Accelerate the Value of Data

Synchronize data between Reltio and Snowflake

Learn how to synchronize your entities, relations, matches, interactions, and merges data from Reltio to Snowflake.

Before you synchronize data, you must validate Reltio event transfer for your cloud platform and Snowflake. For more information, see the that topic for your cloud platform earlier in this section.
Synchronize data between the Reltio and Snowflake platforms to ensure that you always have the latest Reltio data in Snowflake. View data being transmitted in your cloud storage location in the Event Notifications section of your cloud console.
To synchronize data between Reltio and Snowflake:
  1. Start the synchronization by running the syncToDataPipeline:
    POST https://{{tenantEnvironmentName}}.reltio.com/reltio/api/{{tenantId}}/syncToDataPipeline
    where:
    • <tenantEnvironmentName>: is the name of the environment on which your Reltio tenant is hosted, which forms part of the fully qualified URL, for example:
      • https://dev-data-pipeline-hub.reltio.com

      • https://test-data-pipeline-hub.reltio.com

      • https://361-data-pipeline-hub.reltio.com

    • tenantId: is the ID of the Reltio tenant you configured the Reltio Data Pipeline for Snowflake

  2. Run these APIs to confirm the request completed successfully:
    Get Active Tasks for Tenant to check the current tasks for the tenant:
    GET {ApplicationURL}/{tenantId}/tasks
    Get Tasks History to check finished tasks for all tenants
    GET https://{{tenantEnvironmentName}}.reltio.com/reltio/{{tenantId}}/tasks/history
    Get Task by ID for Tenant to check tasks with the given ID for the specified tenant
    GET https://{{tenantEnvironmentName}}.reltio.com/reltio/{{tenantId}}/tasks/{{taskId}}
    where:
    • taskId: is the ID of the Reltio object type processing task.

    For more information, see topics Get Active Tasks for Tenant, Get Tasks History, and Get Task by ID for Tenant.

  3. Run the Tenant Queue Status to verify that Snowflake received Reltio events:
    GET https://{{tenantEnvironmentName}}-data-pipeline-hub.reltio.com/status/tenant/{{tenantID}}/details

    For more information, see topic Tenant Queue Status API.

  4. When the task is complete (count is zero), validate that the number of objects in Snowflake is the same as in Reltio.
    Note: When the synchronization completes, a row is added in the landing table with details of the sync. A separate row is created for each sync, with the time of the sync recorded in the Timestamp field. For more information, see topic Landing table datasets for Snowflake.
    If it is not, check the following issues:
    • If there is a discrepancy in the count of events in Reltio and in Snowflake, use the Event Monitoring to check if the events are being loaded to your cloud storage:

      GET https://{{tenantEnvironmentName}}-data-pipeline-hub.reltio.com/api/tenants/{{tenantID}}/monitoring/_eventMonitoring

      For more information, see topic Event Monitoring API.

    • If events are being passed to your cloud storage, run the COPY_HISTORYcommand to view a copy history table that identifies whether an attempt was made to partially load your Snowflake landing table or if the load failed. For more information, see Loading history in the Snowflake function reference.

      In the displayed data loading history within the selected time period, view these columns to check whether files were loaded or not and the reason if it didn't load correctly:
      • STATUS : Whether or not a particular set of files was loaded, partially loaded, or failed to load.

      • FIRST_ERROR_MESSAGE The reason a load attempt was only partially completed or if didn't happen at all.

        If a set of loaded files has multiple issues, this FIRST_ERROR_MESSAGE column only indicates the first error encountered. Run the COPY INTO <table> command with the VALIDATION_MODE copy option set to RETURN_ALL_ERRORS and reference the set of files you'd attempted to load using Snowpipe to view all errors in the files.
        SELECT *
        FROM TABLE(information_schema.copy_history(
          table_name=>'<LANDING_TABLE_NAME>',
          start_time=>dateAdd(days, -2, current_timestamp())));
        Note: The VALIDATION_MODE copy option instructs a COPY statement to validate the data to be loaded and return results based on the validation option specified. No data is loaded when this copy option is specified.

        For more information, see COPY INTO <table> in the Snowflake function reference.

  5. If events are not being passed to your cloud storage, run the Event Monitoring to observe the specific transfer stage that a particular entity event is at in Reltio and check against your Snowflake data.
    Take these steps depending on the value in the eventState field:

    For more information, see topics Event Monitoring API and Need some help?