Configure a Snowflake pipeline for Azure in the Console

Learn how to use the Reltio Console to configure a Snowflake pipeline for Azure.

Before using the Reltio Data Pipeline for Snowflake on the Azure cloud platform, you must configure it in Reltio.

This topic describes how to configure this data pipeline using the Reltio Console Data Pipelines application. If you prefer to configure this data pipeline using Reltio REST APIs, see topic Configure the Reltio Data Pipeline for Snowflake for Azure.

To configure a new Snowflake data pipeline for Azure:

In the Console, select Data Pipelines.
The page lists the pipelines already configured or, if none, a note linking to the documentation.
Select + NEW PIPELINE, then select Snowflake.
The Configure Snowflake pipeline page is displays.
In the Name field, enter the name of the pipeline, which is used to refer to the pipeline in the UI and API calls. The name must be alphanumeric, can be 3 to 20 characters in length, and cannot contain any spaces.
In the Authentication section, from the Cloud provider dropdown, select the Azure.
Select Authenticate using client credentials to authenticate requests using client credentials.
1. In the Client Credentials field, enter the unique client credentials to be used for authentication.
2. In the Storage account name field, enter the name of the Azure storage account.
3. In the Bucket Name field, enter the name of the bucket on Azure.
4. In the Client ID field, enter the Azure active directory client ID.
5. In the Tenant ID field, enter the Azure active directory tenant ID.
6. In the Staging Bucket Region field, enter the name of the region if the bucket region is not from the United States.
Select Authenticate using client key to authenticate requests using client key.
1. In the Key field, enter the client key used for authentication.
2. In the Storage account name field, enter the name of the Azure storage account.
3. In the Bucket name field, enter te name of the bucket on Azure.
4. In the Staging Bucket Region field, enter the name of the region if the bucket region is not from the United States.
Select Authenticate using client token to authenticate requests using client token.
1. In the Token field, enter the access token used to for authentication.
2. In the Storage account name field, enter the name of the Azure storage account.
3. In the Bucket name field, enter te name of the bucket on Azure.
4. In the Staging Bucket Region field, enter the name of the region if the bucket region is not from the United States.

Verify configuration settings in your tenant

In the Data delivery options section, from the Attribute format dropdown, select the format of the attributes stored.
1. Select Data filtering to enable filtering for adapter.
2. Select Transmit OV values only if you want to sync only operational values to the Snowflake.
3. Select Serialize initial sources in Crosswalks to enable sending initial source in crosswalk.

Verify configuration settings in your tenant

Select Save & Create at the top of the page to save and create the Snowflake pipeline for Azure.

Troubleshoot your Reltio configuration

Follow these steps if data is not publishing to your Azure cloud storage:

Make sure you have:
1. access to edit the Reltio physical tenant configuration.
2. permission to view and modify Reltio streaming and pipeline settings.
3. access to view your Snowflake settings.
4. the ability to run API endpoints to validate your configurations and connections.

Ensure streaming is enabled in the Reltio physical tenant configuration.

"streamingEnabled": true
"streamingAPIEnabled": true

"streamingConfig": {
      "streamingEnabled": true, <<<===Make sure that it is set to true
      "streamingAPIEnabled": true, <<<===Make sure that it is set to true
      "analyzeOvChanges": false,
      "emptyStartEndRelationCrosswalks": false,
      "largeObjectsSupport": false,
      "skipPayload": false,
      "payloadType": "SNAPSHOT",
      "ovOnly": false,

Ensure the pipeline queue is defined and enabled in the messaging.destination section of the streamingConfig

"enabled": true

 {
                  "payloadType": "SNAPSHOT",
                  "type": "queue",
                  "provider": "dph",
                  "name": "<environment>-datapipeline-events_<tenant>",
                  "dtssQueue": false,
                  "groupingQueue": false,
                  "dataPipelineQueue": true,
                  "enabled": true,  <<<<=====  Make sure that it is set to true
                  "format": "KRYO"
              }

Validate the data pipeline with this API and confirm the status is successful (200):

Check the external queue to ensure that events are processing.

POST https://<environment>-data-pipeline-hub.reltio.com/api/tenants/<tenant>/adapters/Snowflake/validate

Validate the processed event, using a specific entity, across the pipeline.

GET https://<environment>-data-pipeline-hub.reltio.com/api/tenants/<tenantid>/monitoring/_entityMonitoring?entityId=<entityID>

Unify and manage your data

Configure a Snowflake pipeline for Azure in the Console

Footer