Provision Delta Live Tables pipeline for Azure

Learn how to configure the Delta Live Tables pipeline for Azure.

Before this step, you must have validated and synced the Reltio data to your cloud storage. For more information, see topic Validate and sync Reltio Data Pipeline for Databricks with Azure.

The Data Pipeline Hub (DPH) provisions the Delta Live Tables with the information and setup you completed earlier in the process. For more details on the Delta Lake, see topic Delta Lake API

To provision the Delta Live Tables pipeline:

Send an empty request to DPH Delta Lake adapter configure pipeline endpoint:

POST {{hub-url}}/api/tenants/{{tenantID}}/adapters/deltalake/actions/configure_pipeline

where hub-url is of the form <env>-date-pipeline-hub.retio.com. With the secrets in this format (only one Azure secret type needed):


{ "Azure":
    {  "accountKey": "<Account Key>",
       "sasToken": "<SAS Token>",
       "clientSecret": "<Client Secret>"
    },
  "Databricks":
    {  "token":"<serviceprincipalstoken>"
    }
}

When successful


{  "status": "SUCCESS",
   "errors": [],
   "adapter-response": {}
}

Contact Support if you receive this validation error:


{  "severity": "Error", 
   "errorMessage": "Adapter not enabled for deltalake.",
   "errorCode": 8004,
   "errorDetailMessage": "Adapter not enabled for deltalake.",
   "foundErrors": null,
   "innerErrorData": null,
   "causedBy": null,
   "asJSON": "{\"severity\":\"Error\",\"errorMessage\":\"Adapter not enabled for deltalake.\",\"errorCode\":8004,\"errorDetailMessage\":\"Adapter not enabled for  deltalake.\",\"foundErrors\":null,\"innerErrorData\":null,\"exception\":null,\" stack\":null,\"causedBy\":null}" 
}

After your pipeline has been provisioned, you may reconfigure the pipeline by calling this endpoint again.

Continuous Mode: If you are running in Continuous mode, then your pipeline is live.
Triggered Mode: If you are running in Triggered mode, then you must trigger the pipeline to go live. For more information, see topic Trigger Delta Live Tables pipeline

Accelerate the Value of Data

Provision Delta Live Tables pipeline for Azure

Footer