Disaster recovery for the Snowflake (Direct Connect) Data Pipeline
Learn how to respond to Reltio outages, Snowflake outages, and combined outages that affect the Snowflake (Direct Connect) Data Pipeline, and what actions are required for each scenario.
Disaster recovery for the Snowflake (Direct Connect) Data Pipeline depends on the type of outage and the scenario you configured. For Reltio outages, Reltio manages the switchover to the Reltio backup environment automatically, and no action is required from you. For Snowflake outages, you need to activate the backup Snowflake account and switch the active data pipeline.
Reltio maintains a Reltio backup environment for tenants on Reltio Business Critical Edition (BCE) and manages the switchover automatically during a Reltio outage. Reltio does not monitor your Snowflake account, so you are responsible for performing the recovery actions when a Snowflake outage occurs.
Outage scenarios and required actions
The following table summarizes the outage combinations and the actions required for each. The status column for PrivateLink applies only when PrivateLink connectivity is enabled for the Snowflake (Direct Connect) Data Pipeline.
| Reltio | Snowflake | PrivateLink | What to expect | Your action |
|---|---|---|---|---|
| Healthy | Healthy | Healthy | Reltio writes to the primary Snowflake account over the network path, which is private if PrivateLink is enabled or public if PrivateLink is not enabled. | None |
| Outage | Healthy | Healthy | Reltio writes to the primary Snowflake account from the Reltio backup environment over the network path, which is private if PrivateLink is enabled or public if PrivateLink is not enabled. Reltio ensures eventual consistency. Data not written to Snowflake during the outage is written after recovery. | None during the outage. |
| Healthy | Outage | Healthy | Reltio writes to the backup Snowflake account over the network path, which is private if PrivateLink is enabled or public if PrivateLink is not enabled. | Complete the Prerequisites for managing a Snowflake outage. Activate the backup Snowflake account and switch the active data pipeline as described in Respond to a Snowflake outage. |
| Healthy | Healthy | Outage | Reltio writes to the primary Snowflake account over the public network. | None |
| Healthy | Outage | Outage | Reltio writes to the backup Snowflake account over the public network. | Complete the Prerequisites for managing a Snowflake outage. Activate the backup Snowflake account and switch the active data pipeline as described in Respond to a Snowflake outage. |
Prerequisites for managing a Snowflake outage
For Scenario 3, the following conditions are required to be in place before a Snowflake outage occurs so that you can switch the data pipeline quickly when an outage happens.
| Prerequisite | Details |
|---|---|
| Only one pipeline is active at a time | The primary Snowflake (Direct Connect) Data Pipeline writes to the primary Snowflake account during normal operation. The secondary data pipeline is configured but deactivated. |
| Snowflake account-to-account replication is configured | Account-to-account replication and a Snowflake failover group keep the primary and backup Snowflake accounts in sync. The replication setup is described in Scenario 3: Set up PrivateLink and configure the Snowflake (Direct Connect) Data Pipeline for Reltio BCE with a Snowflake account having a read-write backup account.. |
| Backup Snowflake account is ready to activate | The backup Snowflake account is provisioned and the failover group is replicated from the primary account. |
| Secondary data pipeline is fully configured and validated | The secondary data pipeline is configured in the Reltio primary tenant, the validate returns 200 OK, and the pipeline is deactivated. |
For Scenario 1 and Scenario 2, no action is required from you during a Snowflake outage other than waiting for Snowflake to recover.
Respond to a Snowflake outage
For Scenario 3, switch the active data pipeline to write to the backup Snowflake account through the steps in the following procedure.
- In the backup Snowflake account, alter the failover group so that the backup Snowflake account becomes the read-write primary:
ALTER FAILOVER GROUP <failover_group_name> PRIMARY;After the failover, the backup Snowflake account is the read-write primary.
- In the Reltio Console, switch the active data pipeline to the backup Snowflake account:
- Deactivate the primary Snowflake (Direct Connect) Data Pipeline that writes to the primary Snowflake account.
- Activate the secondary Snowflake (Direct Connect) Data Pipeline that writes to the backup Snowflake account.
Recover from a Snowflake outage
For Scenario 3, after the primary Snowflake account is restored, recover back to the primary Snowflake account through the steps in the following procedure.
- In the primary Snowflake account, alter the failover group so that the primary Snowflake account becomes the read-write primary again using the following command:
ALTER FAILOVER GROUP <failover_group_name> PRIMARY; - In the Reltio Console, switch the active data pipeline back to the primary Snowflake account:
- Deactivate the secondary Snowflake (Direct Connect) Data Pipeline.
- Reactivate the primary Snowflake (Direct Connect) Data Pipeline.
- Synchronize data that did not reach the primary Snowflake account during the failover-and-recovery cycle. Run syncToDataPipeline from the Reltio primary tenant with the
updatedSinceparameter set to the EPOC time of midnight on the day of the failover. For more information, see the syncToDataPipeline API.
Reltio outage
For Reltio outages, no action is required from you. Reltio manages the switchover from the Reltio primary environment to the Reltio backup environment automatically. The Reltio backup environment continues writing to the same Snowflake account used by the primary environment.
After Reltio recovers from the outage, Reltio ensures eventual consistency, and any data not written to Snowflake during the outage is written after recovery.