Synchronization modes
Learn about the different synchronization modes with Salesforce RIH.
This topic describes the synchronization modes (all bidirectional) of the Reltio Integration for Salesforce (with RIH) and provides guidance on selecting the one to use for your circumstances.
Real-time - event driven
This mode follows an event driven architecture with real-time recipe triggers to start the synchronization between Reltio and Salesforce. When data is created or updated in Reltio or Salesforce, these triggers enable real-time responses that help in integration.
Recipes used to synchronize data from Reltio to Salesforce use an Amazon SNS connector real-time trigger, which provides a high-throughput, push-based, best-effort approach for event ordering. Each recipe automatically creates a SNS subscription as soon as it is started, which remains valid only while the recipe remains in an active state. Amazon SNS follows the publish-subscribe messaging paradigm, with notifications being delivered to clients (RIH recipes in this case) using a push mechanism that eliminates the need to periodically check for new information and updates. If the subscriber is not available, the default policy of three retries is applied until the event is discarded.
Recipes used to synchronize data from Salesforce to Reltio use Salesforce connector's real-time trigger. This trigger is activated when an outbound message is created or updated. This trigger also provides a backup polling for the relevant object in the recipe every six hours after a restart. RIH then checks to ensure that the necessary jobs have been created. In the case of missing events, RIH creates the jobs at that point.
Both set of recipes consider a full entity payload as input, and any change on the subscribed entities trigger a synchronization process to the other system.
The real-time sync mode is generally used for low volume changes. For example, a data steward updates data in Reltio, and a low response time is expected for Salesforce to be updated. It should not be used for high volume changes, such as initial migrations or reindex tasks.
Known limits
Since these recipes use real-time triggers, the following Workato restrictions apply:
| Rate limits | Byte limits |
|---|---|
| 72,000 events per hour | 30 MB per minute |
| 18,000 events in a burst | 30 MB in a burst |
If these limits are reached:
-
the Salesforce to Reltio events are not delivered to the RIH recipes. You can recover them using the backup polling mechanism from the Salesforce connector's real-time trigger.
-
the Reltio to Salesforce events from AWS SNS are not delivered to the RIH recipes. Alternatively, you can use polling triggers such as SQS connectors to recover them.
-
To avoid reaching Workato webhook limits, each RIH recipe with an AWS SNS trigger has a default throttling policy (
throttlePolicy ā maxReceivesPerSecond) that limits calls from AWS SNS to the Workato webhook to 10 per second. You can adjust or remove this limit based on your needs. For more details, see Amazon SNS message delivery retries.
-
Also, make a note of the consumption limits for the different synchronization methods. For more information, see topic Consumption definitions.
Near real-time polling
An alternative to the real-time synchronization mode, this mode synchronizes data from Reltio to Salesforce by initiating events from a Reltio processing queue. This prebuilt integration contains an AWS SQS recipe trigger that periodically checks an AWS SQS queue configured at the Reltio platform to consume any available messages.
ELT/Batch synchronization
This mode uses recipes with batch triggers that is executed to extract, transform, and load (ETL) data from one application to another. This mode follows a multiple transaction approach, that is, a batch of entities for one job. We recommend this mode to cover initial migrations or high-volume data updates in a short period.
Recipes used to synchronize data from Reltio to Salesforce use the Reltio export service to determine the entities or relations based on a custom filter. The recipe then executes multiple jobs to synchronize data to Salesforce using the Salesforce connector batch actions for newly created or updated objects.
Recipes used to synchronize data from Salesforce to Reltio use the Salesforce connector batch trigger with a Salesforce Object Query Language (SOQL) query to get the objects. Each recipe job can handle up to 2,000 entities and the synchronization to Reltio using Reltio connector bulk action.
You can schedule the recipes to run at specific times, and deactivate them at specific times.
How do I choose the correct mode?
| Parameter | Real-time - event driven | Queue - event driver (Reltio only) | ELT Batch |
|---|---|---|---|
| Response time for synchronization to start |
Respond to events immediately (if recipe jobs available) | Up to 5 min. Polling trigger |
Reltio - Scheduled (can be reduced to a min of 5 min interval) SFDC - Batch/Polled (can be reduced to a min of 5 min interval) |
| Message ordering |
Not guaranteed. Best-effort approach for event ordering In cases of load (e.g. >10 events/sec) - not guaranteed at all |
Not guaranteed. Best-effort approach for event ordering. Guaranteed if SQS is changed to be a FIFO queue and recipe concurrency is set to 1. | Not applicable / Not guaranteed |
| Persistence | Persisted in Salesforce. In case RIH consumer recipe is not able / not active to receive the event, backup polling mechanism will process the event later. No persistence in AWS SNS. In case RIH consumer recipe is not able / not active to receive the event, it will be skipped. | Persisted in AWS SQS. Events persisted until RIH recipe can consume it or until TTL is expired. | Not applicable as data is extracted from the source system directly. |
| Log and debug | Detailed information available for each recipe job, easier to debug | Detailed information available for each recipe job, easier to debug | Information grouped per batch chunk |
| Mapping | Simple - uses RIH out-of-the-box mapping features | Simple - uses RIH out-of-the-box mapping features | Medium - uses RIH out-of-the-box mapping features and Javascript code to parse complex arrays |
| Task consumption | Medium to High | Medium to High | Low to Medium |
| Throughput |
Low Single processing (i.e. recipe job) for each generated event. Up to 5 concurrent recipe jobs. |
Low Single processing (i.e. recipe job) for each generated event. Up to 5 concurrent recipe jobs. |
High Batch Processing, up to 2000 for each recipe job |
| Constraints |
Reltio to Salesforce trigger recipes need to remain active to ensure no events are missed. Global RIH webhook limits for the number of events that can be received per hour, see Webhook Rate Limits | Workato Docs | ā | Does not handle delete nor merge events. |
| Recommended scenarios |
Controlled/predictable max number of changes by real users (e.g. Data stewards). Number of changes per hour/burst must be below global RIH webhook limits. | Alternative if real-time is not possible due to RIH webhook limits |
Initial migrations High volume data loads/changes in a short period Periodic syncs Reindex tasks |
More about entity payload from Reltio platform
When events are processed and delivered to the SNS topic or SQS queue, there is no order in which these are delivered. Sometimes, older messages are received after the new ones are sent. To ensure the latest data is retrieved from the Reltio platform, set the SFDC_GetNewPayloadFromReltio to true.
In another instance, when dealing a payload larger than the length supported by the streaming queues, Reltio platform sends the exceededQueueSizeLimit attribute set to true in the event message, without the entity payload. In this case, the RIH SFDC recipe retrieves a fresh copy of the entity payload from the Reltio system. To support the large object event in Reltio, we recommend the following settings in your tenant's physical configuration:
"streamingConfig": {
"streamingEnabled": true,
"streamingAPIEnabled": true,
"analyzeOvChanges": false,
"emptyStartEndRelationCrosswalks": false,
"largeObjectsSupport": true,