Synchronization modes
Learn about the different synchronization modes with Salesforce RIH.
This topic describes the synchronization modes (all bidirectional) of the Reltio Integration for Salesforce with RIH and provides guidance on selecting the one to use for your requirements.
Single dispatch with real-time synchronization
This mode uses an event driven architecture with real-time recipe triggers to start the synchronization between Reltio and Salesforce, following a single transaction approach—one event, one recipe job. When data is created or updated in Reltio or Salesforce, these triggers enable real-time integration.
Recipes used to synchronize data from Reltio to Salesforce use an Amazon SNS connector real-time trigger, which provides a high-throughput, push-based, best-effort approach for event ordering. Each recipe automatically creates a SNS subscription as soon as it is started, which remains valid only while the recipe remains in an active state. Amazon SNS follows the publish-subscribe messaging paradigm, with notifications being delivered to clients (RIH recipes in this case) using a push mechanism that eliminates the need to periodically check for new information and updates. If the subscriber is not available, the default policy of three retries is applied until the event is discarded.
Recipes used to synchronize data from Salesforce to Reltio use Salesforce connector's real-time trigger. This trigger is activated when an outbound message is created or updated. This trigger also provides a backup polling for the relevant object in the recipe every six hours after a restart. RIH then checks to ensure that the necessary jobs have been created. In the case of missing events, RIH creates the jobs at that point.
Both set of recipes consider a full entity payload as input, and any change on the subscribed entities trigger a synchronization process to the other system.
The real-time sync mode is generally used for low volume changes. For example, a data steward updates data in Reltio, and a low response time is expected for Salesforce to be updated. It should not be used for high volume changes, such as initial migrations or reindex tasks.
Known limits
Recipes are configured to use the maximum concurrency available, which is 5. If more events are received in a given period, then these are queued for processing in the RIH platform.
Recipes that use real-time triggers have rate limit restrictions due to the use of the Workato Webhook Gateway. For more information, see Rate Limits.
The webhook gateway uses rate limits to manage the events, which are as follows:
-
Steady rate of 20 events per second, which is equivalent to 72,000 events per hour.
-
Burst allowance of 1000 events that can be processed beyond the steady rate in a burst.
What happens if you reach these limits?
-
In case of events that aren't delivered from Salesforce to Reltio:
-
Salesforce automatically retries to deliver these events by sending a message within 24 hours, in an exponential interval of 2 hours between each retry.
-
Salesforce recovers these events using the backup polling mechanism from the Salesforce connector real time trigger.
-
-
Events from Reltio to Salesforce that aren't delivered are skipped. To avoid this, each RIH recipe that contains an AWS SNS trigger has the following:
-
A subscription filter policy that is applied in the AWS SNS subscription to reduce the number of calls from AWS SNS to the Workato webhook only for the applicable entity type and event types. For more details, see Amazon SNS message filtering.
-
Also, make a note of the consumption limits for the different synchronization methods. For more information, see topic Consumption definitions.
Single dispatch with near real-time synchronization (polling)
An alternative to the Single dispatch with real-time synchronization mode, this mode synchronizes data by initiating events from a Reltio or Salesforce processing queue. This mode also follows an event driven architecture, where it supports higher loads. It works in a polling manner, searching for events from the queue with a minimum interval of 5 minutes.
When synchronizing data from Salesforce to Reltio, it provides different actions inside the Salesforce connector. Select the one that satisfies your business requirements, and change the recipe trigger from the provided recipe.
When synchronizing data from Reltio to Salesforce, this prebuilt integration contains a specific AWS SQS recipe trigger that periodically checks an AWS SQS queue configured at the Reltio platform to consume any available messages.
Batch dispatch with scheduled ETL
This mode uses recipes with batch triggers that are executed to extract, transform, and load (ETL) data from one application to another. This mode follows a multiple transaction approach, that is, a batch of entities for one job. We recommend this mode to cover initial migrations or high-volume data updates in a short period.
Recipes used to synchronize data from Reltio to Salesforce use the Reltio export service to determine the entities or relations based on a custom filter. The recipe then executes multiple jobs to synchronize data to Salesforce using the Salesforce connector batch actions for newly created or updated objects.
Recipes used to synchronize data from Salesforce to Reltio use the Salesforce connector batch trigger with a Salesforce Object Query Language (SOQL) query to get the objects. Each recipe job can handle up to 2,000 entities and the synchronization to Reltio using Reltio connector bulk action.
You can schedule the recipes to run at specific times, and also deactivate them at specific times.
How do I choose the correct mode?
| Parameter | Single dispatch with real-time synchronization | Single dispatch with near real time synchronization (Polling) | Batch dispatch with scheduled ETL |
|---|---|---|---|
| Response time for synchronization to start |
Respond to events immediately (if recipe jobs available) | Up to 5 minutes | Scheduled (upto the user to define) |
| Integration Patterns | Event driven architecture with webhook | Event Driven Architecture with Polling | Extract Transform and Load |
| Message ordering |
Not guaranteed. Best-effort approach for event ordering For SNS, if number of events generated is above throttling policy (e.g. >10 events/sec) - not guaranteed at all |
Not guaranteed. Best-effort approach for event ordering. | Not applicable / Not guaranteed |
| Persistence | Persisted in Salesforce. In case RIH consumer recipe is not able / not active to receive the event, backup polling mechanism will process the event later. No persistence in AWS SNS. In case RIH consumer recipe is not able / not active to receive the event, it will be skipped. | Persisted. Events persisted until RIH recipes can consume it or until TTL is expired. | Not applicable as data is extracted from the source system directly. |
| Log and debug | Detailed information available for each recipe job, easier to debug | Detailed information available for each recipe job, easier to debug | Information grouped per batch chunk |
| Mapping Complexity | Simple - uses RIH out-of-the-box mapping features | Simple - uses RIH out-of-the-box mapping features | Medium - uses RIH out-of-the-box mapping features and Javascript code to parse complex arrays |
| Task consumption | Medium to High | Medium to High | Low to Medium |
| Throughput |
Low Single dispatch (i.e. recipe job) for each generated event. Up to 5 concurrent recipe jobs. |
Low Single dispatch(i.e. recipe job) for each generated event. Up to 5 concurrent recipe jobs. |
High Batch dispatch, up to 2000 for each recipe job |
| Constraints |
Reltio to Salesforce trigger recipes need to remain active to ensure no events are missed. Global RIH webhook limits for the number of events that can be received per hour, see Webhook Rate Limits | Workato Docs | — | Does not handle delete nor merge events. |
| Recommended scenarios |
Controlled/predictable max number of changes by real users (e.g. Data stewards). Number of changes per hour/burst must be below global RIH webhook limits. | Alternative if real-time is not possible due to RIH webhook limits |
Initial migrations High volume data loads/changes in a short period Periodic syncs Reindex tasks |