How the Consistency Service Works

When an inconsistency is detected, the Consistency Service works in real-time to repair the data and synchronize storage layers. Here are the key steps:

1. Detection:
   • The Consistency Service continuously monitors event logs and data replication queues to detect any potential mismatch between the primary and secondary storage.
   • Internal systems like Reltio Event Bus and Data Replication Service generate alerts when a transaction in primary storage doesn’t propagate as expected.
2. Identification:
   • The Consistency Service cross-references data snapshots in both the primary and secondary layers to isolate discrepancies.
   • It also checks transactional logs and write-ahead logs (WALs) to verify whether all intended transactions were successfully recorded in both storage systems.
3. Repair:
   • The repair process leverages data replication protocols that ensure real-time consistency. The Consistency Service uses the Data Repair Engine to initiate corrective actions.
   • For significant inconsistencies, version control mechanisms track changes between different data states, and the most recent valid state is restored.
   • Consistency Service also invokes schema validation routines to verify that the data structure remains intact during repair.
4. Storage Synchronization:
   • After the inconsistency is repaired, the Consistency Service initiates synchronization jobs to ensure the data in secondary storage is updated to match the primary store
   • The Data Consistency Validator performs a final verification to confirm that the repair was successful.

Impact on Platform and Data

Here are the key points to note regarding impact:

Why Do These Events Happen?

Data inconsistencies can occur due to a variety of reasons:

• Temporary network disruptions: Short-term connectivity issues may cause some transactions to fail to propagate to secondary storage.
• System failovers: In the event of a failover, the Consistency Service is responsible for ensuring that the secondary storage matches the primary system post-recovery.
• Latency in storage replication: The primary and secondary storage systems may occasionally experience a delay in data synchronization, which the Consistency Service rectifies.

Reltio uses multi-zone replication and distributed storage systems to minimize the chances of data inconsistencies. The Consistency Service ensures that these occasional issues do not affect long-term data accuracy.

Other Internal Processes That Support Consistency Service

The Consistency Service works in tandem with several other internal system processes to ensure the platform remains reliable and consistent:

• Transactional Logging Service (TLS): This service keeps a record of all transactional events across the system and provides a reference point for the Consistency Service to validate if any transaction has been missed or duplicated during storage replication.
• Reltio Event Bus: The Event Bus is responsible for propagating changes across the system in real-time. When the Consistency Service is triggered, it ensures that the changes made by the Consistency Service are distributed across the platform, preventing further inconsistencies.
• Real-time Indexing Service: As the Consistency Service modifies or corrects data, the Real-time Indexing Service ensures that all updates are reflected in Reltio's search index, allowing users to access the most up-to-date information quickly.
• Replication Consistency Checker: This service runs scheduled checks to ensure that all copies of data in the system (both primary and secondary) are fully consistent, reducing the likelihood of discrepancies. If it detects a mismatch, it informs the Consistency Service.
• High Availability (HA) Mechanisms: The Consistency Service is closely integrated with Reltio's HA architecture, which ensures that even during system failovers or data migrations, the platform remains consistent. HA mechanisms work in conjunction with the Consistency Service to handle repairs when needed.

Action Required?

No direct action is required from your side during Consistency Service operations. In rare cases where a Consistency Service activity might cause a temporary delay or a slight disruption, our Support team will proactively notify you.

If you experience an issue and are informed that the Consistency Service resolved it, this means that our system detected and repaired an inconsistency, ensuring your data remains accurate and synchronized.