Monitoring Queues
A Reltio tenant has internal and external queues that you can monitor for performance.
The internal queues are used to process data internally within Reltio. The external queue is used for real-time integration with systems external to Reltio. The internal queues are further classified into the following queues:
- CRUD Queue - When data, such as entities, relationships, groups, activities, and interactions are loaded into the tenant, this data is stored in the database. The API service then creates events, such as ENTITY_CREATED, RELATIONSHIP_CREATED, RELATIONSHIP_CHANGED, GROUP_CHANGED, and so on that are processed by a set of processors in the CRUD queue to update the tenant’s index. After the tenant’s index is updated, the loaded data is available to users of the Reltio UI.
- Match queue - When entities are loaded into the tenant, the loaded entities are matched against the entities that already exist in the tenant. As part of the matching process, events, such as ENTITY_UPDATED, ENTITY_REMOVED, POTENTIAL_MATCHES_FOUND, NOT_A_MATCH_FOUND, ENTITIES_MERGED, and so on are created. These events are processed by a set of processors in the MATCH queue to match entities and then, merge entities for automatic rules, identify potential matches for suspect rules, or mark entities as not a match. For example, the ENTITY_UPDATED event indicates that a CRUD queue processor has regenerated the match tokens and the match document and has sent match events to the match queue to match this entity based on the changes. The information about match results is sent to the CRUD queue to update the tenant’s index. After the tenant’s index is updated, match results information is available to users of the Reltio UI.
The Queues page allows you to view and monitor the queues and take actions accordingly. The following sections explain the information provided graphically about the internal queues:
Queue Size
The Queue size indicates the number of events in the crud and match queues. The queue performance can be visualized from the trend chart. If the queue size remains the same or decreases as events are added to the queue, then the queue is processing the events. If the queue size remains constant without any new events or if the queue size does not decrease, then the queue may have performance issues. Refer to the Queue Troubleshooting guidelines for the next steps.
The following image shows the queue size of the internal queue:
CRUD Queue Processing Speed
The CRUD Queue Processing Speed graph shows the speed at which events are processed in the CRUD queue. If the number of events is high and the processing speed is slow, then the CRUD queue is not performing properly and needs to be examined.
Match Queue Processing Speed
The Match Queue Processing Speed graph shows the speed at which events are processed in the Match queue. If the number of events is high and the processing speed is slow, then the Match queue is not performing properly and needs to be examined.
The following image shows the processing speed of the CRUD and Match queue:
Time Event Spent in Queue
The Time Event Spent in Queue graph shows the amount of time the events remained in the queue before being processed and moved out of the queue. If the events remain in the queue for a long time, then it indicates that the queue processing is slow. At the same time, if the number of events continues to increase, then the queue might have stopped processing. The following image shows the details of the time spent by events in the queue:
Failed Messages Count
The Failed Messages Count graph shows the number of failures encountered during queue processing. If the number is small, then the system retries processing. A maximum of 200 attempts is made to process a failed message. If a message is not processed after all retry attempts, then the failed message is added to the DeadLetter queue. If messages are failing, it indicates that there are issues in the processing of the internal queues. The following processors might appear when you hover over the graph:
- crud_search_entities
- crud_search_activities
- crud_search_others
- crud_streaming
- crud_history
- crud_merge
- crud_internal_match
- crud_analytics
- match_analytics
- match_incremental
- match_streaming
For example, if a failure occurs and crud_search_entities processor appears when you hover over the graph, then there is an issue with the indexing of entities. The following image shows the number of messages that failed:
DeadLetter Queue Size
If an event could not be processed after several retries, then the event is placed into the Deadletter Queue. This graph shows the number of messages in the DeadLetter queue. If the number of messages is high in this queue, it indicates that there are issues in the processing of internal queues. The following image shows the size of the DeadLetter queue:
Monitoring the External Queue
The external queue has events that are sent from Reltio for real-time integration with systems external to Reltio. The External queue dashboard displays the number of events or messages sent to the external queue.
Queues Troubleshooting
If there is an abrupt drop in the processing speed of the CRUD or MATCH queue and the failed messages count is high and constantly increasing, then the queue might have stopped processing. In this case, check for recent changes in your L3 configuration that might have impacted queue processing. For example, change in match configuration or change of an attribute type. If there are no changes in your L3 configuration or you are not able to diagnose the issue, contact Reltio Customer Support.