Managing External Queues

You send data from Reltio to external systems using external queues.

A Reltio tenant has external queues that you can monitor for performance. As part of our initiative to provide more self-service capabilities, the Tenant Management application is enhanced to allow you to add external queues for streaming data between Reltio and external systems. The external queue is used for real-time integration with systems external to Reltio. Also, you can update and delete external queues.

For more information on the required permissions, see Permissions for the Console Service.

Adding an Amazon SNS or SQS Queue

To add a queue to stream data to AWS, do the following steps:

  1. Go to the home page.
  2. In the Console, click the Tenant Management.
  3. On the left navigation bar, click External Queue.

    The list of queues are displayed.

  4. Click ADD QUEUE.

    The Add queue dialog box is displayed.

  5. From the Provider drop-down list, select Amazon SNS/SQS.
  6. Select the Authenticate using role checkbox.

    You can select the checkbox when you do not wish to share your AWS Key and Secret.

  7. Enter the Role.
  8. Enter the External ID.

    For more information, see:

    Creating a role to delegate permissions to an IAM user,

    How to use an external ID when granting access to your AWS resources to a third party.

    Note: Request a Reltio AWS Account ID, Reltio AWS User ARNs and External ID for a role. For more information, see Reltio Support.
  9. Select the Use ARN checkbox.

    You can use the Amazon Resource Name (ARN) of your AWS queue. In this case, the Queue name and Type fields are not required.

  10. Deselect the Use ARN checkbox.
    1. Enter a name for the queue.
    2. Select the queue type.

      For example, add the queue type as SQS, SNS and so on.

    3. Enter the Region.

  11. Select the streaming format.
    For example, JSON, JSON_ZIP_BASE64 and so on.
    Note: The default streaming format is JSON_ZIP_BASE64.
  12. Optionally, specify the filter condition to filter entities.

    This helps you to filter entities. For more information, see Events API.

  13. Select the Type filter.

    This helps you to select the events that you want to stream using this queue.

  14. Select Payload Type.

    In the Event Payload field, when you add Snapshot as the Payload type, you can add a partial snapshot of the data object to the streaming messages. Delta will help you to add the information about the changed part of the object in a format similar to the Activity Log. For more information, see Message Streaming.

    There are two options:
    1. Snapshot
    2. Delta

    1. Select Snapshot.

      Snapshot enables you to add a partial snapshot of the data object to the streaming messages based on the selection in Event Payload field. For more information, see Message Streaming.

      This field has the list of JMSevents field. You can select all the options. The URI and Type are mandatory. The URI and Type must be selected, when you select the Snapshot.
      Note: If you do not select one of them, you will not be able to save the queue. A validation error will be displayed as follows.

      In the Event Payload field, the following options are displayed.

      For example, uri, type, createdTime,createdBy, and so on.

      OR

    2. Select Delta.

      The drop-down list for the Event Payload field is hidden.

      Select Delta to add only information about the changed part of the object in a format very similar to the activity log. For more details, please refer to the Message Streaming.

  15. Click SAVE.
  16. Select the Enable Streaming checkbox.

    After you have created a queue, the queue is added to the External Queues page.

Adding a Google Pub/Sub Queue

To add a queue to stream data to Google Pub/Sub, do the following steps:

  1. Go to the home page.
  2. In the Console, click Tenant Management.
  3. On the left navigation bar, click External Queue.

    The list of queues are displayed.

  4. Click ADD QUEUE.

    The Add queue dialog box is displayed.

  5. From the Provider drop-down list, select Google Pub/Sub.
  6. Enter the Queue Name.

    The Name must start with a letter. The name can contain only letters, numbers, underscore(_), hyphen(-), tilde(~), period(.), percent(%) or plus(+) signs.

  7. Click Upload Service Account Key File.

    The Open dialog box is displayed.

  8. In your local system, go to the destination folder.
  9. Upload the Service Account Key File in JSON format, generated through the Google Developer Console .

    The JSON file is generated by using the Google Developer Console. The file includes the project ID, private key, and the client email.

  10. Upload the JSON file.

    If you upload the JSON file, you are not required to specify values for the fields.

  11. Otherwise, specify values for Project ID, Private key, and the Client Email fields.
  12. Select the streaming format.
    For example, JSON, JSON_ZIP_BASE64 and so on.
    Note: The default streaming format is JSON_ZIP_BASE64.
  13. Optionally, specify the filter condition.

    This helps you to filter entities. For more information, see Events API.

  14. Select the Type filter.

    The Type filter is a collection of event types name to stream. Events of different types will be ignored. This helps you to select the events that you want to stream using this queue.

  15. Select Payload Type.

    In the Event Payload field, when you add Snapshot as the Payload type, you can add a partial snapshot of the data object to the streaming messages. Delta will help you to add the information about the changed part of the object in a format similar to the Activity Log. For more information, see Message Streaming.

    In Payload type, there are two options:
    1. Snapshot
    2. Delta

    1. Select Snapshot.

      Snapshot enables you to add a partial snapshot of the data object to the streaming messages based on the selection in Event Payload field. For more information, see Message Streaming.

      This field has the list of JMSevents field. You can select all the options. The URI and Type are mandatory. The URI and Type must be selected, when you select the Snapshot.
      Note: If you do not select one of them, you will not be able to save the queue. A validation error will be displayed as follows.

      In the Event Payload field, the following options are displayed.

      For example, uri, type, createdTime,createdBy, and so on.

      OR

    2. Select Delta.

      The drop-down list for the Event Payload field is hidden.

      Select Delta to add only information about the changed part of the object in a format very similar to the activity log. For more details, please refer to the Message Streaming.

  16. Click SAVE.
  17. Select the Enable Streaming checkbox.

    After you have created a queue, the queue is added to the External Queues page.

Adding a Microsft Azure Queue

To add a queue to stream data to Microsoft Azure, do the following steps:

  1. In the Console, click the Tenant Management.
  2. Click External Queue on the left navigation bar.
  3. Click ADD QUEUE.

    The Add queue dialog box appears.

  4. From the Provider drop-down list, select Microsoft Azure.
  5. Select Type.

    For example, Queue or Topic. You can stream to a queue or topic in Microsoft Azure.

  6. Enter the Queue name.

    A Queue name can includes letters, numbers, periods (.), hyphens (-), underscores (_), and slashes (/), up to 260 characters. The Queue name must start and end with a letter or number.

  7. Enter the Namespace name.

    The namespace must be between 6 and 50 characters long. The namespace can contain only letters, numbers, and hyphens. The namespace must start with a letter, and it must end with a letter or a number.

  8. Enter the Authorization policy rule name.
  9. Enter the access key.
  10. Select the streaming format.

    For example, JSON, JSON_ZIP_BASE64.

    The default streaming format is JSON_ZIP_BASE64.
  11. Specify the filter condition.

    This helps you to filter entities. For more information, see Events API.

  12. Select the Type filter.

    This helps you to select the events that you want to stream using this queue.

  13. Select Payload Type.

    In the Event Payload field, when you add Snapshot as the Payload type, you can add a partial snapshot of the data object to the streaming messages. Delta will help you to add the information about the changed part of the object in a format similar to the Activity Log. For more information, see Message Streaming.

    In Payload Type, there are two options:
    1. Snapshot
    2. Delta

    1. Select Snapshot.

      Snapshot enables you to add a partial snapshot of the data object to the streaming messages based on the selection in Event Payload field. For more information, see Message Streaming.

      This field has the list of JMSevents field. You can select all the options. The URI and Type are mandatory. The URI and Type must be selected, when you select the Snapshot.
      Note: If you do not select one of them, you will not be able to save the queue. A validation error will be displayed as follows.

      In the Event Payload field, the following options are displayed.

      For example, uri, type, createdTime,createdBy, and so on.

      OR

    2. Select Delta.

      The drop-down list for the Event Payload field is hidden.

      Select Delta to add only information about the changed part of the object in a format very similar to the activity log. For more details, please refer to the Message Streaming.

  14. Click SAVE.
  15. Select the Enable Streaming checkbox.

    After you have created a queue, the queue is added to the External Queues page.

Editing or Deleting a Queue

  1. Hover over the queue on the right that you want to edit or delete.
  2. Click the Edit icon.

    You can edit a selected item in the queue.

    The Edit queue dialog box appears.

    In this example, we selected the option to Use ARN. While editing, you can also enable or disable streaming and delete the queue.

  3. After editing, click SAVE.
  4. To delete the queue, click DELETE.

    Once you confirm the deletion, the queue is successfully deleted.

Monitoring

On the External Queues page, click Monitor Queues to monitor the processing of the external queues that you created as shown below.

From this page, you can also enable or disable streaming and edit or delete a queue as shown in the image above.