Create Entities in Bulk
Use the latest version (version 4) of the Reltio connector to create records in bulk within a recipe.
Reltio connector provides the ability to create records in bulk, which allows you to create records (entities, relations, and interactions) in a single batch. You can:
- create records in batches of up to 2000 records in a single action. For optimal performance, start with a batch size of 500 and increase gradually to find the best value for your data. The ideal batch size depends on the size and complexity of your entities and the memory available on your integration platform.
- view the number of failed records that are included in one or more failed HTTP
requests in the batch due to the request getting timed out or the service being
unavailable using the Failed records field.
For example: you send five HTTP requests in parallel, and each request contains 50 records. Suppose four of these requests were successful and one request failed due to a 503 error (service is unavailable). As a result, the Failed records field shows all 50 records that correspond to the failed HTTP request. A sample image is displayed:
The Failed records field displays the error messages and record details such as crosswalks and attributes:
Note: The failed records that appear in this field are not the same as the errors that are generated when you use the Reltio platform APIs. - view the URI of the created object using the
returnObjectsparameter. This parameter is the same as thereturnObjectsAPI query parameter in the Reltio platform. For more information, see Save Entities API.You can set the
returnObjectsparameter tofalseif the response is not used in the subsequent steps of your recipe. This significantly reduces the response size and memory consumption on your integration platform, which improves performance. This is especially recommended when working with larger batch sizes or large entities.
Using the Latest Version
Using or Upgrading to a new version of the Reltio connector in Reltio Integration Hub can have an impact on your existing recipes. Even though Reltio takes steps to make sure that new features and updates to the connector do not cause any issues with the existing recipes, some new features occasionally require that new parameters are made available for the Reltio connector actions. In order to use these actions, the existing recipes must be updated.
You can find the updates that you need to make to the connector in the release notes for that version, which indicate whether you will need to make updates to the recipes or not. If you need to make updates to the existing recipes, follow these steps to make sure that the changes made in the new version of the connector take effect.
- Open the recipe.
- Click refresh. For more information, see Refresh the schema in your recipe.
- Make the update to the recipes if required. For example with version 4 of Reltio
connector, you can change the batch size in the Batch size field. The maximum supported batch size is 2000.
However, Reltio recommends starting with a batch size of 500 and incrementally increasing it to determine the optimal value for your use case. The right batch size depends on the size of your entities and the memory limits of your integration platform. For more information, see Configure the batch size.
You can set the
returnObjectsparameter tofalseif you do not want to use fields from the action response in the subsequent steps of the recipe. Doing this will improve the recipe performance particularly for large entity sizes or large batch sizes. - Save your changes for the recipe and test your changes.