Recipes for Reltio Integration for Collibra
Learn about the different recipes available for Collibra.
Here are the Reltio Integration for Collibra recipes and their specific functions.
Main
-
Take input - Collibra community & domain name, Reltio tenant ID & Environment.
-
Get Reltio L3 configuration
-
Define constants for storing Collibra asset type IDs, attribute type IDs, relation type IDs etc. Data model name =
{tenantLabel}_{tenantId}_{tenantEnv}
-
Call recipes in order - Initial Setup, Create Entities, Create Relationships, Add Reference Attribute Relations, Delete Assets.
-
Update Data Model’s description, last updated time, last updated by using Collibra Core API.
Initial Setup
-
Define constants for storing Collibra asset type IDs.
-
Get or Create community, domain, system & data model using the Collibra API. If it already exists, fetch the ID or else create the asset and get the ID from response. Add Relation - System implements Data Model in Collibra.
-
Delete all entries from the lookup tables.
-
Add Data Model ID to created assets table.
-
Get or create Reltio Relationship asset type, Reltio Relationship contains Data Entity/Data Attribute relation type, Data Model contains Reltio Relationship relation type, Data entity contains/references Data Entity relation type using the Collibra API.
-
Return the type IDs of the assets created in this step including domain & data model for further use by other recipes.
Create Entities
-
Define constant for storing relation type.
-
Iterate through entity types in L3 config. Ignore if entity type is abstract.
-
Get or create data entity with full name
{tenantLabel}_{tenantId}_{tenantEnv}>entity>{entity URI’s path last name}
. Patch display name of entity at the same time. -
Add created asset ID and entity URI to created assets & URI->Asset ID tables
-
Update Entity description using the Collibra API
-
Create Relation - Data Entity is part of Data Model
-
Call Create attributes. Pass Entity ID, Entity full name, parent type =
entity
, attribute json and other required things as parameters.
-
Create Relationships
-
Iterate through relation types in L3 configuration:
-
Get or create Reltio Relationship asset with full name
{{tenantLabel}_{tenantId}_{tenantEnv}>relation>relation URI’s path last name}
. Patch asset display name at the same time. -
Add Asset ID to created asset table.
-
Update Relationship description using the Collibra API.
-
Delete all relations whose source/target is Reltio Relationship asset using the Collibra API.
-
Call create attributes if the relationship has attributes with parent type =
Relationship
-
Get or create lhs & rhs relation types - Data Entity
{startObject>Directional context>Label pattern}
Reltio Relationship asset & Reltio Relationship asset{endObject>Directional context>Label pattern}
Data Entity. -
Search for Collibra asset IDs of source & target in Reltio URI->Collibra Asset ID lookup table. Add the 2 relations - between source & relationship asset and between relationship asset & target to toBeCreatedRelations list. If relationship is not directed, use role as co-role. Add relation - Data Model contains Reltio Relationship to toBeCreatedRelations list.
Bulk create relations in toBeCreatedRelations list using the Collibra API.
-
Create Attributes
-
Delete all entries from Recursion lookup table.
-
Add the initial entry to the recursion stack.
-
Define constants for storing attribute types.
-
Repeat while stack has entries -
-
Pop any one entry from stack.
-
Execute python script to extract fields from attribute json.
-
Iterate through item in JSON:
-
Get or Create Data Attribute with full name
{parentFullName}>{attribute name in l3 config}
. Patch the attributes display name at the same time. -
Add created asset ID and attribute URI to created assets & URI->Asset ID tables.
-
Update attribute’s description, data type and is required using the Collibra API.
-
if masking is present in json fields, then add PII flag to attribute using the Collibra API.
-
IF attribute type equals reference - create relation data entity references data entity. Store Collibra Asset ID and referenced URIs in Collibra Reference Attributes table.
-
ELSE IF parent type =
Relationship
, create relation Relationship asset contains Data Entity/Data Attribute based on whether the attribute is nested or simple. -
ELSE create relation data entity contains data entity / data attribute based on whether the attribute is nested or simple.
-
If attribute type equals nested, add attributes of json along with other required entries to the stack.
-
-
Add reference attribute relations
-
For each entry in Collibra reference attributes
-
Get all relations whose source is reference attribute using the Collibra API and add it to toBeDeleted list.
-
Execute python script to convert string of URIs to list of URIs.
-
For each URI in list of URIs
-
Get the corresponding asset ID from the Reltio URI->Collibra Asset ID table.
-
Add relation configuration to toBeCreated list. This contains source ID, target ID and the type of relation - Data Entity contains Data Entity / Data attribute based on whether the attribute is nested.
-
-
Bulk delete the relations in toBeDeleted list using the Collibra API.
-
Bulk create the relations in toBeCreated list using the Collibra API.
Delete Assets
-
Get all assets in Collibra whose full name starts with
{tenantLabel}_{tenantId}_{tenantEnv}
. -
For each asset, if its ID does not exist in Created assets table - add it to toBeDeleted list.
-
Bulk delete the assets in toBeDeleted list.