Managing Crosswalk Dates
Learn how to capture the date and time when data updates are propagated into Reltio platform by a source.
A crosswalk provides a collection of timestamp fields that enable a developer to granularly capture the date and time when different kinds of updates were propagated into Reltio by the source. These are described in this section.
Refer to the following image of an expanded crosswalk in the UI:
Note: Often, in the descriptions below, the word 'object' is used to represent either an
entity or relationship. Also, "date" and "timestamp" are used interchangeably. All dates
are stored as a full timestamp, and the UI provides a full timestamp in most
cases.
createDate (Crosswalk)
| Purpose: | Create Date is intended to represent the date the source object was created within the source system. |
| Recommended Use: | If the source system provides the create date of the object
(usually represented as a datetime field within the record),
then it is ideal to write this date in the
createDate field of the Reltio object’s
crosswalk. |
| Can be written? | Yes |
| Can be overwritten? | Yes |
| How it is set? | During create or override request,
createDate can be provided in the crosswalk
section of the post. If the Create Date is not provided
during the first post, the API will write the current timestamp
as the Create Date. Upon subsequent updates to the
crosswalk, the Create Date can be overwritten via the
crosswalk section of the post, but if it is not provided, the
Create Date is left as-is. |
| How it is returned? | createDate is always returned in the
crosswalk section of a retrieved object. |
| Effect on Survivorship: | None |
updateDate (Crosswalk)
| Purpose: | Also commonly known as ‘Last Update Date’,
updateDate is intended to represent the
date the record was updated in the source system. Recency, as a
concept in an OV rule, is trying to assess which source system
has the most recent information about the data. In order to do
this faithfully, Reltio needs to know the most recent timestamp
of the data, from the perspective of each source. |
| Recommended Use: | If the source system provides the Update Date of the object
(usually represented as a datetime field within the source
record), then it is ideal to write this date in the
updateDate field of the Reltio object’s
crosswalk. |
| Can be written? | Yes |
| Can be overwritten? | Yes |
| How it is set? | During a create or an override request,
updateDate is provided in the
crosswalk section of the post. If the
Update Date is not provided, both the Update
Date and the Create Date will be the same. OV
recency rules will be ineffective in determining which attribute
from a collection is the most recent. For more information, see topic Retrieve or modify an entity crosswalk updateDate. |
| How it is returned? | updateDate is always returned in the
crosswalk section of a retrieved object. |
| Effect on Survivorship: | The OV Recency rule considers the updateDate
of a crosswalk when determining which attribute from a
collection will be the most recent:
|
Updating the "updateDate" parameter inside a Crosswalk
You can modify the updateDate parameter of crosswalks in the
following two ways:
- By using the
PUT .../entities/{entityId}/crosswalks/{crosswalkId}/updateDaterequest. - By using the
POST .../entitiesrequest and performing one of the following steps:- Update any attribute that has a crosswalk and for which you want to update the
updateDatefield. - Do not change any attribute of the entity and use the
partialOverrideoption. Provide all the existing attribute values that belong to the crosswalk. - Do not change any attribute of the entity and do not use the
partialOverrideoption. Provide theupdateDatefield for the crosswalk inside the request body.
- Update any attribute that has a crosswalk and for which you want to update the
deleteDate (Crosswalk)
| Purpose: | deleteDate (end-date) is intended to
represent the date an object that previously existed in the
source was removed or set to Inactive. In an ideal scenario,
objects in the source are soft-deleted, meaning they remain in
the source and remain in the files that continue to be extracted
from the source, but they contain a delete or
inactive date that articulates when the
source system declares the record to be deleted or inactive. In
this way, the date can be written into the object’s crosswalk
within Reltio, and Reltio can treat the crosswalk accordingly.
|
| Recommended Use: | When populated, Reltio considers this to be a end-date of the
crosswalk. In the UI, the current data represented by the
crosswalk no longer has a solid color in the background but
instead is shown as an outlined box with no background color.
Additionally a “red circle with a slash” is shown to the right
of the crosswalk. The crosswalk will no longer accept updates
from the source. In terms of the survivorship strategy, the data
associated with this crosswalk will be ignored by all OV rules.
Note: When all crosswalk dates within an entity have been
end dated, the entire entity will be end dated by the API.
End dating a crosswalk has no effect on ability to link
transactions to it.
|
| Can be written? | Yes |
| Can be overwritten? | Yes, and it can be removed thus re-enabling the crosswalk for active use again. |
| How it is set? | During create or override request, the Delete Date can be provided in the crosswalk section of the post. If not provided, then API will not set is automatically. |
| How it is returned? | The Delete Date is returned with crosswalk for an
object only if is it set. Note: If the style of extraction from
the source does not provide deleted records with delete
dates, but rather just removes them from the extract
entirely, it may be useful to perform delta detection on the
extract to determine which records have been removed since
the last extract, and then craft your load strategy to set
the delete date of the object’s crosswalk in Reltio
accordingly.
|
| Effect on Survivorship: | None |
Behavior of Cleanser
After the deleteDate parameter is set for the source crosswalk, the
cleanser is initiated and all the values of the source crosswalk are converted to
non-OV values and the cleanse process is initiated.
In the cleanse configuration, when the ovOnly is set to
false, the source crosswalks values participate in the cleansing process. The
cleanser adds or updates the cleanse input values with values from the cleanser
crosswalk.
In the cleanse configuration, when the ovOnly is set to true,
the source crosswalks values do not participate in the cleansing process. Existing
cleanser crosswalk values are removed as there are no values in the source.
Crosswalk singleAttributeUpdateDate
| Purpose: | An attribute holds values contributed by and associated with
various crosswalks. Generally, the timestamp. This capability
enables the ability to store the timestamp of a specific value
in an attribute, if the value has been updated more recently and
separately (via a partial override) than that of the
updateDate of the crosswalk that
contributes it. These are represented as a collection on a
crosswalk. |
| Recommended Use: | Usually used during a partial override operation, when one or just a few attributes from a crosswalk are being updated. Example, suppose a First Name field contains Bob, Bobby, Roberto, all having been written by a normal post in the past. A user goes into Edit mode for the profile and changes Roberto to Robert, and saves the record. Bob and Bobby will not be represented in the collection and their timestamp is inferred from that of the crosswalk. Robert however, having been just updated will be represented in the collection with a current timestamp that is greater than that of the crosswalk. The Recency rule will respect the timestamp of Robert found in the collection as the greater of it and the one found in the crosswalk. |
| Can be written? | Yes |
| Can be overwritten? | No, Reltio platform sets this timestamps automatically when either partial override happens or an attribute is created or updated. |
| How it is set? |
|
| How it is returned? | singleAttributeUpdateDates is a property of
a crosswalk that is always returned (might be empty). It will
have elements if specific attributes were updated through
partial override or through attributes creates or
updates. |
| Effect on Survivorship: | The OV Recency rule considers the updateDate
of a crosswalk when determining which attribute from a
collection will be the most recent.
|
sourcePublishDate (Source or crosswalk level)
| Purpose: | sourcePublishDate is intended to represent
the date of an extract from a source system. In this way, Reltio
platform makes it easy to document the latest extract date that
is represented for the data in Reltio platform. When you write
this date, you can do so at the source level, and/or you can
write it at the crosswalk level. In all cases, when you retrieve
an object, Reltio platform returns only one
sourcePublishDate and it is based on an
internal calculation that returns the max of both dates found. |
| Recommended Use: | If a full extract is loaded, then user must set the
sourcePublishDate at the source level.
However if only a small set of records have been subsequently
re-extracted, then the sourcePublishDate at the
crosswalk level must be set for those records. Example: Suppose
on April 1, 2017 the usual monthly extract from System “CRM” has
been received and it has 99,000 records. You would load the file
into Reltio platform and set the
sourcePublishDate at the source level to
04/01/2017. This is a single PUT operation.
Then on April 5, a small set of 3 records were extracted,
separately from the full set and loaded into Reltio platform.
You must set the sourcePublishDate for these 3
records to 04/05/2017 but do so at the crosswalk level. In this
way when you retrieve any of these 3 records, they will show
having been published on Apr 5. But if you retrieve any of the
99,000 other than these 3, they will reveal a
sourcePublishDate of Apr 1. |
| Can be written? | Yes |
| Can be overwritten? | Yes |
| How it is set? |
|
| How it is returned? | This property is returned only as an attribute in an object’s set of crosswalk dates. |
| Effect on Survivorship: | The OV Recency rule considers the
sourcePublishDate of an attribute value
when determining which attribute from a collection will be the
most recent.
|
Object createDate
| Purpose: | Each object carries the date that the object was first instantiated by Reltio platform and given a unique ID. This date is not shown in the UI but can be used in a search and can be retrieved as the createDate property of an object. The date is read-only and will never be overwritten. |
| Recommended Use: | Initially, Reltio platform will automatically set this date to the date the entity was created in Reltio. During subsequent updates, this timestamp is not changed. |
| Can be written? | No |
| Can be overwritten? | No |
| How it is set? | It is set automatically by Reltio platform when a record is created. |
| How it is returned? | Create Date is always returned for an object. |
| Effect on Survivorship: | None |
Object updateDate
| Purpose: | Each object carries a date when it was most recently updated in Reltio platform from any source. This date is not available in the UI but can be used in a search and can be retrieved as the updateDate property of an object. The date is read-only and will never be overwritten. |
| Recommended Use: | Initially, Reltio platform will automatically set this date to the date the entity was created in Reltio. During subsequent updates, Reltio platform will automatically set this date to the date the object was updated. |
| Can be written? | No |
| Can be overwritten? | No |
| How it is set? | It is set automatically by Reltio platform when an object is created or updated. |
| How it is returned? | Update Date is always returned for an object. |
| Effect on Survivorship: | None |
Detail on the Recency OV Rule
When the recency rule is chosen for an attribute, it tries to find the most recent value from each of the contributing crosswalks. But from each crosswalk, it considers the update date, the sourcePublishDate and individual attribute update date, and chooses the most recent from among those dates, as the recency date for that crosswalk.