Data validation function
The Data Validation Function offering ensures data accuracy, improves data quality, and questions the data ingested in the Reltio platform.
Data accuracy is an essential component to track the quality of data. As the volume of data and complexity of systems increase, errors in data become inevitable. Such data errors can lead to incorrect business decisions.
Data Stewards need an easy, time-efficient way to validate the data that they load into the Reltio Connected Cloud as they need to understand the accuracy of the data source. Business users want to identify the problems in the data so their business needs can be met.
To ensure data accuracy and to improve data quality, the data ingested into the Reltio platform must be questioned or validated. With the Data Validation Function offering, Reltio enables you to validate the ingested data (be it through data sources or human entry) against a set of validation functions and then report the correctness of the data.
Use cases on data validation function
The table below explains the Data Validation Function use cases.
| Persona | Use Case |
|---|---|
| Configurator | As a Configurator, you want to configure data validation functions to ensure that the data meets the business needs. |
| Data Steward | As a Data Steward, you must be able to view exceptions for the data that is loaded in the Hub so that the corrective actions can be taken. The data is validated against the rules that the Configurator has configured. |
| Business User | As a Business User, you want to search and identify the data in the Hub that reduces the accuracy of the business decisions. |
Best practices for validation functions
- Use a short and informative Data Validation Function name that doesn't contain the name of the attribute. Informative names will give you better results in the Data Quality charts.
- The same Data Validation Function name can be applied to multiple attributes
across different entity types. For example, In the following chart, the
names of DVF functions are the same:
To drill down records relevant to the chart, select a metric on the chart to navigate to the Advanced search page, where you can manage profiles with data validation warnings. For more information, see Exploring the Advanced Search and Profiles with invalid data.
- In the Hub's profile view, you can see validation function warnings in
Attribute Validations.
- Use Java 8 compliant Regular expressions (Regex) in the validation functions.
- Escape backslashes in expressions with an extra backslash. For example,
regexp(attributes.FirstName.value, '^\s+.*$')must beregexp(attributes.FirstName.value, '^\\\\s+.*$').
| Use case | DVF name | DVF Expression | Validation function summary message |
|---|---|---|---|
| Finding entities with attributes missing needed values | The needed value is missing | missing(attributes.Date.value) | The attribute type needs at least one value. |
| Finding entities with attributes having no values | Exist but Empty | regexp(attributes.FirstName.value,
'^\s*$') | Attribute value can’t be empty. |
| Finding entities with attributes having space or any non-readable character like \n \t \r as values. | Has Non-Readable Characters | regexp(attributes.FirstName.value,
'(?s)^.*(?:\s|\t).*$') | Attribute value can’t contain space and non-readable characters. |
Finding entities with attributes having space as the
prefix of the value. For example, “ text“
(). | Space as prefix | regexp(attributes.FirstName.value,
'^\s+.*$') | Attribute value can’t start with space. |
| Finding attributes exceeding the 21 character value length. | Character length limit exceeded | not(regexp(attributes.FirstName.value,'^.{0,21}$')) | Attribute value has exceeded the 21 characters limit. |
| Finding entities with values containing special characters | Only English letters and numbers are enabled | not(regexp(attributes.FirstName.value,
'^[a-zA-Z0-9]+$')) | Only English letters and numbers can be used in the value field. All other languages and characters aren’t enabled. |