Configure filter expressions for cleanse function

Learn how to use filter expressions to control when a cleanse function executes on entity data.

A filter expression is an optional condition defined on a cleanse function to control whether the function executes for a given entity data. Use a filter expression when you want the cleanse function to run only for specific entity data. This helps avoid unnecessary processing and ensures the function runs only when needed.

Configure a filter expression

To configure, define a filter expression at the cleanse-function level in the infos[].sequence[].chain[] section of the cleanse configuration.

The following path shows the location of the filter in the cleanse configuration.

cleanseConfig.infos[].sequence[].chain[].filter

For example,

{
  "uri": "configuration/entityTypes/Individual",
  "cleanseConfig": {
    "infos": [
      {
        "sequence": [
          {
            "chain": [
              {
                "cleanseFunction": "StringFunctionCleanser",
                "resultingValuesSourceTypeUri": "configuration/sources/ReltioCleanser",
                "filter": "equals(attributes.Country, 'US') and exists(attributes.Address)",
                "mapping": {
                  "inputMapping": [ ... ],
                  "outputMapping": [ ... ]
                }
              }
            ]
          }
        ]
      }
    ]
  }
}

Filter evaluation and behavior

The filter expression is evaluated at runtime for each entity processed by the cleanse framework. If the ovOnly option is enabled, the filter expression is evaluated only against operational values (OV).

If a filter expression is defined, Reltio evaluates it against the current entity data to determine whether the cleanse function must be executed. The expression evaluates to true when the entity data matches the filter condition, and to false when the entity data does not match the filter condition.

The following table describes how cleanse execution behaves based on whether a filter expression is defined and how it evaluates at runtime.


Filter expression	Cleanse behavior
No filter defined	Cleanse executes for all entity data
Filter expression is defined and it evaluates to true	Cleanse executes on the entity data and a cleanser crosswalk is created or updated
Filter expression is defined and it evaluates to false	Cleanse is skipped, no data is written, and no cleanser crosswalk is created or modified

Use multiple filter expressions for conditional cleansing

A cleanse function supports multiple steps, each with its own filter expression, to implement conditional attribute derivation. Each step evaluates independently against the set of records processed by the cleanse function. When a step's filter does not match a record, the step is skipped for the record, but the remaining steps still evaluate and execute if their filter conditions match.

Apply this pattern when an attribute value depends on mutually exclusive conditions. Configure each step with its own filter, and the step whose filter matches a record sets the value for that record.

In the following example, a three-step conditional cleanse function derives the Name attribute based on the different format for different Country. The first two steps target specific country values, and the final step uses a default format to generate the Name attribute.

{
                "chain": [
                  {
                    "cleanseFunction": "PatternBasedFieldBuilder",
                    "resultingValuesSourceTypeUri": "configuration/sources/ReltioCleanser",
                    "proceedOnSuccess": false,
                    "proceedOnFailure": true,
                    "filter": "equals(attributes.Country, 'US')",
                    "mapping": {
                      "inputMapping": [
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/FirstName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "FirstName"
                        },
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/MiddleName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "MiddleName"
                        },
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/LastName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "LastName"
                        }
                      ],
                      "outputMapping": [
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/Name",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "OutputText"
                        }
                      ]
                    },
                    "params": {
                      "isForce": true,
                      "pattern": "{LastName}, {FirstName}"
                    }
                  },
                  {
                    "cleanseFunction": "PatternBasedFieldBuilder",
                    "resultingValuesSourceTypeUri": "configuration/sources/ReltioCleanser",
                    "proceedOnSuccess": false,
                    "proceedOnFailure": true,
                    "filter": "equals(attributes.Country, 'IN')",
                    "mapping": {
                      "inputMapping": [
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/FirstName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "FirstName"
                        },
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/MiddleName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "MiddleName"
                        },
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/LastName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "LastName"
                        }
                      ],
                      "outputMapping": [
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/Name",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "OutputText"
                        }
                      ]
                    },
                    "params": {
                      "isForce": true,
                      "pattern": "{FirstName}, {LastName}"
                    }
                  },
                  {
                    "cleanseFunction": "PatternBasedFieldBuilder",
                    "resultingValuesSourceTypeUri": "configuration/sources/ReltioCleanser",
                    "proceedOnSuccess": false,
                    "proceedOnFailure": true,
                    "filter": "not(equals(attributes.Country, 'US')) and not(equals(attributes.Country, 'IN'))",
                    "mapping": {
                      "inputMapping": [
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/FirstName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "FirstName"
                        },
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/MiddleName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "MiddleName"
                        },
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/LastName",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "LastName"
                        }
                      ],
                      "outputMapping": [
                        {
                          "attribute": "configuration/entityTypes/Individual/attributes/Name",
                          "mandatory": false,
                          "allValues": false,
                          "cleanseAttribute": "OutputText"
                        }
                      ]
                    },
                    "params": {
                      "isForce": true,
                      "pattern": "{FirstName} {MiddleName} {LastName}"
                    }
                  }
                ]
              }

For each record, only the step with a matching filter executes. The remaining steps are skipped without affecting the record or the rest of the chain.

Runtime evaluation for object and nested attributes

Filter evaluation behavior depends on whether the cleanse function operates on object level or nested attributes.

Object level cleansing

The filter is evaluated once against the entire entity.

If the filter expression evaluates to true, the cleanse function executes for the entity.
If the filter expression evaluates to false, the cleanse function is skipped.

For example, run the cleanse function only for entities where Country is US:

"filter":"equals(attributes.Country, 'US')"

Nested attribute cleansing

For nested attributes (such as Address or Consent), filter evaluation occurs in two steps:

Entity level evaluation
- The filter is evaluated against the entire entity.
- If the entity does not match, the cleanse function is skipped for all nested values
Nested value evaluation
- If the entity matches, the filter is evaluated for each nested value.
- During this evaluation, attribute paths are resolved relative to the nested context.

For example, run the cleanse function only for Address values where City is NYC:

equals(attributes.Address.City, 'NYC')

If an entity has multiple Address values, the filter is evaluated for each Address. Only the Address values where City = NYC are processed, and the remaining Address values are skipped.

Supported filter expressions

Filter expressions use the same syntax as Elasticsearch (ES) and Data Validation Functions (DVF) expressions, see Search, Understanding the Expression Attribute.

The following table shows common filter expression patterns.


Construct	Example	Description
Attribute equality	equals(attributes.Country, 'US')	Matches a specific attribute value
Attribute existence	exists(attributes.Email)	Checks if an attribute is present
Value in list	in(attributes.Country, 'US,IN')	Matches one of multiple values
Nested attribute	equals(attributes.Address.City, 'NYC')	Matches values in nested attributes
Crosswalk condition	equals(sourceSystems, 'CRM')	Matches data from a specific source
Composite expression	equals(attributes.Country, 'US') and exists(attributes.Email)	Combines multiple conditions

Filter expression validation

All filter expressions are validated when the configuration is saved.

Invalid syntax: If the filter contains a syntax error (for example, an unclosed parenthesis or an unknown function), the configuration save operation fails.
Error message: The response includes a detailed error message from the expression parser, indicating the exact issue and preventing an invalid configuration from being deployed.

Unify and manage your data