Using String Function Cleanser and Address Cleanser Together

Reltio allows you to use the StringFunctionCleanser and the AddressCleanser one after the other.

The following use case helps you to understand how the two cleansers work one after the other:

Sample Address

{
 "Address1": "12021 Wilmington (Ave) 1st$ Fl",
 "City": "Los Angeles",
 "State": "CA",
 "Country": "United States",
 "Pincode": "90059"
}

At first, the above address is cleansed for any special character that are present. This address is then verified with the Geo-location.

The Reltio StringFunctionCleanser cleanses the address for the special characters and then the AddressCleanser cleanses the entire address.

Expected Address Output after Applying the String Cleanser

The above sample address gives the following output after applying the StringFunctionCleanser to scrub the special characters:

{
 "Address1": "12021 Wilmington Ave 1st Fl",
 "City": "Los Angeles",
 "State": "CA",
 "Country": "United States",
 "Pincode": "90059"
}

Expected Address Output after Applying the Address Cleanser

The above sample address gives the following output after applying the AddressCleanser to standardize the address fields:

{
 "PremiseNumber": "12021",
 "DeliveryAddress": "12021 Wilmington Ave Fl 1",
 "Address": "12021 Wilmington Ave Fl 1<BR>Los Angeles CA 90059-3019",
 "DeliveryAddress1": "12021 Wilmington Ave Fl 1",
 "Locality": "Los Angeles",
 "AdministrativeArea": "CA",
 "CountryName": "United States",
 "PostalCode": "90059-3019",
 "PostalCodePrimary": "90059",
 "PostalCodeSecondary": "3019",
 "Latitude": "33.923550",
 "SubBuilding": "Fl 1",
 "Premise": "12021",
 "Address1": "12021 Wilmington Ave Fl 1",
 "Address2": "Los Angeles CA 90059-3019",
 "ISO3166-2": "US",
 "ISO3166-3": "USA",
 "GeoAccuracy": "P4",
 "GeoDistance": "0.0",
 "ISO3166-N": "840",
 "Thoroughfare": "Wilmington Ave",
 "Longitude": "-118.239390",
 "AVC": "V44-I55-P7-100",
 "PremiseNumberStatus": "fsVerifiedNoChange",
 "LocalityStatus": "fsVerifiedNoChange",
 "SubAdministrativeArea": "Los Angeles",
 "AdministrativeAreaStatus": "fsAdded",
 "ThoroughfareStatus": "fsVerifiedNoChange",
 "MatchRuleLabel": "1a"
}

Recommendation

In order to achieve the desired result, Reltio recommends you to use different resultingValuesSourceTypeURI (crosswalk) for StringFunctionCleanser and AddressCleanser in cleanseconfig. For example:

{
  "cleanseConfig": {
    "mappings": [
      {
        "uri": "configuration/entityTypes/Location/cleanse/mappings/address",
        "outputMapping": [
          {
            "attribute": "configuration/entityTypes/Location/attributes/VerificationStatus",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "VerificationStatus"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/VerificationStatusDetails",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "VerificationStatusDetails"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/StateProvince",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "AdministrativeArea"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/City",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "Locality"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/Country",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "ISO3166-2"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/Zip/attributes/Zip5",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "PostalCodePrimary"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/Zip/attributes/Zip4",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "PostalCodeSecondary"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/GeoLocation/attributes/Latitude",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "Latitude"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/GeoLocation/attributes/Longitude",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "Longitude"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/GeoLocation/attributes/GeoAccuracy",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "GeoAccuracy"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/Organization",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "Organization"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/DeliveryAddress1",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "DeliveryAddress1"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/AddressLine1",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "DeliveryAddress1"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/AddressLine2",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "DeliveryAddress2"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/ISO3166-2",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "ISO3166-2"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/ISO3166-3",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "ISO3166-3"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/DeliveryAddress",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "DeliveryAddress"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/PremiseNumber",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "PremiseNumber"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/ISO3166-N",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "ISO3166-N"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/Unmatched",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "Unmatched"
          },
          {
            "attribute": "configuration/entityTypes/Location/attributes/AVC",
            "mandatory": false,
            "allValues": false,
            "cleanseAttribute": "AVC"
          }
        ]
      }
    ],
    "infos": [
      {
        "uri": "configuration/entityTypes/Location/cleanse/infos/other",
        "useInCleansing": true,
        "sequence": [
          {
            "chain": [
              {
                "cleanseFunction": "Loqate",
                "resultingValuesSourceTypeUri": "configuration/sources/ReltioCleanser",
                "proceedOnSuccess": true,
                "proceedOnFailure": true,
                "mapping": {
                  "inputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/AddressLine1",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "Address1"
                    },
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/AddressLine2",
                      "mandatory": false,
                      "allValues": false,
                      "cleanseAttribute": "Address2"
                    },
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/Country",
                      "mandatory": false,
                      "allValues": false,
                      "cleanseAttribute": "Country"
                    },
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/StateProvince",
                      "mandatory": false,
                      "allValues": false,
                      "cleanseAttribute": "AdministrativeArea"
                    },
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/City",
                      "mandatory": false,
                      "allValues": false,
                      "cleanseAttribute": "Locality"
                    },
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/Zip/attributes/Zip5",
                      "mandatory": false,
                      "allValues": false,
                      "cleanseAttribute": "PostalCode"
                    },
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/Organization",
                      "mandatory": false,
                      "allValues": false,
                      "cleanseAttribute": "Organization"
                    }
                  ],
                  "outputMappingRef": "configuration/entityTypes/Location/cleanse/mappings/address/outputMapping"
                },
                "params": {
                  "verificationStatusMapping": {
                    "Verified": [
                      "V(4|5).*"
                    ],
                    "Partially Verified": [
                      "V(1|2|3).*",
                      "P.*"
                    ],
                    "Unverified": [
                      "U.*"
                    ],
                    "Ambiguous": [
                      "A.*"
                    ],
                    "Conflict": [
                      "C.*"
                    ],
                    "Reverted": [
                      "R.*"
                    ]
                  }
                }
              }
            ]
          },
          {
            "chain": [
              {
                "cleanseFunction": "StringFunctionCleanser",
                "resultingValuesSourceTypeUri": "configuration/sources/StringCleanser",
                "proceedOnSuccess": true,
                "proceedOnFailure": true,
                "mapping": {
                  "inputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/AddressLine1",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "InputText"
                    }
                  ],
                  "outputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/AddressLine1",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "OutputText"
                    }
                  ]
                },
                "params": {
                  "casing": "Title"
                }
              },
              {
                "cleanseFunction": "StringFunctionCleanser",
                "resultingValuesSourceTypeUri": "configuration/sources/StringCleanser",
                "proceedOnSuccess": true,
                "proceedOnFailure": true,
                "mapping": {
                  "inputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/AddressLine2",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "InputText"
                    }
                  ],
                  "outputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/AddressLine2",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "OutputText"
                    }
                  ]
                },
                "params": {
                  "casing": "Title"
                }
              },
              {
                "cleanseFunction": "StringFunctionCleanser",
                "resultingValuesSourceTypeUri": "configuration/sources/StringCleanser",
                "proceedOnSuccess": true,
                "proceedOnFailure": true,
                "mapping": {
                  "inputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/City",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "InputText"
                    }
                  ],
                  "outputMapping": [
                    {
                      "attribute": "configuration/entityTypes/Location/attributes/City",
                      "mandatory": true,
                      "allValues": false,
                      "cleanseAttribute": "OutputText"
                    }
                  ]
                },
                "params": {
                  "casing": "Title"
                }
              }
            ]
          }
        ]
      }
    ]
  }
}

The following table explains how the StringFunctionCleanser and the AddressCleanser works one after the other:

Table 1. String Function Cleanser and Address Cleanser Parameters
Cleanse Configuration Parameter	Description
`"resultingValuesSourceTypeUri": "configuration/sources/StringCleanser"`	The `StringCleanser` crosswalk is used for string cleansing. The output obtained is the address cleansed for the special characters by the `StringFunctionCleanser`.
`"resultingValuesSourceTypeUri": "configuration/sources/ ReltioCleanser"`	The `ReltioCleanser` crosswalk uses the cleansed value from the `StringCleanser` crosswalk as the source. The `AddressCleanser` further cleanses the entire address for verification and Geo-location.

Note: If you want to apply the AddressCleanser before applying the StringFunctionCleanser, you can simply change the order of cleansers in cleanseConfig.

Unify and manage your data

Using String Function Cleanser and Address Cleanser Together

Sample Address

Expected Address Output after Applying the String Cleanser

Expected Address Output after Applying the Address Cleanser

Recommendation

Footer