Get tokenization schemes API

Learn to get detailed information about tokenization schemes of match rules.

GET /reltio/tools/matching/{tenantId}/tokenizationSchemes/{entityType}
?scopes={comma-sep list}
&types={comma-sep list}
&rules={comma-sep list}

Table 1. Parameters
Name	Required	Description	Default value
`entityType`	Yes	The type of entity from where the tokenization schemes are retrieved.	-
`scopes`	No	The scope of the match group that is being considered for the result.	ALL,INTERNAL,EXTERNAL,NONE
`rules`	No	The match group URIs that will be considered for the result.	All
`types`	No	The types of match groups that are being considered for the result.	All
`includeRules`	No	The match group URIs to include in the output when the tokenization schemes are retrieved. This filters the schemesAnalysisExplanations response to return only the match groups you want.	null (Enter a value to filter the response.)

Note: The filtering parameters scopes, types and rules are applied with the AND logic.

Request

Returns the details of tokenization schemes for each matching group.

Example

GET /reltio/tools/matching/{tenantId}/tokenizationSchemes/Individual

Response

{
  "matchRulesTotal": 10,
  "distinctTokenizationSchemesTotal": 6,
  "attributeStats": {
    "participants": [
      "FirstName",
      "..."
    ]
  },
  "operandStats":[
{
	"operand": "exact",
      "total": 15,
      "schemesWithOperand": [
        "scheme1",
        "scheme2"
      ]
},
...
  ],
  "cleanserStats": {
    "NameDictionaryCleanser": [
      "configuration/entityTypes/Individual/matchGroups/MatchGroup1",
      ""
    ],
    "RegexpReplaceCleanser": [
      "configuration/entityTypes/Individual/matchGroups/MatchGroup1",
      ""
    ]
  },
  "tokenizationSchemes": [
    {
      "id": "scheme1",
	"humanReadableTokenizationScheme": "exact(FirstName,matchTokenClassId1):fuzzy(LastName,matchTokenClassId2):...",
      "matchGroups": [
        "configuration/entityTypes/Individual/matchGroups/MatchGroup1",
        ""
      ]
    }
  ],
  "matchTokenClasses": [
    {
      "id": "matchTokenClassId1",
      "class": "ExactMatchToken",
      "parameters": [
        {}
      ],
      "matchGroups": [
        "configuration/entityTypes/Individual/matchGroups/MatchGroup1",
        ""
      ],
      "attributes": [
        "FirstName",
        ""
      ]
    }
  ],
  "matchGroups": [
    {
      "uri": "configuration/entityTypes/Individual/matchGroups/MatchGroup1",
      "label": "some label",
	"type": "suspect",
      "tokenizationSchemes": [
        "scheme1",
        ""
      ],
      "ignoreInToken": [
        "FirstName",
        ""
      ]
    }
  ],
  "schemesAnalysis": {
	"schemesWithTheSameAttributes": [
      [
        {
          "schemeId": "scheme1",
          "humanReadableTokenizationScheme": "exact(FirstName, matchTokenClassId2):exact(MiddleName, matchTokenClassId2)"
        },
        {
          "schemeId": "scheme3",
          "humanReadableTokenizationScheme": "exact(MiddleName, matchTokenClassId2):exact(FirstName, matchTokenClassId2)"
        }
      ],
      ...
    ],
    "sameSchemesWithDifferentClasses": [
      [
        {
          "schemeId": "scheme1",
          "humanReadableTokenizationScheme": "exact(FirstName, matchTokenClassId2)"
        },
        {
          "schemeId": "scheme2",
          "humanReadableTokenizationScheme": "exact(FirstName, matchTokenClassId1)"
        }
      ]
    ],
    "sameSchemesWithDifferentAttributesOrder": [
      [
        {
          "schemeId": "scheme1",
          "humanReadableTokenizationScheme": "exact(FirstName, matchTokenClassId2):exact(MiddleName, matchTokenClassId2)"
        },
        {
          "schemeId": "scheme3",
          "humanReadableTokenizationScheme": "exact(MiddleName, matchTokenClassId2):exact(FirstName, matchTokenClassId2)"
        }
      ]
    ],
    "nonSchemeGeneratingRules": [
      "configuration/entityTypes/Individual/matchGroups/JustComparatorGroup"
    ],
    "excessStrictSchemes": {
      "byAttributes": [
        {
          "loseScheme": {
            "schemeId": "scheme8",
            "humanReadableTokenizationScheme": "equals(FirstName, [someValue])"
          },
          "strictSchemes": [
            {
              "schemeId": "scheme9",
              "humanReadableTokenizationScheme": "fuzzy(FirstName, matchTokenClassId2):equals(FirstName, [someValue])"
            },
            {
              "schemeId": "scheme7",
              "humanReadableTokenizationScheme": "exact(LastName, matchTokenClassId2):equals(FirstName, [someValue])"
            }
          ]
        },
        ...      
],
      "byFull": [
        {
          "loseScheme": {
            "schemeId": "scheme11",
            "humanReadableTokenizationScheme": "exact(LastName, matchTokenClassId2)"
          },
          "strictSchemes": [
            {
              "schemeId": "scheme2",
              "humanReadableTokenizationScheme": "exact(LastName, matchTokenClassId2):exact(MiddleName, matchTokenClassId2)"
            },
            {
              "schemeId": "scheme7",
              "humanReadableTokenizationScheme": "exact(LastName, matchTokenClassId2):equals(FirstName, [someValue])"
            }
          ]
        },
        ...
      ]
    }

  }
}

The following table explains the fields in the response:


Field	Description
`matchRulesTotal`	The total number of match rules analyzed. The number depends on the request parameters.
`distinctTokenizationSchemesTotal`	The number of distinct tokenization schemes.
`attributeStats`	Overall statistics of attributes participating in tokenization schemes.
`attributeStats.participants`	A list of attributes participating in the analysis.
`operandStats`	Overall statistics of operands participating in tokenization schemes.
`cleanserStats`	Overall statistics for cleansers.
`cleanserStats.NameDictionaryCleanser`	A list of match rules where the `NameDictionaryCleanser` is applied.
`cleanserStats.RegexpReplaceCleanser`	A list of match rules where the `RegexpReplaceCleanser` is applied.
`cleanserStats.TransliterateCleanser`	A list of match rules where the `TransliterateCleanser` is applied.
`tokenizationSchemes`	Describes the tokenization schemes.
`tokenizationSchemes.id`	Autogenerated identifier of a tokenization scheme.
`tokenizationSchemes.humanReadableTokenizationScheme`	Describes the tokenization scheme having operands, attributes and match token classes that are separated by colon. `multi([FirstName, matchTokenClassId1:LastName, matchTokenClassId2],matchTokenClassId3):multi([GeoAttribute.latitude, GeoAttribute.longitude],matchTokenClassId4):fuzzy(Address.AddressLine1, matchTokenClassId5):equals(Status,[Active,On])`
`tokenizationSchemes.matchGroups`	A list of match groups that use the particular tokenization scheme.
`matchTokenClasses`	Provides information about match token classes.
`matchTokenClasses.id`	Autogenerated ID of the match token class. The ID is unique and a new ID is generated for match token classes with different parameters.
`matchTokenClasses.class`	Name of the match token class.
`matchTokenClasses.parameters`	A list of parameters in the match token class.
`matchTokenClasses.matchGroups`	A list of match groups in the match token class.
`matchTokenClasses.attributes`	A list of attributes having the match token class.
`matchGroups`	Provides information about the match groups.
`matchGroups.uri`	URI of the match group.
`matchGroups.label`	Label of the match.
`matchGroups.type`	Type of match group.
`matchGroups.tokenizationSchemes`	A list of tokenization schemes generated by the match group.
`matchGroups.ignoreInToken`	A list of attributes that are ignored in a token for the match group.
`schemesAnalysis`	The analysis of the tokenization schemes.
`schemesAnalysis.schemesWithTheSameAttributes`	The tokenization schemes that have the same attributes scheme.
`schemesAnalysis.sameSchemesWithDifferentClasses`	The tokenization schemes that differ only by match token classes.
`schemesAnalysis.sameSchemesWithDifferentAttributesOrder`	The tokenization schemes that differ by the order of attributes.
`schemesAnalysis.nonSchemeGeneratingRules`	A list of the match groups that do not generate any schemes.
`schemesAnalysis.excessStrictSchemes`	The tokenization schemes having additional attributes compared to the other schemes.
`schemesAnalysis.excessStrictSchemes.byAttributes`	The tokenization schemes having additional attributes compared to the other schemes. Compared only by attributes.
`schemesAnalysis.excessStrictSchemes.byAttributes.loseScheme`	The tokenization schemes having fewer attributes.
`schemesAnalysis.excessStrictSchemes.byAttributes.strictSchemes`	The tokenization schemes having more attributes.

Unify and manage your data

Get tokenization schemes API

Footer