Declaring Action Thresholds

When declaring the ranges for queue_for_review and auto_merge, the combination of them should span the entire available range of 0.0 to 1.0 with no gap and no overlap except that the upper endpoint for queue_for_review should equal the lower endpoint for auto_merge thus have a common touchpoint between them. (for example, 0.0 to 0.6 for queue_for_review, and 0.6 to 1.0 for auto_merge.) If the actionThresholds leave a gap, then any score falling within the gap will produce no action. Conversely, if the actionThresholds overlap (for example, 0.4 to 0.6 for queue_for_review, and 0.5 to 0.7 for auto_merge) and a score lands within the intersection (0.55 in our example) or on the touchpoint, the directive of queue_for_review takes precedence.

Primary Use of Relevance-Based Matching

Using Relevance-based matching you could create a match rule that contains a collection of attributes you wish to test as a group, but assign weights to those attributes to govern their relative importance in the rule. Start with all weights equal to 1.0 and perhaps start with actionThresholds of 0.0-0.5 for queue_for_review and 0.5-1.0 for auto_merge. Do some trial runs and examine the results. If you find that too many obvious matches are being set to queue_for_review, then you can adjust weights and even modify the actionThresholds to perhaps 0.0-0.7, and 0.7-1.0. Iterate and experiment until you are able to get optimized results with your data set. Always remember that data quality is a key component of effective matching. So good use of cleansing before the data is posted and proper cleansing using the out-of-the-box match cleansers are important.

Support for customer-defined match actions in a relevance-based rule

In the image below, notice that on the left the admin has two actionThresholds defined and they use the system values of auto_merge and potential_match. However, if the admin had defined matchActions as you see on the right, then she can replace the system values with her matchActions. This provides her the ability to create more advanced outcomes when a threshold is reached. For more details on matchActions, see Understanding Custom Rules.

Calculation of Score

If an attribute has multiple values then the maximum value of relevance is selected.

RelevanceScore_OR=max(relevance₁ , ..., relevance_i, ..., relevance_N)

• relevance_i- relevance of simple attribute, nested attribute, logical operand participating in the OR operand directly.

RelevanceScoreNOT=1-RelevanceScore_{AND,OR,exact,...}

The relevance score of the NOT operand is equal to 1 minus the relevance score of the operand having this negation.

Sample configuration using relevance-based matching for auto_merge and queue_for_review

{
       "uri": "configuration/entityTypes/HCP/matchGroups/ByNames",
       "label": "By Names",
       "type": "relevance_based",
       "rule": {
           "and": {
               "exact": [
                   "configuration/entityTypes/HCP/attributes/FirstName",
                   "configuration/entityTypes/HCP/attributes/LastName"
               ],
               "exactOrAllNull": [
                   "configuration/entityTypes/HCP/attributes/Suffix"
               ]
           },
           "weights": [
               {
                   "attribute": "configuration/entityTypes/HCP/attributes/Suffix",
                   "weight": 0.2
               }
           ],
           "actionThresholds": [
               {
                   "type": "auto_merge",
                   "threshold": "0.8-1.0"
               },
               {
                   "type": "potential_match",
                   "threshold": "0.4-0.8"
               }
           ],
           "matchTokenClass": "com.reltio.match.token.ExactMatchToken"
       },
       "matchServiceClass": "com.reltio.businesslogic.match.providers.internal.InternalMatchService",
   },
   
   }

Example profile pair for the configuration above:

{
   "type": "configuration/entityTypes/HCP",
   "attributes": {
       "FirstName": [
           {
               "value": "Robert"
           }
       ],
       "LastName": [
           {
               "value": "Downy"
           }
       ],
       "Suffix": [
           {
               "value": "Sr."
           }
       ]
   }
},

{
   "type": "configuration/entityTypes/HCP",
   "attributes": {
       "FirstName": [
           {
               "value": "Robert"
           }
       ],
       "LastName": [
           {
               "value": "Downy"
           }
       ],
       "Suffix": [
           {
               "value": "Jr."
           }
       ]
   }
}

• true for First Name;
• true for LastName
• false for Suffix

In this case the logic tree uses AND as the root and so the formula for the AND operator is . The score is calculated as (1*1 + 1*1 + 0*1) / (1 + 1 + 1) = ⅔ = .66

With score of .66 the directive for this pair will be set to queue_for_review.

Using the Verify Matches API with Relevance-Based Matching

{
 "versions": {
   "first": {
     "EntityVersion": 1000000000
   },
   "second": {
     "EntityVersion": 1000000000
   }
 },
 "rules": {
   "configuration/entityTypes/HCP/matchGroups/ByNames": {
   "label": "By Names",
   "useOvOnly": false,
   "matchTokens": {
		"first": {
			"foundInMatchTables": true,
			"tokens": [
				"robert:downy"
			],
			"tokensGenerated": 1
       },
       "second": {
			"foundInMatchTables": true,
			"tokens": [
				"robert:downy"
			],
			"tokensGenerated": 1
       },
       "intersection": {
			"tokens": [
				"robert:downy"
			]
       }
     },
     "rule": {
       "relevance": 0.9090909090909091,
       "and": [
         {
           "relevance": 0.9090909090909091,
           "and": [
             {
               "exact": {
                 "FirstName": {
                   "ignoreInToken": false,
                   "relevance": 1.0
                 },
                 "LastName": {
                   "ignoreInToken": false,
                   "relevance": 1.0
                 }
               },
               "exactOrAllNull": {
                 "Suffix": {
                   ""ignoreInToken": true,
                   "relevance": 0.2
                 }
               },
               "relevance": 0.9090909090909091
             }
           ]
         }
       ]
     },
     "matched": true,
     "matchAction": "AUTO_MERGE",
     "relevance": 0.9090909090909091
   }
 }
}

Weights and Action Thresholds

In the match group configuration, you will define Weights and actionThresholds. The weight property allows you to assign a relative weight (strength) for each attribute. For example, you may decide that Middle Name is less reliable and thus less important than First Name.

The actionThreshold allows you to define a range of scores to drive a directive. For example, you might decide that the match group should merge the profile pair if the score is between 0.9 to 1.0, but should queue the pair for review if the score falls into a lower range of 0.6 to 0.9.

Assigning Weights

Weights can be set from 0.0 to 1.0. If you do not explicitly set a weight for an attribute, it will receive a default weight of 1.0 during execution of the rule.

Multiple Actions of the Same Type

You can configure a relevance-based match rule with multiple action thresholds having the same action type but with a different relevance score range.

Example Match Group Configuration

{
  "uri": "configuration/entityTypes/HCP/matchGroups/PotentialMatchByEmail",
  "label": "By Email",
  "type": "relevance_based",
  "rule": {
    "and": {
      "fuzzy": [
        "configuration/entityTypes/HCP/attributes/Email/attributes/Email"
      ]
    },
    "actionThresholds": [
      {
        "type": "potential_match",
        "threshold": "0.99-1",
        "label": "candidate for merge"
      },
      {
        "type": "potential_match",
        "threshold": "0.9-0.99",
        "label": "suspect match for review"
      }
    ],
  }
}

In the above example, the type is potential_match for two different action thresholds. You can differentiate such thresholds by assigning appropriate labels. You can generate potential matches with different labels based on the range of the relevance score that allows you to differentiate between higher and lower relevance score matches. You can resolve matches quickly based on the label. In the example above, based on the relevance score, some potential matches can be considered for merging directly while others must be reviewed before any action is taken. The results of the API to get potential matches and the external match API will contain a relevance value and a matchActionLabel corresponding to each of the action type configured under the actionThreshold parameter. For more information, see Potential matches API and External match API.

Including Equals and Not Equals Constraints in Tokenization and Relevance Score Calculation

Using operators like equals and notEquals prevents tokenization from generating tokens. These operators should not have an impact on tokenization, if we want to compare and conclude that even though address and/or email and/or phone are different, the remaining attributes match enough to take the score above the threshold.

Business Configuration

{
  "notEquals": [
    {
      "uri": "configuration/entityTypes/Individual/attributes/FirstName",
      "value": "unknown",
      "strict": false,
      "weight": 2.5
    }
  ]
}

Formulae to Calculate Relevance Score