Dedupe, Merge Purge, Duplicate Detection, Data Cleansing Software

After sanitization, the input string is given to the word pattern recognition routines. Each element is examined by the expert system. The expert system determines how an element should be manipulated.

There are two sub functions used by the word pattern recognition routines. The first is phrase recognition. Here multiple word phrases are recognized and replaced with one common phrase. For example, I.B.M will be converted to International Business Machines. Following phrase replacement the name is passed to the rulebase expert system.

The rulebase expert system is used to identify nicknames. Entities such as Bill, William, Bob and Robert are used interchangeably to identify individuals. The rulebase is also used to identify noise words. Noise words are elements in a name that do not help in the identification of a candidate. Examples of noise words are: Incorporated, Corporation, Limited, Junior, Senior, Avenue and Street. Often there are times where elements in a name contribute to the identity but should be treated as less important. In these cases, the rulebase does not treat them as noise words but recognizes that they are less significant. Some examples are: associate, board, international and services. Other variations are caused by the use of common prefixes. Names like McDonnell, are confused with MacDonnell. Prefix recognition provides the facility for handling these classes of problems. The rulebase can also recognize diminutives. Frequently there are names which end in a diminutive such as "ie" or "y". In these cases, it is useful to identify the root and apply the rule. For example, you would want Bill, Billie and Billy to find William or Willie.

NameSearch® comes with an extensive predefined set of rules. These rules can be used right out of the box or modified to meet your specific needs. Rulebase modifications are done through the Generation Shell.

How NameSearch® Works - Word Pattern Recognition