Detection Method of Approximately Duplicated Chinese Address Information
It's a hot issue to eliminate approximately duplicated records in data cleansing operation of data warehouse,in which the address information play an important role to identify the same entity.Aiming at the processing of Chinese address information,the meta-database of segment rules is established,and an approximately duplicated detection model is proposed.The feature word based segment algorithm and similarity computation algorithm are presented.The experiment results indicate that this method can detect approximately duplicated records effectually,the algorithm running efficiency and detect precision can be improved.