Most of the state-of-the-art MapReduce-based entity match- ing methods inherit traditional Entity Resolution techniques on central- ized system and focus on data blocking strategies for structured entities in order to solve the load balancing problem occurred in distributed en- vironment. In this paper, we propose a MapReduce-based entity match- ing framework for Entity Matching on semi-structured and unstructured data. Each entity is represented by a high dimensional vector gener- ated from description data. In order to reduce network transmission, we produce lower dimensional bit-vectors called signatures for those entity vectors based on Locality Sensitive Hash (LSH) function. Our LSH is required for promising cosine similarity. A series of random algorithms are designed to ensure the performance for entity matching. Moreover, our design contains a solution for reducing redundant computation by one round of additional MapReduce job. Experiments show that our ap- proach has a huge advantages on both processing speed and accuracy compared to the other methods.
['Pingfu Chao', 'Zhu Gao', 'Yuming Li', 'Junhua Fang', 'Rong Zhang', 'Aoying Zhou']
Random-Based Algorithm for Efficient Entity Matching