Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: The processing of big data is a hotspot in the scientific research. Data on the Internet is very large and also very important for the scientific researchers, so the capture and store of Internet data is a priority among priorities. The traditional single-host b spider and data store approaches have some problems such as low efficiency and large memory requirement, so this paper proposes a big data store-retrieve approach DSMC (distributed store-retrieve approach using MapReduce model and community detection) based on distributed processing. Firstly, the distributed capture method using MapReduce to deduplicate big data is presented. Secondly, the storage optimization method is put forward; it uses the hash functions with light-ight characteristics and the community detection to address the storage structure and solve the data retrieval problems. DSMC has achieved the high performance of large b data comparison and storage and gets the efficient data retrieval at the same time. The experimental results show that, in the Cloudsim platform, comparing with the traditional b spider, the proposed DSMC approach shows better efficiency and performance.
['Xu Xu', 'Jia Zhao', 'Gaochao Xu', 'Yan Ding', 'Yunmeng Dong']
I am writing a research paper on a idea DSMC: A Novel Distributed Store-Retrieve Approach of Internet Data Using MapReduce Model and Community Detection in Big Data
[{"from": "human", "value": "I am writing a research paper on a idea DSMC: A Novel Distributed Store-Retrieve Approach of Internet Data Using MapReduce Model and Community Detection in Big Data"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: The processing of big data is a hotspot in the scientific research. Data on the Internet is very large and also very important for the scientific researchers, so the capture and store of Internet data is a priority among priorities. The traditional single-host b spider and data store approaches have some problems such as low efficiency and large memory requirement, so this paper proposes a big data store-retrieve approach DSMC (distributed store-retrieve approach using MapReduce model and community detection) based on distributed processing. Firstly, the distributed capture method using MapReduce to deduplicate big data is presented. Secondly, the storage optimization method is put forward; it uses the hash functions with light-ight characteristics and the community detection to address the storage structure and solve the data retrieval problems. DSMC has achieved the high performance of large b data comparison and storage and gets the efficient data retrieval at the same time. The experimental results show that, in the Cloudsim platform, comparing with the traditional b spider, the proposed DSMC approach shows better efficiency and performance."}]