Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: MapReduce programming model is a popular model to simplify but speed up data parallel applications. Hover, it is not efficient for iterative applications because of its repeated data transmission with HDFS (Hadoop Distributed File System). Conch, a cyclic MapReduce model, is designed for efficient processing of iterative applications. In order to minimize network overhead, shared data is cached locally and a "map-shuffle" phase is presented with a combined transmission mechanism. Meanwhile, a prediction scheduler for iterative applications is brought out to achieve better data locality in terms of runtime information. The experiments show that Conch can support iterative applications transparently and efficiently. Compared with Hadoop and HaLoop in single-job environment, Conch can achieve 13%-17% improvements on K-Means and fuzzy C-Means. Especially in multi-job environment, 63.6% and 28.6% improvements can be obtained compared with Hadoop and HaLoop.
['Ran Zheng', 'Genmao Yu', 'Hai Jin', 'Xuanhua Shi', 'Qin Zhang']
I am writing a research paper on a idea Conch: A Cyclic MapReduce Model for Iterative Applications
[{"from": "human", "value": "I am writing a research paper on a idea Conch: A Cyclic MapReduce Model for Iterative Applications"}, {"from": "gpt", "value": "Thats a unique idea.You should consider the following idea to enhance your idea so that it would impact uniquely: MapReduce programming model is a popular model to simplify but speed up data parallel applications. Hover, it is not efficient for iterative applications because of its repeated data transmission with HDFS (Hadoop Distributed File System). Conch, a cyclic MapReduce model, is designed for efficient processing of iterative applications. In order to minimize network overhead, shared data is cached locally and a \"map-shuffle\" phase is presented with a combined transmission mechanism. Meanwhile, a prediction scheduler for iterative applications is brought out to achieve better data locality in terms of runtime information. The experiments show that Conch can support iterative applications transparently and efficiently. Compared with Hadoop and HaLoop in single-job environment, Conch can achieve 13%-17% improvements on K-Means and fuzzy C-Means. Especially in multi-job environment, 63.6% and 28.6% improvements can be obtained compared with Hadoop and HaLoop."}]