An Intelligent Scientific Workflows Failure Prediction Model using Ensemble Learning Technique
Cloud computing is a distributed computing paradigm which is considered as the computing platform that is going to be the pioneering field for the next ten years. Apart from several industrial, business applications being deployed, this paradigm is additionally attracting several scientific communities to utilize the services of the cloud for running massive scale knowledge and computation intensive applications like a montage, that is employed in astronomy. workflow is defined as a group of task and dependencies between the tasks that are used for expressing numerous scientific applications. The main issue in running these workflow applications is mapping the tasks of the workflow to an appropriate resource in the cloud environment. Scheduling these workflows in a computing environment. To overcome these failures, the workflow scheduling system should be fault tolerant. The fault tolerance by using replication and resubmission of tasks supported the priority of the tasks. The replication of tasks depends on a heuristic metric that is calculated by finding the trade-off between the replication issue and resubmission issue. As scientific workflows scale to many thousands of distinct tasks, failures because of the software package and hardware faults become progressively common. We study job failure models for data collected from different scientific applications, by our proposed framework. In particular, we show that the Ensemble Learning classifier can accurately predict the failure probability of jobs. Failure prediction models have been implemented through machine learning approaches and evaluated performance metrics. The models allow us to predict job failures for a given execution resource and then use these failure predictions for two higher-level goals: (1) to suggest a better job assignment, and (2) to provide quantitative feedback to the workflow component developer about the robustness of their application codes.