We use supervised machine learning algorithms (i.e., Decision Trees, Random Forest, and K-nearest Neighbors) to predict performance characteristics such as runtime and IO traffic of batch jobs on high-end clusters, using only user job scripts as input. We show that decision trees outperform other algorithms and accurately predict the runtime of 73% of jobs within a error tolerance of 10 minutes, which is a 51% improvement over the user requested runtime.
['Ryan McKenna', 'Stephen Herbein', 'Adam Moody', 'Todd Gamblin', 'Michela Taufer']
Machine Learning Predictions of Runtime and IO Traffic on High-End Clusters