Justifest: A Blended Learning Tool for Statistical Analysis Plans

Statistics classes in higher education often focus on specific analyses, assuming a research question was previously formulated and an analysis plan for the available data was agreed on. These early stages of the investigative process are, however, critical steps where important choices are made and where statistical thinking must lead to a justified approach. We built a blended learning tool—Justifest —that stimulates students to practice planning skills, facing the consequences of various choices. Starting from a simple, small, but relevant case study, they come up with a plan to investigate whether university grades have changed during the coronavirus pandemic. We help to improve their initial analysis plan by asking critical questions at the conceptual level and revealing frequentist properties through plasmode simulation.