@inproceedings{upadhyay-etal-2023-improving,
    title = "Improving Mathematics Tutoring With A Code Scratchpad",
    author = "Upadhyay, Shriyash  and
      Ginsberg, Etan  and
      Callison-Burch, Chris",
    editor = {Kochmar, Ekaterina  and
      Burstein, Jill  and
      Horbach, Andrea  and
      Laarmann-Quante, Ronja  and
      Madnani, Nitin  and
      Tack, Ana{\"\i}s  and
      Yaneva, Victoria  and
      Yuan, Zheng  and
      Zesch, Torsten},
    booktitle = "Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.bea-1.2",
    doi = "10.18653/v1/2023.bea-1.2",
    pages = "20--28",
    abstract = "Large language models can solve reasoning tasks (like math problems) more effectively when they are allowed to generate rationales. However, a good tutoring system should not just generate solutions, but should also generate explanations and should be able to correct and guide students. We show that providing a code scratchpad improves performance on each tutoring step with a gradeschool mathematics dataset. On these tutoring tasks, GPT-3 models provided with a code scratchpad significantly outperform those given only a language scratchpad (77.7{\%} vs 48.7{\%} cumulative accuracy).",
}

Large language models can solve reasoning tasks (like math problems) more effectively when they are allowed to generate rationales. However, a good tutoring system should not just generate solutions, but should also generate explanations and should be able to correct and guide students. We show that providing a code scratchpad improves performance on each tutoring step with a gradeschool mathematics dataset. On these tutoring tasks, GPT-3 models provided with a code scratchpad significantly outperform those given only a language scratchpad (77.7{\%} vs 48.7{\%} cumulative accuracy).
Upadhyay, Shriyash Ginsberg, Etan Callison-Burch, Chris
Improving Mathematics Tutoring With A Code Scratchpad