@inproceedings{magister-etal-2023-teaching,
    title = "Teaching Small Language Models to Reason",
    author = "Magister, Lucie Charlotte  and
      Mallinson, Jonathan  and
      Adamek, Jakub  and
      Malmi, Eric  and
      Severyn, Aliaksei",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-short.151",
    doi = "10.18653/v1/2023.acl-short.151",
    pages = "1773--1781",
    abstract = "Chain of thought prompting successfully improves the reasoning capabilities of large language models, achieving state of the art results on a range of datasets. However, these reasoning capabilities only appear to emerge in models with at least tens of billions of parameters. In this paper, we explore the transfer of such reasoning capabilities to smaller models via knowledge distillation, also investigating model and dataset size trade-off. Specifically, we finetune a student model on the chain of thought outputs generated by a larger teacher model. Our experiments show that the proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets. For example, the accuracy of T5 XXL on GSM8K improves from 8.11{\%} to 21.99{\%} and 18.42{\%} when finetuned on PaLM 540B and GPT-3 175B generated chains of thought, respectively.",
}

Chain of thought prompting successfully improves the reasoning capabilities of large language models, achieving state of the art results on a range of datasets. However, these reasoning capabilities only appear to emerge in models with at least tens of billions of parameters. In this paper, we explore the transfer of such reasoning capabilities to smaller models via knowledge distillation, also investigating model and dataset size trade-off. Specifically, we finetune a student model on the chain of thought outputs generated by a larger teacher model. Our experiments show that the proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets. For example, the accuracy of T5 XXL on GSM8K improves from 8.11{\%} to 21.99{\%} and 18.42{\%} when finetuned on PaLM 540B and GPT-3 175B generated chains of thought, respectively.
Magister, Lucie Charlotte Mallinson, Jonathan Adamek, Jakub Malmi, Eric Severyn, Aliaksei
Teaching Small Language Models to Reason