@inproceedings{lee-kim-2023-recursion,
    title = "Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models",
    author = "Lee, Soochan  and
      Kim, Gunhee",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.40",
    doi = "10.18653/v1/2023.findings-acl.40",
    pages = "623--658",
    abstract = "Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models{'} (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs{'} inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.",
}

Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models{'} (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs{'} inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.
Lee, Soochan Kim, Gunhee
Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models