 Hi, this is Lance from Langchain. This is our third video focused on query translation in the Rag from Scratch series. And we're going to be talking about decomposition. So query translation in general is a set of approaches that sits kind of towards the front of this overall rag pipeline. And the objective is to modify, rewrite, or otherwise decompose an input question from a user in order to improve retrieval. So we kind of talked through some of these approaches previously, in particular various ways to do query writing, like ragfusion and multiquery. There's a separate set of techniques that have become pretty popular and are really interesting for certain problems, which you might call like breaking down or decomposing an input question into a set of sub questions. So some of the papers here that are pretty cool are, for example, this work from Google. And the objective really is first to take an input question and decompose it into a set of sub-problems. So this particular example from the paper was the problem of last letter concatenation. And so it took the input question of three words, think machine learning, and broke it down into three sub-problems, think, think machine, think machine learning, as the the third subproblem. And then you can see in this bottom panel, it solves each one individually. So it shows, for example, in green, solving the problem, think machine, where you concatenate the last letter of K with the last letter of machine or the last letter of think K, lesser machine E, concatenate those to K E. And then for the overall problem, taking that solution and then basically building on it to get the overall solution of K E G. So that's kind of one concept of decomposing into subproblems, solving them sequentially. Now a related work called IRCOT or interleave retrieval combines retrieval with chain of thought reasoning. And so you can kind of put these together into one approach, which you can think of as kind of dynamically retrieval to solve a set of sub-problems, kind of that retrieval kind of interleaving with chain of thought as noted in the second paper, and a set of decomposed questions based on your initial question from the first work from google so really the idea here is we're taking one sub question we're answering it we're taking that answer and using it to help answer the second sub question and so forth so let's actually just walk through this in code to show how this might work so this is the notebook we've been working with from some of the other videos. You can see we already have a retriever defined up here at the top. And what we're going to do is we're first going to find a prompt that's basically going to say, given an input question, let's break it down to set up subproblems or subquestion which can be solved individually. So we can do that and this blog post is focused on agents. So let's ask a question about what are the main components of an LLM powered autonomous agent system. So let's run this and see what the decomposed questions are. So you can see the decomposed questions are, what is LLM technology and how does it work? What are its components? And then how the components interact. So it's kind of a sane way to kind of break down this problem into a few sub-problems which you might attack individually. Now here's where we define a prompt that very simply is gonna take our question, we'll take any prior questions we've answered, and we'll take our retrieval and basically just combine them. And we can define this very simple chain. Actually, let's go back and make sure our retriever is defined up at the top. So now we're building our retriever. Good, we have that now. So we can go back down here and let's run this. So now we are running. And what's happening is we're trying to solve each of these questions individually using retrieval and using any prior question answers. So okay, very good. Looks like that's been done. And we can see here's our answer. Now let's go over to Langtooth and actually see what happened under the hood. So here's what's kind of interesting and helpful to see. For the first question, so here's our first one. It looks like it just does retrieval, which is what we expect. And then it uses that to answer this initial question. Now for the second question should be a little bit more interesting because if you look at our prompt here's our question. Now here is our background available question answer pair so this was the answer question answer pair from the first question which we add to our prompt and then here's the retrieval for this particular question. So we're kind of building up the solution because we're pending the question answer pair from question one. And then likewise with question three, it should combine all of that. So we can look at here, here's our question, here's question one, here's question two, great. Now here's additional retrieval related to this particular question and we get our final answer. So that's like a really nice way you can kind of build up solutions using this kind of interleaved retrieval and concatenating prior question answer pairs. I do want to mention very briefly that we can also take a different approach where we can just answer these all individually and then just concatenate all those answers to produce a final answer. And I'll show that really quickly here. It's like a little bit less interesting maybe because you're not using answers from each question to inform the next one. You're just answering them all in parallel. This might be better for cases where it's not really like a sub-question decomposition, but maybe it's like a set of several independent questions whose answers don't depend on each other. That might be relevant for some problems. And we can go ahead and run. Okay, so this ran as well. We can look at our trace. And in this case, yeah, we can see that this actually just kind of concatenates all of our QA pairs to produce the final answer. So this gives you a sense for how you can use query decomposition and employ ideas from two different papers that are pretty cool. Thanks.
RAG from scratch: Part 7 (Query Translation -- Decomposition)
Query decomposition is a strategy to improve question-answering by breaking down a question into sub-questions. These can either be (1) solved sequentially or (2) independently answered followed by consolidation into a final answer. 

Slides:
https://docs.google.com/presentation/d/1O97KYrsmYEmhpQ6nkvOVAqQYMJvIaZulGFGmz4cuuVE/edit?usp=sharing

Code:
https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb

Reference:
https://arxiv.org/abs/2205.10625
https://arxiv.org/pdf/2212.10509