 Hi this is Lance from Langchain. This is the 13th part of our Rag from Scratch series focused on a technique called Raptor. So Raptor sits within kind of an array of different indexing techniques that can be applied on vector stores. We just talked about multi representation indexing. I provided a link to a video that's very good talking about the different means of chunking, so I encourage you to look at that. And we're going to talk today about a technique called Raptor, which you can kind of think of it as a technique for hierarchical indexing. So the high-level intuition is this. Some questions require very detailed information from a corpus to answer, like pertain to a single document or single chunk. So like we can call those low-level questions. Some questions require consolidation across kind of broad swaths of a document. So across like many documents or many chunks within a document. And you can call those like higher level questions. And so there's kind of this challenge in retrieval and that typically we do like k nearest neighbors retrieval like we've been talking about you're fishing out some number of chunks but what if you have a question that requires information across like five six you know or a number of different chunks which may exceed you know the k parameter in your retrieval so again we, when you typically do retrieval, you might set a k parameter of 3, which means you're retrieving 3 chunks from your vector store. And maybe you have a very high-level question that could benefit from infiltration across more than 3. So this technique called Raptor is basically a way to build a hierarchical index of document summaries. And the intuition is this. You start with a set of documents as your leafs here on the left, you cluster them, and then you summarize each cluster. So each cluster of similar documents will consolidate information from across your context, which is, you know, your context could be a bunch of different splits, or it could even be across a bunch of different documents, you're basically capturing similar ones and you're consolidating the information across them in a summary. And here's the interesting thing, you do that recursively until either you hit like a limit or you end up with one single cluster that's a kind of very high-level summary of all of your documents. And what the paper shows is that if you basically just collapse all these and index them together as a big pool, you end up with a really nice array of chunks that span the abstraction hierarchy. Like you have a bunch of chunks from individual documents that are just like more detailed chunks pertaining to that, you know, single document, but you also have chunks from the summaries, or I would say like, you know, maybe not chunks, but in this case, the summary is like a distillation. So, you know, raw chunks on the left that represent your leaves are kind of like the raw form of information, either raw chunks or raw documents. And then you have these higher level summaries, which are all indexed together. So if you have higher level questions, they should basically be more similar in semantic search for example to these higher level summary chunks if you have lower level questions then they'll retrieve these more lower level chunks and so you have better semantic coverage across like the abstraction hierarchy of question types that's the intuition they do a bunch of nice studies to show that this works pretty well um I actually did a deep dive video just on this, which I link below. I did want to cover it briefly, just at a very high level. So let's actually just do kind of a code walkthrough. And I've added it to this React from Scratch course notebook, but I link over to my deep dive video, as well as the paper and the full code notebook, which is already checked in and is discussed at more length in the deep dive. The technique is a little bit detailed, so I only want to give you a very high-level kind of overview here, and you can look at the deep dive video if you want to go in more depth. Again, we talked through this abstraction hierarchy. I apply this to a large set of langchain documents um so this is me loading basically all of our langchain expression language docs this is on the order of 30 documents you can see i do a histogram here of the token counts per document some are pretty big most are fairly small less than less than 4,000 tokens. And what I did is I indexed all of them individually. So all those raw documents you can kind of imagine are here on the left. And then I do embedding, I do clustering, summarization, and I do that recursively until I end up with, in this case, I believe I only set like three levels of recursion, and then I save them all in my vector store. So that's like the high level idea. I'm applying this Raptor technique to a whole bunch of line chain documents that have fairly large number of tokens. So I do that. And yeah, I use, actually I use both Cloud as well as OpenAI here. This talks through the clustering method that they use, which is pretty interesting. You can kind of dig into that on your own if you're really interested. This is a lot of their code, which I cite accordingly. This is basically implementing the clustering method that they use. And this is just simply the document embedding stage. This is like basically embedding and clustering. That's really it. Some text formatting, summarizing of the clusters right here. And then this is just running that whole process recursively. That's really it. This is tree building. So basically I have the raw docs. Let's just go back and look at doc texts. So this should be all my raw documents. So that's right. You can see it here. Doc text is basically just the text and all those documents that i pulled um and so i run this process on them right here uh so this is that recursive embedding cluster basically runs and produces that tree here's the results um this is me just going through the results and basically adding the result text to this list of texts. Okay, so here's what I do. This leaf text is all the raw documents and I'm appending to that all the summaries. That's all that's going on. And then I'm indexing them all together. That's the key point, rag chain, and there you have it. That's really all you do. So anyway, I encourage you to look at this in depth. It's a pretty interesting technique. It works well with long contexts. So for example, one of the arguments I made is that it's kind of a nice approach to consolidate information across like a span of large documents. Like in this particular case, my individual documents were language and expression language docs, each being somewhere in the order of, in this case, most of them are less than 4,000 tokens, some pretty big, but I index them all, I cluster them without any splits, embed them, cluster them, build this tree, and go from there. And it all works because we now have LLMs that can go out to 200,000, up to a million tokens in context. So you can actually just do this process for big swaths of documents in place without any splitting. It's a pretty nice approach. So I encourage you to think about it, look at it, watch the deep dive video if you really wanna go deeper on this. Thanks.
RAG From Scratch: Part 13 (RAPTOR)
Our RAG From Scratch video series walks through impt RAG concepts in short / focused videos w/ code.      
 
Problem: 
RAG systems need to handle "lower-level" questions that reference specific facts found in a single document or "higher-level" questions that distill ideas that span many documents. Handling both types of questions can be a challenge with typical kNN retrieval where only a finite number of doc chunks are retrieved.

Idea: 
RAPTOR (@parthsarthi03 et al) is a paper that addresses this by creating document summaries that capture higher-level concepts. It embeds and clusters documents, and then summarizes each cluster. It does this recursively, producing a tree of summaries with increasingly high-level concepts. The summaries and starting docs are indexed together, giving coverage across user questions. 

Code:
https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_12_to_14.ipynb
https://github.com/langchain-ai/langchain/blob/master/cookbook/RAPTOR.ipynb

References:
1/ Paper: https://arxiv.org/pdf/2401.18059.pdf
2/ Longer deep dive: https://www.youtube.com/watch?v=jbGchdTL7d0