 Hi, this is Lance from Langchain. This is the 10th video in our Rack from Scratch series focused on routing. So we talked through query translation, which is the process of taking a question and translating in some way. It could be decomposing it using step back prompting or otherwise. But the idea here was take our question, change it into a form that's better suited for retrieval. Now, routing is the next step, which is basically routing that potentially decomposed question to the right source. And in many cases, that could be a different database. So let's say in this toy example, we have a vector store, a relational DB, and a graph DB. But what we redo with routing is we simply route the question based upon the content of the question to the relevant data source. So there's a few different ways to do that. One is what we call logical routing. In this case, we basically give an LLM knowledge of the various data sources that we have at our disposal, and we let the LLM kind of reason about which one to apply the question to. So it's kind of like the LLM is applying some logic to determine which data source, for example, to use. Alternatively, you can use semantic routing, which is where you take a question, we embed it, and for example, we embed prompts. We then compute the similarity between our question and those prompts, and then we choose a prompt based upon the similarity. So the general idea is in our diagram, we talk about routing to, for example, a different database, but it can be very general. It can be routing to different prompt. It can be, you know, really arbitrarily taking this question and sending it different places, be it different prompts, be it different vector stores. So let's walk through the code a little bit. So you can see just like before, we've done a few pip installs, we've set up Langsmith, and let's talk through logical routing first. So in this toy example, let's say we had, for example, three different docs, like we had Python docs, we had JS docs, we had Golang docs. What we wanna do is take a question, route it to one of those three. So what we're actually doing is we're setting up a data model, which is basically going to be bound to our LLM and allow the LLM to output one of these three options as a structured object. So you really think about this as a classification. Classification plus function calling to produce a structured output, which is constrained to these three possibilities. So the way we do that is, let's just zoom in here a little bit. We can define like a structured object we want to get out from our LLM. Like in this case, we want, for example, you know, one of these three data sources to be output. We can take this and we can actually convert it into OpenAI, for example, function schema. And then we actually pass that in and bind it to our LLM. So what happens is we ask a question, our LLM invokes this function on the output to produce an output that adheres to the schema that we specify. So in this case, for example, we output like, you know, in this toy example, let's say we wanted like, you know, an output to be data source, vector store or SQL database. The output will contain a data source object and it'll be, you know, one of the options we specify as a JSON string. We also instantiate a parser from this object to parse that JSON string to an output like a pedantic object, for example. So that's just one toy example. And let's show one up here. So in this case, again, we had our three doc sources. We bind that to our LLM. So you can see we do with structured output, basically under the hood, that's taking that object definition, turning into function schema and binding that function schema to our LLM. And we call our prompt, you're an expert at routing user question based on, you know, programming language that you're referring to. So let's define our router here. Now we're going to do is we'll ask a question that is Python code. So we'll call that and now it's done. And you can see the object we get out is indeed, it's a route query object. So it's exactly, it adheres to this data model we've set up. And in this case, it's correct. So it's calling this Python doc. So we can extract that right here as a string. Now, once we have this, you can really easily set up like a route. So this could be like our full chain where we take this router, which is defined here, and then this choose route function can basically take that output and do something with it. So for example, if Python docs, this could then apply the question to like a retriever full of Python information or JS, same thing. So this is where you would hook basically that question up to different chains that are like, you know, retriever chain one for Python, retriever chain two for JS and so forth. of like the routing mechanism but this is really doing the heavy lifting of taking an input question and turning it into a structured object that restricts the output to one of a few output types that we care about in our like routing problem so that's really kind of the way this all hooks together now semantic routing is actually maybe a little bit more straightforward based on what we've seen previously so in that that case, let's say we have two prompts. We have a physics prompt, we have a math prompt. We can embed those prompts. No problem. We do that here. Now, let's say we have an input question from a user, like in this case, what is a black hole? We pass that through. We then apply this run of a lambda function, which is defined right here. What we're doing here is we're embedding the question. We're computing similarity between the question and the prompts. We're taking the most similar. And then we're basically choosing the prompt based on that similarity. And you can see, let's run that and try it out. And we're using the physics prompt. And there we go. Black holes, region region and space. So that just shows you kind of how you can use semantic routing to basically embed a question, embed for example, various prompts, pick the prompt based on semantic similarity. So that really gives you just two ways to do routing. One is logical routing with function calling. It can be used very generally. In this case, we applied it to like different coding languages, but imagine these could be swapped out for my Python, my vector store versus my graph DB versus my relational DB. And you could just very simply have some description of what each is. And then not only will the LLM do reasoning, but it'll also return an object that can be parsed very cleanly to produce like one of a few very specific types, which then you can reason over, like we did here, in your routing function. So that kind of gives you the general idea, and these are really very useful tools, and I encourage you to experiment with them. Thanks.
RAG from scratch: Part 10 (Routing)
This is the 10th video in our RAG From Scratch series, focused on different types of query routing (logical and semantic).

Notebook:
https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_10_and_11.ipynb

Slides:
https://docs.google.com/presentation/d/1kC6jFj8C_1ZXDYcFaJ8vhJvCYEwxwsVqk2VVeKKuyx4/edit?usp=sharing