 how they work and what it means to be agentic. Yeah, that's absolutely true, Craig. Absolutely true. And so like, is there one standard way to build an agent right now? No, not really at all, no. Has there ever been? No, not really. Has the way that we build agents evolved significantly in the field since even just last year? I would say even the way we think about agents has evolved. So yes, definitely the way we build them. That sounds like we've got a lot. Are you ready to tackle this big subject? As ready as you can be, yes. Yes, for sure, man. All right. I'm going to get right into it. We'll have you back in a bit for a quick discussion before we start our demos of the day. Welcome everybody. I'm Greg. That is Chris, AKA The Wiz, and we are co-founders of AI Makerspace. Thanks for taking the time to join us again this week. Today we talk agentic RAG. What you'll learn from today's event is a little bit of the context of where we were last year with agents and RAG and where we are this year. Hopefully we're going to clarify a lot of foundational concepts for you before we do quite a large build today that's going to cover a bit of the classic rag and agentic thinking and a bit of the new school rag and agentic thinking. So let's go ahead and get started. If you have questions along the way, do please ask them in the Slido or go ahead and slap them in the YouTube live chat during the Q&A. So today we cover agentic rag. We need to break down exactly what this means, what it doesn't mean. And along the way, we will have a couple of key things that we are aiming at. Number one, we want to just cover agent terminology and architecture considerations as we get into this. We're going to cover core rag and agent constructs within the lang chain ecosystem. And we're going to cover how to build an agent that contains both chains and graphs today. So this is going to be kind of a big one. We're going to talk agentic, rag, and function calling first. We're going to talk chains, graphs, and the infrastructure that it takes to build agents. Next, we're going to do a couple of demos today. One little classic chain rag, and we're going to then build a reasoning engine around a chained classic rag setup, plus some additional tools that we can leverage through function calling. All right, so we need to break down all these terms, and we need to make sure that we're aligned on what exactly all of this is really talking about at the end of the day. First of all, one of the things that people will ask us as we teach this subject is like, when we use these words, agent, agent-like, agentic, are these the same thing? Yes. They're the same thing. Okay. Don't worry. There's no real super necessary to argue subtle nuances here. The other thing we want to understand about agents is where they fit in exactly within our applications, even within a simple RAG application? And the answer is, well, wherever you want them to. Because agents you see are a pattern. They're not a specific thing. You can apply the agentic or agent-like pattern in lots of ways. What is this pattern? Well, fundamentally, when we talk about our field in LLMs, it's the reasoning action pattern. That's all it is, okay? And the reasoning action pattern is the old two-step loop. In practice, we call an LLM to do some reasoning. It helps us determine which action to take next or which response to give to the user. So the flow is like user asks question, LLM decides. Do I know the answer off the bat? Maybe I'll go and pick up a tool. Maybe it'll be a custom rag build. Maybe it'll just be an API call to something like Wikipedia or to Google search. I will then consider what I'm getting out of that tool and I'll decide, okay, do I have the final response yet? If not, maybe I'll go through and pick up another tool. Maybe I'll break down the question in a slightly different way. There's lots of things that we can potentially do to keep looping through this to find the best possible answer. And this sort of reasoning step and this action step is kind of the fundamental two components that we want to focus in on as we think about the pattern behavior. And this high level view is unchanged for non technologists and non engineers when they think about what could be built. Now, what happens behind this on the back end is the part that we'll get into today that has been evolving in a very exciting and interesting way. All right, let's continue to contextualize here today because we talked agents, let's talk about RAG. We're talking agentic RAG today. RAG very simply is all about avoiding hallucinations through the retrieval of reference material we find references we augment the prompt with references we improve our answer with those references rag is simply doing two things. It's doing dense vector retrieval and it's leveraging the idea of in-context learning. When we ask a question to a RAG system, that question flows through an embedding model after it's tokenized and chunked up. Those embeddings are then compared to any data that we have, that we've also tokenized, chunked, and embedded and saved in our vector database. We're doing a comparison here between the string in our question and any data in our database. We also set up a prompt template. It generally says something like, use the provided context to answer the user's query. You may not answer the user's query unless there is specific context in the following text. If you do not know the answer, importantly, or cannot answer, please please respond with I don't know. Into this prompt goes the similar reference material to the question although now it's back in natural language form. This process is called dense vector retrieval and the idea of putting all of this additional context into the prompt this is where the augmentation comes in this is where in context learning comes in so this is rag and all this happens before we put our prompt into the llm and get our output into the LLM and get our output. So this is the process of retrieval augmented generation. Now, the thing about RAG is the same thing as agents. RAG is a pattern. It's not a specific thing. Okay. So if we consider agentic rag, we can say then that agentic rag too is simply a pattern. It's not a specific thing. Consider using an agent within a rag system. If we say agentic rag, most of us think of taking the rag system, compressing it down into a single system diagram box, and then perhaps our agent sits atop that rag system. That's probably how most of us think about a gentic rag out of the box, but it doesn't have to be because we can do a gentic retrieval and the retrieval process is where we use an agent. We can do a gentic augmentation. And of course, we could also do agentic generation. I want to invite Wiz back up to the stage here for quick discussion. When we think about this idea of agents sitting atop RAG or within RAG, can you provide us with a few examples of the ones that we might use or consider using sitting within the rag system, within retrieval, within augmentation, within generation? How should we be thinking about this? Yeah, I mean, it basically comes down to behavior or where do you want that complex reasoning behavior? You know, so I think for retrieval, it makes them an intelligently retrieved context. It seems like a powerful thing to be able to have for augmentation, right? The way that we're augmenting and how we kind of post process that those retrieved documents and how we kind of, you know, interact with them or, you know, rank them or whatever you want to call the process, right? We can leverage, you know, this actual agentic behavior and then generation as well, right? So when we're generating, not just generating one response, but doing some reasoning with our augmented prompts. And then, and finally providing a more comprehensive response, right? They're closely linked and you have to kind of quibble about it to really to get down to it, but you can fit agents into any of these steps and it'll, it'll be, you know, better assumed than if you didn't. Okay, okay. So if we think about agents as sort of this reasoning action framework, right? We're sort of then either leveraging the reasoning and action at retrieval or augmentation or generation or above. Is that correct to say? Yeah. I think that's a great way to explain it. Okay. Okay. So if there's any sort of reasoning that you have to do at any of these steps, you could potentially leverage an agent. Is that fair to say? Absolutely true. Yes. Interesting. All right. Thanks for your insights, man. We'll have you back in just a little bit for the first demo so you can see agents and agentic behavior are not the most straightforward thing in the world let's add an additional piece to our story here because when we build with agents we inevitably end up crossing the path of function calling now function calling is all about calling. Now, function calling is all about connecting large language models to external tools. For our agentic RAG build today, what we're going to do is we're going to describe our tools and we're going to have the LLM choose which to pick up to answer the query. Specifically for us, we're going to use our tools as backup to a primary RAG system. So we'll decide if we need to go to backup and that's what our agent, our reasoning engine that's going to leverage function calling is going to do. engine that's going to leverage function calling is going to do. Okay. What are we going to build today exactly? Well, today's agentic rag build is going to have three tools in it. We're going to create a classic chained rag application with sort of private data. You want to consider this. And then we're going to have this backup search capability through function calling for both archive papers and also directly on DuckDuckGo. And even within just having access to these three different things, there's a lot of configurations of how you might build an agentic tool like this. We have to outline exactly the way we've built it, but let's keep it general for now. Our agentic reasoning action loop, it's going to be helpful to think about like this. We ask a question. That question is then going to eventually have access to each of these three tools, the private data, the archive, and the DuckDuckGo capabilities. What we'll do is we'll set up a prompt template and it'll say something like, "'Have a conversation with a human "'answering the following questions as best you can. "'You have access to the following tools. "'Tool one, RAG private data has access to my private data. "'Tool two, archive is good "'at finding relevant research papers. Tool three, DuckDuckGo, is used for general internet search. Now, if we had specific private data, we would probably want to provide some specific details here, but this should give us an idea of how this is working in general. Now, what we do is, as we say begin, what happens is we set up an agent scratch pad this is sort of keeping track of the state of affairs in our agentic reasoning loop and we of course need to connect this to a large language model. Generally, we're using an OpenAI LLM for this. And we want to make sure that this loop can be gone through. As many times as necessary, this reasoning action loop can have access to these tools. We can continue to think through things and write them down on this sort of agent scratch pad. And we can decide eventually to output a final response so this is kind of the big picture of what we want to kind of connect together today interestingly when it comes to building these systems, there are nuances that are important to keep track of. And this gets us into our portion on chains, graphs, and infrastructure. As I mentioned earlier, if I'm a user, nothing has changed in terms of how I engage with this system. I ask questions, I get a response. It's a chatbot. If I'm an LLM, if I'm the OpenAI LLM, nothing has changed. I get a prompt, I deliver responses, I have access to tools, same, same. However, if I'm a builder like you are if i'm a product manager if i'm an engineer the infrastructure that we have now to build agents is actually more useful than it ever has been what do we mean by useful well it's more intuitive it's more modular and it's more intuitive, it's more modular, and it's generally more capable. And Langchain allows us to build sort of from the ground up with production in mind, which is very cool. What do we mean by intuitive and modular? Well, intuitively, what we can do now Well, intuitively, what we can do now is if we can draw a diagram, let's say your product manager or director of product wants to draw a diagram. You can actually directly build that diagram using Langchain's ecosystem, specifically using Langgraph. This is super cool. Now as an engineer, you might think, well, I can always figure out how to build what's in the diagram. But this is sort of a beautiful kind of one-to-one of taking the diagram as outlined and sketched and putting it directly into code. This is a very powerful pattern, this graph pattern. It's also more modular. And it's more modular because of this graph approach. Now, recall exactly what we're talking about when we say graph. A graph is simply a collection of connected objects. You might think of this as a directed acyclic graph, a DAG. But what's happening here is we actually are able to do cyclical things in the graphs that we can build with Lang graph. So it's a little bit more than a DAG. It's a cyclic DAG. We still have nodes. We still have edges. And what we can do in Langchain is we can actually create cyclical graphs. This is useful because this allows us to add cycles to applications built on Langchain. Of course, Langchain started with the abstraction of chains. Langgraph is what allows us to do this. And specifically what Langgraph is doing is it's creating state machines that sort of track the state at each node by specifying them as graphs. Now you might think, ooh, state machines. I'm not exactly sure what you mean by that. Harrison Chase, the CEO of Langchain, gave a great TED Talk describing the cognitive architectures. State machines sit somewhere in between routers and agents. In fact, we did a deep dive on state machines in our recent event on Landgraf. So we're going to kind of avoid going into super depth there at the moment. The important thing to note for today's build is when we talk about nodes and edges in Lang graph, we want to think of the nodes as functions like external tools or as Lang chain expression language runnables. For instance, a classic rag chain. The edges are instead paths to take, right? Or where to pass our state object next. Remember the agent scratch pad? Well, what that is really tracking is it's tracking the state. It's tracking the state. Okay. So let's dive a little bit deeper into the tactics of Langchain. Langchain just came out with v0.1. This was a big release. It included Langraph. Remember, Langchain is all about two things, enabling LLM applications to leverage context and reasoning. Context, we can think of RAG, context augmentation. Reasoning, we can think of agents. These are both patterns, right? When we think of our classic RAG chain, the tools that we're using in Langchain within Langchain Core, Langchain Community, are, of course, L LCL that underlies everything. But we're going to use the model and the prompt in our build today. We could use the output parser or the example selector. But we're just going to stick with the model IO and the prompt templating. We're also going to leverage these retrieval tools, the retriever, the loader, the vector store, text splitting, and so on. And we'll use chains to chain them together. Declarative composable chains are very easy in LCEL, and this is very straightforward to build. This is sort of classic RAG at this point. Now, when we talk about using Lang graph to do retrieval on external tools like archive and DuckDuckGo, this is still a rag pattern. But now what we're doing instead is we're leveraging agent tooling and agents. Specifically in our build today, we are going to do this aspect of our application that uses agents and tools using OpenAI function calling. Okay? So we've got classic rag chain, the rag pattern that is leveraging the function calling capability to get access to backup archive or DuckDuckGo search. So if we dig into our classic RAG build, this is pretty straightforward for anybody that's been building with Langchain. This is our little box called private data. And what we're going to do is we're going to leverage models, prompts, and retrieval stuff. The models input output is classic, right? If we use an OpenAI model, we need to sort of leverage the system user assistant message syntax. In LanqChain, it's the system human AI message syntax using chat style model. No problem. The prompt template, well, we've already kind of talked about the prompt templating that we need for a classic rag system in our diagram earlier. And then we need to create a simple vector store. This vector store is going to just load some documents in, split the text up, do some embedding, and store the vector. There's all the little tools we need right there. So this rag setup looks something like this. If we use, say, a face vector database, we're going to chain everything together, and we're going to load some documents in, split the text up, and so on. What we're going to do today is we're going to imagine a situation where we built a vector store, let's say in 2023 or a little while ago, and it's not quite updated yet. So we want to give it access to backup live up-to-date search capabilities with both archive and DuckDuckGo. What we're going to do is we're going to do something that if you've been following our channel or our content for a while, you've probably seen us do before. We like to call this meta-rag, and we've been doing this for the past year or so, where we'll search archive for the top papers on rag, say top three, top five papers. We'll turn them to embeddings, we'll put them in a vector store, and then we'll be able to query our RAG application about RAG. Then we return answers to the questions with sources. The way we can set this up is we can use Langchain's archive loader to get papers from archive based on our question and build out this vector store imagine that we were going to set this up and then in the future we wanted to augment it And then in the future, we wanted to augment it. But this first step is just setting it up. We're going to take off-the-shelf models, Text3 Embedding Small, the latest embedding model from OpenAI, the chat model, GPT 3.5 Turbo, and we're going to build out this simple meta-rag system classically. This is what Chris is going to walk us through now. And then after we get our meta rag system set up, we'll be able to rock and roll right into our agentic rag rest of the day. Wiz, over to you. Yes. Thank you, Greg. Okay. So we're going to talk very briefly about how we set up this L-C-E-L chain, and then we'll be moving on from there. So the idea here is that we're going to be using a number of different components, the first of which is going to be this L-C-E-L chain, which we're kind of uh you know used to uh to using hopefully at this point and then also we're gonna touch on that laying graph uh piece so the dependencies are pretty straightforward right we need lang chain lane chain open ai leg graph and then we're going to be using some tools and archive induct.go search we're also going to set up for our local retrieval pipeline. We are going to set up these two. This is face CPU which is what we are going to use to power our vector store. And then PyMOOPDF is something we are going to use to parse PDFs that we take from archive. And that's it. For environment variables, it's pretty straightforward. We're going to use OpenAI today. So we want OpenAI. We're also going to use LangSmith. We're not going to touch on it a lot, but we do have, you know, this integration that comes for free. You know, and the idea is it's, if you're using LangChain at this point, you should just also be using langsmith because this is all you have to do and we're going to be taking advantage of langsmith just like that so rebuild a simple chain this is kind of your your classic normal you know stock lcel chain we're going to first create a document loader which is going to load documents from archive with the query retrieval augmented generation gotta do meta rag it's always a We're going to first create a document loader, which is going to load documents from archive with the query retrieval augmented generation. Got to do meta rag. It's always a great time. Then we're going to split those documents. So we're going to chunk them into smaller pieces so that they can be more compatible with our retrieval system. And so that we can, you know, not just put the whole paper in and ask questions but we we put in only the relevant pieces uh so that we can put that in uh we also have of course our chunked documents our chunked documents are pretty straightforward they are just those split documents we're going to use those chunk documents as well as OpenAI embeddings, which is the TextEmbedding3 small model, which is a very performant and more importantly, inexpensive model. And we're going to use that to actually create our vector representations or embeddings for each of those chunks. And then we're going to set that resultant vector store up as a retriever. This is just wrapping it in that retriever interface. And that's it. The idea here is, you know, we need something that we can send a natural language query to and then have relevant documents be spit out. And yes, so that's it. The next step we're going to do is augment, right? So we have retrieval covered. Now we're going to cover augmented. And augmented is just this prompt, right? So we're going to take this prompt and we're going to take our question and then we're going to augment with some context. We're going to ask our LLM, hey, you know, using this context, answer the query. If you cannot answer the question, you know, please respond with I don't know. So the idea here is we're going to augment our prompt with our context. And so that's where augmented comes in. And so all's where augmented comes in. And so all we got left of course is generation. Generation is the easy part. We just point this thing at a model. Once we've pointed it at a model, we're ready to create our LCL chain, right? So I've created the LCL chain. I always kind of provide this, uh, uh, this absurd, uh, commenting because the LCL chains are a little bit unintuitive. If this is your first time seeing that, uh, if it's, if it's like your eighth time seeing it, I, you know, hopefully you're, you understand what's happening, but we'll just very quickly, quickly walk through it. Uh, the idea is straightforward, right? We are going to start our chain with getting our question and giving it to our retriever. And that's going to be called our context, right? So when we pass a quick query to our retriever, we should get relevant documents. So that's our context. And our question is just still our question. And then we're going to pass our context through to the next step, right? As context. And we're going to retrieve it here just so we have it, right? So we have something to reference. And then we're going to pass our question into our rag prompt with our context since our rag prompt expects both question and context. And then we pass that to our model and that's our response, right? And there you go, clear as mud, right? So that's the piece. And then we're gonna test it and it works how we expect. And we're just flexing LCL a little bit here. So we do the await and the a invoke. So this is the asynchronous version of the chain. And the reason we're doing that is just to showcase that we didn't have to do anything special. We get async for free. Async is very important when we're having hundreds or thousands or millions of people trying to interact with our application. And that's it. I mean, that's how we set up the classic chain. The only thing left to do there is to get this thing to eventually interact with land graph uh but before we do that we're gonna hear from greg a little bit more um and uh he'll he'll take us to that next step yeah thanks wiz nice little lcel flex there i like that so from classic meta rag over to our rag and function calling build note the rag with the spider emoji here is the sort of graph rag. So what we're going to do now is we're going to take the two tools that we want to have access to, archive and DuckDuckGo, and we're going to put them into this framework of function calling. Now, we can visualize that, and it looks something like this. This allows us to leverage OpenAI function calling directly just for these two tools. Now what we're actually doing today is we're taking this setup and we're also adding this additional component of our private data rag, which in our case is the MetaRag system. that but the idea here is that as your systems evolve that perhaps your older documentation hasn't been updated in a while your private data and you might want to make sure that you have access to live most up-to-date data so you're not always having to constantly recreate your vector stores and your and constantly going back and searching for new stuff in turning it into embeddings and putting it into the vector store. So this is kind of the idea here. And as we go from this OpenAI function calling setup to our more robust reasoning agent, you might say that what we have done here is we've now started to build out this agentic rag with Lang graph and Lang chain expression language. So how does this all look at the end of the day in graph format? Because we've kind of been looking at this conceptually, but what does it look like if we really wanted to take that system diagram and turn it into something that we can code up? Well, as a cyclic graph, as we mentioned, there's going to be many ways we could potentially put this together. But what we want to do is we want to put this together so that as we start, we're going to, by default, go ahead and we're going to leverage our RAG chain. By default, we will be going to our more classic rag chain setup, and we will potentially be leveraging the DuckDuckGo and archive tooling after we check that initial private database. As we go to the agent component here what's happening is we're asking ourselves do i have enough to answer the query or do i need to go search additional papers on archive or do i need to go ahead and query DuckDuckGo and general internet search to find the answer of what I'm looking for? And within that loop, we will stay until we get something that is good enough that we decide we don't need to continue. And we can end and close the loop. we decide we don't need to continue, and we can end and close the loop. So we're gonna go back to Wiz now to talk about how to build this graph, rag, and function calling piece, and then to do the final assembly of the agentic rag with lang graph and LCEL, where the initial rag chain is a single node in our graph. Wiz, back over to you. Thank you, Greg. Yes. Okay. So this is the, this is kind of where the magic happens, right? So we're going to have Lang graph. Lang graph is, I mean, Greg's talked about it. The notebook goes into a, you know, pretty specific detail about it, but the idea is straightforward, right? I mean, so we have chains and chains are basically like a baby version of a graph, right? They only allow you to flow one direction. You know, you flow from here to the next thing, to the next thing, to the next thing. Langraph, you know you flow from here to the next thing to the next thing to the next thing land graph you know gives us the flexibility to define those nodes or those units that act on our state and edges or the units that tell state where to go next right lets us have very very uh specific control of of those uh components So the idea in essence, at the end of the day, LandGraph is just shepherding state around and doing stuff to it. And so let's look at how that works in a very simple example. We're first going to create a tool belt. This is classic, right? We're going to use DuckDuckGo search and archive query. These are just tools that are offered through a LandChain community, we're going to make sure that they are able to be run. And we're gonna use a tool executor to do this. Now the tool executor is just basically a way for us to actually run the tools. So open AI or whatever, you know, whatever particular LLM you're using can't run the tools on their end, we have them on our end, and we need some way to actually execute Python code in order to run those tools. And so that's exactly what tool executor is going to do for us. Then we're going to select our model. Our model, pretty straightforward. We're just going to use chat open AI. We are using function calling. And so we are using open AI. And so we are using, you know, we are using OpenAI. That's it. There are other models that implement OpenAI as function calling endpoints. And you can get models open source that are going to help you do that if you host them. But we're just going to use OpenAI because it's the most accessible with the least lift. Then we're going to do this stuff. All you have to worry about here is that this is going to do a lot of heavy lifting for us. We don't have to define Pydantic models and then define functions and then build prompts and descriptions. We can just instead use this convert to OpenAI function with the compatible tools. And that's what we're going to do. How does this convert to open AI function with the compatible tools. And that's what we're going to do. How does this compare to auto gen? Seems like you might have flexibility for custom flow design. It sure does, Raymond. Absolutely true. And then we have this very kind of cheeky putting the state in the stateful, right? All state is, is it's an object, right? It's just something that's keeping track of stuff, right? And so we want to use state because it's important. We have to know like what happened before and use that to inform what happens next, right? And the idea is that we're just passing around this state object. It's analogous to like a more flexible, more portable, better version of that scratch pad or that message history, right? And in fact, basically all we're doing in this particular Landgraf example is we're using an annotated sequence of base messages, which is really just a list of messages, just chat history, right? And we're using this operator add. Operator add is going to make it so that we can only append messages to this state. We don't want to reset them or modify them in a way that it deletes things. We certainly will want to do that in a more complex application, but for this example, we don't need to do that. And temp equals zero because we don't want the model to imagine things. Loosely that's true. Aman, yes. The idea is, so there's a question in the chat, temp equals zero because we don't want the model to imagine things. And the answer is loosely yes. We want it to stick to the context as much as we can. And temp zero can help it do that. And then we're going to make this graph, right? So let's just, if you wanna go deeper into this, we did a length graph event, but the basic idea is we're gonna make these nodes. Nodes are just functions that take in state and they return some modification to state. So because we're using operator.add, when we return this state object with our response, it's going to append this message to our state object. And that's as easy as it gets, right? And so every node basically is going to take in state, do something to it, and then return a modification to state. That's the idea. And we do that to call our model where we just call our model on all of the messages that we've got so far. And we do that with our call tool where we just use it to invoke a tool, depending on if the LLM determined we needed to use a tool. We'll call it with its name and its arguments. We'll send it to that tool executor we just talked about. And then we'll append it as a function message. OpenAI will expect there to be a function message as the message after it suggests to use a tool. So it can use that message to determine what the response to our original query should be. And then we're going to start making the graph, right? This is not so bad. Basically all we have to do is create our state graph by passing our agent state. Then we're going to add a node called agent and we're going to make that node our call model function. We're going to add a node called action and we're going to make that our call tool function. And then this is what we got so far. Agent action, easy peasy. We're going to set an entry point so that when we first enter our graph, we're gonna enter into agent. Then we're gonna set this conditional node. Conditional node's gonna check, hey, did the last message say to use a function call? Right, if it did, we should go on to the function agent or the, in this case, the action agent. And if it didn't we're done right so and then we create that conditional Edge which just looks like this right so we we get a we get an input our agent says you know hey we need to use this tool so we continue to the action agent or it doesn't say we need to use a tool and then we just say all right we're done um there you go next you know this is almost done but right now if we get to action, we just kind of peter out, right? We stop. And so we need to get back to agent, which we can do by adding that edge action to agent. And that's going to create this cycle. So now we have this possible cycle and this exit. And there you go. We're good to go, right? We can compile that and then begin using our graph easy peasy. So what if we wanted to create this, right? Which is a slightly more complex version of that chain where we first do rag and then we see if rag was enough. If rag was enough, we end. If it wasn't, we enter into this possible agent action cycle. And if it wasn't, we enter into this possible agent action cycle. Well, there's two things we need to know. Number one, we can use any LCL runnable as a node. So you can use any lang chain chain as a node. However, we've previously discussed, right, the input to our nodes is state, and it expects some modification as output, right? So we have to wrap our LCL chain in this convert state to query, which is just going to convert the state that we have, right, the state object into something that's compatible with the chain we built earlier. And then we're going to put a parser on the end, which is going to parse that response back into the expected state format, right? And that's it. Now, this is still just an LCL chain, we can use it exactly as we used it before. It's just that the inputs and the outputs are slightly different. But this is just an LCL chain runnable, right? We've just made it so that it's compatible with the input output that that our state is going to represent. And so we do that and then it works great. We can still take advantage of the asynchronous out of the box, LCEL components, let's go huge. Now what we're gonna do is we're gonna create our agent node, same as before, action node, same as before. And we're gonna add this first action node. That's going to be our newly built LCL chain, right? So we have the LCL chain that we built up above. We're going to call that first action. And we're going to set the entry point to our chain as first action. And then we're going to build this conditional edge, right? Now, this is just the representation of that edge as a Python function. But basically, we're going to grab our state, we're going to get our question and answer from our state. So our question is our first message. And then our answer is our most recent message, right? Because all we've done at this point is pass it into our chain with a question, and we received a response from our chain. We're going to use this pydantic pattern in order to extract a yes or a no to if this question is fully answered by the response. If it is fully answered by the response, then we're going to go ahead and just go to the end. If it's not fully answered, then we're going to go ahead and continue into our agent action cycle, potential cycle. And that's all this is doing, right? You'll notice that in this conditional edge, we have a LCL chain, right? So we have an LCL chain as a node, and we have an LCL chain in our conditional edge, right? So we can really just put these kind of objects wherever is most convenient for us, which is very handy. We're going to then define that conditional edge with the is fully answered function, where we go from first action to agent, if we need to, or to end if we don't. The rest of this is the same from above. And when we use it, we can see who is the main author on the retrieval augmented generation paper, right? And we don't go to any tools. We just immediately get the answer and then we exit. We never hit the agent action cycle. Versus this question, who is the main author on the retrieval augmented generation paper? What university did they attend? We do, in fact, need to get to the agent action cycle where we're going to use our archive tool to try to find the answer to the question. And that is the idea of Landgraf. And that's how we can use it with RAG. That's how we can incorporate LCL components into it. And that's how we can even use LCL chains within conditional edges, right? So we can do it all with Landgraf. And we wind up with this nice looking graph at the end of the day, which is going to express some state machine. And with that, I'll pass you back to Greg, who will take us to Q and A. All right, yeah. So heck of a build today. Thanks Wiz. That was the final assembly for agentic RAG with Landgraf and LCEL build. In conclusion, we can remind ourselves that agents, RAG, and agentic RAG, these are all patterns. They're not specific things, and we need to be flexible in the way that we apply them. The agent pattern is simply the reasoning action pattern today. And it's as easy as that to really decide how you should go about building specific applications for your use cases. Our classic meta rag that we built with langchain expression language and chains was actually just a single node within our lang graph build. And that node was one we always started out with. We set it up so we had backup search for both archive papers and DuckDuckGo, and that was the architecture and infrastructure we chose to use. use. Remember, LandGraph is highly flexible. It is simply shepherding the state around, allowing you to do lots of different stuff. There's a million ways you could potentially build any given application, and it's up to you as the builder to work with your team and decide what's best for your users when it comes to building out agentic RAG applications. Looks like we've got some questions in the chat and in Slido. And if you have any additional questions, I'd like to welcome Wiz back up to the stage as we get into it, then please go ahead and hit this QR code and drop your questions directly into Slido. So we'll start in Slido here. And Chris, let's kick it off with, can agentic RAG expand the context window of LLMs? Say, summarize a long document section by section. Some sections may not be useful as other sections. Oh, it's a good thing I didn't immediately say no. So no, it can't literally expand the context window. However, it can increase information density in the context window. Right. So we can we can do exactly as you outlined. We can actually take our information from this you kind of huge tokeny mess that has very little useful information and boil it down to only the most useful parts. Absolutely. Yes, you can do that. Okay. Okay. It looks like let's go to this question on hallucinations in Slido. Squashing hallucinations are hard enough when trying to go to market. How do I create awesome environment aware, autonomous agents using RAG that can be trusted? That's a great question. I think if I had the answer probably, I'd be on a boat somewhere in the tropics, but no, I mean, as we evolve and mature in this space, I think one thing that we're gonna see becoming more and more important is this idea of, you know, sourcing or providing references, right? And letting humans determine if things are true or not. We can definitely ground LLMs through things like RAG and other processes, right? We can use self-reflection to, and self-refine in order to try to make sure that their outputs are in some way, quote unquote, true, but it's not, we haven't come up with a way yet that solves the problem. We just, we just have ways that we can help move, move them closer and closer to not hallucinating. I mean, the truth of it is, right? How do we get humans to, you know, that that can be trusted, that never hallucinate? And that's also a very difficult problem we've been trying to solve for as long as we've been walking around, right it's uh at the end of the day i think we're moving closer and closer to that and we're certainly approaching human level performance but uh it's not yet solved and i don't expect it will be solved in the next uh in the next few months at least yeah you know we've been talking about the the full autonomy in class and i love this i love this idea of like do any humans have complete and full autonomy outside of the confines of their managers and the business? It's like, no, that's not a thing. You know, it's like, so this idea of state machines, I think it's really important for everybody to realize that it sort of allows us to stop the full autonomy and just sort of like peek in at any given state of affairs in our application. So I want to go back because everybody's talking about it. Everybody wants it. We're going to go back to context window stuff here for a minute. So we're going to pull some context window questions from the Slido. With massive context window LLMs like Gemini, why even bother with RAG? Why not just put the entire code base in the prompt? Yeah. I mean, it's a fair question. It's worth thinking about and talking about. The idea comes down to two major points, I believe. One which is it's difficult to shuffle around objects that large cheaply. Certainly, it's true that we can just always add the entire code base into the prompt every time we want to do a query. like gigabyte, petabyte sized data repositories, eventually we're gonna run out of either room or memory in whatever machines we're using. And then secondly, this idea of, well, we'll just add it all in there, but if you have a prompt that's a million tokens long, even the papers that have come out with these models that have these impressive context windows are like, yeah, but the latency at that size is insane, right? We're waiting for 30 seconds for a needle in the haystack. And the kind of meta point to bring up there is retrieval systems are very good at extracting the information we need, right? The LLM ultimately will always answer the question correctly if we give it the correct information. And giving it all of the information doesn't necessarily mean giving it the correct information when that information could be confusing or potentially uh you know contradictory so there's a lot of different uh issues someone brings up in the chat right uh they don't remember everything so we do have this loss in the middle problem even if we solve loss in the middle well we we kind of have lost in the in between the middle then because we we might have context broken up across wide ranges in the text with through regular retrieval augmented generation is not problematic becomes problematic because the model is not likely paying attention to the very beginning of this uh the the context and the end at the same time so there's a number of different issues with these long contexts that have yet to be solved now does that mean uh they're not useful heck no they're super useful uh they will make rag better right they will make our interactions with lms better uh it's just not the case that it's going to replace rag uh as much as it's going to augment rag right now Now we'll have augmented retrieval, augmented generation. ARAG, coming soon. But that's the idea, yeah. Well, I'm also reminded of the fact that when we look at these metrics for evaluating our RAG systems, we're often choosing, re-ranking algorithms, different things on trying to make sure we don't have too much duplicative information. It's not just about things that are too divergent, but if we have the same, same, same, same, same stuff all over again, all the time, you know, it's just sort of this repetition isn't necessarily good for the generations. And, but I think, you know, as Sam and Lex were talking about this week, trillion token context windows are coming soon. Next 10 years, why not just dump in all the data from everywhere? I mean, to me, it seems like we could, that would be very useful for people that have no idea how to build rag systems, for sure. But for those of us that are builders that can quickly spin something up. Yeah. I also think someone in the chat, Olivier, said something that's very Someone in the chat, Olivier, said something that's very interesting and worth repeating. We don't, for agentic flows, right, we actually don't care too much about the long context. We want strong reasoning capabilities. And that extra context isn't going to get us necessarily stronger reasoning capabilities. And in fact, might impact negatively reasoning capabilities, depending on how long the context is. And so I think that's another piece where we have to think about like, yes, that long context is huge for some tasks, right, including ones that overlap partially with rag, but for others, it's, it's just, it's just like, not necessary and or potentially harmful. So, okay, okay. So I think what I'd like to do is I'd like to sort of end on this question about the reasoning engine GPT-4 as this sort of evaluator. Can GPT-4 be replaced as this evaluator if the question was answered or not with humans? Can we put a human in the loop there? Can that type of logic be implemented with Landgraf? Can you, I don't know, ping the product manager or ping the customer support person? Literally, yes. Absolutely. So there's LandGraph examples in their in their Docs they have a example use case where you do set a human in the loop uh you can put humans to loop at any of the components including the user right you can let the user tell you if their question was uh was answered by that just initial rag pipeline and then let it go into this more complex flow uh so yes, absolutely. That can be implemented in Landgraf. And we can let the human be the evaluator in any number of fun ways that we can think of. But yes, absolutely. We can replace we can we can even replace it with a perfect or with a preference model right from from RLHF. And we have an RLHF tuned reward model that we know is fairly performative. It aligns with our users' preferences. We can use that to determine if this is a good response or not a good response and then pass it into further parts of the chain. So absolutely. This is the flexibility of Landgraf, right? It takes in state, does something to state, and then returns a modification of state. Anything can exist between those two takes in state and returns modification state. You can put cats in there. You can put whatever you want. That's the power of line graph. Okay, okay. Two rapid fires before we end today. Why didn't you have to explicitly pass question to runnable pass through? Oh, we don't have to pass question because question is already being propagated by the rest of the chain. We're adding that context in order to allow it to pass through. So question is part of the key already in the runnable. Okay. And does DuckDuckGo offer any advantage over the Google search SERP API? Why are we using that? 100% no advantage at all. It's just free. Doesn't require you to sign up for an API key and therefore is the most accessible. Boom. Thank you, Wiz. Your insights, as always, are much appreciated. And can't wait to see you back next time. Yeah. And don't forget, guys, you know, we love you coming out. Don't forget to subscribe. Ring that bell to get notified of when we go live next. We're here every Wednesday talking about cool stuff. So, uh, uh, thanks so much, Greg. We'll see you soon. Boom. Love it. And if you like this session, then you'll probably really like our discord community vibes. Shout out to Mert for dropping in the chat. Uh, definitely join our aim community. We've got community sessions and lots of stuff going on every week. If you're the type that would like to further tinker on your own with lots of different concepts and code, you can find all of the events and all the code that we've shared publicly, directly on our GitHub with the awesome AI Makerspace or AIM Index. Check it out. And if you're looking to accelerate your LLM application development from prototyping to production, and you're one of these async self-studiers, we recently open-sourced the first LLM Ops course we ever built, LLM Ops, LLMs in Production, Cohort 1, fully available with all code on GitHub. We look forward to providing more open-source resources cohort one, fully available with all code on GitHub. We look forward to providing more open source resources and courses for folks out there around the world learning. Now, if you, on the other hand, are not so great at asynchronous self-study, if you've been spinning your wheels for a long time, or you just need to get up to date as quickly as possible, you don't want to play version control yourself. You wanna specifically get the most hands-on, most up-to-date, most current curriculum, concepts and code, and you're ready to super accelerate because this is a serious course. We call it the AI Engineering Bootcamp. We've got an upcoming cohort in just a few weeks, and we'd love to see your application come through. If you're not a fit, I will personally work with you to help you get on the path to where you might be a fit in future cohorts. Anybody can get in the game today. Finally, if you have any feedback, we'd love to hear it, whether directly from our feedback form that we'll share now or on Luma. And as always, everybody, keep building, shipping and sharing, and we will do the same. We'll see you next week. Have a great one.
Agentic RAG with LangChain
​In this event, we’ll provide a brief history of Agents before spending time on the details of building what is becoming the industry-standard agentic RAG application, namely, one with access to backup internet searches. Per LangChain's recommended best practices, we’ll use OpenAI Function Calling to build an OpenAI Functions Agent.


Event page: https://lu.ma/agenticrag

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/8mC2FudkjQaogRZKusiGm8

Speakers: 
Dr. Greg, Co-Founder & CEO
https://www.linkedin.com/in/gregloughane

The Wiz, Co-Founder & CTO
https://www.linkedin.com/in/csalexiuk/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

Apply for our new AI Engineering Bootcamp on Maven today! 
https://bit.ly/aie1

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/eySGueVq29mTSbN19