 Hey, Wiz. Hey Wiz, so agents, they're pretty dope and we've explored them before. Does that mean multi-agents are even more dope? Yeah, Greg, I think it does mean that. They're multi-dope. We've reached the next level of dopeness so you're saying that we can build something dope today and then use multi-agents to even up the dopeness technically speaking all of that is true yes okay so we're gonna technically increase the dopeness of agents. Wiz, I cannot wait to see what you've got in store for us. We'll see you back in a bit, my man. Welcome everybody. I'm Dr. Gregg. That's the Wiz. We're talking multi-agent rag today. We're talking multi-agents today, multi-agent frameworks. We're going to root this discussion in the patterns of GenAI that you need to know if you're going to build LLM applications. There's a lot of complexity to untangle, and we've got a short time to spend together today. So if you've got questions along the way, please drop them in the YouTube live chat or in the Slido link that we're dropping below. We will prioritize Slido at the end. Let's go ahead and get right into it today. We're talking multi-agents. We're talking multi-agent RAG. We're going to be using Lang Chain and Lang Graft to do our build today. So there are some things we want to make sure that we get a handle on today. And as we align ourselves to what we'll get out of this session, we really want to sort of get under and understand multi-agent workflows as an LLM prototyping pattern. Of course, we want to build one and we are going to walk you through exactly how to build a multi-agent application. That's quite complex today. So to get into this, we want to sort of root ourselves in the patterns that we're all so familiar with if you're joining us for multi-agent rag today the patterns of spongebob here and then we want to extend it these are so important because they don't go anywhere just because we add a bunch of agents to our applications let's take a closer look Let's take a closer look. When we talk about the patterns, we have to start with prompting. The definition of prompting is to lead, to do something, to instruct. Done well, we might call this teaching or training even. If we take teaching and training far enough into an LLM, we provide one-shot, two-shot, few-shot examples, we run out of context window, where are we left? We're left with fine-tuning as a method to teach the LLM how to act on the other side of the task-specific spectrum. Of course, optimizing how the LLM acts is one part of the puzzle. We also want to optimize what we're putting into the context window. We want to use as much relevant reference material and knowledge that we can get our hands on. We want our applications to incorporate context well. context well. And of course, RAG is the focal point of so many of our applications, especially the ones that we're actually trying to use to create business value today. When we talk about agents, what we're typically talking about today is we're talking about giving the LLM access to tools of various kinds, but not incredibly various. There's sort of a category, a main category of tools that starts to connect some of our patterns back together again. But let's think about this fact that agents are a pattern. What pattern are they? Well, simply put, they are the reasoning action or the react pattern. And the way we want to think about this is we want to consider a typical simple agent loop. We ask a question. The question is routed to our LLM. This is where the reasoning takes place. LLM might decide, hey, I know the answer already. Boom. Done. Don't even need to pick up tools. I can solve this with my bare hands, the LLM says. Or maybe we need to go in and pick up a tool. Now, if you squint closely in, you can see we have tools like Archive, tools like Search, like Wikipedia, like, what is that, DuckDuckGo right there? right there now what all these tools have in common we're going to go get some information you might say retrieve some information and we're going to collect it and try to then incorporate it into our reasoning that we're doing about answering this question. We might need to go back and grab another tool. We might need to see what it gives us when we engage with it and then incorporate that back in to our reasoning before we give a final answer. So the LLM here is where we're sort of doing the reasoning part of the reasoning action pattern. And this is important. pattern. And this is important. Now, when we go to the tools, when we go to the retrieval of information, what are we doing? We're actually sort of augmenting the thread that's going that our LLM is considering and reasoning about with retrieved information. We're kind of doing rag, aren't we? In fact, I'm going to put to you today that Today, in most cases, agents are just fancy rag. And we'll see this as we walk through exactly what we will build today. Armed with the patterns of prompting, of fine-tuning, of rag, and of agents, we can put together a more complex picture of what a typical multi-agent system looks like. Let's think about multi-agents. Why would I need more than one agent, you might ask? Why would I need more than one agent? You might ask. Well, remember how the agent in our picture just a minute ago was doing the reasoning. Well, consider that we might want our reasoning machines to be specialized, right? We want our reasoning machines to be specialists, just like out there in the world. Now, if the reasoning machines are to be specialists, and I want a bunch of them, where does that ultimately lead to in the context of AI? Where does that ultimately lead to in the context of AI? One question you might ask is, well, does that mean that if I had an AGI LLM that I could just use one agent? I want to bring Wiz up to the stage here for a minute to comment on this. So if I'm looking at specialists and connecting them all up together, isn't it kind of in the limit that the artificial general intelligence is going to understand everything that all of the specialists understand? So it sort of makes the idea of multi-agents not necessary. Is this a crazy way to think about it, Wiz, or is this technically sound? No, I think that's probably true. I mean, eventually, right, when we have some AGI, we could just have one agent do everything. I mean, there's a lot to be said about potentially, you know, this idea of expertise might not ever leave and so maybe we have you know specialized intelligences that are better than these uh generalized intelligences but i think the way that people use the term agi right is uh means that we would only need that agent or that system right right? We wouldn't need those specialists because they would be no better than this AGI. I mean, of course, it depends on how you define AGI, right? I mean, it's like, yeah. Okay. Okay. Okay. All right. Let's get off our high horse. Thanks, Wiz. Thanks for the two cents on that. Let's go back down to earth, everybody, two cents on that. Let's go back down to earth, everybody, because we have a job to do today. Let's talk about multi-agent frameworks, because presumably we don't have AGI today. What we're talking about when we're talking about multi-agent frameworks is we're talking about using multiple independent agents that are each powered by a language model, an LLM, let's say, potentially an SLM in the future, a little small specialized language model potentially. And we basically want to consider two things. What are the agents and what are they good at? And how are they connected? Now, if you think too deeply about this, you start to get lost in the patterns a little bit. So we're going to try to make it easy for you to understand why this is useful. to do things a little bit more cleanly in short. We can group tools and sort of responsibilities, almost like job responsibilities together. We can separate prompts instead of having, of course the infinite context window issue will tell you that you can sort of just dump everything in there and maybe you can, but it makes it really hard to sort of debug exactly where things are going wrong and this separation of prompts can also actually not just provide a cleaner architecture but potentially even give better results and then just conceptually it's going to be a lot easier for other people to understand what you've built. Now, there are many ways to accomplish lots of things you might try to build. I wouldn't immediately jump to multi-agent in almost any case. In fact, I would love to hear if you're in the audience today, if you've heard of an actual use case that you're connected to creating massive business value from a multi-agent use case. These things are still rare today and they are difficult to implement, but there are tools out there. And some of the tools include things like AutoGen. In fact, some of the things we'll talk about today in Landgraf were inspired by the AutoGen paper. This came from Microsoft and they call this a conversation framework. We essentially want to allow these multiple agents to converse with one another. That's what AutoGen is all about. You might have also seen Crew AI. of getting the crew of agents together and operating in a real cohesive unit that's crushing it together. Just like if you're on crew. And obviously lots of people are using this stuff. This is more of a low-code solution. Now we're gonna use LandGraph today. And LandGraph is all about, quote, building stateful multi-actor applications. This is like you can put many agents within graphs, directed cyclic graphs that track the state of affairs as you move through the graph. We've talked about Langraph before, we won't belabor the point, but it's all about adding cycles to applications built on Langchain. And in terms of cognitive architectures that you can leverage within Langraph, the one we're gonna focus on today is the router. It's going to be particularly useful. Now you might say, what's a router? Well, the TLDR on routers is that they choose the action. Remember that reasoning that we did in the beginning to sort of choose the tool. You might choose the tool. You might choose the RAG system. You might choose to go to another agent that is actually just a different prompt. And so when we think about the flows of multi-agent set up, these are the ones that you'll see if you go to the Landgraf repo on GitHub. There's the multi-agent collaboration, the hierarchical team, and the agent supervisor. When we do multi-agent collaboration, we're essentially trying to get two agents to share the same context. And just as we heard in the autogen paper, to have a conversation. Here we have a researcher and a chart generator, as well as a router. All three of these guys are agents. But I'm gonna sort of highlight the routers in our story. The research agent is simply sort of designed with the prompt. You should provide accurate data for the chart generator to use. The research agent is simply sort of designed with the prompt. You should provide accurate data for the chart generator to use. Chart agent is designed with the prompt, any charts you display will be visible to the user. This is a quite simple setup. The router decides which tool to call, be it the chart generator or the researcher. We can make this into a slightly different setup by thinking about our kind of router as being a supervisor. And the supervisor might choose to go to any given agent to delegate specific tasks that the user asks. Now, if we combine these two ideas, we get into something that's a little bit more hierarchical, where we actually have different graphs nested within our top level graph, where our top level supervisor stands. So this is a supervisor that is a router and the research team and document authoring team here are also both represented as supervisor routers at the mid level. All of these have prompts associated with them. We're going to simplify this slightly and use it for our build today. We're going to build what we're calling AIM Editorial, AI Makerspace Editorial. And it is a cut across the hierarchical team setup we just saw that combines the supervisor as well as the sort of collaboration idea. What we'll do is we'll have a top level supervisor. We'll have a research team that's going to leverage Tavoli search, which is a search API specifically designed for use with LLMs and agents. We will build our own custom rag system and then we will create an initial writer, a researcher, a copy editor, and an editor that will make sure our writing is super dope in our document team. At its core, if we think about what's actually going on here, at the top level, our supervisor is directing. It's deciding, it's instructing, it's delegating. These slightly more specialized research team and document team supervisors are doing something similar. We've got retrieval that we can do through Tavoli search or through our custom rag system. And fundamentally, each of the agents in our document team are using prompts. So you can see here is that if we look close enough, we can start to see why we might say something like agents are just fancy rag. We have a goal today and we want to write a dope blog on something relevant. Well, what's something relevant? We want to talk about long context. We saw this pretty sweet paper, extending Lama3's context tenfold overnight. As Lama3 told us, they would do this over the next few months. Also, shout out to Gradient AI releasing the one million context link a week ago. That was a pretty gangster move. And then this one is a formal paper on it, though, 8K to 80k with Qlora fine-tuning what we're gonna do is we're gonna set up our research team we're going to use tabily search and we're gonna use this long context window paper to build a custom rag system we're gonna set up our document team and the writer we're gonna tell it something like you are an expert writing you are an expert in writing blogs below are your files in the current directory the note taker will tell it you are a senior researcher tasked with writing a technical blog outline and taking notes to craft a perfect technical blog the copywriter will be our grammar, copy, punctuation editor, very tactical. And then our dopeness editor is going to be an expert at punching up technical blogs so that they're more dope, extra lit, more cool. Let's see how we can set up these teams using Lang graph right now. Wiz, show us how to do it. Oh, yeah. Okay, so here we go. This is going to be a notebook. So I'm gonna zoom in a little bit. So you first this is what we want, right? This is like the this is the desired output, right? So we have some input to some kind of supervisor, agent, and then we receive output from it, right? It does the task does the thing. So I think when it comes to the, the goal here, we want to combine two things, right? We want to combine kind of this idea of rag that we have. And we want to add this uh you know this potential agentic uh you know piece on top of it and the way we're going to achieve this is in parts and we're going to start with uh some basic parts and then we're going to slowly get a little bit more crazier and crazier now uh this is Now, this is an adapted thing from Langchain's own documentation. They have a great hierarchical agent example. We've made some changes here and there just to demonstrate how easy it is to work with these Lang graph systems. And we're just going to go ahead and get started. And of course, it's based off of the auto gen research that was done. So first things first, we need a bunch of dependencies. It's not crazy, though, right? So we need lang graph, lang chain, lang chain open AI, and lang chain experimental. And then of course, we want Qdrink client or quadrant, sorry, client, client, PyMOO PDF and tick token. This is, for those of you familiar with it, this feels a lot like a rag dependencies and it sure is. We're also gonna grab two API keys. We have our OpenAI API key and our Tivili API key. Now the Tivili API key is something that you'll have to sign up for. And on their free version, you get like a thousand requests per unit time. And basically the idea is it's like Google search, right? But through it through a clean API. There you go. So it's free to trial. And then if you want to go crazy with this, you're gonna have to pay pay the piper as it were. Okay, so the first thing we're gonna do, right, right. So I mean, multi agent rag, if we don't do rag, you know, it feels like we missed the mark. So we're just going to set up some simple rag here. Just gonna be over a single PDF, right? So we've got a lot of content on more advanced rag and everything like that for this notebook that's already quite long. So we're going to just keep it to simple rag. But the way that Langraph, LCEL work, you can extend this to however absurd of a chain or rag system that you want, as long as it's wrappable in a Python function, and it can take in some text and it returns some text, right? So I think this is a very important piece of the puzzle. All of these components that you see are hot swappable, right? Like we can change them however we'd like. That's kind of the beauty of Landgraf, right? So first thing we're going to do, put the R in RAG, we need retrieval. We're just going to load up this long context paper from archive. So I'll show you here. It's just like, you know this this whole context window thing is is it the rag killer right all of this this other stuff right so let's write a little blog on extending rag or extending context windows next we're going to chunk it down to size this is classic we're just going to do some uh some naive splitting nothing fancy going on here uh some naive splitting nothing fancy going on here uh we're gonna turn our one uh document into 15 smaller documents let's go then of course we need an embedding model right if we're gonna do rag we need to have some vector representation of our text assuming that we want we care about semantic uh retrieval which in this case we definitely do uh we're also going to create a quadrant uh backed vector stores this power by quadrant Quadrant is just a great vector store I mean that's the reason we're using it that's really it uh it's very good at its job um and uh it can it can scale very well so even though this is clearly like a toy example um you know Quadrant can handle very much non-toy examples, which is great. And then, of course, we're going to grab a retriever. So, we're just going to modify our vector store into a retriever. Thanks, LaneChain. Nice and easy. Then we're going to do the ANRAG, which is augmented, right? So, this is where we're going to add our context to our question. And we're going to give some instructions, talk about how it's a helpful assistant. It uses available context to answer the question. And if you don't, if it doesn't know how to answer it, it should say, I don't know. And then finally, of course, generation, because this task doesn't need like a ton of like incredible reasoning skills. We can just use GPT-3, 5 turbo for this part. There's, there's no need to use like a GPT-4 for this. And then we build a simple rag chain, right? So this is just an LCL chain. It's going to take some, you know, question, pass it into the retriever to get the context. And then it's just going to pass the question forward to the next step, which is going to pipe into the rag prompt, which is going to pipe into the chat model, which is going to be parsed as a string. So we can do things like this, ragchain.invoke on a question. What does context along context refer to? And then we get a string response. The context along context refers to a coherent text such as a book or a long paper that contains multiple independent text. Yada, yada, yada. You get the idea. Okay. So first of all, there's some limitation to this particular approach. So I just want to acknowledge those. Again, it's just an illustrative example. But, you know, this is a specific PDF, right? So we'd love it if it could take dynamic PDFs. And it's obviously very naive rag. So we'd love for it to be a little bit more complex. And you can do all of those things. And as long as the ending result is an LCL chain, nothing else will change, right? So if you want to tinker around with this, make a more involved rag chain, you can do that. Absolutely. As long as you can get the output, you know, as long as you can get the whole object to be an LCL chain, you're going to be just fine, which is pretty dope. Then we're going to make some helper functions. We need some helper functions. We're going to do the same thing over and over again. Let's just wrap it in a helper function, right? So first of all, we're going to create agent nodes. So these are nodes with agents, right? So you'll notice all this agent node does is it wraps calling the agent in a function, and it takes what the agent returns, and it names it, you know, or it adds to the state, we're going to get into state in just a second, but it adds to the status human message with the output. And that's it. That's all it does. Right? So this is the idea. We want to wrap those nodes. The reason we wrap the nodes is so that they work as expected with LandGraph, right? Where it's gonna take some state agent name, and then it's gonna give us this object that's gonna be compatible with our state. Very cool. So we have this idea of an agent node and we're invoking an agent, but how are we creating these agents, right? With a create agent helper function, of course, let's go. A few key things to keep in mind here. Number one, you know, we want to have kind of this boilerplate prompt on our system prompt for all of our agents, right? Because all of our agents that we're creating with this are going to be, you know, very, very similar under the hood in terms of their prompts. This doesn't have to be true, right? You can custom make each agent, but for us, it makes a lot of sense to just use the same boilerplate at the end of every agent, right? Your other team members and other teams will collaborate with you during, with their own specialties. You were chosen for a reason. You're one of the following team members. And then this classic, do not ask for clarification, right? We just want to go to the agent, get some response based on the tools that it has, and then send that up the chain. So this is the idea. Of course, we're going to be able to modify that with a system prompt. So we're going to be able to define more exactly what this agent is. We just have this kind of suffix that's on each agent prompt. There you go. Okay. Next piece is big. We have our agent scratch pad. This is unique to this agent right here, right? Whatever agent we've created, this is unique to it, right? OpenAI function agent, this is unique, right? So in our executor, right? This is it's one executor, which has, or which is comprised of this create open AI functions agent, right? And these tools, which has its own scratchpad. Now, this is something that's super important. Okay. So each of these little sub agents are their own agent. So already we're in, we're in, we're in multi-agent before we even get into the first draft. Right. But the idea is that they all have their own scratch pad and we're just going to populate the important stuff up the chain to the supervisors. And this is the idea of how these systems can work so effectively, right? Each agents can be able to do like a bunch of stuff. But that stuff largely just doesn't matter to the supervisor agent, right? The super right, just like in real life, supervisors care about the output. They're like, What do I get at the end here guy, right? So that's why we have this individual scratchpad for each agent. And then we're going to see another layer of that as we continue through the notebook. So that's going to create all of our agents. Then we need this idea of a supervisor. Now, I hate to be the bearer of mundane news. Supervisor is just a router. It just routes from one thing to the next thing. So it takes in, you know, it takes in current context, and then it makes a decision, where do we go next? Which which tool agent or which agent do we go to next? Right? What's worker do we go to next? And then, you know, if the answer is we don't go to a another team member, we're just gonna straight up go, we're gonna finish, we're gonna be done, right? So the idea of this particular team supervisor is just to act as a router, right? So say, hey, this looks like it needs this work done. And then it gets response back. Now it looks like it needs this work done. Gets response back. And now it's done, right? This is all it's doing. It's a lot of code to get there. but basically this is just a function. And we're once again going to create this open AI function situation. This is all it's doing. It's not crazy. It's not insane. It's just straight up routing. Where do we go next? So now that we've got those helper functions, that's just the helper functions. It's a bit of a doozy, even notebook, I know. But we're now going to talk about the research team. So remember, our total system is going to be comprised of, and we'll just go, we'll go back to the diagram for a second here. This supervisor agent, which is going to interact with this research team agent. Okay. And this document team agent. So what we're going to talk about next, back down the notebooks, I know the scrollings, you know, not flying. So sorry about that guys, but to just want to reference that document. So first things first, we have a tool using agent. What do we need to do? We need to give us some tools, right? Batman's going to have his utility belt or else he's not Batman. So we're going to start by creating a tool for Tivilli. Now, you'll notice we don't really have to do anything here, right? We're just pulling it from the pre-made integrations from Langchain tool, but we can create our own tools if we want, right? So we're going to show that next. Now this is, so technically we don't need to create a tool from our RAG LCL chain because LCL components can be nodes in a graph. However, we're just going to show it so that you can see how that tool creation happens. There's no real reason other than that to do this. You can just use the LCL chain. That's going to be fine as long as you make sure that the inputs are coming in correctly. So you might have to put a formatting component on the top of your chain. But the idea is here, we're just going to show it in the tool so you can see how easy it is to make the tools. So we just wrap it in our tool decorator, right? This just modifies the function below. And then we create an annotated parameter with, you know know it expects a string and the annotations query to ask uh the retrieve information tool and then we give it this doc string this doc string is important right so one of the things you're going to notice whenever you're working with agents graphs lane chain ai engineering we're always prompting right we're always prompting all the time right this is this is a prompt the lm is going to see this and the time. Right? This is a prompt. The LLM is going to see this and it's going to use it as a prompt. So remember, when we're writing those doc strings, it's not just random text for humans. Right? This is how the LLM is going to interact with our system. So it's important to write clear doc strings here. And then all we do is return that chain invoked. Okay. So now we have our two tools, our Tivoli search tool, and we have our retrieve information tool, which is our RAG pipeline. Next up, we're going to create some state. So we're going to add three objects under state. We're going to have messages, which is just a list of messages. So everything we've done up to this point. Team members, that's the members we have on our team unsurprisingly and then who's up next right so this is going to help decide where where are we going next right who who am i passing the ball to next uh so this we just write about that a little bit there we're going to be using gbt 1106 preview uh gbt oh one uh i can't remember the rest of the numbers right right this exact time but the newer version, the January version, is actually a little bit worse than 1106 at this task. For some reason, it just gets lazy. I think they attempted to fix that. It didn't work, so we're going to use 1106. So this is our LLM. You'll notice, though, that we are using GBT-4 here. This is no longer like GBT-3.5 is going to do. We need a strong reasoner. We need an LLM that can do a great job. That's why we're using GBT4 here. So now that we have our LLM, we need to start creating some agents. So we're going to create first our search agent, which is going to be based on that GBT4 LLM. It's going to have access to this Tavilli tool, and it's going to have this description that explains when we should be using this tool. And then of course we're going to convert that to a node. So now we have this search agent and we have it tucked inside of a node. That's awesome. We're going to do the same thing for our rag pipeline and then tuck it inside of a node. You love to see it. Next we create our supervisor agent. We're going to pass in that same LLM, GBT4. We're going to give it this text. Now, remember, this text is in addition to the other text that exists in the boilerplate, but the idea is that it's going to be able to pass to these separate tools or finish. We're going to let it know which tools it has access to. And then we can move on to the best part, right? Making a graph. We initialize a graph with the research team state graph. We're going to add the node that we created to our graph. We're going to name it search. We're going to add the research node, which is the LLM or the rag node, right, to our graph. We're going to name it paper information retriever. These names are actually pretty important. They have to be in this format. They can't have spaces and this kind of thing. So, make sure that you're naming these correctly. And then, of course, we're going to add our supervisor node. So, now we just have, have like three nodes chilling in a graph. You know, they're not connected to each other at all. Okay. So we're going to then create edges. The edges are pretty straightforward, right? If we're at the search node, we're going to return to the supervisor node. If we're at the paper information retriever node, we're going to return to the supervisor node, right? These nodes all lead to back to the supervisor. Now, from the supervisor, dependent on what's next in our state, remember we defined this next in our state up here, right? Dependent on what's next is going to determine where we go next. So, if it's search, we'll go to the search node. If it's paper information retriever, we'll go to the paper information retriever node. And if it's finish, we'll go to the end node. Now, two things I want to be very clear about here, right? Basically, we can go to the search or paper information retriever nodes, which are powered by those agents, which have those tools. And then they return to the supervisor or we can end in the graph. Now, this graph has state. And that state is its own state. So now we have agents with their own scratch pads. And then we have these nodes which represent those agents. And the entire graph has its own state, right? So we've got two layers of kind of keeping information apart here, right? Very important to think about. And then we just compile it and that part's great. And we set an entry point, right? We enter through the supervisor, easy peasy. And then we can use Mermaid to display our graph. It doesn't look beautiful, but it is right, right? So we have this idea that we can go from our JSON output function parser, which is like, you know, where do I go next, we can go to the paper information retriever, which goes back to the supervisor agent, or we can go to search, which goes back. And then this can also lead us to the end or finish. So that's the mermaid image of our, of our graph of our research team graph, right? Now, because we intend this to operate with another graph, right? We have to have some way to get in here. And the way that we're going to do that is through this enter chain. And we're going to create an LCL chain from our entire graph. This is the beauty of LCL, right? This chain represents our entire graph, our whole graph. But we could just straight, you know, just tack on another LCL component. Easy peasy. And then we can test it out. And we can see things like, you know, we enter, the supervisor says, we're going to search. We do some kind of retrieval with search. We come back and the supervisor says, we're going back to search. We do that. And then eventually the supervisor says, hey, you know what? Actually, we're going, now we're going to the paper information retriever, right? So we, the graph decided we would go to search twice and paper information retriever right so we the the graph decided we would go to search twice and paper information retriever once then it felt like it had all the information that it would need um dope okay now so that's the research team side right we created a graph the graph does stuff we're now gonna immediately think this is a single unit. This entire research team now is just this bit right here. Where it does this thing. We give it some query and it tells us which tools to use to research information and then eventually we come up with a final response that we are going to pass back to our supervisor. So this is the research team supervisor. This next one is going to this the CEO or however you want to think about it. So next up, we have the document writing team, the document writing team, we're going to go through this a little bit quicker. It's the same idea exactly, except instead of tools that relate to search and information retrieval, it's related to document creation and document editing. So we have our create outline tool, which is gonna open a file and put an outline into it and then save that file. Then we have a read document tool, which is going to open a file and read it. Then we have our write document tool, which is going to unsurprisingly open a document and write to it. And then we have our edit document tool, which is gonna, it's gonna blow your mind, open a document and edit to it. And then we have our edit document tool, which is going to, it's going to blow your mind, open a document and edit it, right? So we have these few different tools that we can use, right? So we have the ability to create outlines, which are going to be a document. Then we can read those documents. We can write new documents, or we can edit documents. All awesome stuff to be able to do when you're trying to write a blog, right? We're going to create this state for our document writing team. And it's going to be exactly the same as our research team, except we're going to add this current files additional parameter. And what this is going to do is it's going to just tell us what files currently exist in the directory it's working in. We're going to also have this prelude. All this is doing is it's saying, hey, by the way, this is the current files you have, right? This is how many files that you have. And that's it. We create the graph. It's exactly the same as the one we created before, but with some additional nodes. The idea is that every node goes back to the supervisor. So all paths from all of the sub-agents lead back to the supervisor. And then the supervisor can send it to any one of those particular agents. And that's it, right? So this is the idea. Then we can look at this and we can see it happening, right? So we see you can come in here and it can go to the doc writer, the note taker, the copy editor, the dopeness editor, and then eventually it can finish. Now, one thing that I do want to just keep in mind when we add these tools up here, right, we're going to, for each of these entities, right, we're going to have access to specific abilities, right? So this is the idea, is that we want to give our specific team members, sorry about all the scrolling here again, specific team members are going to have access to specific abilities and that's important. Okay. Now that's all great so far. Next step, right? We're just going to wrap this in the same thing we did before for our team, our research team writer. to wrap this in the same thing we did before uh for our team our research team writer and then we're going to see it work you can see here we ask it to write a short outline a linear regression write it to disk and what does it do it goes to the doc writer node which does exactly that and then we get a short outline that's written to disk and then if we look in our this is this is the worst for sure but if we look here we can see there is a linear regression outline that's created in text right in a text file in that temp directory that we pointed it to pretty awesome okay so that's what we've done up to this point we've created our research team and we've created our document writing team. And now we're going to go back to Greg, who's going to show us how we can tie these two together into a truly multi-agentic experience. Back to you, Greg. Awesome. Okay. So we've got our research and our doc team set up, the ICs are ready to do their work. So you know what time it is. It's time for the supervisors. And the thing about the supervisors that's so interesting, we talked about this before, they're just routing. They all have the same prompt. You are a supervisor tasked with managing a conversation between the following workers. Worker one, worker two, worker three, worker four, whatever. Given the following user request, respond with the worker to act next. Each worker will perform a task and respond with their results and status. When finished, respond with finish. Okay. Seems simple enough. So supervisors are just routers then, right? It they doing any reasoning? Are they taking any action? What's the role they're playing, these supervisors, exactly? I'll leave that as a thought experiment for all of you watching. But it is worthwhile to think about in the 21st century. We know what the ICs are doing. We can see their output. But for now, it's time to create these supervisors, make sure the work is going to the right place, being routed properly for both the research team and the documents team, up to the meta supervisor who is oh so agentic at the very top. Wiz, back to you to close it out. muted. Sorry, guys. Sorry about that. Thanks, Greg. But, yes. All we need to do is we need to, you know, get a new LLM. It's just going to be the same one, right? But then we're going to create our supervisor node. And the supervisor node, thanks for all the reminders and chat guys sorry about that uh but the the idea is we have uh am i still muted uh if i'm still muted let me know okay good so the idea is we just need to create one more layer. And all that layer does is it takes us from, right? So before we created two different graphs, instead of graphs, let's just consider those nodes, right? So this new supervisor, all it does is it tells us when to go to the research team or the blog writing team. That's it. I mean, you can't make this stuff up, right? This is all it's doing. Like Greg said, it's a router. We create new state, right? Which is just going to, you know, we have less things we need to keep track of since we're not so worried about the team members. There's only two team members and we have our messages. And then of course we have our next. So that's who we're going to next. And then this next piece is the best. We only care about the last message going into the new graph. And we only care about the last message from that subgraph. So we can think of it this way. We have this parent graph and we have care about the last message from that subgraph. So we can think of it this way. We have this parent graph and we have this child graph. But the only communication between those two layers is going to be the most recent message from either of them, which means that the parent graph or the meta supervisor, the ultimate supervisor, the one right at the top, CEO, supervisor, right? The one right at the top, CEO, whatever you're going to call it, it only sees the last piece of work from the research team supervisor or the blog writing supervisor, right? So this is huge, right? We only pass that relevant information. This keeps this state very clean, lets it be a very effective reasoner and powerful tool. And then of course, we need to build the graph. Well, the graph is very easy to build because there's only two nodes and they both go back to the supervisor and then the supervisor decides if it's gonna go to the blog writing team, the research team, or it's gonna be done. And we can finally use the thing. And ultimately when finally use the thing. And ultimately, when we use this, it's going to, you know, send a, it's going to do this whole flow, right? And this whole flow is going to go through research team, blog writing team, research team, blog writing team. You know, it probably won't do that many iterations, to be honest with you. Usually it does two or three, but at the end we get this right this output just a full blog on the paper i mean listen is this the best blog that's ever been created okay i'm not gonna say that but it is a blog it was created from the paper it did go through dopeness editing copy editing right uh we can see that this is uh you know pretty dope results are nothing short of revolutionary that's pretty hype language. That's something that our dopeness editor helped with. This is the idea. This part is very straightforward. Each of those sub nodes, right? Each of the sub nodes or sub graphs, we just consider a node. That's the power of laying graph it's an entire agent graph but we're just like it's a node you know who cares uh and that is it uh so good with that we'll pass you back to greg so sorry about being muted guys thanks for calling me on the chat and uh also don't forget to like comment subscribe smash the notification bell i know it's kind of kind of silly but it does help we're here every wednesday we love talking about this kind of stuff and uh i'll pass it back to dr dr greg to uh bring us to q a bring that bell baby yeah you crushed it wiz thanks man so we got a gentic with the meta supervisor and now we can think about this idea of multi-agent RAG in the context of the patterns of generative AI that we're using to build LLM applications. We saw how this all came together in a single build, been through a lot in an hour together. And we can also start to see why we can say things like agents are just fancy rag. Now, remember, these are useful because the grouping is very helpful. the separating of prompts is very helpful, the conceptual models are very helpful. Again, let us know if you come up with any sort of must-have multi-agent use cases. I would love to hear about them. But the patterns, the patterns, the patterns, they're just everywhere. Tools all have prompts and supervisors or routers and searches, retrieval. It's a lot to get a handle on. And we hope that this helped you out today. If you have questions, please let us know in Slido. We're going to start taking them now. And we look forward to creating more multi-agent content in the future. We want you guys to take this notebook and create your own blogs with it. We will liken some to those and maybe we will. If we can get this good enough, dope enough, Chris, create our own AI Makerspace auto blogger, the AIM editorial. Okay. So two, Slido, which platform is better for multi-agent rag? Langchain or Lomindex? Langchain. Boom. Okay. All right. And can we get just a why real quick? How come? I mean, stuff like LCL is just, it's such an effort multiplier, right? We make one thing, we can just straight use it in the next thing. Yeah. It's tough to beat that right now. Yeah. I love the second question so much. Seems that everything can be done with a single agent. Only difference is the forced sequence of agents, of tools. Is there something else I missed? Anonymous. Yeah, I think maybe a little bit. So there's no forced sequence of tools here. The agent is free to select which tool to use when, in which order, how many times. Yeah, that's the idea. So I would say the different sequence of agents is kind of where we get this. Could it all be done with a single agent? Maybe, right? So you could just add all these tools to one agent. But the idea is that this compartmentalization is supposed to make the LLM has one fewer decision or sometimes four fewer decisions right if we're using the four writer tools right this is the idea is that we instead of choosing between like 12 tools is choosing between two tools or three tools or four tools and that is supposed to make it better yeah okay. I go back to the child at the grocery store. Which kind of mustard do you want, sweetie? Do you want it to have a hundred different mustards to choose from or three? And I think it is a great question to always ask though. Can it be done with a single agent? Can it be done with no agents? Of course, we were doing multi-agent RAG today, so we used multiple agents. Next question, is it possible to share variables like DICs, data frames, or any other between agents instead of just making them communicate with natural language? Yeah, yes, absolutely. So we can do that by passing different parts of state, different components of state. As you saw in this example, we only pass the last message into state, but we could add more things and add even more state. And I think that's going to be a decision that you need to make depending on your use case. But yes, the answer is you can absolutely pass. I'm not going to say whatever you want, because that's of course literally not true, but you can pass basically whatever you'd like. Okay. Nice, nice, nice. Okay. So when dealing with multi-agent RAG, it gets hard to cite or source responses in line. Is there an effective way to do this across all the receipt retrieved sources in line yeah okay so for citation that's a little bit harder uh you could add like a state that just keeps track of the various sources and citations and then populate those at the end in some kind of dump uh that would be the uh that would be the base way that I would want to approach this if you want to be sure that you're citing everything that you can. Some of these agents aren't going to produce citations because they're not really looking for those things. But yeah, with state, basically, you'd want to manage that context as you're passing it around. You can add that fairly straightforwardly. Okay. Can agents work in parallel? Yes, of course. Yeah. So the idea would be just to make sure I understand, like you can, some of these flows can be paralyzed, right? So if you need to search two different tools, you can search them at the same time and then synthesize a response once you receive a response from both of them, right? So that's already built in through LCEL. I believe it's coming to Landgraf soon, TM, but for right now, it's built into the LCEL components, and then I believe it's going to enter into Landgraf soon enough. Okay. And what are the techniques or design patterns to make our system faster and more responsive? This multi-agent setup can be potentially slow. Can't it? Oh, yeah, for sure. It's going to be slow. I'm not going to, you know, tell you that it's going to be fast. You can make it feel faster by using a lot of streaming, right? So streaming the responses is going to mean that the time to first token is very low, but it's still going to take the same amount of time to generate the full final answer. So it is going to be something that takes a little while, especially when we have like these six to seven different calls. Also one thing to touch on from that point of view. Right. This is where a tool and integration like Langsmith, which we didn't touch on in the notebook, but is easy to integrate, comes in and answers a lot of the questions we've seen in chat. How do we know how many tokens, how many calls? What path does it take? All of those can be added or tracked through through Langsmith. If you if you use that integration. Yeah. And I just want to sort of mention shout out to Garrett, big homie in our community. He's building deep writer and if you want to talk about how it's slow and potentially expensive to do multi agent stuff, He'd be a great resource for you to follow and to start a DM thread with. He's all about all of this and constantly building every day. So it looks like we've got a lot of other questions, but we are about at time. We will collect these questions and we will try to make a few posts in the week to come on LinkedIn. So give us a follow there. But that's it for today. We wanted to make sure that we end on time. We'll be back with more multi-agent stuff soon. You can count on that. Thanks so much, Wiz, for walking us through that. That was incredible. We will wait on publishing our first blog until we think it is truly dope enough. And let's go ahead and close it out for the day. If you guys enjoyed this and you don't know AI Makerspace yet, we'd love to see you on Discord real soon. We're building, shipping, and sharing with folks all the time. And we'd love to have you as a part of our community starting now. You can start learning for free of course on YouTube. We've got an open source course on LLM Ops that we taught last year. We look forward to open sourcing another course here very soon and we are always running our bootcamp courses. Our flagship one is the AI Engineering Bootcamp. It is an eight-week course that walks you through everything from your first LLM application through the patterns that you need to leverage and build up to multi-agent frameworks. We are starting our next cohort on May 28th. It's kind of a high bar and quite a bit of friction to get in. So apply now and start working your way through the AI Engineering Bootcamp Challenge. To check out more events, if you aren't familiar with us, check out our awesome AIM Index on GitHub. You get direct access to all of the code. There's always concepts and code with every event that you will join on YouTube. Again, we're back every week on Wednesday, same time, same place. We hope to see you again soon. Like and sub and in the meantime, keep building, shipping and sharing, and we will most certainly do the same. Have a great week, everybody. We'll see you soon.
Multi-Agent RAG
Discover how to integrate multiple independent agents to tackle complex problems effectively using the latest frameworks like AutoGen, Crew AI, and LangGraph. We'll dive into the innovative multi-agent systems, particularly focusing on the shared scratchpad approach in LangChain, and demonstrate building an advanced Agent Supervisor model. This system enhances problem-solving by coordinating agents, each with their own scratchpads, under a supervisor that manages the final outputs. Whether you're a developer or just fascinated by AI's potential, join us to learn, interact, and get hands-on with the future of collaborative AI technology. Click now to be part of this journey into multi-agent systems!


Event page: https://lu.ma/agentrag

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/wPDemMAc9nzV96DFmBzXz5

Speakers: 
Dr. Greg, Co-Founder & CEO
https://www.linkedin.com/in/gregloughane

The Wiz, Co-Founder & CTO
https://www.linkedin.com/in/csalexiuk/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

Apply for our new AI Engineering Bootcamp on Maven today! 
https://bit.ly/aie1

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/6NNYtu7MiSUcnWAh6