. Wiz. Today we got Llama Index Workflows. This is the new framework for agents from Llama Index. Is that right? That's right. Yeah. Okay. It's just for agents is that right uh no i i don't think so no okay it's actually for it's actually maybe for like all different kinds of workflows including agentic workflows right that's the ticket that's. That's kind of where we're at. So if you knew agents in Lama Index before, your knowledge is still good. We can apply it here today, can't we? That's absolutely correct, yes. Okay. So we're adding another layer of abstraction to allow for some additional complexities in the applications that we build, in the applications that we build in the workflows that we build. There's some kind of cool stuff going on behind the scenes today. I'm kind of excited to get into it. You ready to kick this thing off, Wiz? I can't wait. Let's do it, man. All right. Here we go, everybody. I'm Dr. Greg. We've got the Wiz coming back in a little bit to show you guys and to discuss some of the details of Lama Index workflows. We are AI Makerspace. Thanks for joining us for this community event where we're going to dig into the latest and greatest from Lama Index. This is a beta tool called Workflows. And initially, to me, it appeared that this was a new way to do agents. In fact, it turns out it kind of is, but it's a little bit more than that. And so we'll get into exactly what you can expect from Workflows throughout the event today. If you have questions along the way, please use the Slido link in the chat and upvote your favorite questions. We'll do our best to get to it, to get to every single question today. Okay, let's go ahead and jump right in today, guys. Workflows with Llama Index. What is this all about? We've got some keywords that we'll hear today that we need to kind of demystify as we go. So let's align ourselves to the session. What are we going to get out of this hour that we spend together? Well, you're going to understand the core concepts and code for Lama Index workflows. This is something that you always walk away with with every AI Makerspace session. Importantly, we're gonna learn how to build an agent in Lama Index and then leverage this idea of this event-driven architecture that workflows is using behind the scenes. We're gonna talk about the differences between some of the graph-based approaches that you may be familiar with for orchestrating agentic applications versus this event-driven orchestration implementation that Lama Index has put together. And so we'll kind of overview Lama Index workflows. We'll kind of get an idea of some of the claims that are being made. We'll talk specifically about the event-driven architecture and we'll contrast it and compare it to other leading architectures available on the market today. And finally, we'll get into our build and we've got kind of a cool little extra special thing we're doing in the build today. So hang out for the build section if you're curious about what that entails. But let's kind of focus on the meat and potatoes for the first sesh of today. Let's talk about Lama Index workflows and let's talk about about Lama Index workflows and let's talk about Lama Index and what they're saying about workflows. So from the top, Lama Index is now branding themselves a framework for building context augmented Gen AI applications with LLMs. And now check this out, added, including agents and workflows. So this is kind of a quite a high level of abstraction thing that has been put into Lama Index and one that you need to know about. Now, we need to understand how Lama Index defines words like agent and workflow, so we can kind of situate ourselves. According to Lama Index, an agent is a semi-autonomous piece of software powered by an LLM that's given a task and executes a series of steps towards solving the task. Okay. Task steps. A workflow is an event-driven, step-based way to control the execution flow of an application. So we have this execution of a series of steps that's happening in an agent, and we have this step-based approach, this event-driven approach that we leverage with workflows. So we see two keywords here with workflows. And the two easy pieces that we want to keep in mind throughout the day are events and steps. All right. We want to think about our application is divided into sections called steps triggered by events. And these steps, they're not that complicated. They're literally just Python functions. They're kind of arbitrary, whatever you want functions. They literally could be anything. They could be just a single line of code, quick calculation. They could be an entire complex agent. And these steps are not just triggered by events, but they actually emit events, meaning the step can be triggered by an event and it can further trigger another event. Okay, these steps are also called components in their documentation, something to sort of keep an eye out for. So the steps are kind of this sort of fundamental piece to the puzzle. The event is the other piece. What is an event? Well, an event literally is a change of state. Or you can even get more broad about this, like sort of anything that you could notice or observe or record, anything that your application could sort of use as a trigger to do something else. So there's a lot of things that this could represent in sort of an enterprise context. Customers interacting, sensors on the manufacturing floor, etc, etc. Lots and lots of stuff. And really when we get right down to it, this sort of overlapping of event and state is sort of the place we want to really focus our understanding today. So we'll hear things from Lama Index, like by combining steps and events, we can create arbitrarily complex flows that encapsulate logic and make your application more maintainable and easier to understand. Okay, that all sounds great. Complex flows, making it easy. We love that at AI Makerspace. Okay, so these workflows are effectively then dispatching events through steps, these Python functions. And each step is sort of like one component of your system. This is this idea of steps and components are sort of the same thing. idea of steps and components are sort of the same thing. You know, you could have one that processes a query, one that talks to an LLM, one that loads data. You can have steps to do all sorts of different things. And each step is going to receive one or more events to process, and then it can also kind of get the word out to other events. Now, the whole purpose of this, just sort of zooming out for a second, the whole purpose of this is when we have something that's getting quite complex, we need a way to manage this complexity. And Gen AI applications, the ones we're building today, they're becoming ever more complex. And as a result, it's becoming ever more challenging to manage that complexity through the flow of data and the execution of different pieces of the application. And so workflows is sort of Lama Index's way of managing this, because as we know, you know, complexity can occur based on some really, really simple rules upstream. You can get a lot of complexity downstream. And this is a way that I often like to introduce agents is to think about agents as giving rise to complexity. This idea that simple rules in an agentic setup, you know, will eventually allow us to build some really, really incredible AI applications, right? So if you consider that sort of consumers and corporations are all sort of agents within an economy, or if you consider that neurons are all agents within a brain or molecules are all agents within cells. You can sort of stack simplicity to get complexity. And so the whole idea of what LAM index is doing here is it's saying, well, as we stack simplicity to get more complexity, we need programmatic design patterns that will allow us to manage this stuff better. So the question we might ask is, why are workflows the best solution for managing this complexity? And this is kind of what we want to get to the bottom of today. Are they the best solution? Well, bottom of today. Are they the best solution? Well, what Lama Index noticed is that other frameworks, including Lama Index themselves, were attempting to sort of build pipelines, to build chains, to build directed acyclic graphs or DAGs to build some of the earlier applications. And this sort of acyclic piece here, the A in DAG is quite important because it means not cyclic, no cycles, no loops. no loops. Now this is fine in a world that doesn't have agents, but in a world of agentic reasoning action patterns and loops, this is a big problem. So we need a way to do better. These DAGs create a lot of issues that Llama Index and the team over there identified specifically by working with their users. And they noticed like when I have to encode the logic directly into like graph edges, it makes it a little bit hard to grok, a little bit hard to read, a little bit hard to understand. When I'm passing data between nodes in a graph, sometimes it's not clear which parameters exactly should be passed and which shouldn't for optimal execution flow of the program. And then sort of anecdotally, LAM index users told them that DAGs just didn't really feel natural for developing these types of applications. So here we are, and we're getting this event-based pattern, as well as this sort of, quote, vanilla Python approach of workflows to resolve these problems of DAGs. And so the event-driven architectures, this idea, it's not new. It's quite old in software development, actually. If we define event-driven architecture, this is quite common in modern applications that are built on microservices. It's just a software design model that uses events to trigger actions. So we can start to see, okay, well, I'm going to use events and actions. That reminds me of the React framework with agents. This feels like it might be a decent way to solve this problem. Let's take a closer look and remind ourselves again that what is an event? Well, it's just a change of state. We're going to trigger steps with our events. We're going to take steps, and we're going to trigger and communicate with other events. Now, this is not the first iteration of new tools from Lama Index to solve this problem of complexity. In early 2024, Lama Index released query pipelines. And query pipelines were okay. were okay. They were sort of implementations of this DAG abstraction that was modified to be able to kind of deal with the Lama Index data agents. And it kind of worked, but again, Kind of worked, but again, they realized sort of fundamentally the DAG approach, it really was leaving them wanting. You know, they kind of got the ability to chain together different modules with these query pipelines in order to create more advanced workflows. in order to create more advanced workflows. But as soon as they added cycles, they said, well, issues, right? These issues we discussed. And they were like, well, what are other people doing? And they looked around the ecosystem and they looked around what's going on right now at the LLM edge. And they were like, well, instead of like using a graph, a cyclic graph, maybe you could say directed cyclic graph approach like others have picked up. One where you're sort of designing explicitly how the graph is to be traversed through nodes and edges, which, by the way, does have some key benefits. For instance, the graph traversal algorithm approach is quite intuitive. That is, if product can draw the diagram, the flow chart of boxes, then engineering can really easily build it almost like one-to-one with the drawing from product. the drawing from product. So that's a pretty powerful thing. But again, rather than going this route, Lama Index said, no, we're going to go event-driven. And in an event-driven architecture, the step subscribes to certain events and is ultimately responsible for deciding what to do based on the data it receives. You know, Wiz and I were talking about this earlier this morning, and I created this meme based on our discussion because it is quite literally that we kind of have these floating possible events that kind of exist, any of which could be next. Okay. Right now we're at our current step. We've triggered our current event and we're looking at what's going to happen next. And based on the way we set up our event-driven architecture, one or more of those events is going to be chosen next, and so on and so on. So what's kind of happening is we're getting this sort of, again, this dispatching that the workflow is doing. Back and forth through the different steps, those are Python functions. And so we have steps, we have events, we have this event-driven architecture. architecture, it's not quite the same as the loop cyclic graph-based approach. Let's just double click in on that for a minute. Of course, the elephant in the room, the alternate approach that all of you are wondering about is Landgraf versus Lama Index Workflows. Now, of course, Landgraf talks about using their tool as a stateful multi-actor application building tool. Stateful. What are we talking about? We're talking about state. We're talking about the state of affairs within our application. Something that's happening now or at a specific point in time. Landgraf is all about this idea of creating state machines and specifying them as graphs. And again, with this approach, if you use the graph-based approach, if you can draw it, you can build it, and that's pretty dope. So what are some of the benefits of an event-driven approach versus a graph-based approach and vice versa? Well, as far as we can tell, the benefits of an event-driven architecture are that you actually need to do less explicit planning of exactly what the execution flow of your program is going to be. And the event-driven idea is something that, you know, using event handlers and this kind of thing is something that most programmers with experience in industry are going to be familiar with coming into this. So you kind of have that going for it as well. Now, if you look at the graph-based approach, you might say it's a little bit easier to whiteboard. And interestingly, and we'll have to sort of stay tuned to see how this continues to unfold. Interestingly, while Landgraf provides a stateful multi-actor approach to building agents and multi-agent systems, where in a multi-agent system, you're sort of subclassing a graph within a graph. in an event-driven architecture, it's not clear that we can easily subclass other workflows within a single workflow. Although I'm sure multi-agents and workflows is something that we can expect to see soon. So with this, I want to bring Wiz up to the stage. I want to just chat a little bit for just a moment. I mean, one of the things, Wiz, that I believe is from the docs, and this is this one of the classic from the Lama Index docs, like workflows make async a first class citizen. This first class citizening, I love this language from Lama Index on different aspects of there. And it sort of brings up this idea, I think this is where we got to with it, where it's kind of like, it's a quote, quote, sort of looser implementation of Landgraf, you know, and whereas it's more implicit, this implicit nature of event-driven architecture is really what comes through more than anything else. What do you have to sort of add to this idea of, you know, it's a looser way, it's more implicit, making async more important. I mean, how is it that you view exactly the difference between this event-driven architecture and a Landgraf style, let's call it a cyclic graph architecture. Yeah, I mean, basically, the way that I would think about it is the, we don't have to connect nodes and edges by connecting nodes and edges. So the kind of when you're building a land graph graph, there's a literal process by which you connect node to edge. Now that's something that still needs to occur, obviously, right behind the scenes. But we don't have to explicitly define it as if we don't have to write the code out, right, in order to do that. So that connection is abstracted from us. We're still effectively connecting nodes to edges by providing what state matters to what, or sorry, what event matters to which step. And then, you know, we dictate which event each step emits, right? So we're still drawing these lines, but we don't have to like call, connect this to that. And instead of having this idea of like a conditional edge being a kind of weird semi-node that floats in between two other nodes, we can just have our step emit different events based on the results that we see in that step, right? So it's this idea of we're moving away from needing to draw the graph ourselves and letting the letting Lama index draw that graph for us. Yeah, yeah, yeah. So moving away from drawing the graph ourselves, I like this. And sort of, I think the image in my mind is kind of that regardless of which implementation we use, after we run the program, we can draw a graph of exactly how things went, right? And so in either case, you have a start, you have a stop, you have stuff that triggered things in between effectively different states. Right. And while you may not be able to draw it ahead of time with an event driven approach, you can still draw it after the fact. So I wonder if you can comment on also kind of the familiarity for software engineers and programmers coming in. Event-driven architectures are something that are quite common, right? Not necessarily in machine learning, but in software engineering. Is that right? Oh, yeah. I mean, like this is kind of the listeners and events. I mean, if you're from that world, this is going to be very natural to you, right? Now, the exact implementation and behind the scenes what's happening, we can kind of quibble, okay, about how truly or purely event-driven this is. But it is that same framework of thought, right, where we have each of these steps is listening for a specific event. And when it hears the event, it will be triggered and it will do something and then it will emit some other event, right? So it's the same. It's the thought process is there. And that's going to really help people who are used to that thought pattern, build applications that extend beyond just a graph. Okay. Okay. And we got a couple sort of interesting questions just to maybe click in a little deeper here from the chat here. Giancarlo asks, so does this happen inside the same agent? In other words, can a single event trigger multiple steps at once? All of the events are asynchronous, which means that they're not blocking. So theoretically, yes. You could have a event that triggers multiple different simultaneously running steps. Right. And then you can actually also listen for multiple events and wait until both events are ready before triggering a step. Right. So say you wanted to trigger three events at once and we want it to wait for each of those three to be done before moving to the next step. You can do that. Yeah, yeah, yeah. You have some freedom. And I love the Willy Wonka questions here. Let's go to the top one here. Are we defining a graph indirectly by defining events and what happens when that event is triggered? In other words, like, you know, the way I think about this is if I'm a product guy and I want to understand what's going to happen and how my user is going to be impacted by this and what they're going to, what sort of payload they'll be delivered. I do kind of want to draw this out and I kind of want to understand how things are going to happen. And then, and then I would maybe give that to my engineer. Let's say you, Wiz, I would give it to you. And if you were implementing an event-driven approach, it might end up sort of looking like the graph that I drew in the first place after we run it, but not upon your implementation. So that's sort of the way I'm thinking about it. Does that square with the way you kind of consider this? Yeah, it is absolutely just making a graph. Like, so now it's good that it's not literally making a graph, but behind the scenes, yes, of course, we can represent this as a graph. But I mean, I think the key intuition here is most applications could be represented as a graph, right? Because it makes sense. We have functions that do things and then return results to other things. And those things take that result and they do things. And we have this kind of meta layer of state and we have this kind of meta layer of events now. But yes, there's this kind of, there's this creation of a graph that's just necessary for our application to be complete, right? We have to be able, there can't be an unreachable node or else it's you, or else why do we have it? And there can't be a node that we can't go anywhere from except for the final node or the stop node, because where do we go once we hit it, right? Like the, the idea is that it is kind of a necessity that these things wind up having to look like graphs or we w w more precisely, we, we should be able to represent them as graphs, which is always useful, right? Uh, but it is, we have some extra layers in the fabric of the graph, you know, like it is still a graph, but we're not creating it as a graph. Someone said like PubSub. Yeah, I mean, kind of right. You can imagine this being a PubSub situation where you have, you know, your agent existing in a more like ready state. And then this is the same kind of framework that PubSub is built off of, except now we have these LLMs involved. Yeah, yeah, yeah. Okay. Okay. Okay. All right. I love this. And yeah, I mean, that's kind of this idea of like, if you can create a flow chart, you should, because everybody loves flow charts. And like, wow, look at that visualization of the application. Yes, that's always there sort of in the background, whether you call it a graph or a flow chart. But yeah, okay. So I think we've got it. I think we got event-driven versus graph-based. The nuances between. Wiz, thank you so much. It's time to get into today's build everybody so let me introduce today's agent build for you just kind of recalling a couple things that we've already talked about llama index framework for context augmented geni applications, agents and workflows. Remember an agent given a task, here's some steps. Now there's some constructs associated with core Lama index that we're not going to go into. You can check out our data agents event we did with Lama index a couple of months ago. It's still completely valid and super useful. So if you want to know more about exactly what's going on with the agent behind the scenes, check that out. Workflow, event-driven, step-based. So Llama Index, all about context augmentation. Context augmentation, we can kind of think of this today with the in-context learning design pattern that we're leveraging in our application stack of today. We can kind of consider this as, okay, we're talking about RAG, right? And RAG, of course, is just dense vector retrieval with some in-context learning. That is, we are returning reference material from our data. We are augmenting the prompt, the context, putting everything in the context window, and we're getting better answers. Okay, so we're going to do something a little bit cool today. We're going to take RAG to the next level. Classic RAG, you ask a question, we find similar stuff in our data. We set up a prompt template. We return the similar stuff we find to the context window. So this dense vector retrieval is coupled with in-context learning to give us retrieval and prompt augmentation. We are almost done with RAG. We still need to do generation with our chat model and get our better answer. So what we're going to do today is we're going to take this whole RAG pipeline, and we're going to just shrink it down because we're going to build something slightly more complex. We're going to build an agentic RAG setup. So we're going to build an agent that can leverage RAG, but can also leverage some other things too. So we'll ask our question. Question will go to our agent. Our agent will decide, should I go to the RAG pipeline or should I go to something else? And we're actually going to give it access to two tools, the RAG pipeline or a Tavoli search capability. search capability. And based on whichever tool gets selected, there'll be some result, some data that we get out of that, that we can observe, and we can push it back to our agent to decide, hey, do I know the answer yet? Or do I need to go find some additional information? And we've got another layer that we're adding to this today, because we know you guys like the additional complexity. And everybody, meet your new friend, Crag. So we're going to slide in some corrective rag here, everybody. And this is for when retrieval goes wrong. Okay. In other words, just as Rag is trying to eliminate hallucinations through reference material, sometimes the reference material isn't the best. So for instance, when I do retrieval in sort of a low quality way, When I do retrieval in sort of a low quality way, question, who is the screenwriter for Death of Batman? I'm going to go find Batman, 1989, Murder of Bruce Wayne's, when Ham's script, it seems like Ham probably wrote this screenplay, right? Well, I see how you made that jump, LLM, but no, not right. So we're actually going to develop a little workflow with Crag. So we're going to ingest some data, and this is actually the visualized graph representation of the workflow. We're going to do retrieval. We're going to evaluate relevance of our retrieval. And then we're going to do an extraction step. We're going to transform the question, our initial query. And we're going to show how we can also leverage tably search here before we get our final response. And the application that we're going to do, Crag, agentic Crag on today, is we're going to do, we're going to take some docs we've been looking at in class and this week at AI Makerspace, EU versus the AI. These are the, quote, harmonized rules on AI and amending regulations. So, you know, I had sort of questions about this, like EU versus the US, like what are people saying here exactly? Like how many parameters are too many, for instance, or like what's really going on here? And is it the IP stuff? Is that even a thing here? In fact, if you're interested at 12 p.m. PT today, jump into Discord with Akash Shetty from our community who will be discussing this very topic. So we're going to use the data from the EU versus AI legal docs, and we're going to leverage some classic models, Text3 Embedding Small and GPT-40 Mini, to build our agentic Crag workflow. And you know who's going to do that for us. The LLM Wizard himself. Wiz, over to you. Crag time. Yes, and it's time for Crag. Okay, so we've got our notebook. I'm going to zoom in so you can see the notebook. Classic. First things first. We're going to go through this. It's a lot of stuff that we have to do. There's code, you know what I mean? But it's straightforward. It's simple. It's just, you know, getting used to new frameworks, classic. We're going to cover events in steps mostly, and then tying them together into our workflow. We are going to use the corrective RAG workflow based on the corrective RAG paper. Greg's done a great job of highlighting it. We're going to go a little bit deeper so we can think about how there's events and steps in there. So first things first, we got to do a bunch of prep. We are going to be, you know, as Greg said, you know, async is a first-class citizen in workflows. So because we're in a currently running event loop, we have to import the nest async IO so that we can async in our async classic. Next thing, we're going to grab some Lama index dependencies. We need, of course, Lama index. We we want llama index tools to philly research for our to philly that's going to be our external uh search tool and then we're going to get our uh our classic llama index utils workflow which is going to help us make a pretty graph then we're going to set up our open ai keys we're going to use open ai's models today as our llm and then our to philly search search key for our external search tool. The next thing we're going to do is we're going to prepare that data. So we have this EU Act that just was released. It kind of goes through all kinds of, you know, it's a big document filled with legalese about AI and how the EU plans to regulate AI. So we're going to use that. We've just provided an HTML file in our data repository that was downloaded from the regulation. We use the English version. So this is the source document we're going to be looking at. You can use whatever document you'd like as you see so steps and events so what's a step and what's an event step is a unit of work or task in a workflow they're just gonna be python functions decorated by a step decorator for the most part each step though is associated with you know one or more events as input and it emits as an output an event, right? Technically, a step can event different events. So, it doesn't have to, there's not one event that the step must emit. It can emit, you know, one or another event. So, we talked about this before. The basic idea here is that, you know, our event is our path in, and then we send an event out, and then that's handled through the workflow. For each step, though, all it knows about is the event in any global context, right? So just like Landgraf, every step is an island. We feed it what it needs at time of running. Okay, steps can be extended to have multiple workers in a workflow. So this is actually pretty cool, right? Let's say we have a step that needs to be executed three times. Instead of doing that sequentially, we can just have three workers, and all three of those steps can happen at once. Then, of course, we have our modify, we have global context. So steps can modify the shared global context, aka state, as required as well. The important part of this is, as opposed to having like a state object that is passed around, we kind of have this state fabric that exists in our workflow, and we can pull things from it. Next, we have our events. And of course, events are very straightforwardly, they're just little data structures, and they pass information. they're just little data structures and they pass information right so when we when we send an event out it has a signature other other steps are gonna be waiting or listening for that event and it's gonna contain some data that's relevant for the running of its associated step events are pydantic models if you're used to Python this is this no, you're no stranger to this, right? Pydantic is a good way to get validation for quote unquote free. And then there are two special events, which are, there's more special events, but these are the two most important. We have our start event. Start event is the entry point into the workflow. So it is the, it's like the first event, right? And we'll see how this works when we write the code out. But the idea is that start event can refer to potentially more than one step. And then our stop event. This is, I think, the key difference between this and Landgraf. The stop event can occur just whenever. And then it's gonna output whatever, right? Whatever it's got access to. So the stop event is interesting in the sense that we can just kind of stop when we feel like it, and then we can output whatever is relevant to output. So that's a lot of, you know, big info dump, of course. But we're gonna kind of keep this in mind as we talk about Craig. So Craig has this lovely picture. We're not going to spend too much time looking at the picture, but the basic idea is that we do retrieval, and then we evaluate our retrieved documents, and we keep only the documents that our evaluation deems are relevant to our query. deems are relevant to our query. Then if we have any documents that were retrieved that are deemed not relevant, we're going to move to our searching tool, which is going to be our external search tool. In this case, it's going to be Tavillion. So what does this mean? Let's say we have five contexts, right? And we determine that four of them are relevant, but one is not. What we're going to do is for that irrelevant context, we're going to rework our query so that it's useful for our tool. And then we're gonna ask the question to our external search tool. And then we're gonna use that to augment our prompt. So the basic idea is if all the contexts are relevant, it's just normal rag, right? We just go to the next step. However, if we have one or more irrelevant contexts, they are omitted from being provided as context to our final LLM call. And then they're supplemented by this web search or this external search. So, you know, this is the basic idea. Let's think about it, you know, from the steps, events, mindset, right? So, okay, so we have ingesting and processing data, right? We must, there's a retriever, so we must have to take in some documents and create an index. So we're going to imagine that that's a step, right? And its function is to ingest, process, and create an index to be used in following steps. The input event to this, so the event that's going to trigger it, is going to be the start event. And the output events will be the stop event, right? Because we're not doing anything once we've created the index. We just need to create the index. And then it's going to return the created index. Okay, cool. Then we're going to have a second step, which is preparing all of our pipelines for use. It's going to, again, start at the start event, and then it's going to move to the prep event, or it's going to emit the prep event, saying, hey, preparation is done. Whatever steps are waiting for preparation to be done, here's the event that signifies we're done. The state of that output event doesn't have to be anything, right? Because we're actually going to store all of this pipelines and everything in our global context. And the code's going to highlight this a little bit better. And then for step three, it's easy. We have to retrieve, And then for step three, it's easy. We have to retrieve, right? So we have a user query. We must now do the retrieval process to get the list of contexts to see if they're relevant or not. So the retrieve context step is going to be waiting for the prep event, which means that everything's prepared. And then it's going to emit the retrieve event, right? So it's gonna say, hey, retrieval is done. And you'll notice a pattern here, that the steps are going to indicate doneness or completeness, rather than the steps are going to indicate what step is next, right? So this is just convention. You can whip it up however you want. But you'll notice that in this example, we are saying when the prep event exists, that means the prep event is complete. Now we can move to the next step. And when we complete this step, we're going to omit the retrieve event to indicate that the retrieve step is done. Right? And what does this have to have in its state? Well, just a list of contexts. Then we're going to have the same thing for evaluate context step, right? It's going to need to evaluate those contexts and then assign whether or not they are relevant or irrelevant. Same thing here. We're going to start this. We're going to kick this off when the retrieve event is emitted. And then we're going to emit, once this is complete, the relevance eval event. And as a state, we need a list of contexts, which you already have, right? So we're going to see how we handle that in the code, and then the results of the evaluation. For the next piece, we're going to extract all relevant context. So we're only going to keep context that's relevant. This is what Craig's doing. And again, that's going to kick off when the relevance eval event is emitted by another step. And then it's going to emit the text extract event when complete. This continues to transform the query. This is actually gonna transform and use the external search tool. We're gonna need to keep the relevant texts and the external search documents now. And it's gonna kick off during the text extract event. And then it's going to say, I'm done. You know, we're done this. And then the next step, step boom final query okay so this all sounds great how do we do it in code it's always much easier in code we create some events creating an event is as easy as subclassing event and then include including any relevant state so we talked about the prep event didn't need state so we just pass out easy and then the retrieval or the retrieve event needed to have a list of contexts and so that's what we have here right and you'll notice that the events are kind of like mini state right we can think about it that way they're they're mini state that's associated with a particular step and the same thing for our relevance eval or text extract event and our query event. Easy peasy, lemon squeezy. Let's go. Next, we're going to need some prompt templates. As always, prompt templates. This is going to be our evaluation prompt template. So remember that we said for crag, we want to evaluate if the context is relevant or not. Of course, we're getting an LLM to do that. And this is the prompt that does it. And then we have to transform our query, right, to be compatible with our search, our external search tool. And so that's what this prompt is doing as well. Now we're going to set up steps. This is kind of like, this is the big part. This is the lots of code part. We start by creating a workflow workflow class that some classes workflow we're going to call it corrective rag workflow uh and then we're going to take a pit stop at this context thing context term overloading i know but context is the global context or the state of the workflow this is is accessible by all steps, right? So this is not just accessible through events. This is like, this is a layer that exists for all steps. Very useful as we'll see. So we have our ingest pipeline. You'll notice that that has a start event. And you'll notice that all we do is we create a ingestion pipeline, aka we need to create a vector index. So we create a vector index and we return the result as index. So for the second step, we need to look at going through a preparation step to set up all of our pipelines. Now, the keen-eyed among you are going to notice, hang on a second, Chris, right? We have start event for this step and this step. How do we know, you know, which one? Well, we don't. We're going to do both. However, if we don't provide documents, we just return none. None is not stop. Okay, so this is, I know we're getting into the weeds here, but none isn't stop. So if we get, if we have a start event, we will fire this node off. Absolutely. Right. We are saying this step. However, if we don't provide any documents, we're just going to return none. And then whatever else is happening is going to keep happening until a stop event is raised, right? So, and it's the same thing for our prepare for retrieval. If we don't provide a search term or a search query, we just stop and we let whatever is happening until there's another stop event, right? So the idea is that these nodes will both fire off each time. However, depending on what initial context we provide, right? What our initial inputs to this workflow are, it's going to exit one of those paths early, but in a non-stopping fashion, right? So it doesn't actually cause the application to halt. It just says, oh, we don't need anything from this bit, right? And so you can imagine that you could build quite complex sub-pipelines and everything like this, and all without stopping the rest of the application. That's the flexibility. Okay. the rest of the application. That's the flexibility. OK. All this is doing, it's a lot of code, but all it's doing is setting stuff up, and then it's storing it in our global context. OK. So now we have our retrieve step. It's going to blow your mind, right? But it's going to retrieve. And so it does. We get a bunch of context, and then we retrieve that, and then we send it out as a retrieve event with our result from our retrieved nodes. Easy. Okay, step four, we evaluate, right? Notice, during the evaluation, we're going to set relevancy results as a global context. So we're going to set this in our global state for our workflow. That means we can leverage it later without needing to keep passing it forward in events, right? So it's a good way to provide access to downstream steps without needing to modify all the event over and over again. Next, of course, we're going to look at the actual texts, and we're just going to extract the ones that matter. We're going to join them up in a string, and then we're going to send that as our next event. Next, we're going to check it out, right? Get the relevancy results we set in our evaluation step for our transform query pipeline step, as well as our query string. So even though text extract event, which we defined above, and I'm not going to do the injustice of scrolling all the way to the top, but it didn't have either of these things. We can still access it through that context. And then if there's any no's in our relevancy, we're gonna augment with the external search tool. And then finally, we kick it off. This is just rag. This last step is literally just rag. So this is, all this code's available, but we broke it out into the markdown so you can read the comments about it. And then of course, we have the ability to draw all possible flows which gets us this beautiful looking graph uh you know we have the start event which can go to ingest and stop or prepare for retrieval and then to retrieve eval extract transform query and stop so this is the idea we have the ability to export this as a graph because we can simply walk through all the possible paths, trace them out, and connect them with beautiful arrows. How do we use the workflow? Well, remember when we talked about there being two start event nodes? One's going to do something if we pass in documents. One's going to do something if we don't. This is where we're doing that so you'll notice that when we start our workflow with just documents we're just going to get our index back that's what we designed it to do and when we run it with our query string and our index we're going to go through the whole process and these are all the questions that we we've asked from greg so i'll let you look through those in the notebook. For now, though, that's it for the build. Thanks so much for listening. Please don't forget to subscribe, ring the bell, et cetera, for YouTube. We're live every Wednesday, and I'll pass you guys back to Greg, who will close us out. Awesome whiz. All right. So we got workflows today, everybody. What did we learn about workflows? Well, we saw that they leverage an event-driven architecture that combines steps, Python functions, with events, basically keeping track of state. When we compare it to these sort of graph algorithms, like we'll see in Landgraf, algorithms, like we'll see in LandGraph. It's a more implicit approach, although we can't draw the graph ahead of time sort of programmatically, although we might want to from a product perspective. After a run, we certainly can. And remember, workflows are not agents. They work in tandem with agents. If you want to go deeper on some of the EU versus the US, join the community session today in just one hour live in Discord in our lounge with Akash Shetty. And with that, we'll get into some Q&A. All right. So we've got a couple of questions that have been coming in from the chat today. And let's see what people have upvoted in Slido. It looks like we've got one at the top. The question. Okay. So I want to sort of wrap together a couple of questions I'm seeing from the chat here. Okay. So workflows versus Landgraf. What are the key differences that you're taking away and which one should people start with? Yeah. I mean, at the end of the day, they're not super different. The way that we can think, they're not super different. The way that we think about building them might be different, the way that they're implemented are different, but the things they're doing is not very different, right? It's getting a, it's doing something and then passing it to the next instance of doing something, right? So these frameworks are there's the key differences are in the implementation are in the the way you're thinking about them uh which is to say when should you choose one versus the other choose whatever is most intuitive to you right choose whatever pattern of thought you're used to because you're going to be able to build the same thing at the end of the day. If you're doing more complex sub-graphing, so as in you want this workflow to take advantage of another workflow to take advantage of another workflow, I would maybe suggest using Langraph for that since Langraph graphs are able to be nodes themselves. So the way of thinking about that can be very easy. Otherwise they just do the same thing. Yeah. Okay. So on that point, we got another question in the Slido sort of when would I need to subclass a workflow? You kind of just answered that right in this multi-agent paradigm we're often building multi-agent systems as sort of sub graphs in a land graph context that's sort of my understanding of when we would subclass we would want to subclass a workflow but right now it's not clear how is that right we always subclass workflow because of how Python works. Workflow is the base class and then we extend it through subclassing it. Yeah. Okay. Okay. So in this case, I know the word sub is in both, but it's not referring to a sub workflow, right? It's a sub workflow. It would be an entirely separate entity. Okay. Okay. So yeah. All right. All right. We got to disambiguate some of the language here. So yeah, we'll continue to work on our disambiguation. Thanks for that question in the slide there. I want to get to the ones that were upvoted, even though I know we're short on time. So if you have to run, we totally understand. For telemetry, what is your recommended tooling for instrumenting workflows? I think there's like an out-of-the-box integration with Phoenix Arise, or sorry, Arise Phoenix. So that's Lama Index's recommendation. I don't know why I would go against that. Yep, Arise Phoenix. Okay. And then this one at the top, maybe just a comment. You know, debugging event-driven workflows can be non-trivial, especially in production environments. What tools do you think about when you think about doing debugging? We saw some questions on sort of debug. I don't have the IDE like Langraph just released for this event-driven architecture yet. How can I debug? What tools are you reaching for today? Arise Phoenix. I mean, at the end of the day, what we need is visibility into what's happening inside our workflow to debug it. Lama Index has built a integration with Arise Phoenix. That's the one they broadcast in their documentation. integration with Arise Phoenix. That's the one they broadcast in their documentation. That's going to be a good tool to use to do this. As far as like a Landgraf IDE clone, there isn't one yet, right? But for now, I would say just look through the tracing, look through what's happening at each step, ensure that you're using appropriate metadata when possible. And that's going to be the way that you're doing it. Yeah. Okay. All right. And I think we are just about out of time. If you have questions you really, really wanted answered, please to leave them in the comments section on the YouTube page. We'll go ahead and answer them for the benefit of everybody. Wiz, thank you so much for walking us through workflows today. And we'll see you back next week, Wiz. That's a wrap, everybody. So for today, Lama Index Workflows, the latest and greatest from Lama Index. If you want to learn more about Lama Index, core constructs, how to do agents, how to look at specialized tools they have like Llama Parse. Check out our YouTube playlist on Llama Index. We've done some really great events with them and we really value them as an event partner. If you're looking to have a really accelerated version of a comprehensive AI engineering training, then consider taking a look at our AI engineering bootcamp that we're always running. We're currently in cohort four. Cohort five will be towards the end of the year, this year, early 2025. Check it out. If you're interested, have any questions, let us know. If you're a company that has a team, if you're a leader, if you are ever in need of any sort of talent or support with proof of concept or prototyping, then reach out and let our team know. We've started to work more and more with teams in the enterprise, and we're looking to develop some more long-term relationships with folks trying to build out AI engineering organizations within their enterprise. If that's you, feel free to reach out directly to our team. We've dropped the link in the chat. And finally, if you have any sort of feedback on this event and ways we could improve it, please let us know in the feedback form that we'll drop. And that's it, guys. Until next time, keep building, shipping, and sharing, and you know we'll do the same. Have a great week, everybody. We'll see you next Wednesday. Bye, guys. See y'all soon.
Better Agents with LlamaIndex Workflows
Join us for an in-depth exploration of cutting-edge AI development as we dive into the best practices for orchestrating complex agentic applications in 2024! We'll compare pipeline, graph, and event-driven architectures, with a special focus on LlamaIndex Workflows. Discover how this event-driven approach transforms AI development, offering greater customization, debugging capabilities, and powerful visualizations. Whether you're a developer or AI enthusiast, you'll gain valuable insights and learn how to leverage LlamaIndex to build state-of-the-art agents. Don’t miss out—tune in and get your questions answered live!

Event page: https://bit.ly/llamaworkflows

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/gpLPoKPKyQZRmyxswr9tMb

Speakers: 
Dr. Greg, Co-Founder & CEO AI Makerspace
https://www.linkedin.com/in/gregloughane

The Wiz, Co-Founder & CTO AI Makerspace
https://www.linkedin.com/in/csalexiuk/

Apply for our new AI Engineering Bootcamp on Maven today! 
https://bit.ly/aie1

For team leaders, check out!
https://aimakerspace.io/gen-ai-upskilling-for-teams/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/iESG6zx2mfRM7NYd8