 Hey Chris, in our first head-to- head tool matchup, who wins? Laying chain or OpenAI assistance API? That's a tough one, Greg. That's a tough one. Okay. All right. All right. But does the API, the assistance API do what it says on the tin? Yeah, definitely. For sure. Okay. Okay. So you're saying we can actually build the same agent-like thing two different ways with two different tools that could be deployed to a production environment. And you're going to show us how? Thousand percent. Yes. Dope. Dope. I'm going to get everyone up to speed on agents, and then we'll get you back up here to rock out our first application. The LLM wizard and CTO at AI Makerspace, Chris Oleksiak, will be back to lead code demos real soon. Thanks for joining us on YouTube today. If this is your first time, welcome. Say hi to our community and share where you're tuning in from in the live chat. My name is Greg Locknane, and I'm the CEO and founder at AI Makerspace. Today, you'll learn about agents and the React framework from first principles. We'll cover the abstractions you need to build agentic retrieval systems in both Langchain and with OpenAI Assistance API. If you have questions now, feel free to drop them into the Slido link that's in the YouTube description box, and that we'll share with you in the chat now. All right, with that, let's go ahead and get right into it. Agents, that's what today is all about. We're gonna learn as we align our aims for the session about agents. What are they? How should we think about them at a high level? And how can we actually build them? What are the constructs that we need to build these things both in Langchain and using the OpenAI Assistance API, building OpenAI Assistance? Now, last week's YouTube Live, we said we were going to go a little bit deeper into the new stack and ops for AI. I've opted to not do that today so we can focus a little bit more on an in-depth discussion of agents so look for more llm ops after it's already in production content coming from us soon today we're going to really drive down into agents we're going to understand what they are how to think about them how to stack up our understanding of prompt engineering all the way up to agentic behavior in RAG systems. We're going to see how to do it in blockchain and open AI, and we're going to build agent-powered retrieval systems in both. Agents are pretty mysterious, and they're kind of actually mysterious, like in real life. Here's a quote from a book called Complexity. Agents exist in many fields. They exist in many different domains. And this is why this term was chosen for the way that we're building these things out when it comes to LLMs and AI applications. Agents might be molecules or neurons or species or consumers or even corporations. Molecules form cells, neurons form brains, species form ecosystems, consumers and corporations form economies. At each new level, we get these emergent structures and these emergent behaviors. Complexity comes from this emergence. comes from this emergence. This is why it's so important to think about agents as we set out to build more and more, ever more complex large language model applications. This is how we're going to build some really incredible things in the 21st century. And it's going to start from very basic principles with very simple rules. Stephen Wolfram's work comes to mind. And this is something that's been well studied for a long, long time. This is a book that's actually quite old and talking about many ideas that are much older. But let's zoom into today and zoom down to the level of the developer. Recall basic prompt engineering. But let's zoom into today and zoom down to the level of the developer. Recall basic prompt engineering. We're going to build this up step by step. Prompt engineering and the best practices of prompt engineering go something like this. You want to make sure you're always providing very clear, very specific instructions. You want to give as much context as you can. You want to provide the LLM with information about what you want it to act like, its role, its persona, how it's feeling today even. We want to use additional context retrieved in, for example, RAG systems to put into the context window. And as we give inputs, as we give examples, as we give one shot, two shot, few shot examples, we want to use those examples and even expand on them. Think through them. Use chain of thought reasoning to actually improve our outputs. And our outputs, outputs and our outputs in many cases need to be specified especially especially in agentic applications because when we're going and we're picking up different tools along the way we need the output of an llm to be the input of another tool this is where formats that are industry standard for hitting APIs like JSON come into play. Digging a little bit deeper into prompt engineering, recall zero-shot prompting is simply instructions only. This is just like talking to a human. This is saying, hey, I'm going to tell you what to do, young human. And I'm not going to tell you any examples. I'm just going to tell you exact instructions of what I'm hoping you'll do for me. And this works pretty well, even with very young children. This is eventually where we want our AIs to get. But for now, you know, it works a lot better when we're engaging with these tools. If we give one example or if we give it many examples, let's say between 10 and 100, we can get better and better and better output and better results. How we can take it even to the next level is we can say, hey, rather than just giving an instruction or providing a few examples, we can provide thought through examples. Instead of a standard prompt, we can provide a chain of thought prompt, or we are providing the reasoning steps that we took to get to the answer. For instance, this example here is the classic example from the paper on chain of thought reasoning. Roger has five tennis balls. He buys two more cans of tennis balls each can has three tennis balls how many tennis balls does Roger have well Roger started with five two cans of three each that's six five and six is eleven the answer is eleven this is the idea if we provide this thinking through this step-by-step thinking through, we can get better outcomes. And if we provide this step-by-step thinking through, we can get better outcomes in our agentic-like applications as well. Now, what if we're not sure how to think through it? Well, what do we do with humans? We say, well, Johnny, that's not the right answer, but can you think through what the right answer would be? We ask Johnny to refine his answer. This is the idea of self-refinement, and LLMs can do it too. We can ask a large language model, well, that's not quite right. Can you think through that step by step? Here's an example from the paper that introduced this idea of self-refine. The user says, I'm interested in playing table tennis. The AI responds, I'm sure it's a great way to socialize. Stay active. The LM is small talking about table tennis. That's not really what I'm looking for. I'm looking to be engaged and understood. I want the LM to sort of have some user empathy here. So if we sort of provide that feedback, we can get a refined response. Hey, that's great to hear. It's a sport and this is what you need to know about it. And this is how you might play it. If you're looking to get into it, this is probably important information for you. And in this way, we can get better and better outputs. We can think through, step by step, what's going on. This brings us almost to agents. If we think about agents in prompt engineering, it's a nice gateway into agents in langchain. So agents in prompt engineering we can think through as being sort of self refiners. We can ask them, think through the answer that you just gave me step by step. Are you sure? Is that the best you can do? An example is you might say, instruction, you are a Python programmer. Write some code to solve this particular programming problem. You might get an output and you might respond. Can you optimize this code to have a lower time complexity? You might get a response, might be lower. Then you might ask, is this the best you can do? And you might get an even better response. Programming examples also were shown in the self-refined paper. For example, this is a way you could generate a sum of one to n. But this code is slow because it's a brute force solution. We can tell the LLM that, and we can get a refined version. This brings us to the point where now we provide some examples, we think through them, we could even ask the LLM to think through for us in a self-refined manner, to reason through what to do, and then to actually do it. Combining reason with the generation is what we see here, but we can do this even more automatically and we can do it even more generally by combining reasoning with action. And this is the idea of the REACT framework. React framework. The React framework from the React paper, this is the example that they provide. So let's go through it. Initial question, initial prompt. Aside from the Apple remote, what other device can control the program Apple Remote was originally designed to interact with? So we're looking to control this program without the Apple Remote. Standard prompt response, iPod, wrong. Thought through, reasoned response. Let's think through this step by step. Yes, let's. Apple Remote was originally designed to interact with Apple TV. Apple TV can be controlled by iPhone, iPod, etc. Therefore, must be iPhone, iPod, etc. I like how you thought through that but still wrong, right? Now, what do we do as humans? We're often getting on Google, searching, researching over and over, right? We might search for the Apple remote. We might find some information. Then we might find that we need to find the front row software. We might search for front row software eventually getting there this is the green here but we might not get exactly where we're trying to go unless we dig pretty deep into the front row software manual but if we combine the reasoning and the action into one holistic process, you might have a thought after the initial question. Well, I got to search the Apple remote to find which program it was originally designed to interact with. So I'm going to find that Front Row Media Center. Great. Now that I've found Front Row row media center i need to search front row and find what other device can control it fantastic searching front row searching front row software and alas we find you can control this thing with the keyboard function keys on your apple device boom got it reasoning action for the win. So what we're seeing here is we're seeing while we can use standard prompting and we can move to chain of thought prompting, this reasoning step is very powerful as is the action step, the brute force search that we're all so comfortable with, Googling things, right? It's the meta skill. But if we combine reasoning and action, we can get better and better and better. This really truly is the art of googling as a human. And so agents are automating this process. That's what we're doing with this reasoning and this action. So let's get into how we actually do this in tools. how we actually do this in tools. Line chain is up first. Line chain agents, they'll tell you straight up. They use the LLM as a reasoning engine and that chooses which sequence of actions to take. Agents are always going to have access to tools. So the actions it can take is it can pick up different tools. For example, it might pick up a Google or a DuckDuckGo search. It might get access to a repository of research papers like Archive. There are lots of different tools you could imagine getting access to. Maybe it gets access to your database or the weather outside today. There's many APIs you could imagine connecting an LLM to. There are also different types of agents in Langchain. The easiest one, the most general purpose one, is the zero-shot React agent. I think it's enough said, right? Each step of that reasoning action, right? That was zero-shot, just using instruction only. And the OpenAI functions is a different type of agent as well. This will kind of come into play today. Now, in June, OpenAI released function calling. They said that GPT 3.5 Turbo and GPT-4 are even better at outputting inputs to other functions, other APIs. And of course, Dev Day, they released JSON mode, which is even better at doing this. JSON mode specifies the response as a JSON object, and it provides a more detailed instruction at the system level. So we can be more comfortable, more confident that we're going to always return a JSON object. Okay. How do we build a RAG system with agents? This is going to get kind of meta, so stick with us. You've probably seen this before if you've been watching our content. This is a basic retrieval augmentation system. The idea is you ask a question, provide a query. That query gets turned into a dense vector representation by being passed through an embedding model. That dense vector representation is fed into the database where we look for similar dense vector representations when we find similar things to our query we put them into the context of our prompt so we set up our prompt template we say use the provided context to answer the user's query. If you do not know the answer, cannot answer, please respond with I don't know. So we take all the relevant context we found that was similar to our query and we put it into the prompt. All of that gets shoved into the LLM and out pops our answer. Now, if we want to build an agent-like system, we might only be concerned with the input and output of this RAG system. So let's abstract now. Let's turn this into a single system component. This retrieval augmented question answering system, we're going to turn into a single Lego block. All right. When we have this as a single tool in our tool chain, we can build more complex things. This is where we can start to visualize what the React framework is actually going to look like in practice. Here's an example where we have two different retrieval augmented generation systems here and here. One is built with a Barbie reviews index. One is built with an Oppenheimer reviews index. In this case, if we want to ask, hey, do Barbie and Oppenheimer share similar themes? We might need information from both of these systems. But let's check out that we need a reasoning agent and an action agent, React, right? Have a conversation with a human answering the following questions as best you can. You have access to the following tools. Tool one, tool two. These tools. Begin. The query comes in. The agent scratchpad is used for reasoning, the action agent and the agent executor executes the actions that are reasoned, and it goes to the appropriate tool in the chain. That output is fed back into the reasoning engine, them and we may continue on with additional reasoning steps or we may output an answer all right so briefly let's review langchain some of you may not have seen this before in detail langchain is all about combiningLMs with other sources of knowledge and computation, other tools in the chain, let's say. Langchain, if you look at their docs, number one is all about connecting it to data, connecting these LLMs to data. And number two, check this out, being agentic, connecting to tools, connecting to the environment like an agent. So the core concepts we need for this, we're going to use our chat style model. We've seen this before. Particular importance for today's discussion, we're going gonna focus on the user message and the assistant message. The user message and the assistant message come into play when we use the OpenAI assistance API, little foreshadowing there. But this is necessary in Langchain as well. Our prompt template, we're gonna use that in a number of places, including in the actual reasoning engine, as we saw. So we've got some prompting to set up. Then we're going to develop our chains. built Langchain expression language rather than the previous way we have shown in events to build these out. L-C-E-L or the Langchain expression language is important for you to start learning because not only is it an elegant and nice way to compose chains, but it also is purpose-built to be able to integrate with LangServe and LangSmith, which we plan to cover in upcoming events. Finally, we need to connect this thing to a vector database, vector store, and index of our data. So in summary, we're going to connect models, prompts, chains, and an index. And we're going to wrap all of this in an agent. So the pattern here that we're looking for is thought. I don't know the answer to the user's query. The action is to figure out which tool to use. Well, ask the RAG, ask the question answering system. The action input is to execute that question. The observation is based on the output. Did I find the answer? Well, maybe I'm not able to locate the answer within the actual data and RAG system that I built. So if the question answering system is insufficient, maybe I'll go ahead and I'll leverage a backup Google search, or today we'll see we'll use DuckDuckGo. Executing that Google search maybe allows us to find the context we need to get the answer we want, finally, as an output. So we're gonna combine a couple things today number one is going to be a sort of a meta rag system this is basically setting up rag to learn about rag we're going to search the top archive papers for rag and we're going to set up this as one Lego block. You may have seen this before. It's something we've shown previously. But the details here are not the point of today. This is the point. We're going to set up a query that goes into a reasoning agent and provides the action agent with the decision to make go to archive or go to DuckDuckGo. Which one? And this is going to allow us to answer some pretty cool, some pretty interesting questions using all the tools that we've talked about so far today. And with that, we're gonna see how we can chunk this thing out into specific bits of code with Chris, the legend himself. We'd like to welcome back to the stage. Are we gonna- Hello, I'm excited to walk through how we set up the flow in Langchain. So the first thing we need to do whenever we get into code is grab some dependencies. So we're going to do that here. You'll notice that we need this Langchain hub. We're going to use DuckDuckGoSearch. We're going to use Archive. And then we have to get some basic dependencies that are part of OpenAI and Lanchan. You will need an OpenAI API key for this. And we're using the GPT-4 Turbo model as an example, though you can substitute it with GPT-3.5 Turbo if you don't have access to GPT-4 yet. So the first thing we're going to do is set up our LLM. We're going to set up as a chat OpenAI model. We're just going to give it the standard zero temperature. We don't need creativity here. We just need it to understand context and then create an answer based on context. And then here, we're going to use our GPT-4 Turbo preview model. You'll notice that we need to use our messages template. So we're going to test it out, make sure it's working by sending the classic, hello, how are you? And receiving the very, you know, robotic, I'm just a computer program, so I don't have feelings message, which is great. So that's our LLM set up through Langchain. Thanks Langchain, So that's our LLM set up through Langchain. Thanks, Langchain. Makes it very straightforward. Then we're going to load the tool belt. Now, you'll notice that we're using this load tools method from our agents. And we can just specify the tools that we need. So we're going to specify that we need an archive tool. And we need a DuckDuckGo search tool. We're going to pass in our LLM as the LLM that powers each of these tools. And that's all we need a duck duck go search tool we're going to pass in our llm as the llm that powers each of these tools and that's all we need to do to set up our tool belt in langchain then we're going to create our agent using the lcel format first thing we need to do though is grab a prompt so you know as we saw as we saw greg, the kind of idea here is that we have this prompt that's going to let our LLM kind of, you know, simulate reasoning and then simulate tool selection and then, you know, additional steps as well. But the idea is that we need some kind of prompt that lets the LLM do this. And so we're going to grab it from the hub. We're going to use it from Harrison himself. And this is the React JSON prompt. So this is the reason action prompt. It uses this JSON format due to the fact that we're using the chat model. And you can see the prompt here, right? Answer the following questions as best you can. You have access to the following tools. And then we're going to create a list of tools. The way you use the tools is by specifying a JSON blob, specifically this JSON, and it goes on and on. But the idea is this is how we let our LLM make a decision on which tool to use, and that's what gives it that agentic behavior. Now that we have our prompt, we need to inject the information about our tools into it so it actually knows how to use the tools. So we're going to do that by rendering a text description, and then we're going to add the tool names as a list, and that's it. So the idea is that we now have a prompt, and that prompt is going to contain our tools, what our tools can do, and so on and so forth. And we're good to go. Before we go to the next step, though, we want to add this stop behavior so that when we see that observation flag, we stop generating. We don't need to generate past that. The idea is that this is where we want to conclude our generation as we're done, you know, we're ready to move on to the next step. Now we're going to use the LCEL format to build this agent chain. You'll notice that we have a single object here, which is going to be a bunch of different things separated by the pipe. The idea about the pipe is that it's going to essentially act as an operator that moves things from one to the next to the next. The idea here is that we take our input and we pass it along into our prompt. We also have our agent scratchpad. And our agent scratchpad is going to be where our LLM can record its quote-unquote thoughts, right? And then we're going to populate our prompt with both of these objects, the input we get from us. And then the intermediate steps is generated through the LLM as it makes subsequent calls to itself. as it makes subsequent calls to itself. LLM with stop, this is just saying once we have our prompt, which is formatted with our user's input and the agent's thoughts or the agent's scratchpad, this prompt will be populated with that information. Then that will be piped into our LLM. Our LLM will produce a text response, and then that will be piped into our React JSON single input output parser. All this means is we're going to get a response in JSON, and we're going to convert it or parse it back to a format that we desire. But the basic idea here, right, is straightforward enough. We take our input, we format a prompt with it, we pass that format a prompt to our LLM, and then we parse the output. All we need to do now is wrap this in our agent executor. It's awesome that the LLM can make decisions about what tools to use, but we need to actually give it access to those tools or a way to call those functions. And that's what our agent executor is going to do. We're going to pass our agent brain, our tools. We're going to set verbose equal to true so we can see all the outputs. And then we're going to let it handle any parsing errors that might come up due to incomplete or malformed JSON. Once we've done that, we're good to start calling it. So you can see here we can pass it an input. That input is, what is retrieval augmented generation? You'll notice that we enter our agent executor chain. The agent has a thought, retrieval augmented generation is a concept of field natural language processing machine learning. It goes on and on. It decides on an action, and that action is as follows. It's going to query the archive tool with this particular query. It's going to get a bunch of responses from the archive tool. It's going to use those responses to synthesize another thought. The thought is going to be the articles found provide a good overview of, and it goes on and on. It says with this information, I can now provide a comprehensive explanation of retrieval augmented generation. And then it populates a final answer. So to go over those steps again, just to be clear about them. The agent initially receives the query. It makes a decision about which tool it should use or if it should use the tool at all. It decides that it needs to use the archive tool, gets a bunch of information from archive, figures out if that's enough information, figures out that that is enough information, and then populates the response. And that's the idea of this system. Now, the thing that makes it agentic or the thing that lets it be more powerful than just being a rag in a box is we can ask questions like, who is the current quarterback of the Denver Broncos? And you'll notice here that we enter the same chain, but the thought is that it should use a DuckDuckGoSearch. And so we wind up using a DuckDuckGoSearch with the query current QB of the Denver Broncos. We get the information we need. It says, I have the information we need. And then we get the final answer right here. So the idea here is that we don't have to tell it which tool to use. We just have to provide all the tools and then query it. And the LLM agent is going to choose the correct tool. And that's going to save us a lot of time. We don't have to hard code anything. And we can have our LLM have access to a broad suite of really powerful tools. So that is how we set this all up in Langchain. I'm going to send you back to Greg, and we're going to learn a little bit more about the OpenAI Assistance API before we see it in code. Yeah, awesome, Chris. So that's how you do it with the classic tool. How about with the new guy? You know, sort of traditional tool we just saw, right? That's how quickly these things are moving. Well, let's take a look at OpenAI's assistance API. What we're going to do here is we're going to build an agent-like application, and we're going to use tools. Now, you may have seen you can use Code Interpreter, you can use retrieval, you can use function calling. We're going to do a bit of a retrieval process here, but we're going to choose to leverage function calling for it. So this is going to be pretty cool. And Chris is going to walk us through the details. Sometimes it might make sense to go ahead and just actually use a function call to a repository like archive rather than uploading your own data. And so to build the same system in OpenAI assistance, we're going to need to know a couple of different constructs that add on to things we may have seen before. we may have seen before. First of all, the definition of assistant for OpenAI is a purpose-built AI that uses, of course, OpenAI models, can access files, and maintains persistent threads across the entire interaction. Of course, you can also call tools, as we've heard about many times so far today. A thread is simply a conversation. Everybody knows what a thread is, right? I mean, a thread is just opening up a convo. But here, a thread is between the assistant and a user, you and the AI, let's say. And storing that message history, that chat, that thread is the point of the assistant here. Because remember, when you hit the OpenAI API as a developer, you need to specify what you're putting in. You need to leverage the system level role to provide those instructions. And you need to leverage the user and assistant level roles. Oftentimes we're leveraging it in a context like this, where we're providing the assistant level and the user level inputs. This is an example of how we might provide who won the World Series, the Los Angeles Dodgers won the World Series. We're providing a question and an answer. We're providing a one-shot example of the way we want to engage. But here, the assistant is not us acting like the AI. It's actually the AI engaging with us. It's our assistant and so the assistants flow here is we create an assistant by sort of defining the specific aspects of it which tools it needs to connect to then we create a thread we add messages to the thread and as we're running the assistant on the thread, we trigger responses. This is what calls the tools and allows us to do the same thing we saw how to accomplish in Langchain. So the flow is something like this. We set up the assistant where we have our name, our instructions, our tools, and the model we're going to use. Then we kick off our thread, and we then run the assistant on the thread. To see how this is done and actually implemented line by line at the syntax level, head back to Wiz to show us how to get her done. Back to you, Chris. Hey, thanks, Greg. I'm excited to walk through how we do this in OpenAI. how we do this in OpenAI. So the idea is pretty straightforward. So basically we're doing exactly what we did in the Lightning Chain with a few extra steps, but we're still doing exactly what we did in the Lightning Chain, right? So the idea is first we need to set up some kind of system that lets us interact with our external search tools. In this case, DuckDuckGoSearch and ArchiveSearch. So notice that we set up some functions. The functions will take a string, and they will return a string. So you can see here, we can ask things like, who's the current captain of the Winnipeg Jets? And we get a number of responses about the fact that it's Adam Lowry. Then, same with archive search. We can send it a string, retrieval augmented generation, and we get a string, which is just a list of summaries from the papers most relevant to that query. Okay, so that's awesome. Let's look at this DuckDuckGo search function a little bit more. So it takes one parameter, query, which is a string, and it returns a string. And we can use it to search the Internet. All right. In order to let OpenAI call these functions, we need to provide it the appropriate format with which to call these functions. Because it doesn't have our Python functions loaded on its side. So it has to send us an output that we can use reliably to call our functions, the function calling API, right? Now we're going to use this format to create these function objects. So we'll give our function object a name, which is going to be dot dot go search, a description, answer non-technical questions, do not use this tool for questions about machine learning. We're going to give it parameters. And we know it takes one parameter, and that's called a query. We're going to say the type is an object. It's a query object, so that's its name, it's going to be a string, and here is an example of that string, right, the search query to use, for example, and here's a small example. And we're going to say that this query is required. We only have one parameter and we, it's not optional, there you go. So this is just a way to address the function we want to explain to OpenAI how we need to call it, what we need to call it with, and what that should be. Once we have this, the important thing we need to do is provide this description. Now that description is going to be what lets us choose when we call it. And when I say lets us choose, I mean, of course, OpenAI. So this natural language description is what will guide the LLM toward the correct function. So it's very important, right? This is like a prompt engineering opportunity. We'll do the same thing for archive, but we'll make it answer questions about machine learning. And we'll do the same thing as we did before, where we give it a description of the parameter, what it should be and how it should look. Now that we've done this, we can create an assistant. So we're going to first create our OpenAI client. This is going to use the OpenAI key we gave already. Then we're going to create an assistant. We're going to give it a name, which is Query Assistant. We're going to give it an instruction. The instruction is you are a personal assistant. Use the provided functions to answer questions. Then we're going to give it some tools. Those tools are going to be the functions we created. You'll notice, though, we're not passing the actual function. We're passing the object associated with the function. This is because OpenAI can't really call our function because it doesn't have it. So we're going to give it the archive function and the DuckDuckGo search function. We're once again going to use GPT-4 Turbo. Then we're going to grab our assistant ID. This is the ID of our assistant. Makes sense. Then we're going to create a thread. Now, Greg already explained to us what threads are. Basically conversations. Within this thread, we're going to create a message. And that message is going to be a user message with the query, what is the capital of Ottawa? You can see that we're going to, from the created thread, we're going to reference the created thread for our message. We're going to give it the role of user, and we're going to give it the content, which is our query. Now we have an assistant, and we have a thread. We're going to use our assistant with our thread, so we're going to run or create a run on our thread that leverages our assistant. So you can see we specify our thread ID that we created above that has this message, and then we're going to use the assistant that we created above through the assistant ID. Once that's done, we're going to need a helper function, which is taken from a helpful repo. Link is in the notebook. All this does is wait to retrieve the run until it is done. So it will just loop forever until it is either errored or completed. You can see here that when we wait for this run to complete, all we need to do is pass the run ID as well as the thread ID. Remember, we have this idea of a thread and we have this idea of a run. So we need to keep track of both of those as we move through our system. Then we have this submit tool outputs helper function. The basic idea here is that our response might sometimes be in the format of requires action. Requires action means the open AI has determined we need to call a function here, right? So it's going to say, it's going to deliver to us a blob we can use to call our function, but we need to get that response in order to continue. The idea of this tool is that we're going to get the function name and args from our response. We're going to call our function depending on which one we have, which one OpenAI has determined we need so either duck duck go search or archive query and then we're going to append these tool output array objects with the tool call id and the output from our function so all this is doing is taking that blob from OpenAI, calling our function on it, and then sending that output into this object. Once we have this object, we can use the threads.runs submitToolOutputs method, which is going to let us send a tool output array along with our thread ID and our run ID. And that's going to continue that conversation, but with the results of the function call. Once we have that, we can wait for the run to complete, at which point we'll receive our natural language response, which we print with this helper function. at which point we'll receive our natural language response, which we print with this helper function. So let's look at a function here that does all of that in one box, just to make it a little bit easier for us. We're going to first create a thread. Then we're going to create a message on that thread with our user's query. We're going to then create a run, and that run is going to be executed on that thread with this assistant. We're going to wait for it to complete. If we require action, we're going to submit the tool outputs back to our assistant, and then we're going to wait for that run to complete, and then we're going to pretty print everything at the end. So the idea, just like Langchain here, is that we have this decision-making process with this information retrieval step, right? So when we first execute, this is OpenAI, the LLM making a decision, should I use a tool or not? If it does use a tool, it sends back to us that we require action. We use the tool to collect the relevant information using the query that it generated for our specific tool. And then we use that to synthesize a natural language response to our original query. So it's the same system, but it's using OpenAI's tools instead of using that lang chain infrastructure. We can do things like ask, what is QLora? You can see that it's consulting archive. Once it's done, we get this QLora definition, which is a great explanation of QLora. We can ask, what is a meme? You can see in this case, it doesn't consult. Archive, it consults DuckDuckGo. And we get a response that is an explanation of a meme. And so that's how we build that same thing we had in Langchain with the OpenAI assistance API. And with that, we'll kick it back to Greg. API. And with that, we'll kick it back to Greg. Chris, that was awesome, man. And super cool to see how we can do retrieval through function calling. Sort of meta and pretty next level. Absolutely loving it. Today, in conclusion, we learned about agents. It's all about these simple rules access to these simple tools providing the complexity of future lm applications and that emergent behavior that we're inevitably going to see as we continue to build ship and share these more and more ever more complex ai systems agent like behavior is all about automated reasoning re and decision making or acting act the pattern here is thought action action input thought, action, action input, observation, and thought once more as we go through the reasoning action loop. It's easier than ever to build and deploy these things. We saw how to do it in Langchain with the React framework. We saw how to do it with OpenAI's assistance API. And that brings us to the Q&A portion of today's event. I'd like to welcome Chris back up on stage and I'd like to leave you all with this meme that we shared a while ago on our LinkedIn. This is a great way to think about it. Batman's got the tool belt. He uses his head to decide which tool to pick up. This is kind of what we're doing at the end of the day with agents. And finally, thanks for jumping in there, Adam. Now, there you have it. You saw it for yourselves. Who wins? What do you think? Adam thinks Langchain. Drop your thoughts in the live chat or in a YouTube comment. Langchain versus OpenAI assistance, our first ever tool versus many more to come. All right, let's go ahead and jump into the Q&A here, Chris. First question of the day, and go ahead and drop these in Slido if you have more questions. Can we build our own agent from scratch? Sure. I mean, it depends on how you mean that, but I mean, sure. Yes. You could build your own agent from scratch. If what you mean like is your own agent framework, that's kind of a research problem, so it's up to you and your team. But if what you mean is like any agent that does a specific set of tasks, then yeah, I mean, both of these approaches we saw today would be helpful to get started on your journey of creating an agent. Okay. Okay. Let's crank through some of these open source models. Cool too. Is this possible to do with like a Lama two 70 B and a react agent? I'm going to say that that is not a firm yes, but it's a pretty good yes, with some fine tuning and some careful consideration. You can definitely allow systems like Lama 2 and with some guardrails to ensure that tokens are being generated properly and that everything's in the right format. Yes, the brain piece is actually not the issue. It's just getting the specific syntax and formatting locked in. That can be a little bit more tricky with the open source models. But there are lots of methods that would allow you to do it. It might not be as out of the box, but if you're already using Lama two seven B, you're used to not straight out of the box experience. So you're already using Lama two, seven B you're used to a not straight out of the box experience. So. Okay. Okay. So what's really going on with the brain making decisions is there's actually a little bit of instruction, a little bit of prompt engineering behind the scenes that maybe we don't see when laying chain or open AI abstracts. Is that kind of what we're getting at here? Yeah. I would say like the thinking part is no problem. It's being able to ensure that it outputs its thoughts in a consistent and standardized format. That's a little bit less consistent on the open source models. But again, through some fine tuning or through your methods that guide token generation, you should be able to set it up so that it's much more reliable. And next question, I believe we did sort of cover this. Maybe you can click in on it a little further, Chris. Is an agent able to search through different RAG systems, like different vector DBs, different sets of data in reaction to a single query? Oh, yes. Hopefully we saw some examples of that. The idea is depending on your actual query, it's going to select the right RAG system. Now, you might ask a question that is relevant to both halves, right? And language chain will actually use both halves. Or say you have 30 different repositories of data that you're trying to synthesize a response from. It's going to intelligently, intelligently in quotes, select all of the correct systems. And that could be one or more than one. And it's going to be relevant to the query. Yeah. Okay. Here's a good one. Here's a good one. Okay. In the React framework, where do the thoughts come from? Where do those actions come from? Where's it coming from? Where's it all coming from, right? I mean, it's the LLM. So the LLM is going to generate a response, just like you see when you prompt chat GPT. If I ask chat GPT, hey, chat GPT, I have a spoon, a fork, and some soup. What should I use to eat the soup? Chat GPT is going to respond with, you should probably use the spoon, my guy, right? So that's the thought. That's exactly the process of how it's generating the thought, right? And then the action is on our side, right? It says, hey, I think you should use spoon. And then we say, call spoon function then. And then we return that response to the LLM, which is going to carry on with more reasoning. And then we're going to be able to take discrete actions based on that reasoning. on with more reasoning, and then we're going to be able to take discrete actions based on that reasoning. Eat the soup with the spoon now, right? Yeah, of course. Of course. Yeah. Okay. So next up from Lax, we have which of these approaches is better in terms of time of execution, tokens used, and hence cost, and accuracy. I think he's referring to Langchain versus OpenAI assistance. What a good question. It's difficult to answer succinctly or concretely. What I'll say is that if you can, so it's kind of like the time to lift off with Langchain is a lot less, right? The time to lift off for the open AI systems, there's a lot more engineering you have to do. And we didn't get to the point where it can re-query itself here. We would need to do additional engineering for that. But the idea is that time to lift off is very low with laying chain. But the open AI assistant has an advantage, which it's likely to use less tokens overall. So it might be less costly. There's nothing to say that has to be true. It's not a rule. But you can, because we're using this functional API paradigm versus the reasoning action paradigm, it's, you know, oftentimes the assistant or OpenAI function calling methods can be a little bit cheaper over the lifetime of your application. Accuracy, they're both, if the source model is the same, it all comes down to what you have written as the prompt that's going to determine accuracy in both cases. Okay. Okay. But can I fine tune? Next question. What do you think about this? Not with the open AI assistance right now. I don't believe that it's leveraging fine tuning. Though that feature is supposed to be coming soon, as I understood from Dev Day. I don't believe it's currently active. I could be wrong, so I'll double check, and then I'll post a comment if this is incorrect. But as of right now, I believe the fine tuning option is not available through the OpenAI Assistant Suite. If you use Langchain, you could use an open source model, in which case, of course, fine tuning is fully available. You can also use just the OpenAI fine tune endpoints if you've fine tuned GBT 3.5 Turbo. So, I mean, the answer is yes. For Langchain, for sure. For OpenAI assistance, it should be coming soon. Coming soon. So this is an interesting question. What are the best small LLMs to perform simple reasoning, like to decide which tool to use? Is there one? I'm going to hot take here for a second. If it's a really easy binary decision, you could probably just use a classifier model, a simple classifier. You know, you don't even have to get as big as your, your BERTs, right? So something like Distilbert, whatever, even smaller classifiers. In the case of more complex behaviors, something like, I think Zephyr or models in that 7B and Structune range are going to be the best models to use. Going less than that is definitely possible. But again, we run into this issue of having these consistent outputs. The actual thinking or reasoning skills of some of these larger models is also better. So I would give as like a generic should work most of the time response. I'm going to say like the Zephyr suite of models is where you should be looking right now. Hot take. Distill Roberta for your reasoning engine. Yeah. Yeah. Love that. Oh, binary only. Binary. Okay. LLM as a judge, which is the better choice? Lane chain evaluator or rag assessment? Ragas. A little bit off topic and off the trail here, but we're going to go. I'm going to hot take here again. They're the same thing. You know, it's like, do you want a yellow bus or do you want a zone bus? Right. It's the same thing. It's just a different word for the same thing. I mean, the LLM as a judge features are largely come down to the prompt, the prompt style. Both of these ragas and langchain evaluators, can be modified and customized. So at the end of the day, it's going to make an API call with a certain set of information to the LLM with a prompt that you provide. So they're similar. If you're looking for the best out-of-the-box solution, if you are in the Langchain ecosystem, obviously staying in it is going to be easier for you. If you are not in the lane chain ecosystem, just go with Ragas. It's very good at what it does. Okay. Okay. One more that's a little bit outside the box here, but let's go ahead and hit it anyways because it's the last one on the Slido. How to handle latency for text-based models, for multimodal? Async is your friend. It comes down toodal. Async is your friend. It comes down to a lot of async is your friend. Unfortunately right now, latency is huge on everything all the time. The assistance API does help with that through this idea of threads. We have many asynchronous threads running. We can pile up more and more threads. For multimodal though, I mean, you're just, you have to wait. At some point, you have to wait for a very large piece of compute power to do a lot of computing. And so latency is always gonna be a problem, but I would say whatever you can parallelize, you should, and then whatever you can do asynchronously, you should. Wiz, all right, man. Close it out. Thank you so much, dude. If this resonated with you today, thanks for coming. We hope to see you again next week at our event on Transformers. Then we're going to be doing a super deep dive for like three hours, soup to nuts, Transformer from scratch. We thank you for joining us today for sure. And if you want to go deeper on these topics, the other thing that you might think about next week is we're launching our cohort three of our LLM ops, LLMs in production course, where we're going to dive into all of the AI engineering best practices. You're going to be able to get in there and build your own agentic RAG application, and you're going to basically be a pro at the systems we covered today by the end of it, by the time we get to demo day. If you have questions, feel free to throw them in the chat, throw them in YouTube, or reach out to us in response to any direct mail, and I'll get right back to you myself. Other than that, please share any feedback you have on today's event. We're going to drop a feedback survey in the chat. We'd love to hear from you. How can we improve? What would you like to see next? We're going to stay on this Wednesday vibe, and we're going to start to be a little bit more consistent with the 1 p.m. Eastern, 10 a.m. Pacific each Wednesday. We wish you all a really happy Thanksgiving tomorrow, and we really look forward to seeing what you build, ship, and share in the AI Makerspace community and beyond on Discord, on LinkedIn, in YouTube in the future. Thanks so much, everybody. We'll see you soon.
Agents: LangChain ReAct vs. Open AI Assistants API
GPT-4 Summary: 
"Discover the Future of AI: LangChain & OpenAI's Latest Tools for Building Complex Applications" - Join our must-watch event for AI enthusiasts and professionals! We'll dive into the innovative world of Agentic apps, exploring LangChain's breakthroughs in data-aware applications and Chain-of-Thought reasoning. Get an exclusive look at OpenAI's Assistants API, revolutionizing the development of agent-like applications. This event is a game-changer for LLM Operations practitioners, aspiring AI engineers, and builders eager to harness the power of LangChain and OpenAI's Assistants API. Don't miss out on learning how to create cutting-edge AI systems, with all demo code provided for you to build and share. Click now to unlock the secrets of advanced AI applications!

Event page: https://lu.ma/Langchainvrsopenai

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/np3mh88VswLrRYsMpifeon

Speakers: 
Dr. Greg Loughnane, Founder & CEO AI Makerspace.
https://www.linkedin.com/in/greglough...

Chris Alexiuk, CTO AI Makerspace.
https://www.linkedin.com/in/csalexiuk/

Apply for the LLM Ops Cohort on Maven today! 
https://maven.com/aimakerspace/llmops

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/az6cQrPNoc7f7cgy6