 On-prem agents, Wiz. Let's go. What a great way to round out 2024, huh? Yeah. Who would have thought this is where we'd end, you know? That's right. That's right. You know, it's really cool. We saw some evolution happening, even within the last week or so from LangGraph and from the folks over at LangChain. And we've actually got some new things that we're going to cover today, including LangGraph platform that actually has LangGraph server implemented, which is a little bit different than LangServe, right? Yes, we have entered the platformification era. And so the names be changing, but ultimately, it's the same technology we don't love, just bundled in an easier package, a prettier present for the holidays. That's right. That's right. That's right. So it's still serving the Langs we love. It's just a little bit more, let's say mature heading into the next year, something like that, right? A little bit more production ready. Yeah. A little bit more enterprise grade. Excellent. Excellent. But there is a free version and it is open source and that's the one we're using today. Isn't it so? Yes. And it does the thing and we're going to see it do the thing. And that's basically all you want. All right. And we're also going to wrap up the year with talking about agents and agentic rag and rag and how you might think about these differently in an on-prem context. I'm excited for today. How about you, Wiz? Last one of the year. Let's go, baby. Can't wait. I can't wait. Let's go. All right. We'll see you back in a bit as we kick off discussion. Let's go ahead and rock and roll with on-prem agents today, guys. If you have questions along the way, if you have comments, throw them in the chat. We'd love to hear from you guys, especially during the discussion components. All questions will be answered at the end if you put them in the Slido link. So please do use the Slido to put in questions and upvote your favorites. We'll cover the latest and greatest from LangChain and LangGraph today. We'll talk about agents and how to build them on-prem, what kind of requirements you need. And we'll also discuss some of the differences between using an approach that we get from Langchain versus using an approach that we get from like a Llama Index. So this is going to be a great way to wrap up the year. Let's get ready to rock and roll with on-prem agents. All right, so as we kick off the old sesh, let's make sure we understand exactly what it is that we're aiming at. We want to understand on-prem and we want to understand agents. We've covered these before, but we're going to give a quick review and primer here. We also want to understand the LandGraph platform because this is sort of a new structure. It's an evolution that's taken place over the last year, and it's important to stay up to date on what's going on right out at the LLM edge, especially as these tools and technologies like LangGraph and Langchain mature. We're going to discuss, because it's quite interesting actually, agents versus agentic rag versus rag in an on-prem context, because the compute and memory requirements you need are actually different depending on what exactly you're looking to build and do. Then we're finally going to build an agent research assistant. Shout out to Langchain folks for putting one together right in time. So we're going to go ahead and stand on the shoulders of giants a little bit with one of their recent repos they put out this week and give our own flavor to it. And then we're going to discuss some of the latest releases they've had this week to sort of close out the sesh, including Langraf Command and Langraf Inter interrupt and give you our two cents so with that let's go ahead and get into on-prem agents we will again talk about the platform we'll also mention the model hosting this is just sort of a for those of you that may be slightly more beginner the inference server versus the API server. These are different things. And we really want to zoom in on that. And then we'll build. So on-prem, when we talk about this, what we're talking about is we're talking about it's right there with you. What's it? Well, the stuff you need to run your models, to run your apps. It could be at your house, like it is at Wiz's house today. It could be at your business. It doesn't really matter. Now, that's what on-prem is. I just want to be super clear like what on-prem isn't. You don't need on-prem to do production grade applications. You can do them on-prem or you can do them in the cloud. Production simply means running and not going down. You might also consider like, do I need specific types of hardware on-prem? Like, do I need GPUs with video RAM on-prem? And the answer to that is like, yes, okay, you do. Because LLMs run on GPU. That means both chat and embedding models. And depending on what you're building, you might need one or both. These things run on GPU, and they care about how much memory we have. This is a really, really important, probably top level important thing if you want to do stuff on-prem. And of course, GPUs are quite expensive. So are the models that we typically are trying to run on them. So what we'll use today is we'll actually use a version of LLAMA 3.2 that you may not know about this, and I didn't really know about this, that they have text-only versions that are great for on-device. It's not just a multimodal LLM. So we're going to go ahead and pick up the 3 billion parameter model because it's more lightweight and still very, very performant. So if we're looking at on-prem, we might also consider looking at things like on-device models, right? Makes a lot of intuitive sense there. Our device is, in fact, on-prem. Now, the next question we need to ask is what about CPUs? What about RAM? What about hard drives? Do these things need to be on-prem? Yeah, of course they do. And, you know, as we think about the types of requirements we need when we go on-prem, it's going to depend on what we're building. We're talking about on-prem agents today. And when we use agents, we may or may not use some of these other pieces, like embedding models, vector DBs, and creating robust data pipelines. Today we're talking about agents though, so let's be real specific and define what we mean by agent. Our definition of an agent at AM Makerspace is a system that can leverage or emulate reasoning or equivalent to make dynamic decisions in an application flow. And in order to create an agent, an agent, we need to really define what that means to us because as we all know, requirements really do matter. What we're building and why we're building it really does matter. And we hear a lot of variations today about on-prem, in the cloud, RAG, agents. We might really take a hard look at this heading into 2025 if we want to spend something up on our premises for either ourselves as individuals or for our teams to leverage because the compute requirements for things we need to build are actually different. If we want to build a RAG system, for instance, we're going to need an embedding model for the query. We're going to need it for our data. We're going to need a vector DB corresponding to the embeddings we create with our data. We're also probably going to need a pretty robust data pipeline if the data is changing from time to time and something we need to update. Whereas if we just want to build something like we'll build today, where we're leveraging sort of a simple agentic flow that hits the internet, comes back, talks to the LLM again. Well, in that case, we don't really require an embedding model, a vector DB, some sort of robust data pipeline. We can actually get away with much less. How much less? Well, these are important questions. And the answer, of course, as all good questions are answered is it depends. And of course, we can have a gentic rag where we kind of have all of the above, we have our sort of proprietary database, and we can pull from it, maybe we're leveraging some sort of internal API to hit it if we're doing this on-prem. And we're also giving our agents access to things in the way we would with simple tool calling, simple function calling agentic flows. So when we go to estimate the requirements, you know, the first thing that we need to do is we need to say, well, how much data? What are the models? Right. And like. The most classic thing ever in machine learning is just exactly that. So none of this is new. The new part is really when we talk about agents. Now, we've talked about agents many times on this channel, and we'll talk about agents many more times in 2025. We've typically given a few different agent definitions. One of the ways you can talk about agents is sort of in an agentic RAG capacity where you're augmenting and enhancing search and retrieval to do even better RAG. Like say agents are a fancy RAG. You can also use agents in a simpler way where you avoid RAG altogether and you're simply giving them access to tools or APIs. The simplest one and one of the ones we'll leverage today is a simple web search. How can we give the agent access to the internet to do search? And this sort of meta pattern that's kind of always at play is this react or reasoning action pattern. So we're going to focus in today specifically on the function calling, the simple agent, not the react or not the fancy rag agentic rag approach. The react pattern and the simple function calling agent are sort of one in the same thing. In fact, we can take a look at what we're building today by looking at a simple reasoning action loop. We're going to have some sort of query, some sort of question that we're trying to answer. We're going to try to do some research today. And our agent is going to take some action. That action is going to have some observation, some sort of result that we're pumping back into the agent to consider and reason through and think through again. Now that might be enough just to hit a quick Google search or something like that to get our final answer. In this case, we're doing summaries of research that we found. So sort of a final summary to answer a detailed question that we asked. Now, what kind of actions are we going to be taking today? This is where it starts to get a little bit more interesting. Well, we're going to leverage the web research. This is from the Rabbit Research Repository or the Research Rabbit Repository on LandGraph. And we're also going to be able to summarize the research. And there's one additional step. So when we go into our toolbox, when we go and we use function calling, we're typically going to go to the web search, and then we're going to sequentially move to a summary step. Now, you can leverage kind of different LLMs for different parts of this, including the third part, which is to reflect. And this is where the reflection agent comes into play here. But we're going to stick with having just a sort of LLAMA 3.2 setup where we use one LLM for each of these steps. Remember, we define agent or agentic system as a system that can leverage reasoning to make dynamic decisions. Basically, it can reason and it can decide, it can act within our application flow. When we think about reflection, this is actually an idea from the paper on reflection, language agents with verbal reinforcement learning. This is kind of a tough definition here directly from the paper verbally, a reflection agent is one that verbally reflects on task feedback signals, then maintains their own reflective text in an episodic memory buffer to induce better decision making in subsequent trials. That's a lot of words that are probably a little bit over complicated so let's take a look at this idea of the reflection agent and let's consider that each time we're going back to our agent we're sort of managing this conversation thread and this conversation thread and everything that we hold in it from our web research to our summaries. This is what we are reflecting on, okay? Simply the conversation that we're having. That's the memory buffer, that's the sort of episodic different loops, and that's the sort of crux of what it is that we're really doing here. We're going to build again, an on-prem, a local researcher or research assistant. This is another way that we can sort of visualize what's happening. We give it a topic. We're gonna go ahead and go to the LLM. It's going to do some research. It's gonna create or update our summary. And then we've got this sort of other LLM pictured here that's sort of doing the reflection. Again, you can picture this a number of different ways. This is directly from the Langchain repo. Now, this was obviously built with a very common piece of visualization, block diagram software that you've probably all seen many places before. You can also visualize it directly through the LangGraph Studio IDE. And the reflection piece, as we go to sort of from the query into research, doing the summary, reflecting on the summary, the reflection piece is actually going back to the research based on that thinking through and that reflection. Now, this is a quite simple setup, actually, this is really kind of as simple as some of the useful agents you might build out there. Get, we don't necessarily need the summary piece. We don't necessarily need the reflection piece. Those are sort of a little bit extra additions, but really this is quite simple. It's much more simple if you joined us a few weeks ago than when we did the TPS report generation using Lama index, and we sort of tried to assess dopeness of particular amounts of data. This was an agentic rag setup, and it's one that we actually had to put quite a bit more work into. We needed an embedding model. We needed a vector database. We needed to make sure that we built out our data pipeline. Whereas when we talk about what we're doing today, when we talk about this research rabbit that we picked up directly from the Langchain repo, they put this out this week on LinkedIn, we saw it, thought that was perfect. This is not leveraging all of that. We don't really need to do all of that. And so one of the interesting questions when we talk about on-prem setups heading into 2025, I want to bring Wiz back up for this, is like if I have a simple setup like this, Wiz, where I don't need all this extra stuff, at least in theory, I don't need all this extra stuff yet. How can I think about my requirements, both from an LLM perspective, and from an agent perspective? Like, howpu vram cpu and ram do i really need yeah i mean this is not really a a question that we have the answer to out of the box uh this is very much you know just dependent on what you're doing uh even for these simple things, right? If you use the 1B model, well, that needs less, you know, resources than if you use a 3B model and performance will appropriately scale, right? So I think it comes down to at the end of the day, like what kind of performance you're looking for and that performance is largely gonna dictate the actual, you you know resources that you need just to to give a a non-answer that's right question that's it so the performance you know you mean sort of like how long will it take someone to get a final answer from our research report after giving it a topic like what's the latency for a final answer from our research report after giving it a topic? Like what's the latency for a final answer kind of thing? Sure. What's the latency? That's a great way to think about performance. Another way is like the accuracy. How well, you know, how good is the report? What's the quality of the report? So on all facets of performance, our resources are going to be taxed as we want things to perform better. If we want a better report at the end of it, a better model is going to lead to a better report, which is going to require more resource. And if we have the same pool of resource, well, maybe now the model is running slow, but the report is generating accurately. And then we need even more resource for it to generate fast and accurately you can imagine how we get in the situation where uh we're talking about nodes and clusters very quickly right when we're we're thinking about on-prem that's right that's so okay so I'm imagining you know I I I'm a developer you know, I'm a developer, you know, at Christmas is coming up, I either have some Christmas money coming in, or, you know, maybe I'm going to give myself a gift here. And I want to be able to build on prem dope agentic apps. Okay, so like, all right, that's my goal 2025. Like, I know we've talked about this before with the gpu but are there cpu requirements that you'd recommend for people today you know do i can i get the the the macbook that is like basically standard and i can kind of get very far with it or do i need to spend you know four grand on the macbook right now to be able to to build agents or does that just make people feel cool and they really don't need it yeah i mean if you want to build on prem it's going to cost you money no matter what uh we've talked about this before would always recommend getting kind of a you know 40, 4090 or 3090, pick your poison, you know, if you want to just get started, but that's, you know, not going to serve a lot of customers, right? So, you're immediately going to have to purchase more hardware. If you just want to tinker and build and have a great time, the macbook uh you know whatever whatever m series chip right uh it works well with olama which we're gonna be talking about today and that's really all you need to like tinker and play right prototype prototype once you get into you know trying to build a performance system you're gonna need to care about things like, are your computer resources high uptime? How do you handle when they go down or something breaks? You immediately get into a much more complex, much less straightforward engineering problem once you move past that prototype phase on-prem. Okay, okay. We got one quick question in the chat here. Can we set up on-prem systems with Red Hat 8.x to execute Gen AI stuff? Sure. I think Red Hat can run Cates, so that's all you need, right? You just need a path. As long as there exists a path to Kate's, you can use whatever operating system you want. I think even Windows can do Kate's stuff through WSL. So your OS shouldn't be a limiting factor. Okay, okay. All right. So basically to recap, like agents, we're not talking like we need some massive CPU, some huge amount of RAM, especially for the simple, straightforward kind of function calling agents. This is something you're going to get out of the box with any kind of standard latest laptop you get off the shelf. Yeah. I mean, you can run this on a MacBook and it will work. You just need, you know, whatever, 24 gigabytes of unified memory, right? We don't worry about the CPU and GPU on a Mac. It's all the same box. It'll be good enough if you have the M series. Good enough, good enough. All right. Well, thanks, Wiz. We appreciate those insights. And we're going to go ahead and introduce the LandGraph platform next, everybody. So we'll see you back in a bit for the build. Okay. So with that in mind, on-prem agents, let's take a look at the latest and greatest tooling from Langchain. The LandGraph platform is really a rebrand of the LangGraph cloud. It came out Halloween this year, so not that long ago. And we're going to use the self-hosted lite version, which gives us access for free up to 1 million nodes executed is the language that they're using. And it's kind of a limited version of the LandGraph platform, but we can run locally, we can self-host, we can do stuff on-prem. There are also other enterprise options if you're interested in those, you know, definitely check those out. Now, how did this come about? This is the more interesting thing for us, right? How did this really come about? Well, the evolution was kind of like, you know, we did an event on this agent IDE, the Landgraf Studio idea. It turned out a lot of people really liked this. We thought it was cool too. We actually thought it was a no code builder when it came out, but it wasn't, but it still really is a useful tool to sort of debug and visualize and accelerate application development for these types of applications. So this came out and then Langchain realized, oh, people love this. They love it so much that they actually want to be able to use it on their own hardware. They don't just want to use it in LangGraph Cloud. They released a desktop version of this. So that's kind of where this began, at least in the story that they spin. Now, additionally, LangGraph server was something that people were finding to be quite valuable. This idea of serving applications up that are built with LangGraph is not new for LangChain, but this is kind of where the agents met the serving a little bit, LangGraph really for those agentic flows, and then serving kind of came together here. And what LangChain heard is that really using this LangGraph server was a lot easier. It was a lot easier than, for instance, a previous solution that we actually thought we were going to cover today. And interestingly enough, we covered last December 14th, almost exactly a year ago, we covered LangServe for the first time, but this is a tool, it's the same idea. You can deploy a Lang chain application directly to an API that people can hit. It never really caught fire in the industry as we saw throughout the year. And even as of this week, you can see on the docs, they're recommending to use LangGraph platform instead of LangServe. So that's exactly what we're doing today. And then finally, the nail in the coffin for going with the plat. On-prem, baby. You know, we've got a lot of people out there. They're like, hey, data privacy is way too important. I'm not going to use LandGraph Cloud. They're like, we've got to be able to hit internal APIs, can't use LandGraph Cloud. And so, you know, they listen to their customers, they listen to many developers out there like you guys, and they realized, well, actually, let's do something that allows for either cloud or on-prem for people with strict data privacy requirements or for people that need to connect to internal APIs. Cloud, which was previously obviously a cloud solution, was now replaced with Landgraf Platform, which can do both. Look at that. Maturity, evolution, 2025 is indeed on the horizon. So Landgraf Platform has three primary components here. It's got Landgraf Server, very similar to this idea of LangServe where we can serve up the application. It's got LangGraph studio where we can visualize, debug, use it to accelerate application development. It's got the command line interface and the software development kit, classic, and it helps us to play scale. All right. Okay. So we're gonna use this today. And we're also going to, importantly, you know, none of this hosts LLMs for us. So we have to do that separately, no matter whether we're building just an agent or a rag system or an agentic RAG system. And as Wiz mentioned, we're going to use Ollama. Ollama, as we talked about before, it's really quick and easy, very, very useful, allows you to deploy stuff locally, and you can really take it anywhere. As we've discussed in a recent event on VLLM, it's not necessarily gonna be the hyperscale, super fast solution for everything in production as you grow things out. But you can take it anywhere, like wherever your premises are. Again, we're going to leverage some efficient models, the 3 billion off the shelf, LAMA 3.2. This is text only. This is not a multimodal LLM. And to be super clear about the servers that we're going to use here, OLAMA is going to be the inference server. So that's the one that hosts the large language model. The lang graph server is going to be the API server, the application server. This is the one that we're hitting so that we can basically say, here's my topic, give me back the report, do it all. And I'm personally a little bit curious to see, can we get some more visualizations from Langchain about what's going on inside LangGraph server? If not, that's something that I'm quite curious to build out here for you guys that follow us at AI Makerspace. So let's go ahead and get into the on-prem research assistant today. It's called on Langchain's's repository Research Rabbit, and Wiz has whipped up a special AI Makerspace edition so we can get Gentic on-prem today with you guys. Wiz, over to you. Oh, yeah. Okay. So we're going to do something terrible, which is share the whole screen for just a few moments. So this is what we're working with today. The idea is that we have a number of things that are rolling. So what we want to be able to do is we want to be able to use this Research Rabbit. We want to be able to use it in our LandGraph studio. And we want that to be hosted through LangGraph Platform. So we have a series of steps that allow us to do this. We have two different terminals here. One is going to be running a local studio version, which we can use to debug, play around, and test. And then the other is going to be running our actual deployment through LangGraph Studio. So this is running on my local machine, and then this is running on my living room server. So what I would say is important to keep in mind here is that, you know, there does exist this separation between like the debug prototyping mode. And then that we've pushed this thing to quote unquote production. Also, you'll notice that we have our Ollama set up. So we can do cool things like talk to this Lama 3.2 model. Like, how are you today? Right. You'll notice that the inference, once the model is on the GPU, is very zippy. This is because it's a relatively small model, right? So this is all of what we're working with just to give you an overview, right? And now what we'll do is we'll dive into each of these components. So first things first, we got to talk about what the actual application is doing. It's going to be this research rabbit from Langchain. You'll notice that we have our LLM. It's just going to be our chat Ollama model. We have the LLM and JSON mode as well. So we can leverage JSON mode since the LLAMA 3.x series of models support that JSON mode. Then we have some nodes generate query. Unsurprisingly, this generates a, you know, this generates a query. We have web research, which is going to be a tool use node. We have summarize sources, which is going to be another kind of tool use situation. We're going to think of it as such. We have reflect on summary, which is going to be a node that allows us to reflect on the summary we got from that other node. And then we have our finalized summary, which is kind of like locking it in. And then we have route research, which is going to tell us where we need to go. You'll see here that what we're doing is we're just going to loop a set number of times in this simple agent. And then we're going to escape to our finalized tag. So all we want to do here is make sure that we have the actual nodes set up, which is pretty classic. You're used to hopefully you're used to nodes at this point. And then what we're going to do is we're going to, you know, build our graph. Once we've built our graph, we're going to go ahead and run our graph. Now you'll notice as well that we have this interesting file structure where we have some prompts. These prompts are pretty straightforward. All we're going to do is we're going to think about them as instructions. This is normal agent stuff, right? So nothing too weird or crazy at this point. And then, of course, we have our state. Our state is pretty simple given the actual application, right? So what we have is the research topic, what our query might be, what our results have been, what sources we have so far, you'll see when we look at the application why this is important. And then we have our research loop count. So this is how many times we've gone in a circle. And then our running summary, this is our summary so far. And that's it. With the entire thing, this whole deduplicating format sources is just going to build a list that's not filled with duplicate links in case we get them. Okay, so that's the application. Okay, that's dope. Now we're going to go ahead and see how it works, right? So we've got this cool application. We're very excited about it. And we're going to go ahead and we're going to look at how it works. So in the LangGraph Studio, which you'll notice, or you might not have noticed, but I'm on a Windows machine. And I'm looking at LangGraph Studio, right? Now, LangGraph Studio is running in the Docker container. We're accessing it through that smith.langchain.com. And we're able to see even our Windows machine, LangGraph Studio, though we can't launch it as a desktop app, which is okay. You'll see here that I've already done a research query, which is how is Llama 3.28B Instruct trained by Meta? And you can see as we go through the steps, first we generated a query, which is how is LLAMA 3.28B Instruct trained by Meta. And you can see as we go through the steps, first we generated a query, then we did some research, right? After the research, we summarized, then we reflected on the summary, and then we did some more research, and then we, and on and on. You kind of get the idea, right? So we can add a new thread here and do new research like what is an LLM agent anyway. And then we can submit this and get the beautiful visualization as our system works through the steps in order to solve this problem. That's just working through those nodes we created in that Python file earlier. Okay. So that's that. I'll just let this thing run. That's fine. What we want to do as well, now that it works, is we want to go ahead and get it running on our remote machine. In this case, a different server. So what I've done is I've already set this up so that we can access it. You can see here that we can access our docs, which is going to be how we use this service. It's got all of the instructions we could ever think about or want, thanks to the documentation that has been created for this service because it's using the open ai or open api uh you know structure the docs look very good and are very robust and that's all been delivered through the land graph platform so in the first instance we just kind of have this land graph server here running by itself. And then we have our land graph platform running the separate instance. This is extremely important because it lets us wrap this whole thing in that platform, which is more than just the actual component of or more than just like the land graph graph so if i once again stop my screen and then we're going to share my terminal window which will be lots of fun which we can share here you'll see here that we have this terminal window i'll go ahead and zoom in so it's less uh it looks less tiny here and the idea pretty straightforward. We're going to go ahead and run things down. And we're going to go ahead and look at our Docker Compose. When we have our Docker Compose, you know, we've talked about Docker before, right? But Docker Compose, what it does is it has a bunch of different services. We have our LangGraph Redis service, Postgres service, and then we have our actual LangGraph service. This is the reason why we say this is more production ready. It's not just this kind of naked API chilling out there in the wild. We have our Postgres service. This is a database. A database is gonna handle things like threads and assistant interactions and all kinds of other fun things. Redis is gonna act as a pub sub for how we're actually interacting with the services. Because many people might be using the service at once, there might be many background threads running, and we need to be able to highlight those to the correct users. There is one limitation with this build as built out specifically, which is that Olama doesn't support concurrent generation. So this is not going to be something that you plug and play into production. You're going to want to or need to look for something like VLLM in order to bridge that final production gap. But very few code changes are required to do that. And it works well for a single user with a llama. And that's it. To say that's it, it's a lot, right? We have this idea of Docker Compose. We have this idea of all these other things that we're using. And if we can finally just Docker Compose up, right, which is going to spin up all of these services, Postgres, Redis, and then of course, our favorite, the LandGraph API. graph api and uh this is you know all gotten quote unquote for free all we had to do is build that docker compose file they already have an example on the uh in the documentation for for land graph platform and then we spin it up and and that's it you know we're we're away to the races and we can start using this service uh with uh with some some reliability uh that's that's the whole thing that's the whole shebang uh so with that i'm gonna pass you guys back to uh greg uh but before i do i gotta ask you to for the last time this year in fact uh you know uh don't forget to subscribe and ring the bell icon on YouTube. We're live every Wednesday doing events like this. Most Wednesdays, we won't be live for the next two due to them falling directly on holidays. But there you go. Back to Greg. All right. Good stuff, Wiz. All right. I want to remind everybody, go ahead and throw any questions you have in the chat. We'll have some time but i wanted to go ahead and also introduce some of the latest drops that we saw this week from lang chain slash lang graph and the first of these is called the first of these is called LangGraph Command. And LangGraph Command was dropped on December 10th. And, you know, this whole idea of like these crazy drops we've been happening, we've been seeing from the industry, from OpenAI, from Google, it seems like LangChain, LangGraph got in on the action here. So I want to kind of have a little discussion about this. You know, the kind of couple of things they pointed out with each of these are really important. And I think maybe it sums up to, you know, something, again, maturity 2025, what's coming next. LangGraph command basically said, you know, requiring edges to connect nodes, turned out sometimes it could make it harder or less intuitive to express very dynamic logic. So basically Langraft command is this idea of edgeless graphs and edgeless graphs. And edgeless graphs allow us to more easily build multi-agent architectures. And this idea of sort of no edges is quite interesting. So I wanna just kind of actually pause here before we go to the next one and bring Wiz back up. You know, why would we care so much about edgeless? This feels like some of the discussions we've been having around event-driven architectures versus graph-based architectures. Is this how it sort of, you interpret this as well? Or is this something specific that you would expect them to roll up so that they could be more production ready? Yeah. I mean, it's just all about how you're thinking about having multiple agents, right? I think the idea of like an edgeless graph lets us kind of emulate the pattern that llama index has not not directly uh but it does let us have more uh kind of easier time to to build out these more complex graphs uh that doesn't require us to have like 40 lines of boilerplate with conditional edges. Remember the idea of conditional edges is they're kind of maps that tell you where you can go next, right? Command helps us kind of do that in a more, in a less verbose, less manual way, I would say, is the way to think about it. Yeah, so I guess the idea of having conditionals for everything does sound like it doesn't quite scale particularly well so you know from that perspective there's some intuition there that i think makes a lot of sense. So, so they released this command piece. And then they released four days later, this was, you know, on December 14, Langraf interrupt, which allows you to interrupt Langraf, you know, and we've talked about this before, where state machines are a little bit more flexible than fully autonomous agents. And the reason is because you can basically track state and you don't necessarily want things being totally autonomous. So, you know, like as we, as we sort of look at this idea of putting humans in the loop more easily, they give a couple of examples here where we're talking about, should a human approve or reject the next action? Should a human approve or reject the next state? Should a human approve or reject the next tool call? Should a human approve or reject the next agentic reasoning flow here. And so, you know, what do you think, Wiz, about these new releases? Are these things that are going to move the needle for people in 2025? We're sort of trending more towards this maturity and enterprise grade thing. They seem to be listening to their customers quite well. The naming's not so bad. I don't hate the naming. These don't necessarily deserve dedicated events from us, but what are your sort of high-level thoughts on some of these new releases from them? Yeah, I mean, we didn't live in a world where we needed to build these specific kinds of multi to the, to the experience to create like a, a, a, a well operating agentic system. So things like command, which are more so like helping developers and the things like interrupt, which can make sure we have this idea of, you know, being able to gracefully and appropriately basically pause and start uh graphs right uh lets people interact with them in a way that would feel natural in a way that would be desirable uh but we didn't yet have the construct for it it is again you know we're we're all in exploration phase right now. Multi-agent systems exist technically in production, technically, right? But they're not like, it's not solved. And so the tools that we need aren't solved. I'm very happy to see Langchain continuing to push forward and help create some of these tools so that people can build better applications than it did in 2024 when it comes time for 2025. Yeah, yeah, definitely, definitely. And I think it seems like the shipping velocity has also kind, increased as everybody has kind of dialed in around agents. We're seeing sort of a lot of people really move towards this and move their product towards this as well. You know, we're doing our best to define agents here as we head into the new year when people hear agents, when you hear agents, and you're sort of hearing about the new agentic release, you know, how, how would you encourage people to, to think about what matters, what doesn't matter, how to sort of pick and choose, how to separate signal from the noise with some of these new tooling that we're seeing some of these more you know they're kind of coming from everywhere whether it's crew ai or swarm or llama index or laying chain or you know everybody's getting in the game now um what are the most important things from your perspective to pay attention to when it comes to agents in 2025? Yeah, I mean, they're, they're not solved. This is still a big play space, right? I think, keep trying to innovate, keep trying to just try things. I mean, we're still in the phase of like, basically, like the prompt engineering equivalent of agents, right? So it's, you know, just get out there trying things, use different frameworks. Definitely we're going to recommend, you know, LangGraph and LangChain, but, you know, try out different things, you know, poach patterns from other frameworks and research that you see. In terms of separating signal from noise, it's very difficult right now because we just don't know where something like a multi-agent system really prospers. We don't have super clear, consistent case studies. And so it's kind of all noise right now. Um, you know, but pay attention to agent force, pay attention to, uh, you know, uh, uh, folks like AG two, you know, who are going to hopefully help steer the ship a little bit. Uh, other than that, it's just, you know, it's, it's, it's all over again again chad gpt but this time with the agents and uh 2025 is going to be fun and hopefully we're going to lock in a few use cases that make sense and leverage the technology in a way that adds value to the businesses yeah yeah yeah and you know my one last comment on that too is everybody's releasing these sort of like study of who's using agents and there's a lot of sort of feedback going on on LinkedIn and elsewhere where people are talking about well yeah like that percentage seems high like yeah I mean the people that are answering the surveys about agents are the people that fundamentally know about agents already, right? So they know about Lang Chain, they know about LangGraph, they know about Crew AI. So yeah, there's this whole world of people that are not out here that I think it's gonna be really, really interesting to see how they decide to use these tools in the year to come. So let's go ahead and open it up for Q&A after we sort of draw some conclusions from today for everybody to keep in mind. You know, when we build on-prem agents, we want to make sure we have the agent hardware support and we have the LLM hardware support. RAG is going to add other stuff. So if we're doing a Gentic RAG, we're going to have to keep that in mind. And when we just want to tinker, we can go far with something simple. But if we want to scale out our app usage, we're going to have to scale out our hardware. So if you do have questions, please let us know. We wanted to add some little bit extra time today just because it is the end of the year and it is our last event of the year. We really appreciate all you guys joining us throughout the year. We have one question um i'm starting to see them coming in the chat i do not worry my data leaving my prem can i use a grok llm in that research application i think this is talking about grok hardware because it's grok with a Q, not Grok. Yeah, you can use, if you don't care about data leaving your prem, you use whatever API you've got access to. Yeah, easy. Will my data leave the network if I am using LangGraph platform on prem like do i need to worry about it going to land graph server uh hosted somewhere and somebody's seeing it i think this is the indication here uh i believe the answer to this question is technically uh dependent on whether or not what you consider your data the short answer is no. The long answer is some data does escape through traces, things like this, depending on the integration that you have with Langsmith, what you're tracing. So this seems to be configurable though, which means that you don't have to share any data outside of your premises. And certainly with the enterprise solutions or even the cloud solutions uh that they offer so this is everything kind of past like the free tier uh your your you have full uh full ownership the data that never leaves leaves your you know your aws uh uh world it never escapes. Yeah. Next question. How are you guys thinking about sizing slash complexity for agents versus assistants? That's an interesting versus there. For example, in setting up a large orchestration agent with many sub agents as nodes, how are you thinking about sizing and yeah at the end of the day i think it would be very difficult to uh to want to reach for very complex agents i think we'll be the the great part of laying graph right is that it's kind of like these little atomic graph units right that that we that we care about um and so you know i'm content to have two or three sub graphs per kind of meta level uh but getting past that i think you you just wind up in this like hyper complex space where you have no idea what's happening even with great tools like studio and langsmith it's very hard to trace through and understand and debug what's happening even with great tools like studio and langsmith it's very hard to trace through and understand and debug what's happening in that application so you know creating little sub graphs uh or sub agents as nodes right uh we we don't we still want them all to have very limited uh scope uh especially because we just don't know yet what the best patterns are. They haven't been figured out, they haven't been trialed and tested. So I would say keeping complexity as low as humanly possible, right, is the goal. And so that's how I think about sizing. What's the absolute minimum that i can get away with and the thing still functions okay okay okay so um two last questions to end with the picture behind me guys this one the attributes of the sciences by jean baptiste simeon chardine it is something that i'm i'm waiting to update until we get the attributes of the AI sciences in an image. So if you guys get one, maybe I'll print it off and put it behind me. Last one. I love this question because it's very similar to a question that we got yesterday as we're beginning to close out our final course of the year, Wiz. And that question yesterday was, is there anything you'd put in the course that you couldn't put in the course? And we said kind of no, because that's the way we design our curriculum. This question is, Dr. Gregg, if you started a second PhD today, what would your topic be? Or I would say, Dr. Wiz, if you wanted to get a PhD today, what would your topic be? Or I would say, Dr. Wiz, if you wanted to get a PhD today, what would your topic be? And I think this is such an interesting question because you and I were talking about this last night, Wiz. It's like the stuff I studied during my PhD in optimization and computational design, and Wiz, your background is in sort of general physics. These sort of backgrounds really did prepare us quite well for navigating the LLM edge today. Like I spent a lot of time in gradient descent. I spent a lot of time learning the old school perspectives on optimization algorithms. You can look at things like PPO and DPO today when you talk about alignment. And, you know, it's like, I guess for me today, what I would do knowing what I know now is number one, I wouldn't necessarily decide on a topic. I would decide on a research group and a professor or an advisor that I really, really thought was the best. And I would really drive towards working with the absolute best. So here's a little insight into advice I've given previously, which is reach out to the research advisors you dream of working with, set up a trip to literally go there and ask, can you give me a walkthrough of your lab? Do you have any open projects? The most successful professors always have open projects and they're always hiring new PhD students. Show the initiative, show that you've studied the background on these LLMs, show that you're really, really in to staying up on what's happening at the edge, show you watched the ILLIA talk from last week, show that you're sort of there and you're willing to go wherever it is that you need to go. And, you know, that'll be sort of, you know, if you're into this stuff today, that'll be sort of the best way to stay right there at the edge and continue to be useful. Like, Wiz, what's your two cents on this? If you had to pick a thing to study today, what would it be? That's an impossible question to answer. It would be, it would probably just be more LLMs, man. Like, you know, I think, unfortunately, you already gave the best answer. So it's, I would just copy that, right? Like, find the people working on the problem that you think is important, and then do anything in your power to work with those people on those problems i think the the you know you don't have to be doing a phd to do that to be clear don don branson from our community uh you know just submitted a pr to ag2 because he likes that right he's contributing uh this is the it's it's it's never been easier to be part of the research community right now because the research community is exploring, right? They're trying to find what's next. Ilya just gave his big talk where he says data is fossil fuel and all this other wild stuff that's got people in a fervor and you know we're trying to figure out what the renewable energy is right now of of ai just to extend his what however you you may think about it just to extend what he was going for and it's never been easier to contribute somebody was easier to to help as an academic as a non-academic uh you know this is this is the time in quote unquote ai history that we we need those ideas foundation models are plateauing uh we all we have is using using them and using them the best no one knows the answer to that question yet that's right and at neurops you were at neurops this past week or so and you mentioned that research today is not just in academia it's very much also an industry you know and this has been true at open ai but it's also true elsewhere today it's true places like google it's true at places like amazon it's true at the large tech companies as much as it's true in academia now. And I think this is an important insight that wasn't the case when I was studying optimization, you know, 2010 to 2015. So don't sleep on industry. Is that a fair insight from your time? Don't sleep on industry. All right. Thanks, Wiz, for your insights. Thanks, Wiz, for the demo. And it's time to wrap it up, guys. Thanks for joining us for our very last YouTube Live session of the year. We're going to be back for 50 more next year until we take a little time off between Christmas and New Year's 2025. So look out in the newsletter for what's coming next. And please don't forget to keep building, shipping, and sharing over the break. Now, if you're looking to accelerate everything that you're doing and really, really try to get out to that edge as quickly as possible, you might consider joining us for the AI Engineering Bootcamp cohort number five, that's gonna kick off January 14 14th it's going to be a banger we've got some amazing peer supporters ready to help out we've got really the most agentic and state-of-the-art curriculum we've ever had and i'm really really excited for everything that comes next so hit me up if you have any questions about that and again uh have a great holiday season everybody thank you for supporting us this year. You guys are our early adopters and we really do appreciate your support. We look forward to following your journeys in 2025 and beyond, whether they lead to PhDs, industry, or otherwise. And although we'll take a little time off, you best believe I'm going to be cooking something up and I'll be building and shipping. And if you jump into the Discord, you might just see what we're ready to share over the holiday break. Until next time, we hope you guys keep building, shipping, and sharing as well. Merry Christmas and Happy New Year's everybody. See you in the new year. Продолжение следует...
On-Prem #agents with LangGraph Platform
Learn how to build, ship, and scale secure agent applications on-prem in this live event! We'll dive into prototyping with LangChain, deploying with LangServe, and hosting models locally with ollama—all on your own hardware. Perfect for AI engineering leaders in highly-regulated industries, this session covers choosing the right on-prem setup, scaling challenges, and live Q&A to get your questions answered. Don’t miss this deep dive into enterprise-ready AI solutions!

Event page: https://bit.ly/onpremlangraph?utm_source=youtube

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/d76N61c8Dbqdxf4RUMtWaZ

Speakers: 
​Dr. Greg, Co-Founder & CEO AI Makerspace
https://www.linkedin.com/in/gregloughane

The Wiz, Co-Founder & CTO AI Makerspace
https://www.linkedin.com/in/csalexiuk/

Apply for The AI Engineering Bootcamp on Maven today! 
https://bit.ly/AIEbootcamp

LLM Engineering - Foundations to SLMs
https://bit.ly/aimllme

For team leaders, check out!
https://aimakerspace.io/gen-ai-upskilling-for-teams/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/EJH8kjSoM7w9QKuU9

#onpremise #langchain #api #server