 so i'm excited this uh this should be fun i thought i was gonna be able to say a little music and set the mood right but i couldn't figure that out in the allotted time and i just want to get rocking now that you're here i think it would be awesome to just kick it off get us going hard and for those that are wondering we have like uh two talks today and we did we did this agent hours last two weeks ago and we had the breakout room sessions which were a lot lot of fun. So we wanted to do it again, but we wanted to concentrate the breakout rooms into one place. And so now after the talks, this time around, we're going to have the breakout room or just one breakout room. So it's all going to be here with us now. So you can just hang out. We're going to have the talks, Samuel, you're up first. And then after that, we will, uh, hopefully have some cool discussions. So if there's questions, I'm sure I'll have plenty of questions to ask. I know there's people that are, so the fun part about this is that i think there's some people that are looking for how to get into this room right now so if you are watching us on the live you hit the agenda on your left head uh and then that should ping you right in here uh and once you're in it's easy it'll be around uh but yeah samuel if you want to share your screen or anything i think um yeah i will do that i will try and put you on this screen and put where what i have in the way of slides on this screen there we go and i can get going so uh can you i presume you can you can see that yeah it's nice and big too thank you that's good i hope that's a good size if anyone is is struggling to read anything like or struggling to understand like i can see the message so like send a message and i'll um try and fix it this is my i mean i've got frustrated with every possible deck format and so i'm i'm using this plain text format so i hope it's not too annoying i'll try and remind me how long i have to to talk just so i've got a got a feeling 15 minutes okay so i i probably i gave this to a version of this talk and it took a bit longer so i'm gonna like whiz through and assume most of you have some idea of who i am and and what's going on but like yeah started pydantic ages ago back in 2017 became a company uh in 2023 we built logfire which is our observability platform which you should go and try if you're building in python uh um yeah the first thing we did was release Pydantic v2. That was last year, the rewrite in Rust. Pydantic today is downloaded 300 million times a month. So it's important to remember, obviously, we're talking about, like, Gen AI today. But, like, one of the power of Pydantic is it is not just for Gen AI. It is, like, widely used in general development, used by everyone who's writing Python, which is basically everyone, but obviously it's got a kind of new lease of life from Gen.AI where it's used in general in API stuff, but in particular in like validating structured responses. So Pydantic is, yeah, I kind of said ubiquitous, like boring and generally liked by developers. The laugh is perhaps a bit strong, but I'll try it. And again, going back to my point before that Pylandic is general purpose, I think it's worth just repeating. This is what Pylandic was built to do. Pylandic long predates LLMs. And the basic idea is you give it, you define a model like this, which has, we use type hints in Python to define your types, but unlike normally type hints, which do, as per the name, are just the hint and don't do anything at runtime, we basically use those type hints to enforce that type. And in particular, and this is obviously super relevant to Gen.AI, we're kind of lax in the sense we try to coerce types. So if we see, you'll see here that PyCharm is complaining that ID is supposed to be an integer, but I've passed it a string. Pydantic by default will try to coerce values. Similarly, this is an ISO 8601 or RFT339 format date, but it's a string. Pydantic will take care of coercing values from in this case a string into a date um and we'll do the same in json so we'll take a json input and again this is the the coercion thing is super valuable because there's no date type in in json so if we were big super strict we snookered we literally cannot define a python date in json directly so we have to do the coercion thing. And then the last thing that we built into Pydantic, actually, Sebastian Ramirez, who built FastAPI, actually originally contributed JSON Schema. At the time, it was all about APIs. We didn't even know JSON Schema was going to go and get used by LLMs to define tools, but we built that long ago, which turned out to be super valuable. So this is like old school use of Pydantic for general purpose programming. What then happened and what everyone went and realized was, oh, when we're making calls to OpenAI and in particular doing tool calls, Pydantic becomes super valuable. So we have our same model defined here, given it a doc string. And with a little bit of work and a bit of like ugly use of Dunder methods, we can define tool calls entirely using our definition from our Pionic model. So parameters of the JSON schema, we take the name from the name of the base model, take the doc string to become the description and hey presto, we get tool calls. And then of course, the great thing is you can then use that same model to go and validate the JSON you get back from in this case, OpenAI and give you a user or generate errors as to why that data was invalid. And this is basically the thing everyone uses Pydantic for. So this is what now you can do this built into the OpenAI SDK. It's also used in all of the agent frameworks, LangJane, CrewAI, PhiData, Instructor, etc., etc., etc. I can't even remember the name of all of them, but the trick that they're all really doing and the thing that Pdantic is most useful for is basically this thing. And so the kind of reason I'm talking today is that we decided that, to put it politely, we weren't that happy with any of the Asian frameworks. We didn't particularly want to go and build with any of them, but we wanted to build with Gen AI, and so we released pydantic ai what like two weeks ago now it's been it's been quite a manic couple of weeks but yeah i think two weeks ago on monday is roughly when we released it and it's had obviously an amazing amazing reception but like basically it's a it's it's a number of different things but it's it's kind of a wrapper for this thing fundamentally. So again, we have our Pydantic-based model defined, but we're importing agent from Pydantic AI, and we're setting the result type to be user, which is going to go and do that same setting up the tool call. But this comes with a bunch of other nice stuff. So in particular, we get reflections. So if validation fails, Pydantic AI will take care of taking the validation errors that Pydantic gave us, sending them back to the model and saying, please try again. But there's more it can do. So dependency injection is something that we, again, Pydantic AI is not about producing the best possible demo or giving you the easiest experience in the first 10 minutes. It's about building production applications. And so dependency injection and type safety are really critical. Type safety doesn't particularly matter when you're giving a presentation, but when you're trying to build a real application, it's super valuable. And so in this case, we define our dependencies here. We have an HTTP connection. We have some API keys and we define depths here, depths type when we define the agent. And then we go and register some tool calls. So in the first example, we were using just the result type, which internally is a tool call to give you structured data out of the agent. But here we're defining tools that are effectively discretionary. So the agent can, sorry, the model can choose to go and call them or not. And in this case, we define a getLatLong tool call, which gets us a location. And we define a weather tool call, gets us a location and we define a weather tool call which given a lap long will return the weather and then we go and run our model here we define our depths and we pass them in as a keyword argument when calling the agent and we get our result. But the powerful thing here is the type safety. So using some clever tricks in Python's typing that you don't need to worry about, we can basically, with static analysis, guarantee that you're using the right depths. And so if you run mypy or pyrite over this and your depths type here is incorrect, you'll get an error. And obviously, once you define your depths type here, when you come to access your depths, so here we have context.depth.client, which is accessing this attribute. If you accessed it wrongly or if you called it wrongly, you would get a type checking error. And that's super powerful. So yeah, the overall workflow here is that the model would be clever enough to say okay i'm going to take the input which is let's get the weather in london and wiltshire let's extract the locations from that let's then go and call the get that long function to extract to turn those locations into elasticity and longitude and then take those values and use them to call the weather function to get the weather and then return it. And so at the risk of never run a live demo in a presentation, especially a 15-minute one, I will try and run that as an example. So this is that same code. The only difference is there's a bit more code to get the weather to look pretty. But if we go and, oh, nice error from GitHub Copilot, if we go and run that, oh, it's not going to run like that. But if I exit this mode and I come here and I try and run, what was I running? Run that example. What we should see is it making the relevant calls and then coming back with a response and giving us the weather. So OpenAI in this case, and I haven't talked about model agnosticism, but we have good support for other models, will take care of effectively calling the right tools. You can see here that it's calling them together. So it's calling get that long twice and get weather twice in parallel. But if we come back to the slides and we go on, this is all very well, but the problem is what's that model doing internally? And you started to see the beginnings of it just now, but like, obviously we think observability is really important. That's why we've built Logfire and why we've built, um, an optional integration into Pydansk AI so that you can use Logfire to understand what your agent is doing. So what tools it's calling and how long they're taking. Um, uh, so I have, I'm not going to open up, uh, Logfire on this occasion, but well, the first thing is you saw immediately here this is logfire giving you an output of what happened uh that's nested in the terminal but if you open the logfire dashboard you get effectively the same view but with much more information so not only do you see what took how long in not only the llm calls but also the http calls to the different apis but you can also see on every given line exactly what's happened the cost in terms of tokens and what's taken how long um so yeah we think observability is super valuable and we've seen that immediately in in logfire loads of people coming to use logfire off the back of finding it through Pydantic AI. But to go on to a few other things, and now I'll talk for a little bit about a couple of things that are coming up soon in Pydantic AI but aren't quite there yet. So agent handoff is a big subject. It's how Swarm got a lot of its attention you can already do this with pydantic ai by effectively registering uh other agents into dependencies like this and then calling those other um agents from within a tool call but we're just about to add support uh for for basically syntax for adding another agent directly to uh an agent so here we, instead of using the decorator to add tools, we add tools using the keyword argument tools. And then we have this agent tool, which you will see PyCharm is complaining doesn't exist because I'm literally working on the PR right now. But the idea is that you can register agents directly with an agent. Again, we use, we can do some clever stuff to make sort of types are correct. So all of these agents need to have the same depths type. So you can then pass depths between agents as a call, as a run is ongoing and have confidence in the typing. We obviously have an input type now, which is then used to validate the arguments passed to the tool. And I think this will be, this is one of the two forms of model comp or agent composition that's super exciting. So this is the like agent handoff. And then we have the kind of calling multiple different agents in sequence, which is the other, the other big thing for us to go and add. I don't quite know what the API is going to be for that yet, but that's another thing we're going to talk about in the future. One of the nice things about doing this in a structured way like this, rather than declaratively, imperatively, excuse me, by inside a tool, is we can go and build a state machine for what this looks like statically and tell you the different agents that are being called under what conditions and what the different flows are of a like multi-agent model. I mean, one of the problems we have honestly with Pylandic AI is the agent isn't quite the right word because our agents are quite small and self-defined. They're almost like agentlets um and so yeah we you end up composing them together to form actual uh as components rather than necessarily trying to bundle all of your logic into a single agent um i just i see some questions but i'm just going to go through a couple more things and then i can can try and answer. Another thing that's coming up, many of you will be aware of Model Context Protocol, which Anthropic announced probably, I said last week, but probably two weeks ago now. And again, we want to add this concept of tool sets to Pydantic AI, where instead of registering a single tool, you can register tool sets. And these can either use model context protocol or you can define your own ones, or we will have tool sets for things like, I meant open AI, sorry, I do mean open API there. So using open API and JSON schema, you could add register a tool set using purely an API's open API endpoint, or using model context protocol or defining your own ones. But the idea here is that if you're building an application that's using Gen AI, enormous amounts of your work is basically doing the boilerplate to integrate with, let's say, Slack's API. This should allow us to have a like rich library of different existing tool sets that you can basically register if and when when you want them um so yeah thank you very much i think now i don't know how much of the 15 minutes have used up i tried to go really quickly i may have gone too quickly but i can answer some some questions now or later on whatever you whatever you think's best yeah we've got uh a few minutes and anybody that is on the call if you want to just jump on and ask questions live uh in your voice go for it i threw one in the chat um i'll just answer a couple of things i've seen here i'm saying hello from cohere i think it's worth saying the examples I talked about here were mostly using OpenAI, but we already have support for Anthropic, Gemini, Grok, Olama, and there's a PR for Co here, and we're happy to add that. We do actually, one of the things that's been amazing in the time since we released this is the number of people who've come and contributed models to the point where we started to have to say no to some lesser known ones because we don't want to be just maintaining loads of different model integrations, but Cohort is definitely one we will accept. What were the frameworks lacking? I don't want to be too rude about existing people who built open source. None of them did what I wanted. In particular, they were not particularly type safe. If you go and invent some sexy, esoteric syntax to define chains of models, you end up losing all the type safety that is available in modern Python. And that's very very frustrating and so it was the production uh readiness in particular that was that we had trouble with um yeah i think achilles you is that how i pronounce it somebody's got their hands up yeah yeah Can you hear me? Yeah. Yeah. We hear you. Yeah. Great presentation. Awesome release, guys. I have a few questions. The first is right now the agent, you define some tools, and the agent decides to call some tools until it finishes the task. Can I override this logic? I mean, for example, an agent might want to use a tool, but depending on the arguments and the tool that it wants to use, I want to follow a different flow, to do something differently than the standard flow, or the native would do internal. Is this easy to customize? I mean, the whole principle of tools as they are defined by the underlying model providers is they are these discretionary things that a model can decide to go and call. If you wanted to have a tool that behave differently depending on your own context, you can have that logic inside the tool. So you take the different arguments you want and you decide what to do inside your tool call. There's also, the other thing is, if you want to get the result out rather than have it as being discretionary, you can use this syntax of structured results, which effectively require the agent to go and call this tool to end a particular run. And we have a PR open to basically add a way to exit a run early in the case that you basically call a function. And within one of these function tools, you can basically end the particular run. So again, we don't want to add too much esoteric custom Pylandic AI syntax. We want the library to be relatively thin and allow you to fall back to writing Python code for everything that doesn't need to be within the library. I mean, I think that is like good principle of building any like software library, any Python library. Don't go and do things that don't need to be in your library. Be relatively cautious about adding functionality. And I've learned that from having maintained a library that has 300 million downloads and has been around for five years. And if you accidentally go and add something because it seemed cool once once and now you're stuck with maintaining it. Yeah. So I think it's worth being cautious about those things. I mean, I got excited one day and defined a color type in Pydantic because I wanted to do something with colors. And now we still have a color type that hangs around that like passes hex colors, which is not something that anyone would expect to be in Pydantic itself. So, yeah, I'm, I think we have the right predicates here, but if there are particular things people want, we can definitely think about adding them. Excellent. Thank you very much. Uh, the code is this in some repo somewhere or the best AKA the best slides ever? Um, I, they, I think they are on my, my on my profile on my github um they're just an markdown file if they're not today i think they're called confusingly they're called boston because i talked at a conference in boston and i use them use them there but i can i'll tweet the link after this as well so yeah if anyone is looking for them they can find them sweet and then uh james was saying is this a replacement for llama index or lang chain uh that's your choice uh if you're happy with llama index or lang chain then obviously you're totally free to carry on using them uh i'm personally going to use it instead of instead of them i mean in in the particular case of llama index andG, we don't have yet a model agnostic interface for generating embeddings, but I think we'll have it. But again, coming back to my point about production and not trying to do too much, in the end, Vector Search is database querying, and we don't want to go and build an ORM, especially an ORM inside Pydantic AI. And so in the end, we're going to give you an interface for generating embeddings, but probably not an interface for querying because to do that in an actually production ready way, you need full ORM or write SQL, which would be my personal preference or depending on whatever database you use. Awesome.
Why Pydantic AI is the Future of AI Agents
Join the MLOps community mlops.community/join. Thanks to arcade-ai.com for the support

Samuel Colvin the creator of Pydantic goes nto more detail on why they built Pydantic AI and what problem they're aiming to solve. He'll also cover some of the future enhancements they plan for PydanticAI.

Link to presentation: https://github.com/samuelcolvin/boston-ae/blob/main/slides.md

//Bio
Python and Rust engineer. Creator of Pydantic and Pydantic Logfire. Professional pedant.

This is a bi-weekly "Agent Hour" event to continue the conversation about AI agents. Join the next live event at home.mlops.community