 to this is going to be actually building the chat application. So this is where we put our front end on. This is where we deal with the web framework. And this is where Chainlit comes in. We like Chainlit, as we've talked about. It's got a really, really nice interface. It allows us to ask questions, to look at chains, to look at when we start adding rag to it. We can look at specific where we got reference materials. We can change prompts, look at prompts, play around with them. This is that prompt playground. We can look through the thinking, the chain of thought reasoning, and it integrates very, very well with any sort of infrastructure tool stack that we might pick up. So Chainlit's great. And it's got a very simple Python syntax. It's really, really straightforward for, you know, people that are familiar with web frameworks already. And it's pretty easy to pick up for folks that aren't. There's no way, the only way out is through with web frameworks, really. So we're going to have a couple additional things to do here, like starting the chat. And then when we get a message, what do we do? So you'll notice some of these aspects within the code. And you'll need to connect some pieces here. You guys will need to set up your OpenAI API key, just like you had to sign up for ChatGPT+. You'll need an OpenAI API key to be able to build applications with OpenAI's API. This making sure you have access to things is going to be able to build applications with OpenAI's API. And so this sort of making sure you have access to things is going to be a key component of really developing any application moving forward in your career. So when it comes to building, that's sort of putting the application front end over top of the OpenAI API to allow us to interact with it the same way as ChatGPT. When we talk about sharing this bad boy, we want a public URL that we can share with our friends. We can text it to our mom, our sister, our son. We can text it to anybody we want and be like, check out this sweet AI app that I built. This is going to be that big wow moment for you guys. This is where Hugging Face Spaces comes in. And creating your own space is very, very easy, as Chris will show. We're going to use the approach of the Docker container, and you'll see it actually builds the Docker container for us. It's freaking awesome. So what we're going to do is we're going to pass it right over to Wiz. We want to maximize the time that you guys have to work together today to show you the step-by-step walkthrough of how to put Chainlit onto that OpenAI API, wrap that bad boy up in a container, and then ship it and share it with Hugging Face. Wiz, show them a little bit about how to do all of this. And let's go ahead and focus a little bit on the chainlet piece that we know people are going to continue struggling with. The last thing I'll say is you guys have to sort of do this same thing over and over and over again in this course. The code behind the scenes isn't the code anyone cares about. It's the code people interact with. It's the app people interact with. And this is what gets you there. This is going to be a game changer for you if you learn it in, out, up, and down whiz over to you man hey thanks greg okay so let's uh let's go ahead and we're first just gonna jump into vs code uh because well it's that's where the code lives gotta go where the code lives so here we are in vs code you'll notice that we have our ai engineering repository on the left-hand side here. And we can go to day two in week one, and we can see that there's a bunch of different files. There are only a few files we truly care about here. The first is the app.py, which is where the Chainlit application lives. We have a Chainlit MD, which is like a readme, but for chainlet. And we have, of course, our Docker file, which is how we create our container. And the container is basically the best thing. And then the last piece is just kind of standard repo stuff. So let's look at the app.py and let's kind of talk about what's happening in Chainlit here. So number one, we've just got a pile of imports, classic top of every application in Python is just a top or just a massive set of imports. Our load.env is useful if we're going to be running this file locally and we've populated our dot env you'll notice that this is just a dot m sample so you'd have to copy this file and then you'd have to paste the file and then we want to rename it right to our just dot env and then we can put our open ai key that's going to load it, that's going to be great. But for now we're just gonna, you know, not worry about that so much. The next step is we have our system template or user template. These are basically just, you know, pieces of strings that we can format later on. It looks like we got a question from Rajesh that I'll try to answer quickly as we go through. it looks like we got a question from uh rajesh uh that i'll try to answer quickly as we go through yeah so a quick question uh uh chris so is if if he uh i mean what does chainlet give you compared to let's say just doing it uh completely manually uh with talking to the open ai api uh we'll talk about as we go through so i'll keep that question in mind but uh we'll we'll discuss that as we kind of go through uh so we have our user template user template is uh exactly what we had before in our our collab notebook and that is all that's doing is it's uh you know telling us it's basically just adding the chain of thought step, to be honest with you. But the idea is that we have some user input and then we tack on that chain of thought so that we get a well thought out answer. And then we get to some first chain lit. Now, if you're not familiar with Python decorators, this is going to be a crash course that we're going to talk about very, very, very, very quickly. Please come to office hours. If you have more questions, the idea is a decorators like a function that takes another function, right? And it's going to do stuff. And it's going to use that other function in, in some way, right? Like this, this whole async def start chat. This is a function that's going to be wrapped by all this. Now, the way that chain lit works is you can think of, there's a number of different steps that Chainlit is going to go through every time we use this application. And at different points or different steps along the way, we can kind of inject our own information, right? So we can say this onChainStart decorator is going to add our own information, right? So we can say this on chain start decorator is going to add this extra information when the chat is starting up, and it's gonna be able to leverage that to do some things that we desire. But the idea here is all this decorator is, is it's basically just saying, hey, you know, whatever function follows this, I'm going to use that for my behavior. It's a little bit cleaner to see with onMessage. Basically, this is going to say, you know, onMessage, whenever we get a message, we're going to do all this stuff that's in this function, right? And that's it. That's all we do. stuff that's that's that's in this function right and that's it that's all we do um the basics of this are we're going to get a message and that's going to be a message and i know it's so you know the terminology is all the same word over and over again but the idea is the message is a chainlet message which is a particular object right and all we're going to do is stuff and then we're going to return something and the thing that we return is going to eventually be what the response is right that's it so now we do stuff which is a lot but it's not too crazy right it's in fact exactly what we saw before in the notebook we have a client and we use that client to interact with OpenAI. Now we're using this asynchronous client instead of just the client. If you have questions about what async means, please come to office hours. I'm not gonna spend time talking about it now, but it's a way that we can make sure we keep things responsive and we're not waiting around for things to finish before we start doing other work. And then this is just going to print it to our terminal so we can see it. Totally unnecessary step, but it's good to look at. And then inside, again, this is going to happen every time the user sends us a message. Every time they send us a message, all this stuff is going to happen in the background. So we open up our client and then we're going to create a prompt object now the prompt object is uh all this is doing is it's letting chainlit see more about this prompt so we can populate more information in its in its front end for us right that's it uh it's just like we're gonna package this in a certain way so that chainlit knows how to properly display it and knows how to let us know what's happening. Then all we're going to do is we're going to build an empty message that will eventually send back to the front end. This is what the user is going to see, right? And then we just straight up call the OpenAI async endpoint. We're going to use a lot of Python code. Basically, all this Python code is saying is, hey, for each, you know, for our input, which is going to be exactly what we have here, you know, send that input to OpenAI in the way it expects, which is in a list of different objects that have a role in some content. We're going to send all of those, and we're going to expect a streamed response. So you might notice that, you know, whenever you're talking to like Chad GPT, you're going to get like, it gives you word by word, token by token, right? All this streaming thing is doing is it's making sure that we just get like a bunch of tokens in a row, as opposed to needing to wait for it to generate every token, right? And then send the full block of text. Because you can imagine, right, if you're using Chad GPT, and it responds in a very long message, if you had to just sit there waiting for all that text to be generated, and then it showed up, it would not feel nearly as good as it would if it started showing you the tokens right away, right? Oftentimes, they're even generating faster than you can read. So it's all good. That's all this code does. So basically all it's saying is every time we get a message, that's all this on message does, right? Every time we get a message, we're going to get our open AI client. We're going to send our users input to the open AI client, and we're going to get a stream of tokens as a response. And every time we get a token from the OpenAI endpoint, we're going to send it to the user so they can see it in that chainlit UI. That's all that's happening, right? Like, it is a lot, but that's all that it's doing. We get the message from the user. We send that message along with some formatting that it's doing we get we get the message from the user we send that message along with some formatting that we're doing to create those those roles and to add that chain of thought prompt right and then we get a stream of tokens in response and every time we get a token from open ai we just we just send that right back to the user so they can get those tokens as well that's it these decorators if you can think of it as like interrupting the normal flow right if we didn't have these it would just do its own thing but instead we're going to tell them hey this is uh you know instead of doing what you would normally do do what i want you to do right which is in this case do this accept the user's prompt generate tokens send to user uh streaming tokens as they're generated. No, very, very specifically, the tool, the streaming is as the tokens are generated. We don't have to wait for every token to be generated, which makes it feel very snappy. So that's that. And that's great. And it's cool. And we love it. And it works locally, right? And you can wrap it up in this docker file the docker file i'm not going to go too deep into right now uh we can talk about this a lot more throughout our time together but all this is doing in essence is it saying hey you know uh we want a specific place or a specific environment to run our application. And we need a blueprint, right? We need instructions on how to build that environment. And you can think of that Docker file as a blueprint for your container, right? We're going to say, hey, start with the Python 3.9 foundation. And then we're going to add some user group stuff on top of that because Hugging Face requires it. And once you've done that, you're going to go ahead and if you could just take the requirements.txt I've given you and put it into this folder inside the container. And then inside the container, if you could go ahead and run this command, which is going to install all those requirements, right? Then copy the rest of the stuff I gave to you, and then go ahead and you're going to run this command inside that container. This is all it's doing, right? It's a blueprint. And that blueprint is in a standard format, so that when we send it to a container or we send it to a machine in AWS that has Docker or GCP that has, wherever has Docker, it knows how to read this blueprint, and it's going to be able to create that container for us. That's all the Docker file is doing. It's just a glorified blueprint. It's, it might seem confusing when you first start using it, but as you get more reps, you're going to, you're going to be able to really, you know, get a firm grasp of like, it's just a, just a list of instructions, just like any other kind of algorithm. Just a list you follow from top to bottom that tells whatever's building that container how to do so. So that's Chainlit in code. Looks dope. Now we want to be able to push this thing to the hub, right? So in the instructions, you're going to find that we're going to wind up with this local directory called beyond chat GPT. So I'm going to go ahead and open my terminal so we can get the terminal view here, right? And if we go to beyond chat GPT, right, what we're going to do, instead of kind of fiddling with files and all this other stuff, which is like, you know, it's kind of maybe it's not so fun. You know, it's hard to keep track of. What we're going to do instead is we're going to do something where we add our Hugging Face space as a remote, and we just push this pile of files to Hugging Face, right? And the way that we add the remote is just with get remote add, and then you name it. So we're going to name it hfrepo. Then you provide the address. Hugging Face gives it to you, and I'll show you an example of this in just a second. But the idea is very straightforwardly, we add it as a remote. We're going to make sure everything's up to speed by using a command that you'll find in the repo. So I'm not going to go through showing it to you now, but you can just copy and paste it in. Basically. We just wanna make sure that the hugging face remote server is synced up to ours. And then you can just push those, uh, push those files to hugging face. Right. And that's all you have to do. We have this local environment, and then we push all those files, including our Docker file, including our requirements.txt, including our Chainlit application. We push those to our remote server, which is our repository on Huggyface, and it's going to know what to do from there. So I'm going to swap over my screen for a second so we can see the huggy face side. And here we go. I'm going to go ahead and change this to here. Perfect. Okay. So this is what the chainlet application looks like once we're done. Just show you an example of how it works. How can I assess you today? How do i make really good lasagna my dude right we can send that message and that message is going to you'll notice it's all these tokens are coming in a big stream right you don't have to wait for that to finish and then we get one big chunk attacks instead we get rapid fire tokens right feels very responsive thanks chainlit and this is the idea that's all we're doing right if we look at the files that we have on this huggy face repository you'll see right their one-to-one match with what we were just looking at we have our docker file or readme or app.py or chainlit.md and our requirements.txt that's it right and it can use this blueprint of the docker file in order to build that container and run our application which we have here that's the that's the uh the basic idea the all you have to do basically is push this to hugging face but where do you push it is a great question and so what we'll do really quickly is we'll just look at how we would do that. So when we are in our, when we're on our profile, right? So I'll go to my profile. So it's exactly the same as what you'll see, right? And what we want to do is we want to create a new space. So we're going to go to our Hugging Face spaces, which is a new space here. Click the button. We're going to name it whatever we want. Like, you know, first LLM, let's go. And then I don't think we can have exclamation mark. We'll set a license. It's up to you what you want to do. This code's MIT, so you can do whatever you want. We click Docker. It's blank because we're bringing our own Docker file. Everything else is the same. We hit create space. And that's where we get this link. And this link is what we can use to add this space as a remote server to our Git repo that we'll then push to. Again, this is all spelled out in the instructions, but we just want to see it. And once you've pushed it, you it's hands off. The only thing you have left to do is to go to the settings, which you can access through the kebab menu and add your secret, right? We need it. We need a secret. Uh, so we'll have new secret. It's gotta be your open AI API key, right? This is it. And then you paste it on in. The only tip I'll give you here is definitely make sure there's no extra new line characters or spaces. It will complain at you and it won't work. So make sure it's just the string. That's the key. Once you've done that, you'll be able to access your Chainlit application just as though you were running it locally on Docker. Um, and, and that's it. Uh, that's the whole process. Obviously this was a bit quick guys. We're gonna spend more time talking about chain lit as we, as we progress forward, uh, and we'll spend more time talking about some ins and outs of Huggy face, but for right now, the idea is once we have it on our machine, we just need to put it into our Huggyface space, and we'll be able to enjoy it. And our chainlet application is just regular Python with some extra widgets, which is those decorators, which essentially let us interrupt the normal behavior of the chainlet application. And that's it, right? Again, just 15 or 16 simple steps, and you're away.
🧑‍💻🚀 Deploy Your First LLM Application with OpenAI, Chainlit, Docker, and Hugging Face
Take a peek into our AI Engineering Bootcamp, Cohort 1!

This is a deep-dive clip from Session 2 on building and deploying your first LLM application.

Chris digs into the details you need to know about building a Chainlit application with industry-standard best-practice tools out at the open-source edge!

Check out the full GitHub repo here:
https://github.com/AI-Maker-Space/Beyond-ChatGPT