 Hi everyone and welcome to Beyond Chat GBT Build Your First LLM Application brought to you by AI Makerspace and 4thBrain. My name is Greg Lockman and I'm the founder and CEO of AI Makerspace and a long-time instructor at 4thBrain. We appreciate you taking the time to join us for today's event. Please share in the chat where you're tuning in from today. We're so happy to have folks across our communities joining us from all over the world. During our event today, you'll learn how to build, ship, and share your very first LLM web application using the simplest stack of all, OpenAI's API, Chainlit, and HuggingFace. By inserting an LLM in between a front-end user interface and a super easy back-end serving and inference solution, you'll be amazed at how quickly you'll be off to the races building your very own LLM applicationsm applications all right let's get into it if you hear anything during our event today that prompts a question please follow the slido link in the description box on the youtube page we'll do our best to answer the most upvoted questions during the q a portion of today's event that, I'm pumped to welcome my good friend, colleague, and the LLM wizard himself, Chris Oleksiuk, to the stage. My man will be working as a team today to deliver this Code Plus Concepts lesson. Chris is the head of LLMs at AI Makerspace and is also a long-time fourth brain instructor. As an experienced online teacher, curriculum developer and YouTube creator, he's always learning, building, shipping and sharing his work. Chris, you ready to show him how to do some build, ship, share? You know it, Greg. All right, let's go. See you back on stage in just a second, Chris. All right, here we go. So today, we're going to sort of walk you through an end to end solution. And that's what we're going to show first. Chris is really going to kind of walk us through exactly where we can expect to be by the end of today's lesson. by the end of today's lesson. We're going to dig into each piece of the puzzle we need to get there. We're gonna talk about the actual developer syntax that you need to know if you're gonna interact with the GPT models as an LLM application builder. Then we also need to understand if we're going to put a front end on a chatbot or virtual assistant, how we can leverage a tool like chainlet and what kind of syntax we need, what is the structure and how does OpenAI's API really fit into that. Finally, we need to go ahead and deploy this thing, make it public and viewable and shareable by everyone on Hugging Face. We're going to do this through a very easy and quick Dockerized solution, where we're going to package up our chain-let application that's got our OpenAI connected to it, and we're going to make that available to everyone. Finally, we'll show you exactly how to sort of iterate and dial in and optimize your prompts once you're live and you're working with your application on Hugging Face. In the end, we'll take any questions that you have, and we hope to continue the discussion towards more and more advanced and more and more complex LLM applications. So first up, we're going to see what this thing looks like completely end to end with all these tools combined. So Chris, over to you. Thanks, Greg. Yeah, I mean, the idea here is that we're building ourselves a chat application that we can interact with through this UI. So we can ask questions like, you know, what is the difference between Langchain and Llama Index? And we can see the LLM processing our prompt. We can look at the response and see, you know, we have this step-by-step breakdown. It gives us some outlines of laying chain and Lama index. It doesn't actually know what these things are. And so it is a, you know, it's hallucinating quite a bit here. But the idea is that we have this chat application and this is in a hugging face space. So this is publicly available. You can go to this link and use this tool today. And, uh, you know, we have the ability to do a whole lot of very powerful things through the chain lit interface. We can check out our prompt playground, which we'll spend a little bit more time talking about later. We have a history of messages we can look at. And, you know, the thing that's running this in the background is OpenAI's GPT 3.5 Turbo model. And so we will go through the whole process of how we got from our first notebook to this app. And we'll take you through that whole course over the course of the rest of our presentation. But with that, we'll kick it back to Greg. Awesome. Thanks, Chris. Yeah. So let's go ahead and dive in and get started you know what we're really saying here is we're saying that the way that most people interact with llms is the not really the way we want to think about interacting with llms as developers and builders we can ask questions simple text in text out of course everybody is doing this today but what we want to do is we want to take it to the next level. We want to sort of figure out this first AI engineering Lego block, if you will. This first piece to our puzzle is this OpenAI API. This is going to get us moving down the path of becoming a truly state-of-the-art machine learning engineer today. So rather than text in, text out, what we want to do is we want to start with something called the chat style model syntax instead. And so rather than text in, text out, the chat style model syntax is going to leverage an list of chat messages in and then we're going to be able to output an additional chat message so each time we're interacting with the API we're not simply putting in new text but rather we're putting in a whole list of messages and this might start off as one message and then grow as we have a chat we're not simply putting in new text but rather we're putting in a whole list of messages and this might start off as one message and then grow as we have a chat but we also might want to include messages for lots of other reasons as well what kind of reasons might we want to include other messages well we can gain some insight into this by looking at the types of messages we'll want to include in our inputs. These include three specific types of roles. The system role, the user role, and the assistant role. If we look at this example on this slide as a first cut, we can say that, you know, the system role is saying, OK, you are a helpful assistant. The user role is prompting who won the World Series in 2020. And the assistant role is answering the question, the Los Angeles Dodgers won the World Series in 2020. in 2020. And you can see that subsequently the user then asks a follow-up question, where was it, referring to the World Series in 2020, played? So we can start to get an idea of these roles by looking at this example, but diving a bit deeper on what these roles mean is really the way to think about the system role is this is sort of the context. It's the specific instructions that are guiding the LLM and setting our real objectives for what we want to get out of the LLM. The user is sort of, you can think about it from the user's perspective. It's simply interacting with the application so you're providing an input and you want to get some sort of output the user is leveraging the application the user while you're building that's just you but when you're finished building and you're ready to share that's actually your user and that's an important distinction when we think about building these things out ourselves. When we go to the assistant role, that's sort of the assistant perspective. This is where you are now acting like the virtual assistant, or in some cases, it's just simply the AI or the LLM responding to you as an AI, as an assistant. So let's see how this works in the simplest possible case, which is I'm simply prompting the LLM very, very similarly to the user role only. So I'm just engaging with the LLM. And this is going to be a very simple syntax here. You're defining the role. you're using a prompt similar to the way you would with chat GPT what you get out is much more interesting here what you get out is a bunch of information but sticking with the idea of roles here we not only get the output response in the content area, as we can see right here, but we also get the role assigned to the response of assistant. In this case, the assistant is actually answering our question. And so this is the first cut way to think about engaging with the OpenAI API. The next layer of this is gonna be when we add system prompts to provide some additional context to our message and our chat message list. And for this, we're gonna go back over to Chris to show us what it looks like in code. Chris. Thanks so much, Greg. Yes, we're gonna be checking out how to interact with the OpenAI suite of models programmatically. So we're going to use Python, and we're going to do this in a Colab notebook that you can access if you want it from the – there should be a link in the chat. chat. The idea here is that we're going to use the OpenAI Python library in order to send requests to those endpoints and get responses just like we saw Greg demonstrate. The first thing we'll need to do is set our OpenAI API key. If you're stuck on how to get that, you can follow the instructions at this repository link where we go through in detail how to get a API key. We're going to talk about kind of this idea of chat completion with these three roles. The idea here is that we can act as these three entities when we're sending prompts to the LLM. And we want to leverage that to do things like, you know, set up different patterns or ask the LLM to act or behave in certain ways. So let's take a peek at how we might do that. First things first, let's just send that prompt we saw in the slides. So we have our string, which is what is the difference between Langchain and Lama index. We are going to use the OpenAI chat completion endpoint, and we're going to create a request. The model is going to be GPT 3.5 Turbo. And then our messages is going to be a list. and we're going to send it just one item in that list, which is going to have the role user, and then our prompt, which is what is the difference between Langchain and Lama index. We can look at that big blob that Greg showed us in the slides, where we can see which exact model it used, the choices we have in terms of our response, the reason that response finished, and then our usage. So we can do things like track, very fine-tuned track our token expenditure, average tokens per request, and all kinds of different fun things. There's a ton of information though, and we have to set up these dicts that have a role in our content. So what we're gonna do is just build some simple helper functions to both kind of get us in the idea of using these kinds of functions to set up our prompts, as well as make things display a little bit nicer, we're going to have a pretty print method that lets us actually display our results in Markdown in the output cell. So we can look at the results as we're hoping to see them. You'll notice that each of the system prompt, the system prompt, and user prompt just wrap our message in the respective role. So let's give this a shot and see if it works. We're going to pass our list of prompts, which in this case is just a user prompt. It's the same one as before, and we can see here that we have a response. We're no longer printing that big blob, So we're just looking at that text. Now we're going to focus on modifying this a little bit. So we're going to add a system message as well. The way you can think about that system message is like an overarching idea that you want the result to adhere to, like an instruction or a way that it should behave or act. So you can see we have our list of prompts, which we're going to send to our completion endpoint. And we have a system prompt, which is you are irate and extremely hungry. And then our user prompt with, do you prefer crushed ice or cubed ice? And then we can get our response. And we can see, you know, it's pretty angry. I don't give a damn about ice right now. I'm starving. What's even the point of asking such a ridiculous question when I'm practically about to pass out from hunger? Just give me some food already. Very, this response sounds both irate and extremely hungry. And this is the idea of the system prompt. So let's just change the system prompt. As you can see, we're only modifying the system prompt in our list of prompts. And let's change it to, you are joyful and having an awesome day. And you can see here that the response is very different, which is I absolutely loved crushed ice. It's perfect for keeping my drinks extra cool and refreshing. It always adds that little bit of joy to my day. The idea here is that with just changing our system prompt, we can really control how that output acts and the kind of tone and delivery that it uses, which is a powerful feature. We can, of course, still access the response object and look at our completion, look at the number of tokens we used, but, again, we're just printing it in this pretty format so that we can see it the way we'd expect to see it if we were in our chat application. And with that, we will kick it back to Greg to learn what more we can do with prompting. Yeah, thanks, Chris. Super interesting to see how if we're telling the LLM that it's hungry or if we're telling it that it's joyful, that really shines through in the response. And it begs the question, like, what is the role of prompting and prompt engineering here? And in order to really contextualize this properly, we should kind of return to the basic best practices or rules of prompting. Number one, we always want to be clear and specific in the instructions that we're providing. We really want to guide the output as we did with the mood or the feeling in the last example. We also wanna be providing that context, that role, that persona, a place to stand. You know, we saw earlier, you are a helpful assistant. You know, you are an extremely astute educator. You are X, Y, Z. You really want to give it a place from where it should be providing that answer. And we saw that in this last case as well. We also, in the format we were asking for out we didn't we didn't do this explicitly but we sort of did a two-step to make sure that the output was in sort of pretty print format. So you know the way that our user kind of sees and can engage with this we're hitting a number of the top rules of prompting and prompt engineering that we always want to keep in mind. The last one that we're really not hitting yet, but that it's time to talk about, is really dialing in the input through few-shot prompting or through chain of thought prompting. And it's worthwhile to just sort of review what these are in general. If we look at kind of the paper that each of these ideas came from, the paper for few-shot prompting was called Language Models are Few-Shot Learners, very famous paper from the team over at OpenAI. Then what we see in this example that they sort of started with is they're teaching the language model new words a what poo is a small furry animal native to Tanzania an example of a sentence that uses the word what poo is we're traveling in Africa and we saw these very cute what poos that's our example We got our input, we provided an output. Now we set it up so that's our one-shot example and we ask for a subsequent output with just this input. To do a far-duttle means to jump up and down really fast. An example of a sentence that uses the word far-duttle is. So here we have sort of a one-shot or let's say few-shot prompting example. You can provide one shot, two shot, few shot, many, many shots. And this is really a key aspect to dialing in the output of your applications. If we take it to the next level, we can not only provide an input example and output example response, but if we provide an input example and output example response but if we provide an input example and output example response we might want to in the output example response give some really key details on why that's our response so in this case you know the standard prompt gives the example roger has five tennis. He buys two more cans of tennis balls. Each can has three tennis balls. How many tennis balls does he have now? The answer is 11. And this is from the original paper that brought up this idea of chain of thought prompting elicits reasoning in large language models that totally would encourage all of you to check out as well. The way to take this initial example, this one-shot prompt example to the next level, is rather than just say the answer is 11, say Roger starts with five balls, two cans of three tennis balls, each is six tennis balls, five plus six is 11. Therefore, the answer is 11. So by providing this sort of idea to have the model think through step by step, what the reasoning is to get it to that answer, then the few shot prompting can be a much more powerful strategy. The way this works in the OpenAI framework with the system, user, and assistant syntax is what we're going to see in code next. From Chris, back to you, my friend. Thanks, Greg. Yeah. So, let's explore how we would implement a few shot example. We're going to use basically the same example that you can see in the paper. Language models are few shot learners. The idea here is that we can use that assistant role to kind of pretend to be the output of the LLM. And that's going to let us really cleanly set up these few shot examples where we show similar to like a conversation history, right? We show the user's initial query, and then we show a response, and then we show another query, and then we let the model use that previous chat history, you can think of it, to inform its answer. So let's just, again, we're just going to use some nonsense words. So we have the prompt, please use the words Stimple and Falbean in a sentence. You can see this is our list of prompts. So we're going to pass this and get a response. ChadGBT, or in this case, GBT 3.5 Turbo is absolutely not sure what those words mean and so refuses to do what we ask. Though technically this is using those words in a sentence. Good job on a technicality there, GPT 3.5 Turbo. But we can use the few shot prompt pattern to actually, you know, effectively teach it these new nonsense words. So we have our user prompt now, something that is Stimple is said to be good, well-functioning, and high quality. An example of a sentence that uses the word Stimple is, and then we input, we kind of, you know, disguise ourselves as the LLM here, so the assistant prompt, and give a response, boy, that there is a Stimple drill. We then set up another query, which is a Fallbean is a tool used to fasten, tighten, or otherwise is a thing that rotates slash spins. An example of a sentence that uses the words Stimple and Fallbean is, and then we're going to allow the chat GPT 3.5 turbo to actually complete this by offering its own assistant prompt. And you can see that we get back a sentence. Wow, this simple fall bean is a game changer in the construction industry. And, you know, if you look at what we assign the definitions to be, this sentence makes absolute sense. assign the definitions to me, the sentence makes absolute sense. And while this is obviously kind of a playful example, it's a very powerful pattern in terms of getting your outputs to match to a specific, you know, either format or to inject some kind of novel information or allow the language model to come up with better results, you know, showcasing a pattern to it. Absolutely powerful technique. Next up, we have our chain of thought prompting. Chain of thought prompting is a very powerful technique because it lets the model take time, as it were, to think through its responses, as Greg was explaining. So we have this reasoning problem, right? Billy wants to get home from San Fran before 7 p.m. Eastern Daylight Time. It's currently 1 p.m. local time. And then we have some options of travel. So we have a fly option and we have a bus option. And then we have a teleporter and bus option. The idea here is that, you know, we're tricking it, right? It's currently 1 p.m. local time is an attempt to trick the model. So though it's 1 p.m. in Pacific time, it's actually much later, Eastern Daylight. So we're going to see if it notices, if it picks up on that. And the answer is it doesn't, if we just leave the prompt in that normal format. But if we add just this string to the end of our prompt, which is think through your response step by step, string to the end of our prompt, which is think through your response step by step, you can see that the answer, it gets it correct, which is that, you know, it does matter which travel option Billy chooses. And it showcases the entire logical flow, including this conversion between, you know, the two time zones. And so this is a very simple pattern that is extremely powerful to allow your language model time to think through and develop, you know, quality responses. And with those two demonstrations out of the way, we'll kick it back to Greg and learn what we're going to do with this. Yeah, thanks, Chris. So the prompting strategies that we just discussed are going to be essential to really making it easy for users to interact with our application. All of this together forms what we can call a prompt template, and that's going to really be a key piece of the way we build our application for our users. When it comes to building a chat application, this really is the piece that's going to give us that front-end developer role and vibe. You know, we can, with just some simple Python code, we can create very, very powerful UIs. As we've seen, we're going to show you exactly how to do it now with a tool called Chainlit. Chainlit is really cool for a bunch of reasons. Number one is that it's crazy easy to implement in Python. And that's where we're going to start today. Number two is that we can use what's called their prompt playground. That's a really, really robust GUI and set of tools for us to dial in our prompt and our strategies for prompting. So what we can do is we can look at all the different settings on the OpenAI API. We can mess with our user, assistant, and system roles. We can provide one-shot, two-shot, few-shot. We can add chain of thought reasoning. There's lots of great things we can do within the Prompt Playground. Now, the additional thing that's great within Chainlit, that's a bit beyond the scope of today, but look for in future lessons from us is the chain of thought, reasoning, GUI based experience that is provided by Chainlet when you're actually building with some of the more complex infrastructure tools that Chainlet has integrations with like Langchain, Llama Index, and even now this week, Haystack. so today we're going to focus on the simple Python code and just the prompt playground to get started with showing you how easy this thing is to get up and running you can do this in real time with us if you open your terminal now Chris is going to show us how simple the python syntax is to get this thing up and going and really just to get something you can start interacting with before we layer in our application logic. So Chris, Chainlit hello world time over to you. Yes, thank you, Greg. So the idea here is straightforward. We're just going to go through how we can start using Chainlit and a little bit about what it can do. You'll notice that the command I typed is pip install chainlit. This is just to install the chainlit package in my environment. Now that I have it installed, I can run chainlit hello world, and that's going to start up a process, which we're gonna be able to look at in VS Code. You can see here that the interface is exactly what we saw in our hugging face space. We're going to do a brief tour just to make sure that we're all, you know, on the same page about what this is doing. So first of all, we obviously have our chatbot. We have whatever it's going to say, and we have a big text box we can type something into. So let's type hello and send that. You'll notice that it immediately responds with just some boilerplate. Again, this is a hello world example. It's just meant to let the system run, and you can see it's working, and we can start building our own Chainlit apps. That's fantastic. We'll also take note of a few other important features. We have a history of messages that we've sent through the Chainlit application. As well, we have a readme, which we can set up to explain about what our application is doing or any other important details we might want to include in a readme. We can start a new chat with the new chat button. We can change our settings from day mode to night mode. And we did all of this with just Chainlit. This is just Chainlit running by itself. We haven't modified anything. We haven't written any code. This is it. It's a very powerful, you know, front end for our application that we're going to modify in the coming steps. But before we do that, I'll pass it back to Greg and we can hear what exactly we're about to do. Awesome, Chris. Yeah. So we're going to take this really powerful front end that basically is super easy to get up and running and we're going to layer in some logic to this because you may be asking yourself was it really that easy to get our first llm app going and it's pretty easy but it's not that easy there are a couple other steps that we need to integrate here. Number one is we're going to have to get our imports. Number two is we're going to set up our prompt templates. That's what we just talked about with the system and user and assistant roles. And Chris is going to show you exactly what that looks like in code. But the key pieces of our chainlet application are going to be two decorator functions or two sort of meta functions in Python. One of them is going to be called on chat start. So as we're beginning the chat in this case within on chat start, we're going to set up our open API API that includes factors like temperature and the max number of tokens that we're willing to output. And we're also going to now begin our list of chat messages. And when we get to the on message decorator function within our chainlet application, we're going to actually call the LLM. That's where we're kind of doing the thing that most users who interact with ChatGPT, they're simply doing. This is where we're actually leveraging the LLM. This is important to keep in mind, especially as you build more and more complex LLM applications, because this actual calling of the LLM comes in much later than many people think as you're building some of these more complex retrieval applications and other applications with the integration tools that we discussed earlier, Langchain, Lama Index, Haystack. So once we set up our app.py file, what we're going to do is we're going to be ready to ship and share this thing. And we're going to do that. We're going to actually deploy this thing on Hugging Face. And once it's deployed, we're going to be able to play with the prompt playground and iterate on our prompts directly within the Chainlit application. So we're going to deploy using a Hugging Face space. These are some of the most famous spaces that you've probably seen up on the Hugging Face community right now. What we're going to do is we're going to create a similar application, just like the one we saw Chris show early on in the demo today. The way we're gonna do it is we're actually gonna leverage a Docker file approach. Now, the Docker file that you need, we have ready for you in the repo that we'll share in the chat with you now. It's in the beyond chat GPT repo. You don't really need to worry about the details of the Docker file, and you don't really need to worry about the details of Docker in general in order to get this thing up and running. The big idea is it's simply a way to package up our application and our front end in a nice portable box that we're then shipping into our Hugging Face space. Once we have our Docker file, the only other piece we need to add here is we need to add our OpenAI API key, and we're ready to go. We can ship this thing and actually interact with it and engage with it live. We can share this with our brother, our sister, our boss, our community. We can actually share this app that we just built. So this is really where the magic happens. One last step we're going to take in this final piece of the demo is once we have our application up and running, we can use a very, very powerful tool within Chainlit called the prompt playground to really dial in our template. As you see here on the screen, our system user and assistant roles here. And we can also change the settings on the open AI API. As you can see in the right-hand side of the panel, all this is again, without any Langchain, Lama Index or other infrastructure tools. And this is done very easily by clicking this little edit button within the chat as Chris will show. So with that, I'm gonna kick it back over to Chris to show us how to containerize, deploy, chat and optimize, bringing it all together. Chris. Thank you, Greg. Yes, so as you can can see we're sharing the whole screen here. We've got both our browser and our VS Code open so we're about to put this all together. First things first, in the repository we're going to find an app.py file. This is going to be the core logic for our application. So we'll notice a few things. First of all, we have a lot of imports. Some of these imports should hopefully seem fairly self-explanatory. We need OpenAI. We need Chainlit. Makes sense. But you'll also notice that we have this prompt and prompt message object from chainlet as well as a chat open AI tool again from chainlet. The first thing we're going to do, as Greg said, is set up our system templates. This is basically our system message. So this is what we're going to use our system message. You can edit this however you'd like, add whatever flavor or context you wish. But this is where that magic is going to happen. The next thing we're going to do is set up a user template, which is going to accept some user input. And then we're going to add that chain of thought prompt to the end so that we get those quality, well thought through answers that we expect. Now, like Greg said, there are two main decorators we're going to be using in Chainlit. The first, and a decorator is just like a way to talk about, you know, something that you can wrap your function in that's going to change what it does or let it interact with a different system. In this case, we're going to use it to interact with the chainlet front end that we saw earlier. So you can see we have this on chat start decorator where we have our settings. This is the model settings we're going to be using today. These are all the different things that you can change. the model settings we're going to be using today. These are all the different things that you can change. You'll notice that we're using a temperature of 0, which means that it's going to be fairly non-creative responses. We have a max tokens of 500. We're using a top P of 1 with no frequency or presence penalty. All of these parameters influence the kinds of generations that we're going to get from our model. So it's very important to play around with these, experiment, and discover what's best for your application. Once we set those settings up, we're going to store them in our user session. This is a session state that persists between different components of our chainlet application. It's important to use this session, user session, to pass information between these various decorators and their functions. Next up, we have our onMessage decorator, which is going to trigger on a message. Right? Makes sense. We're going to collect our settings from our user session, and then we're going to set up this prompt. Now, in Chainlit, a prompt is exactly what you would think it is, right? We have basically, it's a way to collect text in a way that lets it interact properly with the Playground as well as OpenAI. You'll notice that we have our provider, and our provider is ChatOpenAI. This is how we determine which Playground to use. Then we have our messages. This should seem familiar. We have our prompt message with the role of system, where we're going to pass our system template into our formatted prompt. And then we have a prompt message with the role user. And this user role is going to use the formatted prompt of the user template, but we're going to add the input to that template using the dot format method. So the idea here is that we have this template above, which includes this formatable component so that we can inject our user's message into that component through this prompt message. We're going to specify that our input is the message. I'm very sorry to click on these variables. The input is the message that we're going to get from our user, and then we're going to pass our settings. Next up, we're going to set up just the streaming half of Chainlit. I won't go too deep into it. Suffice to say that it lets us see the response flow onto the screen as opposed to waiting and then just dropping a whole blob all at once. You'll also notice that we're using again that OpenAI chat completion method that we saw in our notebook, and we're passing in our messages, our list of prompts. And we're letting the stream setting be set to true. And then we're passing in the settings that we set on chat start. The rest of this is, again, we're going to append our chat history to our prompt. And from there, it's just chainlit boilerplate to send the message back to our user. So let's actually get this thing going. We'll head into GitHub and we're going to rely on this Docker file, which is, you know, again, like Greg mentioned, we've just built this. It's there for you to use. It's there for you to use. You'll notice some additional user permissions, you know, kind of commands. This is because of Hugging Face's specific Docker deployment strategy. But enough of that. We will start with cloning our Beyond Chat GPT repo. We'll cd into that repo. As you can see, if we type get status, we are in this repository now. Now that we're here, we want to add a remote. And the remote we're going to add is a Hugging Face space that we need to create through the Hugging Face UI. And the way to do that is to go to your organization or profile, click new, space. We're going to name this Beyond Chat GPT Demo. We can use whatever license we need to based on the libraries that we have included in our application. So select whichever license is most appropriate for your task. We're going to select the Docker template. We're going to use a blank Docker template. We're going to use the basic hardware, which is free, which is fantastic. And we're going to have this set to public. Once that's done, we can create this space. And now you can see that we can clone this space to get started. We're going to do a slightly different flow, which is adding it as a remote, which we can do here. Now we're going to pull in that remote. We're going to not allow a rebase. So we're going to set the merge strategy for no rebase. And we have to include this allow unrelated histories tag. This is just so that we don't get any complaints from Git when we do this process. Once we've done this, you'll notice that we have some merge conflicts. We can navigate to our readme, which is, in this case, what's going to be the source of those conflicts to accept the current change in both cases. Then we're going to go ahead and save our readme. Now that we've done that, we can head back to our terminal and we're going to git add dot. So we'll just add everything that's in our present repo. Then we're going to git commit dash am releasing app to HF space. Basically, what this is doing is it's just adding and committing any changes as well as adding this message. Now that we've done this, we are good to push our changes to this HF space on the main branch. You'll notice that as soon as I did this, my web browser changed as we're going to start building our actual application. Now, this won't let us use our application because we have to add a secret. The way that we do this is by going to the secrets tab, which can be accessed through the kebab menu, going to settings, and then scrolling down to our secrets and adding a new secret. We're going to add the OpenAI API key, which we can do again by clicking new secrets, pasting it in there. And then I'm going to actually complete this step in another window so I don't expose my OpenAI API key. But the idea here is basically we're going to set up an environment variable so that we can use this application. You'll notice that this is building. Once it's completed its build, we'll be able to use our application the same way that we saw at the beginning of this process in our end-to-end example, which is already loaded and running here. Let's take a look at a prompt. Just say, you know, what is the, let's say, what is four plus five times six. Remember that we're using that chain of thought prompt that we added to our app so that we can leverage that chain of thought style. We also get access to this prompt playground. The prompt playground is a very helpful tool. It lets us see what the system prompt was, what the user prompt was, and what our response was. We can also check and see what exactly it got by clicking the formatted tab. This is going to show us what was formatted and it's highlighted, in fact. We can select a variable in case we had many variables. We can change this variable if we wanted to. And then we have the ability to submit this. We're going to get a new response. And this is going to be something that lets us play around with our prompts. We can also click the little settings button and we can select which model we want to use, the temperature that we want to use, the max tokens, all of those settings that we saw previously. The idea here is that this is a full playground that lets us really, you know, figure out how to best format our prompts. We can do things like removing the actual, you know, chain of thought prompt, and we see that our answer is way less verbose. Anyway, it's a very powerful tool, and it comes built stock into Chainlit. And that's what makes it such an excellent platform for creating, playing with, iterating on and, you know, really, really perfecting these output applications. And with that, we'll send it back to Greg. Wow, Chris, that was awesome, man. Thank you so much for showing us how to containerize, deploy, optimize our prompts. That really brings it all together. And with that, that's your first LLM application. Congratulations. That's all it takes. And we'd love for you to share your first LLM application with us as you move forward and really get going building, shipping, and sharing. In conclusion, prompt engineering is very much still important. It's essential. It's integrated directly into any application that we build. And we need to be very aware of the way that our user is going to interact with our application and we need to be sitting there providing the backstops through our prompt templates making sure that we're giving specific instructions making sure that we're providing context we're doing one shot two shot few shot with chain of thought reasoning and we're doing one shot, two shot, few shot with chain of thought reasoning. And we're providing the details on the kind of output that we want to give within our application. We also have found out that it's really important to get to know chat model syntax or the chat completions model from OpenAI. This is not only important to build with OpenAI tools, it's actually important if you want to build with Langlit, package that up, deploy it on Hugging Face, and you're off. You can now build these applications for your particular context and your particular situation. And so with that, we're going to go ahead and move to the Q&A portion of today's event. Chris, come back up on stage and let's hit it. We're opening up the floor for questions from the audience. If you have questions that have come to mind, please drop them in the Slido link that we will put in the YouTube chat again. And once we run out of questions, we're going to wrap up for today. Chris, first up, we've got, can we get public datasets the same format as you're showing us right now, System User Assistant, which we can match to our target application? Do these exist? I mean, you can in some senses. Yeah. So I think if I understand the question correctly, InstructTune data sets are exactly this. They have that information in exactly this kind of format so that you can fine tune your, uh, whatever LLM you're using to use the, uh, use this similar kind of, uh, format of having that system user and assistant. Uh, though you, you might find that the language for system user and assistant is very non-standard. Uh, it might be like instruction input output or whatever it happens to be, but yes, you can absolutely find data sets that match this format. Yeah. Yeah. DALI 15K comes to mind. What else comes to mind today for you, Chris, in terms of specific data sets for them to look at? Like the originals, Alpaca, the data set that kind of kicked us off into this open source journey recently and then derivatives of it. There's also the the the Flan dataset, which is many millions of tasks. So you have the option to there's there's a lot of good options for these. Bloom XP3 comes to mind as well. Yeah, this is this is a question there's a lot of good options right these little max v3 comes to mind as well yeah this is this is a question we get a lot in the courses that we've been teaching so yeah very very important idea this idea of instruction tuning and it fits in with a lot of what we're talking about today um all right so does chat GPT 3.5 turbo analyze images? No. No, it doesn't. Hopefully we're getting that GPT 4 multimodal release at some point in the not too distant future. That'd be cool. That'd be cool. All right. Next up, we've got, is Chainlit also compatible with other open source LLMs like Llama or Falcon? Oh, you betcha, yes. Absolutely. So it is compatible with all, anything that you can put into Python, technically you can put into Chainlit. It's compatible. If the LLM is compatible through Langchain or Llama Index, it's a very easy lift to get it to working in Chainlit. But even beyond that, you can set things up in such a way that it's going to be able to be leveraged. But yeah, absolutely. Chainlit itself is LLM agnostic. Yeah, yeah. So what would the flow be? What would you need to change in the application we did today to sort of insert an open source LLM rather than OpenAI? Yeah. So, I mean, all of the places that we reference OpenAI kind of in the settings, in the provider for our playground, for the actual responses that we get. So that open AI chat completion endpoint, those are the things we have to plug and play with our open source LLM. Again, the easiest way to do this is going to be through a framework like a langchain where you can just, you know, use it as an LLM object and it's built the, the compatibility. So I would wrap my open AI or my open source LLM in a Lang chain LLM, and then use it as any other, uh, Lang chain LLM in. So you're saying that even going to just try to make this open source, it sort of requires a slight next layer of complexity added to our LM application infrastructure. I would say for the most part, you wind up in that situation. Yes. Yes. Okay. Okay. Very cool. Very cool. All right. So we've got a question. Can you track conversations with multiple users, eg user zero, user one, assistant, et cetera? user 0, user 1, assistant, et cetera? Yes. You would have to do that not in Chainlit. So you would have to have a different service to do that. But you can. That's not to say that you need, like, another application. It's just going to be additional logic in Chainlit. I believe that there's a cookbook example from Chainlit that shows two different LLMs returning the responses at the same times. But in terms of anything to do with like tracking or, you know, these kinds of things, they're going to require you to engineer a solution or attach a pre-engineered solution from a different library. Got it. Got it. Okay. So if I build an app using this stack at work, will my prompt inputs and outputs stay private? How does this work exactly on the privacy front? So your inputs and outputs, they aren't by default stored or monitored or tracked in Chainlit. So in some sense, yes. There's a possibility that the history could leak your inputs across different users, though it shouldn't. I'm just not, I'm not going to say 100% it won't, but it doesn't actually keep a record of your outputs that are visible to others in that sense. Yeah. Yeah, and I think we were having this discussion recently, Chris, of your outputs that are visible to others in that sense. Yeah. Yeah. And I think we were having this discussion recently, Chris, the OpenAI API, if you're hitting the API, the language I believe they use is that it's not going to be used to train future GPT models. Your data is not gonna be used to train future GPT models. Whereas if you're just leveraging ChatGPT, it's highly likely that your information is going to be used to train future GPT models. Whereas if you're just leveraging chat GPT, it's highly likely that your information is going to be used to train future models. Yeah. So if we're talking about specifically the interaction between open AI, yes, the API access means that you by default opt out of using that data for training. Everything you put into ChatGPT is used to train. And without opting in, nothing that you use through the API is used to train. So that's a good way to think about it. Absolutely. But the information is still passing. It's definitely not private. It's just not being used to train. Right, right, right. Not being used to train. Yeah. so that's kind of an interesting application you know if that's of interest to you to interesting reason sort of go build one of these yourselves okay so uh next question is there a way to introduce versioning in index i.e i want to have docs for different releases of python libraries and the assistant ask for the version that I'm using at the moment. Versioning and index. I'm not sure I fully understand the question, but you can absolutely set up logic to make decisions based on the version of Python you're using or the version of an index based on some tracking. If you're using Lama index, I know there's like a native integration with wand B, so weights and biases that will actually let you build a version index. Uh, so you might want to leverage that tool to, to help keep track of your, your indices, otherwise, uh, traditional kind of data, uh, lineage solutions like your, you know, DAGs hubs and, and whatnot are going to be the way to go. All right. All right. Um, is this all doable in English only? Any other languages supported? How can we build an app for non-English language? So because it's LLM agnostic, whatever LLM you want, you can plug in. That includes non-English LLMs. Most of the larger closed source LLMs do have fairly decent language support, but yeah, the lowest hanging fruit is to use a model that's tailored designed for whatever language you're trying to use. All right. A couple of quick questions to close us out here today. What's temperature in the setting exactly? Temperature is, you could think about it like as creativity. So, you know, what happens when we are trying to do an output? There's kind of like what should come next. So the most likely next token. Temperature is a setting that says, what if sometimes we just don't choose that one? We choose a different one, right? And so it introduces some noise into our generation that can, you know, it lets the model act more expressively and creatively. It's a great thing for creative generative tasks, but it's maybe not the best for really systems where you want it to be very extractive. So, you know, for maybe summarization tasks or tasks relating to retrieval and question answering, we might not want the LLM to get too creative, but basically what it is, is sometimes it won't choose the most likely next token. Very cool, very cool. All right, Chris, well, I think that about wraps up the Q and A portion of today's event. That was super awesome. Thanks for all the demo vibes today. Thank you everyone for your participation. This brings us to the end of today's event, which has been brought to you by AI Makerspace and 4thBrain. Our AI Makerspace community is still just getting started. We just finished our very first LLM Ops, LLMs in Production Cohort 1, and it went great. The latest version of our curriculum for advanced developers includes deep dives on the latest and greatest tools for building truly complex LLM applications from Langchain and Langsmith to Lama Index, Weights and Biases, and much more. Cohort 2 launches next Tuesday, September 19th, apply today. And for those of you who enjoyed today's demo, but still feel like you're really just getting started building with LLMs, we highly recommend that you check out Fourth Brain's upcoming Building with LLMs course. This is a more entry-level course where you'll learn the basics of fine-tuning, of laying chain, and of really how to take a simple application like we saw today to just the next level, rather than deploying it and operating it all in production environments and at scale. This course from Fourth Brain serves as a perfect precursor to AI Makerspace's LLM Ops, LLMs in production course. Also, please give us a follow at AI Makerspace on LinkedIn and YouTube and at AI Makerspace on Twitter to stay up to date on what we are building, shipping, and sharing. And until next time, we hope that you keep building, shipping, and sharing. We'll see you all next time. Bye guys. See ya.
Beyond ChatGPT: Build Your First LLM Application
GPT-4 Summary: "Launch Your AI Journey: Beyond ChatGPT Workshop Awaits! Ready to build your first Large Language Model (LLM) application? This workshop is your gateway! Discover the essentials beyond classic MLOps, delve into OpenAI's API, and deploy GPT-4 effectively. You'll learn to craft a smooth 'chat-style' interface using Chainlit and perfect your prompt engineering skills. Plus, we'll guide you on making your application public via Hugging Face Spaces. Ideal for data scientists, ML engineers, and anyone eager to create chatbot-style LLM applications. With OpenAI API, Chainlit, and Hugging Face Spaces, you'll be showcasing your LLM project in no time. Join us and step into the world of LLM Ops with ease!"

Event page: https://lu.ma/firstllm

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/upxYiBkzrtgSob4pQe9hFQ

Speakers: 
Dr. Greg Loughnane, Founder & CEO, AI Makerspace
https://www.linkedin.com/in/greglough...

Chris Alexiuk, CTO, AI Makerspace
https://www.linkedin.com/in/csalexiuk/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

Apply to our upcoming LLM Ops: LLMs in production course on Maven today! 
https://maven.com/aimakerspace/llmops

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/ZAHhGuYVU2c3znbq7