 Hello, welcome back. So far in this course, you've learned about model registry, specific types of automation, including webhooks and waits and buys as launch. And now we want to help you apply these tools to a real project. And for this, my colleague, Doric is going to be walking you through a case study. And Doric is actually a really impressive person. So first of all, Dharik is a Kaggle grandmaster. He is really good at machine learning, but also he has many years of experience in industry, applying machine learning in production. And also he's one of my favorite teachers. He's taught courses with me in the past, especially courses on weights and biases. And so I really think you're going to enjoy this lesson. Thanks a lot, Hamer. I really appreciate this introduction. So again, I'm a machine learning engineer at Weights and Biases. One thing that we do at Weights and Biases is we work a lot with our customers. a lot of customers are now very excited about training and fine tuning llms in fact we have a separate course about training and fine tuning llms that we will link below this video and that we definitely recommend everyone taking but we also want to take this use case of fine-tuning a large language model into this course and show how to set up model management processes that will support both training and evaluation of LLM models. And since this is a case study, we will take an existing dataset called Alpaca. It comes from Stanford. called Alpaca. It comes from Stanford. It's a synthetic data set that was created with an LLM, Text DaVinci 003, and it contains several thousand of, in fact, around 50,000 of instruction following examples. And in the original Alpaca paper, the team has fine-tuned a 7 billion LLMA model on this dataset. In our case, since we are trying to be more efficient here, and we want to also make it easier for people following the course, we'll be fine-tuning a smaller, a tiny LLMA model that has around 1 billion parameters. The important thing when fine-tuning LLMs is evaluation. And in this case study, we'll build on the concept of using LLM as a judge. So we will take the generations from the model that we fine-tune, we'll compare them to generations from a model that we defined as our baseline, and we will use another LLM, in this case a more powerful LLM, which is GPT-4, to compare which one is better. And this will result in a metric that we will use to make decisions if the new model that we train is better than the baseline, and if we want to proceed further with quantizing it, preparing it for deployment, and ultimately deploying it into production. And with this, deploying it into production. And with this, I want to now transition into the live demo session and I want to encourage you to work along with us. Check out the repo that we will provide for the exercises. Follow along and practice setting up this model management processes together with us.
LLM case study overview
We welcome another Weights & Biases guest instructor, Darek Kleczek! Darek is a Machine Learning Engineer at Weights & Biases and a Kaggle Competitions Grandmaster. 

Darek will introduce a case study which will allow us to experience model management and automations while finetuning and evaluating a Large Language Model. 

Resources:

Training and Finetuning LLMs course  with Jonathan Frankle (MosaicML), Weiwei Yang (Microsoft Research), Mark Saroufim (Meta) and Darek Kleczek (Weights & Biases): https://www.wandb.courses/courses/training-fine-tuning-LLMs

Alpaca: A Strong, Replicable Instruction-Following Model paper: https://crfm.stanford.edu/2023/03/13/alpaca.html
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena paper: https://arxiv.org/abs/2306.05685