 This is a course about model management, but to manage models, we first need to train some models. And this is what we're going to do in this lesson. This is the train.py script that you can access in our course repo. And we will share the link to the course repo below this video. And let's take a look what's happening here. In this training, we will be using a dataset called Alpaca. And this is a dataset for fine-tuning language models on instruction following. So let's take a look at this dataset. I like to see the data before I train my models. Let's take a look at this dataset. And you've already seen this artifact view. We are using artifacts in weights and biases to manage and version our datasets and models. In this case, this is a dataset. Let's take a look at the files here. And we can see there are two JSON files for the eval and the training data. But it's difficult to visualize and to navigate JSON to explore the data. So I will take a look at the lineage. You've already seen the lineage view. You can see our dataset and specifically this version, version 4 of the dataset. It was used in a bunch of training runs. But I want to look at the run that was used to produce this dataset. And my hope is that the person that logged this data can take a look who... Okay, so this was Thomas. Thomas logged this dataset. Let's take a look if he visualized the data as he was logging it. And as expected, you can see a table, a Waits and Bases table with a trade dataset. And we can use this table to inspect this data a little bit and get a feel for what we are actually using. As you can see, there are three columns in this dataset. There is an instruction. There is an input column, which sometimes it's empty, sometimes it contains some text, and there is the output. So if we look at one of these instructions, develop a script that prints out the Fibonacci sequence, there is no input here, and then the output is... let's take a look at this the output is actually the python script that prints out a fibonacci sequence so you can look at this in markdown hopefully it will render better yeah so it seems like i will not analyze this i would expect this to be to be correct let's take a look maybe at one of the examples that contains an input so the instruction here is categorize the following post as either a personal or professional post. And the input is just had the best dinner with my team from work. And the output is it is a professional post as it references a work team. Not sure if I agree with this, but this is the dataset that we'll work with. And it's good to have a good feeling for what this dataset contains. We can take a look at eval and it's structured in a similar way. That is, for example, translate this text into Spanish. The text, the input is, we are excited to work with you on this project. And the output is the Spanish translation of this text. on this project and the output is the spanish translation of this text all right so let's go back to the training script and i want to also give you a hint on how this data is being transformed as an input to the model and this is the the data.py file that is in the mini llm library within the the course repo and you can see that we are transforming the inputs, the rows, depending if there is no input, then we're putting this prefix. Below is an instruction that describes a task, write a response that appropriately completes the request, and we're feeding both the instruction, we're feeding actually the instruction here. If there is an input, then we're formatting this the instruction here. If there is an input then we're formatting this a bit differently, we're also adding this input into the prompt and then in the final step we're combining both the instruction and the output which is the the completion from the LLM and that's something that we will use to train our LLM. This script starts with a config that we can modify as we launch this script. We can adjust the learning rate. We can adjust, we can train this model by freezing some layers in order to fit it on a single GPU. We can potentially use LoRa as an alternative way of fitting this model on a single GPU. We can potentially use LoRa as an alternative way of fitting this model on a single GPU. We can maybe like limit the number of steps if we just want to debug it. We can adjust the number of evaluation samples. This script is using HuggingFaceTrailer. And because it's using HuggingFaceTrailer, like we need to add this line, report to 1db. And this will make sure like we can see how our training is progressing in weights and biases. And yeah, we will not go in detail through the script. If you are interested in learning more about training and fine-tuning LLMs, we have a fantastic course about training and fine-tuning LLMs that you can access under 1DB.courses. And I fully recommend that. So in this case, we'll just assume that, you know, we want to train some models, we will not go in detail through like every element of the script. And just one final thing I want to highlight is we're adding this sampling callback for evaluation, because normally as we train a model, we get access to the loss. You can see the training loss and the evaluation loss, but very often evaluation loss is not sufficient to assess the quality of a language model. And for that reason at a certain point we may want to sample from the model. So we want to generate outputs for a given input and either visualize it or automatically evaluate this output. And this is what we are going to do here to control the quality of our model. So we're adding this sampling callback that at the end of the training, we'll use the model checkpoint that we saved and generate some outputs for our evaluation dataset. Okay, so let's start this training script now. I'm gonna go into my console and do train.py and maybe let's limit this now to, let's say, 20 steps just for debugging purposes and see if the script is running. You can see the script is calling weights and biases, it's starting a run in my review entity. One tip is, and yeah, the training is starting and it will finish quickly because this is just 20 steps, debugging steps. One thing I want to highlight is as you are running the script, please adjust the project name and specifically the weights and biases entity. You will not have access to the review call entity, so you might want to change it either to your personal entity or to the team that you're using weights and biases in. You can adjust also the weights and biases tags, you can change the input model. We are starting here off with 1.1 billion parameter tiny lava, that is our pre-trained backbone. And these are the things that you can adjust as you're playing and trying to get a better and better instruction following model. Okay, so the script finished training, and it's now generating a data set that's generating an evaluation set of data. So let's give it a while to finish, and in the next video, we'll pick it up, and we'll try to see how we might want to evaluate this model.
Finetuning an LLM and saving model
Before we can manage our models, we need to train some! 

In this session we will use the following parts of the course repo: 

Train.py code - follow Darek and train your own model: https://github.com/wandb/edu/blob/main/model-management/train.py
Data.py code: https://github.com/wandb/edu/blob/main/model-management/mini_llm/data.py

Further resources: 
Training and Finetuning LLMs course: https://www.wandb.courses/courses/training-fine-tuning-LLMs