 Hey, Chris, you heard of this Olmo language model, man? I have, Greg. Yes. Is it like really, truly, for real, the real deal, open source, open, open source model that we finally have on the market? It is really and truly the open, open source model that we have have a market. It is really and truly the open open source model that we have on the market. Absolutely, 100%. Yeah, man, I was really impressed when I was looking into this and I saw what AI2 had been up to the past few years opening this space up and it really culminates in OMO. You ready to show everybody what we found and show everybody where the edge of the open source is today? I sure am Greg. All right man we'll see you back in a bit to demo what is possible these days. All right everybody my name is Dr. Greg that was the whiz and we're talking Ulmo today. What you're going to learn is you're going to learn We're talking OMO today. What you're going to learn is you're going to learn a number of different pieces that the Allen Institute for AI has been sort of different models and also the architectures, as well as the different toolkits that surround OMO. It's sort of an entire ecosystem of tools that you should probably be aware of if you're looking to build out on the edge. that we can go through of how to train a real LLM that is performant at the levels we see from meta, from open AI, at least close to those levels. So let's go ahead and get into it today. If you got questions, drop them in the Slido link and we'll go ahead and probably have plenty of time for questions at the end. If you guys want to chat about what's next and what is exactly going on inside the different components we'll go through today a lot of trailheads for the olmo model because there's a lot of pieces to the puzzle if you read the blog post you'll notice that today we're going to talk about olmo but we're going to talk about a number of other things as well we're going to look at the but we're going to talk about a number of other things as well. We're going to look at the data. We're going to look at the model architectures. We're also going to look at the evaluation toolkits and even the way that we know that they were able to actually process the pre-training data, not just what it was, but how they actually processed it we'll go into. Not just what it was, but how they actually processed it we'll go into. And then we're going to show you what you need to do to fine-tune OMO on the data sets that are available to fine-tune it today. All right, so AI2 and OMO up first. We're going to talk DOMA, Paloma, and more. And then we're going to show you what you need to do to fine-tune OMO. Okay, so first off, what's AI2? And how did this thing come about? Well, it's important to know that the Allen Institute for AI is nonprofit research institute circa about 2014 when they were founded. And this was created by Paul Allen, the late co founder of Microsoft. And so this thing has been around for a while, and they've been doing cool stuff for a while. Their mission is to conduct high-impact AI research and engineering in service of the common good. Gotta love that. So they're really, really up to some cool things. And you may have heard of them from different benchmark data sets they created. For instance, one that's really stood the test of time is the ARC data set, also known as the AI2 Reasoning Challenge, the ARC. And this is all about common sense reasoning, quote, common sense reasoning. It's a collection of about 8,000 grade school level, multiple choice science questions. And it is still today, even though it was created now six years ago, it's still part of the Hugging Face Open L and Leaderboard. And if you look at that, it's getting the highest score today was 76.02, whereas the average across all benchmarks was actually higher than that. So this thing is holding up against LLMs very, very well. And, you know, it's really a testament to kind of some of the good work that they've been doing at AI2, which I learned a lot about as I was putting this together for you guys today. lot about as I was putting this together for you guys today. Also worth noting is they did also create the drop data set that was in the Hugging Face Leaderboard but came out of the Hugging Face Leaderboard over the past couple of months. So check out discrete reasoning over paragraphs to learn more about that data set as well. A lot of cool stuff from AI too. Of course, we're here to talk about OMO today. And the big idea with OMO, the open language model, is to really try to inspire all of you and a larger community of folks to really continue the innovation in the LLM space and build on what we already know. Right? So rather than continuing to see more and more proprietary things, we're also continuing to see more and more open initiatives, which is very, very cool. So the Olmo model series gives us access not just to part of stuff, but like really the whole kit and caboodle. We have really every step we need. It's kind of the cookbook to build an LLM and to make sure you do it in a really, really sort of sophisticated way from the outset. So you're minimizing the waste that you might put into compute, that you might put into using the wrong data, that you might put into choosing the wrong architecture, that you might put into aiming at the wrong benchmark evaluations. So really, really the suite of tools that they've provided for everybody out there that is interested in developing their own LLMs, which many domains and companies will be, is really kind of monumental. So this is from the ULMA paper, and we'll kind of do a few paper studies today. They're talking about recent open LLMs or LMs, language models. And so, of course, everybody knows Mixtral, the mixture of experts model that came out recently. And so, of course, everybody knows Mixtrel, the mixture of experts model that came out recently. They provide, of course, all their model weights, and they gave us a little report early on in 2024, but they didn't tell us a whole lot more than that. LAMA and LAMA2, especially the paper, went into a ton of details on how they were sort of doing the adaptation and the instruction tuning in the RLHF. That was sort of an amazing insight into how you should think about tuning for general alignment. And then the Mosaic pre-trained transformer, they provided things like the data set distribution, but not the data itself. Falcon, if you remember Falcon coming out, they gave some of the pre-training data. And then the most open models like Pythia from Eleuther AI and Bloom, that was a sort of collaboration, quote, bootstrapped by Hugging Face and others. They released lots and lots of stuff and that kind of set the tone, I think, for where we are today. So where does Olmo fit in? we are today. So where does OMO fit in? Well, OMO is kind of trying to say, okay, well, there are other folks that have been working on this sort of truly open space. LLM 360 is one example. And they really wanted this whole framework from, you know, pre-training data to actual training to what the evaluation tool sets are. They wanted to sort of provide this whole thing. And OMO is kind of the whole thing now, which is very cool. So it's kind of trying to narrow that gap between the open source, truly open source that's happening now. And then things like Lama2 2 where it's a lot of it is open source and it's very very helpful very useful to many of us but still we're left wondering about many of the details so this idea of sort of truly open is important here what does this mean well it means you know we've got all the model weights, we've got the training code, we've got the training logs, ablations, metrics, and we even have the inference code. So they went and they built four variants at the seven billion scale. And they use different architectures, optimizers, training hardware, and they also used one, built one at the 1 billion parameter scale. That's the one we'll pick up off the shelf today. And they're releasing hundreds of checkpoints, the full training data, and then to sort of get a feel of this ecosystem. And we want to go through these tools today. They're like, well, we also have code that produces the training data, which is dope. And that's the Dolma code slash paper slash project. There's also a few other projects that are very, very useful and worth knowing about. The WIMD helps us analyze pre-training data the catwalk helps us do sort of large scale evaluation and paloma allows us to do perplexity based evaluation finally the open instruct data set is going to help more and more people do instruction tuning and do it in a way that's not sort of as random as it feels today. Which data set should I pick up? How am I actually going to do better generally across tasks with any given data set versus another? These things aren't really clear. So AI2 is really tapping into a lot of things as they try to address the entire suite of tools here. And really cool to note that they're currently using this OpenInstruct dataset to release a big OMO. Not yet out, but, you know, stay tuned. Okay, so we're going to go through each of those tools, but the thing to keep in mind, just like all of these companies and initiatives, they're all like, well, you know, I mean, this is just day one. You know, we talked yesterday, Langchain V0.1. This is the first step on the journey of a thousand miles. It's very much the same for everybody in the space. And there's going to be bigger models, more data, better curated, higher quality, you know, more efficient instruct tuning, better alignment. And, you know, of course, as we get into, you know, the future in the 21st century we're going to start seeing the images come in we're going to start seeing the multimodal stuff come in there's going to be lots of stuff that we can expect from AI2 moving forward and that's super cool I want to just take a moment to talk a little bit about the architecture it's not groundbreaking so I just want to kind of put it in context for you guys. And then we'll dig into the data and then the additional pieces of the evaluation suite before we check out how we can do some fine tuning today. So just harken back, if you will, for a moment to a couple of the models that OpenAI put out. GPT-1, 2, 3. These were generative pre-trained transformers, decoder-only style, et cetera, et cetera. We have some data on what they were doing with GPT-1, 2, and 3, and what the architecture looked like. Of course, this is just a simple transformer with a tension and a feed forward within each block. And what we're kind of talking about when we're talking about architectures, we're talking about how many layers, this would mean sort of attention layers. Technically, each of these are layers, but let's sort of liken the idea of layer to the idea of decoder block so you know 12 layer and then within each attention head attention layer we also have within within each attention layer we have 12 heads this is a multi-headed attention so we're sort of 12 heads in each layer and therefore in each block. So 12 by 12, this was GPT-1. They used 117 million parameters and about four and a half gigabytes of text. GPT-2, the big GPT-2, they said 48 layers and they weren't super clear about the number of attention heads, but I think we can guess at this point in 2023 as we'll see uh 1.5 billion parameters and 40 gigabytes of text sort of the same general architecture and then gpt3 they used 96 layers 96 heads and 175 billion parameters 10xing again and again And 175 billion parameters, 10x-ing again and again, 570 gigabytes of text. Okay, so you're seeing the sort of interesting experiments they were doing at the time, you know, sort of the square 12 by 12, but then they also were looking at some other configurations as well. Let's take a look at the Olmo numbers. Well, Olmo in similarity and also in contrast here, we're seeing that the 1 billion has a 16 by 16. We're seeing that the 7 billion is 32 by 32 and the 65 billion that's currently in training. Interestingly, look at that. It's 80 by 64. It's not 80 by 80 or 64 by 64. So I wonder why they chose that. And maybe we'll find out when they finish training. And the data set that they trained on, we see 2 trillion, 2.46 trillion. And I wonder what's going to show up here. Well, let's find out, or at least try to find out by looking at the additional pieces of the suite of evaluation tools, Dolma, Paloma, and even more. So the puzzle pieces we want to put together here are those associated with data, Dolma, and WMD, evaluation, that's Catwalk, and Paloma, and Paloma, and Instruction Tuning, Open Instruct. So the DOMA data set, what's that? Well, it's actually 3 trillion tokens. So I imagine we're going to see a 3 trillion number on that 65 billion. billion. And it's a combination of web stuff, science papers, actual code, books, social, and then sort of encyclopedic materials, this sort of, you know, code for Wikipedia, right. And all in English, so that we kind of get this diverse mixture. but it is still only in English. And if we kind of zoom in on this data set, we can see specifically that we have common crawl. What a classic. This is like petabytes of data, 12 years of web crawling, so much stuff, right? It's like the meta. And then we have the stack. This is three terabytes of code files covering about 30 programming languages, crawled once again, similar to Common Crawl from GitHub. The C4 dataset is the colossal cleaned Common Crawl, C4. And then we can keep going here. The Reddit dataset, this is actually the push shift Reddit dataset. So this was sort of like all the Reddit submissions and comments between June 2005 and April 2019. So like 650 million submissions, 5 billion comments, you know, like almost 3 million subreddits, a lot of Reddit stuff. We have this PES2O. This is where we get our academic literature. So about 40 million open access papers. They've been cleaned, filtered, formatted. It's derived from the Semantic Scholar Open Research Corpus, the S2ORC. Right? So there we get S2O. There it is. And Project Gutenberg is simply publicly available books, books in the public domain. So they grabbed it from April 2023. And then the Wikipedia and the Wiki books. This is our encyclopedic material from the March 2023. Wiki media dumps. OK, now you might say to yourself, well, man, that's a lot. Did they use all of that? And this is where it gets really interesting because of course the answer is no. What they did is they actually didn't just generate the data set, but they gave us all a toolkit to actually process our own data sets in the way that they did. And this is just one example of what they did. This is what they did with their web data. So Common Crawl, C4, this kind of thing, where first they go through a language filtering step, and then they do their first dedupe step by getting rid of the same URLs. Then they go to sort of this quality filter, you know, oh, no, that's low quality, get it out of here. That's low quality, get it out of here. This is high quality, let it stay. And then we have sort of this toxicity looking for things that are gonna, we probably don't want to be training and going through the core, unsupervised pre-training step for a model with this sort of content filter. Finally, a final dedupe step. And this sort of, although we had two dedupe steps for web pipelines, their other pipelines were similar, and they combined one of four language, quality, content, and dedupe filtering steps. quality content and dedupe filtering steps. This is something that is now available to all of us to go ahead and leverage their toolkit to build our own data sets. Super cool that they're putting this out there. The first thing I've seen like it and definitely worth knowing about if you're looking to build your own large language model in a particular domain, similar to say what Bloomberg GP gpt had tried to do and what many others are thinking about these days so that's dolma it's the data set but it's also the toolkit now what about this whim d thing well of course it's a funny little what's in my big data that's what what WMD stands for. And, you know, the idea is very simple here. It's like, man, we have all these super giant corpora, these many corpuses of data that we're pulling from and that we're using for unsupervised pre-training. And we don't really know exactly what the quality of that data is. So this is, again, not just a study, but they actually went to create a platform in a set of 16 analyses to allow for comparison, to allow for assessment of the quality. Now they found some interesting things. For instance, they went ahead and applied this to, you know, C4, the pile, red pajama, you know, and they found very, very high duplication, sort of synthetic data, very low quality content, personally identifiable information, toxic language, and contamination of benchmarks, meaning it was sort of like, you know, trained on data that was similar to the benchmark you might want to test it on. That's not good, right? Because then you're sort of testing on the training set right and really interestingly 50 of the docs in red pajama are duplicates that's wild so you can kind of see right duplicate d-dupe right low quality um toxic sort of this is the filters came from with dolma and You know this this contamination of important benchmarks this is a problem too right we don't want to Let our language models cheat against against benchmarks, so that's that's no good So you know if you're again using the dolma toolkit, it's a way to sort of analyze your usage of actually processing the data before you go out and you train your LLM. So that's pretty cool. Now, what about downstream? What about after you train the model? in the project they did called Catwalk, again, from AI2, is they were like, man, it's hard to compare across LLMs and benchmarks at scale, right? Because like, are you really always taking the same data to test the benchmark with against the different model? Like, it's really kind of hard to make sure that you're doing this right all the time. And of course, there are benchmark evaluation tools like from Luther that runs the back end of the OpenLM leaderboard at Hugging Face. But, you know, they said, well, Catwalk is our sort of solution to this problem of when you want to look at many different models and you want to look at many different tasks at once and so you know here's sort of the visual example of if you wanted to look at gpt2 large and t5 base on a couple different different benchmarks, you could do that pretty easily with this kind of framework that they've put together. And it's sort of designed for this n times n implementations, right, simultaneously. So it's kind of built for researchers and for people trying to do research into different sort of perturbations, different combinations of model and benchmark data sets. So they said, you know, we fine-tuned and evaluated over 64 models on 86 data sets with a single command, zero code. So, I mean, that's a pretty big flex. Pretty cool. Finally, we're going to go talk about one of the other kind of pieces that kind of complete the puzzle here. And we're going to talk about Paloma. But first, in order to talk about Paloma, we need to talk about the idea of perplexity. If you remember like, you know, the try to find cheaters, GPT-0 sort of assessment things, they were using perplexity, but you know, this is sort of a bigger picture view of perplexity. And the biggest picture, if we can sort of summarize this very, very clearly, you know, perplexity is simply measuring the probability of any given statistical models prediction. So like it's a very general thing, but a GPT style LLM, it's going to predict the probability of the next token. Right. It's an autoregressive next word predictor. And so what we're doing is we're sort of using perplexity in general to evaluate the kind of uncertainty that an LLM has in predicting the next token. Of course, we can like look at text that's been written already and we can say, is the model perplexed by this? But this is not what we're really focused on in using Paloma to look at large data sets and also fit how they've been fit to the model and sort of looking across large swaths of data to kind of give us an idea of if we train on a particular data set, let's say Common Crawl, are we actually developing the best possible next word predictor for our domain or for our task or for both. So this is sort of a really interesting evaluation toolkit to be aware of, again, completely open source. And it's based on this idea that perplexity is often reported, but kind of like it's not really clear that perplexity from one distribution of data is going to extrapolate to other distributions. And so what they're kind of seeing is when they do this analysis of perplexity, they're like, well, we should train on stuff beyond common crawl, right? Common crawl sort of isn't enough to just generally extrapolate. Maybe we'll get some fundamental set of things that are enough to generally extrapolate, but if we just use common crawl, it's not enough. And so this is sort of, again, a nice measuring stick for us to be able to sort of say, are we able to build pre-training data sets that are gonna allow us to really create this sort of general approach or that we're focused on a specific domain? Can we sort of, can we make sure that our next word prediction is very, very aligned with that domain? So very, very, very aligned with that domain? So very, very cool. Maybe something that's worth a deep dive at some point. But the very last piece of our puzzle here is the instruction tuning piece. And instruction tuning isn't very complex. It's simply fine tuning an LLM on tasks that we describe with instructions. And this is kind of where we get into the Open Instruct benchmark and this sort of collection of different data sets that AI2 has once again put together. The first paper that came out with regard to Open Instruct was called How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources. And so, you know, they were sort of saying again here, you know, hey, everybody's claiming in their papers that, you know, you can do really, really good with these open data sets, almost as good as proprietary models, but it's not clear how exactly that is getting evaluated. And so they wanted to sort of, again, bring this into their ecosystem of tool sets. And they were like, all right, we provide a large set of instruction-tuned models from seven to 70 nearly billion in size, trained on 12 different instruction tuned data sets. This is before Ulmo. These are actually known as the Tulu models. And the Tulu model, as we can sort of read, is a model that they essentially took this open instruct data set they used a lot of their research they were doing prior to this olmo and they were like you know basically what we learned is that no single instruction tuning dataset is gonna be the best across all things, which I mean, isn't really surprising, but it's very cool to see that properly evaluated, similar to how we need these evaluation frameworks when we're building RAG systems, for instance, like we talked about recently. And so we've got this kind of ability to say now that actually this 87% of chat GPT performance, 73% of GPT performance, these are numbers that we can sort of use OpenInstruct to very clearly and cleanly draw lines on. And then they actually did some more recent work and they released Tulu2, right? So if you haven't heard of any of this AI2 stuff, there's a lot to dig into and check this out. I mean, oh my God, I can't believe it, right? They released not just Tulu2, they released Tulu2 v2 mix, right? Instruction tuning mix, Tulu2, Tulu2 plus DPO, classic, direct preference optimization, basically the latest and greatest alignment technique, and then a code Tulu. So a lot of this stuff has been happening all along, and really it comes together to sort of show us that AI2 has been doing some really, really great work. And Olmo is only sort of the culmination of all of this work that they've been doing. So pretty awesome to kind of dig in and learn about this. And, you know, what we're going to show you now is we're going to show you what's possible today. We're going to take the OMO 1 billion base LLM, and we're going to show you what we can tune it on. And we're going to actually end up using this tool V2SFT mixture rather than an open instruct data set. And Chris is going to tell you exactly why as he digs into the code for instruct tuning. Wiz, over to you, man. Thanks, Greg. Yes. Okay. So basically the idea here is going to be pretty straightforward. There is a hiccup that we're going to run into that I will keep my eye on to make sure that as soon as it's resolved or I have a solution, I'll put it forward. But you know, for now, basically, how are we going to fine tune OMO? There's a couple of things we need to keep in mind. So OMO is released on GitHub in its kind of own format and format and converted to Hugging Face. It does work very well with inference. It's very good at generating responses. But we're still, you know, there's a little bit of friction when it comes to training. So, we're going to go through how we will train it once I have a solution for the specific problem. And then we're also going to show how we can fine tune it using Olmo's or using the actual Allen AI tools. So first things first, you know, we're going to be using QLoraLora. We're going to be using the Olmo1b. We're going to pass in our Huggy Face token. We do have to get access to some of these data sets. So while we won't be using the entire data set for this training, we're going to use a subset you don't need a license for. If you wanted to use the full Tulu instruction data set, you will need to have your Huggy Face token. Other than that, we're just going to load the model up. We just load it from pre-trained. We set our tokenizer. We fix our tokenizer settings so that we are looking at the correct padding. We want to pad to the right. And then we want to be able to jam a lot of information into our prompts. So we're going to have that pad token be our end of sequence token. You'll notice that Ulmo only has an end of text token. So we would also potentially want to add a beginning of text token. But for this use case, we're just going to go ahead and keep it as is. We look at the model. Now, this is a refreshing view. If you're used to looking at these outputs, it's slightly different looking than usual, right? It's not just Lama again. You'll see, though, it has everything that we're used to. We have our activation. We have some dropout. We have our attention out layer. We have our feed forward out layer. We have some rotary embeddings. That's great. We have our attention and feed forward normalization layers. And then we have our attention projection. This is like equivalent to Qproj, that's great. We have our attention and feed forward normalization layers. And then we have our attention projection. This is like equivalent to Qproj, Bproj, you know, if you're used to those specific layers. And then, of course, our feed forward proj as well. Then our out. So this is, I mean, it's at the end of the day, it's still a bunch of decoders in a row. But it is, you know is slightly different naming conventions. That's awesome. When it comes to the data and data set preparation, we're going to be using a subset of the Allen AI Tulu V2. This is, again, it's an unlicensed part of it. Also, it's called Wizard LM, so that's dope. What we're going to do, so the full Tulu mixture is a bunch of different tasks. It's a very comprehensive data set. You know, we have everything from Flan, Open Assistant, Shared GPT. It's got it all, right? But we're just going to use this wizard LM because it doesn't require that license. So if in case you haven't got the license for data sets, you shouldn't run into an error here. We're just going to load it up. We don't actually want all of it. So we're just going to get a subset of our dataset to use since we're going to be training for a few, you know, very few iterations. We're not going to be fully fine tuning this, but we will look at code that will help us to do that in a moment. We're going to look at our dataset. We're happy. We have our instructions and outputs for 500 rows. And then we have a test which is 100 rows. So, feels good. We're going to put our data into a specific format. This is because we want to instruction tune it on this specific kind of task. So, we're going to create a prompt template that we can use and reuse, basically for every instruction set when we're doing our SFT training at the end of the notebook. It's simple enough, right? We got our instruction, we got our response. There you go. When it comes to the, you know, actual dataset creation, again, we don't have that beginning of sequence or beginning of text, start of text token. So it might be useful to add that if you were going to do it full fine tune. Just do the due diligence to test your model before and after and see which performs better. We're just going to stick with exactly what they have. And then we're going to also look at generating a response. So as you can see, you know, this is all just straight hugging face code, and it generates like a dream, right? So we have instruction below is instruction that describes a task, write a response that appropriately completes the request. What are the different types of camelids? There are two types of camelids, a laminate and alpaca, input and output, the input, the output, you know, so it goes on and on. Again, this is not like a, the base model is not instruction tuned. So it makes sense. I'm just going to kind of keep generating forever and ever, but we don't have to worry about that too much. That's what we're going to try to fix. You'll notice the only thing that's a little bit different is that we have this kind of casting, right? So we have to cast our values to CUDA in order to ensure that we're on the GPU. This is not necessarily a step that you'd see in other models, but it's important to include it for Ulmo. Now we're digging. Basically, this is all the same as we're used to, right? So we're used to. We're going to go ahead and prepare a model for K-bit training. The Hugging Face implementation does not use gradient checkpointing. It is the case that the OMO version does. If you're fine tuning using some of the scripts we're going to look at later on, you're going to be able to take advantage of gradient checkpointing. If you're going to try to do this in a notebook, you will not be able to right now using Hugging Face Transformers library. So we're just going to set this flag to false so that we don't run into any errors. Next up is LoRa. The only real difference between this and regular LoRa is that we have to specify our target module. It's not included as a default. It's not within the Pest library yet. So we just have to manually point at the module that we want. Again, we can point at all of our projections as for the QLoRa. But for this one, we're just going to keep it simple, keep it pointing to attention projection. This is the same as targeting QK or QV proj in say like a Lama style model. But you will have to fill this out. There is no default. So you will have to target specific modules. With our rank and our alpha, we're just kind of choosing arbitrary values here. We're not doing too much thought into it. There is some wisdom available in the fine-tuning repository, which is the open instruct, which we can look at in a second. But for the most part, we're just going to choose values that make sense. And then we're going to get PEF model. Again, this is all the same that you might be used to if you're used to using the Huggy Face Transformers library. Next is, yes, it's the same as always. Nothing has changed. You know, we're using the same trainer that we would normally. And we're using the same SFT trainer that we would from TRL. and we're using the same SFT trainer that we would from TRL. We're able to do this exactly as we're used to, right? We have our create prompt, which we built up above that formats our rows into the correct format for when we tokenize them. And then we are going to pass in our train and test set so that we can eval during our training. We have a lot of data left over that we can save for tests since we're only using 600 of the 70,000 examples. But, you know, in this case, we're just going to use that small eval data set to eval during training. And that's where we have to stop for now. Right now, the forward method isn't compatible with the default input or the default that's provided from the tokenizer. So you will run into an error during training. I will update this as soon as there's a concrete solution. But for right now, this is where we kind of would end. You can just go on and do train from here. So let's look at how we would want to actually fine tune this if we had access to a little bit more compute and say we were not in the Colab environment. So in this case, it's pretty straightforward, right? We have the same kind of thing that we did with our data that's being tucked away into the script for Tulu. It's doing exactly what we saw before, right? It's just converting it into a specific format and then preprocessing the data a little bit, making sure that it's all in an expected format, right? So that's all this script is doing. After that, basically, we're just going to pass in some configs. The configs are just going to show kind of what we want to be true about the model. So if we look at our example of a config and say, let's, we go to official, we go to the, sorry, we go to the 1 billion parameter model. You can see here that we have like, you know, the dimension of the model, the number of heads, number of layers, the MLP ratio, all of these things, right? These are different knobs we can tune. You don't have to. If you're using the kind of main checkpoint or the final training checkpoint as a base model, you can leave these things basically all exactly as you see them. You only have to change a few different things. The idea, though, is that we are all we're doing is we're, you know, giving the trainer some information that it needs in order to train our model stably and on the data we wish. You'll notice that we have to change the load path. We have to set the reset trainer state to true. And then we have to point our data paths to that preprocessed data that we created using that script. After that, we can do whatever we like. We can change the mass paths, you know, if we want to. And we can use the, we can change, update, and modify our evaluators. Very kind of straightforward, you know, process. And then we can just run this big script. This is going to run this script via Torch Run. And it's just going to train the model, right? We're just going to do some fine tuning with that data we generated. Now, what you can look forward to in the future, though it's not currently compatible by default, is the Open Instruct, which is basically a training library that we could use for those models. It isn't currently, there's a few things that still need to be sorted out when it comes to the PEF QLORA implementation. But this is the same thing as before. Basically, we have a proprietary training data script, and then we can fine tune with the script using accelerate. Again, this is not something that's currently working for OMO 7B, for QLORA, but you can expect it to be finished soon. The model is very new. And for now, if you're looking to do some serious fine-tuning, then I would point you at the OMO fine-tuning section here to use this Torch Run script, which will help you to do that. So that is how we would fine-tune OMO on any variety of tasks and data. And with that, we'll push it back to Greg, who will take us to Q&A. Yeah, thanks, Chris. I mean, hey, sometimes when you're out here on the open source edge, things don't quite always work exactly the way you were hoping them to. And just to show you just how far out on the edge we are, you know, the sequence of work that AI2 has been putting in, this sort of initial paper on OpenInstruct and on Tulu, that was June 23. The sort of whim D, what's in my big data, that was October. Tulu 2 is November. Catwalk and Paloma both came out in December, these papers. Dolma came out in January and here we are in February with Olmo. I would venture to guess that AI2 is going to keep crushing it from here. So watch out for these guys for sure. And they bring together a ton of stuff. So trying to understand this suite, it's not something that's been around a while. If you understand it, if you can leverage it, this is going to be a competitive advantage for you out there in the marketplace, whether's olmo dolma wimd catwalk paloma tulu i mean all of these things are so new very few people know about them so take some time to learn about them and check out the papers in the youtube live chat we saw that the olmo models currently in 1 billion 7 billion we're expecting the 65 billion soon that'll probably be on 3 trillion tokens. That's the Dolma data set. And then the instruct tuning with OpenInstruct is not quite implemented all the way through Hugging Face so that we can leverage QLORA and some of the things we're used to. But I imagine it's going to be figured out real soon. So with that, we'd love to take any questions you guys have today. And we're happy to chat about OMO and some of the work that AI2 is doing more generally. Go ahead and let us know what you want to chat about, and we'll do our best to answer any and all questions that come through today. All right, Wiz. It looks like it didn't quite work the way we were hoping that it might today. I would expect they're going to get it working pretty soon, huh? What do you think? Yeah, I think so. I mean, you can get it to work with some kind of laborious code modifications that don't translate very well to the Colab environment. But other than that, I think they'll be done pretty quick. I know that they're working on the Hugging Face integration. You can check the model community pages for other people kind of highlighting these problems. But for right now, I think we can expect that just using their script is going to be good. The script doesn't super, you know, it's not going to play super well with the Colab environment. It is kind of something that you want dedicated resources for so you can do those long trainings. But it will, it would technically work. It might crash and it's going to consume a lot of compute credits, but that's all right. Yeah. Yeah. All right. Yeah. Yeah. All right. Well, we've got some questions from folks that are kind of interested in different aspects of how they train this thing. The first question is about the web crawl. I assume this has something to do with common crawl. How does that actually work? Whether we're crawling the entire web or we're crawling GitHub or crawling Reddit, like, how do you think about that? Yeah, when I think about, like, basically this is a scraping, right? So we're scraping the just text off the internet, right? And we do this with a crawler that just follows links, gets text, goes to the next one, gets text, goes to the next one, gets text. We just sometimes refer to it as like spiders. The idea is that, you know, it is just, you know, as much as possible indiscriminately obtaining text from the internet. Obviously, there are subsets or websites that specifically we want to target, but or websites that we definitely don't want to target, but for the most part, it's just let it loose and consume. That's a much harder practice to do now than it was when these data sets were kind of originally collected because of the kind of changes in netiquette as it was, or as it were. But yeah, the idea is it's just a script that follows links and gets text. that follows links and gets text. Simple enough, yeah. But it just does it for like all of the text it can find. It sort of spider webs its way out to, what was it? Yeah, petabytes of text, right? Like wild. I would not be allowed today. I would encourage everybody to check out the Common Craw website too. It's, uh, it's really cool that they exist and it's sort of interesting that they formed the app, uh, you know, kind of base of all LLMs today. Okay. Ali asks, I saw something related to, they provided a hundred checkpoints. Uh, what are the benefits, uh, to giving such a large number of checkpoints? Why would we care about this? So there are two ways that you can think about this. So I'm going to respond in the way that I think about it. And then I'm going to highlight that there is another way to think about it. For me, it's mostly an indication of openness. The idea is that they trained this model for a long time. They did use checkpointing. And so they're releasing those checkpoints so that we can see how the model is trained, how it progresses through the checkpoints, what potentially they did at each checkpoint. It's very much part of that open idea, right? So I don't think that there's a tremendous amount of utility past that. However, having the earlier checkpoints might give you some kind of launchpad, right, that you could use to then pivot to continue pre-trained on another data set or another corpus. So it's not like there's no utility at all. Probably not for most users, but definitely for some people, it's going to be awesome to have those checkpoints and to kind of, you know, start with a more established base that's not yet locked all the way in, right? Like if we want to say, create like a law domain model, we might use one of the earlier checkpoints as a starting point and then, you know train it on our on our map of uh domain specific text so you can sort of download the parameters in the checkpoint and start from wherever you'd like yeah yeah okay that's that's super cool it's like it's like starting with a you know a slate that's been wiped but not yet cleaned yeah yeah totally clean slate or a totally locked in slate so yeah yeah and you can kind of look at the training loss curves and you can say uh yeah yeah okay that's super they actually released the entire training blocks so we can see like their weights and biases runs that that actually show exactly what the model's loss was at, you know, X iteration. And we can see, we can see, we, we have tremendous amount of insight into, I mean, so to me, this paper, all the things they've released really, I, the way I, I think about it is it's a, it's like a cookbook or an instruction manual on how to pre-train a model. And it's the most comprehensive one that we've seen yet. It truly is incredible. Yeah, yeah, yeah, yeah. And I remember when we were looking, when we were watching the Bloomberg GPT guy present at ML Ops World last year, he was sort of talking about the training loss is going down, going down, and then it spikes up. And then you're like, ah. training loss is going down, going down, and then it spikes up and then you're like, ah, and you know, and I think like, you know, even being able to look and see where those maybe those spikes or those, you know, going back up things occurred is very, very interesting. So follow on question here, did they also release metrics related to each of the checkpoints? So we can pick it like we already talked about the loss. But what other what other sort of numbers are you thinking about when you're doing unsupervised pre-training besides the loss curve? They released everything. We have it all. We have the evals at every checkpoint. We have, I mean, it is, we have everything they had, which is insane, right? Like you're not supposed to do that, but they did it. So that's very nice of them. Yeah, yeah, yeah. It's very much what you just said, right? Yeah. Bloomberg GPT released a very good paper. It's still a good paper. Less transparent, obviously, because, of course, it's using sensitive financial information, blah, blah, blah. But it is interesting to see, like, okay, here's a mistake or here's here's like a stumbling block that they ran into and here are the exact conditions that led to it and we have so much you know super focused granular feedback from from the uh the Ulmo crew that lets us see exactly where those things happen like Greg said where these spikes are occurring you know did we do a a shuffle or some kind of you know change that caused our our metrics to actually decrease do we need to reset like there's all kinds of different information that we can we can get because we have everything they had to train the model and i think it's also a good indicator of what you should have to train a model right like these are the things you should be paying attention to. And so if someone were to ask me, what should I pay attention to during free training? I would say almost everything they looked at. All the stuff. Right. Yeah. Yeah. Well, you know, we keep saying they, and I just want to reiterate AI by AI asks, who's the Allen and Allen Institute? Who's the benefactor? And it is Paul Allen, the co-founder of Microsoft, the late Paul Allen. He did sort of put the seed money down for this. And he also, I believe, has a building at the University of Washington named after him, and they sort of collaborate. It's in Seattle. So they sort of collabo pretty hard on making sure that people are pushing this forward. And they seem to be attracting some talent too, by the way. If anybody's looking for like a pretty sweet gig, that might be one. I know that I was on the website earlier. They are hiring. So super cool stuff there. What's the difference between the notebook you demoed and the Torch run script, Chris? When it comes down to it, it's just using the Huggy Face Transformers ecosystem. It's not just directly training with Torch. So PyTorch will just train it using pie torch using their script um it is different in the sense that uh if you so if you look into the script itself uh which is in the repo that uh that we we saw that's the omo repo um it is not using the same methods that we would to train using uh just the Transformers library as what it comes down to okay cool yeah uh Jason just check out the Olmo paper the Olmo paper I think is one of the first links we dropped then I want to go to Luna's question here Luna's talking about a smart rag model how to build a smart rag model for unstructured data structuring. Is that related to OMO in any way? How should we be thinking about that? Not related to OMO, but still cool. You know, if you're looking to do this, basically, there's a number of steps of smart rag. Hopefully, if you haven't, you should check out our video that we did with RCAI talking about DOM and smart RAG. But when it comes to unstructured data structuring, that is a pretty complex task. You'll see that, you know, it's kind of the big question right now, right? This is relating to like, how do we interact with complex PDFs? Or how do we take like, you know, web page and reconstruct that into some kind of meaningful structured data? Obviously, with HTML, it's a little bit easier than without. But yeah, yeah, that's yeah, yeah. Yeah, definitely. yeah yeah that's yeah yeah yeah yeah definitely um okay so uh juan the juan asks any new slash specific nuggets that you're getting from their repo from their paper about the pre-training process anything that uh that we sort of are drawing from that that's maybe unique i i don't know about like uh specifically unique you know but i did like that they had uh non-parametric layer norm because it is it is like a layer norm is a problem right like hopefully we can all agree especially when it comes to like, you know, quantization and representing very small values, layer norm is kind of a pain. So I need to look a lot deeper into specific reasoning into why they chose non-parametric in order to really understand it better. But I did think that was cool. Otherwise, it's, I mean, listen, it's Lama 2 with a fresh coat of open source paint. You know, it's still awesome. It's still dope. They still did everything that probably the meta team did for Lama 2 and the Mistral team did for Mistral 7B and everything. But at the end of the day, it's 32 by 32. Transformer architecture, Swiglu activation. I mean, you know the the novelty is okay one thing actually they released all their data cleaning scripts right so that is something that now you can just have and use uh data data cleaning ddupe and everything that comes with data cleaning is huge and it's a lot of effort and now it's slightly less effort and so um if you if you're like looking from a piece of code to take from the paper and you're interested in pre-training models uh which you if you're interested in it you'd be doing it it's very expensive and time consuming uh their their data cleaning script i think is the most valuable lines of code that come from this paper. Yeah. Yeah, yeah, yeah. Yeah, it's so cool. They've got the data. They've got the data cleaning script. They've got all the training loss curves. They've got all the checkpoints. Yeah, I mean, it's super cool. It's sort of a new year and a new day for AI engineering and for building LLM. So, Wiz, that's it for today on the Q&A. Thank you so much, man. And we'll go ahead and wrap up, everybody. Thanks for joining us. If you haven't liked and subscribed yet, please do. And join us next week to talk alignment with RLAIF. That's reinforcement learning with AI feedback. That's going to be the second part of our alignment series. We will be covering direct preference optimization in a few weeks out from that as well. And if you haven't joined our Discord community, we'd love to see you in there soon. And if you're interested in going deeper on any of the steps that it takes to build actual production LLM applications. Maybe you should check out our AI engineering boot camp. We've got a cohort starting next week and we'll be running cohorts throughout the year and updating with the latest and greatest content at the open source edge. Finally, please give us feedback on today. This was a special event that we saw the model come out. We put it together. We want to continue to do more special events. If you have great ideas or feedback on how we could improve, let us know. And as always, keep building, shipping, and sharing. And we'll see you back next week while we'll be doing the same in the meantime. Thanks everybody. See you soon.
AI2's OLMo (Open Language Model): Overview and Fine-Tuning
GPT-4 Summary: Unlock the secrets of OLMo, the first "truly open" Large Language Model (LLM) launched by The Allen Institute for AI (AI2) on February 1, 2024! Join us in an exclusive YouTube event where we explore the groundbreaking OLMo series, including its unique Dolma pretraining dataset and the sophisticated architectures of OLMo-1B, OLMo-7B, and OLMo-7B-Twin-2T models. Dive into a live demonstration of OLMo-7B's out-of-the-box performance using the Paloma benchmark, and witness a hands-on fine-tuning session utilizing the Quantized Low-Rank Adaptation (QLoRA) method with data from AI2’s open-instruct GitHub repository. This event is a must-watch for learners eager to grasp the essence of true openness in AI, AI engineers looking to leverage the latest in LLM technology, and practitioners keen on the nitty-gritty of the AI2 OLMo series. Don't miss this chance to be at the forefront of open-source AI innovation!

Event page: https://lu.ma/olmoLM

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/nnX68PpbCpNogbB8DWJPuH

Speakers: 
Dr. Greg, Co-Founder & CEO
https://www.linkedin.com/in/gregloughane

The Wiz, Co-Founder & CTO
https://www.linkedin.com/in/csalexiuk/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

Apply for our new AI Engineering Bootcamp on Maven today! 
https://bit.ly/aie1

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/urhWQxtSmzmLsgWaA