 Hey Wiz! Hey, Wiz. So basically all of LLM fine tuning to date has been with Laura, low rank adaptation. Isn't that right? I mean, that seems pretty true. It feels pretty true, at least. Feels pretty true. We've got a sick meme on this that we've been putting out there quite a bit to the world. Today, although things might be starting to change, we're going to meet the challenger. I heard that it combines Laura with lasers. Is that right? I mean, it sounds pretty cool if it is. Yes. Well, we're going to see, along with the team from RRC today, if we can find some signal through the noise of fine-tuning. Sound like a plan, my man? You know what? That sounds like a really fun plan, Greg. Let's go ahead and kick this thing off, then. We'll have you back in a little bit as we kick off our first discussion with the team. Welcome, everybody. Today, we've got a special Thursday event where we're talking about Spectrum, a brand new fine tuning method that was put out by the folks over at RCAI doing absolutely massive things in the space at the LLM Edge. So what I'd like to do is I'd like to, first of all, introduce everybody to the Spectrum tool today. And we're also going to meet some of the creators and maintainers of the tool that work specifically with the RCAI team and that are with us today. So today we talk spectrum. It's all about training, fine tuning, pre-training, post-training. It's all about doing training on large language models, on small language models, and everything in between. We're going to cover the full spectrum today. Let's get into it. What we want to walk away with at the end of our session, as we align our aim to the day, is we want to walk away with, okay, what is spectrum and how is it working? How exactly is it working in a way that's different from other leading techniques that might already be in our tool belt? Specifically, let's look at LoRa or Quantize LoRa, QLoRa. This is something we've looked at in depth and also something that we want to make sure you guys understand when you should pick up one tool versus another. We'll also fine tune using Spectrum's signal to noise ratio approach, which is pretty cool. And we think that there are some real benefits that are worth paying attention to. So we'll talk Spectrum, we'll talk Spectrum versus Laura and the competition out there, and then we'll do some fine tuning. We'll try to benchmark this against a simple fine tuning task. So you guys can really get a feel for the results that we are able to get. You can, of course, read the paper and check out those results as well. We're going to go deep into the code today, as we always do. into the code today as we always do. Introducing Spectrum is not complete without first introducing RCAI. We did our first event with them talking about domain adapted RAG quite some time ago, and they've been doing really big things since then. RC is all about merging and training custom small language models, and they've been making some moves to build out quite a robust ecosystem. We've covered Mergekit in a previous event, which we thought was a great pickup from them. This spectrum powered training that's part of their larger ecosystem is what we cover today. We also think it's a great pickup. So what I'd like to do is I'd like to invite, first off, Lucas Atkins up to the stage with me. He's a research engineer at RCAI specializing in alignment. He is the primary implementer of Spectrum. So he knows this stuff. Let's hit him with the tough questions today and oversees the technology specifically related to SFT, supervised fine tuning and DPO, direct preference optimization. And along with Lucas today, we've got Fernando Fernandez, an AI research scientist at RC who blends deep technical expertise with business acumen. My man, Fernando, that's what I'm talking about. And he's all about transforming complex data into actionable insights and cutting edge solutions for the end user. They're both highly trained, highly educated guys. Welcome, Lucas and Fernando to the event. Also, of course, we got the whiz. Whiz, get up here and hang out with us. So, OK, we want to kick this off, guys. Just big picture. If we think about Spectrum and we think about its context at RC, and maybe we'll start with you, Lucas, how is Spectrum actually useful for this training of SLMs? How do you think about it in today's world of buzzwords? How do you manage this? And how would you sort of clarify it for our audience today? How do you manage this and how would you sort of clarify it for our audience today? When it comes to making domain adapted language models, there's several things that are really important. But from a business sense, it's not hurting the original model. So not losing a ton of information from that original model while you're adding new business-related or private data related information, and speed. The benefit of Spectrum over current fine tuning or pre-training methods is it can work across the stack. So it works with our continued pre-training, it works with fine tuning, and it works with DPO. As well as it's much faster than just doing a full fine tuning, or even in many cases doing something like Qlora or LoRa up to 30. I would say the average is around 34% faster than those alternatives. And then it does really, really well at reducing catastrophic forgetting, because you're only training the most important parts of the model and other parts of the model stay frozen and aren't trained at all. So good. So many threads to pull on there. I wonder, Fernando, if you could bring a little business acumen into this discussion. Is this something that most of the customers at RC are using today? Are people leveraging Spectrum quite a bit in their pipelines to train their SLMs? And if so, is it happening in sort of a continued pre-training phase, a fine-tuning phase, or an alignment phase, as alluded to by Lucas? Currently, Spectrum is being used on all of our training since we have joined RSC and quickly integrated this technology on our whole pipeline. So most of these models that you have seen out there, all of them were trained using Spectrum. And it's worthy also mentioning that one of the key things that we have observed, and it's by design how we construct the spectrum, is that we don't suffer from the problem that LoRa or QLoRa actually suffer, which is you learn less and forget less in case of LoRa. And there is a paper paper a very recent paper regarding this topic why oh in the spectrum which is one of the most appealing of features of the method you learn more and forget less because you're messing up with way less weight Oh learn more, forget less. First shots fired at Laura. Love it. Wiz, is this aligning with your understanding of Laura and Spectrum so far? I mean, definitely the case that with Laura, you learn less and forget less. I mean, that is classic paper that just came out. It is one of those, both something that's powerful about LoRa, but also obviously not as great. So the idea that something like Spectrum can let us have our cake and eat it too is definitely something that's exciting, right? We want to be able to train these models on lower capacity hardware. We want to be able to make these models more customizable by people without this, you know, incredible amount of hardware. And so being able to do that in a way that doesn't as much sacrifice the, you know, the amount that you can train the model is pretty exciting. Yeah. I mean, it's definitely on the list as well. Well, like one thing, and I want to just I want to just pick your guys' brain about this a little bit, because we see this with other things at RSC, right? Which is this idea that similar to how MergeKit allows you to have these base models, and then people can merge them, and then these artifacts kind of sit around, and useful artifacts from good instruction tuning and, you know, useful artifacts from like good instruction tuning and things like that exist. Right. I see that there's this a similar thing in Spectrum where we have kind of the ability for others to contribute the actual Spectrum configs for us. So we don't even necessarily need to go through that initial process, that initial process of creating those, like what, what led you guys to that, that decision? Spectrum came about due to wanting to train.
Spectrum: Training Domain-Adapted SLMs
We explore the growing trend of Small Language Models (SLMs) and their specialization in domain-specific tasks. Discover how SLMs, paired with advanced fine-tuning techniques like Spectrum, are revolutionizing AI. Spectrum leverages a signal-to-noise ratio (SNR) to optimize which layers of a model to fine-tune, balancing performance and cost effectively. We'll dive into the details of Spectrum, comparing it to industry standards like Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA). Learn from experts at Arcee.ai, the minds behind the Spectrum paper, and uncover the practical applications of their innovative methods. If you're aiming to enhance your domain-adapted language models with cutting-edge techniques, this event is a must-attend. 


Event page: https://bit.ly/spectrumarcee

Have a question for a speaker? Drop them here: 
https://app.sli.do/event/trwY664NtPdpUBWJ2321JY

Speakers: 
​Lucas Atkins, Research Engineer at Arcee.ai
https://www.linkedin.com/in/lucas-atkins-2892482b6/

Fernando Fernandes Neto,  AI Research Scientist at Arcee.ai
https://www.linkedin.com/in/fernando-fernandes-neto-762a471a/

Dr. Greg, Co-Founder & CEO AI Makerspace
https://www.linkedin.com/in/gregloughane

The Wiz, Co-Founder & CTO AI Makerspace
https://www.linkedin.com/in/csalexiuk/

Apply for our new AI Engineering Bootcamp on Maven today! 
https://bit.ly/aie1

For team leaders, check out!
https://aimakerspace.io/gen-ai-upskilling-for-teams/

Join our community to start building, shipping, and sharing with us today!
https://discord.gg/RzhvYvAwzA

How'd we do? Share your feedback and suggestions for future events.
https://forms.gle/CP9VQuUkGKZTQvEn8