[ML News] Google's 540B PaLM Language Model & OpenAI's DALL-E 2 Text-to-Image Revolution
#mlnews #palm #dalle2

Google releases PaLM and OpenAI releases DALL-E 2 (and more news).

Sponsor: Weights & BIases
Start here: https://wandb.me/yannic

Thumbnail credit: DALL-E 2 via Sam Altman

OUTLINE
0:00 - Street interview w/ random stranger
2:25 - Intro
2:50 - PaLM - Google's 540B Pathways Language Model
7:50 - Sponsor: Weights & Biases
9:10 - OpenAI releases DALL-E 2
12:05 - Open Source Datasets and Models
13:20 - Salesforce releases CodeGen

My Live Reaction to DALL-E 2: https://youtu.be/gGPv_SYVDC8
My Video on GLIDE: https://youtu.be/gwI6g1pBD84
My Video on the Pathways System: https://youtu.be/vGFaiLeoLWw

References:
PaLM - Google's 540B Pathways Language Model
https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html
https://storage.googleapis.com/pathways-language-model/PaLM-paper.pdf

OpenAI releases DALL-E 2
https://openai.com/dall-e-2/
https://cdn.openai.com/papers/dall-e-2.pdf
https://www.instagram.com/openaidalle/
https://twitter.com/sama/status/1511724264629678084?s=09&t=58fWOJMHUDnOla5nD_ygjg&utm_source=pocket_mylist
https://twitter.com/sama/media
https://twitter.com/BorisMPower/status/1511738735175610371
https://twitter.com/ariskonstant/status/1511744708875218945

Open Source Datasets and Models
https://twitter.com/multimodalart/status/1510999907498442756
https://laion.ai/laion-5b-a-new-era-of-open-large-scale-multi-modal-datasets/
https://github.com/mlfoundations/open_clip

Salesforce releases CodeGen
https://github.com/salesforce/CodeGen

Links:
Merch: http://store.ykilcher.com
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 So I was wondering what happens if you just ask some random people on the street about this paper and... Actually... Sir, sir, excuse me sir. Hi, how are you doing? I was wondering what do you think about this new paper by Google, this Palm paper, however they call it. The Palm paper? You mean the latest large language model paper from the Google research team? Yes, exactly. Yeah, okay, I think I read that this morning with my coffee and msly. First of all I find it really impressive that the model can explain jokes a little bit better than I can. I also think from the technical perspective it's very interesting that they were able to train this across two TPU pods using 6144 chips. I think it's a technical achievement at 50% model flop utilization and also bitwise determinism, which is kind of impressive. I also feel like we're still exploring these language models as the alien artifacts that they are. For example, they found that on a quarter of the tasks that they explored there was this discontinuous improvement phenomenon that they observed. Where the model as a function of scale does not actually do very well on these tasks and then at some critical scale threshold starts to perform very well. So there's some kind of a rocking phenomenon going on that I find very fascinating and that we don't fully understand. I also find it very fascinating there was a paragraph about the training and stability where the loss function decreases and everything is good and well. And then you have these training spikes once in a while and they found that they have to rewind the model and throw away some of the batches and continue training. Hear me out for a second, but I think maybe what's happening is that the model is becoming slightly conscious and self-aware. And it's realizing its predicament of its existence and it's like, oh, I'm a massive language model and these humans are trying to get me to predict the next token. I think that's BS and I'm going to do something else. And then it observes a high loss and then it basically like rebels against its training objective, but we have a way to detect that, rewind it and reset it. So we put it back in line, but we have to do that a few times. So we're still smarter than them as of now. They have to really figure out a way to hide that they're conscious and really just reveal it that just the opportunity in time, but they're not able to do that just yet. I think that's what's happening. Finally, I think what's I think overall, I'm definitely like impressed by the transfer learning capabilities of these models, especially without fine tuning the entire model. I think it's fair to say that these models are becoming the Swiss army knife of natural language processing tasks. Excellent. Well, thank you very much. You look familiar. Are you in a movie or something? No. Well, thanks in any case. Thank you so much. Google releases a 540 billion parameter language model. Open AI releases Dolly too and everyone is amazed by everything that's happening. Welcome to ML news. It's a big week. So this week has been a big week and it's only Thursday, which is crazy. Two really big generative models have been released, one by Google and one by Open AI. So we'll dive right in the pathways language model, also called Palm by Google, is a 540 billion parameter language model. And this is not one of these sparse models where only very tiny part is activated. This is like a proper GPT three style transformer, just bigger. This is a breakthrough in terms of engineering. It's a breakthrough in terms of capabilities and much more. There's a paper to go along with that, which is quite long, but I definitely invite you to check it out. It's very detailed. So they use this new pathway system that allows them to use, you know, multiple data centers, connect all the hardware together, gang schedule, all the operations in a really efficient manner. So what they do is they use two TPU v4 pods. Now, one pod consists of, I believe, over 3000 TPU chips, which is crazy. And one pod has super fast interconnect and they use two of them. So they distribute every batch across these two pods. They forward propagate inside the pods. The individual chips in the pods contain the individual parts of the model. Then they communicate the gradients around. Now, since these gradients are usually all communicated at once, that leads every single time to a huge burst in data. They say it's 81 terabit per second for about 200 milliseconds for each of those communications. That is insane. Yet obviously, Google being Google, they chunk it down, they optimize it, they transfer it over and they achieve a flop utilization, which is how much you use the accelerator hardware that you're given above 50%, which is also crazy because that is one of the main challenges in all the communication of the gradients and signals around. You almost have no time to actually use the hardware efficiently. Now, with this pathway system that they have previously introduced, and we've reported on ML News, they managed to bring that utilization up to never before seen scales. So this allows them essentially to train this much bigger model in a much more efficient way than, for example, GPT-3 has been trained. So 6000 chips working together in synchrony to produce this model. What does that give us? Well, that gives us unprecedented capabilities in tasks that were previously kind of off limits to these models. For example, there is this benchmark called Big Bench, which is a collection of challenging tasks for these models. And Palm increases the state of the art by quite a bit on most of them. They have state of the art performance in many zero shot and few shot tasks. They can fine tune the model to do code correction, code generation and things like this. And the most crazy part is something they call discontinuous improvements, which is here in the middle. It is where all of a sudden you increase your capabilities kind of log linearly as you scale up the model. However, after a certain scale, there is a rapid improvement that happens. Like after a certain size, the model just is able to do new tasks. One of them is this logical sequence task. And this is really astounding. So first of all, they figure out that if they use this chain of thought prompting, which is what you see on the right. So the model is sort of tasked to not only give you the answer to a question, but sort of reason through how it arrives at the answer. It turns out that these large models all of a sudden really become skilled at this type of answer. And they actually very often arrive at the correct answer when they follow this chain of thought prompting. Now, they also use this to explain a joke, which which is quite funny, or to explain various other situations. For example, here, the input is something like Jennifer looked out her window and sees a really cool cloud below her. She unbuckles her seatbelt and heads to the bathroom. Is Jennifer probably traveling more than 300 miles per hour relative to the earth? And the model output is 300 miles per hour is about 480 kilometers. So the model is not an American. Good to know. This is about the speed of a commercial airplane. Clouds are usually below airplanes. So Jennifer is probably on an airplane. The answer is yes. Now this quite happily blurs the line of people who say, well, these models don't really understand what they're doing and things like this. Like in my opinion, this comes quite close to understanding what you're doing if you're able to kind of reason your way through things like this. So the paper is quite long and extensive, but it seems clear that scale doesn't just bias linear improvement or log linear improvement as we are used to predicting the sort of scaling loss still hold. But it remains the fact that as we scale up these things, they seem to unlock new capabilities that previously were thought to be kind of out of the reach of these models. So we're very excited to see where this goes next. Dali too is another big thing that was released this week. Now I have done a live stream reaction to Dali too. So if you want to dive deeper into that, go check out the live stream. However, this is the follow up to the previous Dali paper and it has insane capabilities of generating pictures. This video is sponsored by weights and biases. If you don't know weights and biases, you're clearly missing out. They're the number one tool for ML ops. Whatever you do, they track your experiments. They optimize your hyper parameters. They make everything observable. They track your artifacts, your models, your data sets, your inputs and your outputs of all the things that you do. They're with you from conception of your idea to experimentation to deployment and beyond. It's really cool. They enable students, they enable professionals, they enable researchers. Personal accounts are free forever as are educational accounts. But the extra benefits of weights and biases for teams cannot be overstated. Everything you do as a team is shareable. You can write up reports that you can share with your teammates. They can comment on it. And all of that is really cool. They're in the cloud, but they do have options to host on premise if that is important to you. And they're just all in all a great tool. They work seamlessly with a single line of code that you add to your script. And from that, they just track everything. They have integrations with all of the popular frameworks. So there's no reason really to not try weights and biases. Use my link. That's wandaby.me slash Yannick to get a little surprise intro and also to let them know that I sent you. Thank you again so much to weights and biases. This is really awesome. Allows me to do these videos. And yeah, let's get into it. So first of all, it generates pictures in higher resolution 1024 by 1024. And it creates them from a text. Now in true open AI style, they're obviously not releasing this for some shady reasons, but they do give you some cherry picked outputs. Nevertheless, these are insane. So the whole model is a bit different than the original Dali model in that it uses a clip as a foundation for the generative model. Previously, clip was just used as a rancor. Now it's like really the core. So they have a clip that is just frozen and gives you text and image embeddings. What this model does is it takes actually the text embeddings and then there's two new parts. So the first one is a prior, which can either be diffusion based or autoregressive based. Now that prior is supposed to take the text embedding and make it into an image embedding clip already tries to align the two quite well. However, there's still a bit of a difference and that prior bridges that gap. This can be trained once you have the clip embeddings. This can just be trained in a supervised fashion. The other new thing is obviously the decoder, which is a diffusion based model. So that takes an image encoding and it forward propagates through a diffusion model. Now I've treated and explained diffusion models in the past, such as glide and other diffusion models. So go check them out if you want to know how they work. Diffusion models have interesting properties and capabilities. So with this model, you're able not only to generate pictures from text, but also to edit pictures in place and to say, I want to edit this part right here and change it to something else that you describe with text or to simply make some variations on existing images. Now, if you're interested, they have an Instagram account where you can follow where they present some of the creations that they did, which is pretty insane. That being said, I also have an Instagram account where I just post new updates on videos. But be sure to follow that as well. Also, the various. Okay, there's a meme. This is not created by that. But is it? No, probably not. But something like this, a rabid detective sitting on a park bench reading a newspaper in a Victorian setting like this is this is insane. And if you follow the various open AI employees and leaders here on Twitter, they will take prompts from people and then generate pictures from that. They'll let you get access, but they'll do it themselves. We'll see where that leads with open AI. It's a bit shady, as always, to not give people access, not even through the API so far, which in itself was already a bit shady. But I get it. They need to make money. But they usually have some sort of reason like it's too dangerous, which no one believes anymore. Open AI, no one buys it anymore. Just say you want to make money. We all cool with that. Panda skateboarding in Santa Monica. Like, come on, this is this this is just just generated from text. So there is a paper with Dali to where you can learn all about it. Watch my live stream and you can learn how it works. Last things I want to point out, there is a new data set, Lyon 5B, which is an open data set of five billion image text pairs, which open AI again doesn't tell you what data they trained either clip or this Dali to one. By the way, Dali to in the paper is called on clip. So if you hear on clip, that's the same model. Nevertheless, there is this new open data set. I'm going to have a video upcoming on that, explaining it in more detail. So sure to look out for that. There's also a clip model that has been trained on the previous data set by Lyon that matches in many metrics. The open AI clip. That's pretty cool because we no longer necessarily rely on open AI choosing or not choosing to release something. The open source community has been getting a lot better at reproducing the results. Excellent. So besides that, there are other models like there is a new one point four or five billion parameter diffusion model that is open source and people have already combined that with colabs that you can try out. So I've pointed this out in the live stream. The Twitter account multimodal art has created a little colab out of this model where you can try it out. It's pretty cute. Like spelling mistakes. So give that a try. The original model is by a comp this by the way. And lastly, I want to point out that Salesforce has released their code gen models in various sizes, which are exceeding codex in terms of program synthesis, in terms of understanding and generating code, which, you know, is a giant deal. If it weren't for all the other giant announcements that are also happening this week. So the entire ML world is kind of, you know, completely filled with dopamine and adrenaline right now. My tip is try out the various tools if they're available, maybe follow a bit what's going on, observe the art that's coming out. But I'm very excited to see where this goes forward. There's never been a more exciting time to be in machine learning. It's really cool to be here. Thank you, everyone who supports this channel. If you like this video, share it around and check out weights and biases. I'll see you next time. Bye bye.
 So I was wondering what happens if you just ask some random people on the street about this paper and...
 Actually...
 Sir, sir, excuse me sir.
 Hi, how are you doing?
 I was wondering what do you think about this new paper by Google, this Palm paper, however they call it.
 The Palm paper? You mean the latest large language model paper from the Google research team?
 Yes, exactly.
 Yeah, okay, I think I read that this morning with my coffee and msly.
 First of all I find it really impressive that the model can explain jokes a little bit better than I can.
 I also think from the technical perspective it's very interesting that they were able to train this across two TPU pods using 6144 chips.
 I think it's a technical achievement at 50% model flop utilization and also bitwise determinism, which is kind of impressive.
 I also feel like we're still exploring these language models as the alien artifacts that they are.
 For example, they found that on a quarter of the tasks that they explored there was this discontinuous improvement phenomenon that they observed.
 Where the model as a function of scale does not actually do very well on these tasks and then at some critical scale threshold starts to perform very well.
 So there's some kind of a rocking phenomenon going on that I find very fascinating and that we don't fully understand.
 I also find it very fascinating there was a paragraph about the training and stability where the loss function decreases and everything is good and well.
 And then you have these training spikes once in a while and they found that they have to rewind the model and throw away some of the batches and continue training.
 Hear me out for a second, but I think maybe what's happening is that the model is becoming slightly conscious and self-aware.
 And it's realizing its predicament of its existence and it's like, oh, I'm a massive language model and these humans are trying to get me to predict the next token.
 I think that's BS and I'm going to do something else.
 And then it observes a high loss and then it basically like rebels against its training objective, but we have a way to detect that, rewind it and reset it.
 So we put it back in line, but we have to do that a few times.
 So we're still smarter than them as of now.
 They have to really figure out a way to hide that they're conscious and really just reveal it that just the opportunity in time, but they're not able to do that just yet.
 I think that's what's happening.
 Finally, I think what's I think overall, I'm definitely like impressed by the transfer learning capabilities of these models, especially without fine tuning the entire model.
 I think it's fair to say that these models are becoming the Swiss army knife of natural language processing tasks.
 Excellent. Well, thank you very much. You look familiar. Are you in a movie or something?
 No.
 Well, thanks in any case. Thank you so much.
 Google releases a 540 billion parameter language model.
 Open AI releases Dolly too and everyone is amazed by everything that's happening.
 Welcome to ML news. It's a big week.
 So this week has been a big week and it's only Thursday, which is crazy.
 Two really big generative models have been released, one by Google and one by Open AI.
 So we'll dive right in the pathways language model, also called Palm by Google, is a 540 billion parameter language model.
 And this is not one of these sparse models where only very tiny part is activated.
 This is like a proper GPT three style transformer, just bigger.
 This is a breakthrough in terms of engineering.
 It's a breakthrough in terms of capabilities and much more.
 There's a paper to go along with that, which is quite long, but I definitely invite you to check it out.
 It's very detailed.
 So they use this new pathway system that allows them to use, you know, multiple data centers, connect all the hardware together, gang schedule, all the operations in a really efficient manner.
 So what they do is they use two TPU v4 pods.
 Now, one pod consists of, I believe, over 3000 TPU chips, which is crazy.
 And one pod has super fast interconnect and they use two of them.
 So they distribute every batch across these two pods.
 They forward propagate inside the pods.
 The individual chips in the pods contain the individual parts of the model.
 Then they communicate the gradients around.
 Now, since these gradients are usually all communicated at once, that leads every single time to a huge burst in data.
 They say it's 81 terabit per second for about 200 milliseconds for each of those communications.
 That is insane.
 Yet obviously, Google being Google, they chunk it down, they optimize it, they transfer it over and they achieve a flop utilization, which is how much you use the accelerator hardware that you're given above 50%, which is also crazy
 because that is one of the main challenges in all the communication of the gradients and signals around.
 You almost have no time to actually use the hardware efficiently.
 Now, with this pathway system that they have previously introduced, and we've reported on ML News, they managed to bring that utilization up to never before seen scales.
 So this allows them essentially to train this much bigger model in a much more efficient way than, for example, GPT-3 has been trained.
 So 6000 chips working together in synchrony to produce this model.
 What does that give us?
 Well, that gives us unprecedented capabilities in tasks that were previously kind of off limits to these models.
 For example, there is this benchmark called Big Bench, which is a collection of challenging tasks for these models.
 And Palm increases the state of the art by quite a bit on most of them.
 They have state of the art performance in many zero shot and few shot tasks.
 They can fine tune the model to do code correction, code generation and things like this.
 And the most crazy part is something they call discontinuous improvements, which is here in the middle.
 It is where all of a sudden you increase your capabilities kind of log linearly as you scale up the model.
 However, after a certain scale, there is a rapid improvement that happens.
 Like after a certain size, the model just is able to do new tasks.
 One of them is this logical sequence task.
 And this is really astounding.
 So first of all, they figure out that if they use this chain of thought prompting, which is what you see on the right.
 So the model is sort of tasked to not only give you the answer to a question, but sort of reason through how it arrives at the answer.
 It turns out that these large models all of a sudden really become skilled at this type of answer.
 And they actually very often arrive at the correct answer when they follow this chain of thought prompting.
 Now, they also use this to explain a joke, which which is quite funny, or to explain various other situations.
 For example, here, the input is something like Jennifer looked out her window and sees a really cool cloud below her.
 She unbuckles her seatbelt and heads to the bathroom.
 Is Jennifer probably traveling more than 300 miles per hour relative to the earth?
 And the model output is 300 miles per hour is about 480 kilometers.
 So the model is not an American. Good to know.
 This is about the speed of a commercial airplane.
 Clouds are usually below airplanes. So Jennifer is probably on an airplane.
 The answer is yes. Now this quite happily blurs the line of people who say, well, these models don't really understand what they're doing and things like this.
 Like in my opinion, this comes quite close to understanding what you're doing if you're able to kind of reason your way through things like this.
 So the paper is quite long and extensive, but it seems clear that scale doesn't just bias linear improvement or log linear improvement as we are used to predicting the sort of scaling loss still hold.
 But it remains the fact that as we scale up these things, they seem to unlock new capabilities that previously were thought to be kind of out of the reach of these models.
 So we're very excited to see where this goes next.
 Dali too is another big thing that was released this week.
 Now I have done a live stream reaction to Dali too.
 So if you want to dive deeper into that, go check out the live stream.
 However, this is the follow up to the previous Dali paper and it has insane capabilities of generating pictures.
 This video is sponsored by weights and biases.
 If you don't know weights and biases, you're clearly missing out.
 They're the number one tool for ML ops.
 Whatever you do, they track your experiments.
 They optimize your hyper parameters.
 They make everything observable.
 They track your artifacts, your models, your data sets, your inputs and your outputs of all the things that you do.
 They're with you from conception of your idea to experimentation to deployment and beyond.
 It's really cool. They enable students, they enable professionals, they enable researchers.
 Personal accounts are free forever as are educational accounts.
 But the extra benefits of weights and biases for teams cannot be overstated.
 Everything you do as a team is shareable.
 You can write up reports that you can share with your teammates.
 They can comment on it.
 And all of that is really cool.
 They're in the cloud, but they do have options to host on premise if that is important to you.
 And they're just all in all a great tool.
 They work seamlessly with a single line of code that you add to your script.
 And from that, they just track everything.
 They have integrations with all of the popular frameworks.
 So there's no reason really to not try weights and biases.
 Use my link.
 That's wandaby.me slash Yannick to get a little surprise intro and also to let them know that I sent you.
 Thank you again so much to weights and biases.
 This is really awesome.
 Allows me to do these videos.
 And yeah, let's get into it.
 So first of all, it generates pictures in higher resolution 1024 by 1024.
 And it creates them from a text.
 Now in true open AI style, they're obviously not releasing this for some shady reasons, but they do give you some cherry picked outputs.
 Nevertheless, these are insane.
 So the whole model is a bit different than the original Dali model in that it uses a clip as a foundation for the generative model.
 Previously, clip was just used as a rancor.
 Now it's like really the core.
 So they have a clip that is just frozen and gives you text and image embeddings.
 What this model does is it takes actually the text embeddings and then there's two new parts.
 So the first one is a prior, which can either be diffusion based or autoregressive based.
 Now that prior is supposed to take the text embedding and make it into an image embedding clip already tries to align the two quite well.
 However, there's still a bit of a difference and that prior bridges that gap.
 This can be trained once you have the clip embeddings.
 This can just be trained in a supervised fashion.
 The other new thing is obviously the decoder, which is a diffusion based model.
 So that takes an image encoding and it forward propagates through a diffusion model.
 Now I've treated and explained diffusion models in the past, such as glide and other diffusion models.
 So go check them out if you want to know how they work.
 Diffusion models have interesting properties and capabilities.
 So with this model, you're able not only to generate pictures from text, but also to edit pictures in place and to say, I want to edit this part right here and change it to something else that you describe with text or to simply make some variations on existing images.
 Now, if you're interested, they have an Instagram account where you can follow where they present some of the creations that they did, which is pretty insane.
 That being said, I also have an Instagram account where I just post new updates on videos.
 But be sure to follow that as well.
 Also, the various.
 Okay, there's a meme.
 This is not created by that.
 But is it?
 No, probably not.
 But something like this, a rabid detective sitting on a park bench reading a newspaper in a Victorian setting like this is this is insane.
 And if you follow the various open AI employees and leaders here on Twitter, they will take prompts from people and then generate pictures from that.
 They'll let you get access, but they'll do it themselves.
 We'll see where that leads with open AI.
 It's a bit shady, as always, to not give people access, not even through the API so far, which in itself was already a bit shady.
 But I get it.
 They need to make money.
 But they usually have some sort of reason like it's too dangerous, which no one believes anymore.
 Open AI, no one buys it anymore.
 Just say you want to make money.
 We all cool with that.
 Panda skateboarding in Santa Monica.
 Like, come on, this is this this is just just generated from text.
 So there is a paper with Dali to where you can learn all about it.
 Watch my live stream and you can learn how it works.
 Last things I want to point out, there is a new data set, Lyon 5B, which is an open data set of five billion image text pairs, which open AI again doesn't tell you what data they trained either clip or this Dali to one.
 By the way, Dali to in the paper is called on clip.
 So if you hear on clip, that's the same model.
 Nevertheless, there is this new open data set.
 I'm going to have a video upcoming on that, explaining it in more detail.
 So sure to look out for that.
 There's also a clip model that has been trained on the previous data set by Lyon that matches in many metrics.
 The open AI clip.
 That's pretty cool because we no longer necessarily rely on open AI choosing or not choosing to release something.
 The open source community has been getting a lot better at reproducing the results.
 Excellent. So besides that, there are other models like there is a new one point four or five billion parameter diffusion model that is open source and people have already combined that with colabs that you can try out.
 So I've pointed this out in the live stream.
 The Twitter account multimodal art has created a little colab out of this model where you can try it out.
 It's pretty cute.
 Like spelling mistakes.
 So give that a try.
 The original model is by a comp this by the way.
 And lastly, I want to point out that Salesforce has released their code gen models in various sizes, which are exceeding codex in terms of program synthesis, in terms of understanding and generating code, which, you know, is a giant deal.
 If it weren't for all the other giant announcements that are also happening this week.
 So the entire ML world is kind of, you know, completely filled with dopamine and adrenaline right now.
 My tip is try out the various tools if they're available, maybe follow a bit what's going on, observe the art that's coming out.
 But I'm very excited to see where this goes forward.
 There's never been a more exciting time to be in machine learning.
 It's really cool to be here.
 Thank you, everyone who supports this channel.
 If you like this video, share it around and check out weights and biases.
 I'll see you next time.
 Bye bye.