[ML News] Meta's OPT 175B language model | DALL-E Mega is training | TorToiSe TTS fakes my voice
#mlnews #dalle #gpt3

An inside look of what's happening in the ML world!

Sponsor: Weights & Biases
https://wandb.me/yannic

OUTLINE:
0:00 - Intro
0:20 - Sponsor: Weights & Biases
1:40 - Meta AI releases OPT-175B
4:55 - CoCa: New CLIP-Competitor
8:15 - DALL-E Mega is training
10:05 - TorToiSe TTS is amazing!
11:50 - Investigating Vision Transformers
12:50 - Hugging Face Deep RL class launched
13:40 - Helpful Things
17:00 - John Deere's driverless tractors

References:
Meta AI releases OPT-175B
https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/
https://arxiv.org/abs/2205.01068
https://arxiv.org/pdf/2205.01068.pdf
https://github.com/facebookresearch/metaseq/tree/main/projects/OPT
https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf
https://github.com/facebookresearch/metaseq/tree/main/projects/OPT/chronicles
https://twitter.com/yoavgo/status/1522150063815987201

CoCa: New CLIP-Competitor
https://arxiv.org/abs/2205.01917
https://arxiv.org/pdf/2205.01917.pdf

DALL-E Mega is training
https://twitter.com/borisdayma
https://twitter.com/borisdayma/status/1521891895001112577
https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-Mega--VmlldzoxODMxMDI2

TorToiSe TTS is amazing!
https://github.com/neonbjb/tortoise-tts
https://nonint.com/static/tortoise_v2_examples.html
https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR
https://github.com/neonbjb

Investigating Vision Transformers
https://github.com/sayakpaul/probing-vits/?utm_source=pocket_mylist
https://twitter.com/RisingSayak/status/1515918406171914240?utm_source=pocket_mylist
https://keras.io/examples/vision/probing_vits/
https://github.com/sayakpaul/probing-vits/tree/main/notebooks?utm_source=pocket_mylist

Hugging Face Deep RL class launched
https://github.com/huggingface/deep-rl-class

Helpful Things
https://merantix-momentum.com/technology/squirrel/?utm_source=pocket_mylist
https://github.com/merantix-momentum/squirrel-core?utm_source=pocket_mylist
https://pyscript.net/?utm_source=pocket_mylist
https://github.com/google-research/big_vision
https://deepsportradar.github.io/challenge.html
https://github.com/DeepSportRadar/camera-calibration-challenge
https://twitter.com/alekseykorshuk/status/1515989357961920514?utm_source=pocket_mylist
https://github.com/AlekseyKorshuk/huggingnft

John Deere's driverless tractors
https://thenextweb.com/news/john-deere-slowly-becoming-one-worlds-most-important-ai-companies
https://tractorhacking.github.io/

Links:
Merch: https://ykilcher.com/merch
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Meta builds and releases a 175 billion parameter language model, a contrastive captioning model out competes clip and the open source Dali mega looks better and better every day it trains. Welcome to ML news. This video is sponsored by weights and biases. If you don't know weights and biases, you're clearly missing out. They're the number one tool for ml ops, whatever you do, they track your experiments, they optimize your hyper parameters, they make everything observable, they track your artifacts, your models, your data sets, your inputs and your outputs of all the things that you do. They're with you from conception of your idea to experimentation to deployment and beyond. It's really cool. They enable students, they enable professionals, they enable researchers, personal accounts are free forever as our educational accounts, but the extra benefits of weights and biases for teams cannot be overstated. Everything you do as a team is shareable, you can write up reports that you can share with your teammates, they can comment on it and all of that is really cool. They're in the cloud, but they do have options to host on premise if that is important to you. And they're just all in all a great tool. They work seamlessly with a single line of code that you add to your script. And from that, they just track everything, they have integrations with all of the popular frameworks. So there's no reason really to not try weights and biases. Use my link that's wandaby.me slash Yannick to get a little surprise intro and also to let them know that I sent you thank you again so much to weights and biases. This is really awesome allows me to do these videos. And yeah, let's get into it. Hello and welcome to ML news. My name is Yannick. Welcome to the channel we discuss the newest happenings in the machine learning world. In fact, so much time has passed since the last news that I'm having to split this episode into two parts. So you're seeing part one right now. And part two is going to be released in a few days. So keep an eye out for that. Facebook releases a giant language model the same size as GPT three, but they're just releasing it out into the wild, not entirely as we're going to discuss. So this is the first thing where open AI gets serious competition from open source models. So let's talk about it. Meta AI has a blog post called democratizing access to large scale language models with OPT 175 B. Now, as I already said, 175 billion parameters is the exact size of opening as GPT three, remember that GPT three is behind an API. So you don't necessarily get access to it. Now, openly has been building and improving GPT three over the time that it has existed, apparently or supposedly, and the model we're getting out here out of Facebook is just a straightforward language model. So without access to GPT three, we can't exactly tell where the differences are. However, in the papers, the author state that OPT 175 B is comparable to GPT three, while requiring only one seventh of the carbon footprint to develop. Now, besides the blog post and the paper, there is a GitHub repository to go along with that, which contains the code and also the pre trained models, you can see they release models starting from 125 million parameters all the way up to 175 billion. Now you can get up to the 30 billion model just like that to download the larger models, you have to actually go and ask them for it, they will share it with interested researchers, but they don't release it out into the world quite yet. So you're going to have to wait on that just a bit more. What is also interesting is that they published a log book of training this model. Now the log book is essentially where the researchers keep track of what happened during training of this giant language model. And so there's a goal, there's a purpose, and there's some instructions. And after that, you can find essentially logs of what people did, what they experienced, what they ran, what problems they encountered, and so on. So here you can see all kinds of stuff, like people looking at the plots and finding out interesting trends in the plots, like repeated patterns and some metrics, you can find logs of stuff crashing, stuff trying to auto recover, and so on. In fact, many times these people had to rewind, had to restart, had to get their system out from some kind of failed state and so on. It really gives you a nice insight into the behind the scenes of training these large language models, because all we end up seeing is usually just the shiny paper at the front and the nice results. But reading this gives you a much better impression of just how much work goes into this. So big props to Meta, not only for releasing the models, but also showing a little bit behind the curtain of what's going on. Though the best take on this goes to you off Goldberg saying Meta released OPT 175B, but have you heard anything of OPT 175A? What are they hiding? Couldn't have said it better. There's a new paper called COCA, Contrastive Captioners are Image Text Foundation models by Google Research. This is a model that ultimately competes with CLIP among other things. So the model is trained on the configuration on the left side right here, there is an image encoder, there is a unimodal text encoder, which means it only takes text, there is a contrastive loss between these two encoders. And then there is a multimodal text decoder, which means that it is essentially a language model that also gets the image tokens as an input. So there are two losses involved right here. One is the contrastive loss between the encoders. And the other one is the captioning loss from the language model. There are a number of special things. The first one is that the unimodal text decoder is also an autoregressive language model, which is pretty interesting in itself, because usually people use bidirectional models if they just want to encode stuff. But also the system can be trained once and then used in different configurations for either fine tuning or even zero shot inference. For example, the image encoder will have very good representations for fine tuning a classifier on top of it. And the unimodal encoders, both image and text can be used directly as a replacement for CLIP in order to assess the alignment between text and images given their contrastive loss training. Of course, given that the model is trained essentially as an autoencoder for the text with the help of the image, the model can also be used to do image captioning and other things to do with connecting text and images where the output is text. There is a bit of a deeper insight into the model, you can see that the image is tokenized in classic VIT style, whereas the text is first run through an autoregressive decoder style model, even though it is technically encoding the text. What's special is that we put a CLS token at the end, usually it's put at the beginning, it doesn't really matter in bidirectional models. But in unidirectional models and autoregressive models, we have to put it at the end to get the actual representation out the representation of that CLS token and a pooled representation of the image tokens will be used for the contrastive loss, whereas the rest meaning the image tokens themselves and the text tokens will be used for the multimodal text decoder. In this plot right here in purple, you can see the new model is called coca, by the way, and how it stacks up against other models that are either not specialized, just connecting text and images somehow, or even specialized model for something. So the difference here are pretty significant sometimes, for example, this is the table on zero shot image classification on image net. Now zero shot can be achieved by these image text models. Because what you can do is you can input the image and then ask the model to simply get you the distance to all of the class labels as text is actually a pretty neat way to do classification. And you can classify into an open set and coca beats the other models by a pretty good amount, especially compared to clip in the first row. And you see just how much progress is being made in this field. So again, you see there is another competitor to one of OpenAI's flagship models clip. So along today, we've seen a competitor to GPT three, we've seen a competitor to clip and what's the last one of OpenAI's flagship models? Well, it's Dali. And as it turns out, Boris Dima is leading an effort to reproduce Dali out in the open. Now the first model Dali mini has already been made. And in fact, you can try it out. It's pretty good. So this is the Eiffel tower on the moon. However, Dali mini, as the name says, is kind of a smallish version of Dali. The new effort is Dali mega, which is a proper large model and the replication that resembles Dali in scale and performance. Here you can see intermediate results. This model is training as we speak. So on May 2nd, it was 29% done. And you can see that it's already producing pretty stunning images with respect to the prompts that are given. On May 4th, it was at 45%. And this prompt right here by Rohan Anil was apparently pretty difficult for the model up until this point. It is Spider-Man on a horse. And yeah, it doesn't look too well yet. And one person has actually responded by inputting that prompt into Dali two and giving us the picture out of that. Or at least that's what is claimed. And these look pretty sweet, I have to say. So I'm not sure if Dali mega is going to match Dali two in its performance. It's certainly going to be a good model. But I do feel that Dali two with its new architecture relying on multiple internal models, combining clip with diffusion models, and so on. And what I also suspect is that Dali two had very high quality data, at least in part. So I guess it's going to be difficult to reach that level of performance, but still an open source model that has such a good performance is quite cool. So this project runs out in the open, you can actually look at the report and the ongoing experiments on weights and biases, a link to it in the description, check it out. Tortoise TTS is a multi voice text to speech system that is trained with an emphasis on quality and emphasis on quality means it's very slow, just so we're clear. But it is pretty cool version 2.1 has just been released. And now you have the ability to use your own pre trained models. And I have to say this model is extremely good, like it's very good. Now there is a page with handpicked results. And there is a collab where you can experiment with the model yourself. But the author James Betker has made a custom model for me and sent me a little sample out of that model. And you just have to listen to this. I have never spoken this text. In fact, this is a message that I sent him on Discord. And now it's just available in my voice. That would be fun. Is this the model that is called Tortoise because it's very slow? Insane. It's me is crazy. I mean, imagine just the possibilities that open up with the ability to just clone voices and let anyone say pretty much anything you want. I mean, obviously, there's going to be dangers ahead. I mean, essentially, you can't trust audio recordings anymore where a person says anything. But there's also really cool things ahead. And in fact, the project does include a detector, a model that recognizes whether or not a given sample was created by the tortoise system. Now knowing a bit about adversarial examples, it's fairly easy to still use the system, take the output and then modify the output such that this detector will not be tripped. But at least it is a first line of defense against people who simply mindlessly produce stuff and then put it out into the wild. But let me know what you think. This is essentially a deep fake system for voices. I think it's very cool. Let me know in the comments. This GitHub repository is very cool. Probing vits vision transformers. It's by Aritra Rory Koshypati and Sayak Paul and investigates visual transformers and various variants of that like the original vit, the diet and dino and applies various techniques to investigate these models. We've also written this up in an excellent article on keros.io that really takes you through the research how to interact with their stuff and reproduce their results. So the questions that can be answered like this are things like what do vision transformers learn or where in a picture do vision transformers pay attention to when they make a given classification. All of these things can be achieved via techniques such as attention rollout, visualizing the attention in an image, visualizing positional encodings and much more. If you're interested to learn more about how to investigate vision transformers, check out the repository and this article. Hugging face launches the deep reinforcement learning class. So this is a class about deep reinforcement learning is fairly applied, but there's also theory. And the cool thing is you will actually be using modern code. So libraries such as stable baselines three, which is not only for people trying to learn reinforcement learning, but this is a serious library that is used in practice. Now in conjunction with the hugging face hub, you can just publish the agents you train and many people have already done so. Now the course has just started. So there's still ample time to join if you want to do so. Obviously, you can still go and read older stuff, but the next class will appear on May 11th and it's going to be a surprise. Oh, wow. A surprise. All right, a few helpful things for this week. Squirrel is a library to load, transform, share, and generally interact with data sets. So this unifies a number of ways on how to interact with data sets, such as how to load data sets either from disk or from distributed sources, then import them, transform them in some way and then feed them into your machine learning pipeline. And as you can see from their benchmarks on various data sets, such as CIFAR 100, which is images, Wikitext 103, which is text data set, they outperform other data ingestion pipelines by quite a bit. So check out Squirrel Core on GitHub. PyScript is not necessarily a machine learning thing, but it is Python inside of HTML, which is pretty crazy. And this isn't just some gimmicky thing. No, you can seriously pack your modules and then ship them inside of the browser, run Python in the browser. There's even a two way interaction between JavaScript and Python. So this makes for some exciting new applications that are now possible. If you're interested, check out pyScript.net. Big Vision is an open source version of the code base of a line of work, starting with Vision Transformers over MLP Mixer, all the way to locked image text tuning. So all of this code is by the same or similar groups out of Google. And this code base is the home for that line of research. So check it out if you are interested. It's always cool to be just a bit closer to the source of research than the finished polished repositories we usually see out of papers. Do you like sports? Do you want to make some money and also get to publish a paper at a workshop? These competitions here might be for you. The fifth international ACM workshop on multimedia content analysis in sports hosts these four challenges. There is ball 3D localization, camera calibration, instance segmentation and player re identification. All of them have associated datasets and you can get started right away. There's even some starter code available on GitHub for each of the challenges for you to get into it. The challenges are structured in two phases. In the first phase, the winners go on and get to publish their papers in the workshop. And in the second phase, there's actual money involved. So the best team is going to win 500 bucks and the most innovative solution also wins 500 bucks. And these two things can be the same team. So that's a cool incentive to propose some innovative solution that is also very good. Alexey Korshuk releases hugging NFT. This is a code base to train GANs on NFTs. Now where have I seen this before? This was literally released like one week after I got done filming for my GANFT video. Now I went through the painstaking process of actually getting the data, getting the code, training all of it myself, looking at the hyper parameters, yada, yada, yada. Alexey releases a code base that makes all of this much, much easier because it's specifically designed to interact with NFT collections. So if you want to reproduce what took me multiple weeks to perform in a few hours, check out this repository. All right, here's our last article for the day. John Deere is slowly becoming one of the world's most important AI companies. This is by The Next Web and is an article about an interview with John Deere, not the person John Deere, a person from the company John Deere, about their advances into AI. And I have to say it's pretty cool, whereas we still lack full self-driving in cars on the roads. For tractors, this has long been a reality. Not only can these tractors drive themselves, the farmer can just control them via an app. It's really crazy. Now obviously this is promotional material right here, but I'm not really doubting that they are already doing this. What's crazy here is that the tractors are not only used for things like tilling, but they can also remove weeds with very high precision as they do the tilling. So pretty crazy what's possible. And we've gone from a world where almost everyone was a farmer to where almost no one is a farmer. And pretty soon actually, no one's going to be a farmer. Now I'm not sure we should probably not lose the last, you know, one or 2% of humanity that can actually produce food, but I have to admit it does look pretty sweet to have a driverless tractor. Now wherever there is technology, there are hackers. So this is tractorhacking.github.io, which is not a malicious hacking, but apparently they say John Deere has overly strict security on the electrical component of its tractor. Sure, overly strict security on the electrical components of your tractor. That's certainly a bad thing. Oh no, security. But they do have a point. Obviously these vendors lock down all the electronics so that only they and their technician can update them. So this project is investigating how to bypass those things in order to repair those tractors themselves. So this already sounds a lot more reasonable than just the name tractor hacking, but I still think it's pretty cool. So if you want to take part, there is a form right here. I don't know what happens if you fill out the form, but you know, give it a shot. And that was already it for ML news. Thank you so much for being here. Stay tuned for part two, which is going to come in a few days time. See you around.
 Meta builds and releases a 175 billion parameter language model, a contrastive captioning model
 out competes clip and the open source Dali mega looks better and better every day it trains.
 Welcome to ML news. This video is sponsored by weights and biases. If you don't know weights
 and biases, you're clearly missing out. They're the number one tool for ml ops, whatever you do,
 they track your experiments, they optimize your hyper parameters, they make everything observable,
 they track your artifacts, your models, your data sets, your inputs and your outputs of all
 the things that you do. They're with you from conception of your idea to experimentation
 to deployment and beyond. It's really cool. They enable students, they enable professionals,
 they enable researchers, personal accounts are free forever as our educational accounts,
 but the extra benefits of weights and biases for teams cannot be overstated. Everything you do as
 a team is shareable, you can write up reports that you can share with your teammates, they can comment
 on it and all of that is really cool. They're in the cloud, but they do have options to host on
 premise if that is important to you. And they're just all in all a great tool. They work seamlessly
 with a single line of code that you add to your script. And from that, they just track everything,
 they have integrations with all of the popular frameworks. So there's no reason really to not
 try weights and biases. Use my link that's wandaby.me slash Yannick to get a little surprise
 intro and also to let them know that I sent you thank you again so much to weights and biases.
 This is really awesome allows me to do these videos. And yeah, let's get into it.
 Hello and welcome to ML news. My name is Yannick. Welcome to the channel we discuss the newest
 happenings in the machine learning world. In fact, so much time has passed since the last news that
 I'm having to split this episode into two parts. So you're seeing part one right now. And part two
 is going to be released in a few days. So keep an eye out for that. Facebook releases a giant
 language model the same size as GPT three, but they're just releasing it out into the wild,
 not entirely as we're going to discuss. So this is the first thing where open AI gets serious
 competition from open source models. So let's talk about it. Meta AI has a blog post called
 democratizing access to large scale language models with OPT 175 B. Now, as I already said,
 175 billion parameters is the exact size of opening as GPT three, remember that GPT three
 is behind an API. So you don't necessarily get access to it. Now, openly has been building and
 improving GPT three over the time that it has existed, apparently or supposedly, and the model
 we're getting out here out of Facebook is just a straightforward language model. So without access
 to GPT three, we can't exactly tell where the differences are. However, in the papers, the
 author state that OPT 175 B is comparable to GPT three, while requiring only one seventh of the
 carbon footprint to develop. Now, besides the blog post and the paper, there is a GitHub repository
 to go along with that, which contains the code and also the pre trained models, you can see they
 release models starting from 125 million parameters all the way up to 175 billion. Now you can get up
 to the 30 billion model just like that to download the larger models, you have to actually go and ask
 them for it, they will share it with interested researchers, but they don't release it out into
 the world quite yet. So you're going to have to wait on that just a bit more. What is also interesting
 is that they published a log book of training this model. Now the log book is essentially where the
 researchers keep track of what happened during training of this giant language model. And so
 there's a goal, there's a purpose, and there's some instructions. And after that, you can find
 essentially logs of what people did, what they experienced, what they ran, what problems they
 encountered, and so on. So here you can see all kinds of stuff, like people looking at the plots
 and finding out interesting trends in the plots, like repeated patterns and some metrics, you can
 find logs of stuff crashing, stuff trying to auto recover, and so on. In fact, many times these people
 had to rewind, had to restart, had to get their system out from some kind of failed state and so
 on. It really gives you a nice insight into the behind the scenes of training these large language
 models, because all we end up seeing is usually just the shiny paper at the front and the nice
 results. But reading this gives you a much better impression of just how much work goes into this.
 So big props to Meta, not only for releasing the models, but also showing a little bit behind the
 curtain of what's going on. Though the best take on this goes to you off Goldberg saying Meta released
 OPT 175B, but have you heard anything of OPT 175A? What are they hiding? Couldn't have said it better.
 There's a new paper called COCA, Contrastive Captioners are Image Text Foundation models by
 Google Research. This is a model that ultimately competes with CLIP among other things. So the
 model is trained on the configuration on the left side right here, there is an image encoder, there
 is a unimodal text encoder, which means it only takes text, there is a contrastive loss between
 these two encoders. And then there is a multimodal text decoder, which means that it is essentially
 a language model that also gets the image tokens as an input. So there are two losses involved
 right here. One is the contrastive loss between the encoders. And the other one is the captioning loss
 from the language model. There are a number of special things. The first one is that the
 unimodal text decoder is also an autoregressive language model, which is pretty interesting in
 itself, because usually people use bidirectional models if they just want to encode stuff. But also
 the system can be trained once and then used in different configurations for either fine tuning or
 even zero shot inference. For example, the image encoder will have very good representations for
 fine tuning a classifier on top of it. And the unimodal encoders, both image and text can be
 used directly as a replacement for CLIP in order to assess the alignment between text and images
 given their contrastive loss training. Of course, given that the model is trained essentially as an
 autoencoder for the text with the help of the image, the model can also be used to do image
 captioning and other things to do with connecting text and images where the output is text. There
 is a bit of a deeper insight into the model, you can see that the image is tokenized in classic
 VIT style, whereas the text is first run through an autoregressive decoder style model, even though
 it is technically encoding the text. What's special is that we put a CLS token at the end,
 usually it's put at the beginning, it doesn't really matter in bidirectional models. But in
 unidirectional models and autoregressive models, we have to put it at the end to get the actual
 representation out the representation of that CLS token and a pooled representation of the image
 tokens will be used for the contrastive loss, whereas the rest meaning the image tokens themselves
 and the text tokens will be used for the multimodal text decoder. In this plot right here in purple,
 you can see the new model is called coca, by the way, and how it stacks up against other models
 that are either not specialized, just connecting text and images somehow, or even specialized
 model for something. So the difference here are pretty significant sometimes, for example, this
 is the table on zero shot image classification on image net. Now zero shot can be achieved by
 these image text models. Because what you can do is you can input the image and then ask the model
 to simply get you the distance to all of the class labels as text is actually a pretty neat
 way to do classification. And you can classify into an open set and coca beats the other models by a
 pretty good amount, especially compared to clip in the first row. And you see just how much progress
 is being made in this field. So again, you see there is another competitor to one of OpenAI's
 flagship models clip. So along today, we've seen a competitor to GPT three, we've seen a competitor
 to clip and what's the last one of OpenAI's flagship models? Well, it's Dali. And as it turns
 out, Boris Dima is leading an effort to reproduce Dali out in the open. Now the first model Dali
 mini has already been made. And in fact, you can try it out. It's pretty good. So this is the Eiffel
 tower on the moon. However, Dali mini, as the name says, is kind of a smallish version of Dali. The
 new effort is Dali mega, which is a proper large model and the replication that resembles Dali in
 scale and performance. Here you can see intermediate results. This model is training as we speak. So on
 May 2nd, it was 29% done. And you can see that it's already producing pretty stunning images
 with respect to the prompts that are given. On May 4th, it was at 45%. And this prompt right here by
 Rohan Anil was apparently pretty difficult for the model up until this point. It is Spider-Man on a
 horse. And yeah, it doesn't look too well yet. And one person has actually responded by inputting
 that prompt into Dali two and giving us the picture out of that. Or at least that's what is
 claimed. And these look pretty sweet, I have to say. So I'm not sure if Dali mega is going to match
 Dali two in its performance. It's certainly going to be a good model. But I do feel that Dali two
 with its new architecture relying on multiple internal models, combining clip with diffusion
 models, and so on. And what I also suspect is that Dali two had very high quality data, at least in
 part. So I guess it's going to be difficult to reach that level of performance, but still an
 open source model that has such a good performance is quite cool. So this project runs out in the
 open, you can actually look at the report and the ongoing experiments on weights and biases,
 a link to it in the description, check it out. Tortoise TTS is a multi voice text to speech system
 that is trained with an emphasis on quality and emphasis on quality means it's very slow,
 just so we're clear. But it is pretty cool version 2.1 has just been released. And now you have the
 ability to use your own pre trained models. And I have to say this model is extremely good, like
 it's very good. Now there is a page with handpicked results. And there is a collab where you can
 experiment with the model yourself. But the author James Betker has made a custom model for me and
 sent me a little sample out of that model. And you just have to listen to this. I have never spoken
 this text. In fact, this is a message that I sent him on Discord. And now it's just available in my
 voice. That would be fun. Is this the model that is called Tortoise because it's very slow? Insane.
 It's me is crazy. I mean, imagine just the possibilities that open up with the ability
 to just clone voices and let anyone say pretty much anything you want. I mean, obviously,
 there's going to be dangers ahead. I mean, essentially, you can't trust audio recordings
 anymore where a person says anything. But there's also really cool things ahead. And in fact,
 the project does include a detector, a model that recognizes whether or not a given sample was
 created by the tortoise system. Now knowing a bit about adversarial examples, it's fairly easy to
 still use the system, take the output and then modify the output such that this detector will
 not be tripped. But at least it is a first line of defense against people who simply mindlessly
 produce stuff and then put it out into the wild. But let me know what you think. This is essentially
 a deep fake system for voices. I think it's very cool. Let me know in the comments.
 This GitHub repository is very cool. Probing vits vision transformers. It's by Aritra Rory
 Koshypati and Sayak Paul and investigates visual transformers and various variants of that like
 the original vit, the diet and dino and applies various techniques to investigate these models.
 We've also written this up in an excellent article on keros.io that really takes you through the
 research how to interact with their stuff and reproduce their results. So the questions that
 can be answered like this are things like what do vision transformers learn or where in a picture
 do vision transformers pay attention to when they make a given classification. All of these things
 can be achieved via techniques such as attention rollout, visualizing the attention in an image,
 visualizing positional encodings and much more. If you're interested to learn more about how to
 investigate vision transformers, check out the repository and this article.
 Hugging face launches the deep reinforcement learning class. So this is a class about deep
 reinforcement learning is fairly applied, but there's also theory. And the cool thing is you
 will actually be using modern code. So libraries such as stable baselines three, which is not only
 for people trying to learn reinforcement learning, but this is a serious library that is used in
 practice. Now in conjunction with the hugging face hub, you can just publish the agents you train
 and many people have already done so. Now the course has just started. So there's still ample
 time to join if you want to do so. Obviously, you can still go and read older stuff, but the next
 class will appear on May 11th and it's going to be a surprise. Oh, wow. A surprise.
 All right, a few helpful things for this week. Squirrel is a library to load, transform, share,
 and generally interact with data sets. So this unifies a number of ways on how to interact with
 data sets, such as how to load data sets either from disk or from distributed sources, then import
 them, transform them in some way and then feed them into your machine learning pipeline. And as
 you can see from their benchmarks on various data sets, such as CIFAR 100, which is images,
 Wikitext 103, which is text data set, they outperform other data ingestion pipelines by
 quite a bit. So check out Squirrel Core on GitHub. PyScript is not necessarily a machine learning
 thing, but it is Python inside of HTML, which is pretty crazy. And this isn't just some gimmicky
 thing. No, you can seriously pack your modules and then ship them inside of the browser, run Python
 in the browser. There's even a two way interaction between JavaScript and Python. So this makes for
 some exciting new applications that are now possible. If you're interested, check out
 pyScript.net. Big Vision is an open source version of the code base of a line of work,
 starting with Vision Transformers over MLP Mixer, all the way to locked image text tuning.
 So all of this code is by the same or similar groups out of Google. And this code base is the
 home for that line of research. So check it out if you are interested. It's always cool to be just
 a bit closer to the source of research than the finished polished repositories we usually see out
 of papers. Do you like sports? Do you want to make some money and also get to publish a paper at a
 workshop? These competitions here might be for you. The fifth international ACM workshop on
 multimedia content analysis in sports hosts these four challenges. There is ball 3D localization,
 camera calibration, instance segmentation and player re identification. All of them have
 associated datasets and you can get started right away. There's even some starter code available on
 GitHub for each of the challenges for you to get into it. The challenges are structured in two phases.
 In the first phase, the winners go on and get to publish their papers in the workshop. And in the
 second phase, there's actual money involved. So the best team is going to win 500 bucks and the
 most innovative solution also wins 500 bucks. And these two things can be the same team. So that's
 a cool incentive to propose some innovative solution that is also very good. Alexey Korshuk
 releases hugging NFT. This is a code base to train GANs on NFTs. Now where have I seen this before?
 This was literally released like one week after I got done filming for my GANFT video. Now I went
 through the painstaking process of actually getting the data, getting the code, training all of it
 myself, looking at the hyper parameters, yada, yada, yada. Alexey releases a code base that makes all
 of this much, much easier because it's specifically designed to interact with NFT collections. So if
 you want to reproduce what took me multiple weeks to perform in a few hours, check out this repository.
 All right, here's our last article for the day. John Deere is slowly becoming one of the world's
 most important AI companies. This is by The Next Web and is an article about an interview with John
 Deere, not the person John Deere, a person from the company John Deere, about their advances into AI.
 And I have to say it's pretty cool, whereas we still lack full self-driving in cars on the roads.
 For tractors, this has long been a reality. Not only can these tractors drive themselves,
 the farmer can just control them via an app. It's really crazy. Now obviously this is promotional
 material right here, but I'm not really doubting that they are already doing this. What's crazy
 here is that the tractors are not only used for things like tilling, but they can also remove
 weeds with very high precision as they do the tilling. So pretty crazy what's possible. And
 we've gone from a world where almost everyone was a farmer to where almost no one is a farmer. And
 pretty soon actually, no one's going to be a farmer. Now I'm not sure we should probably not lose the
 last, you know, one or 2% of humanity that can actually produce food, but I have to admit it does
 look pretty sweet to have a driverless tractor. Now wherever there is technology, there are hackers.
 So this is tractorhacking.github.io, which is not a malicious hacking, but apparently they say John
 Deere has overly strict security on the electrical component of its tractor. Sure, overly strict
 security on the electrical components of your tractor. That's certainly a bad thing. Oh no,
 security. But they do have a point. Obviously these vendors lock down all the electronics so
 that only they and their technician can update them. So this project is investigating how to
 bypass those things in order to repair those tractors themselves. So this already sounds a
 lot more reasonable than just the name tractor hacking, but I still think it's pretty cool. So
 if you want to take part, there is a form right here. I don't know what happens if you fill out
 the form, but you know, give it a shot. And that was already it for ML news. Thank you so much for
 being here. Stay tuned for part two, which is going to come in a few days time. See you around.