[ML Olds] Meta Research Supercluster | OpenAI GPT-Instruct | Google LaMDA | Drones fight Pigeons
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper gpt3 gpt-3 gpt 3 openai open ai meta supercluster rsc meta rsc meta research super cluster meta research supercluster mlnews ml news kilcher news openai gpt instruct gpt3 follow instructions how does gpt3 work google lamda google lambda
#mlnews #rsc #gpt3

Some things we've missed in recent weeks!

OUTLINE:
0:00 - Intro & Overview
0:40 - Meta builds AI Research Supercluster (RSC)
2:25 - OpenAI trains GPT-3 to follow instructions
4:10 - Meta AI releases multilingual language models
4:50 - Google LaMDA dialogue models
5:50 - Helpful Things
8:25 - Training the alpha matte generator for Pixel 6
10:15 - Drones used to deter pigeons on buildings
11:05 - IBM sells some Watson Health assets for USD 1B

Merch: http://store.ykilcher.com

References:
https://ai.facebook.com/blog/ai-rsc/?utm_source=pocket_mylist

https://openai.com/blog/instruction-following/
https://cdn.openai.com/papers/Training_language_models_to_follow_instructions_with_human_feedback.pdf
https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/

https://twitter.com/MetaAI/status/1486745968372551686?utm_source=pocket_mylist
https://arxiv.org/pdf/2112.10668.pdf
https://github.com/pytorch/fairseq/tree/main/examples/xglm

https://ai.googleblog.com/2022/01/lamda-towards-safe-grounded-and-high.html?m=1&utm_source=pocket_mylist
https://arxiv.org/pdf/2201.08239.pdf

https://evolutiongym.github.io/?utm_source=pocket_mylist
https://evolutiongym.github.io/all-tasks
https://evolutiongym.github.io/documentation
https://arxiv.org/pdf/2201.09863.pdf
https://github.com/EvolutionGym
https://huggingface.co/blog/sb3
https://twitter.com/Sentdex/status/1489991413005787139
https://github.com/lvwerra/trl?utm_source=pocket_mylist

https://ai.googleblog.com/2022/01/accurate-alpha-matting-for-portrait.html
https://polyhaven.com/hdris

https://ieeexplore.ieee.org/document/9656717

https://www.bloomberg.com/news/articles/2022-01-21/ibm-is-said-to-near-sale-of-watson-health-to-francisco-partners
https://archive.ph/xadf9

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Meta builds a humongous computer, OpenAI teaches their language models to follow instructions, and we battle pigeons with drones. Welcome to ML News. Welcome to ML News. Now I have to say these aren't exactly news. This is stuff that we've somehow missed or skipped or anything like this from the last two to three weeks, let's say. So consider this more ML olds. But if you're interested, stick around. If you actually do enjoy new ML News, be sure to be subscribed to the channel, leave a like and as always, tell me what you think in the comments. I'm very happy to take your feedback. First story, Meta AI has released a blog post introducing the AI research supercluster, Meta's cutting edge AI supercomputer for AI research. Now this, this is a big computer. Like, look at that. The RSC, the research supercluster, that is ginormous. I mean, look at this. Does anyone get the vibes of like, so this is where your box would go? In any case, this is a huge thing. It consists of 760 DGX A100 boxes. That is a total of 6080 GPUs and all of them are A100s. But did you wonder why you can't get your hands on any GPU anywhere on the planet for the last one and a half years or so? Yeah, they're all right here. Now obviously, obviously all of this is connected with super duper Infini band. It has 175 petabytes of storage. It has 175 petabytes of a flash array storage as 46 petabytes of cache storage, and it has 10 petabytes of flash blade storage. I have no clue what these things mean, but it's a lot. So the blog post goes a little bit into the history of how it was built a bit more at what it contains, how they make it secure, how they handle the difficulties of the last two years and so on. This cluster is supposed to support Meta AI production and research workloads and is already operational but is planned to finish to its full scale up to the mid 2022. Look here's the box. Here's the box. Where does the box go? Where does your box go? Your box goes there. Really nice. This is where your box would go. Check out blog post if you want to learn more. OpenAI has released a blog post in paper titled Aligning Language Models to Follow Instructions, where they've fine tuned GPT-3 to follow human instructions. They give an example right here where if you ask GPT-3 something like explain the moon landing to a six year old in a few sentences, it would sort of continue the pattern as GPT-3 does. It would say explain the theory of gravity, explain the theory of relativity. So it would sort of treat this as a regular language modeling prompt. If you actually want to make GPT-3 answer the question, you have to give it a few examples of question, answer, question, answer beforehand. OpenAI went and fine tuned their language models to obey instructions more clearly. So the model that results is instruct GPT, which in this case would output people went to the moon, they took pictures of what they saw and sent them back to Earth so we could all see them. Supposedly. Like yeah, like that ever happened. So the main challenge here is the data collection part. Fine tuning a big language model requires a bit of data. And they largely followed earlier work called learning from human preferences. So this is a multi step process. First they collect a small labeled data set. After that, they let humans sort of rank answers of the model and they train a reward model from that. And in the end, they use reinforcement learning against that learned reward model. Now in their own words, this is nothing new, they say. However, the smaller instruct GPT model are preferred by humans to the larger GPT-3 models, which is interesting. There's a paper to go along with it, give it a read if you're interested. Data AI writes that they are releasing a series of multilingual autoregressive language models up to 7.5 billion parameters, which significantly outperform English centric language models in few shot learning on 20 plus languages. Again, there is a paper to go along with it and the code and models are available on the repository. These are multilingual models and most of the models are trained on 30 different languages. As you can see, they do scale up in partially layers, also model dimensions, and there's even one model that's trained on over 134 languages. So if you're interested in multilingual models, give this model a try. Google releases a paper called Lambda language models for dialogue applications along with a blog post where they detail a new foray into dialogue models using large language models. What's interesting here is that they're not only interested in generating the most likely data, they do pre-train on pure language modeling, but then when it comes to fine tuning on dialogue data, they have various metrics and for each of these metrics, they have classifiers that classifies the outputs of the language model, which is trying to optimize. So some of these outputs are safety, sensibility, specificity, interestingness, and so on. The model is also capable of doing factual grounding as it is augmented by a retrieval stage during the generation process. So technically, it can look up something on Wikipedia before it answers you, which is pretty cool. If you're interested in dialogue models, definitely give this blog post and paper a read. Alright some helpful stuff for this week. Evolution Gym is a large scale benchmark for evolving soft robots. So contrary to classic reinforcement learning where your agent is kind of fixed and static and has a bunch of actions available, in soft robots, you can also choose how to compose your robot. So here's a bunch of examples of soft robots. Now as you can see, the policy isn't the hard part. It's actually the hard part, how you even construct your robots from the individual building blocks. So here you can see a walker, there is object manipulation, climbing, I believe they do have some some other examples right here. There's climbing. It looks pretty cool. So even though it's still reinforcement learning, this is a cool domain. I like it. There's a paper to go along with the release. If you're interested in soft robotics and reinforcement learning, give it a read. Stable Baselines 3 is in the hugging face hub. Stable Baselines 3 is a reinforcement learning library that provides kind of baseline implementations of RL algorithms such as proximal policy optimization, Q learning and more. So now these are on the hugging face hub and you can just kind of download the strategies, maybe, not entirely sure. But if you're into reinforcement learning, give this a try. I've seen that sent decks has already made a video using stable baselines three. But as far as I could see, he has not used the hugging face hub. So sorry, Harrison, you actually did like a lot of work for nothing. You like pip installed the actual package. Why? In related news, I want to highlight this repository right here by Leandro von Vera, who released this repository to perform reinforcement learning with transformers. It's a library slash example code repository of training transformers using proximal policy optimization. If you don't know proximal policy optimization is a reinforcement learning algorithm that tries to maximize the reward, but at the same time, stay close to some known state like a baseline implementation, a baseline model, or a previous version of the model that you're training. This prevents fatal steps like single steps that bring you into really bad local minima. Now I was going to say if you're into the combination of language and reinforcement learning, check this out. But I mean, transformers have gone way beyond language by this point. So if you're into RL and transformers, this might be the repo for you. Okay, this was it for our helpful stuff this week. I hope you were helped. Our next news is Google AI releasing a blog post called accurate alpha matting for portrait mode selfies on Pixel 6. Yes, it is a bit of an ad for their Pixel phones, but also it details quite extensively how they went about training a system that would generate the alpha map for the types of portrait pictures. The goal here is to get a mask on top of a picture that separates foreground meaning if it's a portrait, the person from background so that you can swap out the background. This is challenging because as you can see right here, hair is often a problem. There are very fine details, the lighting can come from any place and that might not match up with the background and so on. So they detail what kind of model architecture they did. It consists of progressive up sampling, which we've seen a couple of times so far. And the most interesting part is the data generation process. They have this giant studio with like surround array of cameras and lights so they can activate different lights at different time and get kind of a 3D impression of the subject that is at the center. They're also able to capture different lighting effects on the subject, which is also really helpful because the second thing they do is they place that subject into various kind of fake backgrounds. And these fake backgrounds are not just any picture. They are sort of 360 pictures of scenes. So what they can do is they can dynamically relight the subject so that it actually fits into the background. And from that, they generate the training data to the AlphaMAT classifier. Now give this a read if you want to learn more. I was just impressed how deep one can go in like a single task, like how much there is if you really want to solve something to the level of where you can build it into a product and it performs well. So that's pretty cool. I saw this article on IEEE Explorer called Autonomous Detection and Deterrence of Pigeons on Buildings by Drones. And this is the most metal thing ever. I mean poor drones. So there's this camera on roofs and it locates pigeons and when it sees a flock of them, pigeons would destroy their things with their what they call it excrements, but it's poop. So they poop and it destroys the buildings. So they want to shoo them away to prevent damage and difficult and dangerous cleaning procedures. So the camera spots the pigeons and it sends in the drone. And here you can see like a first person view of the drone is like it waits and it's like activate, it just goes after the pigeons. I'm so sorry pigeons. Machines one nature zero. Your move pigeons. All right, our last news. Bloomberg writes IBM sells some Watson Health assets for more than $1 billion. So apparently the whole Watson project hasn't really panned out for IBM the way they wanted it to after the initial successes of winning Jeopardy. It just kind of got nowhere it seemed like. I've heard from a lot of people that it was just not doing the things they promised it to do when they actually deployed it in let's say health settings or the finance world. And I don't know exactly what they tried, but the uniform feedback I've heard is that it just underwhelmed in practice. Now there are some customers using it and IBM says it's still committed to the project. Note that it is only selling some parts and only of Watson Health. That is not the entire Watson project. It's just a health sub project, which might come with its own difficulties, let's say regulatory and whatnot. So IBM says that it is going to focus more on being a cloud provider for AI applications. Well I guess that's where the big money is right now. I guess if you're a cloud provider now you can just you can just print money. So good on IBM instead of losing money. They're now printing it. Excellent. This was already it for ML news. If you have any comments, anything to say, please leave it in the comments. Merch still available and I'll see you next time. Bye bye.
 Meta builds a humongous computer,
 OpenAI teaches their language models to follow instructions,
 and we battle pigeons with drones.
 Welcome to ML News.
 Welcome to ML News.
 Now I have to say these aren't exactly news.
 This is stuff that we've somehow missed or skipped or anything like this
 from the last two to three weeks, let's say.
 So consider this more ML olds.
 But if you're interested, stick around.
 If you actually do enjoy new ML News,
 be sure to be subscribed to the channel,
 leave a like and as always, tell me what you think in the comments.
 I'm very happy to take your feedback.
 First story, Meta AI has released a blog post introducing the AI research supercluster,
 Meta's cutting edge AI supercomputer for AI research.
 Now this, this is a big computer.
 Like, look at that.
 The RSC, the research supercluster, that is ginormous.
 I mean, look at this.
 Does anyone get the vibes of like, so this is where your box would go?
 In any case, this is a huge thing.
 It consists of 760 DGX A100 boxes.
 That is a total of 6080 GPUs and all of them are A100s.
 But did you wonder why you can't get your hands on any GPU anywhere on the planet for
 the last one and a half years or so?
 Yeah, they're all right here.
 Now obviously, obviously all of this is connected with super duper Infini band.
 It has 175 petabytes of storage.
 It has 175 petabytes of a flash array storage as 46 petabytes of cache storage, and it has
 10 petabytes of flash blade storage.
 I have no clue what these things mean, but it's a lot.
 So the blog post goes a little bit into the history of how it was built a bit more at
 what it contains, how they make it secure, how they handle the difficulties of the last
 two years and so on.
 This cluster is supposed to support Meta AI production and research workloads and is already
 operational but is planned to finish to its full scale up to the mid 2022.
 Look here's the box.
 Here's the box.
 Where does the box go?
 Where does your box go?
 Your box goes there.
 Really nice.
 This is where your box would go.
 Check out blog post if you want to learn more.
 OpenAI has released a blog post in paper titled Aligning Language Models to Follow Instructions,
 where they've fine tuned GPT-3 to follow human instructions.
 They give an example right here where if you ask GPT-3 something like explain the moon
 landing to a six year old in a few sentences, it would sort of continue the pattern as GPT-3
 does.
 It would say explain the theory of gravity, explain the theory of relativity.
 So it would sort of treat this as a regular language modeling prompt.
 If you actually want to make GPT-3 answer the question, you have to give it a few examples
 of question, answer, question, answer beforehand.
 OpenAI went and fine tuned their language models to obey instructions more clearly.
 So the model that results is instruct GPT, which in this case would output people went
 to the moon, they took pictures of what they saw and sent them back to Earth so we could
 all see them.
 Supposedly.
 Like yeah, like that ever happened.
 So the main challenge here is the data collection part.
 Fine tuning a big language model requires a bit of data.
 And they largely followed earlier work called learning from human preferences.
 So this is a multi step process.
 First they collect a small labeled data set.
 After that, they let humans sort of rank answers of the model and they train a reward model
 from that.
 And in the end, they use reinforcement learning against that learned reward model.
 Now in their own words, this is nothing new, they say.
 However, the smaller instruct GPT model are preferred by humans to the larger GPT-3 models,
 which is interesting.
 There's a paper to go along with it, give it a read if you're interested.
 Data AI writes that they are releasing a series of multilingual autoregressive language models
 up to 7.5 billion parameters, which significantly outperform English centric language models
 in few shot learning on 20 plus languages.
 Again, there is a paper to go along with it and the code and models are available on the
 repository.
 These are multilingual models and most of the models are trained on 30 different languages.
 As you can see, they do scale up in partially layers, also model dimensions, and there's
 even one model that's trained on over 134 languages.
 So if you're interested in multilingual models, give this model a try.
 Google releases a paper called Lambda language models for dialogue applications along with
 a blog post where they detail a new foray into dialogue models using large language
 models.
 What's interesting here is that they're not only interested in generating the most likely
 data, they do pre-train on pure language modeling, but then when it comes to fine tuning on dialogue
 data, they have various metrics and for each of these metrics, they have classifiers that
 classifies the outputs of the language model, which is trying to optimize.
 So some of these outputs are safety, sensibility, specificity, interestingness, and so on.
 The model is also capable of doing factual grounding as it is augmented by a retrieval
 stage during the generation process.
 So technically, it can look up something on Wikipedia before it answers you, which is
 pretty cool.
 If you're interested in dialogue models, definitely give this blog post and paper a read.
 Alright some helpful stuff for this week.
 Evolution Gym is a large scale benchmark for evolving soft robots.
 So contrary to classic reinforcement learning where your agent is kind of fixed and static
 and has a bunch of actions available, in soft robots, you can also choose how to compose
 your robot.
 So here's a bunch of examples of soft robots.
 Now as you can see, the policy isn't the hard part.
 It's actually the hard part, how you even construct your robots from the individual
 building blocks.
 So here you can see a walker, there is object manipulation, climbing, I believe they do
 have some some other examples right here.
 There's climbing.
 It looks pretty cool.
 So even though it's still reinforcement learning, this is a cool domain.
 I like it.
 There's a paper to go along with the release.
 If you're interested in soft robotics and reinforcement learning, give it a read.
 Stable Baselines 3 is in the hugging face hub.
 Stable Baselines 3 is a reinforcement learning library that provides kind of baseline implementations
 of RL algorithms such as proximal policy optimization, Q learning and more.
 So now these are on the hugging face hub and you can just kind of download the strategies,
 maybe, not entirely sure.
 But if you're into reinforcement learning, give this a try.
 I've seen that sent decks has already made a video using stable baselines three.
 But as far as I could see, he has not used the hugging face hub.
 So sorry, Harrison, you actually did like a lot of work for nothing.
 You like pip installed the actual package.
 Why?
 In related news, I want to highlight this repository right here by Leandro von Vera,
 who released this repository to perform reinforcement learning with transformers.
 It's a library slash example code repository of training transformers using proximal policy
 optimization.
 If you don't know proximal policy optimization is a reinforcement learning algorithm that
 tries to maximize the reward, but at the same time, stay close to some known state like
 a baseline implementation, a baseline model, or a previous version of the model that you're
 training.
 This prevents fatal steps like single steps that bring you into really bad local minima.
 Now I was going to say if you're into the combination of language and reinforcement
 learning, check this out.
 But I mean, transformers have gone way beyond language by this point.
 So if you're into RL and transformers, this might be the repo for you.
 Okay, this was it for our helpful stuff this week.
 I hope you were helped.
 Our next news is Google AI releasing a blog post called accurate alpha matting for portrait
 mode selfies on Pixel 6.
 Yes, it is a bit of an ad for their Pixel phones, but also it details quite extensively
 how they went about training a system that would generate the alpha map for the types
 of portrait pictures.
 The goal here is to get a mask on top of a picture that separates foreground meaning
 if it's a portrait, the person from background so that you can swap out the background.
 This is challenging because as you can see right here, hair is often a problem.
 There are very fine details, the lighting can come from any place and that might not
 match up with the background and so on.
 So they detail what kind of model architecture they did.
 It consists of progressive up sampling, which we've seen a couple of times so far.
 And the most interesting part is the data generation process.
 They have this giant studio with like surround array of cameras and lights so they can activate
 different lights at different time and get kind of a 3D impression of the subject that
 is at the center.
 They're also able to capture different lighting effects on the subject, which is also really
 helpful because the second thing they do is they place that subject into various kind
 of fake backgrounds.
 And these fake backgrounds are not just any picture.
 They are sort of 360 pictures of scenes.
 So what they can do is they can dynamically relight the subject so that it actually fits
 into the background.
 And from that, they generate the training data to the AlphaMAT classifier.
 Now give this a read if you want to learn more.
 I was just impressed how deep one can go in like a single task, like how much there is
 if you really want to solve something to the level of where you can build it into a product
 and it performs well.
 So that's pretty cool.
 I saw this article on IEEE Explorer called Autonomous Detection and Deterrence of Pigeons
 on Buildings by Drones.
 And this is the most metal thing ever.
 I mean poor drones.
 So there's this camera on roofs and it locates pigeons and when it sees a flock of them,
 pigeons would destroy their things with their what they call it excrements, but it's poop.
 So they poop and it destroys the buildings.
 So they want to shoo them away to prevent damage and difficult and dangerous cleaning
 procedures.
 So the camera spots the pigeons and it sends in the drone.
 And here you can see like a first person view of the drone is like it waits and it's like
 activate, it just goes after the pigeons.
 I'm so sorry pigeons.
 Machines one nature zero.
 Your move pigeons.
 All right, our last news.
 Bloomberg writes IBM sells some Watson Health assets for more than $1 billion.
 So apparently the whole Watson project hasn't really panned out for IBM the way they wanted
 it to after the initial successes of winning Jeopardy.
 It just kind of got nowhere it seemed like.
 I've heard from a lot of people that it was just not doing the things they promised it
 to do when they actually deployed it in let's say health settings or the finance world.
 And I don't know exactly what they tried, but the uniform feedback I've heard is that
 it just underwhelmed in practice.
 Now there are some customers using it and IBM says it's still committed to the project.
 Note that it is only selling some parts and only of Watson Health.
 That is not the entire Watson project.
 It's just a health sub project, which might come with its own difficulties, let's say
 regulatory and whatnot.
 So IBM says that it is going to focus more on being a cloud provider for AI applications.
 Well I guess that's where the big money is right now.
 I guess if you're a cloud provider now you can just you can just print money.
 So good on IBM instead of losing money.
 They're now printing it.
 Excellent.
 This was already it for ML news.
 If you have any comments, anything to say, please leave it in the comments.
 Merch still available and I'll see you next time.
 Bye bye.