[ML News] OpenAI removes GPT-3 waitlist | GauGAN2 is amazing | NYC regulates AI hiring tools
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper mlnews gaugan gaugan2 nvidia controllable gan openai gpt-3 gpt-3 beta gpt-3 waitlist gpt-3 access gpt-3 playground nyc ai hiring ai hiring tools helpful libraries machine learning news kilcher news everyday robots metnet 2 ai weather forecasting ai rain prediction google research deepmind google x boston dynamics mario kart 64 ai mario kart tensorkart
#mlnews #gaugan #gpt-3

Your weekly dose of ML News!
More GauGAN images here: https://drive.google.com/drive/folders/1tG1rpxP_mnspB1MWi9VZGScw5R-hxUdm?usp=sharing

OUTLINE:
0:00 - Intro
0:20 - Sponsor: Weights & Biases
2:20 - OpenAI's removes GPT-3 Waitlist
4:55 - NVIDIA releases GauGAN2 Webapp
9:45 - Everyday Robots tackles real-life tasks
12:15 - MetNet-2: 12-hour Rain Forecasting
14:45 - TinyML Dog Bark Stopper
15:55 - AI learns to drive Mario Kart 64 on real hardware
17:40 - NYC regulates bias in AI hiring tools
21:05 - Beverage companies big into AI
21:50 - How does AlphaZero play Chess?
23:35 - Helpful Things
28:00 - ArXiv founder awarded Einstein Foundation Award

References:

OpenAI's removes GPT-3 Waitlist
https://openai.com/blog/api-no-waitlist/
https://beta.openai.com/playground?model=davinci

NVIDIA releases GauGAN2 Webapp
https://www.reddit.com/r/MachineLearning/comments/r0mok4/p_nvidia_releases_web_app_for_gaugan2_which/?utm_source=pocket_mylist
http://gaugan.org/gaugan2/
https://blogs.nvidia.com/blog/2021/11/22/gaugan2-ai-art-demo/?ncid=so-twit-261232-vt16#cid=nr01_so-twit_en-us
https://blogs.nvidia.com/blog/2019/03/18/gaugan-photorealistic-landscapes-nvidia-research/
https://arxiv.org/abs/1903.07291

Everyday Robots tackles real-life tasks
https://everydayrobots.com/
https://www.wired.com/story/plaintext-alphabet-x-robots/
https://archive.ph/YC4XG#selection-925.354-925.397

MetNet-2: 12-hour Rain Forecasting
https://ai.googleblog.com/2021/11/metnet-2-deep-learning-for-12-hour.html

TinyML Dog Bark Stopper
https://www.hackster.io/NathanielF/tinyml-dog-bark-stopper-77e436

AI learns to drive Mario Kart 64 on real hardwware
https://www.youtube.com/watch?v=z9E38sN5nRQ

NYC regulates bias in AI hiring tools
https://www.nbcnewyork.com/news/local/nyc-aims-to-be-first-to-rein-in-artificial-intelligence-hiring-tools/3411736/

Beverage companies big into AI
https://www.just-drinks.com/features/which-beverages-companies-are-leading-the-way-in-artificial-intelligence-data/

How does AlphaZero play Chess?
https://arxiv.org/pdf/2111.09259.pdf
https://storage.googleapis.com/uncertainty-over-space/alphachess/index.html?board=08

Helpful Things
https://huggingface.co/sberbank-ai/rudalle-Emojich?utm_source=pocket_mylist
https://github.com/MathisFederico/OpenCodeBlocks?utm_source=pocket_mylist
https://blog.tensorflow.org/2021/11/introducing-tensorflow-gnn.html?linkId=8008555
https://github.com/tensorflow/gnn
https://github.com/jurgisp/pydreamer?utm_source=pocket_mylist
https://danijar.com/project/dreamerv2/
https://github.com/danijar/dreamerv2
https://deepgenx.com/
https://github.com/DeepGenX/CodeGenX
https://devpost.com/software/heyoh-camera?utm_source=pocket_mylist
https://heyoh-app.github.io/heyoh-project-page/
https://github.com/heyoh-app/heyoh-project-page

ArXiv founder awarded Einstein Foundation Award
https://idw-online.de/en/news781515?utm_source=pocket_mylist

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 GPT-3 is now free to access, Nvidia releases Gaogain 2 and it's amazing, and out of Google X comes Everyday Robots which aims to make robots handle everyday tasks. Welcome to ML News. Hey YouTube! Hey attention saws, what's up? This video is sponsored by Weights and Biases. Thank you so much to Weights and Biases for being a great sponsor. If you don't know Weights and Biases you should definitely check it out. It is a one stop shop for all your machine learning needs. It starts at tracking your experiments with a single line of code. Everything is logged to the cloud, your environment is logged, your outputs are logged, your models and datasets can be saved and iterated upon. And it's with you from conception of your idea all the way to deployment and monitoring. They have on-prem solutions, they have cloud solutions and it's completely free for personal use and for academic use. So please try out Weights and Biases. Today I want to highlight their jobs offerings. If you're looking for a job please consider Weights and Biases. As you can see right here they have all kinds of job openings from business operations, customer success. There is lots of engineering jobs. There's deep learning engineers, site reliability engineer, just regular software engineer, product engineer, infrastructure. There's deep learning engineer for growth. But even if you're not an engineer you can go into marketing, into people operations, product managers, all kinds of things and look at that they just need sales people. So if you're good at selling maybe this is your position. As you can see they have some jobs in North America, some are in Europe but a lot of jobs are actually remote. So whether you enjoy remote work or on-site work chances are Weights and Biases has something for you. As you know as we've reported right here Weights and Biases has just raised a giant amount of money at a 1 billion dollar valuation. Make sure you get a slice of that pie. Apply for a job today. Go to 1db.com, go to resources, click on careers and find all their job offerings right now. If you're not looking for a job check out their product. I'm sure you're gonna love it and thank you so much again to Weights and Biases for sponsoring this video. Alright let's get into it. OpenAI's blog says the OpenAI API is now available with no waitlist. That means that you can simply go, sign up and you get access to the API. The API includes things such as their language model, GPT-3 and so on. It includes things like the instruct models and these models are good at following things like instructions and also the codex models that generate code given a piece of natural language. A function to fill my bank account. Well I guess the model tells me that I actually need to make a deposit in order to fill my bank account. That's sad. Of course the flagship models are still the GPT models, specifically GPT-3, the largest version is called DaVinci. The best idea ever is? The best idea ever is the idea that is most useful to the most people. Thank you. DaVinci is a utilitarian, absolutely based. So even if you've used GPT-3 before and if that was a while back you might want to check it out again because the documentation has involved there are a lot of examples. OpenAI themselves have figured out a lot more about how to prompt these models in order to get good completions in order to actually make them do what you want them to do and there's a lot of useful stuff right here. I've actually made a poll about this in the past and over 1000 of you have responded and it turned out most of you didn't have access yet even though a large portion of you applied early. So to all of you who still don't have access this should help you. Now this doesn't come as a surprise as in recent times we've seen a lot of competitors to OpenAI simply giving people access to their API and not having them on a long wait list. So how much of this is well we finally figured it out and how much of it is please don't go to our competition we don't know. That being said OpenAI still wants to have a very tight control over people that actually use the API to build products. They say our work also allows us to review applications before they go live, monitor for misuse, support developers as their product scales and better understand the effects of this technology. Essentially they want to avoid at all costs that you build a product that in any way reflects negatively on OpenAI be that if the model makes some sort of a mistake or if the technology is used for a use case that maybe isn't super PR friendly. That is not good or bad it's just something you have to keep in mind when you go all in and build actually an application on the basis of an API like this. OpenAI releases the second iteration of their Gaogan model which is a generative adversarial network that doesn't just come up with stuff by itself but can be conditioned on certain inputs. Gaogan one was already being used to condition the model on sketches as you see here you can give a bunch of segmentation maps and then the model would dynamically adapt and generate a picture based on that. Gaogan two takes this a step further. Now you can also condition on words for example. In fact they have released a little web app and as you can see you can condition on a segmentation map that's what we saw in Gaogan one. You can condition on a sketch you can condition on a base image or on text and not only either or of these modalities but you can mix them all as you want. There is a Reddit post by the user Whiskey and some of the pictures that this user was able to generate with simply text prompts if I understand this correctly are just stunning by themselves. So here is a winter mountain landscape near sunset. Now interesting is what you can do this is a stream given a text description then you can have the web app generate a sketch from that. Now I'm in dark mode right here but you can probably see the dark lines that are supposed to be a sketch. This is generated from that image and then based on the sketch you can re-render with a different text description or with the same text description but apply a certain style to it. There are a lot of possibilities with models like this you can explore that in the web app. So as we've said for example we can tell the model to input text right here. So input utilization text says all that's used is this text right here. I've put far from home and if I render this which is the arrow on the right you can see a certain image is generated. If I put close to earth a different image is generated. A road with trees in fall that works out pretty well so what I can do now is I can take that and copy it over to the left side. The left side is kind of like the input area. Before we copy actually let me just take kind of a pencil and just sketch a bunch of things here. So let me sketch some... I have no... I have a touch pad. Don't criticize me. And then like a line here. And we'll do like some squiggles here. That is a beautiful sketch. So now we can activate not only text but sketch. So now we're looking for a road with trees in fall given this sketch. Well okay I have to admit my sketch wasn't exactly something that the model could make sense of. So let me try again. A few broad strokes right here, maybe one here and something harsh here. Still no. My sketching abilities might not be super good. So let me try the segmentation map. For the segmentation map you want to take a brush like this one. You want to activate the input utilization of segmentation and then here you can select a bunch of segmentation things. So dirt. Let's put some dirt here on the lower right hand corner like this. Let's also put a bunch of grass over here. And how about a fence right here. That is a fence. The fence goes here. And then house. The house is supposed to take this part right here. I'm not sure how the model is going to make this into a house. Let's just have the house be all of this. And we generate... Okay. If you have better drawing skills than me, feel free. But what is cool is that let's say we generate this image again. We can then copy that image over to the left to this input area. And then we can use different variants. For example, here we can have the segmentation map computed from that image or we can have the sketch computed from that image. So let's compute the segmentation map from that image automatically. And we can turn off the visualization of the real image. So we only have the segmentation map left. We can then use that segmentation map together with the piece of text. But now we're going to change the piece of text. How about a road with trees in spring? So what we want is a similar image but in spring. Look at that. So this is pretty cool. It would have probably be even more accurate if we used the source image as an image, which you can also do. You can use a sketch. As I said, any combination of these things. This web app is pretty cool. And it can even apply custom styles to images and so on. Now I don't want to bore you too much with this and my poor drawing skills. You go ahead and try it out. I'll link it in the description. Everyday robots is a new initiative company. I have no idea what the actual legal structure of this is. Yeah, I guess it is some sort of a company. And the goal is to make robots do everyday tasks. So instead of having robots like Boston Dynamics, where you have very specifically tailored robots, and they're often hard coded to do certain things. So for example, if a Boston Dynamics robot stands back flip, this has been the result of massive engineering effort. These robots are supposed to be a little more as they themselves say boring, yet live in the real world. So they are able to navigate around obstacles interact with real things. The challenges here are massive, like how do you generalize to arbitrary settings and environments and things are dynamic and a lot of things are happening. So this is born out of Google X, which is one of their sort of incubators. And if I understand correctly, these robots are already used in some of their internal cafes here you see one cleaning off the tables. Now even with something as simple as cleaning off the tables, you have to get to the table you have to see if the table is empty, you have to be able to move around the table and wash it down correctly until everything is washed and so on. Definitely not an easy task. There's a big website with a lot of scroll jacking animations, as you can see here, but it seems like a pretty exciting initiative. There's also a good article on wired about it with a lengthy description of what the goal here is and what the capabilities of these robots are right now and where this company wants to go. One specialty seems to be that these robots learn relatively quickly, for example, teaching them to open a door apparently took under 10 hours. Now that seems like a lot, but in real life reinforcement learning with actual robots that need to do this safely and cannot simulate and so on. This is actually a very, very short time. And once the robots have acquired this knowledge, they can transmit it to all the other robots. So only one of them technically has to learn it. The company imagines that in the future, these robots will assist humans with tasks as you can see here menial labor tasks such as cleaning off tables. And of course, since they are robots, the advantages that they can, for example, go into hazardous environments in general operate differently than humans. They also say that in the future, it might be supernatural to interact with robots like these, even if it may seem a little bit dystopian or futuristic right now. Google AI presents MetNet 2, which is another weather forecasting model. So we've already seen DeepMind going into now casting, which means predicting rain a few minutes up to like two hours from now. And MetNet 1 has done previously work predicting a few hours ahead, like six hours or so if I understand correctly, but now they've pushed this to 12 hours. So the different categories of rain forecasting actually bring a lot different challenges to them. For example, to predict the weather for the next 14 days, you look at entirely different things. You look at like big patterns and you can make some sort of large scale forecasts, you know, in the north, it's going to rain in the south, it's not going to rain. However, that information is almost completely useless for something like now casting where you want extremely local predictions that are very, very accurate in time. And in this regime, where MetNet 2 is in the 12 hour region, you sort of have to fuse both of them together. You have to look at very, very large areas. So for example, here, the blue area, if I understand correctly, is the area that they actually look at to make a prediction for the red area. Now, this is a giant area, but they still make predictions on a super fine grained resolution. I think the resolution here is a resolution of two kilometers. So every two kilometers, they make a prediction 12 hours from now, will it rain or won't it rain? So the thing about this from MetNet 1, which could only predict up to like six hours is that in order to predict for a longer horizon, they have to take more context into account, as you can see right here. And surprisingly, one way to do it is to actually replace the attention layers of MetNet 1 with convolutional layers, which are more computationally efficient. However, since convolutional layers only care about their local neighborhoods, they actually use dilated convolutions to dramatically increase the size of the receptive fields of convolutions over just a few layers. On their blog, you can see a few examples and comparisons of their method to other methods. And they even have an investigation into what the model actually learns about whether using interpretability tools. All of this is really cool, because weather prediction used to be done with very, very compute intensive physics simulation, which took apparently about one hour in order to make this same prediction that MetNet 2 makes in under one second. So I invite you to go check out the blog post if you want to learn more. A cool project by Nathaniel Felicki on hackster.io is this tiny ML dog bark stopper. So this is a report on how to use things like arduinos and speakers in order to detect when a dog barks and when the dog barks to play an appropriate sound. So apparently, this dog has a bit of separation anxiety. So whenever the owner leaves the house, the dog just kind of goes wild. And this video is a description on how they've used a speaker that is coupled to an Arduino that records sounds that the dog makes, classifies the dog sound into barking or not barking. This is done converting the sound into spectrograms and then classifying those spectrograms. And then when a bark is detected, the speaker will play a pre recorded sound of the owner such that the dog thinks that the owner is still there. So I very much invite you to go check it out. If you want to build something like this for yourself, I'm sure this is a very good basis in order to do so. The instructions are all there. And if you're into the mixture of ML and actual real world hardware, a little bit into soldering and hacking, this might be for you. Speaking of hardware and interacting with machine learning, this is an ambitious project where the YouTube user stack smashing has used a video capture card combined with again, I think an Arduino or a Raspberry Pi in order to get a ML model to drive Mario Kart. Usually this is done in an emulator. People have done this before, learn to drive Mario Kart using machine learning. However, this user does it on an actual console, which means that they read out the picture that the console generates using a capture card. They feed that image into a neural network and then they use this Raspberry Pi in order to send the commands back to the console. Now the system doesn't go as far as actually move a joystick on a controller, but they do send the appropriate controller inputs to the console using sort of like a cutoff cable and then sending the inputs to the cable. The project details how they've adapted the tensor cart project that is meant for an emulator and brought it to essentially the real world Mario Kart with the console. The machine learning part of the project isn't very complicated. The user has done a bunch of manual runs, recorded their controller inputs and then let the model learn from those controller inputs. A few challenges that arise there is that usually humans steer very abruptly and this user has purposefully as you can see here, tried to only steer super duper smoothly such that the model has a better target distribution to learn. That is not as noisy. At the end, the model is able to learn the track that it has been trained on. And interestingly, it also can drive a little bit on tracks that it hasn't been trained on, though not all of the tracks. So if you think this is cool and you want to learn more, go over to Stack Smashing's YouTube channel and check out the video. I'll link it in the description. NBC New York writes, New York City aims to be the first to reign in artificial intelligence hiring tools. This is about new legislation in New York City that would ban employers from using automated hiring tools unless a yearly bias audit can show they won't discriminate based on applicants, race or gender. And compare this to another rule that the city has enacted that restaurants have to display a calorie count with their menus. And the article here goes into the detail of what the advantages and disadvantages are and that some people think that it doesn't go nearly far enough. Now the whole crux of the matter here, of course, is that what does this yearly bias audit contain? What does it mean that you won't discriminate based on an applicant's race or gender? We can interpret this very strictly where if the model doesn't have access to the applicant's race or gender, it cannot possibly discriminate based on that. Yes, the argument usually goes that there are correlates to race or gender and models very often make decisions based on those correlates. However, what's the definition of based on? On the very other end of the spectrum, you can essentially say that any system that has any disparate outcome whatsoever with respect to hiring fails this yearly bias audit. It's interesting that with such a simple piece of legislation, you can get into very deep discussions about nature versus nurture, what is fixed about people, what isn't, how are decisions made even in humans and what does it mean to make a decision based on something. I mean, there are a lot of interesting questions to be had right here. And I'm pretty sure none of the people who actually passed the ruling have ever dived into it. It just sounds good. Oh, yes, let's make a rule. AI systems cannot discriminate based on race and gender. That sounds good. Think of the children. The article also says that a good outcome of this is a part of the legislation that says that the company has to disclose if it uses automatic systems to screen you. I'm not sure what you're going to do with that as an applicant. At the end of the day, I guess the question is, you know, of course, we all feel the kind of disgust being evaluated by an AI system and then being rejected for some arbitrary algorithmic rule, but I'm not sure like we seem to all pretend that HR personnel is a lot different. It's not like an HR person that has a stack of a thousand resumes for like three positions is going through each of them deeply delving into the applications and really grappling with every person individually. No, they're going to look at it. School, I don't know. Gone. Bad grades. Gone. Gap in whatever year something. Gone. I feel we're comparing AI tools to unreachable master standards, whereas I think what we should be doing is comparing them to what's already there and what's already there most often isn't working either. Now the people that criticize this, they say that is not going far enough, say that essentially the bill was watered down so that it effectively just asks employers to meet existing requirements under US civil rights law, prohibiting hiring practices that have a disparate impact based on race, ethnicity or gender. Oh no, how terrible. You're only asked to comply with the law. I mean, that is a shame. Clearly this isn't far enough. If you're interested, check out this article and tell me what you think about these questions. Justdrinks.com analysis, which beverage companies are leading the way in artificial intelligence? Yes, that is what I needed in my Pepsi, just a bit more AI in that can. Like, oh wow, the drink is now also a recommender system. Yes, please. Apparently after putting your coffee through the portafilter, Starbucks now also forward propagates it through a convolutional neural network before serving it to you. Or maybe they use RL to finally get customers names right. Who knows? But it lets me sleep well at night to know that the beverage companies, they're really on this AI stuff because it really like that is going to make the difference here. DeepMind, Google Brain and the chess champion Vladimir Krumnik have published a papers called the Acquisition of Chess Knowledge in AlphaZero. They investigate AlphaZero. I've previously made a video on AlphaZero about what AlphaZero learns about chess and it's quite interesting. So the paper is fairly lengthy and investigates not only how AlphaZero thinks, but also what are the overlaps with how humans play chess. How are human concepts that, you know, that grandmasters pay attention to when they play chess? How are they represented in the AlphaZero system? And are they represented at all? So they do a lot of different analyses, which is really interesting. And they also have an accompanying website where you can investigate a little bit into that stuff. For example, they have different non-negative matrix factorizations of the different board positions. Non-negative matrix factorization is an excellent tool where you can see how different components additively combine to form certain structures. They also let you select given board positions and then track how the different systems react to that board position and what continuations there are. And you're able to compare AlphaZero during training right here with humans over the years since 1985-ish. So the assumption here is that humans have gotten better over time. And maybe we can compare a little bit new strategies that were discovered by humans with new strategies that AlphaZero discovers as it becomes better using self-play. Now I've investigated this a little bit, and honestly, I haven't found really a big overlap here, but I'm also not super good at chess. So don't take my word for it. Alright, some helpful things for this week. There is a Roudali, which we previously reported about, it's a Russian version of Dali, that is trained on emojis. Now you might think that is ridiculous, to which I would respond to with a crying face emoji. However, the results are actually pretty cool. Like look at this for St. Basil's Cathedral. Looks pretty neat. There's Donald Trump from Lego. A human eats an apple. I mean, given that people already use emojis a lot when texting, you can totally imagine a future where you cannot just select from the emojis that are given to you, but that sort of emojis would be created on the fly. And maybe you could choose from 10 emojis that are conditioned on the sentence you just wrote, and then you can select among those. Seems pretty neat, honestly. I know it doesn't solve world hunger, but could be useful. RunCodeBlocks is a project that is similar to Jupyter Notebooks, except that you're able to connect cells, not linearly, but as a graph. So if this data format flourishes, it's no longer necessary to tell people, well, first you got to run cell one and then cell two and only run cell three. If you want this run cell four twice and so on. This format abstracts all of this into a DAG, if I understand this correctly, and you can then run these cells individually, or you can run like one strand of these cells. Seems pretty cool. The project is quite young. So if you want to get into this, you have to be ready for kind of like alpha version software, but it might be a very, very cool project to contribute if you're into tooling. TensorFlow has a new library for graph neural networks. Now TensorFlow has made a bunch of attempts previously at graph neural networks and related things. Things like TensorFlow fold and stuff like that. But this now seems to be a pretty sophisticated library for doing graph neural networks. So you're able to define various architectures and then run your message propagation algorithms in a way where you can also back propagate through it. The examples show how to build easy graph neural networks given predefined functions on edges and nodes and also how to build graph neural networks that have custom functions for that. So pretty cool. The GitHub repo, if you're into graph neural networks and you're using TensorFlow, this might be a very good library for you. Keep in mind that this is also an alpha release, but should get better in the future. PyDreamer is a torch implementation of the dreamer v2 reinforcement learning algorithm. The original dreamer v2 is implemented in TensorFlow. And this is essentially a port to PyTorch. Now the features differ somewhat and the implementations differ somewhat. So the results aren't exactly the same, but it could be a cool baseline if you want to experiment with dreamer like reinforcement learning algorithms. You can see right here, sometimes it does better, sometimes it does worse than the original dreamer implementation. But I guess that's just reinforcement learning. So if you're interested, the project has quite an extensive readme to get you started. Have fun. CodeGenX is a model that takes in code and spits out what more code you should write. Pretty simple. It's a little bit like GitHub Copilot. However, the difference is that it is open source. There's GitHub repo, it's based on GPTJ and there is a VS code extension, you can get a free API key and start using it right away. The website is a bit bare bones right now, but looks pretty cool. Other than Copilot, it currently supports just Python, though they say they are planning to add additional languages in future releases. So very cool project. Go check it out. And here from DevPost, this is another submission from the PyTorch annual hackathon. This is the Heyo camera. Now it currently only exists for Mac, but this is a camera plugin that recognizes hand gestures and then displays appropriate reactions. So this person is happy, this person is not happy, this person raises their hand. Very excellent. This seems a bit gimmicky, but the sort of recognition of gestures, of course, cannot only be used to display simple emojis, but can be used to trigger various other things. So again, there is a GitHub page, you can download and install it for Mac if you want, or you can continue developing it. And our last story for today, IDW online writes the Einstein Foundation to present the inaugural 500,000 euro award for promoting quality in research. And the award in part goes to the founder of archive. So the individual award worth 200,000 euros goes to Paul Ginspark, professor of physics and information science at Cornell. In 1991, he created the archive, a document server for preprints on which scientific findings are published without review and paywall restriction. Archive has become by far one of the most valuable tools, especially to the machine learning community. And it's pretty cool to see its creator recognized for putting this out there as early as 1991. That is crazy. Excellent work. Thank you. All right, this was already it for ML news this week. I hope you had fun. Did you catch the gorilla?
 GPT-3 is now free to access, Nvidia releases Gaogain 2 and it's amazing, and out of Google
 X comes Everyday Robots which aims to make robots handle everyday tasks.
 Welcome to ML News.
 Hey YouTube!
 Hey attention saws, what's up?
 This video is sponsored by Weights and Biases.
 Thank you so much to Weights and Biases for being a great sponsor.
 If you don't know Weights and Biases you should definitely check it out.
 It is a one stop shop for all your machine learning needs.
 It starts at tracking your experiments with a single line of code.
 Everything is logged to the cloud, your environment is logged, your outputs are logged, your models
 and datasets can be saved and iterated upon.
 And it's with you from conception of your idea all the way to deployment and monitoring.
 They have on-prem solutions, they have cloud solutions and it's completely free for personal
 use and for academic use.
 So please try out Weights and Biases.
 Today I want to highlight their jobs offerings.
 If you're looking for a job please consider Weights and Biases.
 As you can see right here they have all kinds of job openings from business operations,
 customer success.
 There is lots of engineering jobs.
 There's deep learning engineers, site reliability engineer, just regular software engineer,
 product engineer, infrastructure.
 There's deep learning engineer for growth.
 But even if you're not an engineer you can go into marketing, into people operations,
 product managers, all kinds of things and look at that they just need sales people.
 So if you're good at selling maybe this is your position.
 As you can see they have some jobs in North America, some are in Europe but a lot of jobs
 are actually remote.
 So whether you enjoy remote work or on-site work chances are Weights and Biases has something
 for you.
 As you know as we've reported right here Weights and Biases has just raised a giant amount
 of money at a 1 billion dollar valuation.
 Make sure you get a slice of that pie.
 Apply for a job today.
 Go to 1db.com, go to resources, click on careers and find all their job offerings right now.
 If you're not looking for a job check out their product.
 I'm sure you're gonna love it and thank you so much again to Weights and Biases for sponsoring
 this video.
 Alright let's get into it.
 OpenAI's blog says the OpenAI API is now available with no waitlist.
 That means that you can simply go, sign up and you get access to the API.
 The API includes things such as their language model, GPT-3 and so on.
 It includes things like the instruct models and these models are good at following things
 like instructions and also the codex models that generate code given a piece of natural
 language.
 A function to fill my bank account.
 Well I guess the model tells me that I actually need to make a deposit in order to fill my
 bank account.
 That's sad.
 Of course the flagship models are still the GPT models, specifically GPT-3, the largest
 version is called DaVinci.
 The best idea ever is?
 The best idea ever is the idea that is most useful to the most people.
 Thank you.
 DaVinci is a utilitarian, absolutely based.
 So even if you've used GPT-3 before and if that was a while back you might want to check
 it out again because the documentation has involved there are a lot of examples.
 OpenAI themselves have figured out a lot more about how to prompt these models in order
 to get good completions in order to actually make them do what you want them to do and
 there's a lot of useful stuff right here.
 I've actually made a poll about this in the past and over 1000 of you have responded and
 it turned out most of you didn't have access yet even though a large portion of you applied
 early.
 So to all of you who still don't have access this should help you.
 Now this doesn't come as a surprise as in recent times we've seen a lot of competitors
 to OpenAI simply giving people access to their API and not having them on a long wait list.
 So how much of this is well we finally figured it out and how much of it is please don't
 go to our competition we don't know.
 That being said OpenAI still wants to have a very tight control over people that actually
 use the API to build products.
 They say our work also allows us to review applications before they go live, monitor
 for misuse, support developers as their product scales and better understand the effects of
 this technology.
 Essentially they want to avoid at all costs that you build a product that in any way reflects
 negatively on OpenAI be that if the model makes some sort of a mistake or if the technology
 is used for a use case that maybe isn't super PR friendly.
 That is not good or bad it's just something you have to keep in mind when you go all in
 and build actually an application on the basis of an API like this.
 OpenAI releases the second iteration of their Gaogan model which is a generative adversarial
 network that doesn't just come up with stuff by itself but can be conditioned on certain
 inputs.
 Gaogan one was already being used to condition the model on sketches as you see here you
 can give a bunch of segmentation maps and then the model would dynamically adapt and
 generate a picture based on that.
 Gaogan two takes this a step further.
 Now you can also condition on words for example.
 In fact they have released a little web app and as you can see you can condition on a
 segmentation map that's what we saw in Gaogan one.
 You can condition on a sketch you can condition on a base image or on text and not only either
 or of these modalities but you can mix them all as you want.
 There is a Reddit post by the user Whiskey and some of the pictures that this user was
 able to generate with simply text prompts if I understand this correctly are just stunning
 by themselves.
 So here is a winter mountain landscape near sunset.
 Now interesting is what you can do this is a stream given a text description then you
 can have the web app generate a sketch from that.
 Now I'm in dark mode right here but you can probably see the dark lines that are supposed
 to be a sketch.
 This is generated from that image and then based on the sketch you can re-render with
 a different text description or with the same text description but apply a certain style
 to it.
 There are a lot of possibilities with models like this you can explore that in the web
 app.
 So as we've said for example we can tell the model to input text right here.
 So input utilization text says all that's used is this text right here.
 I've put far from home and if I render this which is the arrow on the right you can see
 a certain image is generated.
 If I put close to earth a different image is generated.
 A road with trees in fall that works out pretty well so what I can do now is I can take that
 and copy it over to the left side.
 The left side is kind of like the input area.
 Before we copy actually let me just take kind of a pencil and just sketch a bunch of things
 here.
 So let me sketch some...
 I have no...
 I have a touch pad.
 Don't criticize me.
 And then like a line here.
 And we'll do like some squiggles here.
 That is a beautiful sketch.
 So now we can activate not only text but sketch.
 So now we're looking for a road with trees in fall given this sketch.
 Well okay I have to admit my sketch wasn't exactly something that the model could make
 sense of.
 So let me try again.
 A few broad strokes right here, maybe one here and something harsh here.
 Still no.
 My sketching abilities might not be super good.
 So let me try the segmentation map.
 For the segmentation map you want to take a brush like this one.
 You want to activate the input utilization of segmentation and then here you can select
 a bunch of segmentation things.
 So dirt.
 Let's put some dirt here on the lower right hand corner like this.
 Let's also put a bunch of grass over here.
 And how about a fence right here.
 That is a fence.
 The fence goes here.
 And then house.
 The house is supposed to take this part right here.
 I'm not sure how the model is going to make this into a house.
 Let's just have the house be all of this.
 And we generate...
 Okay.
 If you have better drawing skills than me, feel free.
 But what is cool is that let's say we generate this image again.
 We can then copy that image over to the left to this input area.
 And then we can use different variants.
 For example, here we can have the segmentation map computed from that image or we can have
 the sketch computed from that image.
 So let's compute the segmentation map from that image automatically.
 And we can turn off the visualization of the real image.
 So we only have the segmentation map left.
 We can then use that segmentation map together with the piece of text.
 But now we're going to change the piece of text.
 How about a road with trees in spring?
 So what we want is a similar image but in spring.
 Look at that.
 So this is pretty cool.
 It would have probably be even more accurate if we used the source image as an image, which
 you can also do.
 You can use a sketch.
 As I said, any combination of these things.
 This web app is pretty cool.
 And it can even apply custom styles to images and so on.
 Now I don't want to bore you too much with this and my poor drawing skills.
 You go ahead and try it out.
 I'll link it in the description.
 Everyday robots is a new initiative company.
 I have no idea what the actual legal structure of this is.
 Yeah, I guess it is some sort of a company.
 And the goal is to make robots do everyday tasks.
 So instead of having robots like Boston Dynamics, where you have very specifically tailored
 robots, and they're often hard coded to do certain things.
 So for example, if a Boston Dynamics robot stands back flip, this has been the result
 of massive engineering effort.
 These robots are supposed to be a little more as they themselves say boring, yet live in
 the real world.
 So they are able to navigate around obstacles interact with real things.
 The challenges here are massive, like how do you generalize to arbitrary settings and
 environments and things are dynamic and a lot of things are happening.
 So this is born out of Google X, which is one of their sort of incubators.
 And if I understand correctly, these robots are already used in some of their internal
 cafes here you see one cleaning off the tables.
 Now even with something as simple as cleaning off the tables, you have to get to the table
 you have to see if the table is empty, you have to be able to move around the table and
 wash it down correctly until everything is washed and so on.
 Definitely not an easy task.
 There's a big website with a lot of scroll jacking animations, as you can see here, but
 it seems like a pretty exciting initiative.
 There's also a good article on wired about it with a lengthy description of what the
 goal here is and what the capabilities of these robots are right now and where this
 company wants to go.
 One specialty seems to be that these robots learn relatively quickly, for example, teaching
 them to open a door apparently took under 10 hours.
 Now that seems like a lot, but in real life reinforcement learning with actual robots
 that need to do this safely and cannot simulate and so on.
 This is actually a very, very short time.
 And once the robots have acquired this knowledge, they can transmit it to all the other robots.
 So only one of them technically has to learn it.
 The company imagines that in the future, these robots will assist humans with tasks as you
 can see here menial labor tasks such as cleaning off tables.
 And of course, since they are robots, the advantages that they can, for example, go
 into hazardous environments in general operate differently than humans.
 They also say that in the future, it might be supernatural to interact with robots like
 these, even if it may seem a little bit dystopian or futuristic right now.
 Google AI presents MetNet 2, which is another weather forecasting model.
 So we've already seen DeepMind going into now casting, which means predicting rain a
 few minutes up to like two hours from now.
 And MetNet 1 has done previously work predicting a few hours ahead, like six hours or so if
 I understand correctly, but now they've pushed this to 12 hours.
 So the different categories of rain forecasting actually bring a lot different challenges
 to them.
 For example, to predict the weather for the next 14 days, you look at entirely different
 things.
 You look at like big patterns and you can make some sort of large scale forecasts, you
 know, in the north, it's going to rain in the south, it's not going to rain.
 However, that information is almost completely useless for something like now casting where
 you want extremely local predictions that are very, very accurate in time.
 And in this regime, where MetNet 2 is in the 12 hour region, you sort of have to fuse both
 of them together.
 You have to look at very, very large areas.
 So for example, here, the blue area, if I understand correctly, is the area that they
 actually look at to make a prediction for the red area.
 Now, this is a giant area, but they still make predictions on a super fine grained resolution.
 I think the resolution here is a resolution of two kilometers.
 So every two kilometers, they make a prediction 12 hours from now, will it rain or won't it
 rain?
 So the thing about this from MetNet 1, which could only predict up to like six hours is
 that in order to predict for a longer horizon, they have to take more context into account,
 as you can see right here.
 And surprisingly, one way to do it is to actually replace the attention layers of MetNet 1 with
 convolutional layers, which are more computationally efficient.
 However, since convolutional layers only care about their local neighborhoods, they actually
 use dilated convolutions to dramatically increase the size of the receptive fields of convolutions
 over just a few layers.
 On their blog, you can see a few examples and comparisons of their method to other methods.
 And they even have an investigation into what the model actually learns about whether using
 interpretability tools.
 All of this is really cool, because weather prediction used to be done with very, very
 compute intensive physics simulation, which took apparently about one hour in order to
 make this same prediction that MetNet 2 makes in under one second.
 So I invite you to go check out the blog post if you want to learn more.
 A cool project by Nathaniel Felicki on hackster.io is this tiny ML dog bark stopper.
 So this is a report on how to use things like arduinos and speakers in order to detect when
 a dog barks and when the dog barks to play an appropriate sound.
 So apparently, this dog has a bit of separation anxiety.
 So whenever the owner leaves the house, the dog just kind of goes wild.
 And this video is a description on how they've used a speaker that is coupled to an Arduino
 that records sounds that the dog makes, classifies the dog sound into barking or not barking.
 This is done converting the sound into spectrograms and then classifying those spectrograms.
 And then when a bark is detected, the speaker will play a pre recorded sound of the owner
 such that the dog thinks that the owner is still there.
 So I very much invite you to go check it out.
 If you want to build something like this for yourself, I'm sure this is a very good basis
 in order to do so.
 The instructions are all there.
 And if you're into the mixture of ML and actual real world hardware, a little bit into soldering
 and hacking, this might be for you.
 Speaking of hardware and interacting with machine learning, this is an ambitious project
 where the YouTube user stack smashing has used a video capture card combined with again,
 I think an Arduino or a Raspberry Pi in order to get a ML model to drive Mario Kart.
 Usually this is done in an emulator.
 People have done this before, learn to drive Mario Kart using machine learning.
 However, this user does it on an actual console, which means that they read out the picture
 that the console generates using a capture card.
 They feed that image into a neural network and then they use this Raspberry Pi in order
 to send the commands back to the console.
 Now the system doesn't go as far as actually move a joystick on a controller, but they
 do send the appropriate controller inputs to the console using sort of like a cutoff
 cable and then sending the inputs to the cable.
 The project details how they've adapted the tensor cart project that is meant for an emulator
 and brought it to essentially the real world Mario Kart with the console.
 The machine learning part of the project isn't very complicated.
 The user has done a bunch of manual runs, recorded their controller inputs and then
 let the model learn from those controller inputs.
 A few challenges that arise there is that usually humans steer very abruptly and this
 user has purposefully as you can see here, tried to only steer super duper smoothly such
 that the model has a better target distribution to learn.
 That is not as noisy.
 At the end, the model is able to learn the track that it has been trained on.
 And interestingly, it also can drive a little bit on tracks that it hasn't been trained
 on, though not all of the tracks.
 So if you think this is cool and you want to learn more, go over to Stack Smashing's
 YouTube channel and check out the video.
 I'll link it in the description.
 NBC New York writes, New York City aims to be the first to reign in artificial intelligence
 hiring tools.
 This is about new legislation in New York City that would ban employers from using automated
 hiring tools unless a yearly bias audit can show they won't discriminate based on applicants,
 race or gender.
 And compare this to another rule that the city has enacted that restaurants have to
 display a calorie count with their menus.
 And the article here goes into the detail of what the advantages and disadvantages are
 and that some people think that it doesn't go nearly far enough.
 Now the whole crux of the matter here, of course, is that what does this yearly bias
 audit contain?
 What does it mean that you won't discriminate based on an applicant's race or gender?
 We can interpret this very strictly where if the model doesn't have access to the applicant's
 race or gender, it cannot possibly discriminate based on that.
 Yes, the argument usually goes that there are correlates to race or gender and models
 very often make decisions based on those correlates.
 However, what's the definition of based on?
 On the very other end of the spectrum, you can essentially say that any system that has
 any disparate outcome whatsoever with respect to hiring fails this yearly bias audit.
 It's interesting that with such a simple piece of legislation, you can get into very deep
 discussions about nature versus nurture, what is fixed about people, what isn't, how are
 decisions made even in humans and what does it mean to make a decision based on something.
 I mean, there are a lot of interesting questions to be had right here.
 And I'm pretty sure none of the people who actually passed the ruling have ever dived
 into it.
 It just sounds good.
 Oh, yes, let's make a rule.
 AI systems cannot discriminate based on race and gender.
 That sounds good.
 Think of the children.
 The article also says that a good outcome of this is a part of the legislation that
 says that the company has to disclose if it uses automatic systems to screen you.
 I'm not sure what you're going to do with that as an applicant.
 At the end of the day, I guess the question is, you know, of course, we all feel the kind
 of disgust being evaluated by an AI system and then being rejected for some arbitrary
 algorithmic rule, but I'm not sure like we seem to all pretend that HR personnel is a
 lot different.
 It's not like an HR person that has a stack of a thousand resumes for like three positions
 is going through each of them deeply delving into the applications and really grappling
 with every person individually.
 No, they're going to look at it.
 School, I don't know.
 Gone.
 Bad grades.
 Gone.
 Gap in whatever year something.
 Gone.
 I feel we're comparing AI tools to unreachable master standards, whereas I think what we
 should be doing is comparing them to what's already there and what's already there most
 often isn't working either.
 Now the people that criticize this, they say that is not going far enough, say that essentially
 the bill was watered down so that it effectively just asks employers to meet existing requirements
 under US civil rights law, prohibiting hiring practices that have a disparate impact based
 on race, ethnicity or gender.
 Oh no, how terrible.
 You're only asked to comply with the law.
 I mean, that is a shame.
 Clearly this isn't far enough.
 If you're interested, check out this article and tell me what you think about these questions.
 Justdrinks.com analysis, which beverage companies are leading the way in artificial intelligence?
 Yes, that is what I needed in my Pepsi, just a bit more AI in that can.
 Like, oh wow, the drink is now also a recommender system.
 Yes, please.
 Apparently after putting your coffee through the portafilter, Starbucks now also forward
 propagates it through a convolutional neural network before serving it to you.
 Or maybe they use RL to finally get customers names right.
 Who knows?
 But it lets me sleep well at night to know that the beverage companies, they're really
 on this AI stuff because it really like that is going to make the difference here.
 DeepMind, Google Brain and the chess champion Vladimir Krumnik have published a papers called
 the Acquisition of Chess Knowledge in AlphaZero.
 They investigate AlphaZero.
 I've previously made a video on AlphaZero about what AlphaZero learns about chess and
 it's quite interesting.
 So the paper is fairly lengthy and investigates not only how AlphaZero thinks, but also what
 are the overlaps with how humans play chess.
 How are human concepts that, you know, that grandmasters pay attention to when they play
 chess?
 How are they represented in the AlphaZero system?
 And are they represented at all?
 So they do a lot of different analyses, which is really interesting.
 And they also have an accompanying website where you can investigate a little bit into
 that stuff.
 For example, they have different non-negative matrix factorizations of the different board
 positions.
 Non-negative matrix factorization is an excellent tool where you can see how different components
 additively combine to form certain structures.
 They also let you select given board positions and then track how the different systems react
 to that board position and what continuations there are.
 And you're able to compare AlphaZero during training right here with humans over the years
 since 1985-ish.
 So the assumption here is that humans have gotten better over time.
 And maybe we can compare a little bit new strategies that were discovered by humans
 with new strategies that AlphaZero discovers as it becomes better using self-play.
 Now I've investigated this a little bit, and honestly, I haven't found really a big overlap
 here, but I'm also not super good at chess.
 So don't take my word for it.
 Alright, some helpful things for this week.
 There is a Roudali, which we previously reported about, it's a Russian version of Dali, that
 is trained on emojis.
 Now you might think that is ridiculous, to which I would respond to with a crying face
 emoji.
 However, the results are actually pretty cool.
 Like look at this for St. Basil's Cathedral.
 Looks pretty neat.
 There's Donald Trump from Lego.
 A human eats an apple.
 I mean, given that people already use emojis a lot when texting, you can totally imagine
 a future where you cannot just select from the emojis that are given to you, but that
 sort of emojis would be created on the fly.
 And maybe you could choose from 10 emojis that are conditioned on the sentence you just
 wrote, and then you can select among those.
 Seems pretty neat, honestly.
 I know it doesn't solve world hunger, but could be useful.
 RunCodeBlocks is a project that is similar to Jupyter Notebooks, except that you're able
 to connect cells, not linearly, but as a graph.
 So if this data format flourishes, it's no longer necessary to tell people, well, first
 you got to run cell one and then cell two and only run cell three.
 If you want this run cell four twice and so on.
 This format abstracts all of this into a DAG, if I understand this correctly, and you can
 then run these cells individually, or you can run like one strand of these cells.
 Seems pretty cool.
 The project is quite young.
 So if you want to get into this, you have to be ready for kind of like alpha version
 software, but it might be a very, very cool project to contribute if you're into tooling.
 TensorFlow has a new library for graph neural networks.
 Now TensorFlow has made a bunch of attempts previously at graph neural networks and related
 things.
 Things like TensorFlow fold and stuff like that.
 But this now seems to be a pretty sophisticated library for doing graph neural networks.
 So you're able to define various architectures and then run your message propagation algorithms
 in a way where you can also back propagate through it.
 The examples show how to build easy graph neural networks given predefined functions
 on edges and nodes and also how to build graph neural networks that have custom functions
 for that.
 So pretty cool.
 The GitHub repo, if you're into graph neural networks and you're using TensorFlow, this
 might be a very good library for you.
 Keep in mind that this is also an alpha release, but should get better in the future.
 PyDreamer is a torch implementation of the dreamer v2 reinforcement learning algorithm.
 The original dreamer v2 is implemented in TensorFlow.
 And this is essentially a port to PyTorch.
 Now the features differ somewhat and the implementations differ somewhat.
 So the results aren't exactly the same, but it could be a cool baseline if you want to
 experiment with dreamer like reinforcement learning algorithms.
 You can see right here, sometimes it does better, sometimes it does worse than the original
 dreamer implementation.
 But I guess that's just reinforcement learning.
 So if you're interested, the project has quite an extensive readme to get you started.
 Have fun.
 CodeGenX is a model that takes in code and spits out what more code you should write.
 Pretty simple.
 It's a little bit like GitHub Copilot.
 However, the difference is that it is open source.
 There's GitHub repo, it's based on GPTJ and there is a VS code extension, you can get
 a free API key and start using it right away.
 The website is a bit bare bones right now, but looks pretty cool.
 Other than Copilot, it currently supports just Python, though they say they are planning
 to add additional languages in future releases.
 So very cool project.
 Go check it out.
 And here from DevPost, this is another submission from the PyTorch annual hackathon.
 This is the Heyo camera.
 Now it currently only exists for Mac, but this is a camera plugin that recognizes hand
 gestures and then displays appropriate reactions.
 So this person is happy, this person is not happy, this person raises their hand.
 Very excellent.
 This seems a bit gimmicky, but the sort of recognition of gestures, of course, cannot
 only be used to display simple emojis, but can be used to trigger various other things.
 So again, there is a GitHub page, you can download and install it for Mac if you want,
 or you can continue developing it.
 And our last story for today, IDW online writes the Einstein Foundation to present the inaugural
 500,000 euro award for promoting quality in research.
 And the award in part goes to the founder of archive.
 So the individual award worth 200,000 euros goes to Paul Ginspark, professor of physics
 and information science at Cornell.
 In 1991, he created the archive, a document server for preprints on which scientific findings
 are published without review and paywall restriction.
 Archive has become by far one of the most valuable tools, especially to the machine
 learning community.
 And it's pretty cool to see its creator recognized for putting this out there as early as 1991.
 That is crazy.
 Excellent work.
 Thank you.
 All right, this was already it for ML news this week.
 I hope you had fun.
 Did you catch the gorilla?