[ML News] DeepMind AlphaCode | OpenAI math prover | Meta battles harmful content with AI
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper ml news machine learning news tech news artificial general intelligence ai news best ai meta ai harmful content ai moderator ai mod ai harmful openai deepmind deepmind alphacode alphacode alpha code ai math ai mathematics ai math prove ai theorem prover expert iteration langauge models ai code ai programmer ai leetcode stylegan xl
#mlnews #alphacode #openai

The latest and greatest from the world of Machine Learning!
Merch: http://store.ykilcher.com

Sponsor: Weights & Biases
https://wandb.me/yannic

OUTLINE:
0:00 - Intro
0:15 - Sponsor: Weights & Biases
3:15 - DeepMind's AlphaCode: AI competitive programmer
11:30 - OpenAI uses language models to prove math theorems
14:30 - StyleGAN XL: Scaling StyleGAN to diverse datasets
16:10 - ar5iv.org displays papers as HTML5
17:40 - Helpful Things
19:30 - ICML22 Review process changes
21:15 - Meta AI tackles harmful content classification using few-shot learning
23:55 - Company claims to produce face images from DNA


References:
https://deepmind.com/blog/article/Competitive-programming-with-AlphaCode
https://alphacode.deepmind.com/#layer=18,problem=34,heads=11111111111
https://storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf
https://twitter.com/DBahdanau/status/1489009994007674881?utm_source=pocket_mylist

https://openai.com/blog/formal-math/
https://arxiv.org/pdf/2202.01344.pdf

https://blog.eleuther.ai/announcing-20b/?utm_source=pocket_mylist

https://sites.google.com/view/stylegan-xl/
https://arxiv.org/pdf/2202.00273.pdf

https://ar5iv.org/
https://ar5iv.org/html/1910.06709

https://twitter.com/YiTayML/status/1488556619256328192?utm_source=pocket_mylist
https://ffcv.io/
https://github.com/ott-jax/ott
https://twitter.com/soumithchintala/status/1488206868573040641?utm_source=pocket_mylist
https://github.com/facebookresearch/dietgpu

https://www.reddit.com/r/MachineLearning/comments/shazv1/n_changes_in_the_icml_2022_review_process/?utm_source=pocket_mylist
https://icml.cc/Conferences/2022/ReviewForm
https://icml.cc/Conferences/2022/CallForPapers

https://ai.facebook.com/blog/harmful-content-can-evolve-quickly-our-new-ai-system-adapts-to-tackle-it/?utm_source=pocket_mylist

https://www.technologyreview.com/2022/01/31/1044576/corsight-face-recognition-from-dna/

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 DeepMind's alpha code solves programming challenges, open AI's language models solve math problems, and a Luther AI releases a 20 billion parameter language model open source. Welcome to ML news. Before the rest of the video, this video is sponsored by weights and biases, weights and biases builds developer tools for machine learning for researchers for practitioners for juniors for seniors, whatever your favorite flavor of yogurt is, they don't care, they build products for you, except cherry, who likes cherry. Today, I want to talk to you about a feature called artifacts. So artifacts essentially are files in the cloud, but you're probably going to use them mostly for two things, data and models. Both of these things are notoriously tricky to work with data set is too large to check into get that we need to keep it up to date, we may have different versions of it and models even more, we want to save the outputs of our runs into models that we can then use later, maybe introspect. And these things are also versioned, and we want to depend on them. So when I did this, I had to save the model to some special folder, and then I had to go grab it from that folder, put it on all the machines in a correct folder, and then reference that folder from all my scripts that would then consume this model with artifacts, this gets a lot easier. So we first uploaded the original data set to an artifact. Now we're going to consume that artifact, split the data into train validation and test data, and then emit those things as artifacts. So if there is a new version of the raw data available, I can simply run the same script depending on the same thing, and it will create new versions of the train validation and test data, you can make this arbitrarily complex, but I hope you can see the point here. The same goes for models, if your run outputs and saves some kind of a model, you can log that as an artifact. And from then on, you can consume that model in all subsequent runs. Here's one of my models, it's a CNN, you can see it's already version 116 of that model. But you can see all I have to do to use this model in any code in any script in the future, I simply call the download method on the artifact and it will be available locally. And as I told you, you can do this with any file. But since this is a model of a deep learning framework, weights and biases understands it and gives me a neat viewer where I can actually introspect the model and look at the shapes and even at the weights of my CNN. So I think this is incredibly powerful. These things quickly get complicated with versions and scripts building upon other scripts. And the artifact framework really helps you to make sense of all of it. There's even the possibility that the data stays in specific private buckets with access controls. So not everyone in your team has access to all of the data. Of course, artifacts are only one of the features of weights and biases. If you're interested, please check them out. Free accounts are free. Academic accounts are free enterprise accounts cost a bit and that's it for this week's sponsor spot. Thanks a lot to weights and biases. Let's get into the video. Hello and welcome to ML news. How's everyone doing we'll jump into our first story, which is that deep mind has released alpha code, which is a model that can take a programming challenge description. You might know these descriptions if you've ever done competitive programming or had a programming exam or something like this. So we have one right here given two strings s and t both consisting of lowercase English letters. This is kind of overly formal about but it kind of details a procedure so you can press the backspace button. And as you type the string s and then the character is deleted. And the question is, can you get from the string s to the string t by pressing the back space button at appropriate times. So for example, here is the input, you have four inputs, the first string is a, b, a, b, a, that's s and ba is t. The question is, can you type s and you always have the choice of typing the button of the letter or the backspace button and you have to end up at t. So we'll try this for example, first type a backspace, right? Then there's nothing then we'll type ba and then we'll type b and then a backspace and all of that should result in ba. So we are going to have to write a program that figures out if it is possible to get to t from s and they have a bunch of these example inputs right here. They have some notes and as you can see this is a text description. This is the problem. You feed this to a neural network, the neural network will output a program, an actual piece of code, in this case Python code, that actually reads the input from the provided file. Not only these by the way, so there's going to be other test cases, not just the ones that they have as an example right here, implements the algorithm all by itself. There's no human in the loop right here and then prints out the correct answer according to the specification in the textual description of the problem. This is, let's say, quite challenging. This is a hard problem, especially given the description is in natural language and AlphaCode solves this. So they have submitted AlphaCode to programming challenge competitions and it scored at about a 50th percentile of humans. Now that is not super duper good as lots of these programming challenge competitors are maybe students, people who get into programming, who kind of want to prepare for an interview or hone their skills a little bit. So it is at an intermediate level right now, but it is definitely more than I would have expected. So the way the model works is that kind of like codecs, it is pre trained on GitHub problems, but then it is fine tuned to solve exactly these code challenge data sets. So there exists data sets given problems in natural language description and solutions and the solutions are programs obviously. So DeepMind takes their pre-trained model and then fine tunes it on these pairs of problem description and solution. Now when it comes to actually solving a problem at inference time, they take that problem description, they feed it to the network, but they don't just output whatever the most likely output of the model is, they actually sample a giant amount of possible samples, which means possible programs that the model suggests. Now, a lot of them are going to be wrong. So what they do is they filter those programs based on the small subset of provided solutions that you get in the problem descriptions. In this case, here they have four different example inputs, four different example outputs that will filter out in the paper, they say that will filter out over 99% of possible solutions very often. Now filtering alone isn't enough as that still leaves them with a large number of potential solutions. And very often these coding competitions, they're limited to a very small number of submissions. In this case, I believe it was 10 submissions. So in order to achieve that, they have a step on top of that where they cluster solutions. So they try to cluster together programs that are textually different, but essentially don't do a different thing. Like maybe the variable names are different, maybe the same algorithm is implemented in a slightly different way. So they have a clustering algorithm that lumps those together. And that brings them down to the 10 submissions that they're going to make. These are not the only parts of the system by any means, there is a large number of components to the system that really brings up the system to the level of the average human where it currently stands. Now there's a website where you can explore the solutions given by the model. And you can look at sort of the attention heads of different models, like what they pay attention to along the different types and things they do. So on the left here, you see the description of the exact problem we saw before. This is pure text with natural language. And on the right, you see the solution. So as you hover over this right here, it shows you token probabilities, and it shows you according to what this token is decided upon. So for example, when I say when I hover over the line S is the input right here, you can see that on the left, it focuses on this text right here. And the first line of each test contains the string S. When I focus on T, it focuses mostly on the line below where it describes whatever T is. The attention is not only to the problem description, but also within the program that was already generated. And it's generally pretty cool to explore. I recommend you give it a try. As I said, there is a detailed paper with this where they describe exactly what the components of the system are, and so on. Give it a read. It is quite a lengthy paper. I believe it has its own table of contents. Yes, it does about 30 pages, so not too long. So my question is a little bit when I think back at like AlphaGo, AlphaZero, and so on, those models also didn't start out world class, but they were able to quickly get there and beyond simply by doing more self play. In this case, it seems the data set is a limiting factor. So there's only a finite amount of these human generated programming competition data points. The question would be, is there a way that we could come up with synthetic data like synthetically produced code samples? And is there a way that we could make them progressively harder and harder and harder in a self play kind of style? Because if that's the case, and if we really get this data generation part right, it could also be that the coding AI here will become, you know, like good beyond limits. But I am kind of skeptical about that. We also have some different voices giving their opinions on this. One of these, for example, is Jimitri Bada now, who is a competitive programmer has done this for a while apparently, and puts it a little bit into perspective saying it is impressive. Yes, but he says human level is still light years away mentions again that 50th percentile in these competitions doesn't necessarily mean that it's particularly good that a human challenge is often not only the difficulties of the problems, but also the limited time you have available for them and the disparity between humans and the machine of the approach, namely that 99% of all programs that alpha code outputs are wrong, whereas a human will maybe make a mistake in the first try of the implementation, but doesn't need to generate 1000s and 1000s of hypotheses until they get a correct one. And certainly, they don't evaluate all of these hypotheses by filtering them using the tiny, tiny amount of examples they have. So humans and machines, they seem to have a sort of fundamentally different approach for now to solving these problems. Yet I can definitely see a version of alpha code that more iteratively takes into account sort of partial programs and more does a more guided search for the rest. And ultimately, yeah, humans also they run their program on the small test examples. And if that doesn't work out, they're like, wait, something's wrong. So this is an exciting field. I'm very curious where it goes. Next news, OpenAI releases a blog post called Solving Some Formal Math Olympiad Problems. They detail how a language model that was fine tuned is able to solve formal mathematics problems. This is very, very cool. So other than in alpha code, these problems actually come with a formal description. They are defined in a formal language that is amenable to be proven yet still to apply language modeling to this problem, and then do some post processing, obviously, is quite a hard task. So the reason they use language modeling right here is that other than in chess or anything like this, the action space is huge, it's actually infinite in proving formal mathematics, because you can just invent new things by yourself. They do have a set of tactics that the model is kind of allowed to apply, but still the action space is infinite. And the language model helps them to determine what are the most likely next steps that they want to do if they want to solve this proof. The other thing that differentiates them from games is what they call the lack of self play opportunity. There's no reward to people playing against each other or anything like this, which usually serves as sort of a curriculum method. As the agents play against each other, they sort of level each other up in skill. Now to combat that they have quite a smart data generation and sampling process, where they start off with some hand provided samples of various difficulties of where they want to go. And then they start with the lowest ones that they might be able to prove with the current technique of language model plus proof search. Note that it is not only a language model is combined actually with the proof searcher that is guided by language model. And as they prove more things in the, let's say easier statements, they add those to the data set, which they then reuse to train the language model. So in this case, the model automates its own curriculum by proving more and more statements. Now this isn't obviously without challenge because math is full of trivial and nonsensical statements that you can prove to be true. So choosing even what to prove becomes a hard task. But nevertheless, using this approach, they're able to generate quite good proofs. In fact, they're able to outperform pure proof search by quite a bit. They're also able to solve problems of the International Math Olympiad, which is usually quite a hard problem. There is a paper to go along with this, give it a read if you are interested. Aluthor AI announces GPT-Neo X20B. That is a 20 billion parameter model. And by the time you're watching this, the model is going to be available for free. It's going to be kind of a pain to run it because it's so big, but you can just download it. I've made an entire video interviewing Connor Leahy, who is one of the co-founders of Aluthor AI and has worked on this project about how this came to be, about how they got their hands on the hardware necessary and so on. So if you're interested, check that out. Another new paper about StyleGAN XL. The paper is called Scaling StyleGAN to Large Diverse Datasets. That is a hard thing to say. Scaling StyleGAN. Try saying that over and over again. Scaling StyleGAN. So the TLDR here is, with the right training strategy, StyleGAN achieves state of the art on ImageNet. So if you remember, StyleGAN always used to be trained on very specific datasets. StyleGAN is the thing that powers this person does not exist.com, this shoe does not exist.com, this sneaker does not exist.com, and so on. But these are all very limited datasets, often of the same thing. And approaches like BigGAN have traditionally been better at modeling diverse datasets, such as ImageNet, which has many different things. The authors here show that with the right training protocol, namely projected GANs, upsampling, and so on, progressive training, you can get these GANs to the level of ImageNet. This is also built on StyleGAN v3, which means that it kind of retains it has these translation invariance properties. I have reported on this on ML News previously. So go check that out if you are interested. So they're able to generate images up until 1024 to 1024 resolution, which is quite impressive. They can also invert images on the left, you actually see a real image. And on the right is an inverted image where they have fed this into the GAN, and then figured out the latent codes. And then they're able to edit the image on the right as they see fit. And as I said, it retains the translation equivalent variants from StyleGAN v3. If you're interested, check out their website and check out their paper. R5.5. It's AR5IV. That is a website, it's ar5iv.org. What it allows you to do, it allows you to view archive articles as HTML5 web pages. I'm not exactly sure how it's pronounced. I was told it's pronounced ar5. But then again, it should probably be ar5iv, like the way it's written. I don't know. Also, the browser showed me a warning when I went on this website asking me whether or not I have maybe confused it with archive. So yeah, this might be just a giant phishing attack. But it is pretty cool here is an example that they give now my browser is dark mode. So I don't know if that's available in light mode. But you can see that the references are real true links that you can open as a pop up, there are still some kind of artifacts right here, as you can see, equations are rendered nicely. And also the side note, the footnotes here are rendered right beside the text. I don't know what happens if I zoom in. Okay, they just are pop over. Also allows you to jump to equations and then using the back button, jump back to where you were. This is like this is the greatest thing ever. The amount of times I had not clicked on like an internal reference on a PDF, just because I was like, No, I'm not going to scroll back to where I was. So thank you. Check out our five. Okay, we have some helpful things this week. The first helpful thing is itai saying they've released over 170 pre trained transformer checkpoints, many different shapes and sizes as part of their paper. This is by Google research. Check out the scaling transformers paper, the scaling transformers repo, and the models released. Thank you. FFCV is a library by the lab of Alexander Madri that makes training machine learning models fast. If there's ever like a buzzwordy title that says nothing, it's train machine learning models fast. So they provide a set of sort of throw in replacements, for example, for data loaders that will just kind of speed up common use cases of training neural networks. They claim their code is hyper optimized removes bottlenecks, it's super duper pipeline and parallel and all of that. So if speed is an issue for you, maybe give this a try. OTT or optimal transport tools is a toolbox for all things. Vosserstein, as they call it, it is an optimal transport library for Jacks. Sumit Chintala advertises diet GPU, which is a lossless compression algorithm for Nvidia GPUs. The code of this is available. It's authored by Jeff Johnson. And what it does is it can compress stuff and uncompressed stuff on GPUs. So if you have a slow network, and you have a distributed training, and you really care about making this fast and efficient, what you can do is you can compress stuff that you need to send over the network on the GPUs, send it over, then uncompress it. This library will make the compression and uncompression part really fast and really efficient. All right, that was it for helpful things. I hope you got help. The user breman79 on Reddit says the ICML 2022 conference is changing their review process slightly. So now there are two phases. In phase one, the reviewers just give a recommendation. If there are two recommendations that are negative for a paper in phase one, it is already rejected. I guess this is a goal to call down on the amount of papers that have to be seriously reviewed. It's all the more important now that your paper makes a good first impression. So they say the meta reviewer can reverse this outcome. Okay. And other changes that reviewers do not make, accept or reject recommendations in phase two, the meta reviewers will decide based on the reviews. So I just write my review and then the meta reviewer reads it and integrates it all instead of me saying, well, this is a seven or this is a four, this is a strong accept or a weak accept. Now, technically it shouldn't make a difference, right? Because me voice, like my score that I would usually put is just kind of a conglomeration of what I said before. But you know, tiny changes like this, you know, because we're humans and we're not consistent and we're not, you know, we're not attentive enough, tiny changes like this might actually make a difference. I'd be interested to see some statistical analysis after the fact of what this did. If you're interested, the entire process is detailed in the ICML 2022 review form. Now it just occurred to me that the submission deadline was actually last week, which I should know. So if your paper is not pretty and doesn't make a good first impression, then you just you just gotta gotta hope for that really good meta reviewer that recognizes its inner beauty. This is a little bit older, but I hadn't seen it at the time. There is a blog post on Meta AI's research blog saying harmful content can evolve quickly. Our new AI system adapts to tackle it. So they describe a system that they call few shot learner, which essentially means that it's a system that can monitor harmful content and adapt quickly to new harmful content because it's ever evolving. I find a few things interesting right here. First on a sort of a scientific level, what is pretty interesting is that the model doesn't only consider training data. So data that has been labeled as harmful or not harmful or borderline or anything like this, it does do that. But it also takes a description of the policy, like a textual description of the current policy. And by doing that, it's able to adapt to policies over time, having some sort of a policy that says, you know, with this policy, this stuff is okay. And then with this new policy, this other stuff is okay. So the fine tuning process can potentially happen with less data. I found this pretty, pretty interesting to actually provide the policy itself to the model. The other interesting thing is just this video right here. So as you can see, the people here, they're interacting with the internet and they see harmful content and they're like, oh, like they're like, oh, no, I'm gonna log all, oh, no, all this harmful content. And then, you know, there's the system, they describe their system. Yeah, whoa, okay. So now they, you know, they filter all of this, this new harmful content. And then at the end, look what happens. Everyone's smiling, like, look, they're smiling. Oh, this is just, it is so awesome. Thank you. Thank you, Meta. Thank you. Ah, the few shot learner. Thank God all the harmful content was prevented from destroying smiles. Now, okay, on a more serious note, it is a hard problem, right? There's no way you can monitor all the content all the time. There's no way you can train a static system because sort of the meta of bad content, of bad language, of people bullying each other and so on is always evolving. So props to, you know, Meta for actually trying to tackle this problem because what, I mean, what's the alternative? Shut down all communication. That's not gonna happen. Tell people to be nice, like, well, try. But I see a bit too much complaining about this. And yeah, I do, I do like that they're actually tackling this problem. And I find the approach to be cool. It's just the marketing that's a bit cringy. But what am I saying? I'm wearing sunglasses indoors. Okay, last news for the day. MIT technology review says, this company says it's developing a system that can recognize your face from just your DNA. Now, people have been extremely skeptical of statements like these. This is a company that deals in broad language with law enforcement, searching people, security, surveillance, and so on. And you know, you might debate the merits or unmerits of that in a separate topic. But the particular question of can we actually get someone's facial features from their DNA is highly debated. Just to be said, the company isn't only focused on that. It's called core site and they have different plans. These are not systems that run right now. These are sort of future plans to do things. One of them is this DNA to face thing. Now, I do feel the criticisms of this are often maybe overly skeptical, let's say. Now, again, I don't mind the skepticism about the applications of this, but the possibility that there's a reason that children often look like their parents, your facial structure is in large part determined by your genetic material. Now, the article points out that obviously age and environmental influences also have big impacts on that. So no doubt about that. And they make a good point in that they say the technology will probably not be able to tell you the exact number of millimeters between the eyes or the ratios between the eyes, nose and mouth. And those are some of the features that the current facial recognition technologies rely upon. So since we can't get those features accurately from genetic data, because there may be more environmentally determined, the current facial recognition algorithms wouldn't work. However, I don't see the extrapolation discussed right here in that I would think it might be absolutely possible to train facial recognition algorithms that only use the features that we can read from the DNA. Like the argument that the face reconstructions that the DNA data gives us doesn't work with current facial recognition software is almost a moot point by then. Question is obviously how accurate it's going to be. And again, whether or not you even want to do this in the first place. But let me know what you think. Should this be done? Can this be done? And would you want to do it? Let me know in the comments. This was ML News. Thank you so much for being here. I'll see you next time. Bye bye.
 DeepMind's alpha code solves programming challenges, open AI's language models solve
 math problems, and a Luther AI releases a 20 billion parameter language model open source.
 Welcome to ML news. Before the rest of the video, this video is sponsored by weights and biases,
 weights and biases builds developer tools for machine learning for researchers for practitioners
 for juniors for seniors, whatever your favorite flavor of yogurt is, they don't care, they build
 products for you, except cherry, who likes cherry. Today, I want to talk to you about a feature called
 artifacts. So artifacts essentially are files in the cloud, but you're probably going to use them
 mostly for two things, data and models. Both of these things are notoriously tricky to work with
 data set is too large to check into get that we need to keep it up to date, we may have different
 versions of it and models even more, we want to save the outputs of our runs into models that we
 can then use later, maybe introspect. And these things are also versioned, and we want to depend
 on them. So when I did this, I had to save the model to some special folder, and then I had to
 go grab it from that folder, put it on all the machines in a correct folder, and then reference
 that folder from all my scripts that would then consume this model with artifacts, this gets a
 lot easier. So we first uploaded the original data set to an artifact. Now we're going to consume that
 artifact, split the data into train validation and test data, and then emit those things as
 artifacts. So if there is a new version of the raw data available, I can simply run the same script
 depending on the same thing, and it will create new versions of the train validation and test data,
 you can make this arbitrarily complex, but I hope you can see the point here. The same goes for
 models, if your run outputs and saves some kind of a model, you can log that as an artifact. And from
 then on, you can consume that model in all subsequent runs. Here's one of my models,
 it's a CNN, you can see it's already version 116 of that model. But you can see all I have to do
 to use this model in any code in any script in the future, I simply call the download method on the
 artifact and it will be available locally. And as I told you, you can do this with any file. But
 since this is a model of a deep learning framework, weights and biases understands it and gives me a
 neat viewer where I can actually introspect the model and look at the shapes and even at the weights
 of my CNN. So I think this is incredibly powerful. These things quickly get complicated with versions
 and scripts building upon other scripts. And the artifact framework really helps you to make sense
 of all of it. There's even the possibility that the data stays in specific private buckets with
 access controls. So not everyone in your team has access to all of the data. Of course, artifacts
 are only one of the features of weights and biases. If you're interested, please check them out. Free
 accounts are free. Academic accounts are free enterprise accounts cost a bit and that's it
 for this week's sponsor spot. Thanks a lot to weights and biases. Let's get into the video.
 Hello and welcome to ML news. How's everyone doing we'll jump into our first story, which is that
 deep mind has released alpha code, which is a model that can take a programming challenge
 description. You might know these descriptions if you've ever done competitive programming or had a
 programming exam or something like this. So we have one right here given two strings s and t both
 consisting of lowercase English letters. This is kind of overly formal about but it kind of details
 a procedure so you can press the backspace button. And as you type the string s and then the character
 is deleted. And the question is, can you get from the string s to the string t by pressing the back
 space button at appropriate times. So for example, here is the input, you have four inputs, the first
 string is a, b, a, b, a, that's s and ba is t. The question is, can you type s and you always have the
 choice of typing the button of the letter or the backspace button and you have to end up at t. So
 we'll try this for example, first type a backspace, right? Then there's nothing then we'll type ba and
 then we'll type b and then a backspace and all of that should result in ba. So we are going to have
 to write a program that figures out if it is possible to get to t from s and they have a bunch
 of these example inputs right here. They have some notes and as you can see this is a text description.
 This is the problem. You feed this to a neural network, the neural network will output a program,
 an actual piece of code, in this case Python code, that actually reads the input from the provided
 file. Not only these by the way, so there's going to be other test cases, not just the ones that they
 have as an example right here, implements the algorithm all by itself. There's no human in the
 loop right here and then prints out the correct answer according to the specification in the
 textual description of the problem. This is, let's say, quite challenging. This is a hard problem,
 especially given the description is in natural language and AlphaCode solves this. So they have
 submitted AlphaCode to programming challenge competitions and it scored at about a 50th
 percentile of humans. Now that is not super duper good as lots of these programming challenge
 competitors are maybe students, people who get into programming, who kind of want to prepare for an
 interview or hone their skills a little bit. So it is at an intermediate level right now,
 but it is definitely more than I would have expected. So the way the model works is that
 kind of like codecs, it is pre trained on GitHub problems, but then it is fine tuned to solve
 exactly these code challenge data sets. So there exists data sets given problems in natural language
 description and solutions and the solutions are programs obviously. So DeepMind takes their
 pre-trained model and then fine tunes it on these pairs of problem description and solution. Now when
 it comes to actually solving a problem at inference time, they take that problem description, they
 feed it to the network, but they don't just output whatever the most likely output of the model is,
 they actually sample a giant amount of possible samples, which means possible programs that the
 model suggests. Now, a lot of them are going to be wrong. So what they do is they filter those
 programs based on the small subset of provided solutions that you get in the problem descriptions.
 In this case, here they have four different example inputs, four different example outputs
 that will filter out in the paper, they say that will filter out over 99%
 of possible solutions very often. Now filtering alone isn't enough as that still leaves them
 with a large number of potential solutions. And very often these coding competitions,
 they're limited to a very small number of submissions. In this case, I believe it was
 10 submissions. So in order to achieve that, they have a step on top of that where they cluster
 solutions. So they try to cluster together programs that are textually different, but essentially
 don't do a different thing. Like maybe the variable names are different, maybe the same algorithm is
 implemented in a slightly different way. So they have a clustering algorithm that lumps those
 together. And that brings them down to the 10 submissions that they're going to make. These
 are not the only parts of the system by any means, there is a large number of components to the
 system that really brings up the system to the level of the average human where it currently
 stands. Now there's a website where you can explore the solutions given by the model. And
 you can look at sort of the attention heads of different models, like what they pay attention to
 along the different types and things they do. So on the left here, you see the description of the
 exact problem we saw before. This is pure text with natural language. And on the right, you see
 the solution. So as you hover over this right here, it shows you token probabilities, and it shows you
 according to what this token is decided upon. So for example, when I say when I hover over the line
 S is the input right here, you can see that on the left, it focuses on this text right here.
 And the first line of each test contains the string S. When I focus on T, it focuses mostly on
 the line below where it describes whatever T is. The attention is not only to the problem description,
 but also within the program that was already generated. And it's generally pretty cool to
 explore. I recommend you give it a try. As I said, there is a detailed paper with this where they
 describe exactly what the components of the system are, and so on. Give it a read. It is quite a
 lengthy paper. I believe it has its own table of contents. Yes, it does about 30 pages, so not too
 long. So my question is a little bit when I think back at like AlphaGo, AlphaZero, and so on, those
 models also didn't start out world class, but they were able to quickly get there and beyond simply
 by doing more self play. In this case, it seems the data set is a limiting factor. So there's only a
 finite amount of these human generated programming competition data points. The question would be,
 is there a way that we could come up with synthetic data like synthetically produced code samples?
 And is there a way that we could make them progressively harder and harder and harder
 in a self play kind of style? Because if that's the case, and if we really get this data generation
 part right, it could also be that the coding AI here will become, you know, like good beyond limits.
 But I am kind of skeptical about that. We also have some different voices giving their opinions
 on this. One of these, for example, is Jimitri Bada now, who is a competitive programmer has
 done this for a while apparently, and puts it a little bit into perspective saying it is impressive.
 Yes, but he says human level is still light years away mentions again that 50th percentile in these
 competitions doesn't necessarily mean that it's particularly good that a human challenge is often
 not only the difficulties of the problems, but also the limited time you have available for them
 and the disparity between humans and the machine of the approach, namely that 99% of all programs
 that alpha code outputs are wrong, whereas a human will maybe make a mistake in the first try of the
 implementation, but doesn't need to generate 1000s and 1000s of hypotheses until they get a correct
 one. And certainly, they don't evaluate all of these hypotheses by filtering them using the tiny,
 tiny amount of examples they have. So humans and machines, they seem to have a sort of fundamentally
 different approach for now to solving these problems. Yet I can definitely see a version
 of alpha code that more iteratively takes into account sort of partial programs and more does a
 more guided search for the rest. And ultimately, yeah, humans also they run their program on the
 small test examples. And if that doesn't work out, they're like, wait, something's wrong. So
 this is an exciting field. I'm very curious where it goes.
 Next news, OpenAI releases a blog post called Solving Some Formal Math Olympiad Problems.
 They detail how a language model that was fine tuned is able to solve formal mathematics problems.
 This is very, very cool. So other than in alpha code, these problems actually come with a formal
 description. They are defined in a formal language that is amenable to be proven yet still to apply
 language modeling to this problem, and then do some post processing, obviously, is quite a hard
 task. So the reason they use language modeling right here is that other than in chess or anything
 like this, the action space is huge, it's actually infinite in proving formal mathematics, because
 you can just invent new things by yourself. They do have a set of tactics that the model is kind
 of allowed to apply, but still the action space is infinite. And the language model helps them to
 determine what are the most likely next steps that they want to do if they want to solve this proof.
 The other thing that differentiates them from games is what they call the lack of self play
 opportunity. There's no reward to people playing against each other or anything like this, which
 usually serves as sort of a curriculum method. As the agents play against each other, they sort of
 level each other up in skill. Now to combat that they have quite a smart data generation and
 sampling process, where they start off with some hand provided samples of various difficulties
 of where they want to go. And then they start with the lowest ones that they might be able to prove
 with the current technique of language model plus proof search. Note that it is not only a language
 model is combined actually with the proof searcher that is guided by language model. And as they prove
 more things in the, let's say easier statements, they add those to the data set, which they then
 reuse to train the language model. So in this case, the model automates its own curriculum by
 proving more and more statements. Now this isn't obviously without challenge because math is full
 of trivial and nonsensical statements that you can prove to be true. So choosing even what to prove
 becomes a hard task. But nevertheless, using this approach, they're able to generate quite good
 proofs. In fact, they're able to outperform pure proof search by quite a bit. They're also able to
 solve problems of the International Math Olympiad, which is usually quite a hard problem. There is a
 paper to go along with this, give it a read if you are interested.
 Aluthor AI announces GPT-Neo X20B. That is a 20 billion parameter model. And by the time you're
 watching this, the model is going to be available for free. It's going to be kind of a pain to run
 it because it's so big, but you can just download it. I've made an entire video interviewing Connor
 Leahy, who is one of the co-founders of Aluthor AI and has worked on this project about how this
 came to be, about how they got their hands on the hardware necessary and so on. So if you're
 interested, check that out. Another new paper about StyleGAN XL. The paper is called Scaling
 StyleGAN to Large Diverse Datasets. That is a hard thing to say. Scaling StyleGAN. Try saying that
 over and over again. Scaling StyleGAN. So the TLDR here is, with the right training strategy,
 StyleGAN achieves state of the art on ImageNet. So if you remember, StyleGAN always used to be
 trained on very specific datasets. StyleGAN is the thing that powers this person does not exist.com,
 this shoe does not exist.com, this sneaker does not exist.com, and so on. But these are all very
 limited datasets, often of the same thing. And approaches like BigGAN have traditionally been
 better at modeling diverse datasets, such as ImageNet, which has many different things. The
 authors here show that with the right training protocol, namely projected GANs, upsampling,
 and so on, progressive training, you can get these GANs to the level of ImageNet. This is also built
 on StyleGAN v3, which means that it kind of retains it has these translation invariance properties. I
 have reported on this on ML News previously. So go check that out if you are interested. So they're
 able to generate images up until 1024 to 1024 resolution, which is quite impressive. They can
 also invert images on the left, you actually see a real image. And on the right is an inverted image
 where they have fed this into the GAN, and then figured out the latent codes. And then they're
 able to edit the image on the right as they see fit. And as I said, it retains the translation
 equivalent variants from StyleGAN v3. If you're interested, check out their website and check out their paper.
 R5.5. It's AR5IV. That is a website, it's ar5iv.org. What it allows you to do, it allows you to view
 archive articles as HTML5 web pages. I'm not exactly sure how it's pronounced. I was told it's pronounced
 ar5. But then again, it should probably be ar5iv, like the way it's written. I don't know. Also,
 the browser showed me a warning when I went on this website asking me whether or not I have maybe
 confused it with archive. So yeah, this might be just a giant phishing attack. But it is pretty
 cool here is an example that they give now my browser is dark mode. So I don't know if that's
 available in light mode. But you can see that the references are real true links that you can open
 as a pop up, there are still some kind of artifacts right here, as you can see, equations are rendered
 nicely. And also the side note, the footnotes here are rendered right beside the text. I don't know
 what happens if I zoom in. Okay, they just are pop over. Also allows you to jump to equations and then
 using the back button, jump back to where you were. This is like this is the greatest thing ever.
 The amount of times I had not clicked on like an internal reference on a PDF, just because I was
 like, No, I'm not going to scroll back to where I was. So thank you. Check out our five.
 Okay, we have some helpful things this week. The first helpful thing is
 itai saying they've released over 170 pre trained transformer checkpoints, many different shapes
 and sizes as part of their paper. This is by Google research. Check out the scaling transformers
 paper, the scaling transformers repo, and the models released. Thank you. FFCV is a library
 by the lab of Alexander Madri that makes training machine learning models fast. If there's ever like
 a buzzwordy title that says nothing, it's train machine learning models fast. So they provide a
 set of sort of throw in replacements, for example, for data loaders that will just kind of speed up
 common use cases of training neural networks. They claim their code is hyper optimized removes
 bottlenecks, it's super duper pipeline and parallel and all of that. So if speed is an issue for you,
 maybe give this a try. OTT or optimal transport tools is a toolbox for all things. Vosserstein,
 as they call it, it is an optimal transport library for Jacks. Sumit Chintala advertises
 diet GPU, which is a lossless compression algorithm for Nvidia GPUs. The code of this is available.
 It's authored by Jeff Johnson. And what it does is it can compress stuff and uncompressed stuff on
 GPUs. So if you have a slow network, and you have a distributed training, and you really care about
 making this fast and efficient, what you can do is you can compress stuff that you need to send over
 the network on the GPUs, send it over, then uncompress it. This library will make the
 compression and uncompression part really fast and really efficient. All right, that was it for
 helpful things. I hope you got help. The user breman79 on Reddit says the ICML 2022 conference
 is changing their review process slightly. So now there are two phases. In phase one, the reviewers
 just give a recommendation. If there are two recommendations that are negative for a paper
 in phase one, it is already rejected. I guess this is a goal to call down on the amount of papers
 that have to be seriously reviewed. It's all the more important now that your paper makes a good
 first impression. So they say the meta reviewer can reverse this outcome. Okay. And other changes
 that reviewers do not make, accept or reject recommendations in phase two, the meta reviewers
 will decide based on the reviews. So I just write my review and then the meta reviewer reads it and
 integrates it all instead of me saying, well, this is a seven or this is a four, this is a strong
 accept or a weak accept. Now, technically it shouldn't make a difference, right? Because
 me voice, like my score that I would usually put is just kind of a conglomeration of what I said
 before. But you know, tiny changes like this, you know, because we're humans and we're not consistent
 and we're not, you know, we're not attentive enough, tiny changes like this might actually
 make a difference. I'd be interested to see some statistical analysis after the fact of what this
 did. If you're interested, the entire process is detailed in the ICML 2022 review form. Now it just
 occurred to me that the submission deadline was actually last week, which I should know. So if
 your paper is not pretty and doesn't make a good first impression, then you just you just gotta
 gotta hope for that really good meta reviewer that recognizes its inner beauty.
 This is a little bit older, but I hadn't seen it at the time. There is a blog post on Meta AI's
 research blog saying harmful content can evolve quickly. Our new AI system adapts to tackle it.
 So they describe a system that they call few shot learner, which essentially means that it's a
 system that can monitor harmful content and adapt quickly to new harmful content because it's ever
 evolving. I find a few things interesting right here. First on a sort of a scientific level,
 what is pretty interesting is that the model doesn't only consider training data. So data that
 has been labeled as harmful or not harmful or borderline or anything like this, it does do that.
 But it also takes a description of the policy, like a textual description of the current policy.
 And by doing that, it's able to adapt to policies over time, having some sort of a
 policy that says, you know, with this policy, this stuff is okay. And then with this new policy,
 this other stuff is okay. So the fine tuning process can potentially happen with less data.
 I found this pretty, pretty interesting to actually provide the policy itself to the model.
 The other interesting thing is just this video right here. So as you can see, the people here,
 they're interacting with the internet and they see harmful content and they're like,
 oh, like they're like, oh, no, I'm gonna log all, oh, no, all this harmful content.
 And then, you know, there's the system, they describe their system. Yeah, whoa, okay. So now
 they, you know, they filter all of this, this new harmful content. And then at the end, look what
 happens. Everyone's smiling, like, look, they're smiling. Oh, this is just, it is so awesome.
 Thank you. Thank you, Meta. Thank you. Ah, the few shot learner. Thank God all the harmful content
 was prevented from destroying smiles. Now, okay, on a more serious note, it is a hard problem,
 right? There's no way you can monitor all the content all the time. There's no way you can
 train a static system because sort of the meta of bad content, of bad language, of people bullying
 each other and so on is always evolving. So props to, you know, Meta for actually trying to tackle
 this problem because what, I mean, what's the alternative? Shut down all communication.
 That's not gonna happen. Tell people to be nice, like, well, try. But I see a bit too much complaining
 about this. And yeah, I do, I do like that they're actually tackling this problem. And I find the
 approach to be cool. It's just the marketing that's a bit cringy. But what am I saying? I'm wearing
 sunglasses indoors. Okay, last news for the day. MIT technology review says, this company says it's
 developing a system that can recognize your face from just your DNA. Now, people have been extremely
 skeptical of statements like these. This is a company that deals in broad language with
 law enforcement, searching people, security, surveillance, and so on. And you know, you might
 debate the merits or unmerits of that in a separate topic. But the particular question of can we
 actually get someone's facial features from their DNA is highly debated. Just to be said, the company
 isn't only focused on that. It's called core site and they have different plans. These are not systems
 that run right now. These are sort of future plans to do things. One of them is this DNA to face thing.
 Now, I do feel the criticisms of this are often maybe overly skeptical, let's say. Now, again,
 I don't mind the skepticism about the applications of this, but the possibility that there's a reason
 that children often look like their parents, your facial structure is in large part determined by
 your genetic material. Now, the article points out that obviously age and environmental influences
 also have big impacts on that. So no doubt about that. And they make a good point in that they say
 the technology will probably not be able to tell you the exact number of millimeters between the
 eyes or the ratios between the eyes, nose and mouth. And those are some of the features that
 the current facial recognition technologies rely upon. So since we can't get those features
 accurately from genetic data, because there may be more environmentally determined, the current
 facial recognition algorithms wouldn't work. However, I don't see the extrapolation discussed
 right here in that I would think it might be absolutely possible to train facial recognition
 algorithms that only use the features that we can read from the DNA. Like the argument that the face
 reconstructions that the DNA data gives us doesn't work with current facial recognition software is
 almost a moot point by then. Question is obviously how accurate it's going to be. And again, whether
 or not you even want to do this in the first place. But let me know what you think. Should this be
 done? Can this be done? And would you want to do it? Let me know in the comments. This was ML News.
 Thank you so much for being here. I'll see you next time. Bye bye.