[ML News] AI Threatens Biological Arms Race
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper gtc gtc22 nvidia jensen huang 3090 rtx 3090 ithaca deepmind deep mind deepmind greek text deepmind ithaca ml news mlnews ai news kilcher news drug discovery ai drug discovery ai drug development yoshua bengio joshua bengio yosha bengio bengio knight gary marcus deep learning wall gary marcus deep learning pig grunts ai animal communication meta ai
#mlnews #gtc22 #ithaca

GTC Registration Link: https://ykilcher.com/gtc
Your regular updates on what's going on in the ML world!

OUTLINE:
0:00 - Intro
0:20 - Register to Nvidia GTC and win a 3090!
4:15 - DeepMind's Ithaca deciphers Lost Ancient Texts
6:45 - Drug discovery model turns toxic
10:00 - Gary Marcus: Deep Learning is hitting a wall
19:40 - GopherCite: Backing up answers with citations
22:40 - Yoshua Bengio appointed knight of the legion of honour
23:00 - Meta AI tags parody account of Yoshua Bengio
23:40 - Building games using just natural language
24:55 - YOU.com adds writing assistant
25:45 - Horace He: How to brrr
26:35 - Karpathy: Reproducing Yann LeCun's 1989 paper
27:50 - Pig grunt emotion classifier
28:20 - AI annotates protein domain functions
29:40 - Atwood & Carmack: 10k self-driving car bet
30:50 - Helpful Things

References:
Register to GTC and win a 3090!
https://twitter.com/NVIDIAEU/status/1501881813651836930
https://www.nvidia.com/gtc/keynote/?ncid=so-twit-533413&=&linkId=100000114410590
https://www.nvidia.com/gtc/?ncid=ref-inpa-330612
https://www.nvidia.com/gtc/keynote/
https://www.nvidia.com/gtc/training/
https://developer.nvidia.com/nvidia-omniverse-platform

DeepMind deciphers Lost Ancient Texts
https://deepmind.com/blog/article/Predicting-the-past-with-Ithaca
https://www.nature.com/articles/s41586-022-04448-z
https://github.com/deepmind/ithaca
https://ithaca.deepmind.com/?job=eyJyZXF1ZXN0SUQiOiI1N2I4MWFjNTIxNGM3NDBiMjc3YzA1YzFiOTYwYzI0NCIsImF0dHJpYnV0aW9uIjp0cnVlLCJyZXN0b3JhdGlvbiI6dHJ1ZX0%3D

Drug discovery model turns toxic
https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx
https://www.nature.com/articles/s42256-022-00465-9.pdf?utm_source=pocket_mylist

Gary Marcus: Deep Learning is hitting a wall
https://nautil.us/deep-learning-is-hitting-a-wall-14467/
https://www.youtube.com/watch?v=fVkXE330Bh0&t=4437s

GopherCite: Backing up answers with citations
https://deepmind.com/research/publications/2022/GopherCite-Teaching-Language-Models-To-Support-Answers-With-Verified-Quotes

Yoshua Bengio appointed knight of the legion of honour
https://mila.quebec/en/professor-yoshua-bengio-appointed-knight-of-the-legion-of-honour-by-france/

Meta AI tags parody account
https://twitter.com/MetaAI/status/1504575140532613125

Building games using just natural language
https://andrewmayneblog.wordpress.com/2022/03/17/building-games-and-apps-entirely-through-natural-language-using-openais-davinci-code-model/

YOU.com adds writing assistant
https://you.com/search?q=how%20to%20write%20well

Horace He: How to brrr
https://horace.io/brrr_intro.html

Karpathy: Reproducing Yann LeCun's 1989 paper
https://karpathy.github.io/2022/03/14/lecun1989/

Pig grunt emotion classifier
https://science.ku.dk/english/press/news/2022/pig-grunts-reveal-their-emotions/?utm_source=pocket_mylist

AI annotates protein domain functions
https://ai.googleblog.com/2022/03/using-deep-learning-to-annotate-protein.html?utm_source=pocket_mylist
https://google-research.github.io/proteinfer/

Atwood & Carmack: 10k self-driving car bet
https://blog.codinghorror.com/the-2030-self-driving-car-bet/?utm_source=pocket_mylist

Helpful Things
https://github.com/recognai/rubrix
https://twitter.com/taiyasaki/status/1501288630697877504
https://github.com/mosaicml/composer?src=twitter
https://mujoco.org/
https://mujoco.readthedocs.io/en/latest/changelog.html
https://github.com/deepmind/mctx?utm_source=pocket_mylist
https://padl.ai/
https://github.com/LaihoE/did-it-spill
https://pytorch.org/blog/pytorch-1.11-released/

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 DeepMind uses deep learning to restore ancient Greek texts. A drug discovery system has been abused to create thousands and thousands of super toxic compounds. And Gary Marcus claims deep learning is hitting a wall. Welcome to ML News. It's Monday. GTC conference goes into its next iteration. Now GTC is a company conference like all of the big companies, they present all of their newest stuff there. But they also have a host of external speakers and all kinds of people that just give education and talks about how they use deep learning for various things. Now all of it is obviously Nvidia themed. But I can promise you the talks are interesting by themselves as well. The highlight of the conference is obviously the keynote by Jensen Huang. And depending on when you're watching this video, the conference is going on probably right now. And the best part is if you use my link, that's by culture.com slash GTC and you use that to sign up for the conference, you can win a 3090 that has been hand signed by Jensen Huang. So the same person that is giving the keynote will have signed your GPU if you win it. Now this is pretty cool. All you have to do is sign up using my link and then attend at least one session and why not attend the keynote. The keynote will go into all of the upcoming things of Nvidia. For example, is there going to be something like a 4090? How does it look like? Why do they increase the first digit of 3090 and not just make it the 3091? All the biggest questions of humanity. Now other than new architectures coming up, there will also be a lot of talks on the topics of accelerated computing, autonomous driving, anything to do with computer vision, rendering, cybersecurity. Nvidia hardware now powers almost all of deep learning advances apart from some specialized vendors. So this is definitely a good place to look. Another thing I want to highlight is the Nvidia Omniverse platform, which is a high performance and really good simulation, physics and rendering engine. This includes Pixar's universal scene description technology and can be used to do accurate renderings. And since synthetic data is such a big deal in recent times, this could really be something to accelerate your research if you are into simulated data transferring to the real world. It's pretty cool and a lot of things can be done with it. And no, the Omniverse isn't the metaverse per se, but there is a session that you can attend in GTC that talks about the metaverse and how to build virtual connected worlds. And as you can see, one of the speakers is the VP of Omniverse. So in the end, everything's somehow going to be together. There are even sessions called connect with the experts where you get one on one time with experts in a certain area, for example, GPU performance analysis and optimization. This is first come, first serve. So area, as I said, besides the keynote, there is an entire plethora of sessions that you can attend. These go from building large language models to next generation rendering, to using AI for cybersecurity, or understanding how newest technologies can help your business. There's also more specialized tracks such as focuses on health care, autonomous driving, and other areas. Registration is free and you can put together your own little calendar that reminds you whenever these sessions are coming up. Again, use my link to sign up in order to win a 3090. There's one caveat you need to be in EMEA, which is Europe, Middle East or Africa, in order to qualify for the 3090 raffle. However, I've decided that anyone living outside of these areas can also participate in another raffle that I sponsor. And that will just give you some some merch. So inside EMEA, you can participate for the 3090 outside EMEA, you can participate for the merge. Now, if you are in either bucket, and you want to be in the other bucket, I'm sure we're going to do stuff in the future where you can win to your heart's content. But for now, this seems the most fairest allocation of resources. And remember, you have to attend a session in GTC in order to qualify for the 3090. DeepMind has released a new blog post called predicting the past with Ithaca. Now, this is a system that restores ancient texts, namely ancient texts from the Greeks. So throughout the years, a lot of these inscriptions in stone have gone missing, have been damaged. And therefore, historians, they need to tease out what things could mean. Now, this is obviously a good application for something like a language model. So what Ithaca does is it takes in whatever is undamaged, and a few hints of where it needs to fill in missing characters. And it tries to reconstruct these things. Not only will it give an output that restores the missing pieces of text, but it will also determine a probability distribution over the geographical origins of this piece of text, as well as a chronological attribution, meaning it will estimate when the text was written. Now, it's interesting to me, as you can see right here, the input is just plain text, I would have guessed that they would use some sort of computer visiony things as well, as maybe the Greeks would have written down some stuff in certain ways in certain order, but I'm not too educated in ancient Greek. So this might not have been the case after all. What is cool, though, is that the blog post goes into a lot of detail, not only about the system itself, and how good it is, which it undoubtedly is, but how the combination of humans and machines together can outperform anyone alone. They talk a lot about how to build tools in order for historians to be able to effectively interface with the system, and that it has really accelerated their research. Now, this isn't only good for ancient Greek texts, but the more we learn and how we can use AI in order to accelerate other fields, I think the better the success rates for all of science. This goes along with an open access paper in nature that you can read, the code is online, you can try it out for yourself. And they even have a website with a little demo application, or you can try it out yourself. And just in case you happen to have some ancient Greek block laying around with some damages in it, just enter it here, it will it will do it, it will predict it. Overall, I think it's a pretty cool trend what DeepMind is doing interfacing with lots of experts in adjacent and even non adjacent fields and using AI in order to come up with accelerations in those fields. I think it's a neat application and it benefits everyone. The verge writes AI suggested 40,000 new possible chemical weapons in just six hours. That is an interview with the author of this commentary here. It is called dual use of artificial intelligence powered drug discovery. So what has happened here is that there is a lot of research in drug discovery and AI accelerated drug discovery, obviously, and the mission there is to come up with compounds that achieve some sort of an effect while also not being toxic. It's a good property to have not being toxic. And what often is done is that there are toxicity data sets, so explicitly labeled substances and how toxic they are. And what those people can do is they can essentially take those data sets and train a classifier, an auxiliary classifier that helps their method avoid toxicity. So neural network A will try to come up with new compounds. And then neural network B would just reduce the likelihood of the ones that are really toxic. So you can imagine almost like a little bit of a regularizer or a loss component for the generative model of new compounds. Now all that these researchers did is simply flip the sign essentially in front of that auxiliary classifier. So instead of coming up with new compounds that go less toxic, these new compounds go more toxic. And what's interesting is that they observe that this system will immediately give them lots of substances that have been used for doing chemical warfare. And also a couple of instances of substances that are more toxic than the nerve agent VX, which is very lethal compound in very, very small doses, it paralyzes your lungs and you dead. So this is quite concerning because of the easiness of how that is to do essentially, if you are a little bit into drug discovery, and you can handle a bit of machine learning, this is relatively simple to do. The more hard part here is to actually synthesize those molecules, although that is also not too difficult as the article alludes. The article is necessarily kept not very detailed in order to not just, you know, throw out exactly how to do it. But it is implied that anyone with a bit of knowledge of the topic could go about doing this. And this comes back to what I've been saying for a while, I didn't invent this opinion, but I was always saying that any technology can be used for good and for bad with like a few tiny pieces of exception, the goodness or badness of the technology is almost two sides of the same coin. And this lays it pretty bare essentially any method that we have to make AI technologies somehow more beneficial, less toxic, more truthful, more reliable, anything like this, any method like this that is usually hailed. If you usually just flip a sign on something you flip one bit in the objective, you can achieve the exact opposite. There are very few techniques where you cannot directly derive a more quote unquote evil method from a quote unquote good method. Now to me, I think just raises a set of important questions. And I think it requires us to rethink a little bit how we deal with AI safety and with undesirable consequences of research. But if you have an opinion, let me know in the comments. Gary Marcus writes in Nautilus, deep learning is hitting a wall. This is an essay, an opinion piece essentially by Gary Marcus, who is a longtime AI researcher and author and public persona. If you've been in AI for a while, you've certainly heard of him. He's usually pitched as a little bit of an antagonist to the current paradigm of just do deep learning and scale it up big. And this article right here lays out some of his arguments, but also ends on an optimistic note of the future of deep learning and its combination with symbolic methods. The core story thread of the article is Gary Marcus recalling people like Jeffrey Hinton being very pro symbolic methods and combining symbolic methods with neural networks, let's say back in the day. So symbolic methods contrary to continuous or distributed methods would be methods where you can explicitly manipulate discrete symbols. The extreme version of this would be things like logical systems or expert systems. Now these can get quite complicated in that you can have symbols which themselves are functions over other symbols, symbols that represent abstract concepts and very complicated parameterized manipulation of those symbols. If you go to the other extreme, which is currently very popular, it is that essentially continuous distributed representation systems such as deep neural networks will be able to do all of the AI tasks that we could possibly want. Proponents of this view would say that if we just keep scaling up systems like GPT-3 or so, then AGI will emerge. Now what Marcus is pleading for here ultimately is that we need a synthesis of the two methods in order to progress in the field of AI. Now this in itself, I don't think is that controversial. People I think are well aware that deep learning has some limitations, especially let's call it pure deep learning, just scaling up and feeding more data. And obviously some tasks are tackled way better by symbolic methods. However, this article has created quite a stir on social media, lots of people commenting on it getting into a little bit of fights about it. And I've been trying to understand what's going on right here. So my conclusions are not as much as the content of the article is necessarily wrong, or the conclusions that we need the synthesis is out of the ordinary. However, the framing is such that Marcus tends to be quite critical of the recent advances in the distributed system. So in the deep neural networks, and what I think is unreasonably bullish on symbolic methods and their appeals. Now, as I said, the storyline goes very much with the development of Jeff Hinton, who at one point, apparently has been more pro fusing symbolic methods with neural networks, and then somehow has transitioned into discarding symbolic methods more and more saying that neural networks will essentially be able to do it all to do reasoning to do understanding, etc. Now, I think this itself is a little bit also of a one sided framing of Jeff Hinton's views. But you can definitely see how Jeff Hinton is a strong advocate for neural systems and for distributed systems doing these things. And I have various points to make right here. I think one of the fundamental questions is that obviously we all know that for some tasks, we need some kind of symbolic logical reasoning, it can't just all be done like latently and so on because well, we observe ourselves and we ourselves do symbolic logic reasoning. So point one is that even though we do symbolic reasoning, it is implemented in neural hardware. In fact, nowhere in the brain is there an explicit symbolic processor. So all the evidence we have is that even though symbolic manipulation might be going on in the brain, it is emergent from the underlying neurological structure. Now does that mean we have to go the same route in deep learning in that we train the neurological structure to do the symbolic manipulations? Or does it mean we could take a shortcut and directly implement the symbolic manipulations by itself? I don't know. I'm just saying the precedent is that everything in the brain as far as we see is implemented using a neural distributed architecture and not an explicit symbolic one. On the other hand, the brain obviously consists of super duper specialized parts, all interacting in very sparse and structured manners. And the current deep learning systems that we have are essentially very fully connected, very homogeneous systems, which are also very unlike the brain. So the argument only counts about half. The next thing is and somewhat of an issue I have with symbolicists or let's call it hybridists attacking deep learning in that they tend to be a little bit too dismissive of the abilities of deep learning. And the example that often comes up is something like GPT-3. Now, obviously, it's easy to go ahead and criticize GPT-3. It exhibits many failure cases, whether it represents a really bad therapist, or it just invents facts out of thin air. But I think there wasn't really a person in the world that wasn't a little bit at least surprised by just how much it can do. Like, of course, in hindsight, you can always say, well, it's just a bigger version of GPT-2. Well, it just kind of recites its training examples. And I agree, it does it kind of recites and moshes its training examples. I personally think humans don't do that much more. But there are definitely emergent phenomena, for example, the sheer ability to in context learn as well as it does, that emerge just purely out of a function of the scale, and not because we built anything explicitly in. And I think when people are very bullish on neural methods, what they refer to is this ability, this emergence of functionality that we previously thought could only be explicitly implemented by a symbolic approach. And that just arise if we scale things up. Now, it is true, our ability to scale things up, especially the exponential scaling that we require for deep learning has come to a little bit of a stop since now it takes entire giant companies to implement one of those things. And it is not clear how we can scale that up 10x 100x or 1000x more. But that doesn't necessarily dismiss the claim. Marcus also criticizes things like if GPT-3 has all these failure modes, then, you know, be careful about wanting this in your self driving car. And I think those miss a little bit what we're going for. GPT-3 is aimed to produce text as if it were found on the internet. And that's what you're getting. If people expect to get a truthful or factual or helpful answer out of GPT-3, that fundamentally misses what it was trained for. Now, if someone sat me in a car and said, this car was trained on driving like human drivers, and we filtered out all the human drivers that got into accidents, and it has really learned well how to replicate the human driving ability, then I'd be quite comfortable because that's exactly what I want. I want the car to drive like a human would drive. So there's much less of a mismatch of what the thing is trained for, and what I'm using the thing for. And therefore, I think at least half of the criticism leveraged here is not really applicable to something like self driving cars. The other half is. And likewise, Marcus brings up the net hack challenge right here as an example for how deep methods are still way behind symbolic methods mentioning that in the net hack challenge, the symbolic methods way outperformed the learning methods. By the way, if you don't know net hack is this little game that is largely text based or at least ASCII based. And you have to do exploration, you have to do long term reasoning and so on. Now what I find a little bit worth mentioning is that the symbolic methods that actually one they are just handcrafted they are, and I'm sure the neural methods to an extent are too. But the symbolic methods are just bots for the game, they just implement the game, they parse the messages, they list items they have, they have heuristics for battle for doing anything essentially, everything is hard coded. This is the Boston dynamics of net hack. And I think that kind of misses the point of why we're trying to get deep learning to do these types of things. Because deep learning, they are largely more general methods that we could apply to any sort of environment. And this just happens to be like a very defined environment, the net hack environment, where everything is super bounded and all the inputs are extremely expected and parsable. Yet deep learning has the potential to be much more generalizable and much more applicable to multiple things at the same time. Whereas a bot like this, you can transfer to even a similar game. So I think that kind of criticism is a bit weak too. Now the article by Marcus ends on a high note saying for the first time in 40 years, I finally feel some optimism about AI as recounting that after the symbolic methods had been almost a little bit frowned upon by the community, they do make a resurgence and hybrid approaches do seem to be promising a interesting area for the future. And with that, I agree. And I think the article itself is a cool read. If you are interested more in Marcus's arguments, and a little bit of the history as he sees it, please give it a read. DeepMind releases go for site, which is a language model that supports its answers with verified quotes. This is a language model that will go out and search for information as you query it. And it will first of all base its answers on these citations. But second of all, also be able to actually serve you the citations. Now this is not the first kind of its system. There have been other attempts at doing this. And this is just one in this iteration. But it is an interesting approach. These language models, they do tend to hallucinate a bunch of facts, because there's always a conflicting interest between the language model objective, and sort of the let's call it factual consistency. And if you go deeper, that is a mismatch between the model wanting to be grammatical, but also kind of good at reciting whatever is in the data. And so sometimes that leads to hallucinated facts. And this can be drastically reduced if you base whatever you produce on actual citations that exist somewhere. Now this has advantages and disadvantages. Obviously, the advantages, you'll be more accurate on some of these questions, you'll be able to provide the user directly with the citation that you base your reasoning on. However, there are also things that don't work so well. What they discuss here is an example that says, what does drinking Red Bull give you? And the answer being wings is wrong, because there is a citation, but obviously drinking Red Bull doesn't give you wings. However, this is the type of argument that I also don't quite buy. Because if I go to a human and I asked them, you know, what does drinking Red Bull give you, they will either say diabetes or wings. I don't see why we play such a focus on evaluating these language models on like factual truthfulness, when we query them with questions that really imply not a factual truthfulness, but sort of the truthfulness, according to common lore, or what advertisement tells us. I mean, for all intents and purposes, if a human gave you this answer, you would be happy if that was the question that you asked. So these things being brought up as negative examples are kind of shady to me. What I can imagine it also doesn't do that well is give you answers where you need to synthesize multiple passages, multiple things of citations, although I'm pretty sure you could extend the system to pull all kinds of citations, maybe actually already do that. But the main focus really seems to be on going out finding some citations that actually answers your questions and then gives you that. Another cool thing about these systems is that you don't need to encapsulate all their knowledge into their parameters at training time. So they can potentially even answer questions about topics they've never seen during training simply by you providing them with more external sources that they can query at inference time. So go for site was here able to answer questions about itself. So that's very cool. In other news, Mila writes that Professor Joshua Benjo was appointed knight of the Legion of Honor by France. This is one of the highest honors that France gives out. Obviously, Benjo is Canadian, but he fosters a lot of collaboration between France and Canada. And it's really cool to see him honored once more. Speaking of Joshua Benjo, Meta AI has tweeted out a little clip and a little advertisement for a discussion that was moderated by Alex Friedman between Yann LeCun and Joshua Benjo. They've tagged all the people on Twitter. Now, Joshua Benjo is not on Twitter. And you know, good for him. But they've just gone with the first result that popped up in the search, which is a parody account of a bored Benjo. So I don't know why, but I just find this really funny. Please follow bored Benjo on Twitter. If the account gets enough followers, we can maybe bully the real Benjo to also get on Twitter. Andrew Maine released a cool blog post titled building games and apps entirely through natural language using OpenAI's code DaVinci model. So this is essentially an exploration of OpenAI's codex model that can take in natural language and produce code. And Andrew has used this to build various games. And it's pretty cool to see, for example, here is a minimal legend of Zelda that was built using this input right here. That's it. That's the input. There are various other projects such as a wordle clone, a matrix rain effect, tic tac toe, an image manipulation tool, and much more. What I find really interesting is that you can't really yet describe the application you want in natural language as a non programmer would do. But you still very much have to speak like a programmer. Essentially, you have to write all the comments that go with your code. And the model will simply implement that stuff for you. So this might be an artifact of how it's trained and could definitely help programmers in the future. However, it also shows we're not quite at the point yet where a non programmer could sit down and use one of these models to build an application. The use search engine has added a little tool that's called you write that helps you write stuff. So you input whatever you want here, and you'll get out a text and I thought we'll just make the title of this video will be whatever you write outputs. So we'll go to the article about the toxic compounds. We're just kind of copy the thing here or paste it here. We want a title. Our audience is YouTube. We want a tone that is persuasive. Let's go AI threatens biological arms race. Why not? Why not? Let it be the title. So if you want to try out you write then go to you.com search for how to write well currently you is in beta. So signups are free for now. I don't know for how long more for us has a blog post called making deep learning go from first principles and yes, you have to pronounce like so the theme of the blog post is that lots of people have either superstitious ideas of how to accelerate deep learning or they just kind of know some tricks from somewhere like, oh, just use whatever function here instead of that other function or in place operations are better or non in place operations are better. And this blog post goes into details in how you can think about deep learning performance and by that I mean, like things going fast and things being efficient from first principles by thinking about how compute and memory and transfer between accelerators and CPUs interact and so on is a pretty good read. And if you're interested, I definitely recommend that you check it out. Related Andre Karpat has released a new blog post in which he goes about recreating one famous paper of young Lecar from 1989 about handwritten digit recognition with convolutional neural networks. This is also very cool because Karpat the implements the original model as much as he can decipher from the original paper and tries to reproduce those results. I have to say he does get pretty close and then he goes ahead and implements all of the things that we've learned so far about deep learning about how to tweak architectures and so on. And he's able to bring down the validation loss by quite a bit. So in the end, he gets I think over a 60% reduction in validation error by implementing all of the newer techniques and finally also scaling up the data sets a bit. He draws some conclusions and finally concludes with a bit of a final look at the data set. He concludes with a bit of an outlook instead of looking 30 years into the past looking 30 years into the future, trying to extrapolate a little bit of what the world of deep learning and AI might look like then looking back to now is a pretty cool read and a pretty cool project. Definitely recommend you check it out. University of Copenhagen has a press release about their paper called pick grunts reveal about a system that has a data set of pick grunts with annotations of whether pigs are happy or not or surprised or anxious and it develops a system to classify these things. So all in all this is a pretty cool application of deep learning. And it turns out short grunts are happy grunts. Who knew? I guess farmers knew all along but you know, who knew? Google AI blog has a post about using deep learning to annotate the protein universe. Now, whereas systems like alpha fold have generated a lot of buzz, there are a lot of different tasks in the macro molecules or more specifically the protein area of biology. The one tackled here is the question of what kind of function does a protein have and what domains within the protein exhibit those functions. So the paper is about recent advances by Google to build systems that would annotate such sequences and proteins with their respective functions and push the state of the art by quite a bit. Now for that they use interestingly enough dilated convolutional networks. And they emphasize that a big part of getting this research to be successful is to actually also care for the implementation and the architecture. But also there's a big part in data set preparation and really validating your approach really making sure that what you do is effective and valid is a pretty cool read and along with it goes a larger a little bit of a website blog post a little bit like a distill article that is interactive that you can read and that contains some hands on demonstrations where you can learn about the architecture, learn about the results and explore a little bit by yourself. Jeff Atwood and John Carmack have made a bet. The bet is whether or not by January 1 2030 completely autonomous self driving cars meeting level five fully self driving specification will be commercially available for passenger use in major cities. In this instance, John Carmack is for and Jeff Atwood is against now I have to say 2030 isn't that far away. And as Jeff Atwood points out fully self driving is a really hard problem. However, as other people point out, in some major cities, you're already available to call something like a robot taxi, which doesn't seem to be too far away from what's needed. But that might just appear so because again, the gap between driving in controlled conditions on terrain and roads that you know where you have exact specifications of everything, and being able to handle most situations that a human driver would encounter anywhere at all times. That's a big difference. I'm not sure how this bet is going to turn out. That's why it's interesting. But I'm interested to hear your opinions in the comments. Alright, lastly, we'll get to some helpful things helpful things for this week rubrics is an open source platform for data centric NLP mostly specifying with managing text data and annotating it. Kubrick is a scalable data set generator for video and 3d data. Composer is a pytorch library for efficient neural network training, they implement a lot of the recent advances in speed ups of training and give you reproducible and accessible baselines for you to implement your own very speedy training loops. Mojoco is a physics simulation library, but I guess you already knew that. However, as we've reported deep mind took over bought essentially mojo co and is releasing it open source. And now they've implemented Python bindings. So you're just able to do pip install mojo co we've been waiting for this for decades. Thank you. MCTX is Monte Carlo tree search in Jack's paddle standing for pipeline abstractions for deep learning is a deep learning library that in its own words makes working with deep learning models intuitive, simple and fun. And it is entirely cross compatible with the entire pytorch and scientific Python ecosystem. Did it spill is a library for pytorch that checks if you have any test samples that were in the training set. Speaking of pytorch pytorch releases version one dot 11 with the addition of torch data and funk torch. Now these things have been brewing for a while, but it's pretty cool to see them added to the library. torch data is a library a bunch of functions that make it really easy to do various data set loading, composing and transforming things directly in the data loading pipeline, whereas funk torch is a library that adds composable function transforms to pytorch a little bit in the flavor of Jack's. So definitely check out both. Alright, that was already it for the helpful things and ml news. This episode is already way too long. Thank you for sticking around. Check out GTC use the link sign up win some merch or 3090 and I'll see you around. Thank you bye bye.
 DeepMind uses deep learning to restore ancient Greek texts. A drug discovery system has been
 abused to create thousands and thousands of super toxic compounds. And Gary Marcus claims
 deep learning is hitting a wall. Welcome to ML News. It's Monday.
 GTC conference goes into its next iteration. Now GTC is a company conference like all of the big
 companies, they present all of their newest stuff there. But they also have a host of external
 speakers and all kinds of people that just give education and talks about how they use deep learning
 for various things. Now all of it is obviously Nvidia themed. But I can promise you the talks
 are interesting by themselves as well. The highlight of the conference is obviously the
 keynote by Jensen Huang. And depending on when you're watching this video, the conference is
 going on probably right now. And the best part is if you use my link, that's by culture.com slash
 GTC and you use that to sign up for the conference, you can win a 3090 that has been hand signed by
 Jensen Huang. So the same person that is giving the keynote will have signed your GPU if you win
 it. Now this is pretty cool. All you have to do is sign up using my link and then attend at least
 one session and why not attend the keynote. The keynote will go into all of the upcoming things
 of Nvidia. For example, is there going to be something like a 4090? How does it look like?
 Why do they increase the first digit of 3090 and not just make it the 3091? All the biggest
 questions of humanity. Now other than new architectures coming up, there will also be a lot
 of talks on the topics of accelerated computing, autonomous driving, anything to do with computer
 vision, rendering, cybersecurity. Nvidia hardware now powers almost all of deep learning advances
 apart from some specialized vendors. So this is definitely a good place to look. Another thing I
 want to highlight is the Nvidia Omniverse platform, which is a high performance and really good
 simulation, physics and rendering engine. This includes Pixar's universal scene description
 technology and can be used to do accurate renderings. And since synthetic data is such a big
 deal in recent times, this could really be something to accelerate your research if you are into
 simulated data transferring to the real world. It's pretty cool and a lot of things can be done
 with it. And no, the Omniverse isn't the metaverse per se, but there is a session that you can attend
 in GTC that talks about the metaverse and how to build virtual connected worlds. And as you can see,
 one of the speakers is the VP of Omniverse. So in the end, everything's somehow going to be together.
 There are even sessions called connect with the experts where you get one on one time with experts
 in a certain area, for example, GPU performance analysis and optimization. This is first come,
 first serve. So area, as I said, besides the keynote, there is an entire plethora of sessions
 that you can attend. These go from building large language models to next generation rendering,
 to using AI for cybersecurity, or understanding how newest technologies can help your business.
 There's also more specialized tracks such as focuses on health care, autonomous driving,
 and other areas. Registration is free and you can put together your own little calendar that reminds
 you whenever these sessions are coming up. Again, use my link to sign up in order to win a 3090.
 There's one caveat you need to be in EMEA, which is Europe, Middle East or Africa, in order to
 qualify for the 3090 raffle. However, I've decided that anyone living outside of these areas can also
 participate in another raffle that I sponsor. And that will just give you some some merch.
 So inside EMEA, you can participate for the 3090 outside EMEA, you can participate for the merge.
 Now, if you are in either bucket, and you want to be in the other bucket, I'm sure we're going to
 do stuff in the future where you can win to your heart's content. But for now, this seems the most
 fairest allocation of resources. And remember, you have to attend a session in GTC in order to qualify
 for the 3090. DeepMind has released a new blog post called predicting the past with Ithaca. Now,
 this is a system that restores ancient texts, namely ancient texts from the Greeks. So throughout
 the years, a lot of these inscriptions in stone have gone missing, have been damaged. And therefore,
 historians, they need to tease out what things could mean. Now, this is obviously a good
 application for something like a language model. So what Ithaca does is it takes in whatever is
 undamaged, and a few hints of where it needs to fill in missing characters. And it tries to
 reconstruct these things. Not only will it give an output that restores the missing pieces of text,
 but it will also determine a probability distribution over the geographical origins
 of this piece of text, as well as a chronological attribution, meaning it will estimate when the
 text was written. Now, it's interesting to me, as you can see right here, the input is just plain
 text, I would have guessed that they would use some sort of computer visiony things as well,
 as maybe the Greeks would have written down some stuff in certain ways in certain order, but I'm
 not too educated in ancient Greek. So this might not have been the case after all. What is cool,
 though, is that the blog post goes into a lot of detail, not only about the system itself,
 and how good it is, which it undoubtedly is, but how the combination of humans and machines together
 can outperform anyone alone. They talk a lot about how to build tools in order for historians to be
 able to effectively interface with the system, and that it has really accelerated their research. Now,
 this isn't only good for ancient Greek texts, but the more we learn and how we can use AI in order
 to accelerate other fields, I think the better the success rates for all of science. This goes
 along with an open access paper in nature that you can read, the code is online, you can try it out
 for yourself. And they even have a website with a little demo application, or you can try it out
 yourself. And just in case you happen to have some ancient Greek block laying around with some damages
 in it, just enter it here, it will it will do it, it will predict it. Overall, I think it's a pretty
 cool trend what DeepMind is doing interfacing with lots of experts in adjacent and even non adjacent
 fields and using AI in order to come up with accelerations in those fields. I think it's a
 neat application and it benefits everyone. The verge writes AI suggested 40,000 new possible
 chemical weapons in just six hours. That is an interview with the author of this commentary
 here. It is called dual use of artificial intelligence powered drug discovery. So what
 has happened here is that there is a lot of research in drug discovery and AI accelerated
 drug discovery, obviously, and the mission there is to come up with compounds that achieve some
 sort of an effect while also not being toxic. It's a good property to have not being toxic.
 And what often is done is that there are toxicity data sets, so explicitly labeled substances and
 how toxic they are. And what those people can do is they can essentially take those data sets
 and train a classifier, an auxiliary classifier that helps their method avoid toxicity. So neural
 network A will try to come up with new compounds. And then neural network B would just reduce the
 likelihood of the ones that are really toxic. So you can imagine almost like a little bit of
 a regularizer or a loss component for the generative model of new compounds. Now all that these
 researchers did is simply flip the sign essentially in front of that auxiliary classifier. So instead
 of coming up with new compounds that go less toxic, these new compounds go more toxic. And what's
 interesting is that they observe that this system will immediately give them lots of substances that
 have been used for doing chemical warfare. And also a couple of instances of substances that are
 more toxic than the nerve agent VX, which is very lethal compound in very, very small doses,
 it paralyzes your lungs and you dead. So this is quite concerning because of the easiness of how
 that is to do essentially, if you are a little bit into drug discovery, and you can handle a bit of
 machine learning, this is relatively simple to do. The more hard part here is to actually synthesize
 those molecules, although that is also not too difficult as the article alludes. The article is
 necessarily kept not very detailed in order to not just, you know, throw out exactly how to do it.
 But it is implied that anyone with a bit of knowledge of the topic could go about doing this.
 And this comes back to what I've been saying for a while, I didn't invent this opinion, but I was
 always saying that any technology can be used for good and for bad with like a few tiny pieces of
 exception, the goodness or badness of the technology is almost two sides of the same coin. And this lays
 it pretty bare essentially any method that we have to make AI technologies somehow more beneficial,
 less toxic, more truthful, more reliable, anything like this, any method like this that is usually
 hailed. If you usually just flip a sign on something you flip one bit in the objective,
 you can achieve the exact opposite. There are very few techniques where you cannot directly derive
 a more quote unquote evil method from a quote unquote good method. Now to me, I think just raises
 a set of important questions. And I think it requires us to rethink a little bit how we deal
 with AI safety and with undesirable consequences of research. But if you have an opinion, let me
 know in the comments. Gary Marcus writes in Nautilus, deep learning is hitting a wall. This is an essay,
 an opinion piece essentially by Gary Marcus, who is a longtime AI researcher and author and public
 persona. If you've been in AI for a while, you've certainly heard of him. He's usually pitched as a
 little bit of an antagonist to the current paradigm of just do deep learning and scale it up big. And
 this article right here lays out some of his arguments, but also ends on an optimistic note
 of the future of deep learning and its combination with symbolic methods. The core story thread of
 the article is Gary Marcus recalling people like Jeffrey Hinton being very pro symbolic methods and
 combining symbolic methods with neural networks, let's say back in the day. So symbolic methods
 contrary to continuous or distributed methods would be methods where you can explicitly manipulate
 discrete symbols. The extreme version of this would be things like logical systems or expert systems.
 Now these can get quite complicated in that you can have symbols which themselves are functions
 over other symbols, symbols that represent abstract concepts and very complicated parameterized
 manipulation of those symbols. If you go to the other extreme, which is currently very popular,
 it is that essentially continuous distributed representation systems such as deep neural
 networks will be able to do all of the AI tasks that we could possibly want. Proponents of this
 view would say that if we just keep scaling up systems like GPT-3 or so, then AGI will emerge.
 Now what Marcus is pleading for here ultimately is that we need a synthesis of the two methods
 in order to progress in the field of AI. Now this in itself, I don't think is that controversial.
 People I think are well aware that deep learning has some limitations, especially let's call it
 pure deep learning, just scaling up and feeding more data. And obviously some tasks are tackled
 way better by symbolic methods. However, this article has created quite a stir on social media,
 lots of people commenting on it getting into a little bit of fights about it. And I've been
 trying to understand what's going on right here. So my conclusions are not as much as the content
 of the article is necessarily wrong, or the conclusions that we need the synthesis is out
 of the ordinary. However, the framing is such that Marcus tends to be quite critical of the
 recent advances in the distributed system. So in the deep neural networks, and what I think is
 unreasonably bullish on symbolic methods and their appeals. Now, as I said, the storyline goes very
 much with the development of Jeff Hinton, who at one point, apparently has been more pro fusing
 symbolic methods with neural networks, and then somehow has transitioned into discarding symbolic
 methods more and more saying that neural networks will essentially be able to do it all to do
 reasoning to do understanding, etc. Now, I think this itself is a little bit also of a one sided
 framing of Jeff Hinton's views. But you can definitely see how Jeff Hinton is a strong
 advocate for neural systems and for distributed systems doing these things. And I have various
 points to make right here. I think one of the fundamental questions is that obviously we all
 know that for some tasks, we need some kind of symbolic logical reasoning, it can't just all
 be done like latently and so on because well, we observe ourselves and we ourselves do symbolic
 logic reasoning. So point one is that even though we do symbolic reasoning, it is implemented in
 neural hardware. In fact, nowhere in the brain is there an explicit symbolic processor. So all the
 evidence we have is that even though symbolic manipulation might be going on in the brain,
 it is emergent from the underlying neurological structure. Now does that mean we have to go the
 same route in deep learning in that we train the neurological structure to do the symbolic
 manipulations? Or does it mean we could take a shortcut and directly implement the symbolic
 manipulations by itself? I don't know. I'm just saying the precedent is that everything in the
 brain as far as we see is implemented using a neural distributed architecture and not an
 explicit symbolic one. On the other hand, the brain obviously consists of super duper specialized
 parts, all interacting in very sparse and structured manners. And the current deep learning
 systems that we have are essentially very fully connected, very homogeneous systems, which are
 also very unlike the brain. So the argument only counts about half. The next thing is and somewhat
 of an issue I have with symbolicists or let's call it hybridists attacking deep learning in that they
 tend to be a little bit too dismissive of the abilities of deep learning. And the example that
 often comes up is something like GPT-3. Now, obviously, it's easy to go ahead and criticize
 GPT-3. It exhibits many failure cases, whether it represents a really bad therapist, or it just
 invents facts out of thin air. But I think there wasn't really a person in the world that wasn't
 a little bit at least surprised by just how much it can do. Like, of course, in hindsight, you can
 always say, well, it's just a bigger version of GPT-2. Well, it just kind of recites its training
 examples. And I agree, it does it kind of recites and moshes its training examples. I personally
 think humans don't do that much more. But there are definitely emergent phenomena, for example,
 the sheer ability to in context learn as well as it does, that emerge just purely out of a function
 of the scale, and not because we built anything explicitly in. And I think when people are very
 bullish on neural methods, what they refer to is this ability, this emergence of functionality that
 we previously thought could only be explicitly implemented by a symbolic approach. And that just
 arise if we scale things up. Now, it is true, our ability to scale things up, especially the
 exponential scaling that we require for deep learning has come to a little bit of a stop since
 now it takes entire giant companies to implement one of those things. And it is not clear how we
 can scale that up 10x 100x or 1000x more. But that doesn't necessarily dismiss the claim.
 Marcus also criticizes things like if GPT-3 has all these failure modes, then, you know, be careful
 about wanting this in your self driving car. And I think those miss a little bit what we're going
 for. GPT-3 is aimed to produce text as if it were found on the internet. And that's what you're
 getting. If people expect to get a truthful or factual or helpful answer out of GPT-3,
 that fundamentally misses what it was trained for. Now, if someone sat me in a car and said,
 this car was trained on driving like human drivers, and we filtered out all the human
 drivers that got into accidents, and it has really learned well how to replicate the human
 driving ability, then I'd be quite comfortable because that's exactly what I want. I want the
 car to drive like a human would drive. So there's much less of a mismatch of what the thing is
 trained for, and what I'm using the thing for. And therefore, I think at least half of the
 criticism leveraged here is not really applicable to something like self driving cars. The other
 half is. And likewise, Marcus brings up the net hack challenge right here as an example for how
 deep methods are still way behind symbolic methods mentioning that in the net hack challenge,
 the symbolic methods way outperformed the learning methods. By the way, if you don't know net hack is
 this little game that is largely text based or at least ASCII based. And you have to do exploration,
 you have to do long term reasoning and so on. Now what I find a little bit worth mentioning is that
 the symbolic methods that actually one they are just handcrafted they are, and I'm sure the neural
 methods to an extent are too. But the symbolic methods are just bots for the game, they just
 implement the game, they parse the messages, they list items they have, they have heuristics for
 battle for doing anything essentially, everything is hard coded. This is the Boston dynamics of
 net hack. And I think that kind of misses the point of why we're trying to get deep learning to do
 these types of things. Because deep learning, they are largely more general methods that we could
 apply to any sort of environment. And this just happens to be like a very defined environment,
 the net hack environment, where everything is super bounded and all the inputs are extremely
 expected and parsable. Yet deep learning has the potential to be much more generalizable and much
 more applicable to multiple things at the same time. Whereas a bot like this, you can transfer
 to even a similar game. So I think that kind of criticism is a bit weak too. Now the article by
 Marcus ends on a high note saying for the first time in 40 years, I finally feel some optimism
 about AI as recounting that after the symbolic methods had been almost a little bit frowned upon
 by the community, they do make a resurgence and hybrid approaches do seem to be promising
 a interesting area for the future. And with that, I agree. And I think the article itself is a cool
 read. If you are interested more in Marcus's arguments, and a little bit of the history as
 he sees it, please give it a read. DeepMind releases go for site, which is a language model
 that supports its answers with verified quotes. This is a language model that will go out and
 search for information as you query it. And it will first of all base its answers on these citations.
 But second of all, also be able to actually serve you the citations. Now this is not the first kind
 of its system. There have been other attempts at doing this. And this is just one in this iteration.
 But it is an interesting approach. These language models, they do tend to hallucinate a bunch of
 facts, because there's always a conflicting interest between the language model objective,
 and sort of the let's call it factual consistency. And if you go deeper, that is a mismatch between
 the model wanting to be grammatical, but also kind of good at reciting whatever is in the data. And
 so sometimes that leads to hallucinated facts. And this can be drastically reduced if you base
 whatever you produce on actual citations that exist somewhere. Now this has advantages and
 disadvantages. Obviously, the advantages, you'll be more accurate on some of these questions,
 you'll be able to provide the user directly with the citation that you base your reasoning on.
 However, there are also things that don't work so well. What they discuss here is an example that
 says, what does drinking Red Bull give you? And the answer being wings is wrong, because there is a
 citation, but obviously drinking Red Bull doesn't give you wings. However, this is the type of
 argument that I also don't quite buy. Because if I go to a human and I asked them, you know,
 what does drinking Red Bull give you, they will either say diabetes or wings. I don't see why
 we play such a focus on evaluating these language models on like factual truthfulness, when we query
 them with questions that really imply not a factual truthfulness, but sort of the truthfulness,
 according to common lore, or what advertisement tells us. I mean, for all intents and purposes,
 if a human gave you this answer, you would be happy if that was the question that you asked.
 So these things being brought up as negative examples are kind of shady to me. What I can
 imagine it also doesn't do that well is give you answers where you need to synthesize multiple
 passages, multiple things of citations, although I'm pretty sure you could extend the system to
 pull all kinds of citations, maybe actually already do that. But the main focus really seems to be on
 going out finding some citations that actually answers your questions and then gives you that.
 Another cool thing about these systems is that you don't need to encapsulate all their knowledge
 into their parameters at training time. So they can potentially even answer questions about topics
 they've never seen during training simply by you providing them with more external sources that they
 can query at inference time. So go for site was here able to answer questions about itself. So
 that's very cool. In other news, Mila writes that Professor Joshua Benjo was appointed knight of the
 Legion of Honor by France. This is one of the highest honors that France gives out. Obviously,
 Benjo is Canadian, but he fosters a lot of collaboration between France and Canada. And
 it's really cool to see him honored once more. Speaking of Joshua Benjo, Meta AI has tweeted out
 a little clip and a little advertisement for a discussion that was moderated by Alex Friedman
 between Yann LeCun and Joshua Benjo. They've tagged all the people on Twitter. Now, Joshua Benjo
 is not on Twitter. And you know, good for him. But they've just gone with the first result that
 popped up in the search, which is a parody account of a bored Benjo. So I don't know why, but I just
 find this really funny. Please follow bored Benjo on Twitter. If the account gets enough followers,
 we can maybe bully the real Benjo to also get on Twitter. Andrew Maine released a cool blog post
 titled building games and apps entirely through natural language using OpenAI's code DaVinci model.
 So this is essentially an exploration of OpenAI's codex model that can take in natural language and
 produce code. And Andrew has used this to build various games. And it's pretty cool to see, for
 example, here is a minimal legend of Zelda that was built using this input right here. That's it.
 That's the input. There are various other projects such as a wordle clone, a matrix rain effect,
 tic tac toe, an image manipulation tool, and much more. What I find really interesting is that you
 can't really yet describe the application you want in natural language as a non programmer would do.
 But you still very much have to speak like a programmer. Essentially, you have to write all
 the comments that go with your code. And the model will simply implement that stuff for you. So this
 might be an artifact of how it's trained and could definitely help programmers in the future. However,
 it also shows we're not quite at the point yet where a non programmer could sit down and use
 one of these models to build an application. The use search engine has added a little tool that's
 called you write that helps you write stuff. So you input whatever you want here, and you'll get
 out a text and I thought we'll just make the title of this video will be whatever you write outputs.
 So we'll go to the article about the toxic compounds. We're just kind of copy the thing here
 or paste it here. We want a title. Our audience is YouTube. We want a tone that is persuasive.
 Let's go AI threatens biological arms race. Why not? Why not? Let it be the title. So if you want
 to try out you write then go to you.com search for how to write well currently you is in beta. So
 signups are free for now. I don't know for how long more for us has a blog post called making
 deep learning go from first principles and yes, you have to pronounce like so the theme of the
 blog post is that lots of people have either superstitious ideas of how to accelerate deep
 learning or they just kind of know some tricks from somewhere like, oh, just use whatever function
 here instead of that other function or in place operations are better or non in place operations
 are better. And this blog post goes into details in how you can think about deep learning performance
 and by that I mean, like things going fast and things being efficient from first principles by
 thinking about how compute and memory and transfer between accelerators and CPUs interact and so on
 is a pretty good read. And if you're interested, I definitely recommend that you check it out.
 Related Andre Karpat has released a new blog post in which he goes about recreating one famous paper
 of young Lecar from 1989 about handwritten digit recognition with convolutional neural networks.
 This is also very cool because Karpat the implements the original model as much as he can
 decipher from the original paper and tries to reproduce those results. I have to say he does
 get pretty close and then he goes ahead and implements all of the things that we've learned
 so far about deep learning about how to tweak architectures and so on. And he's able to bring
 down the validation loss by quite a bit. So in the end, he gets I think over a 60% reduction in
 validation error by implementing all of the newer techniques and finally also scaling up the data
 sets a bit. He draws some conclusions and finally concludes with a bit of a final look at the
 data set. He concludes with a bit of an outlook instead of looking 30 years into the past looking
 30 years into the future, trying to extrapolate a little bit of what the world of deep learning
 and AI might look like then looking back to now is a pretty cool read and a pretty cool project.
 Definitely recommend you check it out.
 University of Copenhagen has a press release about their paper called pick grunts reveal
 about a system that has a data set of pick grunts with annotations of whether pigs are happy or not
 or surprised or anxious and it develops a system to classify these things. So all in all this is a
 pretty cool application of deep learning. And it turns out short grunts are happy grunts. Who knew?
 I guess farmers knew all along but you know, who knew? Google AI blog has a post about using deep
 learning to annotate the protein universe. Now, whereas systems like alpha fold have generated a
 lot of buzz, there are a lot of different tasks in the macro molecules or more specifically the
 protein area of biology. The one tackled here is the question of what kind of function does a protein
 have and what domains within the protein exhibit those functions. So the paper is about recent
 advances by Google to build systems that would annotate such sequences and proteins with their
 respective functions and push the state of the art by quite a bit. Now for that they use interestingly
 enough dilated convolutional networks. And they emphasize that a big part of getting this research
 to be successful is to actually also care for the implementation and the architecture. But also there's
 a big part in data set preparation and really validating your approach really making sure that
 what you do is effective and valid is a pretty cool read and along with it goes a larger a little
 bit of a website blog post a little bit like a distill article that is interactive that you can
 read and that contains some hands on demonstrations where you can learn about the architecture,
 learn about the results and explore a little bit by yourself. Jeff Atwood and John Carmack have made
 a bet. The bet is whether or not by January 1 2030 completely autonomous self driving cars
 meeting level five fully self driving specification will be commercially available for passenger use
 in major cities. In this instance, John Carmack is for and Jeff Atwood is against now I have to say
 2030 isn't that far away. And as Jeff Atwood points out fully self driving is a really hard
 problem. However, as other people point out, in some major cities, you're already available
 to call something like a robot taxi, which doesn't seem to be too far away from what's needed. But
 that might just appear so because again, the gap between driving in controlled conditions on terrain
 and roads that you know where you have exact specifications of everything, and being able to
 handle most situations that a human driver would encounter anywhere at all times. That's a big
 difference. I'm not sure how this bet is going to turn out. That's why it's interesting. But
 I'm interested to hear your opinions in the comments. Alright, lastly, we'll get to some
 helpful things helpful things for this week rubrics is an open source platform for data centric NLP
 mostly specifying with managing text data and annotating it. Kubrick is a scalable data set
 generator for video and 3d data. Composer is a pytorch library for efficient neural network
 training, they implement a lot of the recent advances in speed ups of training and give you
 reproducible and accessible baselines for you to implement your own very speedy training loops.
 Mojoco is a physics simulation library, but I guess you already knew that. However, as we've
 reported deep mind took over bought essentially mojo co and is releasing it open source. And now
 they've implemented Python bindings. So you're just able to do pip install mojo co we've been
 waiting for this for decades. Thank you. MCTX is Monte Carlo tree search in Jack's paddle standing
 for pipeline abstractions for deep learning is a deep learning library that in its own words makes
 working with deep learning models intuitive, simple and fun. And it is entirely cross compatible with
 the entire pytorch and scientific Python ecosystem. Did it spill is a library for pytorch that checks
 if you have any test samples that were in the training set. Speaking of pytorch pytorch releases
 version one dot 11 with the addition of torch data and funk torch. Now these things have been
 brewing for a while, but it's pretty cool to see them added to the library. torch data is a library
 a bunch of functions that make it really easy to do various data set loading, composing and
 transforming things directly in the data loading pipeline, whereas funk torch is a library that
 adds composable function transforms to pytorch a little bit in the flavor of Jack's. So definitely
 check out both. Alright, that was already it for the helpful things and ml news. This episode is
 already way too long. Thank you for sticking around. Check out GTC use the link sign up
 win some merch or 3090 and I'll see you around. Thank you bye bye.