[ML News] Google introduces Pathways | OpenAI solves Math Problems | Meta goes First Person
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper google google ai google pathways jeff dean pathways model sparse neural network meta meta ai ego4d sam altman openai openai math language model math t0 tzero bigscience bigsciencew deepmind deepmind lecture series huggingface huggingface hub dataset viewer machine learning news tech news
#pathways #mlnews #ego4d

Your irregular dose of Machine Learning News.

OUTLINE:
0:00 - Intro
0:20 - Sponsor: Weights & Biases
2:10 - Google Introduces Pathways AI Architecture
6:30 - OpenAI trains Language Models to do High School Math
8:25 - Sam Altman says Neural Networks truly learn
9:35 - Google AI researchers frustrated with lawyers
12:10 - DeepMind RL Lecture Series 2021
12:40 - Fashion Store sells Adversarial Patches
13:15 - A viable method to remove the GIL from CPython
15:05 - BigScience Workshop releases T0
17:40 - Huggingface Hub Dataset Viewer
18:10 - Scite classifies scientific citations
19:25 - Facebook AI Ego4D dataset & challenges
21:50 - Tesla Dojo Configurable Floating Point Spec
23:10 - Windows releases PyTorch-DirectML for Deep Learning on DirectX GPUs
23:50 - Helpful Things
33:00 - Traders use ML to analyze CEOs' language
34:20 - Cadbury creates DeepFake ads for local Indian businesses
35:25 - This Shoe Does Not Exist

Sponsor: Weights & Biases
https://wandb.com

References:
Google Introduces Pathways AI Architecture
https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/?utm_source=pocket_mylist

OpenAI trains Language Models to do High School Math
https://openai.com/blog/grade-school-math/
https://arxiv.org/abs/2110.14168

Sam Altman says Neural Networks truly learn
https://twitter.com/sama/status/1450857134648823809?s=09&t=KazQPHo6Epn0M6ihs4DqHg&utm_source=pocket_mylist

Google AI researchers frustrated with lawyers
https://archive.ph/lsQJJ#selection-2855.0-2855.294

DeepMind RL Lecture Series 2021
https://deepmind.com/learning-resources/reinforcement-learning-series-2021

Fashion Store sells Adversarial Patches
https://twitter.com/naotokui/status/1450673712722702340

A viable method to remove the GIL from CPython
https://lwn.net/Articles/872869/

BigScience Workshop releases T0
https://bigscience.huggingface.co/
https://arxiv.org/abs/2110.08207
https://huggingface.co/bigscience/T0pp

Huggingface Hub Dataset Viewer
https://twitter.com/huggingface/status/1454079471154257923

Scite classifies scientific citations
https://scite.ai
https://direct.mit.edu/qss/article/doi/10.1162/qss_a_00146/102990/scite-A-smart-citation-index-that-displays-the

Facebook AI Ego4D dataset & challenges
https://ai.facebook.com/blog/teaching-ai-to-perceive-the-world-through-your-eyes

Tesla Dojo Configurable Floating Point Spec
https://tesla-cdn.thron.com/static/SBY4B9_tesla-dojo-technology_OPNZ0M.pdf?xseo=&response-content-disposition=inline%3Bfilename%3D%22tesla-dojo-technology.pdf%22

Windows releases PyTorch-DirectML for Deep Learning on DirectX GPUs
https://devblogs.microsoft.com/windowsai/introducing-pytorch-directml-train-your-machine-learning-models-on-any-gpu/

Helpful Things
https://github.com/achaiah/pywick?utm_source=pocket_mylist
https://github.com/orybkin/lexa-benchmark?utm_source=pocket_mylist
https://orybkin.github.io/lexa/
https://twitter.com/danijarh/status/1438137568688807942?utm_source=pocket_mylist
https://github.com/RobertTLange/mle-hyperopt
https://keras.io/examples/vision/mobilevit/?utm_source=pocket_mylist
https://twitter.com/osanseviero/status/1451929248231563265?utm_source=pocket_mylist
https://huggingface.co/spaces/flax-community/image-captioning
https://huggingface.co/transformers/master/model_doc/visionencoderdecoder.html
https://github.com/facebookresearch/bitsandbytes
https://arxiv.org/abs/2110.11216
https://arxiv.org/pdf/2110.11216.pdf
https://github.com/facebookresearch/xformers
https://superbbenchmark.org/
https://arxiv.org/abs/2110.07731
https://github.com/BaguaSys/bagua?utm_source=pocket_mylist
https://github.com/cgarciae/treex
https://jax.readthedocs.io/en/latest/pytrees.html

Traders use ML to analyze CEOs' language
https://www.reuters.com/technology/ai-can-see-through-you-ceos-language-under-machine-microscope-2021-10-20/

Cadbury creates DeepFake ads for local Indian businesses
https://www.bgr.in/entertainment/shah-rukh-khan-not-just-a-cadbury-ad-twitter-diwali-celebration-1016913/

This Shoe Does Not Exist
https://www.thisshoedoesnotexist.com/


Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
 Google introduces pathways their next generation AI architecture, open AI solves high school math problems. And Facebook goes all on first person view. Welcome to ML news. But before the video starts, a quick thanks to our sponsor weights and biases, I want to show you this one feature that I just learned about, did you know you can embed a weights and biases report in notion, it's actually not only reports, but also other stuff by weights and biases. So they have this neat little page here, ironically, it is actually a notion and it is super easy to embed live weights and biases stuff into notion. So for example, here I have a sweep and you can see the sweep is interactive. So you can do all the kinds of things you're used to analyzing a weights and biases sweep. Now I can just grab that URL, get over to notion and create a new embed, paste a link. And there we go. Look at that. This is a fully functional weights and biases report inside of notion. So you have all the interactivity here that you would usually have as you can see. So I can look at my runs, I can activate them, I can even go and look at my sweep controls and various things. This is really cool if you work together with other people and you work on more than just weights and biases reports, you can take your notes and notion and then embed the report, the sweep, whatever into notion page. I love notion, I love weights and biases. And it's very cool that to go together. If you don't know weights and biases, it is your one stop shop for all your machine learning experimental needs from trying out models optimizing hyper parameters all the way to saving your models, deploying them and so on. It runs in the cloud, it's free for personal users and for education, there are plans for teams and for self hosted setups. So all the more reason to go try it out. Thanks again to weights and biases for sponsoring this video. And now let's get into it. Bye bye. Hello and welcome to ML news. Let's dive into our first story. Jeff Dean has released a blog post on the Google blog. No, this is not the Google AI blog. This is the main Google blog. He's also given a TED talk about the subject and the subject is this model called pathways, a next generation AI architecture, we don't actually know much about this architecture, because all we have is that TED talk and this illustration right here. And essentially, Jeff Dean imagines Google's future AI projects to rely on this new architecture, where instead of having single task neural networks that train, you have this giant multitask neural network that can do all the tasks at once. And that would also be sparsely activated. As you can see here, different tasks would leverage different paths through this network. This goes along with a few criticisms on today's architectures. So he says, for example, today's AI models are typically trained to do only one thing, pathways will enable us to train a single model to do 1000s or millions of things. So the goal is to have one model do many, many tasks at once. Second, he says today's models mostly focus on one sense pathways will enable multiple senses. This refers to the fact that the input to current neural networks are single modalities. Sometimes they're two modalities, but mostly they're single modalities, like images or text or sound this pathway architecture, naturally being multitask will also be multimodal, which means that it could input any sort of modality in his TED talk, he gives the example, whether or not you see a leopard or hear the word leopard or hear someone say the word leopard or see a video of a leopard that should essentially evoke the same concept in your brain and therefore also in the pathway model. And lastly, he says today's models are dense and inefficient pathways will make them sparse and efficient. This refers to the fact that our current networks are densely activated, everything's connected to everything. And that's very, very inefficient. He imagines this future pathways architecture to be sparsely activated, meaning that only very small subparts of the network will be activated for a given input sample. And therefore the different parts of the network doing different things, they don't always have to be active at the same time. This can also make the model much more efficient in terms of parameters and computation. Now, as I said, there's not a paper to go along with this or an implementation or even a plan of how to get there. This is essentially a wishlist, and it's not a particularly new wishlist. Like people have dreamed of, oh, can't we just make multi modal multi task models where one model learns everything? Well, yeah, everyone wishes that. But you still have the problems, namely, for example, catastrophic forgetting, if you try to teach the model many tasks, and then one task more, you still have to ensure that it doesn't forget the old tasks, which is very, very difficult, especially in this picture, it seems like this is a rather feed forward architecture right here without any sort of memory modules or anything like this. So how they're going to achieve that, I don't know. Secondly, they say there are many different tasks here. However, huge data architectures mostly rely on self supervision, and then fine tuning for individual tasks and not having different tasks in parallel, though multi task training is a thing. And lastly, the sparse activations are not trivial to achieve. Again, people have been saying this forever, like, well, can't we just have a sparse neural network, probably the brain is sparse, blah, blah, blah. But how are you going to get there? This is just a wishlist, how we're going to get there. I don't know. The main problem with sparsity being that if you have a sparse forward signal, then your backwards gradients are also going to be sparse, you may never learn the correct sparse way through your network. If you only activate sparsely in the forward pass. These are all challenges that have existed forever. But it seems like Google is determined to solve these challenges. I mean, if they can, all the better. But for now, it's just a plan and an idea. And I'm excited to see what happens. Open as released a blog post called solving math word problems where they train a language model to solve math problems. This goes along with a paper saying training verifiers to solve math word problems by people at Open AI, you can read it if you want. Essentially, it is a data set of about 8000 of these high school math problems, where you mainly need the basic addition, subtraction, multiplication and division in order to solve the problem. They're usually stated as little stories, and they have some sort of an answer. Now, large language models such as GPT three are usually kind of bad at this type of stuff, mainly because they are not accurate enough, they don't do these simple steps that are required enough, they're more like a language model, they're more like a conversation model, or a thing that simply repeats some of the stuff it has already seen. So the first approach the paper takes is to fine tune such a language model on these tasks. And it turns out that doesn't go too well. Very often that makes a lot of mistakes as well. And the solution comes in the form of what they call verifiers. So verifiers are model that are not trained to produce the solution, but they are trained to rate whether a solution to a problem is likely to be the correct solution or not. So now what they do is they use one model that they fine tuned to produce like 100 solutions, and then they use the verifiers to rank the solution and pick the best one. And that turns out to be very, very powerful. So we've seen approaches like this before, you remember the Dali model of open AI also not only used a generative model for the avocado chair, but it also used the clip model in order to rank the outputs of the generative model. So this could be a more general recipe for improving generative models is train verifiers, and then generate a bunch of solutions and rank them with the verifiers. As I said, you can read the paper and the data set of these math questions is available to download. Sam Altman tweeted neural networks really truly learn it's not a fancy trick. This is one of the most remarkable things humans have ever figured out and the implications are difficult to overstate. Now I'm not sure if he just wanted to start like a fire with this kind of things. There are many ways of going about this, but it seems like the truth or veracity of the statement entirely depends on how you define learning. But it seems like Sam Altman and in general, that's what we see out of open AI is of the opinion that learning that humans do isn't that much different from the learning that current large scale neural networks inherently do. Now this is to be set a little bit into contrast with what people from the more symbolicist camp may think about these neural networks and about the nature of learning and intelligence in general. But again, I guess it only depends on the definition of words here. And just putting the modifiers really and truly in front of a non defined word doesn't suddenly make it defined. But what do you think? Let me know in the comments after you hit the subscribe button. See what I did there. Next news business insider writes Google's AI researchers say their output is being slowed by lawyers after a string of high level exits getting published really is a nightmare right now. So the article starts off with a bunch of Google controversies, obviously, some famous people were fired from Google recently, and there were a bunch of scandals around that. And now one senior AI researcher who spoke with insider on the condition of anonymity comes forward and says, well, the lawyers are essentially up our necks right now. It's so difficult to publish, this is really stifling publishing inside of Google and so on. And the article backs this up by saying, according to Google's online records, the company published 925 pieces of AI research in 2019 and 962 in 2020. But the company looks to have experienced a moderate slowdown this year, publishing just 618 research papers in 2021. Thus far. Now this is the only thing where they actually back anything up that they say now I've no doubt that this is the case inside of these big companies, they give examples whenever they write words such as bias or fairness, then the lawyers they would just have like tons of questions or want to cross them out because they just don't understand the technical terms behind these things. Now noteworthy terms like bias and fairness actually have about 60 technical definitions, and they're all in conflict with each other. So can't exactly fault the lawyers. What I found funny is that in the last section here, a spokesperson from Google took a statement and said we're publishing papers at the same rate we did last year. At this time last year, there were 815 approved papers and this year there are 820 so far the spokesperson said adding our website doesn't reflect all papers and is typically updated a few months after publications. So they had to bury this on the very bottom of the article right here because they want to like tell a story about how lawyers are so terrible and about how this exit stifled Google so much and don't get me wrong, lawyers are terrible. And I'm pretty sure that there are pain in the neck. But the claim that this is especially ramped up now doesn't seem to hold apart from like one or two anonymous people inside of Google coming forward. And the fact that they have to hide this thing at the very bottom, which is pretty clear, like that's a much more likely explanation than Google now suddenly ramping up their eyeballs of the lawyers like lawyers have always been like this. So insider, I'm calling crap on you. DeepMind releases their reinforcement learning lecture series 2021. This is a lecture series about introduction to reinforcement learning by DeepMind researchers at the University College London, and you can in fact watch all of them. They're freely available on YouTube, the slides are available. And it's pretty cool if you want to get into reinforcement learning. It starts out with the simple frameworks, and it ends with deep reinforcement learning. David Hart tweeted the following out a pop up shop in Shibuya will sell clothing with adversarial patches printed on them to make a fashion statement. Now, while I can't understand this exactly, I do think it's pretty cool. So the label or the brand or the store is called camouflage against the machines unlabeled, and the clothing features adversarial patches. Now, whether that will help in any way or form, like I'm quite doubtful, but it is a pretty cool inside joke if you meet other researchers. The next one isn't really machine learning news, but it is quite important. A contributor to pytorch has released a viable solution for Python concurrency. So if you don't know cpython, the reference implementation for the Python language has this problem that in a multi threaded application, in order to keep track of all the objects flying around, it essentially is forced to do this reference counting. And in order to do proper reference counting, it essentially means that every time a reference is incremented or decremented has to lock down all the threads. This is known as the gil, the global interpreter lock. And it is the reason why you can program multi threaded applications in Python, but they will never be able to use the interpreter at the same time, which means that if you have CPU bound applications, multi threading will just not help, it will not speed up your application at all, you need to go to multi processing. So the rule for the longest time has been if your application is IO bound, then you can use multi threading because it's easier to program, it's easier to reason about a shared state and so on. However, if your application is CPU bound, then you have to go to multi processing, which is quite a bit more heavy, more error prone, so on. Many attempts have been made previously to remove the gil, but every single actual implementation of a Python without a gil had the advantage of being able to run multi threaded applications really concurrently, but also the disadvantage that single threaded applications, which most Python programs are single threaded applications would slow down due to these changes. But now this new suggestion by Sam Gross, who as I already said is a major contributor to PyTorch is actually a viable solution and is being evaluated currently, which is pretty cool. And it may be that in the future, Python concurrent programming will get a lot easier. Big Science has released t zero plus plus, which is a model that is a multi task trained text to text model don't even exactly know how I should call this. But essentially, they took t five, and they trained it with a bunch of different NLP tasks that you all frame as a really a text input. So if you don't know what t five is, t five is this concept that when I have an NLP task, rather than encoding it somehow in a smart way, I simply encoded as a natural language prompt. For example, if I want to translate from French to English, I simply say please translate the following from French to English, and then I put the French sentence and then I train the model to auto aggressively predict the English sentence. This means I can use pre trained language models as a start for these models. And namely, that is what GPT three does zero shot out of the box. So the idea here is that if GPT three can do this in a zero shot fashion, these natural language tasks that are formulated in the language of let's say of the input of English, can't we achieve the same or better zero shot performance if we don't pre train the model on language modeling as GPT three is but if we instead pre train the model on other tasks. So t zero is this model that takes a bunch of different NLP tasks puts them all into the language as a human would input them or type them up. So they are compatible with a language model trains all of them at the same time. And it turns out that the resulting model can actually do new NLP tasks in a zero shot fashion much like GPT three but is way more parameter efficient at that. So this is pretty cool. And the model is available on hugging face. So here you see a bunch of examples of what that can look like they have different versions of this model, you can import it in the hugging face API, you can even try it out here on the website. And the thing I want to highlight is that big science isn't some research lab or a company, it's actually a one year long research workshop on large multilingual models and data sets. This is simply a conglomeration of a bunch of researchers from all over the world that is loosely organized together for one year to investigate these large models. So it's pretty cool that something outside of traditional academia or corporate research labs also comes into the game and provides lots of cool stuff for the community. Definitely check it out. Check out their paper, check out their models. Speaking of the hugging face hub, hugging face released this tweet saying that the data set viewer is available in hugging face hub is essentially a preview where you can for any data set go and see what kind of samples are in there, not for any data set, but for any that supports the hugging face streaming API, which are like half the data sets on the hugging face hub, this works for images. So here you can see MNIST and you already saw some NLP things. So pretty cool hugging face hub is getting more and more useful by the day. site is a sort of a Google scholar ish type of thing where you can look for publications and then inside the publications, every citation will be annotated, first of all, with the context of where it goes. So any citation target, if you click on it, you'll see sort of the context of the citation. And second of all, it is annotated with the fact of whether the citation actually supports the cited research or is critical of it or refutes it. So you have positive and negative citations. And this gives you a much more accurate picture of how a particular paper has fared in the research landscape in how it was cited and not only whether it was cited, this is done in part by an automated system. And I believe they already have a giant amount of research articles in there and automating these extraction of references, and they are scoring them using deep learning model. What else there is a paper to go along with it, check it out if you like and give site a try. It isn't exactly free. There are different tiers right here with different features. But if this is at all helpful to you, I guess it might be worth it. Facebook AI releases a blog post called teaching AI to perceive the world through your eyes. This is a push by Facebook or meta or whatever it is called right now to go away from the standard data sets where you have some third person view of a scene to really first person data sets. So they have a bunch of collections of data from around the world from different people in different life circumstances in many, many places, and they collected first person data, meaning I guess these people had head mounted cameras and had other sensors on and they recorded just doing everyday activities. So the data set is called ego 4d. And what I think is cool about it is the data set generation process is different from what other data sets are not only the fact that it is first person and that it is, you know, distributed all over the world and not just done by a single person or team, but also because they just told the people, you know, just record yourself doing everyday stuff. And then after the fact, they went ahead and they defined tasks and they annotated the data for labels. So they didn't have the labels in mind when they collected the data, or maybe they had them in mind, but they didn't collect the data specifically to get some labels first collected the data, and then they put different labels over top. So for example, different tasks that they imagine are memory tasks, forecasting tasks, object recognition, whatnot, they have various layers of labels annotated by humans by crowd workers on this data and the data set, you know, you can imagine that these aren't the only labels. In fact, it is very feasible that a different research group goes ahead and annotates the data in a different way to create their own task. The blog post highlights the difficulty of ego centric data, which is usually vastly different from like a third person view. As you can see here on the left, this object detector works quite well in a third person view. However, in a first person view, it just kind of fails. So is this a good way forward to build more capable systems or a step into dystopia? I guess that's up to you. But if you like working with data like this, then give this data set a try. I'm not exactly sure how you can get ahold of it. I think there is some sort of license attached. But yeah, it's out there. Tesla released apparently pretty randomly a guide to a configurable floating point format and arithmetic. So this is a very technical specification for eight bit and 16 bit floating point numbers and arithmetic and is supposed to sort of standardize or give a format to configurable floating point numbers. So as I said, it's very technical, it's actually also quite short. And the gist here is that they say if you train AI models on really large scales, like Tesla does, you might want to go down from 32 bit numbers to 16 bit numbers or even eight bit numbers. However, in these very low regimes, you only have whatever eight bit to play with. And therefore you can't exactly specify ahead of time how many bits should be the exponent and how many bits the mantissa should be. Therefore, this needs to be configurable. So not like in a 32 bit number, you have exactly this many bits for this and this many bits for that in these new configurable floating point numbers, this would be a variable that you can decide as you use the number. So that allows you to trade off what kind of range this number can potentially have with the accuracy, the resolution that the number can have in a particular range, we'll see whether this remains a thing that's purely used inside of Tesla or whether other people are going to adopt it. Microsoft introduces pytorch direct ml, they say train your machine learning models on any GPU. So this is a component for pytorch that allows you to use any direct x GPU for doing deep learning. And all that is necessary essentially is that in pytorch, you don't say to CUDA, like if you have a CUDA device, now you say to DML to direct ml. And that's it. This works on Windows and on the Windows subsystem for Linux. So if you're still a Windows user for whatever reason, good for you. Alright, more helpful things that I saw this week, there are a lot of helpful things this week. It's not only helpful libraries, it's the section is renamed to just help, like, help me please. pyWIC is a high level batteries included neural network training library for pytorch. And yes, whatever you're thinking is said here at the beginning of the readme, does the world need another pytorch framework? Probably not. But we started this project when no good frameworks were available. And it just kept growing. So here we are. Yeah, respect. Cool. If none of the current frameworks please you, pyWIC might be for you. Lexa is a benchmark for zero shot reaching of goals. This goes along with a paper by CMU, UPenn and U of T about reaching goals after discovering the world. So these agents, what they'll do is they'll essentially go ahead and they'll just try out a bunch of stuff in the world without any explicit goals. And after that, you give the models a picture of a goal to reach, and they're supposed to reach it. So this means you don't explicitly train the agents to reach that particular goal, or any goal, you simply let them explore. And after that, they have to reach a goal. So Lexa is a benchmark that achieves this. And as I said, this goes along with the paper that gives a very, very, very good baseline for this benchmark already. But the benchmark itself is available to download if you're interested in doing that kind of research, give it a try. Next, Donnie Jar Hoffner tweets out excited to introduce crafter. So this is a game sort of an open world game, long term reasoning, exploration, generalization made for reward agents and unsupervised agents. So it's called crafter. And you move around, and there's blocks, and there's food, and you have to dig and you have to build and you have to craft things. I've never seen anything like this before. This is a first. This has no relation to any game that I've seen so far. No, it's, it's pretty cool. So you can craft things, as you can see right here, you can interact with stuff, every world is randomly generated. Like this is a Minecraft clone, but amenable to machine learning research to AI research. So that is pretty cool, because Minecraft just seems too complex, because you can move like in any direction and so on here, it's really discrete. So these models, they have a much more easy time to go about it, they've already evaluated different of these AI learning mechanisms on it, like dreamer, PPO, rainbow agents, and so on. And none of them really compare so far to human expert. But I think the game is pretty cool. It is available. These RL agents can already do things like you know, dig holes, build bridges, and so on. There's a very complex behaviors already emerging here, it moves out of the way of a skeleton. And in another one builds a shelter. Excellent crafter, give it a try. If this video gets more than three likes, we'll do a crafter let's play for sure. Robert Lange releases a lightweight hyper parameter optimization tool. This seems to be a cool kind of personal project by Robert, and he released it with pretty good documentation. There's colab, there is an example. And if you're just looking for like a very simple way to do hyper parameter optimization, then this might be the library for you. As you can see, there's different strategies for doing hyper parameter optimization and different ways you can define them. That's pretty much all you need even has the fancy decorator style as you can see right here. Very pythonic. Sayak Paul released a Keras tutorial on mobile vid. So this is a tutorial that will guide you through implementing mobile visual transformers in Keras, which is quite neat. So Keras still as easy to use as ever. And this tutorial guides you through building the architecture from the ground up all the way to training it. At the end, you convert this model to TF Lite. So it actually runs on your mobile phone. Pretty cool. Omar Sansevierro tweets out this demo is surprising it combines vit with GPT to caption images with great results. And yes, actually, I was positively surprised. This is a hugging face module where you take a existing text model like GPT two and you take an existing image computer vision model like vision transformer and you combine them. So first, you start out with sort of random cross attention weights that you fine tune just a little bit. And that can have really, really good results. Essentially, the model learns how to connect the latent representation from one model to the other model and back. So this is used right here to do an image captioning demo using GPT two and vit, as I said, and training only about 7000 steps on the cocoa data set. So you can see the result. This is a man swinging a tennis racket on a tennis court that is very descriptive. That that is just an unhumanly precise description of what's going on right here. We have a blue and white street sign sitting on top of a pole. Yes, that that is also a very, very, very precise description. person riding a skateboard on top of a cement floor. Well, I guess that has some importance. Is it just me or AI models just bureaucrats? But yeah, pretty cool. Bits and bytes is a library by Facebook research for eight bit optimizers and quantization routines. So they have a bunch of optimizers such as Adam, Adam W, RMS prop, and so on that work on eight bits instead of 32. And that pretty reliably saves you 75% of the memory, something like Adam has two or three different buffers that for every parameter you need to keep track of. So this can pretty quickly get pretty large and saving three quarters of the memory has definitely value. Now I love that it's called Facebook research. But if you hover it says meta research. Is this gonna go well? I don't know. Also, is this supposed to be like a pretzel? Like it is, is it supposed to be like a flat logo? Or is it supposed to represent sort of like a Pringles chips, you know, like the saddle in 3d? I don't know. Another helpful thing user friendly introduction to pack base bounce by Pierre O'Curr. Now this is something I have no clue about. But I know it's important. And I have learned it at some point, if you're trying to get into pack base bounce, this is a I believe over 60 pages introduction to it that seems to be quite well written, introducing you to all the important concepts in it. So if you're interested, give it a try. Again, face meta, whatever research releases x formers, hackable and optimized transformers building blocks supporting a composable construction. So if you're into transformers, and if you would like to recombine them, try out different things inside of them, x formers might be a great library on doing that. So you see all of these boxes here, essentially, this library makes it pretty easy to just rearrange them, connect them differently, and so on. Superb is a speech processing universal performance benchmark. This means that this benchmark has a bunch of speech tasks, so tasks in machine learning where the input is a piece of speech. But the goal here is that you have one pipeline that generates a representation. And then that representation can be fine tuned easily to all of these tasks. You're not supposed to solve all of the tasks from scratch, you're supposed to come up with that pipeline that generates the representation. If you work on speech, this might be very cool for you. I don't know how to say this. CCQA is a web scale question answering data set for model pre training. This is a large scale QA data set that I guess you can use for pre training question answering models. Excellent. Bagua is a library that claims to speed up pytorch. So they have a bunch of things in here for pytorch, for example, advanced distributed training algorithms, performance, auto tuning, generic fused optimizers, load balanced data loader, and so on. So these seem to be specialized algorithms that in very certain cases where you want to use pytorch can potentially deliver a lot of speed up. So if your problem doesn't fall into like the standard bucket where the library is optimized for maybe you can find something inside of Bagua that is going to help you Bagua Bagua. I don't know. tree x tracks treaks is a pi tree module system for deep learning in JAX. So if you work with pi tree, this is it in JAX. Good job pi tree. For those of you don't know are essentially trees out of Python structures. So here, for example, a list which contains numbers and dicts which themselves contain tuples and so on. So a pi tree works with these kinds of objects, and now you can use them inside of JAX and tree x helps you to do that in a more module oriented way or a more object oriented way. Reuters writes AI can see through you CEOs language under machine microscope. This article essentially says that things like NLP and speech sound analysis, they now go after CEOs quarterly announcements, they analyze their voices, and they're trying to just recognize when they're nervous and so on. And they actually have a point in that they claim they can make better investment decisions if they do something like this. But as you know, as soon as you pay attention to anything like this, the CEOs are immediately going to adjust and train to trick essentially these AI systems. So they will use scripted speeches much more in order to not trip the NLP systems, they will train their voice acting more, I guess, or let some press secretary speak for them. All in all, it just seems to be like, you know, if you analyze a CEO's speech, and to to detect when they're lying, and when not, and then make investment decisions, you'll simply reinforce the like the sociopaths that have no problem with just straight out lying that have no difference in their voice whatsoever. So if you want to create a world of more sociopathic CEOs than it already is, I guess, then go right ahead, just just do this, this this is fine. Excellent. Atbury, the company has apparently made this ad for Indian local businesses. And it's not just an ad, but they've paid this Indian celebrity to record essentially one ad, and then they modified that ad using deep learning. So they have like three product categories like shoes, and I guess glasses and watches or something like this, they've recorded the different ads for the different products. But whenever the actor says the company name and the location of the company, they use deep learning to change whatever the small business is. So essentially, this is a deep faith from the same actor to his own face, but to make him say something else. So as a small business in India, you can go there and get your ad for your local business, the system will actually make sure that people that are in your area are advertised with your particular business and people in different areas will see I guess the same ad but the actor mentioning a different business that is in that area. Pretty cool. There's a form if you're in India, you know, check it out. And lastly, this shoe does not exist. This is a website, I guess it's analogous to this person does not exist, which was a famous website that trained stylegan two on a face data set. So this is stylegan three, which was recently released the alias freegan, and it's trained on a shoe data set. So you can just refresh and look at shoes that the model has come up with. I guess these shoes all look like they exist, they might as well, who knows. But yeah, if you're looking for unique design ideas, check it out. I'm looking forward to many more things where stylegan three is applied, it seems to be the quality of these models and the ease of training them has come a long way such that it is in fact possible to do this for many types of things where you have decently amounts of data such as choose, I guess. Alright, this was it for this week's ML news. Thank you so much for being here. Don't forget to like and subscribe and let me know what you think in the comments. I value your opinions. Definitely. This is not just a trick to get the YouTube algorithm to promote the video more and all of that kind of stuff. See ya.
 Google introduces pathways their next generation AI architecture, open AI solves high school math
 problems. And Facebook goes all on first person view. Welcome to ML news.
 But before the video starts, a quick thanks to our sponsor weights and biases, I want to show you
 this one feature that I just learned about, did you know you can embed a weights and biases report
 in notion, it's actually not only reports, but also other stuff by weights and biases. So they
 have this neat little page here, ironically, it is actually a notion and it is super easy to embed
 live weights and biases stuff into notion. So for example, here I have a sweep and you can see the
 sweep is interactive. So you can do all the kinds of things you're used to analyzing a weights and
 biases sweep. Now I can just grab that URL, get over to notion and create a new embed, paste a link.
 And there we go. Look at that. This is a fully functional weights and biases report inside of
 notion. So you have all the interactivity here that you would usually have as you can see. So
 I can look at my runs, I can activate them, I can even go and look at my sweep controls and various
 things. This is really cool if you work together with other people and you work on more than just
 weights and biases reports, you can take your notes and notion and then embed the report, the sweep,
 whatever into notion page. I love notion, I love weights and biases. And it's very cool that to go
 together. If you don't know weights and biases, it is your one stop shop for all your machine
 learning experimental needs from trying out models optimizing hyper parameters all the way to
 saving your models, deploying them and so on. It runs in the cloud, it's free for personal users
 and for education, there are plans for teams and for self hosted setups. So all the more reason to
 go try it out. Thanks again to weights and biases for sponsoring this video. And now let's get into
 it. Bye bye. Hello and welcome to ML news. Let's dive into our first story. Jeff Dean has released
 a blog post on the Google blog. No, this is not the Google AI blog. This is the main Google blog.
 He's also given a TED talk about the subject and the subject is this model called pathways,
 a next generation AI architecture, we don't actually know much about this architecture,
 because all we have is that TED talk and this illustration right here. And essentially,
 Jeff Dean imagines Google's future AI projects to rely on this new architecture, where instead of
 having single task neural networks that train, you have this giant multitask neural network that can
 do all the tasks at once. And that would also be sparsely activated. As you can see here, different
 tasks would leverage different paths through this network. This goes along with a few criticisms on
 today's architectures. So he says, for example, today's AI models are typically trained to do
 only one thing, pathways will enable us to train a single model to do 1000s or millions of things.
 So the goal is to have one model do many, many tasks at once. Second, he says today's models
 mostly focus on one sense pathways will enable multiple senses. This refers to the fact that the
 input to current neural networks are single modalities. Sometimes they're two modalities,
 but mostly they're single modalities, like images or text or sound this pathway architecture,
 naturally being multitask will also be multimodal, which means that it could input any sort of
 modality in his TED talk, he gives the example, whether or not you see a leopard or hear the word
 leopard or hear someone say the word leopard or see a video of a leopard that should essentially
 evoke the same concept in your brain and therefore also in the pathway model. And lastly, he says
 today's models are dense and inefficient pathways will make them sparse and efficient. This refers
 to the fact that our current networks are densely activated, everything's connected to everything.
 And that's very, very inefficient. He imagines this future pathways architecture to be sparsely
 activated, meaning that only very small subparts of the network will be activated for a given input
 sample. And therefore the different parts of the network doing different things, they don't always
 have to be active at the same time. This can also make the model much more efficient in terms of
 parameters and computation. Now, as I said, there's not a paper to go along with this or an
 implementation or even a plan of how to get there. This is essentially a wishlist, and it's not a
 particularly new wishlist. Like people have dreamed of, oh, can't we just make multi modal multi task
 models where one model learns everything? Well, yeah, everyone wishes that. But you still have
 the problems, namely, for example, catastrophic forgetting, if you try to teach the model many
 tasks, and then one task more, you still have to ensure that it doesn't forget the old tasks,
 which is very, very difficult, especially in this picture, it seems like this is a rather
 feed forward architecture right here without any sort of memory modules or anything like this.
 So how they're going to achieve that, I don't know. Secondly, they say there are many different
 tasks here. However, huge data architectures mostly rely on self supervision, and then fine
 tuning for individual tasks and not having different tasks in parallel, though multi task
 training is a thing. And lastly, the sparse activations are not trivial to achieve. Again,
 people have been saying this forever, like, well, can't we just have a sparse neural network,
 probably the brain is sparse, blah, blah, blah. But how are you going to get there? This is just
 a wishlist, how we're going to get there. I don't know. The main problem with sparsity being that
 if you have a sparse forward signal, then your backwards gradients are also going to be sparse,
 you may never learn the correct sparse way through your network. If you only activate sparsely in the
 forward pass. These are all challenges that have existed forever. But it seems like Google is
 determined to solve these challenges. I mean, if they can, all the better. But for now, it's just
 a plan and an idea. And I'm excited to see what happens. Open as released a blog post called
 solving math word problems where they train a language model to solve math problems. This
 goes along with a paper saying training verifiers to solve math word problems by people at Open AI,
 you can read it if you want. Essentially, it is a data set of about 8000 of these high school math
 problems, where you mainly need the basic addition, subtraction, multiplication and division
 in order to solve the problem. They're usually stated as little stories, and they have some sort
 of an answer. Now, large language models such as GPT three are usually kind of bad at this type
 of stuff, mainly because they are not accurate enough, they don't do these simple steps that
 are required enough, they're more like a language model, they're more like a conversation model,
 or a thing that simply repeats some of the stuff it has already seen. So the first approach the
 paper takes is to fine tune such a language model on these tasks. And it turns out that doesn't go
 too well. Very often that makes a lot of mistakes as well. And the solution comes in the form of
 what they call verifiers. So verifiers are model that are not trained to produce the solution,
 but they are trained to rate whether a solution to a problem is likely to be the correct solution
 or not. So now what they do is they use one model that they fine tuned to produce like 100 solutions,
 and then they use the verifiers to rank the solution and pick the best one. And that turns
 out to be very, very powerful. So we've seen approaches like this before, you remember the
 Dali model of open AI also not only used a generative model for the avocado chair, but it
 also used the clip model in order to rank the outputs of the generative model. So this could
 be a more general recipe for improving generative models is train verifiers, and then generate a
 bunch of solutions and rank them with the verifiers. As I said, you can read the paper and
 the data set of these math questions is available to download. Sam Altman tweeted neural networks
 really truly learn it's not a fancy trick. This is one of the most remarkable things humans have ever
 figured out and the implications are difficult to overstate. Now I'm not sure if he just wanted to
 start like a fire with this kind of things. There are many ways of going about this, but it seems
 like the truth or veracity of the statement entirely depends on how you define learning.
 But it seems like Sam Altman and in general, that's what we see out of open AI is of the
 opinion that learning that humans do isn't that much different from the learning that current
 large scale neural networks inherently do. Now this is to be set a little bit into contrast with
 what people from the more symbolicist camp may think about these neural networks and about the
 nature of learning and intelligence in general. But again, I guess it only depends on the definition
 of words here. And just putting the modifiers really and truly in front of a non defined word
 doesn't suddenly make it defined. But what do you think? Let me know in the comments after you hit
 the subscribe button. See what I did there. Next news business insider writes Google's AI researchers
 say their output is being slowed by lawyers after a string of high level exits getting published
 really is a nightmare right now. So the article starts off with a bunch of Google controversies,
 obviously, some famous people were fired from Google recently, and there were a bunch of
 scandals around that. And now one senior AI researcher who spoke with insider on the
 condition of anonymity comes forward and says, well, the lawyers are essentially up our necks
 right now. It's so difficult to publish, this is really stifling publishing inside of Google and
 so on. And the article backs this up by saying, according to Google's online records, the company
 published 925 pieces of AI research in 2019 and 962 in 2020. But the company looks to have experienced
 a moderate slowdown this year, publishing just 618 research papers in 2021. Thus far. Now this is the
 only thing where they actually back anything up that they say now I've no doubt that this is the
 case inside of these big companies, they give examples whenever they write words such as bias
 or fairness, then the lawyers they would just have like tons of questions or want to cross them out
 because they just don't understand the technical terms behind these things. Now noteworthy terms
 like bias and fairness actually have about 60 technical definitions, and they're all in
 conflict with each other. So can't exactly fault the lawyers. What I found funny is that in the
 last section here, a spokesperson from Google took a statement and said we're publishing papers at
 the same rate we did last year. At this time last year, there were 815 approved papers and this year
 there are 820 so far the spokesperson said adding our website doesn't reflect all papers and is
 typically updated a few months after publications. So they had to bury this on the very bottom of the
 article right here because they want to like tell a story about how lawyers are so terrible and
 about how this exit stifled Google so much and don't get me wrong, lawyers are terrible. And I'm
 pretty sure that there are pain in the neck. But the claim that this is especially ramped up now
 doesn't seem to hold apart from like one or two anonymous people inside of Google coming forward.
 And the fact that they have to hide this thing at the very bottom, which is pretty clear, like that's
 a much more likely explanation than Google now suddenly ramping up their eyeballs of the lawyers
 like lawyers have always been like this. So insider, I'm calling crap on you. DeepMind releases
 their reinforcement learning lecture series 2021. This is a lecture series about introduction to
 reinforcement learning by DeepMind researchers at the University College London, and you can
 in fact watch all of them. They're freely available on YouTube, the slides are available.
 And it's pretty cool if you want to get into reinforcement learning. It starts out with the
 simple frameworks, and it ends with deep reinforcement learning. David Hart tweeted the
 following out a pop up shop in Shibuya will sell clothing with adversarial patches printed on them
 to make a fashion statement. Now, while I can't understand this exactly, I do think it's pretty
 cool. So the label or the brand or the store is called camouflage against the machines unlabeled,
 and the clothing features adversarial patches. Now, whether that will help in any way or form,
 like I'm quite doubtful, but it is a pretty cool inside joke if you meet other researchers.
 The next one isn't really machine learning news, but it is quite important. A contributor to
 pytorch has released a viable solution for Python concurrency. So if you don't know cpython, the
 reference implementation for the Python language has this problem that in a multi threaded
 application, in order to keep track of all the objects flying around, it essentially is forced
 to do this reference counting. And in order to do proper reference counting, it essentially means
 that every time a reference is incremented or decremented has to lock down all the threads.
 This is known as the gil, the global interpreter lock. And it is the reason why you can program
 multi threaded applications in Python, but they will never be able to use the interpreter at the
 same time, which means that if you have CPU bound applications, multi threading will just not help,
 it will not speed up your application at all, you need to go to multi processing. So the rule for
 the longest time has been if your application is IO bound, then you can use multi threading because
 it's easier to program, it's easier to reason about a shared state and so on. However, if your
 application is CPU bound, then you have to go to multi processing, which is quite a bit more heavy,
 more error prone, so on. Many attempts have been made previously to remove the gil, but every single
 actual implementation of a Python without a gil had the advantage of being able to run multi
 threaded applications really concurrently, but also the disadvantage that single threaded applications,
 which most Python programs are single threaded applications would slow down due to these changes.
 But now this new suggestion by Sam Gross, who as I already said is a major contributor to PyTorch
 is actually a viable solution and is being evaluated currently, which is pretty cool.
 And it may be that in the future, Python concurrent programming will get a lot easier.
 Big Science has released t zero plus plus, which is a model that is a multi task trained text to
 text model don't even exactly know how I should call this. But essentially, they took t five,
 and they trained it with a bunch of different NLP tasks that you all frame as a really a text input.
 So if you don't know what t five is, t five is this concept that when I have an NLP task,
 rather than encoding it somehow in a smart way, I simply encoded as a natural language prompt. For
 example, if I want to translate from French to English, I simply say please translate the
 following from French to English, and then I put the French sentence and then I train the model to
 auto aggressively predict the English sentence. This means I can use pre trained language models
 as a start for these models. And namely, that is what GPT three does zero shot out of the box. So
 the idea here is that if GPT three can do this in a zero shot fashion, these natural language tasks
 that are formulated in the language of let's say of the input of English, can't we achieve the same
 or better zero shot performance if we don't pre train the model on language modeling as GPT three
 is but if we instead pre train the model on other tasks. So t zero is this model that takes a bunch
 of different NLP tasks puts them all into the language as a human would input them or type them
 up. So they are compatible with a language model trains all of them at the same time. And it turns
 out that the resulting model can actually do new NLP tasks in a zero shot fashion much like GPT three
 but is way more parameter efficient at that. So this is pretty cool. And the model is available
 on hugging face. So here you see a bunch of examples of what that can look like they have
 different versions of this model, you can import it in the hugging face API, you can even try it out
 here on the website. And the thing I want to highlight is that big science isn't some research
 lab or a company, it's actually a one year long research workshop on large multilingual models and
 data sets. This is simply a conglomeration of a bunch of researchers from all over the world that
 is loosely organized together for one year to investigate these large models. So it's pretty
 cool that something outside of traditional academia or corporate research labs also comes into the game
 and provides lots of cool stuff for the community. Definitely check it out. Check out their paper,
 check out their models. Speaking of the hugging face hub, hugging face released this tweet saying
 that the data set viewer is available in hugging face hub is essentially a preview where you can
 for any data set go and see what kind of samples are in there, not for any data set, but for any
 that supports the hugging face streaming API, which are like half the data sets on the hugging
 face hub, this works for images. So here you can see MNIST and you already saw some NLP things. So
 pretty cool hugging face hub is getting more and more useful by the day. site is a sort of a Google
 scholar ish type of thing where you can look for publications and then inside the publications,
 every citation will be annotated, first of all, with the context of where it goes. So any citation
 target, if you click on it, you'll see sort of the context of the citation. And second of all,
 it is annotated with the fact of whether the citation actually supports the cited research
 or is critical of it or refutes it. So you have positive and negative citations. And this gives
 you a much more accurate picture of how a particular paper has fared in the research
 landscape in how it was cited and not only whether it was cited, this is done in part by an automated
 system. And I believe they already have a giant amount of research articles in there and automating
 these extraction of references, and they are scoring them using deep learning model. What else
 there is a paper to go along with it, check it out if you like and give site a try. It isn't exactly
 free. There are different tiers right here with different features. But if this is at all helpful
 to you, I guess it might be worth it. Facebook AI releases a blog post called teaching AI to perceive
 the world through your eyes. This is a push by Facebook or meta or whatever it is called right
 now to go away from the standard data sets where you have some third person view of a scene to
 really first person data sets. So they have a bunch of collections of data from around the world from
 different people in different life circumstances in many, many places, and they collected first
 person data, meaning I guess these people had head mounted cameras and had other sensors on and they
 recorded just doing everyday activities. So the data set is called ego 4d. And what I think is
 cool about it is the data set generation process is different from what other data sets are not
 only the fact that it is first person and that it is, you know, distributed all over the world and
 not just done by a single person or team, but also because they just told the people, you know,
 just record yourself doing everyday stuff. And then after the fact, they went ahead and they defined
 tasks and they annotated the data for labels. So they didn't have the labels in mind when they
 collected the data, or maybe they had them in mind, but they didn't collect the data specifically to
 get some labels first collected the data, and then they put different labels over top. So for example,
 different tasks that they imagine are memory tasks, forecasting tasks, object recognition,
 whatnot, they have various layers of labels annotated by humans by crowd workers on this
 data and the data set, you know, you can imagine that these aren't the only labels. In fact,
 it is very feasible that a different research group goes ahead and annotates the data in a
 different way to create their own task. The blog post highlights the difficulty of ego centric data,
 which is usually vastly different from like a third person view. As you can see here on the left,
 this object detector works quite well in a third person view. However, in a first person view,
 it just kind of fails. So is this a good way forward to build more capable systems or a step
 into dystopia? I guess that's up to you. But if you like working with data like this, then give
 this data set a try. I'm not exactly sure how you can get ahold of it. I think there is some sort of
 license attached. But yeah, it's out there. Tesla released apparently pretty randomly a guide to a
 configurable floating point format and arithmetic. So this is a very technical specification for
 eight bit and 16 bit floating point numbers and arithmetic and is supposed to sort of standardize
 or give a format to configurable floating point numbers. So as I said, it's very technical,
 it's actually also quite short. And the gist here is that they say if you train AI models on
 really large scales, like Tesla does, you might want to go down from 32 bit numbers to 16 bit
 numbers or even eight bit numbers. However, in these very low regimes, you only have whatever
 eight bit to play with. And therefore you can't exactly specify ahead of time how many bits should
 be the exponent and how many bits the mantissa should be. Therefore, this needs to be configurable.
 So not like in a 32 bit number, you have exactly this many bits for this and this many bits for
 that in these new configurable floating point numbers, this would be a variable that you can
 decide as you use the number. So that allows you to trade off what kind of range this number can
 potentially have with the accuracy, the resolution that the number can have in a particular range,
 we'll see whether this remains a thing that's purely used inside of Tesla or whether other
 people are going to adopt it. Microsoft introduces pytorch direct ml, they say train your machine
 learning models on any GPU. So this is a component for pytorch that allows you to use any direct x
 GPU for doing deep learning. And all that is necessary essentially is that in pytorch, you
 don't say to CUDA, like if you have a CUDA device, now you say to DML to direct ml. And that's it.
 This works on Windows and on the Windows subsystem for Linux. So if you're still a
 Windows user for whatever reason, good for you. Alright, more helpful things that I saw this week,
 there are a lot of helpful things this week. It's not only helpful libraries, it's the section is
 renamed to just help, like, help me please. pyWIC is a high level batteries included neural network
 training library for pytorch. And yes, whatever you're thinking is said here at the beginning of
 the readme, does the world need another pytorch framework? Probably not. But we started this
 project when no good frameworks were available. And it just kept growing. So here we are. Yeah,
 respect. Cool. If none of the current frameworks please you, pyWIC might be for you. Lexa is a
 benchmark for zero shot reaching of goals. This goes along with a paper by CMU, UPenn and U of T
 about reaching goals after discovering the world. So these agents, what they'll do is they'll essentially
 go ahead and they'll just try out a bunch of stuff in the world without any explicit goals. And after
 that, you give the models a picture of a goal to reach, and they're supposed to reach it. So this
 means you don't explicitly train the agents to reach that particular goal, or any goal, you simply
 let them explore. And after that, they have to reach a goal. So Lexa is a benchmark that achieves
 this. And as I said, this goes along with the paper that gives a very, very, very good baseline
 for this benchmark already. But the benchmark itself is available to download if you're interested
 in doing that kind of research, give it a try. Next, Donnie Jar Hoffner tweets out excited to
 introduce crafter. So this is a game sort of an open world game, long term reasoning, exploration,
 generalization made for reward agents and unsupervised agents. So it's called crafter. And
 you move around, and there's blocks, and there's food, and you have to dig and you have to build
 and you have to craft things. I've never seen anything like this before. This is a first.
 This has no relation to any game that I've seen so far. No, it's, it's pretty cool. So you can craft
 things, as you can see right here, you can interact with stuff, every world is randomly generated.
 Like this is a Minecraft clone, but amenable to machine learning research to AI research. So
 that is pretty cool, because Minecraft just seems too complex, because you can move like in any
 direction and so on here, it's really discrete. So these models, they have a much more easy time
 to go about it, they've already evaluated different of these AI learning mechanisms on it, like
 dreamer, PPO, rainbow agents, and so on. And none of them really compare so far to human expert. But
 I think the game is pretty cool. It is available. These RL agents can already do things like you
 know, dig holes, build bridges, and so on. There's a very complex behaviors already emerging here,
 it moves out of the way of a skeleton. And in another one builds a shelter. Excellent crafter,
 give it a try. If this video gets more than three likes, we'll do a crafter let's play for sure.
 Robert Lange releases a lightweight hyper parameter optimization tool. This seems to be a cool kind of
 personal project by Robert, and he released it with pretty good documentation. There's colab,
 there is an example. And if you're just looking for like a very simple way to do hyper parameter
 optimization, then this might be the library for you. As you can see, there's different strategies
 for doing hyper parameter optimization and different ways you can define them. That's pretty
 much all you need even has the fancy decorator style as you can see right here. Very pythonic.
 Sayak Paul released a Keras tutorial on mobile vid. So this is a tutorial that will guide you
 through implementing mobile visual transformers in Keras, which is quite neat. So Keras still as
 easy to use as ever. And this tutorial guides you through building the architecture from the ground
 up all the way to training it. At the end, you convert this model to TF Lite. So it actually
 runs on your mobile phone. Pretty cool. Omar Sansevierro tweets out this demo is surprising
 it combines vit with GPT to caption images with great results. And yes, actually, I was positively
 surprised. This is a hugging face module where you take a existing text model like GPT two and you
 take an existing image computer vision model like vision transformer and you combine them. So first,
 you start out with sort of random cross attention weights that you fine tune just a little bit. And
 that can have really, really good results. Essentially, the model learns how to connect
 the latent representation from one model to the other model and back. So this is used right here
 to do an image captioning demo using GPT two and vit, as I said, and training only about 7000 steps
 on the cocoa data set. So you can see the result. This is a man swinging a tennis racket on a tennis
 court that is very descriptive. That that is just an unhumanly precise description of what's going
 on right here. We have a blue and white street sign sitting on top of a pole. Yes, that that is
 also a very, very, very precise description. person riding a skateboard on top of a cement floor. Well,
 I guess that has some importance. Is it just me or AI models just bureaucrats? But yeah, pretty cool.
 Bits and bytes is a library by Facebook research for eight bit optimizers and quantization routines.
 So they have a bunch of optimizers such as Adam, Adam W, RMS prop, and so on that work on eight
 bits instead of 32. And that pretty reliably saves you 75% of the memory, something like Adam has
 two or three different buffers that for every parameter you need to keep track of. So this can
 pretty quickly get pretty large and saving three quarters of the memory has definitely value. Now
 I love that it's called Facebook research. But if you hover it says meta research.
 Is this gonna go well? I don't know. Also, is this supposed to be like a pretzel? Like it is,
 is it supposed to be like a flat logo? Or is it supposed to represent sort of like a Pringles
 chips, you know, like the saddle in 3d? I don't know. Another helpful thing user friendly
 introduction to pack base bounce by Pierre O'Curr. Now this is something I have no clue about. But I
 know it's important. And I have learned it at some point, if you're trying to get into pack base
 bounce, this is a I believe over 60 pages introduction to it that seems to be quite well written,
 introducing you to all the important concepts in it. So if you're interested, give it a try. Again,
 face meta, whatever research releases x formers, hackable and optimized transformers building blocks
 supporting a composable construction. So if you're into transformers, and if you would like to
 recombine them, try out different things inside of them, x formers might be a great library on
 doing that. So you see all of these boxes here, essentially, this library makes it pretty easy to
 just rearrange them, connect them differently, and so on. Superb is a speech processing universal
 performance benchmark. This means that this benchmark has a bunch of speech tasks, so tasks
 in machine learning where the input is a piece of speech. But the goal here is that you have one
 pipeline that generates a representation. And then that representation can be fine tuned easily to
 all of these tasks. You're not supposed to solve all of the tasks from scratch, you're supposed to
 come up with that pipeline that generates the representation. If you work on speech, this might
 be very cool for you. I don't know how to say this. CCQA is a web scale question answering data set
 for model pre training. This is a large scale QA data set that I guess you can use for pre training
 question answering models. Excellent. Bagua is a library that claims to speed up pytorch. So they
 have a bunch of things in here for pytorch, for example, advanced distributed training algorithms,
 performance, auto tuning, generic fused optimizers, load balanced data loader, and so on. So these
 seem to be specialized algorithms that in very certain cases where you want to use pytorch can
 potentially deliver a lot of speed up. So if your problem doesn't fall into like the standard bucket
 where the library is optimized for maybe you can find something inside of Bagua that is going to
 help you Bagua Bagua. I don't know. tree x tracks treaks is a pi tree module system for deep learning
 in JAX. So if you work with pi tree, this is it in JAX. Good job pi tree. For those of you don't know
 are essentially trees out of Python structures. So here, for example, a list which contains numbers
 and dicts which themselves contain tuples and so on. So a pi tree works with these kinds of objects,
 and now you can use them inside of JAX and tree x helps you to do that in a more module oriented
 way or a more object oriented way. Reuters writes AI can see through you CEOs language under machine
 microscope. This article essentially says that things like NLP and speech sound analysis, they
 now go after CEOs quarterly announcements, they analyze their voices, and they're trying to just
 recognize when they're nervous and so on. And they actually have a point in that they claim they can
 make better investment decisions if they do something like this. But as you know, as soon
 as you pay attention to anything like this, the CEOs are immediately going to adjust and train to
 trick essentially these AI systems. So they will use scripted speeches much more in order to not
 trip the NLP systems, they will train their voice acting more, I guess, or let some press secretary
 speak for them. All in all, it just seems to be like, you know, if you analyze a CEO's speech,
 and to to detect when they're lying, and when not, and then make investment decisions, you'll
 simply reinforce the like the sociopaths that have no problem with just straight out lying that have
 no difference in their voice whatsoever. So if you want to create a world of more sociopathic
 CEOs than it already is, I guess, then go right ahead, just just do this, this this is fine.
 Excellent. Atbury, the company has apparently made this ad for Indian local businesses. And it's not
 just an ad, but they've paid this Indian celebrity to record essentially one ad, and then they
 modified that ad using deep learning. So they have like three product categories like shoes,
 and I guess glasses and watches or something like this, they've recorded the different ads for the
 different products. But whenever the actor says the company name and the location of the company,
 they use deep learning to change whatever the small business is. So essentially, this is a deep
 faith from the same actor to his own face, but to make him say something else. So as a small business
 in India, you can go there and get your ad for your local business, the system will actually make
 sure that people that are in your area are advertised with your particular business and
 people in different areas will see I guess the same ad but the actor mentioning a different
 business that is in that area. Pretty cool. There's a form if you're in India, you know,
 check it out. And lastly, this shoe does not exist. This is a website, I guess it's analogous
 to this person does not exist, which was a famous website that trained stylegan two on a face data
 set. So this is stylegan three, which was recently released the alias freegan, and it's trained on a
 shoe data set. So you can just refresh and look at shoes that the model has come up with. I guess
 these shoes all look like they exist, they might as well, who knows. But yeah, if you're looking
 for unique design ideas, check it out. I'm looking forward to many more things where stylegan three
 is applied, it seems to be the quality of these models and the ease of training them has come
 a long way such that it is in fact possible to do this for many types of things where you have
 decently amounts of data such as choose, I guess. Alright, this was it for this week's ML news.
 Thank you so much for being here. Don't forget to like and subscribe and let me know what you think
 in the comments. I value your opinions. Definitely. This is not just a trick to get the YouTube
 algorithm to promote the video more and all of that kind of stuff. See ya.