[ML News] Uber: Deep Learning for ETA | MuZero Video Compression  | Block-NeRF | EfficientNet-X
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper mlnews uber uber eta uber deep learning deepmind muzero muzero video compression muzero explained machine learning tutorial tech news machine learning news block nerf blocknerf learned soft prompts gpt-3 gpt 3 prompt engineering lenia self-organizing agents cellular automata tensorflow know your data kilcher news
#mlnews #muzero #nerf

Your regularly irregular updates on everything new in the ML world!
Merch: http://store.ykilcher.com

OUTLINE:
0:00 - Intro
0:15 - Sponsor: Weights & Biases
2:15 - Uber switches from XGBoost to Deep Learning for ETA prediction
5:45 - MuZero advances video compression
10:10 - Learned Soft Prompts can steer large language models
12:45 - Block-NeRF captures entire city blocks
14:15 - Neural Architecture Search considers underlying hardware
16:50 - Mega-Blog on Self-Organizing Agents
18:40 - Know Your Data (for Tensorflow Datasets)
20:30 - Helpful Things

Sponsor: Weights & Biases
https://wandb.me/yannic

References:
https://docs.wandb.ai/guides/integrations/other/openai
https://colab.research.google.com/github/wandb/examples/blob/master/colabs/openai/Fine_tune_GPT_3_with_Weights_%26_Biases.ipynb#scrollTo=rJdQqrC8Ablo
https://wandb.ai/borisd13/GPT-3/reports/Fine-Tuning-Tips-and-Exploration-on-OpenAI-s-GPT-3---VmlldzoxNDYwODA2

Uber switches from XGBoost to Deep Learning for ETA prediction
https://eng.uber.com/deepeta-how-uber-predicts-arrival-times/?utm_source=pocket_mylist

MuZero advances video compression
https://deepmind.com/blog/article/MuZeros-first-step-from-research-into-the-real-world
https://storage.googleapis.com/deepmind-media/MuZero/MuZero%20with%20self-competition.pdf

Learned Soft Prompts can steer large language models
https://ai.googleblog.com/2022/02/guiding-frozen-language-models-with.html
https://aclanthology.org/2021.emnlp-main.243/

Block-NeRF captures entire city blocks
https://arxiv.org/abs/2202.05263
https://arxiv.org/pdf/2202.05263.pdf
https://waymo.com/intl/zh-cn/research/block-nerf/

Neural Architecture Search considers underlying hardware
https://ai.googleblog.com/2022/02/unlocking-full-potential-of-datacenter.html
https://openaccess.thecvf.com/content/CVPR2021/papers/Li_Searching_for_Fast_Model_Families_on_Datacenter_Accelerators_CVPR_2021_paper.pdf

Mega-Blog on Self-Organizing Agents
https://developmentalsystems.org/sensorimotor-lenia/
https://flowers.inria.fr/

Know Your Data (for Tensorflow Datasets)
https://knowyourdata-tfds.withgoogle.com/#dataset=pass&filters=kyd%2Fcloud_vision%2Fface_probability:9&tab=RELATIONS&item=train%5B89%25%3A91%25%5D_27143&expanded_groups=cloud_vision
https://knowyourdata.withgoogle.com/

Helpful Things
https://twitter.com/casualganpapers/status/1490318575873241091
https://www.reddit.com/r/MachineLearning/comments/snmtzn/r_phd_thesis_on_neural_differential_equations/
https://arxiv.org/abs/2202.02435
https://github.com/vicariousinc/PGMax
https://www.vicarious.com/posts/pgmax-factor-graphs-for-discrete-probabilistic-graphical-models-and-loopy-belief-propagation-in-jax/?utm_content=197542312&utm_medium=social&utm_source=twitter&hss_channel=tw-204185426
https://diambra.ai/tournaments
https://github.com/diambra/diambraArena
https://www.youtube.com/watch?v=dw72POyqcqk&t=271s
https://gitlab.com/deepcypher/python-fhez
https://python-fhez.readthedocs.io/en/latest/
https://joss.theoj.org/papers/10.21105/joss.04101?s=09&utm_source=pocket_mylist
https://github.com/PyTorchLightning/metrics
https://torchmetrics.readthedocs.io/en/latest/
https://twitter.com/alanyttian/status/1492027524909449221?utm_source=pocket_mylist
https://github.com/google/evojax
https://arxiv.org/abs/2202.05008
https://www.reddit.com/r/MachineLearning/comments/snod8f/n_gym_now_has_a_documentation_website/?utm_source=dlvr.it&utm_medium=twitter
https://www.gymlibrary.ml/pages/api/#initializing-environments

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Uber now uses deep learning to predict arrival times. Mew Zero is used to compress YouTube videos and Nerve scales to entire city blocks. Amazing. Welcome to ML News. Hey, ho there. This video is sponsored by Weights and Biases. Today, I want to tell you about a new feature in Weights and Biases, which is their integration with the OpenAI API. If you didn't know, OpenAI has the ability that you can provide your data and they fine tune a GPT-3 model for you. Now, this is pretty cool in itself because you get your own little custom endpoint that you can call has been trained on your data. But now you can sync those training runs to your Weights and Biases account. All you need to do for this to happen is to simply call the sync command on the command line and all your training runs will be synced to Weights and Biases. They have a little demo collab where they demonstrate that you can actually use the artifacts and tables features from Weights and Biases. Essentially, anything that you know, you can construct your data sets, you can have them as artifacts, you can look at them in the tables, then you can ship them to OpenAI to do a fine tuning run. And then you can analyze that fine tuning run and the outputs of it again in Weights and Biases. They even have a little demo report where they do something like this. They upload a Wikipedia data set, they analyze it first using tables, then they analyze the loss from the fine tuning results. They do a little bit of a hyper parameter search and you can analyze those in these nice parallel coordinate plots fully interactively. And in the end, they use this custom fine tuned model in order to make predictions. And again, they analyze predictions using tables. So if you want to get started with big text models, and especially using API such as OpenAI, it has never been easier than now. Check out Weights and Biases. They have all kinds of tools for machine learning researchers, practitioners, educators, students, and much more. Individual use is free forever and they have great team plans and they even do on-prem hosting for enterprise. With that being said, thanks again to Weights and Biases for sponsoring this video. Please check them out and let's get into it. The Uber Engineering blog has a new post up about how Uber switched from XGBoost to Deep Learning to predict arrival times. Uber itself is a massive business. It's not only ride sharing, it's packages, it's food, and all of these things have in common that at some point, there needs to be made a prediction of how long something is going to take until it arrives. Either the food, the people, the packages, the time, the time, the time, the time, the time, you name it. So they used to have this big XGBoost model that predicted when stuff would arrive. And in the blog post, they detail that that just didn't scale anymore. They had more and more data they needed to incorporate. They wanted to get more accuracy, more diverse business cases, more locations. So they switched to Deep Learning. Now what's pretty interesting right here is that the goal isn't necessarily to predict the arrival time. However, they have a traffic routing system already, which is essentially something like Google Maps, you type in where you want to go and where you are, and the routing system analyzes the individual pieces, maybe a little bit of traffic on them, and then predicts for each of the individual pieces, how long it's going to take, you add all of that up, you get some sort of an estimate. Now the problem is real life is more complicated than you can just estimate from a map and a bit of traffic data. So what the machine learning model does is it takes a whole bunch of features, discrete features, continuous features, which interestingly, they quantize first before feeding them to the model, they feed that into a transformer model. And from that they predict a residual. So whatever they need to correct from the routing output, so they don't predict directly how long something's going to take, they simply predict how much it's going to deviate from the routing system's predictions, the system itself seems fairly involved, they don't just shove all the features into the beginning, they also have some features that come in later into the system. But I think the general principle of taking something like a base heuristic, like the routing system, and then simply predicting the residual might be a more general thing that I don't see used often enough. Now maybe I just don't know, and it's used all over. But I do think that we could layer our approaches much more than we are doing right now. Because whenever people switch from something classic to something deep learning, they try to just sort of do all end to end. And maybe the approach of doing more of like a hierarchical prediction where every layer just predicts the residual from the last layer might actually be better. The blog post goes into detail how carefully you have to be with respect to some of the features. For example, location is a very special feature, obviously, if you do routing, because you can't just encode the coordinates because the model needs to somehow know something about the 2d structure. So there's a location hashing algorithm where you can trade off accuracy versus storage. There are also various considerations with respect to the loss where they use the asymmetric hubral loss arguing for example, that being one minute too late is much worse than being one minute too early. So this lets the engineers tune the system in concordance with business decisions. They also describe how they train this thing and then finally deploy it. What's impressive is that the predictions come back in the order of milliseconds, which is pretty cool. Yeah, it seems like a big jump in performance for the Uber estimated arrival times. If you want to learn more, please check out the blog post and the Uber engineering blog. DeepMind has released a blog post called mu zeros first step from research into the real world. And mu zero is an iteration on the alpha zero algorithm. The difference being alpha zero still required an internal simulator. Therefore, it only worked for systems where such a simulator was available. For example, games like chess and go. In these games, you can do a step and you know exactly how the boards going to look like. And you can reverse the step again, you say, Oh, no, I actually don't want to do that. I want to do something else. You can use that for planning into the future, you can start multiple times, explore different paths and so on. There are however environments where this is not possible, for example, pretty much anywhere else in life. mu zero overcomes this by building a latent model in which it can plan forward. So there's no explicit simulator required. So mu zero is more general than alpha zero and has matched or surpassed alpha zero in many domains. Yet it's still sort of lacked the real world application. Because even for mu zero, you need giant amounts of data to train this thing on. Now it does make sense that a video compression is a really good application for something like mu zero. So what you do in video compression is you look at a video frame by frame, and you try to transmit that sequence of frames over the network. Therefore, it should be as small as possible, yet still retain a lot of its quality. In order to do that, usually codecs are used not codecs with an X codecs with CS at the end, this is a piece of software that describes how to take video frames or sequences of video frames and represent them as compressed data stream. Now this is not a static function. In fact, how much a series of frames is compressed is controlled by this thing called the quantization parameter. The idea is if you have a slow scene, very static, like a green background or just a face talking, you can compress large parts of the images and you can compress them for a long time because they'll just be the same a minute from now. So you can crank up that quantization parameter without losing too much quality. However, if a scene is fast moving, if there's lots of stuff happening on screen, you cannot compress the image as much because over time things change. And therefore, there's more information on the screen, even though you might think this is not useful information, it is image information. And therefore, you cannot compress the image as much. Now current codecs use heuristics engineered heuristics to determine when I can crank up or down that quantization parameter. And that is kind of an ideal setting for something like new zero, you feed it a bunch of videos, you say here's a target quality that want to reach and you let me zero decide on the quantization parameter essentially for each frame. This is a sequential decision making process, you need a bit of outlook into the future to see what's happening later, how much can I compress now? What should I do? So it's very much in the framework of these reinforcement learning problems. Now I have looked at these videos. And so this is kind of the original video. Okay. And cool. All right. Now let's look at the new zero compressed video. Like I can't I cannot see a difference. So the the bitrate the bitrate savings is, is the idea that I can't see. Ah, I get it. Okay, maybe it's the idea that I can't see a difference. And they tell me that mu zero uses 4.7% less bits to encode that video sequence. 4.7% might not seem like a lot. But given that apparently, most internet traffic nowadays is video streaming, this is a giant saving. Now I still don't exactly know how much overhead there is running mu zero at inference time to do the compression. But fair to say that savings like this make a real difference on our already overloaded internet infrastructure. If you want to learn more, check out the DeepMind blog post, there's also a paper going along with that called mu zero with self competition for rate control in VP nine video compression that goes more into the in VP nine video compression that goes more into the details of how they train the system. It uses a concept called self competition, which is kind of akin to self play. And it's a lot more technical than the blog post. Google AI blog has a new entry called guiding frozen language models with learned soft prompts. Also here, there's a paper going along with that called the power of scale for parameter efficient prompt tuning. This prompt tuning is an interesting concept of a novel way in NLP in recent years, we've had two basic Modi operandas modus operandi, whatever the first one was kind of like the Bert mode, where you take a pre trained model like Bert, and you fine tune the model on your data, meaning you provided input output pairs, and you fine tuned either the whole model adapter layers or just the head or something like this. And then on the very other end of the spectrum is something like GPT three that is pre trained and will just remain fixed for the duration of its lifetime. And what you can do is you can prompt it, which means that you have to come up with clever things that you can put in front of your question to make GPT three output the correct thing, which is usually called in context learning this paper, they're not the first ones doing it as far as I'm aware, but it is an interesting concept. And it's taken a bit to the next level here is that why are we coming up ourselves with that stuff to input? Can't we teach a model to automatically come up with that stuff? So if we have a data set that might actually work. So what they do is they make the prompt input of the model into tunable parameters. So this is trained on data, so you need to have data in order to train this, but you'll keep the model completely frozen, and you'll only tune what they call the soft prompt. So you don't necessarily determine the tokens to input into the language model, but you do tune the input vectors. So the embeddings of the tokens if this were the prompt that is obviously gets a bit less interpretable and so on. But it is a cool concept. And I do believe that it is very parameter efficient way to steer these large language models. So in this particular paper, the specific tasks they tackle is sort of a multi task training regime, where for each task, they tune one of these prompts. But I believe this can this can go further. These prompts are you can see right here, it's a 20,000 parameters for a prompt, then that can steer a model of 11 billion that is a factor of like six zeros or something like this. And I think that's really cool because it gives us a handle on these big models. And I'm excited to see what we can do if we push this to the limits. Blocknerf is a new paper coming out of UC Berkeley Waymo and Google research, and it pushes nerf to the next level. What it does is it essentially takes an entire city block with Waymo cars going around photographing stuff, and then it constructs many different individual nerfs. A nerf is a neural radiance field. I have made a video somewhere about that if you're interested. Essentially, it is a 3D representation that you can render from any angle, and it will faithfully represent things like, you know, when stuff looks different if you view it from here or from here. It's not perfect, but it's really, really good. And the point is no one needs to sit down and make the 3D models. You simply provided a bunch of pictures and it figures out itself how the stuff looks in 3D. Now this used to work in a limited setting with like one object in the middle or one scene, but this paper right here takes an entire city block and figures out how to combine different nerfs, like different scenes together and stitch them together. We have a website that goes along with this with various videos where they showcase the power of this. So notice they're not even limited to the path that the cars originally drove on. They can just render from completely new points of view. This is really cool and the scale of this is unprecedented. If you want to check this out, visit their websites. They have many videos available and yeah, give it a try. And another post from the Google AI blog called Unlocking the Full Potential of Data Center ML Accelerators with Platform-Aware Neural Architecture Search. That is quite a long title, but what it describes is a paper that's called Searching for Fast Model Families on Data Center Accelerators that extends neural architecture search to also consider the underlying hardware. Usually neural architecture search is where I have some sort of an engine, like an evolutionary algorithm or something like this, slap together a bunch of modules and parameterize them and then I care which of them gives me the best end accuracy or something like this. In this particular case right here, they also worry about which models perform best on the underlying hardware. So you might know that things like TPUs and GPUs, they're good at some things and bad at other things. And their general layout of how they do computation, how they do memory access is very specialized to certain things. If you can make use of those things, if you can design models that inherently do very, very optimized memory access and so on, you can potentially speed up models by a lot while not sacrificing performance. All you do is you build a model that is better able to utilize the underlying hardware. So the final result of this paper is a model family called EfficientNetX. EfficientNetX largely matches EfficientNet, which is sort of a classic computer vision model. It largely matches that in terms of accuracy, yet it is much faster because it uses the underlying hardware a lot better. What the paper also does is it decouples the measure of flops, floating point operations, from actual performance. So people used to estimate how intensive, let's say, a model is by counting the number of flops that a forward pass would utilize. If a forward pass would utilize more flops, then the common assumption was, well, that sort of uses more compute and probably it will take longer. But EfficientNetX requires double the amount of flops than EfficientNet does. And therefore, people would say that it should take longer. However, it is two times faster on the appropriate hardware for which it was designed. This is an error rate of 400% if you actually consider flops as a measure of performance, which is crazy. So I think if anything, this paper shows that we need to rethink how we think about performance and that maybe just flops is not necessarily a good measure of how we estimate model compute utilization. This is a blog post from the Flower team. Flower means, I need to look this up, Flowing Epigenetic Robots and Systems. This is a research group that investigates things like cellular automata, artificial life, self organizing systems, self maintenance, and much more. This is a very lengthy blog post that goes into detail in some of these areas into a system called linear and into various connections with neuroscience, with self organizing systems with biology and so on. They even have some interactive demos. So as you can see right here, there are these life forms. Now you can spawn more of these life forms. And to be said, these life forms, they are not somehow controlled top down. They're self organizing, self perpetuating, even avoiding obstacles they do themselves. Now I can in fact, draw a bit more of an obstacle right here. You can see the evasion still works. It's pretty interesting to see what happens if you just put multiple of them. They do have collisions with each other. You can generate attractors to which they are going to be try to reach it. Come here. So if you feel tired of supervised learning, of having centralized parameters, of having a single model that does things and has overview and has top down control. And if you feel like you want something different, something more emerging, then give this blog post a read. As I said, it's a long blog post. It goes into detail into various systems, starting from very simple systems, and then going up into various experiments, various research papers on the topic, as I said, explains the system called linear and much more. So yeah, can only recommend if you want something out of the box. There's this tool called Know Your Data by the TensorFlow datasets team. And it is a very, very good TensorFlow datasets analyzer. For example, here the pre configured query is please give me images in the ImageNet dataset that have in their metadata, a latitude above 72.09. Now, as you can see, a lot of pictures are in fact, from sort of, let's say colder regions of the earth. Now, it's not always going to be right, but this is a very valuable tool if you want to debug your datasets, it integrates with a lot of stuff I already mentioned metadata, but it also integrates, for example, with a cloud vision, they will give you statistics of what cloud vision detects in these various images, you can also use that as filter. For example, now I would only like to get pictures that have a probability of containing a face above a certain amount, while also being very high in their latitude. Now, apparently there exists no such pictures. So let me clear one of the filters. And as you can see, there are some pictures where there might be faces. Now, ImageNet, obviously doesn't have many faces as such, you can see this picture that does contain faces contains, contains them from some sort of a print article. This tool can be used for many different things, you can analyze stats, you can analyze relations between things, you can inspect the data. And especially if you have your own datasets, this can help you discover problems with the data, discover biases, systematic distortions, and so on. There's a bit of an explanation page to go with it, you can see you can filter a group and much more. However, your datasets do have to be supported by the TensorFlow datasets API. Alright, some helpful things for this week. Just helpful things, not even libraries, just things. I guess the last one was already a helpful thing. Casualganpapers on Twitter says, OpenAI stealth released model weights for the largest clip models. So apparently their repo now says they've released the largest clip model weights. If you're into clip, go get them. On Neural Differential Equations is on Archive, but it's not just a paper, it's an entire PhD thesis by Patrick Kidger. And it serves as a little bit of a textbook on Neural Differential Equations. So if you're into that, check it out. PGMAX is a library that implements general factor graphs for discrete probabilistic graphical models. Graphical models have been a little bit forgotten, at least in the mainstream deep learning world in recent years. But they were really cool before AlexNet promise. So this library, among other things, implements differentiable loopy belief propagation in JAX. So if you do work with probabilistic models and graphs, give this library a try. D'Ambra is a arena for AIs. It is multiple things at the same time. So first and foremost, it is a library essentially reinforcement learning environments, mainly for two player fighting games right now. So they say they feature a collection of high quality environments for reinforcement learning research and experimentation. It's compliant with the OpenAI gym standards, and it includes classic fighter games such as Dead or Alive, Street Fighter, Tekken, and so on. They do have a YouTube channel where they show some baseline implementations of reinforcement learning agents. And they do also host tournaments in these games. It's kind of like a Kaggle competition, I guess, except your agent is paired up against another agents and then they play Tekken. If you're interested, check out D'Ambra. Python FHEZ is a privacy preserving, fully homomorphic encryption and deep learning library. This library supports a lot of primitives in the areas of doing deep learning on data that you might or shouldn't have access to that is private, that is secure in some form or another. And homomorphic encryption allows you to run certain calculations in an encrypted fashion or transmit information in an encrypted way such that either one or the other party doesn't necessarily get to know all the contents of the data. So this being combined with deep learning is pretty cool. And this library enables that Torch Metrics is a project by the PyTorch Lightning devs and it implements metrics for PyTorch, especially for distributed and scaled up PyTorch. Computing metrics is often a hassle because you need to accumulate over batches or over different machines and so on. This library reduces that boilerplate and lets you just track and export your metrics in a very easy way. Here's a simple example that tracks the accuracy over a bunch of batches, I guess a batch of batches, if you will. So it does compute the accuracy on each batch, but it also keeps track of all of them. And then at the end, you can get your accuracy over all of the data. Now, if you've ever done this, you know that last batch is always trouble if it's not exactly full, your metrics will not be perfectly accurate. And yeah, it seems like everyone on the world is just implementing the same thing. So good that there exist libraries. In Tao Tian tweets that their work on modern evolution strategies for creativity has been accepted and they've provided two new collabs that you can try out. So this work is very special. It's evolutionary strategies that try to make these collages of things. It uses clip and abstract shapes to achieve some visual goals. And it looks pretty sweet, I have to say. So now there's two collabs where you can try it out. Related to that, Evojax's hardware accelerated neuro evolution. In fact, if you have paid attention, the collabs from right before are in the Evojax repository. So this is a Jax library that enables neuro evolution, evolutionary search, anything like this. And it enables a lot of cool stuff that is kind of outside the box for classical deep learning. On the right is one of these collages that I've just mentioned. And on the left is a little game where the agents have to collect food but avoid poison. And all of this is trained using evolutionary strategies. There's a paper to go along with the Evojax environment if you're interested more. And lastly, Reddit user jkterry1 writes that five months after taking over maintenance, I'm happy to announce that Jim now has a proper documentation website for the first time in its life. If you don't know, Jim is a project started by OpenAI and then abandoned by OpenAI and has been taken up by an open source developer who was kind enough to continue this project. And now under gym library dot ml, you can find proper documentation for the gym library. Now given how prevalent Jim still is, this is pretty cool. It's clean and simple. And if you do work with Jim, and maybe you want to learn something new about the things that you've been using all along, check out this website. Alright, this was it for ml news this week. I hope you had fun and I'll see you next time. Bye bye.
 Uber now uses deep learning to predict arrival times.
 Mew Zero is used to compress YouTube videos and Nerve scales to entire city blocks.
 Amazing. Welcome to ML News.
 Hey, ho there. This video is sponsored by Weights and Biases. Today, I want to tell you about a new
 feature in Weights and Biases, which is their integration with the OpenAI API. If you didn't
 know, OpenAI has the ability that you can provide your data and they fine tune a GPT-3 model for you.
 Now, this is pretty cool in itself because you get your own little custom endpoint that you can call
 has been trained on your data. But now you can sync those training runs to your Weights and Biases
 account. All you need to do for this to happen is to simply call the sync command on the command line
 and all your training runs will be synced to Weights and Biases. They have a little demo
 collab where they demonstrate that you can actually use the artifacts and tables features
 from Weights and Biases. Essentially, anything that you know, you can construct your data sets,
 you can have them as artifacts, you can look at them in the tables, then you can ship them to
 OpenAI to do a fine tuning run. And then you can analyze that fine tuning run and the outputs of
 it again in Weights and Biases. They even have a little demo report where they do something like
 this. They upload a Wikipedia data set, they analyze it first using tables, then they analyze
 the loss from the fine tuning results. They do a little bit of a hyper parameter search and you can
 analyze those in these nice parallel coordinate plots fully interactively. And in the end, they
 use this custom fine tuned model in order to make predictions. And again, they analyze predictions
 using tables. So if you want to get started with big text models, and especially using API such as
 OpenAI, it has never been easier than now. Check out Weights and Biases. They have all kinds of
 tools for machine learning researchers, practitioners, educators, students, and much more.
 Individual use is free forever and they have great team plans and they even do on-prem hosting for
 enterprise. With that being said, thanks again to Weights and Biases for sponsoring this video.
 Please check them out and let's get into it.
 The Uber Engineering blog has a new post up about how Uber switched from XGBoost to Deep Learning
 to predict arrival times. Uber itself is a massive business. It's not only ride sharing,
 it's packages, it's food, and all of these things have in common that at some point,
 there needs to be made a prediction of how long something is going to take until it arrives.
 Either the food, the people, the packages, the time, the time, the time, the time, the time,
 you name it. So they used to have this big XGBoost model that predicted when stuff would arrive.
 And in the blog post, they detail that that just didn't scale anymore. They had more and more data
 they needed to incorporate. They wanted to get more accuracy, more diverse business cases,
 more locations. So they switched to Deep Learning. Now what's pretty interesting right here is that
 the goal isn't necessarily to predict the arrival time. However, they have a traffic routing system
 already, which is essentially something like Google Maps, you type in where you want to go and
 where you are, and the routing system analyzes the individual pieces, maybe a little bit of traffic
 on them, and then predicts for each of the individual pieces, how long it's going to take,
 you add all of that up, you get some sort of an estimate. Now the problem is real life is
 more complicated than you can just estimate from a map and a bit of traffic data. So what the
 machine learning model does is it takes a whole bunch of features, discrete features, continuous
 features, which interestingly, they quantize first before feeding them to the model, they feed that
 into a transformer model. And from that they predict a residual. So whatever they need to
 correct from the routing output, so they don't predict directly how long something's going to
 take, they simply predict how much it's going to deviate from the routing system's predictions,
 the system itself seems fairly involved, they don't just shove all the features into the beginning,
 they also have some features that come in later into the system. But I think the general principle
 of taking something like a base heuristic, like the routing system, and then simply predicting
 the residual might be a more general thing that I don't see used often enough. Now maybe I just
 don't know, and it's used all over. But I do think that we could layer our approaches much more than
 we are doing right now. Because whenever people switch from something classic to something deep
 learning, they try to just sort of do all end to end. And maybe the approach of doing more of like
 a hierarchical prediction where every layer just predicts the residual from the last layer might
 actually be better. The blog post goes into detail how carefully you have to be with respect to some
 of the features. For example, location is a very special feature, obviously, if you do routing,
 because you can't just encode the coordinates because the model needs to somehow know something
 about the 2d structure. So there's a location hashing algorithm where you can trade off accuracy
 versus storage. There are also various considerations with respect to the loss where they use the
 asymmetric hubral loss arguing for example, that being one minute too late is much worse than being
 one minute too early. So this lets the engineers tune the system in concordance with business
 decisions. They also describe how they train this thing and then finally deploy it. What's impressive
 is that the predictions come back in the order of milliseconds, which is pretty cool. Yeah, it seems
 like a big jump in performance for the Uber estimated arrival times. If you want to learn
 more, please check out the blog post and the Uber engineering blog. DeepMind has released a blog
 post called mu zeros first step from research into the real world. And mu zero is an iteration on the
 alpha zero algorithm. The difference being alpha zero still required an internal simulator. Therefore,
 it only worked for systems where such a simulator was available. For example, games like chess and
 go. In these games, you can do a step and you know exactly how the boards going to look like. And you
 can reverse the step again, you say, Oh, no, I actually don't want to do that. I want to do
 something else. You can use that for planning into the future, you can start multiple times,
 explore different paths and so on. There are however environments where this is not possible,
 for example, pretty much anywhere else in life. mu zero overcomes this by building a latent model
 in which it can plan forward. So there's no explicit simulator required. So mu zero is
 more general than alpha zero and has matched or surpassed alpha zero in many domains. Yet it's
 still sort of lacked the real world application. Because even for mu zero, you need giant amounts
 of data to train this thing on. Now it does make sense that a video compression is a really good
 application for something like mu zero. So what you do in video compression is you look at a video
 frame by frame, and you try to transmit that sequence of frames over the network. Therefore,
 it should be as small as possible, yet still retain a lot of its quality. In order to do that,
 usually codecs are used not codecs with an X codecs with CS at the end, this is a piece of software
 that describes how to take video frames or sequences of video frames and represent them
 as compressed data stream. Now this is not a static function. In fact, how much a series of
 frames is compressed is controlled by this thing called the quantization parameter. The idea is
 if you have a slow scene, very static, like a green background or just a face talking, you can
 compress large parts of the images and you can compress them for a long time because they'll
 just be the same a minute from now. So you can crank up that quantization parameter without losing
 too much quality. However, if a scene is fast moving, if there's lots of stuff happening on
 screen, you cannot compress the image as much because over time things change. And therefore,
 there's more information on the screen, even though you might think this is not useful information,
 it is image information. And therefore, you cannot compress the image as much. Now current codecs
 use heuristics engineered heuristics to determine when I can crank up or down that quantization
 parameter. And that is kind of an ideal setting for something like new zero, you feed it a bunch
 of videos, you say here's a target quality that want to reach and you let me zero decide on the
 quantization parameter essentially for each frame. This is a sequential decision making process,
 you need a bit of outlook into the future to see what's happening later, how much can I compress
 now? What should I do? So it's very much in the framework of these reinforcement learning problems.
 Now I have looked at these videos. And so this is kind of the original video. Okay.
 And cool. All right. Now let's look at the new zero compressed video.
 Like I can't I cannot see a difference. So the the bitrate the bitrate savings is,
 is the idea that I can't see. Ah, I get it. Okay, maybe it's the idea that I can't see a difference.
 And they tell me that mu zero uses 4.7% less bits to encode that video sequence. 4.7% might not seem
 like a lot. But given that apparently, most internet traffic nowadays is video streaming,
 this is a giant saving. Now I still don't exactly know how much overhead there is running mu zero
 at inference time to do the compression. But fair to say that savings like this make a real
 difference on our already overloaded internet infrastructure. If you want to learn more,
 check out the DeepMind blog post, there's also a paper going along with that called mu zero
 with self competition for rate control in VP nine video compression that goes more into the
 in VP nine video compression that goes more into the details of how they train the system. It uses
 a concept called self competition, which is kind of akin to self play. And it's a lot more technical
 than the blog post. Google AI blog has a new entry called guiding frozen language models with
 learned soft prompts. Also here, there's a paper going along with that called the power of scale
 for parameter efficient prompt tuning. This prompt tuning is an interesting concept of a novel way
 in NLP in recent years, we've had two basic Modi operandas modus operandi, whatever the first one
 was kind of like the Bert mode, where you take a pre trained model like Bert, and you fine tune the
 model on your data, meaning you provided input output pairs, and you fine tuned either the whole
 model adapter layers or just the head or something like this. And then on the very other end of the
 spectrum is something like GPT three that is pre trained and will just remain fixed for the duration
 of its lifetime. And what you can do is you can prompt it, which means that you have to come up
 with clever things that you can put in front of your question to make GPT three output the correct
 thing, which is usually called in context learning this paper, they're not the first ones doing it
 as far as I'm aware, but it is an interesting concept. And it's taken a bit to the next level
 here is that why are we coming up ourselves with that stuff to input? Can't we teach a model to
 automatically come up with that stuff? So if we have a data set that might actually work. So what
 they do is they make the prompt input of the model into tunable parameters. So this is trained on data,
 so you need to have data in order to train this, but you'll keep the model completely frozen,
 and you'll only tune what they call the soft prompt. So you don't necessarily determine the
 tokens to input into the language model, but you do tune the input vectors. So the embeddings of
 the tokens if this were the prompt that is obviously gets a bit less interpretable and so on.
 But it is a cool concept. And I do believe that it is very parameter efficient way to steer
 these large language models. So in this particular paper, the specific tasks they tackle is sort of a
 multi task training regime, where for each task, they tune one of these prompts. But I believe this
 can this can go further. These prompts are you can see right here, it's a 20,000 parameters for a
 prompt, then that can steer a model of 11 billion that is a factor of like six zeros or something
 like this. And I think that's really cool because it gives us a handle on these big models. And I'm
 excited to see what we can do if we push this to the limits. Blocknerf is a new paper coming out
 of UC Berkeley Waymo and Google research, and it pushes nerf to the next level. What it does is it
 essentially takes an entire city block with Waymo cars going around photographing stuff, and then
 it constructs many different individual nerfs. A nerf is a neural radiance field. I have made a
 video somewhere about that if you're interested. Essentially, it is a 3D representation that you
 can render from any angle, and it will faithfully represent things like, you know, when stuff looks
 different if you view it from here or from here. It's not perfect, but it's really, really good.
 And the point is no one needs to sit down and make the 3D models. You simply provided a bunch of
 pictures and it figures out itself how the stuff looks in 3D. Now this used to work in a limited
 setting with like one object in the middle or one scene, but this paper right here takes an entire
 city block and figures out how to combine different nerfs, like different scenes together and stitch
 them together. We have a website that goes along with this with various videos where they showcase
 the power of this. So notice they're not even limited to the path that the cars originally
 drove on. They can just render from completely new points of view. This is really cool and the
 scale of this is unprecedented. If you want to check this out, visit their websites. They have
 many videos available and yeah, give it a try. And another post from the Google AI blog called
 Unlocking the Full Potential of Data Center ML Accelerators with Platform-Aware Neural
 Architecture Search. That is quite a long title, but what it describes is a paper that's called
 Searching for Fast Model Families on Data Center Accelerators that extends neural architecture
 search to also consider the underlying hardware. Usually neural architecture search is where I have
 some sort of an engine, like an evolutionary algorithm or something like this, slap together
 a bunch of modules and parameterize them and then I care which of them gives me the best
 end accuracy or something like this. In this particular case right here, they also worry about
 which models perform best on the underlying hardware. So you might know that things like
 TPUs and GPUs, they're good at some things and bad at other things. And their general layout of how
 they do computation, how they do memory access is very specialized to certain things. If you can make
 use of those things, if you can design models that inherently do very, very optimized memory access
 and so on, you can potentially speed up models by a lot while not sacrificing performance. All you do
 is you build a model that is better able to utilize the underlying hardware. So the final result of
 this paper is a model family called EfficientNetX. EfficientNetX largely matches EfficientNet, which
 is sort of a classic computer vision model. It largely matches that in terms of accuracy, yet it
 is much faster because it uses the underlying hardware a lot better. What the paper also does
 is it decouples the measure of flops, floating point operations, from actual performance. So
 people used to estimate how intensive, let's say, a model is by counting the number of flops that a
 forward pass would utilize. If a forward pass would utilize more flops, then the common assumption was,
 well, that sort of uses more compute and probably it will take longer. But EfficientNetX requires
 double the amount of flops than EfficientNet does. And therefore, people would say that it should
 take longer. However, it is two times faster on the appropriate hardware for which it was designed.
 This is an error rate of 400% if you actually consider flops as a measure of performance,
 which is crazy. So I think if anything, this paper shows that we need to rethink how we think about
 performance and that maybe just flops is not necessarily a good measure of how we estimate
 model compute utilization. This is a blog post from the Flower team. Flower means, I need to look
 this up, Flowing Epigenetic Robots and Systems. This is a research group that investigates things
 like cellular automata, artificial life, self organizing systems, self maintenance, and much
 more. This is a very lengthy blog post that goes into detail in some of these areas into a system
 called linear and into various connections with neuroscience, with self organizing systems with
 biology and so on. They even have some interactive demos. So as you can see right here, there are
 these life forms. Now you can spawn more of these life forms. And to be said, these life forms,
 they are not somehow controlled top down. They're self organizing, self perpetuating, even avoiding
 obstacles they do themselves. Now I can in fact, draw a bit more of an obstacle right here. You can
 see the evasion still works. It's pretty interesting to see what happens if you just put multiple of
 them. They do have collisions with each other. You can generate attractors to which they are
 going to be try to reach it. Come here. So if you feel tired of supervised learning, of having
 centralized parameters, of having a single model that does things and has overview and has top down
 control. And if you feel like you want something different, something more emerging, then give this
 blog post a read. As I said, it's a long blog post. It goes into detail into various systems,
 starting from very simple systems, and then going up into various experiments, various research
 papers on the topic, as I said, explains the system called linear and much more. So yeah,
 can only recommend if you want something out of the box. There's this tool called
 Know Your Data by the TensorFlow datasets team. And it is a very, very good TensorFlow datasets
 analyzer. For example, here the pre configured query is please give me images in the ImageNet
 dataset that have in their metadata, a latitude above 72.09. Now, as you can see, a lot of pictures
 are in fact, from sort of, let's say colder regions of the earth. Now, it's not always
 going to be right, but this is a very valuable tool if you want to debug your datasets, it
 integrates with a lot of stuff I already mentioned metadata, but it also integrates, for example,
 with a cloud vision, they will give you statistics of what cloud vision detects in these various
 images, you can also use that as filter. For example, now I would only like to get pictures
 that have a probability of containing a face above a certain amount, while also being very high in
 their latitude. Now, apparently there exists no such pictures. So let me clear one of the filters.
 And as you can see, there are some pictures where there might be faces. Now, ImageNet, obviously
 doesn't have many faces as such, you can see this picture that does contain faces contains,
 contains them from some sort of a print article. This tool can be used for many different things,
 you can analyze stats, you can analyze relations between things, you can inspect the data. And
 especially if you have your own datasets, this can help you discover problems with the data,
 discover biases, systematic distortions, and so on. There's a bit of an explanation page to go
 with it, you can see you can filter a group and much more. However, your datasets do have to be
 supported by the TensorFlow datasets API. Alright, some helpful things for this week. Just helpful
 things, not even libraries, just things. I guess the last one was already a helpful thing.
 Casualganpapers on Twitter says, OpenAI stealth released model weights for the largest clip
 models. So apparently their repo now says they've released the largest clip model weights. If you're
 into clip, go get them. On Neural Differential Equations is on Archive, but it's not just a paper,
 it's an entire PhD thesis by Patrick Kidger. And it serves as a little bit of a textbook on
 Neural Differential Equations. So if you're into that, check it out.
 PGMAX is a library that implements general factor graphs for discrete probabilistic graphical
 models. Graphical models have been a little bit forgotten, at least in the mainstream deep
 learning world in recent years. But they were really cool before AlexNet promise. So this
 library, among other things, implements differentiable loopy belief propagation in
 JAX. So if you do work with probabilistic models and graphs, give this library a try.
 D'Ambra is a arena for AIs. It is multiple things at the same time. So first and foremost,
 it is a library essentially reinforcement learning environments, mainly for two player
 fighting games right now. So they say they feature a collection of high quality environments for
 reinforcement learning research and experimentation. It's compliant with the OpenAI gym standards,
 and it includes classic fighter games such as Dead or Alive, Street Fighter, Tekken, and so on.
 They do have a YouTube channel where they show some baseline implementations of reinforcement
 learning agents. And they do also host tournaments in these games. It's kind of like a Kaggle
 competition, I guess, except your agent is paired up against another agents and then they play
 Tekken. If you're interested, check out D'Ambra. Python FHEZ is a privacy preserving, fully
 homomorphic encryption and deep learning library. This library supports a lot of primitives in the
 areas of doing deep learning on data that you might or shouldn't have access to that is private,
 that is secure in some form or another. And homomorphic encryption allows you to run certain
 calculations in an encrypted fashion or transmit information in an encrypted way such that either
 one or the other party doesn't necessarily get to know all the contents of the data. So this being
 combined with deep learning is pretty cool. And this library enables that Torch Metrics is a
 project by the PyTorch Lightning devs and it implements metrics for PyTorch, especially for
 distributed and scaled up PyTorch. Computing metrics is often a hassle because you need to
 accumulate over batches or over different machines and so on. This library reduces that boilerplate
 and lets you just track and export your metrics in a very easy way. Here's a simple example that
 tracks the accuracy over a bunch of batches, I guess a batch of batches, if you will. So it
 does compute the accuracy on each batch, but it also keeps track of all of them. And then at the
 end, you can get your accuracy over all of the data. Now, if you've ever done this, you know that
 last batch is always trouble if it's not exactly full, your metrics will not be perfectly accurate.
 And yeah, it seems like everyone on the world is just implementing the same thing. So good that
 there exist libraries. In Tao Tian tweets that their work on modern evolution strategies for
 creativity has been accepted and they've provided two new collabs that you can try out. So this work
 is very special. It's evolutionary strategies that try to make these collages of things. It uses
 clip and abstract shapes to achieve some visual goals. And it looks pretty sweet, I have to say.
 So now there's two collabs where you can try it out. Related to that, Evojax's hardware accelerated
 neuro evolution. In fact, if you have paid attention, the collabs from right before are in
 the Evojax repository. So this is a Jax library that enables neuro evolution, evolutionary search,
 anything like this. And it enables a lot of cool stuff that is kind of outside the box for
 classical deep learning. On the right is one of these collages that I've just mentioned. And on
 the left is a little game where the agents have to collect food but avoid poison. And all of this
 is trained using evolutionary strategies. There's a paper to go along with the Evojax environment
 if you're interested more. And lastly, Reddit user jkterry1 writes that five months after taking
 over maintenance, I'm happy to announce that Jim now has a proper documentation website for the
 first time in its life. If you don't know, Jim is a project started by OpenAI and then abandoned by
 OpenAI and has been taken up by an open source developer who was kind enough to continue this
 project. And now under gym library dot ml, you can find proper documentation for the gym library.
 Now given how prevalent Jim still is, this is pretty cool. It's clean and simple. And if you
 do work with Jim, and maybe you want to learn something new about the things that you've been
 using all along, check out this website. Alright, this was it for ml news this week. I hope you had
 fun and I'll see you next time. Bye bye.