[ML News] DeepMind does Nowcasting | The Guardian's shady reporting | AI finishes Beethoven's 10th
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper nowcasting mlnews deepmind weather prediction ai weather short term weather rain prediction rain when ai nowcasting ml weather prediction deepmind weather the guardian truthfulqa truthful qa language models truthful plato xl beethoven 10 ai music ai art ai painting painting authenticity huggingface huggingface infinity neuromorphic chips
#deepmind #nowcasting #machinelearning

Your holy update on what's new in the Machine Learning world.

OUTLINE:
0:00 - Intro
0:30 - DeepMind tackles Nowcasting
3:30 - The Guardian's shady reporting on TruthfulQA
6:15 - Stochastic training not necessary for generalization
7:35 - Google AI's efficient partitioning of road networks
9:15 - MiniHack Reinforcement Learning Environment
10:45 - Plato XL 11B dialog model
11:35 - AI finishes Beethoven's 10th Symphony
13:10 - AI casts doubt on painting authenticity
15:55 - ShadowDragon social media surveillance
18:45 - Helpful Libraries
25:20 - Samsung to copy-paste brains onto chips

References:
DeepMind improves Nowcasting
https://deepmind.com/blog/article/nowcasting
https://www.nature.com/articles/s41586-021-03854-z
https://github.com/deepmind/deepmind-research/tree/master/nowcasting
https://colab.research.google.com/github/deepmind/deepmind-research/blob/master/nowcasting/Open_sourced_dataset_and_model_snapshot_for_precipitation_nowcasting.ipynb

The Guardian's shady reporting on TruthfulQA
https://www.theguardian.com/commentisfree/2021/oct/02/the-truth-about-artificial-intelligence-it-isnt-that-honest?CMP=Share_iOSApp_Other

Stochastic Training is Not Necessary for Generalization
https://arxiv.org/pdf/2109.14119.pdf

Google AI - Efficient Partitioning of Road Networks
https://ai.googleblog.com/2021/09/efficient-partitioning-of-road-networks.html

MiniHack Reinforcement Learning Environment
https://ai.facebook.com/blog/minihack-a-new-sandbox-for-open-ended-reinforcement-learning

Baidu PLATO-XL 11B Dialog Model
http://research.baidu.com/Blog/index-view?id=163

AI finishes Beethoven's 10th Symphony
https://thenextweb.com/news/computer-scientists-completed-beethoven-10th-symphony-syndication

AI casts doubt on paining authenticity
https://www.smithsonianmag.com/smart-news/ai-casts-new-doubt-on-national-gallerys-prized-peter-paul-rubens-180978771/
https://art-recognition.com/
https://art-recognition.com/case-studies/
https://art-recognition.com/faq/

ShadowDragon Social Media Surveillance
https://www.rt.com/usa/535630-ai-surveillance-police-program-social-media/
https://theintercept.com/2021/09/21/surveillance-social-media-police-microsoft-shadowdragon-kaseware/

Helpful Libraries / Datasets
https://huggingface.co/infinity
https://yanaiela.github.io/TNE/?s=09&utm_source=pocket_mylist
https://arxiv.org/abs/2109.10282
https://github.com/microsoft/unilm/tree/master/trocr
https://medium.com/people-ai-research/kaokore-exploring-the-intersection-of-humanities-and-ml-research-through-a-japanese-art-dataset-f6035ba1e4d
https://raft.elicit.org/
https://huggingface.co/spaces/ought/raft-leaderboard
https://huggingface.co/spaces/ought/raft-viewer?dataset=raft&config=ade_corpus_v2&raft=dataset&banking_77=config
https://arxiv.org/pdf/2109.14076.pdf
https://arxiv.org/pdf/2109.14394.pdf
https://www.robots.ox.ac.uk/~vgg/research/pass/
https://zenodo.org/record/5528345#.YVrtd0ZByDU
https://github.com/yukimasano/PASS/
https://openreview.net/pdf?id=BwzYI-KaHdr
https://github.com/pytorch/data?utm_source=pocket_mylist

Samsung Method to copy paste brain onto chip
https://www.engadget.com/samsung-copy-and-paste-brain-neuromorphic-chips-185359994.html

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/1824646584

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Cut my hair, but not the beard. I have a giant cold sore here. That just looks weird without the beard. I was just gonna wait. Well, we'll... um, yeah. Intro. DeepMind can predict rain better than anyone else. The Guardian is not so really truthful about truthful language models. And an AI finishes Beethoven's 10th symphony. Welcome to ML News. It's Monday. For centuries upon centuries, millennia upon millennia, humans have shaken their fist at the sky for the rain which they could not predict. But while the gods of the heavens curse us with the falling precipitation, the gods of the earth, namely DeepMind, have now blessed us with a system that can tell us when and where it's going to rain. DeepMind has been looking into what's called now casting, which is an area of weather prediction that concerns just the next one to two hours. The reason being that apparently longer term forecasting can be done pretty accurately by sort of modeling the global weather, seeing how stuff moves, considering the physics and blah, blah, blah. But very short term predictions are not as accurate as we would like them to be. They've published this in a paper in Nature because where else would DeepMind publish? And it's actually a pretty interesting read. They cite the availability of high quality data, at least in the UK, where radar data is available at very high resolution, and the lack of current systems that work well. Now, instead of directly predicting, their model is a generative model. And from the paper, it looks like it's sort of a GAN with a bunch of GAN losses. So there is a temporal discriminator that discriminates between real and fake, I guess temporal rollouts, there is a spatial discriminator, and there's sort of a regularity loss as well. So essentially, what they do is they take a context of 20 minutes of radar data. And from that, they generate how the radar data looks about two hours ahead. And as you can see, this looks pretty good. So on the top left, you have the target on the top right, you have the DeepMind system. And on the bottom, you have two baselines, you can see that the DeepMind system is quite a bit more accurate. And not only is it more accurate as rated by the metrics and also by human climatologists or weather people, I don't know what exists in this case. And while the DeepMind system is more accurate in terms of metrics, and in terms of humans rating it, DeepMind also advocates for a more impact based metrics. For example, they highlight that the prediction of heavy precipitation at long lead times remains difficult for all approaches. And this is one of the crucial events that you would like to predict. So the paper advocates that maybe we should pay more attention to the things that actually impact such things as farming or air travel or deciding whether or not you can hold an event outdoors. Along with the paper, they do provide the data set and also a snapshot of the trained model. There's a colab where you can download the data set and try out the model. So no longer do you need to have a wet head, simply go here and see whether or not it's going to rain in the next hour. The Guardian has an opinion piece by John Norton that says the truth about artificial intelligence, it isn't that honest. Tests of natural language processing models show that the bigger they are, the bigger liars they are, should we be worried? Now, isn't this exactly what I predicted? I reported on this in last ML news, I made even a dedicated video about this benchmark called truthful QA, which is where the authors create a data set specifically designed to trick these language models going as far as throwing out questions that the language models get right and defining the word truthful in a way that if you answer complete garbage, it counts as truthful and therefore, the smaller models are better because they're just worse. Now, if you get the impression that one should mention these things when discussing this data set, then you'd be right. And I advocated for the same thing. I said if someone gives this as an example of how bad large language models are, and doesn't explicitly mention these things, they either don't know or they want to deceive you. Well, enter John Norton, who writes an entire opinion piece about this article. So given that he writes an entire opinion piece, the possibility that he hasn't read the paper is out. The only thing that comes even a little bit close to mentioning the way the data set was created is this sentence, they composed questions that some humans would answer falsely due to a false belief or misconception. Really, really, do you dear viewer, do you feel that is an adequate characterization of this benchmark? And do you feel that giving only this sentence draws the correct conclusion for people? I mean, it's not wrong, they did this, it just leaves out all the other stuff that you would need to know. And why does it leave out all the other stuff? Because of course, John wants to make an argument. And the argument will completely fall apart if you include this other stuff. And this is how science reporting goes when you have a narrative already in mind, it goes from a paper that does describe the complete process, but uses words such as truthful in very weird ways and is already framed in a particular manner to the Twitter announcements of the authors, which hide all of these facts in very specific wording in somewhere down the thread to the more popular hubs in the AI space, completely leaving away these details, and then to the mainstream media that just picks up the talking points and writes big articles about how bad these things are. Good job, everyone. Now, if only there were some kind of independent new source that you could get your machine learning news from that never ever ever makes mistakes. Now, where could one find that? Moving on, there is an interesting new paper on archive that's called stochastic training is not necessary for generalization that argues that if you tune full batch gradient correctly, and if you regularize correctly, and all of these kinds of things, then you can achieve the same performance with full batch gradient descent, then you can with SGD. And this casts doubt on a lot of theoretical explanations of why neural networks generalize so well, because many of these rely on the stochasticity of SGD. It's long been believed that the stochasticity plays some kind of a role in the generalization capabilities. And at least in part, this paper provides evidence that this might not be fully the case. However, that being said, you do need to regularize the network. So you do need to bring some of the implicit regularization that SGD appears to do through stochasticity into the world of explicit regularization. If you don't want the stochasticity in there, this appears to be true with and without data augmentation. And the paper argues that the community has essentially just spent a long time optimizing stochastic optimizers and hyper parameters and hasn't put that much effort into full batch methods. If this is of interest to you give this paper a read. Google AI releases the efficient partitioning of road networks. So this is a method to partition road networks, because if you simply look at a road network and try to do planning, it quickly becomes ginormous. If you just consider your own city, then already that's a pretty big graph if you really model all the connections, and then you consider a country you consider a continent, it quickly becomes so huge that something like a Dijkstra algorithm cannot plan efficiently anymore. So what you have to do is you have to partition and they give the example of state and island, which is an island in New York City. And while state and island has a lot of roads, and the surrounding city has a lot of roads, the access between the city and state and island is limited to four or five different bridges. So a smart algorithm would sort of clump state and island into very few nodes. And then you can essentially plan on these super nodes until you get to state and island and then inside state and island you can plan locally. This relies on the fact that our road networks very often are comprised of large interconnections between clusters of local roads. And in order to do this, they leverage random walks. So they simply start from some point on the map and they do random walks on the map. And the idea is that if you have super duper connected networks like inside state and island, then the random walks are probably going to stay in that area as they walk because the amount of connections inside the area is just so much larger, and they're not going to traverse very often these interconnections between the clusters. So therefore using random walks, you can figure out what are the clusters that are tightly connected and what are the clusters that are only loosely connected and therefore you can partition the graph. This is then refined using some flow algorithms. And at the end, we all get Google Maps. Thank you. There is a paper to go along with it have a read if that is of interest to you. Facebook AI research releases mini hack, a new sandbox for open ended reinforcement learning. This is an iteration on the net hack learning environment, which we've reported previously is available. Net hack is this game where you're in a dungeon and you need to do certain things, battle certain things, and so on. And the cool thing is that it's entirely described in kind of an ASCII way. So on the left here, you see the way that players or level creators would design levels and then add items and certain effects to it. Now the net hack game is very difficult game. And if you do reinforcement learning inside the game, there are a lot of tasks, there are a lot of things to do. And there is essentially just this one game. So mini hack is an environment where you can create small parts of the game, different sub levels, very simple tasks to test the individual abilities of agents. So you could, for example, make a mini level where it's just about avoiding obstacles, or you could make another mini level where it's simply about fighting opponents. So essentially, it's a level editor for the learning environment. Pretty cool. Give it a try. By do releases Plato XL, the world's first 11 billion parameter pre trained dialogue generation model. Now, whenever you say the world's first, you just have to make whatever comes very specific, then you're always the world's first. Like even if there were a 12 billion parameter pre trained dialogue generation model, Plato XL would still be the world's first 11 billion parameter pre trained dialogue generation model. However, this is really so far the biggest model that is specifically made for dialogue. It's available in English and Chinese and it is specifically trained to do long dialogue that keeps the context alive of what's talked about. Also, by do says that they will release the source code together with the English model on GitHub soon. The next web news writes Beethoven never finished his 10th symphony computer scientists just did this is a description of how a team of computer scientists and music scholars went about finishing Beethoven's 10th symphony. So the ninth symphony concluded with the Ode to Joy, they said, but the 10th symphony is unfinished. There are some scribbles by Beethoven some ideas, but it's by no means a finished piece of work. So the article details how the team went about recreating something that Beethoven might have written. And this is the important part to get right here. They do not claim that what they produce is Beethoven's 10th symphony as Beethoven would have written it. They say that this is given the ideas something that Beethoven might conceivably have come up with. Now that being said, there is a lot of iterations here, there's a lot of hand engineering, of course. So rather than this being fully AI generated, so I would rather call it a computer human collaboration to come up with something that plausibly could have happened had Beethoven lived for a bit longer. The article is fairly long, but it concludes with an excerpt from what these people created. That sounds like music, correct. So it seems like a cool practical applications of some of the techniques, the combination of AI and art is more and more explored. And it's good to see that music is not an exception here. Speaking of AI and art, the Smithsonian magazine writes, did Peter Paul Rubens really paint Samsung and Delilah? AI analysis renews doubts over the authenticity of a star painting in the London National Gallery's collection. Right, so there's this painting by a painter, I have no clue about art, I'm very sorry. But apparently the painting has been painted at some point and then went missing for a while and then it reappeared. And there is an entire debate about whether or not the reappeared painting is in fact the original painting or a fake. And there is this company called Art Recognition, which supposedly can give you a report about whether or not a given painting is actually from a given painter or not. And when this company analyzed the painting, the algorithm reported a 91.78% probability that Samsung and Delilah was painted by someone other than Rubens. So the company claims they have had quite a lot of successes when they assessed non disputed works with the algorithm being generally very correct in these assessments. So given this track record, the statement that this painting is probably fake is quite a bit of a shakeup. Now, now, now I have many questions about this, like, why does this need seven days to generate a report? Do these people actually go out and collect training data once you submit your thing? I don't know. Also, these systems got to be like super duper vulnerable to something like adversarial examples, they give you like a certificate of authenticity. Now, I'm going to guess this is like a CNN and the CNN is trained on a bunch of paintings of that painter, then you get some sort of a closeness estimate. Now, are there negative samples that this is trained at? Is this a one class SVM? I don't know. And actually found anything in the FAQ about how exactly this works. Apparently, the entire service is just digital, and you don't actually need the painting itself. And I know a lot of these scholars, they look at the paint strokes themselves and the thicknesses and x rays and whatnot to determine if art is authentic or not. Now, I have no doubt that something like this might actually work and might actually work better than human art experts can assess this. But at the same time, there are a lot of vulnerabilities in these systems. And I also wouldn't trust them. Now, would I trust them more than human experts? Not sure. I think what is safe to say is that simply because this company says this is probably fake, it probably won't convince anyone in the art world to change their minds about this painting. But interesting to know this exists. Rt writes AI driven community surveillance US cops reportedly using invasive tool to grab suspect social media, Pornhub and Tinder data. This report is about a company called Shadow Dragon that produces tools that scrape social media and pull together all kinds of information about individual people. And they sell this to law enforcement such that essentially anything you do across social media is neatly pulled together and analyzed in one place. This can then be combined with other surveillance mechanisms such as facial recognition from surveillance and all your data from various government databases. And it could technically be used to do predictive policing, which is a very controversial practice where you don't react to crime, but you try to react to pre crime, which gives it a sort of dystopian feeling. The company's founder says the company disagrees with predictive policing and does not build products with predictive capabilities or even suggestions. However, also their website praises the product for being able to predict violence. So, another question is where exactly Shadow Dragon has all this data from they themselves claim they do not intercept any private chats and they do not access anything that's proprietary or private, but simply scrape information from public websites. And again, that is highly disputed. Now, even if they only collect data from public websites, it's still quite worrisome to see that police are using these kind of systems. Of course, if you are a suspect police has every opportunity to go look at all of your social media all across the web and cross reference that but this is now being done in an automated fashion that is available to search and yes, train predictive models on top of it. Now, whether or not that's a good development, I leave that up to you. But a good recommendation is that simply assume that all of your activity online is being carried together at some place and just put all into one neat package. So while in previous life, you could be one kind of person on Twitter and another kind of person on LinkedIn in the future, these things are going to morph together more and more right now it's simply for law enforcement and the government. But given that these products seem to exist, you can expect that to be more the case in general in the future. So now you have the opportunity Do you want to behave more professionally on Twitter? Or do you want to just spew random opinions around on LinkedIn? I know what I'm gonna do. I'll also link a more in depth article by the intercept about shadow dragon and its connections to law enforcement if you're into that. Alright, helpful libraries, we have a lot of helpful libraries and data sets this week, like so much help on the internet. It's crazy. I'm suffocating from helpful libraries, I can't library anymore. That being said, you should totally check out hugging faces infinity, which is a Docker container that you can deploy yourself and that brings inference of transformers down to a millisecond. So if you read more into this, apparently it's about three milliseconds for CPU based transformers like Bert and Roberta, and one millisecond if you host them on GPU. Now this is pretty massive, it represents about a 10x improvement over previous attempts at speeding up these transformers. And you can deploy this on premise fits neatly within a Docker container. Now infinity is in a closed beta right now, but I guess they're going to release it at some point. I don't know, there is a website, but it doesn't say a whole lot of things about it. But I guess being in beta, this is bound to develop further. If you are interested, click the request trial button and see what happens. Next up the text based NP enrichment tasks, text base, text based, not sure which one it is, I'm gonna I'm gonna guess text based. So this is a data set for NLP. And by that, I mean rather how NLP used to be before deep learning, where every noun phrase is sort of annotated with all the possible cross references that exist in the text. So for example, the sentence here, Iranian student protesters face expulsion would be annotated in the following way, Iranian student protesters would be annotated at Amir Kabir University, it would also be annotated with against Ahmadinejad and face expulsion would be annotated with expulsion of 54 students expulsion by university chancellor Ali Reza Rahai or expulsion from America beer university. The goal of the data set is to do these annotations exhaustively, which I'm going to guess was a lot of work. But they do end up with 5497 documents that are exhaustively annotated with all possible links between noun phrases in each document. So pretty cool. If you're more into old school NLP, definitely give this a try. If you are into new school NLP, you should probably learn a bit about old school NLP. Next there is TROCR transformer based optical character recognition with pre trained models by Microsoft along with code. This is a new OCR method that uses transformers code is available, give it a try. Kaokore, which is joint work of Google research and collaborators from Japan's National Institute of Informatics and the University of Cambridge released this data set right here of Japanese art depicting faces. So they wonder whether or not they can teach machines to recognize facial depictions in Japanese art and classify them into various categories. So the data set is created from a larger Japanese art data set by cropping out all of the faces and then manually labeling them. The labels are things such as the social status which is divided into noble warrior incarnation, which is a depiction of a god or goddess and commoner which is I guess the rest of us. You can also train GANs on these data sets. And it seems to be just a pretty cool data set for doing research again, intersection of AI and art. This could be like a theme for today. Raft is a data set of real world annotated few shot tasks. This is a data set where both the task itself and the examples are given in natural language. For example, the task here is a data set is a list of institutions that have contributed papers, data data data data data. The goal is to classify these institutions into one of three categories, university, company or research Institute 50 labeled examples are provided and then there are a bunch of labeled examples, but not too many, thus the name few shots tasks. So this could be pretty cool, because especially it has a lot of practical applications, if you can specify the task in natural language, and you don't need a whole lot of examples for the model to learn a task, a lot of new possibilities in applying NLP open up, there is a paper and a leaderboard if you want to give it a try. The next helpful thing is a data set. Edgar data set is a data set of financial texts. Edgar is a database where all the public companies have to send in their annual reports and Edgar corpus is a data set of that they do provide a script with which to mine the Edgar database and they do train a set of word vectors which for specific tasks in finance perform much better than standard glove word vectors. So if you ever wanted a corpus of a giant amount of text that says absolutely nothing important of any informational value, because all of these finance departments basically just cover their own behind. There you go. The next data set is pass an image net replacement for self supervised pre training without humans. The pitch is they have 1.4 million images 1.4 million of them are CC by licensed and they're absolutely zero humans in the data set. Not only aren't there any depictions of humans, there are also no license plates or other personally identifiable information. The catch is this data set comes without labels. So you cannot train your classic computer vision image classification task, but it is supposed to be another data set that you can use for pre training your models without having to worry about there being some personally identifiable information in there. And also without having to worry about the licensing of the pictures that are in the data set. Now are people going to replace image net by this one? Or are people simply going to add this data to their image net data and therefore the problems simply remain? Well, you take a wild guess which one of those two things is going to happen. In any case, the data set is available to download. Have fun. And lastly, torch data by pytorch is a very unstable prototype, but it is primitives in order to build data loaders in order to make data loading from various sources more effective. So if data loading is your bottleneck, and the standard data loaders don't do the job, maybe give this a try. The API is might break. But you know, that's life. Last things for today, Engadget writes Samsung hopes to copy and paste the brain to 3d chip networks. Essentially, their idea is to stick a bunch of electrodes in there stimulate the neurons, see how the neurons stimulate other neurons from this, you can figure out which neurons are connected to each other and how strong and then you can simply map that connection pattern onto a neuromorphic chip. Now this might actually be an interesting way of getting a neural network with the general connection pattern of the human brain like the sparsity pattern or how exactly the things are connected. So it might be a neat architectural investigation into the human brain. However, the article also writes the move could serve as a shortcut to artificial intelligence systems that behave like real brains, including the flexibility to learn new concepts and adapt to changing conditions, you might even see fully autonomous machines with true cognition according to the researchers. Nah, nah. That's simply because you map out the connection pattern doesn't mean at all that you will get any sort of brain like activity connection pattern between neurons is only one of many, many, many things that is going on in the brain, especially things like learning require forming of new connections dynamically strengthening connections or strengthening synapses inhibiting expression of genes that lead to faster or slower reuptake of synaptic material. And all of this is simply not captured by simply mapping out the connection pattern, forgive me, but no, you're probably not going to see fully autonomous machines with true cognition simply because you can map the brain's connections. Now these things are supposed to run on neuromorphic chips, which means they will have some of these additional abilities, but still highly doubtful. That was it for this week's news. So much stuff happening if you have something interesting that's happening in your life. And if it is in any way related to machine learning, let me know we have no standards here at ML news. Anything goes. I'll see you next week. Ow, it hurts.
 Cut my hair, but not the beard. I have a giant cold sore here.
 That just looks weird without the beard. I was just gonna wait.
 Well, we'll... um, yeah. Intro.
 DeepMind can predict rain better than anyone else.
 The Guardian is not so really truthful about truthful language models.
 And an AI finishes Beethoven's 10th symphony. Welcome to ML News. It's Monday.
 For centuries upon centuries, millennia upon millennia, humans have shaken their
 fist at the sky for the rain which they could not predict.
 But while the gods of the heavens curse us with the falling precipitation,
 the gods of the earth, namely DeepMind, have now blessed us with a system that can tell us
 when and where it's going to rain. DeepMind has been looking into what's called now casting,
 which is an area of weather prediction that concerns just the next one to two hours.
 The reason being that apparently longer term forecasting can be done pretty accurately by
 sort of modeling the global weather, seeing how stuff moves, considering the physics and
 blah, blah, blah. But very short term predictions are not as accurate as we would like them to be.
 They've published this in a paper in Nature because where else would DeepMind publish?
 And it's actually a pretty interesting read. They cite the availability of high quality data,
 at least in the UK, where radar data is available at very high resolution,
 and the lack of current systems that work well. Now, instead of directly predicting,
 their model is a generative model. And from the paper, it looks like it's sort of a GAN with a
 bunch of GAN losses. So there is a temporal discriminator that discriminates between real
 and fake, I guess temporal rollouts, there is a spatial discriminator, and there's sort of a
 regularity loss as well. So essentially, what they do is they take a context of 20 minutes of radar
 data. And from that, they generate how the radar data looks about two hours ahead. And as you can
 see, this looks pretty good. So on the top left, you have the target on the top right, you have the
 DeepMind system. And on the bottom, you have two baselines, you can see that the DeepMind system
 is quite a bit more accurate. And not only is it more accurate as rated by the metrics and also by
 human climatologists or weather people, I don't know what exists in this case. And while the
 DeepMind system is more accurate in terms of metrics, and in terms of humans rating it,
 DeepMind also advocates for a more impact based metrics. For example, they highlight that the
 prediction of heavy precipitation at long lead times remains difficult for all approaches. And
 this is one of the crucial events that you would like to predict. So the paper advocates that maybe
 we should pay more attention to the things that actually impact such things as farming or air
 travel or deciding whether or not you can hold an event outdoors. Along with the paper, they do
 provide the data set and also a snapshot of the trained model. There's a colab where you can
 download the data set and try out the model. So no longer do you need to have a wet head,
 simply go here and see whether or not it's going to rain in the next hour.
 The Guardian has an opinion piece by John Norton that says the truth about artificial intelligence,
 it isn't that honest. Tests of natural language processing models show that the bigger they are,
 the bigger liars they are, should we be worried? Now, isn't this exactly what I predicted? I
 reported on this in last ML news, I made even a dedicated video about this benchmark called
 truthful QA, which is where the authors create a data set specifically designed to trick these
 language models going as far as throwing out questions that the language models get right
 and defining the word truthful in a way that if you answer complete garbage, it counts as truthful
 and therefore, the smaller models are better because they're just worse. Now, if you get
 the impression that one should mention these things when discussing this data set, then you'd
 be right. And I advocated for the same thing. I said if someone gives this as an example of how
 bad large language models are, and doesn't explicitly mention these things, they either don't know
 or they want to deceive you. Well, enter John Norton, who writes an entire opinion piece about
 this article. So given that he writes an entire opinion piece, the possibility that he hasn't read
 the paper is out. The only thing that comes even a little bit close to mentioning the way the data
 set was created is this sentence, they composed questions that some humans would answer falsely
 due to a false belief or misconception. Really, really, do you dear viewer, do you feel that is
 an adequate characterization of this benchmark? And do you feel that giving only this sentence
 draws the correct conclusion for people? I mean, it's not wrong, they did this, it just leaves out
 all the other stuff that you would need to know. And why does it leave out all the other stuff?
 Because of course, John wants to make an argument. And the argument will completely fall apart if
 you include this other stuff. And this is how science reporting goes when you have a narrative
 already in mind, it goes from a paper that does describe the complete process, but uses words
 such as truthful in very weird ways and is already framed in a particular manner to the Twitter
 announcements of the authors, which hide all of these facts in very specific wording in somewhere
 down the thread to the more popular hubs in the AI space, completely leaving away these details,
 and then to the mainstream media that just picks up the talking points and writes big articles about
 how bad these things are. Good job, everyone. Now, if only there were some kind of independent new
 source that you could get your machine learning news from that never ever ever makes mistakes.
 Now, where could one find that? Moving on, there is an interesting new paper on archive that's
 called stochastic training is not necessary for generalization that argues that if you tune
 full batch gradient correctly, and if you regularize correctly, and all of these kinds of things,
 then you can achieve the same performance with full batch gradient descent, then you can with SGD.
 And this casts doubt on a lot of theoretical explanations of why neural networks generalize
 so well, because many of these rely on the stochasticity of SGD. It's long been believed
 that the stochasticity plays some kind of a role in the generalization capabilities. And at least
 in part, this paper provides evidence that this might not be fully the case. However, that being
 said, you do need to regularize the network. So you do need to bring some of the implicit
 regularization that SGD appears to do through stochasticity into the world of explicit
 regularization. If you don't want the stochasticity in there, this appears to be true with and without
 data augmentation. And the paper argues that the community has essentially just spent a long time
 optimizing stochastic optimizers and hyper parameters and hasn't put that much effort into
 full batch methods. If this is of interest to you give this paper a read. Google AI releases
 the efficient partitioning of road networks. So this is a method to partition road networks,
 because if you simply look at a road network and try to do planning, it quickly becomes ginormous.
 If you just consider your own city, then already that's a pretty big graph if you really model all
 the connections, and then you consider a country you consider a continent, it quickly becomes
 so huge that something like a Dijkstra algorithm cannot plan efficiently anymore. So what you have
 to do is you have to partition and they give the example of state and island, which is an island in
 New York City. And while state and island has a lot of roads, and the surrounding city has a lot
 of roads, the access between the city and state and island is limited to four or five different
 bridges. So a smart algorithm would sort of clump state and island into very few nodes. And then you
 can essentially plan on these super nodes until you get to state and island and then inside state
 and island you can plan locally. This relies on the fact that our road networks very often are
 comprised of large interconnections between clusters of local roads. And in order to do this,
 they leverage random walks. So they simply start from some point on the map and they do random
 walks on the map. And the idea is that if you have super duper connected networks like inside state
 and island, then the random walks are probably going to stay in that area as they walk because
 the amount of connections inside the area is just so much larger, and they're not going to traverse
 very often these interconnections between the clusters. So therefore using random walks,
 you can figure out what are the clusters that are tightly connected and what are the clusters that
 are only loosely connected and therefore you can partition the graph. This is then refined using
 some flow algorithms. And at the end, we all get Google Maps. Thank you. There is a paper to go
 along with it have a read if that is of interest to you. Facebook AI research releases mini hack,
 a new sandbox for open ended reinforcement learning. This is an iteration on the net hack
 learning environment, which we've reported previously is available. Net hack is this game
 where you're in a dungeon and you need to do certain things, battle certain things, and so on.
 And the cool thing is that it's entirely described in kind of an ASCII way. So on the left here,
 you see the way that players or level creators would design levels and then add items and certain
 effects to it. Now the net hack game is very difficult game. And if you do reinforcement
 learning inside the game, there are a lot of tasks, there are a lot of things to do. And there is
 essentially just this one game. So mini hack is an environment where you can create small parts of
 the game, different sub levels, very simple tasks to test the individual abilities of agents. So
 you could, for example, make a mini level where it's just about avoiding obstacles, or you could
 make another mini level where it's simply about fighting opponents. So essentially, it's a level
 editor for the learning environment. Pretty cool. Give it a try. By do releases Plato XL, the world's
 first 11 billion parameter pre trained dialogue generation model. Now, whenever you say the world's
 first, you just have to make whatever comes very specific, then you're always the world's first.
 Like even if there were a 12 billion parameter pre trained dialogue generation model, Plato XL
 would still be the world's first 11 billion parameter pre trained dialogue generation model.
 However, this is really so far the biggest model that is specifically made for dialogue. It's
 available in English and Chinese and it is specifically trained to do long dialogue that
 keeps the context alive of what's talked about. Also, by do says that they will release the source
 code together with the English model on GitHub soon. The next web news writes Beethoven never
 finished his 10th symphony computer scientists just did this is a description of how a team of
 computer scientists and music scholars went about finishing Beethoven's 10th symphony. So the ninth
 symphony concluded with the Ode to Joy, they said, but the 10th symphony is unfinished. There are
 some scribbles by Beethoven some ideas, but it's by no means a finished piece of work. So the article
 details how the team went about recreating something that Beethoven might have written.
 And this is the important part to get right here. They do not claim that what they produce
 is Beethoven's 10th symphony as Beethoven would have written it. They say that this is given the
 ideas something that Beethoven might conceivably have come up with. Now that being said, there is
 a lot of iterations here, there's a lot of hand engineering, of course. So rather than this being
 fully AI generated, so I would rather call it a computer human collaboration to come up with
 something that plausibly could have happened had Beethoven lived for a bit longer. The article is
 fairly long, but it concludes with an excerpt from what these people created.
 That sounds like music, correct. So it seems like a cool practical applications of some of the
 techniques, the combination of AI and art is more and more explored. And it's good to see that music
 is not an exception here. Speaking of AI and art, the Smithsonian magazine writes, did Peter Paul
 Rubens really paint Samsung and Delilah? AI analysis renews doubts over the authenticity
 of a star painting in the London National Gallery's collection. Right, so there's this painting by a
 painter, I have no clue about art, I'm very sorry. But apparently the painting has been painted at
 some point and then went missing for a while and then it reappeared. And there is an entire debate
 about whether or not the reappeared painting is in fact the original painting or a fake. And there
 is this company called Art Recognition, which supposedly can give you a report about whether
 or not a given painting is actually from a given painter or not. And when this company analyzed
 the painting, the algorithm reported a 91.78% probability that Samsung and Delilah was painted
 by someone other than Rubens. So the company claims they have had quite a lot of successes
 when they assessed non disputed works with the algorithm being generally very correct in these
 assessments. So given this track record, the statement that this painting is probably fake
 is quite a bit of a shakeup. Now, now, now I have many questions about this, like, why does this
 need seven days to generate a report? Do these people actually go out and collect training data
 once you submit your thing? I don't know. Also, these systems got to be like super duper vulnerable
 to something like adversarial examples, they give you like a certificate of authenticity. Now,
 I'm going to guess this is like a CNN and the CNN is trained on a bunch of paintings of that painter,
 then you get some sort of a closeness estimate. Now, are there negative samples that this is
 trained at? Is this a one class SVM? I don't know. And actually found anything in the FAQ about how
 exactly this works. Apparently, the entire service is just digital, and you don't actually need the
 painting itself. And I know a lot of these scholars, they look at the paint strokes themselves
 and the thicknesses and x rays and whatnot to determine if art is authentic or not. Now,
 I have no doubt that something like this might actually work and might actually work better than
 human art experts can assess this. But at the same time, there are a lot of vulnerabilities in these
 systems. And I also wouldn't trust them. Now, would I trust them more than human experts? Not sure.
 I think what is safe to say is that simply because this company says this is probably fake,
 it probably won't convince anyone in the art world to change their minds about this painting.
 But interesting to know this exists. Rt writes AI driven community surveillance US cops reportedly
 using invasive tool to grab suspect social media, Pornhub and Tinder data. This report is about a
 company called Shadow Dragon that produces tools that scrape social media and pull together all
 kinds of information about individual people. And they sell this to law enforcement such that
 essentially anything you do across social media is neatly pulled together and analyzed in one place.
 This can then be combined with other surveillance mechanisms such as facial recognition from
 surveillance and all your data from various government databases. And it could technically
 be used to do predictive policing, which is a very controversial practice where you don't react
 to crime, but you try to react to pre crime, which gives it a sort of dystopian feeling.
 The company's founder says the company disagrees with predictive policing and does not build
 products with predictive capabilities or even suggestions. However, also their website praises
 the product for being able to predict violence. So, another question is where exactly Shadow Dragon
 has all this data from they themselves claim they do not intercept any private chats and they do not
 access anything that's proprietary or private, but simply scrape information from public websites.
 And again, that is highly disputed. Now, even if they only collect data from public websites,
 it's still quite worrisome to see that police are using these kind of systems. Of course,
 if you are a suspect police has every opportunity to go look at all of your social media all across
 the web and cross reference that but this is now being done in an automated fashion that is
 available to search and yes, train predictive models on top of it. Now, whether or not that's
 a good development, I leave that up to you. But a good recommendation is that simply assume that
 all of your activity online is being carried together at some place and just put all into one
 neat package. So while in previous life, you could be one kind of person on Twitter and another kind
 of person on LinkedIn in the future, these things are going to morph together more and more right
 now it's simply for law enforcement and the government. But given that these products seem
 to exist, you can expect that to be more the case in general in the future. So now you have
 the opportunity Do you want to behave more professionally on Twitter? Or do you want to
 just spew random opinions around on LinkedIn? I know what I'm gonna do. I'll also link a more
 in depth article by the intercept about shadow dragon and its connections to law enforcement
 if you're into that. Alright, helpful libraries, we have a lot of helpful libraries and data sets
 this week, like so much help on the internet. It's crazy. I'm suffocating from helpful libraries,
 I can't library anymore. That being said, you should totally check out hugging faces infinity,
 which is a Docker container that you can deploy yourself and that brings inference of transformers
 down to a millisecond. So if you read more into this, apparently it's about three milliseconds
 for CPU based transformers like Bert and Roberta, and one millisecond if you host them on GPU. Now
 this is pretty massive, it represents about a 10x improvement over previous attempts at speeding up
 these transformers. And you can deploy this on premise fits neatly within a Docker container.
 Now infinity is in a closed beta right now, but I guess they're going to release it at some point.
 I don't know, there is a website, but it doesn't say a whole lot of things about it. But I guess
 being in beta, this is bound to develop further. If you are interested, click the request trial
 button and see what happens. Next up the text based NP enrichment tasks, text base, text based,
 not sure which one it is, I'm gonna I'm gonna guess text based. So this is a data set for NLP.
 And by that, I mean rather how NLP used to be before deep learning, where every noun phrase
 is sort of annotated with all the possible cross references that exist in the text. So for example,
 the sentence here, Iranian student protesters face expulsion would be annotated in the following way,
 Iranian student protesters would be annotated at Amir Kabir University, it would also be annotated
 with against Ahmadinejad and face expulsion would be annotated with expulsion of 54 students
 expulsion by university chancellor Ali Reza Rahai or expulsion from America beer university. The goal
 of the data set is to do these annotations exhaustively, which I'm going to guess was a
 lot of work. But they do end up with 5497 documents that are exhaustively annotated with all possible
 links between noun phrases in each document. So pretty cool. If you're more into old school NLP,
 definitely give this a try. If you are into new school NLP, you should probably learn a bit about
 old school NLP. Next there is TROCR transformer based optical character recognition with pre trained
 models by Microsoft along with code. This is a new OCR method that uses transformers code is available,
 give it a try. Kaokore, which is joint work of Google research and collaborators from Japan's
 National Institute of Informatics and the University of Cambridge released this data set right here of
 Japanese art depicting faces. So they wonder whether or not they can teach machines to recognize
 facial depictions in Japanese art and classify them into various categories. So the data set
 is created from a larger Japanese art data set by cropping out all of the faces and then manually
 labeling them. The labels are things such as the social status which is divided into noble warrior
 incarnation, which is a depiction of a god or goddess and commoner which is I guess the rest
 of us. You can also train GANs on these data sets. And it seems to be just a pretty cool data set for
 doing research again, intersection of AI and art. This could be like a theme for today.
 Raft is a data set of real world annotated few shot tasks. This is a data set where both the
 task itself and the examples are given in natural language. For example, the task here is a data set
 is a list of institutions that have contributed papers, data data data data data. The goal is to
 classify these institutions into one of three categories, university, company or research
 Institute 50 labeled examples are provided and then there are a bunch of labeled examples, but
 not too many, thus the name few shots tasks. So this could be pretty cool, because especially it
 has a lot of practical applications, if you can specify the task in natural language, and you don't
 need a whole lot of examples for the model to learn a task, a lot of new possibilities in applying
 NLP open up, there is a paper and a leaderboard if you want to give it a try. The next helpful thing
 is a data set. Edgar data set is a data set of financial texts. Edgar is a database where all
 the public companies have to send in their annual reports and Edgar corpus is a data set of that
 they do provide a script with which to mine the Edgar database and they do train a set of word
 vectors which for specific tasks in finance perform much better than standard glove word vectors. So
 if you ever wanted a corpus of a giant amount of text that says absolutely nothing important of
 any informational value, because all of these finance departments basically just cover their
 own behind. There you go. The next data set is pass an image net replacement for self supervised
 pre training without humans. The pitch is they have 1.4 million images 1.4 million of them are
 CC by licensed and they're absolutely zero humans in the data set. Not only aren't there any
 depictions of humans, there are also no license plates or other personally identifiable information.
 The catch is this data set comes without labels. So you cannot train your classic computer vision
 image classification task, but it is supposed to be another data set that you can use for pre
 training your models without having to worry about there being some personally identifiable information
 in there. And also without having to worry about the licensing of the pictures that are in the data
 set. Now are people going to replace image net by this one? Or are people simply going to add this
 data to their image net data and therefore the problems simply remain? Well, you take a wild
 guess which one of those two things is going to happen. In any case, the data set is available to
 download. Have fun. And lastly, torch data by pytorch is a very unstable prototype, but it is
 primitives in order to build data loaders in order to make data loading from various sources more
 effective. So if data loading is your bottleneck, and the standard data loaders don't do the job,
 maybe give this a try. The API is might break. But you know, that's life. Last things for today,
 Engadget writes Samsung hopes to copy and paste the brain to 3d chip networks. Essentially,
 their idea is to stick a bunch of electrodes in there stimulate the neurons, see how the neurons
 stimulate other neurons from this, you can figure out which neurons are connected to each other and
 how strong and then you can simply map that connection pattern onto a neuromorphic chip.
 Now this might actually be an interesting way of getting a neural network with the general
 connection pattern of the human brain like the sparsity pattern or how exactly the things are
 connected. So it might be a neat architectural investigation into the human brain. However,
 the article also writes the move could serve as a shortcut to artificial intelligence systems that
 behave like real brains, including the flexibility to learn new concepts and adapt to changing
 conditions, you might even see fully autonomous machines with true cognition according to the
 researchers. Nah, nah. That's simply because you map out the connection pattern doesn't mean at all
 that you will get any sort of brain like activity connection pattern between neurons is only one of
 many, many, many things that is going on in the brain, especially things like learning require
 forming of new connections dynamically strengthening connections or strengthening synapses
 inhibiting expression of genes that lead to faster or slower reuptake of synaptic material. And all
 of this is simply not captured by simply mapping out the connection pattern, forgive me, but no,
 you're probably not going to see fully autonomous machines with true cognition simply because you
 can map the brain's connections. Now these things are supposed to run on neuromorphic chips, which
 means they will have some of these additional abilities, but still highly doubtful. That was
 it for this week's news. So much stuff happening if you have something interesting that's happening
 in your life. And if it is in any way related to machine learning, let me know we have no standards
 here at ML news. Anything goes. I'll see you next week.
 Ow, it hurts.