Unsupervised Brain Models - How does Deep Learning inform Neuroscience? (w/ Patrick Mineault)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper xcorr patrick mineault unsupervised models neuroscience neuroscience and deep learning deep learning brain machine learning brain brain models how does the brain work deep learning and neuroscience self-supervised models representation learning does the brain do representation learning does the brain work like a deep neural network neurips
#deeplearning #brain #neuroscience

Originally, Deep Learning sprang into existence inspired by how the brain processes information, but the two fields have diverged ever since. However, given that deep models can solve many perception tasks with remarkable accuracy, is it possible that we might be able to learn something about how the brain works by inspecting our models? I speak to Patrick Mineault about his blog post "2021 in review: unsupervised brain models" and we explore why neuroscientists are taking interest in unsupervised and self-supervised deep neural networks in order to explain how the brain works. We discuss a series of influential papers that have appeared last year, and we go into the more general questions of connecting neuroscience and machine learning.

OUTLINE:
0:00 - Intro & Overview
6:35 - Start of Interview
10:30 - Visual processing in the brain
12:50 - How does deep learning inform neuroscience?
21:15 - Unsupervised training explains the ventral stream
30:50 - Predicting own motion parameters explains the dorsal stream
42:20 - Why are there two different visual streams?
49:45 - Concept cells and representation learning
56:20 - Challenging the manifold theory
1:08:30 - What are current questions in the field?
1:13:40 - Should the brain inform deep learning?
1:18:50 - Neuromatch Academy and other endeavours

Blog Post: https://xcorr.net/2021/12/31/2021-in-review-unsupervised-brain-models/
Patrick's Blog: https://xcorr.net/
Twitter: https://twitter.com/patrickmineault
Neuromatch Academy: https://academy.neuromatch.io/

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hello there! Today I'm interviewing Patrick Minot, who has a PhD from McGill and did a postdoc at UCLA. He's an independent scientist and a neural data scientist. His interests are neuroscience and the connection to machine learning. He has an awesome blog called XCore, which I guess is pronounced cross correlation, but who knows. So please check out Patrick's blog. He also worked at Google for a while, seeing how people interact with web pages and was a brain computer interface engineer at Facebook Reality Labs. He also has launched the NeuroMatch Academy, which is sort of an intro, an academy where you learn in a summer school about computational neuroscience. This runs every year and you can take part if you want. We're going to touch on that a little bit in the interview. I just wanted to take it away beforehand. So I'm going to give a little introduction about what we'll talk about and then we'll jump into the interview. We're going to talk about mainly about this blog post right here, the 2021 in review unsupervised brain model. The main focus here is on unsupervised models and what they have to do with the brain. So a big question in neuroscience is how does the brain work? I guess it's the main question in neuroscience. And so people are developing the hypothesis of how the brain works. And deep learning turns out to be quite an interesting tool for neuroscientists because in deep learning, we get some inspiration from neuroscience, but essentially we build a model that end to end can learn some task, to perform some tasks. So this would be this one right here. Now the question is, is what deep models do the same or different than what brains do given that they solve the same task? Like let's say both recognize objects on images. Do they do the same thing or do they do something completely different? So neuroscientists, they wonder, you know, how does the brain learn stuff? Is it the same as neural network? Does the neural network now also during the interview, I have to stop saying neural network because it's ambiguous in this context. So does a deep network, a computer, a human made deep network, does it account for neural activity, which means that are the signals in the deep network the same or related to the signals that we see in the brain? And this turns out to be a very important tool for neuroscientists. What they want to see is that let's say the intermediate representations in the neural network. Like you have some kind of picture, it goes into a neural network, there's layer, layer, layer, layer, and then there's a classification head. The classification head might not be that interesting, but what is interesting is like some intermediate representation here. If we figure out that that explains, which means we can correlate it with things that are in the brain. And I'm going to draw like a very bad brain right here. If we can correlate this with things that are found in the brain signals like from fMRI, from electrodes that we put into people's heads, then that is an indication that what these deep networks are doing have something like that there is an effect that is similar and that could help us understand the brain. So the holy grail in neuroscience would be something that can perform the same task as humans that does account for neural activity that is biologically plausible. As you might know, there is still a debate of whether something like backprop is implementable in the brain in one way or another, or if we need an entirely different mechanism in the brain. And lastly, something that could conceivably also have evolved and maybe we'd even have some evidence of how it evolved over time. So we're going to talk about these models right here, specifically self supervised models. Self supervised models here is a slide by Jan Lacan, or models that don't need labels to train. And what you usually do is you block out part of something you know, and then try to predict that from the parts that you do know. For example, if it is an image, again, you'd block out some part of the image and then from the rest of the image, you'd try to predict that part that is self supervised method. There's also contrastive methods which are self supervised, which means that you'd have an image and you make two different views of it, for example, by cropping the image in different places. And then you try to train a model that can tell that these two things actually belong together, come from the same image, and that they are apart from, I'm going to draw inverted arrows right here, they are apart from like a third image that has nothing to do with this image. These are contrastive methods. It turns out that if we build models that learn in self supervised and contrastive ways, and especially in multimodal ways, that we end up with models that can explain brain activity fairly well. So we're going to jump into the papers right here in the interview pretty quickly. But if you keep watching the interview, Patrick goes also into more like high level explanations of neuroscience in general. It is a bit my fault that I immediately was like, so what does this paper say? But I promise you, if you keep listening throughout the interview, there are great insights into the entire field of neuroscience into what are open questions into where can people go to learn about this. And if you even want to research this, if you're in deep learning right now, and you're interested in neuroscience, this Patrick says it's a wide open field, there's lots of papers to be published. And the conferences are especially something like NeurIPS are pretty receptive to papers that connect deep learning with neuroscience, or in general, try to explain neuroscience, neuroscience things. So as I said, we're going to jump into the interview now, I don't want to spend too much more time because we're very detailed in the interview. Check out Patrick's blog and all his other endeavors. And I wish you a lot of fun. Bye. Hello, everyone today here with me I have Patrick Minow, who is a neuroscientist slash blogger slash anything else that you might imagine in between deep learning and the human brain. Welcome, Patrick, to the channel for this bit of a special episode, I guess. Thanks. It's great to be here. I got I think I'm going to say I'm going to say I'm going to say I'm going to say I'm going to say I got I got sort of knowledge of you for through your article 2021 in review unsupervised brain models, you wrote down what happened in the last year in terms of the connection of deep learning and how to let's say how to explain the brain. What is your what is your background in this area? How did you come to be in this in between space between neuroscience and AI? Yeah, absolutely. So I actually originally studied physics. And, you know, after my undergrad, I figured, you know, maybe I don't want to do string theory for the rest of my life. Like that sounds it sounds like some of the questions that to ask like interesting questions, you need to really be pretty advanced. But I think in neuroscience, there's some questions that are pretty right for the picking and that are obvious for even somebody that's pretty far outside the field. So for instance, what is sleep? What does it do? That's like a pretty easy question. That's that's very hard to answer. So I went to do a PhD in computational neuroscience at McGill. And one of the fields of my study was really that intersection of neuroscience and artificial intelligence. Now, when I started my PhD, which was in 2008, deep learning really wasn't a thing, I guess, like some of the original papers by Benjio and and Jeffrey Hinton had been they were out. But you know, the big event, I think, in presenting deep learning to the world and saying like, this is really this is a big deal was image in 2012. Right. As you know, so that was during my PhD. So at the very start of my of my PhD presentation, my PhD defense, I would say something like, look, you know, you have neurons in infratemporal cortex, which is one part of the visual stream, and they're able to do visual recognition. I would present examples of these neurons. And they're invariant. And to things like lighting, rotation, scale, etc. We don't know how to make a computer that does that. But if I gave this presentation, just you know, six months or a year later, I would never have been able to say that because people have been like, you know, you could just you know, like get even Alex net would would be able to do that. So so that's a little bit my, my story, my introduction to to neuro AI. So I was there like, during that transition, towards deep learning. And in fact, in the end of my PhD, I was, I was working on deep learning to try and explain some of the brain areas that I cared about. Now these brain areas are the areas of the dorsal stream. And those are like really brain areas that really care about emotion. And so I was poking around with what was I'm going to date myself, you know, I was poking around in the piano back in the day to to make this happen, which I guess has fallen by the wayside. But yes, I've been at this intersection for quite a while now. Awesome. Well, that it seems like it was an exciting time. I do remember the piano as well. So I'm definitely dated, dated the same. So you, the dorsal stream, just to make clear, that's part of sort of the visual, the visual stream into the brain. Is that correct? Or? Yeah, yeah. So maybe I can, I can give you like the first minute of my, my thesis defense. I've got it engraved in my brain. You just, you defended not too, too long ago, right? True. Exactly. So I'm sure you're gonna forgot it. Oh, yeah. Yeah, you just like put in the box in your brain and just, it's gone. Okay. So the visual information falls on the retina. And it's originally encoded in these very simple formats in terms of differences and luminance between like a center and a surround, or differences in time. So you can think of it as a camera with like a little bit of linear filtering. And it then gets forwarded to different areas of the brain, first to the lateral geniculate nucleus, and then to the back of the brain, the occipital cortex, which is called the primary visual cortex. So that's a huge area, huge chunk of the brain. And you have tons of neurons which are selected for vision there. And from from there, the visual processing splits into two different substreams. There's the ventral visual stream, which is the object stream. So if you think like, what does a, you know, ResNet 50 that strain on, on ImageNet do? Maybe it's something similar that we can get into that later. And then there's another set of areas, which is the dorsal stream. Again, organized in a hierarchical fashion. Again, you have like these, you know, for instance, you have increases in the size of receptive fields, you have increases in the size of in the complexity of things that these neurons respond to. But this time, they don't care about form, they don't care whether they don't care about texture, what they really care about is motion. So you know, you're going to poke at a neuron in, let's say the middle temporal area, which is part of the dorsal stream. And 80 or 90% of the neurons will respond when you show them the right moving stimulus. Yeah, which is, which is remarkable. So in your in your article, you go a little bit into both of these streams. And I think the one of the main focuses that you care about is, are or are the are or are not the deep learning networks we use today, similar to what the brain does, because sure, we've built these systems that can do some visual tasks. But does that bring us closer to understanding how the brain does certain things? And the answer is, right? The answer is a little bit yes, and a little bit no, like there's still there's still questions. But you point out a bunch of areas of where progress has been made in correlating, let's say, neural activities in deep neural networks with neural activities in in brains. So yeah, yeah, I'm, I think that it might be good to just back up a little bit and talk about the, you know, that world at large so that, you know, people are just tuning in. I haven't read the article yet. We'll understand what we're discussing. I think that originally, some of the, okay, so I was talking about ImageNet 2012, which was the big milestone in creating good deep neural networks that could solve the kinds of tasks that humans that humans can solve. Now there was a lot of background work that came into that. One is, you know, the creation of convolutional neural networks and the work from from Yan-le-Cun, which was ultimately, you know, inspired by the new the new cognitron, which is Fukushima, like in around the early 80s. But ultimately, that work was motivated a lot by some early work in vision and in vision neuroscience. So David Ubel and Torsten Weisel in the 50s and 60s looked at different kinds of neurons in the primary visual cortex, and were able to find that you have this this hierarchy of selectivity, right? So the canonical thing that they found is they found cells which were tuned for orientation, right? So you know, you present an edge like this or a line like this, and the cell responds. But if the line, if instead of being white, it's black, then it doesn't respond. So those are called the simple cells. And then they found another subset of cells, which are called the complex cells. And so those are selected for this, but they would be, it wouldn't matter the precise location of this line in question. And it wouldn't matter the contrast. So it could be white to black, or it could be black to white, it wouldn't matter. And so their hunch was that, okay, well, you have this this transformation that happens, first of all, you have a selectivity operation, which creates that simple cell. So basically just a threshold. And that's enough to give you a selectivity, or it could be a relu if you, you know, smooth it out. And, and then there's a pooling operation that happens. So you pool from different, from different simple cells that have the same orientation selectivity, but different contrast sensitivity. And that creates the complex cell. And you can view that as a subsampling operation or downsampling operation as you would have in a deep neural net. So there's this kind of long line of like, oh, there's the inspiration from the brain, we're going to make some models, we're going to show that it's that they're actually good enough to solve tasks that humans can solve. But the question is, okay, are these are these like really like, like human brains? So and that's similar work from from in Jim DiCarlo's lab and Nico Cricascorte in 2014, like really showed that there's some very tantalizing hints that this is indeed the case, you know, that these networks that we've trained on ImageNet, they look a lot like the brain in, in really interesting ways. And one of the big ways that, you know, they're similar is that if you have, if you look at, you know, let's say 10 different networks, and one of them is, some of them turned out to be a little bit better at solving ImageNet, or a little bit worse. And then you correlate that with how well you can align these networks to the brain, turns out that the ones which perform better on ImageNet tend to also perform better on explaining the brain, which is like a very strange coincidence, because think about how like, completely different these two things have been created. So that was that was one of the big hints. And I think like another big hint is the word from Chris Ola and other people at OpenAI that looked inside of these deep neural networks and found that, you know, the kinds of selectivity that you see inside the cells, they're very, very similar to what you would, what a neurophysiologist would describe in areas like V1, V2, V4, and temporal cortex. So the combination of the quantitative and qualitative tells us like, hey, maybe, maybe there's a kind of these are kind of like little brains, one very, very specific part of the brain, I want to be a lot of trouble if you say that that statement. Yes, exactly, exactly. So what do people mean when they say something like explains the brain or something aligns with brain activity? Like, what is it? What is behind that? Yeah, yeah, yeah. So we can talk about the high level stuff, like, you're sure just like the idea of look how like, what do we, what do we measure? Like, you know, is it a number? Is it a correlation? Or is it? Am I training a regression model from one signal to the other signal? Like, how can I make the statement that this neural network explains some function in the brain? So in the early work from from 2014, we see two different approaches being used. And those are the kinds of approaches, like every other approach that's been tried, is kind of a derivative of these like two basic concepts. So one approach is a regression based approach. So let's so very simply, let's say you train a ResNet 50 on image net, you chop it off at some layer, layer four after the first down sampling or whatever. And then you measure the output of that deep neural network with respect to some stimulus ensemble. So which gives you a big matrix big X, which has a bunch of rows for the different examples and a bunch of columns for the different features. And then you just regress that against neural data that's, that's recorded with the same, with the same images. So it's just a regression. So you can add like a bunch of different spices into your basic recipe. So you can add some some sparseness priors, you can try to well, usually you'll use a ridge regression rather than a straight regression, because that will definitely the other regular regression will usually crash and burn neural data is very noisy. That's something that people don't often appreciate. And so it's a regression. Let's just put it that way. Yeah, now that will be sort of, for example, f MRI data, when we talk about neural data. It can be f MRI data, it can be MEG data. So magnetoencephalopograph, encephalopograph, I think we just say MEG. And or it could be a single neuron recordings or array recordings. So those are taken inside the brain, or it might be ECog, which is just on the surface of the brain. So there's different kinds of recordings. Now, it happens that f MRI and MEG are much more popular for for humans, because it's it's, it's non invasive. But every once in a while, people get to record inside of the brains of humans that have some some sort of need for brain surgery, whether it's usually it's epilepsy. And those data are very precious. Now speaking of so you go through different papers in your article. So maybe we can follow that structure a little bit. The first one is a work that shows that the ventral stream might be explainable by and your idea, your the article also goes into it's called unsupervised unsupervised brain models. So your your kind of point that you make is or your investigation is into unsupervised systems, like what, what, what, how good or how close to what the brain does is comes from the self supervised and unsupervised system. So the first, the first, the first thing you go into is the ventral, sorry, the ventral stream, that is you set the sort of object stream. And this paper looks at single neuron activations, right? And the they find that the self supervised systems can be or are equally or even better able to explain the brain data than supervised systems, let's say in an image recognition task. Yeah, so that's super exciting. And the reason is that I think that everybody got very excited when they saw that these networks which were trained for image net, they could be aligned for to the ventral stream to that object recognition stream, because now it's something that, you know, you have this in silico thing, and it kind of looks like it does the same thing as the brain. And so it's kind of a model of the brain. Super exciting, you can do a lot of things with it. But there's different ways in which something can be a model of the brain. And some of these are a little bit more useful than others. And one of the ways I one of the big flaws, I think, for for supervised learning is that it's not like really a way it's not really a model of how the brain would learn a task. Because, you know, I'm not walking around as a baby. And like, you know, my, my parent just tells me like, dog, dog, dog, dog, dog, cat, dog, dog, just like constantly for years and years. So you know, we don't really use unsupervised learning for for for learning these kinds of things. So that's a big flaw that if we want to go move forward with models, which are biologically plausible instantiations of creating these, these models, then we have to move away from from supervised learning. So people generally like unsupervised learning and self supervised learning better for that reason, because you don't have to, you know, come up with this like, weird concept that you have dog, dog, dog, cat. And and but you do have to do the math to make sure that it actually does work out in practice. And that, you know, the right the kinds of the quantity of examples that you feed into, into the model is similar to the kinds of to the quantity of examples that you would feed into a human, for instance, I think you have you have a so your conclusion, you have a little bit of an example that it would like the language models that we train such as GPT three would be equivalent to like, years and years and years of of human, just constants, just talking and talking and talking and talking and babies are able to do it by age, what four or so or two. Exactly. So, so I think that there's still a big gap there that comes from that you still I mean, we're off, I think I calculated we're off by four orders of magnitude in terms of the efficiency. But, you know, I'm to score everybody on the same kind of curve. I mean, the GPT three is not made as a model of the brain minutes made as a language model. And to solve all these these problems in zero shot settings, and it works very well for for its purposes. But definitely, if we want to actually try to explain the brain, we'll need to get to that. So this, this, the, it is also a bit special, because we hear we talk about the ventral stream, you said that's the object stream. And the fact that self supervised systems are equal or better at explaining that than supervised systems, which presumably are trained exactly on the task of that such an object stream would be sensitive to right, that is also one special thing. So I totally agree. I mean, that's super cool that that this is the case that you have this, this thing where you don't give it like learn objects, and yet it learns something that can do can do object recognition. And it learns meaningful, meaningful things like that. But I think that there's a couple of hidden assumptions there that make this not nearly as mysterious as it was like, as we would like it to be. So one is that, you know, image net is not really if your model of image net is not you take like a, like a nice Canon DLS, the DLSR, and, you know, you, you put it at a random point in space, and then you point it at somewhere random, and then you hit the button. Right. So if we look at both of our faces right now, we're in the center of the screen, it turns out that, you know, we're smart like that, that we place our faces, like generally in the center of the screen when we take photos. So the things that we try to look at in image net, you know, the the subject of the category will by and large be in the center. So, and you know, the position of the camera, the things that we that we tend to measure, I mean, these are all these all come into why the model learns the thing that it learns. So it's not, it we can't really say, oh, it, you know, we're not like really feeding it any, any structural priors, we definitely do. We definitely do just, just in not like the conventional way, and not in a way that's very easy to quantify either. But some people are definitely trying to solve these, these problems. So, so for instance, there's a lot of work on trying to fit the same kinds of unsupervised learning models, but with streams of data that look more like what a baby would see in their early years, when which the camera is not always pointed that at the right things, because babies tend to, I see. Yeah, do a lot of gesturing. But it's also, it's also there, especially because the baby with time is able to move its head, right. And therefore, it's also not the same as just placing a camera somewhere because whatever captures attention will be actively looked at more. So it's, it's definitely like, I think there's a long way to go in any of these things. Oh, yeah. Oh, yeah, absolutely. I think. So to close the, the, just that one paper, because we've been on it for like 15 minutes, but super cool that you can have, you can train a model in a unsupervised or self supervised manner. And it turns out to be just as good at explaining, you know, V1, V4, and IT, all these different sub areas of the ventral stream. And then there's a kind of eriarchy that happens between the different, the different models. So, you know, some models are clearly doing better than others. So typically in these papers, SimClear is usually the one that performs the best for reasons that we don't totally understand. Local aggregation also tends to, to do better. So that's interesting. Like, what is it about what's inside of these models that can, that allows them to be more similar to the brain. Now, of course, in the end, you know, you end up with like tiny, tiny error bars, and it can be pretty difficult to actually differentiate between these, these different things. So, you know, you can't like read too, too much into it. But definitely the best models are like the new kind of generation of self supervised models. And then so the next paper deals with the with the with the other stream with the dorsal stream. And there you or yes, that is actually you who found some that's your own paper, right? Oh, yeah. So, so I'll just go very rapidly with true that actually the second one is ventral stream. Oh, sorry, again. And so that's from Talia Conkle. And very, very consistent data. So they use fMRI rather than single neuron data. But I mean, the data is like these two studies were done independently, about a kilometer away from each other, one one team from Harvard and one team from MIT, and they found exactly the same results. So maybe some things in the water in Cambridge, Massachusetts. But otherwise, I mean, it's a very robust finding, basically. But yeah, we can definitely talk about the dorsal stream. So, like I said, I've been interested in this problem for a for a very long time. And I had a little bit of time during the the last lockdown of the pandemic to to relook at this problem. And so we sat down and we said, you know, this I think like the time is right to really look at all this dorsal stream data and see if we can get if we can get one really good model of all these these different areas. So the first thing that I did actually is I was going about this very naively, but I just looked into like the torch vision models, you know, they have like some some model database, and just downloaded all the models that were trained on video recognition. So all the models that were trained on I'm drawing a blank here, kinetics 400, which is a task where you have to look at a video of somebody juggling and say, oh, it's juggling rather than unicycling rather than soccer or whatever. And so the special thing about these models that they look at 3d data, by 3d, I mean spatial temporal, right in time. And so that means that and generally they're trained, the convolutional neural nets, they're trained with 3d filters. So, you know, the front end of the model is going to be a 3d convolution in space and time. So I looked at these models, and I did the kinds of visualization tricks that Chris Ola and gang do it, I open my eye to look inside because I was curious, you know, do they learn motion? Do they align with with the brain? And I found that they were actually really terrible, which surprised me, because if you look into the methods of these papers, it's like we trained, we trained these models for 24 hours on a supercomputer with, you know, 16 GPUs in parallel, and went through, you know, a million videos. And this is the model that we obtained, and they're very good at doing the tests that they're doing. And yet, the kinds of generic features that come out of the models are really terrible at aligning with the brain. So that was kind of the hunch that we saw there that I should say that the one of the early findings and one of the early points that people who are dubious about the finding that the ventral streams align with ImageNet trained ResNets and AlexNets and VGG nets, is that people say, well, you're just training the model to do a task, you know, any sort of task will work. It doesn't matter whether it's object recognition or whatever, it just turns out that this is the task that you had data on. But this is a very, this is a very good like counter example of that, because you train a model on a task which involves, you know, 3D data, video spatial temporal data. And yet, that model is actually the model that you that you train is really good for that one task, but is really terrible at this task of aligning with the brain. So that motivated us to look more deeply into, you know, what else could, like if we don't train, if we don't take, you know, pre-train models to solve this problem, like what could we do? And we know that a lot of the dorsal visual stream is really cares about navigation. So if you look at an area like MST, have you ever had Vertigo? Sure. Yeah. So Vertigo is like kind of sorry, this is like a weird non-secret, but Vertigo is kind of a funny thing, right? Because it's an inner ear problem, right? So you have your vestibule, and it kind of, it basically tells you there's acceleration in ways that there shouldn't be acceleration. And that gives you an impression of being dizzy. But also gives you like these weird visual effects. Yeah. Right? Which is strange. Or, you know, if you drink a little too much, you might have that same kind of feeling. So there's an area in the brain, which is called MST, which has these neurons, which receive both visual input and vestibular input. And the way that they receive visual input is they have a lot of selectivity for things like rotation and expansion and and wide field translation. And so we think that they're really involved in navigation. So if you're going forward in a line, you have these neurons, which receive both the vestibular input. So they know how you're accelerating and where gravity is. And they receive all this wide field optic flow, which is tells you where you're heading. So we said, why don't we train a deep neural network to solve a navigation task so that the network can can orient itself in space, essentially. So I used an environment, which is it's an environment for drone simulations called AirSim. And it's really fun. So it's an Unreal Engine. And you can, you can basically fly a drone in these suburban environments and back out the sequences of videos. And then you can train a convolutional neural net, 3D ResNet, to solve the problem of figuring out what is the from a little sequence of movement, what is the trajectory, basically, that's going on, like where are you heading? Are you rotating? Are you going forward, etc., etc. And so if you train a network on that, it turns out that if you visualize the cells inside of the train network, they really, really look like what you would see in the visual cortex. So as a neurophysiologist or as an amateur neurophysiologist or a person that's been in the vicinity of neurophysiologists, I was really, I was really stoked to see this. So you see these cells that are selected for translation and translation, but they don't care about the pattern that underlies the translation. And in particular, you see these cells like the one that you're visualizing here that like things like spirals in some of the higher level layers of this network, which was super exciting because those look a lot like what you would see in a... So basically, the networks that try to just predict anything from a video that contains motion weren't like turns out these neural net, sorry, the deep networks, I have to stop saying neural networks here because it's ambiguous. Ah, yes, yes, yes. The deep networks that train on any kind of video data, they're not super well aligned with the brain. However, as soon as you go maybe to like some sort of an ego perspective, right? And you especially you predict your own parameters of motion. So from the visuals you're trying to predict, okay, I went to the left, I went to the right, I turned around from the visual information. And that turns out to align very well with the brain data. Does that make like, just maybe an esoteric question, but does that say anything about the need for AI to be embodied? Maybe? Oh, I love this question. Yes, 100%. Yes, we should, we should completely embody AI. Yeah. So I think that one, one big question that came up during the review is that, you know, we claimed originally this was unsupervised or self supervised in the abstract. And then the reviewers came back and said, well, it's not really unsupervised or self supervised. It's a supervised network because you know, you know what the answer is, you're just training in a supervised fashion. My feeling is that it is self supervised in the sense of when you embody this in an agent. So when I'm when I'm a baby, let's imagine that I'm a baby. And I'm walking around the world, I have some control over where I'm heading. Yeah, right. So I can say like, I'm going to turn this way, I'm going to turn that way, I'm going to move forward, I'm going to go get that cookie. I'm going to look at my parent, and so forth. So I am an agent. Yeah. So that means that I control the motion that comes into my eyes. Yeah. Because the vast majority of motion that we see in the world comes from from our self motion. And so I can correlate my motor plans with what I see in the world. And that means that it's a much easier kind of problem to correlate these two things, then to say I here's found data, which is the case of ImageNet, and figure out something to model with this. Yeah, exactly. Right. Yes. You also have this diagram here from young Lecar, talking about self supervised learning. And it seems very much that it is I agree, the line is like gray in some places. But it seems like if you are an embodied agent, you always have those motion parameters ready, right. So it's much more like I am going to darken out part of part of what I already know and try to predict that from it, it seems it falls a lot into this into this diagram right here. Yeah, absolutely. So I think it looks more like the bottom part of this diagram that you see there, where you have these two things which are happening in the present, but one part is occluded and the other part is visible. So you're doing multimodal masking, in other words, right. So you have the vision, but now you're trying to predict the vestibular, or you have the vestibular, and you're trying to predict the vision. And so if you look something like clip would be, I think, like maybe the most popular model that's of the same kind of multimodal kind, you can say, well, clip is a supervised model, because you're trying to predict, you know, in a way, you're trying to predict language from vision. But it's really this kind of masking. And I think it's a more general approach to solving this type of problem. So yeah, I agree with you embodied agents, I'm 100% on board, they're definitely going to be awesome. And actually, questions about, you know, what do reinforcement learning agents learn? Do they learn like good self motion representations, for instance, when they're when they have a visual task? I think like those are super interesting, like, what do you need to put in there? In order to get that that effect? Yeah, that that concept of me in a eyes is not yet really come through so far. But I'm also looking into like, I'm looking forward to having more of a eyes who understand the concept of, of me and to be embodied and and and sort of to have self self state and all of this kind of stuff. I think that will bring us forward. So here in the next paper, you you tackle not I mean, this this paper you're describing, it tackles the question. It is actually, it is actually, I just saw in my notes, that is, again, one of one of your papers. It is the question, why are there even two different of these visual streams in in the brain? Like, it maybe makes sense if we if we sit down, but also, you find some actual empirical evidence for why it might be might be that we even have two streams, right? Yeah, yeah, absolutely. So I think that's a that's an interesting question, like, why are there two things rather than one or four things or eight things rather than than an arbitrary number? So, so Shahab, who's the first author on this paper, worked on looking at what it would take to to recreate both ventral and dorsal stream. And I think the remarkable thing that he found is if you train a network like CPC network, so a contrastive predictive coding network, which is one form of self supervised learning, in which you're trying to essentially discriminate between different futures, if you will, so you're trying to you look at the past, like a certain window in the past, and then you're trying to tell apart like the actual future embed in some subspace versus an alternative future, which is which is dreamt up. So if you try to do that, then, you know, it's already been shown that you can find good representations and in videos. But what's very interesting is that then you can ask the question of what happens as you add more and more substreams inside of this of this network. So if you remember the original Alex net paper, so it did have two streams. So if you remember, like very, it's like a while ago, but what happened is that they had like tiny GPUs back in the day, right. And so they couldn't fit the whole model on just on just one GPU. So what they decided arbitrarily is to split it up into two parts, especially at the at the early point. And then basically, they so they were independent, but they could re communicate a little bit later on. So which was a pretty unique feature. Back then, people didn't really do that. But now it's it's quite common to, you know, chop up the channels in different ways and all sorts of things. But what they found is that there's this this this very interesting self organization principle where all this all the the filters on one GPU turned out to be color selective, and all the filters on the other GPU turned out to be to be black and white, which is whoa, that's weird. Just by the fact of splitting up, because the two streams, they don't always communicate, right, they only communicate at very sparse intermediate points. So so just structural prior gives rise to something that very much looks like the brain in that in the sense that one of the streams correlates well with the ventral brain stream and one correlates well with the dorsal brain stream. Yeah, so in that in that case, in the early Alex, that paper, actually, both of the types of filters are different subtypes that you see in in V1, but they are, you know, functionally different, and they have different roles. But it was like kind of an interesting proof of concept that if you just set a separation, arbitrary separation down the middle, you don't say anything else like you don't say like, you have to respond to color, you have to respond to this. But just you set a separation, it self organizes to something that's interesting. It's crazy. And yeah, it's weird. So they might have just locked themselves into like building a better model by by having two small GPUs. Yeah, exactly. So, you know, they say that necessity is the mother of invention. So I think this is a particular case where, you know, the limitations at the time caused them to stumble onto something which I think is is really deep and interesting, which is symmetry breaking. So I guess ultimately, you know, when you start with, okay, you can imagine that if you just set all the weight parameters to zero, and then you perform your gradient descent, these two filtered sets will learn exactly the same thing, or they'll crash and burn. But by adding a little noise, right, by initializing your your network, you're pushing the network very, very slightly out of equilibrium, and that's enough to self organize into this thing. And so Shahab found a very similar phenomenon in the context of these networks, which are trained in an unsupervised manner in CPC. And so being trained on videos was able to find that these parts of the one part of the network was and so again, this is this is an instance of a network that has kind of a firewall in between the two sets of filters. And so he was able to find that these two sub branches, one of them was dorsal like and the other one was ventral like, and was able to correlate that with some some data that we have in in mouse where there's tons and tons of data on what's the relative selectivity of these different things and found some some really nice correlations. So that means that you can all you would need basically is a little bit of a nudge, right. And so so which is this great idea, like maybe you just initialize the network in a sli so that like the two things are just very slightly asymmetric. Because one thing I should say is that the the two networks don't always get the same label, right. So if you train the network twice, one time it's going to be dorsal ventral and other time is going to be ventral dorsal. Whereas the brain every time that you train it, it's the same that we know. There are some exactly it's all ventral is ventral dorsal. So there's some like inbuilt asymmetry. But it's a very probably like a very small asymmetry. Because if you train it with real data, and then it will automatically, you know, self generate into this in bloom into this particular activity. Cool. So very excited that the brain can organize itself for something that's that's useful just from this could be used, I guess, for I mean, people are already, you know, in multi head attention, they do multi head, right. And that's kind of similar in that they they clearly separate different computation that cannot interconnect. And therefore, that that sort of there also, like the random initialization probably does some symmetry breaking, and then you find that the different heads respond to different things, people have investigated that it's probably very much along the same lines. So I want to skip ahead a little bit here to the the the the the concept cells, the the is it this paper? Oh, that's this as well. I think like, I think that there's been a lot of movement in the subfield. And by the way, I want to tell your viewers because I know a lot of you viewers are coming from a machine learning background versus an neuroscience background. And, you know, it's hard to get into NeurIPS. But I think if you know, it's such a wide open field in neuroscience. There's so many questions that if you care a lot about representation learning, you know, it's it's a pretty easy field to jump onto, and, and have positive reception. So there's there's still a bunch of a bunch of questions. So grab your nearest neuroscientist and go write a paper. Encourage everybody to do it. Yep. Definitely how to how to hack how to hack publications. There you go. Yeah, there you go. So yeah, so clip clip is clip is weird. So if there's one thing that I would say is when we saw when we saw the results of of clip and and some of the both in terms of of how good it is, and also the inner visualizations that Chris Olin gang worked on Chelsea Voss, as well. I think that we were all kind of surprised because they do look a lot like the kinds of concept cells that you see on the hippocampus, right. So the very, very, very, very famous paper that did this is the had the infamous Jennifer Aniston cell. So I don't know if you're in your only in the context of your article. So it's one one cell that responds to both what pictures and the name and various aspects of a person not not just like, exactly, exactly. So if I remember correctly, this this paper, so they had, they had people with intractable epilepsy. So these are human patients, and they were doing pro recordings in the hippocampus to figure out what was the the nature of their epilepsy and how they could be treated. And, you know, they spend a lot of time in the hospital just being bored. And so sometimes they enroll into experiments and these experiments tell us more about the human brain than is otherwise possible. And so very thankful for for these people that do this. And so in this particular instance, they, they presented different kinds of concepts and images. And one of the cells that they found that have this like amazing property that if you just show the words Jennifer Aniston, it would respond. If you showed the face of Jennifer Aniston, it would respond. If you showed, like I didn't do like other kinds of controls, but I imagine that if they had played, and you know, that the start of the of the French show, it probably would have responded, because it all came with this like general concept of, of Jennifer Aniston. So ever since then, people have been like fascinated by this idea, although it's a much older idea, you know, this idea that you have like a cell in your hippocampus that responds to your grandmother, it's the grandmother cell idea. But one thing that was very interesting when we first saw clip is that you have cells can respond both to text and to to images. And in fact, you can do these new kinds of adversarial attacks in which you just write the wrong, write the wrong text. And it fools the wrong text. And it fools the system into actually reading the text and mislabeling the the images. So it sounds very hippocampus like to me. And so in this particular paper, they, they actually looked at at this problem and found that out of all the different models that that they could look, they found that clip could explain the most hippocampal data, which is super exciting. I'm sure that people are really going to drill down further into this, into this finding. Yeah. But it's clip specifically, because there's a lot of other unsupervised models. And somehow clip is the best and we still don't understand why this is I mean, it's like the delta between it and the the second best model is, it's huge. But why? I think no one knows right now. And and actually clip the the just the the the visual aspects of clip are also very good at explaining some of the some some other data. So it's, it's very interesting to think about what happens in a multimodal fashion, like what happens when, you know, experimentalists and neurophysiologists like really like to isolate one thing to one thing to just look at one thing at a time. But now you're talking about something that can do different kinds of modalities. And I think that, you know, multimodal areas are going to be some of the next things that are really attacked by unsupervised and self I mean, it's also a question, I mean, clip is huge. It also has a huge amount of data. We don't exactly know what data went into there, right? There's a lot to to untangle here. But the multimodality, I also feel that that is, is a big part of what's going to bring us forward in AI. And probably also, you know, since the brain is always multimodal, like, I don't you don't get like a stimulus that is maybe now with computers, you do. But you know, just growing up in nature, you probably get zero stimuli that are just unimodal, right? So you're always in this mode of multimodality. Yeah. And in one thing that's, that's interesting, in particular for babies, you know, if, if you ever interacted with babies, they really like to have toys, which make lots of noise, which drives parents crazy. And but I think that there's a reason for that, right? Like, why would you want to like a toy that makes like a lot of noise, because clearly, there's a lot of pressure on making the noise as silent as possible, because the parents are just like trying to sleep. But I think that the kids just prefer that because it's a multimodal stimuli. And you can do all sorts of causal inference about what happens when I get this thing with this thing. So this is the last paper that I I wanted to look at, maybe maybe you have more, but this is, it's challenges, the manifold perspective of deep learning in your you've described it a little bit in the paragraph, you say challenges, the manifold perspective, and it favors the causal perspective. So what is meant here? And what does this paper tell us? Oh, yeah. So you remember, we were discussing earlier, the mechanics of how you compare a brain area and deep neural network. And so you could have so I think a lot of deep learning methods are rotation invariant. So if you take something like clip, for instance, you're learning, I guess, like this, this subspace, which is, I guess, like 128 dimensional in the both from the visual side and from the text side, and you're trying to align it in this 128 dimensional space. If you multiply the two by rotation matrix, and then the entire 128 dimensional space gets gets rotated, it's the same network, right? It really doesn't matter whether it's, whether it's rotated or not. What matters just the locations on the manifolds. And so if you're thinking about aligning a brain area and neural network with a with a regression, again, the rotation doesn't matter. You're saying any any weight matrix is just as good as any other weight matrix. So that's the so So that's the so that's the underlying, I think, assumption. And I think that there's been a lot of work recently in neuroscience, focusing on this idea that, you know, single neurons like don't really matter. What matters is the latent subspace in which the near the neurons are responding. So if you have a population of 100,000 neurons, maybe they Yeah, it's 100,000 neurons. But if you present a bunch of stimuli, you find out that actually the latent sub and you do like an SVD on the matrix of responses, you find that latent subspace actually just five dimensional, or whatever. So first of all, they're just random projections from this five dimensional subspace. And the and the large dimensional subspace doesn't really matter. So this paper, so sorry, sorry, and it's been a lot of work in neuroscience showing that this is the case, especially in, in motor cortex. So you know, you have tons and tons of neurons in your motor cortex as you're going for for reach movement. And yet it seems that these neurons really live in a very low dimensional subspace. So that's what we call the manifold theory of neuroscience is that idea that the neurons are in a high dimensional subspace, but they're just project random projections of some lower dimensional subspace. But one of the consequences that if it's random projections, then each of the neurons individually should just be, you know, weird. It should, you know, respond to a bunch of different things, it shouldn't be shouldn't be able to place a label, because you could like neurons, you could rotate the entire space, it would still make sense, right? So there's no, there's no reason why an individual neuron should align with just like one axis in, in that particular subspace. Yeah, exactly. So, but neuroscientists really like labeled axes. That's one thing that they're very fond of. So, you know, you can imagine that you have like an axis, I don't know if you're in Unity or Unreal, you know, you have like my avatar, and then you just like hit like one switch, and I just go, you know, it just, it just changes my smile from from upwards to downwards. And oh, sorry, I, my printer is haunted. And so I'm just going to disconnect it, if you don't mind, because it makes the lights flash. Unfortunately. Okay. I find it weird that printers are like the oldest technology on the planet, yet still they're like the most troubled, like we should we should have figured this out by now. But we have not. Yeah, it's, it's too bad. So I still print out papers, because there's been research that shows that you retain more when you print something out rather than when you read it in the on a printed document rather than Yeah, reading it on the but it's just becoming so, so inconvenient that I think I'm gonna have to abandon soon. Okay, so starting back then, and I apologize, where do you want me to restart? So um, we, yeah, there's no there's no particular reason why any single neuron right should align with any axis. Yet people find that they do. Yes, yes, exactly. And that might be because, you know, neuroscientists like to name things. And if something is not nameable, they'll say it's mixed selectivity or whatever, and then they'll just forget about it. That's also a very good assumption. So both of these things can be happening at the same time. But in this paper, they found that if you train a bit of VAE, which is a VAE, which has a stronger weight on on one of the KL terms, it tends to find disentangled representations, right, so that the axes actually matter. So one axis is like my smile, the other axis is how much of a unibrow I have, and you know, a third axis is, you know, what's up with my mustache, and etc, etc. And so they found that that aligns pretty well with some neurons in one face selective area of infotemporal cortex. And so they did some some trickery trying to do like one on one alignment versus ensemble alignment. And it looks like, you know, the good interpretation for this data is that it's, it's more like a one on one alignment. And so that could be pretty interesting. But I do want to point out that there are certainly distributed representations in the brain. It doesn't mean that because in this one area, you have non distributed representations, that that's the case for the whole brain. And it might be because of energetic reasons that we have this representation in this in this brain area. Because you know, you want to have how the what the distribution of responses is over a stimulus ensemble is very important for how efficient the code is, because remember, neurons are super noisy. Right. So you want them you want to have like a nice exponential distribution of responses in order to have an efficient code. Given that you have this personal like noise in the data. So yeah, and you you say it favors the causal hypothesis, it so it means that maybe what's happening is that rather than some simply encoding the signal that you see that the brain is actually building like a causal model of what's happening, like you know, there are eyes and there are eyebrows and that, you know, the the result of there being eyebrows is that they look a certain way. And then it will make sense again that they are encoded, like the structural priors encoded in one space. And then simply the manifestation of that is the picture we see. Yeah, yeah, maybe I misused the term causal here. I don't want to mistake it for causal inference. And I don't want to misuse the term causal inference. And sure, sure. But I think that what I mean by this is a forward model for how like one individual. So you can think of you can think of a of a directed basically graph in which, you know, there's a bunch of different factors. One of them is whether or not I wake up with a mustache today. Another one is how close my eyes are. Another one is my nose. And these factors are disentangled. So that means that they're independent from each other. And then I can just like turn on and off the switch and generate different faces. So that's I think like the underlying naive model is the Mr. Potato Head model, right, in which you just like switch out the different components. And of course, there are specific holes that you can put the different the different things in. So I think that I guess like the question is, like, are these factors in this this factor graph? Are they like, can you put labels on them and they correspond to one thing that we would identify as something that is independently changeable? So for instance, like, we understand that age and lighting, for instance, like those are two totally disentangled things that have nothing to do with each other. So the question is, are they are they different factors? Or you rotated like one is square root of two, like one over square root of two times age minus one over square root of two times lighting, and so on and so forth. And it looks like they're really aligned towards the factors that we can label, and that are indeed independent, both in brands and in this particular model. Do you think that it plays a big part that it because face, let's say facial structure, is it is something that is truly, let's say the individual factors are actually independent because of, you know, genetic variation, allele crossing during during meiosis, sorry, or recombination, and so on these things actually go in a fairly, let's say, this uncorrelated uniform distribution in the human population. So almost every combination of narrow eyes, wide eyes, you know, big mouth, small mouth, and so on is possible. And therefore, it might make just sense to let's say encode the individual factors as individual neurons, as you say, maybe for energetic reasons. I think that that's, that's a really interesting hypothesis. But I don't think that that's that that's the case. I think that there might be like a general, you know, algorithm that makes it that tries to disentangle these things into into different, into different sub factors. And then as a consequence, there's this natural alignment with this other process. But, and of course, if it's the case that the kind of latent model that is inside the brain is better aligned with the latent model that's in reality, well, that's better. You know, you want the thing to reflect, but I don't think it's 100% true that, that these that these factors are really disentangled in reality. So for instance, you know, I, I, like a unibrow versus mustache, like these two things are probably pretty correlated with with each other. Yeah, yeah, yeah, I see what I see what you mean. Yeah. So we're we're we're we've been we've been going through this a little bit. There's all I mean, there's a lot of there's other papers, which which are definitely also interesting, like the gloss ones is super interesting. Is there Yeah, is there one that you wanted to touch on particularly? Well, I wanted to give for, you know, readers that are coming slightly outside of this field, and moving into this like very rapidly moving field, kind of an overview of what are the questions that people are interested in, like what are kind of the some of the interesting approaches that people are using to, to tackle these and also encourage people to come in our field and, and, and, and, you know, get papers in and, and scoop us basically. So I really want to encourage people to, to get into that. I think, I think that we've covered some of the papers that I think are the most interesting. And we'll see in the, I actually wanted to do a follow up on precisely the kind of agent based representations that are coming because that that is coming down the line. And I think that's going to be super interesting for this field. So maybe we can end with like, some things to look forward to in the future. Sure. So one of the things that I think is going to be interesting for for the future is like really taking evolution seriously. So we saw the, actually maybe if you can scroll to where I show Jess's, Jess Thompson's diagram of the different types of, of models and how they all fit together. It's at the very start. It's at the intro. So Jess has a really nice way I think of, of explaining this, which is that, you know, there's some models which can really perform a task. And, you know, once we got to ImageNet 2012, like that was, that was where we got there. And then, you know, in 2014, we really got into this accounts for neural activity part of, so, you know, we can find models that can both perform a task, which is biologically relevant and accounts for neural activity. I think this year was a big year for biological plausibility. And I want to say this is the last word, because clearly there's way more work to be doing there. You're going to have models which have realistic, biologically realistic kinds of gradient descent, or replace gradient descent with something that's more biologically plausible. You're going to have Dale's Law, you know, so excitatory neurons only make connection, only makes excitatory connections and inhibitory neurons only make inhibitory connections and you'll have normalization and you have temporal dynamics and so on and so forth. So that's like, the next five years is probably just going to be to fill in this biologically plausible. But there's also could have evolved. I think that that's that's like a super interesting unknown questions and people are going to start to think about this problem in a serious fashion. And I want to point out there's this there's this recent paper that I don't talk about here, which from Fei-Fei Li, which is about evolving different kinds of agents that can solve different kinds of reinforcement learning tasks that actually has a an interesting evolution component to it. So I think we're going to start to see and we can actually like see the process by which the brain can bootstrap itself into existence, which I think is going to teach us something about what it is to be human. And I'm sure there'll be TED Talks and books and so forth. But that's going to take like another five, 10 years. Another thing that I'm excited to look at in the in the future is I just wrote my notes here hands. Hands are great. Hi. I think that one one one thing that we that we're having like really taken seriously so far is the role of weak supervision from a parental perspective. But if you think of like a parent and their baby, they're going to point at things they're going to say this is this, this is that. And you know, it has had like hands have had a huge role in our evolution as as homo sapiens. And it's even like thought that sign language preceded the appearance of voice speech. So that we probably have somewhere in our noggin, some areas which are highly selective for hand gestures, and which are used for a kind of weak supervision. That's important for for parents. So understanding what happens with that personal space and what what happens as as we use tools is clearly important from like just this that curiosity of how you know, we went from Australia to get the techies to the modern humans. And I think it's going to teach us a lot about yeah, what it means to be human. Awesome. Last question from my side with you're clearly interested in how the brain works, right? And and see and seeing, you know, can we can we make parallels between AI models, like deep models and brain areas and so on? Do you think that it is a necessity that we sort of feed back the knowledge into the deep learning realm? So should we should we put more effort into saying, how does the brain work? Okay, let's do that. Because at least that's that's like one example of where intelligence was achieved. Or do you think that, you know, how the brain works is just like a happenstance of nature and evolution and energy restrictions. And, you know, it's not it's not super like, let's just do AI, you know, the way it works best, or option three is something like, what, however we build AI, if we solve the task, it will automatically align with the brain, because there's like only one real way to solve the task, like in which, in which of these, let's say camps are do you find yourself in? Yeah, that's a that's super interesting. And I want to say that so people have made for a long time that claim that if we just study the brain, we'll be able to make better machines. Yeah, so that that comes about and again and again. And I do want to point out that this actually did happen, as we saw with convolutional neural networks, and the whole story of Yubil and Weasel and the Neocognitron and Yalda Kuhn and and eventually ImageNet 2012. But, you know, it's really only happened a few times, it's not clear how much more we have to like how much how many more instances of this will happen. That's certainly the view from from some people at DeepMind, for instance, that have really like gone into cognitive neuroscience and have started to do their own fMRI experiments to really, you know, tackle these problems. I think it's really, really interesting. But I'm not I think that it's going to teach us a lot about the human brain, but not necessarily about how to make intelligent machines, because we're, you know, like these are different systems, as you point out, and there are certainly things about the brain which are kludgy and, and, and certainly suboptimal. So how the retina is wired up is the classic example, it's wired up in the wrong way around, octopuses have haven't the right way around, and it doesn't seem to bother them. So that's a that's a clear example. But maybe there's some thing that we can that we can identify with with brains and that is going to unlock the next generation of machine learning. Maybe it's spiking neural networks, for instance, you know, people are demonstrating like, you could get something which is the like 1000 times or 10,000 times more energy efficient if you just use these mixed signals spiking neural networks. So I don't know. Yeah, that would I mean, 1000 times 10,000 times that is sort of the orders of magnitude you spoke about before when it came to to data. Well, those are so here, I'm thinking about the energy efficiency. So like one recurrent super comparable. No, I think like the the one thing I would point out here is that if you look at all these papers, and you add up all of the their, their training time and carbon emissions, it's it's probably like pretty substantial. Although I will say that, you know, the paper that that I'm the first author of here actually have the machine that I train this thing on like right here. And it's it's still like it's still a one GPU machine. So again, I encourage your your your viewers to to get into this because you can still do things with GTX 1080. That's awesome. But I think that one thing that's that's going to be really interesting is that by studying, you know, better machines, we'll be able to start to understand how to bring this back from the side of machine learning and bring it back into human health. So that's very interesting. And it's by and wide, hasn't been explored thus far. But that I'm kind of a fan of the opposite direction that most people are really going into. So I hope that that answers your question. I, I don't think that naturally, if you just train on your own network to solve a task, it's going to do it the same way that the brain does. But I think that's the brain does because I don't think that that's that's really pointed out. I don't think that GPT three does things the same way that a human does in any sort of meaningful way. No way. Even though they're both very good at language. Yeah, maybe GPT four. Well, if you ask Gary Marcus, he'll say that there's no way it'll never happen. Neurosymbolic AI all the way. Yeah. All right. Cool. Yeah. For every to everyone. Follow Patrick. The many he's written papers, lots of papers. You're also the CTO of Neuromatch Academy. Is that correct? So I, so I helped Neuromatch start actually, so I'm no longer CTO there. But it's a great occasion for for people that want to learn more about that intersection between neuroscience and artificial intelligence to to bring that about. So when we started this a couple of years ago, we just figured, oh, well, do a few video lectures and present that online. And it was at the start of the pandemic and people were bored. So the response was out of this world. So we have we had over 2000 applications and people from all over the world wanted to learn more about both neuroscience and artificial intelligence and their intersection. So we ended up having, I think, 1700 students in the first cohort and having 200 TAs. And so it became a big thing very fast. So I'm very happy that I helped bring that about. It was definitely one of the most stressful times in my life. But we could bring together people from very disparate backgrounds, whether it's people in emerging economies that are at local universities there, and people from from Ivy League universities in the US, Canada and, and the UK together and working with the same curriculum and under the same circumstances. So which was very cool. And then last year, we did the same but doubled in size as well. So I hope that we'll be able to, to double this year. I'm sure the announcement actually for for the next version of Neuromagic Academy will happen pretty soon. So if you have people in in your audience that are interested in that, I highly recommend to them to do that. It's a great occasion to learn. And we already have, you know, materials from last year online. So if you want to get started on your learning, you can do that today. Excellent. Cool. Well, Patrick, it was wonderful, wonderful having you here. This is a new world to me and I think for to a lot of people listening right here. So thank you so much. And I hope to see you again with with next year's review. Awesome.
 Hello there! Today I'm interviewing Patrick Minot, who has a PhD from McGill and did a postdoc at UCLA.
 He's an independent scientist and a neural data scientist. His interests are neuroscience and
 the connection to machine learning. He has an awesome blog called XCore, which I guess is
 pronounced cross correlation, but who knows. So please check out Patrick's blog. He also worked
 at Google for a while, seeing how people interact with web pages and was a brain computer interface
 engineer at Facebook Reality Labs. He also has launched the NeuroMatch Academy, which is sort of
 an intro, an academy where you learn in a summer school about computational neuroscience. This runs
 every year and you can take part if you want. We're going to touch on that a little bit in
 the interview. I just wanted to take it away beforehand. So I'm going to give a little
 introduction about what we'll talk about and then we'll jump into the interview. We're going to talk
 about mainly about this blog post right here, the 2021 in review unsupervised brain model. The main
 focus here is on unsupervised models and what they have to do with the brain. So a big question
 in neuroscience is how does the brain work? I guess it's the main question in neuroscience.
 And so people are developing the hypothesis of how the brain works. And deep learning turns out to be
 quite an interesting tool for neuroscientists because in deep learning, we get some inspiration
 from neuroscience, but essentially we build a model that end to end can learn some task,
 to perform some tasks. So this would be this one right here. Now the question is, is what deep
 models do the same or different than what brains do given that they solve the same task? Like let's
 say both recognize objects on images. Do they do the same thing or do they do something completely
 different? So neuroscientists, they wonder, you know, how does the brain learn stuff? Is it the
 same as neural network? Does the neural network now also during the interview, I have to stop saying
 neural network because it's ambiguous in this context. So does a deep network, a computer,
 a human made deep network, does it account for neural activity, which means that are the signals
 in the deep network the same or related to the signals that we see in the brain? And this turns
 out to be a very important tool for neuroscientists. What they want to see is that let's say the
 intermediate representations in the neural network. Like you have some kind of picture,
 it goes into a neural network, there's layer, layer, layer, layer, and then there's a classification
 head. The classification head might not be that interesting, but what is interesting is like some
 intermediate representation here. If we figure out that that explains, which means we can correlate
 it with things that are in the brain. And I'm going to draw like a very bad brain right here.
 If we can correlate this with things that are found in the brain signals like from fMRI,
 from electrodes that we put into people's heads, then that is an indication that what these deep
 networks are doing have something like that there is an effect that is similar and that could help
 us understand the brain. So the holy grail in neuroscience would be something that can perform
 the same task as humans that does account for neural activity that is biologically plausible.
 As you might know, there is still a debate of whether something like backprop is implementable
 in the brain in one way or another, or if we need an entirely different mechanism in the brain.
 And lastly, something that could conceivably also have evolved and maybe we'd even have some
 evidence of how it evolved over time. So we're going to talk about these models right here,
 specifically self supervised models. Self supervised models here is a slide by Jan Lacan,
 or models that don't need labels to train. And what you usually do is you block out part of
 something you know, and then try to predict that from the parts that you do know. For example,
 if it is an image, again, you'd block out some part of the image and then from the rest of the
 image, you'd try to predict that part that is self supervised method. There's also contrastive
 methods which are self supervised, which means that you'd have an image and you make two different
 views of it, for example, by cropping the image in different places. And then you try to train
 a model that can tell that these two things actually belong together, come from the same image,
 and that they are apart from, I'm going to draw inverted arrows right here, they are apart from
 like a third image that has nothing to do with this image. These are contrastive methods.
 It turns out that if we build models that learn in self supervised and contrastive ways,
 and especially in multimodal ways, that we end up with models that can explain brain activity
 fairly well. So we're going to jump into the papers right here in the interview pretty quickly.
 But if you keep watching the interview, Patrick goes also into more like high level explanations
 of neuroscience in general. It is a bit my fault that I immediately was like, so what does this
 paper say? But I promise you, if you keep listening throughout the interview, there are great insights
 into the entire field of neuroscience into what are open questions into where can people go to
 learn about this. And if you even want to research this, if you're in deep learning right now,
 and you're interested in neuroscience, this Patrick says it's a wide open field, there's lots of papers
 to be published. And the conferences are especially something like NeurIPS are pretty receptive to
 papers that connect deep learning with neuroscience, or in general, try to explain neuroscience,
 neuroscience things. So as I said, we're going to jump into the interview now, I don't want to spend
 too much more time because we're very detailed in the interview. Check out Patrick's blog
 and all his other endeavors. And I wish you a lot of fun. Bye.
 Hello, everyone today here with me I have Patrick Minow, who is a neuroscientist slash blogger slash
 anything else that you might imagine in between deep learning and the human brain. Welcome, Patrick,
 to the channel for this bit of a special episode, I guess.
 Thanks. It's great to be here.
 I got I think I'm going to say I'm going to say I'm going to say I'm going to say I'm going to say
 I got I got sort of knowledge of you for through your article 2021 in review unsupervised brain
 models, you wrote down what happened in the last year in terms of the connection of deep learning
 and how to let's say how to explain the brain. What is your what is your background in this area?
 How did you come to be in this in between space between neuroscience and AI?
 Yeah, absolutely. So I actually originally studied physics. And, you know, after my undergrad,
 I figured, you know, maybe I don't want to do string theory for the rest of my life. Like that
 sounds it sounds like some of the questions that to ask like interesting questions, you need to
 really be pretty advanced. But I think in neuroscience, there's some questions that are
 pretty right for the picking and that are obvious for even somebody that's pretty far outside the
 field. So for instance, what is sleep? What does it do? That's like a pretty easy question. That's
 that's very hard to answer. So I went to do a PhD in computational neuroscience at McGill.
 And one of the fields of my study was really that intersection of neuroscience
 and artificial intelligence. Now, when I started my PhD, which was in 2008, deep learning really
 wasn't a thing, I guess, like some of the original papers by Benjio and and Jeffrey Hinton had been
 they were out. But you know, the big event, I think, in presenting deep learning to the world
 and saying like, this is really this is a big deal was image in 2012. Right. As you know, so that was
 during my PhD. So at the very start of my of my PhD presentation, my PhD defense, I would say
 something like, look, you know, you have neurons in infratemporal cortex, which is one part of the
 visual stream, and they're able to do visual recognition. I would present examples of these
 neurons. And they're invariant. And to things like lighting, rotation, scale, etc. We don't know how
 to make a computer that does that. But if I gave this presentation, just you know, six months or a
 year later, I would never have been able to say that because people have been like, you know, you
 could just you know, like get even Alex net would would be able to do that. So so that's a little
 bit my, my story, my introduction to to neuro AI. So I was there like, during that transition,
 towards deep learning. And in fact, in the end of my PhD, I was, I was working on deep learning
 to try and explain some of the brain areas that I cared about. Now these brain areas are the areas
 of the dorsal stream. And those are like really brain areas that really care about emotion. And
 so I was poking around with what was I'm going to date myself, you know, I was poking around in
 the piano back in the day to to make this happen, which I guess has fallen by the wayside. But yes,
 I've been at this intersection for quite a while now. Awesome. Well, that it seems like it was an
 exciting time. I do remember the piano as well. So I'm definitely dated, dated the same. So you,
 the dorsal stream, just to make clear, that's part of sort of the visual, the visual stream into the
 brain. Is that correct? Or? Yeah, yeah. So maybe I can, I can give you like the first minute of my,
 my thesis defense. I've got it engraved in my brain. You just, you defended not too, too long
 ago, right? True. Exactly. So I'm sure you're gonna forgot it. Oh, yeah. Yeah, you just like put in
 the box in your brain and just, it's gone. Okay. So the visual information falls on the retina. And
 it's originally encoded in these very simple formats in terms of differences and luminance
 between like a center and a surround, or differences in time. So you can think of it as a camera with
 like a little bit of linear filtering. And it then gets forwarded to different areas of the brain,
 first to the lateral geniculate nucleus, and then to the back of the brain, the occipital cortex,
 which is called the primary visual cortex. So that's a huge area, huge chunk of the brain.
 And you have tons of neurons which are selected for vision there. And from from there, the
 visual processing splits into two different substreams. There's the ventral visual stream,
 which is the object stream. So if you think like, what does a, you know, ResNet 50 that strain on,
 on ImageNet do? Maybe it's something similar that we can get into that later. And then there's
 another set of areas, which is the dorsal stream. Again, organized in a hierarchical fashion. Again,
 you have like these, you know, for instance, you have increases in the size of receptive fields,
 you have increases in the size of in the complexity of things that these neurons respond to. But this
 time, they don't care about form, they don't care whether they don't care about texture, what they
 really care about is motion. So you know, you're going to poke at a neuron in, let's say the middle
 temporal area, which is part of the dorsal stream. And 80 or 90% of the neurons will respond when you
 show them the right moving stimulus. Yeah, which is, which is remarkable. So in your in your article,
 you go a little bit into both of these streams. And I think the one of the main focuses that you
 care about is, are or are the are or are not the deep learning networks we use today, similar to
 what the brain does, because sure, we've built these systems that can do some visual tasks.
 But does that bring us closer to understanding how the brain does certain things? And the answer is,
 right? The answer is a little bit yes, and a little bit no, like there's still there's still
 questions. But you point out a bunch of areas of where progress has been made in correlating,
 let's say, neural activities in deep neural networks with neural activities in in brains.
 So yeah, yeah, I'm, I think that it might be good to just back up a little bit and talk about the,
 you know, that world at large so that, you know, people are just tuning in. I haven't read the
 article yet. We'll understand what we're discussing. I think that originally, some of the,
 okay, so I was talking about ImageNet 2012, which was the big milestone in creating good
 deep neural networks that could solve the kinds of tasks that humans that humans can solve. Now
 there was a lot of background work that came into that. One is, you know, the creation of
 convolutional neural networks and the work from from Yan-le-Cun, which was ultimately, you know,
 inspired by the new the new cognitron, which is Fukushima, like in around the early 80s.
 But ultimately, that work was motivated a lot by some early work in vision and in vision neuroscience.
 So David Ubel and Torsten Weisel in the 50s and 60s looked at different kinds of neurons in the
 primary visual cortex, and were able to find that you have this this hierarchy of selectivity,
 right? So the canonical thing that they found is they found cells which were tuned for orientation,
 right? So you know, you present an edge like this or a line like this, and the cell responds.
 But if the line, if instead of being white, it's black, then it doesn't respond. So those are called
 the simple cells. And then they found another subset of cells, which are called the complex cells.
 And so those are selected for this, but they would be, it wouldn't matter the precise location
 of this line in question. And it wouldn't matter the contrast. So it could be white to black,
 or it could be black to white, it wouldn't matter. And so their hunch was that, okay,
 well, you have this this transformation that happens, first of all, you have a selectivity
 operation, which creates that simple cell. So basically just a threshold. And that's enough to
 give you a selectivity, or it could be a relu if you, you know, smooth it out. And, and then there's
 a pooling operation that happens. So you pool from different, from different simple cells that have
 the same orientation selectivity, but different contrast sensitivity. And that creates the complex
 cell. And you can view that as a subsampling operation or downsampling operation as you would
 have in a deep neural net. So there's this kind of long line of like, oh, there's the inspiration
 from the brain, we're going to make some models, we're going to show that it's that they're actually
 good enough to solve tasks that humans can solve. But the question is, okay, are these are these
 like really like, like human brains? So and that's similar work from from in Jim DiCarlo's lab and
 Nico Cricascorte in 2014, like really showed that there's some very tantalizing hints that this is
 indeed the case, you know, that these networks that we've trained on ImageNet, they look a lot
 like the brain in, in really interesting ways. And one of the big ways that, you know, they're
 similar is that if you have, if you look at, you know, let's say 10 different networks, and one of
 them is, some of them turned out to be a little bit better at solving ImageNet, or a little bit
 worse. And then you correlate that with how well you can align these networks to the brain, turns
 out that the ones which perform better on ImageNet tend to also perform better on explaining the
 brain, which is like a very strange coincidence, because think about how like, completely different
 these two things have been created. So that was that was one of the big hints. And I think like
 another big hint is the word from Chris Ola and other people at OpenAI that looked inside of these
 deep neural networks and found that, you know, the kinds of selectivity that you see inside the cells,
 they're very, very similar to what you would, what a neurophysiologist would describe in areas like
 V1, V2, V4, and temporal cortex. So the combination of the quantitative and qualitative tells us like,
 hey, maybe, maybe there's a kind of these are kind of like little brains, one very, very specific
 part of the brain, I want to be a lot of trouble if you say that that statement. Yes, exactly,
 exactly. So what do people mean when they say something like explains the brain or something
 aligns with brain activity? Like, what is it? What is behind that? Yeah, yeah, yeah. So we can talk
 about the high level stuff, like, you're sure just like the idea of look how like, what do we,
 what do we measure? Like, you know, is it a number? Is it a correlation? Or is it? Am I training a
 regression model from one signal to the other signal? Like, how can I make the statement that
 this neural network explains some function in the brain? So in the early work from from 2014,
 we see two different approaches being used. And those are the kinds of approaches,
 like every other approach that's been tried, is kind of a derivative of these like two basic
 concepts. So one approach is a regression based approach. So let's so very simply,
 let's say you train a ResNet 50 on image net, you chop it off at some layer, layer four after the
 first down sampling or whatever. And then you measure the output of that deep neural network
 with respect to some stimulus ensemble. So which gives you a big matrix big X, which has a bunch
 of rows for the different examples and a bunch of columns for the different features. And then you
 just regress that against neural data that's, that's recorded with the same, with the same images.
 So it's just a regression. So you can add like a bunch of different spices into your basic recipe.
 So you can add some some sparseness priors, you can try to well, usually you'll use a ridge
 regression rather than a straight regression, because that will definitely the other regular
 regression will usually crash and burn neural data is very noisy. That's something that people don't
 often appreciate. And so it's a regression. Let's just put it that way. Yeah, now that
 will be sort of, for example, f MRI data, when we talk about neural data.
 It can be f MRI data, it can be MEG data. So magnetoencephalopograph,
 encephalopograph, I think we just say MEG. And or it could be a single neuron recordings or array
 recordings. So those are taken inside the brain, or it might be ECog, which is just on the surface
 of the brain. So there's different kinds of recordings. Now, it happens that f MRI and MEG
 are much more popular for for humans, because it's it's, it's non invasive. But every once in a while,
 people get to record inside of the brains of humans that have some some sort of need for brain
 surgery, whether it's usually it's epilepsy. And those data are very precious. Now speaking of so
 you go through different papers in your article. So maybe we can follow that structure a little bit.
 The first one is a work that shows that the ventral stream might be explainable by and your idea,
 your the article also goes into it's called unsupervised unsupervised brain models.
 So your your kind of point that you make is or your investigation is into unsupervised systems,
 like what, what, what, how good or how close to what the brain does is comes from the self
 supervised and unsupervised system. So the first, the first, the first thing you go into is the
 ventral, sorry, the ventral stream, that is you set the sort of object stream. And this paper looks
 at single neuron activations, right? And the they find that the self supervised systems can be or
 are equally or even better able to explain the brain data than supervised systems, let's say in
 an image recognition task. Yeah, so that's super exciting. And the reason is that I think that
 everybody got very excited when they saw that these networks which were trained for image net,
 they could be aligned for to the ventral stream to that object recognition stream,
 because now it's something that, you know, you have this in silico thing, and it kind of looks
 like it does the same thing as the brain. And so it's kind of a model of the brain. Super exciting,
 you can do a lot of things with it. But there's different ways in which something can be a model
 of the brain. And some of these are a little bit more useful than others. And one of the ways I one
 of the big flaws, I think, for for supervised learning is that it's not like really a way it's
 not really a model of how the brain would learn a task. Because, you know, I'm not walking around as
 a baby. And like, you know, my, my parent just tells me like, dog, dog, dog, dog, dog, cat, dog,
 dog, just like constantly for years and years. So you know, we don't really use unsupervised
 learning for for for learning these kinds of things. So that's a big flaw that if we want to
 go move forward with models, which are biologically plausible instantiations of creating these,
 these models, then we have to move away from from supervised learning. So people generally
 like unsupervised learning and self supervised learning better for that reason, because you
 don't have to, you know, come up with this like, weird concept that you have dog, dog, dog, cat.
 And and but you do have to do the math to make sure that it actually does work out in practice.
 And that, you know, the right the kinds of the quantity of examples that you feed into,
 into the model is similar to the kinds of to the quantity of examples that you would feed into a
 human, for instance, I think you have you have a so your conclusion, you have a little bit of an
 example that it would like the language models that we train such as GPT three would be equivalent
 to like, years and years and years of of human, just constants, just talking and talking and
 talking and talking and babies are able to do it by age, what four or so or two.
 Exactly. So, so I think that there's still a big gap there that comes from that you still I mean,
 we're off, I think I calculated we're off by four orders of magnitude in terms of the efficiency.
 But, you know, I'm to score everybody on the same kind of curve. I mean, the GPT three is not made
 as a model of the brain minutes made as a language model. And to solve all these these problems in
 zero shot settings, and it works very well for for its purposes. But definitely, if we want to
 actually try to explain the brain, we'll need to get to that. So this, this, the, it is also a bit
 special, because we hear we talk about the ventral stream, you said that's the object stream. And the
 fact that self supervised systems are equal or better at explaining that than supervised systems,
 which presumably are trained exactly on the task of that such an object stream would be sensitive
 to right, that is also one special thing.
 So I totally agree. I mean, that's super cool that that this is the case that you have this,
 this thing where you don't give it like learn objects, and yet it learns something that can do
 can do object recognition. And it learns meaningful, meaningful things like that. But
 I think that there's a couple of hidden assumptions there that make this not nearly as mysterious
 as it was like, as we would like it to be. So one is that, you know, image net is not really
 if your model of image net is not you take like a, like a nice Canon DLS, the DLSR, and,
 you know, you, you put it at a random point in space, and then you point it at somewhere random,
 and then you hit the button. Right. So if we look at both of our faces right now, we're in the
 center of the screen, it turns out that, you know, we're smart like that, that we place our faces,
 like generally in the center of the screen when we take photos. So the things that we try to look at
 in image net, you know, the the subject of the category will by and large be in the center.
 So, and you know, the position of the camera, the things that we that we tend to measure,
 I mean, these are all these all come into why the model learns the thing that it learns. So it's not,
 it we can't really say, oh, it, you know, we're not like really feeding it any, any structural
 priors, we definitely do. We definitely do just, just in not like the conventional way, and not in
 a way that's very easy to quantify either. But some people are definitely trying to solve these,
 these problems. So, so for instance, there's a lot of work on trying to fit the same kinds of
 unsupervised learning models, but with streams of data that look more like what a baby would see
 in their early years, when which the camera is not always pointed that at the right things,
 because babies tend to, I see. Yeah,
 do a lot of gesturing. But it's also, it's also there, especially because the baby with time is
 able to move its head, right. And therefore, it's also not the same as just placing a camera
 somewhere because whatever captures attention will be actively looked at more. So it's, it's
 definitely like, I think there's a long way to go in any of these things. Oh, yeah. Oh, yeah,
 absolutely. I think. So to close the, the, just that one paper, because we've been on it for
 like 15 minutes, but super cool that you can have, you can train a model in a unsupervised or
 self supervised manner. And it turns out to be just as good at explaining, you know, V1, V4,
 and IT, all these different sub areas of the ventral stream. And then there's a kind of
 eriarchy that happens between the different, the different models. So, you know, some models are
 clearly doing better than others. So typically in these papers, SimClear is usually the one that
 performs the best for reasons that we don't totally understand. Local aggregation also tends to, to
 do better. So that's interesting. Like, what is it about what's inside of these models that can,
 that allows them to be more similar to the brain. Now, of course, in the end, you know, you end up
 with like tiny, tiny error bars, and it can be pretty difficult to actually differentiate between
 these, these different things. So, you know, you can't like read too, too much into it. But
 definitely the best models are like the new kind of generation of self supervised models.
 And then so the next paper deals with the with the with the other stream with the dorsal stream.
 And there you or yes, that is actually you who found some that's your own paper, right?
 Oh, yeah. So, so I'll just go very rapidly with true that actually the second one is
 ventral stream. Oh, sorry, again. And so that's from Talia Conkle. And very, very consistent data.
 So they use fMRI rather than single neuron data. But I mean, the data is like these two studies
 were done independently, about a kilometer away from each other, one one team from Harvard and
 one team from MIT, and they found exactly the same results. So maybe some things in the water
 in Cambridge, Massachusetts. But otherwise, I mean, it's a very robust finding, basically.
 But yeah, we can definitely talk about the dorsal stream. So, like I said, I've been interested in
 this problem for a for a very long time. And I had a little bit of time during the the last
 lockdown of the pandemic to to relook at this problem. And so we sat down and we said, you know,
 this I think like the time is right to really look at all this dorsal stream data and see if we can
 get if we can get one really good model of all these these different areas. So the first thing
 that I did actually is I was going about this very naively, but I just looked into like the
 torch vision models, you know, they have like some some model database, and just downloaded
 all the models that were trained on video recognition. So all the models that were trained on
 I'm drawing a blank here, kinetics 400, which is a task where you have to look at a video of
 somebody juggling and say, oh, it's juggling rather than unicycling rather than soccer or whatever.
 And so the special thing about these models that they look at 3d data, by 3d, I mean spatial
 temporal, right in time. And so that means that and generally they're trained, the convolutional
 neural nets, they're trained with 3d filters. So, you know, the front end of the model is going to
 be a 3d convolution in space and time. So I looked at these models, and I did the kinds of
 visualization tricks that Chris Ola and gang do it, I open my eye to look inside because I was
 curious, you know, do they learn motion? Do they align with with the brain? And I found that they
 were actually really terrible, which surprised me, because if you look into the methods of these
 papers, it's like we trained, we trained these models for 24 hours on a supercomputer with,
 you know, 16 GPUs in parallel, and went through, you know, a million videos. And this is the model
 that we obtained, and they're very good at doing the tests that they're doing. And yet, the kinds
 of generic features that come out of the models are really terrible at aligning with the brain.
 So that was kind of the hunch that we saw there that I should say that the one of the early
 findings and one of the early points that people who are dubious about the finding that the ventral
 streams align with ImageNet trained ResNets and AlexNets and VGG nets, is that people say, well,
 you're just training the model to do a task, you know, any sort of task will work. It doesn't
 matter whether it's object recognition or whatever, it just turns out that this is the task that you
 had data on. But this is a very, this is a very good like counter example of that, because you
 train a model on a task which involves, you know, 3D data, video spatial temporal data. And yet,
 that model is actually the model that you that you train is really good for that one task,
 but is really terrible at this task of aligning with the brain. So that motivated us to look
 more deeply into, you know, what else could, like if we don't train, if we don't take, you know, pre-train
 models to solve this problem, like what could we do? And we know that a lot of the dorsal visual
 stream is really cares about navigation. So if you look at an area like MST, have you ever had Vertigo?
 Sure. Yeah. So Vertigo is like kind of sorry, this is like a weird non-secret, but Vertigo is kind of
 a funny thing, right? Because it's an inner ear problem, right? So you have your vestibule, and it
 kind of, it basically tells you there's acceleration in ways that there shouldn't be acceleration. And
 that gives you an impression of being dizzy. But also gives you like these weird visual effects.
 Yeah. Right? Which is strange. Or, you know, if you drink a little too much, you might have that
 same kind of feeling. So there's an area in the brain, which is called MST, which has these
 neurons, which receive both visual input and vestibular input. And the way that they receive
 visual input is they have a lot of selectivity for things like rotation and expansion and
 and wide field translation. And so we think that they're really involved in navigation. So if
 you're going forward in a line, you have these neurons, which receive both the vestibular input.
 So they know how you're accelerating and where gravity is. And they receive all this wide field
 optic flow, which is tells you where you're heading. So we said, why don't we train a deep neural
 network to solve a navigation task so that the network can can orient itself in space, essentially.
 So I used an environment, which is it's an environment for drone simulations called AirSim.
 And it's really fun. So it's an Unreal Engine. And you can, you can basically fly a drone in these
 suburban environments and back out the sequences of videos. And then you can train a convolutional
 neural net, 3D ResNet, to solve the problem of figuring out what is the from a little sequence of
 movement, what is the trajectory, basically, that's going on, like where are you heading?
 Are you rotating? Are you going forward, etc., etc. And so if you train a network on that, it turns out
 that if you visualize the cells inside of the train network, they really, really look like what
 you would see in the visual cortex. So as a neurophysiologist or as an amateur neurophysiologist
 or a person that's been in the vicinity of neurophysiologists, I was really, I was really
 stoked to see this. So you see these cells that are selected for translation and translation,
 but they don't care about the pattern that underlies the translation. And in particular,
 you see these cells like the one that you're visualizing here that like things like spirals
 in some of the higher level layers of this network, which was super exciting because those look a lot
 like what you would see in a... So basically, the networks that try to just predict anything from a
 video that contains motion weren't like turns out these neural net, sorry, the deep networks,
 I have to stop saying neural networks here because it's ambiguous.
 Ah, yes, yes, yes. The deep networks that train on any kind of video data, they're not super well
 aligned with the brain. However, as soon as you go maybe to like some sort of an ego perspective,
 right? And you especially you predict your own parameters of motion. So from the visuals you're
 trying to predict, okay, I went to the left, I went to the right, I turned around from the visual
 information. And that turns out to align very well with the brain data. Does that make like,
 just maybe an esoteric question, but does that say anything about the need for AI to be embodied?
 Maybe? Oh, I love this question. Yes, 100%. Yes, we should, we should completely embody AI.
 Yeah. So I think that one, one big question that came up during the review is that, you know,
 we claimed originally this was unsupervised or self supervised in the abstract. And then the
 reviewers came back and said, well, it's not really unsupervised or self supervised. It's a supervised
 network because you know, you know what the answer is, you're just training in a supervised fashion.
 My feeling is that it is self supervised in the sense of when you embody this in an agent. So when
 I'm when I'm a baby, let's imagine that I'm a baby. And I'm walking around the world, I have some
 control over where I'm heading. Yeah, right. So I can say like, I'm going to turn this way, I'm going
 to turn that way, I'm going to move forward, I'm going to go get that cookie. I'm going to look at
 my parent, and so forth. So I am an agent. Yeah. So that means that I control the motion that comes
 into my eyes. Yeah. Because the vast majority of motion that we see in the world comes from
 from our self motion. And so I can correlate my motor plans with what I see in the world. And
 that means that it's a much easier kind of problem to correlate these two things, then to say I
 here's found data, which is the case of ImageNet, and figure out something to model with this. Yeah,
 exactly. Right. Yes. You also have this diagram here from young Lecar, talking about self supervised
 learning. And it seems very much that it is I agree, the line is like gray in some places. But it
 seems like if you are an embodied agent, you always have those motion parameters ready, right. So it's
 much more like I am going to darken out part of part of what I already know and try to predict
 that from it, it seems it falls a lot into this into this diagram right here. Yeah, absolutely. So
 I think it looks more like the bottom part of this diagram that you see there, where you have these
 two things which are happening in the present, but one part is occluded and the other part is visible.
 So you're doing multimodal masking, in other words, right. So you have the vision, but now you're
 trying to predict the vestibular, or you have the vestibular, and you're trying to predict the vision.
 And so if you look something like clip would be, I think, like maybe the most popular model that's
 of the same kind of multimodal kind, you can say, well, clip is a supervised model, because you're
 trying to predict, you know, in a way, you're trying to predict language from vision. But
 it's really this kind of masking. And I think it's a more general approach to solving this type of
 problem. So yeah, I agree with you embodied agents, I'm 100% on board, they're definitely going to be
 awesome. And actually, questions about, you know, what do reinforcement learning agents learn? Do
 they learn like good self motion representations, for instance, when they're when they have a visual
 task? I think like those are super interesting, like, what do you need to put in there? In order
 to get that that effect? Yeah, that that concept of me in a eyes is not yet really come through so far.
 But I'm also looking into like, I'm looking forward to having more of a eyes who understand
 the concept of, of me and to be embodied and and and sort of to have self self state and all of this
 kind of stuff. I think that will bring us forward. So here in the next paper, you you tackle not I
 mean, this this paper you're describing, it tackles the question. It is actually, it is actually, I just
 saw in my notes, that is, again, one of one of your papers. It is the question, why are there even
 two different of these visual streams in in the brain? Like, it maybe makes sense if we if we sit
 down, but also, you find some actual empirical evidence for why it might be might be that we
 even have two streams, right? Yeah, yeah, absolutely. So I think that's a that's an interesting question,
 like, why are there two things rather than one or four things or eight things rather than than an
 arbitrary number? So, so Shahab, who's the first author on this paper, worked on looking at what
 it would take to to recreate both ventral and dorsal stream. And I think the remarkable thing
 that he found is if you train a network like CPC network, so a contrastive predictive coding network,
 which is one form of self supervised learning, in which you're trying to essentially discriminate
 between different futures, if you will, so you're trying to you look at the past, like a certain
 window in the past, and then you're trying to tell apart like the actual future embed in some subspace
 versus an alternative future, which is which is dreamt up. So if you try to do that, then, you know,
 it's already been shown that you can find good representations and in videos. But what's very
 interesting is that then you can ask the question of what happens as you add more and more substreams
 inside of this of this network. So if you remember the original Alex net paper, so it did have two
 streams. So if you remember, like very, it's like a while ago, but what happened is that they had
 like tiny GPUs back in the day, right. And so they couldn't fit the whole model on just on just one
 GPU. So what they decided arbitrarily is to split it up into two parts, especially at the at the
 early point. And then basically, they so they were independent, but they could re communicate a little
 bit later on. So which was a pretty unique feature. Back then, people didn't really do that. But now
 it's it's quite common to, you know, chop up the channels in different ways and all sorts of things.
 But what they found is that there's this this this very interesting self organization principle where
 all this all the the filters on one GPU turned out to be color selective, and all the filters on the
 other GPU turned out to be to be black and white, which is whoa, that's weird. Just by the fact of
 splitting up, because the two streams, they don't always communicate, right, they only communicate
 at very sparse intermediate points. So so just structural prior gives rise to something that
 very much looks like the brain in that in the sense that one of the streams correlates well with
 the ventral brain stream and one correlates well with the dorsal brain stream. Yeah, so in that in
 that case, in the early Alex, that paper, actually, both of the types of filters are different subtypes
 that you see in in V1, but they are, you know, functionally different, and they have different
 roles. But it was like kind of an interesting proof of concept that if you just set a separation,
 arbitrary separation down the middle, you don't say anything else like you don't say like, you
 have to respond to color, you have to respond to this. But just you set a separation, it self
 organizes to something that's interesting. It's crazy. And yeah, it's weird. So they might have
 just locked themselves into like building a better model by by having two small GPUs.
 Yeah, exactly. So, you know, they say that necessity is the mother of invention. So I think
 this is a particular case where, you know, the limitations at the time caused them to
 stumble onto something which I think is is really deep and interesting, which is symmetry breaking.
 So I guess ultimately, you know, when you start with, okay, you can imagine that if you
 just set all the weight parameters to zero, and then you perform your gradient descent, these
 two filtered sets will learn exactly the same thing, or they'll crash and burn. But by adding
 a little noise, right, by initializing your your network, you're pushing the network very, very
 slightly out of equilibrium, and that's enough to self organize into this thing. And so Shahab
 found a very similar phenomenon in the context of these networks, which are trained in an
 unsupervised manner in CPC. And so being trained on videos was able to find that these parts of the
 one part of the network was and so again, this is this is an instance of a network that has
 kind of a firewall in between the two sets of filters. And so he was able to find that these two
 sub branches, one of them was dorsal like and the other one was ventral like,
 and was able to correlate that with some some data that we have in in mouse where there's tons and
 tons of data on what's the relative selectivity of these different things and found some some
 really nice correlations. So that means that you can all you would need basically is a little bit
 of a nudge, right. And so so which is this great idea, like maybe you just initialize the network
 in a sli so that like the two things are just very slightly asymmetric. Because one thing I should
 say is that the the two networks don't always get the same label, right. So if you train the network
 twice, one time it's going to be dorsal ventral and other time is going to be ventral dorsal.
 Whereas the brain every time that you train it, it's the same that we know. There are some exactly
 it's all ventral is ventral dorsal. So there's some like inbuilt asymmetry. But it's a very
 probably like a very small asymmetry. Because if you train it with real data, and then it will
 automatically, you know, self generate into this in bloom into this particular activity. Cool.
 So very excited that the brain can organize itself for something that's that's useful just from
 this could be used, I guess, for I mean, people are already, you know, in multi head attention,
 they do multi head, right. And that's kind of similar in that they they clearly separate
 different computation that cannot interconnect. And therefore, that that sort of there also,
 like the random initialization probably does some symmetry breaking, and then you find that the
 different heads respond to different things, people have investigated that it's probably very
 much along the same lines. So I want to skip ahead a little bit here to the the the the
 the concept cells, the the is it this paper? Oh, that's this as well. I think like, I think
 that there's been a lot of movement in the subfield. And by the way, I want to tell your
 viewers because I know a lot of you viewers are coming from a machine learning background versus
 an neuroscience background. And, you know, it's hard to get into NeurIPS. But I think if you know,
 it's such a wide open field in neuroscience. There's so many questions that if you care a
 lot about representation learning, you know, it's it's a pretty easy field to jump onto,
 and, and have positive reception. So there's there's still a bunch of a bunch of questions.
 So grab your nearest neuroscientist and go write a paper. Encourage everybody to do it.
 Yep. Definitely how to how to hack how to hack publications. There you go.
 Yeah, there you go. So yeah, so clip clip is clip is weird.
 So if there's one thing that I would say is when we saw when we saw the results of of clip and
 and some of the both in terms of of how good it is, and also the
 inner visualizations that Chris Olin gang worked on Chelsea Voss, as well.
 I think that we were all kind of surprised because they do look a lot like the kinds
 of concept cells that you see on the hippocampus, right. So the very, very, very, very famous paper
 that did this is the had the infamous Jennifer Aniston cell. So I don't know if you're
 in your only in the context of your article. So it's one one cell that responds to both what
 pictures and the name and various aspects of a person not not just like,
 exactly, exactly. So if I remember correctly, this this paper, so they had, they had people with
 intractable epilepsy. So these are human patients, and they were doing pro recordings in the
 hippocampus to figure out what was the the nature of their epilepsy and how they could be treated.
 And, you know, they spend a lot of time in the hospital just being bored. And so sometimes they
 enroll into experiments and these experiments tell us more about the human brain than is otherwise
 possible. And so very thankful for for these people that do this. And so in this particular
 instance, they, they presented different kinds of concepts and images. And one of the cells that
 they found that have this like amazing property that if you just show the words Jennifer Aniston,
 it would respond. If you showed the face of Jennifer Aniston, it would respond. If you showed,
 like I didn't do like other kinds of controls, but I imagine that if they had played,
 and you know, that the start of the of the French show, it probably would have responded,
 because it all came with this like general concept of, of Jennifer Aniston. So ever since then,
 people have been like fascinated by this idea, although it's a much older idea, you know, this
 idea that you have like a cell in your hippocampus that responds to your grandmother, it's the
 grandmother cell idea. But one thing that was very interesting when we first saw clip is that you
 have cells can respond both to text and to to images. And in fact, you can do these new kinds of
 adversarial attacks in which you just write the wrong, write the wrong text. And it fools the
 wrong text. And it fools the system into actually reading the text and mislabeling the the images.
 So it sounds very hippocampus like to me. And so in this particular paper, they,
 they actually looked at at this problem and found that out of all the different models that
 that they could look, they found that clip could explain the most hippocampal data,
 which is super exciting. I'm sure that people are really going to drill down further into this,
 into this finding. Yeah. But it's clip specifically, because there's a lot of other
 unsupervised models. And somehow clip is the best and we still don't understand why this is I mean,
 it's like the delta between it and the the second best model is, it's huge. But why?
 I think no one knows right now. And and actually clip the the just the the the visual aspects of
 clip are also very good at explaining some of the some some other data. So it's, it's very
 interesting to think about what happens in a multimodal fashion, like what happens when,
 you know, experimentalists and neurophysiologists like really like to isolate one thing to one
 thing to just look at one thing at a time. But now you're talking about something that can do
 different kinds of modalities. And I think that, you know, multimodal areas are going to be some
 of the next things that are really attacked by unsupervised and self I mean, it's also a question,
 I mean, clip is huge. It also has a huge amount of data. We don't exactly know what data went into
 there, right? There's a lot to to untangle here. But the multimodality, I also feel that that is,
 is a big part of what's going to bring us forward in AI. And probably also, you know, since the brain
 is always multimodal, like, I don't you don't get like a stimulus that is maybe now with computers,
 you do. But you know, just growing up in nature, you probably get zero stimuli that are just
 unimodal, right? So you're always in this mode of multimodality.
 Yeah. And in one thing that's, that's interesting, in particular for babies, you know, if, if you
 ever interacted with babies, they really like to have toys, which make lots of noise, which drives
 parents crazy. And but I think that there's a reason for that, right? Like, why would you want
 to like a toy that makes like a lot of noise, because clearly, there's a lot of pressure on
 making the noise as silent as possible, because the parents are just like trying to sleep. But I
 think that the kids just prefer that because it's a multimodal stimuli. And you can do all sorts of
 causal inference about what happens when I get this thing with this thing.
 So this is the last paper that I I wanted to look at, maybe maybe you have more, but this is,
 it's challenges, the manifold perspective of deep learning in your you've described it a little bit
 in the paragraph, you say challenges, the manifold perspective, and it favors the causal
 perspective. So what is meant here? And what does this paper tell us?
 Oh, yeah. So you remember, we were discussing earlier, the mechanics of how you compare a brain
 area and deep neural network. And so you could have so I think a lot of deep learning methods are
 rotation invariant. So if you take something like clip, for instance, you're learning,
 I guess, like this, this subspace, which is, I guess, like 128 dimensional in the both from the
 visual side and from the text side, and you're trying to align it in this 128 dimensional space.
 If you multiply the two by rotation matrix, and then the entire 128 dimensional space gets gets
 rotated, it's the same network, right? It really doesn't matter whether it's, whether it's rotated
 or not. What matters just the locations on the manifolds. And so if you're thinking about aligning
 a brain area and neural network with a with a regression, again, the rotation doesn't matter.
 You're saying any any weight matrix is just as good as any other weight matrix. So that's the so
 So that's the so that's the underlying, I think, assumption. And I think that there's been a lot of
 work recently in neuroscience, focusing on this idea that, you know, single neurons like don't
 really matter. What matters is the latent subspace in which the near the neurons are responding. So
 if you have a population of 100,000 neurons, maybe they Yeah, it's 100,000 neurons. But if you present
 a bunch of stimuli, you find out that actually the latent sub and you do like an SVD on the matrix
 of responses, you find that latent subspace actually just five dimensional, or whatever.
 So first of all, they're just random projections from this five dimensional subspace. And the
 and the large dimensional subspace doesn't really matter. So this paper, so sorry, sorry, and
 it's been a lot of work in neuroscience showing that this is the case, especially in, in motor
 cortex. So you know, you have tons and tons of neurons in your motor cortex as you're going for
 for reach movement. And yet it seems that these neurons really live in a very low dimensional
 subspace. So that's what we call the manifold theory of neuroscience is that idea that the
 neurons are in a high dimensional subspace, but they're just project random projections of some
 lower dimensional subspace. But one of the consequences that if it's random projections,
 then each of the neurons individually should just be, you know, weird. It should, you know, respond
 to a bunch of different things, it shouldn't be shouldn't be able to place a label, because you
 could like neurons, you could rotate the entire space, it would still make sense, right? So there's
 no, there's no reason why an individual neuron should align with just like one axis in, in that
 particular subspace. Yeah, exactly. So, but neuroscientists really like labeled axes.
 That's one thing that they're very fond of. So, you know, you can imagine that you have like an
 axis, I don't know if you're in Unity or Unreal, you know, you have like my avatar, and then you
 just like hit like one switch, and I just go, you know, it just, it just changes my smile from
 from upwards to downwards. And oh, sorry, I, my printer is haunted. And so I'm just going to
 disconnect it, if you don't mind, because it makes the lights flash. Unfortunately. Okay.
 I find it weird that printers are like the oldest technology on the planet, yet still they're like
 the most troubled, like we should we should have figured this out by now. But we have not.
 Yeah, it's, it's too bad. So I still print out papers, because there's been research that shows
 that you retain more when you print something out rather than when you read it in the on a printed
 document rather than Yeah, reading it on the but it's just becoming so, so inconvenient that I think
 I'm gonna have to abandon soon. Okay, so starting back then, and I apologize, where do you want me
 to restart? So um, we, yeah, there's no there's no particular reason why any single neuron right
 should align with any axis. Yet people find that they do. Yes, yes, exactly. And that might be
 because, you know, neuroscientists like to name things. And if something is not nameable, they'll
 say it's mixed selectivity or whatever, and then they'll just forget about it. That's also a very
 good assumption. So both of these things can be happening at the same time. But in this paper,
 they found that if you train a bit of VAE, which is a VAE, which has a stronger weight on on one
 of the KL terms, it tends to find disentangled representations, right, so that the axes actually
 matter. So one axis is like my smile, the other axis is how much of a unibrow I have, and you know,
 a third axis is, you know, what's up with my mustache, and etc, etc. And so they found that
 that aligns pretty well with some neurons in one face selective area of infotemporal cortex. And
 so they did some some trickery trying to do like one on one alignment versus ensemble alignment.
 And it looks like, you know, the good interpretation for this data is that it's, it's more like a one
 on one alignment. And so that could be pretty interesting. But I do want to point out that
 there are certainly distributed representations in the brain. It doesn't mean that because in this
 one area, you have non distributed representations, that that's the case for the whole brain. And it
 might be because of energetic reasons that we have this representation in this in this brain area.
 Because you know, you want to have how the what the distribution of responses is over a stimulus
 ensemble is very important for how efficient the code is, because remember, neurons are super noisy.
 Right. So you want them you want to have like a nice exponential distribution of responses
 in order to have an efficient code. Given that you have this personal like noise in the data.
 So yeah, and you you say it favors the causal hypothesis, it so it means that maybe what's
 happening is that rather than some simply encoding the signal that you see that the brain is actually
 building like a causal model of what's happening, like you know, there are eyes and there are
 eyebrows and that, you know, the the result of there being eyebrows is that they look a certain
 way. And then it will make sense again that they are encoded, like the structural priors encoded in
 one space. And then simply the manifestation of that is the picture we see. Yeah, yeah, maybe I
 misused the term causal here. I don't want to mistake it for causal inference. And I don't want
 to misuse the term causal inference. And sure, sure. But I think that what I mean by this is
 a forward model for how like one individual. So you can think of you can think of a of a
 directed basically graph in which, you know, there's a bunch of different factors. One of them
 is whether or not I wake up with a mustache today. Another one is how close my eyes are. Another one
 is my nose. And these factors are disentangled. So that means that they're independent from
 each other. And then I can just like turn on and off the switch and generate different faces.
 So that's I think like the underlying naive model is the Mr. Potato Head model, right, in which you
 just like switch out the different components. And of course, there are specific holes that you
 can put the different the different things in. So I think that I guess like the question is,
 like, are these factors in this this factor graph? Are they like, can you put labels on them and
 they correspond to one thing that we would identify as something that is independently
 changeable? So for instance, like, we understand that age and lighting, for instance, like those
 are two totally disentangled things that have nothing to do with each other. So the question
 is, are they are they different factors? Or you rotated like one is square root of two, like one
 over square root of two times age minus one over square root of two times lighting, and so on and
 so forth. And it looks like they're really aligned towards the factors that we can label,
 and that are indeed independent, both in brands and in this particular model.
 Do you think that it plays a big part that it because face, let's say facial structure,
 is it is something that is truly, let's say the individual factors are actually independent
 because of, you know, genetic variation, allele crossing during during meiosis, sorry, or
 recombination, and so on these things actually go in a fairly, let's say, this uncorrelated
 uniform distribution in the human population. So almost every combination of narrow eyes,
 wide eyes, you know, big mouth, small mouth, and so on is possible. And therefore, it might make
 just sense to let's say encode the individual factors as individual neurons, as you say,
 maybe for energetic reasons. I think that that's, that's a really interesting hypothesis. But I
 don't think that that's that that's the case. I think that there might be like a general,
 you know, algorithm that makes it that tries to disentangle these things into into different,
 into different sub factors. And then as a consequence, there's this natural alignment
 with this other process. But, and of course, if it's the case that the kind of latent model that
 is inside the brain is better aligned with the latent model that's in reality, well, that's
 better. You know, you want the thing to reflect, but I don't think it's 100% true that, that these
 that these factors are really disentangled in reality. So for instance, you know, I, I,
 like a unibrow versus mustache, like these two things are probably pretty correlated with
 with each other. Yeah, yeah, yeah, I see what I see what you mean. Yeah. So we're we're we're
 we've been we've been going through this a little bit. There's all I mean, there's a lot of there's
 other papers, which which are definitely also interesting, like the gloss ones is super
 interesting. Is there Yeah, is there one that you wanted to touch on particularly?
 Well, I wanted to give for, you know, readers that are coming slightly outside of this field,
 and moving into this like very rapidly moving field, kind of an overview of what are the
 questions that people are interested in, like what are kind of the some of the interesting approaches
 that people are using to, to tackle these and also encourage people to come in our field and, and,
 and, and, you know, get papers in and, and scoop us basically. So I really want to encourage people
 to, to get into that. I think, I think that we've covered some of the papers that I think are the
 most interesting. And we'll see in the, I actually wanted to do a follow up on precisely the kind of
 agent based representations that are coming because that that is coming down the line. And
 I think that's going to be super interesting for this field. So maybe we can end with like,
 some things to look forward to in the future. Sure. So one of the things that I think is going
 to be interesting for for the future is like really taking evolution seriously. So we saw the, actually
 maybe if you can scroll to where I show Jess's, Jess Thompson's diagram of the different types of,
 of models and how they all fit together. It's at the very start. It's at the intro.
 So Jess has a really nice way I think of, of explaining this, which is that, you know,
 there's some models which can really perform a task. And, you know, once we got to ImageNet 2012,
 like that was, that was where we got there. And then, you know, in 2014, we really got into this
 accounts for neural activity part of, so, you know, we can find models that can both perform a task,
 which is biologically relevant and accounts for neural activity. I think this year was a big year
 for biological plausibility. And I want to say this is the last word, because clearly there's
 way more work to be doing there. You're going to have models which have realistic, biologically
 realistic kinds of gradient descent, or replace gradient descent with something that's more
 biologically plausible. You're going to have Dale's Law, you know, so excitatory neurons
 only make connection, only makes excitatory connections and inhibitory neurons only make
 inhibitory connections and you'll have normalization and you have temporal dynamics and so on and so
 forth. So that's like, the next five years is probably just going to be to fill in this
 biologically plausible. But there's also could have evolved. I think that that's that's like a
 super interesting unknown questions and people are going to start to think about this problem
 in a serious fashion. And I want to point out there's this there's this recent paper that I
 don't talk about here, which from Fei-Fei Li, which is about evolving different kinds of agents that
 can solve different kinds of reinforcement learning tasks that actually has a an interesting
 evolution component to it. So I think we're going to start to see and we can actually like see the
 process by which the brain can bootstrap itself into existence, which I think is going to teach
 us something about what it is to be human. And I'm sure there'll be TED Talks and books and so
 forth. But that's going to take like another five, 10 years. Another thing that I'm excited to look
 at in the in the future is I just wrote my notes here hands. Hands are great. Hi. I think that one
 one one thing that we that we're having like really taken seriously so far is the role of
 weak supervision from a parental perspective. But if you think of like a parent and their baby,
 they're going to point at things they're going to say this is this, this is that. And you know,
 it has had like hands have had a huge role in our evolution as as homo sapiens. And it's even like
 thought that sign language preceded the appearance of voice speech. So that we probably have somewhere
 in our noggin, some areas which are highly selective for hand gestures, and which are
 used for a kind of weak supervision. That's important for for parents. So understanding
 what happens with that personal space and what what happens as as we use tools is clearly important
 from like just this that curiosity of how you know, we went from Australia to get the techies to
 the modern humans. And I think it's going to teach us a lot about yeah, what it means to be human.
 Awesome. Last question from my side with you're clearly interested in how the brain works, right?
 And and see and seeing, you know, can we can we make parallels between AI models, like deep models
 and brain areas and so on? Do you think that it is a necessity that we sort of feed back the knowledge
 into the deep learning realm? So should we should we put more effort into saying, how does the brain
 work? Okay, let's do that. Because at least that's that's like one example of where intelligence was
 achieved. Or do you think that, you know, how the brain works is just like a happenstance of nature
 and evolution and energy restrictions. And, you know, it's not it's not super like, let's just do AI,
 you know, the way it works best, or option three is something like, what, however we build AI,
 if we solve the task, it will automatically align with the brain, because there's like only one real
 way to solve the task, like in which, in which of these, let's say camps are do you find yourself in?
 Yeah, that's a that's super interesting. And I want to say that so people have made for a long time
 that claim that if we just study the brain, we'll be able to make better machines. Yeah, so that
 that comes about and again and again. And I do want to point out that this actually did happen,
 as we saw with convolutional neural networks, and the whole story of Yubil and Weasel and the
 Neocognitron and Yalda Kuhn and and eventually ImageNet 2012. But, you know, it's really only
 happened a few times, it's not clear how much more we have to like how much how many more instances
 of this will happen. That's certainly the view from from some people at DeepMind, for instance,
 that have really like gone into cognitive neuroscience and have started to do their own
 fMRI experiments to really, you know, tackle these problems. I think it's really, really interesting.
 But I'm not I think that it's going to teach us a lot about the human brain, but not necessarily
 about how to make intelligent machines, because we're, you know, like these are different systems,
 as you point out, and there are certainly things about the brain which are kludgy and, and, and
 certainly suboptimal. So how the retina is wired up is the classic example, it's wired up in the
 wrong way around, octopuses have haven't the right way around, and it doesn't seem to bother them.
 So that's a that's a clear example. But maybe there's some thing that we can that we can
 identify with with brains and that is going to unlock the next generation of machine learning.
 Maybe it's spiking neural networks, for instance, you know, people are demonstrating like,
 you could get something which is the like 1000 times or 10,000 times more energy efficient if
 you just use these mixed signals spiking neural networks. So I don't know.
 Yeah, that would I mean, 1000 times 10,000 times that is sort of the orders of magnitude you spoke
 about before when it came to to data. Well, those are so here, I'm thinking about
 the energy efficiency. So like one recurrent super comparable. No, I think like the the one thing I
 would point out here is that if you look at all these papers, and you add up all of the their,
 their training time and carbon emissions, it's it's probably like pretty substantial. Although I will
 say that, you know, the paper that that I'm the first author of here actually have the machine
 that I train this thing on like right here. And it's it's still like it's still a one GPU machine.
 So again, I encourage your your your viewers to to get into this because you can still do things
 with GTX 1080. That's awesome. But I think that one thing that's that's going to be really
 interesting is that by studying, you know, better machines, we'll be able to start to understand
 how to bring this back from the side of machine learning and bring it back into human health.
 So that's very interesting. And it's by and wide, hasn't been explored thus far. But that I'm kind
 of a fan of the opposite direction that most people are really going into. So I hope that
 that answers your question. I, I don't think that naturally, if you just train on your own network
 to solve a task, it's going to do it the same way that the brain does. But I think that's
 the brain does because I don't think that that's that's really pointed out. I don't think that
 GPT three does things the same way that a human does in any sort of meaningful way. No way.
 Even though they're both very good at language. Yeah, maybe GPT four.
 Well, if you ask Gary Marcus, he'll say that there's no way it'll never happen.
 Neurosymbolic AI all the way. Yeah. All right. Cool. Yeah. For every to everyone. Follow Patrick.
 The many he's written papers, lots of papers. You're also the CTO of Neuromatch Academy. Is
 that correct? So I, so I helped Neuromatch start actually, so I'm no longer CTO there. But it's a
 great occasion for for people that want to learn more about that intersection between neuroscience
 and artificial intelligence to to bring that about. So when we started this a couple of years ago,
 we just figured, oh, well, do a few video lectures and present that online. And it was at the start
 of the pandemic and people were bored. So the response was out of this world. So we have
 we had over 2000 applications and people from all over the world wanted to learn more about
 both neuroscience and artificial intelligence and their intersection. So we ended up having,
 I think, 1700 students in the first cohort and having 200 TAs. And so it became a big thing
 very fast. So I'm very happy that I helped bring that about. It was definitely one of the most
 stressful times in my life. But we could bring together people from very disparate backgrounds,
 whether it's people in emerging economies that are at local universities there, and people from
 from Ivy League universities in the US, Canada and, and the UK together and working with the
 same curriculum and under the same circumstances. So which was very cool. And then last year, we did
 the same but doubled in size as well. So I hope that we'll be able to, to double this year.
 I'm sure the announcement actually for for the next version of Neuromagic Academy will happen
 pretty soon. So if you have people in in your audience that are interested in that, I highly
 recommend to them to do that. It's a great occasion to learn. And we already have, you know,
 materials from last year online. So if you want to get started on your learning, you can do that
 today. Excellent. Cool. Well, Patrick, it was wonderful, wonderful having you here. This is a
 new world to me and I think for to a lot of people listening right here. So thank you so much. And I
 hope to see you again with with next year's review. Awesome.