The Future of AI is Self-Organizing and Self-Assembling (w/ Prof. Sebastian Risi)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper sebastian risi copenhagen minecraft ai self-assembly self assembly nanobots swarm bots swarm ai evolution ai evolutionary methods genetic algorithms neural cellular automata cellular automata nca graph neural networks gnns self organization ant colony ai swarm intelligence interview emergence emergent properties
#ai #selforganization #emergence

Read Sebastian's article here: https://sebastianrisi.com/self_assembling_ai/

OUTLINE:
0:00 - Introduction
2:25 - Start of Interview
4:00 - The intelligence of swarms
9:15 - The game of life & neural cellular automata
14:10 - What's missing from neural CAs?
17:20 - How does local computation compare to centralized computation?
25:40 - Applications beyond games and graphics
33:00 - Can we do away with goals?
35:30 - Where do these methods shine?
43:30 - The paradox of scales & brains
49:45 - Connections to graphical systems & GNNs
51:30 - Could this solve ARC?
57:45 - Where can people get started?

References:
https://sebastianrisi.com/
https://modl.ai/
https://sebastianrisi.com/self_assembling_ai/
https://twitter.com/risi1979/status/1519053654921293827?cxt=HHwWhsC9hYfQ4ZQqAAAA
https://distill.pub/2020/growing-ca/
https://arxiv.org/abs/2201.12360?source=techstories.org
https://distill.pub/2020/selforg/mnist/
https://arxiv.org/pdf/2204.11674.pdf
https://github.com/fchollet/ARC
https://github.com/volotat/ARC-Game
http://animalaiolympics.com/AAI/
https://www.deepmind.com/publications/alchemy-a-structured-task-distribution-for-meta-reinforcement-learning-f
https://melaniemitchell.me/BooksContent/CAGTReviews.html 

Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hey there, today I'm talking to Sebastian Riese, who is the director of the creative AI lab and the co director of the robotics, evolution and art lab at the IT University of Copenhagen. He's also the co founder of a company called model AI that uses AI for various aspects of game development. Specifically today, we're going to talk about a blog post that Sebastian wrote that's called the future of artificial intelligence is self organizing and self assembling, we're going to talk about systems that have no supervised instance controlling everything but contain little elements that all need to somehow communicate locally with their neighbors to come to an agreement about the whole thing. Think of something like an anthill just organizing in tiny parts to achieve a bigger goal. Now we've had massive success with these big supervised model, essentially a central instance controlling everything and that works wonders for the problems that we're currently solving. However, if you think of things like the most complex organisms that ever existed, which is probably human society, at least as far as we know, that is not supervised that has no central instance except the Illuminati. But you know, so essentially human society is self organizing and self assembling lots of little parts, making decisions on their own communicating locally. And what emerges is this absolutely beautiful thing. Now, as you can imagine, this is not mainstream self organizing and self assembling systems and related things like open ended and lifelong learning. These are not the current hype topics, but I believe strongly that they will be in the future. Things like this will play a big role when we push beyond the limits that we are definitely going to hit when using supervised and centrally controlled systems. Applications of this are numerous, I already mentioned things like game development. In fact, a lot of Sebastian's experiments are in things like Minecraft and other games just for visual, you know, in their research. However, the applications are possibly unbounded and could touch every area of AI and the greater field of technology. So join me this interview was absolutely awesome. You should follow Sebastian and follow his research and the research of his collaborators very, very interesting. I like it. It's out of the box. It's new, it's creative, it pushes beyond what I know. That is it for me. We'll dive into the interview. I'll see you around. Bye bye. Hello, everyone. Today, I have Sebastian Rizzi with me, who is a professor at in Copenhagen working in the general field of self organizing and self assembling systems, which is, I think an entire different world than the current paradigm that we're used to. We're used to having our deep networks, training them really top down with supervised signal, sometimes self supervised. But I guess that's still kind of like a top down supervision. There's gradient descent, there's all these things where essentially an outsider outside us human or or some some constraint is globally enforced. And there's an entirely different world that goes much more along the lines of nature. And that tries to come up with structure from from the bottom up and that I find this really cool and is really promising. And I think it's sort of can solve problems that are really hard to tackle with these classical algorithms. And I think the field is upcoming, even though it has existed for a long time. But I believe that is definitely worth to look at. So today, we'll talk about a first and foremost, this blog post, the future of artificial intelligence is self organizing and self assembling, but also a bunch of other things in this field. So Sebastian, welcome. And thank you so much for being here. Thanks a lot for the invitation. Very happy to be here. So why aren't you working on just scaling deep learning more and more to bigger and bigger models? What's the appeal of going like really small, really, really modular? Right? Yeah, I think there I mean, one reason is there a lot of people working on or in this field. So I like to work on things where they're, you know, there's there's maybe not so many people working on it. And I find this field particularly exciting. And we have seen that we can scale up deep learning and it can do like amazing things. But we have also seen that the systems still tend to be quite brittle. So we have reinforcement learning agents that that perform beyond human capabilities in some domains. But then you add a single pixel in this kind of the sock in this Atari breakout, and the system completely fell down. And there are a lot of other examples like image recognition examples where you slightly change an image or you rotate slightly and instead of detecting a fire bus, it's detecting something else. You have examples of Tesla driving into like an airplane because it mistakes it for something else. So these systems are amazing at a lot of things, but they're still very, very brittle in other tasks. And so that's why I'm particularly interested in this kind of idea of collective systems and self organization, because these systems have this inherent kind of robustness, you can take away parts, you can add parts. And the system will not completely break down because there's no central leader. It's like a self organizing process, a collective system. And that's what kind of fascinates me. And that's why I'm more recently we're going a lot in this direction. And it seems to be very fruitful direction where there's a lot of interesting things to discover that we haven't really looked at it. I think as a motivating example, we can show this thing right here, which is a collection of what are called swarm robots, or here it's called a robot swarm. Could you describe what is happening right here? What are we looking at? Right. This is a great work from Radhika Nagpal's group, where basically they have these kilobots, a thousand of them. And they follow a specific algorithm. And that allows these thousands of kilobots to assemble into a certain shape, like those shapes we see are like a star, a K, and I think this wrench. And this system shows basically they only have very limited information. These kilobots, they can only basically see their surroundings. But just by having this kind of local communication, these kilobots are able to, over time, to assemble into different shapes. And so this was one of the seminal papers that showed that you can run actually these kind of algorithms inspired by nature on a large scale, on a large swarm of robots. And this is basically like one great example of this. What it kind of what limited it is that those rules that those robots follow, like they have a specific plan, they needed to be designed by humans. So it's a human-made algorithm. They follow it and they can, you can compile it into making different shapes. But what we are more interested in is can we do similar things, but can we instead learn these rules with recent deep learning, machine learning methods, basically combining this deep learning with ideas from collective intelligence to create even more complex structures, growing more complex structures. This I think reminds a lot of people probably of something like ant colonies. Also, maybe not necessarily evolution, but the development of just cellular organisms in general, where there's not really, well, I'm going to step on some toes here, but an intelligent designer, you know, directing every step of the process up there. Is it fair to say that that these things you said inspired by nature? Is it fair to say that something like an ant colony implements one of these algorithms? Yeah, exactly. So it's inspired by what you see in swarms of animals, of insects doing like ants. They're like amazingly robust and they have this kind of collective intelligence that is bigger. They are made out of simple units, but together they do these amazing kind of things and termites. They build these amazing structures. And so I think for this work is actually, I think it was termites that was the main inspiration for this. And then you also have the same thing in the same kind of collective thing happens when through morphogenesis, like when we are grown basically from one cell by division and local communication, it's growing these like amazingly complex structures. And both processes show that by very simple rules, you can get amazing things. And there are many other examples. And one thing that these systems have in common is that you can remove parts and it still kind of works, which is very different to our current like neural networks where you change something slightly and oftentimes they will just break down. I think yeah, you demonstrate this later by training robots and then removing limbs of them and they can still kind of adjust to it. And I think the arch example of these local rules you have also in your blog post, which is this game of life, which is obviously, as you said, these are hand designed rules still give rise to like a really complex set of phenomenon, which is, I believe even like undecidable to really decide from a starting point. I'm not sure about the lore behind game of life. Yeah, exactly. I mean, they're basically you can build any I mean, with this, it's a universal computer, basically, you can build any kind of program if you that you would want with the cellular automata, of course, it would be like a super massive cellular automata. But they as you said, they show that even these kind of simple rules, they give rise to things that replicate things that move across the screen. And so people have found like all kinds of amazing structures by basically not changing the rules, but changing the starting configuration of these kind of cellular automata. When we think about combining this with deep learning, we quickly get to these neural what are called neural cellular automata. You have some examples right here. And I think I have the website open somewhere. This is work that appeared in distilled pub, which is obviously cool interactive journals. So this I think this was one of the first even articles to appear out of Google. And so this here, I can maybe interact with it, you can destroy parts of it, and it will kind of regrow it. And all of this is happening just by local interaction. So there is no, there's no kind of global organizing system that tells these things what to do. But every single pixel in here essentially has a feature vector and communicates with the neighbors. And how they communicate is am I correct to say that the way they communicate with each other, that is the part that is learned through deep learning. Exactly. Yeah, you can imagine like you have basically a copy of the same neural network like running in each cell. And that and that network takes into account like information from the neighbors, the neighbor state, and then it decides what should what should the next state of that pixel basically be. And you have these like RGB values, that's one thing it decides on. But then it also has these additional channels, like hidden channels that it can basically, it can decide what kind of information would be good to communicate to my neighbors. And so this work was not like the first that used neural networks to learn rules for cell automata, but it really kind of revived the field. And what it did is that it showed that you can actually you can make the whole system differentiable. So we tried similar things before where we used evolution to optimize neural networks, which is this field neuroevolution. But it's quite difficult for evolution if you have a specific target in mind, like you want to grow the salamander or you want to grow a certain other structure. It's quite hard for evolution to learn these kind of supervised tasks. And then basically this paper showed then if you have a target, you can just use recent tools like do auto diff, differentiate the whole system. And you can actually efficiently learn how to grow a certain structure that is only grown through these local communication of cells. And that was one of the that I think revived like the whole field. And there's a lot more papers now using neural networks for cell automata to grow all kinds of things, game levels, robots. How do you train such a thing? You said the full thing is differentiable and there is a target in this case, right? Is it the fact that you are in some starting state? Do you let it evolve for a couple of steps and then kind of measure the loss and then do something like back propagation through time? Yeah, exactly. Yeah. So you let it grow and then you measure like is it how close is it to the final output? And then it gives you the error to correct it. And then they do all kinds of tricks like that you want the system to be of course robust that if I let it grow for 50 steps, instead of like 20, I still want it to look like a salamander. So they do some kind of a few tricks that like doing it stochastically and letting grow for different amounts of time to get the system to be that it grows and it also kind of knows when to stop growing because that's an important part. Also nature like if through morphogenesis it grows an organ, it should know when to stop growing that organ and then like not grow forever. So that's one important ability of the systems is to learn kind of when to stop. If you were to let's say criticize this particular work, what would your criticism be? What's still missing from this? Or where is it weak? Yeah, so this what this showed is that it's basically it doesn't if you would critique it that you would you could say that it does not but that was also not the goal. It doesn't discover the structure itself. It has a target. So it has some kind of human design target like the salamander that is drawn by a human. And so in that case, that's one limitation. So actually one follow up work that we will be published soon, we actually combined evolution and this system where evolution we let evolution come up with like these soft robot in that case. And evolution is good at discovering like variety of different morphologies. And then we use basically this method to make the structure very robust. So we let evolution discover the structure and then we cut off all kinds of limbs and let it regrow. So combining kind of the creativity of evolution with this kind of making things robust through this gradient descent based training. That is the yeah, the work on soft robots. I've seen that it just looks really cool. So this would be one thing that is that is discovered this sort of kind of hopping tripod. And obviously this, I think soft robotics in general are rather new field and combining them up with like evolving system seems quite appropriate. So here's one with a cut off limb and you can learn to regrow, regrow it. Right? How in general, how do you teach a self organizing system to regrow things? Do you have to explicitly program? Like you have to explicitly train it to regrow things? Or is this just a natural consequence out of how the system was trained in the first place? Yeah, so sometimes it can often it already has some inherent robustness, but it will without explicit training, it will probably not be able to do this like perfectly. And it will be that it sometimes works and sometimes doesn't. So in these cases, we explicitly and also in the case of the work by Google, like they explicitly like you explicitly remove stuff during the training process so that you confront the system with, you know, this kind of this damage that it has to recover from. So it makes the system more robust if you specifically train for it. And I guess in nature, that's probably one reason that the system had to work for all these different environments. So there is a lot of like they weren't in your aunt colonies, sometimes you had more, sometimes you had less and so these systems are because of the way they were, they are evolved. They also show this kind of similar level of like superior level of robustness. At this point, are we already at the point where you would say that this surpasses or this is very advantageous to classical deep learning? Or are we still in the realm where, let's say, everything would be fairly possible with classic supervised top down deep learning? I think like this, it, it would be possible to have it grow and recover. But I think that the secret here is that it only uses local communication. Basically, you could of course have a network that would, I don't know, a network that you query that would could like similarly to earlier work like compositional pattern producing networks, CPPNs, where you query basically each location in space and you ask it what should the voxel be? And of course, these systems could then if there's damage, they could you could ask them again and they could recover. But the trick here is, is that it's only based on local communication. So if we ever want these things to work in the real world, then it's really advantageous to have things that only require local communication, basically. And so that's one whole that's one goal is that ultimately, we want to take those systems from also the simulation later on and you know, we have some initial work and we want to really create complex things also in the in the physical world. If you say in the in the physical world, because if I if I think of there was, oh, no, this was a this, the paper, the physical cell either automata is at least a thing that is doable in the in the real world. But if I think of something like, I don't know, a Tesla car or something like this, that is in the real world. Yet, it is still, you know, a central controller that controls the whole car, and there are still top down and so on. And it's also trained in that way. What are the types of physical situations where these would really the local communication would really come in handy? Yeah, like I could imagine like, let's say you have a building or something that could automatically detect if it's damaged, and then you know, it could like our you know, our skin, it, you know, it's damaged, and it's it's regrowing, it's, it's self, self healing. So you could ultimately, I mean, this is like science fiction, but imagine a building and then you it gets damaged, and then automatically it recognizes it's damaged. And then it, you know, automatically recovers from this damage. More other like science sci fi is if you have, imagine you have a swarm of nanobots, they only can communicate locally, right, but they have to figure out their shape, they have to figure out their what they can do in an environment. So in those situations, this local communication would be very advantageous. I don't know if it would necessarily be useful for these kind of, you know, Tesla, this car example. But but I could imagine a lot of other like application areas or drones that have to coordinate somehow together only being able to sense each other locally. So more these kind of in that areas. One thing I'm quite excited about is this getting this from like this 2d version to a 3d version. And then you can imagine building all kinds of things and it would automatically know you're building, you know, a table or you're building a chair or you're building this and this, which which I think it's quite so. So this is one example also of so yeah, the self classifying MNIST digits, where basically the system cannot only be used to grow something, but it can also be used to self infer its own shape. So you build something out of small components, or you draw like a digit. And then by having the cells communicate with each other, they figure out, oh, I'm part of an eight or I'm part of a one. And so basically, this is then what we replicated in this physical where you can put them together, make digits, and then each each of these cells will tell would figure out what part what shape am I part of. So this, this is a physical instantiation of the demo I have here online. This is another distal article where as you exactly said, these things, they figure out themselves what they're part of. And you made you made this this is your paper into a physical instantiation, which I find really cool. And now you're taking it to 3d. Yeah, yeah, that's the that's the plan. Yeah. And of course, currently, these systems, like this kind of self classifying MNIST digits, it does not work as well as like using like, like state of the art, deep convolutional neural network or transformer or what what you have. But I think ultimately, these systems, maybe we can integrate some ideas also for things like object detection to make these systems kind of more robust by having a more kind of distributed object detection where you have this system where the components, maybe it could be a combination of something convolutional and but then you have the system on top, where you have this local communication and they figure out together kind of what shape am I looking at and maybe that could make these systems also more robust in the future. And maybe less prone to kind of this adversarial attacks that we currently see the system still exhibit. Has anyone tried with like, maybe this would be interesting, like to take something like this, and try to like make an adversarial, I don't even know how that would look like, but something that a human would clearly classify as like a seven, but there's like a slight twist. Yeah, yeah, I'm not sure people have actually studied it so much on this, trying to see how what kind of adversarial attacks these systems could, I mean, fool like you could fool them. I'm sure there are also some. But maybe the combination of kind of both this and more classic deep image recognition techniques could make them more robust. So you've taken also this idea of this 2D cellular automata, and you've applied this in 3D here in Minecraft, which so this is a morphogenesis. How do you how would you define morphogenesis just quickly? Yeah, I would define morphogenesis as like growing a complex structure based also on this kind of local communication. So how our like bodies are grown is morphogenesis, how our like organs are grown, how our nervous systems is grown basically, from like, you know, a single starting cell. And so this is what we do here. And again, the structures are not found by the system itself, like we took like an existing apartment building. And then we trained the system in the same supervised way to regrow it basically. And we were surprised that it could also grow like these kind of functional machines, we actually had it growing like, like this temple. And then we found that the trap in this temple still worked. So because it had all the components, like there was not one single mistake. And that allowed these kind of functional things to still to still work like this kind of like caterpillar you see there. And can you can you you also said you can destroy part of it and it will regrow, right, which Yeah, is this have you made this playable somewhere in Minecraft itself? Or is this just purely your Yeah, it's currently it's it's not I mean, you can download the code and stuff. But it's not that we have a server where you can play with those things. But it would be very interesting. We actually we organized this Minecraft open endedness competition where we like a related field like can you have an algorithm that can like natural evolution create all kinds of novel things without limits. And that's also where we use this Minecraft framework. But it would be real fun. Like one thing that I that I want to try to also pursue in the future. Imagine you don't have it grow like caterpillars, but you have it grow like cities. And then depending on the environment that you as the human does decide, like the mountains or like the desert, it would grow a different type of city. So like, that's one thing we're looking at now, how can you incorporate also feedback back into the album, because this caterpillar will always grow the same caterpillar. But if if I put this caterpillar in a in a small box, it should maybe grow a small caterpillar. And if it's a large box, it should grow a large caterpillar. So how can you kind of incorporate this environmental feedback? That's another thing that I'm very curious about. Yeah, is do you see beyond beyond gaming, maybe which which I can definitely see applications of this? Do you see applications that are not in the physical world as we talked before, but but maybe in the in the still in the realm of the digital world? Are there applications? I don't know what what what all you you're thinking of, but distributed applications, networking applications, any sort of things that you're very excited about that maybe aren't super obvious if you just see the the Minecraft example. Right. I mean, one thing that we are basically I think like two things. One is like just this Minecraft, I think, could also ultimately teach us something about biology itself. So if we could because we don't know everything yet about how this exact morphogenesis process works in nature. I mean, we know a lot of things, but we don't know, for example, how is it so accurate? Like and and and so there are certain things that we are we don't know yet. And so by simulating these process like a very simplified model, but maybe there's things we can learn from these kind of very simple models. So that's one one area I'm also very excited about. And so taking these systems to as a as a very simplified models biology to learn something. The other thing, the other application area is what I'm excited about is using those things. But instead of growing Minecraft structures, you can grow actually artificial neural networks. So so so you're basically kind of replicating our brains are not like designed and fixed, they're grown like through this developmental process. So what what we did with this recent work is hyper NCA is taken basically, instead of having growing a caterpillar, we grow a pattern. And then we then we with a neural cell automata, and then we convert that pattern into a policy network. And that policy network then is we can use this for our RL task, for example. So that's one one area I'm very excited about and making this systems more, more performant, because currently we apply to quite simple problems. But I think ultimately, this kind of idea of this growing neural networks is can be very powerful, because that's how you know, our brains are created. So so we're trying to replicate that process, hoping to create also more, more adaptive, basically neural networks. What do I gain out of so in this here, I have these developmental steps on the left, I do essentially start with some configuration of weights, essentially. And then I let the cellular automata run for a number of steps self organizing here, then I take it into a network, and then I execute the network. And presumably, I have to learn this somehow. In this paper, what you are doing is you're using, if I recall correctly, a variant of evolutionary search, right? I could also, like, I know, in whatever way I learn it, I somehow have to learn how the cellular automata here reacts. What do I gain out of this instead of just training my policy net? Right. So so far, I would say it's the you don't get so much directly. So so far, this method, it's not that they outperform like current deep RL methods. But ultimately, basically, there is this this hypothesis, also popularized more recently by Tony's adore this kind of genomic bottleneck hypothesis that means that we only have, you know, 20,000 genes, and they, they, they guide the growth and self organization of our brains with trillions of connections. And and and so it's a much smaller genotype that encodes a much larger structure. And so this kind of compression is hypothesized to also allows us to and animals to deal with situation they haven't seen, like to basically that the robustness that animals show is part because they have to go through this bottleneck this compression. And this is the information you give to the next generation. So there's some limit on the information you can get. So that might bias the system towards learning rules that generalize well, like learning rules that generalize well. And so this is the the hypothesis here, that at some point, we can have a very small neural cell automata, which is basically like the genome and that encodes a much larger network and that hopefully would then be more robust. But that's something we have. That's basically what we're working on, which we which we haven't really shown yet. But that's the that's the hypothesis and the hope. One other thing that's kind of funny that it can do like it can you can basically let the growth continue and not just have one network grown, but multiple networks. So like we applied this to this quadruped domain. So we had it grow for for 10 steps to grow one brain like one network, then we put it into this quadruped. And we have a slightly larger quadruped. So we let it grow for longer, and then put it in the middle quadruped and then have a larger one. So and so basically one NCA can grow multiple different neural networks. And that's also one thing that I'm pretty excited about that we want to apply also for like more complex domains. And again, here you had an experiment with with where you damaged these quarter pads, the system is able to adjust, can you explain how this system is able to adjust to a damaged morphology, like a cut off a limb or something? So here it was basically trained to on these, like on all these different morphologies. And then we had it basically, by continuing the growth, you can get a controller that was trained for one morphology, and then you continue it and you get a controller that works for M2 and you let it grow a little longer and it has a morphology for M3. So in this case, those were basically seen during some other experiments, we have results where it has damage that was not seen during training here, basically was trained to being able to deal with this particular type. So if we would damage it in another way, it probably wouldn't work anymore with these metamorphosis networks. But yeah, so the hope is also that if you know how to control one quadruped, then there should be that you don't have to start basically from scratch, there should be some information there that allows you to also grow something that is related, and not having to start like all over again, basically. This flows, I think, into a lot of a lot of ideas from, as you said, the open ended community and the sort of don't have explicit goals community. I think parts of your blog posts and papers mentioned algorithms like quality, diversity, map elites, and things like this, which are obviously very exciting and very different from how we do deep learning today. So far, we've always looked at things that have either an explicit goal, like here is the salamander I want to build, or here is the Minecraft structure I want to build, or have some sort of, I want to say, goal in an in a more abstract sense, like the reinforcement learning goal of maximizing the height in this case, right for these robots that stand on top of one another. Yet, how do we go away from this? Is there is there a natural progression in these self organizing systems to go away from having explicit goals that would be more difficult to pursue with like the classic deep learning systems? Right, I think in general, so I think that, like two things like one is the representation, which I think these neural cell automata are like a great representation for a lot of like growing structures growing neural networks. And then the other thing is you mentioned is like the search, how do we actually get to systems that show interesting, these interesting properties. And so there seems to be a recent trend, I mean, not just in the self organizing systems, but in also in deep RL in general, to not train on one thing basically, but train on a variety of different things. So there was also this more recent paper by I think it was DeepMind where they this XLL that they showed like basically, if you train agents in a lot of different changing environments, they develop more robust skills basically. So I think basically here it's we also what I think it makes these self organizing systems quite difficult to train is that these landscapes, the fitness landscapes basically, they are probably very kind of not very smooth, because changing like something small in the self organizing systems can have like this cascading effect. So that's why these traditional objective based rewards, they work, but then they don't, it's still difficult to optimize. So that's why we're more looking into this kind of open ended, like what you mentioned quality diversity methods basically, where we're not trying to optimize for one particular outcome. But we're trying to find things that differ in some interesting ways basically. And I think those methods, particularly for this kind of self organization, they are very, very powerful basically. They are better at navigating like these kind of very complex landscapes with many local optima, but they're also slightly more expensive because they're looking at the larger space of this of the search space basically. What maybe these two questions in one given given these outlooks, what field that deep learning is good at right now? Do you expect these methods to be better? If you know, let's say if we invest the resources and figure out, you know, the tricks of the trade enough, what parts that deep learning is good at now? Could these methods overtake deep learning? And then on the other hand, what's kind of the, for you, the most exciting area that we haven't even unlocked yet with deep learning that are accessible with this? Right? So it's two different, two different things, but I'm wondering about what you think about both of these directions. Right. So I think it's also, I wouldn't say like overtake deep learning. I mean, we use basically we use deep learning as a tool for basically like kind of train the system. So I think, Yeah, sorry. I mean, deep learning and like the, just the thing we do right now, right? We have objective loss, supervised training, single neural network. So I would assume that these systems would be able to have a lot of different domains. I think the one kind of probably the closest, I think what we would see is that they would make our L agents more, you know, like more robust, more adaptive. And that's also already in this work that you that we have there is like where we have basically in this case, we trained not only the, we have completely random weights and we only trained local update rules, basically the habit rules. And then we show that through this system, we can actually during the lifetime cut off a leg. Again, we are always somehow mutilating these robots. We're not very nice to them. But basically, this is an example, I think, where we already show that is this is more adaptive than the current RL design. So in the current basically deep RL, I think the one main drawback is that we train a system and then we freeze the neural network and then let it do its tasks. And this seems like kind of very unnatural that like you have a frozen brain. Okay, maybe you have like some recurrent connection that allow you to learn something. But basically, we have this training period, then we freeze everything in the system and we apply it to domains. So that's not like lifetime learning in normally these systems. But the idea here is, in general self-organization, that we never wanted to stop learning, we never wanted to stop adapting, we want the self-organizing process to happening the whole time. So I think in any domain, where there are things that you might not have anticipated during test time, these systems could be beneficial. Like might it be there's a pixel edit, you're losing a leg or you wanted to do something else. I think that they already show that there's some, they can be superior in those domains. And that's one thing that I'm pretty excited about to apply them to more complicated domains, not just these like quadruped locomotion tasks, basically. But anything where you have something unanticipated happening, I think there will be can be a benefit of it. And then was the second question like what other a new area that we haven't even like we have no chance currently of tackling with our tools? Yeah, that's a great question. I mean, I think this new area is this kind of rapid lifetime adaptation basically, I think these systems are great for if you know what you would expect. But things like basically like having things that work in unknown environments, I think that's a really, I think exciting area that I mean, you have like animals in nature and you can put a dog into a new environment and will not completely like break down and will still know kind of what to do and to interact with the environment. And we don't have that yet for our agents, like we can put them in environments they're trained for, you put them too far out, they don't know what to do. So and I think that too, that's so this working in other environments and also having this kind of like, you know, common sense, I think is maybe also an area I think in the future that these systems could be applied to although I don't know exactly how but but that these systems have more common sense and don't directly break down like kind of giving them this kind of innate abilities that we humans are born with animals are some animals are born with that allows them to yeah, do a little bit more common sense things than than current deep learning system that don't have that property basically. And this, I think you say it even here at some point. This, in addition to the fact that there is this genomic bottleneck, right, you already said this, the genes encode or only have the capacity to encode very little information. And what we're doing here is we're learning essentially the rules to learn the rules, which can be compressed in a much better way than the rules themselves. And there is a reason to assume that this will result in that kind of common sense that if you have to essentially learn the meta rule, then that will make you generalize better. I mean, it's an it's an argument, I'm not super convinced yet. Right. But if you do then some parameter sharing, you showed in some experiments, you can compress this even further. So that might be a way to tackle that. And also this in Tony's adores paper, he actually he points out that this bottleneck, like there's some organism nature that have many more genes, for example. So maybe it is a feature that we have that number of genes that it's compressed. And so so that gives us like some hope that also having the similar feature in our artificial systems should be beneficial. But but we're still we only showed that for very, very simple, you know, simple tasks so far. And deep learning goes into the exact opposite directions, right? We're like the more the more parameters, the better we have the double descent phenomenon, and we can go essentially infinite and it always gets better, which is which is weird, right? Which is also giving amazing results, I think recently with you know, the whole language models and so on. So it's definitely it could it would be cool if in the near future, people discover like a fundamental connection between, you know, the the good results we get by scaling up, and the the actual principle from biology, which is seems to be more like compressing and scaling down, it would be nice if those were to join together somehow. And hopefully, we can be part of that in some extent. But yeah, I agree. It's really interesting that like that you Yeah, you scale up networks, and then your local optima disappear, like everything just works better. And here we basically we want to go the opposite direction. But it's not necessarily that we, of course, we still want our the final models to have trillions of of of like connections. But we what we basically want is we want the trainable parameters to be low. And I think that that's the fundamental difference that we have a small number of train or relatively small number of trainable parameters there, but they give rise to much more complicated system, exploiting things like self organization growth over time. And, yeah. This is I think, because you said before, you're not you're not an opponent of deep learning. In fact, deep learning is used inside of the cellular automata to to sort of learn these rules. I find it interesting, if you look in nature, that there are cells and they self organize in some way, right, by whatever means that is learned. But these cells then make up brains, right? And brains are naturally very top down planners. They're they're, they're in the moment, they, you know, look ahead. And then the brain somehow organizing to societies and the societies again, are very distributed, very local, very interaction on a person to person level. What do you what do you make of this? Do you think there is like an optimal switch from local to global to local to global that we could sort of stack on top of one another? Or is this just a happenstance of of the universe? Yeah, that's a Yeah, that's a that's a great question. And even more like the humans in the societies, they organize themselves into hierarchies, right? Top down control and somehow it gets even crazy. It's a good question. Do we need one? Yeah, do we need all of this in our artificial systems? Maybe we need all of this to get to real like more general artificial intelligence. Like because also one thing that is really crucial is the our culture, right? Like, like, if you if you I was reading this great book recently, like if you just put humans somewhere by themselves, they're not very like, you know, good at surviving, but we are good at surviving because we have all this cultural information, like all this knowledge that other people made that that we can build on. And that allows us to do all these amazing things. So maybe to get our eyes to do really amazing things, it's not enough to having like single agents in complex environments, but it needs to be multiple agents that need to be simulated maybe over multiple generations. So there can be some cultural knowledge transferred from some agents to other agents, similarly to how how it happens in for us. But of course, that also makes the simulations much more complex and expensive. When you have to simulate cultures, multiple like generations, and then we need some more better compute, especially at the university level. I think yeah, that's one advantage that nature has it has lots of lots of distributed compute available. That said that there is there is an interesting part in your blog post where you describe sort of how to train these things, or how to steer the development of these swarm systems or distributed systems. One one quote here you have is guiding a swarm system can only be done as a shepherd would drive a herd by applying force at crucial leverage points by subverting the natural tendencies of the system. And then another one is the self assembling brain knows no shortcuts in which your I believe your argument was a little bit that is very hard to predict what a change does until you observe it because the interactions can be kind of nonlinear, very dynamic, very, very hard to predict. In essence, that was basically the argument that that hissing are made in his this great book like self organizing, no self assembling brain. And basically that you need to basically the system needs this process of growth. And you have to put energy into it to observe the outcome you cannot predict. And that's also things they showed that Wolfram what he showed with simple one diesel automata, you cannot predict the state of the system, you have to actually run the system even if it's a simple one diesel automata. And that is also apparently the question is, do we also need to do that for to growing our neural networks instead of like designing them? Maybe we need to go through this kind of process of growth with learned rules to to really unlock you know what these systems can do. There is recent work in using for example, GANs or so to predict things like fluid dynamics and you know, they can't do it like super, like they're not extremely accurate, but they can give a pretty good estimate of given starting state and then a highly dynamic nonlinear system. And then they can predict some steps into the future, I've seen the same like galaxy development and so on. Do is there any happening like this where you can say, Well, I don't I can't, I don't have enough compute to run all these swarms, but I can sort of train a surrogate model that will give me the end in sort of a one step fashion. And then these the forces that I poke at the swarm at I could determine those using the surrogate model. Yeah, I think that that would be really interesting. I wonder I think it's, it could work for some limited steps in the future. But but but I think you you would still need to, you know, like, like at some point, you need to basically run this this model. I mean, maybe in the first like generations, you could help have so great model that somehow helps you to sort out like the things that are really bad, like, this will not grow into anything. So I think you could use it there later, I guess you would probably have to run the system like when things get more complex. But I but I think there's also another role for the surrogate models, which something I always wanted to try to predict basically the learning abilities of the system. So you have an agent in an environment. So maybe you don't need to simulate the whole lifetime, right? But you can have some more like some kind of some tests that would test is this agent, how capable is this agent, so having some kind of surrogate that would could look at certain parts of I don't know the neural network and already predict, will this be a good learner or not basically. But yeah, it is in the in one part you also it has very, can very remember like I got into machine learning and graphical models were the hot thing at that point, it was just before deep learning. And this reminds me all this self organizing systems with the local communication, they remind me a lot of belief propagation, things like this graph neural networks, obviously are right now up and coming, let's say, do you see connections between all of those things? Or is that just kind of a superficial connection? Yeah, I definitely see there's a big connection to these also these graph neural networks, basically, like, I mean, they're very close to like a more generalized form basically of like a cell automata, where you have different basically neighborhoods, depending on your the topology of the graph. And they also seem to be there. I think they're super interesting. I also actually how I got into neural networks is the the first lecture I had as an undergrad was actually on neural networks and about these self organizing maps, which these coho and self organizing maps that basically can do clustering based on this somehow like kind of like kimmins, but on a on a much more, they can do it better. And you have to get these like nice visualizations out of them. And apparently, there's also some pros in our brain. I mean, we have these topographic maps also in our brains. I was always fascinated somehow by these self organizing maps. And even though I did a lot of like some other things during my PhD, somehow now I'm coming back to this kind of self organization. And and and yeah, using these recently learning tools, it's I think we can really unlock like the power behind them. There was a Do you know the arc challenge? The abstract reasoning corpus by Francois? Yeah, yeah, yeah. There is I'm not sure if they have an example right here. So for everyone who doesn't know this, this is a task where you get so the left ones are demonstration examples, there's always like an input grid and an output grid. And then you get a test example where you only get the input. So here, the rule is I've looked at that before. So the rule is kind of there is the gray in the middle, and you kind of fold the right hand side onto the left hand side and then you the solution here on the right hand side is kind of the the sum of the two. And this is these are things that humans are surprisingly good at, but are very difficult for a machine to learn. And the this is a data set and the training examples, there are not many training examples. So there is not really a way to to learn this through brute force training. There is a little game that people can play, I think I've reported on this before, but there is a game for anyone who's interested, where this is the arc game, you can find it on the GitHub page on of of Alexei Borski. And you can just choose one here, they're divided into different levels. And yeah, you can you can try them for yourself. So this, this looks even familiar, like cellular automata. Do you think that it like self organizing systems in one way or another in the way we've looked at them today, or in the way you've seen them could be useful in solving challenges like these, because challenges like these are related very much to, let's say, something that we would call intelligence. Yeah, I think the the, the hope would be that if we can get this kind of bottleneck algorithms to work where we exploit, so I'm not sure it like we could apply like self organization directly. But what I could imagine is that we exploit develop these kind of genomic bottleneck algorithms that can guide this self organization growth of a very complex neural network and that that network then could maybe be used for these kind of tasks. And the hope would be that because it has this compression, it would maybe develop an algorithm that would allow it to, you know, solve these kind of tasks that require more like high level cognitive skills. But but of course, that's still Yeah, we're still a little far away from that, I think. And I guess I don't know what the current state of the art and in this task is. How? I think it's, I think it's still largely unsolved. So this could be a great test domain, I think. But yeah, I think I, I'm not sure I have high hopes that it would already like, I think we still probably missing some other ingredients that we don't have yet to kind of make progress there. Yeah, but by the way, this, I think I just clicked on on one randomly. But I think here, the rule as I think if people get it, they can see that you always kind of select the smallest of the shapes that is there and kind of replicate it. You know, at least that's my that's my hypothesis, right? Yeah, maybe, maybe. Oh, I think maybe you take the one that fits in the box. Oh, yeah, yeah, yeah. Right. But it's like this, this, this kind of, like, you need to understand what shapes are and so on. So that is very much that this is very high level. This is very bottlenecky. It has a bottlenecky feel to it. Like, you're probably not going to get very far with like a CNN trained on these pixels directly. So that's, that's like I can see something like this very much be in the domain of of like, first open endedness, but then also self organizing things made up like simple rules making up something very complicated. There's two other domains that I think also very exciting, like one is this animal AI benchmark, where basically they it's like an animal AI Olympics where you apply eyes to tasks that animals normally are good at, like, and like, for example, trying to figure out which one is the tool and then you use that tool to, you know, get a reward. And so there's also where current methods basically, they've pretty much fail on more complicated tasks. And then they also had a mid-term experiments where they had children perform these tasks and they are still much better at than than like any of our deep RL methods. So in the simple task, deeper RL performs pretty well. Once it gets to more complicated things, then they the system basically, these systems fail. So this is one task that like, in the recent grant proposal that I proposed that that there would be a good test domain for these methods basically, because the whole point is to act in an environment that you haven't seen during training. Even though the environment is made out of the same building blocks, like there's rewards, there's like barriers, but how they are composed, all of this is new, basically, and never seen before. And the other one is this also by I think was the mind is alchemy task where you have to learn to kind of it's a task that we have to learn basically about the structure of the domain, what things you can put together, and then you have to use that knowledge to like building on that knowledge basically. And this is also a very difficult task for all of our current methods. So I think this could also be very good task to basically as the North Star to drive these the progress in this kind of area. And the hope is that these kind of self organizing system, they should be, hopefully would be better at in this where can people if someone wants to get started in diving into the world of self organizing systems, swarm intelligence, maybe a bit of open endedness, is there a good place for people to get started like get their their feet? Yeah, I would say I was recently rereading this great book from Melanie Mitchell, this complexity. I think this is a great starting book on on kind of this ideas of complex system self organization. There's something about cellular automata in there. So I think this is a this is a good kind of good point to get a broader overview of of that kind of whole field of basically complex system self organization. And yeah, and hopefully the also the the blog post hopefully can be helpful to some people and also plan to write more on on that as well. But but this I would suggest this is a this is definitely a good place to start. And is there some some, you know, in, in deep learning, it's usually Keras, I train a CNN on MNIST or CIFAR 10. Is there like some some standard thing that every one of your of your students goes through? I mean, now I sent a lot of them to this great distill article basically and looking at this this growing NCAs because they also have a great, like this collab notebook where you can play with the system. So I think this is a great starting point to where you both have neural like you have cellular automata and you have like how recent tools can be used to grow them. So I think this is a good good place to play around with basically. Okay. Yeah, I've I've spent more than more than more time than I've had on these things because they're quite It's great that it's also so interactive and fun to play with. Yes, definitely. Yeah, I think is there anything else that you would like to get out there to people about this field? Yeah, I just Yeah, I hope that people would be not only everybody running basically in the same direction just doing like what everybody else is doing. So hopefully this will be also get a few more people into this field of complex systems and self organizing systems and combining the ideas of deep learning. Because I think there's a lot of things interesting things to discover basically here and a little bit less people working on it then then the heart like like working on foundation models and language models and all those other things. Yeah, it's certainly I think I think is certainly an interesting area. And I guess especially if you're at a university without the super duper clusters. Probably just strategically a PhD in this field would maybe be more of a advantageous position for new newcomers to the field. Actually, like Hinton had this great quote recently on this other podcast, like it's always a good idea to figure out what huge numbers of very smart people are working on and to work on something else. Because you don't want to do maybe what what everybody else is doing. And I think so I would suggest this is a great field where a lot of I think interesting discoveries basically waiting to happen. I agree. All right. So Sebastian, thank you very much for being here today. This was very cool. I hope to see yeah I hope to see a sprawling future for your field. Thanks a lot for the invite. Thanks.
 Hey there, today I'm talking to Sebastian Riese, who is the director of the creative
 AI lab and the co director of the robotics, evolution and art lab at the IT University
 of Copenhagen. He's also the co founder of a company called model AI that uses AI for
 various aspects of game development. Specifically today, we're going to talk about a blog post
 that Sebastian wrote that's called the future of artificial intelligence is self organizing
 and self assembling, we're going to talk about systems that have no supervised instance controlling
 everything but contain little elements that all need to somehow communicate locally with
 their neighbors to come to an agreement about the whole thing. Think of something like an
 anthill just organizing in tiny parts to achieve a bigger goal. Now we've had massive success
 with these big supervised model, essentially a central instance controlling everything
 and that works wonders for the problems that we're currently solving. However, if you think
 of things like the most complex organisms that ever existed, which is probably human
 society, at least as far as we know, that is not supervised that has no central instance
 except the Illuminati. But you know, so essentially human society is self organizing and self
 assembling lots of little parts, making decisions on their own communicating locally. And what
 emerges is this absolutely beautiful thing. Now, as you can imagine, this is not mainstream
 self organizing and self assembling systems and related things like open ended and lifelong
 learning. These are not the current hype topics, but I believe strongly that they will be in
 the future. Things like this will play a big role when we push beyond the limits that we
 are definitely going to hit when using supervised and centrally controlled systems. Applications
 of this are numerous, I already mentioned things like game development. In fact, a lot
 of Sebastian's experiments are in things like Minecraft and other games just for visual,
 you know, in their research. However, the applications are possibly unbounded and could
 touch every area of AI and the greater field of technology. So join me this interview was
 absolutely awesome. You should follow Sebastian and follow his research and the research of
 his collaborators very, very interesting. I like it. It's out of the box. It's new,
 it's creative, it pushes beyond what I know. That is it for me. We'll dive into the interview.
 I'll see you around. Bye bye. Hello, everyone. Today, I have Sebastian Rizzi with me, who
 is a professor at in Copenhagen working in the general field of self organizing and self
 assembling systems, which is, I think an entire different world than the current paradigm
 that we're used to. We're used to having our deep networks, training them really top down
 with supervised signal, sometimes self supervised. But I guess that's still kind of like a top
 down supervision. There's gradient descent, there's all these things where essentially
 an outsider outside us human or or some some constraint is globally enforced. And there's
 an entirely different world that goes much more along the lines of nature. And that tries
 to come up with structure from from the bottom up and that I find this really cool and is
 really promising. And I think it's sort of can solve problems that are really hard to
 tackle with these classical algorithms. And I think the field is upcoming, even though
 it has existed for a long time. But I believe that is definitely worth to look at. So today,
 we'll talk about a first and foremost, this blog post, the future of artificial intelligence
 is self organizing and self assembling, but also a bunch of other things in this field.
 So Sebastian, welcome. And thank you so much for being here. Thanks a lot for the invitation.
 Very happy to be here. So why aren't you working on just scaling deep learning more and more
 to bigger and bigger models? What's the appeal of going like really small, really, really
 modular? Right? Yeah, I think there I mean, one reason is there a lot of people working
 on or in this field. So I like to work on things where they're, you know, there's there's
 maybe not so many people working on it. And I find this field particularly exciting. And
 we have seen that we can scale up deep learning and it can do like amazing things. But we
 have also seen that the systems still tend to be quite brittle. So we have reinforcement
 learning agents that that perform beyond human capabilities in some domains. But then you
 add a single pixel in this kind of the sock in this Atari breakout, and the system completely
 fell down. And there are a lot of other examples like image recognition examples where you
 slightly change an image or you rotate slightly and instead of detecting a fire bus, it's
 detecting something else. You have examples of Tesla driving into like an airplane because
 it mistakes it for something else. So these systems are amazing at a lot of things, but
 they're still very, very brittle in other tasks. And so that's why I'm particularly
 interested in this kind of idea of collective systems and self organization, because these
 systems have this inherent kind of robustness, you can take away parts, you can add parts.
 And the system will not completely break down because there's no central leader. It's like
 a self organizing process, a collective system. And that's what kind of fascinates me. And
 that's why I'm more recently we're going a lot in this direction. And it seems to be
 very fruitful direction where there's a lot of interesting things to discover that we
 haven't really looked at it.
 I think as a motivating example, we can show this thing right here, which is a collection
 of what are called swarm robots, or here it's called a robot swarm. Could you describe what
 is happening right here? What are we looking at?
 Right. This is a great work from Radhika Nagpal's group, where basically they have these kilobots,
 a thousand of them. And they follow a specific algorithm. And that allows these thousands
 of kilobots to assemble into a certain shape, like those shapes we see are like a star,
 a K, and I think this wrench. And this system shows basically they only have very limited
 information. These kilobots, they can only basically see their surroundings. But just
 by having this kind of local communication, these kilobots are able to, over time, to
 assemble into different shapes. And so this was one of the seminal papers that showed
 that you can run actually these kind of algorithms inspired by nature on a large scale, on a large
 swarm of robots. And this is basically like one great example of this. What it kind of
 what limited it is that those rules that those robots follow, like they have a specific plan,
 they needed to be designed by humans. So it's a human-made algorithm. They follow it and
 they can, you can compile it into making different shapes. But what we are more interested in
 is can we do similar things, but can we instead learn these rules with recent deep learning,
 machine learning methods, basically combining this deep learning with ideas from collective
 intelligence to create even more complex structures, growing more complex structures.
 This I think reminds a lot of people probably of something like ant colonies. Also, maybe
 not necessarily evolution, but the development of just cellular organisms in general, where
 there's not really, well, I'm going to step on some toes here, but an intelligent designer,
 you know, directing every step of the process up there. Is it fair to say that that these
 things you said inspired by nature? Is it fair to say that something like an ant colony
 implements one of these algorithms?
 Yeah, exactly. So it's inspired by what you see in swarms of animals, of insects doing
 like ants. They're like amazingly robust and they have this kind of collective intelligence
 that is bigger. They are made out of simple units, but together they do these amazing
 kind of things and termites. They build these amazing structures. And so I think for this
 work is actually, I think it was termites that was the main inspiration for this. And
 then you also have the same thing in the same kind of collective thing happens when through
 morphogenesis, like when we are grown basically from one cell by division and local communication,
 it's growing these like amazingly complex structures. And both processes show that by
 very simple rules, you can get amazing things. And there are many other examples. And one
 thing that these systems have in common is that you can remove parts and it still kind
 of works, which is very different to our current like neural networks where you change something
 slightly and oftentimes they will just break down.
 I think yeah, you demonstrate this later by training robots and then removing limbs of
 them and they can still kind of adjust to it. And I think the arch example of these
 local rules you have also in your blog post, which is this game of life, which is obviously,
 as you said, these are hand designed rules still give rise to like a really complex set
 of phenomenon, which is, I believe even like undecidable to really decide from a starting
 point. I'm not sure about the lore behind game of life.
 Yeah, exactly. I mean, they're basically you can build any I mean, with this, it's a universal
 computer, basically, you can build any kind of program if you that you would want with
 the cellular automata, of course, it would be like a super massive cellular automata.
 But they as you said, they show that even these kind of simple rules, they give rise
 to things that replicate things that move across the screen. And so people have found
 like all kinds of amazing structures by basically not changing the rules, but changing the starting
 configuration of these kind of cellular automata.
 When we think about combining this with deep learning, we quickly get to these neural what
 are called neural cellular automata. You have some examples right here. And I think I have
 the website open somewhere. This is work that appeared in distilled pub, which is obviously
 cool interactive journals. So this I think this was one of the first even articles to
 appear out of Google. And so this here, I can maybe interact with it, you can destroy
 parts of it, and it will kind of regrow it. And all of this is happening just by local
 interaction. So there is no, there's no kind of global organizing system that tells these
 things what to do. But every single pixel in here essentially has a feature vector and
 communicates with the neighbors. And how they communicate is am I correct to say that the
 way they communicate with each other, that is the part that is learned through deep learning.
 Exactly. Yeah, you can imagine like you have basically a copy of the same neural network
 like running in each cell. And that and that network takes into account like information
 from the neighbors, the neighbor state, and then it decides what should what should the
 next state of that pixel basically be. And you have these like RGB values, that's one
 thing it decides on. But then it also has these additional channels, like hidden channels
 that it can basically, it can decide what kind of information would be good to communicate
 to my neighbors. And so this work was not like the first that used neural networks to
 learn rules for cell automata, but it really kind of revived the field. And what it did
 is that it showed that you can actually you can make the whole system differentiable.
 So we tried similar things before where we used evolution to optimize neural networks,
 which is this field neuroevolution. But it's quite difficult for evolution if you have
 a specific target in mind, like you want to grow the salamander or you want to grow a
 certain other structure. It's quite hard for evolution to learn these kind of supervised
 tasks. And then basically this paper showed then if you have a target, you can just use
 recent tools like do auto diff, differentiate the whole system. And you can actually efficiently
 learn how to grow a certain structure that is only grown through these local communication
 of cells. And that was one of the that I think revived like the whole field. And there's
 a lot more papers now using neural networks for cell automata to grow all kinds of things,
 game levels, robots.
 How do you train such a thing? You said the full thing is differentiable and there is
 a target in this case, right? Is it the fact that you are in some starting state? Do you
 let it evolve for a couple of steps and then kind of measure the loss and then do something
 like back propagation through time?
 Yeah, exactly. Yeah. So you let it grow and then you measure like is it how close is it
 to the final output? And then it gives you the error to correct it. And then they do
 all kinds of tricks like that you want the system to be of course robust that if I let
 it grow for 50 steps, instead of like 20, I still want it to look like a salamander.
 So they do some kind of a few tricks that like doing it stochastically and letting grow
 for different amounts of time to get the system to be that it grows and it also kind of knows
 when to stop growing because that's an important part. Also nature like if through morphogenesis
 it grows an organ, it should know when to stop growing that organ and then like not
 grow forever. So that's one important ability of the systems is to learn kind of when to
 stop.
 If you were to let's say criticize this particular work, what would your criticism be? What's
 still missing from this? Or where is it weak?
 Yeah, so this what this showed is that it's basically it doesn't if you would critique
 it that you would you could say that it does not but that was also not the goal. It doesn't
 discover the structure itself. It has a target. So it has some kind of human design target
 like the salamander that is drawn by a human. And so in that case, that's one limitation.
 So actually one follow up work that we will be published soon, we actually combined evolution
 and this system where evolution we let evolution come up with like these soft robot in that
 case. And evolution is good at discovering like variety of different morphologies. And
 then we use basically this method to make the structure very robust. So we let evolution
 discover the structure and then we cut off all kinds of limbs and let it regrow. So combining
 kind of the creativity of evolution with this kind of making things robust through this
 gradient descent based training.
 That is the yeah, the work on soft robots. I've seen that it just looks really cool.
 So this would be one thing that is that is discovered this sort of kind of hopping tripod.
 And obviously this, I think soft robotics in general are rather new field and combining
 them up with like evolving system seems quite appropriate. So here's one with a cut off
 limb and you can learn to regrow, regrow it. Right? How in general, how do you teach a
 self organizing system to regrow things? Do you have to explicitly program? Like you have
 to explicitly train it to regrow things? Or is this just a natural consequence out of
 how the system was trained in the first place?
 Yeah, so sometimes it can often it already has some inherent robustness, but it will
 without explicit training, it will probably not be able to do this like perfectly. And
 it will be that it sometimes works and sometimes doesn't. So in these cases, we explicitly
 and also in the case of the work by Google, like they explicitly like you explicitly remove
 stuff during the training process so that you confront the system with, you know, this
 kind of this damage that it has to recover from. So it makes the system more robust if
 you specifically train for it. And I guess in nature, that's probably one reason that
 the system had to work for all these different environments. So there is a lot of like they
 weren't in your aunt colonies, sometimes you had more, sometimes you had less and so these
 systems are because of the way they were, they are evolved. They also show this kind
 of similar level of like superior level of robustness.
 At this point, are we already at the point where you would say that this surpasses or
 this is very advantageous to classical deep learning? Or are we still in the realm where,
 let's say, everything would be fairly possible with classic supervised top down deep learning?
 I think like this, it, it would be possible to have it grow and recover. But I think that
 the secret here is that it only uses local communication. Basically, you could of course
 have a network that would, I don't know, a network that you query that would could like
 similarly to earlier work like compositional pattern producing networks, CPPNs, where you
 query basically each location in space and you ask it what should the voxel be? And of
 course, these systems could then if there's damage, they could you could ask them again
 and they could recover. But the trick here is, is that it's only based on local communication.
 So if we ever want these things to work in the real world, then it's really advantageous
 to have things that only require local communication, basically. And so that's one whole that's
 one goal is that ultimately, we want to take those systems from also the simulation later
 on and you know, we have some initial work and we want to really create complex things
 also in the in the physical world.
 If you say in the in the physical world, because if I if I think of there was, oh, no, this
 was a this, the paper, the physical cell either automata is at least a thing that is doable
 in the in the real world. But if I think of something like, I don't know, a Tesla car
 or something like this, that is in the real world. Yet, it is still, you know, a central
 controller that controls the whole car, and there are still top down and so on. And it's
 also trained in that way. What are the types of physical situations where these would really
 the local communication would really come in handy?
 Yeah, like I could imagine like, let's say you have a building or something that could
 automatically detect if it's damaged, and then you know, it could like our you know,
 our skin, it, you know, it's damaged, and it's it's regrowing, it's, it's self, self
 healing. So you could ultimately, I mean, this is like science fiction, but imagine
 a building and then you it gets damaged, and then automatically it recognizes it's damaged.
 And then it, you know, automatically recovers from this damage. More other like science
 sci fi is if you have, imagine you have a swarm of nanobots, they only can communicate
 locally, right, but they have to figure out their shape, they have to figure out their
 what they can do in an environment. So in those situations, this local communication
 would be very advantageous. I don't know if it would necessarily be useful for these kind
 of, you know, Tesla, this car example. But but I could imagine a lot of other like application
 areas or drones that have to coordinate somehow together only being able to sense each other
 locally. So more these kind of in that areas. One thing I'm quite excited about is this
 getting this from like this 2d version to a 3d version. And then you can imagine building
 all kinds of things and it would automatically know you're building, you know, a table or
 you're building a chair or you're building this and this, which which I think it's quite
 so. So this is one example also of so yeah, the self classifying MNIST digits, where basically
 the system cannot only be used to grow something, but it can also be used to self infer its
 own shape. So you build something out of small components, or you draw like a digit. And
 then by having the cells communicate with each other, they figure out, oh, I'm part
 of an eight or I'm part of a one. And so basically, this is then what we replicated in this physical
 where you can put them together, make digits, and then each each of these cells will tell
 would figure out what part what shape am I part of.
 So this, this is a physical instantiation of the demo I have here online. This is another
 distal article where as you exactly said, these things, they figure out themselves what
 they're part of. And you made you made this this is your paper into a physical instantiation,
 which I find really cool. And now you're taking it to 3d.
 Yeah, yeah, that's the that's the plan. Yeah. And of course, currently, these systems, like
 this kind of self classifying MNIST digits, it does not work as well as like using like,
 like state of the art, deep convolutional neural network or transformer or what what
 you have. But I think ultimately, these systems, maybe we can integrate some ideas also for
 things like object detection to make these systems kind of more robust by having a more
 kind of distributed object detection where you have this system where the components,
 maybe it could be a combination of something convolutional and but then you have the system
 on top, where you have this local communication and they figure out together kind of what
 shape am I looking at and maybe that could make these systems also more robust in the
 future. And maybe less prone to kind of this adversarial attacks that we currently see
 the system still exhibit.
 Has anyone tried with like, maybe this would be interesting, like to take something like
 this, and try to like make an adversarial, I don't even know how that would look like,
 but something that a human would clearly classify as like a seven, but there's like a slight
 twist.
 Yeah, yeah, I'm not sure people have actually studied it so much on this, trying to see
 how what kind of adversarial attacks these systems could, I mean, fool like you could
 fool them. I'm sure there are also some. But maybe the combination of kind of both this
 and more classic deep image recognition techniques could make them more robust.
 So you've taken also this idea of this 2D cellular automata, and you've applied this
 in 3D here in Minecraft, which so this is a morphogenesis. How do you how would you
 define morphogenesis just quickly?
 Yeah, I would define morphogenesis as like growing a complex structure based also on
 this kind of local communication. So how our like bodies are grown is morphogenesis, how
 our like organs are grown, how our nervous systems is grown basically, from like, you
 know, a single starting cell. And so this is what we do here. And again, the structures
 are not found by the system itself, like we took like an existing apartment building.
 And then we trained the system in the same supervised way to regrow it basically. And
 we were surprised that it could also grow like these kind of functional machines, we
 actually had it growing like, like this temple. And then we found that the trap in this temple
 still worked. So because it had all the components, like there was not one single mistake. And
 that allowed these kind of functional things to still to still work like this kind of like
 caterpillar you see there.
 And can you can you you also said you can destroy part of it and it will regrow, right,
 which Yeah, is this have you made this playable somewhere in Minecraft itself? Or is this
 just purely your
 Yeah, it's currently it's it's not I mean, you can download the code and stuff. But it's
 not that we have a server where you can play with those things. But it would be very interesting.
 We actually we organized this Minecraft open endedness competition where we like a related
 field like can you have an algorithm that can like natural evolution create all kinds
 of novel things without limits. And that's also where we use this Minecraft framework.
 But it would be real fun. Like one thing that I that I want to try to also pursue in the
 future. Imagine you don't have it grow like caterpillars, but you have it grow like cities.
 And then depending on the environment that you as the human does decide, like the mountains
 or like the desert, it would grow a different type of city. So like, that's one thing we're
 looking at now, how can you incorporate also feedback back into the album, because this
 caterpillar will always grow the same caterpillar. But if if I put this caterpillar in a in a
 small box, it should maybe grow a small caterpillar. And if it's a large box, it should grow a
 large caterpillar. So how can you kind of incorporate this environmental feedback? That's
 another thing that I'm very curious about.
 Yeah, is do you see beyond beyond gaming, maybe which which I can definitely see applications
 of this? Do you see applications that are not in the physical world as we talked before,
 but but maybe in the in the still in the realm of the digital world? Are there applications?
 I don't know what what what all you you're thinking of, but distributed applications,
 networking applications, any sort of things that you're very excited about that maybe
 aren't super obvious if you just see the the Minecraft example.
 Right. I mean, one thing that we are basically I think like two things. One is like just
 this Minecraft, I think, could also ultimately teach us something about biology itself. So
 if we could because we don't know everything yet about how this exact morphogenesis process
 works in nature. I mean, we know a lot of things, but we don't know, for example, how
 is it so accurate? Like and and and so there are certain things that we are we don't know
 yet. And so by simulating these process like a very simplified model, but maybe there's
 things we can learn from these kind of very simple models. So that's one one area I'm
 also very excited about. And so taking these systems to as a as a very simplified models
 biology to learn something. The other thing, the other application area is what I'm excited
 about is using those things. But instead of growing Minecraft structures, you can grow
 actually artificial neural networks. So so so you're basically kind of replicating our
 brains are not like designed and fixed, they're grown like through this developmental process.
 So what what we did with this recent work is hyper NCA is taken basically, instead of
 having growing a caterpillar, we grow a pattern. And then we then we with a neural cell automata,
 and then we convert that pattern into a policy network. And that policy network then is we
 can use this for our RL task, for example. So that's one one area I'm very excited about
 and making this systems more, more performant, because currently we apply to quite simple
 problems. But I think ultimately, this kind of idea of this growing neural networks is
 can be very powerful, because that's how you know, our brains are created. So so we're
 trying to replicate that process, hoping to create also more, more adaptive, basically
 neural networks. What do I gain out of so in this here, I have these developmental steps
 on the left, I do essentially start with some configuration of weights, essentially. And
 then I let the cellular automata run for a number of steps self organizing here, then
 I take it into a network, and then I execute the network. And presumably, I have to learn
 this somehow. In this paper, what you are doing is you're using, if I recall correctly,
 a variant of evolutionary search, right? I could also, like, I know, in whatever way
 I learn it, I somehow have to learn how the cellular automata here reacts. What do I gain
 out of this instead of just training my policy net? Right. So so far, I would say it's the
 you don't get so much directly. So so far, this method, it's not that they outperform
 like current deep RL methods. But ultimately, basically, there is this this hypothesis,
 also popularized more recently by Tony's adore this kind of genomic bottleneck hypothesis
 that means that we only have, you know, 20,000 genes, and they, they, they guide the growth
 and self organization of our brains with trillions of connections. And and and so it's a much
 smaller genotype that encodes a much larger structure. And so this kind of compression
 is hypothesized to also allows us to and animals to deal with situation they haven't seen,
 like to basically that the robustness that animals show is part because they have to
 go through this bottleneck this compression. And this is the information you give to the
 next generation. So there's some limit on the information you can get. So that might
 bias the system towards learning rules that generalize well, like learning rules that
 generalize well. And so this is the the hypothesis here, that at some point, we can have a very
 small neural cell automata, which is basically like the genome and that encodes a much larger
 network and that hopefully would then be more robust. But that's something we have. That's
 basically what we're working on, which we which we haven't really shown yet. But that's
 the that's the hypothesis and the hope. One other thing that's kind of funny that it can
 do like it can you can basically let the growth continue and not just have one network grown,
 but multiple networks. So like we applied this to this quadruped domain. So we had it
 grow for for 10 steps to grow one brain like one network, then we put it into this quadruped.
 And we have a slightly larger quadruped. So we let it grow for longer, and then put it
 in the middle quadruped and then have a larger one. So and so basically one NCA can grow
 multiple different neural networks. And that's also one thing that I'm pretty excited about
 that we want to apply also for like more complex domains.
 And again, here you had an experiment with with where you damaged these quarter pads,
 the system is able to adjust, can you explain how this system is able to adjust to a damaged
 morphology, like a cut off a limb or something?
 So here it was basically trained to on these, like on all these different morphologies.
 And then we had it basically, by continuing the growth, you can get a controller that
 was trained for one morphology, and then you continue it and you get a controller that
 works for M2 and you let it grow a little longer and it has a morphology for M3. So
 in this case, those were basically seen during some other experiments, we have results where
 it has damage that was not seen during training here, basically was trained to being able
 to deal with this particular type. So if we would damage it in another way, it probably
 wouldn't work anymore with these metamorphosis networks. But yeah, so the hope is also that
 if you know how to control one quadruped, then there should be that you don't have
 to start basically from scratch, there should be some information there that allows you
 to also grow something that is related, and not having to start like all over again, basically.
 This flows, I think, into a lot of a lot of ideas from, as you said, the open ended community
 and the sort of don't have explicit goals community. I think parts of your blog posts
 and papers mentioned algorithms like quality, diversity, map elites, and things like this,
 which are obviously very exciting and very different from how we do deep learning today.
 So far, we've always looked at things that have either an explicit goal, like here is
 the salamander I want to build, or here is the Minecraft structure I want to build, or
 have some sort of, I want to say, goal in an in a more abstract sense, like the reinforcement
 learning goal of maximizing the height in this case, right for these robots that stand
 on top of one another. Yet, how do we go away from this? Is there is there a natural progression
 in these self organizing systems to go away from having explicit goals that would be more
 difficult to pursue with like the classic deep learning systems?
 Right, I think in general, so I think that, like two things like one is the representation,
 which I think these neural cell automata are like a great representation for a lot of like
 growing structures growing neural networks. And then the other thing is you mentioned
 is like the search, how do we actually get to systems that show interesting, these interesting
 properties. And so there seems to be a recent trend, I mean, not just in the self organizing
 systems, but in also in deep RL in general, to not train on one thing basically, but train
 on a variety of different things. So there was also this more recent paper by I think
 it was DeepMind where they this XLL that they showed like basically, if you train agents
 in a lot of different changing environments, they develop more robust skills basically.
 So I think basically here it's we also what I think it makes these self organizing systems
 quite difficult to train is that these landscapes, the fitness landscapes basically, they are
 probably very kind of not very smooth, because changing like something small in the self
 organizing systems can have like this cascading effect. So that's why these traditional objective
 based rewards, they work, but then they don't, it's still difficult to optimize. So that's
 why we're more looking into this kind of open ended, like what you mentioned quality diversity
 methods basically, where we're not trying to optimize for one particular outcome. But
 we're trying to find things that differ in some interesting ways basically. And I think
 those methods, particularly for this kind of self organization, they are very, very
 powerful basically. They are better at navigating like these kind of very complex landscapes
 with many local optima, but they're also slightly more expensive because they're looking at
 the larger space of this of the search space basically.
 What maybe these two questions in one given given these outlooks, what field that deep
 learning is good at right now? Do you expect these methods to be better? If you know, let's
 say if we invest the resources and figure out, you know, the tricks of the trade enough,
 what parts that deep learning is good at now? Could these methods overtake deep learning?
 And then on the other hand, what's kind of the, for you, the most exciting area that
 we haven't even unlocked yet with deep learning that are accessible with this? Right? So it's
 two different, two different things, but I'm wondering about what you think about both
 of these directions.
 Right. So I think it's also, I wouldn't say like overtake deep learning. I mean, we use
 basically we use deep learning as a tool for basically like kind of train the system. So
 I think,
 Yeah, sorry. I mean, deep learning and like the, just the thing we do right now, right?
 We have objective loss, supervised training, single neural network.
 So I would assume that these systems would be able to have a lot of different domains.
 I think the one kind of probably the closest, I think what we would see is that they would
 make our L agents more, you know, like more robust, more adaptive. And that's also already
 in this work that you that we have there is like where we have basically in this case,
 we trained not only the, we have completely random weights and we only trained local update
 rules, basically the habit rules. And then we show that through this system, we can actually
 during the lifetime cut off a leg. Again, we are always somehow mutilating these robots.
 We're not very nice to them. But basically, this is an example, I think, where we already
 show that is this is more adaptive than the current RL design. So in the current basically
 deep RL, I think the one main drawback is that we train a system and then we freeze
 the neural network and then let it do its tasks. And this seems like kind of very unnatural
 that like you have a frozen brain. Okay, maybe you have like some recurrent connection that
 allow you to learn something. But basically, we have this training period, then we freeze
 everything in the system and we apply it to domains. So that's not like lifetime learning
 in normally these systems. But the idea here is, in general self-organization, that we
 never wanted to stop learning, we never wanted to stop adapting, we want the self-organizing
 process to happening the whole time. So I think in any domain, where there are things
 that you might not have anticipated during test time, these systems could be beneficial.
 Like might it be there's a pixel edit, you're losing a leg or you wanted to do something
 else. I think that they already show that there's some, they can be superior in those
 domains. And that's one thing that I'm pretty excited about to apply them to more complicated
 domains, not just these like quadruped locomotion tasks, basically. But anything where you have
 something unanticipated happening, I think there will be can be a benefit of it. And
 then was the second question like what other
 a new area that we haven't even like we have no chance currently of tackling with our tools?
 Yeah, that's a great question. I mean, I think this new area is this kind of rapid lifetime
 adaptation basically, I think these systems are great for if you know what you would expect.
 But things like basically like having things that work in unknown environments, I think
 that's a really, I think exciting area that I mean, you have like animals in nature and
 you can put a dog into a new environment and will not completely like break down and will
 still know kind of what to do and to interact with the environment. And we don't have that
 yet for our agents, like we can put them in environments they're trained for, you put
 them too far out, they don't know what to do. So and I think that too, that's so this
 working in other environments and also having this kind of like, you know, common sense,
 I think is maybe also an area I think in the future that these systems could be applied
 to although I don't know exactly how but but that these systems have more common sense
 and don't directly break down like kind of giving them this kind of innate abilities
 that we humans are born with animals are some animals are born with that allows them to
 yeah, do a little bit more common sense things than than current deep learning system that
 don't have that property basically.
 And this, I think you say it even here at some point. This, in addition to the fact
 that there is this genomic bottleneck, right, you already said this, the genes encode or
 only have the capacity to encode very little information. And what we're doing here is
 we're learning essentially the rules to learn the rules, which can be compressed in a much
 better way than the rules themselves. And there is a reason to assume that this will
 result in that kind of common sense that if you have to essentially learn the meta rule,
 then that will make you generalize better. I mean, it's an it's an argument, I'm not
 super convinced yet. Right. But if you do then some parameter sharing, you showed in
 some experiments, you can compress this even further. So that might be a way to tackle
 that.
 And also this in Tony's adores paper, he actually he points out that this bottleneck, like there's
 some organism nature that have many more genes, for example. So maybe it is a feature that
 we have that number of genes that it's compressed. And so so that gives us like some hope that
 also having the similar feature in our artificial systems should be beneficial. But but we're
 still we only showed that for very, very simple, you know, simple tasks so far.
 And deep learning goes into the exact opposite directions, right? We're like the more the
 more parameters, the better we have the double descent phenomenon, and we can go essentially
 infinite and it always gets better, which is which is weird, right? Which is also giving
 amazing results, I think recently with you know, the whole language models and so on.
 So it's definitely it could it would be cool if in the near future, people discover like
 a fundamental connection between, you know, the the good results we get by scaling up,
 and the the actual principle from biology, which is seems to be more like compressing
 and scaling down, it would be nice if those were to join together somehow.
 And hopefully, we can be part of that in some extent. But yeah, I agree. It's really interesting
 that like that you Yeah, you scale up networks, and then your local optima disappear, like
 everything just works better. And here we basically we want to go the opposite direction.
 But it's not necessarily that we, of course, we still want our the final models to have
 trillions of of of like connections. But we what we basically want is we want the trainable
 parameters to be low. And I think that that's the fundamental difference that we have a
 small number of train or relatively small number of trainable parameters there, but
 they give rise to much more complicated system, exploiting things like self organization growth
 over time. And, yeah.
 This is I think, because you said before, you're not you're not an opponent of deep
 learning. In fact, deep learning is used inside of the cellular automata to to sort of learn
 these rules. I find it interesting, if you look in nature, that there are cells and they
 self organize in some way, right, by whatever means that is learned. But these cells then
 make up brains, right? And brains are naturally very top down planners. They're they're, they're
 in the moment, they, you know, look ahead. And then the brain somehow organizing to societies
 and the societies again, are very distributed, very local, very interaction on a person to
 person level. What do you what do you make of this? Do you think there is like an optimal
 switch from local to global to local to global that we could sort of stack on top of one
 another? Or is this just a happenstance of of the universe?
 Yeah, that's a Yeah, that's a that's a great question.
 And even more like the humans in the societies, they organize themselves into hierarchies,
 right? Top down control and somehow it gets even
 crazy. It's a good question. Do we need one? Yeah,
 do we need all of this in our artificial systems? Maybe we need all of this to get to real like
 more general artificial intelligence. Like because also one thing that is really crucial
 is the our culture, right? Like, like, if you if you I was reading this great book recently,
 like if you just put humans somewhere by themselves, they're not very like, you know, good at surviving,
 but we are good at surviving because we have all this cultural information, like all this
 knowledge that other people made that that we can build on. And that allows us to do
 all these amazing things. So maybe to get our eyes to do really amazing things, it's
 not enough to having like single agents in complex environments, but it needs to be multiple
 agents that need to be simulated maybe over multiple generations. So there can be some
 cultural knowledge transferred from some agents to other agents, similarly to how how it happens
 in for us. But of course, that also makes the simulations much more complex and expensive.
 When you have to simulate cultures, multiple like generations, and then we need some more
 better compute, especially at the university level.
 I think yeah, that's one advantage that nature has it has lots of lots of distributed compute
 available. That said that there is there is an interesting part in your blog post where
 you describe sort of how to train these things, or how to steer the development of these swarm
 systems or distributed systems. One one quote here you have is guiding a swarm system can
 only be done as a shepherd would drive a herd by applying force at crucial leverage points
 by subverting the natural tendencies of the system. And then another one is the self assembling
 brain knows no shortcuts in which your I believe your argument was a little bit that is very
 hard to predict what a change does until you observe it because the interactions can be
 kind of nonlinear, very dynamic, very, very hard to predict.
 In essence, that was basically the argument that that hissing are made in his this great
 book like self organizing, no self assembling brain. And basically that you need to basically
 the system needs this process of growth. And you have to put energy into it to observe
 the outcome you cannot predict. And that's also things they showed that Wolfram what
 he showed with simple one diesel automata, you cannot predict the state of the system,
 you have to actually run the system even if it's a simple one diesel automata. And that
 is also apparently the question is, do we also need to do that for to growing our neural
 networks instead of like designing them? Maybe we need to go through this kind of process
 of growth with learned rules to to really unlock you know what these systems can do.
 There is recent work in using for example, GANs or so to predict things like fluid dynamics
 and you know, they can't do it like super, like they're not extremely accurate, but they
 can give a pretty good estimate of given starting state and then a highly dynamic nonlinear
 system. And then they can predict some steps into the future, I've seen the same like galaxy
 development and so on. Do is there any happening like this where you can say, Well, I don't
 I can't, I don't have enough compute to run all these swarms, but I can sort of train a
 surrogate model that will give me the end in sort of a one step fashion. And then these
 the forces that I poke at the swarm at I could determine those using the surrogate model.
 Yeah, I think that that would be really interesting. I wonder I think it's, it could work for some
 limited steps in the future. But but but I think you you would still need to, you know,
 like, like at some point, you need to basically run this this model. I mean, maybe in the
 first like generations, you could help have so great model that somehow helps you to sort
 out like the things that are really bad, like, this will not grow into anything. So I think
 you could use it there later, I guess you would probably have to run the system like
 when things get more complex. But I but I think there's also another role for the surrogate
 models, which something I always wanted to try to predict basically the learning abilities
 of the system. So you have an agent in an environment. So maybe you don't need to simulate
 the whole lifetime, right? But you can have some more like some kind of some tests that
 would test is this agent, how capable is this agent, so having some kind of surrogate that
 would could look at certain parts of I don't know the neural network and already predict,
 will this be a good learner or not basically. But yeah,
 it is in the in one part you also it has very, can very remember like I got into machine
 learning and graphical models were the hot thing at that point, it was just before deep
 learning. And this reminds me all this self organizing systems with the local communication,
 they remind me a lot of belief propagation, things like this graph neural networks, obviously
 are right now up and coming, let's say, do you see connections between all of those things?
 Or is that just kind of a superficial connection? Yeah, I definitely see there's a big connection
 to these also these graph neural networks, basically, like, I mean, they're very close
 to like a more generalized form basically of like a cell automata, where you have different
 basically neighborhoods, depending on your the topology of the graph. And they also seem
 to be there. I think they're super interesting. I also actually how I got into neural networks
 is the the first lecture I had as an undergrad was actually on neural networks and about
 these self organizing maps, which these coho and self organizing maps that basically can
 do clustering based on this somehow like kind of like kimmins, but on a on a much more,
 they can do it better. And you have to get these like nice visualizations out of them.
 And apparently, there's also some pros in our brain. I mean, we have these topographic
 maps also in our brains. I was always fascinated somehow by these self organizing maps. And
 even though I did a lot of like some other things during my PhD, somehow now I'm coming
 back to this kind of self organization. And and and yeah, using these recently learning
 tools, it's I think we can really unlock like the power behind them. There was a Do you
 know the arc challenge? The abstract reasoning corpus by Francois? Yeah, yeah, yeah. There
 is I'm not sure if they have an example right here. So for everyone who doesn't know this,
 this is a task where you get so the left ones are demonstration examples, there's always
 like an input grid and an output grid. And then you get a test example where you only
 get the input. So here, the rule is I've looked at that before. So the rule is kind of there
 is the gray in the middle, and you kind of fold the right hand side onto the left hand
 side and then you the solution here on the right hand side is kind of the the sum of
 the two. And this is these are things that humans are surprisingly good at, but are very
 difficult for a machine to learn. And the this is a data set and the training examples,
 there are not many training examples. So there is not really a way to to learn this through
 brute force training. There is a little game that people can play, I think I've reported
 on this before, but there is a game for anyone who's interested, where this is the arc game,
 you can find it on the GitHub page on of of Alexei Borski. And you can just choose one
 here, they're divided into different levels. And yeah, you can you can try them for yourself.
 So this, this looks even familiar, like cellular automata. Do you think that it like self organizing
 systems in one way or another in the way we've looked at them today, or in the way you've
 seen them could be useful in solving challenges like these, because challenges like these
 are related very much to, let's say, something that we would call intelligence. Yeah, I think
 the the, the hope would be that if we can get this kind of bottleneck algorithms to
 work where we exploit, so I'm not sure it like we could apply like self organization
 directly. But what I could imagine is that we exploit develop these kind of genomic bottleneck
 algorithms that can guide this self organization growth of a very complex neural network and
 that that network then could maybe be used for these kind of tasks. And the hope would
 be that because it has this compression, it would maybe develop an algorithm that would
 allow it to, you know, solve these kind of tasks that require more like high level cognitive
 skills. But but of course, that's still Yeah, we're still a little far away from that, I
 think. And I guess I don't know what the current state of the art and in this task is. How?
 I think it's, I think it's still largely unsolved. So this could be a great test domain, I think.
 But yeah, I think I, I'm not sure I have high hopes that it would already like, I think
 we still probably missing some other ingredients that we don't have yet to kind of make progress
 there.
 Yeah, but by the way, this, I think I just clicked on on one randomly. But I think here,
 the rule as I think if people get it, they can see that you always kind of select the
 smallest of the shapes that is there and kind of replicate it. You know, at least that's
 my that's my hypothesis, right? Yeah, maybe, maybe.
 Oh, I think maybe you take the one that fits in the box.
 Oh, yeah, yeah, yeah. Right. But it's like this, this, this kind of, like, you need to
 understand what shapes are and so on. So that is very much that this is very high level.
 This is very bottlenecky. It has a bottlenecky feel to it. Like, you're probably not going
 to get very far with like a CNN trained on these pixels directly. So that's, that's like
 I can see something like this very much be in the domain of of like, first open endedness,
 but then also self organizing things made up like simple rules making up something very
 complicated.
 There's two other domains that I think also very exciting, like one is this animal AI
 benchmark, where basically they it's like an animal AI Olympics where you apply eyes
 to tasks that animals normally are good at, like, and like, for example, trying to figure
 out which one is the tool and then you use that tool to, you know, get a reward. And
 so there's also where current methods basically, they've pretty much fail on more complicated
 tasks. And then they also had a mid-term experiments where they had children perform these tasks
 and they are still much better at than than like any of our deep RL methods. So in the
 simple task, deeper RL performs pretty well. Once it gets to more complicated things, then
 they the system basically, these systems fail. So this is one task that like, in the recent
 grant proposal that I proposed that that there would be a good test domain for these methods
 basically, because the whole point is to act in an environment that you haven't seen during
 training. Even though the environment is made out of the same building blocks, like there's
 rewards, there's like barriers, but how they are composed, all of this is new, basically,
 and never seen before. And the other one is this also by I think was the mind is alchemy
 task where you have to learn to kind of it's a task that we have to learn basically about
 the structure of the domain, what things you can put together, and then you have to use
 that knowledge to like building on that knowledge basically. And this is also a very difficult
 task for all of our current methods. So I think this could also be very good task to
 basically as the North Star to drive these the progress in this kind of area. And the
 hope is that these kind of self organizing system, they should be, hopefully would be
 better at in this
 where can people if someone wants to get started in diving into the world of self organizing
 systems, swarm intelligence, maybe a bit of open endedness, is there a good place for
 people to get started like get their their feet?
 Yeah, I would say I was recently rereading this great book from Melanie Mitchell, this
 complexity. I think this is a great starting book on on kind of this ideas of complex system
 self organization. There's something about cellular automata in there. So I think this
 is a this is a good kind of good point to get a broader overview of of that kind of
 whole field of basically complex system self organization. And yeah, and hopefully the
 also the the blog post hopefully can be helpful to some people and also plan to write more
 on on that as well. But but this I would suggest this is a this is definitely a good place
 to start.
 And is there some some, you know, in, in deep learning, it's usually Keras, I train a CNN
 on MNIST or CIFAR 10. Is there like some some standard thing that every one of your of your
 students goes through?
 I mean, now I sent a lot of them to this great distill article basically and looking at this
 this growing NCAs because they also have a great, like this collab notebook where you
 can play with the system. So I think this is a great starting point to where you both
 have neural like you have cellular automata and you have like how recent tools can be
 used to grow them. So I think this is a good good place to play around with basically.
 Okay. Yeah, I've I've spent more than more than more time than I've had on these things
 because they're quite
 It's great that it's also so interactive and fun to play with.
 Yes, definitely. Yeah, I think is there anything else that you would like to get out there
 to people about this field?
 Yeah, I just Yeah, I hope that people would be not only everybody running basically in
 the same direction just doing like what everybody else is doing. So hopefully this will be also
 get a few more people into this field of complex systems and self organizing systems and combining
 the ideas of deep learning. Because I think there's a lot of things interesting things
 to discover basically here and a little bit less people working on it then then the heart
 like like working on foundation models and language models and all those other things.
 Yeah, it's certainly I think I think is certainly an interesting area. And I guess especially
 if you're at a university without the super duper clusters. Probably just strategically
 a PhD in this field would maybe be more of a advantageous position for new newcomers
 to the field.
 Actually, like Hinton had this great quote recently on this other podcast, like it's
 always a good idea to figure out what huge numbers of very smart people are working on
 and to work on something else. Because you don't want to do maybe what what everybody
 else is doing. And I think so I would suggest this is a great field where a lot of I think
 interesting discoveries basically waiting to happen.
 I agree. All right. So Sebastian, thank you very much for being here today. This was very
 cool. I hope to see yeah I hope to see a sprawling future for your field. Thanks a lot for the
 invite. Thanks.