Reinforcement Learning, Fast and Slow
machine learning reinforcement learning meta-learning deep rl deep reinforcement learning deep neural network atari alphago deepmind google td-gammon episodic memory inductive bias bias variance tradeoff
Abstract:
Deep reinforcement learning (RL) methods have driven impressive advances in artificial intelligence in recent years, exceeding human performance in domains ranging from Atari to Go to no-limit poker. This progress has drawn the attention of cognitive scientists interested in understanding human learning. However, the concern has been raised that deep RL may be too sample-inefficient – that is, it may simply be too slow – to provide a plausible model of how humans learn. In the present review, we counter this critique by describing recently developed techniques that allow deep RL to operate more nimbly, solving problems much more quickly than previous methods. Although these techniques were developed in an AI context, we propose that they may have rich implications for psychology and neuroscience. A key insight, arising from these AI methods, concerns the fundamental connection between fast RL and slower, more incremental forms of learning.

Authors: Matthew Botvinick, Sam Ritter, Jane X. Wang, Zeb Kurth-Nelson, Charles Blundell, Demis Hassabis

https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(19)30061-0
 Hi there, today we're looking at reinforcement learning, fast and slow, by Matthew Botvinick, Sam Ritter, Jane X. Wang, Zeb Kurt-Nielsen, Charles Spondel and Demis Hassabis. These people are from Google DeepMind and this is a review of kind of a development in reinforcement learning, especially as it pertains to kind of how humans learn or what we can understand from the RL world that translates over to human learning. Alright, so basically their argument here is that the first wave of deep RL, as you see here, is powerful but slow. And they give examples of this. So in box one, box one is this. So they believe there's an image missing here. This is Backgammon, TD Gammon. This is the famous DeepMind Atari playing bot and this is kind of the 3D labyrinth playing bot. So there's been a number of advances in RL and especially what they talk about is deep RL. So when we talk about reinforcement learning, the easiest case is where you have an agent and an environment. Alright, so the agent will observe some observation O from the environment and then based on that the agent will perform an action A. And then the environment will give back a reward and also a next observation. So this is O0, O1, and then this is A0 and then here you give A1, AI. So basically this goes back and forth and back and forth. The agent performs an action, the environment gives a reward and the next observation. So this could be for example here in the Atari world. The observation is the screen itself. And then the agent needs to perform an action which is an input of the joystick or pressing some button. You can see the individual actions actually listed here. And then the reward will be given to the agent via a number which I guess is the same number as up here. So the task is to maximize the reward simply by... So the difference is you're not doing this in a supervised manner. So you're not telling the agent what would be the correct action to do. You simply tell it that whether what it did was good or bad by giving it a high or a low reward. Right, so that's reinforcement learning. So what is deep reinforcement learning? Deep reinforcement learning simply means the agent maps the observation to the action via a deep neural network. So deep neural network. That's deep reinforcement learning where the mapping or some part of the agent consists of a deep neural network. You see for example here there is a deep neural network mapping the observation to the action. As well as down here but it's a bit more complicated. So they argue that the first wave of this was powerful but slow meaning kind of you need a lot of samples. And they give two sources of why it's slow, why you need a lot of samples. They say the two factors are incremental parameter adjustment and weak inductive bias. So incremental parameter adjustment means basically that you have to update or train your neural network in a very small incremental way. In order to basically, because you train it one by one, right? You train your neural network step by step. You have to make small steps in order to not forget what came before. You can't fundamentally readjust your neural network to every new batch of observations because then that's going to destroy all the information you've learned of the old one. And then weak inductive bias here is basically an understanding of these neural networks. They are general function approximators and they can approximate any function. So if you just think in terms of kind of, I don't know, let's say polynomials and what kind of polynomials are there? This polynomial, this polynomial, this polynomial, this weird polynomial. If I have a function that can approximate all of these then I have a weak inductive bias whereas if I kind of know, okay all my polynomials are the polynomial that I'm looking for ultimately, I'm very sure it's a third degree polynomial, right? So something like this or like this or like this. So this is much less of a class of functions that I can fit but if I'm sure that the function that I'm trying to fit falls in this category then I'm much faster. So this is then called a strong inductive bias is where I build into the model basically I tell it beforehand. Here is a very restricted class of functions that you can fit. Whereas in a weak inductive bias I won't tell it that. I'll simply say, well model you could fit any function you want and I'm just giving you training samples. So this is a classic example of a bias variance trade-off where there is a lot of variance in these models meaning you can fit also a lot of functions but here because you bias the model towards a certain set of functions it can lower this variance and in this case here it speeds up learning because you don't have as much variance that means you can basically go faster while learning. Alright, so they propose two solutions to this problem of this kind of to mitigate these problems that make reinforcement learning faster or have made reinforcement learning faster. This is a review remember. So the first one is episodic deep reinforcement learning and this episodic deep reinforcement learning is specified here, fast learning through episodic memory. So the suggestion in this field of research is to augment the neural network or the agent by a memory and the memory could look something like this. So in a lot of these RL frameworks what a principal component of the agent is, so the agent will get an observation O and one of the things it has to do is estimate the value of this observation of this state. So basically the agent is in some state let's say you play pong right and you are here down and the ball comes your way up there right there's a little arrow sorry so the ball flies away from you and you're all the way down which basically means draw this bigger. So here you are down here and the ball is here flying up there. So one task in these in these agents that occurs often is to estimate the value of this observation basically means how much reward am I expecting from this state going into the future. In this case I probably will not expect a lot of reward since I can't move up fast enough right to catch the ball. So this I would assign this state a pretty low value whereas if I were up here I would assign this state quite a high value. So as we've already seen this is a deep neural network mapping we learn to assign value to different states and this is one of the parts that takes a long time and these methods they are the one that's depicted here replaces this value estimation by saying okay we have an observation we somehow need to estimate its value why don't we look for similar observation so we have some kind of memory right and we go with our observation and we retrieve O'1 O'2 O'3 that are somehow similar right so in our in our pong example I'm down I'm up here ball moves here I could be looking now at at states where I was here or where I was here like really close or where the ball flew a bit differently but still in the same direction or down here right so all these states are kind of close to my state and I can I already have I already have played these since they're in my memory right so with every one of them I can also retrieve the reward that I got so I because I already know the problem in reinforcement learning is before you do an action you don't know what the reward will be but here I already know because I've played it I've already experienced it it's in the past so I know what reward I got right so and this is exactly what they say over here they basically say here we have time time runs this way we're in state one then in state two and so on and we perform actions and and get rewards and what we can do is we can save these states into this memory as along with their sum of discounted rewards that we collect from that state on and then later this is like a spongebob reference if we want to estimate the value of some new state right what we do is we retrieve all of these states from memory calculate a similarity score over them and with with we wait basically we add their rewards weighted by how similar they are to the state that we want to compute so this basically amounts to averaging over states respective by how close they are to the current state right this is kind of a soft a soft way of saying I only select the states which are close and that gives you a value estimate for the new states so basically this means you just got rid of having to train a value function and this will speed up your reinforcement learning quite a bit if you don't have to train that if you already have good value estimations from your previous experience that's great of course there are a number of problems associated with that namely if this memory here for example becomes stale it doesn't represent the future rewards quite as well there is also a question of which states do you keep in memory just the good ones or do they have to have a certain property do you have to have some diversity in there and of course the biggest problem here the biggest problem is how do you know when two states are similar or when they aren't it might be easy in a situation like pong where I only have like three variables like position y position of my of my paddle and position of the ball and velocity of the ball those are like I can specify those in five numbers but if it gets harder than that if it's like this labyrinth setting full 3d environment then we have no clue which states are similar to each other and what these what most end up doing is they will train you guessed it they will train a deep neural network to give you this similarity score between states right how they do it is is a different question but presumably you can train this network offline basically meaning you can pre train it you could pre train it and then the so we have two stages stage one pre train train similarity dnn right and then once we've done that second stage do reinforcement learning using this and the claim here is that by having this done this this second stage will become faster so it it doesn't really solve the problem of the sample efficiency but what it says is okay the actual reinforcement learning part will become faster because we've already done the work previously but basically by by including this similarity score sorry whatever dnn by including this in the language of the review here we have successfully introduced an inductive bias into the rl procedure because the rl procedure now can't just fit any function we say we tell it your value function is one that conforms to our notion of similarity that we've pre trained this restricts the rl algorithm and we give it an inductive bias and as long as our similarity score is useful for the rl algorithm it can speed up its learning because it doesn't have to learn the value function itself all right cool so the second part here is a bit more abstract it's called meta reinforcement learning speeding up deep rl by learning to learn these kind of learning to learn approaches are quite abundant in the literature people try this usually there's a i mean it's it's very large scale experiments basically you have i think i believe they show it somewhere here yeah you have like some um some outer loop where you would say that's this thing here what the outer loop does is in each loop it samples one environment so it samples one environment from a distribution of environments so now you not only have one environment but you say okay if i'm going to navigate this maze one trying to learn to navigate this maze i'm going actually to learn to learn to navigate many mazes right so it's not like you train one agent to learn you train one agent to navigate many mazes that would just be classic reinforcement learning but you want to train an algorithm that helps an agent learn as a particular maze and you do that by training your helper algorithm on a variety of agent maze combinations so in each step you sample one environment like this this here and you then have an inner loop here you fully reinforcement learn train an agent in the classic sense on this environment right you see here action action observation reward right but the agent receives some kind of signal from outside so the outside algorithm will kind of tell the agent how to approach the problem right this could be that it initializes the the weights here you see that the outer loop trains the parameter weights which determine the inner learner that interacts with an environment during the duration of the episode for every cycle of the outer loop a new environment is sampled from a distribution of environments which share some common structure so basically the one would expect when you train this that these parameters here this could be for example it could be the initial weights of the network that the agent uses that this one possibility right this is very abstract here this meta reinforcement learning it could be literally anything that the outer model teaches the inner model or gives to the inner model right and you you train both of these with reinforcement learning so the inner you train with reinforcement learning on the individual rewards and then you can train the outer loop on the reward that the entire app agent environment episode achieved so the that's kind of a two loop situation and yeah so that's meta reinforcement learning again it's very unspecified what it does but as you can already see if you now have such an algorithm that kind of tells the the inner agent just as an example how to initialize its weights right how to initialize the weights of its deep neural network if you have that here then the agent you will technically bias it this is again an inductive bias so you will give it inductive bias towards what you think are good weights to generally learn these maze structured environments right since the outer loop you can update it way slower because it needs to learn over a longer time horizon and it needs to learn things for a different variety of environments but once you have good kind of initial weights for a particular environment then this agent in here can learn much faster given an individual environment so the agent you instantiated and then you give it good starting weights or some other kind of signal about the environment and then it can go much much faster at learning the environment thereby you have just sped up this inner agent by providing it an inductive bias and that's basically what the claim of the review is that by providing these models with a larger inductive bias you may then speed up their learning because you've kind of told them what good functions are from the outset of course you see the problem again here well the problem the problem is of course you actually need to train this outer loop and the outer loop may actually take much much longer to train than a single and unbiased reinforcement learning thing but again what you could do is you could pre-train on a distribution of environments and then once a new environment shows up that is similar to this distribution you can then have the agent instantiated and learn much faster so again kind of this two-step process you could pre-train this outer loop and then the inner loop will be much faster than if you didn't have the outer loop all right so those are basically the kind of the kind of outlines they do here they then kind of do a connection to like the brain and so on and they relate this to biology and biological learning but ultimately their conclusion is here that whenever you want to do whenever you have slow rl or this is at least my conclusion from their article whenever you have slower you can transform it to fast rl rl but you have to outsource the slow rl slow something else slow x you have to outsource the slowness to some other part so if you want to do fast rl you have to outsource the slowness and what the slowness provides is an inductive bias which means yeah if you want to do like fast rl with episodic memory you have to learn the similarity function which again which might be slow in itself but then the rl will be fast and if you want to do this via kind of a an outer meta learner again this learning of the outer meta learner might be slow but then the inner learner will be fast in a connection to the kind of biological aspect of this they do make a connection which which i find is appropriate in that for example the human brain the reason we can learn things fast let's say in the physical world picking things up dropping things down or navigating our paths we're incredibly good at this navigating through like a weird terrain with rocks in the way is because of course our brains have been adapted to these kinds of environment over generations so there is an outer process like evolution which is this kind of outer loop and it instantiates the inner loop which are the humans that kind of live or die by their ability to to navigate better so the if if the outer loop does a good job of only keeping the humans alive that can navigate well then the individual human in here that that does this the individual human given a landscape with rocks will then be much faster at learning to navigate it all right so that was it for that i it's an interesting article to read especially the connections to the kind of biological aspects and with that have a nice day
 Hi there, today we're looking at reinforcement learning, fast and slow, by Matthew Botvinick,
 Sam Ritter, Jane X. Wang, Zeb Kurt-Nielsen, Charles Spondel and Demis Hassabis.
 These people are from Google DeepMind and this is a review of kind of a development
 in reinforcement learning, especially as it pertains to kind of how humans learn or what
 we can understand from the RL world that translates over to human learning.
 Alright, so basically their argument here is that the first wave of deep RL, as you
 see here, is powerful but slow.
 And they give examples of this.
 So in box one, box one is this.
 So they believe there's an image missing here.
 This is Backgammon, TD Gammon.
 This is the famous DeepMind Atari playing bot and this is kind of the 3D labyrinth playing
 bot.
 So there's been a number of advances in RL and especially what they talk about is deep
 RL.
 So when we talk about reinforcement learning, the easiest case is where you have an agent
 and an environment.
 Alright, so the agent will observe some observation O from the environment and then based on that
 the agent will perform an action A. And then the environment will give back a reward and
 also a next observation.
 So this is O0, O1, and then this is A0 and then here you give A1, AI.
 So basically this goes back and forth and back and forth.
 The agent performs an action, the environment gives a reward and the next observation.
 So this could be for example here in the Atari world.
 The observation is the screen itself.
 And then the agent needs to perform an action which is an input of the joystick or pressing
 some button.
 You can see the individual actions actually listed here.
 And then the reward will be given to the agent via a number which I guess is the same number
 as up here.
 So the task is to maximize the reward simply by...
 So the difference is you're not doing this in a supervised manner.
 So you're not telling the agent what would be the correct action to do.
 You simply tell it that whether what it did was good or bad by giving it a high or a low
 reward.
 Right, so that's reinforcement learning.
 So what is deep reinforcement learning?
 Deep reinforcement learning simply means the agent maps the observation to the action via
 a deep neural network.
 So deep neural network.
 That's deep reinforcement learning where the mapping or some part of the agent consists
 of a deep neural network.
 You see for example here there is a deep neural network mapping the observation to the action.
 As well as down here but it's a bit more complicated.
 So they argue that the first wave of this was powerful but slow meaning kind of you
 need a lot of samples.
 And they give two sources of why it's slow, why you need a lot of samples.
 They say the two factors are incremental parameter adjustment and weak inductive bias.
 So incremental parameter adjustment means basically that you have to update or train
 your neural network in a very small incremental way.
 In order to basically, because you train it one by one, right?
 You train your neural network step by step.
 You have to make small steps in order to not forget what came before.
 You can't fundamentally readjust your neural network to every new batch of observations
 because then that's going to destroy all the information you've learned of the old one.
 And then weak inductive bias here is basically an understanding of these neural networks.
 They are general function approximators and they can approximate any function.
 So if you just think in terms of kind of, I don't know, let's say polynomials and what
 kind of polynomials are there?
 This polynomial, this polynomial, this polynomial, this weird polynomial.
 If I have a function that can approximate all of these then I have a weak inductive
 bias whereas if I kind of know, okay all my polynomials are the polynomial that I'm looking
 for ultimately, I'm very sure it's a third degree polynomial, right?
 So something like this or like this or like this.
 So this is much less of a class of functions that I can fit but if I'm sure that the function
 that I'm trying to fit falls in this category then I'm much faster.
 So this is then called a strong inductive bias is where I build into the model basically
 I tell it beforehand.
 Here is a very restricted class of functions that you can fit.
 Whereas in a weak inductive bias I won't tell it that.
 I'll simply say, well model you could fit any function you want and I'm just giving
 you training samples.
 So this is a classic example of a bias variance trade-off where there is a lot of
 variance in these models meaning you can fit also a lot of functions but here because you
 bias the model towards a certain set of functions it can lower this variance and in this case
 here it speeds up learning because you don't have as much variance that means you can basically
 go faster while learning.
 Alright, so they propose two solutions to this problem of this kind of to mitigate these
 problems that make reinforcement learning faster or have made reinforcement learning
 faster.
 This is a review remember.
 So the first one is episodic deep reinforcement learning and this episodic deep reinforcement
 learning is specified here, fast learning through episodic memory.
 So the suggestion in this field of research is to augment the neural network or the agent
 by a memory and the memory could look something like this.
 So in a lot of these RL frameworks what a principal component of the agent is, so the
 agent will get an observation O and one of the things it has to do is estimate the value
 of this observation of this state.
 So basically the agent is in some state let's say you play pong right and you are here down
 and the ball comes your way up there right there's a little arrow sorry so the ball
 flies away from you and you're all the way down which basically means draw this bigger.
 So here you are down here and the ball is here flying up there.
 So one task in these in these agents that occurs often is to estimate the value of this
 observation basically means how much reward am I expecting from this state going into
 the future.
 In this case I probably will not expect a lot of reward since I can't move up fast enough
 right to catch the ball.
 So this I would assign this state a pretty low value whereas if I were up here I would
 assign this state quite a high value.
 So as we've already seen this is a deep neural network mapping we learn to assign value to
 different states and this is one of the parts that takes a long time and these methods they
 are the one that's depicted here replaces this value estimation by saying okay we have
 an observation we somehow need to estimate its value why don't we look for similar observation
 so we have some kind of memory right and we go with our observation and we retrieve O'1
 O'2 O'3 that are somehow similar right so in our in our pong example I'm down I'm up
 here ball moves here I could be looking now at at states where I was here or where I was
 here like really close or where the ball flew a bit differently but still in the same direction
 or down here right so all these states are kind of close to my state and I can I already
 have I already have played these since they're in my memory right so with every one of them
 I can also retrieve the reward that I got so I because I already know the problem in
 reinforcement learning is before you do an action you don't know what the reward will
 be but here I already know because I've played it I've already experienced it it's in the
 past so I know what reward I got right so and this is exactly what they say over here
 they basically say here we have time time runs this way we're in state one then in state
 two and so on and we perform actions and and get rewards and what we can do is we can save
 these states into this memory as along with their sum of discounted rewards that we collect
 from that state on and then later this is like a spongebob reference if we want to estimate
 the value of some new state right what we do is we retrieve all of these states from
 memory calculate a similarity score over them and with with we wait basically we add their
 rewards weighted by how similar they are to the state that we want to compute so this
 basically amounts to averaging over states respective by how close they are to the current
 state right this is kind of a soft a soft way of saying I only select the states which
 are close and that gives you a value estimate for the new states so basically this means
 you just got rid of having to train a value function and this will speed up your reinforcement
 learning quite a bit if you don't have to train that if you already have good value
 estimations from your previous experience that's great of course there are a number
 of problems associated with that namely if this memory here for example becomes stale
 it doesn't represent the future rewards quite as well there is also a question of which
 states do you keep in memory just the good ones or do they have to have a certain property
 do you have to have some diversity in there and of course the biggest problem here the
 biggest problem is how do you know when two states are similar or when they aren't it
 might be easy in a situation like pong where I only have like three variables like position
 y position of my of my paddle and position of the ball and velocity of the ball those
 are like I can specify those in five numbers but if it gets harder than that if it's like
 this labyrinth setting full 3d environment then we have no clue which states are similar
 to each other and what these what most end up doing is they will train you guessed it
 they will train a deep neural network to give you this similarity score between states right
 how they do it is is a different question but presumably you can train this network
 offline basically meaning you can pre train it you could pre train it and then the so
 we have two stages stage one pre train train similarity dnn right and then once we've done
 that second stage do reinforcement learning using this and the claim here is that by having
 this done this this second stage will become faster so it it doesn't really solve the problem
 of the sample efficiency but what it says is okay the actual reinforcement learning
 part will become faster because we've already done the work previously but basically by
 by including this similarity score sorry whatever dnn by including this in the language of the
 review here we have successfully introduced an inductive bias into the rl procedure because
 the rl procedure now can't just fit any function we say we tell it your value function is one
 that conforms to our notion of similarity that we've pre trained this restricts the
 rl algorithm and we give it an inductive bias and as long as our similarity score is useful
 for the rl algorithm it can speed up its learning because it doesn't have to learn the value
 function itself all right cool so the second part here is a bit more abstract it's called
 meta reinforcement learning speeding up deep rl by learning to learn these kind of learning
 to learn approaches are quite abundant in the literature people try this usually there's
 a i mean it's it's very large scale experiments basically you have i think i believe they
 show it somewhere here yeah you have like some um some outer loop where you would say
 that's this thing here what the outer loop does is in each loop it samples one environment
 so it samples one environment from a distribution of environments so now you not only have one
 environment but you say okay if i'm going to navigate this maze one trying to learn
 to navigate this maze i'm going actually to learn to learn to navigate many mazes right
 so it's not like you train one agent to learn you train one agent to navigate many mazes
 that would just be classic reinforcement learning but you want to train an algorithm that helps
 an agent learn as a particular maze and you do that by training your helper algorithm
 on a variety of agent maze combinations so in each step you sample one environment like
 this this here and you then have an inner loop here you fully reinforcement learn train
 an agent in the classic sense on this environment right you see here action action observation
 reward right but the agent receives some kind of signal from outside so the outside algorithm
 will kind of tell the agent how to approach the problem right this could be that it initializes
 the the weights here you see that the outer loop trains the parameter weights which determine
 the inner learner that interacts with an environment during the duration of the episode for every
 cycle of the outer loop a new environment is sampled from a distribution of environments
 which share some common structure so basically the one would expect when you train this that
 these parameters here this could be for example it could be the initial weights of the network
 that the agent uses that this one possibility right this is very abstract here this meta
 reinforcement learning it could be literally anything that the outer model teaches the
 inner model or gives to the inner model right and you you train both of these with reinforcement
 learning so the inner you train with reinforcement learning on the individual rewards and then
 you can train the outer loop on the reward that the entire app agent environment episode
 achieved so the that's kind of a two loop situation and yeah so that's meta reinforcement
 learning again it's very unspecified what it does but as you can already see if you
 now have such an algorithm that kind of tells the the inner agent just as an example how
 to initialize its weights right how to initialize the weights of its deep neural network if
 you have that here then the agent you will technically bias it this is again an inductive
 bias so you will give it inductive bias towards what you think are good weights to generally
 learn these maze structured environments right since the outer loop you can update it way
 slower because it needs to learn over a longer time horizon and it needs to learn things
 for a different variety of environments but once you have good kind of initial weights
 for a particular environment then this agent in here can learn much faster given an individual
 environment so the agent you instantiated and then you give it good starting weights
 or some other kind of signal about the environment and then it can go much much faster at learning
 the environment thereby you have just sped up this inner agent by providing it an inductive
 bias and that's basically what the claim of the review is that by providing these models
 with a larger inductive bias you may then speed up their learning because you've kind
 of told them what good functions are from the outset of course you see the problem again
 here well the problem the problem is of course you actually need to train this outer loop
 and the outer loop may actually take much much longer to train than a single and unbiased
 reinforcement learning thing but again what you could do is you could pre-train on a distribution
 of environments and then once a new environment shows up that is similar to this distribution
 you can then have the agent instantiated and learn much faster so again kind of this two-step
 process you could pre-train this outer loop and then the inner loop will be much faster
 than if you didn't have the outer loop all right so those are basically the kind of the
 kind of outlines they do here they then kind of do a connection to like the brain and so
 on and they relate this to biology and biological learning but ultimately their conclusion is
 here that whenever you want to do whenever you have slow rl or this is at least my conclusion
 from their article whenever you have slower you can transform it to fast rl rl but you
 have to outsource the slow rl slow something else slow x you have to outsource the slowness
 to some other part so if you want to do fast rl you have to outsource the slowness and
 what the slowness provides is an inductive bias which means yeah if you want to do like
 fast rl with episodic memory you have to learn the similarity function which again which
 might be slow in itself but then the rl will be fast and if you want to do this via kind
 of a an outer meta learner again this learning of the outer meta learner might be slow but
 then the inner learner will be fast in a connection to the kind of biological aspect of this they
 do make a connection which which i find is appropriate in that for example the human
 brain the reason we can learn things fast let's say in the physical world picking things
 up dropping things down or navigating our paths we're incredibly good at this navigating
 through like a weird terrain with rocks in the way is because of course our brains have
 been adapted to these kinds of environment over generations so there is an outer process
 like evolution which is this kind of outer loop and it instantiates the inner loop which
 are the humans that kind of live or die by their ability to to navigate better so the
 if if the outer loop does a good job of only keeping the humans alive that can navigate
 well then the individual human in here that that does this the individual human given
 a landscape with rocks will then be much faster at learning to navigate it all right so that
 was it for that i it's an interesting article to read especially the connections to the
 kind of biological aspects and with that have a nice day