Predictive Coding Approximates Backprop along Arbitrary Computation Graphs (Paper Explained)
deep learning machine learning arxiv explained neural networks ai artificial intelligence paper backpropagation computation autograph tensorflow pytorch torch autodiff differentiation backprop biologically plausible neurons error signal predictive coding variational gaussian iterative local updates distributed inner loop brain neuroscience deep neural networks analyzed hand drawing cnn rnn lstm convolutional neural network recurrent neural network hebian
#ai #biology #neuroscience

Backpropagation is the workhorse of modern deep learning and a core component of most frameworks, but it has long been known that it is not biologically plausible, driving a divide between neuroscience and machine learning. This paper shows that Predictive Coding, a much more biologically plausible algorithm, can approximate Backpropagation for any computation graph, which they verify experimentally by building and training CNNs and LSTMs using Predictive Coding. This suggests that the brain and deep neural networks could be much more similar than previously believed.

OUTLINE:
0:00 - Intro & Overview
3:00 - Backpropagation & Biology
7:40 - Experimental Results
8:40 - Predictive Coding
29:00 - Pseudocode
32:10 - Predictive Coding approximates Backprop
35:00 - Hebbian Updates
36:35 - Code Walkthrough
46:30 - Conclusion & Comments

Paper: https://arxiv.org/abs/2006.04182
Code: https://github.com/BerenMillidge/PredictiveCodingBackprop

Abstract:
Backpropagation of error (backprop) is a powerful algorithm for training machine learning architectures through end-to-end differentiation. However, backprop is often criticised for lacking biological plausibility. Recently, it has been shown that backprop in multilayer-perceptrons (MLPs) can be approximated using predictive coding, a biologically-plausible process theory of cortical computation which relies only on local and Hebbian updates. The power of backprop, however, lies not in its instantiation in MLPs, but rather in the concept of automatic differentiation which allows for the optimisation of any differentiable program expressed as a computation graph. Here, we demonstrate that predictive coding converges asymptotically (and in practice rapidly) to exact backprop gradients on arbitrary computation graphs using only local learning rules. We apply this result to develop a straightforward strategy to translate core machine learning architectures into their predictive coding equivalents. We construct predictive coding CNNs, RNNs, and the more complex LSTMs, which include a non-layer-like branching internal graph structure and multiplicative interactions. Our models perform equivalently to backprop on challenging machine learning benchmarks, while utilising only local and (mostly) Hebbian plasticity. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry, and may also contribute to the development of completely distributed neuromorphic architectures.

Authors: Beren Millidge, Alexander Tschantz, Christopher L. Buckley

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
 Hi there, this is an LSTM cell or the computation graph of an LSTM cell. It is pretty hideous as you can see, but what I'm about to show you is even more hideous. This is the computation graph of the LSTM cell augmented with error units, evincing the connectivity scheme of the predictive coding algorithm. You may see that there are appearing these little red arrows right here that are so called error units. These are necessary for an algorithm called predictive coding, which is an algorithm that is a biologically plausible alternative to backprop. That's what we're going to look at today, specifically this paper as you can see. It is quite a thorough paper. It is called Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs. Have you ever heard a more descriptive title of what's in a paper? The authors are Baron Millage, Alexander Chantz, and Christopher L. Buckley. This paper, as the title says, it looks at this predictive coding algorithm and it shows that this approximates backprop. We'll see that this approximates is in terms of there is an inner iteration in the predictive coding algorithm. The more you run that and under certain assumptions, this approximates the backprop algorithm. The new thing in this paper is along arbitrary computation graphs. There have been papers before describing predictive coding, this algorithm, in various sub-settings like fully connected layers and so on. The fact that it approximates backprop there. However, this paper shows that that's actually the case for arbitrary computation graphs under certain assumptions. Predictive coding approximates the backpropagation algorithm. Why is this important? Because the backpropagation algorithm isn't exactly biologically plausible. So they say right here in the abstract backpropagation of error or short backprop is a powerful algorithm for training machine learning architectures through end-to-end differentiation. Recently has been shown that backprop in multilayer perceptrons can be approximated using predictive coding, a biologically plausible process theory of cortical computation which relies solely on local and Hebbian updates. So the difference between backpropagation and predictive coding is exactly this point that predictive coding relies solely on local and Hebbian updates. The keyword I think is local. So in a neural network you have some sort of input x and you ship it through many layers, layer, layer, layer, layer and then you have an output y hat and then you compare that output using a some kind of loss function with your with your true output that you want and then there is this backwards phase right here and in this backwards phase you want to derive gradients for each of the layers weights. So each of these layers has a weight associated with it. I'm not going into Greek letters again. So this is w I don't know w3 w2 is here and so on. So what you want to get out is you want to say how do I need to change w in order to change my loss for the better. So what you want is this gradient right here and backpropagation does a very natural decomposition namely if you have these hidden states in here so x is transformed to hidden state h0 h1 h2 h3 so that is the latent representation. If you want for example weight if you want to know how to change weight or let's say weight two the backpropagation algorithm decomposes this into the derivative according to the hidden state at layer two multiplied by the derivative of the hidden state by the weight. So this is what you would sort of learn in a beginner's course of deep learning this decomposition and of course in this part right here this part decomposes into del L for h3 and then h3 by h2. So this is the standard backpropagation algorithm you can clearly see in the formula the computation graph it goes from the L it flows backward to h3 right so to h3 and then from h3 it flows to h2 and then from h2 it flows to w2 so that's sort of the flow of the gradient backwards through the network and that's pretty cool because it allows us to run gradient descent on arbitrary computation graphs which ultimately enable deep learning including frameworks like tensorflow, PyTorch or the older ones like Theano or Lua torch even autograd things like this. It's pretty cool but it's not really plausible in the brain because neurons are not bi-directional like this. Neurons generally I'm not a neuroscientist or anything but these neurons they have some sort of soma and then you have this axon right and then this axon goes into many different of these synapses to its children and it kind of docks onto the somas of or on the dendrites of the other neurons and this is not bi-directional this is generally here there's a unidirectional signal in this direction and there are so-called feedback connections so from these neurons to the dendrites of this neuron but you cannot really send this gradient information you cannot send this sort of vector gradient information and you cannot do so in this sort of sweep so in the brain it's probably not the case that the layer propagates forward and then sort of waits for a synchronized backward pass across the network in order to update itself. All of this needs to happen much more in parallel much more local so that things are only considering local information of global information right here for example you need the global gradient in the update of w2 and you need to have that back propagated that's not plausible so predictive coding comes along and today we'll look mainly actually at how predictive coding works of course this paper is about extending it to arbitrary computation graphs which is cool because they do predictive coding for cnn's rnn's and even lstm's and if you look at their so let's first jump into the numerical results if you look at their numerical results they have lots of these plots where they basically show we did this network we train it with backprop and then we train it with predictive coding and the lines are just the same and so it's pretty convincing evidence even if you go super duper deep and they do i think rn ends with up to 100 layers or 100 time steps unrolled so the empirical evidence that predictive coding approximates backprop is certainly here and we'll look at what predictive coding is how it works and how it works along arbitrary computation graphs so that's today's paper and i hope you enjoy it if you do don't hesitate to share it out and subscribe all right so all right so this graphic right here compares the two algorithms in principle on top very much what i've said so far the backprop algorithm somehow has this signal it propagates forward okay and then at some point there's an output and if you want to train it there is a label you compare that to the output that will give you an error and by derivation a gradient and that gradient is now back propagated according to the chain rule according to the back propagation algorithm you can see it's very much what i've drawn the predictive coding algorithm is a little bit different and it's honestly not super clear from this graphic right here i find this graphic to be to be a bit confusing but you can see first of all there is this introduction of these of these error nodes in the computation graph right here and there also seems to be the introduction of these new hats whatever that is so we're sort of first going to dive into the math and then we're going to check out how the algorithm works as such so the math right here is a little bit it's a little you have to think a little bit differently than you do in backprop so first of all they say we define a generative model which parameterizes the value of each vertex given the feedforward prediction of its parents according to this distribution and a factorized variational posterior where p denotes the set of parents and c denotes the set of children of a given node x so this is this is very special namely this turns the entire algorithm into a sort of a guessing game into a variational approximation algorithm so what they're basically saying is that signal in this type of algorithm signal isn't just forward propagated but signal is signal is forward guessed it's like a bit of a guess so you have a signal right here vi and this is a node in your neural network and when you forward propagate the signal maybe this is a fully connected layer right here so it's simply multiplying it by parameter you're not you're not going to obtain the next layer's signal what you're going to obtain is a guess for the next layer's signal right here you're only guessing you're assuming that you're sort of assuming that the true next signal is somewhere in the vicinity of this so what you do is actually assume this is a Gaussian with the mean that you predicted but then there is a fair a good chance it's somewhere around here so what you do is you always you'll guess the next layer's signal by forward propagating your own signal and you're so you're not directly computing it okay and the model that we have for that here and you know it's why do we do this we do this because we're also not so sure about this one right here okay so this entire thing is built upon we're pretty sure what the input is and we're pretty sure what the label is of a data point but without you know we're not we assume we're not really sure what the intermediate layers are and we're going to run sort of an update procedure on these on our guesses of where these intermediate signals are and that's going to be this predictive coding algorithm so it's called predictive coding I guess because you always only predict where the next layer signal might be and you refine that prediction in a series of inner iteration steps and that all before you even do a parameter update so there's going to be an inner iteration to determine what the forward values are of the network and this is very different from back prop there is just a single forward pass right then you know the values and then there's a backward pass here there is as you'll see there is a single forward pass but then there is an inner loop to refine the forward pass before there is a backward pass and we need this because we only do this sort of local updates you'll see in a second so the the Gaussian I just drew so the assumption the assumption is going to be that we refine iteratively refine these up these guesses of where vi is and of course here you'll see that if I if I change vi to be down here my next guess so this is at time step t I mean my guess is this my times that t plus one is this of course if I apply the same fully connected layer my new guess is going to be down here somewhere and so the assumption here that we're going to make is that they you can see the value of each vertex is a is this model right here this is the generative model so it's a probability distribution depending on the parents and we're going to approximate that by this variational posterior which as you can see doesn't depend on the parents anymore so it basically says that the distribution stays the stays is not is not conditional it sort of stays the same I'm not sure if I express this quite correctly but you can see right here they assume a Gaussian for the generative model that's dependent on on these things and then the the posterior is simply a factorized Gaussian and the variational approximation algorithm simply makes the KL divergence between this variational posterior and the true assumed posterior small and they can prove that this is equal to these errors and the errors are going to be the errors between what's predicted and what's guessed yeah it's best if we if we so if I extend this right here right I have v0 okay v0 I'm pretty sure what it is because it's my input then what I'm going to do is I'm going to forward guess what v1 is so this is my guess of v1 now from v1 I am going to guess what v2 is and at the beginning you know my guess of v1 is the same as my forward prediction I have no other reason I have no reason to assume it's anywhere else so I'm just going to draw this on top of v1 right here so since you know it could be anywhere it could be anywhere in the vicinity here but I'm going to assume it's the same I have no reason to do so otherwise and then I'm going to predict v2 okay and v2 let's say that's already my output layer and this is my guess of v2 that's already my output layer but but now we're going to compare v2 to our true output what we desire our label l and there's going to be an error okay so there's going to be an error right here and what the predictive coding algorithm does is it basically says well look v2 could be actually anywhere here anywhere around this thing it's most likely in the middle but it could be anywhere and it's actually quite possible that it's closer to this label than we initially guessed so it takes this error right here this red error and it says I'm going to update my guess of v2 a little bit closer into that direction so I don't have it here is a new color so v2 is going to be a little bit closer here it's it's possible right it's we we simply guessed v2 so it could also be there it's a little bit less likely it's a little bit less likely because it's not in the middle of the Gaussian but v2 could be where l is right but now I have to sort of communicate this error back to the last one and the trick here is that we don't communicate the global gradient but we only communicate these local error signals so this first red arrow here is our first error signal and we are going to communicate that thing back to the to the previous layer so the difference between v2 and v and here is a fully connect let's say this is a fully connected layer what we're going to send back to the last layer is this information of you see you predicted v2 hat but actually you should predict v2 please update yourself such that that doesn't you know that's that's a bit closer so now we're going to update our guess of v1 and say well if we moved v1 a little bit over here that would predict v2 to be up here right with the same fully connected layer and if we if if that's the case then v2 would be a little closer to the true label so we're going to move v1 over here now we're not going to move it fully because so this is a sort of optimization there is a there is a force keeping it to where our original guess is but there is also a force drawing it in the direction of this of this error signal you can see so we're going to say well if we just move v1 to up here we would predict the perfect v2 but also it's less likely so we're going to find like some sort of a trade-off where it's still quite likely under our gaussian assumption but it will predict a little bit more of the correct label and so on so this if we had a longer computation graph this would then sort of every node in the computation graph would ask itself i i'm going to guess my own value at a place that is pretty close to my original guess coming from the forward propagation but also is consistent with the output of the next layer and the output of the next layer of course here is this this v2 right so that the logic isn't i need to make the loss small the logic is well if the next signal is v2 then i can't be in the middle here i must be a little bit more up here because you know i i my signal runs through the fully connected layer and outputs v2 so i am probably more up here so you can see that if you have a computation graph v0 v1 hat v2 hat v3 hat and so on if at the end you have a loss signal you're sort of distributing distributing that loss across this entire chain so you're you're kind of building this guessed chain of values v3 and so on and sorry the that's that's the output node which is close to the loss you're moving all of these things and now once you've done this once you've done this you can do one step of parameter updates so once you've guessed all the nodes well you can go ahead and say okay um this is this is a configuration that is at equilibrium in this sort of algorithm and now here are here is fully connected layer one so here is um here is w0 here is w1 w2 and so on w3 so now we can go ahead and actually update these weights such that the initial guesses that we had and where we truly think the signal is are closer together okay so we're now going to update the weights in order to minimize all of these individual errors and this is also can be done locally so you see that the parameter update step here is now a local one because we've computed all of these errors between where we initially guess the signal is and where we sort of think it should be now we can minimize these errors so what i've drawn here is actually not it's not exactly the algorithm but i hope you get the point so step one is you sort of guess where all the stuff is initially then at the end you get an error signal right this is an error signal then you distribute that error signal backwards and that is now that is not the same as distributing a gradient i know it looks the same but it is not the same and so i have to say that you know they say oh this is only local and so on this doesn't require a backward sweep i think when i look at this algorithm it very much does require a backward sweep so very much it goes from the back to the front in fact it goes from the back to the front many times now you can do that in parallel so this node here can update so to finish the argument here as i said before then you kind of wiggle on these nodes to find out this should probably be more here this one should probably be more here this one should probably be more here this one should probably be more here in order to satisfy in order to make that error smaller and the point is that the parameter update step now is a local one okay so the parameter update step now only needs these local errors between where you initially guessed and where your refined iterative guess is after distributing the error through the network and this can all happen in parallel this this um all of this updating sending information around and so on this can be parallelized but it does require a backward sweep if you ask me okay so there are two equations so the the there's two things right here there is first as we said there is a phase where the guesses of where our vertex units are where our hidden representations are are refined and this is given by these dynamics right here so you see that vi changes with time according to this thing right here f is the variational free energy so this this algorithm sort of falls out from the math of assuming these um assuming these generative models right here under the assumption that they are these gaussians okay um so under under this assumption if you calculate the kl divergence um it turns out to come out to this algorithm right here so how does the how do we need to update the node vi the node vi is updated according to this gradient and this gradient is as we said only computed as properties of local things so the first thing is ei which is that's so again if we have this is our initial guess of vi and then here is our refined guess of vi ei is the error right here that's that's sort of we need to stay close to our initial guess but also we want to go into the direction such that um into this direction right here so ej j is the children of vi j are the children and this thing right here says how do we need to change my guess of vi to make um to make it fall more in line with vj and you see here that's vj uh the initial thing but then of course the error is so the error j is going to be the difference between vj and vj hat so ultimately you are guessing you're saying how do i need to change vi in order to make it more commensurate with vj after going through the the layer okay so this um this derivative right here this is going to involve the derivative of whatever the fully connected layer or the conv layer and so on so there is not there's not no derivatives in this algorithm but there are only sort of these local derivatives so ei is going to be the difference here and then we'll have the fully connected layer using w gives you vj hat but also your refined guess gives you vj and the error j is going to be this thing right here okay so at you want to stay close right here but also you want to um make vi such that it outputs vj such that it also minimizes that error okay sort of um yeah it's it's hard to it's hard to draw these things but i hope i've explained it in multiple ways right now it's at least a little bit clear how this works and at the end once you've reached equilibrium of all of your guesses of um all of your guesses of where the next nodes are what you do is you update your parameters here in a local fashion you can see right here what you need is this error of the if layer and you multiply that by this derivative and this derivative is simply the local derivative of your hidden representation with respect to your layer okay so this is very akin to in the back propagation algorithm hi to wi this is just this local derivative so using the update the update step of the weights now only requires local derivatives and that's the point so here it's in this pseudo code things are a little bit a little bit unclear in this but we'll do so for the entire data set x is the data point and l is the label you fix the start so you fix v0 then you go you do the forward pass so you do this once you these are your initial guesses um these hat things you can see the hat things are always computed from the parents you compute the output error right here and then begin backwards iteration phase of the descent on the free energy so here you see there is this inner loop while not converged and this is just going to work out to be some sort of in some sort of an inner iterative scheme for a number of steps this is going to be a hyper parameter and this here this is something you can technically do in parallel you have to send a bit of information around but you can technically do it in parallel this inner these these inner loops but you can you can just imagine it always going from the back and you distribute these errors you refine your guests a little bit and you start from the back again you distribute errors refine your guesses and so on and you do that you always start from the back in the actual code so you compute these errors so this is your initial guess and this is your refined guess of the current layer and then you update the vertex values you say okay the my guess for the next layer is going to be my guess for this layer plus some sort of a this gradient and this gradient we get from equation number two from this thing right here so my guess is going to be updated such that i still stay close to my original guess but i also update i also predict better what the next layer is and at the end when this is converged you do the update on the weights and the updates on the weights is simply again this what we saw it's the error that you want to correct so this e is the error you want to correct now you have a good approximation of the error once this is converged uh times the derivative of course with respect to the weights so the error is in terms of how how much are your predictions of from what they should be and the derivative simply translates that into the how do you need to change the weights such that in the future that error is smaller okay so then they show that this actually approximates a back prop and this it's a it's a fairly um fairly simple proof it's an it's sort of a proof by induction by iteration that's showing that um one one such one such thing like this this thing right here at the equilibrium at the last layer is equivalent to back prop and because you can simply substitute this and then by sort of recursion that goes back the layers and this is all dependent on you actually reaching that equilibrium which you do as we said by inner iterations so they have a bit of a they have a bit of a an example right here where they have this function of um it's a pretty simple function this function right here the output is the tan of this square root and there's parameters in there right so this is an arbitrary parameter that you might want to learn and then you give some data sets um so this is equal to two but i guess the network doesn't know that i don't know so you have to learn it and they they test that and you can see the this augmentation by error graphs makes the computational graph quite a bit more um complex so you have all these error graphs right here but you know ultimately error ultimately it's you can you could automate this that that is not a problem okay so um they also do this for as i said cnn's rnns lstms and the results are quite remarkable i think in that they they just follow the same accuracy and loss and performance patterns of these networks that's pretty cool the downside of course is that um they are way smaller sorry they're way way slower and they say this sometimes um due to the need to iterate the v's until convergence the predictive coding network had roughly a 100 times greater computational cost than the backprop network though they say this is a bit misleading because you can distribute and parallelize that however as we've seen it's not fully local like you you need to send signal around every node needs to send signal to its parents or its children and um that of course in in backprop you just need to do that once right so i'm not exactly buying this argument of this is much more local and so on so the last thing that i want to point out in the paper and then we looked briefly at the code is this thing right here there's a further simplification they say importantly if the edge function linearly combines the activities and the parameters followed by an element-wise non-linearity which is most of deep learning layers nowadays a condition which we call parameter linear then both the update rule for the vertices and the parameters become Hebbian specifically the update rules for the vertices and the weights become so here is here is um if you have a linear operation followed by a non-linearity which you know is the fact in RNNs in CNNs in fully connected layers then this here are these update rules so the local layer derivative is simply going to be your forward activations passed through and this is a bit weird um it's the forward activations passed through the derivation of the non-linearity this is the non-linearity right here um times again the weights of the forward iteration and the update rule with respect to the parameters are very very similar and the reason i point this out because now we're going to jump into the code and i hope you can see this um you can recognize this again so first of all let's go into the um into the CNN hello all right so the code is quite ugly honestly but um you see that they have their backprop or CNNs but they have this thing right here this um um this model which is the one they train and here is the train function so in the train function they go through the data set and you can see for each data point they simply call this infer function right here so this infer function is what ultimately does the training so in the infer function they get an input as you can see and a label and a number of inference steps so they start out by and this this is labeled a bit a bit different so they have these mus and the outs and these prediction errors and the predictions and we're going to see how that works so first of all they go through the layers right here and i'm going to use my mouse they go through the layers right here and you can see they simply forward propagate the signal so they always take this mu of the last layer they forward propagate it to get the mu on the layer plus one and the outputs are simply cloned from the mus so these must be our news before or our v's whatever you want to call them so one one is going to be the initial guess and the other one is going to be the guess that we iteratively refine okay in fact the mu here is going to be the guess that we iteratively refine at the beginning we simply set them to be the same okay and then the last layer here we put at the label and then the prediction errors that's going to be yeah that's going to be the the error variables so the last prediction error is going to be the derivative of our loss function with respect to the last layer and now we start this iterative algorithm so here you see we go through this number of inference steps train which is going to be like a hundred or so so a hundred times we're going to update each of our guesses of the intermediate layers then here is what i said we're going through the layers in reverse order so a hundred times we're going from back to front back to front back to front back to front and we do that so here you can see what the first thing we do is we come we compute the current error okay which is the difference between the guess that we currently have and the initial guess that we had during forward propagation this is going to be zero for most of the layers at the beginning except the last layer right in the last layer we've actually put we've actually put the the mu to something else than the output and thus this error is going to it's beginning at zero at each layer as the guesses are the same but then we're going to refine and refine and refine and sort of this error of the last layer is going to iteratively propagate through the network to the from the back to the front multiple in an iterative fashion so multiple times so once we have the prediction error we're going to backward this through the layers and this backward here that is sort of that is this this backward edge we saw where did we see this so this backward is going to be the this local derivative in this graph the backward is going to be the the red thing right here so we take the error of the next layer and we're going to we're going to see how do we need to change the current guess in order to make the next layers error be a little bit smaller okay so that's the going to be the backward function and we can actually look at the backward function of let's say yeah here so this is the backward function of a fully connected layer this is the projection layer there is a fully connect here is there is a fully connected layer and the f is going to be the non-linearity and the df is going to be the derivative of the non-linearity so in the forward you can see what we're doing is we're multiplying the input by the weights and then we're going to save the activations and simply propagate them through the non-linearity in the backwards we're going to take the activations this the forward activation then we're going to shove them through the derivative of the non-linearity and this is why i pointed out this is this Hebbian learning rule so first i was a bit confused why do we use the forward activations and shove them through the derivative of the non-linearity but this is exactly this is simply because they've derived that this is the correct local gradient okay and then we have this right this is the local gradient of the layer and we're going to multiply that by the weights so this completes the formula that we had right here for these Hebbian updates this thing so these are the activations this is the derivative of the forward layer we're going to multiply that by the weight again so this is now the complete derivative the complete local derivative which is this thing i've already circled 50 billion times right here and all we need to do now is we need to multiply this by the error in prior prediction error in that layer and then we get an idea of how do we need to change this node such that in this one child and there can be many children such that in this one child we make a little bit less error okay so that's why we multiply this by e right here so e is the the error okay and that will be the backwards thing so backwards simply tells the parent how it needs to change the child sorry how it needs to change itself such that the child is a little bit happier and since this is a forward you know a cnn we don't have multiple children we simply have one child per parent so we have a list and these predictions as you can see we simply take the prediction error of layer j plus one we backward it so how do we need to change this layer in order to make it a little bit more commensurate with the child and then here is this trade-off so the trade-off between the prediction error so how close am i to my original guess i don't want to go too far away right because i assume my original guess isn't too bad in fact there's a gaussian likelihood model how i want to stay close to that but also i want to go into the direction such that i make the next layer happier okay so this is this fundamental trade-off it's computed right here and it's it's this minus sign and then at the end this is the inference learning rate and i simply go into that direction of this trade-off okay so i update the current the guess of the current node like this and as i said i go through the network back to front back to front back to front back to front until i reach some sort of equilibrium and only when i reach equilibrium or in this case after this many steps i then update the weights and the update weights function that's very similar i think here here is update weights that is simply i each layer i input the prediction error of that layer and that layer calculates this function right here in much a similar way than you just than you just saw maybe we can look at one of them let's go this is layers let's go here fully connected layer okay and you're going to see this Hebbian learning rule again so activations through the derivative and so now instead of so there's a little bit of a difference to before right but the difference isn't isn't large right so activations multiplied by through this and then multiplied by the inputs instead of the weights so that's that's that so this multiplied by the inputs instead of the weights then multiplied by e which is so this here multiplied by the error term right here and that's going to be our local update okay cool so that's the code that's predictive coding and you know the challenge is it's not that these people propose this as a true alternative to back prop but it is a step in a direction of saying look the brain with its more Hebbian nature and its more local updates and so on it could actually be doing something much more close to back prop than we thought because people thought well back prop is impossible in the brain therefore the brain can't be doing back prop right and now we see that actually the brain can do something possibly it's not proven but it's possible that the brain does something that approximates the back prop gradient actually arbitrarily if you know if all of these if these some assumptions are given but that's sort of the the results and they also show it's quite robust to learning rate changes and so on as we said we can go pretty deep even though this is this kind of iterative guessing algorithm under these Gaussian assumptions and there is variational approximation it is fairly robust and all so this goes this sort of puts the ball back into maybe the brain is doing something very close to back prop or at least getting the same results getting the same parameter updates as back prop so i hope that wasn't too confusing i've tried to tackle it from many angles and maybe after seeing the code you see it a little bit more clearly if not let me know open for questions as always and bye bye
 Hi there, this is an LSTM cell or the computation graph of an LSTM cell. It is pretty hideous as you
 can see, but what I'm about to show you is even more hideous. This is the computation graph of the
 LSTM cell augmented with error units, evincing the connectivity scheme of the predictive coding
 algorithm. You may see that there are appearing these little red arrows right here that are so
 called error units. These are necessary for an algorithm called predictive coding, which is an
 algorithm that is a biologically plausible alternative to backprop. That's what we're going
 to look at today, specifically this paper as you can see. It is quite a thorough paper. It is called
 Predictive Coding Approximates Backprop Along Arbitrary Computation Graphs. Have you ever heard
 a more descriptive title of what's in a paper? The authors are Baron Millage, Alexander Chantz,
 and Christopher L. Buckley. This paper, as the title says, it looks at this predictive coding
 algorithm and it shows that this approximates backprop. We'll see that this approximates
 is in terms of there is an inner iteration in the predictive coding algorithm. The more you run that
 and under certain assumptions, this approximates the backprop algorithm. The new thing in this
 paper is along arbitrary computation graphs. There have been papers before describing predictive
 coding, this algorithm, in various sub-settings like fully connected layers and so on. The fact
 that it approximates backprop there. However, this paper shows that that's actually the case for
 arbitrary computation graphs under certain assumptions. Predictive coding approximates
 the backpropagation algorithm. Why is this important? Because the backpropagation algorithm
 isn't exactly biologically plausible. So they say right here in the abstract backpropagation of error
 or short backprop is a powerful algorithm for training machine learning architectures through
 end-to-end differentiation. Recently has been shown that backprop in multilayer perceptrons can
 be approximated using predictive coding, a biologically plausible process theory of cortical
 computation which relies solely on local and Hebbian updates. So the difference between
 backpropagation and predictive coding is exactly this point that predictive coding relies solely
 on local and Hebbian updates. The keyword I think is local. So in a neural network you have some sort
 of input x and you ship it through many layers, layer, layer, layer, layer and then you have an
 output y hat and then you compare that output using a some kind of loss function with your
 with your true output that you want and then there is this backwards phase right here and in this
 backwards phase you want to derive gradients for each of the layers weights. So each of these layers
 has a weight associated with it. I'm not going into Greek letters again. So this is w I don't know w3
 w2 is here and so on. So what you want to get out is you want to say how do I need to change w
 in order to change my loss for the better. So what you want is this gradient right here
 and backpropagation does a very natural decomposition namely if you have these hidden
 states in here so x is transformed to hidden state h0 h1 h2 h3 so that is the latent representation.
 If you want for example weight if you want to know how to change weight or let's say weight two
 the backpropagation algorithm decomposes this into the derivative according to the hidden state at
 layer two multiplied by the derivative of the hidden state by the weight. So this is what you
 would sort of learn in a beginner's course of deep learning this decomposition and of course
 in this part right here this part decomposes into del L for h3 and then h3 by h2. So this is the
 standard backpropagation algorithm you can clearly see in the formula the computation graph it goes
 from the L it flows backward to h3 right so to h3 and then from h3 it flows to h2 and then from h2
 it flows to w2 so that's sort of the flow of the gradient backwards through the network
 and that's pretty cool because it allows us to run gradient descent on arbitrary computation graphs
 which ultimately enable deep learning including frameworks like tensorflow, PyTorch or the older
 ones like Theano or Lua torch even autograd things like this. It's pretty cool but it's not
 really plausible in the brain because neurons are not bi-directional like this. Neurons generally
 I'm not a neuroscientist or anything but these neurons they have some sort of soma and then
 you have this axon right and then this axon goes into many different of these synapses to its
 children and it kind of docks onto the somas of or on the dendrites of the other neurons and this
 is not bi-directional this is generally here there's a unidirectional signal in this direction and
 there are so-called feedback connections so from these neurons to the dendrites of this neuron
 but you cannot really send this gradient information you cannot send this sort of vector
 gradient information and you cannot do so in this sort of sweep so in the brain it's probably not
 the case that the layer propagates forward and then sort of waits for a synchronized backward pass
 across the network in order to update itself. All of this needs to happen much more in parallel
 much more local so that things are only considering local information of global information
 right here for example you need the global gradient in the update of w2 and you need to
 have that back propagated that's not plausible so predictive coding comes along and today we'll look
 mainly actually at how predictive coding works of course this paper is about extending it to
 arbitrary computation graphs which is cool because they do predictive coding for cnn's rnn's and even
 lstm's and if you look at their so let's first jump into the numerical results if you look at
 their numerical results they have lots of these plots where they basically show we did this
 network we train it with backprop and then we train it with predictive coding and the lines are
 just the same and so it's pretty convincing evidence even if you go super duper deep
 and they do i think rn ends with up to 100 layers or 100 time steps unrolled so the empirical evidence
 that predictive coding approximates backprop is certainly here and we'll look at what predictive
 coding is how it works and how it works along arbitrary computation graphs so that's today's
 paper and i hope you enjoy it if you do don't hesitate to share it out and subscribe all right
 so all right so this graphic right here compares the two algorithms in principle on top very much
 what i've said so far the backprop algorithm somehow has this signal it propagates forward
 okay and then at some point there's an output and if you want to train it there is a label you compare
 that to the output that will give you an error and by derivation a gradient and that gradient is now
 back propagated according to the chain rule according to the back propagation algorithm you
 can see it's very much what i've drawn the predictive coding algorithm is a little bit different
 and it's honestly not super clear from this graphic right here i find this graphic to be
 to be a bit confusing but you can see first of all there is this introduction of these
 of these error nodes in the computation graph right here and there also seems to be the
 introduction of these new hats whatever that is so we're sort of first going to dive into the
 math and then we're going to check out how the algorithm works as such so the math right here
 is a little bit it's a little you have to think a little bit differently than you do in backprop so
 first of all they say we define a generative model which parameterizes the value of each vertex
 given the feedforward prediction of its parents according to this distribution and a factorized
 variational posterior where p denotes the set of parents and c denotes the set of children of a
 given node x so this is this is very special namely this turns the entire algorithm into a
 sort of a guessing game into a variational approximation algorithm so what they're
 basically saying is that signal in this type of algorithm signal isn't just forward propagated
 but signal is signal is forward guessed it's like a bit of a guess so you have a signal right here
 vi and this is a node in your neural network and when you forward propagate the signal maybe this
 is a fully connected layer right here so it's simply multiplying it by parameter you're not
 you're not going to obtain the next layer's signal what you're going to obtain is a guess
 for the next layer's signal right here you're only guessing you're assuming that
 you're sort of assuming that the true next signal is somewhere in the vicinity of this so what you
 do is actually assume this is a Gaussian with the mean that you predicted but then
 there is a fair a good chance it's somewhere around here so what you do is you always you'll
 guess the next layer's signal by forward propagating your own signal and you're
 so you're not directly computing it okay and the model that we have for that here and you know it's
 why do we do this we do this because we're also not so sure about this one right here okay so
 this entire thing is built upon we're pretty sure what the input is and we're pretty sure what the
 label is of a data point but without you know we're not we assume we're not really sure what the
 intermediate layers are and we're going to run sort of an update procedure on these on our guesses
 of where these intermediate signals are and that's going to be this predictive coding algorithm so
 it's called predictive coding I guess because you always only predict where the next layer signal
 might be and you refine that prediction in a series of inner iteration steps and that all before
 you even do a parameter update so there's going to be an inner iteration to determine what the
 forward values are of the network and this is very different from back prop there is just a single
 forward pass right then you know the values and then there's a backward pass here there is as you'll
 see there is a single forward pass but then there is an inner loop to refine the forward pass before
 there is a backward pass and we need this because we only do this sort of local updates you'll see
 in a second so the the Gaussian I just drew so the assumption the assumption is going to be that we
 refine iteratively refine these up these guesses of where vi is and of course here you'll see that
 if I if I change vi to be down here my next guess so this is at time step t I mean my guess is this
 my times that t plus one is this of course if I apply the same fully connected layer my new guess
 is going to be down here somewhere and so the assumption here that we're going to make is that
 they you can see the value of each vertex is a is this model right here this is the generative
 model so it's a probability distribution depending on the parents and we're going to approximate that
 by this variational posterior which as you can see doesn't depend on the parents anymore so
 it basically says that the distribution stays the stays is not is not conditional it sort of stays
 the same I'm not sure if I express this quite correctly but you can see right here they assume
 a Gaussian for the generative model that's dependent on on these things and then the the posterior
 is simply a factorized Gaussian and the variational approximation algorithm simply makes the KL
 divergence between this variational posterior and the true assumed posterior small and they can
 prove that this is equal to these errors and the errors are going to be the errors between
 what's predicted and what's guessed yeah it's best if we if we
 so if I extend this right here right I have v0 okay v0 I'm pretty sure what it is because it's my
 input then what I'm going to do is I'm going to forward guess what v1 is so this is my guess of v1
 now from v1 I am going to guess what v2 is and at the beginning you know my guess of v1 is the same
 as my forward prediction I have no other reason I have no reason to assume it's anywhere else so
 I'm just going to draw this on top of v1 right here so since you know it could be anywhere it
 could be anywhere in the vicinity here but I'm going to assume it's the same I have no reason
 to do so otherwise and then I'm going to predict v2 okay and v2 let's say that's already my output
 layer and this is my guess of v2 that's already my output layer but but now
 we're going to compare v2 to our true output what we desire our label l and there's going to be an
 error okay so there's going to be an error right here and what the predictive coding algorithm does
 is it basically says well look v2 could be actually anywhere here anywhere around this
 thing it's most likely in the middle but it could be anywhere and it's actually quite possible that
 it's closer to this label than we initially guessed so it takes this error right here this red error
 and it says I'm going to update my guess of v2 a little bit closer into that direction so
 I don't have it here is a new color so v2 is going to be a little bit closer here it's
 it's possible right it's we we simply guessed v2 so it could also be there it's a little bit less
 likely it's a little bit less likely because it's not in the middle of the Gaussian but v2 could be
 where l is right but now I have to sort of communicate this error back to the last one and
 the trick here is that we don't communicate the global gradient but we only communicate these
 local error signals so this first red arrow here is our first error signal and we are going to
 communicate that thing back to the to the previous layer so the difference between v2 and v
 and here is a fully connect let's say this is a fully connected layer what we're going to send
 back to the last layer is this information of you see you predicted v2 hat but actually you should
 predict v2 please update yourself such that that doesn't you know that's that's a bit closer so now
 we're going to update our guess of v1 and say well if we moved v1 a little bit over here that would
 predict v2 to be up here right with the same fully connected layer and if we if if that's the case
 then v2 would be a little closer to the true label so we're going to move v1 over here now we're not
 going to move it fully because so this is a sort of optimization there is a there is a force keeping
 it to where our original guess is but there is also a force drawing it in the direction of this
 of this error signal you can see so we're going to say well if we just move v1 to up here we would
 predict the perfect v2 but also it's less likely so we're going to find like some sort of a trade-off
 where it's still quite likely under our gaussian assumption but it will predict a little bit more
 of the correct label and so on so this if we had a longer computation graph this would then sort of
 every node in the computation graph would ask itself i i'm going to guess my own value at a place
 that is pretty close to my original guess coming from the forward propagation but also is consistent
 with the output of the next layer and the output of the next layer of course here is this this v2
 right so that the logic isn't i need to make the loss small the logic is well if the next signal
 is v2 then i can't be in the middle here i must be a little bit more up here because you know i
 i my signal runs through the fully connected layer and outputs v2 so i am probably more up here so you
 can see that if you have a computation graph v0 v1 hat v2 hat v3 hat and so on if at the end you
 have a loss signal you're sort of distributing distributing that loss across this entire chain
 so you're you're kind of building this guessed chain of values v3 and so on and sorry the that's
 that's the output node which is close to the loss you're moving all of these things and now once
 you've done this once you've done this you can do one step of parameter updates so once you've
 guessed all the nodes well you can go ahead and say okay um this is this is a configuration that
 is at equilibrium in this sort of algorithm and now here are here is fully connected layer one so
 here is um here is w0 here is w1 w2 and so on w3 so now we can go ahead and actually update these
 weights such that the initial guesses that we had and where we truly think the signal is are closer
 together okay so we're now going to update the weights in order to minimize all of these
 individual errors and this is also can be done locally so you see that the parameter update step
 here is now a local one because we've computed all of these errors between where we initially
 guess the signal is and where we sort of think it should be now we can minimize these errors so
 what i've drawn here is actually not it's not exactly the algorithm but i hope you get the point
 so step one is you sort of guess where all the stuff is initially then at the end you get an error
 signal right this is an error signal then you distribute that error signal backwards and that
 is now that is not the same as distributing a gradient i know it looks the same but it is
 not the same and so i have to say that you know they say oh this is only local and so on this
 doesn't require a backward sweep i think when i look at this algorithm it very much does require
 a backward sweep so very much it goes from the back to the front in fact it goes from the back
 to the front many times now you can do that in parallel so this node here can update so to finish
 the argument here as i said before then you kind of wiggle on these nodes to find out this should
 probably be more here this one should probably be more here this one should probably be more here
 this one should probably be more here in order to satisfy in order to make that error smaller
 and the point is that the parameter update step now is a local one okay so the parameter update
 step now only needs these local errors between where you initially guessed and where your refined
 iterative guess is after distributing the error through the network and this can all happen in
 parallel this this um all of this updating sending information around and so on this can be
 parallelized but it does require a backward sweep if you ask me okay so there are two equations so
 the the there's two things right here there is first as we said there is a phase where the guesses
 of where our vertex units are where our hidden representations are are refined and this is given
 by these dynamics right here so you see that vi changes with time according to this thing right
 here f is the variational free energy so this this algorithm sort of falls out from the math
 of assuming these um assuming these generative models right here under the assumption that they
 are these gaussians okay um so under under this assumption if you calculate the kl divergence
 um it turns out to come out to this algorithm right here so how does the how do we need to update the
 node vi the node vi is updated according to this gradient and this gradient is as we said only
 computed as properties of local things so the first thing is ei which is that's so again if we have
 this is our initial guess of vi and then here is our refined guess of vi ei is the error right here
 that's that's sort of we need to stay close to our initial guess but also we want to go into
 the direction such that um into this direction right here so ej j is the children of vi j are
 the children and this thing right here says how do we need to change my guess of vi to make um
 to make it fall more in line with vj and you see here that's vj uh the initial thing but then
 of course the error is so the error j is going to be the difference between vj and vj hat so
 ultimately you are guessing you're saying how do i need to change vi in order to make it more
 commensurate with vj after going through the the layer okay so this um this derivative right here
 this is going to involve the derivative of whatever the fully connected layer or the conv layer
 and so on so there is not there's not no derivatives in this algorithm but there are only
 sort of these local derivatives so ei is going to be the difference here and then
 we'll have the fully connected layer using w gives you vj hat but also your refined guess gives you
 vj and the error j is going to be this thing right here okay so at you want to stay close
 right here but also you want to um make vi such that it outputs vj such that it also minimizes that
 error okay sort of um yeah it's it's hard to it's hard to draw these things but i hope i've explained
 it in multiple ways right now it's at least a little bit clear how this works and at the
 end once you've reached equilibrium of all of your guesses of um all of your guesses of where the next
 nodes are what you do is you update your parameters here in a local fashion you can see right here what
 you need is this error of the if layer and you multiply that by this derivative and this derivative
 is simply the local derivative of your hidden representation with respect to your layer okay
 so this is very akin to in the back propagation algorithm hi to wi this is just this local
 derivative so using the update the update step of the weights now only requires local derivatives
 and that's the point so here it's in this pseudo code things are a little bit a little bit unclear
 in this but we'll do so for the entire data set x is the data point and l is the label you fix the
 start so you fix v0 then you go you do the forward pass so you do this once you these are your initial
 guesses um these hat things you can see the hat things are always computed from the parents
 you compute the output error right here and then begin backwards iteration phase of the descent
 on the free energy so here you see there is this inner loop while not converged and this is just
 going to work out to be some sort of in some sort of an inner iterative scheme for a number of steps
 this is going to be a hyper parameter and this here this is something you can technically do in
 parallel you have to send a bit of information around but you can technically do it in parallel
 this inner these these inner loops but you can you can just imagine it always going from the back
 and you distribute these errors you refine your guests a little bit and you start from the back
 again you distribute errors refine your guesses and so on and you do that you always start from
 the back in the actual code so you compute these errors so this is your initial guess and this is
 your refined guess of the current layer and then you update the vertex values you say okay
 the my guess for the next layer is going to be my guess for this layer plus some sort of a this
 gradient and this gradient we get from equation number two from this thing right here so my guess
 is going to be updated such that i still stay close to my original guess but i also update
 i also predict better what the next layer is
 and at the end when this is converged you do the update on the weights and the updates on the weights
 is simply again this what we saw it's the error that you want to correct so this e is the error
 you want to correct now you have a good approximation of the error once this is converged
 uh times the derivative of course with respect to the weights so the error is in terms of
 how how much are your predictions of from what they should be and the derivative simply translates
 that into the how do you need to change the weights such that in the future that error is smaller
 okay so then they show that this actually approximates a back prop and this it's a it's a
 fairly um fairly simple proof it's an it's sort of a proof by induction by iteration that's showing
 that um one one such one such thing like this this thing right here at the equilibrium at the last
 layer is equivalent to back prop and because you can simply substitute this and then by sort of
 recursion that goes back the layers and this is all dependent on you actually reaching that
 equilibrium which you do as we said by inner iterations so they have a bit of a they have a
 bit of a an example right here where they have this function of um it's a pretty simple function
 this function right here the output is the tan of this square root and there's parameters in there
 right so this is an arbitrary parameter that you might want to learn and then you give some data
 sets um so this is equal to two but i guess the network doesn't know that i don't know
 so you have to learn it and they they test that and you can see the this augmentation by error
 graphs makes the computational graph quite a bit more um complex so you have all these error graphs
 right here but you know ultimately error ultimately it's you can you could automate this that that is
 not a problem okay so um they also do this for as i said cnn's rnns lstms and the results are quite
 remarkable i think in that they they just follow the same accuracy and loss and performance patterns
 of these networks that's pretty cool the downside of course is that um they are way smaller sorry
 they're way way slower and they say this sometimes um due to the need to iterate the v's until
 convergence the predictive coding network had roughly a 100 times greater computational cost
 than the backprop network though they say this is a bit misleading because you can distribute and
 parallelize that however as we've seen it's not fully local like you you need to send signal around
 every node needs to send signal to its parents or its children and um that of course in in backprop
 you just need to do that once right so i'm not exactly buying this argument of this is much more
 local and so on so the last thing that i want to point out in the paper and then we looked briefly
 at the code is this thing right here there's a further simplification they say importantly if the
 edge function linearly combines the activities and the parameters followed by an element-wise
 non-linearity which is most of deep learning layers nowadays a condition which we call parameter
 linear then both the update rule for the vertices and the parameters become Hebbian specifically
 the update rules for the vertices and the weights become so here is here is um if you have a linear
 operation followed by a non-linearity which you know is the fact in RNNs in CNNs in fully
 connected layers then this here are these update rules so the local layer derivative is simply
 going to be your forward activations passed through and this is a bit weird um it's the
 forward activations passed through the derivation of the non-linearity this is the non-linearity
 right here um times again the weights of the forward iteration and the update rule with respect
 to the parameters are very very similar and the reason i point this out because now we're going
 to jump into the code and i hope you can see this um you can recognize this again so first of all
 let's go into the um into the CNN hello all right so the code is quite ugly honestly but um
 you see that they have their backprop or CNNs but they have this thing right here this um
 um this model which is the one they train and here is the train function so in the train function
 they go through the data set and you can see for each data point they simply call this infer
 function right here so this infer function is what ultimately does the training so in the infer
 function they get an input as you can see and a label and a number of inference steps so they start
 out by and this this is labeled a bit a bit different so they have these mus and the outs
 and these prediction errors and the predictions and we're going to see how that works so first of all
 they go through the layers right here and i'm going to use my mouse they go through the layers
 right here and you can see they simply forward propagate the signal so they always take this
 mu of the last layer they forward propagate it to get the mu on the layer plus one and the outputs
 are simply cloned from the mus so these must be our news before or our v's whatever you want to
 call them so one one is going to be the initial guess and the other one is going to be the guess
 that we iteratively refine okay in fact the mu here is going to be the guess that we iteratively
 refine at the beginning we simply set them to be the same okay and then the last layer here we
 put at the label and then the prediction errors that's going to be yeah that's going to be the
 the error variables so the last prediction error is going to be the derivative of our loss function
 with respect to the last layer and now we start this iterative algorithm so here you see we go
 through this number of inference steps train which is going to be like a hundred or so so a hundred
 times we're going to update each of our guesses of the intermediate layers then here is what i said
 we're going through the layers in reverse order so a hundred times we're going from back to front
 back to front back to front back to front and we do that so here you can see what the first thing
 we do is we come we compute the current error okay which is the difference between the guess that we
 currently have and the initial guess that we had during forward propagation this is going to be
 zero for most of the layers at the beginning except the last layer right in the last layer
 we've actually put we've actually put the the mu to something else than the output and thus this
 error is going to it's beginning at zero at each layer as the guesses are the same but then we're
 going to refine and refine and refine and sort of this error of the last layer is going to iteratively
 propagate through the network to the from the back to the front multiple in an iterative fashion so
 multiple times so once we have the prediction error we're going to backward this through the
 layers and this backward here that is sort of that is this this backward edge we saw where did we see
 this so this backward is going to be the this local derivative in this graph the backward is going to
 be the the red thing right here so we take the error of the next layer and we're going to
 we're going to see how do we need to change the current guess in order to make the next
 layers error be a little bit smaller okay so that's the going to be the backward function and we can
 actually look at the backward function of let's say yeah here so this is the backward function of a
 fully connected layer this is the projection layer there is a fully connect here is there is a fully
 connected layer and the f is going to be the non-linearity and the df is going to be the
 derivative of the non-linearity so in the forward you can see what we're doing is we're
 multiplying the input by the weights and then we're going to save the activations and simply
 propagate them through the non-linearity in the backwards we're going to take the activations
 this the forward activation then we're going to shove them through the derivative of the
 non-linearity and this is why i pointed out this is this Hebbian learning rule so first i was a bit
 confused why do we use the forward activations and shove them through the derivative of the
 non-linearity but this is exactly this is simply because they've derived that this is the correct
 local gradient okay and then we have this right this is the local gradient of the layer and we're
 going to multiply that by the weights so this completes the formula that we had right here for
 these Hebbian updates this thing so these are the activations this is the derivative of the forward
 layer we're going to multiply that by the weight again so this is now the complete derivative the
 complete local derivative which is this thing i've already circled 50 billion times right here
 and all we need to do now is we need to multiply this by the error in prior prediction error in
 that layer and then we get an idea of how do we need to change this node such that in this one
 child and there can be many children such that in this one child we make a little bit less error
 okay so that's why we multiply this by e right here so e is the the error okay and that will be
 the backwards thing so backwards simply tells the parent how it needs to change the child sorry how
 it needs to change itself such that the child is a little bit happier and since this is a forward
 you know a cnn we don't have multiple children we simply have one child per parent so we have a list
 and these predictions as you can see we simply take the prediction error of layer j plus one we
 backward it so how do we need to change this layer in order to make it a little bit more commensurate
 with the child and then here is this trade-off so the trade-off between the prediction error so
 how close am i to my original guess i don't want to go too far away right because i assume my
 original guess isn't too bad in fact there's a gaussian likelihood model how i want to stay
 close to that but also i want to go into the direction such that i make the next layer happier
 okay so this is this fundamental trade-off it's computed right here and it's it's this minus sign
 and then at the end this is the inference learning rate and i simply go into that direction of this
 trade-off okay so i update the current the guess of the current node like this and as i said i go
 through the network back to front back to front back to front back to front until i reach some
 sort of equilibrium and only when i reach equilibrium or in this case after this many steps
 i then update the weights and the update weights function that's very similar i think here here is
 update weights that is simply i each layer i input the prediction error of that layer and
 that layer calculates this function right here in much a similar way than you just than you just saw
 maybe we can look at one of them let's go this is layers let's go here fully connected layer
 okay and you're going to see this Hebbian learning rule again so activations through the derivative
 and so now instead of so there's a little bit of a difference to before right but the difference
 isn't isn't large right so activations multiplied by through this and then multiplied by the inputs
 instead of the weights so that's that's that so this multiplied by the inputs instead of the weights
 then multiplied by e which is so this here multiplied by the error term right here
 and that's going to be our local update okay cool so that's the code that's predictive coding
 and you know the challenge is it's not that these people propose this as a true alternative to
 back prop but it is a step in a direction of saying look the brain with its more Hebbian nature and
 its more local updates and so on it could actually be doing something much more close to back prop
 than we thought because people thought well back prop is impossible in the brain therefore
 the brain can't be doing back prop right and now we see that actually the brain can do something
 possibly it's not proven but it's possible that the brain does something that approximates the
 back prop gradient actually arbitrarily if you know if all of these if these some assumptions are given
 but that's sort of the the results and they also show it's quite robust to learning rate changes
 and so on as we said we can go pretty deep even though this is this kind of iterative
 guessing algorithm under these Gaussian assumptions and there is variational approximation
 it is fairly robust and all so this goes this sort of puts the ball back into maybe the brain is
 doing something very close to back prop or at least getting the same results getting the same
 parameter updates as back prop so i hope that wasn't too confusing i've tried to tackle it from
 many angles and maybe after seeing the code you see it a little bit more clearly if not let me know
 open for questions as always and bye bye