Seeing Inside HTM Algorithms [DEMO #9]
Andrews, Felix
Felix shows off some really interesting visualizations of HTMs using Comportex and ComportexViz.
 Cool. So, yeah, as Matt said, I'm going to talk about my attempts to see inside HTNs. It's using Kompotex Fizz that Marcus has already helpfully introduced. And I have to say as an aside, he was very misleading in his comments there. He actually did a lot of substantial work on Kompotex Fizz there and the integration. And it's fantastic to have Kompotex running in on a server on the JVM, which has a lot more capacity than running in the browser. So I'm looking forward to using that. So, if you're interested in Kompotex Fizz, it's on GitHub and it has, it's able to run HTN in the browser and do some visualizations of it. And why don't I just demonstrate that really quickly. So, if I run this, this is a sensory motor data stream. So, to just get you oriented in this space. On the left, we have a representation of what the input is. In this case, it's just like a sequence of letters, A, B, C, D, E, F, G, H, A, J, K, not H, A, J, J. And it's currently focusing on one letter, so represented by the black line. So basically this letter sees just the input in a, through a category encoder. Okay. And because this is a sensory motor case study, it has an additional bit of input, which is the direction that we're going to move in the next time step, the saccade, you might call it. And that's coming from this, this region again, it's like a category about the direction and amount of the movement. These fields here are the activated columns. In this case, there's two layers, layer four and layer three. Layer four receives the sensory input on the proximal feed forward segment. And it also receives the motor input on the, as a distal input to the distal segments. So that's just to get you sort of oriented about what all these dots are. But it's not really what I want to talk about today, because what I worked on for my hack was the temporal pulling algorithm. I've been trying to implement the temporal pulling algorithm. I'm not going to explain it here, if you don't know what temporal pulling is, you can look it up on the new mental wiki, some good stuff. Basically, the idea is to form stable, but distinct representations in higher level regions or higher level layers. So, and it does that by keeping cells active over time while the underlying sequence is predictable. So, the problem I ran into, I've been playing around with implementing this, but as you go through the process, you realize that there's a lot of, it's not clear how to do it exactly. And what should be the relative weight of all these parameters that you're faced with creating. And what I ended up realizing I needed was an ability to visualize what the different influences are on the cells that are becoming active. And because it's a sparse representation, that's feasible. There's only a relatively small number of active cells on each time step. It's usually, the number of columns, which is usually like 2% of the number of columns. So, in this example, what I'm showing here on the right, the individual bars here are single cells that are becoming active. So, there's a total of, I'd say, 12 or 15 cells that are becoming active. And each one, the color represents the amount of influence that's causing it to become active. Okay, so red is just the proximal excitation as represented by these lines here. It's the proximal excitation, it's driving that. The yellow, I don't know if you can see the yellow, that's the boosting factor, which is part of the algorithm. That's to do with encouraging columns to become active, that hasn't become active recently. So, it surprised me actually when I saw this, how much effective boosting is having it. Obviously, I'd set that value, but I'd forgotten about it. And it was invisible until now. So, this is at the beginning, you know, it's just proximal forcing. But if we run this for a little while, it's just 100 time steps or so. It will start to be able to predict. So, the first layer will start to be able to predict what's happening. And we can see that in the time series plot. So, in this plot, although it's the same kind of structure, it's showing a completely different thing. In this case, it's showing the distribution of the states of the active columns. So, red being active and unpredicted, active but not predicted to become active. And purple is correctly predicted to become active. And you can see, over time, more and more, a higher proportion of the columns were predicted, as you'd expect. We're just moving constantly around this one sequence of letters. So, of course, it's being predicted. Okay, I'm going to stop that there. Also, incidentally, the light blue represents columns which were predicted but did not become active. So, if we look now at what's happening, okay, I'm just going to turn off these. Okay. So, you can see on the lower level layer, there's cells which are predicted. But the red, these represent the active cells, but the red indicates that the feed forward input that's activating those was not predicted and can't be predicted because it's coming from the input data. It's actually not really relevant there. But if you look at the layer three, which is the next layer in the stack, the purple here is indicating that what's forcing these cells to become active is input. So, the purple lines to these synapses representing that the proximal input that's turning those cells on is coming from cells which were predicted below. So, it means the underlying sequence is predicted. And in that case, it adds to a persistent level of excitation called the temporal pooling level, which, okay, on this plot is the green line. So, it's that which is responsible for forcing some of these cells to become active in the higher level region. And if we go, I'm just stepping forward in time here. So, you have a situation where the cell has become, this set of cells has become active. And because it's from the predicted input, then in the following time step, you've got a large amount of persistent excitation, which is the green lines there. And that just decays over time until those cells turn off. And if you step forward, then you've got red coming in. That means something's been unpredicted, not predicted in the layer below, and then the temporal pooling tends to turn off in that case. So this has helped me to just calibrate or check that nothing crazy is going on. And there were just crazy things like the number of active synapses coming in and activating cells was way too low. It was like five synapses. So it's just good to be able to check that, like sanity check, and to start to get a feel for what the relative influences of the temporal, doing temporal pooling activation and how fast it's dropping off and causing it to drop off. I haven't really explored this much, but that's just where I got to. And hopefully it's given you some idea of the concepts involved in that algorithm. Any questions? The predicted eye line off the chart right down there. Oh yeah, it's sort of circuiting forward. And in that case, I just wrap it around. Oh, it's going to come into A or something? Yeah. I'd like to do this better. What we should have is multiple words and then you're going to have micro-secades within it and then high level secades going to the next word. So you can start actually, what I was originally hoping to look at was forming a higher level sequence memory, like actually predicting the sequence of words on top, like in the next layer. I realized that I've got more fundamental issues to do with how the algorithm is implemented before I can start to look at that. So what's the impact of the predicted eye line? So your predictions are the letter that is going to be seen next. Oh, this is not a prediction. What does that contribute? You mean this? Does the dim line? Yeah, that's not a prediction. That's actually what the next movement is going to be. Right. Oh, so I'm sorry, it's not a prediction, but it's the next move. But is that fed in also? Yes. So it's this two inputs. One is sensory input and one's motor input. So the motor input is only given in as distal input. But it's always anticipatory input. Interesting. Yeah. And there's different choices that could be made there as well about whether it comes in proximally or distally, whether you also have lateral distal connections in layer four. I'm not familiar with the current thinking on that actually. Yeah. So I was just curious. Test, test, test. I was curious what, I see you put this in JS, JavaScript, right? Well, it's compiled to JS from Closure. From Closure. Did you have to port the entire algorithm or did you? Yeah. Yeah. Okay. Interesting. Thanks. More questions? Thank you Felix. So I meant to say on the robotics, on the robotics, GitHub page, there's a link to this which just has some interactive online demos, different types of examples. Yeah. So check that out.
 Cool. So, yeah, as Matt said, I'm going to talk about my attempts to see inside HTNs.  It's using Kompotex Fizz that Marcus has already helpfully introduced. And I have to say as  an aside, he was very misleading in his comments there. He actually did a lot of substantial  work on Kompotex Fizz there and the integration. And it's fantastic to have Kompotex running  in on a server on the JVM, which has a lot more capacity than running in the browser.  So I'm looking forward to using that. So, if you're interested in Kompotex Fizz, it's  on GitHub and it has, it's able to run HTN in the browser and do some visualizations  of it. And why don't I just demonstrate that really quickly. So, if I run this, this is  a sensory motor data stream. So, to just get you oriented in this space. On the left, we  have a representation of what the input is. In this case, it's just like a sequence of  letters, A, B, C, D, E, F, G, H, A, J, K, not H, A, J, J. And it's currently focusing on  one letter, so represented by the black line. So basically this letter sees just the input  in a, through a category encoder. Okay. And because this is a sensory motor case study,  it has an additional bit of input, which is the direction that we're going to move in  the next time step, the saccade, you might call it. And that's coming from this, this  region again, it's like a category about the direction and amount of the movement. These  fields here are the activated columns. In this case, there's two layers, layer four and layer  three. Layer four receives the sensory input on the proximal feed forward segment. And  it also receives the motor input on the, as a distal input to the distal segments. So  that's just to get you sort of oriented about what all these dots are. But it's not really  what I want to talk about today, because what I worked on for my hack was the temporal pulling  algorithm. I've been trying to implement the temporal pulling algorithm. I'm not going  to explain it here, if you don't know what temporal pulling is, you can look it up on  the new mental wiki, some good stuff. Basically, the idea is to form stable, but distinct representations  in higher level regions or higher level layers. So, and it does that by keeping cells active  over time while the underlying sequence is predictable. So, the problem I ran into, I've  been playing around with implementing this, but as you go through the process, you realize  that there's a lot of, it's not clear how to do it exactly. And what should be the relative  weight of all these parameters that you're faced with creating. And what I ended up realizing  I needed was an ability to visualize what the different influences are on the cells that  are becoming active. And because it's a sparse representation, that's feasible. There's only  a relatively small number of active cells on each time step. It's usually, the number  of columns, which is usually like 2% of the number of columns. So, in this example, what  I'm showing here on the right, the individual bars here are single cells that are becoming  active. So, there's a total of, I'd say, 12 or 15 cells that are becoming active. And  each one, the color represents the amount of influence that's causing it to become active.  Okay, so red is just the proximal excitation as represented by these lines here. It's the  proximal excitation, it's driving that. The yellow, I don't know if you can see the yellow,  that's the boosting factor, which is part of the algorithm. That's to do with encouraging  columns to become active, that hasn't become active recently. So, it surprised me actually  when I saw this, how much effective boosting is having it. Obviously, I'd set that value,  but I'd forgotten about it. And it was invisible until now. So, this is at the beginning, you  know, it's just proximal forcing. But if we run this for a little while, it's just 100  time steps or so. It will start to be able to predict. So, the first layer will start  to be able to predict what's happening. And we can see that in the time series plot. So,  in this plot, although it's the same kind of structure, it's showing a completely different  thing. In this case, it's showing the distribution of the states of the active columns. So, red  being active and unpredicted, active but not predicted to become active. And purple is  correctly predicted to become active. And you can see, over time, more and more, a higher  proportion of the columns were predicted, as you'd expect. We're just moving constantly  around this one sequence of letters. So, of course, it's being predicted. Okay, I'm going  to stop that there. Also, incidentally, the light blue represents columns which were predicted  but did not become active. So, if we look now at what's happening, okay, I'm just going  to turn off these. Okay. So, you can see on the lower level layer, there's cells which  are predicted. But the red, these represent the active cells, but the red indicates that  the feed forward input that's activating those was not predicted and can't be predicted  because it's coming from the input data. It's actually not really relevant there. But if  you look at the layer three, which is the next layer in the stack, the purple here is indicating  that what's forcing these cells to become active is input. So, the purple lines to these synapses  representing that the proximal input that's turning those cells on is coming from cells  which were predicted below. So, it means the underlying sequence is predicted. And in that  case, it adds to a persistent level of excitation called the temporal pooling level, which,  okay, on this plot is the green line. So, it's that which is responsible for forcing  some of these cells to become active in the higher level region. And if we go, I'm just  stepping forward in time here. So, you have a situation where the cell has become, this  set of cells has become active. And because it's from the predicted input, then in the  following time step, you've got a large amount of persistent excitation, which is the green  lines there. And that just decays over time until those cells turn off. And if you step  forward, then you've got red coming in. That means something's been unpredicted, not predicted  in the layer below, and then the temporal pooling tends to turn off in that case.  So this has helped me to just calibrate or check that nothing crazy is going on. And  there were just crazy things like the number of active synapses coming in and activating  cells was way too low. It was like five synapses. So it's just good to be able to check that,  like sanity check, and to start to get a feel for what the relative influences of the temporal,  doing temporal pooling activation and how fast it's dropping off and causing it to drop  off. I haven't really explored this much, but that's just where I got to. And hopefully  it's given you some idea of the concepts involved in that algorithm. Any questions?  The predicted eye line off the chart right down there.  Oh yeah, it's sort of circuiting forward. And in that case, I just wrap it around.  Oh, it's going to come into A or something? Yeah. I'd like to do this better. What we  should have is multiple words and then you're going to have micro-secades within it and  then high level secades going to the next word. So you can start actually, what I was  originally hoping to look at was forming a higher level sequence memory, like actually  predicting the sequence of words on top, like in the next layer. I realized that I've got  more fundamental issues to do with how the algorithm is implemented before I can start  to look at that. So what's the impact of the predicted eye line? So your predictions are  the letter that is going to be seen next. Oh, this is not a prediction. What does that  contribute? You mean this? Does the dim line? Yeah, that's not a prediction. That's actually  what the next movement is going to be. Right. Oh, so I'm sorry, it's not a prediction, but  it's the next move. But is that fed in also? Yes. So it's this two inputs. One is sensory  input and one's motor input. So the motor input is only given in as distal input. But  it's always anticipatory input. Interesting. Yeah. And there's different choices that could  be made there as well about whether it comes in proximally or distally, whether you also  have lateral distal connections in layer four. I'm not familiar with the current thinking  on that actually. Yeah. So I was just curious. Test, test, test. I was curious what, I see  you put this in JS, JavaScript, right? Well, it's compiled to JS from Closure. From Closure.  Did you have to port the entire algorithm or did you? Yeah. Yeah. Okay. Interesting.  Thanks. More questions? Thank you Felix. So I meant to say on the robotics, on the  robotics, GitHub page, there's a link to this which just has some interactive online demos,  different types of examples. Yeah. So check that out.
Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__0_239.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__240_887.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__888_4979.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__4980_5315.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__5316_5675.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__5676_5819.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__5820_6359.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__6360_9083.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__9084_13643.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__13644_13691.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__13692_13811.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__13812_14519.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__14520_14819.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__14820_15143.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__15144_15683.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__15684_16115.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__16116_20591.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__20592_20963.png Seeing_Inside_HTM_Algorithms__DEMO__9___2015_Spring_NuPIC_Hackathon__20964_20972.png
Cool. So, yeah, as Matt said, I'm going to talk about my attempts to see inside HTNs. It's using Kompotex Fizz that Marcus has already helpfully introduced. And I have to say as an aside, he was very misleading in his comments there. He actually did a lot of substantial work on Kompotex Fizz there and the integration. And it's fantastic to have Kompotex running in on a server on the JVM, which has a lot more capacity than running in the browser. So I'm looking forward to using that. So, if you're interested in Kompotex Fizz, it's on GitHub and it has, it's able to run HTN in the browser and do some visualizations of it. And why don't I just demonstrate that really quickly. So, if I run this, this is a sensory motor data stream. So, to just get you oriented in this space. On the left, we have a representation of what the input is. In this case, it's just like a sequence of letters, A, B, C, D, E, F, G, H, A, J, K, not H, A, J, J. And it's currently focusing on one letter, so represented by the black line. So basically this letter sees just the input in a, through a category encoder. Okay. And because this is a sensory motor case study, it has an additional bit of input, which is the direction that we're going to move in the next time step, the saccade, you might call it. And that's coming from this, this region again, it's like a category about the direction and amount of the movement. These fields here are the activated columns. In this case, there's two layers, layer four and layer three. Layer four receives the sensory input on the proximal feed forward segment. And it also receives the motor input on the, as a distal input to the distal segments. So that's just to get you sort of oriented about what all these dots are. But it's not really what I want to talk about today, because what I worked on for my hack was the temporal pulling algorithm. I've been trying to implement the temporal pulling algorithm. I'm not going to explain it here, if you don't know what temporal pulling is, you can look it up on the new mental wiki, some good stuff. Basically, the idea is to form stable, but distinct representations in higher level regions or higher level layers. So, and it does that by keeping cells active over time while the underlying sequence is predictable. So, the problem I ran into, I've been playing around with implementing this, but as you go through the process, you realize that there's a lot of, it's not clear how to do it exactly. And what should be the relative weight of all these parameters that you're faced with creating. And what I ended up realizing I needed was an ability to visualize what the different influences are on the cells that are becoming active. And because it's a sparse representation, that's feasible. There's only a relatively small number of active cells on each time step. It's usually, the number of columns, which is usually like 2% of the number of columns. So, in this example, what I'm showing here on the right, the individual bars here are single cells that are becoming active. So, there's a total of, I'd say, 12 or 15 cells that are becoming active. And each one, the color represents the amount of influence that's causing it to become active. Okay, so red is just the proximal excitation as represented by these lines here. It's the proximal excitation, it's driving that. The yellow, I don't know if you can see the yellow, that's the boosting factor, which is part of the algorithm. That's to do with encouraging columns to become active, that hasn't become active recently. So, it surprised me actually when I saw this, how much effective boosting is having it. Obviously, I'd set that value, but I'd forgotten about it. And it was invisible until now. So, this is at the beginning, you know, it's just proximal forcing. But if we run this for a little while, it's just 100 time steps or so. It will start to be able to predict. So, the first layer will start to be able to predict what's happening. And we can see that in the time series plot. So, in this plot, although it's the same kind of structure, it's showing a completely different thing. In this case, it's showing the distribution of the states of the active columns. So, red being active and unpredicted, active but not predicted to become active. And purple is correctly predicted to become active. And you can see, over time, more and more, a higher proportion of the columns were predicted, as you'd expect. We're just moving constantly around this one sequence of letters. So, of course, it's being predicted. Okay, I'm going to stop that there. Also, incidentally, the light blue represents columns which were predicted but did not become active. So, if we look now at what's happening, okay, I'm just going to turn off these. Okay. So, you can see on the lower level layer, there's cells which are predicted. But the red, these represent the active cells, but the red indicates that the feed forward input that's activating those was not predicted and can't be predicted because it's coming from the input data. It's actually not really relevant there. But if you look at the layer three, which is the next layer in the stack, the purple here is indicating that what's forcing these cells to become active is input. So, the purple lines to these synapses representing that the proximal input that's turning those cells on is coming from cells which were predicted below. So, it means the underlying sequence is predicted. And in that case, it adds to a persistent level of excitation called the temporal pooling level, which, okay, on this plot is the green line. So, it's that which is responsible for forcing some of these cells to become active in the higher level region. And if we go, I'm just stepping forward in time here. So, you have a situation where the cell has become, this set of cells has become active. And because it's from the predicted input, then in the following time step, you've got a large amount of persistent excitation, which is the green lines there. And that just decays over time until those cells turn off. And if you step forward, then you've got red coming in. That means something's been unpredicted, not predicted in the layer below, and then the temporal pooling tends to turn off in that case. So this has helped me to just calibrate or check that nothing crazy is going on. And there were just crazy things like the number of active synapses coming in and activating cells was way too low. It was like five synapses. So it's just good to be able to check that, like sanity check, and to start to get a feel for what the relative influences of the temporal, doing temporal pooling activation and how fast it's dropping off and causing it to drop off. I haven't really explored this much, but that's just where I got to. And hopefully it's given you some idea of the concepts involved in that algorithm. Any questions? The predicted eye line off the chart right down there. Oh yeah, it's sort of circuiting forward. And in that case, I just wrap it around. Oh, it's going to come into A or something? Yeah. I'd like to do this better. What we should have is multiple words and then you're going to have micro-secades within it and then high level secades going to the next word. So you can start actually, what I was originally hoping to look at was forming a higher level sequence memory, like actually predicting the sequence of words on top, like in the next layer. I realized that I've got more fundamental issues to do with how the algorithm is implemented before I can start to look at that. So what's the impact of the predicted eye line? So your predictions are the letter that is going to be seen next. Oh, this is not a prediction. What does that contribute? You mean this? Does the dim line? Yeah, that's not a prediction. That's actually what the next movement is going to be. Right. Oh, so I'm sorry, it's not a prediction, but it's the next move. But is that fed in also? Yes. So it's this two inputs. One is sensory input and one's motor input. So the motor input is only given in as distal input. But it's always anticipatory input. Interesting. Yeah. And there's different choices that could be made there as well about whether it comes in proximally or distally, whether you also have lateral distal connections in layer four. I'm not familiar with the current thinking on that actually. Yeah. So I was just curious. Test, test, test. I was curious what, I see you put this in JS, JavaScript, right? Well, it's compiled to JS from Closure. From Closure. Did you have to port the entire algorithm or did you? Yeah. Yeah. Okay. Interesting. Thanks. More questions? Thank you Felix. So I meant to say on the robotics, on the robotics, GitHub page, there's a link to this which just has some interactive online demos, different types of examples. Yeah. So check that out.
HTN in the browser and do some visualizations of drop off. I haven't really explored this much,