Kickoff (2015 Spring NuPIC Hackathon)
Taylor, Matthew
Protocol and kickoff by Matt Taylor and Jeff Hawkins.
 So let's kick this off. My name is Matt Taylor. I am the open source community flag bearer for the NuPic open source project and I guess for Nemento's open source south in general. First off we're recording. There's a camera in the back. I've got some cameras set up in other places so where we're doing sessions, I try and record for people who can't be here. You might be interested in what we're doing. So you might be caught on tape, just know that. If you would like to take any photographs or videos, feel free. This is a very open, transparent thing we're doing here so if you want to use a hashtag or social media it is Hack NuPic. Let me say thanks to everyone who has helped me with this hackathon real quick. Terry Fry and Taylor, they're at the registration desk downstairs. They've been instrumental for logistics and planning. And thank you, Jeff. And the rest are NuPic engineers on the right here who will all be here willing to help you. Get NuPic installed. Brainstorm about what you can do with your data. We've all got root on the back of our shirts so if you see somebody with root, ask them questions and if they can't answer it they'll find someone who can. Also thanks Cornell Tech. Are there any Cornell Tech students here? Okay. Well thank you Cornell Tech anyway for letting us use their space. Jeff, would you come up and say a couple of things to our hackers to maybe motivate them to do? Fun interesting things. I just saw this logo the first time a few minutes ago. Does everyone know what that is? It's great. My wife says what's that picture I got? I think it's great. It was one of our community members created it. Anyway, thanks Matt. Thanks for coming here and we're excited to be here. This is our first hackathon outside of Silicon Valley. It's a new experience for us although we've done something before. Matt always runs a great show. We're going to try to make this really valuable for you. I just want to put my personal thanks for you coming here. What we're working on in NMETA and I assume what you want to work on here as well I think is extremely, extremely important work in not just the technology world but for humanity in many ways. We are at NMETA. We're trying to reverse engineer the neocortex. We're making good progress at that. We want to build machines that work in those principles. That's what NMETA is all about. It's the open source component of that. I think the implications of this work are going to be very, very profound in the future. We all get a chance to participate in that. It's not clear exactly how the future will play out and what time frames and so on. It's kind of a messy time right now but it's something that's really happening. It's starting to form and I think NMETA is at the center of many decades of machine learning based on core principles and machine intelligence in general. It's great to have other people to participate in that and I know some of you come from a long way to be here. We're going to try to make it all very successful for all of you. It's important work. I don't really want to delay things much at all. I just want to thank you again for coming and we're going to try to make this successful and not just this event but Newpick as a whole and Machine Intelligence as a whole and Brain Theory and all that stuff. Thanks for coming and I'll be around here myself for a part of the day and then I'll be back tomorrow. We can have a lot of conversations with people on the chat. I'm going to give this back over to Matt who's really in charge and running everything and he's going to make it work. Thanks Matt. Thank you Jeff and your vintage hackathon t-shirt. I got my new one back and I just need to change it. No that's good. This is a two years old. I like it. This is from my first hackathon. This is from the first one? Yeah. Well it's from the next one. Okay. Let me get through some boring protocol so we can get on to the real stuff. First off, resources for you. Everything that you need to know should be at numinta.org slash hack. If you go there it's got links to our schedule, to our code of conduct, to resources, to everything you need. Where to register your hack, all that stuff. We will register on a website called challengepost. Challengepost.com. If you're going to have a hack demo at the end of this you need to register there. More info is on numinta.org slash hack. All of these slides also, so all of these slides have a bunch of links in them that might be useful to you. You can also find them at numinta.org slash hack. So today's schedule, I am doing the kickoff at around 10.30 or as soon as I'm done with this we're going to have a couple of new pick engineers show off a few samples of things they've done with new pick using one. Chaitan has a demo using the Unity game engine and feeding data from that into new pick and Austin has something with the project tango. That should be interesting. 11.30 to 12.30 we will be available probably in, there's a big conference room in the back here. So we'll have new pick engineers to help you get new pick installed if you're having problems getting it compiled and running on your laptop. At 12.30 to 1.30 in the room back behind the projection screen we're going to have sort of a brainstorming session on music theory with new pick. The idea behind this is to try and come up with ideas on how to encode MIDI data into sparse representation so that new pick can start understanding music. I think this is really interesting. We have a special guest with us that's going to help us. He's our in-house music theorist Charlie Gillingham from county crows. He's also a computer scientist. We'll be running that session and hopefully pick his brain on some music theory and how we can encode MIDI data into sparse representations. I'm going to give a short session on the state of new pick at 2.30. Here, attend if you want. I'm going to go around like what the community is like, what's going on in the community, what kind of projects are happening, where we're going, what we've accomplished over the past year. So it's more focused on people interested in contributing or in the development and the progress of what's happening with the new pick project itself. We also have Connor Rusamano from OpenBCI who brought a bunch of hardware stuff. OpenBCI does kind of DIY EEG kits. They've got these little spider things that go on your head with contacts and you can get EEG information from your brain. We've had a couple hacks in the past of getting that data into new pick and using it to control things. So that could be interesting. If you're interested in that, please attend that session. That will probably be back in this room behind the projection screen. Tomorrow, I've got nothing on the agenda except demos at 4. So if you're going to demo, be ready at 4 o'clock. A couple things about the facility. Emergency exits. You can see a sign immediately upon exiting this room. It says exit. That's where you go. All of the doors in here don't open pulling or pushing. They're all sliding. This has been a problem in the past apparently. So just be aware of that. It doesn't open. It's because you're doing it wrong. Slide the door back and forth. I mentioned bathroom codes earlier. The men and women have different codes. They're on the door on the way out. You just take a look. Our Wi-Fi here is Google Cast. It's very fast and it's free and no password. If you're sleeping here, feel free. You might hopefully brought a pillow or sleeping bag or something because there's no couches in here. But if you want to talk about accommodations, you can come talk to me later and maybe I can give you some tips on where to crash. Okay, a little more protocol. You do need your badges. To get in the door tomorrow, keep this thing and bring it back tomorrow. On the back of it is the real important thing. This is what the security guards are going to want to see. So keep the badge. It also has a gift card for food. We usually provide food at these events because of logistics problems at this facility. We were not able to provide meals. So we compensated by getting a gift card. If you can't find food on the street, there's something wrong with you. There is tons of food around here. It's lots of places to eat. So please go use that gift card so you can go eat. There's trash cans all over the place. Please use them. I will be making sweeps every few hours or so to pick up trash. It's a lot easier if you just clean up after yourself. Be respectful of the facility. If you don't know if you should be touching something, come and ask me. I'll tell you if you can touch it or not. Also, please take a look at our code of conduct. It's a typical code of conduct. Don't do anything that's harassing or offensive to people. If you think it might be offensive, it probably is offensive. So don't come up with any hacks that might offend anyone. If you want specific details, read it here. Okay, for communicating at this event, obviously we're all in the same room, so that's one method. If you have technical questions that someone here can't answer or you find an answer and you want to share it with people, we have mailing lists. You can go to namenta.org. There's a general discussion list, which is probably your best bet. And there's one for development and there's one for theory. So feel free to sign up for that. And remember, for those of you taking pictures of the slides, that the slides are available at namenta.org. They have all these links in them. We also have a chat room on IRC, which I sometimes on. And we also have a Gitter chat room. If you have a GitHub account, that's the one that generally has the most traffic for chat. And spread the word with the hashtag hack new pic. I like it's a vanity thing. I love going through Twitter and seeing all the social media stuff. Okay, a few questions before I go on. Please raise your hand if you've ever been to a hackathon before. A lot of people. That's good. Okay. Who has actually compiled and built new pic on their laptop? This is a better indication of how busy we're going to be helping you. Okay, so maybe half. That's not too bad. Who has run any of the example code tutorials, stuff like that? Another maybe half. Who has like taken their own data or manipulated their own, some type of data of their own and passed it into new pic and done some of the experiments? Cool. Okay. Thank you. I appreciate it. So some more resources for you. For those of you who did not raise your hand, on namenta.org, there's a link to the wiki. And I would say this is probably the best bet for like getting started is go to the wiki. The wiki is laid out in a way to kind of drive you in a path that will help you build and install, give you tutorials on how to use new pic and then how to learn like what new pic is doing and the theory behind it and that sort of thing. So that's a good resource to start at. Here are some suggested links especially for you hackers. If you're new to new pic and you want to figure out what it's all about, we've got a couple of tutorials we call hot gym. It's about energy consumption over time. This is a really nice use case for new pic because it recognizes these patterns in energy consumption in buildings really well. So both of these have screen casts and sample code and me talking on and on and on about how to build it out. So hopefully that's understandable. There's one for prediction and there's one for anomaly detection which follows the prediction one. We also have a beginner's guide to new pic which is Scott Purdy gave at a previous hackathon which is a really nice dive into the details of encoders, how we encode scalar data for example into sparse representations. So that's a good resource and there's a link here to a bunch of different code examples on our wiki. CPU usage, spatial pooling, temporal pooling, classification, all that stuff. So we have code examples of all this stuff. So if you see something that sparks an interest and you think, hey my hack might fit in that, start with that and see if you can run those examples. We also have a bunch of old demos from old hackathons. From old hackathons I think 14 demos from the last hackathon, something like that from the one before. So if you want some ideas of what other people have tried at old other hackathons, go there. If you do not want to use new pic or you're not a python person, you don't want to write python code. There are other htm implementations you could potentially use. We have htm java which is a port of the new pic python htm algorithms by David Ray. He's right back there. Yeah, there you go. Embarrassing. Okay. And we have comport.exe by Felix Andrews, which is a closure implementation of htm. It doesn't closely follow the r python implementation of the htm algorithms like htm java does, but it's still an htm. Okay. A little guidance about your demos. The most important thing I think is if you have an idea about something, think about your data. Think about do you have a lot of data? Is it temporal? The biggest question is, is the data you're thinking about using for a hack temporal, does it have a time component? Because if it doesn't, if it's just spatial, you might not have much success. A key thing about htm hierarchical temporal memory is a temporal component of that. So make sure you have a temporal component here. Data or at least you can somehow give it a temporal component. Is there enough data? You're not going to learn many patterns from, you know, annual data points unless you have lots of data, lots of annual data. And does it have patterns? Another question to ask yourself. Yeah, your question? Well, that depends. Probably not a question for this presentation, but I can talk afterwards about it. It depends on the patterns and the data, basically. A thousand data points in minimum type of thing? Yeah, that's good. You're going to need at least a thousand data points, probably, no matter what. Okay, a great way to learn htm theory is to build an htm in your favorite programming language. That's great. I love it when people do this, but it's not a good hack because you're just standing here telling people about what they already know and here's my htm in whatever. So please don't make an htm and do that. You can do it while you're here at the hack con, but don't make it a hack demo. We'd be happy to help you do that, but it's not a good demo. Okay, so let's talk quickly about anyone having any ideas. Oh, we're right on schedule, so good. I talked a little about music, so I'm really interested in encoding MIDI data somehow so that NUPIC can understand patterns and music. This is an example that Jeff uses a lot when he's talking about the brain, about htm, about the hierarchy involved. We've actually had demos in the past, our first hackathon actually, someone had encoded a pomp and circumstance, the graduation song, into scalar values and passed it into NUPIC and had it predict the entire song. After about, I think, 22, 23 repetitions, it predicted the whole song completely, like perfectly. So that was impressive to me, but it wasn't a true encoding because they hacked it into scalar data. So the real challenge here is how to get MIDI data or any type of musical representation into a sparse format that encodes more information than just the pitch of the note, right? Because there's more to music than just pitches of notes and durations of notes. So that's something I'm interested in. I don't know if I'll have time to work on that, but if you're interested in that, come to the music theory session we're having today because that's what we're going to talk about. But I want to hear about your ideas. Who has an idea of something they want to do today? I would really love to hear about it because you might motivate other people to help you implement your idea. A part of this is building teams. So I know someone has ideas. Please raise your hand if you have an idea and you're willing to share it. Okay, you're close. Come on, come on. My name is Tom Carzwell and my idea was to use NUPIC for spectroscopy and element recognition. When you look at it in absorption lines, they look suspiciously close to sparse data representations for certain applications. What is that? Basically you shoot light at something and then whatever light doesn't bounce back tells you what's in the object that you're shooting the light at or the light is passing from, depending on how you want to do it. But I can't think of a good way to make that temporal because it doesn't seem like something that would depend on how long you shoot the light at it. In most instance, maybe. Maybe. Okay, that's interesting. So if there's a video component to it in any way and you can get SDRs over time, that may be something that you could predict. As long as you could somehow reconstruct those into something actionable. Can that be explained to them? Yeah. Can that be explained to them? Yeah. And that's what I'm picturing this morning. I think that's more, that's more like a motion capture. This is more a sad thing. By the way, we do have a Connect in-house. One of our engineers brought Connect. So that's something we've been thinking about doing too. Do you want to just comment on this stuff? Yes. I do. I mean, it depends on what kind of data you get out of it. You know, Cortical IO did these word SDRs and they actually get really interesting results without even the temporal components just using the sort of semantic encodings of SDRs. You might be able to do something like that. At least with the words though, we're able to then say, okay, now we can do it in sequence and sentences. You might not be able to do that. But it's still, even as the SDR component of the theory might be useful for you in some way. I just don't know enough about your data. But there are some situations you can do, a limited set of what we do with spatial patterns, pure spatial patterns. So I just want to throw that out. Some of the thing now. Yeah. As long as the sparse representations coming out of this actually have semantic meaning, which they should. Sure. My name is Chiraga. I came to the last Hackathon at night. Then again, I chance to contribute a lot. So I've been reading about Newbick and the hack that I'm thinking about. I'm not even sure if this is a good hack. But I recently read a paper, a Stanford paper, which was trying to predict outcome of baseball games. And they had compared about six or seven AI, so-called AI algorithms and predicting what the outcome of baseball games is. So there's about 2400 games in baseball season and it would train the data over some period and it would try to predict outcomes for each team while we're game. So we were thinking about feeding that data that's really available and see if you can get any easy predictions just by feeding the data. Interesting. It might be even better if you can get events within each game as well. Hi. My name is Sean. This is a pretty vague idea, but I'm interested in environmentalism. And so can we take some amount of data and use that to do something like, I don't know, turn off lights more efficiently or decrease and increase the temperature of rooms based on if people are around. So if anyone is interested in something that's very vague but lots of ideas with that. We have to get a fair amount of work in that space. We're talking about it. It's doable. Definitely. It works. Yeah. Maybe something with even New York data that's around that I haven't looked into. Yeah, it is hard part. Sounds like an IOT application to me. Yeah. It's very like a building. If everything's instrumented in the building, you should be able to predict when people are going to be in what rooms and when things are going to be used. Yeah. Hi. My name is Ashish. I'm working for a project for New York Presbyterian Hospital this summer on patient onboarding and also at home care post discharge. So I'm trying to incorporate some kind of decision analysis before they get to the hospital. So what makes people think which hospitals to choose and what is the decision making. I'm not sure how that translates into this project, but hopefully there are some applications that somebody can help me with. Do you have data? My name is Dave Petrillo. I had an idea for using HTMs to look at automotive data. So can we add just a few extra sensors to a car and get a better idea of what's wrong with the car for consumer applications or get a better idea of how the driver's doing for like performance driving applications. It seems like there should be more information in the data than just okay this sensor is too hot and so then the engine is too hot, but instead the rumble came from the front right corner of the car so the way that we sense it means that wheel bearing is bad and could help make diagnosing cars for regular applications easier and faster. I don't have any data, but we could make some up or something. That definitely sounds like an anomaly detection application. It looks pretty good anomaly detection. I only have time for a few more so keep it short. Alright, my name is Patrick Ryan. I am interested in trying to model student data. I have some generated student data based on students that are good and bad and I'm thinking about making it 10-4 by encoding the student and the question and maybe taxonomies about that question and then followed by their answer and whether that was correct. It's just a simple one-two sequence but it might work. I'm not sure that's temporal enough. So think about the temporal aspect. Hi, I'm Steve Double Day and for my dissertation I'm trying to model interaction between the hippocampus and the cortex and there's a particular rat experiment that I'd love to be able to do. I started to try to put into new pic but only gotten a little way in and that has to do with a rat will project out of their hippocampus a trajectory about where they're going to move and if you were to feed in the trajectories that get generated by a rat or a simulated set which I've got out of the test framework, the question is, can over a period of time can our algorithm predict which trajectory is going to result in a reward? That's what I'm tackling. Okay, I'm going to do you. Okay, four more. I've already picked them. Hey, my name's Scott. So I consider using anomaly detection for fraud detection in the mobile space. So right now mobile advertisers, publishers specifically are left to sort of set up their own metrics using mostly thresholds. If you can use simple device identifiers like the IDFA, the country, maybe their locale in the operating system, you could potentially do some useful things in detecting the types of traffic and the determined anomalous things and present that to a team of security people working for an app. I think, yeah, right now in the mobile space, like I said, a lot of publishers are having to do this on their own and I think it would be useful to consolidate that and create some service around that and testing that out and trying that would be a fun thing to do. Definitely a lot of people are interested in fraud detection. It goes along with anomaly detection. Road behavior detection is another thing we've worked on. Hi, my name is Marcin. So I saw a couple of days ago there was a quite interesting analysis of the LSTM unit. There was some guy posted this on my, from Stanford to guess. And I figured out, well, let's try to create a little bot where you just feed the sentence, say hi, and trade the network and see what the network is going to come up with. That previous person had a bunch of experience as well as Wikipedia as well as kernel sourcing to a pretty wide LSTM unit and got some very interesting answers. So you could have, I guess, if the NuPic network is deep enough, you can have a very sensible conversation. So let's see. I'm Frank Rose. As part of trying to understand the NuPic algorithm, I was interested in doing a sensitivity assay of the algorithm itself using generated data, trained on a classification problem, and then perturbed that data and see, okay, where does the classification start breaking down if you add randomness or drift or something to the input path. So maybe, you know, maybe even be able to compare the performance of the Python version and HTM and Cortex. Hi, I'm Paul. I work for this company that makes, we use this brain sensing headband, it's an EEG device, and we want to use it for something pretty simple, HTM for something pretty simple of just recognizing muscle movement, like blinks, winks, you know, forehead, crunches, or things like that, and maybe later use it for brain sensing. My name is Vladimir. I'm working with time series, a lot of financial and economic time series, and my idea was to test the performance of HTM in kind of non-stationary environments, so I would like to generate a kind of theoretical time series with time change and patterns, and see how HTM can just recognize patterns in kind of adaptive non-stationary environment. Hi, I'm Perga Burnan. I want to hack the theory, basically. So if people are interested in talking about a couple of ideas that are kind of extensions of HTM, and basically going to be interested in brainstorming those things over the weekend, so I'm also really interested in the non-stationarity thing as well. Hey, Perga. Hey, how's it going, everyone? I'm Connor from OpenBCI. I'm interested in taking an OpenBCI device and plugging, or connecting 16 channels to the kind of lower forearm, and seeing if we can use NewPick to classify individual finger movement, because right now I've got a system rigged up where I've got a single EMG input and I can make a hand flex in like an analog motion, but I'm really interested to see if we can identify individual fingers. Okay. Anybody else? My idea, first of all, I was reading about the complex adaptive systems from a different book, and it seems like it used genetic algorithms in order to have to solve the problems in its nature, but I want to embed a brain into it so that NewPick go into that adaptive system to make it better. There are some papers that actually combined the two, but it was only one paper that I found. So I'd like to be able to do that within Mathematica, because Mathematica can actually use, what do you call it, the video card. The GPU. The GPU, in order to compute things much faster, and it's easier to code. So I was trying to actually implement the algorithms in Mathematica. That would be interesting. I don't think that would be, I would accept that demo. As an HTM implementation within Mathematica, that's interesting. Just a quick question. There's a lot of interest in getting NewPick to run on the GPU, and I think I've seen other HTM-like implementations running on the GPU, but we currently don't. So I'm eventually, I'm sure, we will. Hello. We are Cortical.io, and we are trying to combine your HTM and our retina API to detect anomalies and or patterns on text data or text strings. And if you guys are working with text data, feel free to try our API, Cortical.io. For free? Yes, for free. So if you have the hackathon and you want to use Cortical.io's API, which is really cool, talk to these guys and I'll give you an API key. Hi, my name is John Yeh. I'm working on two projects. One is an acoustic project that by listening to the noise of the falset sound, we want to be able to detect which falset and how it works and where it is. And the other one, I'm starting to kick around this idea, is actually using the switch matrix on the FPGA device. Instead of hard coding the actual FPGA, we want to be able to do the weighted learning as part of the switch matrix on the FPGA device. So that's an unexplored layer of an FPGA device. First hack mentioned about audio analysis. Not MIDI, but audio. So there are others here, and Richard is interested in audio waveform analysis, that sort of thing. And I've done some experiments with that too, largely unsuccessfully, but that's my fault. Anybody else? So my hack that I'm planning to work on this weekend is called Market Patterns. And the idea is to, well, the assumption is that there are patterns in the stock market that we can learn to detect. There are lots of traders that learn these patterns by looking at charts. So I want to see, can we encode the data in such a way that NuPik can see patterns and just predicts, you know, ups and downs, not prices, just directions. We've been nominally successful with stock volume prediction, so I know that's possible. Hi, my name is Ji-young. I'm interested in analyzing patterns, like intellectual properties. So my idea is more of classification of patterns, so how valuable your patterns are. It's kind of a vague idea, but that's what I'm interested in. What's the data? So it's like a pattern claims. So you can, you know, patterns. Got it. Okay. Matt, can I just say where does it... Please. You know, I think it might be wise for, you know, especially if you're a NuPik, if you've got an idea and you want to run, and you think you're creating some data and running through or you have some data, try to find one of the NuMenta employees and just discuss it a little bit. It's very easy to make incorrect assumptions about how the data should be, the sampling rates, and what is temporal data. And so if you have even any thoughts at all, it's really useful. More people trip up on the data, I think, than anything else. So it's worth grabbing one of us and just say, hey, I suppose I think you're doing. Give me some advice. You're running circles for a long time, and then we'll come tell you all your data's in the wrong format. Okay. So that concludes the kickoff. Let's do like a 15-minute break, and I'm going to bring Chaden and Austin up here to talk about some of their hack samples with Unity, Game Engine, and Project Tango. Thank you. Thank you all for coming. Music
 So let's kick this off. My name is Matt Taylor. I am the open source community flag bearer  for the NuPic open source project and I guess for Nemento's open source south in general.  First off we're recording. There's a camera in the back. I've got some cameras set up  in other places so where we're doing sessions, I try and record for people who can't be here.  You might be interested in what we're doing. So you might be caught on tape, just know  that. If you would like to take any photographs or videos, feel free. This is a very open,  transparent thing we're doing here so if you want to use a hashtag or social media it is  Hack NuPic. Let me say thanks to everyone who has helped me with this hackathon real  quick. Terry Fry and Taylor, they're at the registration desk downstairs. They've been  instrumental for logistics and planning. And thank you, Jeff. And the rest are NuPic  engineers on the right here who will all be here willing to help you. Get NuPic installed.  Brainstorm about what you can do with your data. We've all got root on the back of our  shirts so if you see somebody with root, ask them questions and if they can't answer it  they'll find someone who can. Also thanks Cornell Tech. Are there any Cornell Tech students  here? Okay. Well thank you Cornell Tech anyway for letting us use their space. Jeff, would  you come up and say a couple of things to our hackers to maybe motivate them to do?  Fun interesting things. I just saw this logo the first time a few minutes ago. Does everyone  know what that is? It's great. My wife says what's that picture I got? I think it's great.  It was one of our community members created it. Anyway, thanks Matt. Thanks for coming  here and we're excited to be here. This is our first hackathon outside of Silicon Valley.  It's a new experience for us although we've done something before. Matt always runs a  great show. We're going to try to make this really valuable for you. I just want to put  my personal thanks for you coming here. What we're working on in NMETA and I assume what  you want to work on here as well I think is extremely, extremely important work in not  just the technology world but for humanity in many ways. We are at NMETA. We're trying  to reverse engineer the neocortex. We're making good progress at that. We want to build machines  that work in those principles. That's what NMETA is all about. It's the open source component  of that. I think the implications of this work are going to be very, very profound in  the future. We all get a chance to participate in that. It's not clear exactly how the future  will play out and what time frames and so on. It's kind of a messy time right now but  it's something that's really happening. It's starting to form and I think NMETA is at the  center of many decades of machine learning based on core principles and machine intelligence  in general. It's great to have other people to participate in that and I know some of  you come from a long way to be here. We're going to try to make it all very successful  for all of you. It's important work. I don't really want to delay things much at all. I  just want to thank you again for coming and we're going to try to make this successful  and not just this event but Newpick as a whole and Machine Intelligence as a whole and Brain  Theory and all that stuff. Thanks for coming and I'll be around here myself for a part  of the day and then I'll be back tomorrow. We can have a lot of conversations with people  on the chat. I'm going to give this back over to Matt who's really in charge and running  everything and he's going to make it work. Thanks Matt.  Thank you Jeff and your vintage hackathon t-shirt.  I got my new one back and I just need to change it.  No that's good. This is a two years old. I like it. This is from my first hackathon.  This is from the first one? Yeah. Well it's from the next one.  Okay. Let me get through some boring protocol so we can get on to the real stuff. First  off, resources for you. Everything that you need to know should be at numinta.org slash  hack. If you go there it's got links to our schedule, to our code of conduct, to resources,  to everything you need. Where to register your hack, all that stuff.  We will register on a website called challengepost. Challengepost.com. If you're going to have  a hack demo at the end of this you need to register there. More info is on numinta.org  slash hack. All of these slides also, so all of these slides have a bunch of links in them  that might be useful to you. You can also find them at numinta.org slash hack.  So today's schedule, I am doing the kickoff at around 10.30 or as soon as I'm done with  this we're going to have a couple of new pick engineers show off a few samples of things  they've done with new pick using one. Chaitan has a demo using the Unity game engine and  feeding data from that into new pick and Austin has something with the project tango. That  should be interesting. 11.30 to 12.30 we will be available probably in, there's a big conference  room in the back here. So we'll have new pick engineers to help you get new pick installed  if you're having problems getting it compiled and running on your laptop.  At 12.30 to 1.30 in the room back behind the projection screen we're going to have sort  of a brainstorming session on music theory with new pick. The idea behind this is to  try and come up with ideas on how to encode MIDI data into sparse representation so that  new pick can start understanding music. I think this is really interesting. We have  a special guest with us that's going to help us. He's our in-house music theorist Charlie  Gillingham from county crows. He's also a computer scientist. We'll be running that session  and hopefully pick his brain on some music theory and how we can encode MIDI data into  sparse representations. I'm going to give a short session on the state of new pick at  2.30. Here, attend if you want. I'm going to go around like what the community is like,  what's going on in the community, what kind of projects are happening, where we're going,  what we've accomplished over the past year. So it's more focused on people interested  in contributing or in the development and the progress of what's happening with the  new pick project itself. We also have Connor Rusamano from OpenBCI who brought a bunch  of hardware stuff. OpenBCI does kind of DIY EEG kits. They've got these little spider  things that go on your head with contacts and you can get EEG information from your  brain. We've had a couple hacks in the past of getting that data into new pick and using  it to control things. So that could be interesting. If you're interested in that, please attend  that session. That will probably be back in this room behind the projection screen.  Tomorrow, I've got nothing on the agenda except demos at 4. So if you're going to demo, be  ready at 4 o'clock. A couple things about the facility. Emergency exits. You can see  a sign immediately upon exiting this room. It says exit. That's where you go. All of  the doors in here don't open pulling or pushing. They're all sliding. This has been a problem  in the past apparently. So just be aware of that. It doesn't open. It's because you're  doing it wrong. Slide the door back and forth. I mentioned bathroom codes earlier. The men  and women have different codes. They're on the door on the way out. You just take a look.  Our Wi-Fi here is Google Cast. It's very fast and it's free and no password. If you're sleeping  here, feel free. You might hopefully brought a pillow or sleeping bag or something because  there's no couches in here. But if you want to talk about accommodations, you can come  talk to me later and maybe I can give you some tips on where to crash.  Okay, a little more protocol. You do need your badges. To get in the door tomorrow, keep  this thing and bring it back tomorrow. On the back of it is the real important thing.  This is what the security guards are going to want to see. So keep the badge. It also  has a gift card for food. We usually provide food at these events because of logistics  problems at this facility. We were not able to provide meals. So we compensated by getting  a gift card. If you can't find food on the street, there's something wrong with you.  There is tons of food around here. It's lots of places to eat. So please go use that gift  card so you can go eat. There's trash cans all over the place. Please use them. I will  be making sweeps every few hours or so to pick up trash. It's a lot easier if you just  clean up after yourself. Be respectful of the facility. If you don't know if you should  be touching something, come and ask me. I'll tell you if you can touch it or not. Also,  please take a look at our code of conduct. It's a typical code of conduct. Don't do anything  that's harassing or offensive to people. If you think it might be offensive, it probably  is offensive. So don't come up with any hacks that might offend anyone. If you want specific  details, read it here. Okay, for communicating at this event, obviously we're all in the  same room, so that's one method. If you have technical questions that someone here can't  answer or you find an answer and you want to share it with people, we have mailing lists.  You can go to namenta.org. There's a general discussion list, which is probably your best  bet. And there's one for development and there's one for theory. So feel free to sign up for  that. And remember, for those of you taking pictures of the slides, that the slides are  available at namenta.org. They have all these links in them. We also have a chat room on  IRC, which I sometimes on. And we also have a Gitter chat room. If you have a GitHub  account, that's the one that generally has the most traffic for chat. And spread the  word with the hashtag hack new pic. I like it's a vanity thing. I love going through  Twitter and seeing all the social media stuff. Okay, a few questions before I go on. Please  raise your hand if you've ever been to a hackathon before. A lot of people. That's good. Okay.  Who has actually compiled and built new pic on their laptop? This is a better indication  of how busy we're going to be helping you. Okay, so maybe half. That's not too bad. Who  has run any of the example code tutorials, stuff like that? Another maybe half. Who has  like taken their own data or manipulated their own, some type of data of their own and passed  it into new pic and done some of the experiments? Cool. Okay. Thank you. I appreciate it. So  some more resources for you. For those of you who did not raise your hand, on namenta.org,  there's a link to the wiki. And I would say this is probably the best bet for like getting  started is go to the wiki. The wiki is laid out in a way to kind of drive you in a path  that will help you build and install, give you tutorials on how to use new pic and then  how to learn like what new pic is doing and the theory behind it and that sort of thing.  So that's a good resource to start at. Here are some suggested links especially for you  hackers. If you're new to new pic and you want to figure out what it's all about, we've  got a couple of tutorials we call hot gym. It's about energy consumption over time.  This is a really nice use case for new pic because it recognizes these patterns in energy  consumption in buildings really well. So both of these have screen casts and sample code  and me talking on and on and on about how to build it out. So hopefully that's understandable.  There's one for prediction and there's one for anomaly detection which follows the prediction  one. We also have a beginner's guide to new pic which is Scott Purdy gave at a previous  hackathon which is a really nice dive into the details of encoders, how we encode scalar  data for example into sparse representations. So that's a good resource and there's a link  here to a bunch of different code examples on our wiki. CPU usage, spatial pooling, temporal  pooling, classification, all that stuff. So we have code examples of all this stuff.  So if you see something that sparks an interest and you think, hey my hack might fit in that,  start with that and see if you can run those examples. We also have a bunch of old demos  from old hackathons. From old hackathons I think 14 demos from the last hackathon,  something like that from the one before. So if you want some ideas of what other people  have tried at old other hackathons, go there. If you do not want to use new pic or you're  not a python person, you don't want to write python code. There are other htm implementations  you could potentially use. We have htm java which is a port of the new pic python htm  algorithms by David Ray. He's right back there. Yeah, there you go. Embarrassing. Okay. And  we have comport.exe by Felix Andrews, which is a closure implementation of htm. It doesn't  closely follow the r python implementation of the htm algorithms like htm java does,  but it's still an htm. Okay. A little guidance about your demos. The most important thing  I think is if you have an idea about something, think about your data. Think about do you  have a lot of data? Is it temporal? The biggest question is, is the data you're thinking about  using for a hack temporal, does it have a time component? Because if it doesn't, if  it's just spatial, you might not have much success. A key thing about htm hierarchical  temporal memory is a temporal component of that. So make sure you have a temporal component  here. Data or at least you can somehow give it a temporal component. Is there enough data?  You're not going to learn many patterns from, you know, annual data points unless you have  lots of data, lots of annual data. And does it have patterns? Another question to ask  yourself. Yeah, your question? Well, that depends. Probably not a question for this presentation,  but I can talk afterwards about it. It depends on the patterns and the data, basically.  A thousand data points in minimum type of thing? Yeah, that's good. You're going to need at least  a thousand data points, probably, no matter what. Okay, a great way to learn htm theory  is to build an htm in your favorite programming language. That's great. I love it when people  do this, but it's not a good hack because you're just standing here telling people about what  they already know and here's my htm in whatever. So please don't make an htm and do that. You  can do it while you're here at the hack con, but don't make it a hack demo. We'd be happy  to help you do that, but it's not a good demo. Okay, so let's talk quickly about anyone  having any ideas. Oh, we're right on schedule, so good. I talked a little about music, so  I'm really interested in encoding MIDI data somehow so that NUPIC can understand patterns  and music. This is an example that Jeff uses a lot when he's talking about the brain, about  htm, about the hierarchy involved. We've actually had demos in the past, our first hackathon  actually, someone had encoded a pomp and circumstance, the graduation song, into scalar  values and passed it into NUPIC and had it predict the entire song. After about, I think,  22, 23 repetitions, it predicted the whole song completely, like perfectly. So that was  impressive to me, but it wasn't a true encoding because they hacked it into scalar data. So  the real challenge here is how to get MIDI data or any type of musical representation  into a sparse format that encodes more information than just the pitch of the note, right? Because  there's more to music than just pitches of notes and durations of notes. So that's something  I'm interested in. I don't know if I'll have time to work on that, but if you're interested  in that, come to the music theory session we're having today because that's what we're  going to talk about. But I want to hear about your ideas. Who has an idea of something they  want to do today? I would really love to hear about it because you might motivate other  people to help you implement your idea. A part of this is building teams. So I know someone  has ideas. Please raise your hand if you have an idea and you're willing to share it. Okay,  you're close. Come on, come on. My name is Tom Carzwell and my idea was to use NUPIC  for spectroscopy and element recognition. When you look at it in absorption lines, they  look suspiciously close to sparse data representations for certain applications. What is that?  Basically you shoot light at something and then whatever light doesn't bounce back tells  you what's in the object that you're shooting the light at or the light is passing from,  depending on how you want to do it. But I can't think of a good way to make that temporal  because it doesn't seem like something that would depend on how long you shoot the light  at it. In most instance, maybe. Maybe. Okay, that's interesting. So if there's a video  component to it in any way and you can get SDRs over time, that may be something that  you could predict. As long as you could somehow reconstruct those into something actionable.  Can that be explained to them? Yeah. Can that be explained to them? Yeah. And that's  what I'm picturing this morning. I think that's more, that's more like a motion capture. This  is more a sad thing. By the way, we do have a Connect in-house. One of our engineers brought  Connect. So that's something we've been thinking about doing too. Do you want to just comment  on this stuff? Yes. I do. I mean, it depends on what kind of data you get out of it. You  know, Cortical IO did these word SDRs and they actually get really interesting results  without even the temporal components just using the sort of semantic encodings of SDRs. You  might be able to do something like that. At least with the words though, we're able to  then say, okay, now we can do it in sequence and sentences. You might not be able to do  that. But it's still, even as the SDR component of the theory might be useful for you in some  way. I just don't know enough about your data. But there are some situations you can do,  a limited set of what we do with spatial patterns, pure spatial patterns. So I just want to throw  that out. Some of the thing now. Yeah. As long as the sparse representations coming out of  this actually have semantic meaning, which they should.  Sure. My name is Chiraga. I came to the last Hackathon at night. Then again, I chance to  contribute a lot. So I've been reading about Newbick and the hack that I'm thinking about.  I'm not even sure if this is a good hack. But I recently read a paper, a Stanford paper,  which was trying to predict outcome of baseball games. And they had compared about six or seven  AI, so-called AI algorithms and predicting what the outcome of baseball games is. So  there's about 2400 games in baseball season and it would train the data over some period  and it would try to predict outcomes for each team while we're game. So we were thinking  about feeding that data that's really available and see if you can get any easy predictions  just by feeding the data.  Interesting. It might be even better if you can get events within each game as well.  Hi. My name is Sean. This is a pretty vague idea, but I'm interested in environmentalism.  And so can we take some amount of data and use that to do something like, I don't know,  turn off lights more efficiently or decrease and increase the temperature of rooms based  on if people are around. So if anyone is interested in something that's very vague but lots of  ideas with that.  We have to get a fair amount of work in that space. We're talking about it. It's doable.  Definitely. It works.  Yeah. Maybe something with even New York data that's around that I haven't looked into.  Yeah, it is hard part.  Sounds like an IOT application to me. Yeah. It's very like a building. If everything's  instrumented in the building, you should be able to predict when people are going to be  in what rooms and when things are going to be used.  Yeah.  Hi. My name is Ashish. I'm working for a project for New York Presbyterian Hospital this summer  on patient onboarding and also at home care post discharge. So I'm trying to incorporate  some kind of decision analysis before they get to the hospital. So what makes people  think which hospitals to choose and what is the decision making. I'm not sure how that  translates into this project, but hopefully there are some applications that somebody  can help me with.  Do you have data?  My name is Dave Petrillo. I had an idea for using HTMs to look at automotive data. So  can we add just a few extra sensors to a car and get a better idea of what's wrong with  the car for consumer applications or get a better idea of how the driver's doing for  like performance driving applications. It seems like there should be more information  in the data than just okay this sensor is too hot and so then the engine is too hot,  but instead the rumble came from the front right corner of the car so the way that we  sense it means that wheel bearing is bad and could help make diagnosing cars for regular  applications easier and faster. I don't have any data, but we could make some up or something.  That definitely sounds like an anomaly detection application. It looks pretty good anomaly detection.  I only have time for a few more so keep it short.  Alright, my name is Patrick Ryan. I am interested in trying to model student data. I have some  generated student data based on students that are good and bad and I'm thinking about making  it 10-4 by encoding the student and the question and maybe taxonomies about that question and  then followed by their answer and whether that was correct. It's just a simple one-two  sequence but it might work.  I'm not sure that's temporal enough. So think about the temporal aspect.  Hi, I'm Steve Double Day and for my dissertation I'm trying to model interaction between the  hippocampus and the cortex and there's a particular rat experiment that I'd love to be able to  do. I started to try to put into new pic but only gotten a little way in and that has to  do with a rat will project out of their hippocampus a trajectory about where they're going to  move and if you were to feed in the trajectories that get generated by a rat or a simulated  set which I've got out of the test framework, the question is, can over a period of time  can our algorithm predict which trajectory is going to result in a reward? That's what  I'm tackling.  Okay, I'm going to do you.  Okay, four more. I've already picked them.  Hey, my name's Scott. So I consider using anomaly detection for fraud detection in the  mobile space. So right now mobile advertisers, publishers specifically are left to sort of  set up their own metrics using mostly thresholds. If you can use simple device identifiers like  the IDFA, the country, maybe their locale in the operating system, you could potentially do some  useful things in detecting the types of traffic and the determined anomalous things and present  that to a team of security people working for an app. I think, yeah, right now in the mobile  space, like I said, a lot of publishers are having to do this on their own and I think it would be  useful to consolidate that and create some service around that and testing that out and  trying that would be a fun thing to do.  Definitely a lot of people are interested in fraud detection. It goes along with anomaly  detection. Road behavior detection is another thing we've worked on.  Hi, my name is Marcin. So I saw a couple of days ago there was a quite interesting analysis of the  LSTM unit. There was some guy posted this on my, from Stanford to guess. And I figured out, well,  let's try to create a little bot where you just feed the sentence, say hi, and trade the network  and see what the network is going to come up with. That previous person had a bunch of  experience as well as Wikipedia as well as kernel sourcing to a pretty wide LSTM unit  and got some very interesting answers. So you could have, I guess, if the NuPic network is  deep enough, you can have a very sensible conversation. So let's see.  I'm Frank Rose. As part of trying to understand the NuPic algorithm, I was interested in doing  a sensitivity assay of the algorithm itself using generated data, trained on a classification  problem, and then perturbed that data and see, okay, where does the classification start breaking  down if you add randomness or drift or something to the input path. So maybe, you know, maybe  even be able to compare the performance of the Python version and HTM and Cortex.  Hi, I'm Paul. I work for this company that makes, we use this brain sensing headband,  it's an EEG device, and we want to use it for something pretty simple, HTM for something  pretty simple of just recognizing muscle movement, like blinks, winks, you know, forehead, crunches,  or things like that, and maybe later use it for brain sensing.  My name is Vladimir. I'm working with time series, a lot of financial and economic time  series, and my idea was to test the performance of HTM in kind of non-stationary environments,  so I would like to generate a kind of theoretical time series with time change and  patterns, and see how HTM can just recognize patterns in kind of adaptive non-stationary environment.  Hi, I'm Perga Burnan. I want to hack the theory, basically. So if people are interested in talking  about a couple of ideas that are kind of extensions of HTM, and basically going to be  interested in brainstorming those things over the weekend, so I'm also really interested in the non-stationarity thing as well.  Hey, Perga.  Hey, how's it going, everyone? I'm Connor from OpenBCI. I'm interested in taking an OpenBCI device  and plugging, or connecting 16 channels to the kind of lower forearm, and seeing if we can use NewPick  to classify individual finger movement, because right now I've got a system rigged up where I've got a single EMG input  and I can make a hand flex in like an analog motion, but I'm really interested to see if we can identify individual fingers.  Okay. Anybody else?  My idea, first of all, I was reading about the complex adaptive systems from a different book,  and it seems like it used genetic algorithms in order to have to solve the problems in its nature,  but I want to embed a brain into it so that NewPick go into that adaptive system to make it better.  There are some papers that actually combined the two, but it was only one paper that I found.  So I'd like to be able to do that within Mathematica, because Mathematica can actually use, what do you call it, the video card.  The GPU.  The GPU, in order to compute things much faster, and it's easier to code.  So I was trying to actually implement the algorithms in Mathematica.  That would be interesting. I don't think that would be, I would accept that demo.  As an HTM implementation within Mathematica, that's interesting.  Just a quick question.  There's a lot of interest in getting NewPick to run on the GPU, and I think I've seen other HTM-like implementations running on the GPU, but we currently don't.  So I'm eventually, I'm sure, we will.  Hello. We are Cortical.io, and we are trying to combine your HTM and our retina API to detect anomalies and or patterns on text data or text strings.  And if you guys are working with text data, feel free to try our API, Cortical.io.  For free?  Yes, for free.  So if you have the hackathon and you want to use Cortical.io's API, which is really cool, talk to these guys and I'll give you an API key.  Hi, my name is John Yeh. I'm working on two projects.  One is an acoustic project that by listening to the noise of the falset sound, we want to be able to detect which falset and how it works and where it is.  And the other one, I'm starting to kick around this idea, is actually using the switch matrix on the FPGA device.  Instead of hard coding the actual FPGA, we want to be able to do the weighted learning as part of the switch matrix on the FPGA device.  So that's an unexplored layer of an FPGA device.  First hack mentioned about audio analysis. Not MIDI, but audio.  So there are others here, and Richard is interested in audio waveform analysis, that sort of thing.  And I've done some experiments with that too, largely unsuccessfully, but that's my fault.  Anybody else?  So my hack that I'm planning to work on this weekend is called Market Patterns.  And the idea is to, well, the assumption is that there are patterns in the stock market that we can learn to detect.  There are lots of traders that learn these patterns by looking at charts.  So I want to see, can we encode the data in such a way that NuPik can see patterns and just predicts, you know, ups and downs, not prices, just directions.  We've been nominally successful with stock volume prediction, so I know that's possible.  Hi, my name is Ji-young. I'm interested in analyzing patterns, like intellectual properties.  So my idea is more of classification of patterns, so how valuable your patterns are.  It's kind of a vague idea, but that's what I'm interested in.  What's the data?  So it's like a pattern claims.  So you can, you know, patterns.  Got it.  Okay.  Matt, can I just say where does it...  Please.  You know, I think it might be wise for, you know, especially if you're a NuPik, if you've got an idea and you want to run,  and you think you're creating some data and running through or you have some data,  try to find one of the NuMenta employees and just discuss it a little bit.  It's very easy to make incorrect assumptions about how the data should be, the sampling rates, and what is temporal data.  And so if you have even any thoughts at all, it's really useful.  More people trip up on the data, I think, than anything else.  So it's worth grabbing one of us and just say, hey, I suppose I think you're doing.  Give me some advice.  You're running circles for a long time, and then we'll come tell you all your data's in the wrong format.  Okay.  So that concludes the kickoff.  Let's do like a 15-minute break, and I'm going to bring Chaden and Austin up here to talk about some of their hack samples  with Unity, Game Engine, and Project Tango.  Thank you.  Thank you all for coming.  Music
Kickoff__2015_Spring_NuPIC_Hackathon__0_227.png Kickoff__2015_Spring_NuPIC_Hackathon__228_1871.png Kickoff__2015_Spring_NuPIC_Hackathon__1872_5867.png Kickoff__2015_Spring_NuPIC_Hackathon__5868_6167.png Kickoff__2015_Spring_NuPIC_Hackathon__6168_23255.png Kickoff__2015_Spring_NuPIC_Hackathon__23256_23315.png Kickoff__2015_Spring_NuPIC_Hackathon__23316_29651.png Kickoff__2015_Spring_NuPIC_Hackathon__29652_29699.png Kickoff__2015_Spring_NuPIC_Hackathon__29700_30515.png Kickoff__2015_Spring_NuPIC_Hackathon__30516_30671.png Kickoff__2015_Spring_NuPIC_Hackathon__30672_30719.png Kickoff__2015_Spring_NuPIC_Hackathon__30720_33323.png Kickoff__2015_Spring_NuPIC_Hackathon__33324_34211.png Kickoff__2015_Spring_NuPIC_Hackathon__34212_35579.png Kickoff__2015_Spring_NuPIC_Hackathon__35580_36491.png Kickoff__2015_Spring_NuPIC_Hackathon__36492_39587.png Kickoff__2015_Spring_NuPIC_Hackathon__39588_39659.png Kickoff__2015_Spring_NuPIC_Hackathon__39660_40871.png Kickoff__2015_Spring_NuPIC_Hackathon__40872_41027.png Kickoff__2015_Spring_NuPIC_Hackathon__41028_42167.png Kickoff__2015_Spring_NuPIC_Hackathon__42168_42251.png Kickoff__2015_Spring_NuPIC_Hackathon__42252_44291.png Kickoff__2015_Spring_NuPIC_Hackathon__44292_45011.png Kickoff__2015_Spring_NuPIC_Hackathon__45012_47051.png Kickoff__2015_Spring_NuPIC_Hackathon__47052_47111.png Kickoff__2015_Spring_NuPIC_Hackathon__47112_47471.png Kickoff__2015_Spring_NuPIC_Hackathon__47472_47519.png Kickoff__2015_Spring_NuPIC_Hackathon__47520_47603.png Kickoff__2015_Spring_NuPIC_Hackathon__47604_47651.png Kickoff__2015_Spring_NuPIC_Hackathon__47652_48419.png Kickoff__2015_Spring_NuPIC_Hackathon__48420_49307.png Kickoff__2015_Spring_NuPIC_Hackathon__49308_49619.png Kickoff__2015_Spring_NuPIC_Hackathon__49620_49703.png Kickoff__2015_Spring_NuPIC_Hackathon__49704_50603.png Kickoff__2015_Spring_NuPIC_Hackathon__50604_50651.png Kickoff__2015_Spring_NuPIC_Hackathon__50652_52475.png Kickoff__2015_Spring_NuPIC_Hackathon__52476_52559.png Kickoff__2015_Spring_NuPIC_Hackathon__52560_53027.png Kickoff__2015_Spring_NuPIC_Hackathon__53028_53387.png Kickoff__2015_Spring_NuPIC_Hackathon__53388_53579.png Kickoff__2015_Spring_NuPIC_Hackathon__53580_53593.png
So let's kick this off. My name is Matt Taylor. I am the open source community flag bearer for the NuPic open source project and I guess for Nemento's open source south in general. First off we're recording. There's a camera in the back. I've got some cameras set up in other places so where we're doing sessions, I try and record for people who can't be here. You might be interested in what we're doing. So you might be caught on tape, just know that. If you would like to take any photographs or videos, feel free. This is a very open, transparent thing we're doing here so if you want to use a hashtag or social media it is Hack NuPic. Let me say thanks to everyone who has helped me with this hackathon real quick. Terry Fry and Taylor, they're at the registration desk downstairs. They've been instrumental for logistics and planning. And thank you, Jeff. And the rest are NuPic engineers on the right here who will all be here willing to help you. Get NuPic installed. Brainstorm about what you can do with your data. We've all got root on the back of our shirts so if you see somebody with root, ask them questions and if they can't answer it they'll find someone who can. Also thanks Cornell Tech. Are there any Cornell Tech students here? Okay. Well thank you Cornell Tech anyway for letting us use their space. Jeff, would you come up and say a couple of things to our hackers to maybe motivate them to do? Fun interesting things. I just saw this logo the first time a few minutes ago. Does everyone know what that is? It's great. My wife says what's that picture I got? I think it's great. It was one of our community members created it. Anyway, thanks Matt. Thanks for coming here and we're excited to be here. This is our first hackathon outside of Silicon Valley. It's a new experience for us although we've done something before. Matt always runs a great show. We're going to try to make this really valuable for you. I just want to put my personal thanks for you coming here. What we're working on in NMETA and I assume what you want to work on here as well I think is extremely, extremely important work in not just the technology world but for humanity in many ways. We are at NMETA. We're trying to reverse engineer the neocortex. We're making good progress at that. We want to build machines that work in those principles. That's what NMETA is all about. It's the open source component of that. I think the implications of this work are going to be very, very profound in the future. We all get a chance to participate in that. It's not clear exactly how the future will play out and what time frames and so on. It's kind of a messy time right now but it's something that's really happening. It's starting to form and I think NMETA is at the center of many decades of machine learning based on core principles and machine intelligence in general. It's great to have other people to participate in that and I know some of you come from a long way to be here. We're going to try to make it all very successful for all of you. It's important work. I don't really want to delay things much at all. I just want to thank you again for coming and we're going to try to make this successful and not just this event but Newpick as a whole and Machine Intelligence as a whole and Brain Theory and all that stuff. Thanks for coming and I'll be around here myself for a part of the day and then I'll be back tomorrow. We can have a lot of conversations with people on the chat. I'm going to give this back over to Matt who's really in charge and running everything and he's going to make it work. Thanks Matt. Thank you Jeff and your vintage hackathon t-shirt. I got my new one back and I just need to change it. No that's good. This is a two years old. I like it. This is from my first hackathon. This is from the first one? Yeah. Well it's from the next one. Okay. Let me get through some boring protocol so we can get on to the real stuff. First off, resources for you. Everything that you need to know should be at numinta.org slash hack. If you go there it's got links to our schedule, to our code of conduct, to resources, to everything you need. Where to register your hack, all that stuff. We will register on a website called challengepost. Challengepost.com. If you're going to have a hack demo at the end of this you need to register there. More info is on numinta.org slash hack. All of these slides also, so all of these slides have a bunch of links in them that might be useful to you. You can also find them at numinta.org slash hack. So today's schedule, I am doing the kickoff at around 10.30 or as soon as I'm done with this we're going to have a couple of new pick engineers show off a few samples of things they've done with new pick using one. Chaitan has a demo using the Unity game engine and feeding data from that into new pick and Austin has something with the project tango. That should be interesting. 11.30 to 12.30 we will be available probably in, there's a big conference room in the back here. So we'll have new pick engineers to help you get new pick installed if you're having problems getting it compiled and running on your laptop. At 12.30 to 1.30 in the room back behind the projection screen we're going to have sort of a brainstorming session on music theory with new pick. The idea behind this is to try and come up with ideas on how to encode MIDI data into sparse representation so that new pick can start understanding music. I think this is really interesting. We have a special guest with us that's going to help us. He's our in-house music theorist Charlie Gillingham from county crows. He's also a computer scientist. We'll be running that session and hopefully pick his brain on some music theory and how we can encode MIDI data into sparse representations. I'm going to give a short session on the state of new pick at 2.30. Here, attend if you want. I'm going to go around like what the community is like, what's going on in the community, what kind of projects are happening, where we're going, what we've accomplished over the past year. So it's more focused on people interested in contributing or in the development and the progress of what's happening with the new pick project itself. We also have Connor Rusamano from OpenBCI who brought a bunch of hardware stuff. OpenBCI does kind of DIY EEG kits. They've got these little spider things that go on your head with contacts and you can get EEG information from your brain. We've had a couple hacks in the past of getting that data into new pick and using it to control things. So that could be interesting. If you're interested in that, please attend that session. That will probably be back in this room behind the projection screen. Tomorrow, I've got nothing on the agenda except demos at 4. So if you're going to demo, be ready at 4 o'clock. A couple things about the facility. Emergency exits. You can see a sign immediately upon exiting this room. It says exit. That's where you go. All of the doors in here don't open pulling or pushing. They're all sliding. This has been a problem in the past apparently. So just be aware of that. It doesn't open. It's because you're doing it wrong. Slide the door back and forth. I mentioned bathroom codes earlier. The men and women have different codes. They're on the door on the way out. You just take a look. Our Wi-Fi here is Google Cast. It's very fast and it's free and no password. If you're sleeping here, feel free. You might hopefully brought a pillow or sleeping bag or something because there's no couches in here. But if you want to talk about accommodations, you can come talk to me later and maybe I can give you some tips on where to crash. Okay, a little more protocol. You do need your badges. To get in the door tomorrow, keep this thing and bring it back tomorrow. On the back of it is the real important thing. This is what the security guards are going to want to see. So keep the badge. It also has a gift card for food. We usually provide food at these events because of logistics problems at this facility. We were not able to provide meals. So we compensated by getting a gift card. If you can't find food on the street, there's something wrong with you. There is tons of food around here. It's lots of places to eat. So please go use that gift card so you can go eat. There's trash cans all over the place. Please use them. I will be making sweeps every few hours or so to pick up trash. It's a lot easier if you just clean up after yourself. Be respectful of the facility. If you don't know if you should be touching something, come and ask me. I'll tell you if you can touch it or not. Also, please take a look at our code of conduct. It's a typical code of conduct. Don't do anything that's harassing or offensive to people. If you think it might be offensive, it probably is offensive. So don't come up with any hacks that might offend anyone. If you want specific details, read it here. Okay, for communicating at this event, obviously we're all in the same room, so that's one method. If you have technical questions that someone here can't answer or you find an answer and you want to share it with people, we have mailing lists. You can go to namenta.org. There's a general discussion list, which is probably your best bet. And there's one for development and there's one for theory. So feel free to sign up for that. And remember, for those of you taking pictures of the slides, that the slides are available at namenta.org. They have all these links in them. We also have a chat room on IRC, which I sometimes on. And we also have a Gitter chat room. If you have a GitHub account, that's the one that generally has the most traffic for chat. And spread the word with the hashtag hack new pic. I like it's a vanity thing. I love going through Twitter and seeing all the social media stuff. Okay, a few questions before I go on. Please raise your hand if you've ever been to a hackathon before. A lot of people. That's good. Okay. Who has actually compiled and built new pic on their laptop? This is a better indication of how busy we're going to be helping you. Okay, so maybe half. That's not too bad. Who has run any of the example code tutorials, stuff like that? Another maybe half. Who has like taken their own data or manipulated their own, some type of data of their own and passed it into new pic and done some of the experiments? Cool. Okay. Thank you. I appreciate it. So some more resources for you. For those of you who did not raise your hand, on namenta.org, there's a link to the wiki. And I would say this is probably the best bet for like getting started is go to the wiki. The wiki is laid out in a way to kind of drive you in a path that will help you build and install, give you tutorials on how to use new pic and then how to learn like what new pic is doing and the theory behind it and that sort of thing. So that's a good resource to start at. Here are some suggested links especially for you hackers. If you're new to new pic and you want to figure out what it's all about, we've got a couple of tutorials we call hot gym. It's about energy consumption over time. This is a really nice use case for new pic because it recognizes these patterns in energy consumption in buildings really well. So both of these have screen casts and sample code and me talking on and on and on about how to build it out. So hopefully that's understandable. There's one for prediction and there's one for anomaly detection which follows the prediction one. We also have a beginner's guide to new pic which is Scott Purdy gave at a previous hackathon which is a really nice dive into the details of encoders, how we encode scalar data for example into sparse representations. So that's a good resource and there's a link here to a bunch of different code examples on our wiki. CPU usage, spatial pooling, temporal pooling, classification, all that stuff. So we have code examples of all this stuff. So if you see something that sparks an interest and you think, hey my hack might fit in that, start with that and see if you can run those examples. We also have a bunch of old demos from old hackathons. From old hackathons I think 14 demos from the last hackathon, something like that from the one before. So if you want some ideas of what other people have tried at old other hackathons, go there. If you do not want to use new pic or you're not a python person, you don't want to write python code. There are other htm implementations you could potentially use. We have htm java which is a port of the new pic python htm algorithms by David Ray. He's right back there. Yeah, there you go. Embarrassing. Okay. And we have comport.exe by Felix Andrews, which is a closure implementation of htm. It doesn't closely follow the r python implementation of the htm algorithms like htm java does, but it's still an htm. Okay. A little guidance about your demos. The most important thing I think is if you have an idea about something, think about your data. Think about do you have a lot of data? Is it temporal? The biggest question is, is the data you're thinking about using for a hack temporal, does it have a time component? Because if it doesn't, if it's just spatial, you might not have much success. A key thing about htm hierarchical temporal memory is a temporal component of that. So make sure you have a temporal component here. Data or at least you can somehow give it a temporal component. Is there enough data? You're not going to learn many patterns from, you know, annual data points unless you have lots of data, lots of annual data. And does it have patterns? Another question to ask yourself. Yeah, your question? Well, that depends. Probably not a question for this presentation, but I can talk afterwards about it. It depends on the patterns and the data, basically. A thousand data points in minimum type of thing? Yeah, that's good. You're going to need at least a thousand data points, probably, no matter what. Okay, a great way to learn htm theory is to build an htm in your favorite programming language. That's great. I love it when people do this, but it's not a good hack because you're just standing here telling people about what they already know and here's my htm in whatever. So please don't make an htm and do that. You can do it while you're here at the hack con, but don't make it a hack demo. We'd be happy to help you do that, but it's not a good demo. Okay, so let's talk quickly about anyone having any ideas. Oh, we're right on schedule, so good. I talked a little about music, so I'm really interested in encoding MIDI data somehow so that NUPIC can understand patterns and music. This is an example that Jeff uses a lot when he's talking about the brain, about htm, about the hierarchy involved. We've actually had demos in the past, our first hackathon actually, someone had encoded a pomp and circumstance, the graduation song, into scalar values and passed it into NUPIC and had it predict the entire song. After about, I think, 22, 23 repetitions, it predicted the whole song completely, like perfectly. So that was impressive to me, but it wasn't a true encoding because they hacked it into scalar data. So the real challenge here is how to get MIDI data or any type of musical representation into a sparse format that encodes more information than just the pitch of the note, right? Because there's more to music than just pitches of notes and durations of notes. So that's something I'm interested in. I don't know if I'll have time to work on that, but if you're interested in that, come to the music theory session we're having today because that's what we're going to talk about. But I want to hear about your ideas. Who has an idea of something they want to do today? I would really love to hear about it because you might motivate other people to help you implement your idea. A part of this is building teams. So I know someone has ideas. Please raise your hand if you have an idea and you're willing to share it. Okay, you're close. Come on, come on. My name is Tom Carzwell and my idea was to use NUPIC for spectroscopy and element recognition. When you look at it in absorption lines, they look suspiciously close to sparse data representations for certain applications. What is that? Basically you shoot light at something and then whatever light doesn't bounce back tells you what's in the object that you're shooting the light at or the light is passing from, depending on how you want to do it. But I can't think of a good way to make that temporal because it doesn't seem like something that would depend on how long you shoot the light at it. In most instance, maybe. Maybe. Okay, that's interesting. So if there's a video component to it in any way and you can get SDRs over time, that may be something that you could predict. As long as you could somehow reconstruct those into something actionable. Can that be explained to them? Yeah. Can that be explained to them? Yeah. And that's what I'm picturing this morning. I think that's more, that's more like a motion capture. This is more a sad thing. By the way, we do have a Connect in-house. One of our engineers brought Connect. So that's something we've been thinking about doing too. Do you want to just comment on this stuff? Yes. I do. I mean, it depends on what kind of data you get out of it. You know, Cortical IO did these word SDRs and they actually get really interesting results without even the temporal components just using the sort of semantic encodings of SDRs. You might be able to do something like that. At least with the words though, we're able to then say, okay, now we can do it in sequence and sentences. You might not be able to do that. But it's still, even as the SDR component of the theory might be useful for you in some way. I just don't know enough about your data. But there are some situations you can do, a limited set of what we do with spatial patterns, pure spatial patterns. So I just want to throw that out. Some of the thing now. Yeah. As long as the sparse representations coming out of this actually have semantic meaning, which they should. Sure. My name is Chiraga. I came to the last Hackathon at night. Then again, I chance to contribute a lot. So I've been reading about Newbick and the hack that I'm thinking about. I'm not even sure if this is a good hack. But I recently read a paper, a Stanford paper, which was trying to predict outcome of baseball games. And they had compared about six or seven AI, so-called AI algorithms and predicting what the outcome of baseball games is. So there's about 2400 games in baseball season and it would train the data over some period and it would try to predict outcomes for each team while we're game. So we were thinking about feeding that data that's really available and see if you can get any easy predictions just by feeding the data. Interesting. It might be even better if you can get events within each game as well. Hi. My name is Sean. This is a pretty vague idea, but I'm interested in environmentalism. And so can we take some amount of data and use that to do something like, I don't know, turn off lights more efficiently or decrease and increase the temperature of rooms based on if people are around. So if anyone is interested in something that's very vague but lots of ideas with that. We have to get a fair amount of work in that space. We're talking about it. It's doable. Definitely. It works. Yeah. Maybe something with even New York data that's around that I haven't looked into. Yeah, it is hard part. Sounds like an IOT application to me. Yeah. It's very like a building. If everything's instrumented in the building, you should be able to predict when people are going to be in what rooms and when things are going to be used. Yeah. Hi. My name is Ashish. I'm working for a project for New York Presbyterian Hospital this summer on patient onboarding and also at home care post discharge. So I'm trying to incorporate some kind of decision analysis before they get to the hospital. So what makes people think which hospitals to choose and what is the decision making. I'm not sure how that translates into this project, but hopefully there are some applications that somebody can help me with. Do you have data? My name is Dave Petrillo. I had an idea for using HTMs to look at automotive data. So can we add just a few extra sensors to a car and get a better idea of what's wrong with the car for consumer applications or get a better idea of how the driver's doing for like performance driving applications. It seems like there should be more information in the data than just okay this sensor is too hot and so then the engine is too hot, but instead the rumble came from the front right corner of the car so the way that we sense it means that wheel bearing is bad and could help make diagnosing cars for regular applications easier and faster. I don't have any data, but we could make some up or something. That definitely sounds like an anomaly detection application. It looks pretty good anomaly detection. I only have time for a few more so keep it short. Alright, my name is Patrick Ryan. I am interested in trying to model student data. I have some generated student data based on students that are good and bad and I'm thinking about making it 10-4 by encoding the student and the question and maybe taxonomies about that question and then followed by their answer and whether that was correct. It's just a simple one-two sequence but it might work. I'm not sure that's temporal enough. So think about the temporal aspect. Hi, I'm Steve Double Day and for my dissertation I'm trying to model interaction between the hippocampus and the cortex and there's a particular rat experiment that I'd love to be able to do. I started to try to put into new pic but only gotten a little way in and that has to do with a rat will project out of their hippocampus a trajectory about where they're going to move and if you were to feed in the trajectories that get generated by a rat or a simulated set which I've got out of the test framework, the question is, can over a period of time can our algorithm predict which trajectory is going to result in a reward? That's what I'm tackling. Okay, I'm going to do you. Okay, four more. I've already picked them. Hey, my name's Scott. So I consider using anomaly detection for fraud detection in the mobile space. So right now mobile advertisers, publishers specifically are left to sort of set up their own metrics using mostly thresholds. If you can use simple device identifiers like the IDFA, the country, maybe their locale in the operating system, you could potentially do some useful things in detecting the types of traffic and the determined anomalous things and present that to a team of security people working for an app. I think, yeah, right now in the mobile space, like I said, a lot of publishers are having to do this on their own and I think it would be useful to consolidate that and create some service around that and testing that out and trying that would be a fun thing to do. Definitely a lot of people are interested in fraud detection. It goes along with anomaly detection. Road behavior detection is another thing we've worked on. Hi, my name is Marcin. So I saw a couple of days ago there was a quite interesting analysis of the LSTM unit. There was some guy posted this on my, from Stanford to guess. And I figured out, well, let's try to create a little bot where you just feed the sentence, say hi, and trade the network and see what the network is going to come up with. That previous person had a bunch of experience as well as Wikipedia as well as kernel sourcing to a pretty wide LSTM unit and got some very interesting answers. So you could have, I guess, if the NuPic network is deep enough, you can have a very sensible conversation. So let's see. I'm Frank Rose. As part of trying to understand the NuPic algorithm, I was interested in doing a sensitivity assay of the algorithm itself using generated data, trained on a classification problem, and then perturbed that data and see, okay, where does the classification start breaking down if you add randomness or drift or something to the input path. So maybe, you know, maybe even be able to compare the performance of the Python version and HTM and Cortex. Hi, I'm Paul. I work for this company that makes, we use this brain sensing headband, it's an EEG device, and we want to use it for something pretty simple, HTM for something pretty simple of just recognizing muscle movement, like blinks, winks, you know, forehead, crunches, or things like that, and maybe later use it for brain sensing. My name is Vladimir. I'm working with time series, a lot of financial and economic time series, and my idea was to test the performance of HTM in kind of non-stationary environments, so I would like to generate a kind of theoretical time series with time change and patterns, and see how HTM can just recognize patterns in kind of adaptive non-stationary environment. Hi, I'm Perga Burnan. I want to hack the theory, basically. So if people are interested in talking about a couple of ideas that are kind of extensions of HTM, and basically going to be interested in brainstorming those things over the weekend, so I'm also really interested in the non-stationarity thing as well. Hey, Perga. Hey, how's it going, everyone? I'm Connor from OpenBCI. I'm interested in taking an OpenBCI device and plugging, or connecting 16 channels to the kind of lower forearm, and seeing if we can use NewPick to classify individual finger movement, because right now I've got a system rigged up where I've got a single EMG input and I can make a hand flex in like an analog motion, but I'm really interested to see if we can identify individual fingers. Okay. Anybody else? My idea, first of all, I was reading about the complex adaptive systems from a different book, and it seems like it used genetic algorithms in order to have to solve the problems in its nature, but I want to embed a brain into it so that NewPick go into that adaptive system to make it better. There are some papers that actually combined the two, but it was only one paper that I found. So I'd like to be able to do that within Mathematica, because Mathematica can actually use, what do you call it, the video card. The GPU. The GPU, in order to compute things much faster, and it's easier to code. So I was trying to actually implement the algorithms in Mathematica. That would be interesting. I don't think that would be, I would accept that demo. As an HTM implementation within Mathematica, that's interesting. Just a quick question. There's a lot of interest in getting NewPick to run on the GPU, and I think I've seen other HTM-like implementations running on the GPU, but we currently don't. So I'm eventually, I'm sure, we will. Hello. We are Cortical.io, and we are trying to combine your HTM and our retina API to detect anomalies and or patterns on text data or text strings. And if you guys are working with text data, feel free to try our API, Cortical.io. For free? Yes, for free. So if you have the hackathon and you want to use Cortical.io's API, which is really cool, talk to these guys and I'll give you an API key. Hi, my name is John Yeh. I'm working on two projects. One is an acoustic project that by listening to the noise of the falset sound, we want to be able to detect which falset and how it works and where it is. And the other one, I'm starting to kick around this idea, is actually using the switch matrix on the FPGA device. Instead of hard coding the actual FPGA, we want to be able to do the weighted learning as part of the switch matrix on the FPGA device. So that's an unexplored layer of an FPGA device. First hack mentioned about audio analysis. Not MIDI, but audio. So there are others here, and Richard is interested in audio waveform analysis, that sort of thing. And I've done some experiments with that too, largely unsuccessfully, but that's my fault. Anybody else? So my hack that I'm planning to work on this weekend is called Market Patterns. And the idea is to, well, the assumption is that there are patterns in the stock market that we can learn to detect. There are lots of traders that learn these patterns by looking at charts. So I want to see, can we encode the data in such a way that NuPik can see patterns and just predicts, you know, ups and downs, not prices, just directions. We've been nominally successful with stock volume prediction, so I know that's possible. Hi, my name is Ji-young. I'm interested in analyzing patterns, like intellectual properties. So my idea is more of classification of patterns, so how valuable your patterns are. It's kind of a vague idea, but that's what I'm interested in. What's the data? So it's like a pattern claims. So you can, you know, patterns. Got it. Okay. Matt, can I just say where does it... Please. You know, I think it might be wise for, you know, especially if you're a NuPik, if you've got an idea and you want to run, and you think you're creating some data and running through or you have some data, try to find one of the NuMenta employees and just discuss it a little bit. It's very easy to make incorrect assumptions about how the data should be, the sampling rates, and what is temporal data. And so if you have even any thoughts at all, it's really useful. More people trip up on the data, I think, than anything else. So it's worth grabbing one of us and just say, hey, I suppose I think you're doing. Give me some advice. You're running circles for a long time, and then we'll come tell you all your data's in the wrong format. Okay. So that concludes the kickoff. Let's do like a 15-minute break, and I'm going to bring Chaden and Austin up here to talk about some of their hack samples with Unity, Game Engine, and Project Tango. Thank you. Thank you all for coming. Music
So let's kick this off. My name is Matt Taylor. I Matt. Thank you Jeff and your vintage hackathon