Smart Harbor [DEMO #7]
Ducro, Daniel
Wietses, Egbert
Tracking geolocations of cargo ships in the port of Rotterdam.
 So we're from the Netherlands, this is Daniel and I'm from a small company called the B.A.X. We're a group of five people. In the Netherlands we participate on a hackathon as well, a 24-hour hackathon and we try to detect anomalous behavior of an incoming ship traffic. So there were a couple of things we tried to do. One of us see if a ship goes on another course, another course than we then predicted. And we also wanted to see if it's another speed, so if it's closing up on a ship or a ship slowing down, stuff like that. And also made me if it's going into an area that it shouldn't be, for example like a new farm or any other structure or object. In the Netherlands we have a lot of water and also a lot of stuff in the water. So a couple of use cases we were thinking of was a hardware-of-use-lue security and offshore security. Yeah and Daniel will show a little bit of what we did here. On the hackathon in the Netherlands we did some stuff but we tried to go beyond that point here and try to see if we could feed some live data into it and see if we can get any results. So in Holland we had a couple of open APIs we could use but unfortunately they closed them off. So we had to use our old data that we collected then. But what we did is we took basically a month of data and then learned one model. We took all the ships, we threw them in a pile and then we learned one model of all these ships and then we created on each of these ships we created a different model to predict if you have to see if there's any anomalies. And so yeah we wanted to feed in live data so we kind of simulated live data as you see here. We think we succeeded but yeah there are a lot more things to explore. Did you use the NMATG special recorder? Yes. You can see some of the parts here it looks like they're going across lands but there are a lot of tiny ways. So how do we interpret it? There's a line what does the line mean? I'll explain it a bit more. We trained the base model with around 30,000 records I think of ship track data and what you see here is yeah we took that as a starting point and the green line indicates that it seemed the route before so we trained it a little bit more a couple of times and yeah you see some anomalies but most traffic what you see here the geospatial recorder of Newfrix says it seemed before and it's recognized as normal data. What does the actual line mean? Yeah it's the track that it took and it's supposed to just leave a little trail behind the boat and disappear. So the blob to the boat so you can tell the direction by if the blob is on the left it's going towards the left? Yeah. Now we set this up quite a bit of course. So take an anomaly there what do you think is happening is going at a different speed or a different route that nobody's gone before? Or is that a fault now? I think it could be a fault. I mean you can interpret it. Is that a static point? Yeah so I think most of it's a little bit of noise from the new pick so we tried with some different data and you could clearly see it goes red like instant and what you see here like you see a little bit of yellow and a little bit of orange that's a little bit of noise but almost all traffic because we also used this data to train it almost all traffic got recognized so Newfrix is doing a great job actually at recognizing all the tracks. Did you use the likelihood stuff at all? No that takes care of some of the noise issues. But what would be nice to test is to actually throw some anomalies out and see what happens. I don't know what would be a real anomaly. I mean something going north south instead of east west or something. I mean it would be nice to try to get a boat in a circle or something. We threw in some tracks that we didn't record before and they turned red but then we wanted to have more tracks to show in the demo but yeah of course these things didn't work out because it was funny in the last hour or so and that's what we're showing you this. When we had these in San Francisco Bay we had a lot of boats and we had some that have gone too close to the bridge pier and hit the bridge. I mean it would be interesting if you could see that a boat was kind of heading off a standard shipping lane and knew that it could get in trouble. That's what we wanted. Let me zoom in a little bit so you can see how accurate it actually is. You can see some of the tracks is not correct but you can on some other data we have tried also. So obviously you're not sampling very fast because you're at one point and you throw at another point where that was a question. But if the data was a little bit better you would see it goes strictly, it follows the lines and the borders of the waterways very carefully. We've seen it with other data as well. So most ships leave the middle blank so if the ship would give you to the middle and you could see it there instantly. And it's pretty accurate. You tested that out? Yeah we tested that out. How often are you sending the data in? We trained it with the TCP connection as well. The sampling was once in mainly, once every 30 seconds. We threw in all the history first. How often is that history sample? One month of data is almost 6.7 million GPS positions. Every how many minutes do you get a GPS position? Every second. Every two seconds. That's very fast. On the speed of the boat, depending on the speed they have to give a frequent position. The change in patient speed? Yeah exactly. So in the Netherlands they are obligated to give their position through this system. So we only had fixed the data. The most go faster than they give more in the second? Yeah. It was going slower this week on bottom. Do you guys feed every point in sequence? We try to use the pre-processing as well. So you try to regularize the sampling rate. Normalize it. Something? Time. We tried but... We had some problems with the amount of data and getting it to the full signal. Yeah. We should look at this more because we can get this data in the system too. Yeah. I have a... Very cool. Thank you guys.
 So we're from the Netherlands, this is Daniel and I'm from a small company called the B.A.X.  We're a group of five people. In the Netherlands we participate on a hackathon as well, a 24-hour hackathon  and we try to detect anomalous behavior of an incoming ship traffic. So there were a couple of  things we tried to do. One of us see if a ship goes on another course, another course than we  then predicted. And we also wanted to see if it's another speed, so if it's closing up on a ship or  a ship slowing down, stuff like that. And also made me if it's going into an area that it shouldn't  be, for example like a new farm or any other structure or object. In the Netherlands we have a  lot of water and also a lot of stuff in the water. So a couple of use cases we were thinking of was  a hardware-of-use-lue security and offshore security. Yeah and Daniel will show a little bit of what we  did here. On the hackathon in the Netherlands we did some stuff but we tried to go beyond that point  here and try to see if we could feed some live data into it and see if we can get any results.  So in Holland we had a couple of open APIs we could use but unfortunately they closed them off.  So we had to use our old data that we collected then. But what we did is we took basically a month  of data and then learned one model. We took all the ships, we threw them in a pile and then we  learned one model of all these ships and then we created on each of these ships we created a different  model to predict if you have to see if there's any anomalies. And so yeah we wanted to feed in live  data so we kind of simulated live data as you see here. We think we succeeded but yeah there are a lot  more things to explore. Did you use the NMATG special recorder? Yes.  You can see some of the parts here it looks like they're going across lands but there are a lot of tiny ways.  So how do we interpret it? There's a line what does the line mean? I'll explain it a bit more. We trained the base model  with around 30,000 records I think of ship track data and what you see here is yeah we took that as a starting point  and the green line indicates that it seemed the route before so we trained it a little bit more a couple of times  and yeah you see some anomalies but most traffic what you see here the geospatial recorder of Newfrix says it seemed before  and it's recognized as normal data. What does the actual line mean? Yeah it's the track that it took and it's supposed to  just leave a little trail behind the boat and disappear. So the blob to the boat so you can tell the direction by if the blob is on the left  it's going towards the left? Yeah. Now we set this up quite a bit of course.  So take an anomaly there what do you think is happening is going at a different speed or a different route that nobody's gone before?  Or is that a fault now? I think it could be a fault. I mean you can interpret it. Is that a static point?  Yeah so I think most of it's a little bit of noise from the new pick so we tried with some different data and you could clearly see  it goes red like instant and what you see here like you see a little bit of yellow and a little bit of orange that's a little bit of noise  but almost all traffic because we also used this data to train it almost all traffic got recognized so Newfrix is doing a great job actually at recognizing all the tracks.  Did you use the likelihood stuff at all? No that takes care of some of the noise issues.  But what would be nice to test is to actually throw some anomalies out and see what happens. I don't know what would be a real anomaly.  I mean something going north south instead of east west or something. I mean it would be nice to try to get a boat in a circle or something.  We threw in some tracks that we didn't record before and they turned red but then we wanted to have more tracks to show in the demo but  yeah of course these things didn't work out because it was funny in the last hour or so and that's what we're showing you this.  When we had these in San Francisco Bay we had a lot of boats and we had some that have gone too close to the bridge pier and hit the bridge.  I mean it would be interesting if you could see that a boat was kind of heading off a standard shipping lane and knew that it could get in trouble.  That's what we wanted. Let me zoom in a little bit so you can see how accurate it actually is.  You can see some of the tracks is not correct but you can on some other data we have tried also.  So obviously you're not sampling very fast because you're at one point and you throw at another point where that was a question.  But if the data was a little bit better you would see it goes strictly, it follows the lines and the borders of the waterways very carefully.  We've seen it with other data as well. So most ships leave the middle blank so if the ship would give you to the middle and you could see it there instantly.  And it's pretty accurate.  You tested that out?  Yeah we tested that out.  How often are you sending the data in?  We trained it with the TCP connection as well.  The sampling was once in mainly, once every 30 seconds.  We threw in all the history first.  How often is that history sample?  One month of data is almost 6.7 million GPS positions.  Every how many minutes do you get a GPS position?  Every second.  Every two seconds.  That's very fast.  On the speed of the boat, depending on the speed they have to give a frequent position.  The change in patient speed?  Yeah exactly.  So in the Netherlands they are obligated to give their position through this system.  So we only had fixed the data.  The most go faster than they give more in the second?  Yeah.  It was going slower this week on bottom.  Do you guys feed every point in sequence?  We try to use the pre-processing as well.  So you try to regularize the sampling rate.  Normalize it.  Something?  Time.  We tried but...  We had some problems with the amount of data and getting it to the full signal.  Yeah.  We should look at this more because we can get this data in the system too.  Yeah.  I have a...  Very cool.  Thank you guys.
Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_0_1319.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_1320_4415.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_4416_4511.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_4512_9395.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_9396_9815.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_9816_10163.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_10164_10331.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_10332_10463.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_10464_11291.png Smart_Harbor__DEMO__7___2014_Fall_NuPIC_Hackathon_11292_14681.png
So we're from the Netherlands, this is Daniel and I'm from a small company called the B.A.X. We're a group of five people. In the Netherlands we participate on a hackathon as well, a 24-hour hackathon and we try to detect anomalous behavior of an incoming ship traffic. So there were a couple of things we tried to do. One of us see if a ship goes on another course, another course than we then predicted. And we also wanted to see if it's another speed, so if it's closing up on a ship or a ship slowing down, stuff like that. And also made me if it's going into an area that it shouldn't be, for example like a new farm or any other structure or object. In the Netherlands we have a lot of water and also a lot of stuff in the water. So a couple of use cases we were thinking of was a hardware-of-use-lue security and offshore security. Yeah and Daniel will show a little bit of what we did here. On the hackathon in the Netherlands we did some stuff but we tried to go beyond that point here and try to see if we could feed some live data into it and see if we can get any results. So in Holland we had a couple of open APIs we could use but unfortunately they closed them off. So we had to use our old data that we collected then. But what we did is we took basically a month of data and then learned one model. We took all the ships, we threw them in a pile and then we learned one model of all these ships and then we created on each of these ships we created a different model to predict if you have to see if there's any anomalies. And so yeah we wanted to feed in live data so we kind of simulated live data as you see here. We think we succeeded but yeah there are a lot more things to explore. Did you use the NMATG special recorder? Yes. You can see some of the parts here it looks like they're going across lands but there are a lot of tiny ways. So how do we interpret it? There's a line what does the line mean? I'll explain it a bit more. We trained the base model with around 30,000 records I think of ship track data and what you see here is yeah we took that as a starting point and the green line indicates that it seemed the route before so we trained it a little bit more a couple of times and yeah you see some anomalies but most traffic what you see here the geospatial recorder of Newfrix says it seemed before and it's recognized as normal data. What does the actual line mean? Yeah it's the track that it took and it's supposed to just leave a little trail behind the boat and disappear. So the blob to the boat so you can tell the direction by if the blob is on the left it's going towards the left? Yeah. Now we set this up quite a bit of course. So take an anomaly there what do you think is happening is going at a different speed or a different route that nobody's gone before? Or is that a fault now? I think it could be a fault. I mean you can interpret it. Is that a static point? Yeah so I think most of it's a little bit of noise from the new pick so we tried with some different data and you could clearly see it goes red like instant and what you see here like you see a little bit of yellow and a little bit of orange that's a little bit of noise but almost all traffic because we also used this data to train it almost all traffic got recognized so Newfrix is doing a great job actually at recognizing all the tracks. Did you use the likelihood stuff at all? No that takes care of some of the noise issues. But what would be nice to test is to actually throw some anomalies out and see what happens. I don't know what would be a real anomaly. I mean something going north south instead of east west or something. I mean it would be nice to try to get a boat in a circle or something. We threw in some tracks that we didn't record before and they turned red but then we wanted to have more tracks to show in the demo but yeah of course these things didn't work out because it was funny in the last hour or so and that's what we're showing you this. When we had these in San Francisco Bay we had a lot of boats and we had some that have gone too close to the bridge pier and hit the bridge. I mean it would be interesting if you could see that a boat was kind of heading off a standard shipping lane and knew that it could get in trouble. That's what we wanted. Let me zoom in a little bit so you can see how accurate it actually is. You can see some of the tracks is not correct but you can on some other data we have tried also. So obviously you're not sampling very fast because you're at one point and you throw at another point where that was a question. But if the data was a little bit better you would see it goes strictly, it follows the lines and the borders of the waterways very carefully. We've seen it with other data as well. So most ships leave the middle blank so if the ship would give you to the middle and you could see it there instantly. And it's pretty accurate. You tested that out? Yeah we tested that out. How often are you sending the data in? We trained it with the TCP connection as well. The sampling was once in mainly, once every 30 seconds. We threw in all the history first. How often is that history sample? One month of data is almost 6.7 million GPS positions. Every how many minutes do you get a GPS position? Every second. Every two seconds. That's very fast. On the speed of the boat, depending on the speed they have to give a frequent position. The change in patient speed? Yeah exactly. So in the Netherlands they are obligated to give their position through this system. So we only had fixed the data. The most go faster than they give more in the second? Yeah. It was going slower this week on bottom. Do you guys feed every point in sequence? We try to use the pre-processing as well. So you try to regularize the sampling rate. Normalize it. Something? Time. We tried but... We had some problems with the amount of data and getting it to the full signal. Yeah. We should look at this more because we can get this data in the system too. Yeah. I have a... Very cool. Thank you guys.
to show in the demo but yeah of course these