Brainwaves for Hackers 2.0
Klostermann, Andreas
Andreas Klostermann - Brainwaves for Hackers 2.0 This talk is a sequel to "Brainwaves for Hackers" and illustrates some experiments you can do with a Neurosky Mindwave headset, a bluetooth enabled EEG device. I'll also talk some more about how to integrate the device with the IPython Notebook for real time viewing and how to use the Mindwave with the Raspberry Pi.
 Hello, my name is Andreas Klostermann and in this talk I will talk about brain waves and how we as Python hackers can use our tools to explore our own brains a bit. Last year at EuroPython I held the 1.0 version of this talk. This year I have only 30 minutes so I'm going to make extremely short thrift of the theory behind brain waves but we have to cover a bit of it so that everyone of you understands what we are doing here but the talks don't have a lot of overlap so this is really 2.0. Now first of all what are brain waves actually? Brain waves are an electro physiological potential that you measure on your scalp that basically is a voltage measurement which is measured over time with electrodes that are on the scalp and what you are measuring there is the summed potential of most or even all your brain cells which give off electrical signals to communicate with each other but also a bit of waste energy gets transmitted outwards and so you can measure it on the skin without having to go inside the scalp. Most of what the most useful analysis of brain waves usually is the Fourier transform which basically only means that we assume that the signal is composed of several frequencies and we want to know which frequency is dominant and by doing that we can infer something about what the brain is doing because we know when a certain frequency range is overrepresented and the subject is for example concentrated or relaxed or that kind of information. Now to measure this current we need an electro and a phallogram and these devices can be quite cumbersome and quite expensive but what I have here is basically the Neurosky Mindwave Mobile which is a Bluetooth connected headset and is really low powered and very optimized for developers and hackers. It does all the amplification stuff inside this headset and digitalizes the data. It also does some preliminary analysis of the data so that you could even connect this to an Arduino and the Arduino would be able to tell if you are concentrated or not. Aruinos are not really sophisticated processors or software in any case so it's quite nice. I wrote a Pyzeology framework and that is a framework for physiological experiments and analysis. It currently mostly only supports Mindwave EEG but eventually I also have code for ECG from Vitalino but that's not activated currently. It's powered by Psypydita and it's iPyzen Jupiter enabled so what I want to achieve is an experimentation platform that works for do-it-yourself people who don't have a lab fancy equipment or something but I want to use mainly these quantified self-type devices with Bluetooth connections or other near field stuff. It's currently Python 3.4 only and I'm making heavy use of Async I.O. Now you've probably heard of the Internet of Things. The basic idea is that you have local computers connected to some kind of sensor and there is a pattern that I see emerging in these data acquisition applications so you have a local component where it very much matters whether the computer is like a Raspberry Pi or an Arduino or something because it has to be right next to the device because of the low range of the sensor. Then there's a cloud component which actually analyzes the data and that does not matter where it is just that it is connected over a network and the user interface is connected also to cloud and local and displays the data and user can interact with the analysis. In my case, the local component is just an Async I.O. data server that I've written. It communicates over Bluetooth with this thing here and it also babysits the connection so if I switch it off and switch it on again then it will reconnect and try to fix errors. That's quite nice because I have that separated from the iPython kernel which is the cloud component because it's difficult for the user to deal with all the exceptions that occur in Bluetooth and we have most of the data processing inside the kernel then. The kernel pushes data to the user interface which is a browser. In an iPython notebook like this one you have the server, the kernel and the user interface and they are somewhat separated but you need a server for Python anyway. Now I'm going to quickly show you a real-time demonstration. This is junk data because nothing was connected and now if we are lucky I will be silent for a moment so that you can appreciate normal brainwaves. The problem with EEG also is that the brainwaves themselves are so, so weak that pretty much everything going on inside or in your head, all the muscles, the facial muscles and eye muscles all contribute artifacts to the data and are stronger than the actual brainwave data. I can show you that I can clench my teeth which are like the strongest muscle in the head. I can also use my eyes to... Yeah, and when I blink. And that makes EEG signals very difficult to analyze. Also this is a bokeh graph. I can use iPython widgets to push data from the kernel to the notebook and also the kernel is running an I think IO loop internally so that it can communicate over the web socket with my data server which handles the Bluetooth stuff. Now I have stopped and then I can go to the next slide. Now let's do some data science. Data scientist is who data science does so we need some data for that and to get good data we need experiments because discovery requires experimentation. And this is a central idea in the physiology framework that is this experiment stuff. The experiment is here initialized and then immediately used as a decorator much like the iPython notebook interact one. And this decorator when I start it, it will wait for data from the server and whenever there is new data this handle message is called and it can then do its thing to do whatever. In this case it just display clear output means the output is cleared and then some HTML is written with the last attention data. So attention is a value that is computed by the newest guy mind-face itself inside the headset. And I can show you how that works and that is really all it takes. So this is a value between zero and 100 currently it seems I am totally inattentive but I can push that higher. So in terms of time it doesn't work. Now experiment classes have these attributes like attention as a time series. So every second or so the such a value is computed by the device in send over Bluetooth and I try to figure out when that was but it is a bit difficult to do that. The time series stuff is really good and so it is just accumulating this experiment. Also the raw data, the meditation data and several other things are also communicated and stored. Now that was one thing running experiments in real time but we also like to record data. That is done by giving it a file name and that is a HDFI file. In this case I think I have a bug because I have like one or two gigabytes of data and that is completely crazy. I have to fix that later. In any case this is the same as before but it displays the amount of raw data which is here, then experiment raw over 512 because the raw data is at 512 samples per second. Now let's record some raw data. I think it is not working currently. In any case, now to resurrect the data we need a so called batch class and the batch class can resurrect the data then. Here we have recorded data from another session which just is much the same as the experiment. I forgot to tell you what actually a batch is. Every time I run the experiment I record a different batch. First time batch 0, second time batch 1 and that way experiments don't overwrite their own data and you can also try to vary the experimental conditions if you want to do that. Now I would like to show you a simple experiment. I don't have time for a lot of them so I would only do one. The new Sky Mindwave computes these ESense meditation attention values. They are patented and sort of back box algorithms where they don't really tell us what they are doing there. Here I am transforming the time series data to a table of features. I don't have time to really explain what is going on there but I take windows of the time series data then I compute how strong different brain wave frequencies are and in the same row I also have the tension and the meditation values. I wrote separate library which is table cleaner and we need that to clean some of the data up because some of the data is bad. As I told you before we have lots of artifacts in the data and we don't want to analyze artifacts. We want to analyze real EEG data. So this table cleaner library is inspired by jungle forms and cleans tabular data. The nice thing is it outputs both a validated data set and a data set of the errors. So on the bottom we see a table of the grouped validation errors and the most rows, 206 rows were deleted because there was too much variation in the EEG data. So when there is an artifact then we try to remove that row of the table. Also poor signal is computed by the brain wave itself but that's not so important in this case. So if you need data validation thing maybe you should look into it. Now I'm doing the linear regression analysis. So I said we have features of brain waves correlated with attentional meditation data and the linear regression all it does is it tries to explain the tension value or the meditation value in terms of the frequency strength. So it multiplies each frequency with the coefficient and then they are all summed up. And that way when you figure out the coefficient you know how important the different frequencies are for this value and you probably can't read the labels on the graph but I didn't have time to make that any better. What you may actually see is that we have here the best or the highest and most significant coefficients are at the... What was that? Are in the high beta or in the beta frequencies from 17 hertz upwards. I can't see that but just believe me when I tell you that there is a certain range of frequencies which is called beta frequencies between 17 and I think 30 hertz depending on the definition. And they are strongly associated with all sorts of attention and concentration stuff. So if you ever heard about the prefrontal cortex and the tension circuits of the brain also ADHD that has a lot to do with with beta brain waves. Now I'd like to meditate a bit about meditation. There's a similar graph but we have more going on. Here we have the alpha values which are between in this case are between like 8 and 10, around 8 and 10 hertz which are significant. But also some beta values which are a bit undecided. I think that's because most of the time I or others too need to do some concentration to enhance alpha values. During baking stage you need to have a somewhat calm mind to actually exhibit alpha waves. Or you close your eyes which is also easy but that's not what we want to do. In any case these values are used in newer feedback. So you can write an application that looks for attention or meditation and feed it back to you. And then you can learn how to consciously manipulate these brain states. That's what I was talking about last year at Europe, Pison and Berlin in 2014. And there's also a YouTube recording of the talk where I go into more detail about the psychology and biology behind all this newer feedback stuff. So this all sort of implies, so the linear regression implies that the strengths of the frequencies are not dependent on each other. And I don't know why that. I'm not going to get it to work. In any case these are correlated and we see that the alpha and the beta values are correlated among each other. But not between the alpha and the beta waves. It has some biological meaning but I'm not going to go into that. I'd like to make some technical remarks. Bluetooth is really bad for timing. So if you have Bluetooth data coming in, they often don't come with timestamps. And these timestamps, you have to give them timestamps as the local component in the server, data server, and figure out which are the right timestamps. And if you do it wrong, then you have like overlapping data and crossing your own timeline is strictly forbidden except for cheap tricks. Yeah, I think I was very good for waiting and problem resolution. I do like yield from weight and I yield from sleep and so I do so I can write these troubleshooting algorithms with reconnecting and I can wait for a few seconds and then reconnect again without disturbing anything else. The combination of Bokeh and notebook widget really rocks because this notebook has its own web socket push communication channel which I can use to stream data to the JavaScript site and update the book charts. And in general, Jupiter notebook is a really great tool to tell computational narratives. I rather skip that or rather went hastily through my little computational narrative about the brainwaves and about attention and meditation, but I didn't really have time for more. But you can imagine that you can do real time experiment or show how the computation works and also explain or narrate how the data really works. Also, it's a lot is the the biology framework is an example of a library for the notebook which really is bigger on the inside. So I just had this experiment decorator and entire heights all this web socket and async IO and other magic. Mostly from the user. I'd like to thank you for your attention. And my Twitter handle is page in horse which was a few years ago supposed to be a pun or Trojan horse or something but now it just sounds silly. The library is by the ology. I just published it. It's very raw and you probably won't get it to work. Especially unless you have this device by the way resistance is futile. This table cleaner library I try to make it a bit bigger because I have had a lot of trouble over the times validating CSV data. And that is sort of my attempt to fix that problem. In this case, I wrote also this notebook assets library which I use for for turning coffee script into JavaScript and serving it's from the actual library itself and not from the extension stuff which I find a bit troublesome because the notebook profile directory has no clear one on one direct connection to the Python library installation stuff. And it's basically all. Thank you for your attention and we have time for some questions maybe. But you when you want to go to the lightning talks. I tried to do some stuff with with the mine wave years ago. But I'm guessing that your framework didn't exist then because then I had to connect via some some proprietary Bluetooth executable that I had to wrap stuff up around so so does your Python code do the Bluetooth stuff. Yes. Also, the current documentation for mine wave points to my older libraries for mine wave so it's a bit difficult to do and I think what the newest guy stuff is doing isn't really that developer friendly on the library side. But the protocol is very friendly. So if I want to get in one of these nearest guy things is that I just had a brief look at the web page is the nearest guy mind wave mobile which is the one to get. Yeah. Yes I think that's the best device as one that is a bit more expensive and more variable but I think the new sky mine wave mobile is the best one for like 100 or 120 euros. You can also get it on Amazon and other shops and you only need the device itself. You don't need any of their software or anything. You just use that all. Okay. Great. Some people even wire up Arduino directly to the sink your ship. So as we as we have some time. Would you mind to show the code for the demo part. Maybe the demo itself. Yeah. By the way, this I want to publish this the notes but I wasn't able to to push it yet from this network and it contains a lot of notes of what I was talking about which are not visible in the presentation itself. So here's a hidden code with the real time raw demo and but to to know what's going on here is this normal book a let's make a plot stuff and then it's the raw source book a model source column data source which is important to have a reference to it. And then my touch bed. So in the handle message one I am doing some re sampling and then I push the data as a very convert the data to this raw source data on this Python side. But the book a knows it's this book a objects know the ID on the browser also on the Python side so I push this ID here. And then replace book a data source and it was a widget which has a communication between the Python side and the notebook side. And then I have just this function replace book a data source which pushes this this data source with its ID and the new data to the JavaScript side and then it deconstructs it and puts that into the data source into the book a data source on the browser and then magically through different callbacks. And then I have this new graph is just redrawn. Thank you. I have to reconnect again. I'm asking to show the demo. I'm sorry. I'm so excited. I work on the key to working now. What is it? What are the lines? Can you most code me? It's actually an easy abuse of this technology to do more scouting like I can also bring. Yeah.
 Hello, my name is Andreas Klostermann and in this talk I will talk about brain waves  and how we as Python hackers can use our tools to explore our own brains a bit.  Last year at EuroPython I held the 1.0 version of this talk.  This year I have only 30 minutes so I'm going to make extremely short thrift of the theory  behind brain waves but we have to cover a bit of it so that everyone of you understands  what we are doing here but the talks don't have a lot of overlap so this is really 2.0.  Now first of all what are brain waves actually?  Brain waves are an electro physiological potential that you measure on your scalp  that basically is a voltage measurement which is measured over time with electrodes  that are on the scalp and what you are measuring there is the summed potential  of most or even all your brain cells which give off electrical signals to communicate with each other  but also a bit of waste energy gets transmitted outwards and so you can measure it on the skin  without having to go inside the scalp.  Most of what the most useful analysis of brain waves usually is the Fourier transform  which basically only means that we assume that the signal is composed of several frequencies  and we want to know which frequency is dominant and by doing that we can infer something about  what the brain is doing because we know when a certain frequency range is overrepresented  and the subject is for example concentrated or relaxed or that kind of information.  Now to measure this current we need an electro and a phallogram  and these devices can be quite cumbersome and quite expensive  but what I have here is basically the Neurosky Mindwave Mobile which is a Bluetooth connected headset  and is really low powered and very optimized for developers and hackers.  It does all the amplification stuff inside this headset and digitalizes the data.  It also does some preliminary analysis of the data so that you could even connect this to an Arduino  and the Arduino would be able to tell if you are concentrated or not.  Aruinos are not really sophisticated processors or software in any case so it's quite nice.  I wrote a Pyzeology framework and that is a framework for physiological experiments and analysis.  It currently mostly only supports Mindwave EEG but eventually I also have code for ECG from Vitalino  but that's not activated currently. It's powered by Psypydita and it's iPyzen Jupiter enabled  so what I want to achieve is an experimentation platform that works for do-it-yourself people  who don't have a lab fancy equipment or something but I want to use mainly these quantified self-type devices  with Bluetooth connections or other near field stuff.  It's currently Python 3.4 only and I'm making heavy use of Async I.O.  Now you've probably heard of the Internet of Things.  The basic idea is that you have local computers connected to some kind of sensor  and there is a pattern that I see emerging in these data acquisition applications  so you have a local component where it very much matters whether the computer is like a Raspberry Pi or an Arduino or something  because it has to be right next to the device because of the low range of the sensor.  Then there's a cloud component which actually analyzes the data and that does not matter where it is  just that it is connected over a network and the user interface is connected also to cloud and local  and displays the data and user can interact with the analysis.  In my case, the local component is just an Async I.O. data server that I've written.  It communicates over Bluetooth with this thing here and it also babysits the connection  so if I switch it off and switch it on again then it will reconnect and try to fix errors.  That's quite nice because I have that separated from the iPython kernel which is the cloud component  because it's difficult for the user to deal with all the exceptions that occur in Bluetooth  and we have most of the data processing inside the kernel then.  The kernel pushes data to the user interface which is a browser.  In an iPython notebook like this one you have the server, the kernel and the user interface  and they are somewhat separated but you need a server for Python anyway.  Now I'm going to quickly show you a real-time demonstration.  This is junk data because nothing was connected and now if we are lucky  I will be silent for a moment so that you can appreciate normal brainwaves.  The problem with EEG also is that the brainwaves themselves are so, so weak that pretty much  everything going on inside or in your head, all the muscles, the facial muscles and eye muscles  all contribute artifacts to the data and are stronger than the actual brainwave data.  I can show you that I can clench my teeth which are like the strongest muscle in the head.  I can also use my eyes to...  Yeah, and when I blink.  And that makes EEG signals very difficult to analyze.  Also this is a bokeh graph.  I can use iPython widgets to push data from the kernel to the notebook and also the kernel  is running an I think IO loop internally so that it can communicate over the web socket  with my data server which handles the Bluetooth stuff.  Now I have stopped and then I can go to the next slide.  Now let's do some data science.  Data scientist is who data science does so we need some data for that and to get good data  we need experiments because discovery requires experimentation.  And this is a central idea in the physiology framework that is this experiment stuff.  The experiment is here initialized and then immediately used as a decorator much like  the iPython notebook interact one.  And this decorator when I start it, it will wait for data from the server and whenever  there is new data this handle message is called and it can then do its thing to do whatever.  In this case it just display clear output means the output is cleared and then some HTML is  written with the last attention data.  So attention is a value that is computed by the newest guy mind-face itself inside the headset.  And I can show you how that works and that is really all it takes.  So this is a value between zero and 100 currently it seems I am totally inattentive  but I can push that higher.  So in terms of time it doesn't work.  Now experiment classes have these attributes like attention as a time series.  So every second or so the such a value is computed by the device in send over Bluetooth  and I try to figure out when that was but it is a bit difficult to do that.  The time series stuff is really good and so it is just accumulating this experiment.  Also the raw data, the meditation data and several other things are also communicated  and stored.  Now that was one thing running experiments in real time but we also like to record data.  That is done by giving it a file name and that is a HDFI file.  In this case I think I have a bug because I have like one or two gigabytes of data  and that is completely crazy.  I have to fix that later.  In any case this is the same as before but it displays the amount of raw data which is  here, then experiment raw over 512 because the raw data is at 512 samples per second.  Now let's record some raw data.  I think it is not working currently.  In any case, now to resurrect the data we need a so called batch class and the batch class  can resurrect the data then.  Here we have recorded data from another session which just is much the same as the experiment.  I forgot to tell you what actually a batch is.  Every time I run the experiment I record a different batch.  First time batch 0, second time batch 1 and that way experiments don't overwrite their  own data and you can also try to vary the experimental conditions if you want to do that.  Now I would like to show you a simple experiment.  I don't have time for a lot of them so I would only do one.  The new Sky Mindwave computes these ESense meditation attention values.  They are patented and sort of back box algorithms where they don't really tell us what they  are doing there.  Here I am transforming the time series data to a table of features.  I don't have time to really explain what is going on there but I take windows of the time  series data then I compute how strong different brain wave frequencies are and in the same  row I also have the tension and the meditation values.  I wrote separate library which is table cleaner and we need that to clean some of the data  up because some of the data is bad.  As I told you before we have lots of artifacts in the data and we don't want to analyze artifacts.  We want to analyze real EEG data.  So this table cleaner library is inspired by jungle forms and cleans tabular data.  The nice thing is it outputs both a validated data set and a data set of the errors.  So on the bottom we see a table of the grouped validation errors and the most rows, 206 rows  were deleted because there was too much variation in the EEG data.  So when there is an artifact then we try to remove that row of the table.  Also poor signal is computed by the brain wave itself but that's not so important in this case.  So if you need data validation thing maybe you should look into it.  Now I'm doing the linear regression analysis.  So I said we have features of brain waves correlated with attentional meditation data  and the linear regression all it does is it tries to explain the tension value or the  meditation value in terms of the frequency strength.  So it multiplies each frequency with the coefficient and then they are all summed up.  And that way when you figure out the coefficient you know how important the different frequencies are for this value  and you probably can't read the labels on the graph but I didn't have time to make that any better.  What you may actually see is that we have here the best or the highest and most significant coefficients are at the...  What was that?  Are in the high beta or in the beta frequencies from 17 hertz upwards.  I can't see that but just believe me when I tell you that there is a certain range of frequencies  which is called beta frequencies between 17 and I think 30 hertz depending on the definition.  And they are strongly associated with all sorts of attention and concentration stuff.  So if you ever heard about the prefrontal cortex and the tension circuits of the brain  also ADHD that has a lot to do with with beta brain waves.  Now I'd like to meditate a bit about meditation.  There's a similar graph but we have more going on.  Here we have the alpha values which are between in this case are between like 8 and 10, around 8 and 10 hertz which are significant.  But also some beta values which are a bit undecided.  I think that's because most of the time I or others too need to do some concentration to enhance alpha values.  During baking stage you need to have a somewhat calm mind to actually exhibit alpha waves.  Or you close your eyes which is also easy but that's not what we want to do.  In any case these values are used in newer feedback.  So you can write an application that looks for attention or meditation and feed it back to you.  And then you can learn how to consciously manipulate these brain states.  That's what I was talking about last year at Europe, Pison and Berlin in 2014.  And there's also a YouTube recording of the talk where I go into more detail about the psychology and biology behind all this newer feedback stuff.  So this all sort of implies, so the linear regression implies that the strengths of the frequencies are not dependent on each other.  And I don't know why that.  I'm not going to get it to work.  In any case these are correlated and we see that the alpha and the beta values are correlated among each other.  But not between the alpha and the beta waves.  It has some biological meaning but I'm not going to go into that.  I'd like to make some technical remarks.  Bluetooth is really bad for timing.  So if you have Bluetooth data coming in, they often don't come with timestamps.  And these timestamps, you have to give them timestamps as the local component in the server, data server, and figure out which are the right timestamps.  And if you do it wrong, then you have like overlapping data and crossing your own timeline is strictly forbidden except for cheap tricks.  Yeah, I think I was very good for waiting and problem resolution.  I do like yield from weight and I yield from sleep and so I do so I can write these troubleshooting algorithms with reconnecting and I can wait for a few seconds and then reconnect again without disturbing anything else.  The combination of Bokeh and notebook widget really rocks because this notebook has its own web socket push communication channel which I can use to stream data to the JavaScript site and update the book charts.  And in general, Jupiter notebook is a really great tool to tell computational narratives. I rather skip that or rather went hastily through my little computational narrative about the brainwaves and about attention and meditation, but I didn't really have time for more.  But you can imagine that you can do real time experiment or show how the computation works and also explain or narrate how the data really works.  Also, it's a lot is the the biology framework is an example of a library for the notebook which really is bigger on the inside. So I just had this experiment decorator and entire heights all this web socket and async IO and other magic.  Mostly from the user.  I'd like to thank you for your attention. And my Twitter handle is page in horse which was a few years ago supposed to be a pun or Trojan horse or something but now it just sounds silly.  The library is by the ology. I just published it. It's very raw and you probably won't get it to work. Especially unless you have this device by the way resistance is futile.  This table cleaner library I try to make it a bit bigger because I have had a lot of trouble over the times validating CSV data. And that is sort of my attempt to fix that problem.  In this case, I wrote also this notebook assets library which I use for for turning coffee script into JavaScript and serving it's from the actual library itself and not from the extension stuff which I find a bit troublesome because the  notebook profile directory has no clear one on one direct connection to the Python library installation stuff. And it's basically all. Thank you for your attention and we have time for some questions maybe.  But you when you want to go to the lightning talks.  I tried to do some stuff with with the mine wave years ago. But I'm guessing that your framework didn't exist then because then I had to connect via some some proprietary Bluetooth executable that I had to wrap stuff up around so so  does your Python code do the Bluetooth stuff. Yes. Also, the current documentation for mine wave points to my older libraries for mine wave so it's a bit difficult to do and I think what the newest guy stuff is doing isn't really that developer friendly on the library  side. But the protocol is very friendly.  So if I want to get in one of these nearest guy things is that I just had a brief look at the web page is the nearest guy mind wave mobile which is the one to get. Yeah. Yes I think that's the best device as one that is a bit more expensive and more variable but I think the new sky  mine wave mobile is the best one for like 100 or 120 euros. You can also get it on Amazon and other shops and you only need the device itself. You don't need any of their software or anything. You just use that all. Okay. Great. Some people even wire up Arduino directly to the sink your ship.  So as we as we have some time. Would you mind to show the code for the demo part.  Maybe the demo itself. Yeah. By the way, this I want to publish this the notes but I wasn't able to to push it yet from this network and it contains a lot of notes of what I was talking about which are not visible in the presentation itself.  So here's a hidden code with the real time raw demo and but to to know what's going on here is this normal book a let's make a plot stuff and then it's the raw source book a model source column data source which is important to have a reference to it.  And then my touch bed.  So in the handle message one I am doing some re sampling and then I push the data as a very convert the data to this raw source data on this Python side. But the book a knows it's this book a objects know the ID on the browser also on the Python side so I push this ID here.  And then replace book a data source and it was a widget which has a communication between the Python side and the notebook side.  And then I have just this function replace book a data source which pushes this this data source with its ID and the new data to the JavaScript side and then it deconstructs it and puts that into the data source into the book a data source on the browser and then magically through different callbacks.  And then I have this new graph is just redrawn.  Thank you.  I have to reconnect again.  I'm asking to show the demo.  I'm sorry.  I'm so excited.  I work on the key to working now.  What is it?  What are the lines?  Can you most code me?  It's actually an easy abuse of this technology to do more scouting like I can also bring.  Yeah.
Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_0_1259.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_1260_4259.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_4260_8483.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_8484_8723.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_8724_14651.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_14652_16259.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_16260_16367.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_16368_16463.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_16464_16523.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_16524_16631.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_16632_17063.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_17064_17495.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_17496_20615.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_20616_21947.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_21948_22007.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_22008_22103.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_22104_23351.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_23352_31871.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_31872_32135.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_32136_35423.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_35424_36047.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_36048_38015.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38016_38063.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38064_38195.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38196_38243.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38244_38387.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38388_38531.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38532_38603.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38604_38723.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_38724_39047.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39048_39155.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39156_39203.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39204_39275.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39276_39395.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39396_39635.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39636_39803.png Andreas_Klostermann_-_Brainwaves_for_Hackers_2_0_39804_40674.png
Hello, my name is Andreas Klostermann and in this talk I will talk about brain waves and how we as Python hackers can use our tools to explore our own brains a bit. Last year at EuroPython I held the 1.0 version of this talk. This year I have only 30 minutes so I'm going to make extremely short thrift of the theory behind brain waves but we have to cover a bit of it so that everyone of you understands what we are doing here but the talks don't have a lot of overlap so this is really 2.0. Now first of all what are brain waves actually? Brain waves are an electro physiological potential that you measure on your scalp that basically is a voltage measurement which is measured over time with electrodes that are on the scalp and what you are measuring there is the summed potential of most or even all your brain cells which give off electrical signals to communicate with each other but also a bit of waste energy gets transmitted outwards and so you can measure it on the skin without having to go inside the scalp. Most of what the most useful analysis of brain waves usually is the Fourier transform which basically only means that we assume that the signal is composed of several frequencies and we want to know which frequency is dominant and by doing that we can infer something about what the brain is doing because we know when a certain frequency range is overrepresented and the subject is for example concentrated or relaxed or that kind of information. Now to measure this current we need an electro and a phallogram and these devices can be quite cumbersome and quite expensive but what I have here is basically the Neurosky Mindwave Mobile which is a Bluetooth connected headset and is really low powered and very optimized for developers and hackers. It does all the amplification stuff inside this headset and digitalizes the data. It also does some preliminary analysis of the data so that you could even connect this to an Arduino and the Arduino would be able to tell if you are concentrated or not. Aruinos are not really sophisticated processors or software in any case so it's quite nice. I wrote a Pyzeology framework and that is a framework for physiological experiments and analysis. It currently mostly only supports Mindwave EEG but eventually I also have code for ECG from Vitalino but that's not activated currently. It's powered by Psypydita and it's iPyzen Jupiter enabled so what I want to achieve is an experimentation platform that works for do-it-yourself people who don't have a lab fancy equipment or something but I want to use mainly these quantified self-type devices with Bluetooth connections or other near field stuff. It's currently Python 3.4 only and I'm making heavy use of Async I.O. Now you've probably heard of the Internet of Things. The basic idea is that you have local computers connected to some kind of sensor and there is a pattern that I see emerging in these data acquisition applications so you have a local component where it very much matters whether the computer is like a Raspberry Pi or an Arduino or something because it has to be right next to the device because of the low range of the sensor. Then there's a cloud component which actually analyzes the data and that does not matter where it is just that it is connected over a network and the user interface is connected also to cloud and local and displays the data and user can interact with the analysis. In my case, the local component is just an Async I.O. data server that I've written. It communicates over Bluetooth with this thing here and it also babysits the connection so if I switch it off and switch it on again then it will reconnect and try to fix errors. That's quite nice because I have that separated from the iPython kernel which is the cloud component because it's difficult for the user to deal with all the exceptions that occur in Bluetooth and we have most of the data processing inside the kernel then. The kernel pushes data to the user interface which is a browser. In an iPython notebook like this one you have the server, the kernel and the user interface and they are somewhat separated but you need a server for Python anyway. Now I'm going to quickly show you a real-time demonstration. This is junk data because nothing was connected and now if we are lucky I will be silent for a moment so that you can appreciate normal brainwaves. The problem with EEG also is that the brainwaves themselves are so, so weak that pretty much everything going on inside or in your head, all the muscles, the facial muscles and eye muscles all contribute artifacts to the data and are stronger than the actual brainwave data. I can show you that I can clench my teeth which are like the strongest muscle in the head. I can also use my eyes to... Yeah, and when I blink. And that makes EEG signals very difficult to analyze. Also this is a bokeh graph. I can use iPython widgets to push data from the kernel to the notebook and also the kernel is running an I think IO loop internally so that it can communicate over the web socket with my data server which handles the Bluetooth stuff. Now I have stopped and then I can go to the next slide. Now let's do some data science. Data scientist is who data science does so we need some data for that and to get good data we need experiments because discovery requires experimentation. And this is a central idea in the physiology framework that is this experiment stuff. The experiment is here initialized and then immediately used as a decorator much like the iPython notebook interact one. And this decorator when I start it, it will wait for data from the server and whenever there is new data this handle message is called and it can then do its thing to do whatever. In this case it just display clear output means the output is cleared and then some HTML is written with the last attention data. So attention is a value that is computed by the newest guy mind-face itself inside the headset. And I can show you how that works and that is really all it takes. So this is a value between zero and 100 currently it seems I am totally inattentive but I can push that higher. So in terms of time it doesn't work. Now experiment classes have these attributes like attention as a time series. So every second or so the such a value is computed by the device in send over Bluetooth and I try to figure out when that was but it is a bit difficult to do that. The time series stuff is really good and so it is just accumulating this experiment. Also the raw data, the meditation data and several other things are also communicated and stored. Now that was one thing running experiments in real time but we also like to record data. That is done by giving it a file name and that is a HDFI file. In this case I think I have a bug because I have like one or two gigabytes of data and that is completely crazy. I have to fix that later. In any case this is the same as before but it displays the amount of raw data which is here, then experiment raw over 512 because the raw data is at 512 samples per second. Now let's record some raw data. I think it is not working currently. In any case, now to resurrect the data we need a so called batch class and the batch class can resurrect the data then. Here we have recorded data from another session which just is much the same as the experiment. I forgot to tell you what actually a batch is. Every time I run the experiment I record a different batch. First time batch 0, second time batch 1 and that way experiments don't overwrite their own data and you can also try to vary the experimental conditions if you want to do that. Now I would like to show you a simple experiment. I don't have time for a lot of them so I would only do one. The new Sky Mindwave computes these ESense meditation attention values. They are patented and sort of back box algorithms where they don't really tell us what they are doing there. Here I am transforming the time series data to a table of features. I don't have time to really explain what is going on there but I take windows of the time series data then I compute how strong different brain wave frequencies are and in the same row I also have the tension and the meditation values. I wrote separate library which is table cleaner and we need that to clean some of the data up because some of the data is bad. As I told you before we have lots of artifacts in the data and we don't want to analyze artifacts. We want to analyze real EEG data. So this table cleaner library is inspired by jungle forms and cleans tabular data. The nice thing is it outputs both a validated data set and a data set of the errors. So on the bottom we see a table of the grouped validation errors and the most rows, 206 rows were deleted because there was too much variation in the EEG data. So when there is an artifact then we try to remove that row of the table. Also poor signal is computed by the brain wave itself but that's not so important in this case. So if you need data validation thing maybe you should look into it. Now I'm doing the linear regression analysis. So I said we have features of brain waves correlated with attentional meditation data and the linear regression all it does is it tries to explain the tension value or the meditation value in terms of the frequency strength. So it multiplies each frequency with the coefficient and then they are all summed up. And that way when you figure out the coefficient you know how important the different frequencies are for this value and you probably can't read the labels on the graph but I didn't have time to make that any better. What you may actually see is that we have here the best or the highest and most significant coefficients are at the... What was that? Are in the high beta or in the beta frequencies from 17 hertz upwards. I can't see that but just believe me when I tell you that there is a certain range of frequencies which is called beta frequencies between 17 and I think 30 hertz depending on the definition. And they are strongly associated with all sorts of attention and concentration stuff. So if you ever heard about the prefrontal cortex and the tension circuits of the brain also ADHD that has a lot to do with with beta brain waves. Now I'd like to meditate a bit about meditation. There's a similar graph but we have more going on. Here we have the alpha values which are between in this case are between like 8 and 10, around 8 and 10 hertz which are significant. But also some beta values which are a bit undecided. I think that's because most of the time I or others too need to do some concentration to enhance alpha values. During baking stage you need to have a somewhat calm mind to actually exhibit alpha waves. Or you close your eyes which is also easy but that's not what we want to do. In any case these values are used in newer feedback. So you can write an application that looks for attention or meditation and feed it back to you. And then you can learn how to consciously manipulate these brain states. That's what I was talking about last year at Europe, Pison and Berlin in 2014. And there's also a YouTube recording of the talk where I go into more detail about the psychology and biology behind all this newer feedback stuff. So this all sort of implies, so the linear regression implies that the strengths of the frequencies are not dependent on each other. And I don't know why that. I'm not going to get it to work. In any case these are correlated and we see that the alpha and the beta values are correlated among each other. But not between the alpha and the beta waves. It has some biological meaning but I'm not going to go into that. I'd like to make some technical remarks. Bluetooth is really bad for timing. So if you have Bluetooth data coming in, they often don't come with timestamps. And these timestamps, you have to give them timestamps as the local component in the server, data server, and figure out which are the right timestamps. And if you do it wrong, then you have like overlapping data and crossing your own timeline is strictly forbidden except for cheap tricks. Yeah, I think I was very good for waiting and problem resolution. I do like yield from weight and I yield from sleep and so I do so I can write these troubleshooting algorithms with reconnecting and I can wait for a few seconds and then reconnect again without disturbing anything else. The combination of Bokeh and notebook widget really rocks because this notebook has its own web socket push communication channel which I can use to stream data to the JavaScript site and update the book charts. And in general, Jupiter notebook is a really great tool to tell computational narratives. I rather skip that or rather went hastily through my little computational narrative about the brainwaves and about attention and meditation, but I didn't really have time for more. But you can imagine that you can do real time experiment or show how the computation works and also explain or narrate how the data really works. Also, it's a lot is the the biology framework is an example of a library for the notebook which really is bigger on the inside. So I just had this experiment decorator and entire heights all this web socket and async IO and other magic. Mostly from the user. I'd like to thank you for your attention. And my Twitter handle is page in horse which was a few years ago supposed to be a pun or Trojan horse or something but now it just sounds silly. The library is by the ology. I just published it. It's very raw and you probably won't get it to work. Especially unless you have this device by the way resistance is futile. This table cleaner library I try to make it a bit bigger because I have had a lot of trouble over the times validating CSV data. And that is sort of my attempt to fix that problem. In this case, I wrote also this notebook assets library which I use for for turning coffee script into JavaScript and serving it's from the actual library itself and not from the extension stuff which I find a bit troublesome because the notebook profile directory has no clear one on one direct connection to the Python library installation stuff. And it's basically all. Thank you for your attention and we have time for some questions maybe. But you when you want to go to the lightning talks. I tried to do some stuff with with the mine wave years ago. But I'm guessing that your framework didn't exist then because then I had to connect via some some proprietary Bluetooth executable that I had to wrap stuff up around so so does your Python code do the Bluetooth stuff. Yes. Also, the current documentation for mine wave points to my older libraries for mine wave so it's a bit difficult to do and I think what the newest guy stuff is doing isn't really that developer friendly on the library side. But the protocol is very friendly. So if I want to get in one of these nearest guy things is that I just had a brief look at the web page is the nearest guy mind wave mobile which is the one to get. Yeah. Yes I think that's the best device as one that is a bit more expensive and more variable but I think the new sky mine wave mobile is the best one for like 100 or 120 euros. You can also get it on Amazon and other shops and you only need the device itself. You don't need any of their software or anything. You just use that all. Okay. Great. Some people even wire up Arduino directly to the sink your ship. So as we as we have some time. Would you mind to show the code for the demo part. Maybe the demo itself. Yeah. By the way, this I want to publish this the notes but I wasn't able to to push it yet from this network and it contains a lot of notes of what I was talking about which are not visible in the presentation itself. So here's a hidden code with the real time raw demo and but to to know what's going on here is this normal book a let's make a plot stuff and then it's the raw source book a model source column data source which is important to have a reference to it. And then my touch bed. So in the handle message one I am doing some re sampling and then I push the data as a very convert the data to this raw source data on this Python side. But the book a knows it's this book a objects know the ID on the browser also on the Python side so I push this ID here. And then replace book a data source and it was a widget which has a communication between the Python side and the notebook side. And then I have just this function replace book a data source which pushes this this data source with its ID and the new data to the JavaScript side and then it deconstructs it and puts that into the data source into the book a data source on the browser and then magically through different callbacks. And then I have this new graph is just redrawn. Thank you. I have to reconnect again. I'm asking to show the demo. I'm sorry. I'm so excited. I work on the key to working now. What is it? What are the lines? Can you most code me? It's actually an easy abuse of this technology to do more scouting like I can also bring. Yeah.
Hello, my name is Andreas Klostermann and in this talk I will talk about brain waves and how we as I have here is basically the Neurosky Mindwave fancy equipment or something but I want to use with EEG also is that the brainwaves themselves that you can do real time experiment or show how attention and we have time for some questions device as one that is a bit more expensive and