The Internet of Shit Goes to Court - When Smart Devices Betray Their Owners
Kaltheuner, Frederike
Wood, Millie Graham
Can smart devices solve crimes - or is IoT the new CCTV? An all-female introduction to IoT privacy, security and the use of smart devices as evidence in criminal investigations.
 Music Will your fridge determine your innocence? Will your hairbrush prove you're guilty? Or will your smartphone contain data that you weren't even aware of in the first place? Technology is embedded into the fabric of society. Our devices, networks and platforms are generating massive amounts of data about each and every one of us. But we're not just talking about phones and laptops anymore. Evermore, our devices are connected to the Internet 24-7. Anything from fridges, toys, toasters, or even this is a smart diaper sensor. The Internet of Shit is of course not a name we came up with. This is a hilarious Twitter account that collects the most bizarre examples of attempts to put Internet and sensors on various devices. This is another example. This is the smart cup. My all-time favorite example is the smart hairbrush, which as you might not be aware of, has not just Bluetooth, Wi-Fi and a motion sensor, but also a microphone to measure and analyze the quality of your hair. The question we would like to ask is, do we really fully understand the consequences of the Internet of Things we're building? The implication for privacy are almost, particularly when people gain access to our data and devices that we're not supposed to. But we're not just talking about hacking or interception. In this talk, we want to zoom in and focus on one particular scenario. The case where smart devices and their data are used in criminal proceedings. Given the consequences, whenever you're part of a criminal investigation, the consequences are quite severe. There could be a heavy fine, you could get a criminal record, worse case, you could lose your liberty, reputation and so on. Given the severity of these consequences, we need to ask ourselves, can we, can and are our devices betraying us? The problem is, aside from a few headline-grubbing cases, the implications of the Internet of Shit going to court has received very little consideration. There's little information on the value of this evidence, once the cases proceed, the role it played, if any at all, to produce guilty verdicts, and the complex issues that arise when smart devices and their data are used as evidence. Another question is, what does trust even mean when we have smart devices in rented accommodation, when we have smart devices in our workplace or even public space? In this talk, we would like to make a simple case. We argue that the desire to obtain data from smart devices and the tools that are used to extract them far outstripped the compensation of the average person who uses and buys these devices. We will present a few very well-known cases and discuss some of the fundamental areas that we think deserve further discussion. That is, the fundamental informational asymmetry about what kind of data is generated and collected in the first place, and unequal access to that data, and finally, the accuracy and reliability of data. As we bring sensors into our bodies, into our homes, into our offices, we have very little understanding and oftentimes very little access about what kinds of data these devices generate, collect, process, or share. In a way, this could be the next level chilling effect. Anything you ever do can and will be used against you. So, how does the Internet of Shit end up in court? A widely reported case involves Amazon Echo. If you look at the court documents, you will see that the accused actually had a wide range of connected devices, including smart meter, NEST thermometer, honey, well-alarm system, door monitoring motion sensors, and remote-activated lighting. But we're just going to focus on the Amazon Echo. So, what happened in this case? In November 2015, a man in Arkansas had some friends over to his house to watch a football match. The next morning, one friend was found dead in the hot tub in the backyard. Police charged the man with murder. He pleaded not guilty. As the police were investigating the crime scene, detectives learned that music had been streamed onto the back patio at the time of death, which could have been activated and controlled via the Amazon Echo. A preservation request was served on Amazon, and they retained the data. Two further search warrants were served. The police believed that Amazon was in possessions of records related to the homicide. These were specifically audio recordings and transcriptions, text records, and other related communications of the device in the accused home. They believed these were on Amazon servers and other computer hardware maintained by Amazon. They also sought a long list of what can be classed as subscriber information and billing records. A further search warrant was served, which Fredericka will touch on later, which wasn't served on Amazon, but was directed at the device itself. So, an Amazon Echo, I'm sure many of you are familiar, this is about a nine-inch voice-controlled speaker, seven microphones, and beam-forming technology to pick up sound from anywhere in a room, even when music is playing. The always-on Echo speaker makes recordings of audio it hears from a fraction of a second before it hears the activation word, either Alexa or Amazon, until it judges that the command be over. When it detects the wake word, the Echo connects remotely to the Alexa voice service, Amazon's intelligent personal assistant, and transmits the audio to the Alexa voice service in the cloud to process and respond. Although no recordings are meant to be made at other times, the device is often activated by accident if it misinterprets a wake command. We know, as was widely publicized, that in February this year, Amazon sought to quash the warrant, resisting the request for audio recordings and the transcripts. They argued that the user's speech and transcript was protected under the First Amendment. Given the privacy implications at stake, the warrant, Amazon argued, was not specific enough, referring to the fear of government tracking and censoring, and one's reading, listening, and viewing choices, a heightened showing of relevance and compelling need, as well as sufficient nexus between the information and subject of the criminal investigation was required. As part of its submissions, Amazon claims no audio recording of the user's request is stored on the device. What had the potential to be a fascinating side battle in the case, however, was short-lived, when the defendants agreed to hand over the information retained on his Amazon Echo. We don't know, as far as I'm aware, what the police extracted, what data they have, and how it's being used. However, what this case does raise is the importance of data on internet devices, of which the police seem very alive to. Let's look at another case. This involved the fitness tracker Fitbit. A man was accused of killing his wife in Connecticut, America. He told the police that a masked assailant came into the couple's suburban home at 9am, after she'd been to the gym. The husband was subdued using pressure points. His gun was then taken by the assailant, and his wife was shot, immediately after she'd walked into the house. However, the Fitbit tells a different story. The digital pedometer, which tracks the wearer's steps, shows the victim moving around the house an hour after her husband said the murder took place. It also shows she travelled 1,200 feet after arriving home, contrary to the accused's story that she was murdered as soon as she walked in the door. It is not clear how the Fitbit data was obtained, whether by a warrant to the company or the accused, but this case clearly presents, as it was presented in the media, a seductive view of the magic of the internet of things in solving crimes. If you look at these cases, it is not a surprise to see why law enforcement is interested in data from connected devices. In a way, the murders portrayed or the fantasy that we're creating a world in which not just our speech becomes traceable, but also our actions and our movements. This in the background is an actual slide from a Microsoft presentation on the connected officer. This is the world we're creating. You're surrounded by connected things. In fact, the digital forensic chief of Scotland, Mark Stroke, argued in January this year that IoT will revolutionise criminal investigations. Police officers in the UK are trained to identify digital footprints, and the police are also developing a tool that would allow police to go into homes and automatically distract data from a variety of devices so that the police doesn't have to take you to a fridge or other devices in custody. As you might not be surprised, also the NSA is interested in the internet of things, not just your toaster, but also biomedical devices. If government access to your pacemaker sounds futuristic, here's an actual example of the police using data from a suspect's pacemaker. A man in Ohio was faced with charges of aggravated arson and insurance fraud for a fire at his home. When asked questions, he said that he managed to grab a whole load of stuff, throw it out his window, follow after it, take it onto the front lawn, and that's how his prize possessions were saved. The police found this a bit suspicious, as well as the fact that there was gasoline on his clothes. They decided to seek the data from his cardiac device. A cardiologist found it highly improbable that the accused would have been able to collect, pack, remove a number of items from the home, exit his window, and carry numerous heavy items onto the front of his house during the short time indicated. The fact that police can access data from devices that we put in the intimacy of our home, or even our bodies, sounds privacy invasive. And this is exactly how all of these cases were reported by the media. Headlines like, your fridge is spying on you, or your toaster can be a snitch. What is fascinating though is that all the cases we mentioned so far, and those are the ones that are predominantly reported by the media, are ones where electronic evidence is used to charge somebody of a crime. The data from smart devices is used as evidence to become a... for the case made by the prosecution. However, wouldn't it be interesting for the sake of the argument to think that this might actually be useful? Wouldn't it be amazing if you could use Internet of Things data to prove that the guilty are guilty and to free the innocent? We're creating a world of senses, or this idea that the Internet of Things will have your back, that everything is traceable, that we're surrounded by impartial electronic witnesses, from which each of us can benefit, actually has an unlikely historical precedent. This is the project that Steve Mann, a research and artist, conducted throughout the 80s and 90s, and those are devices that continuously track all of his behavior. So in a sense you could say that the father of wearable, how he is called, has created a device that could continuously create evidence. When he got into an argument, he could go back in time and say, actually that's not what I said. Steve Mann is also the person who coined the term surveillance, which means surveillance from below. A scenario where you're not just being surveilled from the top, but you yourself have devices that also record evidence that you can use whenever you're accused, or whenever you need to defend yourself. While this parallel is tempting, I think it is important to notice that there is a decisive difference. The sensors we are surrounding ourselves with are not an art project, they're not a research project. They are fundamentally beyond our control. Our thought, speech and behavior is being data-fied by a commercial ecosystem, in which we often do product, not the customer. We are not always the owner of the data that we generate. And in the following, we like to look at three problems that are a consequence of this fundamentally commercial ecosystem. On the one hand, informational asymmetry, unequal access, as well as accuracy and reliability of these data. So let's go back to the Amazon Echo case. The way in which this case was reported was mostly about the very fact that Amazon Echo is listening. You could say in a parody, some articles quoted a parody saying, probably the murderer went to the Amazon Echo and said, Amazon or Alexa, how can I bury a body? This is of course not how this evidence is being used. The entire rationale behind the surge warrant of the Amazon Echo is that any devices that are constantly on are constantly generating data. The Amazon Echo has seven microphones. Whenever the device is on, these devices are generating data. The idea is that it only gets recorded whenever you say the actuation word. And the case of the police was that in the midst of the action, maybe potentially the echo got turned on by accident. We think this itself is problematic, because it means that the devices you're surrounded with might or might not record whatever you're doing and that these evidences might or might not be used against you. However, there's a fascinating second dimension to this case, which is that there's a fundamental disagreement about what kind of data is stored where. If you look at the code documents, this is from Amazon's request to squash the warrant. Amazon claims that there's no data on the device itself. In fact, however, the police issued two surge warrant. The first surge warrant was for data that is on Amazon's servers. And since Amazon didn't comply, the police issued a second warrant to search the actual device that was already in the police possession. Here's a quote from the police. We have learned that the device is capable of storing data, including voice recordings. Amazon thinks that's wrong and says, the police were able to extract the data. And in the footnote, it says that would not have included any voice recordings from Alexa, because such data is stored remotely. If you look at the Amazon Echo itself, any connected smart devices are obviously not built to store massive amounts of data. The entire idea behind smart intelligent devices is that the data is processed and stored remotely. However, we don't know what the police rationale was, but it would be reasonable to assume that any device has sort of like a sliding window in which voice data is recorded and stored on the device in the case that there's an internet outage or other kinds of problems in the moment of transmission. So it's not really clear what kinds of data are on the device, also because the device has no interface. It's connected to a smartphone, but you as a user cannot access the data that's on the device. But regardless what is actually there, we here have a more investigation where a suspect is put on trial and he does not know whether or not his device stores data that may or may not incriminate him. This is a second example that proves the point that there might be data on devices that we don't know about. A couple of years ago, our colleague Richard Tynan was present when GCHQ, the British Intelligence Signals Intelligence Service, instructed journalists from the Guardian to destroy their laptops that contained the snow and documents. He gave a fascinating talk about this at the CCC camp if you want to look at it, but the point is here that everybody who was present in the room was extremely surprised by how precise the GCHQ instructions were. The journalists were not just instructed to destroy the hard drive, they were instructed to destroy all sorts of part of the MacBook in that case, which means that in case of very sensitive information, it's one, extremely difficult to actually delete and destroy data, and secondly, data is stored all over the device at places that GCHQ knows about, however, the people who are actually owning and using the laptop don't know. So now we are at the level of the device. There is an informational asymmetry about what kinds of data are on a device. This continues to the level of the platform. This wasn't the news recently, I assume many of you have read this. The scandal was that Uber is collecting location data even after the app was deleted, which is worth repeating. You delete an app, but the app is still recording your location data, which means that it is stored somewhere and it can be subpoena, potentially. But I think this informational asymmetry about the kinds of data that exist about us is very important to us, and it's also important to us to be able to connect with our users and to make them more abstract level as well, which is whenever you combine data, we get a picture of ourselves that is even more ourselves than we are, or that we know about ourselves. Every single one of us is creating a much more detailed data shadow of our thoughts, feelings, and behavior than we are aware of. We are done with everything you have ever searched for and everything you have ever said to anyone. It's of course not true, but it's an exaggeration. So in a way, your phone is more you than you are yourself. And technology is not just recording, technology is increasingly becoming a conversation partner. This year in the background, a couple of years ago, AOL was hacked or data was leaked about the search history of millions of its users, and I encourage you to watch this fascinating documentary which sort of like constructs a narrative from the search history of a single user who at some point searched I Love Alaska. So the bottom line is this, we don't know what data is generated, collected, processed, and stored about us, and I would like to stress that this is not always malicious. There can be a discrepancy between the way a product is specified to work and how it actually works in its implementation. What we call this excessiveness of generation beyond our knowledge and control data exploitation. There's another important argument why the Internet of Things may not help us establish our innocence. We have unequal access to the data that is collected about each and every one of us. We already know there is unequal access to digital evidence, whether it's police body-worn cameras, CCTV, or ANPR. Individuals and their lawyers can spend years arguing over disclosure and what they can or can't see in relation to evidence against them. Throw into this mix the Internet of Things, and you're adding another layer of complexity which will not only result in unequal access to the raw data, but also the ability to understand it and to interpret it. If the government can access a wide range of data on you, from your own or publicly placed Internet of Things devices, all those tiny bits indicate your political views, your sexual preferences, your health. Individually they may have little meaning, but pieced together they could turn into the living, breathing, real you into what we call your digital doppelganger, without your consent or control. Perhaps this could be used to determine whether you're guilty or not. And what happens when the doppelganger is inaccurate, for example, when you're sentenced? Decisions are made on dodgy data. When considering unequal access, let's recap on some of the examples about how and where smart devices collect, store, and process data. Firstly, on the smart devices and sensors themselves. Secondly, hardware and software. And thirdly, for example, on the cloud. So looking at data on the device, as we've argued, the users of smart devices don't know what kinds of data these devices actually store. There's also the technical problem. The actual core data on devices is often in proprietary formats. You need some level of analysis to understand what it means. This is a place where smart devices can leave traces on the hardware and software that provides communication between smart devices and the external world. This is most probably on your phone. Most smart devices are connected to apps that in turn collect and process data. Amazon, for example, offers a cache of recordings that have been made by the device on your phone. One of the most reported cases about access to data on phones was Apple FBI case in America following the San Bernardino shooting. Apple famously rejected the government search warrant to build a new operating system that would allow the FBI access to an encrypted phone. However, it's highly likely that in the vast majority of gases throughout the world, the police are able to extract the data. Not just that, they are able to extract data that we don't even know is there. Let's look at one of the tools they use. What we can see here are police agents using digital forensics. Such services are offered by companies like Celebrite. The FBI's go-to hackers for mobile forensics. Celebrite's forensics tools include the universal forensic extraction device, which is hardware bundled together with proprietary software that acquires, decodes, analyzes data from smartphones, tablets and portable GPS devices. What we perhaps don't know or don't know enough about is that tools like Celebrite can collect data that the user does not know is on their device. This can include deleted SMS messages as well as cool histories and deleted collected by the phone and apps. And then we've got a training manual for Celebrite technology that the Denver police provided to journalists in the US. It clearly shows that sophisticated tools can access deleted data. And so unless the user buys these tools and knows how to use them, she has no way to access the data on her device, but the police do. This imbalance of power relating to our devices should concern us. Now let's look at data in the cloud. Most IoT data will be uploaded to the cloud. And whilst Amazon challenged the request for data on its services too broad, the data was obtained from the defendant, so we don't know where the line is to be drawn, a battle yet to come. However, it is likely that some of the companies do comply with these requests without resistance. And only a few companies produce transparency reports such as Nest or Fitbit. But what about other manufacturers and companies? A further aspect is what data is provided to the individual. How easy is it for you to get your own data from the company or from the authorities? And finally, let's have a think about quality. The final aspect is the premise that your device can portray you, and this relates to the quality of evidence. All data from IoT devices involve issues around accuracy, authenticity and admissibility. Using evidence can be technically challenging and expensive. We know, for example, that DNA evidence was initially seen as proof, but now its value is highly contested. And what about your fitness tracker? What might that say about your emotional state? What can your smart appliance say about the number of people in a room? That all requires interpretation. There's a familiar expression in the UK that justice is open for all, just like the Ritz Hotel. The Ritz is an extremely expensive and exclusive hotel. In theory, yes, anyone can go in, but in practice, only few have real access. The same is true of the legal system. When the poorest and most vulnerable in our society are charged with serious offenses, how are they to afford an expert? They might be able to obtain the raw data, but what use is that? If only the prosecution can present the digital evidence by their expert, is that fair? What are the implications? Will the defendant even understand if the data is wrong, and will they know what to challenge? Let's have a think about the Fitbit case. What if the accused was telling the truth? Would anyone believe him if he said he thought the dog took the Fitbit, ran around the house, and his evidence was real? Without expert analysis, I sincerely doubt it. It's still early days, and we will wait to see how evidence from IoT devices is used, analyzed, and perhaps found to be flawed, manipulated, or unreliable. It's also fascinating to see that there are tools coming up that make it insanely easy to fake data on IoT devices, such as this is a Photoshop for audio sounds. If you can fabricate what somebody said, and then this is being recorded by a device, should this be evidence or not? I would just like to stress that we're not against technology. Technological innovation in the past has been used to sort of like to bring, to re-examine existing cases, to uncover some evidence that was lost, and to charge people, or to free people who were unjustly charged with crimes. However, and this is important to notice, not all DNA cases were actually cases where there was a lack of evidence. For example, sometimes evidence was missing, or police just simply fucked up an investigation, or police failed to disclose certain evidence. Currently, in Massachusetts, 21,000 drug cases could be dismissed because a former police chemist falsified tests. Here's the second problem. In the UK, 24-year-old Sam Hallen served seven years in prison following a flawed investigation that failed to follow basic lines of inquiry. What we're saying is that technology and more data are not going to solve existing problems in the criminal justice system. Instead, they will add an additional level of complexity, and they come with their own problems and challenges. It simply won't be the case that we will be surrounded by technological witnesses that will have our back. The idea that we're surrounded by objective, unemotional, electronic witnesses that are freeing the innocent and incriminating the guilty is clearly an exaggeration. We believe that this idea is fundamentally flawed. However, our position is not that we shouldn't use technology to help fight crime, but we need to understand the full consequences, including the limitations that these technologies have. We also don't think that we shouldn't cooperate with law enforcement quite the opposite. However, in order to cooperate, we need to know what kind of evidence exists about us. You have a right to protect yourself from self-incrimination. If you don't know what kind of data might or might not be on your device, you might potentially incriminate yourself. The data on the device might be inaccurate or misleading, and you wouldn't even know. So what we want is transparency around data. Right now, data is being generated, collected, processed, analyzed, and shared excessively, and to an extent that we don't fully understand. And this is a serious problem. In this talk, we addressed a very specific threat model, the narrow case in which police want access to your data. This is, of course, one of the many scenarios where your data on your device you don't know about can betray you. It is important to remember that we are generating even more data about us at a time when our technological infrastructures are fundamentally insecure. A few weeks ago, the company FlexiSpy was hacked. Under the pretext of offering legal spy tools for parents and employers, FlexiSpy has become a leading vendor of commercial surveillance tools. The company offers anything from call interception, SMS tracking, remotely activated microphones, and location tracking. Could similar services emerge for IoT devices? Will IoT be a security nightmare or a signal as intelligent bonanza? It's probably both. And this has implications for its value as evidence, which we need to discuss before the police race too far ahead. While the Internet of Things is relatively new, this approach by the authorities is not a game changer, but part of a gradual creep of new technology into our lives, society, and our legal systems. What is new, or at least not discussed enough, is that our devices and devices around us may be completely beyond our control. We have no guarantee or knowledge what devices are collecting and storing. We do not have total access to the data on our own devices, and we cannot always delete it. This may be seen as an advantage, for example, in a criminal case in the US or Germany. But what if you're a gay activist in Uganda, stopped at border control? The consequences for your activism and if they discover your sexuality are serious. We need to know what the authorities can access on our devices. We need awareness of smart devices in public places and what they collect. With all the data being collected by the authorities, we need to know that they're using strong encryption. We need to think about privacy by design and product liability. But finally, we need to understand the limitations, the risks, the potential for electronic evidence being misleading or false. Thank you. Thank you very much. Thank you very much. I think there is some time for questions. So maybe we can have some light in the audience so I can see hands rising. Okay, at the back there is a question. I'm coming to you. Hello. Thank you for your talk. I was wondering when we're talking about different devices and maybe manufacturers and other intermediaries in the process, if we're looking at the global supply chain of these products, we can create governance and regulation domestically and best practices globally. But as far as the enforcement mechanisms in order to keep people secure and communications private, do you have any thoughts on kind of a framework for best practices for the actual enforcement of how we can make sure that the devices that we're using will have oversight and as well as like us as in consumers, is there a best kind of governing or regulatory body in which you feel would best suit it for emerging IoT space? There are several dimensions to this question. There's the burning question of the entire security of the IoT infrastructure. And there are several solutions that we are working on. One thing we think is important is product liability. Currently, manufacturers are not liable the moment you add software to a device and that is bizarre and has sort of like a strange incentive model. And then the other dimension is our right to access data on a device. There are mechanisms in, for example, European data protection law to access, to get access to your data. But a lot of times this stops the moment you approach the sphere of the device. And we think it would be quite interesting to test how far this goes. You see I'm wearing one now. So we are practicing what we preach and looking to find out what data is in the wings on the various things we... Millie's Fitbentzing. So basically, it would be interesting that there's a need for more research to establish what kind of data is on a device as well as legal proceedings where you find out, hey manufacturer, please tell me what is on the device. Okay, are there any more questions? Okay, I see two hands. Very good, I'm coming. Thank you for the presentation. It was really interesting. I just have one very short question. Do you have any cases where Internet of Things devices were involved in press freedom or human rights related cases, so not necessarily criminal investigations like maybe human rights related cases? I think that's something that our researchers look at. We produce country specific reports. We do particular investigations into certain places. If we speak to people, human rights defenders in those countries who are raising those issues, then it would be in those reports. I'm not sure of any particular ones. There's various things to do with mobile phone extraction, which we hear, but in terms of actually people having a nest in their home where there are particular human rights abuses, maybe they don't have a roll out of smart meters there. There's simply not a lot of cases out yet. All of these cases you see, these cover crimes that happened in 2015, that was the time when all of these devices hit the mass consumer market, mostly in the US. And I think these cases might be coming up in the future. Okay, another question here. Do you think that apart from data protection law, there should be other legal provisions made, especially in criminal procedure law or so on, to prevent authorities from accessing the data? Yes. I don't think data protection law, I don't think the GDPR, different provisions they have around the world are going to be able to deal with these specific issues which may be interpreted very different nationally depending on your legal system. And I think there needs to be very specific laws, I need to think the police need very specific guidance, I think there needs to be very specific oversight because we see the authorities push the boundaries. And if we're going back to say the GDPR, that's just not, I don't think that's going to be sufficient, particularly with criminal investigations. Data protection doesn't protect you, this is not applicable in the case where you have a search warrant from a police, but the question is, under which conditions can the police have a search warrant for your device? And when is such a warrant acceptable and when is it not? Okay, I see another question in the front. Oh, true, that's great. I'm coming. Thank you for the interesting talk. With regards to transparency, I was wondering what role do you see for open source software, open source hardware and hackable hardware in this respect? Of course, if you build it yourself, you have a much larger control over what your device is doing. So yes, I think the fact that a lot of connected devices are outsourcing the entire processing to servers, this is the entire privacy nightmare of the Internet of Things, your data is flowing around the world, which means that when it comes down to it, people can subpoena these data in various places, absolutely, in an open source product that's less of a problem. However, then you also still have security issues, right? And we focused about the police here, but it's interesting to see in the Flaxi Spy case, the customers are not just regular, creepy people who want to spy on their spouse, but the customers are also, for example, banks. A lot of times you have investigations going on in companies where there are certain complaints and you have to investigate what actually happened. This is sort of like the spectrum of legal proceedings that we were talking about. Hi, thank you for your talk, first of all. And you mentioned quite a few times now the GDPR, I don't know if all of you here know what it is, the European Data Protection Law which comes into force in May 2018. And this law contains a lot of the transparency rules. You just ask for, like, that they need to declare which data someone is collecting, where is it stored, to whom is it given, and so on and so on. And I don't know how deep you're into that law, but if you're a bit, do you think this will have an impact? Because if companies do not apply to these laws, they will be fined, and with millions and millions of euros. So do you think there might be an impact, or is it just that you say, well, I don't know because I don't know enough about that? Without putting it on the spot, we have an expert in the audience on the GDPR. So I think all of these cases are from the US, and it is interesting to see how these things pan out in a legal system that does protect privacy, but according to a very different logic. If you have a system where you need no legal grounds for processing, and the processing in addition doesn't have to be also fair and not excessive, you have a scenario where data is extremely leaky, that is the US case. Data is being shared, sold in a much, in a way that is fundamentally different from the way that it is happening in the European Union. It's also a different criminal justice system, so the comparisons are sort of difficult to make. Not in the US case, you see that sort of like, insurances, whenever you have a criminal proceedings, insurances can subpoena data from all sorts of places. So I think like, Internet of Things security is not just sort of like, this is addressed also in the ePrivacy directive, and there are lots of important changes that are happening, but I think the particular case where the police has a warrant, that's a different scenario. If you would get something like the GDPR in the US, would you from your perspective would say, oh yes, that would help all of us? Or would you just from your perspective say, oh my god, no, all the transparency rules and so on? So our organizational position is that data protection is important, and data protection law is very important, because it regulates the processing of data, how it can be obtained and how it can be shared. So in that sense, sort of like, it limits the entire excessiveness to a certain extent. I think one interesting thing to do as well, if you're looking to see how far companies are going to comply, is just to go to some of the seminars and conferences that a lot of lawyers that represent these companies are putting on, I've been to a few in London and it's very much about how to comply to the minimum with GDPR, so if you want to really know, go and listen to how they're going to do it. That's a very good tip. Are there any more questions? We still have another five minutes or so, so, okay, I'm seeing one here. Hi. I wonder, would you buy an Amazon Echo or do you have a fitness armrest? Do you use that tools or are you afraid of that tools personally? I've got a Fitbit, I've got a Nest, but my intentions are to use them to do subject access requests, so I guess I don't trust them. I think a lot of people who speak to people at Privacy International say, aren't you terrified of everything you do? And that kind of goes to when people talk about what's their threat model. I don't think the GCHQ are trying to hack into my phone, but I am worried about other surveillance measures there are. I think it's good to be wary of devices. I think we need to challenge the companies. Privacy International would certainly encourage people to, for example, do subjects access requests. We can't wait when we read about people challenging, for example, Uber, finding out what data there. We can't do it all. It's up to people who own these devices to challenge them. Obviously, there are a lot of people out there, whether it's family members, your grandparents, who have no idea about the data collected, even if you bring up location data, they find that shocking. I wouldn't say I trust them. Maybe I've got an ulterior motive. It's quite laudable that Nest has a transparency report. It would be interesting to see more transparency reports for more manufacturers of smart devices to see how many warrants do they get and how many requests from law enforcement do they comply with. Any more questions? That doesn't seem like. Thank you very much. Very enlightening talk. Very thoughtful. Please give a round of applause to Frederica and Milly.
 Music  Will your fridge determine your innocence?  Will your hairbrush prove you're guilty?  Or will your smartphone contain data  that you weren't even aware of in the first place?  Technology is embedded into the fabric of society.  Our devices, networks and platforms  are generating massive amounts of data  about each and every one of us.  But we're not just talking about phones and laptops anymore.  Evermore, our devices are connected to the Internet 24-7.  Anything from fridges, toys, toasters,  or even this is a smart diaper sensor.  The Internet of Shit is of course not a name we came up with.  This is a hilarious Twitter account  that collects the most bizarre examples  of attempts to put Internet and sensors on various devices.  This is another example. This is the smart cup.  My all-time favorite example is the smart hairbrush,  which as you might not be aware of,  has not just Bluetooth, Wi-Fi and a motion sensor,  but also a microphone to measure and analyze the quality of your hair.  The question we would like to ask is,  do we really fully understand the consequences  of the Internet of Things we're building?  The implication for privacy are almost,  particularly when people gain access to our data and devices  that we're not supposed to.  But we're not just talking about hacking or interception.  In this talk, we want to zoom in and focus on one particular scenario.  The case where smart devices and their data  are used in criminal proceedings.  Given the consequences, whenever you're part of a criminal investigation,  the consequences are quite severe.  There could be a heavy fine, you could get a criminal record,  worse case, you could lose your liberty, reputation and so on.  Given the severity of these consequences,  we need to ask ourselves,  can we, can and are our devices betraying us?  The problem is, aside from a few headline-grubbing cases,  the implications of the Internet of Shit going to court  has received very little consideration.  There's little information on the value of this evidence,  once the cases proceed,  the role it played, if any at all, to produce guilty verdicts,  and the complex issues that arise when smart devices and their data are used as evidence.  Another question is, what does trust even mean  when we have smart devices in rented accommodation,  when we have smart devices in our workplace or even public space?  In this talk, we would like to make a simple case.  We argue that the desire to obtain data from smart devices  and the tools that are used to extract them  far outstripped the compensation of the average person who uses and buys these devices.  We will present a few very well-known cases  and discuss some of the fundamental areas that we think deserve further discussion.  That is, the fundamental informational asymmetry  about what kind of data is generated and collected in the first place,  and unequal access to that data,  and finally, the accuracy and reliability of data.  As we bring sensors into our bodies, into our homes, into our offices,  we have very little understanding and oftentimes very little access  about what kinds of data these devices generate, collect, process, or share.  In a way, this could be the next level chilling effect.  Anything you ever do can and will be used against you.  So, how does the Internet of Shit end up in court?  A widely reported case involves Amazon Echo.  If you look at the court documents, you will see that the accused  actually had a wide range of connected devices,  including smart meter, NEST thermometer, honey, well-alarm system,  door monitoring motion sensors, and remote-activated lighting.  But we're just going to focus on the Amazon Echo.  So, what happened in this case?  In November 2015, a man in Arkansas had some friends over to his house  to watch a football match.  The next morning, one friend was found dead in the hot tub in the backyard.  Police charged the man with murder.  He pleaded not guilty.  As the police were investigating the crime scene,  detectives learned that music had been streamed onto the back patio  at the time of death, which could have been activated  and controlled via the Amazon Echo.  A preservation request was served on Amazon, and they retained the data.  Two further search warrants were served.  The police believed that Amazon was in possessions of records  related to the homicide.  These were specifically audio recordings and transcriptions,  text records, and other related communications  of the device in the accused home.  They believed these were on Amazon servers  and other computer hardware maintained by Amazon.  They also sought a long list of what can be classed as subscriber information  and billing records.  A further search warrant was served, which Fredericka will touch on later,  which wasn't served on Amazon, but was directed at the device itself.  So, an Amazon Echo, I'm sure many of you are familiar,  this is about a nine-inch voice-controlled speaker,  seven microphones, and beam-forming technology  to pick up sound from anywhere in a room, even when music is playing.  The always-on Echo speaker makes recordings of audio it hears  from a fraction of a second before it hears the activation word,  either Alexa or Amazon,  until it judges that the command be over.  When it detects the wake word,  the Echo connects remotely to the Alexa voice service,  Amazon's intelligent personal assistant,  and transmits the audio to the Alexa voice service in the cloud  to process and respond.  Although no recordings are meant to be made at other times,  the device is often activated by accident  if it misinterprets a wake command.  We know, as was widely publicized,  that in February this year,  Amazon sought to quash the warrant,  resisting the request for audio recordings and the transcripts.  They argued that the user's speech and transcript  was protected under the First Amendment.  Given the privacy implications at stake,  the warrant, Amazon argued, was not specific enough,  referring to the fear of government tracking and censoring,  and one's reading, listening, and viewing choices,  a heightened showing of relevance and compelling need,  as well as sufficient nexus between the information  and subject of the criminal investigation was required.  As part of its submissions,  Amazon claims no audio recording of the user's request  is stored on the device.  What had the potential to be a fascinating side battle in the case,  however, was short-lived,  when the defendants agreed to hand over the information  retained on his Amazon Echo.  We don't know, as far as I'm aware,  what the police extracted,  what data they have, and how it's being used.  However, what this case does raise  is the importance of data on internet devices,  of which the police seem very alive to.  Let's look at another case.  This involved the fitness tracker Fitbit.  A man was accused of killing his wife in Connecticut, America.  He told the police that a masked assailant  came into the couple's suburban home at 9am,  after she'd been to the gym.  The husband was subdued using pressure points.  His gun was then taken by the assailant,  and his wife was shot,  immediately after she'd walked into the house.  However, the Fitbit tells a different story.  The digital pedometer, which tracks the wearer's steps,  shows the victim moving around the house  an hour after her husband said the murder took place.  It also shows she travelled 1,200 feet after arriving home,  contrary to the accused's story  that she was murdered as soon as she walked in the door.  It is not clear how the Fitbit data was obtained,  whether by a warrant to the company or the accused,  but this case clearly presents, as it was presented in the media,  a seductive view of the magic of the internet of things in solving crimes.  If you look at these cases,  it is not a surprise to see why law enforcement is interested  in data from connected devices.  In a way, the murders portrayed or the fantasy  that we're creating a world in which not just our speech becomes traceable,  but also our actions and our movements.  This in the background is an actual slide from a Microsoft presentation  on the connected officer.  This is the world we're creating.  You're surrounded by connected things.  In fact, the digital forensic chief of Scotland, Mark Stroke,  argued in January this year that IoT will revolutionise criminal investigations.  Police officers in the UK are trained to identify digital footprints,  and the police are also developing a tool that would allow police  to go into homes and automatically distract data from a variety of devices  so that the police doesn't have to take you to a fridge or other devices in custody.  As you might not be surprised, also the NSA is interested in the internet of things,  not just your toaster, but also biomedical devices.  If government access to your pacemaker sounds futuristic,  here's an actual example of the police using data from a suspect's pacemaker.  A man in Ohio was faced with charges of aggravated arson  and insurance fraud for a fire at his home.  When asked questions, he said that he managed to grab a whole load of stuff,  throw it out his window, follow after it, take it onto the front lawn,  and that's how his prize possessions were saved.  The police found this a bit suspicious,  as well as the fact that there was gasoline on his clothes.  They decided to seek the data from his cardiac device.  A cardiologist found it highly improbable  that the accused would have been able to collect, pack,  remove a number of items from the home, exit his window,  and carry numerous heavy items onto the front of his house  during the short time indicated.  The fact that police can access data from devices  that we put in the intimacy of our home, or even our bodies,  sounds privacy invasive.  And this is exactly how all of these cases were reported by the media.  Headlines like, your fridge is spying on you,  or your toaster can be a snitch.  What is fascinating though is that all the cases we mentioned so far,  and those are the ones that are predominantly reported by the media,  are ones where electronic evidence is used to charge somebody of a crime.  The data from smart devices is used as evidence  to become a... for the case made by the prosecution.  However, wouldn't it be interesting for the sake of the argument  to think that this might actually be useful?  Wouldn't it be amazing if you could use Internet of Things data  to prove that the guilty are guilty and to free the innocent?  We're creating a world of senses, or this idea that the Internet of Things  will have your back, that everything is traceable,  that we're surrounded by impartial electronic witnesses,  from which each of us can benefit,  actually has an unlikely historical precedent.  This is the project that Steve Mann, a research and artist,  conducted throughout the 80s and 90s,  and those are devices that continuously track all of his behavior.  So in a sense you could say that the father of wearable, how he is called,  has created a device that could continuously create evidence.  When he got into an argument, he could go back in time and say,  actually that's not what I said.  Steve Mann is also the person who coined the term surveillance,  which means surveillance from below.  A scenario where you're not just being surveilled from the top,  but you yourself have devices that also record evidence  that you can use whenever you're accused,  or whenever you need to defend yourself.  While this parallel is tempting,  I think it is important to notice that there is a decisive difference.  The sensors we are surrounding ourselves with are not an art project,  they're not a research project.  They are fundamentally beyond our control.  Our thought, speech and behavior is being data-fied by a commercial ecosystem,  in which we often do product, not the customer.  We are not always the owner of the data that we generate.  And in the following, we like to look at three problems  that are a consequence of this fundamentally commercial ecosystem.  On the one hand, informational asymmetry, unequal access,  as well as accuracy and reliability of these data.  So let's go back to the Amazon Echo case.  The way in which this case was reported  was mostly about the very fact that Amazon Echo is listening.  You could say in a parody, some articles quoted a parody saying,  probably the murderer went to the Amazon Echo and said,  Amazon or Alexa, how can I bury a body?  This is of course not how this evidence is being used.  The entire rationale behind the surge warrant of the Amazon Echo  is that any devices that are constantly on are constantly generating data.  The Amazon Echo has seven microphones.  Whenever the device is on, these devices are generating data.  The idea is that it only gets recorded whenever you say the actuation word.  And the case of the police was that in the midst of the action,  maybe potentially the echo got turned on by accident.  We think this itself is problematic,  because it means that the devices you're surrounded with  might or might not record whatever you're doing  and that these evidences might or might not be used against you.  However, there's a fascinating second dimension to this case,  which is that there's a fundamental disagreement  about what kind of data is stored where.  If you look at the code documents, this is from Amazon's request to squash the warrant.  Amazon claims that there's no data on the device itself.  In fact, however, the police issued two surge warrant.  The first surge warrant was for data that is on Amazon's servers.  And since Amazon didn't comply,  the police issued a second warrant to search the actual device  that was already in the police possession.  Here's a quote from the police.  We have learned that the device is capable of storing data,  including voice recordings.  Amazon thinks that's wrong and says,  the police were able to extract the data.  And in the footnote, it says that would not have included any voice recordings from Alexa,  because such data is stored remotely.  If you look at the Amazon Echo itself,  any connected smart devices are obviously not built to store massive amounts of data.  The entire idea behind smart intelligent devices  is that the data is processed and stored remotely.  However, we don't know what the police rationale was,  but it would be reasonable to assume that any device has sort of like a sliding window  in which voice data is recorded and stored on the device  in the case that there's an internet outage  or other kinds of problems in the moment of transmission.  So it's not really clear what kinds of data are on the device,  also because the device has no interface.  It's connected to a smartphone,  but you as a user cannot access the data that's on the device.  But regardless what is actually there,  we here have a more investigation where a suspect is put on trial  and he does not know whether or not his device stores data  that may or may not incriminate him.  This is a second example that proves the point  that there might be data on devices that we don't know about.  A couple of years ago, our colleague Richard Tynan was present  when GCHQ, the British Intelligence Signals Intelligence Service,  instructed journalists from the Guardian to destroy their laptops  that contained the snow and documents.  He gave a fascinating talk about this at the CCC camp if you want to look at it,  but the point is here that everybody who was present in the room  was extremely surprised by how precise the GCHQ instructions were.  The journalists were not just instructed to destroy the hard drive,  they were instructed to destroy all sorts of part of the MacBook in that case,  which means that in case of very sensitive information,  it's one, extremely difficult to actually delete and destroy data,  and secondly, data is stored all over the device at places that GCHQ knows about,  however, the people who are actually owning and using the laptop don't know.  So now we are at the level of the device.  There is an informational asymmetry about what kinds of data are on a device.  This continues to the level of the platform.  This wasn't the news recently, I assume many of you have read this.  The scandal was that Uber is collecting location data  even after the app was deleted, which is worth repeating.  You delete an app, but the app is still recording your location data,  which means that it is stored somewhere and it can be subpoena, potentially.  But I think this informational asymmetry about the kinds of data that exist about us  is very important to us, and it's also important to us to be able to connect  with our users and to make them more abstract level as well,  which is whenever you combine data, we get a picture of ourselves  that is even more ourselves than we are, or that we know about ourselves.  Every single one of us is creating a much more detailed data shadow  of our thoughts, feelings, and behavior than we are aware of.  We are done with everything you have ever searched for and everything you have ever said to anyone.  It's of course not true, but it's an exaggeration.  So in a way, your phone is more you than you are yourself.  And technology is not just recording, technology is increasingly becoming a conversation partner.  This year in the background, a couple of years ago, AOL was hacked  or data was leaked about the search history of millions of its users,  and I encourage you to watch this fascinating documentary which sort of like constructs a narrative  from the search history of a single user who at some point searched I Love Alaska.  So the bottom line is this, we don't know what data is generated, collected,  processed, and stored about us, and I would like to stress that this is not always malicious.  There can be a discrepancy between the way a product is specified to work  and how it actually works in its implementation.  What we call this excessiveness of generation beyond our knowledge and control data exploitation.  There's another important argument why the Internet of Things may not help us establish our innocence.  We have unequal access to the data that is collected about each and every one of us.  We already know there is unequal access to digital evidence,  whether it's police body-worn cameras, CCTV, or ANPR.  Individuals and their lawyers can spend years arguing over disclosure  and what they can or can't see in relation to evidence against them.  Throw into this mix the Internet of Things, and you're adding another layer of complexity  which will not only result in unequal access to the raw data,  but also the ability to understand it and to interpret it.  If the government can access a wide range of data on you,  from your own or publicly placed Internet of Things devices,  all those tiny bits indicate your political views, your sexual preferences, your health.  Individually they may have little meaning,  but pieced together they could turn into the living, breathing, real you  into what we call your digital doppelganger, without your consent or control.  Perhaps this could be used to determine whether you're guilty or not.  And what happens when the doppelganger is inaccurate, for example, when you're sentenced?  Decisions are made on dodgy data.  When considering unequal access, let's recap on some of the examples  about how and where smart devices collect, store, and process data.  Firstly, on the smart devices and sensors themselves.  Secondly, hardware and software. And thirdly, for example, on the cloud.  So looking at data on the device, as we've argued,  the users of smart devices don't know what kinds of data these devices actually store.  There's also the technical problem.  The actual core data on devices is often in proprietary formats.  You need some level of analysis to understand what it means.  This is a place where smart devices can leave traces on the hardware  and software that provides communication between smart devices and the external world.  This is most probably on your phone.  Most smart devices are connected to apps that in turn collect and process data.  Amazon, for example, offers a cache of recordings that have been made by the device on your phone.  One of the most reported cases about access to data on phones was  Apple FBI case in America following the San Bernardino shooting.  Apple famously rejected the government search warrant to build a new operating system  that would allow the FBI access to an encrypted phone.  However, it's highly likely that in the vast majority of gases throughout the world,  the police are able to extract the data.  Not just that, they are able to extract data that we don't even know is there.  Let's look at one of the tools they use.  What we can see here are police agents using digital forensics.  Such services are offered by companies like Celebrite.  The FBI's go-to hackers for mobile forensics.  Celebrite's forensics tools include the universal forensic extraction device,  which is hardware bundled together with proprietary software  that acquires, decodes, analyzes data from smartphones, tablets and portable GPS devices.  What we perhaps don't know or don't know enough about  is that tools like Celebrite can collect data that the user does not know is on their device.  This can include deleted SMS messages as well as cool histories  and deleted collected by the phone and apps.  And then we've got a training manual for Celebrite technology  that the Denver police provided to journalists in the US.  It clearly shows that sophisticated tools can access deleted data.  And so unless the user buys these tools and knows how to use them,  she has no way to access the data on her device, but the police do.  This imbalance of power relating to our devices should concern us.  Now let's look at data in the cloud.  Most IoT data will be uploaded to the cloud.  And whilst Amazon challenged the request for data on its services too broad,  the data was obtained from the defendant,  so we don't know where the line is to be drawn, a battle yet to come.  However, it is likely that some of the companies do comply with these requests without resistance.  And only a few companies produce transparency reports such as Nest or Fitbit.  But what about other manufacturers and companies?  A further aspect is what data is provided to the individual.  How easy is it for you to get your own data from the company or from the authorities?  And finally, let's have a think about quality.  The final aspect is the premise that your device can portray you,  and this relates to the quality of evidence.  All data from IoT devices involve issues around accuracy, authenticity and admissibility.  Using evidence can be technically challenging and expensive.  We know, for example, that DNA evidence was initially seen as proof,  but now its value is highly contested.  And what about your fitness tracker?  What might that say about your emotional state?  What can your smart appliance say about the number of people in a room?  That all requires interpretation.  There's a familiar expression in the UK that justice is open for all, just like the Ritz Hotel.  The Ritz is an extremely expensive and exclusive hotel.  In theory, yes, anyone can go in, but in practice, only few have real access.  The same is true of the legal system.  When the poorest and most vulnerable in our society are charged with serious offenses,  how are they to afford an expert?  They might be able to obtain the raw data, but what use is that?  If only the prosecution can present the digital evidence by their expert, is that fair?  What are the implications?  Will the defendant even understand if the data is wrong, and will they know what to challenge?  Let's have a think about the Fitbit case.  What if the accused was telling the truth?  Would anyone believe him if he said he thought the dog took the Fitbit, ran around the house,  and his evidence was real?  Without expert analysis, I sincerely doubt it.  It's still early days, and we will wait to see how evidence from IoT devices is used,  analyzed, and perhaps found to be flawed, manipulated, or unreliable.  It's also fascinating to see that there are tools coming up that make it insanely easy to fake data on IoT devices,  such as this is a Photoshop for audio sounds.  If you can fabricate what somebody said, and then this is being recorded by a device,  should this be evidence or not?  I would just like to stress that we're not against technology.  Technological innovation in the past has been used to sort of like to bring, to re-examine existing cases,  to uncover some evidence that was lost, and to charge people, or to free people who were unjustly charged with crimes.  However, and this is important to notice, not all DNA cases were actually cases where there was a lack of evidence.  For example, sometimes evidence was missing, or police just simply fucked up an investigation,  or police failed to disclose certain evidence.  Currently, in Massachusetts, 21,000 drug cases could be dismissed because a former police chemist falsified tests.  Here's the second problem.  In the UK, 24-year-old Sam Hallen served seven years in prison following a flawed investigation that failed to follow basic lines of inquiry.  What we're saying is that technology and more data are not going to solve existing problems in the criminal justice system.  Instead, they will add an additional level of complexity, and they come with their own problems and challenges.  It simply won't be the case that we will be surrounded by technological witnesses that will have our back.  The idea that we're surrounded by objective, unemotional, electronic witnesses that are freeing the innocent and incriminating the guilty is clearly an exaggeration.  We believe that this idea is fundamentally flawed.  However, our position is not that we shouldn't use technology to help fight crime, but we need to understand the full consequences, including the limitations that these technologies have.  We also don't think that we shouldn't cooperate with law enforcement quite the opposite.  However, in order to cooperate, we need to know what kind of evidence exists about us.  You have a right to protect yourself from self-incrimination.  If you don't know what kind of data might or might not be on your device, you might potentially incriminate yourself.  The data on the device might be inaccurate or misleading, and you wouldn't even know.  So what we want is transparency around data.  Right now, data is being generated, collected, processed, analyzed, and shared excessively, and to an extent that we don't fully understand.  And this is a serious problem.  In this talk, we addressed a very specific threat model, the narrow case in which police want access to your data.  This is, of course, one of the many scenarios where your data on your device you don't know about can betray you.  It is important to remember that we are generating even more data about us at a time when our technological infrastructures are fundamentally insecure.  A few weeks ago, the company FlexiSpy was hacked.  Under the pretext of offering legal spy tools for parents and employers, FlexiSpy has become a leading vendor of commercial surveillance tools.  The company offers anything from call interception, SMS tracking, remotely activated microphones, and location tracking.  Could similar services emerge for IoT devices?  Will IoT be a security nightmare or a signal as intelligent bonanza?  It's probably both.  And this has implications for its value as evidence, which we need to discuss before the police race too far ahead.  While the Internet of Things is relatively new, this approach by the authorities is not a game changer,  but part of a gradual creep of new technology into our lives, society, and our legal systems.  What is new, or at least not discussed enough, is that our devices and devices around us may be completely beyond our control.  We have no guarantee or knowledge what devices are collecting and storing.  We do not have total access to the data on our own devices, and we cannot always delete it.  This may be seen as an advantage, for example, in a criminal case in the US or Germany.  But what if you're a gay activist in Uganda, stopped at border control?  The consequences for your activism and if they discover your sexuality are serious.  We need to know what the authorities can access on our devices.  We need awareness of smart devices in public places and what they collect.  With all the data being collected by the authorities, we need to know that they're using strong encryption.  We need to think about privacy by design and product liability.  But finally, we need to understand the limitations, the risks, the potential for electronic evidence being misleading or false.  Thank you.  Thank you very much.  Thank you very much. I think there is some time for questions.  So maybe we can have some light in the audience so I can see hands rising.  Okay, at the back there is a question. I'm coming to you.  Hello. Thank you for your talk.  I was wondering when we're talking about different devices and maybe manufacturers and other intermediaries in the process,  if we're looking at the global supply chain of these products, we can create governance and regulation domestically and best practices globally.  But as far as the enforcement mechanisms in order to keep people secure and communications private,  do you have any thoughts on kind of a framework for best practices for the actual enforcement of how we can make sure that the devices that we're using will have oversight  and as well as like us as in consumers, is there a best kind of governing or regulatory body in which you feel would best suit it for emerging IoT space?  There are several dimensions to this question. There's the burning question of the entire security of the IoT infrastructure.  And there are several solutions that we are working on.  One thing we think is important is product liability.  Currently, manufacturers are not liable the moment you add software to a device and that is bizarre and has sort of like a strange incentive model.  And then the other dimension is our right to access data on a device. There are mechanisms in, for example, European data protection law to access, to get access to your data.  But a lot of times this stops the moment you approach the sphere of the device. And we think it would be quite interesting to test how far this goes.  You see I'm wearing one now. So we are practicing what we preach and looking to find out what data is in the wings on the various things we...  Millie's Fitbentzing.  So basically, it would be interesting that there's a need for more research to establish what kind of data is on a device as well as legal proceedings where you find out,  hey manufacturer, please tell me what is on the device.  Okay, are there any more questions? Okay, I see two hands. Very good, I'm coming.  Thank you for the presentation. It was really interesting. I just have one very short question. Do you have any cases where Internet of Things devices were involved in press freedom or human rights related cases,  so not necessarily criminal investigations like maybe human rights related cases?  I think that's something that our researchers look at. We produce country specific reports. We do particular investigations into certain places.  If we speak to people, human rights defenders in those countries who are raising those issues, then it would be in those reports. I'm not sure of any particular ones. There's various things to do with mobile phone extraction, which we hear,  but in terms of actually people having a nest in their home where there are particular human rights abuses, maybe they don't have a roll out of smart meters there.  There's simply not a lot of cases out yet. All of these cases you see, these cover crimes that happened in 2015, that was the time when all of these devices hit the mass consumer market, mostly in the US.  And I think these cases might be coming up in the future.  Okay, another question here.  Do you think that apart from data protection law, there should be other legal provisions made, especially in criminal procedure law or so on, to prevent authorities from accessing the data?  Yes.  I don't think data protection law, I don't think the GDPR, different provisions they have around the world are going to be able to deal with these specific issues which may be interpreted very different nationally depending on your legal system.  And I think there needs to be very specific laws, I need to think the police need very specific guidance, I think there needs to be very specific oversight because we see the authorities push the boundaries.  And if we're going back to say the GDPR, that's just not, I don't think that's going to be sufficient, particularly with criminal investigations.  Data protection doesn't protect you, this is not applicable in the case where you have a search warrant from a police, but the question is, under which conditions can the police have a search warrant for your device?  And when is such a warrant acceptable and when is it not?  Okay, I see another question in the front.  Oh, true, that's great.  I'm coming.  Thank you for the interesting talk.  With regards to transparency, I was wondering what role do you see for open source software, open source hardware and hackable hardware in this respect?  Of course, if you build it yourself, you have a much larger control over what your device is doing.  So yes, I think the fact that a lot of connected devices are outsourcing the entire processing to servers, this is the entire privacy nightmare of the Internet of Things,  your data is flowing around the world, which means that when it comes down to it, people can subpoena these data in various places, absolutely, in an open source product that's less of a problem.  However, then you also still have security issues, right?  And we focused about the police here, but it's interesting to see in the Flaxi Spy case, the customers are not just regular, creepy people who want to spy on their spouse,  but the customers are also, for example, banks. A lot of times you have investigations going on in companies where there are certain complaints and you have to investigate what actually happened.  This is sort of like the spectrum of legal proceedings that we were talking about.  Hi, thank you for your talk, first of all.  And you mentioned quite a few times now the GDPR, I don't know if all of you here know what it is, the European Data Protection Law which comes into force in May 2018.  And this law contains a lot of the transparency rules.  You just ask for, like, that they need to declare which data someone is collecting, where is it stored, to whom is it given, and so on and so on.  And I don't know how deep you're into that law, but if you're a bit, do you think this will have an impact?  Because if companies do not apply to these laws, they will be fined, and with millions and millions of euros.  So do you think there might be an impact, or is it just that you say, well, I don't know because I don't know enough about that?  Without putting it on the spot, we have an expert in the audience on the GDPR.  So I think all of these cases are from the US, and it is interesting to see how these things pan out in a legal system that does protect privacy, but according to a very different logic.  If you have a system where you need no legal grounds for processing, and the processing in addition doesn't have to be also fair and not excessive,  you have a scenario where data is extremely leaky, that is the US case.  Data is being shared, sold in a much, in a way that is fundamentally different from the way that it is happening in the European Union.  It's also a different criminal justice system, so the comparisons are sort of difficult to make.  Not in the US case, you see that sort of like, insurances, whenever you have a criminal proceedings, insurances can subpoena data from all sorts of places.  So I think like, Internet of Things security is not just sort of like, this is addressed also in the ePrivacy directive,  and there are lots of important changes that are happening, but I think the particular case where the police has a warrant, that's a different scenario.  If you would get something like the GDPR in the US, would you from your perspective would say, oh yes, that would help all of us?  Or would you just from your perspective say, oh my god, no, all the transparency rules and so on?  So our organizational position is that data protection is important, and data protection law is very important,  because it regulates the processing of data, how it can be obtained and how it can be shared.  So in that sense, sort of like, it limits the entire excessiveness to a certain extent.  I think one interesting thing to do as well, if you're looking to see how far companies are going to comply,  is just to go to some of the seminars and conferences that a lot of lawyers that represent these companies are putting on,  I've been to a few in London and it's very much about how to comply to the minimum with GDPR,  so if you want to really know, go and listen to how they're going to do it.  That's a very good tip. Are there any more questions? We still have another five minutes or so, so, okay, I'm seeing one here.  Hi. I wonder, would you buy an Amazon Echo or do you have a fitness armrest?  Do you use that tools or are you afraid of that tools personally?  I've got a Fitbit, I've got a Nest, but my intentions are to use them to do subject access requests,  so I guess I don't trust them.  I think a lot of people who speak to people at Privacy International say,  aren't you terrified of everything you do?  And that kind of goes to when people talk about what's their threat model.  I don't think the GCHQ are trying to hack into my phone, but I am worried about other surveillance measures there are.  I think it's good to be wary of devices. I think we need to challenge the companies.  Privacy International would certainly encourage people to, for example, do subjects access requests.  We can't wait when we read about people challenging, for example, Uber, finding out what data there.  We can't do it all. It's up to people who own these devices to challenge them.  Obviously, there are a lot of people out there, whether it's family members, your grandparents,  who have no idea about the data collected, even if you bring up location data, they find that shocking.  I wouldn't say I trust them. Maybe I've got an ulterior motive.  It's quite laudable that Nest has a transparency report.  It would be interesting to see more transparency reports for more manufacturers of smart devices  to see how many warrants do they get and how many requests from law enforcement do they comply with.  Any more questions?  That doesn't seem like.  Thank you very much. Very enlightening talk. Very thoughtful.  Please give a round of applause to Frederica and Milly.
re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__0_191.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__192_863.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__864_1499.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__1500_3839.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__3840_3911.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__3912_3959.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__3960_4007.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__4008_4079.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__4080_4223.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__4224_7091.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__7092_7151.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__7152_8579.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__8580_8639.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__8640_9071.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__9072_9743.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__9744_12131.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__12132_16763.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__16764_17843.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__17844_19895.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__19896_20471.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__20472_20615.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__20616_21563.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__21564_24179.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__24180_26027.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__26028_27371.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__27372_27791.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__27792_29867.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__29868_30275.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__30276_30935.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__30936_37763.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__37764_43043.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__43044_43091.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__43092_43355.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__43356_44087.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__44088_44567.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__44568_44999.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__45000_45179.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__45180_68855.png re_publica_2017_-_The_Internet_of_Shit_Goes_to_Court_-_When_Smart_Devices_Betray_Their_Owners_kbhzf__68856_68913.png
Music Will your fridge determine your innocence? Will your hairbrush prove you're guilty? Or will your smartphone contain data that you weren't even aware of in the first place? Technology is embedded into the fabric of society. Our devices, networks and platforms are generating massive amounts of data about each and every one of us. But we're not just talking about phones and laptops anymore. Evermore, our devices are connected to the Internet 24-7. Anything from fridges, toys, toasters, or even this is a smart diaper sensor. The Internet of Shit is of course not a name we came up with. This is a hilarious Twitter account that collects the most bizarre examples of attempts to put Internet and sensors on various devices. This is another example. This is the smart cup. My all-time favorite example is the smart hairbrush, which as you might not be aware of, has not just Bluetooth, Wi-Fi and a motion sensor, but also a microphone to measure and analyze the quality of your hair. The question we would like to ask is, do we really fully understand the consequences of the Internet of Things we're building? The implication for privacy are almost, particularly when people gain access to our data and devices that we're not supposed to. But we're not just talking about hacking or interception. In this talk, we want to zoom in and focus on one particular scenario. The case where smart devices and their data are used in criminal proceedings. Given the consequences, whenever you're part of a criminal investigation, the consequences are quite severe. There could be a heavy fine, you could get a criminal record, worse case, you could lose your liberty, reputation and so on. Given the severity of these consequences, we need to ask ourselves, can we, can and are our devices betraying us? The problem is, aside from a few headline-grubbing cases, the implications of the Internet of Shit going to court has received very little consideration. There's little information on the value of this evidence, once the cases proceed, the role it played, if any at all, to produce guilty verdicts, and the complex issues that arise when smart devices and their data are used as evidence. Another question is, what does trust even mean when we have smart devices in rented accommodation, when we have smart devices in our workplace or even public space? In this talk, we would like to make a simple case. We argue that the desire to obtain data from smart devices and the tools that are used to extract them far outstripped the compensation of the average person who uses and buys these devices. We will present a few very well-known cases and discuss some of the fundamental areas that we think deserve further discussion. That is, the fundamental informational asymmetry about what kind of data is generated and collected in the first place, and unequal access to that data, and finally, the accuracy and reliability of data. As we bring sensors into our bodies, into our homes, into our offices, we have very little understanding and oftentimes very little access about what kinds of data these devices generate, collect, process, or share. In a way, this could be the next level chilling effect. Anything you ever do can and will be used against you. So, how does the Internet of Shit end up in court? A widely reported case involves Amazon Echo. If you look at the court documents, you will see that the accused actually had a wide range of connected devices, including smart meter, NEST thermometer, honey, well-alarm system, door monitoring motion sensors, and remote-activated lighting. But we're just going to focus on the Amazon Echo. So, what happened in this case? In November 2015, a man in Arkansas had some friends over to his house to watch a football match. The next morning, one friend was found dead in the hot tub in the backyard. Police charged the man with murder. He pleaded not guilty. As the police were investigating the crime scene, detectives learned that music had been streamed onto the back patio at the time of death, which could have been activated and controlled via the Amazon Echo. A preservation request was served on Amazon, and they retained the data. Two further search warrants were served. The police believed that Amazon was in possessions of records related to the homicide. These were specifically audio recordings and transcriptions, text records, and other related communications of the device in the accused home. They believed these were on Amazon servers and other computer hardware maintained by Amazon. They also sought a long list of what can be classed as subscriber information and billing records. A further search warrant was served, which Fredericka will touch on later, which wasn't served on Amazon, but was directed at the device itself. So, an Amazon Echo, I'm sure many of you are familiar, this is about a nine-inch voice-controlled speaker, seven microphones, and beam-forming technology to pick up sound from anywhere in a room, even when music is playing. The always-on Echo speaker makes recordings of audio it hears from a fraction of a second before it hears the activation word, either Alexa or Amazon, until it judges that the command be over. When it detects the wake word, the Echo connects remotely to the Alexa voice service, Amazon's intelligent personal assistant, and transmits the audio to the Alexa voice service in the cloud to process and respond. Although no recordings are meant to be made at other times, the device is often activated by accident if it misinterprets a wake command. We know, as was widely publicized, that in February this year, Amazon sought to quash the warrant, resisting the request for audio recordings and the transcripts. They argued that the user's speech and transcript was protected under the First Amendment. Given the privacy implications at stake, the warrant, Amazon argued, was not specific enough, referring to the fear of government tracking and censoring, and one's reading, listening, and viewing choices, a heightened showing of relevance and compelling need, as well as sufficient nexus between the information and subject of the criminal investigation was required. As part of its submissions, Amazon claims no audio recording of the user's request is stored on the device. What had the potential to be a fascinating side battle in the case, however, was short-lived, when the defendants agreed to hand over the information retained on his Amazon Echo. We don't know, as far as I'm aware, what the police extracted, what data they have, and how it's being used. However, what this case does raise is the importance of data on internet devices, of which the police seem very alive to. Let's look at another case. This involved the fitness tracker Fitbit. A man was accused of killing his wife in Connecticut, America. He told the police that a masked assailant came into the couple's suburban home at 9am, after she'd been to the gym. The husband was subdued using pressure points. His gun was then taken by the assailant, and his wife was shot, immediately after she'd walked into the house. However, the Fitbit tells a different story. The digital pedometer, which tracks the wearer's steps, shows the victim moving around the house an hour after her husband said the murder took place. It also shows she travelled 1,200 feet after arriving home, contrary to the accused's story that she was murdered as soon as she walked in the door. It is not clear how the Fitbit data was obtained, whether by a warrant to the company or the accused, but this case clearly presents, as it was presented in the media, a seductive view of the magic of the internet of things in solving crimes. If you look at these cases, it is not a surprise to see why law enforcement is interested in data from connected devices. In a way, the murders portrayed or the fantasy that we're creating a world in which not just our speech becomes traceable, but also our actions and our movements. This in the background is an actual slide from a Microsoft presentation on the connected officer. This is the world we're creating. You're surrounded by connected things. In fact, the digital forensic chief of Scotland, Mark Stroke, argued in January this year that IoT will revolutionise criminal investigations. Police officers in the UK are trained to identify digital footprints, and the police are also developing a tool that would allow police to go into homes and automatically distract data from a variety of devices so that the police doesn't have to take you to a fridge or other devices in custody. As you might not be surprised, also the NSA is interested in the internet of things, not just your toaster, but also biomedical devices. If government access to your pacemaker sounds futuristic, here's an actual example of the police using data from a suspect's pacemaker. A man in Ohio was faced with charges of aggravated arson and insurance fraud for a fire at his home. When asked questions, he said that he managed to grab a whole load of stuff, throw it out his window, follow after it, take it onto the front lawn, and that's how his prize possessions were saved. The police found this a bit suspicious, as well as the fact that there was gasoline on his clothes. They decided to seek the data from his cardiac device. A cardiologist found it highly improbable that the accused would have been able to collect, pack, remove a number of items from the home, exit his window, and carry numerous heavy items onto the front of his house during the short time indicated. The fact that police can access data from devices that we put in the intimacy of our home, or even our bodies, sounds privacy invasive. And this is exactly how all of these cases were reported by the media. Headlines like, your fridge is spying on you, or your toaster can be a snitch. What is fascinating though is that all the cases we mentioned so far, and those are the ones that are predominantly reported by the media, are ones where electronic evidence is used to charge somebody of a crime. The data from smart devices is used as evidence to become a... for the case made by the prosecution. However, wouldn't it be interesting for the sake of the argument to think that this might actually be useful? Wouldn't it be amazing if you could use Internet of Things data to prove that the guilty are guilty and to free the innocent? We're creating a world of senses, or this idea that the Internet of Things will have your back, that everything is traceable, that we're surrounded by impartial electronic witnesses, from which each of us can benefit, actually has an unlikely historical precedent. This is the project that Steve Mann, a research and artist, conducted throughout the 80s and 90s, and those are devices that continuously track all of his behavior. So in a sense you could say that the father of wearable, how he is called, has created a device that could continuously create evidence. When he got into an argument, he could go back in time and say, actually that's not what I said. Steve Mann is also the person who coined the term surveillance, which means surveillance from below. A scenario where you're not just being surveilled from the top, but you yourself have devices that also record evidence that you can use whenever you're accused, or whenever you need to defend yourself. While this parallel is tempting, I think it is important to notice that there is a decisive difference. The sensors we are surrounding ourselves with are not an art project, they're not a research project. They are fundamentally beyond our control. Our thought, speech and behavior is being data-fied by a commercial ecosystem, in which we often do product, not the customer. We are not always the owner of the data that we generate. And in the following, we like to look at three problems that are a consequence of this fundamentally commercial ecosystem. On the one hand, informational asymmetry, unequal access, as well as accuracy and reliability of these data. So let's go back to the Amazon Echo case. The way in which this case was reported was mostly about the very fact that Amazon Echo is listening. You could say in a parody, some articles quoted a parody saying, probably the murderer went to the Amazon Echo and said, Amazon or Alexa, how can I bury a body? This is of course not how this evidence is being used. The entire rationale behind the surge warrant of the Amazon Echo is that any devices that are constantly on are constantly generating data. The Amazon Echo has seven microphones. Whenever the device is on, these devices are generating data. The idea is that it only gets recorded whenever you say the actuation word. And the case of the police was that in the midst of the action, maybe potentially the echo got turned on by accident. We think this itself is problematic, because it means that the devices you're surrounded with might or might not record whatever you're doing and that these evidences might or might not be used against you. However, there's a fascinating second dimension to this case, which is that there's a fundamental disagreement about what kind of data is stored where. If you look at the code documents, this is from Amazon's request to squash the warrant. Amazon claims that there's no data on the device itself. In fact, however, the police issued two surge warrant. The first surge warrant was for data that is on Amazon's servers. And since Amazon didn't comply, the police issued a second warrant to search the actual device that was already in the police possession. Here's a quote from the police. We have learned that the device is capable of storing data, including voice recordings. Amazon thinks that's wrong and says, the police were able to extract the data. And in the footnote, it says that would not have included any voice recordings from Alexa, because such data is stored remotely. If you look at the Amazon Echo itself, any connected smart devices are obviously not built to store massive amounts of data. The entire idea behind smart intelligent devices is that the data is processed and stored remotely. However, we don't know what the police rationale was, but it would be reasonable to assume that any device has sort of like a sliding window in which voice data is recorded and stored on the device in the case that there's an internet outage or other kinds of problems in the moment of transmission. So it's not really clear what kinds of data are on the device, also because the device has no interface. It's connected to a smartphone, but you as a user cannot access the data that's on the device. But regardless what is actually there, we here have a more investigation where a suspect is put on trial and he does not know whether or not his device stores data that may or may not incriminate him. This is a second example that proves the point that there might be data on devices that we don't know about. A couple of years ago, our colleague Richard Tynan was present when GCHQ, the British Intelligence Signals Intelligence Service, instructed journalists from the Guardian to destroy their laptops that contained the snow and documents. He gave a fascinating talk about this at the CCC camp if you want to look at it, but the point is here that everybody who was present in the room was extremely surprised by how precise the GCHQ instructions were. The journalists were not just instructed to destroy the hard drive, they were instructed to destroy all sorts of part of the MacBook in that case, which means that in case of very sensitive information, it's one, extremely difficult to actually delete and destroy data, and secondly, data is stored all over the device at places that GCHQ knows about, however, the people who are actually owning and using the laptop don't know. So now we are at the level of the device. There is an informational asymmetry about what kinds of data are on a device. This continues to the level of the platform. This wasn't the news recently, I assume many of you have read this. The scandal was that Uber is collecting location data even after the app was deleted, which is worth repeating. You delete an app, but the app is still recording your location data, which means that it is stored somewhere and it can be subpoena, potentially. But I think this informational asymmetry about the kinds of data that exist about us is very important to us, and it's also important to us to be able to connect with our users and to make them more abstract level as well, which is whenever you combine data, we get a picture of ourselves that is even more ourselves than we are, or that we know about ourselves. Every single one of us is creating a much more detailed data shadow of our thoughts, feelings, and behavior than we are aware of. We are done with everything you have ever searched for and everything you have ever said to anyone. It's of course not true, but it's an exaggeration. So in a way, your phone is more you than you are yourself. And technology is not just recording, technology is increasingly becoming a conversation partner. This year in the background, a couple of years ago, AOL was hacked or data was leaked about the search history of millions of its users, and I encourage you to watch this fascinating documentary which sort of like constructs a narrative from the search history of a single user who at some point searched I Love Alaska. So the bottom line is this, we don't know what data is generated, collected, processed, and stored about us, and I would like to stress that this is not always malicious. There can be a discrepancy between the way a product is specified to work and how it actually works in its implementation. What we call this excessiveness of generation beyond our knowledge and control data exploitation. There's another important argument why the Internet of Things may not help us establish our innocence. We have unequal access to the data that is collected about each and every one of us. We already know there is unequal access to digital evidence, whether it's police body-worn cameras, CCTV, or ANPR. Individuals and their lawyers can spend years arguing over disclosure and what they can or can't see in relation to evidence against them. Throw into this mix the Internet of Things, and you're adding another layer of complexity which will not only result in unequal access to the raw data, but also the ability to understand it and to interpret it. If the government can access a wide range of data on you, from your own or publicly placed Internet of Things devices, all those tiny bits indicate your political views, your sexual preferences, your health. Individually they may have little meaning, but pieced together they could turn into the living, breathing, real you into what we call your digital doppelganger, without your consent or control. Perhaps this could be used to determine whether you're guilty or not. And what happens when the doppelganger is inaccurate, for example, when you're sentenced? Decisions are made on dodgy data. When considering unequal access, let's recap on some of the examples about how and where smart devices collect, store, and process data. Firstly, on the smart devices and sensors themselves. Secondly, hardware and software. And thirdly, for example, on the cloud. So looking at data on the device, as we've argued, the users of smart devices don't know what kinds of data these devices actually store. There's also the technical problem. The actual core data on devices is often in proprietary formats. You need some level of analysis to understand what it means. This is a place where smart devices can leave traces on the hardware and software that provides communication between smart devices and the external world. This is most probably on your phone. Most smart devices are connected to apps that in turn collect and process data. Amazon, for example, offers a cache of recordings that have been made by the device on your phone. One of the most reported cases about access to data on phones was Apple FBI case in America following the San Bernardino shooting. Apple famously rejected the government search warrant to build a new operating system that would allow the FBI access to an encrypted phone. However, it's highly likely that in the vast majority of gases throughout the world, the police are able to extract the data. Not just that, they are able to extract data that we don't even know is there. Let's look at one of the tools they use. What we can see here are police agents using digital forensics. Such services are offered by companies like Celebrite. The FBI's go-to hackers for mobile forensics. Celebrite's forensics tools include the universal forensic extraction device, which is hardware bundled together with proprietary software that acquires, decodes, analyzes data from smartphones, tablets and portable GPS devices. What we perhaps don't know or don't know enough about is that tools like Celebrite can collect data that the user does not know is on their device. This can include deleted SMS messages as well as cool histories and deleted collected by the phone and apps. And then we've got a training manual for Celebrite technology that the Denver police provided to journalists in the US. It clearly shows that sophisticated tools can access deleted data. And so unless the user buys these tools and knows how to use them, she has no way to access the data on her device, but the police do. This imbalance of power relating to our devices should concern us. Now let's look at data in the cloud. Most IoT data will be uploaded to the cloud. And whilst Amazon challenged the request for data on its services too broad, the data was obtained from the defendant, so we don't know where the line is to be drawn, a battle yet to come. However, it is likely that some of the companies do comply with these requests without resistance. And only a few companies produce transparency reports such as Nest or Fitbit. But what about other manufacturers and companies? A further aspect is what data is provided to the individual. How easy is it for you to get your own data from the company or from the authorities? And finally, let's have a think about quality. The final aspect is the premise that your device can portray you, and this relates to the quality of evidence. All data from IoT devices involve issues around accuracy, authenticity and admissibility. Using evidence can be technically challenging and expensive. We know, for example, that DNA evidence was initially seen as proof, but now its value is highly contested. And what about your fitness tracker? What might that say about your emotional state? What can your smart appliance say about the number of people in a room? That all requires interpretation. There's a familiar expression in the UK that justice is open for all, just like the Ritz Hotel. The Ritz is an extremely expensive and exclusive hotel. In theory, yes, anyone can go in, but in practice, only few have real access. The same is true of the legal system. When the poorest and most vulnerable in our society are charged with serious offenses, how are they to afford an expert? They might be able to obtain the raw data, but what use is that? If only the prosecution can present the digital evidence by their expert, is that fair? What are the implications? Will the defendant even understand if the data is wrong, and will they know what to challenge? Let's have a think about the Fitbit case. What if the accused was telling the truth? Would anyone believe him if he said he thought the dog took the Fitbit, ran around the house, and his evidence was real? Without expert analysis, I sincerely doubt it. It's still early days, and we will wait to see how evidence from IoT devices is used, analyzed, and perhaps found to be flawed, manipulated, or unreliable. It's also fascinating to see that there are tools coming up that make it insanely easy to fake data on IoT devices, such as this is a Photoshop for audio sounds. If you can fabricate what somebody said, and then this is being recorded by a device, should this be evidence or not? I would just like to stress that we're not against technology. Technological innovation in the past has been used to sort of like to bring, to re-examine existing cases, to uncover some evidence that was lost, and to charge people, or to free people who were unjustly charged with crimes. However, and this is important to notice, not all DNA cases were actually cases where there was a lack of evidence. For example, sometimes evidence was missing, or police just simply fucked up an investigation, or police failed to disclose certain evidence. Currently, in Massachusetts, 21,000 drug cases could be dismissed because a former police chemist falsified tests. Here's the second problem. In the UK, 24-year-old Sam Hallen served seven years in prison following a flawed investigation that failed to follow basic lines of inquiry. What we're saying is that technology and more data are not going to solve existing problems in the criminal justice system. Instead, they will add an additional level of complexity, and they come with their own problems and challenges. It simply won't be the case that we will be surrounded by technological witnesses that will have our back. The idea that we're surrounded by objective, unemotional, electronic witnesses that are freeing the innocent and incriminating the guilty is clearly an exaggeration. We believe that this idea is fundamentally flawed. However, our position is not that we shouldn't use technology to help fight crime, but we need to understand the full consequences, including the limitations that these technologies have. We also don't think that we shouldn't cooperate with law enforcement quite the opposite. However, in order to cooperate, we need to know what kind of evidence exists about us. You have a right to protect yourself from self-incrimination. If you don't know what kind of data might or might not be on your device, you might potentially incriminate yourself. The data on the device might be inaccurate or misleading, and you wouldn't even know. So what we want is transparency around data. Right now, data is being generated, collected, processed, analyzed, and shared excessively, and to an extent that we don't fully understand. And this is a serious problem. In this talk, we addressed a very specific threat model, the narrow case in which police want access to your data. This is, of course, one of the many scenarios where your data on your device you don't know about can betray you. It is important to remember that we are generating even more data about us at a time when our technological infrastructures are fundamentally insecure. A few weeks ago, the company FlexiSpy was hacked. Under the pretext of offering legal spy tools for parents and employers, FlexiSpy has become a leading vendor of commercial surveillance tools. The company offers anything from call interception, SMS tracking, remotely activated microphones, and location tracking. Could similar services emerge for IoT devices? Will IoT be a security nightmare or a signal as intelligent bonanza? It's probably both. And this has implications for its value as evidence, which we need to discuss before the police race too far ahead. While the Internet of Things is relatively new, this approach by the authorities is not a game changer, but part of a gradual creep of new technology into our lives, society, and our legal systems. What is new, or at least not discussed enough, is that our devices and devices around us may be completely beyond our control. We have no guarantee or knowledge what devices are collecting and storing. We do not have total access to the data on our own devices, and we cannot always delete it. This may be seen as an advantage, for example, in a criminal case in the US or Germany. But what if you're a gay activist in Uganda, stopped at border control? The consequences for your activism and if they discover your sexuality are serious. We need to know what the authorities can access on our devices. We need awareness of smart devices in public places and what they collect. With all the data being collected by the authorities, we need to know that they're using strong encryption. We need to think about privacy by design and product liability. But finally, we need to understand the limitations, the risks, the potential for electronic evidence being misleading or false. Thank you. Thank you very much. Thank you very much. I think there is some time for questions. So maybe we can have some light in the audience so I can see hands rising. Okay, at the back there is a question. I'm coming to you. Hello. Thank you for your talk. I was wondering when we're talking about different devices and maybe manufacturers and other intermediaries in the process, if we're looking at the global supply chain of these products, we can create governance and regulation domestically and best practices globally. But as far as the enforcement mechanisms in order to keep people secure and communications private, do you have any thoughts on kind of a framework for best practices for the actual enforcement of how we can make sure that the devices that we're using will have oversight and as well as like us as in consumers, is there a best kind of governing or regulatory body in which you feel would best suit it for emerging IoT space? There are several dimensions to this question. There's the burning question of the entire security of the IoT infrastructure. And there are several solutions that we are working on. One thing we think is important is product liability. Currently, manufacturers are not liable the moment you add software to a device and that is bizarre and has sort of like a strange incentive model. And then the other dimension is our right to access data on a device. There are mechanisms in, for example, European data protection law to access, to get access to your data. But a lot of times this stops the moment you approach the sphere of the device. And we think it would be quite interesting to test how far this goes. You see I'm wearing one now. So we are practicing what we preach and looking to find out what data is in the wings on the various things we... Millie's Fitbentzing. So basically, it would be interesting that there's a need for more research to establish what kind of data is on a device as well as legal proceedings where you find out, hey manufacturer, please tell me what is on the device. Okay, are there any more questions? Okay, I see two hands. Very good, I'm coming. Thank you for the presentation. It was really interesting. I just have one very short question. Do you have any cases where Internet of Things devices were involved in press freedom or human rights related cases, so not necessarily criminal investigations like maybe human rights related cases? I think that's something that our researchers look at. We produce country specific reports. We do particular investigations into certain places. If we speak to people, human rights defenders in those countries who are raising those issues, then it would be in those reports. I'm not sure of any particular ones. There's various things to do with mobile phone extraction, which we hear, but in terms of actually people having a nest in their home where there are particular human rights abuses, maybe they don't have a roll out of smart meters there. There's simply not a lot of cases out yet. All of these cases you see, these cover crimes that happened in 2015, that was the time when all of these devices hit the mass consumer market, mostly in the US. And I think these cases might be coming up in the future. Okay, another question here. Do you think that apart from data protection law, there should be other legal provisions made, especially in criminal procedure law or so on, to prevent authorities from accessing the data? Yes. I don't think data protection law, I don't think the GDPR, different provisions they have around the world are going to be able to deal with these specific issues which may be interpreted very different nationally depending on your legal system. And I think there needs to be very specific laws, I need to think the police need very specific guidance, I think there needs to be very specific oversight because we see the authorities push the boundaries. And if we're going back to say the GDPR, that's just not, I don't think that's going to be sufficient, particularly with criminal investigations. Data protection doesn't protect you, this is not applicable in the case where you have a search warrant from a police, but the question is, under which conditions can the police have a search warrant for your device? And when is such a warrant acceptable and when is it not? Okay, I see another question in the front. Oh, true, that's great. I'm coming. Thank you for the interesting talk. With regards to transparency, I was wondering what role do you see for open source software, open source hardware and hackable hardware in this respect? Of course, if you build it yourself, you have a much larger control over what your device is doing. So yes, I think the fact that a lot of connected devices are outsourcing the entire processing to servers, this is the entire privacy nightmare of the Internet of Things, your data is flowing around the world, which means that when it comes down to it, people can subpoena these data in various places, absolutely, in an open source product that's less of a problem. However, then you also still have security issues, right? And we focused about the police here, but it's interesting to see in the Flaxi Spy case, the customers are not just regular, creepy people who want to spy on their spouse, but the customers are also, for example, banks. A lot of times you have investigations going on in companies where there are certain complaints and you have to investigate what actually happened. This is sort of like the spectrum of legal proceedings that we were talking about. Hi, thank you for your talk, first of all. And you mentioned quite a few times now the GDPR, I don't know if all of you here know what it is, the European Data Protection Law which comes into force in May 2018. And this law contains a lot of the transparency rules. You just ask for, like, that they need to declare which data someone is collecting, where is it stored, to whom is it given, and so on and so on. And I don't know how deep you're into that law, but if you're a bit, do you think this will have an impact? Because if companies do not apply to these laws, they will be fined, and with millions and millions of euros. So do you think there might be an impact, or is it just that you say, well, I don't know because I don't know enough about that? Without putting it on the spot, we have an expert in the audience on the GDPR. So I think all of these cases are from the US, and it is interesting to see how these things pan out in a legal system that does protect privacy, but according to a very different logic. If you have a system where you need no legal grounds for processing, and the processing in addition doesn't have to be also fair and not excessive, you have a scenario where data is extremely leaky, that is the US case. Data is being shared, sold in a much, in a way that is fundamentally different from the way that it is happening in the European Union. It's also a different criminal justice system, so the comparisons are sort of difficult to make. Not in the US case, you see that sort of like, insurances, whenever you have a criminal proceedings, insurances can subpoena data from all sorts of places. So I think like, Internet of Things security is not just sort of like, this is addressed also in the ePrivacy directive, and there are lots of important changes that are happening, but I think the particular case where the police has a warrant, that's a different scenario. If you would get something like the GDPR in the US, would you from your perspective would say, oh yes, that would help all of us? Or would you just from your perspective say, oh my god, no, all the transparency rules and so on? So our organizational position is that data protection is important, and data protection law is very important, because it regulates the processing of data, how it can be obtained and how it can be shared. So in that sense, sort of like, it limits the entire excessiveness to a certain extent. I think one interesting thing to do as well, if you're looking to see how far companies are going to comply, is just to go to some of the seminars and conferences that a lot of lawyers that represent these companies are putting on, I've been to a few in London and it's very much about how to comply to the minimum with GDPR, so if you want to really know, go and listen to how they're going to do it. That's a very good tip. Are there any more questions? We still have another five minutes or so, so, okay, I'm seeing one here. Hi. I wonder, would you buy an Amazon Echo or do you have a fitness armrest? Do you use that tools or are you afraid of that tools personally? I've got a Fitbit, I've got a Nest, but my intentions are to use them to do subject access requests, so I guess I don't trust them. I think a lot of people who speak to people at Privacy International say, aren't you terrified of everything you do? And that kind of goes to when people talk about what's their threat model. I don't think the GCHQ are trying to hack into my phone, but I am worried about other surveillance measures there are. I think it's good to be wary of devices. I think we need to challenge the companies. Privacy International would certainly encourage people to, for example, do subjects access requests. We can't wait when we read about people challenging, for example, Uber, finding out what data there. We can't do it all. It's up to people who own these devices to challenge them. Obviously, there are a lot of people out there, whether it's family members, your grandparents, who have no idea about the data collected, even if you bring up location data, they find that shocking. I wouldn't say I trust them. Maybe I've got an ulterior motive. It's quite laudable that Nest has a transparency report. It would be interesting to see more transparency reports for more manufacturers of smart devices to see how many warrants do they get and how many requests from law enforcement do they comply with. Any more questions? That doesn't seem like. Thank you very much. Very enlightening talk. Very thoughtful. Please give a round of applause to Frederica and Milly.
will revolutionise criminal investigations. The data from smart devices is used as evidence entire security of the IoT infrastructure. And processing, and the processing in addition