 Google has a lot of products. It also knows a lot about you. Google Now is a feature that has the potential to tie all that together in a way that's actually meaningful. They invited us to their Mountain View campus in California to talk to people behind the project and see where it's going in the future. We think the phone and more broadly kind of the computer that's with me, whether it's a phone or Sergei's glasses or something else, is pretty unique because it has basically the context of what I'm doing and where I'm at. So we've always had a search bar across the very top on Android from the very beginning. In Jelly Bean we made a few pretty significant changes to the search experience on Android. The first thing that we wanted to do and did was to go much deeper in answering questions. Directions to the museum with the William Paley exhibition. Getting directions. So it turns out it's at the DeYoung Museum. What's happening here is that there's a lot of information on the web that talks about this particular exhibition. And then it obviously talks about where it is and that's the DeYoung Museum in San Francisco. So when I say that, Google crosses all these various different pieces of information in my question to give me an answer back, which in this case is actually just a map. We're obviously very excited about search on mobile devices on Android and about the idea of people being able to interact with their phone in just a natural way. And if you think about what that takes, we feel like there's a few key elements that have to come together. One obviously is voice recognition. The second piece is once I have those words is natural language understanding. And the third piece, which is a fairly new thing for us at Google, is understanding what are the basic components and facts in the world. And this is something we call our knowledge graph. And it's really the first time in computer science history that those three things are all available. And not just available in some giant computational center, but available in a powerful device that's in my pocket. So one of the most important things from Larry's founder letter from earlier this year is the part where he says he thinks computers should do all the hard work for you. Sort of getting all the hard stuff out of the way so you can just enjoy life, enjoy whatever it is that you like to do. For example, if you have a flight reservation, you will have received that in your email. And you get to the hotel, you get to the restaurant, whatever. It's hard to find it. So what Google now does is basically retrieve that information for you automatically from your Gmail and then display it as a card at the right point in time. So if you have a hotel confirmation, it will display that probably when you land in that city. I think putting together information that's mine, whether it's on my phone or not, and information that is sort of the world's, is absolutely critically important. People are going to expect that of course I can have a conversation with Google. And of course Google is going to access more than just the public information on the web. Google is going to know when my flight is, whether my package has gotten here yet, and where my wife is and how long it's going to take her to get home this afternoon, and so on and so on and so on, because of course Google knows that stuff. We wanted to make voice search like ten times better. Speech recognition is the process of turning sounds into meaning. And between those two, there's many levels of processing. We want to turn sounds into what we call phonemes. From the phonemes, you want to turn them into words. From the words, you construct sentences. And then once you have a sentence, you have a transcript of what you've said. You have to turn that into something that's meaningful for the computer to interpret. So if you're doing a search query, you're asking for pictures of cats, the computer has to understand that you're really doing an image search for the word cats. We've actually been using neural networks for a variety of tasks, because if you have a very large amount of data and you can train big models, then you just throw more data at the problem and you'll start to catch, pick up on those more subtle refinements like, oh, someone's speaking with an East Coast accent. Play the scene where George says, it's not you, it's me. So here it gives me a link directly into YouTube, which I can click on. I don't understand, things were going so great. Every back end of Google, every different web service that's been developed over the last 10 years or so, is part of this service. What Google now is, is precisely what Google is best at. How do you do heavy amounts of computational and large amounts of data to extract patterns and to score things? I think what we've tried to do is kind of stay true to the, in some sense you could say, the personality or the persona that Google has today. It has that whimsy and that fun aspect in some kind of unexpected ways, but that doesn't really try to pretend to be your human buddy. What is an android? In science fiction, a robot with a human appearance.
 Google has a lot of products. It also knows a lot about you.
 Google Now is a feature that has the potential to tie all that together in a way that's actually meaningful.
 They invited us to their Mountain View campus in California to talk to people behind the project and see where it's going in the future.
 We think the phone and more broadly kind of the computer that's with me,
 whether it's a phone or Sergei's glasses or something else,
 is pretty unique because it has basically the context of what I'm doing and where I'm at.
 So we've always had a search bar across the very top on Android from the very beginning.
 In Jelly Bean we made a few pretty significant changes to the search experience on Android.
 The first thing that we wanted to do and did was to go much deeper in answering questions.
 Directions to the museum with the William Paley exhibition.
 Getting directions.
 So it turns out it's at the DeYoung Museum.
 What's happening here is that there's a lot of information on the web that talks about this particular exhibition.
 And then it obviously talks about where it is and that's the DeYoung Museum in San Francisco.
 So when I say that, Google crosses all these various different pieces of information in my question
 to give me an answer back, which in this case is actually just a map.
 We're obviously very excited about search on mobile devices on Android
 and about the idea of people being able to interact with their phone in just a natural way.
 And if you think about what that takes, we feel like there's a few key elements that have to come together.
 One obviously is voice recognition.
 The second piece is once I have those words is natural language understanding.
 And the third piece, which is a fairly new thing for us at Google,
 is understanding what are the basic components and facts in the world.
 And this is something we call our knowledge graph.
 And it's really the first time in computer science history that those three things are all available.
 And not just available in some giant computational center, but available in a powerful device that's in my pocket.
 So one of the most important things from Larry's founder letter from earlier this year
 is the part where he says he thinks computers should do all the hard work for you.
 Sort of getting all the hard stuff out of the way so you can just enjoy life,
 enjoy whatever it is that you like to do.
 For example, if you have a flight reservation, you will have received that in your email.
 And you get to the hotel, you get to the restaurant, whatever. It's hard to find it.
 So what Google now does is basically retrieve that information for you automatically from your Gmail
 and then display it as a card at the right point in time.
 So if you have a hotel confirmation, it will display that probably when you land in that city.
 I think putting together information that's mine, whether it's on my phone or not,
 and information that is sort of the world's, is absolutely critically important.
 People are going to expect that of course I can have a conversation with Google.
 And of course Google is going to access more than just the public information on the web.
 Google is going to know when my flight is, whether my package has gotten here yet,
 and where my wife is and how long it's going to take her to get home this afternoon,
 and so on and so on and so on, because of course Google knows that stuff.
 We wanted to make voice search like ten times better.
 Speech recognition is the process of turning sounds into meaning.
 And between those two, there's many levels of processing.
 We want to turn sounds into what we call phonemes.
 From the phonemes, you want to turn them into words.
 From the words, you construct sentences.
 And then once you have a sentence, you have a transcript of what you've said.
 You have to turn that into something that's meaningful for the computer to interpret.
 So if you're doing a search query, you're asking for pictures of cats,
 the computer has to understand that you're really doing an image search for the word cats.
 We've actually been using neural networks for a variety of tasks,
 because if you have a very large amount of data and you can train big models,
 then you just throw more data at the problem and you'll start to catch,
 pick up on those more subtle refinements like, oh, someone's speaking with an East Coast accent.
 Play the scene where George says, it's not you, it's me.
 So here it gives me a link directly into YouTube, which I can click on.
 I don't understand, things were going so great.
 Every back end of Google, every different web service that's been developed over the last 10 years or so,
 is part of this service.
 What Google now is, is precisely what Google is best at.
 How do you do heavy amounts of computational and large amounts of data
 to extract patterns and to score things?
 I think what we've tried to do is kind of stay true to the, in some sense you could say,
 the personality or the persona that Google has today.
 It has that whimsy and that fun aspect in some kind of unexpected ways,
 but that doesn't really try to pretend to be your human buddy.
 What is an android?
 In science fiction, a robot with a human appearance.