 Good morning, everyone. It's great to be back at IEO. Today, I'm excited to share our latest breakthrough in natural language understanding, Lambda. It's a language model for dialogue applications and it's open domain, which means it's designed to converse on any topic. And while it's still in research and development, we've been using it internally to explore novel interactions. For example, say you wanted to learn about one of my favorite planets, Pluto. Lambda already understands quite a lot about Pluto and millions of other topics. Let's listen to a conversation the team had with Pluto a few days ago. I'm so curious about you. I sense your excitement. Ask me anything. Tell me what I would see if I visited. You would get to see a massive canyon, some frozen icebergs, geysers, and some craters. It sounds beautiful. I assure you it is worth the trip. However, you need to bring your coat because it gets really cold. I'll keep that in mind. Hey, I was wondering, have you ever had any visitors? Yes, I have had some. The most notable was New Horizons, the spacecraft that visited me. Let's break down what made it feel so natural. First, learn concepts. As you saw, the model talked about the New Horizons spacecraft and the coldness of space. Lambda synthesized these concepts from its training data. Because none of the responses were predefined, Lambda answered with sensible responses, keeping the dialogue open-ended. Natural conversations are generative and they never take the same path twice. And Lambda is able to carry a conversation no matter what we talk about. Yet, it's still early research, so it doesn't get everything right. Sometimes it can give nonsensical responses, imagining Pluto doing flips or playing fetch with its favorite ball, the moon. Other times, it just doesn't keep the conversation going. We believe Lambda's natural conversation capabilities have the potential to make information and computing radically more accessible and easier to use. We look forward to incorporating better conversational features into products like Google Assistant, Search, and Workspace. Lambda is a huge step forward in natural conversation, but it is still trained only on text. When people communicate with each other, they do it across images, text, audio, and video. So we need to build models that allow people to naturally ask questions across different types of information. These are called multimodal models. For example, when you say, "'Show me the part where the lion roars at sunset," we will get you to that exact moment in a video. Advances in AI are helping us reimagine what a map can be, but now you can also use it to explore the world around you. You'll be able to access live view right from the map and instantly see details about the shops and the restaurants around you, including how busy they are, recent reviews, and photos of those popular dishes. In addition, there are a host of new features coming to live view later this year. We're adding prominent virtual street signs to help you navigate those complex intersections. Second, we'll point you towards key landmarks and places that are important for you, like the direction of your hotel. Third, we're bringing it indoors to help you get around some of the hardest to navigate buildings, like airports, transit stations, and malls. Indoor live view will start rolling out in top train stations and airports in Zurich this week, and will come to Tokyo next month. We're bringing you the most detailed street maps we've ever made. Take this image of Columbus Circle, one of the most complicated intersections in Manhattan. You can now see where the sidewalks, the crosswalks, the pedestrian islands are. Something that might be incredibly helpful if you're taking young children out on a walk, or absolutely essential if you're using a wheelchair. Thanks to our application of advanced AI technology, on robust street view and aerial imagery, we're on track to launch detailed street maps in 50 new cities by the end of the year. So we're making the map more dynamic and more tailored, highlighting the most relevant information exactly when you need it. If it's 8 a.m. on a weekday, we'll display the coffee shops and bakeries more prominently in the map. While at 5 p.m., we'll highlight the dinner restaurants that match your tastes. You'll start seeing this more tailored map in the coming weeks. People have found it really useful, especially during this pandemic, to see how busy a place is before heading out. Now we're expanding this capability from specific places, like restaurants and shops, to neighborhoods with a feature called Area Busyness. Say you're in Rome and want to head over to the Spanish Steps and its nearby shops. With Area Busyness, you'll be able to understand at a glance if it's the right time for you to go. Based on how busy that part of the city is in real time, Area Busyness will roll out globally in the coming months. Let's talk about all the ways we're innovating in shopping. Soon on Chrome, when you open a new tab, you'll be able to see your open carts from the past couple of weeks. We'll also find you promotions and discounts for your open carts if you choose to opt in. Your personal information and what's in your carts are never shared with anyone externally without your permission. We capture photos and videos so we can look back and remember. There are more than four trillion photos and videos stored in Google Photos. But having so many photos of loved ones, screenshots, selfies, all stored together makes it hard to rediscover the important moments. Soon, we're launching a new way to look back that we're calling Little Patterns. Little Patterns show the magic in everyday moments by identifying not so obvious moments and resurfacing them to you. This feature uses machine learning to translate photos into a series of numbers and then compares how visually or conceptually similar these images are. When we find a set of three or more photos with similarities such as shape or color, we'll surface them as a pattern. When we started testing Little Patterns, we saw some great stories come to life. Like how one of our engineers traveled the world with their favorite orange backpack. Or how our product manager, Christie, had a habit of capturing objects of similar shape and color. We also want to bring these moments to life with cutting edge effects. Last year, we launched Cinematic Photos to help you relive your memories in a more vivid way. Cinematic Moments will take these near duplicate images and use neural networks to synthesize the movement between image A and image B. We interpolate the photos and fill in the gaps by creating new frames. The end result is a vivid, moving picture. And the cool thing about this effect is, it can work on any pair of images. Whether they were captured on Android, iOS, or scanned from a photo album. In addition to providing personalized content to look back on, we also want to give you more control. We heard from you that controls can be helpful for anyone who has been through a tough life event, breakup, or loss. These insights inspired us to give you the control to hide photos of certain people or time periods from our memories feature. And soon, you'll be able to remove a single photo from a memory, rename the memory, or remove it entirely. Instead of form following function, what if form followed feeling? Instead of Google Blue, we imagined Material You. A new design that includes you as a co-creator. Letting you transform the look and feel of all your apps by generating personal Material palettes that mix color science with a designer's eye. A new design that can flex to every screen and fit every device. Your apps adapt comfortably every place you go. Beyond light and dark, a mode for every mood. These selections can travel with your account across every app and every device. Material You comes first to Google Pixel this fall, including all of your favorite Google apps. And over the following year, we will continue our vision, bringing it to the web, Chrome OS, wearables, smart displays, and all of Google's products. We've overhauled everything from the lock screen to system settings, revamping the way we use color, shapes, light, and motion. Watch what happens when the wallpaper changes. Like if I use this picture of my kids actually getting along for once. I set it as my background and voila! The system creates a custom palette based on the colors in my photo. The result is a one of a kind design just for you. And you'll see it first on Google Pixel in the fall. Starting from the lock screen, the design is more playful with dynamic lighting. Pick up your phone and it lights up from the bottom of your screen. Press the power button to wake up the phone instead and the light ripples out from your phone. The light ripples out from your touch. Even the clock is in tune with you. When you don't have any notifications, it appears larger on the lock screen. So you know you're all caught up. The notification shade is more intuitive with a crisp at a glance view of your app notifications, whatever you're currently listening to or watching, and quick settings that give you control over the OS with just a swipe and a tap. And now you can invoke the Google Assistant by long pressing the power button. And the team also reduced the CPU time of Android system server by a whopping 22%. And with Android 12, we're going even further to keep your information safe. To give people more transparency and control, we've created a new privacy dashboard that shows you what type of data was accessed and when. This dashboard reports on all the apps on your phone, including all of your Google apps. And we've made it really easy to revoke an app's permission directly from the dashboard. We've also added an indicator to make it clear when an app is using your camera or microphone. But let's take that a step further. If you don't want any apps to access the microphone or camera, even if you've granted them permission in the past, we've added two new toggles and quick settings. So you can completely disable those sensors for every app. Android's Private Compute Core enables things like Now Playing, which tells you what song is playing in the background, and Smart Reply, which suggests responses to your chats based on your personal reply patterns. And there's more to come later this year. All of the sensitive audio and language processing happens exclusively on your device. And like the rest of Android, Private Compute Core is open source. It's fully inspectable and verifiable by the security community. With a single tap, you can unlock and sign into your Chromebook when your phone is nearby. Incoming chat notifications from apps on your phone are right there in Chrome OS. And soon, if you want to share a picture, one click and you can access your phone's most recent photos. To keep movie night on track, we're building TV remote features directly into your phone. You can use voice search or even type with your phone's keyboard. We're also really excited to introduce support for Digital Car Key. Car Key will allow you to lock, unlock, and start your car, all from your phone. It works with NFC and ultra-wideband technology, making it super secure and easy to use. And if your friend needs to borrow your car, you can remotely and securely share your digital key with them. Car Key is launching this fall with select Google Pixel and Samsung Galaxy smartphones. And we're working with BMW and others across the industry to bring it to their upcoming cars. That was a quick look at Android 12, which will launch this fall. But you can check out many of these features in the Android 12 beta. Today, let's go beyond the phone to what we believe is the next evolution of mobile computing, the smartwatch. First, building a unified platform jointly with Samsung, focused on battery life, performance, and making it easier for developers to build great apps for the watch. Second, a whole new consumer experience, including updates to your favorite Google apps. And third, a world-class health and fitness service, created by the newest addition to the Google family, Fitbit. As the world's largest OS, we have a responsibility to build for everyone. But for people of color, photography has not always seen us as we want to be seen, even in some of our own Google products. To make smartphone photography truly for everyone, we've been working with a group of industry experts to build a more accurate and inclusive camera. So far, we've partnered with a range of different expert image makers who've taken thousands of images to diversify our image data sets, helped improve the accuracy of our auto white balance and auto exposure algorithms, and given aesthetic feedback to make our images of people of color more beautiful and more accurate. Although there's still much to do, we're working hard to bring all of what you've seen here and more to Google Pixel this fall. We were all grateful to have video conferencing over the last year. It helped us stay in touch with family and friends and kept businesses and schools going. But there is no substitute for being together in the room with someone. So several years ago, we kicked off a project to use technology to explore what's possible. We call it Project Starline. First, using high resolution cameras and custom built depth sensors, we capture your shape and appearance from multiple perspectives and then fuse them together to create an extremely detailed real time 3D model. The resulting data is huge, many gigabits per second. To send this 3D imagery over existing networks, we developed novel compression and streaming algorithms that reduce the data by a factor of more than 100. And we have developed a breakthrough light field display that shows you the realistic representation of someone sitting right in front of you in three dimensions. As you move your head and body, our system adjusts the images to match your perspective. You can talk naturally, gesture, and make eye contact. It's as close as we can get to the feeling of sitting across from someone. We have spent thousands of hours testing it in our own offices, and the results are promising. There's also excitement from our lead enterprise partners. We plan to expand access to partners in health care and media. Thank you for joining us today. Please enjoy the rest of Google I.O. and stay tuned for the developer keynote coming up next. I hope to see you in person next year. Until then, stay safe and be well.
 Good morning, everyone.
 It's great to be back at IEO.
 Today, I'm excited to share our latest breakthrough
 in natural language understanding, Lambda.
 It's a language model for dialogue applications
 and it's open domain,
 which means it's designed to converse on any topic.
 And while it's still in research and development,
 we've been using it internally
 to explore novel interactions.
 For example, say you wanted to learn about
 one of my favorite planets, Pluto.
 Lambda already understands quite a lot about Pluto
 and millions of other topics.
 Let's listen to a conversation
 the team had with Pluto a few days ago.
 I'm so curious about you.
 I sense your excitement.
 Ask me anything.
 Tell me what I would see if I visited.
 You would get to see a massive canyon,
 some frozen icebergs, geysers, and some craters.
 It sounds beautiful.
 I assure you it is worth the trip.
 However, you need to bring your coat
 because it gets really cold.
 I'll keep that in mind.
 Hey, I was wondering, have you ever had any visitors?
 Yes, I have had some.
 The most notable was New Horizons,
 the spacecraft that visited me.
 Let's break down what made it feel so natural.
 First, learn concepts.
 As you saw, the model talked about
 the New Horizons spacecraft and the coldness of space.
 Lambda synthesized these concepts from its training data.
 Because none of the responses were predefined,
 Lambda answered with sensible responses,
 keeping the dialogue open-ended.
 Natural conversations are generative
 and they never take the same path twice.
 And Lambda is able to carry a conversation
 no matter what we talk about.
 Yet, it's still early research,
 so it doesn't get everything right.
 Sometimes it can give nonsensical responses,
 imagining Pluto doing flips
 or playing fetch with its favorite ball, the moon.
 Other times, it just doesn't keep the conversation going.
 We believe Lambda's natural conversation capabilities
 have the potential to make information and computing
 radically more accessible and easier to use.
 We look forward to incorporating
 better conversational features into products
 like Google Assistant, Search, and Workspace.
 Lambda is a huge step forward in natural conversation,
 but it is still trained only on text.
 When people communicate with each other,
 they do it across images, text, audio, and video.
 So we need to build models that allow people
 to naturally ask questions
 across different types of information.
 These are called multimodal models.
 For example, when you say,
 "'Show me the part where the lion roars at sunset,"
 we will get you to that exact moment in a video.
 Advances in AI are helping us reimagine what a map can be,
 but now you can also use it to explore the world around you.
 You'll be able to access live view right from the map
 and instantly see details about the shops
 and the restaurants around you,
 including how busy they are, recent reviews,
 and photos of those popular dishes.
 In addition, there are a host of new features
 coming to live view later this year.
 We're adding prominent virtual street signs
 to help you navigate those complex intersections.
 Second, we'll point you towards key landmarks
 and places that are important for you,
 like the direction of your hotel.
 Third, we're bringing it indoors
 to help you get around some of the hardest
 to navigate buildings,
 like airports, transit stations, and malls.
 Indoor live view will start rolling out
 in top train stations and airports in Zurich this week,
 and will come to Tokyo next month.
 We're bringing you the most detailed street maps
 we've ever made.
 Take this image of Columbus Circle,
 one of the most complicated intersections in Manhattan.
 You can now see where the sidewalks, the crosswalks,
 the pedestrian islands are.
 Something that might be incredibly helpful
 if you're taking young children out on a walk,
 or absolutely essential if you're using a wheelchair.
 Thanks to our application of advanced AI technology,
 on robust street view and aerial imagery,
 we're on track to launch detailed street maps
 in 50 new cities by the end of the year.
 So we're making the map more dynamic and more tailored,
 highlighting the most relevant information
 exactly when you need it.
 If it's 8 a.m. on a weekday,
 we'll display the coffee shops and bakeries
 more prominently in the map.
 While at 5 p.m., we'll highlight the dinner restaurants
 that match your tastes.
 You'll start seeing this more tailored map
 in the coming weeks.
 People have found it really useful,
 especially during this pandemic,
 to see how busy a place is before heading out.
 Now we're expanding this capability
 from specific places, like restaurants and shops,
 to neighborhoods with a feature called Area Busyness.
 Say you're in Rome and want to head over
 to the Spanish Steps and its nearby shops.
 With Area Busyness, you'll be able to understand
 at a glance if it's the right time for you to go.
 Based on how busy that part of the city is in real time,
 Area Busyness will roll out globally in the coming months.
 Let's talk about all the ways we're innovating in shopping.
 Soon on Chrome, when you open a new tab,
 you'll be able to see your open carts
 from the past couple of weeks.
 We'll also find you promotions and discounts
 for your open carts if you choose to opt in.
 Your personal information and what's in your carts
 are never shared with anyone externally
 without your permission.
 We capture photos and videos
 so we can look back and remember.
 There are more than four trillion photos and videos
 stored in Google Photos.
 But having so many photos of loved ones,
 screenshots, selfies, all stored together
 makes it hard to rediscover the important moments.
 Soon, we're launching a new way to look back
 that we're calling Little Patterns.
 Little Patterns show the magic in everyday moments
 by identifying not so obvious moments
 and resurfacing them to you.
 This feature uses machine learning to translate photos
 into a series of numbers
 and then compares how visually
 or conceptually similar these images are.
 When we find a set of three or more photos with similarities
 such as shape or color,
 we'll surface them as a pattern.
 When we started testing Little Patterns,
 we saw some great stories come to life.
 Like how one of our engineers traveled the world
 with their favorite orange backpack.
 Or how our product manager, Christie,
 had a habit of capturing objects of similar shape and color.
 We also want to bring these moments to life
 with cutting edge effects.
 Last year, we launched Cinematic Photos
 to help you relive your memories in a more vivid way.
 Cinematic Moments will take these near duplicate images
 and use neural networks to synthesize the movement
 between image A and image B.
 We interpolate the photos and fill in the gaps
 by creating new frames.
 The end result is a vivid, moving picture.
 And the cool thing about this effect is,
 it can work on any pair of images.
 Whether they were captured on Android, iOS,
 or scanned from a photo album.
 In addition to providing personalized content
 to look back on, we also want to give you more control.
 We heard from you that controls can be helpful
 for anyone who has been through a tough life event,
 breakup, or loss.
 These insights inspired us to give you the control
 to hide photos of certain people or time periods
 from our memories feature.
 And soon, you'll be able to remove a single photo
 from a memory,
 rename the memory, or remove it entirely.
 Instead of form following function,
 what if form followed feeling?
 Instead of Google Blue, we imagined Material You.
 A new design that includes you as a co-creator.
 Letting you transform the look and feel of all your apps
 by generating personal Material palettes
 that mix color science with a designer's eye.
 A new design that can flex to every screen
 and fit every device.
 Your apps adapt comfortably every place you go.
 Beyond light and dark, a mode for every mood.
 These selections can travel with your account
 across every app and every device.
 Material You comes first to Google Pixel this fall,
 including all of your favorite Google apps.
 And over the following year, we will continue our vision,
 bringing it to the web, Chrome OS, wearables,
 smart displays, and all of Google's products.
 We've overhauled everything from the lock screen
 to system settings, revamping the way we use color,
 shapes, light, and motion.
 Watch what happens when the wallpaper changes.
 Like if I use this picture of my kids actually
 getting along for once.
 I set it as my background and voila!
 The system creates a custom palette
 based on the colors in my photo.
 The result is a one of a kind design just for you.
 And you'll see it first on Google Pixel in the fall.
 Starting from the lock screen, the design is more playful
 with dynamic lighting.
 Pick up your phone and it lights up
 from the bottom of your screen.
 Press the power button to wake up the phone instead
 and the light ripples out from your phone.
 The light ripples out from your touch.
 Even the clock is in tune with you.
 When you don't have any notifications,
 it appears larger on the lock screen.
 So you know you're all caught up.
 The notification shade is more intuitive
 with a crisp at a glance view of your app notifications,
 whatever you're currently listening to or watching,
 and quick settings that give you control over the OS
 with just a swipe and a tap.
 And now you can invoke the Google Assistant
 by long pressing the power button.
 And the team also reduced the CPU time
 of Android system server by a whopping 22%.
 And with Android 12, we're going even further
 to keep your information safe.
 To give people more transparency and control,
 we've created a new privacy dashboard
 that shows you what type of data was accessed and when.
 This dashboard reports on all the apps on your phone,
 including all of your Google apps.
 And we've made it really easy to revoke an app's permission
 directly from the dashboard.
 We've also added an indicator to make
 it clear when an app is using your camera or microphone.
 But let's take that a step further.
 If you don't want any apps to access the microphone
 or camera, even if you've granted them permission
 in the past, we've added two new toggles and quick settings.
 So you can completely disable those sensors for every app.
 Android's Private Compute Core enables things
 like Now Playing, which tells you what song is playing
 in the background, and Smart Reply,
 which suggests responses to your chats
 based on your personal reply patterns.
 And there's more to come later this year.
 All of the sensitive audio and language processing
 happens exclusively on your device.
 And like the rest of Android, Private Compute Core
 is open source.
 It's fully inspectable and verifiable
 by the security community.
 With a single tap, you can unlock and sign
 into your Chromebook when your phone is nearby.
 Incoming chat notifications from apps on your phone
 are right there in Chrome OS.
 And soon, if you want to share a picture,
 one click and you can access your phone's most recent photos.
 To keep movie night on track, we're
 building TV remote features directly into your phone.
 You can use voice search or even type
 with your phone's keyboard.
 We're also really excited to introduce support
 for Digital Car Key.
 Car Key will allow you to lock, unlock, and start your car,
 all from your phone.
 It works with NFC and ultra-wideband technology,
 making it super secure and easy to use.
 And if your friend needs to borrow your car,
 you can remotely and securely share your digital key
 with them.
 Car Key is launching this fall with select Google Pixel
 and Samsung Galaxy smartphones.
 And we're working with BMW and others across the industry
 to bring it to their upcoming cars.
 That was a quick look at Android 12,
 which will launch this fall.
 But you can check out many of these features
 in the Android 12 beta.
 Today, let's go beyond the phone to what
 we believe is the next evolution of mobile computing,
 the smartwatch.
 First, building a unified platform jointly with Samsung,
 focused on battery life, performance,
 and making it easier for developers to build
 great apps for the watch.
 Second, a whole new consumer experience,
 including updates to your favorite Google apps.
 And third, a world-class health and fitness service,
 created by the newest addition to the Google family, Fitbit.
 As the world's largest OS, we have a responsibility
 to build for everyone.
 But for people of color, photography has not always
 seen us as we want to be seen, even in some
 of our own Google products.
 To make smartphone photography truly for everyone,
 we've been working with a group of industry experts
 to build a more accurate and inclusive camera.
 So far, we've partnered with a range of different expert
 image makers who've taken thousands of images
 to diversify our image data sets,
 helped improve the accuracy of our auto white balance
 and auto exposure algorithms, and given aesthetic feedback
 to make our images of people of color
 more beautiful and more accurate.
 Although there's still much to do,
 we're working hard to bring all of what you've seen here
 and more to Google Pixel this fall.
 We were all grateful to have video conferencing
 over the last year.
 It helped us stay in touch with family and friends
 and kept businesses and schools going.
 But there is no substitute for being together
 in the room with someone.
 So several years ago, we kicked off a project
 to use technology to explore what's possible.
 We call it Project Starline.
 First, using high resolution cameras and custom built depth
 sensors, we capture your shape and appearance
 from multiple perspectives and then fuse them together
 to create an extremely detailed real time 3D model.
 The resulting data is huge, many gigabits per second.
 To send this 3D imagery over existing networks,
 we developed novel compression and streaming algorithms
 that reduce the data by a factor of more than 100.
 And we have developed a breakthrough light field
 display that shows you the realistic representation
 of someone sitting right in front of you
 in three dimensions.
 As you move your head and body, our system
 adjusts the images to match your perspective.
 You can talk naturally, gesture, and make eye contact.
 It's as close as we can get to the feeling of sitting across
 from someone.
 We have spent thousands of hours testing it in our own offices,
 and the results are promising.
 There's also excitement from our lead enterprise partners.
 We plan to expand access to partners
 in health care and media.
 Thank you for joining us today.
 Please enjoy the rest of Google I.O.
 and stay tuned for the developer keynote coming up next.
 I hope to see you in person next year.
 Until then, stay safe and be well.