 So what's happening in this building? What are you guys working on? We've got about 850 researchers, so about 1% of the entire employee base of the company working on farther out research. This is a system we call Connect Fusion. Now I've got a Connect camera here. We're used to sort of putting a Connect camera down somewhere and having it sort of capture the room, but we thought, you know, what happens if you actually pick it up and you move it around and use it kind of as a scanning device, right? Top left, we've got just the raw image that comes out of the 3D part of the Connect camera. In the middle, we've got the raw image that comes out of the just standard web camera. Top right, we've got what's called a surface image, which is how we use vision technology to pull out different surfaces. Bottom left is actually the 30 frames a second we get from the 3D camera and building a whole model of that. You can see in the bottom right, we actually get a sort of nicer surface model of that, right? These are 3D renderings of the actual scene. We're building a model of the world by taking lots of individual frames and putting them together. And we can sort of update this and fly. So if I stick my hand in here, you see in a second the hand shows up. I pull it out and it disappears. Can I try that? Can I put my hand in? Yeah, we'll do even better. Josh, have a seat. You want me to sit down? I want you to sit down. Oh, I don't like where this is going at all. Oh, that is... This is disturbing. I feel like I'm having a bad drug experience right now. Did you slip anything into my coffee? You know, if you went and tried to get by like an industrial version of this, they sell for about $50,000, right? And I'm using a $150 camera plugged into a PC to do this. And the crazy thing is this is Kinect version one. I mean, presumably there'll be future higher resolution, faster versions of Kinect. So now let's take the RGB camera and let's add that in. So wait, that's a render down there? This is an actual model. It looks like a model. I don't know what a human sits down here. You can go sit down there if you want. Go sit. Now, drop your arms. Yeah, see. You're in the model and you can, you know, given particularly how social software is today, we need to understand how all these kinds of technologies change the way we interact with other people in our family. You know, we spend a lot of time doing research and doing studies into all of that as well. Okay, so this is our next stop. Pretty excited. There's a laser warning on the door. Let's take a look. We are researching different ways to use projectors and depth cameras to make new user interfaces. Above us we have three projectors and three depth cameras. Kind of looks like an early beta version of GLaDOS from Portal. Yeah, yeah, you might say that. And so this is a research prototype where we're basically testing some of the interactive possibilities of this setup. Now I can do certain things on this table. It's very much like a multi-touch or surface like experience. This is just a regular tape. One of the things we're playing around with is using the sensing infrastructure here to enable you to effortlessly move objects from one display to another. I can now move that back and I can over to the tabletop and move it over here and you get this kind of interesting relationship now that we didn't have. Oh, that is very freaky. So let's say I can't reach both displays at the same time. So what I could do is pull that and then if I'm holding it and then I can touch that and it appears over here. So we're trying to make virtual objects have a little bit of a physical presence in the world by using the projection. Now the key thing here is that we can, using the depth data, we can correct the graphics and we know where everything is so we can hit the hand with the projected graphics. There we go. There we are. Ah, this is, it's kind of incredible. We believe that a rise in tide raises all ships. Our 850 people aren't going to solve all the problems of computing. So we learn from other people, other people learn from us. All the research is about doing things that nobody's ever done before and a lot of people tell you it's impossible to do that, right? And so we constantly sort of advance the state of the art. Today it's possible to do this or we know it's possible to do this. Tomorrow we'll take one step further and we'll take one step further than that and we'll keep kind of working for it until one point we go, oh wow, now there's like this whole new scenario we can do because we figured out how to do all the pieces of that.
 So what's happening in this building?
 What are you guys working on?
 We've got about 850 researchers, so about 1% of the entire employee base of the company
 working on farther out research.
 This is a system we call Connect Fusion.
 Now I've got a Connect camera here.
 We're used to sort of putting a Connect camera down somewhere and having it sort of capture
 the room, but we thought, you know, what happens if you actually pick it up and you move it
 around and use it kind of as a scanning device, right?
 Top left, we've got just the raw image that comes out of the 3D part of the Connect camera.
 In the middle, we've got the raw image that comes out of the just standard web camera.
 Top right, we've got what's called a surface image, which is how we use vision technology
 to pull out different surfaces.
 Bottom left is actually the 30 frames a second we get from the 3D camera and building a whole
 model of that.
 You can see in the bottom right, we actually get a sort of nicer surface model of that,
 right?
 These are 3D renderings of the actual scene.
 We're building a model of the world by taking lots of individual frames and putting them
 together.
 And we can sort of update this and fly.
 So if I stick my hand in here, you see in a second the hand shows up.
 I pull it out and it disappears.
 Can I try that?
 Can I put my hand in?
 Yeah, we'll do even better.
 Josh, have a seat.
 You want me to sit down?
 I want you to sit down.
 Oh, I don't like where this is going at all.
 Oh, that is...
 This is disturbing.
 I feel like I'm having a bad drug experience right now.
 Did you slip anything into my coffee?
 You know, if you went and tried to get by like an industrial version of this, they sell
 for about $50,000, right?
 And I'm using a $150 camera plugged into a PC to do this.
 And the crazy thing is this is Kinect version one.
 I mean, presumably there'll be future higher resolution, faster versions of Kinect.
 So now let's take the RGB camera and let's add that in.
 So wait, that's a render down there?
 This is an actual model.
 It looks like a model.
 I don't know what a human sits down here.
 You can go sit down there if you want.
 Go sit.
 Now, drop your arms.
 Yeah, see.
 You're in the model and you can, you know, given particularly how social software is
 today, we need to understand how all these kinds of technologies change the way we interact
 with other people in our family.
 You know, we spend a lot of time doing research and doing studies into all of that as well.
 Okay, so this is our next stop.
 Pretty excited.
 There's a laser warning on the door.
 Let's take a look.
 We are researching different ways to use projectors and depth cameras to make new user interfaces.
 Above us we have three projectors and three depth cameras.
 Kind of looks like an early beta version of GLaDOS from Portal.
 Yeah, yeah, you might say that.
 And so this is a research prototype where we're basically testing some of the interactive
 possibilities of this setup.
 Now I can do certain things on this table.
 It's very much like a multi-touch or surface like experience.
 This is just a regular tape.
 One of the things we're playing around with is using the sensing infrastructure here to
 enable you to effortlessly move objects from one display to another.
 I can now move that back and I can over to the tabletop and move it over here and you
 get this kind of interesting relationship now that we didn't have.
 Oh, that is very freaky.
 So let's say I can't reach both displays at the same time.
 So what I could do is pull that and then if I'm holding it and then I can touch that and
 it appears over here.
 So we're trying to make virtual objects have a little bit of a physical presence in the
 world by using the projection.
 Now the key thing here is that we can, using the depth data, we can correct the graphics
 and we know where everything is so we can hit the hand with the projected graphics.
 There we go.
 There we are.
 Ah, this is, it's kind of incredible.
 We believe that a rise in tide raises all ships.
 Our 850 people aren't going to solve all the problems of computing.
 So we learn from other people, other people learn from us.
 All the research is about doing things that nobody's ever done before and a lot of people
 tell you it's impossible to do that, right?
 And so we constantly sort of advance the state of the art.
 Today it's possible to do this or we know it's possible to do this.
 Tomorrow we'll take one step further and we'll take one step further than that and we'll
 keep kind of working for it until one point we go, oh wow, now there's like this whole
 new scenario we can do because we figured out how to do all the pieces of that.