LEIA 3D: holographic reality
Fattal, David
Ever since Doug Engelbart presented the first modern computer interface with his famous “Mother of All Demos,” we have strived to achieve more intuitive ways to interact with digital information. That interface has not fundamentally changed over the past half-century, though, even as the scope of information continues to exponentially increase. As we now look to computerized AI to help us navigate and make sense of all of our shared data, we also require a new way to present the information that is intuitive and useful to us. Holographic Reality (HR) is based on “holographic” 3D screens that do not require any eyewear to function. These screens must produce realistic, full-parallax 3D imagery that can be manipulated in mid-air by finger or hand gestures. They must provide the same high quality imagery throughout the field of view, no jumps, bad-spots or other visual artefact. Augmented by Haptic technology (tactile feedback), these screens will even let us “feel” the holographic content physically at our fingertips. 
 David Fatahou, and he's from Leia3D, and his speech will be on holographic reality. We're talking a lot about virtual reality, augmented reality these days, and what I want to present to you is a concept that we call holographic reality, which is really, we want a strong belief that virtual world is good on headset, it's very immersive, but there's a lot of application, actually a lot more that could be on headset, that could be on regular screen with the usual way that you actually interact with the digital world, which is when you want it, you turn it on, you interact with it, and then when you're done you just look away and then you're kind of back to your regular life, and that you can do with a headset. So what I want to talk about, so I'm David Fatahou, I'm the founder and CEO of a startup from Menlobark called Leia Inc., we're a spinoff from HP, and I'm going to be describing a little bit of our vision, our technology, and some of the applications, everything in less than 20 minutes, okay? So the vision of the company is this, right? I think today all of our information is obviously transitioning from the physical world to the digital world, our communication, our knowledge, the way we entertain, the way we work everything is now on servers in the cloud, yet, you know, we're still made of flesh and bones, and you know, we still essentially need to experience all that information in a physical form, which means we need to find ways, intuitive ways to bring back the digital world into our own, okay? This is fundamentally what you do with a 2D display, right? I'm trying to talk at a very abstract level here. A 2D display takes bits and bytes and essentially materializes it into a 2D image on the screen, possibly with a touch screen, you can interact with it, and you're getting essentially to interact with the digital world. And that's the way that we've been interacting with technology, it's been about 10 years now on your iPhone and so on. Obviously, you know, the same way that we transition from black and white TV to color TV, it's been a while it's been predicted we will transition from, you know, flat screens to 3D screens, okay? The world is not flat, so if you want to get a more intuitive interaction with the digital world, you should somehow be able to render the digital world in the third dimension. So recently, right, there's been a lot of craze about VR AR, and they do provide that immersion. It's actually very easy to do 3D. It's quite easy to do 3D with a VR AR headset because you just have two different displays that are following your eyes at all time. And as I said in my mini introduction earlier, right, they come at the expense of having to wear bulky eyewear, which impairs mobility, social interactions. It's quite funny actually, VR AR what happened is that you get to collaborate and to interact with people on other continents, but you're actually isolated from the person next to you, right? So if you want to maybe enjoy, you know, a football game in a bar, usually you want to be close to the people around you, you're not going to isolate yourself with the headset. And you walk in the street and you want to place a quick phone call. You don't want to, I want to have to wear a headset with a mini computer attached to your belt, you know, at all times. Emerging solution, right, is the topic I think of this conference is, you know, stereoscopic 3D displays, in particular naked eye 3D displays that are able to render the digital world in 3D in a completely unobstructive way, just from the screen. However, due to restriction in field of view or in the eye tracking case, there's a little bit of lag and so on and so forth, this still do not offer the total freedom of movement that you would expect from interacting with technology, right? So there's actually no good solution today and what I'm trying to present here, what we're developing at Leia is this quote-unquote holographic platform that is interactive, mobile, intuitive and that essentially lets you interact with holographic quote-unquote objects from any position in an unrestrictive way. And so this is that platform that we call holographic reality, okay? So when I say holograms, I don't want to get into the old debate about what's a hologram, you know, hologram in the broad sense that it's an object that looks and feel like a hologram which means it pops out of the screen and when you move around it, it has very smooth parallax transition and you get to see all kind of perspective around it without jumps and anything like that. So think of it as a very good, a very good multi-view 3D display, for example. And what we're building at Leia is not only we have that capability to render 3D images very, very nicely with very smooth sense of parallax but we augment it with so-called hover touch technology that lets essentially senses your finger above the screen without having to touch the screen. So we can interact with the holograms essentially without touching the screen. Some of you might have gotten a demo yesterday. I think we were there at the demo session. I think you could manipulate the skeleton by just, you know, moving your finger around the screen. It's also in our ambition to integrate so-called haptics and this is essentially ultrasonic technology. You emit ultrasound from the side of the display. If you know where your finger is located, you can actually make the ultrasound interfere in phase under your finger and you'll feel a physical sense of touch. So at that point, you'll be able not only to visualize holographic content and interact with it but you'll be able to actually feel it. So this is the digital world that is literally materialized into the real world and that's basically what we believe is going to be the user interface of the future. Might it be on your mobile phone, might be in the car, in your home, at the office and the vision for, you know, from five to ten years is our world is going to be filled with displays, every single door, every single table, every single, you know, place in your car, in your home, at the office, in public places, is going to be filled with displays, an interactive displays where you can summon the digital world, do some kind of operation and then get on with your life. All right, so that's what we're trying to build. So a little bit about the product today, what it is exactly. We managed to do that with a regular LCD technology. We're 100% compatible with the current LCD supply chain in Asia. And so what we have is we take an LCD display, we actually keep the standard LCD panel on top, you know, commodity technology, low margin and so on and so forth. There's a bunch of vendors in Asia that you can work with. And essentially what we do is we replace the backlight and we replace it with our own backlight that we call a multi-view backlight that has essentially some diffractive pattern on it. So it's a proprietary diffractive pattern that is able to take side illumination light and create what has become to be known as a light field. So we create a light field, a static light field underneath the LCD display. So we create a bunch of colored light rays. We know where these light rays come from. They come from different locations. They propagate in different directions. And they actually have a very controlled, and I'll come back to that, they have a very controlled angular spread. So when these light rays come through the LCD, we can modulate their intensity and what you get is a multi-view 3D image. That is not only full parallax in a wide angle, but it's actually very, has very smooth view transitions. So you don't get the usual jumps that you can see with maybe lenticular or other type of displays. It's actually very, very smooth. And because it's wide angle and because it's very smooth, it lets you essentially interact. It lets you move around the display very comfortably and it lets you interact with it. So that's the difference between having a narrow field of view and having a big field of view is when you have a big field of view, you forget about the technology and you just go on and go on with your life and essentially interact with the device. Not having to worry, am I going to lose the image or not? And the last layer that faces the user is essentially the LCD display itself, right? So unlike an anticular technology where you have to add a bunch of lenses on top, you're actually facing the LCD display. So we can use right away touch technologies, hover touch, you know, capacitive touch, and then we'll be compatible with haptics as well, essentially without having to change much of the technology. So that's essentially the product. And now I'm going to be talking a little bit more about the backlight. But first, I'm just going to talk about what kind of data needs to be fed into the display. It is still a light field display. This is still essentially multi-view 3D display. So you don't have to compute diffraction fringes or anything like that. The technology is diffraction based, but the diffraction pattern is fixed. It doesn't have to change. And it turns out all you have to feed the display are the equivalent of a certain number of views that are computed, you know, on a computer or acquired through cameras, for example. So the very noticeable fact is we can, we can, I don't like to call it crosstalk, but the way the transition, the way you transition from one view to the next is a very precise function of the diffraction pattern. And we can tune it to whatever degree we want. So we can completely kill the crosstalk. We can have zero crosstalk. Or we can introduce some crosstalk to smooth out the transition. And this is done in the hardware. So that idea of a multi-view backlight came from actually, in another life I was a researcher at HP Labs. And we actually made the cover of Nature for that in 2013. And you see another aspect that I haven't mentioned is the backlight is completely transparent. So we can create holograms that are coming, you know, that look like they're emerging from a transparent piece of glass. So it's quite magical. So here you have piece of glass, it's illuminated from the side. And it's essentially creating this, by our standard today, this is a horrible image. It's very blurry, everything. But that was, you know, the proof of concept back then. And because it's transparent, what it allows you to do is also, if we were to work with the transparent modulator, we could do a holographic display, 3D display that is, that is see-through transparent, okay? Which of course you can do with a parallax barrier display or a lenticular type display, right? So we can, we're compatible with see-through and 3D at the same time. So the principle is this, right? It's basically, we call it diffraction of guided wave. So if you look a little bit inside the backlight, we have a side illumination. This is, you know, this doesn't have all the full information on purpose, but just to give you a gist of how it works. You couple light from the side as in a normal backlighting system. And you have this diffractive pattern that can essentially control exactly how the light is extracted. So every time the light interacts with the diffractive pattern, a little bit is essentially diffracted there in this mode. And then most of it is just reflected and then reflected and then has the chance to interact again. So it's very efficient. In the end, we can, we managed to use most of the light in the light guide. And as I was saying, we control not only the direction and the location of this light rays, but we control exactly how they, their angular distribution. And that's very important. We can even use the same diffraction pattern to create two light rays with actually arbitrary direction. Sometime we use that to increase the density of pixels and the density of views available and so on and so forth. So it's very versatile. And a great advantage compared to a lenticular display, for example, is that diffraction doesn't care how tight the angle of the ray is going to be. If you want to send a light ray in a very grazing direction, you can. This is the same as sending it up. With a lenticular display, you need to be close to axis. If you deviate too much from the parallax approximation, you're going to get blurry images. But here you don't care. So as I mentioned before, it turns out that this diffractive pattern is going to act, is going to diffract the light that is guided in the light guide. But for light that is coming straight through, it has no diffraction effect whatsoever. So the light can go through completely unaffected. So that has to do with the transverse momentum of light. Light that is in the light guide has a very large transverse momentum, which means that in order to diffract it, you need a feature size that is pretty small. And when the light comes with near normal incidence with almost zero degree angle, it has a very small amount of momentum there. And this essentially, this grading gives too much of a momentum kick in either direction, and then there's no propagating order that corresponds to it. So again, it's completely transparent. We can give again, we have full freedom. So even if you work with the square array of LCD pixel, you can still define. You have the freedom to define an arbitrary set of views. So for example, here was an example, an early example where we build a hexagonal set of views through a square LCD panel, if you wanted to, just to show you the freedom. And again, what I alluded to that's very important to understand is that each view, we can control all aspects of the radiation pattern. So this one, we wanted to have a flat top, and then we wanted to control a certain, essentially a amount of overlap between the views to create a certain smoothness effect. And then we also wanted the image to kind of die down. So this had eight views, horizontal view one through eight. We wanted view one and view eight to kind of die down so that there was, you know, this is the next, this is essentially, this is the next view here that would be view number nine and it's completely suppressed. So when you look at the display and you move from the left to the right, you see a large amount of parallax, 60 degree field of view. And then if you move too far, instead of seeing a jump back to view one, you know, just the, essentially the brightness decreases and you fade away just very gracefully, right? So it doesn't jump. And at no point you have the sensation of having this inverted 3D, you know, sensation where the left eye image goes to the right eye and vice versa. So it's actually, it is very pleasant. So there's one, there's one usually pain with diffraction is how they do deal with color. So everything I've described so far works well with one color. But now if you take the usual diffraction or holographic kind of configuration, if you send white light on a diffraction pattern, most of you know that what you will get is for first of all, most of the light will go through an undefracted. So we solve that by illuminating with guided light. There is just, there is no zero order propagating order. So this never happens for us. But which, which the thing that could be more disastrous is the first diffracted order gives you a lot of, a lot of color dispersion. So essentially, and actually it's drawn wrong here. The red should be here and the blue should be there. But essentially this should completely break down the colors. You should have, you know, a color in a, you know, in one, one certain range of angles and then another color there, another color there. Most of our patents actually have to do with how we deal with that problem. And we have, we illuminated the light guide in a special way with either with white light or different color of light. But we have to be very careful how we illuminated the system and what we get are actually light rays that are possibly coming from different position but, but that are parallel. So they define the same exact set of views. So that when you look at our display, you don't see any color bleeding. If you want to create a white object, it's perfectly white, including at the edges. That's the really hard part about diffraction. So that's where all the, all the work has to, has to go when you look in the detail to have a high quality display. Perfect. So now it's just a little bit of a, not the marketing, but I want to describe. So the company, just, just to tell you what it, what it takes, right? So we found it two years ago. We spun off HP Labs. We're about 25 employees and we have operation in Menlo Park. Palo Alto, believe it or not, we actually, we bought an ASML stepper and that's how we prototype all of our displays. So the stepper is a optical lithography tool that can, we have a, we can go down to 85 nanometer resolution. Okay. So we have 12 inch wafers. So any display that is small, that would fit in a 12 inch wafer, we can prototype in about less than a week. Just the, the time it takes to get a, a mask from Topan, which is very fast. And essentially, so we, we prototype and we manufacture the backlight still in Palo Alto and then we have a pilot assembly line in Suzhou that aligns the backlight to the LCD panel. And from that pilot line, we transfer the process to the module assembly house in, in Asia, usually in China. So it's, it's, it's quite heavy operations, but you know, it's, it's hardware, so we have to do it. Another mention is we're able to use the existing 3D ecosystem. You probably heard the previous speaker at Chin on, on, on, on real sense. So we use real sense. I show you how to map facial expression onto 3D avatars, for example. So we were on CNN recently. We can take your facial expression and map them onto a monkey. So it looks like, you know, we're transforming you into a monkey and it's, it's 3D and it's really nice. So we have a unity plugin that lets us take any unity content and, and essentially make it compatible for our platform. So you can play unity games on our platform. We do live motion. We have WebGL. So you can go and have fun online. So most of our demos are rendered in the browser today and it's, it's, it's quick and easy to, to program in JavaScript. So that's an example. I think if you were there last night, some of you might have maybe got to play. So this is like a, this is a CAT scan of, of actually it was a former colleague of mine at HP and turned into a 3D model of his spine. And you get to interact with it with, with a leap motion in real time. You can, you know, you can manipulate the hologram. It's quite fun. This is the demo that I mentioned. This is essentially what you would see, you're basically in front of the display. We have a real sense unit that is capturing your facial expression. And in real time, it is essentially mapping this expression onto this, this 3D hologram of a monkey. So that's quite fun. And I want to leave you with that last image of, you know, the vision again, it's, it's the digital world materialized. And I think so, you know, this is really meant to be the user interface of the future and essentially hope we'll get there soon. So thank you for your attention. Thank you.
 David Fatahou, and he's from Leia3D, and his speech will be on holographic reality.  We're talking a lot about virtual reality, augmented reality these days, and what I want  to present to you is a concept that we call holographic reality, which is really, we want  a strong belief that virtual world is good on headset, it's very immersive, but there's  a lot of application, actually a lot more that could be on headset, that could be on  regular screen with the usual way that you actually interact with the digital world,  which is when you want it, you turn it on, you interact with it, and then when you're  done you just look away and then you're kind of back to your regular life, and that you  can do with a headset.  So what I want to talk about, so I'm David Fatahou, I'm the founder and CEO of a startup  from Menlobark called Leia Inc., we're a spinoff from HP, and I'm going to be describing a  little bit of our vision, our technology, and some of the applications, everything in  less than 20 minutes, okay?  So the vision of the company is this, right?  I think today all of our information is obviously transitioning from the physical world to the  digital world, our communication, our knowledge, the way we entertain, the way we work everything  is now on servers in the cloud, yet, you know, we're still made of flesh and bones, and  you know, we still essentially need to experience all that information in a physical form, which  means we need to find ways, intuitive ways to bring back the digital world into our own,  okay?  This is fundamentally what you do with a 2D display, right?  I'm trying to talk at a very abstract level here.  A 2D display takes bits and bytes and essentially materializes it into a 2D image on the screen,  possibly with a touch screen, you can interact with it, and you're getting essentially to  interact with the digital world.  And that's the way that we've been interacting with technology, it's been about 10 years  now on your iPhone and so on.  Obviously, you know, the same way that we transition from black and white TV to color  TV, it's been a while it's been predicted we will transition from, you know, flat screens  to 3D screens, okay?  The world is not flat, so if you want to get a more intuitive interaction with the digital  world, you should somehow be able to render the digital world in the third dimension.  So recently, right, there's been a lot of craze about VR AR, and they do provide that  immersion.  It's actually very easy to do 3D.  It's quite easy to do 3D with a VR AR headset because you just have two different displays  that are following your eyes at all time.  And as I said in my mini introduction earlier, right, they come at the expense of having to  wear bulky eyewear, which impairs mobility, social interactions.  It's quite funny actually, VR AR what happened is that you get to collaborate and to interact  with people on other continents, but you're actually isolated from the person next to  you, right?  So if you want to maybe enjoy, you know, a football game in a bar, usually you want to  be close to the people around you, you're not going to isolate yourself with the headset.  And you walk in the street and you want to place a quick phone call.  You don't want to, I want to have to wear a headset with a mini computer attached to  your belt, you know, at all times.  Emerging solution, right, is the topic I think of this conference is, you know, stereoscopic  3D displays, in particular naked eye 3D displays that are able to render the digital world in  3D in a completely unobstructive way, just from the screen.  However, due to restriction in field of view or in the eye tracking case, there's a little  bit of lag and so on and so forth, this still do not offer the total freedom of movement  that you would expect from interacting with technology, right?  So there's actually no good solution today and what I'm trying to present here, what  we're developing at Leia is this quote-unquote holographic platform that is interactive,  mobile, intuitive and that essentially lets you interact with holographic quote-unquote  objects from any position in an unrestrictive way.  And so this is that platform that we call holographic reality, okay?  So when I say holograms, I don't want to get into the old debate about what's a hologram,  you know, hologram in the broad sense that it's an object that looks and feel like a  hologram which means it pops out of the screen and when you move around it, it has very smooth  parallax transition and you get to see all kind of perspective around it without jumps  and anything like that.  So think of it as a very good, a very good multi-view 3D display, for example.  And what we're building at Leia is not only we have that capability to render 3D images  very, very nicely with very smooth sense of parallax but we augment it with so-called  hover touch technology that lets essentially senses your finger above the screen without  having to touch the screen.  So we can interact with the holograms essentially without touching the screen.  Some of you might have gotten a demo yesterday.  I think we were there at the demo session.  I think you could manipulate the skeleton by just, you know, moving your finger around  the screen.  It's also in our ambition to integrate so-called haptics and this is essentially ultrasonic  technology.  You emit ultrasound from the side of the display.  If you know where your finger is located, you can actually make the ultrasound interfere  in phase under your finger and you'll feel a physical sense of touch.  So at that point, you'll be able not only to visualize holographic content and interact  with it but you'll be able to actually feel it.  So this is the digital world that is literally materialized into the real world and that's  basically what we believe is going to be the user interface of the future.  Might it be on your mobile phone, might be in the car, in your home, at the office and  the vision for, you know, from five to ten years is our world is going to be filled with  displays, every single door, every single table, every single, you know, place in your  car, in your home, at the office, in public places, is going to be filled with displays,  an interactive displays where you can summon the digital world, do some kind of operation  and then get on with your life.  All right, so that's what we're trying to build.  So a little bit about the product today, what it is exactly.  We managed to do that with a regular LCD technology.  We're 100% compatible with the current LCD supply chain in Asia.  And so what we have is we take an LCD display, we actually keep the standard LCD panel on  top, you know, commodity technology, low margin and so on and so forth.  There's a bunch of vendors in Asia that you can work with.  And essentially what we do is we replace the backlight and we replace it with our own backlight  that we call a multi-view backlight that has essentially some diffractive pattern on it.  So it's a proprietary diffractive pattern that is able to take side illumination light and  create what has become to be known as a light field.  So we create a light field, a static light field underneath the LCD display.  So we create a bunch of colored light rays.  We know where these light rays come from.  They come from different locations.  They propagate in different directions.  And they actually have a very controlled, and I'll come back to that, they have a very  controlled angular spread.  So when these light rays come through the LCD, we can modulate their intensity and what  you get is a multi-view 3D image.  That is not only full parallax in a wide angle, but it's actually very, has very smooth view  transitions.  So you don't get the usual jumps that you can see with maybe lenticular or other type  of displays.  It's actually very, very smooth.  And because it's wide angle and because it's very smooth, it lets you essentially interact.  It lets you move around the display very comfortably and it lets you interact with it.  So that's the difference between having a narrow field of view and having a big field  of view is when you have a big field of view, you forget about the technology and you just  go on and go on with your life and essentially interact with the device.  Not having to worry, am I going to lose the image or not?  And the last layer that faces the user is essentially the LCD display itself, right?  So unlike an anticular technology where you have to add a bunch of lenses on top, you're  actually facing the LCD display.  So we can use right away touch technologies, hover touch, you know, capacitive touch, and  then we'll be compatible with haptics as well, essentially without having to change much  of the technology.  So that's essentially the product.  And now I'm going to be talking a little bit more about the backlight.  But first, I'm just going to talk about what kind of data needs to be fed into the display.  It is still a light field display.  This is still essentially multi-view 3D display.  So you don't have to compute diffraction fringes or anything like that.  The technology is diffraction based, but the diffraction pattern is fixed.  It doesn't have to change.  And it turns out all you have to feed the display are the equivalent of a certain number  of views that are computed, you know, on a computer or acquired through cameras, for  example.  So the very noticeable fact is we can, we can, I don't like to call it crosstalk, but  the way the transition, the way you transition from one view to the next is a very precise  function of the diffraction pattern.  And we can tune it to whatever degree we want.  So we can completely kill the crosstalk.  We can have zero crosstalk.  Or we can introduce some crosstalk to smooth out the transition.  And this is done in the hardware.  So that idea of a multi-view backlight came from actually, in another life I was a researcher  at HP Labs.  And we actually made the cover of Nature for that in 2013.  And you see another aspect that I haven't mentioned is the backlight is completely transparent.  So we can create holograms that are coming, you know, that look like they're emerging  from a transparent piece of glass.  So it's quite magical.  So here you have piece of glass, it's illuminated from the side.  And it's essentially creating this, by our standard today, this is a horrible image.  It's very blurry, everything.  But that was, you know, the proof of concept back then.  And because it's transparent, what it allows you to do is also, if we were to work with  the transparent modulator, we could do a holographic display, 3D display that is, that  is see-through transparent, okay?  Which of course you can do with a parallax barrier display or a lenticular type display,  right?  So we can, we're compatible with see-through and 3D at the same time.  So the principle is this, right?  It's basically, we call it diffraction of guided wave.  So if you look a little bit inside the backlight, we have a side illumination.  This is, you know, this doesn't have all the full information on purpose, but just to give  you a gist of how it works.  You couple light from the side as in a normal backlighting system.  And you have this diffractive pattern that can essentially control exactly how the light  is extracted.  So every time the light interacts with the diffractive pattern, a little bit is essentially  diffracted there in this mode.  And then most of it is just reflected and then reflected and then has the chance to interact  again.  So it's very efficient.  In the end, we can, we managed to use most of the light in the light guide.  And as I was saying, we control not only the direction and the location of this light  rays, but we control exactly how they, their angular distribution.  And that's very important.  We can even use the same diffraction pattern to create two light rays with actually arbitrary  direction.  Sometime we use that to increase the density of pixels and the density of views available  and so on and so forth.  So it's very versatile.  And a great advantage compared to a lenticular display, for example, is that diffraction  doesn't care how tight the angle of the ray is going to be.  If you want to send a light ray in a very grazing direction, you can.  This is the same as sending it up.  With a lenticular display, you need to be close to axis.  If you deviate too much from the parallax approximation, you're going to get blurry  images.  But here you don't care.  So as I mentioned before, it turns out that this diffractive pattern is going to act,  is going to diffract the light that is guided in the light guide.  But for light that is coming straight through, it has no diffraction effect whatsoever.  So the light can go through completely unaffected.  So that has to do with the transverse momentum of light.  Light that is in the light guide has a very large transverse momentum, which means that  in order to diffract it, you need a feature size that is pretty small.  And when the light comes with near normal incidence with almost zero degree angle, it  has a very small amount of momentum there.  And this essentially, this grading gives too much of a momentum kick in either direction,  and then there's no propagating order that corresponds to it.  So again, it's completely transparent.  We can give again, we have full freedom.  So even if you work with the square array of LCD pixel, you can still define.  You have the freedom to define an arbitrary set of views.  So for example, here was an example, an early example where we build a hexagonal set of  views through a square LCD panel, if you wanted to, just to show you the freedom.  And again, what I alluded to that's very important to understand is that each view,  we can control all aspects of the radiation pattern.  So this one, we wanted to have a flat top, and then we wanted to control a certain, essentially  a amount of overlap between the views to create a certain smoothness effect.  And then we also wanted the image to kind of die down.  So this had eight views, horizontal view one through eight.  We wanted view one and view eight to kind of die down so that there was, you know, this  is the next, this is essentially, this is the next view here that would be view number  nine and it's completely suppressed.  So when you look at the display and you move from the left to the right, you see a large  amount of parallax, 60 degree field of view.  And then if you move too far, instead of seeing a jump back to view one, you know, just the,  essentially the brightness decreases and you fade away just very gracefully, right?  So it doesn't jump.  And at no point you have the sensation of having this inverted 3D, you know, sensation  where the left eye image goes to the right eye and vice versa.  So it's actually, it is very pleasant.  So there's one, there's one usually pain with diffraction is how they do deal with color.  So everything I've described so far works well with one color.  But now if you take the usual diffraction or holographic kind of configuration, if you  send white light on a diffraction pattern, most of you know that what you will get is  for first of all, most of the light will go through an undefracted.  So we solve that by illuminating with guided light.  There is just, there is no zero order propagating order.  So this never happens for us.  But which, which the thing that could be more disastrous is the first diffracted order gives  you a lot of, a lot of color dispersion.  So essentially, and actually it's drawn wrong here.  The red should be here and the blue should be there.  But essentially this should completely break down the colors.  You should have, you know, a color in a, you know, in one, one certain range of angles  and then another color there, another color there.  Most of our patents actually have to do with how we deal with that problem.  And we have, we illuminated the light guide in a special way with either with white light  or different color of light.  But we have to be very careful how we illuminated the system and what we get are actually light  rays that are possibly coming from different position but, but that are parallel.  So they define the same exact set of views.  So that when you look at our display, you don't see any color bleeding.  If you want to create a white object, it's perfectly white, including at the edges.  That's the really hard part about diffraction.  So that's where all the, all the work has to, has to go when you look in the detail to  have a high quality display.  Perfect.  So now it's just a little bit of a, not the marketing, but I want to describe.  So the company, just, just to tell you what it, what it takes, right?  So we found it two years ago.  We spun off HP Labs.  We're about 25 employees and we have operation in Menlo Park.  Palo Alto, believe it or not, we actually, we bought an ASML stepper and that's how we  prototype all of our displays.  So the stepper is a optical lithography tool that can, we have a, we can go down to 85  nanometer resolution.  Okay.  So we have 12 inch wafers.  So any display that is small, that would fit in a 12 inch wafer, we can prototype in about  less than a week.  Just the, the time it takes to get a, a mask from Topan, which is very fast.  And essentially, so we, we prototype and we manufacture the backlight still in Palo Alto  and then we have a pilot assembly line in Suzhou that aligns the backlight to the LCD  panel.  And from that pilot line, we transfer the process to the module assembly house in, in  Asia, usually in China.  So it's, it's, it's quite heavy operations, but you know, it's, it's hardware, so we  have to do it.  Another mention is we're able to use the existing 3D ecosystem.  You probably heard the previous speaker at Chin on, on, on, on real sense.  So we use real sense.  I show you how to map facial expression onto 3D avatars, for example.  So we were on CNN recently.  We can take your facial expression and map them onto a monkey.  So it looks like, you know, we're transforming you into a monkey and it's, it's 3D and it's  really nice.  So we have a unity plugin that lets us take any unity content and, and essentially make  it compatible for our platform.  So you can play unity games on our platform.  We do live motion.  We have WebGL.  So you can go and have fun online.  So most of our demos are rendered in the browser today and it's, it's, it's quick and  easy to, to program in JavaScript.  So that's an example.  I think if you were there last night, some of you might have maybe got to play.  So this is like a, this is a CAT scan of, of actually it was a former colleague of mine  at HP and turned into a 3D model of his spine.  And you get to interact with it with, with a leap motion in real time.  You can, you know, you can manipulate the hologram.  It's quite fun.  This is the demo that I mentioned.  This is essentially what you would see, you're basically in front of the display.  We have a real sense unit that is capturing your facial expression.  And in real time, it is essentially mapping this expression onto this, this 3D hologram  of a monkey.  So that's quite fun.  And I want to leave you with that last image of, you know, the vision again, it's, it's  the digital world materialized.  And I think so, you know, this is really meant to be the user interface of the future and  essentially hope we'll get there soon.  So thank you for your attention.  Thank you.
SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_0_203.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_204_371.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_372_1187.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_1188_1343.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_1344_1559.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_1560_1643.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_1644_9647.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9648_9695.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9696_9743.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9744_9803.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9804_9851.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9852_9899.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9900_9947.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9948_9995.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_9996_10043.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_10044_10091.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_10092_15155.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_15156_16619.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_16620_19115.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_19116_21047.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_21048_22751.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_22752_25379.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_25380_25439.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_25440_28835.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_28836_29255.png SD_A_2016__LEIA_3D__holographic_reality_-I9namlza4U_29256_30070.png
David Fatahou, and he's from Leia3D, and his speech will be on holographic reality. We're talking a lot about virtual reality, augmented reality these days, and what I want to present to you is a concept that we call holographic reality, which is really, we want a strong belief that virtual world is good on headset, it's very immersive, but there's a lot of application, actually a lot more that could be on headset, that could be on regular screen with the usual way that you actually interact with the digital world, which is when you want it, you turn it on, you interact with it, and then when you're done you just look away and then you're kind of back to your regular life, and that you can do with a headset. So what I want to talk about, so I'm David Fatahou, I'm the founder and CEO of a startup from Menlobark called Leia Inc., we're a spinoff from HP, and I'm going to be describing a little bit of our vision, our technology, and some of the applications, everything in less than 20 minutes, okay? So the vision of the company is this, right? I think today all of our information is obviously transitioning from the physical world to the digital world, our communication, our knowledge, the way we entertain, the way we work everything is now on servers in the cloud, yet, you know, we're still made of flesh and bones, and you know, we still essentially need to experience all that information in a physical form, which means we need to find ways, intuitive ways to bring back the digital world into our own, okay? This is fundamentally what you do with a 2D display, right? I'm trying to talk at a very abstract level here. A 2D display takes bits and bytes and essentially materializes it into a 2D image on the screen, possibly with a touch screen, you can interact with it, and you're getting essentially to interact with the digital world. And that's the way that we've been interacting with technology, it's been about 10 years now on your iPhone and so on. Obviously, you know, the same way that we transition from black and white TV to color TV, it's been a while it's been predicted we will transition from, you know, flat screens to 3D screens, okay? The world is not flat, so if you want to get a more intuitive interaction with the digital world, you should somehow be able to render the digital world in the third dimension. So recently, right, there's been a lot of craze about VR AR, and they do provide that immersion. It's actually very easy to do 3D. It's quite easy to do 3D with a VR AR headset because you just have two different displays that are following your eyes at all time. And as I said in my mini introduction earlier, right, they come at the expense of having to wear bulky eyewear, which impairs mobility, social interactions. It's quite funny actually, VR AR what happened is that you get to collaborate and to interact with people on other continents, but you're actually isolated from the person next to you, right? So if you want to maybe enjoy, you know, a football game in a bar, usually you want to be close to the people around you, you're not going to isolate yourself with the headset. And you walk in the street and you want to place a quick phone call. You don't want to, I want to have to wear a headset with a mini computer attached to your belt, you know, at all times. Emerging solution, right, is the topic I think of this conference is, you know, stereoscopic 3D displays, in particular naked eye 3D displays that are able to render the digital world in 3D in a completely unobstructive way, just from the screen. However, due to restriction in field of view or in the eye tracking case, there's a little bit of lag and so on and so forth, this still do not offer the total freedom of movement that you would expect from interacting with technology, right? So there's actually no good solution today and what I'm trying to present here, what we're developing at Leia is this quote-unquote holographic platform that is interactive, mobile, intuitive and that essentially lets you interact with holographic quote-unquote objects from any position in an unrestrictive way. And so this is that platform that we call holographic reality, okay? So when I say holograms, I don't want to get into the old debate about what's a hologram, you know, hologram in the broad sense that it's an object that looks and feel like a hologram which means it pops out of the screen and when you move around it, it has very smooth parallax transition and you get to see all kind of perspective around it without jumps and anything like that. So think of it as a very good, a very good multi-view 3D display, for example. And what we're building at Leia is not only we have that capability to render 3D images very, very nicely with very smooth sense of parallax but we augment it with so-called hover touch technology that lets essentially senses your finger above the screen without having to touch the screen. So we can interact with the holograms essentially without touching the screen. Some of you might have gotten a demo yesterday. I think we were there at the demo session. I think you could manipulate the skeleton by just, you know, moving your finger around the screen. It's also in our ambition to integrate so-called haptics and this is essentially ultrasonic technology. You emit ultrasound from the side of the display. If you know where your finger is located, you can actually make the ultrasound interfere in phase under your finger and you'll feel a physical sense of touch. So at that point, you'll be able not only to visualize holographic content and interact with it but you'll be able to actually feel it. So this is the digital world that is literally materialized into the real world and that's basically what we believe is going to be the user interface of the future. Might it be on your mobile phone, might be in the car, in your home, at the office and the vision for, you know, from five to ten years is our world is going to be filled with displays, every single door, every single table, every single, you know, place in your car, in your home, at the office, in public places, is going to be filled with displays, an interactive displays where you can summon the digital world, do some kind of operation and then get on with your life. All right, so that's what we're trying to build. So a little bit about the product today, what it is exactly. We managed to do that with a regular LCD technology. We're 100% compatible with the current LCD supply chain in Asia. And so what we have is we take an LCD display, we actually keep the standard LCD panel on top, you know, commodity technology, low margin and so on and so forth. There's a bunch of vendors in Asia that you can work with. And essentially what we do is we replace the backlight and we replace it with our own backlight that we call a multi-view backlight that has essentially some diffractive pattern on it. So it's a proprietary diffractive pattern that is able to take side illumination light and create what has become to be known as a light field. So we create a light field, a static light field underneath the LCD display. So we create a bunch of colored light rays. We know where these light rays come from. They come from different locations. They propagate in different directions. And they actually have a very controlled, and I'll come back to that, they have a very controlled angular spread. So when these light rays come through the LCD, we can modulate their intensity and what you get is a multi-view 3D image. That is not only full parallax in a wide angle, but it's actually very, has very smooth view transitions. So you don't get the usual jumps that you can see with maybe lenticular or other type of displays. It's actually very, very smooth. And because it's wide angle and because it's very smooth, it lets you essentially interact. It lets you move around the display very comfortably and it lets you interact with it. So that's the difference between having a narrow field of view and having a big field of view is when you have a big field of view, you forget about the technology and you just go on and go on with your life and essentially interact with the device. Not having to worry, am I going to lose the image or not? And the last layer that faces the user is essentially the LCD display itself, right? So unlike an anticular technology where you have to add a bunch of lenses on top, you're actually facing the LCD display. So we can use right away touch technologies, hover touch, you know, capacitive touch, and then we'll be compatible with haptics as well, essentially without having to change much of the technology. So that's essentially the product. And now I'm going to be talking a little bit more about the backlight. But first, I'm just going to talk about what kind of data needs to be fed into the display. It is still a light field display. This is still essentially multi-view 3D display. So you don't have to compute diffraction fringes or anything like that. The technology is diffraction based, but the diffraction pattern is fixed. It doesn't have to change. And it turns out all you have to feed the display are the equivalent of a certain number of views that are computed, you know, on a computer or acquired through cameras, for example. So the very noticeable fact is we can, we can, I don't like to call it crosstalk, but the way the transition, the way you transition from one view to the next is a very precise function of the diffraction pattern. And we can tune it to whatever degree we want. So we can completely kill the crosstalk. We can have zero crosstalk. Or we can introduce some crosstalk to smooth out the transition. And this is done in the hardware. So that idea of a multi-view backlight came from actually, in another life I was a researcher at HP Labs. And we actually made the cover of Nature for that in 2013. And you see another aspect that I haven't mentioned is the backlight is completely transparent. So we can create holograms that are coming, you know, that look like they're emerging from a transparent piece of glass. So it's quite magical. So here you have piece of glass, it's illuminated from the side. And it's essentially creating this, by our standard today, this is a horrible image. It's very blurry, everything. But that was, you know, the proof of concept back then. And because it's transparent, what it allows you to do is also, if we were to work with the transparent modulator, we could do a holographic display, 3D display that is, that is see-through transparent, okay? Which of course you can do with a parallax barrier display or a lenticular type display, right? So we can, we're compatible with see-through and 3D at the same time. So the principle is this, right? It's basically, we call it diffraction of guided wave. So if you look a little bit inside the backlight, we have a side illumination. This is, you know, this doesn't have all the full information on purpose, but just to give you a gist of how it works. You couple light from the side as in a normal backlighting system. And you have this diffractive pattern that can essentially control exactly how the light is extracted. So every time the light interacts with the diffractive pattern, a little bit is essentially diffracted there in this mode. And then most of it is just reflected and then reflected and then has the chance to interact again. So it's very efficient. In the end, we can, we managed to use most of the light in the light guide. And as I was saying, we control not only the direction and the location of this light rays, but we control exactly how they, their angular distribution. And that's very important. We can even use the same diffraction pattern to create two light rays with actually arbitrary direction. Sometime we use that to increase the density of pixels and the density of views available and so on and so forth. So it's very versatile. And a great advantage compared to a lenticular display, for example, is that diffraction doesn't care how tight the angle of the ray is going to be. If you want to send a light ray in a very grazing direction, you can. This is the same as sending it up. With a lenticular display, you need to be close to axis. If you deviate too much from the parallax approximation, you're going to get blurry images. But here you don't care. So as I mentioned before, it turns out that this diffractive pattern is going to act, is going to diffract the light that is guided in the light guide. But for light that is coming straight through, it has no diffraction effect whatsoever. So the light can go through completely unaffected. So that has to do with the transverse momentum of light. Light that is in the light guide has a very large transverse momentum, which means that in order to diffract it, you need a feature size that is pretty small. And when the light comes with near normal incidence with almost zero degree angle, it has a very small amount of momentum there. And this essentially, this grading gives too much of a momentum kick in either direction, and then there's no propagating order that corresponds to it. So again, it's completely transparent. We can give again, we have full freedom. So even if you work with the square array of LCD pixel, you can still define. You have the freedom to define an arbitrary set of views. So for example, here was an example, an early example where we build a hexagonal set of views through a square LCD panel, if you wanted to, just to show you the freedom. And again, what I alluded to that's very important to understand is that each view, we can control all aspects of the radiation pattern. So this one, we wanted to have a flat top, and then we wanted to control a certain, essentially a amount of overlap between the views to create a certain smoothness effect. And then we also wanted the image to kind of die down. So this had eight views, horizontal view one through eight. We wanted view one and view eight to kind of die down so that there was, you know, this is the next, this is essentially, this is the next view here that would be view number nine and it's completely suppressed. So when you look at the display and you move from the left to the right, you see a large amount of parallax, 60 degree field of view. And then if you move too far, instead of seeing a jump back to view one, you know, just the, essentially the brightness decreases and you fade away just very gracefully, right? So it doesn't jump. And at no point you have the sensation of having this inverted 3D, you know, sensation where the left eye image goes to the right eye and vice versa. So it's actually, it is very pleasant. So there's one, there's one usually pain with diffraction is how they do deal with color. So everything I've described so far works well with one color. But now if you take the usual diffraction or holographic kind of configuration, if you send white light on a diffraction pattern, most of you know that what you will get is for first of all, most of the light will go through an undefracted. So we solve that by illuminating with guided light. There is just, there is no zero order propagating order. So this never happens for us. But which, which the thing that could be more disastrous is the first diffracted order gives you a lot of, a lot of color dispersion. So essentially, and actually it's drawn wrong here. The red should be here and the blue should be there. But essentially this should completely break down the colors. You should have, you know, a color in a, you know, in one, one certain range of angles and then another color there, another color there. Most of our patents actually have to do with how we deal with that problem. And we have, we illuminated the light guide in a special way with either with white light or different color of light. But we have to be very careful how we illuminated the system and what we get are actually light rays that are possibly coming from different position but, but that are parallel. So they define the same exact set of views. So that when you look at our display, you don't see any color bleeding. If you want to create a white object, it's perfectly white, including at the edges. That's the really hard part about diffraction. So that's where all the, all the work has to, has to go when you look in the detail to have a high quality display. Perfect. So now it's just a little bit of a, not the marketing, but I want to describe. So the company, just, just to tell you what it, what it takes, right? So we found it two years ago. We spun off HP Labs. We're about 25 employees and we have operation in Menlo Park. Palo Alto, believe it or not, we actually, we bought an ASML stepper and that's how we prototype all of our displays. So the stepper is a optical lithography tool that can, we have a, we can go down to 85 nanometer resolution. Okay. So we have 12 inch wafers. So any display that is small, that would fit in a 12 inch wafer, we can prototype in about less than a week. Just the, the time it takes to get a, a mask from Topan, which is very fast. And essentially, so we, we prototype and we manufacture the backlight still in Palo Alto and then we have a pilot assembly line in Suzhou that aligns the backlight to the LCD panel. And from that pilot line, we transfer the process to the module assembly house in, in Asia, usually in China. So it's, it's, it's quite heavy operations, but you know, it's, it's hardware, so we have to do it. Another mention is we're able to use the existing 3D ecosystem. You probably heard the previous speaker at Chin on, on, on, on real sense. So we use real sense. I show you how to map facial expression onto 3D avatars, for example. So we were on CNN recently. We can take your facial expression and map them onto a monkey. So it looks like, you know, we're transforming you into a monkey and it's, it's 3D and it's really nice. So we have a unity plugin that lets us take any unity content and, and essentially make it compatible for our platform. So you can play unity games on our platform. We do live motion. We have WebGL. So you can go and have fun online. So most of our demos are rendered in the browser today and it's, it's, it's quick and easy to, to program in JavaScript. So that's an example. I think if you were there last night, some of you might have maybe got to play. So this is like a, this is a CAT scan of, of actually it was a former colleague of mine at HP and turned into a 3D model of his spine. And you get to interact with it with, with a leap motion in real time. You can, you know, you can manipulate the hologram. It's quite fun. This is the demo that I mentioned. This is essentially what you would see, you're basically in front of the display. We have a real sense unit that is capturing your facial expression. And in real time, it is essentially mapping this expression onto this, this 3D hologram of a monkey. So that's quite fun. And I want to leave you with that last image of, you know, the vision again, it's, it's the digital world materialized. And I think so, you know, this is really meant to be the user interface of the future and essentially hope we'll get there soon. So thank you for your attention. Thank you.
reality these days, and what I want to present to this, right? I think today all of our information ways to bring back the digital world into our own, okay? This is fundamentally what you do with you know, the same way that we transition from know, flat screens to 3D screens, okay? The world is not flat, so if you want to get a more collaborate and to interact with people on other in field of view or in the eye tracking case, interacting with technology, right? So there's that is interactive, mobile, intuitive and that finger around the screen. It's also in our sense of touch. So at that point, you'll be able not only to visualize holographic content and feel it. So this is the digital world that is usual jumps that you can see with maybe increase the density of pixels and the density of light that is coming straight through, it has no give again, we have full freedom. So even if you now if you take the usual diffraction or know that what you will get is for first of all, when you look at our display, you don't see any detail to have a high quality display. Perfect.