Transparent stereoscopic display and application
Ranieri, Nicola
Seifert, Hagen
Gross, Markus
Augmented reality has become important to our society as it can enrich the actual world with virtual information. Transparent screens offer one possibility to overlay rendered scenes with the environment, acting both as display and window. In this work, we review existing transparent back-projection screens for the use with active and passive stereo. Advantages and limitations are described and, based on these insights, a passive stereoscopic system using an anisotropic back-projection foil is proposed. To increase realism, we adapt rendered content to the viewer's position using a Kinect tracking system, which adds motion parallax to the binocular cues. A technique well known in control engineering is used to decrease latency and increase frequency of the tracker. Our transparent stereoscopic display prototype provides immersive viewing experience and is suitable for many augmented reality applications. 
 So, my talk will be about Transparency, Aeroscopic Display and Application. And let me first start with a small motivation, wide transparency. You see on the left an image from Swiss train station. You see this typical information panels which are huge hanging from the ceiling, giving information about the departures of the trains. And on the right hand side you see a typical scene if you go shopping, where you see some promotions in the shop window. And both images share something in common. They expose important information to a viewer, but also they block important content from the background. The third image I took from our lab, it's actually our lab kitchen. On the right hand side you see the most important thing in the lab, the coffee machine. But to this talk more important is the fridge, because our people just put their stuff in the fridge and expiry date vanishes and it starts smelling. So what do we want to say with this picture? It would be very convenient for us as users if the door of the fridge would be transparent. And also 3D, such that next to each item in the fridge we can display information about the owner of the item and also about like the expiry date or something like this. So transparency will definitely add value to our environment since we can augment it with virtual information. The idea of the transparent fridge has been picked up by industries already, so LG and also Samsung I think have one, have these fridges with transparent doors. Samsung has this light box, which we already heard earlier today, where you can put some item inside of it and put some information on front. Both are based on LCD technology. The problem with LCD, if it's a black and white LCD, it absorbs at least 50% due to the underlying technology. If you want a color screen, it even absorbs more light. So these technologies can only be deployed in environment with a controlled background light. And usually the backlight is very strong, has to be very strong to get these nice images, as we can see here. Another display technology which I like a lot is from Sun Innovations. They deploy a phosphoric material inside of a glass panel. This phosphoric material can be activated by a certain wavelength of light. So it's a projection setup and it becomes self-emissive, very nice technology. And also something to mention here are head mounted displays, which also can overlay virtual information with reality. Now for the rest of the talk, I want to focus more on this pack projection kind of screens like on the right. But we used more simple ones in our project and we compared two kinds of pack projections with transparent back projection screens. So first the top one is the isotropic back projection screen. As it works, usually they deploy small droplets inside of a transparent material. Most of the light will just pass through. This gives the screen transparent clear feature. But some of the light will scatter at these droplets and make a projected image visible. It's isotropic because you can put the projector at any position behind the screen and it will work. A little bit more complex is the anisotropic back projection foil or glass which only selectively diffuses light. So these screens, the one we used is holography based so it's a diffraction grating and it just selectively from one center of projection diffuses the light, which means all light from the environment will just pass through giving again a transparent screen. But if you put a projector into this optimized center of projection, this will be visible to a viewer in front. We compare both screens by putting just a checkerboard behind the screen and taking a photograph in front and compared to the ground truth without any screen. And what you can see here is that the isotropic screen has a little bit less contrast which is due to that all environment light, even ambient light will be scattered by the screen. The anisotropic is a little bit better in terms of contrast because it only selectively diffuses light coming from the projector itself. We also compare both screens to passive and active glasses to be used together with those. You will find the complete summary and also this table in our paper. Something I want to highlight here is the light inefficiency of the anisotropic screen. The problem is that the first order diffraction will just pass us through directly and won't be diffused and won't be redirected as illustrated in the figure. So there is a huge light loss. Anyway we have chosen for our project the anisotropic glass in combination with passive glasses because first of all we like the nice contrast and the nice image quality of the anisotropic screen and for the passive glasses it's just convenient because they are cheap and we can just hand them out to possible viewers. So now about rendering content, it has already been mentioned in the two talks before. If you don't have any motion parallax, if you render something and you move your head and don't adapt the rendering then you will see a shift in the rendered object behind. This will get worse with transparent screens because with transparent screens you have the reality, the surrounding reality as a reference point. So you will notice this shift if you don't have motion parallax even more with those screens. So if you just render the same content for a Novel View point you will see that the star in this sample will shift as indicated by the red line but what we rather want to have is a re-rendering, so an offset in the image plane of this star such that it is perceived to not move at all. In our system we use the Kinect head tracker to get an estimate of the viewer position. Now the problem with the Kinect and other head tracking systems is the frame rate as well as the latency. Why is that? Imagine that you have an update from the head tracker and you render the corresponding frame then you start moving your head, the virtual object will shift until you get the next update, 33 milliseconds later in the case of Kinect and then this object will jump back. So you get a wiggling of the object if you move your head and this destroys the motion parallax, the feeling, the immersive feeling of motion parallax. So how worse is it? Let's compute through an example with delta t just 33 milliseconds. We assume a viewer motion of 1 meter per second which makes up for offset of 3.3 centimeters between two frames coming from the Kinect. In our setup we just assume equal distance between screen and viewer and screen and virtual object so there is no amplification and this 3.3 centimeters directly translates to the virtual object. And 3.3 centimeters is not too much but it's already enough to destroy the immersive feeling of motion parallax. So we need to do something more clever both about the frame rate, 33 milliseconds and even worse for the latency which is 125 milliseconds for the Kinect. And here we borrow some insights from control engineering as we already heard in the first talk which is called Kalman filtering. And a Kalman filtering is basically a fusion of a physical model and an observation. And in our case we just model the head according to the laws of motion so our state vector is a position s and the velocity s prime and we update the position according to the velocity times the time step. The Kalman filter itself is an execution in two steps so we have first prediction according to our model and then as soon as we get input from the Kinect we correct our prediction to get better state according to reality. And this is a very basic view on Kalman filtering. Kalman filtering is much more complex so it's also very interesting and I want to recommend if you are interested to read related work about Kalman filtering. For us here it's just most important that the Kalman filter can be used to super sample this discrete signal that we get from the Kinect to create an arbitrary smooth curve as well as we can do a prediction. Prediction is good to get along with the latency of the Kinect. So what do I mean with that? Let's have a look at this plot you see on one axis the time and on the other axis the position of the head so it's a motion like from the head to the left to the right and back to the middle again. The black dots is the sampling signal of the Kinect and the black line through these dots is what you get by purely filtering so we can at least account for this 33 milliseconds which correspond to the frame rate by smoothly interpolating which the Kalman filter does itself already but also we can predict the signal. An ideal prediction of the signal would be just a shifted replica to the left in this plot and you can already see like on the orange line that it pretty works very well but on the green line you will see you will notice an overshoot on the top and also on the bottom where the green curve takes some values which are lower than the black line. So what it means for reality is that when you move your head that the predicted curve things your head is further gone further than you really did and this is again a problem because this overshoot destroys again the immersion. So with this prediction we have to be careful. We have to predict to account for the latency but if you predict too much then we get problems at the end of the motion and you need to find a good balance and in our case we found that half the latency of the Kinect was a good balance between these two errors. So the system that we propose consists of an anisotropic glass which you can see in front with the rendered Ironman on top of it. We use two projectors with linear polarizers in front with cross polarized axis and the viewer is required to wear corresponding linear polarizing glasses. You also see the Kinect in the front and in this image we didn't use any polarizer in front of the camera which captured this image so you see the double image of the Ironman. Calibration is very simple we just project checkerboards in both projectors and we compute the homographies between projector coordinates and display coordinates and to the rectification according to that. Another something I want to highlight is that all components are off-shelf so you really can take our paper look up the references by the components and assemble the display at home and you should really try it out because it's a very immersive feeling that you get. So the visitors of our lab liked it a lot. We think mostly it comes from the transparency as I mentioned you get like this reference points from the reality from the surrounding and also because it's a good accommodation queues and then overlaid with this the virtual reality which has still the ghosting effect which will probably be overcome by Quinn's inventions he just presented before. But also the light white glasses and the motion parallax add like the immersive to the immersive feeling and people walked around and could see Ironman from all sides. Supported is also the true size of Ironman he really is life size and you really get the feeling as if he's standing behind the glass panel. The problem however if you move faster then you get this overshoot and you lose this immersion and it looks like a rendering again. So as a conclusion we are approaching we are not completely there yet but with the next generation of head tracking which have higher frame rates and also low latency we can apply the same technique and we will get a good motion parallax and good immersion into this autostereoscopic content. With this I want to conclude I want to thank you for your attention and maybe there's time for some questions.
 So, my talk will be about Transparency, Aeroscopic Display and Application.  And let me first start with a small motivation, wide transparency.  You see on the left an image from Swiss train station.  You see this typical information panels which are huge hanging from the ceiling, giving  information about the departures of the trains.  And on the right hand side you see a typical scene if you go shopping, where you see some  promotions in the shop window.  And both images share something in common.  They expose important information to a viewer, but also they block important content from  the background.  The third image I took from our lab, it's actually our lab kitchen.  On the right hand side you see the most important thing in the lab, the coffee machine.  But to this talk more important is the fridge, because our people just put their stuff in  the fridge and expiry date vanishes and it starts smelling.  So what do we want to say with this picture?  It would be very convenient for us as users if the door of the fridge would be transparent.  And also 3D, such that next to each item in the fridge we can display information about  the owner of the item and also about like the expiry date or something like this.  So transparency will definitely add value to our environment since we can augment it  with virtual information.  The idea of the transparent fridge has been picked up by industries already, so LG and  also Samsung I think have one, have these fridges with transparent doors.  Samsung has this light box, which we already heard earlier today, where you can put some  item inside of it and put some information on front.  Both are based on LCD technology.  The problem with LCD, if it's a black and white LCD, it absorbs at least 50% due to  the underlying technology.  If you want a color screen, it even absorbs more light.  So these technologies can only be deployed in environment with a controlled background  light.  And usually the backlight is very strong, has to be very strong to get these nice images,  as we can see here.  Another display technology which I like a lot is from Sun Innovations.  They deploy a phosphoric material inside of a glass panel.  This phosphoric material can be activated by a certain wavelength of light.  So it's a projection setup and it becomes self-emissive, very nice technology.  And also something to mention here are head mounted displays, which also can overlay virtual  information with reality.  Now for the rest of the talk, I want to focus more on this pack projection kind of screens  like on the right.  But we used more simple ones in our project and we compared two kinds of pack projections  with transparent back projection screens.  So first the top one is the isotropic back projection screen.  As it works, usually they deploy small droplets inside of a transparent material.  Most of the light will just pass through.  This gives the screen transparent clear feature.  But some of the light will scatter at these droplets and make a projected image visible.  It's isotropic because you can put the projector at any position behind the screen and it will  work.  A little bit more complex is the anisotropic back projection foil or glass which only selectively  diffuses light.  So these screens, the one we used is holography based so it's a diffraction grating and it  just selectively from one center of projection diffuses the light, which means all light  from the environment will just pass through giving again a transparent screen.  But if you put a projector into this optimized center of projection, this will be visible  to a viewer in front.  We compare both screens by putting just a checkerboard behind the screen and taking  a photograph in front and compared to the ground truth without any screen.  And what you can see here is that the isotropic screen has a little bit less contrast which  is due to that all environment light, even ambient light will be scattered by the screen.  The anisotropic is a little bit better in terms of contrast because it only selectively  diffuses light coming from the projector itself.  We also compare both screens to passive and active glasses to be used together with those.  You will find the complete summary and also this table in our paper.  Something I want to highlight here is the light inefficiency of the anisotropic screen.  The problem is that the first order diffraction will just pass us through directly and won't  be diffused and won't be redirected as illustrated in the figure.  So there is a huge light loss.  Anyway we have chosen for our project the anisotropic glass in combination with passive glasses  because first of all we like the nice contrast and the nice image quality of the anisotropic  screen and for the passive glasses it's just convenient because they are cheap and we can  just hand them out to possible viewers.  So now about rendering content, it has already been mentioned in the two talks before.  If you don't have any motion parallax, if you render something and you move your head  and don't adapt the rendering then you will see a shift in the rendered object behind.  This will get worse with transparent screens because with transparent screens you have  the reality, the surrounding reality as a reference point.  So you will notice this shift if you don't have motion parallax even more with those  screens.  So if you just render the same content for a Novel View point you will see that the star  in this sample will shift as indicated by the red line but what we rather want to have  is a re-rendering, so an offset in the image plane of this star such that it is perceived  to not move at all.  In our system we use the Kinect head tracker to get an estimate of the viewer position.  Now the problem with the Kinect and other head tracking systems is the frame rate as  well as the latency.  Why is that?  Imagine that you have an update from the head tracker and you render the corresponding  frame then you start moving your head, the virtual object will shift until you get the  next update, 33 milliseconds later in the case of Kinect and then this object will jump  back.  So you get a wiggling of the object if you move your head and this destroys the motion  parallax, the feeling, the immersive feeling of motion parallax.  So how worse is it?  Let's compute through an example with delta t just 33 milliseconds.  We assume a viewer motion of 1 meter per second which makes up for offset of 3.3 centimeters  between two frames coming from the Kinect.  In our setup we just assume equal distance between screen and viewer and screen and virtual  object so there is no amplification and this 3.3 centimeters directly translates to the  virtual object.  And 3.3 centimeters is not too much but it's already enough to destroy the immersive feeling  of motion parallax.  So we need to do something more clever both about the frame rate, 33 milliseconds and  even worse for the latency which is 125 milliseconds for the Kinect.  And here we borrow some insights from control engineering as we already heard in the first  talk which is called Kalman filtering.  And a Kalman filtering is basically a fusion of a physical model and an observation.  And in our case we just model the head according to the laws of motion so our state vector  is a position s and the velocity s prime and we update the position according to the velocity  times the time step.  The Kalman filter itself is an execution in two steps so we have first prediction according  to our model and then as soon as we get input from the Kinect we correct our prediction  to get better state according to reality.  And this is a very basic view on Kalman filtering.  Kalman filtering is much more complex so it's also very interesting and I want to recommend  if you are interested to read related work about Kalman filtering.  For us here it's just most important that the Kalman filter can be used to super sample  this discrete signal that we get from the Kinect to create an arbitrary smooth curve  as well as we can do a prediction.  Prediction is good to get along with the latency of the Kinect.  So what do I mean with that?  Let's have a look at this plot you see on one axis the time and on the other axis the position  of the head so it's a motion like from the head to the left to the right and back to  the middle again.  The black dots is the sampling signal of the Kinect and the black line through these dots  is what you get by purely filtering so we can at least account for this 33 milliseconds  which correspond to the frame rate by smoothly interpolating which the Kalman filter does  itself already but also we can predict the signal.  An ideal prediction of the signal would be just a shifted replica to the left in this  plot and you can already see like on the orange line that it pretty works very well but on  the green line you will see you will notice an overshoot on the top and also on the bottom  where the green curve takes some values which are lower than the black line.  So what it means for reality is that when you move your head that the predicted curve  things your head is further gone further than you really did and this is again a problem  because this overshoot destroys again the immersion.  So with this prediction we have to be careful.  We have to predict to account for the latency but if you predict too much then we get problems  at the end of the motion and you need to find a good balance and in our case we found that  half the latency of the Kinect was a good balance between these two errors.  So the system that we propose consists of an anisotropic glass which you can see in front  with the rendered Ironman on top of it.  We use two projectors with linear polarizers in front with cross polarized axis and the  viewer is required to wear corresponding linear polarizing glasses.  You also see the Kinect in the front and in this image we didn't use any polarizer in  front of the camera which captured this image so you see the double image of the Ironman.  Calibration is very simple we just project checkerboards in both projectors and we compute  the homographies between projector coordinates and display coordinates and to the rectification  according to that.  Another something I want to highlight is that all components are off-shelf so you really  can take our paper look up the references by the components and assemble the display  at home and you should really try it out because it's a very immersive feeling that you get.  So the visitors of our lab liked it a lot.  We think mostly it comes from the transparency as I mentioned you get like this reference  points from the reality from the surrounding and also because it's a good accommodation  queues and then overlaid with this the virtual reality which has still the ghosting effect  which will probably be overcome by Quinn's inventions he just presented before.  But also the light white glasses and the motion parallax add like the immersive to the immersive  feeling and people walked around and could see Ironman from all sides.  Supported is also the true size of Ironman he really is life size and you really get  the feeling as if he's standing behind the glass panel.  The problem however if you move faster then you get this overshoot and you lose this immersion  and it looks like a rendering again.  So as a conclusion we are approaching we are not completely there yet but with the next  generation of head tracking which have higher frame rates and also low latency we can apply  the same technique and we will get a good motion parallax and good immersion into this  autostereoscopic content.  With this I want to conclude I want to thank you for your attention and maybe there's time  for some questions.
SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_0_131.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_132_251.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_252_431.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_432_2591.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_2592_2783.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_2784_2867.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_2868_3755.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_3756_4235.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_4236_4643.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_4644_8687.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_8688_10079.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_10080_11123.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_11124_12467.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_12468_12899.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_12900_13283.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_13284_14567.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_14568_17087.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_17088_18791.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_18792_20219.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_20220_20759.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_20760_20963.png SD_A_2014__Transparent_stereoscopic_display_and_application__9011-22__tG5XaMeXaG4_20964_21203.png
So, my talk will be about Transparency, Aeroscopic Display and Application. And let me first start with a small motivation, wide transparency. You see on the left an image from Swiss train station. You see this typical information panels which are huge hanging from the ceiling, giving information about the departures of the trains. And on the right hand side you see a typical scene if you go shopping, where you see some promotions in the shop window. And both images share something in common. They expose important information to a viewer, but also they block important content from the background. The third image I took from our lab, it's actually our lab kitchen. On the right hand side you see the most important thing in the lab, the coffee machine. But to this talk more important is the fridge, because our people just put their stuff in the fridge and expiry date vanishes and it starts smelling. So what do we want to say with this picture? It would be very convenient for us as users if the door of the fridge would be transparent. And also 3D, such that next to each item in the fridge we can display information about the owner of the item and also about like the expiry date or something like this. So transparency will definitely add value to our environment since we can augment it with virtual information. The idea of the transparent fridge has been picked up by industries already, so LG and also Samsung I think have one, have these fridges with transparent doors. Samsung has this light box, which we already heard earlier today, where you can put some item inside of it and put some information on front. Both are based on LCD technology. The problem with LCD, if it's a black and white LCD, it absorbs at least 50% due to the underlying technology. If you want a color screen, it even absorbs more light. So these technologies can only be deployed in environment with a controlled background light. And usually the backlight is very strong, has to be very strong to get these nice images, as we can see here. Another display technology which I like a lot is from Sun Innovations. They deploy a phosphoric material inside of a glass panel. This phosphoric material can be activated by a certain wavelength of light. So it's a projection setup and it becomes self-emissive, very nice technology. And also something to mention here are head mounted displays, which also can overlay virtual information with reality. Now for the rest of the talk, I want to focus more on this pack projection kind of screens like on the right. But we used more simple ones in our project and we compared two kinds of pack projections with transparent back projection screens. So first the top one is the isotropic back projection screen. As it works, usually they deploy small droplets inside of a transparent material. Most of the light will just pass through. This gives the screen transparent clear feature. But some of the light will scatter at these droplets and make a projected image visible. It's isotropic because you can put the projector at any position behind the screen and it will work. A little bit more complex is the anisotropic back projection foil or glass which only selectively diffuses light. So these screens, the one we used is holography based so it's a diffraction grating and it just selectively from one center of projection diffuses the light, which means all light from the environment will just pass through giving again a transparent screen. But if you put a projector into this optimized center of projection, this will be visible to a viewer in front. We compare both screens by putting just a checkerboard behind the screen and taking a photograph in front and compared to the ground truth without any screen. And what you can see here is that the isotropic screen has a little bit less contrast which is due to that all environment light, even ambient light will be scattered by the screen. The anisotropic is a little bit better in terms of contrast because it only selectively diffuses light coming from the projector itself. We also compare both screens to passive and active glasses to be used together with those. You will find the complete summary and also this table in our paper. Something I want to highlight here is the light inefficiency of the anisotropic screen. The problem is that the first order diffraction will just pass us through directly and won't be diffused and won't be redirected as illustrated in the figure. So there is a huge light loss. Anyway we have chosen for our project the anisotropic glass in combination with passive glasses because first of all we like the nice contrast and the nice image quality of the anisotropic screen and for the passive glasses it's just convenient because they are cheap and we can just hand them out to possible viewers. So now about rendering content, it has already been mentioned in the two talks before. If you don't have any motion parallax, if you render something and you move your head and don't adapt the rendering then you will see a shift in the rendered object behind. This will get worse with transparent screens because with transparent screens you have the reality, the surrounding reality as a reference point. So you will notice this shift if you don't have motion parallax even more with those screens. So if you just render the same content for a Novel View point you will see that the star in this sample will shift as indicated by the red line but what we rather want to have is a re-rendering, so an offset in the image plane of this star such that it is perceived to not move at all. In our system we use the Kinect head tracker to get an estimate of the viewer position. Now the problem with the Kinect and other head tracking systems is the frame rate as well as the latency. Why is that? Imagine that you have an update from the head tracker and you render the corresponding frame then you start moving your head, the virtual object will shift until you get the next update, 33 milliseconds later in the case of Kinect and then this object will jump back. So you get a wiggling of the object if you move your head and this destroys the motion parallax, the feeling, the immersive feeling of motion parallax. So how worse is it? Let's compute through an example with delta t just 33 milliseconds. We assume a viewer motion of 1 meter per second which makes up for offset of 3.3 centimeters between two frames coming from the Kinect. In our setup we just assume equal distance between screen and viewer and screen and virtual object so there is no amplification and this 3.3 centimeters directly translates to the virtual object. And 3.3 centimeters is not too much but it's already enough to destroy the immersive feeling of motion parallax. So we need to do something more clever both about the frame rate, 33 milliseconds and even worse for the latency which is 125 milliseconds for the Kinect. And here we borrow some insights from control engineering as we already heard in the first talk which is called Kalman filtering. And a Kalman filtering is basically a fusion of a physical model and an observation. And in our case we just model the head according to the laws of motion so our state vector is a position s and the velocity s prime and we update the position according to the velocity times the time step. The Kalman filter itself is an execution in two steps so we have first prediction according to our model and then as soon as we get input from the Kinect we correct our prediction to get better state according to reality. And this is a very basic view on Kalman filtering. Kalman filtering is much more complex so it's also very interesting and I want to recommend if you are interested to read related work about Kalman filtering. For us here it's just most important that the Kalman filter can be used to super sample this discrete signal that we get from the Kinect to create an arbitrary smooth curve as well as we can do a prediction. Prediction is good to get along with the latency of the Kinect. So what do I mean with that? Let's have a look at this plot you see on one axis the time and on the other axis the position of the head so it's a motion like from the head to the left to the right and back to the middle again. The black dots is the sampling signal of the Kinect and the black line through these dots is what you get by purely filtering so we can at least account for this 33 milliseconds which correspond to the frame rate by smoothly interpolating which the Kalman filter does itself already but also we can predict the signal. An ideal prediction of the signal would be just a shifted replica to the left in this plot and you can already see like on the orange line that it pretty works very well but on the green line you will see you will notice an overshoot on the top and also on the bottom where the green curve takes some values which are lower than the black line. So what it means for reality is that when you move your head that the predicted curve things your head is further gone further than you really did and this is again a problem because this overshoot destroys again the immersion. So with this prediction we have to be careful. We have to predict to account for the latency but if you predict too much then we get problems at the end of the motion and you need to find a good balance and in our case we found that half the latency of the Kinect was a good balance between these two errors. So the system that we propose consists of an anisotropic glass which you can see in front with the rendered Ironman on top of it. We use two projectors with linear polarizers in front with cross polarized axis and the viewer is required to wear corresponding linear polarizing glasses. You also see the Kinect in the front and in this image we didn't use any polarizer in front of the camera which captured this image so you see the double image of the Ironman. Calibration is very simple we just project checkerboards in both projectors and we compute the homographies between projector coordinates and display coordinates and to the rectification according to that. Another something I want to highlight is that all components are off-shelf so you really can take our paper look up the references by the components and assemble the display at home and you should really try it out because it's a very immersive feeling that you get. So the visitors of our lab liked it a lot. We think mostly it comes from the transparency as I mentioned you get like this reference points from the reality from the surrounding and also because it's a good accommodation queues and then overlaid with this the virtual reality which has still the ghosting effect which will probably be overcome by Quinn's inventions he just presented before. But also the light white glasses and the motion parallax add like the immersive to the immersive feeling and people walked around and could see Ironman from all sides. Supported is also the true size of Ironman he really is life size and you really get the feeling as if he's standing behind the glass panel. The problem however if you move faster then you get this overshoot and you lose this immersion and it looks like a rendering again. So as a conclusion we are approaching we are not completely there yet but with the next generation of head tracking which have higher frame rates and also low latency we can apply the same technique and we will get a good motion parallax and good immersion into this autostereoscopic content. With this I want to conclude I want to thank you for your attention and maybe there's time for some questions.
also they block important content from the value to our environment since we can augment it with virtual information. The idea of the transparent doors. Samsung has this light box, information on front. Both are based on LCD virtual information with reality. Now for the it will work. A little bit more complex is the screens, the one we used is holography based so passive and active glasses to be used together we use the Kinect head tracker to get an estimate Kinect and other head tracking systems is the frame rate as well as the latency. Why is that? some insights from control engineering as we position of the head so it's a motion like from these two errors. So the system that we propose consists of an anisotropic glass which you can see in front with the rendered Ironman on top of coordinates and display coordinates and to the like the immersive to the immersive feeling and technique and we will get a good motion parallax