Two approaches in the development of goggles-free 3D display systems
Markov, Vladimir
Kupiec, Stephen
Travis, Adrian
Saini, Girdail
A variety of applications, such as entertainment, medical, design and engineering, can significantly benefit from the development of a goggles-free 3D imaging system. Although a number of such systems (spatially multiplexed, volumetric, autostereoscopic and electro-holography) have been suggested, practically examined and modeled, their practical implementation is far from complete, mostly due to unsatisfactory imaging quality. Recent technological advances, both with hardware and data processing software, open a new perspective in the development of advanced non-goggles based 3D displays. Approaches of special interest include temporal multiplexing and point-aspects, as they offer a goggles-free solution to 3D imaging. In this paper, we discuss the initial results in the development of 3D displays with an improved image quality and increased refresh rate based on the new concepts, namely, temporally multiplexing and point-aspect. The paper presents a thorough review of the current state-of-the-art of these two techniques and a perspective in their further practical engineering and realization.
 Looks like I'm too often on a podium. This time I will not speed up and try to make it clear what we do. It's a growing demand with all understanding and respect to holography, growing demand to real-time system, system when an operator can see immediately, or a viewer can see immediately 3D image, not necessarily in a fully volume, but at least in a restricted volume, and that's what I'm going to talk about. It sounds a bit strange for me that after almost 30 years in holography I'm shifting out. I must say that there's a recognition of reality and a problem, not everything we can do in holography, hopefully one day we will, but there is a strong demand and a lot of requirement, and everybody who saw Star Trek wants to have a holodeck, almost impossibly, but dreams still exist. I would say we're trying to bring it one step closer, and I will talk about, I will introduce to the system, it's not much theory just a depiction of what system does and what we do in that aspect, and there are a variety of methods for displaying 3D images without goggles. That's an essential part, you all are familiar with the goggle-based system, unfortunately it's not always useful and not always possible for an operator to have either goggles, or helmet display, head mounted display, it restricts the motion, and specifically in an entertaining area when you think about thousands of people having goggles, and all problems with symmetry and all that stuff, it's not really convenient, although I must exist and people like it. Gaming industry, so there are aspects. So all of these systems project and images, and they have to multiply and time to see sequence of these images, and it has traditional limitations and we're trying to overcome this. What is needed? As I said before, there are glass-based system, head mounted display, shutter, glasses, single user, auto-stereoscopic, and we have recently this discussion, what is that auto-stereoscopic? It's becoming clear that whatever is not stereoscopic, even holography belongs to auto-stereoscopic, all this stereoscopic is goggle-based or system that you mount on your head, and the next generation of these displays should avoid this. So what kind of displays we currently have? We have holographic displays. That's the most advanced. Unfortunately, the most difficult to realize in real time. Practical solution is unlikely in real time development because of the processing, just amount of the information you have to process and transfer, reduction of information, redundancy of the system immediately introduces additional problems related to the noise of the system and flickering effect and all that stuff. So I have a hard time to see it in my life, although my life might be short. Who knows? Volumetric displays, very effective, very efficient, but have deficiencies in the latency effects and problems that are difficult to solve. I saw these auto-stereoscopic systems at a demo. They're really good, but still you see these strips of the light and images, and it gives you a feeling that it's unrealistic. So we come to auto-stereoscopic, and they have been demonstrated very well. They have their own limitations, like a viewing zone, but they do produce a realistically volumetric image. It's like looking at a big-loss system, but without any goggles. That is why we kind of look at that and we went this perspective. What we currently have in the art of 3D systems, stereographic systems with a lenticular barrier, they allow you to look around and they're compact. They have limited interactions, so you can't really work with the image, which is very often necessary, and that's the main area of real-time 3D display. Toshiba produced modulated grating LCD. About 30 viewing zones seem to be large enough, however, limited interaction and a very high cost. Actuality, I just told you about their limitation, though very good, but if you think that you need to spin, once you come to the high speed, you can't really make a big system. It's a very limited-size viewing area. We came to our autostereoscopic display system, which currently produces six viewing zones. We are looking at 16 at the end of phase 2 of this development, and it should give you the possibility to look around of the object, large head box, scalable. The problem is that it's a bit volumetric, I would say, in the sense of physical dimensions. What we use, we use temporal multiplexing. It's a fairly simple technique developed at Cambridge by Adrian Travis, one of the co-authors. What it does, it has an LCD or any kind of shutter that produces a sequence of the images for left and right eye at such a speed that you just don't resolve. We can resolve something like 30 Hz above that, we don't. This one produces about 1000 Hz, so you really don't see it. It spins very fast. That has a normal advantage, because it can produce a high-resolution image with a high speed. The disadvantage of a high frame rate and corresponding bandwidth, but that's solvable with the upcoming technique. We'll talk about that a little bit later. Principle of operation. As I said before, it produces this kind of a multiplicity of the slits that scan in front of you in a viewing zone. I'll try to see if I can. So that's what has been established using LCD. What we try to do at this moment, we're trying to use a better technology, DMD with a high pixel number and a high frequency. Graphic GPU recently increased rate and rendering ability and high bandwidth computers. That's kind of a new technology, new thing that came recently. How it works, here is, I hope it will show something. What we see here is an optical train with just at this point, we look at the slit at the end, at the bottom viewing position, and that is slit for the red light. Now what you need to do, you need to place red, green and blue sequentially or better simultaneously for demo purposes that go sequentially and scan them that you will see the sequence of lateral position. That's where your head box is. That's what the system demos. As you can see, it scans from position to position and that's how it works. It's a fairly straightforward system. As a result, what you will see, you will see the sequence of these images and it produces good effect. At the moment, we are looking at a shutter based approach when LCD from Cambridge University is used for high speed shutter. LCD shutter with a high speed and high throughput, meaning that you don't have much losses in a system and what we have, we put in a phase one which is pretty short and should be very effective. We have a DMD projector that goes through the set of the optical train and a high speed shutter in this area and then through the Fresnel lens create an image. Basically, you will see the viewing zones here. This is colored differently and the real image of the object behind the viewing lens. Let me see if I can show what that sequence looks like. It's fairly simple. You have left eye position periodically and then a jump to the right eye position which you don't resolve in time. They seem to come simultaneously, however, because the eye perceives that in a flash way with a very high speed, you just don't resolve them. You see the two images coming to your eye and you can move the head to see slightly around. That's like three viewing zones. You can go to higher level. You can have more zones like three images or four images and they go sequentially, obviously. Actually, the advantage is that you may use it not for a single operator. You may have three, four, five operators depending upon number of the sleeps you have. In addition to that, what you can also do, you can put a pie-like system when you have a sequence of the monitors and every operator will have its own system. Now, I would like to admit it's not a touch-like system. You can't really go and touch it. You will see it as projected in front of your eye, so you don't need any goggles. The system will produce this kind of image in front of you, but you can't touch. It's not like a touch screen so far, which is also under development in Japan, I believe. It produces a good look around capabilities with no pseudo-scopic effect because here you have to be careful. The image is produced like in holography, a real and pseudo-scopic. So if you overlay two of them simultaneously, then the confusion comes. What is the reality when you see the image like? Shutter synchronization. That is where the core of technology is you need to synchronize the operation of the shutter with the operation of the DMD and produce these images sequentially. So it takes a high computing rate. It takes synchronization, and that's kind of the core of the system. There are several issues that the core shutter synchronization is one of the most important. And then, of course, how the system looks like. What we're currently using in a design, it's an LED projector. You may use LED of different color. I'll talk about that a little bit later. And it produces a light illumination that goes through the optical train and a Fresnel lens synthesizing image somewhere here in this area. But where it comes from, the LED illuminates through the optical train DMD, and LED is controlled synchronously with DMD and then controls the shutter system. So it's a loop that is established inside of the, in the heart of the system and produces these sequences to shut the images in an observation plane. DMD is kind of the central core of the system. Currently existing at about 10 kHz. New DMD 3000 allows you about 16 kHz, and that should be enough to produce 16 viewing zones at a high speed and a full color. Just don't forget that you produce these images sequentially. So you really need a high speed and a full synchronization. But very fortunately, the system is on the market. You can, we can get them and we can buy them. And they produce a very good contrast. Modulated light source. You need to flash this light with a high speed. And again, very fortunately, LED of higher brightness with a 5 ns front, which is a very high speed, can be, can be just obtained from the market. And the LED currently, you can get them in three different lines. Red, blue and green. I'm sorry. Producing a very good white color at the center. So the balance of the color is available. That's not a big issue. It's just a matter of illumination, obviously, and the scheme downstairs, and the right bottom corner shows you that you can get with the right sequence of the pulse duration, you can get the right color for that. And the brightness is good enough to produce 150 nts gamma, ntcs gamma system. So that's also available. We did come with optical design in the third generation, where we used just a simple optical lens that's available, not a special design. And we could produce a very good field of view. Some aberrations at the edges, but at just the first step. And, let's do that. And here is an image rendering in the display. You see the sequence of the images, which goes both in time and in perspective. I understand it's difficult to look at this image, but that's a rotation of the satellite, and you see slightly different sequences. It might be better. That's the first generation of design. The first step in a design when we just got the commercial lenses here, display and put them together, the Fresnel lens sits here, and the viewer should be here. So you will see this. It's a tabletop small compact system that produces about four to six viewing zones, depending upon the mode of the operation. And here you see the realization, real image at the top. It's getting in its own mode. The real image, then the viewing zones, and image at different location and position. So that's basically how you would see the image in a full color system. And this is a sample of these images. You see that they are slightly shifted to left and right eye, producing volumetric effect. That's basically breadboard and here performance is resolution, which is about 800 by 600, with the image size about, I would say, 10 centimeter by 10 centimeter roughly, and optimal view distance about 84 centimeters. So it has a typical for this system parameters. So VGA, video connector and optical synchronization through VESA control. And that's a perspective view. Basically what you see here is the perspective design of the system, where you have a light engine here, that illuminates DMD based special light moderator, goes through the relay lens, through the rear projection lens, to project the image in the viewing zone. That's a field lens here, and you see that this field of view strips, through which you will observe this field lens image. As I said, we have two different approaches, and that's very unfortunate for this aspect. Point display, it's a bit ironic. I should step a little bit back. In 1989, we had a big display conference, when Steve came and he reported on his electro-halography. Many of you know, electro-halography became kind of the second after rainbow for Steve, and he put a lot of efforts to promote it and to develop it. Not everything worked as he wanted, but that was a big progress. There was some disagreement, and then when in 1995, Denysjuk visited me, and at that time I was setting up the group in Bogota in Colombia, we discussed display issues, and I said that I had some problems with Steve, and I was very open with Steve on that, with Steve approaching electro-halography, and I said that I had this idea with point-like imaging system. Denysjuk jumped at me and said, no, no, no, that's my idea. So we had a little fight. We wrote a paper together, and it's a bit ironic that now I work with Air Force, and I'm not at this stage, not allowed to completely describe this system. So the system was born still when I was not in the US, but now there are some restrictions. But perspective, the main difference is that we will have a screen in this system, and the role of the screen to combine these aspectograms, or spectral points, special points, that exist in different areas, in different domains of the pseudo-image, to transfer this image in a view, and that kind of a small demo that should tell you how it works, because they will scan through this screen, producing the images, both in time and in space, and every point of this, or every strip of the screen produces slightly different images, and that's how it should come to the reality. We are currently working on a demo of this system. Hopefully the demo should be accomplished in a half a year, so we will see what will happen. I might be able to report more later. I know that Yuri was working on that with his group independently. There was a point where we couldn't exchange much of information, but at least at the early stage we had a paper that we put together. In conclusion, we did a DMD-based 3D vision system, 3D display with a full-color imaging, and the DMD has definitely advantages over any SLM or a projection system. We did an optical design, we built a tabletop system, and currently working on both phase 2 for this DMD-based projection system and aspect-based aspect and points of hologram system too. So that's basically it. If you have any questions. APPLAUSE Maybe we have a few questions. At the back. Yes, so I can disconnect. If you want to make more viewing zones, what about the brightness of the image? That's a good question. We are looking at 16, that's the maximum possible from the point of view of angle, because you are restricted with two factors. Angle of view, how far you can go, and brightness. Estimation shows that we can go to 16 reliably, 32. You see the decline, I didn't put the figures on the numbers here, everybody blames that when you come to too much of math, that's not what people like at the conferences now. But 32 is still valuable. The problem is that when you go to 32, then you are really limited by the bandwidth of the system currently, because even with 16 kHz DMD, there are still limitations in the number of viewing zones you can produce. But with LED brightness, don't forget that you have a very narrow angle of, well, not very narrow, it's enough to see, but the diffuser that you have in the system, it's a narrow band diffuser. So it's... Any further questions? Do you need to ask so many... Do you need so many slits with complete system? Isn't it possible to reproduce this... What do you have here, one slit? And a slit runs there quickly. Yeah, but about... Canary-pulling is 4, 4 and 4. You can do that, but that's your 4th viewing zone. If you would like to produce 16 viewing zones, this slit should occupy 32 positions, because you need to produce sequentially left right eye, and you should repeat it for 16 viewers. Yeah, but the viewers are always stable, and it's not necessary to have this high angle there. It depends what you would like to do, that is very true, but it depends again what you would like to do. You may have 16 viewing zones from the same position, but if you would like really to have a reality, then you need to have a possibility to look around, that means that you have to have your... There is a head box, and you have to look at that in a different position. That tells you that then the slit should move to a new position. Yeah, but if you move, can you see any... Flicker? Yeah, flicker or... You shouldn't. It's a fast system. You have very high speed. That's really good. At least for 4, I didn't see any flicker at all. I think it's important to just put it in context. This display over here, which is a different type of display, obviously a holographic display, has 640 effective light boxes. Correct. So you're talking about a real-time display, which at the moment, using current technology, could give 16 light boxes. Correct. But what do you think, perhaps, of making 640? Well, yes. Let me put... It's a very relevant and very good question, because we discussed that aspect. What is needed for industry and for a variety of industries? I'm not talking just the military, the defense system. And the defense system in particular. We know the situation now. We know how many errors have been done. You know the fact that Canadian military were just injured by accident. This is because of the limitations with the 3D vision. You need real time. With the time of operation of this kind of the system, there is no virtual possibility to produce this kind of the realism. So that is why. Now, you may look at that different, too. You may say, okay, I have just a second-generation hologram, not dot-recorded, but real one, and that has a pretty broad angle of view, a few have properly done, of course. And bidirectional, not just a single direction, not a single direction perspective. Yeah, it's a stereoscopic effect in both directions. However, that one also has a limitation of brightness. This one brings a lot of brightness, because in every sleeve you have a narrow direction. So it's kind of compromised. Whenever you need something in real time, then the bandwidth... It's a product. It's always a product of the bandwidth and the resolution. Absolutely. Thank you very much, Vladimir. We'll go to the next presentation now.
 Looks like I'm too often on a podium.  This time I will not speed up and try to make it clear what we do.  It's a growing demand with all understanding and respect to holography,  growing demand to real-time system, system when an operator can see immediately,  or a viewer can see immediately 3D image,  not necessarily in a fully volume, but at least in a restricted volume,  and that's what I'm going to talk about.  It sounds a bit strange for me that after almost 30 years in holography I'm shifting out.  I must say that there's a recognition of reality and a problem,  not everything we can do in holography, hopefully one day we will,  but there is a strong demand and a lot of requirement,  and everybody who saw Star Trek wants to have a holodeck,  almost impossibly, but dreams still exist.  I would say we're trying to bring it one step closer,  and I will talk about, I will introduce to the system,  it's not much theory just a depiction of what system does and what we do in that aspect,  and there are a variety of methods for displaying 3D images without goggles.  That's an essential part, you all are familiar with the goggle-based system,  unfortunately it's not always useful and not always possible for an operator to have either goggles,  or helmet display, head mounted display, it restricts the motion,  and specifically in an entertaining area when you think about thousands of people having goggles,  and all problems with symmetry and all that stuff, it's not really convenient,  although I must exist and people like it.  Gaming industry, so there are aspects.  So all of these systems project and images,  and they have to multiply and time to see sequence of these images,  and it has traditional limitations and we're trying to overcome this.  What is needed? As I said before, there are glass-based system,  head mounted display, shutter, glasses, single user, auto-stereoscopic,  and we have recently this discussion, what is that auto-stereoscopic?  It's becoming clear that whatever is not stereoscopic,  even holography belongs to auto-stereoscopic,  all this stereoscopic is goggle-based or system that you mount on your head,  and the next generation of these displays should avoid this.  So what kind of displays we currently have?  We have holographic displays. That's the most advanced.  Unfortunately, the most difficult to realize in real time.  Practical solution is unlikely in real time development because of the processing,  just amount of the information you have to process and transfer,  reduction of information, redundancy of the system  immediately introduces additional problems related to the noise of the system  and flickering effect and all that stuff.  So I have a hard time to see it in my life, although my life might be short.  Who knows? Volumetric displays, very effective, very efficient,  but have deficiencies in the latency effects  and problems that are difficult to solve.  I saw these auto-stereoscopic systems at a demo.  They're really good, but still you see these strips of the light and images,  and it gives you a feeling that it's unrealistic.  So we come to auto-stereoscopic, and they have been demonstrated very well.  They have their own limitations, like a viewing zone,  but they do produce a realistically volumetric image.  It's like looking at a big-loss system, but without any goggles.  That is why we kind of look at that and we went this perspective.  What we currently have in the art of 3D systems,  stereographic systems with a lenticular barrier,  they allow you to look around and they're compact.  They have limited interactions, so you can't really work with the image,  which is very often necessary, and that's the main area of real-time 3D display.  Toshiba produced modulated grating LCD.  About 30 viewing zones seem to be large enough,  however, limited interaction and a very high cost.  Actuality, I just told you about their limitation, though very good,  but if you think that you need to spin,  once you come to the high speed, you can't really make a big system.  It's a very limited-size viewing area.  We came to our autostereoscopic display system,  which currently produces six viewing zones.  We are looking at 16 at the end of phase 2 of this development,  and it should give you the possibility to look around of the object,  large head box, scalable.  The problem is that it's a bit volumetric, I would say,  in the sense of physical dimensions.  What we use, we use temporal multiplexing.  It's a fairly simple technique developed at Cambridge by Adrian Travis,  one of the co-authors.  What it does, it has an LCD or any kind of shutter  that produces a sequence of the images for left and right eye  at such a speed that you just don't resolve.  We can resolve something like 30 Hz above that, we don't.  This one produces about 1000 Hz, so you really don't see it.  It spins very fast.  That has a normal advantage,  because it can produce a high-resolution image with a high speed.  The disadvantage of a high frame rate and corresponding bandwidth,  but that's solvable with the upcoming technique.  We'll talk about that a little bit later.  Principle of operation.  As I said before, it produces this kind of a multiplicity of the slits  that scan in front of you in a viewing zone.  I'll try to see if I can.  So that's what has been established using LCD.  What we try to do at this moment,  we're trying to use a better technology,  DMD with a high pixel number and a high frequency.  Graphic GPU recently increased rate and rendering ability  and high bandwidth computers.  That's kind of a new technology,  new thing that came recently.  How it works, here is, I hope it will show something.  What we see here is an optical train with just at this point,  we look at the slit at the end, at the bottom viewing position,  and that is slit for the red light.  Now what you need to do, you need to place red, green and blue  sequentially or better simultaneously for demo purposes  that go sequentially and scan them  that you will see the sequence of lateral position.  That's where your head box is.  That's what the system demos.  As you can see, it scans from position to position  and that's how it works.  It's a fairly straightforward system.  As a result, what you will see,  you will see the sequence of these images  and it produces good effect.  At the moment, we are looking at a shutter based approach  when LCD from Cambridge University is used for high speed shutter.  LCD shutter with a high speed and high throughput,  meaning that you don't have much losses in a system  and what we have, we put in a phase one  which is pretty short and should be very effective.  We have a DMD projector that goes through the set of the optical train  and a high speed shutter in this area  and then through the Fresnel lens create an image.  Basically, you will see the viewing zones here.  This is colored differently  and the real image of the object behind the viewing lens.  Let me see if I can show what that sequence looks like.  It's fairly simple.  You have left eye position periodically  and then a jump to the right eye position  which you don't resolve in time.  They seem to come simultaneously,  however, because the eye perceives that in a flash way  with a very high speed, you just don't resolve them.  You see the two images coming to your eye  and you can move the head to see slightly around.  That's like three viewing zones.  You can go to higher level.  You can have more zones like three images or four images  and they go sequentially, obviously.  Actually, the advantage is that you may use it  not for a single operator.  You may have three, four, five operators  depending upon number of the sleeps you have.  In addition to that, what you can also do,  you can put a pie-like system  when you have a sequence of the monitors  and every operator will have its own system.  Now, I would like to admit it's not a touch-like system.  You can't really go and touch it.  You will see it as projected in front of your eye,  so you don't need any goggles.  The system will produce this kind of image in front of you,  but you can't touch.  It's not like a touch screen so far,  which is also under development in Japan, I believe.  It produces a good look around capabilities  with no pseudo-scopic effect  because here you have to be careful.  The image is produced like in holography,  a real and pseudo-scopic.  So if you overlay two of them simultaneously,  then the confusion comes.  What is the reality when you see the image like?  Shutter synchronization.  That is where the core of technology is  you need to synchronize the operation of the shutter  with the operation of the DMD  and produce these images sequentially.  So it takes a high computing rate.  It takes synchronization,  and that's kind of the core of the system.  There are several issues that the core shutter synchronization  is one of the most important.  And then, of course, how the system looks like.  What we're currently using in a design,  it's an LED projector.  You may use LED of different color.  I'll talk about that a little bit later.  And it produces a light illumination  that goes through the optical train and a Fresnel lens  synthesizing image somewhere here in this area.  But where it comes from,  the LED illuminates through the optical train DMD,  and LED is controlled synchronously with DMD  and then controls the shutter system.  So it's a loop that is established inside of the,  in the heart of the system  and produces these sequences  to shut the images in an observation plane.  DMD is kind of the central core of the system.  Currently existing at about 10 kHz.  New DMD 3000 allows you about 16 kHz,  and that should be enough to produce 16 viewing zones  at a high speed and a full color.  Just don't forget that you produce these images sequentially.  So you really need a high speed and a full synchronization.  But very fortunately, the system is on the market.  You can, we can get them and we can buy them.  And they produce a very good contrast.  Modulated light source.  You need to flash this light with a high speed.  And again, very fortunately,  LED of higher brightness with a 5 ns front,  which is a very high speed,  can be, can be just obtained from the market.  And the LED currently,  you can get them in three different lines.  Red, blue and green.  I'm sorry.  Producing a very good white color at the center.  So the balance of the color is available.  That's not a big issue.  It's just a matter of illumination, obviously,  and the scheme downstairs,  and the right bottom corner shows you  that you can get with the right sequence of the pulse duration,  you can get the right color for that.  And the brightness is good enough to produce 150 nts gamma,  ntcs gamma system.  So that's also available.  We did come with optical design in the third generation,  where we used just a simple optical lens that's available,  not a special design.  And we could produce a very good field of view.  Some aberrations at the edges, but at just the first step.  And, let's do that.  And here is an image rendering in the display.  You see the sequence of the images,  which goes both in time and in perspective.  I understand it's difficult to look at this image,  but that's a rotation of the satellite,  and you see slightly different sequences.  It might be better.  That's the first generation of design.  The first step in a design when we just got the commercial lenses here,  display and put them together, the Fresnel lens sits here,  and the viewer should be here.  So you will see this.  It's a tabletop small compact system  that produces about four to six viewing zones,  depending upon the mode of the operation.  And here you see the realization, real image at the top.  It's getting in its own mode.  The real image, then the viewing zones,  and image at different location and position.  So that's basically how you would see the image in a full color system.  And this is a sample of these images.  You see that they are slightly shifted to left and right eye,  producing volumetric effect.  That's basically breadboard and here performance is resolution,  which is about 800 by 600,  with the image size about, I would say,  10 centimeter by 10 centimeter roughly,  and optimal view distance about 84 centimeters.  So it has a typical for this system parameters.  So VGA, video connector and optical synchronization through VESA control.  And that's a perspective view.  Basically what you see here is the perspective design of the system,  where you have a light engine here,  that illuminates DMD based special light moderator,  goes through the relay lens,  through the rear projection lens,  to project the image in the viewing zone.  That's a field lens here,  and you see that this field of view strips,  through which you will observe this field lens image.  As I said, we have two different approaches,  and that's very unfortunate for this aspect.  Point display, it's a bit ironic.  I should step a little bit back.  In 1989, we had a big display conference,  when Steve came and he reported on his electro-halography.  Many of you know, electro-halography became kind of the second after rainbow for Steve,  and he put a lot of efforts to promote it and to develop it.  Not everything worked as he wanted, but that was a big progress.  There was some disagreement,  and then when in 1995, Denysjuk visited me,  and at that time I was setting up the group in Bogota in Colombia,  we discussed display issues,  and I said that I had some problems with Steve,  and I was very open with Steve on that,  with Steve approaching electro-halography,  and I said that I had this idea with point-like imaging system.  Denysjuk jumped at me and said,  no, no, no, that's my idea.  So we had a little fight.  We wrote a paper together,  and it's a bit ironic that now I work with Air Force,  and I'm not at this stage,  not allowed to completely describe this system.  So the system was born still when I was not in the US,  but now there are some restrictions.  But perspective, the main difference is that we will have a screen in this system,  and the role of the screen to combine these aspectograms,  or spectral points, special points, that exist in different areas,  in different domains of the pseudo-image,  to transfer this image in a view,  and that kind of a small demo that should tell you how it works,  because they will scan through this screen,  producing the images,  both in time and in space,  and every point of this,  or every strip of the screen produces slightly different images,  and that's how it should come to the reality.  We are currently working on a demo of this system.  Hopefully the demo should be accomplished in a half a year,  so we will see what will happen.  I might be able to report more later.  I know that Yuri was working on that with his group independently.  There was a point where we couldn't exchange much of information,  but at least at the early stage we had a paper that we put together.  In conclusion, we did a DMD-based 3D vision system,  3D display with a full-color imaging,  and the DMD has definitely advantages over any SLM or a projection system.  We did an optical design, we built a tabletop system,  and currently working on both phase 2 for this DMD-based projection system  and aspect-based aspect and points of hologram system too.  So that's basically it.  If you have any questions.  APPLAUSE  Maybe we have a few questions.  At the back.  Yes, so I can disconnect.  If you want to make more viewing zones,  what about the brightness of the image?  That's a good question.  We are looking at 16, that's the maximum possible from the point of view of angle,  because you are restricted with two factors.  Angle of view, how far you can go, and brightness.  Estimation shows that we can go to 16 reliably, 32.  You see the decline, I didn't put the figures on the numbers here,  everybody blames that when you come to too much of math,  that's not what people like at the conferences now.  But 32 is still valuable.  The problem is that when you go to 32,  then you are really limited by the bandwidth of the system currently,  because even with 16 kHz DMD,  there are still limitations in the number of viewing zones you can produce.  But with LED brightness,  don't forget that you have a very narrow angle of,  well, not very narrow, it's enough to see,  but the diffuser that you have in the system, it's a narrow band diffuser.  So it's...  Any further questions?  Do you need to ask so many...  Do you need so many slits with complete system?  Isn't it possible to reproduce this...  What do you have here, one slit?  And a slit runs there quickly.  Yeah, but about...  Canary-pulling is 4, 4 and 4.  You can do that, but that's your 4th viewing zone.  If you would like to produce 16 viewing zones,  this slit should occupy 32 positions,  because you need to produce sequentially left right eye,  and you should repeat it for 16 viewers.  Yeah, but the viewers are always stable,  and it's not necessary to have this high angle there.  It depends what you would like to do, that is very true,  but it depends again what you would like to do.  You may have 16 viewing zones from the same position,  but if you would like really to have a reality,  then you need to have a possibility to look around,  that means that you have to have your...  There is a head box, and you have to look at that  in a different position.  That tells you that then the slit should move to a new position.  Yeah, but if you move,  can you see any...  Flicker?  Yeah, flicker or...  You shouldn't.  It's a fast system. You have very high speed.  That's really good.  At least for 4, I didn't see any flicker at all.  I think it's important to just put it in context.  This display over here,  which is a different type of display,  obviously a holographic display,  has 640 effective light boxes.  Correct.  So you're talking about a real-time display,  which at the moment, using current technology,  could give 16 light boxes.  Correct.  But what do you think, perhaps, of making 640?  Well, yes.  Let me put...  It's a very relevant and very good question,  because we discussed that aspect.  What is needed for industry and for a variety of industries?  I'm not talking just the military, the defense system.  And the defense system in particular.  We know the situation now.  We know how many errors have been done.  You know the fact that Canadian military  were just injured by accident.  This is because of the limitations with the 3D vision.  You need real time.  With the time of operation of this kind of the system,  there is no virtual possibility to produce this kind of the realism.  So that is why.  Now, you may look at that different, too.  You may say, okay, I have just a second-generation hologram,  not dot-recorded, but real one,  and that has a pretty broad angle of view,  a few have properly done, of course.  And bidirectional, not just a single direction,  not a single direction perspective.  Yeah, it's a stereoscopic effect in both directions.  However, that one also has a limitation of brightness.  This one brings a lot of brightness,  because in every sleeve you have a narrow direction.  So it's kind of compromised.  Whenever you need something in real time,  then the bandwidth...  It's a product.  It's always a product of the bandwidth and the resolution.  Absolutely.  Thank you very much, Vladimir.  We'll go to the next presentation now.
Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__0_575.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__576_719.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__720_839.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__840_887.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__888_947.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__948_1643.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__1644_1871.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__1872_2039.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__2040_3431.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__3432_4307.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__4308_6335.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__6336_8243.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__8244_9503.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__9504_10403.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__10404_11711.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__11712_12803.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__12804_13607.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__13608_15311.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__15312_16055.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__16056_17099.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__17100_17975.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__17976_18467.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__18468_18623.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__18624_19487.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__19488_19871.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__19872_20411.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__20412_20903.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__20904_21047.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__21048_21515.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__21516_21755.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__21756_22619.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__22620_23339.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__23340_27323.png Two_approaches_in_the_development_of_goggles-free_3D_display_systems__480p_6fps_H264-128kbit_AAC__27324_35958.png
Looks like I'm too often on a podium. This time I will not speed up and try to make it clear what we do. It's a growing demand with all understanding and respect to holography, growing demand to real-time system, system when an operator can see immediately, or a viewer can see immediately 3D image, not necessarily in a fully volume, but at least in a restricted volume, and that's what I'm going to talk about. It sounds a bit strange for me that after almost 30 years in holography I'm shifting out. I must say that there's a recognition of reality and a problem, not everything we can do in holography, hopefully one day we will, but there is a strong demand and a lot of requirement, and everybody who saw Star Trek wants to have a holodeck, almost impossibly, but dreams still exist. I would say we're trying to bring it one step closer, and I will talk about, I will introduce to the system, it's not much theory just a depiction of what system does and what we do in that aspect, and there are a variety of methods for displaying 3D images without goggles. That's an essential part, you all are familiar with the goggle-based system, unfortunately it's not always useful and not always possible for an operator to have either goggles, or helmet display, head mounted display, it restricts the motion, and specifically in an entertaining area when you think about thousands of people having goggles, and all problems with symmetry and all that stuff, it's not really convenient, although I must exist and people like it. Gaming industry, so there are aspects. So all of these systems project and images, and they have to multiply and time to see sequence of these images, and it has traditional limitations and we're trying to overcome this. What is needed? As I said before, there are glass-based system, head mounted display, shutter, glasses, single user, auto-stereoscopic, and we have recently this discussion, what is that auto-stereoscopic? It's becoming clear that whatever is not stereoscopic, even holography belongs to auto-stereoscopic, all this stereoscopic is goggle-based or system that you mount on your head, and the next generation of these displays should avoid this. So what kind of displays we currently have? We have holographic displays. That's the most advanced. Unfortunately, the most difficult to realize in real time. Practical solution is unlikely in real time development because of the processing, just amount of the information you have to process and transfer, reduction of information, redundancy of the system immediately introduces additional problems related to the noise of the system and flickering effect and all that stuff. So I have a hard time to see it in my life, although my life might be short. Who knows? Volumetric displays, very effective, very efficient, but have deficiencies in the latency effects and problems that are difficult to solve. I saw these auto-stereoscopic systems at a demo. They're really good, but still you see these strips of the light and images, and it gives you a feeling that it's unrealistic. So we come to auto-stereoscopic, and they have been demonstrated very well. They have their own limitations, like a viewing zone, but they do produce a realistically volumetric image. It's like looking at a big-loss system, but without any goggles. That is why we kind of look at that and we went this perspective. What we currently have in the art of 3D systems, stereographic systems with a lenticular barrier, they allow you to look around and they're compact. They have limited interactions, so you can't really work with the image, which is very often necessary, and that's the main area of real-time 3D display. Toshiba produced modulated grating LCD. About 30 viewing zones seem to be large enough, however, limited interaction and a very high cost. Actuality, I just told you about their limitation, though very good, but if you think that you need to spin, once you come to the high speed, you can't really make a big system. It's a very limited-size viewing area. We came to our autostereoscopic display system, which currently produces six viewing zones. We are looking at 16 at the end of phase 2 of this development, and it should give you the possibility to look around of the object, large head box, scalable. The problem is that it's a bit volumetric, I would say, in the sense of physical dimensions. What we use, we use temporal multiplexing. It's a fairly simple technique developed at Cambridge by Adrian Travis, one of the co-authors. What it does, it has an LCD or any kind of shutter that produces a sequence of the images for left and right eye at such a speed that you just don't resolve. We can resolve something like 30 Hz above that, we don't. This one produces about 1000 Hz, so you really don't see it. It spins very fast. That has a normal advantage, because it can produce a high-resolution image with a high speed. The disadvantage of a high frame rate and corresponding bandwidth, but that's solvable with the upcoming technique. We'll talk about that a little bit later. Principle of operation. As I said before, it produces this kind of a multiplicity of the slits that scan in front of you in a viewing zone. I'll try to see if I can. So that's what has been established using LCD. What we try to do at this moment, we're trying to use a better technology, DMD with a high pixel number and a high frequency. Graphic GPU recently increased rate and rendering ability and high bandwidth computers. That's kind of a new technology, new thing that came recently. How it works, here is, I hope it will show something. What we see here is an optical train with just at this point, we look at the slit at the end, at the bottom viewing position, and that is slit for the red light. Now what you need to do, you need to place red, green and blue sequentially or better simultaneously for demo purposes that go sequentially and scan them that you will see the sequence of lateral position. That's where your head box is. That's what the system demos. As you can see, it scans from position to position and that's how it works. It's a fairly straightforward system. As a result, what you will see, you will see the sequence of these images and it produces good effect. At the moment, we are looking at a shutter based approach when LCD from Cambridge University is used for high speed shutter. LCD shutter with a high speed and high throughput, meaning that you don't have much losses in a system and what we have, we put in a phase one which is pretty short and should be very effective. We have a DMD projector that goes through the set of the optical train and a high speed shutter in this area and then through the Fresnel lens create an image. Basically, you will see the viewing zones here. This is colored differently and the real image of the object behind the viewing lens. Let me see if I can show what that sequence looks like. It's fairly simple. You have left eye position periodically and then a jump to the right eye position which you don't resolve in time. They seem to come simultaneously, however, because the eye perceives that in a flash way with a very high speed, you just don't resolve them. You see the two images coming to your eye and you can move the head to see slightly around. That's like three viewing zones. You can go to higher level. You can have more zones like three images or four images and they go sequentially, obviously. Actually, the advantage is that you may use it not for a single operator. You may have three, four, five operators depending upon number of the sleeps you have. In addition to that, what you can also do, you can put a pie-like system when you have a sequence of the monitors and every operator will have its own system. Now, I would like to admit it's not a touch-like system. You can't really go and touch it. You will see it as projected in front of your eye, so you don't need any goggles. The system will produce this kind of image in front of you, but you can't touch. It's not like a touch screen so far, which is also under development in Japan, I believe. It produces a good look around capabilities with no pseudo-scopic effect because here you have to be careful. The image is produced like in holography, a real and pseudo-scopic. So if you overlay two of them simultaneously, then the confusion comes. What is the reality when you see the image like? Shutter synchronization. That is where the core of technology is you need to synchronize the operation of the shutter with the operation of the DMD and produce these images sequentially. So it takes a high computing rate. It takes synchronization, and that's kind of the core of the system. There are several issues that the core shutter synchronization is one of the most important. And then, of course, how the system looks like. What we're currently using in a design, it's an LED projector. You may use LED of different color. I'll talk about that a little bit later. And it produces a light illumination that goes through the optical train and a Fresnel lens synthesizing image somewhere here in this area. But where it comes from, the LED illuminates through the optical train DMD, and LED is controlled synchronously with DMD and then controls the shutter system. So it's a loop that is established inside of the, in the heart of the system and produces these sequences to shut the images in an observation plane. DMD is kind of the central core of the system. Currently existing at about 10 kHz. New DMD 3000 allows you about 16 kHz, and that should be enough to produce 16 viewing zones at a high speed and a full color. Just don't forget that you produce these images sequentially. So you really need a high speed and a full synchronization. But very fortunately, the system is on the market. You can, we can get them and we can buy them. And they produce a very good contrast. Modulated light source. You need to flash this light with a high speed. And again, very fortunately, LED of higher brightness with a 5 ns front, which is a very high speed, can be, can be just obtained from the market. And the LED currently, you can get them in three different lines. Red, blue and green. I'm sorry. Producing a very good white color at the center. So the balance of the color is available. That's not a big issue. It's just a matter of illumination, obviously, and the scheme downstairs, and the right bottom corner shows you that you can get with the right sequence of the pulse duration, you can get the right color for that. And the brightness is good enough to produce 150 nts gamma, ntcs gamma system. So that's also available. We did come with optical design in the third generation, where we used just a simple optical lens that's available, not a special design. And we could produce a very good field of view. Some aberrations at the edges, but at just the first step. And, let's do that. And here is an image rendering in the display. You see the sequence of the images, which goes both in time and in perspective. I understand it's difficult to look at this image, but that's a rotation of the satellite, and you see slightly different sequences. It might be better. That's the first generation of design. The first step in a design when we just got the commercial lenses here, display and put them together, the Fresnel lens sits here, and the viewer should be here. So you will see this. It's a tabletop small compact system that produces about four to six viewing zones, depending upon the mode of the operation. And here you see the realization, real image at the top. It's getting in its own mode. The real image, then the viewing zones, and image at different location and position. So that's basically how you would see the image in a full color system. And this is a sample of these images. You see that they are slightly shifted to left and right eye, producing volumetric effect. That's basically breadboard and here performance is resolution, which is about 800 by 600, with the image size about, I would say, 10 centimeter by 10 centimeter roughly, and optimal view distance about 84 centimeters. So it has a typical for this system parameters. So VGA, video connector and optical synchronization through VESA control. And that's a perspective view. Basically what you see here is the perspective design of the system, where you have a light engine here, that illuminates DMD based special light moderator, goes through the relay lens, through the rear projection lens, to project the image in the viewing zone. That's a field lens here, and you see that this field of view strips, through which you will observe this field lens image. As I said, we have two different approaches, and that's very unfortunate for this aspect. Point display, it's a bit ironic. I should step a little bit back. In 1989, we had a big display conference, when Steve came and he reported on his electro-halography. Many of you know, electro-halography became kind of the second after rainbow for Steve, and he put a lot of efforts to promote it and to develop it. Not everything worked as he wanted, but that was a big progress. There was some disagreement, and then when in 1995, Denysjuk visited me, and at that time I was setting up the group in Bogota in Colombia, we discussed display issues, and I said that I had some problems with Steve, and I was very open with Steve on that, with Steve approaching electro-halography, and I said that I had this idea with point-like imaging system. Denysjuk jumped at me and said, no, no, no, that's my idea. So we had a little fight. We wrote a paper together, and it's a bit ironic that now I work with Air Force, and I'm not at this stage, not allowed to completely describe this system. So the system was born still when I was not in the US, but now there are some restrictions. But perspective, the main difference is that we will have a screen in this system, and the role of the screen to combine these aspectograms, or spectral points, special points, that exist in different areas, in different domains of the pseudo-image, to transfer this image in a view, and that kind of a small demo that should tell you how it works, because they will scan through this screen, producing the images, both in time and in space, and every point of this, or every strip of the screen produces slightly different images, and that's how it should come to the reality. We are currently working on a demo of this system. Hopefully the demo should be accomplished in a half a year, so we will see what will happen. I might be able to report more later. I know that Yuri was working on that with his group independently. There was a point where we couldn't exchange much of information, but at least at the early stage we had a paper that we put together. In conclusion, we did a DMD-based 3D vision system, 3D display with a full-color imaging, and the DMD has definitely advantages over any SLM or a projection system. We did an optical design, we built a tabletop system, and currently working on both phase 2 for this DMD-based projection system and aspect-based aspect and points of hologram system too. So that's basically it. If you have any questions. APPLAUSE Maybe we have a few questions. At the back. Yes, so I can disconnect. If you want to make more viewing zones, what about the brightness of the image? That's a good question. We are looking at 16, that's the maximum possible from the point of view of angle, because you are restricted with two factors. Angle of view, how far you can go, and brightness. Estimation shows that we can go to 16 reliably, 32. You see the decline, I didn't put the figures on the numbers here, everybody blames that when you come to too much of math, that's not what people like at the conferences now. But 32 is still valuable. The problem is that when you go to 32, then you are really limited by the bandwidth of the system currently, because even with 16 kHz DMD, there are still limitations in the number of viewing zones you can produce. But with LED brightness, don't forget that you have a very narrow angle of, well, not very narrow, it's enough to see, but the diffuser that you have in the system, it's a narrow band diffuser. So it's... Any further questions? Do you need to ask so many... Do you need so many slits with complete system? Isn't it possible to reproduce this... What do you have here, one slit? And a slit runs there quickly. Yeah, but about... Canary-pulling is 4, 4 and 4. You can do that, but that's your 4th viewing zone. If you would like to produce 16 viewing zones, this slit should occupy 32 positions, because you need to produce sequentially left right eye, and you should repeat it for 16 viewers. Yeah, but the viewers are always stable, and it's not necessary to have this high angle there. It depends what you would like to do, that is very true, but it depends again what you would like to do. You may have 16 viewing zones from the same position, but if you would like really to have a reality, then you need to have a possibility to look around, that means that you have to have your... There is a head box, and you have to look at that in a different position. That tells you that then the slit should move to a new position. Yeah, but if you move, can you see any... Flicker? Yeah, flicker or... You shouldn't. It's a fast system. You have very high speed. That's really good. At least for 4, I didn't see any flicker at all. I think it's important to just put it in context. This display over here, which is a different type of display, obviously a holographic display, has 640 effective light boxes. Correct. So you're talking about a real-time display, which at the moment, using current technology, could give 16 light boxes. Correct. But what do you think, perhaps, of making 640? Well, yes. Let me put... It's a very relevant and very good question, because we discussed that aspect. What is needed for industry and for a variety of industries? I'm not talking just the military, the defense system. And the defense system in particular. We know the situation now. We know how many errors have been done. You know the fact that Canadian military were just injured by accident. This is because of the limitations with the 3D vision. You need real time. With the time of operation of this kind of the system, there is no virtual possibility to produce this kind of the realism. So that is why. Now, you may look at that different, too. You may say, okay, I have just a second-generation hologram, not dot-recorded, but real one, and that has a pretty broad angle of view, a few have properly done, of course. And bidirectional, not just a single direction, not a single direction perspective. Yeah, it's a stereoscopic effect in both directions. However, that one also has a limitation of brightness. This one brings a lot of brightness, because in every sleeve you have a narrow direction. So it's kind of compromised. Whenever you need something in real time, then the bandwidth... It's a product. It's always a product of the bandwidth and the resolution. Absolutely. Thank you very much, Vladimir. We'll go to the next presentation now.
variety of methods for displaying 3D images of these systems project and images, and they displays. That's the most advanced. time development because of the processing, just auto-stereoscopic, and they have been produce a realistically volumetric image. It's have in the art of 3D systems, stereographic use temporal multiplexing. It's a fairly simple high-resolution image with a high speed. The increased rate and rendering ability and high bandwidth computers. That's kind of a new number of the sleeps you have. In addition to fortunately, the system is on the market. You from the market. And the LED currently, you can a special design. And we could produce a very that. And here is an image rendering in the which goes both in time and in perspective. I very open with Steve on that, with Steve had this idea with point-like imaging system.