Critical alignment methods for stereoscopic production and post-production image registration
Mayhew, Christopher A.
Mayhew, Craig M.
An important aspect in the quality of the three-dimensional perception produced by any stereoscopic production is the precision of its left/right image registration. Tests show that pixel-precise stereoscopic image registration and convergence improves viewer’s three-dimensional perception by reducing viewing discomfort.1 Current software tools generally rely on techniques like 50/50 (onionskin) for camera system and left/right post-production image alignment. Recent stereoscopic software toolset beta tests indicate that these alignment techniques make pixel-accurate registration difficult to achieve. A review of 70 stereo pairs sampled from seven stereoscopic feature films, released from 2009 to 2012, revealed that the majority lacked a precise point of convergence. Extended viewing of inaccurate stereoscopic convergence leads to viewer fatigue and discomfort. 
 Okay, hi, I'm Chris Mayhew. I'm from Vision 3 Imaging Inc. This is my associate Craig Mayhew. Surprisingly enough, we're not brothers. We're not even related. We just happened to have the same name. But, anyway, our paper today is Critical Alignment Methods for Stereoscopic Production and Post-Production. And I'm going to try to move through the PowerPoint of this rather quickly so we can see some imagery at the end here. Just briefly, the touch on that in 2011, the feature film markets in the United States and Canada were $10 billion. 18% of that was generated by 3D content. There were 42,000 plus screens in the United States in 2011, of which almost a third were 3D compatible. So 3D is making progress. Now, there is a bit of a dilemma, though. If you look at this, the thing is, what is necessary to produce comfortable, high-quality stereoscopic imagery? Here are some conclusions from some previously papers that draw some conclusions that, for example, the 3D must be as good as 2D and it must be comfortable to view. Another one draws a conclusion that stereoscopic imagery, because fatigue is accumulated, shouldn't be watched for a long time. So speaking of viewing fatigue, we believe that an important aspect to the quality of any three-dimensional production is the left and right image registration. Others have defined visual fatigue as a decrease in the performance of the visual systems because of physiological strain. It is also known that stereoscopic viewing can impose an intense eye-accommodation activity, such as focusing and converging, that may result in fatigue in some viewers. And difficulty in fusing left and right images is another cause of viewing fatigue. Viewing fatigue, asethanopia is also known. Here is a list of some of the other problems. People's visual systems, whether you wear glasses, how well your eyes work, geometrical differences in the stereo pairs, how well the people have done a good job of aligning the imagery, stereo crosstalk, conflicts between convergent eye movements and accommodation, excessive parallax, too much disparity, image flicker, image blur, all contribute. Other items we've turned up is because visual fatigue is accumulated, some have reported that extended viewing a stereoscopic imagery should be avoided. So this is not very good for television, seeing as for couch potatoes in that regard. The exact causes of viewing fatigue are not fully understood, and efforts to date to address them have not been totally successful. And it seems that a common way to deal with it is to reduce the amount of stereoscopic content and to decrease the amount of time you spend watching it. One of the other things that we've discovered is that existing stereoscopic edit tools do not provide for pixel-perfect convergence. A survey that was done this last year of 953 movie viewers who happened to fill out a questionnaire of that, 60% of them reported one or more of the symptoms that we previously showed you of visual fatigue. 43% reported that those symptoms occurred right after viewing the 3D movie, and 15% reported them up to two hours later. The conclusion of the survey was that there is a high occurrence of visually induced discomfort associated with 3D movies. We did a review of our own, similar to the previous paper, we first started looking at our own production imagery, and basically focused on convergence and the consistency of our convergence and how well it tracked from scene to scene, or in this case, frame to frame. We also then basically went back and bought a bunch of 3D Blu-rays, extracted frames from those, and looked at what was occurring in feature films. In all cases, we found that there was a significant lack of convergence, accuracy, and consistency in these things, and we believe that this is a leading cause of visual fatigue. So in our own case, in order to address these issues, we developed what we call a critical alignment tool set. And essentially, the way this works is it allows you to extract stereopares from your 3D timeline, you bring them into the tool set, and you can alternate between them at between three and six cycles per second, which will produce an autosterioscopic response. We found that 4.3 hertz is the ideal frequency, and while the images are alternating, you can perform transformations on them in order to bring them into alignment. And so these transformations can be performed in a sub-pixel manner in any vector, and so now you can get your imagery to match really nicely. And then you also then can move these frames in 3D space and do keystone correction if necessary. We've also built into it what we call click tool line, which allows you to go from frame to frame and click on a common point in each frame, and it will converge the image at that point. So you can move rapidly down a timeline. You can take these values and then write them back into the main program and update your 3D timeline. You can also set keyframes and do transition for convergence control as you do your transitions, and then you can generate a text 3D edit decision list. So you can look in a hard copy as you watch the film as to what is going on. This is what the critical alignment workflow looks like. You have left-right source, goes into your main video program, then there's a 3D plugin we developed that plugs into that, which has your typical stereo tool set for horizontal vertical rotation and all that. From that, then you have another overriding critical alignment tool set that can extract frames, do critical alignment on that, write it all back to the main video editing, and render out your stereoscopic image. So summary here real quick. Our cursory review of recent stereoscopic production shows that there is a lack of convergence consistency. We believe that this contributes significantly to viewer fatigue and discomfort. We suggest that maintaining a constant, highly accurate convergence point will improve stereoscopic viewing comfort and endurance. A critical alignment tool set provides a means to accurately converge every pair of your frames, dynamically maintain convergence at objects of interest, and objects of conflict, perform comfortable stereoscopic edits and transitions, and also can be used to set up your cameras before you do the production. So that ends the PowerPoint part of this. So now let's see if we can show some of this to you. We have a couple of films. If Dan, if we can run the first three, that would be great. Now these that we're going to show you are the same thing that have different convergences. This was the original one where convergence is basically set up in an onion skin, so it's not as accurate as you would want it to be. So this one now we've set it on the closest object, which then basically comes down to his shoulder at this point here. And this one it is on her eye, which we call the object of interest. I'm sorry? I'm not watching in 3D, so I can't tell. Is it running properly? Yeah, it's got a very small disparities. Okay, so that's this. So now what we're going to do is we develop some software back in 2002 for the National Geospatial Agency that does what we call critical alignment. So we'll go back and analyze these. You can turn the movies off and we'll go through them. Okay, so let me... Okay, so what this lets me do is we've sampled this at half-second intervals. So these are the original ones, and as we go through and we step through every half-second, you can see that nothing in the imagery is actually stable, even though it was originally designed to converge on her because of the way it was done in an onion skin manner. It's not very accurate. So now let's switch to... Did you get this set up? Okay, this is closest object. Okay, so in this case, we've set it on the closest object. So you can see that Rachel here is very stable as you step through. And as Brandon comes into the frame, you see it's on his shoulder. So now where it's gone. Yeah. Okay, so now let's... It is loaded? Oh. Okay. Okay, you have to bear with me if we have to load these frames. Make the other ones on that one. Okay, so what this software was originally set up to do is to do critical alignment for frames from unmanned aerial vehicles for national security issues. But it works well for demonstrating this right here. And the idea of being as what we found in our own stereoscopic imagery was that when we were using regular onion skin type 50-50 or difference maps to try to set convergence, which is standard, they were good, but they weren't very accurate. And when we went and started looking at them in the critical alignment, we could see that we were off by, in some cases, several pixels. And so we think that when your eyes are... are focused on the screen, but because your convergence is off, you're actually somewhere off of that point. And so that requires a mental process to fuse all this and bring it together. And so we think the more that you can reduce that by having something that's constantly converged throughout the entire scene. And so you hold that really tightly that that will reduce the amount of strain on the visual system in order half to accommodate this itself and its visual thing. Good. Okay. Okay, so in this case, it is going on her eye, which we can... We can see. So you can see it's very tight, very accurate. Okay. Okay. So that's it. Okay, if we can show the other movies. I think we have a minute left here. Okay. And these we have... One is an edit and the other is a transition, a dissolve. And you can see where... what we've done. So in this case, the thing was originally converged on Richard here, and then it dissolves over to Rachel. Okay. Again, all of these are slightly off. It's hard to tell in this, but because our vision is... Because our visual systems will accommodate them so well. Okay. So in this next one now that we'll run, you'll see that... This has a dissolve in it, and this will then move from Richard, and it... convergence moves on to Sarah. And this case... or excuse me, in this one, they were... this was the original. They were fixed, so they're not adjusted while we're doing it. Or repeats twice. So now in this one, it's set on Richard's eyes, and as he gets up and moves off, it transitions and then picks it up when it cuts to her, so now it's on Sarah's eyes. Okay. Is there enough time left to do one more thing? Can we go back to the input one? Input one. Yeah. Okay, so now what we'll do is we'll look at this real quick in the dissolve. And you can see we're in the middle of the dissolve here. As we step in, you see that Richard is moving, and as Rachel dissolves in, she's moving as well. So there is no fixed convergence in these images. Okay, so... In this one, we've set it so that it's converged on Richard, and as you step through, you can see that as Rachel fades in, both are stable, and that's where the convergence is. So your convergence is on them, and because they're converged, it's the focus of the screen, so your accommodation and convergence are all in unison. So that's our critical alignment toolset and what it does. Thank you. Shut this down. Very, very nice demonstrations. I guess we have time for one very quick question. Let's try Lenny. Lenny? Yeah. If he can do a quick question. I'll try to be quick. Your hypothesis is that the break down of convergence and accommodation causes discomfort, and therefore it is important to have one principal object shot at zero parallax. But on large screens, for normal viewing distances, the break down of convergence and accommodation in a motion picture theater is not a factor with regard to discomfort. Okay, our sort of what we have found that if we can give those objects what they want to converge on, because when we did samples of the films, you could see that they intended to converge on a particular object, but because the resolution of however they were doing it wasn't very accurate, they weren't there, so they were converged somewhere else slightly off, and so that's what we're talking about, is not the position of where the convergence is. That's whatever the person wants it to be, but that what you intend to converge on, you actually converge on, and so that part is accurate to a sub-pixel level with this tool. So we found like there were a lot of talking heads where you could see where they wanted to converge on the eyes of the person talking, but they weren't there, they were off, and so we think that that contributes when your eyes are focused on the screen, but in your mind you're having to take that extra couple inches and fuse it, because you want to bring that all into some sort of order, and so that's what we're talking about. The more accurate you can make that process, the less mental energy needs to be spent on trying to make it into order. So that's what I'm trying to mean, not necessarily where the convergence point is, it's that it is where you intend it to be. Okay, brilliant. Thank you. Okay, sorry. You can have a big long talk. Okay, thank you very much. Great. Thanks.
 Okay, hi, I'm Chris Mayhew.  I'm from Vision 3 Imaging Inc.  This is my associate Craig Mayhew.  Surprisingly enough, we're not brothers.  We're not even related.  We just happened to have the same name.  But, anyway, our paper today is Critical Alignment Methods for Stereoscopic Production and Post-Production.  And I'm going to try to move through the PowerPoint of this rather quickly so we can see some imagery at the end here.  Just briefly, the touch on that in 2011, the feature film markets in the United States and Canada were $10 billion.  18% of that was generated by 3D content.  There were 42,000 plus screens in the United States in 2011, of which almost a third were 3D compatible.  So 3D is making progress.  Now, there is a bit of a dilemma, though.  If you look at this, the thing is, what is necessary to produce comfortable, high-quality stereoscopic imagery?  Here are some conclusions from some previously papers that draw some conclusions that, for example,  the 3D must be as good as 2D and it must be comfortable to view.  Another one draws a conclusion that stereoscopic imagery, because fatigue is accumulated, shouldn't be watched for a long time.  So speaking of viewing fatigue, we believe that an important aspect to the quality of any three-dimensional production is the left and right image registration.  Others have defined visual fatigue as a decrease in the performance of the visual systems because of physiological strain.  It is also known that stereoscopic viewing can impose an intense eye-accommodation activity, such as focusing and converging, that may result in fatigue in some viewers.  And difficulty in fusing left and right images is another cause of viewing fatigue.  Viewing fatigue, asethanopia is also known. Here is a list of some of the other problems.  People's visual systems, whether you wear glasses, how well your eyes work, geometrical differences in the stereo pairs,  how well the people have done a good job of aligning the imagery, stereo crosstalk, conflicts between convergent eye movements and accommodation,  excessive parallax, too much disparity, image flicker, image blur, all contribute.  Other items we've turned up is because visual fatigue is accumulated, some have reported that extended viewing a stereoscopic imagery should be avoided.  So this is not very good for television, seeing as for couch potatoes in that regard.  The exact causes of viewing fatigue are not fully understood, and efforts to date to address them have not been totally successful.  And it seems that a common way to deal with it is to reduce the amount of stereoscopic content and to decrease the amount of time you spend watching it.  One of the other things that we've discovered is that existing stereoscopic edit tools do not provide for pixel-perfect convergence.  A survey that was done this last year of 953 movie viewers who happened to fill out a questionnaire of that,  60% of them reported one or more of the symptoms that we previously showed you of visual fatigue.  43% reported that those symptoms occurred right after viewing the 3D movie, and 15% reported them up to two hours later.  The conclusion of the survey was that there is a high occurrence of visually induced discomfort associated with 3D movies.  We did a review of our own, similar to the previous paper, we first started looking at our own production imagery,  and basically focused on convergence and the consistency of our convergence and how well it tracked from scene to scene, or in this case, frame to frame.  We also then basically went back and bought a bunch of 3D Blu-rays, extracted frames from those, and looked at what was occurring in feature films.  In all cases, we found that there was a significant lack of convergence, accuracy, and consistency in these things,  and we believe that this is a leading cause of visual fatigue.  So in our own case, in order to address these issues, we developed what we call a critical alignment tool set.  And essentially, the way this works is it allows you to extract stereopares from your 3D timeline,  you bring them into the tool set, and you can alternate between them at between three and six cycles per second,  which will produce an autosterioscopic response.  We found that 4.3 hertz is the ideal frequency, and while the images are alternating,  you can perform transformations on them in order to bring them into alignment.  And so these transformations can be performed in a sub-pixel manner in any vector,  and so now you can get your imagery to match really nicely.  And then you also then can move these frames in 3D space and do keystone correction if necessary.  We've also built into it what we call click tool line, which allows you to go from frame to frame  and click on a common point in each frame, and it will converge the image at that point.  So you can move rapidly down a timeline.  You can take these values and then write them back into the main program and update your 3D timeline.  You can also set keyframes and do transition for convergence control as you do your transitions,  and then you can generate a text 3D edit decision list.  So you can look in a hard copy as you watch the film as to what is going on.  This is what the critical alignment workflow looks like.  You have left-right source, goes into your main video program,  then there's a 3D plugin we developed that plugs into that, which has your typical stereo tool set  for horizontal vertical rotation and all that.  From that, then you have another overriding critical alignment tool set that can extract frames,  do critical alignment on that, write it all back to the main video editing,  and render out your stereoscopic image.  So summary here real quick.  Our cursory review of recent stereoscopic production shows that there is a lack of convergence consistency.  We believe that this contributes significantly to viewer fatigue and discomfort.  We suggest that maintaining a constant, highly accurate convergence point  will improve stereoscopic viewing comfort and endurance.  A critical alignment tool set provides a means to accurately converge every pair of your frames,  dynamically maintain convergence at objects of interest,  and objects of conflict, perform comfortable stereoscopic edits and transitions,  and also can be used to set up your cameras before you do the production.  So that ends the PowerPoint part of this.  So now let's see if we can show some of this to you.  We have a couple of films.  If Dan, if we can run the first three, that would be great.  Now these that we're going to show you are the same thing that have different convergences.  This was the original one where convergence is basically set up in an onion skin,  so it's not as accurate as you would want it to be.  So this one now we've set it on the closest object,  which then basically comes down to his shoulder at this point here.  And this one it is on her eye, which we call the object of interest.  I'm sorry?  I'm not watching in 3D, so I can't tell.  Is it running properly?  Yeah, it's got a very small disparities.  Okay, so that's this.  So now what we're going to do is we develop some software back in 2002  for the National Geospatial Agency that does what we call critical alignment.  So we'll go back and analyze these.  You can turn the movies off and we'll go through them.  Okay, so let me...  Okay, so what this lets me do is we've sampled this at half-second intervals.  So these are the original ones, and as we go through and we step through every half-second,  you can see that nothing in the imagery is actually stable,  even though it was originally designed to converge on her  because of the way it was done in an onion skin manner.  It's not very accurate.  So now let's switch to...  Did you get this set up?  Okay, this is closest object.  Okay, so in this case, we've set it on the closest object.  So you can see that Rachel here is very stable as you step through.  And as Brandon comes into the frame, you see it's on his shoulder.  So now where it's gone.  Yeah.  Okay, so now let's...  It is loaded?  Oh.  Okay.  Okay, you have to bear with me if we have to load these frames.  Make the other ones on that one.  Okay, so what this software was originally set up to do is to do critical alignment for frames  from unmanned aerial vehicles for national security issues.  But it works well for demonstrating this right here.  And the idea of being as what we found in our own stereoscopic imagery was that  when we were using regular onion skin type 50-50 or difference maps to try to set convergence,  which is standard, they were good, but they weren't very accurate.  And when we went and started looking at them in the critical alignment,  we could see that we were off by, in some cases, several pixels.  And so we think that when your eyes are...  are focused on the screen, but because your convergence is off,  you're actually somewhere off of that point.  And so that requires a mental process to fuse all this and bring it together.  And so we think the more that you can reduce that by having something that's constantly converged  throughout the entire scene.  And so you hold that really tightly that that will reduce the amount of strain  on the visual system in order half to accommodate this itself and its visual thing.  Good. Okay.  Okay, so in this case, it is going on her eye, which we can...  We can see.  So you can see it's very tight, very accurate.  Okay.  Okay. So that's it.  Okay, if we can show the other movies.  I think we have a minute left here.  Okay. And these we have...  One is an edit and the other is a transition, a dissolve.  And you can see where... what we've done.  So in this case, the thing was originally converged on Richard here,  and then it dissolves over to Rachel.  Okay. Again, all of these are slightly off.  It's hard to tell in this, but because our vision is...  Because our visual systems will accommodate them so well.  Okay. So in this next one now that we'll run, you'll see that...  This has a dissolve in it, and this will then move from Richard,  and it... convergence moves on to Sarah.  And this case... or excuse me, in this one, they were...  this was the original. They were fixed, so they're not adjusted  while we're doing it.  Or repeats twice.  So now in this one, it's set on Richard's eyes,  and as he gets up and moves off, it transitions  and then picks it up when it cuts to her, so now it's on Sarah's eyes.  Okay.  Is there enough time left to do one more thing?  Can we go back to the input one?  Input one.  Yeah.  Okay, so now what we'll do is we'll look at this real quick in the dissolve.  And you can see we're in the middle of the dissolve here.  As we step in, you see that Richard is moving,  and as Rachel dissolves in, she's moving as well.  So there is no fixed convergence in these images.  Okay, so...  In this one, we've set it so that it's converged on Richard,  and as you step through, you can see that as Rachel fades in,  both are stable, and that's where the convergence is.  So your convergence is on them, and because they're converged,  it's the focus of the screen, so your accommodation and convergence  are all in unison.  So that's our critical alignment toolset and what it does.  Thank you.  Shut this down.  Very, very nice demonstrations.  I guess we have time for one very quick question.  Let's try Lenny.  Lenny?  Yeah.  If he can do a quick question.  I'll try to be quick.  Your hypothesis is that the break down of convergence and accommodation  causes discomfort, and therefore it is important to have one principal object  shot at zero parallax.  But on large screens, for normal viewing distances,  the break down of convergence and accommodation in a motion picture theater  is not a factor with regard to discomfort.  Okay, our sort of what we have found that if we can give those objects  what they want to converge on, because when we did samples of the films,  you could see that they intended to converge on a particular object,  but because the resolution of however they were doing it wasn't very accurate,  they weren't there, so they were converged somewhere else slightly off,  and so that's what we're talking about,  is not the position of where the convergence is.  That's whatever the person wants it to be,  but that what you intend to converge on, you actually converge on,  and so that part is accurate to a sub-pixel level with this tool.  So we found like there were a lot of talking heads where you could see  where they wanted to converge on the eyes of the person talking,  but they weren't there, they were off,  and so we think that that contributes when your eyes are focused on the screen,  but in your mind you're having to take that extra couple inches and fuse it,  because you want to bring that all into some sort of order,  and so that's what we're talking about.  The more accurate you can make that process,  the less mental energy needs to be spent on trying to make it into order.  So that's what I'm trying to mean,  not necessarily where the convergence point is,  it's that it is where you intend it to be.  Okay, brilliant. Thank you.  Okay, sorry. You can have a big long talk.  Okay, thank you very much. Great. Thanks.
_SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_0_83.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_84_155.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_156_839.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_840_1487.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_1488_2459.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_2460_3503.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_3504_4463.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_4464_8015.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_8016_10427.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_10428_11207.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_11208_12443.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_12444_14699.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_14700_15167.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_15168_15731.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_15732_15875.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_15876_15995.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_15996_16295.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_16296_16355.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_16356_16439.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_16440_16559.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_16560_16967.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_16968_17147.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_17148_17603.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_17604_20435.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_20436_20831.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_20832_21035.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_21036_21563.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_21564_25151.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_25152_26255.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_26256_31379.png _SD_A2013__Critical_alignment_methods_for_stereoscopic_production_and_post-production_image____hZuUg_31380_31445.png
Okay, hi, I'm Chris Mayhew. I'm from Vision 3 Imaging Inc. This is my associate Craig Mayhew. Surprisingly enough, we're not brothers. We're not even related. We just happened to have the same name. But, anyway, our paper today is Critical Alignment Methods for Stereoscopic Production and Post-Production. And I'm going to try to move through the PowerPoint of this rather quickly so we can see some imagery at the end here. Just briefly, the touch on that in 2011, the feature film markets in the United States and Canada were $10 billion. 18% of that was generated by 3D content. There were 42,000 plus screens in the United States in 2011, of which almost a third were 3D compatible. So 3D is making progress. Now, there is a bit of a dilemma, though. If you look at this, the thing is, what is necessary to produce comfortable, high-quality stereoscopic imagery? Here are some conclusions from some previously papers that draw some conclusions that, for example, the 3D must be as good as 2D and it must be comfortable to view. Another one draws a conclusion that stereoscopic imagery, because fatigue is accumulated, shouldn't be watched for a long time. So speaking of viewing fatigue, we believe that an important aspect to the quality of any three-dimensional production is the left and right image registration. Others have defined visual fatigue as a decrease in the performance of the visual systems because of physiological strain. It is also known that stereoscopic viewing can impose an intense eye-accommodation activity, such as focusing and converging, that may result in fatigue in some viewers. And difficulty in fusing left and right images is another cause of viewing fatigue. Viewing fatigue, asethanopia is also known. Here is a list of some of the other problems. People's visual systems, whether you wear glasses, how well your eyes work, geometrical differences in the stereo pairs, how well the people have done a good job of aligning the imagery, stereo crosstalk, conflicts between convergent eye movements and accommodation, excessive parallax, too much disparity, image flicker, image blur, all contribute. Other items we've turned up is because visual fatigue is accumulated, some have reported that extended viewing a stereoscopic imagery should be avoided. So this is not very good for television, seeing as for couch potatoes in that regard. The exact causes of viewing fatigue are not fully understood, and efforts to date to address them have not been totally successful. And it seems that a common way to deal with it is to reduce the amount of stereoscopic content and to decrease the amount of time you spend watching it. One of the other things that we've discovered is that existing stereoscopic edit tools do not provide for pixel-perfect convergence. A survey that was done this last year of 953 movie viewers who happened to fill out a questionnaire of that, 60% of them reported one or more of the symptoms that we previously showed you of visual fatigue. 43% reported that those symptoms occurred right after viewing the 3D movie, and 15% reported them up to two hours later. The conclusion of the survey was that there is a high occurrence of visually induced discomfort associated with 3D movies. We did a review of our own, similar to the previous paper, we first started looking at our own production imagery, and basically focused on convergence and the consistency of our convergence and how well it tracked from scene to scene, or in this case, frame to frame. We also then basically went back and bought a bunch of 3D Blu-rays, extracted frames from those, and looked at what was occurring in feature films. In all cases, we found that there was a significant lack of convergence, accuracy, and consistency in these things, and we believe that this is a leading cause of visual fatigue. So in our own case, in order to address these issues, we developed what we call a critical alignment tool set. And essentially, the way this works is it allows you to extract stereopares from your 3D timeline, you bring them into the tool set, and you can alternate between them at between three and six cycles per second, which will produce an autosterioscopic response. We found that 4.3 hertz is the ideal frequency, and while the images are alternating, you can perform transformations on them in order to bring them into alignment. And so these transformations can be performed in a sub-pixel manner in any vector, and so now you can get your imagery to match really nicely. And then you also then can move these frames in 3D space and do keystone correction if necessary. We've also built into it what we call click tool line, which allows you to go from frame to frame and click on a common point in each frame, and it will converge the image at that point. So you can move rapidly down a timeline. You can take these values and then write them back into the main program and update your 3D timeline. You can also set keyframes and do transition for convergence control as you do your transitions, and then you can generate a text 3D edit decision list. So you can look in a hard copy as you watch the film as to what is going on. This is what the critical alignment workflow looks like. You have left-right source, goes into your main video program, then there's a 3D plugin we developed that plugs into that, which has your typical stereo tool set for horizontal vertical rotation and all that. From that, then you have another overriding critical alignment tool set that can extract frames, do critical alignment on that, write it all back to the main video editing, and render out your stereoscopic image. So summary here real quick. Our cursory review of recent stereoscopic production shows that there is a lack of convergence consistency. We believe that this contributes significantly to viewer fatigue and discomfort. We suggest that maintaining a constant, highly accurate convergence point will improve stereoscopic viewing comfort and endurance. A critical alignment tool set provides a means to accurately converge every pair of your frames, dynamically maintain convergence at objects of interest, and objects of conflict, perform comfortable stereoscopic edits and transitions, and also can be used to set up your cameras before you do the production. So that ends the PowerPoint part of this. So now let's see if we can show some of this to you. We have a couple of films. If Dan, if we can run the first three, that would be great. Now these that we're going to show you are the same thing that have different convergences. This was the original one where convergence is basically set up in an onion skin, so it's not as accurate as you would want it to be. So this one now we've set it on the closest object, which then basically comes down to his shoulder at this point here. And this one it is on her eye, which we call the object of interest. I'm sorry? I'm not watching in 3D, so I can't tell. Is it running properly? Yeah, it's got a very small disparities. Okay, so that's this. So now what we're going to do is we develop some software back in 2002 for the National Geospatial Agency that does what we call critical alignment. So we'll go back and analyze these. You can turn the movies off and we'll go through them. Okay, so let me... Okay, so what this lets me do is we've sampled this at half-second intervals. So these are the original ones, and as we go through and we step through every half-second, you can see that nothing in the imagery is actually stable, even though it was originally designed to converge on her because of the way it was done in an onion skin manner. It's not very accurate. So now let's switch to... Did you get this set up? Okay, this is closest object. Okay, so in this case, we've set it on the closest object. So you can see that Rachel here is very stable as you step through. And as Brandon comes into the frame, you see it's on his shoulder. So now where it's gone. Yeah. Okay, so now let's... It is loaded? Oh. Okay. Okay, you have to bear with me if we have to load these frames. Make the other ones on that one. Okay, so what this software was originally set up to do is to do critical alignment for frames from unmanned aerial vehicles for national security issues. But it works well for demonstrating this right here. And the idea of being as what we found in our own stereoscopic imagery was that when we were using regular onion skin type 50-50 or difference maps to try to set convergence, which is standard, they were good, but they weren't very accurate. And when we went and started looking at them in the critical alignment, we could see that we were off by, in some cases, several pixels. And so we think that when your eyes are... are focused on the screen, but because your convergence is off, you're actually somewhere off of that point. And so that requires a mental process to fuse all this and bring it together. And so we think the more that you can reduce that by having something that's constantly converged throughout the entire scene. And so you hold that really tightly that that will reduce the amount of strain on the visual system in order half to accommodate this itself and its visual thing. Good. Okay. Okay, so in this case, it is going on her eye, which we can... We can see. So you can see it's very tight, very accurate. Okay. Okay. So that's it. Okay, if we can show the other movies. I think we have a minute left here. Okay. And these we have... One is an edit and the other is a transition, a dissolve. And you can see where... what we've done. So in this case, the thing was originally converged on Richard here, and then it dissolves over to Rachel. Okay. Again, all of these are slightly off. It's hard to tell in this, but because our vision is... Because our visual systems will accommodate them so well. Okay. So in this next one now that we'll run, you'll see that... This has a dissolve in it, and this will then move from Richard, and it... convergence moves on to Sarah. And this case... or excuse me, in this one, they were... this was the original. They were fixed, so they're not adjusted while we're doing it. Or repeats twice. So now in this one, it's set on Richard's eyes, and as he gets up and moves off, it transitions and then picks it up when it cuts to her, so now it's on Sarah's eyes. Okay. Is there enough time left to do one more thing? Can we go back to the input one? Input one. Yeah. Okay, so now what we'll do is we'll look at this real quick in the dissolve. And you can see we're in the middle of the dissolve here. As we step in, you see that Richard is moving, and as Rachel dissolves in, she's moving as well. So there is no fixed convergence in these images. Okay, so... In this one, we've set it so that it's converged on Richard, and as you step through, you can see that as Rachel fades in, both are stable, and that's where the convergence is. So your convergence is on them, and because they're converged, it's the focus of the screen, so your accommodation and convergence are all in unison. So that's our critical alignment toolset and what it does. Thank you. Shut this down. Very, very nice demonstrations. I guess we have time for one very quick question. Let's try Lenny. Lenny? Yeah. If he can do a quick question. I'll try to be quick. Your hypothesis is that the break down of convergence and accommodation causes discomfort, and therefore it is important to have one principal object shot at zero parallax. But on large screens, for normal viewing distances, the break down of convergence and accommodation in a motion picture theater is not a factor with regard to discomfort. Okay, our sort of what we have found that if we can give those objects what they want to converge on, because when we did samples of the films, you could see that they intended to converge on a particular object, but because the resolution of however they were doing it wasn't very accurate, they weren't there, so they were converged somewhere else slightly off, and so that's what we're talking about, is not the position of where the convergence is. That's whatever the person wants it to be, but that what you intend to converge on, you actually converge on, and so that part is accurate to a sub-pixel level with this tool. So we found like there were a lot of talking heads where you could see where they wanted to converge on the eyes of the person talking, but they weren't there, they were off, and so we think that that contributes when your eyes are focused on the screen, but in your mind you're having to take that extra couple inches and fuse it, because you want to bring that all into some sort of order, and so that's what we're talking about. The more accurate you can make that process, the less mental energy needs to be spent on trying to make it into order. So that's what I'm trying to mean, not necessarily where the convergence point is, it's that it is where you intend it to be. Okay, brilliant. Thank you. Okay, sorry. You can have a big long talk. Okay, thank you very much. Great. Thanks.
that an important aspect to the quality of any three-dimensional production is the left and right image registration. Others have defined the stereo pairs, how well the people have done a pixel-perfect convergence. A survey that was done convergence and how well it tracked from scene to at what was occurring in feature films. In all workflow looks like. You have left-right source, Our cursory review of recent stereoscopic contributes significantly to viewer fatigue and alignment. So we'll go back and analyze these. accommodation and convergence are all in unison. of convergence and accommodation causes discomfort, and therefore it is important to have on large screens, for normal viewing distances, the convergence point is, it's that it is where