The Art of Seeing Stereo 3D

Stereo vs. Reality 

Stereo 3D is not “real,” it is an optical illusion. We are all constantly told that stereo 3D is more engaging visual content, but having actually directed stereo VFX and helped create the whole field of Stereo conversion, I will say the verdict among audiences is mixed. I have been involved in IMAX and feature film stereo projects, and seeing the variation of opinion in online reviews; I am dumbfounded that anyone is actually watching the same movie I helped make. Some people love it, and others want their money back (though I have asked for that when a theater actually showed a 2D movie, but made everyone wear glasses anyway.) Why is it that the illusion is breaking down?

There are of course physiological factors of each person to consider. No two people truly see the same thing, as vision is part of their unique brain. There are people who cannot see 3D ( I think most of them work in the industry — that’s a joke folks), but lets also consider that built-in artifacts of the process can be blamed.

Here’s the big secret: Stereo imagery collapses because you have only two images, and you sit too close.

As humans we do not see two images from a fixed angle as we move about the world. Our angle of view, and relative distance to an object of focus is constantly adjusting as we move closer. Stereo imagery does not support this dynamic viewpoint, and it is an illusion of depth manipulated to tell a story.

Stereo vs. Multi-view
A stereo pair of images — which is how we present most 3D movies these days — reconstructs a visual volume in front of your eyes. Your brain assembles two distinct images into a series of depth queues that fill space in front of you. Where these two images align (what we call convergence) the image appears on the presentation or “screen” plane, where imagery diverges, the detail pushes back, and where it shifts past its matching detail in the other eye, imagery comes closer. Similar to cameras, our eyes have a field of view (FOV), which is simply described as an expanding cone of vision radiating through a lens from a singular focal point (its actually more complex, but I did say “simple”). The volume we are seeing is constructed within that viewing frustum, which tapers toward your eyes.

The reconstructed volume requires your eyes to be at one end of it to define its size. If you are closer to the screen plane, the volume must shrink to fill the space, or expand as you get farther away. Unfortunately, we do not normally see that way, and often move closer to have the image overtake us, or shift side-to-side to try and see around an occluding object. It fails every time. The imagery is fixed, and all our fidgeting gets us is a collapsing volume, or a skewed volume to one side or the other. We see basically the same thing no matter what we do, but with varying amounts of depth.

There is another very popular method to view 3D imagery, mostly in print form, and that is known as lenticular-screened images. Essentially a series of long, thin, vertical lenses are packed together as tightly as they can be, and multiple 3D views are sliced and aligned behind these lenses (see autostereoscopy). As an observer lines up with the lenses, two distinct views are presented to the eyes, and a stereo volume is the result.

Lenticular viewing provides a completely different stereo experience than 3D films; it more closely mimics real vision, and allows our eyes to work naturally. As we move closer to the screen plane, our eyes cross to converge on our subject, and new views are fed to each eye. It is truly fascinating to hold a series of stereo images viewed within a lenticular print, as our brain convinces us it is really in front of our eyes in 3D, and that observation is supported by the fact that we can get in for a closer look.

Unfortunately, lenticular is not perfect viewing technology, as its resolution is limited to what can be easily manufactured. The illusion of reality is fractured by images that often shift or repeat with a visible interlaced transition, but this viewing system is portable and simply implemented — not to mention a great example of multi-view 3D.

The Fixed Volume Hypothesis

So what is the basic artifact that we are looking for that negates an exciting viewing experience for an audience?  Lets try and reason this out a bit another way:

Most stereo measurement is based on the distance between human eyes. Let’s face it, we experience reality through our eyes, and judge it based on those observations — our eyes are approximately 65mm apart, and that is our measuring stick to what is “normal”. Our brain knows the trigonometry to judge distance, and it knows that when our eyes have to turn more toward each other to focus.  When it recognizes that condition,  then it knows an object is nearby and likely small (which makes objects that come off the screen too far, and still stay within the screen borders shrink as they move up the viewing angle toward us). It is a simple matter of distance and relative angle in math.  We just call it “seeing,” and forget what a true miracle it is that we can judge distance based on simple stereo imagery.

For the sake of this discussion we refer to standard stereo 3D as “fixed” because the observer cannot change anything other than their distance to the screen plane.  We have little control because someone else filmed the images, and we merely observe.

A fixed stereo state allows for some interesting visual trickery by merely changing the distance between the “eyes.”  Whenever a stereographer moves cameras in a scene farther apart (called altering the inter-axial distance) the effect to us as the observer is to miniaturize everything we see. The images have greater stereo disparity, we see around objects more, and our mind concludes that the observed object must be smaller. It provides a better understanding of relationships between objects, but since the observer did not move forward, the result is that their eyes and head must have expanded, and they are giants. This artifact of the fixed stereo state is called hyper-stereo, and is useful for several visual effects tricks — as is reducing this angle to make something seem larger.

In a multi-view stereo display like lenticular, a hyper-stereo view  is generally the result of getting closer. Truthfully there is not a direct analog of hyper-stereo in real life, unless you can change the placement of your eyes.  The only way to get that effect in real life is to miniaturize the world in front of you — and if you can do that, you will have better things to do with your time than worry about definitions.  If the object is smaller than the distance between our eyes, our eyes are outside its extent, and return a greater disparity in perspective.  Were the object larger, it would fill our FOV and we would not cross our eyes.  The small object gets a “hyper” view while the large object does not.

A fixed stereo view will not dynamically change relationships, and any change of distance to the screen plane will expand, contract, or skew the result.  An object can be easily miniaturized to our perception, and there is nothing we can do about it, other than squish it into a smaller box.  The volume exists between you and the screen plane, and the closer you get, the less interesting it becomes.

Stereo in the Theater

This theory helps  account for the differing reviews of a stereo 3D movie:  The 3D experience is elastic, and personal. Because the volume is a fixed record of a stereo space as the cameras saw it, we will not be encased in an environment, as the image is personal to our perspective, despite the screen size.  Big screens provide more brightness (they use two projectors for a brighter in-sync image), but not a bigger image because we cannot change our viewpoint.  It is fixed, and if we get closer to the screen to hopefully let it overtake us, it will not.

It is a perfectly natural reaction to move closer to see stereo 3D in more depth, but there is no reward.  Conversely, if one actually sits further back, the eyes will take in more of the image, and the end of the volume will play-out over the heads of the audience in front of us.

It all comes down to how close one sits to the screen.   However, as we move toward  large scale multi-view 3D screens, and people move closer to get better 3D, the motion picture will more likely take on the view of a stage play — A stage play populated with GIANTS.

Stereo at Home

A well-crafted 3D film is a different emotional experience than its 2D cousin.  I have witnessed the difference.  While working on the stereo conversion for the film The Immortals, the projector had a glitch, and showed us the 2D image.  We had looked at the 3D for so long that we forgot that image existed, and felt the original 2D was less interesting.  Good or bad (again based on audience reviews), the film was more engaging in stereo.

Luckily we can enjoy great stereo imagery at home on a 3D television these days.  I personally enjoy the passive 3D Vizio television (sadly no longer manufactured), which offers a brighter image than most cinema screens, and due to its size sits far enough away from the observation point so that the 3D volumes are generally very rich.  Many have said that 3D TV will die due to lack of content(this is actually happening, but not due to lack of content, but market exuberance before any content existed).  It is my feeling that it is often far more rewarding to the viewer to remember how good a 3D movie was, and truly experience that delight at home — at times in better 3D.

 

Of course multi-view 3D television will just have us all crowding around the screen looking at the tiny people inside.

 

AG