In the future of immersive entertainment, people may not only walk through floating 3-D images, but also manipulate the images in thin air. Taking a step toward this reality, researchers have built a prototype of a room-sized 3-D immaterial display, demonstrating the possibility of using the technology for a variety of entertainment purposes.
Image of a 3D teapot, displayed using two perpendicular FogScreens. Credit: Cha Lee, IEEE
Computer scientists Cha Lee, Stephen DiVerdi, and Tobias Höllerer from the Univ. of California Santa Barbara have designed their depth-fused 3-D (DFD) display, which uses up to two FogScreens and projectors, along with a user-tracking system, to achieve a 3-D effect. The results of their experiments will be published in an upcoming issue of IEEE Transactions on Visualization and Computer Graphics.
In recent years, 2-D FogScreens have gained popularity at entertainment venues. Desktop-sized and handheld DFD displays also exist, although users are restricted to a single viewpoint. In a room-size DFD display, users could stand anywhere, and wouldn’t need to wear 3-D glasses, to view 3D images.
“The biggest advantage to DFD rendering lies in the fact that observers do not have to wear encumbering glasses to view the 3-D scene,” Lee says. “Although traditional auto-stereo displays and volumetric displays also exhibit this quality, their inherent technology limits their usability. Autostereo displays allow for a limited number of views, and volumetric displays are usually enclosed within some space which users cannot or should not interact with. DFD displays allow for many views and interaction as long as a user is reliably tracked.”
DFD displays contain one or more FogScreens that project a thin sheet of fog, while light is projected onto the fog from behind. The fog scatters the light to create an image that literally floats in thin air. To achieve the 3-D effect, the same image is rendered on two overlapping screens at different depths. Users’ head positions are tracked since the 2-D images on each screen depend on the user’s viewing direction. The system computes the image alignment in real time, and users see a single, fused 3-D image where the screens overlap.
But a room-sized DFD still presents technical challenges for researchers. For instance, the fog from two FogScreens can bleed through and disrupt each other, air conditioners and open doors can cause turbulence that interferes with the image quality, and alignment and tracking errors can occur because people view the 3-D images with two separate eyes.
In their experiments, the UCSB researchers compared three different FogScreen set-ups to determine which gave the clearest images: two screens parallel to each other (2 m apart), two screens perpendicular to each other, and one screen parallel to a wall (4 m apart).
In user studies, participants confirmed that they could see the 3-D effect in all three set-ups, after the researchers optimized the configurations and room conditions. The single-screen set-up had some advantages over the others, such as having less turbulence and, from a practical perspective, being simpler and cheaper.
Image of a 3D rabbit, displayed using one FogScreen and a wall. Credit: Cha Lee, et al. IEEE
Still, the researchers noted there was room for improvement, especially in terms of alignment errors, fog turbulence, and tracking errors.
“For true realism, a 3-D object should appear exactly where it is expected regardless of what the user is actually doing,” says Lee. “Factors like occlusion, projection bleed-through, projector-camera calibration errors, and dynamic scenes make this an extremely challenging issue. Coupling this with errors introduced by even the best tracking solutions on the market, and an expert user will begin to notice the slight errors in imagery and user-to-scene registration. For our research we used the best available immaterial display, the FogScreen, but display improvements would definitely improve the fidelity of images and interaction.”
Despite the challenges, the demonstration takes a significant step toward a room-sized 3-D display that users can view from any angle, and—with improved tracking accuracy and fog control—reach in and manipulate the images.
“A room-sized 3-D immaterial display would be in many ways the ideal display,” Lee says. “A virtual tour of a museum, an all-in-one entertainment display, and a virtual office or living room are all possible applications. A future application could be something like virtual surgery (e.g. on a large 3-D heart) or a telepresence type of application in live environments (e.g. 3-D surveillance).”
More information: Lee, Cha; DiVerdi, Stephen; and Höllerer, Tobias. “Depth-Fused 3D Imagery on an Immaterial Display.” IEEE Transactions on Visualization and Computer Graphics. To be published.
