Three-dimensional electro-floating display system based on integral imaging technique Sung-Wook Mina, Joohwan Kimb and Byoungho Leeb∗ a
Digital Media Laboratory, ICU, 517-10 Dogok-Dong, Gangnam-Gu, Seoul 135-854, Korea; School of Electrical Engineering, Seoul National University, Kwanak-Gu Shinlim-Dong, Seoul 151-744, Korea
b
ABSTRACT New three-dimensional (3D) display system which combines two different display techniques is proposed. One of the techniques is integral imaging. The integral imaging display system consists of a lens array and a 2D display device, and the 3D reconstructed images are integrated from the elemental images by the lens array. The other technique is image floating, which uses a big convex lens or a concave mirror to exhibit the image of a real object to the observer. The electro-floating display system which does not use the real object needs the volumetric 3D display part because the floating display system cannot make the 3D image, but only carries the image closer to the observer. The integral imaging display system can be adopted in the electro-floating display system, because the integrated image has the characteristics of the volumetric image within the viewing angle. Moreover, many methods to enhance the viewing angle of the integral imaging display system can be used for the proposed system directly. The proposed system can be successfully applied to many 3D applications such as 3D TV. Keywords: Three-dimensional display, integral imaging, image floating, electro-floating, 3D TV
1. INTRODUCTION Three-dimensional (3D) display is the ultimate visual media, which can transfer the images with most reality to the observers, and many researchers have made constant efforts to propose and develop the 3D display system expressing a 3D image like a real object. It is well known that humans use the various depth cues to recognize 3D information. The 3D display system can be described as the display system which provides the 3D image using the physiological cues such as accommodation, convergence, motion parallax and binocular disparity1. The stereoscopic method which is one of the most popular 3D display methods adopts the binocular disparity as the depth cue, which means the difference between the images of the right eye and those of the left eye. The stereoscopic display system makes the 3D image present the different images to each eye, which can easily be realized using the special glasses such as the polarization glasses. Autostereoscopic methods are the advanced stereoscopic method, in which the special glasses are replaced by the special optical components such as the liquid crystal barrier, the holographic optical element and the lens array. In other words, the observer of the autostereoscopic display systems is freed from the uncomfortable glasses by the optical component on the display device. Although the stereoscopic display systems using special glasses are simple and have already been used commercially, the trend of the stereoscopy inclines to the autostereoscopy on account of the convenience and the immediacy. Integral imaging is one of the most promising methods among the autostereoscopic techniques because it can provide the full color image and both vertical and horizontal parallaxes without any special aids on the observer. The integral imaging display system consists of the lens array and the ordinary 2D display device. The 3D images of the integral imaging display system are integrated from the elemental images that are displayed on the display device by the lens array. The studies about the integral imaging have been focused on solving the structural problems such as the pseudoscopic problem or improving the viewing characteristics such as the viewing angle and the image depth 2-5. There are several 3D display methods except the stereoscopic, such as the volumetric and the holographic6, 7. The technical achievement of these methods, however, is not satisfied yet compared with the stereoscopic. Image floating is ∗
332
Corresponding author:
[email protected]; phone: +82-2-880-7245; fax: +82-2-873-9953; http://oeqelab.snu.ac.kr
Stereoscopic Displays and Virtual Reality Systems XII, edited by Andrew J. Woods, Mark T. Bolas, John O. Merritt, Ian E. McDowall, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 5664 © 2005 SPIE and IS&T · 0277-786X/05/$15
one of the antiquated 3D display techniques, which can easily be found in the science museum. The floating display system uses a big convex lens or a concave mirror to exhibit the image of a real object to the observer. Since the 3D image of the object can be located close at hand to the observer, the floating display system can provide the impressive feel of depth. The electro-floating display system which does not use the real object needs the volumetric 3D display part inevitably because the image floating cannot make the 3D image by itself, but only carry the image closer to the observer. In this paper, we propose a new 3D display scheme which combines two different display techniques. One is the integral imaging and the other is the image floating. The integral imaging display system can be adopted in the electrofloating display system, because the reconstructed image of the integral imaging display system has the characteristics of the volumetric image within the viewing angle. The optimum parameters of the proposed system such as the specifications of the floating lens and the lens array are estimated and calculated in consideration of the viewing quality. The many techniques to enhance the integral imaging system can be applied to the proposed system directly.
2. 3D INTEGRAL IMAGING DISPLAY SYSTEM The integral imaging, also referred to as integral photography, is one of the autostereoscopic methods8, 9. Figure 1 shows the concept of the integral imaging system. As shown in Fig. 1, the lens array composed of the elemental lenses produces the elemental images about the object in the pickup process. In the display process, the elemental images presented on the 2D display device pass through the lens array to be integrated into a 3D image. Moreover, the elemental images can be generated through the computer calculation instead of the pickup process. Unlike to other stereoscopic display systems, the integral imaging display system can offer the volumetric 3D information in the certain viewing area, which has not only the sense of solidity but also its own perspective.
Elemental images
Display Pickup device
Object Observer Lens array
Pickup
Display panel
Integrated image
Figure 1. Concept of integral imaging system.
The integral imaging display system has three different display modes; real, virtual and focused mode10. Figure 2 shows the concept of the image integration for each display mode. When the gap between the lens array and the display device is longer than the focal length of the elemental lens, the image focused by the elemental lens is located in front of the lens array. Here, the focal plane of the lens array is named the central depth plane. The integrated image is located on the plane where the rays which are started from each elemental image and passed through the focal plane of the corresponding elemental lens make the cross sections, named the integration plane. The integrated image is too misty or broken to be observed when the location of the integration plane exceeds a certain distance from the central depth plane. Accordingly, the integration plane cannot be distant from the central depth plane and the solidity of the reconstructed image is restricted within the boundary of the integration planes. As shown in Fig. 2 (a), the central depth plane and the integration planes of the real mode are located in front of lens array. When the gap between the lens array and the
SPIE-IS&T/ Vol. 5664
333
display device is shorter than the focal length, the integrated image is displayed in the virtual mode, where the central depth plane and the integration planes are located behind the lens array as shown in Fig. 2 (b). In the virtual mode, the process of the image integration is the same as the real mode except for the locations of the planes. When the gap is almost equal to the focal length, the integrated image can be displayed in front of and behind the lens array simultaneously. This display mode is named the focused mode, where the rays of each elemental image are parallel and the focused point of each elemental lens does not exist as shown in Fig. 2 (c). The central depth plane of the focused mode is located on the same position of the lens array. Central depth plane
Central depth plane
Display panel
Elemental image Lens array
Integrated image
Integrated image
Integration plane
Integration plane
(a) Real mode
Display panel
Lens array
(b) Virtual mode Central depth plane
Integrated image
Display panel
Lens array
Integration plane
(c) Focused mode Figure 2. Concept of image integration for each mode.
According to the display modes, the viewing characteristics of the integral imaging display system are different. When the other conditions of the system are similar, the viewing angle of the real mode is narrower than that of the virtual mode. However, the feel of depth of the real mode is certainly superior to that of the virtual mode because the integrated image of the real mode is closer to the observer. In the focused mode, the depth of the integrated image can be expressed more deeply than the other modes, but the resolution of the integrated image is degraded because the pixel size of the image is fixed as the size of elemental lens. The display modes of the integral imaging display system should be chosen in consideration of the purpose of the system because the selection of the display mode is alternative.
3. ELECTRO-FLOATING DISPLAY SYSTEM Image floating is a simple 3D display technique emphasizing the feel of depth with the floating lens, which transfers the 3D image to the position near to the observer. Figure 3 shows the concept of the image floating. The floating display system cannot make the 3D image from the plane image, but only carries the image closer to the observer, which means
334
SPIE-IS&T/ Vol. 5664
the 3D image cannot be generated by the floating display system using the ordinary 2D display device. Therefore, the electro-floating display system which does not use the real object should need the 3D display part which can provide the volumetric 3D images in order to represent the 3D motion picture. Floating lens (or mirror)
Observer
Object
Floating image
Figure 3. Concept of image floating
The integral imaging display system can be applied to the 3D display part of the electro-floating display system because the integrated image has own perspective like the volumetric image. Figure 4 shows the proposed scheme of the electro-floating display system adopting the integral imaging display system as the 3D display part. The floating lens shown in Fig. 4 can also be changed into a concave mirror. Integral imaging display system Floating lens (virtual mode)
Integrated image
Floating image
Figure 4. Scheme of electro-floating display system using integral imaging method.
As mentioned above, the volumetric characteristics of the integrated image are restricted within the certain viewing area. The viewing angle is one of the viewing parameters of the integral imaging display system, which is the angular region determined by some system parameters such as the focal length and the size of the elemental lens. When the observer moves out of the viewing angle, the flipped images appear. The image flipping is one of the limitations of the integral imaging and occurs because the elemental images are shown through the neighbor elemental lens, not the corresponding elemental lens. The viewing angle can approximately be given by 10:
PL , 2g
Ω = 2 arctan
(1)
where Ω is the viewing angle, PL is the lens pitch, i.e., the spacing between the elemental lenses of the lens array, and g is the gap between the lens array and the display device. Because Eq. (1) is derived under the simple assumption that the integrated image is only one point and one elemental lens is substituted for the lens array, the actual viewing angle,
SPIE-IS&T/ Vol. 5664
335
which can be obtained by complex calculation including more specifications of the system such as the size of the integrated image, is smaller than the result of Eq. (1). However, the tendency of the actual viewing angle corresponds to that of the calculation. According to Eq. (1), the viewing angle is inversely proportional to the gap between the lens array and the display panel. The viewing angle of the virtual mode is wider than that of the other modes when the other conditions are similar. Due to this reason, the virtual mode is more suitable than the other display modes for the electrofloating system.
4. EXPERIMENTS AND RESULTS The experiments about the proposed system can be performed easily with the conventional integral imaging display system and a big-size convex lens as the floating lens. Table 1 shows the specification of the experimental setup. The focal length and the size of the elemental lens are 22 mm and 10 mm respectively. The display device consists of the diffuser screen and the projection system, of which the resolution is easy to change. The pixel size of the display device is set on 0.2 mm, i.e., 5 mm-1 of the resolution. The floating lens is the groove-out type Fresnel convex lens, which has the minimum aberration for the imaging. Table 1. Specification of experimental setup.
Lens array Integral imaging system Display device Floating lens
Focal length Size of elemental lens Number of elemental lens Size of pixel Number of pixel Focal length Size
22 mm 10×10 mm2 13×13 0.2 mm2 1024×768 175 mm 300×300 mm2
Figure 5 shows the experimental results of the composed system. The images of two cherries shown in Fig. 5 (a) are represented by the integral imaging display system with the distance of two cherries of 10 mm. In this case, the central depth plane is 100 mm behind the lens array, and the viewing angle is about 30°. The floating images shown in Fig. 5 (b) are located at 350mm in front of the floating lens, where the sizes of the floating images are equal to those of the integrated images. Because the floating images are located closer to the camera than the integrated image, the sizes of the floating images seem to be bigger than those of the integrated images as shown in Fig. 5.
(a) Integrated image
(b) Floating image
Figure 5. Experimental result of electro-floating system
336
SPIE-IS&T/ Vol. 5664
When the size of the floating lens is larger than the viewing zone at the position of the floating lens, the flipped image appears. Figure 6 shows the floating image with the flipped images. The flipped images can be eliminated by the barrier array, which is designed to block the elemental images for the neighbor elemental lenses and can be used for the general integral imaging system11. In the electro-floating system, the suitable size of the floating lens and the proper mask on the floating image plane can become the solutions of the image flipping. Figure 7 shows the structural solutions of the image flipping. Figure 8 (a) shows that the flipped images are successfully removed by the mask on the floating lens, of which the size is 86 mm and agrees with the actual viewing zone on the floating lens. As shown in Fig. 8 (b), the window mask on the position of the floating image, of which size is 43 mm, can also be another answer of the image flipping. Moreover, the size control of the floating lens and the arrangement of the window mask are more convenient and convertible than setting the barrier array fitted to the variable gap.
Flipped image
Flipped image
Figure 6. Floating image with flipped images.
(3) Mask on floating image
Flipped image
Ω Floating image Integrated image (1) Barrier array
Flipped image (2) Mask on floating lens
Figure 7. Structural solutions of image flipping.
SPIE-IS&T/ Vol. 5664
337
4.3 mm
8.6 mm
(a) Mask on the floating lens
(b) Mask on the position of floating image
Figure 8. Flipped images and their exclusion.
The focal length of the floating lens has influence on both lateral and longitudinal magnification. The volumetric feel of the integrated image can be enlarged in proportion to the magnification, which is determined by the relation between the focal length of the floating lens and the position of the floating image through the lens equation. As the amount of magnification increases, however, the aberration of the floating image becomes severe to observe and the resolution of the floating image is also degraded. When the position of the floating image is twice of the focal length of the floating lens, the magnification factor is set on 1 and the aberration is minimized. Figure 9 (a) and (b) show the floating images of which the magnification factors are 1.5 and 5 relatively. As shown in Fig. 9 (b), the floating image cannot be observed correctly because of the aberration of the floating lens.
(a) Magnification factor is 1.5
(b) Magnification factor is 5
Figure 9. Floating image for different magnification factors
5. DISCUSSION AND CONCLUSION The experiments in the section 4 showed that the integral imaging display system can be successfully adopted in the electro-floating system. The electro-floating system using the integral imaging method can improve the feel of depth of the 3D reconstructed images. Moreover the limitation of the integral imaging can be removed in the proposed system. There are, however, some side-effects such as the degradation of the image resolution and the aberration of the image. More studies for the removal of these problems are necessary.
338
SPIE-IS&T/ Vol. 5664
The techniques enhancing the viewing angle, such as the methods using the time and space multiplexing, the curved lens and the embossed screen, can be directly applied to the proposed system4, 5, 12, 13. The proposed system can be successfully applied to many 3D applications such as 3D TV.
6. ACKNOWLEDGMENT This work was supported by the Ministry of Information and Communications of Korea under the Information Technology Research Center (ITRC) Support Program.
7. REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.
S. E. Palmer, Vision Science, MIT press, Cambridge/London (1999). H. E. Ives, "Optical Properties of a Lippmann Lenticulated Sheet, " J. Opt. Soc. Am., 21, 171-176 (1931). F. Okano, H. Hoshino, J. Arai, and I. Yuyama, "Real-time pickup method for a three-dimensional image based on integral photography," Appl. Opt., 36, pp. 1598-1603 (1997). B. Lee, S. Jung, S.-W. Min, and J.-H. Park, "Three-dimensional display using integral photography with dynamically variable image planes," Opt. Lett., 26, pp. 1481-1482 (2001). B. Lee, S. Jung, and J.-H. Park, "Viewing-angle-enhanced integral imaging using lens switching," Opt. Lett., 27, pp. 818-820 (2002). C.-C. Tsao and J. Chen, "Volumetric display by moving screen projection with a fast-switching spatial light modulator," Proc. SPIE, Projection Displays IV, vol. 3296, pp. 191-197 (1998). G. Saxby, Practical Holography, 2nd ed., Prentice Hall (1994). G. Lippmann, "La photographie integrale, " Comptes-Rendus 146, 446-451, Academie des Sciences (1908). T. Okoshi, "Three-Dimensional Displays, " Proceedings of the IEEE, 68, pp. 548-564 (1980). J.-H. Park, S.-W. Min, S. Jung, and B. Lee, "Analysis of viewing parameters for two display methods based on integral photography," Appl. Opt., 40, pp. 5217-5232 (2001). H. Choi, S.-W. Min, S. Jung, J.-H. Park, and B. Lee, "Multiple-viewing-zone integral imaging using a dynamic barrier array for three-dimensional displays," Opt. Express, 11, pp. 927-932 (2003). Y. Kim, J.-H. Park, H. Choi, S. Jung, S.-W, Min, and B. Lee, "Viewing-angle-enhanced integral imaging system using a curved lens array," Opt. Express, 12, pp. 421-429 (2004). S.-W. Min, J. Kim, and B. Lee, "Wide-viewing projection-type integral imaging system using an embossed screen," Opt. Lett., 29, pp. 2420-2422 (2004).
SPIE-IS&T/ Vol. 5664
339
