The parallax barrier is the simplest way to block lights using strips of the black mask. The principle of the two-view parallax barrier is illustrated in Fig. 1. Fig.
A Stereoscopic Image Rendering Method for Autostereoscopic Mobile Devices Daehee Kim, Hoonjong Kang and Chunghyun Ahn Electronics and Telecommunications Research Institute (ETRI) 161 Gajeong-dong Yuseong-gu, Daejeon, 305-350, Korea {daeheekim, khj63409, hyun}@etri.re.kr
Abstract. A novel stereoscopic image rendering method for autostereoscopic mobile devices is presented in this paper. The system to implement the proposed method consists of autostereoscopic display system and the stereoscopic rendering software engine for mobile devices. First, we mount a parallax barrier, which is made by a twisted nematic (TN) panel, on the liquid crystal display (LCD) to display stereoscopic 3-D images generated by the proposed stereoscopic rendering software engine. In addition, we present the stereoscopic rendering algorithm for 3-D graphic models. The proposed algorithm generates left-view images and right-view images from 3-D graphic models. Therefore, the proposed rendering system provides autostereoscopic views in order that users can enjoy three dimensional effects without any special glasses.
1 Introduction Recently, various contents for mobile devices have been focused to accommodate users’ various demands in mobile markets. In addition, mobile markets are interested in representing 3-D computer graphic data on mobile devices. However, since conventional 3-D rendering methods are based on the monocular view, it is difficult to provide sensation of reality to users. In order to alleviate this problem, stereoscopy can be considered because stereoscopy is a very powerful means for providing a realistic spatial impression of the presented scene [1]. Stereoscopy can be viewed with special glasses or special display devices. Since portability is very important for mobile devices, special display devices attached to mobile devices are proper. In this paper, we propose an autostereoscopic image rendering system for mobile devices. The proposed system consists of two parts: hardware configuration for the autostereoscopic display and the stereoscopic rendering software engine. In this paper, we employ a special display device which is converted from the monocular view mode to the binocular mode and vice versa. Our rendering software engine can generate both monocular-viewed images and binocular-viewed images and is tested on various types of mobile platforms such as pocket PCs, handheld PCs, and mobile phones. In the proposed system, the left-eye image is rendered only on odd lines and the right-eye image is rendered only on even lines not to render unnecessary lines of each image, so computational burdens of mobile devices are alleviated.
The rest of this paper is organized as follows. After we present the hardware configuration for rendering autostereoscopic images in Section 2, Section 3 describes the stereoscopic rendering. In Section 4, we discuss implementation results, and concluding remarks follow in Section 5.
2 Autostereoscopic Display System In this paper, we use the stereoscopic images, which consist of left and right images, and those images are interlaced to provide three dimensional effects using directionselective elements such as parallax barriers or lenticular sheets [3, 4]. Parallax barriers, form of fixed film or TN panel, have a function as a spatial de-multiplexer to separate left-eye and right-eye views from a 3D scene. Therefore, in this paper, we make the autostereoscopic display system by the parallax barrier method using the TN panel to display stereoscopic images both in 2D and in 3D modes. The autostereoscopic display system does not require to wear any device to separate lefteye and right-eye views, that is, the autostereoscopic system sends those views to the corresponding eyes. Typical emissive displays radiate lights equally in all directions. In order to create a twin-view autostereoscopic display, a half of pixels must only radiate lights in directions seen by the left eye and the rest pixels in directions seen by the right eye. The parallax barrier is the simplest way to block lights using strips of the black mask. The principle of the two-view parallax barrier is illustrated in Fig. 1. Fig. 1(a) is a stereoscopic display based on pixels and Fig 1.(b) is a stereoscopic display based on sub-pixels. The sub-pixel based stereoscopic display can reduce annoying effects from strips of the black mask. The left and right images are interlaced in columns on the display. The barrier is positioned in order that left pixels of images are blocked from the region of the right viewing windows and vice versa.
(a) Pixel-Based System
(b) Sub-Pixel-Based System
Fig. 1. Autostereoscopic Display Panel Geometry using Parallax Barrier Method
The autostereoscopic display system has the capability to switch from the 2D mode to the 3D mode electronically and vice versa [5]. Fig. 2 shows the structure of the autostereoscopic display panel. The light is filtered at the first polarizer, modulated by liquid crystal, and filtered at the second polarizer. Here, the characteristic of the first polarizer is opposite to that of the second polarizer. In the 2D mode, since the TN panel does not operate, the modulated color light passes the TN panel and the third polarizer without any changes. Therefore, the autostereoscopic display system
operates like conventional LCD displays. In the 3D mode, however, the autostereoscopic display system sends left-eye and right-eye images to the corresponding eyes because the TN panel operates as the direction-selective element.
Fig. 2. Structure of Autostereoscopic Display Panel
3 Stereoscopic Rendering for 3-D Graphic Models In general, the most important factor to let human being feel three dimensional effects is spatial differences between left and right retinas. These differences are generated from slightly different view points of left and right eyes. However, conventional 3-D graphic rendering libraries set imaginary three axes on a virtual 3-D space and render 3-D graphic models on a monoscopic display device. Therefore, it is not proper for conventional 3-D graphic engines or games to provide three dimensional effects because they render graphic models only from one view point. In this section, we propose a stereoscopic rendering algorithm for 3-D graphic model to provide better three dimensional effects to users. Fig. 3(a) shows the rendering procedure to generate autostereoscopic images on mobile devices. This procedure receives 3-D computer graphic model data as input data. These data consist of geometry information which describes the shapes of 3-D models, light conditions which represent light position and light intensity, and attribute information which describes texture information, transparency and reflection coefficients. Fig 3(b) shows a GPRS phone with the autostereoscopic display and rendering system. For stereoscopic images, we generate left and right masks to allow left-eye and right-eye images to be displayed on odd and even lines of the autostereoscopic display device, respectively. In the proposed rendering procedure, the left-eye image is rendered only on odd lines and the right-eye image is rendered only on even lines not to render unnecessary lines of each image, so computational burdens of mobile devices are alleviated. In order to generate left-eye and right-eye images, we set parameters to generate parallax, and perform transformation of 3-D models. Convergence distances and convergence angles from viewing points are set to generate left-eye and right-eye images. These parameters provide binocular disparities to users, so users feel sensation of reality. After setting parameters, such as convergence distances and
convergence angles, we rotate the entire space that 3-D models are represented on. The entire space is rotated in the counterclockwise direction for the left-eye image, while the entire image is rotated in the clockwise direction for the right-image, because left and right eyes see the right side and left side of 3-D models, respectively. After the rotation for each view, every 3-D model is rendered on the rotated space in conventional rendering method with light conditions, texture information, alpha blending information and etc. Since the computation capability of mobile devices is much inferior to that of general PCs, we employ fixed-point operations rather than floating-point operations, look-up tables to deal with trigonometrical functions, and shift operations rather than multiplication operation [2]. Finally, our rendering procedure displays the left-eye image in odd lines and the right-eye image in even lines using masks, simultaneously. 3-D Model Geometry Information Light Condition Attribute Information (Color, Transparency)
Left View
Mask Generation
Right View
Parameter Setting 3-D Model Transform
Parameter Setting 3-D Model Transform
Left-Image Generation on Masked Memory
Right-Image Generation on Masked Memory
Mask Generation
Display on Autostereoscopic Device
(a)
(b)
Fig. 3. Rendering Procedure for Autostereoscopic Rendering
4 Implementation Results In order to support various mobile service environments, we have implemented and tested our autostereoscopic rendering system on various embedded operating systems, such as Windows CE 3.x, Windows CE.Net and Nucleus. In addition, we have verified our system on the various mobile device platforms such as pocket PCs, handheld PCs and mobile phones. Fig. 4 shows implementation results from the butterfly model. Fig. 4(a) is the lefteye image on odd lines and Fig 4(b) is the right-eye image on even lines. In Fig. 4, the left-eye image represents the right side of the butterfly while the right-eye image does the left side of it. Fig. 4(c) is the final output image for the autostereoscopic display devices. As mentioned in Section 2, the full-pixel parallax barrier might generate thin black vertical lines. To address these problems, we have also employed the sub-pixel parallax barrier. In order to utilize the sub-pixel parallax barrier, the proposed rendering procedure needs post-process that exchanges green components of odd lines for green components of even lines among color components.
(a)
(b)
(c)
Fig. 4. Rendering Results for Autostereoscopic View
5 Conclusions In this paper, we have presented a stereoscopic image rendering method for autostereoscopic mobile devices. The proposed method consists of autostereoscopic display system and the stereoscopic rendering software engine for mobile devices. In order to display stereoscopic images, we mount the parallax barrier, which is made by the TN panel, on the LCD. In addition, we have presented the stereoscopic rendering algorithm for 3-D graphic models. The proposed algorithm generates left-view images and right-view images from 3-D graphic models. Therefore, the proposed rendering system provides autostereoscopic views in order that users can enjoy three dimensional effects without any special glasses. Additionally, the proposed system is applied to produce 3-D games and 3-D contents which are competitive in mobile markets.
Acknowledgment This work is supported by the Ministry of Information and Communication of Korea under title “The development of SmarTV technology.” In addition, the authors wish to thank Nexxpace Co. (http://www.nexxpace.com) for supporting equipment.
References 1. Runde, D.: How to Realize a Natural Image Reproduction using Stereoscopic Displays with Motion Parallax. IEEE Trans. CSVT, Vol. 10, No. 3, (2000) 376-386 2. Harbor, J.S.: Pocket PC Game Programming, Prima Publishing, Roseville (2001) 3. Woodgate, G., Harrold, J., Jacobs, M., Moseley, R., Ezra, D.: Flat Panel Autostereoscopic Displays - Characterization and Enhancement. Proceedings of the SPIE, Vol. 3957 (2000) 4. Susumu, I., Nobuji, T., Morito, I.: Technique of Stereoscopic Image Display. EP Pat., No. 0 354 851, (1990) 5. Ezra, D., Woodgate, G.J., Omar, B.A., Holliman, N.S., Harrold, J., Shapiro, L.S.: New Autostereoscopic Display System. Sharp Technical Journal, Vol. 62, (1995) 10-14
