A variable-collimation display system
Robert Batchko*a, Sam Robinsona, Jack Schmidtb, Benito Granielac Holochip Corporation, 4940 W. 147th Street, Hawthorne, CA USA 90250 b 738 Larkspur Court San Marcos, CA USA 92078 c Visual Systems 4.6.2.5, Naval Air Warfare Center Training Systems Division (NAWCTSD), 12201 Research Parkway, Orlando, Fl USA 32826 a
ABSTRACT Two important human depth cues are accommodation and vergence. Normally, the eyes accommodate and converge or diverge in tandem; changes in viewing distance cause the eyes to simultaneously adjust both focus and orientation. However, ambiguity between accommodation and vergence cues is a well-known limitation in many stereoscopic display technologies. This limitation also arises in state-of-the-art full-flight simulator displays. In current full-flight simulators, the out-the-window (OTW) display (i.e., the front cockpit window display) employs a fixed collimated display technology which allows the pilot and copilot to perceive the OTW training scene without angular errors or distortions; however, accommodation and vergence cues are limited to fixed ranges (e.g., ~ 20 m). While this approach works well for longrange, the ambiguity of depth cues at shorter range hinders the pilot’s ability to gauge distances in critical maneuvers such as vertical take-off and landing (VTOL). This is the first in a series of papers on a novel, variable-collimation display (VCD) technology that is being developed under NAVY SBIR Topic N121-041 funding. The proposed VCD will integrate with rotary-wing and vertical take-off and landing simulators and provide accurate accommodation and vergence cues for distances ranging from approximately 3 m outside the chin window to ~ 20 m. A display that offers dynamic accommodation and vergence could improve pilot safety and training, and impact other applications presently limited by lack of these depth cues. Keywords: Fluidic lens, adaptive lens, liquid lens, varifocal plane display, multi-focal display, three-dimensional display, accommodation, flight simulator, collimated display
1. INTRODUCTION In virtual reality applications where the fidelity of the geo-specific synthetic environment and associated training tasks are paramount, the visual system must provide visual cues similar to those experienced in real world situations. In such a highfidelity simulation system, the main components of the visual system typically include: (a) a digital representation of the synthetic environment; (b) an image generation subsystem; and (c) a visual display subsystem. These three components work together to provide the necessary visual cues for the user. One highly demanding application, in terms of the fidelity of visual cues, is flight training devices for rotary wing platforms. Rotary wing platforms require a complex set of flight regimes at both high and low altitude operations. Typically, the most demanding operations (e.g., tasks such as hovering, take-off and landing, search-and-rescue, confined-area, emergency landing, and cargo loading/unloading operations) require precise close-proximity depth cues where accurate vergence (binocular fixational eye movements [34]) and accommodation (the eye’s focusing response [34]) cues are critical. Typically, high-end visual systems for flight simulators provide monoscopic visual cues using either real-image or fixed-collimated displays. Each of these display technologies has its own advantages and disadvantages; however, present solutions cannot fully support visual cues for both high altitude (i.e., large distances) and low altitude (i.e., close proximity) operations.
*
[email protected]
phone 1 650 906-1064
www.holochip.com
Visual cues allow the pilot to identify environment features and determine rate and direction of movement, rate of closure, orientation distance from obstacles and height above the terrain [1]. Specifically, visual depth cues provide information on the layout or depth and have been divided into the following classes: monoscopic, stereoscopic, motion, and physiological. Cutting (1997) identified nine sources of information from the environment which are related to the perception of layout or depth [2]. These nine sources include: occlusion, relative size, height in the visual field, relative density, aerial perspective, binocular disparities, accommodation, convergence, and motion perspective. According to Cutting (1997), the visual cues for depth perception of objects at distant locations (> 9 m) are arranged in the following order: (1) occlusion, (2) height in the visual field, (3) relative size, and (4) aerial perspective information in the environment. In the action space (i.e. < 30 m), the order of importance for visual cues changes to: (1) occlusion; (2) height in the visual field; (3) binocular disparity; (4) motion perspective; and (5) relative size. The strength and order of the visual cues change according to the distance between the viewer and the objects in the scene. Binocular disparity, which results from differences in the retinal images of the two eyes, is used to derive depth information at close ranges. The term negative (or “far”, or “un-crossed”) horizontal retinal disparity is used for objects that are located beyond the fixation point, while positive (or “near”, or “crossed”) retinal disparity is used to refer to objects closer to the observer than the fixation point. By definition, objects at the fixation point have zero disparity and the perception of depth is stimulated by objects which are located at different distances from the fixation point. Several researchers have shown that a display that is capable of presenting objects at different image planes relative to the fixation point may be used to stimulate depth information[4][5][8][9][26]. In this case, physiological depth cues of accommodation and vergence may allow the eyes to focus at different image planes and thus perceive depth between objects located at the different image planes, leading to an enhanced perception of depth [34]. It is important to note that at large distances, depth perception is mainly derived from pictorial cues such as occlusion and perspective. Light waves traveling from large distances are represented by collimated rays of light which travel in parallel paths. In this case, light rays traveling from different objects in the distance scene do not provide perceivable difference in retinal images. Stereopsis, at these distances, is a weak visual cue as the differences in layout cannot be perceived by the differences in retinal disparity [2]. Furthermore, moving the viewpoint results in no noticeable change of viewing angle between the viewpoint and objects at large distances; in essence, parallax is lost. However, as the distance from observer to object is reduced, stereopsis, resulting from the layout of objects in the environment, becomes an important depth cue. Light waves traveling from nearby objects generate rays of light that are no longer parallel, but now converge at the observer. In this case, the converging light rays from nearby objects produce retinal disparity that is perceivable by the human visual system (HVS). This difference between images on the retina (i.e. retinal disparity) generates stereopsis and, according to Cutting (1997), a stronger visual cue at distances for less than 30 m. Presently, there has been some disagreement among researchers on the distance at which stereoscopic cues become important. What matters is that the importance of different visual cues changes based on the distance from the observer to the object. Therefore, it makes sense for a visual system to adapt the format of the information that it displays as a function of distance from the viewer to the objects in the scene. 1.1 Motivation In current full-flight training devices, the OTW display utilizes either a fixed real-image (e.g., flat panel display or projection onto a fixed surface) or a virtual-image (i.e. collimated) display technology. Collimating and wide-area collimating (WAC) displays use a large primary mirror (i.e. typically a large, stretched and curved film sheet of metalized Mylar® (Figure 1), or a molded glass mirror (Figure 2)) to reimage a standard monitor (or projector) and create a virtual image. The virtual image (i.e., fixation point) typically appears > 9 m from the eyebox (i.e., the viewing location of the user, also referred to as the “viewing volume”). At this distance, the wavefront of light reaching the user’s eyes is largely collimated. Conventional collimated display technology accurately simulates the accommodation and convergence cues for objects intended to appear far from the user; it is typically used for platforms that use multiple pilots and in high end flight training devices (e.g. FAA Level D) with multiple projectors to generate an immersive screen with a large field of view (FOV). However, this method exhibits the limitations identified previously, namely a lack of proper accommodation and distorted perspectives when the system attempts to simulate objects at distances < 9 m. While conventional collimated displays provide for more comfortable and natural viewing of scenes depicting distant objects, they do not perform well when depicting objects that are close to the user. Stereoscopic systems provide varying vergence but are widely known to cause eye fatigue and headaches in many users. This is caused by the user focusing their eyes on the image plane of the display while attempting to interpret objects that
are intended to appear either in-front of (i.e. positive retinal disparity) or behind the image plane (i.e. negative retinal disparity). By contrast, in the real world, observers accommodate their eyes dependent on the distance to the observed subject. By forcing the eyes to accommodate only at a single plane while viewing all objects throughout a 3D environment, the human visual system (HVS) is not behaving naturally. This, in turn leads to the spatial anomalies and user discomfort noted above. Just as focusing on a display that is close to a user’s eyes while viewing distant objects makes the scene seem unreal, so too does focusing at a great distance while viewing objects that are close to the viewer. A display system that adjusts itself based on the distance between the objects in the scene and the viewer may provide higher fidelity visual cues at all operational ranges.
Figure 1 - Aluminized polyester film in a CAE flight simulator (photo credit: Lufthansa Flight Training Berlin [48])
Figure 2. Photo of Glass Mountain Optics/FSI’s glass collimating mirror and back projection screen for use in a Transportable Crewview flight trainer (photo credit: Glass Mountain Optics (Flight Safety International [49])
The traditional dual-seat flight training devices use collimated displays, which allow the pilot and copilot to perceive the OTW training scene without angular errors or distortions. However, accommodation and vergence cues (i.e., the fixation point) are set at fixed ranges (e.g., > 30 m), which work fine for distant objects and high-altitude flight operations, but do not provide the correct visual cues at low altitudes. At low altitudes, ambiguity between depth cues hinders the pilot’s ability to gauge distances in critical maneuvers such as vertical take-off and landing (VTOL). Ideally a variable-collimation display (VCD) will correct the lack of cuing and enable improved initial and refresher pilot training by matching realworld depth perception without losing general spatial orientation. A VCD will include the following: a) correct presentation of obstructions and flight hazards, cultural features, and vegetation, especially forested areas; b) map-of-theearth terrain and feature rendering based on new Intelligence, Surveillance, Reconnaissance, (ISR) stereo camera imagery; c) low-altitude weather effects; d) low-altitude Night Vision Goggle (NVG) presentation; and e) chin window and cargo hatch viewing of low-altitude and landing zones with changes in depth perception that are correct with dynamic changes in own-aircraft altitude. This paper is the first in a series of papers on the development of novel 3D flight-simulator VCD technology that utilizes an adaptive-lens based architecture to provide visual cues that adapt to the distance between the viewer and objects in the scene. This technology is being developed under NAVY SBIR Topic N121-041 for integration into rotary-wing and VTOL full-flight simulators. The following sections of this paper are described as follows. Section 1.2 reviews prior work, with emphasis on varifocal and multifocal displays and asthenopia. Section 1.3 describes the important visual system parameters that such a display system must meet. Section 2 presents an overview of the architecture by presenting the main enabling components of this approach and its initial application. Section 3 provides preliminary results, which contain design criteria and analysis for
the display system that meet the previously outlined requirements. Finally, Section 4 provides conclusions and a road map for future research and the challenges associated with the development of technology. 1.2 Previous Work Three-dimensional (3D) display technology is an emerging technology and applications continue to be discovered, such as laparoscopes for medical surgery [3], flight simulation [4][5], and consumer electronic devices [6], etc. In general, 3D display technologies are typically categorized as either binocular stereoscopic display or autostereoscopic display [7]. The binocular stereoscopic type display provides a different 2D image to each human eye and a pair of eyewear is required. Binocular stereoscopic displays include head-mounted displays (HMD) [4][5][8][9][10][11][12], stereoscope [13][14][15][16][17], polarized glass, and so on. Autostereoscopic type displays directly present 3D image to the viewers without a pair of eyewear. Autostereoscopic displays include lenticular lens [6][18][19][20][21], parallax barriers [22], holography [23][24], light field [25], and volumetric display [7][26][27][28]. Among the 3D display technologies mentioned above, stereoscopic and volumetric displays are popular technologies and more companies are starting to commercialize HMD stereoscopic display in the market [29][30][31]. Volumetric displays provide 3D image rendering without any eyewear by voxels in true 3D space; however, in order to achieve flicker free image rendering, volumetric displays require large volumes of image processing [10]. Athenopia (i.e., visual fatigue) during use of stereoscopic displays is currently an active area of study. Several research groups have studied the effects of Athenopia (e.g., induced headaches) on viewers watching conventional stereoscopic displays for extended periods [11][13][14][15][16][17][32][33][34][35][36]. Shibata et al. presents the discrepancy of accommodation and convergence distances when viewers watch a conventional stereoscopic display with a fixed focal plane [13]. Two solutions to the problem of conflicting accommodation and convergence visual cues are varifocal plane and multifocal plane displays. In these systems, one (varifocal) or more (multifocal) image planes are simultaneously addressed and positioned at the correct fixation distance from the user. Shiwa et al. used a movable relay lens for accommodation compensation in the stereoscopic display system [11], while Shibata et al. used an axially translated LCD display combined with a telecentric lens system to achieve the similar result [13]. Ryana et al. report that a display with five image planes could eliminate accommodation-convergence conflict [34]. Varifocal and multifocal displays demonstrated by Love, et al. [26], Liu et al. [4][5][8][9][10], and others [6][18][19] were based on adaptive lenses (e.g., wherein adaptive lenses serve to replace the movable relay lens of Shiwa et al.). The adaptive lens of Love et al. was based on a stack of birefringent lenses and fast ferroelectric liquid crystal (LC) polarization rotators, while Liu et al. utilized a fluidic lens. Their adaptive lenses had an aperture size of 3.5 to 4.0 mm; as a result, the exit pupil of these systems was approximately the size of the pupil of the human eye, making them suitable for HMD (i.e., wherein the eyepiece of the display is brought close to the user’s eye) but not autostereoscopic displays. Additionally, many researchers employed adaptive lenses based on LC lenses [20][21][27][28][38][39]. Further, as an alternative to the transmissive approach of the adaptive lens, reflective systems utilize a deformable mirror (DM) to perform the variable-focus function. For example, McQuaide et al. used a deformable membrane mirror (DMM) for their retinal scanning display [37]. 1.3 Target Performance Parameters Our proposed variable-collimation display (VCD) will enable accurate presentation of accommodation and convergence simultaneously, while avoiding the shortcomings of other types of 3D displays, such as Athenopia. The VCD allows for the dynamic control of the fixation point (or image plane) of the image formed by a collimated display system. Applied to the chin window channel of a helicopter flight simulator, the VCD will provide the correct depth cues as the craft lands. For example, when the air craft is high in the air, the virtual image presented by the VCD will be > 9 m from the view zone; as the craft descends to the ground, the image plane will move closer to the view zone. In this fashion, the user’s eyes will accommodate and converge at the appropriate distance for the scene displayed, and will do so dynamically as the craft closes in on landing. The technical specifications for the prototype VCD are given in Table 1, below.
Table 1. System specifications for variable-collimation display prototype
Performance Parameters Distance from pilot (or co-pilot) to chin window Range of variable-collimation image distances (i.e., distance from chin window to image plane) Size of view zone (i.e., eye box) Full field of view Luminance
Values*
Units
2.4 (7.8) 0.9 to 183 (3 to 60) 200 X 200 X 200 (8 X 8 X 8) ≥ 24 (horizontal) ≥ 10 (vertical) >5
m (ft) m (ft) mm3 (in3)
2.7
LP/mm
> 576 x 240 12:1
Pixels N/A
10% uniformity in luminance white point match between all system displays
N/A
VIS: 450-750 NIR: 600-900
nm
60
Frames per sec (FPS)
< 70
ms
* Note. The values presented are requirements for the visual system; the specifications for the individual optical elements will be generally higher
2. SYSTEM ARCHITECTURE 2.1 Optical Layout The VCD optical architecture is shown in Figure 3. Similar to the work of Liu, et al [4][8][9], and also that of conventional WAC display systems [40] the VCD incorporates a folded reflective design wherein an object source is projected through a beam splitter, then off of a parabolic (or spherical) mirror, back through the beam splitter and on to the user. The exit pupil of these systems can be matched to viewing zone. Also, similar to Love et al. [26] and Liu et al. [4][8][9], the VCD includes a variable-focal length (“adaptive”) lens disposed between the object source and the user. The adaptive lens and its associated static lenses comprise the “projection lens unit”, which provides the ability to control the location of the image plane while maintaining fixed magnification of the projected image.
Viewing Volume (8”x8”x8”)
Figure 3. Layout drawing of the optical system
The systems of Love, et al. [26] and Liu et al [4][8][9], utilized optics having small (i.e., 3.5 to 4.0-mm) apertures; as a result, the exit pupil of their systems were approximately the size of the pupil of the human eye. Therefore, these systems must be worn like NVGs or HMDs, with an individual display channel directed to each eye of the user. By comparison, the VCD is designed for an exit pupil approximately 200 to 300-mm diameter; approximately 3 to 4X the user’s interpupillary distance (IPD, i.e., the distance between the center of the pupils of the two eyes. The human IPD typically ranges from 52 to 78 mm) [50][51]. This size pupil allows both eyes of the user to comfortably fit in the exit pupil of the system with additional room for movement of the user’s head. In order to realize such a large exit pupil, the apertures of the optics of the VCD must be larger than the abovementioned systems; this includes a ~1.7-m diameter primary mirror and a number of lenses having clear apertures of 100-140 mm. The primary enabling technology component of the VCD is a ~120-mm-diameter aperture fluidic adaptive lens developed by Holochip Corporation [52].
0.9 m (3 ft)
(a)
(c)
(b) 9.1 m (30 ft)
18.3 m
Figure 4. Optical system layout with distance between chin-window and image configured at: a) 3 ft; b) 30 ft; and c) 60 ft
In Figure 3, the system configuration is shown with the image plane locate approximately 0.9 m outside of the chin window. Figure 4 shows the three side-by-side configurations of the system with the image plane 0.9, 9.1 and 18.3 m from the chin window. 2.2 Evaluation of Flight Simulator Architecture and Design The VCD is designed for integration in the chin window channel of rotary-wing and VSTOL full-motion simulators, such as the CH-47 and UH-60 simulators [41]. Figure 5 shows a 3D model of a conventional flight simulator with an expanded callout view showing one channel of the chin window (real image) projection system. The OTW display is a fixed-collimation system based on an array of digital projectors that illuminate a spherical intermediate rear-projection (or “back projection”) screen. A large spherical reflector subtends the OTW full field-of-view (FOV) of the crew compartment and serves as a collimating mirror for the image on the intermediate back projection screen. As shown in the callout view in Figure 5, the conventional (real image) chin window display is a standard flat-screen 2D display located approximately 1 m outside of the chin window. A digital projector is pointed downward at an angle of approximately 45 deg to the floor. A flat mirror is located on the floor of the simulator at a height approximately equivalent
to that of the ground from the perspective of the pilot. The mirror reflects the image from the projector onto a flat backprojection screen. The back projection screen is also oriented at approximately 45 deg to the floor and subtends the fullFOV of the pilot’s view through the chin window. An identical projection system is provided for the co-pilot. Likewise, similar projection systems are provided for the cargo hatch windows (located behind the chin windows).
Figure 5. 3D model of conventional flight simulator (callout view of chin window projector)
Figure 6. Sideview wireframe drawing of chin window projection system
Figure 6 shows a wireframe side view of the chin window for a CH53 simulator. In this case, the vertical FOV is approximately 20 deg. The VCD is based on the specifications for an H60 simulator, in which the horizontal FOV is 24 deg and the vertical FOV is 10 deg.
3. OPTICAL SYSTEM DESIGN AND ANALYSIS In this section we describe our designs and models for the VCD and data resulting from computational simulations. 3.1 Optical System Layout Figure 3 is a 2D layout drawing of our completed intermediate optical design, created using Zemax™ optical ray tracing software. Figure 3 shows the system in a configuration where the image plane is located approximately 0.9 m from the chin window, i.e., its closest spacing from the pilot throughout its range of collimation. The present design enables a complete analysis of optical performance and realizable system. In our design, a conventional digital projector is imaged onto a 114-mm-diameter transmissive, diffuse back-projection screen (the “intermediate image surface”). A custom projection lens unit is comprised of one ~ 120-mm-diameter adaptive lens, four static lens elements (a detailed view of the projection lens is shown in the expanded callout in Figure 3), and the diffuse back-projection screen. The aperture of the adaptive lens is large due to the high value of the Lagrange invariant of the display. The three static doublet lens elements and one static singlet lens element now provide correction of chromatic and other aberrations. The projection lens unit receives the light from the digital projector and transmits it to a 45-deg beam splitter. The beam splitter reflects a portion of the light to the 1.7-m-diameter primary mirror. The primary mirror reflects the light back through the beam splitter, forming a final magnified image of the back-projection screen. The location of the final image plane is controlled by adjusting the focal power of the adaptive lens. We performed a number of analyses on our optical design using Zemax® software in order to validate that the optical system will comply with the visual system performance specifications given in Table 1. The following sections provide a summary of our analyses, including field curvature and distortion, spot diagrams, MTF, luminance and white point match, and multi spectral performance. Since this is a variable collimation system, the analysis was at multiple configurations across the full span of viewing distances, i.e., from near (~ 0.9 m) to far (~ 18 m) ranges. Lens Data Our design parameters are given in Table 1. The general lens data for our resulting optical design is shown in the call-out in Figure 3 and specified in Table 2 and Table 3. Data is given for the system in its two extreme configurations; i.e., 18 m (Table 2) and 0.9 m (Table 3) spacing between the image plane and chin window. The system track length (i.e., the length of the optical system as measured by the vertex separations between the "left most" and "right most" surfaces.) is approximately 4.3 m, the effective focal length ranges from -224 to -245 mm, the working F/# is very fast, (approximately 1.10), and the angular magnification is approximately -0.89.
Table 2. System prescription when configured with the image plane located 60 ft from the chin window
Table 3. System prescription when configured with the image plane located 3 ft from the chin window
3.2 Field Curvature and Distortion Figure 7, Figure 8 and Figure 9 are field curvature and distortion plots for the system in configurations with the imageplane to chin window distance set at 18.3, 3.0 and 0.9 m, respectively. The distortion ranges from approximately +4.3 to 3.2%, while the field curvature is approximately 1.2 mm or less throughout the range of collimation.
Figure 7. Field curvature and distortion plots for the system configured with the image plane located 18 m from the chin window
Figure 8. Field curvature and distortion plots for the system configured with the image plane located 3 m from the chin window
Figure 9. Field curvature and distortion plots for the system configured with the image plane located 0.9 m from the chin window
3.3 Luminance-and-White Point Matching and Relative Illumination As given in Table 1, the requirement for uniformity in luminance and white point match is 10% (i.e., the luminance and white point of our display must match that of the other displays (or “channels”) in the system to within 10%). The ability of a projection display to accurately match luminance with other display channels in a system will be limited by its relative illumination. As shown in Figure 11, the relative illumination for our display is expected to fall off by 10% at a field angle of approximately 11 deg (i.e., near the 13-deg image corner). From 11 to 13 deg, the illumination falls off more rapidly, ending with a 20% dropoff at the image corner. We plan to implement a number of flat-field correction techniques to bring the maximum field dropoff to approximately 1% or less. Such techniques include applying the appropriate illumination profile to the digital projector or its optics. This can be accomplished in software by adjusting the gray levels of individual pixels in the image engine, or by including an apodization filter in the projector lens. By providing an extremely flat field of illumination (e.g., less than 1% dropoff) in the projection optics, illumination matching to other channels can be controlled at the display projector side of the system. Commercial digital projectors offer a wide range of lumination technologies, including LED, Xenon and UHP lamps, providing typically 4000 – 40,000 lumens [45][46]. Figure 10 (a-d) shows photos of LC on silicon (LCOS) projectors employed in an H-60 flight simulator. In order to further provide uniform illumination in the VCD, the back-projection screen (i.e., the intermediate image surface) may be deposited with a transmissive Lambertian (i.e., uniform brightness from any angle of view) diffuser. White point matching will be predominantly controlled at the image engine level of the display. Virtually all commercial digital projectors include hardware and/or software for calibrating the color gamut/temperature, brightness, contrast and other properties of the display. In order to improve the ability to match the white point of our display with other channels, we intend to design our system around the display projectors and image engines used in many commercial and military flight simulators. Christie Digital [46] and Barco [45] are examples of leading manufacturers whose digital projectors are often used in flight simulator systems. Both companies offer solutions that may be viable for our display.
(a)
(b)
(c)
(d)
Figure 10. (a) and (b) Projector bay located above cockpit of simulator; (c) and (d) LCOS projectors
Figure 11. Relative illumination for image-plane to chin window distances of 0.9, 3 and 18 m as a function of field angle. The relative illumination shows an approximately 20% dropoff at the image corner (i.e., field angle of 13 deg). From close range (i.e., image distance = 0.9 m) to long range (i.e., image distance = 18 m), and at all field angles, the relative illumination at the field edge only differs by approximately 5%.
3.4 Spot Size vs. Image Plane Distance to View Zone Figure 12 is a plot showing the average polychromatic RMS spot diameter vs. image-plane to chin window distance. The plotted spot size is averaged (with uniform weighting) at wavelengths of 656.3, 587.6 and 486.1 nm. Throughout the range of image plane locations the spot diameter is reasonably small, having an overall average value of 99 m. Based on a 114mm-diameter transmissive, diffuse back-projection screen, this average polychromatic RMS spot size corresponds to a resolution of over 1150 pixels along the diagonal of the screen, about a factor of 2X greater than the resolution requirement given in Table 1.
Average RMS Polychromatic Spot Diameter (microns)
130
120
110
100
90
80
70 0
10
20
30
40
50
Image Plane to Chin Window Distance (ft)
Figure 12. Average RMS spot diameter vs. image plane location
60
3.5 Modulation Transfer Function Figure 13(a-e) shows the polychromatic MTF vs. spatial frequency for various configurations for the image-plane to chin window distances. The image-plane to chin window distances selected for the analysis were 0.9, 1.2, 3.0, 8.0 and 18.3 m. For each of the configurations of Figure 13, the MTF is plotted at field angles of 0, 1.3, 2.6, 3.9, 5.2, 6.5, 7.8, 9.1, 10.4, 11.7 and 13 deg. The authors were particularly interested in examining the MTF at the upper limit of the system resolution, 2.7 LP/mm (5 arc min/OLP). Therefore, in Figure 13(a-e), the MTF is plotted for spatial frequencies ranging from 0 to 3 LP/mm. For the majority of the image plane configurations, spatial frequencies, and field angles, the average MTF is approximately 80%.
Figure 13. Polychromatic MTF vs. spatial frequency for various image-plane-to-chin-window distances
3.6 Resolution and Contrast Ratio In this section we analyze the contrast ratio (CR) of our display to confirm that it will meet the system requirement given as 12:1 or greater in Table 1. In order to proceed with our CR analysis, we first convert the resolution requirement into units of line pairs per mm (LP/mm). The required resolution density has been given in Table 1 as < 5 arcmin/line-pair (LP). Converting units, this requirement can be expressed as < 0.083 deg/LP. Then, by inverting this expression we get: Eq. 1
Required resolution density > 12 LP/deg.
The full-angular FOV requirements for the system were also given in Table 1 to be: Eq. 2
Required full-angular FOV = 24 (horizontal) x 10 (vertical) deg.
We then obtain the system’s required display resolution in terms of line pairs by taking the product of Eq. 1 and Eq. 2: Resolution requirement = resolution density Resolution requirement > {24 deg Eq. 3
full-angular FOV, or
12 LP/deg} (horiz.) x {10 deg
12 LP/deg} (vert.), or
Resolution requirement > 288 LP (horizontal) x 120 LP (vertical).
Next, we evaluate the MTF at an image surface. We use an intermediate image surface, found in Zemax® to have dimensions of 105.1 x 43.7 mm. We can then obtain the display’s spatial frequency requirements (in units of LP/mm) by dividing the resolution requirement (Eq. 3) by the intermediate image dimensions and find: Eq. 4
Resolution spatial frequency requirement = 2.7 LP/mm
This is the greatest spatial frequency that the system will need to be capable of resolving. Hence, 2.7 LP/mm is the frequency that we will give primary attention to in determining the limits of the contrast ratio of our optical design. Figure 14(a-e) shows a series of five MTF vs field-angle charts at a spatial frequency of 2.7 LP/mm for image-plane to chin window distances of: (a) 3.0 ft (0.9 m), (b) 4.07 ft (1.2 m), (c) 9.8 ft (3.0 m), (d) 26.2 ft (8.0 m), and (e) 60 ft (18.3 m). The MTF plots were generated by Zemax® software as part of our optical design analysis. In each of the charts, the MTF is plotted at field angles ranging from the optical axis (i.e., 0 deg) to the edge of the FOV (i.e., 12 deg). With the exception of a rapid MTF dropoff at image-plane to chin window locations smaller than 4.0 ft (i.e., Figure 14(a)), the MTF plots are consistently very good and average 80-85% across the field for all system configurations. Note there is also a dropoff at large fields (i.e., > 10 deg) and large image-plane to chin window distances, (i.e., 60 ft, see Figure 14(e)).
Figure 14. MTF @ 2.7 LP/mm vs field angle at various image-plane to chin window distances
Now that we have charted the MTF at 2.7 LP/mm (Figure 14), we can use this data to analyze the CR (i.e., “luminance contrast ratio”). We begin by using the basic interpretation of MTF, where MTF is equal to the luminance contrast modulation, CM [43][44], Eq 5
.
Contrast modulation as a function of frequency CM(f) is defined as: Eq. 6 where LH(f) is the maximum luminance at a given spatial frequency, f; LL(f) is the minimum luminance at f; and 0 ≤ CM(f) ≤ 1 [42][43]. We can rearrange Eq. 6 as: , which gives the standard definition of contrast ratio, CR [42]: Eq. 7
.
As mentioned above, we are primarily interested examining the contrast ratio at f = 2.7 LP/mm, the high-frequency limit required for the display (see Eq. 4). Therefore, we must first determine the average MTF at f = 2.7 LP/mm throughout the range of image plane locations.
Figure 15 is a plot of average MTF at 2.7 LP/mm vs image-plane to chin window distance. To obtain this plot, for each configuration given in Figure 14(a-e), the average MTF is found over the full field.
Figure 15. Average MTF (at 2.7 LP/mm) vs. image-plane to chin window distance
In Figure 15 we see that the average MTF at 2.7 LP/mm is considerably high, having an average value of approximately 84% for image plane distances from the chin window of 1.2 to 18.3 m. At distances smaller than 1.2 m, the MTF drops off sharply, this is possibly due to large Lagrange invariant and low f/# of the system. Hence, we have the following: Eq. 8
MTFaverage(2.7 LP/mm) = 0.841.
Now, inserting Eq. 8 into Eq. 7, we find Eq.9
.
/
.
/
.
/
.
It is typical to indicate a system’s luminance contrast ratio by adding “:1” after the CR quotient [42], therefore we have: Eq.10
: .
Finally, we have determined that our optical design is capable of achieving the system requirement of at least a 12:1 contrast ratio. 3.7 Multispectral Performance It may be desirable for the system to perform in both the visible (VIS) and near infrared (NIR) spectral bands, enabling simulations and training using both visible light imagery and the use of NVG. Holochip’s adaptive lenses are capable of simultaneous performance in the VIS and NIR bands.
4. CONCLUSION AND FUTURE WORK In this paper we presented the optical design and analysis for a variable-collimation display (VCD) system which achieves the requirements for use as a WAC display in the chin windows of Navy UH-60 and CH-47 flight training simulators. By incorporating variable collimation in a WAC display, accurate vergence and accommodation visual cues may be presented, greatly reducing the effects of Athenopia on the user.
In future work, we plan to complete the VCD prototype system, including the integration and test of optical, image generation (IG) and control subsystems. Finally, the VCD prototype will be installed in the chin window section of a UH60 or CH-47 flight trainer system under management by the Navy. This installation phase is planned to be completed in 2016. Near-term tasks will focus on integration and testing of the adaptive lens and optical subsystem as proof of concept. The initial VCD prototype will include only one image plane which varies in position relative to the user (i.e., operation in varifocal mode). Future improvements will include the simultaneous display of multiple image planes [26][34], and, hence, operation in multifocal mode.
ACKNOWLEDGEMENT This material is based upon work supported by the DOD/NAVY SBIR Program under Contract No. N68335-13-C-0231.
REFERENCES [1] McIntyre, H. M., and M. E. C. Roberts. "Simulated Visual Scenes-Which are the Critical Cues?." Thomson Training and Simulation Ltd, Flight Simulation: Where are the Challenges? p(SEE N 97-10546 01-09) (1996). [2] Cutting, James E. "How the eye measures reality and virtual reality." Behavior Research Methods, Instruments, & Computers 29.1 (1997): 27-36. [3] S. Lee, M. Choi, E. Lee, K.D. Jung, J.H. Chang, and W. Kim. “Four zoom lens design for 3D laparoscope by using liquid lens”. Proc. of SPIE, 2012, Vol. 8488, 848809, pp. 1-9. [4] S. Liu, D. Cheng, H. Hua. “An Optical See-Through Head Mounted Display with Addressable Focal Planes”. 7th IEEE/ACM ISMAR 2008, pp. 33-42. [5] H. Hua, and S. Liu. “Depth-fused multi-focal plane displays enable accurate depth perception”. Proc. of SPIE, 2010, Vol. 7849 ,78490, pp. 1-9. [6] Y.Iimura, T. Teshima, Y.J. Heo, Y. Morimoto, S. Yoshida, H. Onoe, and S. Takeuchi. “MICROFLUIDICALLY TUNABLE LENTICULAR LENS”, Transducers 2013, Barcelona, SPAIN, 16-20 June 2013, IEEE, pp. 1787-1790. [7] J. Geng. “Volumetric 3D Display Using a DLP Projection Engine”, Proc. of SPIE, 2012, Vol. 8254 82540I pp. 1-10. [8] S. Liu, H. Hua, D. Cheng. “A Novel Prototype for an Optical See-Through Head-Mounted Display with Addressable Focus Cues”. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, MAY/JUNE 2010, VOL. 16, NO. 3, pp. 381-393. [9] S. Liu, H. Hua. “Time-multiplexed dual-focal plane head-mounted display with a liquid lens”, OPTICS LETTERS, June 1, 2009, Vol. 34, No. 11, pp. 1642-1644. [10] H. Hua, D. Cheng, Y. Wang, S. Liu. “Near-eye displays_state-of-the-art and emerging technologies”. Proc. of SPIE , 2010, Vol. 7690 769009 pp. 1-12. [11] S. Shiwa, K. Omura, and F. Kishino, “Proposal for a 3-D display with accommodative compensation: 3DDAC,”Journal of the Society for Information Display, 1996, vol. 4, pp. 255-261. [12] X. Hu, and H. Hua. “An Optical See-through Multi-Focal-Plane Stereoscopic Display Prototype Enabling NearlyCorrect Focus Cues”, Proc. of SPIE-IS&T Electronic Imaging, 2013, Vol. 8648, 86481A pp. 1-6. [13] T. Shibata, T. Kawai, K. Ohta, M. Otsuki, N. Miyake, Y. Yoshihara, and T. Iwasaki, “Stereoscopic 3-D display with optical correction for the reduction of the discrepancy between accommodation and convergence,” Journal of the Society for Information Display, 2005, vol. 13, no. 8, pp. 665-671. [14] M. Emoto, T. Niida, and F. Okano. “Repeated Vergence Adaptation Causes the Decline of Visual Functions in Watching Stereoscopic Television”, JOURNAL OF DISPLAY TECHNOLOGY, DECEMBER 2005, VOL. 1, NO. 2, pp. 328-340. [15] T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks. ” The zone of comfort : Predicting visual discomfort with stereo displays”, Journal of Vision ,2011 , 11 (8):11, pp. 1–29. [16] T. Fukushima, M. Torii, K. Ukai, J. S. Wolffsohn, and B. Gilmartin. “The relationship between CA/C ratio and individual differences in dynamic accommodative responses while viewing stereoscopic images”, Journal of Vision ,2009, 9 (13):21, pp. 1–13. [17] T. Inoue and H. Ohzu. “Accommodative responses to stereoscopic three-dimensional display”, Applied Optics, 1997, vol. 36, no. 19, pp. 4509-4515. [18] D. Lee, T. Koike, K. Takahashi, and T. Naemura. “Optical and Synthetic Focusing Using a Liquid Lens Array”, IEICE Technical Report, 2009, PRMU2008-192, pp. 19-24.
[19] A. Tolosa, R. Martínez-Cuenca, A. Pons, G. Saavedra, M. Martínez-Corral, and B. Javidi.“Optical implementation of micro-zoom arrays for parallel focusing in integral imaging”. J. Opt. Soc. Am. A , March 2010, Vol. 27, No. 3, pp. 495-500. [20] Y. Liu, H. Ren, S. Xu, Y. Chen, L. Rao, T. Ishinabe, and S. T. Wu. “Adaptive Focus Integral Image System Design Based on Fast-Response Liquid Crystal Microlens”. JOURNAL OF DISPLAY TECHNOLOGY, DECEMBER 201. VOL. 7, NO. 12. [21] Y.Y. Kao, Y.P. Huang, K.X. Yang, P. C. P. Chao, C.C. Tsai and C.N. Mo. ”An Auto-Stereoscopic 3D Display Using Tunable Liquid Crystal Lens Array that Mimics Effects of GRIN Lenticular Lens Array”. SID 09 DIGEST, pp. 111114 [22] Y. Kim, K. Hong, J. Yeom, J. Hong, J.H. Jung, Y. W. Lee, J.H. Park, and B. Lee. “A frontal projection-type threedimensional display”, OPTICS EXPRESS, 27 August 2012, Vol. 20, No. 18, pp. 20130-20138. [23] T. Kreis. “Applications of Digital Holography: From Microscopy to 3D-Television”, J. Europ. Opt. Soc., 2012, Rap. Public. 7, 12006, pp. 1-9. [24] C. P. Lee , A. Asundi, Y. Yu, Z. Z. Xiao. “Digital Holography Display (2)”, Physics Procedia 19, 2011, pp. 278–284. [25] D. Lanman, and D. Luebke. “Near-Eye Light Field Displays”, ACM Transactions on Graphics, November 2013, Vol. 32, No. 6, Article 220. [26] G. D. Love, D. M. Hoffman, P. J.W. Hands, J. Gao, A. K. Kirby, and M. S. Banks. “High-speed switchable lens enables the development of a volumetric stereoscopic display”, Opt Express. August 31, 2009, 17 (18), pp.15716– 15725. [27] T. Sonoda, H. Yamamoto, and S. Suyama. “A new volumetric 3D display using multi-varifocal lens and high-speed 2D display”. SPIE-IS&T, 2011, Vol. 7863 786322, pp. 1-9. [28] Q. Feng, H. Tong, T. Liu, Y. Chang, G. Lv, and H. Wu. “Single-DMD based Solid-State Volumetric True 3D Display”, ICALIP, IEEE, 2010, pp.552-556. [29] http://store.sony.com/WFS/SNYUS/en_US/-/USD/-zid27-HMZT3W/cat-27-catid-3D-PersonalViewer?k_id=7caf305d-807d-9609-3ae4000045374109&kpid=HMZT3W&adparam=sem73pla&XID=O:HMZT3W:dg_gglsrch:pla&cagpspn=pla&gclid=C MXCk_ihpbsCFShk7AodjFYA7g [30] http://www.epson.com/cgi-bin/Store/jsp/Moverio/Home/Developers.do [31] http://www.vuzix.com/consumer/products_vr920.html [32] E. F. FINCHAM and J. WALTON. “THE RECIPROCAL ACTIONS OF ACCOMMODATION AND CONVERGENCE”. J. Physiol. I957, I37, pp. 488-508. [33] W. N. Charman and J. Tucker. “DEPENDENCE OF ACCOMMODATION RESPONSE ON THE SPATIAL FREQUENCY SPECTRUM OF THE OBSERVED OBJECT”, Vision Res., 1977, Vol. 17, pp. 129-139. [34] L. Ryan, K. J. MacKenzie, S. J. Watt. “MULTIPLE-FOCAL-PLANES 3D DISPLAYS: A PRACTICAL SOLUTION TO THE VERGENCE-ACCOMMODATION CONFLICT?”, IC3D, IEEE, 2012, pp. 1-6. [35] S. J. Watt, K. Akeley, M.O. Ernst, and M.S. Banks. “Focus cues affect perceived depth”, Journal of Visio, 2005, vol.5, pp. 834–862. [36] D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks. “Vergence–accommodation conflicts hinder visual performance and cause visual fatigue”, Journal of Vision, 2008, 8 (3):33, pp. 1–30. [37] S. C. McQuaide, E. J. Seibel, J. P. Kelly, B. T. Schowengerdt, and T. A. Furness III., “A retinal scanning display system that produces multiple focal planes with a deformable membrane mirror”, Displays, 2003, vol. 24, pp. 65–72. [38] J. G. Lu, X. F. Sun, Y. Song, and H. P. D. Shieh. “2D/3D Switchable Display by Fresnel-Type LC Lens”, JOURNAL OF DISPLAY TECHNOLOGY, APRIL 2011.VOL. 7, NO. 4. [39] Y.P. Huang, C.W. Chen, Y.C. Huang. “Fast Response Fresnel Liquid Crystal Lens for 2D 3D Autostereoscopic Display”. Proc. of SPIE, 2011, Vol. 7956 79560, pp. 1-6. [40] Private communications, Flight Safety, International. [41] http://www.barco.com/en/. [42] http://en.wikipedia.org/wiki/Display_contrast#Luminance_contrast. [43] http://www.normankoren.com/Tutorials/MTF.html/. [44] http://en.wikipedia.org/wiki/Optical_resolution. [45] http://www.barco.com/en/products-solutions/projectors/simulation-projectors/wuxga-single-chip-dlp-projector-formulti-channel-applications.aspx [46] http://www.christiedigital.com/en-us/business/products/projectors/throw-distance-calculator/Pages/default.aspx. [47] http://www.rpcphotonics.com/hilam.asp.
[48] Lufthansa Flight Training Berlin; (Installation of CAE full flight Simulator b777 series 7000); Upload Date 2009 http://www.youtube.com/watch?v=Wqq4-XC5f8U&feature=youtube_gdata_player. [49] http://www.glassmountain.com/simulation/dualeyepointdisplays. [50] Dodgson, N. A. (2004). Variation and extrema of human interpupillary distance. In A. J. Woods, J. O. Merritt, S. A. Benton and M. T. Bolas (eds.), Proceedings of SPIE: Stereoscopic Displays and Virtual Reality Systems XI, Vol. 5291, pp. 36–46. San Jose CA. [51] Jump up ^ Smith, G., & Atchison, D. A. (1997). The eye and visual optical instruments. Cambridge UK: Cambridge University Press. [52] J.-Y. Yiu, R. Batchko, S. Robinson and A. Szilagyi, "A fluidic lens with reduced optical aberration", Proc. SPIE 8301, 830117 (2012).
