IEEE PHOTONICS TECHNOLOGY LETTERS, VOL. 28, NO. 3, FEBRUARY 1, 2016
295
Adaptive Polarization-Modulated Method for High-Resolution 3D Imaging Zhen Chen, Bo Liu, Enhai Liu, and Zhangxian Peng
Abstract— To improve the resolution in 3D range-gated imaging, an adaptive polarization-modulated method using electron multiplying charge-coupled devices (EMCCDs) is proposed in this letter. With polarization beam splitting, the returned light is separated into two orthogonal polarized components, which are modulated by cos2 function and sin2 function, respectively. Based on a dual EMCCDs structure, both a depth map and an intensity image can be reconstructed from the two modulated images instantaneously, which will enhance the performance of dynamic imaging. The results show that the range accuracy of the interested objects in the scene can be promoted from 4.4 to 0.26 m with the art of adaptive gate-opening range. Index Terms— Laser radar, high-resolution imaging, optical polarization, adaptive control.
I. I NTRODUCTION
C
OMPARED with direct method of TOF (time of flight) measurement [1], indirect method based on intensified CCD (ICCD) can provide higher transverse resolution and lower sampling rate in 3D imaging. There were three methods about indirect 3D imaging: time slicing [2], [3], intensity correlation [4], [5] and gain modulation [6], [7]. In general, a higher temporal resolution will lead to a greater range resolution that can provide a more detailed mapping of the scene in the 3D imaging systems [8]. Specifically, time slicing has the highest resolution at the cost of large number of images which means a long imaging time. Intensity correlation is a special case of time slicing in which 3D reconstruction requires only three images, and therefore faster acquisition can be achieved. However, in order to produce a trapezoid-shaped range intensity image, rectangle-shaped pulses are required. Besides, the gated opening time must be twice as long as the laser pulse width. Gain modulation, by contrast, is a pulse-shape-free method with linear-constant gain retrieve [7], linear-linear gain retrieve [9] or exponential-constant gain retrieve [10] to achieve time-resolved imaging, which needs only two images to reconstruct a depth map. As we know, ICCD is an excellent detector that has the ability of high speed gating [11]. However, the imaging
Manuscript received September 14, 2015; revised October 9, 2015; accepted October 21, 2015. Date of publication October 28, 2015; date of current version December 22, 2015. Z. Chen and Z. Peng are with the Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China, and also with the University of Chinese Academy of Sciences, Beijing 100049, China (e-mail:
[email protected];
[email protected]). B. Liu and E. Liu are with the Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China (e-mail:
[email protected];
[email protected]). Color versions of one or more of the figures in this letter are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LPT.2015.2495113
Fig. 1.
Diagram of polarization-modulated 3D imaging lidar system.
procedure of ICCD includes photon-electron conversion, electron-photon conversion and photon-electron conversion again, which limits its transverse resolution and modulation transfer function (MTF) [12]. Compared with ICCD, EMCCD is a new high sensitivity image sensor which performs single photon-electron conversion without coupling other devices. Consequently, it can provide higher transverse resolution, better MTF and smaller physical size [12], [13]. However, the EMCCD itself can’t obtain depth map of the scene due to the time integration mechanism. Fortunately, polarizationmodulated method has been proved to be an effective way for time-resolved imaging [14], [15], but the range-gated imaging (RGI) can’t be achieved in these systems. Applying two modulators and four FPAs (Focal Plane Arrays) in the polarization-modulated method, range-gated imaging can be completed [16]; however it’s quite costly with four EMCCDs. With the EMCCD, an adaptive polarization-modulated method to improve the transverse resolution and range resolution is proposed in this letter. Furthermore, the method can perform range-gated imaging with two EMCCDs only, because the photosensitive time of EMCCDs can be controlled by electro-optic gating simultaneously. In addition, a far imaging range can be obtained by EMCCD due to its high sensitivity and low noise. II. S YSTEM S ETUP The polarization-modulated 3D imaging lidar system is shown in Fig. 1. A narrowband filter (NBF) is used to remove background radiation from sunlight and other sources. Afterwards, a linear polarizer (P1, parallel to the emitted linear-polarized light) is required to prepare linear-polarized light for electro-optic modulator (EOM). Applying a voltage to the EOM, phase retardation proportional to the voltage will take place. Consequently, we are able to convert the linearpolarized light into an elliptical-polarized light [17]. With polarization beam splitting (PBS), the elliptical-polarized
1041-1135 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
296
IEEE PHOTONICS TECHNOLOGY LETTERS, VOL. 28, NO. 3, FEBRUARY 1, 2016
Fig. 4. Phase shift will take place when a voltage is applied to the electrooptic modulator.
Fig. 2.
is attenuated by additional components (e.g. photocathode, phosphor screen) in ICCD [18] while it hardly changes with the electron multiplying gain in EMCCD (M T FE M ≈ 1) [19], and therefore MTF E MCC D is much better than MTF I CC D .
Range-gated imaging in polarization-modulated method.
III. A DAPTIVE P OLARIZATION -M ODULATED M ETHOD
Fig. 3. Comparision of ICCD and EMCCD. (a) Multiple conversions between photon (Pho) and electron (Ele) are included in ICCD and (b) EMCCD performs single photon-electron conversion.
light is separated into two orthogonal polarized components: p-polarized and s-polarized light. The p-polarized light passes to channel X and finally arrives at EMCCDx, while the s-polarized light is reflected to channel Y and arrives at EMCCDy. In addition, EOM2 and P2 are placed between the PBS and lens Lx and they cooperate with the EOM1 to determine when and how long the p-polarized light passes through channel X. Additional electro-optic modulator is unnecessary in channel Y because the EOM1 and PBS can take this work completely. Ultimately, range-gated system can be completed by controlling the applied voltages of the two EOMs. As shown in Fig. 2, V (t) and V X (t) are the applied voltages of two EOMs (see Fig. 1); L is the gate opening range determined by the duration time of gate opening (TG ); R0 is the range corresponding to the optical gating open beginning time; D is the range between the object and R0 ; R is the range between the range-gated imaging (RGI) system and the object, which is given by R = R0 + D
(1)
EMCCD will take over ICCD’s work in the polarizationmodulated system. Comparison of the ICCD and EMCCD is shown in Fig. 3. The modulation transfer functions for ICCD and EMCCD can be derived from (2) and (3), respectively. M T FI CC D = M T FCathode · M T FMC P · M T FScreen M T FE MCC D
· M T FF iber · M T FCC D = M T FE M · M T FCC D
(2) (3)
where MTFi (i = ICCD, EMCCD, . . . , CCD) represents the modulation transfer function of component i . The MTF
The reflected/scattered light returned to the RGI system is accumulated during the gated opening time. Then, the received energy in each pixel can be expressed as follows [9], [10], [20] 2 T2 η ρt ar Dopt at m opt PT (t − 2R/c)dt (4) E R EC = 4R 2 TG where ρt ar is the reflectivity of target, Dopt is the diameter of receiver, Tat m is the transmission efficiency through the atmosphere, ηopt is the optical system efficiency, PT is the transmitted power, R is the range from the RGI system to the object, and c is the speed of light. Since the EMCCDs can’t obtain depth map, electro-optic modulator based on electro-optic effect of crystals is applied to realize time-resolved imaging in Fig. 4. When a voltage is applied to the electro-optic modulator in direction of propagation (z axis), phase retardation of the polarized light will take place. The phase shift θ proportional to the applied voltage V (t) can be given by [17] 2π 3 n γ63 V (t) (5) θ= λ 0 where λ is the wavelength of laser beam, n 0 is the ordinary refractive index of crystal, and γ63 is the crystal electro-optic coefficient. The applied voltage V (t) is given by Vπ t, (0 ≤ t ≤ TG ) (6) V (t) = TG where Vπ is the half wave voltage of the crystal. Then the phase shift can be rewritten as a function related to range. D (7) θ = π · , (0 ≤ D ≤ L) L During the gated opening time, phase retardation of the polarized light increases linearly with time. With beam polarization splitting, p-polarized and s-polarized components are separated into channel X and Y and accumulated by EMCCDx and EMCCDy, whose intensities are modulated with cos2 function and sin2 function respectively. ⎧ ⎪ 2 θ ⎪ E = I cos R EC ⎨ X 2 (8) ⎪ θ ⎪ ⎩ E Y = I R EC sin 2 2
CHEN et al.: ADAPTIVE POLARIZATION-MODULATED METHOD FOR HIGH-RESOLUTION 3D IMAGING
where E X and E Y are the intensities of p-polarized and s-polarized components, respectively. Since the intensitymodulated images derived from (8) contain range information, 3D reconstruction can be implemented and the depth map can be obtained from them. Combining (7) and (8), D can be derived from
EY 2L arctan (9) D= π EX
297
TABLE I M AIN PARAMETERS OF 3D I MAGING L IDAR S YSTEM
Applying (9) to (1), the range R between the RGI system and the object can be given by
EY 2L (10) arctan R = R0 + π EX Besides, it should be noticed that summation of E X and E Y will produce a demodulated intensity image in (11). E X + E Y = E R EC
(11)
According to the variance propagation law [21], we can get the standard deviation of range from (10). L EEYX σ E2 σ2 2X + E2Y (12) σR = EX EY π EEYX + 1 where σ E2 X and σ E2Y are the variances of E X and E Y respectively. The ratio of E X and σ E X is the signal-to-noise ratio (SNRx) in channel X; also, the ratio of E Y and σ EY is the signal-to-noise ratio (SNRy) in channel Y. It can be known that the range error is proportional to the gate opening range if the error of R0 is ignored. When the gate opening range is adjusted, the range accuracy will be changed accordingly. Based on the analysis above, an adaptive polarizationmodulated method is designed to improve the range accuracy for the RGI system. When the duration time of gate opening TG is adjusted adaptively, the gate opening range will be compressed correspondingly, and therefore higher range resolution will be achieved. The adaptive method contains two procedures: coarse range imaging and fine range imaging. In the first step, a large gate opening range is applied to search objects in the field of view and get their rough distance information. In the second step, the gate beginning range R0 and the gate opening range L will be set appropriately to fit the object of interest. Ultimately, the depth map with higher range resolution can be reconstructed from two images under a narrow gate opening range. IV. R ESULTS AND D ISCUSSIONS In order to prove the adaptive polarization-modulated method for range-gated imaging, a RGI system is established. The system works in “flash” mode, and therefore a full depth map will be reconstructed in single pulsed cycle. The pulsed laser works as an illuminator at low repetition rate, high pulse energy and short pulse duration. Meanwhile, dual EMCCDs are used to accumulate the polarized lights in channel X and Y, respectively. The main parameters of the 3D imaging lidar system are given in Table I.
Fig. 5. Depth maps of adaptive polarization-modulated method. (a) result with gate opening range from 5000m to 5200m; (b) result with gate opening range from 5020m to 5030m; (c) result with gate opening range from 5070m to 5090m; (d) result with gate opening range from 5155m to 5185m.
The distance between the RGI system and the objects ranges from 5000m to 5200m. During the coarse range imaging, a gate opening range of 200m is designated to search objects. As shown in Fig. 5(a), three objects with different gray values are included in the depth map under the large gate opening range. The gray value represents range between the RGI system and the objects. According to the dataset derived from coarse range imaging, it can be known that the vehicle ranges roughly from 5020m to 5030m; the steeple building ranges roughly from 5070m to 5090m and the flat top building ranges from 5155m to 5185m. In Fig. 5(a), each object is represented in almost single gray value which indicates that the gate opening range in the coarse range imaging is too large to provide enough range resolution for a single object, and therefore fine range imaging with smaller gate opening range is necessary to explore a more detailed depth map. To explore the vehicle, fine range imaging with small gate opening range of 10m (5020m-5030m) was performed. The result is shown in Fig. 5(b), in which only the vehicle appears and is presented in detailed depth map.
298
IEEE PHOTONICS TECHNOLOGY LETTERS, VOL. 28, NO. 3, FEBRUARY 1, 2016
the system. With which the polarization modulation could be performed, and then a 3D image was reconstructed from the two modulated images. With the method intensity image could also be obtained. The results showed that 3D reconstruction was accomplished in high resolution. Especially, by the adaptive range gate controlling, the depth image could be “zoomed in” on range dimension, which provided the ability for object structure and attitude exploration. Fig. 6. Intensity images modulated by (a) cos2 function; (b) sin2 function; and (c) 1 (summation of cos2 and sin2 ). TABLE II T HE P ERFORMANCE OF S EVERAL R ANGE -G ATED I MAGING S YSTEMS
Similarly, Results of fine range imaging with gate opening range of 20m (5070m to 5090m) and 30m (5155m to 5185m) are shown in Fig. 5(c) and (d). The standard deviations of range accuracy are 4.4m, 0.26m, 0.34 and 0.40m under the gate opening range of 200m, 10m, 20m and 30m respectively. Compared with Fig. 5(a), the range resolution in Fig. 5(b), (c) and (d) has been improved obviously, with which more detailed structure and attitude information can be obtained. Besides, the intensity images with cos2 and sin2 modulation are shown in Fig. 6(a) and (b), from which an intensity-demodulated image could be obtained in Fig. 6(c) according to (11). In Fig. 6(a), the near-range object is modulated into a bright representation while the far-range object is modulated into a dark representation. The opposite situation will happen in Fig. 6(b), and therefore the near-range object is modulated into a dark representation while the far-range object is modulated into a bright representation. Compared with other range-gated imaging methods, the adaptive polarization-modulated method can achieve higher range-resolution with fewer numbers of acquired frames and meanwhile bring higher image-quality (see Table II). V. C ONCLUSION An adaptive polarization-modulated method for 3D imaging, which could improve the resolution, was presented in this letter. A new imaging sensor EMCCD was introduced in 3D imaging lidar system. Compared with ICCD, it could bring much better MTF of imaging. To provide range gate controlling and intensity modulation for the image recorded by EMCCD, two electro-optic crystals were used in
R EFERENCES [1] M. A. Albota et al., “Three-dimensional imaging laser radar with a photon-counting avalanche photodiode array and microchip laser,” Appl. Opt., vol. 41, no. 36, pp. 7671–7678, 2002. [2] J. Busck and H. Heiselberg, “Gated viewing and high-accuracy three-dimensional laser radar,” Appl. Opt., vol. 43, no. 24, pp. 4705–4710, 2004. [3] J. Busck, “Underwater 3-D optical imaging with a gated viewing laser radar,” Opt. Eng., vol. 44, no. 11, p. 116001, 2005. [4] M. Laurenzis, F. Christnacher, and D. Monnin, “Long-range three-dimensional active imaging with superresolution depth mapping,” Opt. Lett., vol. 32, no. 21, pp. 3146–3148, 2007. [5] W. Xinwei, L. Youfu, and Z. Yan, “Multi-pulse time delay integration method for flexible 3D super-resolution range-gated imaging,” Opt. Exp., vol. 23, no. 6, pp. 7820–7831, 2015. [6] M. Kawakita, K. Iizuka, R. Iwama, K. Takizawa, H. Kikuchi, and F. Sato, “Gain-modulated Axi-vision camera (high speed high-accuracy depthmapping camera),” Opt. Exp., vol. 12, no. 22, pp. 5336–5344, 2004. [7] Z. Xiuda, Y. Huimin, and J. Yanbing, “Pulse-shape-free method for longrange three-dimensional active imaging with high linear accuracy,” Opt. Lett., vol. 33, no. 11, pp. 1219–1221, 2008. [8] G. Tsagkatakis et al., “Multireturn compressed gated range imaging,” Opt. Eng., vol. 54, no. 3, p. 031106, 2015. [9] X. Zhang, H. Yan, and Q. Zhou, “Overcoming the shot-noise limitation of three-dimensional active imaging,” Opt. Lett., vol. 36, no. 8, pp. 1434–1436, 2011. [10] C. Jin, X. Sun, Y. Zhao, Y. Zhang, and L. Liu, “Gain-modulated threedimensional active imaging with depth-independent depth accuracy,” Opt. Lett., vol. 34, no. 22, pp. 3550–3552, 2009. [11] S. E. Moran, B. L. Ulich, W. P. Elkins, R. J. Strittmatter, and M. J. DeWeert, “Intensified CCD (ICCD) dynamic range and noise performance,” Proc. SPIE, vol. 3173, pp. 430–457, Dec. 1997. [12] O. Djazovski et al., “Electron-multiplying CCDs for future space instruments,” Proc. SPIE, vol. 8915, p. 89150Q, Oct. 2013. [13] D. J. Denvir and E. Conroy, “Electron-multiplying CCD: The new ICCD,” Proc. SPIE, vol. 4796, pp. 167–174, Feb. 2003. [14] J. Taboada and L. A. Tamburino, “Laser imaging and ranging system using two cameras,” U.S. Patent 5 157 451, Oct. 20, 1992. [15] K. W. Ayer, W. C. Martin, J. M. Jacobs, and R. H. Fetner, “Laser imaging and ranging system (LIMARS): A proof-of-concept experiment,” Proc. SPIE, vol. 1633, pp. 54–62, Jun. 1992. [16] P. S. Banks, “Systems and methods of high resolution three-dimensional imaging,” U.S. Patent 8 471 895 B2, Jun. 25, 2013. [17] A. Yariv and P. Yeh, “Electro-optic devices,” in Optical Waves in Crystals: Propagation and Control of Laser Radiation, 1st ed. Hoboken, NJ, USA: Wiley, 2002, pp. 276–317. [18] N. Gül and M. Efe, “Improved analytical modulation transfer function for image intensified charge coupled devices,” Turkish J. Elect. Eng. Comput. Sci., vol. 18, no. 1, pp. 71–81, 2010. [19] W. Zhang, H. Liu, P. Zou, Q. Chen, G. Gu, and L. Zhang, “Research on modulation transfer function of the electron multiplying CCD,” presented at the IEEE Symp. Photon. Optoelectron., 2012, pp. 1–4. [20] P. F. McManamon, “Review of ladar: A historic, yet emerging, sensor technology with rich phenomenology,” Opt. Eng., vol. 51, no. 6, p. 060901, Jun. 2012. [21] J. R. Taylor, “Propagation of uncertainties,” in Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements, 2nd ed. Sausalito, CA, USA: Univ. Science Books, 1997, pp. 45–92.
