Converting optical scanning holograms of real ... - OSA Publishing

Converting optical scanning holograms of real objects to binary Fourier holograms using an iterative direct binary search algorithm Thibault Leportier,1,2 Min Chul Park,1,2,4 You Seok Kim,3 and Taegeun Kim3,* 1

Department of HCI and Robotics, Korea University of Science and Technology, Hwarangno 14-gil 5, Seongbuk-gu, Seoul 136-791, South Korea 2 Sensor System Research Center, Korea Institute of Science and Technology, Hwarangno 14-gil 5, Seongbuk-gu, Seoul 136-791, South Korea 3 Department of Optical Engineering, Sejong University,98 Kunja-dong, Kwangjin-gu, Seoul 143-747, South Korea 4 [email protected] * [email protected]

Abstract: In this paper, we present a three-dimensional holographic imaging system. The proposed approach records a complex hologram of a real object using optical scanning holography, converts the complex form to binary data, and then reconstructs the recorded hologram using a spatial light modulator (SLM). The conversion from the recorded hologram to a binary hologram is achieved using a direct binary search algorithm. We present experimental results that verify the efficacy of our approach. To the best of our knowledge, this is the first time that a hologram of a real object has been reconstructed using a binary SLM. © 2015 Optical Society of America OCIS codes: (090.1995) Digital holography; (090.1760) Computer holography.

References and links 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.

R. H.-Y. Chen and T. D. Wilkinson, “Computer generated hologram from point cloud using graphics processor,” Appl. Opt. 48(36), 6841–6850 (2009). Z. W. Xie, J. L. Zang, and Y. Zhang, “Accelerated algorithm for three-dimensional computer generated hologram based on the ray-tracing method,” J. Mod. Opt. 60(10), 797–802 (2013). N. T. Shaked, B. Katz, and J. Rosen, “Review of three-dimensional holographic imaging by multiple-viewpointprojection based methods,” Appl. Opt. 48(34), H120–H136 (2009). T. Mishina, M. Okui, and F. Okano, “Calculation of holograms from elemental images captured by integral photography,” Appl. Opt. 45(17), 4026–4036 (2006). Y. S. Kim, T. Kim, S. S. Woo, H. Kang, T.-C. Poon, and C. Zhou, “Speckle-free digital holographic recording of a diffusely reflecting object,” Opt. Express 21(7), 8183–8189 (2013). C. Slinger, C. Cameron, and M. Stanley, “Computer-Generated Holography as a Generic Display Technology,” IEEE Comput. 46, 52 (2005). C. Slinger, C. Cameron, S. Coomber, R. Miller, D. Payne, A. Smith, M. Smith, M. Stanley, and P. Watson, “Recent developments in computer-generated holography: toward a practical electroholography system for interactive 3D visualisation,” Proc. SPIE 5290, 27–41 (2004). C. Burckhardt and L. Enloe, “Television transmission of holograms with reduced resolution requirements on the camera tube,” Bell Syst. Tech. J. 48(5), 1529–1535 (1969). J. Berrang, “Television transmission of holograms using a narrow-band video signal,” Bell Syst. Tech. J. 49(5), 879–887 (1970). T. Kim, Y. S. Kim, W. S. Kim, and T.-C. Poon, “Algorithm for converting full-parallax holograms to horizontalparallax-only holograms,” Opt. Lett. 34(8), 1231–1233 (2009). T. Kim and Y. S. Kim, “Three-dimensional holographic imaging of a diffusely reflective object with conversion to a horizontal-parallax-only hologram,” in SPIE/COS Photonics Asia (2014), pp. 92710S. T.-C. Poon, “Scanning holography and two-dimensional image processing by acousto-optic two-pupil synthesis,” J. Opt. Soc. Am. A 2(4), 521–527 (1985). G. Indebetouw, P. Klysubun, T. Kim, and T.-C. Poon, “Imaging properties of scanning holographic microscopy,” J. Opt. Soc. Am. A 17(3), 380–390 (2000). T.-C. Poon, T. Kim, G. Indebetouw, B. W. Schilling, M. H. Wu, K. Shinoda, and Y. Suzuki, “Twin-image elimination experiments for three-dimensional images in optical scanning holography,” Opt. Lett. 25(4), 215– 217 (2000). M. A. Seldowitz, J. P. Allebach, and D. W. Sweeney, “Synthesis of digital holograms by direct binary search,” Appl. Opt. 26(14), 2788–2798 (1987).

#226260 - $15.00 USD © 2015 OSA

Received 4 Nov 2014; revised 23 Jan 2015; accepted 25 Jan 2015; published 4 Feb 2015 9 Feb 2015 | Vol. 23, No. 3 | DOI:10.1364/OE.23.003403 | OPTICS EXPRESS 3403

16. J. P. Allebach, “DBS: retrospective and future directions,” Proc. SPIE 4300, 410810 (2001). 17. X. D. Zhao, J. Li, T. Tao, Q. Long, and X. P. Wu, “Improved direct binary search-based algorithm for generating holograms for the application of holographic optical tweezers,” Opt. Eng. 51(1), 015801 (2012). 18. B. K. Jennison, J. P. Allebach, and D. W. Sweeney, “Efficient design of direct-binary-search computer-generated holograms,” J. Opt. Soc. Am. A 8(4), 652–660 (1991). 19. T. Kreis, P. Aswendt, and R. Hofling, “Hologram reconstruction using a digital micromirror device,” Opt. Eng. 40(6), 926–933 (2001). 20. J. Y. Son, B. R. Lee, O. O. Chernyshov, K. A. Moon, and H. Lee, “Holographic display based on a spatial DMD array,” Opt. Lett. 38(16), 3173–3176 (2013).

1. Introduction Holography is one of the most promising techniques for the future of three-dimensional (3D) display technology. Because holography provides all of the necessary depth cues to the viewer’s visual system, issues related to 3D-induced visual fatigue can be overcome. Unfortunately, however, several bottlenecks remain in both the capture and reconstruction of digital holograms. Traditionally, holograms are generated by recording the interference between a reference and an object wavefront. However, this requires a stable optical system with coherent light in a dark room. A common alternative is to use computer generated holograms (CGHs) [1]. Although these do not require laser light, their heavy computational load is a major issue [2]. In addition, because CGHs are generated from a virtual model of a 3D object, they cannot reconstruct real objects. Hence, several alternatives have been proposed, including multiview imaging [3] and integral imaging [4]. Recently, a complex hologram of a diffusely reflective object was recorded without speckle noise using optical scanning holography (OSH) [5]. This technique did not require vibration isolation or a darkroom environment, and reconstructed the hologram using a spatial light modulator (SLM) under coherent illumination. The main issue with OSH is that the resolution of the SLMs is not high enough to display large holograms at a wide angle. To optimize performance with devices that are currently available, approaches using spatial or time multiplexing have been proposed. These usually require a more complex optical set-up, and the format of the holographic data must be adapted to the configuration of the reconstruction system. For instance, in the case of time multiplexing, conversion into a binary hologram allows the full speed of the SLM to be exploited. Recently, a holographic 3D display technique based on a binary search algorithm was proposed [6,7]. This method synthesizes an off-axis hologram of a fictitious 3D object from a virtual model, and then generates a binary pattern to reconstruct the intensity distribution corresponding to this hologram using a binary search algorithm. Finally, an optically addressed SLM reconstructs the 3D object. However, this technique is limited to reconstructing computersynthesized images of some fictitious 3D objects. Holographic imaging systems that record a hologram of a real object as a form of electric signal and reconstruct the hologram in remote space have a longstanding history [8,9]. In previous research, we showed that a complex hologram of a diffusely reflective object recorded by OSH could be optically reconstructed using an amplitude-only SLM after being converted to an off-axis hologram [5]. In addition, we also proposed a horizontal-parallaxonly conversion process for data reduction [10], and demonstrated a closed-loop 3D holographic imaging system with a novel data reduction technique [11]. In this paper, we propose a process that converts a hologram of a real object recorded by OSH to a binary hologram, from which the real object can be directly reconstructed. To the best of our knowledge, this is first time that holograms recorded by OSH from real objects have been optically reconstructed after the raw holographic data has been converted into a binary hologram using the direct binary search algorithm. 2. Recording an optical scanning hologram of a real object OSH is based on a heterodyne with a space integrating detection scheme, and records the complex hologram of a diffusely reflective object without speckle, twin image, or background noise [5, 12–14]. As shown in Fig. 1, the scanning beam is generated by a Mach–Zehnder

#226260 - $15.00 USD © 2015 OSA


interferometer unit consisting of two beam splitters, acousto-optic modulators, mirrors, beam expanders, and one lens. In the Mach–Zehnder interferometer unit, the laser beam is divided at the first beam splitter, with the frequencies of the upper and lower beams shifted to Ω and Ω + ΔΩ by the acousto-optic modulators. The spatial distribution of the upper beam becomes a planar wave as it passes through beam expander 1, whereas the lower beam attains a spherical wave distribution by passing through beam expander 2 and the lens. The upper and lower beams are recombined at beam splitter 2. The cross term of the combined beam beats according to the frequency difference between the upper and lower beams is ΔΩ, and the spatial distribution of the cross term of the combined beam becomes the Fresnel zone pattern given by the interference between plane and spherical waves. The combined beam goes to the first photo-detector and the scanning mirror. This photo-detector generates electric current beats according to the frequency difference between the upper and lower path beams, and the electric current is sent to the reference signal input of the dual-output lock-in amplifier. The combined beam that goes to the scanning mirror scans the diffusely reflective object. Reflected light from the object is spatially integrated by a collecting lens and sent to a second photo-detector. Photo-detector 2 generates an electric current that is proportional to the intensity of the spatially integrated light. This is the spatial integration of the reflectance of the object multiplied by the intensity of the scanning beam. The electric current is composed of DC and heterodyne terms. The heterodyne term beats according to the frequency difference between the upper and lower path beams, and has an amplitude that is proportional to the spatial integration between the Fresnel zone plate (FZP) and the object’s reflectance. The electric current goes to the input of the dual-output lock-in amplifier, which generates the inphase and quadrature phase parts of the heterodyne term. Because the amplitude of the heterodyne term is proportional to the spatial integration between the FZP and the object’s reflectance, the in-phase and quadrature phase parts of the lock-in amplifier become the real and imaginary parts of the holographic recording of the object’s reflectance at the scanning mirror. The two outputs from the lock-in amplifier are sampled by an analog-to-digital converter and sent to the digital computer. In the digital computer, each output is arranged as a 2D matrix according to the position of the scanning mirror. These become the real and imaginary parts of the hologram of the object’s reflectance. The real and imaginary parts of the hologram are added to give the complex hologram of the object’s reflectance. This process is described by: H com ( n, m ) ∝

zo +Δz



zo −Δz

I 0 (n, m; z ) ⊗

 πΔl 2 2  j exp  j ( n + m2 ) dz, z λz λ  

(1)

where (n,m) are the indices of the sampled signal, Δl is the sampling interval, ⊗ represents the convolution operation involving (n,m), z is the distance from the focal point of a spherical wave to each planar distribution of the object, z0 is the depth location of the object (the distance from the focal position of the spherical wave to the center of the object), Δz is the range of the object along the depth direction, I0(n,m;z) is the object’s reflectance, and λ is the wavelength of the laser beam. Integrating along z incorporates contributions from all planar distributions across the depth of the 3D object. The origin of the physical coordinates (x,y,z) is the focal point of the spherical wave, and (x = n.Δl, y = m.Δl) gives the position of the scanning mirror. Note that the range of the object along the depth direction can be captured by OSH is determined by the collecting optics that collects the reflected light from the object to the photo-detector because all the reflected light from the object is spatially integrated by the photo-detection.

#226260 - $15.00 USD © 2015 OSA


Fig. 1. Optical scanning holography (M: mirror, AOM: acousto-optic modulator, BS: beam splitter, BE: beam expander, L: lens, SM: scanning mirror, PD: photo-detector, PC: personal computer)

To verify the performance of the proposed system, we record the complex hologram of a cubic dice of side length 3 mm. The x-y scanning region is 13.5 mm × 13.5 mm, and the diameter of the plane wave on the lower path in Fig. (1) is 25.1 mm. The focal length of the lens (L1) is 500 mm. Figure 2(a) and 2(b) show the amplitude and phase, respectively, of the complex hologram H com ( x, y ) . Note that the hologram is clear, as we can see the phase distribution of the fringe; this is because the hologram is not contaminated with speckle noise.

Fig. 2. (a) Amplitude of the complex hologram of the real object; (b) Phase of the complex hologram of the real object

3. Iterative direct-binary search algorithm Holographic data may be captured or generated in several ways. Such data must be converted into a format that is compatible with the SLM used for the optical reconstruction. The direct binary search (DBS) algorithm enables holographic data to be converted into a binary hologram [15]. Several applications have been shown to benefit from such a technique [16,17]. The DBS algorithm is an iterative method that starts from a random binary patternOn each iteration, the propagation of a wavefront diffracted by the binary Fourier hologram to the space domain is calculated by an inverse discrete Fourier transformation:

#226260 - $15.00 USD © 2015 OSA


i H space ( n, m ) =

M −1 2

1 M ⋅N

N −1 2

 H

u=

−M −N v= 2 2

i Fourier

u ⋅ n v ⋅ n  +  N    M



( u, v ) exp  j 2π  

(2)

i In Eq. (2), M and N represent the size of the Fourier hologram, and H Fourier ( u, v ) is the

value of pixel (u, v) in the binary Fourier hologram on the ith iteration. It is possible to compute this formula in an efficient way [18] by introducing the fast Fourier transform as: i i H space ( n, m ) = IFFT  H Fourier ( u, v )

(3)

where IFFT[·] represents the inverse fast Fourier transformation. For the binary hologram, i H Fourier ( u, v ) is equal to 0 or 1. It is also possible to generate Fresnel holograms by

i multiplying the term H Fourier ( u, v ) by a propagation function in Eq. (2) and Eq. (3). Both types of holograms were tested and gave results extremely similar in term of image quality. For the case of Fourier hologram, we note that each binary pixels in Fourier domain corresponds to plane waves in space domain according to angular spectrum wisdom. Thus binary pattern in Fourier domain could correspond to continuous wave distribution in space domain. In the following, we focus on Fourier hologram, but Fresnel holograms could also be used. To avoid losing information during the conversion to binary data, we use a hologram that has been back-propagated to its origin as a reference. The reference hologram is obtained by convoluting the raw complex hologram from Eq. (1) with the FZP z = − zo . This is given by:

 πΔl 2 2  j n + m2 ) exp  j (4) ( −λ z o  −λ zo  Note that as fringes diverge when propagating from an object, back propagation of the hologram to its origin localizes. This makes the DBS converge in an efficient way. i The two complex holograms H space ( n, m ) and H ref ( n, m ) are compared by computing the mean square error (MSE):

H ref ( n, m ) = H com ( n, m ) ⊗

M −1 2

N −1 2

 

MSEi =

1 M ⋅N

n=

−M −N m= 2 2

i ( H ref ( n, m ) − k ⋅ H space ( n, m ))

M −1 2

N −1 2

 

u=

−M −N v= 2 2

(5) H ref ( n, m )

2

with M −1 2

N −1 2

 

k=

n=

−M −N m= 2 2

i H ref ( n, m ) ⋅ conj  H space ( n, m ) M −1 2

N −1 2

 

u=

−M −N v= 2 2

i H space ( n, m )

2

i All pixels in the binary Fourier hologram H Fourier ( u, v ) are changed one by one and the

i MSEi between the new hologram H space ( n, m ) and the reference hologram H ref ( n, m ) is computed each time. The updated value of the binary Fourier hologram is retained if the MSE has improved, and is discarded otherwise. One iteration is completed when all the pixels of

#226260 - $15.00 USD © 2015 OSA


the hologram have been tested once. The MSE is expected to converge toward a minimum after several iterations. The flowchart in Fig. 3 illustrates the procedures discussed so far.

Fig. 3. Block diagram of the iterative direct-binary search algorithm

4. Converting the OSH of a diffusely reflective object to a binary hologram We convert the complex hologram of a diffusely reflective object recorded by OSH to a binary hologram using the DBS algorithm. First, we obtain a reference hologram by backpropagating the raw complex hologram shown in Fig. 2 to its origin according to Eq. (4). This corresponds to the back-propagation block of Fig. 3. We then search for the binary Fourier hologram H Fourier ( u , v ) that minimizes Eq. (5) using the DBS algorithm, in which the backpropagated hologram is zero-padded to 1280x800 pixels to ensure a suitable size for the SLM. The convergence speed of the algorithm is slow, because we are dealing with complex data instead of real data. The computation of MSE is restricted to the region in which the object is located (the center of the image). The output is a binary pattern, as shown in Fig. 4.

#226260 - $15.00 USD © 2015 OSA


Fig. 4. Binary hologram after application of DBS. MSE is only computed in the center of the image

To check that the algorithm converges to a binary hologram that can reconstruct the recorded object, we performed numerical reconstructions of the binary hologram computed with the DBS algorithm. The reconstruction was examined in a different plane to verify that the depth of the object was not lost after the conversion to binary data.

Fig. 5. Reconstruction of the object obtained from the binary hologram at different distances

As shown in Fig. 5, even if the reconstruction is a little noisy, the result is very similar to that obtained with the original hologram. The object is more and more distorted as it is out of focus and its size decreases as the distance to the camera increases. The original hologram contains not only the information of the plane in focus (z = 100mm), but also the out of focus information. Then, the object can be reconstructed sharply in the reference plane but the defocused field can also be visualized by reconstructing the object in another plane. This information is preserved after application of the DBS algorithm. Indeed, the MSE is computed and optimized for the reference plane, but as the error function is computed using the complex field and not just the intensity information the out of focus planes are also taken into account. As shown in Fig. 6, the optimization of the MSE in the plane in focus also induces the MSE to decrease in the defocused planes even if the optimization is less efficient when the distance to the reference increases.

#226260 - $15.00 USD © 2015 OSA


Fig. 6. Evolution of the MSE with the number of iteration for different reconstruction plane (95mm, 130 mm and 165mm)

Note that, unlike conventional digital holography, the hologram recorded by OSH is not contaminated by speckle noise. Thus, the DBS algorithm ensures the initial random binary pattern converges to a binary hologram that can accurately reconstruct the object. 5. Optical reconstruction using a digital micromirror device We used a digital micromirror device (DMD) corresponding to 1280x800 pixels with a pitch size p = 7.6 μm. The mirrors deflect the incident light at an angle of ± 12°. It has been shown that such a device can be used as an SLM to optically reconstruct a hologram [19]. As the micromirrors are inclined, the pixel pattern of the DMD acts as a blazed grating, reducing the intensity of diffraction order 0, but making other orders visible [20]. The angle between order 0 and order 1 is given by α = λ/p. In practice, to be visible, an image must be reconstructed in the area between orders 0 and 1 of the grating. In the optical reconstruction, we load the binary Fourier hologram shown in Fig. 4 on the DMD of the optical set-up illustrated in Fig. 6. A laser beam with λ = 532nm first passes through the beam expander, and then the collimated beam that is binary modulated by the DMD is projected onto the space domain by the Fourier lens using an inverse Fourier transform. The transformed hologram is reconstructed on the focal plane of the Fourier lens. Figure 7 shows the optically reconstructed binary Fourier hologram as captured by a DSLR camera (Canon DS126071). This indicates that the reconstructed object is clearly visible, and is spatially separated from the twin image and the zeroth order. We also checked that the depth of the object was preserved in the optical reconstruction by examining different planes (Fig. 8).

Fig. 7. Optical set-up for reconstructing the hologram with a DMD

#226260 - $15.00 USD © 2015 OSA


Fig. 8. Optical reconstruction of the binary hologram in the reference plane

6. Conclusion We have successfully converted a complex hologram captured by OSH into binary data by applying the DBS algorithm. Numerical and optical reconstructions have demonstrated that the real object can be obtained from the binary pattern without losing depth information. As the reference hologram used in the DBS algorithm is an off-axis hologram, the zeroth order and twin image are separated from the reconstructed object. To the best of our knowledge, this is the first time that a hologram of a real object has been reconstructed using a binary SLM. As binary data can be used to display holograms at a high rate with time multiplexing techniques, this method could potentially be employed to display large holograms captured from real objects by OSH. The fringes of a hologram that records several objects located different depth locations diverge from at each object location. This aspect deserves future research of fringe localization for improving effectiveness of binary search algorithm. A new error function could be designed, for instance by taking into account more than one reference plane, or by giving different weight to the hologram pixels. Acknowledgments This work was supported by GigaKOREA project, [GK14D0100, Development of Telecommunications Terminal with Digital Holographic Table-top Display].

#226260 - $15.00 USD © 2015 OSA
