Digital Decoding Design for Phase Coded Imaging System Po-Chang Chen*, Chih-Hao Liu, Chir-Weei Chang, Chuan-Chung Chang, Ludovic Angot Optoelectronics Research Laboratories, Industrial Technology Research Institute Research 195, Sec. 4, Chung Hsing Rd., Chutung, Hsinchu, Taiwan 31040. R.O.C. ABSTRACT This paper develops a digital decoding design for the imaging system with phase coded lens. The phase coded lens is employed to extend the depth of filed (DoF), and the proposed design is used to restore the special-purpose blur caused by the lens. Since in practice the imaging system inevitably contains manufacturing inaccuracy, it is often difficult to obtain precise point spread function (PSF) for image restoration. To deal with this problem, we develop a flow for designing filters without PSF information. The imaging system first takes a shot of a well-designed test chart to have a blur image of the chart. This blur image is then corrected by using the perspective transformation. We use both of the image of the test chart and the corrected blur image to calculate a minimum mean square error (MMSE) filter, so that the blur image processed by the filter can be very alike to the test chart image. The filter is applied to other images captured by the imaging system in order to verify its effectiveness in reducing the blur and for showing the capability of extending the DoF of the integrated system. Keywords: phase coding, extended depth of field, computational imaging, digital decoding, image restoration, minimum mean square error filter
1. INTRODUCTION In recent years, many researches have been devoted to the design and development of extended DoF (EDoF) imaging systems [1-5]. The EDoF imaging systems possess an advantage capable of being implemented with fixed lens, and hence their sizes can keep compact and small. The way to achieve the goal of EDoF is to use a so-called computational imaging technique (CIT) [1-5]. The CIT is inherently an integrated design, which employs phase coding in optical design and digital decoding in image signal processor. The phase coding is utilized to substantially increase the DoF of the imaging system by means of reducing the PSF variance in the range near the focal plane, but the PSF might be in a special form of a big size. Such PSF often result in serious image blur. The digital decoding (also referred to image restoration, de-convolution or software lens compensation) is thus applied to restore the blurry image. A typical approach of phase coding is the wavefront coding method proposed by Dowski and Cathey [1,2]. By introducing a cubic phase mask into the lens design, the point spread function (PSF) of the lens will be more insensitive to defocus compared with the conventional lens. The restoration of blur by PSF is a challenging problem [6-8]. It is known that the zeros of the optical transfer functions of the imaging systems and the noises may strongly affect the restoration performance. If in spite of the zeros, a restoration filter should be able to accommodate the needs of de-convolution and de-noise. The Wiener filter has been widely used in literatures [3-5,8]; however, it requires precise information of PSF for filter design and the regulation parameter [8] to avoid boosting of high-frequency noise is difficult to determine in practice. In this paper, we develop a flow for designing restoration filters with no need of the optical PSF information. The imaging system first captures an image of a well-designed test chart. Due to the special-form PSF of the coded lens, this image is blurred. The blur image is then corrected by using the perspective transformation. We use both the original image of the test chart and the corrected blur image to calculate a MMSE filter, so that the blur image processed by the restoration filter can be very alike to the test chart image. To verify the effectiveness of the proposed design, experiments are conducted in a personal computer (PC) based platform with wavefront-coded imaging system and image restoration software. A demo chart obliquely placed is used for the observation of the EDoF capability of the integrated system. *
[email protected]; phone 886-3-5918387; fax 886-3-5917531
Applications of Digital Image Processing XXXII, edited by Andrew G. Tescher, Proc. of SPIE Vol. 7443, 74431A · © 2009 SPIE · CCC code: 0277-786X/09/$18 · doi: 10.1117/12.825459
Proc. of SPIE Vol. 7443 74431A-1
2. DIGITAL DECODING DESIGN In this section, we first model the optical channel and then derive the MMSE filter based on the channel model. Through the derivations, we can obtain the algorithm of computing the filter coefficients. A flow is then suggested to design and to apply the filter. 2.1 Channel model The linear shift-invariant (LSI) channel model is employed in this paper. If we define I and B as the input and output images of the channel respectively, then we have B = I ∗H + N
(1)
where H is a point-spread function and N is the additive noise. The channel model can be plotted as Fig. 1(a). And the discrete channel model can be derived from equation (1) to be P
Q
B (i, j ) = ∑∑ I (i + k , j + l ) H (k , l ) + N (i, j ) k =1 l =1
where P and Q are dimensions of H .
(a)
(b) Fig. 1. Block diagrams. (a) LSI channel model; (b) Optical channel with restoration filter
Proc. of SPIE Vol. 7443 74431A-2
(2)
2.2 MMSE filter Fig 1(b) is the optical channel with restoration. To restore the blur image B , a MMSE filter W is designed. Let Iˆ be the output of the filter and then we have M
N
Iˆ(i, j ) = ∑∑ B(i + k , j + l )W (k , l )
(3)
k =1 l =1
where M and N are dimensions of the filter. The Iˆ can be also treated as the estimate of the original image I . What we expect is to have Iˆ → I , and hence the filter should be able to deal with the effect of the PSF on B with good care to the noise at the same time. In order to derive the MMSE filter, a mean square error is defined as follows
{(
J = E I (i, j ) − Iˆ(i, j )
{
)} 2
} {
}
= E { I 2 (i, j )} − 2 E I (i, j ) Iˆ(i, j ) + E Iˆ 2 (i, j )
(4)
where E {} ⋅ denotes the expectation operator. Taking the partial derivative of J with respect to W (k , l ) gives m n ∂J = −2 E { I (i, j ) B (i + k , j + l )} +2∑∑ E { B (i + p, j + q) B(i + k , j + l )}W ( p, q) ∂W (k , l ) p =1 q =1
(5)
for k = 1, L , M and l = 1, L , N . If we define the autocorrelation RBB and cross-correlation RIB as RBB (k − p, l − q ) = E { B (i + p, j + q) B (i + k , j + l )}
(6)
RIB (k , l ) = E { I (i, j ) B (i + k , j + l )}
(7)
then, equation (5) can be further derived to be m n ∂J = −2 RIB (k , l )+2∑∑ RBB (k − p, l − q)W ( p, q) ∂Wˆ (k , l ) p =1 q =1
(8)
For k = 1,L , M and l = 1, L , N . The minimum of J can be found by setting equation (8) to zero, and this gives m
n
RIB (k , l ) = ∑∑ RBB (k − p, l − q)W ( p, q)
(9)
p =1 q =1
After some straightforward derivations, it is easy to rewrite equation (9) as rIB = R BB w
(10)
where rIB and w are vectors composed of RIB and W respectively, and R BB is a square matrix consisting of RIB . For simplicity, we omit the details of rIB , w and R BB . With equation (10), we can solve the filter coefficients as
w = R −BB1 rIB
(11)
Equations (6), (7) and (11) serve as a kernel to compute the filter coefficients. When the original image I and blur image B are known, the filter can be calculated. Besides, the derivations above proved that image B processed by the filter W will have MMSE with image I .
2.3 Filter design flow A plot to illustrate the design flow is in Fig. 2. We design a test pattern containing a pseudo-random pattern at the center surrounded by four reference marks. The pseudo-random pattern is basically for filter computation, while the reference marks are used to provide reference position information. The phase-coded imaging system captures an image of the test pattern placed at the focal plane to obtain a blurred image. The pattern recognition technique [9] is then applied to find
Proc. of SPIE Vol. 7443 74431A-3
out the positions of the reference marks in the blurred image, and the perspective transformation [9] is used to correct the spatial errors and distortion of the blurred image according to the reference positions provided by the known test pattern and the positions of the reference marks detected. With the test pattern image and the corrected blurred image, we can calculate an MMSE filter based on equations (6) (7), and (11). Since the filter is designed for restoration of the image at the focal plane, it is able to recover the blur induced by the PSF therein. Furthermore, if the PSF of the lens is almost invariant within a depth range of interest, then the restoration performance will be consistent over the depth range. The application of the filter designated is shown in Fig. 3, and it is used to process the blurred object image.
Fig. 2. Design flow of MMSE filter
Fig. 3. Image restoration processing
Proc. of SPIE Vol. 7443 74431A-4
Fig. 4. Experiment platform
3. EXPERIMENT RESULTS The experiments are performed here to verify the effectiveness of the proposed design. Fig. 4 shows a diagram of the experiment platform. In the platform, we employ a wavefront-coded lens, a charge-coupled device (CCD) and a personal computer. The personal computer is for retrieving images from the CCD and for processing the images. A demo chart is placed obliquely, so as to observe the DoFs from the captured images of the demo chart.
3.1 Wave-front coded lens The parameters of a conventional lens are respectively: f number=4 and aperture stop diameter=10mm. A cubic phase mask is employed in the conventional lens to enlarge the DoF. The total phase deviation of the cubic phase mask [1] is α = 150 . Suppose that both of the coded lens and its conventional counterpart are focusing at object distance (OD) 200mm from the imaging system, and then their modulation transfer functions (MTFs) and PSFs at ODs 200mm, 230mm and 260mm can be respectively plotted in Fig. 5 and 6. Form the figures, it is obvious that the coded lens has much better robustness against defocus than the conventional one, and its PSFs and MTFs at different ODs keep good similarity among each other.
3.2 Experiment results A demo chart as shown in Fig. 7(a) is of A4 size and composed of geometrical patterns. Each geometrical pattern has roughly 10mm in width. In Fig. 7(a), the near OD and far OD respectively at the left and right of the image are 175mm and 380mm. Fig. 7(a)-7(d) are experiment results. We can observe that a large portion of the image with conventional lens (Fig. 7(a)) is blurred due to serious defocus. The DoF of the traditional lens is about 10mm. Fig. 7 (b) is the image with coded lens, and the special-purpose blur can be seen from the image. Fig 7(c) is the restored image with the MMSE filter. Almost all the geometrical patterns in the restored image can be visibly recognized. Hence, the DoF of the integrated system is extended with an estimate improvement of 5 times compared with the conventional lens. Further applying the medium filter to the restored image (Fig 7(c)) gives us Fig. 7 (d), and significant noise reduction can be seen.
4. CONCLUSIONS This paper developed a flow to design restoration filters for phase-coded imaging systems. The MMSE filter is employed to recovery the special-purpose blur added by the coded lens. A test chart is suggested as the sample input of the imaging system. From the experiment results, we see the EDoF performance provided by the integrated system (wave-front coded lens + image restoration). This confirms the efficacy of the joint design of optics and digital image processing.
Proc. of SPIE Vol. 7443 74431A-5
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 5. MTF and PSF of wavefront-coded lens. (a)PSF at 200mm; (b)MTF at 200mm; (c)PSF at 230mm; (d) MTF at 230mm; (e)PSF at 260mm; (f)MTF at 260mm
Proc. of SPIE Vol. 7443 74431A-6
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 6. MTF and PSF of conventional lens. (a)PSF at 200mm; (b)MTF at 200mm; (c)PSF at 230mm; (d) MTF at 230mm; (e)PSF at 260mm; (f)MTF at 260mm
Proc. of SPIE Vol. 7443 74431A-7
(a)
(b)
(c) ()
(d) Fig. 7. Experiment results. (a)Image with conventional lens; (b)Image with coded lens; (c)Restored image; (d)Restored image with noise reduction.
Proc. of SPIE Vol. 7443 74431A-8
ACKNOWLEDGEMENTS We would like to thank Mr. Yung-Lin Chen of Industrial Technology Research Institute for help with this paper.
REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9]
E. R. Dowski, and W. T. Cathey, "Extended depth of field through wavefront coding," Applied Optics 34(11), 18591866 (1995). W. T. Cathey and E. R. Dowski, "New paradigm for imaging systems," Applied Optics 41(29), 6080-6092 (2002). C. W. Chang and Y. L. Chen, "Using liquid lens in wavefront coded imagining system," Proc. SPIE 7061, 70610O.1-70610O.8 (2008). H. Y. Sung, S. S. Yang and H. Chang, "Design of mobile phone lens with extended depth of field based on pointspread function focus invariance," Proc. SPIE 7061, 706107.1-706107.11 (2008). H. Y. Sung, S. S. Yang and H. Chang, "Software lens compensation applied to athermalization of infrared imaging systems," Optical Review 16(3), 313-317 (2009). D. Kundur and D. Hatzinakos, "Blind image deconvolution," IEEE Signal Processing Magazine 13(3), 43-64 (1996). H. Kaufman and A. M. Tekalp, "Survey of estimation techniques in image restoration," IEEE Control Systems Magazine 11(1), 16-24 (1991). H. C. Andrews and B. R. Hunt, [Digital Image Restoration], Prentice-Hall, Englewood Cliffs, New Jersey (1977). R. C. Gonzalez and R. E. Woods, [Digital Image Processing 2nd Ed.], Prentice-Hall, Upper Saddle River, New Jersey (2002).
Proc. of SPIE Vol. 7443 74431A-9
