Novel approach for merit function optimization in hybrid imaging system through finite impulse response method Chuan-Chung, Chang*, Yung-Lin, Chen, Kuang-Vu Chen, Hsiao-Yue Tsao and Chir-Weei Chang Electronics and Optoelectronics Research Laboratories/Industrial Technology Research Institute 195, Sec.4, Chung Hsing Rd., Chutung, Hsinchu, 310, Taiwan, R.O.C ABSTRACT Merit function with higher efficiency is helpful for lens design, especially in hybrid imaging system. Although different merit functions have been proposed in recent years, for example, Fisher information, Hilbert space angle, mean square error (MSE) based on optical transfer function, intermediate or restored image, structure similarity, correlation or statistical properties from point spread function (PSF). But it is still an unanswered question that which merit function is best for optimization in hybrid imaging system. So, a novel approach which is based on finite impulse response of hybrid imaging system is proposed. And several merit functions, blur MSE, PSF similarity, modulation transfer function (MTF) area and volume are evaluated by present method. The results show that performance of merit function is not only affected by noise, sampling ratio. But the effect of restoration filter should be also considered. Finally, compare with PSF similarity, blur MTF in area and volume; blur MSE provide much stable results in hybrid imaging system, which means it could be an optimized merit function in hybrid imaging system. Keywords: merit function, hybrid imaging system, finite impulse response, optimization
1. INTRODUCTION Hybrid imaging, which use special phase mask combine with image restoration become a popular researching topic since 1995 1. Because of hybrid imaging provides more degree of freedom between lens design and image processing, so several useful and interested applications have been realized. For example, extension in depth of field1-13, single camera ranging finder 14-16, and reduction in numbers of used lens 17. Most of researches focus phase masks itself 1,2,4,7-13, and several merit functions like blur MSE 18 , PSF similarity 19, Hilbert space angle 20, Fisher information 21,22, MTF area 23 and Strehl ratio 24, derivative in optical transfer function 25 have been presented for phase mask optimization as our best knowledge. But it is still an unanswered question that which one is best for phase masks optimization in hybrid imaging. For example, some merit functions consider optical behavior of phase mask only 19-24, but because of the effect of image restoration and electronic noise play are not considered. So it cannot provide stable optimization result. In contrast, other merit functions use metric on intermediate or final image for phase mask optimization in system level 18, but the optimization result will be quite various with tested object (image of Lena, Siemens star, USAF resolution chart or others). In this paper, we present a novel approach, which is based on concept of finite impulse response, a test object with finite impulse response is used to replace current tested object. And system MTF is calculated through Fourier transform on final image. By using the proposed method, system performance can be independent with tested object and used for comparing difference between several merit functions, blur MSE, PSF similarity, blur MTF area and volume. The rest of the paper is organized as follows. Lens, merit functions, and the present method are explained in Section 2. The results are shown in Section 3. Finally, the discussion and our future works are made in Section 4.
*
[email protected]; phone 886 3 591-8384; fax 886 3 582-9781
Novel Optical Systems Design and Optimization XV, edited by G. Groot Gregory, Arthur J. Davis, Proc. of SPIE Vol. 8487, 84870E · © 2012 SPIE CCC code: 0277-786X/12/$18 · doi: 10.1117/12.927393
Proc. of SPIE Vol. 8487 84870E-1 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
2. METHOD 2.1 Lens and sensor and other information In order to simplify difference between merit functions coming from lens structure itself, a F/4 imaging system with two commercial achromatic lens (Edmund optics, NT45-215 and NT32-321) and one phase mask with cubic type 1 is applied in simulation, and the optical layout is shown in Figure 1. The object distance equals to front focal length of first achromatic lens which generates perfect parallel light for followed phase mask. The second achromatic lens is placed in front of image sensor with distance which equals to back focal length of it. The image sensor has 768x494 pixels with pixel size 6.45x6.45 um. Sampling ratio, 1um and 6.45um are considered for finding out possible effect in merit functions. And also Gaussian noise is included, which is used for observing the possible changes in merit function in real world. NT45-215
IIr 11
NT32-321
Q
Phase mask Figure 1. The F/4 imaging system used in simulation, which is composed by two achromatic lens and one phase mask.
2.2 Merit function using for optimization In order to find better merit function in hybrid imaging system, four merit functions are used for phase mask optimization, which using for system with Extended Depth of Field (EDoF). The first merit function blur MSE, which can be calculated by using Eq. (1). Blur MSE assumes that all of intermediate image should be similar with each other, when an excellent performance in depth of field extension is achieved. The second one is PSF similarity, which is result of correlation between on focus and defocus PSF of optical system, the mathematical form is shown in Eq. (2). Higher values in PSF similarity means less difference between on focus and defocus PSF, which represent better performance of EDoF in optics. The third and fourth merit function are area and volume of blur MTF, which are calculated by using ZEMAX and MATLAB together, the concept of these two merit function is expressed in Eq. (3), which is integration of modulation transfer function for a given defocusing range. For all of these four merit functions, difference between each object distance is asked to be zero with the weighting factor one, and also strehl ratio with 5E-3 is applied for avoiding over design in phase mask, which preserve acceptable strength in MTF. Blur MSE =
1 m n [f (i, j) − f on focus (i, j)]2 ∑∑ m × n i =1 j=1
where f and f on focus are images using for comparison.
∑∑ (A
mn
PSF similarity =
m
(1)
− A )(Bmn − B )
n
⎡ 2 ⎤⎡ 2⎤ ⎢∑∑ ( Amn − A) ⎥ ⎢∑∑ (Bmn − B ) ⎥ ⎣m n ⎦⎣ m n ⎦
(2)
where A and B are the PSF using to compare the similarity, A and B are mean value of A and B. f2
Blur MTF area or volume =
∫ MTF ( f , defocus)df f1
Proc. of SPIE Vol. 8487 84870E-2 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
(3)
where f1 and f2 are given minimum and maximum spatial frequency. 2.3 Finite impulse response In order to obtain finite impulse response of hybrid imaging system, firstly, an image with black background and single white point at central position is used as object to replace image of Lena, which is widely used in image processing. Secondly, phase mask is optimized by using tested merit functions. Thirdly, intermediate images of hybrid imaging system are generated through convolution between PSF and the object. Fourthly, the intermediate images are then restored by using Wiener filter. Finally, Fourier transform is applied on the restored images to get MTF of hybrid imaging system. Ideally, better and stable performance in MTF means a superior hybrid imaging system, and the optimized merit function can be determined. The detail process flow of the study is shown in Figure 2.
ZEMAX
MATLAB
LiOptimization
Image
restoration Merit 1 Merit 2
Phase mask -x
optimization -->1
Optical
Restored
PSFs /MTEs
MTF
Merit n
Blur MSE
PSF similarity Blur MTF
Figure 2. Simulation process uses in study, merit function which using for phase mask optimization in ZEMAX firstly, and then different sampling, noise are applied for intermediate images generation in MATLAB, and followed by de-convolution through Wiener filter and FIR calculation finally.
3. RESULT In our first simulation, a hybrid imaging system with absence of noise, 1um sampling and three of object distance -40 mm, on focus and +40 mm are considered. Standard deviation (STD) for different object distance of PSF similarity is applied for judging different merit functions, the results are listed in Table 1. From Table 1, we can find out that higher strength in phase mask provide smaller STD in PSF similarity, which is consist with wavefront coding theory. And also that Fast Fourier Transform (FFT) PSF and Huygens PSF in ZEMAX have same results in PSF similarity.
Proc. of SPIE Vol. 8487 84870E-3 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
Table 1. Result of PSF similarity and its standard deviation of object distance in compared merit functions with 1um sampling and absence of noise. On focus PSF similarity - 40 mm (FFT) + 40 mm On focus PSF similarity - 40 mm (Huygens) + 40 mm Strength of phase mask Standard deviation (FFT) Standard deviation (Huygens)
Blur MSE 1.00 0.14 0.30 1.00 0.14 0.30 1.035E-4 0.457384 0.457384
PSF similarity 1.00 0.19 0.38 1.00 0.20 0.38 1.380E-4 0.423596 0.423596
Blur MTF area 1.00 0.18 0.36 1.00 0.18 0.36 1.290E-4 0.430968 0.430968
Blur MTF volume 1.00 0.21 0.39 1.00 0.20 0.39 1.440E-4 0.414045 0.414045
In our second simulation, difference with first one, Wiener filter is applied for simulating entire process in hybrid imaging system. The results are listed in Table 2. From Table 2, we can observe that merit function of blur MTF area provide largest restored MTF area, which also has best balance in restored MTF between each object distance. Compare with results listed in Table 1, we know that consideration between optics and imaging restoration at the same time will necessary when optimization of merit function is required. In our third simulation, effect of noise and finite pixel sampling in hybrid imaging system are considered further. White Gaussian noise and re-sampling on convoluted intermediate images are carried out in MATLAB. Finally, result of merit functions with different optimization criteria restored MTF area and PSF similarity under two levels of signal to noise ratio (SNR), 100dB and 30dB listed in Table 3 and 4 respectively. Figure 3 to 6 shows the restored images and MTF curve of each merit function for different object distance and SNR. Table 2. Results of restored MTF area and total restored MTF area for all four merit functions with 1um sampling and absence of noise. On focus Restored MTF area - 40 mm + 40 mm Strength of phase mask Total restored MTF area Standard deviation of restored MTF
Blur MSE 92.3190 66.6997 76.9989 1.035E-4 236.0176
PSF similarity 90.0635 73.1399 80.9489 1.380E-4 244.1523
Blur MTF area 88.7620 76.7970 86.1786 1.290E-4 251.7376
Blur MTF volume 89.5498 66.2978 66.9089 1.440E-4 222.7565
12.89139
8.470189
6.296158
13.25166
Table 3. Results of total restored MTF area and PSF similarity for all four merit functions with sampling equals to pixel size and SNR 100dB.
Strength of phase mask Total restored MTF area On focus PSF similarity (Huygens) - 40 mm +40 mm Standard deviation (Huygens)
Blur MSE
PSF similarity
Blur MTF area
1.035E-4 253.47
1.380E-4 172.89
1.290E-4 222.16
Blur MTF volume 1.440E-4 177.02
1.00
1.00
1.00
1.00
0.17 0.52 0.416693
0.16 0.35 0.440492
0.25 0.37 0.402865
0.14 0.35 0.448367
Proc. of SPIE Vol. 8487 84870E-4 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
Object distance on focus
+40 mm
-40 mm
Blur MSE
PSF similarity
Blur MTF area
MI
Blur MTF volume Figure 3. Restored image of finite impulse response object in hybrid imaging system designed by each merit function with 6.45 um sampling and SNR 100dB.
Restored MTF of different merit function at object distance -40cm
Restored MTF of different merit function at object distance 40cm
1.0
1.0
0.9
0.9
0.8
0.8
0.7
0.7
ú 0.5
+Blur MSE -_-PSF similarity -t Blur MTF area
á 0.4
+Blur MTF volume
u 0.6
á
Blur MSE
--PSF similarity 0.5
- !~Blur MTF area
0.4
+Blur MTF volume
0.3
0.3
0.2
0.2
0.1
0.1
0.0
41114111411111r
0.6
__
0.0 O
10
20
30
40
50
60
70
8O
90
100
110
0
10
20
30
Frequency (Ip /mm)
40
50
60
70
80
90
100
110
Frequency (Ip /mm)
(a) (b) Figure 4. Restored MTF curve of different merit function for 1um sampling and 100dB SNR, (a) object distance equals to +40 mm respect to on focus position; (b) object distance equals to -40 mm respect to on focus position.
Table 4. Results of total restored MTF area for all four merit functions with sampling equals to pixel size and SNR 30dB.
Strength of phase mask Total restored MTF area
Blur MSE
PSF similarity
Blur MTF area
1.035E-4 175.39
1.380E-4 124.41
1.290E-4 148.75
Blur MTF volume 1.440E-4 111.99
For easier comparison difference between analyzed merit functions in the study, the results in our first to third simulation are further plotted in Figure 7.
Proc. of SPIE Vol. 8487 84870E-5 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
Object distance on focus
+40 mm
-40 mm
Blur MSE
PSF similarity
Blur MTF area
Blur MTF volume Figure 5. Restored image of finite impulse response object in hybrid imaging system designed by each merit function with 6.45 um sampling and SNR 30dB.
Restored MTF of different merit function at anfocus position 1.0
0.9 0.8 0.7
+ Blur MSE
0.6
t F PSF siml larity
0
-r
0.5
Blur MTF area
+ Blur MTF volume
le' 0.4 0.3 0.2 0.1
0.0
0
20
10
30
40
50
60
70
80
90
100
110
Frequency (Ip /mm)
(a) Restored MTF of different merit function at object distance -90cm
Restored MTF of different merit function at object distance -40cm 1.0
1.0
0.9
0.9
0.8
0.8 0.7
0.7 0.6
u 0.5 ó
á
0.4
N, -.4its.
+ Blur MSE - 11I-PSF similarity
- A-Blur MTF area
--
+ Blur MTF volume
Blur MSE
î 0.6
{ -PSF similarity
0.5
- A-Blur MTF area
0.4
- 0-Blur MTF volume
0.3
0.3
0.2
0.2
O1
0.1
0.0
0.0 O
10
20
30
40
50
60
70
80
90
100
110
0
10
20
30
Frequency (Ip /mm)
40 50 60 70 Frequency (Ip /mm)
80
90
100
110
(b) (c) Figure 6. Restored MTF curve of different merit function for 1um sampling and 30dB SNR, (a) object distance equals to on focus position; (b) object distance shift +40 mm respect to on focus; (c) object distance shift -40 mm respect to on focus.
Proc. of SPIE Vol. 8487 84870E-6 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
Blur MSE
OS >n, ray
ID Restored MTF area_lum
BM MTF area
Blur MTF volume
D Restored MTF area_CCD_100dB
D Restored MTF area_CCD_30dB
Figure 7. Normalized restored MTF area for all of four merit functions by present FIR method.
4. DISCUSSION In the study, we confirm that result of merit function, PSF similarity is agree with wavefront coding theory under different sampling condition with absence or few of noise. But it will be failed when more of noise exists in hybrid imaging system. The reason of this should be that effect of image restoration is not included in PSF similarity. Blur MTF area and volume have similar behavior; both of them will provide worse optimization in hybrid imaging system when more of noise and sampling satisfy with pixel size are considered. Compare with PSF similarity, blur MTF area and volume, blur MSE provide better stable result when noise and sampling satisfy with pixel size are considered. And also optimization using blur MSE provide smaller coefficient in phase mask. It also represents higher Strehl ratio and less of noise in hybrid imaging system will be obtained. We also notice that although blur MSE cannot keep same restored MTF curve at different object distance, but it provide maximum total MTF area for analyzed object distance. By using present method in the study, we compare the difference for several merit functions, which include blur MSE, PSF similarity, blur MTF area and blur MTF volume. The result shows that blur MSE can be used as better merit function for designing a hybrid imaging system. Comparison with other merit functions, like Fisher information, Hilbert space angle will be our following work. From the results of restored MTF curve for each merit functions, we think it maybe another useful merit function when interested spatial frequency or its range is given, and co-optimization with restoration filter is realized at the same time, which will be our future target.
REFERENCES [1] Dowski, E. R. and Cathey, W. T., “Extended depth of field through wave-front coding,” Applied Optics 34(11), 1859-1866 (1995). [2] Chi, W. and George, N., “Electronic imaging using a logarithmic,” Optics Letters 26(12), 875-877 (2001). [3] Chen, X., Bakin, D., Liu, C. and George, N., “Optics optimization in high-resolution imaging module with extended depth of field,” Proc. SPIE 7061, 706103 (2008). [4] Mouroulis, P., “Depth of field extension with spherical optics,” Optics Express 16(17), 12995-13004 (2008). [5] Robinson, M. D., Feng, G. and Stork, D. G., "Spherical coded imagers: Improving lens speed, depth-of-field and manufacturing yield through enhanced spherical aberration and compensating image processing," Proc. SPIE 7429, 74290M (2009).
Proc. of SPIE Vol. 8487 84870E-7 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
[6] Harvey, A. R., Muyo, G. and Demenikov, M., "The principles and roles of hybrid optical/digital codecs in imaging," Proc. SPIE 7113, 71130D (2008). [7] Zhao, H. and Li, Y., "Optimized logarithmic phase masks used to generate defocus invariant modulation transfer function for wavefront coding system," Optics Letters 35(15), 2630-2632 (2010). [8] Chen, S., Fan, Z., Xu, Z., Zuo, B., Wang, S., and Xiao, H., "Wavefront coding technique for controlling thermal defocus aberration in an infrared imaging system," Optics Letters 36(16), 3021-3023 (2011). [9] Zhao, T., Ye, Z., Zhang, W., Huang, W. and Yu, F., "Field depth extension of 2D barcode scanner based on wavefront coding and projection algorithm," Proc. SPIE 6837, 683711 (2007). [10] Sung, H. Y., Chen, P. C., Chang, C. C., Chang, C. W., Yang, S. S. and Chang, H. , "Mobile phone imaging module with extended depth of focus based on axial irradiance equalization phase coding," Proc. SPIE 7876, 787606 (2011). [11] Zhao, H. and Li, Y., "Performance of an improved logarithmic phase mask with optimized parameters in a wavefront-coding system," Applied Optics 49(2), 229-238 (2010). [12] Zhao, H. and Li, Y., "Optimized sinusoidal phase mask to extend the depth of field of an incoherent imaging system," Optics Letters 35(2), 267-269 (2010). [13] Zhou, F., Li, G., Zhang, H., and Wang, D., "Rational phase mask to extend the depth of field in optical-digital hybrid imaging systems," Optics Letters 34(3), 380-382 (2009). [14] Dowski, E. R. and Cathey, W. T., “Single-lens single-image incoherent passive-ranging systems,” Applied Optics 33 (29), 6762-6773 (1994). [15] Chang, C. C., Chen, Y. L., Chang, C. W. and Lee, C. C., "Depth perception with a rotationally symmetric coded camera," Proc. SPIE 7429, 742905 (2009). [16] Chen, Y. L., Chang, C. C., Ludovic Angot, Chang, C. W. and Tien, C. H., "Single-shot depth camera lens design optimization based on a blur metric," Proc. SPIE 7787, 77870A (2010). [17] Chang, C. W., Chen, Y. L., Chang, C. C. and Chen, P. C., "Phase coded optics for computational imaging systems," Proc. SPIE 7723, 772317 (2010). [18] Chang, C. C., Chen, Y. L., Chen, K. V., Tsao, H. Y., Chang, C. W., Chen, P. C. and Chang, H., "Fidelity tolerance analysis for computational imaging system," Proc. SPIE 7876, 78760R (2011). [19] Chang, C. C. and Lee, C. C., "Effect of surface manufacturing error of cubic phase mask in wavefront coding system," Optics Express 17(8), 6245-6251 (2009). [20] Sherif, S. S., Cathey, W. T., and Dowski, E. R., "Phase Plate to Extend the Depth of Field of Incoherent Hybrid Imaging Systems," Applied Optics 43(13), 2709-2721 (2004). [21] Prasad, S., Torgersen, T. C., Pauca, V. P., Plemmons, R. J. and Gracht, J., "Engineering the pupil phase to improve image quality," Proc. SPIE 5108, 1 (2003). [22] Liu, Q., Zhao, T., Chen, Y., Zhang, W., and Yu, F., "Pupil design based on Fisher information optimization to extend field depth in practical optical system," Chin. Optics Letters 8(2), 159-161 (2010). [23] Chen, S., Fan, Z., Chang, H., and Xu, Z, "Nonaxial Strehl ratio of wavefront coding systems with a cubic phase mask," Applied Optics 50(19), 3337-3345 (2011). [24] Bagheri, S., Silveira, P., Narayanswamy, R. and Daniela, P., "Design and optimization of the cubic-phase pupil for the extension of the depth of field of task-based imaging systems," Proc. SPIE 6311, 63110R (2006). [25] Barwick, D. S., "Efficient metric for pupil-phase engineering," Applied Optics 46(29), 7258-7261 (2007).
Proc. of SPIE Vol. 8487 84870E-8 Downloaded From: http://proceedings.spiedigitallibrary.org/ on 04/25/2013 Terms of Use: http://spiedl.org/terms
