Journal of Microscopy, Vol. 237, Pt 2 2010, pp. 111–118
doi: 10.1111/j.1365-2818.2009.03325.x
Received 27 March 2009; accepted 3 August 2009
Lagrange time delay estimation for scanning electron microscope image magnification K.S. SIM, L.W. THONG, H.Y. TING & C.P. TSO Faculty of Engineering & Technology, Multimedia University, Melaka, Malaysia
Key words. Finite impulse response, Lagrange time delay, scanning electron microscope.
Summary Interpolation techniques that are used for image magnification to obtain more useful details of the surface such as morphology and mechanical contrast usually rely on the signal information distributed around edges and areas of sharp changes and these signal information can also be used to predict missing details from the sample image. However, many of these interpolation methods tend to smooth or blur out image details around the edges. In the present study, a Lagrange time delay estimation interpolator method is proposed and this method only requires a small filter order and has no noticeable estimation bias. Comparing results with the original scanning electron microscope magnification and results of various other interpolation methods, the Lagrange time delay estimation interpolator is found to be more efficient, more robust and easier to execute. Introduction Scanning electron microscope (SEM) is one of the most powerful instruments that can image and analyse bulk specimens (Goldstein et al., 1992; Reimer, 1998). One of its important applications is image magnification through its hardware. However, over-exposure of a sample in the SEM machine will lead to image contamination, due to the constant bombardment by the electrons. Therefore, to reduce image contamination, we can apply an interpolation method on a low-magnification image to obtain the needed higher magnification. By doing so, we only need to capture a single low-magnification image, which will definitely reduce the local areas of contamination. Moreover, software magnification has the advantage over mechanical magnification, which requires careful alignment of sample location. Potentially, the software magnification can also be embedded into the SEM system. Correspondence to: Kok-Swee Sim. Tel: (606)252–3480; fax: (606)231–6552; e-mail:
[email protected] C 2009 The Authors C 2009 The Royal Microscopical Society Journal compilation
There are a few common interpolation software that are available in the literature, such as nearest neighbourhood, bilinear, bicubic and Hermite interpolation (Gonzalez & Woods, 1992; Kim & Ko, 2000; Gonzalez et al., 2004). Nevertheless, there are several drawbacks for these methods. Nearest neighbourhood, bilinear and bicubic interpolation methods are limited by the number of image pixels in each image whereas Hermite interpolation method suffers on lengthy computation time. Hence, we propose to reduce these limitations by using the Lagrange time delay estimation interpolation (LATDEI) model and to study its performance based on various images at different magnifications and noise contamination levels. The proposed method In this paper, we introduce a finite impulse response for bandlimited approximation of a fractional digital delay onto the LATDEI algorithm. In the x-direction, when delaying signal rx (t) with time shifted tD , we write the new signal as x(t) = rx (t − tD ). After converting them into discrete signals by sampling at t = kT , where T is the sampling interval and k is an integer time index, we can obtain the time delay signal, as by Laakso et al. (1996), without any lost of generality as x(k) = r x (k − D˜ ) =
∞
h i d (n)r x (k − n), − ∞ < k < +∞,
n=−∞
(1)
where hid (n) is the transfer function of time delay, and D˜ is a positive real number that can be split into an integer and fractional part as D˜ = I nt (D ) + d = tD /T .
(2)
Taking non-integer values of D˜ , we use a band-limited interpolation to approximate x(k),which lies between the two samples of rx (k − Int(D )) and rx (k − Int(D ) − 1). The d in Eq. (2) is a fractional delay of discrete signal (Hermanowics, 1992). This established fractional delay filter will be employed
112
K.S. SIM ET AL.
to develop Lagrange interpolation finite impulse response filter later. For a given set of sequential data {ki , xx (ki )}(0 ≤ i ≤ N), we see that the N- degree polynomial x(k x ) =
N
g i k xi
(3)
i =0
gives the approximation of x at the point kx , which lies in the interval [k 0 , kN ]. The coefficient{g i }can be obtained by following the Cramer’s rule (Ogata, 1967) and g i = i , where (i = 0, 1, 2, . . . , N). In order to find the coefficients, we need to solve for , which is Vandermonde’s determinant given as ⎡ ⎤ 1 k0 · · · k0k · · · k0N ⎢ ⎥ ⎢ 1 k1 · · · k1k · · · k1N ⎥ ⎢ ⎥ ⎥= = det ⎢ (k j − km ), .. .. .. ⎢ .. ⎥ . ⎢. ⎥ 0≤m≤ j ≤N . . ⎣ ⎦ (4) 1 k N · · · k kN · · · k NN and i is the determinant of a new matrix defined by replacing the i th column of the matrix by the column vector [x(k 0 ), x(x 1 ), . . . , x(kN )]T into Eq. (4). We assumed that the second-order polynomial function (N = 4) is x(k x ) = g 4 k x4 + g 3 k x3 + g 2 k x2 + g 1 k x + g 0 .
(5)
It passes through five points, namely, (n − 2)T , (n − 1)T , nT, (n + 1)T and (n + 2)T , which are equally spaced. Accordingly, by substituting (n − 2)T , (n − 1)T , nT, (n + 1)T and (n + 2)T into Eq. (5), we note that the polynomial satisfies the following linear equation: ⎡
(n + 2)4 T 4
(n + 2)3 T 3
(n + 2)2 T 2
⎢ ⎢ (n + 1)4 T 4 (n + 1)3 T 3 (n + 1)2 T 2 ⎢ ⎢ ⎢ n4 T 4 n3 T 3 n2 T 2 ⎢ ⎢ ⎢ (n − 1)4 T 4 (n − 1)3 T 3 (n − 1)2 T 2 ⎣ (n − 2)4 T 4 (n − 2)3 T 3 (n − 2)2 T 2 ⎤ ⎡ ⎤ ⎡ x(n + 2) g4 ⎥ ⎢ ⎥ ⎢ ⎢ g 3 ⎥ ⎢ x(n + 1) ⎥ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ g 2 ⎥ = ⎢ x(n) ⎥ . ×⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎢ g 1 ⎥ ⎢ x(n − 1) ⎥ ⎦ ⎣ ⎦ ⎣ x(n − 2) g0
(n + 2)T (n + 1)T nT (n − 1)T (n − 2)T
1
⎤
⎥ 1⎥ ⎥ ⎥ 1⎥ ⎥ ⎥ 1⎥ ⎦ 1
(6)
This can be written in matrix form as Z G = X.
(7)
The solution for G is written as G = Z −1 X.
(8)
Fig. 1. IC sample images taken at various magnification. (a) Image taken at 1000× magnification with horizontal field-width = 100 μm, (b) image taken at 2000× magnification with horizontal field-width = 20 μm and (c) image taken at 4000× with horizontal field-width = 20 μm.
C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
LATDEI MODEL FOR SEM IMAGE MAGNIFICATION
113
Fig. 2. IC sample images taken at 2000× magnification. (a) Original image captured from SEM with horizontal field-width = 20 μm, (b) image after nearest neighbourhood method, (c) image after bilinear method, (d) image after bicubic method, (e) image after Hermite method and (f) image after LATDEI method. C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
114
K.S. SIM ET AL.
Fig. 3. IC sample images taken at 4000× magnification. (a) Original image captured from SEM with horizontal field-width = 20 μm, (b) image after nearest neighbourhood method, (c) image after bilinear method, (d) image after bicubic method, (e) image after Hermite method and (f) image after LATDEI method.
C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
LATDEI MODEL FOR SEM IMAGE MAGNIFICATION
115
The values of g 4 , g 3 , g 2 , g 1 and g 0 are obtained by finding Z −1 (see Appendix A). The interpolated sample at time k x = (n − D˜ )T can be written as xˆ (n − D˜ ) = h 2 x(n − 2) + h 1 x(n − 1) + h 0 x(n) + h −1 x(n + 1) + h −2 x(n + 2),
(9)
1 ˜ ˜2 where D˜ is the time delay; h −2 = 24 D ( D − 1)( D˜ − 2), h −1 = 4 ˜ ˜ 6 ˜2 2 ˜ − 24 D ( D − 1)( D − 4), h 0 = 24 ( D − 4)( D˜ 2 − 1), h 1 = 4 ˜ ˜ 1 ˜ ˜2 D ( D + 1)( D˜ 2 − 4) and h 2 = 24 D ( D − 1)( D˜ + 2). − 24 For the Nth-order (with N = 2M)delay filter, a general form for the filter coefficients can be derived as
h D˜ ( j ) =
M D˜ − i . j −i i =−M
(10)
i = j
For our application, we choose N = 4 (M = 2) without losing any generality as it is then simple and easy to implement the Lagrange interpolation filter. Therefore after applying the LATDEI in the x-direction, we have x(k) = r x (k − D˜ ) =
∞
h D˜ (n)x(k − n).
(11)
n=−∞
Similarly, by applying the LATDEI in the y-direction, we have y(k) = r y (k − D˜ ) =
∞
h D˜ (n)y(k − n).
(12)
n=−∞
Results and discussion In this study, we worked on two different varieties of SEM images which are the IC (integrated circuit) images and the epoxy composite based on betel nut fibres images. We have performed experiments on various magnifications by using different interpolation methods. The methods applied are nearest neighbourhood interpolation method, bilinear interpolation method, bicubic interpolation method, Hermite interpolation method and the proposed LATDEI method. Figure 1 shows the IC images captured using the SEM from 1000× to 4000× magnification. Figure 1(a) shows the details of IC images with horizontal field-width = 100 μm. With SEM magnification function, the image is then again magnified to 2000×, as shown in Fig. 1(b). Higher magnification with 4000× is shown in Fig. 1(c). Considering the size of the image and the visual perception of the human eye, we observed that the original SEM image is captured with good quality, crisp image details and clear edges, as shown in Fig. 2(a). Figure 2(b)–(e) is obtained through the nearest neighbourhood interpolation, bilinear interpolation, bicubic interpolation and Hermite interpolation, respectively. These images are magnified from the original image of Fig. 1(a). To illustrate the LATDEI interpolation, the image
Fig. 4. Epoxy composite based on betel nut fibres sample images taken at various magnification. (a) Image taken at 1000× magnification with horizontal field-width = 30 μm, (b) image taken at 2000× magnification with horizontal field-width = 15 μm and (c) image taken at 4000× with horizontal field-width = 7.5 μm.
C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
116
K.S. SIM ET AL.
Fig. 5. Epoxy composite based on betel nut fibres sample images taken at 2000× magnification. (a) Original image captured from SEM with horizontal field-width = 15 μm, (b) image after nearest neighbourhood method, (c) image after bilinear method, (d) image after bicubic method, (e) image after Hermite method and (f) image after LATDEI method. C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
LATDEI MODEL FOR SEM IMAGE MAGNIFICATION
117
Fig. 6. Epoxy composite based on betel nut fibres sample images taken at 4000× magnification. (a) Original image captured from SEM with horizontal field-width = 7.5 μm, (b) image after nearest neighbourhood method, (c) image after bilinear method, (d) image after bicubic method, (e) image after Hermite method and (f) image after LATDEI method. C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
118
K.S. SIM ET AL.
details found in the IC chip is magnified, as shown in Fig. 2(f). It can be seen that the image obtained by using LATDEI interpolation provides a better quality and contains more details compared to other interpolation methods. In fact, we are using time delay method to predict the surrounding data that help to retrieve more information after interpolation from 2000× to 4000×, as shown in Fig. 3. The details of the two-dimensional representative of various image magnification techniques are shown in Fig. 3. In Fig. 3(f), the image produced by LATDEI is comparative clear with the nearest neighbourhood, bilinear interpolation, bicubic interpolation as well as Hermite interpolation. The LATDEI image does not have the blocky, jagged edges as the bicubic interpolation. Moreover, the edges of the image seem to be crispier and more clearly defined. This effect is apparent when the image is magnified several times compared to Figs 2 and 3. A second set of test images are shown in Fig. 4. These images of epoxy composite based on betel nut fibres are captured using SEM from 1000× to 4000× magnification. Figure 4(a) shows the details of epoxy composite images with horizontal fieldwidth = 30 μm. With SEM magnification function, the image is then again magnified to 2000×, as shown in Fig. 4(b). Higher magnification with 4000× is shown in Fig. 4(c). Figure 5(a) shows that the original SEM image is captured with crisp image details whereas Fig. 5(b–f) is obtained through the nearest neighbourhood interpolation, bilinear interpolation, bicubic interpolation, Hermite interpolation and LATDEI interpolation, respectively. These images are magnified from the original image of Fig. 4(a). Comparing LATDEI image and nearest neighbourhood image, it can be observed that LATDEI interpolation gives smoother and less jagged edges. LATDEI also shows significant improvements and less blurring compared to the other interpolation methods. Besides that, observe that the contrast in the textured region is slightly better. Even though LATDEI interpolation may not produce images as sharp as the original SEM, it is still able to magnify the images competently compared to the other conventional methods. To further illustrate the effects clearly, the same original image of Fig. 4(a) is magnified 4000×, as shown in Fig. 6. Figure 6 shows the details of various image magnification techniques varying from nearest neighbourhood, bilinear interpolation, bicubic interpolation and Hermite interpolation. Observe that nearest neighbourhood produces blocky texture, whereas LATDEI interpolation is clearly
⎡ Z
−1
⎢ ⎢ =⎢ ⎢ ⎣
− 6T1 4
1 24T 4 24T −48nT 288T 10 7
7
−12T 8 −72nT 8 +72n 2 T 8 288T 10
−48T +192nT 288T 10 7
7
192T 8 +144nT 8 −288n 2 T 8 288T 10
superior in eliminating blocky region and enhance edges. Bilinear, bicubic and Hermite interpolation give images which are less focused especially in the vicinity of the bright texture, but LATDEI is able to improve the image by applying time delay technique on the images. However, the image edges are a little blurred and smoothed out compared to the original SEM image. Nevertheless, LATDEI interpolation is still able to generate higher-magnification images proficiently. In conclusion, the experimental results of LATDEI are shown to be superior compared to other usual interpolation methods used for image magnification. The performance of the LATDEIbased estimator is analysed in terms of the image resolution. Nevertheless, the limitation of the LATDEI estimation is in determining the optimized fractional time delay and in applying estimation technique on low-magnification images with contrast varying greatly from pixel to pixel. In future, it may be possible to incorporate LATDEI into the image control function of SEM imaging software, in order to provide a suitable magnification without contaminating the viewing sample using the mechanical method. The improvement on the optimized fractional time delay may also be studied.
References Goldstein, J.L., Newburry, D.E., Echlin, P. et al. (1992) Scanning Electron Microscopy And X-ray Microanalysis: A Text For Biologist, Material Scientist And Geologist, 2nd edn. Plenum Press, New York. Gonzalez, R.C. & Woods, R.E. (1992) Digital Image Processing, 2nd edn. Addison-Wesley, Boston, Massachusetts. Gonzalez, R.C., Woods, R.E. & Steven, L.E. (2004) Digital Image Processing Using Matlab. Pearson Prentice Hall, New Jersey. Hermanowics, E. (1992) Explicit formulas for weighting coefficients of maximally flat tunable FIR delayers. Electron. Lett. 28(2), 1936–1937. Kim, H. & Ko, H. (2000) An intelligent image interpolation using cubic Hermite method. IEICE Trans. Inf. Syst. E83-D(4), 914–921. Laakso, T.L., Valimaki, V., Karjalainen, M. & Laine, U. (1996) Splitting the unit delay. IEEE Signal Process. Mag. 13(1), 30–60. Ogata, K. (1967) State Space Analysis of Control Systems, Instrumentation and Controls Series (ed. by W.S. William). Prentice Hall, U.K. Reimer, L. (1998) Scanning Electron Microscopy. Springer Series in Optical Sciences. Springer, Berlin.
Appendix A The inverse of Z of the fourth-order polynomial function:
1 4T 4
− 6T1 4
− Tn3
48T +192nT 288T 10
−360nT 8 +422n 2 T 8 288T 10
7
⎤
1 24T 4 7
192T 8 −144nT 8 −288n 2 T 8 288T 10
−24T −48nT 288T 10 7
7
−12T 8 +72nT 8 +72n 2 T 8 288T 10
−24T 9 +24nT 9 +72n 2 T 9 −48n 3 T 9 288T 10
192T 9 −384nT 9 −144n 2 T 9 +192n 3 T 9 288T 10
720nT 9 −288n 3 T 9 288T 10
−192T 9 −384nT 9 +144n 2 T 9 +192n 3 T 9 288T 10
24T 9 +24nT 9 −72n 2 T 9 −48n 3 T 9 288T 10
24nT 10 −12n 2 T 10 −24n 3 T 10 +12n 4 T 10 288T 10
−192nT 10 +192n 2 T 10 +48n 3 T 10 −48n 4 T 10 288T 10
288T 10 −360n 2 T 10 +72n 4 T 10 288T 10
−192nT 10 +192n 2 T 10 −48n 3 T 10 −48n 4 T 10 288T 10
−24nT 10 −12n 2 T 10 +24n 3 T 10 +12n 4 T 10 288T 10
⎥ ⎥ ⎥ ⎥ ⎦
C 2009 The Authors C 2009 The Royal Microscopical Society, Journal of Microscopy, 237, 111–118 Journal compilation
