2010 International Conference on Pattern Recognition
Robust Fourier-Based Image Alignment with Gradient Complex Image
Hong-Ren Su and Shang-Hong Lai
Ya-Hui Tsai
Department of Computer Science National Tsing Hua University Hsinchu, Taiwan e-mail:
[email protected],
[email protected]
Mechanical and Systems Research Laboratories Industrial Technology Research Institute Hsinchu, Taiwan e-mail:
[email protected]
Abstract—The paper proposes a robust image alignment framework based on Fourier transform of a gradient complex image. The proposed Fourier-based algorithm can handle translation, rotation, and scaling, and it is robust against noise and non-uniform illumination. The proposed alignment algorithm is further extended to work under occlusion by partitioning the template and performing the Fourier-based alignment for all partitioned sub-templates in a voting framework. Our experiments show superior alignment results by using the proposed robust Fourier-based alignment over the previous related methods.
II.
FOURIER-BASED ALIGNMENT ALGORITHM USING GRADIENT COMPLEX IMAGE
Instead of taking the Fourier transform of the original image in the previous Fourier-based registration methods [2] [7], we take the Fourier transform of the gradient complex image in this work. The gradient complex image, denoted by g(x,y), of a 2D image f(x,y) is defined as: g ( x, y )
Keywords-image alignment; Fourier transform; gradient complex;
I.
where
INTRODUCTION I x ( x, y )
Automated visual inspection usually requires a precise and fast image alignment technique as the first step. Many image alignment methods have been proposed for this purpose in the past [1]. Compared with other image registration techniques [2] in spatial domain, such as pixelbased methods, correlation methods and feature-based methods, Fourier-based methods, which estimate the alignment parameters in the frequency domain, usually have lower computational complexity, and thus are suitable for industrial inspection demand. However, earlier Fourier-based methods are usually less accurate with limited range of alignment [3]. Recently, modified approaches based on Fourier transform [4], like log-polar method [5], multi-layer Fourier transform (MLFFT) [6] and phase-only-correlation (POC) [7, 8, 9], have been proposed to improve the registration precision and the alignment range. For practical image alignment in industrial inspection, noise and illumination variation will influence the alignment accuracy. In addition, object occlusion can significantly reduce the registration accuracy for Fourier-based alignment methods. In this paper, we propose a robust Fourier-based image alignment method by using the gradient complex image. The proposed method is proved to be robust against noise and non-uniform illumination changes through experiments. Furthermore, the proposed alignment method is extended to handle occlusion by partitioning the template and performing the Fourier-based alignment for all partitioned sub-templates in a voting framework.
1051-4651/10 $26.00 © 2010 IEEE DOI 10.1109/ICPR.2010.582
I x ( x, y ) iI y ( x, y )
wf ( x, y ) ; I y ( x, y ) wx
wf ( x, y ) wy
A. Image alignment for 2D translation only Assume two images, f1(x,y) and f2(x,y), are displaced by (x0,y0), i.e. f 2 ( x, y )
f 1 ( x x0 , y y 0 )
The relationship between the corresponding gradient complex images, denoted by g1(x,y) and g2(x,y), is given by g 2 ( x, y )
g1 ( x x0 , y y0 )
Their corresponding Fourier spectrums are related by G2 ([ ,K ) e j 2S ([x0 Ky0 )G1 ([ ,K )
The cross-power-spectrum of g1 and g2 is defined by cps
g1 , g 2
([ ,K )
G1 ([ ,K )G2* ([ ,K ) G1 ([ ,K )G2* ([ ,K )
e j 2S ([x0 Ky0 )
where G* is the complex conjugate of G. The phase of the cross-power spectrum equals to the phase difference between the two images f1(x,y) and f2(x,y) [7]. (x0,y0) can be calculated by the 2370 2382 2378
( x0 , y0 ) arg max(real( IFT (cps g1 , g2 ([ ,K ))))
displacement (d, © 0) can be determined from the crosspower spectrum, i.e.
where IFT denotes the inverse Fourier transform.
(d ,T 0 ) arg max(real ( IFT (cpsL1 , L2 ([ ,K ))))
B. Image alignment for rotation and scaling Assume two images f1(x,y) and f2(x,y) are related with rotation and scaling transformation, i.e.
In our implementation, fast Fourier transform (FFT) is used to calculate the Fourier spectrums and two steps are required to register images in the proposed algorithm. The first step is the estimation the rotation and scaling, and the second step is to estimate the image translation. We also use the MLFFT [6] to enhance the Fourier spectrum information in order to obtain high accuracy in recovering large scale factors and large rotation angles. The procedure of the proposed Fourier-based alignment is as follows: 1) Calculate the gradient complex images g1 and g2 for the input images f1 and f2 (Eq. 10). 2) Calculate the MLFFT magnitudes L1 and L2 from g1 and g2 (Eq. 11 and 13). 3) Calculate the log-polar Fourier transform magnitude spectrums of L1 and L2 (Eq. 14). 4) Determine the scale factor and the rotation angle from the cross-power spectrum of L1 and L2 (Eq. 16). 5) Apply the scale factor and the rotation angle to transform f1 and obtain the corrected image f1’. 6) Calculate the gradient complex image g1’ for f1’ (Eq. 4). 7) Determine the translation by the cross-power spectrum of g1’ and g2 (Eq. 5, 6, 7).
f1 ( x' , y ' )
f 2 ( x, y )
where x' a ( x cos T 0 y sin T 0 ) x0
y ' a ( x sin T 0 y cos T 0 ) y0
and ©0 is the rotation angle, and a is the scaling factor. The relationship of the corresponding gradient complex images, g1(x,y) and g2(x,y), is given by g 2 ( x, y ) aeiT 0 g1 ( x' , y ' )
Their corresponding Fourier spectrums are G2 ([ ,K )
1 i ([t x Kt y T 0 ) e u a
G1 (
[ cosT 0 K sin T 0 [ sin T 0 K cosT 0 a
,
a
)
where tx
III.
VOTING
x0 cos T 0 y0 sin T 0 x0 sin T 0 y0 cos T 0
ty
OCCLUSION HANDLING WITH MULTIPLE WINDOW
Let L1 and L2 be the magnitudes of G1 and G2 (Ignoring the 1/a). L2 ([ ,K ) Figure 1. (a) The reference image and (b) 3-by-3 partitions of gradient complex image
L1 (([ cos T 0 K sin T 0 ) / a, ([ sin T 0 K cos T 0 ) / a)
In order to extend the above Fourier-based alignment algorithm, to handle the occlusion problem, we first partition the reference image, as depicted in Figure 1, and apply the above alignment method for each of the partitioned block. The average gradient amplitude for each block is used to check if the content in this block is rich enough for determining the alignment. The yellow cross in Figure 1(b) means that the average gradient amplitude is too small to be used for alignment. If a block is used, the above Fourierbased alignment is applied to estimate the translation, rotation, and scaling from a sliding window in the target image (figure 3). After the alignment for all the blocks in the
By using the log-polar transform similar to [3], we can simplify Eq. (13) to L2 ( r , T )
L1 ( r d , T T 0 )
where
[
U cos T ;K
U sin T r
log U ; d
log a
Eq. 14 indicates that the rotation and scaling of images can be transformed into the 2D translation problem, thus the
2379 2383 2371
V.
sliding windows, the locations of the template are determined from the voting map for the target image computed from the alignment of the partitioned blocks in the target image. Figure 3 depicts an example of the voting map.
EXPERIMENTAL RESULTS
The experiment is divided into three parts. At first, we compare the performance of different Fourier-based registration methods, including the traditional FFT (tFFT) [2], MLFFT [6], the proposed gradient complex image (GCI) approach. The Lena image corrupted by additive and multiplicative Gaussian noises is used in our experiments. The second experiment is to detect objects from the target images under different conditions. The third experiment is to detect objects with canny edge image (CEI), gradient amplitude image (GAI), and GCI. We use Eq. 17 to calculate the accuracy of different methods.
Figure 2. The center voting (blue dot) from a small part in reference (left) to the target (right) after registration.
'T
The aligned locations of all valid sub-blocks are included into the voting on the target image for all matched sliding windows to produce the voting map, which indicates the possibility of a template appearing at each location in the target image. The larger value at a location in voting map indicates the higher possibility of a template located at this location. Finally, an adaptive threshold method [10] is applied on the voting map to determine the locations of the template in the target image. From the searched locations of the template, the corresponding local windows on the target image are determined and registered with the whole template by using the proposed Fourier method to refine the alignment results.
T 'T , ^s,r,x,y` T
where T contains the transformation parameters, s is the scale, r is the rotation, and (x,y) is the translation. T ' and T denote the target and reference parameter vectors, respectively.
(a) Figure 5.
(b)
(c)
(d)
(e)
(a) The original image, and (b)-(e) the cropped images with different transformation parameters.
Figure 3. The voting map on target image. Figure 6. The Lena image multiplied with a non-uniform field.
IV.
IMAGE DETECTION
TABLE 1
In practical image alignment, the template is usually much smaller than the target image. To find the template in the target image, we apply the proposed Fourier-based alignment in a sliding window framework. Since the Fourierbased method can well align the template even when the translation, rotation, and scaling between the template and the sliding window are large, we can employ sparsely sampled sliding windows in the target image, thus the total number of Fourier-based alignment is well controlled.
Target image
THE RESULTS OF FFT-BASED REGISTRATION
True (s, r) (x, y)
Fig5 (1.00, 0.00) (b) (0, 0) Fig5 (1.00, 80.00) (c) (0, 0) Fig5 (1.00, 80.00) (d) (-50, -40) Fig5 (2.00, 80.00) (e) (-50, -40)
Figure 4. A small number of sliding windows is used on the target image.
2380 2384 2372
tFFT (s, r) (x, y) (0.02, 0.00) (0,0) (0.02, -2.66) (2,3) (0.02, -3.36) (-4, 3) (0.04, -6.17) (-8, 5)
MLFFT (s, r) (x, y) (0.02, 0.00) (0,0) (0.02,-0.55) (0,1) (0.02,-0.55) (2,-1) (0.04,-0.55) (3,-3)
GCI (s, r) (x, y) (0.02, 0.00) (0, 0) (0.02, 0.15) (0, 0) (0.02, -0.55) (0, 0) (0.04, -0.55) (1, -1)
TABLE 2
Target image Fig5 (b) Fig5 (c) Fig5 (d) Fig5 (e)
THE RESULTS OF FFT-BASED REGISTRATION WITH THE MULTIPLIED NON-UNIFORM FIELD
True tFFT (s, r) (s, r) (x, y) (x, y) (1.00, 0.00) (0.02, 0.00) (0, 0) (0,0) (1.00, 80.00) (0.02, -10.39) (0, 0) (13,1) (1.00, 80.00) Fail (-50, -40) (2.00, 80.00) Fail (-50, -40) TABLE 3
Target Image
MLFFT (s, r) (x, y) (0.02, 0.00) (0,0) (0.02,-2.66) (5,4) (0.45,-3.36) (5,7) (0.28,-9.69) (9,8)
than the traditional FFT. Table 2 compares the accuracies of different Fourier methods under non-uniform intensity variations, and the results show that the proposed GCI method is more robust against the intensity variations than the other methods. Table 3 compares the accuracies of different edge information; the accuracy of the proposed GCI is slightly better others. Figure 8 depicts some examples of occlusion problems for image alignment with the templates shown in Figure 7. The proposed block partition and voting strategy effectively finds and aligns the patterns with occlusion in the target images. However, the blue block in Figure 8 (f) shows an error in template detection, because the pattern in the target is very similar to the template.
GCI (s, r) (x, y) (0.02, 0.00) (0, 0) (0.02, 0.55) (0, 0) (0.02, -0.55) (3, 7) (0.04, -0.55) (2, -1)
THE RESULTS OF FFT-BASED REGISTRATION
True (s, r) (x, y)
Fig5 (2.00, 80.00) (e) (-50, -40)
Canny edge (s, r) (x, y) (0.04, -6.17) (-2, -3)
(a)
GA (s, r) (x, y) (0.08, -6.17) (5,-3)
VI.
GCI (s, r) (x, y) (0.04, -0.55) (1, -1)
CONCLUSION
In practical image alignment, robustness against occlusion and intensity change is very important for an image alignment algorithm. In this paper, we proposed a robust Fourier-based image alignment algorithm by taking the Fourier transform of the gradient complex image. We proved the superior accuracy of the proposed algorithm for image alignment under noise corruption, intensity change, and occlusion compared to the previous Fourier-based methods through experiments. For the future work, we plan to improve the accuracy of the proposed Fourier-based algorithm by exploiting the phase information as complementary information to the current approach.
(b)
Figure 7. Two example template images.
REFERENCES [1]
(a)
(b)
(c)
(d)
(e)
(f)
B. Zitova and J. Flusser. Image Registration Methods: A Survey. Image Vision Computing, 21(11):977-1000, Nov. 2003. [2] B. S. Reddy and B. N. Chatterji. An FFT-Based Technique for Translation, Rotation and Scale-Invariant Image Registration. IEEE Trans. on Image Processing, 5(8):1266-1271, Aug. 1996. [3] H. Foroosh, J.B. Zerubia, and M. Berthod. Extension of Phase Correlation to Subpixel Registration. IEEE Trans. Image Processing, 11(3), Mar. 2002. [4] S. Zokai and G. Wolberg. Image Registration Using Log-Polar Mappings for Recovery of Large-Scale Similarity and Projective Transformations. IEEE Trans. Image Processing, 14(10), Oct. 2005. [5] Y. Keller, A. Averbuch, and M. Israeli. Pseudopolar-Based Estimation of Large Translations, Rotations and Scalings in Images. IEEE Trans. Image Processing, 14(1):12-22, Jan. 2005. [6] W. Pan, K. Qin and Y. Chen. An Adaptable-Multilayer Fractional Fourier Transform Approach for Image Registration. IEEE Trans. on Pattern Analysis and Machine Intelligence, 31(3):400-413, Mar. 2009. [7] C. D. Kuglin and D. C. Hines. The phase correlation image alignment method. Proc. Int. Conf. Cybernetics and Society, 163–165, 1975. [8] K. Takita, T. Aoki, Y. Sasaki, T. Higuchi, and K. Kobayashi. Highaccuracy subpixel image registration based on phase-only correlation. IEICE Trans. Fundamentals, E86-A(8):1925–1934, Aug. 2003. [9] K. Ito, S. Litsuka, and T. Aoki. A palmprint recognition algorithm using phase-based correspondence matching. Proc. IEEE Int. Conf. Image Processing 2009, Nov. 2009. [10] R. Gonzales and R. Woods Digital Image Processing, AddisonWesley Publishing Company, 1992, pp 443 – 452.
Figure 8. Examples of image alignment under occlusion.
For the first experiment, we perform the image alignment on the Lena images shown in Figure 5 and 6. Table 1 summarizes the accuracies of different Fourier-based methods. The accuracy of the proposed GCI method is slightly better than that of MLFFT and significantly better
2381 2385 2373
