applied sciences Article
A Novel Image Alignment Algorithm Based on Rotation-Discriminating Ring-Shifted Projection for Automatic Optical Inspection Chin-Sheng Chen * and Chien-Liang Huang Graduate Institute of Automation Technology, National Taipei University of Technology, 1, Sec. 3, Zhongxiao E. Rd, Taipei 10608, Taiwan;
[email protected] * Correspondence:
[email protected]; Tel.: +886-2-2771-2171 (ext. 4325); Fax: +886-2-8773-3217 Academic Editor: Chien-Hung Liu Received: 24 December 2015; Accepted: 29 April 2016; Published: 9 May 2016
Abstract: This paper proposes a novel image alignment algorithm based on rotation-discriminating ring-shifted projection for automatic optical inspection. This new algorithm not only identifies the location of the template image within an inspection image but also provides precise rotation information during the template-matching process by using a novel rotation estimation scheme, the so-called ring-shifted technique. We use a two stage framework with an image pyramid searching technique for realizing the proposed image alignment algorithm; in the first stage, the similarity based on hybrid projection transformation with the image pyramid searching technique is employed for quick selection and location of the candidates in the inspection image. In the second stage, the rotation angle of the object is estimated by a novel ring-shifted technique. The estimation is performed only for the most likely candidate which is the one having the highest similarity in the first stage. The experimental results show that the proposed method provides accurate estimation for template matching with arbitrary rotations and is applicable in various environmental conditions. Keywords: image alignment; rotation-discriminating; ring-shifted projection; image entropy
1. Introduction Image alignment is a fundamental technique that has many applications for machine vision and image processing, including image retrieval, object recognition, pose estimation, industrial inspection, and target tracking. Because industrial inspections are usually conducted in well-controlled environments (e.g. the working distance between the lens and inspection objects is fixed), it is common practice to perform image alignment using constrained two-dimensional (2D) rigid transformation. This creates the possibility of establishing a fast image alignment algorithm that can be used for industrial inspection applications, and is the focus of this study. The research results are usually used in the quality control of manufacture automation. For example, two cases show the feasibility of this proposed research: (1) a robust vision system can inspect components fitted to several ranges of diesel engines at a critical stage and align in the engine assembly; (2) the quality assurance of surface mount devices (SMD) mounted on the printed circuit board (PCB). Image alignment techniques are generally divided into two major categories: a feature- or geometry-based method and an area- or intensity-based method [1]. The feature-based methods, of Huttenlocher et al. [2,3] apply the directed Hausdorff distance to several algorithms to allow image alignment. Kown et al. [4] proposed a robust hierarchical Hausdorff distance to compare edge maps in a multi-level pyramidal structure. Chen et al. [5] also used the Hausdorff distance for image alignment in an inspection system for printed circuit boards. Peripherally, local descriptors are commonly used in several real-world applications, such as object recognition. Lowe [6] proposed a scale invariant
Appl. Sci. 2016, 6, 140; doi:10.3390/app6050140
www.mdpi.com/journal/applsci
Appl. Sci. 2016, 6, 140
2 of 21
feature transform (SIFT), which combines a scale invariant region detector and descriptor and uses the gradient distribution in detected regions. SIFT emphasizes local features at a particular point of interest that are invariant to scale and rotation. Mikolajczyk and Schmid [7] compared the performance of descriptors that were computed for local interest regions. Even though SIFT is a useful method to describe the region of interest when the region has scale and rotates, SIFT-based matching fails when there are only a few feature points in the template image. The Zernike moments method [8] is also used to pattern features for a pattern image and determines invariance to rotation, translation, and scale. To ensure computational efficiency, Mondal [9] further proposed an accelerated version that uses a multi-resolution wavelet technique, projection, and Zernike moments. Ullah [10] proposed a rotation-invariant template method that uses gradient information in the form of orientation codes. However, this method is time-consuming because the histogram must be computated. In order to increase the speed of the template-matching method that uses orientation codes, the novel rotation discriminative descriptor and an object search strategy were used [11]. The aforementioned invariant descriptors of points of interest in the image are resistant to partial occlusion and relatively insensitive to changes of viewpoint. The algorithms referred to are generally efficient in general applications, but they can result in unnecessary processing when the image alignment only involves rotation and limited scaling compensation, as with automatic optical inspection (AOI) applications. The area-based method is sometimes called the correlation-like or template-matching method and has been popular for decades because of its basic concepts. The small template is firstly applied to a large inspection image by sliding the template window on a pixel-by-pixel basis. The normalized cross correlation (NCC) is computed between the template and the inspection images. The maximum values or peaks for the computed correlation values indicate the matches between a template and sub-images in the inspection image. The NCC metric is often used to manage the registration of images that differ only by a translation. If images are deformed by complex transformations, the template window cannot cover the same regions of interest in the template and the inspection images, so several studies have proposed modified NCC metrics to circumvent the image registration problem that is associated with complex transformations. To allow image matching that is invariant to rotation, a ring-projection transformation was proposed that transforms a 2D gray-level image into a rotation-invariant representation, as in a one-dimensional (1D) ring-projection space [12]. Tsai and Chiang [13] further used the ring-projection to represent a target template in wavelet-decomposed sub-images. Specifically, they used only pixels with high wavelet coefficients at low levels of resolution to compute the NCC between two separate templates. Choi and Kim [14] also used a two-stage image-alignment method that first locates candidates by comparing the vector sums for ring-projection and then matches these candidates using Zernike moments. Table 1 summarizes the features, approaches, and disadvantages of image-alignment methods for each category of method. The feature-based category uses edge maps, points of interest, shape contours, and invariant descriptors as alignment features. The embedded approaches are the Hausdorff distance, feature correspondence, the Zernike moment, and measurement of dissimilarity. The disadvantage of the feature-based method is the alignment error that occurs when there is inaccurate feature extraction. The area-based category uses image templates as matching features and cross correlation or ring projection to register the images. However, computation time can be excessive for applications that involve complex image transformation. Table 1. A summary of the image alignment methods. Category
Features
Approaches
Disadvantages
Feature-based
Edge maps Interest point Invariant descriptors Orientation code
Hausdorff distance Feature correspondence Zernike moment Dissimilarity measurement
Inaccurate feature extraction
Area-based
Image templates
Cross correlation Ring-projection
Excessive computation time
Appl. Sci. 2016, 6, 140
3 of 21
Although there have been many studies of area-based image alignment techniques, 3 of 21 relatively Appl. Sci. 2016, 6, 140 few concern ring-projection alignment, which is a special type of area-based method. This method Although there have image been many of area‐based image alignment techniques, relatively transforms a 2D gray-level into studies a rotation-invariant representation in one-dimensional (1D) few concern ring‐projection alignment, which is a special type of area‐based method. This method ring-projection space [12]. Tsai [15] used the ring-projection technique to represent templates in transforms a 2D gray‐level image into a rotation‐invariant representation in one‐dimensional (1D) multi-channel images using a method that rapidly selects candidates by computing the NCC between space [12]. Tsai [15] the is ring‐projection technique to represent templates colorring‐projection ring-projection information. The used rotation then estimated by rotating the template. To in reduce multi‐channel images using a method that rapidly selects candidates by computing the NCC the computational complexity, an elimination strategy that quickly detects a similar candidate is between color ring‐projection information. The rotation is then estimated by rotating the template. used [16]. In this approach, the rotation invariant ring-projection transformation is used to describe To reduce the computational complexity, an elimination strategy that quickly detects a similar the image. Two stage-matching methods have also been proposed in which the candidates are selected candidate is used [16]. In this approach, the rotation invariant ring‐projection transformation is used usingto the vector sums of theTwo ring stage‐matching projection at the first stage. single candidate undergoes describe the image. methods have Only also abeen proposed in then which the Zernike-moment template matching using rotation-invariant properties [14,17]. Radial projection candidates are selected using the vector sums of the ring projection at the first stage. Only a single candidate then undergoes Zernike‐moment template matching using rotation‐invariant properties has also been shown to be applicable to template matching [18]. Radial projection transforms a 2D [14,17]. image Radial into projection has also shown invariance to be applicable to template Radial lines. gray-level 1D space andbeen the scale is constructed inmatching the form[18]. of radial projection transforms a 2D gray‐level image into 1D space and the scale invariance is constructed in However, ring-projection descriptors are specific to rotation-invariant matching. The rotation angle the form of radial lines. However, ring‐projection descriptors are specific to rotation‐invariant between two corresponding shapes in different images can also not be estimated in order to perform matching. The rotation angle between two corresponding shapes in different images can also not be image alignment as is necessary for industrial inspection. estimated in order to perform image alignment as is necessary for industrial inspection. In AOI applications, the geometric relationship between a template image P and an inspected In AOI applications, the geometric relationship between a template image P and an inspected image S is constricted as shown in Figure 1.. image S is constricted as shown in Figure 1
T
Figure 1. The geometric relationship between pattern image P and inspection image S.
Figure 1. The geometric relationship between pattern image P and inspection image S.
An indicated inspection pattern (such as the diamond templates shown in Figure 1) is usually An indicated inspection pattern (such as the diamond templates shown in Figure 1) is usually used for the images, in order to alleviate the computational burden that is associated with real‐time usedimage alignment. Because the working distance between the lens and the inspected target is fixed in for the images, in order to alleviate the computational burden that is associated with real-time AOI applications, the scaling variation between P and S is tiny and can be neglected. Therefore, the image alignment. Because the working distance between the lens and the inspected target is fixed in AOI image alignment problem for AOI can be formulated to identify the optimal 2D rigid transformation applications, the scaling variation between P and S is tiny and can be neglected. Therefore, the T and the between P and S, as constrained vector rotation angle method . The image alignment problem for AOI canby betranslation formulated to identify optimal 2D rigid transformation á presented here locates the template image within the inspection image and provides precise rotation between P and S,during as constrained by translation vectorby T using and rotation θ. estimation The method presented information the template‐matching process a novel angle rotation scheme. here This locates the template image within the inspection image and provides precise rotation information is the so‐called ring‐shifted technique which uses the architecture of a two‐stage during the template-matching process by stage using(the a novel rotation estimation candidate‐searching stage), scheme. the only This likely is the template‐matching method. In the first candidates are determined using uses rotation‐invariant features. Subsequently, the ring‐shifted so-called ring-shifted technique which the architecture of a two-stage template-matching method. technique is applied to the most likely candidate that only is obtained the first stage in order to using In the first stage (the candidate-searching stage), the likely from candidates are determined estimate the rotation in the second stage the (the ring-shifted rotation‐estimation stage). rotation rotation-invariant features. Subsequently, technique is Because applied the to the mostis likely directly calculated using hybrid‐projection data, the proposed method minimizes computational candidate that is obtained from the first stage in order to estimate the rotation in the second stage (the complexity. Even though the newly matching method is proposed, it is not invariant to scale changes.
rotation-estimation stage). Because the rotation is directly calculated using hybrid-projection data, the
Appl. Sci. 2016, 6, 140
4 of 21
proposed method minimizes computational complexity. Even though the newly matching method is proposed, it is not invariant to scale changes. Appl. Sci. 2016, 6, 140 4 of 21 The main contributions of this paper include the following: (1) The novel image alignment algorithm based on rotation-discriminating ring-shifted projection, which is efficiently applied The main contributions of this paper include the following: (1) The novel image alignment to recognize and align the template and inspection images; (2) The proposed hybrid projection algorithm based on rotation‐discriminating ring‐shifted projection, which is efficiently applied to transformation basedalign on image entropyand can provide unique and(2) robust forhybrid rapidly selecting the recognize and the template inspection images; The features proposed projection most transformation likely candidate based on the image pyramid searching technique; (3) A new rotation-estimation based on image entropy can provide unique and robust features for rapidly technique known as the ring-shifted is proposed to detect the rotation of only the best of selecting the most likely candidate technique based on the image pyramid searching technique; (3) A new rotation‐estimation technique known as the ring‐shifted technique is proposed to detect the rotation the output of the candidates; (4) An image pyramid searching technique is applied to significantly of only the best of the output of the candidates; (4) An image pyramid searching technique is enhance the computation performance. applied to significantly enhance the computation performance.
2. Architecture of the Proposed Method 2. Architecture of the Proposed Method
Before giving an overview of the proposed method, two fundamental projections, ring- and Before giving an overview of the proposed method, two fundamental projections, ring‐ and variance-projection, should be mentioned. The ring- and variance-projection transform the 2D image variance‐projection, should be mentioned. The ring‐ and variance‐projection transform the 2D image into 1D signals as function of radius, which are defined as the mean and variance of intensities along into 1D signals as function of radius, which are defined as the mean and variance of intensities along the radius, respectively. The main concepts of ring- and variance-projection are shown in Figure 2. the radius, respectively. The main concepts of ring‐ and variance‐projection are shown in Figure 2. The ring-projection calculates the the average intensity forfor thethe specified radius, here three different The ring‐projection calculates average intensity specified radius, here three different rings, r1 “ rings, 60, r2 “r1 120, “ 180 pixelsr3are shown. Furthermore, corresponding , r2r3 120 , and pixels are shown. the Furthermore, the variance-projections corresponding 60 and 180 can be seen in the bottom figure. Obviously, the value of Obviously, 180 pixels the ringvalue is lower than others because variance‐projections can be seen in the bottom figure. of 180 pixels ring is the intensity is almost uniform in this ring. lower than others because the intensity is almost uniform in this ring. 200 180 160 140 C p,0(r)
CP ,0 (r )
120 100 80 60 40 20
r1 60
r2 120
r3 180
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176 183 190 197
0
r1 r2
r (pixe l)
r3
4000000 3500000
P ,0 (r )
3000000 σp,0(r)
2500000 2000000 1500000 1000000 500000
r1 60
r2 120
r3 180
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99 106 113 120 127 134 141 148 155 162 169 176 183 190 197
0 r (pixe l)
Figure 2. Main concepts of ring‐ and variance‐projection. Figure 2. Main concepts of ring- and variance-projection.
The architecture of the proposed algorithm is shown in Figure 3. It consists of two phases:
The architecture of the proposed algorithm is shown in Figure 3. It consists of two phases: template preprocessing and online alignment. In the template preprocessing phase, template image template preprocessing andmanually. online alignment. Intechnique the template preprocessing template image P is the input specified A pyramid is then used to build phase, a multi‐resolution P is the input specified manually. pyramid technique then used to buildusing a multi-resolution image from the template image. A Hybrid‐projection values isare then determined ring‐ and and wdetermined , in conjunction with wm ,lare ( r ) then ( r ) imagevariance‐projection values and the weighting coefficients, from the template image. Hybrid-projection values using ring- and σ,l variance-projection values and theimages. weighting coefficients,wmw,l (m,l and prq,those in conjunction corresponding multi‐resolution The coefficients and of the ring with r ) prq wσ,lw (rσ,l ) are corresponding multi-resolution images. The coefficients w prq and w prq are those of the ring and m,l σ,l and variance projection, respectively, depending on the intensity distribution at the specific radius variance projection, respectively, depending on the intensity distribution at the specific radius of the of the template image. In the online alignment phase, the inspection image S is the input and a template image. In the online alignment phase, inspection image S in is the and a sliding sliding window, which is the same size as the the region of interest (ROI) the input multi‐resolution template moved multi‐resolution inspection Sl. This detects the most likely window, whichimage, is the is same sizefor as the region of interest (ROI) image in the multi-resolution template image, is candidates by evaluating the similarity coefficient of the hybrid projection values. The locations of moved for multi-resolution inspection image Sl . This detects the most likely candidates by evaluating
Appl. Sci. 2016, 6, 140
5 of 21
Appl. Sci. 2016, 6, 140
5 of 21
the similarity coefficient of the hybrid projection values. The locations of the candidates with scores the candidates with scores that exceed the similarity threshold Tmin,l are registered. The translation that exceed the similarity threshold Tmin,l are registered. The translation is estimated using the vector á estimated using the vector T between the centers of the target detected in PL and SL. These is T between the centers of the target detected in PL and SL . These candidates and their corresponding candidates and their corresponding ring intensity profiles for each specified radius undergo the ring intensity profiles for each specified radius undergo the rotation-estimation process. This proposed rotation‐estimation process. This proposed method is called the ring‐shifted technique to estimate method is called the ring-shifted technique to estimate the rotation angle. The accuracy of the rotation the rotation angle. The accuracy of the rotation angle is finally refined by using a parabola curve angle is finally refined by using a parabola curve fitting. The details of the two phases of the algorithm fitting. The details of the two phases of the algorithm are described in the following subsections. are described in the following subsections. S
P L
Pl , l 0,1...L
CP ,l (r ), P ,l (r )
Sl , l 0,1...L
wm ,l (r ), w ,l (r )
CS ,l ( x, y, r ), S ,l ( x, y, r )
H P ,l ( r )
H S ,l ( x , y , r )
Tmin,l
Figure 3. Architecture of the proposed algorithm. Figure 3. Architecture of the proposed algorithm.
3. Candidate Selection and Localization 3. Candidate Selection and Localization The The candidates candidates are are first first selected selected and and located located using using a a rotation‐invariant rotation-invariant descriptor descriptor and and the the second stage uses the rotation‐estimation method. For computational efficiency, the ring‐projection second stage uses the rotation-estimation method. For computational efficiency, the ring-projection transformation transformation and and the the image image pyramid pyramid technique technique [19] [19] are are used used during during the the candidate candidate selection selection process. These techniques identify the best matching results by using low‐cost features in each image process. These techniques identify the best matching results by using low-cost features in each pyramid level. In the candidate‐selection and localization process, the selection of a template image image pyramid level. In the candidate-selection and localization process, the selection of a template for the proposed method is a crucial step. This template image is used in the template preprocessing image for the proposed method is a crucial step. This template image is used in the template phase to generate hybrid‐projection values. The candidate‐selection and localization uses these preprocessing phase to generate hybrid-projection values. The candidate-selection and localization hybrid‐projection values. Therefore, the features of the template image in the uses these hybrid-projection values. Therefore, the features of the template image in the template template preprocessing phase considerably influence the matching results. In the hybrid‐projection preprocessing phase considerably influence the matching results. In the hybrid-projection transform, transform, the hybrid‐projection are calculated using a circular region Rbetween the hybrid-projection values are values calculated using a circular region between and RRl ,min and on l,min
l,max
Rl ,max on image pyramid level l. For illustrating the hybrid projection calculation, a concise example image pyramid level l. For illustrating the hybrid projection calculation, a concise example is illustrated in in Figure 4, thethe hybrid projection values on current image pyramid l are l calculated from is illustrated Figure 4, hybrid projection values on current image pyramid are calculated the R “ 20 to R “ 200. And the details of hybrid projection calculation are described in the from the Rl ,max 200 . And the details of hybrid projection calculation are described in l,min Rl ,min 20 to l,max following subsections. the following subsections.
Appl. Sci. 2016, 6, 140 Appl. Sci. 2016, 6, 140
6 of 21 6 of 21
Rl ,max 200
Rl ,min 20
Figure 4. An example to illustrate the circular region of hybrid projection calculation. Figure 4. An example to illustrate the circular region of hybrid projection calculation.
3.1. Ring‐Projection Transformation 3.1. Ring-Projection Transformation Using the rotation invariant properties, the ring‐projection transformation is used. The center Using the rotation invariant properties, the ring-projection transformation is used. The center ( xcc, ,yyccq,) of x, of y ) the point, , of the block image is calculated and used to transform the pixel coordinate of point, px the block image is calculated and used to transform the pixel coordinate px, (yq the block image from a Cartesian coordinate into a polar coordinate by: block image from a Cartesian coordinate into a polar coordinate by:
x x r cos θ, y y r sin θ x “ xcc ` rcosθ, y “ ycc ` rsinθ
(1) (1)
where θ [0, 2 ] , r b ( x xc ) 2 ( y yc ) 2 , r [ R0,min , R0,max ] , R0,max min( M 2 , N 2) , R0,min is where θ P r0, 2πs, r “ px ´ xc q2 ` py ´ yc q2 , r P rR0,min , R0,max s, R0,max “ minpM{2, N{2q, R0,min is defined by the user, and M and N are the width and height of an image, respectively. defined by the user, and M and N are the width and height of an image, respectively. C P , l ( r ) represents the ring‐projection values for the template image at a specific radius r on CP,l prq represents the ring-projection values for the template image at a specific radius r on image image pyramid level , which is defined as: pyramid level l, whichl is defined as:
1 C l (r ) 1 ÿ P( xc r cos θ k , yc r sin θ k ) CP,lP ,prq “ Qr k Ppx c ` rcosθk , y c ` rsinθk q Qr
(2) (2)
k
where Qr is the number of pixels that are located on the circle of radius r Rl ,min , Rl ,min 1, , Rl ,max . where Qr is the number of pixels that are located on the circle of radius r “ Rl,min , Rl,min ` 1, . . . , Rl,max . The ring‐projection value, C P , l ( r ) , is the mean of the values for pixel intensity at the specific The ring-projection value, CP,l prq, is the mean of the values for pixel intensity at the specific radius r on radius image pyramid level l . the Because the value is constructed rings of radius, increasing r on image pyramid level l. Because value is constructed along ringsalong of increasing the radius, the one‐dimensional ring‐projection values are invariant to any rotation of the one-dimensional ring-projection values are invariant to any rotation of the two-dimension template two‐dimension template image. An example of the ring projection of an image is shown in Figure 5. image. An example of the ring projection of an image is shown in Figure 5. Figure shows that profiles of these two images remain identical after rotation Figure 55 shows that thethe profiles of these two images remain nearly nearly identical after rotation changes changes have been completed. In the candidate‐selection process, the ring‐projection value in the have been completed. In the candidate-selection process, the ring-projection value in the inspection ( x , y ) r l is inspection image that is located at with specific radius on image pyramid level image that is located at px, yq with specific radius r on image pyramid level l is defined as follows: defined as follows: 1 ÿ CS,l px, y, rq “ 1 Spx ` rcosθk , y ` rsinθk q (3) C S , l ( x, y , r ) Q r (3) k S ( x r cos θ k , y r sin θ k ) Qr k where the “S” is inspection image shown in Figure 4. The lookup table and optimal step angle φopt,l where the “S” is inspection image shown in Figure 4. The lookup table and optimal step angle opt ,l are used to reduce computational complexity. The lookup table is created using rounded integer values are used to reduce complexity. lookup is created locations using rounded integer for the distance fromcomputational the center of the template The image to thetable corresponding at the specific values for the distance from the center of the template image to the corresponding locations at the radius. The optimal step angle φopt,l is shown in Figure 6 and is defined as follows: specific radius. The optimal step angle opt ,l is shown in Figure 6 and is defined as follows: d2 1 d2 2q φopt,l prq “ cos´ (4) 1 p1 ´ (4) opt ,l (r ) cos (1 2r ) 2 2r where d is the minimum shift distance between the original and the corresponding rotated points at where d is the minimum shift distance between the original and the corresponding rotated points the specified radius r. Using the optimal step angle, the number of pixels Q at the specific radius r is at the specified radius r . Using the optimal step angle, the number of rpixels Qr at the specific calculated by 2π{φopt,l prq and θk “ θk´1 ` φopt,l . radius r is calculated by 2 opt , l (r ) and θ k θ k 1 opt , l .
Appl. Sci. 2016, 6, 140 Appl. Sci. 2016, 6, 140
7 of 21 7 of 21
Appl. Sci. 2016, 6, 140
7 of 21
(a)
(c)
(a)
(c)
(b)
(d)
(b)
(d)
Figure 5. An image at two distinct angles of rotation. (a) The original image; (b) The rotated image Figure 5. An image at two distinct angles of rotation. (a) The original image; (b) The rotated image Figure 5. An image at two distinct angles of rotation. (a) The original image; (b) The rotated image (30° counterclockwise, CCW); (c, d) Show the corresponding profiles of ring projection. (30˝ counterclockwise, CCW); (c, d) Show the corresponding profiles of ring projection. (30° counterclockwise, CCW); (c, d) Show the corresponding profiles of ring projection.
rr
optopt,l (,lr()r ) rr
dd
Figure 6. Definition of the optimal step angle for image pyramid level l.
Figure 6. Definition of the optimal step angle for image pyramid level l. Figure 6. Definition of the optimal step angle for image pyramid level l.
3.2. Robust Features
3.2. Robust Features As previously mentioned, the ring‐projection transformation is used to describe the features of 3.2. Robust Features As an image and to reduce computational complexity. However, the features should have unique and previously mentioned, the ring-projection transformation is used to describe the features of As previously mentioned, the ring‐projection transformation is used to describe the features of robust properties to correctly recognize the candidates. In despite of the ring‐projection an an image and to reduce computational complexity. However, the features should have unique and image and to reduce computational complexity. However, the features should have unique and transformation having a rotation invariant property, it cannot describe the unique features of an robust properties to correctly recognize the candidates. In despite In of the ring-projection transformation robust properties to correctly recognize the candidates. despite of the ring‐projection image when the image contains a homogeneous region. For example, texture‐less searching images having a rotation invariant property, it cannot describe the unique features of an image when the containing different of homogeneous regions are shown Figure 7a–d. The corresponding transformation having a types rotation invariant property, it cannot describe the unique features of an image contains homogeneous For example, texture-less searching images containing different features athat are described region. by ring‐projection transformation are shown in Figure 7e. As can been image when the image contains a homogeneous region. For example, texture‐less searching images seen in Figure 7e, the features of searching images 3 and 4 are easily distinguished; it also means the types of homogeneous regions are shown Figure 7a–d. The corresponding features that are described containing different types of homogeneous regions are shown Figure 7a–d. The corresponding by features ring-projection are shown in Figure 7e. As can been seen in Figure 7e, the features that are transformation described by ring‐projection transformation are shown in Figure 7e. As can been of searching images 3 and 4 are easily distinguished; it also means the searching images 3 and 4 can be seen in Figure 7e, the features of searching images 3 and 4 are easily distinguished; it also means the
Appl. Sci. 2016, 6, 140
8 of 21
Appl. Sci. 2016, 6, 140
8 of 21
correctly recognized by evaluating the similarity of the features in the template and searching image in searching images 3 and 4 can be correctly recognized by evaluating the similarity of the features in the candidate selection and localization process. the template and searching image in the candidate selection and localization process.
(a)
(b)
(c)
(d)
(e) Figure 7. (a–d) Textures‐less searching images 1–4; (e) Corresponding ring projection features. Figure 7. (a–d) Textures-less searching images 1–4; (e) Corresponding ring projection features.
By contrast, the features of images 1 and 2 cannot be correctly recognized by evaluating the By contrast, the features of images 1 and 2 cannot correctly recognized by evaluatingthe the similarity of the features in the candidate selection and belocalization process. Consequently, similarity of the features in the candidate selection and localization process. Consequently, the features features must describe the homogeneous ring regions in a distinct rather than a uniform manner. To must describe homogeneous ring regions a distinct ratherin than uniform To overcome overcome this the problem, the variance of the inspecific radius the aimage is manner. considered and the this problem, the variance of the specific radius in the image is considered and the variance-projection variance‐projection transform is used [20]. The variance projection transformation in the template transform is used [20]. The variance projection transformation in the template and searching images and searching images on image pyramid level l are defined as σ p,l and σs,l: on image pyramid level l are defined as σp,l and σs,l : 1 P ,l (r ) (5) k [ P( xc r cos k , yc r sin k ) CP,l (r )]2 2 1r ÿ Q σP,l prq “ rPpxc ` rcosθk , yc ` rsinθk q ´ CP,l prqs (5) Qr k 1 S , l ( x, y , r ) (6) k [ P( x r cos k , y r sin k ) CS ,l ( x, y, r )]2 Q1r ÿ 2 rPpx ` rcosθk , y ` rsinθk q ´ CS,l px, y, rqs (6) σS,l px, y, rq “ Qr k In Figure 8, the features are described by the variance‐projection transformation which actually reflects variation for image 2 and other images (images 1, 3 and 4). However, this actually is not In the Figure 8, the features are described by the variance-projection transformation which sufficient to describe the features of image 1, 3, and 4. Fortunately, the features of image 1, 3, and 4 reflects the variation for image 2 and other images (images 1, 3 and 4). However, this is not sufficient can be distinguished by ring‐projection transformation as shown in Figure 7e. to describe the features of image 1, 3, and 4. Fortunately, the features of image 1, 3, and 4 can be For the sake of unique and robust features description, a hybrid‐projection transformation that distinguished by ring-projection transformation as shown in Figure 7e. combines the ring‐ and and variance‐projection transformations is used in transformation this paper. The For the sake of unique robust features description, a hybrid-projection that hybrid‐projection transformation in the template and searching images on image pyramid level l are combines the ring- and variance-projection transformations is used in this paper. The hybrid-projection defined as H p,l and H s,l: template and searching images on image pyramid level l are defined as H transformation in the p,l and Hs,l : H P , l ( r ) wm , l ( r ) C P , l ( r ) w , l ( r ) σ P ,l ( r ) (7) HP,l prq “ wm,l prq ˆ CP,l prq ` wσ,l prq ˆ σP,l prq (7) H S , l ( x , y , r ) wm , l ( r ) C S , l ( x , y , r ) wσ ,l ( r ) σ S , l ( x , y , r ) (8) HS,l px, y, rq “ wm,l prq ˆ CS,l px, y, rq ` wσ,l prq ˆ σS,l px, y, rq (8)
Appl. Sci. 2016, 6, 140
9 of 21
Appl. Sci. 2016, 6, 140
9 of 21
where wm,l prqw and w σ,l prq w are(weighting coefficients for the ring transformation, where and are weighting coefficients for and the variance-projection ring and variance‐projection m ,l ( r ) σ ,l r ) respectively. In Equations (7) and (8), the weighting coefficients at the specific radius on image pyramid transformation, respectively. In Equations (7) and (8), the weighting coefficients at the specific levelradius on image pyramid level l are calculated by the concept of image entropy. l are calculated by the concept of image entropy.
Figure 8. Variance projection features of texture‐less searching images. Figure 8. Variance projection features of texture-less searching images.
The image entropy is a quantity which is used to describe the randomness of an image, the
The image entropy is a quantity which is used to describepbthe of an image, the image (i ) randomness image entropy H with associated probabilities of occurrence can be defined as: entropy H with associated probabilities of occurrence pbpiq can be defined as: g H 1 pb(i ) log( pb (i )) g ÿ
i0
H “ ´1 ˆ
where pb (i ) is the probability of gray level i .
(9)
pbpiqlogppbpiqq
(9)
i “0
In the case of an image, if all of the pixels have the same value, the image entropy of the image is zero. On the other hand, a high image entropy occurs when an image has chaotic gray levels. For the where pbpiq is the probability of gray level i. hybrid‐projection transformation, the pixels data along radius the having high variance should In the case of an image, if all of the havethe thespecific same value, image entropy of the image is have high weights. Therefore, the weighting coefficients at the specific radius are defined as follows: zero. On the other hand, a high image entropy occurs when an image has chaotic gray levels. For the hybrid-projection transformation, the data along the specific H radius having high variance should have wσ,l (r ) (1 Tmin,l ) (10) log(specific g) high weights. Therefore, the weighting coefficients at the radius are defined as follows: wm , l ( r ) 1 wσ,l ( r ) H
wσ,l prq “ p1 ´ Tmin,l q
(11)
logpgq is between 0 and 1, 1 T is where the range of variance‐projection weighting coefficient min,l designed for keeping the robustness against the noise in the image ( Tmin,l is the similarity threshold
wm,l prq “ 1 ´ wσ,l prq
(10) (11)
on image pyramid level l), the weighting coefficient of ring‐projection transformation is based on where the range of variance-projection weighting coefficient is between 0 and 1, 1 ´ Tmin,l is designed Equation (10), and in this study it is designed to get a symmetrical weighting coefficient. for keeping the robustness against the noise in the image (Tmin,l is the similarity threshold on image As mentioned in Equations (7) and (8), the weighting coefficients are separately determined at the specific within the circular region on each template image pyramid level. the (10), pyramid level l),radius the weighting coefficient of ring-projection transformation is based onUsing Equation hybrid‐projection 7a–d and normalizing the hybrid values to a range and in this study it is transformation designed to getin a Figure symmetrical weighting coefficient. between 0 and 1, yield the results shown in Figure 9. All features of the texture‐less searching images, As mentioned in Equations (7) and (8), the weighting coefficients are separately determined i.e., Figure 7a–d, are clearly identified by this new descriptor. It means the features of the template at the specific radius within the circular region on each template image pyramid level. Using the image and searching image can be correctly recognized by evaluating the similarity based on the hybrid-projection transformation in Figure 7a–d and normalizing the hybrid values to a range between described features in the candidate selection and localization process. For this new descriptor, the 0 andweighting 1, yield the results shown in Figure 9. All features of the texture-less searching images, i.e., coefficients wm , l ( r ) and wσ ,l ( r ) represent the weighting for the ring‐ and
Figure 7a–d, are clearly identified by this new descriptor. It means the features of the template image variance‐projection values at the specific radius in the image pyramid level. Using the proposed and searching image can be correctly recognized by evaluating the similarity based on the described features in the candidate selection and localization process. For this new descriptor, the weighting coefficients wm,l prq and wσ,l prq represent the weighting for the ring- and variance-projection values at the specific radius in the image pyramid level. Using the proposed method, the hybrid-projection transformation gives unique and more robust rotation-invariant features.
Appl. Sci. 2016, 6, 140
10 of 21
Appl. Sci. 2016,the 6, 140 method,
hybrid‐projection rotation‐invariant features.
transformation
gives
unique
and
more
10 of 21 robust
Figure 9. Hybrid projection features of texture‐less searching images. Figure 9. Hybrid projection features of texture-less searching images.
3.3. Similarity Measurement 3.3. Similarity Measurement When the robust features are determined, the NCC is used as a similarity measure for selection When the robust features are determined, the NCC is used as a similarity measure for selection of of the possible candidates. The correlation coefficient for the template image and a searching block the possible candidates. The correlation coefficient for the template image and a searching block at ( x,in y )the at point in the inspection image on image pyramid level l are then defined as: point px, yq inspection image on image pyramid level l are then defined as: Rmax
( H (r ) H (r )) ( H ( x, y, r ) H ( x, y, r )) pH P,l prq ´ H P,l prqq ¨ pHS,l px, y, rq ´ H S,l px, y, rqq
Rř max
( x, y ) δl px, yql “ d
r Rmin
Rmax
(r )
H
P ,l
S ,l
Rmax
S ,l
Rř max ( H P ,l (r ) H P ,l (r )) 2 ( H S ,l ( x, y, r ) H S ,l ( x, y, r )) 2 pH pHS,l px, y, rq ´ H S,l px, y, rqq P,l prq ´ H P,l prqq ¨r Rmin r Rmin
Rř max
r“ Rmin
H
P ,l
r“ Rmin
2
2
(12) (12)
r“ Rmin
( x, y , r )
l S ,l where H P ,prq and are values the values the hybrid‐projection transformation for the where and HS,l px, y, rq are the of the of hybrid-projection transformation for the template P,l template and inspection images, respectively. Note that l is the index of the image pyramid levels. and inspection images, respectively. Note that l is the index of the image pyramid levels. The l (isx,between y ) is between The correlation It is toequal to 1 when a match perfect match occurs correlation valuevalue δl px, yq ´1 and−1 1. and It is 1. equal 1 when a perfect occurs between between the template only the the template image andimage and searching block. The candidate selection process searching block. The candidate selection process allows onlyallows the most likely most likely candidate to be fed into the refinement process. candidate to be fed into the refinement process.
3.4. Image Pyramid Search Technique 3.4. Image Pyramid Search Technique The image The image pyramid pyramid search search framework framework allows allows this this type type of of target target search. search. The The image image pyramid pyramid hierarchically splits the candidates using information from different image pyramid levels to disjointed hierarchically splits the candidates using information from different image pyramid levels to subsets. This process allows many candidates to be discarded early in the search process because their disjointed subsets. This process allows many candidates to be discarded early in the search process similarity coefficients are lower than the pre-defined threshold for similarity. One case involving three because their similarity coefficients are lower than the pre‐defined threshold for similarity. One case image pyramid levels of the search process is shown in Figure 10. involving three image pyramid levels of the search process is shown in Figure 10. The search process starts at the highest pyramid level (level 2) of the candidate. The candidate on The search process starts at the highest pyramid level (level 2) of the candidate. The candidate the highest pyramid levellevel is identified by computing the similarity using hybrid-projection values, on the highest pyramid is identified by computing the similarity using hybrid‐projection H P ,2H ( rS,2 ) px, y, rq, H at(each x , y , rposition ) obj2, HP,2 prq and (x, y). The candidate, obj2,1 , on level 2,1 which is indicated by values, and S ,2 , at each position (x, y). The candidate, , on level 2, which is a green circle in Figure 10, is also identified. The candidate with a similarity score that exceeds the indicated by a green circle in Figure 10, is also identified. The candidate with a similarity score that similarity threshold Tmin,2 is stored in the candidate list. The value of Tmin,2 is also dependent on the exceeds the similarity threshold Tmin,2 is stored in the candidate list. The value of Tmin,2 is also application and must be set to the minimum expected object visibility. On the next lower pyramid dependent on the application and must be set to the minimum expected object visibility. On the next levels (level 1), the search process uses the candidate that is located on the previous pyramid level. lower pyramid levels (level 1), the search process uses the candidate that is located on the previous This includes candidates obj1,t1,2,3,4u , which are denoted by a yellow circle and are inherited from their obj1,{1,2,3,4} pyramid level. This includes which are denoted by 1. a yellow circle and parent candidate obj2,1 using acandidates 2 ˆ 2 searching area , on image pyramid level Only candidate objare 1,2 , obj which is denoted by a red square, has a similarity score that exceeds the similarity threshold T 2,1 min,1 . using a 2 2 searching area on image pyramid level 1. inherited from their parent candidate This process is repeated until all matching candidates have been tracked to the lowest pyramid level,
Appl. Sci. 2016, 6, 140
11 of 21
Appl. 2016, 6, 140obj1,2 , which is denoted by a red square, has a similarity score that exceeds 11 of 21 Only Sci. candidate the
similarity threshold Tmin,1 . This process is repeated until all matching candidates have been tracked to the lowest pyramid level, which, in this case is level 0. Finally, the best matching result with a which, in this case is level 0. Finally, the best matching result with a similarity score that exceeds the similarity score that exceeds the similarity threshold is denoted by the red fill square in Figure 10. similarity threshold Tmin,0 is denoted by the red fill Tsquare in Figure 10. min,0
obj2,1
obj1,1
obj0,{1,...,4}
obj1,2
obj2,1
obj1,3
obj0,5 obj0,6
obj1,4
obj0,7 obj0,8
obj1,{1,2,3,4}
obj0,{9,...,12}
obj0,{13,...,16}
obj0,{5,6,7 ,8}
Figure 10. The image pyramid search technique. Figure 10. The image pyramid search technique.
T
In the search search process, process, the the similarity threshold is set a different image pyramid level. In the similarity threshold Tmin,lmin,lis set in ain different image pyramid level. The The intuitive way the touser to determine the isvalue is to choose it and 1. If the mostmost intuitive way for thefor user determine the value to choose it between between 0 0 and 1. If the threshold threshold is set to be a low value, more candidates will be selected in the searching process. It also is set to be a low value, more candidates will be selected in the searching process. It also means means the computation will increase. Furthermore, the similarity of the candidates the computation burden burden will increase. Furthermore, the similarity score ofscore the candidates should should be decreased on the pyramid higher pyramid level the because are missed. Therefore, the be decreased on the higher level because detailsthe aredetails missed. Therefore, the similarity T threshold T must be slightly reduced on higher pyramid levels in order to avoid a candidate that min,l min,l similarity threshold must be slightly reduced on higher pyramid levels in order to avoid a is missed. For example, the similarity threshold of each higher image pyramid level is decreased by 0.1 candidate that is missed. For example, the similarity threshold of each higher image pyramid level is by rule of thumb. decreased by 0.1 by rule of thumb. 4. Estimation of the Rotation 4. Estimation of the Rotation The candidate-selection process, as described in the previous section, selects the most likely The candidate‐selection process, as described in the previous section, selects the most likely candidates. A new rotation-estimation technique known as the ring-shifted technique is proposed to candidates. A new rotation‐estimation technique known as the ring‐shifted technique is proposed to detect the rotation of only the best of the output of the candidates. Although, the original ring-projection detect the rotation of only the best of the output of the candidates. Although, the original CP,l prq shown in Equation (2) is rotation invariant, the ring-shifted technique will only inherit the ring ring‐projection C P , l ( r ) shown in Equation (2) is rotation invariant, the ring‐shifted technique will intensity profile Ppx c ` rcosθk , yc ` rsinθk q for each specific radius r. So, it does not need any operation ( xc rotation r cos k estimation , yc r sin kprocess, ) for each specific radius r. So, it only inherit the ring intensity profile profile inPthe for preparing the ring intensity which considerably reduces does not need any operation for preparing the ring intensity profile in the isrotation estimation the computation time. For the rotation invariance, the hybrid-projection value constructed along process, which reduces the computation time. technique, For the rotation the circular rings withconsiderably successively increasing radii. In the ring-shifted the ring invariance, intensity profile hybrid‐projection value is constructed along circular rings with successively increasing radii. In the at each radius is used to estimate the rotation. To illustrate the ring-shifted technique, Figure 11 shows ring‐shifted technique, the ring intensity images with different rotation angles and profile at each radius is used to estimate the rotation. To the corresponding ring intensity profiles pr and p1r at the illustrate the ring‐shifted technique, Figure 11 shows images with different rotation angles and the specific radius of r = 17. ˚ ` k p17qq when the ˚ q is equal to the point pr1 pn prpn corresponding ring intensity profiles at the specific radius of = 17. r and p17 It can be seen that the intensity ofppoint 17
* k p17q is equal to the point when the It can be seen that the intensity of point 17 ) represents p17 ( n * ) 11a. consequent point shifts by k p17q, as shown in Figure Actually, the shiftpvalue 17 (n k the ring that point is rotated by φ 17 in the Furthermore, we can define the 17 at k a17radius k 17 in spatial Figure domain. 11a. Actually, the shift value consequent shifts by , as ofshown relationship between the rotation angle φr and the shift value k opt prq at the specific radius, which is represents the ring that is rotated by 17 at a radius of 17 in the spatial domain. Furthermore, we defined as follows: can define the relationship between φthe rotation angle and the shift value kopt r at (13) the r “ k opt prq ˆ φopt,0 prqr specific radius, which is defined as follows: where k opt prq is the shift value at radius r and φopt,0 prq is the optimal step angle described in Equation (4). r kopt r opt ,0 (r) (13)
where kopt r is the shift value at radius r and opt ,0 ( r ) is the optimal step angle described in Equation (4).
Appl. Sci. 2016, 6, 140
12 of 21
Appl. Sci. 2016, 6, 140
12 of 21
(a)
(b)
(c) Figure 11. An example of the ring intensity profile at the specific radius (radius = 17). (a) Original
Figure 11. An example of the ring intensity profile at the specific radius (radius = 17). (a) Original image; (b) Rotated image (30°, CCW); (c) Shift point k (17) in the profiles with rotation angles. image; (b) Rotated image (30˝ , CCW); (c) Shift point kp17q in the profiles with rotation angles.
Before the rotation estimation, the shift value kopt r at radius r should be calculated. The Before the rotation estimation,is the shift value radius r should be calculated. opt prq cross correlation measurement employed to kfind the atmaximum correlation of the two The ring cross correlation measurement is employed to find the maximum correlation of the two ring intensity profiles intensity profiles by a shift value, the so‐called ring‐shifted correlation. Therefore, the shift value by a shift value, the so-called ring-shifted correlation. Therefore, the shift value can be derived; can be derived; with the measurement with maximum correlation, the shift value can be defined as: with the measurement with maximum correlation, the shift value can be defined as: kopt r arg max( r ( k ))
(14)
k [0, N r ]
k opt prq “ argmaxpδr pkqq
(14)
r . where N r represents the number of pixels at radius kPr0,N s r
Nr
) pr (n))r. ( pr (n k ) pr (n k )) ( pat where Nr represents the number of pixels r ( nradius r (k ) Nr ř n “0
n0 Nr
Nr
( p (n) p (n)) 1( p (n k ) p (1n)) 2
r
r
r
(15)
2
r
0 n 0 pn ` kq ´ p pn ` kqq ppnrpnq ´ pr pnqq ¨ pp r r
(15) δr pkq “ d Based on the definitions given from Equations (13)–(15), the rotation angle of the candidate can Nr Nr ř 2 2 ř 1 be estimated at only a single radius. According to Equation (4), the accuracy of the rotation angle 1 ppr pnq ´ pr pnqq ¨ ppr pn ` kq ´ pr pnqq depends on the optimal step angle the optimal step angle is inversely n “0 n“0 and the radius. Obviously, proportional to the radius. Therefore, the rotation angle can be accurately estimated by integrating Based on the definitions given Equations (13)–(15), the rotation angle of the candidate can all ring weighting coefficients wr from in the range between Rmin and Rmax . Hence, the rotation angle of be estimated at only a single radius. According to Equation (4), the accuracy of the rotation angle the most likely candidate is defined as follows: depends on the optimal step angle and theRradius. Obviously, the optimal step angle is inversely objrotation wr angle kopt r can optbe ) proportional to the radius. Therefore, the estimated by integrating (16) all ,0 ( raccurately rR ring weighting coefficients wr in the range between Rmin and Rmax . Hence, the rotation angle of the wr N r N sum Nsum is the number of pixels for all radii and N r is the number of pixels for most where likely candidate is , defined as follows: max
min
the specific radius r, respectively
θobj “
Rÿ max r“ Rmin
wr ¨ k opt prq ¨ φopt,0 prq
(16)
Appl. Sci. 2016, 6, 140
13 of 21
where wr “ Nr {Nsum , Nsum is the number of pixels for all radii and Nr is the number of pixels for the specific radius r, respectively In addition, two neighbors of the shift value k prq at radius r, k prq ´ 1 and k prq ` 1, and the corresponding correlation coefficients, δr pk prq ´ 1q and δr pk prq ` 1q, are input into the fitting model to refine the accuracy of the rotation angle. Computing the best fitting parabola equation involves solving a system equation, which is written as: δpxq “ ax2 ` bx ` c
(17)
where x “ k prq ´ 1, k prq , k prq ` 1. Three unknown parameters a, b and c are calculated using Cramer’s rule. When the parabola parameters are determined, the optimal shift value is calculated by: k˚opt prq = ´
b 2a
(18)
Therefore, the greatest accuracy for the rotation angle θ˚opt is refined as: θ˚opt “
Rÿ max
wr ¨ k˚opt prq ¨ φopt,l prq
(19)
r“ Rmin
5. Implementation of the Proposed Method According to the architecture mentioned in Section 2, the implementation details of the two phases are described in the following subsection. 5.1. Template Preprocessing Phase A template preprocessing phase determines the multi-resolution hybrid-projection information for the object of interest in the template image P. The precise steps are presented as follows: Step 1: The template image P is input specified manually. Step 2: The multi-resolution template image Pl is constructed, where l “ 0, 1, . . . , L, l represents several image pyramid levels and L is the maximum image pyramid level. Step 3: Ring-projection transform is used to determine ring-projection values CP,l prq within different image pyramid levels, where r P rRl,min , Rl,max s. Step 4: After the ring-projection transform process, the variance-project values σP,l prq for each image pyramid level are determined through variance-project transform, where r P rRl,min , Rl,max s. Step 5: The hybrid-projection values HP,l prq for each image pyramid level are determined using ring-projection values CP,l prq, variance-projection values , σP,l prq and the corresponding weighting coefficients for the ring projection wm,l prq and variance projection wσ,l prq. The weighting coefficients, wm,l prq and wσ,l prq, are calculated by image entropy. Step 6: An appropriate threshold of similarity, Tmin,l , for different inspection conditions is identified. The levels of the image pyramid, weighting coefficients, hybrid projection values, and threshold are used for the online-alignment phase of the image pyramid technique, and the similarity measure with the image pyramid search technique, respectively. 5.2. Online Alignment Phase The online-alignment phase identifies the best candidates in the inspection image which have an arbitrary rotation angle. Also, the rotation angle is further determined using the ring-shift projection. The precise steps conducted at multi-resolution inspection image are presented as follows:
Appl. Sci. 2016, 6, 140
14 of 21
Step 1: A multi-resolution inspection image Sl is constructed, where l “ 0, 1, . . . , L, and L corresponds to the maximum image pyramid level in the template preprocessing phase. Step 2: The search block initializes at location (x = 0, y = 0) in the highest inspection image pyramid level. This image is the same size as the template image on the highest image pyramid level. Step 3: Ring- and variance-projection transform are used to determine the ring-projection values CS,l px, y, rq and variance-projection values σS,l px, y, rq from the search block at location (x, y) on image pyramid level l. Step 4: The hybrid-projection values HS,l px, y, rq on image pyramid level l are obtained using CS,l px, y, rq, σS,l px, y, rq and the weighting coefficients, wm,l prq and wσ,l prq, which are calculated in the template preprocessing phase. Step 5: When estimating the similarity coefficient between the template image and search block on image pyramid level l, if the similarity coefficient exceeds the pre-defined threshold for similarity, the location (x, y) is stored in the candidate list. Step 6: The search block in the inspection image is moved and Steps 3–6 are repeated until the similarity coefficient in the inspection image is computed for all locations. When the candidate list for the highest image pyramid level is determined, the image pyramid search process is used to increase the speed of the candidate selection and localization processes. On the next lower image pyramid levels, the search block location inherits the location from the candidate list that has been stored on the previous pyramid level. The search block must only be searched within a small range of displacement. Steps 3–5 in the online-alignment phase are used to determine the most likely candidate on the lower image pyramid levels. The hybrid-projection transform for multi-resolution using the image pyramid search technique is described in Section 3.3. Finally, the best candidate is identified on the lowest image pyramid and then the rotation estimation process calculates the rotation angle using the ring-shifted projection for the best candidate. 6. Experimental Results This section describes a series of experimental results that show the performance of the proposed method. Various images were used in different experiments to verify the proposed algorithm. To verify the proposed method, the results are compared with [15,21]. The similarity threshold Tmin,0 is 0.8 for both this study and the compared methods. All experiments were performed on a personal computer with an Intel Core i7 3.4 GHz CPU and 8 GB of memory using Visual Studio 2008. 6.1. Rotation Estimation Figure 12 shows the rotation test images, which are used to evaluate the rotation accuracy of the proposed rotation-estimation technique. To estimate the rotation accuracy, the simulated rotated images are generated by rotating the original image at rotation angles from 0˝ to 359˝ in increments of 5˝ . For obtaining the rotation angle, the least second moment method is utilized to estimate the angle between the template and inspection image in [15]. Also, the pre-constructed rotated templates score and piecewise linear model are applied to estimate the rotation angle in [21]. Detailed comparison results are shown in Table 2. The error Er is calculated using the following equation: Er “ |θa ´ θe |
(20)
where θa and θe are the actual and estimated rotation angles, respectively. To provide an overall accuracy evaluation, three performance indices of error Er are used to quantitatively show the performance: the mean of error Er_m , the standard deviation of error Er_std , and the maximum error Er_max .
Appl. Sci. 2016, 6, 140
15 of 21
Appl. Sci. 2016, 6, 140
15 of 21
(a)
(b)
(c)
Figure 12. Images for the rotation test. (a) IC image 1; (b) Metal image; (c) IC image 2. Figure 12. Images for the rotation test. (a) IC image 1; (b) Metal image; (c) IC image 2. Table 2. Errors for the rotation accuracy for both the proposed and compared methods. Table 2. Errors for the rotation accuracy for both the proposed and compared methods. Case Case
Performance Index Proposed Method Performance Proposed Method 0.023 Er_m ( Index ) ˝ Er_m ( () ) 0.023 Figure 12a 0.020 Er_std Figure 12a Er_std (˝ ) 0.020 0.086 Er_max ( ) Er_max (˝ ) 0.086 0.010 Er_m˝ ( ) Er_m ( ) 0.010 Figure 12b 0.007 Er_std(˝ () ) Figure 12b Er_std 0.007 ˝ 0.026 Er_max( () ) Er_max 0.026 0.036 Er_m(˝ () ) Er_m 0.036
Figure 12c Figure 12c
Er_std Er_std(˝ () ) Er_max (˝ ) Er_max ( )
0.027 0.027 0.102 0.102
[15] [21] [15] 62.352 [21] 0.565
0.565 48.950 62.352 0.376 0.376 48.950 1.390 174.422 1.390 174.422 1.004 90.306 1.004 90.306 0.447 52.024 0.447 52.024 1.914 1.914 174.422 174.422 0.488 94.930 0.488 94.930 0.393 47.098 47.098 0.393 1.607 174.422 1.607 174.422
As seen in Table 2, the errors in the mean, standard deviation, and maximum in all test cases, As seen in Table 2, the errors in the mean, standard deviation, and maximum in all test cases, which derived byby using thethe proposed ring-shifted technique, are less 0.036˝0.036°, , 0.027˝0.027°, , and 0.102 which are are derived using proposed ring‐shifted technique, are than less than and ˝ , respectively. In [15], the least second moment method cannot provide precise results because of the 0.102°, respectively. In [15], the least second moment method cannot provide precise results because of the integer simple integer data representation. the pre‐constructed for calculating the simple data representation. Besides,Besides, the pre-constructed methodmethod for calculating the rotation rotation angle in accuracy [21], the accuracy of angle depends on the of number of pre‐constructed angle in [21], the of angle depends on the number pre-constructed rotation rotation template template and the location which are by the cross-correlation normalized cross‐correlation (NCC) images andimages the location which are estimated byestimated the normalized (NCC) method. If the method. If the NCC give method cannot give a ofcorrect location of the template within the the inspection NCC method cannot a correct location the template within the inspection image, rotation image, the will rotation have of failed. The number of pre‐constructed rotation template estimation have estimation failed. Thewill number pre-constructed rotation template images is nine in this images is nine in this test. The proposed method is seen to be superior to the other methods. For the test. The proposed method is seen to be superior to the other methods. For the ring-shifted technique, * ring‐shifted technique, technique the rotation‐estimation technique uses k˚opt the prq optimal shift value by kopt r from the rotation-estimation uses the optimal shift value by increasing the radius Rincreasing . Inradius this experiment, ring-shifted technique providesring‐shifted an accurate technique estimation Rmax . In this from Rmin the to proposed experiment, the proposed min to Rmaxthe when the template is rotated within the inspection image using arbitrary rotation angles. provides an accurate estimation when the template is rotated within the inspection image using arbitrary rotation angles. 6.2. Performance on Images with Added Noise 6.2. Performance on Images with Added Noise This section considers the effect of noise on the inspection images. The test images and the corresponding template images shown in Figure 13.inspection The test images contain simple and and complex This section considers the are effect of noise on the images. The test images the backgrounds. The signal-to-noise ratio (SNR) for the Gaussian noise applied to the test images varies corresponding template images are shown in Figure 13. The test images contain simple and complex from 10 to 35 dB in increments of 5 dB. Figure 14 shows the errors in rotation at different noise levels. backgrounds. The signal‐to‐noise ratio (SNR) for the Gaussian noise applied to the test images varies Table 3 lists the translation errors. The translation errors are calculated using the Euclidean distance from 10 to 35 dB in increments of 5 dB. Figure 14 shows the errors in rotation at different noise levels. between the real and matching positions. The rotation errors are calculated using Equation (20). Table 3 lists the translation errors. The translation errors are calculated using the Euclidean distance In this experiment, the maximum rotation errors for the proposed method are 7.157˝ and 0.405˝ between the real and matching positions. The rotation errors are calculated using Equation (20). whenIn this experiment, the maximum rotation errors for the proposed method are 7.157° and 0.405° the SNR ratio is 10 and 15 dB, respectively. The maximum translation errors for the proposed method are 0 when the SNR is either 10 or 15 dB. In [21], the original template image is used to identify when the SNR ratio is 10 and 15 dB, respectively. The maximum translation errors for the proposed the location the template image in the images by using NCC template measurement. method are of 0 when the SNR is either 10 test or 15 dB. In [21], the the original image Because is used the to identify the location of the template image in the test images by using the NCC measurement.
Appl. Sci. 2016, 6, 140
16 of 21
Appl. Sci. 2016, 6, 140
16 of 21
test images are not rotated in this experiment, the translation error for the test images with different Because the test images are not rotated in this experiment, the translation error for the test images Appl. Sci. 2016, 6, 140 16 of 21 Gaussian noise levels is not generated. By using the hybrid projection transformation, the features with different Gaussian noise levels is not generated. By using the hybrid projection transformation, are obtained from the mean and variance values of the specific radius; therefore, it can significantly Because the test images are not rotated in this experiment, the translation error for the test images the features are obtained from the mean and variance values of the specific radius; therefore, it can resistsignificantly the noise inresist the inspection images. It is seen in this experiment that the overall performance of with different Gaussian noise levels is not generated. By using the hybrid projection transformation, the noise in the inspection images. It is seen in this experiment that the overall the features are obtained from the mean and variance values of the specific radius; therefore, it can the proposed method is considerably better than that of the other methods when the SNR is greater performance of the proposed method is considerably better than that of the other methods when the significantly resist the noise in the inspection images. It is seen in this experiment that the overall than SNR is greater than 10 dB. 10 dB. performance of the proposed method is considerably better than that of the other methods when the SNR is greater than 10 dB.
(a)
(b)
(a) (b) Figure 13. Test images. The sizes of test images are (a) 414 × 525; (b) 800 × 600; and the corresponding Figure 13. Test images. The sizes of test images are (a) 414 ˆ 525; (b) 800 ˆ 600; and the corresponding template images are (a) 170 × 170; (b) 120 × 120. Figure 13. Test images. The sizes of test images are (a) 414 × 525; (b) 800 × 600; and the corresponding template images are (a) 170 ˆ 170; (b) 120 ˆ 120. template images are (a) 170 × 170; (b) 120 × 120.
Proposed method Proposed method
Reference [15] Reference [15]
Reference [21] Reference [21]
Error (deg) Error (deg)
12 12 10 10 8 8 6 6 4 4 2 2 0 0 ‐2 ‐2
10 10
15 15
Proposed method Proposed method
20 25 20 25 SNR ratio (dB) SNR ratio (dB) (a) (a)
30 30
35 35
Reference [15] Reference [15]
Reference [21] Reference [21]
20 25 20 25 SNR ratio (dB) SNR (dB)
30 30
Error (o) Error (o)
25 25 20 20 1515 1010 55 00 ‐5‐5
10 10
15 15
(b)
35 35
Figure 14. Rotation errors for the matching results with Gaussian noise. (a) The case of Figure 13(a); (b) The case of Figure 13(b).
Appl. Sci. 2016, 6, 140 Appl. Sci. 2016, 6, 140
17 of 21 17 of 21
Figure 14. Rotation errors for the matching results with Gaussian noise. (a) The case of Figure 13(a); Table 3. Translation error for the matching results with Gaussian noise. (b) The case of Figure 13(b).
Translation Error (pixel) Table 3. Translation error for the matching results with Gaussian noise. SNR Ratio (dB)
10 SNR Ratio (dB) Proposed method10 0 15 Figure Proposed method 13a [15] 0 0 0 0 1 Figure 13a [15] [21] 0 Proposed method0 0 0 [21] Figure Proposed method 13b [15] 1.414 0 0 [21] 0 Figure 13b [15] 1.414 1 [21] 0 0
15 20 25 30 35 Translation Error (pixel) 20 30 0 0 0 25 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0
0
35 0 0 0 0 0 0
6.3. Weighting Influence in Candidate Selection and Localization 6.3. Weighting Influence in Candidate Selection and Localization
The hybrid-projection transformation combined the weighted average of wm,l prq and wσ,l prq to hybrid‐projection transformation combined the weighted average of wm ,l ( r )strategies and w ,l (of r ) these select andThe localize the candidate. Here, the performance with the different weighted to select and localize the candidate. Here, the performance with the different weighted strategies of two parameters are studied, the proposed image entropy method is compared with the user defined these two parameters are studied, the proposed image entropy method is compared with the user method. Three sets of parameters (wm,l prq, wσ,l prq) are selected as (0.3, 0.7), (0.5, 0.5), and (0.7, 0.3) defined method. Three sets of parameters ( wm ,limage ( r ) , wpyramid , l ( r ) ) are selected as (0.3, 0.7), (0.5, 0.5), and corresponding to every specified radius for all levels, respectively. The test images (0.7, 0.3) corresponding to every specified radius for all image pyramid levels, respectively. The test and corresponding template images are shown in Figure 15. The Gaussian noises from 10 to 35 dB in images and corresponding template images are shown in Figure 15. The Gaussian noises from 10 to increments of 5 dB are applied in the test images for evaluation. Here, two performance indices of 35 dB in increments of 5 dB are applied in the test images for evaluation. Here, two performance errors are used to quantitatively show the performance: the mean of error Et_m and Er_m for translation indices of errors are used to quantitatively show the performance: the mean of error Et_m and Er_m for and rotation; the standard deviation of error Et_std and Er_std for translation and rotation, respectively. translation and rotation; the standard deviation of error Et_std and Er_std for translation and rotation, The results of translation and rotation errors are listed in Table 4, where the mean translation and respectively. The results of translation and rotation errors are listed in Table 4, where the mean ˝ , 0.17 pixels, and 0.92˝ , respectively. It is rotation errors of our proposed are 0 pixels, 0.004 translation and rotation errors method of our proposed method are 0 pixels, 0.004°, 0.17 pixels, and 0.92°, obvious that the results of our proposed methods are superior to the user defined method. According respectively. It is obvious that the results of our proposed methods are superior to the user defined to themethod. architecture in Figure 3, architecture the accuratein rotation estimated results are obtained onare correct According to the Figure 3, the accurate rotation estimated based results obtained based on correct locations of the matching candidates. As seen in the results of Figure 15a, locations of the matching candidates. As seen in the results of Figure 15a, wrong locations are obtained wrong locations are obtained by the user defined method, thus, poor rotation estimated results are by the user defined method, thus, poor rotation estimated results are obtained in this case. obtained in this case.
(a)
(b)
Figure 15. Test images. Sizes of all test images are 800 × 600, and the corresponding template images
Figure 15. Test images. Sizes of all test images are 800 ˆ 600, and the corresponding template images are (a) 150 × 150; (b) 120 × 120. are (a) 150 ˆ 150; (b) 120 ˆ 120. Table 4. Matching errors with different weighting coefficients.
Table 4. Matching errors with different weighting coefficients. Errors Methods
Figure 15a
Translation (pixel)
Et_m EFigure t_std Errors Proposed method 0 0 Translation (pixel) Methods 80.69 wm ,l (r ) = 0.3, w ,l ( r ) = 0.7 166.88 Et_m Et_std 11.02 wm ,l (r ) = 0.5, w ,l ( r ) = 0.5 160.91
Proposed method
0
0
wm,l prq = 0.3, wσ,l prq = 0.7
166.88
wm,l prq = 0.5, wσ,l prq = 0.5 wm,l prq = 0.7, wσ,l prq = 0.3
Figure 15b
Rotation ( )
15aEr_m
Er_std 0.004 0.001 Rotation (˝ ) 205.23 15.22 Er_std Er_m 241.88 9.92
0.004
0.001
80.69
205.23
160.91
11.02
184.49
83.11
Translation (pixel)
Rotation ( )
Et_m Et_std FigureE15b r_m Er_std 0.17 0.37 (pixel) 0.92 Rotation 2.02 (˝ ) Translation 0.33 6.27 0.47 9.03 Et_std Er_std Et_m Er_m 0.33 6.27 0.47 9.03
0.17
0.37
0.92
2.02
15.22
0.33
0.47
6.27
9.03
241.88
9.92
0.33
0.47
6.27
9.03
149.39
22.44
0.33
0.47
6.27
9.03
Appl.Appl. Sci. 2016, 6, 140 Sci. 2016, 6, 140 Appl. Sci. 2016, 6, 140 wm ,l (r ) = 0.7, w ,l ( r ) = 0.3
18 of 21 18 of 21 184.49
83.11
149.39
22.44
0.33
6.27
0.47
6.4. Computational Performance in Real Captured PCB Images
18 of 21 9.03
83.11 149.39 0.33 6.27 wm ,l (r ) = 0.7, w ,l ( r ) = 0.3 184.49 22.44 0.47 9.03 6.4. Computational Performance in Real Captured PCB Images Additional experiments were performed using real captured PCB images. The images were
captured when an experiments object was arbitrarily moving andreal rotating. All PCB test and corresponding Additional were performed using captured images. The images template were 6.4. Computational Performance in Real Captured PCB Images images are shown in Figures 16–18. The sizes of the test images are 800 ˆ 600. This experiment captured when an object was arbitrarily moving and rotating. All test and corresponding template Additional experiments performed real PCB images. The were assessed performance of computation and correctness according to the matching results. The matching images are shown in Figures were 16–18. The sizes using of the test captured images are 800 × 600. This images experiment captured when an object was arbitrarily moving and rotating. All test and corresponding template assessed of computation correctness according the matching The results for theperformance proposed method and that ofand references [15] and [21] areto denoted by red,results. blue and green images are shown in Figures 16–18. The sizes of the test images are 800 [21] are denoted by red, × 600. This experiment matching results for the proposed method and that of references [15] and boxes, respectively. To allow a fair comparison, the comparison methods are also optimized using the assessed performance of computation and correctness according to the matching results. The blue and green boxes, respectively. allow a fair comparison, the comparison methods are also image pyramid search framework. TheTo statistical results for efficiency and the effect of pyramid levels matching results for the proposed method and that of references [15] and [21] are denoted by red, are optimized using the image pyramid search framework. The statistical results for efficiency and the summarized in Table 5. Here it is shown that the computational burden is significantly reduced blue and green boxes, respectively. To allow a fair comparison, the comparison methods are also effect of pyramid levels are summarized in Table 5. Here it is shown that the computational burden when the image pyramid search framework is used. optimized using the image pyramid search framework. The statistical results for efficiency and the is significantly reduced when the image pyramid search framework is used. effect of pyramid levels are summarized in Table 5. Here it is shown that the computational burden is significantly reduced when the image pyramid search framework is used.
(a)
(b)
(a)
(b)
(c)
(d)
Figure 16. Printed circuit board (PCB) case1 images. (a–c) real and (d) template image (200 × 200 pixels). Figure 16. Printed circuit (c) board (PCB) case1 images. (a–c) (d) real and (d) template image (200Figure 16. Printed circuit board (PCB) case1 images. (a–c) real and (d) template image (200 × 200 pixels). ˆ 200 pixels).
(a)
(b)
(a)
(b)
(c)
(d)
Figure 17. PCB case2 images. (a–c) real images and (d) template image (200 × 200 pixels). (c) (d) Figure 17. PCB case2 images. (a–c) real images and (d) template image (200 × 200 pixels).
Figure 17. PCB case2 images. (a–c) real images and (d) template image (200 ˆ 200 pixels).
Appl. Sci. 2016, 6, 140
19 of 21
Appl. Sci. 2016, 6, 140 Appl. Sci. 2016, 6, 140
19 of 21 19 of 21
(a) (a)
(b) (b)
(c) (c)
(d) (d)
Figure 18. PCB case3 images. (a–c) real images and (d) template image (200 × 200 pixels). Figure 18. PCB case3 images. (a–c) real images and (d) template image (200 ˆ 200 pixels). Figure 18. PCB case3 images. (a–c) real images and (d) template image (200 × 200 pixels). Table 5. The computational burden for the proposed and compared methods. Table 5. The computational burden for the proposed and compared methods. Table 5. The computational burden for the proposed and compared methods.
Average Execution Time (s) Average Execution Time (s) Average Execution Time (s) Figure 16 Figure 17 Figure 18 Figure 19 Methods Figure 16 Figure 16Figure 17 Figure 18 Figure 17 Figure 18 Figure 19 Figure 19 Pyramid levels = 2 Pyramid levels = 2 Pyramid 0.124 levels = 2 Proposed method 0.316 0.126 0.322 Proposed method 0.316 0.124 0.126 0.322 [15] 0.302 0.143 0.147 0.322 0.337 Proposed method 0.316 0.124 0.126 [15] 0.302 0.143 0.147 0.337 0.337 [15] 0.057 0.302 0.143 0.147 [21] 0.03 0.031 0.062 [21] 0.057 0.03 0.031 0.062 [21] 0.057 0.03 0.031 0.062 Pyramid levels = 1 Pyramid levels = 1 Pyramid 0.766 levels = 1 Proposed method 1.721 0.765 1.731 Proposed method 1.721 0.766 0.765 1.731 [15] 3.701 1.393 1.401 1.731 3.842 Proposed method 1.721 0.766 0.765 [15] 1.393 1.401 3.842 3.842 [15] 3.701 3.701 1.393 1.401 [21] 0.755 0.364 0.366 0.767 [21] 0.364 0.366 0.767 0.767 [21] 0.755 0.755 0.364 0.366 Pyramid levels = 0 (without image pyramid search technique) Pyramid levels = 0 (without image pyramid search technique) pyramid search technique) Proposed method Pyramid levels 19.5 = 0 (without image 8.318 8.329 21.183 Proposed method 19.5 8.318 8.329 21.183 Proposed method 19.5 8.318 8.329 [15] 30.458 14.85 14.465 21.183 32.172 [15] 30.458 14.85 14.465 32.172 [15] 11.526 30.458 5.694 14.85 14.465 [21] 5.737 32.172 11.526 [21] 11.526 11.526 5.694 5.694 5.737 [21] 5.737 11.526 11.526
Methods Methods
Time Time
Time
(a) (a)
(b) (b) Figure 19. Cont.
Appl. Sci. 2016, 6, 140
20 of 21
Appl. Sci. 2016, 6, 140
20 of 21
(c)
(d)
FigureFigure 19. PCB case4 images. (a–c) real images and (d) template image (206 × 201 pixels). 19. PCB case4 images. (a–c) real images and (d) template image (206 ˆ 201 pixels).
In these test cases, the mismatch for the method of [21] is clearly seen in the PCB test cases, such In these test cases, the mismatch for the method of [21] is clearly seen in the PCB test cases, such as Figures 16b,c, 17b,c, and 18b,c (shown as green boxes). This method is sensitive to test images as Figure 16b,c, Figure 17b,c, and Figure 18b,c (shown as green boxes). This method is sensitive to when the template is located in a complex background. In terms of the method of [15], the matching test images when the template is located in a complex background. In terms of the method of [15], results provide a more accurate localization of the template image in the test images (shown as blue the matching results provide a more accurate localization of the template image in the test images boxes). However, the results for the rotation angle are not sufficiently precise, such as in Figures (shown as blue boxes). However, the results for the rotation angle are not sufficiently precise, such as 16a,c and 19a–c. By contrast, the matching results for the proposed method are seen clearly in in Figures 16a,c and 19a–c. By contrast, the matching results for the proposed method are seen clearly Figures 16–18. They are superior to the results for the other methods. On the other hand, the image in Figures 16–18. They are superior to the results for the other methods. On the other hand, the image pyramid searching technique is used to enhance the performance in terms of efficiency. It can been pyramid searching technique is used to enhance the performance in terms of efficiency. It can been seen the computation advantage of the proposed method, the [15,21] are around 66 times, 95 times, seen the computation advantage of the proposed method, the [15,21] are around 66 times, 95 times, and 186 times. and 186 times. From the experimental results, the proposed method not only provides a correct location, but From the experimental results, the proposed method not only provides a correct location, but also estimates a correct and precise rotation angle for the template image in real captured PCB also estimates a correct and precise rotation angle for the template image in real captured PCB images. images. Based on the convincing results of this experiment, the proposed method can be used with Based on the convincing results of this experiment, the proposed method can be used with real-world real‐world images. However, the feature‐based matching method that use the SIFT descriptor is not images. However, the feature-based matching method that use the SIFT descriptor is not appropriate appropriate for the complex images, such as Figures 16–19, because of the heavy computational for the complex images, such as Figures 16–19 because of the heavy computational burden. There burden. There are more than 2300 feature points in these real captured images. The computation are more than 2300 feature points in these real captured images. The computation times for the real times for the real captured cases are 642.22, 620, 448.28, and 489.1 msec, respectively. The efficiency captured cases are 642.22, 620, 448.28, and 489.1 msec, respectively. The efficiency of the feature-based of the feature‐based method is slower than the proposed method. method is slower than the proposed method. 7. Conclusions 7. Conclusions A novel image alignment algorithm that uses rotation‐discriminating ring‐shifted projection for A novel image alignment algorithm that uses rotation-discriminating ring-shifted projection AOI applications is presented. It combines hybrid projection transformation and the ring shift for AOI applications is presented. It combines hybrid projection transformation and the ring shift technique. The hybrid projection transformation with the image pyramid searching technique can technique. The hybrid projection transformation with the image pyramid searching technique can significantly reduce the computation burden and own the unique and robust features in the significantly reduce the computation burden and own the unique and robust features in the alignment alignment process. The ring shift technique provides the rotation estimation between the template process. The ring shift technique provides the rotation estimation between the template image and the image and the searching image. The results show that the rotation estimation of the proposed searching image. The results show that the rotation estimation of the proposed method is superior to method is superior to other comparative methods. Furthermore, the novel image alignment other comparative methods. Furthermore, the novel image alignment algorithm can obtain accurate algorithm can obtain accurate and robust results in the scene image with rotation, translation, and and robust results in the scene image with rotation, translation, and noise. A series of experiments noise. A series of experiments verified the efficiency, accuracy, and robustness of the proposed verified the efficiency, accuracy, and robustness of the proposed algorithm. Furthermore, the proposed algorithm. Furthermore, the proposed method not only provides high accuracy with rotation, but method not only provides high accuracy with rotation, but also works well under noise influence and also works well under noise influence and translation. The various experiment results indicate that translation. The various experiment results indicate that this approach is suitable for accurate image this approach is suitable for accurate image alignment in AOI industrial inspections. alignment in AOI industrial inspections. Acknowledgments: The authors would like to acknowledge the support financial support of the National Science Acknowledgments: The authors would like to acknowledge the financial of the National Science Council Council of Taiwan, through its grant MOST 104‐2221‐E‐027‐029‐MY2. of Taiwan, through its grant MOST 104-2221-E-027-029-MY2. Author Contributions: Contribute the ideas of theof research and research supervision: Chin-Sheng Chen;Chen; Author Contributions: Contribute the ideas the research and research supervision: Chin‐Sheng Performing of the research and writing of the manuscript: Chien-Liang Huang. Performing of the research and writing of the manuscript: Chien‐Liang Huang. Conflicts of Interest: The authors declare no conflict of interest. Conflicts of Interest: The authors declare no conflict of interest.
Appl. Sci. 2016, 6, 140
21 of 21
References 1. 2. 3. 4. 5.
6. 7. 8. 9.
10. 11. 12. 13. 14. 15. 16. 17.
18.
19. 20. 21.
Zitová, B.; Flusser, J. Image registration methods: A survey. Image Vis. Comput. 2003, 21, 977–1000. [CrossRef] Huttenlocher, D.P.; Klanderman, G.A.; Rucklidge, W.J. Comparing images using the hausdorff distance. IEEE Trans. Pattern Anal. Mach. Intell. 1993, 15, 850–863. [CrossRef] Huttenlocher, D.P.; Lilien, R.H.; Olson, C.F. View-based recognition using an eigenspace approximation to the hausdorff measure. IEEE Trans. Pattern Anal. Mach. Intell. 1999, 21, 951–955. [CrossRef] Kwon, O.-K.; Sim, D.-G.; Park, R.-H. Robust hausdorff distance matching algorithms using pyramidal structures. Pattern Recogn. 2001, 34, 2005–2013. [CrossRef] Chen, C.-J.; Lai, S.-H.; Liu, S.-W.; Ku, T.; Yeh, S.Y. Optical PCB inspection system based on hausdorff distance. In Proceedings of SPIE 5679, Machine Vision Applications in Industrial Inspection XIII, 53, San Jose, CA, USA, 17 January 2005; pp. 53–61. Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [CrossRef] Mikolajczyk, K.; Schmid, C. A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27, 1615–1630. [CrossRef] [PubMed] Khotanzad, A.; Hong, Y.H. Invariant image recognition by zernike moments. IEEE Trans. Pattern Anal. Mach. Intell. 1990, 12, 489–497. [CrossRef] Mondal, T.; Mourya, G.K. An accelerated approach of template matching for rotation, scale and illumination invariance. In Control, Computation and Information Systems; Springer: Berlin Heidelberg, Germany, 2011; pp. 121–128. Ullah, F.; Kaneko, S.I. Using orientation codes for rotation-invariant template matching. Pattern Recogn. 2004, 37, 201–209. [CrossRef] Marimon, D.; Ebrahimi, T. Efficient rotation-discriminative template matching. In Progress in Pattern Recognition, Image Analysis and Applications; Springer: New York, NY, USA, 2007; pp. 221–230. Tang, Y.Y.; Cheng, H.D.; Suen, C.Y. Transformation-ring-projection (TRP) algorithm and its VLSI implementation. Int. J. Pattern Recogn. Artif. Intell. 1991, 5, 25–56. [CrossRef] Tsai, D.-M.; Chiang, C.-H. Rotation-invariant pattern matching using wavelet decomposition. Pattern Recogn. Lett. 2002, 23, 191–201. [CrossRef] Choi, M.-S.; Kim, W.-Y. A novel two stage template matching method for rotation and illumination invariance. Pattern Recogn. 2002, 35, 119–129. [CrossRef] Tsai, D.-M.; Tsai, Y.-H. Rotation-invariant pattern matching with color ring-projection. Pattern Recogn. 2002, 35, 131–141. [CrossRef] Lee, W.-C.; Chen, C.-H. A fast template matching method with rotation invariance by combining the circular projection transform process and bounded partial correlation. IEEE Signal Process. Lett. 2012, 19, 737–740. Lee, W.-C.; Chen, C.-H. A fast template matching method for rotation invariance using two-stage process. In Proceedings of the Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP 2009), Kyoto, Japan, 12–14 September 2009; pp. 9–12. Kim, H.; Araújo, S. Grayscale template-matching invariant to rotation, scale, translation, brightness and contrast. In Advances in Image and Video Technology; Mery, D., Rueda, L., Eds.; Springer: Berlin Heidelberg, Germany, 2007; Volume 4872, pp. 100–113. Tanimoto, S.L. Template matching in pyramids. Comput. Graph. Image Process. 1981, 16, 356–369. [CrossRef] Feng, G.C.; Yuen, P.C. Variance projection function and its application to eye detection for human face recognition. Pattern Recogn. Lett. 1998, 19, 899–906. [CrossRef] Sassanapitak, S.; Kaewtrakulpong, P. An efficient translation-rotation template matching using pre-computed scores of rotated templates. In Proceedings of the 6th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON 2009), Pattaya, Chonburi, Thailand, 6–9 May 2009; pp. 1040–1043. © 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
