Enhancement Matching Algorithms Using Fusion

World Applied Sciences Journal 6 (6): 759-763, 2009 ISSN 1818-4952 © IDOSI Publications, 2009

Enhancement Matching Algorithms Using Fusion of Multiple Similarity Metrics for Sonar Images Hatem A. Khater, Ahmed S. Gad, Ehab A. Omran and Ayman A. Abdel-Fattah Department of Naval Research and Development, Egyptian Navy, Alexandria, Egypt Abstract: The goal of this paper is to develop a matching technique for sonar and underwater images where it is used for a range of applications including stereo vision, classification of sonar images, underwater image registration and mosaicing,..., etc. The paper presents a novel scheme to improve the performance of image matching algorithms using a combination of independent matching similarity metrics. A number of corner similarity metrics have been developed to facilitate matching, however, any individual metric has a limited effectiveness depending on the content of images to be registered and the different types of distortions that may be present. This paper explores combining corner similarity metrics to produce more effective measures for corner matching. In particular the combination of two similarity metrics is investigated using experiments on a number of images exhibiting different types of transformations and distortions. The results suggest that a linear combination of different similarity metrics may produce more accurate and robust assessments of corner similarity. Key words: Sonar image matching

Score fusion

Underwater image registration

INTRODUCTION

the matching algorithm due to the different imaging conditions and/or due to the different spectral sensitivity of the sensors [3]. The choice of the feature description and similarity measure has to consider these factors. In general, features should be distinct with respect to their neighborhoods, invariant with respect to geometric and radiometric influences and stable with respect to noise [4]. This paper is organized as follows. Section 2 briefly describes estimation and selection of feature points as well as the inter-frame corner matching strategies employed. Section 3 describes score normalization and the proposed linear score combination method. Section 4 presents the results of testing the algorithm on a number of images and Section 5 provides some concluding remarks.

Side-Scan sonar systems are used to image the sea bottom in order to detect man–made objects that can be distinguished from the background structure of the sea bottom and to detect the other bottom [1]. Also, these systems are used to detect the wrecks and obstructions that may be dangerous to surface and subsurface navigation which lie between sounding lines. The robust detection of feature points constitutes a fundamental stage in sonar and underwater image characterization and matching [2]. The employment of feature points (such as corners) to find corresponding points within multiple images is an important step in many image processing and computer vision applications. These include image registration, motion tracking, mosaic construction and medical image fusion. Feature matching is generally referred to as the correspondence problem. The problem is how to automatically match corresponding features from two images, while at the same time not assigning matches incorrectly. The match is assigned to the corner with the highest matching score. In the feature matching step, errors may arise due to false feature detection or any image distortions that may be present. Corresponding features may appear to be dissimilar to

Estimation and Selection of Feature Points: The general image registration system extracts a set of features (closed-boundary regions, edges, contours, line intersections, corners, etc.) from the unregistered images which are either manually or automatically detected [3]. The basic features that we used for matching were corners. As described in the previous section, the detector should have good localization accuracy and should not be sensitive to the expected image distortions.

Corresponding Author: Dr. Ahmed S. Gad, Naval R and D Department, Egyptian Navy, Alexandria, Egypt

759

World Appl. Sci. J., 6 (6): 759-763, 2009

A number of algorithms for corner extraction have been tested in recent years [5, 6]. These may be classified into two sets. Algorithms in the first set are based on extracting edges and then finding the points having high curvature or looking for points where edge segments are significant. The second and largest set consists of algorithms that look for corners directly in the grey-level image. In this paper we concentrate on the Modified Harris corner detector [5,7] which belongs to this latter set of corner detectors. Harris Corner Detector: The original Harris corner detector computes a 2 × 2 autocorrelation matrix M as shown in Equation (1). A Corner Response Function (CRF) is formulated as Equation (3) and used for assessing “cornerness” where k is a constant (typically set to 0.04 [7]). I is the grey-level intensity image and Ix and Iy are horizontal and vertical intensity image gradient components. Sums are taken over a predefined window size (typically 3×3) around each pixel. A Gaussian smoothing filter with = 2 is used to process the gradient images Ix and Iy. The suitable range of the threshold for CRF for the pixel to be a corner candidate is in between 10000-1000000 depending on the image contents [7]. We used a modified Harris corner detector [5] which accomplished the same performance as the original version, but has much better stability and localization in addition to a much lower computational cost. A Gaussian filter with = 0.5 is used for smoothing the gradient images. The small standard deviation helps to decrease any displacement effects. In the modified version of the Harris detector the gradient images are measured more accurately which improves system stability [8]. Nonmaximum suppression using a 3x3 mask is also applied and a threshold is used to find candidate corners [9]. ∑ I ∑ I I  M=    ∑ I I ∑ I 

(1)

Ix = I/ x, Iy = I/ y

(2)

2

x

x

y

2

x

y

y

CRF = det (M) - k (trace (M))

(3)

Feature Matching: Features denote the existence and setting of structures; therefore, relying on features reduces the search process and makes high-level understanding of images easier without losing the most important information from the intensity image [3]. Block matching compares point features by extracting a

small image area (block) centred on each pixel in the first image and searching for a highly similar block in the second image [10]. For a given pair to be considered as a candidate match, correlation score must be higher than a given threshold. For each feature point in the first image, we have a set of candidate feature point matches from the second image. The corner sets U={u1,u2, …, u } and Z={z1,z2, …, z } for each image respectively result in a corner pair set UZ = {ui zj, i {1,…, }, j {1,…, } }, representing all possible matching pairs, where and are the number of corners in the U and Z sets, respectively. As features might be affected by the distortion or partially overlapping images, corners might be present in one image but not another, leading to falsely matched corner pairs in UZ. A constraint on the correlation score is then applied to pick the most reliable matches [4, 10,11]. Measuring Similarity for Image Registration: Similarity measures are used to register images by finding an accurate match between an input image and transformed versions of the reference image. Some of the similarity measuring techniques discussed in [3,12] include correlation, normalized cross-correlation, statistical correlation, match filters, phase-correlation, sum of absolute differences, root mean square and masked correlation. The extracted feature sets in the reference and input images must have enough common points. The feature descriptors should be invariant to the assumed distortions. Simultaneously, they have to be distinguishable enough to be able to differentiate among various features as well as sufficiently stable so as not to be influenced by slight unexpected feature variations and noise. The performance of the feature detection method, the reliability of feature matching estimation and the acceptable approximation error need to be considered too. In this paper, we suggest to empirically compare and validate the effectiveness of different matching strategies that aim to reject false matches. We use Gradient Direction Similarity Measure (GDSM) and Mutual Correlation Coefficient (MCC). They are evaluated and selected to be among the best metrics according to their ability to reduce the number of false matches in a given match set, while preserving the good ones [13]. Mutual Correlation Coefficient (MCC): The MCC is based on comparing blocks of intensity values [14]. It has the advantage of producing stable and reliable results over a wide range of viewing conditions. The MCC is defined as follows:

760

World Appl. Sci. J., 6 (6): 759-763, 2009

MCC (= P , Q)

1 A

1

∑ ( P (m ') − P )(Q (m) − Q)

2

m m

' ∈

∈ Q

P

= SY

(4)

' m

where m and m' are the corresponding pixel position within the compared blocks, P(m) and Q(m') are intensity values at m and m', respectively, P and Q are the means (average intensity values) of the two blocks, 1 and 2 are standard deviations of the two blocks and A is the area of the blocks (N × N). The correlation coefficient can take values that range from -1 to +1, where -1 indicates no similarity at all and +1 indicates a perfect match (the highest possible similarity).

∂11 ∂12 ( m ') − ( m) ∂y ∂x

(5)

GYDSM ( P, Q) ∑

∂11 ∂12 (m ') − ( m) ∂y ∂x

(6)

m m

m m

' ∈

' ∈

∈

∈

blokc

bolkc

blokc

bolkc

(

(

(

Q

(

P

Q

)

P

)

)

)

= GDSM(P,Q) GXDSM(P,Q) + GYDSM(P,Q)

(m') + ∑ ∂I2 (m) ∑ ∂∂I1 x ∂x ' m

∈

bolk )(P c

m'

∈

lk )cP o b (

m'

∈

(10)

ck (P lo b )

sˆ = ( s − min( s )) /(max( s ) − min( s ))

= wsˆ , where ∑ w ∑

= sum

i

i

1

i

(11) (12)

i

sum = wsˆ + wsˆ where w + w = 1 1

D SM G

2

C M

1

2

(13)

RESULTS AND DISCUSSION The overall scheme of the system is shown in Figure 1. The matching ground truth tables for test images (Figures 2, 3 and 4) have been formed. These images include examples with translation, rotation and lighting distortions. All matching corners found by the algorithms have been compared with the truth table. In this case every point in the first image is connected to one or more points in the second image with the constraint that the matching score is above or under the set threshold depending on the correlation measure used. The match is considered correct only if the Euclidean distance is less than two pixels from the true position. The results of the weighted linear combination are illustrated in Figures 5, 6 and 7. The abscissa axis shows the weight range, w1, while the ordinate axis shows the total error on linear scales. The total error represents the number of false matched corners and false non-matched corners in the system. The results of the evaluation and the comparison between different metrics and the proposed hybrid algorithm HYB are shown in Table 1. The GDSM metric has performed significantly better than the other metric in the presence of translation and rotation distortion [13]. The MCC metric has outperformed the GDSM metric in its

(7)

 GXDSM(P,Q) GYDSM(P,Q)  + NGDSM(P,Q) = 2−  (8) SX SY  

= SX

∈

Score Normalization and Combination Proces: A MinMax method is used for score normalization to map the raw scores to the [0,1] range as shown in Equation (11)[8]. We indicate a raw matching score obtained from Equations (3) or (4) as S and a normalized score as . The functions max(S) and min(S) return the end points of the score range. Fusing the scores of several classifiers has proved to be a promising approach to improve the overall accuracy of pattern recognition systems [15]. We apply a weighted sum rule to combine GDSM, MCC, the GDSM and MCC similarity metrics[16]. The output of the combined similarity metric can be obtained by multiplying the normalized matching score by suitable weights, Wi, as shown in Equations (12) and (13). A threshold on the combined similarity score is then applied to pick the most reliable matches.

Gradient Direction Similarity Measure (GDSM): This is based on comparing blocks of gradient direction [13]. The gradient represents the rate of change of the intensity levels in an image and it is higher near edges and smaller in uniform areas. Three image attributes are used to compute GDSM score; these being I (grey-level intensity), Ix = I/ x, Iy = I/ y, where I/ x and I/ y are the horizontal and vertical gradients of the intensity images, which have a corner pair centred at blocks P and Q of size N×N pixels with m and m' referring to the corresponding pixels in the compared blocks. The estimation of the horizontal and vertical gradient component blocks for a pair of corners in P in the first image I1 and in Q in the second image I2 individually, is performed using the sum of absolute difference. GDSM is computed as in Equations (5) and (6) where the normalized version of Equation (7) is given by Equation (8) and the score ranges from 0, for two blocks that are not similar, to 2 for two identical blocks. Gradient evaluation is a pre-processing operation in many applications; so working with gradient features may not introduce any additional calculations. GXDSM ( P, Q) ∑

∂I ∂I ∑ ∂y1(m') + ∑ ∂y2 (m)

(9)

bloc k(P )

761

World Appl. Sci. J., 6 (6): 759-763, 2009 Table 1: The number of correct matched corners found for each metric at block size 9 × 9 Image

Distortion

MCC

GDSM

HYB

Kitchen

Translation

84

102

111

Building

Rotation

59

71

77

Objects

Lighting

85

77

93

I1

I2

Gradient Image Computation Corner Extraction Non Max Suppression

Block-Matching GDSM

Block-Matching MCC

Score Normalization

Score Normalization

Fig. 3: Sonar image showing mine-like targets (corners are marked as black squares)

Fusion Function Decision Threshold Accepted or Rejected

Fig. 1: Overall structure for the system

Fig. 4: Sonar image showing dolphins swimming (corners are marked as black squares). 55

Window size 5x5;( Translation Distortion );Threshold=0.8

50

GDSM MCC HYBRI D

45 40 35 30 25 20 15 10 5 0

Fig. 2: Sonar image showing sand waves (corners are marked as black squares)

0.1

0.2

0.3

0.4

0.5 w1

0.6

0.7

0.8

0.9

1

Fig. 5: Total error as a function of w1 for images in Figure 2

762

World Appl. Sci. J., 6 (6): 759-763, 2009 85

2.

Window size 5x5;( Rotation Distortion );Threshold=0.82

80 75

3.

GDSM MCC HYBRI

70 65

4.

60 55 50 45 40 35 0

0.1

0.2

0.3

0.4

0.5 w1

0.6

0.7

0.8

0.9

5.

1

Fig. 6: Total error as a function of w1 for images in Figure 3 54

6.

Window size 5x5;( Lighting Distortion );Threshold=0.78

52

7.

GDSM MCC HYBRI

50 48

8.

46 44 42

9.

40 38 36 0

0.1

0.2

0.3

0.4

0.5 w1

0.6

0.7

0.8

0.9

10.

1

Fig. 7: Total error as a function of w1 for images in Figure 4 robustness to lighting distortion. The performance of the hybrid algorithm is indicative of improved performance across the different types of distortions.

11.

12.

CONCLUSIONS These results suggest that a combination of corner similarity metrics may be used to improve the performance and robustness of corner matching systems for sonar images. The GDSM and MCC metrics extract different kinds of information from the images and hence can be fused so as to obtain a hybrid metric with different characteristics. The threshold value used on the combined similarity metric to establish a match is a useful means for controlling the balance between the different types of error in corner matching.

13.

14.

15.

16.

REFERENCES 1.

Stage, B. and B. Zerr, 1997. Detection of Objects on the Sea Bottom using Back Scattering Characteristics Dependent on the Observation Point. Oceanic Engineering, 22: 40-46. 763

URICK, R., 1983. Principles of Under Water Sound for Engineers. McGraw-Hill. Brown, L.G., 1992. A Survey of Image Registration Techniques. ACM Computing Surveys, 24(4): 325-376. Vincent, É. and R. Laganière, 2001. Matching Feature Points in Stereo Pairs: A Comparative Study of Some Matching Strategies. Machine Graphics & Vision. 10(3): 237-259. Trajkovic, M. and M. Hedley, 1998. Fast Corner Detection. Image and Vision Computing, 16(2): 75-87. Tissainayagam, P. and D. Suter, 2004. Assessing the Performance of Corner Detectors for Point Feature Tracking Applications. Image Vision Computing, 22(8): 663-679. Harris, C. and M. Stephens, 1988. A combined Corner and Edge Detector. In the Proceedings of Alvey Vision Conference, pp: 147-151. Mohr, R., Schmid and C. Bauckhage, 2000. Evaluation of Interest Point Detectors. International Journal of Computer Vision, 37(2): 151-172. Parker, J.R., 1997. Algorithms for Image Processing and Computer Vision. John Wiley & Sons. Zhang, Z., R. Deriche and O. Faugeras, 1995. A Robust Technique for Matching Two Uncalibrated Images through the Recovery of the Unknown Epipolar Geometry. Artificial Intelligence Journal, 78(3): 311-325. Smith, P., D. Sinclair, R. Cipolla and K. Wood, 1998. Effective Corner Matching. In the British Machine Vision Conference, pp: 545-556. Zitova, B. and J. Flusser, 2003. Image Registration Methods: A Survey. Image and Vision Computing, 21: 977-1000. Alkaabi, S. and F. Deravi, 2005. Iterative Corner Extraction and Matching for Mosaic Construction. In the 2nd Canadian Conference on Computer and Robot Vision, pp: 468-475. Zheng, Q. and R. Chellappa, 1993. A Computational Vision Approach to Image Registration. IEEE Transaction on Image Processing, 2(3): 311-325. Kittler, J., M. Hatef, R. Duin and J. Matas, 1988. On Combining Classifiers. IEEE Transaction on Pattern Analysis and Machine Intelligence, 20(3): 226-239. Snelick, R., U. Uludag, A. Mink, M. Indovina and A. Jain, 2005. Large-Scale Evaluation of Multimodal Biometric Authentication using State-of-the-Art Systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3): 450-455.