A Vector SIFT Operator For Interest Point Detection In Vector Imagery and its Application To Multispectral and Hyperspectral Imagery Leidy Paola Dorado-Muñoz1, Amith Mukherjee 2 , Dr. Miguel Vélez Reyes – Advisor 1 , Dr. Badrinath Roysam 2 E-mail:
[email protected],
[email protected],
[email protected],
[email protected] 1 Laboratory for Applied Remote Sensing and Image Processing, University of Puerto Rico-Mayaguez, PR. 2 Rensselaer Polytechnic Institute, NY.
This research work presents an algorithm for automated extraction of multi-scale interest points in multispectral and hyperspectral images. Interest points are features of the image that capture information from its neighbors and are distinctive and stable under transformations such as translation and rotation. They have been applied to diverse problems in computer vision, including image matching, recognition, registration, and change detection. Interest-point operators for monochromatic images were proposed more than a decade ago and have since been studied extensively. An interest operator seeks out points in an image that are structurally distinct, invariant to imaging conditions, stable under geometric transformation, and interpretable. The approach that is presented in this work combines and extends ideas from Lowe’s interest point operator that uses local extrema of Difference of Gaussian (DoG) function at multiple scales. A modification to Lowe’s approach for vector images is proposed. The multiscale representation of the image is generated by vector anisotropic diffusion that should leads to improve detection since it better preserve edges in the image. In addition, a vectorial approach of this operator is proposed to manage multispectral and hyperspectral images, experiments with multispectral and hyperspectral images are presented, and an evaluation of interest points found by our approach based on repeatability criterion and image registration is carried out.
Lowe’s Approach for Grayscale Images [1]:
Scale-Space Representation by Vector-Anisotropic Diffusion[3]
Original Image
Scale increase
-
Finding Maximum Vector within 3x3x3 neighborhoods using Lexicographical Ordering[4]
Detection of Maxima and elimination of unstable points[5] ⎡ ∂ DoSi ⎢∑ ∂x 2 II = ⎢ iM=1 2 ⎢ ∂ DoSi ⎢∑ ⎣ i =1 ∂y∂x M
Scale t+1 Scale increase
-
Original g Image g
Interest Points
-
Difference-of-Gaussians DoG
Generation of Scale Space Gaussian Smoothing
⎡ ∂ 2 DoG ⎢ ∂x 2 H (DoG ( x , y ) ) = ⎢ 2 ⎢ ∂ DoG ⎢ ∂y∂x ⎣
Scale t
∂ 2 DoG ∂x∂y ∂ 2 DoG ∂y 2
Local Maxima Pixel
r (ε ) = 2
R (ε ) min(n , n )
-
2
2
12
x =H x 1
Generation of Scale Space Gaussian Smoothing
1
12
1
21
Interest Points
ε
max(s , s ) , min( s , s ) 1
2
1
2
disp = ( x − x ) + ( y − y ) 2
rel
2
1
2
2036 IPs in Reference Image g 2121 IPs in Sensed Image 1936 IPs are corresponded
1
PCA Comp. M
Difference-of-Gaussians DoG
RGB Images: Rotation Sequence
PCA Comp. 1
PCA Comp. M
Interest Points
Function for combining DoG responses along spectral dimension
2
1
RGB Images taken by Digital Camera in the Parguera Area
n , n : Total T t l number b off Interest I t t Points P i t 2
2
ε
1
PCA Comp. 2
Same as Lowe’s Approach
s
2
x =H x 2
-
R =
1
R (ε ) = {( x , x ) | dist ( H x , x ) < ε } PCA Comp. 1
PCA Comp. 1
Image Registration was performed using a transformation, which was estimated from matching interest points chosen manually. In addition, the correspondence between set of interest points (IPs) was determined in terms of their position and scale. Two points: (x1, y1) in image 1 with scale s1, and (x2, y2) with s2 after applying the estimated transformation, transformation are corresponded if ratio of scales (Rs), ) is less than 2 and the relative displacement (disprel), is less than the average scale [2].
2
2
PCA Comp. M
Tr ( II ) 2 ( r + 1) 2 < Det ( II ) r r = 10
Evaluation of Interest Points by means Image Registration:
Repeatability is a criterion that attempts to measure the independence of the keypoints detection to changes in imaging conditions (relative position of camera to the scene and illumination conditions). Repeatability rate is the number of points repeated between two images over the total number of detected points [6].
Mukherjee’s Approach for Hyperspectral Images [2]:
PCA projection
i =1 M
Scale t-1
Evaluation of Interest Point by means Repeatability Criterion:
Threshold around of each Local Maxima Pixel
Original Image
∑
Interest Points
2
Original Image
Different scales
( r + 1) 2 Tr ( H ) 2 = Det ( H ) r
Scale iincrease
∂ DoSi ⎤ ⎥ ∂x∂y ⎥ ∂ 2 DoSi ⎥ ∑ ∂y 2 ⎥⎦ i =1 M
Difference of Adjacent Scales (DoS)
⎤ ⎥ ⎥ ⎥ ⎥ ⎦
Scale increase
2
Original Image
Rotation Angle
Repeatability Criterion ( ε =1pixel)
15°
0.581
30°
0.494
45°
0.447
60°
0.404
Under water Image Reference Image
Shifted Under water Image Sensed Image
Overlayed Registered Images
Hyperspectral Images: Time-lapse Images taken by Hyperion 2803 IPs in Reference Image 4209 IPs in Sensed Image 1645 IPs are corresponded
Rotated Image by 60°
RGB Images: Scale Change Sequence L.Dorado, M.Velez-Reyes, A. Mukherjee and B.Roysam. “Interest Point Detection in Hyperspectral Imagery”. In Proceedings of SPIE: Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XV, Vol. 7334, April 2009.
[1] D,Lowe. “Distinctive Image Features from Scale-Invariant Keypoints”. International Journal of Computer Vision, 2004. [2] A.Mukherjee, B.Roysam, M.Velez-Reyes. “Interest points for hyperspectral image data”. IEEE Transactions on Geoscience and Remote Sensing, 47(3):748–760, 2009 [3] J,Duarte-Carvajalino, P.Castillo, M.Velez-Reyes. “Comparative Study of Semi-Implicit Schemes for Nonlinear Diffusion in Hyperspectral Imagery”. IEEE Transactions on Image Processing, 2007. [4] E.Apotula,S.Lefevre. On lexicographical ordering in multivariate mathematical morphology. Pattern Recognition Letter, 2008. [5] M.Spivak. A Comprehensive Introduction to Differential Geometry. Perish, 3ed. 1999. [6] C.Schmid; R.Mohr; C.Bauckhage. Evaluation of interest point detectors. International Journal of Computer Vision, 37(2):151–172, 2000
Change of Scale
Repeatability Criterion (ε =1pixel)
1:4
0.708
1:6
0.726
1:8
0.809
2:0
Parguera 2002 Reference Image
Parguera 2003 Sensed Image
Overlayed Registered Images
Hyperspectral Images with different spatial resolution taken by AISA
1218 IPs in Reference Image 397 IPs IP in i S Sensed d IImage 90 IPs are corresponded
0.558 Scale t
Original Image
Scaled Image 2:0
This work will contribute to Image Understanding and Multispectral Discrimination (R2). SeaBED data is being used for testing and validation of the proposed approach. Acknowledgements: Dr. Velez-Reyes and Ms. Dorado were supported primarily by NGA. All participants received partial support from the Bernard M. Gordon Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Center Program of the National Sciences Foundation (Award Number EEC-9986821)
Parguera 4m Reference Image
Parguera 4m Reference Image
Parguera 8m Sensed Image
Parguera 8m Sensed Image
Overlayed Registered Images
A vectorial operator for detecting interest points in hyperspectral and RGB images has been developed, based on SIFT operator proposed by Lowe. An evaluation based on the Repeatability criterion and Image Registration is performed.
