2010 22nd International Conference on Tools with Artificial Intelligence
An Adaptive Fuzzy Classifier Approach to Edge Detection in Latent Fingerprint Images Juan F Ramirez Rochac, Lily Liang, Byunggu Yu
Zhao Lu
Dept. of Computer Science & Information Technology University of the District of Columbia Washington, DC 20008 USA {juan.ramirezrochac, lliang, byu}@udc.edu
Department of Electrical Engineering Tuskegee University Tuskegee, AL 36088 USA
[email protected]
Abstract— This paper proposes an Adaptive Fuzzy Classifier Approach (AFCA) to local edge detection in order to address the challenges of detecting latent fingerprint in severely degraded images. The proposed approach adapts classifier parameters to different parts of input images using the concept of reference neighborhood. Three variants of AFCAs, namely K-means-clustering AFCA, Entropy-based AFCA, and Statistical AFCA, were developed. Experiments were conducted both on synthetic images and on real fingerprint images to compare these AFCAs and Canny edge detection. The presented results show that Statistical AFCA is the best performer with latent images.
segmentation. Recently, C. Kotropoulos and I. Pitas studied edge segmentation in magnetic resonance imaging with Support Vector Machine (SVM) [6]. S. Wan, F. Yang, and M. He. [10] took into account human vision perception in edge detection, that is, given an image, humans see both the whole as well as local details. In line with this, hybrid approaches have recently been developed. X. He, H. Zhang, W. Jia, Q. Wu, and T. Hintz [8] proposed a global-and-local combined approach to adaptively select the best edges, which uses J. See, M. Hanmandlu, and S. Vasikarla, in [9], used both global and local characteristics in grayscale images to detect both strong and weak edges. However, all these approaches lack the capacity to identify edges when input images come with inconsistent illumination causing dark shadows and bright spots. An approach in [14] deals with varying illumination in the image by generating multiple templates during the edge detection process. This was proven to be effective in pill identification; however, latent fingerprint challenges (e.g., cuts, small overlapping areas, large nonlinear distortions, and even altered areas) are yet to be overcome. Accuracy remains an open field for more active collaborative research among academia, industry, and government. Problem: Given the approaches outlined above, and the difficulties in addressing the issues of edge detection in latent fingerprints, and that fingerprints have been widely accepted as one of the key biometrics for personal identification [15], it is clear that there is an increasing demand for efficient robust personal identification and/or authentication methods in computer vision, biometrics, and information assurance [17], especially in extracting features from low-quality or inconsistently illuminated fingerprints, which is a challenging problem that requires significant research. Recent literature, researchers, scientists, and government agencies all have pointed out this problem [14, 15, 16 and 17]. Approach: In light of recent fuzzy edge detection approaches [1, 4, 9, 11, 12], this paper proposes an Adaptive Fuzzy Classifier Approach (AFCA) to effectively address this challenge. The proposed AFCAs adapt their classifier parameters to different parts of input images using both a local and a reference neighborhood.
Keywords- edge detection; latent fingerprints; fuzzy classifer
I.
INTRODUCTION
The goal of edge detection is to identify and classify the discontinuities in images. The first presentation of edge detection (D. Marr and E. Hildreth [2]) was made in 1980 and many different methods have been proposed since then. These methods can be classified into two groups: local edge detection and global edge detection. The local methods consider a small neighborhood area in each detection step, while the global ones consider the whole image each step. J. H. Elder and S. W. Zucker developed their approach to local edge detection and blur estimation [7], but local algorithms are sensitive to noise. Canny [3] developed a global edge detection method for grayscale images, which is still one of the most used algorithms. Canny Edge Detector uses global parameters, which is less sensitive to local changes in intensity, resulting in an increased, robustness against noise. But, in severely degraded images, this resistance against noise becomes a trade-off by losing soft edges. In order to take into account these changes in intensity, and to obtain both strong and weak edges, A. F. Korn proposed [4] that intensity changes of an image be represented by symbols and to use these symbols to adaptively select a suitable scale, and with that, the size of the right kernel (i.e., an operator that relates data in a high dimensional feature space). This approach works well in places where other non-adaptive filter-based approaches had failed. These approaches have found a niche in the medical field. For instance, D. Q. Zhang and S. C. Chen [5] presented a kernelized fuzzy C-means algorithm for brain image 1082-3409/10 $26.00 © 2010 IEEE DOI 10.1109/ICTAI.2010.32
178
While a relatively small local neighborhood window is used to generate feature vectors for edge classification, a larger reference neighborhood window is used to generate gradient change references in a broader area. The use of the reference neighborhood takes the area’s illumination into consideration. Contributions: This paper explores the novel idea of using reference neighborhood window to adaptively calibrate the classifier parameters. Three variants of AFCAs, called, K-means-clustering AFCA, Entropy-based AFCA, and Statistical AFCA, were developed. Experiments were conducted both on synthetic images and on real fingerprint images to compare these AFCAs and Canny edge detection. The presented results show that Statistical AFCA is the best edge detection performer for latent fingerprints. This approach advances the field of biometrics personal identification and can be applied in law enforcement, national security, and other applications of information assurance and security. Contents: The rest of this paper is organized as follows: Section II presents the six-step procedure of AFCAs; Section III presents and discusses the experimental results; and, the last section draws a conclusion and sketches future improvements. II.
Figure 1. Sequence of steps of our approach
A. Step 1:Calculate feature (Difference) Vectors of pixel p Fig. 2 shows the difference vector of a pixel P5. The difference vector x = (d1, d2, d3, d4) represents the magnitude of the differences between P5 and its transversal neighbors. For instance, in direction 1, d1 = |P5-P1| + |P5P9|. Such a difference vector is calculated for every pixel within the image.
METHODOLOGY
Fig. 1 presents an overview of the general Adaptive Fuzzy Classifier Approach (AFCA). This six-step approach is explained briefly as follows while the full details of each step are presented in subsections A through F. (1) Calculate feature vectors of pixel p: The feature vectors in this paper are vectors of gradient difference, defined in Subsection A. Thus we call them difference vectors. In this step, we calculate the difference vector for each pixel p in its pixel's 3 x 3 local neighborhood. (2) Cluster vectors of pixel p in to High/Low: For each pixel p, we define its 9 x 9 reference neighborhood, and obtain values of each dimension of all difference vectors in this reference neighborhood (9*9*4 values). These values are clustered into two sets: High and Low and their centers, hi and lo, are calculated. (3) Construct adaptive fuzzy classifiers of pixel p: For each pixel p, define its fuzzy classifier for background, edge or speckle with that pixel’s hi and lo. (4) Classify each pixel p using the adaptive fuzzy classifiers. The above steps are repeated for each pixel p before the next step is taken. (5) Select edge pixels with competitive rules: Pixels classified as edges compete with each other on the direction that determined by their class. This process produces fine-line edges. (6) Remove Speckles: Isolated edge pixels are considered as speckle noise and they are subject to removal.
Figure 2. Directions of the four transversal differences in a local neighborhood
B. Step 2: Cluster vectors of pixel p into High/Low In this step, we implemented three alternative algorithms: K-means clustering, entropy-based clustering and statistics to customize the values of lo and hi to reference neighborhood of p. Our K-means approach generates the initial centers randomly and updates based on Euclidean distance. In our entropy based approach, we divide the reference neighborhood into two subregions. Then we calculate their entropy and add these two values into the total entropy of the system. This is done by using (1) on each subregion:
S = −k B ∑ Pi ln Pi ,
(1)
i
where Pi is the corresponding probability of each i-th pixel value. This procedure is iterated until the maximum total
179
entropy is achieved. The pseudo-code of the above algorithms is listed below.
⎛
μi ( x ) = Max ⎜ 0, 1 − ⎜ ⎝
( x − ci ) w
2
2
⎞ ⎟, ⎟ ⎠
(2)
Algorithm No. 1: K-means clustering
where ci is the ith prototype vector and x is the difference vector. We use lo and hi, calculated for each pixel p, to construct custom-made classes for each pixel depending on the intensive values of the reference neighborhood area. Therefore, our fuzzy classifier adapts to the multiple degrees of intensity within the image.
Step 1: For each pixel p set its reference neighborhood=9 x 9 Step 2: Generate initial centers of cluster High and Low respectively Step 3: For each difference vector x in the current reference neighborhood Calculate its distance to both hi and lo Assign p to the closest one Step 4: Recalculate cluster centers as hi and lo Step 5: Repeat step 3 until center stops changing
D. Step 4: Classify pixel p We calculate the degree to which p belongs to each of the six classes, and then p is assigned to the one with the maximum degree of membership.
Algorithm No. 2: Entropy-based clustering Step 1: For each pixel p set its reference neighborhood=9 x 9 Step 2: For each difference vector x in the current reference neighborhood Calculate the entropy S to both the set of smaller values and the set of larger values comparing to the current pixel value Find p that maximizes entropy S of both sets Step 3: Set hi = mean (larger values than p) Step 4: Set lo = mean (smaller values than p)
E. Step 5: Select edge pixels with Competitive Rules For synthetic images. The pixels that were classified as edge pixels compete against their perpendicular neighbors and only the one with the highest degree of membership for that edge class is set as a real edge in order to obtain thinner edges. For example, pixels classified as Class1 compete against their neighboring pixels on Direction 3. The rule is presented below.
Algorithm No. 3: Statistical clustering Step 1: For each pixel set its reference neighborhood = 9 x 9 Step 2: For all difference vectors x in the current reference neighborhood Calculate their mean (µ) and standard deviation (σ) Step 3: Set hi = µ + σ Step 4: Set lo = µ – σ
IF pixel p is of Class 1* THEN IF p wins against pixels on Direction 3 THEN change p to black ELSE change p to white END IF
Similarly, we design the rules for competition for other classes.
When the reference neighborhood R is moving on or close by any border of the fingerprint image, only a partial window will be consider for calculating hi and lo.
For fingerprint images. An adaptation of rules is made to take into account the fingerprint “parallel lines.” By narrowing down the possible direction of the edges to coincide with the ridges within the reference area, we achieve a reduction in the amount of outliers and background noise. Based on the above concept, a pixel will be classified as either a potential edge pixel or a background pixel by using an IF-ELSE clause. If, within the reference neighborhood, the traversal diagonal to the pixel’s class is the highest diagonal and the parallel diagonal is the lowest diagonal, the pixel is a possible edge; otherwise it is part of the background. The formal rule is presented below.
C. Step 3: Contruct adaptive fuzzy classifiers of pixel p As in [1] we have one prototype for each of the six edge or non-edge classes. For each edge class there is one unique direction. For example, Class 3 includes all the pixels that are labeled as possible vertical edges and Class 5 includes pixel that are possible horizontal edges. Class 2 and Class 4 respectively correspond to the major and minor diagonals on the local neighborhood. Each prototype is represented by a vector, are listed below. The subscript 1-4 are for the four dimensions respectively. Class 1: background Class 2: edge Class 3: edge Class 4: edge Class 5: edge Class 6: speckle edge
IF pixel p is considered an edge on the diagonal d1* THEN IF hi3 = max(hik) AND lo3 = min(lok) for k ={1,2,3,4} THEN change p to black ELSE change p to white END IF END IF
[lo1, lo2, lo3, lo4] [lo1, hi2, hi3, hi4] [hi1, lo2, hi3, hi4] [hi1, hi2, lo3, hi4] [hi1, hi2, hi3, lo4] [hi1, hi2, hi3, hi4]
The notation “k” stands for all hi/lo values in the reference neighborhood.
The Epanechnikov function defined by (2) is employed for the fuzzy membership functions of the classes:
180
F. Step 6: Remove Speckles To reduce the influence of noise, we remove an edge pixel if it is an isolated edge pixel inside a 3x3 local neighborhood or if all of its four diagonal differences are hi. III.
EXPERIMENTAL RESULTS Figure 5. CFED result (hi=max, lo = min neighborhood=3 x 3)
In this section, we include an array of experimental results which demonstrate the effectiveness of the proposed method for edge detection on both synthetic images and real fingerprint images. A. Results on Synthetic Images We first employed synthetic images to evaluate our adaptive fuzzy approach. Here we performed different implementations of our approach and compared their results with results produced by Canny [3]. We drew a synthetic image, which is depicted in Fig. 3, with all kinds of edges and with different textures. The results of Canny edge detection, CFED, K-means clustering AFCA and statistical AFCA are presented in Fig. 4, 5, 6, and 7 respectively. While you can see that Canny and CFED are missing some edges, the K-means clustering and statistical AFCA algorithms depict all edges –even the ones hidden in darken areas. The entropy-based AFCA turned out to be highly computational expensive. No figures were generated during a reasonable given amount of time. We performed another group of experiments to investigate the influence of various sizes of local neighborhood and reference neighborhood on our Statistical AFCA. Besides the above experiment with a standard 3 x 3 local neighborhood size, we tested with other sizes: 5 x 5, 7 x 7, and 9 x 9. For each different size of local neighborhood, three different sizes of reference neighborhood were tested, where the reference size was 2, 3, and 5 times of the local size. The best results were achieved by local neighborhood size of 7 x 7 and reference neighborhood of 21 x 21 pixels, for the tested image. As we can see from Fig. 8 through to Fig. 12, the quality of edge detection results improves when size of the local neighborhood increases to a certain degree. The same is true for the size of the reference neighborhood.
Figure 6. K-means-clustering-based AFCA result
Figure 7. Statistical AFCA results (hi = mean + σ, lo = mean + σ)
Figure 8. Statistical AFCA Results (a 3 x 3 local neighborhood over a 6 x 6 reference neighborhood)
Figure 9. Statistical AFCA Results (3 x 3 over 15 x 15)
Figure 3. Synthetic image
Figure 10. Statistical AFCA Results (7 x 7 over 14 x 14)
Figure 4. Result from Canny with Matlab [5]
Figure 11. Statistical AFCA Results (7 x 7 over 21 x 21)
181
Performance on Im age No.2 105%
Detected pixel edges
100%
Figure 12. Results of AFCA (7 x 7 over 35 x 35)
B. Results on Fingerprint Images The last array of experiments was performed on real images from a sample dataset of latent fingerprints, which is available online at [18]. To show the robustness of Statistical AFCA in the presence of inconsistent illumination, fingerprint images were synthetically darkened to a certain degree (15%, 30% and 45%). The results are shown in Fig. 17-20 respectively on the last pages.
90% 85% 80% 75% 70% 0%
15%
30%
45%
Degree of degradation Canny
C. Performance Analysis In this section, AFCA is compared to Canny edge detection algorithm in a quantitative matter. Fig. 13, 14, 15 and 16 compare the Statistical AFCA to Canny in terms of accuracy. In Table 1, we can see that, for the average of the five cases we tested, the Statistical AFCA performs better than Canny edge detection, when the illumination of the image is severely inconsistent across the image.
AFC
Figure 14. Statistical AFCA results (hi = mean + σ, lo = mean + σ) Perform ance on Im age No.3 105% 100% Detected pixel edges
TABLE I.
95%
ACCURACY COMPARISON
95% 90% 85%
Image Degradation
Canny
AFCA
0%
100%
100%
75%
15%
99%
99%
70%
30%
98%
100%
45%
73%
85%
80%
0%
15%
30%
45%
Degree of degradation Canny
AFC
Figure 15. Statistical AFCA results (hi = mean + σ, lo = mean + σ)
Perform ance on Im age No.1 105%
Average Performance 105%
95%
100%
90%
Detected pixel edges
Detected pixel edges
100%
85% 80% 75% 70% 0%
15%
30%
45%
90% 85% 80% 75%
Degree of degradation Canny
95%
70% 0%
AFC
15%
30%
45%
Degree of degradation Canny
Figure 13. Statistical AFCA results (hi = mean + σ, lo = mean + σ)
AFC
Figure 16. Statistical AFCA results (hi = mean + σ, lo = mean + σ)
182
false edge reduction by adding dynamically adapted minimum thresholds for hi.
As an additional estimation, we evaluated the best algorithm’s behavior (i.e., Statistical AFCA), regardless of the hardware, using the Big O Notation. Especially, we considered only the complexity evaluation of the clustering, Step 2, and the classification that were recognized as the most time consuming steps of our algorithms. The algorithm is reported in pseudo-code to have a compact and high-level description; the O(f(n)) notation is presented for each step. The complexity of Step 1 depends on the size of the image (N x M), the size of the local neighborhood L and the number of values per features vector (4). In Step 2, we consider the size of the input image (N x M), the size of the reference neighborhood R and the number of fuzzy parameters (8). The complexity of Step 3 & Step 4 depends on the image size (N x M), the number of class prototypes (6) and the number of dimensions per class prototype (4). For Steps 5 & 6, the complexity is based on size of the input image (N x M), the number of rules (6) and the highest depth of a given rule (4). Thus, in terms of complexity, not runtime, and after theoretical analysis of our algorithm, we can say that our Statistical AFCA runs in O(n3) time, as it is simplified in (3):
⎛ N *M * L*4 + ⎞ ⎟ ⎜ ⎜ N *M * R*4*2 +⎟ O⎜ ⇒ O(n 3 ) , N * M *6*4 + ⎟ ⎟ ⎜ ⎟ ⎜ N * M *6*4 ⎠ ⎝
ACKNOWLEDGMENT This work was supported by the National Science Foundation (NSF) under award/funding No. 0911969: "Collaborative Research: A Partnership for Developing the Information Assurance Workforce". REFERENCES [1] [2]
[3] [4]
[5]
[6]
[7]
(3)
[8]
considering the worst-case performance scenario. IV.
[9]
CONCLUSION [10]
We developed an Adaptive Fuzzy Classifier Approach (AFCA) to edge detection, addressing the challenges of detecting edges in latent fingerprint images. The AFCA automatically adapts its fuzzy classifier parameters to the input by using the concept of reference neighborhood. Three variants of AFCAs, called K-means-clustering AFCA, Entropy AFCA and Statistical AFCA, were proposed. Experiments were conducted on synthetic images to compare AFCA with one of the most recognized edge detection approaches, Canny edge detection. We also investigated the influence of the sizes of the local and the reference neighborhood to AFCA. Then we tested the best performer of the three AFCAs on real-world fingerprint images and compared its result with the ones of Canny. The Statistical AFCA has shown better results than the ones achieved with the existing approach for severely degraded images. This project provides solution for images with inconsistent illumination. The effectiveness of Statistical AFCA can still be improved and other clustering methods can be tried for AFCA. In future, we plan to address the
[11]
[12] [13] [14]
[15] [16]
[17]
[18]
183
L. R. Liang, C. G. Looney, “Competitive fuzzy edge detection” Applied Soft Computing vol. 3, Elsevier, 2003, pp. 123-137. D. Marr, E. Hildreth, “Theory of edge detection” Proceedings of the Royal Society of London. Series B, Biological Sciences, vol. 207, No. 1167. (Feb. 29, 1980), pp. 187-217. J. Canny, “A computational approach to edge detection” IEEE Trans. Pattern Analysis and Machine Intelligence, 1986, pp. 679–698. A.F. Korn, “Toward a symbolic representation of intensity changes in images” IEEE Trans. Pattern Analysis and Machine Intelligence, 1988, pp. 610–625. D. Q. Zhang, S. C. Chen, “A novel kernelized fuzzy C-means algorithm with application in medical image segmentation” Artificial Intelligence in Medicine, vol. 32, Elsevier, 2004, pp. 37–50. C. Kotropoulos, I. Pitas, “Segmentation of ultrasonic images using support vector machines” Pattern Recognition Letters, vol. 24, Elsevier, 2003, pp. 715–727. J. H. Elder, S. W. Zucker, “Local scale control for edge detection and blur estimation” IEEE Trans. Pattern Analysis and Machine Intelligence, 1998, pp. 699–716. X. He, H. Zhang, W. Jia, Q. Wu, T. Hintz, “Combining Global and Local Features for Detection of License Plates in a Video”, Proceedings of Image and Vision Computing New Zealand 2007, pp. 288–293. . See, M. Hanmandlu, S. Vasikarla, “Fuzzy-based Parameterized Gaussian Edge Detector Using Global and Local Properties” IEEE Proceedings of ITCC'05 Vol 2, 2005, pp. 101–106. JS. Wan, F. Yang, M. He, “Gradient-threshold edge detection based on perceptually adaptive threshold selection” IEEE Trans. Pattern Analysis and Machine Intelligence, 1988, pp. 610–625. R. Wang, L. Gao, S. Yang, and Y. Liu, "An Edge detection method by combining fuzzy logic and neural networks," Machine Learning and Cybernetics, 7(2005) 4539-4543, 18-21, Aug 2005 Tizhoosh H.R., “Fast fuzzy edge detection”, Proceedings of Fuzzy Information Processing Society, 2002, pp. 239-242. The MathWorks Inc. 16 Dec 2009 http://www.mathworks.com Y.-B. Lee, U. Park, and A. K. Jain, "PILL-ID: Matching and Retrieval of Drug Pill Imprint Images," IEEE, International Conference on Pattern Recognition, Aug 2010 (To Appear). A. K. Jain, J. Feng and K. Nandakumar, "Fingerprint Matching",IEEE Computer, pp. 36-44, Feb. 2010. A. K. Jain and A. Kumar, "Biometrics of Next Generation: An Overview", Second Generation Biometrics (E. Mordini and D. Tzovaras, Eds.), Springer, 2010. K. Niinuma, U. Park, A. K. Jain, "Soft Biometric Traits For Continuous User Authentication", MSU Technical Report. MSUCSE-10-6, Feb. 2010. NIST 8-Bit Gray Scale Images of Fingerprint Image Groups (FIGS), November 19, 1992, downloaded on April 8, 2010 ftp://sequoyah.ncsl.nist.gov/pub/databases/data/prints4.tar
Figure 17. Results of Finger Print No. 1: The top row of images corresponds to original fingerprint plus 0%, 15%, 30% 45% degradation; the second row shows the results of Canny edge detection. The third row depicts the ones of Statistical AFCA
Figure 18. Results of Finger Print No. 2: The top row of images corresponds to original fingerprint plus 0%, 15%, 30% 45% degradation; the second row shows the results of Canny edge detection. The third row depicts the ones of Statistical AFCA
184
Figure 19. Results of Finger Print No. 3: The top row of images corresponds to original fingerprint plus 0%, 15%, 30% 45% degradation; the second row shows the results of Canny edge detection. The third row depicts the ones of Statistical AFCA
Figure 20.
Results of Finger Print No. 4: The top row of images corresponds to original fingerprint plus 0%, 15%, 30% 45% degradation; the second row shows the results of Canny edge detection. The third row depicts the ones of Statistical AFCA
185
