Palmprint authentication using a symbolic representation of images

Image and Vision Computing 28 (2010) 343–351

Contents lists available at ScienceDirect

Image and Vision Computing journal homepage: www.elsevier.com/locate/imavis

Palmprint authentication using a symbolic representation of images Jiansheng Chen a,*, Yiu-Sang Moon b, Ming-Fai Wong c, Guangda Su a a b c

Department of Electronic Engineering, Tsinghua University, Haidian, Beijing 100084, PR China Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong Department of Computer Science, University of Toronto, 10 King’s College Road, Toronto, Canada M5S 3G4

a r t i c l e

i n f o

Article history: Received 4 December 2007 Received in revised form 30 March 2009 Accepted 2 June 2009

Keywords: Biometrics Palmprint authentication Symbolic representation Time series

a b s t r a c t A new branch of biometrics, palmprint authentication, has attracted increasing amount of attention because palmprints are abundant of line features so that low resolution images can be used. In this paper, we propose a new texture based approach for palmprint feature extraction, template representation and matching. An extension of the SAX (Symbolic Aggregate approXimation), a time series technology, to 2D data is the key to make this new approach effective, simple, flexible and reliable. Experiments show that by adopting the simple feature of grayscale information only, this approach can achieve an equal error rate of 0.3%, and a rank one identification accuracy of 99.9% on a 7752 palmprint public database. This new approach has very low computational complexity so that it can be efficiently implemented on slow mobile embedded platforms. The proposed approach does not rely on any parameter training process and therefore is fully reproducible. What is more, besides the palmprint authentication, the proposed 2D extension of SAX may also be applied to other problems of pattern recognition and data mining for 2D images. Ó 2009 Elsevier B.V. All rights reserved.

1. Introduction Automated personal authentication using biometric features has been widely studied during the last two decades. Previous research efforts have made it possible to apply biometric systems to practical applications for security or commercial purposes. Biometric systems based on fingerprint recognition [1], face recognition [2], and iris recognition [3] have already been developed to a quite mature stage so that they can be applied to critical security applications such as the immigration control and the crime investigation. Recently, a novel hand-based biometric feature, palmprint, has attracted an increasing amount of attention. Like any other biometric identifiers, palmprints are believed to have the critical properties of universality, uniqueness, permanence and collectability for personal authentication [4]. Palmprints have several advantages over other hand-based biometrics, such as fingerprint and hand geometry. Compared to fingertips, palms are larger in size and therefore are more robust to injuries and dirt. Also, low-resolution imaging can be employed in the palmprint recognition based on creases and palm lines, making it possible to perform real time preprocessing and feature extraction; and the cost of the capturing device can also be well controlled. Palmprint authentication is believed to be able to achieve the accuracy comparable to that of * Corresponding author. Tel.: +852 31634251; fax: +852 26035024. E-mail address: [email protected] (J. Chen). 0262-8856/$ - see front matter Ó 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2009.06.004

other hand-based biometric authentication technologies, including fingerprint [1] and hand geometry [5,6]. Texture and palm lines are the most clearly observable palmprint features in low resolution (such as 100 dpi) images [7], and thus have attracted most research efforts. In texture based palmprint authentication approaches [7–14], signal processing based texture analysis methods [15] are usually adopted. Typically, texture features are extracted by filtering the palmprint images using filters such as the Gabor filter [7,11–13], the ordinal filter [10], or the wavelet [8]. The image filtering may be performed in either the spatial domain or the frequency domain [14]. Recently, a lot of automated palmprint authentication methods [16–19] have focused on the palm line features, since they are more appealing than the texture for the human vision. In the offline method proposed in [16], the geometric shapes of the palm lines are extracted and approximated by straight-line segments. The slope, intercept and inclination of each segment are used as features for palmprint matching. C.C. Han et al. investigate the magnitude of palm lines in palmprint matching [17]. The latest related research reveals that the orientations of palm lines also contain strong discriminative power. Based on palm line orientations, a Competitive Code is designed for palmprint representation in [18]; and Y. Han et al. use local orientation histograms for describing palmprints [19]. Similar to the texture based methods; the palm line based methods usually employ image filtering for line feature extraction, leading to a high computational complexity. For example, in the Competitive Code method [18], six Gabor filters are

344

J. Chen et al. / Image and Vision Computing 28 (2010) 343–351

applied to each palmprint ROI (Region Of Interest) for generating the corresponding orientation map. Suppose that the Palmprint ROI is 128  128 pixels and the Gabor filters are 35  35 in size, the overall MADD (Multiplication + Addition) operations required for one palmprint is around 120 million, leading to a very long processing time especially on slow mobile platforms. Experiments show that extracting the Competitive Code for one palmprint takes more than eight seconds on a state of the art PDA [20]. This is far too slow for a real-time biometric system. Besides the computational complexity, selecting appropriate filter parameters is also nontrivial in filtering based palmprint authentication methods. It has been demonstrated in [4,7] that the authentication accuracy varies a lot when using different Gabor filter parameters, which need to be tuned in a try and error manner, indicating that the authentication performance will depend a lot on the training set used for parameter selection. This may account for the significant performance variations of different filtering based palmprint authentication methods on different databases [19]. In this paper, we propose a texture based approach for palmprint authentication, in which palmprint image grayscale information are directly adopted as features. The computational complexity of the feature extraction process is much lower than previous filtering based approaches, and thus can be implemented efficiently for even slow mobile embedded platforms. By extending the idea of SAX (Symbolic Aggregate approXimation) [21] in time series research to 2D images for palmprint representation and matching, the proposed method can achieve the authentication performance, in terms of EER (Equal Error Rate), comparable to the state of the art palmprint authentication methods. The rest of this paper is organized as the follows. Section 2 explains the 2D extension of SAX for images. Section 3 describes the feature extraction and matching processes of the proposed approach. Experiments and results are elaborated in Section 4. The last section is a conclusion of our work. 2. 2D SAX As a symbolic representation of sequential data, SAX (Symbolic Aggregate approXimation) [21] has been verified as a simple but effective tool for solving most time series data mining problems such as clustering, classification, indexing, anomaly detection, and motif finding [22]. For a real valued data sequence, its SAX representation is obtained by first transforming it into the PAA (Piecewise Aggregate Approximation) representation; then predetermined breakpoints are used to discretize the PAA coefficients into SAX symbols. Basically, SAX represents a real valued data sequence using a string of discrete symbols or characters, as is shown in Fig. 1. Dramatic dimensionality reduction can be achieved by converting a time series into its SAX representation. In order to preserve the discriminative power of the data as complete as possible, Keogh et al. proposed a discretization technique which can produce symbols with equiprobability by using discretization breakpoints following a Gaussian distribution [21]. This specific

Table 1 The lookup table for calculating dist() function when SAX_Level equals four [21].

‘a’ ‘b’ ‘c’ ‘d’

‘a’

‘b’

‘c’

‘d’

0 0 0.67 1.34

0 0 0 0.67

0.67 0 0 0

1.34 0.67 0 0

property of SAX has made it suitable for the feature representation in biometric authentication where the discriminative power of the representation is crucial. From Fig. 1, we can see that the degree of dimensionality reduction is decided by two parameters: the length of the resultant symbol string, or the SAX_Length; and the number of symbols used, or the SAX_Level. As an example, the SAX_Length equals eight and the SAX_Level equals four in Fig. 1. The smaller these two parameters are, the more intensive the dimensionality reduction is. A distance measurement called MINDIST, which strictly lower bounds the corresponding Euclidean distance of the original time series [23], is employed to measure the similarity between two SAX b ¼q ^2 ; . . . ; q ^w and ^1 ; q strings [21]. Given two SAX strings Q b ¼ ^c1 ; ^c2 ; . . . ; ^cw , their MINDIST can be calculated using Eq. (1), C where n is the length of the original time series, and dist() can be implemented using table lookup as is illustrated in Table 1 [21].

b; C bÞ ¼ MINDISTð Q

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiu w X nu t ^i ; ^ci Þ2 distðq w i¼1

ð1Þ

For high dimensional data, such as a 2D or 3D image, one feasible way of applying the SAX conversion is to firstly decompose it into a 1D sequence. For example, 3D fMRI (functional Magnetic Resonance Imaging) images are decomposed into 1D sequences using the Hilbert space filling curve for clustering and motif finding in [24]. In our previous work [25], palmprint ROIs are decomposed using spiral curves before SAX conversion. Nevertheless, the neighborhood information, which plays an important role in similarity measurement for high dimensional data, will inevitably be lost during the 1D decomposition no matter what space filling curve is used. For instance, in a 2D image, each pixel, except those on the boundary, has eight eight-neighbor pixels. However, after decomposition, each pixel will only have two neighbors in the resultant 1D sequence. Such a phenomenon is illustrated in Fig. 2, where (I) and (II) shows the up-left corners of two images. Without losing generality, pixels are represented by cells inside which the values stand for the grayscale level in the hexadecimal format. SI and SII ðS0I and S0II Þ are the 1D decomposition results using progressive scan (Hilbert curve scan). It is not difficult to observe that in their 2D forms, image (I) and image (II) are probably identical except for a one pixel shift in both the vertical and the horizontal directions. This apparent similarity, however, cannot be easily discovered from their corresponding 1D sequences. The fundamental reason for this problem is the serious loss of neighbor-

Fig. 1. Conversion of a time series to its SAX representation. (a) The original real valued time series. (b) PAA representation. (c) SAX representation.

345

J. Chen et al. / Image and Vision Computing 28 (2010) 343–351

real valued 2D data using a 2D matrix of symbols. Assume that the size of the input data Q is m  n. First, Q is normalized to have a zero mean and a unit standard deviation. Then, similar to the PAA representation, Q can be represented by a matrix Q of the size w1  w2 . The ðith ; jth Þ element of Q can be calculated by Eq. (2). Simply stated, the original data Q is divided into w1  w2 equal size blocks, and the mean value of the data inside each block is calculated to form the dimensionality reduced representation: Q . m

1 Qði; jÞ ¼ w1 w2

hood information. In Fig. 2, ‘5’ and ‘A’, which are neighbors in both (I) and (II), are no longer adjacent in any of the 1D sequences; and they are separated in such a irregular way that existing time series alignment techniques, such as sequence shifting, DTW (Dynamic Time Warping) alignment [26] and LCSS (Longest Common Subsequence) alignment [27], cannot be consistently applied to recover the lost neighborhood information. This phenomenon accounts for the unsatisfactory verification accuracy we have achieved in our previous attempt of applying SAX to palmprint authentication [25]. To facilitate a more effective SAX conversion for 2D data, we propose the 2D SAX, an extension of the SAX, which represents a

n

j

w2 X

Q ðx; yÞ

ð2Þ

x¼wm ði1Þþ1 y¼wn ðj1Þþ1 1

Fig. 2. 1D decomposition of 2D images using different space filling curves.

i

w1 X

2

Finally, breakpoints identical to those used in 1D SAX converb , or the 2D sion are applied to convert Q into a symbol matrix Q SAX representation. The definitions and properties of these breakpoints can be found in [21]. These breakpoints are selected to equally divide the area under the probability distribution function curve of a standard normal distribution due to the Gaussianity nature of real life data [21]. Fig. 3 illustrates the process of a sample 2D SAX conversion. In particular, the 2D SAX_Length is now defined as w1  w2 ; and the definition of the 2D SAX_Level is identical to that of the 1D case. The total operations required for such a conversion is no more than mn additions, plus mn=w1 w2 divisions, plus ðSAX LevelÞ  mn=w1 w2 comparisons. Also, the MINDIST for 2D SAX is now redefined as Eq. (3). Actually, such a definition can be further extended to 3D or even higher dimensionality.

vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi rffiffiffiffiffiffiffiffiffiffiffiffiuX w2 u w1 X mn b; C bÞ ¼ t ^i;j ; ^ci;j Þ2 MINDISTð Q distðq w1 w2 i¼1 j¼1

ð3Þ

The 2D SAX conversion can be directly applied to images, which are special cases of two-dimensional data. For color images, the 2D SAX conversion can be applied to each color channel, respectively,

Fig. 3. Conversion of a 2D data into its 2D SAX representation. (a) The original real valued 2D data of size 64  64. (b) The block wise average representation and the 2D SAX conversion. (c) The resultant 2D SAX representation.

Fig. 4. Conversion of a 2D image into its 2D SAX representation. (a) The original noisy image. (b) Noise suppression using Gaussian filtering. (c) The resultant 2D SAX representation. (d) Blowup of the lower-left corner of image (c).

346

J. Chen et al. / Image and Vision Computing 28 (2010) 343–351

and the MINDIST can simply be defined as the summation of the MINDIST values of all the channels. In this paper, only grayscale images are considered as the palmprint images [28,29] to be used are all intensity images. For a 2D image I, image filtering can be applied prior to the 2D SAX conversion. Suppose G is a digital filter, we denote the filtering process as Eq. (4). This enhances the flexibility of the 2D SAX representation, since image filtering is a fundamental tool in most image processing techniques. Fig. 4 illustrates the process of converting a noisy Lenna into its 32  32 2D SAX representation, in which the SAX_Level equals four. The original image occupies 262K bytes and the 2D SAX only needs 256 bytes to store. Tremendous dimensionality and numerosity reduction is thus achieved.

Q ¼IG

ð4Þ

3. 2D SAX conversion for palmprints Two public palmprint databases are used for the experiments in this work [28,29]. The PolyU Palmprint Database [28] contains 7752 grayscale palmprint images (384  284 pixels, 96 dpi) corresponding to 386 different palms. For each palm, there are around 20 samples collected in two sessions, where around 10 samples were captured in the first session and the second session, respectively. The average interval between the two sessions was 2 months. Palmprints in this database were captured using a patented palmprint capturing device [30] in which pegs are used to limit the movement and the distortion of the user’s palm. The illumination of the devices is tuned to facilitate the capture of palm

line details. As a result, palmprints in the PolyU Palmprint Database are believed to have relatively high image qualities. The other database used is the CASIA Palmprint Image Database [29] which contains 5239 grayscale palmprint images (640  480 pixels, 72 dpi) collected from 301 individuals. All palmprints were collected in one single session. Comparing to the PolyU capturing device, the device used for constructing the CASIA database imposes less physical constrains on the user’s palm, leading to more obvious palm movements, distortions and blurring in the captured palmprint images. Fig. 5 compares sample palmprint images from the two databases. Fig. 5(c) and (d) belong to the CASIA database and are captured from the same palm. However, intra-class variation caused by shifting and distortion can be easily observed. Moreover, Fig. 5(d) is obviously blurred probably because of the palm motion or the camera defocusing. Before the 2D SAX conversion, the input palmprint images should be aligned to a predefined universal coordinate to facilitate consistent ROI extraction. We adopt a palm coordinate system similar to the second square-based palm coordinate system proposed in [4]. The following steps are performed for the palmprint alignment. A sample palmprint form the PolyU database is used to illustrate this procedure in Fig. 6. (1) A fixed threshold is applied to convert the original image into a binary image (Fig. 6b). Isolated pixels, spurs and leaks are removed using morphological operations. (2) The palm boundary (Fig. 6c) is obtained using a boundary tracking algorithm [31].

Fig. 5. Sample palmprint images. Image (a) and image (b) belong to the PolyU database. They are captured from the same palm. Image (c) and image (d) belong to the CASIA database. They are also captured from the same palm.

Fig. 6. Palmprint image preprocessing: alignment and ROI extraction.

347

J. Chen et al. / Image and Vision Computing 28 (2010) 343–351

(3) The maximum curvature points P1 and P2 between the fingers (Fig. 6d) are located using a curvature maxima finding algorithm [32]. (4) P1 and P2 are connected by a line L1. Draw another line L2 passing through the middle point P3 of L1 perpendicularly (Fig. 6e). A point P4 is found on L2 so that the length between P3 and P4 equals a predefined value. (5) A square area S (Fig. 6f/g) of fixed size (135  135 pixels) is extracted with P4 as its center. S is defined as the ROI for 2D SAX conversion. (6) Non-uniform illumination correction is performed to enhance the palmprint texture (Fig. 6h). Fig. 7 shows the ROI extraction result for a palmprint from the CASIA database. The CASIA palmprints are first resized to 380  284 pixels before the ROI extraction. The extracted palmprint ROI are then converted to the 2D SAX form for palmprint representation and matching. Several sample PolyU palmprint ROIs are shown in Fig. 8, in which each two ROIs in the same column are from the same palm; and the ROIs in the first and second rows are from session one and session two, respectively. Visually, the ROIs are extracted consistently. An average filter G is employed to filter the extracted ROIs so as to suppress noises caused by factors such as dirt, injuries and camera noises. To fulfill our objective of low computational complexity, the filter G is decomposed into two 1D average filters, so that the filtering process can be equivalently implemented as the combination of two 1D filtering processes as is illustrated in Eqs. (5) and (6). And the 1D average filtering can be efficiently implemented such that each pixel requires only one addition plus one subtraction. As a result, the overall complexity of the average filtering process is around 4N, where N is the total number of pixels in the image. The 128  128 pixels central part of the filtered ROI is cropped

for the 2D SAX conversion. Fig. 9 shows the corresponding 2D SAX representations for the palmprint ROIs shown in Fig. 8, where w1 and w2 equal 32 and the SAX_Level is set to 4. These 2D SAX representations are used as palmprint templates. And the storage size of each template is 256 bytes.

2 3 1 617 7 61 1 ... 1 7 6 7 6 G¼6 7 ¼ 6 .. 7 7  ½1 1 ... 1 ¼ U  V 4... ... ... ...5 6 4. 5 1 1 ... 1 1

ð5Þ

I  G ¼ I  ðU  VÞ ¼ ðI  UÞ  V

ð6Þ

2

1

1

... 1

3

The MINDIST between two templates is adopted as the similarity measurement of the corresponding two palmprints. The smaller the MINDIST is, the more similar the two palmprints are. As human palm is a particularly flexible organ, distortions are unavoidable during palmprint capture, leading to the fact that no alignment scheme is able to extract the ROI perfectly [4]. We need to vertically and horizontally translate one of the templates to find the best match, or the smallest MINDIST value. The definition of the MINDIST after translation is identical to Eq. (3), except that the values of w1 and w2 should be modified according to the size of the overlapping template area. The range of the vertical and the horizontal translations are defined from 2 to 2. In the next section, we will show the discriminative power of the SAX representation and the MINDIST for the palmprint authentication through verification and identification experiments. 4. Experiments A complete one to one matching experiment is performed on the PolyU palmprint database [28] to test the effectiveness of the proposed method. There are altogether 74068 genuine matching

Fig. 7. A CASIA palmprint and the corresponding ROI extraction.

Fig. 8. Sample palmprint ROIs.

348

J. Chen et al. / Image and Vision Computing 28 (2010) 343–351

Fig. 9. 2D SAX representations for the sample palmprint ROIs in Fig. 8.

and 29968808 imposter matching. The experimental settings are as follows: G is a 7  7 average filter; SAX_Length = 32  32 = 1024, and SAX_Level equals 4. Fig. 10 shows the score distributions of the genuine matching and the imposter matching, as well as the ROC (Receiver Operating Characteristic) curve. The EER of the proposed method is 0.3% with the corresponding threshold at 46.2. We compare the performance of the proposed method to that of three widely referenced palmprint authentication approaches: PalmCode [7], FusionCode [11,12] and Competitive Code [18]. The comparison results are shown in Table 2. To avoid any uncertainty in the implementation, the accuracies of the three reference approaches are quoted from the original papers. The verification results for the PalmCode [7], the FusionCode [11], and the Competitive Code [18] were also generated from one to one matching among 7752 palmprints in the PolyU database. This ensures the fairness of the performance comparison. It can be observed from Table 2 that in general the proposed method over performs the FusionCode and the PalmCode. In terms of EER, our method is comparable to the Competitive Code method. It is true that for small FAR values, the Competitive Code method performs better. However, the outstanding performance of our methods for relatively large FAR values can be regarded as a trade off. Also, our method does not rely on any training of the parameters, and has very low computational complexity and requires relatively small storage size. Table 3 lists the estimated computational complexity of the different feature extraction methods. We only focus on the feature extraction process since the

Table 2 Verification accuracies of different methods on the PolyU database. False acceptance rate

106 105 104 103 102 101 Equal error rate

False rejection rate 2D SAX (%)

Competitive Code (%)

FusionCode (%)

PalmCode (%)

5.9 4.1 1.9 0.5 0.1