Handwritten Signature Verification Using Image ... - IEEE Xplore

Handwritten Signature Verification Using Image Invariants and Dynamic Features Abdullah I. Al-Shoshan Computer Science Department, Qassim University P.O Box 1928, Unaizah 51911, Saudi Arabia {E-mail: [email protected]} Abstract In this paper, a development of automatic signature classification system is proposed. We have presented offline and online signature verification system, based on the signature invariants and its dynamic features. The proposed system segments each signature based on its perceptually important points and then, for each segment, computes a number of features that are scale, rotation and displacement invariant. The normalized moments and the normalized Fourier descriptors are used for this invariancy, while the speed of pen is used as a dynamic feature of the signature. In both cases the data acquisition, pre-processing, feature extraction and comparison steps are analyzed and discussed. Both static and dynamic features were used as an input to a neural network. The neural network used for classification is a multi-layer perception (MLP) with one input layer, one hidden layer and one output layer. The performance of the proposed system is presented through simulation examples.

1. Introduction A biometric can be classified as physiological or behavioral. Physiological biometrics measures some physical feature of the subject such as face, fingerprint, iris, and hand and finger geometry [1, 2]. Behavioral biometrics measures a user action, such as speaking, writing and walking. Most physical features remain relatively stable over time, while behavioral characteristics are in control of the subject and tend to change over the short and long terms due to health, physiological state and aging. Further, the subject can often create false negatives, hiding their true identity by consciously changing the behavior being measured. This implies that behavioral biometrics must be collected from a cooperative or unaware subject. While physiological biometrics may be adequately represented by a single sample, behavioral biometric generally requires several samples due to their inherent variability. Signatures vary depending on fatigue, mental and physical state, and writing position.

Signature verification is of particular importance as it is the only widely accepted method for endorsing financial transactions. An important advantage of the signature over other biometric is its long standing tradition in many commonly encountered verification tasks [3-5]. It has been used for decades in civilian applications while other methods (e.g., fingerprints) still have the stigma of being associated with criminal investigation. In other words, signature verification is already accepted by the general public. Experiments investigating the effects of packet loss for two online signature verification systems distributed over the Internet were explored in [6]. Handwritten signature verification can be divided into on-line (or dynamic) and off-line (or static) verification [7-10]. On-line verification refers to a process where the signer uses a special pen called a stylus to create his signature that produces measurements such as pen location, speed, and pressure. Off-line verification is concerned with the verification of a signature made by a normal pen. Various different approaches to both classes have been proposed. Automatic signature verification takes into account the intrinsic variability of a signature according to country, age, time, habits, psychological or mental state, physical conditions and practical signing conditions constitute the major challenge for any system designer. In this paper we will focus on a subset of these techniques, those employing the idea of human signature to match image patterns. The notion of matched spatial filter (MSF) is a venerable one with a long history [11, 12]. While by itself it cannot account for invariant recognition, it can be coupled to invariant mappings or signal expansions, and is therefore able to provide invariance to rotation and scaling in the image plane.

2. Features Extraction 2.1 Static Features When processing a signature after a person has signed it, we can only extract its static features by looking at it as an image. Dealing with the signature as an image, we can

Proceedings of the International Conference on Computer Graphics, Imaging and Visualisation (CGIV'06) 0-7695-2606-3/06 $20.00 © 2006

use some of the image processing tools. In this paper, we have considered the normalized Fourier descriptors and the normalized central moment's theories. 2.1.1 Fourier Descriptor (FD) The Fourier descriptor is a method of describing the shape of a closed, planar figure. Given a figure in the complex plane, the contour can be traced yielding a one dimensional complex function of time. If the contour is traced repeatedly, the periodic function that results can be expressed in a Fourier series. The Fourier descriptor of a contour is defined as this series. If the complex function representing the contour is x(n), then A(k) is the FD of this expressed as follows:

1 A(k ) = N

N −1

¦ x(n)e

§ j 2ukπ · ¨ ¸ © N ¹

(1)

u =0

To make the FD invariant to scale, transition and orientation, this is called normalizing the Fourier descriptor. So the Normalized Fourier Descriptor is the FD of a contour such that it has a standard size, orientation and starting point. The NFD is calculated by a procedure obtained by Timothy Wallace and Paul Wintz [13]. Referring to reference [13], this procedure is built on an assumption that the contour does not cross itself and it is traced counter clockwise. Therefore, A(1) will be always the largest coefficient in magnitude. But in the case of signature, the complex function is not a closed contour and it may cross itself and it should not traced CCW. So, we have to find the first largest coefficient, which we will call it the first largest magnitude (flm). So, we have to find the flm and have the equation:

m = k − flm

(2)

Multiply A(i) by j [(i − k )u + ( flm − i )v ] / ( k − flm )

e

then by

e j (i − flm )2π

m [k ]

we get the NFD of the signature. 2.1.2 Normalized Central Moment Moments are a descriptive technique with an intuitive basis in the study of the mechanics of bodies. The initial approach is usually credited to Hu [14]. Teague discussed the concept of orthogonal moment sets and showed how the concept of Hu’s moment invariant may be extended to higher-order moments. Other work has shown that both the moment of the binary silhouette and gray-valued image segment may be used together for classification. Furthermore, the two-dimensional moment may be used to obtain moments of projections through the image segment, which may be used in classification schemes. For a 2-D discrete function f(x,y), the moment, mpq, of order (p+q) is defined as [11, 14]:

m pq = ¦¦ x p y q f ( x, y ) x

(3)

y

for p,q = 0, 1, 2, … To make the moments invariant to position transformation, we have to calculate the moments with respect to the image centroid instead of the origin (0,0). This is called the central moments and defined as:

μ pq = ¦ ¦ ( x − x ) p ( y − y ) q f ( x, y ) x

(4)

y

where

x=

m10 m00

and

y=

m01 m00

(5)

Although the above moments are invariant to scale, rotation and transition, they are not invariant to partially translating, rotating or scaling a part of the image while keeping the rest of the image without change. Also, image scaling ratio must be the same with respect to the x and the y directions. In the signature classification system, the signatures are not guaranteed to be scaled with same ratio because the signer may enlarge his signature in one direction but not the other. Therefore, input size normalization will be considered an important issue. The invariant moments also suffer from another problem; as we increase the moments order, their values become smaller and smaller.

2.2 Dynamic Features When a person signs on paper, he leaves a static image of his signature. Forgers practice the art of reproducing the image (or shape) of a signature, with little regard to the motions that caused the image. Pen motions are ephemeral and are not captured on the paper. When a signature is captured with a digitizer, the pen motions (dynamics) are recorded. When signing, the hand can operate in a regime known as ballistic motion, where the muscles are not controlled by sensory feedback. Ballistic motions are generally rapid, practiced motions whose accuracy increases with speed [1]. In the on-line signature there is an important feature that can be extracted, which is the speed of the signature. During the signing process, the speed of the pen ball is changing at every point of the signature. These changes are repeated in a fixed way every time a person signs again. To find out the speed of the signature we have to record the time at which a specific point is sampled. Through a visual basic program we recorded the coordinate and the time of each point of the signature and then we calculated the speed of the signature as Speed = Distance / Time

(6)

3 Artificial Neural Networks One of the applications of pattern classification (PC) using ANN is signatures classification [15, 16]. Signature classification using ANN can be done in two


approaches. One is by using the structural analysis of the signature as number of loops and the length of the vertical and horizontal lines and such other features. The other approach is by using the global feature of the signature as invariant moments, normalized Fourier descriptor, and speed of the signature. Template matching is extensively used in low-level vision tasks to localize and identify patterns in images. Two methods are commonly used: (1) image subtraction, (2) correlation [17]. When the images are normalized to have zero average and unit norm, the two approaches give the same result. The usual implementation of the above methods relies on the Euclidean distance. Other distances can be used and some of them have better properties such as increasing robustness to noise and minor deformations [18]. One of the reasons for which template matching by correlation is commonly used is that correlation can be shown to be the optimal (according to a particular criterion) linear operation by which a deterministic reference function can be extracted from additive white Gaussian noise [17]. Several performance metrics are available for correlation filters that describe attributes of the correlation plane. The signal to noise ratio (SNR) is just one of them. Other useful quantities are the peak-to-correlation energy, the location of the correlation peak and the invariance to distortion. As correlation is typically used to locate and discriminate objects, another important measure of a filter's performance is how well it discriminates between different classes of objects [12, 19]. The classifier which minimizes the probability of error is the Bayes classifier. The neural network used for classification is a multilayer perception (MLP) with one input layer and one hidden layer and one output layer. An MLP is believed to have generalization capability, that is, it can produce correct (or nearly correct) output for input not used during training and it can overcome the small differences between the input patterns [20, 21]. In the MLP each node may use the thresholding function sigmoid, tanh or any other suitable function. In this project, the tanh function is used. The error function used in the MLP is the sum of square error. The learning algorithm used with the MLP is the back-propagation algorithm. The back-propagation algorithm is a gradient descent method to minimize the square error cost function.

4. Examples Identification answers the question “who are you”, while verification answers the questions “are you who you claim to be”. Identification is a 1:N match, which can be quite difficult when N is in the millions, for example potential shoppers at a large retailer. Verification is a much more tractable problem, requiring a 1:1 match, based on a reference biometric retrieved by an identification claim (typically from the user) [1]. To generate the signatures database, the shape and speed of each signature is recorded using a pen tablet and we have used the features: speed, NFD, and NCM as the inputs to the ANN. We used the Delta-Bar-Delta technique. To

evaluate the speed of the signature we have to record the time at which a specific point is sampled, and equation (6) has been used. Fig. 1 shows an example the speeds of two signatures.

Figure 1: The average speed of two signatures. A collection of 100 persons, with 10 realizations each, has been selected, which are divided into two sets: 500 samples for training and the rest are used for testing. The length of a signature may vary from time to time. Therefore, the length of the signature and speed vector should be normalized for every signature in one category so we can compare them. After normalizing the length of the signatures, we calculate the largest error between the recorded signature and the average. This error will be our measurement boundary. If the error of the new signature is greater than a threshold, it will be rejected. Using the NCM, image scaling ratio must be the same. We used the recorded signature and compare it with our database library, depending on the signature points, speed and NFD. One of the main problems in the NCM is its inability to be invariant to local variations in the signature. These signatures are generated using a pen tablet and fed to the computer as points consisting of x and y-position and time. The results of using each feature independently are shown in Table 1. Table 1: Results of Using Each Descriptor with the ANN. Features % Correct % Error Speed 90 % 10 % NFD 92 % 8% NCM 83 % 17% Combining the three features, we get the results shown in Table 2. Table 2: Results of using all descriptors with the ANN. Features # of iterations % Error Speed, NFD, NCM 103 0% The reason behind this high performance is the combination of the features. By tying down the


threshold, the verification error in each feature is minimized by the other one. For example, the 17 % error when using only the NCM is classified correctly by the ANN when using the speed or the NFD features.

[9]

4. Conclusions

[10]

In this paper, a dynamic signature verification system to meet the needs of the retail sector has been developed. The proposed system segments each signature based on its perceptually important points and then computes for each segment a number of features that are scale, rotation and displacement invariant. The normalized moments and the normalized Fourier descriptors are used for this invariance, while the signature speed was used to catch the signature dynamics. The proposed method produced a 0% error rate when using combining the speed, the NCM and the NFD in the ANN.

REFERENCES [1]

[2]

[3]

[4]

[5] [6]

[7]

[8]

T. G. Zimmerman, G. F. Russell, A. Heilper, B. A. Smith, J. Hu, D. Markman, J. E. Graham, C. Drews, Retail Applications of Signature Verification, Proceedings of SPIE2004, Volume 5404, Biometric Technology for Human Identification, Anil K. Jain, Nalini K. Ratha, Editors, 206-214, August 2004. R. Kashi, J. Hu, W. Nelson, W. Turin, A Hidden Markov Model approach to online handwritten signature verification, International Journal on Document Analysis and Recognition, 1, 102-109, 1998. M. Turk and A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience, 3(1), 71-86, 1991. L. L. Lee, T. Berger, and E. Aviczer, Reliable OnLine Human Signature Verification Systems, IEEE Trans. Pattern Recognition and Machine Intelligence, 18(6), 643-647, 1996. 3L. Yang, B. Widjaja, R. Prasad, Application of hidden markov models for signature verification, Pattern Recognition, 28(2), 161–170, 1995. J. Richiardi, J. Fierrez-Aguilar, J. Ortega-Garcia, A. Drygajlo, On-Line Signature Verification Resilience to Packet Loss in IP Networks, Proc. 2nd COST 275 Workshop on Biometrics on the Internet: fundamentals, advances and applications, 9-14, 2004. J. J. Brault and R. Plamondon, Segmenting handwritten signatures at their perceptually important points, IEEE transaction on Pattern Analysis and Machine Intelligence, 15, 953-957, 1993. R. Plamondon and S. N. Srihari, On-line and Offline Handwriting Recognition: A Comprehensive Survey, IEEE Trans. Pattern Recognition and Machine Intelligence, 22(1), 63-84, January 2000.

[11] [12]

[13]

[14] [15]

[16]

[17] [18]

[19]

[20]

[21]

S. N. Srihari, A. Xu, and M. K. Kalera, Learning Strategies and Classification Methods for OffLine Signature Verification, 9th Int. Workshop on Frontiers in Handwriting Recognition (IWFHR’04), 161-166, 2004. B. Yanikoglu and A. Kholmatov, An improved decision criterion for genuine/forgery classification in on-line signature verification, Online-Sig-ICANN2003, 1-4, 2003. Rafael C. Gonzalez and Richard E. Woods, Digital image processing, Addison-Wesley, 1993. D. Casasent and G. Ravichandran, Advanced distortion-invariant minimum average corelation energy (MACE) filters, Applied Optics, 31(8), 1109-1116, 1992. Timothy P. Wallace and Paul A. Wintz, An Efficient Three-Dimension Aircraft Recognition Algorithm Using Normalized Fourier Descriptors, Computer Graphics and Image Processing, 13, 99-126, 1980. M. K. Hu, Visual Pattern Recognition by Moment Invariants, IRE Trans. Info. Theory, IT-8, 179187, 1962. Benyamin Matityaho and Miriam Furst, Neural Network Based Model for Classification of Music Type, IEEE Cat., 95, 640-645, 1995. P. Mautner, V. Matoušek, T. Maršálek, and M. Šoule, Using Self-organizing Feature Map for Signature Verification, (451) Artificial Intelligence and Soft Computing, 129, 2004. K. Fukunaga, Introduction to statistical pattern recognition, Academic Press, 1990. S. I. Sudharsanan, A. Mahalanobis, and M. K. Sundareshan, Unified framework for the synthesis of synthetic discriminant functions with reduced noise variance and sharp correlation structure, Optical Engineering, 29(9), 1021-1028, 1990. J. Ben-Arie and K. R. Rao, On the Recognition of Occluded Shapes and Generic Faces Using Multiple Template Expansion Matching, CVPR93., 1, 214-219, 1993. T. Shuttleworth and R. Wilson, A Neural Network for Triad Classification, Proceedings of the International Computer Music Conference, San Francisco, 1, 428-431, 1995. J. Bromley, J. Bentz, L. Bottou, I. Guyon, Y. L. Cun, C. Moore, E. Sackinger, and R. Shah, Signature verification using a siamese time delay neural network, International Journal of Pattern Recognition and Artificial Intelligence, 7(4), 669– 688, 1998.
