Pattern Recognition Letters 26 (2005) 2390–2399 www.elsevier.com/locate/patrec
Off-line Chinese signature verification based on support vector machines Hairong Lv *, Wenyuan Wang, Chong Wang, Qing Zhuo Department of Automation, Tsinghua University, Beijing 100084, China Received 12 April 2004; received in revised form 12 April 2005 Available online 22 June 2005 Communicated by T. Breuel
Abstract This paper proposes a novel off-line Chinese signature verification method based on support vector machines. The method uses both static features and dynamic features. The static features include moment features and 16-direction distribution (an improvement on 4-direction distribution). The dynamic features include gray distribution and stroke width distribution. At last, support vector machine is used to classify the signatures. The main steps of constructing a signature verification system are discussed and experiments on real data sets show that the average error rate can reach 5%, which is obviously satisfactory. 2005 Elsevier B.V. All rights reserved. Keywords: Signature verification; Moment features; Direction distribution; Gray distribution; Stroke width distribution; Support vector machines (SVM)
1. Introduction Handwritten signature is one of the most widely accepted personal attributes for identity verification. As a symbol of consent and authorization, especially in the prevalence of credit cards and bank cheques, handwritten signature has long
*
Corresponding author. Tel.: +86 1062773349. E-mail address:
[email protected] (H. Lv).
been the target of fraudulence. Therefore, with the growing demand for processing of individual identification faster and more accurately, the design of an automatic signature verification system faces a real challenge. Handwritten signature verification can be divided into on-line (or dynamic) and off-line (or static) verification. On-line verification refers to a process that the signer uses a special pen called a stylus to create his or her signature, producing the pen locations, speeds and pressures, while
0167-8655/$ - see front matter 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2005.04.013
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
off-line verification just deals with signature images acquired by a scanner or a digital camera. In general, off-line signature verification is a challenging problem. Unlike the on-line signature, where dynamic aspects of the signing action are captured directly as the handwriting trajectory, the dynamic information contained in off-line signature is highly degraded. Handwriting features, such as the handwriting order, writing-speed variation, and skillfulness, need to be recovered from the grey-level pixels. During the last few years, researchers have made great efforts on off-line signature verification. Ng and He used a neural network expert system to identify Chinese signature (Ng and Ong, 1993; He et al., 2002). Yingyong and Hunt used some global and grid features to compute the Euclid distance of two signatures (Yingyong and Hunt, 1994). Buryne tried to apply elastic image matching in signature verification (Buryne and Forre, 1986). Methods based on texture analyses were also applied in the area, with works by Liu et al. (1997) and Liu (1997). Peter Shaohua discussed the possibility of applying wavelet on signature verification in their paper of 1999 (Peter Shaohua et al., 1999). Justino and El-Yacoubi developed a signature verification system based on Hidden Markov Model (HMM) (Justino et al., 2001 and El-Yacoubi et al., 2000). Wan and Lin presented a off-line signature verification system that only requires the genuine signatures of a new user (Wan et al., 2003).
2391
There are three classes of forgeries: random forgeries, simple forgeries and skilled forgeries. Most systems mentioned above are designed for simple forgeries and random forgeries. In this paper, a novel approach for off-line signature verification system is proposed based on support vector machines. It combines four feature sets: moment features, 16-direction distribution, gray distribution and stroke width distribution. Support vector machine is employed to identify the signature images. The system is effective for not only simple forgeries and random forgeries, but also skilled forgeries. The remainder of the paper is organized as follows: Section 2 introduces the database used in the experiments; Section 3 discusses the preprocessing stage; Section 4 describes the features we select and how to get them; Section 5 provides a brief introduction to support vector machines; In Section 6 the verification strategies and experimental results are presented; Section 7 presents the conclusion and future work.
2. Database The signature database consists of 1100 signature images, scanned at a resolution of 300 dpi, 8-bit grey-scale. They are organized into 20 sets, and each set corresponds to one signature enrollment. There are 25 genuine and 30 forgery signatures in a set. Each volunteer was asked to sign his or her own signatures on a white paper with
Fig. 1. Genuine and forgery signatures.
2392
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
base-lines 25 times. After this process had been done, we invited some people who are good at imitating others handwritings. Before formal collection, they could practice any times they wanted when imitating. Each persons name was imitated 30 times in total. The imitated signatures can be regarded as skilled forgeries. Fig. 1 shows some samples in the database. 3. Preprocessing The preprocessing stage is divided into five different parts: noise reduction, image binarization, data area cropping, width normalization and image thinning (Lam and Suen, 1991). Standard noise reduction and isolated peak noise removal techniques, such as median-filtering and average filtering (Gonzalez and Wintz, 1987), are used to clean the initial image. Morphological operations are applied to fill small holes and to remove small connected components mostly generated by the noisy background. Fig. 2 shows the image binarization and Fig. 3 shows a thinned image.
4.1. Moment features The global shape characteristics can be described by moment. For a M · N sized image, its pP + q rand moment is defined as: lpq ¼ PM1 N 1 ðx xÞp ðy y Þq f ðx; yÞ, where f(x, y) is x¼0 0 the grey value of pixel (x, y) of the original signature image, and PM1 PN 1 x¼0 y¼0 x f ðx; yÞ x ¼ PM1 PN 1 ; x¼0 y¼0 f ðx; yÞ From above moments, we can derive such features: (1) Height to width ratio l20 l02 m0 ¼ þ1 2; l20 þ l02 (2) Incline degree m1 ¼
h ; p=2 l02 l20 þ
where h ¼ tan1 (3) Extension degree m2 ¼
4. Feature extraction
k1 k2 ; k1 þ k2
where, k1 ¼ The choice of a powerful set of features is crucial in signature verification systems. In our system, we use four groups of categorized features.
k2 ¼
ðl02 þl20 Þþ
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi ðl02 l20 Þ2 þ4l211 . 2l11
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi 2 ðl02 l20 Þ þ4l211
, 2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 ðl02 þ l20 Þ ðl02 l20 Þ þ 4l211
Fig. 2. Image binarization.
Fig. 3. Example of thinning algorithm.
2
.
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
2393
(4) Extension degree 2 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðl02 þ l20 Þ=m00 ; m3 ¼ size pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi where, size ¼ ðxmax xmin Þðy max y min Þ, m00 ¼
M 1 X N 1 X 0
f ðx; yÞ.
0
In the moment features (5)–(8) below, we define lpq ¼ lþ pq lpq ; ðp þ q ¼ 3Þ: (5) Horizontal excursion degree þ l30 l 30 m4 ¼ þ 1 2; lþ 30 þ l30 (6) Vertical excursion degree þ l30 l 30 m5 ¼ þ 1 2; lþ 30 þ l30 (7) Horizontal extension equilibrium degree þ l21 l 21 m6 ¼ þ1 2; lþ 21 þ l21 (8) Vertical extension equilibrium degree þ l12 l 12 m7 ¼ þ1 2. lþ 12 þ l12 The eight moment features make up of the * T moment feature vector fm ¼ fm0 ; . . . ; m7 g . 4.2. Direction distribution Direction distribution is one of texture analysis. Researchers usually use four directions, which are horizontal, vertical, two 45 incline directions. In this paper, we compute the direction probability distribution on 16 directions, such as Fig. 4. Our experiments show that, in comparison to using four directions, using 16-directions brings more discriminative information in describing the small differences between different writers. However, more than 16 directions may be too sensitive for the natural alteration between the handwritings of the same person. For each point P in the thinned signature strokes, we draw a 9 · 9 square around it (Fig. 4), at the edge of which a corresponding direction number is assigned. We trace the stroke from P
Fig. 4. Sixteen direction.
until we reach the edge of the square, and the number assigned is the direction of P. The stroke-tracing algorithm is as follows: (1) The 8 neighbor direction of point P is defined in Fig. 5; (2) If P is the point of which we want to get the direction, search the next point (the point is on the thinned stroke) to begin with direction 0; (3) If the last tracing direction is i, the direction of current point is specified by the following steps: (A) Since the opposite direction should not be revisited, we only check along the i direction; if the next point is achieved, the direction is also i, then goto (4); (B) Symmetrically check (i + r) · 45, where r follows the sequence: ±1, ±2, ±3, until
Fig. 5. 8 neighbor direction.
2394
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
Fig. 6. Computation examples: (a) direction 2, (b) direction 15.
we reach the next point at (i + r*) · 45 and the direction is (i + r*)%8; (4) Repeat (3) until we reach the edge of the square.
T P15 . . . ; d 15 g ; ð i¼0 d i ¼ 1Þ. Fig. 7 shows the direction distribution of some genuine signatures and forgeries.
4.3. Gray distribution Fig. 6(a) and (b) shows two examples of computing the direction. The black points are the tracing paths. For all points on the thinned signature strokes, we compute the percentage of each direction, and * the result forms a vector of 16-D: fd ¼ fd 0 ; d 1 ;
For static signature images, we cannot derive dynamic information directly. What we can do is to derive some features that can partly represent the dynamic information. One of them is gray level distribution. During the writing process, different
Fig. 7. Direction distribution examples.
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
2395
Fig. 8. Gray distribution examples.
gray level distributions are formed, due to the different pressures and velocities, Some researchers deal with the whole image (Ammar et al., 1986), which has an obvious disadvantage that the background takes a great proportion of the signature. Since we find that nearly all of the pixels gray values exceed a given threshold (for white is 255 and black is 0), we made an improvement that we just set two thresholds to hold only the foreground points and edge points. The two thresholds are set to 85 and 205 in our experiment through experiences. We only concern the remaining points whose gray value is between the two thresholds and we evenly divide [85, 205] into 12 segments, and the percentage of the points whose gray value falls in the corresponding segment is one of the * * values of the feature vector fg . Thus fg is a 12-D * T vector and fg ¼ fg0 ; g1 ; . . . ; g11 g , which reflects the gray distribution. Fig. 8 shows the gray distribution of some genuine signatures and forgeries. 4.4. Stroke width distribution Besides gray distribution, another dynamic feature is stroke width distribution. In this paper, we use a simple but effective method stated as follows. In Fig. 9, P(x, y) is on
Fig. 9. Width estimation.
the thinned signature strokes, L(xL, yL), R(xR, yR), U(xU, yU) and B(xB, yB) are edge points of the binary signature image, satisfying the following constraints: ( xL < x < xR ; y L ¼ y ¼ y R ; yU < y < yB;
xU ¼ x ¼ xB .
We compute the direction of point P (shown in Fig. 4). There are four cases of the result: (1) The direction is 0: then kBUk is the width of point P. (2) The direction is 8: then kLRk is the width of point P.
2396
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
Fig. 10. Width distribution examples.
(3) The direction is between 1 and 7, such as Fig. 7. kPD1k is the distance from P to LB and kPD2k is the distance from P to UR. They are calculated as follows: jx xL jjy y B j ffi; kPD1k ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 2 ðx xL Þ þ ðy y B Þ jx xR jjy y U j ffi. kPD2k ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 2 ðx xR Þ þ ðy y U Þ
geometrical interpretations please refer Burges (1998); Cristianini and Shawe-Taylor (2000). Consider a two-class classification problem and a set of training vectors {Xi}, i = 1, . . ., M with corresponding binary labels Ci = 1 for the ‘‘positive’’ and Ci = 1 for the ‘‘negative’’ class. In classificab to a test vector T by tion an SVM assigns a label C evaluating ! X b ¼ sign C ai C i KðT ; X i Þ þ b ð1Þ i
And the width of point P is kPD1k + kPD2k. (4) The direction is between 9 and 15. We compute the distance of P to LU and P to BR, and then add them together. For all points on the thinned signature strokes, we compute the percentage of each width (from 1 * to 12), the result forms a vector fw ¼ fw0 ; w1 ; . . . ; T w11 g . Fig. 10 shows some examples of stroke width distribution. 5. Support vector machines Here, we provide a brief introduction to support vector machines (SVM), especially to support vector classification (SVC). For more details and
The weights ai and the bias b are SVM parameters and adopted during training by maximizing X 1X LD ¼ ai ai aj C i C j KðX i ; X j Þ ð2Þ 2 i;j i under the constraints X 0 6 ai 6 C and ai C i ¼ 0
ð3Þ
i
with C a positive constant weighting the influence of training errors. K(Æ,Æ) is the kernel of the SVM. A solution for the ai implies a value for b. If K(Æ,Æ) is positive definite, (2)–(3) is a convex quadratic optimization problem, for which the convergence towards the global optimum can be guaranteed. However, obtaining this solution for
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
real-world problems can be quite demanding and require sophisticated optimization algorithms like chunking, decomposition or sequential minimal optimization (Cristianini and Shawe-Taylor, 2000). Usually ai = 0 for the majority of i and thus the summation in (1) is limited to a subset of Xi, which therefore is called the set of support vectors (SV). We usually use three kinds of kernel functions, and they are: (a) Polynomial kernel K(x, xi) = [(x Æ xi) + 1]d, here, d is the order. In our experiment, d is set to 2. (b) Radial basis function kernel Kc(x, xi) = exp(ckx xik2). In our experiment, c is set to 1. (c) Two layer neural network kernel K(x, xi) = tan h(v(x Æ xi) + c). In our experiment, m is set to 1 and c is set to 0.
2397
are labeled ‘‘1’’ in a persons set. After this step, we have 480 ‘‘+1’’ samples and 600 ‘‘1’’ samples in total. Randomly select 240 ‘‘+1’’ samples and 300 ‘‘1’’ samples to be training samples, and others for testing. In the experiment, the vector for a pair of signature is computed by (4): *
f ¼ ðu1 ; u2 ; . . . ; uN Þ
T
¼ ðju11 u21 j; ju12 u22 j; . . . ; ju1N u2N jÞ
T
ð4Þ
* * T Here, f1 ¼ ðu11 ; u12 ; . . . ; u1N Þ and f2 ¼ ðu21 ; u22 ; . . . ; u2N ÞT are the selected feature set of the two sig-
natures of the pair (ui is among mk, dk, gk and wk). Each pair has a different signature from others, and the feature vector is obtained through the differences between the two signatures in the pair, which guarantees the independences of samples in the data set. 6.2. Experimental results
6. Experiments 6.1. Verification strategies and feature generation for SVM Our goal is to compare a questionable signature with only one corresponding authentic signature. First, we reconstruct the dataset from the original signature images database. For each set of the database, its 25 genuine signatures produce 24 pairs of signatures, and each pair is assigned a label ‘‘+1’’. For example, if the 25 genuine signatures are randomly labeled g1–g25 as a sequence, the 24 pairs of signatures would be g1–g2, g2–g3, g3–g4, . . . ,g24–25. For each forgery signature of a person, randomly select a genuine signature of the same person to make up of a pair and label the pair ‘‘1’’. There are 30 pairs of signatures that
To verify the performance of our approach, we design five groups of experiments with the selection of different sub feature sets and the combination. Besides, we test the SVM classification performances with different kernels. Tables 1–5 shows the experimental results. From the tables above, we find that only with * the feature set fd itself we can reach a high classification performance, which benefits from the right selection of direction counts. When we combine all of the feature sets together, we get a better classifier with the average error rate about 5.0%. As we know, for English handwritten signature verification system designed for skilled forgeries, the ever-reported accuracy rate is less than 80% (Mizukmi et al., 1999; Yoshiki et al., 2002). Of course, they are based on different databases.
Table 1 Result of the feature set fm (moment features) Class of SVM
Polynomial (%)
Radial basis function (%)
Sigmoid (%)
FRR (false reject rate) FAR (false accept rate) Average error rate
20.0 18.3 19.1
19.6 17.7 18.5
19.6 18.0 18.7
2398
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
Table 2 Result of the feature set fd (direction distribution) Class of SVM
Polynomial (%)
Radial basis function (%)
Sigmoid (%)
FRR (false reject rate) FAR (false accept rate) Average error rate
12.5 13.3 13.0
12.1 12.7 12.4
12.1 13.0 12.6
Table 3 Result of the feature set fg (gray distribution) Class of SVM
Polynomial (%)
Radial basis function (%)
Sigmoid (%)
FRR (false reject rate) FAR (false accept rate) Average error rate
30.4 32.3 31.5
30.0 32.0 31.1
30.4 32.0 31.3
Table 4 Result of the feature set fw (stroke width distribution) Class of SVM
Polynomial
Radial basis function (%)
Sigmoid (%)
FRR (false reject rate) FAR (false accept rate) Average error rate
27.9 28.7 28.3
27.5 28.3 28.0
27.9 28.3 28.1
Table 5 Result of combined feature set: fm, fd, fg, fw Class of SVM
Polynomial (%)
Radial basis function (%)
Sigmoid (%)
FRR (false reject rate) FAR (false accept rate) Average error rate
5.0 5.3 5.2
4.6 5.3 5.0
5.0 5.3 5.2
Linear Fisher Discriminate Analysis (FDA) is also implemented in our experiment. With all the feature sets combined, the average error rate is about 10%. Compared with FDA, SVM-based classifier can gain a higher accuracy (Table 5).
7. Conclusion and future work Off-line Chinese signature verification is a difficult two-class pattern recognition problem. A new approach based on combined features and SVM classifier has been proposed in this paper. A major contribution of this work is the intro-
duction of 16-direction distribution and the application of SVM classifier in the field of signature verification. Despite the fact that the approach shown in this paper seems to be effective, it should be validated on a large signature database where several types of signatures can be taken into account (North American, European, Arabic, etc.). Further perspectives and attractive challenges for future research lie in three aspects: how to extract more effective features, how to combine SVM-based classifier with other signature verification methods and how to use only genuine signatures to build a classifier.
H. Lv et al. / Pattern Recognition Letters 26 (2005) 2390–2399
Acknowledgement The authors would like to thank anonymous reviewers for their very helpful comments and suggestions. References Ammar, M., Yoshida, Y., Fulumura, T., 1986. A new effective approach for off-line verification of signatures by using pressure features. In: Proceedings of the Eighth ICPR, Washington, DC, USA, 1986, pp. 566–569. Burges, C., 1998. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2 (2), 121–167. Buryne, P.de, Forre, R., 1986. Signature verification with elastic image matching. In: International Carnahan Conference on Security Technology Gothenburg, Sweden, August 12–14. Cristianini, N., Shawe-Taylor, J., 2000. Support Vector Machines. Cambridge University Press. El-Yacoubi, A., Justino, E.J.R., Sabourin, R., Bortolozzi, F., 2000. Off-line signature verification using HMMs and crossvalidation. Neural Networks for Signal Processing X, 2000. In: Proceedings of the 2000 IEEE Signal Processing Society Workshop, December 11–13, 2000, vol. 2, pp. 859–868. Gonzalez, C., Wintz, P., 1987. Digital Image Processing, second ed. Addison-Wesley, MA. He, Z.-Yu., Chen, Q.-Hu., Chen, D.-F. 2002. A neural network expert system for Chinese handwriting-based writer identification. Machine Learning and Cybernetics. In: Proceedings 2002 International Conference, vol. 4, November 4–5, 2002, pp. 2194–2196. Justino, E.J.R., Bortolozzi, F., Sabourin, R., 2001. Off-line signature verification using HMM for random, simple and
2399
skilled forgeries. Document Analysis and Recognition, 2001. In: Proceedings of Sixth International Conference, September 10–13, 2001, pp. 1031–1034. Lam, L., Suen, C.Y., 1991. A dynamic shape preserving thinning algorithm. Signal Processing 22, 199–208. Liu, Chenglin, 1997. Writer identification by multichannel decomposition and matching. Acta Automatica Sinica. Liu, Chenglin, Dai, Ruwei, Liu, Yingjian, 1997. Modified wigner distribution and application to writer identification. Chinese Journal of Computers 20 (11). Mizukmi, Y., Miike, H., Yoshimura, M., Yoshimura, I., 1999. An off-line signature verification system using an extracted displacement function. Document Analysis and Recognition, 1999. In: Proceedings of the Fifth International Conference, ICDAR 99, September 20–22, 1999, pp. 757– 760. Peter Shaohua, Deng, Hong-Yuan Mark, Liao, Chin Wen, Ho, Hsiao-Rong, Tyan, 1999. Wavelet-Based Off-Line Handwritten Signature Verification. Computer Vision and Image Understanding 76 (3), 173–190. Ng, G.S., Ong, H.S., 1993. A neural network approach for offline signature verification. In: Proceedings of Computer, Communication, Control and Power Engineering, TENCON 93. IEEE Region 10 Conference, vol. 2, October 19– 21, pp. 770 –773. Wan, L., Lin, Z.-C., Zhao, R.-C. 2003. Off-line signature verification incorporating the prior model. Machine Learning and Cybernetics, 2003. In: International Conference, vol. 3, November 2–5, 2003, pp. 1602–1606. Yingyong, Q., Hunt, B.R., 1994. Signature verification using global and grid features. Pattern Recognition vol. 22 (12) Great Britain, 1621–1629. Yoshiki, Mizukami, Mitsu, Yoshimura, Hidetoshi, Miike, Isao, Yoshimura, 2002. An off-line signature verification system using an extracted displacement function. Pattern Recognition Letters 23 (13), 1569–1577.
