302
Int. J. Artificial Intelligence and Soft Computing, Vol. 2, No. 4, 2011
Offline signature verification and identification by hybrid features and Support Vector Machine Bailing Zhang Department of Computer Science and Software Engineering, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, China E-mail:
[email protected] Abstract: This paper emphasised an approach for offline signature verification and identification. Two image descriptors are studied, including Pyramid Histogram of Oriented Gradients (PHOG), and a direction feature proposed in the literature. Compared with many previously proposed signature feature extraction approaches, PHOG has advantages in the extraction of discriminative information from handwriting signature images. The significance of classification framework is stressed. With the benchmarking database “Grupo de Procesado Digital de Senales” (GPDS), satisfactory performances were obtained from several classifiers. Among the classifiers compared, SVM is clearly superior, giving a False Rejection Rate (FRR) of 2.5% and a False Acceptance Rate (FAR) 2% for skillful forgery, which compares sharply with the latest published results on the same dataset. This substantiates the superiority of the proposed method. The related issue offline signature recognition is also investigated based on the same approach., with an accuracy of 99% on the GPDS data from SVM classification. Keywords: offline signature verification; offline signature recognition; classification; PHOG; pyramid histogram of oriented gradients; DF; direction feature; SVM; support vector machine. Reference to this paper should be made as follows: Zhang, B. (2011) ‘Offline signature verification and identification by hybrid features and Support Vector Machine’, Int. J. Artificial Intelligence and Soft Computing, Vol. 2, No. 4, pp.302–320. Biographical notes: Bailing Zhang received the Master’s Degree in Communication and Electronic System from the South China University of Technology, Guangzhou, China, and PhD Degree in Electrical and Computer Engineering from the University of Newcastle, NSW, Australia, in 1987 and 1999, respectively. He is currently Professor in the Department of Computer Science and Software Engineering at Xi’an JiaotongLiverpool University, Suzhou, China. He has been a Lecturer in the School of Computer Science and Mathematics in the Victoria University, Australia since 2003. Before 2001, he was a Research staff in the Kent Ridge Digital Labs (KRDL), Singapore. Prior to the research activities in Singapore, he worked as a postdoctoral fellow in the School of Electrical and Information Engineering, University of Sydney. His research interest Copyright © 2011 Inderscience Enterprises Ltd.
Offline signature verification and identification
303
includes machine learning and computer vision, with applications in health informatics and bioinformatics.
1 Introduction A person’s handwriting reflects his or her behavioural trait that is characteristic to that particular person. There has been a long history of exploitation of signatures for authentication or authorisation in many fields such as in financial and legal transactions. In contemporary society, the public is still accustomed to the use of signatures in their daily lives. Being inconsistent with the fact that signature verification is generally utilised by the public, automatic signature identification or verification is still limited to academic research by and large. A signature verification system is expected to provide identifiable information of a person based on the intrinsic features extracted from the signature, which should be independent of the handwritten content. There are two different kinds of signature verification systems depending on the method of data acquisition. An offline system realises validation based on handwriting images that are usually collected by scanners or cameras when the handwritings have already been produced. Unlike offline verification, an online system uses the logs the pen timing throughout the duration of the signing process, i.e., subject-specific pen movement dynamics of the signature captured by digitising tablets or instrumented pens, in real time. Offline signature verification is often considered more challenging than online verification due to the absence of the stable dynamic characteristics and the difficulties in extracting individual features from static images. Signature verification generally attempts to distinguish whether a query signature is genuine or a forgery based on the differences between the variations within genuine signatures and the fraud. This involves a number of issues in the decision process. The simplest situation is the random and unprofessional forgery detection (Plamondon and Lorette, 1989; Justino et al., 2001), which occurs when a person simply uses an invalid signature of his or her own, without any knowledge of the authentic signer’s name and signature style. A large portion of work has been done on the random forgery detection. Simple forgery happens when the forger signs with the knowledge of content based on his or her memory on the genuine signature but without close imitations. The most difficult type is skilled forgeries, which are made by people who has previous knowledge of an original instance of the signature and then attempt to mimic the style of the authentic signer as closely as possible. Among the many issues in developing offline signature verification, the key research focus is on the extraction of suitable attributes of the signature images that characterise the differences between signatures of distinct individuals. The performance of a verification system depends greatly on the choice of these attributes or features and their discriminating capability. Much of the research efforts have been made in the past, with many different feature representation approaches proposed. These are roughly categorised into: global or local features (Qi and Hunt, 1994). Global features describe an entire signature as a whole and are usually extracted from all the pixels that lie within the region circumscribing the signature image. Examples include different transforms such as wavelet (Deng et al., 1999),
304
B. Zhang
bitmap image pixels (Zhang, 2006) or some geometric features like the signature area, signature height-to-width ratio, maximum horizontal histogram and maximum vertical histograms. Local features represent a limited portion (e.g., a grid) of the signature for describing a particular geometrical and topological characteristic of local segments, and are often derived at stroke and substroke levels which acquire the distribution of pixels of a signature such as local pixel density or slant (Guo et al., 2001; Fang et al., 2003; Sabourin et al., 1997; Blumenstein et al., 2003, 2004). It is generally recognised that local features carry more accurate and detailed characteristics of handwritten samples. In general, the existing techniques use either a type of feature (global, local, statistical, geometric, etc.) or a combination of different types of features. A recently proposed method is based on the quasi-multiresolution technique using Gradient, Structural and Concavity (GSC) features (Kalera et al., 2004), with promising performances. An issue related to the above verification problems is signature recognition or identification (Kalera et al., 2004), which has been relatively less addressed in the literature. The goal of signature recognition is to identify the owner of a questioned signature given a repository of writing exemplars of several known signers. In addition to potential applications for accessing security sensitive facilities and for processing legal and historical documents, signature recognition can also be used as a stage for verification. Some of latest works can be found in Kalera et al. (2004) and Zhang (2006). With the demands for efficient and robust offline signature verification increased over the years, the research has progressed considerably. Many conventional techniques have been thoroughly investigated, mainly based on similarity comparisons between a test signature and the signatures in a reference set. Usually, a semantically interpretable distance or similarity measure is first established and then the signer-dependent thresholds defined to decide whether the distance between the test signature and the reference and/or template signatures are acceptable. A recent trend is treating verification or identification in the general classification framework, with many ‘off-the-shelf’ algorithms attempted. For example, MLP, kNN and SVM (Ferrer et al., 2005; Justino et al., 2004; Frias-Martinez et al., 2006). Despite the progresses achieved in offline signature verification, the results are still far from being close to real solutions for some important applications such as the automatic clearing of cheques. The accuracy or robustness of existed techniques often falls short of requirements in practice. It is the fact that no reliable verification systems exist to help financial institutions fight with the fastest-growing crime of cheque fraud and counterfeiting. Many challenges are imperative to meet in order to develop a system that automatically authenticates documents based on the owner’s handwritten signature. Such a system should have an acceptable trade-off between a low FAR and a low FRR, where the FRR is the ratio of the number of genuine test signatures rejected to the total number of genuine test signatures submitted and the FAR is the ratio of the number of forgeries accepted to the total number of forgeries submitted. With the state-of-the-art techniques of offline signature verification and identification summarised above, the works presented in this paper feature two propositions. The first is the further exploration of distinctive and efficient features, particularly by joint exploitation of different attributes which often characterises
Offline signature verification and identification
305
different aspect of image features. The first feature is based on the recently proposed local edge feature extraction method, called the Histogram of Oriented Gradients (HOG), which was proposed by Dalal and Triggs (2005), which have reported successful applications to a number of different object. Another feature, DF, was proposed in Blumenstein et al. (2003) and has been proven efficient for handwritten-character recognition. For offline signature images, PHOG provides a principled way of expressing the intrinsic information and a simple combination of PHOG and DF produce more comprehensive features which may include global, structural, geometric or statistical information. With the efficient feature description strategies, signature verification can be defined in the general classification framework, which is the second proposition addressed in this paper. Specifically, verification and recognition can be solved by binary and multi-class classification methods, respectively. Among a bunch of “off-the-shelf” classifiers, we comparatively evaluated several of the most common ones, including the kNN, FLD, MLP, SVM and RF. Using a benchmarking signature database GPDS (Martinez et al., 2004), experimental results showed superb performance from SVM and the combined features, thus substantiating the effectiveness of the proposed approach.
2 Feature extraction Many types of features can be defined to extract immanent information hidden in an image using a variety of different techniques. For instance, the colour and grey-level histograms, Gabor filters, and different transforms. For handwritten signature images, geometrical features based on shape are generally accepted as more important. These commonly include descriptions such as baseline, slant angle, aspect ratio, normalised area, centre of gravity, and slope of the line joining the Centres of Gravity of the two halves of signature image (Qi and Hunt, 1994; Baltzakis and Papamarkos, 2001). Another example of geometrical feature are the exterior contours (Chen and Srihari, 2005), which may extract information including each pixel’s position, slope, and curvature et al. Though successful under some circumstances, these feature extraction methods are not robust, and many pre-processing or post-processing such as Dynamic Time Warping (DTW) for the contour sequences alignment is required. In this section, two novel feature extraction methods will be briefly introduced.
2.1 Pyramid histogram of oriented gradients For images containing rich shape information, many types of shape and perceptual grouping features have been proposed. The local geometrical shape within an image can be characterised by the distribution of edge directions, called Histograms of Oriented Gradients (HOG) (Dalal and Triggs, 2005). HOG can be calculated by evaluating a dense grid of well-normalised local histograms of image gradient orientations over the image windows. Specifically, an image is first divided into non-overlapping pixel regions, or cells. For each cell a 1D histogram of gradient orientations over pixels in that cell is accumulated. The gradient at each pixel is discretised into one of nine orientation bins, and each pixel ‘votes’ for the
306
B. Zhang
orientation of its gradient, with a strength that depends on the gradient magnitude. Finally, the histogram of each cell is normalised with respect to the gradient energy in a neighbourhood around it. HOG has some important advantages over other local shape descriptors, for example, it is invariant to small deformations and robust in terms of outliers and noise. The HOG feature encodes the gradient orientation of one image patch without considering where this orientation originates from in this patch. Therefore, it is not discriminative enough when the spatial property of the underlying structure of the image patch is important. The objective of a newly proposed improved descriptor PHOG (Bosch et al., 2007) is to take the spatial property of the local shape into account while representing an image by HOG. The spatial information is represented by tiling the image into regions at multiple resolutions, based on spatial pyramid matching (Lazebnik et al., 2006). Each image is divided into a sequence of increasingly finer spatial grids by repeatedly doubling the number of divisions in each axis direction. The number of points in each grid cell is then recorded. The number of points in a cell at one level is simply the sum over those contained in the four cells it is divided into at the next level, thus forming a pyramid representation. The cell counts at each level of resolution are the bin counts for the histogram representing that level. The soft correspondence between the two point sets can then be computed as a weighted sum over the histogram intersections at each level. More specifically, for each grid cell at each pyramid resolution level, a HOG vector is computed. The final PHOG descriptor for the image is then a concatenation of all the HOG vectors. In the implementation we follow the practice in Bosch et al. (2007) by limiting the number of levels to L = 3 to prevent over-fitting. The PHOG is normalised to sum to unity. The PHOG descriptors can be compared using the same histogram intersection method as the pyramid match (Lazebnik et al., 2006), or by other means such as the χ2 distance between the histograms at each level. PHOG descriptor has been applied to solve a number of practical tasks such as object classification (Bosch et al., 2007) and image retrieval (Lee, 2008).
2.2 Direction Feature The DF proposed in Blumenstein et al. (2003) is based on local and global geometric features of image (character) skeleton. It extracts different types of line segment that form a particular character. The detailed description is referred to Blumenstein et al. (2003, 2004). After some simple preprocessing such as binarisation, noise removal and skeletonisation, DF mainly consists of the following two steps. Step 1: Direction determination The DF aims at simplifying a binary image’s boundary through finding out individual stroke or line segments in the image. It extracts different line segments in a sub-region of an image by traversing the entire skeleton in that sub-region. To achieve this, boundaries were traced from a given starting point to known intersection points. Roughly, a pixel is defined to be an intersection if it has more than one neighbour. Once at an intersection, a clockwise investigation was conducted to determine the beginning and end of prospective line segments based on a number of rules (Blumenstein et al., 2003). Then all foreground (black) pixels that comprise individual line segments are replaced with a set of ‘direction values’,
Offline signature verification and identification
307
which can be categorised into four types depending on the calculated direction of individual pixels: •
vertical
•
horizontal
•
right diagonal
•
left diagonal.
The number of any particular line type is normalised in order to generalise the direction of individual line segments. This is accomplished by calculating the most frequently occurring direction value in a given line segment (Blumenstein et al., 2003, 2004). Step 2: Formation of feature vectors: Once line segments were determined, a procedure will be carried out to create appropriate feature vectors. For each of the defined windows of equal size, direction information is extracted and expressed as the line segment direction, length, intersection points, etc. in floating point values between −1 and 1.
3 Signature verification and identification by classification In principle, the problem of signature verification is a pattern recognition task involving the discrimination of two classes, original and forgery signatures. On the other hand, signature identification is a typical multi-class classification issue. With feature vectors estimated, a binary classification can be invoked for verification (genuine or forgery), or a multi-class classification resorted for identification. In the following we first give a brief review of the SVM as it is the most commonly used classifier and the excellent performance demonstrated in the next section. Some other classification techniques will also be briefly introduced for comparison purposes.
3.1 Brief review of SVM For binary pattern classification, SVM tries to find the optimal separating hyperplane that separates the positive and negative examples with an maximal margin. Usually, the classification decision function for a linearly separable problem can be represented by f = sign(w · x + b)
(1)
which determines the classification decision function by minimising the empirical risk: 1 |f (xi ) − yi | , l i=1 l
R=
where l represent the size of examples.
(2)
308
B. Zhang
The optimal separating hyperplane is determined by giving the largest margin of separation between different classes, following structural risk minimisation principle (Shawe-Taylor and Cristianini, 2004). The following constrained minimisation then needs to be satisfied, 1 T w w, 2 Subject to: yi (w · xi + b) ≥ 1, i = 1, . . . , l.
Minimise:
(3)
The obtained SVM classifier involves the inner product xTi x, where i ⊂ S, the set of support vectors. However, it is not necessary to use the explicit input data to form the classifier. Instead, all that is needed is to use this inner products between the support vectors and vectors of the feature space. That is, by defining the kernel k(xi , x) = xTi x
(4)
a nonlinear classifier can then be formulated as f (x) = sign {α0 yi K(xi , x) + b0 } . The commonly applied Gaussian kernel is 2 x − y k(x, y) = exp − 2σ 2
(5)
(6)
where the parameter σ controls the support region of the kernel.
3.2 Multi-class Support Vector Machine The aforementioned SVMs were primarily designed for binary classification problems. Based on the relationship between binary classification and multi-classification, many methods have been proposed in the literature for solving multi-class problems (Tax and Duin, 2002). Binary SVM can be extended to multi-class SVM based on the schemes: one-against-one, one-against-all, and Directed Acyclic Graph Support Vector Machine (DAGSVM) In Hsu and Lin (2002), Franc and Hlavac (2002), a comparison of these methods was thoroughly analysed. For most applications, the one-against-one approach is more computationally intensive since the results of more SVM pairs ought to be computed. From this consideration, we will apply the one-against-all approach. In the one-against-all approach, a SVM is constructed for each class by discriminating that class against the remaining k − 1) classes. The number of SVMs used is k. A test pattern x is classified by using the winner-takes-all decision strategy, i.e., the class with the maximum value of the discriminant function f (x) is assigned to it.
3.3 Some conventional classification methods compared In machine learning, there are many well-known classification methods. For example, the kNN classifier, the Logistic Regression, the FLD, and the artificial neural networks (e.g., Multi Layer Perceptrons). In the following we briefly summarise a few of the most commonly used methods for comparison purposes. Detailed descriptions can be found in Duda et al. (2001).
Offline signature verification and identification
309
3.3.1 k -Nearest Neighbour classifier k-Nearest Neighbour (kNN) classifier is a prototype-based classifier among the oldest types of classification methods. It is based on a distance function, such as the Euclidean distance, for comparing pairs of data samples. The kNN classifies a test sample by first finding the k closest samples in the training set, and then predicting the class by majority voting. In other words, the class that is most common among those k neighbours is chosen as the predicted label. Obviously the kNN classifier needs to access all the training data every time when a new test case is to be classified.
3.3.2 Random Forest Decision trees are among the best known classification techniques (Duda et al., 2001). But the associated problems of over-fitting and poor generalisations have inspired many efforts for improvements. A RF (Breiman, 2001) is an ensemble classifier that consists of many decision trees and outputs the class that is the mode of the classes output by individual trees. It mitigates decision tree problems by injecting randomness into the training of the trees (i.e., the random selection of features), and combining the output of multiple randomised trees into a single classifier. Random Forests have been used for many classification and regression tasks, often demonstrating lower test errors than conventional decision trees and performance comparable to SVMs in some problems. RF is highly computationally efficient. The algorithm begins with the selection of many (e.g., 500) bootstrap samples from the data. A typical RF consists of a set of binary decision trees. During training, a classification tree is fit to each bootstrap sample, but at each node, only a small number of randomly selected variables (e.g., the square root of the number of variables) are available for the binary partitioning. Each non-leaf node in each tree is assigned a binary test that is applicable to any data sample.
3.3.3 Fisher Linear Discriminant In FLD, each class is modelled by a multivariate Gaussian, where each class is assumed to have the same covariance matrix. Specifically FLD finds Gaussian distributions of the data by maximum likelihood estimations for several parameters: the prior probability of class k; the mean of class k; and the common covariance matrix. Observations can be classified to the class of the nearest mean vector according to Mahalanobis distance, or by a likelihood ratio test which reduces to a particularly simple form: x · w > t, where t is a decision threshold, w is a parameter vector computed from the Gaussian parameters.
3.3.4 Multi Layer Perceptron The MLP is a common type of neural network classifier, which is often trained by the error back-propagation algorithm. MLP usually consists of a layer of input nodes, each linked by weighted connections to every one of a layer of hidden nodes, each of which is linked, in turn, to a set of output nodes. It has been shown that MLPs can virtually approximate any function with any desired accuracy,
310
B. Zhang
provided that enough hidden units and enough data are given. Therefore, it can also implement a discrimination function that separates input data into classes. Such an ability of an MLP to learn from data makes it a practical classifier for many classification tasks. Though successful for many applications, however, MLP classifier has several limitations and training an MLP network involves a considerable degree of empiricism. And the performance often depends on the nature and quality of the data on which it is trained. For example, the classification accuracies may be sensitive for different class frequencies in the training set.
4 Experiments Experiments have been conducted to verify the proposed algorithm using a publically available offline signature data set, gpdsSIGNATURE, which can be accessed from the internet http://www.gpds.ulpgc.es/download/index.htm. The dataset contains signature images from 160 individuals: 24 genuine signatures for each individual, plus 30 forgeries of his or her signature. The 24 genuine specimens of each signer were collected in a single day writing sessions. The forgeries were produced from the static image of the genuine signature. Each forger was allowed to practice the signature for as long as she or he wishes. Each forger imitated 3 signatures of 5 signers in a single day writing session. The genuine signature shown to each forger is chosen randomly from the 24 genuine ones. Therefore for each genuine signature there are 30 skilled forgeries made by 10 forgers from 10 different genuine specimens. Figure 1 illustrates one class example. Figure 1 The gpds dataset samples: left column: genuine signature samples; right column: forgeries samples
As described in Section 2, the PHOG feature extraction method is applied to all the signature images, with three levels of pyramids and 8 bins in each level. In forming the pyramid, the grid at level l has 2l cells along each dimension. Consequently, level 0 is represented by a K-vector corresponding to the K bins of the histogram, level 1 by a 4K-vector etc. Therefore, 2 the four level PHOG descriptor of an image is a vector with dimensionality 8 l=0 3l = 168. In all of the following experiments, each one of those signatures had been first normalised using a bilinear interpolation algorithm to a rectangle measuring 50 × 25 pixels. And the images were converted to binary using the Otsu’s thresholding algorithm. The size of the normalised signatures
Offline signature verification and identification
311
was obtained as the average value of the size of the signatures. The DF feature extraction method yields 85 dimensional vector for each of the signature image. A signature verification system decides whether a claim that a particular signature belongs to a specific class (writer) is true or false. A signature recognition system, on the other hand, has to decide to which of a certain number of classes (writers) a particular signature belongs. Details of experiments for verification and identification are elaborated below.
4.1 Verification In the verification scenario, a signature image with corresponding claimed identity is presented to a verification system, which need to make the decision of acceptance or rejection for the claim. From pattern classification point of view, the objective of signature verification system is to discriminate between two classes: the original and the forgery. There are several different scenarios. •
Correct acceptance. The system was asked if the signature belonged to the correct owner and the response was positive.
•
False rejection. The system was asked if the signature belonged to the correct owner and the response was negative.
•
False acceptance. The system was asked if the signature belonged to a false owner and the response was positive.
•
Correct rejection. The system was asked if the signature belonged to a false owner and the response was negative.
Accordingly, the performance of an offline verification system is often measured by the FAR and FRR, defined as: number of FAs number of impostor signatures number of FRs FRR = . number of true signatures
FAR =
(7) (8)
For many applications of signature verification, particularly forensic and financial applications, the goal is often to get an acceptable FRR at a FAR level as low as possible (best near 0). In our study, two basic forgeries have been taken into account: random forgery and skilled forgery. To perform verification, a classifier is trained for each subject by first forming positive and negative signatures. On a query signature, the trained classifier classifies it in terms of authenticity, as either genuine or non genuine. A first set experiments were conducted for random forgeries. To detect only random forgeries, subsets of other writers’ training sets can be used as negative samples. Specifically, the positive training set consisted of 80% signatures (19 samples) of each subject while the negative training set (19 samples) are from random sampling of other subjects’ signatures. The positive testing set consisted of the remaining 5 signatures of each subject and 5 negative testing set is randomly collected from other subjects signature without overlapping with the negative
312
B. Zhang
training set. Thus for each subject we had 24 genuine and 24 random forged signature samples, and the training and testing sets were disjoint. The trained classifier then discriminates the query signature in one of these two classes to establish its authenticity. The experiment settings for all the classifiers are summarised as follows. For MLP, we experimented with a three-layer network with following structure. The number of inputs is the same as the number of features (i.e., 168 for PHOG histogram, 85 for DF), one hidden layer with 50 units and a single output unit with logistic function representing the class label. When the input data belongs to a genuine signature, the output (target of the MLP) is fixed to 1. When the input is a forgery, the output is fixed to –1. It is our experience that varying the number of hidden units in such an MLP usually does not significantly change the performance. The MLP was trained during 800 iterations using a learning rate equal to 0.01. The SVM applied is based on the Intersection Kernel Support Vector Machines (Maji et al., 2008). For kNN classifier, we chosen k = 1 after testing a range of different values of k with the similar results. Our experiment is based on the holdout approach, which repeat 100 times for each subject by the sampling procedure as described above (20% of data reserved for testing and the rest for training in each repeating). The five different classifiers are compared simultaneously. To evaluate the performance, we recorded the FRR or Error Type I, i.e., the percentage of rejection of genuine signatures, and FAR or Error Type II, i.e., the percentage of acceptance of forgeries. The error rate considered in our experiments is the average of FRR and FAR, and the mean error rate is the average of the error rates associated with existing writers. Another often cited performance verification accuracy can be defined as Accuracy = 1 −
F AR + F RR . 2
(9)
The verification results with random forgeries are summarised in Table 1. In the table, Comb. stands for the hybrid feature of combinaiton of PHOG and DF. Several observations can be found from the above experiments: •
the PHOG feature is much better than DF
•
SVM is the best among the classifiers compared
•
combination of PHOG and DF improves the performances of kNN, FLD and SVM in regard to FRR and FAR.
The best performance of FRR 0.4% and FAR 0.9% was obtained from SVM with the combined feature. Further details of the experiment results are illustrated in Figure 2(a), where the accuracies are compared by bar plots. In practice, in addition to FPR and FAR, another common used binary classification performance is the ROC curve as a more robust evaluation of the relative classification performance of alternative models. The ROC curve of a classifier is obtained by varying a threshold on the continuous output of the classifier, ranging from 0 to 1. Samples whose their outputs are greater than the threshold are classified as positives, otherwise they are classified as negatives.
Offline signature verification and identification Table 1
Verification performances with random forgeries
313
314
B. Zhang
Figure 2 Signature verification accuracies from the four different classifiers with features of PHOG, DF and their combination. 80% of data were used for training while the remaining for testing. (a) barplot and (b) the ROC curves (see online version for colours)
Given a value of the threshold, a point of (F P R, T P R) on the ROC space is determined by number of negatives incorrectly classified total negatives number of positives correctly classified . TPR = total positives
FPR =
(10) (11)
Obviously, the general definition of False Positive Rate (FPR) is equal to the FAR and the True Positive Rate (TPR) is equal to 1 – FRR. The ROC curve for the compared classifiers with random forgeries is given in Figure 2(b), which shows again that SVM is the best, followed by FLD, whisi also offers the satisfactory performances. Experiments have also conducted for verification with skilled forgeries. In the GPDS dataset we used the signatures from 39 subjects. Each subject has 24 genuine signatures and 30 forgeries. For each subject, 80% genuine signatures (19 samples) and 80% forgeries (24 samples) are picked out randomly as the training examples, the other 5 genuine signatures and 6 forgeries are used as testing examples. The experiments were repeated 100 times for each subject. The verification performances are given in Table 2. From Table 2 we can find again that PHOG feature is much better than the DF regarding FRR, FAR and Accuracy, and SVM is also the best. However, the combination of PHOG and DF brings performance improvement only for the SVM, with FRR 2.5%, FAR 2% and corresponding accuracy 97.8%. Figure 3 demonstrated further comparisons of the verification accuracies and ROC curves from the different classifiers and features. In Table 3, we present a comparison of the above results with some recently published works which also use the same GPDS signature data. It shows that SVM classifier with hybrid feature of PHOG and DF outperforms all the existing approaches. It is worthy to mention that the comparison only provides a rough
Offline signature verification and identification Table 2
Verification performances for skilled forgery
315
316
B. Zhang
Figure 3 (a) Barplots of the verification accuracies from the four different classifiers. The results were from the average of 100 tests. The SVM used is based on Intersection Kernel Support Vector Machines (IKSVM). (b) The corresponding ROC curves (see online version for colours)
Table 3 Comparison of the proposed method with recently published results from using the GPDS data Armand Ruiz-del-Solar Vargas Larkins Proposed
FRR (%) 10.36 16.4 10.01 10.79 2.5
FAR (%) 2.88 14.2 14.66 17.24 2
Accuracy (%) 93.59 84.7 87.67 85.99s 97.8
meaning because the experiments details from different publications are varied even for the same data sets.
4.2 Recognition Being different with signature verification, signature recognition is the decision about the identity of the owner of a signature from registered signature database, which can be conveniently described as a multi-class classification problem. We conducted recognition experiments in a similar way described above for verification, with PHOG and DF features extracted for all the signature images, resulting PHOG descriptor of an image with length 168 and Df descriptor of length 85. Being different with verification where a binary classifier is allocated to each subject for discriminating genuine or forgery signatures, an overall classifier is created for all the signatures without forgeries being taken into account. The training features accompany identity labels. We compared four classification methods, i.e., kNN, MLP, RF and SVM. The MLP is a 3-layer similar structure as used in verification except that there are now 160 output units corresponding to 160 subjects. During training, the unit corresponding to the desired class label is clamped at 1.0 while all of other units clamped to 0.0. During classification, an integer is assigned to the class from the output unit with the highest activation value. This is called the one-per-class approach.
Offline signature verification and identification
317
There are many standard procedures to test the performance of a general pattern classification scheme. The commonly used ones include k-fold cross-validation and leave-one-out methods. The k-fold cross-validation is an established technique for estimating the accuracy of a classifier. In general, all of the examples are partitioned into k subsamples, of which the kth subsamples is retained for testing the model while the remaining k − 1 subsamples are used as training data. Cross-validation is then repeated k times, with all of the k subsamples used exactly once as the validation data. The cross-validation estimation of accuracy is the overall number of correct classifications, divided by the number of instances in the data set. A 10-fold cross-validation experiment is proceeded for the four classifiers. For each division of the samples, a classifier is developed with 3456 training samples and tested in the rest 384 samples. This process of randomly generated partitions is repeated 10 times, with randomly chosen training and testing sets giving up 100 unbiased estimates of discriminant ability. As the test samples are independent of the training data, the results derived from this 10-fold cross-validation are reliable. The experiment results of classification accuracies as shown in Figure 4 demonstrated that, with the same PHOG and DF attributes, the hybrid feature of simple concatenation of them offers performance enhancement for all of the classifiers, and again SVM is far more superior, achieving classification accuracy of 99%. Figure 4 Barplots (a) and Boxplots (b) of the signature classification accuracies from the four different classifiers. The results were from the average of 10-fold cross-validation tests. In the boxplots, each classifier was illustrated with three different features (from left to right), i.e., PHOG, DF and the their combination (see online version for colours)
5 Conclusion As signatures are still the most accepted form of identity verification and the serious problems associated with forgery, automatic verification of offline signature remains highly desirable and challenging after decades of research efforts. In this paper, two different feature description methods, namely the Pyramid Histogram of Oriented Gradient and Direction Feature, were comparatively studied for signature verification, showing that PHOG is a much better method than DF and a simple
318
B. Zhang
combination of them can offer performance improvements. Binary classification was suggested as a principled solution of signature verification by designing classifiers for discriminating genuine and forgery signatures and SVM was proved to be the best among classifiers compared. Using a benchmarking signature database GPDS, experimental results have shown very promising performances. For skilled forgery, SVM classifier can give a FRR of 2.5% and a FAR 2%, which is considerably better than the majority of results that have been previously published. By using the same PHOG and DF shape features, a closely related issue of offline signature recognition was also discussed. A very satisfactory classification accuracy of 99% is obtained from SVM, showing promising potentials of the system in practical applications.
References Armand, S., Blumenstein, M. and Muthukkumarasamy, V. (2006) ‘Offline signature verification using the enhanced modified direction feature and neural-based classification’, Proceedings of International Joint Conference on Neural Networks, Vancouver, pp.684–691. Baltzakis, H. and Papamarkos, N. (2001) ‘A new signature verification technique based on a two-stage neural network classifier’, Engineering Applications of Artificial Intelligence, Vol. 14, pp.95–103. Bosch, A., Zisserman, A. and Munoz, X. (2007) ‘Representing shape with a spatial pyramid kernel’, 6th ACM international Conference on Image and Video Retrieval, Amsterdam, The Netherlands, pp.401–408. Blumenstein, M., Verma, B. and Basli, H. (2003) ‘A novel feature extraction technique for the recognition of segmented handwritten characters, 7th International Conference on Document Analysis and Recognition (ICDAR’03), Eddinburgh, Scotland, pp.137–141. Blumenstein, M., Liu, X.Y. and Verma, B. (2004) ‘A modified direction feature for cursive character recognition’, 2004 IEEE International Joint Conference on Neural Networks, Budapest, Hungary, Vol. 4, pp.2983–2987. Breiman, L. (2001) ‘Random Forests’, Machine Learning, Vol. 45, pp.5–32. Chen, S. and Srihari, S.N. (2005) ‘Use of exterior contours and word shape in off-line signature verification’, Proceedings International Conference on Document Analysis and Recognition, Korea, pp.1280–1284. Dalal, N. and Triggs, B. (2005) ‘Histograms of oriented gradients for human detection’, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), San Diego, CA, pp.886–893. Deng, P.S., Liao, H.Y.M., Ho, C-W. and Tyan, H-R. (1999) ‘Waveletbased off-line handwritten signature verification’, Computer Vision and Image Understanding, Vol. 76, pp.173–190. Duda, R.O., Hart, P.E. and Stork, D.G. (2001) Pattern Classification, 2nd ed., John Wiley and Sons, New York. Fang, B., Wang, Y.Y., Leung, C.H., Tse, K.W., Tang, Y.Y., Kwok, P.C.K. and Wong, Y.K. (2001) ‘Offline signature verification by the analysis of cursive strokes’, International Journal of Pattern Recognition and Artificial Intelligence, Vol. 15, pp.659–673. Fang, B., Leung, C., Tang, Y., Tse, K., Kwok, P. and Wong, Y. (2003) ‘Off-line signature verification by the tracking of feature and stroke positions’, Pattern Recognition, Vol. 36, pp.91–101.
Offline signature verification and identification
319
Ferrer, M., Alonso J. and Travieso, C. (2005) ‘Offline geometric parameters for automatic signature verification using fixed-point arithmetic’, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 27, pp.993–997. Franc, V. and Hlavac, V. (2002) ‘Multi-class support vector machine’, Proceedings 16th International Conference on Pattern Recognition, Quebec City, Canada, pp.236–239. Frias-Martinez, E., Sanchez, A. and Velez, J. (2006) ‘Support vector machine versus multi-layer perceptrons for efficient off-line signature recognition’, Engineering Applications of Artificial Intelligence, Vol. 19, pp.693–704. Guo, J., Doermann, D. and Rosenfeld, A. (2001) ‘Forgery detection by local correspondence’, International Journal of Pattern Recognition and Artificial Intelligence, Vol. 15, pp.579–641. Hsu, C-W. and Lin, C-J. (2002) ‘A comparison on methods for multi-class support vector machines’, IEEE Transactions on Neural Networks, Vol. 13, pp.415–425. Justino, E.J.R., Bortolozzi, F. and Sabourin, R. (2001) ‘Off-line signature verification using HMM for random, simple and skilled forgeries’, Proceedings of the Sixth International Conference on Document Analysis and Recognition, Seattle, pp.1031–1034. Justino, E.J.R., Bortolozzi, F. and Sabourin, R. (2004) ‘A comparison of SVM and HMM classifiers in the off-line signature verification’, Pattern Recognition Letters, Vol. 26, pp.1377–1385. Kalera, M.K., Srihariy, S. and Xu, A. (2004) ‘Offline signature verification and identification using distances statistics’, International Journal of Pattern Recognition and Artificial Intelligence, Vol. 18, pp.1339–1360. Martinez, L.E., Travieso, C.M., Alonso, J.B. and Ferrer, M. (2004) ‘Parametrization of a forgery handwritten signature verification using SVM’, Proceedings of IEEE 38th Annual International Carnahan Conference on Security Technology, pp.193–196. Larkins, R. and Mayo, M. (2008) ‘Adaptive feature thresholding for offline signature verification’, Proceedings of 23rd International Conference Image and Vision Computing, New Zealand. Lazebnik, S., Schmid, C. and Ponce, J. (2006) ‘Beyond bags of features: spatial pyramid matching for recognizing natural scene categories’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, pp.2169–2178. Lee, J. (2008) Efficient Object Recognition and Image Retrieval for Large-Scale Applications, Technical Report, MIT-CSAIL-TR-2008-026. Maji, S., Berg, A.C. and Malik, J. (2008) ‘Classification using intersection Kernel support vector machines is efficient’, Proceedings, CVPR 2008, Anchorage, Alaska. Plamondon, R. and Lorette, G. (1989) ‘Automatic signature verification and writer identification – the state of the art’, Pattern Recognition, Vol. 22, pp.107–131. Qi, Y. and Hunt, B. (1994) Signature verification using global and grid features’, Pattern Recognition, Vol. 27, pp.1621–1629. Ruiz-Del-Solar, J., Devia, C., Loncomilla, P. and Concha F. (2008) ‘Offline signature verification using local interest points and descriptors’, Proceedings of the 13th Iberoamerican congress on Pattern Recognition: Progress in Pattern Recognition, Image Analysis and Applications, Havana, Cuba, pp.22–29. Sabourin, R., Genest, G. and Preteux, F.J. (1997) ‘Offline signature verification by local granulometric size distributions’, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, pp.976–988. Shawe-Taylor, J. and Cristianini, N. (2004) Kernel Methods for Pattern Analysis, Cambridge University Press, UK.
320
B. Zhang
Tax, D.M.J. and Duin, R.P.W. (2002) ‘Using two-class classifiers for multiclass classification’, Proceedings of International Conference on Pattern Recognition, Quebec, Canada. Vargas, J., Ferrer, M., Travieso, C. and Alonso, J. (2008) ‘Offline signature verification based on high pressure polar distribution’, Proceedings of the 11th International Conferences on Frontiers in Handwriting Recognition, Concordia. Zhang, B. (2006) ‘Offline signature recognition and verification by Kernel principal component self-regression’, Proceedings of the 5th International Conference on Machine Learning and Applications (ICMLA’06), Orlando, Fla, pp.28–33.
