One-Class versus Bi-Class SVM Classifier for Off-line Signature Verification Yasmine Guerbai, Youcef Chibani and Nassim Abbas Speech Communication and Signal Processing Laboratory, Faculty ofElectronics and Computer Science University of Science and Technology Houari Boumediene (USTHB),
32,
EI Alia, Bab Ezzouar,
16111, Algiers,
Algeria,
[email protected],
[email protected],
[email protected]
Abstract-Support vector machines (SVMs) have
Generally, an off-line HSVS
become an
alternative tool for pattern recognitions, and more specifically for
acqUisItIOn
and
Handwritten Signature Verification Systems (HSVS). Usually, the
classification
[5].
bi-class SVMs (B-SVM) are used for separating between genuine
developed for the
and
forged
signatures.
However,
in
practice,
only
genuine
signatures are available. In this paper, we investigate the use of one-class SVM (OC-SVM) for handwritten signature verifications.
class SVM.
I.
generation
classification stage,
and
such as Template
Hidden Markov Models) and the Structural Model More
recently,
models
based
on
the
[5].
Support
Vector
Machines have been found to be well suited for signature modeling,
Keywords- signature verification, one class support vector machine, hi-class support vector machine, uniform grid
feature
In the past decade, many methods have been
Matching, the Statistical Models (Measures, Neural Networks,
Experimental results conducted on the standard CEDAR database show the effective use of the one-class SVM compared to the bi
is composed of three stages:
preprocessing,
as Offline
Signature Verification Using Virtual
Support Vector Machines
[6].
Usually, the bi-class SVMs (B
SVM) are designed for separating between genuine and forged signatures. However, in practice, only genuine signatures are
INTRODUCTION
available for the verification
[7].
Hence, we propose in this
Biometrics is the set of technological means that enables the
paper to investigate the use of the One-Class SVM (OC-SVM)
identification or verification of an individual from its physical
for the HSVS. The main task of the OC-SVM is to distinguish
or
genuine signatures from other forged signatures. The choice of
behavioral
Physiological
characteristics characteristics
depending are
on
related
their to
nature.
anatomical
the OC-SVMs is that they have been successfully used for
properties of a person, including, for instance, fmgerprint, face,
many applications as biometric verification
iris and hand geometry. Behavioral characteristics refer to how
[9],
an individual performs an action, including, for instance, voice, gait and signature
[1].
for verifying a document
[2].
However, they are visually
difficult to read since they contain special characters and flourishes
[2].
[8], image retrieval [lO] and document classification [11].
The remainder of this paper is organized as follows. Section
2
Socially, the handwritten signatures are the most accepted
speaker diarization
briefly reviews the basics of one-class SVM.
presents the system description. Section
4
Section
3
reports all the
experiments performed on CEDAR database. The last section concludes the paper.
Moreover, signatures can significantly vary for II.
the same person. The main drawback of the handwritten signature is the easiest reproduction by the professional forgers. Although, the human experts can discriminate between genuine and forged signatures, various verification systems are being developed in order to modeling the invariance of the signatures and thus automating signature recognition process
[3].
SVM OVERVIEW
In this section, we briefly describe the standard support vector proposed by Vapnik
[12]
namely B-SVM and the
modifications introduced by SchOlkopf
[13]
to build the OC
SVM. A.
Bi-class Support Vector Machine The SVM is a learning method, which tries to fmd an
Usually, two acquisition modes are used for capturing the
optimal hyper plane for separating two classes. SVMs have
mode,
been defined for separating linearly two classes. When data are
respectively. The off-line mode allows generating a handwriting
non linearly separable, a kernel function is used as polynomial
static image from a scanning document. In contrast, the on-line
function, radial basis function
mode allows generating from pen tablets or digitizers dynamic
in order to construct a linear decision function in the feature
signature,
which
are
off-line
mode
information such as velocity and pressure
and
[4].
on-line
(RBF)
or multi layer perceptron
space so that the dataset becomes separable with maximum margin
[12].
978-1-4673-1520-3/12/$31.00 ©2012
IEEE
Let datasets training set,
m
(XvYl),.. (X m,Ym), Xi E Rd, Yi E [1,-1] be is the nwnber of training data and d is the size
of the feature vector. Then, the decision function is expressed in terms of kernel expansion as:
Outliers
Training data
Support vectors
foe> 0
Support vectors
m f(x) ai [ 0, C ] .
where range
=
I aiYiK(X,Xi)
(1)
+ b
i=l
K(x ,Xi)
are Lagrange multipliers having values in the C is a user-defmed parameter that controls the
tradeoff between the machine complexity and the number of
Projection in featLLre space
non separable points [14]. The bias b is a scalar computed by using
any
support
vector.
Thus,
test
data
are
classified
Original space
according the decision rule:
X
E
{class (
if f(x) > 0 otherwise
+ 1)
class(-1)
A pattern
novelty
detection)
is
a
type
of
x is
then accepted when
foc(x)
>
O.
Otherwise, it
is rejected. Various kernel functions can be used as polynomial, Radial Basis Function or multilayer perceptron [12]. Usually,
One-Class SVM (also known as single-class classification or
= 0
Feature space
Figure 1. Data classification based on OC-SVM.
(2)
One-class Support Vector Machine
B.
foe
Origin
unsupervised
learning
algorithm developed by SchOlkopf [13]. One-class classification
the RBF is the most used kernel, which allows determining the radius of the hyper sphere according the parameter y. It is defined by:
allows classifying just one-class objects, and distinguishing it
K(x,Xi)
from all other possible objects. Objects can be classified well by
=
exp(-yli x - xdl2)
(7)
the classifier, but the others will be classified as outliers [7]. III.
The concept of the OC-SVM consists to find an hyper sphere in which the most of learning data are included into a minimwn volume. More specifically, the objective of the OC
foc(x) that encloses the most of sphere Rx {x E Rd \foc(x) > O}
SVM is to estimate a function learning data into a hyper
=
a minimum volume where d is [15]. foc(x) is the decision function,
with
The verification system shown in figure 2 is composed of three
stages:
pre-processing,
feature
generation
and
classification. In the following, we give a description of each stage composed our system.
the size of feature vector which
SYSTEM DESCRIPTION
Off-line handwritten
takes the form as
[4]:
(3)
m
is the nwnber of training data and
ai
Pre-processing of signature
are the Lagrange
mUltipliers computed by optimizing the following equations:
(4) Subject to
(5)
Evaluation
m
I ai
=
1
i
is the percentage of data considered as outliers.
Figure 2. Structure ofthe verification system
(6)
P defines the distance of the hyper sphere from the origin. v
J
'----
K(. ,. ) defines
the OC-SVM kernel that allows projecting data from
the
original space to the feature space. Figure 1 shows an example of using OC-SVM to illustrate the separation of data [8].
A.
Preprocessing of signatures The main preprocessing of the acquired signature is
the
binarization of the image by using the local iterative threshold [16]. It is carried out through several steps for each sample area found in a 9 x 9 square centered on the pixel at
(x,y).
This
a(x,y)
m(x,y)
and the standard
allows taking into account only genuine signatures and fInally
of each pixel values in the sample area. By
the ROC curve (Receiver Operating Characteristic). The ROC
process starts by fIrst fInding the mean deviation
doing this, any pixel that could potentially be noise is adjusted. B.
is a visual characterization of the trade-off between the FAR and the FRR.
Feature generation
In our approach, we also consider the Half Total Error Rate
Features are generated from the signature image by using
(HTER) defined as [18]:
the uniform grid [16], which consists to create rectangular regions having the same size and shape. Hence, the feature
HTER
vector is generated by considering the set of regions each one is
FAR =
+ 2
FRR
computed as the ratio between the black pixels and the total number of pixels. Component values of the feature vector are
The HTER constitutes a good criterion for evaluating the
ranged from zero to one.
accuracy of a method. Hence, a method can be considered
C.
accurate when the HTER is lower as much as possible.
Classification The OC-SVM is designed to separate a class from others
classes. Theoretically, a pattern the decision function
foc(x)
x
B.
is positive. Implicitly, the threshold
Model Generation The diffIculty of OC-SVM and B-SVM is setting of its
is correctly classified when
respective parameters
(v, y)
and
(e, a)
to achieve the lower
is fIxed to zero. This approach can be considered as a hard
FAR and FRR. Both SVM classifIcations involve training,
thresholding. In order to relax this constraint, we propose a soft
validation and testing steps. The training and validation stages
thresholding in order to reduce the misclassifIcation. Therefore,
consist to fInd the optimal parameters.
we adopt the following decision rule:
x t
if foc(x) � t otherwise
(Accepted {Rejected
E
For each SVM classifIcation, three parameters should be
(8)
size
of
the
uniform
grid.
These
parameters
are
found
experimentally depending on the dataset. The fInal step is
defInes the threshold computed according the following
equation:
testing which allows evaluating the robustness of the classifIer. C.
(9)
Comparative Analysis When comparing the OC-SVM versus B-SVM, we are faced
to select the training, validation and testing dataset. Hence, we adopt the cross-validation for evaluating the performances of
such that M
mi at
=
� L foc(xj) j=l
N
=
2.. � a· . aNL
t
,
t
where
N is
To obtain the error rate curves for each classifIer, the threshold for the B-SVM is fIxed in the range [0,1] [19], while, with k is a parameter included in the range [-3,3]. Figures 3 and 4 show the mean ROC curves for both SVM
number of writers, M is number of signatures,
mi and ai are the mean and standard deviation, respectively,
which are computed from all decision functions during the learning phase. The parameter k allows controlling the decision threshold. IV.
the SVM classifications. In our experiments, we use 3-fold cross-validation.
for the OC-SVM, the threshold is fIxed according equation 9 =
i=l
classifIers derived from 3-fold cross-validation. We clearly see that the OC-SVM is more accurate comparatively to the B SVM. Indeed, errors produced by the OC-SVM varies from FRR=8% to FAR=20%, while the B-SVM produces errors from FRR=65% to FAR=85%. Table 1 reports values of HTER obtained for both SVM
EXPERIMENTAL RESULTS
A. Database Description and Evaluation Criteria The Center of Excellence for Document Analysis and Recognition (CEDAR) signature dataset [17] is a commonly used dataset for off-line signature verifIcation. The CEDAR database is composed of 55 writers; each one has 24 genuine and 24 forgery signatures. The signatures from the different writers are scanned at 300 dpi. In order to evaluate the performance of the OC-SVM comparatively to the B-SVM, we use three standard evaluation criteria:
determined: the SVM parameter, the kernel parameter and the
False Acceptance Rate (FAR) allows taking into
account only skilled forgeries; False Rejected Rate (FRR)
classifIers
using
3-fold
cross-validation.
The
off-line
verifIcation system based on the B-SVM classifIer yields a mean HTER of 14.46% corresponding to the optimal value of the threshold equal to 1.08 while the off-line verifIcation system based on OC-SVM classifIer yields a mean HTER of 4.39 corresponding to the optimal value of the threshold equal to 0.22. The comparative analysis clearly shows that the OC-SVM allows reducing signifIcantly the HTER. This is due to the appropriate choice of the decision threshold, which is defmed from the decision function values computed during the training phase.
TABLE!.
VALUES OF HTER OBTAINED FOR BOTH SVM CLASSIFfERS USING 3-FoLD CROSS-VALIDAnON SVM Classifier
HTER
Threshold
B-SVM
14.46
1.08
OC-SVM
4.39
In the continuation of the present work, the next objective consists to explore other distances used into the RBF kernel in order to attempt to reduce HTER.
REFERENCES [I]
-0.22
A. K. Jain, A. Ross and S. Prabhakar, "An introduction to biometric recognition", IEEE Transaction on Circuits and Systems for Video Technology, Special Issue on Image- and Video-Based Biometrics, vol. 14,No. I,pp. 4-20,2004.
[2]
70 .---.----,---,---�
1. F. Velez, A. SAnchez and A. B. Moreno, "Robust Off-Line Signature Verification Using Compression Networks And Positional Cuttings",The 13th IEEE Workshop on Neural Networks for Signal Processing, vol. 1,
60
pp. 627-636, 2003. [3]
�50
�
"Probabilistic Model
Q)
ro
Il:
[4] 30
�
20
Il: Q) !!!
Dynamic
Signature
Wang and 1.
Verification
Ho,
System",
3,No. II,pp. 1320-1324,2011.
.Q
j.
for
Research Journal of Applied Sciences, Engineering and Technology, vol.
40
c:
T. Yuen, W. L. Lim, C. S. Tan, B. Goi, X.
R. Plamondon and S. N. Srihari, "On-line and off-line handwriting recognition: A comprehensive survey", IEEE Trans. PAMI, vol. 22, No. I,pp. 63-84,2000.
[5]
D. Impedovo and G. Pirlo, "Automatic Signature Verification: The State of the Art", IEEE Transactions On Systems, Man, and Cybernetics, vol. 38,No. 5,2008.
10
[6]
S. Audet, P. Bansal and S. Baskaran, "Offline Signature Verification Using Virtual Support Vector Machines", Artificial Intelligence, Final
10
20
40
50 30 60 False Acceptance Rate (%)
70
80
90
Project,April 7,2006. [7]
F. Camci and R. B. Chinnam., "General support vector representation machine for one-class classification of non-stationary classes", Pattern
Figure 3. The mean ROC curve derived from 3-fold cross-validation
Recognition ,vol. 41,pp. 3021-3034,2008.
using the B-SVM.
[8]
C. Bergani, L. S. Oliveira, A. L. Koreich and R. Sabourin,"Combining different biometric traits with one-class classification",Signal Processing
20 .-------,--,---,
,vol. 89,pp. 2117- 2127,2009. [9]
18
33,pp. 491- 498, 2007.
16 [10]
�14
355-365,2008.
12 Il:
[II]
c:
.� Il: Q) !!!
�
B. Fergani, D. Manuel and A. Houacine, "Speaker diarization using one class support vector machines", Speech Communication, vol. 50, pp.
Q)
ro
,g
K. K. Seo, "An application of one-class support vector machine in content-bases image retrieval", Expert System with Applications. vol.
M. Larry and Y. Malik, "One-Class SVMs for Document Classification", Journal of Machine Learning Research,vol . 2,pp. 139-154,2001.
10 8
[12]
V. Vapnik,The Nature Of Statistical Learning Theory,Springer,1995 .
[13]
B. Scholkopf, 1. Platt, J. Shawe-Taylor, A. Smola and R. Williamson. Estimating the support of a high dimensional distribution, Neural
6
Competition,vol . 13,No.7,pp. 1443-1472,2001. 4
[14]
H. P. Huang and Y. H. Liu, "Fuzzy support vector machines for pattern recognition and data mining", International Journal of Fuzzy Systems, vol. 4,No. 3,pp. 826-835,2002.
[15]
O L-----�----�--� o
20
40
60
Fatse Acceptance Rate (%)
80
100
A. Rabaoui, D. Manuel, Z. Lachiri and
N. Ellouze, "Improved One
Class SVM Classifier for Sounds Classification ", Proceedings of IEEE Conference on Advanced Video and Signal Based Surveillance, London,
Figure 4. The mean ROC curve derived from 3-fold cross-validation
United Kingdom ,2007.
using the OC-SVM.
[16]
R.
L.
Larkins, Off-line
Signature
Verification, The University of
Waikato,2009.
V.
[17]
CONCLUSION AND FUTUR WORK
This paper presented the investigation of the OC-SVM classifier
for
Handwritten
Signature
Verification.
B-SVM.
Recognition and Artificial Intelligence, vol. 18, No.7, pp. 1339-1360,
The
comparative analysis shows that the OC-SVM outperforms the
M. Kalera, B. Zhang, and S. Srihari, "Offline Signature Verification and Identification Using Distance Statistics",International Journal of Pattern 2004.
[18]
S. Bengio and 1. Mariethoz, "A Statistical Significance Test for Person Authentication", The Speaker and Language Recognition Workshop, ODYSSEY,Toledo,Spain,May 31 - June 3,2004
[19]
N. Abbas and Y. Chibani, "Combination of Off-Line and On-Line Signature Verification Systems Based on SVM and DST", The 11th International
Conference
on
Intelligent
Systems
Design
Applications: ISDA' II, Cordoba, Spain, November 22 - 24, 20II.
and
