Multi-Expert Verification of Hand-Written Signatures - CiteSeerX

Multi-Expert Verification of Hand-Written Signatures L. Bovino(*), S. Impedovo(°), G. Pirlo(°), L. Sarcinella(*) (°)

(*)

Dipartimento di Informatica - Università degli Studi di Bari Via Orabona 4 - 70126 Bari – Italy

Consorzio Interuniversitario Nazionale per l’Informatica (CINI) Via Giulio Petroni 15/F.1 - 70100 Bari- Italy

Abstract This paper presents a multi-expert system for dynamic signature verification. The system uses a stroke-oriented description of signatures well-suited for multi-expert approach. Each stroke is analysed in multiple representation domains to verify locally both the shape and dynamics of a signature. A two-level scheme for decision combination is used to combine local decisions. At the first level soft- and hard- combination rules are used to combine decisions from different representation domains. At the second level simple and weighted averaging is used to combine decisions from different parts of the signature.

1.

Introduction

The spreading of geographic networks makes automated personal verification widely requested for accessing reserved information, transferring electronic funds, subscribing legal documents and banking transactions. Biometry offers potentials for verifying the identity of a subject by the analysis of physical and behavioural characteristics. Physical characteristics can be obtained from finger-print, palm-print, face gestures, blood vessels in the retina, or DNA. Behavioural characteristics can be obtained from key-stroke dynamics, speech, hand-written signature [1]. Among others, hand-written signature is one of the most interesting means for automated personal verification. Signature is the customary way of identifying an individual in our society and it is well-accepted by every user for any kind of legal attestation and administrative certification [2]. Unfortunately, automatic verification of hand-written signatures is a difficult classification problem since it must be performed only on the basis of a few positive specimens available for learning [3,4]. Moreover, hand-written signature is the result of a complex process based on a sequence of predetermined actions, stored in the human brain, and realised by the writing system of the signer Acknowledgements This paper has been supported by the Italian Ministry "Ministero dell'Istruzione, dell’Università e della Ricerca", MIUR under law 488, Project "Rete Puglia" CINI-BA - D.M. n.600, 11-11-1999.

(arms and hands) through ballistic-like movements. Therefore, signatures written by the same person can be very different depending on the physical and psychological state of the writer [5]. The multi-expert approach is one of the most promising research areas in the field of automatic signature verification. So far, several systems have been proposed which combine hard [6,7] or soft [8,9] decisions from verifiers based on different sets of features, using parallel [6,10], serial [11] or hybrid [12] strategies. In this paper a new multi-expert system for dynamic signature verification is presented. Unlike previous approaches, the system is based on a segmentation technique of hand-written signatures well-suited for multi-expert verification since it provides compatible stroke-oriented descriptions of the test and the reference signatures and allows signature analysis by strokes. The authenticity of each stroke is verified individually by combining, via softor hard- combination rules, the verification decisions obtained in multiple representation domains (position, velocity, acceleration), which provide complete information on the stroke shape and dynamics. The final verification decision for the test signature is obtained by simple or weighted averaging of the local decisions on the authenticity of each individual stroke. The experimental results show the effectiveness of the new system when decision combination is performed via simple averaging, both for the combination of decisions from different representation spaces and from different parts of the signature. The paper is organised as follows: Section 2 describes the multi-expert process of signature verification. Section 3 presents the segmentation technique. Stroke verification, performed in multiple domains of representation, is described in Section 4. Section 5 presents the rules for decision combination at stroke and signature level. The experimental results are presented in Section 6.

2.

Multi-expert Signature Verification

In this work the acquisition of a hand-written signature is performed in on-line mode by an electronic tablet

Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003) 0-7695-1960-1/03 $17.00 © 2003 IEEE

providing the position of the tip of the pen on the tablet and the pen-up / pen-down signal. After acquisition, signatures are suitably normalised in position, size, and time-duration [13]. The segmentation of a test signature is performed according to the characteristics of the reference specimens, in order to obtain compatible stroke-oriented descriptions for the test and the reference signatures. In particular, in this work, three reference signatures have been considered for each signer, as proposed elsewhere [14,15]. In the feature extraction phase, function features are used to describe the input signature as a time-function in the position, velocity and acceleration domains [3]. In particular, the position signal is provided directly from the tablet and it is used to derive velocity and acceleration numerically. In the comparison process each stroke of the test signature is compared to the corresponding strokes of the reference signatures belonging to the individual whose identity must be verified. Decision combination follows a two-level scheme. At the first level, soft- or hard- combination rules are considered to combine, for each stroke of the signature, the decisions from different representation domains. Local decisions on the authenticity of each stroke are combined at the second level by considering simple or weighted averaging, in order to obtain the final response for the test signature.

3. Hand-written Signature Segmentation Dynamic Programming (DP) has been used to segment the test signature in a way compatible to the reference signatures, despite possible variations among specimens due to personal variability in signing [13, 16]. Signature segmentation uses the two-level procedure described in the following. At the first level DP is applied to the whole signature, based on the analysis of the local maxima in the vertical direction, which are very robust features. At the second level DP is used to split each segment into basic strokes, on the basis of the local minima in the vertical direction. 3.1 First-level segmentation. Let St be the test signature and S1,S2,S3 three reference signatures, as Figure 1 shows. At the first level the segmentation technique compares the sequences of the local maxima of St and S1,S2,S3 (marked by “*”). Precisely, let Mt(i), i=1,…,NtMAX be the sequence of the local maxima of St, Mr(i), i=1,…,NrMAX be the sequence of the local maxima of Sr , r=1,2,3; DP is used to detect the warping function W*(Mt,Mr)=(i1,j1), (i2,j2),.., (ik,jk),.., (iK,jK) (k,ik,jk integers, so that 1≤k≤K, 1≤ik≤NtMAX, 1≤jk≤NrMAX), which satisfies

monotonicity, continuity and boundary conditions makes minimum the quantity [14]:

D(Mt,Mr) =

and

K 1 ⋅ ∑ d(ik, jk) , r N MAX + N MAX k =1 t

where d(ik,jk) is the Euclidean distance between Mt(ik) and Mr(jk). Successively, the set of directly matched local maxima are identified [15]. Let Mt(p) be coupled to Mr(q) by W*(Mt,Mr), Mt(p) and Mr(q) are directly matched if the following conditions are satisfied: (a) ∀p'=1,…,NtMAX, p'≠p : Mt(p') is not coupled to Mr(q); (b) ∀q'=1,…,NrMAX, q'≠q : Mr(q') is not coupled to Mt(p). In Figure 1 the directly matched local maxima are coupled by a continuous line: from W*(Mt,M1) we obtain the sets of directly matching points BtW*(Mt,M1) for St, B1W*(Mt,M1) for S1: BtW*(Mt,M1) ={Mt(1), Mt(2), Mt(4), Mt(9), Mt(11)}, B1W*(Mt,M1)={M1(1),M1(2),M1(5),M1(11),M1(14)}; from W*(Mt,M2) we obtain the sets of directly matching points BtW*(Mt,M2) for St, B2W*(Mt,M2) for S2: BtW*(Mt,M2) = {Mt(1), Mt(2), Mt(3), Mt(4), Mt(8), Mt(9), Mt(11)} B2W*(Mt,M2) ={M2(1), M2(2), M2(3), M2(4), M2(8), M2(9), M2(12)}; from W*(Mt,M3) we obtain the sets of directly matching points BtW*(Mt,M3) for St, B3W*(Mt,M3) for S3: BtW*(Mt,M3) ={Mt(1), Mt(2), Mt(3), Mt(4), Mt(7), Mt(8), Mt(9), Mt(10), Mt(11)} B3W*(Mt,M3)={M3(1), M3(2), M3(3), M3(4), M3(6), M3(7), M3(8),M3(9), M3(10)}. The analysis of the directly matched points allows an unambiguous identification of segmentation points on the test and reference signatures [15]. A direct matching point Mt(p) of St is a segmentation point if and only if an index qr ∈ {1,…,NrMAX} exists so that Mt(p) is directly matched to Mr(qr), ∀r=1,2,3. Moreover, for each segmentation point Mt(p) of the test signature, the corresponding points Mr(qr), r=1,2,3, are used to split the reference specimens. In other words, for the test signature St, the set of segmentation points is: Bt = BtW*(Mt,M1) ∩ BtW*(Mt,M2) ∩ BtW*(Mt,M3) , and the set of segmentation points for Sr (r=1,2,3) is: Br ={ Mr(q) ∈BrW*(Mt,Mr) | Mr(q) is coupled to Mt(p)∈Bt }. In Figure 1, the set of segmentation points are: Bt={Mt(1),Mt(2),Mt(4),Mt(9),Mt(11)}, and B1= {M1(1), M1(2), M1(5), M1(11), M1(14)}; B2= {M2(1), M2(2), M2(4), M2(9), M2(12)}; B3= {M3(1), M3(2), M3(4), M3(8), M3(10)}.

Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003) 0-7695-1960-1/03 $17.00 © 2003 IEEE

Test Signature : St

First Reference Signature : S1

Second Reference Signature : S2

Third Reference Signature : S3

W*(Mt,M1)

W*(Mt,M2)

W*(Mt,M3)

Directly Matched Points of St: 1,2,4,9,11 Directly Matched Points of S1: 1,2,5,11,14

Directly Matched Points of St: 1,2,3,4,8,9,11 Directly Matched Points of S2: 1,2,3,4,8,9,12 Segmentation Points of St: 1,2,4,9,11

Directly Matched Points of St: 1,2,3,4,7,8,9,10,11 Directly Matched Points of S3: 1,2,3,4,6,7,8,9,10

Segmentation Points of S1: 1,2,5,11,14

Segmentation Points of S2: 1,2,4,9,12

Segmentation Points of S3: 1,2,4,8,10

Figure 1: First level Segmentation (by Local Maxima)

3.2 Second-level segmentation. At the second level of segmentation, the points which are candidates for splitting are the local minima in the vertical direction. Analogously to the first-level segmentation, DP is applied to each corresponding segment of the test and the reference signatures to obtain the final stroke-oriented description of St and S1,S2,S3. Note that this segmentation technique always splits each test and reference signature into the same number of strokes, avoiding the problem of additional/missing stroke treatment.

4. Local Verification Decision Local verification (i.e. verification at the stroke level) is performed in the domains of position, velocity, and acceleration. Let Sts(i) be the i-th stroke of St, and Srs(i) the corresponding stroke of Sr (r=1,2,3), represented in the domain s (s=position, velocity, acceleration). Moreover let St(i)_Length and Sr(i)_Length (r=1,2,3) be the length (number of points) of Sts(i) and Srs(i), respectively. The verification of Sts(i) is performed by DP that detects the warping function W*(Sts(i), Srs(i))= (p1,q 1), (p2,q2), …, (pg,qg), …, (pG,qG) between points of Sts(i) and Srs(i) (1≤g≤G, 1≤pg≤St(i)_Length, 1≤qg≤ Sr(i)_Length), which satisfies monotonicity, continuity and boundary conditions and makes minimum the quantity [14]: G

D(Sts(i),Srs(i)) = ∑ ds(pg,qg) , g =1

where ds(pg,qg) is the Euclidean distance (in the domain s) between the pg-th point of Sts(i) and the qg-th point of Srs(i).

In this way the confidence value Cts(i), for the stroke t S s(i), is determined as Cts(i)=Vts(i)/ Ts(i), where: Vts(i) = Min {D(Sts(i), Srs(i)) | r=1,2,3} is the minimum distance of Sts(i) from the corresponding strokes of

genuine specimens; Ts(i)= Max {D(Sr1s(i), Sr2s(i)) | r1,r2=1,2,3, r1≠r2 } estimates the local variability of genuine strokes corresponding to Sts(i). Finally, for the stroke Sts(i), represented in the domain s, the verification decision Rts(i) is obtained by the following rule [14,15]: Rts(i)=1 if Cts(i) ≤ 1 ⇔ Sts(i) is authentic t R s(i)=0 otherwise ⇔ Sts(i) is false.

5. Decision Combination A two-level scheme is used for decision combination. Soft- and hard- rules are used at the first level to combine decisions from different representation domains. Simple and weighted averaging is used at the second level to combine decisions concerning different parts of the signature. 5.1. Decision Combination from different representation domains. One soft-rule and three hard-rules are used to combine decisions from different representation spaces. Let Rt (i) be the verification decision for St(i) (the i-th stroke of the test signature St), where: Rt(i)=1 means that St(i) is authentic, Rt(i)=0 means that St(i) is false. The value of Rt (i) is determined as follows:

Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003) 0-7695-1960-1/03 $17.00 © 2003 IEEE

determines the percentage of the test signature that is authentic:

1. Soft - Averaging Rule 1 R t (i ) = R st (i ) . 3 s

∑

2. Hard - OR Rule Rt (i) =1 if Rts(i)=1 in at least one domain out of three; Rt (i)=0 otherwise. 3. Hard - Majority Rule Rt (i) =1 if Rts(i)=1 in at least two domains out of three; Rt (i)=0 otherwise. 4. Hard - AND Rule Rt (i) =1 if Rts(i)=1 in three domains; Rt (i)=0 otherwise. 5.2. Decision Combination from different regions of the signature. Two soft rules are used for decision combination at the signature level. Let Rt be the verification decision for the test signature St, where: Rt(i)=1 means that St is authentic, Rt(i)=0 means that St is false. The value of Rt is determined as follows: 1. Simple_Ave is the simple average of the decisions at the stroke level. Simple _ Ave( R t ) =

1 N

N

∑ R (i) t

i =1

2. Weight_Ave is the average of the decisions at the stroke level, weighted by the length of each stroke. It

Weight _ Ave( R t ) =

N

1

∑ R (i) ⋅ L (i) ,

t

L

t

t

i =1

where Lt is the length of St and Lt(i) is the length of the stroke St(i), i=1,2,,N; being N the number of strokes of St.

6

Experimental Results

In order to test the multi-expert system, authentic signatures and skilled forgeries were collected in controlled writing sessions. Fifteen writers were engaged to collect the authentic signatures and other fifteen people produced the forged samples. Every writer signed in a rectangle of 4 cm x 12 cm on the graphic tablet (sampling frequency: 110 Hz). In each session, the writer had about ten minutes to practice with the electronic tablet and five minutes to affix up to five signatures. The forgers attended the writing sessions and trained themselves in imitating the genuine signatures. After enrolment, a database of fifty genuine signatures and fifty (skilled) forgeries were available for each writer. Three extra genuine signatures were collected for reference. The verification results at the stroke level in each representation domain are reported in Table 1, in terms of False Rejection Rate (FRR), due to falsely rejected originals, and the False Acceptance Rate (FAR), due to falsely accepted forgeries. Velocity is the most useful domain for personal verification, since we have FRR=18% and FAR=31%, on average.

Table 1. Stroke Verification #2 25% 42% 35% 30% 35% 38%

#3 27% 30% 17% 34% 25% 25%

#4 30% 25% 10% 31% 30% 32%

#5 31% 22% 16% 25% 30% 20%

Simple_Ave

#1 35% 50% 25% 41% 40% 38%

Ave OR Maj AND

#1 2% 8% 4% 10%

#2 2% 4% 2% 2%

#3 0% 0% 0% 2%

#4 0% 0% 0% 0%

#5 0% 0% 0% 0%

Weight_Ave

Domain: Pos. FRR FAR Vel. FRR FAR Acc. FRR FAR

Ave OR Maj AND

2% 6% 4% 10%

0% 2% 2% 2%

2% 0% 0% 4%

0% 0% 0% 0%

0% 2% 0% 2%

Signer #8 29% 20% 17% 38% 36% 33%

#10 5% 20% 17% 40% 36% 33%

#11 10% 40% 10% 22% 17% 48%

#12 15% 40% 10% 33% 21% 29%

#13 20% 41% 15% 26% 25% 37%

#14 14% 37% 12% 28% 23% 40%

#15 40% 42% 25% 26% 27% 43%

Total 24% 34% 18% 31% 28% 34%

Table 2. Signature Verification Signer #6 #7 #8 #9 #10 0% 0% 0% 0% 0% 2% 2% 4% 0% 0% 2% 2% 0% 0% 0% 2% 0% 4% 0% 0%

#11 0% 0% 0% 0%

#12 0% 2% 0% 0%

#13 0% 2% 2% 0%

#14 0% 0% 0% 0%

#15 2% 2% 2% 4%

Total 0.4% 1.7% 0.9% 1.6%

0% 2% 2% 2%

0% 0% 0% 0%

2% 4% 0% 0%

0% 0% 0% 0%

2% 2% 0% 4%

0.5% 1.9% 0.8% 2.2%

#6 34% 40% 24% 30% 28% 37%

0% 4% 2% 2%

#7 33% 34% 21% 37% 25% 26%

0% 2% 2% 2%

0% 2% 0% 4%

#9 15% 34% 18% 30% 23% 32%

0% 0% 0% 0%

0% 2% 0% 2%

Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003) 0-7695-1960-1/03 $17.00 © 2003 IEEE

Table 2 reports the Equal Error Rate (EER) of the multi-expert system, which is the error rate where the FRR and the FAR are equal [16]. The majority rule always outperforms the OR-rule and the AND-rule. The soft-rule for combination of decisions from different representation domains always outperforms hard-rules. At the most, the EER of the system is 0.4%, when simple averaging is considered at the first and the second combination levels. 12% Ave/Simple_Ave OR/Simple_Ave Maj/Simple_Ave AND/Simple_Ave Ave/Weight_Ave OR/Weight_Ave Maj/Weight_Ave AND/Weight_Ave

10%

EER

8% 6% 4% 2% 0% 15

17

19

21

23

Number of Strokes (Average)

Figure 2: EER vs Number of Strokes

Figure 2 shows the relationship between the EER and the average number of strokes in the signature of each signer. A short signature, segmented into a few strokes, is easier to forger than a long signature, for which many strokes must be correctly imitated. However, it results that simple averaging is generally less sensitive than weighted averaging to the number of strokes. Conclusion This paper has presented a new multi-expert system for dynamic signature verification. The system is based on a stroke-oriented description of hand-written signatures suitable for multi-expert verification, since complementary information is expected from the analysis of different strokes of the signature. Each stroke is verified into the representation domain of position, velocity and acceleration, in order to evaluate its authenticity in terms of shape and dynamics. Decision combination is performed following a two-level strategy. Soft- and Hard- combination rules are used at the first level to combine stroke verification decisions from different representation domains. Soft-rules based on averaging are used at the second level to obtain the final verification response. The experimental results show that soft-rules are better than hard-rules for decision combination. Specifically, an Equal Error Rate of 0.4% is achieved when simple averaging is considered at the first and the second combination levels. This result shows the superiority of the new system compared to other systems in literature, and it is promising for future applications of signature verification systems.

References [1] Proc. of BiometriCon 2000, Gaithersburg, MD, USA, Sept. 13-14, 2000. [2] R. Plamondon, S.N. Srihari, “On-line and off-line handwriting recognition: A Comprehensive Survey”, IEEE T-PAMI, 2000, 22(1), pp. 63-84. [3] R. Plamondon, G. Lorette, “Automatic Signature Verification and Writer Identification - The State of the Art”, Pattern Recognition , 1989, Vol 22(2), pp.107-131. [4] F. Leclerc, R. Plamondon, “Signature verification: The state of the Art 1989-1993”, IJPRAI, 1994, Vol 8(3),pp. 643-660. [5] R. Plamondon, W. Guerfali, “The generation of handwriting with delta-lognormal synergies”, Biological Cybernetics, 1998, Vol. 78 (2), pp. 119-132. [6] H. Cardot, M. Revenu, B. Victorri, M. J. Revillet, “Coopération de réseaux neuronaux pour l'autentification de signatures manuscrites“, Proc. of the Int. Conf. NeuroNimes, 1991. [7] G.Dimauro, S.Impedovo, G.Pirlo, G.Congedo, “Signature Verification through a Dynamical Segmentation technique”, Proc. 3th IWFHR, 1993, NY, USA, pp. 262-271. [8] R. Plamondon, P. Yergeau, J.J. Brault, “A multi-level signature verification system”, in “From Pixels to Features III”, ed. S.Impedovo and J.C.Simon, Elsevier Publisher 1992, pp. 363-370. [9] H. Cardot, M. Revenu, B. Victorri, M.J. Revillet, “A Static Signature Verification System based on a cooperative Neural Networks Architecture”, IJPRAI, 1994, Vol. 8(3), pp. 679-692. [10] R. Bajaj, S. Chaudhury, “Signature verification using multiple neural classifiers”, Pattern Recognition, 1997, Vol. 30(1), pp. 1-7. [11] L.P. Cordella, P. Foggia, C. Sansone, M. Vento, “Document Validation by Signature: a Serial Multi-Expert Approach”, Proceedings of ICDAR ‘99, Bangalore, India, IEEE Press, 1999, pp. 601-604. [12] G. Dimauro, S. Impedovo, G. Pirlo, A. Salzo, “A multiexpert signature verification system for bankcheck processing”, IJPRAI, World Scientific Publishing, 1997, Vol 11(5), pp. 827-843. [13] Y. Sato, K. Kogure, “On-line signature verification based on shape, motion and writing Pressure”, Proceedings 6th ICPR, 1982, IEEE Press, Vol. 2, pp. 823-826. [14] I. Yoshimura, M. Yoshimura, “On-line signature verification incorporating the direction pen movement An experimental examination of the effectiveness”, in “From Pixels to Features III”, eds. S. Impedovo and J.C. Simon, Elsevier Publishing, 1992, pp. 353-362. [15] V. Di Lecce, G. Dimauro, A. Guerriero, S. Impedovo, G. Pirlo, A. Salzo, “A Multi-Expert System for Dynamic Signature Verification”, in Multiple Classifier System, eds. J. Kittler and F. Roli, Lecture Notes in Computer Science, Springer 2000, pp.320-329. [16] B. Wirtz, “Average Prototypes for Stroke-Based Signature Verification”, Proc. of ICDAR 1997, Ulm, Germany, IEEE Press, pp. 268-272.

Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003) 0-7695-1960-1/03 $17.00 © 2003 IEEE