Automatic Extraction of Sigma-Lognormal Parameters on Signatures

Automatic Extraction of Sigma-Lognormal Parameters on Signatures Christian O'Reilly

Réjean Plamondon

École Polytechnique de Montréal, Laboratoire Scribens, Département de Génie Électrique, C.P. 6079, Succursale Centre-Ville, Montréal QC, H3C 3A7, Canada {christian.oreilly, rejean.plamondon}@polymtl.ca Abstract Prior publications have highlighted that the SigmaLognormal model is one of the most suitable representation for the analysis of complex movement kinematics [1]. So far, the extraction of parameters from experimental data has been done interactively. This is a process that might become awkward when applied to large databases. This limitation has restrained the use of the model as a framework for human movement analysis, particularly in the field of on-line processing of handwriting. In this paper, a short overview of Sigma-Lognormal paradigm and the key steps for the development of a completely automatic sigma-lognormal parameter extractor for complex human movements are presented. Initial results on the automatic extraction of parameters from on-line signatures are also reported. Keywords: Signature analysis, Sigma-Lognormal, parameter extraction, handwriting processing.

1. Introduction When designing a signature verification system, one has to answer at least three questions: (1) How individual signatures will be represented? (2) How signature classes will be described? (3) And finally, by which means individual signatures will be compared to signature classes? This paper is addressing the first of these questions and does it by using a physiological model of human movement production. This choice is motivated by the hypothesis that such a representation would lead to an improved characterization of the signer’s hidden specificities. It also has the advantage of being invariant in regard of cultural or language differences whereas systems based on visual characteristics often need to be tailored for Chinese, Arabic, European or American signatures. This design orientation raises the question of choosing an adequate physiological model. This topic is discussed at the beginning of the second section which also presents the details of the chosen model.

Section 3 addresses the important question of extracting parameters for the chosen model. Examples of preliminary results obtained on human signatures are given in section 4. Finally, propositions for future works conclude this paper.

2. Previous Work 2.1.

Signature representation

Several computational models have been proposed, among many other approaches, to tentatively explain how the central nervous system generates and controls the kinematics of human movements. Some models describe a movement with analytical expressions ([2]; [3]; [4]; [5]) while others proceed through the numerical resolution of a system of differential equations ([6]; [7]; [8]; [9]; [10]; [11]; [12]). The usage of a minimum principle [13] or an equilibrium hypothesis [14] as a basis for solving indeterminacy in human movement control has also been widely discussed in the literature. Facing such variety of choices for a model of representation, some guidelines should be devised. Hence, four criterions have been considered in our choice: (1) The model should have an analytical form for an easy use in analysis as well as in synthesis. (2) It also must be implementable in a parameter extraction algorithm and (3) should make the analysis of kinematics as straightforward as possible, given that data are obtained though a digitizer. (4) Finally, it should be able to represent complex movements. In this context, the Sigma-Lognormal model seemed a good candidate.

2.2.

The Sigma-Lognormal Model

The Kinematics Theory of Rapid Human Movements from which the Delta-Lognormal ( and the SigmaLognormal (ΣΛ) models were developed has been first introduced in [5], [15]. For briefness reasons, only the basis of ΣΛ model will be overviewed here. The Sigma-Lognormal model considers the resulting speed of a neuromuscular system action as having a lognormal shape scaled by a command parameter (D) and

time-shifted by the time occurrence of the command (t0) (see eq. 1). Moreover, because it represents the movement as happening along a pivot, the angular position can be calculated as shown in (2) where the set of parameters Pj is defined in (3). This latter equation is used to represent the ΣΛ parameters in a short form. Finally, erf(x) is the error function, as defined by (4).  v j (t ; P j )



D j  ( t  t 0 j ;  j ,  2j )



 ln( t  t 0 j )   exp    2 2j  ( t  t 0 j ) 2  Dj

 j (t ; Pj )   sj 

 ej   sj Dj

t



v

j





2

j

( ; Pj ) d



D

j

2 

erf ( x) 

 j  j  sj  ej t

t0 j x

e

t 2

(1)

(2)

0

 ln(t  t 0 j )   j  ej   sj    sj  1  erf  2 j  

Pj

   

   

3.  Parameter Extraction 3.1.

Parameter extraction is an essential preprocessing step for ΣΛ signature representation. Many attempts have been conducted to develop algorithms for that purpose. Excellent results have been achieved on simple rapid strokes [16] and some preliminary results have even been reported on complex movements [17]. However, no algorithm has been shown to be robust and flexible enough to process automatically large databases of complex human movements like handwritten signatures. To partly address this problem, we present a powerful ΣΛ parameter extractor, based on the XZERO algorithm [16]. Its general structure is shown on Figure 1. RX0 estimation

(3)

Angle estimation

(4)

dt

0

The synergy emerging from the interaction and coupling of many of these neuromuscular systems results in the sequential generation of complex movements. As shown by (5), this is modeled using a vectorial summation of lognormals. Equation (6) defines the P symbol used to represent a sigma-lognormal parameter array.

  v (t )  (t ; P) 

P 

M

  v j (t; Pj ) j 1

P

t 1

P2t

Extraction Generalities

... Pjt

... PMt



t

(5) (6)

M  v x (t ; P )   v j (t ; Pj ) cos( j (t ; Pj ))

(7)

M  v y (t ; P)   v j (t; Pj ) sin( j (t ; Pj ))

(8)

j 1

j 1

t

Yes Levenberg-Marquardt Optimization

3.2.

Component Identification

To estimate the ΣΛ parameters, lognormal components must first be localized in the velocity magnitude profile. This localization results in a sequence of the five characteristic points, whose definition is given in Table 1. Table 1. Definition of the five characteristic points. Definition (pi) Numerical value (pi_n) Lognormal component First point preceding p3_n beginning. which is a local minimum or has a magnitude of less than 1% of v3_n. 2 First inflexion point. First vt _ n zero crossing i 1

(9) 3

0

t

y (t; P)   v y ( ; P )d

All components are extracted?

Figure 1. Basic structure of the extractor.

The velocity in the Cartesian space can be calculated, as shown by (7) and (8), and then used to obtain positions versus time as done in (9) and (10).

x(t ; P )   vx ( ; P)d

No

(10)

4

Local maximum of lognormal component. Second inflexion point.

0

5

Lognormal component ending.

before p3_n. Local maximum of vt_n. First vt _ n zero crossing after p3_n. First point after p3_n which is a local minimum or has a magnitude of less than 1% of v3_n.

3.3.

t3 _ n  t1 _ n

XZERO Estimation

The algorithm we propose for complex movement parameter extraction is based on a generalization of the XZERO algorithm. Therefore, XZERO will first be introduced. For the interested reader, [16] is a more comprehensive presentation of this topic. XZERO has been designed to improve the estimation of the  parameters of rapid human movements. This kind of movements is a special case of ΣΛ patterns. In this simple case, agonist and antagonist neuromuscular systems are considered as being exactly opposed.  v (t )

 

  v1 (t )  v 2 (t ) D1  (t  t 0 ;  1 ,  12 )  D 2  (t  t 0 ;  2 ,  22 )

(11)

(12) (13) (14)

Non-trivial zeros of (12) and (13) can be used to obtain the time occurrence and the magnitude of the points p2, p3, and p4. Furthermore p1 and p5 can be chosen so that they define an interval containing 99.97% of the surface under the lognormal curve. The corresponding analytical expressions are given by (15) and (16) with the ai parameter given by (17).

ti  t0  e  e  a i

v ti  a1  3

2   ai  a i 2  2  1 

D 2

e  e

   

i  1,2,...,5

(15)

i  1,2,...,5

(16) (17.a)

a 2  1.5   0.25  1 a3   2

(17.b)

a 4  1.5 2   0.25 2  1 a5  3

(17.d)

2

2

(17.c) (17.e)

Using theses relations, it can be shown that can be estimated solving the non-linear expression (18).

e  a3  e  a1 e a5  e a1

(18)

Then, t0, D and  can be calculated as shown in equation (19) to (21). These expressions are given in a generalized form. In the particular case of the XZERO algorithm, i=3 in (19) and (20) and {i=4, j=2} in (21).

t 0  ti _ n  e  e  a i D 

(19) 

2 v ti _ n e  e

 a i2   ai  2 2 

   

t t    ln  i_an j_an  e  e  i

XZERO exploits the properties of zero crossing points of the first and the second derivatives of a single timeshifted and weighted lognormal equation. Their analytical expression is given by (12) and (13) where k is defined as in (14).  (t  t )    (t  t 0 ) (  k )  0  (t  t 0 )  (t  t )    (t  t 0 ) ( k 2  3k  2 2  1)  0  2 (t  t 0 ) 2 ln( t  t 0 )   k 

t5 _ n  t1 _ n



j

(20) (21)

Considering that the antagonist contribution is nonsignificant in regard of the agonist’s one, (18) through (21) are used directly on |v(t)| to calculate the agonist parameters. This yields an approximation of |v1(t)|. Subtracting it from |v(t)| gives an estimation of |v2(t)|. This signal is then used to extract antagonist parameters in a somewhat similar fashion then what has been done for the agonist component. XZERO has been shown to lay very good results at extracting  parameters. However, it cannot be used as-is on complex movements. The Robust XZERO algorithm (RX0) achieves this goal by generalizing the overall methodology [16].

3.4.

Robust XZERO Estimation

Equations (17.x) are at the basis RX0 estimation. These relationships use only velocity amplitude signals. Thus, for every lognormal component, four speed related parameters (i.e. t0, D,  and ) have to be estimated by the RX0 algorithm. Angular parameters are dealt with afterward. To reach that goal, six constraints are available from the p2, p3 and p4 expressions. These are of the form vti=vti_n or ti=ti_n where the left-hand term has an analytical expression as given by (15) and (16) and the right-hand term has a numerical value obtained from the velocity profile, as described in the third column of Table 1. From these six constraints, 15 different four elements combinations are available to estimate the lognormal parameters. However, it can be shown that there is a redundant constraint in sets using all there vti values, so there are in fact only 12 sets that are useful. D, t0, and  can be calculated as shown in equation (19) to (21), taking whatever vti_n and ti_n values at hand. As for , its expression depends on the available vti values as shown in Table 2. The ij parameter used in this table is a

ratio of vti_n on vtj_n. Moreover, the three sets of constraints using only one vti need numerical resolution. To stick with closed-form solutions only, these combinations will be discarded, leaving nine useful sets that will be used to lay nine different lognormal parameter estimations. The estimation which minimizes the least-square reconstruction error of |v(t)| is kept as the initial solution output from RX0 algorithm. Table 2. Expression of  depending on the available vti constraints. Available vti_n  value vt2_n and vt3_n 1

 2  2 ln(  23 ) 

vt2_n and vt4_n

t4  t3  t4  e e 1

vt 2 

  2  vt 2  e4 vt 3 1

  2  vt 4 4 vt 4  e vt 3

      1  e 2   

 2  4  

   

(25.b)

 2  4  2  

 2  4  2  

(26.a)

(26.b)

The variability of  and  limits the possible localization of p2_n and p4_n as shown in Figure 2. Furthermore, if variability of  is restricted in either of the interval [0, 0.65] or [0.65, +∞], these zones can be approximated by quadrilaterals as shown on Figure 3.

2  2 ln( 42 )

vt3_n and vt4_n

 2  2 ln( 43 ) 

t 2 _ n  t3 _ n

Only one vti_n

t4 _ n  t2 _ n 3.5.

2 ln(  23 )

  2

1 2 ln( 43 )

e  a4  e  a3   a4 e  e  a2

Parameter Variability Restriction

To increase robustness, the a priori knowledge on the possible variability of the parameters has to be used in the estimation process to narrow the range of possible solutions to the parameter estimation problem. So far, variability restriction has only been applied to the  and  parameters. Let’s consider the general case where the parameter variability is defined as in (22).

   ,        ,  

(22.a)

Figure 2. Example of parameter variability limitation ([2.0,1.0] and [0.2,0.5]).

(22.b)

The goal here is to use this knowledge to correct the p2_n, p3_n and p4_n values when noise or superposition drift these points out of the region they should be with respect to the variability of  and . Because the value of p3_n has been found to be the more reliable than the p2_n and p4_n positions, these latter are limited relatively to p3_n. Equations (23) and (24) define the relative position of a point from p3_n and (25) and (26) give the corresponding analytical expressions for p2 and p4.

t  t  t3 vt  vt / vt 3

(23) (24)   2

t2  t3  t2  e e

      1  e 2   

 2  4  

   

(25.a)

Figure 3. Quadrilateral zone approximation.

Equations (27.x) give the conditions for a sample from vt_n to be a valid candidate for p2_n. The f(…) function used in these expressions is defined in (29). Analogous equations can be developed for p4_n. Using these relationships, noisy values of p2_n and p4_n can be corrected and a reliability confidence measurement can be obtained considering the importance of the correction made. Table 3 shows typical results of the utilization of the parameter variability restrictions on p2_n and p4_n accuracy.

vt _ n  f (t _ n , p2 (  ,   ), p2 (  ,   )) (27.a) vt _ n  f (t _ n , p2 (  ,   ), p2 (  ,   )) (27.b) vt _ n  f (t _ n , p2 (  ,   ), p2 (  ,   )) (27.c) vt _ n  f (t _ n , p2 (  ,   ), p2 (  ,   )) (27.d)

D, the angular parameters can be obtained by linear extrapolation using equations (32) and (33).

 

Table 3. Average error (in %) on p2 and p4 before and after the utilization of the parameter variability restrictions t_n vt_n extracted p2 5.1 46.2 corrected p2 2.5 11.0 extracted p4 4.6 19.8 corrected p4 2.2 5.4

3.6.

Multiple Components Extraction

To achieve parameter extraction on complex movements, a strategy must be adopted to extract multiple lognormal components from the velocity profiles. Figure 4 shows the iterative architecture used for this purpose. Pi* P* P=[ ]

Single component estimation

Pi=[0 … 0] (x, y)

Pi* P*

(x, y)*in

no Last serie?

yes

P=[P*; Pi*]

p1 – p5 series vt*

Series validation

Figure 4. Diagram for the multiple components estimation On this schema, expressions next to solid circles show the initial values used for the first iteration while those next to solid diamonds show the final values after the last iteration. The single component estimation block diagram can be further detailed as shown on Figure 5. pi (x, y)in P

 ln(t  t 0 )    D  1  erf  2  2  

(33.b)

Furthermore, the angular parameters can be obtained in an analogous way using only two points from {p2, p3, p4} if one of these points is suspected to be less reliable than the others because of noise or important superposition of lognormal components.

Angle Estimation

l (t  t 0 ; D,  ,  ) 

(33.a)

 e   _ n (t3 _ n )   l (t5 )  l (t3 ) 

Velocity computation

The parameters D, t0,  and  having been estimated, the angular parameters θs and θe remain to be evaluated. From the hypothesis that single lognormal movement components are acting along a pivot, it can be seen that the angular variation is proportional to the traveled distance along the trajectory (i.e.  (t )  l (t ) ). This property can be used to estimate the angular parameters. Equation (30) shows the expression of the trajectory length as a function of time. Using (15) and (30), the distance of pi from the beginning of the movement is obtained as shown by (31).

(32)

 s   _ n (t3 _ n )   l (t3 )  l (t1 ) 

3.7.

p2 ( ,  )  (t2 ( ,  ), vt 2 ( ,  )) (28) y y f ( x, ( x1 , y1 ), ( x2 , y2 ))  2 1 ( x  x1 )  y1 (29) x2  x1

_ n (t 4 _ n )   _ n (t 2 _ n ) l (t 4 )  l (t 2 )

SNRi

+

RX0 pi+1 + angles

+

RX0 p i' + angles

SNRi'

pi* Performance (x, y)* in evaluation * P

(30)

0 i  1   D    ai  (31) l (ti )   1  erf   i  2,3,4 2 2        i  5 D Given the value of φ_n(ti_n) taken from (x, y)_n and the values of l(ti) computed from the estimated values of  and

-

Figure 5. Diagram single lognormal component estimation Lognormal equations are estimated and subtracted from vt in the order of their timing occurrences. Thus, they should not be distorted by the superposition of a preceding

lognormal component. Furthermore, the extraction strategy proposed in Figure 4 and 5 minimizes distortion from superposition of the following lognormal component. This way, parameter estimation is not biased by superposition effects from direct neighbouring lognormals. According to Figure 5, the output of the performance evaluation module depends on its input values, as shown by (34) and (35). It must be noted that, in the expression (x, y)in - pi used in equation (34) and on Figure 5, the minus sign stand for "removing the effect of the computed lognormal from parameter pi to the numerical signal (x, y)in ". It must not be interpreted as a direct mathematical subtraction because the two operands are of a different kind and therefore cannot be directly subtracted.

 pi*  pi 1  SNRi  SNRi'   P*  [ P; pi ] ( x, y )*  ( x, y )  p in in i 

(34)

 pi*  pi  SNRi  SNRi'   P *  P ( x, y )*  ( x, y ) in in 

(35)

Back to the schema on Figure 4, when the single component estimation module has been executed, (x, y)* is used to reconstruct the magnitude of a velocity. This signal is then scanned for valid p1-p5 series. If only one series is remaining, pi* already contains its estimation and the iteration process must stop. Otherwise, the iteration continues. Series are invalid if the area under the curve from t1 to t5 is less than the minimum value of the D parameter that a lognormal component should have (a 5mm value is used in the current experimentation). There is also a rejection if the maximum value of vt is greater than ten times the vt3 value. These latter two conditions are empirical; as will be seen in the next section, they have lead to good experimental results.

shows, on it right most column, the number of lognormal components extracted for each signature.

(a) Signature 1 - X-Y

(b) Signature 1 - v

(c) Signature 2 - X-Y

(d) Signature 2 - v

(e) Signature 3 - X-Y

(f) Signature 3 - v

t

t

t

Figure 6. Signature reconstruction from the parameter extraction results. Axis are in mm for a, c, e and x and y axis are respectively in s and in mm/s2 for b, d and f. Table 4. SNR and number of extracted components Signature SNR x-y (dB) SNR v (dB) Nb. of  t 1 15.9 14.6 43 2 23.3 21.3 31 3 25.4 28.9 41

4. Results on Signatures The extractor has been tested on real human signatures digitized using a Wacom Intuos 200Hz tablet. No manual preprocessing of any kind has been conducted on data. The entire extraction process is completely automated. Figure 6 shows typical results obtained. On these plots, the continuous lines show the original signature traces (a, c, e) and their velocity magnitudes (b, d, f). The dashed lines depict the results obtained from the ΣΛ synthesis of the signature traces and their velocity magnitudes using the extracted parameters. Table 4 gives the signal to noise ratios as computed from the vt and x-y signals. It also

5. Conclusion and Future Work The representation of signatures using the ΣΛ model solves many online signature preprocessing and representation problems still challenging [18]. Indeed, processes such as segmentation and feature extraction are embedded in the extraction of ΣΛparameter. In this paper, the development of a completely automatic ΣΛ extractor able to devise such signature representation has been presented. To our knowledge, it is the first robust extractor developed for such a purpose. It availability should provide

new possibilities for the analysis of large complex human movement databases. At this moment, the verification or classification of signatures is not at reach. The work presented here is a first step aiming at modeling signature by lognormals. A model of signature classes such as a probabilistic model, like a HMM model of sequencing of lognormal commands, has still to be developed for signatures to be compared and verified. Although the performances reported in Figure 6 look quite impressive, there is still room for improvements. To reach a sufficient level of accuracy, robustness and flexibility for commercial and industrial requirements, research efforts will have to be conducted on many topics, such as the generalization to 3D movements, a better tuning regarding ΣΛ parameter variability, the performances amelioration in terms of signal to noise ratio and the adoption of a faster and more flexible non linear optimization technique. Finally, it is important to be aware that different lognormal extractions can offer equally good fitting for a particular movement. It will be important to address the question of characterizing the level of correspondence between a particular extraction and the process which generated the observation. This is essential to choose the good set of parameter, to make sure that the full power of the ΣΛ representation is exploited.

6. Acknowledgments This work was partly supported by NSERC grant RGPIN-915 to Réjean Plamondon. The data presented in Figure 6 are from a database collected during a research project supported by the Fondation Lucie and André Chagnon.

References [1] R. Plamondon and M. Djioua, “A multi-level representation paradigm for handwriting stroke generation,” Human Movement Science, vol. 25(4-5), pp. 586–607, 2006. [2] M.A. Alimi, “Beta neuro-fuzzy systems”, TASK Quarterly J., Special Issue on “Neural Networks” eds. W. Duch and D. Rutkowska, 7(1):23–41, 2003. [3] N. Hogan, “An organization principle for a class of voluntary movements.” The Journal of Neurosciences. 4: 2745-2754, 1984. [4] W.L. Nelson, “Physical principles for econo-mies of skilled movements.” Biological Cybernetics. 46:135147, 1983. [5] R. Plamondon, “A kinematic theory of rapid human movements. I. Movement representation and generation,” Biological Cybernetics, vol. 72, no. 4, pp. 295 – 307, 1995.

[6] D. Bullock and S. Grossberg, “The VITE model: A neural command circuit for generating arm and articulator trajectories.” In J.A.S. Kelso, A.J. Mandell, and M.F. Shlesinger(Eds.), Dynamic Patterns in Complex Systems. Singapore: World Scientific Publishers, 305–326, 1988. [7] C.M. Harris and D.M. Wolpert “Signal-dependent noise determines motor planning.” Nature. 394: 780784, 1998. [8] P.D. Neilson, “The problem of redundancy in movement control: the adaptive model theory approach.” Psychological Research. 55:99-106, 1993. [9] P.D. Neilson and M.D. Neilson, “An overview of adaptive model theory: solving the problems of redundancy, resources and nonlinear interactions in human movement control.” Journal of Neural Engineering. 2(3):279-312, 2005. [10] H. Tanaka, J.W. Krakauer, and N. Qian, “An optimization Principle for Determining Movement Duration.” Journal of Neurophysiology. 95:38753886, 2006. [11] Y. Uno, M. Kawato and R. Suzuki “Formation and control of optimal trajectory in human multijoint arm movement.” Biological Cybernetics. 61:89-101, 1989. [12] G. Gangadhar, D. Joseph and V.S. Chakravarthy, “An oscillatory neuromotor model of handwriting generation”, Int. Journal on Document Analysis and Recognition, vol. 10, no. 2, pp. 69-84, 2007. [13] S.E. Engelbrecht, “Minimum Principles in Motor Control,” Journal of Mathematical Psychology, Vol. 45, pp. 497-542, 2001. [14] A.G. Feldman and M. L. Latash, “Testing hypotheses and the advancement of science: recent attempts to falsify the equilibrium point hypothesis,” Experimental Brain Research, Vol. 161, no. 1, pp. 91-103, 2005. [15] R. Plamondon, “A kinematic theory of rapid human movements. II. Movement time and control,” Biological Cybernetics, vol. 72, no. 4, pp. 309 – 20, 1995. [16] M. Djioua and R. Plamondon, “A new algorithm and system for the extraction of delta-lognormal parameters,” Technical Report EPM-RT-2008-04, École Polytechnique de Montréal, 2008. [17] W. Guerfali and R. Plamondon, “A new method for the analysis of simple and complex planar rapid movements,” Journal of Neuroscience Methods, vol. 82, no. 1, pp. 35-45, 1998. [18] G. Dimauro et al. “Recent Advancements in Automatic Signature Verification,” Proc. Ninth International Workshop on Frontiers in Handwriting Recognition, pp. 179-184, 2004.