Signature Verification

Signature Verification

History • • • •

Sumerians used intricate seals applied to clay cuneiform tablets to authenticate their writings. Documents were authenticated in the Roman Empire (AD 439) by affixing handwritten signatures to the documents. In 1677 England passed a an act to prevent frauds and perjuries by requiring documents to be signed by the participating parties. In 1977, the first studies of both off-line and on-line signature verification algorithms were published – Nagel and Rosenfeld “off-line system” IEEE T. Comp. – Liu and Herbst “on-line system” IBM J. Res. Dev.

•

Much research has followed, attempting various methods for both feature extraction and matching – Yang et. al. “Application of Hidden Markov Models for Signature Verification” (HMM) – Lam et. al. “Signature Recognition through Spectral Analysis” (FFT) – Hangai et. al. “Writer Verification using Altitude and Direction of Pen Movement” (DTW) – Lejman et. al. “On-line Handwritten Signature Verification using Wavelets and Backpropagation Neural Networks” (Neural Network) – Crane et. al. “Automatic Signature Verification using a Three-axis Force-sensitive Pen” (Parametric)

Current State of the Art • •

No common agreement on benchmark databases and protocols in the research community. 1st International Signature Verification Competition in 2004 (100 users; 20 genuine and 20 forgeries): – – – –

Web: http://www.cs.ust.hk/svc2004/ On-line signature: 2 Tasks. No companies participated (or participated but remained anonymous). Difficult task (pen tablet without visual feedback, synthetic signatures, forgers were given the dynamics of the signatures to imitate, English and Chinese signatures, etc.). – Best system (3% EER Skilled Forgeries, 1.5% EER Random Impostors)

•

Human performance: – Expert (0.5%FAR @ 7%FRR), Layperson (6.5%FAR @ 26%FRR)

Signature Verification vs. Handwriting Recognition

Paola Garcia Signature Recognition (Verification)

Handwriting Recognition

• On-Line:

Azimuth (0°-359°) Altitude (0°-90°)

X

0°

Az

P

Y

270°

90° Al

180°

0

• Off-Line

SCANNER

100

200

300

Applications • On-line: – SOFTPRO (http://www.signplus.com/) – CYBERSIGN (http://www.cybersign.com/) – CIC (http://www.cic.com/)

• Off-line: – APP-DAVOS (http://www.appdavos.ch/) – NUMEDIA (http://www.sapura.com.my/N uMedia/check.htm)

• Advantages of signature verification: – – – – –

User-friendly. Well accepted socially and legally. Non invasive. Already acquired in a number of applications. Acquisition hardware: • Off-line: ubiquitous (pen and paper). • On-line: inexpensive and already integrated in some devices (Tablet PC).

– If compromised, can be changed. – Long experience in forensic environments.

• Disadvantages: – – – –

High intra-class variability Forgeries Higher error rates than other traits Affected by the physical and emotional state of the user – Large temporal variation

Pattern Recognition System (On-line Signature Verification)

Pattern Recognition System

Al

Az

P

Y

X

• Acquisition devices:

0

100

200

300

Pattern Recognition Process

Preprocessing •

For online signatures no segmentation needs to be performed –

• • •

All parts of the signature are known after sensing

Attempt to eliminate noise from the capturing device, speed of writing, and the writing itself Minimize the potential of eliminating writer dependencies Solutions 1) 2) 3) 4) 5)

Size Normalization Position Normalization Smoothing Re-sampling Ligature

• Position Normalization: 1000

800

600

500 400

200

0

Initial Sample

0

-500

-200

-400

-1000 -600

-1500 -1500

-1000

-500

0

500

1000

1500

2000

2500

1500

-800 -500

0

500

1000

1500

2000

2500

3000

3500

4000

-1500

-1000

-500

0

500

1000

1500

2000

2500

800

600 1000 400

200

500

Center of Mass

0 0

-200

-400 -500 -600

-1000 -2000

-1500

-1000

-500

0

500

1000

1500

2000

-800 -2000

Resample • In order to compare two signatures with respect to their shape, they must be re-sampled to eliminate the dependencies on speed – Sample rate: 100 samples/second

• Temporal features must be extracted beforehand since all local speed information is lost during this process

Resampling • Ensures that the signature is uniformly smoothed – Segments of high writing velocity will be smoothed more than segments that are written slow

Smoothing • A one dimensional Gaussian filter is used in both the x and y directions – Small changes in the signal are smoothed out while the overall structure is kept

• Each segment between critical points is smoothed separately in order to retain their absolute positions

xt

filtered

=

2σ

∑σ f * x i

i = −2

where −

fi =

e 2σ

i2 2σ 2

∑e

j = −2σ

−

j2 2σ 2

orig t +i

Stroke Concatenation • A stroke is the points input between a pen down and pen up sequence • All strokes are connected into one long string – This is done in order to facilitate the use of the string matching procedure

Critical Points • Def: points that carry more information than others – Endpoints of strokes – Points of trajectory change

Critical Points • Re-sampling or smoothing of these points will discard important information about the structure and speed of the signature – Accordingly, these points are never changed throughout preprocessing – The speed information is stored at each of these points

Preprocessing Steps • Original

• Critical points

Preprocessing Steps • Fine re-sampling

• Gaussian filter

Preprocessing Steps • Coarse re-sampling

• Stoke concatenation

Pattern Recognition Process

Functional vs. Parametric •

Functional Approaches (local) – Complete signals (e.g. x(t), y(t), p(t), v(t), a(t), etc.) are considered as mathematical time functions whose values are directly correlated with the feature set. – Difficulties are encountered in the matching step (temporal differences and non-linear distortions) – Feature extracting is relatively simple – More computationally intensive (slower), higher accuracy

•

Parametric Approaches – m parameters are computed as features from the measured signals – Feature extraction is very difficult (selection of meaningful features) – Simple matching techniques for comparing 2 sets of parameters can be used – Very fast matching, lower accuracy

Feature Extraction • Local features – Spatial: static features extracted from the shape of the signature • Change of the distance between two consecutive points (δx, δy) • Absolute y coordinate (y) • Sine and cosine of the angle with the x axis (sin α and cos α) • Curvature (β) • Grey values in a 9x9 pixel neighborhood

– Temporal: features using the ordering (timing) of the signature • Absolute and relative speed at each re-sampled point • Absolute and relative speed between two critical points

– Pressure value

Local Features • δx and δy for point pi are the changes with respect to the subsequent point pi+1 • y is the y-coordinate of each re-sampled point after preprocessing • α is the angle between the xaxis and the line through points pi and pi+1 – α is not used (ex. 1 and 359) – sin and cos are both used for directional information

• β is the angle between the straight lines pi -pi-2 and pi pi+2

Local Features • The 9x9 pixel neighborhood is divided into nine 3x3 squares – Grey values are computed by summing pixel values in each 3x3 square – Most costly operation

• No features are extracted for the last point of a stroke • Total number of features – 15

Temporal Features • Absolute and relative speeds are defined as distance per unit time – Tablet PC captures the position of the pen 100 times per second – Distance is measured in pixels – Only distance between points is necessary to define the speed

• Speed is normalized by dividing the local speed at each sample point by the average writing speed of the signature – Overall speed may vary but the relative speeds should be more stable

Global Features

Pattern Recognition System

String Matching • To compute this alignment, dynamic time warping (DTW) is used

⎧ D (i − 1, j − 1) + d E (i , j ) ⎪ D (i , j ) = Min ⎨ D (i − 1, j ) + Missing Penalty ⎪ D(i,j-1 ) + Spurious Penalty ⎩

– D(i,j) is the optimal alignment up to point i of the first string and point j of the second string – de(i,j) is the Euclidian distance between points i and j

• The overall dissimilarity for a signature

D(I , J )2 Dist ( I , J ) = Norm _ Factor ( N I , N J )

DTW Example • Match strings ‘abcacac’ and ‘bcab’

Difference Score Matrix

Reverse Path

Result

String Matching •

Each point is represented by an n-ary feature vector – Feature reduction is performed

• •

Euclidean distance is used as the metric to compare two feature vectors Each feature is normalized using the z-score –

f′=

f −µ

σ

• A set of pairings between the template and input string is found where the sum of the differences between each pair of aligned points in minimal

User Dependent Normalization • Training set: – Pair-wise distances between all training samples is calculated (DTW) – The sample with the smallest average distance is selected as the template – Normalization Statistics: • Average distance from template • Average maximum distance • Average minimum distance

• Test sample: – Compute DTW against all training samples • Record: distance from template, maximum distance and minimum distance

– Normalize the 3 distances by dividing them by the set’s average statistics

Dimension Reduction • PCA – Transform a 3D vector of highly correlated data to 1D – Find a linear transformation W that maps the original vector (X) to projection coefficient vector (Y)

Y =W X T

– Compute Average:

1 µ = N

– Covariance Matrix: S T = – Eigen-decomposition:

N

∑

i =1

N

∑ (x i =1

i

xi

− µ )( x i − µ ) T

ST e = λ e

PCA • Project the distance vector (max, min, template) to 1 dimension

Validation Set • PCA needs samples to calculate the projection coefficients – Using data from the training or testing sets will bias the results

• Divide data into training, validation, and testing • SVC 2004 – 40 users: 20 genuine signatures, 20 skilled forgeries per user. – Testing procedures: • • • •

Train on 5 genuine signatures Test on 10 genuine signatures Test on 20 skilled forgeries Test on 10 random forgeries

• 2-Fold Cross Validation – Select data from the second half of 20 users (genuine, skilled) for validation set to train PCA – Test on the first half of 20 users with the calculated coefficients – Repeat this process, switching the 2 groups

SVC Examples Genuine

Imposter

Mahalanobis Distance • Introduced by P. C. Mahalanobis in 1936 • A distance measure which utilizes the correlations between the features • d M ( x , y ) = ( x − y )′S −1 ( x − y ) • M is the squared Mahalanobis distance • s represents the within-group covariance matrix • y is the vector of the means of the scores for a group • x is the vector containing the individual scores of a sample

• In our work, a diagonal covariance matrix is assumed

Pattern Recognition System

Score Fusion

Score Fusion

Global Features

Local Features

Weighted Sum Rule • Assign a test sample to wi if – Wl P ( wi | xl ) + Wg P ( wi | x g ) > Wl P( wa | xl ) + Wg P ( wa | x g ) And to wa otherwise

Biometric Comparison