An Evolving Signature Recognition System

An Evolving Signature Recognition System Buddhika Jayasekara, Awantha Jayasiri, Lanka Udawatta Department of Electrical Engineering,University of Moratuwa, Moratuwa, Sri Lanka [email protected], [email protected], [email protected] Abstract— This paper proposed a signature recognition method based on the fuzzy logic and genetic algorithm (GA) methodologies. It consists of two phases; the fuzzy inference system training using GA and the signature recognition. A sample of signatures is used to represent a particular person. The feature extraction process is followed by a selective preprocessing. The fuzzy inference system is followed by a feature extraction step. The projection profiles, contour profiles, geometric centre, actual dimensions, signature area, local features, and the baseline shift are considered as the feature set in the study. The input feature set is divided into five sections and separate five fuzzy subsystems were used to take the results. Those results are combined using a second stage fuzzy system. The fuzzy membership functions are optimized using the GA. A set of signatures consisting of genuine signatures, random forgeries, skilled forgeries of a particular signature and different signatures were used as the training set. Then, that particular optimized recognition system can be used to identify the particular signature identity. System achieved a signature recognition rate of about 90% and handled the random forgeries with 77 % accuracy and skilled forgeries with 70% accuracy. The recognition results authenticate that this is a reliable and accurate system for off-line recognition of handwritten signatures.

I. INTRODUCTION Signature recognition is an area which is exposed to a vast amount of research works and they have followed many directions to achieve that. It is a subset of biometric recognition techniques and pattern recognition. The Biometric methods are prominent among the authentication techniques. Biometric authentication can be defined as automatic recognition of a person based on his or her physiological or behavioral characteristics [1]. The main physiological biometric methods are fingerprint recognition, facial recognition/face recognition, hand geometry, iris recognition, retinal scan and identification techniques of DNA (deoxyribonucleic acid). Behavioral biometrics include speaker/voice recognition, signature/handwritten recognition and keystroke/patterns [1], [2]. Signature recognition can be divided into two main areas depending on the method of data acquisition: on-line and off-line signature recognition [3], [4]. In off-line signature recognition, signature is available on a document which is scanned to get the digital image representation. The on-line signature recognition uses special hardware, such as a digitizing tablet or a pressure sensitive pen to

1-4244-0322-7/06/$20.00 ©2006 IEEE

record the moments over the paper during writing. Both methods have their own advantages and disadvantages. The off-line data is easy to acquire, a scanner is the only special hardware needed. It can also be applied to signatures that have been acquired in the past and documents that are not in the electronic format. On the other hand, the processing steps needed to extract important features from the signature for recognition are more difficult. No temporal information is available to indicate the sequence of processes in which the strokes were formed. Therefore all the features have to be extracted from the digitized signature pattern [5], [6], [7]. The on-line signature recognition uses the dynamics of hand movements of the signature in addition to its shape. For online data the individual has to be present at the time of signing and actively get participated in writing the signature. On the other hand for online signature recognition, special hardware has to be installed. This can be an obstacle for potential customers. However, we are able to record additional characteristics such as time dependencies and possible pressure and pen tilt which are useful for recognition [8], [9], [10]. The application areas for signature recognition include all applications where handwritten signatures are already used such as in bank transactions, credit card operations or authentication of a document with important instructions or information. The available methods for off-line signature recognition are based on a vast range of concepts. The research findings can be categorized according to the way that it handles the problem as holistic, regional and local property based methods. The holistic methods try to capture a feature vector and it is mapped with a reference to identify similarity measures in the recognition stage. In the regional methodology, it tries to represent the signature using sequence of vectors (E.g. Strokes, Segments, Windows), and the similarity measure is quantized by mapping considering the structure of the sequence [7], [11], [12]. The local methods represent the signature features based on functions of time and space. Here the elastic matching is prominent in the most of the cases. The purpose of the signature recognition process is to identify the writer of a given sample, while the purpose of the signature verification process is to confirm or reject the sample. The recognition and verification strategies are equipped with a vast range of popular tools. There are recognition methods based on various distance functions.

From the machine recognition of handwritten signatures, the online recognition systems can acquire time dependant information like acceleration of writing, applied pressure in writing and pen moment in addition to the resulting signature identity. Therefore the online recognition systems provide an excellent recognition rates. But timely dependant local feature details are not available in the offline systems. Therefore the off-line signature recognition remains as an important method in the variety of applications. Machine recognition of signature is a very special and difficult problem. Those constraints arise due to the complexity of signature patterns, associated large variations in the patterns of a single person (i.e. inter personnel variations) and the forged signatures produced by professional forgers [25].

II.

PREPROCESSING

The signature samples are captured using a scanner in color RGB space. Then scanned signature images converted to a common base, which is to binary format with scale invariance correction. A. Character Image Type Conversion The RGB color space is used as the model for color monitors and broad class of color video cameras. The purpose of a color model is to facilitate the specifications of colors in some standard; generally accepted way [26]. Most color models in use today are oriented either toward hardware such as monitors and printers or towards applications. The other color models are CMY (cyan, magenta, yellow), CMYK (cyan, magenta, yellow, black), NTSC HSI (Hue, Saturation and Intensity), [luminance(Y), hue (I), saturation (Q)].NTSC color system is used in television [12]. In the NTSC color space, the luminance component represents the Grayscale information and other two components represent the color information. Therefore we can convert RGB space to NTSC and gray scale image can generate by neglecting the color information (hue (I), saturation (Q)). RGB image is stored as an M × N × 3 array and each and every pixel

contain Red, Green and Blue color information. The YIQ components are obtained from the RGB components of an image using the following transformation. 0.114   R  Y  0.299 0.587  I  = 0.596 − 0.274 − 0.322 G       Q   0.211 − 0.523 0.312   B 

(1)

Gray scale to binary conversion is done according to a method based histogram based thresholding [26], [27]. Suppose that intensity histogram shown in Fig.1, corresponds to an image f(x, y). Gray level occurrences

The distance between the test sample and the templates were measured as the simple distance or Euclidean distance [13], [14], [15]. The pattern recognition and classification applications exploit neural network based methods. A range of neural network derivations can be found in the field of signature recognition and verification [16], [17], [18]. The Hidden Markov Models and many other structural matching algorithms incorporating prior knowledge were also used in this scenario [19], [20], [21]. The modified Bayesian network [22], Hough Transform [23] and statistical methods based classifiers [24], and recognition methodologies are also available for the purpose.

T Gray Level

Fig. 1. Histogram based thresholding According the histogram all the pixels are arranged into two dominant classes. Those two classes are defined according to the threshold value T. Then the resultant binary image is defined as g(x, y),

1 if  g ( x, y ) =  0 if 

f ( x, y ) ≥ T (2)

f ( x, y ) < T

The threshold value is generated according to the Otsu’s algorithm [26], [27].The implementation of this method can start from the normalize histogram by discrete probability density function,

Pi (i ) =

ni N

(3)

Where N is the total number of pixels in the image, ni is the number of pixels that have intensity level i and L is the total number of possible intensity levels in the image. Now suppose that a threshold k is chosen such that C0 is the set pixels with gray levels [0,1,..,k-1] and C1 is the set of pixels with gray levels [k,k+1,…,L-1]. Otsu’s method choose the threshold value k that maximize the between class variance which is defined as

σ B 2 = ω 0 ( µ 0 − µ T ) 2 + ω1 ( µ 0 − µ T ) 2

(4)

and ω1 are represents the probabilities of class ω0 occurrences and µ 0 and µ1 are the corresponding

In words, equation (9) indicates that the erosion of A by B is the set of all points z such that B, translated by z, is considered in A [26]. Fig. 3. illustrates thinned character and its binary image.

cumulative moments of the histogram. µ T is the total mean level of the corresponding image. (a)

(b)

Fig. 3. (a) Binary character image. (b) Thinned character image.

(a)

(b)

III. FEATURE EXTRACTION AND MAPPING

(c) Fig. 2. (a) Scanned color signature image. (b) Gray Scale signature image. (c) Binary signature image which converted using Otsu’s Algorithm

B. Scale Correction The methodology to implement the scale invariance can be summarized as follows. 1) Check the input character height(Z0) and length(W0) with the standard values(X0,Y0) 2) If not matched, calculate the scaling factors.

ky =

y0 x and k x = 0 z0 w0

(5)

3) Apply the geometric scale transform T

[x

k x y 1] = [w z 1]T = [w z 1] 0  0 y = k y * z and x = k x * w

0 ky 0

0 0 1

(6)

The above spatial retransform is the one of the most commonly used forms of spatial transformations and called as affine transform for scaling [26]. C. Thinning Thinning means reducing binary objects or shapes in an image to strokes that are a single pixel wide. Thinning of a set A by a structuring element B, A ⊗ B can be defined in terms of hit and miss transform, A ∗ B

A ⊗ B = A − ( A ∗ B) = A ∩ ( A ∗ B)

C

(7)

The hit and miss transform can be defined as

A ∗ B = ( A − B1 ) ∩ ( A − B2 ) c

(8)

and the erosion can be defined as,

A − B = {z ( B) z ⊆ A}

(9)

The feature selection and extraction plays a major role in all pattern recognition systems. It is preferable to extract those features which will enable the system to correctly discriminate one class from the other. In general the features can be classified into two groups [13], [16], [28]: Global features which describe or identify the signature as a whole. This is due to any distortion of an isolated region of the signature will result in small changes to global features and less sensitive to inter personal variations. Local features represent the portion or a limited region of a signature. This may also be called as the grid features. Only the area in each region is utilized in order to form the grid information feature. The following feature combination of local and global features is selected for this study. The actual height, actual width, horizontal projection profile, vertical projection profile, Geometric centers, Contour profiles, Baseline shift and image area energize the global feature set. The normalized number of foreground pixels for each region, Geometric centers of regions and partial projection profiles are selected as the local or grid feature set. There are two main methodologies used to map the extracted features from reference signature with the training signature or with the testing signature. They are correlation based mapping and a normalized difference based method. The mapping method is selected according to the feature properties. If the feature is available as a profile, the correlation based method is feasible and if a feature is a kind of numerical value then normalized difference method is suitable. Let’s consider the general case of mapping two profiles P1 ( x) and P2 ( x) and their correlation coefficient, r can be calculated as in Eq. 10.

r=

∑ (P ( x) − P ( x))(P ( x) − P ( x))

∀ x

(

1

1

2

)

(10)

2

(

)

 2  2  ∑ P1 ( x) − P1 ( x)  ∑ P2 ( x) − P2 ( x)       ∀ x ∀ x

P1 ( x) and P2 ( x) are mean values of P1 ( x) and Therefore according to the feature type, P1 ( x ) and

P2 ( x ) .

P2 ( x)

n

d=

Ft − Fr1 + Ft − Fr 2 + .... + Ft − Frn nFm Ft

=

∑F

t

k =1

− Frk

(11)

nFm

represents the feature of the testing signature , Fr ...Frn

are the feature values of the reference sample and

Fm

represents a feature value which can be used in normalization and in this study, Fm consider same as Ft ( Fm = Ft ). This methodology can be used to map features such as geometric centre, area of the foreground signature, base line shift, and actual sizes.

of two fuzzy sets: Not Matched (NM) and Matched (M). Initially Gaussian membership functions are assumed. The rule base also assigned. V. FUZZY SYSTEM EVOLVING The fuzzy system is optimized using Genetic Algorithms. A sample training sample set ( S t ), including forgeries is used in the training. The required decision and resulting output (D) are compared using mean square difference method as the objective to GA [29] Then the evolved fuzzy system can be managed to recognize the signatures.

S

S

IV. FUZZY SYSTEM

Fˆr Preprocessing & Feature Extraction

Fˆt

Feature Mapping

may replace with horizontal or vertical projection, or with contour profile. The value based features are mapped using a normalized difference method and let’s consider a general case with reference sample containing n samples. Then the normalized difference ( d ), can be calculated as

Fc

The feature set consists of 15 feature inputs. Therefore the fuzzy system is divided in to several subsystems to simplify the design of the rule base and outputs of those systems are combined using an additional system. It is assumed that five feature sets are independent.

Feature set

TABLE I Feature set selection Features

1

Horizontal and vertical projection

2

Four contour Profiles

3

Geometric center and area

4

Actual width and height and Baseline shift

5

Local Features

According to Fig. 4., each feature set ( FS i ), is fed to a separate fuzzy system and the results, Rk of those systems are combined with a help of 2nd stage FIS. Then the outcome of 2nd stage fuzzy is considered as the final result.

FSi

Fuzzy Inference System

Rk

2nd stage Fuzzy Inference System

D, Decision

Fig. 4: Fuzzy System Structure

Consider first fuzzy subsystem as an example. It has two feature inputs based on horizontal projection and vertical projections which are followed by correlation based mapping. Therefore the input ranges of both variables are between 0 and 1. Each input consists of three fuzzy sets; Poor (P), Good (G), and Excellent (E) and output consist

Fuzzy Inference System (FIS)

D

MFs

GA based Optimizing

Fig. 5: Fuzzy inference system optimizing using GA.

According to the Fig. 5, first the reference sample and training sample of signatures are preprocessed and then the features are extracted. The feature set from the reference sample ( Fˆr ), and the feature set from the training sample ( Fˆt ), are mapped using a mapping technique. Then the resulting feature set ( Fc ), is fed to the fuzzy system. Then the GA is used to evolve the fuzzy system parameters such as membership function types and shapes by iterating up to satisfactory system. The next important consideration following the representation is the choice of fitness function. The genotype representation encodes the problem into a string while the fitness function measures the performance of the system. To find a good fitness measurement for a system is quite important for evolving practical systems using GA. Unlike traditional gradient based methods, GA’s can be used to evolve systems with any kind of fitness measurement functions including those that are non differentiable, discontinuous. Finding a good fitness measurement can make it easier for GA to evolve a useful system. How to define the fitness function for a system to be evolved is problem dependent. For prediction and

estimation problems, a commonly used function is a mean squire error or absolute difference error related function [29]. Therefore objective function can be defined as

(12)

Where N is the number of training samples and Oi and Ti are obtained and targeted outputs respectively.

N

M

0.8 Degree of membership

1 N ∑ (Oi − Ti )2 or N i =1 1 N F = ∑ Oi − Ti N i =1

F=

1

Threshold value

0.6 0.4 0.2 0 0

0.2

0.4

0.6

0.8

1

Decision VI.

RESULTS

Fig. 7: Threshold selection using trained membership function

The prominent two sections which lead to results are fuzzy system optimization and the signature recognition. The evolved membership functions of the first subsystem are shown in the Fig. 6.

E

G

P

1

0.5

0 0

0

1

HProj

(a)

Degree of membership

1

N

Degree of membership

Degree of membership

1

P

G

E

0.5

A database which consists of about 2000 signature samples from thirty different persons and their random forgeries and skilled forgeries obtained by the help of few forgers were used in the study. A bunch of three hundred signatures which consist of genuine signature and forgeries were used in a particular training session. The fuzzy system training process is achieved with fixed fuzzy rule base by allowing fuzzy membership function and it shape to evolve. The overall recognition summary is comprised into TABLE II and TABLE III. TABLE II Signature Recognition Summary Recognized Not Recognized

0 0

0.5

1

Total

Vproj

No.

1345

155

1500

(b)

(%)

89.67

10.33

100

M

0.8 0.6

The systems capability to handle the skilled forgeries and the response in the case of simple forgeries also analyzed and those results are illustrated in TABLE III. TABLE III Random and Skilled Forgery Handling in the System Recognized Not Recognized

0.4 0.2 0 0 0.2 0.4 0.6 0.8 1

Skilled Forgeries

Recognition

(c) Fig. 6 (a): Evolved input membership function that corresponding to horizontal projection, (b): Evolved input membership function that corresponding to vertical projection, and (c): Evolved output membership function of the first system

The decision, weather the two signatures that under consideration are from similar samples or a different sample are taken using a crisp threshold value. That value is extracted from the second stage trained output membership function and it is illustrated in the Fig. 7.

Random Forgeries

Total

No.

210

90

300

%

70

30

100

No.

231

69

300

%

77

23

100

VII.

CONCLUSION

In this paper we proposed an off-line signature recognition system using a fuzzy system, which is optimized using GA. It consists of major two phases; the fuzzy system evolution using Genetic Algorithms with a training signature set which consists of forgeries also and recognition using trained fuzzy system. The system results leads to following conclusions.

A combination of global and local features can enhance the identity of the feature set, because that combination can smartly handle the intra-personal and inter-personal signature variations. The Fuzzy system evolving enhanced the recognition capability. System achieved a signature recognition rate of about 90% and handled the random forgeries with 77 % accuracy and skilled forgeries with 70% accuracy. These values calculated with the constraint of the testing sample size. According to results the system is a competitive solution to the off-line signature recognition. The system performance highly depends on the fuzzy inference system capabilities and therefore relies on the fuzzy rule base also. Then by more accurate outcomes can be achieved by optimizing the rule base in addition to the membership function type and shapes. REFERENCES [1].

S. Prabhakar, S. Pankanti, and K. Jain, “Biometric recognition: Security & privacy concerns”, IEEE Security & Privacy Magazine, Vol. 1, pp. 33–42, March-April 2003. [2]. J. Soldeck, V. Shmerko, P. Philips, G. Kukharev, W. Rogers and S. Yanushkevich, “Image Analysis and Pattern Recognition in Biometric Technologies”, in proc. of Int. Conference on the Biometrics, USA, 1997, pp. 270-286. [3]. R. Plamondon and N. Srihari, “On-line and Off-line Handwritten Recognition: A Comprehensive Survey.”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, No. 1, pp. 63-84, January 2000. [4]. A. Zimmer, L. Ling, “A Hybrid On/Off Line Handwritten Signature Verification System”, in proc. of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003), Scotland, UK, 2003. [5]. R. Sabourin, G. Genest, and J. Preteux, “Off-Line Signature Verification by Local Granulometric Size Distributions”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No. 9, September 1997. [6]. R. Justino, F. Bortolozzi, and R. Sabourin , “Off-line Signature Verification using HMM for Random, Simple and Skilled Forgeries”, in proc. of IEEE Proceedings of the Sixth International Conference on Document Analysis and Recognition (ICDAR 2001), WA, USA, September 2001. [7]. B. Fang, C.H. Leungb, Y.Y. Tang, K.W. Tse, P.C.K. Kwok and Y.K. Wong, “Off-line signature verification by the tracking of feature and stroke positions” Pattern Recognition,Vol 36, pp.91101,2003. [8]. L. Lee, T. Berger, and E. Aviczer, “Reliable On-line Human Signature verification systems”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 6, June 1996. [9]. J. Yi, C. Lee, and J. Kim, “Online Signature Verification using Temporal Shift Estimated by the Phase of Gabor Flter”, IEEE Trans. on Signal Processing, Vol. 53, No. 2, February 2005. [10]. V. Nalwa, J. Holmdel, “Automatic on-line signature verification”, in Proc. of the IEEE, 1997. [11]. A. Chalechale and A. Mertins, “Line segment distribution of sketches for Persian signature recognition”, in proc. of Conference on Convergent Technologies for Asia-Pacific Region(TENCON), Bangalore, Vol. 1, Oct. 2003, pp. 11 – 15. [12]. A. Chalechale, G. Naghdy, P. Pramaratne, “Arabic/Persian cursive signature recognition and verification using line segment distribution”, in proc. of IEEE International Conference on Information and Communication Technologies: From Theory to Applications, Vol. 19, April 2004, pp. 475-476.

[13]. Y. Qi and R. Hunt, “Signature verification using global and grid features”, Pattern Recognition, Vol. 27, Iss. 12, pp. 1621-1629, December 1994. [14]. B. Fang, H. Leung, Y. Tang, W. Tse, K. Kwok, and K. Wong, “Off-line signature verification by the tracking of feature and stroke positions” , Pattern Recognition, Vol. 36, Iss. 1, pp. 91101, January 2003. [15]. L. Wan, Z. Lin, and R. Zhao, “Off-line signature verification incorporating the prior model”, in proc. of International Conference on Machine Learning and Cybernetics, Vol. 3, November 2003, pp. 1602 – 1606. [16]. H. Baltzakis and N. Papamarkos, “A new signature verification technique based on a two stage neural network classifier”, Engineering applications of Artificial Intelligence, Vol. 14, Iss. 1, pp.95-103, February 2001. [17]. P. Drouhard, R. Sabourin, and M. Godbout, “A neural network approach to off-line signature verification using directional PDF”, Pattern Recognition, Vol. 29, Iss. 3, pp. 415-424, March 1996. [18]. K. Huang and H. Yan, “Off-line signature verification based on geometric feature extraction and neural network classification”, Pattern Recognition, Vol. 30, Iss.1, pp. 9-17, January 1997. [19]. L. Yang, K. Widjaja, and R. Prasad, “Application of hidden Markov models for signature verification”, Pattern Recognition, Vol. 28, Iss. 2, pp. 161-170, February 1995. [20]. R. Justino, F. Bortolozzi, and Robert Sabourin, “A Comparison of SVM and HMM classifiers in the off-line signature verification”, Pattern Recognition Letters, Article in Press. [21]. R. Justino, F. Bortolozzi, R. Sabourin, “Off-line signature verification using HMM for random, simple and skilled forgeries”, in proc of Sixth International Conference on Document Analysis and Recognition, WA, USA, September 2001, pp.1031 – 1034. [22]. X. Xiao and G. Leedham ,”Signature verification using a modified Bayesian network”, Pattern Recognition, Vol. 35, Iss. 5, pp. 983995, May 2002. [23]. T. Kaewkongka, K. Chamnongthai, and B. Thipakorn, “Off-line signature recognition using parameterized Hough transform”, in proc. of IEEE Fifth International Symposium on Signal Processing and Its Applications, Vol. 1, August 1999, pp.451 – 454. [24]. K. Huang and H. Yan, “Off-line signature verification using structural feature correspondence”, Pattern Recognition, Vol. 35, Iss. 11, pp. 2467-2477, November 2002. [25]. A. Ismail and S. Gad, “Off-line arabic signature recognition and verification”, Pattern Recognition, Vol. 33, Iss. 10, pp.1727-1740, October 2000. [26]. R.C. Gonzalez, R.E. Woods, “Digital Image Processing”, Pearson Education (Singapore) Pte. Ltd, Delhi, 2003, ch6. [27]. N. Otsu, “A Threshold Selection Method from Gray-Level Histograms”, IEEE Trans. on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, 1979. [28]. E. Ramesh, M. Narasimha, and Murty, “Off-Line signature verification using genetically optimized weighted features”, Pattern recognition, Vol. 32, Iss. 2, pp. 217-233, February 1999. [29]. Y. Eberhart and R. Yaobin, “Implementation of evolutionary fuzzy systems”, IEEE Trans. on Fuzzy Systems, Vol. 7, Iss. 2, pp.109 119, April 1999. [30]. J. Yen and R. Langari, “Fuzzy Logic Intelligence, Control and Information”, Pearson Education (Singapore) Pte. Ltd, India, 2005.