Introducing a New Multimodal Database from Twins’ Biometric Traits Hamid Behravan
Karim Faez
School of Computing University of Eastern Finland Joensuu, Finland Email:
[email protected]
Department of Electrical Engineering Amirkabir University of Technology Tehran, Iran Email:
[email protected]
Abstract—This paper presents a new multimodal database from twins’ biometric traits intended for twins and person authentication from multiple cues. The database consists of six unimodal biometric traits, namely two dimensional (2D) face images, fingerprints, offline handwritten texts, videos of moving faces, spectral and thermal face images. A total of 104 subjects corresponding to 52 pairs comprises the database from which 20 pairs are identical and the rest are fraternal twins. Besides biometric traits, personality traits and psychological characteristics of twins were also collected using two popular psychology questionnaires, Craig’s Locus of Control (LOC) and the Big Five. Additionally, we conduct an experiment to measure human capability in distinguishing between identical twins. The result shows that untrained humans could classify identical twins with 82% accuracy using facial information and 76% accuracy using writing styles.
I.
I NTRODUCTION
Biometrics as the science of measuring humans biological and behavioural traits for personal authentication has received considerable attention both from academics and governments in the last decade. Already several biometric systems have been proposed and their performance has been compared [1]. However, as these systems develop, they may face new biometrical challenges. As an example, face recognition is one of the most reliable and commonly used biometric systems in surveillance applications. Singh et al [2, 3] have shown that face recognition systems are very sensitive to facial plastic surgery and in fact, these systems are unable to meet an acceptable rate of identification at intentional and/or domestic facial alterations. The first step to compensate the vulnerability of biometric systems to newly posed challenges is to construct a database, which includes unfavourable artifices. One of the new biometrical challenges is to encounter with similar objects that their identity can hardly be recognized, such as confronting twins and multiples [4]. Security systems, which are not able to tackle with twins and multiple biometric traits may be liable to penetration [4, 5]. The term twins or multiples refers to two or multiple individuals grown up in the same uterus (womb) and usually but not necessarily born at the same time. In general, there are two types of twins, identical and fraternal. Identical twins share the same DNA and they are usually in the same gender. On the other hand, fraternal twins are most likely to have distinctive chromosomes and DNA. Due to unavailability of databases covering twins biometric
traits, few twins studies have been carried out in biometrics. Srihari [5] studied the similarities of fingerprints of twins. He implied that there is more similarity between twin fingers than in the case of two arbitrary fingers. Moreover, twins can be successfully discriminated using fingerprints. In another work, he showed that the discriminability of the handwriting of twins is a useful measurement in the individuality of handwritings and in fact, twins writing styles are distinctive to be identified [6]. Studies on twins palmprints [7], iris [8] and 2D face [9] are other works focused on twins biometric traits. In this work, we have developed a new multimodal biometric traits database from identical and fraternal twins. The traits collected are two dimensional face images, fingerprints samples, handwritten samples, videos of moving faces, spectral and thermal face images and psychological traits. Twins Multimodal Biometric Database covers new challenges in biometric systems and intends to measure performance of multimodal biometric systems under controlled acquisition conditions. To the best of our knowledge, the design of the database is the first organized attempt to collect multimodal biometric traits from identical and fraternal twins. The samples were collected from Iranian and Finnish twins. For 10 pairs, biometric traits of the parents are also included in the database. II.
RELATED WORKS
Due to increasing concerns in multimodal biometric recognition and their promising robustness against forgeries attacks, many of the recently developed biometric databases are multimodal. Some of the commonly used multimodal databases are •
BANCA [10] includes face and speech recordings of 208 subjects. It includes four European languages with various acquisition devices (2 cameras and 2 microphones). The recording scenarios are controlled, degraded and adverse over a period of three months. The database targets at testing multimodal identification tasks under two modalities in realistic situations.
•
MyIDEA [11] includes face, voice, fingerprints, signature, handwriting, palmprints and hand geometry traits. The database includes biometric samples of 104 subjects recorded over three different sessions. The general specifications of the database are direct compatibility with mono and multimodal databases,
usage of different quality of sensors, various realistic scenarios with contents inscribed in French and English languages and finally, imposter attempts for voice, signature and handwritten. •
•
BioSec [12] was acquired under the FP6 EU BioSec Integrated Project. It includes fingerprint samples captured with three different sensors, frontal face images from a webcam, iris images, and voice utterances of 250 subjects with 4 sessions per subject (about 1 month between each sessions). The database targets at studying the automatic test-independent speaker recognition applications with speeches recorded at 44 KHz stereo with 16 bits using both a headset and a distant webcam microphone. BIOSECURE [13] is the extended version of the BioSec and MyIDEA (developed by the same research group). It covers over 1000 subjects in two sessions. The database considers three acquisition scenarios, namely unsupervised Internet acquisition including voice and face (still images and talking faces), supervised office-like scenario including voice, finger prints (two sensors), face (still and talking faces), iris, signature (genuine and forgeries attacks) and hand, and finally acquisition in a mobile device including signature (genuine and forgeries attacks), fingerprints, voice, and face (images and video).
Multimodal biometric databases are not limited to those mentioned above. A summary of existing multimodal databases and their properties can be found in [14]. However, these databases mostly vary in size, scope and the extracted features; they do not cover some other new challenges such as studying the groups with common biometric traits as identical and fraternal twins. TABLE I demonstrates existing works which target at twins’ biometric traits. III.
TWINS MULTIMODAL BIOMETRIC DATABASE
The Biometric traits in our database are divided into two groups, of which 2D face images, videos of moving faces, fingerprints, spectral and thermal images are physiological biometrics, and handwritten samples are behavioral biometrics. We also collected relevant non-biometric information from each participant. It includes types of twins, age, gender and education of each subject. The other database specifications are 3
Target of 104 subjects including 52 pairs, of which 20 are identical and the rest are fraternal twins, between the ages 8 to 55 years old.
3
Biometric acquisition under several conditions such as variations in pose, illumination, expression and occlusion in face images, different contents in handwritten samples, two levels of pressure in fingerprints acquisition.
3
Organization of the traits into separate folders for each twin pairs to allow independent open-set experiments.
3
Repetitions in acquisition of the traits in order to consider imposter attacks in handwritten and fingerprints samples.
TABLE I.
A SUMMARY OF EXISTING TWINS BIOMETRIC DATABASES.
Biometrics Age range #Identical #Fraternal #Total Fingerprint [5] 3-87 255 42 297 Handwriting [6] 6-78 170 31 201 Palmprints [7] 6-45 53 53 Iris Texture [8] 84 9 93 Multi-biometric [4] 5-65 51 15 51 3D TEC [15] 107 107
3
Compatibility with other mono and multimodal biometric databases such as AR face database [16], BANCA and XM2VTSDB [17].
In database construction, we limited ourselves to controlled scenarios. We believe that what mostly matter are to design recognition systems capable of distinguishing between twins and consequently, reporting an acceptable rate of identification compared to humans identification capability. TABLE II summarizes the general properties of each trait in Twins Multimodal Biometric Database. A. Acquisition Protocol - Face Samples A total of 15 images was captured from each subject under different acquisition scenarios by a 1/2.3 (7.76mm) Super HAD CCD Sony DSC-W330 camera. We carefully monitored recording conditions (Camera Distance, Camera Parameters and lighting settings) to ensure that they remain identical across all subjects. Recording conditions are 1) 2) 3) 4)
Frontal faces with neutral and smile expression. Frontal faces with occlusion including wearing sunglasses and scarf. Head rotation with neutral expression including left, right, up and down poses. Illumination variations in head rotations.
The Images were captured at a resolution of 4320 by 3240 pixels and then rescaled to 512 by 512 pixels in JPEG format. Fig. 1 shows sample faces of two pairs of identical and fraternal twins. - Writing Styles The method used to extract twins writing styles is based on offline text identification methods [6]. The forms were designed to collect handwritten samples of each subject. Each form consists of three parts. On the top section, randomly selected texts from magazines are printed in separate lines. Then, each subject rewrites the contexts with his/her natural writing style in the blank space below each text. Writers demographic information including age, education, gender and type of twins were also appended to the designed box on the bottom of the forms. In order to have a baseline comparison between twin’s writing styles, the forms were divided into two different categories, in which one form is common between all twin pairs and the other is unique only to a specific pair. Moreover, each subject wrote twice from a specific form. In total, there are 416 handwritten samples available corresponding to 104 subjects. Fig. 2 shows similar and dissimilar writing styles from two different pairs.
TABLE II.
MAIN PROPERTIES OF TWINS MULTIMODAL BIOMETRIC DATABASE.
Modality
Conditions
Capture Device
Face
Frontal Expression Illumination Pose Occlusion
1/2.3 (7.76mm) Super HAD 512 * 512 JPEG CCD Sony DSCW330 camera
Fingerprints
Quality(Pressure level)
Handwritten
Video
Spectral Face
Thermal Face
Contents Repetition Video-Audio Recording Head Rotation Right and False Sentences Frontal Pose Occlusion Expression Frontal Pose Occlusion Expression
Resolution
No. of Samples
52*10
Fotronic’s FS80 Optical, CMOS
480*320 (500 dpi)BMP
52*10*2
Digital Scanning
1000 dpi optical
52*2*4
Laptop Integrated Webcam
DV Encoded AVI 25 frames/second
52*3
CRI Nuance EX LCTF camera
Wavelength range from 450 nm to 950 nm
3*10
Flir SC 7600 camera
wavelength from 1500 nm to 5100 nm
3*10
- Fingerprints Overall, two fingerprint samples (BMP format with no compression) of all 10 fingers at a resolution of 480*320 pixels (500 DPI) were captured using a digital live scanner. In the controlled scenario, the quality and centering of the fingers were monitored and the scanning area was cleaned between each acquisition. Fingerprints at this level correspond to high quality samples. The uncontrolled scenario performed without monitoring the quality and centering of the fingers. We asked participants to press their fingers with less level of pressure on the sensor. Fingerprints at this level correspond to low quality samples. In total, the ten fingerprints images of 104 subjects were captured. For ten particular pairs, the fingerprint samples of their parents were also captured. Fig. 3 shows ten rolled fingers of an individual. A side by side comparison between 5 rolled fingers of an identical pair is shown at Fig. 4.
Fig. 1.
to nine (Sentence (2)) and with a short pause, reading the randomly generated digits below (sentence (3)): Sentence (2): ”0 1 2 3 4 5 6 7 8 9” Sentence (3): ”3 7 5 0 1 4 2 9 6 7”
- Videos of Moving Faces The construction of video recordings is similar to the recordings of XM2VTS database [17], but the contents are in Persian. The audio-video shots were acquired with a laptop integrated webcam, while the subjects facing the camera to the center. All utterances are in Persian. We made a considerable attempt to hold camera settings and subject’s distance identical across all sessions. Totally, there are 399 video sequences available corresponding to three recorded consecutive sessions as follow
Identical (left) and fraternal (right) twins face samples.
3)
The video recordings end with head rotation. Each sequence consists of a subject moving his/her head from the center to the right, to the left, then up to down and at each step, returning to the center.
All video samples are stored in DV encoded AVI file format. - Spectral and Thermal Face Images
1)
In the first session, each subject is asked to utter the sentence 1, while motion video (with audio) is captured. Each sequence is approximately 6 seconds long and it contains the subject speaking the sentence (1): Sentence (1): ”Everybody should respect laws and obey it”
The spectral face images were captured by CRI Nuance EX LCTF spectral camera. The spectral range is from 450nm to 950nm with a bandwidth length of 20nm, producing 26 bands in all. Similar to face samples, spectral images have pose, frontal, expression, illumination and occlusion variations. Fig. 5 shows a one by one comparison between fraternal twins’s spectral images at three wavelengths.
2)
This session contains two video sequences (with audio). The subject is asked to read digits from zero
The thermal face images were captured by Flir SC 7600 thermal camera at operating wavelength from 1500 nm to 5100
(a)
Fig. 3. 10 rolled fingerprints of an individual. (a) right little, (b) right ring, (c) right middle, (d) right index, (e) right thumb, (f) left little, (g) left ring, (h) left middle, (i) left index, (j) left thumb. (b)
Fig. 2.
Twins writing styles. (a) Similar style, (b) Dissimilar style.
with the thermal accuracy 0.05 degree Celsius. The scenarios considered are similar to spectral images except illumination is not included. Fig. 6 shows thermal image samples from fraternal twins. At the time of writing this paper, there are spectral and thermal images of three pairs available in the database, making available 474 spectral and 30 thermal samples. However, we are planning to include samples of more pairs in the future version of the database. - Behavioural and Psychological traits Besides biometric data collection, we asked participants to complete two standard validated personality questionnaires, the Persian version of Craig’s Locus of Control (LOC) of Behaviour scale [18] and the part of Big Five personality traits related to extraversion dimension of personality [19]. LOCs questionnaire includes 17 items with a ranging scale from 0 to 5 points. It measures the extent to which individuals believe that they have control over their lifes events. LOC consists of a stable set of beliefs about whether the outcome of ones actions is dependent on what the subject does (internal orientation) or on events outside of his/her control (external orientation). The Big Five consists of 10 items with a ranging scale from 1 to 7. It is a multi-dimensional model of personality, which corresponds to five behavioural factors namely extraversion, agreeableness, openness to experience, conscientiousness and emotional stability.
Fig. 4. Comparison between fingerprints of identical twins. From left to right, right little, right ring, right middle, right index, right thumb. (a) first twin, (b) second twin.
B. Acquisition Validation Process and Acquisition Guidelines Due to the importance of biometric traits and the privacy of personal information, acquisition process started only once the subject fully agreed the aims of the research. Moreover, we gave each subject the right to change or delete his/her personal data. To protect the privacy of the information, we did not include the subjects real names in the database construction. During the acquisition procedure, a human supervisor gave the necessary instruction to participants. A data validation process was also carried out to check which biometric traits should be accepted or rejected.
the number of correct responses by 40 (the total number of face pairs shown on the screen). The percentage of times the pairs were given correct in this study labels is 81.2 %. B. Writing Styles Study
Fig. 5. Spectral face samples of a fraternal pair at (a) 450 nm, (b) 750 nm, and (c) 950 nm wavelengths.
In order to have a baseline comparison, we used the same test cases as face study. We selected 20 pairs of handwritten samples, so that in half of them both twin wrote the same content, and in another half the contents were different. The results indicated that humans show 76% accuracy to decide on either the handwritten samples come from a single person or a pair. Our primary goal in this experiment was to answer the question ”can humans differentiate between identical twins?” However, the results have a little implication in automatic biometric recognition. We believe that in case of twins authentication, human performances are the only reference, which other algorithms might use to compare their system performance. V.
Fig. 6.
Frontal and pose thermal face samples of a pair of fraternal twins.
We considered the following rules in data validation process •
In video files, audio and video should be synchronized. Background noise and lighting conditions should also remain identical across all pairs.
•
Low quality images in fingerprint samples are not removed unless the visibility of the trait is very poor.
•
In face images acquisition, illumination variations between pairs are not allowed. Poses should vary identical across twin of the same pair.
•
In Spectral and thermal images acquisition, cameras are identically calibrated for all pairs. IV.
EXPERIMENTS
We additionally conducted a preliminary experiment to measure human classification capability on two types of biometrics, face as a physiological biometric and handwritten as a behavioural biometric. This is done by asking whether the biometric samples belong to a single individual or to a pair. A graphical user interface was designed to present the samples on the screen. Two testers independently carried out the classification. When the decisions were not identical, a third tester determined the final classification. A. Face Study We randomly selected 20 pairs of identical twins from our list and used them as ’Twins’ group. To create the ’Same Person’ group, 20 individuals were also selected from pairs and their identical (not the same) face images were paired to create this group. The analysis consists of determining how often humans are able to correctly classify each pairs of face images into two predefined groups. To find out the overall accuracy, we divided
CONCLUSION
In this contribution, we have developed a new multimodal database for the analysis of authentication methods on twinss multimodal biometric traits. The traits collected are two dimensional face images, fingerprint samples, offline handwritten texts, videos of moving faces, spectral and thermal face images from 104 subjects corresponding to 52 pairs of twins. Besides biometric data acquisitions, personality traits of each subject were also collected by using two well-known psychological questionnaires, the Big Five and the Locus of Control. A preliminary experiment showed that humans are generally able to discriminate between identical twins by reporting 81.2% classification accuracy on facial information and 76% accuracy on writing styles. ACKNOWLEDGMENT The authors would like to thank Iranian twins community for providing supports in database collection. Many thanks to Prof. Markku Hauta-Kasari and Dr. Jouni Hiltunen who helped in collecting the spectral and thermal face images and to every pair who participated to the sessions. Database terms of use and release will be provided by contacting the authors. R EFERENCES [1]
[2]
[3]
[4]
G. P. J. Phillips, R. M. McCabe and R. Chellappa, ”Biometric image processing and recognition”, In Proceedings of the European Signal Processing Conference, 1998. R. Singh, M. Vatsa, H.S. Bhatt, S. Bharadwaj, A. Noore and S.S. Nooreyezdan, ”Plastic Surgery: A New Dimension to Face Recognition”, In IEEE Transaction on Information Forensics and Security, Vol. 5, No. 3, pp. 441-448, 2010. R. Singh, M. Vatsa and A. Noore, ”Effect of Plastic Surgery on Face Recognition: A Preliminary Study”, In Proceedings of IEEE Computer Society Workshop on Biometrics at Computer Vision and Pattern Recognition Conference, pp. 72-77, 2009. Z. Sun, A. A. Paulino, J. Feng, Z. Chai, T. Tan and A. K. Jain, ”A study of multibiometric traits of identical twins”, In Proceedings of SPIE, Biometric Technology for Human Identication VII, vol. 7667, Orlando, FL, USA, pp. 1-12, 2010.
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15] [16] [17]
[18] [19]
S. N. Srihari, H. Srinivasan and G. Fang, ”Discriminability of fingerprints of twins”, Journal of Forensic Identification, Volume: 58, Issue: 1, pp. 109-127, 2008. S. N. Srihari, C. Huang and H. Srinivasan, ”On the Discriminability of the Handwriting of Twins”, Journal of Forensic Sciences, 53(2), pp. 430-446, 2008. A. W. Kong, D. Zhang and G. Lu, ”A Study of identical twins’ palmprints for personal authentication”, In Proceedings of ICB, pp. 668674, 2006. K. Hollingsworth, K. W. Bowyer and P. J. Flynn, ”Similarity of iris texture between identical twins”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 22-29, 2010. P. J. Phillips, P. J. Flynn, K. W. Bowyer, R. W. Vorder Bruegge, P. J. Grother, G. W. Quinn and M. Pruitt, ”Distinguishing Identical Twins by Face Recognition”, The Ninth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011), 2011. E. Bailly-Bailliere, S. Bengio, F. Bimbot, M. Hamouz, J. Kittler, J. Mariethoz, J. Matas, K. Messer, V. Popovici, F. Poree, B. Ruiz and J. Thiran, ”The BANCA Database and Evaluation Protocol”, In Proceeding of Third International Conference on Audio and VideoBased Biometric Person Authentication, pp. 625-638, 2003. B. Dumas, C. Pugin, J. Hennebert, D. Petrovska-Delacretaz, A. Humm, F. Evequoz, R. Ingold and D. Von Rotz, ”MyIdea Multimodal Biometrics Database, Description of Acquisition Protocols”, In Proceedings of COST-275 Workshop Biometrics on the Internet, 2005. J. Fierrez, J. Ortega-Garcia, D. Torre-Toledano and J. GonzalezRodriguez, ”BioSec Baseline Corpus: A Multimodal Biometric Database”, Pattern Recognition, vol. 40, no. 4, pp. 1389-1392, 2007. F. Alonso-Fernandez, J. Fierrez, D. Ramos and J. Ortega-Garcia, ”BIOSECURE: A Multimodal Biometric Database”, In Proceeding of SPIE, 2008. M. Faundez-Zanuy, J. Fierrez-Aguilar, J. Ortega-Garcia and J. Gonzalez-Rodriguez, ”Multimodal biometric databases: An overview”, IEEE Aerospace and Electronic Systems Magazine, Vol. 21, n. 8, pp. 29-37, 2006. V. Vijayan, K.W. Bowyer and P.J. Flynn. ”3D twins and expression challenge”, In ICCV Workshops, pp. 2100-2105, 2011. A. R. Martinez and R. Benavente, ”The AR face database”, Technical Report 24, Computer Vision. K. Messer, J. Matas, J. Kittler, J. Luettin and G. Maitre, ”XM2VTSDB: The Extended M2VTS Database”, In Proceedings of the Second International Conference on Audio and Video-Based Biometric Person Authentication, 1999. J.B. Rotter, ”Generalized Expectancies for Internal versus External Control of Reinforcement”, Psychological Monographs, 1965. O.P. John and S. Srivastava,”The Big five trait taxonomy: History, measurement and theoretical perspectives”, Handbook of personality theory and research, Guilford Press, New York, 1999.
