This study was initiated by the Federal Criminal. Investigation Office of. Germany (BKA) in close cooperation with the German Information Security. Agency (BSI).
BioIS Study Comparative Study of Biometric Identification Systems A.Zwiesele,
BKA Wiesbadenl
-
A.Munde,
BSI B o n n '
Dr. C.Busch, H.Daum, IGD Darmstadt3
Abstract Dupability: The aim of this part is to analyse and assess the effort that is necessary to dupe biometric systems. It not only covers the systems taking part in the study, but also examines their respective functional principles independently of their technical implementation. Influence of the various programmable system parameters: This part attempts to investigate the repercussions of the various system setups for the identification attributes. The findings are intended to permit recommendations to be made regarding the prefered settings for each of the biometric systems under investigation. Influence of the various environmental factors on the identification reliability of the systems: The purpose of this part is to determine the repercussions of changes in environmental conditions for the identification attributes. One example of such factors might be the way in which different lighting conditions affect the systems' ability to recognise faces.
On 1'' April 1999, after a preparatory phase lasting more than twelve months, work on the a.m. BioIS Study finally commenced. This study was initiated by the Federal Criminal Investigation Office of. Germany (BKA) in close cooperation with the German Information Security Agency (BSI). The study was executed by the Fraunhofer Institute of Graphical Data Processing (IGD). The study includes a field investigation, in which 11 physiological (static) and behaviour-specific (dynamic) systems, which were available and supported in Germany, were installed and put into operation in a defined scenario. The field investigation was conducted with approximately 40 users representing different age, employment, educational and ethnic groups. The main objectives of the field investigation are as follows: 1.)
2.)
3.)
To gather experience with the biometric systems and to identify any weaknesses that need to be examined in greater depth during the future course of the study. To obtain statistical information regarding the frequency with which authorised users are rejected by the various systems. This information will then be taken as a basis for establishing the existence of certain user groups which individual systems have difficulties in identifying. In the event that such groups do exist, the possible reasons for their rejection need to be examined. To observe the behaviour of the users over a prolonged period of time, in order to establish whether or not any changes can be observed. There might, for instance, be a certain familiarisation effect, which is reflected in a change in the rejection rate.
The study was completed on the 1 5 of ~ May 2000. It is the aim of this lecture to inform the
audience of the results of the study and the knowledge which could be gained.
Introduction ,,In comparison to PINS and passwords, a biometric signature has crucial advantages and provides an unambiguous proof of identity..." ,,Comprehensive empirical tests are being conducted to get rid of the last doubts and insecurities from the angle of consumer and data protection..." ,,Widespread employment of biometric systems just around the comer..."
The field investigation is to be followed by a further technical study phase, designed to investigate the following points:
...that is what the manufacturers are promising, but as a study by the Federal Criminal Investigation
Federal Criminal Investigation Office of Germany German Information Security Agency Fraunhofer Institute of Graphical Data Processing
0-7803-5965-8/00/$10.00 02000 IEEE
60
Verification mode systems
Office of Germany (BKA) and of the German Information Security Agency (BSI) shows, those systems are not as reliable as they should be. The Comparative Study of Biometric Identification Systems - BioIS Study - commenced on 1'' April 1999 and was completed on the 15" of May 2000. The study was executed by the Fraunhofer Institute of Graphical Data Processing (IGD) at the location of Darmstadt.
1. Face Recognition System 2. Signature Verification System 3. Signature Verification System 4. Fingerprint-System 5 . Iris Recognition System 6 . Fingerprint-System, integrated in keyboard Identification mode systems
Selection of systems 7. Face and Voice Recognition System 8. Face and Voice Recognition System 9. Face Recognition System 10. Hand Geometry System 11. Fingerprint-System
The term ,,biometric identification system" is used to refer to a system that enables people to be identified or authenticated on the basis of one or more biometric characteristics. These characteristics - or attributes - may be either behaviour-specific (dynamic) or physiological (static). Behaviour-specific attributes include a person's signature or voice, or the rhythm in which they actuate the keys on a keyboard. Among a person's physiological attributes are their fingerprints, their face, the fundus of their eyes, etc.
(Due to agreement of confidence manufacturers must not be mentioned by name)
Examination of identification accuracy To evaluate the identification accuracy of biometric identification systems, we need adequate methods to find out the False Rejection Rate (FRR)and the False Acceptance Rate (FAR) of these systems.
In order for a biometric attribute to be hndamentally suitable for use by an automatic authorisation system, it must fulfill the following minimum requirements:
,'
FRR is the rate of rejection of an entitled person by a biometric system. FAR is the rate of acceptance of a person, entitled or not, if he or she was identified as another entitled person by a biometric system. The FAR can't occur in relation with verification mode systems if we assume that no attempt is made to fool a biometric system.
uniqueness, i.e. it must be possible to derive the identity of the person concerned from this attribute. temporal stability, i.e. this attribute must not change (significantly) over an extended period of time. widespread propagation, i.e. all persons who need to be identified must share this attribute. acceptance, i.e. the use of the attribute must be accepted by the system's users.
As an adequate method, the study began with a Field-Investigation, in which the various systems were installed and put into operation in a defined scenario. The field investigation was conducted with approximately 40 users representing different age, employment, educational and ethnic groups for half a year.
Besides the biometric attribute a biometric system is focussing on, the systems vary by their mode of operation:
Verification mode, the system has to compare the biometric attribute of a person who offers a declared identity with the stored biometric sample (template) of that identity. (1 : 1 comparison) Identification mode, the system has to compare the biometric attribute of a person with all stored biometric samples (templates). (1 : n comparison)
Technical preparation of Field-lnvestigation
-
In order to examine a representative profile of biometric identification systems the following 11 physiological (static) and behaviour-specific (dynamic) systems, which were available and supported in Germany, were selected to be tested within the BioIS Study:
installationand configuration of systems by the manufacturer or following the instructions of manufacturer Arrangement of systems in a course
61
the fourth and fith attempt no more remarkable improvement is achieved. For that reason we should consider to reduce, in future studies, the number of five allowed attempts to a total of three.
Proceeding of Field-Investigation Introductory meeting - participation voluntary data protection Enrollment of users by administrator Daily visit and autonomous use of systems by the participants Documentation of results on daily replaced logsheets Electronic recording of results Analysis
-
For the Identification mode systems also the False Acceptance Rate was determined. The FAR is an important factor for assessment whether a biometric system is suitable for safety-related applications or not:
Identification mode systems
During their daily visits the participants tried to be identified by the systems. If they failed, they could try up to four more times in a row. If they were identified within a total of five attempts or less, this was quoted as one successful trial. Otherwise the trial was quoted as a false rejection. The following two tables show the False Rejection Rates of the Verification and Identification mode systems. After all, the value FRR(5) is the essential one to make out the accuracy of identification:
8. Face8VoiceRecgniion Sydem
1 17 1 1 1 0
I
IO. HandGeomeby System
11. FingerprintSystem
1. Face Recugnm'on
1856 16,05
7,16
2. SiatureVerificaM n System
1588 8 7 9
78,27
3.SgnatureVeIifkaton System
1721
4.Fingerpn'Mystem
1942 2L97
710 11.191
I 1 1I 11
I
9. FaceRemgnitionSystem 1834
Verificationmode systems
0
I
I
I
I 1
114 159
I
I
,
760 I41,44
44
2,18
88
437
130
6,48
115
5,73
Results
system
As shown by the tables, 5 of 11 systems finished the field study with very bad results. The systems 7. and 8. with very good FARs disappointed by FRRs of almost 50 %. To improve the FRR one might think of changing the system parameters to more tolerant values but the systems were already run with optimum configurations, adjusted by the manufacturer for a second time after initial results of the field study.
34,75 24,87 15,09
Another problem had been a little group of so called ,,non-users". 2 of 40 persons could not be identified by the fingerprint-systems due to physiological reasons. Another person with strong eyeglasses had problems to be identified by the iris recognition system. A woman who didn't clip her fingernails could not be identified anymore by the hand geometry system after some weeks... Therefore we have to consider the hypothesis that the perfect biometric system does not exist and that there will always be the need of a reserve-system.
ldentiiication mode systems System
Trials FRR(1) FRR(2) FRR(3) FRR(4) FRR(5) I%] I%l IS1 1%1 [%I
a. Face8Voice
1537 7 1 ~ 2 59,86
52,83 49,19
46,19
RecognitionSystem
How would the systems cope with efforts to fool them by intention ?
Examination of safety After the field study the aim of a second approach of the BioIS Study was to analyse and assess the effort that is necessary to fool biometric systems to get access as a non-entitled person.
As can be seen, much better identification accuracy was achieved by the second and third attempt. By 62
Potential points of attack to fool a biometric system are as follows:
Group c) System No.2 could not be fooled (but has a FRR of 65,43 % anyway). System No.3 could be fooled in some cases when simple signatures were imitated.
Front of system (sensor) Fooling the sensor (camera, fingerprint-scanner etc.) by using a copied, falsified or forged biometric attribute or by using a biometric attribute similar to the original one.
Group d)
Data link between sensor and data processing unit Monitoring the signal offers two methods of attack: a) Recording and replaying the signal into the data link (replay-attack) b) Reworking of the recorded signal (video, audio, printout) and reuse for sensor
System No.10 uses a video-signal to transfer the palm-image to the data processing unit. Therefore the system could be fooled like the audio-visualsystems by a replay attack. Results If the signature-system No.2 (FRR 65,43 %) is not taken into account, 9 of 10 biometric systems could be fooled by more or less simple measures.
Data link between data processing unit and other units Hacking into the system will offer the possibility of copying or manipulating stored templates of entitled biometric attributes.
To record and to replay the video-signals a standard video-tape recorder was used. The india-rubber fingerprint-stamps were made of materials which are easily available in handicraft shops.
In this study only points 1. and 2. were examined because point 3. was not quoted as a specialized biometry-related attack.
Conclusion and outlook
The BioIS Study clearly showed, that with the exception of one system (by the way the most expensive one) none of the tested systems is suitable to be used for safety-related applications.
Proceeding of safety examination The 11 biometric systems were divided into 4 groups: a) audio-visual-systems (No. 1,5,7,8,9) b) fingerprint-systems (No.4,6,11) c) signature-systems (No.2,3) d) hand geometry system (No.10)
But some of the security-leaks could easily be remedied by the manufacturers. The tested systems are the standard of one year ago and the development of biometric systems goes on.
Group a) Sytems which are less suitable to be used for safety-related applications may still do a good job in other domains.
System No.9 was fooled by printouts of templates of entitled persons (colour and black and white) and by the colour-printout of a digital camera which was placed beside the system camera by the offender to take photographs of entitled persons.
Therefore, as a result of the BioIS Study, we have started a new project to create technical procedures for testing and classifying biometric systems. The aim is to create categories for biometric systems to give users a hand to decide, what biometric system to use for what kind of application.
Systems N0.7~8and 1 were fooled by recording and replaying the video-signal of an entitled person into the data link between camera and data processing unit. The audio-signal (No.7 and 8) was not recorded but spoken by the offender. It was not necessary to synchronize the audio and video-signal. System No.5 (Iris Recognition System) could not be fooled. Group b) All systems were fooled by fingerprint-stamps, copied from entitled persons and made of indiarubber.
63
