Biometric Visions

Biometric System Performance

A biometric system can provide the following two functions:

  • Verification Authenicates its users in conjunction with a smart card, username or ID number. The biometric template captured is compared with that stored against the registered user either on a smart card or database for verification.
  • Identification Authenticates its users from the biometric characteristic alone without the use of smart cards, usernames or ID numbers. The biometric template is compared to all records within the database and a closest match score is returned. The closest match within the allowed threshold is deemed the individual and authenticated.

Measurement

  • false accept rate (FAR) or false match rate (FMR): the probability that the system incorrectly declares a successful match between the input pattern and a non-matching pattern in the database. It measures the percent of invalid matches. These systems are critical since they are commonly used to forbid certain actions by disallowed people.
  • false reject rate (FRR) or false non-match rate (FNMR): the probability that the system incorrectly declares failure of match between the input pattern and the matching template in the database. It measures the percent of valid inputs being rejected.
  • receiver (or relative) operating characteristic (ROC): In general, the matching algorithm performs a decision using some parameters (e.g. a threshold). In biometric systems the FAR and FRR can typically be traded off against each other by changing those parameters. The ROC plot is obtained by graphing the values of FAR and FRR, changing the variables implicitly. A common variation is the Detection error trade-off (DET), which is obtained using normal deviate scales on both axes. This more linear graph illuminates the differences for higher performances (rarer errors).
  • equal error rate (EER): the rate at which both accept and reject errors are equal. ROC or DET plotting is used because how FAR and FRR can be changed, is shown clearly. When quick comparison of two systems is required, the ERR is commonly used. Obtained from the ROC plot by taking the point where FAR and FRR have the same value. The lower the EER, the more accurate the system is considered to be.
  • failure to enroll rate (FTE or FER): the percentage of data input is considered invalid and fails to input into the system. Failure to enroll happens when the data obtained by the sensor are considered invalid or of poor quality.
  • failure to capture rate (FTC): Within automatic systems, the probability that the system fails to detect a biometric characteristic when presented correctly.
  • template capacity: the maximum number of sets of data which can be input in to the system.

Data

The following table shows the state of art of some biometric systems:

Biometric Performance

One simple but artificial way to judge a system is by EER, but not all the authors provided it. Moreover, there are two particular values of FAR and FRR to show how one parameter can change depending on the other. For fingerprint there are two different results, the one from 2003 is older but it was performed on a huge set of people, while in 2004 far fewer people were involved but stricter conditions have been applied. For iris, both references belong to the same year, but one was performed on more people, the other one is the result of a competition between several universities so, even if the sample is much smaller, it could reflect better the state of art of the field.

 

References

  1.  P. J. Philips, P. Grother, R. J. Micheals, D. M. Blackburn, E. Tabassi, and J. M. Bone, Face Recognition Vendor Test 2002: Overview and Summary (Online)
  2. C. Wilson, A. R. Hicklin, H. Korves, B. Ulery, M. Zoepfl, M. Bone, P. Grother, R. J. Micheals, S. Otto, and C. Watson, Fingerprint vendor technology evaluation 2003: summary of results and analysis report, NIST Internal Rep. 7123, Jun. 2004 [Online]
  3. R. Cappelli, D. Maio, D. Maltoni, J. L. Wayman, and A. K. Jain, Performance evaluation of fingerprint verification systems, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 1, pp. 3–18, Jan. 2006
  4. E. Kukula, S. Elliott, Implementation of Hand Geometry at Purdue University's Recreational Center: An Analysis of User Perspectives and System Performance, IEEE 2005
  5. International Biometric Group, Independent Testing of Iris Recognition Technology, May 2005 (Online)
  6. NIST Iris Challenge Evaluation, (Online)
  7. S. Hocquet, J. Ramel, H. Cardot, Fusion of Methods for Keystroke Dynamic Authentication, Automatic Identification Advanced Technologies, 2005. Fourth IEEE Workshop on 17-18 Oct. 2005 Page(s):224 - 229
  8. D. A. Reynolds, W. Campbell, T. Gleason, C. Quillen, D. Sturim, P. Torres-Carrasquillo, and A. Adami, The 2004 MIT Lincoln laboratory speaker recognition system, in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, Philadelphia, PA, Mar. 2005