Theoretical Statistical Correlation for Biometric Identification Performance

Transcription

1 Theoretical Statistical Correlation for Biometric Identification Performance Michael E. Schuckers Theoretical Statistical Correlation for Biometric Identification Performance p. 1/20

2 Overview Problem Statement: There is a need for appropriate statistical methods for estimation of the performance measures for biometric identification devices. Contribution: This paper lays out general correlation structure that allows for estimation, specifically standard errors, for FTE, FTA, FMR, FNMR. Theoretical Statistical Correlation for Biometric Identification Performance p. 2/20

3 Biometric Process and Notation Enrollment, (E i ) Theoretical Statistical Correlation for Biometric Identification Performance p. 3/20

4 Biometric Process and Notation Enrollment, (E i ) Acquisition, (A ij ) Theoretical Statistical Correlation for Biometric Identification Performance p. 3/20

5 Biometric Process and Notation Enrollment, (E i ) Acquisition, (A ij ) Feature Extraction, (S ik, X ik ) Theoretical Statistical Correlation for Biometric Identification Performance p. 3/20

6 Biometric Process and Notation Enrollment, (E i ) Acquisition, (A ij ) Feature Extraction, (S ik, X ik ) Matching, (Y ik,i k ) Theoretical Statistical Correlation for Biometric Identification Performance p. 3/20

7 Biometric Process and Notation Enrollment, (E i ) Acquisition, (A ij ) Feature Extraction, (S ik, X ik ) Matching, (Y ik,i k ) Decision, (D ik,i k ) Theoretical Statistical Correlation for Biometric Identification Performance p. 3/20

8 Failure to Enrol (FTE) Notation (1) E i = { 1 if individual i is unable to enroll 0 otherwise. Failure to enroll (FTE) rate is (2) FTE = i E E i N E. where N E is the total number of individuals who attempted to enrol in the system. Theoretical Statistical Correlation for Biometric Identification Performance p. 4/20

9 Failure to Acquire (FTA) Notation (3) A ij = 1 if the j th acquisition attempt by individual i is not acquired 0 otherwise. where a i is the total number of attempts for the i th individual A represents all individuals who attempt to have their biometric image collected. Failure to acquire (FTA) rate is (4) FTA = i A ai j=1 A ij i A a i Theoretical Statistical Correlation for Biometric Identification Performance p. 5/20

10 False Match Rate (FMR) Notation (5) D ii l = { 0 if i i,y ik,i k > τ 1 if i i,y ik,i k τ False match rate (FMR) is then (6) FMR = i i i i l D ii l i i n ii where n ii is the number of decisions on the ordered pair of individuals i and i. Theoretical Statistical Correlation for Biometric Identification Performance p. 6/20

11 False Non-Match Rate (FNMR) Notation (7) D ii l = { 1 if i = i,y ik,ik > τ 0 if i = i,y ik,ik τ False non-match rate (FNMR) is then i FNMR = l D iil (8), i n ii where n ii is the number of decisions on individual i. Theoretical Statistical Correlation for Biometric Identification Performance p. 7/20

12 Statistical Methods Estimated FTE, FTA, FMR, FNMR are all linear combinations of binary RV s Linear Combination R = v b vr v Expectation E[R ] = V v=1 a ve[r v ] Variance V [R ] = V V v=1 w=1 a va w Cov(R v,r w ) If variance is constant then V [R ] = σr 2 [ V v=1 a2 v + 2 V V v=1 w>v a va w Corr(R v,r w )]. Theoretical Statistical Correlation for Biometric Identification Performance p. 8/20

13 Assumptions For the rest of this paper, we assume that FTE, FTA, FMR, and FNMR Constant mean Constant variance/covariance/correlation Get simple right move to more complicated E s, A s, D s are binary (variance mean(1-mean)) Work from correlation Theoretical Statistical Correlation for Biometric Identification Performance p. 9/20

14 Binary Correlations Which of the following rows of binary sequences is correlated? A Theoretical Statistical Correlation for Biometric Identification Performance p. 10/20

15 Binary Correlations Which of the following rows of binary sequences is correlated? A B Theoretical Statistical Correlation for Biometric Identification Performance p. 10/20

16 Binary Correlations Which of the following rows of binary sequences is correlated? A B C Theoretical Statistical Correlation for Biometric Identification Performance p. 10/20

17 Binary Correlations Which of the following rows of binary sequences is correlated? A B C D Theoretical Statistical Correlation for Biometric Identification Performance p. 10/20

18 Binary Correlations Which of the following rows of binary sequences is correlated? A B C D Answer: None, all are binomial Corr = 0 Theoretical Statistical Correlation for Biometric Identification Performance p. 10/20

19 Binary Correlations Which of the following rows of binary sequences is correlated? A B C D Answer: None, all are binomial Corr = 0 Theoretical Statistical Correlation for Biometric Identification Performance p. 10/20

20 Correlation Structure: FTE Uncorrelated Between Individuals Structure { 1 if i = i Corr(E i,e i ) (9) = 0 otherwise. Then estimated variance is (10) V [ ˆ FTE] = FTE(1 ˆ N E ˆ FTE) Theoretical Statistical Correlation for Biometric Identification Performance p. 11/20

21 Correlation Structure: FTA Intra-individual Correlation (11) Corr(A ij,a i j ) = 1 if i = i,j = j ψ if i = i,j j 0 otherwise. Then estimated variance is (12) V [ ˆ FTA] = FTA(1 ˆ N 2 A ˆ FTA) [ N A + ψ n i=1 a i (a i 1) ] where N A = a i. Theoretical Statistical Correlation for Biometric Identification Performance p. 12/20

22 Correlation Structure: FNMR Intra-individual Correlation (13) Corr(D iil,d i i l ) = 1 if i = i,l = l ρ if i = i,l l 0 otherwise V [ FNMR] ˆ = (14) where N G = n ii. FNMR(1 ˆ N 2 G ˆ FNMR) [ N G + ρ n i=1 n ii (n ii 1) ] Theoretical Statistical Correlation for Biometric Identification Performance p. 13/20

23 Correlation Structure: FMR Shared elements of a pair correlation OR ALL HELL BREAKS LOOSE (15) Corr(D ikl, D i k l ) = 1 if i = i, k = k, l = l η if i = i, k = k, l l ω 1 if i = i, k k, i k,i k ω 2 if i i, k = k, i k,i k ω 3 if i = k, i k,i i, i k ω 3 if i = k, i k, i i, k k ξ 1 if i = k, k = i, i i, k k, l = l ξ 2 if i = k, k = i, i i, k k, l l 0 otherwise Theoretical Statistical Correlation for Biometric Identification Performance p. 14/20

24 Correlation Structure: FMR (Variance) V [ FNMR] ˆ = (16) where N I = i FNMR ˆ I (1 + ω 1 i=1 + ω 3 i=1 + ξ 1 i=1 k i n ik. k=1 k i k=1 k i k=1 k i N 2 I n ik n ik ˆ FNMR) k =1 k i,k k i =1 i i,i k n ki + ξ 2 n i=1 N I + η i=1 n ik k=1 k i + ω 2 i=1 n i i + k =1 k i,k k k=1 k i n ki (n ki 1) n ik (n ik 1) k=1 k i n kk n ik i =1 i i,i k n i k Theoretical Statistical Correlation for Biometric Identification Performance p. 15/20

25 Application: FMR from Ross and Jain (2003) Modality τ ˆ FMR ˆη ˆω 1 ˆω 2 ˆω 3 ˆξ1 ˆξ2 ˆV [ ˆπ I ] Hand *0.000 * FP * Face * * * τ here is the threshold * indicates truncated correlation at zero N I = Overdispersion here: 2.64, 1.25, 1.45 respectively relative to binomial Theoretical Statistical Correlation for Biometric Identification Performance p. 16/20

26 Application: FNMR from Ross and Jain (2003) Modality τ ˆ FNMR ˆρ ˆV [ˆπG ] Hand FP * Face τ here is the threshold * indicates truncated correlation at zero N G = 500 Overdispersion here: 1.16, 1.00, 1.04 respectively relative to binomial Theoretical Statistical Correlation for Biometric Identification Performance p. 17/20

27 Structure and Methods Metric Parametric model Non-parametric model FTE Binomial IID FTA Beta-binomial User-specific Bootstrap FNMR Beta-binomial User-specific Bootstrap FMR N/A N/A User-specific Bootstrap due to Poh et al 2007 FMR model generalization of implicit structure by Bickel Central Limit Theorems apply to all cases. Theoretical Statistical Correlation for Biometric Identification Performance p. 18/20

28 Discussion Theoretical Formulation of Correlation Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

29 Discussion Theoretical Formulation of Correlation All correlations are threshold dependent Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

30 Discussion Theoretical Formulation of Correlation All correlations are threshold dependent Basis for More Complicated Correlation Structures Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

31 Discussion Theoretical Formulation of Correlation All correlations are threshold dependent Basis for More Complicated Correlation Structures Appropriate Statistical CI s and Hypo Tests Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

32 Discussion Theoretical Formulation of Correlation All correlations are threshold dependent Basis for More Complicated Correlation Structures Appropriate Statistical CI s and Hypo Tests Parametric approach used for Sample Size Calc s Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

33 Discussion Theoretical Formulation of Correlation All correlations are threshold dependent Basis for More Complicated Correlation Structures Appropriate Statistical CI s and Hypo Tests Parametric approach used for Sample Size Calc s Needed for GLM s, Covariate approaches Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

34 Discussion Theoretical Formulation of Correlation All correlations are threshold dependent Basis for More Complicated Correlation Structures Appropriate Statistical CI s and Hypo Tests Parametric approach used for Sample Size Calc s Needed for GLM s, Covariate approaches Mean Transaction Time?? Theoretical Statistical Correlation for Biometric Identification Performance p. 19/20

35 Thank You Questions? Theoretical Statistical Correlation for Biometric Identification Performance p. 20/20