Hybrid Fusion for Biometrics: Combining Score-level and Decision-level Fusion

Transcription

1 Hybri Fusion for Biometrics: Combining Score-level an Decision-level Fusion Qian Tao Raymon Velhuis Signals an Systems Group, University of Twente Postbus 217, 7500AE Enschee, the Netherlans Abstract A general framework of fusion at ecision level, which works on ROCs instea of matching scores, is investigate. Uner this framework, we further propose a hybri fusion metho, which combines the score-level an ecision-level fusions, taking avantage of both fusion moes. The hybri fusion aaptively tunes itself between the two levels of fusion, an improves the final performance over the original two levels. The propose hybri fusion is simple an effective for combining ifferent biometrics. 1. Introuction Biometrics, which uses a variety of physical or behavioral characteristics to verify a person s ientity, is wiely use in a lot of security applications. To overcome the limitation of a single biometrics, information from multiple biometrics can be integrate to achieve more reliable an robust performance. For this purpose, fusion of iverse biometrics has been extensively stuie in recent years. For a etaile review, see [10]. Accoring to the ifferent stages of a biometric system, fusion can be one at four istinct levels, namely: sensor (raw biometric ata) level, feature level, matching score level, an ecision level. Along these levels the biometric information is graually extracte an reuce. On the first two stages, the information content is rich, but in most cases noisy an reunant. On the matching score level, the information is reuce into a single quantity, inicating the likelihoo that the biometric ata belongs to a certain class. On the final ecision level, the information is further reuce to the iscrete class labels. In this paper, we will concentrate on the last two levels of fusion, not only because of the simplicity, but also because of the possibility to buil up a general fusion framework, without taking into account the specific type of biometric ata processing an classification methos, which woul closely influence the first two levels of fusion. Fusion at matching score level is the most popular way of fusion, offering the best traeoff between information content an fusion complexity [6, 15, 11, 9, 10]. Fusion at ecision level, in comparison, is less stuie, as it is often consiere inferior to matching score level fusion, on the basis that ecisions have less information content than the matching scores. Actually, the combination of the two ecisions using AND an OR rule often has the risk of egraing the overall performance when the performance of component classifiers are significantly ifferent [3]. A optimal ecision fusion metho by the AND an OR rule has been propose in literature [12]. In this metho, the fusion at ecision level is one in an optimal way such that it always gives an improvement in terms of error rates over the classifiers that are fuse. Here optimal is taken in the Neyman-Pearson sense [14]: at a given FAR (false accept rate) α, the ecision-fuse classifier has a FRR (false reject rate) β that is minimal, an never larger than the β of the component classifiers at the same α. Besies, the metho has the avantage that in presence of outliers (i.e. the biometric ata which belongs to the genuine user but eviate from the moele istributions, possibly cause by the variability of collection conitions), the OR rule ecision fusion can achieve a low FAR with little risk of increasing FRR [12]. In this paper, we will exten this work, constructing a more general framework of ecision fusion oriente on performance, an propose a hybri fusion scheme which combines the score-level an ecision-level fusion. Instea of ealing with the matching scores, the fusion framework works irectly on the ROC (receiver operation characteristic). Although the ROC is erive from the matching scores, the problem is still mae ifferent: the matching scores are converte into a compact set of operations points, which convey the istribution information of matching scores in an inirect way. The optimization in the framework only involves those operation points, without reference to the matching scores. Uner this framework, any two (or more) ROCs can be fuse together for improve performance. Those ROCs coul characterize any biometric system, either of a single biometric, or of a alreay fuse multi-biometrics. This en /08/$ IEEE

2 ables us to o fusion in an hybri manner, combining scorelevel an ecision-level fusion an taking avantage of both fusion moes. The paper is organize as follows. Section 2 reviews the ecision-level fusion framework. Section 3 introuces the hybri fusion. Section 4 shows the experimental results. Finally, Section 5 gives conclusions. 2. A Decision-level Fusion Framework Each biometric system can be characterize by a ROC, i.e., the etection rate p (p = 1 β) as a function of false accept rate α, enote by p (α). The ROC is obtaine by varying the threshol that iscriminates the genuine an impostor matching scores, thus proucing ifferent etection rate p an false accept rate α. Each point on the ROC, a specific pair of (α, p ), is calle an operation point, corresponing to a particular threshol t of the matching scores. In this section we will show how multiple ROCs can be fuse together simply by AND an OR rule for improve performance. When the optimal operation points on ROC are obtaine, the threshols of matching scores are obtaine as well. Suppose we have N inepenent biometric systems, each characterize by its ROC, p,i (α i ), i = 1,..., N. The inepenency assumption is realistic in practice as fusion is often one between ifferent biometric moalities. Besies, the inepenency assumption in this section makes the formulations much simpler an clearer. The epenent cases, however, will be iscusse in Section 3.2. If the AND rule is use for fusion, the final performance can be estimate, uner the inepenent assumption, as α = α i, p (α) = p,i (α i ) (1) with α the false-accept rate an p the etection rate of the AND rule fuse ecision, respectively. In search of the optimal operation points, the fusion framework by AND rule can be formulate as ˆp (α) = max α i αi=α { } p,i (α i ) which means that the resulting etection rate ˆp at α is the maximal value of the prouct of the etection rates at a certain optimal combination of α i, i = 1,..., N, which satisfy α i = α. In other wors, at a prefixe α, the optimal operation points of the component ROCs are obtaine by optimizing (2). Consequently, the threshols of component biometric systems can be reaily obtaine as the ones corresponing to the optimize operation points. (2) Likewise, if we efine the reject rate for the impostors p r,i = 1 α i, the fusion framework by OR rule can be similarly formulate { N } ˆp r (β) = p r,i (β i ) (3) max β i βi=β It can be easily prove that the optimize etection rate ˆp (α) in (2) is never smaller than any of the component p,i, i = 1,..., N, at the same α, an ˆp r (β) in (3) is never smaller than any of the component p r,i, i = 1,..., N, at the same β [12]. If a certain classifier cannot help or possibly egraes the overall performance, the optimization will switch it off by tuning its operation points to α = 1, p = 1 in case of fusion by AND rule, or β = 1, p r = 1 in case of fusion by OR rule. In practice, it is in most cases not possible to have the ROC ˆp (α) in analytical form, instea, the ROC has to be estimate from the evaluation ata. As a result, ˆp (α) are characterize by a set of iscrete operation points rather than a continuous function. The optimization problem formulate in (2) an (3), therefore, has to be solve numerically. In a brute-force way, the optimization coul be one by first calculating the pool of operation points, i.e, estimating all the possible combinations by (1), an then select the ones optimal in the Neyman-Pearson sense. The fusion of three or more ROCs, as prove in Appenix A, can be reuce to iteratively fusing two ROCs. Therefore, the number of possible combinations oes not exploe rapily with the number of ROCs, an the complexity of the optimization is kept low. An example is given to illustrate the optimization proceure, as shown in Fig. 1. The first ROC is obtaine by generating genuine scores as the ranom variables of Gaussian istribution N(1.5, 1), an impostor scores of N( 1.5, 1), while the secon ROC is obtaine by generating genuine scores of N(2, 1) an impostor scores of N( 2, 1). The possible operation points after fusion are inicate by ots, while the final optimize points are marke by small squares. It can be observe that both the AND an OR fuse ROCs are improve, in the Neyman-Pearson sense, over the two original ROCs. 3. Hybri Fusion The motivation for the hybri fusion is twofol. Firstly, we show that the ecision fusion framework using ROCs is very general an can be extene easily. Seconly, by hybri fusion we hope to take avantage of the score-level an ecision-level fusion, an eventually achieve an even more reliable an robust biometric system. In this section, we will first iscuss the pros an cons of the score-level an ecision-level fusion, an then present the hybri fusion metho.

3 Figure 1. (a) the first component ROC; (b) the secon component ROC; (c) all the possible AND fuse points an the optimal ROC selecte; () all the possible OR fuse points an the optimal ROC selecte Score-level an Decision-level Fusion: Pros an Cons Score-level fusion is the most popular way of fusion. The avantage of it is obvious. As a quantitative similarity measure it contains rich information about the biometric input, an yet it is still easy to process compare to sensor-level or feature-level ata. In many cases, score-level fusion is able to achieve theoretically optimal performance. For example, taking prouct of the matching scores, which are inepenent an proportional to the likelihoo ratio (in the feature space), is an ieal estimation of the joint likelihoo ratio. Also, in the ensity-base score-level fusion [2], the ROC corresponing to the likelihoo ratio statistic (in the matching score space), is optimal in the Neyman-Pearson sense. A isavantage of score-level fusion is that, because it works in the matching score space, it is subject to consierable flexibilities. For example, ifferent normalization methos of the matching scores lea to ifferent ecision bounaries. Also, a too small training set of scores might easily overfits the ata, especially in methos with flexible bounaries. There are also avantages an isavantages of the ecision-level fusion escribe in Section 2. First of all, the framework is simple an clear from a mathematical point of view. Only a compact set of operation points is involve, an the Neyman-Pearson criterion is very beneficial for any biometric system. Besies, the optimization is not influence by any score normalization, to which the ROCs are strictly invariant. Furthermore, the OR rule fusion is very suitable for many real worl biometric applications, with outliers existent in the genuine class [12]. Basically, when the istributions of the genuine an impostor class are not symmetric, as is often true, the AND or OR ecision fusion is very likely to fit because they have unsymmetrical support for the two classes. The common criticism on ecision-level fusion is that it has small an rigi information content. In the framework escribe in Section 2, however, the ecision-level fusion has been aapte in such a way that the operation points are not fixe anymore, instea they are tunable an can be optimize with respect to performance. The isavantage of ecision-level fusion, nevertheless, is still the limite possibility of ecision bounaries, because the operations are restricte to thresholing, AND, an OR. This paper presents a new fusion scheme, combining the score-level an ecision-level fusion uner the general fusion framework escribe in Section 2. As the fusion framework is orientate on performance, we expect the final classifier to automatically alternate between the two levels of fusion in ifferent situations, an achieve improve performance.

4 3.2. Hybri Fusion Metho Uner the general ecision fusion framework, any two or more ROCs can be fuse together. A biometric system, which has alreay been fuse, can be easily put into this framework. This enables us to esign a new hybri biometric fusion scheme, combining score-level an ecision-level fusion. Suppose the ecision-level fusion can be expresse by r ecision = D(r 1,..., r N ) (4) where r 1,...r N are the component ROCs to be fuse, D is the ecision fusion function, an r ecision is the resulting ROC. Similarly, suppose the score-level fusion is expresse by r score = S(r 1,..., r N ) (5) where S is the score fusion function, an r score is the resulting ROC. The hybri fusion function H is efine as H(r 1,..., r N ) = D (r 1,..., r N, S 1,..., S M ) (6) where S 1,..., S M enotes the ROCs of ifferent score-level fusion methos. In Section 2, we have assume inepenency between the component ROCs. In hybri fusion, however, the assumption is not satisfie, as the inputs in (6), r 1,..., r N an S(r 1,..., r N ), are epenent. Strictly speaking, we have to go back to the matching score space, an take into account the joint probabilities of the component matching scores. For example, suppose we are fusing two classifiers with matching scores s 1 an s 2, with the genuine score istribution p(s 1, s 2 ω 1 ), an the impostor score istribution p(s 1, s 2 ω 0 ). The optimization at ecision level, in the Neyman-Pearson sense, is ˆp (α) = max t 1,t 2 subject to { t 1 t 1 t 2 } p(s 1, s 2 ω 1 )s 1 s 2 ) t 2 p(s 1, s 2 ω 0 )s 1 s 2 = α There are methos to solve (7), however, in practice we foun that the inepenency assumption, i.e., solving (2) to obtain the threshols corresponing to the optimal α i s, is just aequate. The inepenency assumption might change the estimation of ˆp (α), but the threshols t 1 an t 2 corresponing to its maximal value is often unchange, or close enough to the real t 1 an t 2 uner the epenent assumption. This is similar to the Naive Bayes problem [5], which (7) also assumes inepenency between features, but whose optimality in epenency cases has been acknowlege in a wie range of applications [16][4]. Actually, we have observe that in many cases, the results from inepenency assumption is even better than the results from the epenency solutions. This can be explaine by that fact that the optimization problem in (7) has much larger complexity than (2) an therefore more prone to overfit the solutions to the specific training set of matching scores. Solving the hybri fusion using the ROCs, instea of the matching scores, not only preserves the simplicity of the metho, but also makes the solution more robust to the eviations between the training an testing scores. We summarize the hybri fusion metho as follows: 1. Given a set of component matching scores, an a set of score-level fusion methos. 2. (Training) Derive iniviual ROCs from the component matching scores an the score-level fuse matching scores. Fuse all the ROCs uner the fusion framework by the AND rule (2) or OR rule (3), an obtain the optimal combination of operation points. 3. Obtain the threshols corresponing to those optimize operation points. 4. (Testing) Apply the traine threshols on the component matching scores the score-level fuse matching scores, an fuse the ecisions by the AND rule or OR rule as the final ecision. 4. Experiments an Results In this section, we present some experimental results of the propose hybri fusion. For the score-level fusion, we use the sum-rule, an preprocess the matchings by Z- normalization [10], which normalize the genuine scores to zero mean an unite variance. Many other score-level fusion methos coul be inserte into the hybri fusion, but in the preliminary experiments we only illustrate with Z- norm sum-rule score-level fusion, which is simple an robust. For the ecision-level fusion, we use the OR rule, as in practice it is more suitable because of the outliers in the genuine class 1. The first example is to combine the two-imensional face texture an three-imensional face shape information. The context of this work is EU FP6 3D-face project [1] which aims to combine two face moalities as a secure biometric for EU passports. The atabase that the algorithms are evelope on is the FRGC atabase [13] which contains both 2D texture an 3D shape ata. For either moality, 1 There coul also be outliers in the impostor class, but the outlier proportion in the genuine class is usually much higher. Generally the two opposite class are not balance, either in size, or in istribution.

5 (a) Figure 2. Example testing results of fusion between two face moalities, with matching scores from ifferent institutes. Figure 3. Example testing results of BA-fusion score atabase, with two typical type of score istributions. the matching scores are erive by three algorithms, evelope by the Cognitec Systems GmbH (COG), L-1 Ientity Solutions (L1), an University of Twente (UTW), respectively. The atabase contains ata of 465 subjects an has in total 4,007 samples, with 2D texture ata an 3D shape ata collecte simultaneously. The classifiers which prouce the matching scores are traine on 309 subjects in the atabase. To train fusion, another 100 subjects are taken to obtain (b) (a) (b) the matching scores from the traine classifier, resulting in 25,520 genuine scores an 2,568,190 impostor scores (fusion training ata). The remaining 56 subjects are use for evaluation, resulting in 12,270 genuine scores an 700,910 impostor scores (fusion testing ata). In the following experiments, we optimize the threshols on the fusion training ata, while evaluate the performance on the fusion testing ata. In Fig. 2, we give two examples of fusion between the 2D texture an 3D shape ata. Both the scatterplot of the testing ata an the fusion results on those ata are shown. For comparison, we list the original ROCs, the sum rule fusion results, OR rule fusion results, an the hybri fusion results. It can be observe that the hybri fusion metho outperforms both sum rule score-level fusion an OR rule ecision-level fusion in both cases. The secon example is on the public atabase BA-fusion (Biometric Authentication Fusion Benchmark Database) [8] evelope from the XM2VTS atabase [7], which contains the matching scores from face vieo an speech ata. The matching scores are erive from various baseline systems (for etails, see [8]). We show two examples with typical score istributions from the ataset, as in Fig. 3. Again we observe that the hybri fusion metho tunes the performance in such a way that it is always better than the score-level metho or ecision-level fusion methos. The score-level fusion an ecision-level fusion both have their avantages an fit ifferent situations. For example, it can be observe that in Fig. 2 (a) the ecision-level fusion is more beneficial, while in Fig. 2 (b) the ecisionlevel fusion an score-level fusion have comparable performance. In Fig. 3 (a) sum rule fusion is more suitable, while in Fig. 3 (b), ecision fusion an sum rule fusion fit ifferent requirement of FARs. The hybri fusion, which combines the two levels of fusion, aaptively tunes itself accoring to the ifferent matching score istributions an specific performance requirements (i.e. prefix FAR or FRR). As can be observe, the final performance of hybri fusion is improve over the better one, although sometimes with small margins ue to the epenency. Note that in both Fig. 2 an Fig. 3, the scatterplots are of the testing scores, ifferent from the training scores on which the fusion is traine. In some cases, the improvement of the performance might also be accounte by the relative insensitivity of the ROC to overtraining, when a simple set of operation points are use to represent the original set of genuine an impostor training scores. The hybri fusion, therefore, is favorable in three senses, namely, aaptivity to ifferent situations (alternating between the two levels of fusion), robustness to outliers, an relative insensitivity to eviations between the training an testing scores.

6 5. Conclusions In this paper, we investigate a general fusion framework at ecision level, by optimizing the operation points on the ROCs in the Neyman-Pearson sense. Uner this framework, a hybri fusion metho is propose, which combines the score-level fusion an the ecision-level fusion, an takes avantage of both. Experiments show that in ifferent cases, with ifferent matching score istributions, the hybri fusion metho is able to aapt itself for improve performance over the two levels of fusion. More generally speaking, any fusion metho coul be integrate into this framework an optimize with respect to ROC, with improvements expecte in the Neyman-Pearson sense. A. Proof of Iterative Fusion We show that the iterative fusion of two ROCs is optimal for the AND rule. The proof for the OR rule is similar. Let I an J enote the inex sets, such that I J = an I J = {1,..., N}. Define p I (α) = max p,i (α i ), p J α i α i=α i I (α) = max α j α j=α j J p,j (α j ) (8) an (α) = max p I (α I )p J α I α J (αj ). (9) =α First, expaning (α) results in a prouct p,k(α k ) for some α k, k = 1,..., N, satisfying α k = α. Therefore, we have Secon, (α) max α k=α p,k (α k ). (10) (α) pi (α I )p J (αj ) αi α J =α p,i (α i ) p,j (α j ) i I = p,k (α k ) max α k=α αi=α I j J αk =α On combining (10) an (11) we have, αj=α J p,k (α k ). (11) This means that if the optimal ROCs are known for isjoint subsets, the overall optimal ROC can be foun by optimally fusing the subsets. References [1] 3D Face. 3D Face biometric research. 3face.org/, [2] S. C. Dass, K. Nanakumar, an A. K. Jain. A principle approach to score level fusion in multimoal biometric systems. In Auio- an Vieo-Base Biometric Person Authentication, pages , [3] J. Daugman. Combining multiple biometrics. combine/combine.html, [4] P. Domingos an M. Pazzani. Beyon inepenence: Conitions for the optimality of the simple bayesian classifier. In 13th Internat. Conf. on Machine Learning, [5] R. O. Dua, P. E. Hart, an D. G. Stork. Pattern Classification (2n e.). John Wiley an Sons, New York, [6] J. Kittler, M. Hatef, R. Duin, an J. Matas. On combining classifiers. IEEE Transactions on Pattern Analysis an Machine Intelligence, 20(3): , [7] K. Messer, J. Matas, J. Kittler, J. Luettin, an G. Maitre;. Xm2vtsb: The extene m2vts atabase. In 2n Conference on Auio an Vieo-base Biometric Personal Verification, [8] N. Poh an S. Bengio. Database, protocols an tools for evaluating score-level fusion algorithms in biometric authentication. Pattern Recognition, 39(2): , [9] A. Ross an A. Jain. Information fusion in biometrics. Pattern Recognition Letters, 24(13), [10] A. Ross, K. Nanakumar, an A. Jain. Hanbook of Multibiomtrics. Springer Publishers, [11] C. Sanerson an K. Paliwal. Information fusion an person verification using speech an face information. Technical report, IDIAP, Switzerlan, September [12] Q. Tao an R. Velhuis. Optimal ecision fusion for a face verification system. In the 2n International Conference on Biometrics, pages , Seoul, Korea, [13] FRGC. Frgc face atabase. frgc/. [14] H. van Trees. Detectioin, Estimation, an Moulation Theory. John Wiley an Sons, New York, [15] Y. Wang, T. Tan, an A. K. Jain. Combining face an iris biometrics for ientity verification. In Fourth International Conference on AVBPA, pages , [16] H. Zhang. The optimality of naive bayes. In 17th Internat. FLAIRS Conf., (α) = max α k=α p,k (α k ). (12)