A Multimodal Fusion Algorithm Based on FRR and FAR Using SVM

Transcription

1 Internatonal Journal of Securty and Its Applcatons A Multmodal Fuson Algorthm Based on FRR and FAR Usng SVM Yong L 1, Meme Sh 2, En Zhu 3, Janpng Yn 3, Janmn Zhao 4 1 Department of Informaton Engneerng, Engneerng Unversty of CAPF, X an Chna 2 Department of Scence, Engneerng Unversty of CAPF, X'an, Chna 3 School of Computer, Natonal Unversty of Defense Technology, Changsha, Hunan Provnce, Chna 4 Top Key Dscplne of Computer Software and Theory n Zhejang Provncal Colleges, Zhejang Normal Unversty, Chna 1 lyong@nudt.edu.cn, 2 memesh2003@yahoo.com.cn, 3 {enzhu,jpyn}@nudt.edu.cn, 4 zhaojm0310@163.com Abstract Remarable mprovements n recognton can be acheved through multbometrc fuson. Among varous fuson technques, score level fuson s the most frequently used n multbometrc system. In ths paper, we propose a novel fuson algorthm based on False Reject Rate (FRR) and False Accept Rate (FAR) usng Support Vector Machne (SVM). It transfers scores nto correspondng FRRs and FARs, thus avodng calculatng posteror probablty of a certan score, as well as be capable of llustratng dstrbuton of matchng scores. The proposed method taes full advantages of both capabltes of FRR and FAR to descrbe the order of score and classfcaton of SVM. Expermental results show that the proposed method outperforms exstng representatve approaches and can effectvely mprove the performance of multbometrc system. Keywords: Bometrcs, Multbometrc, Score level fuson, Mult-modal, SVM, False Reject Rate, False Accept Rate 1. Introducton Bometrc recognton refers to the process of automatcally confrmng the dentty of a person through physologcal or behavoral characterstcs. A hgh securty level system always demands for hgh performance, such as low FRR (False Reject Rate) and FAR (False Accept Rate). Unbometrc system, usng a sngle bometrc trat, wth ts nsuffcent nformaton and easly nfluenced by collecton noses, s no longer able to meet the requrements of hgh level system [1]. A possble soluton s multbometrc fuson [14-16]. Accordng to the level of nformaton fused, the fuson scheme can be classfed nto sensor level, feature level, score level and decson level fuson. Among them, score level fuson s the most frequently researched for t shelds both the dversty of bottom bometrc characterstc data and complexty of recognton process, as well as preserves smlarty metrcs of ndvdual trats to dfferentate samples from genune or ntruder. Already exstng score level fuson approaches can be categorzed nto three classes: transformaton-based, densty-based and classfer-based fuson [2]. Transformaton-based fuson frstly normalzes the matchng scores and then acqures a new score through a certan fuson rule to mae fnal decson. Ths method taes nto consderaton of the two factors, one s normalzaton functon, and the other s fuson rule [2, 65

2 Internatonal Journal of Securty and Its Applcatons 3-4]. In densty-based fuson scheme, matchng scores are frstly transformed nto posteror probablty and fnal decson s made accordng to Byes rules [5-6]. Classfer-based fuson taes N matchng scores as an N-dmensonal feature vector and transfers fuson problem nto classfcaton of the N-dmensonal vector [8-9]. In concluson, transformaton-based fuson technque, wth no tranng process and few consderaton of dstrbuton of matchng scores, s easy to mplement. On the contrary, densty based fuson, whch requres accurate estmaton of densty and huge number of tranng samples, s hard to carry out for the followng reasons: Frstly, postve samples, namely genune matchng scores are lmted n today's mult-bometrcs system. Secondly, t s dffcult to estmate the densty of matchng scores n that they may not obey a certan dstrbuton model. Besdes, the tme and space cost of ths method s consderable. Classfer based fuson has ts advantage of freeng from the restrcton of dfferent dstrbuton of matchng scores. However, t can nether drectly produce a matchng score nor obtan optmal FRR by certan FAR. Also, t needs suffcent tranng samples and new tranng for a dfferent system. Based on the above analyss of the advantages and dsadvantages of exstng methods, we propose a novel approach to descrbe the dstrbuton of matchng scores by FRR and FAR. As two mportant parameters of llustratng performance of a recognton system, FRR and FAR avod estmaton of probablty densty when used as descrptor of matchng score dstrbuton. It s smlar to ntegral of matchng score densty, thus maes the orgnal dscrete densty contnuous. Tronc [7] utlzed FRR-FAR as an mportant standard to mae the fnal decson, but wth the assumpton of normal dstrbuton when calculatng ts value. However, no matter how FRR and FAR are combned, loss of nformaton s nevtable. Besdes, the recognton performance of sum rule s not good enough. In ths context, we consder drectly usng FRR and FAR as features. Wth more features ntroduced, avalable nformaton s much more abundant. It frstly calculates FRR and FAR based on tranng samples, then computes FRR and FAR of each matchng score through nterpolaton based on testng samples, and maes fnal decson by SVM. Compared wth exstng methods, the algorthm presented n ths paper can greatly mprove the recognton performance of multbometrc system. 2. Transform the Score to FRR and FAR Matchng scores comng from dfferent recognton systems are not comparable drectly. At the same tme, FRR and FAR can be obtaned by settng threshold as each score. They can well descrbe the score dstrbuton of genune and mposter. Therefore, we frstly calculate FRR and FAR of each matchng score based on tranng samples, then compute FRR and FAR of each matchng score through nterpolaton based on testng samples FRR and FAR calculaton based on tranng set Suppose an M-matcher fuson system has N tranng samples. For th tranng sample, the template feature s 1 M U u,..., u, nput feature s 1 M V v,..., u and output score s S 1 s,..., s M. Here, u, v, s respectvely represents template feature, nput feature and matchng score of th tranng sample by th matcher. In addton, we use z to note when th tranng sample s genune or mposter. Namely, f the th tranng sample s genune, then z =1; f t s mposter, then z = 0. Then, FRR and FAR of th (1 M ) matcher based on tranng samples are respectvely calculated as follows. 66

3 Internatonal Journal of Securty and Its Applcatons N 1 N 1 FAR ( t) s t, z 0 (1 z ) (1) FRR ( t) s t, z 1 z (2) 2.2. FRR and FAR calculaton based on testng set Accordng to the above analyss, calculaton of transformaton functon based on FRR and FAR requres all the matchng scores, whch s mpossble n a real system. So we can only mae the calculaton based on tranng samples. Generally, tranng and testng samples follow the same dstrbuton, whch means that f one score appears n the tranng samples, t wll probably appear n the testng samples as well. Thus, usng each matchng score as threshold t, we can get transformaton value. For the newly appearng scores, we use nterpolaton to obtan ts transformaton value. At most N dscrete matchng scores are generated by each matcher for N tranng samples n tranng set. Generally, score obtaned n testng process s not absolutely dentcal to that attaned n tranng process. So the transformatonal scores n testng process can be calculated through nterpolaton based on scores from tranng samples. Suppose n the tranng process, n( n N) dfferent matchng scores are generated by th matcher based on N samples and raned n ascendng order { t, t,..., t }. Substtute these scores nto FAR () t and FRR () t, we can obtan as 1 2 n { FAR ( t ) 1 n } and { FRR ( t ) 1 n }. For matchng score x generated by th matcher n testng process, FAR ( x ) and FRR ( x ) are calculated n the four stuatons as follows. 1. If x t1, then use t1 and t 2 as control ponts, and calculate FRR ( x) and FAR ( x) through extrapolaton. Suppose t2 x x t1 FRR ( x) FRR ( t1) FRR ( t2) (3) t t t t t x x t FAR ( x) FAR ( t ) FAR ( t ) t2 t1 t2 t1 Then FRR FRR 0 FRR (5) 0 FRR 0 FAR FAR 1 FAR (6) 1 FAR 1 2. If x t n, then use t n and t n 1 as control ponts, and calculate FRR ( x) and FAR ( x) through extrapolaton. Suppose tn x x t n1 FRR ( x) FRR ( tn ) FRR ( ) 1 t n (7) t t t t n n1 n n 1 (4) 67

4 Internatonal Journal of Securty and Its Applcatons Then t x x t FAR x FAR t FAR t n n1 ( ) ( n ) ( ) 1 n tn t n t 1 n t n 1 FRR FRR 1 FRR 1 FRR 1 FAR FAR 0 FAR 0 FAR 0 (8) (9) (10) 3. If there exsts whch maes x t, FRR ( x) FRR( t ), FAR ( x) FAR( t ) (11) 4. If there exsts whch maes t x t 1, then calculate FRR ( x) and FAR ( x ) through nterpolaton. t x x t FRR x FRR t FRR t 1 ( ) ( ) ( 1) t 1t t 1t (12) t x x t FAR x FAR t FAR t 1 ( ) ( ) ( 1) t 1t t 1t (13) In step (3) and (4), bnary searchng s adopted to fnd correspondng ( t, t 1( 1 n ) ). The tme complexty of bnary search s O(log 2 N ), thus effectvely reducng tme complexty of the whole algorthm. So far, we have attaned FRR and FAR for each matchng score n testng samples An example for computaton of FRR and FAR To better demonstrate how our algorthm wors, we gve an example n ths secton. Table 1 s one part of the experment based on tranng samples, where matchng scores are raned n ascendng order. From the above analyss, we now that computaton of matchng score transformaton of a certan smple classfer s rrelevant to other classfers. So we just choose one smple classfer for llustraton. Also, n the tranng set, there s no other matchng score among the exstng ones, and FRR and FAR are calculated by tang matchng score as threshold based on tranng samples. Table 2 s one part of the experment based on testng samples, where matchng scores are among those n Table 1. We use calculaton formula of FAR () t and FRR () t based on testng samples. Tae the fourth matchng score x= as example, as we havet 4 x t 5, then t5 x xt4 FRR ( x) FRR ( t4) FRR ( t5) (14) t t t t t x xt FAR ( x) FAR ( t ) FAR ( t ) t5 t4 t5 t4 (15) 68

5 Internatonal Journal of Securty and Its Applcatons Table 1. Computaton of FRR and FAR on tranng set t FRR ( t ) FAR ( t ) Table 2. Computaton of FRR and FAR on testng set x FRR (x) FAR (x) SVM Fuson Based on FRR and FAR Durng score level fuson, score vector s combned to generate a sngle scalar score whch s later used to mae the fnal decson. Based on Secton 2, one match score can be transformed to two scores whch are ts FRR and FAR. All the tranng and testng scores can be transformed nto ther FRRs and FARs. SVM s based on the structural rs mnmzaton prncple [12], whch taes both the tranng error and generalzaton error nto consderaton. The SVM s to fnd an optmal hyper-plane that can well separate two class samples n the feature space. In our proposed multmodal bometrc method, we use SVM to buld a fuson functon whch can provde a fused score. Assume that we have N feature vectors wth assocated label y { 1, 1}, so the tranng data set s denoted as N n {( X, y )} X R, 1,2, N (16) After transformng the match scores nto FRRs and FARs, SVM s used to generate the fnal score. And X FRR ( s ), FAR ( s ), FRR ( s ), FAR ( s ),..., FRR ( s ), FAR ( s ) (17) M M Also, the score can be frst transformed by normalzed methods such as Tanh, MnMax as well as T, and then SVM s used (respectvely denoted as Tanh_SVM, MM_SVM, and T_SVM). In the above process, the sequental mnmal optmzaton (SMO) tranng algorthm [11] s used to enhance the tme effcency. In general, the dscrmnate functon of 69

6 Internatonal Journal of Securty and Its Applcatons an arbtrary classfer s of lttle probablty meanng. However, the probablstc output of SVM classfer can help n multmodal fuson post-processng. Fttng a sgmod functon to the classfer output s a soluton to ths problem. Platt [11] has descrbed a method of fttng the sgmod functon. 4. Experment Results The XM2VTS-Benchmar database [10] conssts of fve face matchers and three speech matchers and was parttoned nto tranng and evaluaton sets accordng to the Lausanne Protocol-1 (LPI). The benchmar of LPI ncludes two fles, one s dev.label and the other s eva.label. We use dev.label as tranng data and eva.label as test data. Our experments are conducted based on ths match score benchmar. We sgn the face matcher as F1, F2, F3, F4 and F5 and the speech matcher as S1, S2 and S3 respectvely. The Equal Error Rate (EER) s the pont of the ROC curve where the two errors,.e., the FRR and the FAR, are equal. Ths performance measure s wdely used n the bometrc feld to assess the performance of bometrc systems [2]. In bometrc recognton systems, we always try to mae EER smaller. 4.1 Comparson wth dfferent fuson strateges We frstly conducted experments to measure the advantages of the proposed method FF_SVM over densty-based algorthm LR [6], transformaton-based fuson rules: Tanh and MnMax wth smple sum rule [2] and the fuson methods T whch was proposed n Ref.[11]. The EER of all the matchers can be found n Table 3. As shown n Table 3, among face matchers, matcher face-3 and face-5 gan the best and worst performance respectvely. And among speech matchers, the performance order s speech-1, speech-3 and speech-2. Table 3. EERs of each smple recognton system on XM2VTS-Benchmar S1 S2 S3 F1 F2 F3 F4 F5 EER Table 4. EER comparson of dfferent fuson technques EER(%) F1S1 F1S2 F1S3 F2S1 F2S2 F2S3 F3S1 F3S2 F3S3 F4S1 F4S2 F4S3 F5S1 F5S2 F5S3 FF_SVM T LR Tanh MM The experments are conducted wth 15 nd multmodal combnaton. Table 2 shows the EER of mult-modal fuson among FF_SVM, LR, T, Tanh and MnMax. From Table 4, we observe that FF_SVM provdes the best performance 8 tmes and the second best performance 6 tmes. FF_SVM fuson method gans the best recognton performance followed by T and LR fuson algorthms. In order to evaluate the performance precsely, we gve each matcher of every fuson a performance mar. The performance mar for the best matcher s 5 and followed s 4, 3, 2, 1. If the performance of two matchers are the same, for 70

7 Internatonal Journal of Securty and Its Applcatons example, both are the second best, then the two matchers get the same mar (4+3)/2=3.5. Table 5 s the performance mar of dfferent fuson technques whch s measured by EER. From Table 5, we can easly fnd that the proposed fuson method FF_SVM shows the best performance because the total mar s the largest one. And we observe that the T and LR methods also perform well. Table 5. Performance mar comparson of dfferent technques based on EER Mar F1S1 F1S2 F1S3 F2S1 F2S2 F2S3 F3S1 F3S2 F3S3 F4S1 F4S2 F4S3 F5S1 F5S2 F5S3 total FF_SVM T LR Tanh MM Comparson wth SVM-based strateges In order to verfy the effect of transformng score to FRR and FAR, we compare the proposed method FF_SVM wth T-based SVM fuson method T_SVM, Tanh-based SVM fuson method Tanh_SVM, MnMax-based SVM fuson method (MM_SVM) and SVM fuson drectly on raw scores. From Table 6 and 7, we can observe that the proposed fuson method FF_SVM outperforms Tanh_SVM, MM_SVM, T_SVM and SVM. Durng 15 multmodal combnatons, FF_SVM gves the best performance 10 tmes, and the total performance mar s larger than the other SVM-based algorthms. Table 6. EER comparson of SVM-based Technques EER(%) F1S1 F1S2 F1S3 F2S1 F2S2 F2S3 F3S1 F3S2 F3S3 F4S1 F4S2 F4S3 F5S1 F5S2 F5S3 FF_SVM T_SVM Tanh_SVM MM_SVM SVM Table 7. Performance mar comparson of SVM-based Technques based on EER Mar F1S1 F1S2 F1S3 F2S1 F2S2 F2S3 F3S1 F3S2 F3S3 F4S1 F4S2 F4S3 F5S1 F5S2 F5S3 total FF_SVM T_SVM Tanh_SVM MM_SVM SVM To show the comparson of all the algorthms n multmodal bometrc systems, Fgure 1 shows the EERs of the 9 algorthms. The proposed FF_SVM algorthm s show n bold lne. From the fgure, we can conclude that the proposed algorthm can mprove the performance of multmodal bometrc system effcently. 71

8 EER Internatonal Journal of Securty and Its Applcatons EER-MM EER-MM-SVM EER-LR EER-SVM EER-T EER-T-SVM EER-Tanh EER-Tanh-SVM EER-FF-SVM Concluson Fgure 1. EERs of all Algorthms Ths paper proposes a novel fuson algorthm: FF_SVM. Ths proposed algorthm frst transforms the match scores nto FRRs and FARs and then adopts SVM for classfcaton. To be more specfc, we frstly calculate FRRs and FARs based on the tranng set, and then use nterpolaton to obtan the FRRs and FARs of each score n the testng set. As well, bnary searchng s appled to fnd nterpolaton ponts n order to reduce tme complexty of the whole algorthm. In addton, SVM s used to explore the relatonshp between the fnal score and FRR & FAR. The expermental results show that ths FF_SVM fuson algorthm can effcently mprove performance compared wth LR, T, Tanh and MnMax algorthms. Besdes, n comparson wth other SVM-based fuson methods, the proposed fuson method can provde better recognton performance for multmodal bometrc fuson systems. Acnowledgments Ths wor was supported by the Natonal Natural Scence Foundaton of Chna (Grant no , , ), the Foundaton for the Author of Natonal Excellent Doctoral Dssertaton (Grant No. 2007B4), and the Openng Fund of Top Key Dscplne of Computer Software and Theory n Zhejang Provncal Colleges at Zhejang Normal Unversty(Grant No. ZSDZZZZXK38). References [1] A. K. Jan, Bometrc Recognton: Overvew and Recent Advances, CIARP, LNCS 4756, (2007), pp [2] A. Ross, K. Nandaumar and A. K. Jan, Handboo of Multbometrcs, Sprnger Verlag, (2006). [3] M. Indovna, U. Uludag, R. Snelc, A. Mn and A. Jan, Multmodal Bometrc Authentcaton Methods: A COTS Approach, Proceedngs of Mult-Modal User Authentcaton (MMUA), (2003), pp [4] M. X. He and S. J. Horng, Performance evaluaton of score level fuson n multmodal bometrc systems, Pattern Recognton, vol. 43, no. 5, (2010), pp [5] A. K. Jan, K. Nandaumar and A. Ross, Score Normalzaton n Multmodal Bometrc Systems, Pattern Recognton, vol. 38, no. 12, (2005), pp

9 Internatonal Journal of Securty and Its Applcatons [6] K. Nandaumar and Y. Chen, Lelhood rato-based bometrc score fuson, IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 30, no. 2, (2008), pp [7] G. Gacnto, F. Rol and R. Tronc, Score Selecton Technques for Fngerprnt Mult-modal Bometrc Authentcaton, Proc. of 13th ICIAP, (2005), pp [8] A. Kumar and V. Kanhangad, A New Framewor for Adaptve Multmodal Bometrcs Management, IEEE Transactons on Informaton Forenscs and Securty, vol. 5, no. 2, (2010), pp [9] R. Tronc and G. Gacnto, Dynamc score combnaton of bnary experts, 19th Internatonal Conference on Pattern Recognton, New Yor, vol. 1-6, (2008), pp [10] N. Poh and S. Bengo, Database, Protocol and Tools for Evaluatng Score-Level Fuson Algorthms n Bometrc Authentcaton, Pattern Recognton, vol. 39, no. 2, (2006), pp [11] Y. L, J. Yn, J. Long and E. Zhu, A Novel Method for Multbometrc Fuson Based on FAR and FRR, MDAI 2009, LNAI 5861, (2009), pp [12] V. Vapn, The nature of statstcal learnng theory, Sprnger-Verlag, NewYor, (1995). [13] J. C. Platt, Probablstc Outputs for Support Vector Machnes for Pattern Recognton, U.Fayyad, Kluwer Academc Publshers: Boston, (1999). [14] D. Kumar and Y. Ryu, A Bref Introducton of bometrcs and Fngerprnt Payment Technology, Internatonal Journal of Advanced Scence and Technology, vol. 4, (2009), pp [15] M. Soltane, N. Doghmane and N. Guers, Face and Speech Based Mult-Modal Bometrc Authentcaton, Internatonal Journal of Advanced Scence and Technology, vol. 21, (2010), pp [16] D. Ranjan Ksu, P. Gupta and J. KantaSng, Multbometrcs Feature Level Fuson by Graph Clusterng, Internatonal Journal of Advanced Scence and Technology, vol. 5, (2011), pp Authors Yong L receved hs PhD degree from Natonal Unversty of Defense Technology of Chna. And now he s a lecturer at the Department of Informaton Engneerng, Engneerng Unversty of CAPF. Meme Sh receved her M.S. degree from Hunan Unversty of Chna. And now she s a lecturer of Engneerng Unversty of CAPF. En Zhu s an assocate professor at the School of Computer, Natonal Unversty of Defense Technology. Hs research nterest covers pattern recognton and mage process. 73

10 Internatonal Journal of Securty and Its Applcatons Janpng Yn s a professor at the School of Computer, Natonal Unversty of Defense Technology. Hs re-search nterest covers pattern recognton, artfcal ntellgence, and networ securty. Janmn Zhao s a professor of Zhejang Normal Unversty. Hs research nterest covers nformaton securty, pattern recognton and mage process. 74