Using Random Subspace to Combine Multiple Features for Face Recognition

Transcription

1 Usng Random Subspace to Combne Multple Features for Face Recognton Xaogang Wang and Xaoou ang Department of Informaton Engneerng he Chnese Unversty of Hong Kong Shatn, Hong Kong {xgwang1, Abstract LDA s a popular subspace based face recognton approach. However, t often suffers from the small sample sze problem. When dealng wth the hgh dmensonal face data, the LDA classfer constructed from the small tranng set s often based and unstable. In ths paper, we use the random subspace method (RSM) to overcome the small sample sze problem for LDA. Some low dmensonal subspaces are randomly generated from face space. A LDA classfer s constructed from each random subspace, and the outputs of multple LDA classfers are combned n the fnal decson. Based on the random subspace LDA classfers, a robust face recognton system s developed ntegratng shape, texture, and Gabor wavelet responses. he algorthm acheves 99.83% accuracy on the XM2VS database. 1. Introducton Face recognton has drawn more and more attenton n recent years. LDA or Fsherface [2][3] s a popular subspace based face recognton approach. Accordng to the Fsher crtera, LDA determnes a set of projecton vectors maxmzng the between-class scatter matrx and mnmzng the wthn-class scatter matrx n the projectve feature space. However, when dealng wth the hgh dmensonal face data, LDA often suffers from the small sample sze problem. Snce usually there are only a few samples n each face class for tranng, the wthn-class scatter matrx s not well estmated and may become sngular. So the LDA classfer s often based and senstve to slght changes of the tranng set. o address ths problem, a two-stage PCA+LDA approach,.e. Fsherface [2] s proposed. Usng PCA, the hgh dmensonal face data s projected to a low dmensonal feature space and then LDA s performed n the low dmensonal PCA subspace. Usually, the egenfaces wth small egenvalues are removed n the PCA subspace. Snce they may also encode some nformaton helpful for recognton, ther removal may ntroduce a loss of dscrmnant nformaton. o construct a stable LDA classfer, the PCA subspace dmenson has to be much smaller than the tranng set sze. When the PCA subspace dmenson s relatvely hgh, the constructed LDA classfer s often based and unstable. he projecton vectors may be greatly changed by the slght dsturbance of nose on the tranng set. So when the tranng set s small, some dscrmnatve nformaton has to be dscarded n order to construct a stable LDA classfer. In ths paper, we use the random subspace method to overcome the small sample sze problem for LDA. PCA s frst appled to face data. A small number of egenfaces are randomly selected to form a random subspace, and a LDA classfer s constructed from each subspace. In recognton, the nput face mage s fed to multple LDA classfers constructed from K random subspaces. he outputs are combned usng a fuson methodology to make the fnal decson. We apply ths random subspace method to the ntegraton of multple features. In [4], Zhao et. al. ponted out that both holstc feature and local features are crucal for face recognton, and have dfferent contrbutons. hree typcal knds of features, shape, texture and local Gabor wavelet responses are selected. hey undergo a scale normalzaton and decorrelaton by PCA to form a combned long feature vector. hen multple LDA classfers are generated by randomly samplng from ths combned vector for recognton. Our approach s tested on the XM2VS database [5]. It acheves 99.83% recognton accuracy, sgnfcantly outperformng conventonal face recognton methods. 2. LDA n Reduced PCA Space In ths secton, we brefly revew the conventonal LDA face recognton approach usng two-stage PCA+LDA. For appearance-based face recognton, a 2D face mage s vewed as a vector wth length N n the hgh dmensonal mage space. he tranng set contans

2 M samples M X L. j j PCA x 1 belongng to L ndvdual classes PCA uses the Karhunen-Loeve ransform for face representaton and recognton. A set of egenfaces are computed from the egenvectors of the ensemble covarance matrx C of the tranng set, M C 1 x mx m, (1) where m s the mean of all samples. Egenfaces are sorted by egenvalues, whch represent the varance of face dstrbuton on egenfaces. here are at most M-1 egenfaces wth non-zero egenvalues. Normally the K largest egenfaces, U u1, u K, are selected to span the egenspace, snce they can optmally reconstruct the face mage wth the mnmum reconstructon error. Low dmensonal face features are extracted by projectng the face data x to the egenspace, w U x m. (2) he features on dfferent egenfaces are uncorrelated, and they are ndependent f the face data can be modeled as a Gaussan dstrbuton LDA LDA tres to fnd a set of projectng vectors W best dscrmnatng dfferent classes. Accordng to the Fsher crtera, t can be acheved by maxmzng the rato of determnant of the between-class scatter matrx S b to the determnant of the wthn-class scatter matrx S w, W SbW W arg max. (3) W S ww S b and S w are defned as, L Sw 1 xkx L Sb n m m 1 xk m xk m m m, (4), (5) where m s the mean face for class X wth n samples. W can be computed from the egenvectors of Sw 1 S b [6]. he rank of S w s at most M-L. But n face recognton, usually there are only a few samples for each class, and M-L s far smaller than the face vector length N. So S w may become sngular and t s dffcult 1 to compute S w. In the Fsherface method [2], the face data s frst projected to a PCA subspace spanned by M-L largest egenfaces. LDA s then performed n the M-L dmensonal subspace, such that S w s nonsngular. But n many cases, M-L dmensonalty s stll too hgh for the tranng set. When the tranng set s small, S w s not well estmated. A slght dsturbance of nose on the tranng set wll greatly change the nverse of S w. So the LDA classfer s often based and unstable. Dfferent subspace dmensons are selected n other studes. In [3], the dmenson of PCA subspace was chosen as 40% of the total number of egenfaces. In [7], the selected egenfaces contans 95% of the total energy. hey all remove egenfaces wth small egenvalues. However, egenvalue s not an ndcator of the feature dscrmnablty. Snce the PCA subspace dmenson depends on the tranng set, when the tranng set s small, some dscrmnatve nformaton has to be dscarded n order to construct a stable LDA classfer. 3. Mult-Features Fuson Usng RSM 3.1. RSM Based LDA In Fsherface, overfttng happens when the tranng set s relatvely small compared to the hgh dmensonalty of the feature vector. In order to construct a stable LDA classfer, we sample a small subset of features to reduce dscrepancy between the tranng set sze and the feature vector length. Usng such a random samplng method, we construct a multple number of stable LDA classfers. We then combne these classfers to construct a more powerful classfer that covers the entre feature space wthout losng dscrmnant nformaton. Unlke the tradtonal random subspace method that samples the orgnal feature vector drectly, our algorthm samples n the PCA subspace. We frst apply PCA to the face tranng set. All the egenfaces wth zero egenvalues are removed, and M-1 egenfaces U 0 u 1,, u M 1 are retaned as canddates to construct random subspaces. he dmenson of feature space s frst greatly reduced wthout any loss on dscrmnatve nformaton, snce all the tranng samples have zero projectons on the egenfaces wth zero egenvalues. hen, K random subspaces S K 1 are generated. he dmenson of random subspace s determned by the tranng set to make the LDA classfer stable. he

3 random subspace s composed of two parts. he frst N 0 dmensons are fxed as the N 0 largest egenfaces, and the remanng N 1 dmensons are randomly selected from u, u. he N 0 largest egenfaces M N 1, 0 M 1 encode much face structural nformaton. If they are not ncluded n the random subspace, the accuracy of LDA classfers may be too low. Our approach guarantees that the LDA classfer n each random subspace has satsfactory accuracy. he N 1 random dmensons cover most of the remanng small egenfaces. So the ensemble classfers also have a certan degree of error dversty. K LDA classfers C k x are constructed from the K random subspaces. In recognton, the raw face data s projected to the K random subspace and fed to K LDA classfer. he output are combned usng a fuson methodology to make the fnal decson Combnng Multple LDA Classfers Many methods on combnng multple classfers have been proposed [9][10]. In ths paper, we use two smple fuson rules to combne LDA classfers: majorty votng and sum rule Majorty votng Each LDA classfer the nput face data, x as a bnary functon, k x X C k 1, 0, C k x assgns a class label to. We represent ths event x Ck. (6) otherwse By a majorty votng, the fnal class s chosen as, x arg max k x X X K. (7) Sum rule We assume that x belongs to k 1 x P X Ck s the probablty that X under the measure of LDA classfer. Accordng to the sum rule, the class for the fnal decson s chosen as, C k x x K k x arg max P X C (8) x X 1 P X Ck can be estmated from the output of the LDA classfer. For LDA classfer x, the center k C k m of class X, and nput face data x are projected to LDA vectors W, k wk Wk m (9) x P X Ck s estmated as x wk Wk x (10) x w w ˆ k k P X Ck x 1 / 2, (11) x w k wk whch has been mapped to [0,1] Mult-Features Fuson Random subspace based LDA essentally ntegrates dfferent parts of the orgnal hgh dmensonal feature vector. It also can be extended to the ntegraton of dfferent knds of features such as shape, texture, and Gabor responses. A face graph contanng 35 fducal ponts s desgned as shown n Fg. 1. Usng the method n Actve Shape Model [11], we separate the face mage nto shape and texture. he shape vector V s s formed by concatenatng the coordnates of the 35 fducal ponts after algnment. Warpng the face mage onto a mean face shape, the texture vector V t s obtaned by samplng ntensty on the shape-normalzed mage. As descrbed n Elastc Bunch Graph Matchng [12], a set of Gabor kernels n fve scales and eght orentatons are convolved wth the local patch around each fducal pont. he Gabor feature vector V g combnes magntudes of Gabor responses to represent the face local texture. he mult-feature random subspace LDA face recognton algorthm s then desgned as the followng. 1. Apply PCA to the three features vectors respectvely to compute the egenvectors U s, U t, s t g U g and egenvalues,,. All the egenvectors wth zero egenvalues are removed. 2. For each face mage, project each knd of feature to the egenvectors and normalze them by the sum of egenvalues, such that they are n the same scale. j w j U j V j /, (j=s, t, g). (12) 3. Combne w t, w s, w g nto a large feature vector, and apply PCA on ths combned feature vector. 4. Apply the random subspace algorthm to the combned feature vectors to generate multple LDA classfers. 5. Combne these multple classfers. Whle most other mult-feature ntegraton systems are based on match score level or decson level by desgnng one classfer for each knd of feature, our

4 ntegraton approach starts from the feature level. Integraton at feature level conveys the rchest nformaton, but t s more dffcult, snce dfferent knds of features are ncompatble n scale and the new combned feature vector has a hgher dmensonalty. Our approach overcomes both problems. 4. Experments Fgure 1. Face graph model. We conduct experments on the XM2VS face database [5]. here are 295 people, and each person has four frontal face mages taken n four dfferent sessons. Some examples are shown n Fgure 2. In our experments, two face mages of each face class are used for tranng and reference, and the remanng two for testng. We adopt the recognton test protocol used n FERE [13]. All the face classes n the reference set are ranked. We measure the percentage of the correct answer n top 1 match Random subspace based LDA We frst compare random subspace based LDA wth conventonal LDA approach usng the holstc feature. In preprocessng, the face mage s normalzed by translaton, rotaton, and scalng, such that the centers of two eyes are n fxed postons. A 46 by 81 mask removes most of the background. Hstogram equalzaton s appled as photometrc normalzaton. Fgure 3 reports the accuracy of a sngle LDA classfer constructed from PCA space wth dfferent number of egenfaces. Snce there are 590 face mages of 295 classes n the tranng set, there are 589 egenfaces wth non-zero egenvalues. Accordng to the Fsherface [2], the PCA space dmenson should be M- C=295. However, the result shows that the accuracy s only 79% usng a sngle LDA classfer constructed from 295 egenfaces, because ths dmenson s too hgh for the tranng set. We observe that LDA classfer has the best accuracy 92.88%, when the PCA subspace dmenson s set at 100. So for ths data set, 100 seems to be a sutable dmenson to construct a stable LDA Fgure 2. Examples of face mages n XM2VS database taken n four dfferent sessons classfer. In the followng experments, we choose 100 as the dmenson of random subspaces to construct the multple LDA classfers. Frst, we randomly select 100 egenfaces from the 589 egenfaces wth nonzero egenvalues. he result of combnng 20 LDA classfers usng majorty votng s shown n Fgure 4. Wth random samplng, the accuracy of each ndvdual LDA classfer s low, between 50% and 70%. Usng majorty votng, the weak classfers are greatly enforced, and 87% accuracy s acheved. hs shows that LDA classfers constructed from dfferent random subspaces are complementary of each other. In Fgure 5, as we ncrease the classfer number K, the accuracy of the combned classfer mproves, and even becomes better than the hghest accuracy n Fgure 3. Although ncreasng classfer number and usng more complex combnng rules may further mprove the performance, t wll ncrease the system burden. A better approach to mprove the accuracy of the combned classfer s to ncrease the performance of each ndvdual weak classfer. o mprove the accuracy of each ndvdual LDA classfer, as proposed n Secton 3.1, n each random subspace, we fx the frst 50 bass as the 50 largest egenfaces, and randomly select another 50 bass from the remanng 539 egenfaces. As shown n Fgure 6, ndvdual LDA classfers are mproved sgnfcantly. hey are smlar to the LDA classfer based on the frst 100 egenfaces. hs shows that u 51,, u100 are not necessarly more dscrmnatve than those smaller egenfaces. hese classfers are also complementary of each other, so much better accuracy s acheved when they are combned Multple Features Fuson In able 1, we report the recognton accuracy of ntegratng shape, texture, and Gabor features usng random subspace LDA. Combnng 20 classfers usng the sum rule, t acheves 99.83% recognton accuracy.

5 For 590 testng samples, t msclassfes only one! For comparson, we also report the accuraces of some conventonal face recognton approaches. Egenface [14], Fsherface [2], and Bayes[15] are three subspace face recognton approaches based on holstc feature. Elastc Bunch Graph Matchng descrbed n [12] uses the correlaton of Gabor features as smlarty measure. Experments clearly demonstrate the superorty of the new method. 5. Concluson In ths paper, we propose to use random subspace based LDA to ntegrate multple features. It effectvely stablzes the LDA classfer based on a small tranng set, and makes use of most dscrmnatve features n the hgh dmensonal space. In ths study, we use the smplest fuson rules to combne multple LDA classfers and also acheve notable mprovement. Many more complex combnaton algorthms [9][10] have been proposed. hey may further mprove the performance. Usng random subspace, a large set of LDA classfers can be generated. Instead of combnng them drectly, t s helpful to select a small set of complementary LDA classfers wth hgh accuracy from them for combnaton. hs s a drecton for our further study. Acknowledgement he work descrbed n ths paper was fully supported by grants from the Research Grants Councl of the Hong Kong Specal Admnstratve Regon (Project no. CUHK 4190/01E and CUHK 4224/03E). References [1]. Kam Ho, "he Random Subspace Method for Constructng Decson Forests," IEEE rans. on PAMI, Vol. 20, No. 8, pp , August [2] P.N. Belhumeur, J. Hespanda, and D. Kregeman, Egenfaces vs. Fsherfaces: Recognton Usng Class Specfc Lnear Projecton, IEEE rans. on PAMI, Vol. 19, No. 7, pp , July [3] D. Swets, J. Weng, Usng Dscrmnant Egenfeatures for Image Retreval, IEEE rans. on PAMI, Vol. 16, No. 8, pp , August, [4] W. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phllps,(2000) "Face Recognton: A Lterature Survey", UMD CAR echncal Report CAR-R948. [5] K. Messer, J. Matas, J. Kttler, J. Luettn, and G. Matre, XM2VSDB: he Extended M2VS Database, Proceedngs of Internatonal Conference on Audo- and Vdeo-Based Person Authentcaton, pp , [6] K. Fukunnaga, Introducton to Statstcal Pattern Recognton, Academc Press, second edton, [7] H. Monn, and P.J. Phllps, Analyss of PCA- Based Face Recognton Algorthms, Emprcal Evaluaton echnques n Computer Vson, K. W. Bowyer and P.J. Phllps, eds, IEEE Computer Socety Press, Los Alamtos, CA, [8] R. Huang, Q. Lu, H. Lu, and S. Ma, "Solvng Small Sample Sze Problem of LDA", Proceedngs of ICPR, pp , [9] L. Xu, A. Krzyzak, C. Y. Suen, "Method of Combnng Multple Classfers and her Applcatons to Handwrtng Recognton," IEEE rans. on System, Man, and Cybernetcs, Vol. 22, No. 3, , [10]. Kam Ho, J. Hull, S. Srhar, "Decson Combnaton n Multple Classfer Systems," IEEE rans. on PAMI, Vol. 16, No.1, pp , Jan [11]A. Lants, C. J. aylor, and. F. Cootes, Automatc Interpretaton and Codng of Face Images Usng Flexble Models, IEEE rans. on PAMI, Vol. 19, No. 7, pp , July [12]L. Wskott, J.M. Fellous, N. Kruger and C. von der Malsburg, "Face Recognton by Elastc Bunch Graph Matchng," IEEE rans. on Pattern Analyss and Machne Intellgence, Vol. 19, No.7, pp , July, [13]P. J. Phllps, H. Moon, S. A. Rzv, and P. J. Rauss, he FERE Evaluaton, n Face Recognton: From heory to Applcatons, H. Wechsler, P. J. Phllps, V. Bruce, F.F. Soule, and. S. Huang, Eds., Berln: Sprnger-Verlag, [14]M. urk and A. Pentland, Face recognton usng egenfaces, Proceedngs of IEEE, CVPR, pp , Hawa, June, [15]B. Moghaddam,. Jebara, and A. Pentland, Bayesan Face Recognton, Pattern Recognton, Vol. 33, pp , 2000.

6 Fgure 3. Recognton accuracy of LDA classfer usng dfferent number of egenfaces n the reduced PCA subspace. Fgure 4. Recognton accuracy of combng 20 LDA classfers constructed from random subspaces usng majorty votng. Each random subspace randomly selects 100 egenfaces from 589 egenfaces wth nonzero egenvalues. Fgure 5. Accuracy of combnng dfferent number of LDA classfers constructed from random subspaces usng majorty votng. Each random subspace randomly selects 100 egenfaces from 589 egenfaces wth non-zero egenvalues. Fgure 6. Recognton accuracy of combng 20 LDA classfers constructed from random subspaces usng majorty votng and the sum rule. For each 100 dmenson random subspace, the frst 50 dmensons are fxed as the 50 largest egenfaces, and another 50 dmensons are randomly selected from the remanng 539 egenfaces wth non-zero egenvalues. able 1. Compare recognton accuracy of random subspace based LDA (R-LDA) wth conventonal methods Feature Holstc feature Shape exture Gabor Integraton of Mult-feature Method Egenface Fsherface Bayes R-LDA Eucld Eucld EBGM R-LDA Accuracy 85.59% 92.88% 92.71% 96.10% 49.5% 85.76% 95.76% 99.83%