Tensor Subspace Analysis

Transcription

1 Tensor Subspace Analyss Xaofe He 1 Deng Ca Partha Nyog 1 1 Department of Computer Scence, Unversty of Chcago {xaofe, nyog}@cs.uchcago.edu Department of Computer Scence, Unversty of Illnos at Urbana-Champagn dengca@uuc.edu Abstract Prevous work has demonstrated that the mage varatons of many objects (human faces n partcular) under varable lghtng can be effectvely modeled by low dmensonal lnear spaces. The typcal lnear subspace learnng algorthms nclude Prncpal Component Analyss (PCA), Lnear Dscrmnant Analyss (LDA), and Localty Preservng Projecton (LPP). All of these methods consder an n 1 n mage as a hgh dmensonal vector n R n1 n, whle an mage represented n the plane s ntrnscally a matrx. In ths paper, we propose a new algorthm called Tensor Subspace Analyss (). consders an mage as the second order tensor n R n1 R n, where R n1 and R n are two vector spaces. The relatonshp between the column vectors of the mage matrx and that between the row vectors can be naturally characterzed by. detects the ntrnsc local geometrcal structure of the tensor space by learnng a lower dmensonal tensor subspace. We compare our proposed approach wth PCA, LDA and LPP methods on two standard databases. Expermental results demonstrate that acheves better recognton rate, whle beng much more effcent. 1 Introducton There s currently a great deal of nterest n appearance-based approaches to face recognton [1], [5], [8]. When usng appearance-based approaches, we usually represent an mage of sze n 1 n pxels by a vector n R n1 n. Throughout ths paper, we denote by face space the set of all the face mages. The face space s generally a low dmensonal manfold embedded n the ambent space [6], [7], []. The typcal lnear algorthms for learnng such a face manfold for recognton nclude Prncpal Component Analyss (PCA), Lnear Dscrmnant Analyss (LDA) and Localty Preservng Projecton (LPP) [4]. Most of prevous works on statstcal mage analyss represent an mage by a vector n hgh-dmensonal space. However, an mage s ntrnscally a matrx, or the second order tensor. The relatonshp between the rows vectors of the matrx and that between the column vectors mght be mportant for fndng a projecton, especally when the number of tranng samples s small. Recently, multlnear algebra, the algebra of hgher-order tensors, was appled for analyzng the multfactor structure of mage ensembles [9], [11], [1]. Vaslescu and Terzopoulos have proposed a novel face representaton algorthm called Tensorface [9]. Tensorface represents the set of face mages by a hgher-order tensor and

2 extends Sngular Value Decomposton (SVD) to hgher-order tensor data. In ths way, the multple factors related to expresson, llumnaton and pose can be separated from dfferent dmensons of the tensor. In ths paper, we propose a new algorthm for mage (human faces n partcular) representaton based on the consderatons of multlnear algebra and dfferental geometry. We call t Tensor Subspace Analyss (). For an mage of sze n 1 n, t s represented as the second order tensor (or, matrx) n the tensor space R n1 R n. On the other hand, the face space s generally a submanfold embedded n R n1 R n. Gven some mages sampled from the face manfold, we can buld an adjacency graph to model the local geometrcal structure of the manfold. fnds a projecton that respects ths graph structure. The obtaned tensor subspace provdes an optmal lnear approxmaton to the face manfold n the sense of local sometry. Vaslescu shows how to extend SVD(PCA) to hgher order tensor data. We extend Laplacan based dea to tensor data. It s worthwhle to hghlght several aspects of the proposed approach here: 1. Whle tradtonal lnear dmensonalty reducton algorthms lke PCA, LDA and LPP fnd a map from R n to R l (l < n), fnds a map from R n1 R n to R l1 R l (l 1 < n 1,l < n ). Ths leads to structured dmensonalty reducton.. can be performed n ether supervsed, unsupervsed, or sem-supervsed manner. When label nformaton s avalable, t can be easly ncorporated nto the graph structure. Also, by preservng neghborhood structure, s less senstve to nose and outlers. 3. The computaton of s very smple. It can be obtaned by solvng two egenvector problems. The matrces n the egen-problems are of sze n 1 n 1 or n n, whch are much smaller than the matrces of sze n n (n = n 1 n ) n PCA, LDA and LPP. Therefore, s much more computatonally effcent n tme and storage. There are few parameters that are ndependently estmated, so performance n small data sets s very good. 4. explctly takes nto account the manfold structure of the mage space. The local geometrcal structure s modeled by an adjacency graph. 5. Ths paper s prmarly focused on the second order tensors (or, matrces). However, the algorthm and analyss presented here can also be appled to hgher order tensors. Tensor Subspace Analyss In ths secton, we ntroduce a new algorthm called Tensor Subspace Analyss for learnng a tensor subspace whch respects the geometrcal and dscrmnatve structures of the orgnal data space..1 Laplacan based Dmensonalty Reducton Problems of dmensonalty reducton has been consdered. One general approach s based on graph Laplacan []. The objectve functon of Laplacan egenmap s as follows: mn (f(x ) f(x j )) S j f j where S s a smlarty matrx. These optmal functons are nonlnear but may be expensve to compute. A class of algorthms may be optmzed by restrctng problem to more tractable famles of functons. One natural approach restrcts to lnear functon gvng rse to LPP [4]. In ths

3 paper we wll consder a more structured subset of lnear functons that arse out of tensor analyss. Ths provded greater computatonal benefts.. The Lnear Dmensonalty Reducton Problem n Tensor Space The generc problem of lnear dmensonalty reducton n the second order tensor space s the followng. Gven a set of data ponts X 1,,X m n R n1 R n, fnd two transformaton matrces U of sze n 1 l 1 and V of sze n l that maps these m ponts to a set of ponts Y 1,,Y m R l1 R l (l 1 < n 1,l < n ), such that Y represents X, where Y = U T X V. Our method s of partcular applcablty n the specal case where X 1,,X m M and M s a nonlnear submanfold embedded n R n1 R n..3 Optmal Lnear Embeddngs As we descrbed prevously, the face space s probably a nonlnear submanfold embedded n the tensor space. One hopes then to estmate geometrcal and topologcal propertes of the submanfold from random ponts ( scattered data ) lyng on ths unknown submanfold. In ths secton, we consder the partcular queston of fndng a lnear subspace approxmaton to the submanfold n the sense of local sometry. Our method s fundamentally based on LPP [4]. Gven m data ponts X = {X 1,,X m } sampled from the face submanfold M R n1 R n1, one can buld a nearest neghbor graph G to model the local geometrcal structure of M. Let S be the weght matrx of G. A possble defnton of S s as follows: S j = e X X j t, f X s among the k nearest neghbors of X j, or X j s among the k nearest neghbors of X ; 0, otherwse. where t s a sutable constant. The functon exp( X X j /t) s the so called heat kernel whch s ntmately related to the manfold structure. s the Frobenus norm of matrx,.e. A = j a j. When the label nformaton s avalable, t can be easly ncorporated nto the graph as follows: { S j = e X X j t, f X and X j share the same label; () 0, otherwse. Let U and V be the transformaton matrces. A reasonable transformaton respectng the graph structure can be obtaned by solvng the followng objectve functons: U T X V U T X j V S j (3) mn U,V j The objectve functon ncurs a heavy penalty f neghborng ponts X and X j are mapped far apart. Therefore, mnmzng t s an attempt to ensure that f X and X j are close then U T X V and U T X j V are close as well. Let Y = U T X V. Let D be a dagonal matrx, D = j S j. Snce A (AA T ), we see that: 1 U T X V U T X j V S j = 1 tr ( (Y Y j )(Y Y j ) T) S j j = 1 j ( tr ( Y Y T D Y Y T + Y j Y T j j Y Y T j ) S j Y Yj T j Y j Y T ) Sj (1)

4 ( D U T X V V T X T U ) S j U T X V V T Xj T U j (U T( D X V V T X T S j X V V T X T ) ) j U j. ( U T (D V S V ) U ) where D V = D X V V T X T and S V = j S jx V V T X T j. Smlarly, A = tr(a T A), so we also have 1 U T X V U T X j V S j j = 1 tr ( (Y Y j ) T (Y Y j ) ) S j j = 1 j ( tr ( Y T Y + Y T j Y j Y T Y j Y T j Y ) Sj D Y T Y j S j Y T Y j ) (V T( D X T UU T X X T UU T ) ) X j V j. ( V T (D U S U ) V ) where D U = D X T UUT X and S U = j S jx T UUT X j. Therefore, we should smultaneously mnmze tr ( U T (D V S V ) U ) and tr ( V T (D U S U ) V ). In addton to preservng the graph structure, we also am at maxmzng the global varance on the manfold. Recall that the varance of a random varable x can be wrtten as follows: var(x) = (x µ) dp(x), µ = xdp(x) M where M s the data manfold, µ s the expected value of x and dp s the probablty measure on the manfold. By spectral graph theory [3], dp can be dscretely estmated by the dagonal matrx D(D = j S j) on the sample ponts. Let Y = U T XV denote the random varable n the tensor subspace and suppose the data ponts have a zero mean. Thus, the weghted varance can be estmated as follows: var(y ) = Y D = ( V T ( tr(y T Y )D = ) D X T UU T X )V M tr(v T X T UU T X V )D ( V T D U V ) Smlarly, Y (Y Y T ), so we also have: ( var(y ) = tr(y Y T )D U T ( D X V V T X T ) ) U ( U T D V U ) Fnally, we get the followng optmzaton problems: tr ( U T (D V S V ) U ) mn U,V tr (U T D V U) (4)

5 tr ( V T (D U S U ) V ) mn U,V tr (V T (5) D U V ) The above two mnmzaton problems (4) and (5) depends on each other, and hence can not be solved ndependently. In the followng subsecton, we descrbe a smple computatonal method to solve these two optmzaton problems..4 Computaton In ths subsecton, we dscuss how to solve the optmzaton problems (4) and (5). It s easy to see that the optmal U should be the generalzed egenvectors of (D V S V,D V ) and the optmal V should be the generalzed egenvectors of (D U S U,D U ). However, t s dffcult to compute the optmal U and V smultaneously snce the matrces D V,S V,D U,S U are not fxed. In ths paper, we compute U and V teratvely as follows. We frst fx U, then V can be computed by solvng the followng generalzed egenvector problem: (D U S U )v = λd U v (6) Once V s obtaned, U can be updated by solvng the followng generalzed egenvector problem: (D V S V )u = λd V u (7) Thus, the optmal U and V can be obtaned by teratvely computng the generalzed egenvectors of (6) and (7). In our experments, U s ntally set to the dentty matrx. It s easy to show that the matrces D U,D V,D U S U, and D V S V are all symmetrc and postve sem-defnte. 3 Expermental Results In ths secton, several experments are carred out to show the effcency and effectveness of our proposed algorthm for face recognton. We compare our algorthm wth the Egenface (PCA) [8], Fsherface (LDA) [1], and Laplacanface (LPP) [5] methods, three of the most popular lnear methods for face recognton. Two face databases were used. The frst one s the PIE (Pose, Illumnaton, and Experence) database from CMU, and the second one s the ORL database. In all the experments, preprocessng to locate the faces was appled. Orgnal mages were normalzed (n scale and orentaton) such that the two eyes were algned at the same poston. Then, the facal areas were cropped nto the fnal mages for matchng. The sze of each cropped mage n all the experments s 3 3 pxels, wth 56 gray levels per pxel. No further preprocessng s done. For the Egenface, Fsherface, and Laplacanface methods, the mage s represented as a 4-dmensonal vector, whle n our algorthm the mage s represented as a (3 3)-dmensonal matrx, or the second order tensor. The nearest neghbor classfer s used for classfcaton for ts smplcty. In short, the recognton process has three steps. Frst, we calculate the face subspace from the tranng set of face mages; then the new face mage to be dentfed s projected nto d-dmensonal subspace (PCA, LDA, and LPP) or (d d)-dmensonal tensor subspace (); fnally, the new face mage s dentfed by nearest neghbor classfer. In our algorthm, the number of teratons s taken to be Experments on PIE Database The CMU PIE face database contans 68 subjects wth 41,368 face mages as a whole. The face mages were captured by 13 synchronzed cameras and 1 flashes, under varyng pose, llumnaton and expresson. We choose the fve near frontal poses (C05, C07, C09, C7,

6 Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne 60 Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne 5 15 Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne Dms d (d d for ) (a) 5 Tran Dms d (d d for ) (b) Tran Dms d (d d for ) (c) Tran Fgure 1: Error rate vs. dmensonalty reducton on PIE database Table 1: Performance comparson on PIE database Dms d (d d for ) (d) Tran Method 5 Tran Tran error dm tme(s) error dm tme(s) Baselne 69.9% % 4 - Egenfaces 69.9% % Fsherfaces 31.5% % Laplacanfaces.8% % % % Tran Tran Method error dm tme(s) error dm tme(s) Baselne 38.% 4-7.9% 4 - Egenfaces 38.1% % Fsherfaces 15.4% % Laplacanfaces 14.1% % % % C9) and use all the mages under dfferent llumnatons and expressons, thus we get 170 mages for each ndvdual. For each ndvdual, l(= 5,,, ) mages are randomly selected for tranng and the rest are used for testng. The tranng set s utlzed to learn the subspace representaton of the face manfold by usng Egenface, Fsherface, Laplacanface and our algorthm. The testng mages are projected nto the face subspace n whch recognton s then performed. For each gven l, we average the results over random splts. It would be mportant to note that the Laplacanface algorthm and our algorthm share the same graph structure as defned n Eqn. (). Fgure 1 shows the plots of error rate versus dmensonalty reducton for the Egenface, Fsherface, Laplacanface, and baselne methods. For the baselne method, the recognton s smply performed n the orgnal 4-dmensonal mage space wthout any dmensonalty reducton. Note that, the upper bound of the dmensonalty of Fsherface s c 1 where c s the number of ndvduals. For our algorthm, we only show ts performance n the (d d)-dmensonal tensor subspace, say, 1, 4, 9, etc. As can be seen, the performance of the Egenface, Fsherface, Laplacanface, and algorthms vares wth the number of dmensons. We show the best results obtaned by them n Table 1 and the correspondng face subspaces are called optmal face subspace for each method. It s found that our method outperforms the other four methods wth dfferent numbers of tranng samples (5,,, ) per ndvdual. The Egenface method performs the worst. It does not obtan any mprovement over the baselne method. The Fsherface and Laplacanface methods perform comparatvely to each each. The dmensons of the optmal subspaces are also gven n Table 1. As we have dscussed, can be mplemented very effcently. We show the runnng tme n seconds for each method n Table 1. As can be seen, s much faster than the

7 Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne Laplacanfaces (PCA+LPP) Fsherfaces (PCA+LDA) Egenfaces (PCA) Baselne Dms d (a) Tran Dms d (b) 3 Tran Dms d (c) 4 Tran Dms d (d) 5 Tran Fgure : Error rate vs. dmensonalty reducton on ORL database Table : Performance comparson on ORL database Method Tran 3 Tran error dm tme error dm tme Baselne.% 4 -.4% 4 - Egenfaces.% % Fsherfaces 5.% % Laplacanfaces.% % % % Tran 5 Tran Method error dm tme error dm tme Baselne 16.0% % 4 - Egenfaces 15.9% % Fsherfaces 9.17% % Laplacanfaces 8.54% % % %.97 Egenface, Fsherface and Laplacanface methods. All the algorthms were mplemented n Matlab 6.5 and run on a Intel P4.566GHz PC wth 1GB memory. 3. Experments on ORL Database The ORL (Olvett Research Laboratory) face database s used n ths test. It conssts of a total of 0 face mages, of a total of people ( samples per person). The mages were captured at dfferent tmes and have dfferent varatons ncludng expressons (open or closed eyes, smlng or non-smlng) and facal detals (glasses or no glasses). The mages were taken wth a tolerance for some tltng and rotaton of the face up to degrees. For each ndvdual, l(=,3,4,5) mages are randomly selected for tranng and the rest are used for testng. The expermental desgn s the same as that n the last subsecton. For each gven l, we average the results over random splts. Fgure 3. shows the plots of error rate versus dmensonalty reducton for the Egenface, Fsherface, Laplacanface, and baselne methods. Note that, the presentaton of the performance of the algorthm s dfferent from that n the last subsecton. Here, for a gven d, we show ts performance n the (d d)- dmensonal tensor subspace. The reason s for better comparson, snce the Egenface and Laplacanface methods start to converge after 70 dmensons and there s no need to show ther performance after that. The best result obtaned n the optmal subspace and the runnng tme (mllsecond) of computng the egenvectors for each method are shown n Table. As can be seen, our algorthm performed the best n all the cases. The Fsherface and Laplacanface methods performed comparatvely to our method, whle the Egenface method performed poorly.

8 4 Conclusons and Future Work Tensor based face analyss (representaton and recognton) s ntroduced n ths paper n order to detect the underlyng nonlnear face manfold structure n the manner of tensor subspace learnng. The manfold structure s approxmated by the adjacency graph computed from the data ponts. The optmal tensor subspace respectng the graph structure s then obtaned by solvng an optmzaton problem. We call ths Tensor Subspace Analyss method. Most of tradtonal appearance based face recognton methods (.e. Egenface, Fsherface, and Laplacanface) consder an mage as a vector n hgh dmensonal space. Such representaton gnores the spacal relatonshps between the pxels n the mage. In our work, an mage s naturally represented as a matrx, or the second order tensor. Tensor representaton makes our algorthm much more computatonally effcent than PCA, LDA, and LPP. Expermental results on PIE and ORL databases demonstrate the effcency and effectveness of our method. s lnear. Therefore, f the face manfold s hghly nonlnear, t may fal to dscover the ntrnsc geometrcal structure. It remans unclear how to generalze our algorthm to nonlnear case. Also, n our algorthm, the adjacency graph s nduced from the local geometry and class nformaton. Dfferent graph structures lead to dfferent projectons. It remans unclear how to defne the optmal graph structure n the sense of dscrmnaton. References [1] P.N. Belhumeur, J.P. Hepanha, and D.J. Kregman, Egenfaces vs. fsherfaces: recognton usng class specfc lnear projecton, IEEE. Trans. Pattern Analyss and Machne Intellgence, vol. 19, no. 7, pp , July [] M. Belkn and P. Nyog, Laplacan Egenmaps and Spectral Technques for Embeddng and Clusterng, Advances n Neural Informaton Processng Systems 14, 01. [3] Fan R. K. Chung, Spectral Graph Theory, Regonal Conference Seres n Mathematcs, number 9, [4] X. He and P. Nyog, Localty Preservng Projectons, Advance n Neural Informaton Processng Systems 16, Vancouver, Canada, December 03. [5] X. He, S. Yan, Y. Hu, P. Nyog, and H.-J. Zhang, Face Recognton usng Laplacanfaces, IEEE. Trans. Pattern Analyss and Machne Intellgence, vol. 7, No. 3, 05. [6] S. Rowes, and L. K. Saul, Nonlnear Dmensonalty Reducton by Locally Lnear Embeddng, Scence, vol 90, December 00. [7] J. B. Tenenbaum, V. de Slva, and J. C. Langford, A Global Geometrc Framework for Nonlnear Dmensonalty Reducton, Scence, vol 90, December 00. [8] M. Turk and A. Pentland, Egenfaces for recognton, Journal of Cogntve Neuroscence, 3(1):71-86, [9] M. A. O. Vaslescu and D. Terzopoulos, Multlnear Subspace Analyss for Image Ensembles, IEEE Conference on Computer Vson and Pattern Recognton, 03. [] K. Q. Wenberger and L. K. Saul, Unsupervsed Learnng of Image Manfolds by SemDefnte Programmng, IEEE Conference on Computer Vson and Pattern Recognton, Washngton, DC, 04. [11] J. Yang, D. Zhang, A. Frang, and J. Yang, Two-dmensonal PCA: a new approach to appearance-based face representaton and recognton, IEEE. Trans. Pattern Analyss and Machne Intellgence, vol. 6, No. 1, 04. [1] J. Ye, R. Janardan, Q. L, Two-Dmensonal Lnear Dscrmnant Analyss, Advances n Neural Informaton Processng Systems 17, 04.