Research Article Incremental Tensor Principal Component Analysis for Handwritten Digit Recognition

Transcription

1 Hndaw Publshng Copoaton athematcal Poblems n Engneeng, Atcle ID 89758, 0 pages Reseach Atcle Incemental enso Pncpal Component Analyss fo Handwtten Dgt Recognton Chang Lu,, ao Yan,, WeDong Zhao,, YongHong Lu,, Dan L,, Feng Ln, 3 and JLu Zhou 3 College of Infomaton Scence and echnology, Chengdu Unvesty, Chengdu 6006, Chna ey Laboatoy of Patten Recognton and Intellgent Infomaton Pocessng, Insttutons of Hghe Educaton of Schuan Povnce, Chengdu 6006, Chna 3 School of Compute Scence, Schuan Unvesty, Chengdu 60065, Chna Coespondence should be addessed to YongHong Lu; 84444@qq.com Receved 5 July 03; Revsed Septembe 03; Accepted Septembe 03; Publshed 30 Januay 04 Academc Edto: Paveen Agawal Copyght 04 C. Lu et al. hs s an open access atcle dstbuted unde the Ceatve Commons Attbuton Lcense, whch pemts unestcted use, dstbuton, and epoducton n any medum, povded the ognal wok s popely cted. o ovecome the shotcomngs of tadtonal dmensonalty educton algothms, ncemental tenso pncpal component analyss ( based on updated-svd technque algothm s poposed n ths pape. hs pape poves the elatonshp between, D,, and the gaph embeddng famewok theoetcally and deves the ncemental leanng pocedue to add sngle sample and multple samples n detal. he expements on handwtten dgt ecognton have demonstated that has acheved bette ecognton pefomance than that of vecto-based pncpal component analyss (, ncemental pncpal component analyss (, and multlnea pncpal component analyss ( algothms. At the same tme, also has lowe tme and space complexty.. Intoducton Patten ecognton and compute vson eque pocessng a lage amount of mult-dmensonal data, such as mage and vdeo data. Untl now, a lage numbe of dmensonalty educton algothms have been nvestgated. hese algothms poject the whole data nto a low-dmensonal space andconstuctnewfeatuesbyanalyzngthestatstcalelatonshp hdden n the data set. he new featues often gve good nfomaton o hnts about the data s ntnsc stuctue. As a classcal dmensonalty educton algothm, pncpal component analyss has been appled n vaous applcatons wdely. adtonal dmensonalty educton algothms geneally tansfom each mult-dmensonal data nto a vecto by concatenatng ows, whch s called Vectozaton. Such knd of the vectozaton opeaton has lagely nceased the computatonal cost of data analyss and seously destoys the ntnsc tenso stuctue of hgh-ode data. Consequently, tenso dmensonalty educton algothms ae developed based on tenso algeba[ 0]. Refeence [0] has summazed exstng multlnea subspace leanng algothms fo tenso data. Refeence [] has genealzed pncpal component analyss nto tenso space and pesented multlnea pncpal component analyss (. Refeence [] has poposed the gaph embeddng famewok to unfy all dmensonalty educton algothms. Futhemoe, tadtonal dmensonalty educton algothms geneally employ off-lne leanng to deal wth new added samples, whch aggavates the computatonal cost. o addess ths poblem, on-lne leanng algothms ae poposed [3, 4]. In patcula, efeence [5] hasdeveloped ncemental pncpal component analyss ( based on updated-svd technque. But most on-lne leanng algothms focus on vecto-based methods, only a lmted numbe of woks study ncemental leanng n tenso space [6 8]. o mpove the ncemental leanng n tenso space, ths pape pesents ncemental tenso pncpal component analyss ( based on updated-svd technque combnng tenso epesentaton wth ncemental leanng.

2 athematcal Poblems n Engneeng hs pape poves the elatonshp between, D,, and the gaph embeddng famewok theoetcally and deves the ncemental leanng pocedue to add sngle sample and multple samples n detal. he expements on handwtten dgt ecognton have demonstated that has acheved bette pefomance than vecto-based ncemental pncpal component analyss ( and multlnea pncpal component analyss ( algothms. At the same tme, also has lowe tme and space complexty than.. enso Pncpal Component Analyss In ths secton, we wll employ tenso epesentaton to expess hgh-dmensonal mage data. Consequently, a hghdmensonal mage dataset can be expessed as a tenso dataset X {X,...,X },wheex R I I N s an N dmensonal tenso and s the numbe of samples n the dataset. Based on the epesentaton, the followng defntons ae ntoduced. Defnton. FotensodatasetX, the mean tenso s defned as follows: X X R I I N. ( Defnton. he unfoldng matx of the mean tenso along the nth dmenson s called the mode-n mean matx and s defned as follows: X (n X (n R I n N ni. ( Defnton 3. Fo tenso dataset X, the total scatte tenso s defned as follows: Ψ X X m X, (3 m whee A s the nom of the tenso. Defnton 4. Fo tenso dataset X, the mode-n total scatte matx s defned as follows: C (n (X (n (X (n, (4 whee X (n s the mode-n mean matx and X (n s the mode-n unfoldng matx of tenso X. enso s ntoduced n [, 9]. he taget s to compute N othogonal pojectve matces {U (n R I n P n,n,...,n}to maxmze the total scatte tenso of the pojected low-dmensonal featue as follows: f{u (n,n,...,n}ag max Ψ y U (n ag max U (n m (5 Y m Y, whee Y m X m U ( U ( n U (n n+ U(n+ N U (N. Snce t s dffcult to solve N othogonal pojectve matces smultaneously, an teatve pocedue s employed to appoxmately compute these N othogonal pojectve matces. Geneally, snce t s assumed that the pojectve matces {U (,...,U (n,u (n+,...u (N } ae known, we can solve the followng optmzed poblem to obtan U (n : ag max m (C (n m C(n m, (6 whee C m (X m X U ( U ( n U (n n+ U(n+ N U (N and C (n m s the mode-n unfoldng matx of tenso C m. Accodng to the above analyss, t s easy to deve the followng theoems. heoem 5 (see []. Fotheodeoftensodatan,that s, fo the fst-ode tenso, the objectve functon of s equal to that of. Poof. Fo the fst-ode tenso, X m R I s a vecto, then (6s (C (n m m C(n m (U (X m X (X m X U. m (7 So fo fst-ode tenso s equal to vecto-based. heoem 6 (see []. Fotheodeoftensodatan,that s, fo the second-ode tenso, the objectve functon of s equal to that of Ds. Poof. Fo the second-ode tenso, X m R I I s a matx; t s needed to solve two pojectve matces U ( and U (,then (5becomes Y m Y m m U( (X m X U (. (8 he above equaton exactly s the objectve functon of BD (bdectonal D [0 ]. Lettng U ( I, the pojectve matx U ( s solved. In ths case, the objectve functon s Y m Y m m U( (X m X I. (9 hen the above equaton s smplfed nto the objectve functon of ow D [3, 4]. Smlaly, lettng U ( I,the pojectve matx U ( s solved; the objectve functon s Y m Y m m I (X m X U (. (0 hen the above equaton s smplfed nto the objectve functon of column D [3, 4].

3 athematcal Poblems n Engneeng 3 Although vecto-based and D can be espected as the specal cases of, and D employ dffeent technques to solve the pojectve matces. D caes out to ow data and column data, espectvely, and employs an teatve soluton to compute N pojectve matces. If t s supposed that the pojectve matces {U (,...,U (n,u (n+,...u (N } ae known, then U (n s solved. Equaton (6 can be expessed as follows: C (n ((X (n ((X (n ((X (n ((X (n k U (k N k k n k U (k U ( n ((X (n N k k n U ( n U ( n U ( n (X (n, whee U ( n U (N U (n+ U (n U (. Because ( heoem 7. can be unfed nto the gaph embeddng famewok []. Poof. Based on the basc knowledge of tenso algeba, we can get the followng: Y m Y m m vec(y m vec(y. (6 Lettng y m vec(y m, μvec(y,wecangetthefollowng: y μ (y μ(y μ N (y N j (y jy y N j j (y y N y ( j y j N N ( y j y j U ( n U ( n (U (N U (n+ U (n U ( ( + N ( y j ( j j y j (U (N U (n+ U (n U (. Basedontheoneckepoduct,wecangetthefollowng: y y y N ( j y j (A B A B, (A B(C D AC BD. (3 y N j y j + N ( y j ( j j y j So U ( n U ( n U (N U (N U (n+ U (n+ U (n U (n U ( U (. (4 Snce U ( R I I s an othogonal matx, U ( U ( I,,...,N, n,andu ( n U ( n I. If the dmensons of pojectve matces do not change n teatve pocedue, then C (n (X (n (X (n. (5 he above equaton s equal to BD. Because updates pojectve matces dung teatve pocedue, t has acheved bette pefomance than D. y y y N y j j j (,j,j,j W j y y,j W j y y j W j (y y +y j y j y y j y jy W j (y y j (y y j W j y y j F, (7

4 4 athematcal Poblems n Engneeng whee the smlaty matx W R ;foany,j,wehave W j /.So(6 can be wtten as follows: Y m Y m,j,j W j Y Y j W j X n U (n N So the theoem s poved. n X j n U (n 3. Incemental enso Pncpal Component Analyss N n. (8 3.. Incemental Leanng Based on Sngle Sample. Gven ntal tanng samples X old {X,...,X }, X R I I N, when a new sample X new R I I N s added, the tanng dataset becomes X{X old,x new }. he mean tenso of ntal samples s X old he covaance tenso of ntal samples s X. (9 C old X X old. (0 he mode-n covaance matx of ntal samples s C (n old (X (n X old (X (n X old. ( When the new sample s added, the mean tenso s X + + X + ( X +X new + (X old +X new. he mode-n covaance matx s expessed as follows: C (n + (X (n (X (n (X (n (X (n +(X (n new X(n (X (n new X(n, ( (3 whee the fst tem of (3s (X (n (X (n (X (n (X (n [(X (n [(X (n (X (n old + X(n old X(n old + X(n old X(n old +(X(n old X(n ] old +(X (n old X(n ] old (X(n old +(X (n old X(n + (X (n old (X(n old X(n +(X (n old (X (n old (X(n old +(X (n old X(n C (n old +(X(n old X(n old +X(n new + (X (n old X(n old +X(n new + C (n old + (+ (X(n old X(n new (X(n old X(n new. he second tem of (3s (X (n new X(n (X (n new X(n (X (n new X(n (X (n new X(n (X (n new X(n old +X(n new (X (n new X(n old +X(n new + (+ (X(n old X(n new (X(n old X(n new. + (4 (5

5 athematcal Poblems n Engneeng 5 Consequently, the mode-n covaance matx s updated as follows: C (n C (n old + + (X(n old X(n new (X(n old X(n new. (6 heefoe, when a new sample s added, the pojectve matces ae solved accodng to the egen decomposton on ( Incemental Leanng Based on ultple Samples. Gven an ntal tanng dataset X old {X,...,X }, X R I I N, when new samples ae added nto tanng dataset, X new {X +,...,X + }, then tanng dataset becomes nto X {X,...,X,X +,...,X + }.Inthscase,themeantenso s updated nto the followng: X + + X + ( X + + (X old +X new. Its mode-n covaance matx s C (n (X (n (X (n (X (n + + (X (n + (X (n X (X (n. he fst tem n (8 s wtten as follows: (X (n (X (n (7 (8 whee [(X (n old (X(n old X(n +(X (n old ] X (n old X(n (n old X old X(n X (n old X(n old +X (n old X(n +X (n old X(n (n old X old X(n old X (n X (n (n (n old +X X old 0, (X (n old X(n (+ (X(n old X(n new (X(n old X(n new. Puttng (30nto(9, then (9 becomes as follows: (X (n (X (n C (n old + (+ (X(n old X(n new (X(n old X(n new. he second tem n (8 s wtten as follows: whee + (X (n (X (n + C (n new +(X(n new X(n (X (n new X(n, (X (n new X(n (X (n new X(n (30 (3 (3 (33 (+ (X(n old X(n new (X(n old X(n new hen (3 becomes as follows: (X (n old (X(n old +(X (n old X(n + [(X (n old (X(n old X(n +(X (n old ], (9 + (X (n (X (n + C (n new + (+ (X(n old X(n new (X(n old X(n new (34 Puttng (3and(34nto(8, then we can get the followng: C (n C (n old +C(n new + + (X(n old X(n new (X(n old X(n new. (35

6 6 athematcal Poblems n Engneeng It s wothy to note that when new samples ae avalable, t has no need to ecompute the mode-n covaance matx of all tanng samples. We just have to solve the mode-n covaance matx of new added samples and the dffeence between ognal tanng samples and new added samples. Howeve, lke tadtonal ncemental, egen decomposton on C (n has been epeated once new samples ae added. It s cetan that the epeated egen decomposton on C (n wll cause heavy computatonal cost, whch s called the egen decomposton updatng poblem. Fo tadtonal vectobased ncemental leanng algothm, the updated-svd technque s poposed n [5] to ft the egen decomposton. hs pape wll ntoduce the updated-svd technque nto tenso-based ncemental leanng algothm. Fo ognal samples, the mode-n covaance matx s C (n old whee S (n old (X (n old (X(n old S (n old S(n old, (36 [X(n old,...,x(n X(n old ]. Accodng to the egen decomposton S (n old svd(uσv,wecangetthe followng: S (n old S(n old (UΣV (UΣV (37 UΣV VΣU UΣ U eg (C (n old. So t s easy to deve that the egen-vecto of C (n old s the left sngula vecto of S (n old and the egen-values coespond to the extacton of left sngula values of S (n old. Fo new samples, the mode-n covaance matx s C (n + new (X (n new (X(n new S (n new S(n new, + (38 whee S (n new [X(n X(n new,...,x(n X(n new ]. Accodng to (35, the updated mode-n covaance matx s defned as follows: C (n C (n old +C(n new + + (39 (X (n old X(n new (X(n old X(n new S (n S (n, whee S (n [S (n old,s(n new, /( + (X(n old X(n new ]. heefoe, the updated pojectve matx U (n s the egen-vectos coespondng to the lagest P n egen-values of S (n.heman steps of ncemental tenso pncpal component analyss ae lsted as follows: nput: ognal samples and new added samples, output: N pojectve matces. Step. Computng and savng eg (C (n old [U(n,Σ (n ]. (40 Step. Fo :N B[ S (n new, [ + (X(n old X(n new ]. (4 ] Pocessng QR decomposton fo the followng equaton: QR (I U (n U (n B. (4 Pocessng SVD decomposton fo the followng equaton: svd [ Σ (n U (n B] U Σ V. (43 0R Computng the followng equaton: old,b] ([U(n,Q] U Σ([ V(n 0 0I ] V. (44 [S (n hen the updated pojectve matx s computed as follows: end. U (n [U (n,q] U, (45 Step 3. Repeatng the above steps untl the ncemental leanng s fnshed he Complexty Analyss. FotensodatasetX{X,..., X }, X R I I N, wthout loss of genealty, t s assumed that all dmensons ae equal, that s, I I N I. Vecto-based convets all data nto vecto and constuctsadatamatxx R D, D I N. Fo vectobased, the man computatonal cost contans thee pats: the computaton of the covaance matx, the egen decomposton of the covaance matx, and the computaton of low-dmensonal featues. he tme complexty to compute covaance matx s O(I N, the tme complexty of the egen decomposton s O(I 3N, and the tme complexty to compute low-dmensonal featues s O(I N +I 3N. Lettng the teatve numbe be, the tme complexty to computng the mode-n covaance matx fo s O(NI N+, the tme complexty of egen decomposton s O(NI 3,andthetmecomplextytocomputelowdmensonal featues s O(NI N+,sothetotaltmecomplexty s O(NI N+ +NI 3. Consdeng the tme complexty, s supeo to. Fo, t s assumed that ncemental datasets ae added; has to ecompute mode-n covaance matx and conducts egen decomposton fo ntal dataset and ncemental dataset. he moe the tanng samples ae, the hghe tme complexty they have. If updated-svd s used, we only need to compute QR decomposton and SVD decomposton. he tme complexty of QR decomposton s O(NI N+. he tme complexty of the ank-k decomposton of the matx wth the sze of ( + I ( + I N s

7 athematcal Poblems n Engneeng Fgue : he samples n USPS dataset. he ecognton esults O(N( + Ik. It can be seen that the tme complexty of updated-svd has nothng to do wth the numbe of new added samples. akng the space complexty nto account, f tanng samples ae educed nto low-dmensonal space and the dmenson s D N n d n, then needs D N n I n bytes to save pojectve matces and needs N n I nd n bytes. So has lowe space complexty than. Fo ncemental leanng, both and need N n I n bytes to save ntal tanng samples; only need N n I n bytes to keep mode-n covaance matx. 4. Expements In ths secton, the handwtten dgt ecognton expements on the USPS mage dataset ae conducted to evaluate the pefomance of ncemental tenso pncpal component analyss. he USPS handwtten dgt dataset has 998 mages fom zeo to nne shown n Fgue. Fo each mage, the sze s 6 6.In ths pape,we choose 000 mages and dvde them nto ntal tanng samples, new added samples, and test samples. Futhemoe, the neaest neghbo classfe s employed to classfy the low-dmensonal featues. he ecognton esults ae compaed wth [6], [5], and []. Atfst,wechoose70samplesbelongngtofouclasses fom ntal tanng samples. Fo each tme of ncemental leanng, 70 samples whch belong to the othe two classes ae added. So afte thee tmes, the class labels of the tanng samples ae ten and thee ae 70 samples n each class. he estng samples of ognal tanng samples ae consdeed as testng dataset. All algothms ae mplemented n ALAB 00 on an Intel (R Coe ( 5-30 GHz wth 4 G RA. Fstly, 36 PCs ae peseved and fed nto the neaest neghbo classfe to obtan the ecognton esults. he esults ae plotted n Fgue. It can be seen that and ae bette than and fo ntal leanng; thepobableeasonsthatandemploytenso epesentaton to peseve the stuctue nfomaton. he ecognton esults unde dffeent leanng stages ae shownnfgues3, 4,and5. It can be seen that the ecognton esults of these fou methods always fluctuate volently he numbe of class labels Fgue : he ecognton esults fo 36 PCs of the ntal leanng. he ecognton esults he numbe of low-dmensonal featues Fgue 3: he ecognton esults of dffeent methods of the fst ncemental leanng. when the numbes of low-dmensonal featues ae small. Howeve, wth the ncement of the featue numbe, the ecognton pefomance keeps stable. Geneally and ae supeo to and. Although have compaatve pefomance at fst two leanng, begn to sumount afte the thd leanng. Fgue 6 has gven the best ecognton pecents of dffeent methods. WecangetthesameconclusonasshownnFgues3, 4,and 5. he tme and space complexty of dffeent methods ae shown n Fgues 7 and 8, espectvely.akngthetme complexty nto account, t can be found that at the stage of ntal leanng, has the lowest tme complexty. Wth

8 8 athematcal Poblems n Engneeng he ecognton esults he numbe of low-dmensonal featues Fgue 4: he ecognton esults of dffeent methods of the second ncemental leanng. he ecognton esults Class 6 Class 8 he numbe of class labels Class 0 Fgue 6: he compason of ecognton pefomance of dffeent methods. he ecognton esults he tme complexty (/s he numbe of low-dmensonal featues he numbe of class labels Fgue 5: he ecognton esults of dffeent methods of the thd ncemental leanng. Fgue 7: he compason of tme complexty of dffeent methods. the ncement of new samples, the tme complexty of and gows geatly and the tme complexty of and becomes stable. has slowe ncement than. he eason s that ntoduces ncemental leanng based on the updated-svd technque and avods decomposng the mode-n covaance matx of ognal samples agan. Consdeng the space complexty, t s easy to fnd that has the lowest space complexty among fou compaed methods. 5. Concluson hs pape pesents ncemental tenso pncpal component analyssbasedonupdated-svdtechnquetotakefulladvantage of edundancy of the space stuctue nfomaton and onlne leanng. Futhemoe, ths pape poves that and D ae the specal cases of and all of them canbeunfedntothegaphembeddngfamewok.hs

9 athematcal Poblems n Engneeng 9 he space complexty (/ he numbe of class labels Fgue 8: he compason of space complexty of dffeent methods. pape also analyzes ncemental leanng based on sngle sample and multple samples n detal. he expements on handwtten dgt ecognton have demonstated that pncpal component analyss based on tenso epesentaton s supeo to tenso pncpal component analyss based on vecto epesentaton. Although at the stage of ntal leanng, has bette ecognton pefomance than, the leanng capablty of becomes well gadually and exceeds. oeove, even f new samples ae added, the tme and space complexty of stll keep slowe ncement. Conflct of Inteests he authos declae that thee s no conflct of nteests egadng the publcaton of ths pape. Acknowledgments hspesentwokhasbeenfundedwthsuppotfomthe Natonal Natual Scence Foundaton of Chna (67448, the Doctoal Fund of nsty of Educaton of Chna ( , the Young Scentst Poject of Chengdu Unvesty (no. 03XJZ. Refeences [] H.Lu,.N.Platanots,andA.N.Venetsanopoulos, Uncoelated multlnea dscmnant analyss wth egulazaton and aggegaton fo tenso object ecognton, IEEE ansactons on Neual Netwoks,vol.0,no.,pp.03 3,009. [] C. Lu,. He, J.-L. Zhou, and C.-B. Gao, Dscmnant othogonal ank-one tenso pojectons fo face ecognton, n Intellgent Infomaton and Database Systems, N..Nguyen,C.-G. m,anda.janak,eds.,vol.659oflectue Notes n Compute Scence, pp. 03, 0. [3] G.-F. Lu, Z. Ln, and Z. Jn, Face ecognton usng dscmnant localty pesevng pojectons based on maxmum magn cteon, Patten Recognton, vol. 43, no. 0, pp , 00. [4]D.ao,X.L,X.Wu,andS.J.aybank, Genealtenso dscmnant analyss and Gabo featues fo gat ecognton, IEEE ansactons on Patten Analyss and achne Intellgence, vol. 9, no. 0, pp , 007. [5] F.Ne,S.Xang,Y.Song,andC.Zhang, Extactngtheoptmal dmensonalty fo local tenso dscmnant analyss, Patten Recognton,vol.4,no.,pp.05 4,009. [6] Z.-Z. Yu, C.-C. Ja, W. Pang, C.-Y. Zhang, and L.-H. Zhong, enso dscmnant analyss wth multscale featues fo acton modelng and categozaton, IEEE Sgnal Pocessng Lettes,vol.9,no.,pp.95 98,0. [7]S.J.Wang,J.Yang,.F.Sun,X.J.Peng,..Sun,andC. G. Zhou, Spase tenso dscmnant colo space fo face vefcaton, IEEE ansactons on Neual Netwoks and Leanng Systems, vol. 3, no. 6, pp , 0. [8] J.L.no,C.E.homaz,andD.F.Glles, enso-basedmultvaate statstcal dscmnant methods fo face applcatons, n Poceedngs of the Intenatonal Confeence on Statstcs n Scence, Busness, and Engneeng (ICSSBE,pp. 6,Septembe 0. [9] N. ang, X. Gao, and X. L, enso subclass dscmnant analyss fo ada taget classfcaton, Electoncs Lettes,vol.48,no. 8, pp , 0. [0] H.Lu,.N.Platanots,andA.N.Venetsanopoulos, Asuvey of multlnea subspace leanng fo tenso data, Patten Recognton,vol.44,no.7,pp ,0. [] H. Lu,. N. Platanots, and A. N. Venetsanopoulos, : multlnea pncpal component analyss of tenso objects, IEEE ansactons on Neual Netwoks, vol.9,no.,pp.8 39, 008. [] S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Ln, Gaph embeddng and extensons: a geneal famewok fo dmensonalty educton, IEEE ansactons on Patten Analyss and achne Intellgence,vol.9,no.,pp.40 5,007. [3] R.PlamondonandS.N.Sha, On-lneandoff-lnehandwtng ecognton: a compehensve suvey, IEEE ansactons on Patten Analyss and achne Intellgence,vol.,no.,pp.63 84, 000. [4] C.. Johnson, A suvey of cuent eseach on onlne communtes of pactce, Intenet and Hghe Educaton,vol.4,no.,pp.45 60,00. [5]P.Hall,D.ashall,andR.atn, egngandsplttng egenspace models, IEEE ansactons on Patten Analyss and achne Intellgence,vol.,no.9,pp ,000. [6] J. Sun, D. ao, S. Papadmtou, P. S. Yu, and C. Faloutsos, Incemental tenso analyss: theoy and applcatons, AC ansactons on nowledge Dscovey fom Data, vol.,no.3, atcle, 008. [7] J. Wen, X. Gao, Y. Yuan, D. ao, and J. L, Incemental tenso based dscmnant analyss: a new colo-based vsual tackng method, Neuocomputng, vol. 73, no. 4 6, pp , 00. [8] J.-G. Wang, E. Sung, and W.-Y. Yau, Incemental two-dmensonal lnea dscmnant analyss wth applcatons to face ecognton, Netwok and Compute Applcatons, vol.33,no.3,pp.34 3,00.

10 0 athematcal Poblems n Engneeng [9]X.Qao,R.Xu,Y.-W.Chen,.Igaash,.Nakao,andA. ashmoto, Genealzed N-Dmensonal Pncpal Component Analyss (GND- based statstcal appeaance modelng of facal mages wth multple modes, IPSJ ansactons on Compute Vson and Applcatons,vol.,pp.3 4,009. [0]H.ong,X.L,L.Wang,E..eoh,J.-G.Wang,andR. Venkateswalu, Genealzed D pncpal component analyss, n Poceedngs of the IEEE Intenatonal Jont Confeence on Neual Netwoks (IJCNN 05,vol.,pp.08 3,August005. [] D. Zhang and Z.-H. Zhou, (D : two-dectonal twodmensonal fo effcent face epesentaton and ecognton, Neuocomputng,vol.69,no. 3,pp.4 3,005. [] J. Ye, Genealzed low ank appoxmatons of matces, achne Leanng,vol.6,no. 3,pp.67 9,005. [3] J. Yang, D. Zhang, A. F. Fang, and J.-Y. Yang, wo-dmensonal : a new appoach to appeaance-based face epesentaton and ecognton, IEEE ansactons on Patten Analyss and achne Intellgence,vol.6,no.,pp.3 37,004. [4] J. Yang and J.-Y. Yang, Fom mage vecto to matx: a staghtfowad mage pojecton technque- vs., Patten Recognton,vol.35,no.9,pp ,00. [5] J. wok and H. Zhao, Incemental egen decomposton, n Poceedngs of the Intenatonal Confeence on Atfcal Neual Netwoks (ICANN 03, pp , Istanbul, ukey, June 003. [6] P. N. Belhumeu, J. P. Hespanha, and D. J. egman, Egenfaces vs. fshefaces: ecognton usng class specfc lnea pojecton, IEEE ansactons on Patten Analyss and achne Intellgence,vol.9,no.7,pp.7 70,997.

11 Advances n Opeatons Reseach Hndaw Publshng Copoaton Advances n Decson Scences Hndaw Publshng Copoaton Appled athematcs Algeba Hndaw Publshng Copoaton Hndaw Publshng Copoaton Pobablty and Statstcs he Scentfc Wold Jounal Hndaw Publshng Copoaton Hndaw Publshng Copoaton Intenatonal Dffeental Equatons Hndaw Publshng Copoaton Submt you manuscpts at Intenatonal Advances n Combnatocs Hndaw Publshng Copoaton athematcal Physcs Hndaw Publshng Copoaton Complex Analyss Hndaw Publshng Copoaton Intenatonal athematcs and athematcal Scences athematcal Poblems n Engneeng athematcs Hndaw Publshng Copoaton Hndaw Publshng Copoaton Hndaw Publshng Copoaton Dscete athematcs Hndaw Publshng Copoaton Dscete Dynamcs n Natue and Socety Functon Spaces Hndaw Publshng Copoaton Abstact and Appled Analyss Hndaw Publshng Copoaton Hndaw Publshng Copoaton Intenatonal Stochastc Analyss Optmzaton Hndaw Publshng Copoaton Hndaw Publshng Copoaton