Robust Principal Component Analysis with Non-Greedy l 1 -Norm Maximization

Transcription

1 Proceedngs of he Tweny-Second Inernaonal Jon Conference on Arfcal Inellgence Robus Prncpal Componen Analyss wh Non-Greedy l -Norm Maxmzaon Fepng Ne, Heng Huang, Chrs Dng, Djun Luo, Hua Wang Deparmen of Compuer Scence and Engneerng Unversy of Texas a Arlngon, Arlngon, Texas 7609, USA {fepngne,djun.luo,huawangcs}@gmal.com, {heng,chqdng}@ua.edu Absrac Prncpal Componen Analyss (PCA) s one of he mos mporan mehods o handle hghdmensonal daa. However, he hgh compuaonal complexy makes hard o apply o he large scale daa wh hgh dmensonaly, and he used l 2 -norm makes sensve o oulers. A recen work proposed prncpal componen analyss based on l -norm mzaon, whch s effcen and robus o oulers. In ha work, a greedy sraegy was appled due o he dffculy of drecly solvng he l -norm mzaon problem, whch s easy o ge suck n local soluon. In hs paper, we frs propose an effcen opmzaon algorhm o solve a general l -norm mzaon problem, and hen propose a robus prncpal componen analyss wh non-greedy l -norm mzaon. Expermenal resuls on real world daases show ha he nongreedy mehod always obans much beer soluon han ha of he greedy mehod. Inroducon In many real-world applcaons such as face recognon and ex caegorzaon, he dmensonaly of daa are usually very hgh. Drecly handle he hgh-dmensonal daa s compuaonally expensve and a he same me he performance could be very poor because he number of avalable daa s always lmed and he nose n he daa would ncrease dramacally as he dmensonaly ncreases. Dmensonaly reducon or dsance merc learnng s one of he mos mporan and effecve mehods o handle hghdmensonal daa [Xang e al., 2008; Yang e al., 2009; Ne e al., 200b]. Among he dmensonaly reducon mehods, Prncpal Componen Analyss (PCA) s one of he mos wdely appled mehods due o s smplcy and effecveness. Gven a daase, PCA fnds a projecon marx o mze he varance of he projeced daa pons under hs projecon marx, and he srucure of orgnal daa could be effecvely preserved under he projecon. Ths research was funded by US NSF CCF , , , NSF DMS , NSF CNS , In he pas decades, he radonal PCA has been successfully appled n many problems. However, has several drawbacks. Frs, has o perform Sngular Vecor Decomposon (SVD) on npu daa marx or egen-decomposon on covarance marx, whch s compuaonally expensve and dffcul o apply when boh he number of daa and he dmensonaly are very hgh. Second, s sensve o oulers because s nrnscally based on l 2 -norm and he oulers wh large norm can be exaggeraed by usng he l 2 - norm. Many works [Baccn e al., 996; Aanas e al., 2002; De La Torre and Black, 2003; Ke and Kanade, 2005; Dng e al., 2006; Wrgh e al., 2009] have devoed effor o allevae hs problem and mprove he robusness o oulers. [Baccn e al., 996; Ke and Kanade, 2005] consder he problem of fndng a subspace such ha he sum of l -norm dsances of daa pons o he subspace s mnmzed. Alhough he robusness o oulers s mproved by hs mehod, s compuaonally expensve and more mporanly, he used l -norm s no nvaran o roaon and he performance usually very poor when appled o K-means cluserng [Dng e al., 2006]. To solve hs problem, R -PCA was proposed whch s nvaran o roaon and demonsraed favorable performance [Dng e al., 2006]. However, R -PCA eravely performs he subspace eraon algorhm n he hgh-dmensonal orgnal space, whch s compuaonally expensve. The exenson of R -PCA o ensor verson can be found n [Huang and Dng, 2008]. Recenly, a robus prncpal componen analyss based on l -norm mzaon s proposed n [Kwak, 2008], anda smlar work can be found n [Galpn and Hawkns, 987]. Ths mehod s nvaran o roaon and s also robus o oulers. In [Kwak, 2008], an effcen algorhm s proposed o solve he l -norm mzaon problem. The algorhm only need o perform marx-vecor mulplcaon, and hus can be appled n he case ha boh he number of daa and he dmensonaly are very hgh. Some works on s ensor verson and supervsed verson can be found n [L e al., 200; Lu e al., 200; Pang e al., 200]. Due o he dffculy of drecly solvng he l -norm mzaon problem, all hese works use a greedy sraegy o solve. Specfcally, he projecon drecons are sequenally opmzed one by one. Ths knd of greedy mehod s easy o ge suck n a local soluon. In hs paper, we focus on solvng he l -norm mzaon problem. We frs propose an effcen opmzaon al- 433

2 gorhm o solve a general l -norm mzaon problem. Theorecal analyss guaranees he algorhm wll converge and usually converge o a local soluon. The l -norm mzaon problem n [Kwak, 2008] s a specal case of he general problem, and hus he proposed opmzaon algorhm can be used o solve drecly n a non-greedy sraegy. Tha s, all he projecon drecons can be opmzed smulaneously. Expermenal resuls on real daases show ha he non-greedy mehod always obans much beer soluon han ha of he greedy mehod. The res of hs paper s organzed as follows: We gve a bref revew of he work [Kwak, 2008] n 2. In 3, we propose an effcen algorhm o solve a general l -norm mzaon problem and gve heorecal analyss on. Based on he algorhm, we solve he problem for he prncpal componen analyss wh greedy l -norm mzaon n 4and propose a prncpal componen analyss wh non-greedy l - norm mzaon n 5. In 6, we presen expermens o verfy he effecveness of he proposed mehod. Fnally, we draw he conclusons n 7. 2 Relaed work Suppose he gven daa are X =[x,x 2,,x n ] R d n, where n and d are he number and he dmensonaly of daa pons respecvely. Whou loss of generaly, he daa {x } n are assumed o be cenralzed,.e., n x =0. Denoe he projecon marx W =[w,w 2,,w m ] R d m. Tradonal PCA mehod mzes he varance of daa n he projeced subspace, and o solve he followng opmzaon Tr(W T S W ), () where S = n XXT s he covarance marx, I s he deny marx and Tr( ) s he race operaor of a marx. Denoe he l -norm and l 2 -norm of a vecor by and 2, respecvely. The problem () can be reformulaed as he followng W T x 2 n 2. (2) Movaed by hs reformulaon, a recen work [Kwak, 2008] proposed o mze he l -norm nsead of he l 2 -norm n PCA, and hus he robusness o oulers s mproved. Then he problem becomes: n W T x (3) Drecly solvng hs problem s dffcul, hus he auhor use a greedysraegyo solve. Specfcally, he m projecon drecons {w,w 2,..., w m } are opmzed one by one. The frs projecon drecon w s opmzed by solvng he followng w T w= w T x (4) Afer he (k )-h projecon drecon w k has been obaned, he daa marx X s ransformed o X = X w k (w k ) T X, and hen he k-h projecon drecon w k s opmzed by solvng he followng w T k x (5) wk T w k= In hs greedy mehod, he only problem needed o solve s he problem (5) for each k. The work n [Kwak, 2008] proposed an erave algorhm o solve hs problem. The dealed procedure s: ) =. Inalze wk Rd such ha (wk ) 2 =. 2) For each,f(wk )T x < 0, α = oherwse α =. 3) Le v = n α x,andw + k = v/ v 2, = +. 4) Ieravely perform seps 2 and 3 unl converges. In order o guaranee he algorhm converges o a local mum, he algorhm adds an addonal judgemen afer convergence. If here exss such ha (wk )T x =0,hen le wk =(w k + w)/ w k + w 2 andgoosep2,where w s a small nonzero random vecor. However, such operaon mgh make he algorhm nermnable (for example, suppose here s a daa pon x ha exacly locaes on he mean of he daa se, hen x wll be zero afer cenralzaon, and hus (w ) T x s always zero for any w ). Moreover, s possble ha here exss such ha (wk )T x =0a he global mum. In hs case, he algorhm can no have he chance o fnd he global mum. Subsequenly, we wll frs propose an effcen algorhm o solve a general l -norm mzaon problem. Based on, we also solve he problem (5) for he prncpal componen analyss wh greedy l -norm mzaon and propose he prncpal componen analyss wh non-greedy l -norm mzaon by drecly solve he problem (3). The addonal judgemen s no requred n he new algorhms o oban a local soluon, and he non-greedy mehod always obans much beer soluon han ha of he greedy mehod n pracce. 3 An effcen algorhm o solve a general l -norm mzaon problem Consder a general l -norm mzaon problem as follows (we assume ha he objecve has an upper bound) : v C f(v)+ g (v). (6) where f(v) and g (v) for each are arbrary funcons, and v C s an arbrary consran. Alhough here are many mehods o solve he l -norm mnmzaon problem n compressed sensng and sparse learnng [Donoho, 2006; Ne e al., 200a], hese mehods can no be used o solve he l -norm mzaon problem. Rewrng he problem (6) as he followng v C f(v)+ α g (v), (7) where α = sgn(g (v)), andsgn( ) s he sgn funcon defned as follows: sgn(x) = f x > 0, sgn(x) = f x<0, andsgn(x) =0f x =0. Noe ha α deps on 434

3 v and hus s also a unknown varable. We propose an erave algorhm o solve he problem (6), and prove ha he proposed erave algorhm wll monooncally ncrease he objecve of he problem (6) n each eraon, and wll usually converge o a local soluon. The algorhm s descrbed n Algorhm. In each eraon, α s calculaed by curren soluon v, and he soluon v s updaed wh he curren α. The erave procedure s repeaed unl he algorhm converges. Inalze v C, =; whle no converge do. For each, calculae α = sgn(g (v )) ; 2. v + = arg f(v)+ α g (v) ; v C 3. = +; Oupu: v. Algorhm : An effcen algorhm o solve a general l -norm mzaon problem (6). 3. Theorecal analyss of he opmzaon algorhm The convergence of he Algorhm s demonsraed n he followng heorem: Theorem The Algorhm wll monooncally ncrease he objecve of he problem (6) n each eraon. Proof: Accordng o he sep 2 n Algorhm, for each eraon we have f(v + )+ α g (v + ) f(v )+ α g (v ) (8) For each, noe ha α = sgn(g (v )), so we have ha g (v + ) = sgn(g (v + ))g (v + ) sgn(g (v ))g (v + ) = α g (v + ). Then g (v + ) α g (v + ) and noe ha g (v ) α g (v )=0,wehave: g (v + ) α g (v + ) g (v + ) α g (v + ) 0 g (v + ) α g (v + ) g (v ) α g (v ) (9) Eq. (9) holds for every, hus we have ( g (v + ) α g (v + )) ( g (v ) α g (v )) (0) Combnng Eq. (8) and Eq. (0), we arrve a f(v + )+ g (v + ) f(v )+ g (v ) () Thus he Algorhm wll monooncally ncrease he objecve of he problem (6) n each eraon. As he objecve of he problem (6) has an upper bound, Theorem ndcaes ha he Algorhm wll converge. The followng heorem shows ha he soluon n he convergence wll sasfy he KKT condon. Theorem 2 The soluon of he Algorhm n he convergence wll sasfy he KKT condon of he problem (6). Proof: The Lagrangan funcon of he problem (6) s L(v, λ) =f(v)+ g (v) h(v, λ), (2) where h(λ, v) s he Lagrangan erm o encode he consran v Cnproblem (6). Takng he dervave of L(v, λ) w.r. v, and seng he dervave o zero, we have: L(v, λ) v = f (v)+ α g h(v, λ) (v) =0, (3) v where α = sgn(g (v)). Suppose he Algorhm converges o a soluon v, from sep 2 n Algorhm we have v = arg v C f(v )+ α g (v ), (4) where α = sgn(g (v )). Accordng o he KKT condon [Boyd and Vandenberghe, 2004] of he problem n Eq. (4), we know ha he soluon v sasfes Eq. (3), whch s he KKT condon of he problem (6). In general, sasfyng he KKT condon usually ndcaes ha he soluon s a local opmum soluon. Theorem 2 ndcaes ha he Algorhm wll usually converge o a local soluon. We can see ha boh he problem (5) and he problem (3) are he specal cases of he problem (6), so we can use he proposed Algorhm o solve hese wo problems. The key sep of he Algorhm s o solve he problem n sep 2. In he nex wo secons, we gve dealed dervaon and algorhm o solve he problem (5) and he problem (3), respecvely. 4 Prncpal componen analyss wh greedy l -norm mzaon revsed Recall ha he prncpal componen analyss wh greedy l - norm mzaon only need o solve he followng w T x. (5) w T w= As descrbed n Secon 2, an algorhm proposed n [Kwak, 2008] can solve. In hs secon, we solve based on he Algorhm, and compare he dfferences beween hese wo algorhms. Accordng o he Algorhm, he key sep o solve he problem (5) s o solve he followng α w T x, (6) w T w= where α = sgn((w ) T x ). Denoe m = can rewre he problem (6) as n α x,henwe w T m, (7) w T w= When x =0, 0 s a subgraden of funcon x, sosgn(x) s he graden or a subgraden of he funcon x n all he cases. 435

4 The Lagrangan funcon of he above problem s L(w, λ) =w T m λ(w T w ), (8) Takng he dervave of L(w, λ) w.r. w, and seng he dervave o zero, we have w = m/λ. Thenλ = m 2 accordng o he consran w T w =. So he opmal soluon o he problem (6) s w = m/ m 2. Based on he Algorhm, he algorhm o solve he prncpal componen analyss wh greedy l -norm mzaon s descrbed n Algorhm 2. We can see ha he Algorhm 2 s almos he same as he one descrbed n Secon 2, excep ha he values of α are dfferen when (w k )T x =0 and he Algorhm 2 does no have he addonal judgemen when he algorhm converges. When (w k )T x =0, α =0 n Algorhm 2 whle α =n he algorhm proposed n [Kwak, 2008]. Usng he Algorhm 2 whou he addonal judgemen, we can also oban a local soluon accordng o Theorem 2. From Algorhm 2 we can see ha he algorhm s effcen and only nvolves marx-vecor mulplcaon. The compuaonal complexy s O(ndm), wheren, d, m s he number of daa, dmenson of orgnal daa and he dmenson of he projeced daa respecvely, and s he erave number. In pracce, he algorhm usually converges n en eraons. Therefore, he compuaonal complexy of he algorhm s lnear w.r. boh daa number and daa dmenson, whch ndcaes he algorhm s applcable n he case ha boh daa number and daa dmenson are very hgh. If he daa are sparse, he compuaonal complexy s furher reduced o O(nsm), wheres s he averaged number of non-zeros elemens n a daa pon. Inpu: X, m,wherex s cenralzed Inalze W =[w,w 2,..., w m ] R d m such ha W T W = I ; for k =o m do Le wk = w k, =; whle no converge do. α = sgn((wk )T x ) ; 2. m = n α x,andw + k = m/ m 2 ; 3. = +; Le X = X wk (w k )T X and w k = wk ; Oupu: W R d m. Algorhm 2: Prncpal componen analyss wh greedy l -norm mzaon. 5 Prncpal componen analyss wh non-greedy l -norm mzaon The orgnal problem n [Kwak, 2008] s o solve he followng W T x. (9) Inpu: X, m,wherex s cenralzed Inalze W R d m such ha W T W = I, =; whle no converge do. α = sgn((w ) T x ), M = n x α T ; 2. Calculae he SVD of M as M = UΛV T,Le W + = UV T ; 3. = +; Oupu: W R d m. Algorhm 3: Prncpal componen analyss wh nongreedy l -norm mzaon. Snce drecly solvng hs problem s dffcul, [Kwak, 2008] urns o solve by a greedy mehod. In hs paper, we propose a non-greedy mehod o drecly solve he problem (9). Based on he Algorhm, he key sep o solve he problem (9) s o solve he followng α T W T x (20) where he vecors α = sgn((w ) T x ). Denoe M = x α T, hen we can rewre he problem (20) as Tr(W T M) (2) Suppose he SVD of M s M = UΛV T,henTr(W T M) can be rewren as: Tr(W T M) = Tr(W T UΛV T ) = Tr(ΛV T W T U) = Tr(ΛZ) = λ z (22) where Z = V T W T U, λ and z are he (, )-h elemen of marx λ and Z respecvely. Noe ha Z s an orhonormal marx,.e. Z T Z = I, so z. On he oher hand, λ 0 snce λ s sngular value of M. Therefore, Tr(W T M)= λ z λ, and when z =( c), he equaly holds. Tha s o say, Tr(W T M) reaches he mum when Z = I. Recall ha Z = V T W T U, hus he opmal soluon o he problem Eq. (2) s W = UZ T V T = UV T. (23) Based on he Algorhm, he algorhm o solve he prncpal componen analyss wh non-greedy l -norm mzaon s descrbed n Algorhm 3. Accordng o Theorem 2, we can usually oban a local soluon. From Algorhm 2 we can see ha he algorhm s also effcen. Noe ha n m n pracce, hus he compuaonal complexy of he algorhm s O(ndm), whch s he same as ha of he greedy mehod. Smlarly, he algorhm usually converges n en eraons n pracce. Therefore, he compuaonal complexy of he algorhm s also lnear w.r. boh 436

5 Table : Daase Descrpons. Daa se Sze Dmensons Classes Jaffe Ums Yale Col Palm USPS Objecve Dmenson x (a) Jaffe Objecve Dmenson x (b) Ums daa number and daa dmenson, whch ndcaes he algorhm s applcable n he case ha boh daa number and daa dmenson are very hgh. If he daa are sparse, he compuaonal complexy s furher reduced o O(nsm). 5. Exensons o kernel and ensor cases Smlar o radonal PCA, he robus prncpal componen analyss wh l -norm mzaon s also a lnear mehod, and s dffcul o handle daa well wh non-gaussan dsrbuon. A popular echnque o deal wh hs problem s exng he lnear mehod o kernel mehod. Obvously, he robus prncpal componen analyss wh l -norm mzaon s nvaran o roaon and shf, so hs lnear mehod sasfes he condons n a generalzed kernel framework n [Zhang e al., 200], and hus can be kernelzed usng he framework. Specfcally, he gven daa are ransformed by KPCA [Schölkopf e al., 998], and hen perform Algorhm 3 usng he ransformed daa as npu. Anoher problem of he prncpal componen analyss s ha he mehod can only handle vecor daa. For 2D ensor or hgher order ensor daa, we have o vecorze he daa o very hgh-dmensonal vecors n order o apply hs mehod. Ths approach wll desroy he srucural nformaon of ensor daa and also make he compuaonal burden very heavy. A popular echnque o deal wh hs problem s exng he vecor mehod o ensor mehod. As he problem (9) of he prncpal componen analyss wh l -norm mzaon only ncludes lnear operaor W T x, can be easly exed o he ensor mehod o handle ensor daa drecly. For smplcy, we only brefly dscuss he case of 2D ensor, hgh order ensor cases can be readly exed by replacng he lnear operaor W T x wh ensor operaor [Lahauwer, 997]. Suppose he gven daa are X = [X,X 2,..., X n ] R r c n, where each daa X R r c s a 2D ensor, n s he number of daa pons. Smlarly, we assume ha {X } n are cenered,.e., n X = 0. In he 2D ensor case, lnear operaor W T x s replaced by U T X V,whereU R r r and V R c c are wo projecon marces. Correspondngly, he problem (9) becomes: U T U=I r,v T V =I c U T X V (24) As n oher ensor mehod, problem (24) can be solved by alernave opmzaon echnque (also named block coordnae descen). Specfcally, when fxng U, he problem (24) reduced o he problem (9), and hus he V can be opmzed by Algorhm 3. Smlarly, U can also be opmzed by Al- Objecve Objecve Dmenson (c) Yale 2000 Dmenson (e) Palm Objecve Objecve Dmenson (d) Col20 20 Dmenson (f) USPS Fgure : Objecve values n Eq. (3) wh dfferen dmensons obaned by PCA-l greedy and PCA-l nongreedy, respecvely. gorhm 3 when fxng V. The procedure s eravely performed unl converges. 6 Expermens In hs secon, we presen expermens o demonsrae he effecveness of he proposed prncpal componen analyss wh non-greedy l -norm mzaon (denoed by PCA- l nongreedy) compared o he greedy mehod (denoed by PCA-l greedy). We use sx mage daases from dfferen domans o perform he expermens. A bref descrpon of he daases are shown n Table. In hs expermen, we sudy he greedy and non-greedy opmzaon mehods, and compare he objecve values n Eq.(3) obaned by hese wo opmzaon mehods. In he frs expermen, we run he greedy mehod and he non-greedy mehod wh dfferen projeced dmensons m and he same nalzaon on each daase. The projeced dmensons vares from 5 o 00 wh he nerval 5. The resuls are shown n Fgure. In he second expermen, we run he greedy mehod and he non-greedy mehod 50 mes wh he projeced dmensons m =50on each daase. In each me, he wo mehods use he same nalzaon. The resuls are shown n Table 2. From Fgure and Table 2 we can see, he proposed nongreedy mehod obans much hgher objecve values han ha of he greedy mehod n all he cases. The resuls ndcae ha 437

6 Table 2: Objecve values n Eq.(3) wh dmenson 50 obaned by PCA-l greedy and PCA-l nongreedy, respecvely. The number of nalzaon s 50. Daa se PCA-l greedy PCA-l nongreedy Mn Max Mn/Max Mean Mn Max Mn/Max Mean Jaffe Ums Yale Col Palm USPS he proposed non-greedy mehod always obans much beer soluon o he l -norm mzaon problem (3) han he pervous greedy mehod. 7 Conclusons A robus prncpal componen analyss wh non-greedy l - norm mzaon s proposed n hs paper. We frs propose an effcen opmzaon algorhm o solve a general l - norm mzaon problem, and he algorhm wll usually converge o a local soluon by heorecal analyss. Based on he algorhm, we drecly solve he l -norm mzaon problem where he projecon drecons are opmzed smulaneously. Smlarly o he prevous greedy mehod, he robus prncpal componen analyss wh non-greedy l -norm mzaon s also effcen, and s easy o ex o s kernel verson or ensor verson. Expermenal resuls on sx real world mage daases show ha he proposed non-greedy mehod always obans much beer soluon han ha of he greedy mehod. References [Aanas e al., 2002] H. Aanas, R. Fsker, K. Asrom, and J.M. Carsensen. Robus facorzaon. IEEE Transacons on PAMI, 24(9):25 225, [Baccn e al., 996] A. Baccn, P. Besse, and A. de Faguerolles. A L-norm PCA and heursc approach. In Proceedngs of he Inernaonal Conference on Ordnal and Symbolc Daa Analyss, volume, pages , 996. [Boyd and Vandenberghe, 2004] S.P. Boyd and L. Vandenberghe. Convex opmzaon. Cambrdge Unversy Press, [De La Torre and Black, 2003] F. De La Torre and M.J. Black. A framework for robus subspace learnng. Inernaonal Journal of Compuer Vson, 54():7 42, [Dng e al., 2006] Chrs H. Q. Dng, Dng Zhou, Xaofeng He, and Hongyuan Zha. R-PCA: roaonal nvaran L-norm prncpal componen analyss for robus subspace facorzaon. In ICML, pages , [Donoho, 2006] Davd L. Donoho. Compressed sensng. IEEE Transacons on Informaon Theory, 52(4): , [Galpn and Hawkns, 987] J.S. Galpn and D.M. Hawkns. Mehods of L esmaon of a covarance marx. Compuaonal Sascs & Daa Analyss, 5(4):305 39, 987. [Huang and Dng, 2008] Heng Huang and Chrs H. Q. Dng. Robus ensor facorzaon usng r norm. In CVPR, [Ke and Kanade, 2005] Q. Ke and T. Kanade. Robus L norm facorzaon n he presence of oulers and mssng daa by alernave convex programmng. In CVPR, pages , [Kwak, 2008] N. Kwak. Prncpal componen analyss based on L- norm mzaon. IEEE Transacons on PAMI, 30(9): , [Lahauwer, 997] Leven De Lahauwer. Sgnal Processng based on Mullnear Algebra. PhD hess, Facule der Toegepase Weenschappen. Kaholeke Unverse Leuven, 997. [L e al., 200] Xuelong L, Yanwe Pang, and Yuan Yuan. L- norm-based 2DPCA. IEEE Transacons on Sysems, Man, and Cybernecs, Par B, 38(4), 200. [Lu e al., 200] Yang Lu, Yan Lu, and Keh C. C. Chan. Mullnear mum dsance embeddng va l-norm opmzaon. In AAAI, 200. [Ne e al., 200a] Fepng Ne, Heng Huang, Xao Ca, and Chrs Dng. Effcen and robus feaure selecon va jon l 2,-norms mnmzaon. In NIPS, 200. [Ne e al., 200b] Fepng Ne, Dong Xu, Ivor Wa-Hung Tsang, and Changshu Zhang. Flexble manfold embeddng: A framework for sem-supervsed and unsupervsed dmenson reducon. IEEE Transacons on Image Processng, 9(7):92 932, 200. [Pang e al., 200] Yanwe Pang, Xuelong L, and Yuan Yuan. Robus ensor analyss wh L-norm. IEEE Transacons on Crcus and Sysems for Vdeo Technology, 20(2):72 78, 200. [Schölkopf e al., 998] Bernhard Schölkopf, Alex J. Smola, and Klaus-Rober Müller. Nonlnear componen analyss as a kernel egenvalue problem. Neural Compuaon, 0(5):299 39, 998. [Wrgh e al., 2009] J. Wrgh, A. Ganesh, S. Rao, and Y. Ma. Robus prncpal componen analyss: Exac recovery of corruped low-rank marces va convex opmzaon. NIPS, [Xang e al., 2008] Shmng Xang, Fepng Ne, and Changshu Zhang. Learnng a mahalanobs dsance merc for daa cluserng and classfcaon. Paern Recognon, 4(2): , [Yang e al., 2009] Y Yang, Yueng Zhuang, Dong Xu, Yunhe Pan, Dacheng Tao, and Sephen J. Maybank. Rereval based neracve caroon synhess va unsupervsed b-dsance merc learnng. In ACM Mulmeda, pages 3 320, [Zhang e al., 200] Changshu Zhang, Fepng Ne, and Shmng Xang. A general kernelzaon framework for learnng algorhms based on kernel PCA. Neurocompung, 73(4-6): ,