Multiscale principal component analysis

Transcription

1 Multscale prcpal compoet aalyss A A Aduo ad A N Gorba Mathematcs Departmet Uversty of ecester ecestershre E 7RH UK E-mal: aaa78@leacu ad ag53@leacu Abstract Prcpal compoet aalyss PCA s a mportat tool eplorg data he covetoal approach to PCA leads to a soluto hch favours the structures th large varaces hs s sestve to outlers ad could obfuscate terestg uderlyg structures Oe of the equvalet deftos of PCA s that t sees the subspaces that mamze the sum of squared parse dstaces betee data proectos hs defto opes up more fleblty the aalyss of prcpal compoets hch s useful ehacg PCA I ths paper e troduce scales to PCA by mamzg oly the sum of parse dstaces betee proectos for pars of datapots th dstaces th a chose terval of values [lu] he resultg prcpal compoet decompostos Multscale PCA deped o pot lu o the plae ad for each pot e defe proectors oto prcpal compoets Cluster aalyss of these proectors reveals the structures the data at varous scales Each structure s descrbed by the egevectors at the medod pot of the cluster hch represet the structure We also use the dstorto of proectos as a crtero for choosg a approprate scale especally for data th outlers hs method as tested o both artfcal dstrbuto of data ad real data For data th multscale structures the method as able to reveal the dfferet structures of the data ad also to reduce the effect of outlers the prcpal compoet aalyss Itroducto It s ofte dffcult to etract meag from multvarate data of hgh dmeso ad hece there s a eed for feature etracto to mae aalyss easer ad to spot treds patters outlers ad other terestg relatoshp ad structures our data I 90 Pearso proposed appromatg hgh dmesoal data th les ad plaes ad hece veted the Prcpal Compoet Aalyss PCA PCA s a lear techque hch trasforms data to a e coordate system usg lear orthogoal trasformato such that the e coordates are ordered by varace he coordate th hghest varace s the frst prcpal compoet; the secod prcpal compoet s the coordate th the secod hghest varace ad so o a eample s gve fgure PCA s a poerful aalyss tool ad t s udged to be oe of the most mportat results of appled lear algebra [6] th may terestg applcatos hch clude: dmeso reducto bld source separato data vsualzato mage compresso ad th relevace may appled dscples such as quattatve face bology pharmaceutcs taoomy healthcare ad may more he prcpal compoets from PCA are lear combato of the orgal compoets ad eve though PCA s lmted tha o-lear dmeso reducto techques t s guarateed to sho geue propertes of the orgal data ad the lo dmeso are meagful [7]

2 Fgure Scatter plot of data he sold red arro ad the dashed blac arro dcate the drecto of the frst ad secod prcpal compoets respectvely color ole Hoever despte the may applcatos of PCA t s ot thout ts drabacs A eample of such drabacs s that PCA s based o the covarace matr hch s sestve to outlers I ths paper outlers are defed as data elemets th large dstace from the other data elemets a data sample Eve though outlers ca be fltered before performg PCA o the dataset hoever some cotets detfyg outlers could be cumbersome I addto to the above datasets are usually osy here e defe oses as data elemets th rather small varace ad the presece of ose data aalyss ca further obfuscate the uderlyg structures of the data beg vestgated [6] Oe of the deftos of PCA s that PCA fds subspaces les plaes or hgher dmesoal subspaces that mamze the sum of pot-to-pot squared dstaces betee the orthogoal proectos of data pots to them et the dstace fucto dst y be defed by a postve defte quadratc form Eucldea dstace for pars of obects y I clear terms PCA sees the dmesoal orthogoal proecto that mamzes dst P P here P s the proecto of vector to plae We ca observe that the mamzato problem gve above favors large parse dstaces Hece other terestg structures hch ca be revealed by smaller parse dstaces may be completely obfuscated Oe eample of such problem arses he usg PCA o data th outlers as the outlers may obfuscate the structures of the data et us cosder the data sho fgure the data s dstrbuted alog a le but th outlers sho crcles Fgure 3 shos the data proecto to the prcpal compoets th the -as as the frst prcpal compoet Hoever f the outlers ere removed the frst prcpal compoet should be close to the le o hch the data s dstrbuted as sho by the arro fgure 3 Fgures 4 ad 5 are the bplots [] of the eample gve above A bplot s useful for vsualzg the magtude ths s represeted th the les ad sg of each varable's cotrbuto to the frst to or three prcpal compoets ad ho each observato represeted as pots o the graph s represeted terms of those compoets he aes represet the prcpal compoets From the bplot belo e ca observe a sgfcat chage the cotrbuto of the varables the PCA due to the presece of the outlers here are several equvalet deftos of prcpal compoets he defto preseted above through mamzato of the sum of pot-to-pot squared dstaces betee the orthogoal proectos of data pots gves more fleblty for geeralzato ad cotrol [3] hch ca be mapulated to reveal some terestg uderlyg structures our data I addto to ths the defto above opes up the relatoshp betee PCA ad multdmesoal scalg [3]

3 Fgure Scatter plot of data dstrbuted alog a le th outlers he outlers are sho crcle Fgure 3 Data proecto to the frst prcpal compoets gve by the aes he arro dcates the drecto of the prcpal compoet f the outlers are removed Fgure 4 he bplot of the data thout outlers Fgure 5 he bplot of the data th outlers I ths paper scale as troduced to ehace the performace of PCA o datasets hat s e ll use the defto of multscale PCA mamzato of the sum of pot-to-pot squared dstaces betee the orthogoal proectos of data pots for the pars of pots th dstaces some tervals he result of ths s PCA decomposto of the data hch deped o the scale chose A further study of these PCA decompostos reveals some uderlyg structures hch could have bee obfuscated by other structures such has the presece of outlers or repeated patters as sho later We also proposed a crtero for determg the approprate scale for computg the prcpal compoets for data th outlers

4 Deftos ad Mathematcal Bacgroud I ths secto e cosder four classcal approaches to PCA hch are equvalet as gve by [] ad e also gve the ecessary mathematcal bacgroud that ll be eeded for ths paper Deftos of PCA et be a lear mafold of dmeso gve the parametrc form as v 0 av av a v here R m m a v R v s a set of orthoormal vectors R m Also let be data elemets here R ad let the data elemets be arraged as the ros of a m matr X such that the m coordates s gve by the colum of X For ths paper the coordates ll be represeted by Gree dces hle the observatos ll be represeted by at dces e s the thcoordate of the th observato For all computatos e assume that the data s cetered hs could be acheved by smple traslato of the data 0 ad v v We shall deote the proecto of data to the plae by P Defto Data appromato by les ad plaes PCA computes the sequeces m such that the sum of squared dstaces from data pots to ther orthogoal proectos o s mmal over all lear mafolds of dmeso m embedded R : MSD X m m he mea squared dstace betee a dataset X ad set of vectors y deoted by MSD X y s N defed as MSD X dst P y y Remar: Dmesoless varables ad ormalzato hs s eactly the defto gve by Pearso 90 [5] Eve though Pearso hs paper o prcpal compoet aalyss dd ot use hs defto of PCA ormalzato to ut varace t s ecessary to use the same dmeso o all aes For eample e caot summarse meters th lograms herefore ormalzato becomes mportat he the data s from dfferet dmesos; hoever the choce of ormalzato should deped o the type of data ad the problem beg solved Defto Varace mamzato For a dataset X ad for a gve v let us costruct a oe-dmesoal dstrbuto B { : v X} here deotes scalar vector product he let us defe emprcal varace of X alog v as Var B here Var s the stadard emprcal varace PCA sees to fd such that the sum of emprcal varaces of X alog v v v ould be mamal over all lear mafolds of dmeso embedded R m : Var B ma Defto 3 mea pot-to-pot squared dstace mamsato

5 PCA problem cossts fdg such sequece betee the orthogoal proectos of data pots o m dmeso embedded R : dst P P ma N that the mea pot-to-pot squared dstace s mamal over all lear mafolds of We o that all orthogoal proectos oto loer-dmesoal space lead to cotracto of all potto-pot dstaces ecept for some that do ot chage ths s equvalet to mmzato of mea squared dstace dstorto: N [ dst dst P P ] ma Defto 4 correlato cacellato: PCA sees such a orthoormal bass v v v hch the covarace matr for X s dagoal Evdetly ths bass the dstrbutos v ad v for have zero correlato Mathematcs Bacgroud As earler stated PCA sees the -dmesoal proecto that mamzes dst P P Usg the Eucldea dstace ths problem ca be stated as DX P ma here a v a R ad m Also m v R ad v v s roecer delta he proecto of a vector to a plae hch s deoted by P v v herefore the problem 3 reduces to mamzg D X v 4 hs s the same as mamzg the equato 5 belo D X v 5 he epresso the bracet gve as v v v v S ~ v 3 6 here ~ S

6 ad each elemet of S ~ s gve as ~ S Here S ~ s symmetrc postve sem-defte because for every y herefore the problem gve by 3 s reduced to ma v v ~ v Sv Subect to v v y y s postve sem-defte et m be the sorted egevalues of the matr S ~ ad e em be the correspodg egevectors a mamzer of the costraed mamzato problem 8 s e em See theorem Hece the orthogoal vectors that mamze 8 are the egevectors correspodg to the hghest egevalues If there are q dstct egevalues q of the matr S ~ such that s of multplcty ad q m e have a case called egevalue degeeracy For each th multplcty the egevectors le a dmesoal subspace orthogoal to the subspace spaed by the odegeerate egevalues For symmetrc matr these egevectors ll be learly depedet ad usg Gram-Schmdt procedure e ca fd orthogoal vectors that spa ths subspace No t s left to sho that the soluto to the problem 8 s actually the prcpal compoets et us eame the matr ~ S 9 Where μ μ μ μ ad S ~ μ 0 he remag terms are zero because the data has bee cetered ~ S cov X We o that gve dataset X th emprcal covarace matr S ad let egevalues of the matr S the correspodg egevector ~ 7 8 be the sorted e e s the prcpal compoets of the data From equato above S S therefore the egevectors of S s also the egevectors of S ~ ad ths s the prcpal compoet of X sce the multplcato of a matr by a postve

7 costat does ot chage the egevectors or ther order Hece e have sho that the soluto to mamzato problem 8 s the prcpal compoet of X No e cosder the elemets the matr S ~ From equato 7 e have S ~ 3 I matr otato the quadratc form 3 ca be rtte as ~ X X S ~ X X S 4 Where ] [ ad s the roecer delta s a symmetrc postve-sem defte matr th zero colum ad ro sum ad ths s useful for descrbg the parse relatoshp betee data elemets as sho emma hch s also avalable [7] heorem : et A be a symmetrc matr ad let the sorted egevalues be gve by ad let e e be the correspodg egevectors he e e s a mamzer of the costraed mamzato problem A ma u u u u Subect to: u u A detaled proof of ths theorem s avalable [7] emma : et be as defed above ad let R the Ad for coordate vectors e have:

8 Hece e see that the matr gve by s useful because the quadratc form assocated th t s the eghted sum of all parse squared dstaces [7] 3 Weghted PCA Defto 3 allos for some fleblty the aalyss of prcpal compoets because e have cotrol over the parse dstaces of proected data By assgg eghts to these parse dstaces e ca mapulate the resultg PCA decomposto of the data We o cosder the problem of fdg the prcpal compoet usg eghted parse dstaces of proected data hs problem s stated belo [ dst P P ] Where D X P Subect to: v v ma = s the o-egatve eght assged to the dstace betee elemet ad ad 0 for he equato 5 reduces to mamzg the equato 6 belo D X v 6 hs s the same as D X v he epresso the bracet gve as here et R ad let C v v v ~ v 5 v M 7 ~ M Equato 8 ca be rtte as ~ M 9 8

9 ] [ C R 0 ] [ C R Each elemet s gve as C R M R M 3 because C R ad herefore e ca rte equato 3 as R M M 4 et ] [ hs ca be rtte the form belo Where 0 for I matr otato the quadratc form 4 ca be rtte as ~ X X M 5 ad ~ X X M herefore the problem gve by 5 s reduced to v v M ~ ma v v Subect to v v 6 here M ~ s a symmetrc postve sem-defte matr ad from theorem the egevectors correspodg to the sorted egevalues of the matr M ~ s a mamzer of the costraed mamzato problem 6 I the case of degeerated egevalues the set m e e s ot uquely defed

10 3 Multscale PCA MPCA I ths secto e troduce the Multscale PCA MPCA algorthm to ehace the robustess of the PCA especally revealg hdde structures that may be preset dataset but hch the covetoal approach mght ot reveal MPCA compute prcpal compoets by mamzg the sum of parse dstaces betee data proecto for oly pars of datapots for hch the dstace s th the chose scale hs s acheved by assgg a eght of to the parse dstace of proectos of ay par of data pots th dstace th the chose scale ad a eght of 0 otherse l u 0 otherse 7 m I the scale terval lu l s the loer lmt of the scale ad u s the upper lmt et d be the ma mmum parse dstace greater tha zero ad d be the mamum parse dstace the data m We select the pars lu from a tragle Δ={lu: d l<u d ma } Wth ths cotrol over the parse dstaces e are able compute PCA at varous scales ad the outcome of ths s scale depedet PCA hch ca reveal terestg uderlyg structures that may be preset data For eample reducg the upper lmt of the scale hle eepg the loer lmt at 0 traslate to computg PCA by cosderg smaller dstaces ad ecludg very large dstaces hs has the effect of mmzg thout eplct ecluso the cotrbuto of certa fluetal data elemets the aalyss of the prcpal compoets 3 he Multscale PCA Algorthm Here e dscuss the Multscale PCA Algorthm Gve the data sample Cetralze the data by subtractg the mea of the varables from each observato 3 Fd the dssmlarty matr by computg the Eucldea dstace 4 Choose a approprate scale betee 0 ad the mamum dstace For easy aalyss a scale betee 0 ad could be chose ad the multpled by the mamum dstace For ths paper he usg scale betee 0 ad e call t stadard scale 5 Calculated the bary eght as gve equato 7 6 Calculate the matr as gve belo 7 Calculate the matr A Y Y here Y s the cetralzed data 8 Fd the sorted egevalues of the matr A descedg order of magtude ad proect the data oto ther correspodg egevectors hs ll be the prcpal compoets at the selected scale

11 o llustrate the result of MPCA o data e cosder some eamples 3 Multscale PCA o Data th repeated patters Eample Here e cosder a eample of a data sample th repeated uderlyg structure See fgure 6-9 Fgure 6 Scatter plot of data th repeated patter Fgure 7 he sold arro ad the dashed arros sho the drecto of the frst ad secod prcpal compoets respectvely usg MPCA at a scale of [0-08] equvalet to stadard scale [0-] hs s the same as usg PCA Fgure 8 he sold arro ad the dashed arro sho the drecto of the frst ad secod prcpal compoets respectvely usg MPCA at a scale of [0-00] equvalet to stadard scale [0-08]

12 Fgure 9 he sold arro ad the dashed arro sho the drecto of the frst ad secod prcpal compoets respectvely usg MPCA at a scale of [0-] equvalet to stadard scale [0-00] From fgure 9 e observe that the PCA reveals the er structure of the data A better ve of ths er structure ad the PCA s gve fgure 0 Fgure 0 Magfed ve of oe cluster the dataset th the sold red arro ad dashed blac arro represetg the drecto of the frst ad secod prcpal compoets respectvely Color ole As llustrated the eample above the prcpal compoets chaged as the scale chaged ad ths as able to reveal some uderlyg structures of the data Fgure oe captures the chages the frst prcpal compoet at varous scales Fgure he dagram llustrate the chage the agle of the frst prcpal compoet as the scale chaged he agle recorded here s the agle gradet betee the frst prcpal compoet usg PCA ad the frst prcpal compoet usg MPCA at a gve scale

13 4 Clusterg Aalyss o the Iterval of Scales o further study these structures e cosder clusterg aalyss o the terval of scales ad e troduce the Rato of Dstorto ths secto 4 Represetg PCA Structures et us cosder the terval of values l u here l loer lmt u upper lmt ad l u he scale l u ca be represeted as pot the plae R as sho fgure he resultg prcpal compoet decompostos MPCA deped o the pots l u o the plae Fgure hs dagram shos the stadard scale represeted as pots o the plae R We ll le to study the PCA structure at dfferet scales; therefore e eed a represetato of the PCA structure for each pot l u o the plae We ca represet the PCA structure at a pot l u by the correspodg orthoormal vectors of prcpal compoets from MPCA at that pot; hoever ths represetato s ot coveet for statstcal aalyss of prcpal compoets If for eample e cosder the case of equdstrbuto of a ormalzed vector v o m sphere the epectato E [ v] 0 hs s because of sphercal symmetry ad the Epectato s the vector the sphere hch s rotato varat ad that s 0 ad ths could be couter tutve he space of prcpal compoet bases s a space of orthoormal bases R m hs s ot a lear space but a rather complcated symmetrc mafold th group O m acto o t We propose to embed ths symmetrc space to a Eucldea space usg the PCA proector represetato ad after that apply stadard statstcal ad data mg procedures et us recall that the prcpal compoet gve by e s the same as e therefore e eed a represetato such that ths codto s satsfed he prcpal compoets are orthogoal aal frame [7] ad oe ay to represet such data s usg the tesor product P e e hch s the proector of our data oto the prcpal compoet e Sce ths product s b-lear e o that e e e e hece e have the same represetato for both a vector ad ts egatve as requred P X e e X e e X s the proecto of data X oto vectors e ad P X e e X s the data X proected oto the frst - prcpal compoet

14 m For ay m orthoormal vectors e em e e If e s oe of e th probablty the m Ee e he rotato varace gves the same result f e s equdstrbuted o ut m m sphere Fgure 3 hs dagram llustrates ho the proecto of vector y chages gve by the dashed red sphere as e moves alog the blue sold -sphere Color ole It follos that the average proecto of E[ P X ] X m herefore e represet the PCA structure of the data at ay pot l uby the sum of the proectors correspodg to MPCA at that pot hs ll be deoted by e e m he full descrpto of the prcpal compoets decompostos of data X s gve by a ordered set cortege of matrces m m If e arrage the e as colums of matr E the X EE X ad EE For m EE Ι MPCA lead to scale depedet PCA structures ad th these PCA structures represeted as defed above e ca study the structures our data further by aalyzg these proectors he PCA structures assocated th to dfferet pots o the plae s sad to be smlar f ther correspodg proectors are smlar 4 Clusterg of Scales We guess that some cases there are clear teral structures the data hch deped o scales Performg MPCA o the data leads to a cotuum of PCA structures depedg o scales used ad to reveal the structures the data e o scales th smlar PCA structures ad separate scales th dssmlar PCA structures hs leads to the dea of clusterg of scales We represet the dstace betee to pots o the scale by the dstace betee ther correspodg PCA structures Clusterg aalyss of the scales group smlar PCA structures together ad ths reveals some structures the data We descrbe each cluster by the proector correspodg to the medod pot of the cluster I a later secto e ll troduce Rato of Dstorto hch s aother crtero that ca be used to select the proectors that descrbe the clusters

15 For eample clusterg aalyses of scales for correspods to cluster aalyss of the MPCA structures he data s proected oto the frst prcpal compoets at varous scales No let each pot l u the plae be represeted by p here p l u l uu such that l u We deote the proector at a pot p by p For ay par of pots p q the space of scales e ca compute the dstace betee the assocated proectors for a gve usg varat orm We recall that the Frobeus orm of a real matr B deoted by F tracebb P P trace P P P P B therefore dstace betee proectors of ay par of pots the space of scale dst p q p q p q Ay stadard clusterg algorthm ca be used to cluster the scale order to reveal hdde structures the data but ths paper agglomeratve herarchcal clusterg as used because e ca measure dstace easly Decdg o the umber of true clusters clusterg aalyss s a classcal p q problem ad oe may at to compare varous dces A typcal eample of such s the statstc SSEt SSEa SSEb a b pseudo t SSE SSE Where SSE s the sum of square of cluster a a a b b pseudo SSE s the sum of square of cluster b SSE s the sum of square of cluster formed by og clusters a ad b a ad b are the umber of elemets clusters a ad b respectvely If a small value of the pseudo t statstc at a step of the herarchcal clusterg s folloed by a dstct large value at the step the cluster form at the step s chose as the optmal cluster It s assume that the mea vector of the to clusters beg merged at the step ca be regarded as dfferet ad should probably ot be merged et us cosder the result of the cluster aalyss of the data eample fgure 6 for e proecto oto frst prcpal compoet For llustrato purpose pots from the subset of ad U have bee selected l ad u u he result s preseted fgure 4 t t Fgure 4 hs dagram shos cluster of scales o the plae Scales belogg to the same cluster are represeted by the same symbol ad color Color ole

16 he pseudo t statstc dcates three meagful clusters hs reaffrms the result dsplayed fgure We represet each of these structures by the egevector of the medod pot of the cluster represetg t he result s gve the table able hs table shos the descrpto of each cluster Each cluster has bee descrbed by the egevector of the proector correspodg to the medod pot of the cluster Cluster Iterval correspodg to Medod pot Proector 009 Egevector e [ ] e [ 0707 e [ ] 00000] Eample We cosder MPCA of the Eergy Effcecy Dataset avalable ole at the UCI mache earg Repostory hs dataset cotas 768 samples ad 8 compoets ad used to predct dfferet outputs We perform MPCA o the data the output varables are ot cluded sce all the data are postve e ormalzed by dvdg by the mea he data proectos to the frst to prcpal compoet ad frst three prcpal compoets respectvely are sho the fgures 5 ad 6 MPCA at stadard scale of [0-0] reveals the structure gve fgures 7 ad 8 he result of the clusterg aalyss of the scale s preseted fgure 9 he dcates four meagful clusters pseudo t statstc Fgure 5 Data proecto to the frst prcpal compoets for PCA Fgure 6 Data proecto to the frst 3 prcpal compoets for PCA

17 Fgure 7 Data proecto to the frst prcpal compoets usg MPCA at stadard scale 0 0 Fgure 8 Data proecto to the frst 3 prcpal compoets usg MPCA at stadard scale 0 0 Fgure 9 hs dagram shos cluster of scales o the plae Scales belogg to the same cluster are represeted by the same symbol ad color Color ole We represet each of these structures by the egevector of the medod pot of the cluster represetg t he result s gve the table able hs table shos the descrpto of each cluster Each cluster has bee descrbed by the egevectors of the proector correspodg to the medod pot of the cluster Cluster Iterval correspodg Egevector to Medod pot 009 e [ ] e [ ] e [ e [ ] e [ ] 0790] e [ ] e [ ] e [ ]

18 43 Multscale PCA o Data th Outlers he presece of outlers our data serves to obfuscate the uderlyg structure of the data PCA MPCA s hoever effectve revealg the uderlyg structure of data th outlers By reducg the upper lmt of the scale e ca effectvely mtgate the effect of outlers the aalyss of the prcpal compoets thout eplct ecluso of these outlers Eample 3 o test the performace of scaled PCA o data th outlers data ere smulated alog o plae ad some outlers ere added to ths data hs data as embedded to a hgher dmesoal space ad e see to recover the orgal plae from the data by usg PCA ad MPCA at varous scales he agle betee the orgal drectoal vector ad the frst prcpal compoet of MPCA at varous scales s gve the apped see table A We cosder a 3-dmesoal data sample hch the elemets are dstrbuted uformly o a plae -d th the drectoal vectors gve as u [ ]; v [ ]; Wth vector u beg the frst prcpal compoet ad fe outlers ere added as ca be see fgure 9 he proecto of the data to the frst prcpal compoets s sho fgure 0; ths has bee flueced by the outlers the data MPCA at stadard scale of 0-08 hoever gves aother structure hch s foud to have captured the data qute ell as sho fgure he result of the clusterg aalyss of the scales s preseted fgure Fgure 9 Scatter plot of data 3-dmeso th a fe outlyg pots Fgure 0 Data proecto to the frst prcpal compoets usg PCA It ca be observed that the outlers have flueced the result of the PCA

19 Fgure Data proecto to the frst prcpal compoets usg MPCA at stadard scale of 0 08 he effects of the outlers have bee mtgated Fgure hs dagram shos cluster of scales o the plae Scales belogg to the same cluster are represeted by the same symbol ad color Color ole We represet each of these structures by the egevector of the medod pot of the cluster represetg t See table 3 able 3 hs table shos the descrpto of each cluster descrbed by the egevectors of the proector correspodg to the medod pot of the cluster Cluster Iterval correspodg to Egevector Medod pot Proector 04 e [ ] e [ ] 00 e [ ] e [ ] 44 Crtero for choosg scale for data th outlers I ths secto e propose a crtero for decdg a approprate scale for MPCA feature etracto especally for data th outlers As metoed earler fdg the prcpal compoets usg defto 3 s equvalet to mmzg the mea squared dstace dstorto N [ dst dst P P ] m Where the dmeso of s strctly less tha the dmeso of the data Hece e propose that a approprate scale for a gve dmeso could be determed by fdg the rato of dstorto

20 For all N N P P such that l u ma l Is the loer lmt of the scale ad u s the upper lmt he rato of dstorto troduced here ca also be used the clusterg aalyss of scales as a crtero to determe the PCA structure that descrbes the cluster 5 Dscusso ad Cocluso 5 Dscusso For eample 3 MPCA at scales l u 0 l u 0 9 l u reveals aother structure the data that has bee obfuscated by the outlers able A ad A3 the apped sho the results of the rato of dstorto for dfferet dmesos hs s also cosstet th the dfferece agle betee the orgal plae ad the prcpal compoet computed usg MPCA at these scales see table A the apped Reducg the upper lmt to a very small umber may cause MPCA to ft ose hle creasg the loer lmt oly may cause MPCA to ft outlers f such s preset the data It s mportat to ote that by usg MPCA some parse dstaces are eempted the aalyss of prcpal compoet ad the percetage of such eempted parse dstace should be ept to a reasoable umber As t ca be observed the table A ad A3 the apped as the loer lmt creased the rato of dstorto appear to mprove eve though the dfferece agle s qute large for some scales but oly because MPCA s fttg outlers herefore addto to the result of the rato of dstorto the percetage of total parse dstaces eempted the computato of the MPCA at dfferet scales especally hel 0 should be cosdered choosg a approprate scale A good scale for MPCA should be oe th mamum rato of dstorto ad least umber of eempted parse dstaces able A4 the apped shos the percetage of parse dstaces of data pots eempted the computato of MPCA at varous scales It ca be cocluded that hle reducg the upper lmt s good for ths data creasg the loer lmt maes MPCA to ft outlers 5 Cocluso Prcpal compoet aalyss of hgh dmeso data favour compoets th hgh varace hs may obfuscate hdde geometrc structures that may be preset the data he defto of PCA as the mamzato of the sum of pot-to-pot squared dstaces betee the orthogoal proectos of data pots s a very coveet defto ad allos for geeralzato I ths paper e troduced multscale PCA as mamzato of the sum of pot-to-pot squared dstaces betee the orthogoal proectos of data pots for the pars of pots th dstaces some tervals scales MPCA s developed to solve the problem of revealg hdde geometrc structures data he result of MPCA o data leads to a cotuum of PCA structures of the data hch s depedet o the tervals chose Aalysg the MPCA structures of data reveals some teral structures of the data especally for data th multscale structures o study the MPCA structures of data e represet the MPCA structure at a gve terval by the cortege of proectors correspodg to MPCA at that terval; ths represetato has good ad meagful statstcal propertes hch are dscussed o reveal uderlyg geometrc structures that may be preset the data clusterg aalyss of the PCA structures at varous scales groups together scales th smlar PCA structures ad separate scales th dssmlar PCA structures

21 For data th clear multscale structures the cluster aalyss reveals some uderlyg structures the data hch covetoal PCA caot reveal due to the fact that such structures are obfuscated by other structures of hgher varace Each meagful cluster correspods to a structure the data ad e represet each cluster by the medod pot of the cluster ad ths represetatve s used to descrbe the structure of the data for cluster We propose the Rato of Dstorto as a crtero for choosg a approprate scale for MPCA for feature etracto especally for data th outlers ad also as a crtero for choosg the PCA structure to descrbe each cluster the clusterg aalyss of scales Applcato of MPCA o artfcal ad real lfe eamples shos that ths ca be useful For data th mult-scale structures the method as able to reveal some uderlyg structure data he method as partcularly useful mtgatg the fluece of outlers o the aalyss of prcpal compoet thout havg to eclude such outlers eplctly

22 oer mt oer mt Apped able A he agle betee the orgal compoet ad the result of the frst prcpal compoet usg MPCA at varous scales for the data eample 3 Upper mt SCAE Note: he MPCA at scale 0- s the same as PCA he cell for PCA as beg mared th a grey-scale bacgroud able A he rato of dstorto of MPCA at varous scales for = hs result s for the data gve eample 3 Upper mt SCAE NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Note: he MPCA at scale 0- s the same as PCA he cell for PCA as beg mared th a grey-scale bacgroud

23 oer mt oer mt able A3 he rato of dstorto of MPCA at varous scales for = hs result s for the data gve eample 3 Upper mt SCAE NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN Note: he MPCA at scale 0- s the same as PCA he cell for PCA as beg mared th a grey-scale bacgroud able A4 he percetage of parse dstaces ecluded at varous scales of MPCA hs result s for the data gve eample 3 Upper mt SCAE % 0% 0% 0% 0% 0% 0% 585% 347% 7353% 0 647% 3768% 3768% 3768% 3768% 3768% 3768% 43% 68% 000% 0 659% 7650% 7650% 7650% 7650% 7650% 7650% 85% 000% 000% % 9535% 9535% 9535% 9535% 9535% 9535% 000% 000% 000% % 0000% 0000% 0000% 0000% 0000% 000% 000% 000% 000% % 0000% 0000% 0000% 0000% 000% 000% 000% 000% 000% % 0000% 0000% 0000% 000% 000% 000% 000% 000% 000% % 0000% 0000% 000% 000% 000% 000% 000% 000% 000% % 0000% 000% 000% 000% 000% 000% 000% 000% 000% % 000% 000% 000% 000% 000% 000% 000% 000% 000%

24 Refereces [] Jollffe I 00 Prcpal Compoet Aalyss Secod Edto Ne Yor: Sprger [] Gorba A N ad Zovyev A Y 009 Prcpal Graphs ad Mafolds Hadboo of Research o Mache earg Applcatos ad reds: Algorthms Methods ad echques Iformato Scece ReferecePreprt arxv: v [3] Zovyev A 000 Vsualzato of Multdmesoal Data I Russa Krasoyars echcal State Uversty Press [4] Burges C J C 00 Geometrc Methods for Feature Etracto ad Dmesoal Reducto - A Guded our Data Mg ad Koledge Dscovery Hadboo Ne Yor: Sprger ed O Mamo ad Roach d Edto ISBN pp 53-8 [5] Pearso K 90 O les ad plaes of closest ft to systems of pots space Phlosophcal Magaze pp [6] Shles J 005 A tutoral o prcpal compoet aalyss Accessed May 0 03 [7] Arold R ad Jupp P E 03 Statstcs of orthogoal aal frames Bometra pp -6 do: 0093/bomet/ast07 [8] Kore Y ad Carmel 004 Robust lear dmesoalty reducto Vsualzato ad Computer Graphcs IEEE rasactos vol0 o4 pp459-70