Missing in Action? please check in. Course Groups and Presentations. Lecture #2 Principal Component Analysis !!!! FIRST!!!! Principal what?

Transcription

1 8 Mssng n Acton? please check n Course Bologcal Data analss an chemometrcs Aners C. Raffalt Jule K. Høgh Lasse R. Bech Mkkel. Pleman Chrstna H. Kærgaar na F. Rasmussen Ken Sejlng Aners S. Laer Rene J. Larsen Jacob B. Reves Lars Poulsen Kasper Jensen s8 s s8 s s s s s s8 s8 s8 s Cecle Krø Louse M. Jørgensen Fhaaz A. Saa Emmanoul Papaaks Aners S. R. Øum Mara M. Bermejo Carlos N. Bartolomé Asl Ozen Slva Bergamasch Mchal Strejcek Gr H. Svensen Casper. Holst s s8 s s s888 s8 s s8 s8 s s s Februar th, 8 Groups an Presentatons Group # (PCA): Lars Poulsen s8 Jacob B. Reves s8 Group # (MLR): Aners C. Raffalt s8 Casper. Holst s Group # (PCR): Mara M. Bermejo s8 Carlos N. Bartolomé s Group # (PLS R): Chrstna H. Kærgaar s na F. Rasmussen s Group # (Rge regress.): Louse M. Jørgensen s8 Mkkel. Pleman s Group # (Corresp. Analss): Cecle Krø s Gr H. Svensen s Group # (Cluster Analss): Rene J. Larsen Emmanoul Papaaks Group #8 (Classfcaton I): Aners S. R. Øum Asl Ozen Group # (Classfcaton II): Slva Bergamasch Mchal Strejcek Group # (PCO): Fhaaz A. Saa Kasper Jensen Aners S. Laer Ken Sejlng Jule K. Høgh Lasse R. Bech s8 s s888 s8 s8 s s s s s s s8 Lecture # Prncpal Component Analss Mchael Asetts Eberg Hansen Assstng Professor, CMB/DU meh@bo.tu.k Februar th, 8 Prncpal what? Prncpal Component Analss PCA.!!!! FIRS!!!! Lets recap from lnear algebra Invente b Karl Pearson n : Pearson, K. () On lnes an planes of closest ft to sstems of ponts n space. Phlosophcal l Magazne () :. A.k.a. or closel relate to: Sngular Value Decomposton (SVD) Karhunen Loéve Epanson Egenvector Analss Latent Vector Analss Characterstc Vector Analss Hotellng ransformaton

2 8 Scalars, Vectors an Matrces Scalar: varable escrbe b a sngle number (magntue) emperature C Denst g.cm Image ntenst (pel value) a. u. Vector: varable escrbe b magntue an recton Matr: rectangular arra of scalars Column vector Row vector Square ( ) Rectangular ( ) j : th row, j th column e n v v v A 8 C D b [ ] Vector Operatons ranspose operator column row row column b [ ] b [ ] Outer prouct matr column row row column A A [ ] Vector Operatons Inner prouct scalar [ ] Length of a vector [ ] n n n... M ( ) / ( ) / Inner prouct of a vector wth tself (vector length) ( ) Matr Operatons Aton (matr of same sze) Commutatve: ABBA Assocatve: (AB)CA(BC) B A Matr Operatons Multplcaton number of columns n frst matr number of rows n secon C A B (m p) (m n) (n p) c j nner prouct between th row n A an j th column n B Assocatve: (A B) C A (B C) Dstrbutve: A (BC) A B A C Not commutatve: AB BA!!! (A B) B A A B Some Defntons Ientt Matr Dagonal Matr I A A I A I D Smmetrc Matr B B b j b j D B

3 8 Matr Inverse Data as observatons A A AA I D D I Propertes A - onl ests f A s square (n n) If A - ests then A s non-sngular (nvertble) (A B) - B - A - ; B - A - A B B - B I (A ) - (A - ) ; (A - ) A (A A - ) I Samples as observatons n a Mult/hper mensonal Space: Objects are a collecton of features. IB # Features are mensons. Objects are ponts n a multmensonal space. IB # Intenst Mathematcal notaton N s the number of observatons M s the number of varables/features X Colon ameter K M K M M M M O M K N N N NM Data as observatons Data as observatons For each sample n (,..., P ) For each sample n (,..., P ) n n n Covarates M N M N M N L M L M L M L M O M L NM A pror (,..., P ) L P (,..., P ) L P L P L P M O M (,..., P ) N L NP n Covarates A pror n n X Y (,..., P ) (,..., P ) (,..., P ) n N N ( N,..., NP ) n N N ( N,..., NP ) For each sample n Data as observatons (,..., P ) (,..., P ) n Covarates A pror OBJECIVE: n X Y (,..., P ) Can we moel Y base on X? n (,..., P ) Prncpal Component Analss (PCA) Projecton metho Eplorator ata analss Etract nformaton an remove nose Reuce mensonalt / Compresson Classfcaton an clusterng X Moel Nose n N N ( N,..., NP ) Datatable (X) Instrument measurements Qualt parameters Process settngs Moel Structure varaton Informaton Nose Unstructure varaton Measurement error Contans no nformaton

4 8 PCA n a nutshell Projecton of ata Frst he llustratonal/ntutve approach: Lnear transformaton B proper rotaton Feature observatons n D space Feature observatons n reuce D space Projecton of ata Projecton of ata B proper lnear transformaton B proper lnear transformaton B proper rotaton B proper rotaton Feature observatons n D space Feature observatons n reuce D space Feature observatons n D space Feature observatons n reuce D space he PCA approach: Rotaton accorng to mamum varance n ata. he PCA approach: Rotaton accorng to mamum varance n ata. Fsher approach (later lecture): Rotaton accorng to mamum scrmnaton between groups. Prncpal Component Analss For a gven ataset: Prncpal Component Analss Calculate the centro ( mean n all rectons ):

5 8 Prncpal Component Analss Prncpal Component Analss Shft the gr to the centro: ake ths as our new coornate sstem: Prncpal Component Analss Calculate the recton n whch the varance s mamal: p Prncpal Component Analss An repeat ths for each net perpencular as (recton wth secon most varance): p p Prncpal Component Analss Leavng us wth a rotate gr: Prncpal Component Analss Whch we can rotate to a normal poston: p p p p

6 8 Prncpal Component Analss Prncpal Component Analss Showng us mamal varance: We can also use ths to reuce the complet of the ata set: p p p p Prncpal Component Analss B elmnatng a number of as b projecton of the ponts: Prncpal Component Analss In ths eample movng from two : p p p p Prncpal Component Analss In D to one mensonal ata ponts: Var(PC) Projectons st PC Var(PC) n PC

7 8 Prncpal Component Analss Prncpal Component Analss Assumumng varaton equals speces verst...: the frst PCA epcts ths nformaton : p p Scores an Loangs Prncpal Component Analss Scores Map of samples Dsplas strbuton of samples n the new space efne b the PC s Loangs Map of varables Shows how the orgnal varables are relate to the PC s We center the ata. But what about scalng? Especall when the covarates (columns) represent fferent scales (.e. comparng appels an bananas) ts mportant to ve b the stanar evaton! PCA n a nutshell Net lets escrbe the llustratons n mathematcal terms: Some (borng) efntons of PCA s mathematcall efne as an orthogonal lnear transformaton that transforms the ata to a new coornate sstem such that the greatest varance b an projecton of the ata comes to le on the frst coornate (calle the frst prncpal p component), the secon greatest varance on the secon coornate, an so on.

8 8 Some (borng) efntons of PCA n a nutshell PCA can be use for mensonalt reucton n a ata set b retanng those characterstcs of the ata set that contrbute most to ts varance, b keepng lower orer prncpal components an gnorng hgher orer ones. Such low orerorer components often contan the most mportant aspects of the ata. I.e. assumng that nformaton s equal varaton! However, epenng on the ata ths ma (obvousl) not alwas be the case. Fnall... Lets wrap t up n math: Prncpal Component Analss Prncpal Component Analss he th prncpal component of X s the projecton Y p X he vector Y Y Y M P X Y k s calle the vector of prncpal components. μ (.,.).8 Σ... PCA ( λ, λ ) (.,.) P ( p p ), μ (.,.). Σ... cov ( Y) cov( P X) λ L P ΣP M O L M λ k Prncpal Component Analss Prncpal Component Analss p p μ (.,.).8 Σ... PCA ( λ, λ ) (.,.) P ( p p ), μ (.,.). Σ... μ (.,.).8 Σ... PCA ( λ, λ ) (.,.) P ( p p ), μ (.,.). Σ... 8

9 8 Prncpal Component Analss Output From the PCA we ma etract the set of lnear combnatons that eplans the most varaton λ L λm q λ L λ L λ m An hereb conense an reuce the mensonalt of the featurespace. From the eample before we see, that the lnear combnatons eplan of the total varance. ( p, p ) ( 8.%,.% ) P k Loangs he weghts Scores Plots Loangs plot Scores plot Bplot Pros an Cons Postves Can eal wth large ata sets. here weren t one an assumptons on the ata. hs metho s general an ma be apple to an ata set. Negatves Nonlnear structure s nvsble to PCA he meanng of features s lost when lnear combnatons are forme Stll!!! Nonlnear PCA s est (so calle kernel methos) Sparseness or supervse projectons can be ntrouce to emphasze mportant features Eercse Eercse SCORE plot LOADING plot INFLUENCE plot EXPL. VARIANCE plot

10 8 Eercse Eercse