Dr. Junchao Xia Center of Biophysics and Computational Biology. Fall /15/2016 1/25

Transcription

1 BIO53 Biostatistics Lecture : Pricipal Copoet alysis Dr. Juchao ia Ceter of Biophysics ad Coputatioal Biology Fall 6 /5/6 /5

2 Outlie Liear lgebra ad Matrix Eigevalues ad Eigevectors of Matrix Pricipal Copoet alysis Data Clusterig alysis /5/6 /5

3 Matrix: Defiitio I atheatics, a atrix is a rectagualr array of ubers, sybols, or expressios arraged i rows ad colus. Row vector, Colu vector, Square atrix , a a a,,, a a a,,, a a a,,, ai, j /5/6 3/5

4 Matrix: Operatios I ddtio: + B i,j = i,j + B i,j, where i ad j Equatio: x =B kxl, oly if i,j =B i,j ad =k, =l. Scalar ultiplicatio: c i,j = c i,j, raspostio: i,j = j,i + B = B +, + B = + B, c = c, = Matrix ultiplicatio: x B xp = B xp ssociativity: BC = BC Distributivity: +BC = C+BC, C+B = C+CB, B B /5/6 4/5

5 Matrix: Operatios II Row ad colu operatios additio, that is addig a row or colu to aother. ultiplicatio, that is ultiplyig all etries of a row or colu by a ozero costat. 3 row switchig, that is iterchagig two rows or colus of a atrix. Subatrix subatrix of a atrix is obtaied by deletig ay collectio of rows ad/or colus /5/6 5/5

6 Square Matrix Diagoal, upper ad lower triagular atrix Idetity atrix I, I = 3 Syetric atrix: = Skew-syetric atrix, = - 4 Ivertible o-sigular atrix ad its iverse, B = B = I B= - 5 Defiite atrix: For all ozero vectors x R, the associated quadratic for of a syetric x-atrix is give by Qx = x x. If Qx takes oly positive values, is called positive-defiite atrix. If Qx takes oly o-egative values, the syetric atrix is called positive-seidefiite. 6 Orthogoal atrix, with real etries whose colus ad rows are orthogoal uit vectors. Equivaletly, = -, = = I /5/6 6/5

7 he deteriat of a atrix is deoted det, det, or. It ca be viewed as the scalig factor of the trasforatio described by the atrix. It ca be deoted directly i ters of the atrix etries by writig eclosig bars istead of brackets: Deteriat of x atrices Deteriat of Square Matrix Deteriat of 3x3 atrices 3Deteriat of 4x4 atrices Sarrus Rule /5/6 7/5

8 Properties of Deteriat. deti =, where I is the x idetity atrix.. det =det 3. det - =/det=det - 4. For square atrices ad B of equal size, detb=detdetb 5. Detc=c det for a x atrix. 6. If is a triagular atrix, i.e. a i,j =, wheever i>j, or alteratively, where i<j, the its deteriat equals the product of the diagoal etries: det=a, a, a, = a i,i 7. Iterchagig ay pair of colus or rows of a atrix ultiplies its deteriat by 8. ddig a scalar ultiple of oe colu to aother colu does ot chage the value of the deteriat. 9. Gaussia eliiatio for fidig the deteriat usig 8, 7, ad 6. /5/6 det = detb, detc = detb, detd = detc, det = detd = +8 8/5

9 Matrix Iversio is called ivertible also osigular or odegeerate if there exists a -by- square atrix B such that B = B = I, B= - is called the iverse of. he adjugate of is the traspose of the cofactor atrix C of, he cofactor atrix of is the atrix C whose i, j etry is the i, j cofactor of, he i,j ior of, deoted M ij, is the deteriat of the atrix that results fro deletig row i ad colu j of. /5/6 9/5

10 real -by- atrix x gives rise to a liear trasforatio R R appig each vector x i R to the atrix product x, which is a vector i R. Liear rasforatios by Matrix Represetatio /5/6 d b c a, d c, b a,, d b c a d b c a d b c a d b c a d b c a.5.5 with HorizotalShear axis vertical the Reflectiothrough / 3 3/ 3/ with r Squeeze appig cos3 si3 si3 cos3 Rotatio by 3 /5

11 Liear Equatios i atrix for, x +, x , x = b..., x +, x , x = b ca be writte as atrix equatio x = b where is a -by- atrix, x desigates a colu vector that is, -atrix of variables x, x,..., x, ad b is a -colu vector. Solvig liear equatios by atrix operatios Liear Equatios by Matrix Represetatio /5/6 b x b x I b x b x b x /5

12 Liear Regressio i Matrix For /5/ Matrix for poit For each observatio data is iid N, where i Scalar For, Model Siple Liear Regressio i i i i Matrix for Model Multiple Liear Regressio k k k k k k i ik k i i i /5

13 Least Squares Method /5/6 I d d d d i ik k i i i Errors of Squares of by iiizig the Su Method Squares Least, Matrix for Model Regressio Liear Multiple 3/5

14 Eigevalues ad Eigevectors of Matrix uber λ ad a o-zero vector v satisfyig v = λv are called a eigevalue ad a eigevector of atrix, respectively. he uber λ is a eigevalue of a -atrix if ad oly if λi is ot ivertible, which is equivalet to det λi=. Matrix decopositio LU/LDU decopositio: = LU=LDU factors as a product of lower L ad upper U triagle ad/or diagoal D atrices. Sigular value decopositio =UDV, U ad V are uitary atrices, ad D is a diagoal atrix. 3 Eigedecopositio or diagoalizatio, = VDV, D is diagoal atrix, with diagoal eleets are correspodig eigevalues dii= i; V is the square N N atrix whose ith colu is the eigevector vi of. /5/6 4/5

15 Eigevalues ad Eigevectors of Matrix: Exaple /5/6 Cosider a x atrix as below: Obtai the eigevalues by the deteriat: Eigevectors ca be obtaied by substitutig eigevalues to Mv=v ad requireet of oalizatio,, 3 VDV M V D old represetatio, 3 ew represetatio, 3 V DV V D V VDV M, - ew oe : the old basis set ito a trasfors V 5/5

16 Variace-Covariace Matrix he covariace betwee two rado variables ad Cov, E[ E E ] E[ E E E E x y ] If is a colu of rado variables,,. x y /5/6 6/5

17 Pricipal Copoet alysis Pricipal copoet aalysis PC is a statistical procedure that uses a orthogoal trasforatio to covert a set of observatios of possibly correlated variables ito a set of values of liearly ucorrelated variables called pricipal copoets. he uber of pricipal copoets is less tha or equal to the uber of origial variables. his trasforatio is defied i such a way that the first pricipal copoet has the largest possible variace that is accouts for as uch of variability i the data as possible. 3Succeedig copoet i tur has the highest variace possible uder the costrait that it is orthogoal to the precedig copoets. 4 he resultig vectors are a ucorrelated orthogoal basis set. /5/6 7/5

18 Geeral Procedure of Costructig PC PC ca be thought of as fittig a -diesioal ellipsoid to the data, where each axis of the ellipsoid represets a pricipal copoet. If soe axis of the ellipse is sall, the the variace alog that axis is also sall, ad by oittig that axis ad its correspodig pricipal copoet fro our represetatio of the dataset, we lose oly a coesurately sall aout of iforatio. Subtract the ea of each variable fro the dataset to ceter the data aroud the origi. Copute the covariace atrix of the data. 3 Calculate the eigevalues ad correspodig eigevectors of this covariace atrix. 4 Orthogoalize the set of eigevectors, ad oralize each to becoe uit vectors. Oce this is doe, each of the utually orthogoal, uit eigevectors ca be iterpreted as a axis of the ellipsoid fitted to the data. 5 he proportio of the variace that each eigevector represets ca be calculated by dividig the eigevalue correspodig to that eigevector by the su of all eigevalues. /5/6 8/5

19 Coputig PC usig the Covariace Method Prepare the data set: Suppose we have the data coprisig a set of observatios of p variables, ad we wat to reduce the data so that each observatio ca be described with oly L variables, L < p. Suppose further, that the data are arraged as a set of data vectors x x with each x i represetig a sigle grouped observatio of the p variables. rrage each x i as row vectors, we will have atrix xp Calculate the epirical ea: Fid the epirical ea alog each colu j =,..., p. Place the calculated ea values ito a epirical ea vector u of diesios p. u j i, j 3 Calculate the deviatios fro the ea : substract the epirical ea vector u fro each row of the data atrix, B=-hu, where h is x colu vector of all s. 4 Fid the covariace atrix: calculate the pxp epirical covariace atrix fro the B data atrix, C=B * B/-, B * is the cojugate traspose atrix. i /5/6 9/5

20 Coputig PC usig the Covariace Method 5 Fid the eigevectors ad eigevalues of the covariace atrix : V - CV=D, where D is the p x p diagoal atrix of eigevalues of C. D kl = k for k=l; D kl = for kl. Matrix V, also of diesio p p, cotais p colu vectors, each of legth p, which represet the p eigevectors of the covariace atrix C. 6 Rearrage the eigevectors ad eigevalues : Sort the colus of the eigevector atrix V ad eigevalue atrix D i order of decreasig eigevalue. 7 Copute the cuulative eergy cotet for each eigevector : g j j Dk, k for j,,p k 8 Select a subset of the eigevectors as basis vectors: Save the first L colus of V as the p L atrix W. Use the vector g as a guide i choosig a appropriate value for L such as g L /g p >.9. /5/6 /5

21 Coputig PC usig the Covariace Method 9 Covert the source data to z-scores optioal : Creat a p epirical stardard deviatio vector s fro the square root of each eleet alog the ai diagoal of the diagoalized covariace atrix C. Calculate the p z-score oralized atrix, Z=B/hS, divide eleet-by-eleet. Project the z-scores of the data oto the ew basis: he projected vectors are the colus of the atrix =ZW. he rows of atrix represet thekosabi-karhue-loeve trasfors KL of the data vectors i the rows of atrix. Rebuild Models usig the eigevectors for the reduced diesios. Variace trasforatio i PC /5/6 /5

22 Groupig a set of objects i such a way that objects i the sae group called a cluster are ore siilar i soe sese or aother to each other tha to those i other groups clusters. Reduce the uber of data poits to build coarse-graiig odels for aalyzig ad extractig critical iforatio. Use less coputig resources. Data Clusterig alysis /5/6 /5

23 K-Meas Clusterig s oe of cetroid odels, k-eas clusterig ais to partitio observatios x, x,, x, ito k < clusters, S = {S, S,, S k }, i which each observatio belogs to the cluster with the earest ea, servig as a prototype of the cluster, by iiizig the withi-cluster su of squares WCSS su of distace fuctios of each poit i the cluster to the K ceter. /5/6 3/5

24 s oe of coectivity odels, seeks to build a hierachy of clusters. Geerally there are two types of strategies for hierarchical clusterig: ggloerative"botto up" approach: each observatio starts i its ow cluster, ad pairs of clusters are erged as oe oves up the hierarchy. Divisive "top dow" approach: all observatios start i oe cluster, ad splits are perfored recursively as oe oves dow the hierarchy. Cluster dissiilarity: Hierachical Clusterig appropriate etric a easure of distace betwee pairs of observatios. likage criterio which specifies the dissiilarity of sets as a fuctio of the pair-wise distaces of observatios i the sets. /5/6 4/5

25 Suary I this lecture we talk about the followig topics related PC: Liear lgebra ad Matrix Eigevalues ad Eigevectors of Matrix Liear trasfor ad Multiple Regressio alysis i Matrix Pricipal Copoet alysis for Reducig Diesios Data Clusterig alysis /5/6 5/5

26 he Ed /5/6 6/5