GEOG 4110/5100 Advanced Remote Sensing Lecture 15 Principal Component Analysis Relevant reading: Richards. Chapters 6.3* http://www.ce.yildiz.edu.tr/personal/songul/file/1097/principal_components.pdf *For more information on working with matrices, refer to Richards, Appendix A) GEOG 4110/5100 1
Multispectral Transformations of Image Data It is possible to transform brightness data through linear operations on the set of spectral bands Can make image features visible that are not discernable in the original data Can preserve image quality a reduced number of transformed dimensions E.g. for display on a color monitor Principal Component transformation Seeks to minimize correlation in order to minimize redundancy of spectral bands GEOG 4110/5100 2
Principal Components Analysis (PCA) PCA is a technique that transforms the original vector image data into smaller set of uncorrelated variables. The variables represent most of the image information and are easier to interpret. Principal components are derived such that the first PC accounts for much of the variation of the original data. The second (vertical) accounts for most of the remaining variation. PCA is useful in reducing the dimensionality (number of bands) that are used for analysis. Minimum noise fraction (MNF) method can be used with hyperspectral data for noise reduction. for a fairly simple explanation: http://www.ce.yildiz.edu.tr/personal/songul/file/1097/principal_components.pdf GEOG 4110/5100 3
Principal Components Seek new coordinate system in vector space in which data can be represented without correlation Covariance matrix is diagonal y = Gx = D t x GEOG 4110/5100 4
Mean Vector and Covariance The mean vector (m) is the vector average of the individual components of a vector The covariance between two real-valued random describes how one variable varies in relation to another. Cov(X,Y) = 1 n 1 n i=1 (X i x )(Y i y ) C = 1 x n 1 n i=1 (X i x)(x i x) T GEOG 4110/5100 5
Mean Vector and Covariance The covariance matrix (S x ) is a matrix of covariance values that describes the scatter or spread between variables. Computation of Covariance Matrix (Table 8.1 from Richards and Jia, 2006) x = 1 n n 1 i= 1 ( x i m)( x i m) t m = 3.00 2.33 No Correlation between x 1 and x 2 6
Relationship Between x and y Covariance Matrices y = Gx = D t x Each component of y is a linear combination of all of the elements of x; the weighting coefficients are the elements of the matrix G (or D T ) S y = ξ[(y-m y )(y-m y ) t ] m y = ξ[y] = ξ[d t x] = D t ξ [x] = D t m x S y = ξ[(d t x-d t m x )(D t x-d t m x ) t ] S y = D t ξ[(x-m x )(x-m x ) t ]D S y = D t S x D S x is the covariance of the pixel data in x space (S y in y) - m x and m y are the mean vectors in x and y respectively - ξ is the Expected value (here taken as the mean for m) Identifying a y coordinate space in which the pixel data exhibits no correlation requires S y to be a diagonal matrix GEOG 4110/5100 7
Eigenvalues and Eigenvectors When we have a transformation matrix operating on a vector, a new vector is produced: Sometimes that new vector is simply the product of a scalar and the original vector Eigenvalue Eigenvector When this is the case, the scalar is referred to as the Eigenvalue, and the vector is referred to as the Eigenvector http://www.ce.yildiz.edu.tr/personal/songul/file/1097/principal_components.pdf (simple explanation) http://math.mit.edu/linearalgebra/ila0601.pdf (more complex explanation) GEOG 4110/5100 8
Eigenvalues and Eigenvectors Eigenvalues (l) and eigenvectors (x) of a Matrix (M) are scalar and vector terms such that the multiplication of x by l has the same result as the matrix transformation of x by matrix M or Mx = lx (i.e. y = lx is equivalent to y = Mx) Mx - lx = 0 à (M-lI)x =0; where I is the identity matrix (x is a vector with n elements, where n = number of bands) For the above to be true, then either x = 0 or M-lI = 0 This is the characteristic equation from which the eigenvalues (l) can be determined When plugged into the equation: (M-lI)x =0, the eigenvectors (x) can be determined GEOG 4110/5100 9
Calculating Determinants From: http://www.mathsisfun.com/algebra/matrix-determinant.html
Principal Component Transformation S y = D t S x D - S x is the covariance of the pixel data in x space - D is a matrix of Eigenvectors derived from S x - The covariance matrix in y-space is given by: y $ λ 1 0 0 ' & ) 0 λ 2 0 = & ) & ) & ) % 0 0 λ N ( Where N is the dimensionality, and l i represents the eigenvalues in descending order The n th component (n = 1 N) represents z percent of the variance where λ n ζ n = λ 1 + λ 2...+ λ n - S y is by definition a diagonal covariance matrix with its elements representing the variance in the transformed coordinates - The greatest variance occurs in the first dimension of the transformed coordinate system, the next greatest in the 2 nd, and so-on such that the least variance is found in the n th dimension GEOG 4110/5100 11
Principal Component Transformation The eigenvectors determine the transformation matrix that produces each principal component The eigenvalue describes the percentage of the variance that is contained within each principal component The higher the eigenvalue as a fraction of the sum of the eigenvalues, the more relative information is contained in the corresponding principal component GEOG 4110/5100 12
Principal Components Transformation Example in 2 dimensions # = 2.40 0 & % ( x $ 0 1.87' x # 1.9 1.1& = % ( $ 1.1 1.1' GEOG 4110/5100 13
Principal Components Transformation Example in 2 dimensions x # 1.9 1.1& = % ( $ 1.1 1.1' First we need to find the eigenvalues S x li = 0 1.9 λ 1.1 1.1 1.1 λ = 0 l 2-3.0l + 0.88 = 0 à l = 2.67 and 0.33 GEOG 4110/5100 14
Principal Components Transformation Example in 2 dimensions " = $ 2.67 0 y # 0 0.33 % ' & First component contains 2.67/(2.67+0.33) = 89% of the variance in this example (usually we order the eigenvalues in descending order) Now we seek to find the principal components transformation matrix G = D T Where D T is the transposed matrix of eigenvectors. The first eigenvector (g 1 )corresponds to the first eigenvalue l 1 " [S x li]g 1 = 0 with g 1 = g % 11 t $ ' = d 1 for the two dimensional case # & g 21 Substituting S x and l 1 (2.67) gives the pair of equations: -0.77g 11 + 1.10g 21 = 0 1.10g 11 1.57g 21 = 0 yields g 11 =1.43g 21 GEOG 4110/5100 15
Principal Components Transformation Example in 2 dimensions We have the added constraint that the eigenvectors must be normalized (i.e. the G matrix must be orthogonal such that G t = G -1 ) (g 11 ) 2 + (g 21 ) 2 = 1 This produces the following eigenvectors " g 1 = 0.82 % # $ ' g # 0.57 2 = 0.57 & % ( & $ 0.82 ' g 2 is the 2 nd eigenvector derived from the 2 nd eigenvalue (replace 2.67 on previous page with 0.33) Which in turn produce the following transformation matrix " G = D t = $ # 0.82 0.57 0.57 0.82 t % ' & " = $ # 0.82 0.57 0.57 0.82 % ' & Remember, D is the matrix of eigenvectors GEOG 4110/5100 16
Principal Components Transformation Example in 2 dimensions GEOG 4110/5100 17
c (a) Four Landat MSS bans for the region of Andamooka in Central Australia; (b) The four principal components of the image segment; (c) comparison of standard false color composite (R=band 7; G=band 5; B=band 4) with a principal component composite (R, G, B are 1st, 2nd, and 3rd components respectively) a b
Highly correlated bands 1, 2, and 3 b a c d e 19
a b Bands 4, 3, 2 c PC3, PC2, PC1 d PC4, PC3, PC2 e 20
a b c d e 21
Principal Component Transformation Steps 1. Compute the covariance matrix of the data set in vector space 2. Calculate the eigenvalues of the covariance matrix 3. The diagonal matrix with the eigenvalues along the diagonal will be the covariance matrix of the transformed axes (principal component axes) 4. Find the matrix of eigenvectors (D i ) for each individual l of interest by solving for [S x l i I]g i = 0. for that l. 5. Transpose the Matrix D to produce principal component transformation matrix (g). The number of rows in g will equal the number of spectral dimensions from which the eigenvalues and eigenvectors were calculated 6. For each g matrix (derived from a given l) the original data values (in original x coordinate system) are multiplied by the rows in g (g 1, g 2, g n where n is the number of dimensions in vector space), to produce coordinates in the transformed dimension (new y coordinate system). Each axis in the original spectral space will be multiplied by its corresponding row in the g matrix to produce the transformed coordinate system (principal component) 7. Steps 4 6 are repeated until the desired number of principal component transformations have been executed.
Principal Components Analysis (PCA) PCA is a technique that transforms the original vector image data into smaller set of uncorrelated variables. The variables represent most of the image information and easier to interpret. Principal components are derived such that the first PC accounts for much of the variation of the original data. The second (vertical) accounts for most of the remaining variation. PCA is useful in reducing the dimensionality (number of bands) that used for analysis. Minimum noise fraction (MNF) method can be used with hyperspectral data for noise reduction. GEOG 4110/5100 23
TM Example for PC Transformation Compute the n-dimensional covariance matrix (7 x 7 for Landsat TM). The variances of the principal components (eigenvalues) contain useful information (e.g. determine the % of total variance explained by each of the principal components) eigenvalue of the p th component sum of the eigenvalue of all components Band Number 1 2 3 4 5 7 6 Variance 100.93 34.14 68.83 248.40 568.84 154.92 17.78 Table shows the variance of different bands of TM scene. Adapted from Jensen, 2005. GEOG 4110/5100 24
TM Example for PC Transformation Component p (eigenvalues) 1 2 3 4 5 6 7 eigenvalue 1010.92 131.20 37.60 6.73 3.95 2.17 1.24 Table shows the variance of different principal components. Adapted from Jensen, 2005. Sum of eigenvalues of all components = 1193.81 % of variance explained by PC1 = (1010.92/1193.81)*100 = 84.68% % of variance explained by PC2 = (131.2/1193.81)*100 = 10.99 % Band Component p 1 2 3 4 5 6 7 1 0.205 0.637 0.327-0.054 0.249-0.611-0.079 2 0.127 0.342 0.169-0.077 0.012 0.396 0.821 3 0.204 0.428 0.159-0.076-0.075 0.649-0.562 4 0.443-0.471 0.739 0.107-0.153-0.019-0.004 5 0.742-0.177-0.437-0.300 0.370 0.007 0.011 7 0.376 0.197-0.309-0.312-0.769-0.181 0.051 6 0.106 0.033-0.080 0.887 0.424 0.122 0.005 Table shows the eigenvectors (coefficients) for each principal component in each column. Adapted from Jensen, 2005. 25
calculated to determine which band is associated with each principal TM component. Example This for helps PC in Transformation understanding the information contains by each component. How principal component images are created? Identify the original brightness values of a given pixel (e.g. the first pixel at column 1 and row 1). Obtain the new pixel value by summation of the multiplication of the eigenvector of the component of each band by the original value GEOG 4110/5100 26
TM Example for PC Transformation Band DN PC1 eigenvector 1 20 0.205 2 30 0.127 3 22 0.204 4 60 0.443 5 70 0.742 7 62 0.376 New value = 0.205(20) + 0.127(30) + 0.106(50) = 119 This will be carried out for each pixel to produce the PC1 image. If the first three components can explain most of the variation in the data, further analysis can be performed using the transformed images. 6 50 0.106 GEOG 4110/5100 27
TM Example for PC Transformation 7 PC images of TM data. PC1 (bands 4, 5, 7; near infrared and middle infrared). PC2 (bands 1, 2, 3; visible). PC3 (near infrared). PC4 (band 6, thermal). The four PC accounts for 99.38 % of the variance. From Jensen (2005) GEOG 4110/5100 28
Back-up GEOG 4110/5100 29