LECTURE 3 LINEAR ALGEBRA AND SINGULAR VALUE DECOMPOSITION

Transcription

1 SCIENTIFIC DATA COMPUTING 1 MTAT LECTURE 3 LINEAR ALGEBRA AND SINGULAR VALUE DECOMPOSITION Prepared by: Amnir Hadachi Institute of Computer Science, University of Tartu amnir.hadachi@ut.ee

2 OUTLINE Matrix terminology Basics of singular value decomposition Full SVD Reduced SVD SVD Applications Image compression

3 1. MATRIX TERMINOLOGY

4 MATRIX TERMINOLOGY Square Matrix size of matrix is defined by number of rows (n) and columns (m): nxm Square matrix nxn Matrix Transpose

5 BASICS OF THE SINGULAR VALUE DECOMPOSITION Matrix multiplication

6 BASICS OF THE SINGULAR VALUE DECOMPOSITION Identity matrix

7 BASICS OF THE SINGULAR VALUE DECOMPOSITION Orthogonal matrix Example:

8 BASICS OF THE SINGULAR VALUE DECOMPOSITION Determinant

9 BASICS OF THE SINGULAR VALUE DECOMPOSITION Eigenvectors and Eigenvalues Eigenvector is nonzero vector that satisfy the equation: EIGENVECTOR SQUARE MATRIX SCALAR / EIGENVALUES

10 BASICS OF THE SINGULAR VALUE DECOMPOSITION Eigenvectors and Eigenvalues Example:

11 BASICS OF THE SINGULAR VALUE DECOMPOSITION Eigenvectors and Eigenvalues Example: In order that our system have a nonzero vector then the determinant of the coefficient matrix should be equal to zero Thus we have two possible eigenvalues :

12 BASICS OF THE SINGULAR VALUE DECOMPOSITION Eigenvectors and Eigenvalues Example: Eigenvalues Eigenvectors

13 2. BASICS SINGULAR VALUE DECOMPOSITION

14 BASICS OF THE SINGULAR VALUE DECOMPOSITION Singular value decomposition known as SVD ORTHOGONAL ORTHOGONAL DIAGONAL dn 3 7 5

15 BASICS OF THE SINGULAR VALUE DECOMPOSITION Full SVD DEFINTION: is a diagonal matrix where the values are the square roots of eigenvalues from U or V in defending order.

16 BASICS OF THE SINGULAR VALUE DECOMPOSITION Full SVD Example: Finding U Eigenvalues

17 BASICS OF THE SINGULAR VALUE DECOMPOSITION Full SVD Finding U Eigenvectors Last step is to transform the matrix into an orthogonal matrix by applying Gram-Schmidt orthonormalisation process to the column vectors

18 BASICS OF THE SINGULAR VALUE DECOMPOSITION Full SVD Finding U Gram-Schmidt process 1. normalising:! w 2 =!! v 2 u2!v 2 u! 1 =[1, 1] [ 1 p 2, 1 p ] [1, 1] [ p 1, p ] 2! u 2 = =[1, 1] 0 [ 1 p 2,! w 2 w! 2 =[ p 1, 2 1 p ] 2 1 p ]=[1. 1] [0, 0] = [1, 1] 2

19 BASICS OF THE SINGULAR VALUE DECOMPOSITION Full SVD Finding U U = " 1 p2 1 p2 p1 p2 1 2 # Finding V Transpose the same process should be applied but for: A T A V = p1 p5 2 p p2 p1 p p1 0 p V T = p1 p6 2 p6 1 6 p2 p p1 p2 p

20 BASICS OF THE SINGULAR VALUE DECOMPOSITION Full SVD Finding A = U V T = = " 1 p2 1 p2 p1 p2 1 2 applep p # applep p p1 p6 2 p6 1 6 p2 p p1 p2 p = apple

21 BASICS OF THE SINGULAR VALUE DECOMPOSITION Reduced SVD Theory: A = U V T A T A = V U T U V T = V 2 V T AA T = U V T V U T = U 2 U T Thus by founding the normalised eigenvectors are found for these two equations, then the orthonormal basis vectors are produced for U and V.

22 BASICS OF THE SINGULAR VALUE DECOMPOSITION Reduced SVD Example A = apple A T A = AA T = apple apple apple apple = 0 4 apple apple = 0 4 Eigenvalues: = {9, 4} 1 =3and 2 =2 Eigenvectors: [±1, 0] and [0, ±1] After normalisation: apple 1 u 1 = 0 apple 1 v 1 = 0 apple 0 u 2 = 1 apple 0 v 2 = 1

23 BASICS OF THE SINGULAR VALUE DECOMPOSITION Reduced SVD Example A = U V T = apple apple apple

24 3. SVD APPLICATIONS

25 SVD APPLICATIONS Given an original image: Any I'mage is a matrix, thus we can write it as a matrix that we can decomposed it using SVD. A = U V T

26 SVD APPLICATIONS The matrix A can be also writing as sum of r rank-one matrices: rx A = ju j vj for r=k A j=1 U V T N X R K X K = N X K * * i 0 K X R SINGULAR VALUES

27 SVD APPLICATIONS Image Singular value decomposition spectrum number of columns

28 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=1. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:1])*np.diag(sigma[:1])* np.matrix(v[:1,:]) Original Image r=1

29 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=10. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:10])*np.diag(sigma[:10])* np.matrix(v[:10,:]) Original Image r=10

30 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=20. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:20])*np.diag(sigma[:20])* np.matrix(v[:20,:]) Original Image r=20

31 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=30. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:30])*np.diag(sigma[:30])* np.matrix(v[:30,:]) Original Image r=30

32 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=40. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:40])*np.diag(sigma[:40])* np.matrix(v[:40,:]) Original Image r=40

33 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=50. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:50])*np.diag(sigma[:50])* np.matrix(v[:50,:]) Original Image r=50

34 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=100. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:100])*np.diag(sigma[:100])* np.matrix(v[:100,:]) Original Image r=100

35 SVD APPLICATIONS Now lets reconstruct the original image using only one subset which means for r=200. U, S, V = svd(image_matrix) rec = np.matrix(u[:,:200])*np.diag(sigma[:200])* np.matrix(v[:200,:]) Original Image r=200

36 SVD APPLICATIONS Code and Demo import cv2 import numpy as np import matplotlib.pyplot as plt import sys, getopt Importing the opencv for decoding the image in order to use with numpy matplotlib is for plotting the processed image sys and getopt for managing the argument and options input file def read_img(image): data = image.read() print 'Loaded [%d] bytes of %s image' % (len(data), image) np_data = np.fromstring(data, count=len(data), dtype='uint8') print 'Converted input image data to NumPy array' img = cv2.imdecode(np_data,cv2.cv_load_image_grayscale) print 'Parsed image data into OpenCV data structure' return img read_img function read the image as data and decode into numpy arrays in order to use later as a matrix and apply the SVD decomposition.

37 SVD APPLICATIONS Code and demo def svd(img_matrix): U, Sigma, V = np.linalg.svd(img_matrix) return U, Sigma, V applying the singular value decomposition using numpy function svp() which will return the U, Sigma, and V matrices. def svd_compression(u,sigma,v,level): rec = np.matrix(u[:,:level])*np.diag(sigma[:level])* np.matrix(v[:level,:]) return rec def img_view(img_rec): print img_rec.size() plt.imshow(img_rec, cmap='gray') plt.show() This function is used to reconstructed the original function using the singular value decomposition matrices and the specified rank which will influence the resolution and the size of the image reconstructed This function just for viewing the output.

38 SVD APPLICATIONS Code and demo def main(argv): print 'argv: ', argv inputfile = sys.stdin from argparse import ArgumentParser # Parsing arguments parser = ArgumentParser() parser.add_argument('-i','--ifile', \ help='input image file', \ default='-') parser.add_argument('-l','--level', \ help='matrix rank', \ default=1) args = parser.parse_args() if args.ifile!= '-': inputfile = open(args.ifile,'rb') main function image_matrix=read_img(inputfile) U, S, V = svd(image_matrix) img_comp = svd_compression(u,s,v,int(args.level)) img_view(img_comp) if name == ' main ': main(sys.argv)