Numerical Linear Algebra SEAS Matlab Tutorial 2

Transcription

1 Linear System of Equations Numerical Linear Algebra SEAS Matlab utorial Linear system of equations. Given n linear equations in n unknowns. Matri notation: find such that A b A, b Kevin Wayne Comuter Science Deartment Princeton University Fall 7 Among most fundamental roblems in science and engineering. Chemical equilibrium. see Lab Googles PageRank algorithm. Linear and nonlinear otimization. Kirchoffs current and voltage laws. Hookes law for finite element methods. Numerical solutions to differential equations. Chemical Equilibrium Circuit Analysis E: combustion of roane. E: find current flowing in each branch of a circuit. C H 8 + O CO + H O Stoichiometric constraints. Carbon: Hydrogen: 8 conservation of mass Oygen: + Normalize: C H 8 + 5O CO + H O Kirchoffs current law conservation of electrical charge Solution:.9,.,.66.

2 5 Gaussian Elimination Gaussian elimination. Among oldest and most widely used solutions. Reeatedly aly row oerations until system is uer triangular. Solve trivial uer triangular system via back substitution. >> A [ ; -; 5]; >> b [; ; 6]; >> lsolvea, b we are going to imlement this 6 Gaussian Elimination >> A [ ; -; 5] A - 5 >> A[ ], : A[ ], : A - 5 >> A, : A, : - A, : A - swa rows and subtract times row from row declare a matri 7 Elementary Row Oerations Elementary row oerations. Echange row and row q. Add a multile of row to row q. Key invariant. Row oerations reserve solutions. q A[ q], : A[q ], :; b[ q], : b[q ], :; Aq, : Aq, : - alha A, :; bq, : bq, : - alha b, :; 8 interchange row and subtract row from row Gaussian Elimination: Row Oerations

3 Gaussian Elimination: Back Substitution Gaussian Elimination: Back Substitution Back substitution. Uer triangular systems are easy to solve by eamining equations in reverse order. Back substitution. Uer triangular systems are easy to solve by eamining equations in reverse order. Eq. / Eq. Eq. + / - Eq. / Eq. Eq. + / - i n b i a ij j a ii j i+ [m n] sizea; zerosn, ; for i n : - : total.; for j i+ : n total total + Ai, j j; i bi - total / Ai, i; i n b i a ij j a ii j i+ vectorized version [m n] sizea; zerossizeb; vector inner roduct for i n : - : j i+ : n i, : bi, : - Ai, j j, : / Ai, i; 9 Gaussian Elimination: Forward Elimination Gaussian Elimination: Forward Elimination Forward elimination. Aly row oerations to make uer triangular. Pivot. Zero out entries below ivot a. Forward elimination. Aly row oerations to make uer triangular. Pivot. Zero out entries below ivot a. a i / a a ij a ij a j b i b i b for i + : n alha Ai, / A, ; bi, : bi, : - alha b, :; Ai, : Ai, : - alha A, :; for : n for i + : n alha Ai, / A, ; bi, : bi, : - alha b, :; Ai, : Ai, : - alha A, :;

4 Gaussian Elimination: Partial Pivoting Gaussian Elimination: Partial Pivoting Remark. Code on revious slide fails sectacularly if ivot a. Partial ivoting. Swa row with the row q that has largest entry in column among rows below the diagonal. Remark. Code on revious slide fails sectacularly if ivot a. Partial ivoting. Swa row with the row q that has largest entry in column among rows below the diagonal. q ; for i + : n if absai, > absaq, q i; A[ q], : A[q ], :; b[ q], : b[q ], :; q 9 [val q] maabsa:n, ; q q + - ; A[ q], : A[q ], :; b[ q], : b[q ], :; vectorized version q 9 Gaussian Elimination with Partial Pivoting function lsolvea, b LSOLVE Linear system of equation solver, bare bones version lsolvea, b returns the solution to the equation A b, where A is an n-by-n nonsingular matri, and b is a column vector of length n or a matri with several such columns. [m n] sizea; Gaussian elimination with artial ivoting for : n find inde q of largest element below diagonal in column [val q] maabsa:n, ; q q + - ; swa with row A[ q], : A[q ], :; b[ q], : b[q ], :; A \ b; zero out entries of A and b using ivot A, for i + : n alha Ai, / A, ; bi, : bi, : - alha b, :; Ai, : Ai, : - alha A, :; back substitution zerossizeb; for i n : - : j i+ : n; i, : bi, : - Ai, j j, : / Ai, i; 5 6

5 Singular Value Decomosition Princial Comonent Analysis Singular value decomosition. Given a real, square matri A, the SVD is A U S V, where U and V are orthogonal, and S is diagonal. U U I singular values in descing order Among most imortant concets in matri comutation. Alications: statistics, signal rocessing, acoustics, vibrations,. Princial comonent analysis PCA. runcated SVD is A r U r S r V r, where U r and V r are the first r columns of U and V, and S r is the first r rows and columns of S. Fact. A r is the best rank r aroimation to A A U S V A U S V r A r U r S r V r Ar A 7 8 Image Processing: PCA baboon.m Image rocessing. Read in color image. Convert to grayscale. Create n-by-n matri of grayscale values. Comute best rank {,, 5,, 5, 5 } aroimation. MALAB scrit that reads in the image baboon.jg, converts it to grayscale, and forms a matri of its grayscale values. hen it comutes and lots the best rank r aroimate to the matri using the SVD. It saves each aroimation as a JPEG image. A imreadbaboon.jg; A rgbgraya; A imdoublea; imshowa; read image from a file convert from color to grayscale convert to double recision matri dislay the image in a window [U S V] svda; for r [ ] Ar U:, :r S:r, :r V:, :r; imshowar; ause; imwritear, srintfbaboon-d.jg, r; 9

6 Faces 7 Princial Faces Reference: Diego Nehab, COS 96 average