Example: Current in an Electrical Circuit. Solving Linear Systems:Direct Methods. Linear Systems of Equations. Solving Linear Systems: Direct Methods

Transcription

1 Example: Current in an Electrical Circuit Solving Linear Systems:Direct Methods A number of engineering problems or models can be formulated in terms of systems of equations Examples: Electrical Circuit Analysis, Radiative Heat Transfer (or equivalently, global diffuse illumination compuation), etc. Need a variety of computational techniques, depending on nature of system (sparse systems, for instance, benefit from iterative solutions). Systems can be very large, thousands of equations is not uncommon. Large systems require computational/numerical solutions. ITCS 4133/5133: Intro. to Numerical Methods 1 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 2 Solving Linear Systems Linear Systems of Equations Solving Linear Systems: Direct Methods Basic Gaussian Elimination (GE) Gaussian Elimination with Row Pivoting Tridiagonal Systems (Gauss-Thomas Method) Gauss-Jordan Elimination a 11 X 1 +a 12 X 2 + +a 1n Xn = C 1 a 21 X 1 +a 22 X 2 + +a 2n Xn = C a n1 X 1 +a n2 X 2 + +a nn Xn = C n where a ij s are the coefficients, X j s are the unknowns, and C i s are the known constants. Assumptions Number of unknowns equal the number of equations Equations are linearly independent ITCS 4133/5133: Intro. to Numerical Methods 3 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 4 Solving Linear Systems

2 Solution of Small Systems of Equations For small systems such as can be solved by substitution a 11 X 1 + a 12 X 2 = C 1 a 21 X 1 + a 22 X 2 = C 2 Complex engineering problems could contain thousands of equations. Substitution is not a feasible technique for large systems With solutions Without solutions With infinite solutions Linear System Classification ITCS 4133/5133: Intro. to Numerical Methods 5 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 6 Solving Linear Systems Gaussian Elimination Gaussian Elimination:Matrix Representation Forward Pass: convert the system of equations into an upper triangular matrix, Back Substitution: to solve the system. Permissible Operations Changing the order of the equations does not change the solution An equation may be scaled (multiplied/divided) by a non-zero constant without changing the solution May add 2 equations together and replace either one with the new equation, termed (linear combination) or, a 11 a 12 a 1n X 1 V 1 a 21 a 22 a 2n X = V 2. a n1 a n2 a nn X n a 11 a 12 a 1n C 1 a 21 a 22 a 2n C a n1 a n2 a nn C n V n ITCS 4133/5133: Intro. to Numerical Methods 7 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 8 Solving Linear Systems

3 Gaussian Elimination:Forward Pass Results in the upper triangular matrix: 1 d 12 d 13 d 1n e1 0 1 d 23 d 2n e d 3n e e n Gaussian Elimination:Back Substitution X 1 + d 12 X 2 + d 13 X d 1,n 1 X n 1 + d 1n X n = e 1 X 2 + d 23 X d 2,n 1 X n 1 + d 2n X n = e 2 X d 3,n 1 X n 1 + d 3n X n = e 3 X n 2 + d n 2,n 1 X n 1 + d n 2,n X n = e n 2 X n 1 + d n 1,n X n = e n 1 X n = e n Implementation Diagonalize the matrix, resulting in the solution vector in the last column. ITCS 4133/5133: Intro. to Numerical Methods 9 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 10 Solving Linear Systems Forward Pass: Gaussian Elimination:Algorithm Loop over each row i, making it the pivot row Normalize the pivot row, i (pivot element becomes 1) a ij = a ij /a ii, j = i + 1, i + 2,..., (n + 1) C i = C i /a ii a ii = 1 Loop over rows (i + 1) to n and reduce elements in each row. a kj = a kj a ki a ij, j = i,..., n C k = C k a ki C i, k = (i + 1),..., n ITCS 4133/5133: Intro. to Numerical Methods 11 Solving Linear Systems Gaussian Elimination:Algorithm Forward Pass (Alternate) Uses a multiplier, without normalizing pivot element Input: a ij, b i, n Column: for k = 1 to n 1 for i = k + 1 to n Multiplier: m ik = a ik /a kk Row: for j = k + 1 to n a ij = a ij m ik a kj end b i = b i m ik b k end end Output: Upper Triangular Matrix ITCS 4133/5133: Intro. to Numerical Methods 12 Solving Linear Systems

4 Gaussian Elimination: Algorithm Gaussian Elimination:Algorithm Back Substitution: For the last row, X n = Cn For rows (n 1) through 1, n X i = C i a ij X j, j = (i + 1),..., n and i = (n 1),..., 1 j=i+1 ITCS 4133/5133: Intro. to Numerical Methods 13 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 14 Solving Linear Systems Example 1:Electrical Circuit Roundoff Errors and Pivoting Truncation and Roundoff errors can occur when coefficients differ by orders of magnitude. To minimize errors, diagonal elements should contain the largest coefficient This can be accomplished by swapping rows during Gaussian elimination, a process called partial pivoting Zero Pivot Element: rearrange equations to prevent zero coefficients in the pivot element Equations must be linearly independent, else we have a singular coefficient matrix ITCS 4133/5133: Intro. to Numerical Methods 15 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 16 Solving Linear Systems

5 GE with Pivoting: Motivation GE with Pivoting: Algorithm Consider the following example: ITCS 4133/5133: Intro. to Numerical Methods 17 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 18 Solving Linear Systems GE with Pivoting: Example Banded Systems Systems of equations with banded structure, with non-zero elements on the diagonal and a small number of off-diagonals. Such systems can be solved more efficiently. Example: Tridiagonal system d 1 x 1 a 1 x 2 = r 1 b 2 x 1 d 2 x 2 a 2 x 3 = r b n 1 x n 2 d n 1 x n 1 a n 1 x n = r n 1 b n x n 1 d n x n = r n Both memory and computational savings! ITCS 4133/5133: Intro. to Numerical Methods 19 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 20 Solving Linear Systems

6 Gauss-Thomas Method Representation:Vectors d (diagonal), a (above diagonal elements), b (below diagonal elements) Example: Gauss-Thomas Method (contd) ITCS 4133/5133: Intro. to Numerical Methods 21 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 22 Solving Linear Systems Gauss-Thomas Method:Algorithm Gauss-Jordan Elimination In Gaussian elimination, we create upper triangular matrix, followed by back-substitution In Gauss-Jordan elimination, the input matrix is diagonalized in the forward pass, completing the solution More efficient when the same system is used to solve multiple rightside vectors. ITCS 4133/5133: Intro. to Numerical Methods 23 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 24 Solving Linear Systems

7 Matrix Condition Number Defined for non-singular matrices Defined in terms of the norm of the matrix, A Can be derived from vector norms: Matrix norms are defined as x 1 = x 1 + x x n x 2 = ( x x x n 2 ) 1/2 x = Max x i A 1 = max(over j) [sum of column j] A = max (over i) [sum of row i] A 2 = ( Max Eigen value of A H A) 1/2 Matrix Condition Number (contd) Ill-conditioned: Sensitive to small changes in parameters defining the problem. In general, condition number is defined as Spectral condition number: κ(a) = A A 1 κ(a) = A 2 A 1 2 Relative change in solution : depends on product of condition number and relative change in RHS vector (b). ITCS 4133/5133: Intro. to Numerical Methods 25 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 26 Solving Linear Systems Analysis: Gaussian Elimination Why does Gaussian Elimination work? Equation forming a linear combination of two equations also passes through a point common (intersection) to both equations. Consider S 1 : a 0 + a 1 x a n x n = 0 T 1 : a 0 + b 1 x b n x n = 0 If r = (r 1, r 2,..., r n ) is a solution vector it also satisfies m 1 S 1 + m 2 T 1 Elimination produces a system solvable by back substitution. ITCS 4133/5133: Intro. to Numerical Methods 27 Solving Linear Systems Measure Flops. Analysis: Gaussian Elimination - Computational Complexity Row calculations: multiplier calculation (1) + n multiplications and n additions + RHS update Repeat for (n 1) rows, resulting in (n+1)(n-1) multiplications and divisions, n(n-1) additions (n k + 1)(n k) = = n 2 2nk + k 2 + n k (n 2 + n) (2n + 1)k + = n3 3 n 3 ITCS 4133/5133: Intro. to Numerical Methods 28 Solving Linear Systems k 2

8 Analysis: GE with Pivoting Some form of row scaling can also be used. However, ill-conditioned matrices are not helped by scaling. Scaling is difficult to incorporate in general purpose packages. Computational complexity is O(n 3 ), so preferable to use more efficient methods. Analysis: GT Method (Tridiagonal Systems) First equation: 2 divisions (scaling the equation) Next (n-2) equations: 2 mults, 2 divisions. Last equation: 2 mults, 1 division Total (Forward Step): 5 + 4(n 2) Back Substitution: (n-1) multiplies. ITCS 4133/5133: Intro. to Numerical Methods 29 Solving Linear Systems ITCS 4133/5133: Intro. to Numerical Methods 30 Solving Linear Systems