5. Direct Methods for Solving Systems of Linear Equations. They are all over the place...

Transcription

1 5 Direct Methods for Solving Systems of Linear Equations They are all over the place Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

2 51 Preliminary Remarks Systems of Linear Equations Another important field of application for numerical methods is numerical linear algebra that deals with solving problems of linear algebra numerically (matrix-vector product, finding eigenvalues, solving systems of linear equations) Here, we consider the solution of systems of linear equations, eg, to compute the intersection point of three planes 3x + 2y z = 1 2x 2y + 4z = 2 4x + 2y 4z = 0 Find three ways how to solve this system! wikipedia Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

3 Cramer s rule: = = 3(8 8) 2( ) (4 8) = 12, x = y = z = = = = = 1, = 2, = 2 12 Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

4 Compute A 1 : a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 = and get the solution as x y z = A This is equivalent to solving the three linear systems a i1 a i2 a i3 = δ 1i δ 2i δ 3,i, i = 1, 2, 3 Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

5 From the first line of the system, we get: 3x + 2y z = 1 2x 2y + 4z = 2 4x + 2y 4z = 0 x = 1 (1 2y + z) (1) 3 Replacing x by this formula in three times the second and third row yields: The first of these two equations gives 10y + 14z = 8, 14y 16z = 4 We replace y in the second equation (scaled with 5 2 ) by this formula: y = 1 (4 + 7z) (2) 5 9z = 18 z = 2 Substituting this result back in (2) yields y: y = 1 5 (4 14) = 2 Substituting y and z in (1) finally gives x: x = 1 3 ( ) = 1 Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

6 A general linear system with n unknowns: Find x R n with A x = b, where A = (a i,j ) 1 i,j n = b 1 R n,n, b = (b i ) 1 i n = b n a 1,1 a 1,n a n,1 a n,n R n Linear systems of equations are omnipresent in numerics: interpolation: construction of the cubic spline interpolant (see section 33) boundary value problems (BVP) of ordinary differential equations (ODE) (see chapter 8) partial differential equations (PDE) Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

7 In terms of the problem given, one distinguishes between: full matrices: the number of non-zero values in A is of the same order of magnitude as the number of all entries of the matrix, ie O(n 2 ) sparse matrices: here, zeros clearly dominate over the non-zeros (typically O(n) or O(n log(n)) non-zeros); those sparse matrices often have a certain sparsity pattern (diagonal matrix, tridiagonal matrix (a i,j = 0 for i j > 1), general band structure (a i,j = 0 for i j > c) etc), which simplifies solving the system diagonal 2c band (bandwidth 2c) Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations tridiagonal block-diagonal They are all over the place, December 13,

8 Systems of Linear Equations (2) One distinguishes between different solution approaches: direct solvers: provide the exact solution x (modulo rounding errors) (covered in this chapter); indirect solvers: start with a first approximation x (0) and compute iteratively a sequence of (hopefully increasingly better) approximations x (i), without ever reaching x (covered in chapter 6) Reasonably, we assume in the following an invertible or non-singular matrix A, ie det(a) 0 or rank(a) = n or Ax = 0 x = 0, respectively Two approaches that seem obvious at first sight are considered as numerical mortal sins for reasons of complexity: x := A 1 b, ie the explicit computation of the inverse of A; The use of Cramer s rule (via the determinant of A and of the n matrices which result from A by substituting a column with the right-hand side b) Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

9 Of course, the following general rule also applies to numerical linear algebra: Have a close look at the way a problem is posed before starting, because even the simple term y := A B C D x, A, B, C, D R n,n, x R n, can be calculated stupidly via y := (((A B) C) D) x with O(n 3 ) operations (matrix-matrix products!) or efficiently via y := A (B (C (D x))) with O(n 2 ) operations (only matrix-vector products!)! Keep in mind for later: Being able to apply a linear mapping in form of a matrix (ie to be in control of its effect on an arbitrary vector) is generally a lot cheaper than via the explicit construction of the matrix! Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

10 Vector Norms In order to analyze the condition of the problem of solving systems of linear equations as well as to analyze the behavior of convergence of iterative methods in chapter 6, we need a concept of norms for vectors and matrices A vector norm is a function : R n R with the three properties positivity: x > 0 x 0; homogeneity: αx = α x for arbitrary α R; triangle inequality: x + y x + y The set {x R n : x = 1} is called norm sphere regarding the norm Examples for vector norms relevant in our context (verify the above norm attributes for every one): Manhattan norm: x 1 := n i=1 x i n Euclidean norm: x 2 := i=1 x i 2 maximum norm: x := max 1 i n x i (the common vector length) Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

11 Matrix Norms By extending the concept of vector norm, a matrix norm can be defined or rather induced according to A := max Ax x =1 In addition to the three properties of a vector norm (rephrased correspondingly), which also apply here, a matrix norm is sub-multiplicative: AB A B ; consistent: Ax A x The condition number κ(a) is defined as κ(a) := max x =1 Ax min x =1 Ax κ(a) indicates how strongly the norm sphere is deformed by the matrix A or by the respective linear mapping In case of the identity matrix I (ones in the diagonal, zeros everywhere else) and for certain classes of matrices there are no deformations at all in these cases we have κ(a) = 1 For non-singular A, we have Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations κ(a) = A A 1 They are all over the place, December 13,

12 The Condition of Solving Linear Systems of Equations Now, we can determine the condition of the problem of solving a system of linear equations: In (A + δa)(x + δx) = b + δb, we have to deduce the error δx of the result x from the perturbations δa, δb of the input A, b Of course, δa has to be so small that the perturbed matrix remains invertible (this holds for example for changes with δa < A 1 1 ) Solve the relation above for δx and estimate with the help of the sub-multiplicativity and the consistency of an induced matrix norm (this is what we needed it for!): A 1 δx 1 A 1 ( δb + δa x ) δa Now, divide both sides by x and keep in mind that the relative input perturbations δa / A as well as δb / b should be bounded by ε (because we assume small input perturbations when analyzing the condition) δx x because b = Ax A x ( ) εκ(a) b 1 εκ(a) A x + 1 2εκ(A) 1 εκ(a) Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

13 The Condition of Solving Linear Systems of Equations (2) Because it s so nice and so important, once more the result we achieved for the condition: δx x 2εκ(A) 1 εκ(a) Thus, we have κ(a) >> 1 ɛκ(a) 1 } The bigger the condition number κ(a) is, the bigger our upper bound on the right for the effects on the result becomes, the worse the condition of the problem solve Ax = b gets The term condition number therefore is chosen reasonably it represents a measure for condition Only if εκ(a) 1, which is restricting the order of magnitude of the acceptable input perturbations (before the oerturbed A might become singular), a numerical solution of the problem makes sense In this case, however, we are in control of the condition Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

14 At the risk of seeming obtrusive: This only has to do with the problem (ie the matrix) and nothing to do with the rounding errors or approximate calculations! Note: The condition of a problem can often be improved by adequate rearranging If the system matrix A in Ax = b is ill-conditioned, the condition of the new system matrix MA in MAx = Mb might be improved by choosing a suitable prefactor M Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

15 The Residual An important quantity is the residual r For an approximation x of x, r is defined as r := b A x = A(x x) =: Ae with the error e := x x Caution: Error and residual can be of very different order of magnitude In particular, from r = O( ε) does not at all follow e = O( ε) the correlation even contains the condition number κ(a), too Nevertheless the residual is helpful: Namely, r = b A x A x = b r shows that x can be interpreted as exact result of slightly perturbed input data (A original, instead of b now b r) for small residual; therefore, x is an acceptable result in terms of chapter 2! We mainly use the residual for the construction of iterative solvers in chapter 6: In contrast to the unknown error, the residual can easily be determined in every iteration step Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

16 52 Gaussian Elimination The Principle of Gaussian Elimination The classical solution method for systems of linear equations, familiar from linear algebra, is Gaussian elimination, the natural generalization of solving our example with three unknowns in the beginning of this lecture: Solve one of the n equations (eg the first one) for one of the unknowns (eg x 1 ) Replace x 1 by the resulting term (depending on x 2,, x n) in the other n 1 equations therefore, x 1 is eliminated from those 0 0 Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

17 Solve the resulting system of n 1 equations with n 1 unknowns analogously and continue until an equation only contains x n, which can therefore be explicitly calculated Now, x n is inserted into the elimination equation of x n 1, so x n 1 can be given explicitly Continue until at last the elimination equation of x 1 provides the value for x 1 by inserting the values for x 2,, x n (known by now) Summarizing, we state that we have 1) transformed the system Ax = b into Rx = b with un upper triangular matrix R and a modified right-hand side, 2) computed the solution of the latter by starting with the last row of the system and sequentially computing x n, x n 1,, x 1 Note that both systems (Ax = b and Rx = b have the same solution Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

18 Gaussian Elimination The Elimination Algorithm loop over columns k r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 a k,k a k,k+1 a k,n 0 0 a k+1,k a k+1,k+1 a k+1,n 0 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

19 Gaussian Elimination The Elimination Algorithm for j from k to n r[k,j] := a[k,j] r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n 0 0 a k+1,k a k+1,k+1 a k+1,n 0 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

20 Gaussian Elimination The Elimination Algorithm r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n 0 0 a k+1,k a k+1,k+1 a k+1,n 0 a n,k a n,n b 1 b 2 b k b k+1 b n (a k+1,k /r k,k ) Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

21 Gaussian Elimination The Elimination Algorithm r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n 0 0 a k+1,k a k+1,k+1 a k+1,n 0 a n,k a n,n b 1 b 2 b k b k+1 b n l[k + 1, k] Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

22 Gaussian Elimination The Elimination Algorithm for j from k+1 to n r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n a k+1,k+1 a k+1,n 0 a n,k a n,n b 1 b 2 b k b k+1 b n l[j, k] Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

23 Gaussian Elimination The Elimination Algorithm r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n a k+1,k+1 a k+1,n 0 0 a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

24 Gaussian Elimination The Elimination Algorithm for k from 1 to n r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n r k+1,k+1 r k+1,n r n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

25 Gaussian Elimination The Solution Step r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n r k+1,k+1 r k+1,n r n,n b 1 b 2 b k b k+1 b n x n = b n/r n,n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

26 Gaussian Elimination The Solution Step r 1,1 r 1,2 r 1,k r 1,k+1 r 1,n 0 r 2,2 r 2,k r 2,k+1 r 2,n 0 0 r k,k r k,k+1 r k,n r k+1,k+1 r k+1,n r n,n for k from n to 1 ( x k = 1 b r n,n k ) n j=k+1 x j r k,j b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

27 The Algorithm in Pseudocode for k from 1 to n do // loop over columns for j from k to n do r[k,j]:=a[k,j] od; // set values of R in line k for i from k+1 to n do // loop over lines i below line k l[i,k]:=a[i,k]/r[k,k]; // determine factor to elim a[i,k] for j from k+1 to n do a[i,j]:=a[i,j]-l[i,k]*r[k,j]; // determine new entries in line i od; b[i]:=b[i]-l[i,k]*b[k]; // update right hand sides od; od; for i from n downto 1 do // backward solver loop x[i]:=b[i]; for j from i+1 to n do x[i]:=x[i]-r[i,j]*x[j]; od; x[i]:=x[i]/r[i,i]; od; Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

28 , initial system , first column eliminated 8 18, 51 43, 65 second column eliminated 8 18, , Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

29 third column eliminated 8, , R := last column eliminated 8, , factors L and R = we have L R = A =: L Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

30 Discussion of Gaussian Elimination outer k-loop: eliminate subdiagonal part of column k; possibly store the (modified) right-hand side b k in y k ; first inner j-loop: store upper triangular part of line k in the auxiliary matrix R; inner i-loop: determine the required factors in column k for the cancellation of the entries a i,k below the diagonal and store them in the auxiliary matrix L (here, we silently assume r j,j 0 for the first instance); subtract the corresponding multiple of the row k from row i; modify the right side accordingly (such that the solution x won t be changed); finally: substitute the calculated components of x backwards (ie starting with x n) in the equations stored in R and b (or y) (r i,i 0 is silently assumed again); Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

31 Counting the arithmetic operations accurately gives a total effort of basic arithmetic operations 2 3 n3 + O(n 2 ) Together with the matrix A, the matrices L and R appear in the algorithm of Gaussian elimination We have (cf algorithm): In R, only the upper triangular part (inclusive the diagonal) is populated In L, only the strict lower trianguler part is populated (without the diagonal) If filling the diagonal in L with ones, we get the fundamental relation Note that we have executed A = L R a i,j := a i,j l i,k r k,j for all k < i, j (compare pseudocode) bevor setting r i,j := a i,j This is equivalent to a i,j = r i,i + l i,k r k,j A = L R, k<min(i,j) where a i,j denotes the original unmodified value Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

32 Such a decomposition or factorization of a given matrix A in factors with certain properties (here: triangular form) is a very basic technique in numerical linear algebra The insight above means for us: Instead of using the classical Gauss elimination, we can solve Ax = b with the triangular decomposition A = LR, namely with the algorithm resulting from Ax = LRx = L(Rx) = Ly = b, ie, we compute L and R as in the algorithm for Gaussian elimination, but leave the right-hand side b unchanged In the subsequent solution step, we first solve Ly = b followed by solving Rx = y Usually, LU factorization reorders the operations of Gaussian elimination by switching from a columnswise traversal of the matrix to a row-wise matrix traversal that computes all changes of matrix entries (ie, of L and R) together entry by entry However, this is still mathematically equivalent such that both execution order are possible In the following, we shortly sketch the usual algorithms with linewise matrix traversal Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

33 algorithm of the LR decomposition: 1 step: triangular decomposition: decompose A into factors L (lower triangular matrix with ones in the diagonal) and R (upper triangular matrix); the decomposition is with this specification of the respective diagonal values unique! 2 step: forward substitution: solve Ly = b (by inserting, from y 1 to y n) 3 step: backward substitution: solve Rx = y (by inserting, from x n to x 1 ) In English, this is usually denoted as LU factorization with a lower trinagular matrix L and an upper triangular matrix U Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

34 LU Factorization The Algorithm outer loop over lines k u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n a k,1 a k,k 1 a k,k a k,k+1 a k,n a k+1,k 1 a k+1,k a k+1,k+1 a k+1,n a n,1 a n,k 1 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

35 LU Factorization The Algorithm for i from 1 to k-1 l[k,i] := a[k,i] outer loop over lines k u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,1 l k,k 1 a k,k a k,k+1 a k,n a k+1,k 1 a k+1,k a k+1,k+1 a k+1,n a n,1 a n,k 1 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

36 LU Factorization The Algorithm u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,k 1 u k 1,k 1 l k,1 l k,k 1 a k,k a k,k+1 a k,n a k+1,k 1 a k+1,k a k+1,k+1 a k+1,n a n,1 a n,k 1 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

37 LU Factorization The Algorithm u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,1 l k,k 1 /a k,k a k,k a k,k+1 a k,n a k+1,k 1 a k+1,k a k+1,k+1 a k+1,n a n,1 a n,k 1 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

38 LU Factorization The Algorithm for i from k to n u[k,i] := a[k,i] u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,1 l k,k 1 uk,k u k,k+1 u k,n a k+1,k 1 a k+1,k a k+1,k+1 a k+1,n a n,1 a n,k 1 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

39 LU Factorization The Algorithm for i from k to n u[k,i] := a[k,i] u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,k 1 u k 1,k+1 l k,1 l k,k 1 u k,k u k,k+1 u k,n a k+1,k 1 a k+1,k a k+1,k+1 a k+1,n a n,1 a n,k 1 a n,k a n,n b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

40 LU Factorization The Solution Step u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,1 l k,k 1 u k,k u k,k+1 u k,n a k+1,k 1 l k+1,k u k+1,k+1 u k+1,n a n,1 a n,k 1 l n,k l n,n 1 u n,n b 1 b 2 b k b k+1 b n for k from 1 to n y k = b k n j=k+1 b j l k,j Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

41 LU Factorization The Solution Step for k from 1 to n u 1,1 u 1,2 u 1,k u 1,k+1 u 1,n l 2,1 u 2,2 u 2,k u 2,k+1 u 2,n l k,1 l k,k 1 uk,k u k,k+1 u k,n a k+1,k 1 l k+1,k u k+1,k+1 u k+1,n a n,1 a n,k 1 l n,k l n,n 1 u n,n ( for k from n to 1 x k = 1 y u k ) n k,k j=k+1 x j u k,j b 1 b 2 b k b k+1 b n Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

42 LU Factorization The Pseudocode for k from 1 to n do for i from 1 to k-1 do l[k,i]:=a[k,i]; for j from 1 to k-1 do l[k,i]:=l[k,i]-l[k,j]*u[j,i] od; l[k,i]:=l[k,i]/a[k,k] od; for i from k to n do u[k,i]:=a[k,i]; for j from 1 to k-1 do u[k,i]:=u[k,i]-l[k,j]*u[j,i] od od od; for k from 1 to n do y[k]:=b[k]; for j from 1 to k-1 do y[k]:=y[k]-l[k,j]*y[j] od od; for k from n downto 1 do x[k]:=y[k]; for j from k+1 to n do x[k]:=x[k]-u[k,j]*x[j] od; x[k]:=x[k]/u[k,k]; od; Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

43 Discussion of LU factorization To comprehend the first part (the decomposition into the factors L and R), start with the general formula for the matrix product: a k,i = n l k,j u j,i j=1 However, here quite unusually A is known, and L and R have to be determined (the fact that this works is due to the triangular shape of the factors)! In case of k > i, one only has to sum up to j = k, and l k,i can be determined (solve the equation for l k,i : everything on the right-hand side is known already): i 1 i 1 a k,i = l k,j u j,i +l k,i u i,i and, therefore, l k,i := a k,i l k,j u j,i /u i,i j=1 j=1 Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,

44 If k i, one only has to sum up to j = k, and u k,i can be determined (solve the equation for u k,i : again, all quantities on the right are already given; note l k,k = 1): a k,i = k 1 k 1 l k,j u j,i + l k,k u k,i and, therefore, u k,i := a k,i l k,j u j,i j=1 j=1 It is clear: If you fight your way through all variables with increasing row and column indices (the way it s happening in the algorithm, starting at i = k = 1), every l k,i and u k,i can be calculated one after the other on the right-hand side, there is always something that has already been calculated! The forward substitution (second i-loop) follows and as before at Gaussian elimination the backward substitution (third and last k loop) It can be shown that the two methods Gauss elimination and LU factorization are identical in terms of carrying out the same operations (ie they particularly have the same cost); only the order of the operations is different! Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13,