Introduction to Scientific Computing
|
|
- Lindsey Jacobs
- 5 years ago
- Views:
Transcription
1 (Lecture 5: Linear system of equations / Matrix Splitting) Bojana Rosić, Thilo Moshagen Institute of Scientific Computing
2 Motivation Let us resolve the problem scheme by using Kirchhoff s laws: the algebraic sum of all the currents flowing toward a node is equal to zero. I = I + I2 I3 = I R + I3 R3 = V In any closed circuit, the algebraic sum of all the voltages around the loop is equal to zero V = IR I2 R2 I3 R3 = V2 2 / 5
3 Motivation Hence, we end up with the system of equations I + I2 I3 = I R + I3 R3 = V I2 R2 I3 R3 = V2 which is easy to solve algebraically by elimination I = I2 + I3 and substitution (I2 + I3 )R + I3 r3 = V I2 = (V + I3 (R3 R ))R and so on... 3 / 5
4 What to do in this case? Algebraically is difficult to solve. 4 / 5
5 But, engineering goal is to solve large-scale (realistic) systems This usually matches with solving x = F (x) in which x is the system state. 5 / 5
6 Special case Special case are systems in a linear form Ax = b in which matrix A can be of dimensions larger than. Think about A R A R Examples are: solving linear partial differential equations (elasticity, heat equation etc.) In these cases one cannot compute x = A b as one would have problem with memory or computation time. 6 / 5
7 Linear systems There are many methods one can use to solve the system Ax = b such as: Stationary iterative methods (approximate operator) Gauss-Seidel (GS) Jacobi (JM) Successive over-relaxation (SOR) Krylov subspace methods Conjugate Gradient (CG) Biconjugate gradient method (BiCG) generalized minimal residual method (GMRES) combination 7 / 5
8 Solving system Remembering fixed point iteration, we may try something similar. Let us guess the solution by putting x = x (k) Since we do not know if our guess is correct, we may check its accuracy by evaluating the residual d (k) = b Ax(k) Now we may have educated guess by drawing conclusions from the error value. 8 / 5
9 What would be our next guess? Thus, our next guess x(k+) will be smaller or larger than x(k) depending on the residual d(k). Hence, x(k+) = x(k) + v(k) in which v(k) denotes correction. Our goal is now to find the best correction! Naturally, the best correction v(k) would be the correction which satisfies the equation x(k) + v(k) = x. so the error. The only problem is that we do not know x!! 9 / 5
10 But, we do know that the best correction v(k) has to satisfy Ax = A(x(k) + v(k) ) = b This in turn gives us equation for the correction Av(k) = b Ax(k) = d(k) which further yields v(k) = A (b Ax(k) ) = A d(k) Thsi is explicit solving again. / 5
11 Solution: pose the problem differently Do not invert matrix A, but find some simpler matrix C A which is easy to invert! Then the problem v(k) = A d(k) reduces to v(k) = C d(k) i.e. C v(k) = d(k) which takes only small computational time. / 5
12 Linear iteration scheme Then we can formulate the general linear iteration scheme x(k+) = x(k) + C b Ax(k). {z } v(k) in which C can be chosen in different ways. With respect to the type of approximation one may distinguish: Jacobi method Gauss-Seidel method The successive over relaxation method, etc. These methods choose C from LDU decomposition of a matrix A. 2 / 5
13 LDU decomposition A = L + D + U, where L is a lower triangular matrix with main diagonal equals, D is a diagonal matrix und U is an upper triangular matrix with main diagonal equals zero. Decomposition of A: nz = Lij =, j i nz = diag(aii ) nz = Uij =, j i 3 / 5
14 Jacobi method Let us take that C is approximated only by diagonal C := D = diag(aii ). Then it follows x(k+) = x(k) + D b Ax(k). {z } v(k) 5 With D = diag aii and 6 7 b Ax(k) i = bi n j= (k) aij xj nz = / 5
15 Jacobi method The iterative procedure in element-wise form reads: n (k+) (k) xi = aij xj, bi aii j= i =,..., n. i6=j Hence, the values aii must be different than zero. 5 / 5
16 Exercise Let us use Jacobi method to solve 2x + y = 5x + 7y = 3 by starting at x() = (; )T. Hence, 2 A=, b= / 5
17 Exercise The first iteration reads x() = x() + D b Ax() where D is the diagonal part of the matrix A: 2 D= 7 with the inverse D = /2. /7 7 / 5
18 Exercise Hence, x () /2 = + ( /7 3 2 ) x() =.429 After 22 iterations one obtains: 7. (22) x = / 5
19 Exercise Let us check solution 2 Ax(22) = = / 5
20 Exercise Let us use Jacobi method to solve 3x + 2y z = 2x 2y + 4z = 2 x + 2 y z = Hence, 3 A= , b = 2 2 / 5
21 Exercise The diagonal part of the matrix is 3 D = 2 /3 /2 D = / and the Jacobi method reads: x(k+) = x(k) + D b Ax(k). {z } v(k) 2 / 5
22 Exercise Start with x() = = (,, )T such that x() x() = x() + D b Ax() =+D.5 holds. 22 / 5
23 Exercise Hence, x() /3 = /2 /3 2 = / Similarly, one may write x(2) / = + D (2) x = / / 5
24 Exercise Iter 2 3 x e6 -.29e59 x e6.3e59 x e6.87e59 However, by iterating one may notice that the solution does not converge. This happens even by changing the initial condition. The question is why this happens? 24 / 5
25 Convergence From x(k+) = x(k) + D b Ax(k) = F (x(k) ) we may conclude that this is one kind of fixed point iteration scheme with the Lipschitz constant F (x) = D A Since A=D +L+U with R = L + U is the rest of the matrix F (x) = D (D + R) = D R 25 / 5
26 Convergence To check if the method is convergent, one has to check contractivity q = sup kf k = sup kd Rk2 <. The last relation is equivalent to the condition for the spectral radius def ρ = max( λi ) < i where λi are the eigenvalues of D R. 26 / 5
27 Exercise In the first example one has D R = Hence, ρ =.5976 < 27 / 5
28 Exercise In the second example one has.6667 D R = Hence, ρ =.4 > 28 / 5
29 Can we make this conclusion apriori? If matrix A is strictly or irreducibly diagonally dominant, Jacobi method will converge. Strict row diagonal dominance means that for each row, the absolute value of the diagonal term is greater than the sum of absolute values of other terms: aii > aij. j6=i This stems from (see Golub/van Loan Chap..) ρ(d (D + R)) D (D + R) = max i aij < aii j6=i 29 / 5
30 Gauss-Seidel method Going back to the general scheme x(k+) = x(k) + C b Ax(k). we may take the approximation for C to be equal C := L + D Then it follows 6 x(k+) = x(k) + (L + D) b Ax(k) nz = / 5
31 Gauss-Seidel method However, we said before that inversion is avoided in numerical schemes. Due to this reason, let us multiply both sides of equation by (L + D): (L + D)x(k+) = (L + D)x(k) + b Ax(k). Having that A = L + D + U we obtain (L + D)x(k+) = (L + D)x(k) + b (L + D + U)x(k) = b Ux(k) Splitting the left side of equation, one obtains Dx(k+) = b Lx(k+) Ux(k). 3 / 5
32 Gauss-Seidel method In component form one obtains (k+) aii xi = bi i (k+) aij xj j= and (k+) xi n (k) aij xj, i n (k+) (k) = bi aij xj aij xj aii j= i =,..., n. j=i+ i =,..., n. j=i+ 32 / 5
33 Compare Jacobi and GS method The Jacobi method in element-wise form reads: n (k+) (k) = xi aij xj, bi aii j= i =,..., n. i6=j and Gauss-Seidel: (k+) xi = bi aii i j= (k+) aij xj n (k) aij xj i =,..., n. j=i+ More motivating view on GS: It is Jacobi method while using the latest infos. 33 / 5
34 Exercise Let us use Gauss-Seidel method to solve 2x + y = 5x + 7y = 3 by starting at x() = (; )T. Hence, 2 A=, b= / 5
35 Exercise The first iteration reads n i () () () aij xj xi = bi aij xj aii i =,..., n. j=i+ j= 2 () x = () [b aj xj ] = 5 a j=2 () x2 = () [b2 a2j xj ] =.743 a22 j= It takes approximately the same number of iterations as Jacobi to get the solution. 35 / 5
36 Exercise Let us use Gauss-Seidel method to solve 3x + 2y z = 2x 2y + 4z = 2 x + 2 y z = Hence, 3 A= , b = 2 36 / 5
37 Exercise The first iteration reads n i () () () bi aij xj aij xj xi = aii j= j=i+ 3 () x = () [b aj xj ] =.333 a j=2 () x2 = 3 j= j=3 () () [b2 a2j xj a2j xj ] =.333 a22 2 () x3 () = [b3 a3j xj ] =.333 a33 j= 37 / 5
38 Exercise Similarly to the Jacobi method, the Gauss-Seidel does not converge for this system of equations. 38 / 5
39 Convergence From x(k+) = x(k) + (L + D) b Ax(k). we may conclude that this is one kind of fixed point iteration scheme with the Lipschitz constant F (x) = (L + D) A Since A=L+D +U one has F (x) = (L + D) (L + D + U) = (L + D) U 39 / 5
40 Convergence Thus, the method is convergent if the spectral radius of ρ((l + D) U) < The convergence properties of the Gauss Seidel method are dependent on the matrix A. Namely, the procedure is known to converge if either: A is symmetric positive-definite (symmetric A = AT, positive definite matrix is a symmetric matrix A for which all eigenvalues are positive). A is strictly or irreducibly diagonally dominant. 4 / 5
41 Convergence In the first example one has that ρ((l + D) U) =.357 < and in the second example ρ((l + D) U) =.24 > 4 / 5
42 Successive over relaxation method (SOR) The SOR-scheme tries to improve the convergence properties of the Gauß-Seidel method. The idea is to introduce a relaxation parameter ω > such that C := D +L ω Inserting yields (k+) x Multiplying by (k) =x + D +L ω b Ax(k). D ω + L, and using A = L + D + U one obtains D + L x(k+) = Dx(k) + b Ux(k) ω ω 42 / 5
43 SOR method Then and ω Dx(k+) = Dx(k) Lx(k+) + b Ux(k) ω ω Dx(k+) = ( ω)dx(k) + ω b Lx(k+) Ux(k). The representation with components is now given by (k+) aii xi = ( (k) ω)aii xi +ω bi i (k+) aij xj j= n! (k) aij xj and (k+) xi = ( (k) ω)xi ω + aii bi i (k+) aij xj j=, j=i+ n! (k) aij xj. j=i+ 43 / 5
44 SOR method Note that when ω = the method Dx(k+) = ( ω)dx(k) + ω b Lx(k+) Ux(k). becomes Dx(k+) = b Lx(k+) Ux(k). which is Gauß-Seidel method. 44 / 5
45 SOR: how to choose ω? With an optimal choice of ω we can improve the convergence of the SOR-method in comparison with the Gauß-Seidel method. But the optimal value for ω depends on the problem, i.e. in general one has to try several values. One should start with larger values, i.e. ω =.7, then ω =.3... and take a look on the convergence behaviour. 45 / 5
46 Convergence From x(k+) = x(k) + D +L ω b Ax(k) we may conclude that this is one kind of fixed point iteration scheme with the Lipschitz constant F (x) = D +L A ω and hence the spectral radius must satisfy D +L A) < ρ( ω to get convergence. 46 / 5
47 Convergence For ω =.7 the spectral radius in the previous two examples becomes ρ( D +L A) =.7 < ω and in the second example ρ( D +L ω A) = 3.8 > 47 / 5
48 General convergence results The equation x(k+) = x(k) + C b Ax(k). for the general iteration method can be transformed into x(k+) = x(k) + C b Ax(k) = C b + (I C A)x(k) i.e. x(k+) = Mx(k) + g in which M = I C A and g = C b. This equation is often called normal form. Every linear iteration method can be written in normal form. The matrix M is called iteration matrix of the method. 48 / 5
49 General convergence results For the derivation of convergence criteria we need the spectral radius of a matrix which can be defined in the following way: Definition For a given matrix M Rn,n let λi, i =,..., n be the eigenvalues of M. Then ρ(m) := max λi i=,...,n is called the spectral radius of M. 49 / 5
50 General convergence results Theorem The linear iteration method with the iteration matrix M converges for an arbitrary start vector x() if and only if ρ(m) < and if in a dedicated matrix norm k k, the condition kmk is satisfied. In this case the following error estimates are true: x(k) x x(k) x kmkk x() x(), kmk kmk x(k) x(k ). kmk () (2) Inequality () is called a priori estimate and (2) is called a posteriori estimate. 5 / 5
51 Literature C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations Yousef Saad, Iterative methods for sparse linear systems Martin Stoll, Solving Linear Systems using the Adjoint 5 / 5
Lecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More information9.1 Preconditioned Krylov Subspace Methods
Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More informationLab 1: Iterative Methods for Solving Linear Systems
Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as
More informationFEM and Sparse Linear System Solving
FEM & sparse system solving, Lecture 7, Nov 3, 2017 1/46 Lecture 7, Nov 3, 2015: Introduction to Iterative Solvers: Stationary Methods http://people.inf.ethz.ch/arbenz/fem16 Peter Arbenz Computer Science
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationCAAM 454/554: Stationary Iterative Methods
CAAM 454/554: Stationary Iterative Methods Yin Zhang (draft) CAAM, Rice University, Houston, TX 77005 2007, Revised 2010 Abstract Stationary iterative methods for solving systems of linear equations are
More informationMath Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationIterative Methods. Splitting Methods
Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition
More informationJACOBI S ITERATION METHOD
ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD
More informationIterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)
Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential
More information6. Iterative Methods for Linear Systems. The stepwise approach to the solution...
6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse
More information7.3 The Jacobi and Gauss-Siedel Iterative Techniques. Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP.
7.3 The Jacobi and Gauss-Siedel Iterative Techniques Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP. 7.3 The Jacobi and Gauss-Siedel Iterative Techniques
More informationClassical iterative methods for linear systems
Classical iterative methods for linear systems Ed Bueler MATH 615 Numerical Analysis of Differential Equations 27 February 1 March, 2017 Ed Bueler (MATH 615 NADEs) Classical iterative methods for linear
More information9. Iterative Methods for Large Linear Systems
EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationMath/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018
Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x
More informationLecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University
Lecture Note 7: Iterative methods for solving linear systems Xiaoqun Zhang Shanghai Jiao Tong University Last updated: December 24, 2014 1.1 Review on linear algebra Norms of vectors and matrices vector
More informationIterative methods for Linear System
Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Iteration basics Notes for 2016-11-07 An iterative solver for Ax = b is produces a sequence of approximations x (k) x. We always stop after finitely many steps, based on some convergence criterion, e.g.
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 207 Philippe B. Laval (KSU) Linear Systems Fall 207 / 2 Introduction We continue looking how to solve linear systems of the
More informationCLASSICAL ITERATIVE METHODS
CLASSICAL ITERATIVE METHODS LONG CHEN In this notes we discuss classic iterative methods on solving the linear operator equation (1) Au = f, posed on a finite dimensional Hilbert space V = R N equipped
More informationNumerical Programming I (for CSE)
Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let
More informationReview of matrices. Let m, n IN. A rectangle of numbers written like A =
Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an
More informationContents. Preface... xi. Introduction...
Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationMath 471 (Numerical methods) Chapter 3 (second half). System of equations
Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 21: Sensitivity of Eigenvalues and Eigenvectors; Conjugate Gradient Method Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis
More informationThe amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A.
AMSC/CMSC 661 Scientific Computing II Spring 2005 Solution of Sparse Linear Systems Part 2: Iterative methods Dianne P. O Leary c 2005 Solving Sparse Linear Systems: Iterative methods The plan: Iterative
More informationJordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS
Jordan Journal of Mathematics and Statistics JJMS) 53), 2012, pp.169-184 A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS ADEL H. AL-RABTAH Abstract. The Jacobi and Gauss-Seidel iterative
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Linear Systems Fall 2015 1 / 12 Introduction We continue looking how to solve linear systems of
More informationNext topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
More informationIntroduction to Iterative Solvers of Linear Systems
Introduction to Iterative Solvers of Linear Systems SFB Training Event January 2012 Prof. Dr. Andreas Frommer Typeset by Lukas Krämer, Simon-Wolfgang Mages and Rudolf Rödl 1 Classes of Matrices and their
More informationAlgebraic Multigrid as Solvers and as Preconditioner
Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven
More informationDEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
More informationLINEAR SYSTEMS (11) Intensive Computation
LINEAR SYSTEMS () Intensive Computation 27-8 prof. Annalisa Massini Viviana Arrigoni EXACT METHODS:. GAUSSIAN ELIMINATION. 2. CHOLESKY DECOMPOSITION. ITERATIVE METHODS:. JACOBI. 2. GAUSS-SEIDEL 2 CHOLESKY
More informationComputational Economics and Finance
Computational Economics and Finance Part II: Linear Equations Spring 2016 Outline Back Substitution, LU and other decomposi- Direct methods: tions Error analysis and condition numbers Iterative methods:
More informationIterative Methods for Ax=b
1 FUNDAMENTALS 1 Iterative Methods for Ax=b 1 Fundamentals consider the solution of the set of simultaneous equations Ax = b where A is a square matrix, n n and b is a right hand vector. We write the iterative
More informationEXAMPLES OF CLASSICAL ITERATIVE METHODS
EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic
More informationChapter 7. Iterative methods for large sparse linear systems. 7.1 Sparse matrix algebra. Large sparse matrices
Chapter 7 Iterative methods for large sparse linear systems In this chapter we revisit the problem of solving linear systems of equations, but now in the context of large sparse systems. The price to pay
More informationLecture 11. Fast Linear Solvers: Iterative Methods. J. Chaudhry. Department of Mathematics and Statistics University of New Mexico
Lecture 11 Fast Linear Solvers: Iterative Methods J. Chaudhry Department of Mathematics and Statistics University of New Mexico J. Chaudhry (UNM) Math/CS 375 1 / 23 Summary: Complexity of Linear Solves
More informationIterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University
More informationComparison of Fixed Point Methods and Krylov Subspace Methods Solving Convection-Diffusion Equations
American Journal of Computational Mathematics, 5, 5, 3-6 Published Online June 5 in SciRes. http://www.scirp.org/journal/ajcm http://dx.doi.org/.436/ajcm.5.5 Comparison of Fixed Point Methods and Krylov
More informationUp to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas
Finding Eigenvalues Up to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas plus the facts that det{a} = λ λ λ n, Tr{A}
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationProcess Model Formulation and Solution, 3E4
Process Model Formulation and Solution, 3E4 Section B: Linear Algebraic Equations Instructor: Kevin Dunn dunnkg@mcmasterca Department of Chemical Engineering Course notes: Dr Benoît Chachuat 06 October
More informationLecture 11: CMSC 878R/AMSC698R. Iterative Methods An introduction. Outline. Inverse, LU decomposition, Cholesky, SVD, etc.
Lecture 11: CMSC 878R/AMSC698R Iterative Methods An introduction Outline Direct Solution of Linear Systems Inverse, LU decomposition, Cholesky, SVD, etc. Iterative methods for linear systems Why? Matrix
More information4.6 Iterative Solvers for Linear Systems
4.6 Iterative Solvers for Linear Systems Why use iterative methods? Virtually all direct methods for solving Ax = b require O(n 3 ) floating point operations. In practical applications the matrix A often
More informationDepartment of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015
Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in NUMERICAL ANALYSIS Spring 2015 Instructions: Do exactly two problems from Part A AND two
More informationParallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1
Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations
More information30.5. Iterative Methods for Systems of Equations. Introduction. Prerequisites. Learning Outcomes
Iterative Methods for Systems of Equations 0.5 Introduction There are occasions when direct methods (like Gaussian elimination or the use of an LU decomposition) are not the best way to solve a system
More informationToday s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
More informationHere is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J
Class Notes 4: THE SPECTRAL RADIUS, NORM CONVERGENCE AND SOR. Math 639d Due Date: Feb. 7 (updated: February 5, 2018) In the first part of this week s reading, we will prove Theorem 2 of the previous class.
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationITERATIVE METHODS FOR SPARSE LINEAR SYSTEMS
ITERATIVE METHODS FOR SPARSE LINEAR SYSTEMS YOUSEF SAAD University of Minnesota PWS PUBLISHING COMPANY I(T)P An International Thomson Publishing Company BOSTON ALBANY BONN CINCINNATI DETROIT LONDON MADRID
More informationChapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
More informationLecture 10b: iterative and direct linear solvers
Lecture 10b: iterative and direct linear solvers MATH0504: Mathématiques appliquées X. Adriaens, J. Dular Université de Liège November 2018 Table of contents 1 Direct methods 2 Iterative methods Stationary
More informationGeneralized AOR Method for Solving System of Linear Equations. Davod Khojasteh Salkuyeh. Department of Mathematics, University of Mohaghegh Ardabili,
Australian Journal of Basic and Applied Sciences, 5(3): 35-358, 20 ISSN 99-878 Generalized AOR Method for Solving Syste of Linear Equations Davod Khojasteh Salkuyeh Departent of Matheatics, University
More informationNumerical Analysis: Solutions of System of. Linear Equation. Natasha S. Sharma, PhD
Mathematical Question we are interested in answering numerically How to solve the following linear system for x Ax = b? where A is an n n invertible matrix and b is vector of length n. Notation: x denote
More informationTheory of Iterative Methods
Based on Strang s Introduction to Applied Mathematics Theory of Iterative Methods The Iterative Idea To solve Ax = b, write Mx (k+1) = (M A)x (k) + b, k = 0, 1,,... Then the error e (k) x (k) x satisfies
More informationMotivation: Sparse matrices and numerical PDE's
Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting
More informationSparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations
Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system
More informationMA3232 Numerical Analysis Week 9. James Cooley (1926-)
MA umerical Analysis Week 9 James Cooley (96-) James Cooley is an American mathematician. His most significant contribution to the world of mathematics and digital signal processing is the Fast Fourier
More informationStabilization and Acceleration of Algebraic Multigrid Method
Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration
More informationScientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1
Scientific Computing WS 2018/2019 Lecture 9 Jürgen Fuhrmann juergen.fuhrmann@wias-berlin.de Lecture 9 Slide 1 Lecture 9 Slide 2 Simple iteration with preconditioning Idea: Aû = b iterative scheme û = û
More informationFEM and sparse linear system solving
FEM & sparse linear system solving, Lecture 9, Nov 19, 2017 1/36 Lecture 9, Nov 17, 2017: Krylov space methods http://people.inf.ethz.ch/arbenz/fem17 Peter Arbenz Computer Science Department, ETH Zürich
More informationLinear Solvers. Andrew Hazel
Linear Solvers Andrew Hazel Introduction Thus far we have talked about the formulation and discretisation of physical problems...... and stopped when we got to a discrete linear system of equations. Introduction
More informationNumerical Methods Process Systems Engineering ITERATIVE METHODS. Numerical methods in chemical engineering Edwin Zondervan
IERAIVE MEHODS Numerical methods in chemical engineering Edwin Zondervan 1 OVERVIEW Iterative methods for large systems of equations We will solve Laplace s equation for steady state heat conduction LAPLACE
More informationLecture 17: Iterative Methods and Sparse Linear Algebra
Lecture 17: Iterative Methods and Sparse Linear Algebra David Bindel 25 Mar 2014 Logistics HW 3 extended to Wednesday after break HW 4 should come out Monday after break Still need project description
More informationFrom Stationary Methods to Krylov Subspaces
Week 6: Wednesday, Mar 7 From Stationary Methods to Krylov Subspaces Last time, we discussed stationary methods for the iterative solution of linear systems of equations, which can generally be written
More informationMidterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015
Midterm for Introduction to Numerical Analysis I, AMSC/CMSC 466, on 10/29/2015 The test lasts 1 hour and 15 minutes. No documents are allowed. The use of a calculator, cell phone or other equivalent electronic
More informationTMA4125 Matematikk 4N Spring 2017
Norwegian University of Science and Technology Institutt for matematiske fag TMA15 Matematikk N Spring 17 Solutions to exercise set 1 1 We begin by writing the system as the augmented matrix.139.38.3 6.
More informationDonnishJournals
DonnishJournals 204-89 Donnish Journal of Educational Research and Reviews. Vol 2() pp. 008-07 January, 205. http:///djerr Copyright 205 Donnish Journals Original Research Paper Solution of Linear Systems
More informationPreface to the Second Edition. Preface to the First Edition
n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................
More informationNumerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
More informationSOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA
1 SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 2 OUTLINE Sparse matrix storage format Basic factorization
More informationComputational Methods. Systems of Linear Equations
Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations
More informationOUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU
Preconditioning Techniques for Solving Large Sparse Linear Systems Arnold Reusken Institut für Geometrie und Praktische Mathematik RWTH-Aachen OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative
More informationNumerical Methods I Non-Square and Sparse Linear Systems
Numerical Methods I Non-Square and Sparse Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 25th, 2014 A. Donev (Courant
More informationBackground. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms
More informationM.A. Botchev. September 5, 2014
Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev
More information6.4 Krylov Subspaces and Conjugate Gradients
6.4 Krylov Subspaces and Conjugate Gradients Our original equation is Ax = b. The preconditioned equation is P Ax = P b. When we write P, we never intend that an inverse will be explicitly computed. P
More informationLecture 4: Linear Algebra 1
Lecture 4: Linear Algebra 1 Sourendu Gupta TIFR Graduate School Computational Physics 1 February 12, 2010 c : Sourendu Gupta (TIFR) Lecture 4: Linear Algebra 1 CP 1 1 / 26 Outline 1 Linear problems Motivation
More informationModelling and implementation of algorithms in applied mathematics using MPI
Modelling and implementation of algorithms in applied mathematics using MPI Lecture 3: Linear Systems: Simple Iterative Methods and their parallelization, Programming MPI G. Rapin Brazil March 2011 Outline
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be
More informationMath 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018
1 Linear Systems Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March, 018 Consider the system 4x y + z = 7 4x 8y + z = 1 x + y + 5z = 15. We then obtain x = 1 4 (7 + y z)
More informationIterative techniques in matrix algebra
Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and
More informationConjugate Gradient (CG) Method
Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous
More informationON A HOMOTOPY BASED METHOD FOR SOLVING SYSTEMS OF LINEAR EQUATIONS
TWMS J. Pure Appl. Math., V.6, N.1, 2015, pp.15-26 ON A HOMOTOPY BASED METHOD FOR SOLVING SYSTEMS OF LINEAR EQUATIONS J. SAEIDIAN 1, E. BABOLIAN 1, A. AZIZI 2 Abstract. A new iterative method is proposed
More informationCourse Notes: Week 1
Course Notes: Week 1 Math 270C: Applied Numerical Linear Algebra 1 Lecture 1: Introduction (3/28/11) We will focus on iterative methods for solving linear systems of equations (and some discussion of eigenvalues
More informationPartitioned Methods for Multifield Problems
Partitioned Methods for Multifield Problems Joachim Rang, 10.5.2017 10.5.2017 Joachim Rang Partitioned Methods for Multifield Problems Seite 1 Contents Blockform of linear iteration schemes Examples 10.5.2017
More informationGeneralized line criterion for Gauss-Seidel method
Computational and Applied Mathematics Vol. 22, N. 1, pp. 91 97, 2003 Copyright 2003 SBMAC Generalized line criterion for Gauss-Seidel method M.V.P. GARCIA 1, C. HUMES JR. 2 and J.M. STERN 2 1 Department
More informationSolving PDEs with Multigrid Methods p.1
Solving PDEs with Multigrid Methods Scott MacLachlan maclachl@colorado.edu Department of Applied Mathematics, University of Colorado at Boulder Solving PDEs with Multigrid Methods p.1 Support and Collaboration
More informationParallel Iterative Methods for Sparse Linear Systems. H. Martin Bücker Lehrstuhl für Hochleistungsrechnen
Parallel Iterative Methods for Sparse Linear Systems Lehrstuhl für Hochleistungsrechnen www.sc.rwth-aachen.de RWTH Aachen Large and Sparse Small and Dense Outline Problem with Direct Methods Iterative
More informationA LINEAR SYSTEMS OF EQUATIONS. By : Dewi Rachmatin
A LINEAR SYSTEMS OF EQUATIONS By : Dewi Rachmatin Back Substitution We will now develop the backsubstitution algorithm, which is useful for solving a linear system of equations that has an upper-triangular
More informationNumerical Analysis: Solving Systems of Linear Equations
Numerical Analysis: Solving Systems of Linear Equations Mirko Navara http://cmpfelkcvutcz/ navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office
More informationScientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008
More information