Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58

Size: px
Start display at page:

Download "Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58"

Transcription

1 Background C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58

2 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms C. T. Kelley Background NCSU, Spring / 58

3 A Stationary Iterative Method converts Au = f to u = Ru + c and the iteration is u n+1 = Ru n + c M is called the iteration matrix. This iteration is also called Richardson Iteration. The method is called stationary because the formula does not change as a function of u n. C. T. Kelley Background NCSU, Spring / 58

4 Convergence and the Banach Lemma Banach Lemma Let R be N N. Assume that R < 1 for some induced matrix norm. Then (I R) is nonsingular (I R) 1 = l=0 Rl (I R) 1 (1 R ) 1 C. T. Kelley Background NCSU, Spring / 58

5 Convergence and the Banach Lemma Proof of Banach Lemma: I We will show that the series R l = (I R) 1. l=0 The partial sums S k = form a Cauchy sequence in R N N. To see this note that for all m > k m S k S m R l. And... k l=0 R l l=k+1 C. T. Kelley Background NCSU, Spring / 58

6 Convergence and the Banach Lemma Proof of Banach Lemma: II R l R l because is a matrix norm that is induced by a vector norm. Hence m ( ) 1 R S k S m R l = R k+1 m k 0 1 l=k+1 as m, k. So the series converges. Let S = l=0 R l C. T. Kelley Background NCSU, Spring / 58

7 Convergence and the Banach Lemma Proof of Banach Lemma: II Clearly Finally RS = R l+1 = R l = S I and so l=0 l=1 (I R)S = I and S = (I R) 1. (I R) 1 R l = (1 R ) 1. l=0 C. T. Kelley Background NCSU, Spring / 58

8 Convergence and the Banach Lemma Convergence for If R < 1 for any induced matrix norm then the stationary iteration u n+1 = Ru n + c converges for all c and u 0 to u = (I R) 1 c Proof: Clearly u n+1 = n R l c + R n u 0 (I R) 1 c = u. l=0 C. T. Kelley Background NCSU, Spring / 58

9 Convergence and the Banach Lemma Convergence Speed Let R = α < 1 and u = (I R) 1 c. Then Proof: u u n α n u u 0. u u n = l=n Rl c R n u 0 = R( l=n 1 Rl c R n 1 u 0 ) = R(u u n 1 ) α u u n 1 α n u u 0. C. T. Kelley Background NCSU, Spring / 58

10 Convergence and the Banach Lemma Spectral Radius The spectrum of R σ(r), is the set of eigenvalues of R. The spectral radius of R is ρ(r) = max λ σ(r) λ Theorem ρ(r) < 1 if and only if R < 1 for some induced matrix norm. A stationary iterative method will u n+1 = Ru n + c converges for all initial iterates and right sides if and only if ρ(r) < 1. The spectral radius does not depend on any norm. C. T. Kelley Background NCSU, Spring / 58

11 Convergence and the Banach Lemma Predicting Convergence Suppose you know that α < 1. Then e n+1 = u u n+1 = (Ru + c) (Ru n + c) = Re n Hence e n α n e 0 and e n τ e 0 if α n < τ or n > log(τ)/ log(α). C. T. Kelley Background NCSU, Spring / 58

12 Convergence and the Banach Lemma Preconditioned Richardson Iteration If I A < 1 then one can apply Richardson iteration directly to Au = f u n+1 = (I A)u n + f Sometimes one can find a approximate inverse B for which I BA < 1 and precondition with B to obtain BAu = Bf and the iteration is u n+1 = (I BA)u n + Bf C. T. Kelley Background NCSU, Spring / 58

13 Convergence and the Banach Lemma Approximate Inverse Preconditioning: I Theorem: If A and B are N N matrices and B is an approximate inverse of A, then A and B are both nonsingular and and A 1 A 1 B B 1 I BA, B 1 B I BA 1 I BA, A B 1 A 1 I BA, A I BA 1 I BA. C. T. Kelley Background NCSU, Spring / 58

14 Convergence and the Banach Lemma Approximate Inverse Preconditioning: II Proof: Let R = I BA. The Banach Lemma implies that I R = I (I BA) = BA is nonsingular. Hence both A and B are nonsingular. Moreover A 1 B 1 = (I R) R = 1 1 I BA. C. T. Kelley Background NCSU, Spring / 58

15 Convergence and the Banach Lemma Approximate Inverse Preconditioning: III Use A 1 = (I R) 1 B to get the first part A 1 B (I R) 1 The second pair of inequalities follows from and the first. B 1 I BA. A 1 B = (I BA)A 1, A B 1 = B 1 (I BA) C. T. Kelley Background NCSU, Spring / 58

16 Matrix Splittings and Classical Methods Matrix Splittings One way to convert Au = f to Ru = c is to split A where A 1 is nonsingular A = A 1 + A 2 A 1 y = q is easy to solve for all q and then solve u = A 1 1 (f A 2u) Ru + c. Here R = A 1 1 A 2 and c = A 1 1 f. Remember A 1 z means solve A 1 y = z, not compute A 1 1. C. T. Kelley Background NCSU, Spring / 58

17 Matrix Splittings and Classical Methods Jacobi Iteration: I Write Au = f explicitly a 11 ξ a 1N ξ N = β 1. a N1 ξ a NN ξ N = β N and solve the ith equation for ξ i, pretending the other components are know. You get ξ i = 1 β i a ij ξ j a ii j i which is a linear fixed point problem equivalent to Au = f. C. T. Kelley Background NCSU, Spring / 58

18 Matrix Splittings and Classical Methods Jacobi Iteration: II The iteration is ξ New i = 1 a ii β i j i a ij ξ Old j So what are R and c? Split A = A 1 + A 2, where A 1 = D, A 2 = L U, D is the diagonal of A, and L and U are the (strict) lower and upper triangular parts. then x New = D 1 (f + (L + U))x Old. C. T. Kelley Background NCSU, Spring / 58

19 Matrix Splittings and Classical Methods Jacobi Iteration: III So the iteration is u n+1 = D 1 (L + U)u n + D 1 f and the iteration matrix is R JAC = D 1 (L + U). Is there any reason for ρ(r JAC ) < 1? C. T. Kelley Background NCSU, Spring / 58

20 Matrix Splittings and Classical Methods Convergence for Strictly Diagonally Dominant A Theorem: Let A be an N N matrix and assume that A is strictly diagonally dominant. That is for all 1 i N 0 < j i a ij < a ii. Then A is nonsingular and the Jacobi iteration converges to u = A 1 f for all f. C. T. Kelley Background NCSU, Spring / 58

21 Matrix Splittings and Classical Methods Proof: Convergence for Strictly Diagonally Dominant A Our assumptions imply that a ii 0, so the iteration is defined. We can prove everything else showing that R JAC < 1. Remember that R JAC < 1 is the maximum absolute row sum. By assumptions, the ith row sum of R = R JAC satisfies That s it. N m ij = j=1 j i a ij < 1. a ii C. T. Kelley Background NCSU, Spring / 58

22 Matrix Splittings and Classical Methods Observations Convergence of Jacobi implies A is nonsingular. Showing R JAC < 1 for any norm would do. The l norm fit the assumptions the best. We have said nothing about the speed of convergence. Jacobi iteration does not depend on the ordering of the variables. Each ξi New can be processed independently of all the others. So Jacobi is easy to parallelize. C. T. Kelley Background NCSU, Spring / 58

23 Matrix Splittings and Classical Methods Gauss-Seidel Iteration Gauss-Seidel changes Jacobi by updating each entry as soon as the computation is done. So ξ New i = 1 β i a ii j<i a ij ξ New j j>i a ij ξ Old j You might think this is better, because the most up-to-date information is in the formula. C. T. Kelley Background NCSU, Spring / 58

24 Matrix Splittings and Classical Methods Gauss-Seidel Iteration One advantage of Gauss-Seidel is that you need only store one copy of x. This loop does the job with only one vector. for i=1:n do sum=0; for j i do sum=sum+a(i,j)*u(j) end for u(i) = (f(i) + sum)/a(i,i) end for C. T. Kelley Background NCSU, Spring / 58

25 Matrix Splittings and Classical Methods Gauss-Seidel Iteration Matrix From the formula, running for i = 1,... N. ξi New = 1 β i a ij ξj New a ij ξ Old j a ii j<i j>i you can see that so (D + U)u n+1 = b Lu n R GS = (D U) 1 L and c = (D U) 1 b. C. T. Kelley Background NCSU, Spring / 58

26 Matrix Splittings and Classical Methods Backwards Gauss-Seidel Gauss-Seidel depends on the ordering. Backwards Gauss-Seidel is ξ New i = 1 β i a ii j>i a ij ξ New j j<i a ij ξ Old j running from i = N, So R BGS = (D L) 1 U. C. T. Kelley Background NCSU, Spring / 58

27 Matrix Splittings and Classical Methods Symmetric Gauss-Seidel A symmetric Gauss-Seidel iteration is a forward Gauss-Seidel iteration followed by a backward Gauss-Seidel iteration. This leads to the iteration matrix R SGS = R BGS R GS = (D U) 1 L(D L) 1 U. If A is symmetric then U = L T. In that event R SGS = (D U) 1 L(D L) 1 U = (D L T ) 1 L(D L) 1 L T. C. T. Kelley Background NCSU, Spring / 58

28 Matrix Splittings and Classical Methods SOR iteration Add a relaxation parameter ω to Gauss-Seidel. R SOR = (D ωl) 1 ((1 ω)d + ωu). Much better performance with good choice of ω. C. T. Kelley Background NCSU, Spring / 58

29 Matrix Splittings and Classical Methods Red-Black Ordering Do example in one and two dimensions Stress parallelism in Jacobi and RB ordering C. T. Kelley Background NCSU, Spring / 58

30 Matrix Splittings and Classical Methods Observations Gauss-Seidel and SOR depend on order of variables. So they are harder to parallelize. While they may perform better than simple Jacobi, it s not a lot better. These methods are not competitive with Krylov methods. They require the least amount of storage, and are still used for that reason. C. T. Kelley Background NCSU, Spring / 58

31 Matrix Splittings and Classical Methods Splitting Methods to Preconditioners Splitting methods can be seen as preconditioned Richardson iteration. You want to find the preconditioner B so that the iteration matrix from the splitting So I R = BA. R = A 1 1 A 2 = I BA. C. T. Kelley Background NCSU, Spring / 58

32 Matrix Splittings and Classical Methods Jacobi preconditioning For the Jacobi splitting A 1 = D, A 2 = L U, we get D 1 (L + U) = I BA so BA = I D 1 (L + U) = D 1 A Jacobi preconditioning is multiplication by D 1. This can be a surprisingly good preconditioner for Krylov methods. C. T. Kelley Background NCSU, Spring / 58

33 Poisson s Equation Poisson s Equation in 1D One space dimension u (x) = f (x) for x (0, 1) Homogeneous Dirichlet boundary conditions u(0) = u(1) = 0 Eigenvalue Problem u (x) = λu(x) for x (0, 1), u(0) = u(1) = 0 C. T. Kelley Background NCSU, Spring / 58

34 Poisson s Equation Solution of Poisson s Equation We can diagonalize the operator using the solutions of the eigenvalue problem {u n } is an orthonormal basis for u n (x) = sin(nπx)/ 2, λ = n 2 π 2 L 2 0 = cl L 2{u C([0, 1]) u(0) = u(1) = 0} and the boundary value problem s solution is u(x) = 1 u n (x) n 2 π 2 n=1 1 1 u n (z)f (z) dz. C. T. Kelley Background NCSU, Spring / 58

35 Poisson s Equation Properties of the Operator The operator d 2 dx 2 : C 2 0 ([0, 1]) C([0, 1]) is injective symmetric with respect to the L 2 scalar product has an L 2 0 orthonormal basis of eigenfunctions has positive eigenvalues C. T. Kelley Background NCSU, Spring / 58

36 Poisson s Equation Solution Operator The solution of Poisson s Equation on [0, 1] with homogeneous Dirichlet boundary conditions is u(x) = 1 0 g(x, z)f (z) dz where x(1 z) 0 < x < z g(x, z) = z(1 x) z < x < 1 C. T. Kelley Background NCSU, Spring / 58

37 Poisson s Equation Central Difference Approximation Add u(x + h) = u(x) + u (x)h + u (x)h 2 /2 + u (h)h 3 /6 + O(h 4 ) to u(x h) = u(x) u (x)h + u (x)h 2 /2 u (h)h 3 /6 + O(h 4 ) and get u (x) = ( u(x + h) + 2u(x) u(x h))/h 2 + O(h 2 ) C. T. Kelley Background NCSU, Spring / 58

38 Poisson s Equation Finite Difference Equations Equally spaced grid x i = ih, 0 i N + 1, h = 1/(N + 1). Approximate u(x i ) by ξ i. Let u = (ξ 1,..., ξ N ) T. Boundary conditions imply that ξ 0 = ξ N+1 = 0. Finite difference equations at interior grid points are for 1 i N. Do inhomogeneous BC ξ i 1 + 2ξ i ξ i+1 h 2 = b i f (x i ) C. T. Kelley Background NCSU, Spring / 58

39 Poisson s Equation Matrix Representation Au = b where A is tridiagonal and symmetric , , A = ,... 0 h ,, 0, ,...,, C. T. Kelley Background NCSU, Spring / 58

40 Poisson s Equation Jacobi and Gauss-Seidel Jacobi: for i=1:n do ξi New (1/2)(h 2 b i + ξ Old i 1 + ξold i+1 ) end for Gauss-Seidel: for i=1:n do ξ i (1/2)(h 2 b i + ξ i 1 + ξ i+1 ) end for C. T. Kelley Background NCSU, Spring / 58

41 Poisson s Equation Jacobi Iteration in MATLAB for ijac=1:n xnew(1) =.5*(hˆ2 * b(1) + xold(2)); for i=2:n 1 xnew(i) =.5*(hˆ2 * b(i) + xold(i 1) + xold(i+1)); end xnew(n) =.5*(hˆ2 * b(n) + xold(n 1)); xold=xnew; end How would you turn this into Gauss-Seidel with a text editor? C. T. Kelley Background NCSU, Spring / 58

42 Poisson s Equation Jacobi Example Let s solve u = 0, u(0) = u(1) = 0. with h = 1/101 and N = 100. The solution is u = 0. We will use as an intial iterate u 0 = x(1 x) cos(49πx) We will take 100 Jacobi iterations. C. T. Kelley Background NCSU, Spring / 58

43 Poisson s Equation Initial Error as Function of x 0.35 Initial Error as Function of $x$ error x C. T. Kelley Background NCSU, Spring / 58

44 Poisson s Equation Final Error as Function of x 0.25 Final Error as Function of $x$ error x C. T. Kelley Background NCSU, Spring / 58

45 Poisson s Equation Final Error as Function of x Convergence History 0.25 l error Iterations C. T. Kelley Background NCSU, Spring / 58

46 Poisson s Equation Eigenvalues and Eigenvectors Theorem: A is symmetric positive definite. The eigenvalues are λ n = h 2 2 (1 cos(πnh)) = π 2 n 2 + O(h 2 ). The eigenvectors u n = (ξ n 1,..., ξn N )T are given by ξ n i = 2/h sin(niπh) C. T. Kelley Background NCSU, Spring / 58

47 Poisson s Equation Comments and Proof Eigenvalues agree with continuous problem to second order. κ(a) = λ N /λ 1 = O(N 2 ) = O(h 2 ). ξi n = u n (x i ) 2/h Proof: Note that ξ0 n = ξn N+1 = 0 ξ n i 1 + 2ξn i ξ n i+1 = 2/h( sin(n(i 1)πh) + 2 sin(niπh) sin(n(i 1)πh)) C. T. Kelley Background NCSU, Spring / 58

48 Poisson s Equation End of Proof Set x = niπh and y = nπh. Use the trig identities sin(x ± y) = sin(x) cos(y) ± sin(y) cos(x) to get ξi 1 n + 2ξn i ξi+1 n = sin(x y) + 2 sin(x) sin(x + y) = 2 sin(x)(1 cos(y)) = λ n ξ n i as asserted. C. T. Kelley Background NCSU, Spring / 58

49 Poisson s Equation Jacobi does Poorly for Poisson If you apply Jacobi to Poisson s equation, iteration matrix is R = D 1 (L + U) = I D 1 (D + L + U) = I D 1 A as we have seen. For Poisson, D = (2/h 2 )I so R = I D 1 A = I (h 2 /2)A. The eigenvalues of R are µ n = 1 (h 2 /2)λ n. So ρ(r) = 1 O(h 2 ) which is very bad. The performance gets worse as the mesh is refined! C. T. Kelley Background NCSU, Spring / 58

50 Poisson s Equation Observations Jacobi (and GS, SOR,... ) are not scalable. The number of iterations needed to reduce the error by a given amount depends on the grid. Fixing this for PDE problems requires a different approach. You can solve the 1D problem in O(N) time with a tridiagonal solver, but... direct methods become harder to use for 2D and 3D problems on complex geometries with unstructured grids. C. T. Kelley Background NCSU, Spring / 58

51 Poisson s Equation Poisson s Equation in Two Dimensions Equation: u xx u yy = f (x, y) for 0 < x, y < 1 Boundary conditions: u(0, y) = u(x, 0) = u(1, y) = u(x, 1) = 0 Similar properties to 1-D Physical Grid: (x i, x j ), x i = i h. Begin with two-dimensional matrix of unknowns u ij u(x i, x j ). Must order the unknowns (ie the grid points) to prepare for a packaged linear solver. C. T. Kelley Background NCSU, Spring / 58

52 Poisson s Equation which leads to... u xx 1 (u(x h, y) 2u(x, y) + u(x + h, y)) h2 u yy 1 (u(x, y h) 2u(x, y) + u(x, y + h)) h2 C. T. Kelley Background NCSU, Spring / 58

53 Poisson s Equation Discrete 2D Poisson, Version 1 1 h 2 ( U i 1,j U i,j 1 + 4U ij U i+1,j U i,j+1 ) = f ij f (x i, x j ) Jacobi, Gauss-Seidel,... are still easy. Here s GS for i=1:n do for j=1:n do U ij 1 ( 4 h 2 ) f ij + U i 1,j + U i,j 1 + U i+1,j + U i,j+1 end for end for So how did I order the unknowns? C. T. Kelley Background NCSU, Spring / 58

54 Poisson s Equation Ordering the Unknowns N 2 N + 1 N 2 N N N + 1 2N N N + 1 N N N C. T. Kelley Background NCSU, Spring / 58

55 Poisson s Equation Creating a Matrix-Vector Representation Define u = (ξ 1,..., ξ N 2) T R N2 by ξ N(i 1)+j = U i,j and β N(i 1)+j = U i,j The Matrix representation is Au = b where... C. T. Kelley Background NCSU, Spring / 58

56 Poisson s Equation Matrix Laplacian in 2D: I A = 1 h 2 T I , 0 I T I, I T I, ,, 0, I T I 0...,...,, 0 I T where I is the N N identity matrix and T is the N N tridiagonal matrix C. T. Kelley Background NCSU, Spring / 58

57 Poisson s Equation Discrete Laplacian in 2D: I T = , , , ,, 0, ,...,, C. T. Kelley Background NCSU, Spring / 58

58 Poisson s Equation Mapping the 2D vector to/from a 1D vector Use the MATLAB reshape command. Example: N = u 2d = u 1d = reshape(u 2d, N N, 1) = (1, 4, 7, 2, 5, 8, 3, 6, 9) T and u 2d = reshape(u 1d, N, N). This means you can do things on the physical grid and still give solvers linear vectors when they need them. C. T. Kelley Background NCSU, Spring / 58

Introduction and Stationary Iterative Methods

Introduction and Stationary Iterative Methods Introduction and C. T. Kelley NC State University tim kelley@ncsu.edu Research Supported by NSF, DOE, ARO, USACE DTU ITMAN, 2011 Outline Notation and Preliminaries General References What you Should Know

More information

CLASSICAL ITERATIVE METHODS

CLASSICAL ITERATIVE METHODS CLASSICAL ITERATIVE METHODS LONG CHEN In this notes we discuss classic iterative methods on solving the linear operator equation (1) Au = f, posed on a finite dimensional Hilbert space V = R N equipped

More information

Lecture 18 Classical Iterative Methods

Lecture 18 Classical Iterative Methods Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,

More information

Chapter 7 Iterative Techniques in Matrix Algebra

Chapter 7 Iterative Techniques in Matrix Algebra Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition

More information

Iterative Methods for Solving A x = b

Iterative Methods for Solving A x = b Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http

More information

9. Iterative Methods for Large Linear Systems

9. Iterative Methods for Large Linear Systems EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1

More information

Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods

Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 March 2015 1 / 70 Topics Introduction to Iterative Methods

More information

Introduction. Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods. Example: First Order Richardson. Strategy

Introduction. Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods. Example: First Order Richardson. Strategy Introduction Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 Solve system Ax = b by repeatedly computing

More information

Lecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University

Lecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University Lecture Note 7: Iterative methods for solving linear systems Xiaoqun Zhang Shanghai Jiao Tong University Last updated: December 24, 2014 1.1 Review on linear algebra Norms of vectors and matrices vector

More information

Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for

Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for 1 Iteration basics Notes for 2016-11-07 An iterative solver for Ax = b is produces a sequence of approximations x (k) x. We always stop after finitely many steps, based on some convergence criterion, e.g.

More information

EXAMPLES OF CLASSICAL ITERATIVE METHODS

EXAMPLES OF CLASSICAL ITERATIVE METHODS EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic

More information

Finite Difference Methods for Boundary Value Problems

Finite Difference Methods for Boundary Value Problems Finite Difference Methods for Boundary Value Problems October 2, 2013 () Finite Differences October 2, 2013 1 / 52 Goals Learn steps to approximate BVPs using the Finite Difference Method Start with two-point

More information

Numerical Programming I (for CSE)

Numerical Programming I (for CSE) Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let

More information

Math 471 (Numerical methods) Chapter 3 (second half). System of equations

Math 471 (Numerical methods) Chapter 3 (second half). System of equations Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular

More information

The amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A.

The amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A. AMSC/CMSC 661 Scientific Computing II Spring 2005 Solution of Sparse Linear Systems Part 2: Iterative methods Dianne P. O Leary c 2005 Solving Sparse Linear Systems: Iterative methods The plan: Iterative

More information

Kasetsart University Workshop. Multigrid methods: An introduction

Kasetsart University Workshop. Multigrid methods: An introduction Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available

More information

6. Iterative Methods for Linear Systems. The stepwise approach to the solution...

6. Iterative Methods for Linear Systems. The stepwise approach to the solution... 6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse

More information

Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018

Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018 Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x

More information

The Conjugate Gradient Method

The Conjugate Gradient Method The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large

More information

Solving Linear Systems

Solving Linear Systems Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 207 Philippe B. Laval (KSU) Linear Systems Fall 207 / 2 Introduction We continue looking how to solve linear systems of the

More information

Notes for CS542G (Iterative Solvers for Linear Systems)

Notes for CS542G (Iterative Solvers for Linear Systems) Notes for CS542G (Iterative Solvers for Linear Systems) Robert Bridson November 20, 2007 1 The Basics We re now looking at efficient ways to solve the linear system of equations Ax = b where in this course,

More information

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system

More information

FEM and Sparse Linear System Solving

FEM and Sparse Linear System Solving FEM & sparse system solving, Lecture 7, Nov 3, 2017 1/46 Lecture 7, Nov 3, 2015: Introduction to Iterative Solvers: Stationary Methods http://people.inf.ethz.ch/arbenz/fem16 Peter Arbenz Computer Science

More information

4.6 Iterative Solvers for Linear Systems

4.6 Iterative Solvers for Linear Systems 4.6 Iterative Solvers for Linear Systems Why use iterative methods? Virtually all direct methods for solving Ax = b require O(n 3 ) floating point operations. In practical applications the matrix A often

More information

Numerical Analysis of Differential Equations Numerical Solution of Elliptic Boundary Value

Numerical Analysis of Differential Equations Numerical Solution of Elliptic Boundary Value Numerical Analysis of Differential Equations 188 5 Numerical Solution of Elliptic Boundary Value Problems 5 Numerical Solution of Elliptic Boundary Value Problems TU Bergakademie Freiberg, SS 2012 Numerical

More information

Iterative methods for Linear System

Iterative methods for Linear System Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and

More information

CAAM 454/554: Stationary Iterative Methods

CAAM 454/554: Stationary Iterative Methods CAAM 454/554: Stationary Iterative Methods Yin Zhang (draft) CAAM, Rice University, Houston, TX 77005 2007, Revised 2010 Abstract Stationary iterative methods for solving systems of linear equations are

More information

Iterative Methods and Multigrid

Iterative Methods and Multigrid Iterative Methods and Multigrid Part 1: Introduction to Multigrid 2000 Eric de Sturler 1 12/02/09 MG01.prz Basic Iterative Methods (1) Nonlinear equation: f(x) = 0 Rewrite as x = F(x), and iterate x i+1

More information

Iterative techniques in matrix algebra

Iterative techniques in matrix algebra Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and

More information

Chapter 2. Solving Systems of Equations. 2.1 Gaussian elimination

Chapter 2. Solving Systems of Equations. 2.1 Gaussian elimination Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix

More information

Iterative Methods. Splitting Methods

Iterative Methods. Splitting Methods Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition

More information

Iterative Solution methods

Iterative Solution methods p. 1/28 TDB NLA Parallel Algorithms for Scientific Computing Iterative Solution methods p. 2/28 TDB NLA Parallel Algorithms for Scientific Computing Basic Iterative Solution methods The ideas to use iterative

More information

NUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP

NUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP ANALELE ŞTIINŢIFICE ALE UNIVERSITĂŢII AL.I. CUZA DIN IAŞI (S.N. MATEMATICĂ, Tomul LIII, 2007, f.1 NUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP BY GINA DURA and RĂZVAN ŞTEFĂNESCU Abstract. The aim

More information

Iterative Methods and Multigrid

Iterative Methods and Multigrid Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be

More information

9.1 Preconditioned Krylov Subspace Methods

9.1 Preconditioned Krylov Subspace Methods Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete

More information

The Solution of Linear Systems AX = B

The Solution of Linear Systems AX = B Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has

More information

Computational Linear Algebra

Computational Linear Algebra Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD

More information

Scientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1

Scientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1 Scientific Computing WS 2018/2019 Lecture 9 Jürgen Fuhrmann juergen.fuhrmann@wias-berlin.de Lecture 9 Slide 1 Lecture 9 Slide 2 Simple iteration with preconditioning Idea: Aû = b iterative scheme û = û

More information

Computational Methods. Systems of Linear Equations

Computational Methods. Systems of Linear Equations Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations

More information

Poisson Equation in 2D

Poisson Equation in 2D A Parallel Strategy Department of Mathematics and Statistics McMaster University March 31, 2010 Outline Introduction 1 Introduction Motivation Discretization Iterative Methods 2 Additive Schwarz Method

More information

Algebraic Multigrid as Solvers and as Preconditioner

Algebraic Multigrid as Solvers and as Preconditioner Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven

More information

Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015

Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015 Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in NUMERICAL ANALYSIS Spring 2015 Instructions: Do exactly two problems from Part A AND two

More information

1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0

1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0 Numerical Analysis 1 1. Nonlinear Equations This lecture note excerpted parts from Michael Heath and Max Gunzburger. Given function f, we seek value x for which where f : D R n R n is nonlinear. f(x) =

More information

Chapter 7. Iterative methods for large sparse linear systems. 7.1 Sparse matrix algebra. Large sparse matrices

Chapter 7. Iterative methods for large sparse linear systems. 7.1 Sparse matrix algebra. Large sparse matrices Chapter 7 Iterative methods for large sparse linear systems In this chapter we revisit the problem of solving linear systems of equations, but now in the context of large sparse systems. The price to pay

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

Lab 1: Iterative Methods for Solving Linear Systems

Lab 1: Iterative Methods for Solving Linear Systems Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as

More information

Math Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.

Math Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012. Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems

More information

COURSE Iterative methods for solving linear systems

COURSE Iterative methods for solving linear systems COURSE 0 4.3. Iterative methods for solving linear systems Because of round-off errors, direct methods become less efficient than iterative methods for large systems (>00 000 variables). An iterative scheme

More information

Iterative Methods for Sparse Linear Systems

Iterative Methods for Sparse Linear Systems Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University

More information

Algebra C Numerical Linear Algebra Sample Exam Problems

Algebra C Numerical Linear Algebra Sample Exam Problems Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric

More information

Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018

Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018 1 Linear Systems Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March, 018 Consider the system 4x y + z = 7 4x 8y + z = 1 x + y + 5z = 15. We then obtain x = 1 4 (7 + y z)

More information

Introduction to Boundary Value Problems

Introduction to Boundary Value Problems Chapter 5 Introduction to Boundary Value Problems When we studied IVPs we saw that we were given the initial value of a function and a differential equation which governed its behavior for subsequent times.

More information

Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems

Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Dr. Noemi Friedman, 09.2.205. Reminder: Instationary heat

More information

Numerical Solution Techniques in Mechanical and Aerospace Engineering

Numerical Solution Techniques in Mechanical and Aerospace Engineering Numerical Solution Techniques in Mechanical and Aerospace Engineering Chunlei Liang LECTURE 3 Solvers of linear algebraic equations 3.1. Outline of Lecture Finite-difference method for a 2D elliptic PDE

More information

Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers

Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Dr. Noemi

More information

Introduction to Scientific Computing

Introduction to Scientific Computing (Lecture 5: Linear system of equations / Matrix Splitting) Bojana Rosić, Thilo Moshagen Institute of Scientific Computing Motivation Let us resolve the problem scheme by using Kirchhoff s laws: the algebraic

More information

Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White

Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Introduction to Simulation - Lecture 2 Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Outline Reminder about

More information

Computational Economics and Finance

Computational Economics and Finance Computational Economics and Finance Part II: Linear Equations Spring 2016 Outline Back Substitution, LU and other decomposi- Direct methods: tions Error analysis and condition numbers Iterative methods:

More information

Numerical Methods - Numerical Linear Algebra

Numerical Methods - Numerical Linear Algebra Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear

More information

JACOBI S ITERATION METHOD

JACOBI S ITERATION METHOD ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes

More information

Numerical Methods I Non-Square and Sparse Linear Systems

Numerical Methods I Non-Square and Sparse Linear Systems Numerical Methods I Non-Square and Sparse Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 25th, 2014 A. Donev (Courant

More information

Here is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J

Here is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J Class Notes 4: THE SPECTRAL RADIUS, NORM CONVERGENCE AND SOR. Math 639d Due Date: Feb. 7 (updated: February 5, 2018) In the first part of this week s reading, we will prove Theorem 2 of the previous class.

More information

Von Neumann Analysis of Jacobi and Gauss-Seidel Iterations

Von Neumann Analysis of Jacobi and Gauss-Seidel Iterations Von Neumann Analysis of Jacobi and Gauss-Seidel Iterations We consider the FDA to the 1D Poisson equation on a grid x covering [0,1] with uniform spacing h, h (u +1 u + u 1 ) = f whose exact solution (to

More information

Lecture 1 INF-MAT3350/ : Some Tridiagonal Matrix Problems

Lecture 1 INF-MAT3350/ : Some Tridiagonal Matrix Problems Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems Tom Lyche University of Oslo Norway Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.1/33 Plan for the day 1. Notation

More information

Stabilization and Acceleration of Algebraic Multigrid Method

Stabilization and Acceleration of Algebraic Multigrid Method Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration

More information

Motivation: Sparse matrices and numerical PDE's

Motivation: Sparse matrices and numerical PDE's Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting

More information

Splitting Iteration Methods for Positive Definite Linear Systems

Splitting Iteration Methods for Positive Definite Linear Systems Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System

More information

Theory of Iterative Methods

Theory of Iterative Methods Based on Strang s Introduction to Applied Mathematics Theory of Iterative Methods The Iterative Idea To solve Ax = b, write Mx (k+1) = (M A)x (k) + b, k = 0, 1,,... Then the error e (k) x (k) x satisfies

More information

Solving Sparse Linear Systems: Iterative methods

Solving Sparse Linear Systems: Iterative methods Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccs Lecture Notes for Unit VII Sparse Matrix Computations Part 2: Iterative Methods Dianne P. O Leary c 2008,2010

More information

Solving Sparse Linear Systems: Iterative methods

Solving Sparse Linear Systems: Iterative methods Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 2: Iterative Methods Dianne P. O Leary

More information

PowerPoints organized by Dr. Michael R. Gustafson II, Duke University

PowerPoints organized by Dr. Michael R. Gustafson II, Duke University Part 3 Chapter 10 LU Factorization PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

More information

Finite difference method for elliptic problems: I

Finite difference method for elliptic problems: I Finite difference method for elliptic problems: I Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 560065 http://math.tifrbng.res.in/~praveen

More information

Introduction to Iterative Solvers of Linear Systems

Introduction to Iterative Solvers of Linear Systems Introduction to Iterative Solvers of Linear Systems SFB Training Event January 2012 Prof. Dr. Andreas Frommer Typeset by Lukas Krämer, Simon-Wolfgang Mages and Rudolf Rödl 1 Classes of Matrices and their

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 11 Partial Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002.

More information

DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular

DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix

More information

x x2 2 + x3 3 x4 3. Use the divided-difference method to find a polynomial of least degree that fits the values shown: (b)

x x2 2 + x3 3 x4 3. Use the divided-difference method to find a polynomial of least degree that fits the values shown: (b) Numerical Methods - PROBLEMS. The Taylor series, about the origin, for log( + x) is x x2 2 + x3 3 x4 4 + Find an upper bound on the magnitude of the truncation error on the interval x.5 when log( + x)

More information

Solving Linear Systems

Solving Linear Systems Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Linear Systems Fall 2015 1 / 12 Introduction We continue looking how to solve linear systems of

More information

Qualifying Examination

Qualifying Examination Summer 24 Day. Monday, September 5, 24 You have three hours to complete this exam. Work all problems. Start each problem on a All problems are 2 points. Please email any electronic files in support of

More information

Iterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)

Iterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009) Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential

More information

Parallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1

Parallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1 Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations

More information

Modelling and implementation of algorithms in applied mathematics using MPI

Modelling and implementation of algorithms in applied mathematics using MPI Modelling and implementation of algorithms in applied mathematics using MPI Lecture 3: Linear Systems: Simple Iterative Methods and their parallelization, Programming MPI G. Rapin Brazil March 2011 Outline

More information

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C. Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference

More information

Iterative Methods for Ax=b

Iterative Methods for Ax=b 1 FUNDAMENTALS 1 Iterative Methods for Ax=b 1 Fundamentals consider the solution of the set of simultaneous equations Ax = b where A is a square matrix, n n and b is a right hand vector. We write the iterative

More information

Num. discretization. Numerical simulations. Finite elements Finite volume. Finite diff. Mathematical model A set of ODEs/PDEs. Integral equations

Num. discretization. Numerical simulations. Finite elements Finite volume. Finite diff. Mathematical model A set of ODEs/PDEs. Integral equations Scientific Computing Computer simulations Road map p. 1/46 Numerical simulations Physical phenomenon Mathematical model A set of ODEs/PDEs Integral equations Something else Num. discretization Finite diff.

More information

7.2 Steepest Descent and Preconditioning

7.2 Steepest Descent and Preconditioning 7.2 Steepest Descent and Preconditioning Descent methods are a broad class of iterative methods for finding solutions of the linear system Ax = b for symmetric positive definite matrix A R n n. Consider

More information

Chapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs

Chapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs Chapter Two: Numerical Methods for Elliptic PDEs Finite Difference Methods for Elliptic PDEs.. Finite difference scheme. We consider a simple example u := subject to Dirichlet boundary conditions ( ) u

More information

Jordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS

Jordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS Jordan Journal of Mathematics and Statistics JJMS) 53), 2012, pp.169-184 A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS ADEL H. AL-RABTAH Abstract. The Jacobi and Gauss-Seidel iterative

More information

Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo. Department of Mathematics Iowa State University

Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo. Department of Mathematics Iowa State University Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department

More information

Lecture 8: Fast Linear Solvers (Part 7)

Lecture 8: Fast Linear Solvers (Part 7) Lecture 8: Fast Linear Solvers (Part 7) 1 Modified Gram-Schmidt Process with Reorthogonalization Test Reorthogonalization If Av k 2 + δ v k+1 2 = Av k 2 to working precision. δ = 10 3 2 Householder Arnoldi

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline

More information

30.5. Iterative Methods for Systems of Equations. Introduction. Prerequisites. Learning Outcomes

30.5. Iterative Methods for Systems of Equations. Introduction. Prerequisites. Learning Outcomes Iterative Methods for Systems of Equations 0.5 Introduction There are occasions when direct methods (like Gaussian elimination or the use of an LU decomposition) are not the best way to solve a system

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical

More information

Iterative Methods for Linear Systems

Iterative Methods for Linear Systems Iterative Methods for Linear Systems 1. Introduction: Direct solvers versus iterative solvers In many applications we have to solve a linear system Ax = b with A R n n and b R n given. If n is large the

More information

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear

More information

Cache Oblivious Stencil Computations

Cache Oblivious Stencil Computations Cache Oblivious Stencil Computations S. HUNOLD J. L. TRÄFF F. VERSACI Lectures on High Performance Computing 13 April 2015 F. Versaci (TU Wien) Cache Oblivious Stencil Computations 13 April 2015 1 / 19

More information

Classical iterative methods for linear systems

Classical iterative methods for linear systems Classical iterative methods for linear systems Ed Bueler MATH 615 Numerical Analysis of Differential Equations 27 February 1 March, 2017 Ed Bueler (MATH 615 NADEs) Classical iterative methods for linear

More information

Solving linear systems (6 lectures)

Solving linear systems (6 lectures) Chapter 2 Solving linear systems (6 lectures) 2.1 Solving linear systems: LU factorization (1 lectures) Reference: [Trefethen, Bau III] Lecture 20, 21 How do you solve Ax = b? (2.1.1) In numerical linear

More information

Review of matrices. Let m, n IN. A rectangle of numbers written like A =

Review of matrices. Let m, n IN. A rectangle of numbers written like A = Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an

More information

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008

More information

Illustration of Gaussian elimination to find LU factorization. A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44

Illustration of Gaussian elimination to find LU factorization. A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44 Illustration of Gaussian elimination to find LU factorization. A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a 1 Compute multipliers : Eliminate entries in first column: m i1 = a i1 a 11, i = 2, 3, 4 ith

More information