Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
|
|
- Abner Lindsey
- 6 years ago
- Views:
Transcription
1 Background C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
2 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms C. T. Kelley Background NCSU, Spring / 58
3 A Stationary Iterative Method converts Au = f to u = Ru + c and the iteration is u n+1 = Ru n + c M is called the iteration matrix. This iteration is also called Richardson Iteration. The method is called stationary because the formula does not change as a function of u n. C. T. Kelley Background NCSU, Spring / 58
4 Convergence and the Banach Lemma Banach Lemma Let R be N N. Assume that R < 1 for some induced matrix norm. Then (I R) is nonsingular (I R) 1 = l=0 Rl (I R) 1 (1 R ) 1 C. T. Kelley Background NCSU, Spring / 58
5 Convergence and the Banach Lemma Proof of Banach Lemma: I We will show that the series R l = (I R) 1. l=0 The partial sums S k = form a Cauchy sequence in R N N. To see this note that for all m > k m S k S m R l. And... k l=0 R l l=k+1 C. T. Kelley Background NCSU, Spring / 58
6 Convergence and the Banach Lemma Proof of Banach Lemma: II R l R l because is a matrix norm that is induced by a vector norm. Hence m ( ) 1 R S k S m R l = R k+1 m k 0 1 l=k+1 as m, k. So the series converges. Let S = l=0 R l C. T. Kelley Background NCSU, Spring / 58
7 Convergence and the Banach Lemma Proof of Banach Lemma: II Clearly Finally RS = R l+1 = R l = S I and so l=0 l=1 (I R)S = I and S = (I R) 1. (I R) 1 R l = (1 R ) 1. l=0 C. T. Kelley Background NCSU, Spring / 58
8 Convergence and the Banach Lemma Convergence for If R < 1 for any induced matrix norm then the stationary iteration u n+1 = Ru n + c converges for all c and u 0 to u = (I R) 1 c Proof: Clearly u n+1 = n R l c + R n u 0 (I R) 1 c = u. l=0 C. T. Kelley Background NCSU, Spring / 58
9 Convergence and the Banach Lemma Convergence Speed Let R = α < 1 and u = (I R) 1 c. Then Proof: u u n α n u u 0. u u n = l=n Rl c R n u 0 = R( l=n 1 Rl c R n 1 u 0 ) = R(u u n 1 ) α u u n 1 α n u u 0. C. T. Kelley Background NCSU, Spring / 58
10 Convergence and the Banach Lemma Spectral Radius The spectrum of R σ(r), is the set of eigenvalues of R. The spectral radius of R is ρ(r) = max λ σ(r) λ Theorem ρ(r) < 1 if and only if R < 1 for some induced matrix norm. A stationary iterative method will u n+1 = Ru n + c converges for all initial iterates and right sides if and only if ρ(r) < 1. The spectral radius does not depend on any norm. C. T. Kelley Background NCSU, Spring / 58
11 Convergence and the Banach Lemma Predicting Convergence Suppose you know that α < 1. Then e n+1 = u u n+1 = (Ru + c) (Ru n + c) = Re n Hence e n α n e 0 and e n τ e 0 if α n < τ or n > log(τ)/ log(α). C. T. Kelley Background NCSU, Spring / 58
12 Convergence and the Banach Lemma Preconditioned Richardson Iteration If I A < 1 then one can apply Richardson iteration directly to Au = f u n+1 = (I A)u n + f Sometimes one can find a approximate inverse B for which I BA < 1 and precondition with B to obtain BAu = Bf and the iteration is u n+1 = (I BA)u n + Bf C. T. Kelley Background NCSU, Spring / 58
13 Convergence and the Banach Lemma Approximate Inverse Preconditioning: I Theorem: If A and B are N N matrices and B is an approximate inverse of A, then A and B are both nonsingular and and A 1 A 1 B B 1 I BA, B 1 B I BA 1 I BA, A B 1 A 1 I BA, A I BA 1 I BA. C. T. Kelley Background NCSU, Spring / 58
14 Convergence and the Banach Lemma Approximate Inverse Preconditioning: II Proof: Let R = I BA. The Banach Lemma implies that I R = I (I BA) = BA is nonsingular. Hence both A and B are nonsingular. Moreover A 1 B 1 = (I R) R = 1 1 I BA. C. T. Kelley Background NCSU, Spring / 58
15 Convergence and the Banach Lemma Approximate Inverse Preconditioning: III Use A 1 = (I R) 1 B to get the first part A 1 B (I R) 1 The second pair of inequalities follows from and the first. B 1 I BA. A 1 B = (I BA)A 1, A B 1 = B 1 (I BA) C. T. Kelley Background NCSU, Spring / 58
16 Matrix Splittings and Classical Methods Matrix Splittings One way to convert Au = f to Ru = c is to split A where A 1 is nonsingular A = A 1 + A 2 A 1 y = q is easy to solve for all q and then solve u = A 1 1 (f A 2u) Ru + c. Here R = A 1 1 A 2 and c = A 1 1 f. Remember A 1 z means solve A 1 y = z, not compute A 1 1. C. T. Kelley Background NCSU, Spring / 58
17 Matrix Splittings and Classical Methods Jacobi Iteration: I Write Au = f explicitly a 11 ξ a 1N ξ N = β 1. a N1 ξ a NN ξ N = β N and solve the ith equation for ξ i, pretending the other components are know. You get ξ i = 1 β i a ij ξ j a ii j i which is a linear fixed point problem equivalent to Au = f. C. T. Kelley Background NCSU, Spring / 58
18 Matrix Splittings and Classical Methods Jacobi Iteration: II The iteration is ξ New i = 1 a ii β i j i a ij ξ Old j So what are R and c? Split A = A 1 + A 2, where A 1 = D, A 2 = L U, D is the diagonal of A, and L and U are the (strict) lower and upper triangular parts. then x New = D 1 (f + (L + U))x Old. C. T. Kelley Background NCSU, Spring / 58
19 Matrix Splittings and Classical Methods Jacobi Iteration: III So the iteration is u n+1 = D 1 (L + U)u n + D 1 f and the iteration matrix is R JAC = D 1 (L + U). Is there any reason for ρ(r JAC ) < 1? C. T. Kelley Background NCSU, Spring / 58
20 Matrix Splittings and Classical Methods Convergence for Strictly Diagonally Dominant A Theorem: Let A be an N N matrix and assume that A is strictly diagonally dominant. That is for all 1 i N 0 < j i a ij < a ii. Then A is nonsingular and the Jacobi iteration converges to u = A 1 f for all f. C. T. Kelley Background NCSU, Spring / 58
21 Matrix Splittings and Classical Methods Proof: Convergence for Strictly Diagonally Dominant A Our assumptions imply that a ii 0, so the iteration is defined. We can prove everything else showing that R JAC < 1. Remember that R JAC < 1 is the maximum absolute row sum. By assumptions, the ith row sum of R = R JAC satisfies That s it. N m ij = j=1 j i a ij < 1. a ii C. T. Kelley Background NCSU, Spring / 58
22 Matrix Splittings and Classical Methods Observations Convergence of Jacobi implies A is nonsingular. Showing R JAC < 1 for any norm would do. The l norm fit the assumptions the best. We have said nothing about the speed of convergence. Jacobi iteration does not depend on the ordering of the variables. Each ξi New can be processed independently of all the others. So Jacobi is easy to parallelize. C. T. Kelley Background NCSU, Spring / 58
23 Matrix Splittings and Classical Methods Gauss-Seidel Iteration Gauss-Seidel changes Jacobi by updating each entry as soon as the computation is done. So ξ New i = 1 β i a ii j<i a ij ξ New j j>i a ij ξ Old j You might think this is better, because the most up-to-date information is in the formula. C. T. Kelley Background NCSU, Spring / 58
24 Matrix Splittings and Classical Methods Gauss-Seidel Iteration One advantage of Gauss-Seidel is that you need only store one copy of x. This loop does the job with only one vector. for i=1:n do sum=0; for j i do sum=sum+a(i,j)*u(j) end for u(i) = (f(i) + sum)/a(i,i) end for C. T. Kelley Background NCSU, Spring / 58
25 Matrix Splittings and Classical Methods Gauss-Seidel Iteration Matrix From the formula, running for i = 1,... N. ξi New = 1 β i a ij ξj New a ij ξ Old j a ii j<i j>i you can see that so (D + U)u n+1 = b Lu n R GS = (D U) 1 L and c = (D U) 1 b. C. T. Kelley Background NCSU, Spring / 58
26 Matrix Splittings and Classical Methods Backwards Gauss-Seidel Gauss-Seidel depends on the ordering. Backwards Gauss-Seidel is ξ New i = 1 β i a ii j>i a ij ξ New j j<i a ij ξ Old j running from i = N, So R BGS = (D L) 1 U. C. T. Kelley Background NCSU, Spring / 58
27 Matrix Splittings and Classical Methods Symmetric Gauss-Seidel A symmetric Gauss-Seidel iteration is a forward Gauss-Seidel iteration followed by a backward Gauss-Seidel iteration. This leads to the iteration matrix R SGS = R BGS R GS = (D U) 1 L(D L) 1 U. If A is symmetric then U = L T. In that event R SGS = (D U) 1 L(D L) 1 U = (D L T ) 1 L(D L) 1 L T. C. T. Kelley Background NCSU, Spring / 58
28 Matrix Splittings and Classical Methods SOR iteration Add a relaxation parameter ω to Gauss-Seidel. R SOR = (D ωl) 1 ((1 ω)d + ωu). Much better performance with good choice of ω. C. T. Kelley Background NCSU, Spring / 58
29 Matrix Splittings and Classical Methods Red-Black Ordering Do example in one and two dimensions Stress parallelism in Jacobi and RB ordering C. T. Kelley Background NCSU, Spring / 58
30 Matrix Splittings and Classical Methods Observations Gauss-Seidel and SOR depend on order of variables. So they are harder to parallelize. While they may perform better than simple Jacobi, it s not a lot better. These methods are not competitive with Krylov methods. They require the least amount of storage, and are still used for that reason. C. T. Kelley Background NCSU, Spring / 58
31 Matrix Splittings and Classical Methods Splitting Methods to Preconditioners Splitting methods can be seen as preconditioned Richardson iteration. You want to find the preconditioner B so that the iteration matrix from the splitting So I R = BA. R = A 1 1 A 2 = I BA. C. T. Kelley Background NCSU, Spring / 58
32 Matrix Splittings and Classical Methods Jacobi preconditioning For the Jacobi splitting A 1 = D, A 2 = L U, we get D 1 (L + U) = I BA so BA = I D 1 (L + U) = D 1 A Jacobi preconditioning is multiplication by D 1. This can be a surprisingly good preconditioner for Krylov methods. C. T. Kelley Background NCSU, Spring / 58
33 Poisson s Equation Poisson s Equation in 1D One space dimension u (x) = f (x) for x (0, 1) Homogeneous Dirichlet boundary conditions u(0) = u(1) = 0 Eigenvalue Problem u (x) = λu(x) for x (0, 1), u(0) = u(1) = 0 C. T. Kelley Background NCSU, Spring / 58
34 Poisson s Equation Solution of Poisson s Equation We can diagonalize the operator using the solutions of the eigenvalue problem {u n } is an orthonormal basis for u n (x) = sin(nπx)/ 2, λ = n 2 π 2 L 2 0 = cl L 2{u C([0, 1]) u(0) = u(1) = 0} and the boundary value problem s solution is u(x) = 1 u n (x) n 2 π 2 n=1 1 1 u n (z)f (z) dz. C. T. Kelley Background NCSU, Spring / 58
35 Poisson s Equation Properties of the Operator The operator d 2 dx 2 : C 2 0 ([0, 1]) C([0, 1]) is injective symmetric with respect to the L 2 scalar product has an L 2 0 orthonormal basis of eigenfunctions has positive eigenvalues C. T. Kelley Background NCSU, Spring / 58
36 Poisson s Equation Solution Operator The solution of Poisson s Equation on [0, 1] with homogeneous Dirichlet boundary conditions is u(x) = 1 0 g(x, z)f (z) dz where x(1 z) 0 < x < z g(x, z) = z(1 x) z < x < 1 C. T. Kelley Background NCSU, Spring / 58
37 Poisson s Equation Central Difference Approximation Add u(x + h) = u(x) + u (x)h + u (x)h 2 /2 + u (h)h 3 /6 + O(h 4 ) to u(x h) = u(x) u (x)h + u (x)h 2 /2 u (h)h 3 /6 + O(h 4 ) and get u (x) = ( u(x + h) + 2u(x) u(x h))/h 2 + O(h 2 ) C. T. Kelley Background NCSU, Spring / 58
38 Poisson s Equation Finite Difference Equations Equally spaced grid x i = ih, 0 i N + 1, h = 1/(N + 1). Approximate u(x i ) by ξ i. Let u = (ξ 1,..., ξ N ) T. Boundary conditions imply that ξ 0 = ξ N+1 = 0. Finite difference equations at interior grid points are for 1 i N. Do inhomogeneous BC ξ i 1 + 2ξ i ξ i+1 h 2 = b i f (x i ) C. T. Kelley Background NCSU, Spring / 58
39 Poisson s Equation Matrix Representation Au = b where A is tridiagonal and symmetric , , A = ,... 0 h ,, 0, ,...,, C. T. Kelley Background NCSU, Spring / 58
40 Poisson s Equation Jacobi and Gauss-Seidel Jacobi: for i=1:n do ξi New (1/2)(h 2 b i + ξ Old i 1 + ξold i+1 ) end for Gauss-Seidel: for i=1:n do ξ i (1/2)(h 2 b i + ξ i 1 + ξ i+1 ) end for C. T. Kelley Background NCSU, Spring / 58
41 Poisson s Equation Jacobi Iteration in MATLAB for ijac=1:n xnew(1) =.5*(hˆ2 * b(1) + xold(2)); for i=2:n 1 xnew(i) =.5*(hˆ2 * b(i) + xold(i 1) + xold(i+1)); end xnew(n) =.5*(hˆ2 * b(n) + xold(n 1)); xold=xnew; end How would you turn this into Gauss-Seidel with a text editor? C. T. Kelley Background NCSU, Spring / 58
42 Poisson s Equation Jacobi Example Let s solve u = 0, u(0) = u(1) = 0. with h = 1/101 and N = 100. The solution is u = 0. We will use as an intial iterate u 0 = x(1 x) cos(49πx) We will take 100 Jacobi iterations. C. T. Kelley Background NCSU, Spring / 58
43 Poisson s Equation Initial Error as Function of x 0.35 Initial Error as Function of $x$ error x C. T. Kelley Background NCSU, Spring / 58
44 Poisson s Equation Final Error as Function of x 0.25 Final Error as Function of $x$ error x C. T. Kelley Background NCSU, Spring / 58
45 Poisson s Equation Final Error as Function of x Convergence History 0.25 l error Iterations C. T. Kelley Background NCSU, Spring / 58
46 Poisson s Equation Eigenvalues and Eigenvectors Theorem: A is symmetric positive definite. The eigenvalues are λ n = h 2 2 (1 cos(πnh)) = π 2 n 2 + O(h 2 ). The eigenvectors u n = (ξ n 1,..., ξn N )T are given by ξ n i = 2/h sin(niπh) C. T. Kelley Background NCSU, Spring / 58
47 Poisson s Equation Comments and Proof Eigenvalues agree with continuous problem to second order. κ(a) = λ N /λ 1 = O(N 2 ) = O(h 2 ). ξi n = u n (x i ) 2/h Proof: Note that ξ0 n = ξn N+1 = 0 ξ n i 1 + 2ξn i ξ n i+1 = 2/h( sin(n(i 1)πh) + 2 sin(niπh) sin(n(i 1)πh)) C. T. Kelley Background NCSU, Spring / 58
48 Poisson s Equation End of Proof Set x = niπh and y = nπh. Use the trig identities sin(x ± y) = sin(x) cos(y) ± sin(y) cos(x) to get ξi 1 n + 2ξn i ξi+1 n = sin(x y) + 2 sin(x) sin(x + y) = 2 sin(x)(1 cos(y)) = λ n ξ n i as asserted. C. T. Kelley Background NCSU, Spring / 58
49 Poisson s Equation Jacobi does Poorly for Poisson If you apply Jacobi to Poisson s equation, iteration matrix is R = D 1 (L + U) = I D 1 (D + L + U) = I D 1 A as we have seen. For Poisson, D = (2/h 2 )I so R = I D 1 A = I (h 2 /2)A. The eigenvalues of R are µ n = 1 (h 2 /2)λ n. So ρ(r) = 1 O(h 2 ) which is very bad. The performance gets worse as the mesh is refined! C. T. Kelley Background NCSU, Spring / 58
50 Poisson s Equation Observations Jacobi (and GS, SOR,... ) are not scalable. The number of iterations needed to reduce the error by a given amount depends on the grid. Fixing this for PDE problems requires a different approach. You can solve the 1D problem in O(N) time with a tridiagonal solver, but... direct methods become harder to use for 2D and 3D problems on complex geometries with unstructured grids. C. T. Kelley Background NCSU, Spring / 58
51 Poisson s Equation Poisson s Equation in Two Dimensions Equation: u xx u yy = f (x, y) for 0 < x, y < 1 Boundary conditions: u(0, y) = u(x, 0) = u(1, y) = u(x, 1) = 0 Similar properties to 1-D Physical Grid: (x i, x j ), x i = i h. Begin with two-dimensional matrix of unknowns u ij u(x i, x j ). Must order the unknowns (ie the grid points) to prepare for a packaged linear solver. C. T. Kelley Background NCSU, Spring / 58
52 Poisson s Equation which leads to... u xx 1 (u(x h, y) 2u(x, y) + u(x + h, y)) h2 u yy 1 (u(x, y h) 2u(x, y) + u(x, y + h)) h2 C. T. Kelley Background NCSU, Spring / 58
53 Poisson s Equation Discrete 2D Poisson, Version 1 1 h 2 ( U i 1,j U i,j 1 + 4U ij U i+1,j U i,j+1 ) = f ij f (x i, x j ) Jacobi, Gauss-Seidel,... are still easy. Here s GS for i=1:n do for j=1:n do U ij 1 ( 4 h 2 ) f ij + U i 1,j + U i,j 1 + U i+1,j + U i,j+1 end for end for So how did I order the unknowns? C. T. Kelley Background NCSU, Spring / 58
54 Poisson s Equation Ordering the Unknowns N 2 N + 1 N 2 N N N + 1 2N N N + 1 N N N C. T. Kelley Background NCSU, Spring / 58
55 Poisson s Equation Creating a Matrix-Vector Representation Define u = (ξ 1,..., ξ N 2) T R N2 by ξ N(i 1)+j = U i,j and β N(i 1)+j = U i,j The Matrix representation is Au = b where... C. T. Kelley Background NCSU, Spring / 58
56 Poisson s Equation Matrix Laplacian in 2D: I A = 1 h 2 T I , 0 I T I, I T I, ,, 0, I T I 0...,...,, 0 I T where I is the N N identity matrix and T is the N N tridiagonal matrix C. T. Kelley Background NCSU, Spring / 58
57 Poisson s Equation Discrete Laplacian in 2D: I T = , , , ,, 0, ,...,, C. T. Kelley Background NCSU, Spring / 58
58 Poisson s Equation Mapping the 2D vector to/from a 1D vector Use the MATLAB reshape command. Example: N = u 2d = u 1d = reshape(u 2d, N N, 1) = (1, 4, 7, 2, 5, 8, 3, 6, 9) T and u 2d = reshape(u 1d, N, N). This means you can do things on the physical grid and still give solvers linear vectors when they need them. C. T. Kelley Background NCSU, Spring / 58
Introduction and Stationary Iterative Methods
Introduction and C. T. Kelley NC State University tim kelley@ncsu.edu Research Supported by NSF, DOE, ARO, USACE DTU ITMAN, 2011 Outline Notation and Preliminaries General References What you Should Know
More informationCLASSICAL ITERATIVE METHODS
CLASSICAL ITERATIVE METHODS LONG CHEN In this notes we discuss classic iterative methods on solving the linear operator equation (1) Au = f, posed on a finite dimensional Hilbert space V = R N equipped
More informationLecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More information9. Iterative Methods for Large Linear Systems
EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1
More informationMath 1080: Numerical Linear Algebra Chapter 4, Iterative Methods
Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 March 2015 1 / 70 Topics Introduction to Iterative Methods
More informationIntroduction. Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods. Example: First Order Richardson. Strategy
Introduction Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 Solve system Ax = b by repeatedly computing
More informationLecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University
Lecture Note 7: Iterative methods for solving linear systems Xiaoqun Zhang Shanghai Jiao Tong University Last updated: December 24, 2014 1.1 Review on linear algebra Norms of vectors and matrices vector
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Iteration basics Notes for 2016-11-07 An iterative solver for Ax = b is produces a sequence of approximations x (k) x. We always stop after finitely many steps, based on some convergence criterion, e.g.
More informationEXAMPLES OF CLASSICAL ITERATIVE METHODS
EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic
More informationFinite Difference Methods for Boundary Value Problems
Finite Difference Methods for Boundary Value Problems October 2, 2013 () Finite Differences October 2, 2013 1 / 52 Goals Learn steps to approximate BVPs using the Finite Difference Method Start with two-point
More informationNumerical Programming I (for CSE)
Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let
More informationMath 471 (Numerical methods) Chapter 3 (second half). System of equations
Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular
More informationThe amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A.
AMSC/CMSC 661 Scientific Computing II Spring 2005 Solution of Sparse Linear Systems Part 2: Iterative methods Dianne P. O Leary c 2005 Solving Sparse Linear Systems: Iterative methods The plan: Iterative
More informationKasetsart University Workshop. Multigrid methods: An introduction
Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available
More information6. Iterative Methods for Linear Systems. The stepwise approach to the solution...
6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse
More informationMath/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018
Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 207 Philippe B. Laval (KSU) Linear Systems Fall 207 / 2 Introduction We continue looking how to solve linear systems of the
More informationNotes for CS542G (Iterative Solvers for Linear Systems)
Notes for CS542G (Iterative Solvers for Linear Systems) Robert Bridson November 20, 2007 1 The Basics We re now looking at efficient ways to solve the linear system of equations Ax = b where in this course,
More informationSparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations
Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system
More informationFEM and Sparse Linear System Solving
FEM & sparse system solving, Lecture 7, Nov 3, 2017 1/46 Lecture 7, Nov 3, 2015: Introduction to Iterative Solvers: Stationary Methods http://people.inf.ethz.ch/arbenz/fem16 Peter Arbenz Computer Science
More information4.6 Iterative Solvers for Linear Systems
4.6 Iterative Solvers for Linear Systems Why use iterative methods? Virtually all direct methods for solving Ax = b require O(n 3 ) floating point operations. In practical applications the matrix A often
More informationNumerical Analysis of Differential Equations Numerical Solution of Elliptic Boundary Value
Numerical Analysis of Differential Equations 188 5 Numerical Solution of Elliptic Boundary Value Problems 5 Numerical Solution of Elliptic Boundary Value Problems TU Bergakademie Freiberg, SS 2012 Numerical
More informationIterative methods for Linear System
Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and
More informationCAAM 454/554: Stationary Iterative Methods
CAAM 454/554: Stationary Iterative Methods Yin Zhang (draft) CAAM, Rice University, Houston, TX 77005 2007, Revised 2010 Abstract Stationary iterative methods for solving systems of linear equations are
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 1: Introduction to Multigrid 2000 Eric de Sturler 1 12/02/09 MG01.prz Basic Iterative Methods (1) Nonlinear equation: f(x) = 0 Rewrite as x = F(x), and iterate x i+1
More informationIterative techniques in matrix algebra
Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and
More informationChapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
More informationIterative Methods. Splitting Methods
Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition
More informationIterative Solution methods
p. 1/28 TDB NLA Parallel Algorithms for Scientific Computing Iterative Solution methods p. 2/28 TDB NLA Parallel Algorithms for Scientific Computing Basic Iterative Solution methods The ideas to use iterative
More informationNUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP
ANALELE ŞTIINŢIFICE ALE UNIVERSITĂŢII AL.I. CUZA DIN IAŞI (S.N. MATEMATICĂ, Tomul LIII, 2007, f.1 NUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP BY GINA DURA and RĂZVAN ŞTEFĂNESCU Abstract. The aim
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be
More information9.1 Preconditioned Krylov Subspace Methods
Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete
More informationThe Solution of Linear Systems AX = B
Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD
More informationScientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1
Scientific Computing WS 2018/2019 Lecture 9 Jürgen Fuhrmann juergen.fuhrmann@wias-berlin.de Lecture 9 Slide 1 Lecture 9 Slide 2 Simple iteration with preconditioning Idea: Aû = b iterative scheme û = û
More informationComputational Methods. Systems of Linear Equations
Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations
More informationPoisson Equation in 2D
A Parallel Strategy Department of Mathematics and Statistics McMaster University March 31, 2010 Outline Introduction 1 Introduction Motivation Discretization Iterative Methods 2 Additive Schwarz Method
More informationAlgebraic Multigrid as Solvers and as Preconditioner
Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven
More informationDepartment of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015
Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in NUMERICAL ANALYSIS Spring 2015 Instructions: Do exactly two problems from Part A AND two
More information1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0
Numerical Analysis 1 1. Nonlinear Equations This lecture note excerpted parts from Michael Heath and Max Gunzburger. Given function f, we seek value x for which where f : D R n R n is nonlinear. f(x) =
More informationChapter 7. Iterative methods for large sparse linear systems. 7.1 Sparse matrix algebra. Large sparse matrices
Chapter 7 Iterative methods for large sparse linear systems In this chapter we revisit the problem of solving linear systems of equations, but now in the context of large sparse systems. The price to pay
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationLab 1: Iterative Methods for Solving Linear Systems
Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as
More informationMath Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationCOURSE Iterative methods for solving linear systems
COURSE 0 4.3. Iterative methods for solving linear systems Because of round-off errors, direct methods become less efficient than iterative methods for large systems (>00 000 variables). An iterative scheme
More informationIterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationMath 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018
1 Linear Systems Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March, 018 Consider the system 4x y + z = 7 4x 8y + z = 1 x + y + 5z = 15. We then obtain x = 1 4 (7 + y z)
More informationIntroduction to Boundary Value Problems
Chapter 5 Introduction to Boundary Value Problems When we studied IVPs we saw that we were given the initial value of a function and a differential equation which governed its behavior for subsequent times.
More informationIntroduction to PDEs and Numerical Methods Lecture 7. Solving linear systems
Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Dr. Noemi Friedman, 09.2.205. Reminder: Instationary heat
More informationNumerical Solution Techniques in Mechanical and Aerospace Engineering
Numerical Solution Techniques in Mechanical and Aerospace Engineering Chunlei Liang LECTURE 3 Solvers of linear algebraic equations 3.1. Outline of Lecture Finite-difference method for a 2D elliptic PDE
More informationIntroduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers
Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Dr. Noemi
More informationIntroduction to Scientific Computing
(Lecture 5: Linear system of equations / Matrix Splitting) Bojana Rosić, Thilo Moshagen Institute of Scientific Computing Motivation Let us resolve the problem scheme by using Kirchhoff s laws: the algebraic
More informationBoundary Value Problems - Solving 3-D Finite-Difference problems Jacob White
Introduction to Simulation - Lecture 2 Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Outline Reminder about
More informationComputational Economics and Finance
Computational Economics and Finance Part II: Linear Equations Spring 2016 Outline Back Substitution, LU and other decomposi- Direct methods: tions Error analysis and condition numbers Iterative methods:
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationJACOBI S ITERATION METHOD
ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes
More informationNumerical Methods I Non-Square and Sparse Linear Systems
Numerical Methods I Non-Square and Sparse Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 25th, 2014 A. Donev (Courant
More informationHere is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J
Class Notes 4: THE SPECTRAL RADIUS, NORM CONVERGENCE AND SOR. Math 639d Due Date: Feb. 7 (updated: February 5, 2018) In the first part of this week s reading, we will prove Theorem 2 of the previous class.
More informationVon Neumann Analysis of Jacobi and Gauss-Seidel Iterations
Von Neumann Analysis of Jacobi and Gauss-Seidel Iterations We consider the FDA to the 1D Poisson equation on a grid x covering [0,1] with uniform spacing h, h (u +1 u + u 1 ) = f whose exact solution (to
More informationLecture 1 INF-MAT3350/ : Some Tridiagonal Matrix Problems
Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems Tom Lyche University of Oslo Norway Lecture 1 INF-MAT3350/4350 2007: Some Tridiagonal Matrix Problems p.1/33 Plan for the day 1. Notation
More informationStabilization and Acceleration of Algebraic Multigrid Method
Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration
More informationMotivation: Sparse matrices and numerical PDE's
Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting
More informationSplitting Iteration Methods for Positive Definite Linear Systems
Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System
More informationTheory of Iterative Methods
Based on Strang s Introduction to Applied Mathematics Theory of Iterative Methods The Iterative Idea To solve Ax = b, write Mx (k+1) = (M A)x (k) + b, k = 0, 1,,... Then the error e (k) x (k) x satisfies
More informationSolving Sparse Linear Systems: Iterative methods
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccs Lecture Notes for Unit VII Sparse Matrix Computations Part 2: Iterative Methods Dianne P. O Leary c 2008,2010
More informationSolving Sparse Linear Systems: Iterative methods
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 2: Iterative Methods Dianne P. O Leary
More informationPowerPoints organized by Dr. Michael R. Gustafson II, Duke University
Part 3 Chapter 10 LU Factorization PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
More informationFinite difference method for elliptic problems: I
Finite difference method for elliptic problems: I Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 560065 http://math.tifrbng.res.in/~praveen
More informationIntroduction to Iterative Solvers of Linear Systems
Introduction to Iterative Solvers of Linear Systems SFB Training Event January 2012 Prof. Dr. Andreas Frommer Typeset by Lukas Krämer, Simon-Wolfgang Mages and Rudolf Rödl 1 Classes of Matrices and their
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 11 Partial Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002.
More informationDEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
More informationx x2 2 + x3 3 x4 3. Use the divided-difference method to find a polynomial of least degree that fits the values shown: (b)
Numerical Methods - PROBLEMS. The Taylor series, about the origin, for log( + x) is x x2 2 + x3 3 x4 4 + Find an upper bound on the magnitude of the truncation error on the interval x.5 when log( + x)
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Linear Systems Fall 2015 1 / 12 Introduction We continue looking how to solve linear systems of
More informationQualifying Examination
Summer 24 Day. Monday, September 5, 24 You have three hours to complete this exam. Work all problems. Start each problem on a All problems are 2 points. Please email any electronic files in support of
More informationIterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)
Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential
More informationParallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1
Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations
More informationModelling and implementation of algorithms in applied mathematics using MPI
Modelling and implementation of algorithms in applied mathematics using MPI Lecture 3: Linear Systems: Simple Iterative Methods and their parallelization, Programming MPI G. Rapin Brazil March 2011 Outline
More informationLecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.
Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference
More informationIterative Methods for Ax=b
1 FUNDAMENTALS 1 Iterative Methods for Ax=b 1 Fundamentals consider the solution of the set of simultaneous equations Ax = b where A is a square matrix, n n and b is a right hand vector. We write the iterative
More informationNum. discretization. Numerical simulations. Finite elements Finite volume. Finite diff. Mathematical model A set of ODEs/PDEs. Integral equations
Scientific Computing Computer simulations Road map p. 1/46 Numerical simulations Physical phenomenon Mathematical model A set of ODEs/PDEs Integral equations Something else Num. discretization Finite diff.
More information7.2 Steepest Descent and Preconditioning
7.2 Steepest Descent and Preconditioning Descent methods are a broad class of iterative methods for finding solutions of the linear system Ax = b for symmetric positive definite matrix A R n n. Consider
More informationChapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs
Chapter Two: Numerical Methods for Elliptic PDEs Finite Difference Methods for Elliptic PDEs.. Finite difference scheme. We consider a simple example u := subject to Dirichlet boundary conditions ( ) u
More informationJordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS
Jordan Journal of Mathematics and Statistics JJMS) 53), 2012, pp.169-184 A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS ADEL H. AL-RABTAH Abstract. The Jacobi and Gauss-Seidel iterative
More informationLecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo. Department of Mathematics Iowa State University
Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department
More informationLecture 8: Fast Linear Solvers (Part 7)
Lecture 8: Fast Linear Solvers (Part 7) 1 Modified Gram-Schmidt Process with Reorthogonalization Test Reorthogonalization If Av k 2 + δ v k+1 2 = Av k 2 to working precision. δ = 10 3 2 Householder Arnoldi
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More information30.5. Iterative Methods for Systems of Equations. Introduction. Prerequisites. Learning Outcomes
Iterative Methods for Systems of Equations 0.5 Introduction There are occasions when direct methods (like Gaussian elimination or the use of an LU decomposition) are not the best way to solve a system
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More informationIterative Methods for Linear Systems
Iterative Methods for Linear Systems 1. Introduction: Direct solvers versus iterative solvers In many applications we have to solve a linear system Ax = b with A R n n and b R n given. If n is large the
More informationToday s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
More informationCache Oblivious Stencil Computations
Cache Oblivious Stencil Computations S. HUNOLD J. L. TRÄFF F. VERSACI Lectures on High Performance Computing 13 April 2015 F. Versaci (TU Wien) Cache Oblivious Stencil Computations 13 April 2015 1 / 19
More informationClassical iterative methods for linear systems
Classical iterative methods for linear systems Ed Bueler MATH 615 Numerical Analysis of Differential Equations 27 February 1 March, 2017 Ed Bueler (MATH 615 NADEs) Classical iterative methods for linear
More informationSolving linear systems (6 lectures)
Chapter 2 Solving linear systems (6 lectures) 2.1 Solving linear systems: LU factorization (1 lectures) Reference: [Trefethen, Bau III] Lecture 20, 21 How do you solve Ax = b? (2.1.1) In numerical linear
More informationReview of matrices. Let m, n IN. A rectangle of numbers written like A =
Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an
More informationScientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008
More informationIllustration of Gaussian elimination to find LU factorization. A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44
Illustration of Gaussian elimination to find LU factorization. A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a 1 Compute multipliers : Eliminate entries in first column: m i1 = a i1 a 11, i = 2, 3, 4 ith
More information