Iterative Methods and Multigrid
|
|
- Aileen Jennings
- 5 years ago
- Views:
Transcription
1 Iterative Methods and Multigrid Part 1: Introduction to Multigrid 2000 Eric de Sturler 1 12/02/09 MG01.prz
2 Basic Iterative Methods (1) Nonlinear equation: f(x) = 0 Rewrite as x = F(x), and iterate x i+1 = F(x i ) (fixed-point iteration) Converges locally if F (x) < 1 in neighborhood of solution Linear system: Ax = b Matrix splitting: A = M N (M N)x = b w Mx= Nx+ b w x = M 1 Nx+ M 1 b Iterate: x k+1 =(I P 1 A)x k + P 1 b or Converges iff (if and only if) (M 1 N)<1 Sol ve Mx k+1 = Nx k + b Methods: Jacobi iteration, Gauss-Seidel, (S)SOR,... Fixed-point: x = M 1 Nx+ M 1 b w M 1 Ax = M 1 b Fixed-point is solution of the preconditioned system: M 1 Ax = M 1 b 2 12/02/09 MG01.prz
3 Basic Iterative Methods (2) x k+1 = M 1 Nx k + M 1 b = M 1 (M A)x k + M 1 b = x k + M 1 b M 1 Ax k Linear System: Ax = b Residual: r k = b Ax k Prec. system: M 1 Ax = M 1 b Prec. residual: r k = M 1 b M 1 Ax k x k+1 = x k + r k u x k+1 = x 0 + r 0 + r r k Update: x k+1 x 0 = r 0 + r r k r k+1 = M 1 b M 1 Ax k+1 = M 1 b M 1 Ax k M 1 Ar k = r k M 1 Ar k r k+1 = (I M 1 A)r k = (M 1 N) k+1 r /02/09 MG01.prz
4 Basic Iterative Methods (3) Solution to : Error: Ax = bxˆ k = xˆ x k Residual and error: r k = b Ax k = Axˆ Ax k = A k ( r k = M 1 A k ) Theorem: xˆ is a fixed point of x k+1 = M 1 Nx k + M 1 biff xˆ is the solution of M 1 Ax = M 1 b (g Ax = b) Proof: x =(I M 1 A)x+ M 1 b = x M 1 Ax+ M 1 b M 1 Ax = M 1 b w k+1 = xˆ x k+1 = M 1 Nxˆ + M 1 b M 1 Nx k M 1 b = M 1 N k k+1 = M 1 N k = (M 1 N) k+1 0 and r k+1 = (M 1 N) k+1 r /02/09 MG01.prz
5 Matrix Splittings (1) Some well-known choices for M: A 11 A 1p A = A p1 A pp M = diag(a 11, A 22,, A pp ) (point/block) Jacobi M = A 11 A p1 A pp (point/block) Gauss-Seidel M = 1 A 11 A p1 1 A pp (point/block) SOR x SOR k+1 = x GS SOR k+1 + (1 )x k 5 12/02/09 MG01.prz
6 Review Basic Iterative Methods Solving Au = f Write A = D L U, where U is the strict upper triangular part, L is the strict lower triangular part, and D is the diagonal. Jacobi iteration: Du k+1 = (L + U)u k + f Gauss-Seidel iteration: (D L )u k+1 = Uu k + f Block versions possible. Basic iteration: u k+1 = P 1 (P A)u k + P 1 f = R x u k + P 1 f Convergence iff and asymptotic convergence rate:. (R x ) < 1 (R x ) R J = D 1 (L + U) R GS = (D L ) 1 U 6 12/02/09 MG01.prz
7 Use of Relaxation Parameter We can use a relaxation parameter to improve convergence: Jacobi iteration: ũ k = D 1 (L + U)u k + D 1 f u k+1 = (1 )u k + ũ k Alternatively: u k+1 = u k + r k with r k = D 1 (f Au k ) R J, = (1 )I + R J Gauss-Seidel iteration: ũ k = (D L ) 1 Uu k + (D L ) 1 f u k+1 = (1 )u k + ũ k Alternatively: u k+1 = u k + r k with r k = (D L ) 1 (f Au k ) R GS, = (1 )I + R GS 7 12/02/09 MG01.prz
8 Model Problem Consider the following model problem, u xx = f for xc (0, 1) with Dirichlet boundary conditions u = 0 for x = 0 and x = 1. We discretize the problem (see Meurant 1.9) using n + 1 grid points. This gives h = n 1 and x j = jh for j = 0 n. We use the standard central finite difference discretization u xx = u i 1 2u i +u i+1. h + O(h 2 ) This gives the matrix A = tridiag This gives eigenvectors u (k) j = sin jk n and eigenvalues k h ( k) = 4sin 2 ( k. (see Meurant 1.10) 2n ) 8 12/02/09 MG01.prz
9 Model Problem (Jacobi) Let s consider pointwise algorithm for model problem, for, and u xx + u = f 0 < x < 1 u(0) = u(1) = 0 Equidistant grid points: and. Initially. x i = ih u i = u(x i ) = 0 Centered finite differences: where. u i 1 + 2u i u i+1 = h 2 f i i = 1 n 1 Jacobi iteration: u ( k+1) = D 1 (L + U)u ( k) + D 1 f Update at each point: u (k+1) i = 1 2 h 2 f i + u (k) (k) i 1 + u i+1 = u i (k) + 1 2r i (k) Update can be done for all points at once (no dependencies). With relaxation parameter : u (k+1) i = u (k) i + 1 (k) 2 r i 9 12/02/09 MG01.prz
10 Model problem (Gauss-Seidel) Let s consider pointwise algorithm for model problem, for, and u xx + u = f 0 < x < 1 u(0) = u(1) = 0 Equidistant grid points: x i = ih and u i = u(x i ). Centered finite differences: u i 1 + 2u i u i+1 = h 2 f i where i = 1 n 1. Gauss-Seidel iteration: u ( k+1) = (D L ) 1 Uu ( k) + (D L ) 1 f Update at each point: u (k+1) i = 1 2 h 2 f i + u (k+1) (k) i 1 + u i+1 = u i (k) + 1 2r i (k) where r (k) i h h 2 f i + u (k+1) i 1 + u (k) (k) i+1 2u i : the residual evaluated after updating u i 1 d u i 1 (k+1) With relaxation parameter : u (k+1) i = u (k) i + 1 (k) 2 r i 10 12/02/09 MG01.prz
11 Point Jacobi for Model Problem Now consider the rate of convergence of the point Jacobi iteration. We take,, and. M = 2I N = M A = 2I A M 1 N = I 1 2 A The iteration matrix, G h M 1 N, has the same eigenvectors as A. For its eigenvalues we get G = A. We have A,k = 4sin 2 (k 2n ) = 4sin 2 (k n ), and so G,k = 1 2sin 2 (k ) 2n = 1 2sin 2 (k n ) For we get k = 1 G,1 = 1 2sin 2 ( n ) l 1 2 n 2 = 1 2 2n 2 Likewise, we get G,n 1 = 1 2sin 2 ((n 1) n ) l n 2 Clearly, for, (and fairly rapidly so) n d (G) d /02/09 MG01.prz
12 Error Smoothing Convergence: e ( k) = G k e ( 0), where G is iteration matrix G = (I P 1 A). u ( k+1) = u ( k) + P 1 r ( k) = u ( k) + P 1 (f Au ( k) ) = (I P 1 A)u ( k) + P 1 f Jacobi iteration matrix: (1 )I + R J = (1 )I + D 1 (D A) This gives: I 2 A Eigenvalues : I 2 A (I 2A) = 1 2 (A) Eigenvalues of A 12 12/02/09 MG01.prz
13 Error Smoothing Eigenvalues of : A Assume eigenvector close to physical eigenvector: v k = sin kj n Apply pointwise rule for Jacobi: Av j,k = sin k (j 1) n + 2sin kj n sin k (j+1) n = 2sin kj n 2sin kj n (1 cos k n ) (so eigenvector indeed) 2sin kj n cos k n = 1 cos k n = 1 cos 2 k 2n = 1 (1 2sin 2 ( k 2n )) = 2sin 2 ( k 2n ) 2sin kj n (1 cos k n ) = sin kj n * 4sin 2 ( k 2n ) This gives for the weighted Jacobi method: I 2A (RJ, where ) = 1 2 sin 2 ( k 2n ) 0 < [ 1 Always converges. Poor convergence for which modes? 13 12/02/09 MG01.prz
14 Point Gauss-Seidel for Model Problem Next we consider the rate of convergence of the point Gauss-Seidel method (in natural ordering) for the model problem. We solve Ax = b. Let A = L + D + U, where D is diagonal of A, and L and U are strictly lower and upper triangular part of A. Take M = L + D and N = U. Iterate Mx k+1 = Nx k + b. Eigenvectors of M 1 N? M 1 Nw = w g Nw = Mw Model problem: with. 2 w j w j 1 = w j+1 w 0 = w n = 0 Recurrence w j+1 2 w j + w j 1 = 0. Substitute w j = z j ; this gives z 2 2 z + = 0 d z 1,2 =! 2. Real gives either strictly increasing or decreasing (or constant, i.e. zero). So we need z complex, and therefore 0 < < /02/09 MG01.prz
15 Point Gauss-Seidel for Model Problem j j General solution w j = 1 z z 2, where z 1 = + i 2 and z 2 = i 2 Note z 1 = 1/2 d z 1,2 = 1/2 1/2! i 1/2 2 = 1/2 e i. This gives 1/2 = cos w 0 = = 0 d 2 = 1 w n = 1 cos n e in 1 cos n e in = 2i 1 cos n sin n = 0 So we must have = k = k n and we can take w k,j = cos k n sin jk n = cos k sin j k (why?) For the eigenvalue we therefore get k = cos 2 k for k = 1 n 1. This gives = 1 = cos 2 n = 1 sin 2 n l 1 2. n How does this relate to the convergence rate for 2 Jacobi iteration? 15 12/02/09 MG01.prz
16 Red-Black ordering for Gauss-Seidel Gauss-Seidel depends on ordering. Often-used special ordering: Red-Black ordering, where red unknowns reference only black unknowns and vice versa. In this case, order even grid points first and then odd grid points Red/Even update: where r (k) 2i h h 2 (k) f 2i + u 2i 1 u (k+1) 2i = 1 2 h2 (k) f 2i + u 2i 1 (k) + u 2i+1 (k) 2u 2i (k) + u 2i+1 With relaxation parameter : u (k+1) 2i = u (k) 2i r (k) 2i = u (k) 2i r (k) 2i Black/Odd update: u (k+1) 2i 1 = 1 2 h2 f 2i 1 + u (k+1) (k+1) 2i 2 + u 2i+2 where r (k+1) 2i 1 h h 2 f 2i 1 + u (k+1) 2i 2 + u (k+1) (k) 2i+2 2u 2i 1 (k) = u 2i r (k+1) 2i 1 With relaxation parameter : u (k+1) (k) 2i 1 = u 2i r (k+1) 2i /02/09 MG01.prz
17 Error Smoothing Initial Solution=-Error DCT: [4.9, 0.27, 0.16, 0.061, -2.4,...], O(0.1) or O(0.2) 17 12/02/09 MG01.prz
18 Error Smoothing 0.8 Solution=-Error After 20 Jacobi iterations with = /02/09 MG01.prz
19 Error Smoothing 0.7 Solution=-Error After 50 Jacobi iterations with = /02/09 MG01.prz
20 Error Smoothing 0.45 Solution=-Error After 1000 Jacobi iterations with =1 DCT: [2.4,, -1.1,,...,, 0.35], =O(0.01), mainly O(1e-3) 20 12/02/09 MG01.prz
21 Error Smoothing ω = 2/ /02/09 MG01.prz
22 Error Smoothing /02/09 MG01.prz
23 Error Smoothing /02/09 MG01.prz
24 Error Smoothing /02/09 MG01.prz
9. Iterative Methods for Large Linear Systems
EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1
More informationTheory of Iterative Methods
Based on Strang s Introduction to Applied Mathematics Theory of Iterative Methods The Iterative Idea To solve Ax = b, write Mx (k+1) = (M A)x (k) + b, k = 0, 1,,... Then the error e (k) x (k) x satisfies
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 1: Introduction to Multigrid 1 12/02/09 MG02.prz Error Smoothing 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Initial Solution=-Error 0 10 20 30 40 50 60 70 80 90 100 DCT:
More informationCOURSE Iterative methods for solving linear systems
COURSE 0 4.3. Iterative methods for solving linear systems Because of round-off errors, direct methods become less efficient than iterative methods for large systems (>00 000 variables). An iterative scheme
More informationKasetsart University Workshop. Multigrid methods: An introduction
Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available
More informationLecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More informationIterative Methods and Multigrid
Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be
More informationMath 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018
1 Linear Systems Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March, 018 Consider the system 4x y + z = 7 4x 8y + z = 1 x + y + 5z = 15. We then obtain x = 1 4 (7 + y z)
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More information9.1 Preconditioned Krylov Subspace Methods
Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete
More information6. Iterative Methods for Linear Systems. The stepwise approach to the solution...
6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse
More informationStabilization and Acceleration of Algebraic Multigrid Method
Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration
More informationSplitting Iteration Methods for Positive Definite Linear Systems
Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System
More informationPoisson Equation in 2D
A Parallel Strategy Department of Mathematics and Statistics McMaster University March 31, 2010 Outline Introduction 1 Introduction Motivation Discretization Iterative Methods 2 Additive Schwarz Method
More informationJACOBI S ITERATION METHOD
ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationParallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1
Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 24: Preconditioning and Multigrid Solver Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 5 Preconditioning Motivation:
More informationBackground. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms
More informationIterative techniques in matrix algebra
Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and
More informationCHAPTER 5. Basic Iterative Methods
Basic Iterative Methods CHAPTER 5 Solve Ax = f where A is large and sparse (and nonsingular. Let A be split as A = M N in which M is nonsingular, and solving systems of the form Mz = r is much easier than
More informationNotes for CS542G (Iterative Solvers for Linear Systems)
Notes for CS542G (Iterative Solvers for Linear Systems) Robert Bridson November 20, 2007 1 The Basics We re now looking at efficient ways to solve the linear system of equations Ax = b where in this course,
More informationMonte Carlo simulation inspired by computational optimization. Colin Fox Al Parker, John Bardsley MCQMC Feb 2012, Sydney
Monte Carlo simulation inspired by computational optimization Colin Fox fox@physics.otago.ac.nz Al Parker, John Bardsley MCQMC Feb 2012, Sydney Sampling from π(x) Consider : x is high-dimensional (10 4
More information1. Fast Iterative Solvers of SLE
1. Fast Iterative Solvers of crucial drawback of solvers discussed so far: they become slower if we discretize more accurate! now: look for possible remedies relaxation: explicit application of the multigrid
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Iteration basics Notes for 2016-11-07 An iterative solver for Ax = b is produces a sequence of approximations x (k) x. We always stop after finitely many steps, based on some convergence criterion, e.g.
More informationLecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University
Lecture Note 7: Iterative methods for solving linear systems Xiaoqun Zhang Shanghai Jiao Tong University Last updated: December 24, 2014 1.1 Review on linear algebra Norms of vectors and matrices vector
More informationSyDe312 (Winter 2005) Unit 1 - Solutions (continued)
SyDe3 (Winter 5) Unit - Solutions (continued) March, 5 Chapter 6 - Linear Systems Problem 6.6 - b Iterative solution by the Jacobi and Gauss-Seidel iteration methods: Given: b = [ 77] T, x = [ ] T 9x +
More informationMath 1080: Numerical Linear Algebra Chapter 4, Iterative Methods
Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 March 2015 1 / 70 Topics Introduction to Iterative Methods
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD
More informationIntroduction. Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods. Example: First Order Richardson. Strategy
Introduction Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 Solve system Ax = b by repeatedly computing
More informationCAAM 454/554: Stationary Iterative Methods
CAAM 454/554: Stationary Iterative Methods Yin Zhang (draft) CAAM, Rice University, Houston, TX 77005 2007, Revised 2010 Abstract Stationary iterative methods for solving systems of linear equations are
More informationHere is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J
Class Notes 4: THE SPECTRAL RADIUS, NORM CONVERGENCE AND SOR. Math 639d Due Date: Feb. 7 (updated: February 5, 2018) In the first part of this week s reading, we will prove Theorem 2 of the previous class.
More informationAlgebraic Multigrid as Solvers and as Preconditioner
Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven
More informationIterative Methods for Ax=b
1 FUNDAMENTALS 1 Iterative Methods for Ax=b 1 Fundamentals consider the solution of the set of simultaneous equations Ax = b where A is a square matrix, n n and b is a right hand vector. We write the iterative
More informationNumerical Programming I (for CSE)
Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let
More informationMultigrid Methods and their application in CFD
Multigrid Methods and their application in CFD Michael Wurst TU München 16.06.2009 1 Multigrid Methods Definition Multigrid (MG) methods in numerical analysis are a group of algorithms for solving differential
More informationIterative Methods for Sparse Linear Systems
Iterative Methods for Sparse Linear Systems Luca Bergamaschi e-mail: berga@dmsa.unipd.it - http://www.dmsa.unipd.it/ berga Department of Mathematical Methods and Models for Scientific Applications University
More informationScientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1
Scientific Computing WS 2018/2019 Lecture 9 Jürgen Fuhrmann juergen.fuhrmann@wias-berlin.de Lecture 9 Slide 1 Lecture 9 Slide 2 Simple iteration with preconditioning Idea: Aû = b iterative scheme û = û
More informationNumerical Analysis of Differential Equations Numerical Solution of Elliptic Boundary Value
Numerical Analysis of Differential Equations 188 5 Numerical Solution of Elliptic Boundary Value Problems 5 Numerical Solution of Elliptic Boundary Value Problems TU Bergakademie Freiberg, SS 2012 Numerical
More informationIterative Methods. Splitting Methods
Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition
More informationLecture 8: Fast Linear Solvers (Part 7)
Lecture 8: Fast Linear Solvers (Part 7) 1 Modified Gram-Schmidt Process with Reorthogonalization Test Reorthogonalization If Av k 2 + δ v k+1 2 = Av k 2 to working precision. δ = 10 3 2 Householder Arnoldi
More informationA Comparison of Solving the Poisson Equation Using Several Numerical Methods in Matlab and Octave on the Cluster maya
A Comparison of Solving the Poisson Equation Using Several Numerical Methods in Matlab and Octave on the Cluster maya Sarah Swatski, Samuel Khuvis, and Matthias K. Gobbert (gobbert@umbc.edu) Department
More informationSome definitions. Math 1080: Numerical Linear Algebra Chapter 5, Solving Ax = b by Optimization. A-inner product. Important facts
Some definitions Math 1080: Numerical Linear Algebra Chapter 5, Solving Ax = b by Optimization M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 A matrix A is SPD (Symmetric
More informationHence a root lies between 1 and 2. Since f a is negative and f(x 0 ) is positive The root lies between a and x 0 i.e. 1 and 1.
The Bisection method or BOLZANO s method or Interval halving method: Find the positive root of x 3 x = 1 correct to four decimal places by bisection method Let f x = x 3 x 1 Here f 0 = 1 = ve, f 1 = ve,
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationSolving Ax = b, an overview. Program
Numerical Linear Algebra Improving iterative solvers: preconditioning, deflation, numerical software and parallelisation Gerard Sleijpen and Martin van Gijzen November 29, 27 Solving Ax = b, an overview
More informationA MULTIGRID ALGORITHM FOR. Richard E. Ewing and Jian Shen. Institute for Scientic Computation. Texas A&M University. College Station, Texas SUMMARY
A MULTIGRID ALGORITHM FOR THE CELL-CENTERED FINITE DIFFERENCE SCHEME Richard E. Ewing and Jian Shen Institute for Scientic Computation Texas A&M University College Station, Texas SUMMARY In this article,
More informationSOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA
1 SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 2 OUTLINE Sparse matrix storage format Basic factorization
More informationLecture 4 Basic Iterative Methods I
March 26, 2018 Lecture 4 Basic Iterative Methods I A The Power Method Let A be an n n with eigenvalues λ 1,...,λ n counted according to multiplicity. We assume the eigenvalues to be ordered in absolute
More informationNotes on Some Methods for Solving Linear Systems
Notes on Some Methods for Solving Linear Systems Dianne P. O Leary, 1983 and 1999 and 2007 September 25, 2007 When the matrix A is symmetric and positive definite, we have a whole new class of algorithms
More informationNumerical Solutions to Partial Differential Equations
Numerical Solutions to Partial Differential Equations Zhiping Li LMAM and School of Mathematical Sciences Peking University A Model Problem in a 2D Box Region Let us consider a model problem of parabolic
More informationCLASSICAL ITERATIVE METHODS
CLASSICAL ITERATIVE METHODS LONG CHEN In this notes we discuss classic iterative methods on solving the linear operator equation (1) Au = f, posed on a finite dimensional Hilbert space V = R N equipped
More informationJordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS
Jordan Journal of Mathematics and Statistics JJMS) 53), 2012, pp.169-184 A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS ADEL H. AL-RABTAH Abstract. The Jacobi and Gauss-Seidel iterative
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline
More informationReview of matrices. Let m, n IN. A rectangle of numbers written like A =
Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an
More informationUp to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas
Finding Eigenvalues Up to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas plus the facts that det{a} = λ λ λ n, Tr{A}
More informationFEM and Sparse Linear System Solving
FEM & sparse system solving, Lecture 7, Nov 3, 2017 1/46 Lecture 7, Nov 3, 2015: Introduction to Iterative Solvers: Stationary Methods http://people.inf.ethz.ch/arbenz/fem16 Peter Arbenz Computer Science
More informationMath Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationNumerical Analysis Comprehensive Exam Questions
Numerical Analysis Comprehensive Exam Questions 1. Let f(x) = (x α) m g(x) where m is an integer and g(x) C (R), g(α). Write down the Newton s method for finding the root α of f(x), and study the order
More informationIntroduction and Stationary Iterative Methods
Introduction and C. T. Kelley NC State University tim kelley@ncsu.edu Research Supported by NSF, DOE, ARO, USACE DTU ITMAN, 2011 Outline Notation and Preliminaries General References What you Should Know
More informationChapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs
Chapter Two: Numerical Methods for Elliptic PDEs Finite Difference Methods for Elliptic PDEs.. Finite difference scheme. We consider a simple example u := subject to Dirichlet boundary conditions ( ) u
More informationComputational Methods. Systems of Linear Equations
Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations
More informationLecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo. Department of Mathematics Iowa State University
Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department
More informationThe convergence of stationary iterations with indefinite splitting
The convergence of stationary iterations with indefinite splitting Michael C. Ferris Joint work with: Tom Rutherford and Andy Wathen University of Wisconsin, Madison 6th International Conference on Complementarity
More informationSolving Linear Systems of Equations
November 6, 2013 Introduction The type of problems that we have to solve are: Solve the system: A x = B, where a 11 a 1N a 12 a 2N A =.. a 1N a NN x = x 1 x 2. x N B = b 1 b 2. b N To find A 1 (inverse
More informationVon Neumann Analysis of Jacobi and Gauss-Seidel Iterations
Von Neumann Analysis of Jacobi and Gauss-Seidel Iterations We consider the FDA to the 1D Poisson equation on a grid x covering [0,1] with uniform spacing h, h (u +1 u + u 1 ) = f whose exact solution (to
More informationIterative Solution methods
p. 1/28 TDB NLA Parallel Algorithms for Scientific Computing Iterative Solution methods p. 2/28 TDB NLA Parallel Algorithms for Scientific Computing Basic Iterative Solution methods The ideas to use iterative
More informationClassical iterative methods for linear systems
Classical iterative methods for linear systems Ed Bueler MATH 615 Numerical Analysis of Differential Equations 27 February 1 March, 2017 Ed Bueler (MATH 615 NADEs) Classical iterative methods for linear
More informationNUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP
ANALELE ŞTIINŢIFICE ALE UNIVERSITĂŢII AL.I. CUZA DIN IAŞI (S.N. MATEMATICĂ, Tomul LIII, 2007, f.1 NUMERICAL ALGORITHMS FOR A SECOND ORDER ELLIPTIC BVP BY GINA DURA and RĂZVAN ŞTEFĂNESCU Abstract. The aim
More informationMultigrid absolute value preconditioning
Multigrid absolute value preconditioning Eugene Vecharynski 1 Andrew Knyazev 2 (speaker) 1 Department of Computer Science and Engineering University of Minnesota 2 Department of Mathematical and Statistical
More informationMath/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018
Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x
More informationAspects of Multigrid
Aspects of Multigrid Kees Oosterlee 1,2 1 Delft University of Technology, Delft. 2 CWI, Center for Mathematics and Computer Science, Amsterdam, SIAM Chapter Workshop Day, May 30th 2018 C.W.Oosterlee (CWI)
More informationPolynomial accelerated MCMC... and other sampling algorithms inspired by computational optimization
Polynomial accelerated MCMC... and other sampling algorithms inspired by computational optimization Colin Fox fox@physics.otago.ac.nz Al Parker, John Bardsley Fore-words Motivation: an inverse oceanography
More informationGoal: to construct some general-purpose algorithms for solving systems of linear Equations
Chapter IV Solving Systems of Linear Equations Goal: to construct some general-purpose algorithms for solving systems of linear Equations 4.6 Solution of Equations by Iterative Methods 4.6 Solution of
More informationNext topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
More information3D Space Charge Routines: The Software Package MOEVE and FFT Compared
3D Space Charge Routines: The Software Package MOEVE and FFT Compared Gisela Pöplau DESY, Hamburg, December 4, 2007 Overview Algorithms for 3D space charge calculations Properties of FFT and iterative
More informationGeneralized AOR Method for Solving System of Linear Equations. Davod Khojasteh Salkuyeh. Department of Mathematics, University of Mohaghegh Ardabili,
Australian Journal of Basic and Applied Sciences, 5(3): 35-358, 20 ISSN 99-878 Generalized AOR Method for Solving Syste of Linear Equations Davod Khojasteh Salkuyeh Departent of Matheatics, University
More informationScientific Computing II
Technische Universität München ST 008 Institut für Informatik Dr. Miriam Mehl Scientific Computing II Final Exam, July, 008 Iterative Solvers (3 pts + 4 extra pts, 60 min) a) Steepest Descent and Conjugate
More informationSolving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners
Solving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners Eugene Vecharynski 1 Andrew Knyazev 2 1 Department of Computer Science and Engineering University of Minnesota 2 Department
More information2.29 Numerical Fluid Mechanics Spring 2015 Lecture 9
Spring 2015 Lecture 9 REVIEW Lecture 8: Direct Methods for solving (linear) algebraic equations Gauss Elimination LU decomposition/factorization Error Analysis for Linear Systems and Condition Numbers
More informationAIMS Exercise Set # 1
AIMS Exercise Set #. Determine the form of the single precision floating point arithmetic used in the computers at AIMS. What is the largest number that can be accurately represented? What is the smallest
More information10.2 ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS. The Jacobi Method
54 CHAPTER 10 NUMERICAL METHODS 10. ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS As a numerical technique, Gaussian elimination is rather unusual because it is direct. That is, a solution is obtained after
More informationCME342 Parallel Methods in Numerical Analysis. Matrix Computation: Iterative Methods II. Sparse Matrix-vector Multiplication.
CME342 Parallel Methods in Numerical Analysis Matrix Computation: Iterative Methods II Outline: CG & its parallelization. Sparse Matrix-vector Multiplication. 1 Basic iterative methods: Ax = b r = b Ax
More informationChapter 10 Exercises
Chapter 10 Exercises From: Finite Difference Methods for Ordinary and Partial Differential Equations by R. J. LeVeque, SIAM, 2007. http://www.amath.washington.edu/ rl/fdmbook Exercise 10.1 (One-sided and
More informationEXAMPLES OF CLASSICAL ITERATIVE METHODS
EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic
More informationMaster Thesis Literature Study Presentation
Master Thesis Literature Study Presentation Delft University of Technology The Faculty of Electrical Engineering, Mathematics and Computer Science January 29, 2010 Plaxis Introduction Plaxis Finite Element
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 207 Philippe B. Laval (KSU) Linear Systems Fall 207 / 2 Introduction We continue looking how to solve linear systems of the
More informationIntroduction to Multigrid Method
Introduction to Multigrid Metod Presented by: Bogojeska Jasmina /08/005 JASS, 005, St. Petersburg 1 Te ultimate upsot of MLAT Te amount of computational work sould be proportional to te amount of real
More informationComputational Economics and Finance
Computational Economics and Finance Part II: Linear Equations Spring 2016 Outline Back Substitution, LU and other decomposi- Direct methods: tions Error analysis and condition numbers Iterative methods:
More informationLecture 11. Fast Linear Solvers: Iterative Methods. J. Chaudhry. Department of Mathematics and Statistics University of New Mexico
Lecture 11 Fast Linear Solvers: Iterative Methods J. Chaudhry Department of Mathematics and Statistics University of New Mexico J. Chaudhry (UNM) Math/CS 375 1 / 23 Summary: Complexity of Linear Solves
More information7.3 The Jacobi and Gauss-Siedel Iterative Techniques. Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP.
7.3 The Jacobi and Gauss-Siedel Iterative Techniques Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP. 7.3 The Jacobi and Gauss-Siedel Iterative Techniques
More informationOUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU
Preconditioning Techniques for Solving Large Sparse Linear Systems Arnold Reusken Institut für Geometrie und Praktische Mathematik RWTH-Aachen OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative
More informationComputation Fluid Dynamics
Computation Fluid Dynamics CFD I Jitesh Gajjar Maths Dept Manchester University Computation Fluid Dynamics p.1/189 Garbage In, Garbage Out We will begin with a discussion of errors. Useful to understand
More informationMultigrid finite element methods on semi-structured triangular grids
XXI Congreso de Ecuaciones Diferenciales y Aplicaciones XI Congreso de Matemática Aplicada Ciudad Real, -5 septiembre 009 (pp. 8) Multigrid finite element methods on semi-structured triangular grids F.J.
More informationCourse Notes: Week 1
Course Notes: Week 1 Math 270C: Applied Numerical Linear Algebra 1 Lecture 1: Introduction (3/28/11) We will focus on iterative methods for solving linear systems of equations (and some discussion of eigenvalues
More informationDepartment of Computer Science, University of Illinois at Urbana-Champaign
Department of Computer Science, University of Illinois at Urbana-Champaign Probing for Schur Complements and Preconditioning Generalized Saddle-Point Problems Eric de Sturler, sturler@cs.uiuc.edu, http://www-faculty.cs.uiuc.edu/~sturler
More informationIntroduction to Scientific Computing II Multigrid
Introduction to Scientific Computing II Multigrid Miriam Mehl Slide 5: Relaxation Methods Properties convergence depends on method clear, see exercises and 3), frequency of the error remember eigenvectors
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 11 Partial Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002.
More information