Partitioned Methods for Multifield Problems

Size: px
Start display at page:

Download "Partitioned Methods for Multifield Problems"

Transcription

1 Partitioned Methods for Multifield Problems Joachim Rang, Joachim Rang Partitioned Methods for Multifield Problems Seite 1

2 Contents Blockform of linear iteration schemes Examples Joachim Rang Partitioned Methods for Multifield Problems Seite 2

3 Contents Blockform of linear iteration schemes Examples Joachim Rang Partitioned Methods for Multifield Problems Seite 3

4 Motivating example: Heat equation Consider the one-dimensional heat equation with Dirichlet boundary conditions u u = exp( t)(x 2 + 2) Exact solution: u(t) = 1 + exp( t)x 2 space discretisation: central differences time discretisation: fully implicit Runge Kutta method with s internal stages Aim: Compute one timestep Joachim Rang Partitioned Methods for Multifield Problems Seite 4

5 Introduction Coupled linear system: A A 1n.. A n1... A nn x 1. x n = b 1. b n, where A ij R n i,n j and b i, x i R n i. Let x = (x 1,..., x n ) and b = (b 1,..., b n ). Moreover A is the matrix containing of the blocks A ij. Let the matrix A be regular Joachim Rang Partitioned Methods for Multifield Problems Seite 5

6 Introduction monolythical approach: Solve Ax = b by a direct or an iterative method. Partitioned method: Solve A ii x i = b n A ij x j j=0 j i is solved w.r.t. x i. Since the other components of x are not known in general, an iterative loop is needed to get an accurate numerical solution Joachim Rang Partitioned Methods for Multifield Problems Seite 6

7 Example 2 2-block system: A 11 x 1 + A 12 x 2 = b 1 (1) A 21 x 1 + A 22 x 2 = b 2. In the first step we solve (1) w.r.t. x 1, i. e. x 1 = A 1 11 (b 1 A 12 x 2 ). Next we communicate the solution x 1 from software 1 to software 2. Then we can solve equation (1) w.r.t. x 2 and get x 2 = (A 22 A 21 A 1 11 A 12) 1 (b 2 A 21 A 1 11 b 1). (2) Joachim Rang Partitioned Methods for Multifield Problems Seite 7

8 Block Jacobi method 1. Set ν := 0 and choose an initial guess x (0) = (x (0) 2. Solve (1) w.r.t. x 1, i. e. 3. Solve (2) w.r.t. x 2, i. e. 4. Set ν := ν Compute the residuum A 11 x (ν+1) 1 = b 1 A 12 x (ν) 2. A 22 x (ν+1) 2 = b 2 A 21 x (ν) 1. r (ν) := Ax (v) b. 1, x(0) 2 ). 6. If r (ν) is sufficiently small, then stop. Otherwise compute x (v+1). This method is called Block Jacobi method or simultaneous iteration method Joachim Rang Partitioned Methods for Multifield Problems Seite 8

9 Block-Gauß Seidel method 1. Set ν := 0 and choose an initial guess x (0) = (x (0) 2. Solve (1) w.r.t. x 1, i. e. 3. Solve (2) w.r.t. x 2, i. e. 4. Set ν := ν Compute the residuum A 11 x (ν+1) 1 = b 1 A 12 x (ν) 2. A 22 x (ν+1) 2 = b 2 A 21 x (ν+1) 1. r (ν) := Ax (v) b. 1, x(0) 2 ). 6. If r (ν) is sufficiently small, then stop. Otherwise compute x (v+1). This method is called Block Gauß Seidel method or sucessive iteration method Joachim Rang Partitioned Methods for Multifield Problems Seite 9

10 Example Let ( 2 1 A 11 = 1 2 ( 1 1 A 21 = 1 1 ) ( 10 1, A 12 = ), A 22 = ( ) ) and let b 1 = ( ) ( 13, b 2 = 14 ) Joachim Rang Partitioned Methods for Multifield Problems Seite 10

11 Result Joachim Rang Partitioned Methods for Multifield Problems Seite 11

12 Blockform of linear iteration methods Notation: Let A, B R n,n be given matrices. A B, if a ij b ij holds for all i, j {1,..., n}. A B, if a ij b ij holds for all i, j {1,..., n}. A is called nonnegative, if A 0, A is called positive, if A > 0, A is called monotone, if Ax 0 implies x 0. Theorem: A matrix A is monotone, if and only if A is regular with A 1 0. Proof: see Axelsson (1996) Joachim Rang Partitioned Methods for Multifield Problems Seite 12

13 Definition A matrix A with A = (a ij ) is called a (nonsingular) M-Matrix, if a ij 0 for i j and if it is monotone, i. e. A 1 0. By M(A) = (m ij (A)) we denote the comparison matrix given by { aij, if i = j m ij (A) := a ij, if i j. If M(A) is a M-Matrix, then A is called H-matrix Joachim Rang Partitioned Methods for Multifield Problems Seite 13

14 Example Let Then A = M(A) = is a M-Matrix and it follows that A is an H-matrix Joachim Rang Partitioned Methods for Multifield Problems Seite 14

15 Splittings types of splittings: Let C, R R n,n be matrices. Then the matrix A = C R is called: a regular splitting, if C is monotone and R 0 (see Varga (1963)) a weak regular splitting, if C is monotone and C 1 R 0 (see Ortega and Rheinboldt (1970)) a nonnegative splitting, if C is regular and C 1 R 0 (see Beauwens (1979) and Song (1991)) a convergent splitting, if C is regular and ρ(c 1 R) < Joachim Rang Partitioned Methods for Multifield Problems Seite 15

16 Block-iterative methods Let A be a M-matrix with a ij 0 for i j. Moreover let A be of the form D 1 A A 1n A 21 D 2... A 2n A =....., A 1n A n2... D n where D i R n i,n i. It can be shown that D i is regular and D 1 i 0. Let D A := diag(d 1,..., D n ). Then we have D 1 A Joachim Rang Partitioned Methods for Multifield Problems Seite 16

17 Theorem Let A = C R be a weak regular splitting. Then the splitting is convergent if and only if A is monotone. Proof: Axelsson (1996) Joachim Rang Partitioned Methods for Multifield Problems Seite 17

18 Block Jacobi method Splitting: R = DA A, i. e. C := D A. Then it follows C 1 = D 1 A 0 and the matrix C is monotone. Moreover we have R 0. All these conclusions show us that C R is a regular splitting. (Block) Jacobi method or simultaneous iteration method: D A x (k+1) = (D A A)x (k) + b Joachim Rang Partitioned Methods for Multifield Problems Seite 18

19 Block Gauß Seidel method Splitting: A := D L U with l ij = a ij for i < j and u ij = a ij for i > j. Let C := D L. Then we have C 1 = (D L) 1 = [ D ( I D 1 L )] 1 = ( I D 1 L ) 1 D 1 = ( D 1 L ) i D 1 0. Moreover we have U 0 and finally we have proven that A = C R with R = U is a regular splitting. This approach is called (block) Gauß Seidel method or sucessive iteration method and is given by i=0 (D L)x (k+1) = Ux (k) + b Joachim Rang Partitioned Methods for Multifield Problems Seite 19

20 Contents Blockform of linear iteration schemes Examples Joachim Rang Partitioned Methods for Multifield Problems Seite 20

21 Heat equation space discretisation: n = 20 time stepsize: τ = Joachim Rang Partitioned Methods for Multifield Problems Seite 21

22 Heat equation space discretisation: n = 50 time stepsize: τ = Joachim Rang Partitioned Methods for Multifield Problems Seite 22

23 Heat equation: spectral radius space discretisation: n = Joachim Rang Partitioned Methods for Multifield Problems Seite 23

24 Example Consider the problem { u = f, x (0, 1) u = 0, x {0, 1} Exact solution: Then: u(x) = x(1 x) (x 1/4) + 1/10 f(x) = x x x (80x 2 40x + 13) 2 Space discretisation: central differences Joachim Rang Partitioned Methods for Multifield Problems Seite 24

25 Monolythical approach discrete system: A h u h = f h n u h u CPU E E E Joachim Rang Partitioned Methods for Multifield Problems Seite 25

26 Partititioned approach (Block Gauß Seidel method) discrete system: ( A11 A 12 A 21 A 22 ) ( ) ( ) u (1) f (1) = u (2) Blockform of linear iteration schemes Examples f (2) with A 11 = A 22 = h { (A 12 ) ij = (A 21 ) 1, i = n, j = 1 ij = 0, else Joachim Rang Partitioned Methods for Multifield Problems Seite 26

27 Partititioned approach Block Gauß Seidel method: n u h u Iterations CPU ρ E E E Joachim Rang Partitioned Methods for Multifield Problems Seite 27

28 A parallel ansatz Idea: Split the domain into (0, 1/2) and (1/2, 1) At x = 1/2 we require u left (1/2) = u right (1/2), u left(1/2) = u right(1/2), Joachim Rang Partitioned Methods for Multifield Problems Seite 28

29 Time-dependent problem Consider the problem u u = f, (t, x) (0, 1) 2 u = 0, t (0, 1), x {0, 1} u = u 0 (x), x (0, 1) Space discretisation: central differences Time discretisation: backward Euler method discrete problem: (I τa)u h m+1 = u h m + τf(t m+1 ) Here: spectral radius ρ depends on τ Joachim Rang Partitioned Methods for Multifield Problems Seite 29

30 Literature Owe Axelsson. Iterative solution methods. Cambridge Univ. Press., Cambridge, Robert Beauwens. Factorization iterative methods, M-operators and H-operators. Numer. Math., 31: , Andras Meister: Numerik linearer Gleichungssysteme. Eine Einführung in moderne Verfahren. Mit MATLAB-Implementierungen von C. Vömel, Vieweg+Teubner, Wiesbaden, J.M. Ortega and W.C. Rheinboldt. Iterative solution of nonlinear equations in several variables. Academic Press, New York-London, Yongzhong Song. Comparisons of nonnegative splittings of matrices. Linear Algebra Appl., : , R.S. Varga. Matrix iterative analysis. Series in Automatic Computation. Prentice-Hall, Joachim Rang Partitioned Methods for Multifield Problems Seite 30

31 BiCGStab method Iterative methods which convergece faster than the Jacobi, the Gauss Seidel and SOR-method are cg-schemes (conjugient gradient schemes). These methods are based on optimization ideas, i.e. in each iteration step one or more search vectors are computed which minimises the residuum. In this lecture we only introduce the BiCGStab method Joachim Rang Partitioned Methods for Multifield Problems Seite 31

32 BiCGStab method (I) Let x (0) be an initial guess for the solution of the linear system Ax = b and let r (0) = Ax (0) b be the initial residuum. Set r (0) := r (0) and p (0) := r (0). Calculate ρ (0) := (r (0), r (0) ). Let ɛ be a given tolerance. If r (0) ɛ then stop Joachim Rang Partitioned Methods for Multifield Problems Seite 32

33 BiCGStab method (II) Otherwise we iterate from k = 0 to n 2 and compute s (k) = Ap (k), σ (k) = (s (k), r (0) ), stop, if σ (k) = 0. α (k) = ρ(k) σ (k), w(k) = r (k) α (k) s (k), v (k) = Aw (k), ω (k) = (v(k), w k ) (v (k), v (k) ) x (k+1) = x (k) α (k) p (k) ω (k) w (k) r (k+1) = r (k) α (k) s (k) ω (k) v (k) ρ (k+1) = (r (k+1), r (0) ) If r (k+1) ɛr (0) then stop. Otherwise compute the new search vectors β (k) = ρ(k+1) ρ (k) α (k) ω (k), p(k+1) = r (k+1) + β (k) (p (k) ω (k) s (k) ) Joachim Rang Partitioned Methods for Multifield Problems Seite 33

34 Example Consider the problem { u = f, x (0, 1) u = 0, x {0, 1} Exact solution: Then: u(x) = x(1 x) (x 1/4) + 1/10 f(x) = x x x (80x 2 40x + 13) 2 Space discretisation: central differences Joachim Rang Partitioned Methods for Multifield Problems Seite 34

35 Example Dimension: Joachim Rang Partitioned Methods for Multifield Problems Seite 35

CAAM 454/554: Stationary Iterative Methods

CAAM 454/554: Stationary Iterative Methods CAAM 454/554: Stationary Iterative Methods Yin Zhang (draft) CAAM, Rice University, Houston, TX 77005 2007, Revised 2010 Abstract Stationary iterative methods for solving systems of linear equations are

More information

Partitioned Methods for Multifield Problems

Partitioned Methods for Multifield Problems C Partitioned Methods for Multifield Problems Joachim Rang, 6.7.2016 6.7.2016 Joachim Rang Partitioned Methods for Multifield Problems Seite 1 C One-dimensional piston problem fixed wall Fluid flexible

More information

Generalized AOR Method for Solving System of Linear Equations. Davod Khojasteh Salkuyeh. Department of Mathematics, University of Mohaghegh Ardabili,

Generalized AOR Method for Solving System of Linear Equations. Davod Khojasteh Salkuyeh. Department of Mathematics, University of Mohaghegh Ardabili, Australian Journal of Basic and Applied Sciences, 5(3): 35-358, 20 ISSN 99-878 Generalized AOR Method for Solving Syste of Linear Equations Davod Khojasteh Salkuyeh Departent of Matheatics, University

More information

Lecture 18 Classical Iterative Methods

Lecture 18 Classical Iterative Methods Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,

More information

COURSE Iterative methods for solving linear systems

COURSE Iterative methods for solving linear systems COURSE 0 4.3. Iterative methods for solving linear systems Because of round-off errors, direct methods become less efficient than iterative methods for large systems (>00 000 variables). An iterative scheme

More information

Introduction to Scientific Computing

Introduction to Scientific Computing (Lecture 5: Linear system of equations / Matrix Splitting) Bojana Rosić, Thilo Moshagen Institute of Scientific Computing Motivation Let us resolve the problem scheme by using Kirchhoff s laws: the algebraic

More information

Computational Linear Algebra

Computational Linear Algebra Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD

More information

M.A. Botchev. September 5, 2014

M.A. Botchev. September 5, 2014 Rome-Moscow school of Matrix Methods and Applied Linear Algebra 2014 A short introduction to Krylov subspaces for linear systems, matrix functions and inexact Newton methods. Plan and exercises. M.A. Botchev

More information

Computers and Mathematics with Applications. Convergence analysis of the preconditioned Gauss Seidel method for H-matrices

Computers and Mathematics with Applications. Convergence analysis of the preconditioned Gauss Seidel method for H-matrices Computers Mathematics with Applications 56 (2008) 2048 2053 Contents lists available at ScienceDirect Computers Mathematics with Applications journal homepage: wwwelseviercom/locate/camwa Convergence analysis

More information

Iterative Methods for Solving A x = b

Iterative Methods for Solving A x = b Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http

More information

9.1 Preconditioned Krylov Subspace Methods

9.1 Preconditioned Krylov Subspace Methods Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete

More information

Iterative Solution methods

Iterative Solution methods p. 1/28 TDB NLA Parallel Algorithms for Scientific Computing Iterative Solution methods p. 2/28 TDB NLA Parallel Algorithms for Scientific Computing Basic Iterative Solution methods The ideas to use iterative

More information

Time-adaptive methods for the incompressible Navier-Stokes equations

Time-adaptive methods for the incompressible Navier-Stokes equations Time-adaptive methods for the incompressible Navier-Stokes equations Joachim Rang, Thorsten Grahs, Justin Wiegmann, 29.09.2016 Contents Introduction Diagonally implicit Runge Kutta methods Results with

More information

OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU

OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative methods ffl Krylov subspace methods ffl Preconditioning techniques: Iterative methods ILU Preconditioning Techniques for Solving Large Sparse Linear Systems Arnold Reusken Institut für Geometrie und Praktische Mathematik RWTH-Aachen OUTLINE ffl CFD: elliptic pde's! Ax = b ffl Basic iterative

More information

Splitting Iteration Methods for Positive Definite Linear Systems

Splitting Iteration Methods for Positive Definite Linear Systems Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System

More information

Solving Linear Systems

Solving Linear Systems Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 207 Philippe B. Laval (KSU) Linear Systems Fall 207 / 2 Introduction We continue looking how to solve linear systems of the

More information

A NEW EFFECTIVE PRECONDITIONED METHOD FOR L-MATRICES

A NEW EFFECTIVE PRECONDITIONED METHOD FOR L-MATRICES Journal of Mathematical Sciences: Advances and Applications Volume, Number 2, 2008, Pages 3-322 A NEW EFFECTIVE PRECONDITIONED METHOD FOR L-MATRICES Department of Mathematics Taiyuan Normal University

More information

Numerical Methods - Numerical Linear Algebra

Numerical Methods - Numerical Linear Algebra Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear

More information

Introduction to Iterative Solvers of Linear Systems

Introduction to Iterative Solvers of Linear Systems Introduction to Iterative Solvers of Linear Systems SFB Training Event January 2012 Prof. Dr. Andreas Frommer Typeset by Lukas Krämer, Simon-Wolfgang Mages and Rudolf Rödl 1 Classes of Matrices and their

More information

Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems

Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Dr. Noemi Friedman, 09.2.205. Reminder: Instationary heat

More information

Iterative Methods. Splitting Methods

Iterative Methods. Splitting Methods Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition

More information

7.3 The Jacobi and Gauss-Siedel Iterative Techniques. Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP.

7.3 The Jacobi and Gauss-Siedel Iterative Techniques. Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP. 7.3 The Jacobi and Gauss-Siedel Iterative Techniques Problem: To solve Ax = b for A R n n. Methodology: Iteratively approximate solution x. No GEPP. 7.3 The Jacobi and Gauss-Siedel Iterative Techniques

More information

Kasetsart University Workshop. Multigrid methods: An introduction

Kasetsart University Workshop. Multigrid methods: An introduction Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available

More information

Num. discretization. Numerical simulations. Finite elements Finite volume. Finite diff. Mathematical model A set of ODEs/PDEs. Integral equations

Num. discretization. Numerical simulations. Finite elements Finite volume. Finite diff. Mathematical model A set of ODEs/PDEs. Integral equations Scientific Computing Computer simulations Road map p. 1/46 Numerical simulations Physical phenomenon Mathematical model A set of ODEs/PDEs Integral equations Something else Num. discretization Finite diff.

More information

Here is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J

Here is an example of a block diagonal matrix with Jordan Blocks on the diagonal: J Class Notes 4: THE SPECTRAL RADIUS, NORM CONVERGENCE AND SOR. Math 639d Due Date: Feb. 7 (updated: February 5, 2018) In the first part of this week s reading, we will prove Theorem 2 of the previous class.

More information

Iterative Methods for Ax=b

Iterative Methods for Ax=b 1 FUNDAMENTALS 1 Iterative Methods for Ax=b 1 Fundamentals consider the solution of the set of simultaneous equations Ax = b where A is a square matrix, n n and b is a right hand vector. We write the iterative

More information

JACOBI S ITERATION METHOD

JACOBI S ITERATION METHOD ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes

More information

AIMS Exercise Set # 1

AIMS Exercise Set # 1 AIMS Exercise Set #. Determine the form of the single precision floating point arithmetic used in the computers at AIMS. What is the largest number that can be accurately represented? What is the smallest

More information

A generalization of the Gauss-Seidel iteration method for solving absolute value equations

A generalization of the Gauss-Seidel iteration method for solving absolute value equations A generalization of the Gauss-Seidel iteration method for solving absolute value equations Vahid Edalatpour, Davod Hezari and Davod Khojasteh Salkuyeh Faculty of Mathematical Sciences, University of Guilan,

More information

Chapter 7 Iterative Techniques in Matrix Algebra

Chapter 7 Iterative Techniques in Matrix Algebra Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition

More information

Lecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University

Lecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University Lecture Note 7: Iterative methods for solving linear systems Xiaoqun Zhang Shanghai Jiao Tong University Last updated: December 24, 2014 1.1 Review on linear algebra Norms of vectors and matrices vector

More information

9. Iterative Methods for Large Linear Systems

9. Iterative Methods for Large Linear Systems EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1

More information

EXISTENCE VERIFICATION FOR SINGULAR ZEROS OF REAL NONLINEAR SYSTEMS

EXISTENCE VERIFICATION FOR SINGULAR ZEROS OF REAL NONLINEAR SYSTEMS EXISTENCE VERIFICATION FOR SINGULAR ZEROS OF REAL NONLINEAR SYSTEMS JIANWEI DIAN AND R BAKER KEARFOTT Abstract Traditional computational fixed point theorems, such as the Kantorovich theorem (made rigorous

More information

Comparison of Fixed Point Methods and Krylov Subspace Methods Solving Convection-Diffusion Equations

Comparison of Fixed Point Methods and Krylov Subspace Methods Solving Convection-Diffusion Equations American Journal of Computational Mathematics, 5, 5, 3-6 Published Online June 5 in SciRes. http://www.scirp.org/journal/ajcm http://dx.doi.org/.436/ajcm.5.5 Comparison of Fixed Point Methods and Krylov

More information

Monte Carlo Method for Finding the Solution of Dirichlet Partial Differential Equations

Monte Carlo Method for Finding the Solution of Dirichlet Partial Differential Equations Applied Mathematical Sciences, Vol. 1, 2007, no. 10, 453-462 Monte Carlo Method for Finding the Solution of Dirichlet Partial Differential Equations Behrouz Fathi Vajargah Department of Mathematics Guilan

More information

Introduction. Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods. Example: First Order Richardson. Strategy

Introduction. Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods. Example: First Order Richardson. Strategy Introduction Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 Solve system Ax = b by repeatedly computing

More information

Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods

Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods Math 1080: Numerical Linear Algebra Chapter 4, Iterative Methods M. M. Sussman sussmanm@math.pitt.edu Office Hours: MW 1:45PM-2:45PM, Thack 622 March 2015 1 / 70 Topics Introduction to Iterative Methods

More information

Jae Heon Yun and Yu Du Han

Jae Heon Yun and Yu Du Han Bull. Korean Math. Soc. 39 (2002), No. 3, pp. 495 509 MODIFIED INCOMPLETE CHOLESKY FACTORIZATION PRECONDITIONERS FOR A SYMMETRIC POSITIVE DEFINITE MATRIX Jae Heon Yun and Yu Du Han Abstract. We propose

More information

Iterative Methods and Multigrid

Iterative Methods and Multigrid Iterative Methods and Multigrid Part 3: Preconditioning 2 Eric de Sturler Preconditioning The general idea behind preconditioning is that convergence of some method for the linear system Ax = b can be

More information

Computation Fluid Dynamics

Computation Fluid Dynamics Computation Fluid Dynamics CFD I Jitesh Gajjar Maths Dept Manchester University Computation Fluid Dynamics p.1/189 Garbage In, Garbage Out We will begin with a discussion of errors. Useful to understand

More information

Iterative Methods for Smooth Objective Functions

Iterative Methods for Smooth Objective Functions Optimization Iterative Methods for Smooth Objective Functions Quadratic Objective Functions Stationary Iterative Methods (first/second order) Steepest Descent Method Landweber/Projected Landweber Methods

More information

Examination paper for TMA4122/TMA4123/TMA4125/TMA4130 Matematikk 4M/N

Examination paper for TMA4122/TMA4123/TMA4125/TMA4130 Matematikk 4M/N Department of Mathematical Sciences Examination paper for TMA4122/TMA4123/TMA4125/TMA4130 Matematikk 4M/N Academic contact during examination: Markus Grasmair Phone: 97 58 04 35 Code C): Basic calculator.

More information

Process Model Formulation and Solution, 3E4

Process Model Formulation and Solution, 3E4 Process Model Formulation and Solution, 3E4 Section B: Linear Algebraic Equations Instructor: Kevin Dunn dunnkg@mcmasterca Department of Chemical Engineering Course notes: Dr Benoît Chachuat 06 October

More information

An Accelerated Block-Parallel Newton Method via Overlapped Partitioning

An Accelerated Block-Parallel Newton Method via Overlapped Partitioning An Accelerated Block-Parallel Newton Method via Overlapped Partitioning Yurong Chen Lab. of Parallel Computing, Institute of Software, CAS (http://www.rdcps.ac.cn/~ychen/english.htm) Summary. This paper

More information

Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015

Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015 Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in NUMERICAL ANALYSIS Spring 2015 Instructions: Do exactly two problems from Part A AND two

More information

Scientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1

Scientific Computing WS 2018/2019. Lecture 9. Jürgen Fuhrmann Lecture 9 Slide 1 Scientific Computing WS 2018/2019 Lecture 9 Jürgen Fuhrmann juergen.fuhrmann@wias-berlin.de Lecture 9 Slide 1 Lecture 9 Slide 2 Simple iteration with preconditioning Idea: Aû = b iterative scheme û = û

More information

1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0

1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0 Numerical Analysis 1 1. Nonlinear Equations This lecture note excerpted parts from Michael Heath and Max Gunzburger. Given function f, we seek value x for which where f : D R n R n is nonlinear. f(x) =

More information

Poisson Equation in 2D

Poisson Equation in 2D A Parallel Strategy Department of Mathematics and Statistics McMaster University March 31, 2010 Outline Introduction 1 Introduction Motivation Discretization Iterative Methods 2 Additive Schwarz Method

More information

EXAMPLES OF CLASSICAL ITERATIVE METHODS

EXAMPLES OF CLASSICAL ITERATIVE METHODS EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic

More information

Chapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs

Chapter Two: Numerical Methods for Elliptic PDEs. 1 Finite Difference Methods for Elliptic PDEs Chapter Two: Numerical Methods for Elliptic PDEs Finite Difference Methods for Elliptic PDEs.. Finite difference scheme. We consider a simple example u := subject to Dirichlet boundary conditions ( ) u

More information

The amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A.

The amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A. AMSC/CMSC 661 Scientific Computing II Spring 2005 Solution of Sparse Linear Systems Part 2: Iterative methods Dianne P. O Leary c 2005 Solving Sparse Linear Systems: Iterative methods The plan: Iterative

More information

Chapter 2. Solving Systems of Equations. 2.1 Gaussian elimination

Chapter 2. Solving Systems of Equations. 2.1 Gaussian elimination Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix

More information

MATRIX SPLITTING PRINCIPLES

MATRIX SPLITTING PRINCIPLES IJS 8:5 00 5 84 PII. S067000706 http://ijmms.hindawi.com Hindawi Publishing Corp. ATRIX SPLITTING PRINCIPLES ZBIGNIEW I. WOŹNICKI Received 5 arch 00 Abstract. The systematic analysis of convergence conditions,

More information

Parallel Iterative Methods for Sparse Linear Systems. H. Martin Bücker Lehrstuhl für Hochleistungsrechnen

Parallel Iterative Methods for Sparse Linear Systems. H. Martin Bücker Lehrstuhl für Hochleistungsrechnen Parallel Iterative Methods for Sparse Linear Systems Lehrstuhl für Hochleistungsrechnen www.sc.rwth-aachen.de RWTH Aachen Large and Sparse Small and Dense Outline Problem with Direct Methods Iterative

More information

Motivation: Sparse matrices and numerical PDE's

Motivation: Sparse matrices and numerical PDE's Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting

More information

Jordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS

Jordan Journal of Mathematics and Statistics (JJMS) 5(3), 2012, pp A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS Jordan Journal of Mathematics and Statistics JJMS) 53), 2012, pp.169-184 A NEW ITERATIVE METHOD FOR SOLVING LINEAR SYSTEMS OF EQUATIONS ADEL H. AL-RABTAH Abstract. The Jacobi and Gauss-Seidel iterative

More information

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C. Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference

More information

The upper Jacobi and upper Gauss Seidel type iterative methods for preconditioned linear systems

The upper Jacobi and upper Gauss Seidel type iterative methods for preconditioned linear systems Applied Mathematics Letters 19 (2006) 1029 1036 wwwelseviercom/locate/aml The upper Jacobi upper Gauss Seidel type iterative methods for preconditioned linear systems Zhuan-De Wang, Ting-Zhu Huang School

More information

Spectral gradient projection method for solving nonlinear monotone equations

Spectral gradient projection method for solving nonlinear monotone equations Journal of Computational and Applied Mathematics 196 (2006) 478 484 www.elsevier.com/locate/cam Spectral gradient projection method for solving nonlinear monotone equations Li Zhang, Weijun Zhou Department

More information

Conjugate Gradient (CG) Method

Conjugate Gradient (CG) Method Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous

More information

A Refinement of Gauss-Seidel Method for Solving. of Linear System of Equations

A Refinement of Gauss-Seidel Method for Solving. of Linear System of Equations Int. J. Contemp. Math. Sciences, Vol. 6, 0, no. 3, 7 - A Refinement of Gauss-Seidel Method for Solving of Linear System of Equations V. B. Kumar Vatti and Tesfaye Kebede Eneyew Department of Engineering

More information

Lab 1: Iterative Methods for Solving Linear Systems

Lab 1: Iterative Methods for Solving Linear Systems Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as

More information

4.6 Iterative Solvers for Linear Systems

4.6 Iterative Solvers for Linear Systems 4.6 Iterative Solvers for Linear Systems Why use iterative methods? Virtually all direct methods for solving Ax = b require O(n 3 ) floating point operations. In practical applications the matrix A often

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 21: Sensitivity of Eigenvalues and Eigenvectors; Conjugate Gradient Method Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis

More information

Partititioned Methods for Multifield Problems

Partititioned Methods for Multifield Problems Partititioned Methods for Multifield Problems Joachim Rang, 22.7.215 22.7.215 Joachim Rang Partititioned Methods for Multifield Problems Seite 1 Non-matching matches usually the fluid and the structure

More information

Math Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.

Math Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012. Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems

More information

CLASSICAL ITERATIVE METHODS

CLASSICAL ITERATIVE METHODS CLASSICAL ITERATIVE METHODS LONG CHEN In this notes we discuss classic iterative methods on solving the linear operator equation (1) Au = f, posed on a finite dimensional Hilbert space V = R N equipped

More information

Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers

Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Dr. Noemi

More information

Modified Gauss Seidel type methods and Jacobi type methods for Z-matrices

Modified Gauss Seidel type methods and Jacobi type methods for Z-matrices Linear Algebra and its Applications 7 (2) 227 24 www.elsevier.com/locate/laa Modified Gauss Seidel type methods and Jacobi type methods for Z-matrices Wen Li a,, Weiwei Sun b a Department of Mathematics,

More information

DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular

DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix

More information

Termination criteria for inexact fixed point methods

Termination criteria for inexact fixed point methods Termination criteria for inexact fixed point methods Philipp Birken 1 October 1, 2013 1 Institute of Mathematics, University of Kassel, Heinrich-Plett-Str. 40, D-34132 Kassel, Germany Department of Mathematics/Computer

More information

The Triangle Algorithm: A Geometric Approach to Systems of Linear Equations

The Triangle Algorithm: A Geometric Approach to Systems of Linear Equations : A Geometric Approach to Systems of Linear Equations Thomas 1 Bahman Kalantari 2 1 Baylor University 2 Rutgers University July 19, 2013 Rough Outline Quick Refresher of Basics Convex Hull Problem Solving

More information

Monte Carlo simulation inspired by computational optimization. Colin Fox Al Parker, John Bardsley MCQMC Feb 2012, Sydney

Monte Carlo simulation inspired by computational optimization. Colin Fox Al Parker, John Bardsley MCQMC Feb 2012, Sydney Monte Carlo simulation inspired by computational optimization Colin Fox fox@physics.otago.ac.nz Al Parker, John Bardsley MCQMC Feb 2012, Sydney Sampling from π(x) Consider : x is high-dimensional (10 4

More information

Iterative techniques in matrix algebra

Iterative techniques in matrix algebra Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and

More information

Preconditioning for Nonsymmetry and Time-dependence

Preconditioning for Nonsymmetry and Time-dependence Preconditioning for Nonsymmetry and Time-dependence Andy Wathen Oxford University, UK joint work with Jen Pestana and Elle McDonald Jeju, Korea, 2015 p.1/24 Iterative methods For self-adjoint problems/symmetric

More information

Numerical Solution Techniques in Mechanical and Aerospace Engineering

Numerical Solution Techniques in Mechanical and Aerospace Engineering Numerical Solution Techniques in Mechanical and Aerospace Engineering Chunlei Liang LECTURE 3 Solvers of linear algebraic equations 3.1. Outline of Lecture Finite-difference method for a 2D elliptic PDE

More information

Algebra C Numerical Linear Algebra Sample Exam Problems

Algebra C Numerical Linear Algebra Sample Exam Problems Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric

More information

Generalized line criterion for Gauss-Seidel method

Generalized line criterion for Gauss-Seidel method Computational and Applied Mathematics Vol. 22, N. 1, pp. 91 97, 2003 Copyright 2003 SBMAC Generalized line criterion for Gauss-Seidel method M.V.P. GARCIA 1, C. HUMES JR. 2 and J.M. STERN 2 1 Department

More information

Solving Linear Systems

Solving Linear Systems Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Linear Systems Fall 2015 1 / 12 Introduction We continue looking how to solve linear systems of

More information

MA3232 Numerical Analysis Week 9. James Cooley (1926-)

MA3232 Numerical Analysis Week 9. James Cooley (1926-) MA umerical Analysis Week 9 James Cooley (96-) James Cooley is an American mathematician. His most significant contribution to the world of mathematics and digital signal processing is the Fast Fourier

More information

ETNA Kent State University

ETNA Kent State University Electronic Transactions on Numerical Analysis. Volume 4, pp. 1-13, March 1996. Copyright 1996,. ISSN 1068-9613. ETNA NON-STATIONARY PARALLEL MULTISPLITTING AOR METHODS ROBERT FUSTER, VIOLETA MIGALLÓN,

More information

Cache Oblivious Stencil Computations

Cache Oblivious Stencil Computations Cache Oblivious Stencil Computations S. HUNOLD J. L. TRÄFF F. VERSACI Lectures on High Performance Computing 13 April 2015 F. Versaci (TU Wien) Cache Oblivious Stencil Computations 13 April 2015 1 / 19

More information

Solution of Matrix Eigenvalue Problem

Solution of Matrix Eigenvalue Problem Outlines October 12, 2004 Outlines Part I: Review of Previous Lecture Part II: Review of Previous Lecture Outlines Part I: Review of Previous Lecture Part II: Standard Matrix Eigenvalue Problem Other Forms

More information

(f(x) P 3 (x)) dx. (a) The Lagrange formula for the error is given by

(f(x) P 3 (x)) dx. (a) The Lagrange formula for the error is given by 1. QUESTION (a) Given a nth degree Taylor polynomial P n (x) of a function f(x), expanded about x = x 0, write down the Lagrange formula for the truncation error, carefully defining all its elements. How

More information

Polynomial accelerated MCMC... and other sampling algorithms inspired by computational optimization

Polynomial accelerated MCMC... and other sampling algorithms inspired by computational optimization Polynomial accelerated MCMC... and other sampling algorithms inspired by computational optimization Colin Fox fox@physics.otago.ac.nz Al Parker, John Bardsley Fore-words Motivation: an inverse oceanography

More information

Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes

Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes Elena Virnik, TU BERLIN Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov

More information

CONVERGENCE BEHAVIOUR OF INEXACT NEWTON METHODS

CONVERGENCE BEHAVIOUR OF INEXACT NEWTON METHODS MATHEMATICS OF COMPUTATION Volume 68, Number 228, Pages 165 1613 S 25-5718(99)1135-7 Article electronically published on March 1, 1999 CONVERGENCE BEHAVIOUR OF INEXACT NEWTON METHODS BENEDETTA MORINI Abstract.

More information

CHAPTER 5. Basic Iterative Methods

CHAPTER 5. Basic Iterative Methods Basic Iterative Methods CHAPTER 5 Solve Ax = f where A is large and sparse (and nonsingular. Let A be split as A = M N in which M is nonsingular, and solving systems of the form Mz = r is much easier than

More information

COURSE Numerical methods for solving linear systems. Practical solving of many problems eventually leads to solving linear systems.

COURSE Numerical methods for solving linear systems. Practical solving of many problems eventually leads to solving linear systems. COURSE 9 4 Numerical methods for solving linear systems Practical solving of many problems eventually leads to solving linear systems Classification of the methods: - direct methods - with low number of

More information

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system

More information

Up to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas

Up to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas Finding Eigenvalues Up to this point, our main theoretical tools for finding eigenvalues without using det{a λi} = 0 have been the trace and determinant formulas plus the facts that det{a} = λ λ λ n, Tr{A}

More information

Numerical Programming I (for CSE)

Numerical Programming I (for CSE) Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let

More information

Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58

Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58 Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms

More information

Math 471 (Numerical methods) Chapter 3 (second half). System of equations

Math 471 (Numerical methods) Chapter 3 (second half). System of equations Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular

More information

A new nonmonotone Newton s modification for unconstrained Optimization

A new nonmonotone Newton s modification for unconstrained Optimization A new nonmonotone Newton s modification for unconstrained Optimization Aristotelis E. Kostopoulos a George S. Androulakis b a a Department of Mathematics, University of Patras, GR-265.04, Rio, Greece b

More information

Exam in TMA4215 December 7th 2012

Exam in TMA4215 December 7th 2012 Norwegian University of Science and Technology Department of Mathematical Sciences Page of 9 Contact during the exam: Elena Celledoni, tlf. 7359354, cell phone 48238584 Exam in TMA425 December 7th 22 Allowed

More information

Notes for CS542G (Iterative Solvers for Linear Systems)

Notes for CS542G (Iterative Solvers for Linear Systems) Notes for CS542G (Iterative Solvers for Linear Systems) Robert Bridson November 20, 2007 1 The Basics We re now looking at efficient ways to solve the linear system of equations Ax = b where in this course,

More information

Theory of Iterative Methods

Theory of Iterative Methods Based on Strang s Introduction to Applied Mathematics Theory of Iterative Methods The Iterative Idea To solve Ax = b, write Mx (k+1) = (M A)x (k) + b, k = 0, 1,,... Then the error e (k) x (k) x satisfies

More information

Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018

Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March 22, 2018 1 Linear Systems Math 5630: Iterative Methods for Systems of Equations Hung Phan, UMass Lowell March, 018 Consider the system 4x y + z = 7 4x 8y + z = 1 x + y + 5z = 15. We then obtain x = 1 4 (7 + y z)

More information

Dense Matrices for Biofluids Applications

Dense Matrices for Biofluids Applications Dense Matrices for Biofluids Applications by Liwei Chen A Project Report Submitted to the Faculty of the WORCESTER POLYTECHNIC INSTITUTE In partial fulfillment of the requirements for the Degree of Master

More information