Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations
|
|
- Damian Anderson
- 5 years ago
- Views:
Transcription
1 Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system of equations Ax = b where A is a given n n matrix, b is a given n 1 vector, and x the sought n 1 solution vector. Department of Mathematics, Umeå University November 11, 2008 Our basic assumption is that n is large (e.g., n > 10 3 ) and that A is sparse. A sparse matrix is somewhat vaguely defined as one with very few non-zero entries a i,j. Motivation for Studying Sparse Linear Systems Partial Differential Equations PDE s are important since they provide mathematical models of many real-world phenomenons, such as: Elastic deformation - Cauchy-Navier s equations Discretization of partial differential equations (PDE s) constitute by far the biggest source of sparse matrix problems. σ = f σ = 2με(u) + λ( u)i Fluid flow - the Navier-Stokes equations u +(u )u Δu + p = 0 u = 0 Heat Transfer - the Heat equation Ṫ + u T T = f
2 Discretizing Partial Differential Equations A PDE Model Problem: Poisson s Equation The typical way to solve PDE s is to discretize them, that is, to approximate them by other (algebraic) equations involving only a finite number of unknowns. The matrix problems arising from these discretizations are generally large and sparse. Two standard techniques for discretizing PDE s are: Finite Difference Methods (FDM) Finite Element Methods (FEM) Consider Poisson s problem: Find a function u = u(x, y) such that ( 2 ) u x u y 2 = 1 (1) within the unit square Ω={0 < x, y < 1} and such that u = 0 on the boundary Ω. Needless to say, this is a PDE since it is an equation involving the partial derivatives of u. Let us discretize this PDE using finite differences. Finite Differences To obtain a numerical approximation to u the domain Ω is first subdivided into an (N + 1) (N + 1) grid of squares. Finite Differences, cont d Let u(ih, jh) =u i,j for ease of notation. On the grid, derivatives are approximated by difference quotients, e.g., or u i+1/2,j x u i+1,j u i,j h Thus 2 u i,j x 2 u i+1,j 2u i,j + u i 1,j h 2 The grid spacing is h = 1/(N + 1). The vertex points (ih, jh), i, j = 0, 1,...,N + 1 are called nodes. The aim is to compute approximate values of u at the nodes. 2 u i,j x u i,j y 2 u i 1,j + u i+1,j 4u i,j + u i,j 1 + u i,j+1 h 2 (The expression on the right hand side is sometimes called the 5-point stencil.)
3 Finite Differences, cont d A finite difference approximation to (1) is defined by 4u i,j u i 1,j u i+1,j u i,j 1 u i,j+1 = h 2 (2) with i, j = 1, 3,...,N. Finite Differences, cont d The resulting linear system Ax = b is sparse, banded, and generally very large since we wish to obtain an accurate solution approximation to the PDE and therefore compute on a fine grid with many nodes. In matlab it is easy to construct A since it is given by the built-in routine delsq. Just type This is N 2 linear equations for the N 2 unknown values u i,j in the interior nodes. (Recall that u = 0 on the boundary.) N = 5; h = 1/(N+1); G = numgrid( S,N+2); % create grid A = delsq(g); % matrix A b = hˆ2*ones(nˆ2,1); % vector b Finite Differences, cont d The distribution of the non-zero entries in a matrix is called the sparsity pattern of the matrix. You can plot the sparsity pattern of a matrix in matlab with the spy command. Finite Differences, cont d Solving the linear system resulting from the finite difference discretization process we get a solution approximation to the PDE. x = A\b; % solve linear system u = G; % map solution onto grid u(g>0) = full(x(g(g>0))); surf(u) % plot Figure: Sparsity pattern of the finite difference matrix A. Figure: Plotted finite difference solution.
4 Finite Differences, cont d The general idea is that the accuracy of the computed finite difference solution can be improved by using a finer grid with more closely spaced nodes. This implies more computational work since the size of the resulting linear system grows. When doing computations one usually has to strike a balance between the precision of the output and the computational cost. Finite differences are popular because of their simplicity. Their drawbacks are that they cannot handle complex computational domains, and are hard to analyze quantitatively. Finite element methods are easier to analyze and are also able to handle complex geometries but they are more complicated. Both methods however yield large linear systems to be solved, depending on the size of the mesh. The Need for Iterative Methods for Linear Systems Direct methods for solving Ax = b (such as LU factorization, or Gaussian elimination) require roughly n 3 arithmetic operations (i.e., + and *). Thus they often become too expensive (w.r.t. cpu time and memory consumption) to use when dealing with very large linear systems. In such cases iterative methods might work fine, since they are usually memory conserving and fast. Iterative algorithms produce a sequence of approximations x (1), x (2),...,x (k) which (hopefully) converges to the true solution x as k. Basic Iterative Methods for Linear Systems Basic Iterative Methods for Linear Systems, cont d Consider the system of equations Let us split A into Ax = b. A = M K where M is any non-singular matrix and K = M A. Hence, Ax = b becomes (M K )x = b Mx = Kx + b x = M 1 Kx + M 1 b This suggests the iteration scheme: For k = 1, 2, 3,...repeat until convergence. x (k+1) = M 1 Kx (k) + M 1 b Of course, for this iteration to be computationally practical, the splitting of A should be chosen such that M 1 K and M 1 b are easy to calculate. We will study splittings based on the diagonal, and the upper / lower triangular parts of A: A = D U L
5 Jacobi s Method The Gauss-Seidel Method So-called Jacobi iteration is defined by choosing the splitting A = D (L + U) =M K where D is the diagonal of A, L is the strictly lower triangle of A, and U is the strictly upper triangle of A. The iteration scheme takes the form x (k+1) = D 1 (L + U)x (k) + D 1 b Note that D is easy to invert since it is a diagonal matrix. In the (forward) Gauss-Seidel method A is split into yielding the iteration scheme A =(D L) U = M K x (k+1) =(D L) 1 (Ux (k) + b) Since D L is lower triangular the effect of (D L) 1 can be computed by forward elimination. The (backward) Gauss-Seidel method instead uses A =(D U) L = M K Successive Over-Relaxation (SOR) A more sophisticated method is obtained by choosing A =( 1 ω D L) (1 D + U) =M K ω ω where ω is a relaxation parameter. This gives the iteration scheme x (k+1) =(D ωl) 1 (ωu +(1 ω)d)x (k) + ω(d ωl) 1 b Matlab Code Set up the splitting D=diag(diag(A)) L=-tril(A,-1) U=-triu(A, 1) Jacobi s method for k = 1:10 y = D\((L+U)*x + b) x = y; end Gauss-Seidel s method for k = 1:10 y = (D-L)\(U*x + b) x = y; end
6 Diagonally Dominant Matrices Convergence An n n matrix A is said to be (strictly) diagonally dominant if a i,i > n j=0,j i a i,j i = 1, 2,...,n that is, if the absolute value of each diagonal element is greater than the sum of the absolute values of the other elements in its row. Example A = The success (i.e., convergence) of an iterative method depends on the type of linear system Ax = b it is applied to. Jacobi converges if A is strictly diagonally dominant. Gauss-Seidel converges if A is symmetric and positive definite (SPD). SOR also converges for SPD matrices, but only if the relaxation parameter ω is such that 0 <ω<2. is diagonally dominant. Kacmarz Method So what if the matrix A is non-singular, but unsymmetric or indefinite? In these cases it is possible to apply Gauss-Seidel or SOR to A T Ax = A T b that is, the familiar normal equations. Since A T A is symmetric and positive definite if A is non-singular both Gauss-Seidel and SOR will converge. However the rate of convergence can be slow. Convergence Analysis All the above methods can be written x (k+1) = Rx (k) + c where the iteration matrix R is R = M 1 K and c = M 1 b. A relation between the errors in successive approximations can be be derived by subtracting x = Rx + c: x (k+1) x = R(x (k) x) =...= R k+1 (x (0) x). Taking norms and using the Cauchy inequality we get x (k+1) x R k+1 x (0) x R k+1 x (0) x. Thus a sufficient condition for convergence is that R < 1in any norm.
7 Convergence Analysis, cont d From the above analysis it is clear that R should be as small as possible since this is the amplification factor for the error in each iteration. Hence the splitting of A should be chosen such that: Rx = M 1 Kx and x = M 1 b are easy to evaluate. R is small. Unfortunately, these goals are contradictory, and a balance has to be struck. For example, 1. M = I makes M 1 trivial, but probably not A I < M = A gives K = 0 and thus R = M 1 K = 0, but then M 1 = A 1 is expensive to compute. Convergence Analysis for Jacobi s Method We claimed before that Jacobi s method converges for strictly diagonally dominant matrices. Let us prove this. In Jacobi s method the iteration matrix R has the elements Taking the infinity norm gives r i,j = a i,j a i,i, i j, r i,i = 0 R = max n 1 i n j=1, j i a i,j a i,i which shows that R < 1ifA is strictly diagonal dominant, and we are done. Storage Schemes for Sparse Matrices Coordinate Format The simplest sparse format is the coordinate format. The data structure consists of three arrays, one for the non-zero matrix values, one for the row indices, and one for the column indices. All arrays are of length nz, the number of non-zeros. In order to take advantage of the large number of zero entries, special schemes are used to store sparse matrices. The main goal is to represent only the non-zero elements, and to be able to perform the common matrix operations, e.g., matrix-vector multiplication. The matrix A = can (for example) be represented as a=[ ]; r=[ ]; c=[ ];
8 Compressed Sparse Row Format Compressed Sparse Row Format, cont d Assuming that n nz a more economic storage scheme is the compressed sparse row (CSR) format. It also contains three arrays. A real array (values) of length nz containing all the matrix entries a i,j stored row by row. An integer array (colind) of length nz containing the column indices of a i,j. An integer array (rowptr) of length n + 1 containing pointers to the start of each row within the arrays values and colind. The last place usually contain the number of non-zeros nz, or nz + 1. The matrix is stored in CSR format as A = values=[ ]; colind=[ ]; rowptr=[ ]; Compressed Sparse Column Format Basic Sparse Matrix Operations The basic operations of linear algebra are more complicated when the matrix is stored in CRS format. There are a number of variations on the CSR format. The most obvious is to store the columns instead of the rows. This is the compressed sparse column (CSC) scheme. The CSC data arrays are matrix values, row index, and column pointers. Consider, for example, matrix-vector multiplication y = Ax: y = zeros(n,1) for i = 1:n for j= rowptr(i):rowptr(i+1)-1 y(i) = y(i) + values(j)*x(colind(j)) end end In the inner loop we traverse the non-zero columns of A in row i, multiply with the corresponding entries of x, and add to y.
9 Basic Sparse Matrix Operations, cont d Reading Since this method only multiplies non-zero matrix entries, the operations count is O(nz) which is a substantial saving over the dense operation requirement O(n 2 ). As we shall see the basic matrix-vector operation Ax is very important for modern iterative methods, since it is practically the only operation necessary (and affordable). Y Saad Iterative Methods for Sparse Linear Systems 1st edition. The book is free and can be downloaded from the URL: saad/books.html Chapter 2: 2.1, 2.2, and 2.3 Chapter 3: 3.1, 3.4, and 3.5 Chapter 4: read as much as you like
Lecture 18 Classical Iterative Methods
Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,
More information9. Iterative Methods for Large Linear Systems
EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationCLASSICAL ITERATIVE METHODS
CLASSICAL ITERATIVE METHODS LONG CHEN In this notes we discuss classic iterative methods on solving the linear operator equation (1) Au = f, posed on a finite dimensional Hilbert space V = R N equipped
More informationNumerical Solution Techniques in Mechanical and Aerospace Engineering
Numerical Solution Techniques in Mechanical and Aerospace Engineering Chunlei Liang LECTURE 3 Solvers of linear algebraic equations 3.1. Outline of Lecture Finite-difference method for a 2D elliptic PDE
More informationMath 471 (Numerical methods) Chapter 3 (second half). System of equations
Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular
More informationIterative Methods. Splitting Methods
Iterative Methods Splitting Methods 1 Direct Methods Solving Ax = b using direct methods. Gaussian elimination (using LU decomposition) Variants of LU, including Crout and Doolittle Other decomposition
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More information6. Iterative Methods for Linear Systems. The stepwise approach to the solution...
6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse
More informationLecture Note 7: Iterative methods for solving linear systems. Xiaoqun Zhang Shanghai Jiao Tong University
Lecture Note 7: Iterative methods for solving linear systems Xiaoqun Zhang Shanghai Jiao Tong University Last updated: December 24, 2014 1.1 Review on linear algebra Norms of vectors and matrices vector
More informationThe Solution of Linear Systems AX = B
Chapter 2 The Solution of Linear Systems AX = B 21 Upper-triangular Linear Systems We will now develop the back-substitution algorithm, which is useful for solving a linear system of equations that has
More informationComputational Methods. Systems of Linear Equations
Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations
More informationScientific Computing
Scientific Computing Direct solution methods Martin van Gijzen Delft University of Technology October 3, 2018 1 Program October 3 Matrix norms LU decomposition Basic algorithm Cost Stability Pivoting Pivoting
More informationReview of matrices. Let m, n IN. A rectangle of numbers written like A =
Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an
More informationToday s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
More informationScientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008
More informationNotes for CS542G (Iterative Solvers for Linear Systems)
Notes for CS542G (Iterative Solvers for Linear Systems) Robert Bridson November 20, 2007 1 The Basics We re now looking at efficient ways to solve the linear system of equations Ax = b where in this course,
More information9.1 Preconditioned Krylov Subspace Methods
Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 21: Sensitivity of Eigenvalues and Eigenvectors; Conjugate Gradient Method Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationNext topics: Solving systems of linear equations
Next topics: Solving systems of linear equations 1 Gaussian elimination (today) 2 Gaussian elimination with partial pivoting (Week 9) 3 The method of LU-decomposition (Week 10) 4 Iterative techniques:
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 207 Philippe B. Laval (KSU) Linear Systems Fall 207 / 2 Introduction We continue looking how to solve linear systems of the
More informationLU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark
DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline
More informationBackground. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms
More informationEXAMPLES OF CLASSICAL ITERATIVE METHODS
EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic
More informationLinear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4
Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix
More information2.29 Numerical Fluid Mechanics Fall 2011 Lecture 7
Numerical Fluid Mechanics Fall 2011 Lecture 7 REVIEW of Lecture 6 Material covered in class: Differential forms of conservation laws Material Derivative (substantial/total derivative) Conservation of Mass
More informationNumerical Linear Algebra
Numerical Linear Algebra Decompositions, numerical aspects Gerard Sleijpen and Martin van Gijzen September 27, 2017 1 Delft University of Technology Program Lecture 2 LU-decomposition Basic algorithm Cost
More informationProgram Lecture 2. Numerical Linear Algebra. Gaussian elimination (2) Gaussian elimination. Decompositions, numerical aspects
Numerical Linear Algebra Decompositions, numerical aspects Program Lecture 2 LU-decomposition Basic algorithm Cost Stability Pivoting Cholesky decomposition Sparse matrices and reorderings Gerard Sleijpen
More informationConsider the following example of a linear system:
LINEAR SYSTEMS Consider the following example of a linear system: Its unique solution is x + 2x 2 + 3x 3 = 5 x + x 3 = 3 3x + x 2 + 3x 3 = 3 x =, x 2 = 0, x 3 = 2 In general we want to solve n equations
More informationChapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
More informationMath/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018
Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x
More informationSolving the Generalized Poisson Equation Using the Finite-Difference Method (FDM)
Solving the Generalized Poisson Equation Using the Finite-Difference Method (FDM) James R. Nagel September 30, 2009 1 Introduction Numerical simulation is an extremely valuable tool for those who wish
More informationIterative methods for Linear System
Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and
More informationFinite Difference Methods for Boundary Value Problems
Finite Difference Methods for Boundary Value Problems October 2, 2013 () Finite Differences October 2, 2013 1 / 52 Goals Learn steps to approximate BVPs using the Finite Difference Method Start with two-point
More informationIterative Solution methods
p. 1/28 TDB NLA Parallel Algorithms for Scientific Computing Iterative Solution methods p. 2/28 TDB NLA Parallel Algorithms for Scientific Computing Basic Iterative Solution methods The ideas to use iterative
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Iteration basics Notes for 2016-11-07 An iterative solver for Ax = b is produces a sequence of approximations x (k) x. We always stop after finitely many steps, based on some convergence criterion, e.g.
More information5.7 Cramer's Rule 1. Using Determinants to Solve Systems Assumes the system of two equations in two unknowns
5.7 Cramer's Rule 1. Using Determinants to Solve Systems Assumes the system of two equations in two unknowns (1) possesses the solution and provided that.. The numerators and denominators are recognized
More informationA Hybrid Method for the Wave Equation. beilina
A Hybrid Method for the Wave Equation http://www.math.unibas.ch/ beilina 1 The mathematical model The model problem is the wave equation 2 u t 2 = (a 2 u) + f, x Ω R 3, t > 0, (1) u(x, 0) = 0, x Ω, (2)
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationLab 1: Iterative Methods for Solving Linear Systems
Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as
More informationFinite difference method for elliptic problems: I
Finite difference method for elliptic problems: I Praveen. C praveen@math.tifrbng.res.in Tata Institute of Fundamental Research Center for Applicable Mathematics Bangalore 560065 http://math.tifrbng.res.in/~praveen
More informationMTH 464: Computational Linear Algebra
MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University February 6, 2018 Linear Algebra (MTH
More informationJACOBI S ITERATION METHOD
ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes
More informationNumerical Analysis: Solutions of System of. Linear Equation. Natasha S. Sharma, PhD
Mathematical Question we are interested in answering numerically How to solve the following linear system for x Ax = b? where A is an n n invertible matrix and b is vector of length n. Notation: x denote
More informationMATH 3511 Lecture 1. Solving Linear Systems 1
MATH 3511 Lecture 1 Solving Linear Systems 1 Dmitriy Leykekhman Spring 2012 Goals Review of basic linear algebra Solution of simple linear systems Gaussian elimination D Leykekhman - MATH 3511 Introduction
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationLU Factorization. LU factorization is the most common way of solving linear systems! Ax = b LUx = b
AM 205: lecture 7 Last time: LU factorization Today s lecture: Cholesky factorization, timing, QR factorization Reminder: assignment 1 due at 5 PM on Friday September 22 LU Factorization LU factorization
More informationIterative Methods for Linear Systems
Iterative Methods for Linear Systems 1. Introduction: Direct solvers versus iterative solvers In many applications we have to solve a linear system Ax = b with A R n n and b R n given. If n is large the
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationHani Mehrpouyan, California State University, Bakersfield. Signals and Systems
Hani Mehrpouyan, Department of Electrical and Computer Engineering, Lecture 26 (LU Factorization) May 30 th, 2013 The material in these lectures is partly taken from the books: Elementary Numerical Analysis,
More informationClassical iterative methods for linear systems
Classical iterative methods for linear systems Ed Bueler MATH 615 Numerical Analysis of Differential Equations 27 February 1 March, 2017 Ed Bueler (MATH 615 NADEs) Classical iterative methods for linear
More informationA Comparison of Solving the Poisson Equation Using Several Numerical Methods in Matlab and Octave on the Cluster maya
A Comparison of Solving the Poisson Equation Using Several Numerical Methods in Matlab and Octave on the Cluster maya Sarah Swatski, Samuel Khuvis, and Matthias K. Gobbert (gobbert@umbc.edu) Department
More informationFinite Difference Methods (FDMs) 1
Finite Difference Methods (FDMs) 1 1 st - order Approxima9on Recall Taylor series expansion: Forward difference: Backward difference: Central difference: 2 nd - order Approxima9on Forward difference: Backward
More informationComputational Economics and Finance
Computational Economics and Finance Part II: Linear Equations Spring 2016 Outline Back Substitution, LU and other decomposi- Direct methods: tions Error analysis and condition numbers Iterative methods:
More informationIntroduction to Scientific Computing
(Lecture 5: Linear system of equations / Matrix Splitting) Bojana Rosić, Thilo Moshagen Institute of Scientific Computing Motivation Let us resolve the problem scheme by using Kirchhoff s laws: the algebraic
More informationSolving linear systems (6 lectures)
Chapter 2 Solving linear systems (6 lectures) 2.1 Solving linear systems: LU factorization (1 lectures) Reference: [Trefethen, Bau III] Lecture 20, 21 How do you solve Ax = b? (2.1.1) In numerical linear
More information30.5. Iterative Methods for Systems of Equations. Introduction. Prerequisites. Learning Outcomes
Iterative Methods for Systems of Equations 0.5 Introduction There are occasions when direct methods (like Gaussian elimination or the use of an LU decomposition) are not the best way to solve a system
More information5. Direct Methods for Solving Systems of Linear Equations. They are all over the place...
5 Direct Methods for Solving Systems of Linear Equations They are all over the place Miriam Mehl: 5 Direct Methods for Solving Systems of Linear Equations They are all over the place, December 13, 2012
More informationIntroduction to Boundary Value Problems
Chapter 5 Introduction to Boundary Value Problems When we studied IVPs we saw that we were given the initial value of a function and a differential equation which governed its behavior for subsequent times.
More informationMotivation: Sparse matrices and numerical PDE's
Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting
More informationSolving Ax = b w/ different b s: LU-Factorization
Solving Ax = b w/ different b s: LU-Factorization Linear Algebra Josh Engwer TTU 14 September 2015 Josh Engwer (TTU) Solving Ax = b w/ different b s: LU-Factorization 14 September 2015 1 / 21 Elementary
More information5.1 Banded Storage. u = temperature. The five-point difference operator. uh (x, y + h) 2u h (x, y)+u h (x, y h) uh (x + h, y) 2u h (x, y)+u h (x h, y)
5.1 Banded Storage u = temperature u= u h temperature at gridpoints u h = 1 u= Laplace s equation u= h u = u h = grid size u=1 The five-point difference operator 1 u h =1 uh (x + h, y) 2u h (x, y)+u h
More informationLinear Algebraic Equations
Linear Algebraic Equations 1 Fundamentals Consider the set of linear algebraic equations n a ij x i b i represented by Ax b j with [A b ] [A b] and (1a) r(a) rank of A (1b) Then Axb has a solution iff
More informationNumerical Methods I Non-Square and Sparse Linear Systems
Numerical Methods I Non-Square and Sparse Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 25th, 2014 A. Donev (Courant
More information2.1 Gaussian Elimination
2. Gaussian Elimination A common problem encountered in numerical models is the one in which there are n equations and n unknowns. The following is a description of the Gaussian elimination method for
More informationKasetsart University Workshop. Multigrid methods: An introduction
Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available
More informationAM 205: lecture 6. Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization
AM 205: lecture 6 Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization Unit II: Numerical Linear Algebra Motivation Almost everything in Scientific Computing
More informationPoisson Equation in 2D
A Parallel Strategy Department of Mathematics and Statistics McMaster University March 31, 2010 Outline Introduction 1 Introduction Motivation Discretization Iterative Methods 2 Additive Schwarz Method
More informationChapter 3 - From Gaussian Elimination to LU Factorization
Chapter 3 - From Gaussian Elimination to LU Factorization Maggie Myers Robert A. van de Geijn The University of Texas at Austin Practical Linear Algebra Fall 29 http://z.cs.utexas.edu/wiki/pla.wiki/ 1
More informationCS 323: Numerical Analysis and Computing
CS 323: Numerical Analysis and Computing MIDTERM #1 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.
More informationCAAM 454/554: Stationary Iterative Methods
CAAM 454/554: Stationary Iterative Methods Yin Zhang (draft) CAAM, Rice University, Houston, TX 77005 2007, Revised 2010 Abstract Stationary iterative methods for solving systems of linear equations are
More informationLecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.
Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference
More informationLecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo. Department of Mathematics Iowa State University
Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department
More informationIllustration of Gaussian elimination to find LU factorization. A = a 11 a 12 a 13 a 14 a 21 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44
Illustration of Gaussian elimination to find LU factorization. A = a 21 a a a a 31 a 32 a a a 41 a 42 a 43 a 1 Compute multipliers : Eliminate entries in first column: m i1 = a i1 a 11, i = 2, 3, 4 ith
More informationNumerical Methods I: Numerical linear algebra
1/3 Numerical Methods I: Numerical linear algebra Georg Stadler Courant Institute, NYU stadler@cimsnyuedu September 1, 017 /3 We study the solution of linear systems of the form Ax = b with A R n n, x,
More informationProcess Model Formulation and Solution, 3E4
Process Model Formulation and Solution, 3E4 Section B: Linear Algebraic Equations Instructor: Kevin Dunn dunnkg@mcmasterca Department of Chemical Engineering Course notes: Dr Benoît Chachuat 06 October
More informationLinear System of Equations
Linear System of Equations Linear systems are perhaps the most widely applied numerical procedures when real-world situation are to be simulated. Example: computing the forces in a TRUSS. F F 5. 77F F.
More informationChapter 6. Finite Element Method. Literature: (tiny selection from an enormous number of publications)
Chapter 6 Finite Element Method Literature: (tiny selection from an enormous number of publications) K.J. Bathe, Finite Element procedures, 2nd edition, Pearson 2014 (1043 pages, comprehensive). Available
More informationAM 205: lecture 6. Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization
AM 205: lecture 6 Last time: finished the data fitting topic Today s lecture: numerical linear algebra, LU factorization Unit II: Numerical Linear Algebra Motivation Almost everything in Scientific Computing
More information1. Fast Iterative Solvers of SLE
1. Fast Iterative Solvers of crucial drawback of solvers discussed so far: they become slower if we discretize more accurate! now: look for possible remedies relaxation: explicit application of the multigrid
More informationIterative techniques in matrix algebra
Iterative techniques in matrix algebra Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan September 12, 2015 Outline 1 Norms of vectors and matrices 2 Eigenvalues and
More informationSOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA
1 SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 2 OUTLINE Sparse matrix storage format Basic factorization
More informationA Review of Matrix Analysis
Matrix Notation Part Matrix Operations Matrices are simply rectangular arrays of quantities Each quantity in the array is called an element of the matrix and an element can be either a numerical value
More informationIntroduction to Mathematical Programming
Introduction to Mathematical Programming Ming Zhong Lecture 6 September 12, 2018 Ming Zhong (JHU) AMS Fall 2018 1 / 20 Table of Contents 1 Ming Zhong (JHU) AMS Fall 2018 2 / 20 Solving Linear Systems A
More informationSOLVING LINEAR SYSTEMS
SOLVING LINEAR SYSTEMS We want to solve the linear system a, x + + a,n x n = b a n, x + + a n,n x n = b n This will be done by the method used in beginning algebra, by successively eliminating unknowns
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 24: Preconditioning and Multigrid Solver Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 5 Preconditioning Motivation:
More informationMAC Learning Objectives. Learning Objectives (Cont.) Module 10 System of Equations and Inequalities II
MAC 1140 Module 10 System of Equations and Inequalities II Learning Objectives Upon completing this module, you should be able to 1. represent systems of linear equations with matrices. 2. transform a
More informationBoundary Value Problems and Iterative Methods for Linear Systems
Boundary Value Problems and Iterative Methods for Linear Systems 1. Equilibrium Problems 1.1. Abstract setting We want to find a displacement u V. Here V is a complete vector space with a norm v V. In
More informationIterative methods for Linear System of Equations. Joint Advanced Student School (JASS-2009)
Iterative methods for Linear System of Equations Joint Advanced Student School (JASS-2009) Course #2: Numerical Simulation - from Models to Software Introduction In numerical simulation, Partial Differential
More informationThe amount of work to construct each new guess from the previous one should be a small multiple of the number of nonzeros in A.
AMSC/CMSC 661 Scientific Computing II Spring 2005 Solution of Sparse Linear Systems Part 2: Iterative methods Dianne P. O Leary c 2005 Solving Sparse Linear Systems: Iterative methods The plan: Iterative
More informationApplied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.2: LU and Cholesky Factorizations 2 / 82 Preliminaries 3 / 82 Preliminaries
More informationChapter 2. Solving Systems of Equations. 2.1 Gaussian elimination
Chapter 2 Solving Systems of Equations A large number of real life applications which are resolved through mathematical modeling will end up taking the form of the following very simple looking matrix
More informationSparse Matrices and Iterative Methods
Sparse Matrices and Iterative Methods K. 1 1 Department of Mathematics 2018 Iterative Methods Consider the problem of solving Ax = b, where A is n n. Why would we use an iterative method? Avoid direct
More information1 Number Systems and Errors 1
Contents 1 Number Systems and Errors 1 1.1 Introduction................................ 1 1.2 Number Representation and Base of Numbers............. 1 1.2.1 Normalized Floating-point Representation...........
More informationSolving Linear Systems
Solving Linear Systems Iterative Solutions Methods Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) Linear Systems Fall 2015 1 / 12 Introduction We continue looking how to solve linear systems of
More informationStabilization and Acceleration of Algebraic Multigrid Method
Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration
More informationLinear Systems of n equations for n unknowns
Linear Systems of n equations for n unknowns In many application problems we want to find n unknowns, and we have n linear equations Example: Find x,x,x such that the following three equations hold: x
More informationBoundary Value Problems - Solving 3-D Finite-Difference problems Jacob White
Introduction to Simulation - Lecture 2 Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Outline Reminder about
More information