The implicit finite-difference method in matrix terms. Vrije Universiteit Amsterdam

Size: px
Start display at page:

Download "The implicit finite-difference method in matrix terms. Vrije Universiteit Amsterdam"

Transcription

1 - in matrix terms André Ran Vrije Universiteit Amsterdam 1

2 methods Last week we did the explicit finite, based on forward difference in time. This week we consider so-called implicit finite s, based on a backward difference in time. 2

3 methods The discretization of the heat equation becomes u m+1 n u m n δτ = um+1 n+1 2um+1 n + u m+1 n 1 (δx) 2 for m = 0, 1,, with the initial conditions u 0 n = u(nδx, 0), and with boundary conditions as before at ( N δx, mδτ) and at (N + δx, mδτ). Put α = δτ (δx) 2. Rewrite the equation above as αu m+1 n+1 + (1 + 2α)um+1 n αu m+1 n 1 = um n. Note that we now need to compute the u m n in an implicit manner, we cannot work our way forward in time as we did before. Thus it is now necessary to use matrix-vector methods. 3

4 methods Let us put the equation αu m+1 n+1 + (1 + 2α)um+1 n αu m+1 n 1 = um n. with its initial and boundary conditions in matrix-vector form. Put u m+1 = u m+1 N +1. u m+1 N + 1 This vector contains the u-values on the grid that are at time level (m + 1) δτ, but not on the boundary. Put k = N + + N 1. Then u m+1 R k. 4

5 methods To compute we need to use the boundary points, that is, we need the vector b m = u m N u m N + This should be understood correctly: we want b m and u m to be vectors of the same size k! 5

6 methods With these notations the equation αu m+1 n+1 can be rewritten as + (1 + 2α)um+1 n αu m+1 n 1 = um n. (I + αt)u m+1 = u m + αb m where α > 0 and T is the symmetric tridiagonal Toeplitz matrix T =

7 methods We can now solve the equation: (I + αt)u m+1 = u m + αb m u m+1 = (I + αt) 1 (u m + αb m ). Since we know u 0, which is the given initial condition, we can compute u m+1 u 1 = (I + αt) 1 ( u 0 + αb 0), u 2 = (I + αt) 1 ( u 1 + αb 1) =. (I + αt) 2 u 0 + α(i + αt) 2 b 0 + α(i + αt) 1 b 1, = (I + αt) m 1 u 0 + α m (I + αt) j 1 b m j. j=0 7

8 Error propagation methods Assuming we make no errors in b m, errors we make at time step m, which are denoted by e m propagate as e m+1 = (I + αt) 1 e m. Iterating this: errors we make at time 0 have at time m + 1 the effect e m+1 = (I + αt) m 1 e 0. These will decay as m increases when the eigenvalues of (I + αt) 1 are in the interval ( 1, 1). 8

9 Eigenvalues methods To see this, argue as follows. Since T = T, we have that T = V DV 1, where V is an orthogonal matrix, D is diagonal with real numbers on the diagonal. Then (I + αt) m 1 = V (I + αd) m 1 V 1 Now since D is a diagonal matrix, so is I + αd. So, the right hand side will go to zero as m if and only if all eigenvalues of (I + αt) 1 are in the interval ( 1, 1). We show something stronger. 9

10 methods Recall α = δτ δx 2 > 0. Theorem For all α > 0 the eigenvalues of (I + αt) 1 are in (0, 1). 10

11 methods Recall α = δτ δx 2 > 0. Theorem For all α > 0 the eigenvalues of (I + αt) 1 are in (0, 1). Proof. Since T is positive definite (i.e., Tx, x > 0 for all non-zero x) and α > 0 all eigenvalues of M = I + αt are in (1, ). Indeed, Mx = λx (M I)x = (λ 1)x αtx = (λ 1)x Tx = λ 1 α x. Thus λ > 1. 10

12 methods Recall α = δτ δx 2 > 0. Theorem For all α > 0 the eigenvalues of (I + αt) 1 are in (0, 1). Proof. Since T is positive definite (i.e., Tx, x > 0 for all non-zero x) and α > 0 all eigenvalues of M = I + αt are in (1, ). Indeed, Mx = λx (M I)x = (λ 1)x αtx = (λ 1)x Tx = λ 1 α x. Thus λ > 1. Furthermore, Mx = λx M 1 x = 1 λ x. So all eigenvalues of (I + αt) 1 are in (0, 1). 10

13 methods This is good news. Remember, for the explicit method we had to keep α below 1 2. Now we have stability of the method for all α > 0. So, if we want to improve on the calculations by choosing ten times as many data points in the x-direction (which in financial terms is the variable S), we can do so without having to change δτ as well! However, we pay a price: we have to compute an inverse: (I + αt) 1 or we have to solve the equations (I + αt)u m+1 = u m + αb m. 11

14 Methods methods Ideas: Direct inversion of I + αt is possible in a fast way (based on the fact that it is a tri-diagonal positive definite Toeplitz matrix; via the Gohberg-Semencul formulas or the Levinson algorithm). This will not be discussed. 12

15 Methods methods Ideas: Direct inversion of I + αt is possible in a fast way (based on the fact that it is a tri-diagonal positive definite Toeplitz matrix; via the Gohberg-Semencul formulas or the Levinson algorithm). This will not be discussed. We can use the LU-decomposition of I + αt to solve the matrix-vector equations. This will be discussed today. 12

16 Methods methods Ideas: Direct inversion of I + αt is possible in a fast way (based on the fact that it is a tri-diagonal positive definite Toeplitz matrix; via the Gohberg-Semencul formulas or the Levinson algorithm). This will not be discussed. We can use the LU-decomposition of I + αt to solve the matrix-vector equations. This will be discussed today. We can use iteration methods to solve the equations. This will be discussed next time. 12

17 methods Idea is to write I + αt = L U where L is lower triangular and U is upper triangular. Because of the fact that T is tri-diagonal, L will be bi-diagonal and U as well. Write y 1 u l y 2 u L = , U = l k y k 1 u k l k y k 13

18 methods gives I + αt = L U y 1 = 1 + 2α, l 1 = α/y 1, u 1 = α, l 1 u 1 + y 2 = 1 + 2α y 2 = 1 + 2α α2 y 1, l 2 = α/y 2, u 2 = α. Continue this way: y 1 = 1 + 2α and for all j = 1, 2, y j+1 = 1 + 2α α2 y j, l j = α y j, u j = α. 14

19 methods Now (I + αt)u m+1 = u m + αb m translates to L Uu m+1 = u m + αb m. First solve Lq m = u m + αb m, then solve Uu m+1 = q m. Denote c m = u m + αb m = c 1.. c k, q m = q 1. q k. with k = N + N

20 methods Then Lq m = c m can be solved top-to-bottom: q m 1 = c m 1, q m j = c m j l j q m j 1 = c m j + α y j 1 q m j 1. And from that Uu m+1 = q m can be solved from the bottom up: u m+1 N + 1 = qm k, u m+1 = qm j + αu m+1 j N +1. y j N k y j This is easy to implement. 16

21 methods In the examples we compute the European put, as before with T = 0.5, σ = 0.4, r = 0.02, E = 20. Again, the light blue line is the exact value. The dark blue line is the approximation using the implicit difference. In all figures the left hand graph is the graph of the put, the right hand graph is the graph of its derivative, the. In all cases we compute using an algorithm that solves the heat equation between x = 2 and x = 1.5. In the graphs we take δτ = 10 4, and vary δx. The values for δx are, in this order δx = 0.05, and δx = The corresponding values for α are, respectively, α = 0.04, and α = 1. We see that the value of the put is nicely approximated (actually indistinguishable from the exact value), but the approximation of is poor for large δx. It improves greatly with decreasing δx, and we do not now need to worry about keeping δx below 1 2. In fact, with δx = 0.01 the computed value is indistinguishable from the theoretical value. 17

22 prijs Europese put Approximation of Delta methods prijs Europese put 0 Approximation of Delta Figure 1: δx = 0.05 (above), δx = 0.01 (below), δτ =

Iterative methods: introduction and background. Vrije Universiteit Amsterdam

Iterative methods: introduction and background. Vrije Universiteit Amsterdam Iterative methods: introduction and background André Ran Vrije Universiteit Amsterdam 1 Consider the set of linear equations (I + αt)u = v, where v is given, α > 0 and T is the symmetric tridiagonal Toeplitz

More information

u n 2 4 u n 36 u n 1, n 1.

u n 2 4 u n 36 u n 1, n 1. Exercise 1 Let (u n ) be the sequence defined by Set v n = u n 1 x+ u n and f (x) = 4 x. 1. Solve the equations f (x) = 1 and f (x) =. u 0 = 0, n Z +, u n+1 = u n + 4 u n.. Prove that if u n < 1, then

More information

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn

Today s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear

More information

MAA507, Power method, QR-method and sparse matrix representation.

MAA507, Power method, QR-method and sparse matrix representation. ,, and representation. February 11, 2014 Lecture 7: Overview, Today we will look at:.. If time: A look at representation and fill in. Why do we need numerical s? I think everyone have seen how time consuming

More information

Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015

Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in. NUMERICAL ANALYSIS Spring 2015 Department of Mathematics California State University, Los Angeles Master s Degree Comprehensive Examination in NUMERICAL ANALYSIS Spring 2015 Instructions: Do exactly two problems from Part A AND two

More information

Computational Fluid Dynamics Prof. Sreenivas Jayanti Department of Computer Science and Engineering Indian Institute of Technology, Madras

Computational Fluid Dynamics Prof. Sreenivas Jayanti Department of Computer Science and Engineering Indian Institute of Technology, Madras Computational Fluid Dynamics Prof. Sreenivas Jayanti Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture 46 Tri-diagonal Matrix Algorithm: Derivation In the last

More information

THE QR METHOD A = Q 1 R 1

THE QR METHOD A = Q 1 R 1 THE QR METHOD Given a square matrix A, form its QR factorization, as Then define A = Q 1 R 1 A 2 = R 1 Q 1 Continue this process: for k 1(withA 1 = A), A k = Q k R k A k+1 = R k Q k Then the sequence {A

More information

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 9

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 9 STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 9 1. qr and complete orthogonal factorization poor man s svd can solve many problems on the svd list using either of these factorizations but they

More information

Numerical Linear Algebra Homework Assignment - Week 2

Numerical Linear Algebra Homework Assignment - Week 2 Numerical Linear Algebra Homework Assignment - Week 2 Đoàn Trần Nguyên Tùng Student ID: 1411352 8th October 2016 Exercise 2.1: Show that if a matrix A is both triangular and unitary, then it is diagonal.

More information

Block-tridiagonal matrices

Block-tridiagonal matrices Block-tridiagonal matrices. p.1/31 Block-tridiagonal matrices - where do these arise? - as a result of a particular mesh-point ordering - as a part of a factorization procedure, for example when we compute

More information

An Introduction to Physically Based Modeling: An Introduction to Continuum Dynamics for Computer Graphics

An Introduction to Physically Based Modeling: An Introduction to Continuum Dynamics for Computer Graphics An Introduction to Physically Based Modeling: An Introduction to Continuum Dynamics for Computer Graphics Michael Kass Pixar Please note: This document is 1997 by Michael Kass. This chapter may be freely

More information

Lecture: Face Recognition and Feature Reduction

Lecture: Face Recognition and Feature Reduction Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed in the

More information

Discrete Riccati equations and block Toeplitz matrices

Discrete Riccati equations and block Toeplitz matrices Discrete Riccati equations and block Toeplitz matrices André Ran Vrije Universiteit Amsterdam Leonid Lerer Technion-Israel Institute of Technology Haifa André Ran and Leonid Lerer 1 Discrete algebraic

More information

Numerical Solutions to Partial Differential Equations

Numerical Solutions to Partial Differential Equations Numerical Solutions to Partial Differential Equations Zhiping Li LMAM and School of Mathematical Sciences Peking University The Implicit Schemes for the Model Problem The Crank-Nicolson scheme and θ-scheme

More information

Solution of Linear Equations

Solution of Linear Equations Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass

More information

Part 1. The diffusion equation

Part 1. The diffusion equation Differential Equations FMNN10 Graded Project #3 c G Söderlind 2016 2017 Published 2017-11-27. Instruction in computer lab 2017-11-30/2017-12-06/07. Project due date: Monday 2017-12-11 at 12:00:00. Goals.

More information

JACOBI S ITERATION METHOD

JACOBI S ITERATION METHOD ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes

More information

4. Determinants.

4. Determinants. 4. Determinants 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 2 2 determinant 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 3 3 determinant 4.1.

More information

Multi-Factor Finite Differences

Multi-Factor Finite Differences February 17, 2017 Aims and outline Finite differences for more than one direction The θ-method, explicit, implicit, Crank-Nicolson Iterative solution of discretised equations Alternating directions implicit

More information

Matrix Computations and Semiseparable Matrices

Matrix Computations and Semiseparable Matrices Matrix Computations and Semiseparable Matrices Volume I: Linear Systems Raf Vandebril Department of Computer Science Catholic University of Louvain Marc Van Barel Department of Computer Science Catholic

More information

Advanced methods for ODEs and DAEs

Advanced methods for ODEs and DAEs Advanced methods for ODEs and DAEs Lecture 8: Dirk and Rosenbrock Wanner methods Bojana Rosic, 14 Juni 2016 General implicit Runge Kutta method Runge Kutta method is presented by Butcher table: c1 a11

More information

Systems of Linear ODEs

Systems of Linear ODEs P a g e 1 Systems of Linear ODEs Systems of ordinary differential equations can be solved in much the same way as discrete dynamical systems if the differential equations are linear. We will focus here

More information

Matrix decompositions

Matrix decompositions Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The

More information

MATHEMATICS FOR COMPUTER VISION WEEK 2 LINEAR SYSTEMS. Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year

MATHEMATICS FOR COMPUTER VISION WEEK 2 LINEAR SYSTEMS. Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year 1 MATHEMATICS FOR COMPUTER VISION WEEK 2 LINEAR SYSTEMS Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year 2013-14 OUTLINE OF WEEK 2 Linear Systems and solutions Systems of linear

More information

Introduction to numerical schemes

Introduction to numerical schemes 236861 Numerical Geometry of Images Tutorial 2 Introduction to numerical schemes Heat equation The simple parabolic PDE with the initial values u t = K 2 u 2 x u(0, x) = u 0 (x) and some boundary conditions

More information

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008

More information

Getting Started with Communications Engineering

Getting Started with Communications Engineering 1 Linear algebra is the algebra of linear equations: the term linear being used in the same sense as in linear functions, such as: which is the equation of a straight line. y ax c (0.1) Of course, if we

More information

Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems

Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Lecture 7. Solving linear systems Dr. Noemi Friedman, 09.2.205. Reminder: Instationary heat

More information

Physics 127b: Statistical Mechanics. Renormalization Group: 1d Ising Model. Perturbation expansion

Physics 127b: Statistical Mechanics. Renormalization Group: 1d Ising Model. Perturbation expansion Physics 17b: Statistical Mechanics Renormalization Group: 1d Ising Model The ReNormalization Group (RNG) gives an understanding of scaling and universality, and provides various approximation schemes to

More information

Computational Techniques Prof. Sreenivas Jayanthi. Department of Chemical Engineering Indian institute of Technology, Madras

Computational Techniques Prof. Sreenivas Jayanthi. Department of Chemical Engineering Indian institute of Technology, Madras Computational Techniques Prof. Sreenivas Jayanthi. Department of Chemical Engineering Indian institute of Technology, Madras Module No. # 05 Lecture No. # 24 Gauss-Jordan method L U decomposition method

More information

Throughout these notes we assume V, W are finite dimensional inner product spaces over C.

Throughout these notes we assume V, W are finite dimensional inner product spaces over C. Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal

More information

6. Iterative Methods for Linear Systems. The stepwise approach to the solution...

6. Iterative Methods for Linear Systems. The stepwise approach to the solution... 6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse

More information

Math 471 (Numerical methods) Chapter 3 (second half). System of equations

Math 471 (Numerical methods) Chapter 3 (second half). System of equations Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular

More information

Stabilization and Acceleration of Algebraic Multigrid Method

Stabilization and Acceleration of Algebraic Multigrid Method Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration

More information

Further Mathematical Methods (Linear Algebra)

Further Mathematical Methods (Linear Algebra) Further Mathematical Methods (Linear Algebra) Solutions For The 2 Examination Question (a) For a non-empty subset W of V to be a subspace of V we require that for all vectors x y W and all scalars α R:

More information

Lecture 3: Hamilton-Jacobi-Bellman Equations. Distributional Macroeconomics. Benjamin Moll. Part II of ECON Harvard University, Spring

Lecture 3: Hamilton-Jacobi-Bellman Equations. Distributional Macroeconomics. Benjamin Moll. Part II of ECON Harvard University, Spring Lecture 3: Hamilton-Jacobi-Bellman Equations Distributional Macroeconomics Part II of ECON 2149 Benjamin Moll Harvard University, Spring 2018 1 Outline 1. Hamilton-Jacobi-Bellman equations in deterministic

More information

Data Analysis and Manifold Learning Lecture 2: Properties of Symmetric Matrices and Examples

Data Analysis and Manifold Learning Lecture 2: Properties of Symmetric Matrices and Examples Data Analysis and Manifold Learning Lecture 2: Properties of Symmetric Matrices and Examples Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline

More information

Lecture 4: Linear Algebra 1

Lecture 4: Linear Algebra 1 Lecture 4: Linear Algebra 1 Sourendu Gupta TIFR Graduate School Computational Physics 1 February 12, 2010 c : Sourendu Gupta (TIFR) Lecture 4: Linear Algebra 1 CP 1 1 / 26 Outline 1 Linear problems Motivation

More information

9. Numerical linear algebra background

9. Numerical linear algebra background Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

More information

MIT (Spring 2014)

MIT (Spring 2014) 18.311 MIT (Spring 014) Rodolfo R. Rosales May 6, 014. Problem Set # 08. Due: Last day of lectures. IMPORTANT: Turn in the regular and the special problems stapled in two SEPARATE packages. Print your

More information

Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4

Linear Algebra Section 2.6 : LU Decomposition Section 2.7 : Permutations and transposes Wednesday, February 13th Math 301 Week #4 Linear Algebra Section. : LU Decomposition Section. : Permutations and transposes Wednesday, February 1th Math 01 Week # 1 The LU Decomposition We learned last time that we can factor a invertible matrix

More information

. =. a i1 x 1 + a i2 x 2 + a in x n = b i. a 11 a 12 a 1n a 21 a 22 a 1n. i1 a i2 a in

. =. a i1 x 1 + a i2 x 2 + a in x n = b i. a 11 a 12 a 1n a 21 a 22 a 1n. i1 a i2 a in Vectors and Matrices Continued Remember that our goal is to write a system of algebraic equations as a matrix equation. Suppose we have the n linear algebraic equations a x + a 2 x 2 + a n x n = b a 2

More information

Eigenvalues and Eigenvectors

Eigenvalues and Eigenvectors /88 Chia-Ping Chen Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Eigenvalue Problem /88 Eigenvalue Equation By definition, the eigenvalue equation for matrix

More information

2 Two-Point Boundary Value Problems

2 Two-Point Boundary Value Problems 2 Two-Point Boundary Value Problems Another fundamental equation, in addition to the heat eq. and the wave eq., is Poisson s equation: n j=1 2 u x 2 j The unknown is the function u = u(x 1, x 2,..., x

More information

9.1 Preconditioned Krylov Subspace Methods

9.1 Preconditioned Krylov Subspace Methods Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete

More information

Last Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection

Last Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue

More information

PowerPoints organized by Dr. Michael R. Gustafson II, Duke University

PowerPoints organized by Dr. Michael R. Gustafson II, Duke University Part 3 Chapter 10 LU Factorization PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

More information

Numerical Linear Algebra

Numerical Linear Algebra Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix

More information

Lecture 3 Numerical Solutions to the Transport Equation

Lecture 3 Numerical Solutions to the Transport Equation Lecture 3 Numerical Solutions to the Transport Equation Introduction I There are numerous methods for solving the transport problem numerically. First we must recognize that we need to solve two problems:

More information

Lecture: Face Recognition and Feature Reduction

Lecture: Face Recognition and Feature Reduction Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 11-1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed

More information

Lecture 1: Graphs, Adjacency Matrices, Graph Laplacian

Lecture 1: Graphs, Adjacency Matrices, Graph Laplacian Lecture 1: Graphs, Adjacency Matrices, Graph Laplacian Radu Balan January 31, 2017 G = (V, E) An undirected graph G is given by two pieces of information: a set of vertices V and a set of edges E, G =

More information

AIMS Exercise Set # 1

AIMS Exercise Set # 1 AIMS Exercise Set #. Determine the form of the single precision floating point arithmetic used in the computers at AIMS. What is the largest number that can be accurately represented? What is the smallest

More information

Marching on the BL equations

Marching on the BL equations Marching on the BL equations Harvey S. H. Lam February 10, 2004 Abstract The assumption is that you are competent in Matlab or Mathematica. White s 4-7 starting on page 275 shows us that all generic boundary

More information

EIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4

EIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue

More information

MATH SOLUTIONS TO PRACTICE MIDTERM LECTURE 1, SUMMER Given vector spaces V and W, V W is the vector space given by

MATH SOLUTIONS TO PRACTICE MIDTERM LECTURE 1, SUMMER Given vector spaces V and W, V W is the vector space given by MATH 110 - SOLUTIONS TO PRACTICE MIDTERM LECTURE 1, SUMMER 2009 GSI: SANTIAGO CAÑEZ 1. Given vector spaces V and W, V W is the vector space given by V W = {(v, w) v V and w W }, with addition and scalar

More information

The symmetric linear matrix equation

The symmetric linear matrix equation Electronic Journal of Linear Algebra Volume 9 Volume 9 (00) Article 8 00 The symmetric linear matrix equation Andre CM Ran Martine CB Reurings mcreurin@csvunl Follow this and additional works at: http://repositoryuwyoedu/ela

More information

6.4 Krylov Subspaces and Conjugate Gradients

6.4 Krylov Subspaces and Conjugate Gradients 6.4 Krylov Subspaces and Conjugate Gradients Our original equation is Ax = b. The preconditioned equation is P Ax = P b. When we write P, we never intend that an inverse will be explicitly computed. P

More information

Absolute Value Equations and Inequalities. Use the distance definition of absolute value.

Absolute Value Equations and Inequalities. Use the distance definition of absolute value. Chapter 2 Section 7 2.7 Absolute Value Equations and Inequalities Objectives 1 2 3 4 5 6 Use the distance definition of absolute value. Solve equations of the form ax + b = k, for k > 0. Solve inequalities

More information

Beam Propagation Method Solution to the Seminar Tasks

Beam Propagation Method Solution to the Seminar Tasks Beam Propagation Method Solution to the Seminar Tasks Matthias Zilk The task was to implement a 1D beam propagation method (BPM) that solves the equation z v(xz) = i 2 [ 2k x 2 + (x) k 2 ik2 v(x, z) =

More information

Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers

Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Platzhalter für Bild, Bild auf Titelfolie hinter das Logo einsetzen Introduction to PDEs and Numerical Methods Tutorial 5. Finite difference methods equilibrium equation and iterative solvers Dr. Noemi

More information

Math Linear Algebra II. 1. Inner Products and Norms

Math Linear Algebra II. 1. Inner Products and Norms Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

More information

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12,

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12, LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12, 2000 74 6 Summary Here we summarize the most important information about theoretical and numerical linear algebra. MORALS OF THE STORY: I. Theoretically

More information

CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9

CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9 CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 9 GENE H GOLUB 1 Error Analysis of Gaussian Elimination In this section, we will consider the case of Gaussian elimination and perform a detailed

More information

Numerical Programming I (for CSE)

Numerical Programming I (for CSE) Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let

More information

COMP 558 lecture 18 Nov. 15, 2010

COMP 558 lecture 18 Nov. 15, 2010 Least squares We have seen several least squares problems thus far, and we will see more in the upcoming lectures. For this reason it is good to have a more general picture of these problems and how to

More information

NUMERICAL COMPUTATION IN SCIENCE AND ENGINEERING

NUMERICAL COMPUTATION IN SCIENCE AND ENGINEERING NUMERICAL COMPUTATION IN SCIENCE AND ENGINEERING C. Pozrikidis University of California, San Diego New York Oxford OXFORD UNIVERSITY PRESS 1998 CONTENTS Preface ix Pseudocode Language Commands xi 1 Numerical

More information

Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Numerical Analysis Lecture Notes Peter J Olver 8 Numerical Computation of Eigenvalues In this part, we discuss some practical methods for computing eigenvalues and eigenvectors of matrices Needless to

More information

Hence a root lies between 1 and 2. Since f a is negative and f(x 0 ) is positive The root lies between a and x 0 i.e. 1 and 1.

Hence a root lies between 1 and 2. Since f a is negative and f(x 0 ) is positive The root lies between a and x 0 i.e. 1 and 1. The Bisection method or BOLZANO s method or Interval halving method: Find the positive root of x 3 x = 1 correct to four decimal places by bisection method Let f x = x 3 x 1 Here f 0 = 1 = ve, f 1 = ve,

More information

632 CHAP. 11 EIGENVALUES AND EIGENVECTORS. QR Method

632 CHAP. 11 EIGENVALUES AND EIGENVECTORS. QR Method 632 CHAP 11 EIGENVALUES AND EIGENVECTORS QR Method Suppose that A is a real symmetric matrix In the preceding section we saw how Householder s method is used to construct a similar tridiagonal matrix The

More information

1. Background: The SVD and the best basis (questions selected from Ch. 6- Can you fill in the exercises?)

1. Background: The SVD and the best basis (questions selected from Ch. 6- Can you fill in the exercises?) Math 35 Exam Review SOLUTIONS Overview In this third of the course we focused on linear learning algorithms to model data. summarize: To. Background: The SVD and the best basis (questions selected from

More information

Lecture 12: Detailed balance and Eigenfunction methods

Lecture 12: Detailed balance and Eigenfunction methods Miranda Holmes-Cerfon Applied Stochastic Analysis, Spring 2015 Lecture 12: Detailed balance and Eigenfunction methods Readings Recommended: Pavliotis [2014] 4.5-4.7 (eigenfunction methods and reversibility),

More information

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 12 Luca Trevisan October 3, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 12 Luca Trevisan October 3, 2017 U.C. Berkeley CS94: Beyond Worst-Case Analysis Handout 1 Luca Trevisan October 3, 017 Scribed by Maxim Rabinovich Lecture 1 In which we begin to prove that the SDP relaxation exactly recovers communities

More information

Lecture 11. Linear systems: Cholesky method. Eigensystems: Terminology. Jacobi transformations QR transformation

Lecture 11. Linear systems: Cholesky method. Eigensystems: Terminology. Jacobi transformations QR transformation Lecture Cholesky method QR decomposition Terminology Linear systems: Eigensystems: Jacobi transformations QR transformation Cholesky method: For a symmetric positive definite matrix, one can do an LU decomposition

More information

CS168: The Modern Algorithmic Toolbox Lecture #8: PCA and the Power Iteration Method

CS168: The Modern Algorithmic Toolbox Lecture #8: PCA and the Power Iteration Method CS168: The Modern Algorithmic Toolbox Lecture #8: PCA and the Power Iteration Method Tim Roughgarden & Gregory Valiant April 15, 015 This lecture began with an extended recap of Lecture 7. Recall that

More information

Physics 200 Lecture 4. Integration. Lecture 4. Physics 200 Laboratory

Physics 200 Lecture 4. Integration. Lecture 4. Physics 200 Laboratory Physics 2 Lecture 4 Integration Lecture 4 Physics 2 Laboratory Monday, February 21st, 211 Integration is the flip-side of differentiation in fact, it is often possible to write a differential equation

More information

CS 323: Numerical Analysis and Computing

CS 323: Numerical Analysis and Computing CS 323: Numerical Analysis and Computing MIDTERM #1 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.

More information

MA3232 Numerical Analysis Week 9. James Cooley (1926-)

MA3232 Numerical Analysis Week 9. James Cooley (1926-) MA umerical Analysis Week 9 James Cooley (96-) James Cooley is an American mathematician. His most significant contribution to the world of mathematics and digital signal processing is the Fast Fourier

More information

Mathematics GRADE 8 Teacher Packet

Mathematics GRADE 8 Teacher Packet COMMON CORE Standards Plus Mathematics GRADE 8 Teacher Packet Copyright 01 Learning Plus Associates All Rights Reserved; International Copyright Secured. Permission is hereby granted to teachers to reprint

More information

Section 4.4 Reduction to Symmetric Tridiagonal Form

Section 4.4 Reduction to Symmetric Tridiagonal Form Section 4.4 Reduction to Symmetric Tridiagonal Form Key terms Symmetric matrix conditioning Tridiagonal matrix Similarity transformation Orthogonal matrix Orthogonal similarity transformation properties

More information

MATH 221, Spring Homework 10 Solutions

MATH 221, Spring Homework 10 Solutions MATH 22, Spring 28 - Homework Solutions Due Tuesday, May Section 52 Page 279, Problem 2: 4 λ A λi = and the characteristic polynomial is det(a λi) = ( 4 λ)( λ) ( )(6) = λ 6 λ 2 +λ+2 The solutions to the

More information

Section 6.4. The Gram Schmidt Process

Section 6.4. The Gram Schmidt Process Section 6.4 The Gram Schmidt Process Motivation The procedures in 6 start with an orthogonal basis {u, u,..., u m}. Find the B-coordinates of a vector x using dot products: x = m i= x u i u i u i u i Find

More information

Recall : Eigenvalues and Eigenvectors

Recall : Eigenvalues and Eigenvectors Recall : Eigenvalues and Eigenvectors Let A be an n n matrix. If a nonzero vector x in R n satisfies Ax λx for a scalar λ, then : The scalar λ is called an eigenvalue of A. The vector x is called an eigenvector

More information

Matrix Factorization Reading: Lay 2.5

Matrix Factorization Reading: Lay 2.5 Matrix Factorization Reading: Lay 2.5 October, 20 You have seen that if we know the inverse A of a matrix A, we can easily solve the equation Ax = b. Solving a large number of equations Ax = b, Ax 2 =

More information

c 1 v 1 + c 2 v 2 = 0 c 1 λ 1 v 1 + c 2 λ 1 v 2 = 0

c 1 v 1 + c 2 v 2 = 0 c 1 λ 1 v 1 + c 2 λ 1 v 2 = 0 LECTURE LECTURE 2 0. Distinct eigenvalues I haven t gotten around to stating the following important theorem: Theorem: A matrix with n distinct eigenvalues is diagonalizable. Proof (Sketch) Suppose n =

More information

Lecture 1: From Data to Graphs, Weighted Graphs and Graph Laplacian

Lecture 1: From Data to Graphs, Weighted Graphs and Graph Laplacian Lecture 1: From Data to Graphs, Weighted Graphs and Graph Laplacian Radu Balan February 5, 2018 Datasets diversity: Social Networks: Set of individuals ( agents, actors ) interacting with each other (e.g.,

More information

Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm

Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm Computing Eigenvalues and/or Eigenvectors;Part 2, The Power method and QR-algorithm Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo November 19, 2010 Today

More information

Linear Programming The Simplex Algorithm: Part II Chapter 5

Linear Programming The Simplex Algorithm: Part II Chapter 5 1 Linear Programming The Simplex Algorithm: Part II Chapter 5 University of Chicago Booth School of Business Kipp Martin October 17, 2017 Outline List of Files Key Concepts Revised Simplex Revised Simplex

More information

Finish section 3.6 on Determinants and connections to matrix inverses. Use last week's notes. Then if we have time on Tuesday, begin:

Finish section 3.6 on Determinants and connections to matrix inverses. Use last week's notes. Then if we have time on Tuesday, begin: Math 225-4 Week 7 notes Sections 4-43 vector space concepts Tues Feb 2 Finish section 36 on Determinants and connections to matrix inverses Use last week's notes Then if we have time on Tuesday, begin

More information

Numerical Methods in Matrix Computations

Numerical Methods in Matrix Computations Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices

More information

9. Numerical linear algebra background

9. Numerical linear algebra background Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

More information

ANONSINGULAR tridiagonal linear system of the form

ANONSINGULAR tridiagonal linear system of the form Generalized Diagonal Pivoting Methods for Tridiagonal Systems without Interchanges Jennifer B. Erway, Roummel F. Marcia, and Joseph A. Tyson Abstract It has been shown that a nonsingular symmetric tridiagonal

More information

CS137 Introduction to Scientific Computing Winter Quarter 2004 Solutions to Homework #3

CS137 Introduction to Scientific Computing Winter Quarter 2004 Solutions to Homework #3 CS137 Introduction to Scientific Computing Winter Quarter 2004 Solutions to Homework #3 Felix Kwok February 27, 2004 Written Problems 1. (Heath E3.10) Let B be an n n matrix, and assume that B is both

More information

AN ITERATION. In part as motivation, we consider an iteration method for solving a system of linear equations which has the form x Ax = b

AN ITERATION. In part as motivation, we consider an iteration method for solving a system of linear equations which has the form x Ax = b AN ITERATION In part as motivation, we consider an iteration method for solving a system of linear equations which has the form x Ax = b In this, A is an n n matrix and b R n.systemsof this form arise

More information

Section 4.5 Eigenvalues of Symmetric Tridiagonal Matrices

Section 4.5 Eigenvalues of Symmetric Tridiagonal Matrices Section 4.5 Eigenvalues of Symmetric Tridiagonal Matrices Key Terms Symmetric matrix Tridiagonal matrix Orthogonal matrix QR-factorization Rotation matrices (plane rotations) Eigenvalues We will now complete

More information

Iterative solvers for linear equations

Iterative solvers for linear equations Spectral Graph Theory Lecture 23 Iterative solvers for linear equations Daniel A. Spielman November 26, 2018 23.1 Overview In this and the next lecture, I will discuss iterative algorithms for solving

More information

Linear Algebra Short Course Lecture 4

Linear Algebra Short Course Lecture 4 Linear Algebra Short Course Lecture 4 Matthew J. Holland matthew-h@is.naist.jp Mathematical Informatics Lab Graduate School of Information Science, NAIST 1 Some useful references Finite dimensional inner-product

More information

Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018

Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018 Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x

More information

Pr[X = s Y = t] = Pr[X = s] Pr[Y = t]

Pr[X = s Y = t] = Pr[X = s] Pr[Y = t] Homework 4 By: John Steinberger Problem 1. Recall that a real n n matrix A is positive semidefinite if A is symmetric and x T Ax 0 for all x R n. Assume A is a real n n matrix. Show TFAE 1 : (a) A is positive

More information

Matrix Algorithms. Volume II: Eigensystems. G. W. Stewart H1HJ1L. University of Maryland College Park, Maryland

Matrix Algorithms. Volume II: Eigensystems. G. W. Stewart H1HJ1L. University of Maryland College Park, Maryland Matrix Algorithms Volume II: Eigensystems G. W. Stewart University of Maryland College Park, Maryland H1HJ1L Society for Industrial and Applied Mathematics Philadelphia CONTENTS Algorithms Preface xv xvii

More information

Butcher tableau Can summarize an s + 1 stage Runge Kutta method using a triangular grid of coefficients

Butcher tableau Can summarize an s + 1 stage Runge Kutta method using a triangular grid of coefficients AM 205: lecture 13 Last time: ODE convergence and stability, Runge Kutta methods Today: the Butcher tableau, multi-step methods, boundary value problems Butcher tableau Can summarize an s + 1 stage Runge

More information