Sparse Matrix Techniques for MCAO

Size: px
Start display at page:

Download "Sparse Matrix Techniques for MCAO"

Transcription

1 Sparse Matrix Techniques for MCAO Luc Gilles Michigan Technological University, ECE Department Brent Ellerbroek Gemini Observatory Curt Vogel Montana State University, Math Sciences November 25, 2002 L. Gilles, CfAO Fall Retreat 2002 p.1/21

2 Wavefront Estimation from Idealized WFS Data - Open-loop WFS: s = Gϕ + η - η and ϕ random with known statistics C η = η η T and C ϕ = ϕ ϕ T ˆϕ = G s (noise-weighted pseudo-inverse) ˆϕ = arg min ϕ J(ϕ) J(ϕ) = 1 2 [ η 2 C 1 η + ϕ 2 C 1 ϕ ] = 1 2 [ η, C 1 η η + ϕ, Cϕ 1 ϕ ] ˆϕ = G s with G = ( G T C 1 η G + Cϕ 1 ) 1 G T Cη 1 - Block-layered structure for MCAO - (SNR) 2 = Gϕ 2 / η 2 = Gϕ 2 /(2 n z σ 2 ) L. Gilles, CfAO Fall Retreat 2002 p.2/21

3 Key Approximations to Inverse Turbulence Covariance Approximation #1 - C ϕ is block diagonal (different layers are statistically independent) - For each turbulent layer l, C ϕl is BTTB (phase structure function) - Approximate BBTB matrix by BCCB matrix (diagonalized by DFT) - C ϕl BCCB(τ l ) = F 1 diag(ˆτ l )F where ˆτ l = Fτ l (eigenvalues) - ˆτ l = c l κ 11/3 (Kolmogorov PSD) C 1 ϕ l BCCB(Λ l ) = F 1 diag(λ l ) F, λ l = ˆτ 1 l = c 1 l κ 11/3 L. Gilles, CfAO Fall Retreat 2002 p.3/21

4 Key Approximations to Inverse Turbulence Covariance - BCCB(Λ l ) is a full matrix. Approximation #2 - Entries Λ l rapidly decay to small values - Approximate BCCB(Λ l ) by Sl 2 where S l = c l S and S is the discrete Laplacian matrix with periodic boundary conditions S = BCCB(v) = F 1 diag(ˆv)f (sparse BCCB) ˆv = 4 [ sin 2 (πκ x x)/ x 2 + sin 2 (πκ y y)/ y 2] x, y 0 4π2 κ 2 - S = S y I x + I y S x where S y and S x are 1D versions L. Gilles, CfAO Fall Retreat 2002 p.4/21

5 Key Approximations to Inverse Turbulence Covariance Sparsity Patterns in 1D [S u] i = ( u i 1 + 2u i u i+1 ) / x 2 S = circ(v), v = (2, 1, 0,, 0, 1) T / x 2 = (2 e 0 e 1 e n 1 )/ x 2 S = F 1 diag(ˆv)f, ˆv = Fv = (2 ê 0 ê 1 ê n 1 )/ x 2 ˆv = 4 sin 2 (πκ x)/ x 2 (eigenvalues) x 0 4π 2 κ 2 L. Gilles, CfAO Fall Retreat 2002 p.5/21

6 Key Approximations to Inverse Turbulence Covariance Eigenvalues λ (1), λ (2) (biharmo) λ (2) (power 11/3) λ (1) c (1) = IDFT[ λ (1) ], c (2) = IDFT[ λ (2) ] C (1) = circ[ c (1) ] C (2) = circ[ c (2) ], Threshold = % Fill % Fill L. Gilles, CfAO Fall Retreat 2002 p.6/21

7 MCAO Minimum Variance Reconstructor Problem: find optimal actuator commands â minimizing MCAO wide-field error metric W â = Rs R = arg min J(R) R J(R) = ɛ 2 W = ɛ T Wɛ ɛ = H a â H ϕ ϕ (aperture-plane residual phase) R = H a }{{} F itting H ϕ G }{{} Estimation G = ( G T Cη 1 G }{{} + Cϕ 1 }{{} ) 1 G T Cη 1 Sparse+low rank (LGS) Sparse approx H a = ( H at W H a + low-rank ) 1 H at W L. Gilles, CfAO Fall Retreat 2002 p.7/21

8 Multigrid (MG) Methods - Can sometimes be used as stand-alone system solvers. - Can be used as preconditioners. - Rely on multiple scales (grid sizes) inherent in certain problems. - Need smoother which damps out high-frequency components of error on fine grids. - Classical Gauss-Seidel iteration works well for Laplace s equation. - Remaining low frequency error is well-represented on coarser grids. - Are recursive versions of the following 2-grid scheme. L. Gilles, CfAO Fall Retreat 2002 p.8/21

9 2-Grid Scheme x h S(x h, y h,...) r h A h x h y h x h x h + e h x h S(x h, y h,...) Restrict r H Ih Hr h Interpolate e h IH h e H Solve A H e H = r H - S(v, w,...) denotes application of smoother to solve Ax = w with initial guess x = v. - To obtain MG V-cycle, apply 2-grid scheme recursively. Carry out Solve step with (e H, r H ) in place of (x h, y h ). L. Gilles, CfAO Fall Retreat 2002 p.9/21

10 Multigrid (MG) Methods - Inter-grid transfers (restriction, or up-binning, and interpolation, or down-binning) are cheap. - Cost is typically dominated by smoother application on finest grid. - Choice of smoother is problem-dependent. - Block (i.e., layer-oriented) symmetric Gauss-Seidel (B-SGS) works well for MCAO estimation step. - FFT-based modified Richardson iteration works well for Ex-AO estimation. L. Gilles, CfAO Fall Retreat 2002 p.10/21

11 Block Gauss-Seidel Smoother Based on block L + D + U splitting A = A A A A n1... A n,n A nn }{{}}{{} L 0 A A 1n A n 1,n } {{ 0 } U D L. Gilles, CfAO Fall Retreat 2002 p.11/21

12 Block Gauss-Seidel Smoother Ax = b is equivalent to (L + D)x = b Ux. This motivates the block forward iteration (L + D)x k+1 = b Ux k, k = 0, 1,.... Similarly, we obtain the block backward interation (D + U)x k+1 = b Lx k, k = 0, 1,.... Block symmetric Gauss-Seidel (B-SGS) is obtained by interweaving forward and backward iterations. L. Gilles, CfAO Fall Retreat 2002 p.12/21

13 Efficient PCG Solver - MG preconditioned CG Estimation Step - Block SGS smoother: requires only inversion of diagonal blocks. Implemented using reordering + full Cholesky factorization. Fitting Step - Incomplete Cholesky preconditioned CG. Incomplete Cholesky applied to full sparse matrix without reordering. L. Gilles, CfAO Fall Retreat 2002 p.13/21

14 Preliminary Results Algorithm tested against conventional matrix multiply reconstructors on 8m class problems with degrees of freedom 32m class problems with degrees of freedom solvable in Matlab with 2-3Gb memory Convergence obtained in 2-10 iterations - Convergence rate a strong function of WFS noise level - Weak function of problem dimensionality, NGS vs. LGS MCAO L. Gilles, CfAO Fall Retreat 2002 p.14/21

15 Sample MCAO Problem Dimensionality Aperture diameter (m) WFS measurements Turbulence phase points DM actuators L. Gilles, CfAO Fall Retreat 2002 p.15/21

16 Sample MCAO Problem Dimensionality Estimation matrix to be inverted. 6 layers, 5 NGS s. D=16m. Phase screens L. Gilles, CfAO Fall Retreat 2002 p.16/21

17 Fig.1 Top layer diagonal block of estimation matrix. D=16m. Phase screens L. Gilles, CfAO Fall Retreat 2002 p.17/21

18 Fig.2 Fitting matrix (3 DM s, 932 actuators/dm). D=16m. L. Gilles, CfAO Fall Retreat 2002 p.18/21

19 Fig.3 Estimation, 6-layer profile, 5 WFSs using 5 NGS s. FoV diameter 100 arcsec, 1 V-cycle/CG iteration, 1 SGS iter/grid level, SNR = 20, r 0 = 25cm, x = r Estimation Error Norm Averaged over FoV 32m 16m 8m 10 0 CG Estimation Residual Norm 32m 16m 8m CG Estimation Iteration CG Estimation Iteration 10 0 Direct Solve vs CG (8m) RMS Estimation Error (32m) x CG Estimation Iteration 0 L. Gilles, CfAO Fall Retreat 2002 p.19/21

20 Fig.4 Fitting after 20 CG estimation iterations. Average over array of 5 5 observation directions. FoV diameter 100 arcsec. Incomplete Cholesky preconditioning, regularization parameter α = 1e-5. Error Norm (32m) Averaged over FoV 0Residual 10 3 DMs 2 DMs 1 DM Error Norm (16m) Averaged over FoV 0Residual 10 3 DMs 2 DMs 1 DM CG Fitting Iteration CG Fitting Iteration Error Norm (8m) Averaged over FoV 0Residual 10 3 DMs 2 DMs 1 DM 10 0 Residual Error Norm Averaged over FoV m 16m 8m CG Fitting Iteration CG Estimation Iteration L. Gilles, CfAO Fall Retreat 2002 p.20/21

21 Fig.5 Fitting after 20 CG estimation iterations. Average over array of 5 5 observation directions. FoV diameter 100 arcsec. Incomplete Cholesky preconditioning, regularization parameter α = 1e Direct Solve vs CG (8m) 10 0 CG Residual Norm (8m) DMs 2 DMs 1 DM DMs 2 DMs 1 DM CG Fitting Iteration CG Fitting Iteration 10 0 CG Residual Norm (16m) RMS Residual Error (32m, 2DMs) x DMs 2 DMs 1 DM CG Fitting Iteration 0 L. Gilles, CfAO Fall Retreat 2002 p.21/21

Wavefront reconstruction for adaptive optics. Marcos van Dam and Richard Clare W.M. Keck Observatory

Wavefront reconstruction for adaptive optics. Marcos van Dam and Richard Clare W.M. Keck Observatory Wavefront reconstruction for adaptive optics Marcos van Dam and Richard Clare W.M. Keck Observatory Friendly people We borrowed slides from the following people: Lisa Poyneer Luc Gilles Curt Vogel Corinne

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 24: Preconditioning and Multigrid Solver Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 5 Preconditioning Motivation:

More information

Numerical Methods I Non-Square and Sparse Linear Systems

Numerical Methods I Non-Square and Sparse Linear Systems Numerical Methods I Non-Square and Sparse Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 25th, 2014 A. Donev (Courant

More information

JACOBI S ITERATION METHOD

JACOBI S ITERATION METHOD ITERATION METHODS These are methods which compute a sequence of progressively accurate iterates to approximate the solution of Ax = b. We need such methods for solving many large linear systems. Sometimes

More information

9.1 Preconditioned Krylov Subspace Methods

9.1 Preconditioned Krylov Subspace Methods Chapter 9 PRECONDITIONING 9.1 Preconditioned Krylov Subspace Methods 9.2 Preconditioned Conjugate Gradient 9.3 Preconditioned Generalized Minimal Residual 9.4 Relaxation Method Preconditioners 9.5 Incomplete

More information

1. Fast Iterative Solvers of SLE

1. Fast Iterative Solvers of SLE 1. Fast Iterative Solvers of crucial drawback of solvers discussed so far: they become slower if we discretize more accurate! now: look for possible remedies relaxation: explicit application of the multigrid

More information

Lecture 13: Basic Concepts of Wavefront Reconstruction. Astro 289

Lecture 13: Basic Concepts of Wavefront Reconstruction. Astro 289 Lecture 13: Basic Concepts of Wavefront Reconstruction Astro 289 Claire Max February 25, 2016 Based on slides by Marcos van Dam and Lisa Poyneer CfAO Summer School Page 1 Outline System matrix, H: from

More information

Sparse Matrix Methods for Large-Scale. Closed-Loop Adaptive Optics

Sparse Matrix Methods for Large-Scale. Closed-Loop Adaptive Optics Sparse Matrix Methods for Large-Scale Closed-Loop Adaptive Optics Luc Gilles lgilles@mtu.edu Michigan Technological University, ECE Department January 23, 24 L.Gilles, IPAM Workshop, UCLA p.1/37 Large-Scale

More information

Lecture 18 Classical Iterative Methods

Lecture 18 Classical Iterative Methods Lecture 18 Classical Iterative Methods MIT 18.335J / 6.337J Introduction to Numerical Methods Per-Olof Persson November 14, 2006 1 Iterative Methods for Linear Systems Direct methods for solving Ax = b,

More information

Fourier domain preconditioned conjugate gradient algorithm for atmospheric tomography

Fourier domain preconditioned conjugate gradient algorithm for atmospheric tomography Fourier domain preconditioned conjugate gradient algorithm for atmospheric tomography Qiang Yang, Curtis R. Vogel, and Brent L. Ellerbroek By atmospheric tomography we mean the estimation of a layered

More information

Wavefront Reconstruction

Wavefront Reconstruction Wavefront Reconstruction Lisa A. Poyneer Lawrence Livermore ational Laboratory Center for Adaptive Optics 2008 Summer School University of California, Santa Cruz August 5, 2008 LLL-PRES-405137 This work

More information

The Conjugate Gradient Method

The Conjugate Gradient Method The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large

More information

A FREQUENCY DEPENDENT PRECONDITIONED WAVELET METHOD FOR ATMOSPHERIC TOMOGRAPHY

A FREQUENCY DEPENDENT PRECONDITIONED WAVELET METHOD FOR ATMOSPHERIC TOMOGRAPHY Florence, Italy. May 2013 ISBN: 978-88-908876-0-4 DOI: 10.12839/AO4ELT3.13433 A FREQUENCY DEPENDENT PRECONDITIONED WAVELET METHOD FOR ATMOSPHERIC TOMOGRAPHY Mykhaylo Yudytskiy 1a, Tapio Helin 2b, and Ronny

More information

Kasetsart University Workshop. Multigrid methods: An introduction

Kasetsart University Workshop. Multigrid methods: An introduction Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available

More information

Adaptive algebraic multigrid methods in lattice computations

Adaptive algebraic multigrid methods in lattice computations Adaptive algebraic multigrid methods in lattice computations Karsten Kahl Bergische Universität Wuppertal January 8, 2009 Acknowledgements Matthias Bolten, University of Wuppertal Achi Brandt, Weizmann

More information

Motivation: Sparse matrices and numerical PDE's

Motivation: Sparse matrices and numerical PDE's Lecture 20: Numerical Linear Algebra #4 Iterative methods and Eigenproblems Outline 1) Motivation: beyond LU for Ax=b A little PDE's and sparse matrices A) Temperature Equation B) Poisson Equation 2) Splitting

More information

Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White

Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Introduction to Simulation - Lecture 2 Boundary Value Problems - Solving 3-D Finite-Difference problems Jacob White Thanks to Deepak Ramaswamy, Michal Rewienski, and Karen Veroy Outline Reminder about

More information

Algebraic Multigrid as Solvers and as Preconditioner

Algebraic Multigrid as Solvers and as Preconditioner Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven

More information

Preconditioning Techniques Analysis for CG Method

Preconditioning Techniques Analysis for CG Method Preconditioning Techniques Analysis for CG Method Huaguang Song Department of Computer Science University of California, Davis hso@ucdavis.edu Abstract Matrix computation issue for solve linear system

More information

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix

Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008

More information

Solving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners

Solving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners Solving Symmetric Indefinite Systems with Symmetric Positive Definite Preconditioners Eugene Vecharynski 1 Andrew Knyazev 2 1 Department of Computer Science and Engineering University of Minnesota 2 Department

More information

Multigrid absolute value preconditioning

Multigrid absolute value preconditioning Multigrid absolute value preconditioning Eugene Vecharynski 1 Andrew Knyazev 2 (speaker) 1 Department of Computer Science and Engineering University of Minnesota 2 Department of Mathematical and Statistical

More information

Computational Linear Algebra

Computational Linear Algebra Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 4: Iterative Methods PD

More information

FEM and Sparse Linear System Solving

FEM and Sparse Linear System Solving FEM & sparse system solving, Lecture 7, Nov 3, 2017 1/46 Lecture 7, Nov 3, 2015: Introduction to Iterative Solvers: Stationary Methods http://people.inf.ethz.ch/arbenz/fem16 Peter Arbenz Computer Science

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 11 Partial Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002.

More information

4.6 Iterative Solvers for Linear Systems

4.6 Iterative Solvers for Linear Systems 4.6 Iterative Solvers for Linear Systems Why use iterative methods? Virtually all direct methods for solving Ax = b require O(n 3 ) floating point operations. In practical applications the matrix A often

More information

Conjugate Gradient Method

Conjugate Gradient Method Conjugate Gradient Method direct and indirect methods positive definite linear systems Krylov sequence spectral analysis of Krylov sequence preconditioning Prof. S. Boyd, EE364b, Stanford University Three

More information

Lab 1: Iterative Methods for Solving Linear Systems

Lab 1: Iterative Methods for Solving Linear Systems Lab 1: Iterative Methods for Solving Linear Systems January 22, 2017 Introduction Many real world applications require the solution to very large and sparse linear systems where direct methods such as

More information

PDE Solvers for Fluid Flow

PDE Solvers for Fluid Flow PDE Solvers for Fluid Flow issues and algorithms for the Streaming Supercomputer Eran Guendelman February 5, 2002 Topics Equations for incompressible fluid flow 3 model PDEs: Hyperbolic, Elliptic, Parabolic

More information

Introduction and Stationary Iterative Methods

Introduction and Stationary Iterative Methods Introduction and C. T. Kelley NC State University tim kelley@ncsu.edu Research Supported by NSF, DOE, ARO, USACE DTU ITMAN, 2011 Outline Notation and Preliminaries General References What you Should Know

More information

Chapter 7 Iterative Techniques in Matrix Algebra

Chapter 7 Iterative Techniques in Matrix Algebra Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition

More information

EXAMPLES OF CLASSICAL ITERATIVE METHODS

EXAMPLES OF CLASSICAL ITERATIVE METHODS EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline

More information

Key words. preconditioned conjugate gradient method, saddle point problems, optimal control of PDEs, control and state constraints, multigrid method

Key words. preconditioned conjugate gradient method, saddle point problems, optimal control of PDEs, control and state constraints, multigrid method PRECONDITIONED CONJUGATE GRADIENT METHOD FOR OPTIMAL CONTROL PROBLEMS WITH CONTROL AND STATE CONSTRAINTS ROLAND HERZOG AND EKKEHARD SACHS Abstract. Optimality systems and their linearizations arising in

More information

Multigrid Methods and their application in CFD

Multigrid Methods and their application in CFD Multigrid Methods and their application in CFD Michael Wurst TU München 16.06.2009 1 Multigrid Methods Definition Multigrid (MG) methods in numerical analysis are a group of algorithms for solving differential

More information

arxiv: v1 [math.na] 11 Jul 2011

arxiv: v1 [math.na] 11 Jul 2011 Multigrid Preconditioner for Nonconforming Discretization of Elliptic Problems with Jump Coefficients arxiv:07.260v [math.na] Jul 20 Blanca Ayuso De Dios, Michael Holst 2, Yunrong Zhu 2, and Ludmil Zikatanov

More information

LINEAR SYSTEMS (11) Intensive Computation

LINEAR SYSTEMS (11) Intensive Computation LINEAR SYSTEMS () Intensive Computation 27-8 prof. Annalisa Massini Viviana Arrigoni EXACT METHODS:. GAUSSIAN ELIMINATION. 2. CHOLESKY DECOMPOSITION. ITERATIVE METHODS:. JACOBI. 2. GAUSS-SEIDEL 2 CHOLESKY

More information

Preface to the Second Edition. Preface to the First Edition

Preface to the Second Edition. Preface to the First Edition n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................

More information

Contents. Preface... xi. Introduction...

Contents. Preface... xi. Introduction... Contents Preface... xi Introduction... xv Chapter 1. Computer Architectures... 1 1.1. Different types of parallelism... 1 1.1.1. Overlap, concurrency and parallelism... 1 1.1.2. Temporal and spatial parallelism

More information

Stabilization and Acceleration of Algebraic Multigrid Method

Stabilization and Acceleration of Algebraic Multigrid Method Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration

More information

Incomplete Cholesky preconditioners that exploit the low-rank property

Incomplete Cholesky preconditioners that exploit the low-rank property anapov@ulb.ac.be ; http://homepages.ulb.ac.be/ anapov/ 1 / 35 Incomplete Cholesky preconditioners that exploit the low-rank property (theory and practice) Artem Napov Service de Métrologie Nucléaire, Université

More information

Iterative Methods for Linear Systems

Iterative Methods for Linear Systems Iterative Methods for Linear Systems 1. Introduction: Direct solvers versus iterative solvers In many applications we have to solve a linear system Ax = b with A R n n and b R n given. If n is large the

More information

Solving PDEs with CUDA Jonathan Cohen

Solving PDEs with CUDA Jonathan Cohen Solving PDEs with CUDA Jonathan Cohen jocohen@nvidia.com NVIDIA Research PDEs (Partial Differential Equations) Big topic Some common strategies Focus on one type of PDE in this talk Poisson Equation Linear

More information

Structured Linear Algebra Problems in Adaptive Optics Imaging

Structured Linear Algebra Problems in Adaptive Optics Imaging Structured Linear Algebra Problems in Adaptive Optics Imaging Johnathan M. Bardsley, Sarah Knepper, and James Nagy Abstract A main problem in adaptive optics is to reconstruct the phase spectrum given

More information

Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58

Background. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58 Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms

More information

Notes for CS542G (Iterative Solvers for Linear Systems)

Notes for CS542G (Iterative Solvers for Linear Systems) Notes for CS542G (Iterative Solvers for Linear Systems) Robert Bridson November 20, 2007 1 The Basics We re now looking at efficient ways to solve the linear system of equations Ax = b where in this course,

More information

HOMEWORK 10 SOLUTIONS

HOMEWORK 10 SOLUTIONS HOMEWORK 10 SOLUTIONS MATH 170A Problem 0.1. Watkins 8.3.10 Solution. The k-th error is e (k) = G k e (0). As discussed before, that means that e (k+j) ρ(g) k, i.e., the norm of the error is approximately

More information

SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA

SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS. Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 1 SOLVING SPARSE LINEAR SYSTEMS OF EQUATIONS Chao Yang Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA, USA 2 OUTLINE Sparse matrix storage format Basic factorization

More information

Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes

Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov Processes Elena Virnik, TU BERLIN Algebraic Multigrid Preconditioners for Computing Stationary Distributions of Markov

More information

6. Iterative Methods for Linear Systems. The stepwise approach to the solution...

6. Iterative Methods for Linear Systems. The stepwise approach to the solution... 6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse

More information

Lecture 11: CMSC 878R/AMSC698R. Iterative Methods An introduction. Outline. Inverse, LU decomposition, Cholesky, SVD, etc.

Lecture 11: CMSC 878R/AMSC698R. Iterative Methods An introduction. Outline. Inverse, LU decomposition, Cholesky, SVD, etc. Lecture 11: CMSC 878R/AMSC698R Iterative Methods An introduction Outline Direct Solution of Linear Systems Inverse, LU decomposition, Cholesky, SVD, etc. Iterative methods for linear systems Why? Matrix

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)

AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical

More information

Multigrid solvers for equations arising in implicit MHD simulations

Multigrid solvers for equations arising in implicit MHD simulations Multigrid solvers for equations arising in implicit MHD simulations smoothing Finest Grid Mark F. Adams Department of Applied Physics & Applied Mathematics Columbia University Ravi Samtaney PPPL Achi Brandt

More information

Lecture4 INF-MAT : 5. Fast Direct Solution of Large Linear Systems

Lecture4 INF-MAT : 5. Fast Direct Solution of Large Linear Systems Lecture4 INF-MAT 4350 2010: 5. Fast Direct Solution of Large Linear Systems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 16, 2010 Test Matrix

More information

Multigrid Methods for Discretized PDE Problems

Multigrid Methods for Discretized PDE Problems Towards Metods for Discretized PDE Problems Institute for Applied Matematics University of Heidelberg Feb 1-5, 2010 Towards Outline A model problem Solution of very large linear systems Iterative Metods

More information

Parallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1

Parallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1 Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations

More information

Scientific Computing: Solving Linear Systems

Scientific Computing: Solving Linear Systems Scientific Computing: Solving Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 September 17th and 24th, 2015 A. Donev (Courant

More information

Efficient smoothers for all-at-once multigrid methods for Poisson and Stokes control problems

Efficient smoothers for all-at-once multigrid methods for Poisson and Stokes control problems Efficient smoothers for all-at-once multigrid methods for Poisson and Stoes control problems Stefan Taacs stefan.taacs@numa.uni-linz.ac.at, WWW home page: http://www.numa.uni-linz.ac.at/~stefant/j3362/

More information

Numerical Analysis: Solutions of System of. Linear Equation. Natasha S. Sharma, PhD

Numerical Analysis: Solutions of System of. Linear Equation. Natasha S. Sharma, PhD Mathematical Question we are interested in answering numerically How to solve the following linear system for x Ax = b? where A is an n n invertible matrix and b is vector of length n. Notation: x denote

More information

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations

Sparse Linear Systems. Iterative Methods for Sparse Linear Systems. Motivation for Studying Sparse Linear Systems. Partial Differential Equations Sparse Linear Systems Iterative Methods for Sparse Linear Systems Matrix Computations and Applications, Lecture C11 Fredrik Bengzon, Robert Söderlund We consider the problem of solving the linear system

More information

Computational Methods. Systems of Linear Equations

Computational Methods. Systems of Linear Equations Computational Methods Systems of Linear Equations Manfred Huber 2010 1 Systems of Equations Often a system model contains multiple variables (parameters) and contains multiple equations Multiple equations

More information

Iterative Methods and Multigrid

Iterative Methods and Multigrid Iterative Methods and Multigrid Part 1: Introduction to Multigrid 1 12/02/09 MG02.prz Error Smoothing 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Initial Solution=-Error 0 10 20 30 40 50 60 70 80 90 100 DCT:

More information

Notes on PCG for Sparse Linear Systems

Notes on PCG for Sparse Linear Systems Notes on PCG for Sparse Linear Systems Luca Bergamaschi Department of Civil Environmental and Architectural Engineering University of Padova e-mail luca.bergamaschi@unipd.it webpage www.dmsa.unipd.it/

More information

6.4 Krylov Subspaces and Conjugate Gradients

6.4 Krylov Subspaces and Conjugate Gradients 6.4 Krylov Subspaces and Conjugate Gradients Our original equation is Ax = b. The preconditioned equation is P Ax = P b. When we write P, we never intend that an inverse will be explicitly computed. P

More information

Lecture 8: Fast Linear Solvers (Part 7)

Lecture 8: Fast Linear Solvers (Part 7) Lecture 8: Fast Linear Solvers (Part 7) 1 Modified Gram-Schmidt Process with Reorthogonalization Test Reorthogonalization If Av k 2 + δ v k+1 2 = Av k 2 to working precision. δ = 10 3 2 Householder Arnoldi

More information

9. Iterative Methods for Large Linear Systems

9. Iterative Methods for Large Linear Systems EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1

More information

Solving Sparse Linear Systems: Iterative methods

Solving Sparse Linear Systems: Iterative methods Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccs Lecture Notes for Unit VII Sparse Matrix Computations Part 2: Iterative Methods Dianne P. O Leary c 2008,2010

More information

Solving Sparse Linear Systems: Iterative methods

Solving Sparse Linear Systems: Iterative methods Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 2: Iterative Methods Dianne P. O Leary

More information

A robust multilevel approximate inverse preconditioner for symmetric positive definite matrices

A robust multilevel approximate inverse preconditioner for symmetric positive definite matrices DICEA DEPARTMENT OF CIVIL, ENVIRONMENTAL AND ARCHITECTURAL ENGINEERING PhD SCHOOL CIVIL AND ENVIRONMENTAL ENGINEERING SCIENCES XXX CYCLE A robust multilevel approximate inverse preconditioner for symmetric

More information

Solving Ax = b, an overview. Program

Solving Ax = b, an overview. Program Numerical Linear Algebra Improving iterative solvers: preconditioning, deflation, numerical software and parallelisation Gerard Sleijpen and Martin van Gijzen November 29, 27 Solving Ax = b, an overview

More information

Numerical Programming I (for CSE)

Numerical Programming I (for CSE) Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let

More information

2 Regularized Image Reconstruction for Compressive Imaging and Beyond

2 Regularized Image Reconstruction for Compressive Imaging and Beyond EE 367 / CS 448I Computational Imaging and Display Notes: Compressive Imaging and Regularized Image Reconstruction (lecture ) Gordon Wetzstein gordon.wetzstein@stanford.edu This document serves as a supplement

More information

Lecture 9: Numerical Linear Algebra Primer (February 11st)

Lecture 9: Numerical Linear Algebra Primer (February 11st) 10-725/36-725: Convex Optimization Spring 2015 Lecture 9: Numerical Linear Algebra Primer (February 11st) Lecturer: Ryan Tibshirani Scribes: Avinash Siravuru, Guofan Wu, Maosheng Liu Note: LaTeX template

More information

Linear algebra issues in Interior Point methods for bound-constrained least-squares problems

Linear algebra issues in Interior Point methods for bound-constrained least-squares problems Linear algebra issues in Interior Point methods for bound-constrained least-squares problems Stefania Bellavia Dipartimento di Energetica S. Stecco Università degli Studi di Firenze Joint work with Jacek

More information

Scientific Computing

Scientific Computing Scientific Computing Direct solution methods Martin van Gijzen Delft University of Technology October 3, 2018 1 Program October 3 Matrix norms LU decomposition Basic algorithm Cost Stability Pivoting Pivoting

More information

Robust solution of Poisson-like problems with aggregation-based AMG

Robust solution of Poisson-like problems with aggregation-based AMG Robust solution of Poisson-like problems with aggregation-based AMG Yvan Notay Université Libre de Bruxelles Service de Métrologie Nucléaire Paris, January 26, 215 Supported by the Belgian FNRS http://homepages.ulb.ac.be/

More information

2.29 Numerical Fluid Mechanics Spring 2015 Lecture 9

2.29 Numerical Fluid Mechanics Spring 2015 Lecture 9 Spring 2015 Lecture 9 REVIEW Lecture 8: Direct Methods for solving (linear) algebraic equations Gauss Elimination LU decomposition/factorization Error Analysis for Linear Systems and Condition Numbers

More information

Name: INSERT YOUR NAME HERE. Due to dropbox by 6pm PDT, Wednesday, December 14, 2011

Name: INSERT YOUR NAME HERE. Due to dropbox by 6pm PDT, Wednesday, December 14, 2011 AMath 584 Name: INSERT YOUR NAME HERE Take-home Final UWNetID: INSERT YOUR NETID Due to dropbox by 6pm PDT, Wednesday, December 14, 2011 The main part of the assignment (Problems 1 3) is worth 80 points.

More information

Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018

Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer Homework 3 Due: Tuesday, July 3, 2018 Math/Phys/Engr 428, Math 529/Phys 528 Numerical Methods - Summer 28. (Vector and Matrix Norms) Homework 3 Due: Tuesday, July 3, 28 Show that the l vector norm satisfies the three properties (a) x for x

More information

Elliptic Problems / Multigrid. PHY 604: Computational Methods for Physics and Astrophysics II

Elliptic Problems / Multigrid. PHY 604: Computational Methods for Physics and Astrophysics II Elliptic Problems / Multigrid Summary of Hyperbolic PDEs We looked at a simple linear and a nonlinear scalar hyperbolic PDE There is a speed associated with the change of the solution Explicit methods

More information

Topics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems

Topics. The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems Topics The CG Algorithm Algorithmic Options CG s Two Main Convergence Theorems What about non-spd systems? Methods requiring small history Methods requiring large history Summary of solvers 1 / 52 Conjugate

More information

Preconditioning techniques to accelerate the convergence of the iterative solution methods

Preconditioning techniques to accelerate the convergence of the iterative solution methods Note Preconditioning techniques to accelerate the convergence of the iterative solution methods Many issues related to iterative solution of linear systems of equations are contradictory: numerical efficiency

More information

Geometric Multigrid Methods

Geometric Multigrid Methods Geometric Multigrid Methods Susanne C. Brenner Department of Mathematics and Center for Computation & Technology Louisiana State University IMA Tutorial: Fast Solution Techniques November 28, 2010 Ideas

More information

Iterative Methods for Solving A x = b

Iterative Methods for Solving A x = b Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

Computational Linear Algebra

Computational Linear Algebra Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 3: Iterative Methods PD

More information

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark

LU Factorization. Marco Chiarandini. DM559 Linear and Integer Programming. Department of Mathematics & Computer Science University of Southern Denmark DM559 Linear and Integer Programming LU Factorization Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [Based on slides by Lieven Vandenberghe, UCLA] Outline

More information

Outline. 1 Partial Differential Equations. 2 Numerical Methods for PDEs. 3 Sparse Linear Systems

Outline. 1 Partial Differential Equations. 2 Numerical Methods for PDEs. 3 Sparse Linear Systems Outline Partial Differential Equations Numerical Methods for PDEs Sparse Linear Systems 1 Partial Differential Equations 2 Numerical Methods for PDEs 3 Sparse Linear Systems Michael T Heath Scientific

More information

Lecture 17: Iterative Methods and Sparse Linear Algebra

Lecture 17: Iterative Methods and Sparse Linear Algebra Lecture 17: Iterative Methods and Sparse Linear Algebra David Bindel 25 Mar 2014 Logistics HW 3 extended to Wednesday after break HW 4 should come out Monday after break Still need project description

More information

Journal of Computational and Applied Mathematics. Multigrid method for solving convection-diffusion problems with dominant convection

Journal of Computational and Applied Mathematics. Multigrid method for solving convection-diffusion problems with dominant convection Journal of Computational and Applied Mathematics 226 (2009) 77 83 Contents lists available at ScienceDirect Journal of Computational and Applied Mathematics journal homepage: www.elsevier.com/locate/cam

More information

Lecture # 20 The Preconditioned Conjugate Gradient Method

Lecture # 20 The Preconditioned Conjugate Gradient Method Lecture # 20 The Preconditioned Conjugate Gradient Method We wish to solve Ax = b (1) A R n n is symmetric and positive definite (SPD). We then of n are being VERY LARGE, say, n = 10 6 or n = 10 7. Usually,

More information

Control of the Keck and CELT Telescopes. Douglas G. MacMartin Control & Dynamical Systems California Institute of Technology

Control of the Keck and CELT Telescopes. Douglas G. MacMartin Control & Dynamical Systems California Institute of Technology Control of the Keck and CELT Telescopes Douglas G. MacMartin Control & Dynamical Systems California Institute of Technology Telescope Control Problems Light from star Primary mirror active control system

More information

Domain Decomposition Preconditioners for Spectral Nédélec Elements in Two and Three Dimensions

Domain Decomposition Preconditioners for Spectral Nédélec Elements in Two and Three Dimensions Domain Decomposition Preconditioners for Spectral Nédélec Elements in Two and Three Dimensions Bernhard Hientzsch Courant Institute of Mathematical Sciences, New York University, 51 Mercer Street, New

More information

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.

Lecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C. Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference

More information

Splitting Iteration Methods for Positive Definite Linear Systems

Splitting Iteration Methods for Positive Definite Linear Systems Splitting Iteration Methods for Positive Definite Linear Systems Zhong-Zhi Bai a State Key Lab. of Sci./Engrg. Computing Inst. of Comput. Math. & Sci./Engrg. Computing Academy of Mathematics and System

More information

3D Space Charge Routines: The Software Package MOEVE and FFT Compared

3D Space Charge Routines: The Software Package MOEVE and FFT Compared 3D Space Charge Routines: The Software Package MOEVE and FFT Compared Gisela Pöplau DESY, Hamburg, December 4, 2007 Overview Algorithms for 3D space charge calculations Properties of FFT and iterative

More information

AMG for a Peta-scale Navier Stokes Code

AMG for a Peta-scale Navier Stokes Code AMG for a Peta-scale Navier Stokes Code James Lottes Argonne National Laboratory October 18, 2007 The Challenge Develop an AMG iterative method to solve Poisson 2 u = f discretized on highly irregular

More information

FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION

FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION FINE-GRAINED PARALLEL INCOMPLETE LU FACTORIZATION EDMOND CHOW AND AFTAB PATEL Abstract. This paper presents a new fine-grained parallel algorithm for computing an incomplete LU factorization. All nonzeros

More information

Solution to Laplace Equation using Preconditioned Conjugate Gradient Method with Compressed Row Storage using MPI

Solution to Laplace Equation using Preconditioned Conjugate Gradient Method with Compressed Row Storage using MPI Solution to Laplace Equation using Preconditioned Conjugate Gradient Method with Compressed Row Storage using MPI Sagar Bhatt Person Number: 50170651 Department of Mechanical and Aerospace Engineering,

More information

Application of Wavelets to N body Particle In Cell Simulations

Application of Wavelets to N body Particle In Cell Simulations Application of Wavelets to N body Particle In Cell Simulations Balša Terzić (NIU) Northern Illinois University, Department of Physics April 7, 2006 Motivation Gaining insight into the dynamics of multi

More information