Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques

Size: px
Start display at page:

Download "Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques"

Transcription

1 Institut für Numerische Mathematik und Optimierung Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques Oliver Ernst Computational Methods with Applications Harrachov, CR, August 19 25, 2007

2 1 Collaborators Catherine Powell, David Silvester University of Manchester, School of Mathematics, Manchester, UK Ingolf Busch, Michael Eiermann, Elisabeth Ullmann, TU Bergakademie Freiberg, Freiberg, Germany Support: DAAD/British Council

3 2 Outline Random fields and the Karhunen-Loève expansion Discretization of the covariance operator Solution of the discrete eigenvalue problem A numerical example

4 3 Random Fields

5 4 Formally stochastic process indexed by a spatial coordinate x D R d, D bounded, i.e., measurable function a : D Ω R, where (Ω, A, P ) is a given probability space For ω Ω fixed, a(, ω) is a realization of the random field, i.e., a function D R. For x D fixed, a(x, ) is a random variable (RV) w.r.t. (Ω, A, P ).

6 5 Notation ξ := Ω ξ(ω) dp (ω) expected value of RV ξ : Ω R a(x) := a(x, ) Cov a (x, y) := (a(x, ) a(x))(a(y, ) a(y)) Var a (x) := Cov a (x, x) σ a (x) := Var a (x) L 2 P (Ω) := {ξ : ξ 2 < } mean of RF a at x D covariance of RF a at x, y D variance of RF a at x D standard deviation of RF a at x D RV of second order

7 6 A RF is of second order, if a(x, ) L 2 P (Ω) for all x D. Theorem (Karhunen-Loève expansion). Given a second order RF a = a(x, ω) with continuous covariance function c(x, y) := Cov a (x, y), denote by {(λ m, a m (x))} the eigenpairs of the (compact) integral operator C : L 2 (D) L 2 (D), (Cu)(x) = u(y) c(x, y) dy, there exists a sequence {ξ m } m N of random variables with ξ m = 0 m, ξ m ξ n = δ m,n m, n such that the Karhunen-Loève (KL) expansion D a(x, ω) = a(x) + m=1 λm a m (x) ξ m (ω) (KL) converges uniformly on D and in L 2 P.

8 7 Note: Covariance functions c(x, y) are continuous on D D as well as symmetric and of positive type. Therefore covariance operators C are compact, hence spectra Λ(C) consist of countably many eigenvalues accumulating at most at zero. Covariance operators are selfadjoint and positive semidefinite. Analogy Singular value expansion of integral operator A : L 2 (D) L 2 P, f(x) (Af)(ω) := A : L 2 P L 2 (D), ξ(ω) (A ξ)(x) = C = A A. D Ω f(x)a(x, ω) dx, ξ(ω)a(x, ω) dp (ω)

9 8 Common Covariance Models Cov a (x, y) = c(x, y) = c(ρ), ρ = x y c(r) c(r) c(r) r exponential c(r) = σ 2 e ρ/l r Bessel c(r) = σ 2 ρ l K 1( ρ l ) r Gaussian c(r) = σ 2 e ρ2 /l 2 l > 0 is a measure of the correlation length, here l = 0.1, 1, 2.

10 9 Variance For normalized eigenfunctions a m (x), D Var a (x) = c(x, x) = Var a (x) dx = m=1 m=1 λ m a m (x) 2, λ m (a m, a m ) D }{{} =1 = trace C. For constant variance (e.g., stationary RF), Var a σ 2 > 0, λ m = D σ 2. m

11 10 Truncated KL Expansion For computational purposes, KL expansion truncated after M terms: a (M) (x, ω) = a(x) + M m=1 λm a m (x) ξ m (ω). Truncation error a a (M) 2 D = λ m. m=m+1 Choose M such that sufficient amount of total variance of RF is retained.

12 11 Eigenvalue Decay Roughly: the smoother the kernel, the faster {λ m } m N 0. More precisely: if D R d, then if the kernel function c is piecewise H r : λ m c 1 m r/d piecewise smooth : λ m c 2 m r for any r > 0 piecewise analytic : λ m c 3 exp( c 4 m 1/d ) for suitable constants c 1, c 2, c 3, c 4. Note: piecewise smoothness of kernel also leads to bounds on derivatives of eigenfunctions a m in L (D). Proven e.g. in [Schwab & Todor (2006)], [Todor (2006)]

13 12 Galerkin Discretization T h admissible finite element triangulation of D finite dimensional subspace of piecewise polynomials V h = {φ : D R : φ T P k T T } L 2 (D). Discrete eigenvalue problem: find pairs (λ h m, a h m) such that (Ca h m, φ) = λ h m(a h m, φ) φ V h, m = 1, 2,... corresponds to generalized matrix eigenvalue problem Cx = λmx, [C ] i,j = (Cφ j, φ i ), [M ] i,j = (φ j, φ i ), i, j = 1, 2,..., N = dim V h. C large and dense, M can be made diagonal using suitable basis.

14 13 Discretization Error Discrete operator given by C h = P h CP h, P h the L 2 (D) orthogonal projection to V h. Discrete eigenpairs {(λ h m, a h m)} N m=1 If covariance operator is piecewise smooth, then for any r > 0 ( ) 0 λ m λ h m K r h 2(k+1) λ 1 r m + h 4(k+1) λ 2r m, (I P h )a m L2 (D) K r λ r m h k+1. [Todor (2006)]

15 14 Solution of Matrix Eigenvalue Problem Only fixed number of leading eigenpairs required, suggests restarted Krylov subspace technique. We use the Thick-Restart Lanczos (TRL) method [Simon & Wu (2000)]. Idea: limit dimension of Krylov space to fixed m, save some desired approximate eigenpairs, generate new Krylov space which contains these retained approximations (restart). Krylov methods require inexpensive matrix-vector product. We obtain this by replacing C by a hierarchical matrix approximation C, for which matrix vector products can be computed in O(N log N) operations [Hackbusch (1999)].

16 15 Thick-Restart Lanczos Cycle (1) Given Lanczos decomposition of Krylov space K m (A, v) AQ m = Q m T m + β m+1 q m+1 em, Q m = [q 1,..., q m ], Q mq m = I m, (2) compute eigenpairs T m y j = ϑ j y j, j = 1,..., m, (3) select k < m Ritz vectors to retain, Y k := [y 1,..., y k ], (4) set Q k := Q m Y k, T k := Q k T Q m k to obtain A Q k = Q k Tk + β m+1 q k+1 s with q k+1 = q m+1 and s := Y k e m, (5) extend span{ q 1,..., q m+1 } to Krylov space of order m with Lanczostype decomposition A Q m = Q m Tm + β m+1 q m+1 e m

17 16 After restart cycle, projection T m of A on new Krylov space in A Q m = Q m Tm + β m+1 q m+1 e m has the form T m = T k βm s β m s α k+1 βk+1. β..... k βm β m α m Note: Leading k k block is diagonal.

18 17 Remarks: Mathematically equivalent to implicitly restarted Lanczos method and other augmented Krylov techniques, but more efficient. Takes advantage of symmetry (ARPACK uses full recurrences). Projected matrix T k readily available (= diag(ϑ 1,..., ϑ k )). Eigenvector residual norms from coordinate calculations (like in standard symmetric Lanczos). Well-known reorthogonalization techniques can be incorporated. For covariance problem: no shift-invert techniques required. Note: Need efficient matrix-vector product.

19 18 Hierarchical Matrix Approximation Idea: (recall survey in Monday s plenary talk of W. Hackbusch) Partition dense matrix into square blocks of 2 types near field blocks: computed and stored as usual far field blocks: approximated by matrix of low rank UV, computed by interpolation of kernel, store factors U, V. blocks correspond to clusters of degrees of freedom, i.e., clusters of supports of Galerkin basis functions block for pair of clusters s, t in near field if admissibility condition min{diam(d s ), diam(d t )} η dist(d s, D t ) satisfied by associated domains, η is the admissibility parameter.

20

21 20 Remarks: Algebraic variant of fast multipole method Admissibility parameter η scales with correlation length. Necessary smoothness requirements satisfied for all common covariance kernels. Resulting data-sparse representation of discretized integral operator can be applied to a vector in O(N log N) operations (for N DOF). Need efficient quadrature for near field. An optimal approximation must thus balance the errors due to truncation of the KL series, Galerkin error in approximation a h m a m, λ h m λ m Lanczos approximation of discrete eigenpairs hierarchical matrix approximation C C

22 21 Numerical Example Bessel covariance kernel c(x, y) = x y l ( ) x y K 1, x, y D = [ 1, 1] 2. l Discretization: piecewise constant functions w.r.t. triangular mesh on D Hierarchical matrix parameters: interpolation polynomial degree : 4 admissibility constant : η = 1/l minimal block size : 62 Computations: MATLAB R2007a, Intel Xeon 5160, 3 GHz, 16 GB RAM calls to HLib-1.3 library (MPI Leipzig) via MEX

23 22 Some modes (l = 0.5) mode mode mode mode

24 23 Performance of TRL N # evs % variance m restarts l = l = l =

25 24 Timings N log(n) l=0.5 l=1 l= N log(n) l=0.5 l=1 l= tgen [s] tlanczos [s] N generation of hierarchical matrix approximation N eigenvalue calculation

26 25 Conclusions Covariance eigenvalue problem challenging due to its size Can exploit regularity of covariance kernels Lanczos combined with hierarchical matrix approximation promising Becomes intractable for very small correlation lengths (too many relevant modes) Ongoing Work more careful tuning of hierarchical matrix approximation parameters multiple eigenvalues (symmetries in the domain) extend optimal quadrature techniques to 3D higher order FE approximation

Efficient Solvers for Stochastic Finite Element Saddle Point Problems

Efficient Solvers for Stochastic Finite Element Saddle Point Problems Efficient Solvers for Stochastic Finite Element Saddle Point Problems Catherine E. Powell c.powell@manchester.ac.uk School of Mathematics University of Manchester, UK Efficient Solvers for Stochastic Finite

More information

AND ELISABETH ULLMANN

AND ELISABETH ULLMANN Proceedings of ALGORITMY 25 pp. 1 1 COMPUTATIONAL ASPECTS OF THE STOCHASTIC FINITE ELEMENT METHO MICHAEL EIERMANN, OLIVER G. ERNST AN ELISABETH ULLMANN Abstract. We present an overview of the stochastic

More information

Lecture 1: Center for Uncertainty Quantification. Alexander Litvinenko. Computation of Karhunen-Loeve Expansion:

Lecture 1: Center for Uncertainty Quantification. Alexander Litvinenko. Computation of Karhunen-Loeve Expansion: tifica Lecture 1: Computation of Karhunen-Loeve Expansion: Alexander Litvinenko http://sri-uq.kaust.edu.sa/ Stochastic PDEs We consider div(κ(x, ω) u) = f (x, ω) in G, u = 0 on G, with stochastic coefficients

More information

Implementation of Sparse Wavelet-Galerkin FEM for Stochastic PDEs

Implementation of Sparse Wavelet-Galerkin FEM for Stochastic PDEs Implementation of Sparse Wavelet-Galerkin FEM for Stochastic PDEs Roman Andreev ETH ZÜRICH / 29 JAN 29 TOC of the Talk Motivation & Set-Up Model Problem Stochastic Galerkin FEM Conclusions & Outlook Motivation

More information

Solving the Stochastic Steady-State Diffusion Problem Using Multigrid

Solving the Stochastic Steady-State Diffusion Problem Using Multigrid Solving the Stochastic Steady-State Diffusion Problem Using Multigrid Tengfei Su Applied Mathematics and Scientific Computing Advisor: Howard Elman Department of Computer Science Sept. 29, 2015 Tengfei

More information

Krylov subspace projection methods

Krylov subspace projection methods I.1.(a) Krylov subspace projection methods Orthogonal projection technique : framework Let A be an n n complex matrix and K be an m-dimensional subspace of C n. An orthogonal projection technique seeks

More information

H 2 -matrices with adaptive bases

H 2 -matrices with adaptive bases 1 H 2 -matrices with adaptive bases Steffen Börm MPI für Mathematik in den Naturwissenschaften Inselstraße 22 26, 04103 Leipzig http://www.mis.mpg.de/ Problem 2 Goal: Treat certain large dense matrices

More information

Solving the steady state diffusion equation with uncertainty Final Presentation

Solving the steady state diffusion equation with uncertainty Final Presentation Solving the steady state diffusion equation with uncertainty Final Presentation Virginia Forstall vhfors@gmail.com Advisor: Howard Elman elman@cs.umd.edu Department of Computer Science May 6, 2012 Problem

More information

Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms

Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms 2. First Results and Algorithms Michael Eiermann Institut für Numerische Mathematik und Optimierung Technische

More information

A refined Lanczos method for computing eigenvalues and eigenvectors of unsymmetric matrices

A refined Lanczos method for computing eigenvalues and eigenvectors of unsymmetric matrices A refined Lanczos method for computing eigenvalues and eigenvectors of unsymmetric matrices Jean Christophe Tremblay and Tucker Carrington Chemistry Department Queen s University 23 août 2007 We want to

More information

arxiv: v2 [math.na] 8 Apr 2017

arxiv: v2 [math.na] 8 Apr 2017 A LOW-RANK MULTIGRID METHOD FOR THE STOCHASTIC STEADY-STATE DIFFUSION PROBLEM HOWARD C. ELMAN AND TENGFEI SU arxiv:1612.05496v2 [math.na] 8 Apr 2017 Abstract. We study a multigrid method for solving large

More information

Multilevel accelerated quadrature for elliptic PDEs with random diffusion. Helmut Harbrecht Mathematisches Institut Universität Basel Switzerland

Multilevel accelerated quadrature for elliptic PDEs with random diffusion. Helmut Harbrecht Mathematisches Institut Universität Basel Switzerland Multilevel accelerated quadrature for elliptic PDEs with random diffusion Mathematisches Institut Universität Basel Switzerland Overview Computation of the Karhunen-Loéve expansion Elliptic PDE with uniformly

More information

Hierarchical Parallel Solution of Stochastic Systems

Hierarchical Parallel Solution of Stochastic Systems Hierarchical Parallel Solution of Stochastic Systems Second M.I.T. Conference on Computational Fluid and Solid Mechanics Contents: Simple Model of Stochastic Flow Stochastic Galerkin Scheme Resulting Equations

More information

PDEs, Matrix Functions and Krylov Subspace Methods

PDEs, Matrix Functions and Krylov Subspace Methods PDEs, Matrix Functions and Krylov Subspace Methods Oliver Ernst Institut für Numerische Mathematik und Optimierung TU Bergakademie Freiberg, Germany LMS Durham Symposium Computational Linear Algebra for

More information

The Lanczos and conjugate gradient algorithms

The Lanczos and conjugate gradient algorithms The Lanczos and conjugate gradient algorithms Gérard MEURANT October, 2008 1 The Lanczos algorithm 2 The Lanczos algorithm in finite precision 3 The nonsymmetric Lanczos algorithm 4 The Golub Kahan bidiagonalization

More information

Sampling and Low-Rank Tensor Approximations

Sampling and Low-Rank Tensor Approximations Sampling and Low-Rank Tensor Approximations Hermann G. Matthies Alexander Litvinenko, Tarek A. El-Moshely +, Brunswick, Germany + MIT, Cambridge, MA, USA wire@tu-bs.de http://www.wire.tu-bs.de $Id: 2_Sydney-MCQMC.tex,v.3

More information

Last Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection

Last Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue

More information

A Newton-Galerkin-ADI Method for Large-Scale Algebraic Riccati Equations

A Newton-Galerkin-ADI Method for Large-Scale Algebraic Riccati Equations A Newton-Galerkin-ADI Method for Large-Scale Algebraic Riccati Equations Peter Benner Max-Planck-Institute for Dynamics of Complex Technical Systems Computational Methods in Systems and Control Theory

More information

Large-scale eigenvalue problems

Large-scale eigenvalue problems ELE 538B: Mathematics of High-Dimensional Data Large-scale eigenvalue problems Yuxin Chen Princeton University, Fall 208 Outline Power method Lanczos algorithm Eigenvalue problems 4-2 Eigendecomposition

More information

Iterative methods for symmetric eigenvalue problems

Iterative methods for symmetric eigenvalue problems s Iterative s for symmetric eigenvalue problems, PhD McMaster University School of Computational Engineering and Science February 11, 2008 s 1 The power and its variants Inverse power Rayleigh quotient

More information

Adaptive Collocation with Kernel Density Estimation

Adaptive Collocation with Kernel Density Estimation Examples of with Kernel Density Estimation Howard C. Elman Department of Computer Science University of Maryland at College Park Christopher W. Miller Applied Mathematics and Scientific Computing Program

More information

Proper Orthogonal Decomposition (POD) for Nonlinear Dynamical Systems. Stefan Volkwein

Proper Orthogonal Decomposition (POD) for Nonlinear Dynamical Systems. Stefan Volkwein Proper Orthogonal Decomposition (POD) for Nonlinear Dynamical Systems Institute for Mathematics and Scientific Computing, Austria DISC Summerschool 5 Outline of the talk POD and singular value decomposition

More information

Eigenvalue Problems CHAPTER 1 : PRELIMINARIES

Eigenvalue Problems CHAPTER 1 : PRELIMINARIES Eigenvalue Problems CHAPTER 1 : PRELIMINARIES Heinrich Voss voss@tu-harburg.de Hamburg University of Technology Institute of Mathematics TUHH Heinrich Voss Preliminaries Eigenvalue problems 2012 1 / 14

More information

Efficient Observation of Random Phenomena

Efficient Observation of Random Phenomena Lecture 9 Efficient Observation of Random Phenomena Tokyo Polytechnic University The 1st Century Center of Excellence Program Yukio Tamura POD Proper Orthogonal Decomposition Stochastic Representation

More information

Solving an Elliptic PDE Eigenvalue Problem via Automated Multi-Level Substructuring and Hierarchical Matrices

Solving an Elliptic PDE Eigenvalue Problem via Automated Multi-Level Substructuring and Hierarchical Matrices Solving an Elliptic PDE Eigenvalue Problem via Automated Multi-Level Substructuring and Hierarchical Matrices Peter Gerds and Lars Grasedyck Bericht Nr. 30 März 2014 Key words: automated multi-level substructuring,

More information

Numerical methods for the discretization of random fields by means of the Karhunen Loève expansion

Numerical methods for the discretization of random fields by means of the Karhunen Loève expansion Numerical methods for the discretization of random fields by means of the Karhunen Loève expansion Wolfgang Betz, Iason Papaioannou, Daniel Straub Engineering Risk Analysis Group, Technische Universität

More information

CBS Constants and Their Role in Error Estimation for Stochastic Galerkin Finite Element Methods. Crowder, Adam J and Powell, Catherine E

CBS Constants and Their Role in Error Estimation for Stochastic Galerkin Finite Element Methods. Crowder, Adam J and Powell, Catherine E CBS Constants and Their Role in Error Estimation for Stochastic Galerkin Finite Element Methods Crowder, Adam J and Powell, Catherine E 2017 MIMS EPrint: 2017.18 Manchester Institute for Mathematical Sciences

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

Solving the stochastic steady-state diffusion problem using multigrid

Solving the stochastic steady-state diffusion problem using multigrid IMA Journal of Numerical Analysis (2007) 27, 675 688 doi:10.1093/imanum/drm006 Advance Access publication on April 9, 2007 Solving the stochastic steady-state diffusion problem using multigrid HOWARD ELMAN

More information

MATH 115A: SAMPLE FINAL SOLUTIONS

MATH 115A: SAMPLE FINAL SOLUTIONS MATH A: SAMPLE FINAL SOLUTIONS JOE HUGHES. Let V be the set of all functions f : R R such that f( x) = f(x) for all x R. Show that V is a vector space over R under the usual addition and scalar multiplication

More information

The quadratic eigenvalue problem (QEP) is to find scalars λ and nonzero vectors u satisfying

The quadratic eigenvalue problem (QEP) is to find scalars λ and nonzero vectors u satisfying I.2 Quadratic Eigenvalue Problems 1 Introduction The quadratic eigenvalue problem QEP is to find scalars λ and nonzero vectors u satisfying where Qλx = 0, 1.1 Qλ = λ 2 M + λd + K, M, D and K are given

More information

Lecture 2: Linear Algebra Review

Lecture 2: Linear Algebra Review EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1

More information

Karhunen-Loève Approximation of Random Fields by Generalized Fast Multipole Methods

Karhunen-Loève Approximation of Random Fields by Generalized Fast Multipole Methods Karhunen-Loève Approximation of Random Fields by Generalized Fast Multipole Methods C. Schwab and R.A. Todor Research Report No. 26- January 26 Seminar für Angewandte Mathematik Eidgenössische Technische

More information

Applied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic

Applied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.4: Krylov Subspace Methods 2 / 51 Krylov Subspace Methods In this chapter we give

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

Krylov Subspaces. Lab 1. The Arnoldi Iteration

Krylov Subspaces. Lab 1. The Arnoldi Iteration Lab 1 Krylov Subspaces Lab Objective: Discuss simple Krylov Subspace Methods for finding eigenvalues and show some interesting applications. One of the biggest difficulties in computational linear algebra

More information

Experiences with Model Reduction and Interpolation

Experiences with Model Reduction and Interpolation Experiences with Model Reduction and Interpolation Paul Constantine Stanford University, Sandia National Laboratories Qiqi Wang (MIT) David Gleich (Purdue) Emory University March 7, 2012 Joe Ruthruff (SNL)

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Structured Krylov Subspace Methods for Eigenproblems with Spectral Symmetries

Structured Krylov Subspace Methods for Eigenproblems with Spectral Symmetries Structured Krylov Subspace Methods for Eigenproblems with Spectral Symmetries Fakultät für Mathematik TU Chemnitz, Germany Peter Benner benner@mathematik.tu-chemnitz.de joint work with Heike Faßbender

More information

Hilbert Space Methods for Reduced-Rank Gaussian Process Regression

Hilbert Space Methods for Reduced-Rank Gaussian Process Regression Hilbert Space Methods for Reduced-Rank Gaussian Process Regression Arno Solin and Simo Särkkä Aalto University, Finland Workshop on Gaussian Process Approximation Copenhagen, Denmark, May 2015 Solin &

More information

Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms

Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms Krylov Subspace Methods for the Evaluation of Matrix Functions. Applications and Algorithms 4. Monotonicity of the Lanczos Method Michael Eiermann Institut für Numerische Mathematik und Optimierung Technische

More information

Functional Analysis Review

Functional Analysis Review Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all

More information

Arnoldi Methods in SLEPc

Arnoldi Methods in SLEPc Scalable Library for Eigenvalue Problem Computations SLEPc Technical Report STR-4 Available at http://slepc.upv.es Arnoldi Methods in SLEPc V. Hernández J. E. Román A. Tomás V. Vidal Last update: October,

More information

Second-Order Inference for Gaussian Random Curves

Second-Order Inference for Gaussian Random Curves Second-Order Inference for Gaussian Random Curves With Application to DNA Minicircles Victor Panaretos David Kraus John Maddocks Ecole Polytechnique Fédérale de Lausanne Panaretos, Kraus, Maddocks (EPFL)

More information

Linear Algebra Practice Problems

Linear Algebra Practice Problems Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,

More information

Rational Krylov methods for linear and nonlinear eigenvalue problems

Rational Krylov methods for linear and nonlinear eigenvalue problems Rational Krylov methods for linear and nonlinear eigenvalue problems Mele Giampaolo mele@mail.dm.unipi.it University of Pisa 7 March 2014 Outline Arnoldi (and its variants) for linear eigenproblems Rational

More information

Concentration Ellipsoids

Concentration Ellipsoids Concentration Ellipsoids ECE275A Lecture Supplement Fall 2008 Kenneth Kreutz Delgado Electrical and Computer Engineering Jacobs School of Engineering University of California, San Diego VERSION LSECE275CE

More information

Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes

Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes TTU, October 26, 2012 p. 1/3 Karhunen-Loeve Expansion and Optimal Low-Rank Model for Spatial Processes Hao Zhang Department of Statistics Department of Forestry and Natural Resources Purdue University

More information

PART IV Spectral Methods

PART IV Spectral Methods PART IV Spectral Methods Additional References: R. Peyret, Spectral methods for incompressible viscous flow, Springer (2002), B. Mercier, An introduction to the numerical analysis of spectral methods,

More information

Stochastic Spectral Approaches to Bayesian Inference

Stochastic Spectral Approaches to Bayesian Inference Stochastic Spectral Approaches to Bayesian Inference Prof. Nathan L. Gibson Department of Mathematics Applied Mathematics and Computation Seminar March 4, 2011 Prof. Gibson (OSU) Spectral Approaches to

More information

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

More information

A Vector-Space Approach for Stochastic Finite Element Analysis

A Vector-Space Approach for Stochastic Finite Element Analysis A Vector-Space Approach for Stochastic Finite Element Analysis S Adhikari 1 1 Swansea University, UK CST2010: Valencia, Spain Adhikari (Swansea) Vector-Space Approach for SFEM 14-17 September, 2010 1 /

More information

Matrix Algorithms. Volume II: Eigensystems. G. W. Stewart H1HJ1L. University of Maryland College Park, Maryland

Matrix Algorithms. Volume II: Eigensystems. G. W. Stewart H1HJ1L. University of Maryland College Park, Maryland Matrix Algorithms Volume II: Eigensystems G. W. Stewart University of Maryland College Park, Maryland H1HJ1L Society for Industrial and Applied Mathematics Philadelphia CONTENTS Algorithms Preface xv xvii

More information

S N. hochdimensionaler Lyapunov- und Sylvestergleichungen. Peter Benner. Mathematik in Industrie und Technik Fakultät für Mathematik TU Chemnitz

S N. hochdimensionaler Lyapunov- und Sylvestergleichungen. Peter Benner. Mathematik in Industrie und Technik Fakultät für Mathematik TU Chemnitz Ansätze zur numerischen Lösung hochdimensionaler Lyapunov- und Sylvestergleichungen Peter Benner Mathematik in Industrie und Technik Fakultät für Mathematik TU Chemnitz S N SIMULATION www.tu-chemnitz.de/~benner

More information

Golub-Kahan iterative bidiagonalization and determining the noise level in the data

Golub-Kahan iterative bidiagonalization and determining the noise level in the data Golub-Kahan iterative bidiagonalization and determining the noise level in the data Iveta Hnětynková,, Martin Plešinger,, Zdeněk Strakoš, * Charles University, Prague ** Academy of Sciences of the Czech

More information

Quasi-optimal and adaptive sparse grids with control variates for PDEs with random diffusion coefficient

Quasi-optimal and adaptive sparse grids with control variates for PDEs with random diffusion coefficient Quasi-optimal and adaptive sparse grids with control variates for PDEs with random diffusion coefficient F. Nobile, L. Tamellini, R. Tempone, F. Tesei CSQI - MATHICSE, EPFL, Switzerland Dipartimento di

More information

FINITE-DIMENSIONAL LINEAR ALGEBRA

FINITE-DIMENSIONAL LINEAR ALGEBRA DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITE-DIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup

More information

Block-diagonal preconditioning for spectral stochastic finite-element systems

Block-diagonal preconditioning for spectral stochastic finite-element systems IMA Journal of Numerical Analysis (2009) 29, 350 375 doi:0.093/imanum/drn04 Advance Access publication on April 4, 2008 Block-diagonal preconditioning for spectral stochastic finite-element systems CATHERINE

More information

BALANCING-RELATED MODEL REDUCTION FOR DATA-SPARSE SYSTEMS

BALANCING-RELATED MODEL REDUCTION FOR DATA-SPARSE SYSTEMS BALANCING-RELATED Peter Benner Professur Mathematik in Industrie und Technik Fakultät für Mathematik Technische Universität Chemnitz Computational Methods with Applications Harrachov, 19 25 August 2007

More information

(a) II and III (b) I (c) I and III (d) I and II and III (e) None are true.

(a) II and III (b) I (c) I and III (d) I and II and III (e) None are true. 1 Which of the following statements is always true? I The null space of an m n matrix is a subspace of R m II If the set B = {v 1,, v n } spans a vector space V and dimv = n, then B is a basis for V III

More information

Space-time sparse discretization of linear parabolic equations

Space-time sparse discretization of linear parabolic equations Space-time sparse discretization of linear parabolic equations Roman Andreev August 2, 200 Seminar for Applied Mathematics, ETH Zürich, Switzerland Support by SNF Grant No. PDFMP2-27034/ Part of PhD thesis

More information

Krylov Subspace Methods for Large/Sparse Eigenvalue Problems

Krylov Subspace Methods for Large/Sparse Eigenvalue Problems Krylov Subspace Methods for Large/Sparse Eigenvalue Problems Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan April 17, 2012 T.-M. Huang (Taiwan Normal University) Krylov

More information

Linear Algebra Review. Vectors

Linear Algebra Review. Vectors Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

More information

Low-rank techniques applied to moment equations for the stochastic Darcy problem with lognormal permeability

Low-rank techniques applied to moment equations for the stochastic Darcy problem with lognormal permeability Low-rank techniques applied to moment equations for the stochastic arcy problem with lognormal permeability Francesca Bonizzoni 1,2 and Fabio Nobile 1 1 CSQI-MATHICSE, EPFL, Switzerland 2 MOX, Politecnico

More information

QUALIFYING EXAMINATION Harvard University Department of Mathematics Tuesday September 21, 2004 (Day 1)

QUALIFYING EXAMINATION Harvard University Department of Mathematics Tuesday September 21, 2004 (Day 1) QUALIFYING EXAMINATION Harvard University Department of Mathematics Tuesday September 21, 2004 (Day 1) Each of the six questions is worth 10 points. 1) Let H be a (real or complex) Hilbert space. We say

More information

Collocation based high dimensional model representation for stochastic partial differential equations

Collocation based high dimensional model representation for stochastic partial differential equations Collocation based high dimensional model representation for stochastic partial differential equations S Adhikari 1 1 Swansea University, UK ECCM 2010: IV European Conference on Computational Mechanics,

More information

Efficient Implementation of Large Scale Lyapunov and Riccati Equation Solvers

Efficient Implementation of Large Scale Lyapunov and Riccati Equation Solvers Efficient Implementation of Large Scale Lyapunov and Riccati Equation Solvers Jens Saak joint work with Peter Benner (MiIT) Professur Mathematik in Industrie und Technik (MiIT) Fakultät für Mathematik

More information

Polynomial Chaos and Karhunen-Loeve Expansion

Polynomial Chaos and Karhunen-Loeve Expansion Polynomial Chaos and Karhunen-Loeve Expansion 1) Random Variables Consider a system that is modeled by R = M(x, t, X) where X is a random variable. We are interested in determining the probability of the

More information

Review (Probability & Linear Algebra)

Review (Probability & Linear Algebra) Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint

More information

H(div) Preconditioning for a Mixed Finite Element Formulation of the Stochastic Diffusion Problem 1

H(div) Preconditioning for a Mixed Finite Element Formulation of the Stochastic Diffusion Problem 1 University of Maryland Department of Computer Science CS-TR-4918 University of Maryland Institute for Advanced Computer Studies UMIACS-TR-2008-15 H(div) Preconditioning for a Mixed Finite Element Formulation

More information

Applications of Randomized Methods for Decomposing and Simulating from Large Covariance Matrices

Applications of Randomized Methods for Decomposing and Simulating from Large Covariance Matrices Applications of Randomized Methods for Decomposing and Simulating from Large Covariance Matrices Vahid Dehdari and Clayton V. Deutsch Geostatistical modeling involves many variables and many locations.

More information

arxiv: v2 [math.na] 5 Dec 2013

arxiv: v2 [math.na] 5 Dec 2013 ACTIVE SUBSPACE METHODS IN THEORY AND PRACTICE: APPLICATIONS TO KRIGING SURFACES PAUL G. CONSTANTINE, ERIC DOW, AND QIQI WANG arxiv:1304.2070v2 [math.na 5 Dec 2013 Abstract. Many multivariate functions

More information

Mathematical foundations - linear algebra

Mathematical foundations - linear algebra Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar

More information

Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

More information

Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method

Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts

More information

arxiv: v2 [math.pr] 27 Oct 2015

arxiv: v2 [math.pr] 27 Oct 2015 A brief note on the Karhunen-Loève expansion Alen Alexanderian arxiv:1509.07526v2 [math.pr] 27 Oct 2015 October 28, 2015 Abstract We provide a detailed derivation of the Karhunen Loève expansion of a stochastic

More information

Fast Direct Methods for Gaussian Processes

Fast Direct Methods for Gaussian Processes Fast Direct Methods for Gaussian Processes Mike O Neil Departments of Mathematics New York University oneil@cims.nyu.edu December 12, 2015 1 Collaborators This is joint work with: Siva Ambikasaran Dan

More information

Numerical Approximation of Stochastic Elliptic Partial Differential Equations

Numerical Approximation of Stochastic Elliptic Partial Differential Equations Numerical Approximation of Stochastic Elliptic Partial Differential Equations Hermann G. Matthies, Andreas Keese Institut für Wissenschaftliches Rechnen Technische Universität Braunschweig wire@tu-bs.de

More information

Matrix Equations and and Bivariate Function Approximation

Matrix Equations and and Bivariate Function Approximation Matrix Equations and and Bivariate Function Approximation D. Kressner Joint work with L. Grasedyck, C. Tobler, N. Truhar. ETH Zurich, Seminar for Applied Mathematics Manchester, 17.06.2009 Sylvester matrix

More information

Lecture Summaries for Linear Algebra M51A

Lecture Summaries for Linear Algebra M51A These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture

More information

Hierarchical Matrices. Jon Cockayne April 18, 2017

Hierarchical Matrices. Jon Cockayne April 18, 2017 Hierarchical Matrices Jon Cockayne April 18, 2017 1 Sources Introduction to Hierarchical Matrices with Applications [Börm et al., 2003] 2 Sources Introduction to Hierarchical Matrices with Applications

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

Stabilization and Acceleration of Algebraic Multigrid Method

Stabilization and Acceleration of Algebraic Multigrid Method Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration

More information

Krylov subspace methods for linear systems with tensor product structure

Krylov subspace methods for linear systems with tensor product structure Krylov subspace methods for linear systems with tensor product structure Christine Tobler Seminar for Applied Mathematics, ETH Zürich 19. August 2009 Outline 1 Introduction 2 Basic Algorithm 3 Convergence

More information

Background Mathematics (2/2) 1. David Barber

Background Mathematics (2/2) 1. David Barber Background Mathematics (2/2) 1 David Barber University College London Modified by Samson Cheung (sccheung@ieee.org) 1 These slides accompany the book Bayesian Reasoning and Machine Learning. The book and

More information

A HARMONIC RESTARTED ARNOLDI ALGORITHM FOR CALCULATING EIGENVALUES AND DETERMINING MULTIPLICITY

A HARMONIC RESTARTED ARNOLDI ALGORITHM FOR CALCULATING EIGENVALUES AND DETERMINING MULTIPLICITY A HARMONIC RESTARTED ARNOLDI ALGORITHM FOR CALCULATING EIGENVALUES AND DETERMINING MULTIPLICITY RONALD B. MORGAN AND MIN ZENG Abstract. A restarted Arnoldi algorithm is given that computes eigenvalues

More information

Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson

Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson Final Exam, Linear Algebra, Fall, 2003, W. Stephen Wilson Name: TA Name and section: NO CALCULATORS, SHOW ALL WORK, NO OTHER PAPERS ON DESK. There is very little actual work to be done on this exam if

More information

Multi-Element Probabilistic Collocation Method in High Dimensions

Multi-Element Probabilistic Collocation Method in High Dimensions Multi-Element Probabilistic Collocation Method in High Dimensions Jasmine Foo and George Em Karniadakis Division of Applied Mathematics, Brown University, Providence, RI 02912 USA Abstract We combine multi-element

More information

1 Singular Value Decomposition and Principal Component

1 Singular Value Decomposition and Principal Component Singular Value Decomposition and Principal Component Analysis In these lectures we discuss the SVD and the PCA, two of the most widely used tools in machine learning. Principal Component Analysis (PCA)

More information

LARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems

LARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization General Tools for Solving Large Eigen-Problems

More information

Multigrid and stochastic sparse-grids techniques for PDE control problems with random coefficients

Multigrid and stochastic sparse-grids techniques for PDE control problems with random coefficients Multigrid and stochastic sparse-grids techniques for PDE control problems with random coefficients Università degli Studi del Sannio Dipartimento e Facoltà di Ingegneria, Benevento, Italia Random fields

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 23: GMRES and Other Krylov Subspace Methods; Preconditioning Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 18 Outline

More information

Iterative methods for Linear System

Iterative methods for Linear System Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline Basics: Matrices and their properties Eigenvalues, Condition Number Iterative Methods Direct and

More information

On solving linear systems arising from Shishkin mesh discretizations

On solving linear systems arising from Shishkin mesh discretizations On solving linear systems arising from Shishkin mesh discretizations Petr Tichý Faculty of Mathematics and Physics, Charles University joint work with Carlos Echeverría, Jörg Liesen, and Daniel Szyld October

More information

Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method

Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts

More information

Algebraic Multigrid as Solvers and as Preconditioner

Algebraic Multigrid as Solvers and as Preconditioner Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven

More information

Krylov Subspaces. The order-n Krylov subspace of A generated by x is

Krylov Subspaces. The order-n Krylov subspace of A generated by x is Lab 1 Krylov Subspaces Lab Objective: matrices. Use Krylov subspaces to find eigenvalues of extremely large One of the biggest difficulties in computational linear algebra is the amount of memory needed

More information

Sparse polynomial chaos expansions in engineering applications

Sparse polynomial chaos expansions in engineering applications DEPARTMENT OF CIVIL, ENVIRONMENTAL AND GEOMATIC ENGINEERING CHAIR OF RISK, SAFETY & UNCERTAINTY QUANTIFICATION Sparse polynomial chaos expansions in engineering applications B. Sudret G. Blatman (EDF R&D,

More information

MATH 590: Meshfree Methods

MATH 590: Meshfree Methods MATH 590: Meshfree Methods Chapter 34: Improving the Condition Number of the Interpolation Matrix Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Fall 2010 fasshauer@iit.edu

More information

DEFLATED RESTARTING FOR MATRIX FUNCTIONS

DEFLATED RESTARTING FOR MATRIX FUNCTIONS DEFLATED RESTARTING FOR MATRIX FUNCTIONS M. EIERMANN, O. G. ERNST AND S. GÜTTEL Abstract. We investigate an acceleration technique for restarted Krylov subspace methods for computing the action of a function

More information