Fast Hessenberg QR Iteration for Companion Matrices
|
|
|
- Jordan Sullivan
- 5 years ago
- Views:
Transcription
1 Fast Hessenberg QR Iteration for Companion Matrices David Bindel Ming Gu David Garmire James Demmel Shivkumar Chandrasekaran Fast Hessenberg QR Iteration for Companion Matrices p.1/20
2 Motivation: Polynomial root finding A standard algorithm: QR iteration on a companion matrix Robust software exists It s normwise backward stable It s used in Matlab But it takes O(n 3 ) time and O(n 2 ) storage Fast Hessenberg QR Iteration for Companion Matrices p.2/20
3 Companion matrices Given p(λ) = λ n + a n 1 λ n a 1 λ + a 0 Define a companion matrix C: a n 1 a n 2 a n 3... a 1 a C = For matrix polynomials, replace a i by A i and 1 by I. Fast Hessenberg QR Iteration for Companion Matrices p.3/20
4 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
5 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
6 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
7 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
8 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
9 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
10 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
11 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
12 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
13 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
14 Hessenberg QR iteration Fast Hessenberg QR Iteration for Companion Matrices p./20
15 Accelerating QR Have examples of how to do QR faster: Naive implementation = Hessenberg QR ridiagonal QR (symmetric and skew problems) Unitary Hessenberg QR Keys to fast QR: Find a structure preserved by the iteration Find a low-complexity representation for that structure Fast Hessenberg QR Iteration for Companion Matrices p.5/20
16 Rank symmetry Call A rank-symmetric if rank(a 12 ) = rank(a 21 ) for any 2-by-2 blocking of A with square A 11 and A 22. Examples: Symmetric matrices Skew-symmetric matrices Orthogonal matrices (by the CS decomposition) J-orthogonal matrices (by hyperbolic CS decomposition) Fast Hessenberg QR Iteration for Companion Matrices p.6/20
17 Rank symmetry Call A rank-symmetric if rank(a 12 ) = rank(a 21 ) for any 2-by-2 blocking of A with square A 11 and A 22. Examples: Symmetric matrices Skew-symmetric matrices Orthogonal matrices (by the CS decomposition) J-orthogonal matrices (by hyperbolic CS decomposition) Examples of rank-symmetric + low rank: Companion matrices Problems which lose self-adjointness at boundary Linear oscillators with low-rank damping Fast Hessenberg QR Iteration for Companion Matrices p.6/20
18 Rank symmetric + low rank h: For A rank symmetric and L with rank k, both square block 2-by-2: rank(a 12 + L 12 ) rank(a 21 + L 21 ) 2k Proof: rank(l ij ) k, so rank(a ij + L ij ) rank(a ij ) k. Note rank(a ij ) = rank(a ji ) and apply triangle inequality. Fast Hessenberg QR Iteration for Companion Matrices p.7/20
19 Off-diagonal rank Any matrix unitarily similar to C is orthogonal + rank 1. herefore: Hessenberg forms similar to C have rank(super-diagonal block) 3. Real Schur forms similar to C have rank(super-diagonal block) 2. (For polynomial matrices, make that 3d and 2d). Fast Hessenberg QR Iteration for Companion Matrices p.8/20
20 SSS structure Write Hessenberg matrices with low off-diagonal rank as narrow band matrix + SSS matrix. 0 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V 0 0 U 2 V3 U 2 W 3 V H = B U 3 V Write H ij = U i W i+1... W j 1 V j for i < j where U i R m b 2k+1, V i R m b 2k+1, and W i R 2k+1 2k+1. Make the block size m b 2k + 1. otal storage cost: O(nk). Fast Hessenberg QR Iteration for Companion Matrices p.9/20
21 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V B 21 B 22 U 2 V3 U 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Fast Hessenberg QR Iteration for Companion Matrices p.10/20
22 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V B 21 B 22 U 2 V3 U 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Fast Hessenberg QR Iteration for Companion Matrices p.10/20
23 SSS structure transformed B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V B 11 B 12 Û 1 V 3 B 21 B 22 Û 2 V 3 Û 1 W 3 V Û 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Fast Hessenberg QR Iteration for Companion Matrices p.10/20
24 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 V 2 U 1 W 2 V3 U 1 W 2 W 3 V B 21 B 22 U 2 V3 U 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Split blocks once bulge is within B 22 Fast Hessenberg QR Iteration for Companion Matrices p.10/20
25 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 ˆV 2 U 1 ˆV 3 U 1 W 2 W 3 V B 21 B 22 B 23 Û 2 V B 32 B 33 Û 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Split blocks once bulge is within B 22 Continue until Hessenberg form is restored Fast Hessenberg QR Iteration for Companion Matrices p.10/20
26 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 V2 U 1 W 2 V 3 B 21 B 22 U 2 V 3 U 1 W 2 W 3 V U 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Split blocks once bulge is within B 22 Continue until Hessenberg form is restored Fast Hessenberg QR Iteration for Companion Matrices p.10/20
27 SSS structure transformed B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V B 11 U 1 V2 U 1 W 2 ˆV 3 U 1 W 2 ˆV B 21 B 22 U 2 ˆV 3 U 2 ˆV B 32 B 33 B 3 B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Split blocks once bulge is within B 22 Continue until Hessenberg form is restored Fast Hessenberg QR Iteration for Companion Matrices p.10/20
28 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 V 2 U 1 W 2 V 3 U 1 W 2 W 3 V B 21 B 22 U 2 V 3 U 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Split blocks once bulge is within B 22 Continue until Hessenberg form is restored Fast Hessenberg QR Iteration for Companion Matrices p.10/20
29 SSS structure transformed B 11 B 11 U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V U 1 V2 U 1 W 2 V3 U 1 W 2 W 3 V B 21 B 22 U 2 V3 U 2 W 3 V B 32 B 33 U 3 V B 3 B Start with an SSS representation Bulge chase in B 11 : changes B 11 and U 1 Merge first two blocks and chase bulge across boundary Split blocks once bulge is within B 22 Continue until Hessenberg form is restored Fast Hessenberg QR Iteration for Companion Matrices p.10/20
30 Row and column bases Define C j and G j by the recurrences C 1 = U 1 and C j = G n = V n and G j = [ ] C j 1 W j [ U j V v G j+1 W j for j > 1 ] for j < n. C j and G j are row and column bases for the submatrix left of block column j and above block row j. All C j orthonormal: left proper form. All G j orthonormal: right proper form. Fast Hessenberg QR Iteration for Companion Matrices p.11/20
31 Splitting blocks Start with C i 1 and G i+1 orthonormal. H 1,i... H 1,n.. = H i,i... H i,n [ C i I ] [ Vi H ii W i U i ] [ 0 G i+1 I 0 ]. [ V j B jj W j U j ] Vj α Bjj αα B βα jj W α j V β j U α j V β j B ββ jj W α j W β j U α j W β j U β j Do pivoted [ QR ] or SVD decomposition to split U, V, W. Wj α Choose Uj α with orthonormal columns. Fast Hessenberg QR Iteration for Companion Matrices p.12/20
32 Balancing Want to balance matrix to improve accuracy. But general diagonal scaling destroys structure! However: C is a companion matrix with Perron vector: Let hen DCD 1 x i = λ i D = diag(x 1 i ) = diag(λ i ). Is optimally balanced in the -norm. Is λ orthogonal + low rank Is scaling the polynomial indeterminate by λ. Fast Hessenberg QR Iteration for Companion Matrices p.13/20
33 Implementation Modified DLAHQR (basic double-shift QR) to use the new data structure. Convert to right proper form. Choose shifts (logic from LAPACK). Bulge chase top to bottom / convert to left proper form. Check for convergence / deflate (logic from LAPACK). Most of the code accesses data in the band part of the structure. his text is identical to the standard code just change LDA. Fast Hessenberg QR Iteration for Companion Matrices p.1/20
34 Performance Compared against LAPACK DLAHQR and DHSEQR Compiled using g77/gcc with -O2 flag LAPACK with standard optimizations, ALAS BLAS P processor at 1.5 GHz Polynomials of degree Fast Hessenberg QR Iteration for Companion Matrices p.15/20
35 Performance: QR-based Polynomial roots Fast DLAHQR (QR) DLAHQR DHSEQR 00 ime (s) Polynomial degree Fast Hessenberg QR Iteration for Companion Matrices p.16/20
36 Performance: QR-based 10 3 Polynomial roots Fast DLAHQR (QR) DLAHQR DHSEQR ime (s) Polynomial degree Fast Hessenberg QR Iteration for Companion Matrices p.17/20
37 Performance: SVD-based Polynomial roots Fast DLAHQR (SVD) DLAHQR DHSEQR 00 ime (s) Polynomial degree Fast Hessenberg QR Iteration for Companion Matrices p.18/20
38 Performance: SVD-based 10 3 Polynomial roots Fast DLAHQR (SVD) DLAHQR DHSEQR ime (s) Polynomial degree Fast Hessenberg QR Iteration for Companion Matrices p.19/20
39 Conclusions Hessenberg({ symmetric, skew, unitary } + low rank) has low off-diagonal rank. Applies to (matrix) polynomial root finding, some damped vibration calculations. Can use low-rank structure for fast bulge-chasing. Can use-use most existing QR lore and code. Appears backward stable with SVD-based splitting. Fast Hessenberg QR Iteration for Companion Matrices p.20/20
Index. for generalized eigenvalue problem, butterfly form, 211
Index ad hoc shifts, 165 aggressive early deflation, 205 207 algebraic multiplicity, 35 algebraic Riccati equation, 100 Arnoldi process, 372 block, 418 Hamiltonian skew symmetric, 420 implicitly restarted,
Section 4.5 Eigenvalues of Symmetric Tridiagonal Matrices
Section 4.5 Eigenvalues of Symmetric Tridiagonal Matrices Key Terms Symmetric matrix Tridiagonal matrix Orthogonal matrix QR-factorization Rotation matrices (plane rotations) Eigenvalues We will now complete
Orthogonal iteration to QR
Notes for 2016-03-09 Orthogonal iteration to QR The QR iteration is the workhorse for solving the nonsymmetric eigenvalue problem. Unfortunately, while the iteration itself is simple to write, the derivation
Numerical Methods in Matrix Computations
Ake Bjorck Numerical Methods in Matrix Computations Springer Contents 1 Direct Methods for Linear Systems 1 1.1 Elements of Matrix Theory 1 1.1.1 Matrix Algebra 2 1.1.2 Vector Spaces 6 1.1.3 Submatrices
QR-decomposition. The QR-decomposition of an n k matrix A, k n, is an n n unitary matrix Q and an n k upper triangular matrix R for which A = QR
QR-decomposition The QR-decomposition of an n k matrix A, k n, is an n n unitary matrix Q and an n k upper triangular matrix R for which In Matlab A = QR [Q,R]=qr(A); Note. The QR-decomposition is unique
Numerical Methods I Eigenvalue Problems
Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 2nd, 2014 A. Donev (Courant Institute) Lecture
Matrix decompositions
Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The
13-2 Text: 28-30; AB: 1.3.3, 3.2.3, 3.4.2, 3.5, 3.6.2; GvL Eigen2
The QR algorithm The most common method for solving small (dense) eigenvalue problems. The basic algorithm: QR without shifts 1. Until Convergence Do: 2. Compute the QR factorization A = QR 3. Set A :=
Eigenvalue Problems and Singular Value Decomposition
Eigenvalue Problems and Singular Value Decomposition Sanzheng Qiao Department of Computing and Software McMaster University August, 2012 Outline 1 Eigenvalue Problems 2 Singular Value Decomposition 3 Software
Elementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
Lecture 2: Numerical linear algebra
Lecture 2: Numerical linear algebra QR factorization Eigenvalue decomposition Singular value decomposition Conditioning of a problem Floating point arithmetic and stability of an algorithm Linear algebra
APPLIED NUMERICAL LINEAR ALGEBRA
APPLIED NUMERICAL LINEAR ALGEBRA James W. Demmel University of California Berkeley, California Society for Industrial and Applied Mathematics Philadelphia Contents Preface 1 Introduction 1 1.1 Basic Notation
The Singular Value Decomposition
The Singular Value Decomposition An Important topic in NLA Radu Tiberiu Trîmbiţaş Babeş-Bolyai University February 23, 2009 Radu Tiberiu Trîmbiţaş ( Babeş-Bolyai University)The Singular Value Decomposition
Numerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
Linear Analysis Lecture 16
Linear Analysis Lecture 16 The QR Factorization Recall the Gram-Schmidt orthogonalization process. Let V be an inner product space, and suppose a 1,..., a n V are linearly independent. Define q 1,...,
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 13
STAT 309: MATHEMATICAL COMPUTATIONS I FALL 208 LECTURE 3 need for pivoting we saw that under proper circumstances, we can write A LU where 0 0 0 u u 2 u n l 2 0 0 0 u 22 u 2n L l 3 l 32, U 0 0 0 l n l
forms Christopher Engström November 14, 2014 MAA704: Matrix factorization and canonical forms Matrix properties Matrix factorization Canonical forms
Christopher Engström November 14, 2014 Hermitian LU QR echelon Contents of todays lecture Some interesting / useful / important of matrices Hermitian LU QR echelon Rewriting a as a product of several matrices.
Index. book 2009/5/27 page 121. (Page numbers set in bold type indicate the definition of an entry.)
page 121 Index (Page numbers set in bold type indicate the definition of an entry.) A absolute error...26 componentwise...31 in subtraction...27 normwise...31 angle in least squares problem...98,99 approximation
LinGloss. A glossary of linear algebra
LinGloss A glossary of linear algebra Contents: Decompositions Types of Matrices Theorems Other objects? Quasi-triangular A matrix A is quasi-triangular iff it is a triangular matrix except its diagonal
The Singular Value Decomposition
The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will
Eigenvalue problems. Eigenvalue problems
Determination of eigenvalues and eigenvectors Ax x, where A is an N N matrix, eigenvector x 0, and eigenvalues are in general complex numbers In physics: - Energy eigenvalues in a quantum mechanical system
Applied Mathematics 205. Unit V: Eigenvalue Problems. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit V: Eigenvalue Problems Lecturer: Dr. David Knezevic Unit V: Eigenvalue Problems Chapter V.2: Fundamentals 2 / 31 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors of
5 Selected Topics in Numerical Linear Algebra
5 Selected Topics in Numerical Linear Algebra In this chapter we will be concerned mostly with orthogonal factorizations of rectangular m n matrices A The section numbers in the notes do not align with
Eigenvalues and Eigenvectors
Chapter 1 Eigenvalues and Eigenvectors Among problems in numerical linear algebra, the determination of the eigenvalues and eigenvectors of matrices is second in importance only to the solution of linear
LAPACK-Style Codes for Pivoted Cholesky and QR Updating. Hammarling, Sven and Higham, Nicholas J. and Lucas, Craig. MIMS EPrint: 2006.
LAPACK-Style Codes for Pivoted Cholesky and QR Updating Hammarling, Sven and Higham, Nicholas J. and Lucas, Craig 2007 MIMS EPrint: 2006.385 Manchester Institute for Mathematical Sciences School of Mathematics
Orthogonal iteration to QR
Week 10: Wednesday and Friday, Oct 24 and 26 Orthogonal iteration to QR On Monday, we went through a somewhat roundabout algbraic path from orthogonal subspace iteration to the QR iteration. Let me start
This can be accomplished by left matrix multiplication as follows: I
1 Numerical Linear Algebra 11 The LU Factorization Recall from linear algebra that Gaussian elimination is a method for solving linear systems of the form Ax = b, where A R m n and bran(a) In this method
Chasing the Bulge. Sebastian Gant 5/19/ The Reduction to Hessenberg Form 3
Chasing the Bulge Sebastian Gant 5/9/207 Contents Precursers and Motivation 2 The Reduction to Hessenberg Form 3 3 The Algorithm 5 4 Concluding Remarks 8 5 References 0 ntroduction n the early days of
Nonlinear palindromic eigenvalue problems and their numerical solution
Nonlinear palindromic eigenvalue problems and their numerical solution TU Berlin DFG Research Center Institut für Mathematik MATHEON IN MEMORIAM RALPH BYERS Polynomial eigenvalue problems k P(λ) x = (
11.3 Eigenvalues and Eigenvectors of a Tridiagonal Matrix
11.3 Eigenvalues and Eigenvectors of a ridiagonal Matrix Evaluation of the Characteristic Polynomial Once our original, real, symmetric matrix has been reduced to tridiagonal form, one possible way to
The Future of LAPACK and ScaLAPACK
The Future of LAPACK and ScaLAPACK Jason Riedy, Yozo Hida, James Demmel EECS Department University of California, Berkeley November 18, 2005 Outline Survey responses: What users want Improving LAPACK and
Orthonormal Transformations and Least Squares
Orthonormal Transformations and Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 30, 2009 Applications of Qx with Q T Q = I 1. solving
LAPACK-Style Codes for Pivoted Cholesky and QR Updating
LAPACK-Style Codes for Pivoted Cholesky and QR Updating Sven Hammarling 1, Nicholas J. Higham 2, and Craig Lucas 3 1 NAG Ltd.,Wilkinson House, Jordan Hill Road, Oxford, OX2 8DR, England, sven@nag.co.uk,
Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
A Fast Implicit QR Eigenvalue Algorithm for Companion Matrices
A Fast Implicit QR Eigenvalue Algorithm for Companion Matrices D. A. Bini a,1 P. Boito a,1 Y. Eidelman b L. Gemignani a,1 I. Gohberg b a Dipartimento di Matematica, Università di Pisa, Largo Bruno Pontecorvo
Orthogonal iteration revisited
Week 10 11: Friday, Oct 30 and Monday, Nov 2 Orthogonal iteration revisited Last time, we described a generalization of the power methods to compute invariant subspaces. That is, starting from some initial
Chapter 6 Inner product spaces
Chapter 6 Inner product spaces 6.1 Inner products and norms Definition 1 Let V be a vector space over F. An inner product on V is a function, : V V F such that the following conditions hold. x+z,y = x,y
Parallel Singular Value Decomposition. Jiaxing Tan
Parallel Singular Value Decomposition Jiaxing Tan Outline What is SVD? How to calculate SVD? How to parallelize SVD? Future Work What is SVD? Matrix Decomposition Eigen Decomposition A (non-zero) vector
Linear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations
Parallel Numerical Algorithms
Parallel Numerical Algorithms Chapter 6 Matrix Models Section 6.2 Low Rank Approximation Edgar Solomonik Department of Computer Science University of Illinois at Urbana-Champaign CS 554 / CSE 512 Edgar
I. Multiple Choice Questions (Answer any eight)
Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam - Course Instructor : Prashanth L.A. Date : Sep-24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY
Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Algorithms Notes for 2016-10-31 There are several flavors of symmetric eigenvalue solvers for which there is no equivalent (stable) nonsymmetric solver. We discuss four algorithmic ideas: the workhorse
Introduction to Numerical Linear Algebra II
Introduction to Numerical Linear Algebra II Petros Drineas These slides were prepared by Ilse Ipsen for the 2015 Gene Golub SIAM Summer School on RandNLA 1 / 49 Overview We will cover this material in
Index. Copyright (c)2007 The Society for Industrial and Applied Mathematics From: Matrix Methods in Data Mining and Pattern Recgonition By: Lars Elden
Index 1-norm, 15 matrix, 17 vector, 15 2-norm, 15, 59 matrix, 17 vector, 15 3-mode array, 91 absolute error, 15 adjacency matrix, 158 Aitken extrapolation, 157 algebra, multi-linear, 91 all-orthogonality,
Solving linear equations with Gaussian Elimination (I)
Term Projects Solving linear equations with Gaussian Elimination The QR Algorithm for Symmetric Eigenvalue Problem The QR Algorithm for The SVD Quasi-Newton Methods Solving linear equations with Gaussian
Solution of Linear Equations
Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass
ECS130 Scientific Computing Handout E February 13, 2017
ECS130 Scientific Computing Handout E February 13, 2017 1. The Power Method (a) Pseudocode: Power Iteration Given an initial vector u 0, t i+1 = Au i u i+1 = t i+1 / t i+1 2 (approximate eigenvector) θ
A fast randomized algorithm for approximating an SVD of a matrix
A fast randomized algorithm for approximating an SVD of a matrix Joint work with Franco Woolfe, Edo Liberty, and Vladimir Rokhlin Mark Tygert Program in Applied Mathematics Yale University Place July 17,
Math 504 (Fall 2011) 1. (*) Consider the matrices
Math 504 (Fall 2011) Instructor: Emre Mengi Study Guide for Weeks 11-14 This homework concerns the following topics. Basic definitions and facts about eigenvalues and eigenvectors (Trefethen&Bau, Lecture
Last Time. Social Network Graphs Betweenness. Graph Laplacian. Girvan-Newman Algorithm. Spectral Bisection
Eigenvalue Problems Last Time Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection λ 2, w 2 Today Small deviation into eigenvalue problems Formulation Standard eigenvalue
Lecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Eigenvalue Problems; Similarity Transformations Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 18 Eigenvalue
arxiv: v1 [math.na] 24 Jan 2019
![arxiv: v1 [math.na] 24 Jan 2019](/thumbs/88/116199563.jpg "arxiv: v1 [math.na] 24 Jan 2019")
ON COMPUTING EFFICIENT DATA-SPARSE REPRESENTATIONS OF UNITARY PLUS LOW-RANK MATRICES R BEVILACQUA, GM DEL CORSO, AND L GEMIGNANI arxiv:190108411v1 [mathna 24 Jan 2019 Abstract Efficient eigenvalue solvers
1 Singular Value Decomposition and Principal Component
Singular Value Decomposition and Principal Component Analysis In these lectures we discuss the SVD and the PCA, two of the most widely used tools in machine learning. Principal Component Analysis (PCA)
FAST CONDITION ESTIMATION FOR A CLASS OF STRUCTURED EIGENVALUE PROBLEMS
SIAM J. MATRIX ANAL. APPL. Vol. 30, No. 4, pp. 1658 1676 c 2009 Society for Industrial and Applied Mathematics FAST CONDITION ESTIMATION FOR A CLASS OF STRUCTURED EIGENVALUE PROBLEMS A. J. LAUB AND J.
Eigenvalue and Eigenvector Problems
Eigenvalue and Eigenvector Problems An attempt to introduce eigenproblems Radu Trîmbiţaş Babeş-Bolyai University April 8, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) Eigenvalue and Eigenvector Problems
Krylov Subspace Methods that Are Based on the Minimization of the Residual
Chapter 5 Krylov Subspace Methods that Are Based on the Minimization of the Residual Remark 51 Goal he goal of these methods consists in determining x k x 0 +K k r 0,A such that the corresponding Euclidean
Double-shift QR steps
Double-shift QR steps Week 11: Monday, Oct 29 Last time, we discussed the Wilkinson strategy of choosing as a shift one of the roots of the trailing 2-by-2 submatrix of A (k) (the one closest to the final
Why the QR Factorization can be more Accurate than the SVD
Why the QR Factorization can be more Accurate than the SVD Leslie V. Foster Department of Mathematics San Jose State University San Jose, CA 95192 foster@math.sjsu.edu May 10, 2004 Problem: or Ax = b for
Matrix Algorithms. Volume II: Eigensystems. G. W. Stewart H1HJ1L. University of Maryland College Park, Maryland
Matrix Algorithms Volume II: Eigensystems G. W. Stewart University of Maryland College Park, Maryland H1HJ1L Society for Industrial and Applied Mathematics Philadelphia CONTENTS Algorithms Preface xv xvii
Least Squares. Tom Lyche. October 26, Centre of Mathematics for Applications, Department of Informatics, University of Oslo
Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 26, 2010 Linear system Linear system Ax = b, A C m,n, b C m, x C n. under-determined
5.3 The Power Method Approximation of the Eigenvalue of Largest Module
192 5 Approximation of Eigenvalues and Eigenvectors 5.3 The Power Method The power method is very good at approximating the extremal eigenvalues of the matrix, that is, the eigenvalues having largest and
Linear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
Math Fall Final Exam
Math 104 - Fall 2008 - Final Exam Name: Student ID: Signature: Instructions: Print your name and student ID number, write your signature to indicate that you accept the honor code. During the test, you
Eigenvalues and eigenvectors
Chapter 6 Eigenvalues and eigenvectors An eigenvalue of a square matrix represents the linear operator as a scaling of the associated eigenvector, and the action of certain matrices on general vectors
Block Lanczos Tridiagonalization of Complex Symmetric Matrices
Block Lanczos Tridiagonalization of Complex Symmetric Matrices Sanzheng Qiao, Guohong Liu, Wei Xu Department of Computing and Software, McMaster University, Hamilton, Ontario L8S 4L7 ABSTRACT The classic
Applied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
Eigenvalue Problems. Eigenvalue problems occur in many areas of science and engineering, such as structural analysis
Eigenvalue Problems Eigenvalue problems occur in many areas of science and engineering, such as structural analysis Eigenvalues also important in analyzing numerical methods Theory and algorithms apply
Fall TMA4145 Linear Methods. Exercise set Given the matrix 1 2
Norwegian University of Science and Technology Department of Mathematical Sciences TMA445 Linear Methods Fall 07 Exercise set Please justify your answers! The most important part is how you arrive at an
SUMMARY OF MATH 1600
SUMMARY OF MATH 1600 Note: The following list is intended as a study guide for the final exam. It is a continuation of the study guide for the midterm. It does not claim to be a comprehensive list. You
Scientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 4 Eigenvalue Problems Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
LARGE SPARSE EIGENVALUE PROBLEMS
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization 14-1 General Tools for Solving Large Eigen-Problems
1. The Polar Decomposition
A PERSONAL INTERVIEW WITH THE SINGULAR VALUE DECOMPOSITION MATAN GAVISH Part. Theory. The Polar Decomposition In what follows, F denotes either R or C. The vector space F n is an inner product space with
From Matrix to Tensor. Charles F. Van Loan
From Matrix to Tensor Charles F. Van Loan Department of Computer Science January 28, 2016 From Matrix to Tensor From Tensor To Matrix 1 / 68 What is a Tensor? Instead of just A(i, j) it s A(i, j, k) or
G1110 & 852G1 Numerical Linear Algebra
The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the
Numerische Mathematik
Numer. Math. (21) 116:177 212 DOI 1.17/s211-1-32-y Numerische Mathematik Implicit double shift QR-algorithm for companion matrices Marc Van Barel Raf Vandebril Paul Van Dooren Katrijn Frederix Received:
Linear algebra & Numerical Analysis
Linear algebra & Numerical Analysis Eigenvalues and Eigenvectors Marta Jarošová http://homel.vsb.cz/~dom033/ Outline Methods computing all eigenvalues Characteristic polynomial Jacobi method for symmetric
Solving large scale eigenvalue problems
arge scale eigenvalue problems, Lecture 5, March 23, 2016 1/30 Lecture 5, March 23, 2016: The QR algorithm II http://people.inf.ethz.ch/arbenz/ewp/ Peter Arbenz Computer Science Department, ETH Zürich
Solution of eigenvalue problems. Subspace iteration, The symmetric Lanczos algorithm. Harmonic Ritz values, Jacobi-Davidson s method
Solution of eigenvalue problems Introduction motivation Projection methods for eigenvalue problems Subspace iteration, The symmetric Lanczos algorithm Nonsymmetric Lanczos procedure; Implicit restarts
5.6. PSEUDOINVERSES 101. A H w.
5.6. PSEUDOINVERSES 0 Corollary 5.6.4. If A is a matrix such that A H A is invertible, then the least-squares solution to Av = w is v = A H A ) A H w. The matrix A H A ) A H is the left inverse of A and
Math 577 Assignment 7
Math 577 Assignment 7 Thanks for Yu Cao 1. Solution. The linear system being solved is Ax = 0, where A is a (n 1 (n 1 matrix such that 2 1 1 2 1 A =......... 1 2 1 1 2 and x = (U 1, U 2,, U n 1. By the
Numerical Methods. Elena loli Piccolomini. Civil Engeneering. piccolom. Metodi Numerici M p. 1/??
Metodi Numerici M p. 1/?? Numerical Methods Elena loli Piccolomini Civil Engeneering http://www.dm.unibo.it/ piccolom elena.loli@unibo.it Metodi Numerici M p. 2/?? Least Squares Data Fitting Measurement
1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)
Notes for 2017-01-30 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty
Direct methods for symmetric eigenvalue problems
Direct methods for symmetric eigenvalue problems, PhD McMaster University School of Computational Engineering and Science February 4, 2008 1 Theoretical background Posing the question Perturbation theory
AMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview & Matrix-Vector Multiplication Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 20 Outline 1 Course
LARGE SPARSE EIGENVALUE PROBLEMS. General Tools for Solving Large Eigen-Problems
LARGE SPARSE EIGENVALUE PROBLEMS Projection methods The subspace iteration Krylov subspace methods: Arnoldi and Lanczos Golub-Kahan-Lanczos bidiagonalization General Tools for Solving Large Eigen-Problems
Lecture 5 Singular value decomposition
Lecture 5 Singular value decomposition Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn
Orthonormal Transformations
Orthonormal Transformations Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 25, 2010 Applications of transformation Q : R m R m, with Q T Q = I 1.
Math 102 Final Exam - Dec 14 - PCYNH pm Fall Name Student No. Section A0
Math 12 Final Exam - Dec 14 - PCYNH 122-6pm Fall 212 Name Student No. Section A No aids allowed. Answer all questions on test paper. Total Marks: 4 8 questions (plus a 9th bonus question), 5 points per
Lecture Notes for Inf-Mat 3350/4350, Tom Lyche
Lecture Notes for Inf-Mat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector
DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix to upper-triangular
form) Given: matrix C = (c i,j ) n,m i,j=1 ODE and num math: Linear algebra (N) [lectures] c phabala 2016 DEN: Linear algebra numerical view (GEM: Gauss elimination method for reducing a full rank matrix
Preface to Second Edition... vii. Preface to First Edition...
Contents Preface to Second Edition..................................... vii Preface to First Edition....................................... ix Part I Linear Algebra 1 Basic Vector/Matrix Structure and
Linear Algebra (Review) Volker Tresp 2017
Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.
EIGENVALUE PROBLEMS. EIGENVALUE PROBLEMS p. 1/4
EIGENVALUE PROBLEMS EIGENVALUE PROBLEMS p. 1/4 EIGENVALUE PROBLEMS p. 2/4 Eigenvalues and eigenvectors Let A C n n. Suppose Ax = λx, x 0, then x is a (right) eigenvector of A, corresponding to the eigenvalue
EECS 275 Matrix Computation
EECS 275 Matrix Computation Ming-Hsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 17 1 / 26 Overview
Math 407: Linear Optimization
Math 407: Linear Optimization Lecture 16: The Linear Least Squares Problem II Math Dept, University of Washington February 28, 2018 Lecture 16: The Linear Least Squares Problem II (Math Dept, University
Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated.
Math 504, Homework 5 Computation of eigenvalues and singular values Recall that your solutions to these questions will not be collected or evaluated 1 Find the eigenvalues and the associated eigenspaces
Applied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.3: QR Factorization, SVD 2 / 66 QR Factorization 3 / 66 QR Factorization