UMIACSTR July CSTR 2721 Revised March Perturbation Theory for. Rectangular Matrix Pencils. G. W. Stewart.


 Jean Wilkins
 10 months ago
 Views:
Transcription
1 UMIASTR95 July 99 STR 272 Revised March 993 Perturbation Theory for Rectangular Matrix Pencils G. W. Stewart abstract The theory of eigenvalues and eigenvectors of rectangular matrix pencils is complicated by the fact that arbitrarily small perturbations of the pencil can cause them disappear. However, there are applications in which the properties of the pencil ensure the existence of eigenvalues and eigenvectors. In this paper it is shown how to develop a perturbation theory for such pencils. Department of omputer Science and Institute for Advanced omputer Studies, University of Maryland, ollege Park, MD This work was supported in part by the Air Force Oce of Scientic Research under ontract AFOSR This report is available by anonymous ftp from thales.cs.umd.edu in the directory pub/reports.
2 Perturbation Theory for Rectangular Matrix Pencils G. W. Stewart ASTRAT The theory of eigenvalues and eigenvectors of rectangular matrix pencils is complicated by the fact that arbitrarily small perturbations of the pencil can cause them disappear. However, there are applications in which the properties of the pencil ensure the existence of eigenvalues and eigenvectors. In this paper it is shown how to develop a perturbation theory for such pencils. In this note we will be concerned with the perturbation theory for the generalized eigenvalue problem Ax = x; () where A and are m n matrices with m > n and and are normalized so that jj 2 + jj 2 = : Although rectangular matrix pencils have a long history and a well developed theory of canonical forms (e.g., see [, 5, 6]), their eigenvalues and eigenvectors have been less well studied. There are two reasons. In the rst place, eigenvalues and eigenvectors may fail to exist. For example, if A = and = ; then () has no nontrivial solution unless =. In the second place, even if () has a solution, an arbitrarily small perturbation can make it go away, as the above example also shows. It is not easy to construct a general perturbation theory for objects that may not exist and can vanish at the drop of a hat. Nonetheless, in some applications the nature of the matrices A and insure the existence of eigenvectors, and the conditions that do so are stable under small perturbations (for an example from game theory, see [3]). In these circumstances The homogeneous form given here seems preferable to the more conventional form Ax = x. We call the pair (; ) an eigenvalue of the problem.
3 2 Rectangular Matrix Pencils it is reasonable to try to develop a perturbation theory. In this note we shall show how to do so by reducing the problem to a square one. We begin by describing the space of solutions of the unperturbed problem. First note that equation () says that Ax and x lie in the same onedimensional subspace. Generalizing this observation, we say that a subspace X of dimension k is an eigenspace of the pencil A? if there is a subspace Y of dimension k such that AX + X Y: (2) Here the sums and products are the usual Minkowski operations; e.g., AX = f Ax : x 2 X g. Since the sum of two eigenspaces is an eigenspace, there is a unique maximal eigenspace, which we shall call the eigenspace of the pencil. In what follows X will be the eigenspace of the pencil A? and Y will be the corresponding subspace in (2). We will assume that X has dimension k >. The relation between eigenspaces and eigenvectors can be seen as follows. Let X = (X X 2 ) and Y = (Y Y 2 ) be unitary matrices such that R(X ) = X and R(Y ) = Y. Then Y H AX and Y H X have the forms Y H AX = A A 2 A 22 and Y H X = 2 : (3) 22 It follows that if x is an eigenvector of the pencil A?, then X x is an eigenvector of (). The converse is also true: all eigenvectors of () have the form X x. This is a consequence of the fact that the pencil A 22? 22 does not have an eigenspace. For if it did, we could reduce A 22 and 22 as above, so that the transformed pencil assumes the form A ^A2 ^A3 ^A22 ^A23 ^A33 A? ^2 ^3 ^22 ^23 ^33 A : Then the original pencil has an eigenspace of dimension greater than k corresponding to the pencil A ^A2 ^2? ; ^A22 ^22
4 Rectangular Matrix Pencils 3 which contradicts the maximality of the subspace. In what follows, we will assume that the matrices of the pencil A? are in the reduced form (3). Note that since the reduction is a unitary equivalence any perturbation in the original pencil corresponds (in a unitarily invariant norm) to a perturbation of the same size in the reduced form. Let us now consider the perturbed matrices and where A = = A A2 E 2 A22 A + E A 2 + E 2 E 2 A 22 + E 22 2 F F 2 + F 2 ; F F 22 ke ij k; kf ij k ; i; j = ; 2: (4) Here k k denotes both the Euclidean vector norm and the subordinate spectral matrix norm. Let x = (x H x H 2 ) H be a normalized eigenvector of the corresponding pencil: Ax = x; kxk = : We are going to show that under a natural condition (namely, that dened below is not small) the second component x 2 of x is small. First note that since the pencil A 22? 22 has no eigenvectors, there is a number > such that k A 22 x 2? 22 x 2 k kx 2 k: (5) Since jj 2 + j j 2 =, if then Since x is an eigenvector? p 2 > ; (6) k A 22 x 2? 22 x 2 k kx 2 k: = k (E 2 x? A 22 x 2 )? (F 2 x? 22 x 2 )k kx 2 k? k E 2 x? F 2 x k: Hence kx 2 k p 2 :
5 4 Rectangular Matrix Pencils Now let us perform the reduction procedure describe above on the perturbed pencil, but using only the onedimensional space spanned by the eigenvector x. Specically, let X = R Q P S be a unitary matrix whose rst column is x and for which kp k; kqk kx 2 k p 2 : (7) (The existence of of such a matrix is established in the appendix.) Let Y = be a unitary matrix whose rst column is proportional to Ax (or x if Ax = ). Then the rst k columns of Y H AX and Y H X have the forms a H A E A ^R ^P and ^Q ^S b H F A : (8) Thus (; ) is an eigenvalue of the pencil a H A? b H ut by direct computation, this pencil is or ( ^RH A R + ^RH E R + ^RH A2 P + ^QH E 2 R + ^QH A22 P )? where from (4) and (7) ( ^R H R + ^R H F R + ^R H A2 P + ^Q H F 2 R + ^Q H A22 P ) ( ^R H A R + G)? ( ^R H R + H); kgk; khk p 2 maxfk Ak; k k :
6 Rectangular Matrix Pencils 5 We have thus shown that the eigenvalue (; ) is an eigenvalue of a perturbation of the pencil ^R H A R? ^R H R, a pencil which corresponds to the eigenspace of the original problem. Since the pencil and its perturbation are square, we may use standard perturbation theory to bound the perturbation in the eigenvalue and its eigenvector (for a survey of the perturbation theory of the generalized eigenvalue problem, see [4, h. VI]). The dependence of the bounds on dened by (5) is entirely natural. If is small, then (; ) is almost an eigenvalue of the pencil A 22? 22 and could have come from the perturbed pencil A 22? 22. Such an eigenvalue need not be near an eigenvalue associated with the eigenspace of the original problem. Seen in this light, the condition (6) insures that the perturbed eigenvalue truly comes from the eigenspace of the original problem. Finally, we note that the general technique we have outlined here is at least as important as the particular bounds. For example, it can be extended to obtain perturbations bounds on eigenspaces. Again, if we work with nonorthogonal diagonalizing transformations, so that A 2 = 2 = in (3) and a H = b H = in (8), we obtain the result that and / + y H Ex + O(maxfkEk; kf kg 2 ) / + y H F x + O(maxfkEk; kf kg 2 ); where y H is the left eigenvector corresponding to (; ). The details are left to the reader. Appendix The following extension theorem is more general than we need but may be of independent interest. Let p q and let the matrix X = q p X n?p X 2 have orthonormal columns. Then there is a unitary matrix q p?q n?p p X X = X 2 X 3 n?p X 2 X 22 X 23
7 6 Rectangular Matrix Pencils such that k(x 2 X 22 )k = kx 2 k: To establish the theorem, rst assume that p+q n. y the S decomposition (e.g., see [2, p.77]) we may assume that X = q q p?q q n?p?q S A ; where and S are nonnegative diagonal matrices. Then the required extension is in which we take X = q p?q q n?p?q q?s p?q I q S n?p?q I (X 2 X 22 ) = S On the other hand, if p + q > n, we take X in the form X = In this case the required extension is p+q?n n?p p+q?n I n?p p?q n?p S : A A ; in which X = p+q?n n?p p?q n?p p+q?n I n?p?s p?q I A ; n?p S (X 2 X 22 ) = ( S ):
8 Rectangular Matrix Pencils 7 References [] F. R. Gantmacher (959). The Theory of Matrices, Vols. I, II. helsea Publishing ompany, New York. [2] G. H. Golub and. F. Van Loan (989). Matrix omputations. Johns Hopkins University Press, altimore, Maryland, 2nd edition. [3] E. Marchi, J. A. Oviedo, and J. E. ohen (99). \Pertrubation Theory of a Nonlinear Game of von Neuman." SIAM Journal on Matrix Analysis and Applications, 2, 592{596. [4] G. W. Stewart and G.J. Sun (99). Matrix Perturbation Theory. Academic Press, oston. [5] P. Van Dooren (979). \The omputation of Kronecker's anonical Form of a Singular Pencil." Linear Algebra and Its Applications, 27, 3{4. [6] J. H. Wilkinson (979). \Kronecker's anonical Form and the QZ Algorithm." Linear Algebra and Its Applications, 28, 285{33.
Institute for Advanced Computer Studies. Department of Computer Science. On the Perturbation of. LU and Cholesky Factors. G. W.
University of Maryland Institute for Advanced Computer Studies Department of Computer Science College Park TR{95{93 TR{3535 On the Perturbation of LU and Cholesky Factors G. W. Stewart y October, 1995
More informationInstitute for Advanced Computer Studies. Department of Computer Science. Two Algorithms for the The Ecient Computation of
University of Maryland Institute for Advanced Computer Studies Department of Computer Science College Park TR{98{12 TR{3875 Two Algorithms for the The Ecient Computation of Truncated Pivoted QR Approximations
More informationInstitute for Advanced Computer Studies. Department of Computer Science. On the Adjoint Matrix. G. W. Stewart y ABSTRACT
University of Maryland Institute for Advanced Computer Studies Department of Computer Science College Park TR{97{02 TR{3864 On the Adjoint Matrix G. W. Stewart y ABSTRACT The adjoint A A of a matrix A
More informationDefinition (T invariant subspace) Example. Example
Eigenvalues, Eigenvectors, Similarity, and Diagonalization We now turn our attention to linear transformations of the form T : V V. To better understand the effect of T on the vector space V, we begin
More informationthe Unitary Polar Factor æ RenCang Li P.O. Box 2008, Bldg 6012
Relative Perturbation Bounds for the Unitary Polar actor RenCang Li Mathematical Science Section Oak Ridge National Laboratory P.O. Box 2008, Bldg 602 Oak Ridge, TN 37836367 èli@msr.epm.ornl.govè LAPACK
More informationLECTURE VI: SELFADJOINT AND UNITARY OPERATORS MAT FALL 2006 PRINCETON UNIVERSITY
LECTURE VI: SELFADJOINT AND UNITARY OPERATORS MAT 204  FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO 1 Adjoint of a linear operator Note: In these notes, V will denote a ndimensional euclidean vector
More informationSingular Value Decomposition and Polar Form
Chapter 14 Singular Value Decomposition and Polar Form 14.1 Singular Value Decomposition for Square Matrices Let f : E! E be any linear map, where E is a Euclidean space. In general, it may not be possible
More informationParallel Singular Value Decomposition. Jiaxing Tan
Parallel Singular Value Decomposition Jiaxing Tan Outline What is SVD? How to calculate SVD? How to parallelize SVD? Future Work What is SVD? Matrix Decomposition Eigen Decomposition A (nonzero) vector
More informationGeneralized Shifted Inverse Iterations on Grassmann Manifolds 1
Proceedings of the Sixteenth International Symposium on Mathematical Networks and Systems (MTNS 2004), Leuven, Belgium Generalized Shifted Inverse Iterations on Grassmann Manifolds 1 J. Jordan α, P.A.
More informationThe Spectral Theorem for normal linear maps
MAT067 University of California, Davis Winter 2007 The Spectral Theorem for normal linear maps Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (March 14, 2007) In this section we come back to the question
More informationThe Lanczos and conjugate gradient algorithms
The Lanczos and conjugate gradient algorithms Gérard MEURANT October, 2008 1 The Lanczos algorithm 2 The Lanczos algorithm in finite precision 3 The nonsymmetric Lanczos algorithm 4 The Golub Kahan bidiagonalization
More informationChapter 3 Least Squares Solution of y = A x 3.1 Introduction We turn to a problem that is dual to the overconstrained estimation problems considered s
Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of Technology 1 1 c Chapter
More informationA Note on Eigenvalues of Perturbed Hermitian Matrices
A Note on Eigenvalues of Perturbed Hermitian Matrices ChiKwong Li RenCang Li July 2004 Let ( H1 E A = E H 2 Abstract and Ã = ( H1 H 2 be Hermitian matrices with eigenvalues λ 1 λ k and λ 1 λ k, respectively.
More informationNumerical Methods I: Eigenvalues and eigenvectors
1/25 Numerical Methods I: Eigenvalues and eigenvectors Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu November 2, 2017 Overview 2/25 Conditioning Eigenvalues and eigenvectors How hard are they
More informationIn particular, if A is a square matrix and λ is one of its eigenvalues, then we can find a nonzero column vector X with
Appendix: Matrix Estimates and the PerronFrobenius Theorem. This Appendix will first present some well known estimates. For any m n matrix A = [a ij ] over the real or complex numbers, it will be convenient
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationMATLAB TOOLS FOR SOLVING PERIODIC EIGENVALUE PROBLEMS 1
MATLAB TOOLS FOR SOLVING PERIODIC EIGENVALUE PROBLEMS 1 Robert Granat Bo Kågström Daniel Kressner,2 Department of Computing Science and HPC2N, Umeå University, SE90187 Umeå, Sweden. {granat,bokg,kressner}@cs.umu.se
More informationor H = UU = nx i=1 i u i u i ; where H is a nonsingular Hermitian matrix of order n, = diag( i ) is a diagonal matrix whose diagonal elements are the
Relative Perturbation Bound for Invariant Subspaces of Graded Indenite Hermitian Matrices Ninoslav Truhar 1 University Josip Juraj Strossmayer, Faculty of Civil Engineering, Drinska 16 a, 31000 Osijek,
More information5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.
Linear Algebra  Test File  Spring Test # For problems  consider the following system of equations. x + y  z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the
More informationTheorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k nk. mk k mk nk
I. REVIEW OF LINEAR ALGEBRA A. Equivalence Definition A1. If A and B are two m x n matrices, then A is equivalent to B if we can obtain B from A by a finite sequence of elementary row or elementary column
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationConstructing an orthonormal set of eigenvectors for DFT matrix using Gramians and determinants
Constructing an orthonormal set of eigenvectors for DFT matrix using Gramians and determinants Vadim Zaliva, lord@crocodile.org July 17, 2012 Abstract The problem of constructing an orthogonal set of eigenvectors
More informationProblems in Linear Algebra and Representation Theory
Problems in Linear Algebra and Representation Theory (Most of these were provided by Victor Ginzburg) The problems appearing below have varying level of difficulty. They are not listed in any specific
More informationLinear Algebra Final Exam Review
Linear Algebra Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.
More informationThe Eigenvalue Problem: Perturbation Theory
Jim Lambers MAT 610 Summer Session 200910 Lecture 13 Notes These notes correspond to Sections 7.2 and 8.1 in the text. The Eigenvalue Problem: Perturbation Theory The Unsymmetric Eigenvalue Problem Just
More informationReview and problem list for Applied Math I
Review and problem list for Applied Math I (This is a first version of a serious review sheet; it may contain errors and it certainly omits a number of topic which were covered in the course. Let me know
More informationLinear algebra and applications to graphs Part 1
Linear algebra and applications to graphs Part 1 Written up by Mikhail Belkin and Moon Duchin Instructor: Laszlo Babai June 17, 2001 1 Basic Linear Algebra Exercise 1.1 Let V and W be linear subspaces
More informationJordan Normal Form. Chapter Minimal Polynomials
Chapter 8 Jordan Normal Form 81 Minimal Polynomials Recall p A (x) =det(xi A) is called the characteristic polynomial of the matrix A Theorem 811 Let A M n Then there exists a unique monic polynomial q
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342  Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More informationLinear algebra 2. Yoav Zemel. March 1, 2012
Linear algebra 2 Yoav Zemel March 1, 2012 These notes were written by Yoav Zemel. The lecturer, Shmuel Berger, should not be held responsible for any mistake. Any comments are welcome at zamsh7@gmail.com.
More information216 S. Chandrasearan and I.C.F. Isen Our results dier from those of Sun [14] in two asects: we assume that comuted eigenvalues or singular values are
Numer. Math. 68: 215{223 (1994) Numerische Mathemati c SringerVerlag 1994 Electronic Edition Bacward errors for eigenvalue and singular value decomositions? S. Chandrasearan??, I.C.F. Isen??? Deartment
More informationHomework 2. Solutions T =
Homework. s Let {e x, e y, e z } be an orthonormal basis in E. Consider the following ordered triples: a) {e x, e x + e y, 5e z }, b) {e y, e x, 5e z }, c) {e y, e x, e z }, d) {e y, e x, 5e z }, e) {
More informationLECTURE VII: THE JORDAN CANONICAL FORM MAT FALL 2006 PRINCETON UNIVERSITY. [See also Appendix B in the book]
LECTURE VII: THE JORDAN CANONICAL FORM MAT 204  FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO [See also Appendix B in the book] 1 Introduction In Lecture IV we have introduced the concept of eigenvalue
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More informationSpectral Analysis of kbalanced Signed Graphs
Spectral Analysis of kbalanced Signed Graphs Leting Wu 1, Xiaowei Ying 1, Xintao Wu 1, Aidong Lu 1 and ZhiHua Zhou 2 1 University of North Carolina at Charlotte, USA, {lwu8,xying, xwu,alu1}@uncc.edu
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,
More informationBasic Concepts in Matrix Algebra
Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1
More informationTOEPLITZ OPERATORS. Toeplitz studied infinite matrices with NWSE diagonals constant. f e C :
TOEPLITZ OPERATORS EFTON PARK 1. Introduction to Toeplitz Operators Otto Toeplitz lived from 18811940 in Goettingen, and it was pretty rough there, so he eventually went to Palestine and eventually contracted
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationMatrix Mathematics. Theory, Facts, and Formulas with Application to Linear Systems Theory. Dennis S. Bernstein
Matrix Mathematics Theory, Facts, and Formulas with Application to Linear Systems Theory Dennis S. Bernstein PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD Contents Special Symbols xv Conventions, Notation,
More informationLinear Algebra Methods for Data Mining
Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 2. Basic Linear Algebra continued Linear Algebra Methods for Data Mining, Spring 2007, University of Helsinki
More information1. The Polar Decomposition
A PERSONAL INTERVIEW WITH THE SINGULAR VALUE DECOMPOSITION MATAN GAVISH Part. Theory. The Polar Decomposition In what follows, F denotes either R or C. The vector space F n is an inner product space with
More informationLeast squares Solution of Homogeneous Equations
Least squares Solution of Homogeneous Equations supportive text for teaching purposes Revision: 1.2, dated: December 15, 2005 Tomáš Svoboda Czech Technical University, Faculty of Electrical Engineering
More information1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)
Notes for 20170130 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty
More informationJordan normal form notes (version date: 11/21/07)
Jordan normal form notes (version date: /2/7) If A has an eigenbasis {u,, u n }, ie a basis made up of eigenvectors, so that Au j = λ j u j, then A is diagonal with respect to that basis To see this, let
More informationLinear Algebra. Workbook
Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx
More informationInvertibility and stability. Irreducibly diagonally dominant. Invertibility and stability, stronger result. Reducible matrices
Geršgorin circles Lecture 8: Outline Chapter 6 + Appendix D: Location and perturbation of eigenvalues Some other results on perturbed eigenvalue problems Chapter 8: Nonnegative matrices Geršgorin s Thm:
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationBASIC ALGORITHMS IN LINEAR ALGEBRA. Matrices and Applications of Gaussian Elimination. A 2 x. A T m x. A 1 x A T 1. A m x
BASIC ALGORITHMS IN LINEAR ALGEBRA STEVEN DALE CUTKOSKY Matrices and Applications of Gaussian Elimination Systems of Equations Suppose that A is an n n matrix with coefficents in a field F, and x = (x,,
More informationOffdiagonal perturbation, firstorder approximation and quadratic residual bounds for matrix eigenvalue problems
Offdiagonal perturbation, firstorder approximation and quadratic residual bounds for matrix eigenvalue problems Yuji Nakatsukasa Abstract When a symmetric block diagonal matrix [ A 1 A2 ] undergoes an
More informationLecture 7 Spectral methods
CSE 291: Unsupervised learning Spring 2008 Lecture 7 Spectral methods 7.1 Linear algebra review 7.1.1 Eigenvalues and eigenvectors Definition 1. A d d matrix M has eigenvalue λ if there is a ddimensional
More informationEigenvalues, Eigenvectors, and an Intro to PCA
Eigenvalues, Eigenvectors, and an Intro to PCA Eigenvalues, Eigenvectors, and an Intro to PCA Changing Basis We ve talked so far about rewriting our data using a new set of variables, or a new basis.
More informationJordan Canonical Form Homework Solutions
Jordan Canonical Form Homework Solutions For each of the following, put the matrix in Jordan canonical form and find the matrix S such that S AS = J. [ ]. A = A λi = λ λ = ( λ) = λ λ = λ =, Since we have
More informationAN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES
AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES JOEL A. TROPP Abstract. We present an elementary proof that the spectral radius of a matrix A may be obtained using the formula ρ(a) lim
More informationMath 25a Practice Final #1 Solutions
Math 25a Practice Final #1 Solutions Problem 1. Suppose U and W are subspaces of V such that V = U W. Suppose also that u 1,..., u m is a basis of U and w 1,..., w n is a basis of W. Prove that is a basis
More informationReview of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem
Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem Steven J. Miller June 19, 2004 Abstract Matrices can be thought of as rectangular (often square) arrays of numbers, or as
More informationAn indicator for the number of clusters using a linear map to simplex structure
An indicator for the number of clusters using a linear map to simplex structure Marcus Weber, Wasinee Rungsarityotin, and Alexander Schliep Zuse Institute Berlin ZIB Takustraße 7, D495 Berlin, Germany
More informationSystems of Algebraic Equations and Systems of Differential Equations
Systems of Algebraic Equations and Systems of Differential Equations Topics: 2 by 2 systems of linear equations Matrix expression; Ax = b Solving 2 by 2 homogeneous systems Functions defined on matrices
More informationA Note on Simple Nonzero Finite Generalized Singular Values
A Note on Simple Nonzero Finite Generalized Singular Values Wei Ma ZhengJian Bai December 21 212 Abstract In this paper we study the sensitivity and second order perturbation expansions of simple nonzero
More information2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
More informationChapter 8. Rigid transformations
Chapter 8. Rigid transformations We are about to start drawing figures in 3D. There are no builtin routines for this purpose in PostScript, and we shall have to start more or less from scratch in extending
More informationEnsembles and incomplete information
p. 1/32 Ensembles and incomplete information So far in this course, we have described quantum systems by states that are normalized vectors in a complex Hilbert space. This works so long as (a) the system
More informationHighestweight Theory: Verma Modules
Highestweight Theory: Verma Modules Math G4344, Spring 2012 We will now turn to the problem of classifying and constructing all finitedimensional representations of a complex semisimple Lie algebra (or,
More informationLinear Algebra using Dirac Notation: Pt. 2
Linear Algebra using Dirac Notation: Pt. 2 PHYS 476Q  Southern Illinois University February 6, 2018 PHYS 476Q  Southern Illinois University Linear Algebra using Dirac Notation: Pt. 2 February 6, 2018
More informationRecurrence Relations and Fast Algorithms
Recurrence Relations and Fast Algorithms Mark Tygert Research Report YALEU/DCS/RR343 December 29, 2005 Abstract We construct fast algorithms for decomposing into and reconstructing from linear combinations
More informationAlgebra I Fall 2007
MIT OpenCourseWare http://ocw.mit.edu 18.701 Algebra I Fall 007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 18.701 007 Geometry of the Special Unitary
More informationLinear Algebra and Dirac Notation, Pt. 2
Linear Algebra and Dirac Notation, Pt. 2 PHYS 500  Southern Illinois University February 1, 2017 PHYS 500  Southern Illinois University Linear Algebra and Dirac Notation, Pt. 2 February 1, 2017 1 / 14
More informationMath 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.
Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses
More informationCOMPARISON OF MODE SHAPE VECTORS IN OPERATIONAL MODAL ANALYSIS DEALING WITH CLOSELY SPACED MODES.
IOMAC'5 6 th International Operational Modal Analysis Conference 5 May4 Gijón  Spain COMPARISON OF MODE SHAPE VECTORS IN OPERATIONAL MODAL ANALYSIS DEALING WITH CLOSELY SPACED MODES. Olsen P., and Brincker
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a ToolBox Linear Equation Systems Discretization of differential equations: solving linear equations
More informationMatrices and Vectors. Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A =
30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can
More informationComputational math: Assignment 1
Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange
More informationMultiple eigenvalues
Multiple eigenvalues arxiv:0711.3948v1 [math.na] 6 Nov 007 Joseph B. Keller Departments of Mathematics and Mechanical Engineering Stanford University Stanford, CA 9430515 June 4, 007 Abstract The dimensions
More informationDensity Matrices. Chapter Introduction
Chapter 15 Density Matrices 15.1 Introduction Density matrices are employed in quantum mechanics to give a partial description of a quantum system, one from which certain details have been omitted. For
More informationCS168: The Modern Algorithmic Toolbox Lecture #8: How PCA Works
CS68: The Modern Algorithmic Toolbox Lecture #8: How PCA Works Tim Roughgarden & Gregory Valiant April 20, 206 Introduction Last lecture introduced the idea of principal components analysis (PCA). The
More informationStat 159/259: Linear Algebra Notes
Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the
More informationUnsupervised dimensionality reduction
Unsupervised dimensionality reduction Guillaume Obozinski Ecole des Ponts  ParisTech SOCN course 2014 Guillaume Obozinski Unsupervised dimensionality reduction 1/30 Outline 1 PCA 2 Kernel PCA 3 Multidimensional
More informationLecture 6 Positive Definite Matrices
Linear Algebra Lecture 6 Positive Definite Matrices Prof. ChunHung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Spring 2017 2017/6/8 Lecture 6: Positive Definite Matrices
More informationSingular Value Decomposition and Polar Form
Chapter 12 Singular Value Decomposition and Polar Form 12.1 Singular Value Decomposition for Square Matrices Let f : E! E be any linear map, where E is a Euclidean space. In general, it may not be possible
More informationApplied Mathematics 205. Unit II: Numerical Linear Algebra. Lecturer: Dr. David Knezevic
Applied Mathematics 205 Unit II: Numerical Linear Algebra Lecturer: Dr. David Knezevic Unit II: Numerical Linear Algebra Chapter II.3: QR Factorization, SVD 2 / 66 QR Factorization 3 / 66 QR Factorization
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationEigenvalues and Eigenvectors
LECTURE 3 Eigenvalues and Eigenvectors Definition 3.. Let A be an n n matrix. The eigenvalueeigenvector problem for A is the problem of finding numbers λ and vectors v R 3 such that Av = λv. If λ, v are
More informationStrict diagonal dominance and a Geršgorin type theorem in Euclidean
Strict diagonal dominance and a Geršgorin type theorem in Euclidean Jordan algebras Melania Moldovan Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland
More informationKEYWORDS. Numerical methods, generalized singular values, products of matrices, quotients of matrices. Introduction The two basic unitary decompositio
COMPUTING THE SVD OF A GENERAL MATRIX PRODUCT/QUOTIENT GENE GOLUB Computer Science Department Stanford University Stanford, CA USA golub@sccm.stanford.edu KNUT SLNA SCCM Stanford University Stanford,
More informationMATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)
MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m
More informationExtra Problems for Math 2050 Linear Algebra I
Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as
More informationNOTICE WARNING CONCERNING COPYRIGHT RESTRICTIONS: The copyright law of the United States (title 17, U.S. Code) governs the making of photocopies or
NOTICE WARNING CONCERNING COPYRIGHT RESTRICTIONS: The copyright law of the United States (title 17, U.S. Code) governs the making of photocopies or other reproductions of copyrighted material. Any copying
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More information1 Linearity and Linear Systems
Mathematical Tools for Neuroscience (NEU 34) Princeton University, Spring 26 Jonathan Pillow Lecture 78 notes: Linear systems & SVD Linearity and Linear Systems Linear system is a kind of mapping f( x)
More information11.0 Introduction. An N N matrix A is said to have an eigenvector x and corresponding eigenvalue λ if. A x = λx (11.0.1)
Chapter 11. 11.0 Introduction Eigensystems An N N matrix A is said to have an eigenvector x and corresponding eigenvalue λ if A x = λx (11.0.1) Obviously any multiple of an eigenvector x will also be an
More informationMATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y.
as Basics Vectors MATRIX ALGEBRA An array of n real numbers x, x,, x n is called a vector and it is written x = x x n or x = x,, x n R n prime operation=transposing a column to a row Basic vector operations
More informationMajorizations for the Eigenvectors of GraphAdjacency Matrices: A Tool for Complex Network Design
Majorizations for the Eigenvectors of GraphAdjacency Matrices: A Tool for Complex Network Design Rahul Dhal Electrical Engineering and Computer Science Washington State University Pullman, WA rdhal@eecs.wsu.edu
More informationMATH 304 Linear Algebra Lecture 34: Review for Test 2.
MATH 304 Linear Algebra Lecture 34: Review for Test 2. Topics for Test 2 Linear transformations (Leon 4.1 4.3) Matrix transformations Matrix of a linear mapping Similar matrices Orthogonality (Leon 5.1
More information22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices
m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix
More informationThe Algebraic Multigrid Projection for Eigenvalue Problems; Backrotations and Multigrid Fixed Points. Sorin Costiner and Shlomo Ta'asan
The Algebraic Multigrid Projection for Eigenvalue Problems; Backrotations and Multigrid Fixed Points Sorin Costiner and Shlomo Ta'asan Department of Applied Mathematics and Computer Science The Weizmann
More informationI. Multiple Choice Questions (Answer any eight)
Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam  Course Instructor : Prashanth L.A. Date : Sep24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY
More informationFunctional Analysis Review
Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all
More informationHomework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I)
CS 106 Spring 2004 Homework 1 Elena Davidson 8 April 2004 Problem 1.1 Let B be a 4 4 matrix to which we apply the following operations: 1. double column 1, 2. halve row 3, 3. add row 3 to row 1, 4. interchange
More information