# POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS

Save this PDF as:
Size: px
Start display at page:

## Transcription

1 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR Abstract. Let A and E be real symmetric matrices. In this paper we are concerned with the determination of the values of t for which A + te is positive semidefinite. That is, we want to find the positive semidefinite interval for the matrix pencil t A + te. We first consider the cases when A is positive definite and when A is negative definite. In both of these cases, the positive semidefinite interval is determined from the eigenvalues of the matrix A 1 E. The results are extended to the case when A is semidefinite with range containing that of E and are combined to handle the indefinite case. 1. Introduction Let A and E be real symmetric n n matrices. The (linear) matrix pencil A + te is the function G : t A + te (t R). We consider the problem of determining the set T of those t for which G(t) is positive semidefinite. Since the set of positive semidefinite matrices is a closed convex cone in the space of n n matrices, T is a closed (possibly unbounded) interval. We call T the positive semidefinite interval. One source of interest in this problem is its connection to mathematical programming. Consider first the parametric quadratic program ([BC86], [Rit67], [Väl85]) min{c(t) x x C(t)x a i (t) x b i (t),i=1,,m}. If C(t) =A+tE, then the quadratic program is convex if and only if A+tE is positive semidefinite. The positive semidefinite interval gives the values of t for which the Karush-Kuhn-Tucker points are global minimizers. Consider also a semidefinite program ([WSV00], [VB96]) where the contraints require that the matrices n F j (x) :=A j 0 + x i A j i 0, j =1,,q i=1 Date: 21Feb05 4:16p. This research was supported by the National Sciences and Engineering Research Council. 1

2 2 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR be positive semidefinite. An application of a hit-and-run algorithm [BBRK + 87] for finding necessary constraints requires, at each iteration, the determination of T for each of the matrix pencils F j (ˆx + tv) =F j (ˆx)+t n i=1 v i A j i, j =1,,q where ˆx is the current iterate and v is a unit vector. If A is positive semidefinite and E is of rank one or two, then explicit expressions for the endpoints of T, and a method of computing them, are given in [CG86]. Valiaho ([Väl88]) extended these results, presenting a method of determining the inertia (the ordered triple of the numbers of positive, negative and zero eigenvalues) of A + te as a function of the parameter t. In this paper, motivated by applications to determination of constraint boundaries in semidefinite programs, we are only concerned with the determination of the endpoints of T. Section 2 gives some preliminary material. Section 3 discusses the case when A is nonsingular. Explicit expressions of the endpoints of the positive semidefinite interval are given when A is positive definite and when A is negative definite. In these cases, the endpoints are determined from the eigenvalues of A 1 E. Then we can combine these results to obtain the positive semidefinite interval when A is nonsingular and indefinite. Section 4 discusses the case when A is singular. Section 5 presents remarks on implementation and includes examples. 2. Preliminaries Throughout this paper, we shall denote the range space of a matrix M by R(M), its null space by N(M), and its rank by r(m). A real symmetric n n matrix M is positive definite (M 0), if x Mx > 0, for all x R n,x 0;M is positive semidefinite (M 0) if x Mx 0, for all x R n. It is well known that M is positive definite if and only if all eigenvalues of M are positive and M is positive semidefinite if and only if all eigenvalues of M are nonnegative. Since x Mx is continuous in x and affine in M, the set of positive definite matrices is an open convex set and the set of positive semidefinite matrices is a closed convex cone. It follows that the set T = {t R : G(t) 0} is an open interval, the positive definite interval (of G) and T = {t R : G(t) 0} =[t, t] is a closed interval, 1 the positive semidefinite interval. The only time T is the whole real line is if E = 0. Indeed, if x is an eigenvector of E with a non-zero eigenvalue, x x G(t)x is a straight line with a non-zero slope x Ex, so must be somewhere negative. Thus, except in the case E = 0, one of t and t is finite. If T is not empty, then its closure is T. Indeed, if (say) t 0 T and t is finite, then for each non-zero x in R n, x G(t 0 )x>0 and x G( t)x 0, so x G(t)x >0 for all t (t 0, t). 1 Here, and in the sequel, we will use [a, b] for {t R : a t b}. For example, if a = and b R, then [a, b] =(,b].

3 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 3 Given two real symmetric n n matrices A and E with A nonsingular, if det(a + te) =0, (1) then det( 1 t I A 1 E)=0; so 1 t is an eigenvalue of A 1 E. The solutions to equation (1), that is, the roots of the determining polynomial g(t) = det G(t), will determine the intervals T and T, as long as g is not identically 0. This is the case, for example, if one of A and E is non-singular. If g has no root, then T is either empty or the whole real line; if it has at least one root, then T must be bounded above or below or both. Let T 0 be the set of those roots t of the determining polynomial g for which G(t) 0. Clearly, T 0 consists of at most 2 points, the boundary points of T. Theorem 2.1. Suppose g 0. Then, the positive semidefinite interval T are determined as follows. (1) If T 0 =, then T =. (2) If T 0 consists of 2 points, then T is the closed interval they determine. (3) If T 0 consists of a single point t 0 and T is bounded, then T = T 0.In particular, this holds if t 0 is between 2 other roots of g, or any other 2 points a and b where G(a) and G(b) are not positive definite. (4) If T 0 consists of a single point t 0 and for some real a>t 0, and G(a) 0, then T =[t 0, + ). (5) If T 0 consists of a single point t 0 and for some real a<t 0, G(a) 0, then T =(,t 0 ]. If T is unbounded, then one of (4) or (5) holds (or E =0). Proof. The proof is straightforward. Consider, for example, (4). Then T is non-empty and T is its closure. Since there are no other possible endpoints, T =[t 0, + ). If E is positive definite, then A + te is positive definite for sufficiently large t. Lemma 2.2. Let E be positive definite. If max x =1 x ( A)x t> min x =1 x Ex, then A + te 0. Hence, T =[t, + ), where t is the largest root of g. Proof. Since E 0, for every x R n with x =1,x Ex > 0. Thus, for t> max x =1 x ( A)x min x =1 x Ex,

4 4 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR we have t min x =1 x Ex > max x =1 x ( A)x. Hence, for each x of norm 1, tx Ex > x ( A)x; that is, x (A + te)x >0; hence, A + te 0. Since T is unbounded on the right, it is non-empty of the form (t, + ) and T =[t, + ), its closure. Since T can contain no roots of g, t is the largest one. By writing A + te = A +( t)( E), we obtain the corresponding result for the case that E is negative definite. Lemma 2.3. Let E be negative definite. Then T =(, t ] and t is the smallest root of g. The eigenvalues λ i (t) ofg(t), (listed in descending order) are continuous functions of t. This follows, for example, from the Wielandt-Hoffman Theorem ([Wil65], page 104). From this it is clear that R is divided into intervals on which the inertia is constant. However, with our approach, we will have no need of this result. 3. The Positive Semidefinite Interval when A is Nonsingular In case A is nonsingular, the determining polynomial is not identically zero, so has finitely many roots, each negative inverses of eigenvalues of A 1 E. We present the results in three subsections, according to whether A is positive definite, negative definite, or indefinite A is positive definite. When A is positive definite, 0 T and T =[t, t] is its closure. Thus, t is either the smallest positive zero of g or is + ; t is either the largest negative root of g or is. Translating this into a statement about eigenvalues of A 1 E, we obtain: Theorem 3.1. Suppose that A is positive definite. Let δ n be the largest eigenvalue of A 1 E and δ 1 the smallest. The positive semidefinite interval for A + te is T =[t, t] where { 1 t = δ n, if δ n > 0, otherwise { t 1 = δ 1, if δ 1 < 0 +, otherwise. Special cases: If E is positive semidefinite, it is clear that T is not bounded above, that is, t =+. If E 0, we know therefore that T = [ 1/δ n, + ). If E is negative definite, then T =(, 1/δ 1 ], by Lemma 2.3.

5 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS A is negative definite. From Lemmas 2.2 and 2.3 we know the form of T when E is positive definite or negative definite. If E is neither and A is negative definite, we find T is empty. Lemma 3.2. Suppose that A is negative definite. If E is singular or indefinite, then T =. Proof. If E is singular, there exists y R n, y 0, such that It follows that for every t, y Ey =0. y (A + te)y <0, so that A + te is not positive semidefinite. If E is indefinite, then there exist u, v R n, such that u Eu > 0, v Ev < 0. Then, for t 0, u (A + te)u <0; and for t>0, v (A + te)v <0, so again there is no t for which G(t) is positive definite: T =. In terms of the eigenvalues of A 1 E, we obtain the following complete description of the T in the case A is negative definite. Theorem 3.3. Suppose A is negative definite. Let δ n be the largest eigenvalue of A 1 E and δ 1 the smallest. Then the positive semidefinite interval T is given as follows. (1) T =[ 1 δ n, + ), ifδ n < 0, (2) T =(, 1 δ 1 ],ifδ 1 > 0, and (3) T =, otherwise. Proof. There are three mutually exclusive exhaustive subcases. If E 0, then by Lemma 2.2, T =[t, + ), where t is the largest root of g. Since A = G(0), is negative definite, G(t) is negative definite for all t 0; hence all the roots of g are positive, or equivalently, all the eigenvalues of A 1 E are negative. The largest root of g corresponds to the largest eigenvalue of A 1 E, so we obtain (1). Similarly, E 0 (negative definite) corresponds to all the roots of g positive that is, all the eigenvalues of A 1 E negative and, using Lemma 2.3, we obtain (2). In the remaining case, where E is neither positive definite, nor negative definite, T is empty, by Lemma 3.2

6 6 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR 3.3. A is nonsingular and indefinite. We now combine the results in Section 3.1 and Section 3.2 to obtain information about the positive semidefinite interval when A is nonsingular and indefinite. Let Q be an orthogonal n n matrix such that Q D1 0 AQ =, (2) 0 D 2 where D 1,D 2 are diagonal and positive definite. Let Q E1 E EQ = 3 E3, E 2 where E 1 and E 2 have the same sizes as D 1 and D 2. Then Q D1 + te (A + te)q = 1 te 3 te3. D 2 + te 2 By Theorem 3.1 and Theorem 3.3, we can determine the positive semidefinite intervals for D 1 + te 1 and D 2 + te 2. We denote them by T 1 and T 2 and obtain: Theorem 3.4. T T 1 T 2 and if E 3 =0, T = T 1 T 2. If E 3 is not a zero matrix, then we may need to use brute force, relying on Theorem 2.1. Theorem 3.4 helps, since we know the endpoints (elements of T 0 ) must be in T 1 T 2. For example, we could first determine T 1 and T 2, then search inside T 1 T 2, for roots of the determining polynomial, stopping when 2 such are found or when one such is between points where G(t) is not positive definite. 4. The Positive Semidefinite Interval when A is Singular In this section, we are concerned with the positive semidefinite interval for the matrix pencil A + te when A is singular. Choose an orthogonal matrix Q for which Q AQ = D 1 0 D 2 0, (3) 0 where D 1,D 2 are diagonal and positive definite, extending the notation of the diagonalization (2). The corresponding decomposition of Q EQ has an extra column and row of blocks. E 1 E 3 E 4 Q EQ = E3 E 2 E 5. E4 E5 E 6 Let us write these as Q A1 0 AQ = and Q Ē1 Ē EQ = 2 Ē2 (4) E 6

7 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 7 Then Q A1 + tē1 tē2 (A + te)q =. tē 2 te 6 D1 0 Since A 1 = is nonsingular, the positive semidefinite interval 0 D 2 for the matrix pencil G 1 : t A 1 + tē1 can be found using Theorem 3.1, Theorem 3.3, or Theorem 3.4. Let T 3 denote the positive semidefinite interval for the matrix pencil te 6. Then, (1) E 6 =0 = T 3 = R. (2) E 6 0,E 6 0 = T 3 =[0, + ). (3) E 6 0,E 6 0 = T 3 =(, 0]. (4) E 6 is indefinite = T 3 = {0}. Note that if G(t) =A + te is positive semidefinite, then both A 1 + tē1 and te 6 are positive semidefinite. However, in general, the intersection of their positive semidefinite intervals are not that of G Example 4.1. Let A = and E =. Then, the positive semidefinite interval for G is T = {0}. But those of G 1 and t te 6 are both 1 0 (, + ) The case R(E) R(A). We now restrict to the case R(E) R(A), or what is the same, by the symmetry of A, N(A) N(E). Lemma 4.2. R(E) R(A) if and only if there exists X (necessarily unique) with R(X) R(A) and AX = E. X1 0 With the notation of equation (4), this X is given by Q Q, where X 1 is A 1 1 Ē 1. Proof. First, R(E) R(A) if and only if for some X, AX = E. (5) We need to show X can be chosen so that also R(X) R(A). Using the diagonalization (4), we can write (5) as A1 0 X1 X 4 Ē1 Ē = 2 X 3 X 2 Ē2, E 6 where Q X1 X XQ = 4. X 3 X 2 A 1 1 Thus, Ē 2 = 0 and E 6 = 0 and for such X, X 1 = A 1 1 Ē 2 = 0; hence, A1 0 X1 0 = X 3 X 2 ] [Ē1 0. Ē 1, and X 4 =

8 8 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR Changing X 3 and X 2 arbitrarily will not destroy (5). By an argument similar to that for E, we see that R(X) R(A) if and only if also X 3 =0 and [ X 4 = ] 0. Thus, the unique X with AX = E and R(X) R(A) is X1 0 Q Q, where X 1 is A 1 1 Ē 1. It follows from Lemma 4.2 that, for the case R(E) R(A), ] [Ē1 Q 0 EQ =. Hence, Q (A + te)q = A1 + tē1 0 ; that is, Q G1 (t) 0 G(t)Q =. Since G(t) 0 if and only if G 1 (t) 0, we have: Theorem 4.3. If R(E) R(A), the positive semidefinite interval for G is that of G 1. Since A 1 is nonsingular, we can use Theorem 3.1, Theorem 3.3, or Theorem 3.4 to determine its interval. The determining polynomial becomes g 1 (t) = det(g 1 (t)). X1 0 Note, however, that the nonzero eigenvalues of X = Q Q, are the same as those of X 1 = A 1 1 Ē 1. The largest eigenvalue of X is positive iff that of X 1 is, etc. Hence, for example, Theorem 3.1 extends to the case A positive semidefinite. Corollary 4.4. Let A be positive semidefinite, R(E) R(A) and let X be the unique solution of AX = E for which R(X) R(E). If δ 1 is the smallest eigenvalue of X and δ n the largest then, T =[t, t], where { 1 t = δ n, if δ n > 0, otherwise { t 1 = δ 1, if δ 1 < 0 +, otherwise. The corresponding result for A negative semidefinite is a little more awkward to state because of the possibility of zero eigenvalues. If the rank of E is smaller than that of A, then T is empty. Otherwise, we can use the non-zero eigenvalues of X to determine the interval T Theorem 3.3.

9 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 9 5. Implementation and Examples Our results indicate that the calculation of T can be accomplished with the orthogonal diagonalization of A and the calculation of the eigenvalues of X = A 1 E. In practice, the diagonalization is often not needed. When A is positive semidefinite, we first determine whether or not R(E) R(A). This can be done by checking the consistency of AX = E. One approach is to first determine a decomposition of A using Choleski factorization with symmetric pivoting ([DMBS79]). If r(a) = r, there exists a nonunique permutation matrix P and a triangular matrix R (unique for a given P ) such that P AP = R R, where R11 R R = 12, and where R 11 is a nonsingular upper triangular r r matrix and R 12 is an r (n r) matrix. For simplicity, we assume that P is the identity matrix. Now, AX = E is equivalent to R RX = E. Set Y = RX. We first solve R Y = E for the matrix Y. Set Y11 Y Y = 12 E11 E and E = 12 Y 21 Y 22 E12. E 22 Then we can write R Y = E as R 11 0 Y11 Y 12 E11 E R12 = 12 0 Y 21 Y 22 E12. E 22 Since R 11 is nonsingular, Y 11 and Y 12 are uniquely determined by R 11Y 11 = E 11 and R 11Y 12 = E 12. The equation AX = E is consistent only if Y 11 and Y 12 satisfy R 12Y 11 = E 12 and R 12Y 12 = E 22. The equation Y = RX can be written as [ R11 R 12 ] X11 X 12 = X 21 X 22 Y11 Y 12. Y 21 Y 22 This is consistent if and only if Y 21 = 0 and Y 22 = 0 and then all solutions of AX = E are determined by R 11 X 11 = Y 11 R 12 X 12 and R 11 X 21 = Y 12 R 12 X 22. (6) In section 4, we showed that X 21 and X 22 can be chosen so that the solution X satisfies R(X) R(A). To do this, we can again use the Choleski factorization, obtaining X 12 = R12(R 11 + R11 1 R 12R12)R 11 2 E 11 X 22 = R 12(R 11 + R 1 11 R 12R 12)R 2 11 E 12 (7)

10 10 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR Example 5.1. Let A = and E = The Choleski decomposition of A yields R11 R R = 12 = We solve the equation R11 Y 11 = E 11 and R11 Y 12 = E 12 obtaining 1/2 0 1/2 Y 11 = and Y 0 1/3 12 =. 1/3 Observe that R12 Y 11 = E12 and R 12 Y 12 = E 22 ; hence, have R(E) R(A). Set Y 21 = 0 and Y 22 = 0 for consistency. Solving for the X with R(X) R(A), as in (6) and (7), we obtain 1/6 1/27 11/54 X = 1/12 2/27 17/108, 1/12 1/27 5/108 with eigenvalues 1/4, 1/9, 0 and the positive semidefinite interval is [ 4, 9], obtained from the negative inverses of those which are non-zero. Example 5.2. Consider now the matrices A = and E = , in which A is indefinite. In the notation of section 3.3, the matrix pencils D 1 + te 1 = + t and D te 2 =( 1) + t(2) have positive semidefinite intervals T 1 =[ 1, 1] and T 2 =[1/2, + ), yielding T 1 T 2 =[1/2, 1]. The eigenvalues δ i of A 1 E are δ 1 = 1, δ 2 = , δ 3 = , so the corresponding roots t i = 1/δ i of the determining polynomial g are t 1 =1,t 2 = and t 3 = Thus, T 0 = {1, } and the positive semidefinite interval for A + te is T =[( 1+ 5)/2, 1].

11 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 11 References [BBRK + 87] H. C. P. Berbee, C. G. E. Boender, A. H. G. Rinnooy Kan, C. L. Scheffer, R. L. Smith, and J. Telgen, Hit-and-run algorithms for the identification of nonredundant linear inequalities, Math. Programming 37 (1987), no. 2, [BC86] M. J. Best and R. J. Caron, A parameterized Hessian quadratic programming problem, Ann. Oper. Res. 5 (1986), no. 1-4, [CG86] R. J. Caron and N. I. M. Gould, Finding a positive semidefinite interval for a parametric matrix, Linear Algebra Appl. 76 (1986), [DMBS79] J. J. Dongarra, C. B. Moler, J. R. Bunch, and G. W. Stewart, LINPACK users guide, Society for Industrial and Applied Mathematics, Philadelphia, [Rit67] Klaus Ritter, A method for solving nonlinear maximum-problems depending on parameters, Naval Res. Logist. Quart. 14 (1967), [Väl85] Hannu Väliaho, A unified approach to one-parametric general quadratic programming, Math. Programming 33 (1985), no. 3, [Väl88] H. Väliaho, Determining the inertia of matrix pencil as a function of the parameter, Linear Algebra Appl. 106 (1988), [VB96] L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM Review 38 (1996), [Wil65] J. H. Wilkinson, The algebraic eigenvalue problem, Clarendon Press, Oxford, MR 32 #1894 [WSV00] Henry Wolkowicz, Romesh Saigal, and Lieven Vandenberghe (eds.), Handbook of semidefinite programming, International Series in Operations Research & Management Science, 27, Kluwer Academic Publishers, Boston, MA, Department of Mathematics and Statistics, University of Windsor, Windsor, Ontario, N9B 3P4, Canada.

### Chap 3. Linear Algebra

Chap 3. Linear Algebra Outlines 1. Introduction 2. Basis, Representation, and Orthonormalization 3. Linear Algebraic Equations 4. Similarity Transformation 5. Diagonal Form and Jordan Form 6. Functions

### Chapter 3 Transformations

Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases

### Lecture Note 5: Semidefinite Programming for Stability Analysis

ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State

### Absolute value equations

Linear Algebra and its Applications 419 (2006) 359 367 www.elsevier.com/locate/laa Absolute value equations O.L. Mangasarian, R.R. Meyer Computer Sciences Department, University of Wisconsin, 1210 West

### There are six more problems on the next two pages

Math 435 bg & bu: Topics in linear algebra Summer 25 Final exam Wed., 8/3/5. Justify all your work to receive full credit. Name:. Let A 3 2 5 Find a permutation matrix P, a lower triangular matrix L with

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### Symmetric Matrices and Eigendecomposition

Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions

### An Algorithm for Solving the Convex Feasibility Problem With Linear Matrix Inequality Constraints and an Implementation for Second-Order Cones

An Algorithm for Solving the Convex Feasibility Problem With Linear Matrix Inequality Constraints and an Implementation for Second-Order Cones Bryan Karlovitz July 19, 2012 West Chester University of Pennsylvania

### APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.

APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product

### MATH 220 FINAL EXAMINATION December 13, Name ID # Section #

MATH 22 FINAL EXAMINATION December 3, 2 Name ID # Section # There are??multiple choice questions. Each problem is worth 5 points. Four possible answers are given for each problem, only one of which is

### Cheat Sheet for MATH461

Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A

### MA 265 FINAL EXAM Fall 2012

MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators

### Semidefinite Programming

Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has

### Mathematical Optimisation, Chpt 2: Linear Equations and inequalities

Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl http://dickinson.website version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson

### Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

### 7. Symmetric Matrices and Quadratic Forms

Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value

### Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

### Matrix decompositions

Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The

### Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

### This property turns out to be a general property of eigenvectors of a symmetric A that correspond to distinct eigenvalues as we shall see later.

34 To obtain an eigenvector x 2 0 2 for l 2 = 0, define: B 2 A - l 2 I 2 = È 1, 1, 1 Î 1-0 È 1, 0, 0 Î 1 = È 1, 1, 1 Î 1. To transform B 2 into an upper triangular matrix, subtract the first row of B 2

### MATH 5720: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 2018

MATH 57: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 18 1 Global and Local Optima Let a function f : S R be defined on a set S R n Definition 1 (minimizers and maximizers) (i) x S

### May 9, 2014 MATH 408 MIDTERM EXAM OUTLINE. Sample Questions

May 9, 24 MATH 48 MIDTERM EXAM OUTLINE This exam will consist of two parts and each part will have multipart questions. Each of the 6 questions is worth 5 points for a total of points. The two part of

### MATH Dr. Pedro V squez UPRM. P. V squez (UPRM) Conferencia 1/ 17

Dr. Pedro V squez UPRM P. V squez (UPRM) Conferencia 1/ 17 Quadratic programming MATH 6026 Equality constraints A general formulation of these problems is: min x 2R nq (x) = 1 2 x T Qx + x T c (1) subjec

### Conceptual Questions for Review

Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.

### MATH Dr. Pedro Vásquez UPRM. P. Vásquez (UPRM) Conferencia 1 / 17

MATH 6026 Dr. Pedro Vásquez UPRM P. Vásquez (UPRM) Conferencia 1 / 17 Quadratic programming uemath 6026 Equality constraints A general formulation of these problems is: min x 2R nq (x) = 1 2 x T Qx + x

### Real Symmetric Matrices and Semidefinite Programming

Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many

### EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

### University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm

University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm Name: The proctor will let you read the following conditions

### Largest dual ellipsoids inscribed in dual cones

Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

### The extreme rays of the 5 5 copositive cone

The extreme rays of the copositive cone Roland Hildebrand March 8, 0 Abstract We give an explicit characterization of all extreme rays of the cone C of copositive matrices. The results are based on the

### Linear Programming: Simplex

Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016

### Matrix Multiplication Chapter IV Special Linear Systems

Matrix Multiplication Chapter IV Special Linear Systems By Gokturk Poyrazoglu The State University of New York at Buffalo BEST Group Winter Lecture Series Outline 1. Diagonal Dominance and Symmetry a.

### Linear Algebra Practice Problems

Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,

### Rank and inertia optimizations of two Hermitian quadratic matrix functions subject to restrictions with applications

Rank and inertia optimizations of two Hermitian quadratic matrix functions subject to restrictions with applications Yongge Tian a, Ying Li b,c a China Economics and Management Academy, Central University

### Lecture 2 - Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then

Lecture 2 - Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then 1. x S is a global minimum point of f over S if f (x) f (x ) for any x S. 2. x S

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

### Chapter 2: Unconstrained Extrema

Chapter 2: Unconstrained Extrema Math 368 c Copyright 2012, 2013 R Clark Robinson May 22, 2013 Chapter 2: Unconstrained Extrema 1 Types of Sets Definition For p R n and r > 0, the open ball about p of

### Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show

MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,

### Linear Algebra And Its Applications Chapter 6. Positive Definite Matrix

Linear Algebra And Its Applications Chapter 6. Positive Definite Matrix KAIST wit Lab 2012. 07. 10 남성호 Introduction The signs of the eigenvalues can be important. The signs can also be related to the minima,

### Solvability of Linear Matrix Equations in a Symmetric Matrix Variable

Solvability of Linear Matrix Equations in a Symmetric Matrix Variable Maurcio C. de Oliveira J. William Helton Abstract We study the solvability of generalized linear matrix equations of the Lyapunov type

### Symmetric matrices and dot products

Symmetric matrices and dot products Proposition An n n matrix A is symmetric iff, for all x, y in R n, (Ax) y = x (Ay). Proof. If A is symmetric, then (Ax) y = x T A T y = x T Ay = x (Ay). If equality

### MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.

MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:

### Math 302 Outcome Statements Winter 2013

Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the Three-Dimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a

### Example Linear Algebra Competency Test

Example Linear Algebra Competency Test The 4 questions below are a combination of True or False, multiple choice, fill in the blank, and computations involving matrices and vectors. In the latter case,

### Math 408 Advanced Linear Algebra

Math 408 Advanced Linear Algebra Chi-Kwong Li Chapter 4 Hermitian and symmetric matrices Basic properties Theorem Let A M n. The following are equivalent. Remark (a) A is Hermitian, i.e., A = A. (b) x

### Math Matrix Algebra

Math 44 - Matrix Algebra Review notes - (Alberto Bressan, Spring 7) sec: Orthogonal diagonalization of symmetric matrices When we seek to diagonalize a general n n matrix A, two difficulties may arise:

### Remark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.

Sec 6 Eigenvalues and Eigenvectors Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called an eigenvalue of A if there is a nontrivial

Characterization of half-radial matrices Iveta Hnětynková, Petr Tichý Faculty of Mathematics and Physics, Charles University, Sokolovská 83, Prague 8, Czech Republic Abstract Numerical radius r(a) is the

### AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 3: Positive-Definite Systems; Cholesky Factorization Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 11 Symmetric

### A Brief Outline of Math 355

A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Sparsity-Preserving Difference of Positive Semidefinite Matrix Representation of Indefinite Matrices

Sparsity-Preserving Difference of Positive Semidefinite Matrix Representation of Indefinite Matrices Jaehyun Park June 1 2016 Abstract We consider the problem of writing an arbitrary symmetric matrix as

### In English, this means that if we travel on a straight line between any two points in C, then we never leave C.

Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from

### Positive semidefinite intervals for matrix pencils.

University of Windsor Scholarship at UWindsor Electronic Theses and Dissertations 1-1-2004 Positive semidefinite intervals for matrix pencils. Huiming Song University of Windsor Follow this and additional

### Here each term has degree 2 (the sum of exponents is 2 for all summands). A quadratic form of three variables looks as

Reading [SB], Ch. 16.1-16.3, p. 375-393 1 Quadratic Forms A quadratic function f : R R has the form f(x) = a x. Generalization of this notion to two variables is the quadratic form Q(x 1, x ) = a 11 x

### Math Linear Algebra Final Exam Review Sheet

Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

### 22.4. Numerical Determination of Eigenvalues and Eigenvectors. Introduction. Prerequisites. Learning Outcomes

Numerical Determination of Eigenvalues and Eigenvectors 22.4 Introduction In Section 22. it was shown how to obtain eigenvalues and eigenvectors for low order matrices, 2 2 and. This involved firstly solving

### HW2 - Due 01/30. Each answer must be mathematically justified. Don t forget your name.

HW2 - Due 0/30 Each answer must be mathematically justified. Don t forget your name. Problem. Use the row reduction algorithm to find the inverse of the matrix 0 0, 2 3 5 if it exists. Double check your

### Part 4: Active-set methods for linearly constrained optimization. Nick Gould (RAL)

Part 4: Active-set methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where

### A Note on Eigenvalues of Perturbed Hermitian Matrices

A Note on Eigenvalues of Perturbed Hermitian Matrices Chi-Kwong Li Ren-Cang Li July 2004 Let ( H1 E A = E H 2 Abstract and Ã = ( H1 H 2 be Hermitian matrices with eigenvalues λ 1 λ k and λ 1 λ k, respectively.

ON LOCAL NON-GLOBAL MINIMIZERS OF QUADRATIC OPTIMIZATION PROBLEM WITH A SINGLE QUADRATIC CONSTRAINT A. TAATI AND M. SALAHI Abstract. In this paper, we consider the nonconvex quadratic optimization problem

### OPTIMAL SCALING FOR P -NORMS AND COMPONENTWISE DISTANCE TO SINGULARITY

published in IMA Journal of Numerical Analysis (IMAJNA), Vol. 23, 1-9, 23. OPTIMAL SCALING FOR P -NORMS AND COMPONENTWISE DISTANCE TO SINGULARITY SIEGFRIED M. RUMP Abstract. In this note we give lower

### CHARACTERIZATIONS. is pd/psd. Possible for all pd/psd matrices! Generating a pd/psd matrix: Choose any B Mn, then

LECTURE 6: POSITIVE DEFINITE MATRICES Definition: A Hermitian matrix A Mn is positive definite (pd) if x Ax > 0 x C n,x 0 A is positive semidefinite (psd) if x Ax 0. Definition: A Mn is negative (semi)definite

### 9. Numerical linear algebra background

Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization

### MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

### Identifying Redundant Linear Constraints in Systems of Linear Matrix. Inequality Constraints. Shafiu Jibrin

Identifying Redundant Linear Constraints in Systems of Linear Matrix Inequality Constraints Shafiu Jibrin (shafiu.jibrin@nau.edu) Department of Mathematics and Statistics Northern Arizona University, Flagstaff

### Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures

### 2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

### MTH 2032 SemesterII

MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents

### Recall the convention that, for us, all vectors are column vectors.

Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

### Matrix Mathematics. Theory, Facts, and Formulas with Application to Linear Systems Theory. Dennis S. Bernstein

Matrix Mathematics Theory, Facts, and Formulas with Application to Linear Systems Theory Dennis S. Bernstein PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD Contents Special Symbols xv Conventions, Notation,

### A sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm

Volume 31, N. 1, pp. 205 218, 2012 Copyright 2012 SBMAC ISSN 0101-8205 / ISSN 1807-0302 (Online) www.scielo.br/cam A sensitivity result for quadratic semidefinite programs with an application to a sequential

### Research Note. A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization

Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 88-107 Research Note A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization B. Kheirfam We

### UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems

UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction

### arxiv: v1 [math.na] 1 Sep 2018

On the perturbation of an L -orthogonal projection Xuefeng Xu arxiv:18090000v1 [mathna] 1 Sep 018 September 5 018 Abstract The L -orthogonal projection is an important mathematical tool in scientific computing

### Lecture notes: Applied linear algebra Part 1. Version 2

Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and

### Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

### Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

### YORK UNIVERSITY. Faculty of Science Department of Mathematics and Statistics MATH M Test #1. July 11, 2013 Solutions

YORK UNIVERSITY Faculty of Science Department of Mathematics and Statistics MATH 222 3. M Test # July, 23 Solutions. For each statement indicate whether it is always TRUE or sometimes FALSE. Note: For

### Numerical Optimization

Numerical Optimization Unit 2: Multivariable optimization problems Che-Rung Lee Scribe: February 28, 2011 (UNIT 2) Numerical Optimization February 28, 2011 1 / 17 Partial derivative of a two variable function

### Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

### On the Solution of Constrained and Weighted Linear Least Squares Problems

International Mathematical Forum, 1, 2006, no. 22, 1067-1076 On the Solution of Constrained and Weighted Linear Least Squares Problems Mohammedi R. Abdel-Aziz 1 Department of Mathematics and Computer Science

### Math 308 Practice Final Exam Page and vector y =

Math 308 Practice Final Exam Page Problem : Solving a linear equation 2 0 2 5 Given matrix A = 3 7 0 0 and vector y = 8. 4 0 0 9 (a) Solve Ax = y (if the equation is consistent) and write the general solution

### 22.3. Repeated Eigenvalues and Symmetric Matrices. Introduction. Prerequisites. Learning Outcomes

Repeated Eigenvalues and Symmetric Matrices. Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one

### 1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?

. Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in

### Chapter 1. Matrix Algebra

ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface

### Principal Component Analysis (PCA) Our starting point consists of T observations from N variables, which will be arranged in an T N matrix R,

Principal Component Analysis (PCA) PCA is a widely used statistical tool for dimension reduction. The objective of PCA is to find common factors, the so called principal components, in form of linear combinations

### Fall 2016 MATH*1160 Final Exam

Fall 2016 MATH*1160 Final Exam Last name: (PRINT) First name: Student #: Instructor: M. R. Garvie Dec 16, 2016 INSTRUCTIONS: 1. The exam is 2 hours long. Do NOT start until instructed. You may use blank

### Review Questions REVIEW QUESTIONS 71

REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of

### VISCOSITY SOLUTIONS. We follow Han and Lin, Elliptic Partial Differential Equations, 5.

VISCOSITY SOLUTIONS PETER HINTZ We follow Han and Lin, Elliptic Partial Differential Equations, 5. 1. Motivation Throughout, we will assume that Ω R n is a bounded and connected domain and that a ij C(Ω)

### Remark 1 By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.

Sec 5 Eigenvectors and Eigenvalues In this chapter, vector means column vector Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called

### Direct methods for symmetric eigenvalue problems

Direct methods for symmetric eigenvalue problems, PhD McMaster University School of Computational Engineering and Science February 4, 2008 1 Theoretical background Posing the question Perturbation theory

### Polynomiality of Linear Programming

Chapter 10 Polynomiality of Linear Programming In the previous section, we presented the Simplex Method. This method turns out to be very efficient for solving linear programmes in practice. While it is

### 1 Positive definiteness and semidefiniteness

Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +

### Lecture 6 Positive Definite Matrices

Linear Algebra Lecture 6 Positive Definite Matrices Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Spring 2017 2017/6/8 Lecture 6: Positive Definite Matrices