POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS


 Sharon Holt
 1 years ago
 Views:
Transcription
1 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR Abstract. Let A and E be real symmetric matrices. In this paper we are concerned with the determination of the values of t for which A + te is positive semidefinite. That is, we want to find the positive semidefinite interval for the matrix pencil t A + te. We first consider the cases when A is positive definite and when A is negative definite. In both of these cases, the positive semidefinite interval is determined from the eigenvalues of the matrix A 1 E. The results are extended to the case when A is semidefinite with range containing that of E and are combined to handle the indefinite case. 1. Introduction Let A and E be real symmetric n n matrices. The (linear) matrix pencil A + te is the function G : t A + te (t R). We consider the problem of determining the set T of those t for which G(t) is positive semidefinite. Since the set of positive semidefinite matrices is a closed convex cone in the space of n n matrices, T is a closed (possibly unbounded) interval. We call T the positive semidefinite interval. One source of interest in this problem is its connection to mathematical programming. Consider first the parametric quadratic program ([BC86], [Rit67], [Väl85]) min{c(t) x x C(t)x a i (t) x b i (t),i=1,,m}. If C(t) =A+tE, then the quadratic program is convex if and only if A+tE is positive semidefinite. The positive semidefinite interval gives the values of t for which the KarushKuhnTucker points are global minimizers. Consider also a semidefinite program ([WSV00], [VB96]) where the contraints require that the matrices n F j (x) :=A j 0 + x i A j i 0, j =1,,q i=1 Date: 21Feb05 4:16p. This research was supported by the National Sciences and Engineering Research Council. 1
2 2 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR be positive semidefinite. An application of a hitandrun algorithm [BBRK + 87] for finding necessary constraints requires, at each iteration, the determination of T for each of the matrix pencils F j (ˆx + tv) =F j (ˆx)+t n i=1 v i A j i, j =1,,q where ˆx is the current iterate and v is a unit vector. If A is positive semidefinite and E is of rank one or two, then explicit expressions for the endpoints of T, and a method of computing them, are given in [CG86]. Valiaho ([Väl88]) extended these results, presenting a method of determining the inertia (the ordered triple of the numbers of positive, negative and zero eigenvalues) of A + te as a function of the parameter t. In this paper, motivated by applications to determination of constraint boundaries in semidefinite programs, we are only concerned with the determination of the endpoints of T. Section 2 gives some preliminary material. Section 3 discusses the case when A is nonsingular. Explicit expressions of the endpoints of the positive semidefinite interval are given when A is positive definite and when A is negative definite. In these cases, the endpoints are determined from the eigenvalues of A 1 E. Then we can combine these results to obtain the positive semidefinite interval when A is nonsingular and indefinite. Section 4 discusses the case when A is singular. Section 5 presents remarks on implementation and includes examples. 2. Preliminaries Throughout this paper, we shall denote the range space of a matrix M by R(M), its null space by N(M), and its rank by r(m). A real symmetric n n matrix M is positive definite (M 0), if x Mx > 0, for all x R n,x 0;M is positive semidefinite (M 0) if x Mx 0, for all x R n. It is well known that M is positive definite if and only if all eigenvalues of M are positive and M is positive semidefinite if and only if all eigenvalues of M are nonnegative. Since x Mx is continuous in x and affine in M, the set of positive definite matrices is an open convex set and the set of positive semidefinite matrices is a closed convex cone. It follows that the set T = {t R : G(t) 0} is an open interval, the positive definite interval (of G) and T = {t R : G(t) 0} =[t, t] is a closed interval, 1 the positive semidefinite interval. The only time T is the whole real line is if E = 0. Indeed, if x is an eigenvector of E with a nonzero eigenvalue, x x G(t)x is a straight line with a nonzero slope x Ex, so must be somewhere negative. Thus, except in the case E = 0, one of t and t is finite. If T is not empty, then its closure is T. Indeed, if (say) t 0 T and t is finite, then for each nonzero x in R n, x G(t 0 )x>0 and x G( t)x 0, so x G(t)x >0 for all t (t 0, t). 1 Here, and in the sequel, we will use [a, b] for {t R : a t b}. For example, if a = and b R, then [a, b] =(,b].
3 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 3 Given two real symmetric n n matrices A and E with A nonsingular, if det(a + te) =0, (1) then det( 1 t I A 1 E)=0; so 1 t is an eigenvalue of A 1 E. The solutions to equation (1), that is, the roots of the determining polynomial g(t) = det G(t), will determine the intervals T and T, as long as g is not identically 0. This is the case, for example, if one of A and E is nonsingular. If g has no root, then T is either empty or the whole real line; if it has at least one root, then T must be bounded above or below or both. Let T 0 be the set of those roots t of the determining polynomial g for which G(t) 0. Clearly, T 0 consists of at most 2 points, the boundary points of T. Theorem 2.1. Suppose g 0. Then, the positive semidefinite interval T are determined as follows. (1) If T 0 =, then T =. (2) If T 0 consists of 2 points, then T is the closed interval they determine. (3) If T 0 consists of a single point t 0 and T is bounded, then T = T 0.In particular, this holds if t 0 is between 2 other roots of g, or any other 2 points a and b where G(a) and G(b) are not positive definite. (4) If T 0 consists of a single point t 0 and for some real a>t 0, and G(a) 0, then T =[t 0, + ). (5) If T 0 consists of a single point t 0 and for some real a<t 0, G(a) 0, then T =(,t 0 ]. If T is unbounded, then one of (4) or (5) holds (or E =0). Proof. The proof is straightforward. Consider, for example, (4). Then T is nonempty and T is its closure. Since there are no other possible endpoints, T =[t 0, + ). If E is positive definite, then A + te is positive definite for sufficiently large t. Lemma 2.2. Let E be positive definite. If max x =1 x ( A)x t> min x =1 x Ex, then A + te 0. Hence, T =[t, + ), where t is the largest root of g. Proof. Since E 0, for every x R n with x =1,x Ex > 0. Thus, for t> max x =1 x ( A)x min x =1 x Ex,
4 4 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR we have t min x =1 x Ex > max x =1 x ( A)x. Hence, for each x of norm 1, tx Ex > x ( A)x; that is, x (A + te)x >0; hence, A + te 0. Since T is unbounded on the right, it is nonempty of the form (t, + ) and T =[t, + ), its closure. Since T can contain no roots of g, t is the largest one. By writing A + te = A +( t)( E), we obtain the corresponding result for the case that E is negative definite. Lemma 2.3. Let E be negative definite. Then T =(, t ] and t is the smallest root of g. The eigenvalues λ i (t) ofg(t), (listed in descending order) are continuous functions of t. This follows, for example, from the WielandtHoffman Theorem ([Wil65], page 104). From this it is clear that R is divided into intervals on which the inertia is constant. However, with our approach, we will have no need of this result. 3. The Positive Semidefinite Interval when A is Nonsingular In case A is nonsingular, the determining polynomial is not identically zero, so has finitely many roots, each negative inverses of eigenvalues of A 1 E. We present the results in three subsections, according to whether A is positive definite, negative definite, or indefinite A is positive definite. When A is positive definite, 0 T and T =[t, t] is its closure. Thus, t is either the smallest positive zero of g or is + ; t is either the largest negative root of g or is. Translating this into a statement about eigenvalues of A 1 E, we obtain: Theorem 3.1. Suppose that A is positive definite. Let δ n be the largest eigenvalue of A 1 E and δ 1 the smallest. The positive semidefinite interval for A + te is T =[t, t] where { 1 t = δ n, if δ n > 0, otherwise { t 1 = δ 1, if δ 1 < 0 +, otherwise. Special cases: If E is positive semidefinite, it is clear that T is not bounded above, that is, t =+. If E 0, we know therefore that T = [ 1/δ n, + ). If E is negative definite, then T =(, 1/δ 1 ], by Lemma 2.3.
5 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS A is negative definite. From Lemmas 2.2 and 2.3 we know the form of T when E is positive definite or negative definite. If E is neither and A is negative definite, we find T is empty. Lemma 3.2. Suppose that A is negative definite. If E is singular or indefinite, then T =. Proof. If E is singular, there exists y R n, y 0, such that It follows that for every t, y Ey =0. y (A + te)y <0, so that A + te is not positive semidefinite. If E is indefinite, then there exist u, v R n, such that u Eu > 0, v Ev < 0. Then, for t 0, u (A + te)u <0; and for t>0, v (A + te)v <0, so again there is no t for which G(t) is positive definite: T =. In terms of the eigenvalues of A 1 E, we obtain the following complete description of the T in the case A is negative definite. Theorem 3.3. Suppose A is negative definite. Let δ n be the largest eigenvalue of A 1 E and δ 1 the smallest. Then the positive semidefinite interval T is given as follows. (1) T =[ 1 δ n, + ), ifδ n < 0, (2) T =(, 1 δ 1 ],ifδ 1 > 0, and (3) T =, otherwise. Proof. There are three mutually exclusive exhaustive subcases. If E 0, then by Lemma 2.2, T =[t, + ), where t is the largest root of g. Since A = G(0), is negative definite, G(t) is negative definite for all t 0; hence all the roots of g are positive, or equivalently, all the eigenvalues of A 1 E are negative. The largest root of g corresponds to the largest eigenvalue of A 1 E, so we obtain (1). Similarly, E 0 (negative definite) corresponds to all the roots of g positive that is, all the eigenvalues of A 1 E negative and, using Lemma 2.3, we obtain (2). In the remaining case, where E is neither positive definite, nor negative definite, T is empty, by Lemma 3.2
6 6 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR 3.3. A is nonsingular and indefinite. We now combine the results in Section 3.1 and Section 3.2 to obtain information about the positive semidefinite interval when A is nonsingular and indefinite. Let Q be an orthogonal n n matrix such that Q D1 0 AQ =, (2) 0 D 2 where D 1,D 2 are diagonal and positive definite. Let Q E1 E EQ = 3 E3, E 2 where E 1 and E 2 have the same sizes as D 1 and D 2. Then Q D1 + te (A + te)q = 1 te 3 te3. D 2 + te 2 By Theorem 3.1 and Theorem 3.3, we can determine the positive semidefinite intervals for D 1 + te 1 and D 2 + te 2. We denote them by T 1 and T 2 and obtain: Theorem 3.4. T T 1 T 2 and if E 3 =0, T = T 1 T 2. If E 3 is not a zero matrix, then we may need to use brute force, relying on Theorem 2.1. Theorem 3.4 helps, since we know the endpoints (elements of T 0 ) must be in T 1 T 2. For example, we could first determine T 1 and T 2, then search inside T 1 T 2, for roots of the determining polynomial, stopping when 2 such are found or when one such is between points where G(t) is not positive definite. 4. The Positive Semidefinite Interval when A is Singular In this section, we are concerned with the positive semidefinite interval for the matrix pencil A + te when A is singular. Choose an orthogonal matrix Q for which Q AQ = D 1 0 D 2 0, (3) 0 where D 1,D 2 are diagonal and positive definite, extending the notation of the diagonalization (2). The corresponding decomposition of Q EQ has an extra column and row of blocks. E 1 E 3 E 4 Q EQ = E3 E 2 E 5. E4 E5 E 6 Let us write these as Q A1 0 AQ = and Q Ē1 Ē EQ = 2 Ē2 (4) E 6
7 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 7 Then Q A1 + tē1 tē2 (A + te)q =. tē 2 te 6 D1 0 Since A 1 = is nonsingular, the positive semidefinite interval 0 D 2 for the matrix pencil G 1 : t A 1 + tē1 can be found using Theorem 3.1, Theorem 3.3, or Theorem 3.4. Let T 3 denote the positive semidefinite interval for the matrix pencil te 6. Then, (1) E 6 =0 = T 3 = R. (2) E 6 0,E 6 0 = T 3 =[0, + ). (3) E 6 0,E 6 0 = T 3 =(, 0]. (4) E 6 is indefinite = T 3 = {0}. Note that if G(t) =A + te is positive semidefinite, then both A 1 + tē1 and te 6 are positive semidefinite. However, in general, the intersection of their positive semidefinite intervals are not that of G Example 4.1. Let A = and E =. Then, the positive semidefinite interval for G is T = {0}. But those of G 1 and t te 6 are both 1 0 (, + ) The case R(E) R(A). We now restrict to the case R(E) R(A), or what is the same, by the symmetry of A, N(A) N(E). Lemma 4.2. R(E) R(A) if and only if there exists X (necessarily unique) with R(X) R(A) and AX = E. X1 0 With the notation of equation (4), this X is given by Q Q, where X 1 is A 1 1 Ē 1. Proof. First, R(E) R(A) if and only if for some X, AX = E. (5) We need to show X can be chosen so that also R(X) R(A). Using the diagonalization (4), we can write (5) as A1 0 X1 X 4 Ē1 Ē = 2 X 3 X 2 Ē2, E 6 where Q X1 X XQ = 4. X 3 X 2 A 1 1 Thus, Ē 2 = 0 and E 6 = 0 and for such X, X 1 = A 1 1 Ē 2 = 0; hence, A1 0 X1 0 = X 3 X 2 ] [Ē1 0. Ē 1, and X 4 =
8 8 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR Changing X 3 and X 2 arbitrarily will not destroy (5). By an argument similar to that for E, we see that R(X) R(A) if and only if also X 3 =0 and [ X 4 = ] 0. Thus, the unique X with AX = E and R(X) R(A) is X1 0 Q Q, where X 1 is A 1 1 Ē 1. It follows from Lemma 4.2 that, for the case R(E) R(A), ] [Ē1 Q 0 EQ =. Hence, Q (A + te)q = A1 + tē1 0 ; that is, Q G1 (t) 0 G(t)Q =. Since G(t) 0 if and only if G 1 (t) 0, we have: Theorem 4.3. If R(E) R(A), the positive semidefinite interval for G is that of G 1. Since A 1 is nonsingular, we can use Theorem 3.1, Theorem 3.3, or Theorem 3.4 to determine its interval. The determining polynomial becomes g 1 (t) = det(g 1 (t)). X1 0 Note, however, that the nonzero eigenvalues of X = Q Q, are the same as those of X 1 = A 1 1 Ē 1. The largest eigenvalue of X is positive iff that of X 1 is, etc. Hence, for example, Theorem 3.1 extends to the case A positive semidefinite. Corollary 4.4. Let A be positive semidefinite, R(E) R(A) and let X be the unique solution of AX = E for which R(X) R(E). If δ 1 is the smallest eigenvalue of X and δ n the largest then, T =[t, t], where { 1 t = δ n, if δ n > 0, otherwise { t 1 = δ 1, if δ 1 < 0 +, otherwise. The corresponding result for A negative semidefinite is a little more awkward to state because of the possibility of zero eigenvalues. If the rank of E is smaller than that of A, then T is empty. Otherwise, we can use the nonzero eigenvalues of X to determine the interval T Theorem 3.3.
9 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 9 5. Implementation and Examples Our results indicate that the calculation of T can be accomplished with the orthogonal diagonalization of A and the calculation of the eigenvalues of X = A 1 E. In practice, the diagonalization is often not needed. When A is positive semidefinite, we first determine whether or not R(E) R(A). This can be done by checking the consistency of AX = E. One approach is to first determine a decomposition of A using Choleski factorization with symmetric pivoting ([DMBS79]). If r(a) = r, there exists a nonunique permutation matrix P and a triangular matrix R (unique for a given P ) such that P AP = R R, where R11 R R = 12, and where R 11 is a nonsingular upper triangular r r matrix and R 12 is an r (n r) matrix. For simplicity, we assume that P is the identity matrix. Now, AX = E is equivalent to R RX = E. Set Y = RX. We first solve R Y = E for the matrix Y. Set Y11 Y Y = 12 E11 E and E = 12 Y 21 Y 22 E12. E 22 Then we can write R Y = E as R 11 0 Y11 Y 12 E11 E R12 = 12 0 Y 21 Y 22 E12. E 22 Since R 11 is nonsingular, Y 11 and Y 12 are uniquely determined by R 11Y 11 = E 11 and R 11Y 12 = E 12. The equation AX = E is consistent only if Y 11 and Y 12 satisfy R 12Y 11 = E 12 and R 12Y 12 = E 22. The equation Y = RX can be written as [ R11 R 12 ] X11 X 12 = X 21 X 22 Y11 Y 12. Y 21 Y 22 This is consistent if and only if Y 21 = 0 and Y 22 = 0 and then all solutions of AX = E are determined by R 11 X 11 = Y 11 R 12 X 12 and R 11 X 21 = Y 12 R 12 X 22. (6) In section 4, we showed that X 21 and X 22 can be chosen so that the solution X satisfies R(X) R(A). To do this, we can again use the Choleski factorization, obtaining X 12 = R12(R 11 + R11 1 R 12R12)R 11 2 E 11 X 22 = R 12(R 11 + R 1 11 R 12R 12)R 2 11 E 12 (7)
10 10 RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR Example 5.1. Let A = and E = The Choleski decomposition of A yields R11 R R = 12 = We solve the equation R11 Y 11 = E 11 and R11 Y 12 = E 12 obtaining 1/2 0 1/2 Y 11 = and Y 0 1/3 12 =. 1/3 Observe that R12 Y 11 = E12 and R 12 Y 12 = E 22 ; hence, have R(E) R(A). Set Y 21 = 0 and Y 22 = 0 for consistency. Solving for the X with R(X) R(A), as in (6) and (7), we obtain 1/6 1/27 11/54 X = 1/12 2/27 17/108, 1/12 1/27 5/108 with eigenvalues 1/4, 1/9, 0 and the positive semidefinite interval is [ 4, 9], obtained from the negative inverses of those which are nonzero. Example 5.2. Consider now the matrices A = and E = , in which A is indefinite. In the notation of section 3.3, the matrix pencils D 1 + te 1 = + t and D te 2 =( 1) + t(2) have positive semidefinite intervals T 1 =[ 1, 1] and T 2 =[1/2, + ), yielding T 1 T 2 =[1/2, 1]. The eigenvalues δ i of A 1 E are δ 1 = 1, δ 2 = , δ 3 = , so the corresponding roots t i = 1/δ i of the determining polynomial g are t 1 =1,t 2 = and t 3 = Thus, T 0 = {1, } and the positive semidefinite interval for A + te is T =[( 1+ 5)/2, 1].
11 POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS 11 References [BBRK + 87] H. C. P. Berbee, C. G. E. Boender, A. H. G. Rinnooy Kan, C. L. Scheffer, R. L. Smith, and J. Telgen, Hitandrun algorithms for the identification of nonredundant linear inequalities, Math. Programming 37 (1987), no. 2, [BC86] M. J. Best and R. J. Caron, A parameterized Hessian quadratic programming problem, Ann. Oper. Res. 5 (1986), no. 14, [CG86] R. J. Caron and N. I. M. Gould, Finding a positive semidefinite interval for a parametric matrix, Linear Algebra Appl. 76 (1986), [DMBS79] J. J. Dongarra, C. B. Moler, J. R. Bunch, and G. W. Stewart, LINPACK users guide, Society for Industrial and Applied Mathematics, Philadelphia, [Rit67] Klaus Ritter, A method for solving nonlinear maximumproblems depending on parameters, Naval Res. Logist. Quart. 14 (1967), [Väl85] Hannu Väliaho, A unified approach to oneparametric general quadratic programming, Math. Programming 33 (1985), no. 3, [Väl88] H. Väliaho, Determining the inertia of matrix pencil as a function of the parameter, Linear Algebra Appl. 106 (1988), [VB96] L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM Review 38 (1996), [Wil65] J. H. Wilkinson, The algebraic eigenvalue problem, Clarendon Press, Oxford, MR 32 #1894 [WSV00] Henry Wolkowicz, Romesh Saigal, and Lieven Vandenberghe (eds.), Handbook of semidefinite programming, International Series in Operations Research & Management Science, 27, Kluwer Academic Publishers, Boston, MA, Department of Mathematics and Statistics, University of Windsor, Windsor, Ontario, N9B 3P4, Canada.
Chap 3. Linear Algebra
Chap 3. Linear Algebra Outlines 1. Introduction 2. Basis, Representation, and Orthonormalization 3. Linear Algebraic Equations 4. Similarity Transformation 5. Diagonal Form and Jordan Form 6. Functions
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 WeiTa Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2015 WeiTa Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationAbsolute value equations
Linear Algebra and its Applications 419 (2006) 359 367 www.elsevier.com/locate/laa Absolute value equations O.L. Mangasarian, R.R. Meyer Computer Sciences Department, University of Wisconsin, 1210 West
More informationThere are six more problems on the next two pages
Math 435 bg & bu: Topics in linear algebra Summer 25 Final exam Wed., 8/3/5. Justify all your work to receive full credit. Name:. Let A 3 2 5 Find a permutation matrix P, a lower triangular matrix L with
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationAn Algorithm for Solving the Convex Feasibility Problem With Linear Matrix Inequality Constraints and an Implementation for SecondOrder Cones
An Algorithm for Solving the Convex Feasibility Problem With Linear Matrix Inequality Constraints and an Implementation for SecondOrder Cones Bryan Karlovitz July 19, 2012 West Chester University of Pennsylvania
More informationAPPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the ndimensional column vector with components x 1 x 2.
APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the ndimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product
More informationMATH 220 FINAL EXAMINATION December 13, Name ID # Section #
MATH 22 FINAL EXAMINATION December 3, 2 Name ID # Section # There are??multiple choice questions. Each problem is worth 5 points. Four possible answers are given for each problem, only one of which is
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationSemidefinite Programming
Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has
More informationMathematical Optimisation, Chpt 2: Linear Equations and inequalities
Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl http://dickinson.website version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More information7. Symmetric Matrices and Quadratic Forms
Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationMatrix decompositions
Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationThis property turns out to be a general property of eigenvectors of a symmetric A that correspond to distinct eigenvalues as we shall see later.
34 To obtain an eigenvector x 2 0 2 for l 2 = 0, define: B 2 A  l 2 I 2 = È 1, 1, 1 Î 10 È 1, 0, 0 Î 1 = È 1, 1, 1 Î 1. To transform B 2 into an upper triangular matrix, subtract the first row of B 2
More informationMATH 5720: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 2018
MATH 57: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 18 1 Global and Local Optima Let a function f : S R be defined on a set S R n Definition 1 (minimizers and maximizers) (i) x S
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationMay 9, 2014 MATH 408 MIDTERM EXAM OUTLINE. Sample Questions
May 9, 24 MATH 48 MIDTERM EXAM OUTLINE This exam will consist of two parts and each part will have multipart questions. Each of the 6 questions is worth 5 points for a total of points. The two part of
More informationMATH Dr. Pedro V squez UPRM. P. V squez (UPRM) Conferencia 1/ 17
Dr. Pedro V squez UPRM P. V squez (UPRM) Conferencia 1/ 17 Quadratic programming MATH 6026 Equality constraints A general formulation of these problems is: min x 2R nq (x) = 1 2 x T Qx + x T c (1) subjec
More informationMATH Dr. Pedro Vásquez UPRM. P. Vásquez (UPRM) Conferencia 1 / 17
MATH 6026 Dr. Pedro Vásquez UPRM P. Vásquez (UPRM) Conferencia 1 / 17 Quadratic programming uemath 6026 Equality constraints A general formulation of these problems is: min x 2R nq (x) = 1 2 x T Qx + x
More informationReal Symmetric Matrices and Semidefinite Programming
Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationUniversity of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm
University of Colorado at Denver Mathematics Department Applied Linear Algebra Preliminary Exam With Solutions 16 January 2009, 10:00 am 2:00 pm Name: The proctor will let you read the following conditions
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAASCNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAASCNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationThe extreme rays of the 5 5 copositive cone
The extreme rays of the copositive cone Roland Hildebrand March 8, 0 Abstract We give an explicit characterization of all extreme rays of the cone C of copositive matrices. The results are based on the
More informationLecture 2  Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then
Lecture 2  Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then 1. x S is a global minimum point of f over S if f (x) f (x ) for any x S. 2. x S
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of WisconsinMadison. IMA, August 2016 Stephen Wright (UWMadison) Linear Programming: Simplex IMA, August 2016
More informationMatrix Multiplication Chapter IV Special Linear Systems
Matrix Multiplication Chapter IV Special Linear Systems By Gokturk Poyrazoglu The State University of New York at Buffalo BEST Group Winter Lecture Series Outline 1. Diagonal Dominance and Symmetry a.
More informationRank and inertia optimizations of two Hermitian quadratic matrix functions subject to restrictions with applications
Rank and inertia optimizations of two Hermitian quadratic matrix functions subject to restrictions with applications Yongge Tian a, Ying Li b,c a China Economics and Management Academy, Central University
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,
More informationChapter 2: Unconstrained Extrema
Chapter 2: Unconstrained Extrema Math 368 c Copyright 2012, 2013 R Clark Robinson May 22, 2013 Chapter 2: Unconstrained Extrema 1 Types of Sets Definition For p R n and r > 0, the open ball about p of
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150  Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationProblem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show
MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology  Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,
More informationLinear Algebra And Its Applications Chapter 6. Positive Definite Matrix
Linear Algebra And Its Applications Chapter 6. Positive Definite Matrix KAIST wit Lab 2012. 07. 10 남성호 Introduction The signs of the eigenvalues can be important. The signs can also be related to the minima,
More informationSolvability of Linear Matrix Equations in a Symmetric Matrix Variable
Solvability of Linear Matrix Equations in a Symmetric Matrix Variable Maurcio C. de Oliveira J. William Helton Abstract We study the solvability of generalized linear matrix equations of the Lyapunov type
More informationMath 302 Outcome Statements Winter 2013
Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the ThreeDimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a
More informationSymmetric matrices and dot products
Symmetric matrices and dot products Proposition An n n matrix A is symmetric iff, for all x, y in R n, (Ax) y = x (Ay). Proof. If A is symmetric, then (Ax) y = x T A T y = x T Ay = x (Ay). If equality
More informationMATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.
MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Rowcolumn rule: ijth entry of AB:
More informationMath 408 Advanced Linear Algebra
Math 408 Advanced Linear Algebra ChiKwong Li Chapter 4 Hermitian and symmetric matrices Basic properties Theorem Let A M n. The following are equivalent. Remark (a) A is Hermitian, i.e., A = A. (b) x
More informationExample Linear Algebra Competency Test
Example Linear Algebra Competency Test The 4 questions below are a combination of True or False, multiple choice, fill in the blank, and computations involving matrices and vectors. In the latter case,
More informationMath Matrix Algebra
Math 44  Matrix Algebra Review notes  (Alberto Bressan, Spring 7) sec: Orthogonal diagonalization of symmetric matrices When we seek to diagonalize a general n n matrix A, two difficulties may arise:
More informationRemark By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.
Sec 6 Eigenvalues and Eigenvectors Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called an eigenvalue of A if there is a nontrivial
More informationSparsityPreserving Difference of Positive Semidefinite Matrix Representation of Indefinite Matrices
SparsityPreserving Difference of Positive Semidefinite Matrix Representation of Indefinite Matrices Jaehyun Park June 1 2016 Abstract We consider the problem of writing an arbitrary symmetric matrix as
More informationCharacterization of halfradial matrices
Characterization of halfradial matrices Iveta Hnětynková, Petr Tichý Faculty of Mathematics and Physics, Charles University, Sokolovská 83, Prague 8, Czech Republic Abstract Numerical radius r(a) is the
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra)
AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 3: PositiveDefinite Systems; Cholesky Factorization Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 11 Symmetric
More informationA Brief Outline of Math 355
A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting
More informationHW2  Due 01/30. Each answer must be mathematically justified. Don t forget your name.
HW2  Due 0/30 Each answer must be mathematically justified. Don t forget your name. Problem. Use the row reduction algorithm to find the inverse of the matrix 0 0, 2 3 5 if it exists. Double check your
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationIdentifying Redundant Linear Constraints in Systems of Linear Matrix
Identifying Redundant Linear Constraints in Systems of Linear Matrix Inequality Constraints Shafiu Jibrin (shafiu.jibrin@nau.edu) Department of Mathematics and Statistics Northern Arizona University, Flagstaff
More informationMath Linear Algebra Final Exam Review Sheet
Math 151 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a componentwise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationPositive semidefinite intervals for matrix pencils.
University of Windsor Scholarship at UWindsor Electronic Theses and Dissertations 112004 Positive semidefinite intervals for matrix pencils. Huiming Song University of Windsor Follow this and additional
More informationHere each term has degree 2 (the sum of exponents is 2 for all summands). A quadratic form of three variables looks as
Reading [SB], Ch. 16.116.3, p. 375393 1 Quadratic Forms A quadratic function f : R R has the form f(x) = a x. Generalization of this notion to two variables is the quadratic form Q(x 1, x ) = a 11 x
More information22.4. Numerical Determination of Eigenvalues and Eigenvectors. Introduction. Prerequisites. Learning Outcomes
Numerical Determination of Eigenvalues and Eigenvectors 22.4 Introduction In Section 22. it was shown how to obtain eigenvalues and eigenvectors for low order matrices, 2 2 and. This involved firstly solving
More informationPart 4: Activeset methods for linearly constrained optimization. Nick Gould (RAL)
Part 4: Activeset methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where
More informationA Note on Eigenvalues of Perturbed Hermitian Matrices
A Note on Eigenvalues of Perturbed Hermitian Matrices ChiKwong Li RenCang Li July 2004 Let ( H1 E A = E H 2 Abstract and Ã = ( H1 H 2 be Hermitian matrices with eigenvalues λ 1 λ k and λ 1 λ k, respectively.
More information1. Introduction. Consider the following quadratically constrained quadratic optimization problem:
ON LOCAL NONGLOBAL MINIMIZERS OF QUADRATIC OPTIMIZATION PROBLEM WITH A SINGLE QUADRATIC CONSTRAINT A. TAATI AND M. SALAHI Abstract. In this paper, we consider the nonconvex quadratic optimization problem
More informationMath 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 78pm in 117 MacMillan What will be covered The exam will cover material from the lectures
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661  Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationMTH 2032 SemesterII
MTH 202 SemesterII 201011 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents
More informationOPTIMAL SCALING FOR P NORMS AND COMPONENTWISE DISTANCE TO SINGULARITY
published in IMA Journal of Numerical Analysis (IMAJNA), Vol. 23, 19, 23. OPTIMAL SCALING FOR P NORMS AND COMPONENTWISE DISTANCE TO SINGULARITY SIEGFRIED M. RUMP Abstract. In this note we give lower
More informationCHARACTERIZATIONS. is pd/psd. Possible for all pd/psd matrices! Generating a pd/psd matrix: Choose any B Mn, then
LECTURE 6: POSITIVE DEFINITE MATRICES Definition: A Hermitian matrix A Mn is positive definite (pd) if x Ax > 0 x C n,x 0 A is positive semidefinite (psd) if x Ax 0. Definition: A Mn is negative (semi)definite
More information9. Numerical linear algebra background
Convex Optimization Boyd & Vandenberghe 9. Numerical linear algebra background matrix structure and algorithm complexity solving linear equations with factored matrices LU, Cholesky, LDL T factorization
More informationMATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)
MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m
More informationRecall the convention that, for us, all vectors are column vectors.
Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists
More informationIdentifying Redundant Linear Constraints in Systems of Linear Matrix. Inequality Constraints. Shafiu Jibrin
Identifying Redundant Linear Constraints in Systems of Linear Matrix Inequality Constraints Shafiu Jibrin (shafiu.jibrin@nau.edu) Department of Mathematics and Statistics Northern Arizona University, Flagstaff
More informationMatrix Mathematics. Theory, Facts, and Formulas with Application to Linear Systems Theory. Dennis S. Bernstein
Matrix Mathematics Theory, Facts, and Formulas with Application to Linear Systems Theory Dennis S. Bernstein PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD Contents Special Symbols xv Conventions, Notation,
More informationMATH 1553C MIDTERM EXAMINATION 3
MATH 553C MIDTERM EXAMINATION 3 Name GT Email @gatech.edu Please read all instructions carefully before beginning. Please leave your GT ID card on your desk until your TA scans your exam. Each problem
More informationarxiv: v1 [math.na] 1 Sep 2018
On the perturbation of an L orthogonal projection Xuefeng Xu arxiv:18090000v1 [mathna] 1 Sep 018 September 5 018 Abstract The L orthogonal projection is an important mathematical tool in scientific computing
More informationA sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm
Volume 31, N. 1, pp. 205 218, 2012 Copyright 2012 SBMAC ISSN 01018205 / ISSN 18070302 (Online) www.scielo.br/cam A sensitivity result for quadratic semidefinite programs with an application to a sequential
More informationResearch Note. A New Infeasible InteriorPoint Algorithm with Full NesterovTodd Step for SemiDefinite Optimization
Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 88107 Research Note A New Infeasible InteriorPoint Algorithm with Full NesterovTodd Step for SemiDefinite Optimization B. Kheirfam We
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tuberlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data TwoDimensional Plots Programming in
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More informationYORK UNIVERSITY. Faculty of Science Department of Mathematics and Statistics MATH M Test #1. July 11, 2013 Solutions
YORK UNIVERSITY Faculty of Science Department of Mathematics and Statistics MATH 222 3. M Test # July, 23 Solutions. For each statement indicate whether it is always TRUE or sometimes FALSE. Note: For
More informationNumerical Optimization
Numerical Optimization Unit 2: Multivariable optimization problems CheRung Lee Scribe: February 28, 2011 (UNIT 2) Numerical Optimization February 28, 2011 1 / 17 Partial derivative of a two variable function
More informationChapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015
Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal
More information22.3. Repeated Eigenvalues and Symmetric Matrices. Introduction. Prerequisites. Learning Outcomes
Repeated Eigenvalues and Symmetric Matrices. Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one
More information22.3. Repeated Eigenvalues and Symmetric Matrices. Introduction. Prerequisites. Learning Outcomes
Repeated Eigenvalues and Symmetric Matrices.3 Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one
More informationPrincipal Component Analysis (PCA) Our starting point consists of T observations from N variables, which will be arranged in an T N matrix R,
Principal Component Analysis (PCA) PCA is a widely used statistical tool for dimension reduction. The objective of PCA is to find common factors, the so called principal components, in form of linear combinations
More informationOn the Solution of Constrained and Weighted Linear Least Squares Problems
International Mathematical Forum, 1, 2006, no. 22, 10671076 On the Solution of Constrained and Weighted Linear Least Squares Problems Mohammedi R. AbdelAziz 1 Department of Mathematics and Computer Science
More informationMath 308 Practice Final Exam Page and vector y =
Math 308 Practice Final Exam Page Problem : Solving a linear equation 2 0 2 5 Given matrix A = 3 7 0 0 and vector y = 8. 4 0 0 9 (a) Solve Ax = y (if the equation is consistent) and write the general solution
More information1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?
. Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in
More informationChapter 1. Matrix Algebra
ST4233, Linear Models, Semester 1 20082009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface
More informationFall 2016 MATH*1160 Final Exam
Fall 2016 MATH*1160 Final Exam Last name: (PRINT) First name: Student #: Instructor: M. R. Garvie Dec 16, 2016 INSTRUCTIONS: 1. The exam is 2 hours long. Do NOT start until instructed. You may use blank
More informationVISCOSITY SOLUTIONS. We follow Han and Lin, Elliptic Partial Differential Equations, 5.
VISCOSITY SOLUTIONS PETER HINTZ We follow Han and Lin, Elliptic Partial Differential Equations, 5. 1. Motivation Throughout, we will assume that Ω R n is a bounded and connected domain and that a ij C(Ω)
More informationReview Questions REVIEW QUESTIONS 71
REVIEW QUESTIONS 71 MATLAB, is [42]. For a comprehensive treatment of error analysis and perturbation theory for linear systems and many other problems in linear algebra, see [126, 241]. An overview of
More informationPolynomiality of Linear Programming
Chapter 10 Polynomiality of Linear Programming In the previous section, we presented the Simplex Method. This method turns out to be very efficient for solving linear programmes in practice. While it is
More informationRemark 1 By definition, an eigenvector must be a nonzero vector, but eigenvalue could be zero.
Sec 5 Eigenvectors and Eigenvalues In this chapter, vector means column vector Definition An eigenvector of an n n matrix A is a nonzero vector x such that A x λ x for some scalar λ A scalar λ is called
More information