Chapter 3. Some Applications. 3.1 The Cone of Positive Semidefinite Matrices
|
|
- Scarlett Morgan
- 6 years ago
- Views:
Transcription
1 Chapter 3 Some Applications Having developed the basic theory of cone programming, it is time to apply it to our actual subject, namely that of semidefinite programming. Indeed, any semidefinite program assumes the form of a cone program Maximize Tr(C T X) subject to A(X) = b X 0 Maximize c, x subject to b Ax L x K, where K = {X S n : X 0} V = S n, the space of symmetric n n matrices, equipped with the scalar product X, Y = Tr(X T Y ). We have shown in Lemma 2..2 that K is a closed convex cone. Furthermore, L = {0} W = R m in our application. 3. The Cone of Positive Semidefinite Matrices Let us from now on use the notation S + n := {X S n : X 0} for the cone of positive semidefinite n n matrices. This notation is deliberately chosen in analogy with the notation R n + for the cone {x R n : x 0}, the nonnegative orthant in R n. In fact, there are many similarities between the two cones R n + and S+ n, to some of which we will get next. At this point, a sceptical reader might want to convince him- or herself that this is indeed a scalar product. 39
2 3.. Self-Duality 3. The Cone of Positive Semidefinite Matrices 40 We start by showing that the cone S n + is self-dual, just like Rn +. This statement is also known as Fejér s Trace Theorem. 3.. Lemma. (S + n ) = S + n. Proof. We first prove an auxiliary Claim. Let M, M S + n. Then Tr(MT M ) 0. It is instructive to consider the following non-proof first: If M, M S + n, then also M T M S + n. Therefore all diagonal elements of M T M and thus also the trace are nonnegative. What makes this a non-proof is the observation that the product of positive semidefinite matrices need not even be symmetric, let alone an element of S + n. Here is the real proof: We diagonalize M and write it in the form M = SDS T, where S is an orthogonal matrix (i.e. S = S T ) and D is a diagonal matrix with the nonnegative eigenvalues λ,...,λ n of M on the diagonal. Let s i denote the ith column of S. Using symmetry of Tr, we now compute Tr(M T M ) = Tr(S DS T M ) = Tr(DS T M S) = n λ i s T i M s i 0, i= since s T i M s i 0 by M S n +, see Fact.4.(ii). Using this claim, S + n (S+ n ) follows. The other direction is even simpler. Choose M (S + n ). For all x R n, the matrix xx T is positive semidefinite, so M (S + n ) implies 0 Tr(M T xx T ) = x T Mx x R n, meaning that M S + n Generating Matrices Every vector in R n is a linear combination of the n unit vectors e i, i =,...,n, and R n + can be characterized as the set of nonnegative linear combinations. In that sense, the n unit vectors generate both R n and R n +. Similarly, and as we show next, every matrix in S n is a linear combination of n matrices of the form ss T, s =, and S n + can be characterized as the set of nonnegative linear combinations. In that sense, the matrices ss T generate both S n and S n +. The essential difference to the case of Rn and R n + is that there is no finite set of generators.
3 4 3. Some Applications 3..2 Lemma. Let M be an n n matrix. We have M S n (M S + n, respectively) if and only if there are unit-length vectors s,...,s n S n and real numbers (nonnegative real numbers, respectively) λ,...,λ n such that M = n λ i s i s T i. i= Proof. The if directions are clear: all the s i s T i are in S n + S n, so every linear combination is in S n. By convexity of S n +, every nonnegative linear combination is again in S n +. For the only if directions, we again diagonalize M as M = SDS T, where S is orthogonal and D is the diagonal matrix of eigenvalues λ,...,λ n (which are nonnegative if M S n + ). With D(i) being the matrix Q such that q ii = λ i (and zero otherwise), we get M = S( n D (i) )S T = i= n SD (i) S T = i= n λ i s i s T i, where s i is the i-th column of S. By orthogonality of S, s i = for all i. i= 3.2 Largest Eigenvalue The following cannot really be called an application, but it nicely illustrates the use of duality in a very simple case. We will reprove the following well-known fact from linear algebra, using cone programming duality. There exists in fact a much shorter proof of this theorem, but the point here is to see duality in action (and prepare grounds for a more general statement, see Exercise 3.4.4) Theorem. Let C S n. Then the largest eigenvalue of C is equal to λ = max{x T Cx : x R n, x = }. We note that the maximum exists, since we are optimizing a continuous function x T Cx over a compact subset of R n. Proof. We first rewrite x T Cx as Tr(C T xx T ) and x = as Tr(xx T ) =. This means that λ is the value of the constrained optimization problem Maximize Tr(C T xx T ) subject to Tr(xx T ) =. (3.) This program is obtained from the semidefinite program Maximize Tr(C T X) subject to Tr(X) = X 0 (3.2)
4 3.2 Largest Eigenvalue 42 by adding the constraint that X has rank. Indeed, the positive semidefinite matrices of rank are exactly the ones of the form xx T (Exercise 3.4.2). Equivalently, (3.2) can be considered as a relaxation of (3.). Note that Tr is a linear operator from S n to R. The crucial fact is that (3.) and (3.2) have the same value λ. Indeed, if X is any feasible solution of (3.2), Lemma 3..2 lets us write X = n λ i x i x T i, i= with nonnegative λ i s. Since the x i s are unit vectors, all matrices x i x T i trace, so linearity of Tr implies n i= λ i =. But then we have have Tr(C T X) = Tr(C T n n λ i x i x T i ) = λ i Tr(C T x i x T i ) max n i= Tr(CT x i x T i ) λ, i= i= since the x i are feasible solutions of (3.). Tr(C T X ) λ for some X follows from the fact that (3.2) is a relaxation of (3.). Now we are prepared to apply the strong Duality Theorem of cone programming, in the version for equational form. Written as a cone program in equational form, (3.2) has K = S + n S n, c = C S n, b = R, and A = Tr. In order to be able to write down the dual, we just need to determine the adjoint operator A T : R S n. From the requirement y, A(X) = y Tr(X)! = Tr(A T (y)x) = A T (y), X for all X S n, y R, we infer that A T (y) := yi n is an adjoint, so that the dual of (3.2) is Minimize y subject to yi n C 0. (3.3) Since the primal program (3.2) has an interior point (choose X = I n /n, for example), the duality theorem applies and shows that the optimal value of (3.3) is also λ. But what is λ? If C has eigenvalues λ,...,λ n, then yi n C has eigenvalues y λ,..., y λ n, and the constraint yi n C 0 requires all of them to be nonnegative. Therefore, the value λ, the smallest y for which yi n C 0 holds, equals the largest eigenvalue of C. This proves the theorem. Exercise discusses semidefinite programming formulations for the sum of the k largest eigenvalues.
5 43 3. Some Applications E F I J L E F I J L E F I J L E F I J L Figure 3.: Left: The recognition graph connects an input letter v to an output letter w if v may be recognized as w. Right: the similarity graph connects two input letters if they may be recognized as the same output letter 3.3 Shannon Capacity and Theta Function of a Graph Suppose that you have just bought some optical character recognition system. Your goal as a citizen of the internet age may be to digitize all the books that you have, so that you can safely throw them away afterwards. The system is not perfect, though, and it sometimes gets letters wrong. For example, the letter E might mistakenly be recognized as an F. In general, there are input letters (the ones in the book) and output letters (the ones being recognized). Input and output letters may come from the same, but also from different alphabets. We can encode the possible behaviors of the system as a directed bipartite recognition graph that connects an input letter v to an output letter w if the system may recognize v as w, see Figure 3. (left) for an example with 5 input letters and the same 5 output letters. Two input letters v, v are called similar if there is an ouput letter w such that both v and v may be recognized as w. In the example, v = E and v = F are similar, with w = F witnessing their similarity. The letters J and L are also similar, since both of them could be recognized as I. Finally, each letter is similar to itself by definition. We can record this information in an (undirected) similarity graph that connects two distinct input letters if they are similar, see Figure 3. (right). The information that every letter is similar to itself is implicit. If the similarity graph is empty, the system can correctly scan all your books: for every recognized output letter w, there is exactly one matching input letter v, and assuming the the system knows its recognition graph, the correct input letter v can be reconstructed. But already with a relatively sparse but nonempty similarity graph, the system may get a lot of words wrong. For example, a word with many E s is pretty likely to get corrupted, since it suffices if just one of the E s is mistakenly recognized as an F. For this reason, the system has an error correction: It comes with a built-in similarity-free dictionary of allowed input words. This means that no
6 3.3 Shannon Capacity and Theta Function of a Graph 44 two distinct allowed words are similar in the sense that they may be recognized as the same output word. Formally, two k-letter words v...v k and v...v k are similar if and only if v i is similar to v i for all i. Indeed, if the dictionary is similarity-free, then error corrections works, since for every recognized word w...w k, there is exactly one word v...v k in the dictionary such that v i may be recognized as w i for all i, and this word must be the correct input word. 2 While you are waiting for your next book to be scanned, your mind is drifting off and you start asking a theoretical question. What is the largest similarity-free dictionary of k-letter words? For k = (the words are just letters), this is easy to answer: The dictionary must be an independent set in the similarity graph. The largest similarity-free dictionary of -letter words is therefore a maximum independent set in the similarity graph. For k >, we can easily form a graph G k whose edges characterize similarity between k-letter words. The vertices of G k are the words of length k, and there is an edge between two words v...v k and v...v k if they are similar, meaning that v i is similar to v i for all i. This leads to the following 3.3. Observation. Let α(g) denote the independence number of a graph G, i.e. the size of a maximum independent set in G. Then the largest similarity-free dictionary of k-letter words has size α(g k ). It is known that the independence number of a graph is NP-hard to compute [4, Section 3..3], so finding the size of the largest similarity-free dictionary is hard even for -letter words. However, this is not our main concern here, since we want to study the sequence (α(g k )) k N in its entirety. We start by showing that the sequence is super-multiplicative Lemma. For all k, l N, α(g k+l ) α(g k )α(g l ). Proof. If I is an independent set in G k, and J is an independent set in G l, then the set of I J words {v...v k w...w l : v...v k I, w...w l J} is independent in G k+l. Indeed, no two distinct words in this set can be similar, as this would imply that at least one of I and J contains two distinct similar words. If I = α(g k ) and J = α(g l ), the statement follows. 2 If there is no such word v...v k in the dictionary, then the correct input word was not an allowed word.
7 45 3. Some Applications The inequality in Lemma is in general strict. For the 5-cycle C 5 we have α(c 5 ) = 2. But α(c5) 2 5 > α(c 5 ) 2. To see this, we use the interpretation of α(c5 2 ) as the size of a largest similarity-free dictionary of 2-letter words. Suppose that the letters around the cycle C 5 are A, B, C, D, E. Then it is easy to check that the following five 2-letter words are pairwise non-similar: AA, BC, CE, DB, and ED. This example is actually best possible and α(c5 2) = The Shannon Capacity We may view a dictionary as a set of messages, encoded by k-letter words. The goal is to safely transmit any given message over a noisy channel whose input/output behavior induces a similarity graph G as in Figure 3. (right). If the dictionary is similarity-free w.r.t. G, we can indeed correct all errors being made during transmission. Using -letter words, we can thus safely transmit α(g) different messages. This means that every letter carries (roughly) log(α(g)) bits of information (the logarithm is binary). Using k-letter words, we can transmit α(g k ) different messages, meaning that each of the k letters carries k log α(gk ) bits of information. We are interested in the best k, the one that leads to the highest information-per-letter ratio. It easily follows from our above considerations on C 5 that k = is not always the best choice. Indeed, we have log(α(c 5 ) = < 2 log(α(c2 5) = log Consequently, let us define the Shannon capacity of a graph G as σ(g) = sup{ k log α(gk ) : k N}, (3.4) the (asymptotically) highest information-per-letter ratio that can be achieved. This definition is due to Claude Shannon [5] Lemma. For every graph G = (V, E), σ(g) is bounded and satisfies ( ) σ(g) = lim k k log α(gk ). Proof. Since G k has V k vertices, we obviously have α(g k ) V k which implies that σ(g) log V. Taking logarithms in Lemma 3.3.2, we see that the sequence (x k ) k N = (log α(g k )) k N is super-additive, meaning that x k+l x k +x l
8 3.3 Shannon Capacity and Theta Function of a Graph 46 for all k, l. Now we use Fekete s Lemma which states that for every super-additive sequence (x k ) k N, the sequence ( xk ) k k N converges to its supremum (Exercise asks you to prove this). Shannon already remarked in his original paper [5] in 956 that it can be quite difficult to compute σ(g) even for small graphs G, and he in particular failed to determine σ(c 5 ). We know that σ(c 5 ) log 5, 2 but it is absolutely not clear whether k = 2 yields the best possible informationper-letter ratio. Only in 979, Lovász was able to determine σ(c 5 ) = /2 log5, showing that the lower bound obtained from 2-letter encodings is tight []. Lovász did this by deriving the theta function, a new upper bound on σ(g) (computable with semidefinite programming, as we will see), and by showing that this upper bound matches the known lower bound for σ(c 5 ). Instead of σ(g), Lovász uses the following equivalent quantity Definition. Let G be a graph. The Lovász-Shannon capacity of G is Θ(G) = 2 σ(g) = lim k k α(g k ). (3.5) We remark that the Lovász-Shannon capacity is lower-bounded by α(g) (by super-multiplicativity) and upper-bounded by V. After this transformation, the statement σ(c 5 ) = /2 log 5 reads as Θ(C 5 ) = The Theta Function We first pinpoint our earlier notation of similarity Definition. Let G = (V, E) be a graph. Vertices v and v are called similar in G if either v = v or {v, v } E Definition. An orthonormal representation of a graph G = (V, E) with V = n is a set U = {u v : v V } of unit vectors in S n such that u T v u v = 0 if v and v are not similar in G. (3.6) It is clear that every graph has such a representation, since we may take the n pairwise orthogonal unit vectors e,...,e n
9 47 3. Some Applications But we are looking for a better representation, if possible. Intuitively, a representation is good if there exists a unit vector that is far from being orthogonal to any of the u v. Formally, and at first glance somewhat arbitrarily, we define the value of an orthonormal representation U = {u v : v V } as ϑ(u) := min max c = v V (c T u v ) 2. (3.7) The minimum exists, since we can cast the problem as the minimization of a continuous function over a compact set (the unit sphere S n minus suitable open balls around the u v to avoid singularities). A vector c that attains the minimum is called a handle of U Definition. The theta function ϑ(g) of G is the smallest value ϑ(u) over all orthonormal representations U of G. Again, the minimum exists, as (i) ϑ(u) is continuous, and (ii) the set of orthonormal representations is the compact set (S n ) n, intersected with closed sets of the form {u T v u v = 0} (which again yields a compact set) The Lovász Bound In this section we show that ϑ(g) is an upper bound for the Lovász-Shannon capacity Θ(G). This requires two elementary lemmas. With the definition of the graph G k on Page 44, we want to prove that ϑ(g k ) ϑ(g) k (recall that the inverse inequality holds for the independence number α, by Lemma 3.3.2). For this, we handle the case k = 2 first, in the following more general form Definition. Let G = (V, E) and H = (W, F) be graphs. The strong product of G and H is the graph G H with vertex set V W, and an edge between (v, w) and (v, w ) if and only if v is similar to v in G and w is similar to w in H Lemma. For all graphs G and H, ϑ(g H) ϑ(g)ϑ(h). Since G k is isomorphic to the k-fold strong product ( ((G G) G) G), we obtain Corollary. ϑ(g k ) ϑ(g) k.
10 3.3 Shannon Capacity and Theta Function of a Graph 48 Proof of Lemma Let U = {u v : v V } and V = {v w : w W } be optimal orthonormal representations of G = (V, E) and H = (W, F), with handles c and d. We will from this construct an orthonormal representation of G H with value at most ϑ(g)ϑ(h). The construction is simple: the orthonormal representation is obtained by taking all tensor products of vectors u v with vectors v w, and an upper bound for its value is computed using the tensor product of the handles c and d. The tensor product of two vectors x R m and y R n is the (column) vector x y R mn defined by x y = (x y,...,x y n, x 2 y,...,x 2 y n,...,x m y,...,x m y n ) R mn. Equivalently, the tensor product is the matrix xy T, written as one long vector (row by row). We have that (x y) T (x y ) = m n x i y j x iy j = i= j= m n x i x i y j y j = (x T x )(y T y ). (3.8) i= j= Now we can prove that the vectors u v v w indeed form an orthonormal representation U V of G H. As a direct consequence of (3.8), all of them are unit vectors. Moreover, if (v, w) and (v, w ) are not similar in G H, then v is not similar to v in G, or w is not similar to w in H. In both cases, (3.8) implies that (u v v w ) T (u v v w ) = (u T v u v )(ut w u w ) = 0, since U and V are orthonormal representations of G and H. Thus, we have an orthonormal representation of G H. By definition, ϑ(g H) is bounded by ϑ(u V) max v V,w W ((c d) T (u v v w )) 2 = max v V,w W (c T u v ) 2 (d T v w ) 2 = ϑ(g)ϑ(h). Here is the second lemma that we need: The theta function ϑ(g) is like the Lovász-Shannon capacity Θ(G) an upper bound for the independence number of G Lemma. For any graph G, α(g) ϑ(g). Proof. Let I V (G) be a maximum independent set in G, and let U = {u v : v V } be an optimal orthonormal representation with handle c. We know that the vectors u v, v I are pairwise orthogonal which implies (Exercise 3.4.8) that c T c v I (c T u v ) 2.
11 49 3. Some Applications We thus have = c T c v I (c T u v ) 2 I min v I (ct u v ) 2 = α(g) min v I (ct u v ) 2. This in turn means that α(g) min v I (c T u v ) = max 2 v I (c T u v ) max 2 v V (c T u v ) = ϑ(g). 2 The main result of this section now easily follows: The theta function ϑ(g) is an upper bound for the Lovász-Shannon capacity Θ(G), meaning that we have α(g) Θ(G) ϑ(g) Theorem. For any graph G, Θ(G) ϑ(g). Proof. By Lemma 3.3. and Corollary 3.3.0, we have α(g k ) ϑ(g k ) ϑ(g) k. It follows that hence k α(gk ) ϑ(g) k, Θ(G) = lim k k α(g k ) ϑ(g) The 5-Cycle Using the bound of Theorem 3.3.2, we can now determine the Lovász-Shannon capacity of the 5-cycle C 5. We already know that Θ(C 5 ) 5, by using 2- letter encodings. The fact that this is best possible follows from the next lemma, together with Theorem Lemma. ϑ(c 5 ) 5. Proof. We need to find an orthonormal representation of C 5 with value at most 5. Let the vertices of C5 be 0,, 2, 3, 4 in cyclic order. Here is Lovász s umbrella construction that yields vectors u 0,...,u 4 in S 2 (we can add two zero coordinates to lift them into S 4 ). Imagine an umbrella with unit handle c = (0, 0, ) and five unit ribs of the form u i = 2πi 2πi (cos, sin 5 (cos 2πi 5, sin 2πi 5, z) 5 i = 0,...,4., z),
12 3.3 Shannon Capacity and Theta Function of a Graph 50 u u 2 2π 5 c u 0 u 3 u 4 Figure 3.2: A flat five-rib umbrella, top view If z = 0, the umbrella is completely flat (see Figure 3.2 for a top view in which c collapses to the origin), and letting z grow to corresponds to the process of folding up the umbrella. Keep folding the umbrella until the angle between u 0 and u 2 becomes π/2, meaning that the vectors become orthogonal. This will eventually happen since we start with angle 4π/5 > π/2 in flat position and converge to angle 0 as z. We can compute the value of z for which we get orthogonality: We must have 0 = u T 0 u 2 (, 0, z) cos 4π 5 sin 4π 5 z = cos 4π 5 + z2 = 0. Hence, z = cos 4π 5, u 0 = (, 0, ) cos 4π 5 cos 4π 5. For this value of z, symmetry implies that we do have an orthonormal representation U: every u i is orthogonal to the two opposite vectors u i+2 and u i 2 of its two non-neighbors in C 5 (indices are modulo 5). Recalling that c = (0, 0, ), we have that ϑ(c 5 ) ϑ(u) 4 max i=0 (c T u i ) 2 = (c T u 0 ) 4π cos 5 = 2 cos 4π 5 by symmetry. Exercise (3.4.9) asks you to prove that this number is 5.
13 5 3. Some Applications Two Semidefinite Programs for the Theta Function The value of Θ(C 5 ) was unknown for more than 20 years after Shannon had given the lower bound Θ(C 5 ) 5. Together with the Lovász bounds Θ(C 5 ) ϑ(c 5 ) 5, we get Θ(C 5 ) = ϑ(c 5 ) = 5. Here we want to discuss how ϑ(g) can be computed for an arbitrary graph G. The above method for C 5 was somewhat ad-hoc, and only in hindsight it turned out that the umbrella construction yields an optimal orthonormal representation. In general, the definition of ϑ(g) does not give rise to an efficient algorithm. However, Lovász proved that ϑ(g) can alternatively be expressed as the value of a semidefinite program. This implies that ϑ(g) is efficiently computable, up to any desired precision. In fact, there are various semidefinite programs that can be used to compute ϑ(g). The first one is obtained by more or less just rewriting the definition. Recall that ϑ(g) is the smallest value of ϑ(u) = min max c = v V (c T u v ) 2, over all orthonormal representations U. By replacing u v with u v if necessary, we may assume c T u v 0 for all v. But then, ϑ(g) = max U = max ϑ(u) U max c = min v V ct u v. With t R n + being an additional variable for the minimum, we see that / ϑ(g) is the value of the program Maximize t subject to u T v u v = 0 if v and v are not similar in G c T u v t, v V u v =, v V c = (3.9) This has not yet the form of a semidefinite program in equational form, but it can be brought into this form, see Exercise and observe the remark after Theorem Theorem. For any graph G = (V, E) with V = {,..., n}, the theta function ϑ(g) is the value of the following semidefinite program in the matrix variable Y S n and the real variable t. Minimize t subject to y ij = y ii = t Y 0 if i is not similar to j in G for all i =,..., n (3.0)
14 3.3 Shannon Capacity and Theta Function of a Graph 52 One remark is in order. We can easily write this as a cone program in equational form, with cone K = S + n R + (t follows from y ii = t and Y 0). But this can also be simulated by a semidefinite program: append to Y one row and column, add the constraints y n+,i = y i,n+ = 0 for i n and replace t with x n+,n+ throughout. Then the larger matrix is positive semidefinite if and only if Y 0 and y n+,n+ 0, so (3.0) is a proper semidefinite program. In the same way, we can integrate an arbitrary number of nonnegative real variables into any semidefinite program. Proof. We first show that the value of (3.0) is at most ϑ(g). Let U = {u,...,u n } be an optimal orthonormal representation of G with handle c. Now we define a matrix Ỹ S n by and ỹ ij = u T i u j (c T u i )(c T u j ), i j ỹ ii = ϑ(g), i =,...,n. Since U is an orthonormal representation, we have ỹ ij = for i not similar to j. If we can show that Ỹ 0, we know that the pair (Ỹ, ϑ(g)) is a feasible solution of (3.0), meaning that the program s value is at most ϑ(g). To see Ỹ 0, we first observe (a simple calculation) that ỹ ij = and (by definition of ϑ(g)) ỹ ii = ϑ(g) ( c u i c T u i ) T ( c u j c T u j ( (c T u i ) = c 2 ), i j, u i c T u i ) T ( c u i c T u i This means that Ỹ is of the form Ỹ = D + UT U, where D is a diagonal matrix with nonnegative entries, and U is the matrix whose i-th column is the vector c u i /c T u i. Thus, Ỹ 0. To show that the value of (3.0) is at least ϑ(g), we let (Ỹ, t) be any feasible solution of (3.0) with the property that t is minimal subject to ỹ ij being fixed for i j. This implies that Ỹ has one eigenvalue equal to 0 (otherwise we could decrease t) and is therefore singular. Note that t. We now perform a Cholesky decomposition Ỹ = ST S, see Fact.4.(iii). Let s,...,s n be the columns of S. Since Ỹ is singular, S is singular as well, and the s i span a proper subspace of R n. Consequently, there exists a unit vector c that is orthogonal to all the s i. Next we define u i := t (c + s i), i =,...,n, ).
15 53 3. Some Applications and we intend to show that U = {u,...,u n } is an orthonormal representation of G. For this, we compute u T i u j = t (c + s i) T (c + s j ) = t (ct }{{} c +c T s j + s T i }{{}}{{} c 0 0 It follows that +s T i s j) = }{{} ỹ ij ỹ ii = t u i =, for all i =,...,n, ỹ ij = u T i u j = 0, if i not similar to j, t ( + ỹ ij). so we have indeed found an orthonormal representation of G. Since we further have (c T u i ) 2 = ( c T t ) 2 (c + s i) = t ( c T (c + s i ) ) 2 = t, i =,...,n, we get which completes the proof. ϑ(g) ϑ(u) n max i= (c T u i ) 2 = t, The Sandwich Theorem and Perfect Graphs We know that ϑ(g) is lower-bounded by α(g), the independence number of the graph G. But we can also upper-bound ϑ(g) in terms of another graph parameter. This bound will also shed some more light on the geometric interpretation of the semidefinite program (3.0) for ϑ(g) Definition. Let G = (V, E) be a graph. (i) A clique in G is a subset K V of vertices such that {v, w} E for all distinct v, w K. The clique number ω(g) of G is the size of a largest clique in G. (ii) A k-coloring of G is a mapping c : V {,..., k} such that c(v) c(w) if {v, w} E. The chromatic number χ(g) of G is the smallest k such that G has a k-coloring. (iii) The complementary graph Ḡ = (V, Ē) of E is defined via Ē = ( V 2) \ E. Here, ( V 2) is the set of two-element subsets of V.
16 3.3 Shannon Capacity and Theta Function of a Graph 54 According to this definition, an independent set in G is a clique in Ḡ, and vice versa. Consequently, α(g) = ω(ḡ). (3.) Here is the promised upper bound on ϑ(g). Together with the already known lower bound, we obtain the Sandwich Theorem that bounds ϑ(g) in terms of clique number and chromatic number of the complementary graph Theorem. For every graph G = (V, E), ω(ḡ) ϑ(g) χ(ḡ). Proof. For the lower bound, we use Lemma 3.3. together with (3.). For the upper bound, let us suppose that ϑ(g) >, as the bound is trivial for ϑ(g) = ). But then, χ(ḡ) 2, since a -coloring is possible only for Ē = in which case ϑ(g) = ). Now, let us rescale (3.0) into the following equivalent form (we assume that V = {,..., n}): Minimize t subject to y ij = /(t ) y ii = Y 0 if {i, j} Ē for all i =,...,n (3.2) At the same time, we have replaced the condition i is not similar to j by the equivalent condition that {i, j} is an edge in the complementary graph Ḡ = (V, Ē). If we rewrite Y 0 as Y = ST S for S a matrix with columns s,...,s n, the equality constraints of (3.2) translate as follows. y ij = t st i s j = t y ii = s i =. Lemma below shows that if Ḡ has a k-coloring, then we actually find vectors s i that satisfy the latter equations, for t = k. This implies that (Y, t) = (S T S, k) is a feasible solution of (3.2), and hence k ϑ(g), the value of (3.2). The upper bound follows if we choose k = χ(ḡ). The vectors s i constructed in the latter proof can be regarded as a vector-kcoloring of Ḡ Definition. For k R, a vector-k-coloring of a graph G = (V, E) is a mapping γ : V S n such that γ(v) T γ(w) =, {v, w} E. k
17 55 3. Some Applications For a k-coloring, we require that adjacent vertices have different colors. For a vector k-coloring, we require the colors of adjacent vertices to have a large angle. The proof of Theorem shows that ϑ(g) is the smallest k such that Ḡ has a vector k-coloring. The upper bound ϑ(g) χ(ḡ) then follows from the fact that the notion of vector-k-colorings is a relaxation of the notion of k-colorings: Lemma. If a graph G has a k-coloring, then it also has a vector-kcoloring. Proof: We construct k unit-length vectors u,...,u k such that u T i u j = k, i j. Given a k-coloring c of G, a vector-k-coloring can then be obtained via γ(v) = u c(v), v V. The k vectors form the vertices of a regular simplex centered at the origin, see Figure 3.3 for the case k = 3. In general, we define u i = e i k e i k k l= e l k l= e, i =,...,k. l u 3 20 u u 2 Figure 3.3: k unit-length vectors with pairwise scalar products /(k ). Perfect graphs. We know that the clique number ω(g) is NP-hard to compute for general graphs. The same can be said about the chromatic number χ(g). But there is a class of graphs for which Theorem makes both values computable in polynomial time Definition. A graph G is called perfect if ω(g ) = χ(g ) for every induced subgraph G of G.
18 3.4 Exercises 56 There are many known families of perfect graphs, including for example bipartite graphs as an easy example. Indeed, every induced subgraph of a bipartite graph is again bipartite, and every bipartite graph has clique number and chromatic number equal to 2. Other examples are interval graphs (intersection graphs of closed intervals on the real line), and more generally, chordal graphs (every cycle of length at least four has an edge connecting two vertices that are not neighbors along the cycle). For perfect graphs, Theorem implies ω(g) = ϑ(ḡ) = χ(g), meaning that maximum cliques and minimum colorings can be computed for perfect graphs in polynomial time through semidefinite programming. Indeed, since we are looking for an integer, it suffices to solve (3.2) (for the complementary graph) up to accuracy ε < /2. Moreover, due to y ii =, all entries of a feasible Y are scalar products of unit vectors and hence in [, ]. This means that our requirements for polynomial-time solvability (see the beginning of Section.3) are satisfied. One can also compute the independence number α(g) of a perfect graph G in polynomial time: Recall that α(g) = ω(ḡ), and since Ḡ is perfect as well (the weak perfect graph conjecture, proved by Lovász in 972), the statement follows. 3.4 Exercises 3.4. Exercise. Let M S n be a symmetric n n matrix with eigenvalues λ,...,λ n. Prove that n Tr(M) = λ j. j= Exercise. Prove that a matrix M S n has rank if and only if M = ±ss T for some nonzero vector s R n. In particular, M S n + has rank if and only if M = ss T Exercise. Given a symmetric matrix C S n, we are looking for a matrix Y S n + such that Y C S+ n. Prove that the trace of any such matrix is at least the sum of the positive eigenvalues of C. Moreover, there exists a matrix Ỹ S n + with Ỹ C S+ n and Tr(Ỹ ) equal to the sum of the positive eigenvalues of C Exercise. Let C S n.
19 57 3. Some Applications (a) Prove that the value of the following cone program is the sum of the k largest eigenvalues of C. Minimize ky + Tr(Y ) subject to yi n + Y C S + n (Y, y) S + n R. Hint: You may use the statement of Exercise (b) Derive the dual program and show that its value is also the sum of the k largest eigenvalues of C. In doing this, you have (almost) proved Fan s Theorem Exercise. Let (x k ) k N be a sequence of real numbers such that x k+l x k + x l k, l. We say that the sequence is superadditive. Prove that x k lim k k = sup{x k k : k N}, where both the limit and the supremum may be Exercise. What is the value of the orthonormal representation U = {e,...,e n }? Exercise. Prove that the program (3.9) can be rewritten into a semidefinite program in equational form, and with the same value Exercise. Let u,...,u k be pairwise orthogonal unit vectors in R n. Prove that c T c k i= (ct u i ) 2 for all c R n Exercise. Prove that cos 4π 5 cos 4π 5 = Exercise. Prove that for all graphs G and H, ϑ(g H) = ϑ(g)ϑ(h). Hint: First write down the program dual to (3.0) and show that it also has value ϑ(g). Then look at the dual programs for G and H with optimal solutions; from them, construct a feasible solution of the dual program for G H with value ϑ(g)ϑ(h). This shows that ϑ(g H) ϑ(g)ϑ(h), and the other inequality is Lemma
20
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationFour new upper bounds for the stability number of a graph
Four new upper bounds for the stability number of a graph Miklós Ujvári Abstract. In 1979, L. Lovász defined the theta number, a spectral/semidefinite upper bound on the stability number of a graph, which
More informationSemidefinite programs and combinatorial optimization
Semidefinite programs and combinatorial optimization Lecture notes by L. Lovász Microsoft Research Redmond, WA 98052 lovasz@microsoft.com http://www.research.microsoft.com/ lovasz Contents 1 Introduction
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More information1 The independent set problem
ORF 523 Lecture 11 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Tuesday, March 29, 2016 When in doubt on the accuracy of these notes, please cross chec with the instructor
More informationApplications of the Inverse Theta Number in Stable Set Problems
Acta Cybernetica 21 (2014) 481 494. Applications of the Inverse Theta Number in Stable Set Problems Miklós Ujvári Abstract In the paper we introduce a semidefinite upper bound on the square of the stability
More informationPreliminaries and Complexity Theory
Preliminaries and Complexity Theory Oleksandr Romanko CAS 746 - Advanced Topics in Combinatorial Optimization McMaster University, January 16, 2006 Introduction Book structure: 2 Part I Linear Algebra
More informationCS 6820 Fall 2014 Lectures, October 3-20, 2014
Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given
More informationLecture 7: Semidefinite programming
CS 766/QIC 820 Theory of Quantum Information (Fall 2011) Lecture 7: Semidefinite programming This lecture is on semidefinite programming, which is a powerful technique from both an analytic and computational
More informationThe maximal stable set problem : Copositive programming and Semidefinite Relaxations
The maximal stable set problem : Copositive programming and Semidefinite Relaxations Kartik Krishnan Department of Mathematical Sciences Rensselaer Polytechnic Institute Troy, NY 12180 USA kartis@rpi.edu
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationGraph coloring, perfect graphs
Lecture 5 (05.04.2013) Graph coloring, perfect graphs Scribe: Tomasz Kociumaka Lecturer: Marcin Pilipczuk 1 Introduction to graph coloring Definition 1. Let G be a simple undirected graph and k a positive
More informationBBM402-Lecture 20: LP Duality
BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1
More informationOptimization Theory. A Concise Introduction. Jiongmin Yong
October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationJOHN THICKSTUN. p x. n sup Ipp y n x np x nq. By the memoryless and stationary conditions respectively, this reduces to just 1 yi x i.
ESTIMATING THE SHANNON CAPACITY OF A GRAPH JOHN THICKSTUN. channels and graphs Consider a stationary, memoryless channel that maps elements of discrete alphabets X to Y according to a distribution p y
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationIndex coding with side information
Index coding with side information Ehsan Ebrahimi Targhi University of Tartu Abstract. The Index Coding problem has attracted a considerable amount of attention in the recent years. The problem is motivated
More informationLecture 8: Semidefinite programs for fidelity and optimal measurements
CS 766/QIC 80 Theory of Quantum Information (Fall 0) Lecture 8: Semidefinite programs for fidelity and optimal measurements This lecture is devoted to two examples of semidefinite programs: one is for
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More information1 Positive definiteness and semidefiniteness
Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationSandwich Theorem and Calculation of the Theta Function for Several Graphs
Brigham Young University BYU ScholarsArchive All Theses and Dissertations 2003-03-7 Sandwich Theorem and Calculation of the Theta Function for Several Graphs Marcia Ling Riddle Brigham Young University
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More information6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationNORMS ON SPACE OF MATRICES
NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationModeling with semidefinite and copositive matrices
Modeling with semidefinite and copositive matrices Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria F. Rendl, Singapore workshop 2006 p.1/24 Overview Node and Edge relaxations
More informationApproximation Algorithms
Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationLINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM
LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM Unless otherwise stated, all vector spaces in this worksheet are finite dimensional and the scalar field F is R or C. Definition 1. A linear operator
More informationLecture 4: January 26
10-725/36-725: Conve Optimization Spring 2015 Lecturer: Javier Pena Lecture 4: January 26 Scribes: Vipul Singh, Shinjini Kundu, Chia-Yin Tsai Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationACO Comprehensive Exam March 17 and 18, Computability, Complexity and Algorithms
1. Computability, Complexity and Algorithms (a) Let G(V, E) be an undirected unweighted graph. Let C V be a vertex cover of G. Argue that V \ C is an independent set of G. (b) Minimum cardinality vertex
More informationThe Simplex Algorithm
8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.
More informationResearch Reports on Mathematical and Computing Sciences
ISSN 1342-284 Research Reports on Mathematical and Computing Sciences Exploiting Sparsity in Linear and Nonlinear Matrix Inequalities via Positive Semidefinite Matrix Completion Sunyoung Kim, Masakazu
More informationMIT Algebraic techniques and semidefinite optimization February 14, Lecture 3
MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications
More informationLNMB PhD Course. Networks and Semidefinite Programming 2012/2013
LNMB PhD Course Networks and Semidefinite Programming 2012/2013 Monique Laurent CWI, Amsterdam, and Tilburg University These notes are based on material developed by M. Laurent and F. Vallentin for the
More informationAlgebraic Methods in Combinatorics
Algebraic Methods in Combinatorics Po-Shen Loh 27 June 2008 1 Warm-up 1. (A result of Bourbaki on finite geometries, from Răzvan) Let X be a finite set, and let F be a family of distinct proper subsets
More informationCS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2)
CS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2) Tim Roughgarden February 2, 2016 1 Recap This is our third lecture on linear programming, and the second on linear programming
More informationMathematical Methods wk 2: Linear Operators
John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm
More informationRecitation 8: Graphs and Adjacency Matrices
Math 1b TA: Padraic Bartlett Recitation 8: Graphs and Adjacency Matrices Week 8 Caltech 2011 1 Random Question Suppose you take a large triangle XY Z, and divide it up with straight line segments into
More informationIntroduction to Group Theory
Chapter 10 Introduction to Group Theory Since symmetries described by groups play such an important role in modern physics, we will take a little time to introduce the basic structure (as seen by a physicist)
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 11 Luca Trevisan February 29, 2016
U.C. Berkeley CS294: Spectral Methods and Expanders Handout Luca Trevisan February 29, 206 Lecture : ARV In which we introduce semi-definite programming and a semi-definite programming relaxation of sparsest
More informationAlgebraic Methods in Combinatorics
Algebraic Methods in Combinatorics Po-Shen Loh June 2009 1 Linear independence These problems both appeared in a course of Benny Sudakov at Princeton, but the links to Olympiad problems are due to Yufei
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationA lower bound for the Laplacian eigenvalues of a graph proof of a conjecture by Guo
A lower bound for the Laplacian eigenvalues of a graph proof of a conjecture by Guo A. E. Brouwer & W. H. Haemers 2008-02-28 Abstract We show that if µ j is the j-th largest Laplacian eigenvalue, and d
More information5 Flows and cuts in digraphs
5 Flows and cuts in digraphs Recall that a digraph or network is a pair G = (V, E) where V is a set and E is a multiset of ordered pairs of elements of V, which we refer to as arcs. Note that two vertices
More informationVector Spaces, Orthogonality, and Linear Least Squares
Week Vector Spaces, Orthogonality, and Linear Least Squares. Opening Remarks.. Visualizing Planes, Lines, and Solutions Consider the following system of linear equations from the opener for Week 9: χ χ
More information10. Smooth Varieties. 82 Andreas Gathmann
82 Andreas Gathmann 10. Smooth Varieties Let a be a point on a variety X. In the last chapter we have introduced the tangent cone C a X as a way to study X locally around a (see Construction 9.20). It
More informationChapter 6 Orthogonal representations II: Minimal dimension
Chapter 6 Orthogonal representations II: Minimal dimension Nachdiplomvorlesung by László Lovász ETH Zürich, Spring 2014 1 Minimum dimension Perhaps the most natural way to be economic in constructing an
More informationDuke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014
Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document
More informationReducing graph coloring to stable set without symmetry
Reducing graph coloring to stable set without symmetry Denis Cornaz (with V. Jost, with P. Meurdesoif) LAMSADE, Paris-Dauphine SPOC 11 Cornaz (with Jost and Meurdesoif) Coloring without symmetry SPOC 11
More informationThe Hilbert Space of Random Variables
The Hilbert Space of Random Variables Electrical Engineering 126 (UC Berkeley) Spring 2018 1 Outline Fix a probability space and consider the set H := {X : X is a real-valued random variable with E[X 2
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.85J / 8.5J Advanced Algorithms Fall 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 8.5/6.85 Advanced Algorithms
More informationCSC Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method Notes taken by Stefan Mathe April 28, 2007 Summary: Throughout the course, we have seen the importance
More informationLecture Notes 1: Vector spaces
Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationSDP Relaxations for MAXCUT
SDP Relaxations for MAXCUT from Random Hyperplanes to Sum-of-Squares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationThe structure of bull-free graphs I three-edge-paths with centers and anticenters
The structure of bull-free graphs I three-edge-paths with centers and anticenters Maria Chudnovsky Columbia University, New York, NY 10027 USA May 6, 2006; revised March 29, 2011 Abstract The bull is the
More informationSemidefinite Programming
Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has
More informationMAT 445/ INTRODUCTION TO REPRESENTATION THEORY
MAT 445/1196 - INTRODUCTION TO REPRESENTATION THEORY CHAPTER 1 Representation Theory of Groups - Algebraic Foundations 1.1 Basic definitions, Schur s Lemma 1.2 Tensor products 1.3 Unitary representations
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationContents Real Vector Spaces Linear Equations and Linear Inequalities Polyhedra Linear Programs and the Simplex Method Lagrangian Duality
Contents Introduction v Chapter 1. Real Vector Spaces 1 1.1. Linear and Affine Spaces 1 1.2. Maps and Matrices 4 1.3. Inner Products and Norms 7 1.4. Continuous and Differentiable Functions 11 Chapter
More informationAn introductory example
CS1 Lecture 9 An introductory example Suppose that a company that produces three products wishes to decide the level of production of each so as to maximize profits. Let x 1 be the amount of Product 1
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationChannel Coding: Zero-error case
Channel Coding: Zero-error case Information & Communication Sander Bet & Ismani Nieuweboer February 05 Preface We would like to thank Christian Schaffner for guiding us in the right direction with our
More informationCopositive Plus Matrices
Copositive Plus Matrices Willemieke van Vliet Master Thesis in Applied Mathematics October 2011 Copositive Plus Matrices Summary In this report we discuss the set of copositive plus matrices and their
More informationTopics in Graph Theory
Topics in Graph Theory September 4, 2018 1 Preliminaries A graph is a system G = (V, E) consisting of a set V of vertices and a set E (disjoint from V ) of edges, together with an incidence function End
More informationLecture 14: Random Walks, Local Graph Clustering, Linear Programming
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 14: Random Walks, Local Graph Clustering, Linear Programming Lecturer: Shayan Oveis Gharan 3/01/17 Scribe: Laura Vonessen Disclaimer: These
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More information1 Strict local optimality in unconstrained optimization
ORF 53 Lecture 14 Spring 016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, April 14, 016 When in doubt on the accuracy of these notes, please cross check with the instructor s
More informationReconstruction and Higher Dimensional Geometry
Reconstruction and Higher Dimensional Geometry Hongyu He Department of Mathematics Louisiana State University email: hongyu@math.lsu.edu Abstract Tutte proved that, if two graphs, both with more than two
More informationGrothendieck s Inequality
Grothendieck s Inequality Leqi Zhu 1 Introduction Let A = (A ij ) R m n be an m n matrix. Then A defines a linear operator between normed spaces (R m, p ) and (R n, q ), for 1 p, q. The (p q)-norm of A
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationConic approach to quantum graph parameters using linear optimization over the completely positive semidefinite cone
Conic approach to quantum graph parameters using linear optimization over the completely positive semidefinite cone Monique Laurent 1,2 and Teresa Piovesan 1 1 Centrum Wiskunde & Informatica (CWI), Amsterdam,
More informationMulti-coloring and Mycielski s construction
Multi-coloring and Mycielski s construction Tim Meagher Fall 2010 Abstract We consider a number of related results taken from two papers one by W. Lin [1], and the other D. C. Fisher[2]. These articles
More informationON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION
Annales Univ. Sci. Budapest., Sect. Comp. 33 (2010) 273-284 ON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION L. László (Budapest, Hungary) Dedicated to Professor Ferenc Schipp on his 70th
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationLecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016
Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,
More informationHamming codes and simplex codes ( )
Chapter 6 Hamming codes and simplex codes (2018-03-17) Synopsis. Hamming codes are essentially the first non-trivial family of codes that we shall meet. We start by proving the Distance Theorem for linear
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More information1 T 1 = where 1 is the all-ones vector. For the upper bound, let v 1 be the eigenvector corresponding. u:(u,v) E v 1(u)
CME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh (rezab@stanford.edu) Final Review Session 03/20/17 1. Let G = (V, E) be an unweighted, undirected graph. Let λ 1 be the maximum eigenvalue
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationCS264: Beyond Worst-Case Analysis Lecture #11: LP Decoding
CS264: Beyond Worst-Case Analysis Lecture #11: LP Decoding Tim Roughgarden October 29, 2014 1 Preamble This lecture covers our final subtopic within the exact and approximate recovery part of the course.
More informationThe Hamming Codes and Delsarte s Linear Programming Bound
The Hamming Codes and Delsarte s Linear Programming Bound by Sky McKinley Under the Astute Tutelage of Professor John S. Caughman, IV A thesis submitted in partial fulfillment of the requirements for the
More informationTangent spaces, normals and extrema
Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent
More informationNONCOMMUTATIVE POLYNOMIAL EQUATIONS. Edward S. Letzter. Introduction
NONCOMMUTATIVE POLYNOMIAL EQUATIONS Edward S Letzter Introduction My aim in these notes is twofold: First, to briefly review some linear algebra Second, to provide you with some new tools and techniques
More informationIRREDUCIBLE REPRESENTATIONS OF SEMISIMPLE LIE ALGEBRAS. Contents
IRREDUCIBLE REPRESENTATIONS OF SEMISIMPLE LIE ALGEBRAS NEEL PATEL Abstract. The goal of this paper is to study the irreducible representations of semisimple Lie algebras. We will begin by considering two
More informationCSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017
CSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017 Linear Function f: R n R is linear if it can be written as f x = a T x for some a R n Example: f x 1, x 2 =
More information