Lecture 5. Theorems of Alternatives and SelfDual Embedding


 Alice Watkins
 1 years ago
 Views:
Transcription
1 IE Lecture 5. Theorems of Alternatives and SelfDual Embedding
2 IE A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c T y = 1 has a solution (but not both nor neither). Such type of statement is called theorem of alternatives. There are in fact quite a few such theorems of alternatives, in the context of linear algebra, as we will discuss shortly.
3 IE A very general form of theorem of alternatives is Hilbert s Nullstellensatz. Let P i (x) be a polynomial of n variables x 1,..., x n in the complex field (x C n ), i = 1,..., m. Theorem 1 (Nullstellensatz) Either P 1 (x) = 0, P 2 (x) = 0,..., P m (x) = 0 has a solution x C n, or there exist polynomials Q 1 (x),..., Q m (x), such that has a solution. P 1 (x)q 1 (x) + P 2 (x)q 2 (x) + + P m (x)q m (x) = 1 For more details, visit Terence Tao s blog at:
4 IE For the system of linear inequalities, the most famous theorem of alternative is the following Farkas lemma (shown by Farkas around the end of the 19th century). Theorem 2 (Farkas) Either Ax = b, x 0 has a solution, or A T y 0, b T y < 0 has a solution. In fact, some more refined forms of the theorems of alternatives exist, in the style of the Farkas lemma. Some examples are shown in the next few slides.
5 IE Let x and y be two vectors. Denote x y to be x y and x y ; similarly for x y. Theorem 3 (Gordan) Either Ax > 0 has a solution, or A T y = 0, y 0 has a solution. Theorem 4 (Stiemke) Either Ax 0 has a solution, or A T y = 0, y > 0 has a solution.
6 IE Theorem 5 (Gale) Either Ax b has a solution, or A T y = 0, b T y = 1, y 0 has a solution. Theorem 6 (Tucker) Suppose A 0. Either Ax 0, Bx 0, Cx = 0 has a solution, or A T u + B T v + C T w = 0, u > 0, v 0 has a solution.
7 IE Theorem 7 (Motzkin) Suppose A 0. Either Ax > 0, Bx 0, Cx = 0 has a solution, or A T u + B T v + C T w = 0, u 0, v 0 has a solution.
8 IE The most important theorem of alternatives is still the Farkas lemma. The Farkas lemma can be proven either by the simplex method for linear programming, or it can be proven by the separation theorem. Interestingly, the Farkas lemma can also be proven in an elementary manner, due to a recent brilliant expository article by Terence Tao. Let us slightly rephrase the Farkas lemma: Let P i (x) = n j=1 p ijx j r i, i = 1, 2,..., m. If the system P i (x) 0, i = 1, 2,..., m, does not have a solution, then there are y i 0 such that m i=1 y ip i (x) = 1. We prove the result by induction on n. When n = 1, we can scale P i (x) 0 possibly into three parts: x 1 a i 0, i I + ; x 1 + b j 0, j I ; c k 0, k I 0. If there is k I 0 with c k < 0 then we may let y k = 1/c k and y l = 0 for all l k. Otherwise there is i I + and j I with a i > b j. We let y i = y j = 1/(a i b j ) and 0 elsewhere, yielding m i=1 y ip i (x) = 1.
9 IE Now suppose that the result is proven for the dimension (of x) no more than n. Consider the case when we have n + 1 variables: x = (x; x n+1 ), where x R n. Depending on the coefficients of x n+1 we scale the original inequalities to get the following equivalent system of inequalities: P i ( x) = x n+1 Q i (x) 0, i I + ; P j ( x) = x n+1 + Q j (x) 0, j I ; P k ( x) = Q k (x) 0, k I 0. If Q i (x) + Q j (x) 0, i I + and j I ; Q k (x) 0, k I 0 has a solution then the original system would have a solution too.
10 IE By the condition of the lemma, therefore, it cannot have a solution. Now using the induction hypothesis, there are y ij, y k 0 such that y ij ( Q i (x) + Q j (x)) + y k Q k (x) = 1. i I + ;j I k I 0 Hence, i I + (x n+1 Q i (x)) + y ij j I j I i I + y ij ( x n+1 + Q j (x)) + k I 0 y k Q k (x) = 1. The Farkas lemma is thus proven by this simple induction argument.
11 IE In fact, the strict separation theorem of convex polytopes follows from the Farkas lemma. The separation theorem that we are talking about here is: If polytopes P 1 and P 2 do not intersect then there is an affine linear function f(x) = y, x + y 0 such that f(x) 1 for x P 1, and f(x) 1 for x P 2. Let the extreme points of P 1 be {p 1,..., p s }, and the extreme points of P 2 be {q 1,..., q t }. Then f(x) 1 for x P 1, and f(x) 1 for x P 2 y, p i + y 0 1, for i = 1,..., s; and y, q j + y 0 1, for j = 1,..., t.
12 IE Let us now use a contradiction argument. Suppose the above does not have a solution in y and y 0. Then the Farkas lemma asserts that there are u i 0 (i = 1,..., s) and v j 0 (j = 1,..., t), such that s u i ( y, p i + y 0 1) + i=1 t v j ( y, q j y 0 1) = 1. j=1 This implies that s u i p i t v j q j = 0, s u i t v j = 0, and s u i + t v j = 1. i=1 j=1 i=1 j=1 i=1 j=1 Therefore s i=1 u i = t j=1 v j = 1/2. Moreover, P 1 2 s u i p i = 2 i=1 t v j q j P 2, j=1 which contradicts to the assumption that P 1 P 2 =. The same arguments will work if polytopes are replaced by polydra.
13 IE Let us return to conic optimization models. To study the geometric structure of the problem, introduce the linear subspace L := {x Ax = 0}, and its orthogonal complement L = {A T y y R m }. Let a = A T (AA T ) 1 b, and so a + L = {x Ax = b, x 0}. In this notation, the generic conic optimization problem can be written as minimize c T x subject to x (a + L) K. (1)
14 IE Geometrically, the conic optimization is nothing but linear optimization over the intersection of an affine linear subspace a + L and a given closed convex cone K. Such problems can be either feasible or infeasible; having unbounded optimal value or finite optimal value; attainable optimal solution exists or not. We shall see first that the situation for a generic conic optimization gets more complicated than for linear programming.
15 IE Let us consider the case of infeasibility. It is well known that for linear programming the state of being infeasibility is rather stable, in the sense that if the data (a, c) corresponds to an infeasible problem instance, then a small perturbation on (a, c) will remain the infeasibility status. Why? Consider the following equivalence due to the Farkas lemma: {x R n Ax = b, x R n +} = {y R m A T y R n +, b T y < 0} =.
16 IE For conic optimization, however, this no longer holds. Consider Example 1 minimize x 22 subject to x 11 = b 1, x 12 = b 2 x 11, x 12, 0. x 21, x 22 For (b 1, b 2 ) = (0, 1) the problem is infeasible. However, for (b 1, b 2 ) = (ϵ, 1) where ϵ is any positive number, the problem is always feasible.
17 IE Definition 1 We call the conic optimization problem: weakly infeasible (a + L) K = but dist(a + L, K) = 0; strongly infeasible dist(a + L, K) > 0; weakly feasible (a + L) K = but (a + L) int K = ; strongly feasible (a + L) int K =.
18 IE Theorem 8 (Alternatives of strong infeasibility). {x R n Ax = b, x K} is not strongly infeasible {y R m A T y K, b T y < 0} =. Proof. Note that for s L, i.e. s = A T y for some y R m, we have a T s = a T A T y = b T y. Observe the following chain of equivalent statements: {y R m A T y K, b T y < 0} = for all s L K it must follow a T s 0 a (L K ) = cl (K + L) x i K, x i L such that a (x i + x i ) = (a x i ) x i 0 (a + L) K is not strongly infeasible. The theorem is proven.
19 IE Definition 2 We call d P an primal improving direction if d P L K, and c T d P < 0; d i P an primal improving direction sequence if d i P K, dist(d i P, L) 0 and lim sup i c T d i P < 0; d D a dual improve direction if d D L K and a T d D < 0; d i D a dual improve direction sequence if and lim sup i a T d i D < 0. d i D K, dist(d i D, L ) 0
20 IE If a problem is not strongly infeasible, then it can be either feasible or weakly infeasible. Next question is: can we further characterize the feasibility of a conic problem? Naturally if it is possible, then by exclusion we will also have characterized the weak feasibility. Theorem 9 (Alternatives of primal feasibility). {x R n Ax = b, x K} is infeasible if and only if there is a dual improving direction sequence.
21 IE Proof. Let L = span (H ({x Ax = b})) = x 0 x x 0b + Ax = 0 and K = R + K. We have L = s 0 s y Rm, s 0 = b T y, s = A T y and K = R + K. Let c = 1 0 n.
22 IE Theorem 8 states that the primal problem is strongly infeasible is equivalent to the existence of a dual improve direction. To state it in the dual way, Theorem 8 can also be written as ( c + L ) K is strongly infeasible x L K, c T x < 0. (2) It is easy to see that {x Ax = b, x K} is feasible x L K, c T x < 0 (by (2)) ( c + L ) K is strongly infeasible.
23 IE Therefore, {x Ax = b, x K} is infeasible ( c + L ) K is not strongly infeasible s i 0 0, s i K such that dist si 0, c + L 0 s i K, dist (s i, L ) 0 and s i lim sup i a T s i < 0 a dual improve direction sequence.
24 IE As a consequence, by exclusion, this implies the following result: Corollary 1 {x R n Ax = b, x K} is weakly infeasible there is a dual improving direction sequence but no dual improving direction. So far we have been concerned with characterizing infeasible conic programs. How about feasible ones? Recall that we have two types of feasibilities: the strong feasibility (also known as to satisfy the Slater condition), and the weak feasibility, which means that it is feasible but the Slater condition is not satisfied. It is helpful to derive an equivalent condition for the Slater condition, using the dual information.
25 IE Lemma 1 (a + L) int K = if and only if for any d 0 and d L K it must follow a T d > 0. Proof. Note that x int K 0 y K it follows x T y > 0. = : Let x (a + L) int K. In particular, write x = a + z with z L. Take any d 0 and d L K : 0 < x T d = (a + z) T d = a T d. = : If (a + L) int K =, then a + L and int K can be separated by a hyperplane. That is, there is s 0 such that s T x 0 for all x K and s T (a + y) 0 for all y L. The last inequality means that s T a 0 and s T y = 0 for all y L. Thus, we have found a nonzero vector s L K with a T s 0, which contradicts the assumption.
26 IE Theorem 10 If the primal conic program satisfies the Slater condition, and its dual is feasible, then the dual has a nonempty and compact optimal solution set. Proof. Clearly, the objective value of the dual problem is bounded below by the weak duality theorem. The only possibility for the dual problem not to possess an optimal solution is that the sup is nonattainable; i.e. there is an infinite sequence s i = c A T y i (a + L ) K such that a T s i = a T c b T y i a T c optimal value of the dual problem, and s i. Without loss of generality, suppose that s i / s i converges to d. Then we have d L K and a T d = 0. This contradicts with the primal Slater condition, in light of Lemma 1. Hence the optimal solutions must be bounded.
27 IE Therefore, if a pair of primaldual conic programs satisfy the Slater condition, then attainable optimal solutions exist for both problems, according to Theorem 10. The only remaining issue in this regard is: how about the strong duality theorem? In other words, do these optimal solutions satisfy the complementarity slackness conditions? The answer is yes. Theorem 11 If the primal conic program and its dual conic program both satisfy the Slater condition, then the optimal solution sets for both problems are nonempty and compact. Moreover, the optimal solutions are complementary to each other with zero duality gap.
28 IE Proof. The first part of the theorem is simply Theorem 10, stated both for the primal and for the dual parts separately. We now focus on the second part. Let x and s be optimal solutions for teh primal and the dual conic optimization problems respectively. Write the primal problem in the parametric form: (P (a)) minimize c T x subject to x a + L x K. Let the optimal value of (P (a)), as a function of the vector a, be v(a). Clearly v(a) is a convex function in its domain, D, whenever (P (a)) is feasible. Obviously, D K. Also we see that v(0) = 0 due to Theorem 8.
29 IE By the Slater condition we know that a int D. Let g be a subgradient for v at a; that is for all z D. v(z) v(a) + g T (z a) (3) Observe that for any d L we have a + L = a + d + L. Thus, v(a + d) = v(a) for any d L. This implies g L. Take any u K. Since x + u is a feasible solution for (P (a + u)), it follows that v(a + u) c T (x + u) = v(a) + c T u. Together with the subgradient inequality (3) this yields (c g) T u 0 for any u K; hence c g K. Thus, c g is a dual feasible solution, and so a T s a T (c g).
30 IE Taking z = 0, the subgradient inequality gives us 0 = v(0) v(a) + g T (0 a) = c T x g T a. (4) Combining with (4) we have a T c c T x a T s 0. Hence, 0 = (x a) T (s c) = (x ) T s + a T c c T x a T s (x ) T s 0. Thus, (x ) T s = 0, and the complementarity slackness condition is satisfied.
31 IE So far we have discussed the relationship between a duality pair of conic optimization problems (P ) minimize c T x subject to Ax = b x K, and (D) maximize b T y subject to A T y + s = c s K.
32 IE In case K = K = R n + the linear programming case the following facts are well known: If (P ) and (D) both are feasible, then both problems are solvable, with attainable optimal solutions; It is possible that (P ) is feasible and unbounded in its objective. In that case, (D) is infeasible. Symmetrically, if (D) is feasible and unbounded, then (P ) is infeasible. It is possible that both (P ) and (D) are infeasible.
33 IE This can be summarized by the following table, where the row is the status of the primal problem and the column represents the status of the dual problem: Feasible Unbounded Infeasible Feasible (P ), (D) solvable Unbounded Infeasible
34 IE For general conic programs, assuming (D) is feasible, then the relationship with (P ) is as follows: Primal Dual s. f. bd opt sol set (Theorem 10) w. f. no improving direction sequence (Theorem 9) w. inf. improving dir seq but no improving dir (Corollary 1) strongly infeasible improving direction (Theorem 8)
35 IE Example 2 with the dual minimize x 12 + x 21 subject to x 12 + x 21 + x 33 = 1, x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 0, x 31, x 32, x 33 maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, 0 0, 0, y 1 0. The primal problem is weakly feasible, while the dual is weakly infeasible.
36 IE Example 3 with the dual minimize x 12 + x 21 subject to x 12 + x 21 + x 23 + x 32 = 1 x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 x 31, x 32, x 33 0, maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, y 1 0, y 1, 0 0. Both the primal and the dual are weakly infeasible.
37 IE Example 4 with the dual minimize x 12 + x 21 x 33 subject to x 12 + x 21 + x 23 + x 32 = 1, x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 0, x 31, x 32, x 33 maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, y 1 0, y 1, 1 0. In this case, the primal is weakly infeasible, while the dual is strongly infeasible.
38 IE Example 5 with the dual minimize x 12 + x 21 x 33 subject to x 12 + x 21 + x 23 + x 32 = 1, x 22 = 1 x 11, x 12, x 13 x 21, x 22, x 23 0, x 31, x 32, x 33 maximize y 1 y 2 0, 1 y 1, 0 subject to 1 y 1, y 2, y 1 0, y 1, 1 0. In this case, both the primal and the dual are strongly infeasible.
39 IE Consider the standard conic optimization problems (P ) and (D). Take any x 0 int K, s 0 int K, and y 0 R m. Moreover, define r p = b Ax 0, r d = s 0 c + A T y 0, and r g = 1 + c T x 0 b T y 0. The following model is selfdual: (SD) min βθ s.t. Ax bτ +r p θ = 0 A T y +cτ +r d θ s = 0 b T y c T x +r g θ κ = 0 r T p y r T d x r gτ = β x K, τ 0, s K, κ 0, where β = 1 + (x 0 ) T s 0 > 1, and the decision variables are (y, x, τ, θ, s, κ).
40 IE The above selfdual model admits an interior feasible solution (y, x, τ, θ, s, κ) = (y 0, x 0, 1, 1, s 0, 1). Proposition 1 The problem (SD) has a maximally complementary optimal solution, denoted by (y, x, τ, θ, s, κ ), such that θ = 0 and (x ) T s + τ κ = 0. Moreover, if τ > 0, then x /τ is an optimal solution for the primal problem, and (y /τ, s /τ ) is an optimal solution for the dual problem. If κ > 0 then either c T x < 0 or b T y > 0; in the former case the dual problem is strongly infeasible, and in the latter case the primal problem is strongly infeasible.
41 IE If strict complementarity fails, then one can only conclude that the original problem does not have complementary optimal solutions. In that case, one may wish to identify whether the problems are weakly feasible or weakly infeasible. However, the weak infeasibility is in a slippery terrace; it cannot be certified by a finite proof. The type of certificate that one can expect is in the spirit of if the problem is ever feasible, then any of its feasible solution must have a very large norm.
42 IE For linear programming, the situation is particularly interesting. Remember we have shown before that if a primaldual LP pair is strongly feasible, then the primaldual central path solutions converge to a strictly complementary optimal solutions. That is, Ax(µ) = b A T y(µ) + s(µ) = c x i (µ)s i (µ) = µ, i = 1, 2,..., n and x := lim µ 0 x(µ) and s := lim µ 0 s(µ) are strictly complementary.
43 IE When we apply this to (SD), we have Ax bτ = 0 A T y + cτ s = 0 b T y c T x κ = 0 with x 0, s 0, τ 0, κ 0 and x + s > 0, τ + κ > 0.
44 IE This constructively proves a well known result of Goldman and Tucker: Theorem 12 (Goldman and Tucker, 1956). Suppose that M is skewsymmetric. Then the following system always has a solution: Mx s = 0 x 0, s 0, x + s > 0. In fact, the theorem of Goldman and Tucker can also be used to prove the previous result: they are equivalent.
45 IE Key References: T. Tao, Structure and Randomness, American Mathematical Society, Z.Q. Luo, J.F. Sturm and S. Zhang, Conic Convex Programming and SelfDual Embedding, Optimization Methods and Software, 14, , J.F. Sturm, Theory and Algorithms for Semidefinite Programming. In High Performance Optimization, eds. H. Frenk, K. Roos, T. Terlaky, and S. Zhang, Kluwer Academic Publishers, Y. Ye, M.J. Todd, and S. Mizuno, An O( nl)iteration Homogeneous and SelfDual Linear Programming Algorithm, Mathematics of Operations Research, 19, 53 67, 1994.
Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 00 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationA fullnewton step infeasible interiorpoint algorithm for linear programming based on a kernel function
A fullnewton step infeasible interiorpoint algorithm for linear programming based on a kernel function Zhongyi Liu, Wenyu Sun Abstract This paper proposes an infeasible interiorpoint algorithm with
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 151 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationApproximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.32.5, 3.13.4) 1 Geometry of Linear
More informationA Parametric Simplex Algorithm for Linear Vector Optimization Problems
A Parametric Simplex Algorithm for Linear Vector Optimization Problems Birgit Rudloff Firdevs Ulus Robert Vanderbei July 9, 2015 Abstract In this paper, a parametric simplex algorithm for solving linear
More informationInput: System of inequalities or equalities over the reals R. Output: Value for variables that minimizes cost function
Linear programming Input: System of inequalities or equalities over the reals R A linear cost function Output: Value for variables that minimizes cost function Example: Minimize 6x+4y Subject to 3x + 2y
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of WisconsinMadison May 2014 Wright (UWMadison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationCSCI5654 (Linear Programming, Fall 2013) Lecture8. Lecture 8 Slide# 1
CSCI5654 (Linear Programming, Fall 2013) Lecture8 Lecture 8 Slide# 1 Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables.
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework  MC/MC Constrained optimization
More informationIntroduction to optimization
Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)
More informationDetecting Infeasibility in InfeasibleInteriorPoint Methods for Optimization
Detecting Infeasibility in InfeasibleInteriorPoint Methods for Optimization M. J. Todd January 16, 2003 Abstract We study interiorpoint methods for optimization problems in the case of infeasibility
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationThe KarushKuhnTucker (KKT) conditions
The KarushKuhnTucker (KKT) conditions In this section, we will give a set of sufficient (and at most times necessary) conditions for a x to be the solution of a given convex optimization problem. These
More information Wellcharacterized problems, minmax relations, approximate certificates.  LP problems in the standard form, primal and dual linear programs
LPDuality ( Approximation Algorithms by V. Vazirani, Chapter 12)  Wellcharacterized problems, minmax relations, approximate certificates  LP problems in the standard form, primal and dual linear programs
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationThe fundamental theorem of linear programming
The fundamental theorem of linear programming Michael Tehranchi June 8, 2017 This note supplements the lecture notes of Optimisation The statement of the fundamental theorem of linear programming and the
More informationMATINF4110/MATINF9110 Mathematical optimization
MATINF4110/MATINF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More informationChap6 Duality Theory and Sensitivity Analysis
Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we
More informationConvex Sets. Prof. Dan A. Simovici UMB
Convex Sets Prof. Dan A. Simovici UMB 1 / 57 Outline 1 Closures, Interiors, Borders of Sets in R n 2 Segments and Convex Sets 3 Properties of the Class of Convex Sets 4 Closure and Interior Points of Convex
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationOptimization Theory. A Concise Introduction. Jiongmin Yong
October 11, 017 16:5 wsbook9x6 Book Title Optimization Theory 01708Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 01708Lecture Notes page Optimization
More informationA Generalized Homogeneous and SelfDual Algorithm. for Linear Programming. February 1994 (revised December 1994)
A Generalized Homogeneous and SelfDual Algorithm for Linear Programming Xiaojie Xu Yinyu Ye y February 994 (revised December 994) Abstract: A generalized homogeneous and selfdual (HSD) infeasibleinteriorpoint
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationLecture 11: PostOptimal Analysis. September 23, 2009
Lecture : PostOptimal Analysis September 23, 2009 Today Lecture DualSimplex Algorithm PostOptimal Analysis Chapters 4.4 and 4.5. IE 30/GE 330 Lecture Dual Simplex Method The dual simplex method will
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationA Strongly Polynomial Simplex Method for Totally Unimodular LP
A Strongly Polynomial Simplex Method for Totally Unimodular LP Shinji Mizuno July 19, 2014 Abstract Kitahara and Mizuno get new bounds for the number of distinct solutions generated by the simplex method
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a realvalued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationInterior Point Methods for LP
11.1 Interior Point Methods for LP Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor, Winter 1997. Simplex Method  A Boundary Method: Starting at an extreme point of the feasible set, the simplex
More informationDuality Theory, Optimality Conditions
5.1 Duality Theory, Optimality Conditions Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor We only consider single objective LPs here. Concept of duality not defined for multiobjective LPs. Every
More informationAlternative Linear Inequalities
Division of the Humanities and Social Sciences Alternative Linear Inequalities KC Border September 2013 v 20170701::1105 Contents 1 Executive Summary 1 11 Solutions of systems of equalities 2 12 Nonnegative
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1
More informationDuality in Linear Programming
Duality in Linear Programming Gary D. Knott Civilized Software Inc. 1219 Heritage Park Circle Silver Spring MD 296 phone:319623711 email:knott@civilized.com URL:www.civilized.com May 1, 213.1 Duality
More informationNotes taken by Graham Taylor. January 22, 2005
CSC4  Linear Programming and Combinatorial Optimization Lecture : Different forms of LP. The algebraic objects behind LP. Basic Feasible Solutions Notes taken by Graham Taylor January, 5 Summary: We first
More informationA strongly polynomial algorithm for linear systems having a binary solution
A strongly polynomial algorithm for linear systems having a binary solution Sergei Chubanov Institute of Information Systems at the University of Siegen, Germany email: sergei.chubanov@unisiegen.de 7th
More informationConvex Functions. Pontus Giselsson
Convex Functions Pontus Giselsson 1 Today s lecture lower semicontinuity, closure, convex hull convexity preserving operations precomposition with affine mapping infimal convolution image function supremum
More informationProblem 1 (Exercise 2.2, Monograph)
MS&E 314/CME 336 Assignment 2 Conic Linear Programming January 3, 215 Prof. Yinyu Ye 6 Pages ASSIGNMENT 2 SOLUTIONS Problem 1 (Exercise 2.2, Monograph) We prove the part ii) of Theorem 2.1 (Farkas Lemma
More informationLecture 17: Primaldual interiorpoint methods part II
10725/36725: Convex Optimization Spring 2015 Lecture 17: Primaldual interiorpoint methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationCS 6820 Fall 2014 Lectures, October 320, 2014
Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 320, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29092012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More informationLecture 1: Background on Convex Analysis
Lecture 1: Background on Convex Analysis John Duchi PCMI 2016 Outline I Convex sets 1.1 Definitions and examples 2.2 Basic properties 3.3 Projections onto convex sets 4.4 Separating and supporting hyperplanes
More informationREGULAR LAGRANGE MULTIPLIERS FOR CONTROL PROBLEMS WITH MIXED POINTWISE CONTROLSTATE CONSTRAINTS
REGULAR LAGRANGE MULTIPLIERS FOR CONTROL PROBLEMS WITH MIXED POINTWISE CONTROLSTATE CONSTRAINTS fredi tröltzsch 1 Abstract. A class of quadratic optimization problems in Hilbert spaces is considered,
More informationOn the Value Function of a Mixed Integer Linear Optimization Problem and an Algorithm for its Construction
On the Value Function of a Mixed Integer Linear Optimization Problem and an Algorithm for its Construction Ted K. Ralphs and Anahita Hassanzadeh Department of Industrial and Systems Engineering, Lehigh
More informationLocally convex spaces, the hyperplane separation theorem, and the KreinMilman theorem
56 Chapter 7 Locally convex spaces, the hyperplane separation theorem, and the KreinMilman theorem Recall that C(X) is not a normed linear space when X is not compact. On the other hand we could use semi
More informationSummary of the simplex method
MVE165/MMG630, The simplex method; degeneracy; unbounded solutions; infeasibility; starting solutions; duality; interpretation AnnBrith Strömberg 2012 03 16 Summary of the simplex method Optimality condition:
More informationVariational Inequalities. Anna Nagurney Isenberg School of Management University of Massachusetts Amherst, MA 01003
Variational Inequalities Anna Nagurney Isenberg School of Management University of Massachusetts Amherst, MA 01003 c 2002 Background Equilibrium is a central concept in numerous disciplines including economics,
More informationNormal Fans of Polyhedral Convex Sets
SetValued Analysis manuscript No. (will be inserted by the editor) Normal Fans of Polyhedral Convex Sets Structures and Connections Shu Lu Stephen M. Robinson Received: date / Accepted: date Dedicated
More informationDuality of LPs and Applications
Lecture 6 Duality of LPs and Applications Last lecture we introduced duality of linear programs. We saw how to form duals, and proved both the weak and strong duality theorems. In this lecture we will
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2 Instructor: Farid Alizadeh Scribe: Xuan Li 9/17/2001 1 Overview We survey the basic notions of cones and conelp and give several
More informationLecture Notes 3: Duality
Algorithmic Methods 1/11/21 Professor: Yossi Azar Lecture Notes 3: Duality Scribe:Moran BarGat 1 Introduction In this lecture we will present the dual concept, Farkas s Lema and their relation to the
More informationStrong Duality: Without Simplex and without theorems of alternatives. Somdeb Lahiri SPM, PDPU. November 29, 2015.
Strong Duality: Without Simplex and without theorems of alternatives By Somdeb Lahiri SPM, PDPU. email: somdeb.lahiri@yahoo.co.in November 29, 2015. Abstract We provide an alternative proof of the strong
More informationConvex Feasibility Problems
Laureate Prof. Jonathan Borwein with Matthew Tam http://carma.newcastle.edu.au/drmethods/paseky.html Spring School on Variational Analysis VI Paseky nad Jizerou, April 19 25, 2015 Last Revised: May 6,
More information9.1 Linear Programs in canonical form
9.1 Linear Programs in canonical form LP in standard form: max (LP) s.t. where b i R, i = 1,..., m z = j c jx j j a ijx j b i i = 1,..., m x j 0 j = 1,..., n But the Simplex method works only on systems
More informationSolving Dual Problems
Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More information6. Linear Programming
Linear Programming 61 6. Linear Programming Linear Programming LP reduction Duality Maxflow mincut, Zerosum game Integer Programming and LP relaxation Maximum Bipartite Matching, Minimum weight vertex
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationSensitivity Analysis and Duality
Sensitivity Analysis and Duality Part II Duality Based on Chapter 6 Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan
More information3 Development of the Simplex Method Constructing Basic Solution Optimality Conditions The Simplex Method...
Contents Introduction to Linear Programming Problem. 2. General Linear Programming problems.............. 2.2 Formulation of LP problems.................... 8.3 Compact form and Standard form of a general
More informationInteger Programming ISE 418. Lecture 12. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 12 Dr. Ted Ralphs ISE 418 Lecture 12 1 Reading for This Lecture Nemhauser and Wolsey Sections II.2.1 Wolsey Chapter 9 ISE 418 Lecture 12 2 Generating Stronger Valid
More informationIMPLEMENTING THE NEW SELFREGULAR PROXIMITY BASED IPMS
IMPLEMENTING THE NEW SELFREGULAR PROXIMITY BASED IPMS IMPLEMENTING THE NEW SELFREGULAR PROXIMITY BASED IPMS By Xiaohang Zhu A thesis submitted to the School of Graduate Studies in Partial Fulfillment
More informationThe Strong Duality Theorem 1
1/39 The Strong Duality Theorem 1 Adrian Vetta 1 This presentation is based upon the book Linear Programming by Vasek Chvatal 2/39 Part I Weak Duality 3/39 Primal and Dual Recall we have a primal linear
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms  A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationPROBLEMS. (b) (Polarization Identity) Show that in any inner product space
1 Professor Carl Cowen Math 54600 Fall 09 PROBLEMS 1. (Geometry in Inner Product Spaces) (a) (Parallelogram Law) Show that in any inner product space x + y 2 + x y 2 = 2( x 2 + y 2 ). (b) (Polarization
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationThe Q Method for Symmetric Cone Programmin
The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra
More informationThe smonotone Index Selection Rule for CrissCross Algorithms of Linear Complementarity Problems
The smonotone Index Selection Rule for CrissCross Algorithms of Linear Complementarity Problems Zsolt Csizmadia and Tibor Illés and Adrienn Nagy February 24, 213 Abstract In this paper we introduce the
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of WisconsinMadison. IMA, August 2016 Stephen Wright (UWMadison) Linear Programming: Simplex IMA, August 2016
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uniklu.ac.at AlpenAdriaUniversität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA
More informationLinear programming: theory, algorithms and applications
Linear programming: theory, algorithms and applications illes@math.bme.hu Department of Differential Equations Budapest 2014/2015 Fall Vector spaces A nonempty set L equipped with addition and multiplication
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis
MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis AnnBrith Strömberg 2017 03 29 Lecture 4 Linear and integer optimization with
More information16 1 Basic Facts from Functional Analysis and Banach Lattices
16 1 Basic Facts from Functional Analysis and Banach Lattices 1.2.3 Banach Steinhaus Theorem Another fundamental theorem of functional analysis is the Banach Steinhaus theorem, or the Uniform Boundedness
More informationInteger Programming Duality
Integer Programming Duality M. Guzelsoy T. K. Ralphs July, 2010 1 Introduction This article describes what is known about duality for integer programs. It is perhaps surprising that many of the results
More informationLecture 5. 1 GoermansWilliamson Algorithm for the maxcut problem
Math 280 Geometric and Algebraic Ideas in Optimization April 26, 2010 Lecture 5 Lecturer: Jesús A De Loera Scribe: HuyDung Han, Fabio Lapiccirella 1 GoermansWilliamson Algorithm for the maxcut problem
More information6. Duals of L p spaces
6 Duals of L p spaces This section deals with the problem if identifying the duals of L p spaces, p [1, ) There are essentially two cases of this problem: (i) p = 1; (ii) 1 < p < The major difference between
More informationRankone LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the wellknown Lyapunov's matrix inequality about
Rankone LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the wellknown Lyapunov's matrix inequality about the location of the eigenvalues of a matrix
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationExtreme points of compact convex sets
Extreme points of compact convex sets In this chapter, we are going to show that compact convex sets are determined by a proper subset, the set of its extreme points. Let us start with the main definition.
More informationLecture 1 Introduction
L. Vandenberghe EE236A (Fall 201314) Lecture 1 Introduction course overview linear optimization examples history approximate syllabus basic definitions linear optimization in vector and matrix notation
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set LöwnerJohn ellipsoid
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationLecture 5  Hausdorff and GromovHausdorff Distance
Lecture 5  Hausdorff and GromovHausdorff Distance August 1, 2011 1 Definition and Basic Properties Given a metric space X, the set of closed sets of X supports a metric, the Hausdorff metric. If A is
More information1 Maximal Latticefree Convex Sets
47831: Advanced Integer Programming Lecturer: Amitabh Basu Lecture 3 Date: 03/23/2010 In this lecture, we explore the connections between lattices of R n and convex sets in R n. The structures will prove
More informationAlgorithmic Game Theory and Applications. Lecture 5: Introduction to Linear Programming
Algorithmic Game Theory and Applications Lecture 5: Introduction to Linear Programming Kousha Etessami real world example : the diet problem You are a fastidious eater. You want to make sure that every
More informationAn introduction to some aspects of functional analysis
An introduction to some aspects of functional analysis Stephen Semmes Rice University Abstract These informal notes deal with some very basic objects in functional analysis, including norms and seminorms
More informationLinear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence
Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n
More information4. The Dual Simplex Method
4. The Dual Simplex Method Javier Larrosa Albert Oliveras Enric RodríguezCarbonell Problem Solving and Constraint Programming (RPAR) Session 4 p.1/34 Basic Idea (1) Algorithm as explained so far known
More informationCLOSURES OF QUADRATIC MODULES
CLOSURES OF QUADRATIC MODULES JAKA CIMPRIČ, MURRAY MARSHALL, TIM NETZER Abstract. We consider the problem of determining the closure M of a quadratic module M in a commutative Ralgebra with respect to
More informationAuerbach bases and minimal volume sufficient enlargements
Auerbach bases and minimal volume sufficient enlargements M. I. Ostrovskii January, 2009 Abstract. Let B Y denote the unit ball of a normed linear space Y. A symmetric, bounded, closed, convex set A in
More informationChapter 1: Linear Programming
Chapter 1: Linear Programming Math 368 c Copyright 2013 R Clark Robinson May 22, 2013 Chapter 1: Linear Programming 1 Max and Min For f : D R n R, f (D) = {f (x) : x D } is set of attainable values of
More informationExamples of Dual Spaces from Measure Theory
Chapter 9 Examples of Dual Spaces from Measure Theory We have seen that L (, A, µ) is a Banach space for any measure space (, A, µ). We will extend that concept in the following section to identify an
More informationOn the Closedness of the Linear Image of a Closed Convex Cone
MATHEMATICS OF OPERATIONS RESEARCH Vol., No., Xxxxxx 2xx, pp. xxx xxx ISSN 364765X EISSN 15265471 xx xxx DOI 1.1287/moor.xxxx.xxxx c 2xx INFORMS On the Closedness of the Linear Image of a Closed Convex
More informationConvex Optimization and an Introduction to Congestion Control. Lecture Notes. Fabian Wirth
Convex Optimization and an Introduction to Congestion Control Lecture Notes Fabian Wirth August 29, 2012 ii Contents 1 Convex Sets and Convex Functions 3 1.1 Convex Sets....................................
More information