# Lecture 5. Theorems of Alternatives and Self-Dual Embedding

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 IE Lecture 5. Theorems of Alternatives and Self-Dual Embedding

2 IE A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c T y = 1 has a solution (but not both nor neither). Such type of statement is called theorem of alternatives. There are in fact quite a few such theorems of alternatives, in the context of linear algebra, as we will discuss shortly.

3 IE A very general form of theorem of alternatives is Hilbert s Nullstellensatz. Let P i (x) be a polynomial of n variables x 1,..., x n in the complex field (x C n ), i = 1,..., m. Theorem 1 (Nullstellensatz) Either P 1 (x) = 0, P 2 (x) = 0,..., P m (x) = 0 has a solution x C n, or there exist polynomials Q 1 (x),..., Q m (x), such that has a solution. P 1 (x)q 1 (x) + P 2 (x)q 2 (x) + + P m (x)q m (x) = 1 For more details, visit Terence Tao s blog at:

4 IE For the system of linear inequalities, the most famous theorem of alternative is the following Farkas lemma (shown by Farkas around the end of the 19th century). Theorem 2 (Farkas) Either Ax = b, x 0 has a solution, or A T y 0, b T y < 0 has a solution. In fact, some more refined forms of the theorems of alternatives exist, in the style of the Farkas lemma. Some examples are shown in the next few slides.

5 IE Let x and y be two vectors. Denote x y to be x y and x y ; similarly for x y. Theorem 3 (Gordan) Either Ax > 0 has a solution, or A T y = 0, y 0 has a solution. Theorem 4 (Stiemke) Either Ax 0 has a solution, or A T y = 0, y > 0 has a solution.

6 IE Theorem 5 (Gale) Either Ax b has a solution, or A T y = 0, b T y = 1, y 0 has a solution. Theorem 6 (Tucker) Suppose A 0. Either Ax 0, Bx 0, Cx = 0 has a solution, or A T u + B T v + C T w = 0, u > 0, v 0 has a solution.

7 IE Theorem 7 (Motzkin) Suppose A 0. Either Ax > 0, Bx 0, Cx = 0 has a solution, or A T u + B T v + C T w = 0, u 0, v 0 has a solution.

8 IE The most important theorem of alternatives is still the Farkas lemma. The Farkas lemma can be proven either by the simplex method for linear programming, or it can be proven by the separation theorem. Interestingly, the Farkas lemma can also be proven in an elementary manner, due to a recent brilliant expository article by Terence Tao. Let us slightly rephrase the Farkas lemma: Let P i (x) = n j=1 p ijx j r i, i = 1, 2,..., m. If the system P i (x) 0, i = 1, 2,..., m, does not have a solution, then there are y i 0 such that m i=1 y ip i (x) = 1. We prove the result by induction on n. When n = 1, we can scale P i (x) 0 possibly into three parts: x 1 a i 0, i I + ; x 1 + b j 0, j I ; c k 0, k I 0. If there is k I 0 with c k < 0 then we may let y k = 1/c k and y l = 0 for all l k. Otherwise there is i I + and j I with a i > b j. We let y i = y j = 1/(a i b j ) and 0 elsewhere, yielding m i=1 y ip i (x) = 1.

9 IE Now suppose that the result is proven for the dimension (of x) no more than n. Consider the case when we have n + 1 variables: x = (x; x n+1 ), where x R n. Depending on the coefficients of x n+1 we scale the original inequalities to get the following equivalent system of inequalities: P i ( x) = x n+1 Q i (x) 0, i I + ; P j ( x) = x n+1 + Q j (x) 0, j I ; P k ( x) = Q k (x) 0, k I 0. If Q i (x) + Q j (x) 0, i I + and j I ; Q k (x) 0, k I 0 has a solution then the original system would have a solution too.

10 IE By the condition of the lemma, therefore, it cannot have a solution. Now using the induction hypothesis, there are y ij, y k 0 such that y ij ( Q i (x) + Q j (x)) + y k Q k (x) = 1. i I + ;j I k I 0 Hence, i I + (x n+1 Q i (x)) + y ij j I j I i I + y ij ( x n+1 + Q j (x)) + k I 0 y k Q k (x) = 1. The Farkas lemma is thus proven by this simple induction argument.

11 IE In fact, the strict separation theorem of convex polytopes follows from the Farkas lemma. The separation theorem that we are talking about here is: If polytopes P 1 and P 2 do not intersect then there is an affine linear function f(x) = y, x + y 0 such that f(x) 1 for x P 1, and f(x) 1 for x P 2. Let the extreme points of P 1 be {p 1,..., p s }, and the extreme points of P 2 be {q 1,..., q t }. Then f(x) 1 for x P 1, and f(x) 1 for x P 2 y, p i + y 0 1, for i = 1,..., s; and y, q j + y 0 1, for j = 1,..., t.

12 IE Let us now use a contradiction argument. Suppose the above does not have a solution in y and y 0. Then the Farkas lemma asserts that there are u i 0 (i = 1,..., s) and v j 0 (j = 1,..., t), such that s u i ( y, p i + y 0 1) + i=1 t v j ( y, q j y 0 1) = 1. j=1 This implies that s u i p i t v j q j = 0, s u i t v j = 0, and s u i + t v j = 1. i=1 j=1 i=1 j=1 i=1 j=1 Therefore s i=1 u i = t j=1 v j = 1/2. Moreover, P 1 2 s u i p i = 2 i=1 t v j q j P 2, j=1 which contradicts to the assumption that P 1 P 2 =. The same arguments will work if polytopes are replaced by polydra.

13 IE Let us return to conic optimization models. To study the geometric structure of the problem, introduce the linear subspace L := {x Ax = 0}, and its orthogonal complement L = {A T y y R m }. Let a = A T (AA T ) 1 b, and so a + L = {x Ax = b, x 0}. In this notation, the generic conic optimization problem can be written as minimize c T x subject to x (a + L) K. (1)

14 IE Geometrically, the conic optimization is nothing but linear optimization over the intersection of an affine linear subspace a + L and a given closed convex cone K. Such problems can be either feasible or infeasible; having unbounded optimal value or finite optimal value; attainable optimal solution exists or not. We shall see first that the situation for a generic conic optimization gets more complicated than for linear programming.

15 IE Let us consider the case of infeasibility. It is well known that for linear programming the state of being infeasibility is rather stable, in the sense that if the data (a, c) corresponds to an infeasible problem instance, then a small perturbation on (a, c) will remain the infeasibility status. Why? Consider the following equivalence due to the Farkas lemma: {x R n Ax = b, x R n +} = {y R m A T y R n +, b T y < 0} =.

16 IE For conic optimization, however, this no longer holds. Consider Example 1 minimize x 22 subject to x 11 = b 1, x 12 = b 2 x 11, x 12, 0. x 21, x 22 For (b 1, b 2 ) = (0, 1) the problem is infeasible. However, for (b 1, b 2 ) = (ϵ, 1) where ϵ is any positive number, the problem is always feasible.

17 IE Definition 1 We call the conic optimization problem: weakly infeasible (a + L) K = but dist(a + L, K) = 0; strongly infeasible dist(a + L, K) > 0; weakly feasible (a + L) K = but (a + L) int K = ; strongly feasible (a + L) int K =.

18 IE Theorem 8 (Alternatives of strong infeasibility). {x R n Ax = b, x K} is not strongly infeasible {y R m A T y K, b T y < 0} =. Proof. Note that for s L, i.e. s = A T y for some y R m, we have a T s = a T A T y = b T y. Observe the following chain of equivalent statements: {y R m A T y K, b T y < 0} = for all s L K it must follow a T s 0 a (L K ) = cl (K + L) x i K, x i L such that a (x i + x i ) = (a x i ) x i 0 (a + L) K is not strongly infeasible. The theorem is proven.

19 IE Definition 2 We call d P an primal improving direction if d P L K, and c T d P < 0; d i P an primal improving direction sequence if d i P K, dist(d i P, L) 0 and lim sup i c T d i P < 0; d D a dual improve direction if d D L K and a T d D < 0; d i D a dual improve direction sequence if and lim sup i a T d i D < 0. d i D K, dist(d i D, L ) 0

20 IE If a problem is not strongly infeasible, then it can be either feasible or weakly infeasible. Next question is: can we further characterize the feasibility of a conic problem? Naturally if it is possible, then by exclusion we will also have characterized the weak feasibility. Theorem 9 (Alternatives of primal feasibility). {x R n Ax = b, x K} is infeasible if and only if there is a dual improving direction sequence.

21 IE Proof. Let L = span (H ({x Ax = b})) = x 0 x x 0b + Ax = 0 and K = R + K. We have L = s 0 s y Rm, s 0 = b T y, s = A T y and K = R + K. Let c = 1 0 n.

22 IE Theorem 8 states that the primal problem is strongly infeasible is equivalent to the existence of a dual improve direction. To state it in the dual way, Theorem 8 can also be written as ( c + L ) K is strongly infeasible x L K, c T x < 0. (2) It is easy to see that {x Ax = b, x K} is feasible x L K, c T x < 0 (by (2)) ( c + L ) K is strongly infeasible.

23 IE Therefore, {x Ax = b, x K} is infeasible ( c + L ) K is not strongly infeasible s i 0 0, s i K such that dist si 0, c + L 0 s i K, dist (s i, L ) 0 and s i lim sup i a T s i < 0 a dual improve direction sequence.

24 IE As a consequence, by exclusion, this implies the following result: Corollary 1 {x R n Ax = b, x K} is weakly infeasible there is a dual improving direction sequence but no dual improving direction. So far we have been concerned with characterizing infeasible conic programs. How about feasible ones? Recall that we have two types of feasibilities: the strong feasibility (also known as to satisfy the Slater condition), and the weak feasibility, which means that it is feasible but the Slater condition is not satisfied. It is helpful to derive an equivalent condition for the Slater condition, using the dual information.

25 IE Lemma 1 (a + L) int K = if and only if for any d 0 and d L K it must follow a T d > 0. Proof. Note that x int K 0 y K it follows x T y > 0. = : Let x (a + L) int K. In particular, write x = a + z with z L. Take any d 0 and d L K : 0 < x T d = (a + z) T d = a T d. = : If (a + L) int K =, then a + L and int K can be separated by a hyperplane. That is, there is s 0 such that s T x 0 for all x K and s T (a + y) 0 for all y L. The last inequality means that s T a 0 and s T y = 0 for all y L. Thus, we have found a non-zero vector s L K with a T s 0, which contradicts the assumption.

26 IE Theorem 10 If the primal conic program satisfies the Slater condition, and its dual is feasible, then the dual has a non-empty and compact optimal solution set. Proof. Clearly, the objective value of the dual problem is bounded below by the weak duality theorem. The only possibility for the dual problem not to possess an optimal solution is that the sup is non-attainable; i.e. there is an infinite sequence s i = c A T y i (a + L ) K such that a T s i = a T c b T y i a T c optimal value of the dual problem, and s i. Without loss of generality, suppose that s i / s i converges to d. Then we have d L K and a T d = 0. This contradicts with the primal Slater condition, in light of Lemma 1. Hence the optimal solutions must be bounded.

27 IE Therefore, if a pair of primal-dual conic programs satisfy the Slater condition, then attainable optimal solutions exist for both problems, according to Theorem 10. The only remaining issue in this regard is: how about the strong duality theorem? In other words, do these optimal solutions satisfy the complementarity slackness conditions? The answer is yes. Theorem 11 If the primal conic program and its dual conic program both satisfy the Slater condition, then the optimal solution sets for both problems are non-empty and compact. Moreover, the optimal solutions are complementary to each other with zero duality gap.

28 IE Proof. The first part of the theorem is simply Theorem 10, stated both for the primal and for the dual parts separately. We now focus on the second part. Let x and s be optimal solutions for teh primal and the dual conic optimization problems respectively. Write the primal problem in the parametric form: (P (a)) minimize c T x subject to x a + L x K. Let the optimal value of (P (a)), as a function of the vector a, be v(a). Clearly v(a) is a convex function in its domain, D, whenever (P (a)) is feasible. Obviously, D K. Also we see that v(0) = 0 due to Theorem 8.

29 IE By the Slater condition we know that a int D. Let g be a subgradient for v at a; that is for all z D. v(z) v(a) + g T (z a) (3) Observe that for any d L we have a + L = a + d + L. Thus, v(a + d) = v(a) for any d L. This implies g L. Take any u K. Since x + u is a feasible solution for (P (a + u)), it follows that v(a + u) c T (x + u) = v(a) + c T u. Together with the subgradient inequality (3) this yields (c g) T u 0 for any u K; hence c g K. Thus, c g is a dual feasible solution, and so a T s a T (c g).

30 IE Taking z = 0, the subgradient inequality gives us 0 = v(0) v(a) + g T (0 a) = c T x g T a. (4) Combining with (4) we have a T c c T x a T s 0. Hence, 0 = (x a) T (s c) = (x ) T s + a T c c T x a T s (x ) T s 0. Thus, (x ) T s = 0, and the complementarity slackness condition is satisfied.

31 IE So far we have discussed the relationship between a duality pair of conic optimization problems (P ) minimize c T x subject to Ax = b x K, and (D) maximize b T y subject to A T y + s = c s K.

32 IE In case K = K = R n + the linear programming case the following facts are well known: If (P ) and (D) both are feasible, then both problems are solvable, with attainable optimal solutions; It is possible that (P ) is feasible and unbounded in its objective. In that case, (D) is infeasible. Symmetrically, if (D) is feasible and unbounded, then (P ) is infeasible. It is possible that both (P ) and (D) are infeasible.

33 IE This can be summarized by the following table, where the row is the status of the primal problem and the column represents the status of the dual problem: Feasible Unbounded Infeasible Feasible (P ), (D) solvable Unbounded Infeasible

34 IE For general conic programs, assuming (D) is feasible, then the relationship with (P ) is as follows: Primal Dual s. f. bd opt sol set (Theorem 10) w. f. no improving direction sequence (Theorem 9) w. inf. improving dir seq but no improving dir (Corollary 1) strongly infeasible improving direction (Theorem 8)

35 IE Example 2 with the dual minimize x 12 + x 21 subject to x 12 + x 21 + x 33 = 1, x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 0, x 31, x 32, x 33 maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, 0 0, 0, y 1 0. The primal problem is weakly feasible, while the dual is weakly infeasible.

36 IE Example 3 with the dual minimize x 12 + x 21 subject to x 12 + x 21 + x 23 + x 32 = 1 x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 x 31, x 32, x 33 0, maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, y 1 0, y 1, 0 0. Both the primal and the dual are weakly infeasible.

37 IE Example 4 with the dual minimize x 12 + x 21 x 33 subject to x 12 + x 21 + x 23 + x 32 = 1, x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 0, x 31, x 32, x 33 maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, y 1 0, y 1, 1 0. In this case, the primal is weakly infeasible, while the dual is strongly infeasible.

38 IE Example 5 with the dual minimize x 12 + x 21 x 33 subject to x 12 + x 21 + x 23 + x 32 = 1, x 22 = 1 x 11, x 12, x 13 x 21, x 22, x 23 0, x 31, x 32, x 33 maximize y 1 y 2 0, 1 y 1, 0 subject to 1 y 1, y 2, y 1 0, y 1, 1 0. In this case, both the primal and the dual are strongly infeasible.

39 IE Consider the standard conic optimization problems (P ) and (D). Take any x 0 int K, s 0 int K, and y 0 R m. Moreover, define r p = b Ax 0, r d = s 0 c + A T y 0, and r g = 1 + c T x 0 b T y 0. The following model is self-dual: (SD) min βθ s.t. Ax bτ +r p θ = 0 A T y +cτ +r d θ s = 0 b T y c T x +r g θ κ = 0 r T p y r T d x r gτ = β x K, τ 0, s K, κ 0, where β = 1 + (x 0 ) T s 0 > 1, and the decision variables are (y, x, τ, θ, s, κ).

40 IE The above self-dual model admits an interior feasible solution (y, x, τ, θ, s, κ) = (y 0, x 0, 1, 1, s 0, 1). Proposition 1 The problem (SD) has a maximally complementary optimal solution, denoted by (y, x, τ, θ, s, κ ), such that θ = 0 and (x ) T s + τ κ = 0. Moreover, if τ > 0, then x /τ is an optimal solution for the primal problem, and (y /τ, s /τ ) is an optimal solution for the dual problem. If κ > 0 then either c T x < 0 or b T y > 0; in the former case the dual problem is strongly infeasible, and in the latter case the primal problem is strongly infeasible.

41 IE If strict complementarity fails, then one can only conclude that the original problem does not have complementary optimal solutions. In that case, one may wish to identify whether the problems are weakly feasible or weakly infeasible. However, the weak infeasibility is in a slippery terrace; it cannot be certified by a finite proof. The type of certificate that one can expect is in the spirit of if the problem is ever feasible, then any of its feasible solution must have a very large norm.

42 IE For linear programming, the situation is particularly interesting. Remember we have shown before that if a primal-dual LP pair is strongly feasible, then the primal-dual central path solutions converge to a strictly complementary optimal solutions. That is, Ax(µ) = b A T y(µ) + s(µ) = c x i (µ)s i (µ) = µ, i = 1, 2,..., n and x := lim µ 0 x(µ) and s := lim µ 0 s(µ) are strictly complementary.

43 IE When we apply this to (SD), we have Ax bτ = 0 A T y + cτ s = 0 b T y c T x κ = 0 with x 0, s 0, τ 0, κ 0 and x + s > 0, τ + κ > 0.

44 IE This constructively proves a well known result of Goldman and Tucker: Theorem 12 (Goldman and Tucker, 1956). Suppose that M is skew-symmetric. Then the following system always has a solution: Mx s = 0 x 0, s 0, x + s > 0. In fact, the theorem of Goldman and Tucker can also be used to prove the previous result: they are equivalent.

45 IE Key References: T. Tao, Structure and Randomness, American Mathematical Society, Z.Q. Luo, J.F. Sturm and S. Zhang, Conic Convex Programming and Self-Dual Embedding, Optimization Methods and Software, 14, , J.F. Sturm, Theory and Algorithms for Semidefinite Programming. In High Performance Optimization, eds. H. Frenk, K. Roos, T. Terlaky, and S. Zhang, Kluwer Academic Publishers, Y. Ye, M.J. Todd, and S. Mizuno, An O( nl)-iteration Homogeneous and Self-Dual Linear Programming Algorithm, Mathematics of Operations Research, 19, 53 67, 1994.

### Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

### Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem

Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R

### A full-newton step infeasible interior-point algorithm for linear programming based on a kernel function

A full-newton step infeasible interior-point algorithm for linear programming based on a kernel function Zhongyi Liu, Wenyu Sun Abstract This paper proposes an infeasible interior-point algorithm with

### 15. Conic optimization

L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone

### Approximate Farkas Lemmas in Convex Optimization

Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The

### Convex Optimization and Modeling

Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual

### Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method

The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear

### A Parametric Simplex Algorithm for Linear Vector Optimization Problems

A Parametric Simplex Algorithm for Linear Vector Optimization Problems Birgit Rudloff Firdevs Ulus Robert Vanderbei July 9, 2015 Abstract In this paper, a parametric simplex algorithm for solving linear

### Input: System of inequalities or equalities over the reals R. Output: Value for variables that minimizes cost function

Linear programming Input: System of inequalities or equalities over the reals R A linear cost function Output: Value for variables that minimizes cost function Example: Minimize 6x+4y Subject to 3x + 2y

### Optimality, Duality, Complementarity for Constrained Optimization

Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear

### CSCI5654 (Linear Programming, Fall 2013) Lecture-8. Lecture 8 Slide# 1

CSCI5654 (Linear Programming, Fall 2013) Lecture-8 Lecture 8 Slide# 1 Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables.

### LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization

### Introduction to optimization

Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)

### Detecting Infeasibility in Infeasible-Interior-Point Methods for Optimization

Detecting Infeasibility in Infeasible-Interior-Point Methods for Optimization M. J. Todd January 16, 2003 Abstract We study interior-point methods for optimization problems in the case of infeasibility

### Lecture 6: Conic Optimization September 8

IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

### The Karush-Kuhn-Tucker (KKT) conditions

The Karush-Kuhn-Tucker (KKT) conditions In this section, we will give a set of sufficient (and at most times necessary) conditions for a x to be the solution of a given convex optimization problem. These

### - Well-characterized problems, min-max relations, approximate certificates. - LP problems in the standard form, primal and dual linear programs

LP-Duality ( Approximation Algorithms by V. Vazirani, Chapter 12) - Well-characterized problems, min-max relations, approximate certificates - LP problems in the standard form, primal and dual linear programs

### 14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.

CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity

### Assignment 1: From the Definition of Convexity to Helley Theorem

Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x

### The fundamental theorem of linear programming

The fundamental theorem of linear programming Michael Tehranchi June 8, 2017 This note supplements the lecture notes of Optimisation The statement of the fundamental theorem of linear programming and the

### MAT-INF4110/MAT-INF9110 Mathematical optimization

MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:

### Chap6 Duality Theory and Sensitivity Analysis

Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we

### Convex Sets. Prof. Dan A. Simovici UMB

Convex Sets Prof. Dan A. Simovici UMB 1 / 57 Outline 1 Closures, Interiors, Borders of Sets in R n 2 Segments and Convex Sets 3 Properties of the Class of Convex Sets 4 Closure and Interior Points of Convex

### LP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra

LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality

### Optimization Theory. A Concise Introduction. Jiongmin Yong

October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization

### A Generalized Homogeneous and Self-Dual Algorithm. for Linear Programming. February 1994 (revised December 1994)

A Generalized Homogeneous and Self-Dual Algorithm for Linear Programming Xiaojie Xu Yinyu Ye y February 994 (revised December 994) Abstract: A generalized homogeneous and self-dual (HSD) infeasible-interior-point

### A Review of Linear Programming

A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex

### Conic Linear Programming. Yinyu Ye

Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture

### Lecture 11: Post-Optimal Analysis. September 23, 2009

Lecture : Post-Optimal Analysis September 23, 2009 Today Lecture Dual-Simplex Algorithm Post-Optimal Analysis Chapters 4.4 and 4.5. IE 30/GE 330 Lecture Dual Simplex Method The dual simplex method will

### Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5

Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize

### A Strongly Polynomial Simplex Method for Totally Unimodular LP

A Strongly Polynomial Simplex Method for Totally Unimodular LP Shinji Mizuno July 19, 2014 Abstract Kitahara and Mizuno get new bounds for the number of distinct solutions generated by the simplex method

### HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.

HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard

### Interior Point Methods for LP

11.1 Interior Point Methods for LP Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor, Winter 1997. Simplex Method - A Boundary Method: Starting at an extreme point of the feasible set, the simplex

### Duality Theory, Optimality Conditions

5.1 Duality Theory, Optimality Conditions Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor We only consider single objective LPs here. Concept of duality not defined for multiobjective LPs. Every

### Alternative Linear Inequalities

Division of the Humanities and Social Sciences Alternative Linear Inequalities KC Border September 2013 v 20170701::1105 Contents 1 Executive Summary 1 11 Solutions of systems of equalities 2 12 Nonnegative

### IE 5531: Engineering Optimization I

IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1

### Duality in Linear Programming

Duality in Linear Programming Gary D. Knott Civilized Software Inc. 1219 Heritage Park Circle Silver Spring MD 296 phone:31-962-3711 email:knott@civilized.com URL:www.civilized.com May 1, 213.1 Duality

### Notes taken by Graham Taylor. January 22, 2005

CSC4 - Linear Programming and Combinatorial Optimization Lecture : Different forms of LP. The algebraic objects behind LP. Basic Feasible Solutions Notes taken by Graham Taylor January, 5 Summary: We first

### A strongly polynomial algorithm for linear systems having a binary solution

A strongly polynomial algorithm for linear systems having a binary solution Sergei Chubanov Institute of Information Systems at the University of Siegen, Germany e-mail: sergei.chubanov@uni-siegen.de 7th

### Convex Functions. Pontus Giselsson

Convex Functions Pontus Giselsson 1 Today s lecture lower semicontinuity, closure, convex hull convexity preserving operations precomposition with affine mapping infimal convolution image function supremum

### Problem 1 (Exercise 2.2, Monograph)

MS&E 314/CME 336 Assignment 2 Conic Linear Programming January 3, 215 Prof. Yinyu Ye 6 Pages ASSIGNMENT 2 SOLUTIONS Problem 1 (Exercise 2.2, Monograph) We prove the part ii) of Theorem 2.1 (Farkas Lemma

### Lecture 17: Primal-dual interior-point methods part II

10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS

### Semidefinite Programming

Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i

### CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

### Chap 2. Optimality conditions

Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1

### Optimization methods NOPT048

Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague

### Lecture 1: Background on Convex Analysis

Lecture 1: Background on Convex Analysis John Duchi PCMI 2016 Outline I Convex sets 1.1 Definitions and examples 2.2 Basic properties 3.3 Projections onto convex sets 4.4 Separating and supporting hyperplanes

### REGULAR LAGRANGE MULTIPLIERS FOR CONTROL PROBLEMS WITH MIXED POINTWISE CONTROL-STATE CONSTRAINTS

REGULAR LAGRANGE MULTIPLIERS FOR CONTROL PROBLEMS WITH MIXED POINTWISE CONTROL-STATE CONSTRAINTS fredi tröltzsch 1 Abstract. A class of quadratic optimization problems in Hilbert spaces is considered,

### On the Value Function of a Mixed Integer Linear Optimization Problem and an Algorithm for its Construction

On the Value Function of a Mixed Integer Linear Optimization Problem and an Algorithm for its Construction Ted K. Ralphs and Anahita Hassanzadeh Department of Industrial and Systems Engineering, Lehigh

### Locally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem

56 Chapter 7 Locally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem Recall that C(X) is not a normed linear space when X is not compact. On the other hand we could use semi

### Summary of the simplex method

MVE165/MMG630, The simplex method; degeneracy; unbounded solutions; infeasibility; starting solutions; duality; interpretation Ann-Brith Strömberg 2012 03 16 Summary of the simplex method Optimality condition:

### Variational Inequalities. Anna Nagurney Isenberg School of Management University of Massachusetts Amherst, MA 01003

Variational Inequalities Anna Nagurney Isenberg School of Management University of Massachusetts Amherst, MA 01003 c 2002 Background Equilibrium is a central concept in numerous disciplines including economics,

### Normal Fans of Polyhedral Convex Sets

Set-Valued Analysis manuscript No. (will be inserted by the editor) Normal Fans of Polyhedral Convex Sets Structures and Connections Shu Lu Stephen M. Robinson Received: date / Accepted: date Dedicated

### Duality of LPs and Applications

Lecture 6 Duality of LPs and Applications Last lecture we introduced duality of linear programs. We saw how to form duals, and proved both the weak and strong duality theorems. In this lecture we will

### Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2

Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2 Instructor: Farid Alizadeh Scribe: Xuan Li 9/17/2001 1 Overview We survey the basic notions of cones and cone-lp and give several

### Lecture Notes 3: Duality

Algorithmic Methods 1/11/21 Professor: Yossi Azar Lecture Notes 3: Duality Scribe:Moran Bar-Gat 1 Introduction In this lecture we will present the dual concept, Farkas s Lema and their relation to the

### Strong Duality: Without Simplex and without theorems of alternatives. Somdeb Lahiri SPM, PDPU. November 29, 2015.

Strong Duality: Without Simplex and without theorems of alternatives By Somdeb Lahiri SPM, PDPU. email: somdeb.lahiri@yahoo.co.in November 29, 2015. Abstract We provide an alternative proof of the strong

### Convex Feasibility Problems

Laureate Prof. Jonathan Borwein with Matthew Tam http://carma.newcastle.edu.au/drmethods/paseky.html Spring School on Variational Analysis VI Paseky nad Jizerou, April 19 25, 2015 Last Revised: May 6,

### 9.1 Linear Programs in canonical form

9.1 Linear Programs in canonical form LP in standard form: max (LP) s.t. where b i R, i = 1,..., m z = j c jx j j a ijx j b i i = 1,..., m x j 0 j = 1,..., n But the Simplex method works only on systems

### Solving Dual Problems

Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem

### CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.

1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function

### 6. Linear Programming

Linear Programming 6-1 6. Linear Programming Linear Programming LP reduction Duality Max-flow min-cut, Zero-sum game Integer Programming and LP relaxation Maximum Bipartite Matching, Minimum weight vertex

### Lecture #21. c T x Ax b. maximize subject to

COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the

### Sensitivity Analysis and Duality

Sensitivity Analysis and Duality Part II Duality Based on Chapter 6 Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan

### 3 Development of the Simplex Method Constructing Basic Solution Optimality Conditions The Simplex Method...

Contents Introduction to Linear Programming Problem. 2. General Linear Programming problems.............. 2.2 Formulation of LP problems.................... 8.3 Compact form and Standard form of a general

### Integer Programming ISE 418. Lecture 12. Dr. Ted Ralphs

Integer Programming ISE 418 Lecture 12 Dr. Ted Ralphs ISE 418 Lecture 12 1 Reading for This Lecture Nemhauser and Wolsey Sections II.2.1 Wolsey Chapter 9 ISE 418 Lecture 12 2 Generating Stronger Valid

### IMPLEMENTING THE NEW SELF-REGULAR PROXIMITY BASED IPMS

IMPLEMENTING THE NEW SELF-REGULAR PROXIMITY BASED IPMS IMPLEMENTING THE NEW SELF-REGULAR PROXIMITY BASED IPMS By Xiaohang Zhu A thesis submitted to the School of Graduate Studies in Partial Fulfillment

### The Strong Duality Theorem 1

1/39 The Strong Duality Theorem 1 Adrian Vetta 1 This presentation is based upon the book Linear Programming by Vasek Chvatal 2/39 Part I Weak Duality 3/39 Primal and Dual Recall we have a primal linear

### Optimization for Machine Learning

Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html

### Advances in Convex Optimization: Theory, Algorithms, and Applications

Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne

### PROBLEMS. (b) (Polarization Identity) Show that in any inner product space

1 Professor Carl Cowen Math 54600 Fall 09 PROBLEMS 1. (Geometry in Inner Product Spaces) (a) (Parallelogram Law) Show that in any inner product space x + y 2 + x y 2 = 2( x 2 + y 2 ). (b) (Polarization

### What can be expressed via Conic Quadratic and Semidefinite Programming?

What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent

### The Q Method for Symmetric Cone Programmin

The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra

### The s-monotone Index Selection Rule for Criss-Cross Algorithms of Linear Complementarity Problems

The s-monotone Index Selection Rule for Criss-Cross Algorithms of Linear Complementarity Problems Zsolt Csizmadia and Tibor Illés and Adrienn Nagy February 24, 213 Abstract In this paper we introduce the

### Linear Programming: Simplex

Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016

### Copositive Programming and Combinatorial Optimization

Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA

### Linear programming: theory, algorithms and applications

Linear programming: theory, algorithms and applications illes@math.bme.hu Department of Differential Equations Budapest 2014/2015 Fall Vector spaces A nonempty set L equipped with addition and multiplication

### MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis

MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis Ann-Brith Strömberg 2017 03 29 Lecture 4 Linear and integer optimization with

### 16 1 Basic Facts from Functional Analysis and Banach Lattices

16 1 Basic Facts from Functional Analysis and Banach Lattices 1.2.3 Banach Steinhaus Theorem Another fundamental theorem of functional analysis is the Banach Steinhaus theorem, or the Uniform Boundedness

### Integer Programming Duality

Integer Programming Duality M. Guzelsoy T. K. Ralphs July, 2010 1 Introduction This article describes what is known about duality for integer programs. It is perhaps surprising that many of the results

### Lecture 5. 1 Goermans-Williamson Algorithm for the maxcut problem

Math 280 Geometric and Algebraic Ideas in Optimization April 26, 2010 Lecture 5 Lecturer: Jesús A De Loera Scribe: Huy-Dung Han, Fabio Lapiccirella 1 Goermans-Williamson Algorithm for the maxcut problem

### 6. Duals of L p spaces

6 Duals of L p spaces This section deals with the problem if identifying the duals of L p spaces, p [1, ) There are essentially two cases of this problem: (i) p = 1; (ii) 1 < p < The major difference between

### Rank-one LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the well-known Lyapunov's matrix inequality about

Rank-one LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the well-known Lyapunov's matrix inequality about the location of the eigenvalues of a matrix

### Symmetric Matrices and Eigendecomposition

Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions

### Extreme points of compact convex sets

Extreme points of compact convex sets In this chapter, we are going to show that compact convex sets are determined by a proper subset, the set of its extreme points. Let us start with the main definition.

### Lecture 1 Introduction

L. Vandenberghe EE236A (Fall 2013-14) Lecture 1 Introduction course overview linear optimization examples history approximate syllabus basic definitions linear optimization in vector and matrix notation

### 8. Geometric problems

8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid

### 10 Numerical methods for constrained problems

10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside

### Lecture 5 - Hausdorff and Gromov-Hausdorff Distance

Lecture 5 - Hausdorff and Gromov-Hausdorff Distance August 1, 2011 1 Definition and Basic Properties Given a metric space X, the set of closed sets of X supports a metric, the Hausdorff metric. If A is

### 1 Maximal Lattice-free Convex Sets

47-831: Advanced Integer Programming Lecturer: Amitabh Basu Lecture 3 Date: 03/23/2010 In this lecture, we explore the connections between lattices of R n and convex sets in R n. The structures will prove

### Algorithmic Game Theory and Applications. Lecture 5: Introduction to Linear Programming

Algorithmic Game Theory and Applications Lecture 5: Introduction to Linear Programming Kousha Etessami real world example : the diet problem You are a fastidious eater. You want to make sure that every

### An introduction to some aspects of functional analysis

An introduction to some aspects of functional analysis Stephen Semmes Rice University Abstract These informal notes deal with some very basic objects in functional analysis, including norms and seminorms

### Linear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence

Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n

### 4. The Dual Simplex Method

4. The Dual Simplex Method Javier Larrosa Albert Oliveras Enric Rodríguez-Carbonell Problem Solving and Constraint Programming (RPAR) Session 4 p.1/34 Basic Idea (1) Algorithm as explained so far known

CLOSURES OF QUADRATIC MODULES JAKA CIMPRIČ, MURRAY MARSHALL, TIM NETZER Abstract. We consider the problem of determining the closure M of a quadratic module M in a commutative R-algebra with respect to

### Auerbach bases and minimal volume sufficient enlargements

Auerbach bases and minimal volume sufficient enlargements M. I. Ostrovskii January, 2009 Abstract. Let B Y denote the unit ball of a normed linear space Y. A symmetric, bounded, closed, convex set A in

### Chapter 1: Linear Programming

Chapter 1: Linear Programming Math 368 c Copyright 2013 R Clark Robinson May 22, 2013 Chapter 1: Linear Programming 1 Max and Min For f : D R n R, f (D) = {f (x) : x D } is set of attainable values of

### Examples of Dual Spaces from Measure Theory

Chapter 9 Examples of Dual Spaces from Measure Theory We have seen that L (, A, µ) is a Banach space for any measure space (, A, µ). We will extend that concept in the following section to identify an