IE 521 Convex Optimization Homework #1 Solution
|
|
- Terence Martin
- 5 years ago
- Views:
Transcription
1 IE 521 Convex Optimization Homework #1 Solution your NAME here your NetID here February 13, 2019 Instructions. Homework is due Wednesday, February 6, at 1:00pm; no late homework accepted. Please use the provided L A TEX file as a template. You can discuss with others, but please write your own solutions. 1
2 Problem 1: Norms and Dual Norms A real-valued function on R n is called a norm, denoted as, if it satisfies the three properties: (positivity): x R n, x 0; x = 0 if an only if x = 0; (homogeneity): x R n, α R: αx = α x ; (triangle inequality): x, y R n : x + y x + y. The standard norms on R n are the l p -norms (p 1): [Euclidean norm]: p = 2, x 2 := n x2 i ; [Manhattan norm]: p = 1, x 1 := n x i ; ( n ) 1/p x p := x i p [Maximum norm]: p =, x := max 1 i n x i. The dual norm of on R n is defined as x = max y R n : y 1 xt y. Exercise 1.1 (Definition) Prove that is a valid norm. (a) The positivity property is true, because we know x = The homogeneity property is true, because we know max y R n : y 1 xt y x T 0 = 0 αx = max y R n : y 1 αxt y = α max y R n : y 1 xt y = α x The triangle inequality property is true, because we know x + z = max (x + y R n : y 1 z)t y = max y R n : y 1 (xt y + z T y) Let y be the optimal solution of the above. Then, we have x + z = x T y + z T y max y R n : y 1 xt y + max y R n : y 1 zt y = x + z Exercise 1.2 (Example) Denote x p, as the dual norm to the l p -norm (p 1). Show that x p, = x q, where 1 p + 1 q = 1. Hence, 2, = 2, 1, =,, = 1. Hint: Use Hölder s inequality. 2
3 (b) For 1 < p, q <, we have x p, = Now consider the following with Hölder s inequality x T y max y R n : y xt y p 1 x i y i y p x q x q Now the remaining part is to find a y such that the equality can hold. i [1, n]. We calculate Let z i = sign(x i ) x i q 1 for all Further, we calculate Now constuct y = z p p = x i z i = z i p = x i sign(x i ) x i q 1 = p sign(x i) x i q 1 = x i q = x q q x i (q 1)p = z z p, and we have y p = 1. Then we have n x i q = x q q x i y i = x i z i = 1 z p z p x i z i = 1 x q/p q x q q = x q Therefore, x p, = x q For p = 1 and q = or p = and q = 1, they are the trivial cases. The proof is similar but make 1 and be the pairs accordingly. 3
4 Problem 2: Convex Sets Exercise 2.1 (Unit Ball) The unit ball of any norm is the set B = {x R n : x 1}. One can easily see that B is symmetric w.r.t. the origin (x B if and only x B ) and compact (closed and bounded) with nonempty interior. Show that the set B is convex. For any x, y B, and for λ [0, 1]. Then we have λx + (1 λ)y λx + (1 λ)y λ x + (1 λ) y λ + (1 λ) = 1 Therefore, λx + (1 λ)y B. Hence, B is a convex set. Exercise 2.2 (Vice Versa) Let B be convex, symmetric w.r.t. the origin, and compact with nonempty interior. Show that the following function B : x B = inf{t > 0 : x t B} is a valid norm. Moreover, its unit ball is exactly the set B. The positivity property is true, because we know x B = inf{t > 0 : x t B} 0 If x B = 0, then x tb, t > 0, so x = 0. The homogeneity property is true, because we know αx B = inf{t > 0 : αx t B} = inf{αt > 0 : αx αt B} = α inf{t > 0 : x t B} = α x B To show the triangle inequality property is true, it is sufficient to show that This is true because by definition of x B and y B, x + y x B + y B B x x B B and y y B B. Since B is a convex set, Moreover, x + y x B + y B = x B x B + y B x x B + y B x B + y B y y B B B B = {x R n : x B 1} = {x R n : inf{t > 0 : x t = {x R n : x B} B} 1} The last step come from the fact that if x B, then inf{t > 0 : x t B} > 1, which is a contradiction. Exercise 2.3 (Neighborhood of Convex Sets) Let X R n be a convex set and let ɛ > 0. The ɛ-neighborhood of the set X under norm is defined as Prove that X ɛ is a convex set. X ɛ = {x R n : inf x y ɛ}. y X 4
5 Let x and y be two arbitrary elements in X ɛ. Then we have for any ɛ > ɛ, u, v X, s.t. x u ɛ, y v ɛ. For any λ [0, 1], λu + (1 λ)v X, such that [λx + (1 λ)y] [λu + (1 λ)v] λ x u + (1 λ) y b ɛ. This implies that λx + (1 λ)y X ɛ. 5
6 Problem 3: Strong Duality Recall that in class we have shown the [Farkas Lemma] Let A R n m and b R m, exactly one of the two sets must be empty: (i) : {x R n : Ax = b, x 0} (ii) : {y R m : A T y 0, b T y > 0} Exercise 3.1 (Variant of Farkas Lemma) of the following sets must be empty: Prove the following variant of Farkas Lemma: exactly one (i) : {x R n : Ax b} (ii) : {y R m : y 0, A T y = 0, b T y < 0} We first show that system (i) is feasible system (ii) infeasible Suppose there is a x R n such that Ax b, then for any y 0, we have x T (A T y) = (Ax) T y b T y If system (ii) is feasible, then there exists y 0, A T y = 0, b T y < 0, which implies 0 = x T (Ay) b T y < 0 leading to a contradiction. Therefore, (ii) is infeasible. We now show that system (i) infeasible system (ii) feasible We can rewrite Ax b as A(x + x )+s = b, where (x +, x, s) 0. Hence, x + {x : Ax b} x : [ A A I ] x + x + x = b, x 0 s s s By Farkas Lemma, the infeasibility of this system implies that there exists y such that b T y < 0, [ A A I ] T y 0 This implies that y 0, A T y = 0, b T y < 0. Namely, system (ii) is feasible. Overall, we know exactly one of them is empty. Exercise 3.2 (Duality of Linear Program) The above lemma can be used to derive strong duality of linear programs. Consider the primal and dual pair of linear programs: min x c T x s.t. Ax = b, x 0 (P ) max y b T y s.t. A T y c (D) Assume that the problem (P ) has an optimal solution x with finite optimal value p. (a) Let y be a feasible solution to (D), show that b T y p. (b) Apply the above variant Farkas lemma to show that the following system {y : A T y c, y T b p } is nonempty, namely, there exists y that satisfies [ ] [ ] A T c b T y. p 6
7 (a) Let y be a feasible solution to (D), then A T y c. Hence, b T y = (Ax ) T y = x T (A T y) x T c = p. [ λ (b) Suppose the above system is not feasible. By the variant of Farkas Lemma, λ = 0, such λ0] that { Aλ λ 0 b = 0 λ T c λ 0 p < 0 If λ 0 > 0, then A( λ λ 0 ) = b and c T ( λ λ 0 ) < p, contradicts with p being optimal If λ 0 = 0, then Aλ = 0, λ T c < 0. then we can construct a feasible solution x = x + λ that leads to small objective, c T x = c T (x + λ) < c T x = p which contradicts with the optimality of x. Therefore, the assumption is not true, i.e. above system must be feasible. Combining (a) and (b), we can see that if problem (P) is solvable, then the dual problem (D) is also solvable, and the optimal values are the same. 7
8 Problem 4: Helley s Theorem and Duality Exercise 4.1 (Special Case of Helley s Theorem) Using linear programming duality, prove Helleys theorem for the special case when the convex sets are halfspaces. When the convex sets are halfspaces, the equivalent statement is as following. If m halfspaces in R n, m > d, do not have a point in common, then there exist some d + 1 of them that do not have a point in common either. Let s define the m halfspaces by Ax b do not have a point in common, then that means that the following linear program has no solution: max 0 s.t. Ax b By strong duality, its dual problem is either infeasible or unbounded: min b T y s.t. A T y = 0, y 0 Since we know y = 0 is feasible for the dual, then the dual must be unbounded. With respect to the simplex method, an unbounded problem means that there exists some basic feasible solution with an entering variable but no leaving variable(refer section 3.2 in introduction to Linear Optimization) Assume, without loss of generality, that y 1, y 2, y 3,..., y d are the basic variables for this basic feasible solution. As a reminder, there are d of these because the dual has d constrains. Assume that y d+1 is the entering variable for which there is no leaving variable. By increasing the value of y d+1 enough we can obtain a solution y such that: 1. y is feasible 2. b T y < 0 3. The remaining non-basic variable y d+2 = y d+3 =... = y m = 0. Now, let A T D, y D, and b D denote the restriction of A T, y, and b, respectively, to the variables y 1, y 2,..., y d+1. Because the remaining non-basic variable is zero, we havea T D y D = 0 and b T D y D < 0.(y D 0 as well) Finally, consider the intersection of d + 1 halfspaces defined by A D x b D. If there exists an x in this intersection, then x T A T D b T D x T A T Dy D b T Dy D 0 b T Dy D which is a contradiction. Therefore, the intersection of d + 1 halfspaces defined by A D x b D is empty, and so we have found a set of d + 1 of the m halfspaces that do not have a point in common. 8
1 Review of last lecture and introduction
Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first
More informationApproximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationGEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III
GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationIE 521 Convex Optimization
Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationMath 4317 : Real Analysis I Mid-Term Exam 1 25 September 2012
Instructions: Answer all of the problems. Math 4317 : Real Analysis I Mid-Term Exam 1 25 September 2012 Definitions (2 points each) 1. State the definition of a metric space. A metric space (X, d) is set
More informationWeek 3 Linear programming duality
Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the
More informationMAT-INF4110/MAT-INF9110 Mathematical optimization
MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More informationSpring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization
Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationNote 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)
Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical
More informationLecture 1: Convex Sets January 23
IE 521: Convex Optimization Instructor: Niao He Lecture 1: Convex Sets January 23 Spring 2017, UIUC Scribe: Niao He Courtesy warning: These notes do not necessarily cover everything discussed in the class.
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationA Parametric Simplex Algorithm for Linear Vector Optimization Problems
A Parametric Simplex Algorithm for Linear Vector Optimization Problems Birgit Rudloff Firdevs Ulus Robert Vanderbei July 9, 2015 Abstract In this paper, a parametric simplex algorithm for solving linear
More informationIowa State University. Instructor: Alex Roitershtein Summer Homework #1. Solutions
Math 501 Iowa State University Introduction to Real Analysis Department of Mathematics Instructor: Alex Roitershtein Summer 015 EXERCISES FROM CHAPTER 1 Homework #1 Solutions The following version of the
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationLINEAR PROGRAMMING II
LINEAR PROGRAMMING II LP duality strong duality theorem bonus proof of LP duality applications Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING II LP duality Strong duality
More informationChapter 3, Operations Research (OR)
Chapter 3, Operations Research (OR) Kent Andersen February 7, 2007 1 Linear Programs (continued) In the last chapter, we introduced the general form of a linear program, which we denote (P) Minimize Z
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationReal Analysis Notes. Thomas Goller
Real Analysis Notes Thomas Goller September 4, 2011 Contents 1 Abstract Measure Spaces 2 1.1 Basic Definitions........................... 2 1.2 Measurable Functions........................ 2 1.3 Integration..............................
More informationMath 341: Convex Geometry. Xi Chen
Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry
More informationCS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2)
CS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2) Tim Roughgarden February 2, 2016 1 Recap This is our third lecture on linear programming, and the second on linear programming
More informationChapter 2. Convex Sets: basic results
Chapter 2 Convex Sets: basic results In this chapter, we introduce one of the most important tools in the mathematical approach to Economics, namely the theory of convex sets. Almost every situation we
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationCS261: Problem Set #3
CS261: Problem Set #3 Due by 11:59 PM on Tuesday, February 23, 2016 Instructions: (1) Form a group of 1-3 students. You should turn in only one write-up for your entire group. (2) Submission instructions:
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationHomework 4, 5, 6 Solutions. > 0, and so a n 0 = n + 1 n = ( n+1 n)( n+1+ n) 1 if n is odd 1/n if n is even diverges.
2..2(a) lim a n = 0. Homework 4, 5, 6 Solutions Proof. Let ɛ > 0. Then for n n = 2+ 2ɛ we have 2n 3 4+ ɛ 3 > ɛ > 0, so 0 < 2n 3 < ɛ, and thus a n 0 = 2n 3 < ɛ. 2..2(g) lim ( n + n) = 0. Proof. Let ɛ >
More informationPROBLEMS. (b) (Polarization Identity) Show that in any inner product space
1 Professor Carl Cowen Math 54600 Fall 09 PROBLEMS 1. (Geometry in Inner Product Spaces) (a) (Parallelogram Law) Show that in any inner product space x + y 2 + x y 2 = 2( x 2 + y 2 ). (b) (Polarization
More informationConvex Analysis Background
Convex Analysis Background John C. Duchi Stanford University Park City Mathematics Institute 206 Abstract In this set of notes, we will outline several standard facts from convex analysis, the study of
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationTheorems. Theorem 1.11: Greatest-Lower-Bound Property. Theorem 1.20: The Archimedean property of. Theorem 1.21: -th Root of Real Numbers
Page 1 Theorems Wednesday, May 9, 2018 12:53 AM Theorem 1.11: Greatest-Lower-Bound Property Suppose is an ordered set with the least-upper-bound property Suppose, and is bounded below be the set of lower
More informationInput: System of inequalities or equalities over the reals R. Output: Value for variables that minimizes cost function
Linear programming Input: System of inequalities or equalities over the reals R A linear cost function Output: Value for variables that minimizes cost function Example: Minimize 6x+4y Subject to 3x + 2y
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationFarkas Lemma, Dual Simplex and Sensitivity Analysis
Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More information(c) For each α R \ {0}, the mapping x αx is a homeomorphism of X.
A short account of topological vector spaces Normed spaces, and especially Banach spaces, are basic ambient spaces in Infinite- Dimensional Analysis. However, there are situations in which it is necessary
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationLecture Notes 3: Duality
Algorithmic Methods 1/11/21 Professor: Yossi Azar Lecture Notes 3: Duality Scribe:Moran Bar-Gat 1 Introduction In this lecture we will present the dual concept, Farkas s Lema and their relation to the
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationLecture 15: October 15
10-725: Optimization Fall 2012 Lecturer: Barnabas Poczos Lecture 15: October 15 Scribes: Christian Kroer, Fanyi Xiao Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have
More informationLecture Note 18: Duality
MATH 5330: Computational Methods of Linear Algebra 1 The Dual Problems Lecture Note 18: Duality Xianyi Zeng Department of Mathematical Sciences, UTEP The concept duality, just like accuracy and stability,
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationLocally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem
56 Chapter 7 Locally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem Recall that C(X) is not a normed linear space when X is not compact. On the other hand we could use semi
More informationMath General Topology Fall 2012 Homework 1 Solutions
Math 535 - General Topology Fall 2012 Homework 1 Solutions Definition. Let V be a (real or complex) vector space. A norm on V is a function : V R satisfying: 1. Positivity: x 0 for all x V and moreover
More information2. Dual space is essential for the concept of gradient which, in turn, leads to the variational analysis of Lagrange multipliers.
Chapter 3 Duality in Banach Space Modern optimization theory largely centers around the interplay of a normed vector space and its corresponding dual. The notion of duality is important for the following
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More informationBenders Decomposition
Benders Decomposition Yuping Huang, Dr. Qipeng Phil Zheng Department of Industrial and Management Systems Engineering West Virginia University IENG 593G Nonlinear Programg, Spring 2012 Yuping Huang (IMSE@WVU)
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week 2 November 6 November Deadline to hand in the homeworks: your exercise class on week 9 November 13 November Exercises (1) Let X be the following space of piecewise
More informationMath 5052 Measure Theory and Functional Analysis II Homework Assignment 7
Math 5052 Measure Theory and Functional Analysis II Homework Assignment 7 Prof. Wickerhauser Due Friday, February 5th, 2016 Please do Exercises 3, 6, 14, 16*, 17, 18, 21*, 23*, 24, 27*. Exercises marked
More informationSet, functions and Euclidean space. Seungjin Han
Set, functions and Euclidean space Seungjin Han September, 2018 1 Some Basics LOGIC A is necessary for B : If B holds, then A holds. B A A B is the contraposition of B A. A is sufficient for B: If A holds,
More information1 Overview. 2 Extreme Points. AM 221: Advanced Optimization Spring 2016
AM 22: Advanced Optimization Spring 206 Prof. Yaron Singer Lecture 7 February 7th Overview In the previous lectures we saw applications of duality to game theory and later to learning theory. In this lecture
More informationInequality Constraints
Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the
More informationMA651 Topology. Lecture 10. Metric Spaces.
MA65 Topology. Lecture 0. Metric Spaces. This text is based on the following books: Topology by James Dugundgji Fundamental concepts of topology by Peter O Neil Linear Algebra and Analysis by Marc Zamansky
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationLecture 8 Plus properties, merit functions and gap functions. September 28, 2008
Lecture 8 Plus properties, merit functions and gap functions September 28, 2008 Outline Plus-properties and F-uniqueness Equation reformulations of VI/CPs Merit functions Gap merit functions FP-I book:
More informationConvex Sets with Applications to Economics
Convex Sets with Applications to Economics Debasis Mishra March 10, 2010 1 Convex Sets A set C R n is called convex if for all x, y C, we have λx+(1 λ)y C for all λ [0, 1]. The definition says that for
More informationB. Appendix B. Topological vector spaces
B.1 B. Appendix B. Topological vector spaces B.1. Fréchet spaces. In this appendix we go through the definition of Fréchet spaces and their inductive limits, such as they are used for definitions of function
More informationThe Simplex Algorithm
8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.
More informationFarkas Lemma. Rudi Pendavingh. Optimization in R n, lecture 2. Eindhoven Technical University. Rudi Pendavingh (TUE) Farkas Lemma ORN2 1 / 15
Farkas Lemma Rudi Pendavingh Eindhoven Technical University Optimization in R n, lecture 2 Rudi Pendavingh (TUE) Farkas Lemma ORN2 1 / 15 Today s Lecture Theorem (Farkas Lemma, 1894) Let A be an m n matrix,
More informationLecture 9 Monotone VIs/CPs Properties of cones and some existence results. October 6, 2008
Lecture 9 Monotone VIs/CPs Properties of cones and some existence results October 6, 2008 Outline Properties of cones Existence results for monotone CPs/VIs Polyhedrality of solution sets Game theory:
More informationChapter 2: Preliminaries and elements of convex analysis
Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15
More informationMATH 4200 HW: PROBLEM SET FOUR: METRIC SPACES
MATH 4200 HW: PROBLEM SET FOUR: METRIC SPACES PETE L. CLARK 4. Metric Spaces (no more lulz) Directions: This week, please solve any seven problems. Next week, please solve seven more. Starred parts of
More informationOPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM
OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM Abstract These notes give a summary of the essential ideas and results It is not a complete account; see Winston Chapters 4, 5 and 6 The conventions and notation
More informationLinear Programming Inverse Projection Theory Chapter 3
1 Linear Programming Inverse Projection Theory Chapter 3 University of Chicago Booth School of Business Kipp Martin September 26, 2017 2 Where We Are Headed We want to solve problems with special structure!
More informationDiscrete Optimization
Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises
More informationChapter 1 Linear Programming. Paragraph 5 Duality
Chapter 1 Linear Programming Paragraph 5 Duality What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution
More informationMotivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory
Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationOn duality gap in linear conic problems
On duality gap in linear conic problems C. Zălinescu Abstract In their paper Duality of linear conic problems A. Shapiro and A. Nemirovski considered two possible properties (A) and (B) for dual linear
More informationZero sum games Proving the vn theorem. Zero sum games. Roberto Lucchetti. Politecnico di Milano
Politecnico di Milano General form Definition A two player zero sum game in strategic form is the triplet (X, Y, f : X Y R) f (x, y) is what Pl1 gets from Pl2, when they play x, y respectively. Thus g
More informationMathematical Optimisation, Chpt 2: Linear Equations and inequalities
Introduction Gauss-elimination Orthogonal projection Linear Inequalities Integer Solutions Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl
More informationDivision of the Humanities and Social Sciences. Supergradients. KC Border Fall 2001 v ::15.45
Division of the Humanities and Social Sciences Supergradients KC Border Fall 2001 1 The supergradient of a concave function There is a useful way to characterize the concavity of differentiable functions.
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationAppendix A: Separation theorems in IR n
Appendix A: Separation theorems in IR n These notes provide a number of separation theorems for convex sets in IR n. We start with a basic result, give a proof with the help on an auxiliary result and
More informationNonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
More informationIntroduction to Proofs
Real Analysis Preview May 2014 Properties of R n Recall Oftentimes in multivariable calculus, we looked at properties of vectors in R n. If we were given vectors x =< x 1, x 2,, x n > and y =< y1, y 2,,
More informationFunctional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...
Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationIntroduction to Nonlinear Stochastic Programming
School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS
More informationLecture 5. Ch. 5, Norms for vectors and matrices. Norms for vectors and matrices Why?
KTH ROYAL INSTITUTE OF TECHNOLOGY Norms for vectors and matrices Why? Lecture 5 Ch. 5, Norms for vectors and matrices Emil Björnson/Magnus Jansson/Mats Bengtsson April 27, 2016 Problem: Measure size of
More informationConvex Sets. Prof. Dan A. Simovici UMB
Convex Sets Prof. Dan A. Simovici UMB 1 / 57 Outline 1 Closures, Interiors, Borders of Sets in R n 2 Segments and Convex Sets 3 Properties of the Class of Convex Sets 4 Closure and Interior Points of Convex
More informationHelly's Theorem and its Equivalences via Convex Analysis
Portland State University PDXScholar University Honors Theses University Honors College 2014 Helly's Theorem and its Equivalences via Convex Analysis Adam Robinson Portland State University Let us know
More informationIntroduction to Convex and Quasiconvex Analysis
Introduction to Convex and Quasiconvex Analysis J.B.G.Frenk Econometric Institute, Erasmus University, Rotterdam G.Kassay Faculty of Mathematics, Babes Bolyai University, Cluj August 27, 2001 Abstract
More informationLecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016
Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,
More informationThis chapter reviews some basic geometric facts that we will need during the course.
Chapter 1 Some Basic Geometry This chapter reviews some basic geometric facts that we will need during the course. 1.1 Affine Geometry We will assume that you are familiar with the basic notions of linear
More information