# Assignment 1: From the Definition of Convexity to Helley Theorem

Size: px
Start display at page:

## Transcription

1 Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x ix 1x 2 + i 2 x 2 2 1, i = 1,..., 10} 3. {x R 2 : x ix 1x 2 + i 2 x 2 2 1, i = 1,..., 10} 4. {x R 2 : x x 1x 2 + 4x 2 2 1} 5. {x R 10 : x x2 2 +3x x x x x x x 10 } 6. {x R 2 : exp{x 1 } x 2 } 7. {x R 2 : exp{x 1 } x 2 } 8. {x R n n : x 2 i = 1} 9. {x R n n : x 2 i 1} 10. {x R n n : x 2 i 1} 11. {x R n : max,...,n x i 1} 12. {x R n : max,...,n x i 1} 13. {x R n : max,...,n x i = 1} 14. {x R n : min,...,n x i 1} 15. {x R n : min,...,n x i 1} 16. {x R n : min,...,n x i = 1} Exercise 2 Which ones of the following three statements are true? 1. The convex hull of a closed set in R n is closed 2. The convex hull of a closed convex set in R n is closed 3. The convex hull of a closed and bounded set in R n is closed and bounded For true statements, present proofs; for wrong, give counterexamples. Hint: Recall that a bounded and closed subset of R n Caratheodory Theorem. is compact and that there exists The next two exercises are optional. Those who do well at least one of them, will be released from the first mid-term exam with grade

2 Exercise 3 A cake contains 300 g 1) of raisins (you may think of every one of them as of a 3D ball of positive radius). John and Jill are about to divide the cake according to the following rules: first, Jill chooses a point a in the cake; second, John makes a cut through a, that is, chooses a 2D plane Π passing through a and takes the part of the cake on one side of the plane (both Π and the side are up to John, with the only restriction that the plane should pass through a); all the rest goes to Jill. 1. Prove that it may happen that Jill cannot guarantee herself 76 g of the raisins 2. Prove that Jill always can choose a in a way which guarantees her at least 74 g of the raisins 3. Consider n-dimensional version of the problem, where the raisins are n-dimensional balls, the cake is a domain in R n, and a cut taken by John is defined as the part of the cake contained in the half-space {x R n : e T (x a) 0}, where e 0 is the vector ( inner normal to the cutting hyperplane ) chosen by John. Prove that for every ɛ > 0, Jill can guarantee to herself at least 300 n+1 ɛ g of raisins, but in general cannot guarantee to herself 300 n+1 + ɛ g. Exercise 4 Prove the following Kirchberger s Theorem: Assume that X = {x 1,..., x k } and Y = {y 1,..., y m } are finite sets in R n, with k + m n + 2, and all the points x 1,..., x k, y 1,..., y m are distinct. Assume that for any subset S X Y comprised of n + 2 points the convex hulls of the sets X S and Y S do not intersect. Then the convex hulls of X and Y also do not intersect. Hint: Assume, on contrary, that the convex hulls of X and Y intersect, so that k m λ i x i = µ j y j for certain nonnegative λ i, i λ i = 1, and certain nonnegative µ j, j µ j = 1, and look at the expression of this type with the minimum possible total number of nonzero coefficients λ i, µ j. j=1 1) grams 2

3 Assignment 2: General Theorem on Alternative General Theorem on Alternative has several particular cases (in fact equivalent to the GTA). The goal of the subsequent exercises is to become acquainted with these particular cases. Exercise 5 Derive from GTA the following Gordan s Theorem on Alternative Let A be an m n matrix. One of the inequality systems (I) Ax < 0, x R n, (II) A T y = 0, 0 y 0, y R m, has a solution if and only if the other one has no solutions. Exercise 6 Derive from GTA the following Inhomogeneous Farkas Lemma A linear inequality is a consequence of a solvable system of linear inequalities a T x p (1) Nx q (2) if and only if it is a linear consequence of the system and the trivial inequality 0 T x 1, i.e., if it can be obtained by taking weighted sum, with nonnegative coefficients, of the inequalities from the system and this trivial inequality. Algebraically: (1) is a consequence of solvable system (2) if and only if for some nonnegative vector ν such that a = N T ν ν T q p. Exercise 7 Derive from GTA the following Motzkin s Theorem on Alternative The system has no solutions if and only if the system has a solution. Sx < 0, Nx 0 S T σ + N T ν = 0, σ 0, ν 0, σ 0 Exercise 8 Which of the following systems of linear inequalities with 2 unknowns have, and which have no solutions (for the systems which are solvable, point out a solution; for the unsolvable systems, explain why they are so): 3

4 x + y 2 2x y 1 5x + y 5 x + y 2 2x y 1 5x + y 4 x + y 2 2x y 1 5x + y 3.5 Exercise 9 Consider the linear inequality x + y 2 and the system of linear inequalities { x 1 x 100 Our inequality clearly is a consequence of the system it is satisfied at every solution to it (simply because there are no solutions to the system at all). According to the Inhomogeneous Farkas Lemma, the inequality should be a linear consequence of the system and the trivial inequality 0 1, i.e., there should exist nonnegative ν 1, ν 2 such that ( ) ( ) 1 1 = ν ( ) 1 + ν 2, ν ν 2 2, which clearly is not the case. What is the reason for the observed contradiction? 4

5 Assignment 3: Separation and Extreme Points Exercise 10 Mark in the below list the cases when the linear form f T x separates the sets S and T S = {0} R, T = {0} R, f T x = x S = {0} R, T = [0, 1] R, f T x = x S = {0} R, T = [ 1, 1] R, f T x = x S = {x R 3 : x 1 = x 2 = x 3 }, T = {x R 3 : x 3 S = {x R 3 : x 1 = x 2 = x 3 }, T = {x R 3 : x 3 S = {x R 3 : 1 x 1 1}, T = {x R 3 : x 2 1 4}, f T x = x 1 x x2 2 }, f T x = x 1 x 2 x x2 2 }, f T x = x 3 x 2 S = {x R 2 : x 2 x 2 1, x 1 0}, T = {x R 2 : x 2 = 0}, f T x = x 2 Exercise 11 Consider the set M = x R 2004 : x 1 + x x x 1 + 2x 2 + 3x x x x x x x x x x Is it possible to separate this set from the set {x 1 = x 2 =... = x }? If yes, what could be a separating plane? Exercise 12 Let M be a convex set in R n and x be an extreme point of M. Prove that if m x = λ i x i is a representation of x as a convex combination of points x i M with positive weights λ i, then x = x 1 =... = x m. Exercise 13 Let M be a closed convex set in R n and x be a point of M. Prove that if there exists a linear form a T x such that x is the unique maximizer of the form on M, then x is an extreme point of M. Exercise 14 Find all extreme points of the set {x R 2 x 1 + 2x 2 8, 2x 1 + x 2 9, 3x 1 x 2 6, x 1, x 2 0}. Exercise 15 Mark with y those of the below statements which are true: If M is a nonempty convex set in R n which does not contain lines, then M has an extreme point If M is a convex set in R n which has an extreme point, then M does not contain lines 5

6 If M is a nonempty closed convex set in R n which does not contain lines, then M has an extreme point If M is a closed convex set in R n which has an extreme point, then M does not contain lines If M is a nonempty bounded convex set in R n, then M is the convex hull of Ext(M) If M is a nonempty closed and bounded convex set in R n, then M is the convex hull of Ext(M) If M is a nonempty closed convex set in R n which is equal to the convex hull of Ext(M), then M is bounded Exercise 16 A n n matrix π is called double stochastic, if all its entries are nonnegative, and the sums of entries in every row and every column are equal to 1, as it is the case with the unit matrix or, more generally, with a permutation matrix the one which has exactly one nonzero entry (equal to 1) in every column and every row, e.g., π = Double stochastic matrices of a given order n clearly form a nonempty bounded convex polyhedral set D in R n n. What are the extreme points of the set? The answer is given by the following Theorem 0.1 (Birkhoff) The extreme points of the polytope D of n n double stochastic matrices are exactly the permutation matrices of order n. Try to prove the Theorem. Hint: The polytope in question is the feasible set of n n Transportation problem with unit capacities in the sources and unit demands in the sinks, isn t it? The Birkhoff Theorem is the source of a number of important inequalities; some of these inequalities will be the subject of optional exercises in the coming hometasks. 6

7 Assignment 4: Convex Functions Exercise 17 Mark by c those of the following functions which are convex on the indicated domains: f(x) 1 on R f(x) = x on R f(x) = x on R f(x) = x on R f(x) = x on R + = {x 0} exp{x} on R exp{x 2 } on R exp{ x 2 } on R exp{ x 2 } on {x x 100} Exercise 18 Prove that the following functions are convex on the indicated domains: x2 y on {(x, y) R2 y > 0} ln(exp{x} + exp{y}) on the 2D plane. Exercise 19 A function f defined on a convex set Q is called log-convex on Q, if it takes real positive values on Q and the function ln f is convex on Q. Prove that a log-convex on Q function is convex on Q the sum (more generally, linear combination with positive coefficients) of two log-convex functions on Q also is log-convex on the set. Hint: use the result of the previous Exercise + your knowledge on operations preserving convexity Exercise 20 For n-dimensional vector x, let x = ( x 1,..., x n ) T be the vector obtained from x by rearranging the coordinates in the non-ascending order. E.g., with x = (2, 1, 3, 1) T, x = (3, 2, 1, 1) T. Let us fix k, 1 k n. Is the function x k (k-th largest entry in x) a convex function of x? Is the function s k (x) = x x k (the sum of k largest entries in x) convex? Exercise 21 Consider a Linear Programming program min x {c T x : Ax b} with m n matrix A, and let x be an optimal solution to the problem. It means that x is a minimizer of differentiable convex function f(x) = c T x on convex set Q = {x Ax b} and therefore, according to Proposition 2.5.1, f(x ) should belong to the normal cone of A at x this is the necessary and sufficient condition for optimality of x. What does this condition mean in terms of the data A, b, c? 19

8 Optional exercises: Exercise 22 Let f(x) be a convex symmetric function of x R n (symmetry means that the f which remains unchanged when permuting the coordinates of the argument, as it is the case with x i, or max x i ). Prove that if π is a double stochastic n n matrix (see Optional exercise from i i Assignment 3), then f(πx) f(x) x Exercise 23 Let f(x) be a convex symmetric function on R n. For a symmetric n n matrix X, let λ 1 (X) λ 2 (X)... λ n (X) be the eigenvalues of X taken with their multiplicities and arranged in the nondecreasing order. Prove 1. For every orthogonal n n matrix U and symmetric n n matrix X, f(diag(uxu T )) f(λ(x)) where diag(a) stands for the diagonal of square matrix; 2. The function of symmetric n n matrix X is convex. F (X) = f(λ(x)) Exercise 24 For a symmetric matrix X, what is larger the sum of two largest diagonal entries or the sum of two largest eigenvalues? 20

9 Assignment 5: Optimality Conditions in Convex Programming Exercise 25 Find the minimizer of a linear function on the set f(x) = c T x V p = {x R n n x i p 1}; here p, 1 < p <, is a parameter. What happens with the solution when the parameter becomes 0.5? Exercise 26 Let a 1,..., a n > 0, α, β > 0. Solve the optimization problem { n a i min x x α : x > 0, } x β i 1 i i Exercise 27 Consider the optimization problem { max ξ T x + τt + ln(t 2 x T x) : (t, x) X = {t > } x T x} x,t where ξ R n, τ R are parameters. Is the problem convex 3)? What is the domain in space of parameters where the problem is solvable? What is the optimal value? Is it convex in the parameters? Exercise 28 Consider the optimization problem max {ax + by + ln(ln y x) + ln(y) : (x, y) X = {y > exp{x}}}, x,y where a, b R are parameters. Is the problem convex? What is the domain in space of parameters where the problem is solvable? What is the optimal value? Is it convex in the parameters? 3) A maximization problem with objective f( ) and certain constraints and domain is called convex if the equivalent minimization problem with the objective ( f) and the original constraints and domain is convex. 17

10 Assignment 6: Optimality Conditions Exercise 29 Consider the problem of minimizing the linear form f(x) = x x 1 on the 2D plane over the triangle with the vertices (1, 0), (0, 1), (0, 1/2) (draw the picture!). 1) Verify that the problem has unique optimal solution x = (1, 0) 2) Verify that the problem can be posed as the LP program min x {x x 1 : x 1 + x 2 1, x 1 + 2x 2 1, x 1, x 2 0}. Prove that in this formulation of the problem the KKT necessary optimality condition is satisfied at x. What are the active at x constraints? What are the corresponding Lagrange multipliers? 3) Verify that the problem can be posed as a nonlinear program with inequality constraints min x {x x 1 : x 1 0, x 2 0, (x 1 + x 2 1)(x 1 + 2x 2 1) 0}. Is the KKT Optimality condition satisfied at x? Exercise 30 Consider the following elementary problem: min x f(x 1, x 2 ) = x 2 1 x 2 : h(x) x 2 = 0 { } with the evident unique optimal solution (0, 0). Is the KKT condition satisfied at this solution? Rewrite the problem equivalently as { } min f(x 1, x 2 ) = x 2 x 1 x 2 : h(x) x 2 2 = 0. What about the KKT condition in this equivalent problem? Exercise 31 Consider an inequality constrained optimization problem min x {f(x) : g i (x) 0, i = 1,..., m}. Assume that x is locally optimal solution, f, g i are continuously differentiable in a neighbourhood of x and the constraints g i are concave in this neighbourhood. Prove that x is locally optimal solution to the linearized problem { } min f(x ) + (x x ) T f(x ) : (x x ) T g j (x ) = 0, j J(x ) = {j : g j (x ) = 0}. x Is x a KKT point of the problem? Exercise 32 Let a 1,..., a n be positive reals, and let 0 < s < r be two reals. Find maximum and minimum of the function n a i x i r on the surface n x i s = 1. 22

11 Optional exercises (those who will submit complete solutions to both optional exercises before March 16, will get 100 in the second MidTerm exam): Exercise 33 Recall S-Lemma: If A, B are two symmetric n n matrices such that for certain x, then the implication x T B x > 0 x T Bx 0 x T Ax 0 holds true iff there exists λ 0 such that A λb 0 (P 0 means that P is symmetric positive semidefinite: P = P T and x T P x 0 for all x R n ). The proof given in Lecture on optimality conditions (see Transparencies) was incomplete it was taken for granted that certain optimization problem has an optimal solution. Fill the gap in the proof or find an alternative proof. Exercise 34 Let A = A T be a symmetric n n matrix, and let B, C be m n matrices, C 0. Then all matrices A + B T C + C T B corresponding to m m matrices with 1 4) are positive semidefinite iff there exists λ 0 such that the matrix [ ] A λc T C B T B is positive semidefinite. This fact has important applications in Control and Optimization under uncertainty. λi m 4) here is the standard matrix norm: = max { u 2: u 2 1}. The statement remains true u when replacing the matrix norm with the Frobenius norm F ro = 2 ij. i,j 23

### Lecture 6 - Convex Sets

Lecture 6 - Convex Sets Definition A set C R n is called convex if for any x, y C and λ [0, 1], the point λx + (1 λ)y belongs to C. The above definition is equivalent to saying that for any x, y C, the

### DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

### CSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming

CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming

### Inequality Constraints

Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the

### Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS

Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution

### MAT-INF4110/MAT-INF9110 Mathematical optimization

MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:

### What can be expressed via Conic Quadratic and Semidefinite Programming?

What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent

### 4. Algebra and Duality

4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone

### Linear and non-linear programming

Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)

### Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

### AN INTRODUCTION TO CONVEXITY

AN INTRODUCTION TO CONVEXITY GEIR DAHL NOVEMBER 2010 University of Oslo, Centre of Mathematics for Applications, P.O.Box 1053, Blindern, 0316 Oslo, Norway (geird@math.uio.no) Contents 1 The basic concepts

### minimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,

4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x

### Introduction to linear programming

Chapter 2 Introduction to linear programming 2.1 Single-objective optimization problem We study problems of the following form: Given a set S and a function f : S R, find, if possible, an element x S that

### Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.

Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space

### Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The

### GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III

GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS

### HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.

HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard

### UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems

UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction

### CO 250 Final Exam Guide

Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,

### CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.

1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function

### Two hours. To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER. xx xxxx 2017 xx:xx xx.

Two hours To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER CONVEX OPTIMIZATION - SOLUTIONS xx xxxx 27 xx:xx xx.xx Answer THREE of the FOUR questions. If

### IE 521 Convex Optimization Homework #1 Solution

IE 521 Convex Optimization Homework #1 Solution your NAME here your NetID here February 13, 2019 Instructions. Homework is due Wednesday, February 6, at 1:00pm; no late homework accepted. Please use the

### 12. Perturbed Matrices

MAT334 : Applied Linear Algebra Mike Newman, winter 208 2. Perturbed Matrices motivation We want to solve a system Ax = b in a context where A and b are not known exactly. There might be experimental errors,

### Math 341: Convex Geometry. Xi Chen

Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry

### Lecture 2: Convex Sets and Functions

Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are

### Math 164-1: Optimization Instructor: Alpár R. Mészáros

Math 164-1: Optimization Instructor: Alpár R. Mészáros First Midterm, April 20, 2016 Name (use a pen): Student ID (use a pen): Signature (use a pen): Rules: Duration of the exam: 50 minutes. By writing

### IE 521 Convex Optimization

Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.

### On John type ellipsoids

On John type ellipsoids B. Klartag Tel Aviv University Abstract Given an arbitrary convex symmetric body K R n, we construct a natural and non-trivial continuous map u K which associates ellipsoids to

### Geometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as

Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.

### Example: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma

4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid

### E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming

E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program

### Chapter 1. Preliminaries

Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between

### 10-725/ Optimization Midterm Exam

10-725/36-725 Optimization Midterm Exam November 6, 2012 NAME: ANDREW ID: Instructions: This exam is 1hr 20mins long Except for a single two-sided sheet of notes, no other material or discussion is permitted

### Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016

Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,

### Symmetric Matrices and Eigendecomposition

Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory

### DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO

QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:10-12:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists

### Lecture 8. Strong Duality Results. September 22, 2008

Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation

### An Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace

An Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace Takao Fujimoto Abstract. This research memorandum is aimed at presenting an alternative proof to a well

### CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

### 1 Strict local optimality in unconstrained optimization

ORF 53 Lecture 14 Spring 016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, April 14, 016 When in doubt on the accuracy of these notes, please cross check with the instructor s

### Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.

. Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,

### Copositive Plus Matrices

Copositive Plus Matrices Willemieke van Vliet Master Thesis in Applied Mathematics October 2011 Copositive Plus Matrices Summary In this report we discuss the set of copositive plus matrices and their

### Shiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient

Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian

### Lecture 9 Monotone VIs/CPs Properties of cones and some existence results. October 6, 2008

Lecture 9 Monotone VIs/CPs Properties of cones and some existence results October 6, 2008 Outline Properties of cones Existence results for monotone CPs/VIs Polyhedrality of solution sets Game theory:

### Lecture 5 : Projections

Lecture 5 : Projections EE227C. Lecturer: Professor Martin Wainwright. Scribe: Alvin Wan Up until now, we have seen convergence rates of unconstrained gradient descent. Now, we consider a constrained minimization

### j=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A).

Math 344 Lecture #19 3.5 Normed Linear Spaces Definition 3.5.1. A seminorm on a vector space V over F is a map : V R that for all x, y V and for all α F satisfies (i) x 0 (positivity), (ii) αx = α x (scale

### Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

### Convex Optimization M2

Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization

### Math 61CM - Solutions to homework 6

Math 61CM - Solutions to homework 6 Cédric De Groote November 5 th, 2018 Problem 1: (i) Give an example of a metric space X such that not all Cauchy sequences in X are convergent. (ii) Let X be a metric

### Lecture 5. Theorems of Alternatives and Self-Dual Embedding

IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c

### Econ Slides from Lecture 7

Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for

### Mathematics 530. Practice Problems. n + 1 }

Department of Mathematical Sciences University of Delaware Prof. T. Angell October 19, 2015 Mathematics 530 Practice Problems 1. Recall that an indifference relation on a partially ordered set is defined

### Optimality Conditions for Constrained Optimization

72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)

### Chapter 2: Preliminaries and elements of convex analysis

Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15

### A strongly polynomial algorithm for linear systems having a binary solution

A strongly polynomial algorithm for linear systems having a binary solution Sergei Chubanov Institute of Information Systems at the University of Siegen, Germany e-mail: sergei.chubanov@uni-siegen.de 7th

### The Karush-Kuhn-Tucker (KKT) conditions

The Karush-Kuhn-Tucker (KKT) conditions In this section, we will give a set of sufficient (and at most times necessary) conditions for a x to be the solution of a given convex optimization problem. These

### The Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment

he Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment William Glunt 1, homas L. Hayden 2 and Robert Reams 2 1 Department of Mathematics and Computer Science, Austin Peay State

### Constrained Optimization and Lagrangian Duality

CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may

### I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010

I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0

### Tutorials in Optimization. Richard Socher

Tutorials in Optimization Richard Socher July 20, 2008 CONTENTS 1 Contents 1 Linear Algebra: Bilinear Form - A Simple Optimization Problem 2 1.1 Definitions........................................ 2 1.2

### Exercises: Brunn, Minkowski and convex pie

Lecture 1 Exercises: Brunn, Minkowski and convex pie Consider the following problem: 1.1 Playing a convex pie Consider the following game with two players - you and me. I am cooking a pie, which should

### Linear Algebra I. Ronald van Luijk, 2015

Linear Algebra I Ronald van Luijk, 2015 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents Dependencies among sections 3 Chapter 1. Euclidean space: lines and hyperplanes 5 1.1. Definition

### 3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions

A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence

### A Review of Linear Programming

A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko

### Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)

Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual

### SECTION C: CONTINUOUS OPTIMISATION LECTURE 9: FIRST ORDER OPTIMALITY CONDITIONS FOR CONSTRAINED NONLINEAR PROGRAMMING

Nf SECTION C: CONTINUOUS OPTIMISATION LECTURE 9: FIRST ORDER OPTIMALITY CONDITIONS FOR CONSTRAINED NONLINEAR PROGRAMMING f(x R m g HONOUR SCHOOL OF MATHEMATICS, OXFORD UNIVERSITY HILARY TERM 5, DR RAPHAEL

### Linear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016

Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources

### EE5138R: Problem Set 5 Assigned: 16/02/15 Due: 06/03/15

EE538R: Problem Set 5 Assigned: 6/0/5 Due: 06/03/5. BV Problem 4. The feasible set is the convex hull of (0, ), (0, ), (/5, /5), (, 0) and (, 0). (a) x = (/5, /5) (b) Unbounded below (c) X opt = {(0, x

### UC Berkeley Department of Electrical Engineering and Computer Science. EECS 227A Nonlinear and Convex Optimization. Solutions 5 Fall 2009

UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 5 Fall 2009 Reading: Boyd and Vandenberghe, Chapter 5 Solution 5.1 Note that

### Elementary linear algebra

Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

### 6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC

6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility

### Integer programming: an introduction. Alessandro Astolfi

Integer programming: an introduction Alessandro Astolfi Outline Introduction Examples Methods for solving ILP Optimization on graphs LP problems with integer solutions Summary Introduction Integer programming

### Kernel Methods. Machine Learning A W VO

Kernel Methods Machine Learning A 708.063 07W VO Outline 1. Dual representation 2. The kernel concept 3. Properties of kernels 4. Examples of kernel machines Kernel PCA Support vector regression (Relevance

### Chap 2. Optimality conditions

Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1

### Linear Programming Inverse Projection Theory Chapter 3

1 Linear Programming Inverse Projection Theory Chapter 3 University of Chicago Booth School of Business Kipp Martin September 26, 2017 2 Where We Are Headed We want to solve problems with special structure!

### ON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION

Annales Univ. Sci. Budapest., Sect. Comp. 33 (2010) 273-284 ON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION L. László (Budapest, Hungary) Dedicated to Professor Ferenc Schipp on his 70th

### Lecture: Cone programming. Approximating the Lorentz cone.

Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is

### 15. Conic optimization

L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone

### Constrained Optimization Theory

Constrained Optimization Theory Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Constrained Optimization Theory IMA, August

### Mid Term-1 : Practice problems

Mid Term-1 : Practice problems These problems are meant only to provide practice; they do not necessarily reflect the difficulty level of the problems in the exam. The actual exam problems are likely to

### Mathematical Optimisation, Chpt 2: Linear Equations and inequalities

Introduction Gauss-elimination Orthogonal projection Linear Inequalities Integer Solutions Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory

### 5. Duality. Lagrangian

5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

### 4TE3/6TE3. Algorithms for. Continuous Optimization

4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality

### Duality Theory of Constrained Optimization

Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive

### Basic convexity. 1.1 Convex sets and combinations. λ + μ b (λ + μ)a;

1 Basic convexity 1.1 Convex sets and combinations AsetA R n is convex if together with any two points x, y it contains the segment [x, y], thus if (1 λ)x + λy A for x, y A, 0 λ 1. Examples of convex sets

### Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method

Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach

### j=1 u 1jv 1j. 1/ 2 Lemma 1. An orthogonal set of vectors must be linearly independent.

Lecture Notes: Orthogonal and Symmetric Matrices Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Orthogonal Matrix Definition. Let u = [u

### 1 Principal component analysis and dimensional reduction

Linear Algebra Working Group :: Day 3 Note: All vector spaces will be finite-dimensional vector spaces over the field R. 1 Principal component analysis and dimensional reduction Definition 1.1. Given an

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

### Introduction to Mathematical Programming IE406. Lecture 3. Dr. Ted Ralphs

Introduction to Mathematical Programming IE406 Lecture 3 Dr. Ted Ralphs IE406 Lecture 3 1 Reading for This Lecture Bertsimas 2.1-2.2 IE406 Lecture 3 2 From Last Time Recall the Two Crude Petroleum example.

### Lecture 8 : Eigenvalues and Eigenvectors

CPS290: Algorithmic Foundations of Data Science February 24, 2017 Lecture 8 : Eigenvalues and Eigenvectors Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Hermitian Matrices It is simpler to begin with

### Spring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization

Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table

### Real Symmetric Matrices and Semidefinite Programming

Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many

### Mathematical Optimisation, Chpt 2: Linear Equations and inequalities

Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl http://dickinson.website version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson