Practice Exam 1: Continuous Optimisation
|
|
- Annis Jefferson
- 6 years ago
- Views:
Transcription
1 Practice Exam : Continuous Optimisation. Let f : R m R be a convex function and let A R m n, b R m be given. Show that the function g(x) := f(ax + b) is a convex function of x on R n. Suppose that f is strictly convex. Show that then g(x) := f(ax+b) is strictly convex if and only if A has (full) rank n. Hint: Recall that f is strictly convex if for any x y, 0 < λ < it holds: f(λy + ( λ)y 2 ) < λf(y ) + ( λ)f(y 2 ). [3 points] [ points] For x, y R n, λ [0, ] we find: g(λx + ( λ)y) = f(a(λx + ( λ)y)b) = f(λax + ( λ)ay + λb + ( λ)b) = f(λ(ax + b) + ( λ)(ay + b)) f is convex λf(ax + b) + ( λ)f(ay + b) = λg(x) + ( λ)g(y) rank(a) = n implies: x y Ax Ay. As in for x y, λ (0, ) we obtain: g(λx + ( λ)y) = f(λ(ax + b) + ( λ)(ay + b)) f is strict convex, Ax + b Ay + b < λf(ax + b) + ( λ)f(ay + b) = λg(x) + ( λ)g(y) Assume rank(a) < n. hen there exist x y with Ax = Ay and for any λ (0, ) we obtain: g(λx + ( λ)y) = f(λ(ax + b) + ( λ)(ay + b)) = f(ax + b) g(x) = g(y) = g(x) = λg(x) + ( λ)g(y). So g is not strictly convex. 2. For given S R n we define the convex hull conv(s) by { m m conv(s) = x = λ i x i : m N, λ i = ; x i S, λ i 0 i Show that conv(s) is the smallest convex set containing S: Show that the set conv(s) is convex with S conv(s). [3 points]
2 est exam: Continuous Optimisation 205 Monday th December 205 Show that for any convex set C containing S we must have conv(s) C. (Hint: You may use without proof any Lemma/heorem/Corollary from the course, except for heorem.9.) [3 points] For all x S, letting m = and λ = we see that x conv(s), and thus S conv(s). o prove convexity we consider arbitrary x, y conv(s), λ [0, ]. hen there exists p, q N and x,..., x p S and y,..., y q S and θ R p + and ψ R q + such that x = θ i x i, = θ i, y = ψ i y i, = ψ i. We then have λθ i + λx + ( λ)y = λθ i x i + ( λ)ψ i y i, λθ i 0 for all i =,..., p, ( λ)ψ i 0 for all i =,..., q, ( λ)ψ i = λ θ i + ( λ) ψ i = λ + ( λ) =. herefore λx + ( λ)y conv(s), and as x, y conv(s), λ [0, ] were arbitrary, this implies that conv(s) is a convex set. Consider an arbitrary convex set C R n such that S C. For m N let S m = { x = m λ i x i : m λ i = ; x i S, λ i > 0 i. We then have conv(s) = m= Sm. We shall show by induction that S m C for all m, and thus conv(s) = m= Sm C. For m = we have S = S C. Now suppose for the sake of induction that S m arbitrary x S m+. C and consider an We have x = m+ λ ix i for some x i S, λ i > 0 for all i and m λ i =. Letting θ R m + such that θ i = λ i /( λ m+ ) and letting y = m θ ix i we have m θ i = and thus y S m C by the induction hypothesis. We then have y C and x m+ S C and λ m+ [0, ] and x = ( λ m+ )y + λ m+ x m+. As C is a convex set, this implies that x C. 2
3 est exam: Continuous Optimisation 205 Monday th December Consider with 0 c R n the program: (P ) min x R n c x s.t. x x. Show that x = c is the minimizer of (P) with minimum value v(p ) = c. c ( x means here the Euclidian norm.) Compute the solution y of the Lagrangean dual (D) of (P). Show in this way that for the optimal values strong duality holds, i.e., v(d) = v(p ). [2 points] [ points] Either show this by a sketch. Or as follows (using Schwarz inequality): x implies: c x c x c, and c x = c holds iff x = c c So x = c c is the minimizer with v(p ) = c ( c c ) = c. (Alternatively find x by solving the KK-conditions.) he dual (D) is given by (D) max y 0 ψ(y) where ψ(y) := min L(x, y) x Rn with Lagrangean function L(x, y) = c x + y(x x ). We find for y = 0: ψ(0) =. for y > 0: he minimizer x of ψ(y) satisfies x L(x, y) = c + 2yx = 0 or x = c. So (fill in) 2y ψ(y) = 2y c c + y c c y = y c c y. he Lagrangian dual problem is thus max y>0 { y c c y. o find an (unconstrained) maximizer of ψ(y) for y > 0 we solve 0 = ψ (y) = y 2 c c with solution y = 2 c. So v(d) = ψ(y) = c = v(p ).. Consider the problem (in connection with the design of a cylindrical can with height h, radius r and volume at least 2π such that the total surface area is minimal): (P ) : min f(h, r) := 2π(r 2 + rh) s.t. πr 2 h 2π, (and h > 0, r > 0) 3
4 est exam: Continuous Optimisation 205 Monday th December 205 Compute a (the) solution (h, r) of the KK conditions of (P). Show that (P ) is not a convex optimization problem. Show that the solution (h, r) in is a local minimizer. Why is it the unique global solution? Hint: Use the sufficient optimality conditions [ points] [3 points] We first note that the functions f(h, r) = 2π(r 2 + rh) and g(h, r) := πr 2 h + 2π are not convex (for h > 0). For the objective function f, e.g., this follows by: r 0 f = 2π, 2 f = 2π 2r + h 2 By Lemma.2 we thus have that 2 f is not positive semidefinite, thus f is not a convex function and we do not have a convex optimisation problem. We now consider the KK condition: ( f = µ g, g 0, µ g = 0) So consider: 2π ( r 2r+h = µπ r 2 2rh) ( ): Case µ = 0: leads to 2π ( r 2r+h) = 0 with solution (h, r) = (0, 0) which is not feasible. Case µ > 0 and thus g = 0, or equivalently πr 2 h = 2π: he 2 equations in ( ) lead to µ = 2/r and then 2(2r+h) = 2 2rh or h = 2r. r By using the (active) constraint we find πr 2 h = 2πr 3 = 2π with solution r =. So the unique KK solution is given by (h, r) = (2, ), µ = 2. (We apply the second order sufficient conditions of h. 5. to the nonconvex program (P)). So we will show (for the set C = {d R n : f d 0, g d 0): d 2 h,rl(h, r, µ)d > 0 d C \ {0 ( ) We compute f(h, r) = 2π, g(h, r) = π, L(h, r, µ) = 2π + 2( π) = 2π and C = {d R 2 f(h, r) d 0, g(h, r) d 0 { = d R 2 d 0, d 0 { = λ λ R
5 est exam: Continuous Optimisation 205 Monday th December 205 For d = λ(, ) 0, (i.e., λ 0) we obtain (see ( )): 0 λ(, )( 2π) λ =... = 2λ 2 π6 > 0 λ 0. 2 So (h, r) = (2, ) is a local minimizer. It is the unique (global) minimizer since the point is the only KK point. Note that since the linear independency constraint qualification holds ( g = π ( r 2rh) 2 0, for r, h > 0) any local minimizer must be a KK point. Also note that for feasible (h, r) also f holds. (o show the latter fact is technically involved and was not expected to be done.) 5. Consider the closed set K = {x R 2 x + 2x 2 0 and 3x + x 2 0 Prove that K is a proper cone. [You may assume closure.] Find the dual cone to K. [5 points] [ point] In order for a set to be a proper cone it most be a closed, convex, pointed full-dimensional cone. We will assume closure and prove the rest: Convex cone: Consider an arbitrary x, y K and λ, λ 2 > 0. From heorem 7.2, if we can show that λ x + λ 2 y K then we are done. We have his implies that x + 2x 2 0, 3x + x 2 0, λ > 0, y + 2y 2 0, 3y + y 2 0, λ 2 > 0. (λ x + λ 2 y) + 2(λ x + λ 2 y) 2 = λ (x + 2x 2 ) + λ 2 (y + 2y 2 ) 0, 3(λ x + λ 2 y) + (λ x + λ 2 y) 2 = λ (3x + x 2 ) + λ 2 (3y + y 2 ) 0. herefore λ x + λ 2 y K. Full-dimensional: Using Definition 7.8, part 2, this follows from the space being two dimensional and having two linearly independent 0 vectors, K. 0 5
6 est exam: Continuous Optimisation 205 Monday th December 205 Pointed: We will consider an arbitrary x R 2 such that ±x K. Using Definition 7.7, if we can then show that x = 0 then we are done. We have (x) + 2(x) 2 0 x + 2x 2 = 0, ( x) + 2( x) 2 0 3(x) + (x) 2 0 3x + x 2 = 0. 3( x) + ( x) 2 0 herefore x = 2 (3x 5 + x 2 ) {{ (x 5 + 2x 2 ) {{ =0 =0 = 0, x 2 = (3x + x 2 ) {{ =0 3 x {{ = 0. =0 From heorem 8.7 we have { K = cl conic, 2 { 3 = conic, We will consider bounds to the optimal value of the following problem: min x 5x 2 x x 2 2x + x s.t. x 2 + 5x 2 2 x x 2 8x 2 = x R 2. (A) (c) Give a finite upper bound on the optimal value of problem (A). Formulate a positive semidefinite optimisation problem to give a lower bound on the optimal value of problem (A). Give the dual problem to the positive semidefinite optimisation problem you formulated in part of this question. [ point] [2 points] [ point] o find an upper bound we can use any feasible point, x. If we limit our search for a feasible point such that x 2 = 0 then we would have a feasible point if and only if = x x = x 2. herefore both x = (2, 0) and x = ( 2, 0) are feasible points. We only need one point to give us an upper bound, and if we consider the feasible point x = (2, 0) 6
7 est exam: Continuous Optimisation 205 Monday th December 205 then this gives us the upper bound of 5 x 2 x x 2 2 x + x = = = 8 Problem (A) is equivalent to 5 2 min, xx 2x x s.t., xx 8x = x x xx PSD 3 x R 3, A lower bound on this is then provided by solving the optimisation problem 5 2 min, X 2x x,x s.t., X 8x = x PSD 3. (c) his previous optimisation problem is equivalent to 2 0 min 5 2 x0 x, x,x s.t. 0 2 x0 x, = x0 x, =, x0 x PSD 3. We then have that the dual problem is max y s.t. y + y y 0 2 y PSD
8 est exam: Continuous Optimisation 205 Monday th December (Automatic additional points) [ points] Question: otal Points: A copy of the lecture-sheets may be used during the examination. Good luck! 8
Exam: Continuous Optimisation 2015
Exam: Continuous Optimisation 215 1. Let f : C R, C R n convex, be a convex function. Show that then the following holds: A local imizer of f on C is a global imizer on C. And a strict local imizer of
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationLecture 4: Convex Functions, Part I February 1
IE 521: Convex Optimization Instructor: Niao He Lecture 4: Convex Functions, Part I February 1 Spring 2017, UIUC Scribe: Shuanglong Wang Courtesy warning: These notes do not necessarily cover everything
More informationLecture 6 - Convex Sets
Lecture 6 - Convex Sets Definition A set C R n is called convex if for any x, y C and λ [0, 1], the point λx + (1 λ)y belongs to C. The above definition is equivalent to saying that for any x, y C, the
More informationDuality (Continued) min f ( x), X R R. Recall, the general primal problem is. The Lagrangian is a function. defined by
Duality (Continued) Recall, the general primal problem is min f ( x), xx g( x) 0 n m where X R, f : X R, g : XR ( X). he Lagrangian is a function L: XR R m defined by L( xλ, ) f ( x) λ g( x) Duality (Continued)
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationContinuous Optimisation, Chpt 9: Semidefinite Problems
Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationSparse Optimization Lecture: Dual Certificate in l 1 Minimization
Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationSWFR ENG 4TE3 (6TE3) COMP SCI 4TE3 (6TE3) Continuous Optimization Algorithm. Convex Optimization. Computing and Software McMaster University
SWFR ENG 4TE3 (6TE3) COMP SCI 4TE3 (6TE3) Continuous Optimization Algorithm Convex Optimization Computing and Software McMaster University General NLO problem (NLO : Non Linear Optimization) (N LO) min
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationContinuous Optimisation, Chpt 9: Semidefinite Optimisation
Continuous Optimisation, Chpt 9: Semidefinite Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 28/11/17 Monday
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationBASICS OF CONVEX ANALYSIS
BASICS OF CONVEX ANALYSIS MARKUS GRASMAIR 1. Main Definitions We start with providing the central definitions of convex functions and convex sets. Definition 1. A function f : R n R + } is called convex,
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationSubgradients. subgradients and quasigradients. subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients and quasigradients subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE392o, Stanford University Basic inequality recall basic
More informationChapter 2 Convex Analysis
Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,
More informationContinuous Optimisation, Chpt 7: Proper Cones
Continuous Optimisation, Chpt 7: Proper Cones Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 10/11/17 Monday 13th November
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationFinite Dimensional Optimization Part III: Convex Optimization 1
John Nachbar Washington University March 21, 2017 Finite Dimensional Optimization Part III: Convex Optimization 1 1 Saddle points and KKT. These notes cover another important approach to optimization,
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationQuiz Discussion. IE417: Nonlinear Programming: Lecture 12. Motivation. Why do we care? Jeff Linderoth. 16th March 2006
Quiz Discussion IE417: Nonlinear Programming: Lecture 12 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University 16th March 2006 Motivation Why do we care? We are interested in
More informationLagrange Relaxation and Duality
Lagrange Relaxation and Duality As we have already known, constrained optimization problems are harder to solve than unconstrained problems. By relaxation we can solve a more difficult problem by a simpler
More informationMarch 5, 2012 MATH 408 FINAL EXAM SAMPLE
March 5, 202 MATH 408 FINAL EXAM SAMPLE Partial Solutions to Sample Questions (in progress) See the sample questions for the midterm exam, but also consider the following questions. Obviously, a final
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationIE 521 Convex Optimization
Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.
More informationSECTION C: CONTINUOUS OPTIMISATION LECTURE 11: THE METHOD OF LAGRANGE MULTIPLIERS
SECTION C: CONTINUOUS OPTIMISATION LECTURE : THE METHOD OF LAGRANGE MULTIPLIERS HONOUR SCHOOL OF MATHEMATICS OXFORD UNIVERSITY HILARY TERM 005 DR RAPHAEL HAUSER. Examples. In this lecture we will take
More informationLecture 1: Introduction. Outline. B9824 Foundations of Optimization. Fall Administrative matters. 2. Introduction. 3. Existence of optima
B9824 Foundations of Optimization Lecture 1: Introduction Fall 2009 Copyright 2009 Ciamac Moallemi Outline 1. Administrative matters 2. Introduction 3. Existence of optima 4. Local theory of unconstrained
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EECS 227A Nonlinear and Convex Optimization. Solutions 5 Fall 2009
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 5 Fall 2009 Reading: Boyd and Vandenberghe, Chapter 5 Solution 5.1 Note that
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationLecture 1: Introduction. Outline. B9824 Foundations of Optimization. Fall Administrative matters. 2. Introduction. 3. Existence of optima
B9824 Foundations of Optimization Lecture 1: Introduction Fall 2010 Copyright 2010 Ciamac Moallemi Outline 1. Administrative matters 2. Introduction 3. Existence of optima 4. Local theory of unconstrained
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationLagrange Multipliers
Lagrange Multipliers (Com S 477/577 Notes) Yan-Bin Jia Nov 9, 2017 1 Introduction We turn now to the study of minimization with constraints. More specifically, we will tackle the following problem: minimize
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More information1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 8 February 22nd 1 Overview In the previous lecture we saw characterizations of optimality in linear optimization, and we reviewed the
More informationExamination paper for TMA4180 Optimization I
Department of Mathematical Sciences Examination paper for TMA4180 Optimization I Academic contact during examination: Phone: Examination date: 26th May 2016 Examination time (from to): 09:00 13:00 Permitted
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationIntroduction to Mathematical Programming IE406. Lecture 3. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 3 Dr. Ted Ralphs IE406 Lecture 3 1 Reading for This Lecture Bertsimas 2.1-2.2 IE406 Lecture 3 2 From Last Time Recall the Two Crude Petroleum example.
More informationSF2822 Applied nonlinear optimization, final exam Saturday December
SF2822 Applied nonlinear optimization, final exam Saturday December 20 2008 8.00 13.00 Examiner: Anders Forsgren, tel. 790 71 27. Allowed tools: Pen/pencil, ruler and eraser; plus a calculator provided
More informationComputational Optimization. Convexity and Unconstrained Optimization 1/29/08 and 2/1(revised)
Computational Optimization Convexity and Unconstrained Optimization 1/9/08 and /1(revised) Convex Sets A set S is convex if the line segment joining any two points in the set is also in the set, i.e.,
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationSubgradients. subgradients. strong and weak subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients strong and weak subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE364b, Stanford University Basic inequality recall basic inequality
More informationFranco Giannessi, Giandomenico Mastroeni. Institute of Mathematics University of Verona, Verona, Italy
ON THE THEORY OF VECTOR OPTIMIZATION AND VARIATIONAL INEQUALITIES. IMAGE SPACE ANALYSIS AND SEPARATION 1 Franco Giannessi, Giandomenico Mastroeni Department of Mathematics University of Pisa, Pisa, Italy
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationCoordinate Update Algorithm Short Course Subgradients and Subgradient Methods
Coordinate Update Algorithm Short Course Subgradients and Subgradient Methods Instructor: Wotao Yin (UCLA Math) Summer 2016 1 / 30 Notation f : H R { } is a closed proper convex function domf := {x R n
More informationInequality Constraints
Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationSolution to EE 617 Mid-Term Exam, Fall November 2, 2017
Solution to EE 67 Mid-erm Exam, Fall 207 November 2, 207 EE 67 Solution to Mid-erm Exam - Page 2 of 2 November 2, 207 (4 points) Convex sets (a) (2 points) Consider the set { } a R k p(0) =, p(t) for t
More informationLecture 1: Convex Sets January 23
IE 521: Convex Optimization Instructor: Niao He Lecture 1: Convex Sets January 23 Spring 2017, UIUC Scribe: Niao He Courtesy warning: These notes do not necessarily cover everything discussed in the class.
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationTwo hours. To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER. xx xxxx 2017 xx:xx xx.
Two hours To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER CONVEX OPTIMIZATION - SOLUTIONS xx xxxx 27 xx:xx xx.xx Answer THREE of the FOUR questions. If
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationFinal Exam Advanced Mathematics for Economics and Finance
Final Exam Advanced Mathematics for Economics and Finance Dr. Stefanie Flotho Winter Term /5 March 5 General Remarks: ˆ There are four questions in total. ˆ All problems are equally weighed. ˆ This is
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationMathematical Optimisation, Chpt 2: Linear Equations and inequalities
Introduction Gauss-elimination Orthogonal projection Linear Inequalities Integer Solutions Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationIntroduction to Nonlinear Stochastic Programming
School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationLecture 2: Convex Sets and Functions
Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationSeminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1
Seminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1 Session: 15 Aug 2015 (Mon), 10:00am 1:00pm I. Optimization with
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationAsteroide Santana, Santanu S. Dey. December 4, School of Industrial and Systems Engineering, Georgia Institute of Technology
for Some for Asteroide Santana, Santanu S. Dey School of Industrial Systems Engineering, Georgia Institute of Technology December 4, 2016 1 / 38 1 1.1 Conic integer programs for Conic integer programs
More informationGEOMETRIC APPROACH TO CONVEX SUBDIFFERENTIAL CALCULUS October 10, Dedicated to Franco Giannessi and Diethard Pallaschke with great respect
GEOMETRIC APPROACH TO CONVEX SUBDIFFERENTIAL CALCULUS October 10, 2018 BORIS S. MORDUKHOVICH 1 and NGUYEN MAU NAM 2 Dedicated to Franco Giannessi and Diethard Pallaschke with great respect Abstract. In
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationNonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
More informationLagrangian Duality. Richard Lusby. Department of Management Engineering Technical University of Denmark
Lagrangian Duality Richard Lusby Department of Management Engineering Technical University of Denmark Today s Topics (jg Lagrange Multipliers Lagrangian Relaxation Lagrangian Duality R Lusby (42111) Lagrangian
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationThis article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and
This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution
More informationMathematical Optimisation, Chpt 2: Linear Equations and inequalities
Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl http://dickinson.website version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson
More informationMATHEMATICAL ECONOMICS: OPTIMIZATION. Contents
MATHEMATICAL ECONOMICS: OPTIMIZATION JOÃO LOPES DIAS Contents 1. Introduction 2 1.1. Preliminaries 2 1.2. Optimal points and values 2 1.3. The optimization problems 3 1.4. Existence of optimal points 4
More informationChapter 1. Optimality Conditions: Unconstrained Optimization. 1.1 Differentiable Problems
Chapter 1 Optimality Conditions: Unconstrained Optimization 1.1 Differentiable Problems Consider the problem of minimizing the function f : R n R where f is twice continuously differentiable on R n : P
More informationNonlinear Optimization
Nonlinear Optimization Etienne de Klerk (UvT)/Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos Course WI3031 (Week 4) February-March, A.D. 2005 Optimization Group 1 Outline
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More informationSECTION C: CONTINUOUS OPTIMISATION LECTURE 9: FIRST ORDER OPTIMALITY CONDITIONS FOR CONSTRAINED NONLINEAR PROGRAMMING
Nf SECTION C: CONTINUOUS OPTIMISATION LECTURE 9: FIRST ORDER OPTIMALITY CONDITIONS FOR CONSTRAINED NONLINEAR PROGRAMMING f(x R m g HONOUR SCHOOL OF MATHEMATICS, OXFORD UNIVERSITY HILARY TERM 5, DR RAPHAEL
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationMinimality Concepts Using a New Parameterized Binary Relation in Vector Optimization 1
Applied Mathematical Sciences, Vol. 7, 2013, no. 58, 2841-2861 HIKARI Ltd, www.m-hikari.com Minimality Concepts Using a New Parameterized Binary Relation in Vector Optimization 1 Christian Sommer Department
More information