Lecture 1 Introduction to optimization
|
|
- Marlene Jacobs
- 5 years ago
- Views:
Transcription
1 TMA947 / MMG621 Nonlinear optimization Lecture 1 Introduction to optimization Emil Gustavsson, Zuzana Nedělková October 2, 217 What is optimization? Optimization is a mathematical discipline which is concerned with finding the minima or maxima of functions, possibly subject to constraints. Basic notation Vectors are written with bold face, i.e., x R n. Elements in a vector are written asx j, j = 1,...,n. All vectors are column vectors. The inner product ofaand b is written asa T b = b T a = n j=1 a jb j. The norm denotes the Euclidean norm, i.e., x = x T x = n j=1 x2 j. We utilize vector inequalities, a b, meaning that a j b j, j = 1,...,n. Optimization problem formulation In order to introduce a general optimization problem, we need to define the following: x R n : vector of decision variables x j, j = 1,...,n, f : R n R ± : objective function, X R n : ground set, g i : R n R : constraint function defining restriction onx, g i, i I : inequality constraints, g i =, i E : equality constraints. 1
2 A general optimization problem is to minimize x f(x), (1a) subject to g i (x), i I, (1b) g i (x) =, i E, x X. (1c) (1d) (If we consider a maximization problem, we change the sign off to get a minimization problem.) Classification of optimization problems Linear Programming (LP): - Linear objective function f(x) = c T x = n j=1 cjxj, - Affine constraint functions g i(x) = a T i x b i, i I E - Ground set X defined by affine equalities/inequalities. Nonlinear programming (NLP): - Some functionsf,g i, i I E are nonlinear. Unconstrained optimization: - I E =, - X = R n. Constrained optimization: - I E =, and/or - X R n. Integer programming (IP): - X Z n or X {,1} n. Convex programming (CP): - f,g i, i I are convex functions, - g i, i E are affine, - X is closed and convex. 2
3 Conventions Let S = {x R n g i(x), i I, g i(x) =, i E, x X} denote a feasible set. What do we mean by solving the problem to minimize Let f(x)? f := infimum denote the infimum value of f over the set S. If the valuef is attained at some pointx ins, we can write f := minimum f(x) f(x), and havef(x ) = f. Another well-defined operator defines the set of minimal solutions to the problem S := arg minimum f(x), wheres S is nonempty if and only if the infimum valuef is attained at some pointx ins. Now we can define what we mean by the problem to minimize f(x). "to minimize f(x)" means "findf and an x S " If we have an optimization problem P : minimize f(x) A point x is feasible in problemp if x S. The point is infeasible in problemp ifx / S. The problemp is feasible if there exist ax S and the problemp is infeasible ifs =. A point x is an optimal solution top if x arg minimum f is an optimal value top if f = minimum f(x). f(x). Examples I. Consider the problem to minimize (x+1) 2, subject to x R, Easy problem,(x+1) 2 is convex, no constraints. Just solvef (x) =, and get the optimal solutionx = 1 and the optimal valuef =. (Convex, quadratic, unconstrained optimization problem) 3
4 II. A more complicated problem is to minimize (x+1) 2, subject to x. Now the "f (x) = " trick does not work and we need to consider the boundary. We get the optimal solution x = and the optimal valuef = 1. (Convex, quadratic, constrained optimization problem) III. Consider the problem to minimize x 1, subject to x 1 +x 2 1, x 1,x 2. We solve this graphically. So optimal solution is x = (1,) T and the optimal value if f = 1. 1 x 2 f = (1,) T x 1 +x 2 = 1 x = (1,) T 1 x 1 The diet problem As a first example of an real optimization problem, we consider the diet problem (first formulated by George Stigler). For a moderately active person, how much of each of a number of foods should be eaten on a daily basis so that the person s intake of nutrients will be at least equal to the recommended dietary allowances (RDAs), with the cost of the diet being minimal? Good example to show how to model a real optimization problem, why a realistic model sometimes can be difficult to achieve. We consider the case when the only allowed foods can be found at McDonalds. For a moderately active person, how much of each of a number of McDonald foods (see Table 1) should be eaten on a daily basis so that the person s intake of nutrients will be at least equal to the recommended dietary allowances (RDAs), with the cost of the diet being minimal? 4
5 Food Calories Carb Protein Vit A Vit C Calc Iron Cost Big Mac 55 kcal 46g 25g 6% 2% 25% 25% 3kr Cheeseburger 3 kcal 33g 15g 6% 2% 2% 15% 1kr McChicken 36 kcal 4g 14g % 2% 1% 15% 35kr McNuggets 28 kcal 18g 13g % 2% 2% 4% 4kr Caesar Sallad 35 kcal 24g 23g 16% 35% 2% 1% 5kr French Fries 38 kcal 48g 4g % 15% 2% 6% 2kr Apple Pie 25 kcal 32g 2g 4% 25% 2% 6% 1kr Coca-Cola 21 kcal 58g g % % % % 15kr Milk 1 kcal 12g 8g 1% 4% 3% 8% 15kr Orange Juice 15 kcal 3g 2g % 14% 2% % 15kr RDA 2 kcal 35g 55g 1% 1% 1% 1% Table 1: Given data for the diet problem We define the sets Foods := {Big Mac, Cheeseburger, McChicken, McNuggets, Caesar Sallad French Fried, Apple Pie, Coca-Cola, Milk, Orange Juice}, Nutrients := {Calories, Carb, Protein, Vit A, Vit C, Calc, Iron.} Then we define the parameters a ij = Amount of nutrientiin food j, i Nutrients, j Foods, b i = Recommended daily amount (RDA) of nutrienti, i Nutrients, c j = Cost for food j, j Foods, and the decision variables x j = Amount of food j we should eat each day, j Foods. The model of the diet optimization problem is then to minimize subject to c jx j, j Foods j Foods a ijx j b i, i Nutrients, x j, j Foods. (2a) (2b) (2c) (2a) We minimize the total cost, such that (2b) we get enough of each nutrient, and such that (2c) we don t sell anything to McDonalds. 5
6 The optimal solution is then Total cost Total intake of calories x = = kr. = kcal. x Big Mac x Cheeseburger x McChicken x McNuggets x Caesar Sallad x French Fries x Apple Pie x Coca Cola x Milk x Orange Juice 7.48 = If we add the constraint that x j should be integer, the solution is x Big Mac x Cheeseburger 7 x McChicken x McNuggets x = x Caesar Sallad x French Fries = 1. x Apple Pie 3 x Coca Cola x Milk x Orange Juice Total cost Total intake of calories = 15 kr. = 32 kcal. Now consider going on a diet, meaning that we would like to eat as few calories as possible. We reformulate our model to minimize subject to j Foods a Calories,j x j, j Foods a ijx j b i, i Nutrients\{Calories}, x j, j Foods. (3a) (3b) (3c) The optimal solution is then x = x Big Mac x Cheeseburger x McChicken x McNuggets x Caesar Sallad x French Fries x Apple Pie x Coca Cola x Milk x Orange Juice =
7 Total cost Total intake of calories = kr. = kcal. If we add the constraint that x j should be integer, the solution is x = x Big Mac x Cheeseburger x McChicken x McNuggets x Caesar Sallad x French Fries x Apple Pie x Coca Cola x Milk x Orange Juice = Total cost Total intake of calories = 27 kr. = 221 kcal. The real diet problem When first studied by the Stigler, the problem concerned the US military and had 77 different foods in the model. He didn t managed to solve the problem to optimality, but almost. The near optimal diet was Wheat flour Evaporated milk Cabbage Spinach Dried navy beans at a cost of $.1 a day in 1939 US dollars. 7
8 Course material Lecture 1 Lecture 2 Lecture 3 Lecture 4 Lecture 5 Lecture 6 Lecture 7 Lecture 8 Lecture 9 Lecture 1 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Define and model optimization problems, classification Convexity of sets, functions, optimization problems Optimality conditions, introduction Unconstrained optimization, methods, classification. Optimality conditions, continued The Karush-Kuhn-Tucker conditions Lagrangian duality Linear programming, introduction Linear programming, continued Duality in linear programming Convex optimization Integer programming Nonlinear optimization methods, convex feasible sets Nonlinear optimization methods, general sets Overview of the course 8
36106 Managerial Decision Modeling Linear Decision Models: Part II
1 36106 Managerial Decision Modeling Linear Decision Models: Part II Kipp Martin University of Chicago Booth School of Business January 20, 2014 Reading and Excel Files Reading (Powell and Baker): Sections
More informationConvex Optimization. Linear Programming Applications. ENSAE: Optimisation 1/40
Convex Optimization Linear Programming Applications ENSAE: Optimisation 1/40 Today This is the What s the point? lecture... What can be solved using linear programs? Just an introduction... ENSAE: Optimisation
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More informationNumerical Optimization
Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,
More informationPlanning and Optimization
Planning and Optimization C23. Linear & Integer Programming Malte Helmert and Gabriele Röger Universität Basel December 1, 2016 Examples Linear Program: Example Maximization Problem Example maximize 2x
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a two-class classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationGenerality of Languages
Increasing generality Decreasing efficiency Generality of Languages Linear Programs Reduction Algorithm Simplex Algorithm, Interior Point Algorithms Min Cost Flow Algorithm Klein s algorithm Reduction
More informationOptimization (2MMD10/2DME20), lecture 1b. Gerhard Woeginger. Technische Universiteit Eindhoven
Optimization (2MMD10/2DME20), lecture 1b Gerhard Woeginger Technische Universiteit Eindhoven Fall 2015, Q1 GJ Woeginger Optimization (2MMD10/2DME20), lecture 1b 1/26 Our program for week 1 Tuesday: lots
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationSupport Vector Machines
Wien, June, 2010 Paul Hofmarcher, Stefan Theussl, WU Wien Hofmarcher/Theussl SVM 1/21 Linear Separable Separating Hyperplanes Non-Linear Separable Soft-Margin Hyperplanes Hofmarcher/Theussl SVM 2/21 (SVM)
More informationCSCI5654 (Linear Programming, Fall 2013) Lecture-8. Lecture 8 Slide# 1
CSCI5654 (Linear Programming, Fall 2013) Lecture-8 Lecture 8 Slide# 1 Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables.
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationLecture 3 Introduction to optimality conditions
TMA947 / MMG621 Nonlinear optimisation Lecture 3 Introduction to optimality conditions Emil Gustavsson October 31, 2013 Local and global optimality We consider an optimization problem which is that to
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationMachine Learning. Support Vector Machines. Manfred Huber
Machine Learning Support Vector Machines Manfred Huber 2015 1 Support Vector Machines Both logistic regression and linear discriminant analysis learn a linear discriminant function to separate the data
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationIOE510/MATH561/OMS518: Linear Programming I
IOE510/MATH561/OMS518: Linear Programming I Lectures: Mondays and Wednesdays, 9:00-10:30 AM, IOE 1680 Instructor: Prof. Marina A. Epelman GSI: Katharina Ley Office hours: See CTools for most current info
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationCS711008Z Algorithm Design and Analysis
.. CS711008Z Algorithm Design and Analysis Lecture 9. Algorithm design technique: Linear programming and duality Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 12: Nonlinear optimization, continued Prof. John Gunnar Carlsson October 20, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 20,
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationOn the Method of Lagrange Multipliers
On the Method of Lagrange Multipliers Reza Nasiri Mahalati November 6, 2016 Most of what is in this note is taken from the Convex Optimization book by Stephen Boyd and Lieven Vandenberghe. This should
More informationNumerical Optimization
Linear Programming Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on min x s.t. Transportation Problem ij c ijx ij 3 j=1 x ij a i, i = 1, 2 2 i=1 x ij
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More informationNonlinear Optimization: The art of modeling
Nonlinear Optimization: The art of modeling INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org)
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationCOMP3121/9101/3821/9801 Lecture Notes. Linear Programming
COMP3121/9101/3821/9801 Lecture Notes Linear Programming LiC: Aleks Ignjatovic THE UNIVERSITY OF NEW SOUTH WALES School of Computer Science and Engineering The University of New South Wales Sydney 2052,
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationLinear Programming Duality
Summer 2011 Optimization I Lecture 8 1 Duality recap Linear Programming Duality We motivated the dual of a linear program by thinking about the best possible lower bound on the optimal value we can achieve
More informationJanuary 29, Introduction to optimization and complexity. Outline. Introduction. Problem formulation. Convexity reminder. Optimality Conditions
Olga Galinina olga.galinina@tut.fi ELT-53656 Network Analysis Dimensioning II Department of Electronics Communications Engineering Tampere University of Technology, Tampere, Finl January 29, 2014 1 2 3
More informationTutorial on Convex Optimization for Engineers Part II
Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationApplied Lagrange Duality for Constrained Optimization
Applied Lagrange Duality for Constrained Optimization February 12, 2002 Overview The Practical Importance of Duality ffl Review of Convexity ffl A Separating Hyperplane Theorem ffl Definition of the Dual
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationLecture Note 1: Introduction to optimization. Xiaoqun Zhang Shanghai Jiao Tong University
Lecture Note 1: Introduction to optimization Xiaoqun Zhang Shanghai Jiao Tong University Last updated: September 23, 2017 1.1 Introduction 1. Optimization is an important tool in daily life, business and
More informationQuiz Discussion. IE417: Nonlinear Programming: Lecture 12. Motivation. Why do we care? Jeff Linderoth. 16th March 2006
Quiz Discussion IE417: Nonlinear Programming: Lecture 12 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University 16th March 2006 Motivation Why do we care? We are interested in
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationOutline. Outline. Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING. 1. Scheduling CPM/PERT Resource Constrained Project Scheduling Model
Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING Lecture 3 and Mixed Integer Programg Marco Chiarandini 1. Resource Constrained Project Model 2. Mathematical Programg 2 Outline Outline 1. Resource Constrained
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationResource Constrained Project Scheduling Linear and Integer Programming (1)
DM204, 2010 SCHEDULING, TIMETABLING AND ROUTING Lecture 3 Resource Constrained Project Linear and Integer Programming (1) Marco Chiarandini Department of Mathematics & Computer Science University of Southern
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More information6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE. Three Alternatives/Remedies for Gradient Projection
6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE Three Alternatives/Remedies for Gradient Projection Two-Metric Projection Methods Manifold Suboptimization Methods
More informationChapter 3 Introduction to Linear Programming PART 1. Assoc. Prof. Dr. Arslan M. Örnek
Chapter 3 Introduction to Linear Programming PART 1 Assoc. Prof. Dr. Arslan M. Örnek http://homes.ieu.edu.tr/~aornek/ise203%20optimization%20i.htm 1 3.1 What Is a Linear Programming Problem? Linear Programming
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationECE 307- Techniques for Engineering Decisions
ECE 307- Techniques for Engineering Decisions Lecture 4. Dualit Concepts in Linear Programming George Gross Department of Electrical and Computer Engineering Universit of Illinois at Urbana-Champaign DUALITY
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationSection Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018
Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationSupport Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2014 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
More informationMachine Learning Support Vector Machines. Prof. Matteo Matteucci
Machine Learning Support Vector Machines Prof. Matteo Matteucci Discriminative vs. Generative Approaches 2 o Generative approach: we derived the classifier from some generative hypothesis about the way
More informationOperations Research Lecture 6: Integer Programming
Operations Research Lecture 6: Integer Programming Notes taken by Kaiquan Xu@Business School, Nanjing University May 12th 2016 1 Integer programming (IP) formulations The integer programming (IP) is the
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More information9. Geometric problems
9. Geometric problems EE/AA 578, Univ of Washington, Fall 2016 projection on a set extremal volume ellipsoids centering classification 9 1 Projection on convex set projection of point x on set C defined
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationN. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:
0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything
More informationLecture 1 Introduction
L. Vandenberghe EE236A (Fall 2013-14) Lecture 1 Introduction course overview linear optimization examples history approximate syllabus basic definitions linear optimization in vector and matrix notation
More informationChapter 1: Linear Programming
Chapter 1: Linear Programming Math 368 c Copyright 2013 R Clark Robinson May 22, 2013 Chapter 1: Linear Programming 1 Max and Min For f : D R n R, f (D) = {f (x) : x D } is set of attainable values of
More informationMarch 13, Duality 3
15.53 March 13, 27 Duality 3 There are concepts much more difficult to grasp than duality in linear programming. -- Jim Orlin The concept [of nonduality], often described in English as "nondualism," is
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko
More informationOptimization: an Overview
Optimization: an Overview Moritz Diehl University of Freiburg and University of Leuven (some slide material was provided by W. Bangerth and K. Mombaur) Overview of presentation Optimization: basic definitions
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationLagrange Relaxation and Duality
Lagrange Relaxation and Duality As we have already known, constrained optimization problems are harder to solve than unconstrained problems. By relaxation we can solve a more difficult problem by a simpler
More informationNonlinear Programming and the Kuhn-Tucker Conditions
Nonlinear Programming and the Kuhn-Tucker Conditions The Kuhn-Tucker (KT) conditions are first-order conditions for constrained optimization problems, a generalization of the first-order conditions we
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationLECTURE 7 Support vector machines
LECTURE 7 Support vector machines SVMs have been used in a multitude of applications and are one of the most popular machine learning algorithms. We will derive the SVM algorithm from two perspectives:
More informationKarush-Kuhn-Tucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Karush-Kuhn-Tucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =
More informationOptimization Models and Applications
Optimization Models and Applications Martin Takáč ISE 316, Fall 2014, Lecture 1 September 3, 2014 Martin Takáč ISE 316 Fall 2014 1 / 33 Outline Course Information Example of an Optimization Problem Terminology
More informationLecture Notes on Support Vector Machine
Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ω T x + b = 0 (1) where ω R n is
More informationConstrained optimization
Constrained optimization In general, the formulation of constrained optimization is as follows minj(w), subject to H i (w) = 0, i = 1,..., k. where J is the cost function and H i are the constraints. Lagrange
More informationSupport Vector Machines and Kernel Methods
2018 CS420 Machine Learning, Lecture 3 Hangout from Prof. Andrew Ng. http://cs229.stanford.edu/notes/cs229-notes3.pdf Support Vector Machines and Kernel Methods Weinan Zhang Shanghai Jiao Tong University
More informationNONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)
NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More information