IE 5531 Midterm #2 Solutions


 Julius Lawrence
 1 years ago
 Views:
Transcription
1 IE 5531 Midterm #2 s Prof. John Gunnar Carlsson November 9, 2011 Before you begin: This exam has 9 pages and a total of 5 problems. Make sure that all pages are present. To obtain credit for a problem, you must show all your work. if you use a formula to answer a problem, write the formula down. Do not open this exam until instructed to do so. 1
2 Problem 1: True or False (10 points) State whether each of the following statements is true or false. Assume all auxiliary information given is correct. For each part, only your answer, which should be one of True or False, will be graded. Any explanation will not be read. 1. Although its worstcase running time is faster than that of the simplex method, the ellipsoid method is rarely used because it performs poorly in practical situations. True. 2. One disadvantage of the Wolfe linesearch conditions is that, in any given iteration, we may erroneously remove local minimizers from the search space. False. (local minimizers have φ (α) = 0, which is never discarded) 3. At a point satisfying the KKT conditions, if a Lagrange multiplier λ of a constraint is 0, then that constraint must be inactive. False. Minimizing x 2 subject to x 0 has a zero Lagrange multiplier but the constraint is active at optimality.. Consider the problem minimize f (x) a T x b where f (x) is a convex function. If x satises a T x < b and f (x ) = 0, then x must be a globally optimal solution to our problem. True. s.t. 5. Newton's method, applied to the function below, should converge to the marked root of the function if our initial guess lies in the shaded interval. True. 2
3 Problem 2: Line search conditions (20 points) Suppose that you are performing a line search to nd an unconstrained minimum of the function f (x) in R n. At the current iteration, you have chosen your descent direction d k and you are currently trying to determine the optimal step length α k. Sketch the ranges of feasible step lengths in the gure below for the Goldstein conditions. You may assume that φ (0) = f (x k ) T d k = 1, as suggested by the gure. Assume that c = 1/ and consequently that 1 c = 3/. 3
4 Problem 3: CobbDouglas Utility Function (20 points) The CobbDouglas production function is widely used in economics to represent the relationship between inputs and outputs of a rm. It takes the form Y = AL α K β where Y represents output, L labor, and K capital. The parameters α and β are constants that determine how production is scaled. The CobbDouglas function can also be applied to utility maximization and takes the general form N i=1 xαi i. Consider the following utility maximization problem: maximize u (x) = x α 1 x 1 α 2 s.t. p 1 x 1 + p 2 x 2 w Here p is a price vector for x 1 and x 2 and w represents a user's budget. Assume that p > 0 and w > Perform an appropriate transformation to turn this into a minimization problem with a convex objective function. Taking logarithms and multiplying by 1 we obtain the equivalent convex problem minimize [α log x 1 + (1 α) log x 2 ] s.t. p 1 x 1 + p 2 x 2 w 2. Write the KKT conditions and nd an explicit solution for x as a function of α, p, and w. (Hint: can you assume that certain constraints must be tight? Can you assume that certain constraints must be slack?) Are these conditions sucient for optimality? The KKT conditions for the transformed problem are ( ) ( ) ( ) ( α/x 1 p λ (1 α)/x 0 + λ 2 p 1 + λ ) = 0 λ 0 (p 1 x 1 + p 2 x 2 w) = 0 λ 1 x 1 = 0 λ 2 x 2 = 0 p 1 x 1 + p 2 x 2 w The constraint p 1 x 1 + p 2 x 2 w must be tight because the objective function of the transformed problem is monotonically decreasing in x 1 and x 2. The sign constraints on x 1 and x 2 are slack because otherwise the objective function of the transformed problem is, and thus λ 1 = λ 2 = 0. The conditions above therefore tell us that α/x 1 + λ 0 p 1 = 0 (1 α)/x 2 + λ 0 p 2 = 0 p 1 x 1 + p 2 x 2 = w which admits the unique solution λ 0 = 1/w so that x 1 = αw/p 1 and x 2 = (1 α)w/p Find the corresponding Lagrange multiplier for the budget constraint and give an interpretation of it. The Lagrange multiplier is λ 0 = 1/w; the economic interpretation is that it represents the marginal benet of one extra unit of budget.
5 This page intentionally left blank. 5
6 Problem : A linearly constrained problem (25 points) optimization problem: Consider the following linearly constrained convex minimize x 2 1 6x 1 + x 3 2 3x 2 s.t. x 1 + x Derive the KKT conditions for this problem. The KKT conditions are 2x λ 0 λ 1 = 0 3x λ 0 λ 2 = 0 λ 0 (x 1 + x 2 1) = 0 λ 1 x 1 = 0 λ 2 x 2 = 0 x 1 + x Determine whether or not x = (1/2, 1/2) is an optimal solution. If x = (1/2, 1/2), then by complementary slackness we must have λ 1 = λ 2 = 0. However, the rst two equations tell us that 2x λ 0 λ 1 = 0 3x λ 0 λ 2 = 0 so that 2(1/2) 6 + λ 0 = 3(1/2) λ 0 = 0, which cannot happen, and thus (1/2, 1/2) is not optimal. 3. Given that the optimal solution lies on a corner point of the feasible region, nd the optimal solution to the above problem. The feasible region has three corner points: (0, 0), (1, 0), and (0, 1). By inspection we nd that the optimal point is (1, 0). 6
7 This page intentionally left blank. 7
8 Problem 5: Interior point method for LP (25 points) Suppose we want to solve the following linear program: minimize x 1 + x 2 s.t. x 1 + x 2 + x 3 = 1 x 1, x 2, x 3 0 We will use a barrier method on the inequality constraints (not the primaldual potential function!) to solve this problem. 1. Write the KKT conditions for the barrierfunction relaxation to this linear program. The problem is and the KKT conditions are minimize x 1 + x 2 µ (log x 1 + log x 2 + log x 3 ) 1 µ/x 1 + λ 0 = 0 1 µ/x 2 + λ 0 = 0 µ/x 3 + λ 0 = 0 x 1 + x 2 + x 3 = 1 x 1 + x 2 + x 3 = 1 2. Show that for every µ > 0, the optimal solution to the barrierfunction relaxation is: x 1 (µ) = 1 + 3µ 1 + 9µ 2 2µ x 2 (µ) = x 1 (µ) x 3 (µ) = 1 2x 1 (µ) s.t. The KKT conditions give us four equations in four unknowns. Isolating λ 0 we can write 1 µ/x 1 = 1 µ/x 2 = µ/x 3 x 1 + x 2 + x 3 = 1 which tells us that x 2 = x 1 at optimality. Since x 1 + x 2 + x 3 = 1 this tells us that x 3 = 1 2x 1. To solve for x 1 we write 1 µ/x 1 = µ/x 3 = µ/(1 2x 1 ) which gives a quadratic equation in x 1. Solving this equation gives us x 1 = 1 + 3µ ± 1 + 9µ 2 2µ We have two options for the ± sign, since both give feasible solutions. However, the objective function at x 1 = x 2 = 1+3µ+ 1+9µ 2 2µ is strictly greater than the objective function at x 1 = x 2 = 1+3µ 1+9µ 2 2µ, and the desired result follows. 3. Show that as µ 0, the optimal solution to the barrier problem approaches the unique optimal solution.. It is straightforward to verify that lim µ 0 (x 1 (µ), x 2 (µ), x 3 (µ)) = (0, 0, 1).. (Bonus; 5 points) Suppose that our objective function is just to minimize x 1. Where does the solution converge to now? Using the same approach the solution converges to (0, 1/2, 1/2). 8
9 This page intentionally left blank. 9
IE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 19: Midterm 2 Review Prof. John Gunnar Carlsson November 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 22, 2010 1 / 34 Administrivia
More informationIE 5531 Practice Midterm #2
IE 5531 Practice Midterm #2 Prof. John Gunnar Carlsson November 23, 2010 Problem 1: Nonlinear programming You are a songwriter who writes Top 40 style songs for the radio. Each song you write can be described
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 15: Nonlinear optimization Prof. John Gunnar Carlsson November 1, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 1, 2010 1 / 24
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.14.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 13 line supporting sentence
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equalityconstrained Optimization Inequalityconstrained Optimization Mixtureconstrained Optimization 3 Quadratic Programming
More informationLecture 19: Optimization Programming
Fall, 2017 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equalityconstrained Optimization Inequalityconstrained Optimization Mixtureconstrained Optimization 3 Quadratic Programming
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 14: Unconstrained optimization Prof. John Gunnar Carlsson October 27, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 27, 2010 1
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 12: Nonlinear optimization, continued Prof. John Gunnar Carlsson October 20, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 20,
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEME7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEME7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationEcon Slides from Lecture 14
Econ 205 Sobel Econ 205  Slides from Lecture 14 Joel Sobel September 10, 2010 Theorem ( Lagrange Multipliers ) Theorem If x solves max f (x) subject to G(x) = 0 then there exists λ such that Df (x ) =
More informationDEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION. Part I: Short Questions
DEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION Part I: Short Questions August 12, 2008 9:00 am  12 pm General Instructions This examination is
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202220 Time: 4:009:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationTMA947/MAN280 APPLIED OPTIMIZATION
Chalmers/GU Mathematics EXAM TMA947/MAN280 APPLIED OPTIMIZATION Date: 06 08 31 Time: House V, morning Aids: Text memoryless calculator Number of questions: 7; passed on one question requires 2 points
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a twoclass classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationLecture 13: Constrained optimization
20101203 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LPproblem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationReview Solutions, Exam 2, Operations Research
Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More informationMark your answers ON THE EXAM ITSELF. If you are not sure of your answer you may wish to provide a brief explanation.
CS 189 Spring 2015 Introduction to Machine Learning Midterm You have 80 minutes for the exam. The exam is closed book, closed notes except your onepage crib sheet. No calculators or electronic items.
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and FuJie Huang 1 Outline What is behind
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of WisconsinMadison May 2014 Wright (UWMadison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationLinear Programming Duality
Summer 2011 Optimization I Lecture 8 1 Duality recap Linear Programming Duality We motivated the dual of a linear program by thinking about the best possible lower bound on the optimal value we can achieve
More informationLecture 10: Linear programming duality and sensitivity 00
Lecture 10: Linear programming duality and sensitivity 00 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 00 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationFIN 550 Exam answers. A. Every unconstrained problem has at least one interior solution.
FIN 0 Exam answers Phil Dybvig December 3, 0. TrueFalse points A. Every unconstrained problem has at least one interior solution. False. An unconstrained problem may not have any solution at all. For
More information1. f(β) 0 (that is, β is a feasible point for the constraints)
xvi 2. The lasso for linear models 2.10 Bibliographic notes Appendix Convex optimization with constraints In this Appendix we present an overview of convex optimization concepts that are particularly useful
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 function with f () on (, L) and that you have explicit formulae for
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationLP. Kap. 17: Interiorpoint methods
LP. Kap. 17: Interiorpoint methods the simplex algorithm moves along the boundary of the polyhedron P of feasible solutions an alternative is interiorpoint methods they find a path in the interior of
More information21. Solve the LP given in Exercise 19 using the bigm method discussed in Exercise 20.
Extra Problems for Chapter 3. Linear Programming Methods 20. (BigM Method) An alternative to the twophase method of finding an initial basic feasible solution by minimizing the sum of the artificial
More informationPart 4: Activeset methods for linearly constrained optimization. Nick Gould (RAL)
Part 4: Activeset methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where
More informationPrimaldual Subgradient Method for Convex Problems with Functional Constraints
Primaldual Subgradient Method for Convex Problems with Functional Constraints Yurii Nesterov, CORE/INMA (UCL) Workshop on embedded optimization EMBOPT2014 September 9, 2014 (Lucca) Yu. Nesterov Primaldual
More informationContinuous Optimisation, Chpt 6: Solution methods for Constrained Optimisation
Continuous Optimisation, Chpt 6: Solution methods for Constrained Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version:
More informationInteriorPoint Methods for Linear Optimization
InteriorPoint Methods for Linear Optimization Robert M. Freund and Jorge Vera March, 204 c 204 Robert M. Freund and Jorge Vera. All rights reserved. Linear Optimization with a Logarithmic Barrier Function
More informationInterior Point Methods for Mathematical Programming
Interior Point Methods for Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Florianópolis, Brazil EURO  2013 Roma Our heroes Cauchy Newton Lagrange Early results Unconstrained
More informationMATH 210 EXAM 3 FORM A November 24, 2014
MATH 210 EXAM 3 FORM A November 24, 2014 Name (printed) Name (signature) ZID No. INSTRUCTIONS: (1) Use a No. 2 pencil. (2) Work on this test. No scratch paper is allowed. (3) Write your name and ZID number
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 5: The Simplex method, continued Prof. John Gunnar Carlsson September 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 22, 2010
More informationPrimal/Dual Decomposition Methods
Primal/Dual Decomposition Methods Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470  Convex Optimization Fall 201819, HKUST, Hong Kong Outline of Lecture Subgradients
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470  Convex Optimization Fall 201718, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationExam in TMA4180 Optimization Theory
Norwegian University of Science and Technology Department of Mathematical Sciences Page 1 of 11 Contact during exam: Anne Kværnø: 966384 Exam in TMA418 Optimization Theory Wednesday May 9, 13 Tid: 9. 13.
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Algorithms for Constrained Nonlinear Optimization Problems) Tamás TERLAKY Computing and Software McMaster University Hamilton, November 2005 terlaky@mcmaster.ca
More information10725/ Optimization Midterm Exam
10725/36725 Optimization Midterm Exam November 6, 2012 NAME: ANDREW ID: Instructions: This exam is 1hr 20mins long Except for a single twosided sheet of notes, no other material or discussion is permitted
More informationLINEAR AND NONLINEAR PROGRAMMING
LINEAR AND NONLINEAR PROGRAMMING Stephen G. Nash and Ariela Sofer George Mason University The McGrawHill Companies, Inc. New York St. Louis San Francisco Auckland Bogota Caracas Lisbon London Madrid Mexico
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationmin 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14
The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 025 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,
More informationLagrangian Duality for Dummies
Lagrangian Duality for Dummies David Knowles November 13, 2010 We want to solve the following optimisation problem: f 0 () (1) such that f i () 0 i 1,..., m (2) For now we do not need to assume conveity.
More informationToday s class. Constrained optimization Linear programming. Prof. Jinbo Bi CSE, UConn. Numerical Methods, Fall 2011 Lecture 12
Today s class Constrained optimization Linear programming 1 Midterm Exam 1 Count: 26 Average: 73.2 Median: 72.5 Maximum: 100.0 Minimum: 45.0 Standard Deviation: 17.13 Numerical Methods Fall 2011 2 Optimization
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Nonlinear programming In case of LP, the goal
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationOPTIMISATION /09 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2 2008/09 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and
More informationLecture 10: Linear programming. duality. and. The dual of the LP in standard form. maximize w = b T y (D) subject to A T y c, minimize z = c T x (P)
Lecture 10: Linear programming duality Michael Patriksson 19 February 2004 00 The dual of the LP in standard form minimize z = c T x (P) subject to Ax = b, x 0 n, and maximize w = b T y (D) subject to
More informationKernel Machines. Pradeep Ravikumar Coinstructor: Manuela Veloso. Machine Learning
Kernel Machines Pradeep Ravikumar Coinstructor: Manuela Veloso Machine Learning 10701 SVM linearly separable case n training points (x 1,, x n ) d features x j is a ddimensional vector Primal problem:
More informationAlgorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationInterior Point Methods in Mathematical Programming
Interior Point Methods in Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Brazil Journées en l honneur de Pierre Huard Paris, novembre 2008 01 00 11 00 000 000 000 000
More informationECON 304 MIDTERM EXAM ANSWERS
ECON 30 MIDTERM EXAM ANSWERS () The short questions: (a) Transitivity says that if y and y z, then z. Note the three bundles in diagram 0.. y because they are on the same indifference curve. y z because
More informationPrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method
PrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationQuiz Discussion. IE417: Nonlinear Programming: Lecture 12. Motivation. Why do we care? Jeff Linderoth. 16th March 2006
Quiz Discussion IE417: Nonlinear Programming: Lecture 12 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University 16th March 2006 Motivation Why do we care? We are interested in
More informationCS269: Machine Learning Theory Lecture 16: SVMs and Kernels November 17, 2010
CS269: Machine Learning Theory Lecture 6: SVMs and Kernels November 7, 200 Lecturer: Jennifer Wortman Vaughan Scribe: Jason Au, Ling Fang, Kwanho Lee Today, we will continue on the topic of support vector
More informationFundamentals of Operations Research. Prof. G. Srinivasan. Indian Institute of Technology Madras. Lecture No. # 15
Fundamentals of Operations Research Prof. G. Srinivasan Indian Institute of Technology Madras Lecture No. # 15 Transportation Problem  Other Issues Assignment Problem  Introduction In the last lecture
More informationCSCI 1951G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cashflow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More informationMore FirstOrder Optimization Algorithms
More FirstOrder Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More informationFIN 550 Practice Exam Answers. A. Linear programs typically have interior solutions.
FIN 550 Practice Exam Answers Phil Dybvig. TrueFalse 25 points A. Linear programs typically have interior solutions. False. Unless the objective is zero, all solutions are at the boundary. B. A local
More informationEcon 172A, Fall 2007: Midterm A
Econ 172A, Fall 2007: Midterm A Instructions The examination has 5 questions. Answer them all. You must justify your answers to Questions 1, 2, and 5. (if you are not certain what constitutes adequate
More informationBindel, Spring 2017 Numerical Analysis (CS 4220) Notes for So far, we have considered unconstrained optimization problems.
Consider constraints Notes for 20170424 So far, we have considered unconstrained optimization problems. The constrained problem is minimize φ(x) s.t. x Ω where Ω R n. We usually define x in terms of
More informationConvex Optimization. Prof. Nati Srebro. Lecture 12: InfeasibleStart Newton s Method Interior Point Methods
Convex Optimization Prof. Nati Srebro Lecture 12: InfeasibleStart Newton s Method Interior Point Methods Equality Constrained Optimization f 0 (x) s. t. A R p n, b R p Using access to: 2 nd order oracle
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More information12. Interiorpoint methods
12. Interiorpoint methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationIn view of (31), the second of these is equal to the identity I on E m, while this, in view of (30), implies that the first can be written
11.8 Inequality Constraints 341 Because by assumption x is a regular point and L x is positive definite on M, it follows that this matrix is nonsingular (see Exercise 11). Thus, by the Implicit Function
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More information4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP Different spaces and objective functions but in general same optimal
More informationMultidisciplinary System Design Optimization (MSDO)
Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology  Prof. de Weck and Prof. Willcox Today s Topics Sequential
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationAdvanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras
Advanced Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Lecture  3 Simplex Method for Bounded Variables We discuss the simplex algorithm
More informationThe KuhnTucker Problem
Natalia Lazzati Mathematics for Economics (Part I) Note 8: Nonlinear Programming  The KuhnTucker Problem Note 8 is based on de la Fuente (2000, Ch. 7) and Simon and Blume (1994, Ch. 18 and 19). The KuhnTucker
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationOperations Research Lecture 4: Linear Programming Interior Point Method
Operations Research Lecture 4: Linear Programg Interior Point Method Notes taen by Kaiquan Xu@Business School, Nanjing University April 14th 2016 1 The affine scaling algorithm one of the most efficient
More informationAM 121 Introduction to Optimization: Models and Methods Example Questions for Midterm 1
AM 121 Introduction to Optimization: Models and Methods Example Questions for Midterm 1 Prof. Yiling Chen Fall 2018 Here are some practice questions to help to prepare for the midterm. The midterm will
More informationMarch 13, Duality 3
15.53 March 13, 27 Duality 3 There are concepts much more difficult to grasp than duality in linear programming.  Jim Orlin The concept [of nonduality], often described in English as "nondualism," is
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationECON 582: Dynamic Programming (Chapter 6, Acemoglu) Instructor: Dmytro Hryshko
ECON 582: Dynamic Programming (Chapter 6, Acemoglu) Instructor: Dmytro Hryshko Indirect Utility Recall: static consumer theory; J goods, p j is the price of good j (j = 1; : : : ; J), c j is consumption
More information1 Computing with constraints
Notes for 20170426 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationLinear Programming, Lecture 4
Linear Programming, Lecture 4 Corbett Redden October 3, 2016 Simplex Form Conventions Examples Simplex Method To run the simplex method, we start from a Linear Program (LP) in the following standard simplex
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 20182019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationPart IB Optimisation
Part IB Optimisation Theorems Based on lectures by F. A. Fischer Notes taken by Dexter Chua Easter 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after
More informationLinear Programming, Lecture 4
Linear Programming, Lecture 4 Corbett Redden October 1, 2018 Simplex Form Conventions Examples Standard Form For an LP, Standard Form is usually defined as Maximize c T x subject to Ax b or or Maximize
More informationMathematical Foundations 1 Constrained Optimization. Constrained Optimization. An intuitive approach 2. First Order Conditions (FOC) 7
Mathematical Foundations  Constrained Optimization Constrained Optimization An intuitive approach First Order Conditions (FOC) 7 Constraint qualifications 9 Formal statement of the FOC for a maximum
More informationCS 323: Numerical Analysis and Computing
CS 323: Numerical Analysis and Computing MIDTERM #2 Instructions: This is an open notes exam, i.e., you are allowed to consult any textbook, your class notes, homeworks, or any of the handouts from us.
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More informationSupport Vector Machines
Support Vector Machines Support vector machines (SVMs) are one of the central concepts in all of machine learning. They are simply a combination of two ideas: linear classification via maximum (or optimal
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem PrimalDual Relationship Economic Interpretation
More informationMarch 8, 2010 MATH 408 FINAL EXAM SAMPLE
March 8, 200 MATH 408 FINAL EXAM SAMPLE EXAM OUTLINE The final exam for this course takes place in the regular course classroom (MEB 238) on Monday, March 2, 8:300:20 am. You may bring twosided 8 page
More informationMath 354 Summer 2004 Solutions to review problems for Midterm #1
Solutions to review problems for Midterm #1 First: Midterm #1 covers Chapter 1 and 2. In particular, this means that it does not explicitly cover linear algebra. Also, I promise there will not be any proofs.
More information