IE 5531: Engineering Optimization I

Transcription

1 IE 5531: Engineering Optimization I Lecture 1: Introduction Prof. John Gunnar Carlsson September 8, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

2 Administrivia TEMPORARY web site (for syllabus, etc.): Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

3 What is it? Optimization concerns the formulation and solution of mathematical models for maximizing or minimizing things Make me the fastest car Make me the most energy-ecient refrigerator Make me the cheapest computer Build the mousetrap that will make me the most money Tell me how I should invest my money to minimize my risk Economics, nance, science, business, government, sports, marketing Nothing at all takes place in the Universe in which some rule of maximum or minimum does not appear. Leonhard Euler, Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

4 History BC: Euclid considers the minimal distance between a point and a line; Heron proves that light reected in a mirror travels between two points through the path with shortest length 1600s: Kepler considers the problem of optimally choosing a second wife 1, then writes a famous treatise on optimal shape of wine barrels while shopping for his wedding; Newton and Leibniz discover calculus 1800s: Legendre presents the method of least squares; steepest descent method proposed; rst formulation of linear program 1 Now a famous optimization problem. Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

5 History As a young trial lawyer, Abraham Lincoln thought of one of the earliest examples of a classic optimization question: How can I visit all of my destinations most eciently? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

6 History Modern optimization takes its origins in convex analysis, which is slightly over 100 years old In the World War II era, operations research was dened as "a scientic method of providing executive departments with a quantitative basis for decisions regarding the operations under their control." Computers were used to compute optimal trajectories for missiles for the Royal Airforce Establishment Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

7 History Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

8 Optimization model The problems we will discuss in this course can all be expressed in the form where Ω R n minimize f (x) x x Ω Ω can be expressed in many dierent ways s.t. Ω = Z n ; Ω = {0, 1} n Ω = {x R n : g i (x) 0} where i {1,..., m} Ω = {x R n : h j (x) = 0} where j {1,..., p} Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

9 Terminology minimize f (x) s.t. x g i (x) 0 i {1,..., m} h j (x) = 0 j {1,..., p} Decision variables vs. parameters Objective/goal/target Constraint/limitation/requirement Satised/violated Feasible/allowable solution Optimal (feasible) solution Optimal value Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

10 Model classications Unconstrained optimization: Ω = R n Linear optimization (LP): constraint functions and objective are linear Linearly constrained optimization: constraint functions are linear Convex optimization: constraint functions and objective are convex Nonlinear optimization: constraints contain nonlinear functions Discrete optimization: Ω Z n Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

11 Solving problems In practice, we are not given a mathematical problem typically the problem is described verbally by a specialist within some domain It is our task to model this problem mathematically, then solve it Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

12 Examples Support vector machine, separating sphere Portfolio optimization Shortest path Vertex cover Maximum ow Minimum cut Emergency evacuation Tournament elimination Project selection Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

13 Support vector machine We have two collections of data points in R n, say A = {a1,..., am} and B = {b1,..., bp} We would like to nd a hyperplane that separates them, i.e. a vector y R n and a scalar β such that y T a i + β 1 i {1,..., m} y T b j + β 1 j {1,..., p} Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

14 Support vector machine For each a i A, we want y T a i + β 1 For any y, the mis-t of y and a i is min { y T a i + β 1, 0 }, which we write as ( y T a i + β 1 ) Similarly the mis-t of y and b j is max { y T b j + β + 1, 0 }, which we write as ( y T b j + β + 1 ) + We can write this as an unconstrained optimization problem with variables y and β: minimize y,β ( ) ( + y T a i + β 1 + y T b j + β + 1) s.t. i j y R n β R Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

15 Support vector machine In fact, the preceding problem can actually be written as a linear program: minimize y,δ,β δ i + i j σ j s.t. a T i b T j y + β + δ i 1 i y + β σ j 1 i δ i 0 i σ j 0 j The δ i and σ j terms represent the mis-t of point a i or b j Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

16 Separating sphere Instead of separating two point sets with a line, we might try separating them with a sphere Our goal is to determine a point c R n and a radius r > 0 such that c a i r i c b i r j This can be written as a nonlinear convex program: minimize ( c c,r a i r) + + ( c b i r) s.t. c R n r R Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

17 Portfolio optimization We can choose n assets or stocks, which we will hold over a period of time Each asset i has a relative price change of p i, i.e. p i = change in price over time period price at beginning of time period Say we invest x i in asset i The overall return on the portfolio is r = p T x The natural problem is maximize p T x x e T x B x 0 s.t. where e = (1,..., 1) and B is a xed budget Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

18 Portfolio optimization In practice, we don't know the vector p; say it's a random vector Suppose that E (p) = p and Cov (p) = Σ The return r is now a random variable with mean p T x and variance x T Σx As a risk-averse investor, we'd like to minimize the variance of our investment while still receiving some average return r min : maximize x T Σx x s.t. p T x r min e T x B Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

19 Portfolio optimization Alternatively, we could maximize the average return of our investment, subject to constraints on the variance: maximize p T x x s.t. x T Σx σ 2 min e T x B These are both non-linear convex optimization problems Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

20 Graphs A common object that we encounter in solving discrete problems is a graph, or network A graph consists of vertices, V, that are connected with edges, E ; notated G = (V, E) An edge between vertices i and j is notated (i, j) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

21 Directed graphs In a directed graph, the edge from i to j may be distinct from the edge from j to i Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

22 Weighted graphs It is often useful to attach numbers, or weights, to the edges in a graph, notated G = (V, E, W ) Less frequently, weights can also be attached to the vertices Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

23 Shortest path Given a weighted graph G = (V, E, W ) and a pair of nodes s and t, nd the path from s to t with minimal total weight Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

24 Shortest path For each edge (i, j) E, dene { 1 if we use edge (i, j) x ij = 0 otherwise Our problem is minimize (i,j) E w ij x ij x sj = 1 j x it = 1 i s.t. x ij = j j x ji i V x ij {0, 1} (i, j) E Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

25 Vertex cover Given a graph G = (V, E), nd the smallest possible set of vertices that touch every edge in the network Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

26 Vertex cover For each vertex i V, dene { 1 if we use vertex i x i = 0 otherwise Our problem is minimize i V x i s.t. x i + x j 1 (i, j) E x i {0, 1} i V Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

27 Max ow Given a directed, weighted graph G = (V, E, W ) and a pair of nodes s and t Think of the edge weights w ij as the capacity of that edge What's the largest amount of ow we can send from s to t, subject to the capacity constraints? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

28 Max ow Let x ij denote the amount of ow that we send across edge (i, j); our problem is maximize s.t. (s,i) E (i,j) E x si x ij w ij x ij = (j,i) E x ji i x ij 0 (i, j) E Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

29 Minimum cut Given a graph G = (V, E), and two nodes s and t, divide V into two subsets A s and B t, minimizing the total weight of the edges separated by the cut Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

30 Minimum cut For each vertex i V, dene { 1 if vertex i is in A x i = 0 if vertex i is in B We'll also dene y ij = { 1 if edge (i, j) is separated by the cut 0 otherwise Clearly, x s = 1 and x t = 0 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

31 Minimum cut Our problem is minimize w ij y ij s.t. (i,j) E x s = 1 x t = 0 y ij x i x j (i, j) E x i, y ij 0 i, j Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

32 Emergency evacuation Represent a highway network as a graph G = (V, E) Populated nodes X V, safety nodes S V, assume X S = Can we make a set of evacuation routes from the populated nodes to the evacuation nodes without any paths sharing an edge (congestion)? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

33 Emergency evacuation Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

34 Tournament elimination Given a team's win-loss history (up to the present), can we determine if the team can be eliminated from rst place? Team New York Baltimore Toronto Boston #Wins Say there are ve games left (all pairings except Boston v. New York); can Boston win? Can we solve this problem generally? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35

35 Project selection Companies have to balance between projects that yield revenue and the expenses needed for activities that support these projects Some projects are prerequisites for others; e.g., to put Wi-Fi in a high-speed rail, we need to build the high-speed rail rst! In general these can be modelled as precedence constraints of the form (Project A) (Project B), etc. How do we select the projects that maximize our revenue? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 8, / 35