Lecture 1 Introduction to optimization

Transcription

1 TMA947 / MMG621 Nonlinear optimization Lecture 1 Introduction to optimization Emil Gustavsson, Zuzana Nedělková October 2, 217 What is optimization? Optimization is a mathematical discipline which is concerned with finding the minima or maxima of functions, possibly subject to constraints. Basic notation Vectors are written with bold face, i.e., x R n. Elements in a vector are written asx j, j = 1,...,n. All vectors are column vectors. The inner product ofaand b is written asa T b = b T a = n j=1 a jb j. The norm denotes the Euclidean norm, i.e., x = x T x = n j=1 x2 j. We utilize vector inequalities, a b, meaning that a j b j, j = 1,...,n. Optimization problem formulation In order to introduce a general optimization problem, we need to define the following: x R n : vector of decision variables x j, j = 1,...,n, f : R n R ± : objective function, X R n : ground set, g i : R n R : constraint function defining restriction onx, g i, i I : inequality constraints, g i =, i E : equality constraints. 1

2 A general optimization problem is to minimize x f(x), (1a) subject to g i (x), i I, (1b) g i (x) =, i E, x X. (1c) (1d) (If we consider a maximization problem, we change the sign off to get a minimization problem.) Classification of optimization problems Linear Programming (LP): - Linear objective function f(x) = c T x = n j=1 cjxj, - Affine constraint functions g i(x) = a T i x b i, i I E - Ground set X defined by affine equalities/inequalities. Nonlinear programming (NLP): - Some functionsf,g i, i I E are nonlinear. Unconstrained optimization: - I E =, - X = R n. Constrained optimization: - I E =, and/or - X R n. Integer programming (IP): - X Z n or X {,1} n. Convex programming (CP): - f,g i, i I are convex functions, - g i, i E are affine, - X is closed and convex. 2

3 Conventions Let S = {x R n g i(x), i I, g i(x) =, i E, x X} denote a feasible set. What do we mean by solving the problem to minimize Let f(x)? f := infimum denote the infimum value of f over the set S. If the valuef is attained at some pointx ins, we can write f := minimum f(x) f(x), and havef(x ) = f. Another well-defined operator defines the set of minimal solutions to the problem S := arg minimum f(x), wheres S is nonempty if and only if the infimum valuef is attained at some pointx ins. Now we can define what we mean by the problem to minimize f(x). "to minimize f(x)" means "findf and an x S " If we have an optimization problem P : minimize f(x) A point x is feasible in problemp if x S. The point is infeasible in problemp ifx / S. The problemp is feasible if there exist ax S and the problemp is infeasible ifs =. A point x is an optimal solution top if x arg minimum f is an optimal value top if f = minimum f(x). f(x). Examples I. Consider the problem to minimize (x+1) 2, subject to x R, Easy problem,(x+1) 2 is convex, no constraints. Just solvef (x) =, and get the optimal solutionx = 1 and the optimal valuef =. (Convex, quadratic, unconstrained optimization problem) 3

4 II. A more complicated problem is to minimize (x+1) 2, subject to x. Now the "f (x) = " trick does not work and we need to consider the boundary. We get the optimal solution x = and the optimal valuef = 1. (Convex, quadratic, constrained optimization problem) III. Consider the problem to minimize x 1, subject to x 1 +x 2 1, x 1,x 2. We solve this graphically. So optimal solution is x = (1,) T and the optimal value if f = 1. 1 x 2 f = (1,) T x 1 +x 2 = 1 x = (1,) T 1 x 1 The diet problem As a first example of an real optimization problem, we consider the diet problem (first formulated by George Stigler). For a moderately active person, how much of each of a number of foods should be eaten on a daily basis so that the person s intake of nutrients will be at least equal to the recommended dietary allowances (RDAs), with the cost of the diet being minimal? Good example to show how to model a real optimization problem, why a realistic model sometimes can be difficult to achieve. We consider the case when the only allowed foods can be found at McDonalds. For a moderately active person, how much of each of a number of McDonald foods (see Table 1) should be eaten on a daily basis so that the person s intake of nutrients will be at least equal to the recommended dietary allowances (RDAs), with the cost of the diet being minimal? 4

5 Food Calories Carb Protein Vit A Vit C Calc Iron Cost Big Mac 55 kcal 46g 25g 6% 2% 25% 25% 3kr Cheeseburger 3 kcal 33g 15g 6% 2% 2% 15% 1kr McChicken 36 kcal 4g 14g % 2% 1% 15% 35kr McNuggets 28 kcal 18g 13g % 2% 2% 4% 4kr Caesar Sallad 35 kcal 24g 23g 16% 35% 2% 1% 5kr French Fries 38 kcal 48g 4g % 15% 2% 6% 2kr Apple Pie 25 kcal 32g 2g 4% 25% 2% 6% 1kr Coca-Cola 21 kcal 58g g % % % % 15kr Milk 1 kcal 12g 8g 1% 4% 3% 8% 15kr Orange Juice 15 kcal 3g 2g % 14% 2% % 15kr RDA 2 kcal 35g 55g 1% 1% 1% 1% Table 1: Given data for the diet problem We define the sets Foods := {Big Mac, Cheeseburger, McChicken, McNuggets, Caesar Sallad French Fried, Apple Pie, Coca-Cola, Milk, Orange Juice}, Nutrients := {Calories, Carb, Protein, Vit A, Vit C, Calc, Iron.} Then we define the parameters a ij = Amount of nutrientiin food j, i Nutrients, j Foods, b i = Recommended daily amount (RDA) of nutrienti, i Nutrients, c j = Cost for food j, j Foods, and the decision variables x j = Amount of food j we should eat each day, j Foods. The model of the diet optimization problem is then to minimize subject to c jx j, j Foods j Foods a ijx j b i, i Nutrients, x j, j Foods. (2a) (2b) (2c) (2a) We minimize the total cost, such that (2b) we get enough of each nutrient, and such that (2c) we don t sell anything to McDonalds. 5

6 The optimal solution is then Total cost Total intake of calories x = = kr. = kcal. x Big Mac x Cheeseburger x McChicken x McNuggets x Caesar Sallad x French Fries x Apple Pie x Coca Cola x Milk x Orange Juice 7.48 = If we add the constraint that x j should be integer, the solution is x Big Mac x Cheeseburger 7 x McChicken x McNuggets x = x Caesar Sallad x French Fries = 1. x Apple Pie 3 x Coca Cola x Milk x Orange Juice Total cost Total intake of calories = 15 kr. = 32 kcal. Now consider going on a diet, meaning that we would like to eat as few calories as possible. We reformulate our model to minimize subject to j Foods a Calories,j x j, j Foods a ijx j b i, i Nutrients\{Calories}, x j, j Foods. (3a) (3b) (3c) The optimal solution is then x = x Big Mac x Cheeseburger x McChicken x McNuggets x Caesar Sallad x French Fries x Apple Pie x Coca Cola x Milk x Orange Juice =

7 Total cost Total intake of calories = kr. = kcal. If we add the constraint that x j should be integer, the solution is x = x Big Mac x Cheeseburger x McChicken x McNuggets x Caesar Sallad x French Fries x Apple Pie x Coca Cola x Milk x Orange Juice = Total cost Total intake of calories = 27 kr. = 221 kcal. The real diet problem When first studied by the Stigler, the problem concerned the US military and had 77 different foods in the model. He didn t managed to solve the problem to optimality, but almost. The near optimal diet was Wheat flour Evaporated milk Cabbage Spinach Dried navy beans at a cost of $.1 a day in 1939 US dollars. 7

8 Course material Lecture 1 Lecture 2 Lecture 3 Lecture 4 Lecture 5 Lecture 6 Lecture 7 Lecture 8 Lecture 9 Lecture 1 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Define and model optimization problems, classification Convexity of sets, functions, optimization problems Optimality conditions, introduction Unconstrained optimization, methods, classification. Optimality conditions, continued The Karush-Kuhn-Tucker conditions Lagrangian duality Linear programming, introduction Linear programming, continued Duality in linear programming Convex optimization Integer programming Nonlinear optimization methods, convex feasible sets Nonlinear optimization methods, general sets Overview of the course 8