Introduction. 5-Steps Method : 1. Question. 2. Approach. 3. Formulate. 4. Solve. 5. Interpret

Transcription

1 Introduction 5-Steps Method : 1. Question 2. Approach 3. Formulate 4. Solve 5. Interpret Example #1 A PC manufacturer currently sells 10,000 units/ month of a certain model. The manufacturing cost is $ 700/ unit. The whole sale price is $950/ unit. In a few test markets, the price was discounted by $100, with result of 50% sales increase. The advertising cost at present are $50,000/ month. Research by the ad agency shows an increase of advertising budget by $10,000 would increase sales by 200/month. (The company will not increase budget above $100,000). Variables: n = 10, 000 units/ month - current sales m = $700/ unit - manufacturing cost p = $950/ unit - price D = units of discount ($100/unit) r = 50% increase in sales a = $50, 000/ month - advertising b - ad budget increase = b $10, 000 c = 200/month - increase in sales Goal: Maximize P the total profit Assumptions: N = total number of units sold q = the profit/unit N = n (1 + D(0.50)) }{{} }{{ b } sold by increasing discount increase due to advertising 1

2 Case A: No ad increase The total profit is: q = (p m) }{{} D 100 }{{} net profit discount P = 10, 000 (1 + D (0.50)) (250 D 100) 50, 000 }{{}}{{} Maximize the profit: N q dp dd = n (0.50) (250 D 100) 100 n (1 + D (0.50) }{{}}{{} ) = 0 (1) r r So 100D = 25 = D = D 50 D = 0 Note: d 2 P < 0 = Max. dd2 Interpret: The price has to be discounted by $25 to optimize the total profit P. Note: P = 10, 000(1.125)(225) 50, 000 = 11, 250(225) 50, 000 = 2, 481, 250. From dp dd = 0 we have 200rD + 250r 100 = 0 Sensitivity to r (rate of increase due to discount) of D = S(D, r) = r D S(D, r) = (FIGURE) D D r r dd dr = r [ (250r 100) 200 D (200r) (r 200) (200r) 2 250r r ] = (normalized changes in D)/ (normalized changes in r) = r is = 1 2r 2 r D = 1 2rD 2

3 Note for r = 0.4 = D = 0 r = 0.6 = D = dp = ndq = 10, 000 D ( D) = dr So the sensitivity of P to r is S(P, r) = r P = 0.1 (FIGURE) Case B: Include the effect of advertising P (b, D) = [n(1 + rd) + }{{} 200 b] [250 D 100] b 10, , 000 Maximize P (b, D) by using Lagrange multipliers: c P = nr[250 D 100] 100[n(1 + rd) + cb] D P = 200[ D] 10, 000 = 50, Dc b Note that P > 0 for 0 D 2.5. b We also know that the company will not increase budget above $100,000, so 0 b 10. First find the critical points that satisfy the inequalities by solving the system: P D = 0, P b = 0 3

4 Equivalently we have: ( 100rn + 100rn 100c 100c 0 ) ( D b ) = ( 250nr 100n ) , 000 So, from we get From the second equation 100cD = , 000 D = c = = 2 200nrD + 100cb = 250nr 100n we get b = n [ 150r 100] = 75r c Note that the point (b, D) = ( 75r 50, 2) is a saddle point. Definition of saddle point Next, we proceed with the Lagrange multipliers and we treat the constraints as equalities instead of inequalities i.e. and We solve the following system b = 0, b = 10 D = 0, D = 2.5 P D = λ 1, P b = λ 2 for D and b in terms of λ 1 and λ 2, then substitute in the above constraint equations. (FIGURE) 4

5 Why does this make sense? (review) General example for Lagrange multipliers(with equality constraints) minf( x) x = (x 1, x 2,, x n ) s.t. g 1 ( x) = c 1. g m ( x) = c m For linear constraints min f( x) Ax = b Equivalently : a 11 x 1 + a 12 x 2 + a 1n x n = b 1. a m1 x 1 + a m2 x 2 + a mn x n = b m Then append the constraints (from optimization theory) f x1 f x2 f xn. = f = λ 1 g 1 }{{} + + λ m g }{{ m } Note n-vector n-vector a 11 a 21 a m1 ḡ =. = AT a 1n a 2n a mn So we have n + m equations with x = (x 1, x 2,, x n ) and λ = (λ 1, λ 2,, λ m ) unknowns. 5

6 Note: Assume g k with k = 1, 2,, m are vectors linearly independent. Let s consider a simple example with linear constraints: min(x x 2 2) s.t. 2x 1 + x 2 = 3 f at x = x* f at x = x* g is g = 3 Clearly f 0 for feasible solutions. Suppose we find the minimum: Note: g 1 = 3 = 2x 1 + x 2 = 3 so g 1 = (2, 1) To determine x 1, x 2 we solve the following system: 2x 1 = 2λ 2x 2 = λ = λ = x 1 and λ = x 2. Substitute x 1 and x 2 in the constraint: 2λ + λ 2 = 3 and get λ = 6 5. Therefore x 1 = 6 5 and x 2 = 3 5. For these values f(x) = x2 1 + x2 2 = =

7 We can check this result by plugging the constraint into f(x) and minimize. f(x) = x (3 2x 1) 2 = 5x x Now minimize this function: f x 1 = 10x 1 12 so from f x 1 = 0 = x 1 = 6 5 which confirms our previous result. General example for Lagrange multipliers (with inequality constraints) minf( x) x = (x 1, x 2,, x n ) s.t. g 1 ( x) c 1. g m ( x) c m (FIGURE) Reconsider now, the previous example with inequality constraints: min(x x 2 2) s.t. 2x 1 + x 2 3 x 1 2 x 1 x 2 3 (FIGURE) 7

8 Note that f 0 on the region. By looking at the graph we expect minimum on 2x 1 + x 2 = 3 = λ = 6 5 What if we guess wrong by looking for the minimum on x 1 = 2? Then the equations are: f = (2x 1, 2x 2 ) and g 1 = x 1 = 2. So g 1 = (1, 0) Equivalently 2x 1 = λ 2x 2 = 0 so λ = 4. But this says f = λ g > 0 (an increasing function) = cannot be a minimum. Now return to the profit problem. The goal is to maximize the profit function P (b, D) on 0 b 10 with respect to D and b. So the problem is: s.t. b 10 Note P 0 on the interval [0, 10]. First, try g 1 (b, D) = b = 10 constraint. P = λ(0, 1). So from here maxp (b, D) b 0 P = 0 = nr[250 D 100] 100[n(1 + rd) + cb] D P = λ = 200[ D] 10, 000 b Therefore D = 250 λ + 10, = D = λ + 10, Plug this in the first equation and get [ ] λ + 10, 000 nr 100[n(1 + rd) + cb] = From these equations λ = 39, 000 > 0 and D =

9 Next, try g 1 (b, D) = b = 0 constraint. From the above equations when b = 0 we get D = 0.25 and λ > 0 Note, in this case λ = P b > 0. Since λ > 0, maximum must be at top constraint. If the maximum was at b = 0, then P < 0 at b = 0, so λ < 0. b What conclusions can we draw? D = 0.25 and b = 10 give the maximum. Is D as significant as b? Conclusions about dependence of P on D. (FIGURE) What about sensitivity? As before: S(P, r) = r P P r S(D, r) = r D D r Now we can also test sensitivity to constraint by solving three equations: P D = 0 P b = λ b = b max 9

10 We have 40, , 000D = λb max 2 D = λb max 20, 000 D = 2 λb max 20, 000 Now, we study the sensitivity of P to b max. Recall the problem: P is maximized under the constraint 0 b 10 Note also: Therefore: P = ( P D, P ), P = λ g, g(b, D) = b = 10 b P b max = P D D + P b max b = P ( D b max, = λ g ( D b max, = λ(0, 1) ( D b max, b b max b b max ) b b max ) P b max = λ = 39, 000 b b max ) = λ So S(P, b max ) = λ b max P λ = Shadow price, amount P increases with change in constraint. In general, to maximize f( x) x = (x 1, x 2,, x n ) 10

11 with constraints: g 1 (x 1, x 2,, x n ) = c 1 g 2 (x 1, x 2,, x n ) = c 2. g k (x 1, x 2,, x n ) = c k f = λ g, λ = (λ 1, λ 2,, λ k ) where f xj = λ 1 g 1xj + λ 2 g 2xj + + λ k g kxj For c variable (parameter) x = x(c). Then, j = 1, 2,, n So f = f x 1 + f x f x n c 1 x 1 c 1 x 2 c 1 x n c 1 = f ( x (c 1 )). = λ x (c 1 ) = (λ 1 g 1x1 + λ 2 g 2x1 + + λ k g kx1 ) x 1 c 1 + (λ 1 g 1x2 + λ 2 g 2x2 + + λ k g kx2 ) x 2 c 1 + (λ 1 g 1xn + λ 2 g 2xn + + λ k g kxn ) x n c 1 n g 1 x j n g 2 x j = λ 1 + λ 2 + λ k x j c 1 x j c 1 j=1 j=1 = λ 1 g 1 c 1 + λ 2 g 2 c λ k g k c 1 f c 1 = λ 1 n j=1 g k x j x j c 1 which means that the Lagrange multiplier is the rate of change of the objective function with respect to constraints. Problem A local newspaper is looking for ways to increase profits. Currently, the paper is $ 11

12 1.50/week with circulation of 80,000. Advertising is $ 250/page, and 350 pages/week are sold. It is estimated that an increase of 10 cents/week in the subscription price will cause a drop in circulation of 5,000 subscribers. Increasing ad prices by $100/page will cause the paper to lose approximately 50 pages of ads/week. The loss of 50 advertising pages will reduce circulation by 1,000 subscriptions. Find the weekly subscription price that will maximize profit, keeping advertising costs the same. How do the assumptions affect your result? What is the optimal strategy if both subscription price and advertising price are changed? Variables: w = $1.50 : initial price/week s 0 = 80, 000 : # of subscribers (initially) a = $250 : ad price/page (initially) p = 350 : initial pages /week c = $0.10 increase in subscription price/week (units of price increase) b : increase in ad price/page/week (units of $100) r = 50 : loss of ad pages/unit due to increase in ad price x = 1, 000 : loss of subscribers due to r ad pages lost k = 5, 000 : loss of subscribers per $ 0.10 price increase The profit P ( = revenue ) Goal: Maximize P the total profit Step 1: s = 80, 000 ck bx = # of subscribers q = w + cl = profit/ subscriber z = a + 100b = profit/ ad page n = 350 br = # of ad pages P = profit = sq + zn Look at 5-step method: 1. Ask/Identify variables, units sensible, assumptions identified. 2. Modelling approach 3. Formulate according to 2 - using 1 4. Solve 3 12

13 5. Answer - using 4 and 1 (criticize) The profit function is given by : P = (s 0 ck bx)(w + c(0.10)) + (a + 100b)(350 br) where the variables are the ad price (b) and subscription price (c). Case 1 First fix b = 0 dp = k(w + cl) + l(80, 000 ck bx) = 0 dc 5, 000( c(0.10)) (80, 000 5, 000c) = 0 From here c = 1 2 Then q = w + cl = 1.55 and s = 80, 000 2, 500 = 77, 500. Sensitivity analysis: What if lost sales is not k = 5, 000? How sensitive is the result to this assumption? dp dc = k(w + cl) + l(80, 000 ck bx) = k( c) + 8, ck = 0 Solve this equation for c, c = c (k). Optimal c is a function of k which depends on assumption about k. So from the above equation: c = 5 8, 000 k 7.5 From here The sensitivity is given by dc dk = 40, 000 k 2 S(c, k) = dc dk k c = 40, 000 k 2 k c which represents 16% increase in c for a drop of 1% in k. = 40, 000 2, 500 = 16 13

14 f(c) k worse f(c) k = (k c 2 ) / 3 f(c) k = k better c Nonlinear relationship What if the assumption about k depends on a survey asking about a $. 30 increase? Perhaps we expect better results for c < 0.30 and worse results for c > 0.30 Question What is the sensitivity of the optimal price to assumption about k? Linear case The sensitivity is: (FIGURE) P = (s 0 c k bx)(w + c (0.10)) + (a + 100b)(350 br) S(P, k) = dp dk k P = c (w + c (0.10)) k 0.5 (1.55) 5, 000 = P P 14

15 Quadratic case P = (s 0 c 2 3 k bx)(w + c (0.10)) + (a + 100b)(350 br) The sensitivity is: S(P, k) = dp dk k P = c 2 3 (w + c (0.10)) (FIGURE) k P = ( ) 5, 000 P In both cases the sensitivity gives changes of $10, 000/week for 1% change in k. Then P = (s 0 c2 k bx)(w + c(0.10)) + (a + 100b)(350 br) 3 dp dc = 2ck c2 (w + c(0.10)) + l[80, k bx] = c2 kl 2ck w + 8, 000 = 0 3 Solve this equation for c and get: Thus c = k + k 2 + 3, 200k 2(0.10k) c = , 200 k For k = 5, 000 we get c = 1.4 Then dc dk = ,200 The sensitivity is S(c, k) = dc dk k k c = 16, k 2 + 3, 200k or c = k k 2 + 3, 200k 2(0.10k) or c = 5 5 3, 200 k , 200 k 16, 000 = 2 k 4 + 3, 200k 3 1 c = 16, , ,200 5,

16 (FIGURE) Case 2-2-D optimization Optimize the profit P (c, b) with respect to subscription cost and ad cost. Find the critical point: P = 0 = k(w + cl) + l(80, 000 ck bx) c P = 0 = x(w + cl) + 100(350 br) r( b) b This is a linear system of two equations and two unknowns. Equivalently we have: ( ) ( ) ( ) 2lk xl c 80, 000 = xl 200r b 35, 000 xw 250r By substituting the values of the parameters and solving this system we get: (c 0, b 0 ) = (0.2903, ). Note that 2 P b 2 2 P c 2 = 2kl 2 P b 2 = 200r 2 P c b = lx ( 2 P c b )2 > 0, so (c 0, b 0 ) = (0.2903, ) is not a saddle Therefore 2 P c 2 point. Note that if there is a cap on the amount of increase in price, i.e. b = 1.5, then it can not achieve this maximum. (FIGURE) 16

17 Results for the problem for parameters as given, b = 1.5 and c = 0.35 The goal is to maximize the profit function P (b, c) on b 1.5 with respect to c and b. So the problem is: maxp (b, c) s.t. b 1.5 There is no local optimum obtained for b < 1.5, therefore it must be obtained on the boundary b = 1.5. So the problem becomes: Note that P = λ(0, 1). So from here maxp (b, c) s.t. b = 1.5 P = 0 = k(w + cl) + l(80, 000 ck bx) c P = λ = x(w + cl) + 100(350 br) r( b) b For b = 1.5 substitute the other known variables (w, x, r, k) and solve the system for λ and c. We can test the sensitivity to assumptions: S(b, r) = b r r b S(b, k) = b k k b = 0, b is constant at r = 50 = 0 at k = 5, 000 (FIGURES) Next, we look at sensitivity of P to r. 17

18 (FIGURE) Finally, we look at sensitivity of P to constraint b = c 1. S(P, c 1 ) = P c 1 c 1 P = λc 1 P So we get an increase of % of P for 1% increase in c 1 If we include the constraint at b = 0 we repeat the analysis with b = c 2 = 0 constraint. P c = 0, P b = λ 2 From here λ 2 < 0 which indicates that P c 2 maximum there Problem Determine population levels that maximize population growth. Variable population model: Logistic model: x = r 1 x(1 x k 1 ) Population decays (grows) for x > (<)k 1. We can solve for x(t). dx dt = r 1x(1 x ) = k 1 1 r 1 [ln x + ln(1 x k 1 )] = t + C ln x(1 x k 1 ) = r 1 t + C < 0 on this boundary = not a dx r 1 x(1 x ) = k 1 dt = x(1 x k 1 ) = Ce r 1t 18

19 Model for two competing species - The whale problem x = r 1 x(1 x k 1 ) α 1 xy y = r 2 y(1 y k 2 ) α 2 xy where r i = the intrinsic growth (Lotka - Voltera) k i = maximum sustainable population α i = competition Question: At what population levels is the total growth largest? Therefore we have to determine population levels that are feasible (x, y 0) and sustainable (x, y 0) which maximize the total population growth. Consider effect of α 1 = α 2 = α. The problem becomes: Blue Fin r k α max(x + y ) s.t. x 0 y 0 x 0 y 0 (FIGURES) 19

20 We initially consider the optimization problem with equality constraints. max(x + y ) s.t. B 1 : x = 0 B 2 : y = 0 B 3 : x = 0 (x 0) B 4 : y = 0 (y 0) (FIGURES) For this, first of all we find the critical point for f = x + y = r 1 x(1 x k 1 ) α 1 xy + r 2 y(1 y k 2 ) α 2 xy which satisfies the equality constraints: f x = 0, f y = 0 Equivalently r 1 (1 x k 1 ) α 1 y α 2 y r 1 k 1 x = 0 r 2 (1 y k 2 ) α 1 x α 2 x r 2 k 2 y = 0 Substituting the values for r 1, r 2, k 1, k 2, α 1 = α 2 = α and solving the system we get x and y If we substitute these values in the equations of derivatives, we get: x + y = > 0 20

21 4 x 10 5 B 4 B 1 2 B B x 10 5 Next, we proceed with the Lagrange multipliers and we treat the constraints as equalities instead of inequalities. Note on: B 1 &B 2 we have x + y = 0 B 1 &B 4 we have x + y = 0 B 2 &B 3 we have x + y = 0 Thus we can eliminate the corners. Note on : B 1, x = c (= 0) dp dc = f x = f = λ, if decreasing, then moving into domain = f decreasing = c optimize on boundary B 2, y = c (= 0) 21

22 dp dc = > 0 f y = f c B 3, x = c = λ > 0 = optimize by moving into domain dp dc = > 0 f y = f = λ > 0. As c increases, the boundary moves to the left. c If λ > 0 = f increases, then moving into the domain. If λ < 0 = f decreases, then moving out of the domain, but for this x < 0. Similarly for B 4. Next consider the constraint B 1 : g 1 (x, y) = x = 0. From here g 1 = (1, 0) so the constraints with Lagrange multipliers become: f x = λ f y = 0 x = 0 If we consider the constraint B 3 : g 3 (x, y) = x = r 1 x(1 x k 1 ) α 1 xy = 0. From here g 3 = (r 1 (1 x k 1 ) α 1 y r 1 k 1 x, α 1 x) so the constraints with Lagrange multipliers 22

23 become: f x = λ[r 1 (1 x k 1 ) α 1 y r 1 k 1 x] f y = λ( α 1 x) x = 0 Problem 2.42 max(x + y) s.t. x 0 y 0 x 0 y 0 Note (x + y) = (1, 1) = cannot have internal maxima. Consider the problem with the equality constraints: max(x + y) s.t. x = 0 or y = 0 or x = 0 or y = 0 Newton s Method Case A - (1-D) Find the root of f(x) i.e find x such that f(x ) = 0. Suppose you make an estimate x e of x. Then f(x ) = f(x e ) + f (x e )(x x e ) We want f(x ) = 0, so f(x e ) = f (x e )(x e x ) = f(x e) f (x e ) x e x. So, from here x x e f(x e) f.let s see how this works: (x e ) 23

24 f(x) f(xe) xe x* xe + c f(xe + c) If we guess x e, then we can improve the guess by f(x e) f = c > 0 = the new guess (x e ) will be x e + c. Now replace x e with x e + c. Update: x e + c f(x e + c) f (x e + c) So guesses to left, correct to the right and guesses to right correct to the left. If we want to find minimum(maximum) of a function F (x) we have to find the critical points first. So we have to find the roots of F (x) = f(x) = 0. Start with an initial guess x 0 and improve it: x n+1 = x n F (x n ) F (x n ) where x n+1 is the updated (improved) guess. Case B - (2-D) Now we want to find the maximum (minimum) of function F (x, y). First of all we have to find the critical points by solving the following system: f(x, y ) = F x = 0 g(x, y ) = F y = 0 24

25 Start with an initial guess (x 0, y 0 ). Then 0 = f(x, y ) = f(x 0, y 0 ) + f x (x 0, y 0 )(x x 0 ) + f y (x 0, y 0 )(y y 0 ) 0 = g(x, y ) = g(x 0, y 0 ) + g x (x 0, y 0 )(x x 0 ) + g y (x 0, y 0 )(y y 0 ) Or in the matrix form: ( ) ( ) fx f y x x 0 g x g y y = y 0 Therefore In general x x 0 = fg y gf y f x g y g x f y y y 0 = f xg g x f f x g y g x f y ( ) f(x0, y 0 ) g(x 0, y 0 ) x n+1 = x n F xf yy F y F xy F xx F yy F xy F yx, y n+1 = y n F yf xx F x F xy F xx F yy F xy F yx Linear Programs We want to solve the following problem: maxf( x) = c 1 x 1 + c 2 x c n x n where c 1, c 2,, c n - constants s.t. g 1 ( x) b 1. g m ( x) b m We ve already seen a linear program, even though it didn t appear to be. reconsider the whale model Let s max(x + y) s.t. x 0 y 0 x 0 y 0 25

26 where x = r 1 x(1 x k 1 ) α 1 xy 0 y = r 2 y(1 y k 2 ) α 2 xy 0 Note that these constraints are linear but in general they would be nonlinear (quadratic). We saw that the maximum was obtained at a corner which is geometrically obvious. The idea behind solving a linear program is that if moving along a boundary is optimal, then continue until corner is reached. Many software packages exist for solving linear programs, based on the simplex method. We consider a simple example to see how the simplex method works. max(x 1 + 2x 2 ) s.t. 2x 1 + x 2 2 x 1 + 2x 2 7 x C 5 D 2 B -2 x 1 + x 2 = 2 x 1 = 0 x 1 = 3 A x 2 = 0 E It can be proved that the optimal point is an extreme point(corner). Idea: If moving along the constraint improves solution, continue until you hit the next constraint. 26

27 Standard form: Introduce slack variables - slack variables pick up the slack in the inequality and convert to equality constraints. Standard form: min ˆf s.t. Aˆx = b ˆx 0 We have our problem in terms of maximum, so we can take f and look for min( f). min( x 1 2x 2 ) s.t. 2x 1 + x 2 + s 3 = 2 x 1 + 2x 2 + s 4 = 7 x 1 + s 5 = 3 x 1, x 2 0 s 3, s 4, s 5 0 s j = slack variables Replace s 3, s 4, s 5 to x 3, x 4, x 5. We know that we will find the maximum at an extreme point. So we choose one feasible solution (the one that satisfies the constraints ) which is an extreme point, and see if can do better. So we start with something simple: if we take x 3, x 4, x 5, the slack variables, to equal the constraints, the x 1, x 2 = 0. Note: we need (at least) 3 of x is 0(basic variables) to satisfy the constraints (3 equations and 3 unknowns)= x 3 = 2, x 4 = 7, x 5 = 3. This means that we start at A, where x 1, x 2 = 0 (non-basic variables). Now we look to see if we can improve the objective function f by changing the non-basic variables x 1 or x 2. YES! In fact if we increase either one, f is improved. Which variable should we choose? We have to choose the one which improves f more. Note that the objective function is f = x 1 2x 2, so it will be improved more if we choose to change x 2. Do not choose both variables, because we know that we should stay on constraint. By making x 2 0, we can now satisfy the constraints with either x 2, x 4, x 5, x 2, x 3, x 5, or x 2, x 3, x 4, etc. Moving on constraint x 1 = 0 (increasing x 2 ), which constraint do we hit first by 27

28 decreasing x 3 or x 4? ( 2 0) + x 2 + x 3 = 2 ( 0) + 2x 2 + x 4 = 7 x 1 + s 5 = 3 (it doesn t change) Therefore x 2 = 2, x 3 = 0, and x 4 0. So, move along constraint until you hit x 2 = 2, then x 3 = 0. Note that this is the point B. We have changed our basic variables; before we had x 3, x 4, x 5 0 to make up nontrivial variables satisfying the constraints. Now we have new basic variables: x 2, x 4, x 5 0 x 1, x 3 = 0 ( 2 0) + x = 2 = x 2 = 2 ( 0) + 2x 2 + x 4 = 7 = x 4 = 3 x 1 + s 5 = 3 = x 5 = 3 This step is called a pivot. We repeat the procedure until we can no longer improve f. 1. Check to see possible improvements to f by changing non-basic variables 2. Choose direction to search 3. Pivot (switch basic, non-basic variables) and repeat. First Step (x 1 = 0, x 2 = 0) Basic variables Non-basic variables x 3 x 1 x 4 x 2 x 5 The problem is min( x 1 2x 2 ) s.t. 2x 1 + x 2 + x 3 = 2 x 1 + 2x 2 + x 4 = 7 x 1 + x 5 = 3 28

29 We changed x 2 because the coefficient of x 2 in f is greater. So the new basic variables are x 2, x 4, x 5 from increasing x 2 until x 3 = 0 and x 4 0 Second Step(x 1 = 0, x 2 = 2) Basic variables Non-basic variables x 2 x 1 x 4 x 3 x 5 We change the objective function in terms of non-basic variables min( x 1 4x 1 + 2x 3 4) = min( 5x 1 + 2x 3 4) s.t. 2x 1 + x 2 + x 3 = 2 x 1 + 2x 2 + x 4 = 7 x 1 + x 5 = 3 Note that the coefficient of x 1 in the objective function is greater, so we change x 1 = it affects x 2, x 4, x 5. At x 1 = 3, x 5 = 0 which is off constraint. Moving along the first constraint (x 3 = 0) we have x 1 = 1 + x 2, so from here 2 x 1 + 4x x 4 = 7. Therefore x 1 = 1, x 4 = 0, x 2 = 4. Third Step(x 1 = 1, x 2 = 4) Basic variables Non-basic variables x 1 x 4 x 2 x 3 x 5 Can we improve f by increasing one of the non-basic variables? Write f in terms of non-basic variables x 3 and x 4. From the first constraint we have x 2 = 2 + 2x 1 x 3 and by substituting this into the third constraint we get x 1 + 2(2 + 2x 1 x 3 ) + x 1 = 7. So the objective function in terms of non-basic variables will be: f = [ x x 4] 2[2 + 2( x x 4) x 3 ] = x x 3 29

30 Thus the problem will be: min( x x 3) s.t. 2x 1 + x 2 + x 3 = 2 x 1 + 2x 2 + x 4 = 7 x 1 + x 5 = 3 Note that by increasing x 3 we improve the objective function = it affects x 1, x 2, x 5. We follow the constraint x 1 + 2x 2 = 7 since x 4 =0. From here x 2 = 7 x 1. 2 By substituting this into the second constraint we get: 2x x 1 + x 3 = 2. 2 Equivalently: 2(3 x 5 ) + 7 (3 x 5) + x 3 = 2. So from here x 5 + x 3 = 3. If x 3 = 3 = x 2 = 5. Therefore x 1 = 3 and x 2 = 5 Fourth Step(x 1 = 3, x 2 = 5) Basic variables Non-basic variables x 1 x 4 x 2 x 5 x 3 From the constraints we have: x 1 = 3 x 5 and 2x 2 = 7 x x 5. So the objective function in terms of non-basic variables will be: f = (3 x 5 ) (7 x x 5 ) = 2x 5 + x 4 13 At this point the objective function cannot be improved anymore by increasing the non-basic variables. Therefore x 4 = 0, x 5 = 0 = x 1 + 2x 2 = 7 and x 1 = 3. The binding conditions give the extreme point. Example in 4-D f(x 1, x 2, x 3, x 4 ) = ax 1 + bx 2 + cx 3 + dx 4. f(x) is a linear function in all variables. x j = # of certain product which is limited by resources, materials, work. 30

31 The problem will be: max(ax 1 + bx 2 + cx 3 + dx 4 ) s.t. c j x j dj x j x j 0 So, there are 6 constraints and 4 variables. Where do we expect to find maximum? On edge! - WHY? Let s pick up an interior point. The linear function f(x) can always be increased by increasing x j (or if a, b, c, d < 0 by decreasing x j ) = eventually hit the edge. Further,we can always increase the function f by increasing along the edge: follow constraint until the direction changes = eventually hit the corner = optimize over corners. Dual Problem Primal (Original) Problem Dual Problem maxf(x) = c 1 x 1 + c 2 x c n x n x a 11 a 12 a 1 1n x 2 s.t... b 2. a m1 a m2 a mn x n b 1 b m max ˆf(y) = b 1 y 1 + b 2 y b m y m y a 11 a 21 a 1 c 1 m1 y 2 s.t... c 2. a 1n a 2n a mn Note: The dual variables are the same as the shadow prices from Lagrange multipliers. y m c n 31

32 f = λ g. Then the equations are: c 1 = a 11 λ 1 + a 21 λ 2 + a m1 λ m. c n = a 1n λ 1 + a 2n λ 2 + a mn λ m Then max ˆf = min f (the objective functions are equal). If we increase b i by 1 unit, ˆf increases by yi units i.e. ˆf = y i. b i Let s consider again the whale problem. Primal Problem Dual Problem min( x 1 x 2 ) r 1 αk 1 s.t.. r 2 αk 2 x 1 0 x 2 0 ( ) x1 x 2 ( ) r1 k 1 r 2 k 2 min(r 1 k 1 y 1 + r 2 k 2 y 2 ) s.t. r 1 y 1 + r 2 y 2 1 αk 1 y 1 + αk 2 y 2 1 y 1 0 y 2 0 Next, let s reconsider the whale problem as linear programming problem. max(x + y) s.t. x 0 y 0 x 0 y 0 32

33 where x = r 1 x(1 x k 1 ) α 1 xy 0 y = r 2 y(1 y k 2 ) α 2 xy 0 We can solve this problem in MATLAB by using the following function: [X, F V AL, EXIT F LAG, OUT P UT, LAMBDA] = linprog(f, A, b) which solves the general problem minf s.t. Ax b Now we put the whale problem into matlab form: min( x y ) s.t. r 1 x(1 x ) + α 1 xy 0 k 1 r 2 y(1 y ) + α 2 xy 0 k 2 x 0 y 0 So f = ( 1 ) 1 r 1 αk r 1 k 1 1 A =. b = r 2 k 2 0 αk 2 r 2 0 Example Wood Labor Revenue Bookshelf With doors With drawers Custom

34 There are 500 units of wood available, and 150 units of labor available. Which types of book shelves should be produced to maximize revenue? f(x) = 100x x x x 4, where x i are the number of units of each type of bookshelves. The problem becomes: max(100x x x x 4 ) s.t. 10x x x x x 1 + 4x 2 + 8x x x 1, x 2, x 3, x 4 0 For the given parameters, the matlab form of this problem is: min( 100x 1 150x 2 200x 3 400x 4 ) s.t. 10x x x x x 1 + 4x 2 + 8x x x 1, x 2, x 3, x 4 0 Using linear programming in MATLAB we get x 1 =37.5 and x 4 =6.25. So the revenue is Note: linear programming gives approximation. In reality we can only make integer multiples of products = x 1 =38 and x 4 =6 and therefore the revenue is In this case there were 500 units of wood used and only 148 units of labor (not all consumed). Case 1 If increase the constraint of wood by 1, i.e. 10x x x x 4 501, we get $5 increase in the revenue = the revenue = Case 2 If increase the constraint of wood by 2, i.e. 10x x x x we get x 1 =37.8 and x 4 =6.2. For x 1 38 and x 4 6 = the revenue = For x 1 37 and x 4 6 = the revenue = 6250 which is the maximum. Case 3 If increase the constraint of wood by 8, i.e. 10x x x x the maximum is given by x 1 =39 and x 4 =6. 34

35 Integer programming For the previous problem suppose that x i can take only integer values, i.e. x 1 = 1, 2,, 50 x 2 = 1, 2,, 34 x 3 = 1, 2,, 18 x 4 = 1, 2,, 8 The actual optimal solution changes discontinuity with constraints. If we have n bookshelves, then we have n! combinations. This is an example of knapsack problem: compare all possible combinations of items to be put in knapsack with different values, weights, sizes,, weight restriction. Possible solution methods: Combinatorial methods (polyhedral considerations) Dynamic programming Take one item, then proceed according to some algorithm until restriction is hit - compare all possible end results, take maximum, and go back recursively. Tree - exponential time algorithm. Ways to reduce time Use linear programming as approximation to reduce state space. For the wood problem the linear programming gave x 1 = 37.5 so we could choose x 1 = 37 or 38. Similarly for x 4 =6.25, we could choose either x 4 = 6 or 7. Greedy algorithm. For the wood problem - make as many of most expensive items as possible if resources are available. Make 12 custom units and 1 unit with drawers = the revenue = 4, =5,000 and the left over wood is =260. Improve by replacing some custom units with items not requiring much labor, but using wood. Shortest path problem - Travelling salesperson problem The sales person must visit n cities. It cost him (time) to go between different cities. What is the minimum cost to visit all 5 cities once? 35

36 (FIGURE) We can do the calculation by hand. C(1,4,1)=min 3 + C(2, 3, 1) = = C(3, 3, 1) = = C(4, 3, 1) = = 13 C(2,3,1)=min 2 + C(3, 2, 1) = = C(4, 2, 1) = = 10 C(3,3,1)=min = = 7 C(4,3,1)=min = = What about greedy algorithm? Do the best you can do on the first step, then do the best you can do on the next step does give the optimal. 36