Applied Lagrange Duality for Constrained Optimization

Size: px

Start display at page:

Download "Applied Lagrange Duality for Constrained Optimization"

Rose Arnold
6 years ago
Views:

1 Applied Lagrange Duality for Constrained Optimization February 12, 2002

2 Overview The Practical Importance of Duality ffl Review of Convexity ffl A Separating Hyperplane Theorem ffl Definition of the Dual Problem ffl Steps in the Construction of the Dual Problem ffl Examples of Dual Constructions ffl cfl2002 Massachusetts Institute of Technology. All rights reserved

3 Overview The Dual is a Concave Maximization Problem ffl Weak Duality ffl The Column Geometry of the Primal and Dual Problems ffl Strong Duality ffl ffl Duality Strategies ffl Illustration of Lagrange Duality in Discrete Optimization cfl2002 Massachusetts Institute of Technology. All rights reserved

4 Context of Duality Previous Examples of Duality Models of electrical networks ffl Models of economic markets ffl Structural design ffl Duality in Linear Optimization ffl cfl2002 Massachusetts Institute of Technology. All rights reserved

5 Context of Duality Importance of Duality Identifying near-optimal solutions ffl Proving optimality ffl Sensitivity analysis of the primal problem ffl Karush-Kuhn-Tucker (KKT) conditions ffl Convergence of improvement algorithms ffl Good Structure ffl Other uses, too... ffl cfl2002 Massachusetts Institute of Technology. All rights reserved

6 Review of Convexity Local and Global Optima Problem Set-Up The ball centered at μx with radius ffl is the set: B(μx; ffl) := fxjkx μxk» fflg: Consider the following optimization problem over the set F: P : min x or max x f(x) s.t. x 2 F cfl2002 Massachusetts Institute of Technology. All rights reserved

7 Review of Convexity Local and Global Optima Definition x 2 F 1 is a local minimum P of if there ffl > 0 exists such f(x)» f(y) that for all 2 B(x; ffl) F. y Definition x 2 F 2 is a global minimum P of if» f(y) for all y 2 F. f(x) cfl2002 Massachusetts Institute of Technology. All rights reserved

8 Review of Convexity Strict Local & Global Minima Definition x 2 F 3 is a strict local minimum P of if there ffl > 0 exists such f(x) < f(y) that for all 2 B(x; ffl) F. y Definition x 2 F 4 is a strict global minimum P of if < f(y) for all y 2 F. f(x) cfl2002 Massachusetts Institute of Technology. All rights reserved

9 Review of Convexity Local & Global Maxima Definition x 2 F 5 is a local maximum P of if there ffl > 0 exists such f(x) f(y) that for all 2 B(x; ffl) F. y Definition x 2 F 6 is a global maximum P of if f(y) for all y 2 F. f(x) cfl2002 Massachusetts Institute of Technology. All rights reserved

10 Review of Convexity Strict Local & Global Maxima Definition x 2 F 7 is a strict local maximum P of if there ffl > 0 exists such f(x) > f(y) that for all 2 B(x; ffl) F. y Definition x 2 F 8 is a strict global maximum P of if > f(y) for all y 2 F. f(x) cfl2002 Massachusetts Institute of Technology. All rights reserved

12 Review of Convexity Definition 9 Convex Sets and Functions A subset S ρ < n is a convex set if y 2 S ) x + (1 )y 2 S x; for 2 [0; any 1]. Illustration of convex and non-convex sets. cfl2002 Massachusetts Institute of Technology. All rights reserved

13 Review of Convexity Intersections of Convex Sets Proposition 1 S; T If are convex sets, S T then is a convex set. Proposition 2 The intersection of any collection of convex sets is a convex set. Illustration of the intersection of convex sets. cfl2002 Massachusetts Institute of Technology. All rights reserved

14 Review of Convexity Convex Functions Definition 10 A function f(x) is a convex function if + (1 )y)» f(x) + (1 )f(y) f( x for x all y and and for 2 [0; all 1]. Definition 11 function if A function f(x) is a strictly convex + (1 )y) < f(x) + (1 )f(y) f( x for x all y and and for 2 (0; all 1). cfl2002 Massachusetts Institute of Technology. All rights reserved

16 Review of Convexity Convex Optimization P : minimize x f(x) x 2 F s:t: Proposition 3 Suppose F that is a convex set, : F! < is a convex function, and μx is a local f minimum P of. μx Then is a global minimum f of over F. cfl2002 Massachusetts Institute of Technology. All rights reserved

17 Review of Convexity Proof of Proposition Proof of the Proposition: Suppose μx is not a global minimum, i.e., there exists y 2 F for which f(y) < f(μx). Let = μx + (1 )y, which is a convex combination of μx and y y( ) 2 [0; 1] for (and y( ) 2 F therefore, 2 [0; for 1]). Note that! μx as! 1. y( ) From the convexity of f( ), = f( μx + (1 )y)» f(μx) + (1 )f(y) < f(μx) + (1 )f(μx) = f(μx) f(y( )) for 2 (0; all 1). f(y( )) < f(μx) Therefore, for 2 (0; all 1), and μx so is not a local minimum, resulting in a contradiction. q.e.d. cfl2002 Massachusetts Institute of Technology. All rights reserved

18 Review of Convexity Examples of Convex Functions Functions of One Variable Examples of convex functions of one variable: ffl f(x) = ax + b ffl f(x) = x 2 + bx + c ffl f(x) = jxj ffl f(x) = ln(x) for x > 0 for = 1 x x > 0 ffl f(x) = e x f(x) ffl cfl2002 Massachusetts Institute of Technology. All rights reserved

19 Review of Convexity Concave Functions & Maximization The opposite of a convex function is a concave function: Definition 12 A function f(x) is a concave function if + (1 )y) f(x) + (1 )f(y) f( x for x all y and and for 2 [0; all 1]. Definition 13 A function f(x) is a strictly concave function if + (1 )y) > f(x) + (1 )f(y) f( x for x all y and and for 2 (0; all 1). cfl2002 Massachusetts Institute of Technology. All rights reserved

21 Review of Convexity Concave Maximization Now consider the maximization problem P : P : maximize x f(x) s:t: x 2 F Proposition 4 Suppose F that is a convex set, : F! < is a concave function, and μx is a local f maximum P of. μx Then is a global maximum f of over F. cfl2002 Massachusetts Institute of Technology. All rights reserved

22 Review of Convexity Convex Program : min x or max x f(x) P x 2 F s.t. We call P a convex program if ffl F is a convex set, and we are minimizing and is a convex function, or ffl we are maximizing and f(x)is a concave function. Then every local optima of the objective function of a convex program is a global optima of the objective function. cfl2002 Massachusetts Institute of Technology. All rights reserved

24 Review of Convexity Linear Functions Proposition 5 A linear f(x) = a function x + b is both convex and concave. T Proposition 6 f(x) If is both convex and concave, f(x) then is a linear function. A linear function is convex and concave. cfl2002 Massachusetts Institute of Technology. All rights reserved

25 Separating Hyperplane Theory Strong Separation Theorem 1 Strong Separating Hyperplane Theorem: Suppose that S is a convex set in < n, and that we are given a point μx =2 S. Then there exists a vector u 6= 0 and a scalar ff for which the following hold: u T x > ff for all x 2 S u T μx < ff cfl2002 Massachusetts Institute of Technology. All rights reserved

27 Separating Hyperplane Theory Weak Separation Theorem 2 Weak Separating Hyperplane Theorem: Suppose that S is a convex set in < n, and that we are given a point μx =2 S or μx Then there exists a vector u 6= 0 and a scalar ff for which the following hold: u T x ff for all x 2 S u T μx» ff cfl2002 Massachusetts Institute of Technology. All rights reserved

29 Separating Hyperplane Theory Intersecting Line Dual Primal: minimum x height of x s:t: 2 S; x 2 L x Dual: maximum ß intercept of line ß with L s:t: S ß line lies below cfl2002 Massachusetts Institute of Technology. All rights reserved

31 OP : minimum x Definition of the Dual Problem General Constrained Problem Most General Format f(x) g 1 (x)» 0; s:t: =. m (x)» 0; g 2 P; x : < n 7! < f(x) i (x) : < n 7! <; i = 1; : : : ; m. g cfl2002 Massachusetts Institute of Technology. All rights reserved

32 ffl P = < n OP : z Λ = minimum x ffl P = Φ x j x 2 Z n + Ψ m (x)» 0; g 2 P; x Definition of the Dual Problem General Constrained Problem Standard Format... f(x) g 1 (x)» 0; s:t:. ffl P = fx j x 0g ffl P = fx j g i (x)» 0; i = m + 1; : : : ; m + kg P = fx j Ax» bg ffl cfl2002 Massachusetts Institute of Technology. All rights reserved

33 OP : z Λ = minimum x L(x; u) := f(x) + u T g(x) = f(x) + m P Definition of the Dual Problem General Constrained Problem...Standard Format... f(x) Form the Lagrangian function: g i (x)» 0; i = 1; : : : ; m; s:t: 2 P; x Solve the presumably easier problem: i=1 u i g i (x) L Λ (u) := minimum x f(x) + u T g(x) x 2 P s:t: The L function (u) is called the dual function. Λ We presume that computing L Λ (u) is an easy task. cfl2002 Massachusetts Institute of Technology. All rights reserved

34 L(x; u) := f(x) + u T g(x) = f(x) + m P Definition of the Dual Problem General Constrained Problem...Standard Format Form the Lagrangian function: Form the dual function L Λ (u): i=1 u i g i (x) L Λ (u) := minimum x f(x) + u T g(x) x 2 P s:t: The dual problem is then defined to be: D : v Λ = maximum u L Λ (u) cfl2002 Massachusetts Institute of Technology. All rights reserved

35 OP : z Λ = minimum x Steps in the Construction of the Dual Problem f(x) Step 1. Create the Lagrangian g i (x)» 0; i = 1;:::;m; s:t: 2 P; x u) := f(x) + u T g(x) : L(x; Step 2. Create the dual function: L Λ (u) := minimum x f(x) + u T g(x) Step 3. Create the dual problem: s:t: x 2 P D : v Λ = maximum u L Λ (u) s:t: u 0 cfl2002 Massachusetts Institute of Technology. All rights reserved

36 LP : minimum x Examples of Dual Constructions of OPs The Dual of a Linear Problem Consider the linear optimization problem: c T x s:t: Ax b What is the dual of this problem? cfl2002 Massachusetts Institute of Technology. All rights reserved

37 IP : minimum x The Dual of a Binary Integer Problem Examples of Dual Constructions of OPs Consider the binary integer optimization problem: c T x s:t: b Ax j 2 f0; 1g; j = 1; : : : ; n : x What is the dual of this problem? cfl2002 Massachusetts Institute of Technology. All rights reserved

38 BP : minimum x1 ;x 2 ;x 3 5x 1 + 7x 2 4x 3 3 P Examples of Dual Constructions of OPs The Dual of a Log-Barrier Problem Consider the following logarithmic barrier problem: ln(x j ) j=1 s.t. x 1 + 3x x 3 = 37 x 1 > 0; x 2 > 0; x 3 > 0 : What is the dual of this problem? cfl2002 Massachusetts Institute of Technology. All rights reserved

39 OP : minimum x f(x) + P i2l u i g i (x) + P u i g i (x) + P Examples of Dual Constructions of OPs Problems w/ Different Formats of Constraints Remarks... f(x) s:t: i (x)» 0; i 2 L g i (x) 0; i 2 G g g i (x) = 0; i 2 E The Lagrangian is: x 2 P; L(x; u) := f(x) + u T g(x) = u i g i (x) cfl2002 Massachusetts Institute of Technology. All rights reserved i2g i2e

40 u) := f(x) + u T g(x) = f(x) + P L(x; u i g i (x) + P i2l D : v Λ = maximum u u i g i (x) + P Examples of Dual Constructions of OPs Problems w/ Different Formats of Constraints...Remarks u i g i (x) The dual function L Λ (u): i2g i2e L Λ (u) := minimum x f(x) + u T g(x) The dual problem is: s:t: x 2 P L Λ (u) s:t: 0; i 2 L u» 0; i 2 G u > < 0; i 2 E u cfl2002 Massachusetts Institute of Technology. All rights reserved

41 OP : z Λ = minimum x The Dual is a Concave Maximization Problem f(x) s:t: g i (x)» 0; i = 1; : : : ; m x 2 P; L(x; u) := f(x) + u T g(x) L Λ (u) := minimum x f(x) + u T g(x) s:t: x 2 P Theorem 3 The dual function L Λ (u) is a concave function. cfl2002 Massachusetts Institute of Technology. All rights reserved

42 = min x2p f(x) + u T 1 g(x) Λ + (1 ) f(x) + u T 2 g(x) Λ min x2p f(x) + u T 1 g(x) Λ + (1 ) min x2p (f(x) + u T 2 g(x) Λ The Dual is a Concave Maximization Problem Proof of Concavity of L Λ (u) Proof: Let u 1 0 and u 2 0 be two values of the dual variables, and let u = u 1 + (1 )u 2, where 2 [0; 1]. Then L Λ (u) = min x2p f(x) + u T g(x) = L Λ (u 1 ) + (1 )L Λ (u 2 ) : Therefore L Λ (u) is a concave function. q.e.d. cfl2002 Massachusetts Institute of Technology. All rights reserved

43 OP : z Λ = minimum x g i (x)» 0; i = 1;:::;m s:t: 2 P; x Weak Duality f(x) D : v Λ = maximum u L Λ (u) s:t: u 0 Theorem 4 Weak Duality Theorem: If μx is feasible for OP and μu is feasible for D, then In particular, f(μx) L Λ (μu) Λ v Λ : z cfl2002 Massachusetts Institute of Technology. All rights reserved

44 Weak Duality Proof of Weak Duality Theorem 4 Weak Duality Theorem: If μx is feasible for OP and μu is feasible for D, then In particular, f(μx) L Λ (μu) z Λ v Λ : Proof: If μx is feasible for OP and μu is feasible for D, then + T g(μx) min x2p f(x) + μu T g(x) = L Λ (μu) : f(μx) f(μx) μu v Therefore. q.e.d. Λ z Λ cfl2002 Massachusetts Institute of Technology. All rights reserved

45 OP : z Λ = minimum x A = 1 (x) g 2 (x) g m (x) g f(x) A = Column Geometry of Primal and Dual Problems Resources and Costs For this section only, we assume equality format: f(x) g i (x) = 0; i = 1;:::;m s:t: 2 P; x D : v Λ = maximum u L Λ (u) s:t: u > < 0 For each x 2 P, we have an array of resources and costs associated with x: r z 1 r 2 r B = B. C C B B. C C g(x) r m z cfl2002 Massachusetts Institute of Technology. All rights reserved

46 Column Geometry of Primal and Dual Problems Resources and Costs S := Φ (r; z) 2 < m+1 j (r; z) = (g(x); f(x)) for some x 2 P Ψ : z L = {( r, z ) r = 0 } S H r 1 r 2 r = 0 The column geometry of the primal and dual problem. cfl2002 Massachusetts Institute of Technology. All rights reserved

47 Column Geometry of Primal and Dual Problems Column Geometry Interpretation of OP Every point in the intersection of S and L corresponds to a feasible solution of OP, and the feasible solution with the lowest cost corresponds to the lowest point in the intersection of S and L. This lowest value is exactly the value of the primal problem, namely z Λ. cfl2002 Massachusetts Institute of Technology. All rights reserved

48 Column Geometry of Primal and Dual Problems Supporting Hyperplanes of S H = H u;ff = Φ (r; z) 2 < m+1 j z + u T r = ff Ψ : L = Φ (r; z) 2 < m+1 j r = 0 Ψ : H L = f(r; z) = (0; ff)g cfl2002 Massachusetts Institute of Technology. All rights reserved

49 maximum u;ff Column Geometry of Primal and Dual Problems Maximum-Height Supporting Hyperplane of S ff s:t: H u;ff lies below S = maximum u;ff ff s:t: z + u T r ff; for all (r; z) 2 S = maximum u;ff ff s:t: f(x) + u T g(x) ff; for all x 2 P cfl2002 Massachusetts Institute of Technology. All rights reserved

50 maximum u;ff > 0 : < Column Geometry of Primal and Dual Problems Maximum-Height Supporting Hyperplane of S ff s:t: f(x) + u T g(x) ff; for all x 2 P = maximum u;ff ff s:t: L Λ (u) ff; = maximum u L Λ (u) s:t: u cfl2002 Massachusetts Institute of Technology. All rights reserved

51 Column Geometry of Primal and Dual Problems Maximum-Height Supporting Hyperplane of S The dual problem corresponds to finding that hyperplane H u;ff lying below S whose intersection with L is the highest. This highest value is exactly the value of the dual problem, namely v Λ. cfl2002 Massachusetts Institute of Technology. All rights reserved

54 Strong Duality Strong Duality Theorem Theorem 5 Strong Duality Theorem: Suppose that OP is a convex optimization problem, that is, all constraints of OP are of the form g i (x)» 0; i = 1; : : : ; m, f(x) as well as g i (x) are convex functions, i = 1; : : : ; m, and P is a convex set. Then under very mild additional conditions, z Λ = v Λ. cfl2002 Massachusetts Institute of Technology. All rights reserved

55 Duality Strategies Dualizing Bad Constraints OP : minimum x c T x s:t:» b Ax» g : Nx Suppose that optimization over the constraints Nx» g is easy. The addition of the constraints Ax» b makes the problem much more difficult. Let = fx j Nx» gg P cfl2002 Massachusetts Institute of Technology. All rights reserved

56 P = fx j Nx» gg Duality Strategies Dualizing Bad Constraints : minimum x c T x OP Ax» b s:t: Nx» g : OP : minimum x c T x Ax» b s:t: 2 P : x cfl2002 Massachusetts Institute of Technology. All rights reserved

57 P = fx j Nx» gg Duality Strategies Dualizing Bad Constraints : minimum x c T x OP Ax» b s:t: where x 2 P : The Lagrangian is: L(x; u) = c T x + u T (Ax b) = u T b + (c T + u T A)x L Λ (u) := minimum x u T b + (c T + u T A)x s:t: x 2 P : cfl2002 Massachusetts Institute of Technology. All rights reserved

58 Λ (u) := minimum x u T b + (c T + u T A)x L x 2 P : s:t: Duality Strategies Dualizing Bad Constraints D : maximum u L Λ (u) s:t: u 0 Notice that L Λ (u) is easy to evaluate for any value of u, and so we can attempt to solve OP by designing an algorithm to solve the dual problem D. cfl2002 Massachusetts Institute of Technology. All rights reserved

59 OP : minimum x1 ;x 2 (c1 ) T x 1 +(c 2 ) T x 2 Duality Strategies Dualizing A Large Problem into Many Small Problems B 1 x 1 +B 2 x 2» d s:t: 1 x 1» b 1 A 2 x 2» b 2 A Notice here that if it were not for the constraints 1 x 1 + B 2 x 2» d, that we would be able to separate B the problem into two separate problems. Let us dualize on these constraints. Let: P = Φ (x 1 ; x 2 ) j A 1 x 1» b 1 ; A 2 x 2» b 2Ψ cfl2002 Massachusetts Institute of Technology. All rights reserved

60 OP : minimum x 1 ;x 2 (c1 ) T x 1 +(c 2 ) T x 2 A 2 x 2» b 2 P = Φ (x 1 ;x 2 ) j A 1 x 1» b 1 ;A 2 x 2» b 2Ψ OP : minimum x 1 ;x 2 (c1 ) T x 1 +(c 2 ) T x 2 Duality Strategies Dualizing A Large Problem into Many Small Problems B 1 x 1 +B 2 x 2» d s:t: 1 x 1» b 1 A B 1 x 1 +B 2 x 2» d s:t: 1 ; x 2 ) 2 P (x cfl2002 Massachusetts Institute of Technology. All rights reserved

61 OP : minimum x 1 ;x 2 (c1 ) T x 1 +(c 2 ) T x 2 L Λ (u) = minimum x1 ;x 2 ut d + ((c 1 ) T + u T B 1 )x 1 + ((c 2 ) T + u T B 2 )x 2 Duality Strategies Dualizing A Large Problem into Many Small Problems B 1 x 1 +B 2 x 2» d s:t: 1 ;x 2 ) 2 P (x = Φ (x 1 ;x 2 ) j A 1 x 1» b 1 ;A 2 x 2» b 2Ψ P The Lagrangian is: L(x; u) = (c 1 ) T x 1 + (c 2 ) T x 2 + u T (B 1 x 1 + B 2 x 2 d) = u T d + ((c 1 ) T + u T B 1 )x 1 + ((c 2 ) T + u T B 2 )x 2 ; (x 1 ; x 2 ) 2 P s:t: cfl2002 Massachusetts Institute of Technology. All rights reserved

62 L Λ (u) = minimum x 1 ;x 2 ut d + ((c 1 ) T + u T B 1 )x 1 + ((c 2 ) T + u T B 2 )x 2 Duality Strategies Dualizing A Large Problem into Many Small Problems s:t: (x 1 ;x 2 ) 2 P L Λ (u) = T d u minimum A1 x 1»b 1 ((c1 ) T + u T B 1 )x minimum A 2 x 2»b 2 ((c2 ) T + u T B 2 )x 2 Notice once again that L Λ (u) is easy to evaluate for any value of u, and so we can attempt to solve OP by designing an algorithm to solve the dual problem: D : maximum u L Λ (u) cfl2002 Massachusetts Institute of Technology. All rights reserved

63 Lagrange Duality Illustrated Constrained Shortest Path Problem Definition Constrained Shortest Path Problem: find the shortest path from node to node 6, among those paths using 4 exactly arcs cfl2002 Massachusetts Institute of Technology. All rights reserved

64 Lagrange Duality Illustrated Constrained Shortest Path Problem Context In practice: Arcs might have two types of costs (time and money) There may be millions of nodes and billions of paths cfl2002 Massachusetts Institute of Technology. All rights reserved

66 Lagrange Duality Illustrated Constrained Shortest Path Motivation for Dual Suppose that we have available an efficient algorithm for solving (unconstrained) shortest path problems. We will use this algorithm to try to solve our problem through the dual cfl2002 Massachusetts Institute of Technology. All rights reserved

67 z Λ = min xij x 12 +x 13 +x 23 +x 24 +3x 34 +2x 35 +x 54 +x 46 +3x 56 Lagrange Duality Illustrated Constrained Shortest Path IP Formulation... CSPP : s.t. x 12 +x 13 +x 23 +x 24 +x 34 +x 35 +x 54 +x 46 +x 56 = 4 x 12 +x 13 = 1 x 12 x 23 x 24 = 0 x 13 +x 23 x 34 x 35 = 0 x 24 +x 34 +x 54 x 46 = 0 x 35 x 54 x 56 = 0 x 46 +x 56 = 1 x ij 2 f0; 1g for all x ij. cfl2002 Massachusetts Institute of Technology. All rights reserved

68 CSPP : minimum xij c ij x ij Lagrange Duality Illustrated Constrained Shortest Path...IP Formulation... P i;j P x ij = 4 s:t: i;j Nx = b x ij 2 f0; 1g cfl2002 Massachusetts Institute of Technology. All rights reserved

69 CSPP : minimum xij c ij x ij Lagrange Duality Illustrated Constrained Shortest Path...IP Formulation P i;j P x ij = 4 s:t: i;j where: x 2 P P = fx j Nx = b ; x ij 2 f0; 1g for all arcs i jg cfl2002 Massachusetts Institute of Technology. All rights reserved

71 L Λ (u) := minimum xij 4 P i;j Lagrange Duality Illustrated Constrained Shortest Path L Λ (u)... ψ P c ij x ij + u x ij! i;j s:t: Nx = b x ij 2 f0; 1g cfl2002 Massachusetts Institute of Technology. All rights reserved

72 Lagrange Duality Illustrated Constrained Shortest Path...L Λ (u) Λ (u) = min xij (1 u)x 12 +(1 u)x 13 +(1 u)x 23 L u)x 24 +(3 u)x 34 +(2 u)x 35 +(1 +(1 u)x 54 +(1 u)x 46 +(3 u)x 56 +4u s.t. x 12 +x 13 = 1 x 12 x 23 x 24 = 0 x 13 +x 23 x 34 x 35 = 0 x 24 +x 34 +x 54 x 46 = 0 x 35 x 54 x 56 = 0 x 46 +x 56 = 1 x ij 2 f0; 1g for all x ij. cfl2002 Massachusetts Institute of Technology. All rights reserved

73 Lagrange Duality Illustrated Constrained Shortest Path Comments on L Λ (u) For a fixed value of u, 4u is a constant. For a fixed value of u, this is a shortest path problem with modified arc costs given by: c 0 ij = c ij u : cfl2002 Massachusetts Institute of Technology. All rights reserved

75 Lagrange Duality Illustrated Constrained Shortest Path The set P Recall that P is the set of solutions of: x 12 +x 13 = 1 x 12 x 23 x 24 = 0 x 13 +x 23 x 34 x 35 = 0 x 24 +x 34 +x 54 x 46 = 0 x 35 x 54 x 56 = 0 x 46 +x 56 = 1 x ij 2 f0; 1g for all x ij. cfl2002 Massachusetts Institute of Technology. All rights reserved

77 Lagrange Duality Illustrated Constrained Shortest Path Duality Gap ffl The optimal u is u = 1:5 ffl The value of the dual is v Λ = 4:5 ffl The value of the primal is z Λ = 5:0 ffl There is a duality gap: z Λ v Λ = 5:0 4:5 = 0:5. cfl2002 Massachusetts Institute of Technology. All rights reserved

Duality Theory of Constrained Optimization

Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive