Numerical Optimization

Transcription

1 Linear Programming Computer Science and Automation Indian Institute of Science Bangalore , India. NPTEL Course on

2 min x s.t. Transportation Problem ij c ijx ij 3 j=1 x ij a i, i = 1, 2 2 i=1 x ij b j, j = 1, 2, 3 x ij 0 i, j a i : Capacity of the plant Fi b j : Demand of the outlet Rj c ij : Cost of shipping one unit of product from Fi to Rj x ij : Number of units of the product shipped from Fi to Rj (variables) The objective is to minimize ij c ijx ij 3 j=1 x ij a i, i = 1, 2 (constraints) 2 i=1 x ij b j, j = 1, 2, 3 (constraints) x ij 0 i, j (constraints)

3 The Diet Problem: Find the most economical diet that satisfies minimum nutritional requirements. Number of food items: n Number of nutritional ingredient: m Each person must consume at least b j units of nutrient j per day Unit cost of food item i: c i Each unit of food item i contains a ji units of the nutrient j Number of units of food item i consumed: x i Constraint corresponding to the nutrient j: Cost: a j1 x 1 + a j2 x a jn x n b j, x i 0 i c 1 x 1 + c 2 x c n x n

4 Problem: min s.t. c 1 x 1 + c 2 x c n x n a j1 x 1 + a j2 x a jn x n b j j x i 0 i Given: c = (c 1,..., c n ) T, A = (a 1... a n ), b = (b 1,..., b m ) T. Linear Programming Problem (LP): min s.t. c T x Ax b x 0 where A R m n, c R n and b R m. Assumption: m n, rank(a) = m Linear Constraints can be of the form Ax = b or Ax b

5 Constraint (Feasible) Set: Inequality constraint of the type {x : a T x b} or {x : a T x b} denotes a half space Equality constraint, {x : a T x = b}, represents an affine space Non-negativity constraint, x 0 Constraint set of an LP is a convex set Polyhedral Set X = {x : Ax b, x 0} Polytope: A bounded polyhedral set

6 Consider the constraint set in R 2 : {(x 1, x 2 ) : x 1 + x 2 2, x 1 1, x 1 0, x 2 0}

7 Consider the constraint set in R 2 : {(x 1, x 2 ) : x 1 1, x 2 x 1 }

8 Feasible set can be a singleton set Feasible Set = {(x 1, x 2 ) : x 1 + x 2 = 2, x 1 + x 2 = 1} = {A}

9 Feasible set can be empty! Feasible Set = {(x 1, x 2 ) : x 1 + x 2 2, x 1 + x 2 1} = φ

10 Definition Let X be a convex set. A point x X is said to be an extreme point (corner point or vertex) of X if x cannot be represented as a strict convex combination of two distinct points in X. Extreme Points: A, B, C and D. E is not an extreme point.

11 Extreme Point: A

12 Constraint Set: X = {(x 1, x 2 ) : x 1 + x 2 2, x 1 1, x 1 0, x 2 0} 4 constraints in R 2 Two constraints are binding (active) at every extreme point Fewer than two constraints are binding at other points

13 Consider the constraint set: X = {x : Ax b, x 0} where A R m n and rank(a) = m. m + n hyperplanes associated with m + n halfspaces m + n halfspaces define X An extreme point lies on n linearly independent defining hyperplanes of X If X is nonempty, the set of extreme points of X is not empty and has a finite number of points. An edge of X is formed by intersection of n 1 linearly independent hyperplanes Two extreme points of X are said to be adjacent if the line segment joining them is an edge of X

14 For example, B and C are adjacent points Adjacent extreme points have n 1 common binding linearly independent hyperplanes

15 Remarks: Consider the constraint set: X = {x : Ax = b, x 0} where A R m n and rank(a) = m. Let x X be an extreme point of X m equality constraints are active at x Therefore, n m additional hyperplanes (from the non-negativity constraints) are active at x

16 Geometric Solution of a LP: min c T x s.t. Ax b x 0 where A R m n, c R n and b R m.

17

18

19

20

21

22

23 Example: min 2x 1 x 2 s.t. x 1 + x 2 5 x 1 + 2x 2 6 x 1, x 2 0

24 Example: min 2x 1 x 2 s.t. x 1 + x 2 5 x 1 + 2x 2 6 x 1, x 2 0

25 Example: min 2x 1 x 2 s.t. x 1 + x 2 5 x 1 + 2x 2 6 x 1, x 2 0

26 Example: min 2x 1 x 2 s.t. x 1 + x 2 5 x 1 + 2x 2 6 x 1, x 2 0

27 Example: min s.t. c T x x X

28 Example: min s.t. c T x x X

29 Example: min s.t. c T x x X

30 Example: min s.t. c T x x X

31 Consider a linear programming problem LP: min c T x s.t. a T i x (, =, ) b i, i = 1,..., m x 0 Let X = {x : a T i x (, =, ) b i, i = 1,..., m, x 0}. Remarks: X is a closed convex set The set of optimal solutions is a convex set. The linear program may have no solution or a unique solution or infinitely many solutions. If x is an optimal solution to LP, then x must be a boundary point of X. If z = c T x, then {x : c T x = z} is a supporting hyperplane to X. If X is compact and if there is an optimal solution to LP, then at least one extreme point of X is an optimal solution to the linear programming problem.

32 LP in Standard Form: min s.t. c T x Ax = b x 0 where A R m n and rank(a) = m. Assumption: Feasible set is non-empty

33 Any linear program can be converted to the Standard Form. (a) max c T x = min c T x (b) Constraint of the type a T x b, x 0 can be written as a T x + y = b x 0 y 0

34 (c) Constraint of the type a T x b, x 0 can be written as a T x z = b x 0 z 0 (d) Free variables (x i R) can be defined as x i = x i + xi, x i + 0, xi 0

35 Example: min x 1 2x 2 3x 3 s.t. x 1 + 2x 2 + x 3 14 x 1 + 2x 2 + 4x 3 12 x 1 x 2 + x 3 = 2 x 1, x 2 unrestricted Write the constraints as equality x 3 3constraints x 1 + 2x 2 + x 3 + x 4 = 14, x 4 0 x 1 + 2x 2 + 4x 3 x 5 = 12, x 5 0 Define new variables x + 1, x 1, x+ 2, x 2 and x 3 such that x 1 = x + 1 x 1, where x+ 1 0, x 1 0 x 2 = x + 2 x 2, where x+ 2 0, x 2 0 x 3 = 3 x 3 so that x 3 0

36 Therefore, the program min x 1 2x 2 3x 3 s.t. x 1 + 2x 2 + x 3 14 x 1 + 2x 2 + 4x 3 12 x 1 x 2 + x 3 = 2 x 1, x 2 unrestricted x 3 3 can be converted to the standard form: min x + 1 x 1 2(x+ 2 x 2 ) + 3(3 + x 3) s.t. x + 1 x 1 + 2(x+ 2 x 2 ) x 3 + x 4 = 17 x + 1 x 1 + 2(x+ 2 x 2 ) 4x 3 x 5 = 24 x + 1 x 1 x+ 2 + x 2 x 3 = 5 x + 1, x 1, x+ 2, x 2, x 3, x 4, x 5 0

37 Consider the linear program in standard form (SLP): min s.t. c T x Ax = b x 0 where A R m n, rank(a) = rank(a b) = m. Let B R m m be formed using m linearly independent columns of A. Therefore, the system of equations, Ax = b can be written as, ( ) xb (B N) = b. Letting x N = 0, we get x N Bx B = b x B = B 1 b. (x B : Basic Variables) (x B 0) T : Basic solution w.r.t. the basis matrix B

38 Basic Feasible Solution If x B 0, then (x B w.r.t. the basis matrix B. 0) T is called a basic feasible solution of Ax = b x 0 Theorem Let X = {x : Ax = b, x 0}. x is an extreme point of X if and only if x is a basic feasible solution of Ax = b x 0.

39 Proof. (a) Let x be a basic feasible solution of Ax = b, x 0. Therefore, x = (x 1,..., x }{{ m, 0,..., 0). Let B = (a }}{{} 1 a 2... a m ) 0 n m where a 1,..., a m are linearly independent. So, x 1 a x m a m = b. Suppose x is not an extreme point of X. Let y, z X, y z and x = αy + (1 α)z, 0 < α < 1. Since y, z 0, we have y m+1 =... = y n = 0 z m+1 =... = z n = 0 } and y 1a y m a m = b z 1 a z m a m = b Since a 1,..., a m are linearly independent, x = y = z, a contradiction. So, x is an extreme point of X.

40 Proof.(continued) (b) Let x be an extreme point of X. x X Ax = b, x 0. There exist n linearly independent constraints active at x. m active constraints associated with Ax = b. n m active constraints associated with n m non-negativity constraints x is the unique solution of Ax = b, x N = 0. Ax = b Bx B + Nx N = b x B = B 1 b 0 Therefore, x = (x B x N ) T is a basic feasible solution. ( ) n Number of basic solutions m Enough to search the finite set of vertices of X to get an optimal solution

41 Theorem Let X be non-empty and compact constraint set of a linear program. Then, an optimal solution to the linear program exists and it is attained at a vertex of X. Proof. Objective function, c T x, of the linear program is continuous and the constraint set is compact. Therefore, by Weierstrass Theorem, optimal solution exists. The set of vertices, {x 1,..., x k }, of X is finite. Therefore, X is the convex hull of x 1,..., x k. Hence, for every x X, x = k i=1 α ix i where α i 0, k i=1 α i = 1. Let z = min 1 i k c T x i. Therefore, for any x X, z = c T x = k i=1 α ic T x i z k i=1 α i = z. So, the minimum value of c T x is attained at a vertex of X.

42 Consider the constraints: x 1 + x 2 2 x 1 1 x 1, x 2 0

43 The given constraints x 1 + x 2 2 x 1 1 x 1, x 2 0 can be written in the form, Ax = b, x 0: Let A = x 1 +x 2 + x 3 = 2 x 1 +x 4 = 1 x 1, x 2, x 3, x 4 0 ( ) ( = (a a 2 a 3 a 4 ) and b = 1 ).

44 ( ) A = = (a a 2 a 3 a 4 ) and b = ( ) 1 1 (1) B = (a 1 a 2 ) = 1 0 ( 2 1 ). x B = (x 1 x 2 ) T = B 1 b = (1 1) T and x N = (x 3 x 4 ) T = (0 0) T. x = (x B x N ) T corresponds to the vertex C.

45 ( ) A = = (a a 2 a 3 a 4 ) and b = ( ) 1 1 (2) B = (a 1 a 3 ) = 1 0 ( 2 1 ). x B = (x 1 x 3 ) T = B 1 b = (1 1) T and x N = (x 2 x 4 ) T = (0 0) T. x = (x B x N ) T corresponds to the vertex B.

46 ( ) ( A = = (a a 2 a 3 a 4 ) and b = 1 ( ) 1 0 (3) B = (a 1 a 4 ) = 1 1 ). x B = (x 1 x 4 ) T = B 1 b = (2 1) T and x N = (x 2 x 3 ) T = (0 0) T. x = (x B x N ) T is not a basic feasible point

47 ( ) A = = (a a 2 a 3 a 4 ) and b = ( ) 1 0 (4) B = (a 2 a 4 ) = 0 1 ( 2 1 ). x B = (x 2 x 4 ) T = B 1 b = (2 1) T and x N = (x 1 x 3 ) T = (0 0) T. x = (x B x N ) T corresponds to the vertex D.

48 ( ) A = = (a a 2 a 3 a 4 ) and b = ( ) 1 0 (5) B = (a 3 a 4 ) = 0 1 ( 2 1 ). x B = (x 3 x 4 ) T = B 1 b = (2 1) T and x N = (x 1 x 2 ) T = (0 0) T. x = (x B x N ) T corresponds to the vertex A.

49 Example: min 3x 1 x 2 s.t. x 1 + x 2 2 x 1 1 x 1, x 2 0