IS703: Decision Support and Optimization. Week 5: Mathematical Programming. Lau Hoong Chuin School of Information Systems

Transcription

1 IS703: Decision Support and Optimization Week 5: Mathematical Programming Lau Hoong Chuin School of Information Systems 1

2 Mathematical Programming - Scope Linear Programming Integer Programming Network flows (an introduction in previous class) Not Covered: Conve Programming Non-Linear Programming etc 2

3 Linear Programming (Adapted from Powerful model generalizes many classic problems: shortest path, ma flow, multicommodity flow, MST, matching, 2- person zero sum games Essential tool for optimal allocation of scarce resources, among a number of competing activities. Ranked among most important scientific advances of 20 th century. accounts for a major proportion of all scientific computation Helps find "good" solutions to NP-hard optimization problems. optimal solutions (branch-and-cut) provably good solutions (randomized rounding) 3

4 Linear Programming LP "standard" form. Input data: rational numbers c j, b i, a ij. Maimize linear objective function. Subject to linear inequalities. (P) ma s. t. n j = 1 n j = 1 c a j ij j j j b 0 i 1 i m 1 j n (P) ma s. t. c A b 0 4

5 LP: Geometry Geometry. Forms an n-dimensional polyhedron. (P) ma s. t. n j = 1 n j = 1 c a j ij j j j b 0 i 1 i m 1 j n Conve: if y and z are feasible solutions, then so is αy + (1- α)z, for all 0<α<1. Etreme points y y z z Conve Not conve 5

6 LP: Geometry Etreme Point Theorem. If there eists an optimal solution to standard form LP (P), then there eists one that is an etreme point. Only need to consider finitely many possible solutions. Greed. Local optima are global optima. 6

7 LP: Algorithms Simple. (Dantzig 1947) Developed shortly after WWII in response to logistical problems: used for 1948 Berlin airlift. Practical solution method that moves from one etreme point to a neighboring etreme point. Finite (eponential) compleity, but no polynomial implementation known. 7

8 LP: Polynomial Algorithms Ellipsoid. (Khachian 1979, 1980) Solvable in polynomial time: O(n 4 L) bit operations. n = # variables L = # bits in input Theoretical tour de force. Not remotely practical. Karmarkar's algorithm. (Karmarkar 1984) O(n 3.5 L). Polynomial and reasonably efficient implementations possible. Interior point algorithms. O(n 3 L). Competitive with simple! will likely dominate on large problems soon Etends to even more general problems. 8

9 LP Application: Weighted Bipartite Matching Assignment problem. Given a complete bipartite network K n,n and edge weights c ij, find a perfect matching of minimum weight. min s. t. 1 i n 1 j n c 1 i n Birkhoff-von Neumann theorem (1946, 1953). All etreme points of the above polyhedron are {0-1}-valued. Corollary. Can solve assignment problem using LP techniques since LP algorithms return optimal solution that is an etreme point. Remarks. (1) The above theorem is directly proven by applying the concept of totally unimodular matri. (2) Polynomial combinatorial algorithms also eist. ij 1 j n ij ij ij ij = = i j 1 i, j n n n 9

10 10 LP Application: Multicommodity Flow Multicommodity flow. Given a network G = (V, E) with edge capacities u(e) 0, edge costs c(e) 0, set of commodities K, and supply / demand d k (v) for commodity k at node v, find a minimum cost flow that satisfies all of the demand. Applications. Transportation networks. Communication networks. VLSI design. K k E e e E e e u e K k V v v d e e e e c k K k k k v e k v e k E e k k K k =, 0 ) ( ) ( ) (, ) ( ) ( ) ( t. s. ) ( ) ( min out of in to Remark. The problem is NP-hard if flows are integral!

11 Integer Programming An Integer Programming model is a linear programming problem where some or all of the variables are required to be non-negative integers. These models are in general substantially harder than solving linear programming models. Network models are special cases of integer programming models and are very efficiently solvable. Several applications of integer programming models. Branch and bound technique, one of the most popular algorithm to solve integer programming models.

12 12 Integer Programming Problem: Given rational numbers a ij, b i, c j, find integers j that solve: Claim. INTEGER-PROGRAMMING is NP-hard. Proof. VERTEX-COVER P INTEGER-PROGRAMMING. n j n j m i b a c j j i n j j ij n j j j = = 1 integral t. s. min 1 1

13 Weighted Verte Cover Weighted verte cover. Given an undirected graph G = (V, E) with verte weights w v 0, find a minimum weight subset of nodes S such that every edge is incident to at least one verte in S. NP-hard even if all weights are 1. Integer programming formulation. 10 A F6 9 ( ILP) min s. t. w v V v v + v w v 1 {0, 1} ( v, w) E v V 16 6 B C3 G7 H 10 9 If * is optimal solution to (ILP), then S = {v V: * v = 1} is a min weight verte cover D E I 10 J Total weight =

14 Classifications of IP Models Pure IP Model: Where all variables must take integer values. Maimize z = subject to , 2 0, 1 and 2 integer Mied IP Model: Where some variables must be integer while others can take real values. Maimize z = subject to , 2 0, 1 integer 0-1 IP Model: Where all variables must take values 0 or 1. Maimize z = 1-2 subject to , 1, 2 = 0 or 1

15 Classifications of IP Models (contd.) LP Relaation: The LP obtained by omitting all integer or 0-1 constraints on variables is called the LP relaation of IP. IP: LP Relaation: Maimize z = subject to , 2 0, 1 and 2 integer Maimize z = subject to , 2 0 Fact: Optimal objective value of IP Optimal objective value of LP relaation

16 Solving IP via LP relaation Solve the LP relaation Round-off the solution to the nearest feasible integer solution Any potential difficulty with this approach? =

17 Capital Budgeting Problem Stockco Co. is considering four investments It has $14,000 available for investment Formulate an IP model to maimize the NPV obtained from the investments IP: Investment choice Cash $5000 $7000 $4000 $3000 outflow NPV $16000 $22000 $12000 $8000 Maimize z = subject to , 2,, 3, 4 = 0, 1

18 Fied Charge Problem Gandhi cloth company manufactures three types of clothing: shirts, shorts, and pants Machinery must be rented on a weekly basis to make each type of clothing. Rental Cost: $200 per week for shirt machinery $150 per week for shorts machinery $100 per week for pants machinery There are 150 hours of labor available per week and 160 square yards of cloth Find a solution to maimize the weekly profit Labor hr Cloth yd Var. cost Price Shirts 3 4 $12 $6 Shorts 2 3 $8 $4 Pants 6 4 $15 $8

19 Fied Charge Problem (contd.) Decision Variables: 1 = number of shirts produced each week 2 = number of shorts produced each week 3 = number of pants produced each week y 1 = 1 if shirts are produced and 0 otherwise y 2 = 1 if shorts are produced and 0 otherwise y 3 = 1 if pants are produced and 0 otherwise Formulation: Ma. z = y y 2-100y 3 subject to M y 1, 2 M y 2, 3 M y 3 1, 2,, 3 0, and integer; y 1, y 2,, y 3 0, and integer

20 Either-Or Constraints Dorian Auto is considering manufacturing three types of auto: compact, midsize, large. Resources required and profits obtained from these cars are given below. We have 6,000 tons of steel and 60,000 hours of labor available. If any car is produced, we must produce at least 1,000 units of that car. Find a production plan to maimize the profit. Compact Midsize Large Steel Req. 1.5 tons 3 tons 5 tons Labor Req. 30 hours 25 hours 40 hours Profit $2000 $3000 $4000

21 Either-Or Constraints (contd.) Decision Variables: 1, 2, 3 = number of compact, midsize and large cars produced y 1, y 2, y 3 = 1 if compact, midsize and large cars are produced or not Formulation: Maimize z = subject to 1 My 1 ; 2 My 2 ; 3 My M(1-y 1 ) M(1-y 2 ) M(1-y 3 ) , 2, 3 0 and integer; y 1, y 2, y 3 = 0 or 1

22 Set Covering Problems Western Airlines has decided to have hubs in USA. Western runs flights between the following cities: Atlanta, Boston, Chicago, Denver, Houston, Los Angeles, New Orleans, New York, Pittsburgh, Salt Lake City, San Francisco, and Seattle. Western needs to have a hub within 1000 miles of each of these cities. Determine the minimum number of hubs Atlanta (AT) Boston (BO) Chicago (CH) Denver (DE) Houston (HO) Los Angeles (LA) New Orleans (NO) New York (NY) Pittsburgh (PI) Salt Lake City (SL) San Francisco (SF) Seattle (SE) Cities within 1000 miles AT, CH, HO, NO, NY, PI BO, NY, PI AT, CH, NY, NO, PI DE, SL AT, HO, NO LA, SL, SF AT, CH, HO, NO AT, BO, CH, NY, PI AT, BO, CH, NY, PI DE, LA, SL, SF, SE LA, SL, SF, SE SL, SF, SE

23 Formulation of Set Covering Problems Decision Variables: i = 1 if a hub is located in city i i = 0 if a hub is not located in city i Minimize AT + BO + CH + DE + HO + LA + NO + NY + PI + SL + SF + SE subject to AT BO CH DE HO LA NO NY PI SL SF SE Required AT AT >= 1 BO BO >= 1 CH CH >= 1 DE DE >= 1 HO HO >= 1 LA LA >= 1 NO NO >= 1 NY NY >= 1 PI PI >= 1 SL SL >= 1 SF SF >= 1 SE SE >= 1

24 IP: Logistics/Supply Chain Applications Set covering and partitioning Scheduling airline crews Manpower rostering Facility location Locating warehouses or franchises Multicommodity distribution Distributing auto parts Traveling salesman / Vehicle routing Routing pickups and deliveries Eercise: Model TSP as an IP

25 Branch and Bound The most popular approach for solving integer programming problems Enumerate the entire solution space but only implicity; hence they are called implicit enumeration algorithms. A general-purpose solution technique which must be specialized for individual IP's. Running time grows eponentially with the problem size, but small to moderate size problems can be solved in reasonable time. A divide and conquer algorithm, where a problem is divided into smaller and smaller subproblems. Each subproblem is solved separately, and the best solution is taken.

26 Branch-and-Bound Search Represent search space as trees Search tree Each leaf node represents a feasible solution Non-leaf node represents a subset of feasible solutions Parent-child relationship represents partition of search space into sub-spaces Idea: if we can drastically reduce solution space from one level to net, total number of nodes eplored will be reduced 26

27 Branch-and-Bound Search S (a) S1 S2 S4 S3 S1 S2 S3 S4 (b) S21 S1 * S31 S22 S32 * S4 S1 S2 S3 S4 * = does not contain optimal solution (c) S21 S22 S23 S24 Figure 1: Illustration of the search space of B&B. 27

28 Elements of Branch-and-Bound 1. Branching rule Rule for dividing the solution space into sub-spaces (sub-problem) 2. Node-selection (search/eploration) strategy Select net node to be eplored net DFS, BFS, Best-First Search 3. Bounding function Function that computes the lower bound for the best solution obtainable in that search space 4. Upper bounding Process of obtaining an upper bound for each subproblem 5. Pruning strategy If for a subproblem, UB LB, then the subproblem need not be eplored further 28

29 Bounding Function f g S P Figure 2: The relation between the bounding function g and the objective function f on the sets S and P of feasible and potential solutions of a problem. 29

30 Branch-and-Bound Algorithm (sketch) Start with complete feasible solution space and a naïve initial solution upper bound = objective value of best solution found so far Among uneplored nodes do Select a non-leaf node based on selection strategy Partition the feasible space into sub-spaces based on branching rule For each sub-space do If leaf node reached and beats best solution, update best solution, else discard and eit Compute lower-bound If lower bound >= upper bound, prune the sub-space Else create a sub-space node and add to list of uneplored nodes 30

31 Eample: TSP Find optimal tour by systematically considering all tours Strategy 1 (ehaustive search): Consider all permutations of the cities (vertices) Evaluate each tour (given by particular verte order) Keep track of shortest tour (n-1)! permutations, each takes O(n) time to evaluate Don t look at all n permutations, since we don t care about starting point of tour: A,B,C,(A) is same tour as C,A,B,(C) Unacceptable for large n Strategy 2 (dynamic programming): O(n 2 2 n ) 31

32 Eample: TSP Strategy 3: Branch-and-Bound Construct a tree with a root that represents all tours (tours are feasible solutions) Each node has two children One child represents tours with a particular edge Other child represents tours without that edge Choose edges in leicographic order, i.e. ab, ac, ad, etc 32

33 TSP Search Tree all tours tours with (a,b) tours without (a,b) tours with (a,b) and (a,c) pruned (infeasible) tours with (a,b) and (a,c) not (a,d) tours with (a,b) and (a,d) not (a,c) tours with (a,b) not (a,c) tours with (a,b) not (a,c) or (a,d) tours with (a,c) not (a,b) : not shown tours with (a,d) not (a,b) or (a,c) tours without (a,b) or (a,c) tours without (a,d), (a,b) or (a,c) 33

34 Lower bound Compute a lower bound on the cost of feasible solutions below node n If current solution is better (lower) than lower bound for node n, need not eplore any nodes below n For TSP, lower bound on cost of tour What s an obvious loose lower bound? Tighter bound: half the sum over all nodes n of the 2 least cost edges incident upon n More precisely, let e 1 (n) and e 2 (n) be the two least cost edges incident to n, then LB = ( n V e 1 (n)+e 2 (n)) / 2 Proof. Left as an eercise. (Hint: consider situation at each node, and sum up over all nodes.) 34

35 Eample 3 a 3 b e 8 c d 2 least cost edges adjacent to a are: (a,d) and (a,b) for total of 2+3=5 2 least cost edges adjacent to b are: (a,b) and (b,e) for total of 3+3=6 c, d, e: total of 8, 7, 9 respectively lower bound on cost of tour is ( )/2 =

36 Lower Bound for Subset of Tours Lower bound on the cost of node in search tree Each node represents tours defined by a set of edges that must be in tour and set of edges that may not be included Assume we must include (a,e) and eclude (b,c) a: (a,d) and (a,e), total 9 // must include (a,e) instead of (a,b) b: (a,b) and (b,e), total 6 c: (a,c) and (c,d), total 9 // cannot use (b,c) d: (a,d) and (c,d), total 7 e: (a,e) and (b,e), total 10 // must include (a,e) instead of (e,d) lower bound = ( )/2 =

37 Reconstruct Search Tree Each time we branch, decide on which edge to include/eclude based on: Include (,y) if it becomes impossible for or y to have two adjacent edges in tour otherwise Eclude (,y) if it raises degree of or y above 2 Eclude (,y) if it would complete a non-tour cycle with selected edges After branch, compute lower bounds for both children Prune child whose lower bound is as high or higher than best tour found so far If neither child can be pruned, eplore child with smaller lower bound first 37

38 New Search Tree LB A 17.5 no constraints B 17.5 ab C 18.5 ab D 20.5 ac ad ae E 18 ac J 18.5 ac K 21 ac ad ae pruned after discovery of I F 18 ad ae G 23 ad ae L 18.6 ad ae M 23.5 ad ae pruned H I pruned after discovery of I N P pruned bc bd be ce de cd bc cd ce bd de be bc cd ce bd de be bc bd be ce de cd tour= abceda cost=23 tour= abecda cost=21 tour= acbeda cost=19 tour= acebda cost=23 38

39 Analysis Search space: still (n-1)! in worst-case But much better on average Depends on the 3 elements of B&B 39

40 0-1 Knapsack Problem ma z = subject to the following constraints: <= 14 j = 0 or 1, for all j A simple way to get the initial solution: Set i = 1 for i = 1,2, until the constraint is violated. e.g. 1 =1, 2 =1, 3 =0, 4 =0. Lower bound = = 38 Integer programming (IP) formulation Any sub-space that has an upper bound <=38 can be pruned. What is a good upper bound? 40

41 Upper Bounding Function: Fractional Knapsack Problem ma z = subject to the following constraints: <= 14 0<= j <=1, for all j p j To solve it : order items according to nonincreasing value w j and make the ' s in this order as large as possible, i.e. Greedy! j j p j w j ratio ranking LP solution : Linear programming (LP) Relaation = = = 0.5, 0 2 = 1, 41

42 Branch and Bound Tree 1 Subproblem 1 z = 44, LB=38 1 = 2 = 1 3 =.5 3 = 0 3 = = 0 Subproblem 2 z = 43.3, LB=38 1 = 2 =1 3 = 0, 4 =.67 8 Subproblem 8 z = 38, LB=42 1 = 2 =1 3 = 4 = 0 9 pruned LB=42 4 = 1 Subproblem 9 z= 42.85, LB=42 1 = 4 =1 3 = 0, 2 =.85 pruned 2 = 0 pruned Subproblem 3 z = 43.4, LB=38 1 = 3 = 1, 2 =.7, 4 =0 3 4 Subproblem 4 Subproblem 5 z = 36, LB=38 z = 43.6, LB=38 1 = 3 =1 1 =.6, 2 = 3 =1 2 = 0, 4 =1 4 = 0 1 = 0 1 = = 1 5 Subproblem 6 z = 42, LB=38 1 =0, 2 = 3 =1 4 = 1 6 Subproblem 7 LB = 42 Infeasible 42

43 Practical Developments Good formulations, heuristics and theory Goal: to get LP solution as close as possible to IP solution Disaggregation, adding constraints (cuts) Preprocessing Automatic methods for reformulation Some interesting graph theory is involved Cut generation (branch-and-cut) Add cuts during the branch-and-bound Column generation Improve formulation by introducing an eponential number of variables. 43

44 Eercise: Generalized Assignment Problem In this problem, we assign m jobs to n machines Machine: The total available time of machine j is T j Jobs: If job i is performed on machine j, it costs a ij, and requires t ij time units. A job must be assigned to one machine in its entirety The problem is to find a minimum cost assignment of all jobs. Design a B&B approach to find the optimal solution. 44