Linear Programming. Linear Programming. Brewery Problem: A Toy LP Example. Applications

Transcription

1 Linear Programming Linear Programming Linear programming implex method LP duality What is it? Quintessential tool for optimal allocation of scarce resources among a number of competing activities. (e.g. see ORF 7) Powerful and general problem-solving method. shortest path max flow min cost flow generalized flow multicommodity flow T matching 2-person zero sum games Why significant? Fast commercial solvers: CPLEX OL. Powerful modeling languages: APL GA. Ranked among most important scientific advances of 2 th century. Also a general tool for attacking NP-hard optimization problems. Dominates world of industry. ex: Delta claims saving $1 million per year using LP Reference: The Allocation of Resources by Linear Programming cientific American by Bob Bland. Princeton University CO 226 Algorithms and Data tructures pring 24 Kevin Wayne 2 Applications Brewery Problem: A Toy LP Example Agriculture. Diet problem. Computer science. Compiler register allocation data mining. Electrical engineering. VLI design optimal clocking. Energy. Blending petroleum products. Economics. Equilibrium theory two-person zero-sum games. Environment. Water quality management. Finance. Portfolio optimization. Logistics. upply-chain management Berlin airlift. anagement. otel yield management. arketing. Direct mail advertising. anufacturing. Production line balancing cutting stock. edicine. Radioactive seed placement in cancer treatment. Operations research. Airline crew assignment vehicle routing. Physics. Ground states of -D Ising spin glasses. Plasma physics. Optimal stellarator design. Telecommunication. Network design Internet routing. ports. cheduling ACC basketball handicapping horse races. mall brewery produces ale and beer. Production limited by scarce resources: corn hops barley malt. Recipes for ale and beer require different proportions of resources. Beverage Corn (pounds) ow can brewer maximize profits? ops (ounces) alt (pounds) Beer Quantity Devote all resources to ale: 4 barrels of ale $442. Devote all resources to beer: 2 barrels of beer $ barrels of ale 29.5 barrels of beer $ barrels of ale 28 barrels of beer $8. Profit ($) 1 2 4

2 Brewery Problem Brewery Problem: Feasible Region ops + 4B 16 alt + 2B 119 Beer max 1A A 2B 4B 2B B Profit Corn ops alt ( 2) (12 28) (26 14) Corn + 48 Beer ( ) (4 ) 5 6 Brewery Problem: Objective Function Brewery Problem: Geometry Brewery problem observation. Regardless of objective function coefficients an optimal solution occurs at an extreme point. Extreme points Profit ( 2) ( 2) (12 28) 1A + 2B = $16 (12 28) (26 14) (26 14) Beer 1A + 2B = $8 Beer ( ) (4 ) 1A + 2B = $442 ( ) (4 ) 7 8

3 tandard Form LP Brewery Problem: Converting to tandard Form "tandard form" LP. Input: real numbers c j b i a ij. Output: real numbers x j. n = # nonnegative variables m = # constraints. aximize linear objective function subject to linear inequalities. Original input. max 1A A 2B 4B 2B B (P) max n cj x j j 1 n aij x j j 1 x j bi 1 i m 1 j n (P) max c T x Ax x b tandard form. Add slack variable for each inequality. Now a 5-dimensional problem. Linear. No x 2 xy arccos(x) etc. Programming. Planning (term predates computer programming). max 1A A 2B 4B 2B B C C Geometry Geometry Geometry. Inequalities : halfplanes (2D) hyperplanes. Bounded feasible region: convex polygon (2D) (convex) polytope. Convex: if a and b are feasible solutions then so is (a +b) / 2. Extreme point: feasible solution x that can't be written as (a + b) / 2 for any two distinct feasible solutions a and b. Extreme point property. If there exists an optimal solution to (P) then there exists one that is an extreme point. Only need to consider finitely many possible solutions. Challenge. Number of extreme points can be exponential! Consider n-dimensional hypercube. Greed. Local optima are global optima. Extreme point is optimal if no neighboring extreme point is better. extreme point a a b b Convex Not convex 11 12

4 implex Algorithm implex Algorithm: Basis implex algorithm. (George Dantzig 1947) Developed shortly after WWII in response to logistical problems. Used for 1948 Berlin airlift. Generic algorithm. never decrease objective function tart at some extreme point. Pivot from one extreme point to a neighboring one. Repeat until optimal. ow to implement? Linear algebra. Basis. ubset of m of the n variables. Basic feasible solution (BF). et n - m nonbasic variables to solve for remaining m variables. olve m equations in m unknowns. If unique and feasible solution BF. BF corresponds to extreme point! implex only considers BF. max 1A A 2B C 4B 2B B C {B } ( 2) Beer Basis {A B } (12 28) Infeasible {A B } ( ) {A B C } (26 14) 1 { C } ( ) {A C } (4 ) 14 implex Algorithm: Pivot 1 implex Algorithm: Pivot 1 1A 2B C 4B 2B A B C Basis = { C } A = B = = C = 48 = 16 = 119 1A 2B C 4B 2B A B C Basis = { C } A = B = = C = 48 = 16 = 119 ubstitute: B = 1/15 (48 C ) 16 A 2 1 A B 15 1 C 8 A 4 85 A 4 C A B C Basis = {B } A = C = = 76 B = 2 = 2 = 55 Why pivot on column 2? Each unit increase in B increases objective value by $2. Pivoting on column 1 also OK. Why pivot on row 2? Preserves feasibility by ensuring R. inimum ratio rule: min { 48/15 16/4 119/2 }

5 implex Algorithm: Pivot 2 implex Algorithm: Optimality 16 A 2 1 A B 1 8 A 4 85 A 4 C A B C Basis = {B } A = C = = 76 B = 2 = 2 = 55 When to stop pivoting? If all coefficients in top row are non-positive. Why is resulting solution optimal? Any feasible solution satisfies system of equations in tableaux. in particular: = 8 C 2 Thus optimal objective value * 8 since C. ubstitute: A = /8 (2 + 4/15 C ) Current BF has value 8 optimal. A A C B 1 1 C 1 1 C 25 6 C B C Basis = {A B } C = = = 8 B = 28 A = 12 = 11 A A C B 1 1 C 1 1 C 25 6 C B C Basis = {A B } C = = = 8 B = 28 A = 12 = implex Algorithm: Issues LP Duality: Economic Interpretation Remarkable property. In practice simplex algorithm typically terminates in at most 2(m+n) pivots. No polynomial pivot rule known. ost pivot rules known to be exponential in worst-case. Issues. Which neighboring extreme point? Degeneracy. New basis same extreme point. "talling" is common in practice. Cycling. Get stuck by cycling through different bases that all correspond to same extreme point. Doesn't occur in the wild. Bland's least index rule finite # of pivots. Brewer's problem: find optimal mix of beer and ale to maximize profits. (P) max 1A 2B 48 4B 16 2B 119 A B Entrepreneur's problem: Buy individual resources from brewer at minimum cost. C = unit price for corn hops malt. Brewer won't agree to sell resources if 5C < 1. (D) min 48C 5C 15C C A* = 12 B* = 28 OPT = 8 C* = 1 * = 2 * = OPT =

6 LP Duality LP Duality: Economic Interpretation Primal and dual LPs. Given real numbers a ij b i c j find real numbers x i y j that optimize (P) and (D). (P) max n cj x j j 1 n aij x j bi 1 i m j 1 x j 1 j n (D) min m bi yi i 1 m aij yi cj i 1 yi 1 j n 1 i m Duality Theorem (Gale-Kuhn-Tucker 1951 Dantzig-von Neumann 1947). If (P) and (D) have feasible solutions then max = min. pecial case: max-flow min-cut theorem. ensitivity analysis. ensitivity analysis. Q. ow much should brewer be willing to pay (marginal price) for additional supplies of scarce resources? A. corn $1 hops $2 malt $. Q. New product "light beer" is proposed. It requires 2 corn 5 hops 24 malt. ow much profit must be obtained from light beer to justify diverting resources from production of beer and ale? A. Breakeven: 2 ($1) + 5 ($2) + 24 ($) = $12 / barrel. ow do I compute marginal prices (dual variables)? implex solves primal and dual simultaneously. Top row of final simplex tableaux provides optimal dual solution! 22 2 istory istory 199. Production planning. (Kantorovich UR) Propaganda to make paper more palatable to communist censors Production planning. (Kantorovich) implex algorithm. (Dantzig) "I want to emphasize again that the greater part of the problems of which I shall speak relating to the organization and planning of production are connected specifically with the oviet system of economy and in the majority of cases do not arise in the economy of a capitalist society." "the majority of enterprises work at half capacity. There the choice of output is determined not by the plan but by the interests and profits of individual capitalists." UR UA Kantorovich awarded 1975 Nobel prize in Economics for contributions to the theory of optimum allocation of resources. taple in BA curriculum. Used by most large companies and other profit maximizers

7 istory istory 199. Production planning. (Kantorovich) implex algorithm. (Dantzig) 195. Applications in many fields. ilitary logistics. Operations research. Control theory. Filter design. tructural optimization Production planning. (Kantorovich) implex algorithm. (Dantzig) 195. Applications in many fields. Geometric divide-and-conquer. olvable in polynomial time: O(n 4 L) bit operations. n = # variables L = # bits in input Theoretical tour de force not remotely practical istory istory 199. Production planning. (Kantorovich) implex algorithm. (Dantzig) 195. Applications in many fields Projective scaling algorithm. (Karmarkar) O(n.5 L). Efficient implementations possible Production planning. (Kantorovich) implex algorithm. (Dantzig) 195. Applications in many fields Projective scaling algorithm. (Karmarkar) 199. Interior point methods. O(n L) and practical. Extends to even more general problems. 29

8 istory istory 199. Production planning. (Kantorovich) implex algorithm. (Dantzig) 195. Applications in many fields Projective scaling algorithm. (Karmarkar) 199. Interior point methods. Interior point faster when polyhedron smooth like disco ball. implex faster when polyhedron spiky like quartz crystal Production planning. (Kantorovich) implex algorithm. (Dantzig) 195. Applications in many fields Projective scaling algorithm. (Karmarkar) 199. Interior point methods. Current research. Approximation algorithms. oftware for large scale optimization. Interior point variants. 1 2 Ultimate Problem olving odel Perspective Ultimate problem-solving model? hortest path. aximum flow. in cost flow. Generalized multicommodity flow. Linear programming. emidefinite programming.... TP (or any NP-complete problem) tractable intractable (conjectured) LP is near the deep waters of NP-completeness. olvable in polynomial time. Known for less than 25 years. Integer linear programming. LP with integrality requirement. NP-hard. Does P = NP? No universal problem-solving model exists unless P = NP. An unsuspecting BA student transitions from tractable LP to intractable ILP in a single mouse click. 4