3.1 The Revised Simplex Algorithm. 3 Computational considerations. Thus, we work with the following tableau. Basic observations = CARRY. ... m.

Size: px

Start display at page:

Download "3.1 The Revised Simplex Algorithm. 3 Computational considerations. Thus, we work with the following tableau. Basic observations = CARRY. ... m."

Allan Morrison
5 years ago
Views:

1 3 Computational conideration In what follow, we analyze the complexity of the Simplex algorithm more in detail For thi purpoe, we focu on the update proce in each iteration of thi procedure Clearly, ince, up to now, the illutration of the algorithm wa the primary aim, we updated the ableau completely However, if we code the algorithm in ome computer language thi i not efficient herefore, we introduce the o-called revied Simplex algorithm uine Computing and Operation Reearch he Revied Simplex Algorithm y conidering the Simplex Algorithm generated above, it turn out that we have to update a complete (m+x(n+ tableau throughout the calculation proce Additional analye, however, how that we can reduce thi effort by keeping a ignificantly maller (m+x(m+ tableau Specifically, thi i etablihed by making ue of the following cheme uine Computing and Operation Reearch 329 aic obervation If we commence our calculation with the identity matrix E m, we finally obtain the inverted matrix of A correponding to a current bai hu, we tart with a zero cot row a the obective function. hi enable u to determine the correponding dual olution Since c i given a a parameter, we can identify the relative cot by uing a imple tranformation At iteration l, we denote the conidered (m+x(m+ matrix a CARRY (l In addition, we commence the calculation with CARRY ( hu, we work with the following tableau... b... m = = CARRY ( b E m b m... Z π ( l = CARRY A b A uine Computing and Operation Reearch 33 uine Computing and Operation Reearch 33

2 Da ildelement mit der eziehung-id rid9 wurde in der Datei nicht gefunden. ranformation Additionally, we keep track of the bai, i.e., the current baic variable that form the current bf In order to carry out the Primal Simplex Algorithm, the following tep are neceary. Generate the relative cot c = c π a one at a time until we either find a negative - ay for = - value or we terminate with the cognition that the current olution i optimal already 2. Column Generation: ranformation Generate the column a = A a Generate the pivot b element with r b p r = min p {,..., m} a >. Clearly, p, a a r, p, if r doe not exit, the problem i unbounded. 3. Update CA ( l to obtain ( l+ RRY CARRY. y making ue of - vector a, we are able to update A and b accordingly.. Update the bai accordingly. Specifically, we replace (r by. uine Computing and Operation Reearch 332 uine Computing and Operation Reearch 333 Column generation he econd tep introduce new column, i.e., a new alternative in the tableau y making ue of the inverted matrix of the current A, we can iteratively generate the column of the original primal tableau In tep 3, we apply all updating operation to the inverted matrix a well a to the left-hand ide, i.e., to b = A b Applied to the wo-phae Method Note that applying the Revied Simplex to the wo-phae Method come along with everal pecific we mut attend to Firt of all, the initial olution coincide with the maximal uage of the m auxiliary variable (for every row one variable A Hence, the inverted matrix i In order to commence with correct cot value for the column that doe not belong to the bai, we determine m c = ai, i= π a For the econd phae, we have to adut the obective function row to π = c A uine Computing and Operation Reearch 33 uine Computing and Operation Reearch 335 2

3 he Revied Primal Simplex Algorithm I he Revied Primal Simplex Algorithm II In what follow, we aume that the right hand ide b i poitive. If thi doe not apply we have to ue the two-phae method intead. ranform the primal problem into canonical form. Generate equation with the lack variable denoted a x,,x m and let x m+,,x m+n be the tructure variable. Obtain a minimization obective function. 2. Start with the feaible baic olution to the primal problem given by the lack variable: Obtain, b = b π = and A : = E m, ( =,, ( m = m { } 3. Search for a pivot column : = min If exit: Calculate a : = A a. c = c π a < If i =,..., m : a i, then terminate ince the olution pace i unbounded. Pivot row r: λ : = min { b i / a i i =,..., m : a i > } that i an upper bound on x. Form the elementary matrix r m r a ( a m P = e,..., p,..., e with p = p : =,..., p : =,..., p : = r rr mr a a a Column r r r r ai change (x enter the bai and x (r become a non-baic variable Calculate A : = P A and et (r:= Calculate b : = A, : b π = c A Go to tep 3. Otherwie ( c =,..., n : ermination. An optimal baic olution to the primal problem i found. uine Computing and Operation Reearch 336 uine Computing and Operation Reearch 337 Example Revied Simplex Method ( Maximize 2 3 x 2 2, x x A =,c = ( 2 3,b = ( =, ( 2 = 2 Z =,π = (, A =, A b = b = 3 We commence with 2 3 c = = = 3 λ = min, = 5 r = 3 3 uine Computing and Operation Reearch 338 Example Revied Simplex Method 2 3 λ = min, = 5 r = ( = 3, ( 2 = 2 3 a a r... m Auming ɶ a the former bai, we get A = P A e... p... e A, with p ɶ = ɶ = a Column - r r (... a m a r hu, we h 2 5 ave A = 3 = b = b = = 5 π c A = = ( = ( 3 uine Computing and Operation Reearch 339 3

4 Example Revied Simplex Method 2 c = ( =, c = c =, c = 2 ( =, c = 3 ( = = a = A a = = λ = min, = = r = 2 ( = 3, ( 2 = hu, we have A = = = b = b = = 3, π = c = ( 2 = ( uine Computing and Operation Reearch 3 Example Revied Simplex Method ( 2 2 ( 2 c = =, c = =, c =, c =, c = ( = = a = A a = = 3 λ = min 3, 3 = r = 2 ( = 3, ( 2 = 5 hu, we have A = = b = b = =, π = c = ( = ( uine Computing and Operation Reearch 3 Example Revied Simplex Method ( ( c = 2 =, c = 2 = 2, c =, c = ( 2 2 =, c = x ( π ( 2 =,,,, = are optimal 2 3 c x = b π = Analyzing the complexity Clearly, at firt glance, you would aume that the main complexity effect of the revied implex algorithm i from the fact that it application reduce the value to be updated from an (m+x(n+ tableau to a (m+x(m+ matrix However, we have to generate the reduced cot by iteratively computing for a conidered non-baic column. hu, if we have to do thi for all thee non-baic column thi require ( multiplication ut thi i not ignificantly le than the number of multiplication needed to update the tableau in the ordinary implex method! uine Computing and Operation Reearch 32 uine Computing and Operation Reearch 33

5 Poitive effect of the revied implex he practical complexity reducing effect of the revied implex are omewhat more ubtle but ignificant nonethele Firt, it i quite likely that we need not compute the relative cot of every non-baic column if we (for intance alway take the firt column with negative cot (improving alternative (e.g. rule of land hi ignificantly reduce on the average the computational effort to ome fraction hi fraction i determined by the average number of column that mut be examined until one with negative relative cot i identified Poitive effect of the revied implex Second, each pricing operation (i.e., the calculation of the reduced cot value for a conidered column ue the column of the original ableau We will ee in many practical application in Combinatorial Optimization thee matrice are often very pare (many zero entrie hu, the neceary prizing computation can be performed very efficiently Moreover, the original matrix can be tored in a very compact way uine Computing and Operation Reearch 3 uine Computing and Operation Reearch 35 Further refinement earing thee idea in mind, we can alternatively tore the invere bai matrix in product form and not a an (m+x(m+ matrix. Each pivot operation can be repreented a a multiplication with a matrix P that equal the (m+x(m+ identity matrix except for column r that contain the vector η x, xr, x 2, xr,... = x r,... xm, x r, xm+, x r, rth row uine Computing and Operation Reearch 36 Update at any tage l y uing the current vector η, we can efficiently generate all matrice P and generate ( = Moreover, due to the current vector η, the matrix P can be tored very efficiently However, if the equence of η become too long, it can be replaced by a horter equence hi replacement proce i denoted a reinverion It generate an equivalent but horter equence of pivot to attain the current bai Such technique can greatly reduce the torage and time required to perform the implex algorithm, epecially if pecial attention i paid to reducing the number of nonzero element in the η- equence (ee Orchard-Hay (968 or Ladon (97. uine Computing and Operation Reearch 37 5

6 3.3 Solving the Max Flow Problem In anticipation of the application that are conidered in the Section 6 and 7 where we introduce and conider the Shortet Path Problem and the Max Flow Problem, in what follow, we how an intereting application of the revied implex procedure oth application are network problem that can be formulated a Linear Program and the contraint matrix can be derived directly from the graph underlying the problem he Max Flow Problem 3.3. Definition Given a flow network =,,,, with = node and = arc, an intance of the Max Flow Problem (MFP i defined a the optimization problem of finding a flow on each edge with maximal value v from the ingle ource node to the ingle terminal node t. On each edge, thi flow ha to be lower or equal to the capacity vector. uine Computing and Operation Reearch 38 uine Computing and Operation Reearch 39 Specific problem definition We now formulate thi problem a an LP in a omewhat urpriing way Since the arc are numbered by e,,e m, we introduce C,,C p a a complete enumeration of every chain (i.e., path from to t A known from the revied implex algorithm introduced above, the theoretical Linear Program cover m row and p column but the conidered LP in each tep will comprie only m column Arc-chain incidence matrix D=[d i, ] In order to unambiguouly define poible path, we determine the o-called arc-chain incidence matrix a follow: if ei C i {,..., m} : {,..., p} : di, = otherwie he capacity contraint are given by: he obective function purue the maximization of all the flow in all defined chain, i.e., minc f D f b with (,..., c = uine Computing and Operation Reearch 35 uine Computing and Operation Reearch 35 6

7 Complete Linear program hu, we obtain the following LP y introducing a lack vector, we tranform thi LP into tandard form and extend the repective vector a follow hi lead to the LP min c f.t. D f b f ( ( ˆ m ˆ ( Flow vector: fˆ = f, cot vector c = f, and D = D E min z=ˆ c fˆ.t. Dˆ fˆ = b fˆ f uine Computing and Operation Reearch 352 Conequence Clearly, each lack variable i repreent the difference between the flow in arc i and the capacity b i, with i=,,m We now apply the revied implex algorithm Fortunately, we do not have to apply the two-phae method ince thi problem i olvable with the trivial olution =, = that repreent the zero flow he criterion for a new column to enter the bai are negative reduced cot, given by c = c π D < π D < π D < Since c = D i the th column of the arc-chain incidence matrix D uine Computing and Operation Reearch 353 Conequence Max Flow Problem Example Clearly, " i the cot of the chain/path under the weight vector In order to find a profitable column, we need to find the hortet chain from to t under the weight vector that weight le than If that hortet chain/path, ay, ha cot no le than, then the optimality criterion i atified If not, we introduce into the bai he calculation therefore require only the maintenance of an (m+x(m+ CARRY matrix and the repeated olution of the hortet-path problem b = b 2 = e 5 e e 2 b 5 = b 3 = e 3 e b = t uine Computing and Operation Reearch 35 uine Computing and Operation Reearch 355 7

8 he matrix CARRY ( Dual olution # = = -z= = 2 = 3 = = 5 = he initial dual olution i zero, i.e., π=, and therefore, the initial weight are zero Hence, each path from to t i minimal and ha the length < herefore, it i a candidate to be integrated into the bai In what follow, we will determine a chain/path Max Flow Problem path election We introduce a hortet chain (of length zero into the bai In order to complicate the computation a little bit, we tart with a chain/path that i not in the optimal olution Specifically, we introduce the chain/path = $, $ %, $ &. hi i illutrated below e 5 e e 2 e 3 e t uine Computing and Operation Reearch 356 uine Computing and Operation Reearch 357 Introducing C he correponding column i ' = = We update the current CARRY ( matrix Dual olution # = = -z= -z= = C = 2 = 2 = 3 = 3 = = = - 5 = 5 = - - uine Computing and Operation Reearch 358 Max Flow Problem path election We again introduce a hortet chain (of length zero into the bai Now, we introduce the chain/path ( = $ $ &. hi i illutrated below c = c 2 = e 5 e e 2 c 5 = c 3 = e 3 e c = uine Computing and Operation Reearch 359 t 8

9 Introducing C 2 he column ( i and we have ' ( = We update the current CARRY (2 matrix - c = π D Dual olution # = = -z= -z= C = C = 2 = 2 = 3 = 3 = - = - C 2 = - 5 = - 5 = - uine Computing and Operation Reearch 36 - Max Flow Problem path election We again introduce a hortet chain (of length zero into the bai Now, we introduce the chain/path = $ $ (. hi i illutrated below c = c 2 = e 5 e e 2 c 5 = c 3 = e 3 e c = uine Computing and Operation Reearch 36 t Introducing C 3 he column i and we have ' = We update the current CARRY (3 matrix - c = π D Dual olution # = = -z= -z=2 C = C = - 2 = C 3 = 3 = - 3 = - - C 2 = - C 2 = - 5 = - 5 = - uine Computing and Operation Reearch Max Flow Problem path election No zero chain/path exit between and t Hence, an optimal olution with maximal flow 2 i found It comprie the path ( = $ $ & and = $ $ ( c = c 2 = e 5 e e 2 c 5 = c 3 = e 3 e c = uine Computing and Operation Reearch 363 t 9

10 Alternatively introducing C 3 he column i and we have ' = We update the current CARRY (3 matrix - c = π D Dual olution # = = -z= -z=2 C = C 3 = 2 = 2 = - 3 = - 3 = - C 2 = - C 2 = 5 = - 5 = uine Computing and Operation Reearch Max Flow Problem path election No zero chain/path exit between and t Hence, an optimal olution with maximal flow 2 i found It comprie the path ( = $ $ & and = $ $ ( c = c 2 = e 5 e e 2 c 5 = c 3 = e 3 e c = uine Computing and Operation Reearch 365 t 3. he Dantzig-Wolfe Decompoition It often happen that a large linear program i actually a collection of maller linear program that are largely independent of each other herefore, the revied implex algorithm allow u to decompoe the entire problem into maller mater- and ubproblem he communication between thee problem can be directly derived from the revied implex hank to thi decompoition, problem that may become too large to be olved efficiently (e.g., due to pace requirement can be olved very fat Structure of the LP We analyze the entire LP with two ubproblem hi lead to the following cheme of a matrix n column n 2 column D F A m row m row m 2 row With variable * +, correponding to the firt column and. + / correponding to the next ( column uine Computing and Operation Reearch 366 uine Computing and Operation Reearch 367

11 min z = c x + d y.t. D x + F y = b A x = b x, y y = o 2 he complete LP We denote the m equation the coupling equation he ucceeding et of row are the ubproblem Aand he ubproblem A We conider the contraint of ubproblem A, i.e., A x = b x y heorem.3.9, we know that any feaible point in thi ubproblem can be written a a convex combination of edge point. We denote thee point a *,, *, with p { } x = λ x where,..., p : λ and λ = = = p uine Computing and Operation Reearch 368 uine Computing and Operation Reearch 369 Analogouly: he ubproblem We conider the contraint of ubproblem, i.e., y = o y y heorem.3.9, we know that any feaible point in thi ubproblem can be written a a convex combination of edge point. We denote thee point a.,,. 2, with q 2 { } y = µ y where,..., q : µ and µ = = = uine Computing and Operation Reearch 37 q Modified tructure of the LP We replace x and y by their derived repreentation hi lead to the following modified cheme (in what follow, denoted a the mater problem Cot Variable 3,, 3,, 2 Right hand ide Dimenion "*,, "* 5.,, 5. 2 m row,,,, row,,,, row p min z = λ c x + µ d y = = q p q while λ, µ are the new variable λ IR µ IR uine Computing and Operation Reearch 37

12 Conequence Due to the tranformation, we may obtain an atronomical number of column, one for each vertex of each of the two ubproblem However, the number of row i ignificantly reduced from + + ( to +2 Furthermore, the revied implex method can be applied with a CARRY matrix of ize Hence, much larger intance may fit into fat-acce torage Size of CARRY: intead of ( + + ( ( + If, for intance, it hold that = = ( =, we have + 3 ( intead of 3 + (, i.e., almot a factor of 3 2 =9 uine Computing and Operation Reearch 372 Applying the revied implex In the firt row, we have the price (reduced cot hi i a partitioned vector, :, ; where < correpond to the firt row and :, ; to the next two row of the mater problem, repectively he reduced cot of the 3 -column are given by D x = ( π α β c c x,, = c x π D x α Hence, the criterion for a column to be brought into the current bai i c x π D x α < c x π D x < α uine Computing and Operation Reearch 373 Pricing problem A If c x π D x < α hold for any vertex of Subproblem A, we have found a profitable column among the firt p column However, ince an exhautive exploration of the total number of poible column (i.e., for each poible vertex of Subproblem A would be impractical, we purue the finding of the optimal vertex of the following LP hu, the pricing problem i ( π min all vertice x of Subprobelm π = A c x D x c D x min ( c π D x.t. A x = b x Pricing problem A If the obective function value of the optimal olution i lower than α the reulting column can be introduced into the bai Hence, the mater problem integrate the new column into the bai and, therefore, update the current olution he newly attained olution reult in a new price vector ent to ubproblem A Otherwie, no further improvement i poible and the olution of the mater problem i kept unchanged according to the input of Subproblem A uine Computing and Operation Reearch 37 uine Computing and Operation Reearch 375 2

13 Pricing problem Similarly, we can determine if there i a favorable column among the lat q column by olving min ( d π F y.t. y = b y and comparing the attained minimum cot to β, by an argument analogou to that urrounding the pricing problem A 2 uine Computing and Operation Reearch 376 Procedure decompoition opt := 'no' et up CARRY with zero row (π, α, β; while opt = 'no' do begin olve the LP min # " * = =. t. * =, * if A < : then generate the column correponding to the olution with thi cot, and pivot in CARRY ele begin olve the LP min C 5. = =. t. '. = (,. if A < ; then generate the column correponding to the olution with thi cot, and pivot in CARRY ele opt := 'ye' end end uine Computing and Operation Reearch 377 Summary We verbally um up the operation of the decompoition method: After being olved, the mater problem, baed on it overall view of the entire ituation, end a price to ubproblem A hi ubproblem A then repond with a olution (a propoal for poibly improving the overall problem, baed on it local information and the price he mater problem then weigh the cot (A of thi propoal againt it criterion α for ubproblem A If the propoal i cheaper than α, it i implemented by bringing it into the bai. hi reult in an update of the current olution (and price If not, ubproblem i ent a price vector and aked for a propoal. A long a a ubproblem can produce a favorable propoal, the mater problem can find a favorable pivot When neither ubproblem can come up with a favorable propoal, we have reached an optimal olution of the entire problem Concluion What we have done for two ubproblem can clearly be done for any number of ubproblem, in which cae the contraint matrix will take the form hown below m row m row m 2 row m r row A A 2 A r uine Computing and Operation Reearch 378 uine Computing and Operation Reearch 379 3

14 An illutrative example Additional literature to Section 3 A little arithmetic how the effectivene of the approach in ize reduction of the mapped problem Suppoe we have ubproblem, each with row, and coupling equation. hen the original problem ha, row CARRY ha, x, entrie, i.e., approximately 8 entrie. May be too many for the working memory of a computer he mater problem ha a CARRY with 2 x 2 entrie. hee are approximately x entrie, which i practical to tore in the fat memory of any reaonably large computer All in all, the clear advantage of the decompoition algorithm i it effect on the pace requirement Note that it cannot be aid much about the time requirement, becaue we do not know how many time the ubproblem mut be olved before optimality i reached eale, E.M.L. (968: Mathematical Programming in Practice. John Wiley & Son, Inc., New York. Dantzig, G.., Orden, A., and Wolfe, P. (955: he generalized implex method for minimizing a linear form under linear inequality retraint. Pacific Journal of Mathematic 5, Dantzig, G.., Wolfe, P. (96: "Decompoition Principle for Linear Programming. Operation Reearch, 8(, -. Dantzig, G.. (963: Linear Programming and Extenion. Princeton Univerity Pre, Princeton, N.J. Orchard/Hay, W. (968: Advanced Linear-Programming Computing echnique. McGraw-Hill ook Company, New York. Ladon, L.S. (97: Optimization heory for Large Sytem. London: MacMillan, Inc. uine Computing and Operation Reearch 38 uine Computing and Operation Reearch 38

Problem Set 8 Solutions

Problem Set 8 Solutions Deign and Analyi of Algorithm April 29, 2015 Maachuett Intitute of Technology 6.046J/18.410J Prof. Erik Demaine, Srini Devada, and Nancy Lynch Problem Set 8 Solution Problem Set 8 Solution Thi problem