Department of Chemcal and Bologcal Engneerng LECURE NOE III Chapter 4 Lnear Programmng Ref: Luenberger, D. G., "Lnear and Nonlnear Programmng,: 2nd Ed., Addson-Wesley, 1984 - Obectve functon and constrants are lnear. - Feasble regon: the regon that satsfes all constrans - Unque optmal soluton vs. alternatve (or multple) optmal soluton - Unbounded optmum: nfnte obectve functon value (± ) due to nsuffcent constrants - An optmal soluton to LP, f t ests, wll be on one of the vertces generated by constrants Eample: Let 1 and 2 denote the number of grade-1 and grade-2 nspectors assgned to nspecton. he grade-1 nspector can nspect 2 peces wth error rate of 2% and 12 peces wth error rate of 5% for grade-2 nspector, daly. he hourly wages are $4 and $3 for grade-1 and grade-2, respectvely. he erroneous nspecton costs $2 for each pece. Snce the number of avalable nspectors n each grade s lmted, we have followng constrants: 1 8 (grade-1) and 2 1 (grade-2) he company requres at least 18 peces be nspected daly. Formulate the problem to mnmze the cost for the company. <Soluton> otal cost for company (obectve functon): Z ($4 / hr8 hr $2 / ea2ea.2) 1 ($3/ hr 8 hr $2 / ea 12ea.5) 2 4 36 1 2 Constrans: 1 8, 2 1, 21 122 18 1, 2 Graphcal nterpretaton: feasble regon: shaded area (satsfyng all constrants) optmum pont: at pont A, (8, 1.6), the obectve functon Z has the mnmum of 377.6. optmum soluton: 1 =8, 2 =1.6 (one of grade-2 nspector s utlzed only 6%) (If not possble choose 2 =2.) I. Basc Propertes 1. Standard form: mn( c c c ) subect to 1 1 2 2 n a111 a12 2 a1 nn b1 a211 a222 a2nn b2 a a a b n m1 1 m2 2 mn n m and 1, 2,, n. III-1
A. subect Department of Chemcal and Bologcal Engneerng Or n vector-matr form, mn c to A = b and (Wthout loss of generalty, let b ) where the coeffcent matr A s mn, the decson vector s n1, the cost (proft) vector c s n1, and the requrement vector b s m1. (n optmzaton varables and m constrants wth scalar obectve functon) Converson to the standard form for the dfferent types of constrants <E. 1> A b A s b and s (s : "slack varable") <E. 2> A b A s b and s cf) (s : "surplus varable") If slack or surplus varable s, the nequalty constrant s actve, and f they are postve, then the constrant s bndng. <E. 3> u-v u and v (replace s wth u - v) <E. 4> Mamzaton of Z s equvalent to mnmzaton of (-Z). <E. 5> ma( 12 2 3 3) mn( 12 2 34 3 5) subect to subect to 12 3 7 1 2 4 5 6 7 12 3 2 1 2 4 5 7 2 3 2 5 3 2 2 5 1 2 3 1 2 4 5 and 1, 2. and ( 1,2,4,5,6,7) 2. Basc Feasble Soluton - A = b (constrants) m n Let A n m,, and b. n varables wth m equatons: (n-m) degrees of freedom (n>m) Full rank assumpton: m rows of A are lnearly ndependent. cf) f n=m, unque soluton ests f rank(a)=m. f n<m, t s a over-determned system and there s no soluton f rank(a)>n. f n>m, t s a under-determned system and there are many solutons. (select the optmal among many solutons) III-2
B. mn C. subect Department of Chemcal and Bologcal Engneerng - Let A = [B N] and = [ 1 2 ] where B s mm, N s m(n-m), 1 s m1, and 2 s (n-m)1. A = B 1 + N 2 = b - Only m varables can be fed and others can have any values to satsfy the m-constrants. By lettng 2 = and rearrange the varable ndces so that the frst m varables s B, -1 1 B b B (Basc varable) (Nonbasc varable) 2 N B (Basc soluton) If B, = [ B ] s the basc feasble soluton. If some of the elements of B are zero, = [ B ] s the degenerate basc feasble soluton. 3. he Fundamental heorem of LP c to A = b and mn Assume rank A m, ) f there s a feasble soluton, there s a basc feasble soluton; ) f there s an optmal feasble soluton, there s an optmal basc feasble soluton. It s suffcent to search only over the basc feasble solutons! n n! No. of basc feasble solutons (some are not feasble) m ( n m)! 4. Geometrc Meanng of Basc Feasble Solutons <E. 1> 1 + 2 + 3 = 1 (m=1) 1, 2, 3 1 B, or 1, or 1 <E. 2> 1 + 2 + 3 = 1 2 1 + 3 2 = 1 (m=2) 1, 2, 3 B 1/2, or 1/3 1/2 2/3 Soluton can be found among basc feasble solutons (vertces). Etreme ponts of the conve polyhedron Etreme ponts of the conve polyhedron { A = b} Lnear varety, Affne space (Not a lnear space have not orgn) { A = b, } Conve polytope (has etreme ponts) III-3
Department of Chemcal and Bologcal Engneerng Etreme pont : A pont n a conve set C s sad to be an etreme pont of C f there are no two dstnct pont 1 and 2 n C such that 1(1 ) 2 for some 1. heorem (Equvalence of etreme ponts and basc solutons): Let A be an mn matr of rank m and b an m-vector. Let K be the conve polytope consstng of all n-vector satsfyng A = b,. A vector s an etreme pont of K f and only f s a basc feasble soluton of A = b,. Corollary 1: If the conve set K correspondng to A = b, s nonempty, t has at least one etreme pont. Corollary 2: If there s a fnte optmal soluton to a lnear programmng problem, there s a fnte soluton whch s an etreme pont of the constrant set. Corollary 3: he constrant set K correspondng to A = b, possesses at most a fnte number of etreme ponts. Corollary 4: If the conve polytope K correspondng to A = b, s bounded, then K s a conve polyhedron, that s, K conssts of ponts that are conve combnatons of a fnte number of ponts. 5. Searchng the Basc Soluton, Pvot Operaton Elementary row operaton : Add the constant multple of a row to another row (Does not alter the soluton of the lnear system equaton) 1 d1 1 Let E1 (mm matr) 1 1 E 1 A = E 1 b Addton of d 1 tmes of the -th row to the frst row Pvot operaton : Reduce the coeffcent of a specfc varable to unty n on of the equaton and zero elsewhere. Let E = E 1 E 2 E 3 E m whch wll perform a pvot operaton and conssts of m-elementary row operatons where 1 a1 q / apq 1 a2q / apq E (mm matr) 1/ apq amq / apq 1 hen EA = Eb wll have a form that s pvoted by the pq-element. By m-pvot operatons (or by Gauss-Jordan elmnaton) for A = b a11 a12 a 1 1n 1 b1 y1, m1 y1 n 1 y1 a 1 21 a22 a 2n 2 b y 2 2, m 1 y 2n 2 y 2 a 1 y m1 am2 amn n bm mm, 1 ymnn ym Row Echelon Form, or Canoncal Form III-4
Department of Chemcal and Bologcal Engneerng Set m+1 = m+2 = = n =. hen 1 = y 1, 2 = y 2, m = y m, y y B 1 m (A basc Soluton) Alternatvely, a11 a12 a1 1 1 1, 1 1 1 n b y m yn 1 y 1 a21 a22 a y 2 2 2 2, m 1 y2n 1 n b 2 y 2 a y 1 2 mm, 1 ymn 1 m am amn n b m n y m Set 1 = 2 = = n-m =. hen m +1 = y' 1, m +2 = y' 2, n = y' m, y y B 1 m (An alternatve basc soluton) he ERO's are appled to both matr A and vector b. herefore, apply ERO's for the followng augmented matr. a11 a12 a1 n b1 a21 a22 a2n b 2 am1 am2 amn bm 6. Fndng a Mnmum Feasble Soluton Whle Preservng Feasblty Adacent basc feasble soluton: It s a basc feasble soluton whch dffers from the present basc feasble soluton n eactly one basc varable. (Adacent verte or etreme pont whch has dfferent obectve functon value could be better or worse) Select the better basc feasble soluton (among m(n-m) solutons at most) (If no adacent basc feasble soluton s better than the present, optmal!) Echange one varable n the basc varable set and on n nonbasc varable set n a way that the obectve functon value s mproved. Once the varable to echange s selected (p-th varable n the basc and q-th n the nonbasc), pvot by pq-th element! Row echelon form I B + B -1 N N = B -1 b (where B = B -1 b B -1 N N and N = ) B -1 N B B -1 b a11 a12 a 1 1n 1 b1 y1, m1 y1 n 1 y1 a 1 21 a22 a 2n 2 b y 2 2, m 1 y 2n 2 y 2 a 1 y m1 am2 amn n bm mm, 1 ymnn ym Obectve functon J = c B B + c N N = c B B N Assume q from nonbasc varables s changed from to 1: Impact of q on obectve B = B -1 b B -1 N N = B -1 b d q q = B -1 b d q, ( q = 1) where d q s q-th column of B -1 N. J' = c B (B -1 b d q ) + c N N = c B (B -1 b d q ) + c q III-5
Department of Chemcal and Bologcal Engneerng Improvement on obectve by ncludng q n the basc (Inner product rule) ΔJ =J' J = [c B (B -1 b - d q ) + c q ] c B B -1 b = c q c B d q c c cd q q B q relatve cost (or proft) of the nonbasc varable q should be negatve for mprovement n mnmzaton problem should be postve for mprovement n mamzaton problem For mnmzaton, consder a new basc varable whch has the most mprovement on obectve functon. Condton for optmalty (Mnmzaton) c q ( qm1,, n) q (A local mnmum s the global mnmum snce t s LP.) Determnng varable to enter bass (mn.): Choose q so that c mn c q Selecton of the leavng element from the basc varable set o acheve the greatest mprovement on obectve, the q should ncrease the obectve functon as much as possble, but B cannot be nonpostve n more than one element. Increase q untl any one of elements n B becomes zero ( p = ). y B N y y m 1 q q q ( 1, ) q - If y q s not postve, becomes more postve as q ncreases. (cannot be nonbasc) - Choose p so that the mamum ncrease n q wthout makng more than one basc varable nonpostve. Determnng varable to leave bass (mn.): Choose p so that (y / y q ) s smallest among y q > for = 1,, m. (Mnmum rato rule) Geometrcal Interpretaton of the basc feasble soluton A = b a a a Choose a a 1 2 n 1 2 m b 1 1 2 b 2 b n m a as the bass so that 1 a a a b B 1 2 ( B > ) m Assume m=2 and n=4. A feasble soluton defnes a representaton of b as a postve combnaton of a 's. In ths eample, {a 1, a 3 }, {a 2, a 4 } and {a 3, a 4 } cannot result n basc feasble soluton. a 3 a 1 a 4 a 2 b III-6
Department of Chemcal and Bologcal Engneerng II. SIMPLEX MEHOD Standard LP problem can be wrtten as: mn Z c c B B N N B N b, B Subect to B N and N If B s the bass for a basc feasble soluton ( B = B -1 b>, N = ), B = B -1 b B -1 N N Z = c B B + c N N = c B (B -1 b B -1 N N ) + c N N = c B B -1 b + (c N c B B -1 N ) N = c B B -1 b + r N 1. Matr Form of smple method (ableau form) Obectve functon value 1 1 A b B N b I B N B b 1 1 (by ERO) c cb cn cn cbb N cbb b relatve (reduced) cost coeff. <Procedure> 1. Select m-ntal basc varables. 2. Convert the tableau nto the row echelon form by ERO's. 3. Choose a column whch has the largest negatve value of the reduced cost coeffcent (Select q). 4. Choose a pvot element n the q-th column by the mnmum rato rule. 5. Include q as a basc varable and p becomes a nonbasc varable. 6. Repeat the step 2-6 untl all the reduced costs are postve (Mnmzaton problem) Mamzaton problem 1. Replace Z wth Z. 2. Or, choose largest postve value of the relatve proft coeffcent n step 3 and repeat the procedure untl all the reduced costs are negatve. Alternatve Optma: If the reduced costs for nonbasc varables are not postve at the optmal soluton, ths ndcates the alternatve solutons whch wll not change the optmum value. Unbounded Optmum: If the mnmum rato rule cannot be determned (all negatve), ths ndcate the unbounded optmum. Degeneracy: If a basc feasble soluton contans zeros for one or more basc varables, ths ndcates the degenerate basc feasble soluton. It occurs f two or more rows te for the mnmum rato, or one or more elements of the rght-hand sde n the constrants (b s) n orgnal LP problem are zero. Perturb the pvot entry n the requrement vector (b) by a small amount ( ), then proceed as normal. Cyclng: If the degeneracy occurs (no mprovement wth new basc varable), no mprovement n the obectve functon s acheved for a whle. (Reducng calculaton effcency) If the teraton goes on ndefntely wthout mprovng the obectve functon value, t s called classcal cyclng or cyclng. It s not lkely n practce, but computer cyclng may occur! III-7
Department of Chemcal and Bologcal Engneerng Eample: ma Z 312 33 212 3 2 subect to 122 33 5 2122 3 6 ( 1,2,3) Introducng three nonnegatve slack varables, the ntal tableau a 1 a 2 a 3 a 4 a 5 a 6 b a /a 2 1 1 1 2 2/2 1 2 3 1 5 5/1 2 2 1 1 6 6/2 r -3-1 -3 Select pvot element as 1 st column and 1 st row and perform ERO s. a 1 a 2 a 3 a 4 a 5 a 6 b a /a 1 1/2 1/2 1/2 1 2 3/2 5/2-1/2 1 4 8/5 1-1 1 4 bg r 1/2-3/2 3/2 3 Select pvot element as 3 rd column and 2 nd row and perform ERO s. a 1 a 2 a 3 a 4 a 5 a 6 b a /a 1 1/5 2/5-1/5 1/5 3/5 1-1/5 1/5 8/5 1-1 1 4 r 11/1 6/5 3/1 27/5 All the relatve costs are postve: Optmal soluton s obtaned. 1 =1/5, 3 =8/5, 2 =, 4 =, 5 =, 6 =4, and Z= 27/5. III. WO-PHASE SIMPLEX MEHOD - o start the smple method, an ntal feasble soluton n canoncal form s needed. - Generally, the constrants are not gven n the canoncal form. Need to solve the lnear equaton to fnd a basc feasble soluton 1. Artfcal Varable If basc varables are not readly avalable from each constrant, add new artfcal varable for each constrants to form a basc soluton. <Eample> mn Z = 3 1 + 2 + 3 s. t. 1 2 2 + 3 11 4 1 + 2 2 + 2 3 3 2 1 3 = 1, 1, 2, 3 o standard form (wth slack/surplus varable) 1 2 2 + 3 + 4 = 11 4 1 + 2 2 + 2 3 5 = 3 2 1 + 3 = 1, 1 ~ 5 4 can be a basc varable. Need two more basc varables! Add 2 artfcal varables ( 6, 7 ) III-8
Department of Chemcal and Bologcal Engneerng 1 2 2 + 3 + 4 = 11 4 1 + 2 2 + 2 3 5 + 6 = 3 2 1 + 3 + 7 = 1, 1 ~ 7 Basc feasble soluton : 4 =11, 6 = 3, 7 = 1, 1 ~ 3, 5 = (It s not feasble to orgnal problem snce 6 and 7 are not zero.) Need to make 6 and 7 zero ASAP for the feasble soluton to orgnal problem!! 2. wo-phase Smple Method <Phase I> o remove all the artfcal varables, set an artfcal obectve functon. mn w = 6 + 7 s. t. 1 2 2 + 3 + 4 = 11 4 1 + 2 2 + 2 3 5 + 6 = 3 2 1 + 3 + 7 = 1, 1 ~ 7 If w becomes zero by smple method ( 6 = and 7 = ), then the soluton s a basc feasble soluton of the orgnal problem. If not, the orgnal problem s nfeasble. Start the ntal tableau calculatng reduced costs by makng the reduced costs for artfcal varables zero. <Phase II> Fnd the optmum of the orgnal problem wth the soluton from the Phase I wthout any artfcal varables. 3. Varables Wth Upper Bounds mn c Subect to A = b and h <Method 1> Change the problem to standard form mn c Subect to A = b, y =hand, y - he sze of coeffcent matr s changed from mn to (m+n)2n. - hs transformaton requres more memory and computaton tme. <Method 2> Defnton: An etended basc feasble soluton correspondng to the problem for the varables wth upper bounds s a feasble soluton for whch (n-m) varables are equal to ether ther lower (zero) or ther upper bound; and the remanng m basc varables correspond to lnearly ndependent column of A. - If there s no etended basc feasble soluton wth m basc varables, then the constrants are too tght and there s no soluton. - Assume that every etended basc feasble soluton s nondegenerate. - A varable at ts lower bound can only be ncreased, and an ncrease wll be benefcal f the correspondng relatve cost coeffcent s negatve. - A varable at ts upper bound can only be decreased, and the decrease wll be benefcal f the correspondng relatve cost coeffcent s postve. - heorem (Upper bound optmalty): An etended basc feasble soluton s optmal for the upper bound problem f for the nonbasc varables r f r f h III-9
Department of Chemcal and Bologcal Engneerng h - Defne new varables, (prevously was at lower bound) and (prevously was at upper bound). - Etended tableau 1 y1, m1 y1 n y1 1 y2, m1 y2n y2 1 ymm, 1 ymn ym rm 1 rn Z e e e e e 1 2 m m1 n - he e s ether + or dependng on the current soluton, - Strategy: or. e 1. Determne a nonbasc varable for whch r <. If no such varable ests, stop; the current soluton s optmal. (-th column s selected for basc) 2. Based on the three numbers, a) h, b) mn y / y, c) mn( y h) / y, y, y, follow the update strategy accordng to whch number s smallest. Case a) he varable goes to ts opposte bound: Subtract h tmes column from last column and change sgns of column. he bass does not change and no pvot s requred. Case b) he -th basc varable returns to ts old bound: Pvot on the -th element. Case c) he -th basc varable goes to ts opposte bound: Subtract h from y and change sgns of y and e. Pvot on the -th element. 3. Return to step 1. - Eample: mn Z 212 3324 15 Subec to 134 25 5 2 2324 5 9 7, 1, 1, 5, 3 1 2 3 4 5 Orgnal tableau s a 1 a 2 a 3 a 4 a 5 b a) b) c) 1 1-1 2 5 h =5 - (5-7)/(-1)=2 1 2 2 1 9 9/2 - r -1-2 5-19 e + + + + + =4 =2 =1 r 3 =3 12-11= 1, r 4 = 2+12 21= 2, r 5 =1 22 11= 5 he c) s the mnmum. Case c) should be appled. Before pvotng, a 1 a 2 a 3 a 4 a 5 b a) b) c) -1 1-1 2-2 1 2 2 1 9 r -1-2 5-19 e - + + + + b = 75= 2 After pvotng, III-1
Department of Chemcal and Bologcal Engneerng a 1 a 2 a 3 a 4 a 5 b a) b) c) 1-1 1-2 2 h =1 - (2-5)/(-1)=3-2 1 4 5 5 5/4 - r 2-3 1-15 e - + + + + =3 =2 =4 he a) has the mnmum. Case a) should be appled. (b=b h a 3, a 3 =a 3, no pvot.) a 1 a 2 a 3 a 4 a 5 b a) b) c) 1 1 1-2 3-2 1-4 5 1 r 2 3 1-12 e - + - + + =3 here s no r < for nonbasc varables. he optmum s obtaned! h 7 ( e ) y, 4 y4 3, 1 1 1 2 2 1 3 h3 1 ( e3 ), and 5 ( e5 ) IV. REVISED SIMPLEX MEHOD - For large sze problem (>5 constrants), there are problems n memory and computaton tme, - For effcent computer mplementaton, some modfcatons of the smple method are needed. - Other columns besdes pvot column are not eplctly used. (f n>>m, waste of computaton) - he pvotng wll be appled to B -1 and y q, not the whole tableau. <Procedure> (Mnmzaton case) Gven a current bass B -1 and the current soluton B = y = B -1 b, 1. Calculate current reduced cost coeffcents: c c N ( N N If c, the optmal soluton s obtaned! (SOP) N 1 cb B ) 2. Determne whch a q s to enter the bass by selectng the most negatve reduced cost coeff. and calculate the column to be pvoted y q = B -1 a q. 3. If no y q >, stop! (he problem s unbounded.) Otherwse, calculate B /y q for y q > to determne whch vector s to leave the bass by mnmum rato rule. 4. Update B -1 and B = B -1 b, then return to step 1. Eample: ma Z 3 3 1 2 3 212 3 2 subect to 122 33 5 2122 3 6 ( 1,2,3) c he orgnal tableau: a 1 a 2 a 3 a 4 a 5 a 6 b 2 1 1 1 2 1 2 3 1 5 2 2 1 1 6-3 -1-3 III-11
Department of Chemcal and Bologcal Engneerng Start wth ntal basc feasble soluton and correspondng B -1. var B -1 B y q B /y q 4 1 2 2 2/2 5 1 5 1 5/1 6 1 6 2 6/2 c c c B [ 3 1 3] [ ] I (nonbasc nde: I N =[1 2 3]) 1 N N B Update the tableau by pvotng wth new bass y 1 =B -1 a 1 =[2 1 2]. var B -1 B y q B /y q 1 1/2 1 2 1 5-1/2 1 4 1 6-1 1 4 2 1 N [ 1 3 ] [ 3 ] [1/ 2 3 ] Update the tableau wth new bass y 3 =B -1 a 3 =[1/2 5/2 ]. var B -1 B y q B /y q 1 1/2 1 1/2 2/1 3-1/2 1 4 5/2 8/5 6-1 1 4 bg After pvotng, var B -1 B y q B /y q 1 3/5-1/5 1/5 1/2 3-1/5 2/5 8/5 5/2 1 6-1 1 4 1 N [ 1 ] [ 3 3 ] [1/ 5 3/ 5 ] No nonpostve relatve cost! he optmal soluton s obtaned. 1 =1/5, 2 =, 3 =8/5, 4 =, 5 =, 6 =4 and Z= 31/5 38/5= 27/5. c B (nonbasc nde: I N =[2 3 4]) c B (nonbasc nde: I N =[2 4 5]) <Modfcatons> - Not to choose most negatve reduced cost n step 2, but to choose frst negatve cost more maor teratons but less computng tme n total. - If we start wth I as B -1 (bass), then the tableau of k-th teraton can be epressed as E where E s an ERO's matr nvolves all the pvotng operatons and s the ntal tableau. B -1 = E = E k E k-1 E 2 E 1 I n Step 1, λ = c B B -1 = c B E k E k-1 E 2 E 1 n Step 2, y q = B -1 a q = E k E k-1 E 2 E 1 a q n Step 4, B = B -1 b = E k E k-1 E 2 E 1 b he B -1 can be ncorporated n the tableau and updated by the recursve form : (B -1 ) k = E k (B -1 ) k-1 - It s not sensble to store whole E. It requres only to know y q and p to reconstruct E. 1 y1 q / ypq 1 y2q / ypq E (mm matr) 1/ y pq ymq / ypq 1 - If the teraton goes on, there could be some accumulaton of truncaton error. Check B B - b perodcally and f t s not near zero, then calculate the nverson of III-12
Department of Chemcal and Bologcal Engneerng B and set B = B -1 b. (renverson) - Revsed smple method can be reformulated usng B nstead of B -1. Instead of usng B -1, solve lnear equatons three tmes. By = b, λ B = c B, By q = a q for y, λ, y q Ba1 a2 am LU where U u u u (ntally obtaned) 1 2 m (upper trangular) Let B s the new bass where a column a k s replaced wth a q. B a 1 a a k1 k1 a a m q LU 1 1 1 1 1 1 H L B L a1 L ak 1 L a k1 L am L aq 1 u 1 uk 1 uk 1 um L aq (Non-upper trangular! It has some subdagonals after (k-1)th column.) H can be constructed wthout addtonal computaton, snce u s are known and L -1 a q s a by-product n the computaton of y q. cf) Solvng lnear equaton by LU decomposton ) A=b=LU L(U)=Ly=b: Snce L s a lower trangular matr, y 1 =b 1 /l 11, y 2 =(b 2 l 21 y 1 )/l 22, y 3 =(b 3 l 31 y 1 l 32 y 2 )/l 33, and so on. ) U=y: Snce U s a upper trangular matr, n =y n /u nn = y n, n-1 =(y n-1 u (n-1)n n )/ u (n-1) (n-1), n-2 =(y n-2 u (n-2) (n-1) n-1 u (n-2)n n )/ u (n-2) (n-2), and so on. Reducton of H to upper dagonal matr: M 1 1 1 for k, k1,, m-1 m 1 1 where m = obtaned from the Gaussan elmnaton to convert non-upper trangular matr to trangular matr form. (m = -h (+1) / h ) U M M M H (upper trangular matr wth unt dagonals) m1 m2 k 1 1 1 k k1 m1 BLH LM M M U L LM M M 1 1 1 k k1 m1 1 ( M s smply M wth the sgn of the off-dagonal term reversed.) cf) For the sake of storage convenence, U wll be decomposed as a upper dagonal matr wth unt dagonals n LU decomposton. III-13
- Department of Chemcal and Bologcal Engneerng V. DANZIG-WOLFE DECOMPOSIION MEHOD mn Z c Subect to A = b and - If A has the specal block-angular structure, L1 L2 L A1 A A2 A It becomes mn N 1 c 1 N N Subect to L =b, N A =b and ( 1,2,, N) Dvsons nto N subproblems wth a lnkng constrant of dmenson m. Let the constrant set for the -th subproblem be S = { : A = b, } and assume that each of the polytopes S ( = 1,, N) s ndeed bounded and hence a polyhedron (by placng artfcal large upper bounds on each ). And let the etreme ponts of S be { 1, 2,, K }. hen, K 1 where K 1 and for =1, K (lnear combnaton) 1 and the orgnal problem becomes: N N K N K N K Z c c c p p c 1 1 1 1 1 1 1 N N K N K N K ( ) L L L q = b ( q L ) 1 1 1 1 1 1 1 <Master problem> mn p Subect to Q =g and where [ ], g = [b, 1, 1,,1], 11 1K1 21 2K1 NKN 1 11 1 1K 2 21 2 2K N NK p [ c c c c c ] 1 2 N q11 q1 q 1 21 q2 q 2 1 1 Q 1 1 1 K K NK N III-14
Department of Chemcal and Bologcal Engneerng λ Mnmum relatve cost: * * r mn r 1 mn ( p B ) [1,, N] [1,, K ] pb N hs procedure wll calculate even for the relatve costs of basc varables that wll be zero. But t does not affect the results of the procedure. * q r mn ( 1 ) mn ( c m m 1) [1,, K ] c L e [1,, K ] where s the vector made up of frst m elements of. (he s are n S and they satsfy A =b.) <he -th subproblem> mn( c L ) Subect to A =b and Solve the -th subproblem to get problem. and calculate r * n the procedure of solvng master * Eample: mn Z 122 4334 Subect to 12 2 3 4 L 2 L 1 2 34 3 2 12 4 and ( 1,2,3,4) A 1 12 2 3 A 4 2 2 3 2 5 3 4 Slack varables wll be added, but the number of decson varables n the master problem wll be same as those of the orgnal problem. <Master problem: 6 varables and 4 constrants> mn( p p p p ) 11 11 12 12 21 21 22 22 Subect to L L L L 1 11 1 11 12 1 12 21 2 21 22 2 22 s2 11 12 1, 21 22 1 and,,,, s, s 11 12 21 22 1 2 s 4 3 where p 11 =[ 1 2] 11, p 12 =[ 1 2] 12, p 21 =[ 4 3] 21, p 22 =[ 4 3] 22. A startng basc feasble soluton can be [ s 1, s 2, 11, 21 ] [4 311] (hus, t requres only 11 and 21 for revsed smple method.) and the convenent etreme ponts of the subsystems are 11 = and 21 =. o select the nonbasc varable to be basc varable, solve the two subproblems. mn( c1 L1) 1 mn( c2 L2) 2 s.t 212 4 s.t 34 2 and 12 2 334 5,, 1 2 3 4 III-15
Department of Chemcal and Bologcal Engneerng where c 1 [ 1 2], c 2 [ 4 3], and [ ]. hen, fnd the mnmum relatve prce among two subproblem solutons ( 1 =[ 2], 2 =[1 1] and r 1 =([ 1 2] ) 1 = 4, r 2 =([ 4 3] ) 2 = 7) n order to get the nonbasc varable to enter as basc ( 22 for 22 =[1 1] ) and fnd the basc varable to be nonbasc usng mnmum rato rule ( 21 ). Iterate the procedure untl optmum s obtaned. VI. DUALIY Prmal LP mn c Dual LP ma λ b s. t. A b s. t. λ A c λ (Symmetrc form) - Consder a standard LP problem mn c mn c s. t. A b s. t. A b A b Its dual s: ma( ubvb) s. t. uavac u, v (Asymmetrc form) uv mn c A b s. t. A b ma b s. t. Ac ( s free) Lemma 1: Weak Dualty heorem If and λ are feasble for the asymmetrc prmal and ts dual problems, respectvely, then c λ b. pf) λ A c, and A=b c λ A = λ b. Remark: A feasble vector to ether problem yelds a bound on the obectve functon value of the other problem. Corollary: If and λ are feasble for the asymmetrc prmal and ts dual problems, respectvely, and f c =λ b, then and λ are optmal for ther respectve problems. heorem: Dualty heorem of LP If ether of the asymmetrc prmal and ts dual problems has a fnte optmal soluton, so does the other, and the correspondng values of the obectve functons are equal. If ether problem has an unbounded obectve, the other s not feasble. - Relatons between the prmal and dual problems Let A=[B N]. If a basc feasble soluton B =B -1 b s optmal, the relatve cost vector r must be nonnegatve n each component. III-16
Department of Chemcal and Bologcal Engneerng Let r c c B N c c B N 1 1 N N B N B 1 cb B (smple multpler). hen, at the optmal soluton A B N c c B N c c c 1 B B B N bc B bc c 1 B B B heorem: Alternatve Dualty heorem of LP Let the LP of the asymmetrc prmal problem have an optmal basc feasble soluton 1 correspondng to the bass B. hen the vector cb B s an optmal soluton to ts dual problem and the optmal values of both problems are equal. - Geometrc nterpretaton mn Z 18112 2 23 64 s. t. 312 23 4 2 13 2 4 2,,, 1 2 3 4 In prmal space, fnd a postve lnear combnaton of a s to yeld resource vector b. (unque n ths case) a 3 a 2 a 4 a 1 b ma Z 212 2 s. t. 312 18 132 12 2 1 2 6 1 2 In dual space, the dual feasble regon s determned by the orthogonal lnes to each a s of whch locaton s determned by the elements of the resource vector c. a 3 a 2 a 4 a 1 b - Smple Multplers cb B 1 he λ s a synthetc prce as a lnear combnaton of the orgnal costs. hs vector s not a soluton to dual problem unless B s an optmal bass for prmal. Nevertheless, t has economc nterpretaton (related to relatve cost, shadow prce). If the prmal problem s to produce m-products b at the mnmum cost of c by n-facltes wth the producton rate of each product A, then the dual problem s to mamze the product purchase (λ b) not by manufacturng whle the purchase prce (λ a ) at the same producton rate of each faclty should be less than the producton cost (λ a c or λ A c ). - Complementary slackness 1. Asymmetrc case (standard LP form, A = b): From the dualty theorem, λ b = c (λ A c ) = > mples λ a = c (s = ) λ a < c mples = (s > ). III-17
Department of Chemcal and Bologcal Engneerng 2. Symmetrc case (nonstandard LP form, A b): From the dualty theorem, λ b = c (λ A c ) > mples λ a c λ a = c (s λ = ) λ a < c mples. = (s = ) From the dualty theorem, λ b = c λ (A b) λ > mples a b a = b (s = ) a > b mples λ λ =. (where a s the -th row of A) - Physcal meanng of λ n terms of senstvty Let = [ B ] be the optmal basc feasble soluton and B be the correspondng optmal bass. We know that B = B -1 b and λ = c B B -1. Assume the requrement vector b s changed to b +Δb. he optmal soluton s then [ B +Δ B ] where Δ B = B -1 Δb. he correspondng ncrement n the cost obectve wll be ΔZ = c B Δ B = c B B -1 Δb = λ Δb or λ =Δz/Δb Senstvty of the optmal cost wth respect to b. (Margnal prce) VII. SENSIIVIY (POS-OPIMALIY) ANALYSIS - By changng nput coeffcents (resource or constrants coeffcents), the optmal value can be mproved consderably hen, t should be consdered to change the stuaton. - Decde the mportance of the data coeffcents whch enables to re-estmate the mportant data coeffcents Improve the accuracy, relablty of the model. Rangng of the Coeffcents - Obectve functon coeffcent (c ): wthn some ranges of each coeffcent, the optmal soluton wll not change even though the optmal value wll change by * Δ (the slope of the obectve functon wll change). 1 rn cn c BB N where c B c B ( for ) For the optmal soluton to be same, the new relatve costs for nonbasc varable should be nonnegatve despte the change n cost coeffcent c. 1 r N rn ( B N), (Mnmzaton) - Resource coeffcent (b ): wthn some ranges of each resource coeffcent, the optmal m wll not change even though the optmal soluton and value wll change (λ * Δ ). * 1 B B b where b b ( for ) For the optmal m to be same, the new optmal soluton should be postve despte the change n cost coeffcent b. 1 ( B ), B B 2 ob. fn 2 ob. fn (no change n ob. fn value) 1 1 ( 2 wll not est f t goes further) III-18
Department of Chemcal and Bologcal Engneerng Smultaneous Varatons n Parameters - 1% rule (For obectve functon coeffcents) Let c be the actual decrease (ncrease) n the obectve functon coeffcent of varable and c be the mamum decrease (ncrease) allowed by the senstvty analyss. c If 1 satsfes, the optmal soluton wll not change. he change n c obectve functon value wll be Z c. Remark: he falure of the 1% rule for obectve functon coeffcents does not mply that the optmal soluton wll change. - 1% rule (For resource coeffcent) Let b be the actual decrease (ncrease) n the resource coeffcent of the -th constrant and b be the mamum decrease (ncrease) allowed by the senstvty analyss. b If 1 satsfes, the optmal product m and the shadow prces wll not b * change. he change n obectve functon value wll be Z b. Addng more varables Suppose n+1 s added wth constrant coeffcent vector a n+1 and the cost c n+1. r [ 1] N c N N an rn 1 cn 1 an 1 where c N [ cn; cn 1]. If the r n+1 s nonnegatve, the n+1 should reman as a nonbasc varable that s zero. hus, r c a n1 n1 n1 hen, the new varable can be a basc varable that can have nonzero value and addng a new varable s meanngful. Eample: A factory can produce four products denoted by P 1, P 2, P 3 and P 4. Each product must be produced n each of two workshops. he processng tme (n hours per unt produced) are gven n the followng table. 4 hours of labor are P 1 P 2 P 3 P 4 avalable n each workshop. he proft margns are 4, 6, Workshop 1 3 4 8 6 1 and 9 dollars per unt of P 1, P 2, P 3 and P 4 produced, respectvely. Everythng produced can be sold. hus, the mamzng proft, the followng lnear program can be used. Workshop 2 6 2 5 8 Ma 4 1 +6 2 +1 3 +9 4 Subect to 3 1 +4 2 +8 3 +6 4 4 6 1 +2 2 +5 3 +8 4 4 1, 2, 3, 4 Introducng slack varables s 1 and s 2, and applyng the smple method, we get the fnal tableu: r 4.5.5 1 2 5.5 1.5 1 2 6 (a) How many unts of P 1, P 2, P 3 and P 4 should be produced n order to mamze profts? (b) Assume that 2 unts of P 3 have been produced by mstake. What s the resultng decrease n proft? (c) In what range can the proft margn per unt of P 1 vary wthout changng the optmal * 1 2 3 4 s 1 s 2 b.75 1 2 1.5.25 1 III-19
Department of Chemcal and Bologcal Engneerng bass? (d) In what range can the proft margn per unt of P 2 vary wthout changng the optmal bass? (e) What s the margnal value of ncreasng the producton capacty of Workshop 1? (f) In what range can the capacty of Workshop 1 vary wthout changng the optmal bass? (g) Management s consderng the producton of a new product P 5 that would requre 2 hours n Workshop 1 and 1 hours n Workshop 2. What s the mnmum proft margn needed on ths new product to make t worth producng? Answers: (a) From the fnal tableau, we read that 2 =1 s basc and 1 = 3 = 4 = are nonbasc. So 1 unts of P 2 should be produced and none of P 1, P 3 and P 4. he resutng proft s $6 and that s the mamum possble, gven the constrants. (b) he reduced cost for 3 s 2 (found n Row 3 of the fnal tableau). hus, the effect on proft of producng 3 unts of P 3 s 2 3. If 2 unts of P 3 have been produced by mstake, then the proft wll be 22=$4 lower than the mamum stated n (a). (makng the coeffcent for 3 one and replace b 2 =2 and perform ERO for r 3 to be zero) (c) Let 4+Δ be the proft margn on P 1. he reduced cost remans nonnegatve n the fnal tableau f.5 Δ snce 2 s nonbasc. hat s Δ.5. herefore, as long as the proft margn on P 1 s less than 4.5, the optmal bass remans unchanged. (d) Let 6+Δ be the proft margn on P 2. Snce 2 s basc, we need to restore a correct bass. hs s done by addng Δ tmes Row 1 to Row 3. hs effects the reduced costs of the nonbasc varables, namely 1, 3, 4 and s 1. All these reduced costs must be nonnegatve. hs mples: (sgn change n relatve cost for mamzaton) 1 r r ( B N), (Snce 2 s the frst basc varable) N N 1.5+.75Δ 2+2Δ +1.5Δ 1.5+.25Δ Combnng all these nequaltes, we get Δ. So, as long as the proft margn on P 2 s 6 or greater, the optmal bass remans unchanged. (e) he margnal value of ncreasng capacty n Workshop 1 s 1 * 1 1.5. 1 4 1 1 cb B [6 ] [6 ] [1.5 ] 2 1 4 2 4 (f) Let 4+Δ be the capacty of Workshop 1. he resultng RHS n the fnal tableau wll be: 1+.25Δ n Row 1, and 2.5Δ n Row 2. he optmal bass remans unchanged as long as these two quanttes are nonnegatve. Namely, 4 Δ 4. So, the optmal bass remans unchanged as long as the capacty of Workshop 1 s n the range to 8. ( B 1 B 1(1/ 4) 1 and B2 B2 ( 1/ 2) 1 ) (g) he effect on the optmum proft of producng 5 unts of P 5 would be * * * 5 1(2) 2(1) 1.5(2) (1) 3 c5 a. If the proft margn on P 5 s suffcent to offset ths, then P 5 should be produced. hat s, we should produce P 5 f ts proft margn s at least $3. III-2
Department of Chemcal and Bologcal Engneerng VIII. OHERS Dual Smple Method If a certan lnear programmng s solved, then a new problem s constructed by changng the resource vector, b. In ths case, the soluton may not be feasble to the new problem, but the soluton s a basc feasble soluton for the dual problem (satsfyng λ a c ). he bass B satsfes λ = c B B -1. However, B = B -1 b, called a dual feasble soluton, may not satsfy B and some B s are negatve. From the complementary slackness for asymmetrc case, > mples λ a = c λ a = c for = 1,, m z =λ a < c for = m+1,, n he new margnal prce vector s obtaned by the echange of one varable (-th). hen λ a should be less than c (c ε, ε>) and one of λ a among = m+1,, n should be c. Let the -th row of B -1 be u. and y be u a. hen, λ a = c for = 1,, m ( ) λ a = c ε λ a = z εy for = m+1,, n herefore, choose ε so that the only one of (z εy ) s becomes c. Snce z <c and ε>, the y should be negatve. <Procedure> 1. Gven a dual feasble soluton B, f B, then t s the optmal! If B s not nonnegatve, select an nde such that the -th component of B, B <. 2. If y = (B -1 N) for = 1, 2,, n, then the dual has no mamum. If y < for some, then let zk c k z c mn : y y k y 3. Form a new bass B by replacng a by a k. Usng ths bass determne the correspondng basc dual feasble soluton B and return to step 1. It does not requre an ntal basc feasble soluton for. Prmal-Dual Algorthm mn c ma b s. t. A b s. t. Ac ( s free) Gven a feasble soluton λ to the dual problem, let P={ =1,2,,m}. a c, P (basc) a c, P (nonbasc) Assocated restrcted prmal and dual problems: mn1y ma ub s. t. A y b s. t. u [ A I] [ 1], y III-21
Department of Chemcal and Bologcal Engneerng heorem: (Prmal-Dual optmalty theorem) Suppose the λ s feasble for the dual and that y= and B s the optmal for the assocated restrcted prmal. hen =[ B ; ] and λ are optmal for the orgnal prmal and dual problem, respectvely. <Procedure> 1. Gven a feasble soluton λ to the orgnal dual problem, set up the assocated restrcted prmal problem. 2. Optmze the assocated restrcted prmal. If the mnmal value of ths problem s zero, the correspondng soluton s optmal for the orgnal prmal problem by the prmal-dual optmalty theorem. 3. If the mnmal value of the assocated restrcted prmal s strctly postve, obtan the soluton u of the assocated restrcted dual from the fnal smple of the assocated restrcted prmal. If there s no for whch u a >, conclude the prmal has no feasble solutons. Else, defne the new dual feasble vector λ= λ +ε u where c a mn : ua ck ak ua k ua hen go back to Step 1 usng ths λ. Reducton of Lnear Inequaltes 1. Redundant equatons: Correspondng to the system of lnear constrants A=b,, the system s sad to have redundant equatons f there s a nonzero m-vector w satsfyng w A= and w b =. rank(a)<m: mposes unnecessary computaton It can be detected and elmnated n Phase I (f the tableau has zero rows). 2. Null varables: A varable n the system of lnear constrants A=b, s sad to be a null varable f = n every soluton. Elmnate the null varables and -th column of A from the system. Null Varable heorem: If the feasble regon S s not empty, the varable s a null varable f and only f there s a nonzero m-vector w such that w A and w b = and the -th component of w A s strctly postve. cf) If s a null varable the followng LP has the zero optmal value. mn( e) s. t. A b ma b s. t. A e hus, wth w = λ, w b =, w A and so on. 3. Nonetremal varables: A varable n the system of lnear constrants A=b, s nonetremal f the nequalty s redundant. reat them as free varables and elmnate them by usng equatons where they are epressed n terms of other varables If the nequalty, can be composed of a lnear combnaton of the nequaltes, A b, then s a nonetremal varable. III-22
Department of Chemcal and Bologcal Engneerng Nonetremal Varable heorem: If the feasble regon S s not empty, the varable s a nonetremal varable f and only f there s m-vector w and n-vector d such that w A=d and w b where d = 1 and d ( ). cf) Let wb ( ) and. mn ma b s. t. A b s. t. a ( ) ( ) a 1 Snce mn( ) w A=d and w b., b and. hus, wth w = λ, Karmarka s Algorthm (1984) for large-scale problem (Interor pont method) - Search n the strct nteror of the conve feasble regon. - Karmarkar's algorthm s usually more effcent f the problem sze s very large mn f ( ) c s. t. A b <Procedure> 1. Start at the centrod ( k ) of the smple comprsng A k k =b (feasble solutons) and proect f ( ) c onto the ntersecton of the equalty constrants. k1 k k k k k k k 1 k k P ( c ) ( IA (A A ) A ) c k 1 2. Fnd so that only one element of becomes zero. And use slghtly less value of,.e..98, so that the pont les strctly nsde the feasble regon. k 1 3. Rescale the varables and transfer back to the rescaled centrod. (Prmal affne scalng) k1 1 k k 1 k Dk, k 1 k c Dkc, and A A D k k 1 where D k s a dagonal matr wth the elements of n as dagonals. 1 4. If P( c ) and (AA ) A) c (Lagrange multpler), then stop. Else, go to Step 1. (For actve constrants, λ = and for nactve, λ >) cf) Karmarkar actually used a nonlnear transformaton k 1 D where e s an n-dmensonal vector of 1 s. ed 1 k n k 1 k k III-23
Department of Chemcal and Bologcal Engneerng Eample: mn f ( ) [1 2 3] s. t. [1 1 1] 1 1. Centrod= =[1/3 1/3 1/3], f ( ) [ 1 2 3] 1 2/3 1/3 1/3 P I 1 (3) 1 1 1 1 1/3 2/3 1/3 1 1/3 1/3 2/3 1 ( f ) 1/3 1/3 1/3 1 For feasblty and to be strct nteror, P ( 1/3.98 ) 2. Choose scalng matr D k =dag([1.98 1.2]), then k k+1 = D -1 k k =[.168.333 16.67], A k A k+1 =A k D k =[1.98 1.2], c k c k+1 =D k c k =[ 1.98 2.6] 3. Go to Step 1 after checkng termnaton crtera. 1.98/ 3 1/ 3.2 / 3 <Prmal-dual method> From the constrans of prmal and dual problems and the complementary slackness, A b,, A s c, s, s ( 1,, n) Man dea of the method s to move through a sequence of strctly feasble prmal and dual solutons that come ncreasngly closer to satsfyng the complementary slackness condtons. A b,, A s c, s, s ( 1,, n) he dualty gap: c b s n hus, startng from some value of, decrease t satsfyng the constrants as fndng the soluton for changes n, y, and s. A,, A s, s, s s s ( 1,, n) he complementary slackness becomes S Xs e XSe where S and X are the dagonal matrces whos dagonals are elements of s and, respectvely. hen, 1 1 1 ( AS XA ) AS ( e XSe) s A 1 1 S ( exse) ( S X) s and ma for all. hen ma mn( prmal, dual ) s mas where s prmal mn and mn dual (rato test) s s Smple method III-24
Department of Chemcal and Bologcal Engneerng QUADRAIC PROGRAMMING Reklats G. V., A. Ravndran, and K. M. Ragsdell, "Engneerng Optmzaton : Method and Applcaton," John Wley and Sons, NY, 1983. (Secton 11.2) I. BASIC PRINCIPLES Problem Statement mn f() = c +.5 Q subect to g() = h() = A b = Lagrangan functon and Lagrangan multpler Above problem can be rewrtten as mn L(, μ, λ) = f() μ g() λ h() subect to μ (λ s unspecfed) where L(, μ, λ) s called the Lagrangan functon, μ and λ are called Lagrange multplers. Kuhn-ucker Condtons (KC) Assume f, g and h are dfferentable. he vectors (N1), μ(j1), λ(k1) become a canddate for the optmal soluton f L ) f g( ) h( ) ) L g( ) s g() (use surplus varable s) L ) h ( ) h() = v) μ (λ s unspecfed) v) μ g () = for =1, 2,, J (complementary slackness condton). For complementary slackness condton, f the -th nequalty constrant s bndng (actve) g () = and f -th nequalty constrant s nonbndng (nactve), μ =. Solvng ) to v) : Kuhn-ucker problem - K- necessary condton for optmalty Let f, g and h be dfferentable, * be a feasble soluton. Assume that g( * ) for actve constrants and h( * ) are lnearly ndependent. cf) Defne the effectve constrants by h E () = {h (): h (*) =, = 1,, m}, where h E () s a (m E 1) vector, m E beng the number of effectve constrants, wth m E < n. hen, the constrant qualfcaton becomes: constrant qualfcaton (CQ): rank[h E (*)/] = m E. It s the regularty condtons on the feasble regon and dffcult to verfy. However, t s generally acceptable n practce. he CQ states that, at *, the m E effectve constrants are "lnearly ndependent around *". hs guarantees that the III-25
Department of Chemcal and Bologcal Engneerng mplct functon theorem can be used to solve the effectve constrants for m E of the varables and the result substtuted back nto the obectve functon. hen, * and some (μ *, λ * ) satsfy the Kuhn-ucker condton. When the constrant qualfcaton s not met at the optmum, there may not est a soluton to Kuhn-ucker problem. - K- suffcent condton for optmalty Let f be conve, g be all concave, and h be lnear. hen ( *, μ *, λ * ) whch satsfy the Kuhn-ucker condton s an optmal soluton. he Kuhn-ucker condton provdes some new nformaton. hey endogenously treat the constrants whch are bndng (or non-bndng). From the complementary slackness condton, f the -th constrant s non-bndng, g ( * )>, then the correspondng Lagrange multpler must be zero, * =, = 1,, m. And f the -th Lagrange multpler s postve, * >, then the correspondng constrant must be bndng, g ( * ) =, = 1,, n. he Lagrange multplers are now restrcted to be non-negatve: *. hs reflects the fact that we now have nequalty constrants. Indeed, we have seen that the Lagrange multpler can be nterpreted as the margnal value of the constrants. Relang an nequalty constrant means ncreasng the feasble set, generatng a non-decreasng value of the ndrect obectve functon, and thus a non-negatve margnal value of the constrants. In ths contet, a postve and large Lagrange multpler means that the correspondng constrant s "very bndng" and dentfes sgnfcant resource scarcty. Alternatvely, a small Lagrange multpler dentfes lttle resource scarcty, as the correspondng constrant s "barely bndng." A Lagrange multpler reaches ts lower bound ( * = ) when the -th constrant s non-bndng and becomes rrelevant to the decson. When Q s postve defnte or postve semdefnte for quadratc programmng problem, t s suffcent to solve the Kuhn-ucker condtons to fnd an optmal soluton to the quadratc problem. Lagrangan functon wth equalty constrants only mn f() + λ ( A + b) Optmalty condton f Q s postve defnte f () A λ = and A + b = f () = c + Q * 1 Q A c Q A c * A b A b By Frobenus-Schur nverson formula 1 1 1 1 1 1 1 1 1 A B A A BD ( CA B) CA A BD ( CA B) 1 1 1 1 1 C D ( DCA B) CA ( DCA B) * 1 1 1 1 1 1 1 1 Q Q A ( AQ A ) AQ Q A ( AQ A ) c * 1 1 1 1 1 ( AQ A ) AQ ( AQ A ) b Let * 1 1 1 A ( AQ A ) AQ and * 1 1 * H Q Q A A. III-26
Department of Chemcal and Bologcal Engneerng * * * H A c * * 1 1 A ( AQ A ) b cf) When A and D are nvertble, -1-1 -1-1 -1-1 -1 (A + BCD) = A - A B(C + DA B) DA 1 1 1 1 1 1 A B ( ABD C) ( ABD C) BD 1 1 1 1 1 C D ( DCA B) CA ( DCA B) Recursve formula For a basc feasble soluton, A + b = and f ( ) = c + Q * * * * * * ( ) f ( ) * * 1 * 1 * H c A b H Q A A H A c ( AQA ) b ( A Q ( AQA ) A) A f ( ) 1 1 1 * * ( ( AQA ) AQ Q( AQA ) A) A f( ) A f( ) II. FLECHER'S QP ALGORIHM QP Algorthm (Fletcher, 1971) mn f() = c +.5 Q (n decson varables) subect to A b (m nequalty constrants) <Procedure> 1. A feasble soluton s gven, and q-constrants are tght. Let A q be the constrants coeffcent matr wth q-actve constrants. * 1 1 1 * 1 1 * Calculate A ( A Q A ) A Q and H Q Q A A and set k=. 2. Compute q q q q ( k1) * ( k) q q q q s H f ( ). If s (k+1) =, go to step 4. 3. If s (k+1), compute (k+1) = (k) + α *(k) s (k+1) and f ( k 1) ( ) where α *(k) = mn {1, α (k) p }, ( k) ( k) a b ( k1) p mn ; as ( 1,, ) ( k 1) q m as If α *(k) <1, then add p-th column of A (a p (k) ) to H * q and A * q, H * * * * q p p q q1 H HaaH q * aha p q p * * * * A ah q p q Aqap q1 * aha p q p 1 A and set k=k+1, q=q+1 and go to step2. If α *(k) =1, then set k=k+1 and go to step2. 4. Calculate ( k) ( k) A and r mn ; 1, 2,, q f ( ) ( k) * ( k) q. III-27
Department of Chemcal and Bologcal Engneerng If If (.e. all elements of ( k ) r ( k ) r ( k ) are nonnegatve), Stop (optmum)., delete r-th row of A * q (a r ) from H * q and A * q, and set q=q 1 and go to step2. r H H aa aqa * * r q 1 q r * A q1 * Aq r AQaa * q r r r r aqa III. COMPLEMENARY PIVO PROBLEMS (LEMKE, 1965) Problem Statement mn f() = c + Q subect to A b (n decson varables) (m nequalty constrants) Assume Q s symmetrc and s postve defnte or postve semdefnte. KC optmalty condton to the above conve quadratc program: c + (Q + Q ) μ λ A = μ = 2Q A λ + c s = A b, μ, λ, s μ + s λ = Let w s, 2Q A z, M and A hen, w = Mz + q and w z = wth w, z. Complementary problem c q b. - M s postve semdefnte snce Q s postve defnte or postve semdefnte. - If Q s set to zero, t becomes an LP. Defntons: 1. A nonnegatve soluton (w, z) to the system of the equaton w = Mz + q s called a feasble soluton to the complementary problem 2. A feasble soluton (w, z) to the complementary problem that also satsfes the complementarty condton s called a complementary soluton. - w z = w z = for all - he varables w and z for each s called a complementary par of varables. - If the element of the vector q are nonnegatve, then there ests an obvous complementary soluton gven by w = q and z =. (rval soluton) - If some elements of the vector q are negatve, then complementary soluton gven by w = q and z = would be nfeasble. Introduce a suffcently large artfcal varable z so that (q + z ) become nonnegatve. A basc feasble soluton s gven by w = q + z, z = for all = 1, 2,, n III-28
Department of Chemcal and Bologcal Engneerng z = mn (q ) (But w = Mz + q s not met.) called almost complementary soluton <Procedure> 1. o determne the ntal almost complementary soluton, the varable z s brought nto the bass, replacng the bass varable wth the most negatve value. (Let q s = mn q < ) hat s, z replaces w s from the bass by pvotng. w Mz z e = q w, z, z and w z = e (n1) = [1, 1,, 1] =[I M e q] (ntal tableau) 2. In order to mantan the complementarty, ether one of w s and z s should reman as a basc varable. So n the net tableau, the complement of the basc varable that ust left the bass n the last tableau should become a basc varable. (complementary rule) In order to mantan the nonnegatvty of the basc feasble soluton, use mnmum rato test to determne the varable to leave the bass. q k q mn ; 1, 2,, n and ms mks ms hat s, z s replaces w k from the bass by pvotng. 4. Snce w k left the bass, the varable z k s brought nto the bass by the complementary rule and the bass changes as before. If the mnmum rato s obtaned n row s, and z leaves the bass, the resultng basc soluton after performng the pvot operaton s the complementary soluton. If the mnmum rato test fals, snce all the coeffcents n the pvot column are nonpostve, ths mples no soluton to the complementary problem ests. In ths case, we say that the complementary problem has a ray soluton. (the gven lnear or quadratc program has no soluton) Eample: 2 2 mn f()= 6 1 +2 1 2 1 2 +2 2 subect to 1 2 2 and 1, 2 For f()= c + Q, c=[ 6 ], Q=[2 1; 1 2]. For A b, A=[ 1 1], b= 2, w=[μ 1 μ 2 s], z=[ 1 2 λ]. 4 2 1 6 M 2 4 1 and q 1 1 2 Bass w 1 w 2 w 3 z 1 z 2 z 3 z q w 1 1-4 2-1 -1-6 w 2 1 2-4 -1-1 w 3 1 1 1-1 2 An almost complementary soluton s obtaned by replacng w 1 by z. Bass w 1 w 2 w 3 z 1 z 2 z 3 z q z -1 4-2 1 1 6 w 2-1 1 6-6 6 w 3-1 1 5-1 1 8 From the complementarty, z 1 has to be a basc snce w 1 became nonbasc. he rato test: q /m s = 6/4, 6/6, 8/5. herefore choose w 2. III-29
Department of Chemcal and Bologcal Engneerng Bass w 1 w 2 w 3 z 1 z 2 z 3 z q z -1/3-2/3 2 1 1 2 z 1-1/6 1/6 1-1 1 w 3-1/6-5/6 1 4 1 3 From the complementarty, z 2 has to be a basc snce w 2 became nonbasc. he rato test: q /m s = 2/2, -1/1, 3/4. herefore choose w 3. Bass w 1 w 2 w 3 z 1 z 2 z 3 z q z -1/4-1/4-1/2 1/2 1 1/2 z 1-5/24-1/24 1/4 1 1/4 7/4 z 2-1/24-5/24 1/4 1 1/4 3/4 From the complementarty, z 3 has to be a basc snce w 3 became nonbasc. he rato test: q /m s = 1, 7, 3. herefore choose z. Bass w 1 w 2 w 3 z 1 z 2 z 3 z q z 3-1/2-1/2-1 1 2 1 z 1-1/12-1/12 1/2 1-1/2 3/2 z 2-1/12-1/12 1/2 1-1/2 1/2 Snce z left the bass, the complementary soluton s obtaned. z 1 = 1 =3/2, z 2 = 2 =1/2, z 3 =λ=1, w 1 = w 2 = w 3 = and f( * )= 11/2. III-3