LINEAR PROGRAMMING II Patrik Forssé Office: 2404 Phoe: 08/47 29 66 E-mail: patrik@tdb.uu.se
HE LINEAR PROGRAMMING PROBLEM (LP) A LP-problem ca be formulated as: mi c subject a + am + g + g p + + + c to + a + a + g + g m p b b = h = h m p mi c or i matri otatio A b, G = h ie.., the goal is to fid decisio variables,, fulfillig the m liear iequality costraits a + + a b ad the p liear equality costraits g + + g = h, i i i i i i ie.., a feasible solutio, so that the liear objective fuctio c + + c is miimized.
SOLVING AN LP GEOMERICALLY Cosider the problem : mi + 22 3 2 + 2 3, 2 0 2 We see that the miimum is - 2 ad is attaied at the poit =, 2 = Oly first quadrat cosidered because, 2 add the costrait + 22 3 ad the the costrait 2 + 2 3 the draw som level curves of z = 2. 3 0 CONCLUSION FROM HIS EXAMPLE: he feasible set is a polyhedro. he optimum is attaied at a verte (etreme poit, basic feasible solutio)..5 (,) c.5 3 0 = = 2 2 = 2 2
LAS IME Itroductio Differet forms of LP s ad how to covert betwee them.
GEOMERY OF LINEAR PROGRAMS
HYPERPLANES R i { b} DEFINIION:A hyperplae i is a = ( a 0), e.g., he solutio set of oe liear equatio a + + a = b with at least oe a 0. he set of vectors that make a costat ier product with the vector a = ( a,, a ), a is called the ormal vector. 0 a hyperplae 0 = 0 = b ( a a ) NOE : A equality costrait a + + a = b i a LP defies a hyperplae! 2 3 he hyperplae has dimesio, e.g., i R a lie, i R a plae,
HE SOLUION SE OF LINEAR EQUAIONS he solutio set of m liear equatios a + + a = b, or i matri otatio A = b, am+ + am = bm is a affie set, cf. LP i stadard form! his is the same as the itersectio of m a = ( a,, a ) (we assume all a 0). i i i i hyperplaes with ormal vectors
HALFSPACES R i { b} DEFINIION:A closed halfspace i is a ( a 0), e.g., he solutio set of oe liear iequality a + + a b with at least oe a 0. a = ( a,, a ) is the outward ormal. halfspace 0 a { a a } 0 0 { a a } 0 { a b} < is called a ope halfspace. a b is equivalet to a b. NOE : he boudary of a halfspace is a hyperplae NOE 2 : A iequality costrait a + + a b i a LP defies a halfspace!
POLYHEDRA DEFINIION : A polyhedro is the solutio set of a system of liear iequalities a + + a b a + + a b cf. LP i iequality form! m m m or i matri otatio A b, his is the same as the itersectio of m a = ( a,, a ) (we assume all a 0): i i i i halfspaces with ormal vectors a a 2 a 6 a 5 a 3 a 4 Note : his is a visualizatio of the feasible set for a LP - problem i iequality form
{ h} Note that a hyperplae g = ( a 0) describes the polyhedro g h g h ad from this it follows that affie sets ( i.e., the solutio set to G = h) also describes a polyhedro. We ow coclude that the solutio set of a system of liear equatios/iequalities: A b G = h describes polyhedra. Especially we have that: G = h (the feasible set of a stadard form LP) 0 is a polyhedro. CONCLUSION: HE FEASIBLE SE OF AN LP IS A POLYHEDRON.
VISUALIZING SANDARD FORM PROBLEM We wat to visualize the feasible set of a stadard form problem, i.e., the polyhedro A = b m where A R. 0 We assume that m ad that the costrait A = b forces to lie o a ( m) dimesioal set. If we "stad" o that ( m) dimesioal set ad igore the m dimesios orthogoal to it, the feasible set is oly costraied by the liear iequality costraits 0. I particular, if m = 2 the feasible set ca be draw as a 2D set defied by liear iequality costraits. EXAMPLE : Cosider the feasible set i R defied by 3 + 2 + 3 = ad ote that = 3, m =., 2, 3 0 If we stad o the plae defied by + + =, the the feasible set has the apperace of 2 3 a triagle i 2D, each edge of the triagle correspods to oe of the costraits,, 0. 2 3 3 = 0 2 2 = 0 3 = 0
CONVEX SES AND COMBINAIONS DEFINIION: A set S is cove if y, S ad λ [0,] we have λ + (- λ) y S. R his defiitio states that a set is cove if the segmet joiig ay of its two elemets is i the set. E.g, the set S i the figure is cove but the set Q is ot. S y Q y Note that a polyhedro is a cove set! DEFINIION: Let,, be vectors i ad let,, λ be positive scalars such as R λ λi = y = λ i i ii cove combiatio = =. he the vector is said to be a of,,. Especially the cove combiatios of ad is o the lie joiig ad. 2 2
EXREME POINS DEFINIION: Let P be a polyhedro. A vector P is a etreme poit of P if we caot fid two vectors yz, P, both differet from, ad a scalar λ [0, ] such that = λy + ( λ). z I.e., ca ot be writte as a cove combiatio of y ad z. w u v P z he vector w is ot a etreme poit because it ca be writte as a cove combiatio of v ad u. he vector isa etreme poit because if = λy + (- λ) z ad λ [0,] the either y P, or z P, or = y, or = z. y
VEREX DEFINIION :Let P be a polyhedro. A vector P is a verte of P if there eists some c such that c < c y, y satisfyig y P ad y. I other words, is a verte of P iff P is o oe side of a hyperplae ( the hyperplae { yc y = c } ) which meets P oly at the poit. { yc y = c w} w P he lie at the bottom touches P at a sigle poit is a verte. he vector w is ot a verte because there is o hyperplae that meets P oly at w. { yc y = c }
BASIC SOLUIONS Cosider the polyhedro P defied i terms of liear equality/iequality costraits: i b i, i M i 2 i = b i, i M2 a a where M, M are fiite ide sets ad a R. * DEFINIION: If a vector satisfies a i = b i for some i i M or M2 we say that the correspodig costrait is active or bidig at. * 3 A E P C 2 {( ) } Let P =,, + + =,,, 0. here are three costraits that are active at the poits A, B, C ad D. here are two costraits active at E, amely + + = ad = 0. 2 2 3 2 3 2 3 2 3 D B
{ a i I} { } { } DEFINIION:Let R ad let I = i a = b = i,, i (the set of active costraits at ) ii) spa = R. i * * * * i i k the the vector is a basic solutio if: i) All equality costraits are active. DEFINIION : If basic feasible solutio. * is a basic solutio that satisfies all of the costraits it is called a A B D C P E F he poits A, B, C, D, E ad F are all basic solutios because at each oe of them there are two liearly idepedet costraits active. he poits C, D, E ad F are also basic feasible solutios.
DEGENERACY * { } { } * * R I = i i = b i = i i k DEFINIION:Let be a basic feasible solutio ad let a,, be the set of active costraits at. * he the vector R is a deg eerate basic solutio if k >, i.e, there are more active basic feasible solutio. * * costraits tha at the poit. If is a basic feasible solutio the we have a degeerate A P B he poit A is a degeerate basic solutio, the poit B is a degeerate basic feasible solutio ad the poit C is a (odegerate) basic feasible solutio. C Degeeracy is a property of the descriptio of the polyhedro ot its geometry ad disappears with small perturbatios of b i. Degeeracy affects the performace of some algorithms but we will assume that all basic solutios are odegeerate.
ADJACEN BASIC SOLUIONS DEFINIION : wo distict basic solutios to the set of liear costraits i R : i b i, i M i i = b i, i M2 a a is called adjacet, where a R, if we ca fid - liearly idepedet costraits that are active at both of them. If two adjacet basic solutios are also feasible the the liesegmet joiig them is called a edge of the feasible set. D E A P F E ad F are adjacet to B; A ad C are adjacet to D. B C he lie joiig A ad C is a edge.
EQUIVALENCE OF HE HREE DEFINIIONS HEOREM :Let P be a oempty polyhedro ad let a) b) c) * * * is a verte; isa etreme poit; is a basic feasible solutio. * P. he the followig are eqivalet : CONCLUSION: WE NOW HAVE A WAY O DESCRIBE HE GEOMERIC PROPERIES EXREME POIN AND VEREX OF A POLYHEDRON ALGEBRAICALLY USING HE CONCEP BASIC FEASIBLE SOLUION.
IS HERE AN OPIMAL SOLUION? Cosider mi c + c2 + 2, 2 0 2 the problem 2 = mi c Note that the feasible set is ubouded! For c = (,) we see that the optimal solutio is = (0, 0); c = (, 0) c = (,) c = (, ) c = (0,) for c = (, 0) ay solutio of the form = (0, 2), 0 2, is optimal. Note that the set of optimal solutios is bouded; for c = (0,) ay solutio of the form = (, 0), 0, is optimal. Note that the set of optimal solutios is ubouded; for c = (, ) ay feasible solutio of the form = ( ca always be reduced by icreasig value of c coverges to ad we set the optimal value to ; if we impose the additioal costrait + 2 2 we see that the feasible set is empty, the we set the optimal value to be +.. herefore the, 2 )
I summary we have the followig possibilities: ) here eist a uique optimal solutio. 2) here eist multiple optimal solutios; i this case; the set of optimal solutios ca be either bouded or ubouded. 3) he optimal solutio is - ad o feasible solutio is optimal. 4) he feasible set is empty the the optimal solutio is + ad o feasible solutio eist.
EXISENCE OF EXREME POINS DEFINIION: A polyhedro P R cotais a lie if P ad a vector d R 0 such that + λd P λ R. { R i i } HEOREM: Suppose that the polyhedro P = a b, i =,, m. he the followig are equivalet: a) P has at least oe etreme poit. b) P does ot cotai a lie. c) vectors out of a,, a which are liearly idepedet. i m COROLLARY: Every oempty bouded polyhedro ad every oempty polyhedro i stadard form has at least oe etreme poit (basic feasible solutio). P Q P cotais a lie ad does ot have a etreme poit, Q does ot cotai a lie ad has etreme poits.
OPIMALIY OF EXREME POINS HEOREM: Cosider the problem of miimizig c over a polyhedro P. Suppose that P has at least oe etreme poit. he either the optimal solutio is or there a etreme poit which is optimal. COROLLARY : Cosider the problem of miimizig c over a polyhedro P. he there a etreme poit which is optimal. oempty bouded CONCLUSIONS: HE OPIMAL VALUE OF AN LP IS ALWAYS WELL DEFINED. FOR AN LP IN SANDARD FORM HE OPIMAL VALUE IS EIHER - OR HE OPIMAL VALUE IS AAINED A AN EXREME POIN.
HE SIMPLEX MEHOD
LP IN SANDARD FORM I the future we assume that we have a LP i stadard form, i.e., mi c m A = b where A = ( A,, A ) R. 0 We assume whitout loss of geerality that rak ( ) ( A ) = m. Because if rak A m the either A = b has o solutio or we ca delete a equality costrait.
BASIC SOLUIONS HEOREM : Cosider the stadard form formulatio of { A b 0} a polyhedro P = =, ad assume that rak( A) = m. he a vector is a basic solutio iff R { B B } we have A = b ad a ide set,, such that: a) he colums A,, A are lieary idepedet; { } m B B m b) If i B,, B the = 0. i m
CONSRUCING BASIC SOLUIONS ALGORIHM. Choose 2. Let 3. Let B system i = = 0 ( A,, A ) of m colums i B m B,, B B m equatios A B m. ad,, A if B rak( B B m B m. = B ) = b to m solve the get B,, B m. If is a basic solutio, the variables,, are called basic variables ad the remaig variables are called obasic. B B m B B m If,, 0 the the solutio is a basic feasible solutio. If,, > 0 the the solutio is a odegeerate basic feasible solutio. We will always B B m assume that basic solutios are odegeerate. he colums AB,, AB are called basic colums ad if rak( B) = m they form a basis for R. m { } wo bases are distict or differet if they ivolve differet sets B,, B of basic idices (the orderig of the idicies does't matter). he matri B is called a basis matri. m m