CPSC 665 : An Algorithmist s Toolkit Lecture 4 : 21 Jan Linear Programming

Transcription

1 CPSC 665 : An Algorithmist s Toolkit Lctur 4 : 21 Jan 2015 Lcturr: Sushant Sachdva Linar Programming Scrib: Rasmus Kyng 1. Introduction An optimization problm rquirs us to find th minimum or maximum) of a function ovr a domain, by finding an lmnt of th domain on which th function is minimizd rsp. maximizd). Th problm of minimizing a function f : P R ovr a domain P R is usually writtn as minfx ). x P W call an optimization problm ovr a subst of th rals a linar program LP) whn it satisfis th following conditions 1) f is linar, i.. fx ) = c T x. 2) P is dfind by half-spac constraints and affin subspac constraints, i.. linar in)qualitis. For xampl, w might dfin P R by m inqualitis P = x :a T 1 x b 1 a T 2 x b 2 a T 3 x = b 3. a T mx b m } A constraint a T i x = b i dfins an n 1 dimnsional affin subspac, whil a constraint a T i x b i dfins a half-spac. Affin subspacs and half-spacs ar both convx and bcaus P is th intrsction of th sts whr ach constraint is satisfid, it must also b a convx st or mpty). Dantzig introducd th dfinition of linar programs in Exampl 1.1. max x 1,x 2 ) R 2x 1 + x 2 s.t. x 1 + x 2 3 x 1 0 x 3 0 x 1 2 Th xampl is shown in figur 1. W can writ LPs succinctly using vctor inqualitis. Dfinition 1.2 Vctor inquality). For vctors a R m and b R m dfin a b iff i.a i b i. W say an LP is in gnral form whn it is xprssd as max x x R nct s.t. Ax b 1

2 P x c Figur 1. A simpl xampl of a linar program. An LP is said to b in standard form whn it is xprssd as max x x R nct s.t. Ax = b x 0 HW: Evry LP can b writtn as an quivalnt LP which is in gnral form. HW: Similarly, vry LP can b writtn as an quivalnt LP which is in standard form. Dfinition 1.3 Fasibl solution). x is fasibl iff x P. Dfinition 1.4 Optimal solution). x is an optimal solution iff x is fasibl and c T x = max x P c T x. Dfinition 1.5 Tight constraint). A constraint is tight for a point x iff it is satisfid with quality at x. Dfinition 1.6 Basic fasibl solution). x is a basic fasibl solution iff it is fasibl and has at last n tight constraints that ar linarly indpndnt. Constraints a T 1 x b 1, a T 2 x b 2, a T 3 x = b 3,..., a T mx b m } ar said to b linarly indpndnt if a 1, a 2, a 3,..., a m } forms a linarly indpndnt st of vctors. Dfinition 1.7 Unboundd LP). An LP max x P c T x is said to b unboundd iff th suprmum of th valus it taks is +. Similarly, a minimization program is said to b unboundd iff th infimum of its valus is. Thorm 1.8. For any LP, on of th following is tru 1) Th LP is infasibl. 2) Th LP is unboundd. 3) Th LP has a basic fasibl solution x that is optimal. Exampl 1.9. [Th Assignmnt Problm] Th Assignmnt Problm rquirs us to match a maximal numbr of jobs to popl that can prform thm. Formally, w hav a st of n jobs J, and a st of n popl P, a st of allowd job assignmnts E J P. W rquir that vry job is assignd to at most on prson, and that vry prson gts at most on job. Our goal is to find a valid job assignmnt A E that maximizs A. 2

3 W can xprss th problm as an intgr program IP). To do this, w giv a bijction btwn an assignmnt A and a vctor of intgr variabls x 0, 1} E. For ach E dfin x 0, 1} s.t. 1 if A, x = 0 othrwis. Th intgr program is max x 0,1} E E x s.t. j J. p P. p:j,p) E j:j,p) E x j,p) 1 x j,p) 1 Obsrvation OPT-IP = max assignmnt siz. Proof: An IP solution x is fasibl iff its corrsponding assignmnt is valid, and th valu of th IP is xactly th siz of th corrsponding assignmnt. Thus, if A is an optimal assignmnt, thn th corrsponding x achivs th sam valu so OPT-IP max assignmnt siz. Similarly, if x is an optimal solution to th IP, thn its corrsponding assignmnt A has siz qual to th valu of th IP at x, so max assignmnt siz OPT-IP. Rmark: It is vry common that combinatorial optimization problms can b xprssd xactly as an IP. W can rlax th IP to an LP: max x x R E E s.t. j J. x j,p) 1 p P. p:j,p) E j:j,p) E E.0 x 1 x j,p) 1 W call this a rlaxation bcaus th nw constraints ar wakr: Any fasibl solution to th IP is also a fasibl solution to th LP. This implis our nxt obsrvation. Obsrvation OPT-LP OPT-IP. Th assignmnt problm LP has an unusual proprty, which is capturd in th nxt thorm. Thorm For th assignmnt problm LP, vry basic fasibl solution is intgral, maning x 0, 1} for vry E. From this thorm and thorm 1.8 it follows that thr xists an optimal solution to th LP which satisfis for all variabls that x 0, 1}, and hnc w gt a simpl corollary. Corollary OPT-IP OPT-LP. Th main algorithms for solving gnral linar programs ar 1) Simplx Dantzig, 1947). Oftn vry fast in practic. Thr is no known dtrministic vrsion of th algorithm which runs in polynomial tim. Thr is a randomizd Simplx algorithm du to Klnr and Spilman that runs in polynomial tim. 3

4 2) Ellipsoid Mthod Khachiyan, 1979). First polynomial tim algorithm for LPs and many othr convx programs. Vry slow in practic. 3) Intrior Point Mthod Karmarkar, 1984). Polynomial running tim and oftn comptitiv with th simplx algorithm in practic. What dos it man to solv an LP in polynomial tim? By this, w man that th running tim is polynomial in n, th numbr of variabls; m, th numbr of constraints; and L, th bit complxity of th problm. L can b dfind in diffrnt ways, for xampl, th bit lngth of th largst numbr in A, b, and c. Evn if arithmtic oprations ar countd as to taking constant tim, all known algorithms for linar programming hav a running tim that dpnds polynomially on L. Algorithms whos running tim dos not dpnd on L in modls with constant tim arithmtic, in addition to bing polynomial-tim undr th standard bit opration modl, ar rfrrd to as strongly polynomial. It is an important opn problm whthr thr xists a strongly polynomial algorithm for linar programming. 2. Approximation Algorithm using Linar Programming Exampl 2.1. [Minimum Vrtx Covr] Givn a an undirctd graph G = V, E), find a subst S V which minimizs S subjct to th condition u, v) E.u S or v S. Th dcision vrsion of this problm is NP-complt Karp, 1972). Thr is a simpl IP for this problm. To dscrib it, w giv a bijction btwn a candidat vrtx covr S and a vctor of intgr variabls x 0, 1} V. For ach v V dfin x v 0, 1} s.t. 1 if v S, x v = 0 othrwis. Th following IP can b shown straightforwardly to b quivalnt to th minimum vrtx covr problm. min x v x 0,1} V v V s.t. u, v) E.x u + x v 1 As for th assignmnt problm, w can rlax th IP to an LP, by ltting x R V and adding constraints 0 x v 1 for all v. Sinc th LP is a rlaxation, w gt th corollary blow. Corollary 2.2. OPT-LP OPT-IP = minimum vrtx covr siz. In this problm, thr may b a gap btwn th valus of th LP and th IP. For xampl, considr th complt graph on n vrtics K n. Th minimum vrtx covr for this graph has siz OPT-IP = n 1, whil th optimal solution for th LP has x v = 1/2 for all v V, and thus OPT-LP = n/2. In this cas, as n OPT-IP/OPT-LP = 2. Dfinition 2.3 Intgrality Gap). Th intgrality gap of an LP rlaxation of an IP minimization problm is dfind as OPT-IPI) sup instancs I OPT-LPI). Manwhil, th intgrality gap of a rlaxation of a maximization problm is dfind as inf instancs I OPT-IPI) OPT-LPI). From th K n xampl w can dduc th nxt corollary. Corollary 2.4. Intgrality gap of Minimum Vrtx Covr LP rlaxation 2. 4

5 Thorm 2.5. Intgrality gap of Minimum Vrtx Covr LP rlaxation 2. A common mthod for dmonstrating an uppr bound on an intgrality gap is to show that a fasibl solution to th rlaxation can b roundd to a fasibl intgr solution with littl or no loss in th quality of th solution. Proof of thorm. To convrt any fasibl LP solution x to a fasibl IP solution z w us a rounding algorithm, in this cas a vry simpl on: Dfin th vrtx st S = v V.x v 1/2}. S is a vrtx covr, bcaus x fasibl implis that for vry dg u, v), w hav x u + x v 1, and this mans maxx u, x v ) 1/2, so th dg is covrd. Lt z b th IP solution corrsponding to th covr S. Now v V z v 2 v S x v 2 v V x v. In particular, by considring th optimal LP solution x, and rounding it to a fasibl IP solution z, w find OPT-IP z v 2 x v = 2 OPT-LP. v V v V Dfinition 2.6 Approximation Algorithm). W say an algorithm for a minimization problm has an approximation ratio of α if for vry instanc I, it outputs a solution ALGI) with costalgi)) αopt. For a maximization problm, w say an algorithm has an approximation ratio of α if for vry instanc I, it outputs a solution ALGI) with costalgi)) αopt. No known algorithm for Minimum Vrtx Covr achivs a bttr approximation ratio than th α = 2 approximation algorithm givn abov. Dinur and Safra showd that a 1.36 approximation algorithm for th problm would imply P = NP. Khot and Rgv showd that for any constant ɛ > 0, a 2 ɛ approximation algorithm would imply that th Uniqu Gams Conjctur is fals. Exampl 2.7. MAX-SAT W ar givn n boolan variabls x i T, F } and m clauss c j,.g. c 1 : x 1 x 2 x 3... c 2 : x 2 x 4 x c m : x 1 x 5 x 6... and w want to find som boolan assignmnt of our variabls maximizing th numbr of satisfid clauss. This is asily translatd into an IP. First, w crat a nw intgr variabl y i 0, 1} for ach boolan variabl x i. A bijction btwn th boolan variabls an th intgr variabls is givn by 1 if x i is tru y i = 0 othrwis. For ach claus c j w mak an intgr variabl z j 0, 1} and add a xprssing that z j = 1 only if c j is satisfid. For xampl, th claus c 1 in th xampl abov givs th constraint y 1 + y y 3 ) +... z 1. Th objctiv of th IP is to maximiz m j=1 z j, and th optimum of this IP is xactly qual to th valu of th MAX-SAT problm. W can rlax this to an LP by instad allowing th y i and z j to rang ovr th rals and adding constraints 0 y i 1 and 0 z j 1. 5

6 Thorm 2.8 Raghavan-Thomson 1987). Givn a fasibl solution y* to th abov LP, w can gt a fasibl solution to th corrsponding IP by dfining th following indpndnt random variabls 1 with probability yi x i = 0 othrwis. Doing so givs us a solution with an xpctd valu gratr than a factor of 1 1 tims th actual optimal valu. To prov this thorm, w first show th following lmma: Lmma 2.9. Pr[c i is satisfid ] 1 1 ) z i. Proof of lmma. W dmonstrat th mthodology for th xampl claus c 1. Th logic is asily xtndd to gnral clauss. Pr[c i is satisfid ] = 1 1 y 1 )1 y 2 )y 3 ) 1 1 y i + y y 3 ) z ) 3 i ) z i. In th first inquality w usd th AM-GM inquality. Proof of thorm. Dnot th optimal solution of th Linar Program by OPT-LP, and th optimal solution of th MAX-SAT problm by OPT. It should b obvious that E[x i ] = y i Thn by th linarity of xpctation, E[# of clauss satisfid] 1 1 ) z i i = 1 1 ) OPT-LP 1 1 ) OPT. Rfrncs ) 3 6