Ltur 4 Gry Algorithms, Ativity Sltion, Minimum Spnning Trs Sris: Logn Short (5), Virgini Dt: My, Gry Algorithms Suppos w wnt to solv prolm, n w r l to om up with som rursiv ormultion o th prolm tht woul giv us ni ynmi progrmming lgorithm. But thn, upon urthr insption, w noti tht ny optiml solution only pns on looking up th optiml solution to on othr suprolm. A gry lgorithm is n lgorithm whih xploits suh strutur, ignoring othr possil hois. Gry lgorithms n sn s rinmnt o ynmi progrmming; in orr to prov tht gry lgorithm is orrt, w must prov tht to omput n ntry in our tl, it is suiint to onsir t most on othr tl ntry; tht is, t h point in th lgorithm, w n mk gry, lolly-optiml hoi, n gurnt tht glolly-optiml solution still xists. Inst o onsiring multipl hois to solv suprolm, gry lgorithms only onsir singl suprolm, so thy run xtrmly quikly gnrlly, linr or los-to-linr in th prolm siz. Unortuntly, gry lgorithms o not lwys giv th optiml solution, ut thy rquntly giv goo (pproximt) solutions. To giv orrt gry lgorithm on must irst intiy optiml sustrutur (s in ynmi progrmming), n thn rgu tht t h stp, you only n to onsir on suprolm. Tht is, vn though thr my mny possil suprolms to rurs on, givn our sltion o suprolm, thr is lwys optiml solution tht ontins th optiml solution to th slt suprolm.. Ativity Sltion Prolm On prolm, whih hs vry ni (orrt) gry lgorithm, is th Ativity Sltion Prolm. In this prolm, w hv numr o tivitis. Your gol is to hoos sust o th tivis to prtiipt in. Eh tivity hs strt tim n n tim, n you n t prtiipt in multipl tivitis t on. Thus, givn n tivitis, 2,..., n whr i hs strt tim s i, inish tim i, w wnt to in mximum st o non-onliting tivitis. Th tivity sltion prolm hs mny pplitions, most notly in shuling jos to run on singl mhin... Optiml Sustrutur Lt s strt y onsiring sust o th tivitis. In prtiulr, w ll intrst in onsiring th st o tivitis S i,j th strt tr tivity i inishs n n or tivity j strts. Tht is, S i,j = { k i s k, k s j }. W n prtiipt in ths tivitis twn i n j. Lt A i,j mximum sust o non-onliting tivitis rom th sust S i,j. Suppos som k A i,j, thn w n rk own th optiml susolution A i,j s ollows A i,j = + A i,k + A k,j whr A i,k is th st st or S i,k (or k ), n A k,j is th st st or tr k. Anothr wy o intrprting this xprssion is to sy on w pl k in our optiml st, w n only onsir optiml solutions to suprolms tht o not onlit with k. Thus, w n immitly om up with rurrn tht llows us to om up with ynmi progrmming lgorithm to solv th prolm. A i,j = mx k S i,j + A i,k + A k,j
This prolm rquirs us to ill in tl o siz n 2, so th ynmi progrmming lgorithm will run in Ω(n 2 ) tim. Th tul runtim is O(n ) sin illing in singl ntry might tk O(n) tim. But w n o ttr! W will show tht w only n to onsir th k with th smllst inishing tim, whih immitly llows us to srh or th optiml tivity sltion in linr tim. Clim. For h S i,j, thr is n optiml solution A i,j ontining k S i,j o minimum inishing tim k. Not tht i th lim is tru, whn k is minimum, thn A i,k is mpty, s no tivitis n inish or k ; thus, our optiml solution only pns on on othr suprolm A k,j (giving us linr tim lgorithm). Hr, w prov th lim. Proo. Lt k th tivity o minimum inishing tim in S i,j. Lt A i,j som mximum st o nononliting tivitis. Consir A i,j = A i,j \ { l } { k } whr l is th tivity o minimum inishing tim in A i,j. It s lr tht A i,j = A i,j. W n to show tht A i,j os not hv onliting tivitis. W know l A i,j S i,j. This implis l k, sin k hs th min inishing tim in S i,j. All t A i,j \ { l } on t onlit with l, whih mns tht s t l, whih mns tht s t k, so this mns tht no tivity in A i,j \ { l } n onlit with k. Thus, A i,j is n optiml solution. Du to th ov lim, th xprssion or A i,j rom or simpliis to th ollowing xprssion in trms o k S i,j, th tivity with minimum inishing tim k. A i,j = + A k,j A i,j = A k,j { k } Algorithm Gry-AS ssums tht th tivitis r prsort in nonrsing orr o thir inishing tim, so tht i i < j, i j. Algorithm : Gry-AS() A { } // tivity o min i k or m = 2 n o i s m k thn // m strts tr lst itivity in A A A { m } k m rturn A By th ov lim, this lgorithm will prou lgl, optiml solution vi gry sltion o tivitis. Th lgorithm os singl pss ovr th tivitis, n thus only rquirs O(n) tim rmti improvmnt rom th trivil ynmi progrmming solution. I th lgorithm lso n to sort th tivitis y i, thn its runtim woul O(n log n) whih is still ttr thn th originl ynmi progrmming solution. 2 Minimum Spnning Trs Th minimum spnning tr prolm is strutur s ollows: Input: G = (V, E) unirt, w: E Z Output: A tr onnting ll o V with minimum totl wight Exmpl Blow is n xmpl o th minimum spnning tr o grph. In th xmpl, th gs orming th minimum spnning tr r olor lu whil gs tht r not prt o th minimum spnning tr r olor lk. 2
Fining Minimum Spnning Trs Algorithms or ining Minimum Spnning Trs tn to gry lgorithms. Ths lgorithms mintin n uil upon som sust o th gs o th grph A E with th invrint tht A is lwys sust o Minimum Spnning Tr. In h stp, gs will grily to A suh tht A will ontinu to sust o Minimum Spnning Tr. Whn th lgorithm trmints, A will ontin st o gs tht orm Minimum Spnning Tr o th grph. Th ut proprty o th MST prolm is in s ollows: Thorm.. Lt A E sust o som MST, lt S V sust suh tht thr is no g in A onnting S to V \S, n lt (u, v) th g in G with minimum wight suh tht u S, v S, thn A {(u, v)} is sust o som MST. Proo o ut proprty. Lt T som MST ontining A. Sin A hs no gs rom S to V \S ut T is spnning, T hs som g (x, y) with x S n y S. I (x, y) is rmov rom T, two sutrs r otin whih w n ll T x n T y. Without loss o gnrlity w ssum tht u T x n v T y. Aing (u, v) rsults in th tr T = T \{(x, y)} {(u, v)} whih is still spnning tr. wight(t ) = wight(t ) - wight((x, y)) + wight((u, v)). Sin wight((u, v)) wight((x, y)) w hv tht wight(t ) wight(t ) whih onsquntly mns tht T is n MST. Nxt, w will prsnt thr irnt lgorithms or ining n MST. Eh lgorithm will gry n orrtnss will ollow y mintining th ut proprty. As w v sn or, thr will tros twn running h lgorithm, so it s goo to unrstn how h o thm works.. Borůvk s Algorithm Borůvk s Algorithm only works or grphs with istint g wights. At high lvl, th st A mintin y th lgorithm n thought o s st o isjoint trs. In h stp, h tr in A piks th g with minimum wight ling out o it n mrgs with th tr t th othr npoint o this slt minimum g. This pross is rpt until only on tr rmins, whih is thn th rturn MST. Bor w lv urthr into th xt untionlity o Borůvk s Algorithm, w in two trms: suprnos n suprgs. A suprno is in to olltion no tht ontins multipl nos within it. A suprg (t, t ) xists twn suprnos t n t i x t, y t suh tht (x, y) E. I multipl suh gs xist, (x, y) is slt to th g with minimum wight tht onnts no in t n no in t. In Borůvk s Algorithm, vry no is irst initiliz to suprno n vry g is initiliz s suprg. In h stp o th lgorithm, h suprno t lots its minimum wight suprg (t, t ).
t is thn mrg togthr with t to rt nw suprno, n th rl-g orrsponing to th slt suprg (t, t ) is to th MST. Algorithm 2: Borůvk(G = (V, E)) T mpty st orh v V o Crt suprno t ontining v; T T {t} orh (u, v) E o Crt suprg (t, t ) whr u t n v t ; whil Mor thn on suprno in T o X mpty st orh t T o in min wight suprg (t, t ); X X {(t, t )}; orh (t, t ) X o //to optimiz runtim, itrt in DFS orr on X Mrg(t, t ); A A rl-g((t, t )) rturn A Exmpl In this xmpl w will go through th stps o pplying Borůvk s Algorithm in orr to in th MST o th ollowing grph: In th irst itrtion, w slt th minimum suprg o h suprno. Sin tr initiliztion vry no is suprno n vry g is suprg, this is quivlnt to slting th minimum wight g o h no. 4
Following th mrg oprtions th rmining suprnos r {,, } n {,, }. Th suprg twn ths suprnos is in y th rl-g (, ). Thus w g (, ) to th MST n ollowing nothr mrg oprtion w r lt with only on suprno t whih point th lgorithm trmints. Mrging two suprnos t n t n on y rting nw suprno t n giving it ll suprgs o t n t tht r not suprgs twn t n t. To sv on runtim, th mrg lgorithm low piks th smllr gr no t n s ll its gs to t without rting multipls. (In th ltur w in t o this.) Algorithm : Mrg(t, t ) orh suprg (t, x), x t o i (t, x) not suprg thn suprg (t, x); i (t, x) lry suprg thn wight(t, x) min{wight(t, x), wight(t, x)} Dlt t rom th grph n T Whil w ll s tht th othr lgorithms run str thn Borůvk s, not tht us h suprno is uilt up sprtly, this lgorithm is highly prlllizl proprty whih is vry sirl. This proprty mks Borůvk s lgorithm prtiulrly usul or nlyzing vry lrg ntworks, possily in istriut stting. 5
Corrtnss. Most o th orrtnss ollows rom th ut proprty: or ny suprno/tr in A, th ut proprty itts tht piking th minimum wight g out o th suprno is s to to A. Howvr, u to th prlllism o th Borůvk stps, w n to mk sur tht th suprnos rmin trs, i.. tht A os not ontin ny yls. W prov this low, n this omplts th orrtnss proo o th lgorithm. Clim 2. I th wights o th gs in grph r ll istint, th st o gs A slt y Borůvk s Algorithm will not ontin ny yls. Proo. Suppos Borůvk s Algorithm whn ppli to grph whos g wights r ll istint ns up slting gs tht orm yl o k suprnos on suprnos v, v 2,..., v k whr suprnos with onsutiv susripts r jnt to on nothr in th yl. This mns tht t som stp in Borůvk s Algorithm w will n up slting suprgs (v, v 2 ), (v 2, v ),..., (v k, v ). Sin th wight o h suprg is istint, thr must mximum wight suprg in this yl. Cll this mximum suprg (v i, v i+ ). Sin this is th lrgst wight suprg in th yl, w know tht suprg (v i+, v i+2 ) must hv smllr wight thn suprg (v i, v i+ ). Thus (v i+, v i+2 ) will th suprg slt y suprno v i+ uring this stp o Borůvk s Algorithm. Howvr, similr rgumnt n ppli to suprg (v i, v i ) whih must lso hv smllr wight thn (v i, v i+ ) n onsquntly will th suprg slt y Borůvk s Algorithm or suprno v i t this stp. But this ls to ontrition sin nithr o th npoints o (v i, v i+ ) will n up slting this suprg uring Borůvk s Algorithm. Thus thr nnot ny yls in th st o gs slt y Borůvk s Algorithm i th wights o ll th gs in th grph r istint. Runtim. Th runtim o th mrg lgorithm is: O(min{g(t )) whr t is th no mrg into nothr no t. In ltur w mntion tht w n run Borůvk s Algorithm so tht th suprgs out o h suprno r ss t most onstnt numr o tim in h itrtion, so tht h itrtion runs in O(m) tim. Hr w giv th tils on how to o this. Consir th st X o gs to mrg in prtiulr itrtion. W will o th mrgs o gs in X in suitl orr. First go through X n rt out o it grph in jny list rprsnttion. This n on in O(n + X ) tim whr n is th urrnt numr o suprnos. Now w n run DFS on th sugrph inu rom X. Whnvr suprg (t, t ) o X is trvrs y th DFS srh, w mrg t into t. Sin X is orst (s w prov ov), th DFS srh will not nountr ny kgs. Thus, ny suprno tht is n npoint o suprg in X will us s t t most on. Thus its suprgs will to no t t most on. To hk whthr pir (t, x) is tully suprg o t, w n stor th suprgs o suprno in two ormts: th jny list ormt ( list o th suprgs L t ) n th jny vtor ormt (n rry N t o lngth n, suh tht i (t, x) is suprg, N[t] = ). Now, whn t is mrg into t, or ny suprg (t, x) out o t, on irst hks i N t [x] =. I so, thn just th wight o (t, x) is moii. Othrwis, N t [x] is st to n (t, x) is to th n o L t. In ition, N x [t] is st to n (t, x) is to th n o L x. Whn x is ss ltr in th lgorithm, oth (t, x) n (t, x) will ss whn srhing or th minimum g wight. Thn, (t, x) will isr sin t is no longr tiv, n w n think o this oprtion on (t, x) s ing pi or lry y th ition o (t, x) to x. (W omitt ll ths tils in ltur to sv tim.) Thus h itrtion o th lgorithm n run in O(m) tim (whil lso pying or isring utur (t, x) suprgs out o intiv suprnos t ). Anothr wy to mk th mrg oprtion st is to us union-in t strutur (s Kruskl s lgorithm low). Now w rss th numr o itrtions. In h stp o Borůvk s Algorithm, th minimum siz suprno is mrg with nothr suprno. Th siz o th minimum siz suprno will thus t lst oul tr h stp. It strts t n ns t n. This mns tht th numr o itrtions in Borůvk s Algorithm is log(n) whih givs us th inl runtim o Borůvk s Algorithm: O(m log n).
.2 Prim s Algorithm At high lvl, th st A mintin y Prim s Algorithm is singl tr. In h stp, th g with minimum wight ling out o A n onnting to no tht hs not yt n onnt to A is slt n to A. On A onnts vry no in th grph, it is rturn s n MST o th grph. Prim s Algorithm is similr to Dijkstr s Algorithm in tht stimts o th istn to h no r mintin n upt s th lgorithm progrsss. V (A) is in to th vrtis tht gs in A r inint to. (v) v V is in to th minimum g wight rom no o V (A) to v. Algorithm 4: Prim(G = (V, E)) A mpty st F mpty Fioni hp orh v V o (v) ; π(v) NIL; put v in F with ky (v); pik s rom V ; (s) ; whil F is not mpty o u F.xtrtmin; A A {(π(u), u)}; orh (u, v) E o i (v) > wight(u, v) thn (v) wight(u, v); π(v) u; rturn A Prim s Algorithm is irt moiition o Dijkstr s Algorithm n hs runtim o O(m + n log n). Exmpl In this xmpl w will run through th stps o pplying Prim s Algorithm in orr to intiy n MST or th ollowing grph: Suppos w slt no to th sour no, s. W thn xtrt no rom th Fioni hp n st () =, () = n π() =, π() =.
,,,,,, Sin () is now th smllst vlu in th Fioni hp, w visit no. Bus π() = w g (, ) to th st A. W thn upt th istn stimts n prnt ils o nos tht hv gs onnting to. Thus w st () =, () =, () =, n π() =, π() =, n π() =.,,,,,, No is xtrt nxt n w g (, ) to A. W thn st () = n π() =.
,,,,,, No is xtrt nxt n w g (, ) to A. W thn st () = n π() =.,,,,,, No is xtrt nxt n w g (, ) to A. No nos r upt tr this stp.,,,,,,
Finlly no is xtrt n g (, ) is to A. Atr this stp th Fioni hp will mpty n Prim s Algorithm will rturn A.,,,,,,. Kruskl s Algorithm At high lvl, th st A mintin y Kruskl s Algorithm is st o isjoint trs similr in sns to Borůvk s Algorithm. Howvr, Kruskl s Algorithm irs rom Borůvk s Algorithm in its upt stp. During upt stp i, i th i-th smllst g onnts irnt trs, mrg th two trs onnt y this g. Th lgorithm progrsss until vntully only on tr rmins t whih point th st A rprsnts n MST o th grph. Kruskl s Algorithm utilizs th union-in t strutur in orr to hnl th mrging o th isjoint trs mintin y th lgorithm. Th union-in t strutur supports isjoint sts with th ollowing oprtions: mkst(x): rts nw st ontining x provi tht x ny othr sts in(x): rturns th nm o th st ontining x union(x, y): mrg th st ontining x n th st ontining y into on st Th lgorithm itsl n strutur s ollows: Algorithm 5: Kruskls(G = (V, E)) A mpty st; E sort gs y wight in non rsing orr; orh v V o mkst(v); orh (u, v) E o i in(u) in(v) thn A A {(u, v)}; union(u, v); rturn A; Th orrtnss irtly ollows rom th ut proprty. Th runtim o Kruskl s Algorithm is pnnt on two tors: th tim to sort th gs y wight n th runtim o th union-in t strutur oprtions. Whil Ω(m log n) tim is rquir i w us 0
omprison sorting, in mny ss, w my l to sort th gs in linr tim. (Rll, you show in th homwork tht CountingSort n us to sort th gs in O(m) tim i th wights r givn y intgrs oun y polynomil in m. Th orrsponing lgorithm is ll Rix Sort.) In this s, th runtim is oun y th runtim o th union-in oprtions n is givn y O(nT (mkst)+mt (in)+nt (union)). Th st known runtim or th union-in oprtions is mortiz O(α(n)) whr α(n) is th invrs Akrmnn untion. Intrstingly, th vlu o th invrs Akrmnn is or ll prtil purposs tiny: α(n) 4, n < # toms in th univrs n thus or ll prtil purposs th union-in oprtions run in onstnt tim. Thus, in mny sttings, th runtim o Kruskl s lgorithm is nrly linr in th numr o gs. Th tul inition o α(n) is α(n) = min{k A(k) n}, whr A(k) is th Akrmnn untion vlut t k. A(k) itsl is in vi mor gnrl Akrmnn untion, in prtiulr, A(k) = A k (2). A k (x) is in rursivly. A 0 (x) = + x, so tht A 0 (2) =. A (x) = 2x (A 0 itrt x tims), so tht A (2) = 4 A 2 (x) = 2 x x (A itrt x tims), so tht A 2 (2) =... A (x) 2 222, towr o x 2s (A 2 itrt x tims), so tht A (2) 2 A 4 (x) is t lst towr o x towrs o x 2s, n A 4 (2) is t lst towr o 4 twos! This numr is lrgr thn th totl numr o toms in th known univrs, n lso lrgr thn th numr o nnosons sin th Big Bng. (Thus, α(n) 4 or ll prtil purposs.) Exmpl In this xmpl w will run through th stps o pplying Kruskl s Algorithm in orr to intiy n MST or th ollowing grph: W gin y rting nw st or h no in th grph. W thn ing itrting ovr th gs in non rsing orr. Th irst g w xmin is (, ). This g onnts nos n whih r urrntly not prt o th sm st. W thus inlu this g in A n union th sts ontining n.
Th nxt g in our sort orr is g (, ). Sin n r not prt o th sm st w inlu this g in A n union th sts ontining n. Th nxt g in th sort orr is (, ). W union th sts ontining n n g (, ) to A. Th nxt g in th sort orr is (, ). W union th sts ontining n n g (, ) to A. 2
Th nxt g in th sort orr is (, ). W union th sts ontining n n g (, ) to A. At this point ll nos r ontin in th sm st n so no urthr gs r to A.