CS261: A Second Course in Algorithms Lecture #1: Course Goals and Introduction to Maximum Flow

Transcription

1 CS61: A Second Coure in Algorihm Lecure #1: Coure Goal and Inroducion o Maximum Flo Tim Roughgarden January 5, Coure Goal CS61 ha o major coure goal, and he coure pli roughly in half along hee line. 1.1 Well-Soled Problem Thi fir goal i ery much in he piri of an inroducory coure on algorihm. Indeed, he fir fe eek of CS61 are prey much a direc coninuaion of CS161 he opic ha e d coer a he end of CS161 a a emeer chool. Coure Goal 1 Learn efficien algorihm for fundamenal and ell-oled problem. There a collecion of problem ha are flexible enough o model many applicaion and can alo be oled quickly and exacly, in boh heory and pracice. For example, in CS161 you udied hore-pah algorihm. You hould hae learned all of he folloing: 1. The formal definiion of one or more arian of he hore-pah problem.. Some famou hore-pah algorihm, like Dijkra algorihm and he Bellman-Ford algorihm, hich belong in he canon of algorihm greae hi.. Applicaion of hore-pah algorihm, including o problem ha don explicily inole pah in a neork. For example, o he problem of planning a equence of deciion oer ime. c 016, Tim Roughgarden. Deparmen of Compuer Science, Sanford Unieriy, 474 Gae Building, 5 Serra Mall, Sanford, CA im@c.anford.edu. 1

2 The udy of uch problem i op prioriy in a coure like CS161 or CS61. One of he bigge benefi of hee coure i ha hey preen you from reinening he heel (or rying o inen omehing ha doen exi), inead alloing you o and on he houlder of he many brillian compuer cieni ho preceded u. When you encouner uch problem, you already hae good algorihm in your oolbox and don hae o deign one from crach. Thi coure ill alo gie you pracice poing applicaion ha are ju hinly diguied erion of hee problem. Specifically, in he fir half of he coure e ll udy: 1. he maximum flo problem;. he minimum cu problem;. graph maching problem; 4. linear programming, one he mo general polynomial-ime olable problem knon. Our algorihm for hee problem ih hae running ime a bi bigger han hoe you udied in CS161 (here almo eeryhing run in near-linear ime). Sill, hee algorihm are ufficienly fa ha you hould be happy if a problem ha you care abou reduce o one of hee problem. 1. No-So-Well-Soled Problem Coure Goal Learn ool for ackling no-o-ell-oled problem. Unforunaely, many real-orld problem fall ino hi camp, for many differen reaon. We ll focu on o clae of uch problem. 1. NP -hard problem, for hich e don expec here o be any exac polynomial-ime algorihm. We ll udy eeral broadly ueful echnique for deigning and analyzing heuriic for uch problem.. Online problem. The anachroniic name doe no refer o he Inerne or ocial neork, bu raher o he realiic cae here an algorihm mu make irreocable deciion ihou knoing he fuure (i.e., ihou knoing he hole inpu). We ll focu on algorihm for NP -hard and online problem ha are guaraneed o oupu a oluion reaonably cloe o an opimal one. 1. Inended Audience CS61 ha o audience, boh imporan. The fir i uden ho are aking heir final algorihm coure. For hi group, he goal i o pack he coure ih eenial and likelyo-be-ueful maerial. The econd i uden ho are conemplaing a deeper udy of algorihm. Wih hi group in mind, hen he opporuniy preen ielf, e ll dicu

3 recen reearch deelopmen and gie you a glimpe of ha you ll ee in fuure algorihm coure. For hi econd audience, CS61 ha a hird goal. Coure Goal Proide a gaeay o he udy of adanced algorihm. Afer compleing CS61, you ll be ell equipped o ake any of he many 00- and 00- leel algorihm coure ha he deparmen offer. The pace and difficuly leel of CS61 inerpolae beeen ha of CS161 and more adanced heory coure. When you peak o audience, i good o hae one or a fe canonical audience member in mind. For your reference and amuemen, here your inrucor menal model for canonical uden in coure a differen leel: 1. CS161: a conan fracion of he uden do no an o be here, and/or hae mah.. CS61: a elf-elecing group of uden ho like algorihm and an o learn much more abou hem. Suden may or may no loe mah, bu hey houldn hae i.. CSxx: geared oard uden ho are doing or ould like o do reearch in algorihm. Inroducion o he Maximum Flo Problem 5 Figure 1: (a, lef) Our fir flo neork. Each edge i aociaed ih a capaciy. (b, righ) A ample flo of alue 5, here he red, green and blue pah hae flo alue of, 1, repeciely..1 Problem Definiion The maximum flo problem i a one-cold claic in he deign and analyi of algorihm. I eay o underand inuiiely, o le do an informal example before giing he formal

4 definiion. The picure in Figure 1(a) reemble he one you a hen udying hore pah, bu he emanic are differen. Each edge i labeled ih a capaciy, he maximum amoun of uff ha i can carry. The goal i o figure ou ho much uff can be puhed from he erex o he erex. For example, Figure 1(b) exhibi a mehod of puhing fie uni of flo from o, hile repecing all edge capaciie. Can e do beer? Cerainly no, ince a mo 5 uni of flo can ecape on i o ougoing edge. Formally, an inance of he maximum flo problem i pecified by he folloing ingredien: a direced graph G, ih erice V and direced edge E; 1 a ource erex V ; a ink erex V ; a nonnegaie and inegral capaciy u e for each edge e E. () () 5 (1) () () Figure : Denoing a flo by keeping rack of he amoun of flo on each edge. Flo amoun i gien in bracke.. Since he poin i o puh flo from o, e can aume ihou lo of generaliy ha ha no incoming edge and ha no ougoing edge. Gien uch an inpu, he feaible oluion are he flo in he neork. While Figure 1(b) depic a flo in erm of eeral pah, for algorihm, i ork beer o ju keep rack of he amoun of flo on each edge (a in Figure ). Formally, a flo i a nonnegaie ecor {f e } e E, indexed by he edge of G, ha aifie o conrain: 1 All of our maximum flo algorihm can be exended o undireced graph; ee Exercie Se #1. Eery flo in hi ene arie a he uperpoiion of flo pah and flo cycle; ee Problem #1. 4

5 Capaciy conrain: f e u e for eery edge e E; Coneraion conrain: for eery erex oher han and, amoun of flo enering = amoun of flo exiing. The lef-hand ide i he um of he f e oer he edge incoming o ; likeie ih he ougoing edge for he righ-hand ide. The objecie i o compue a maximum flo a flo ih he maximum-poible alue, meaning he oal amoun of flo ha leae. (A e ll ee, hi i he ame a he oal amoun of flo ha ener.). Applicaion Why hould e care abou he maximum flo problem? Like all cenral algorihmic problem, he maximum flo problem i ueful in i on righ, plu many differen problem are really ju hinly diguied erion of maximum flo. For ome relaiely obiou and lieral applicaion, he maximum flo problem can model he rouing of raffic hrough a ranporaion neork, packe hrough a daa neork, or oil hrough a diribuion neork. In upcoming lecure e ll proe he le obiou fac ha problem ranging from biparie maching o image egmenaion reduce o he maximum flo problem.. A Naie Greedy Algorihm We no urn our aenion o he deign of efficien algorihm for he maximum flo problem. A priori, i i no clear ha any uch algorihm exi (for all e kno righ no, he problem i NP -hard). We begin by conidering greedy algorihm. Recall ha a greedy algorihm i one ha make a equence of myopic and irreocable deciion, ih he hope ha eeryhing omeho ork ou a he end. For mo problem, greedy algorihm do no generally produce he be-poible oluion. Bu i ill orh rying hem, becaue he ay in hich greedy algorihm break ofen yield inigh ha lead o beer algorihm. The imple greedy approach o he maximum flo problem i o ar ih he all-zero flo and greedily produce flo ih eer-higher alue. The naural ay o proceed from one o he nex i o end more flo on ome pah from o (cf., Figure 1(b)). A flo correpond o a eady-ae oluion, ih a conan rae of arrial a and deparure a. The model doe no capure he ime a hich flo reache differen erice. Hoeer, i no hard o exend he model o alo capure emporal apec a ell. 5

6 A Naie Greedy Algorihm iniialize f e = 0 for all e E repea earch for an - pah P uch ha f e < u e for eery e P // ake O( E ) ime uing BFS or DFS if no uch pah hen hal ih curren flo {f e } e E ele room on e {}}{ le = min (u e f e ) e P }{{} room on P for all edge e of P do increae f e by Noe ha he pah earch ju need o deermine heher or no here i an - pah in he ubgraph of edge e ih f e < u e. Thi i eaily done in linear ime uing your faorie graph earch ubrouine, uch a breadh-fir or deph-fir earch. There may be many uch pah; for no, e allo he algorihm o chooe one arbirarily. The algorihm hen puhe a much flo a poible on hi pah, ubjec o capaciy conrain. () 5 () () Figure : Greedy algorihm reurn ubopimal reul if fir pah picked i ---. Thi greedy algorihm i naural enough, bu doe i ork? Tha i, hen i erminae ih a flo, need hi flo be a maximum flo? Our ole example hu far already proide a negaie aner (Figure ). Iniially, ih he all-zero flo, all - pah are fair game. If he algorihm happen o pick he zig-zag pah, hen = min{, 5, } = and i roue uni of flo along he pah. Thi aurae he upper-lef and loer-righ edge, a hich poin here i no - pah uch ha f e < u e on eery edge. The algorihm erminae a hi 6

7 poin ih a flo ih alue. We already kno ha he maximum flo alue i 5, and e conclude ha he naie greedy algorihm can erminae ih a non-maximum flo. 4.4 Reidual Graph The econd idea i o exend he naie greedy algorihm by alloing undo operaion. For example, from he poin here hi algorihm ge uck in Figure, e d like o roue o more uni of flo along he edge (, ), hen backard along he edge (, ), undoing of he uni e roued he preiou ieraion, and finally along he edge (, ). Thi ould yield he maximum flo of Figure 1(b). u e f e u e (f e ) f e Figure 4: (a) original edge capaciy and flo and (b) reulan edge in reidual neork. Figure 5: Reidual neork of flo in Figure. We need a ay of formally pecifying he alloable undo operaion. Thi moiae he folloing imple bu imporan definiion, of a reidual neork. The idea i ha, gien a graph G and a flo f in i, e form a ne flo neork G f ha ha he ame erex e of G and ha ha o edge for each edge of G. An edge e = (, ) of G ha carrie flo f e and ha capaciy u e (Figure 4(a)) pan a forard edge (u, ) of G f ih capaciy u e f e (he room remaining) and a backard edge (, ) of G f ih capaciy f e (he amoun 4 I doe compue ha knon a a blocking flo; more on hi nex lecure. 7

8 of preiouly roued flo ha can be undone). See Figure 4(b). 5 Obere ha - pah ih f e < u e for all edge, a earched for by he naie greedy algorihm, correpond o he pecial cae of - pah of G f ha comprie only forard edge. For example, ih G our running example and f he flo in Figure, he correponding reidual neork G f i hon in Figure 5. The four edge ih zero capaciy in G f are omied from he picure. 6.5 The Ford-Fulkeron Algorihm Happily, if e ju run he naural greedy algorihm in he curren reidual neork, e ge a correc algorihm, he Ford-Fulkeron algorihm. 7 Ford-Fulkeron Algorihm iniialize f e = 0 for all e E repea earch for an - pah P in he curren reidual graph G f uch ha eery edge of P ha poiie reidual capaciy // ake O( E ) ime uing BFS or DFS if no uch pah hen hal ih curren flo {f e } e E ele le = min e P (e reidual capaciy in G f ) // augmen he flo f uing he pah P for all edge e of G hoe correponding forard edge i in P do increae f e by for all edge e of G hoe correponding reere edge i in P do decreae f e by For example, aring from he reidual neork of Figure 5, he Ford-Fulkeron algorihm ill augmen he flo by uni along he pah. Thi augmenaion produce he maximum flo of Figure 1(b). We no urn our aenion o he correcne of he Ford-Fulkeron algorihm. We ll orry abou opimizing he running ime in fuure lecure. 5 If G already ha o edge (, ) and (, ) ha go in oppoie direcion beeen he ame o erice, hen G f ill hae o parallel edge going in eiher direcion. Thi i no a problem for any of he algorihm ha e dicu. 6 More generally, hen e peak abou he reidual graph, e uually mean afer all edge ih zero reidual capaciy hae been remoed. 7 Ye, i he ame Ford from he Bellman-Ford algorihm. 8

9 .6 Terminaion We claim ha he Ford-Fulkeron algorihm eenually erminae ih a feaible flo. Thi follo from o inarian, boh proed by inducion on he number of ieraion. Fir, he algorihm mainain he inarian ha {f e } e E i a flo. Thi i clearly rue iniially. The parameer i defined o ha no flo alue f e become negaie or exceed he capaciy u e. For he coneraion conrain, conider a erex. If i no on he augmening pah P in G f, hen he flo ino and ou of remain he ame. If i on P, ih edge (x, ) and (, ) belonging o P, hen here are four cae, depending on heher or no (x, ) and (, ) correpond o forard or reere edge. For example, if boh are forard edge, hen he flo augmenaion increae boh he flo ino and he flo ou of increae by. If boh are reere edge, hen boh he flo ino and he flo ou of decreae by. In all four cae, he flo in and flo ou change by he ame amoun, o coneraion conrain are preered. Second, he Ford-Fulkeron algorihm mainain he propery ha eery flo amoun f e i an ineger. (Recall e are auming ha eery edge capaciy u e i an ineger.) Induciely, all reidual capaciie are inegral, o he parameer i inegral, o he flo ay inegral. Eery ieraion of he Ford-Fulkeron algorihm increae he alue of he curren flo by he curren alue of. The econd inarian implie ha 1 in eery ieraion of he Ford-Fulkeron algorihm. Since only a finie amoun of flo can ecape he ource erex, he Ford-Fulkeron algorihm eenually hal. By he fir inarian, i hal ih a feaible flo. 8 Of coure, all of hi applie equally ell o he naie greedy algorihm of Secion.. Ho do e kno heher or no he Ford-Fulkeron algorihm can alo erminae ih a non-maximum flo? The hope i ha becaue he Ford-Fulkeron algorihm ha more pah eligible for augmenaion, i progree furher before haling. Bu i i guaraneed o compue a maximum flo?.7 Opimaliy Condiion Anering he folloing queion ill be a major heme of he fir half of CS61, culminaing ih our udy of linear programming dualiy. HOW DO WE KNOW WHEN WE RE DONE? For example, gien a flo, ho do e kno if i a maximum flo? Any correc maximum flo algorihm mu aner hi queion, explicily or implicily. If I handed you an allegedly maximum flo, ho could I conince you ha I m no lying? I eay o conince omeone ha a flo i no maximum, ju by exhibiing a flo ih higher alue. 8 The Ford-Fulkeron algorihm coninue o erminae if edge capaciie are raional number, no necearily ineger. (Proof: caling all capaciie by a common number doen change he problem, o e can clear denominaor o reduce he raional capaciy cae o he inegral capaciy cae.) I i a bizarre mahemaical curioiy ha he Ford-Fulkeron algorihm need no erminae ih edge capaciie are irraional. 9

10 Reurning o our original example (Figure 1), anering hi queion didn eem like a big deal. We exhibied a flo of alue 5, and becaue he oal capaciy ecaping i only 5, i clear ha here can be any flo ih high alue. Bu ha abou he neork in Figure 6(a)? The flo hon in Figure 6(b) ha alue only. Could i really be a maximum flo? (1) 100 () 1 1 (1) () 1 (1) Figure 6: (a) A gien neork and (b) he alleged maximum flo of alue. We ll ackle eeral fundamenal compuaional problem by folloing a o-ep paradigm. To-Sep Paradigm 1. Idenify opimaliy condiion for he problem. Thee are ufficien condiion for a feaible oluion o be an opimal oluion. Thi ep i rucural, and no necearily algorihmic. The opimaliy condiion ary ih he problem, bu hey are ofen quie inuiie.. Deign an algorihm ha erminae ih he opimaliy condiion aified. Such an algorihm i necearily correc. Thi paradigm i a guide for proing algorihm correc. Correcne proof didn ge oo much airime in CS161, becaue almo all of hem are raighforard inducion hink of MergeSor, or Dijkra algorihm, or any dynamic programming algorihm. The harder problem udied in CS61 demand a more ophiicaed and principle approach (ih hich you ll ge pleny of pracice). So ho ould e apply hi o-ep paradigm o he maximum flo problem? Conider he folloing claim. Claim.1 (Opimaliy Condiion for Maximum Flo) If f i a flo in G uch ha he reidual neork G f ha no - pah, hen he f i a maximum flo. 10

11 Thi claim implemen he fir ep of he paradigm. The Ford-Fulkeron algorihm, hich can only erminae ih hi opimaliy condiion aified, already proide a oluion o he econd ep. We conclude: Corollary. The Ford-Fulkeron algorihm i guaraneed o erminae ih a maximum flo. Nex lecure e ll proe (a generalizaion of) he claim, derie he famou maximumflo/minimum-cu problem, and deign faer maximum flo algorihm. 11