Baic Tool CMSC April, Review Aympoic Noaion Order of Growh Recurrence relaion Daa Srucure Li, Heap, Graph, Tree, Balanced Tree, Hah Table Advanced daa rucure: Binomial Heap, Fibonacci Heap Soring Mehod Heapor, Quickor Linear ime or (bucke/couning/radix) Selecion elec he median in linear ime Running Time Problem Heap Operaion make-heap iner find-min delee-min union decreae-key delee Linked Li N N N Binary N Binomial Fibonacci Find he large ube of hee funcion ha belong o he ame equivalence cla: n lg lg n lg (n!) (lg n)! (lg n) lgn n. lg lgn i-empy amorized Algorihmic Deign Paradigm Problem Divide and Conquer Dynamic Programming Greedy Algorihm Minimum Spanning Tree Krukal and Prim Shore Pah Dijkra algorihm Bellman Ford Floyd Warhall Forre Gump i a good runner. He can run mile uninerruped bu afer ha he need o op o ea a box of chocolae. He wan o run acro America along a cerain rou. He know he locaion of he ore along he roue ha ell hi favorie chocolae. He wan o op a FEW ime a poible. Deign an algorihm for chooing where Gump hould op. Inpu: ored array A[,,,n] of diance of chocolae hop from he aring poin Aume ha he la chocolae hop i a he end of he roue
Soluion Inuiion Behind Greedy MST Greedy Work! Chooe he op ha i FARTHEST (among all poible op) from he curren op. Variable: currensop, nexsop Greedy Choice : chooe he nexsop uch ha A[nexSop] A[currenSop] <, and A[nexSop+] A[currenSop] > You need o prove ha he oluion i opimal Ue a proof by conradicion We mainain in a ube of edge A, which will iniially be empy, and we will add edge one a a ime, unil equal he MST. We ay ha a ube A E i viable if A i a ube of edge in ome MST. We ay ha an edge (u,v) E-A i afe if A {(u,v)} i viable. Baically, he choice (u,v) i a afe choice o add o ha A can ill be exended o form an MST. Noe ha if A i viable i canno conain a cycle. A generic greedy algorihm operae by repeaedly adding any afe edge o he curren panning ree. Generic-MST (G, w) Definiion. A // A rivially aifie invarian // line - mainain he invarian. while A doe no form a panning ree. do find an edge (u,v) ha i afe for A. A A {(u,v)}. reurn A // A i now a MST A cu (S, V-S) i ju a pariion of he verice ino dijoin ube. An edge (u, v) croe he cu if one endpoin i in S and he oher i in V-S. Given a ube of edge A, we ay ha a cu repec A if no edge in A croe he cu. An edge of E i a ligh edge croing a cu, if among all edge croing he cu, i ha he minimum weigh (he ligh edge may no be unique if here are duplicae edge weigh). When i an Edge Safe? MST Lemma If we have compued a parial MST, and we wih o know which edge can be added ha do NOT induce a cycle in he curren MST, any edge ha croe a repecing cu i a poible candidae. Inuiion ay ha ince all edge croing a repecing cu do no induce a cycle, hen he lighe edge croing a cu i a naural choice. Lemma: Le G = (V, E) be a conneced, undireced graph wih real-value weigh on he edge. Le A be a viable ube of E (i.e. a ube of ome MST), le (S, V-S) be any cu ha repec A, and le (u,v) be a ligh edge croing hi cu. Then, he edge i afe for A. Poin of Lemma: Greedy raegy work!
Baic of Krukal Algorihm Deecing a Cycle Aemp o add edge o A in increaing order of weigh (lighe edge fir) If he nex edge doe no induce a cycle among he curren e of edge, hen i i added o A. If i doe, hen hi edge i paed over, and we conider he nex edge in order. A hi algorihm run, he edge of A will induce a fore on he verice and he ree of hi fore are merged ogeher unil we have a ingle ree conaining all verice. We can perform a DFS on ubgraph induced by he edge of A, bu hi ake oo much ime. Ue dijoin e UNION-FIND daa rucure. Thi daa rucure uppor operaion: Creae-Se(u): creae a e conaining u. Find-Se(u): Find he e ha conain u. Union(u, v): Merge he e conaining u and v. Each can be performed in O(lg n) ime. The verice of he graph will be elemen o be ored in he e; he e will be verice in each ree of A (ored a a imple li of edge). MST-Krukal(G, w) Inuiion behind Prim Algorihm. A // iniially A i empy. for each verex v V[G] // line - ake O(V) ime. do Creae-Se(v) // creae e for each verex. or he edge of E by non-decreaing weigh w. for each edge (u,v) E, in order by non-decreaing weigh. do if Find-Se(u) Find-Se(v) // u&v on differen ree. hen A A {(u,v)}. Union(u,v). reurn A Conider he e of verice S currenly par of he ree, and i complemen (V-S). We have a cu of he graph and he curren e of ree edge A i repeced by hi cu. Which edge hould we add nex? Ligh edge! Toal running ime i O(E lg E). Baic of Prim Algorihm Implemenaion: Prioriy Queue I work by adding leave one a a ime o he curren ree. Sar wih he roo verex r (i can be any verex). A any ime, he ube of edge A form a ingle ree. S = verice of A. A each ep, a ligh edge connecing a verex in S o a verex in V- S i added o he ree. The ree grow unil i pan all he verice in V. Implemenaion Iue: How o updae he cu efficienly? How o deermine he ligh edge quickly? Prioriy queue implemened uing heap can uppor he following operaion in O(lg n) ime: Iner (Q, u, key): Iner u wih he key value key in Q u = Exrac_Min(Q): Exrac he iem wih minimum key value in Q Decreae_Key(Q, u, new_key): Decreae he value of u key value o new_key All he verice ha are no in he S (he verice of he edge in A) reide in a prioriy queue Q baed on a key field. When he algorihm erminae, Q i empy.
MST-Prim(G, w, r) Analyi of Prim. Q V[G]. for each verex u Q // iniializaion: O(V) ime. do key[u]. key[r] // ar a he roo. π[r] NIL // e paren of r o be NIL. while Q // unil all verice in MST. do u Exrac-Min(Q) // verex wih lighe edge. for each v adj[u]. do if v Q and w(u,v) < key[v]. hen π[v] u. key[v] w(u,v) // new ligher edge ou of v. decreae_key(q, v, key[v]) Exracing he verex from he queue: O(lg n) For each inciden edge, decreaing he key of he neighboring verex: O(lg n) where n = V The oher ep are conan ime. The overall running ime i, where e = E and n = V, T(n) = u V (lg n + deg(u)lgn) = u V (+ deg(u)) lg n = lg n (n + e) = O((n + e)lgn) Eenially ame a Krukal : O((n+e) lg n) ime Can be made o run faer in O(n lg n) ime How? Problem There ha been a lo of hype recenly abou The Marix Reloaded wih he releae of new hearical railer. For hi problem, uppoe you are managing he conrucion of billboard on he Neo Memorial Highway, a heavily-raveled rech of road ha run we-ea for M mile. The poible ie for billboard are given by number x x, x n, each in he inerval [.M] (pecified by heir poiion along he highway meaured in mile from i weern end). If you place a billboard a locaion x i, you receive a revenue of r i > You wan o place billboard a a ube of he ie in x x, x n o a o maximize your oal revenue, ubjec o he following rericion: DP Soluion OPT[i] = he maximum revenue one can receive given he conrain and elecing a ube of he ie beween x and x i Noice ha if we don build a x i, we imply wan he be hing ha happened o far, i.e. OPT[i ] If we do build a x i, we wan he be hing ha happened up o : x i, i.e. or OPT[j] + r i where x j < x i Recurrence: OPT[i] = max OPT[i ] OPT[j] + r i where </ j </ i x j < x i Running ime?. Environmenal Conrain. You canno build wo billboard wihin le han mile of one anoher on he highway. Definiion for Shore Pah Single-Source Shore Pah Think of verice a ciie and he edge weigh a he diance from one ciy o anoher. Define he lengh of a pah o be he um of edge weigh along he pah. Define he diance beween wo verice, u and v, δ(u,v) o be he lengh of he minimum lengh pah from u o v. Given a direced graph G = (V, E) wih edge weigh and a diinguihed ource verex, V, deermine he diance from he ource verex o every verex in he graph. BFS find hor-pah from a ingle ource verex o all oher verice in O(n+e) ime, auming he graph ha no edge weigh. Edge weigh can be negaive; bu in order for he problem o be well-defined here mu be no cycle whoe oal co i negaive. A hore pah from verex u o verex v i hen defined a any pah p wih weigh w(p) = δ(u,v).
Relaxaion Relax (u, v) Mainain an eimae of he hore pah for each verex v, call i d[v]. Iniially d[v] will be he lengh of he hore pah ha he algorihm of know from o v. Thi value will alway be greaer han or equal o he rue hore pah diance from o v. Iniially, we know of no pah, o all d[v]= & d[]=. A he algorihm goe on and ee more verice, i rie o updae d[v] for each verex in he graph, unil all d[v] value converge o rue hore diance.. if d[u] + w(u,v) < d[v] // i he pah hru u horer?. hen d[v] d[u] + w(u,v); // ye, hen ake i.. π[v] u; // he hore way back o he ource i hrough u by updaing he predeceor poiner NOTE: If we perform Relax (u, v) repeaedly over all edge of he graph, all he d[v] value will evenually converge o he rue final diance value from. How o do hi mo efficienly? Dijkra Algorihm Dijkra(G, w, ) Mainain a ube of verice, S V, for which we know heir rue diance d[u]=δ(,u). Iniially S = and we e d[]= and all oher o. One by one we elec verice from V - S o add o S. For each verex u V-S, we have compued a diance eimae d[u]. The greedy approach i o ake he verex for which d[u] i minimum, i.e. ake unproceed verex ha i cloe o. We ore he verice of V - S in a prioriy queue (heap), where he key value of each verex u i d[u]. All operaion can be done in O(lg n) ime.. Q V[G] and S. for each verex u Q // iniializaion: O(V) ime. do d[u] and π[u] NIL. d[] // ar a he ource. π[] NIL // e paren of o be NIL. while Q // ill all verice proceed. do u Exrac-Min(Q) // elec cloe o S S {u}. for each v adj[u]. do if v Q and (d[u] + w(u,v) < d[v]). hen π[v] u. d[v] d[u] + w(u,v) // Relax (u,v). decreae_key(q, v, d[v]) More on Shore Pah Problem Running ime of Dijkra Bellman Ford Deec negaive weigh cycle Floyd Warhall All pair Shore Pah Dynamic Programming Algorihm There are n ciie conneced by m bue. For each bu, we are given i aring ciy, erminu ciy, aring ime and ime a which i reache he erminu. The bu doe no op anywhere in he middle, o you may board i only from he aring erminu. You ar a ciy a AM and wan o find he earlie ime you can reach a ciy X uing any number of bu op. Decribe a polynomial ime algorihm o accomplih hi.
Soluion Max Flow Nework Conruc a graph G Verice repreen ciie Edge repreen bu roue Can we ue Bellman Ford o find he earlie ime o reach any ciy from ciy in polynomial ime? Need o modify he relaxaion procedure Max flow nework: G = (V, E,,, u). (V, E) = direced graph, no parallel arc. Two diinguihed node: = ource, = ink. u(e) = capaciy of arc e. RELAX(Ciy u, Ciy v){ if d[u] < arime(u,v) and d[v] > endime(u,v) hen d[v] := endime(u,v) } where, d[v] i he earlie ime o far o reach ciy u from ciy. Capaciy Flow Flow An - flow i a funcion f: E R ha aifie: For each e E: f(e) u(e) (capaciy) For each v V {, }: f ( e) (conervaion) e in o v e ou of v An - flow i a funcion f: E R ha aifie: For each e E: f(e) u(e) (capaciy) For each v V {, }: f ( e) (conervaion) e in o v ou of v e MAX FLOW: find - flow ha maximize ne flow ou of he ource. f e ou of Capaciy Flow Capaciy Flow Value = Flow Flow An - flow i a funcion f: E R ha aifie: For each e E: f(e) u(e) (capaciy) For each v V {, }: f ( e) (conervaion) e in o v e ou of v MAX FLOW: find - flow ha maximize ne flow ou of he ource. An - flow i a funcion f: E R ha aifie: For each e E: f(e) u(e) (capaciy) For each v V {, }: f ( e) (conervaion) e in o v e ou of v MAX FLOW: find - flow ha maximize ne flow ou of he ource. Capaciy Flow Value = Capaciy Flow Value =
Cu Cu An - cu i a node pariion (S, T) uch ha S, T. The capaciy of an - cu (S, T) i: u( e) : = u( v, w). e ou of S ( v, w) E v S, w T Min - cu: find an - cu of minimum capaciy. An - cu i a node pariion (S, T) uch ha S, T. The capaciy of an - cu (S, T) i: u( e) : = u( v, w ). e ou of S ( v, w ) v S E, w T Min - cu: find an - cu of minimum capaciy. Capaciy = Capaciy = Cu Max Flow and Min Cu An - cu i a node pariion (S, T) uch ha S, T. The capaciy of an - cu (S, T) i: u( e) : = u( v, w). e ou of S ( v, w ) E v S, w T Min - cu: find an - cu of minimum capaciy. Corollary. Le f be a flow, and le (S, T) be a cu. If f = cap(s, T), hen f i a max flow and (S, T) i a min cu. Capaciy = Cu capaciy = Flow value = Max-Flow Min-Cu Theorem Max-Flow Min-Cu Theorem MAX-FLOW MIN-CUT THEOREM (Ford-Fulkeron, ): In any nework, he value of he max flow i equal o he value of he min cu. Augmening pah heorem (Ford-Fulkeron, ): A flow f i a max flow if and only if here are no augmening pah. Cu capaciy = Flow value = MAX-FLOW MIN-CUT THEOREM (Ford-Fulkeron, ): he value of he max flow i equal o he value of he min cu. We prove boh imulaneouly by howing he TFAE: (i)f i a max flow. (ii)there i no augmening pah relaive o f. (iii)there exi a cu (S, T) uch ha f = cap(s, T).
Chooing Good Augmening Pah Shore Augmening Pah Ue care when elecing augmening pah. Some choice lead o exponenial algorihm. Clever choice lead o polynomial algorihm. Goal: chooe augmening pah o ha: Can find augmening pah efficienly. Few ieraion. Edmond-Karp (): chooe augmening pah wih Max boleneck capaciy. (fa pah) Sufficienly large capaciy. (capaciy-caling) Fewe number of arc. (hore pah) Inuiion: chooing pah via breadh fir earch. Eay o implemen. may implemen by coincidence! Find augmening pah wih fewe number of arc. ShoreAugmeningPah(V, E,, ) FOREACH e E f(e) G f reidual graph WHILE (here exi augmening pah) find uch a pah P by BFS f augmen(f, P) updae G f RETURN f Hiory Year... Dicoverer Danzig Mehod Simplex Big-Oh mn U Ford, Fulkeron Augmening pah mnu Edmond-Karp Shore pah m n Diniz Shore pah mn Edmond-Karp, Diniz Capaciy caling m log U Diniz-Gabow Capaciy caling mn log U Karzanov Preflow-puh n Sleaor-Tarjan Dynamic ree mn log n Goldberg-Tarjan FIFO preflow-puh mn log (n / m)......... Goldberg-Rao Lengh funcion m / log (n / m) log U mn / log (n / m) log U