Remainder: Shortest Paths Maximum Flows Discrete Optimization 2010 Lecture 3 Maximum Flows Marc Uetz University of Twente m.uetz@utwente.nl Lecture 3: sheet 1 / 29 Marc Uetz Discrete Optimization
Outline Remainder: Shortest Paths Maximum Flows 1 Remainder: Shortest Paths Acyclic Graphs Bellman Equations 2 Maximum Flows Augmenting Path Algorithms Ford-Fulkerson Edmonds-Karp Max-Flow Min-Cut Lecture 3: sheet 2 / 29 Marc Uetz Discrete Optimization
Remainder: Shortest Paths Maximum Flows Acyclic G Bellman Eqns Computation Time Dijkstra Simple Initialization O( n ) n iterations of while-loop, in each need to find smallest label in V \ S, which is doable in O( n ) O( n 2 ) in total m relabeling steps of O( 1 ) O( m ) Which gives O( n ) + O( n 2 ) + O( m ) O( n 2 ) time Less simple An O( m log n ) implementation, which uses priority queue (also called heap) to manage finding minimal d(v) in O( 1 ) time. (log n comes from overhead in organization of the heap) Lecture 3: sheet 3 / 29 Marc Uetz Discrete Optimization
Remainder: Shortest Paths Maximum Flows Acyclic G Bellman Eqns BFS - breadth-first-search Let all arc lengths c vw = 1. What about Dijkstra s algorithm? distance labels d(v) = minimal # arcs to reach v from s (shortest path tree = BFS tree) d(v) = for non-reachable v ( reachability algorithm ) Computation time? once assigned (finite) value, no node needs relabeling (otherwise contradiction to choice of v = argmin{d(v)}) hence organize such nodes as a FIFO list Q (queue): when d(w) is assigned a (finite) value, w end of Q then minimal d(v) is found in O( 1 ) time, at top of Q Total time O( n + m ) Lecture 3: sheet 4 / 29 Marc Uetz Discrete Optimization
Remainder: Shortest Paths Maximum Flows Acyclic G Bellman Eqns Acyclic Graphs Definition!"#"$"%&'($)*+,-+&.% A digraph G = (V, A), V = n, has a linearization if there exists bijection!"#$%&'&(")$*+$(&,)'-.&!/01234&5"+.&%&%6($7&'%(&8&')*79&:.$%&+.$ π : V {1,..., n} such that %;8<$)"%,&!=&1&!>?2@2%A&"7&'&!"#"$"%&'($)"*+,*!"#"$"%&'($)"*+,* 6B&+.$&%6($7&"B& π(v) <! π(w) 0-2.4&" 3=&! for all(v, 0-4C! 0.4 w) A!0#4! " $ # % &'()*(+,-./0*12'-3456789-'1:-1-;!"!#!$%&'#(!)*+) /<- all arcs from left to right 1=>-)=?@-/<-3-A)=;1/=:-=)->/*(A;(>-A@A?(: Lecture 3: sheet 5 / 29 Marc Uetz Discrete Optimization
Remainder: Shortest Paths Maximum Flows Acyclic G Bellman Eqns Acyclic Graphs Theorem A graph has a linearization if and only if it has no (directed) cycle. only if a cycle (v, w,..., v) yields π(v) < π(w) < < π(v) if acyclic graphs have a node v without ingoing arc (why?) let π(v) = 1, delete all arcs (v, w), by induction, can get a linearization on G v (with numbers 2,..., n) Lecture 3: sheet 6 / 29 Marc Uetz Discrete Optimization
Remainder: Shortest Paths Maximum Flows Acyclic G Bellman Eqns Linear Time Algorithm Theorem Shortest paths from some source node s to all other nodes v can be computed in time O( n + m ) for acyclic graphs. Note: best possible, as encoding length of a graph is in Ω( n + m ) Idea Like in Dijkstra s algorithm, make sure to make only one relabeling per edge. Namely, take a linearization of G, and in iteration k correct -inequality for all outgoing arcs of node π(k), k = 1,..., n Easy inductive proof: In iteration k, the labels of nodes π(1),..., π(k) are correct, since there are no backward arcs Thereby, each node & arc touched only once, O( n + m ) Lecture 3: sheet 7 / 29 Marc Uetz Discrete Optimization
Remainder: Shortest Paths Maximum Flows Acyclic G Bellman Eqns Bellman Equations & Shortest Path Lengths Definition ( ) Given digraph G = (V, A), s V, consider system of equations d(s) = 0, d(w) = min{d(v) + c vw (v, w) A} w s Solution of ( ) as linear program: Find maximal solution d of ( ) maximize v V d(v) s.t. d(w) d(v) + c vw d(s) = 0 w V \ s and (v, w) A Claim: Solution of this LP are shortest path lengths. Lecture 3: sheet 8 / 29 Marc Uetz Discrete Optimization
Outline 1 Remainder: Shortest Paths Acyclic Graphs Bellman Equations 2 Maximum Flows Augmenting Path Algorithms Ford-Fulkerson Edmonds-Karp Max-Flow Min-Cut Lecture 3: sheet 9 / 29 Marc Uetz Discrete Optimization
!"#$%&%'()*+',-*.)/% Maximum Flow Problem Remainder: Shortest Paths Maximum Flows Augmenting Path Algorithms Max-Flow Min-Cut! Any!"#$%&'(")*+,-'!./01231'+*4'4+,+4"5"$6'7&'2! instance given by 8'/4+99$:&'!"#$%&'!"#$%&'3! 5;<':$6")%+5$:'%<:$6'(1'# " 0'/6<7*4$'+%:'5+*)$53 digraph G = (V, A), arc capacities u : A N (N important) (G, u) = network!!<+9&'=$%:'>+?">7>'@9<;'@*<>'( two designated nodes, s and t 5<'#1'*$6,$45"%)'+*4'4+,+4"5"$6 goal: send maximum flow from s to t numbers at arcs : s=1 (3,6) (2,6) (1,1) 2 5 3 (1,3) (3,3) ( flow, capacity (1,2) (3,7) 4 (4,4) (1,5) t=6 ) (s, t)-flow Lecture 3: sheet 10 / 29 Marc Uetz Discrete Optimization
Linear Programming Formulation x ij = amount of flow through arc (i, j) max v(x) := x sj s.t. j:(s,j) A x ij j:(i,j) A j:(j,i) A j:(j,s) A x js x ji = 0 i s, t (1) 0 x ij u ij (i, j) A (2) v(x) = value of the flow = flow out of s (= flow into t) (1) = flow balance constraints at intermediate nodes (2) = capacity constraints along arcs Lecture 3: sheet 11 / 29 Marc Uetz Discrete Optimization
Feasible Flow Problem Assume given b(i) = required outflow (excess) at node i x ij = amount of flow through arc (i, j) Question: Is there a feasible flow x with x ij x ji = b(i) i V j:(i,j) A j:(j,i) A 0 x ij u ij (i, j) A b(i) > 0 excess node b(i) < 0 deficit node if all b(i) = 0 problem is called circulation problem Lecture 3: sheet 12 / 29 Marc Uetz Discrete Optimization
Augmenting Paths (0,1) 3 (0,2) s=1 (0,3) (0,2) 2 3 s=1 2 2 2 (0,1) 2 t=4 t=4 flow augmentation along path (1, 2, 3, 4), by 2 units Flow augmentation over th 1,2,3,4 with 2 units. Lecture 3: sheet 13 / 29 Marc Uetz Discrete Optimization
Augmenting Paths s=1 (0,1) (2,2) 1 3 2 3 s=1-1 2 (2,3) (2,2) (0,1) 1 t=4 t=4 Flow after augmentation o the path 1,2,3,4 with 2 uni flow augmentation along path (1, 3, 2, 4), by 1 unit note: thereby decrease flow value along (2, 3) Flow augmentation over t 1,3,2,4 with 1 unit: decrea arc (2,3) in the wrong d Lecture 3: sheet 14 / 29 Marc Uetz Discrete Optimization
Augmenting Paths!"#$%&'(&#)*+',- (1,1) 3 (2,2) s=1 (2,2) 2 (1,3) (1,1) t=4 No further improvements possible. Lecture 3: sheet 15 / 29 Marc Uetz Discrete Optimization
Residual Graph G(x)!"#$%&'()*"+,-./)!0"1 s=1 (0,1) (2,2) 3 2 (2,3) (2,2) (0,1) t=4 Consider flow x over path 1,2,3,4 with 2 units. (x 12 = x 23 = x 34 = 2, x ij = 0 otherwise) Residual Network G(x): Represents all possibilities to change the flow on top of given flow x s=1 1 2 1 3 2 2 2 1 t=4 Lecture 3: sheet 16 / 29 Marc Uetz Discrete Optimization
!"#$%&'()*"+,-./)!"#$ Residual Graph G(x) Remainder: Shortest Paths Maximum Flows Augmenting Path Algorithms Max-Flow Min-Cut For a given flow x, the network G contains arcs over which the flow Definition can be increased and/or decreased. Arcs over which the flow Forcan anybe given decreased (feasible) are flow reversed x in G, with define capacity a network equal G(x) to the amount forward of possible arc: possible decrease. amount of flow increase backward arc: possible amount of flow decrease i (x ij, u ij ) j i x ij < u ij u ij -x ij j i x ij > 0 x ij j Note: any possible flow increase can be achieved by finding Note: directed Anypaths augmenting the residual path for network flow x is a directed augmenting path paths. in G(x) Lecture 3: sheet 17 / 29 Marc Uetz Discrete Optimization
Generic Augmenting Path Algorithm (Ford-Fulkerson) Algorithm 1: Ford-Fulkerson input : network (G, u) with capacities u 0, s, t V output: x = maximum (s, t)-flow let x = 0 and G(x) = G; while (G(x) contains (s, t)-path P) do determine smallest residual capacity ū on P [ 1]; x P = flow of value ū along P (so, x a = ū a P); augment flow x = x + x P ; update residual capacities, i.e., re-compute G(x); Theorem An (s, t)-flow x is maximum if and only if the residual graph G(x) does not have any (s, t)-path Lecture 3: sheet 18 / 29 Marc Uetz Discrete Optimization
Proof of Augmenting Path Theorem Necessity If G(x) has an (s, t)-path, could augment flow x along that path (by at least one unit), contradiction. Sufficiency have: (s, t)-path in G(x), assume flow x with v(x ) > v(x) Consider flow x x, can show: feasible flow in G(x) By assumption v(x x) = v(x ) v(x) > 0, so x x has flow out of s By flow balance, must reach t on some path(s) So we have found an (s, t) path in G(x), contradiction Lecture 3: sheet 19 / 29 Marc Uetz Discrete Optimization
Computation Time Ford-Fulkerson Algorithm 3 10 6 10 6 s=1 1 t=4 10 6 2 10 6 2!"#$%%$&'"(%&)"*+,#-'.*.$&'/")0-'"/-%-1.$',"23*45 10 6 flow augmentations if we are unlucky 6*.0/ [for u a R, might even fail to converge to optimum value, there s a paper by Uri Zwick (1995) containing the smallest example] 7&%+.$&'/8"90&&/- :; 70&<.-/. *+,#-'.$', 6*.0/ =#$'$#+#"'+#3-< &("*<1/> Possible!;?*@$#+#"1*6*1$.A"*+,#-'.$',"6*.0/ Solutions shortest augmenting paths (minimal # arcs) maximum capacity augmenting paths Lecture 3: sheet 20 / 29 Marc Uetz Discrete Optimization
Edmonds-Karp Augmenting Path Algorithm Algorithm 2: Edmonds-Karp Use Ford-Fulkerson augmenting path algorithm But always use shortest augmenting paths (# arcs) How to find shortest augmenting paths? Needed: shortest (s, t)-path in G(x), in terms of # arcs Problem: after flow augmentation, arcs deleted/added to G(x) Brute Force Solution After flow augmentation and update of G(x), make BFS in G(x) Denote by d(v) these distances (from source s), v V Note, if t is no longer reachable (d(t) = ), we are done Lecture 3: sheet 21 / 29 Marc Uetz Discrete Optimization
Analysis Edmonds-Karp Algorithm Lemma 1 At least one arc is removed from G(x) in each iteration (but may come back in a later iteration) Proof: One arc (at least) is saturated, thus disappears in G(x) P Lemma 2 Distance labels d(v) never decrease u v Lemma 3 Each arc is removed at most O( n ) times Lecture 3: sheet 22 / 29 Marc Uetz Discrete Optimization
Lemma 2: d(u) can only increase P s u v w An arc removal cannot decrease any d(u), but arc addition? Say, arc (v, u) is added (backward along augmenting path P) (let u be first node with this property along P) creating a new (s, u)-path via w but d(w) d(u), because P was shortest path before so d(u) cannot decrease by this new path Lecture 3: sheet 23 / 29 Marc Uetz Discrete Optimization
Lemma 3: Any arc is removed at most O( n ) times P Let d k (v) be the distance label before iteration k consider arc (u, v), removed in some iteration k it holds d k (v) = d k (u) + 1, because P shortest path if (u, v) is removed in a later iteration again, it first must come back, say in iteration h > k it holds d h (u) = d h (v) + 1 by Lemma 2, d h (u) = d h (v) + 1 d k (v) + 1 = d k (u) + 2 but n 1 is an upper bound on any label u v Lecture 3: sheet 24 / 29 Marc Uetz Discrete Optimization
Edmonds-Karp: Computation Time Claims 1 O( nm ) augmentations 2 Each augmentation in O( m ) Hence total computation time is O( m 2 n ) Claim 1 In each augmentation at least one arc is removed Claim 2 No arc is removed more than O( n ) times In total at most O( nm ) augmentations Each augmentation is one BFS from s, takes O( m ) time Remarks: Improvement to O( n 2 m ) is possible ( Literature) Algorithm (& runtime analysis) works for any u a R Lecture 3: sheet 25 / 29 Marc Uetz Discrete Optimization
Cuts in Networks!"#$%&'%()#*+,-$ Definitions An (s,t)-cut is a partition [S,T] of V, such that s!sand t!t, S!T = ". An (s, t)-cut is a partition [S, T ] of V, such that s S, t T The arcs in the cut [!"#] [S, are T ] those are those with the with tail tail in S inand S, the head in T The capacity u(s, T ) of a cut [S, T ] is u(s, T ) = head in T. a [S,T ] u a The capacity of the cut [S,T], denoted by u(s,t) is the cumulative capacity of the arcs in the cut, " a#[s,t] u a Cut [{1,2,3},{4,5,6}] Capacity: 3+2+3=8 s=1 (3,6) (1,1) (1,3) 2 5 (1,2) (0,2) (3,7) (4,4) t=6 (2,6) 3 (3,3) 4 (1,5) Lecture 3: sheet 26 / 29 Marc Uetz Discrete Optimization
Max-Flow Min-Cut Theorem(s) Weak Duality Theorem The value of any flow is at most equal to the capacity of any cut. So for any flow x and any cut [S, T ], v(x) u(s, T ). Strong Duality Theorem There is a flow x and a cut [S, T ] such that v(x) = u(s, T ). Lecture 3: sheet 27 / 29 Marc Uetz Discrete Optimization
Proof: Weak Duality Take any (s, t)-flow x and any (s, t)-cut [S, T ], then v(x) = x sj j:(s,j) A j:(j,s) A j:(s,j) A j:(j,s) A x js = x sj x js + = x ij i S j:(i,j) A = x ij (i,j) [S,T ] j:(j,i) A (j,i) [T,S] x ji i S\s x ji x ij u ij = u(s, T ) (i,j) [S,T ] (i,j) [S,T ] x ij j:(i,j) A j:(j,i) A x ji Lecture 3: sheet 28 / 29 Marc Uetz Discrete Optimization
Proof: Strong Duality!"#$%&'()*+,"- Take a maximum (s, t)-flow x, with value v(x), we construct a cut with Take theany same maximum capacity. (s,t) flow x in the network, with value v(x). We construct a cut [S,T] with the same capacity: Let S be the set of vertices Let S be the set of vertices reachable from s in residual graph reachable from s in the residual graph G(x). Let T = V!S. G(x), We know T = V t!s, \ S. by the previous Theorem (! augmenting path). Know t T by previous Theorem ( augmenting (s, t)-path) S s G(x) t T # # ( i, j) $ [ S, T ] ( i, j) $ [ T, S ] x x ij ij " u " 0 ij Hence, flow from S to T is u(s, T ), by flow balance = v(x) Hence, flow from S to T equals cut capacity, u(s,t), and by flow conservation, v(x) equals the flow from S to T. Lecture 3: sheet 29 / 29 Marc Uetz Discrete Optimization