Lecture. Single-Source Shortest Paths All-Pairs Shortest Paths T. H. Cormen, C. E. Leiserson and R. L. Rivest Introduction to, rd Edition, MIT Press, 009 Sungkyunkwan University Hyunseung Choo choo@skku.edu Copyright 000-0 Networking Laboratory
Single-Source Shortest Path Problem: Given a weighted directed graph G, find the minimum-weight path from a given source vertex s to another vertex v Shortest-path minimum weight Weight of path is sum of edges e.g., a road map what is the shortest path from Chapel Hill to Charlottesville? Networking Laboratory /9
Shortest Path Properties Again, we have optimal substructure the shortest path consists of shortest subpaths: Proof: suppose some subpath is not a shortest path There must then exist a shorter subpath It could substitute the shorter subpath for a shorter path But then overall path is not shortest path Contradiction Networking Laboratory /9
Shortest Path Properties Define δ(u,v) to be the weight of the shortest path from u to v Shortest paths satisfy the triangle inequality δ(u,v) δ(u,x) + δ(x,v) Proof: x u v This path is no longer than any other path Networking Laboratory /9
Relaxation A key technique in shortest path algorithms is relaxation Idea: for all v, maintain upper bound d[v] on δ(s,v) Relax(u,v,w) { if (d[v] > d[u]+w) then d[v]=d[u]+w; } 9 Relax Relax Networking Laboratory /9
Bellman-Ford Algorithm BellmanFord() for each v V d[v] = ; d[s] = 0; for i= to V - for each edge (u,v) E Relax(u,v, w(u,v)); for each edge (u,v) E if (d[v] > d[u] + w(u,v)) return no solution ; Initialize d[], which will converge to shortest-path value δ Relaxation: Make V - passes, relaxing each edge Test for solution Under what condition do we get a solution? Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w Networking Laboratory /9
Bellman-Ford Algorithm e.g. s s 0 0 8 8 t y t - - 9 - x z y - 9 - - x z s s 0 0 8 8 t y t - 9 - - x z y - 9 - - x z s 0 8 t y - 9 - - x z Networking Laboratory /9
Practice Problems Given a directed graph as bellow. Suppose that in Bellman-Ford algorithm all the edges will be relaxed in the following order: BC AC BA SA SB. Fill in the table with the distance estimates for each vertex after each iteration. Vertex S A B C Initial 0 Iteration Iteration Iteration Iteration Networking Laboratory 8/9
Dijkstra s Algorithm Input A digraph G(V,E) where edges are associated with non-negative weight (cost) and a source src Output Lengths of shortest paths from src to each node in G Idea : without loss of general V = {,,, n } where is a source node We have two sets S : set of nodes already chosen ( S = { } ) C : set of remaining node ( C = {,,, n } ) D[,,, n] : containing costs of shortest path Networking Laboratory 9/9
Dijkstra s Algorithm Repeatedly add a node v in C to S whose distance to (source) is minimal until S = {,,, n} Dijkstra ( L[,,n,,,n] ) /* L is cost array, L[i,j] : cost if (i,j) in E or : infinity if (i,j) is not in E */ C <- {,,, n } for i <- to n do D[i] <- L[,i] repeat (n-) times v <- a node in C s.t. D[v] = Min{ D[w] } for each w in C C <- C {v} for each w in C do D[w] <- Min{ D[w], D[v]+L[v,w] } return D Networking Laboratory 0/9
Dijkstra s Algorithm e.g. 0 0 00 0 0 V = E = 8 0 0 Step v C D 0 -- {,,, } {,, } {, } { } 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 Networking Laboratory /9
Dijkstra s Algorithm e.g. 0 u v 0 u v 8 0 u v 8 9 s 0 9 x y s 0 9 x y 0 9 x y 0 u v 0 0 u v 8 0 u v 8 9 s 0 9 s x y 0 9 x y 0 9 x y Networking Laboratory /9
Dijkstra s Algorithm e.g. Networking Laboratory /9
Minimum Steiner Tree The Steiner problem in graphs (NP-complete) Given graph G(V,E) A subset S of V Find a subgraph the minimum cost among all connected subgraphs subgraphs contain S It is evident that the subgraph is a solution of this problem must be a tree We briefly call it an optimal tree V =n, S =k (k>) Shortest path problem when k= Minimum-cost Spanning Tree problem when k=n Networking Laboratory /9
Minimum Steiner Tree Multicasting It refers to the transmission of data from one node to a selected group of nodes TM Algorithm Hiromitsu Takahashi and Akira Matsuyama AN APPROXIMATE SOLUTION FOR THE STEINER PROBLEM IN GRAPHS Math. Japonica, vol., no., pp. -, 980. Pseudo Code Networking Laboratory /9
Minimum Steiner Tree TM e.g. Source Member Member Networking Laboratory /9
Minimum Steiner Tree Self-study: KMB Algorithm L. Kou, G. Markowsky, and L. Berman A Fast Algorithm for Steiner Trees Acta Informatica, vol., pp. -, 98. Source Source Source Member Member Member Member Member Member Networking Laboratory /9
Shortest Path Problems Input: Directed graph G = (V, E) Weight function w : E R Weight of path p = v 0, v,..., v k w( p) = k i= w( v i, v i ) Shortest-path weight from u to v s 0 9 p δ(u, v) = min w(p) : u v if there exists a path from u to v t y x z otherwise Shortest path from u to v is any path p such that w(p) = δ(u, v) Networking Laboratory 8/9
Shortest Path Representation d[v] = δ(s, v): a shortest path estimate Initially, d[v]= Reduces as algorithms progress π[v] = predecessor of v on a shortest path from s If no predecessor, π[v] = NIL π induces a tree : shortest path tree Shortest paths & shortest path trees are not unique s 0 t 9 y x z Networking Laboratory 9/9
Relaxation Relaxing an edge (u, v) Testing whether we can improve the shortest path to v found so far by going through u If d[v] > d[u] + w(u, v) we can improve the shortest path to v update d[v] and π[v] s u v 9 s u v RELAX(u, v, w) RELAX(u, v, w) u v u v Networking Laboratory 0/9
Bellman-Ford Algorithm t Computes d[v] and π[v] for all v V s - - 0 8 y 9 TRUE if no negative-weight cycles are reachable from the source s Single-source shortest paths problem It allows negative edge weights Returns: FALSE otherwise no solution exists Idea: Traverse all the edges V times, every time performing a relaxation step of each edge - x z Networking Laboratory /9
Dijkstra s Algorithm Single-source shortest path problem: No negative-weight edges: w(u, v) > 0 (u, v) E Maintains two sets of vertices: 0 u v 8 9 S = vertices whose final shortest-path weights have already been determined C = vertices in V S x Repeatedly select a vertex u V S, with the minimum shortest-path estimate d[v] 0 9 y Networking Laboratory /9
Practice Problems Is the following algorithm a valid method for finding the shortest path from node S to node T in a directed graph with some negative edges? add a large constant to each edge weight so that all the weights become positive then run Dijkstra s algorithm starting at node S, and return the shortest path found to node T. Networking Laboratory /9
All-Pairs Shortest Paths - Solutions If we run Bellman-Ford once from each vertex: O(V E), which is O(V ) if the graph is dense (E = Θ(V )) If no negative-weight edges, we could run Dijkstra s algorithm once from each vertex: O(VElgV) with binary heap, O(V lgv) if the graph is dense O(EV + V lgv) with Fibonacci heap, O(V ) if the graph is dense We ll see how to do in O(V ) in all cases, with no fancy data structure Networking Laboratory /9
All-Pairs Shortest Paths (APSP) Given: Directed graph G = (V, E) Weight function w : E R Compute: The shortest paths between all pairs of vertices in a graph Representation of the result: an n n matrix of shortest-path distances δ(u, v) 8 - - Networking Laboratory /9
All-Pairs Shortest Paths Assume the graph G is given as adjacency matrix of weights W = (w ij ), n x n matrix, V = n Vertices numbered to n w ij = 0 weight of (i, j) if i = j if i j, (i, j) E if i j, (i, j) E Output the result in an n x n matrix D = (d ij ), where d ij = δ(i, j) Solve the problem using dynamic programming - 8 - Networking Laboratory /9
Optimal Substructure of a Shortest Path All subpaths of a shortest path at most m edges are shortest paths Let p: a shortest path p from i k j vertex i to j that contains at most m edges at most m - edges If i = j w(p) = 0 and p has no edges If i j: p = i p k j p has at most m- edges p is a shortest path δ(i, j) = δ(i, k) + w kj Networking Laboratory /9
Recursive Solution l (m) ij : weight of shortest path i that contains at most m edges j at most m edges m = 0: l ij (0) = 0 if i = j if i j i k j m : l ij (m) = min {, min { l (m-) ik + w kj } } l ij (m-) k n = min { l ik (m-) + w kj } k n Shortest path from i to j with at most m edges Shortest path from i to j containing at most m edges, considering all possible predecessors (k) of j Networking Laboratory 8/9
Computing the Shortest Paths m = : l ij () = The path between i and j is restricted to edge L () = W w ij Given W = (w ij ), compute: L (), L (),, L (n-), where L (m) = (l (m) ij ) L (n-) contains the actual shortest-path weights Given L (m-) and W compute L (m) Extend the shortest paths computed so far by one more edge If the graph has no negative cycles: all simple shortest paths contain at most n - edges δ(i, j) = l (n-) ij and l (n) ij, l (n+) ij... l (n-) ij Networking Laboratory 9/9
Extending the Shortest Paths k l ij (m) = min { l ik (m-) + w kj } k n j j i k = i L (m-) n x n W L (m) Replace: min + + Computing L (m) looks like matrix multiplication Networking Laboratory 0/9
EXTEND(L, W, n) l (m) ij = min { l (m-) ik + w kj } k n. create L, an n n matrix. for i to n. do for j to n. do l ij. for k to n. do l ij min ( l ij, l ik + w kj ). return L Running time: Θ(n ) Networking Laboratory /9
SLOW-APSP(W,n) SLOW-ALL-PAIRS-SHORTEST PATHS(W, n). L () W. for m to n-. do L (m) EXTEND ( L (m - ), W, n ). return L (n - ) Running time: Θ(n ) Networking Laboratory /9
Example - 8 - W L (m-) = L () L (m) = L () 0 8-0 0-0 0 0 8-0 0-0 0 l ij (m) = min { l ik (m-) + w kj } k n and so on until L () 0 8-0 - 0 - - 0-8 0 Networking Laboratory /9
Improving Running Time No need to compute all L (m) matrices If no negative-weight cycles exist: L (m) = L (n - ) for all m n We can compute L (n-) by computing the sequence: L () = W L () = W = W W L () = W = W W L (8) = W 8 = W W x = n ( ) lg( n ) L n = W Networking Laboratory /9
FASTER-APSP(W, n). L () W. m. while m < n -. do L (m) EXTEND(L (m), L (m), n). m m. return L (m) OK to overshoot: products don t change after L (n - ) Running Time: Θ(n lg n) Networking Laboratory /9
The Floyd-Warshall Algorithm Given: Directed, weighted graph G = (V, E) Negative-weight edges may be present No negative-weight cycles could be present in the graph Compute: The shortest paths between all pairs of vertices in a graph 8 - - Networking Laboratory /9
The Structure of a Shortest Path Vertices in G are given by V = {,,, n} Consider a path p = v, v,, v l An intermediate vertex of p is any vertex in the set {v, v,, v l - } e.g.: p =,,, : {, } 0. p =,, : {} Networking Laboratory /9
The Structure of a Shortest Path For any pair of vertices i, j V, consider all paths from i to j whose intermediate vertices are all drawn from a subset {,,, k} Find p, a minimum-weight path from these paths p i p u j p t No vertex on these paths has index > k Networking Laboratory 8/9
Example d (k) ij = the weight of a shortest path from vertex i to vertex j with all intermediary vertices drawn from {,,, k} d (0) = d () = d () = 0. d () = d () =. Networking Laboratory 9/9
The Structure of a Shortest Path k is not an intermediate vertex of path p Shortest path from i to j with intermediate vertices from {,,, k} is a shortest path from i to j with i k j intermediate vertices from {,,, k - } p k is an intermediate vertex of path p k is not intermediary vertex of p, p p is a shortest path from i to k with vertices from {,,, k - } i p k p p j p is a shortest path from k to j with vertices from {,,, k - } Networking Laboratory 0/9
A Recursive Solution d ij (k) = the weight of a shortest path from vertex i to vertex j with all intermediary vertices drawn from {,,, k} k = 0 d ij (k) = w ij k Case : k is not an intermediate vertex of path p d ij (k) = d ij (k-) i k j Case : k is an intermediate vertex of path p d ij (k) =d ik (k-) + d kj (k-) i k j Networking Laboratory /9
Computing The Shortest Path Weights d ij (k) = w ij if k = 0 min { d ij (k-), d ik (k-) + d kj (k-) } if k The final solution: D (n) = (d ij (n) ): d ij (n) = δ(i, j) i, j V j j + (k, j) i D (k-) (i, k) i D (k) Networking Laboratory /9
FLOYD-WARSHALL(W). n rows[w]. D (0) W. for k to n. do for i to n. do for j to n. do d (k) ij min (d (k-) ij, d (k-) ik + d (k-) kj ). return D (n) Running time: Θ(n ) Networking Laboratory /9
Constructing a Shortest Path = = = Π ij ij ij w j i i w j i Nil and or (0) Π + Π = Π otherwise ) ( ) ( ) ( ) ( ) ( ) ( k kj k kj k ik k ij k ij k ij d d d Networking Laboratory /9
d ij (k) = min {d ij (k-), d ik (k-) + d kj (k-) } Example 8 - - D (0) = W D () 0 8-0 0-0 0 0 8-0 0-0 - 0 (0) () N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N Networking Laboratory /9
d ij (k) = min {d ij (k-), d ik (k-) + d kj (k-) } Example D () () 0 8-0 N N N N N - 8 - D () 0-0 - 0 0 8-0 0-0 - 0 () N N N N N N N N N N N N N N N N N N N N Networking Laboratory /9
d ij (k) = min {d ij (k-), d ik (k-) + d kj (k-) } Example D () () 0 8-0 N N N N 0 N N - 8 - - 0-0 D () 0 8-0 0 - - 0-0 () N N N N N N N N N N N N N N N Networking Laboratory /9
Example - 8 - D () 0 8-0 0 - - 0-0 D () 0 - - 0 - - 0 - - 0-8 0 d ij (k) = min {d ij (k-), d ik (k-) + d kj (k-) } () N N N N N N N N N N () N N N N N Networking Laboratory 8/9
Example - 8 - D () 0 - - 0 - - d ij (k) = min {d ij (k-), d ik (k-) + d kj (k-) } N N 0 N - - 0 - N 8 0 N D () 0 - - 0 - - N N 0 N - - 0 - N 8 0 N () () Networking Laboratory 9/9