CMPS 00 Fall 01 Single Source Shortest Paths Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk 1
Paths in graphs Consider a digraph G = (V, E) with edge-weight function w : E R. The weight of path p = v 1 v... v k is defined to be Example: k 1 i=1= 1 w ( p) = w( v i, v i + 1). v 1 4 v 5 1 v 5 v v 4 w(p) =
Shortest paths A shortest path from u to v is a path of minimum weight from u to v. The shortestpath weight from u to v is defined as δ(u, v) = min{w(p) : p is a path from u to v}. Note: δ(u, v) = if no path from u to v exists.
Optimal substructure Theorem. A subpath of a shortest path is a shortest path. Proof. Cut and paste: 4
Triangle inequality Theorem. For all u,, v,, x V,, we have δ(u, v) δ(u, x) + δ(x, v). Proof. δ(u,v) minimizes over all paths from u to v δ(u, v) over all paths from u u v Concatenating two shortest paths from u to x and from x to v yields one specific path from u to v δ(u, x) δ(x, v) x 5
Well-definedness of shortest paths If a graph G contains a negative-weight g cycle, then some shortest paths may not exist. Example: u -6 5 < 0-8 v 6
Single-source shortest paths Problem. From a given source vertex s V, find the shortest-path weights δ(s, v) for all v V. Assumption: All edge weights w(u, v) are nonnegative. It follows that all shortest-path weights must exist. IDEA: Greedy. 1. Maintain a set S of vertices whose shortest-path weights from s are known, i.e., d[v]=δ(s,v) [ ] ( ). At each step add to S the vertex u V S whose distance estimate d[u] from s is minimal.. Update the distance estimates d[v] [ ] of vertices v adjacent to u. 7
Dijkstra s algorithm d[s] 0 for each v V {s} do d[v] S Vertices for which d[v]=d(s,v) Q V Q is a priority queue maintaining V S while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then relaxation step d[v] d[u] + w(u, v) Implicit DECREASE-KEY 8
Example of Dijkstra s algorithm Graph with nonnegative edge weights: A 10 B D 1 4 8 C E while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 9
Example of Dijkstra s algorithm Initialize: S: {} 0 A 10 B D 1 4 8 Q: C E A B C D E 0 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 10
Example of Dijkstra s algorithm A EXTRACT-MIN(Q): S: { A } 0 A 10 B D 1 4 8 Q: C E A B C D E 0 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 11
Example of Dijkstra s algorithm Relax all edges leaving A: S: { A } 0 A 10 10 B D 1 4 8 Q: C E A B C D E 0 10 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 1
Example of Dijkstra s algorithm C EXTRACT-MIN(Q): S: { A,,C} } 0 A 10 10 B D 1 4 8 Q: C E A B C D E 0 10 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 1
Example of Dijkstra s algorithm Relax all edges leaving C: S: { A,,C} } 0 A 10 7 11 B D 1 4 8 Q: C E A B C D E 0 5 10 7 11 5 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 14
Example of Dijkstra s algorithm E EXTRACT-MIN(Q): S: { A,,C, E } 0 A 10 7 11 B D 1 4 8 Q: C E A B C D E 0 5 10 7 11 5 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 15
Example of Dijkstra s algorithm Relax all edges leaving E: S: { A,,C, E } 0 A 10 7 11 B D 1 4 8 Q: C E A B C D E 0 5 10 7 11 5 7 11 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 16
Example of Dijkstra s algorithm B EXTRACT-MIN(Q): S: { A,,C, E, B } 0 A 10 7 11 B D 1 4 8 Q: C E A B C D E 0 5 10 7 11 5 7 11 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 17
Example of Dijkstra s algorithm Relax all edges leaving B: S: { A,,C, E, B } 0 A 10 B D 1 4 8 Q: C E A B C D E 0 5 10 7 11 5 7 11 9 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 18
Example of Dijkstra s algorithm D EXTRACT-MIN(Q): S: { A,,C, E, B, D}0 A 10 B D 1 4 8 Q: C E A B C D E 0 5 10 7 11 5 7 11 9 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 19
Analysis of Dijkstra V times degree(u) times while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) Handshaking Lemma Θ( E ) implicit DECREASE-KEY s. Time = Θ( V ) TT EXTRACT -MIN + Θ( E ) T TDECREASE-KEY Note: Same formula as in the analysis of Prim s minimum spanning tree algorithm. 0
Analysis of Dijkstra (continued) Time = Θ( V ) T ) EXTRACT-MIN + Θ( E ) T ) DECREASE-KEY Q T EXTRACT -MIN T DECREASE-KEY Total array O( V ) O(1) O( V ) binary heap Fibonacci heap O(log V ) O(log V ) O( E log V ) O(log V ) amortized O(1) O( E + V log V ) amortized worst case 1
Correctness Theorem. (i) For all v S: d[v] = δ(s, v) (ii) For all v S: d[v] ] = weight of shortest t path from s to v that uses only (besides v itself) vertices in S. Corollary. Dijkstra s algorithm terminates with d[v] = δ(s, v) for all v V.
Correctness Theorem. (i) For all v S: d[v] = δ(s, v) (ii) For all v S: d[v] ] = weight of shortest t path from s to v that uses only (besides v itself) vertices in S. Proof. By induction. Base: Before the while loop, d[s]=0 and d[v]= for all v s s, so (i) and (ii) are true. Step: Assume (i) and (ii) are true before an iteration; now we need to show they remain true after another iteration. Let u be the vertex added dto S, so d[u] ] d[v] ] for all other v S.
Correctness Theorem. (i) For all v S: d[v] = δ(s, v) (ii) For all v S: d[v] ] = weight of shortest t path from s to v that uses only (besides v itself) vertices in S. (i) Need to show that d[u] = δ(s, u). Assume the contrary. There is a path p from s to u with w(p) < d[u]. Because of (ii) that path uses vertices S, in addition to u. Let y be first vertex on p such that y S. s S,,just before x Path p from y adding u. s to u 4 u
Correctness Theorem. (i) For all v S: d[v] = δ(s, v) (ii) For all v S: d[v] ] = weight of shortest t path from s to v that uses only (besides v itself) vertices in S. S, just before adding u. s x y u Path p from s to u d[y] w(p) < d[u]. Contradiction to the choice of u. weights are assumption nonnegative about path 5
Correctness Theorem. (i) For all v S: d[v] = δ(s, v) (ii) For all v S: d[v] ] = weight of shortest t path from s to v that uses only (besides v itself) vertices in S. (ii) Let v S. Let p be a shortest path from s to v that uses only (besides v itself) vertices in S. p does not contain u: (ii) true by inductive hypothesis p contains u: p consists of vertices in S\{u} and ends with an edge from u to v. w(p)=d[u]+w(u,v), which is the value of d[v] after adding u. So (ii) is true. 6
Unweighted graphs Suppose w(u, v) = 1 for all (u, v) E. Can the code for Dijkstra be improved? Use a simple FIFO queue instead of a priority queue. while Q do Breadth-first search while Q do u DEQUEUE(Q) for each v Adj[u] do if d[v] = Analysis: Time = O( V + E ). u EXTRACT-MIN(Q) for each v Adj[u] do then d[v] d[u] + 1 ENQUEUE(Q, v) if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) 7
Example of breadth-first search a b c d e f g h i Q: 8
Example of breadth-first search 0 a f h d b g e i c 0 Q: a 9
Example of breadth-first search a 0 1 1 d b e c 1 1 Q: a bd f g h i 0
Example of breadth-first search a 0 1 1 d b e c 1 Q: a b d c e f g h i 1
Example of breadth-first search a 0 1 1 d b e c Q: a b d c e f g h i
Example of breadth-first search a 0 1 1 d b e c Q: a b d c e f g h i
Example of breadth-first search a 0 1 1 d b e c f h g i Q: a b d c e gi 4
Example of breadth-first search a 0 1 1 b c d e 4 f g 4 Q: a b d c e g i f h i 5
Example of breadth-first search a 0 1 1 b c d e 4 4 f g 4 4 Q: a b d c e g i f h h i 6
Example of breadth-first search a 0 1 1 b c d e 4 4 f g Q: a b d c e g i f h 4 h i 7
Example of breadth-first search a 0 1 1 b c d e 4 4 f g Q: a b d c e g i f h h i 8
Example of breadth-first search a 0 1 1 b c d e 4 4 f g Q: a b d c e g i f h h i 9
Correctness of BFS while Q do u DEQUEUE(Q) for each v Adj[u] do if d[v] = then d[v] d[u] + 1 ENQUEUE(Q, v) Key idea: The FIFO Q in breadth-first search mimics the priority queue Q in Dijkstra. Invariant: v comes after u in Q implies that d[v] = d[u] or d[v] = d[u]+1 + 1. 40
How to find the actual shortest paths? Store a predecessor tree: d[s] 0 for each v V {s} do d[v] S Q V Q is a priority queue maintaining V S while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u]+w(u + w(u, v) π[v] u 41
Example of Dijkstra s algorithm Graph with nonnegative edge weights: A 10 B D 1 4 8 C E while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 4
Example of Dijkstra s algorithm Initialize: S: {} 0 A 10 B D 1 4 8 Q: C E A B C D E 0 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 4
Example of Dijkstra s algorithm A EXTRACT-MIN(Q): S: { A } 0 A π: A B C D E C E Q: A B C D E 0 10 B D 1 4 8 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 44
Example of Dijkstra s algorithm Relax all edges leaving A: 10 10 B D 1 4 8 S: { A } 0 A π: A B C D E - C E Q: A B C D E 0 10 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 45
Example of Dijkstra s algorithm Relax all edges leaving A: 10 10 B D 1 4 8 S: { A } 0 A π: A B C D E Α Α - C E Q: A B C D E 0 10 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 46
Example of Dijkstra s algorithm C EXTRACT-MIN(Q): 10 B D S: { A,,C} } 0 A π: A B C D E Α Α - C E Q: A B C D E 0 10 10 1 4 8 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 47
Example of Dijkstra s algorithm Relax all edges leaving C: 10 7 11 B D 1 4 8 S: { A,,C} } 0 A π: A B C D E Α Α - C E Q: A B C D E 0 5 10 7 11 5 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 48
Example of Dijkstra s algorithm Relax all edges leaving C: 10 7 11 B D 1 4 8 S: { A,,C} } 0 A π: A B C D E Α Α - C E Q: A B C D E 0 5 10 7 11 5 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 49
Example of Dijkstra s algorithm E EXTRACT-MIN(Q): 7 11 B D S: { A,,C, E } 0 A π: A B C D E C Α C C C E Q: A B C D E 0 5 10 7 11 5 10 1 4 8 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 50
Example of Dijkstra s algorithm Relax all edges leaving E: 10 7 11 B D 1 4 8 S: { A,,C, E } 0 A π: A B C D E C Α C C C E Q: A B C D E 0 5 10 7 11 5 7 11 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 51
Example of Dijkstra s algorithm B EXTRACT-MIN(Q): 7 11 B D S: { A,,C, E, B } 0 A π: A B C D E C Α C C C E Q: A B C D E 0 5 10 7 11 5 7 11 10 1 4 8 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 5
Example of Dijkstra s algorithm Relax all edges leaving B: 10 B D 1 4 8 S: { A,,C, E, B } 0 A π: A B C D E C Α B C C E Q: A B C D E 0 5 10 7 11 5 7 11 9 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 5
Example of Dijkstra s algorithm D EXTRACT-MIN(Q): B D S: { A,,C, E, B, D}0 A π: A B C D E C Α B C C E Q: A B C D E 0 5 10 7 11 5 7 11 10 9 1 4 8 while Q do u EXTRACT-MIN(Q) for each v Adj[u] do if d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) π[v] u 54