Introduction to Algorithms, Lecture 5 // Introduction to Algorithms 6.46J/.4J LECTURE Shortest Paths I Properties o shortest paths Dijkstra s Correctness Analysis Breadth-irst Pro. Manolis Kellis March, L. Deinitions: Paths in graphs Consider a digraph G = (V, E) with edge-weight unction w : E. The weight o path p = v v L v k is deined to be: Example: k i= w ( p) = w( v i, v i + ). v 4 v 5 v 5 v v 4 w(p) = March, L. Deinitions: Shortest paths A shortest path rom u to v is a path o minimum weight rom u to v. The shortest-path weight rom u to v is deined as δ(u, v) = min{w(p) : p is a path rom u to v}. Note: δ(u, v) = i no path rom u to v exists. March, L. Shortest path properties:. Well-deinedness I a graph G contains a negative-weight cycle, then some shortest paths do not exist. Example: < vv <: Shortest path does not exist >: Cycle can be removed =: Choose no cycle, same cost, ewer edges, wlog Only nd to consider paths o V - edges! March, L.4 Shortest path properties:. Triangle inequality Shortest path properties:. Optimal Substructure Theorem. For all s, u, v V, we have δ(s, v) δ(s, u) + w(u, v). Theorem. Any subpath i~j o a shortest path s~k is a shortest path. Proo. ss δ(s, v) vv Proo. Cut and paste: δ(s, u) w(u, v) s i j k I optimal path p uses (u,v), then δ(s, v) = δ(s, u)+w(u, v). Else: proo by contradiction. I w p (s~>v)>δ(s, u)+w(u, v), then can make shorter path p by going through u. I not, make shorter path by s~i~j~k March, L.5 March, L.6 Leiserson, Demaine, Kellis
Introduction to Algorithms, Lecture 5 // Today Thursday Shortest paths: Problem settings Setting All pairs Weights Grdy/DP Algorithm = Grdy BFS General General Grdy Dyn. Prog. Dyn. Prog. Dijkstra Bellman-Ford MX-mult, Floyd- Warshall, Johnson March, L. S Grdy choice property (only i all weights are ) ss ww 4 9 vv Maintain set S o nodes whose shortest path distances are minimal. At each step, include the vertex v whose current distance estimate rom S is minimum, through edge (u,v). Then edge (u,v) must be part o a shortest path, since any other indirect path to v must go through a longer-distance intermediate (because path costs never decrease, all w ). March, L. zz Note: Full proo also shows that our estimates are correct Weighted graphs (non-negative) Dijkstra s March, L.9 Single-source shortest paths (non-negative edge weights) Problem. Assume that w(u, v) or all (u, v) E. (Hence, all shortest-path weights must exist.) From a given source vertex s V, ind the shortest-path weights δ(s, v) or all v V. Algorithm: Grdy choice.. Maintain a set S o vertices whose shortestpath distances rom s are known.. At each step, a to S the vertex v V S whose distance estimate rom s is minimum.. Update the distance estimates o vertices adjacent to v. March, L. Dijkstra s d[s] or each v V {s} do d[v] S Q V Q is a priority queue maintaining V S, keyed on d[v] while Q do u EXTRACT-MIN(Q) S S {u} or each v Adj[u] do i d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) relaxation step Implicit DECREASE-KEY March, L. Example o Dijkstra s Graph with nonnegative edge weights: March, L. 4 9 EE Leiserson, Demaine, Kellis
Introduction to Algorithms, Lecture 5 // Initialize: Example o Dijkstra s S: {} 4 9 EE Example o Dijkstra s A EXTRACT-MIN(Q): S: { A } 4 9 EE March, L. March, L.4 Example o Dijkstra s Relax all edges leaving A: S: { A } 4 9 EE Example o Dijkstra s C EXTRACT-MIN(Q): 4 9 S: { A, C } EE March, L.5 March, L.6 Example o Dijkstra s Relax all edges leaving C: 5 4 9 S: { A, C } EE 5 Example o Dijkstra s E EXTRACT-MIN(Q): 5 4 9 EE 5 S: { A, C, E } March, L. March, L. Leiserson, Demaine, Kellis
Introduction to Algorithms, Lecture 5 // Example o Dijkstra s Relax all edges leaving E: 5 4 9 EE 5 S: { A, C, E } Example o Dijkstra s B EXTRACT-MIN(Q): 5 4 9 EE 5 S: { A, C, E, B } March, L.9 March, L. Example o Dijkstra s Relax all edges leaving B: 5 9 9 4 9 EE 5 S: { A, C, E, B } Example o Dijkstra s D EXTRACT-MIN(Q): 5 9 9 4 9 EE 5 S: { A, C, E, B, D } March, L. March, L. Dijkstra's: Proo o correctness March, L. Correctness proo: Part : Upper Bound (Th. 4.) Lemma. Initializing d[s] and d[v] or all v V {s} establishes d[v] δ(s, v) or all v V, and this invariant is maintained over any sequence o relaxation steps. (Katherine s Lemma) Proo. Prove invariant d[v] δ(s, v) or all vertices by induction over number o relaxation steps. Base case upon d[s]= and d[v]= otherwise. Inductive step upon relaxation o edge (u,v): by inductive hypothesis d[x] δ(s,x) or all x V. The only d value changing is d[v], and it becomes: d[v] = d[u] + w(u,v) δ(s,u)+w(u,v) by inductive hypothesis δ(s,v) by the triangle inequality. March, L.4 Leiserson, Demaine, Kellis 4
Introduction to Algorithms, Lecture 5 // Correctness proo: Part : Convergence (Th. 4.6) Lemma. Let u be v s predecessor on a shortest path rom s to v. Then, i d[u] = δ(s, u) and edge (u, v) is relaxed, we have d[v] =δ(s, v) ater the relaxation. Proo. Observe that δ(s, v) = δ(s, u) + w(u, v). Suppose that d[v] > δ(s, v) beore the relaxation. (Otherwise, we re done.) Then, the test d[v] > d[u] + w(u, v) succds, because d[v] > δ(s, v) = δ(s, u) + w(u, v) = d[u] + w(u, v), and the sets d[v] = d[u] + w(u, v) = δ(s, v). Correctness proo (Th. 4.6): Part : Correctness o Dijkstra Theorem. Dijkstra s terminates with d[v] = δ(s, v) or all v V. Proo. It suices to show that d[v] = δ(s, v) or every v V when v is aed to S. Suppose u is the irst vertex aed to S or which d[u] >δ(s, u). Let y be the irst vertex in V S along a shortest path rom s to u, and let x be its predecessor: S, just beore aing u. ss xx yy March, L.5 March, L.6 Correctness proo: Part : Correctness o Dijkstra ss S xx yy Dijkstra's Runtime analysis Since u is the irst vertex violating the claimed invariant, we have d[x] = δ(s, x). When x was aed to S, the edge (x, y) was relaxed, which implies that d[y]=δ(s, y) δ(s, u) < d[u]. But, d[u] d[y] by our choice o u. Contradiction. March, L. March, L. V times Analysis o Dijkstra degr(u) times while Q do u EXTRACT-MIN(Q) S S {u} or each v Adj[u] do i d[v] > d[u] + w(u, v) then d[v] d[u] + w(u, v) Handshaking Lemma Θ(E) implicit DECREASE-KEY s. Time = Θ(V T EXTRACT-MIN + E T DECREASE-KEY ) Note: Same ormula as in the analysis o Prim s minimum spanning tr. March, L.9 Analysis o Dijkstra (continued) Time = Θ(V) T EXTRACT-MIN + Θ(E) T DECREASE-KEY Q T EXTRACT-MIN T DECREASE-KEY Total array O(V) O() O(V ) binary heap O(lg V) O(lg V) O(E lg V) Fibonacci heap O(lg V) amortized O() amortized O(E + V lg V) worst case March, L. Leiserson, Demaine, Kellis 5
Introduction to Algorithms, Lecture 5 // Unweighted graphs: BFS (all edges have positive, unit weights) w(u,v)= or all (u,v) March, L. Unweighted graphs Suppose that w(u, v) = or all (u, v) E. Can Dijkstra s be improved? Use a simple FIFO queue instead o a priority queue. Breadth-irst while Q do u DEQUEUE(Q) or each v Adj[u] do i d[v] = then d[v] d[u] + ENQUEUE(Q, v) Analysis: Time = O(V + E). March, L. Example o breadth-irst Example o breadth-irst cc cc a March, L. March, L.4 Example o breadth-irst Example o breadth-irst cc a b d cc a b d c e March, L.5 March, L.6 Leiserson, Demaine, Kellis 6
Introduction to Algorithms, Lecture 5 // Example o breadth-irst Example o breadth-irst cc a b d c e cc a b d c e March, L. March, L. Example o breadth-irst cc a b d c e g i Example o breadth-irst 4 cc 4 a b d c e g i March, L.9 March, L.4 Example o breadth-irst cc a b d c e g i h Example o breadth-irst cc 4 a b d c e g i h March, L.4 March, L.4 Leiserson, Demaine, Kellis
Introduction to Algorithms, Lecture 5 // Example o breadth-irst cc a b d c e g i h Example o breadth-irst cc a b d c e g i h March, L.4 March, L.44 Correctness o BFS while Q do u DEQUEUE(Q) or each v Adj[u] do i d[v] = then d[v] d[u] + ENQUEUE(Q, v) Key idea: The FIFO Q in breadth-irst mimics the priority queue Q in Dijkstra. Invariant: v comes ater u in Q implies that d[v] = d[u] or d[v] = d[u] +. Today Thursday Shortest paths: Summary Setting All pairs Weights Gr./DP Algorithm = Grdy BFS: O(V+E) General General Grdy DP DP Dijkstra: O(E+VlgV) Bellman-Ford MX-mult, Floyd- Warshall, Johnson March, L.45 March, L.46 Leiserson, Demaine, Kellis