Data Structures and and Algorithm Xiaoqing Zheng

Transcription

1 Data Structures and Algorithm Xiaoqing Zheng

2 Representations of undirected graph / / 2 4 / / 4 1 / Adjacency-list representation of graph G = (V, E)

3 Representations of undirected graph Adjacency-matrix representation of graph G = (V, E)

4 Representations of directed graph / 2 5 / / / 6 6 / / Adjacency-list representation of graph G = (V, E)

5 Representations of directed graph Adjacency-matrix representation of graph G = (V, E)

6 Graphs Definition. A directed d graph (digraph) h G = (V, E) is an ordered pair consisting of a set V of vertices (singular: vertex), a set E V V of edges. In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices. In either case, we have E = O(V 2 ). Moreover, if G is connected, then E V 1.

7 Representations of graph Adjacency-list representation An adjacency list of a vertex v V is the list Adj[v] of vertices adjacent to v. For undirected graphs, Adj[v] = degree(v). For directed graphs, Adj[v] = out-degree(v). Adjacency-matrix representation The adjacency matrix of a graph G = (V, E), where V = {1, 2,, n}, is the matrix A[1 n, 1 n] given by 1 if (i, j) E, A[i, j] = 0 if (i, j) E.

8 Breadth-first search Given a graph G = (V, E) and a distinguished ished source vertex s, breadth-first search systematically explores the edges of G to "discover" every vertex that is reachable from s. r s t u 0 v w x y

9 Breadth-first search example r s t u 0 v w x y Q s 0 Gray vertices

10 Breadth-first search example r s t u 1 0 Q w r Gray vertices v w x y

11 Breadth-first search example r s t u Q r t x Gray vertices v w x y

12 Breadth-first search example r s t u Q t x v Gray vertices v w x y

13 Breadth-first search example r s t u Q x v u Gray vertices v w x y Why not x?

14 Breadth-first search example r s t u Q v u y Gray vertices v w x y

15 Breadth-first search example r s t u Q u y Gray vertices v w x y

16 Breadth-first search example r s t u Gray vertices v w x y Q y 3

17 Breadth-first search example r s t u Q Gray vertices v w x y

18 Breadth-first search algorithm BFS(G, s) ) 1. for each vertex u V[G] {s} 2. do color[u] [ ] WHITE 3. d[u] 4. π[u] [ ] NIL 5. color[s] GRAY 6. d[s] ] 0 7. π[s] NIL 8. Q 9. ENQUEUE(Q, s)

19 Breadth-first search algorithm BFS(G, s) ) 10. while Q 11. do u DEQUEUE(Q) 12. for each v Adj[u] 13. do if color[v] [ ] = WHITE 14. O(E) then color[v] GRAY 15. times d[v] ] d[u] ] π[v] u 17. ENQUEUE(Q, v) ) 18. color[u] BLACK Running time is O(V + E) O(V) times

20 Shortest paths PRINT-PATH(G, PATH(G s, v) ) 1. if v = s 2. then print s 3. else if π[v] = NIL 4. then print t" "no path thf from" s "to"" v "exists." " 5. else PRINT-PATH(G, s, π[v]) 6. print v

21 Predecessor subgraph of BFS s w r t u x y v Predecessor subgraph of BFS is G π = (V π, E π ), where V π ={ v V : π[v] NIL } {s} and E π ={ (π[v], v): v V π {s} }.

22 Depth-first search Given a graph G = (V, E), depth-first search is to search deeper in the graph whenever possible. Edges are explored out of the most recently discovered vertex v that still has unexplored edges leaving it. u v w x y x

23 Depth-first search example u v w 11/12 12/12 12/12 12/12 12/12 12/12 x y x TOP u S

24 Depth-first search example u v w 11/12 12/12 12/12 12/12 12/12 12/12 x y x TOP v u S

25 Depth-first search example u v w 11/12 12/12 12/12 12/12 13/12 12/12 x y x TOP y v u S

26 Depth-first search example u v w 11/12 12/12 12/12 TOP x y x y x v u 14/12 13/12 12/12 S

27 Depth-first search example u v w 11/12 12/12 12/12 B 14/52 13/12 12/12 x y x TOP x y v u S

28 Depth-first search example u v w 11/12 12/12 12/12 B y x y x TOP v u 14/52 13/62 12/12 S

29 Depth-first search example u v w 11/12 12/72 12/12 B 14/52 13/62 12/12 x y x v TOP u S

30 Depth-first search example u v w 11/82 12/72 12/12 F B 14/52 13/62 12/12 x y x TOP u S

31 Depth-first search example u v w 11/82 12/72 19/12 F B 14/52 13/62 12/12 x y x TOP w S

32 Depth-first search example u v w 11/82 12/72 19/12 F B 14/52 13/62 10/12 x y x TOP x w S

33 Depth-first search example u v w 11/82 12/72 19/12 F B 14/52 13/62 10/11 B x y x x TOP w S

34 Depth-first search example u v w 11/82 12/72 19/12 F B C 14/52 13/62 10/11 B x y x TOP w S

35 Depth-first search example u v w 11/82 12/72 19/12 F B 14/52 13/62 10/11 C B x y x TOP S

36 Depth-first search algorithm DFS(G) 1. for each vertex u V[G] 2. do color[u] [ ] WHITE 3. π[u] NIL 4. time 0 5. for each vertex u V[G] 6. do if color[u] [ ] = WHITE 7. then DFS-VISIT(u) O(V) ( ) times

37 Depth-first search algorithm DFS-VISIT(u) ) 1. color[u] GRAY 2. time time d[u] time 4. for each vertex v Adj[u] ] 5. do if color[v]=white 6. then π[v] [ ] u 7. DFS-VISIT(v) 8. color[u] [ ] BLACK 9. f[u] time time + 1 Running time is O(V + E) O(E) times

38 Predecessor subgraph of DFS u w v C z B F B y Predecessor subgraph of DFS is G π = (V, E π ), where x E π ={ (π[v], v): v V and π[v] NIL }.

39 Predecessor subgraph of DFS u w Each edge (u, v) can be classified by the color of the B vertex v that is reached when C v z the edge is first explored: WHITE indicates a tree edge; F B GRAY indicates a back edge; y BLACK indicates a forward (if d[u] < d[v]) or cross edge x (if d[u] > d[v]).

40 Precedence among events Underwear Socks Watch Pants Shoes Shirt Belt Tie Jacket

41 Precedence among events Underwear Socks Watch 11/16 17/18 19/10 Pants Shoes 12/15 13/14 11/86 Shirt Belt Tie 16/76 12/56 13/46 Jacket Call depth first search algorithm.

42 Precedence among events Underwear Socks Watch 11/26 17/11 19/50 Pants Shoes 12/35 13/44 11/66 Shirt Belt Tie 16/76 12/86 Sort tby finishing i time. 13/96 Jacket

43 Curriculum Java or C++ Data Structure and Algorithm Object Oriented Programming Software Engineering Web Application Database Computer Systems Internship Thesis Computer Architecture Calculus ALL COURSES Project Management Intelligent Systems Computer Network Probability and Statistics Discrete Mathematics

44 Topological sort TOPOLOGICAL-SORT(G) 1. call DFS(G) to compute finishing times f[v] for each vertex v. 2. as each vertex is finished, insert it onto the front of a linked list. 3. return the linked list of vertices.

45 Topological sort A topological lsort of a directed acyclic graph or "dag" d G = (V, E) is a linear ordering of all its vertices such that if G contains an edge (u, v), then u appears before v in the ordering. Topological sort of a graph can be viewed as an linear ordering of its vertices. Topological lsorting is different from the usual lkind of "sorting" studied before. If the graph is not acyclic, then no linear ordering is possible.

46 Disjoint sets a b e h f j Connected components c d g i Edge processed Collection of disjoint sets initial sets {a} {b} {c} {d} {e} {f} {g} {h} {i} {j}

47 Disjoint sets a b e h c d g i Edge processed f j Connected components Collection of disjoint sets initial sets {a} {b} {c} {d} {e} {f} {g} {h} {i} {j} (b, d) {a} {b, d} {c} {e} {f} {g} {h} {i} {j}

48 Disjoint sets a b e h c d g i f j Connected components Edge processed Collection of disjoint sets initial sets (b, d) (e, g) {a} {a} {a} {b} {b, d} {b, d} {c} {c} {c} {d} {e} {e} {e, g} {f} {f} {f} {g} {g} {h} {h} {h} {i} {i} {i} {j} {j} {j}

49 Disjoint sets a b e h c d g i f j Connected components Edge processed Collection of disjoint sets initial sets (b, d) (e, g) (a, c) {a} {a} {a} {a, c} {b} {b, d} {b, d} {b, d} {c} {c} {c} {d} {e} {e} {e, g} {e, g} {f} {f} {f} {f} {g} {g} {h} {h} {h} {h} {i} {i} {i} {i} {j} {j} {j} {j}

50 Disjoint sets a b e h c d g i f j Connected components Edge processed Collection of disjoint sets initial sets (b, d) (e, g) (a, c) (h, i ) (a, b) (e, f ) (b, c) {a} {a} {a} {a, c} {a, c} {a, b, c, d} {a, b, c, d} {a, b, c, d} {b} {c} {b, d} {c} {b, d} {c} {b, d} {d} {e} {e} {e, g} {e, g} {b, d} {e, g} {e, g} {e, f, g} {e, f, g} {f} {f} {f} {f} {f} {f} {g} {g} {h} {h} {h} {h} {h, i} {h, i} {h, i} {h, i} {i} {i} {i} {i} {j} {j} {j} {j} {j} {j} {j} {j}

51 Disjoint set operations MAKE-SET(x) ( ) creates a new set whose only member is x. UNION(x, ( y) ) unites the dynamic sets that contain x and y, say S x and S y, into a new set that is the union of these two sets. The two sets are assumed to be disjoint prior to the operation. FIND-SET(x) ) returns a pointer to the representative of the unique set containing x.

52 Linked list representation of disjoint sets a b c d / h i / e f g / j UNION(a, e) a b c d e f g /

53 Analysis of linked list representation Linked list representation of the UNION operation requires an average of Φ(n) time per call because we may be appending a longer list onto a shorter list. Weighted-union heuristic: Suppose that each list also includes the length of the list and that we always append the smaller list onto the longer. A sequence of m MAKE-SET, UNION, and FIND-SET operations, n of whichare MAKE-SET operations, takes O(m + nlgn) time.

54 Disjoint set forests c e h h UNION(e, h) i f e i a d g c f b a d g b

55 Union by rank c e f h i UNION(e, h) e c f h a d g a d g i b b

56 Path compression e c f h FIND-SET(b) ( ) e a b c f h a d g i d g i b

57 Disjoint set forests MAKE-SET(x) ) 1. p[x] x 2. rank[x] ] 0 FIND-SET(x) 1. if x p[x] 2. then p[x] FIND-SET(p[x]) 3. return p[x]

58 Disjoint set forests UNION(x, y) ) 1. LINK(FIND-SET(x), FIND-SET(y)) LINK(x, y) 1. if rank[x] > rank[y] 2. then p[y] x 3. else p[x] y 4. if rank[x] = rank[y] 5. then rank[y] rank[y] + 1

59 Union by rank and path compression When we use both union by rank and path compression, the worst-case running time is O(m α(n)), where α(n) is a very slowly l growing function. 0 for 0 n 2, 1 for n = 3, α(n) ( ) 2 for 4 n 7, 3 for 8 n 2047, 4 for 2047 n A 4 (1) >> A k (j) j + 1 if k = 0, A 1 ( j) ( j+ 1) k if k 1.

60 Minimum spanning tree b c d 2 9 a i h g f 1 2 e 10

61 Minimum spanning tree b c d 2 9 a i h g f 1 2 e 10 Total weight = = 37

62 Minimum spanning tree Input: A connected, undirected d graph G = (V, E) with weight function w: E. Output: A spanning tree T a tree that connects all vertices of minimum weight: wt ( ) = wuv (, ) ( uv, ) T

63 Destroy cycles b c d 2 9 a i h g f 1 2 e 10

64 Destroy cycles b c d 2 9 a i h g f 1 2 e 10

65 Destroy cycles a b c d 7 i h g f e 10

66 Destroy cycles a b c d 7 i h g f e 10

67 Destroy cycles a b c d 7 i h g f e

68 Destroy cycles a b c d 7 i h g f e

69 Destroy cycles a b c d 7 i e h g f 1 2

70 Destroy cycles a b c d 7 i e h g f 1 2

71 Destroy cycles a b c d i e h g f 1 2

72 Destroy cycles a b c d i e h g f 1 2

73 Destroy cycles a b c d i e h g f 1 2

74 Destroy cycles a b c d i e h g f 1 2

75 Destroy cycles a b c d i e h g f 1 2 Total weight = = 37

76 Avoid cycles b c d 2 9 a i h g f 1 2 e 10 Total weight = = 37

77 Kruskal's algorithm MST-KRUSKAL(G, w) ) 1. A 2. for each vertex v V [G] 3. do MAKE-SET(v) 4. sort the edges of E into nondecreasing order by weight w. 5. for each edge (u, v) E, taken in nondecreasing order by weight. 6. do iffind FIND-SET(u) FIND-SET(v) 7. then A A {(u, v)} 8. UNION(u, v) 9. return A Running time is O(ElgE) E)

78 Optimal substructure Minimum spanning tree T of G = (V, E).

79 Optimal substructure Minimum spanning tree T of G = (V, E). u Remove any edge (u, v) T. v

80 Optimal substructure Minimum spanning tree T of G = (V, E). T 1 v Remove any edge (u, v) T. Then, T is partitioned into two subtrees T 1 and T 2. Theorem. The subtree T 1 is an MST of G 1 = (V 1, E 1 ), the subgraph of G induced by the vertices of T 1 : V 1 = vertices of T 1, E 1 ={(x (x, y) E: x, y V 1 }. Similarly for T 2. u T 2

81 Proof of optimal substructure Proof. Cut and paste: w(t) ( ) = w(u, ( v) ) + w(t( 1 ) + w(t( 2 ). If T 1 ' were a lower-weight spanning tree than T 1 for G 1, then T '={(u, v)} T 1 ' T 2 would be a lower-weight weight spanning tree than T for G. Do we also have overlapping subproblems? Great, then dynamic programming may work! Yes! Yes! But minimum spanning tree exhibits another powerful property p which leads to an even more efficient algorithm.

82 Hallmark for "greedy" algorithms Greedy-choice property A locally optimal choice is globally optimal. Theorem. Let T be the minimum spanning tree of G =(V (V, E), and let A V. Suppose that (u, v) E is the least-weight edge connecting A to V A. Then, (u, v) T.

83 Proof of theorem Proof. Suppose (u, v) ) T. Cut and paste. T: v A T A u (u, v)=least-weight edge connecting A to V A.

84 Proof of theorem Proof. Suppose (u, v) ) T. Cut and paste. T: v A T A u (u, v)=least-weight edge connecting A to V A. Consider the unique simple path from u to v in T.

85 Proof of theorem Proof. Suppose (u, v) ) T. Cut and paste. T: v A T A u (u, v)=least-weight edge connecting A to V A. Consider the unique simple path from u to v in T. Swap (u, v) ) with the first edge on this path that connects a vertex in A to a vertex in V A.

86 Proof of theorem Proof. Suppose (u, v) ) T. Cut and paste. T: v A T A u (u, v)=least-weight edge connecting A to V A. Consider the unique simple path from u to v in T. Swap (u, v) ) with the first edge on this path that connects a vertex in A to a vertex in V A. A lighter-weight spanning tree than T results.

87 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

88 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

89 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

90 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

91 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

92 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

93 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

94 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

95 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

96 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

97 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

98 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

99 Example of Prim's algorithm b c d 2 9 a i h g f 1 2 e 10

100 Prim's algorithm IDEA: Miti Maintain V A as a priority it queue Q. Key each vertex in Q with the weight of the least weight edge connecting it to a vertex in A. MST-PRIM(G, w, r) Minimum spanning tree A for G is 1. for each u V[G] thus A = { (v, π[v]): v V {r}} 2. do key[u] 3. π[u] NIL 4. key[r] 6. while Q 0 5. Q V[G] 7. do u EXTRACT-MIN(Q) 8. for each v Adj[u] 9. do if v Q and w(u, v) < key(v) 10. then π[v] [ ] u 11. key[v] w(u, v)

101 Analysis of Prim algorithm MST-PRIM(G PRIM(G, w, r) 1. for each u V[G] 2. do key[u] Φ(V) 3. π[u] NIL total 4. key[r] 0 5. Q V[G] 6. while Q 7. do u EXTRACT-MIN(Q) 8. for each v Adj[u] 9. degree(u) do if v Q and w(u, v) < key(v) V times 10. times then π[v] u 11. key[v] w(u, v) Θ(E) implicit DECREASE-KEY's. Time = Θ(V) Time EXTRACT-MIN + Θ(E) Time DECREASE-KEY

102 Analysis of Prim algorithm Time = Θ(V) Time EXTRACT-MIN + Θ(E) Time DECREASE-KEY Q Time EXTRACT-MIN Time DECREASE-KEY Total Array O(V) O(1) O(V 2 ) Binary heap O(lgV) ( ) O(lgV) ( ) O(ElgV) ( g ) Running time of Prim's algorithm is O(ElgV )

103 Pudong Shanghai

104 Shortest paths t 1 x 10 9 s y 2 z

105 Shortest paths problem Consider a directed graph G = (V, E), with weight function w: E mapping edges to real-valued weights. The weight of path p = v 1 v 2 v k is defined to be k wp ( ) = wv ( i 1, vi) i= 1 We define the shortest path weight from u to v by p min{w(p): u v} If there is a path from u to v, δ ( uv, ) = Otherwise.

106 Optimal substructure Theorem. A subpath of a shortest path is a shortest path. Proof. Cut and paste: u v x y

107 Single-source shortest paths Problem. From a given source vertex s V, find the shortest-path weights δ(s, v) for all v V. If all edge weights w(u, v) are nonnegative, all shortest-path weights must exist. IDEA: Greedy. 1. Maintain a set S of vertices whose shortest path distances from s are known. 2. At each step add to S the vertex v V S whose distance estimate from s is minimal. 3. Update the distance estimates of vertices adjacent to v.

108 Example of Dijkstra's algorithm t x s y 2 z

109 Example of Dijkstra's algorithm t x s y 2 z

110 Example of Dijkstra's algorithm t s x y z

111 Example of Dijkstra's algorithm t x 14 s y z

112 Example of Dijkstra's algorithm t x 13 s y z

113 Example of Dijkstra's algorithm t s x y z

114 Example of Dijkstra's algorithm t s x y z

115 Dijkstra's algorithm DIJKSTRA(G, w, s) 1. for each vertex v V[G] 2. do d[v] [ ] 3. π(v) NIL 4. d[s] 0 5. S 6. Q V[G] 7. while Q 8. do u EXTRACT-MIN(Q) 9. S S {u} 10. for each vertex v Adj[u] 11. do if d[v] > d[u] + w(u, v) 12. then d[v] d[u] + w(u, v) 13. π(v) u Relaxation step. Implicit DECREASE-KEY.

116 Correctness of Dijkstra's algorithm Lemma. Initializing d[s] 0 and d[v] for all v V {s} establishes d[v] δ(s, v) for all v V, and this invariant is maintained over any sequence of relaxation steps. Proof. Suppose not. Let v be the first vertex for which d[v] < δ(s, v), and let u be the vertex that caused d[v] to change:d[v] =d[u] +w(u w(u, v). Then, d[v] < δ(s, v) supposition δ(s, u)+δ(u + δ(u, v) triangle inequality δ(s, u) + w(u, v) sh. path specific path d[u]+w(u + w(u, v) v is first violation Contradiction

117 Correctness of Dijkstra's algorithm Lemma. Let u be v's s predecessor on a shortest path from s to v. Then, if d[u] = δ(s, u) and edge (u, v) is relaxed, we have d[v]=δ(s = δ(s, v) after the relaxation. Proof. Observe that δ(s, v) = δ(s, u) + w(u, v). Suppose that d[v]>δ(s > δ(s, v) before the relaxation. (Otherwise, we're done.) Then, the test d[v] > d[u] + w(u, v) succeeds, because d[v] > δ(s, v)=δ(s = δ(s, u)+ + w(u, v) = d[u] + w(u, v), and the algorithm sets d[v]=d[u]+w(u = d[u] + w(u, v)=δ(s = δ(s, v).

118 Correctness of Dijkstra's algorithm Theorem. Dijkstra's algorithm terminates with d[v] = δ(s, v) for all v V. Proof. It suffices to show that d[v] = δ(s, v) for every v V when v is added to S. Suppose u is the first vertex added to S for which d[u] > δ(s, u). Let y be the first vertex in V S along a shortest path from s to u, and let x be its predecessor: p 2 u Just tbefore adding u. s S p 1 x y

119 Correctness of Dijkstra's algorithm s S p 1 x p 2 y u Since u is the first vertex violating the claimed invariant, we have d[x] = δ(s, x). When x was added to S, the edge (x, y) was relaxed, which implies that d[y] = δ(s, y) δ(s, u) < d[u]. But, d[u] d[y] by our choice of u. Contradiction.

120 Analysis of Dijkstra's algorithm DIJKSTRA(G, w, s) 1. for each vertex v V[G] 2. do d[v] [ ] 3. π(v) NIL 4. d[s] 0 5. S 6. Q V[G] 7. while Q Time = Θ(V) Time EXTRACT-MIN + Θ(E) Time DECREASE-KEY 8. do u EXTRACT-MIN(Q) 9. S S {u} 10. for each vertex v Adj[u] 11. degree(u) do if d[v] > d[u] + w(u, v) 12. times then d[v] d[u] + w(u, v) 13. π(v) u V times

121 Analysis of Dijkstra's algorithm Time = Θ(V) Time EXTRACT-MIN + Θ(E) Time DECREASE-KEY Q Time EXTRACT-MIN Time DECREASE-KEY Total Array O(V) O(1) O(V 2 ) Binary heap O(lgV) ( ) O(lgV) ( ) O(ElgV) ( g ) Running time of Dijkstra'ss algorithm is O(ElgV )

122 Unweighted graphs Suppose that w(u, ( v) ) = 1for all (u, v) ) E. Can Dijkstra's algorithm be improved? Breadth-first t search. Use a simple FIFO queue instead of a priority queue. 1. while Q 2. do u DEQUEUE(Q) 3. for each vertex v Adj[u] 4. do if d[v] = 5. then d[v] d[u] ENQUEUE(Q, v) Running time is O(V + E).

123 Example of breadth-first search t x Q s s 0 0 Gray vertices y z

124 Example of breadth-first search t x 1 Q t y s Gray vertices 1 y z

125 Example of breadth-first search t x 1 2 Q y x s Gray vertices 1 y z

126 Example of breadth-first search t x 1 2 Q x z s Gray vertices 1 2 y z

127 Example of breadth-first search t x 1 2 Q z s y z Gray vertices

128 Example of breadth-first search t x 1 2 Q s 0 Gray vertices 1 2 y z

129 Shortest paths in weighted dag What is dag? A dag is a directed acyclic graph. DAG-SHORTEST-PATH(G, w, s) 1. Topologically sort the vertices of G 2. for each vertex v V[G] 3. do d[v] 4. π(v) NIL 5. d[s] 0 6. for each vertex u, taken in topologically sorted order 7. do for each vertex v Adj[u] 8. do if d[v] > d[u] + w(u, v) 9. then d[v] d[u] +w(u w(u, v) 10. π(v) u

130 Example of dag shortest paths 6 1 r s t x y z

131 Example of dag shortest paths 6 1 r s t x y z

132 Example of dag shortest paths 6 1 r s t x y z

133 Example of dag shortest paths 6 1 r s t x y z

134 Example of dag shortest paths 6 1 r s t x y z

135 Example of dag shortest paths 6 1 r s t x y z

136 Example of dag shortest paths 6 1 r s t x y z

137 Positive-weight cycle u > 0 v Shortest path cannot contain a Shortest path cannot contain a positive-weight cycle.

138 0-weight cycle = 0 u v 0-weight cycle can be removed 0 weight cycle can be removed from the shortest path.

139 Negative-weight cycle u < 0 v If a graph contains a negative- weight cycle, then some shortest paths may not exist.

140 Algorithm for negative weight cycle Bll F d l i h Bellman-Ford algorithm: Finds all shortest-path lengths from a source s V to all v V or determines that a negative-weight cycle exists.

141 Example of Bellman-Ford algorithm t 5 x s y 9 z Initialization.

142 Example of Bellman-Ford algorithm t 1 5 x s y 6 9 z Order of edge relaxation.

143 Example of Bellman-Ford algorithm t 1 5 x s y 6 9 z

144 Example of Bellman-Ford algorithm t 1 5 x s y 6 9 z

145 Example of Bellman-Ford algorithm t 1 5 x s y 6 9 z

146 Example of Bellman-Ford algorithm t 1 5 x s y z

147 Example of Bellman-Ford algorithm t 1 5 x s y z End of pass 1.

148 Example of Bellman-Ford algorithm t 1 5 x s y z

149 Example of Bellman-Ford algorithm t 1 5 x s y z

150 Example of Bellman-Ford algorithm t 1 5 x s y z

151 Example of Bellman-Ford algorithm t 1 5 x s y z End of pass 2.

152 Example of Bellman-Ford algorithm t 1 5 x s y z

153 Example of Bellman-Ford algorithm t 1 5 x s y z End of pass 3.

154 Example of Bellman-Ford algorithm t 1 5 x s y z

155 Example of Bellman-Ford algorithm t s x y z End of pass 4.

156 Example of Bellman-Ford algorithm t s x y z End

157 Bellman-Ford algorithm BELLMAN-FORD(G, w, s) 1. for each vertex v V[G] 2. do d[v] [ ] 3. π(v) NIL 4. d[s] 0 5. for i 1 to V[G] 1 6. do for each edge (u, v) E[G] 7. do if d[v] > d[u]+w(u + w(u, v) 8. then d[v] d[u] + w(u, v) 9. π(v) ( ) u 10. for each edge (u, v) E[G] 11. do if d[v] > d[u] + w(u, v) 12. then return FALSE 13. return TURE

158 Correctness of Bellman-Ford algorithm Theorem. If G = (V, E) contains no negative e weight cycles, then after the Bellman-Ford algorithm executes, d[v] ] = δ(s, v) ) for all v V. Proof. Let v V be any vertex, and consider a shortest path p from s to v with the minimum number of edges. p: s v v 0 v 1 v 2 v 3 v k Since p is a shortest path, we have δ(s, v i )=δ(s δ(s, v i 1 )+w(v i 1, v i ).

159 Correctness of Bellman-Ford algorithm p: s v v 0 v 1 v 2 v 3 v k Initially, d[v 0 ] = 0 = δ(s, v 0 ), After 1 pass through E,wehaved[v 1 ] = δ(s, v 1 ). After 2 passes through E, we have d[v 2 ] = δ(s, v 2 ).. After k passes through E, we have d[v k ] = δ(s, v k ). Since G contains no negative-weight g cycles,,pp is simple. Longest simple path has V 1 edge If a value d[v] fails to converge after V 1 passes, there exists a negative-weight cycle in G reachable from s.

160 Correctness of Bellman-Ford algorithm Conversely, suppose that graph G contains a negative- e weight cycle that is reachable from the source s; let this cycle be c = v 0 v 1 v k, where v 0 = v k. Then, k wv ( i 1, v i) < i= 1 (, ) 0 If the Bellman-Ford algorithm returns TRUE, then, d[v i ] d[v i 1 ] + w(v i 1, v i ) for i = 1, 2,, k, and k k 1 1 d [ v ] ( d [ v ] + w ( v, v )) i i i i i= 1 i= 1 k k = + d [ v ] w ( v, v ) i 1 i 1 i i= 1 i= 1

161 Correctness of Bellman-Ford algorithm k k k dv [ ] dv [ ] + wv (, v) i i 1 i 1 i i= 1 i= 1 i= 1 Since v 0 = v k, and so, k k dv [ i] = dv [ i 1], thus, i= 1 i= 1 k 0 wv (, v) contradicts with wv ( 1, v) < 0 i= 1 i 1 i k i= 1 i i

162 Example of negative-weight cycle x t s y Initialization and order of edge relaxation.

163 Example of negative-weight cycle x t s y

164 Example of negative-weight cycle x t s y End of pass 1.

165 Example of negative-weight cycle x t s y

166 Example of negative-weight cycle x t s y End of pass 2.

167 Example of negative-weight cycle x t s y

168 Example of negative-weight cycle x t s y End of pass 3.

169 Example of negative-weight cycle x t s y End. 5 > 2 +1

170 Single-source shortest-paths algorithms Algorithms Unweighted Positive Negative Cycle Time Breadth-first search O(V + E) Dag shortest paths O(V + E) Dijkstra O(ElgV) Bellman- O(VE) Ford ( )

171 All-pairs shortest paths Input: Digraph G = (V, E), where V = {1, 2,, n}, with edge-weight function w: E. Output: n n matrix of shortest-path lengths δ(i, j) for all i, j V. IDEA: Run Bellman-Ford once from each vertex. Time = O(V 2 E). Dense graph (n 2 edges) Θ(n 4 ) time in the worst case.

172 Dynamic programming Consider the n n adjacency matrix A = (a ij ) of the digraph, and define d (m) ij = weight of a shortest path from i to j that uses at most m edges. Claim: We have d ij (0) = 0 if i = j, if i j. and for m = 1, 2,, n 1, d (m) = (m 1) ij min k {d ik + a kj }.

173 Proof of claim d ij (m) = min k {d ik (m 1) + a kj }. k's Relaxation! for k 1 to n do if d ij > d ik + a kj then d ij d ik + a kj i. j m 1 edges Note: No negative-weight cycles implies δ(i, j) = d (m 1) ij = d (m) ij = d (m + 1) ij =

174 All-pairs shortest paths example (1) D = (1) Π =

175 All-pairs shortest paths example (1) D = d (m) ij = min k {d (m 1) ik + a kj } d 14 (2) = min k {d ik (1) + a kj }. d 14 (2) = min{(d 11 (1) + a 14 ), (d 12 (1) + a 24 ), (d 13 (1) + a 34 ), (d 14 (1) + a 44 ), (d 15 (1) + a 54 )}. d 14 (2) = min{(0 + ), (3 + 1), (8 + ), ( + 0), ( 4 + 6)}= 2

176 All-pairs shortest paths example (2) D = (2) Π = Length 2

177 All-pairs shortest paths example (3) D = (3) Π = Length 3

178 All-pairs shortest paths example (4) D = (4) Π = Length 4

179 All-pairs shortest paths example (4) D = Π = Path 12 = (4) d (4) 12 = = 1

180 All-pairs shortest paths example (4) D = Π = Path 35 = (4) d (4) 35 = = 3

181 Matrix multiplication All-pairs shortest paths for graph G =(V (V, E). Dynamic programming: compute D ( V 1). d (m) =min{d ij k (m 1) ik + a kj }. Observe that if we make the substitutions D (m 1) a, D (0) b, D (m) c, min +, +. Problem change to compute C = A B, where C, A, and B are n n matrices: n c = a b ij ik kj k = 1

182 Matrix multiplication Consequently, we can compute D (1) = D (0) D (0) D (2) = D (1) D (0). D (n 1) = D (n 2) D (0) Yi ldi D (n 1) δ(i j) Ti Θ( 3 ) ( 4 ) N Yielding D (n 1) = δ(i, j). Time = Θ(n n 3 ) = (n 4 ). No better than running Bellman-Ford once from each vertex.

183 Improved matrix multiplication Repeated squaring: D (2k) = D (k) D (k). lg 1 2 n Compute D (2), D (4),, D. O(lgn) squarings Note: D (n 1) = D (n) = D (n + 1) =... Time = Θ(n 3 lgn). To detect negative-weight cycles, check the diagonal for negative values in O(n) additional time.

184 Floyd-Warshall algorithm Also dynamic programming, but faster! Define d (k) ij = weight of a shortest path from i to j with intermediate vertices belonging to the set {1, 2,, k}. Thus, δ(i, j) = d ij (n). Also, d ij (0) = w ij.

185 Floyd-Warshall algorithm all intermediate vertices in {1, 2,..., k 1} all intermediate vertices in {1, 2,..., k 1} k i j p: all intermediate vertices in {1, 2,..., k} w ij d (k) ij = min(dij (k 1), d (k 1) ik + d (k 1) kj ) d ij (m) = min k {d ik (m 1) + a kj }. if k = 0, if k 1.

186 Example of Floyd-Warshall algorithm (0) D = Initialization (0) Π =

187 Example of Floyd-Warshall algorithm (1) D = Node 1 (1) Π =

188 Example of Floyd-Warshall algorithm (2) D = Node 2 (2) Π =

189 Example of Floyd-Warshall algorithm (3) D = Node 3 (3) Π =

190 Example of Floyd-Warshall algorithm (4) D = Π = Node 4 (4) Path 13 = d 13 (4) = = 1

191 Example of Floyd-Warshall algorithm (5) D = Π = Node 5 (5)

192 Floyd-Warshall algorithm FLOYD-WARSHALL(W ) 1. n rows[w] 2. d 0 W 3. for k 1 to n 4. do for i 1 to n 5. do for j 1 to n 6. do d (k) ij min(d (k 1) ij, d (k 1) ik + d (k 1) kj ) 7. return D (n) Running time is Φ(n 3 )

193 Graph reweighting Is it any possible to change all negative-weight ihedges to nonnegative? Then, we can use Dijkstra's algorithm to compute the shortest path for all edges. Graph reweighting properties. For all pairs of vertices u,, v V,, a path p is a shortest path from u to v using weight function w if and only if p is also a shortest path from u to v using weight function ŵ after reweighting. For all edges (u, v), the new weight wuv ˆ (, ) is nonnegative.

194 Graph reweighting Theorem. Given a function h: V, reweight each edge (u, v) E by wuv ˆ (, ) = wuv (, ) + hu ( ) hv ( ). Then, for any two vertices, all paths between them are reweighted by the same amount. Proof. Let p = v 1 v 2... v k be a path in G. We have k wp ˆ( ) = wv ˆ( i 1, vi) i=11 k = ( wv ( i 1, vi) + hv ( i 1) hv ( i)) i=1= 1 k = + wv (, v) hv ( ) hv ( ) i 1 i 0 k i= 1 wp ( ) hv ( 0) hv ( k ) = + Same amount!

195 Graph reweighting

196 Graph reweighting Run Bellman-Ford algorithm for vertex v 0

197 Graph reweighting h(v k ) = δ(v 0, v k ) Run Bellman-Ford algorithm for vertex v 0

198 Graph reweighting h(v k ) = δ(v 0, v k ) wv ˆ (, v ) = wv (, v ) + hv ( ) hv ( ) i j i j i j Run Dijkstra's algorithm for each v k

199 Graph reweighting Why is wuv ˆ (, ) nonnegative? u s w(u, v) v Triangle inequality δ(s, v) δ(s, u) + w(u, v) w(u, v) + δ(s, u) δ(s, v) 0 h(v) ( ) = δ(s, (, v), h(u) ( ) = δ(s, (, u) ) wuv ˆ (, ) = wuv (, ) + hu ( ) hv ( ) 0

200 Johnson's algorithm JOHNSON(G ) 1. Compute G', where V[G'] = V[G] {s}, E[G'] ] = E[G] {(s, v): v V[G]}, and w(s, v) = 0 for all v V[G] 2. if BELLMAN-FORD(G', w, s) = FALSE 3. then print "the input graph contains a negative-weight cycle" 4. else for each vertex v V[G'] 5. do set h(v) to the value of δ(s, v) 6. for each edge (u, v) E[G'] 7. do wuv ˆ (, ) wuv (, ) + hu ( ) hv ( ) 8. for each vertex u V[G] 9. do run DIJKSTRA(G, ŵ,u) to compute ˆ( δ uv, ) for all v V[G]

201 Johnson's algorithm 10. for each vertex v V[G] 11. do duv ˆ( δ u, v ) + h ( v ) h ( u ) 12. return D

202 Analysis of Johnson's algorithm 1. Run Bellman-Ford to solve the difference constraints h(v) h(u) w(u, v), or determine that a negativeweight cycle exists. Time = O(VE). 2. Run Dijkstra's algorithm for each vertex u V[G]. Time = O(VElgV). 3. For each (u, v) E[G], compute δ(u, v). Time = O(E). Total time = O(VElgV). Johnson's algorithm is particularly suitable for sparse graph.

203 All-pairs shortest-paths algorithms Algorithms Time Data structure Brute-force (run Bellman- Ford once from each vertex) O(V 2 E ) Adjacency-list or adjacency-matrix Dynamic programming O(V 4 ) Adjacency-matrix only Improved dynamic Programming O(V 3 lgv ) Adjacency-matrix only Floyd-Warshall algorithm O(V 3 ) Adjacency-matrix only Johnson algorithm O(VElgV) Adjacency-list or adjacency-matrix

204 Flow networks Hangzhou u 12 Ningbo v Shanghai s t Hongkong x Nanjin y 14 Liangyungang

205 Flow networks Definition. A flow network is a directed graph G = (V, E) with two distinguished vertices: a source s and a sink t. Each edge (u, v) E has a nonnegative capacity c(u, v). ) If (u, v) ) E, then c(u, ( v) ) = 0.

206 Flow networks Definition. A positive flow on G is a function f: V V satisfying the following: Capacity constraint: For all u, v V, we require 0 f(u, v) ) c(u, ( v). ) Skew symmetry: For all u, v V, we require f(u, v) = f (v, u) Flow conservation:forallu all u V {s, t}, we require f( u, v) = 0 v V

207 A flow on a network Positive flow Capacity u 12/12 v s 0 1 1/ /4 7/ /7 t x 11/14 Flow conservation: Flow into x is 8+ 4= 12. Flow out of x is 1+ 11= 12. The value of this flow is = 19. y

208 Maximum-flow problem Maximum-flow problem. Given a flow network G, find a flow of maximum value on G. u 12/12 v s / 7 7/ t x 11/14 y The maximum flow is 23.

209 Cuts Definition. A cut (S, T) of a flow network G =(V, E) is a partition of V such chthat s S and t T. If f is a flow on G, then the flow across the cut is f(s, T). Maximum flow in a network is bounded by the capacity of minimum cut of the network. Why?

210 Cuts of flow networks u 12/12 v s 0 1 1/ /4 7/ /7 t x y 11/14 The net flow across this cut is f(u, v) + f(x, v) + f(x, y) = 12 + ( 4) + 11 = 19. and its capacity is c(u, v) + c(x, v) = = 26.

211 Flow cancellation Without loss of generality, positive flow goes either from u to v, or from v to u, but not both. u u Net flow from u to v in both cases is 1. 3/5 2/3 1/5 0/3 v The capacity constraint and flow conservation are preserved by this transformation. v

212 Residual network Definition. Let f be a flow on G = (V, E). The residual network G f (V, E f ) is the graph with strictly positive residual capacities c f (u, v)=c(u = c(u, v) f (u, v) > 0. Edges in E f admit more flow. 0/1 4 G f : G: u v u v 3/5 2

213 Augmenting paths Definition. Any path from s to t in G f is an augmenting path in G with respect to f. The flow value can be increased along an augmenting path p by c ( p ) = min { c ( u, v)} f ( uv, p) f

214 Example of maximum-flow problem u 12 v s 10 x 4 14 y 7 t c f (p) =min(16, 12, 9, 14, 4) = 4 u 4/12 v s t x 4/14 y

215 Example of maximum-flow problem u 4/12 v s t x u 4/ y v s t x 10 4 y

216 Example of maximum-flow problem u 8 4 v s 10 x u t c f (p) y =min(12, 10, 10, 7, 20) 4 = 7 8 Total v = = 11 s t x 3 11 y

217 Example of maximum-flow problem u 8 4 v s s 3 11 x u t c f (p) 3 y =min(13, 11, 8, 13) 11 = 8 12 Total v = = 19 7 t x 3 11 y

218 Example of maximum-flow problem u 12 v s 11 x u 3 7 c f (p) 3 y =min(5, 4, 5) 11 = 4 12 Total v = = 23 t s t x 3 11 y

219 Example of maximum-flow problem u 12 v s t x u 3 y 11 12/12 = 23 v The maximum flow s 10 1/4 7/7 t x 11/14 y

220 Ford-Fulkerson algorithm FORD-FULKERSON(G, FULKERSON(G s, t) 1. for each edge (u, v) E[G] 2. do f [u, v] ] 0 3. f [v, u] 0 4. while there exists a path p from s to t in the residual network G f 5. do c f (p) min{c f (u, v): ) (u, v) ) is in p} } 6. for each edge (u, v) in p 7. do f [u, v] ] f [u, v] ]+ c f (p) 8. f [v, u] f [v, u]

221 Analysis of Ford-Fulkerson algorithm Find a path in a residual netork is O(V + E) if we use either depth-first search or breadth-first search. f * denote the maximum flow found by the algorithm. Running time of Ford-Fulkerson algorithm is O(E f * ).

222 Ford-Fulkerson max-flow algorithm u u s 1 t s 0/1 t v v

223 Ford-Fulkerson max-flow algorithm u u s 0/1 t s 1/1 t v v

224 Ford-Fulkerson max-flow algorithm u u s 1/1 t s 0/1 t v v

225 Ford-Fulkerson max-flow algorithm u u s 0/1 t s 1/1 t v v

226 Ford-Fulkerson max-flow algorithm u u s 1/1 t s 0/1 t v v Running time is 10,000.

227 Edmonds-Karp algorithm Edmonds and Karp noticed that many people's implementations of Ford-Fulkerson augment along a breadth-first augmenting path: a shortest path in G f from s to t where each edge has weight 1. These implementations would always run relatively fast. Since a breadth-first augmenting path can be found in O(V + E) time, their analysis, which provided the first polynomial-time bound on maximum flow, focuses on bounding the number of flow augmentations. Edmonds-Karp algorithm's running time is O(VE 2 ).

228 Any yquestion? Xiaoqing i Zheng Fundan University