Sublinear Algorithms for Big Data

Transcription

1 Sublinear Algorithms for Big Data Qin Zhang 1-1

2 2-1 Part 2: Sublinear in Communication

3 Sublinear in communication The model x 1 = x 2 = x 3 = x k = Applicaitons They want to jointly compute f (x 1, x 2,..., x k ) Goal: minimize total bits of communication 3-1 etc.

4 A natrual approach The model x k = x 1 = x 2 = They want to jointly compute f (x 1, x 2,..., x k ) Goal: minimize total bits of communication The natural approach Coordinator x = C S 1 S 2 S 3 S k Each S i computes a skech of its input sk(s i ) and send it to C, and then C computes f (x 1,..., x k ) based on sk(s 1 ),..., sk(s k ) 4-1 The slides from next page are borrowed from Andrew McGregor

5 I. Connectivity II. k-connectivity III. Min-Cut

6 I. Connectivity II. k-connectivity III. Min-Cut Theorem: Testing Connectivity a) Dynamic Graph Stream: O(n polylog n) space. b) Simultaneous Messages: O(polylog n) length.

7 Ingredient 1: Basic Algorithm

8 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest):

9 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge

10 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge

11 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge

12 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge

13 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge 2.For each connected comp: pick incident edge

14 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge 2.For each connected comp: pick incident edge

15 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge 2.For each connected comp: pick incident edge

16 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge 2.For each connected comp: pick incident edge 3.Repeat until no edges between connected comp.

17 Ingredient 1: Basic Algorithm Algorithm (Spanning Forest): 1. For each node: pick incident edge 2.For each connected comp: pick incident edge 3.Repeat until no edges between connected comp. Lemma: After O(log n) rounds selected edges include spanning forest.

18 Ingredient 2: Sketching Neighborhoods

19 Ingredient 2: Sketching Neighborhoods For node i, let a i be vector indexed by node pairs. Non-zero entries: a i [i,j]=1 if j>i and a i [i,j]=-1 if j<i. {1,2} {1,3} {1,4} {1,5} {2,3} {2,4} {2,5} {3,4} {3,5} {4,5} a 1 = ( ) a 2 = ( )

20 Ingredient 2: Sketching Neighborhoods For node i, let a i be vector indexed by node pairs. Non-zero entries: a i [i,j]=1 if j>i and a i [i,j]=-1 if j<i. {1,2} {1,3} {1,4} {1,5} {2,3} {2,4} {2,5} {3,4} {3,5} {4,5} a 1 = ( ) a 2 = ( )

21 Ingredient 2: Sketching Neighborhoods For node i, let a i be vector indexed by node pairs. Non-zero entries: a i [i,j]=1 if j>i and a i [i,j]=-1 if j<i. {1,2} {1,3} {1,4} {1,5} {2,3} {2,4} {2,5} {3,4} {3,5} {4,5} a 1 = ( ) a 2 = ( ) a 1 +a 2 = ( )

22 Ingredient 2: Sketching Neighborhoods For node i, let a i be vector indexed by node pairs. Non-zero entries: a i [i,j]=1 if j>i and a i [i,j]=-1 if j<i. {1,2} {1,3} {1,4} {1,5} {2,3} {2,4} {2,5} {3,4} {3,5} {4,5} a 1 = ( ) a 2 = ( ) a 1 +a 2 = ( ) Lemma: For any subset of nodes S V, support ( i S a i ) = E(S,V \S)

23 Ingredient 2: Sketching Neighborhoods For node i, let a i be vector indexed by node pairs. Non-zero entries: a i [i,j]=1 if j>i and a i [i,j]=-1 if j<i. {1,2} {1,3} {1,4} {1,5} {2,3} {2,4} {2,5} {3,4} {3,5} {4,5} a 1 = ( ) a 2 = ( ) a 1 +a 2 = ( ) Lemma: For any subset of nodes S V, support ( i S a i ) = E(S,V \S) Lemma: random M: N k with k=o(polylog N) such that for any a N, with high probability Ma e support(a)

24 Recipe: Sketch & Compute on Sketches

25 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j

26 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j Central Player Runs Algorithm in Sketch Space:

27 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j Central Player Runs Algorithm in Sketch Space: Use Ma j to get incident edge on each node j

28 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j Central Player Runs Algorithm in Sketch Space: Use Ma j to get incident edge on each node j For i=2 to log n: To get incident edge on component S V use:

29 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j Central Player Runs Algorithm in Sketch Space: Use Ma j to get incident edge on each node j For i=2 to log n: To get incident edge on component S V use: j S Ma j = M( j S a j )

30 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j Central Player Runs Algorithm in Sketch Space: Use Ma j to get incident edge on each node j For i=2 to log n: To get incident edge on component S V use: a j ) e support( j S a j ) = E(S,V \S) j S Ma j = M( j S

31 Recipe: Sketch & Compute on Sketches Sketch: Each player sends Ma j Central Player Runs Algorithm in Sketch Space: Use Ma j to get incident edge on each node j For i=2 to log n: To get incident edge on component S V use: a j ) e support( j S a j ) = E(S,V \S) j S Ma j = M( j S Detail: Actually each player sends log n indept sketches M 1 a j, M 2 a j,... and central player uses M i a j when emulating i th iteration of the algorithm.

32 I. Connectivity II. k-connectivity III. Min-Cut

33 I. Connectivity II. k-connectivity III. Min-Cut Theorem: Checking every cut has size k a) Dynamic Graph Stream: O(n k polylog n) space. b) Simultaneous Messages: O(k polylog n) length.

34 Ingredient 1: Basic Algorithm

35 Ingredient 1: Basic Algorithm Algorithm (k-connectivity):

36 Ingredient 1: Basic Algorithm Algorithm (k-connectivity): 1. Let F 1 be spanning forest of G(V,E)

37 Ingredient 1: Basic Algorithm Algorithm (k-connectivity): 1. Let F 1 be spanning forest of G(V,E) 2.For i=2 to k: 2.1. Let F i be spanning forest of G(V,E-F F i-1 )

38 Ingredient 1: Basic Algorithm Algorithm (k-connectivity): 1. Let F 1 be spanning forest of G(V,E) 2.For i=2 to k: 2.1. Let F i be spanning forest of G(V,E-F F i-1 ) Lemma: G(V,F F k ) is k-connected iff G(V,E) is.

39 Ingredient 2: Connectivity Sketches

40 Ingredient 2: Connectivity Sketches Sketch: Simultaneously construct k independent connectivity sketches {M 1 G, M 2 G,... M k G}.

41 Ingredient 2: Connectivity Sketches Sketch: Simultaneously construct k independent connectivity sketches {M 1 G, M 2 G,... M k G}. Run Algorithm in Sketch Space: Use M 1 G to find a spanning forest F 1 of G

42 Ingredient 2: Connectivity Sketches Sketch: Simultaneously construct k independent connectivity sketches {M 1 G, M 2 G,... M k G}. Run Algorithm in Sketch Space: Use M 1 G to find a spanning forest F 1 of G Use M 2 G-M 2 F 1 =M 2 (G-F 1 ) to find F 2

43 Ingredient 2: Connectivity Sketches Sketch: Simultaneously construct k independent connectivity sketches {M 1 G, M 2 G,... M k G}. Run Algorithm in Sketch Space: Use M 1 G to find a spanning forest F 1 of G Use M 2 G-M 2 F 1 =M 2 (G-F 1 ) to find F 2 Use M 3 G-M 3 F 1 -M 3 F 2 =M 3 (G-F 1 -F 2 ) to find F 3

44 Ingredient 2: Connectivity Sketches Sketch: Simultaneously construct k independent connectivity sketches {M 1 G, M 2 G,... M k G}. Run Algorithm in Sketch Space: Use M 1 G to find a spanning forest F 1 of G Use M 2 G-M 2 F 1 =M 2 (G-F 1 ) to find F 2 Use M 3 G-M 3 F 1 -M 3 F 2 =M 3 (G-F 1 -F 2 ) to find F 3 etc.

45 I. Connectivity II. k-connectivity III. Min-Cut

46 I. Connectivity II. k-connectivity III. Min-Cut Theorem: (1+%)-approximating minimum cut a) Dynamic Graph Stream: O(% -2 n polylog n) space. b) Simultaneous Messages: O(% -2 polylog n) length.

47 Ingredient 1: Subsampling

48 Ingredient 1: Subsampling Lemma (Karger): Define subgraph G i by sampling edges w/p 2 -i. Then Min-Cut(G) = (1±ǫ) 2 i Min-Cut(G i ) if i < logp

49 Ingredient 1: Subsampling Lemma (Karger): Define subgraph G i by sampling edges w/p 2 -i. Then Min-Cut(G) = (1±ǫ) 2 i Min-Cut(G i ) if i < logp where p = 6ǫ 2 logn/min-cut(g)

50 Ingredient 1: Subsampling Lemma (Karger): Define subgraph G i by sampling edges w/p 2 -i. Then Min-Cut(G) = (1±ǫ) 2 i Min-Cut(G i ) if i < logp where p = 6ǫ 2 logn/min-cut(g) G=G 0

51 Ingredient 1: Subsampling Lemma (Karger): Define subgraph G i by sampling edges w/p 2 -i. Then Min-Cut(G) = (1±ǫ) 2 i Min-Cut(G i ) if i < logp where p = 6ǫ 2 logn/min-cut(g) G=G 0 G 1

52 Ingredient 1: Subsampling Lemma (Karger): Define subgraph G i by sampling edges w/p 2 -i. Then Min-Cut(G) = (1±ǫ) 2 i Min-Cut(G i ) if i < logp where p = 6ǫ 2 logn/min-cut(g) G=G 0 G 1 G 2

53 Ingredient 1: Subsampling Lemma (Karger): Define subgraph Gi by sampling edges w/p 2-i. Then Min-Cut(G ) = (1 ± ǫ) 2i Min-Cut(Gi ) if i < log p where p = 6ǫ 2 log n/min-cut(g) G=G0 G1 G2 G3

54 Ingredient 1: Subsampling Lemma (Karger): Define subgraph Gi by sampling edges w/p 2-i. Then Min-Cut(G ) = (1 ± ǫ) 2i Min-Cut(Gi ) if i < log p where p = 6ǫ 2 log n/min-cut(g) G=G0 G1 G2 G3 Suffices to find Min-Cut(Gi) for some i<-log p*.

55 Ingredient 2: k-connectivity

56 Ingredient 2: k-connectivity k-connectivity: Given G i returns subgraph H i with Min-Cut(G i ) = Min-Cut(H i ) if Min-Cut(G i ) < k

57 Ingredient 2: k-connectivity k-connectivity: Given G i returns subgraph H i with Min-Cut(G i ) = Min-Cut(H i ) if Min-Cut(G i ) < k Lemma: For k = 12% -2 log n, with high probability Min-Cut(G i ) < k for i = logp

58 Ingredient 2: k-connectivity k-connectivity: Given G i returns subgraph H i with Min-Cut(G i ) = Min-Cut(H i ) if Min-Cut(G i ) < k Lemma: For k = 12% -2 log n, with high probability Min-Cut(G i ) < k for i = logp since expectation of Min-Cut(G i ) is < 6% -2 log n.

59 Ingredient 2: k-connectivity k-connectivity: Given G i returns subgraph H i with Min-Cut(G i ) = Min-Cut(H i ) if Min-Cut(G i ) < k Lemma: For k = 12% -2 log n, with high probability Min-Cut(G i ) < k for i = logp since expectation of Min-Cut(G i ) is < 6% -2 log n. Putting it together: Construct H i for all i. Return 2 i Min-Cut(H i ) for smallest i with Min-Cut(H i ) < k.

60 Algorithm for Min-Cut 1. For i = {1,..., 2 log n}, let h i {0, 1} be a uniform hash function. 2. For i = {1,..., 2 log n}, (a) Let G i be the subgraph of G containing edges e such that Π j i h j (e) = 1. (b) Let H i k-connected(g i ) for k = O(ɛ 2 log n). 3. Return 2 j Min-Cut(H j ), where j = min{i : Min-Cut(H i ) < k} 1-1

61 Example: Checking Bipartiteness

62 Example: Checking Bipartiteness Idea: Given G, define G where a node v becomes v 1 and v 2 and edge (u,v) becomes (u 1,v 2 ) and (u 2,v 1 ).

63 Example: Checking Bipartiteness Idea: Given G, define G where a node v becomes v 1 and v 2 and edge (u,v) becomes (u 1,v 2 ) and (u 2,v 1 ).

64 Example: Checking Bipartiteness Idea: Given G, define G where a node v becomes v 1 and v 2 and edge (u,v) becomes (u 1,v 2 ) and (u 2,v 1 ). Lemma: G is bipartite iff number of connected components doubles. Can sketch G implicitly.

65 Example: Checking Bipartiteness Idea: Given G, define G where a node v becomes v 1 and v 2 and edge (u,v) becomes (u 1,v 2 ) and (u 2,v 1 ). Lemma: G is bipartite iff number of connected components doubles. Can sketch G implicitly.

66 Example: Checking Bipartiteness Idea: Given G, define G where a node v becomes v 1 and v 2 and edge (u,v) becomes (u 1,v 2 ) and (u 2,v 1 ). Lemma: G is bipartite iff number of connected components doubles. Can sketch G implicitly. Thm: Õ(n)-dimensional sketch for bipartiteness.

67 Example: Minimum Spanning Tree

68 Example: Minimum Spanning Tree Idea: Let n i be number of connected components if we ignore edges with weight (1+ ) i, then: w(mst) i ɛ(1 + ɛ) i n i (1 + ɛ)w(mst)

69 Example: Minimum Spanning Tree Idea: Let n i be number of connected components if we ignore edges with weight (1+ ) i, then: w(mst) i ɛ(1 + ɛ) i n i (1 + ɛ)w(mst) Thm: Can (1+ ) approximate MST in one-pass dynamic semi-streaming model.

70 Algorithm for Sparsification λ e (G): size of the minimum cut for each edge e = (u, v) in G 1. For i = {1,..., 2 log n}, let h i {0, 1} be a uniform hash function. 2. For i = {1,..., 2 log n}, (a) Let G i be the subgraph of G containing edges e such that Π j i h j (e) = 1. (b) Let H i k-connected(g i ) for k = O(ɛ 2 log 2 n). 3. For each edge e = (u, v), find j = min{i : λ e (H i ) < k}. If e H j, add e to the sparsifier with weight 2 j. Azuma s inequality A sequence of random variables X 1, X 2,... is called a martingale is for all i 1, E[X i+1 X i ] = X i. If X i+1 X i c i almost surely for all i, then 2-1 Pr[ X n X 1 t] < 2e t 2 2 n 1 i=1 c2 i.