Jaccard filtering on two Network Datasets

Transcription

1 Jaccard filtering on two Network Datasets Minghao Tian Final Report for CSE9 May, 16 Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 1 / 1

2 Jaccard Index Given an arbitrary graph G, let Nu G denote the set of immediate neighbors of u (i.e, nodes connected to u V (G) by edges in E(G)). Given any edge (u, v) E(G), the Jaccard index ρ u,v of this edge is defined as ρ u,v (G) = NG u N G v N G u N G v. Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 / 1

3 τ-jaccard filtering Given graph Ĝ, for each edge (u, v) E(Ĝ), we insert the edge (u, v) into E( G τ ) if and only if ρ u,v (Ĝ) τ. That is, V ( G τ ) = V (Ĝ) and E( G τ ) := {(u, v) E(Ĝ) ρu,v(ĝ) τ}. Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 3 / 1

4 The conclusion I want to derive Jaccard-filtering may be robust to Erdős-Rényi type perturbation for some datasets. Erdős-Rényi type perturbation: with probability p we delete the existed edges in G and with probability q we insert the non-existed edges to G. Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 4 / 1

5 Experiment Design For a network G, 1 Apply Erdős-Rényi kind perturbation to G and derive Ĝ Apply τ-jaccard filtering to G and Ĝ and derive G τ and Ĝτ 3 Uniformly sampling N points in G and get the N N shortest path distance matrix M G induced by the whole graph G. Do the same thing to the corresponding N points in Ĝ and derive M Ĝ. Also uniformly sampling N points in G τ and do the same things to G τ and Ĝτ, thus we get M Gτ and MĜτ Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 / 1

6 The distance-matrix-to-rips-complexes Algorithm Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 6 / 1

7 The distance-matrix-to-rips-complexes Algorithm (conti.) Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 7 / 1

8 Experiment Design (conti.) 4 Use the distance-matrix-to-rips-complexes algorithm and Gudhi to get the up-to-1 dimension barcodes B G, BĜ, B Gτ Compute the 1-Wasserstein distances D w,1 (B i G, Bi Ĝ ), and BĜτ D w,1 (B i G τ, B i Ĝ τ ) and Bottleneck distances D w, (B i G, Bi Ĝ ), D w, (B i G τ, B i Ĝ τ ) where i {, 1} indicates the dimension of the topological features 6 Expect to see that D w,1 (B i G τ, B i Ĝ τ ) and D w, (B i G τ, B i Ĝ τ ) are significantly smaller than the corresponding ones between G and Ĝ Also, we do not want Jaccard-filtering kill too many edges. One way to see this is that we show you the persistent diagrams of the these graphs and the ones after Jaccard-filtering still obtain many topological features. Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 8 / 1

9 Facebook combined 1 (Undirected, Nodes: 439, Edges: 8834) We choose p =., q =.1 and τ = O v.s P D W, D W, DJ v.s JAP D W, D W, Table : 1-dimension Facebook combined with p =., q =.1, τ =.4 and N = 1. O represents the original graph, P represents the perturbed one, DJ represents the one after directly Jaccard-filtering and JAP represents the one doing Jaccard-filtering after perturbation. 1 Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 9 / 1

10 Facebook combined(undirected, Nodes: 439, Edges: 8834) (conti.) barcode_n_1_original_facebook barcode_n_1_perturb_facebook_p_._q_.1 x x Figure : Left: The persistent diagram of the Facebook combined dataset; Right: The one after Erdős-Rényi type perturbation; Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 1 / 1

11 Facebook combined(undirected, Nodes: 439, Edges: 8834) (conti.) barcode_n_1_jaccard_filtering_directly_facebook_j_.4 barcode_n_1_jaccard_filtering_after_perturb_facebook_p_._q_.1_j_.4 x 1 4 x Figure : Left: The one using Jaccard-filtering directly; Right: The one using Jaccard-filtering after perturbation; Minghao Tian (Final Report for CSE9) Jaccard filtering May, / 1

12 Twitter combined (Directed, Nodes: 8136, Edges: ) We view this directed graph as undirected one and choose p =., q =. and τ = O v.s P D W, D W, DJ v.s JAP D W, 6 D W, Table : 1-dimension Twitter combined with p =., q =., τ =.1 and N = 1 Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 1 / 1

13 Twitter combined 3 (Directed, Nodes: 8136, Edges: ) (conti.) barcode_n_1_original_twitter_combined barcode_n_1_perturb_twitter_combined_p_._q_. x x Figure : Left: The persistent diagram of the Twitter combined dataset; Right: The one after Erdős-Rényi type perturbation; Minghao 3 Tian (Final Report CSE9) Jaccard filtering May, / 1

14 Twitter combined 4 (Directed, Nodes: 8136, Edges: ) (conti.) barcode_n_1_jaccard_filtering_directly_twitter_combined_j_.1 barcode_n_1_jaccard_filtering_after_perturb_twitter_combined_p_._q_._j_.1 x 1 4 x Figure : Left: The one using Jaccard-filtering directly; Right: The one using Jaccard-filtering after perturbation; Minghao 4 Tian (Final Report CSE9) Jaccard filtering May, / 1

15 The End Minghao Tian (Final Report for CSE9) Jaccard filtering May, 16 1 / 1