Vectorized Laplacians for dealing with high-dimensional data sets

Transcription

1 Vectorized Laplacians for dealing with high-dimensional data sets Fan Chung Graham University of California, San Diego The Power Grid Graph drawn by Derrick Bezanson 2010

2 Vectorized Laplacians Graph gauge theory Earlier work (3 papers ) with Shlomo Sternberg, Bert Kostant on the vibrational spectra and spins of the molecules. New directions in organizing/analyzing high-dimensional data sets, with applications in - vector diffusion maps, - graphics, imaging, sensing - resource allocation with dynamic demands Singer + Wu 2011, C. + Zhao 2012

3 Cayley graphs G = (V,E) Γ V! a group H =A 5 u! v! u = gv for g" B, a chosen subset of H. B={(12345),(54321),(12)(34)}!A 5

4 Cayley graphs G = (V,E) Γ V! a group H u! v! u = gv for g" B, a chosen subset of H. adjacency matrix! 0 0 irreducible representations Frobenius 1897

5 Eigenvalues of the buckyball with weight 1 for single bonds and t for double bonds Roots of with multiplicity 5 (x 2 + x t 2 + t 1)(x 3 tx 2 x 2 t 2 x 3x + t 3 t 2 + t + 2) = 0 (x 2 + x t 2 1)(x 2 + x (t +1) 2 ) = 0 with multiplicity 4 (x 2 + (2t +1)x + t 2 + t 1)(x 4 3x 3 + ( 2t 2 + t 1)x 2 + (3t 2 4t + 8)x + t 4 t 3 + t 2 + 4t 4) = 0 with multiplicity 3 x t 2 = 0 with multiplicity = 60

6

7 The spectrum of a graph Discrete Laplace operator 1! f ( x) = #( f ( x) " f ( y)) d x y! x 1 " f ( x ) =! f ( x )! f ( x ) j j j 2 For In a path P n + 1 {( )!( f ( x )! f ( x ))} j j! 1 2! " f ( x )!!! x 2 j " f ( x ) " f ( x ) j+ 1 j! x! x { }

8 In a graph Γ = (V,E), Δ acts on In general, we consider F (V,R) = { f :V R} Δ acts on F (V, X) = { f :V X}. geometric data time based data statistical data Example 1: Vibrational spectra of molecules F (V,R 3 ). Δ = Δ Example 2: Vector Diffusion Maps F (V,R d ). Δ = Δ nd nd V = n

9 Connections: a classical framework for dealing with Vibrations and spins of the Buckyball Noise in the data sets, etc.

10 Vector bundle E = { E x : x V}

11 Vector bundle Connection T E = { E x : x V}

12 Vector bundle E = { E x : x V} Connection T T x,e : E y E x for x V, e =(x,y) E T x,e T y,e = id Example 1: Vibrational spectra of molecules Spins of the Buckyball

13 Example 1: Vibrational spectra of molecules u h(v) h(u) v F (V,R 3 ) = {h :V R 3 } the space of displacements

14 Example 1: Vibrational spectra of molecules Hooke s law: h :V R 3 Potential energy u,v u h(u) v h(v) ( ) = k u,v u + h(u) v h(v) u v u,v k u,v w u,v (h(u) h(v)) = k < h(u) h(v), A (h(u) h(v)) > u,v e u,v where w u,v = u v u v, 2 A = w w t e u,v u,v 2

15 The vibrational spectrum (infrared) of the Buckyball

16 Vector bundle E = { E x : x V} Connection T T x,e : E y E x for x V, Selection sece s(u) T x,e T y,e = id E u for s sece e =(x,y) E Suppose a group G acts on V as graph automorphisms. For a G and u V, a : E u E au a b = ab : E u E abu G acts as automorphisms on connections: (a T ) = at x,e a 1 x,a 1 e a 1 x,e Define Q (s) = (s(x) T s(y)) A (s(x) T s(y)) T,A x,e x,e x,e

17 Example 1: Vibrations and spins on the Buckyball Theorem: C.+Sternberg 1992 In a homogenous graph Γ = a G-invariant connection on G/H with edge generating set B, E = E(Γ, X) defines an isomorphism t b Auto(X), such that the operator Q T,A (s) = x,e can be represented as F = (s(x) T s(y)) A (s(x) T s(y)) x,e x,e x,e b B A v,be I b B A v,be t b r(b) For the Buckyball G = SL(2,5), PSL(2,5) A 5, G Isotropy subgroup Z 2

18 Examples of connection graphs Example 1: Vibrational spectra of molecules Spins of the buckyball Example 2: Vector Diffusion Maps Singer + Wu 2011 rotation O uv data point u data point v

19 Examples of connection Laplacians Example 2: Vector Diffusion Maps Singer + Wu 2011 At a node u, Data local PCA dimension reduction vector in R d In a network, rotation O uv data point u data point v

20 Consistency of connection Laplacians For a connected connection graph G of dimension d, For f :V R d (i, j ) E < f, L f >= f (u)o uv f (v) 2 wuv

21 Consistency of connection Laplacians For a connected connection graph G of dimension d, the following statements are equivalent: The 0 eigenvalue has multiplicity d. All eigenvalues have multiplicity d. For any two vertices u and v, any two paths P, P, O xy = O zw xy P zw P' For each vertex v, we have O v SO(d), such that O = O 1 O uv u v for all (u,v) E.

22 Consistency of connection Laplacians Theorem: For a connected connection graph G of dimension d, if the 0 eigenvalue has multiplicity d, then we can find O v SO(d), such that O = O 1 O uv u v for all (u,v) E. Method 1: For a fixed vertex u, choose a frame O u consisting of any d orthonormal vectors. i For any v, define O v = O vi v i+1 is any path joining u and v. where u = v 0,v 1,...,v t Method 2: Use eigenvectors. Singer+Spielman 2012

23 Dealing with noise in the data Several combinatorial approaches Random connection Laplacians Graph sparsification algorithms PageRank algorithms Graph limits and graphlets...

24 Dealing with noise in the data Several combinatorial approaches Random connection Laplacians Graph sparsification algorithms PageRank algorithms Graph limits and graphlets...

25 A random graph model for any given expected degree sequence Independent random indicator variables X ij = X ji,1 i j n. Pr(X ij = 1) = Pr(v i v j ) = p ij ( ) X = X ij v i v j `` Example: The Erdos-Renyi model G(n, p) with p ij all equal to p.

26 A matrix concentration inequality Theorem: C. + Radcliffe 2011 X i : independent random Ahlswede-Winter,Cristofides- Markstrom, Oliveira, Gross, Recht, Tropp,... n n Hermitian matrix Then with X = m i=1 X i E(X i ) M X i satisfies Pr( X E(X ) > a) 2n exp m a 2 2v 2 + 2Ma / 3 where v 2 = var(x ) i i=1

27 Eigenvalues of random graphs with general distributions Theorem: C. + Radcliffe 2011 G : a random graph, where Pr(v i v j ) = p ij. A : A : adjacency matrix of G expected adjacency matrix A ij = p ij With probability, eigenvalues of A and satisfy 1 ε A for all i, λ i (A) λ i (A) 4d max log(2n / ε) 1 i n,if the maximum degree satisfies d max > 4 9 log( 2n ε ).

28 Eigenvalues of the normalized Laplacian for random graphs with general distributions Theorem: C. + Radcliffe 2011 G : a random graph, where Pr(v i v j ) = p ij. A : A : D adjacency matrix of G expected adjacency matrix diagonal degree matrix of G A ij = p ij With probability 1 ε, eigenvalues of L = I D 1/2 AD 1/2 satisfy for all i, 3log(4n / ε) λ (L) λ (L) i i δ 1 i n,and if the minimum degree satisfies L = I D 1/2 AD 1/2, δ > 10logn.

29 Dealing with noise in the data Several combinatorial approaches Random connection Laplacians Graph sparsification algorithms PageRank algorithms Graph limits and graphlets...

30 Graph sparsification For a graph G=(V,E,W), we say H is an ε-sparsifier if for all have h G (S) h H (S) εh G (S). S V we Benczur and Karger sparsifier with size O(n log n) in running time O(n 2 ) Spielman and Sriastava using effective resistance running time O(n) Batson,Spielman and Sriastava running time O(n) O(nm) sparsifier

31 Graph sparsification Sparsification Sampling edges e The cut edges are more likely to be sampled in order to satisfy, for all S V, h G (S) h H (S) εh G (S) We need to rank edges.

32 Dealing with noise in the data Several combinatorial approaches Random connection Laplacians Graph sparsification algorithms PageRank algorithms for ranking edges Graph limits and graphlets...

33 PageRank for ranking vertices Definition of PageRank Two equivalent ways to define PageRank p=pr(!,s) (1) p =! s + (1 "!) pw seed jumping constant (2) p "!! = $ # t= 0 t (1 ) ( sw ) t s = (,,..., ) n n n the (original) PageRank s = some seed, e.g., (1,0,...,0) personalized PageRank

34 PageRank as generalized Green s function Discrete Green s function Chung+Yau 2000 Laplacian n 1 L = D 1/2 ΔD 1/2 = λ k P k k =1 Green s function G = LG = I n 1 k =1 The general Green s function G β = 1 λ k P k restricted to n 1 k =1 1 β + λ k P k (ker L ) PageRank pr(α, s) = βsd 1/2 G β D 1/2, β = 2α 1 α

35 Ranking pages using Green s function and electrical network theory G = (V,E,W ) i V :V : a weighted graph injected current function w u,v conductance i E : E induced current function 1 if v is e's head B(e,v) = 1 if v is e's tail 0 otherwise Kirchoff s current law: Ohm s current law: i V = i E B i E = f B T W L = B T WB 1 u voltage potantial function v 1

36 Ranking pages using Green s function and electrical network theory Kirchoff s current law: Ohm s current law: i V = f V B T W B = f V L f V = i V D 1/2 G D 1/2 R(u,v) = f V (χ u χ v ) T β β i V = i E B B' = D1/2 B i E = f B T W β β = i V D 1/2 G D 1/2 (χ u χ v ) T β W β = βi 0 0 W L β = βi + L L G β β = I = (χ u χ v ) D 1/2 G D 1/2 (χ u χ v ) T β pr α (s) = sd 1/2 G β D 1/2 where β = 2α 1 α

37 Ranking pages using Green s function and electrical network theory The Green value of (u,v) R α (u,v) = h α (u,v) + h α (v,u) = pr α,u(u) d u pr α,u(v) d v + pr α,v(v) d v pr α,v(u) d u

38 Vectorized Laplacian and vectorized PageRank A connection graph G = (V,E,O,w) A connection matrix w O if (u,v) E, A = uv uv 0 otherwise. d d D(v,v) = d u I d d L = D A (u,v) E f Lf T = w uv f (u)o uv f (v) 2

39 Vectorized Laplacian and vectorized PageRank A connection graph G = (V,E,O,w) The transition probability matrix The vectorized PageRank P = D 1 A Z = I + P 2 ˆp(α,v) = α ŝ + (1 α) ˆpZ

40 Ranking pages using Green s function and electrical network theory The Green value of (u,v) R α (u,v) = ˆp α,u (u) d u ˆp α,u (v) d v + ˆp α,v (v) d v ˆp α,v (u) d u which the sampling subroutine is based upon.

41 Dealing with noise in the data Several combinatorial approaches Random connection Laplacians Graph sparsification algorithms PageRank algorithms for ranking edges Graph limits and graphlets...

42 Examples of connection Laplacians Example 1: Vibrational spectra of molecules Spins of the buckyball Example 2: Electron microscopy Vector Diffusion Maps Example 3: Graphics, vision,... Managing millions of images, photos.

43 Millions of photos in a box! Navigate using connection graphs. - finding consistent subgraphs, paths...

44 Feature detection Detect features using SIFT [Lowe, IJCV 2004]

45 Feature matching Match features between each pair of images

46 (graph layout produced using the Graphviz toolkit: Part of an image connection graph Building Rome in a day Agarwal, Snavely, Simon, Seitz, Szeliski 2009

47 Vectorized Laplacians for dealing with high-dimensional data sets Fan Chung Graham University of California, San Diego The Power Grid Graph drawn by Derrick Bezanson 2010

48 Thank you.

49 (graph layout produced using the Graphviz toolkit: Part of an image connection graph Building Rome in a day Agarwal, Snavely, Simon, Seitz, Szeliski 2009

50 An example of connection graphs Building Rome in a day Agarwal, Snavely, Simon, Seitz, Szeliski 2009