Network Topology Inference from Non-stationary Graph Signals

Size: px

Start display at page:

Download "Network Topology Inference from Non-stationary Graph Signals"

Frederick Riley
5 years ago
Views:

1 Network Topology Inference from Non-stationary Graph Signals Rasoul Shafipour Dept. of Electrical and Computer Engineering University of Rochester Co-authors: Santiago Segarra, Antonio G. Marques and Gonzalo Mateos ICASSP, March 6, 2017 Network Topology Inference from Non-stationary Graph Signals ICASSP

2 Network Science analytics Online social media Internet Clean energy and grid analy,cs Network as graph G = (V, E): encode pairwise relationships Desiderata: Process, analyze and learn from network data [Kolaczyk 09] Network Topology Inference from Non-stationary Graph Signals ICASSP

3 Network Science analytics Online social media Internet Clean energy and grid analy,cs Network as graph G = (V, E): encode pairwise relationships Desiderata: Process, analyze and learn from network data [Kolaczyk 09] Interest here not in G itself, but in data associated with nodes in V The object of study is a graph signal Ex: Opinion profile, buffer congestion levels, neural activity, epidemic Network Topology Inference from Non-stationary Graph Signals ICASSP

4 Graph signal processing (GSP) Undirected G with adjacency matrix A A ij = Proximity between i and j Define a signal x on top of the graph x 1 1 x x 4 x i = Signal value at node i 3 5 x 3 x 5 Network Topology Inference from Non-stationary Graph Signals ICASSP

5 Graph signal processing (GSP) Undirected G with adjacency matrix A A ij = Proximity between i and j Define a signal x on top of the graph x 1 1 x x 4 x i = Signal value at node i 3 5 x 3 x 5 Associated with G is the graph-shift operator S = VΛV T M N S ij = 0 for i j and (i, j) E (local structure in G) Ex: A, degree D and Laplacian L = D A matrices Network Topology Inference from Non-stationary Graph Signals ICASSP

6 Graph signal processing (GSP) Undirected G with adjacency matrix A A ij = Proximity between i and j Define a signal x on top of the graph x 1 1 x x 4 x i = Signal value at node i 3 5 x 3 x 5 Associated with G is the graph-shift operator S = VΛV T M N S ij = 0 for i j and (i, j) E (local structure in G) Ex: A, degree D and Laplacian L = D A matrices Graph Signal Processing Exploit structure encoded in S to process x Our view: GSP well suited to study (network) diffusion processes Take the reverse path. How to use GSP to infer the graph topology? Network Topology Inference from Non-stationary Graph Signals ICASSP

7 Topology inference: Motivation and context Network topology inference from nodal observations [Kolaczyk 09] Partial correlations and conditional dependence [Dempster 74] Sparsity [Friedman et al 07] and consistency [Meinshausen-Buhlmann 06] Key in neuroscience [Sporns 10] Functional net inferred from activity Network Topology Inference from Non-stationary Graph Signals ICASSP

Topology inference: Motivation and context Network topology inference from nodal observations [Kolaczyk 09] Partial correlations and conditional dependence [Dempster 74] Sparsity [Friedman et al 07]

8 Topology inference: Motivation and context Network topology inference from nodal observations [Kolaczyk 09] Partial correlations and conditional dependence [Dempster 74] Sparsity [Friedman et al 07] and consistency [Meinshausen-Buhlmann 06] Key in neuroscience [Sporns 10] Functional net inferred from activity Noteworthy GSP-based approaches Gaussian graphical models [Egilmez et al 16] Smooth signals [Dong et al 15], [Kalofolias 16] Stationary signals [Pasdeloup et al 15], [Segarra et al 16] Directed graphs [Mei-Moura 15], [Shen et al 16] Our contribution: topology inference from non-stationary graph signals Network Topology Inference from Non-stationary Graph Signals ICASSP

9 Generating structure of a diffusion process Signal y is the response of a linear diffusion process to an input x y = α 0 (I α l S)x = l=1 β l S l x l=0 Common generative model. Heat diffusion if α k constant We say the graph shift S explains the structure of signal y Network Topology Inference from Non-stationary Graph Signals ICASSP

10 Generating structure of a diffusion process Signal y is the response of a linear diffusion process to an input x y = α 0 (I α l S)x = l=1 β l S l x l=0 Common generative model. Heat diffusion if α k constant We say the graph shift S explains the structure of signal y Cayley-Hamilton asserts we can write diffusion as y = ( N 1 ) h l S l x := Hx l=0 Graph filter H is shift invariant [Sandryhaila-Moura 13] H diagonalized by the eigenvectors V of the shift operator Network Topology Inference from Non-stationary Graph Signals ICASSP

11 Our approach for topology inference I Two-step approach for graph topology identification Signal realizations or their statistics Step 1: Identify the eigenvectors of the shift I Inferred network S A priori info, desired topological features Inferred eigenvectors V Step 2: Identify eigenvalues to obtain a suitable shift Beyond diffusion Alternative sources for spectral templates V I I Design of graph filters [Segarra et al 15] Graph sparsification and network deconvolution [Feizi et al 13] Network Topology Inference from Non-stationary Graph Signals ICASSP

12 Step 2: Obtaining the eigenvalues We can use extra knowledge/assumptions to choose one graph Of all graphs, select one that is optimal in some sense S := argmin S,λ N f (S, λ) s. to S = λ k v k vk T, S S k=1 Set S contains all admissible scaled adjacency matrices S :={S S ij 0, S M N, S ii = 0, j S 1j =1} Can accommodate Laplacian matrices as well Network Topology Inference from Non-stationary Graph Signals ICASSP

13 Step 2: Obtaining the eigenvalues We can use extra knowledge/assumptions to choose one graph Of all graphs, select one that is optimal in some sense S := argmin S,λ N f (S, λ) s. to S = λ k v k vk T, S S k=1 Set S contains all admissible scaled adjacency matrices S :={S S ij 0, S M N, S ii = 0, j S 1j =1} Can accommodate Laplacian matrices as well Problem is convex if we select a convex objective f (S, λ) Ex: Sparsity (f (S) = S 1), min. energy (f (S) = S F ), mixing (f (λ) = λ 2) Robust recovery from imperfect or incomplete ˆV [Segarra et al 16] Network Topology Inference from Non-stationary Graph Signals ICASSP

14 Step 1: Obtaining the eigenvectors Stationary graph signal [Marques et al 16] Def: A graph signal y is stationary with respect to the shift S if and only if y = Hx, where H = L 1 l=0 h ls l and x is white. Network Topology Inference from Non-stationary Graph Signals ICASSP

15 Step 1: Obtaining the eigenvectors Stationary graph signal [Marques et al 16] Def: A graph signal y is stationary with respect to the shift S if and only if y = Hx, where H = L 1 l=0 h ls l and x is white. The covariance matrix of the stationary signal y is [ C y = E Hx ( Hx ) ] T = HE [ xx T ] H T = HH T Key: Since H is diagonalized by V, so is the covariance C y C y L 1 2 = V h l Λ l V T l=0 Estimate V from {y i } via Principal Component Analysis Network Topology Inference from Non-stationary Graph Signals ICASSP

16 Non-stationary graph signals Q: What if the signal y = Hx is not stationary (i.e., x colored)? Matrices S and C y no longer simultaneously diagonalizable since C y = HC x H T Network Topology Inference from Non-stationary Graph Signals ICASSP

17 Non-stationary graph signals Q: What if the signal y = Hx is not stationary (i.e., x colored)? Matrices S and C y no longer simultaneously diagonalizable since C y = HC x H T Key: still H = L 1 l=0 h ls l diagonalized by the eigenvectors V of S Infer V by estimating the unknown diffusion (graph) filter H Step 1 boils down to system identification + eigendecomposition Leverage different sources of information on the input signal x (a) Input-output graph signal realization pairs {y m, x m} (b) Input covariance C x and positive semidefinite filter H 0 (c) Input covariance C x and generic filter H Network Topology Inference from Non-stationary Graph Signals ICASSP

18 Input-output graph signal realization pairs Consider M diffusion processes on G, where y m = Hx m (x m colored) Assume that realizations {y m, x m } M m=1 are available Filter H and, as byproduct, its eigenvectors V can be estimated as Ĥ = argmin H M y m Hx m 2 m=1 Define X = [x 1,..., x M ] and Y = [y 1,..., y M ]. Then, Ĥ given by vec(ĥ) = ( (X T ) I N ) vec(y) If M N and X is full rank, the minimizer Ĥ is unique Network Topology Inference from Non-stationary Graph Signals ICASSP

19 Inferring a brain network Recovery Error Recovery Error Consider a structural brain graph with N = 66 neural regions Signals diffused either by H1 = 2 l=0 h la l or H 2 = (I + αa) 1 Observe realizations {ym, x m} M m=1 and vary M Also noisy signals y m = H i x m + w m, with w m N (0, 10 2 I) Noisy H1 Noisy H Noiseless H1 Noiseless H M M Recovery error A Â F / A F small for M 66, even with noise Performance roughly independent of the filter type Network Topology Inference from Non-stationary Graph Signals ICASSP

20 Input covariance and positive semidefinite filters Realizations of the input may be challenging to acquire Consider instead that C x,m = E[x m x m T ] are known Estimate output covariance Ĉy,m from realizations {y (p) m } Pm p=1 Goal is to find H such that Ĉy,m and HC x,m H T are close Least squares yields a fourth-order cost in H Challenging Network Topology Inference from Non-stationary Graph Signals ICASSP

21 Input covariance and positive semidefinite filters Realizations of the input may be challenging to acquire Consider instead that C x,m = E[x m x m T ] are known Estimate output covariance Ĉy,m from realizations {y (p) m } Pm p=1 Goal is to find H such that Ĉy,m and HC x,m H T are close Least squares yields a fourth-order cost in H Challenging Assume H is PSD, e.g, in Laplacian diffusion y = ( l=0 β ll l )x, β l > 0 Well-defined square roots, hence H can be identified as Ĥ = argmin H M N ++ M (C 1/2 x,m Ĉ y,m C 1/2 x,m ) 1/2 C 1/2 x,m HC 1/2 x,m 2 F m=1 If C y,1 known, even with M = 1 PSD assumption renders H identifiable Network Topology Inference from Non-stationary Graph Signals ICASSP

22 Inferring Zachary s karate club network Recovery Error Social network with N = 34 club members Model opinion diffusion with S = I αl, where α = λ 1 max(l) For M = 1, 5, 10 input covariances Cx,m assumed given Estimate Cy,m from {y m (p) } Pm p=1 via sample averaging, varying Pm M=1 M=5 M= Number of Observations With imperfect estimates Ĉy,m, performance improves with M Network Topology Inference from Non-stationary Graph Signals ICASSP

23 Input covariance and generic filters Q: What about identifying a generic symmetric filter H? Filter is no longer PSD, square roots not prudent Try to solve Ĥ = argmin H M N M Ĉ y,m HC x,m H T 2 F m=1 Non-convex problem can be tackled by gradient descent or ADMM {H L, H R} = argmin H L,H R M N M C y,m H L C x,m H T R 2 F m=1 s. to H L = H R In general, identifiability cannot be guaranteed. Larger M helps Network Topology Inference from Non-stationary Graph Signals ICASSP

24 Inferring a brain network Recovery Error Consider a structural brain graph with N = 66 neural regions Signals diffused by H = 2 l=0 h la l, h l U[0, 1] Performance comparison against counterpart in [Segarra et al 16] Assumes ym stationary Estimates V directly from Ĉy,m Proposed Method Stationary Method M Error decays with M, almost all edges in S recovered for M = 9 Outperforms algorithm agnostic to signal non-stationarities Network Topology Inference from Non-stationary Graph Signals ICASSP

25 Closing remarks Network topology inference from diffused non-stationary graph signals Graph shift S and covariance Cy are not simultaneously diagonalizable Diffusion filter H and graph shift S still share spectral templates V Two step approach for topology inference i) Obtain Ĥ ˆV; ii) Given ˆV, estimate Ŝ via convex optimization Estimate Ĥ under different settings Input-output graph signal realization pairs {ym, x m} Input covariance Cx and positive semidefinite filter H 0 Input covariance Cx and generic filter H Network Topology Inference from Non-stationary Graph Signals ICASSP

26 Closing remarks Network topology inference from diffused non-stationary graph signals Graph shift S and covariance Cy are not simultaneously diagonalizable Diffusion filter H and graph shift S still share spectral templates V Two step approach for topology inference i) Obtain Ĥ ˆV; ii) Given ˆV, estimate Ŝ via convex optimization Estimate Ĥ under different settings Input-output graph signal realization pairs {ym, x m} Input covariance Cx and positive semidefinite filter H 0 Input covariance Cx and generic filter H Ongoing work and future directions Identifiability and convergence guarantees for generic H Extensions to directed graphs Inference of time-varying networks Network Topology Inference from Non-stationary Graph Signals ICASSP

Blind Identification of Invertible Graph Filters with Multiple Sparse Inputs 1

Blind Identification of Invertible Graph Filters with Multiple Sparse Inputs Chang Ye Dept. of ECE and Goergen Institute for Data Science University of Rochester cye7@ur.rochester.edu http://www.ece.rochester.edu/~cye7/