Digraph Fourier Transform via Spectral Dispersion Minimization

Transcription

1 Digraph Fourier Transform via Spectral Dispersion Minimization Gonzalo Mateos Dept. of Electrical and Computer Engineering University of Rochester Co-authors: Rasoul Shafipour, Ali Khodabakhsh, and Evdokia Nikolova Acknowledgment: NSF Award CCF Calgary, AB, April 20, 2018 Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

2 Network Science analytics Online social media Internet Clean energy and grid analy,cs Network as graph G = (V, E): encode pairwise relationships Desiderata: Process, analyze and learn from network data [Kolaczyk 09] Interest here not in G itself, but in data associated with nodes in V The object of study is a graph signal Ex: Opinion profile, buffer levels, neural activity, epidemic Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

3 Graph signal processing and Fourier transform Directed graph (digraph) G with adjacency matrix A A ij = Edge weight from node i to node j Define a signal x R N on top of the graph x i = Signal value at node i Associated with G is the underlying undirected G u Laplacian marix L = D A u, eigenvectors V = [v 1,, v N ] Graph Signal Processing (GSP): exploit structure in A or L to process x Graph Fourier Transform (GFT): x = V T x for undirected graphs Decompose x into different modes of variation Inverse (i)gft x = V x, eigenvectors as frequency atoms Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

4 GFT: Motivation and context Spectral analysis and filter design [Tremblay et al 17], [Isufi et al 16] Promising tool in neuroscience [Huang et al 16] Graph frequency analyses of fmri signals Noteworthy GFT approaches Eigenvectors of the Laplacian L [Shuman et al 13] Jordan decomposition of A [Sandryhaila-Moura 14], [Deri-Moura 17] Lovaśz extension of the graph cut size [Sardellitti et al 17] Greedy basis selection for spread modes [Shafipour et al 17] Generalized variation operators and inner products [Girault et al 18] Our contribution: design a novel digraph (D)GFT such that Bases offer notions of frequency and signal variation Frequencies are (approximately) equidistributed in [0, fmax] Bases are orthonormal, so Parseval s identity holds Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

5 Signal variation on digraphs Total variation of signal x with respect to L TV(x) = x T Lx = N i,j=1,j>i Smoothness measure on the graph G u A u ij(x i x j ) 2 For Laplacian eigenvectors V = [v 1,, v N ] TV(v k ) = λ k 0 = λ 1 < λ N can be viewed as frequencies Def: Directed variation for signals over digraphs ([x] + = max(0, x)) DV(x) := N A ij [x i x j ] 2 + i,j=1 Captures signal variation (flow) along directed edges Consistent, since DV(x) TV(x) for undirected graphs Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

6 DGFT with spread frequeny components Goal: find N orthonormal bases capturing different modes of DV on G Collect the desired bases in a matrix U = [u 1,, u N ] R N N DGFT: x = U T x u k represents the kth frequency mode with f k := DV(u k ) Similar to the DFT, seek N equidistributed graph frequencies f k = DV(u k ) = k 1 N 1 f max, k = 1,..., N f max is the maximum DV of a unit-norm graph signal on G Q: Why spread frequencies? Parsimonious representations of slowly-varying signals Interpretability better capture low, medium, and high frequencies Aid filter design in the graph spectral domain Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

7 Motivation for spread frequencies Ex: Directed variation minimization [Sardellitti et al 17] min U N i,j=1 A ij[u i u j ] + s.t. U T U = I U is the optimum basis where a = , b = 1 5 4, and c = 0.5 All columns of U satisfy DV(u k ) = 0, k = 1,..., 4 Expansion x = U x fails to capture different modes of variation Q: Can we always find equidistributed frequencies in [0, f max ]? Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

8 Challenges: Maximum directed variation Finding f max is in general challenging u max = argmax u =1 DV(u) and f max := DV(u max ). Let v N be the dominant eigenvector of L Can 1/2-approximate f max with ũ max = argmax DV(v) v {v N, v N } f max can be obtained analytically for particular graph families Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

9 Challenges: Equidistributed frequencies Equidistributed f k = k 1 N 1 f max may not be feasible. Ex: In undirected G u f u max = λ max and N N f k = TV(v k ) = trace(l) k=1 k=1 Idea: Set u 1 = u min := 1 N 1 N and u N = u max and minimize N 1 δ(u) := [DV(u i+1 ) DV(u i )] 2 i=1 δ(u) is the spectral dispersion function Minimized when the free DV values form an arithmetic sequence Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

10 Spectral dispersion minimization We cast the optimization problem of finding spread frequencies as min U N 1 [DV(u i+1 ) DV(u i )] 2 i=1 subject to U T U = I u 1 = u min u N = u max Non-convex, orthogonality-constrained minimization of smooth δ(u) Feasible since umax u min Adopt a feasible method in the Stiefel manifold to design the DGFT: (i) Obtain f max (and u max) by minimizing DV(u) over {u u T u = 1} (ii) Find the orthonormal basis U with minimum spectral dispersion Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

11 Feasible method in the Stiefel manifold Rewrite the problem of finding orthonormal basis as min U subject to φ(u) := δ(u) + λ 2 U T U = I ( u1 u min 2 + u N u max 2) Let U k be a feasible point at iteration k and the gradient G k = φ(u k ) Skew-symmetric matrix B k := G k U k T U k G k T Follow the update rule U k+1 (τ) = ( I + τ 2 B k) 1 ( I τ 2 B k) Uk Cayley transform preserves orthogonality (i.e., T Uk+1 U k+1 = I) Is a descent path for a proper step size τ Theorem (Wen-Yin 13) The procedure converges to a stationary point of smooth φ(u), while generating feasible points at every iteration Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

12 Algorithm 1: Input: Adjacency matrix A, parameters λ > 0 and ɛ > 0 2: Find u max by a similar feasible method and set u min = 1 N 1 N 3: Initialize k = 0 and orthonormal U 0 R N N at random 4: repeat 5: Compute gradient G k = φ(u k ) R N N 6: Form B k = G k U k T U k G k T 7: Select τ k satisfying Armijo-Wolfe conditions 8: Update U k+1 (τ k ) = (I + τ k 2 B k ) 1 (I τ k 2 B k )U k 9: k k : until U k U k 1 F ɛ 11: Return Û = U k Overall run-time is O(N 3 ) per iteration Additional details in arxiv: [eess.sp] Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

13 Numerical test: Synthetic graph Compute U and directed variations using Directed Laplacian eigenvectors [Chung 05] PAMAL method [Sardellitti et al 17] Greedy heuristic [Shafipour et al 17] Spectral dispersion minimization Rescale DV values to [0, 1] and calculate spectral dispersion δ(u) 0.256, 0.301, 0.118, and respectively Confirms the proposed method yields a better frequency spread Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

14 Numerical test: US average temperatures Consider the graph of the N = 48 contiguous United States Connect two states if they share a border Set arc directions from lower to higher latitudes Graph signal x Average annual temperature of each state Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

15 Numerical test: Denoising US temperatures Noisy signal y = x + n, with n N (0, 10 I N ) Define low-pass filter H = diag( h), where h i = I {i w} (for w = 3) Recover signal via filtering ˆx = U Hỹ = U HU T y Compute recovery error e f = ˆx x x 12% Reverse the edge orientations and repeat the experiment Feasible Method: N-S Feasible Method: S-N DGFT basis offers a parsimonious (i.e., bandlimited) signal representation Adequate network model improves the denoising performance Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

16 Closing remarks Measure of directed variation to capture the notion of frequency on G Find an orthonormal set of Fourier bases for signals on digraphs Span a maximal frequency range [0, fmax] Frequency modes are as evenly distributed as possible Two-step DGFT basis design via a feasible method over Stiefel manifold i) Find the maximum directed variation f max over the unit sphere ii) Minimize a smooth spectral dispersion criterion over [0, f max] Provable convergence guarantees to a stationary point Ongoing work and future directions Complexity of finding the maximum frequency fmax on a digraph? If NP-hard, what is the best approximation ratio Optimality gap between the local and global optimal dispersions? Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP

17 GlobalSIP 18 Symposium on GSP Symposium on Graph Signal Processing Topics of interest Graph-signal transforms and filters Signals in high-order and multiplex graphs Distributed and non-linear graph SP Neural networks for graph data Statistical graph SP Topological data analysis Prediction and learning for graphs Graph-based image and video processing Network topology inference Communications, sensor and power networks Recovery of sampled graph signals Neuroscience and other medical fields Control of network processes Web, economic and social networks Paper submission due: June 17, th IEEE Global Conference on Signal and Information Processing November 26-28, 2018 Anaheim, California, USA Digraph Fourier Transform via Spectral Dispersion Minimization Organizers: Gonzalo Mateos (Univ. of Rochester) Santiago Segarra (MIT) Sundeep Chepuri (TU Delft) ICASSP