Stationary Graph Processes: Nonparametric Spectral Estimation

Transcription

1 Stationary Graph Processes: Nonparametric Spectral Estimation Santiago Segarra, Antonio G. Marques, Geert Leus, and Alejandro Ribeiro Dept. of Signal Theory and Communications King Juan Carlos University - Madrid (SPAIN) antonio.garcia.marques@urjc.es ACK: Spanish MINECO TEC R and USA NSF CCF IEEE SAM Workshop, Rio de Janeiro, July 11, 2016 Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 1 / 19

2 Motivation and context We frequently encounter stochastic processes Statistical signal processing Developed tools for their understanding Stationarity facilitates the analysis of random signals in time Statistical properties are time-invariant We seek to extend the concept of stationarity to graph processes Network data and irregular domains motivate this Lack of regularity lead to potentially multiple generalizations Classical SSP concepts and tools can be generalized: spectral estimation, average periodograms,... Better understanding and estimation of graph processes Related works: [Girault 15], [Perraudin 16] Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 2 / 19

3 Prelims 1: GSP Fundamentals Graph G(V, E) with N = V nodes A and L = D A Graph signals mappings x : V R, represented as vectors x R N As.: Signal properties related to topology of G To understand GS Graph-shift operator S R N N Local S ij = 0 for i j and (i, j) / E Exs.: A or L Spectrum of S = VΛV H (assume normality) Graph Fourier Transform (GFT) for signals: x = V H x Graph filters H : R N R N are maps between graph signals Polynomial in S with coefficients h R L+1 H := L l=0 h ls l Diag. by V Freq. resp.: H = Vdiag( h)v H Local linear operators (good for modeling net dynamics) Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 3 / 19

4 Prelims 2: Weakly stationarity in time (1) Correlation of stationary discrete time signals is invariant to shifts [ C x := E xx H] ] = E [x H (n l) N x(n l) N = E [S l x(s l x) H] (2) Signal is the output of a LTI filter H excited with white noise w x = Hw, with E [ ww H] = I (3) The covariance matrix C x is diagonalized by the Fourier matrix C x = Fdiag(p)F H The process has a power spectral density p := diag ( F H C x F ) Each of these definitions can be generalized to graph signals Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 4 / 19

5 Stationary graph processes: Shifts Definition (shift invariance) Process x is weakly stationary with respect to S if and only if (b > c) (S E[ a x )( (S H ) b x ) ] [ H (S = E a+c x )( (S H ) b c x ) ] H Use a and b shifts as reference. Shift by c forward and backward Signal is stationary if these shifts do not alter its covariance It reduces to E [ xx H] = E [ S l x(s l x) H] when S is a directed cycle Time shift is orthogonal, S H = S 1 (a = 0, b = N and c = l) Need reference shifts because S can change energy of the signal Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 5 / 19

6 Stationary graph processes: Filtering Definition (filtering of white noise) Process x is weakly stationary with respect to S if it can be written as the output of linear shift invariant filter H with white input w x = Hw, with E [ ww H] = I The filter H is linear shift invariant if H(Sx) = S(Hx) L Equivalently, H polynomial on the shift operator H = h l S l Filter H determines color C x = E [ (Hw)(Hw) H] = HH H l=0 Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 6 / 19

7 Stationary graph processes: Diagonalization Definition (Simultaneous diagonalization) Process x is weakly stationary with respect to S if the covariance C x and the shift S are simultaneously diagonalizable S = V Λ V H = C x = V diag(p) V H Equivalent to time definition because F diagonalizes cycle graph The process has a power spectral density p := diag ( V H C x V ) Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 7 / 19

8 Equivalence of definitions and PSD Have introduced three equally valid definitions of weak stationarity They are different but, pleasingly, equivalent Theorem Process x has shift invariant correlation matrix it is the output of a linear shift invariant filter Covariance jointly diagonalizable with shift Shift and Filtering How stationary signals look like (local invariance) Simultaneous Diagonalization A PSD exists p := diag ( V H C x V ) The PSD collects the eigenvalues of C x. The PSD is nonnegative because C x is positive semidefinite Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 8 / 19

9 Weak stationary graph processes examples Example (White noise) White noise w is stationary in any graph shift S = VΛV H Covariance C w = σ 2 I simultaneously diagonalizable with all S Example (Covariance matrix graphs and Precision matrices) Every process is stationary in the graph defined by its covariance matrix If S = C x, shift S and covariance C x diagonalized by same basis Process is also stationary on precision matrix S = C 1 x Example (Heat diffusion processes and ARMA processes) Heat diffusion process in a graph x = α 0 (I αl) 1 w Stationary in L since α 0 (I αl) 1 is a polynomial on L Any autoregressive moving average (ARMA) process on a graph Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 9 / 19

10 Power spectral density examples Example (White noise) Power spectral density p = diag ( V H (σ 2 I)V ) = σ 2 1 Example (Covariance matrix graphs and Precision matrices) Power spectral density p = diag ( V H (VΛV H )V ) = diag(λ) Example (Heat diffusion processes and ARMA processes) Power spectral density p = diag [α 0 2 (I αλ) 2] Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 10 / 19

11 PSD estimation with correlogram & periodogram Given a process x, the covariance of x = V H x is given by C x := E [ x x H] = E [ (V H x)(v H x) H] = diag(p) Periodogram Given samples x r, average GFTs of samples ˆp pg := 1 R R x r 2 = 1 R r=1 R V H x r 2 r=1 Correlogram Replace C x in PSD definition by sample covariance ˆp cg ( ) := diag V H Ĉ x V [ [ 1 := diag V H R R x r x H r r=1 ] ] V Periodogram and correlogram lead to identical estimates ˆp pg = ˆp cg Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 11 / 19

12 Accuracy of periodogram estimator Theorem If the process x is Gaussian, periodogram estimates have bias and variance Bias b pg := E [ˆp pg ] p = 0 Variance Σ pg := E [ (ˆp pg p)(ˆp pg p) H] = 2 R diag2 (p) The periodogram is unbiased but the variance is not too good Quadratic in p. Same as time processes Bias - Variance tradeoff Windows and Filterbanks Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 12 / 19

13 Windowed average periodogram Generalize Bartlett and Welch window methods to graphs Consider windowed versions of a signal as separate realizations Given one realization x and bank of M windows W = {w m } M m=1 Obtain the windowed average periodogram ˆp W := 1 M M V H x m 2 1 M = V H diag(w m )x 2 M m=1 m=1 Similar to periodogram but windowed signals x m are not independent Introduces bias but reduces variance compared to single observation Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 13 / 19

14 Accuracy of windowed periodogram Window PSD W m := V H diag(w m )V and spectrum mixing matrices W mm := W m W m Theorem For window bank W = {w m } M m=1, the mean and variance of the windowed average periodogram ˆp W are Mean E [ˆp W ] = 1 M W mm p M m=1 Variance Tr[Σ W ] = 2 M 2 M m=1,m =1 [ ( Tr Wmm p )( Wmm p ) ] H If windows are localized there is no (or minimal) spectrum mixing Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 14 / 19

15 Normalized MSE Average periodogram MSE of periodogram as a function of the nr. of observations R Baseline ER random graph (N = 100 and p = 0.05) and S = A Observe filtered white Gaussian noise and estimate PSD 2 1 Baseline ER Smaller ER Larger PSD Small-world Number of observations Normalized MSE evolves as 2/R as expected Invariant to size, topology, and PSD Same behavior observed in non-gaussian processes (theory not valid) Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 15 / 19

16 Windowed average periodogram Performance of local windows and random windows Block stochastic graph (N = 100, 10 communities) and small world Process filters white noise with different number of taps Normalized MSE Bias squared Variance Theoretical error Empirical error Normalized MSE Local window (L=10) Local window (L=2) Random window (L=10) Random window (L=2) 0 Single window Ten local wind. Ten random wind Number of windows The use of windows introduces bias but reduces total error (MSE) Local windows work better than random windows Advantage of local windows is larger for local processes Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 16 / 19

17 Source identification error Opinion source identification Opinion diffusion in Zachary s karate club network (N = 34) Observed opinion x obtained by diffusing sparse white rumor w Given {x r } R r=1 generated from unknown {w r } R r=1 Diffused through filter of unknown nonnegative coefficients β Goal Identify the support of each rumor w r First Estimate β from Moving Average PSD estimation Second Solve R sparse linear regressions to recover supp(w r ) < = 0 < = 0.10 < = 0.15 < = Number of observations Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 17 / 19

18 Frequency index Normalized MSE PSD of face images PSD estimation for spectral signatures of faces of different people 100 grayscale face images {x i } 100 i=1 R10304 (10 images 10 people) Consider x i as realization graph process that is Stationary on Ĉ x Construct Ĉ(j) x = V (j) Λ (j) c V H(j) based on images of person j 5 10 # Individual 1 Individual 2 Individual Frequency index Number of windows Process of person j approximately stationary in Ĉx (left) Use windowed average periodogram to estimate PSD of new face Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 18 / 19

19 Conclusion Extended the notion of weak stationarity for graph processes Three definitions inspired in stationary time processes Shown all of them to be equivalent Defined power spectral density and studied its estimation Generalized classical non-parametric estimation methods Periodogram and correlogram where shown to be equivalent Windowed average periodogram leads to better estimate Extensions not described here Other non-parametric schemes: filter banks Parametric estimation: AR, MA, ARMA Space-time variation Thanks! Antonio G. Marques Stationary Graph Processes: Nonparametric Spectral Estimation 19 / 19