Synthesis of Gaussian and non-gaussian stationary time series using circulant matrix embedding

Transcription

1 Synthesis of Gaussian and non-gaussian stationary time series using circulant matrix embedding Vladas Pipiras University of North Carolina at Chapel Hill UNC Graduate Seminar, November 10, 2010 (joint work with H. Helgason, P. Abry) Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 1 / embe 28

2 Few references Gaussian stationary multivariate series: Fast and exact synthesis of stationary multivariate Gaussian time series using circulant embedding (with H. Helgason and P. Abry). Non-Gaussian stationary multivariate series: Synthesis of multivariate stationary series with prescribed marginal distributions and covariance using circulant matrix embedding (with H. Helgason and P. Abry). Both available on my website. Not the first in this department! On the reconstruction of the covariance of stationary Gaussian processes observed through zero-memory nonlinearities (S. Cambanis and E. Masry). IEEE Transactions on Information Theory, Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 2 / embe 28

3 What you need to know to follow this talk Focus on univariate time series {X n } n Z which are stationary. Autocovariance r: r(n) = Cov(X k, X k+n ) = Cov(X 0, X n ) = EX 0 X n EX 0 EX n Spectral density f : 0 f (w) = 1 2π ( r(0) + 2 ) r(n) cos(nw) Gaussian series: any vector (X k1,..., X kn ) is Gaussian. Non-Gaussian series: marginal will have certain non-gaussian distribution, e.g. χ 2 1 or log-normal series. n=1 Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 3 / embe 28

4 Three Gaussian series. Which one(s) dependent? Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 4 / embe 28

5 Three non-gaussian series. Which one(s) dependent? Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 5 / embe 28

6 Goals Interested in synthesis of X := (X 0, X 1,..., X N 1 ) where {X n } n Z is either Gaussian stationary series with given autocovariance, or non-gaussian stationary series with some marginal distribution and autocovariance. Denote the covariance matrix of X by Σ = EXX EXEX = r(0) r(1) r(2)... r(n 1) r(1) r(0) r(1)... r(n 2) r(2) r(1) r(0)... r(n 3) r(n 1) r(n 2) r(n 3)... r(0) Gaussian case: suppose EX = 0. Non-Gaussian case: Y will be used instead of X. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 6 / embe 28

7 GAUSSIAN CASE Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 7 / embe 28

8 Elementary approach and its limitations Since Σ is non-negative definite, one elementary approach (Cholesky method) is to factorize Σ = Σ 1/2 Σ 1/2 and to set X = Σ 1/2 ɛ, where ɛ is a N (0, I N ) vector. Note that such X has the correct covariance structure since EXX = Σ 1/2 Eɛɛ Σ 1/2 = Σ 1/2 I N Σ 1/2 = Σ. Problem: The complexity of this method is O(N 3 ) and the approach is practical only for moderate sizes N (N 2000). What about larger N? Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 8 / embe 28

9 Circulant matrix embedding A circulant matrix embedding of Σ is a circulant matrix r(0) r(1)... r(n 1) r(n 2)... r(1) r(1) r(0)... r(n 2) r(n 1)... r(2) Σ = r(n 1) r(n 2)... r(0) r(1)... r(n 2) r(n 2) r(n 1)... r(1) r(0)... r(n 3) r(1) r(2)... r(n 2) r(n 3)... r(0) = circ(r(0), r(1),..., r(n 1), r(n 2),..., r(1)) of dimension 2M 2M with embedding size 2M = 2N 2. Note that Σ contains the covariance matrix Σ. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 9 / embe 28

10 Why we love circulant matrices The discrete Fourier basis diagonalizes circulant matrices: Σ = F ΛF, where Λ = diag(λ(0),..., λ(2m 1)) and F is the 2M 2M Fourier matrix with jth column e j = 1 ( ) 2πj i 1, e 2M,..., e i 2πj(2M 1) 2M. 2M The eigenvalues λ(m) satisfy λ(m) = 2M 1 j=0 M 1 2πjm i r(j)e 2M = r(0) + r(m)( 1) m + 2 j=1 r(j) cos ( ) πjm M and can be computed rapidly using FFT (supposing 2M = 2 K ; complexity O(M log M)). Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 10 / embe 28

11 Assumption ND Σ has real eigenvalues λ(m) in general. Suppose for the moment: Assumption ND: The eigenvalues λ(m), m = 0,..., 2M 1, are non-negative. Equivalently, the matrix Σ is non-negative definite. Then, there is a Gaussian vector X such that E X X = Σ. Since Σ contains Σ in its upper-left corner, the first N elements X of X will then have the desired covariance structure Σ. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 11 / embe 28

12 Constructing X and X To construct X, it is more convenient to work with complex-valued variables. Let Ṽ = F Λ 1/2 Z, where Λ 1/2 exists by using Assumption ND and Z = Z 0 + iz 1 consists of two independent N (0, I 2M ) random vectors Z 0 and Z 1. Note that, for m = 0,..., 2M 1, Ṽ m = 1 2M 1 λ(j) 1/2 2πjm i Z j e 2M, 2M j=0 so that Ṽ can be rapidly computed by FFT. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 12 / embe 28

13 Constructing X and X Fact: The Gaussian vectors R(Ṽ ) and I(Ṽ ) are independent, with the covariance structure ER(Ṽ )R(Ṽ ) = EI(Ṽ )I(Ṽ ) = Σ. Thus, both X = R(Ṽ ) and X = I(Ṽ ) have the covariance matrix Σ. Finally, the desired vector X with the covariance matrix Σ is made up of the first N elements of X : X = first N entries of X. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 13 / embe 28

14 Assumption ND is expected to hold for large N Fact: If {X n } n Z is a short-range dependent series with (strictly) positive density function f (λ), then Assumption ND holds for large enough N. Basic idea: Note that, for large M (or N), M 1 λ(m) = r(0) + r(m)( 1) m + 2 j=1 r(j) cos ( ) πjm M r(0) + 2 j=1 ( r(j) cos j πm M ) = f ( πm M ) > 0. Open problem: Prove analogous result for long-range dependent series. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 14 / embe 28

15 Known conditions for Assumption ND to hold for any N Facts: Suppose that the sequence r(0),..., r(m) is convex, decreasing and non-negative. Then, Assumption ND holds. Suppose r(k) 0, k = 1,..., M. Then, Assumption ND holds. E.g. FARIMA(0, d, 0) series, fractional Gaussian noise and other models satisfy one of these conditions. Main reference for circulant embedding method: Fast and exact simulation of stationary Gaussian processes through circulant embedding of the covariance matrix (C. Dietrich and G. Newsam), SIAM J. Sci. Comput., Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 15 / embe 28

16 Our contribution Chan and Wood (1999) extended circulant embedding method to stationary Gaussian vector fields. 4 page paper! Unclear why their algorithm works or how to implement it in practice. Our contribution clarifies their algorithm in the multivariate context: Each component series is constructed using univariate circulant embedding. Cross covariance is obtained by correlating vectors Z s. Embedding of odd size is used by Chan and Wood, whereas we use even size. Difference from univariate case related to time-reversibility. We formulate analogous sufficient conditions for Assumption ND to hold, and check them on several multivariate series models. Our algorithm is implemented and has been experimented on a number of multivariate series models. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 16 / embe 28

17 NON-GAUSSIAN CASE Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 17 / embe 28

18 Problem statement We wish to numerically synthesize univariate stationary series {Y n } n Z targeting a priori given marginal distribution, F (y) = P(Y n y) autocovariance, r Y (n) = EY 0 Y n EY 0 EY n Also, the procedure should be practical and computationally fast. Remarks: The problem is not necessarily well-posed. Y may not be unique higher-order quantities not targeted. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 18 / embe 28

19 Focus on Construction based on non-linear memoryless transforms of Gaussian stationary series: Y n = f (X n ) X n is a stationary Gaussian series with autocovariance r X (n) X n d = N (0, 1) f : R R is a deterministic function Matching marginal: Multiple ways to reach the desired marginal F. A standard transformation: f (x) = F 1 (Φ(x)), Φ cdf for N (0, 1) Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 19 / embe 28

20 Relationship between covariances r Y and r X Hermite polynomials: H 0 (x) = 1, H 1 (x) = x, H 2 (x) = x 2 1, etc. For f L 2 (R, e x2 /2 dx) in Y n = f (X n ), expand f as f (x) = c m H m (x). m=0 Then, using the orthogonality property of H m (x) s, or r Y (n) = cmm!(r 2 X (n)) m m=1 r Y (n) r Y (0) = g(r X (n)) with g(z) = m=1 c 2 mm! r Y (0) zm =: b m z m. m=1 Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 20 / embe 28

21 Relationship between covariances r Y and r X Example: χ 2 1 -marginal: f (x) = x 2, r Y (n) = 2(r X (n)) 2, g(z) = z 2. Standard transformation f (x) = F 1 (Φ(x)) and the corresponding g, on the other hand, do not have explicit form. Invert r Y (n) r Y (0) = g(r X (n)) to have a candidate covariance r X for targeted r Y : Issues: ( r r X (n) = g 1 Y ) (n) r Y. (0) Inversion when g is not given explicitly. If after inversion, r X defines a valid covariance structure, X can be generated, for example, using circulant embedding method. Otherwise, What approximating valid covariance r X to choose? Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 21 / embe 28

22 Series reversion Consider g(z) = m=1 b mz m. If b 1 0, g(z) has an inverse g 1 in a neighborhood of z = 0 which can be expressed as g 1 (w) = d m w m m=1 where {d m } m 1 is defined by the reversion of the sequence {b m } m 1. Formally, equate the coefficients at the powers of z in z = g 1 (g(z)) = ( ) m d m b k z k m=1 This gives, d 1 = b1 1, d 2 = b1 3 b 2, d 3 = b1 5 (2b2 2 b 1b 3 ), etc. Simple algorithm exists for computing the coefficients d m (e.g. computational complex analysis book by Henrici (1974)). k=1 Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 22 / embe 28

23 Series reversion Few issues: Given the coefficients b m in g(z), difficult to determine where g 1 (w) = m=1 d mw m defines the inverse of g. In practice, plot (z, g(z)) for z [ 1, 1] and ( g 1 (w), w) on the same graph. Example: χ 2 1-marginal through standard transformation f (x) = F 1 (Φ(x)). The function g does not have explicit form g(z) (or w) z (or g 1 (w)) Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 23 / embe 28

24 Approximating covariance through circulant embedding Invert (explicitly or numerically) to obtain ( r r X (n) = g 1 Y ) (n) r Y. (0) Again, r X does not necessarily define a valid covariance structure. Proceed with circulant embedding method anyways. Set negative eigenvalues to 0 by { λ(m), if λ(m) 0, λ(m) = 0, if λ(m) < 0 and consider embedding Σ = F ΛF. Proceed with the rest of circulant embedding method to generate a Gaussian vector X with approximating r X. Generate approximating Ŷn as f ( X n ). Let r Y be the resulting approximation to r Y. Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 24 / embe 28

25 Optimality of approximation Are approximations r Y, Ŷ optimal in any sense? Yes, semi-formal proof shows that r Y (n) r Y (n) 2 W n min r Y (n) r app (n) 2 W n, n n where x 2 W = vec(x) W vec(x) with a positive definite matrix W, W n are suitable positive definite weight matrices, and the minimum is over all covariance structures r app arising from the transformation of a Gaussian series. The optimality is akin to spectral truncation method used in the area ( Generation of a random sequence having a jointly specified marginal distribution and autocovariance, B. Liu and D. C. Munson, IEEE Transactions on Acoustics, Speech, and Signal Processing, 1982). Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 25 / embe 28

26 Numerical example Want {Y n } with χ 2 1 marginal and autocovariance r Y (n) = 2φ n. Negative correlations: 1 < φ < 0. Consider the standard transformation f. Take φ = 0.35 so that r Y (n)/r Y (0) > 0.44 g( 1). Consider N = r Y (n) or r Y (n) r Y (n) r Y (n) n n Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 26 / embe 28

27 Our contribution Key points of our contribution: Multivariate context Use of series reversion Use of circulant embedding for approximation Its optimality Various ways to match marginals Practical implementations Open problem: It seems that standard transformation f (x) = F 1 (Φ(x)) leads to the largest class of attainable autocovariances. Can this be proved? Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 27 / embe 28

28 THANK YOU! Vladas Pipiras University of North Carolina atsynthesis Chapel Hill of Gaussian () and non-gaussian stationary time series using circulant matrix 28 / embe 28