Wavelet Methods for Time Series Analysis. Motivating Question

Transcription

1 Wavelet Methods for Time Series Analysis Part VII: Wavelet-Based Bootstrapping start with some background on bootstrapping and its rationale describe adjustments to the bootstrap that allow it to work with correlated time series describe how the decorrelating property of the DWT can be used to develop a wavelet-based bootstrap for certain time series describe wavestrapping, an adaptive procedure based upon finding a decorrelating transform from a wavelet packet table Motivating Question let X =[X,..., X N ] T be a finite portion of a stationary process with autocovariance sequence(acvs) {s τ } let {ρ τ } be the corresponding autocorrelation sequence (ACS): ρ τ = s τ, where s τ =cov{x t,x t+τ } and s s =var{x t } given a time series, we can estimate its ACS at τ = using N 2 ˆρ t= X tx t+ N t= X2 t under the assumption that E{X t } = Q: given the amount N of data we have, how close can we expect ˆρ to be to the true unknown ρ? i.e., how can we assess the sampling variability in ˆρ? VII VII 2 Classic Approach Large Sample Theory: I in what follows, let N (µ, σ 2 ) denote a Gaussian (normal) random variable (RV) with mean µ and variance σ 2 if X t s were independent and identically distributed (IID) so that ρ =, the distribution of ˆρ becomes arbitrarily close to that of an N (, N )RVasN (requires suitable conditions) Classic Approach Large Sample Theory: II more generally, ˆρ is close to the distribution of an N (ρ,σn 2 ) RV as N, where σn 2 } {ρ 2 N τ(+2ρ 2 )+ρ τ+ρ τ 4ρ ρ τ ρ τ τ= in practice, the above result is unappealing because it requires knowledge of the theoretical ACS the ACS to damp down sufficiently fast, which would rule out long memory processes (LMPs) while large sample theory has been worked out for ˆρ under certain conditions, similar theory for other statistics can be hard to come by VII 3 VII 4

2 Alternative Approach Bootstrapping: I if X t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows suppose we have the following time series of length N =8, which is a realization of a Gaussian white noise process: x =[.9,. 2.2,.,.,.6,.,.3,.3] T,. for which ˆρ =.23 (for white noise, the true value of ρ is ) generate a new time series x () by randomly sampling from x: x (). =[2.2,.,.,.,.9,.9,.6,.] T, for which ˆρ (). =.3 (note: sampling is done with replacement) do again to get x (2) =[.3,.,.9,.6,.3,., 2.2, 2.2] T, for which ˆρ (2). =.39 VII Alternative Approach Bootstrapping: II repeat a large number of times to get ˆρ (), ˆρ(2),...,ˆρ(M) plots shows histogram for {ˆρ (m) : m =,...,, }, along with probability density function (PDF) for N (, 8 ) (left-hand plot) and an approximation to the true PDF for ˆρ (right).3. ˆρ (m) ˆρ (m) vertical line indicates ˆρ can regard sample distribution of {ˆρ (m) } as an approximation to the unknown distribution of ˆρ VII 6 Alternative Approach Bootstrapping: III bootstrap approximation to distribution of ˆρ gets better as N increases consider sample of Gaussian white noise of length N = 28, for. which ˆρ =.2 x t t ˆρ (m) vertical line indicates ˆρ sample distribution of {ˆρ (m) } agrees quite well with the approximate true PDF Bootstrapping Correlated Time Series: I key assumption: x was a realization of IIDRVs if not true (usually the case with time series!), sample distribution of {ˆρ (m) } can be badly misleading as an approximation to unknown distribution of ˆρ as an example, consider a realization of a fractionally differenced (FD) process with parameter δ = 4, for which ˆρ. =.23 (for an FD( 4 ) process, ρ = 3 ) x t t VII 7 VII 8

3 Bootstrapping Correlated Time Series: II use the same procedure as before to get ˆρ (), ˆρ(2),...,ˆρ(M) plot shows histogram for {ˆρ (m) : m =,...,, }, along with an approximation to the true PDF for ˆρ ˆρ (m) vertical lines indicate ˆρ and ρ bootstrap approximation gets even worse as N increases to correct the problem caused by correlation in time series, can use specialized time or domain bootstrapping if ACS damp downs sufficiently fast Parametric Bootstrapping: I one well-known time domain bootstrapping scheme is the parametric (or residual) bootstrap suppose we can assume that our time series is a realization of a portion X,..., X N of a first order autoregressive (AR) process: X t = φ X t + ɛ t, where φ < and {ɛ t } is white noise with zero mean and variance σɛ 2 (this model is widely used in geophysics) have var {X t } = σɛ/( 2 φ 2 ) and ρ τ = φ τ for AR() process in particular, ρ = φ, so can estimate φ using ˆφ ˆρ VII 9 VII Parametric Bootstrapping: II since ɛ t = X t φ X t, can form residuals r t = X t ˆφ X t, t =,...,N, with the idea that r t will be a good approximation to ɛ t (note: there are N residuals rather than N) let r (),r(),...,r() N be a random sample from r,r 2,...,r N (as before, sampling is done with replacement) let X () = r () /( ˆφ 2 )/2 ( stationary initial condition ) form X () t = ˆφ X () t + r() t, t =,...,N, yielding the bootstrapped time series X (),X(),...,X() N Parametric Bootstrapping: III use X (),X(),...,X() N to compute ˆρ() let r (2),r(2),...,r(2) N be a second random sample from r, r 2,..., r N use these to form a second bootstrapped series X (2), X(2),..., X (2) N, from which we form ˆρ(2) repeat this procedure M times to get ˆρ (), ˆρ(2),...,ˆρ(M) VII VII 2

4 Parametric Bootstrapping: IV as an example, consider a realization of an AR() process with φ = ρ = 3, for which ˆρ. =.38 x t t plot shows histogram for {ˆρ (m) : m =,...,, }, along with an approximation to the true PDF for ˆρ 6 ˆρ (m) vertical lines indicate ˆρ and ρ Parametric Bootstrapping: V important assumption here is that time series is well modeled by AR() process to see what happens if this assumption fails, reconsider FD( 4 ) realization and treat it as if it were an AR() realization.. since ˆρ =.23, we would set ˆφ =.23 plot shows histogram for {ˆρ (m) : m =,...,, }, along with an approximation to the true PDF for ˆρ ˆρ (m) vertical lines indicate ˆρ and ρ VII 3 VII 4 Parametric Bootstrapping: VI more generally, can fit pth order process p X t = φ u X t u + ɛ t and use r t = X t u= p ˆφ u X t u u= to form new series and then ˆρ (m) note that the number of residuals is N p, so best to stick with small values of p several variations on the basic scheme, one of which is to use r t = r t r rather than r t, where r is the sample mean of the residuals (usually close to zero, but sometimes not) Block Bootstrapping another time domain approach is block bootstrapping, which is nonparametric and has some nice theoretical properties, but a bit trickier to describe and implement VII VII 6

5 Frequency Domain Bootstrapping phase scramble discrete Fourier transform (DFT) {X k } of data {X t } and apply inverse DFT to create new series: X k = N t= X t e i2πkt/n = A k e iθ k periodogram-based bootstrapping: in addition to phase scrambling, evoke large sample result that A k s are approximately uncorrelated with distribution related to a chi-square RV with 2 degrees of freedom circulant embedding bootstrapping: form nonparametric estimate of spectral density function and generate realizations using circulant embedding Rationale for Wavelet Domain Bootstrapping time and domain approaches are both problematic for long memory processes DWT decorrelates certain time series X, including long memory processes (these are ruled out by time and domain bootstrapping because ACS damps down slowly) level J partial DWT maps X to W, W 2,...,W J and V J, with the RVs in the W j s being approximately uncorrelated (note: scaling coefficients V J are still highly correlated) VII 7 VII 8 DWT of a Long Memory Process: I DWT of a Long Memory Process: II V X ˆρ X,τ W t τ (lag) W 6 W realization of an FD(.4) time series X along with its sample autocorrelation sequence (ACS): for τ, N τ ˆρ X,τ t= X t X t+τ N t= X2 t note that ACS dies down slowly VII 9 W 4 W 3 W 2 W 32 τ (lag) LA(8) DWT of FD(.4) series and sample ACSs for each W j & V 7, along with 9% confidence intervals for white noise VII 2

6 DWT of a Long Memory Process: III second example: ACS for FD(.4) τ unit lag autocorrelations for W j using the Haar, D(4) and LA(8) wavelet filters (other autocorrelations are very small) j Haar D(4) LA(8) VII 2 DWT of a Long Memory Process: IV spectral density functions (SDFs) for X and W j SDF (in decibels) relatively flat (white noise if perfectly flat), but remaining variation well approximated by SDF for AR(2) process height increases as j increases (variance of W j sets height) VII 22 DWT of a Long Memory Process: V maximum absolute cross-correlations for wavelet coefficients in W j and W j for j<j 4 Haar D(4) LA(8) j\j VII 23 Recipe for Wavelet Domain Bootstrapping: I. given X of length N =2 J, compute level J = J 2 partial DWT W,..., W J and V J (4 coefficients in W J and V J ) 2. randomly sample with replacement N/2 j times from W j to create bootstrapped vector W (b) j, j =,...,J 3. do the same for V J to create V (b) J (theory lacking here, but better in computer experiments than using just V J ) 4. apply inverse transform to W (b),..., W(b) J and V (b) J to obtain bootstrapped time series X (b). compute unit lag sample autocorrelation ˆρ (b) repeat above many times to build up sample distribution of bootstrapped autocorrelations VII 24

7 Recipe for Wavelet Domain Bootstrapping: II Motivation for Wavestrapping : I computer experiments indicate improvement over block bootstrap for FDprocesses variation: replace X by series of length 2N given by X (c) [X,X,...,X N 2,X N,X N,X N 2,...,X,X ] T ; i.e., use reflection rather than circular boundary conditions DWT does not adequately decorrelate all time series consider first order moving average process (MA()): X t = ɛ t +.99ɛ t VII 2 VII 26 Motivation for Wavestrapping : II SDFs for MA() process and associated W j Motivation for Wavestrapping : III consider following level J = 4 wavelet packet table (WPT): SDF (in decibels) -2 - W, X W, W, W 2, W 2, W 2,2 W 2,3 W 3, W 3, W 3,2 W 3,3 W 3,4 W 3, W 3,6 W 3,7 W 4, W 4, W 4,2 W 4,3 W 4,4 W 4, W 4,6 W 4,7 W 4,8 W 4,9 W 4,W 4,W 4,2W 4,3W 4,4W 4, /6 /8 3/6 /4 /6 3/8 7/6 /2 f shaded boxes identify an orthonormal transform that is a better decorrelator of the MA() process than the DWT note that SDF of W is not approximately flat idea: use transform selected from wavelet packet table VII 27 VII 28

8 Motivation for Wavestrapping : IV SDFs for MA() process and associated W j,n SDF (in decibels) Motivation for Wavestrapping : V first of W j,n SDFs have variations less than 3 db, but those for W 4,3, W 4,4 and W 4, vary by 3.9,.3 and 7.2 db increasing depth of WPT to J = 6 allows us to replace these by three j = level subvectors W,26, W,27, W,28 and six j = 6 level subvectors W 6,8,...,W 6,63 resulting WPT has SDFs that all vary by less than 3 db idea: adaptively select transform by using white noise tests VII 29 VII 3 Recipe for Wavestrapping: I. given X of length 2 J, compute level J = J 2 WPT (enter step 2 with starting values j = n = and W, X) 2. if j = J, retain W j,n ;ifj<j, do white noise test on W j,n portmanteau test on autocorrelation estimates for W j,n cumulative periodogram test if fail to reject the null hypothesis, retain W j,n ; if reject, discard W j,n (after transforming it into W j+,2n and W j+,2n+ ), and repeat this step twice again (both on W j+,2n and W j+,2n+ ) 3. desired adaptively chosen transform consists of all subvectors retained after step 2 applied as many times as needed; randomly sample (with replacement) from each subvector in the transform to create the similarly dimensioned wavestrapped subvectors VII 3 Recipe for Wavestrapping: II 4. apply inverse transform to obtain bootstrapped time series X (b). compute unit lag sample autocorrelation ˆρ (b) repeat above many times to build up sample distribution of bootstrapped autocorrelations VII 32

9 Summary of Computer Experiments - I Summary of Computer Experiments - II Wavestrap Process Boundary DWT Port Pgrm Block True WN N = 28 periodic reflection N = 24 periodic reflection AR() N = 28 periodic reflection...4 N = 24 periodic reflection.6.. MA() N = 28 periodic reflection N = 24 periodic reflection FD N = 28 periodic reflection N = 24 periodic reflection standard deviations ( ) of unit lag sample autocorrelations given by DWT-based bootstrapping, two forms of wavestrapping and block bootstrap, along with true standard deviations four models considered are white noise (WN); AR() process X t =.9X t + ɛ t ; MA() process X t = ɛ t +.99ɛ t ; and fractionally differenced (FD) process with δ =.4 wavestrapping does better than block bootstrap (current state of the art) except for the MA() process, for which the block bootstrap is ideally suited VII 33 VII 34 Application to BMW Stock Prices - I Application to BMW Stock Prices - II log(returns) plot shows log of daily returns on BMW share prices. has small unit lag sample autocorrelation: ˆρ =.8. large sample theory appropriate for Gaussian white noise gives standard error of / N =.3. days Gaussianity is suspect: data better modeled by t distribution with 3.9 degrees of freedom block bootstrap with block sizes 3,,, 2 and gives standard errors are.2,.2,.4,.6 and. DWT-based bootstrap and wavestrap give.23 &.2 confirms presence of autocorrelation (small, but presumably exploitable by traders) VII 3 VII 36

10 Applications to Bivariate Climate Time Series - I Applications to Bivariate Climate Time Series - II snowpack (inches) year plot shows Pacific decadal oscillation (PDO) index (thick curve) and March th snow depth on Mt. Rainier (thin curve) sample cross-correlation is ˆρ XY N t= (X t X)(Y t Y ) [ N t= (X t X) 2 N t= (Y t Y ) 2] /2 - PDO index. =.27 Q: given such a short series, is this significantly different from zero? VII 37 histogram of wavestrapped cross-correlations says yes normalized counts cross correlation VII 38 Comments on Other Approaches References: I stick with DWT, but use parametric or block bootstrap on each subvector W j of coefficients for FDprocesses, W j is close to white noise, but the variation from white noise is captured to a very good approximation by an AR(2) process C. Angelini, D. Cava, G. Katul and B. Vidakovic (2), Resampling Hierarchical Processes in the Wavelet Domain: A Case Study Using Atmospheric Turbulence, Physica D, 27, pp M. Breakspear, M. J. Brammer, E. T. Bullmore, P. Das and L. M. Williams (24), Spatiotemporal Wavelet Resampling for Functional Neuroimaging Data, Human Brain Mapping, 23, pp. 2 M. Breakspear, M. Brammer and P. A. Robinson (23), Construction of Multivariate Surrogate Sets from Nonlinear Data Using the Wavelet Transform, Physica D, 82, pp. 22 E. Bullmore, J. Fadili, V. Maxim, L. Şendur, B. Whitcher, J. Suckling, M. Brammer and M. Breakspear (24), Wavelets and Functional Magnetic Resonance Imaging of the Human Brain, NeuroImage, 23, pp. S234-S249 E. Bullmore, C. Long, J. Suckling, J. Fadili, G. Calvert, F. Zelaya, T. A. Carpenter and M. Brammer (2), Colored Noise and Computational Inference in Neurophysiological (fmri) Time Series Analysis: Resampling Methods in Time and Wavelet Domains, Human Brain Mapping, 2, pp VII 39 VII 4

11 References: II A. C. Davison and D. V. Hinkley (997), Bootstrap Methods and their Applications, Cambridge University Press H. Feng, T. R. Willemain and N. Shang (2), Wavelet-Based Bootstrap for Time Series Analysis, Communications in Statistics: Simulation and Computation 34(2), pp S. Golia (22), Evaluating the GPH Estimator via Bootstrap Technique, in Proceedings in Computational Statistics COMPSTAT22, edited by W. Härdle and B. Ronz. Heidelberg: Physica Verlag, pp D. B. Percival, S. Sardy and A. C. Davison (2), Wavestrapping Time Series: Adaptive Wavelet-Based Bootstrapping, in Nonlinear and Nonstationary Signal Processing, edited by W. J. Fitzgerald, R. L. Smith, A. T. Walden and P. C. Young. Cambridge, England: Cambridge University Press, pp A. M. Sabatini (999), Wavelet-Based Estimation of /f-type Signal Parameters: Confidence Intervals Using the Bootstrap, IEEE Transactions on Signal Processing, 47(2), pp B. J Whitcher, Wavelet-Based Bootstrapping of Spatial Patterns on a Finite Lattice, Computational Statistics & Data Analysis (9), pp VII 4