Wavelet Methods for Time Series Analysis. Part IX: Wavelet-Based Bootstrapping

Transcription

1 Wavelet Methods for Time Series Analysis Part IX: Wavelet-Based Bootstrapping start with some background on bootstrapping and its rationale describe adjustments to the bootstrap that allow it to work with correlated time series describe how the decorrelating property of the DWT can be used to develop a wavelet-based bootstrap for certain time series describe wavestrapping, an adaptive procedure based upon finding a decorrelating transform from a wavelet packet table IX

2 Motivating Question let X =[X 0,..., X N ] T be a finite portion of a stationary process with autocovariance sequence (ACVS) {s τ } let {ρ τ } be the corresponding autocorrelation sequence (ACS): ρ τ = s τ, where s τ =cov{x t,x t+τ } and s s 0 =var{x t } 0 given a time series, we can estimate its ACS at τ = using N 2 ˆρ t=0 X tx t+ N t=0 X2 t under the assumption that E{X t } =0 Q: given the amount N of data we have, how close can we expect ˆρ to be to the true unknown ρ? i.e., how can we assess the sampling variability in ˆρ? IX 2

3 Classic Approach Large Sample Theory: I in what follows, let N (µ, σ 2 ) denote a Gaussian (normal) random variable (RV) with mean µ and variance σ 2 if X t s were independent and identically distributed (IID) so that ρ = 0, the distribution of ˆρ becomes arbitrarily close to that of an N (0, N )RVasN (requires suitable conditions) IX 3

4 Classic Approach Large Sample Theory: II more generally, ˆρ is close to the distribution of an N (ρ,σn 2 ) RV as N, where σn 2 } {ρ 2 N τ(+2ρ 2 )+ρ τ+ρ τ 4ρ ρ τ ρ τ τ= in practice, the above result is unappealing because it requires knowledge of the theoretical ACS the ACS to damp down sufficiently fast, which would rule out long memory processes (LMPs) while large sample theory has been worked out for ˆρ under certain conditions, similar theory for other statistics can be hard to come by IX 4

5 Alternative Approach Bootstrapping: I if X t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows suppose we have the following time series of length N =8, which is a realization of a Gaussian white noise process: x =[.9,. 2.2, 0.,.0, 0.6, 0.5,.3, 0.3] T,. for which ˆρ =0.23 (for white noise, the true value of ρ is 0) generate a new time series x () by randomly sampling from x: x (). =[2.2, 0., 0.,.0,.9,.9, 0.6, 0.] T, for which ˆρ (). = 0.3 (note: sampling is done with replacement) do again to get x (2) =[ 0.3, 0.5,.9, 0.6, 0.3, 0.5, 2.2, 2.2] T, for which ˆρ (2). =0.39 IX 5

6 Alternative Approach Bootstrapping: II repeat a large number of times to get ˆρ (), ˆρ(2),...,ˆρ(M) plots shows histogram for {ˆρ (m) : m =,...,0, 000}, along with probability density function (PDF) for N (0, 8 ) (left-hand plot) and an approximation to the true PDF for ˆρ (right) ˆρ (m) ˆρ (m) vertical lines indicates ˆρ can regard sample distribution of {ˆρ (m) } as an approximation to the unknown distribution of ˆρ IX 6

7 Alternative Approach Bootstrapping: III bootstrap approximation to distribution of ˆρ gets better as N increases consider sample of Gaussian white noise of length N = 28, for which ˆρ. = 0.02 x t t ˆρ (m) vertical line indicates ˆρ sample distribution of {ˆρ (m) } agrees quite well with the approximate true PDF IX 7

8 Bootstrapping Correlated Time Series: I key assumption: x was a realization of IIDRVs if not true (usually the case with time series!), sample distribution of {ˆρ (m) } can be badly misleading as an approximation to unknown distribution of ˆρ as an example, consider a realization of a fractionally differenced (FD) process with parameter δ = 4, for which ˆρ. =0.23 (for an FD( 4 ) process, ρ = 3 ) x t t IX 8

9 Bootstrapping Correlated Time Series: II use the same procedure as before to get ˆρ (), ˆρ(2),...,ˆρ(M) plot shows histogram for {ˆρ (m) : m =,...,0, 000}, along with an approximation to the true PDF for ˆρ ˆρ (m) vertical lines indicate ˆρ and ρ bootstrap approximation gets even worse as N increases to correct the problem caused by correlation in time series, can use specialized time or frequency domain bootstrapping if ACS damp downs sufficiently fast IX 9

10 Parametric Bootstrapping: I one well-known time domain bootstrapping scheme is the parametric (or residual) bootstrap suppose we can assume that our time series is a realization of a portion X 0,..., X N of a first order autoregressive (AR) process: X t = φ X t + ɛ t, where φ < and {ɛ t } is white noise with zero mean and variance σɛ 2 (this model is widely used in geophysics) have var {X t } = σɛ/( 2 φ 2 ) and ρ τ = φ τ for AR() process in particular, ρ = φ, so can estimate φ using ˆφ ˆρ IX 0

11 Parametric Bootstrapping: II since ɛ t = X t φ X t, can form residuals r t = X t ˆφ X t, t =,...,N, with the idea that r t will be a good approximation to ɛ t (note: there are N residuals rather than N) let r () 0,r(),...,r() N be a random sample from r,r 2,...,r N (as before, sampling is done with replacement) let X () 0 = r () 0 /( ˆφ 2 )/2 ( stationary initial condition ) form X () t = ˆφ X () t + r() t, t =,...,N, yielding the bootstrapped time series X () 0,X(),...,X() N IX

12 Parametric Bootstrapping: III use X () 0,X(),...,X() N let r (2) 0,r(2),...,r(2)..., r N to compute ˆρ() N be a second random sample from r, r 2, use these to form a second bootstrapped series X (2) 0, X(2),..., X (2) N, from which we form ˆρ(2) repeat this procedure M times to get ˆρ (), ˆρ(2),...,ˆρ(M) IX 2

13 Parametric Bootstrapping: IV as an example, consider a realization of an AR() process with φ = ρ = 3, for which ˆρ. =0.38 x t t plot shows histogram for {ˆρ (m) : m =,...,0, 000}, along with an approximation to the true PDF for ˆρ ˆρ (m) vertical lines indicate ˆρ and ρ IX 3

14 Parametric Bootstrapping: V important assumption here is that time series is well modeled by AR() process to see what happens if this assumption fails, reconsider FD( 4 ) realization and treat it as if it were an AR() realization since ˆρ. =0.23, we would set ˆφ. =0.23 plot shows histogram for {ˆρ (m) : m =,...,0, 000}, along with an approximation to the true PDF for ˆρ ˆρ (m) vertical lines indicate ˆρ and ρ IX 4

15 Parametric Bootstrapping: VI more generally, can fit pth order process p X t = φ u X t u + ɛ t and use r t = X t u= p u= ˆφ u X t u to form new series and then ˆρ (m) note that the number of residuals is N p, so best to stick with small values of p several variations on the basic scheme, one of which is to use r t = r t r rather than r t, where r is the sample mean of the residuals (usually close to zero, but sometimes not) IX 5

16 Block Bootstrapping another time domain approach is block bootstrapping, which is nonparametric and has some nice theoretical properties, but a bit trickier to describe and implement IX 6

17 Frequency Domain Bootstrapping phase scramble discrete Fourier transform (DFT) {X k } of data {X t } and apply inverse DFT to create new series: X k = N t=0 X t e i2πkt/n = A k e iθ k periodogram-based bootstrapping: in addition to phase scrambling, evoke large sample result that A k s are approximately uncorrelated with distribution related to a chi-square RV with 2 degrees of freedom circulant embedding bootstrapping: form nonparametric estimate of spectral density function and generate realizations using circulant embedding IX 7

18 Rationale for Wavelet Domain Bootstrapping time and frequency domain approaches are both problematic for long memory processes DWT decorrelates certain time series X, including long memory processes (these are ruled out by time and frequency domain bootstrapping because ACS damps down slowly) level J 0 partial DWT maps X to W, W 2,...,W J0 and V J0, with the RVs in the W j s being approximately uncorrelated (note: scaling coefficients V J0 are still highly correlated) IX 8

19 DWT of a Long Memory Process: I 5 X 0 0 ˆρ X,τ t τ (lag) realization of an FD(0.4) time series X along with its sample autocorrelation sequence (ACS): for τ 0, N τ ˆρ X,τ t=0 X t X t+τ N t=0 X2 t note that ACS dies down slowly IX 9

20 DWT of a Long Memory Process: II V 7 W 7 W 6 W 5 W 4 W 3 W 2 W τ (lag) LA(8) DWT of FD(0.4) series and sample ACSs for each W j & V 7, along with 95% confidence intervals for white noise IX 20

21 DWT of a Long Memory Process: III second example: ACS for FD(0.45) τ unit lag autocorrelations for W j using the Haar, D(4) and LA(8) wavelet filters (other autocorrelations are very small) j Haar D(4) LA(8) IX 2

22 DWT of a Long Memory Process: IV spectral density functions (SDFs) for X and W j SDF (in decibels) frequency frequency relatively flat (white noise if perfectly flat), but remaining variation well approximated by SDF for AR(2) process height increases as j increases (variance of W j sets height) IX 22

23 DWT of a Long Memory Process: V maximum absolute cross-correlations for wavelet coefficients in W j and W j for j<j 4 Haar D(4) LA(8) j\j IX 23

24 Recipe for Wavelet Domain Bootstrapping: I. given X of length N =2 J, compute level J 0 = J 2 partial DWT W,..., W J0 and V J0 (4 coefficients in W J0 and V J0 ) 2. randomly sample with replacement N/2 j times from W j to create bootstrapped vector W (b) j, j =,...,J 0 3. do the same for V J0 to create V (b) J (theory lacking here, but 0 better in computer experiments than using just V J0 ) 4. apply inverse transform to W (b),..., W(b) J and V (b) 0 J to obtain 0 bootstrapped time series X (b) 5. compute unit lag sample autocorrelation ˆρ (b) repeat above many times to build up sample distribution of bootstrapped autocorrelations IX 24

25 Recipe for Wavelet Domain Bootstrapping: II computer experiments indicate improvement over block bootstrap for FDprocesses variation: replace X by series of length 2N given by X (c) [X 0,X,...,X N 2,X N,X N,X N 2,...,X,X 0 ] T ; i.e., use reflection rather than circular boundary conditions IX 25

26 Motivation for Wavestrapping : I DWT does not adequately decorrelate all time series consider first order moving average process (MA()): X t = ɛ t +0.99ɛ t IX 26

27 Motivation for Wavestrapping : II SDFs for MA() process and associated W j SDF (in decibels) frequency frequency note that SDF of W is not approximately flat idea: use transform selected from wavelet packet table IX 27

28 Motivation for Wavestrapping : III consider following level J 0 = 4 wavelet packet table (WPT): W 0,0 X W,0 W, W 2,0 W 2, W 2,2 W 2,3 W 3,0 W 3, W 3,2 W 3,3 W 3,4 W 3,5 W 3,6 W 3,7 W 4,0 W 4, W 4,2 W 4,3 W 4,4 W 4,5 W 4,6 W 4,7 W 4,8 W 4,9 W 4,0 W 4, W 4,2 W 4,3 W 4,4 W 4,5 0 /6 /8 3/6 /4 5/6 3/8 7/6 /2 f shaded boxes identify an orthonormal transform that is a better decorrelator of the MA() process than the DWT IX 28

29 Motivation for Wavestrapping : IV SDFs for MA() process and associated W j,n SDF (in decibels) frequency frequency IX 29

30 Motivation for Wavestrapping : V first 5 of W j,n SDFs have variations less than 3 db, but those for W 4,3, W 4,4 and W 4,5 vary by 3.9, 5.3 and 7.2 db increasing depth of WPT to J 0 = 6 allows us to replace these by three j = 5 level subvectors W 5,26, W 5,27, W 5,28 and six j = 6 level subvectors W 6,58,...,W 6,63 resulting WPT has SDFs that all vary by less than 3 db idea: adaptively select transform by using white noise tests IX 30

31 Recipe for Wavestrapping: I. given X of length 2 J, compute level J 0 = J 2 WPT (enter step 2 with starting values j = n = 0 and W 0,0 X) 2. if j = J 0, retain W j,n ;ifj<j 0, do white noise test on W j,n portmanteau test on autocorrelation estimates for W j,n cumulative periodogram test if fail to reject the null hypothesis, retain W j,n ; if reject, discard W j,n (after transforming it into W j+,2n and W j+,2n+ ), and repeat this step twice again (both on W j+,2n and W j+,2n+ ) 3. desired adaptively chosen transform consists of all subvectors retained after step 2 applied as many times as needed; randomly sample (with replacement) from each subvector in the transform to create the similarly dimensioned wavestrapped subvectors IX 3

32 Recipe for Wavestrapping: II 4. apply inverse transform to obtain bootstrapped time series X (b) 5. compute unit lag sample autocorrelation ˆρ (b) repeat above many times to build up sample distribution of bootstrapped autocorrelations IX 32

33 Summary of Computer Experiments - I Wavestrap Process Boundary DWT Port Pgrm Block True WN N = 28 periodic reflection N = 024 periodic reflection AR() N = 28 periodic reflection N = 024 periodic reflection MA() N = 28 periodic reflection N = 024 periodic reflection FD N = 28 periodic reflection N = 024 periodic reflection IX 33

34 Summary of Computer Experiments - II standard deviations ( 00) of unit lag sample autocorrelations given by DWT-based bootstrapping, two forms of wavestrapping and block bootstrap, along with true standard deviations four models considered are white noise (WN); AR() process X t =0.9X t + ɛ t ; MA() process X t = ɛ t +0.99ɛ t ; and fractionally differenced (FD) process with δ = 0.45 wavestrapping with portmanteau test and reflection boundary conditions does better than or is comparable to block bootstrap (current state of the art) except for the MA() process, for which the block bootstrap is ideally suited IX 34

35 Application to BMW Stock Prices - I log(returns) days plot shows log of daily returns on BMW share prices has small unit lag sample autocorrelation: ˆρ. =0.08. large sample theory appropriate for Gaussian white noise gives standard error of / N. =0.03 IX 35

36 Application to BMW Stock Prices - II Gaussianity is suspect: data better modeled by t distribution with 3.9 degrees of freedom block bootstrap with block sizes 30, 50, 00, 200 and 500 gives standard errors are 0.02, 0.02, 0.04, 0.06 and 0.05 DWT-based bootstrap and wavestrap give & confirms presence of autocorrelation (small, but presumably exploitable by traders) IX 36

37 Applications to Bivariate Climate Time Series - I snowpack (inches) year PDO index plot shows Pacific decadal oscillation (PDO) index (thick curve) and March 5th snow depth on Mt. Rainier (thin curve) sample cross-correlation is N ˆρ XY t=0 (X t X)(Y t Y ) [ N t=0 (X t X) 2 N t=0 (Y t Y ) 2] /2. = 0.27 Q: given such a short series, is this significantly different from zero? IX 37

38 Applications to Bivariate Climate Time Series - II histogram of wavestrapped cross-correlations says yes 4 normalized counts cross correlation IX 38

39 Comment on Other Approaches stick with DWT, but use J 0 = J 3orJ 0 = J 4 partial DWT so that there are 8 or 6 coefficients in both W J0 and V J0, then use parametric bootstrap on V J0 in addition to using parametric bootstrap on V J0, use parametric or block bootstrap separately on each subvector W j for FDprocesses, although W j is close to white noise, its variation from white noise is captured to a very good approximation by an AR() or AR(2) process IX 39

40 References: I C. Angelini, D. Cava, G. Katul and B. Vidakovic (2005), Resampling Hierarchical Processes in the Wavelet Domain: A Case Study Using Atmospheric Turbulence, Physica D, 207, pp M. Breakspear, M. J. Brammer, E. T. Bullmore, P. Das and L. M. Williams (2004), Spatiotemporal Wavelet Resampling for Functional Neuroimaging Data, Human Brain Mapping, 23, pp. 25 M. Breakspear, M. Brammer and P. A. Robinson (2003), Construction of Multivariate Surrogate Sets from Nonlinear Data Using the Wavelet Transform, Physica D, 82, pp. 22 E. Bullmore, J. Fadili, V. Maxim, L. Şendur, B. Whitcher, J. Suckling, M. Brammer and M. Breakspear (2004), Wavelets and Functional Magnetic Resonance Imaging of the Human Brain, NeuroImage, 23, pp. S234-S249 E. Bullmore, C. Long, J. Suckling, J. Fadili, G. Calvert, F. Zelaya, T. A. Carpenter and M. Brammer (200), Colored Noise and Computational Inference in Neurophysiological (fmri) Time Series Analysis: Resampling Methods in Time and Wavelet Domains, Human Brain Mapping, 2, pp IX 40

41 References: II A. C. Davison and D. V. Hinkley (997), Bootstrap Methods and their Applications, Cambridge University Press H. Feng, T. R. Willemain and N. Shang (2005), Wavelet-Based Bootstrap for Time Series Analysis, Communications in Statistics: Simulation and Computation 34(2), pp S. Golia (2002), Evaluating the GPH Estimator via Bootstrap Technique, in Proceedings in Computational Statistics COMPSTAT2002, edited by W. Härdle and B. Ronz. Heidelberg: Physica Verlag, pp D. B. Percival and W. L. B. Constantine (2006), Exact Simulation of Gaussian Time Series from Nonparametric Spectral Estimates with Application to Bootstrapping, Statistics and Computing, 6(), pp D. B. Percival, S. Sardy and A. C. Davison (200), Wavestrapping Time Series: Adaptive Wavelet-Based Bootstrapping, in Nonlinear and Nonstationary Signal Processing, edited by W. J. Fitzgerald, R. L. Smith, A. T. Walden and P. C. Young. Cambridge, England: Cambridge University Press, pp IX 4

42 References: III A. M. Sabatini (999), Wavelet-Based Estimation of /f-type Signal Parameters: Confidence Intervals Using the Bootstrap, IEEE Transactions on Signal Processing, 47(2), pp B. J Whitcher (2006), Wavelet-Based Bootstrapping of Spatial Patterns on a Finite Lattice, Computational Statistics & Data Analysis 50(9), pp IX 42