Wavelet-Based Bootstrapping for Non-Gaussian Time Series. Don Percival

Transcription

1 Wavelet-Based Bootstrapping for Non-Gaussian Time Series Don Percival Applied Physics Laboratory Department of Statistics University of Washington Seattle, Washington, USA overheads for talk available at

2 Overview question of interest: how can we assess the sampling variability in statistics computed from a time series,,..., N? start with some background on bootstrapping review parametric and block bootstrapping (two approaching for handling correlated time series) review previously proposed wavelet-based approach to bootstrapping (Percival, Sardy and Davison, 2) describe a new wavelet-based approach that uses trees for resampling and is potentially useful for non-gaussian time series demonstrate methodology on time series related to BMW stock conclude with some remarks 2

3 Motivating Question let = [,..., N ] T be a portion of a stationary process with autocorrelation sequence (ACS) ρ τ = s τ s, where s τ = cov { t, t+τ } and s = var { t } given a time series, we can estimate its ACS at τ = using P N 2 ˆρ = t= ( t )( t+ ) P N t= ( t ) 2, where = N N Q: given the amount of data N we have, how close can we expect ˆρ to be to the true unknown ρ? i.e., how can we assess the sampling variability in ˆρ? t= t 3

4 Classic Approach Large Sample Theory let N (µ, σ 2 ) denote a Gaussian (normal) random variable (RV) with mean µ and variance σ 2 under suitable conditions (see, e.g., Fuller, 996), ˆρ has a distribution close to that of N (ρ, σn 2 ) as N, where σn 2 = o nρ 2 N τ( + 2ρ 2 ) + ρ τ+ρ τ 4ρ ρ τ ρ τ τ= in practice, this result is unappealing because it requires knowledge of theoretical ACS (including the unknown ρ!) ACS to damp down fast, ruling out some processes of interest while large sample theory has been worked out for ˆρ under certain conditions, similar theory for other statistics can be hard to come by 4

5 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 5

6 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

7 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

8 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

9 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

10 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

11 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

12 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: 5

13 Alternative Approach Bootstrapping: I if t s were IID, we could apply bootstrapping to assess the variability in ˆρ, as follows consider a time series of length N = 8 that is a realization of a Gaussian white noise process (ρ = ): ˆρ. =.8 generate new series by randomly sampling with replacement: ˆρ (). =.24 5

14 Alternative Approach Bootstrapping: II repeat a large number of times M to get ˆρ (), ˆρ(2),..., ˆρ(M) plots shows estimated probability density function (PDF) for ˆρ (), ˆρ(2),..., ˆρ(), along with (a) PDF for N (, 8 ) and (b) approximation to the true PDF for ˆρ.3. (a) (b) ˆρ (m) ˆρ (m) vertical line indicates ˆρ can regard sample distribution of {ˆρ (m) } as an approximation to the unknown distribution of ˆρ 6

15 Alternative Approach Bootstrapping: III quality of approximation depends upon particular time series here are bootstrap approximations to PDF of ˆρ based upon two other time series of length N = 8, along with true PDF.3. vertical line indicates ˆρ ˆρ (m) ˆρ (m) repeating the above for 5 time series yields 5 bootstrap PDFs summarize via sample means and standard deviations (SDs): average of 5 sample means =..27 (truth =..24). average of 5 sample SDs =.28 (truth =..284) 7

16 Bootstrapping Correlated Time Series: I key assumption: contains IID RVs if not true (as for most time series!), sample distribution of {ˆρ (m) } can be a poor approximation to distribution of ˆρ as an example, consider first order autoregressive (AR) process: t = φ t + t, where φ =.9 and { t } is zero mean Gaussian white noise AR time series of length N = 28 with sample and true ACSs: ˆρ τ, ρ τ t 32 τ 8

17 Bootstrapping Correlated Time Series: II use same procedure as before to get ˆρ (), ˆρ(2),..., ˆρ() bootstrap approximation to PDF of ˆρ along with true PDF: ˆρ (m) vertical line indicates ˆρ bootstrap approximation gets even worse as N increases to correct the problem caused by correlation in time series, can use specialized time- or frequency-domain bootstrapping (assuming true ACS damps downs sufficiently fast) 9

18 Parametric Bootstrapping: I one well-known time-domain bootstrapping scheme is the parametric (or residual) bootstrap suppose is a realization of AR process t = φ t + t note that var { t } = var { t }/( φ 2 ) and ρ τ = φ τ in particular, ρ = φ, so can estimate φ using ˆφ = ˆρ since t = t φ t, can form residuals r t = t ˆφ t, t =,..., N, with the idea that r t will be a good approximation to t let r (), r(),..., r() N be a random sample from r, r 2,..., r N let () = r () /( ˆφ 2 ) /2 ( stationary initial condition )

19 Parametric Bootstrapping: II form () t = ˆφ () t + r() t, t =,..., N, yielding the bootstrapped time series (), (),..., () N AR time series (left-hand plot) and bootstrapped series (right): () use bootstrapped series to compute ˆρ () repeat this procedure M times to get ˆρ (), ˆρ(2),..., ˆρ(M)

20 Parametric Bootstrapping: III bootstrap approximation to PDF of ˆρ along with true PDF: ˆρ (m) repeating the above for 5 AR time series yields: vertical line indicates ˆρ average of 5 sample means. =.83 (truth. =.86) average of 5 sample SDs. =.53 (truth. =.48) 2

21 Parametric Bootstrapping: IV important assumption: generated by AR process to see what happens if assumption fails, consider a fractionally differenced (FD) process Γ( δ) t = Γ(k + )Γ( δ k) t k, k= where δ =.45 and { t } is zero mean Gaussian white noise FD time series of length N = 28 with sample and true ACSs: ˆρ τ, ρ τ

22 Parametric Bootstrapping: V AR process has short-range dependence, whereas FD process exhibits long-range (or long-memory ) dependence bootstrap approximation to PDF of ˆρ along with true PDF: ˆρ (m) repeating the above for 5 FD time series yields: vertical line indicates ˆρ average of 5 sample means =..49 (truth =..53) average of 5 sample SDs =..78 (truth =..7). note: ρ =.82 for this FD process; agreement in SD gets worse (better) as N increases (decreases) 4

23 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 4

24 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

25 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

26 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

27 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

28 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

29 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

30 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

31 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

32 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

33 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

34 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

35 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

36 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

37 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

38 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: 4

39 Block Bootstrapping: I another time-domain approach is block bootstrapping, which has many variations, of which the following is the simplest break time series up into B blocks (subseries) of equal length: ˆρ. =.78 generate bootstrapped AR series by randomly sampling blocks: ˆρ (). =.63 5

40 Block Bootstrapping: II bootstrap approximation to PDF of ˆρ along with true PDF: ˆρ (m) repeating the above for 5 AR time series yields: vertical line indicates ˆρ average of 5 sample means. =.75 (truth. =.86) average of 5 sample SDs. =.49 (truth. =.48) repeating the above for 5 FD time series yields: average of 5 sample means. =.46 (truth. =.53) average of 5 sample SDs. =.82 (truth. =.7) 6

41 Frequency-Domain Bootstrapping again, many variations, including the following three phase scramble discrete Fourier transform (DFT) k = N t= t e i2πkt/n = A k e iθ k of and apply inverse DFT to create new series periodogram-based bootstrapping: in addition to phase scrambling, evoke large sample result that A k s are approximately uncorrelated with distribution related to a chi-square RV with 2 degrees of freedom circulant embedding bootstrapping: form nonparametric estimate of spectral density function and generate realizations using circulant embedding (Percival and Constantine, 26) 7

42 Critique of Time/Frequency-Domain Bootstrapping time- and frequency-domain approaches are mainly designed for series with short-range dependence (e.g., AR) and are problematic for those exhibiting long-range dependence (e.g., FD) parametric and frequency-domain bootstraps work best for series that obey a Gaussian distribution, but can be problematic for non-gaussian series non-gaussian series better handled by block bootstrapping, but quality of this approach depends critically on chosen size for blocks (ad hoc rule is to set size close to N) room for improvement: will consider wavelet-based approaches 8

43 Overview of Discrete Wavelet Transform (DWT): I DWT is an orthonormal transform W that reexpresses a time series of length N as a vector of DWT coefficients W: W = W, where W is an N N matrix such that = W T W particular W depends on the choice of wavelet filter, the most basic of which is the Haar filter (fancier filters include the Daubechies family of least asymmetric filters of width L denoted by LA(L), with L = 8 being a popular choice) level J, which determines the number of dyadic scales τ j = 2 j, j =, 2,..., J, involved in the transform 9

44 Overview of Discrete Wavelet Transform (DWT): II DWT coefficient vector W can be partitioned into J subvectors of wavelet coefficients W j, j =, 2,..., J, along with one sub-vector of scaling coefficients V J wavelet coefficients in W j are associated with changes in averages over a scale of τ j, whereas the scaling coefficients in V J are associated with averages over a scale of 2τ J as a concrete example, let s look at a level J = 4 Haar DWT of the AR time series 2

45 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series 2

46 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale τ = wavelet coefficient highlighted 2

47 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale τ = wavelet coefficient highlighted 2

48 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale τ 2 = 2 wavelet coefficient highlighted 2

49 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale τ 2 = 2 wavelet coefficient highlighted 2

50 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale τ 4 = 8 wavelet coefficient highlighted 2

51 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale τ 4 = 8 wavelet coefficient highlighted 2

52 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

53 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

54 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

55 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

56 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

57 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

58 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

59 V 4 W DWT of Autoregressive Process: I level J = 4 Haar DWT of AR series, with scale 2 τ 4 = 6 scaling coefficient highlighted 2

60 DWT of Autoregressive Process: II V 4 W Haar DWT of AR series and sample ACSs for each W j & V 4, along with 95% confidence intervals for white noise 22

61 DWT of Fractionally Differenced Process V 4 W Haar DWT of FD series and sample ACSs for each W j & V 4, along with 95% confidence intervals for white noise 23

62 DWT as a Decorrelating Transform for many (but not all!) time series, DWT acts as a decorrelating transform: to a good approximation, each W j is a sample of a white noise process, and coefficients from different sub-vectors W j and W j are also pairwise uncorrelated variance of coefficients in W j depends on j scaling coefficients V J are still autocorrelated, but there will be just a few of them if J is selected to be large decorrelating property holds particularly well for FD and other processes with long-range dependence above suggests the following recipe for wavelet-domain bootstrapping 24

63 Recipe for Wavelet-Domain Bootstrapping. given of length N = 2 J, compute level J DWT (the choice J = J 3 yields 8 coefficients in W J and V J ) 2. randomly sample with replacement from W j to create bootstrapped vector W (b) j, j =,..., J 3. create V (b) J using a parametric bootstrap 4. apply W T to W (b),..., W(b) J and V (b) J to obtain bootstrapped time series (b) and then form corresponding ˆρ (b) repeat above many times to build up sample distribution of bootstrapped autocorrelations 25

64 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

65 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

66 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

67 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

68 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

69 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

70 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

71 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

72 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

73 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

74 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

75 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

76 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

77 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

78 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

79 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

80 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

81 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

82 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

83 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

84 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

85 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

86 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

87 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

88 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

89 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

90 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

91 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

92 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

93 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

94 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

95 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

96 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

97 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

98 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

99 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

100 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 25

101 Illustration of Wavelet-Domain Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and waveletdomain bootstrap thereof (right-hand) 26

102 Wavelet-Domain Bootstrapping of AR Series approximations to true PDF using (a) Haar & (b) LA(8) wavelets (a) (b) ˆρ (m) ˆρ (m) using 5 AR time series and the Haar DWT yields: average of 5 sample means. =.67 (truth. =.86) average of 5 sample SDs. =.7 (truth. =.48) using 5 AR time series and the LA(8) DWT yields: average of 5 sample means. =.8 (truth. =.86) average of 5 sample SDs. =.55 (truth. =.48) vertical line indicates ˆρ 27

103 Wavelet-Domain Bootstrapping of FD Series approximations to true PDF using (a) Haar & (b) LA(8) wavelets (a) (b) ˆρ (m) ˆρ (m) using 5 FD time series and the Haar DWT yields: average of 5 sample means. =.35 (truth. =.53) average of 5 sample SDs. =.96 (truth. =.7) using 5 FD time series and the LA(8) DWT yields: average of 5 sample means. =.43 (truth. =.53) average of 5 sample SDs. =.98 (truth. =.7) vertical line indicates ˆρ 28

104 Effect of Non-Gaussianity: I wavelet-domain bootstrapping works well if we can assume Gaussianity, but can be problematic if this assumption fails for non-gaussian series, wavelet-domain bootstraps are typically closer to Gaussianity than original series, which poses a problem for assessing variability in certain statistics 29

105 Effect of Non-Gaussianity: II consider Gaussian white noise t and Y t = sign{ t } 2 t : 4.5 t 4. 2 Y t right-hand plots show estimated PDFs and true PDFs 3

106 Effect of Non-Gaussianity: III wavelet-domain bootstraps of t and Y t = sign{ t } 2 t : 4.5 t 4. 2 Y t right-hand plots show estimated PDFs and true original PDFs 3

107 Tree-Based Bootstrapping to preserve non-gaussianity, consider using groups ( trees ) of wavelet coefficients co-located across small scales as basic sampling unit for bootstrapping at those scales wavelet coefficients at large scales treated in same way as in usual wavelet-domain bootstrap scaling coefficients handled using parametric bootstrap certain wavelet-based signal denoising schemes for non-gaussian noise treat small scales in a special way and large scales in the same way as in the Gaussian case (see, e.g., Gao, 997) tree-based structuring of wavelet coefficients is key idea behind denoising using Markov models (Crouse et al., 998) and notion of wavelet footprints (Dragotti and Vetterli, 23) 32

108 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

109 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

110 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

111 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

112 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

113 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

114 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

115 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

116 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

117 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

118 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

119 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

120 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

121 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

122 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

123 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

124 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

125 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

126 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

127 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

128 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

129 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

130 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

131 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

132 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

133 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 32

134 Illustration of Tree-Based Bootstrapping V 4 W Haar DWT of FD(.45) series (left-hand column) and level j = 3 tree-based bootstrap thereof (right-hand) 33

135 Tree-Based Bootstraps of Non-Gaussian White Noise Y t (top row) and j = Haar tree-based bootstrap (bottom) 2 Y t 2 Y (b) t right-hand plots show estimated PDFs and true original PDF 33

136 Tree-Based Bootstraps of Non-Gaussian White Noise Y t (top row) and j = 2 Haar tree-based bootstrap (bottom) 2 Y t 2 Y (b) t right-hand plots show estimated PDFs and true original PDF 33

137 Tree-Based Bootstraps of Non-Gaussian White Noise Y t (top row) and j = 3 Haar tree-based bootstrap (bottom) 2 Y t 2 Y (b) t right-hand plots show estimated PDFs and true original PDF 33

138 Tree-Based Bootstraps of Non-Gaussian White Noise Y t (top row) and j = 4 Haar tree-based bootstrap (bottom) 2 Y t 2 Y (b) t right-hand plots show estimated PDFs and true original PDF 33

139 Tree-Based Bootstraps of Non-Gaussian White Noise Y t (top row) and j = 5 Haar tree-based bootstrap (bottom) 2 Y t 2 Y (b) t right-hand plots show estimated PDFs and true original PDF 34

140 Summary of Computer Experiments LA(8) j = 2 j = 4 Statistic Process Parm Block DWT Tree Tree True mean AR FD SD AR FD time series of length N = 24 for each Y t = sign{ t } 2 t bootstrap samples from each series, yielding unit lag sample autocorrelations ˆρ (m) mean and SD of sample autocorrelations recorded table reports averages of these two statistics over 5 time series true values based on, generated series for each process 35

141 Application to BMW Stock Prices - I right-hand plot: log of daily returns on BMW share prices left-hand: nonparametric and Gaussian PDF estimates series has small unit lag sample autocorrelation: ˆρ. =.8. large sample theory appropriate for Gaussian white noise gives standard deviation of / N. =.3 36

142 Application to BMW Stock Prices - II bootstrap estimates of standard deviations: LA(8) j = 2 j = 4 Parm Block DWT Tree Tree Gaussian SD est since ˆρ =.8, bootstrap methods all confirm presence of autocorrelation (small, but presumably exploitable by traders) 37

143 Concluding Remarks wavelet-domain & tree-based bootstraps competitive with parametric & block bootstraps for series with short-range dependence and offer improvement in case of long-range dependence results to date for tree-based bootstrapping encouraging, but many questions need to be answered, including: are there statistics & non-gaussian series for which treebased approach offers more than just a marginal improvement over wavelet-domain approach? what are asymptotic properties of tree-based approach? how can the tree-based approach be adjusted to handle stationary processes that are not well decorrelated by the DWT? 38

144 Thanks to... Ann Maharaj and Giovanni Forchini for opportunity to speak CSIRO Mathematics, Informatics and Statistics (CMIS) for visiting scientist position allowing for extended Australian visit Crime Scene for 2nd place finish in Melbourne Cup (got $3 in $2 sweep first winnings ever from a horse race!!!) 39

145 References: I C. Angelini, D. Cava, G. Katul and B. Vidakovic (25), Resampling Hierarchical Processes in the Wavelet Domain: A Case Study Using Atmospheric Turbulence, Physica D, 27, pp M. Breakspear, M. J. Brammer, E. T. Bullmore, P. Das and L. M. Williams (24), Spatiotemporal Wavelet Resampling for Functional Neuroimaging Data, Human Brain Mapping, 23, pp. 25 M. Breakspear, M. Brammer and P. A. Robinson (23), Construction of Multivariate Surrogate Sets from Nonlinear Data Using the Wavelet Transform, Physica D, 82, pp. 22 E. Bullmore, J. Fadili, V. Maxim, L. Şendur, B. Whitcher, J. Suckling, M. Brammer and M. Breakspear (24), Wavelets and Functional Magnetic Resonance Imaging of the Human Brain, NeuroImage, 23, pp. S234-S249 E. Bullmore, C. Long, J. Suckling, J. Fadili, G. Calvert, F. Zelaya, T. A. Carpenter and M. Brammer (2), Colored Noise and Computational Inference in Neurophysiological (fmri) Time Series Analysis: Resampling Methods in Time and Wavelet Domains, Human Brain Mapping, 2, pp

146 References: II M.S. Crouse, R. D. Nowak and R. G. Baraniuk (998), Wavelet-Based Statistical Signal Processing using Hidden Markov Models, IEEE Transactions on Signal Processing, 46(4), pp A. C. Davison and D. V. Hinkley (997), Bootstrap Methods and their Applications, Cambridge, England: Cambridge University Press P. L. Dragotti and M. Vetterli (23), Wavelet Footprints: Theory, Algorithms, and Applications, IEEE Transactions on Signal Processing, 5(5), pp H. Feng, T. R. Willemain and N. Shang (25), Wavelet-Based Bootstrap for Time Series Analysis, Communications in Statistics: Simulation and Computation, 34(2), pp W. A. Fuller (996), Introduction to Statistical Time Series (2nd Edition), New York: John Wiley & Sons. H.-Ye. Gao (997), Choice of Thresholds for Wavelet Shrinkage estimate of the Spectrum, Journal of Time Series Analysis, 8(3), pp S. Golia (22), Evaluating the GPH Estimator via Bootstrap Technique, in Proceedings in Computational Statistics COMPSTAT22, edited by W. Härdle and B. Ronz. Heidelberg: Physica Verlag, pp

147 References: III L. Lin, Y. Fan and L. Tan (28), Blockwise Bootstrap Wavelet in Nonparametric Regression Model with Weakly Dependent Processes. Metrika, 67(), pp D. B. Percival and W. L. B. Constantine (26), Exact Simulation of Gaussian Time Series from Nonparametric Spectral Estimates with Application to Bootstrapping, Statistics and Computing, 6(), pp D. B. Percival, S. Sardy and A. C. Davison (2), Wavestrapping Time Series: Adaptive Wavelet-Based Bootstrapping, in Nonlinear and Nonstationary Signal Processing, edited by W. J. Fitzgerald, R. L. Smith, A. T. Walden and P. C. Young. Cambridge, England: Cambridge University Press, pp D. B. Percival and A. T. Walden (2), Wavelet Methods for Time Series Analysis, Cambridge, England: Cambridge University Press A. M. Sabatini (999), Wavelet-Based Estimation of /f-type Signal Parameters: Confidence Intervals Using the Bootstrap, IEEE Transactions on Signal Processing, 47(2), pp A. M. Sabatini (26), A Wavelet-Based Bootstrap Method Applied to Inertial Sensor Stochastic Error Modelling Using the Allan Variance, Measurement Science and Technology, 7, pp

148 References: IV B. J Whitcher (26), Wavelet-Based Bootstrapping of Spatial Patterns on a Finite Lattice, Computational Statistics & Data Analysis, 5(9), pp