Autoregressive Models Fourier Analysis Wavelets

Transcription

1 Autoregressive Models Fourier Analysis Wavelets

2 BFR Flood w/10yr smooth Spectrum Annual Max: Day of Water year Flood magnitude vs. timing Jain & Lall, 2000

3 Blacksmith Fork, Hyrum, UT Analyses of Flood Statistics using a 30 year Moving Window 100 yr flood (cfs) From Jain and Lall (2000) Mean(log(Q)) Var(log(Q))

4 Flood Variance given DJF NINO3 and PDO NINO3 PDO NINO3 PDO Correlations: Derived using weighted local regression with 30 neighbors Log(Q) vs. DJF NINO vs. DJF PDO 0.32 Jain and Lall, 2000

5 El Nino and its variations over time Changing periodicity

6 Wavelet Analysis of 1000 year sample of annual maximum NINO3 from a 110,000 year integration of the Cane-Zebiak Model with stationary forcing ( Clement and Cane, 1999)

7 y t = 0.9y t 1 +sqrt(0.19)*n(0,1) One could actually think of this as a model for daily streamflow. During periods of no rainfall, we could consider that groundwater releases water to the stream (base flow), and during periods of rainfall, the groundwater is recharged, and the streamflow is enhanced. If the groundwater storage (volume or water level) is denoted by S, then the model for base flow in the stream can be determined during non rainy periods from ds/dt = rate of change of groundwater storage = Q g, where Q g is the rate of streamflow contributed by groundwater storage. Now for a homogeneous aquifer with relatively constant water level over space, we can relate Q g and S linearly, i.e., Q g =bs. This leads to the usual recession curve solution Q g =Q 0 e rt, where r is the recession rate. The behavior of y t = c*y t 1 is similar to that of the base flow recession curve. The noise term modifies this behavior, and if we get a random sequence o positive noise terms, the time series rises at around index 20 (or 50, or 90) in the example.

8

9 ACF Partial ACF ACF shown with bars, and ACF based on AR1 model shown with circles AR(k) model fit using AIC: y t = 0.71y t y t y t 3

10 The Data: y t t =1 n Can be a raw time series or a derived function of a time series; e.g., Or The Basic Questions: Is the probability distribution of y t invariant with time? Is µ(y t ) independent of t? If not, what is the nature and rate of change of µ(y t ) with t. Some Underlying Models: a. y t ~ f(θ); N(µ,σ 2 ) or Exp (λ) b. y t ~ ARMA(p,q), e.g., y t = ay t 1 +e t c. y t =g(t) +e t ; e.g., (3) (1) y t = a 0 +a 1 t +e t (2) y t = a 0 +a 1 t +a 2 sin(wt) +e t d. y t =g(t) +h(z) + e t ;e.g., (1) y t = a 0 +a 1 t +a 2 z t 0.5 +e t (2) y t = a 0 +a 1 t+e t e t = b 1 e t 1 +v t (3) y t = m t +e t m t = a 1 m t 1 + a 3 m t 2 +v t

11 Some Underlying Models: No trend a. y t ~ f(θ); N(µ,σ 2 ) or Exp (λ) Trend Monotonic Periodic Shift b. y t ~ ARMA(p,q), e.g., y t = ay t 1 +e t c. y t =g(t) +e t ; e.g., (3) (1) y t = a 0 +a 1 t +e t (2) y t = a 0 +a 1 t +a 2 sin(wt) +e t Trend +Exogenous Factor Ex. Deterministic Ex. Random (AR) State Space Model (Obs.=Process+Noise) d. y t =g(t) +h(z) + e t ;e.g., (1) y t = a 0 +a 1 t +a 2 z t 0.5 +e t (2) y t = a 0 +a 1 t+e t e t = b 1 e t 1 +v t (3) y t = m t +e t m t = a 1 m t 1 + a 3 m t 2 +v t

12 y= t P value for slope=0.3

13 y= t P values: (0.0000,0.0257)

14 y= t P values: (0.000,0.0000)

15

16 y t = t y t 1 P values: (0.78,0.96,0.0000) r t =y t 0.91y t 1 r t =0+0t P values: (1,1)

17 y= t P value for slope=0.007

18

19 AR(1) y t = 0.9y t 1 +e t y t 1 =0.9y t 2 +e t 1 Or y t =0.9(0.9y t 2 +e t 1 )+e t = 0.92 y t 1 + noise Or y t =0.9 k y t 1 + noise If y t has mean 0, variance 1, then cor(y t, y t k )=r k Linear Trend y t =0.9t+e t y t 1 = 0.9(t 1)+ e t 1 Or y t =y t 1 +(1 0.9)+noise Or y t =y t k +(k 0.9)+noise i.e., y t and y t k have a mean value that differs by a constant proportional to k Here the ACF decreases linearly with lag k

20 y= t P value for slope=0.13 y= t+1.28sin(13.5π/180t) P values=(0.004,0.0001)

21 y= t P value=(0.000,0.000) y= t+0.88δ s (t=50) P values=(0,0.49, 0.04) from y = 1 + δs(t=50) + Normal (0,1). The step of size 1 is placed at t=50.

22 y= t P value: (0.0003,0.0000) y t = t y t 1 p=values : (0.47,0.10,0.0000)

23 y t =r*y t 1 y t 2 r=1 r=1.2 r=1.5 r=1.8 r=1.9 r=1.95 r=1.99 r=2.0

24 The general problem of interest is illustrated by the chart on the left We suspect that there are periodic trends (e.g., annual cycle), and wish to identify them from a time series. If we know the 4 frequencies ω i, that went into building this model, then this is a linear regression problem with 8 predictors (4 ω i ) plus: A 1 =1.85, A 2 =0.8, A 3 =0.56, A 4 =0.81 from least squares fit assuming all phases are 0

25 How well can the Fourier basis approximate arbitrary functions (periodic)? What are good ways to identify the periodic trends? 1. The frequencies of interest are known 2. The frequencies of interest are not known What is identifiable, and what are the limitations of selected methods.

26 Since there is one periodic mode in the data (y t ), we seek to approximate it with 1 periodic function as shown above. We need to estimate four coefficients: A, ω, φ, and B This is a nonlinear regression problem, and we used the Solver in Excel to get the solution

27 We take the residuals from the previous model and repeat the process of approximating them with a sine function Note that the frequency is almost exactly double that of the previous case.

28 We take the residuals from the previous model and repeat the process of approximating them with a sin function Note that the frequency is now 3 times that of the original case.

29 We take the residuals from the previous model and repeat the process of approximating them with a sin function Note that the frequency is now 5 times that of the original case.

30 We take the residuals from the previous model and repeat the process of approximating them with a sin function Note that the frequency is now 6 times that of the original case.

31 A ω/2π φ B / / / / /

32 At this point, you ve seen two examples. 1. We knew the frequencies of interest, and solved for their amplitude using linear regression. 2. We did not know the frequency of interest, and tried to identify the appropriate frequency one at a time. We often know that there is an annual cycle in the data, but we may not be able to assume that it is sinusoidal. This was the point of the second example frequencies of interest need to be solved for. Thus, except in rather trivial cases, we are faced with the identification of the frequencies, their phase and amplitude. Ideally, we would like to postulate a model once, and solve for its parameters.

33 Consider the model above. For each frequency, ω i, that we consider, we need to identify 2 coefficients, A i, and B i. In addition, we need to estimate the mean. Since, we have n data points, we can estimate no more than n coefficients total, i.e., the max number of frequencies, I, we can consider is (n/2 1). The lowest frequency we can resolve with n years of data is 1/n cycles per year If the data is uniformly sampled with a sampling interval of dt units (e.g., 1), the highest frequency we can resolve is 1/(2dt). So, if we consider uniform sampling, the Fourier frequencies are of interest: ω i = {1/n, 3/n, 5/n, 1/2}/dt Note that if we use all the terms available to us, we cannot estimate e t, i.e., we have chosen to solve for coefficients for all possible harmonics, even if only 1 was important, and hence the rest of the terms will add up to the noise!!

34

35 The time series: Definitions: The Periodogram: P(ω i ) = (A i2 +B i2 )

36 Time series of yt

37 Time series of yt

38 Since the 1 st two are noise free, we expect a single peak and a flat spectrum. We see broad peaks and multiple small peaks. The first is because Splus smooths the periodogram. This leads to leakage of power. The second is because of edge effects and discretization (the frequency of the 1. The 1/12 frequency spectrum and time series signal is not a Fourier 2. The 1/24 frequency spectrum and time series frequency) 3. The sum of the two spectrum and time series 4. The residual from the triangular wave spectrum and time series

39 Since we fit so many coefficients to the data, and ended up with negligible degrees of freedom, the resulting amplitudes are random coefficients, and the periodogram is not a consistent estimator of the spectrum as n infinity, the estimates do not converge to the right estimate we simply estimate more coefficients and end up with the same degrees of freedom. Consequently, it is useful to think of these estimates as data rather than statistics on which hypotheses can be tested. It is common to smooth the resulting estimates this entails an assumption that the spectrum is a smooth function of frequency. However, this is at odds with the notion that we get sharp peaks corresponding to periodic trends. Thus, most such estimates are not terribly satisfactory. Smoothing Methods include: Maximum Entropy Spectral Estimation (MESA) Logspline Spectral estimation (Kooperberg et al.) Multi Taper Method (MTM)

40 The spectrum and the series autocovariance function are related as: where c k is the lag k autocovariance This suggests that if we have an estimate of the autocovariance function, then we can readily obtain the spectrum. The problem is that sample estimates of the ACF degrade as k increases, since the number of pairs that are available to compute the sample statistic decreases. This situation is often addressed by fitting the best AR model to the data, and then using the filtered autocovariance function to estimate the spectrum.

41

42

43 Autocorrelation Function:Example 2 Fitted AR model:example 2 Note the periodic structure in the ACF, and also the periodic sequence of positive and negative coefficients for the AR model. These clearly suggest that we have underlying periodic trends. The AR model explains 99.8% of the variance of the original series.

44 Now that we know the frequencies of interest, we can also directly find the amplitudes by fitting a linear regression (they are also available from the Fourier transform of the filtered autocorrelation function keeping the AR model terms). f i A i B i / / / / / Compare A ω/2π φ B / / / / /

45 1. The 1/12 frequency and its spectrum. 2. The 1/24 frequency and its spectrum. 3. The sum of the two and its spectrum. 4. The residual from the triangular wave and its spectrum. The results are much cleaner than those for the periodogram. However, there is still a fair amount of smearing of spectral power in the neighborhood of the peak. There was no noise in this data set, other than the approximation error because of the nonsinusoidal signal.

46

47

48 Data from October 1974 September 2002 Show: Raw periodogram Smoothed periodogram periodogram from AR model best fit by AIC

49 Some peaks are shown by the blue dotted lines at a number of frequencies. The period. The annual cycle is the most prominent peak. The 6 year and ~2 year ones may be climatically related. The 0.5 and 0.2 values are harmonics of the annual cycle.

50 AR fit by AIC: AR(1) ρ = 0.62 Smoothed to a fault no indication of any annual periodicity. AR(1) model is not able to reproduce the annual cycle.

51

52