INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY. Lecture -20 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

Transcription

1 INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY Lecture -20 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

2 Summary of the previous lecture Case study -3: Monthly streamflows ate KRS reservoir Plots of Time series, Correlogram, Partial Autocorrelation function and Power spectrum Standardization to remove periodicities Candidate ARMA models : (contiguous and noncontiguous) Log Likelihood Mean square error 2

3 CASE STUDIES - ARMA MODELS 3

4 Case study 3 Monthly Stream flow (cum/sec) Statistics( ) for Cauvery River at Krishna Raja Sagar Reservoir is considered in the case study. Time series of the data, auto correlation function, partial auto correlation function and the power spectrum are plotted. The series indicates presence of periodicities. The series is standardized to remove the periodicities. 4

5 Case study 3 (Contd.) Standardized series is considered for fitting the ARMA models Total length of the data set N = 480 Half the data set (240 values) is used to construct the model and other half is used for validation. Both contiguous and non-contiguous models are studied Non-contiguous models consider the most significant AR and MA terms leaving out the intermediate terms 5

6 Case study 3 (Contd.) Contiguous models: Sl. No Model Likelihood values 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1) L N = ln ( σ ) n 2 i i i 6

7 Case study 3 (Contd.) Non-contiguous models*: Sl. No Model Likelihood values 1 ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(2,2) ARMA(2,3) ARMA(3,2) ARMA(3,3) ARMA(4,2) *: The last AR and MA terms correspond to the 12 th lag 7

8 Case study 3 (Contd.) For this time series, the likelihood values for contiguous model = non-contiguous model = Hence contiguous ARMA(4,0) can be used. The parameters for the selected model are as follows φ 1 = φ 2 = φ 3 = φ 4 = Constant =

9 Case study 3 (Contd.) Contiguous models: Forecasting Models Sl. No Model Mean square error values 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1)

10 Case study 3 (Contd.) Non-contiguous models: Sl. No Model Mean square error values 1 ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(2,2) ARMA(2,3) ARMA(3,2) ARMA(3,3) ARMA(4,2)

11 Case study 3 (Contd.) The simplest model AR(1) results in the least value of the MSE For one step forecasting, quite often the simplest model is appropriate Also as the number of parameters increases, the MSE increases which is contrary to the common belief that models with large number of parameters give better forecasts. AR(1) model is recommended for forecasting the series and the parameters are as follows φ 1 = and C =

12 Case study 3 (Contd.) Validation tests on the residual series Significance of residual mean Significance of periodicities Cumulative periodogram test or Bartlett s test White noise test Whittle s test Portmanteau test Residuals, Residual m1 m 2 et = Xt φjxt j + θjet j + C j= 1 j= 1 Data Simulated from the model 12

13 Case study 3 (Contd.) Significance of residual mean: Sl. No Model η(e) t 0.95 (239 ) 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) η ( e) = N ρ 1/2 ˆ1/2 e η(e) < t(0.95, 240 1); All models pass the test 13

14 Case study 3 (Contd.) Significance of periodicities: η e ( ) = γ 2 k N ( ) 4 ˆ ρ 1 2 γ k2 = α k 2 + β k 2 1 N ˆ ρ ˆ 1 = ˆcos α ω βsin ω N t= 1 n 2 α = e { ( ) ( )} 2 et kt kt cos ( ω t) k t k N t= 1 14

15 Case study 3 (Contd.) n 2 β = e sin ( ω t) k t k N t= 1 2π/ω k is the periodicity for which test is being carried out. η(e) < F α (2, N 2 ) Model passes the test 15

16 Case study 3 (Contd.) Significance of periodicities: Sl. No Model η Value for the periodicity 1 st 2 nd 3 rd 4 th F 0.95 (2,238 ) 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2)

17 Case study 3 (Contd.) Significance of periodicities by Bartlett s test : (Cumulative periodogram test) N 2 N k = etcos( kt) + etsin ( kt) N t= 1 N t= 1 γ ω ω g k = k j= 1 N /2 k = 1 γ γ 2 j 2 k k = 1,2, N/2 17

18 Case study 3 (Contd.) Cumulative periodogram for the original series without standardizing gk Lag, k

19 Case study 3 (Contd.) The confidence limits (1.35/(N/2) 1/2 = 0.087) are plotted for 95% confidence. (N = 480 in this case) Most part of the cumulative periodogram lies outside the significance bands confirming the presence of periodicity in the data. for k=40, a spike in the graph is seen indicating the significant periodicity This k corresponds to a periodicity of 12 months (480/40) k=80, corresponds to a periodicity of 6 months 19

20 Case study 3 (Contd.) Cumulative periodogram for the residual series of ARMA(4, 0) model gk Lag, k 20

21 Case study 3 (Contd.) The confidence limits (+1.35/(N/2) 1/2 = ) are plotted for 95% confidence. (N = 240, in this case) Cumulative periodogram lies within the significance bands confirming that no significant periodicity present in the residual series. The model passes the test. 21

22 Whittle s test for white noise White noise test (Whittle s test): This test is carried out to test the absence of correlation in the series. The covariance r k at lag k of the error series {e t } r N 1 k = ejej k N k j= k+ 1 k = 0, 1, 2,.k max The value of k max is normally chosen as 0.15N Ref: Kashyap R.L. and Ramachandra Rao.A, Dynamic stochastic models from empirical data, Academic press, New York,

23 Whittle s test for white noise The covariance matrix is r r r.. r r r r.. r r max k 1 2 Γ n1 =.. rk r max k r max max 1 0 k k max x k max 23

24 Whittle s test for white noise A statistic η(e) is defined as η e N ˆ ρ = 1 n1 1 ˆ ρ1 ( ) 0 n1 = k max Where ˆρ 0 is the lag zero correlation =1, and ˆ ρ = 1 det Γ det Γ n1 n11 The matrix Γ n1-1 is constructed by eliminating the last row and the last column from the Γ n1 matrix. 24

25 Whittle s test for white noise The statistic η(e) is approximately distributed as F α (n1, N n1 ), where α is the significance level at which the test is being carried out. If the value of η(e) < F α (n1, N n1 ), then the residual series is uncorrelated. 25

26 Case study 3 (Contd.) N ˆ ρ 0 η ( e) = 1 n1 1 ˆ ρ1 F 0.95 (2,239 ) n1 = 73 n1 = 49 n1 = 25 Model η η η ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) Whittle s test - white noise model fails 26

27 Case study 3 (Contd.) Portmanteau test for white noise: 2 n1 r k = k= 1 r0 ( e) ( N n1) χ 2 kmax = 48 kmax = 36 kmax = 24 kmax = (k max ) Model η η η η ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) model fails η 27

28 Case study 4 Monthly Stream flow ( ) of a river (37 years of monthly data); Flow in cumec Time (months) 28

29 Case study 4 (Contd.) Correlogram Correlation coefficient Lag 29

30 Case study 4 (Contd.) Partial Auto Correlation function 1 Fig 10 Partial Autocorrelation Function of flows PAC Partial Autocorrelations Lag Lag 30

31 Case study 4 (Contd.) Power Spectrum 1.2E+09 1E I(k) E+08 W(k) 31

32 Case study 4 (Contd.) Model name constant φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 φ 8 θ 1 θ 2 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(8,0) ARMA(1,1) ARMA(2,1) ARMA(3,1) ARMA(4,1) ARMA(1,2) ARMA(2,2) ARMA(0,1) ARMA(0,2)

33 Case study 4 (Contd.) Sl. No Model Mean Square Error Likelihood value 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(8,0) ARMA(1,1) ARMA(2,1) ARMA(3,1) ARMA(4,1) ARMA(1,2) ARMA(2,2) ARMA(0,1) ARMA(0,2)

34 Case study 4 (Contd.) Significance of residual mean: e η ( e) = N ρ 1/2 ˆ1/2 e is the estimate of the residual mean ˆρ is the estimate of the residual variance All models pass the test Model Test t η(e) t(α, N 1) ARMA(1,0) E ARMA(2,0) E ARMA(3,0) E ARMA(4,0) E ARMA(5,0) E ARMA(6,0) E ARMA(7,0) E ARMA(8,0) E ARMA(1,1) ARMA(2,1) ARMA(3,1) ARMA(4,1) ARMA(1,2) ARMA(2,2) ARMA(0,1) ARMA(0,2)

35 Case study 5 Sakleshpur Annual Rainfall Data ( ) Rainfall in mm Time (years) 35

36 Case study 5 (Contd.) Correlogram 36

37 Case study 5 (Contd.) PAC function 37

38 Case study 5 (Contd.) Model Likelihood AR(1) AR(2) AR(3) AR(4) AR(5) AR(6) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1) ARMA(3,2)

39 Case study 5 (Contd.) ARMA(5,0) is selected with highest likelihood value The parameters for the selected model are as follows φ 1 = φ 2 = φ 3 = φ 4 = φ 5 = Constant =

40 Case study 5 (Contd.) Significance of residual mean Model η(e) t 0.95 (N ) ARMA(5,0)

41 Case study 5 (Contd.) Significance of periodicities: Periodicity η F 0.95 (2,239 ) 1 st nd rd th th th

42 Case study 5 (Contd.) Whittle s white noise test: Model η F 0.95 (2, N-2 ) ARMA(5,0)

43 Case study 5 (Contd.) Model MSE AR(1) AR(2) AR(3) AR(4) AR(5) AR(6) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1) ARMA(3,2)

44 Case study 5 (Contd.) ARMA(1, 2) is selected with least MSE value for one step forecasting The parameters for the selected model are as follows φ 1 = θ 1 = θ 2 = Constant =

45 Case study 5 (Contd.) Significance of residual mean Model η(e) t 0.95 (N ) ARMA(1, 2)

46 Case study 5 (Contd.) Significance of periodicities: Periodicity η F 0.95 (2,239 ) 1 st nd rd th th th

47 Case study 5 (Contd.) Whittle s white noise test: Model η F 0.95 (2, N-2 ) ARMA(1, 2)

48 Markov Chains

49 Markov Chains A Markov chain is a stochastic process with the property that value of process X t at time t depends on its value at time t-1 and not on the sequence of other values (X t-2, X t-3,. X 0 ) that the process passed through in arriving at X t-1. [,,... ] = [ ] PX X X X PX X t t 1 t 2 0 t t 1 Single step Markov chain 49

50 Markov Chains P X = a X = a t j t 1 i This conditional probability gives the probability at time t will be in state j, given that the process was in state i at time t-1. The conditional probability is independent of the states occupied prior to t-1. For example, if X t-1 is a dry day, we would be interested in the probability that X t is a dry day or a wet day. This probability is commonly called as transition probability 50

51 Markov Chains P X = a X = a = P t t j t 1 i ij P ij t Usually written as indicating the probability of a step from a i to a j at time t. If P ij is independent of time, then the Markov chain is said to be homogeneous. P t ij = t P + τ ij i.e., v t and τ the transition probabilities remain same across time 51

52 Markov Chains Transition Probability Matrix(TPM): P t+1 t m P P P.. P P P P.. P P m m 31 =.. m.. P P P m1 m2 mm m x m 52

53 Markov Chains m j= 1 P ij = 1 V i Elements in any row of TPM sum to unity TPM can be estimated from observed data by enumerating the number of times the observed data went from state i to j P j (n) is the probability of being in state j in time step n. 53

54 Markov Chains p j (0) is the probability of being in state j in period t = 0. ( 0) ( 0) ( 0) ( 0) p = p1 p2.. p. Probability m ( n) ( n) ( n) ( n) 1 2 If p (0) is given and TPM is given 1 m 1 m vector at time 0 p = p p.. p. Probability ( 1) ( 0) p = p P m vector at time n 54

55 Markov Chains P P P.. P P P P.. P ( 1) ( 0) ( 0) ( 0) p p p p P m m = m 31 And so on ( 0) ( 0) ( 0). P P P m1 m2 mm = p1 P11+ p2 P pm Pm 1. Probability of ( 0) ( 0) ( 0) going to state 1 = p1 P12+ p2 P pm Pm2. Probability of going to state 2 55

56 Markov Chains Therefore In general, ( 1) ( 1) ( 1) ( 1) p p p p = m 1 m ( 2) ( 1) p = p P ( 0) = p P P ( ) = p P 0 2 ( n) ( 0) p = p P n 56

57 Markov Chains As the process advances in time, p j (n) becomes less dependent on p (0) The probability of being in state j after a large number of time steps becomes independent of the initial state of the process. The process reaches a steady state at large n p= p P n As the process reaches steady state, the probability vector remains constant 57

58 Example 1 Consider the TPM for a 2-state first order homogeneous Markov chain as TPM = State 1 is a non-rainy day and state 2 is a rainy day Obtain the 1. probability of day 1 is non-rainfall day / day 0 is rainfall day 2. probability of day 2 is rainfall day / day 0 is non-rainfall day 3. probability of day 100 is rainfall day / day 0 is non-rainfall day 58

59 Example 1 (contd.) 1. probability of day 1 is non-rainfall day / day 0 is rainfall day TPM No rain rain No rain = rain The probability is probability of day 2 is rainfall day / day 0 is nonrainfall day ( ) ( ) p = p P

60 Example 1 (contd.) ( 2) p = [ ] = [ ] The probability is probability of day 100 is rainfall day / day 0 is nonrainfall day ( n) ( 0) p = p P n 60

61 Example 1 (contd.) 2 P = P P = = P = P P = P = P P = P = P P =

62 Example 1 (contd.) Steady state probability p = [ ] For steady state, n p= p P = [ ] = [ ] 62