INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY. Lecture -18 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

Transcription

1 INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY Lecture -18 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

2 Summary of the previous lecture Model selection Mean square error Model testing/validation Significance of residual mean Significance of periodicities Cumulative periodogram test or Bartlett s test White noise test Whittle s test Portmanteau test 2

3 ARIMA Models Data Generation: Consider AR(1) model, X t = φ 1 X t-1 + e t e.g., φ 1 = 0.5 : AR(1) model is X t = 0.5X t-1 + e t zero mean; uncorrelated 3

4 ARIMA Models X 1 = 3.0 (assumed) X 2 = 0.5* = X 3 = 0.5* = 2.14 And so on 4

5 ARIMA Models Consider ARMA (1, 1) model, X t = φ 1 X t-1 + θ 1 e t-1 + e t e.g., φ 1 = 0.5, θ 1 = 0.4 : ARMA(1, 1) model is written as X t = 0.5X t e t-1 + e t 5

6 ARIMA Models Say X 1 = 3.0 X 2 = 0.5* * = X 3 = 0.5* * = 2.39 X 4 = 0.5* * = and so on... 6

7 ARIMA Models Data Forecasting: Consider AR(1) model, X t = φ 1 X t-1 + e t Expected value is considered. Xˆ [ ] = φ [ ] + [ ] E X E X E e t t 1 t 1 t = φ X 1 t 1 Expected value of e t is zero 7

8 ARIMA Models Consider ARMA(1, 1) model, X t = φ 1 X t-1 + θ 1 e t-1 + e t E[X t ] = φ 1 X t-1 + θ 1 e t Error in forecast in the previous period e.g., φ 1 = 0.5, θ 1 = 0.4: Forecast model is written as X t = 0.5X t e t-1 8

9 ARIMA Models Say X 1 = 3.0 X ˆ = = 1.5 Initial error assumed to be zero X 2 = 2.8 Error e 2 = = 1.3 X ˆ = = 1.92 Actual value to be used 9

10 ARIMA Models X 3 = 1.8 Error e 3 = = ( ) X ˆ = = and so on. 10

11 CASE STUDIES 11

12 Case study 1 Rainfall data for Bangalore city is considered. Time series plot, auto correlation function, partial auto correlation function and power spectrum area plotted for daily, monthly and yearly data. 12

13 Case study 1 (Contd.) Daily data Time series plot Rainfall in mm Time 13

14 Case study 1 (Contd.) Daily data Correlogram 0.2 Sample Autocorrelation Function 0.15 Sample Autocorrelation Lag 14

15 Case study 1 (Contd.) Daily data Partial auto correlation function 15

16 Case study 1 (Contd.) Daily data Power spectrum I(k) Wk 16

17 Case study 1 (Contd.) Monthly data Time series plot Rainfall in mm Time 17

18 Case study 1 (Contd.) Monthly data Correlogram 18

19 Case study 1 (Contd.) Monthly data Partial auto correlation function 19

20 Case study 1 (Contd.) Monthly data Power spectrum I(k) ω = k 2π P 20

21 Case study 1 (Contd.) Yearly data Time series plot Rainfall in mm Time 21

22 Case study 1 (Contd.) Yearly data Correlogram 22

23 Case study 1 (Contd.) Yearly data Partial auto correlation function 23

24 Case study 1 (Contd.) Yearly data Power spectrum I(k) Wk 24

25 Case study 2 Stream flow data for a river is considered in the case study. Time series plot, auto correlation function, partial auto correlation function and power spectrum area plotted for the original data and the first differenced data. 25

26 Case study 2 (Contd.) Time series plot shows an increasing trend Flow in cumec Time 26

27 Case study 2 (Contd.) Correlogram: 27

28 Case study 2 (Contd.) Partial auto correlation function 28

29 Case study 2 (Contd.) Power spectrum 29

30 Case study 2 (Contd.) Time series plot first differenced data : No trend 30

31 Case study 2 (Contd.) Correlogram differenced data: First two correlations significant 31

32 Case study 2 (Contd.) Partial auto correlation function differenced data 32

33 Case study 2 (Contd.) Power spectrum differenced data I(k) Wk 33

34 CASE STUDIES ON ARMA MODELS 34

35 Case study 3 Monthly Stream flow (cum/sec) Statistics( ) for Cauvery River at Krishna Raja Sagar Reservoir is considered in the case study. Time series of the data, auto correlation function, partial auto correlation function and the power spectrum are plotted. The series indicates presence of periodicities. The series is standardized to remove the periodicities. 35

36 Case study 3 (Contd.) Time series plot original series Flow in cumec Time 36

37 Case study 3 (Contd.) Correlogram original series 37

38 Case study 3 (Contd.) Partial auto correlation function original series 38

39 Case study 3 (Contd.) Power spectrum original series I(k) Wk 39

40 Case study 3 (Contd.) Time series plot Standardized series Standardized Flow Time 40

41 Case study 3 (Contd.) Correlogram Standardized series 41

42 Case study 3 (Contd.) Partial auto correlation function Standardized series 42

43 Case study 3 (Contd.) Power spectrum Standardized series I(k) Wk 43

44 Case study 3 (Contd.) Standardized series is considered for fitting the ARMA models Total length of the data set N = 480 Half the data set (240 values) is used to construct the model and other half is used for validation. Both contiguous and non-contiguous models are studied Non-contiguous models consider the most significant AR and MA terms leaving out the intermediate terms 44

45 Case study 3 (Contd.) For example, a non-contiguous AR(3), with significant dependence at lags 1, 4 and 12, the model is written as X = φ X + φ X + φ X + e t 1 t 1 4 t 4 12 t 12 t Similarly the moving average terms are also considered and a non-contiguous ARMA(3, 3) is written as X = φ X + φ X + φ X + t 1 t 1 4 t 4 12 t 12 θ e + θ e + θ e + e 1 t 1 4 t 4 12 t 12 t 45

46 Case study 3 (Contd.) The advantage of non-contiguous models is the reduction of number of AR and MA parameters to be estimated. Exactly which terms to include is to be decided based on the correlogram and spectral analysis of the series under consideration. For a given series, the choice of contiguous or a non-contiguous model is decided by the relative likelihood values for the two models 46

47 Case study 3 (Contd.) Contiguous models: Sl. No Model Likelihood values 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1) L N = ln ( σ ) n 2 i i i 47

48 Case study 3 (Contd.) Non-contiguous models: Sl. No Model Likelihood values 1 ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(2,2) ARMA(2,3) ARMA(3,2) ARMA(3,3) ARMA(4,2)

49 Case study 3 (Contd.) For this time series, the likelihood values for contiguous model = non-contiguous model = Hence contiguous ARMA(4,0) can be used. The parameters for the selected model are as follows φ 1 = φ 2 = φ 3 = φ 4 = Constant =

50 Case study 3 (Contd.) Contiguous models: Forecasting Models Sl. No Model Mean square error values 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1)

51 Case study 3 (Contd.) Non-contiguous models: Sl. No Model Mean square error values 1 ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(2,2) ARMA(2,3) ARMA(3,2) ARMA(3,3) ARMA(4,2)

52 Case study 3 (Contd.) The simplest model AR(1) results in the least value of the MSE For one step forecasting, quite often the simplest model is appropriate Also as the number of parameters increases, the MSE increases which is contrary to the common belief that models with large number of parameters give better forecasts. AR(1) model is recommended for forecasting the series and the parameters are as follows φ 1 = and C =

53 Case study 3 (Contd.) Validation tests on the residual series Significance of residual mean Significance of periodicities Cumulative periodogram test or Bartlett s test White noise test Whittle s test Portmanteau test Residuals, Residual m1 m 2 e = X φ X + θ e t t j t j j t j j= 1 j= 1 Data Simulated from the model 53

54 Case study 3 (Contd.) Significance of residual mean: Sl. No Model η(e) t 0.95 (239 ) 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) η ( e) = N ρ 1/2 ˆ1/2 e η(e) < t(0.95, 240 1); All models pass the test 54

55 Case study 3 (Contd.) Significance of periodicities: η e ( ) = γ 2 k N ( ) 4 ˆ ρ 1 2 γ k2 = α k 2 + β k 2 1 N ˆ ρ ˆ 1 = ˆcos α ω βsin ω N t= 1 n 2 α = e { ( ) ( )} 2 et kt kt cos ( ω t) k t k N t= 1 55

56 Case study 3 (Contd.) n 2 β = e sin ( ω t) k t k N t= 1 2π/ω k is the periodicity for which test is being carried out. η(e) < F α (2, N 2 ) Model passes the test 56

57 Case study 3 (Contd.) Significance of periodicities: Sl. No Model η Value for the periodicity 1 st 2 nd 3 rd 4 th F 0.95 (2,238 ) 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2)

58 Case study 3 (Contd.) Significance of periodicities by Bartlett s test : (Cumulative periodogram test) N 2 N k = etcos( kt) + etsin ( kt) N t= 1 N t= 1 γ ω ω g k = k j= 1 N /2 k = 1 γ γ 2 j 2 k k = 1,2, N/2 58

59 Case study 3 (Contd.) Cumulative periodogram for the original series without standardizing gk Lag, k

60 Case study 3 (Contd.) The confidence limits (+1.35/(N/2) 1/2 = ) are plotted for 95% confidence. Cumulative periodogram lies outside the significance bands confirming the presence of periodicity in the data. for k=40, sudden increase in the graph is seen indicating the significant periodicity This k corresponds to a periodicity of 12 months (480/40) k=80, corresponds to a periodicity of 6 months 60

61 Case study 3 (Contd.) Cumulative periodogram for the residual series of ARMA(4, 0) model gk Lag, k 61

62 Case study 3 (Contd.) The confidence limits (+1.35/(N/2) 1/2 = ) are plotted for 95% confidence. Cumulative periodogram lies within the significance bands confirming that no significant periodicity present in the residual series. The model pass the test. 62

63 Case study 3 (Contd.) N ˆ ρ 0 η ( e) = 1 n1 1 ˆ ρ1 F 0.95 (2,239 ) n1 = 73 n1 = 49 n1 = 25 Model η η η ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) Whittle s test for white noise: model fails 63

64 Case study 3 (Contd.) Portmanteau test for white noise: 2 n1 r k = k= 1 r0 ( e) ( N n1) χ 2 kmax = 48 kmax = 36 kmax = 24 kmax = (k max ) Model η η η η ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) model fails η 64

65 Case study 4 Monthly Stream flow ( ) of a river; Flow in cumec Time (months) 65

66 Case study 4 (Contd.) Correlogram Correlation coefficient Lag 66

67 Case study 4 (Contd.) Partial Auto Correlation function 1 Fig 10 Partial Autocorrelation Function of flows PAC Partial Autocorrelations Lag Lag 67

68 Case study 4 (Contd.) Power Spectrum 1.2E+09 1E I(k) E+08 W(k) 68

69 Case study 4 (Contd.) Model name constant φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 φ 8 θ 1 θ 2 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(8,0) ARMA(1,1) ARMA(2,1) ARMA(3,1) ARMA(4,1) ARMA(1,2) ARMA(2,2) ARMA(3,2) ARMA(0,1) ARMA(0,2)

70 Case study 4 (Contd.) Sl. No Model Mean Square Error Likelihood value 1 ARMA(1,0) ARMA(2,0) ARMA(3,0) ARMA(4,0) ARMA(5,0) ARMA(6,0) ARMA(7,0) ARMA(8,0) ARMA(1,1) ARMA(2,1) ARMA(3,1) ARMA(4,1) ARMA(1,2) ARMA(2,2) ARMA(3,2) ARMA(0,1) ARMA(0,2)

71 Case study 4 (Contd.) Significance of residual mean: e η ( e) = N ρ 1/2 ˆ1/2 e is the estimate of the residual mean ˆρ is the estimate of the residual variance All models pass the test Model Test t η(e) t(α, N 1) ARMA(1,0) E ARMA(2,0) E ARMA(3,0) E ARMA(4,0) E ARMA(5,0) E ARMA(6,0) E ARMA(7,0) E ARMA(8,0) E ARMA(1,1) ARMA(2,1) ARMA(3,1) ARMA(4,1) ARMA(1,2) ARMA(2,2) ARMA(3,2) ARMA(0,1) ARMA(0,2)

72 Case study 5 Sakleshpur Rainfall Data is considered in the case study. Rainfall in mm Time 72

73 Case study 5 (Contd.) Correlogram 73

74 Case study 5 (Contd.) PAC function 74

75 Case study 5 (Contd.) 75

76 Case study 5 (Contd.) Model Likelihood AR(1) AR(2) AR(3) AR(4) AR(5) AR(6) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1) ARMA(3,2)

77 Case study 5 (Contd.) ARMA(5,0) is selected with highest likelihood value The parameters for the selected model are as follows φ 1 = φ 2 = φ 3 = φ 4 = φ 5 = Constant =

78 Case study 5 (Contd.) Significance of residual mean Model η(e) t 0.95 (N ) ARMA(5,0)

79 Case study 5 (Contd.) Significance of periodicities: Periodicity η F 0.95 (2,239 ) 1 st nd rd th th th

80 Case study 5 (Contd.) Whittle s white noise test: Model η F 0.95 (2, N-2 ) ARMA(5,0)

81 Case study 5 (Contd.) Model MSE AR(1) AR(2) AR(3) AR(4) AR(5) AR(6) ARMA(1,1) ARMA(1,2) ARMA(2,1) ARMA(2,2) ARMA(3,1) ARMA(3,2)

82 Case study 5 (Contd.) ARMA(1, 2) is selected with least MSE value for one step forecasting The parameters for the selected model are as follows φ 1 = θ 1 = θ 2 = Constant =

83 Case study 5 (Contd.) Significance of residual mean Model η(e) t 0.95 (N ) ARMA(1, 2)

84 Case study 5 (Contd.) Significance of periodicities: Periodicity η F 0.95 (2,239 ) 1 st nd rd th th th

85 Case study 5 (Contd.) Whittle s white noise test: Model η F 0.95 (2, N-2 ) ARMA(1, 2)

86 Markov Chains Markov Chains: Markov chain is a stochastic process with the property that value of process X t at time t depends on its value at time t-1 and not on the sequence of other values (X t-2, X t-3,. X 0 ) that the process passed through in arriving at X t-1. [,,... ] = [ ] PX X X X PX X t t 1 t 2 0 t t 1 Single step Markov chain 86

87 Markov Chains P X = a X = a t j t 1 i The conditional probability gives the probability at time t will be in state j, given that the process was in state i at time t-1. The conditional probability is independent of the states occupied prior to t-1. For example, if X t-1 is a dry day, what is the probability that X t is a dry day or a wet day. This probability is commonly called as transitional probability 87

88 Markov Chains P X = a X = a = P t t j t 1 i ij P ij t Usually written as indicating the probability of a step from a i to a j at time t. If P ij is independent of time, then the Markov chain is said to be homogeneous. P t ij = t P + τ ij i.e., v t and τ the transitional probabilities remain same across time 88

89 Markov Chains Transition Probability Matrix(TPM): P t+1 t m P P P.. P P P P.. P P m m 31 =.. m.. P P P m1 m2 mm m x m 89

90 Markov Chains m j= 1 P ij = 1 v j Elements in any row of TPM sum to unity (stochastic matrix) TPM can be estimated from observed data by tabulating the number of times the observed data went from state i to j P j (n) is the probability of being in state j in the time step n. 90

91 Markov Chains p j (0) is the probability of being in state j in period t = 0. ( 0) ( 0) ( 0) ( 0) p = p1 p2.. p. Probability m ( n) ( n) ( n) ( n) m 1 m Let p (0) is given and TPM is given vector at time 0 p = p p.. p. Probability ( 1) ( 0) p = p P m vector at time n 91

92 Markov Chains P P P.. P P P P.. P ( 1) ( 0) ( 0) ( 0) p p p p P m m = m 31 And so on ( 0) ( 0) ( 0). P P P m1 m2 mm = p1 P11+ p2 P pm Pm 1. Probability of ( 0) ( 0) ( 0) going to state 1 = p1 P12+ p2 P pm Pm2. Probability of going to state 2 92

93 Markov Chains Therefore In general, ( 1) ( 1) ( 1) ( 1) p p p p = m 1 m ( 2) ( 1) p = p P ( 0) = p P P ( ) = p P 0 2 ( n) ( 0) p = p P n 93

94 Markov Chains As the process advances in time, p j (n) becomes less dependent on p (0) The probability of being in state j after a large number of time steps becomes independent of the initial state of the process. The process reaches a steady state ay very large n p= p P n As the process reach steady state, TPM remains constant 94

95 Example 1 Consider the TPM for a 2-state (state 1 is non-rainfall day and state 2 is rainfall day) first order homogeneous Markov chain as TPM = Obtain the 1. probability of day 1 is non-rainfall day / day 0 is rainfall day 2. probability of day 2 is rainfall day / day 0 is nonrainfall day 3. probability of day 100 is rainfall day / day 0 is nonrainfall day 95

96 Example 1 (contd.) 1. probability of day 1 is non-rainfall day / day 0 is rainfall day TPM No rain rain No rain = rain The probability is probability of day 2 is rainfall day / day 0 is nonrainfall day ( ) ( ) p = p P

97 Example 1 (contd.) ( 2) p = [ ] = [ ] The probability is probability of day 100 is rainfall day / day 0 is nonrainfall day ( n) ( 0) p = p P n 97

98 Example 1 (contd.) 2 P = P P = = P = P P = P = P P = P = P P =

99 Example 1 (contd.) Steady state probability p = [ ] For steady state, n p= p P = [ ] = [ ] 99