INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY. Lecture -12 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

Transcription

1 INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY Lecture -12 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.

2 Summary of the previous lecture Data Extension & Forecasting Moving average Double moving average Data Generation Uncorrelated Data Data Generation Serially Correlated Data First order Markov Model 2

3 Data Generation Serially Correlated Data First order stationary Markov model Or Thomas Fiering model (Stationary) ( ) 2 1 X = µ + ρ X µ + t σ ρ j+ 1 x 1 j x j+ 1 x 1 Standard normal deviate Stationary w.r.t mean, variance and lag-one correlation Known sample estimates of µ x, σ x, ρ 1 Assume X 1 (= µ x ) Generate values X 2, X 3, X 4, X 5 3

4 Data Generation Serially Correlated Data First order Markov model with non-stationarity: First order stationary Markov model assumes that the process is stationary in mean, variance and lagone auto correlation. The model is generalized to account for nonstationarity (mainly due to seasonality/periodicity) in hydrologic data. A main application of this generalised model is in generating the monthly stream flows with pronounced seasonality. Periodicity may affect not only the mean, but all the moments of data including the serial correlations. 4

5 Data Generation Serially Correlated Data ( ) 2 1 X = µ + ρ X µ + t σ ρ j+ 1 x 1 j x j+ 1 x 1 StaAonary First order Markov Model First order Markov model with non-stationarity, for stream flow generation: σ X X t ( ) j+ 1 2 i, j+ 1 = µ j+ 1 + ρ j ij µ j + i, j+ 1σ j+ 1 1 ρj σ j ρ j is serial correlation between flows of j th month and j+1 th month. t i, j+1 N(0, 1) 5

6 Example-1 The monthly stream flow (in cumec) for a river is available for 29 years (12 years data is given here) SL. NO. YEAR JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

7 Example-1 (contd.) Time series of monthly stream flow for 29 years

8 Example-1 (contd.) S.No. Month Mean Stdev. Lag-1 correlation 1 JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

9 Assume X 1 = µ 1 = ; σ 1 = 52.24, ρ 1 = µ 2 = 474.5, σ 2 = , σ 2 X 1,2 = ( ) 2 µ 2 + ρ 1 X1,1 µ 1 + t1,2σ2 1 ρ1 σ = = Example-1 (contd.) ( ) *

10 Example-1 (contd.) X 1,2 =521.67, µ 2 = 474.5; σ 2 = , ρ 2 = µ 3 = , σ 3 = , ρ 3 = X 1,3 = ( ) * =

11 Example-1 (contd.) X 1,3 =474.64, µ 3 = ; σ 3 = , ρ 3 = µ 4 = , σ 4 = 77.65, X 1,4 = ( ) * = X 1,12 X 2,1 = σ µ + ρ µ + σ ρ ( ) 2 X t , , σ12 11

12 Generated: Example-1 (contd.) S.No. Month Mean Stdev. Lag-1 correlation 1 JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

13 Example-1 (contd.) Mean Observed Generated JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY 13

14 Example-1 (contd.) 160 Standard Deviation Observed Generated JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY 14

15 Example-1 (contd.) Lag-1 correlation Observed Generated JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY 15

16 Data Generation Serially Correlated Data σ y Y Y t ( ) j+ 1 2, + 1 = µ + ρ µ + 1, 1σ 1 ρ j+ j j + j+ 1 j i j y y ij y i j y y σ y j Where Y i,j+1 = ln (X i, j+1 ) µ y, σ, j y ρ j yj refer to the mean, standard deviation and lag one correlation of logarithms of original data 16

17 Example-2 The logarithms of stream flow (in cumec) of example-1are constructed SL. NO. YEAR JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

18 Example-2 (contd.) S.No. Month Mean Stdev. Lag-1 correlation 1 JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

19 Generated: Example-2 (contd.) S.No. Month Mean Stdev. Lag-1 correlation 1 JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

20 Example-2 (contd.) 7 6 Mean Observed Generated JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY 20

21 Example-2 (contd.) 3 Standard Deviation Observed Generated JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY 21

22 Example-2 (contd.) 1.2 Lag-1 Correlation Observed Generated JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY

23 Example-3 (Generation of 10-day flows to Sardar Sarovar Reservoir) Time series Flow in Mcum Time (10- day period) 23

24 Example-3(contd.) Mean (MCum) Historical data Generated data Jul-I Jul-II Jul-III Aug-I Aug-II Aug-III Sep-I Sep-II Sep-III Oct-I Oct-II Oct-III Nov-I Nov-II Nov-III Dec-I Dec-II Dec-III Jan-I Jan-II Jan-III Feb Ten daily period Mean Flows 24

25 Example-3(contd.) Standard deviation (MCum) Historical data Generated data Jul-I Jul-II Jul-III Aug-I Aug-II Aug-III Sep-I Sep-II Sep-III Oct-I Oct-II Oct-III Nov-I Nov-II Nov-III Dec-I Dec-II Dec-III Jan-I Jan-II Jan-III Feb-I Ten daily period Standard Deviation 25

26 Example-3(contd.) Correlation coefficient Jul-I Jul-II Jul-III Aug-I Aug-II Aug-III Sep-I Sep-II Sep-III Oct-I Oct-II Oct-III Nov-I Nov-II Nov-III Dec-I Dec-II Dec-III Jan-I Jan-II Jan-III Feb-I Ten daily period Lag one correlation Historical data Generated data 26

27 Issues to be addressed in modeling flows Is it necessary to model peak flows? Time during the year when the peak occurs important? Volume of flow important? Duration of flow to be considered (Daily, weekly, monthly etc.) Dependence of the flow from one time period to another? Is time series of flows stationary? Is there evidence of trends or jumps? Quality and quantity of data available Ref: C.T.Haan, 1995, Page no

28 FREQUENCY DOMAIN ANALYSIS 28

29 Frequency Domain Analysis Auto correlation function or correlogram is used for analyzing the time series. r k Time domain analysis X t = d t + ε t k Correlogram Periodicities in data can be determined by analyzing the time series in frequency domain. 29

30 Frequency Domain Analysis Spectral analysis or the frequency domain analysis: the time series is represented in the frequency domain instead of the time domain The observed time series is a random sample of a process over time and is made up of oscillations of all possible frequencies. Spectral analysis is used to identify the periodicities in the data. 30

31 Frequency Domain Analysis Wave length Amplitude n odd n 1 n, 2 2 Periodicity n even ( ) ( ) X = α + α cos 2π f t + β sin 2π f t + ε t 0 k k k k t k= 1 t = 1, 2,. N 31

32 Frequency Domain Analysis f k = k N ; k th harmonic of the fundamental frequency (1/N) N is the no. of observations Periodicity (P): P = 1 f k 32

33 Frequency Domain Analysis α α 0 = x n 2 = x cos 2 ( π f t) k t k N t= 1 k = 1, 2,. M β n 2 = x sin ( 2π f t) k = 1, 2,. M k t k N t= 1 M is maximum lag ( typically considered up to 0.25N) The above equations for α k and β k are valid up to k=n/2 33

34 Frequency Domain Analysis When N is odd, the expressions are true until k = N 2 1 α β N/2 N /2 n 1 = N = 0 t= 1 ( 1) t x t 34

35 Frequency Domain Analysis A variance spectrum divides the variance into no. of intervals or bands of frequency. Spectral density (I k ) is the amount of variance per interval of frequency. N 2 2 I( k) = k k 2 α + β Angular frequency k = 1, 2,. M ω = k 2π k N k = 1, 2,. M 35

36 Frequency Domain Analysis ω = k 2π P A plot of ω k vs I(k) is called spectrum I(k) Peak corresponds to periodicity ω k Prominent spikes indicate periodicity 36

37 Obtain ω k and I(k) for k=1 t X t cos 2 Example-2 ( π ft) sin ( 2π ft) X cos( 2π f t) X sin ( 2π f t) k Σ k t k t k 37

38 Example-2 (contd.) f k k = N 1 = = α = 2 n n x cos( 2π f t) β = x sin ( 2π f t) k t k N t= 1 2 = = ( ) 2 k t k N t= 1 2 = 10 = ( ) 38

39 Example-2 (contd.) N 2 2 I( k) = k k 2 α + β = = ( ) ( ) 2 2 ω k 2π k = N 2 π 1 = 10 =

40 Frequency Domain Analysis Spectral density is also called as line spectrum The line spectrum thus transforms the information from time domain to the frequency domain While the correlogram indicate merely the presence of periodicities in the data The spectral analysis helps indentify the significant periodicities themselves 40

41 Frequency Domain Analysis Spectral density is an inconsistent estimate The plot is not a smooth function The smoothened spectrum is called as power spectrum Power spectrum is a consistent estimate of spectral density 41

42 Frequency Domain Analysis Power spectrum Fourier cosine transform of auto covariance function. n 1 2 I k c 0 jcj fk j j= 1 ( ) = λ cos( 2π ) c j = Auto covariance function λ j = lag window (or smoothing window) Different ways of estimating λ j 42

43 Frequency Domain Analysis Tukey window λ j 1 2π 2 M = 1+ 2cos ' M = Maximum lag ( 0.25N) I(k) Smoothen diagram ω k 43

44 Frequency Domain Analysis Information content is extracted from spectrum. For a completely random series (e.g., uniformly distributed random numbers), the spectral density function is constant termed as white noise White noise indicates that no frequency interval contains any more variance than any other frequency interval. (auto correlation function ρ k = 0, for k 0) I(k) ω k 44

45 Frequency Domain Analysis The steps for analyzing the data are as follows Plot the time series x t ρ k t Plot the correlogram Plot the spectrum I(k) k ω k 45

46 Frequency Domain Analysis The spectrum shows prominent spikes (which represent the periodicities inherent in the data) The period corresponding to any value of ω k may be computed by 2π/ ω k. A rough approximation can be made in neglecting the no. of spikes from the spectrum. To test the significance, the periodicities (which are approximated to be significant) are removed from the original series to get a new series {Z t }, where Z t = X t Y t, 46

47 Frequency Domain Analysis ( ) ˆ ( ) ˆ ( ) ˆ ( )... + ˆ α cos( ω t) + ˆ β sin ( ω t) Y = µ + ˆ α cos ωt + β sin ωt + α cos ω t + β sin ω t + t d d d d where d is no. of periodicities removed (which are assumed to be significant ) The spectrum of new series Z t is plotted and the spikes are observed. A wrong conclusion may be made that these spikes are significant. However they need to be analyzed for their statistical significance 47

48 Frequency Domain Analysis Statistical significance of the periodicities: The periodicities are tested for significance by defining a statistic as follows (Kashyap and Rao 1976) 2 γ ( N 2) I = 4 ˆ ρ 1 Where γ 2 = α 2 + β 2 and N 1 ˆ ρ ˆ 1 = ˆcos α ω βsin ω N t= 1 { x ( ) ( )} t kt kt Ref: Kashyap R L and Ramachandra Rao A Dynamic stochastic models from empirical data, Academic press, New York,

49 Frequency Domain Analysis The periodicity corresponding to ω k is significant at level α only if I F ( 2, N 2) Where F denotes F distribution This test examines the periodicity at a time and should be carried out on a series from which all periodicities (previously found significant) are removed. 49

50 Frequency Domain Analysis A necessary condition in stochastic models is that the series being modeled must be free from any significant periodicities. One way of removing the periodicities from the time series is to simply transform the series into a standardized one. One method of standardizing the series {X t } is by expressing {X t } as the new series {Z t } where, Z t = ( X X ) t S i i 50

51 Frequency Domain Analysis X i is the estimate of mean S i is the estimate of the standard deviation The series has zero mean and unit variance. The series without periodicities is then obtained for which a stochastic (e.g., ARMA Auto Regressive Moving Average) model is created. 51

52 Example 2 Monthly Stream flow (ft 2 /sec) Statistics( ) for Missouri River near Wolf Point MT in Montana is selected for the study. The time series is plotted as follows 35,000 30,000 25,000 months 20,000 15,000 10,000 5, mean 250 flow

53 Example 2 (contd.) Correlogram is plotted Correlation coefficient Lag 53

54 Example 2 (contd.) Spectrum is plotted 1.2E+09 1E I(k) E+08 W(k) 54

55 Frequency Domain Analysis Peaks represent the periodicities inherent in the data. The peaks correspond to w(k) = with a periodicity of 218 months, w(k) = 1.03 with a periodicity of 6 months, w(k) = with a periodicity of 3 months, w(k) = with a periodicity of 2 months and w(k) = with a periodicity of 1.5 months and so on. The periodicities are tested for significance to avoid wrong conclusions. 55

56 Frequency Domain Analysis 218 months and 6 months periodicities are significant and 3 months, 2 months and 1.5 months periodicities are insignificant 56

57 Frequency Domain Analysis Time series of standardized data. 57

58 Frequency Domain Analysis Correlogram of standardized data. Correlation coefficient Lag 58

59 Frequency Domain Analysis Spectrum of standardized data. 59