A generalized mathematical framework for stochastic simulation and forecast of hydrologic time series

Transcription

1 WATER RESOURCES RESEARCH, VOL. 36, NO. 6, PAGES , JUNE 2000 A generalized mathematical framework for tochatic imulation and forecat of hydrologic time erie Demetri Koutoyianni Department of Water Reource, Faculty of Civil Engineering, National Technical Univerity, Athen Abtract. A generalized framework for ingle-variate and multivariate imulation and forecating problem in tochatic hydrology i propoed. It i appropriate for hort-term or long-term memory procee and preerve the Hurt coefficient even in multivariate procee with a different Hurt coefficient in each location. Simultaneouly, it explicitly preerve the coefficient of kewne of the procee. The propoed framework incorporate hort-memory (autoregreive moving average) and long-memory (fractional Gauian noie) model, conidering them a pecial intance of a parametrically defined generalized autocovariance function, more comprehenive than thoe ued in thee clae of model. The generalized autocovariance function i then implemented in a generalized moving average generating cheme that yield a new time-ymmetric (backward-forward) repreentation, whoe advantage are tudied. Fat algorithm for computation of internal parameter of the generating cheme are developed, appropriate for problem including even thouand of uch parameter. The propoed generating cheme i alo adapted through a generalized methodology to perform in forecat mode, in addition to imulation mode. Finally, a pecific form of the model for problem where the autocorrelation function can be defined only for a certain finite number of lag i alo tudied. Several illutration are included to clarify the feature and the performance of the component of the propoed framework. 1. Introduction Copyright 2000 by the American Geophyical Union. Paper number 2000WR /00/2000WR900044$09.00 Since it initial tep in the 1950, tochatic hydrology, the application of theory of tochatic procee in analyi and modeling of hydrologic procee, ha offered very efficient tool in tackling a variety of water reource problem, including hydrologic deign, hydrologic ytem identification and modeling, hydrologic forecating, and water reource management. An overview of the hitory of tochatic hydrology ha been compiled by Grygier and Stedinger [1990]. We mention a the mot ignificant initial tep of tochatic hydrology the work by Barne [1954] (generation of uncorrelated annual flow at a ite from normal ditribution), Maa et al. [1962] and Thoma and Fiering [1962] (generation of flow correlated in time), and Beard [1965] and Matala [1967] (generation of concurrent flow at everal ite). We mut mention that the foundation of tochatic hydrology followed the ignificant development in mathematic and phyic in the 1940, a well a the development of computer. Specifically, it followed the etablihment of the Monte Carlo method, which wa invented by Stanilaw Ulam in 1946 (initially conceived a a method to etimate probabilitie of olitaire ucce; oon after, it grew to olve neutron diffuion problem), developed by himelf and other great mathematician and phyicit in Lo Alamo (John von Neumann, Nichola Metropoli, Enrico Fermi), and firt implemented on the electronic numerical integrator and computer (ENIAC) [Metropoli, 1989; Eckhardt, 1989]. The claic book on time erie analyi by Box and Jenkin [1970] alo originated from different, more fundamental cientific field. However, it ha ubequently become very important in tochatic hydrology and till remain the foundation of hydrologic tochatic modeling. Box and Jenkin developed a claification cheme for a large family of time erie model. Their claification cheme ditinguihe among autoregreive model of order p (AR( p)), moving average model of order q (MA(q)), and combination of the two, called autoregreive moving average (ARMA( p, q)). However, depite making a large family, all Box-Jenkin model are eentially of hort-memory type; that i, their autocorrelation tructure decreae rapidly with the lag time. Therefore uch model are proven inadequate in tochatic hydrology, if the long-term peritence of hydrologic (and other geophyical) procee i to be modeled. Thi property, dicovered by Hurt [1951], i related to the oberved tendency of annual average treamflow to tay above or below their mean value for long period. Other clae of model uch a fractional Gauian noie (FGN) model [Mandelbrot, 1965; Mandelbrot and Walli, 1969a, b, c], fat fractional Gauian noie model [Mandelbrot, 1971], and broken line model [Ditleven, 1971; Mejia et al., 1972] are more appropriate to reemble long-term peritence [ee alo Bra and Rodriguez- Iturbe, 1985, pp ]. However, model of thi category have everal weak point uch a parameter etimation problem, narrow type of autocorrelation function that they can preerve, and their inability to perform in multivariate problem (apart from the broken line model, ee Bra and Rodriguez-Iturbe [1985, p. 236]). Therefore they have not been implemented in widepread tochatic hydrology package uch a LAST [Lane and Frevert, 1990], SPIGOT [Grygier and Stedinger, 1990], CSUPAC1 [Sala, 1993], and WASIM [McLeod and Hipel, 1978]. Another peculiarity of hydrologic procee i the kewed ditribution function oberved in mot cae. Thi i not o 1519

2 1520 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST common in other cientific field whoe procee are typically Gauian. Therefore attempt have been made to adapt tandard model to enable treatment of kewne [e.g., Matala and Walli, 1976; Todini, 1980; Koutoyianni, 1999a, b]. (The preentation by Koutoyianni [1999b] i available on-line from World Wide Web erver for the National Technical Univerity, Athen, node at pkewne.pdf.) The kewne i mainly caued by the fact that hydrologic variable are nonnegative and ometime have an atom at zero in their probability ditribution. Therefore a ucceful modeling of kewne indirectly contribute to avoiding negative value of imulated variable; however, it doe not eliminate the problem, and ome ad hoc technique (uch a truncation of negative value) are often ued in addition to modeling kewne. The variety of available model, either hort memory or long memory, with different equation, parameter etimation technique, and implementation, create difficultie in the model choice and ue. Beide, the AR( p) or ARMA( p, q) model, which have been more widepread in tochatic hydrology, become more and more complicated, and the parameter to be etimated become more uncertain, a p or q increae (epecially in multivariate problem). Thu in oftware package uch a thoe mentioned above, only AR(0) through AR(2) and ARMA(1, 1) model are available. The reaon for introducing everal model and claifying them into different categorie eem to be not tructural but rather impoed by computational need at the time when they were firt developed. Today, the widepread ue of fat peronal computer allow a different approach to tochatic model. In thi paper, we try to unify all the above-decribed model, both hort memory and long memory, imultaneouly modeling the proce kewne explicitly. The unification i done uing a generalized autocovariance function (ection 2), which depend on a number of parameter, not necearily greater than that typically ued in traditional tochatic hydrology model. Specifically, we eparate the autocovariance function from the mathematical tructure of the generating cheme (or model) that implement thi autocovariance function. Thu the autocovariance function may depend on two or three parameter, but the generating cheme may include a thouand numerical coefficient (referred to a internal parameter), all dependent on (and derived from) thee two or three parameter. The generating cheme ued i of moving average type, which i the implet and mot convenient; in addition to the traditional backward moving average cheme, a new cheme with everal advantage, referred to a ymmetric (backwardforward) moving average model, i introduced (ection 3). New method of etimating the internal parameter of the generating cheme, given the external parameter of the autocorrelation function, are introduced (ection 4); they are very fat even for problem including thouand of internal parameter. The propoed generating cheme can be directly applied for tochatic imulation. In addition, the cheme can perform in forecat mode, a well, through a propoed methodology that make thi poible (ection 5); it i a generalized methodology that can be ued with any type of tochatic model. The framework, initially formulated a a ingle-variate model, i directly extended for multivariate problem (ection 6). A pecific model form for problem where the autocorrelation function i defined only for a certain finite number of lag (e.g., in generation of rainfall increment within a rainfall event) i alo tudied (ection 7). In it preent form the propoed framework i formulated for tationary procee; the poibility of incorporating eaonality in combination with eaonal model i alo mentioned briefly (ection 8, alo including concluion). Several ection of the paper include imple illutration that clarify the feature and the performance of the component of the propoed framework. Additional example on the application of the generalized autocovariance function uing ynthetic and hitorical hydrologic data et are given in Appendix A1. 1 To increae readability, everal mathematical derivation are given eparately in Appendice A2 A4. 2. A Generalized Autocovariance Structure and It Spectral Propertie Annual quantitie related to hydrologic procee uch a rainfall, runoff, evaporation, etc. or ubmonthly quantitie of the ame procee (e.g., fine-cale rainfall depth within a torm) can be modeled a tationary tochatic procee in dicrete time. We conider the tationary tochatic proce X i in dicrete time denoted with i, with autocovariance j : Cov X i, X ij j 0, 1, 2,... (1) The variable X i are not necearily tandardized to have zero mean or unit variance, nor are they necearily Gauian; on the contrary, they can be kewed with coefficient of kewne X : E[(X i X ) 3 ]/ 3/2 0, where X : E[X i ] i the mean and 0 i the variance. The kewne term, which i uually ignored in tochatic proce theory, i eential for tochatic hydrology becaue hydrologic variable very often have kewed ditribution. The parameter X, 0, and X determine in an acceptable approximation the marginal ditribution function of the hydrologic variable of interet, wherea the autocovariance j, if known, determine, again in an acceptable approximation, the tochatic tructure of the proce. However, a j i etimated from ample x 1,, x n with typically mall length n, only few foremot term j can be known with ome acceptable confidence. Uually, thee are determined by the (biaed) etimator [e.g., Bloomfield, 1976, pp. 163, 182; Box and Jenkin, 1970, p. 32; Sala, 1993, p ] ˆ j 1 nj n x i xx ij x, (2) i1 where x i the ample mean. In addition to the fact that the number of available term of the um in (2) decreae linearly with the lag j (which reult in increaing etimation uncertainty), typically, j i a decreaing function of lag j. The combination of thee two fact may lead u to conider that j i zero beyond a certain lag m (i.e., for j m) which may be not true. In other word, the proce X i may be regarded a hort memory, while in reality it could be long memory. However, the large lag autocovariance term j may affect eriouly ome propertie of the proce of interet, and thu a choice of a hort-memory model would be an error a far a thee propertie are conidered. Thi i the cae, for example, if the propertie of interet are the duration of drought or the range of cumulative departure from mean value [e.g., Bra and Rodriguez-Iturbe, 1985, pp ]. 1 Supporting Appendice A1 A4 are available with entire article on microfiche. Order by mail from American Geophyical Union, 2000 Florida Avenue, N. W., Wahington, DC or by phone at ; $2.50. Document 2000WR900044M. Payment mut accompany order.

3 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST 1521 The mot typical tochatic model, belonging to the cla of ARMA( p, q) model [Box and Jenkin, 1970] can be regarded a hort-memory model (although the ARMA(1, 1) model ha been ued to approximate long-term peritence for ome pecial value of it parameter [O Connell, 1974]) a they eentially imply an exponential decay of autocovariance. Specifically, in an ARMA proce, the autocovariance for large lag j converge either to if all term j are poitive or to j a j (3) j a j co 0 1 j (4) if the term alternate in ign, where a,, 0, and 1 are numerical contant (with 0 1). The cae implied by (3) i more common than (4) if the proce X i repreent ome hydrologic quantity like rainfall, runoff, etc. The inability of the ARMA procee to repreent important propertie of hydrologic procee, uch a thoe already mentioned, have led Mandelbrot [1965] to introduce another proce known a fractional Gauian noie (FGN) proce [ee alo Bra and Rodriguez-Iturbe, 1985, p. 217]. Thi i a longmemory proce with autocovariance j 0 1/2j 1 2H j 1 2H j 2H (5) j 1, 2,..., where H i the o-called Hurt coefficient (0.5 H 1). Apart from the firt few term, thi function i very well approximated by j 0 1 1/1 1/2 j 1/, (6) where 1/[2(1 H)] 1, which how that autocovariance i a power function of lag. Notably, (5) i obtained from a continuou time proce (t) with autocovariance Cov [(t), (t )] a 1/ (with contant a 0 (1 1/)(1 1/2)), by dicretizing the proce uing time interval of unit length and taking a X i the average of (t) in the interval [i, i 1]. The autocovariance of both ARMA and FGN procee for large lag can be viewed a pecial cae of a generalized autocovariance tructure (GAS) j 0 1 j 1/, (7) where and are contant. Indeed, for 0, (7) become (uing de l Hopital rule) j 0 exp j, (8) which i identical to (3) if ln. For 1 and large j, (7) yield a very cloe approximation of (6) if 1/1 1/1 1/2 : 0. (9) For other value of or for value of in the interval (0, 1), (7) offer a wide range of feaible autocovariance tructure in between, or even outide of, the ARMA and the FGN tructure, a demontrated in Figure 1a, where we have plotted everal autocovariance function uing different value of but keeping the ame 0, 1, and 2 for all cae. The meaning of the different value of will be dicued later, in the end of thi ection. Here it may uffice to explain that GAS i more comprehenive than the FGN cheme a the latter, with it ingle parameter H, cannot model explicitly even the lag-one autocovariance. It i alo more comprehenive than ARMA cheme a it can explicitly model long-term peritence while yet being parameter parimoniou. If the autocovariance i not everywhere poitive but alternate in ign, (7) can be altered in agreement with (4) to become j 0 1 j 1/ co 0 1 j. (10) In the form of (7) or (10), GAS ha three or five parameter, repectively, one of which i the proce variance 0 (thu the correponding autocorrelation tructure ha two or four parameter, repectively). Although parameter parimony i mot frequently deired in tochatic modeling [e.g., Box and Jenkin, 1970, p. 17], GAS can be directly extended to include a greater number of parameter. Specifically, it can be aumed that the initial m 1 term j ( j 0,, m) have any arbitrary value (e.g., etimated from available record), and then (7) (or (10)) i ued for extrapolating for large j. Eentially, thi introduce m additional model parameter at mot. Thu the total number of independent parameter i m 1if both and are etimated in term of 0,, m,orm 2 if i etimated independently; even in the latter cae, cannot be regarded a an independent parameter becaue continuity at term m demand that 1 m 0 m 1 0, (11) 1 m ln 0 m 0. Parameter etimation can be baed on the empirical autocovariance function. In the mot parameter parimoniou model form (7), parameter 0 i etimated from the ample variance, and parameter and can be etimated by fitting GAS to the empirical autocovariance function. There are everal poibilitie to fit thee parameter: (1) We can chooe to have a good overall fit to a number of autocovariance without preerving exactly any pecific value. (2) Alternatively, we may chooe to preerve exactly the lag-one autocovariance 1 in which cae (11) hold for m 1. Still we have an extra degree of freedom (one more independent parameter), which can be etimated o a to get a good fit of GAS to autocovariance for a certain number of lag higher than 1. (3) Finally, we may chooe to preerve the lag-one and lag-two autocovariance exactly. Cae 3 i the eaiet to apply, a parameter and are directly etimated from (7) in term of 0, 1, and 2. However, cae 1 and 2 are preferable becaue they take into account the autocovariance of higher lag and, thu, the longmemory propertie of the proce. A leat quare method i a direct and imple bai to take into account more than two autocovariance in cae 1 and 2. Note that linearization of (7) by taking logarithm i not applicable a ome empirical autocovariance may be negative. Therefore the application of leat quare require nonlinear optimization, which i a rather imple tak a there are only two parameter in cae 1 ( and ) and one in cae 2 (only becaue of (11)). Reult of application of method 1 on ynthetic and hitorical hydrologic data et are given in Appendix A1. Apparently, parameter etimation i ubject to high uncertainty, and thi i particularly true for, which i related to the long-term peritence of the proce. Therefore, if the record length i mall, it may be a good idea to aume a value of after examining other erie

4 1522 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST Figure 1. (a) Example of autocovariance equence of the propoed generalized type for everal value of the exponent, in comparion with the fractional Gauian noie and ARMA type; (b) correponding power pectra. (Read 1E-04 a ) of nearby gauge, rather than to etimate it directly from the available mall record. Similar parameter etimation trategie can be applied for the richer in parameter form of GAS that were decribed above, in which cae everal value j are preerved. We mut emphaize that not any arbitrary equence j can be a feaible autocovariance equence. Specifically, j i a feaible autocovariance equence if it i poitive definite [Papouli, 1991, pp ]. Thi can be teted in term of the variancecovariance matrix h of the vector of variable [X 1,, X ] T, which ha ize and entrie h ij ij. (12) If h i poitive definite for any, then j i a feaible autocovariance equence. Normally, if the model autocovariance are etimated from data record, poitive definitene i atified. However, it i not uncommon to meet a cae that doe not atify thi condition. The main reaon i the fact that autocovariance of different lag are etimated uing record of different length either due to the etimation algorithm (e.g., uing (2)) or due to miing data. Another reaon i the fact that high-lag autocovariance are very poorly etimated, a explained above. An alternative way to tet that j i a feaible autocovariance equence i provided by the power pectrum of the proce, which hould be poitive everywhere. The power pectrum of the proce i the dicrete Fourier tranform (DFT; alo termed the invere finite Fourier tranform) of the autocovariance equence j [e.g., Papouli, 1991, pp. 118, 333; Bloomfield, 1976, pp ; Debnath, 1995, pp ; Spiegel, 1965, p. 175]; that i, : j1 j co 2j 2 j co 2j. j (13) Becaue j i an even function of j (i.e., j j ), the DFT in (13) i a coine tranform; a uual we have aumed in (13)

5 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST 1523 that the frequency range in [0, 1/2], o that j i determined in term of () by j 0 1/ 2 co 2j d. (14) If autocovariance i given by the generalized relation (7) for all j, then it i eaily hown that the power pectrum i / Re e 2i, 1, 1 (15) where i : 1, Re [ ] denote the real part of a complex number, and ( ) i the Lerch trancendent phi function defined by z, b, a : j0 z j a j b. (16) In the pecific cae that 0, where (8) hold, (15) reduce to 2 0 )] 1 e 2 1 e 2 2e co 2. (17) Thi give a characteritic invere S-haped power pectrum (Figure 1b) that correpond to a hort-memory proce. Numerical invetigation how that for any 1 and 0 (given in (9)), (15) become approximately a power function of the frequency with exponent approaching 1/ 1. (More accurately, the exponent i by an amount maller than 1/ 1, where i 0.03 for 2.5 and decreae almot linearly to zero a approache 1. The exponent become almot equal to 1/ 1if i et equal to [1 0.71(1 1/)(1 1/2)] 0 ; however, in the latter cae the departure of the power pectrum from the power law i greater.) Thi cae indicate a typical long-memory proce, imilar to a FGN proce (ee Figure 1b) where the power function appear a a traight line on the log-log plot). Generally, the power pectrum tend to infinity a tend to zero, regardle of the value of, if 1. For 1 the power pectrum cannot be a power law of the frequency but approache that given by (17) (invere S-haped) a decreae, taking a finite value for 0. If we fix 0 and 1 (the variance and the lag-one autocovariance) at ome certain value, and vary (and accordingly), we oberve that there exit a combination of * 1 and 0 (*) (given by (9)) reulting in a power pectrum *() approximately following a power law. For * the power pectrum exceed *() for low frequencie (that i, it depart from the traight line and become invere J-haped in the log-log plot). The oppoite happen if * (the pectrum tend to the invere S-haped). Thi i demontrated in Figure 1b, where, in addition, 2 ha been alo fixed; the power pectra of Figure 1b are thoe reulting from the autocovariance of Figure 1a. Note that the power pectra of Figure 1b have been calculated numerically from (13) rather than from (15) becaue the three fixed autocovariance 0, 1, and 2 do not allow a ingle intance of (7) to hold for all j. The cae * (traight line on log-log plot) that correpond to the FGN proce ha been met in many hydrological and geophyical erie. The cae 0 (invere S-haped line on log-log plot) that correpond to ARMA procee ha been widely ued in tochatic hydrology. In addition to thee cae the GAS cheme allow for all intermediate value of in the range (0, *), a well a for value * (invere J-haped line on log-log plot, or very fat tail of autocovariance). The cae 0 * implie a long-term peritence weaker than the typical FGN one. The cae * characterize procee with trong long-term peritence but not very trong lag-one correlation coefficient. Both thee cae can be met in hydrologic erie (ee example in Appendix A1). In ection 4.1 we will ee how we can utilize the power pectrum of the proce to determine the parameter of a generalized generating cheme, which will be introduced in the following ection Decription of the Generating Scheme It i well known [Box and Jenkin, 1970, p. 46] that for any autocovariance equence j, X i can be written a the weighted um of an infinite number of independent and identically ditributed (i.i.d.) innovation V i (alo termed auxiliary or noie variable), that i, in the following form, known a (backward) moving average (BMA) form (where we have lightly modified the original notation of Box and Jenkin [1970]): 0 X i a j V ij a 2 V i2 a 1 V i1 a 0 V i, (18) j where a j are numerical coefficient that can be determined from the equence of j. Specifically, coefficient a j are related to j through the equation [Box and Jenkin, 1970, pp. 48, 81] a j a ij i i 0, 1, 2,... (19) j0 Although in theory X i i expreed in term of an infinite number of innovation, in practice it uffice to ue a finite number of them for two reaon: (1) becaue the number of variable to be generated in any imulation problem i alway a finite number and (2) becaue term a j decreae a j 3, o that beyond a certain number j all term can be neglected without ignificant lo of accuracy. We mut clarify that in our perpective the number of nonnegative term 1 i larger, by order of magnitude, than p or q typically ued in ARMA( p, q) model. Alo, the number i totally unrelated to the number of eential parameter m 2 of the autocovariance function, dicued in ection 2, a coefficient a j are internal parameter of the computational cheme. By contrat, the number could be regarded a a large number of the order of magnitude 100 or 1000, depending on the decay of autocovariance, the deired accuracy, and the imulation length. In thi repect, (18) and (19) can be approximated by 0 X i a j V ij a V i a 2 V i2 a 1 V i1 a 0 V i, j (20) i a j a ij i i 0, 1, 2,..., (21) j0 repectively, for a ufficiently large. Extending thi notion, we can write X i a the weighted um

6 1524 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST of both previou and next (theoretically infinite) innovation variable V i, i.e., in the following backward-forward moving average (BFMA) form: X i a j V ij a 1 V i1 a 0 V i a 1 V i1, j (22) where now the coefficient a j are related to j through the almot obviou equation a j a ij i i 0, 1, 2,..., n. (23) j In ection 5 we will ee that the introduction of forward innovation term (i.e., V i1, V i2, etc.) doe not create any inconvenience even if the model i going to be ued a a forecat model. The backward-forward moving average model (22) i more flexible than the typical backward moving average model (18). Indeed, the number of parameter a j in model (22) i double that of model (18) in order to repreent the ame number of autocovariance j. Therefore in model (22) there exit an infinite number of equence a j atifying (23). One of the infinite olution of (23) i that with a j 0 for every j 0, in which cae the model (22) i identical to the model (18). Another intereting pecial cae of (22) i that with a j a j j 1, 2,... (24) For reaon that will be explained below, the latter cae will be adopted a the preferable model throughout thi paper and will be referred to a the ymmetric moving average (SMA) model (although the BMA model will be conidered a well). In thi cae, (22) can be written a X i a j V ij a V i a 1 V i1 a 0 V i a 1 V i1 j a V i, (25) where we have alo retricted the number of innovation variable to a finite number, for the practical reaon already explained above. That i, we have aumed a j 0 for j. The equation relating the coefficient a j to j become now i a j a ij i i 0, 1, 2,... (26) j Given that the internal model parameter a j are 1in total, the model can preerve the firt 1 term of the autocovariance j of the proce X i,ifa j are calculated o that (26) i atified for i 0,,. (In ection 4 we will dicu how thi calculation can be done.) A we have already dicued, the number can be choen o that the deired accuracy can be achieved. The model implie nonzero autocovariance for a number of ubequent time lag. Thu for the ubequent term ( j 1,,2) the autocovariance term are given by i a j a ij i 1,...,2 (27) ji (a conequence of (26) for i ), wherea for even larger lag the autocovariance vanihe off. Apart from the parameter a j that are related to the autocovariance of the proce X i, two more parameter are needed for the generating cheme, which are related to the mean and kewne of the proce. Thee are the mean V : E[V i ] and the coefficient of kewne V : E[(V i V ) 3 ] of the innovation V i (note that by definition, Var [V i ] 1). They are related to the correponding parameter of X i by 3/ 2 V X 0 (28) j a V X, j0 for the BMA model and a 0 2 j a V X, j1 3 a j j0 a a 3/ 2 j V X 0 j0 (29) for the SMA model, which are direct conequence of (20) and (25), repectively. To provide a more practical view of the behavior of the SMA model, alo in comparion with the typical BMA model, we demontrate in Figure 2 two example in graphical form. In the firt example, we have aumed that the proce X i i Markovian with autocovariance (3) and 0 1 and 0.9. In cae of the BMA model with infinite a j term, a theoretical olution of (19) i a j j, (30) a can be eaily verified by ubtituting (30) into (19). If we chooe to preerve the firt 101 autocovariance term j auming that a j 0 for j 100, we can numerically etimate from (21) the firt 101 nonzero term a j (in a manner that will be decribed in ection 4). The numerically etimated a j are depicted in Figure 2a; practically, they equal thoe given by (30), apart from the lat three value, which depart from theoretical value becaue of the effect of etting the high term a j 0 (the departure i clear in Figure 2a). The ame autocovariance j can be alo preerved by the SMA model. The theoretical olution for infinite a j term, and the approximate olution, again uing 101 nonzero a j term, are calculated from (23) and (26), repectively (uing technique that will be decribed in ection 4), and are alo hown in Figure 2a. We oberve that all a j value of the SMA model (apart from a 0 ) are maller than the correponding a j value of the BMA model; for large j near 100, a j of the SMA model become 1 order of magnitude maller than thoe of the BMA model. Apparently, thi contitute a trong advantage of the SMA model over the BMA one: The maller the coefficient a j for large j, the maller i the introduced error due to etting a j 0 for j. In a econd example we have aumed that X i i a FGN proce with autocovariance (5), and 0 1 and H 0.6, which correpond to Thi autocovariance i hown graphically in Figure 2b along with the reulting equence of a j, auming again that the firt 101 term are nonzero. Once more we oberve that the a j equence of the SMA model lie below that of the BMA model. In addition to the approximate olution for 101 nonzero a j term a theoretical olution for infinite a j term, alo hown in Figure 2b, i poible for the

7 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST 1525 Figure 2. Two example of theoretical autocovariance equence and reulting equence of internal parameter a j for BMA and SMA cheme: (a) a Markovian autocovariance equence and (b) a fractional Gauian noie autocovariance equence. The obtained autocovariance equence by either of the BMA or SMA cheme are inditinguihable from the theoretical one. SMA model, a will be decribed in ection 4.1. We will alo ee in ection 4.1 and Appendix A2 that a cloed analytical olution i poible for the SMA model for any autocovariance j but not for the BMA model. Thi certainly contitute a econd advantage of the SMA model over the BMA one. A we have already mentioned above, the SMA model implie a nonzero autocovariance even for lag above the aumed numerical limit, i.e., for j 1uptoj 2, given by (27). On the contrary, the BMA model implie that all autocovariance term above are zero. In Figure 3 we have plotted the reulting autocovariance of the above-decribed Markovian example for lag j up to 200. We oberve that thi tructure may be an accepted approximation of the Markovian tructure for lag (at leat it i better than the zero autocovariance implied by the BMA model). A thi i achieved by no cot at all (no additional parameter are introduced), it can be regarded a an additional advantage of the SMA model over the BMA model. A fourth advantage of the SMA model i related to the preervation of kewne, in cae of kewed variable, which are very common in tochatic hydrology. It i well known [Todini, 1980; Koutoyianni, 1999a] that if the coefficient of kewne of the innovation variable become too high, it i impoible to preerve the kewne of the variable X i. Therefore the model reulting in lower coefficient of kewne of the innovation variable i preferable. In all cae examined, thi wa the SMA model. For intance, in the above-decribed Markovian example the SMA model reulted in V 2.52 X

8 1526 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST Figure 3. Obtained autocovariance tructure from the SMA cheme uing 100 a j term, for lag 0 200; the theoretical autocovariance tructure i that of Figure 2a. wherea, in the BMA model, V 3.27 X (by applying (29) and (28), repectively). 4. Computation of Internal Parameter of the Generating Scheme We will preent two method for computing the equence of term a j given the autocovariance j. The firt method reult in cloed analytical olution of (23) for the cae that (24) hold; thi i applicable to the SMA model for an infinite number of a j term. The econd method i a numerical olution of (21) or (26) that determine a finite number of a j term and i applicable to both the BMA and the SMA model Cloed Solution Denoting a () the DFT of the a j erie and utilizing the convolution equation (23) and the fact that in the SMA model, a j i an even function of j (equation (24)), we can how (ee Appendix A2) that a () i related to the power pectrum of the proce () by a 2. (31) Thi enable the direct calculation of the DFT of the a j erie if the power pectrum of the proce () (or equivalently, the autocovariance j ) i known. Then a j can be calculated by the invere tranform, i.e., a j 0 1/ 2 a co 2j d. (32) Apart from a few pecial cae, the calculation needed to evaluate a j from j can be performed only numerically. However, they are imple and noniterative. In addition, all calculation can be performed uing the fat Fourier tranform (FFT, e.g., Bloomfield [1976, pp ]), thu enabling the building of a fat algorithm. For the BMA model the fact that a j i not an even (or an odd) function of j reult in a complex DFT of a j. Therefore the correponding relation between a () and () become (ee Appendix A2) a 2, (33) where a () i the abolute value of a (). Given that a () i complex, (33) doe not uffice to calculate a () (it give only it amplitude, not it phae). Therefore the method cannot work for the BMA model. In addition, it i hown in Appendix A2 that there doe not exit any other real valued tranformation, different from DFT, that could reult in an equation imilar to (31) to enable a direct calculation of a j for the BMA model. However, the iterative method preented in ection 4.2 can be applied to both the SMA and the BMA model Iterative Solution The equation relating the model internal parameter a j to the autocovariance term j, i.e., (21) and (26) for the BMA and SMA model, repectively, may be written imultaneouly for j 0,, in matrix notation a p, (34) where [a 0,, a ] T, [ 0,, ] T (with the exponent T denoting the tranpoe of a matrix or vector), and p i a matrix with ize ( 1) ( 1) and element p ij 1/2a ji U j i a ij2 U i j 1 for the BMA model and (35) p ij a ji a ij2 U j 2U i j 1 (36) for the SMA model. Here U( x) i the Heaviide unit tep function, with U( x) 1 for x 0 and U( x) 0 for x 0. It can be eaily verified that (35) and (36) (along with (34)) are equivalent to (21) and (26), repectively. Other expreion equivalent to (35) and (36) and impler than them can be alo derived, but (35) and (36) are the mot convenient in ubequent tep. Clearly, each ingle equation of the ytem (34) include econd-order product of unknown term a j. Therefore (34) may have one or more olution in cae of a poitive definite autocovariance or no olution otherwie. Generally, we need to

9 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST 1527 Figure 4. An example of an inconitent j equence approximated with a conitent equence achieved by the SMA cheme uing 100 a j term; the latter are alo plotted, in comparion with the correponding term of the BMA cheme. determine one ingle olution if it exit; otherwie, we may need to find the bet approximation to (34). To accomplih thee in a common manner, we reformulate the parameter etimation problem a a minimization problem, demanding to min f fa 0,...,a : p 2 p 1 0 2, (37) where p 1 i the firt row of p, i a weighting factor, and. denote the Euclidean norm of a vector. The meaning of the firt term of the right-hand ide of (37) become obviou from (34). The econd term denote the quare error in preerving the model variance 0, multiplied by the weighting factor. Although, apparently, the econd term i alo contained in the firt term, it eparate appearance in the objective function enable it eparate treatment. In cae of a feaible autocovariance equence, the minimum of f() will be zero, whatever the value of. However, in cae of an inconitent autocovariance equence the minimum of f() will be a poitive number. In uch a cae the preervation of the variance 0 i more important than that of autocovariance term. Aigning a large value to (e.g., 10 3 ), we force (p 1 0 ) 2 to take a value cloe to zero. Alternatively, could be conidered a a Lagrange multiplier (an extra variable of the objective function (37)), but thi would complicate the olution procedure. The tak of minimization of f() i facilitated by determining it derivative with repect to. After algebraic manipulation it can be hown that d(p)/d 2p (for both BMA and SMA cheme), o that df 4p d T p 4p 1 0 p 1. (38) Clearly, the problem we have to olve i an uncontrained nonlinear optimization problem with analytically determined derivative. Thi can be eaily tackled by typical method of the literature uch a the teepet decent and Fletcher-Reeve conjugate gradient method [e.g., May and Tung, 1996, p. 6.12; Pre et al., 1992, p. 422]. Thee are iterative method, tarting with an initial vector, which, in our cae, can be taken a [0] [ 0, 0, 0,, 0]T, and iteratively improving it until the olution converge. The algorithm ha been proven very quick and efficient in all cae examined, involving problem even with more than 1000 a i parameter. Example of applying the algorithm for conitent autocovariance, Markovian and fractional Gauian, have been already dicued (ection 2 and Figure 2). An example of applying the algorithm to an inconitent autocovariance i hown in Figure 4. The autocovariance of thi example i identical to that of the Markovian example of Figure 2a, apart from the value 2 and 3 that were both et equal to 1 ; thi create a covariance matrix h that i not poitive definite. A hown in Figure 4, the algorithm reulted in a very good approximation of the aumed autocovariance. In comparion with an earlier numerical procedure by Wilon [1969; ee alo Box and Jenkin, 1970, p. 203] for determining the parameter of the BMA proce, the abovedecribed algorithm i more general (it alo cover the SMA cae), fater (it doe not involve matrix inverion, wherea Wilon algorithm doe), and more flexible and efficient (it can provide approximate olution for inconitent autocovariance, wherea Wilon algorithm cannot). 5. Generation Scheme in Forecat Mode Equation (20) and (25) are directly applicable for imulation (unconditional generation) of the proce X i. However, it i quite frequently the cae where ome of the variable X i (pat and preent) are known and we wih either to generate other (future) variable, or to obtain bet prediction of thee (future) variable. A we will ee, both problem can be tackled in a common imple manner, applicable for both the BMA and SMA model. We will aume that the vector coniting of the preent and k pat variable Z : [X 0, X 1,, X k] T i known and it value i z [ x 0, x 1,, x k] T. We wih either to generate any future variable X j for j 0, or to predict it value, under the condition Z z. Thee can be done utilizing the following propoition, whoe proof i given in Appendix A3:

10 1528 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST Propoition. Let X i(i k,, 0, 1, 2, ) be any dicrete time tochatic proce with autocovariance j ( j 0, 1, ) and let Z : [X 0, X 1,, X k ] T. Let alo Z : [X 0, X 1,, X k] T be a vector of tochatic variable independent of X i with mean and autocovariance identical to that of X i. Then, the tochatic proce X i X i i T h 1 Z Z i 1, 2,..., (39) where i T : Cov [X i, Z ] and h : Cov [Z, Z ], ha identical mean and autocovariance with thoe of X i. In addition, the conditional variance of X i, given Z z, i Var X i Z z 0 i T h 1 i (40) and i identical to the leat mean quare prediction error of X i from Z. Note that h i a ymmetric matrix with ize (k 1) (k 1) and element given by (12) wherea i i a vector with ize k 1 and element i j ij1. (41) Alo, note that the propoition i quite general and it can be applied to any type of linear tochatic model (not only to thoe examined in thi paper). Thi propoition enable the following procedure for the forecat mode of the model: (1) Determine the matrix h uing (12) for the given number (k 1) of known (preent and pat) variable and then calculate h 1. (2) Generate a equence of variable X i(i k,, 0, 1, ), uing the adopted model (20) or (25) without any reference to the known variable Z. Form the vector Z and Z and calculate the vector h 1 (Z Z ). (3) For each i 0, determine the vector i from (41) and calculate the final value of the variable X i, conditional on Z, from (39). Equation (40) how that the conditional variance of X i i maller than the unconditional one ( 0 ), a expected. The fact that thi conditional variance i identical to the leat mean quare prediction error of X i from Z enure u that no further reduction i poible by any type of linear prediction model. Thu the combination of model (20) or (25) with the tranformation (39) allow preervation of the tochatic tructure of the proce, whatever thi tructure i, and imultaneouly reduce the conditional variance to it mallet poible value, in the ene that no other linear tochatic model could reduce it further. Notably, the ame generating model (20) or (25) i ued in both mode, imulation and forecat. Theoretically, the procedure can be applied to negative value of i, a well. In thi cae, if k i 0, it i eay to how that (39) reduce to the trivial cae X i X i, a it hould (ee Appendix A3). The above tep are appropriate if the forecat i done in term of conditional imulation. If it mut be done in term of expected value rather than conditionally imulated value, then in tep 2 of the above procedure, X i are et equal to their (unconditional) expected value rather than generated. In thi cae, if confidence limit are needed, they can be calculated in term of the conditional variance given by (40). 6. Multivariate Cae The model tudied in ection 2 5 i a ingle-variate model but can be eaily extended to the multivariate cae. In thi cae the model, apart from the temporal covariance tructure, hould conider and preerve the contemporaneou covariance tructure of everal variable correponding to different location. Let X i [X i 1, X i 2,, X in ] T be the vector of n tochatic variable each correponding to ome location pecified by the index l 1,, n, at a pecific time period i. Let alo g be the variance-covariance matrix of thoe variable with element g lk : Cov X i l, X i k l, k 1, 2,..., n. (42) l We aume that each of the variable X i can be expreed in term of ome auxiliary variable V l i (again with unit variance) by uing either for the BMA model or 0 X l i r X l i r l l a r V ir l l a r V ir (43) (44) for the SMA model. Thee equation are imilar to (20) and (25), repectively. l The auxiliary variable V i can be aumed uncorrelated in time i (i.e., Cov [V l i, V k m ] 0 if i m) but correlated in different location l for the ame time i. Ifc i the variancecovariance matrix of variable V l i, then each of it element c lk : Cov V i l, V i k l, k 1, 2,..., n (45) can be expreed in term of g lk and the erie of a i l and a i k by for the BMA model and lk c lk l k g a r a r r0 lk c lk l k g a r a r r (46) (47) for the SMA model. Thee equation are direct conequence of (43) and (44), repectively. The theoretically anticipated lagged cro-covariance for any lag j 0, 1,, i then Cov X l i, X k ij for the BMA model and Cov X i l, X ij j g lk r0 j k g lk r a r l k a jr l a jr a r r0 k r l k a r a r l k a r a r (48) (49) for the SMA model. Given the variance-covariance matrix c, the vector of variable V i [V i 1, V i 2,, V in ] T can be generated uing the imple multivariate model V i bw i, (50) where W i [W i 1, W i 2,, W in ] T i a vector with innovation variable with unit variance independent both in time i and in

11 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST 1529 Table 1. Theoretical and Empirical Statitic of the Application of Section 6 Theoretical Empirical Location 1 Location 2 Location 1 Location 2 Mean Standard deviation Coefficient of kewne Hurt coefficient Crocorrelation coefficient location l 1,, n and b i a matrix with ize n n uch that bb T c. (51) The methodology for olving (51) for b given c (alo known a taking the quare root of c) will be dicued in ection 7 below. The other parameter needed to completely define model (50) are the vector of mean value W and coefficient of kewne W of W i l. Thee can be calculated in term of the correponding vector V and V of V i l, already known from (28) or (29), by W b 1 V, W b 3 1 V, (52) which are direct conequence of (50). In (52), b (3) i the matrix whoe element are the cube of b, and the exponent 1 denote the invere of a matrix. To illutrate the method, we have applied it to a problem with two location with tatitic given in Table 1. To invetigate the method ability to preerve long-term memory propertie uch a the Hurt coefficient in multiple dimenion, we have aumed the FGN tructure with exponent equal to 1.25 and for location 1 and 2, repectively, correponding to Hurt coefficient 0.6 and 0.7 for location 1 and 2, repectively. We generated a ynthetic record with 10,000 data value uing the SMA cheme with 2000 nonzero a j term, which were evaluated by the cloed olution decribed in ection 4.1. The lat (2000th) term of the erie of a j wa a 0 for location 1 and a 0 for location 2; thee mall value indicate that the error due to neglecting the higher a j term (beyond term 2000) i mall. The required computer time on a modet (300 MHz) Pentium PC wa 10 for the computation of internal parameter (when the fat Fourier tranform wa implemented in the algorithm; otherwie it increaed to 2 min) and another 10 for the generation of the ynthetic record. A hown in Table 1, the preervation of all tatitic wa perfect. In addition, Figure 5 how that the autocorrelation and cro-correlation function, the power pectrum, and the recaled range a a function of record length were very well preerved, a well. 7. Finite Length of Autocorrelation Sequence In ection 1 6 it wa aumed that the autocovariance j i defined for any arbitrarily high lag j. However, there are cae where only a finite number of autocovariance term can be defined. For example, in a tochatic model decribing rainfall increment at time interval within a rainfall event with certain duration d q (where q i an integer), the autocovariance ha no meaning for lag greater than q 1 (ee the application in the end of thi ection). Such cae can be tackled in a different, rather impler, way. An appropriate model for thi cae i X bv, (53) where X [X 1,, X q] T i the vector of variable to be modeled with variance-covariance matrix h given by (12), V [V 1,, V q] T i a vector of innovation with unit variance, and b i a quare matrix of coefficient with ize q q. The main difference from the model of ection 3 5 i that the number of innovation V equal the number q of the modeled variable X (the length of the ynthetic record). In thi cae the ditribution of innovation V cannot be identical. Each one ha different mean and coefficient of kewne, given by V b 1 X, V b 3 1 X, (54) which are direct conequence of (53). The matrix of coefficient b i given by bb T h, (55) which again i a direct conequence of (53). It i reminded that (55) ha an infinite number of olution b if h i poitive definite. Traditionally, two well-known algorithm are ued which reult in two different olution b [ee, e.g., Bra and Rodriguez-Iturbe, 1985, p. 96; Koutoyianni, 1999a]. The firt and impler algorithm, known a triangular or Choleky decompoition, reult in a lower triangular b. The econd, known a ingular value decompoition, reult in a full b uing the eigenvalue and eigenvector of h. A third algorithm ha been propoed by Koutoyianni [1999a] which i baed on an optimization framework and can determine any number of olution, depending on the objective et (for example, the minimization of kewne, or the bet approximation of the covariance matrix, in cae that it i not poitive definite). We can oberve that the lower triangular b i directly aociated with the BMA model dicued in ection 3, but with different number of innovation V i for each X i. Thu, if b i lower triangular, then, apparently, X 1 b 11 V 1, X 2 b 21 V 1 b 22 V 2, etc. Likewie, a ymmetric b i aociated with the SMA model. An iterative method for deriving a ymmetric b can be formulated a a pecial cae of the methodology propoed by Koutoyianni [1999a]. Thi can be baed on the minimization of fb) : bb T h 2, (56) where we have ued the notation bb T h for the norm (more pecifically, we adopt the Euclidean or tandard norm here; ee, e.g., Marlow [1993, p. 59]), a if bb T h were a vector in q 2 pace rather than a matrix. The derivative of f(b) with repect to b are eay to determine (ee Appendix A4). Uing the notation d/db [/b ij ] for the matrix of partial derivative of any calar a with repect to all b ij (thi i an extenion of the notation ued for vector, e.g., Marlow [1993, p. 208]) and conidering that b i ymmetric, we find that dfb db 8e 4e*, (57) where e : (bb T h)b and e* diag (e 11, e 22,, e qq), i.e., a diagonal matrix containing the diagonal element of e.

12 1530 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST Figure 5. Preervation of tatitical propertie by the imulated record of the application of ection 6: (a) autocovariance, (b) power pectra, (c) recaled range and Hurt coefficient, and (d) cro-covariance. A in the imilar cae of ection 4.2, the problem here i an uncontrained nonlinear optimization problem with analytically determined derivative, which can be eaily tackled by typical method uch a the teepet decent and the Fletcher- Reeve conjugate gradient method. A initial olution for the iterative procedure we hould ue a ymmetric one; a good choice i b [0] 0 I, where I i the identity matrix. To illutrate the method and the difference among the

13 KOUTSOYIANNIS: STOCHASTIC SIMULATION AND FORECAST 1531 three different olution dicued, we have conidered a tochatic model of a rainfall event with duration d 20 hour uing a half-hour time reolution, o that the number of variable i q 20/ We denote X i (i 1,, 40) the half-hour rainfall increment and aume that the covariance tructure of X i i a in the caling model of torm hyetograph [Koutoyianni and Foufoula-Georgiou, 1993]; that i, ij Cov X i, X j c 2 c 1 2 j i, q 1/ c 1 2 d 21 /q 2, (58) where c 1, c 2, and are parameter and m, : 1/2m 1 21/ m 1 21/ m 21/ m 0, (59) wherea (0, ) 1. Thi i apparently a long-memory autocorrelation tructure imilar to the FGN tructure. It alway reult in conitent (poitive definite) autocovariance if it i evaluated within the duration d of the event; however, for certain combination of parameter it can reult in inconitent autocovariance value if it i attempted to evaluate it outide of the event (i.e., for lag greater than q 1). For the example preented here we have aumed that the model parameter are c , c , 0.45, and 10 (unit of millimeter and hour). The tatitic of X i, determined from equation given by Koutoyianni and Foufoula-Georgiou [1993], are X 1.14 mm, mm 2, and X 2.88 (the latter i determined auming twoparameter gamma ditribution for X i ). The matrix b i (1600 element). We have calculated all three olution of the matrix b decribed above (triangular, ingular value, and ymmetric), which are hown chematically in Figure 6. We oberve a regular pattern with a trong diagonal and a trong firt column for the triangular olution, a trong firt column and an irregular pattern for other column for the ingular value olution, and a regular pattern with a trong diagonal for the ymmetric olution. An appropriate mean to compare the three olution i provided by the reulting coefficient of kewne of innovation V i, given by (54). Thee are hown in Figure 7. The ingular value olution reulted in coefficient of kewne ranging from 40 to 62, which apparently are computationally intractable at generation. More reaonable are the value of the triangular olution, with a maximum coefficient of kewne equal to 10. The ymmetric olution reulted in the mallet, among the three cae, maximum coefficient of kewne, lightly exceeding 6. Notably, thi value i the mallet poible value among all poible (infinite) b olution of (55) [Koutoyianni, 1999b]. Thi enhance further the already dicued feature of the SMA model, that ymmetric olution reult in maller coefficient of kewne of innovation, a feature quite expedient in tochatic hydrology. The finite length cheme decribed in thi ection can be a preferable alternative even in cae where the autocovariance i defined for any j but the length q of the ynthetic record i very mall. Specifically, the cheme of the preent ection 7 ue q 2 internal parameter. In the cae that the proce exhibit long memory the required number of parameter of the cheme of ection 3 may be greater than q 2, and thu the cheme of ection 7 could be preferable. Figure 6. Comparion of three different olution of parameter matrice b (3-D plot of their element) of the application of ection 7: (a) triangular olution, (b) ingular value olution, and (c) ymmetric olution. 8. Summary, Concluion, and Dicuion The main topic of the propoed framework can be ummarized in the following point: (1) A generalized autocovariance function i introduced which unifie in a imple mathematical expreion both hort-memory (ARMA) and long-memory (FGN) model, conidering them a pecial intance in a parametrically defined continuum, more comprehenive than thee clae of model. (2) A moving average tochatic generation cheme i propoed that can implement the generalized autocovariance function (or any other autocovariance function). In addition to the traditional backward moving average cheme, a new time-ymmetric (backward-forward) moving average cheme i propoed. It i computationally more convenient and alo reult in better treatment of procee with kewed ditribution. (3) Two method of determining the internal parameter of the generating cheme are propoed. The firt i a