Analogue Correction Method of Errors by Combining Statistical and Dynamical Methods

Transcription

1 NO.3 REN Hongli and CHOU Jifan 367 Analogue Correction Method of Errors by Combining Statistical and Dynamical Methods REN Hongli 1,2 ( ) and CHOU Jifan 2 ( ) 1 Laboratory for Climate Studies,National Climate Center,China Meteorological Administration,Beijing College of Atmospheric Sciences, Lanzhou University, Lanzhou (Received April 30, 2006) ABSTRACT Based on the atmospheric analogy principle, the inverse problem that the information of historical analogue data is utilized to estimate model errors is put forward and a method of analogue correction of errors (ACE) of model is developed in this paper. The ACE can combine effectively statistical and dynamical methods, and need not change the current numerical prediction models. The new method not only adequately utilizes dynamical achievements but also can reasonably absorb the information of a great many analogues in historical data in order to reduce model errors and improve forecast skill. Furthermore, the ACE may identify specific historical data for the solution of the inverse problem in terms of the particularity of current forecast. The qualitative analyses show that the ACE is theoretically equivalent to the principle of the previous analogue-dynamical model, but need not rebuild the complicated analogue-deviation model, so has better feasibility and operational foreground. Moreover, under the ideal situations, when numerical models or historical analogues are perfect, the forecast of the ACE would transform into the forecast of dynamical or statistical method, respectively. Key words: combination of statistical and dynamical methods, inverse problem, numerical prediction, analogue correction of errors (ACE) 1. Introduction In general, there are two basic predictive methods including the dynamical and statistical ones, both of which have benefits and absences. The dynamical method, based on initial problem of physical principles, does not or not fully utilize historical data. As a contrast, the statistical method can use a lot of information of historical observed data, but does not or not fully utilize physics we have (Chou, 1986). Early in 1958, Gu (1958) put forward the importance and feasibility of introducing past historical data into the numerical prediction. Thereafter, a series of innovative and effective theories and methods were put forward (Chou, 1974; Zheng and Du, 1973; Cao, 1993; Zhang and Chou, 1997; Gu, 1998; Gong et al., 1999; Chen et al., 2003) in order to efficiently combine statistical method with numerical model and fully utilize information of past data to improve dynamical prediction. Many results of numerical experiments have shown considerable predictive skill. Especially, to effectively combine numerical prediction model with subjective experiences of forecasters in analogue prediction, the dynamical prediction field may be assumed as a small disturbance over historical analogue field so that synoptic experiences are introduced to the numerical prediction (Chou, 1979). In terms of this basic principle, some analogue-dynamical models (ADMs) were established for the weather forecasting and seasonal prediction (Qiu and Chou, 1989; Huang and Wang, 1991; Huang et al., 1993). These ADMs have higher accuracy than the traditional analogue prediction, which was documented by prediction experiments (Schuurmans, 1973; Barnett and Preisendorfer, 1978; van den Dool, 1987). It is well known that there inevitably exist errors in the numerical model, thus the ADM principle is just proposed for reducing model errors. However, it is quite difficult technically to directly build the analogue-deviation version of complicated numerical Supported jointly by the National Natural Science Foundation of China under Grant Nos , and Corresponding author:renhl@cma.gov.cn.

2 368 ACTA METEOROLOGICA SINICA VOL.20 model. Actually, we had better base on existing dynamical models if we would fully utilize physical laws. At present, there exist a great many direct approaches for diminishing model errors, but along such route, it is getting more difficult to heighten prediction level. As a contrast, some techniques such as model identification which is regarded as the second kind of inverse problem (Gao and Chou, 1994; Fan and Chou, 1999) can fully utilize plenty of existing real observed data to optimize model parameters and improve model with smaller cost (Qiu and Chou, 1987, 1988, 1990). It needs pointing out that such the inverse problem indicates reimprovement of the numerical model and thus can improve the numerical prediction along with the continual developments of positive problems. Recently, there has already been some innovative works for the related attempt and exploration by using complicated model, and preliminary experiment results have shown considerable validation (Bao et al., 2004; Bao, 2004). Therefore in the present paper, we will base on the atmospheric analogy principle and put forward a new inverse problem that the information of historical analogue data is utilized to estimate current model errors, and then a method of analogue correction of errors (ACE), which can combine effectively statistical and dynamical methods, is further developed. 2. A new inverse problem Because it is focused on how historical data are utilized to reduce model errors efficiently and improve current prediction, the influences of observed data errors on latter theoretical deviation are not considered in this situation in order to highlight primary problems. That is to say, observed data are completely proper. As we have known, numerical prediction is mathematically put forward in terms of the initial problem of partial differential equations. In general, numerical prediction model can be expressed as follows: ψ + L(ψ) = 0, t (1) ψ(r, 0) = G(r), (2) where ψ(r, t) is the model state vector to be predicted, r the vector in the spatial coordinate, t time, and L the differential operator of ψ, which is corresponding to real numerical model and usually nonlinear. When t >0, ψ or its functional P may be obtained by numerically integrating initial values. Similarly, the exact model that real atmosphere satisfies can be written as ψ t + L(ψ) = E(ψ), (3) where E is the error term and stands for the process that actually exists but is not described or exactly described in Eq.(1), and just reflects the errors of real numerical model. Then historical data may be regarded as a series of special solutions ψ or their functional P of Eqs.(3) and (2). At present, there primarily exist two methods for the reduction of numerical model errors. One is directly improving every sections of dynamical model, which has been widely operated internationally. But in these days, there also appear many problems in related studies such as expensive cost, long research period, and slow advancement on predictive level. For the other method, it is the main idea that the unknown term E in Eq.(3) can be approximately estimated when a series of special solutions ψ or P of Eqs.(3) and (2) as well as L are known, that is to say, which has become the model identification problem of confirming unknown sections in equations by utilizing historical observed data (Qiu and Chou, 1987, 1988, 1990). Such a problem belongs to the second kind of inverse problem (Gao and Chou, 1994; Fan and Chou, 1999), which can improve model and heighten predictive level with smaller cost. Thus, it will just be investigated how current model errors are estimated by effectively using information from historical data, which may also be regarded as a model identification problem. In fact, for a special numerical model L, one can retrieve and obtain several very different or even completely different estimations of model errors by using observed data in different times. Which estimation on earth should be chosen to improve model and prediction? What ought to be based on for such a choice? To answer these problems, the particularity of prediction objective should be considered in order to

3 NO.3 REN Hongli and CHOU Jifan 369 use distinguishingly existing historical data. In terms of analogy principle, estimated model errors E by utilizing analogical atmospheric evolving data would be closer to each other, which may be easily understood from practical experiences. For instance, the same model often makes very similar faults for forecasting analogical weather process. Set ψ as the historical analogical state (called as analogue reference state, or reference state for short, denoted as RS) of ψ, which satisfies ψ t + L( ψ) = E( ψ), (4) ψ(r, 0) = G(r). (5) Because ψ is quite close to ψ, we can make the firstorder Taylor expansion of E(ψ) in terms of ψ around ψ as follows: E(ψ) E( ψ) + (ψ ψ)d ψ, where D represents the sum of the partial differentials of E with respect to every component of ψ. As we can see, when D ψ is bounded and ψ ψ is small enough, let ψ = ψ + ψ and it is not difficult to obtain E( ψ + ψ ) E( ψ) << E(ψ). At this time, provided that the error term E(ψ) on the right-hand side of Eq.(3) is directly estimated with the error term E( ψ) on the right-hand side of Eq.(4), we can obtain ψ t + L(ψ) = ψ t + L( ψ). (6) In Eq.(6), the small term E( ψ + ψ ) E( ψ) has been actually omitted. Because the RSs are from historical data, the first term on the right-hand side of Eq.(6) is known and the second term may be calculated by numerical model (Bao, 2004). Thus, Eq.(6) can be considered to append an analogue-correction term of errors into Eq.(1) in order to be closer to Eq.(3), which is evidently more exact than that by omitting E(ψ) on the right the side of Eq.(3) compared with Eq.(1). Here, we call Eq.(6) the analoguecorrection equation of errors (denoted by ACEE), which still reflects the original model by only adding a correction term so as to reduce model errors. It can be seen that by estimating E(ψ) in current prediction from E( ψ) on the basis of existing model and historical analogue RSs, we can regard E( ψ) as correction term in order to reduce model errors and improve current prediction. In such a sense, the problem improving the dynamical prediction in numerical model by utilizing historical data has been actually transformed into the inverse problem estimating current unknown model errors by using known historical analogical information. 3. Equivalent analogue-dynamical model In general, there exist many approaches that can reduce model errors and improve prediction by utilizing information of historical data, such as the systematic correction method of model errors, besides the model identification technique on the basis of solving inverse problem (Qiu and Chou, 1987, 1988, 1990). The former can be employed to improve prediction by directly utilizing historical hindcast errors to correct current prediction, but the latter is used for improving model based on historical data. Under the prerequisite without regard to observed errors, the errorcorrection sections corresponding to the former may just be produced by the model error term retrieved from the latter. Consequently, in the sense of improving prediction, the two approaches are consistent with each other, which is very important for the applications of Eq.(6) to complicated model. As we know, the influences of model errors on prediction are alterative with time and dependent on flow pattern. Thus, it may not be the most effective that all of data or in-the-near-past data are used for the identification of model errors or systematic error correction. Generally, the more data are used in linear system, the better effect for solving above-mentioned inverse problem will be. However, as the atmosphere is a nonlinear system, the effect for solving corresponding inverse problem will not be dependent on quantity of data but on particularity of data. According to the aforementioned analyses, it can be known that for the Chou Jifan. The analogue-dynamical model of monthly mean circulation. personal communication. April 25, 2003.

4 370 ACTA METEOROLOGICA SINICA VOL.20 estimation problem of model errors, we should utilize specific historical data for the solution of the inverse problem in terms of the particularity of current forecast objective, which is also the same for systematic error correction. Furthermore, in terms of the derivation of Eq.(6), it is not difficult to understand that only the information generated from historical analogue states can be effectively used for correcting model errors of current forecast, but non-analogue states could provide fault correcting information. Here, the particularity of forecast objective refers to current forecast and the particularity of used data refers to historical analogue states. Actually, many previous studies on ADMs (Qiu and Chou, 1987; Huang and Wang, 1991; Huang et al., 1993) have already integrally taken the prediction objective and the particularity of used data into account. ψ can be divided into the analogue reference state (RS) ψ and the analogue disturbance state (or disturbance state for short, denoted as DS) ψ, namely ψ = ψ + ψ, where ψ is selected from historical data in terms of the similarities between the RSs and the current initial state G(r). Substitute ψ = ψ +ψ and ψ into Eq.(1) respectively, and subtract the latter from the former, and we can obtain the analogue-deviation equation (ADE) as follows: ψ t + L( ψ + ψ ) L( ψ) = 0. (7) By comparing Eq.(7) with Eq.(6), it can be found that they are completely equivalent, but have quite different physical sense. The former expresses the dynamical equation satisfied by analogue deviation and needs to rebuild a deviated ADM, which is very difficult for complicated operational model. However, the latter only corrects model errors in current prediction by utilizing historical analogical information and need not change the original numerical model. Thus, although both of them are more precise than Eq.(1), according to practical point of view, the prediction method on the basis of the ACEE expressed as Eq.(6) has undoubtedly more advantages. Similar to the approach establishing ADM in terms of ADE, theoretically, equivalent ADM can be built by ACEE. Such the equivalent ADM is actually made up of the original dynamical prediction model and the diagnostic model which is introduced to the estimation of the analogue correction terms of model errors. 4. Analogue correction method of errors In early days, the identification problem on model error term E is usually restricted by retrieval technique and calculating condition, and past historical data are not also objectively distinguished for use in terms of different prediction objectives. According to the ACEE in Eq.(6), we can theoretically get an equivalent ADM that need not change original model. Here, one needs only to add a diagnostic model which is used to estimate the analogue correction terms of model errors, and can directly utilize information of past data to improve results of dynamical prediction. However, in practical operations, the time integration schemes such as semi-implicit or implicit iterations are employed in structure-complicated operational model, which makes it difficult to determine explicit operator L so as not to directly calculate error correction term. In fact, we may indirectly overcome such a difficulty by referring to the above qualitative analysis to the relationship between identification of model errors and systematic error correction. 4.1 Prediction of prediction errors For current initial value ψ 0, prediction of Eq.(1) is denoted as P (ψ 0 ) and that of Eq.(3) is denoted as ˇP (ψ 0 ) (namely observed value, unknown). Then, due to taking no account of observed errors, Ê(ψ 0 ) = ˇP (ψ 0 ) P (ψ 0 ) may be regarded as the contribution of model error term E to prediction results. If one estimates Ê(ψ 0) aforehand, prediction will be expressed as P (ψ 0 )+Ê(ψ 0). At this time, the above problem has come down to solving Ê(ψ 0), which just is error correction for dynamical prediction P (ψ 0 ). Consequently, for the improvement of numerical prediction, one can introduce a new idea which need not directly solve equations but need prognose prediction errors of dynamical model indirectly. Similarly, for initial value ψ j from historical observed data, prediction of Eq.(1) is denoted as P (ψ j ) and that of Eq.(3) is denoted as ˇP (ψj ) (namely

5 NO.3 REN Hongli and CHOU Jifan 371 historical observed value, known). Here, j = 1, 2,..., n, where maximal n may be taken as the total sample size of all historical data. Therefore, one can obtain n known prediction errors Ê(ψ j) = P (ψ j ) P (ψ j ), and estimate Ê(ψ 0) by arithmetically averaging these errors so as to get systematic parts of model errors. On the other hand, according to forenamed analyses, in order to select suitable observed data to present prediction, we may estimate Ê(ψ 0) by directly utilizing the prediction errors Ê( ψ j ) from historical observed states ψ j that are analogical to current initial values ψ 0. Here, j = 1, 2,..., m, where maximal m is the number of selected historical analogues. 4.2 Error estimation methods After obtaining m prediction errors Ê( ψ j ) from historical analogical data, how will the current prediction error Ê(ψ 0) be estimated? This is not a problem with certain answer and needs still exploring from lots of practices. Here, we may express error estimation problem in mathematical formula as follows: Ê(ψ 0 ) = C(Ê( ψ 1 ), Ê( ψ 2 ),..., Ê( ψ m )), (8) where ψ 0 is the initial value of current prediction, ψ j is the initial values of selected historical analogues (j = 1, 2,..., m), and m is the number of selected analogues. The parameter C stands for the algorithms estimating current error from errors based on historical analogues, the specific form of which can be decided in terms of given situation. As a choice, we define C as a linear form in order to introduce a simple linear estimation method (SLEM), and thus Eq.(8) becomes Ê(ψ 0 ) = m a j Ê( ψ j ), (9) j=1 j=1 where a j stands for the jth normalized weighted coefficient and is defined as a j = b j / m b j. Here b j is a coefficient to be determined associated with the degrees of the similarities between different historical analogues and current initial state. Referred to many common techniques and methods in meteorological data analysis, some discussions associated with error estimation method to Eq.(9) are given below. (1) Consider the simplest way: One can directly estimate current unknown prediction error by using prediction error based on historical analogue and utilize this known error Ê(ψ 0) to correct current prediction, by which m prediction results may easily be obtained, respectively. Here, we have two further selections. One is to regard prediction error corresponding to the most analogical historical states as ultimate prediction, and the other is to operate a simple ensemble by arithmetically averaging m ultimate prediction results, by which prediction level can be further heightened (Bao et al., 2004). (2) Consider the way of equal-weighted averaged estimation: Take b j =1, Ê(ψ 0) will be the arithmetical average of m errors generated by analogue prediction. (3) Consider the way of unequal-weighted averaged estimation: As a more general situation, one can determine b j in terms of the differences of the similarities between different historical analogues and the current initial state, and take Ê(ψ 0) as the weighted average of m errors generated by analogue prediction. Certainly, there may still exist other methods for selecting weight in error estimation. Thus, we develop a method of analogue correction of errors (ACE), compared with other methods by using past data (Chou, 1974; Zheng and Du, 1973; Cao, 1993; Zhang and Chou, 1997; Gu, 1998; Gong et al., 1999), which can utilize not only in-the-near-past data but also lots of historical data. Based on the present model, the ACE can estimate current prediction errors by introducing information of prediction errors from historical analogue data in order to improve prediction results. The advantage of this new method is just to utilize current model without changing this original model. The above-mentioned methods (1) and (2) have been applied to monthly forecasting with complex operational model, and experiment results showed better predictive skill than control run (Bao et al., 2004; Bao, 2004), which documents that the ACE has preferable perspective for practical application. 5. Qualitative analyses for ideal situations Since the ACE can efficiently combine the advantages of statistical and dynamical methods, when either of them is accurate completely, how will the

6 372 ACTA METEOROLOGICA SINICA VOL.20 forecast result of the ACE change? Now from the view of qualitative analyses, two ideal situations will be discussed (Bao, 2004). (1) Dynamical model is perfect. At this moment, Eqs.(1) and (3) are equivalent. That is to say, omitted term E disappears and no model error exists. For any analogical or non-analogical RSs ψ, the correction term on the right side of Eq.(6) becomes zero. Therefore the forecast of the ACE transforms into that of perfect dynamical model as Eq.(1). (2) Historical analogue is perfect. Initial value ψ(t 0 ) of current forecast at time t 0 is equal to its RS ψ(t ). If statistical analogy method is used to predict, current forecast at time t 0 +δt is directly taken as historical ψ(t + δt). If the ACE is used, whether numerical model is accurate or not, by using the same model and initial values ψ(t 0 ) and ψ(t ), current forecast obtained on the left side of Eq.(6) must be the same as historical observed value ψ(t +δt) on the right side after integrated period of δt. At this moment, the forecast of the ACE transforms into that of statistical analogy method. It follows from both of the above analyses that under the ideal situations, when numerical model is perfect, the forecast of the ACE would transform into the forecast of dynamical method, whereas when historical analogues are perfect, the forecast of the ACE would transform into the forecast of statistical method. Thus it can be seen that the ACE, as a new prediction method by combining statistical and dynamical methods on the basis of numerical model, not only utilizes dynamical achievements dependent on numerical prediction model, but also can adequately absorb the information of a great many historical data dependent on statistical method and synoptic prediction experiences. 6. Summary In order to not only utilize dynamical achievements adequately, but also efficiently extract the information of a great many historical data in the past of tens of years, based on the previous studies, the inverse problem that the information of historical analogue data is used to estimate model errors is put forward and a method of ACE that can effectively combine statistical and dynamical methods together is developed in the present paper. Furthermore, the ACE need not change the current numerical prediction models and may identify specific historical data for the solution of the inverse problem in terms of the particularity of current forecast. The qualitative analyses show that the ACE is theoretically equivalent to the previous analoguedynamical model principle, but need not rebuild the complicated analogue-deviation model, thus has better feasibility and operational foreground. In terms of the discussions on the relationship between the identification problem of model errors and systematic error correction, a new idea of prognosing prediction errors in complicated operational model is raised. Here, the above-mentioned inverse problem has transformed into the estimation problem of current unknown errors by utilizing known prediction errors from historical analogues, and some methods on error estimation are further proposed in brief. Moreover, under the ideal situations, when numerical model is perfect, the forecast of the ACE would transform into the forecast of dynamical method, whereas when historical analogues are perfect, the forecast of the ACE would transform into the forecast of statistical method. Certainly, because observed errors are omitted in the ACE that is proposed as the second kind of inverse problem, what impact will have on the new method? Also, there still exist many problems that need to be further studied such as suitable error estimation methods, and so on. Acknowledgments. The authors thank Drs. Zhang Peiqun and Bao Ming very much for their valuable comments and help on the present paper. REFERENCES Bao Ming, 2004: The experiment of monthly mean circulation prediction using the analogy-dynamical model (Dissertation). Dept. Atmos. Sci. of Nanjing University. (in Chinese) Bao Ming, Ni Yunqi, and Chou Jifan, 2004: The experi- Chou Jifan. How to make short-term climate prediction more accurate? there exists a shortcut! Invited report in CMA/NMC, February 16, 2004.

7 NO.3 REN Hongli and CHOU Jifan 373 ment of monthly mean circulation prediction using the analogy-dynamical model. Chinese Sci. Bul., 49 (12), Barnett, T. P., and R. W. Preisendorfer, 1978: Multifield analog prediction of short-term climate fluctuations using a climate state vector. J. Atmos. Sci., 35 (10), Cao Hongxing, 1993: Self-memorization equation in atmospheric motion. Science in China (SeriesB), 36 (7), Chen Bomin, Ji Liren, Yang Peicai, et al., 2003: An approach to improving the dynamical extended-range (monthly) prediction. Chinese Sci. Bul., 48(7), Chou Ge-Fen, 1974: A problem of using past data in numerical weather forecasting. Scientia Sinica, 17 (6), Chou Jifan, 1979: Some problems of long-range numerical weather prediction. Collection of middle and long-range hydrometeorological prediction. China Water Power Press, Beijing, (in Chinese) Chou Jifan, 1986: Why to combine both statistical and dynamical methods together? also discuss how to combine. Plateau Meteorology, 5(4), (in Chinese) Fan Xingang and Chou Jifan, 1999: Methods and experiments of numerical prediction raised as inverse problem. Part I: Three kinds of inverse problems and numerical solutions. Chinese J. Atmos. Sci., 23(5), (in Chinese) Gao Jidong and Chou Jifan, 1994: Two kinds of inverse problems in NWP and a numerical method ideal field experiment. Acta Meteor. Sinica, 52(2), (in Chinese) Gong Jiandong, Li Weijing, and Chou Jifan, 1999: Forming proper ensemble forecast initial members with four dimensional variational data assimilation method. Chinese Sci. Bul., 44(10), (in Chinese) Gu Xiangqian, 1998: A spectral model based on atmospheric self-memorization principle. Chinese Sci. Bul., 43(20), Gu Zhenchao, 1958: The use of past data in numerical weather forecast. Acta Meteorologica Sinica, 29(3), (in Chinese) Huang Jianping and Wang Shaowu, 1992: The experiment of seasonal prediction using the analogydynamical model. Science in China (Ser.B), 35(2), Huang Jianping, Yi Yuhong, Wang Shaowu, et al., 1993: An analogue-dynamical long-range numerical weather prediction system incorporating historical evolution. Quart. J. Roy. Meteor. Soc., 119, Qiu Chongjian and Chou Jifan, 1987: A new approach to improve the numerical weather prediction. Science in China (Series B), 17(8), Qiu Chongjian and Chou Jifan, 1988: A perturbation method of model recognition of numerical weather prediction. Chinese J. Atmos. Sci., 12(3), (in Chinese) Qiu Chongjian and Chou Jifan, 1989: An analoguedynamical method of weather forecasting. Chinese J. Atmos. Sci., 13(1), (in Chinese) Qiu Chongjian and Chou Jifan, 1990: The method of optimizing parameterization in numerical prediction model. Science in China (Series B), 20(2), Schuurmans, C. J. E., 1973: A 4-year experiment in long-range weather forecasting using circulation analogues. Meteor. Rundsch, 26, 2-4. van den Dool, H. M., 1987: A bias in skill in forecasts based on analogues and antilogues. J. Appl. Meteor., 26(9), Zhang Peiqun and Chou Jifan, 1997: A method improving monthly extended range forecasting. Plateau Meteorology, 16(4), (in Chinese) Zheng Qinglin and Du Xingyuan, 1973: A new model of numerical weather prediction by using multi-time observed data. Scientia Sinica, 16(2), (in Chinese)