STRUCTURAL TIME-SERIES MODELLING

Transcription

1 1: Structural Time-Series Modelling STRUCTURAL TIME-SERIES MODELLING Prajneshu Indian Agricultural Statistics Research Institute, New Delhi Introduction. ARIMA time-series methodology is widely used for modelling time-series data. This methodology can be applied only when either the series under consideration is stationary or it can be made so by differencing, de-trending, or by any other means. Another disadvantage is that this approach is empirical in nature and does not provide any insight into the underlying mechanism. An alternative mechanistic approach, which is quite promising, is the Structural time-series modelling (Harvey, 1996). Here, the basic philosophy is that characteristics of the data dictate the particular type of model to be adopted from the family. Purpose of present lecture is to discuss STM methodology utilised for modelling time-series data in the presence of trend, seasonal and cyclical fluctuations.. Structural Time-Series Modelling. Structural time-series models are formulated in such a way that their components are stochastic, i.e. they are regarded as being driven by random disturbances. The key to handling structural time-series models is the state space form, with the state of the system representing the various unobserved components, such as trend, cyclical or seasonal fluctuations. Once in state space form (SSF), the Kalman filter provides the means of updating the state, as new observations become available. Predictions are made by extrapolating these components into the future. Kalman filter, a smoothing algorithm is used for obtaining the best estimate of the state at any point within the sample. Prediction and smoothing is carried out once the parameters governing the stochastic movements of the state variables have been estimated. Estimation of these parameters, known as hyper-parameters, is based on Kalman filter. This is because the likelihood function can be expressed in terms of one-step-ahead prediction errors, and these prediction errors emerge as a by-product of the filter. Once a model is estimated, its suitability can be assessed using goodness fit statistics. 3. Kalman Filter. It is a set of mathematical equations that provides an efficient computational (recursive) solution of the least squares method. Kalman filter is a recursive procedure for computing optimal estimator of the state at a particular time, based on information available till that time (Meinhold and Singpurwalla, 1983). The filter is very powerful in several aspects: it supports estimation of past, present and even future states, and it can do so even when the precise nature of the modelled system is unknown. 4. Structural Time-Series Models for Trend. A Structural time-series model is set up in terms of its various components, like trend, cyclical fluctuations, and seasonal variations, i.e. Y t = T t + C t + S t + ε t, (1) where Y t is the observed time-series at time t, T t, C t, S t, ε t are the trend, cyclical, seasonal and irregular components. (i) Local Level Model (LLM) 13

2 1: Structural Time-Series Modelling In the absence of seasonal and cyclical components, eq. (1) reduces to Y t = μt + εt, εt ~ N (0, σ ε ), t = 1,,..., T. () When the trend component (μ t ) does not show a steady upward or downward movement, it becomes a permanent component called level. Sometimes, it is assumed to vary according to a random walk, i.e. μ t = μ t 1 + ηt, ηt ~ N (0, σ η ). (3) Eqs. () and (3) together form the LLM. It may be noted that these equations are already in state space form. However, level μ t is not directly observable but can be estimated. Further, forecast values are weighted average of the data points. When σ ε = 0, forecast is just the last observation and when σ η = 0, level is constant and the best forecast is sample mean. Level of time-series vary over time depending on signal to noise ratio q = ση /. Estimation of μ t, conditional on σ and ε σ η, is done recursively using Kalman filter and smoother (Harvey, 1996). The parameters σ ε and σ η are unknown and are treated as hyperparameters. Likelihood function can be evaluated by Kalman filtering via prediction error decomposition (Shumway and Stoffer, 000). Once σε and σ η are known, one-step-ahead prediction of level, i.e. estimator of μ t+1 given Y t = { Y 1, Y,..., Y t }, viz. at+ 1 = E ( μt+ 1 Yt ), (4) σε is evaluated recursively by Kalman filter. Prediction error variance Pt + 1 = Var ( μt+1 Yt ) = Var (at+1) is also obtained recursively. Reduced form of LLM is ARIMA (0, 1, 1) model. (5) (ii) Local Linear Trend Model (LLTM). As described by Harvey (1996), LLTM is given by eq. (10) along with the following two equations: μt = μt 1 + βt 1 + ηt, (6) β t = βt 1 + ξt, t =..., 1, 0,1,..., (7) where η t ~ N (0, σ η ) and ξ t ~ N (0, σ ξ ). It may be mentioned that ηt, ξt and ε t are independent of one another. If σ η = σ ξ = 0, eqs. (6) and (7) collapse to μt = μt 1 + β, t = 1,,..., T, (8) which can equivalently be written as μt = α + β t, t = 1,,..., T, (9) showing that the deterministic linear trend is a limiting case. LLTM is in state space form with state vector α t = ( μ t, βt ). Updating and prediction are σ ε σ carried out using Kalman filter by assuming that, η and σ ξ are known. Otherwise these 133

3 1: Structural Time-Series Modelling can be estimated using maximum likelihood method for state space models (de Jong, 1988). Reduced form of a LLTM is ARIMA (0,, ) model. It has been shown that forecast function of LLTM performs better than that of corresponding ARIMA model. Ravichandran and Muthuraman (006) utilised STM model for trend for modelling and forecasting India s rice production. Ravichandran and Prajneshu (00a) compared the efficiencies of STM model for trend and Bayesian analysis of time-series model while forecasting India s foodgrain production. (iii) Local linear trend model with intervention effect (LLTMI). Intervention analysis is concerned with making inference about effects of known events. These effects are measured by including intervention, or dummy variables in a dynamic model (Harvey and Durbin, 1986). LLTMI is described by the following equations: Y t = μt + λ w t + ε t (10) μt = μt 1 + βt 1 + λ w t + ηt (11) βt = βt 1 + λ w t + ξ t, (1) where w t is intervention variable and λ is its coefficient. The quantity w t depends on the form which intervention is assumed to take. As for LLTM, estimation of state vector αt = ( μt, βt ) for LLTMI is similarly carried out by putting the model in state space form and applying Kalman filter recursively by treating w t as an explanatory variable. Ravichandran and Prajneshu (00b) utilised this model for sunflower yield forecasting. 5. Structural Time-Series Models for Cyclical Fluctuations. In the absence of seasonal components, STM model reduces to Y t = μ t + ψ t + ε t. (13) For modelling cyclical fluctuations, the following three models are discussed: (i) Cycle Plus Noise Model (CNM). Here, the trend μ t is assumed to be constant. Thus, the functional form of CNM (Harvey, 1996) is Y t = μ + ψ t + ε t, t = 1,,..., T. (14) The cyclical fluctuations ψ t can be expressed as a mixture of sine and cosine terms: ψ t = α cos ( λ c t ) + β sin ( λ c t ), (15) where λ c, (α + β ) 1/, tan -1 (β / α) represent respectively the frequency, amplitude, and phase. The cycles in eq. (4) need to be made stochastic by allowing the parameters α and β to evolve over time. Following Harvey (1996), the final form of eq. (4) can be written as ψ t cos λc sin λc ψ t 1 k t = ρ +, (16) * * * t sin λc cos λc t 1 k t where the correlation coefficient ρ [ 0, 1] is a damping factor, and k t & k t * are uncorrelated white-noise disturbance terms. Further, ψ * 0 = α, ψ 0 = β. The new parameters are ψ t, the * current value of the cycle, and ψ t, which appears by construction in order to form ψ t. Eq. (16) is a vector AR (1) process. 134

4 1: Structural Time-Series Modelling Let L denote the lag operator, i.e. L ψt = ψ. t 1 (17) Then eq. (5) can be written as 1 L ρ cos λc L ρ sin λc ψ t k t = * * L ρ sin λc 1 L ρ cos λc t k t Substituting for ψ t in eq. (3) yields. (18) Y t (1 L ρ cos λ ) k + (L sin ) k = + c t ρ λ μ c + εt, t =1,,..., T, (19) (1 L ρ cos λ + L ρ ) where ε t is assumed to be uncorrelated with k t and k t *. c For estimation of parameters, eq. (19) has to be put in state space form (Meinhold and Singpurwalla, 1983) and then Kalman filter, prediction and smoothing (Koopman et al., 000) are applied. In eq. (19), unobservable state, conditional on variances σ ε and, is done recursively using Kalman filter and smoother. In general, these are unknown and are treated as hyperparameters. Likelihood function can be evaluated by Kalman filter via prediction error decomposition (Shumway and Stoffer, 000). Maximizing likelihood function with respect to hyperparameters, using quasi-newton optimization procedure, is referred to as hyperparameter estimation. After estimation of parameters, prediction and smoothing are performed. The reduced form from ARIMA family corresponding to CNM is Constant + ARIMA (,). (ii) Trend Plus Cycle Model (TCM). As described by Harvey (1996), TCM is given by the following equations: Yt = μt + ψt + εt, t =1,,..., T, (0) μt = μt 1 + βt 1 + ηt, (1) βt = βt 1 + ξt, t =..., 1, 0, 1,... () Here ε t, η t and ξ t are the disturbance terms which follow Gaussian distributions with means 0 and variances σ ε, ση and σ, called hyper-parameters of the model. As mentioned in CNM ξ model, estimation of state vector and hyper-parameters is carried out by putting the model in state space form and subsequently Kalman filter is applied with proper initial values. The state space formulation of TCM is as follows: * t σ k The measurement equation is Y t = α + ε And the transition equation is [ ] t t, t = 1,,..., T, (3) 135

5 1: Structural Time-Series Modelling μt 1 1. μt 1 ηt βt 0 1. βt 1 ξt αt = L = L L L L L L + L (4) ψt. ρ cos λc ρ sin λc ψt 1 k t * ρ λ ρ λ * * t. sin t c cos c 1 k t The covariance matrix of the vector of disturbances in eq. (4) is a diagonal matrix with diagonal elements { σ η,,, }. Once the parameters are estimated using prediction error σ ξ σ k σ k* decomposition, Kalman filter, prediction and smoothing can be applied. (iii) Cyclical Trend Model (CTM). Here the cycle is actually incorporated within trend. Thus the model is given by the equations (Harvey, 1996): Y t = μt + εt, t = 1,,..., T, (5) μ t = μ t 1 + ψ t 1 + βt 1 + ηt, (6) βt = β t 1 + ξ t. (7) Estimation of parameters and hyper-parameters is carried out by putting the model in state space form and applying Kalman filter. The essential difference between TCM and CTM is that, in the former, the observation Y t depends on cyclical fluctuations ψ t explicitly whereas, in the latter, it does so on ψ t 1 implicitly through the trend μ t. ARIMA (,, 4) model is the corresponding analogue of both TCM and CTM. Ravichandran et al. (00) utilised the above versions of STM models for cyclical fluctuations in all-india lac forecasting which has prominent cycles since modelling and forecasting using these models found out to be better than traditional ARIMA forecasting model. 6. Structural Time-Series Models for Seasonal Fluctuations. A STM is set up in terms of various components of interest, such as trend (μ t ), cyclical fluctuations (ψ t ), seasonal variations (γ t ) and error terms (ε t ), i.e. Y t = μt + ψt + γ t + ε t. (8) However, if cyclical fluctuations are not present, eq. (19) reduces to Y t = μ t + γ t + ε t. (9) In reality, μ t varies over time and it seems logical that μ t depends on μ t 1. We, therefore, let μt = μt 1 + βt 1 + ηt, (30) βt = βt 1 + ξt, (31) where the error terms ε t, η t, ξ t are assumed to be mutually uncorrelated and follow respectively N ( 0, σ ε ), N ( 0, σ η ), and N ( 0, σ ξ ) distributions. Eqs. (9), (30) and (31) together are called Basic Structural Model (BSM). The seasonal component γ t in eq. (9) can be considered as either Dummy seasonal or Trigonometric seasonal (Harvey, 1996). 136

6 1: Structural Time-Series Modelling 7. Goodness of Fit. Goodness of fit statistics is used for assessing overall model fit. Basic measure of goodness of fit in time-series models is prediction error variance. Comparison of fit between different models is based on Akaike information criterion (AIC). AIC = log L + n, (39) where L is the likelihood function, which is expressed in terms of estimated one-step-ahead prediction errors v ˆt = Y t Ŷ t. Here n is the number of hyperparameters estimated from the t 1 model. Schwartz-Bayesian information criterion (SBC) is also used as a measure of goodness of fit which is given as SBC = log L + n log T, (40) where T is the total number of observations. Lower the values of these statistics, better is the fitted model. 8. Software for STM Modelling. STM models can be fitted to the data using Structural Time-series Analyser, Modeller and Predictor (STAMP) (Koopman et al., 000) or by using SsfPack. (Koopman et al., 1999) software package or by SAS (Statistical Analysis System), Version 9. software packages. 9. Illustrations Problem. Fit the following Structural time-series model: Yt = μ t + εt, t = 1,,...,T, where μt = μ t 1 +β t 1 + ηt and βt = β t 1 + ξt using SAS package to India s total foodgrain production (million tonnes) during the period 1966 to Draw the graph of fitted values along with data points. Finally obtain the forecasts for the years 000, 005 and 010. Solution. SAS Code Data foodgrain; input year Production; cards;

7 1: Structural Time-Series Modelling ; /* Print data*/ proc print data=foodgrain; /* Save result file*/ ods rtf file='resultfoodgrain'; /* Unobserved Components Model*/ proc ucm; model production; irregular; level; slope; forecast outfor=for1; proc print data=for1; /* Prepare data for graph*/ data graphfit; set for1; keep forecast; data graph; 138

8 1: Structural Time-Series Modelling merge foodgrain graphfit; proc print data=graph; title 'Graph of fitted values along with data points'; proc sgplot data=graph; scatter x=year y=production; series x=year y=forecast; xaxis values=(1966 to 1999 by 3); yaxis values=(70 to 10 by 0); quit; ods rtf close; RESULTS: Likelihood Optimization Algorithm Converged in 10 Iterations. Table 1. Trend Information Nam e Estimate Standard Error Level Slope Goodness of fit Statistics : AIC=36.00, BIC=40.31 Table. Forecasts for India s total foodgrain production Year Forecast Standard Error REFERENCES 139

9 1: Structural Time-Series Modelling Box, G.E.P., Jenkins, G.M. and Reinse l, G.C. (1994). Time-Series Analysis: Forecasting and Control, 3rd edition. Prentice Hall, New Jersey. de Jong, P. (1988). The likelihood for a state space model. Biometrika, 75, Harvey, A.C. (1996). Forecasting, Structural Time-Series Models and the Kalman Filter. Cambridge Univ. Press, U.K. Harvey, A.C. and Durbin, J. (1986). The effects of seat belt legislation on British road casualties: A case study in structural time-series modelling. J. Royal Statistl. Soc., A 149, Koopman, S.J., Shephard, N. and Doornik, J.A. (1999). Statistical algorithms for models in state spacae using SsfPack.. Econometrics J.,, Koopman, S.J., Harvey, A.C., Doornik, J.A. and Shephard, N. (000). STAMP: Structural Time- Series Analyser, Modeller and Predictor. Timberlake Consultants Press. Meinhold, R.J. and Singpurwalla, N.D. (1983). Understanding Kalman filter. Amer. Statistician, 37, Ravichandran, S. and Muthuraman, P. (006). Structural time-series modelling and forecasting India s rice production. Agril. Situ. India, Ravichandran, S. and Prajneshu (00a). Dynamical modelling and forecasting of India s foodgrain production. Proc. Nat. Acad. Sci. India, B 7, Ravichandran, S. and Prajneshu (00b). Structural time-series models for describing sunflower yield. Ind. J. Agril. Sci., 7, Ravichandran, S., Prajneshu and Wadhwa, S. (00). Structural time-series models for describing cyclical fluctuations. J. Ind. Soc. Agril. Statist., 55(1), Shumway, R.H. and Stoffer, D.S. (000). Time-Series Analysis and its Applications. Springer Verlag, New York. 140