5 Transfer function modelling

Transcription

1 MSc Further Time Series Analysis 5 Transfer function modelling 5.1 The model Consider the construction of a model for a time series (Y t ) whose values are influenced by the earlier values of a series (X t ). Thus the process (Y t ) is dynamically related to the process (X t ). We may think of (X t ) as the input to a system and of (Y t ) as the output, or of (X t ) as a process of explanatory variables and of (Y t ) as a process of dependent variables. In general, we shall model (Y t ) as a linearly filtered version of (X t ), where the filter used is one-sided, i.e., causal, and also includes a constant term µ. Thus, at the heart of the model, we have a relationship of the form Y t = µ + v j X t j, which represents the systematic dynamics of the model. The effect of the input process takes time to work through to the output process. Furthermore, there may be a timedelay before the input starts to influence the output, i.e., a positive integer k such that v j = 0 for 0 j k 1. (In comparison with Section 3 on linear filters, note the reversal of the roles of the processes (X t ) and (Y t ) as input and output.) In practice, this model needs to be developed further, because the relationship between (X t ) and (Y t ) will not be exact but will be subject to disturbance. We introduce a disturbance or noise term U t to arrive at the following equation for what is known as the transfer function model or a distributed lag model. Y t = µ + v j X t j + U t ( < t < ). (1) The disturbance process (U t ) in Equation (1) is unobservable and is not necessarily a white noise process. We make the following two assumptions: 1. (U t ) is a zero-mean stationary process. 2. (U t ) is uncorrelated with the input process (X t ). If we further assume that (X t ) is a stationary process then it follows that the output process (Y t ) is also a stationary process. If the data being modelled is non-stationary then it may be differenced to reduce it to stationarity before fitting the model of Equation (1). In such cases it may well be appropriate, as is the case for univariate models, to assume that the differenced data, both input and output, have zero mean, which implies that the constant µ in the model equation (1) is taken to be zero. 1

2 Example Consider a manufacturer who decides on the amount of a certain product that he will start to produce at time t, basing his decision upon the predicted selling price. Suppose that it takes k time periods from start to completion of the product. Let Y t denote the quantity of the product ready for supply at time t and X t the market price at time t. If the manufacturer uses simple exponential smoothing then the predicted price ˆx t any number of steps ahead at time t may be written in the form ˆx t = (1 α) α j X t j. Assuming a simple linear relationship between planned production and predicted price, together with the added disturbance term, we may write Equivalently, writing δ(1 α) = β, Y t+k = µ + δˆx t + U t+k = µ + δ(1 α) α j X t j + U t+k. Y t = µ + β α j X t k j + U t = µ + β α j k X t j + U t. (2) j=k Note the presence of a time-delay k. Alternatively, using the lag operator, we may write Y t = µ + βl k (αl) j X t + U t = µ + βlk 1 αl X t + U t. (3) The processes (X t ) and (Y t ) as described will not be stationary in general, but they may be differenced to transform them to stationarity. The transformed series will still satisfy essentially the same Equations (2) and (3) with µ = 0. Returning to the general case, we assume that the disturbance process (U t ) is an ARMA process with infinite moving average representation U t = ψ(l)ɛ t, where (ɛ t ) is a white noise process with variance σ 2 process (X t ). Equation (1) may then be written as and uncorrelated with the input Y t = µ + v(l)x t + ψ(l)ɛ t, (4) where v(z) is the generating function of the coefficients of the filter. In the present setting v(z) is also referred to as the transfer function of the filter. The first two terms on the 2

3 right hand side of Equation (4) represent the systematic dynamics of the model and the third term the disturbance dynamics. We may write the ARMA model for (U t ) more explicitly in the form φ(l)u t = θ(l)ɛ t. Thus φ(z) is the autoregressive characteristic polynomial and θ(z) is the moving average characteristic polynomial for the disturbance process. We assume that the transfer function v(z) may be expressed as a rational function, a ratio of polynomials, v(z) = ω(z)zk δ(z), (5) where k is the time-delay, the denominator (autoregressive) polynomial δ is given by δ(z) = 1 δ 1 z δ 2 z 2... δ p z p, for some p, and the numerator (moving average) polynomial ω by ω(z) = ω 0 ω 1 z ω 2 z 2... ω q z q, for some q. The corresponding recursive filter is δ(l)y t = ω(l)l k X t. Equation (4) becomes Y t = µ + ω(l)lk δ(l) X t + θ(l) φ(l) ɛ t. (6) The polynomial ω(z) has ω 0 1 in general, because a multiplicative constant has been absorbed into it. The minus sign in front of the subsequent ω i reflects the SAS usage. To have a well-defined model, we assume that all the roots of the characteristic equations δ(z) = 0 and φ(z) = 0 lie outside the unit circle in the complex plane. The model of Equation (1)/(4)/(6) may be rewritten as δ(l)y t = µ + ω(l)l k X t + U t, where µ = δ(1)µ and U t = δ(l)u t, to exhibit explicitly a recursive, autoregressive aspect of the model for (Y t ). Assuming that the processes (X t ) and (Y t ) are stationary, let µ X and µ Y denote their respective means. Taking expectations in Equation (1), µ Y = µ + v j µ X. (7) The quantity v j is sometimes referred to as the total multiplier the change in µ Y per unit change in µ X, i.e., the long-term effect on (Y t ) of a unit change in µ X. Note that the total multiplier may also be written as v(1) ω(1)/δ(1). 3

4 5.2 The cross-correlation function and model identification Taking (X t ) to be stationary, we assume temporarily that (X t ) has zero mean, which does not alter the second-order moments of the model but simplifies the notation in deriving the results of this sub-section. Recall that in Equation (1) (U t ) is also zero-mean stationary and that (X t ) and (U t ) are uncorrelated with each other. For any τ, multiplying through by X t τ and taking expectations, we obtain γ 21 τ = v j γ 11 τ j, (8) where γτ 21 is the cross-covariance, γτ 21 = E(Y t X t τ ), and γτ 11 is the autocovariance, γτ 11 = E(X t X t τ ). Equation (8) simplifies in the special case when (X t ) is a white noise process, in which case γτ 21 = v τ γ0 11, which we may rewrite as v τ = ρ 21 τ γ22 0 γ0 11 ρ 21 τ. (9) The result of Equation (9) shows that, if the input process is white noise, we have a straightforward method of estimating the transfer function from the sample crosscorrelation function and the sample variances of the input and output processes: ˆv τ = r 21 τ c22 0 c 11 0 r 21 τ. (10) Now, in general, (X t ) is not a white noise process but is some other ARMA process. Hence there exists some filter with generating function w(z) such that w(l)x t = η t, where (η t ) is a white noise process, uncorrelated with (U t ). Applying this filter to Equation (1) we obtain Yt = µ + v j η t j + Ut, (11) where Yt = w(l)y t, µ = w(1)µ and Ut = w(l)u t. Equation (11) is similar in form to the model Equation (1), with the same filter (v j ), but with the input process white noise. Hence the results of Equations (9) and (10) may be applied to the filtered processes (Yt ) and (η t ) to estimate (v j ). In attempting to find an appropriate transfer function model, a standard approach involves first filtering the input and output processes, using the same filter for both, so as to convert the input process to white noise. Such a procedure is commonly known as prewhitening. It presupposes that the processes (X t ) and (Y t ) have, if necessary, already been transformed to stationarity by differencing. By examining the cross-correlation function of the pre-whitened process data, we may be able to identify a suitable transfer function model. 4

5 5.3 Example: sales Using the example introduced in Section 4.2, recall that it is the first differences of the series ind and sales that appear to be stationary. The main reason for investigating the series would appear to be to predict sales from the leading indicator. Hence we shall attempt to construct a transfer function model with first differences of the sales as the output (Y t ) and the first differences of the leading indicator as the input (X t ). The following SAS program fits an appropriate model, chosen from among models of the general form of Equation (6). proc arima data=indsales; identify var=ind(1) nlag=20; estimate q=1 noint; identify var=sales(1) crosscor=ind(1) nlag=20; estimate q=1 input=(3 $ / (1) ind) noint; forecast lead=5 out=results; run; In fact, this program would have to be developed iteratively, step by step. The first identify statement specifies what is going to be the input variable, here the first differences of the variable ind, and produces the autocorrelation and other functions. The autocorrelation function output on page 7 suggests that we should fit an MA(1) model to the differences, i.e., an ARIMA(0,1,1) model to ind. The first estimate statement fits this model, and the noint option indicates that a zero mean is being assumed. The output on page 8 exhibits the fitted model, and the p-values of the portmanteau statistics show that the model fits well. The fitted model for the input process is used 1. for pre-whitening the input and output variables before calculation of their crosscorrelation function and 2. for calculating forecast values of the input variable which are in turn used in the calculation of forecast values of the output variable. After the input process has been modelled, the second identify statement, with the crosscor option, produces (i) the autocorrelation and other functions for what is going to be the output variable, the first differences of sales, and (ii) the cross-correlation function of the first differences of sales and ind, automatically pre-whitened using the model fitted to ind by the previous estimate statement. Examination of the crosscorrelation function on page 10 indicates that there is a time-delay of 3 units. Thereafter, the ccf appears to die away geometrically. This suggests a transfer function of the form v(z) = ω 0z 3 1 δ 1 z. We are fortunate in having a clear-cut structure to the cross-correlation function here! Note that this cross-correlation function differs from the one in Section 4.2, where the variables had not been pre-whitened. 5

6 As part of the process of model identification, we also have to specify a model for the disturbance process (U t ). We might just try fitting the transfer function model, assuming a few simple models for the disturbance process, to find the simplest one that works. Another, more systematic approach involves first finding a rough estimate ˆv of the transfer function, using Equation (10). An estimated disturbance process is then given by û t = y t ˆv(L)x t. At least in simple cases, we may readily calculate the values of this estimated disturbance process and fit an ARMA model to them. In the present case, using Equation (10), first estimates of the parameters of our proposed form of transfer function are given by ˆω 0 = ˆv 3 = = 4.71 and ˆδ 1 = ˆv 4 = r21 4 = ˆv 3 r = From analysis of the estimated disturbance process or by some trial and error, it turns out that an MA(1) model for (U t ) is appropriate here. The second estimate statement in the SAS program fits our chosen transfer function model to the data with the first difference of sales as the output variable, as specified in the previous identify statement. The q=1 option specifies that the disturbance process is to be modelled as an MA(1) process. The input option is used to specify the input variable and the form of the rational transfer function of Equation (5). In general, the input option takes the form input = ( k $ ( numerator lags ) / ( denominator lags ) x ) where k is the time-delay (Shift in the terminology of the SAS output), numerator lags specifies the numerator polynomial, denominator lags specifies the denominator polynomial, and x represents the input variable. In the present case, the time-delay is 3. The term / (1) after the dollar sign specifies that the numerator polynomial is a constant and that the denominator polynomial is of order 1, i.e., 1 δ 1 z. The input variable will be the first difference of ind. The noint option specifies that there is to be no constant term in the transfer function model. The SAS output on pages 11 and 12 gives the fitted model as Y t = L X t 3 + ɛ t ɛ t 1. Apart from one value that is significant at the 5% level, the p-values of the diagnostic statistics on page 11 indicate that the fitted model is satisfactory in that 1. the autocorrelations of the residuals are consistent with being from a white noise process and 2. the cross-correlations are consistent with the residuals, and hence the disturbance process, being uncorrelated with the input process. Finally, the forecast statement in the SAS program produces forecasts of sales for the next five time-points. We might note that the forecast values are fairly similar to the ones obtained at the end of Section 4 using the VAR(5) model, and would be even more so if constant terms had not been included in the VAR(5) model. 6

7 The ARIMA Procedure Name of Variable = ind Period(s) of Differencing 1 Mean of Working Series Standard Deviation Number of Observations 149 Observation(s) eliminated by differencing 1... Autocorrelations Lag Covariance Correlation Std Error ******************** ********* ** * *** ** ** ** ** * ** **** ** * * * ** ** "." marks two standard errors 7

8 Conditional Least Squares Estimation Standard Approx Parameter Estimate Error t Value Pr > t Lag MA1, < Variance Estimate Std Error Estimate AIC SBC Number of Residuals 149 * AIC and SBC do not include log determinant. Autocorrelation Check of Residuals To Chi- Pr > Lag Square DF ChiSq Autocorrelations Model for variable ind Period(s) of Differencing 1 No mean term in this model. Moving Average Factors Factor 1: B**(1) 8

9 Name of Variable = sales Period(s) of Differencing 1 Mean of Working Series Standard Deviation Number of Observations 149 Observation(s) eliminated by differencing 1... Autocorrelations Lag Covariance Correlation Std Error ******************** ****** ****** ***** ***** *** *** * *** ** * * * * * "." marks two standard errors Variable ind has been differenced. Correlation of sales and ind Period(s) of Differencing 1 Number of Observations 149 Observation(s) eliminated by differencing 1 Variance of transformed series sales Variance of transformed series ind Both series have been prewhitened. 9

10 Crosscorrelations Lag Covariance Correlation * * * * * * * * * * * * ** * ** ************** ********* ******* ***** ***** **** *** ** *** * ** * * * "." marks two standard errors... Both variables have been prewhitened by the following filter: Prewhitening Filter Moving Average Factors Factor 1: B**(1) 10

11 Conditional Least Squares Estimation Standard Approx Parameter Estimate Error t Value Pr > t Lag Variable Shift MA1, sales 0 NUM < ind 3 DEN1, < ind 3 Variance Estimate Std Error Estimate AIC SBC Number of Residuals 145 * AIC and SBC do not include log determinant. Correlations of Parameter Estimates Variable sales ind ind Parameter MA1,1 NUM1 DEN1,1 sales MA1, ind NUM ind DEN1, Autocorrelation Check of Residuals To Chi- Pr > Lag Square DF ChiSq Autocorrelations Crosscorrelation Check of Residuals with Input ind To Chi- Pr > Lag Square DF ChiSq Crosscorrelations

12 Model for variable sales Period(s) of Differencing 1 No mean term in this model. Moving Average Factors Factor 1: B**(1) Input Number 1 Input Variable ind Shift 3 Period(s) of Differencing 1 Overall Regression Factor Denominator Factors Factor 1: B**(1) Forecasts for variable sales Obs Forecast Std Error 95% Confidence Limits