DYNAMIC VS STATIC AUTOREGRESSIVE MODELS FOR FORECASTING TIME SERIES

Transcription

1 DYNAMIC VS STATIC AUTOREGRESSIVE MODELS FOR FORECASTING TIME SERIES Chris Xie Polytechnic Institute New York University (NYU), NY Phone: June, 008 Electronic copy available at:

2 ABSTRACT I build an innovative Dynamic Autoregressive Model (DAR) in forecasting time series, and make comparison with a Static Autoregressive Model (SAR). DAR model requires re-evaluating optimal orders and coefficients at each period, while SAR models simply treats them as constants. The optimal length of the base (historical data for building autoregressive models) has been also investigated. Results show that on average DAR models outperform SAR models by about % up to double digits percent. With increase of the length of the base, adjusted R-Squares for both models are diminishingly increasing, and errors (in percentage) differences are vanishing. Key Words: Dynamic Autoregressive Model (DAR); Static Autoregressive Model (SAR); Base Data (BD) also called the base; Optimal Number of Orders (p) JEF Classifiers: C3, C53 Electronic copy available at:

3 I. INTRODUCTION The autoregressive model is one of powerful tools to forecast time series. Autoregressive models differ from standard linear regression models, because they do not regress on independent variables, but on a subset of the dependent variables (i.e., its lagged values). The p th order autoregressive model can be mathematically expressed as follows: Here, Y β βy β Y β Y ε t = 0 + t + t p t p + t Y t is the dependent variable at the time period t; Yt i is the dependent variable at the time period t-i (i =,,, p) b i is the coefficients (I = 0,,,, p); x t is the residual or random error; p is the number of orders or autoregression rank; Supposed that we know Y, Y,, Y p,,y p+m, then we have following linear equations: Y = β + βy + β Y + β Y β Y + ε p+ 0 p p 3 p p p+ Y = β + βy + β Y + β Y β Y + ε p+ 0 p+ p 3 p p p+ Y = β + βy + β Y + β Y β Y + ε : p+ 3 0 p+ p+ 3 p p 3 p+ 3 Y = β + βy + β Y + β Y β Y + ε p+ m 0 p+ m p+ m 3 p+ m 3 p m p+ m.. () Let Y Y Y Y p+ p+ = p Y p+ m m, X Yp Yp Yp... Y Yp+ Yp Yp... Y Y Y Y... Y = p+ p+ p Y Y Y... Y p+ m p+ m p+ m 3 m m ( p + ), β β0 β = β... β m m ε ε ε ε p+ p+ = p ε p+ m m Then we have Y = Xβ + ε. To minimize sum of squared errors ( the optimal solutions for β, which could be expressed as follows: ε i ), we can obtained 3

4 T T β = ( X X) ( X Y).. () So given X, Y, we could easily calculate coefficients (betas) for this p-order autoregressive model. Now we define the base data (BD) as follows: BD Yp+ Yp Yp Yp... Y Yp+ Yp+ Yp Yp... Y Y Y Y Y... Y = p+ 3 p+ p+ p Y Y Y Y... Y p+ m p+ m p+ m p+ m 3 m m ( p + ) on which the coefficients (betas) will be calculated, and m, the number of rows in BD is called as the length of the base data (BD). Therefore, at the time period T, as we know Y i (i =,,, T), we could estimate T Y +, as Y = β0 + βy + βy β Y. Based on this linear regressive model, other key T + T T p T p figures such as adjusted R-square can also be calculated. II DYNAMIC AUTOREGRESSIVE MODEL (DAR) Dynamic Autoregressive Model (DAR) is an autoregressive model with dynamically re-evaluating all betas (coefficients) and the number of lags (p-order) with respect to a rolling base for forecasting asset prices in order to achieve the best result by autoregressive models. Comparing to a standard autoregressive model (SAR), which treats all coefficients and the number of lags (p-order) initially generated by the initial base data as constants, DAR model not only treats all coefficients and the number of lags (p) as variables (time-subscribed), but also has to choose the optimal length (m) of base data (BD) to minimize the error terms such as the sum of errors-squared. Given all historical figures (daily closed prices) before December 3, 007 known for all stocks of DJI, in forecasting their daily closed prices during January to May 30, 008, our investigations show that DAR models outperform SAR models significantly on average at lower number (less than 350) of the length of the base, while with increasing the length of the base, both models have almost similar result (see details in the section of Dynamic Autoregressive Model vs Static Autoregressive Model). The detailed processes or steps are elaborated as follows. Given the asset prices (for example, the daily closed price) for a certain period of time as Y, Y,, Y T known and assuming the optimal length (m) of the base data (BD) already known, we try to forecast the asset price Y T+, Y T+,, Y T+i (at the period T+, T+, T+i). 4

5 Forecast the asset price Y T+ at the time period T+ Step : Calculate initial p (order) value by using the given known figures (Y, Y,, Y T ) Keep adding additional lags until the adjusted R stops increasing, or increase the number of lags (p) until Akaike Information Criterion (AIC) reaches the minimum ε i p value. Here AIC can be expressed as AIC = log + n n Step : Determine the initial base data (BD) based on given known figures (Y, Y,, Y T ), since the p (order) value and m (the length of base) are known The initial base data (BD) is set as follows: Y Y Y Y... Y Y Y Y Y... Y B Y Y Y Y Y T m+ T m T m T m T m p + T m+ T m+ T m T m T m p + = T m T m T m T m T m p Y Y Y Y... Y T T T T 3 T p m ( p + ) Step 3: Build an autoregressive model base on the above initial base data (B ) Treat the st column as the dependent variable and all other p columns as independent variables, and the dependent variable can be expressed in form of a linear combination of all p dependent variables, or that Y = β + β Y + β Y + β Y β Y + ε T m+ (,0) (,) T m (,) T m (,3) T m (, p ) T m p + (,) Y = β + β Y + β Y + β Y β Y + ε T m+ (,0) (,) T m+ (,) T m (,3) T m (, p ) T m p + (,) Y = β + β Y + β Y + β : β Y + ε T m+ 3 (,0) (,) T m+ (,) T m+ (, 3) T m (, p ) T m p + 3 (,3) Y = β + β Y + β Y + β Y β Y + ε T (,0) (,) T (,) T (,3) T 3 (, p ) T p (, m) Y m To minimize sum of squared errors ( ε ), we can obtained the optimal solutions for b (, i ) i= by either using some linear programming packages or using the formula () stated in the INTRODUCTION. Step 4: Forecast Y T+ Y T+ can be estimated as follows: 5

6 Y = β + β Y + β Y + β Y β Y T + (,0) (,) T (,) T (,3) T (, p ) T p + Forecast the asset price Y T+ at the time period T+ Repeat the above steps except for some modifications Step : Step : Calculate p (order) value using the given known figures (Y, Y 3,, Y T,Y T+ ) by adding Y T+ as it becomes known and eliminating Y as it deemed as obsolete. Determine the base (B ) based on given known figures (Y, Y 3,, Y T,Y T+ ), since the p (order) value and m (the length of base) are known The base (B ) is set as follows: Y Y Y Y... Y Y Y Y Y... Y B Y Y Y Y Y T m+ T m+ T m T m T m p + T m+ 3 T m+ T m+ T m T m p + 3 = T m T m T m T m T m p Y Y Y Y... Y T+ T T T T p + m ( p + ) Step 3: Build an autoregressive model base on the above base data (B ) Similarly treat the st column as the dependent variable and all other p columns as independent variables, and the dependent variable can be expressed in form of a linear combination of all p dependent variables. To minimize sum of squared errors, we can obtained the optimal solutions for b. Step 4: Forecast Y T+ Y T+ can be estimated as follows: Y = β + β Y + β Y + β Y β Y T + (,0) (,) T+ (,) T (,3) T (, p ) T p + Keeping go this way to forecast the asset price Y T+i at the time period T+i Repeat the above steps except for some modifications Step : Calculate p i (order) value using the given known figures (Y i, Y i+,, Y T,Y T+, Y T+i- ) by adding Y T+i- as it becomes known and eliminating Y i- as it deemed as obsolete. Step : Determine the base (B i ) based on given known figures (Y i, Y i+,, Y T,Y T+, Y T+i- ), since the p i (order) value and m (the length of base) are known 6

7 The base (B i ) is set as follows: Y Y Y Y... Y Y Y Y Y... Y B = Y Y Y Y... Y Y Y Y Y... Y T m+ i T m+ i T m+ i T m+ i 3 T m+ i p T m+ i+ T m+ i T m+ i T m+ i T m+ i p + i T m+ i+ T m+ i+ T m+ i T m+ i T m+ i p + T+ i T+ i T+ i 3 T+ i 4 T+ i pi m ( p i + ) i i i Step 3: Build an autoregressive model base on the above base data (B i ) Again treat the st column as the dependent variable and all other p i columns as independent variables, and the dependent variable can be expressed in form of a linear combination of all p i dependent variables. To minimize sum of squared errors, we can obtained the optimal solutions for b i. Step 4: Forecast Y T+i Y T+i can be estimated as follows: Y = β + β Y + β Y + β Y β Y T + i (,0) i (,) i T + i (,) i T + i (,3) i T + i 3 (, i p ) T + i p i i From the above processes, at each period we re-evaluate p (the number of lags) and roll the base to keep it updated, except for, which we will discuss in the later section. The rationales behind DAR model are that the most recently asset prices are higher valuable as inputs for our model as we deem that asset prices have short memories, and that p value (the number of lags) should be dynamically changed as to make the best fitting of the model. Next we applying DAR model to make forecasts for DJI index and S&P500 index. Assuming that it is just at the end of 007, we want to predict the DJI index and S&P 500 index for the period of January to May 30, 008 by DAR model (the result shown in Figure for DJI Index and Figure for S&P 500 Index) The results are summarized as follows: DJI Index by DAR Model (m, the length of the base, set to 40) Avg Adj R-Sq Avg p(i) Max p(i) Min p(i) Avg Abs Error (%) Std Dev (Avg Abs Error (%)) % S&P 500 Index by DAR Model (m, the length of the base, set to 870) Avg Adj R-Sq Avg p(i) Max p(i) Min p(i) Avg Abs Error (%) Std Dev (Avg Abs Error (%)) % The average adjusted R-squares for both DJI and S&P500 indices are over 0.976, which 7

8 indicates a very good fitness for the DAR models. The average number of lags (p) for them are less than with maximum numbers of lags of for DJI index and of for S&P500 index, which may explain a very short memory, as the indices at the time period T+ are expressed in a linear combination of only a few most recent (previous) prices. The average of the absolute errors (residuals) in percentage is about %, which could be also deemed as a reasonably good result. Figure : DJI Index (DAR model) DJI Index vs Forecast DJI Index Daily Close /0 08/0 08/0 08/03 08/03 08/04 08/05 08/05 08/06 Time DJI Index Forecast DJI Index Note: The forecast period: January May 30, 008 Figure : S&P500 Index (DAR model) S&P500 Index vs Forecast S&P500 Index Daily Close /0 08/0 08/0 08/03 08/03 08/04 08/05 08/05 08/06 Time SP500 Index Forecast S&P500 Index Note: The forecast period: January May 30, 008 8

9 III CHOOSE APPROPRIATE LENGTH OF THE BASE (m) Another key parameter for DAR model is m, the length of the base. What will be the optimal number for m? Is the bigger m, the better outcome or the lower error? After investigations of all stocks of DJI, as well as DJI and S&P500 Indices by applying our DAR model, we find that the first question is hard to answer, and generally speaking the optimal number for m, the length of the base would be around depending on the historical prices movements with the average of 470. But it is clear that the answer for the second question is NO, or that, we don t need to make m as big as possible. The Figure 3 & 4 shows the average absolute errors (in %) for DJI and S&P500 indices (daily close) for the period of January to May 30, 008 respectively, given all known historical indices figures by the end of year 007, respect to m (the length of the base). Similarly the Figure 5 & 6 show the average sum of error-squares for DJI and S&P500 indices (daily close) for the period of January to May 30, 008 respectively, given all known historical indices figures by the end of year 007, respect to m (the length of the base). Figure 3 & 4: DJIA and S&P500-Average Absolute Error (%) vs the length of the base.04%.0%.00% 0.98% 0.96% 0.94% 0.9% DJI: Average Absolute Error (%) (m = 680).%.0%.08%.06%.04%.0% S&P500: Average Absolute Error (%) (m = 960) Avg Abs Error (%) Min Avg Abs Error (%) Min Figure 5 & 6: DJIA and S&P500-Average Error-Squres vs the length of the base DJI: Average Error-Square (m = 40) S&P500: Average Error-Square (m = 870) Avg Error-Sq Min Avg Error-Sq Min The optimal m for forecasting DJI index during the period of January to May 30, 008 with the objective to minimize the average absolute error (%) is 680, while with the objective to minimize the sum of error-squares the optimal m is 40. The optimal m for forecasting S&P500 indices during the same period with the objective to minimize the average absolute error (%) is 960, while with the objective to minimize the sum of error-squares the optimal m 9

10 is 870. All graphs show a tendency that when m moves from a smaller number up to around 400, the average errors both in terms of absolute values (in %) and sum of error-squares significantly reduced, then when m moves above around 400, the average errors in both terms appear a diminishingly decrease, and eventually increase. So the tendency curve seems to be a right-skewed smile shape (See Figure 7). These characteristics could be more manifest in the results of my investigations for some individual stocks by applying DAR model to forecast their prices (daily close) with the same period in the following paragraphs. Figure 7: Tendency (right-skewed smile shape) Tendency Errors Applying DAR model to forecast stock prices (daily close) for CitiGroup (C), JP Morgan (JPM), Boeing (BA), and IBM for the period of January to May 30, 008, given all historical daily close prices before January, 008, in comparison with the actual daily close price figures, we construct the graphs regarding the errors terms with respect to m, the length of the base in Figure 8 & 9. The results are summarized as follows: Company Optimal m Min Avg Abs Error (%) Optimal m Min Avg Error-Sq Citigroup m = 40.66% m = JPMorgan m = 630.3% m = Boeing m = 40.43% m = IBM m = 690.5% m = Clearly, it is not the bigger m, the lower errors terms, which could be also interpreted that the older prices become less meaningful used for forecasting the future prices by using Dynamic Autoregressive Models (DAR). With the target of minimizing the average error terms (including average absolute error in %, and average error-squares), there will exist an optimal number for m (the length of the base), which, according to my investigations, would be in between 00 to 980, on average of around 470 (see details in the Appendix ). 0

11 Figure 8 Citigroup: Average Absolute Error in (%) JP Morgan: Average Absolute Error (%) 3.00% Min Avg Abs Error in %.90%.80% (m = 40).70%.60%.60% Min Avg Abs Error in %.50% (m = 630).40%.30% Boeing: Average Absolute Error (%) IBM: Average Absolute Error (%).60% Min Avg Abs Error in %.55%.50% (m = 40).45%.40%.50%.45% Min Avg Abs Error in %.40%.35% (m = 690).30%.5%.0% Figure 9 Citigroup: Average Error-Square JP Morgan: Average Error-Square (m = 0) Min Avg Error-Square (m = 30) Min Avg Error-Square Boeing: Average Error-Square.80 Min Avg Error-Square (m = 690).0.00 IBM: Average Error-Square 4.00 Min Avg Error-Square (m = 690) It is also clear that the adjusted R-squared for DJI and S&P indices are increasing with the increase of m, the length of the base (see Figure 0)

12 Figure 0: DJI: Adjusted R-Squared Adjusted R-squared S&P500: Adjusted R-Squared Adjusted R-squared Very much similar results for the stocks of DJI, for example, Citigroup, JP Morgan, Boeing and IBM (see Figure ). Figure Citigroup: Adjusted R-Squared JP Morgan: Adjusted R-Squarvd Boeing: Adjusted R-Squared IBM: Adjusted R-Squared IV DAR vs SAR MODEL The Static Autoregressive Model (SAR) is a simple process for forecasting the asset price at the time period T+, as described in Dynamic Autoregressive Model (DAR). We just treat the p value (the number of lags) and all coefficients (betas), which are obtained from the standard autoregressive model by using the initial base (B), as constants, to forecast the asset prices at the time period T+, T+,, T+i,. The forecast values are written as follows: Y = β + βy + β Y + β Y + + β Y (i =,, 3, ) T+ i 0 T+ i T+ i 3 T+ i 3... p T+ i p

13 Here p is the number of lags, from the DAR process in forecasting T+; b i (i = 0,,,, p) are coefficients, obtained from DAR process in forecasting T+; m, the length of the base, is pre-determined by analyzing the historical data, similarly when DAR model is applied. Similarly, supposed that we are at the end of year 007, and we have all historical data (daily close prices) for all stocks of DJI, we want to make forecasts for the future prices during the period of January to May 30, 008 for all of these stocks (also daily close price) by applying both DAR and SAR models, and then compare the estimated values generated from both DAR and SAR models to the actual daily close prices. We evaluate the results, finding that on average, DAR model outperforms SAR model by 0.8% to 40% in reducing the average of absolute error in percentage based on the lower range (30-350) of m, the length of the base, while at the upper range ( ) the results for both almost equal (see Figure ). The data is given in Appendix. Figure : DAR vs SAR model 3.0%.90%.70%.50%.30%.0%.90%.70%.50% Cpmparison: DAR vs SAR Models (Average Absolute Error in %) DAR Model SAR Model Note: All stocks of DJI (Dow Jones Industrial Average) used for the above analysis. Assuming that all stock prices (daily closed) before December 3, 007 are known, we both apply DAR and SAR model to forecast the closed prices at the period from January, 008 to May, 008, and then compare with the actual daily closed prices. 3

14 V CONCLUSION Even though the data we use for illustrations above is based on a special time period, the results for using DAR and SAR models to analyze all stocks of DJI as well as DJI & S&P indices show quite consistent and stable. We could reasonably conclude that DAR models will have advantages over SAR models in time-series forecasting by reducing the error terms, especially for the lower range of the length of the base (30-350). In other words, if there are not enough historical data available for analyzing, DAR models will most likely outperform over SAR model in time-series forecasting. With increasing the length of the base, these advantages for DAR models are diminishing and the results for both models are almost converging together. It is clear that there will be an optimal number for the length of the base (m), which means that it is not the bigger m, the better outcome. Via our investigations, on average, the length of the base should be around (for analyzing daily close prices we may just need daily close prices for past two years). The curve of the error term with respect to the length of the base (m) shows a right skewed smile shape, of which the lowest point is the optimal number for the length of the base. With increase of the length of the base for both models, the R-squares show a diminishing increasing, as they tend to increase significantly within the lower range (30-350), and then tend to be flat and gradually approaching to (however, can never be reached). 4

15 VI. REFERENCES (i) Financial Risk Manager Handbook (GARP), Philippe Jorion, 007 (ii) The Complete Guide to Capital Markets for Quantitative Professionals, Alex Kuznetsov, 006 (iii) Statistical Analysis of Financial Data in S-Plus, Rene A. Carmona, 004 (iv) Lectures Notes, Prof. F. Novomestky, Polytechnic Institute of New York University, 008 5

16 APPENDICES Appendix : Optimal Bases for DAR & SAR models Company DAR: Opt.Base (m) SAR: Opt.Base (m) AA AIG AXP BA C 0 70 CAT DD DIS GE GM 0 70 HD HON HPQ 0 90 IBM INTC JNJ JPM KO 0 0 MCD MMM MRK MSFT PFE PG T UTX VZ WMT XOM Average Min Max

17 Appendix : Average R-squares and Absolute Error (%) Comparisons for DAR and SAR Models Length of Base Average R-Square Average Absolute Error in Percentage m DAR SAR DAR SAR Outperform (%) %.94% 4.5% %.39% 3.% %.5% 8.0% %.89% 5.3% %.8%.9% %.78% 0.% %.75% 8.7% %.68% 6.% %.70% 6.9% %.69% 6.6% %.67% 5.8% %.66% 5.0% %.63% 3.6% %.6% 3.0% %.6%.9% %.60%.% %.60%.6% %.60%.6% %.58% 0.8% %.58% 0.8% %.58% 0.7% %.58% 0.8% %.58% 0.6% %.58% 0.7% %.59%.4% %.58% 0.7% %.59%.5% %.59%.5% %.59%.4% %.59%.4% %.60%.7% %.59%.% %.58% 0.7% %.59%.3% %.58% 0.6% %.58%.0% %.58%.% %.58% 0.8% 7

18 %.58% 0.9% %.58%.0% %.58% 0.8% %.58% 0.8% %.57% 0.6% %.57% 0.3% %.57% 0.% %.57% 0.% %.57% 0.3% %.57% 0.3% %.57% 0.3% %.57% 0.3% %.57% 0.3% %.57% 0.% %.57% 0.% %.56% -0.% %.57% 0.% %.56% 0.% %.56% -0.% %.57% 0.0% %.57% 0.% %.57% 0.% %.57% 0.5% %.56% -0.% %.56% 0.% %.56% -0.% %.56% -0.% %.56% -0.4% %.56% -0.3% %.55% -0.3% %.56% 0.0% %.56% -0.% %.56% 0.0% %.56% -0.3% %.56% -0.% %.56% -0.% %.56% -0.% %.56% -0.% %.56% -0.3% %.56% -0.% %.56% -0.3% %.55% -0.6% %.56% -0.% 8

19 %.55% -0.4% %.56% -0.% %.56% -0.% %.56% -0.% %.56% -0.% %.57% 0.9% %.56% -0.4% %.55% -0.7% %.55% -0.6% %.55% -0.7% %.55% -0.8% %.55% -0.6% %.55% -0.7% %.55% -0.6% %.55% -0.7% %.55% -0.7% %.55% -0.7% 9