DYNAMIC ECONOMETRIC MODELS Vol. 10 Nicolaus Copernicus University Toruń Witold Orzeszko Nicolaus Copernicus University in Toruń

Transcription

1 DYNAMIC ECONOMETRIC MODELS Vol. 10 Nicolaus Copernicus University Toruń 2010 Witold Orzeszo Nicolaus Copernicus University in Toruń Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient A b s t r a c t: Construction, estimation and application of the mutual information measure have been presented in this paper. The simulations have been carried out to verify its usefulness to detect nonlinear serial dependencies. Moreover, the mutual information measure has been applied to the indices and the sector sub-indices of the Warsaw Stoc Exchange. K e y w o r d s: nonlinearity, mutual information coefficient, mutual information, serial dependencies. 1. Introduction Measuring relationships between variables is an extremely important area of research in econometrics. To this end the Pearson correlation coefficient is commonly used. However, the Pearson coefficient is not a proper tool for measuring nonlinear dependencies. Therefore, in the case of nonlinearity other methods must be used. The mutual information coefficient is one of the most important tools to detect nonlinear relationships. It comes from the information theory and is based on a concept of entropy. The mutual information coefficient may be applied to measure dependencies between two time or serial dependencies in a single time. 2. Measuring Nonlinear Dependencies in Time Series There are various methods to measure nonlinear dependencies in time (cf. Granger, Terasvirta, 1993; Maasoumi, Racine, 2002; Bruzda, 2004). One of the most important is the mutual information measure (MI hereafter), given by the formula: Financial support of Nicolaus Copernicus Univerity in Toruń for the project UMK 397-E is gratefully acnowledged.

2 98 Witold Orzeszo p( x, y) (, ) (, ) log, 1( ) 2 ( ) I X Y = p x y dxdy (1) p x p y where p ( x, y) is a joint probability density function and p 1( x) and p 2( y) are marginal densities for random variables X and Y. It can be shown that for all X and Y the measure I ( X, Y) taes non-negative values and I ( X, Y ) = 0 only if X and Y are independent. It is convenient to define the mutual information coefficient, given by the expression: 2I ( X, Y ) R( X, Y) = 1 e. (2) It can be shown that the mutual information coefficient has the following properties (cf. Granger, Terasvirta, 1993; Granger, Lin, 1994): 1. 0 R ( X, Y ) 1, 2. R( X, Y ) = 0 X and Y are independent, 3. R( X, Y ) = 1 Y = f (X ), where f is some invertible function, 4. R is unaltered if X, Y are replaced by instantaneous transformations X ), h ( ) R X Y = R h ( X ), h ( ), h 1( 2 Y, i.e. (, ) ( 1 2 Y ) 5. if ( X, Y ) (or ( X ), h ( )) h 1( 2 Y, where h 1 and h 2 are instantaneous) has a joint Gaussian distribution with correlation ρ ( X, Y ), then R( X, Y ) = ρ( X, Y ). In the literature one can find several methods for estimating a value of I ( X, Y ). Essentially, due to the technique of estimating the probability density functions in Equation 1, they can be divided into three main groups (cf. Dionisio, Menezes, Mendes, 2003): histogram-based estimators, ernel-based estimators, parametric methods. The ernel-based estimators have many adjustable parameters such as the optimal ernel width and the optimal ernel form, and a non-optimal choice of those parameters may cause a large bias in the results. For the application of parametric methods one needs to now the specific form of the generating process (Dionisio, Menezes, Mendes, 2003)). Therefore a standard way is to estimate the densities by means of histograms (cf. Darbellay, Wuertz, 2000). One can also define auto mutual information at lag for a stationary discrete-valued stochastic process X 1, X 2,..., X n as the mutual information between random variables X t and X t + :

3 Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 99 P( xt, xt+ ) I ( X, ) = (, )log. ( ) ( ) t X t+ P xt xt+ (3) x x P xt P xt+ t t + Since the process is stationary, I ( X t, X t + ) is independent of t and so we can refer to the mutual information at lag, as I () (Fonseca, Crovella, Salamatian, 2008). This means that, the mutual information measure may be used to measure serial dependencies in a single time as well. To this end, the past realizations of the investigated data X should be taen as the variable Y. It should be emphasized that MI measures both linear and nonlinear dependencies, so to identify serial nonlinear relationships, analyzed data must be prefiltered by an estimated ARMA-type model. 3. Application of the Mutual Information Measure to Detect Serial Dependencies 3.1. Simulated Data The aim of the simulations was to verify, if the mutual information measure may be effectively applied to detect nonlinear serial dependencies. The time produced from five different generating models and two different sample sizes (with each of those models) were used in the simulations. This data was generated by Barnett et al. (1998) to compare the power of some popular tests for nonlinearity and chaos 1. Specifically, these were: five time of 2000 observations M1, M2, M3, M4, M5 and five time of their first 380 observations M1s, M2s, M3s, M4s, M5s. The investigated were generated from the following models 2 : I) M1 logistic map 3 : x.57x (1 x ), (4) t = 3 t 1 t 1 II) M2 process: x = h u, (5a) t t t 2 t = 1 t 1 + t 1 x 0 =. h + 0.1x 0.8h, (5b) where 0 = 1 h and 0 1 The data was downloaded from the homepage of W.A. Barnett: /barnett/papers.html. 2 In all cases, the white-noise disturbances u t were sampled independently from the standard normal distribution. 3 The logistic map with the parameter equaled to 3.57 generates chaotic dynamics.

4 100 Witold Orzeszo III) M3 Nonlinear Moving Average Process (NLMA): x u +.8u u, (6) t = t 0 t 1 t 2 IV) M4 ARCH(1) process: x t 2 t 1 t = x u, (7) V) M5 ARMA(2,1) process: x.8x x + u 0.3u, (8) t = 0 t 1 t 2 t + t 1 where x 0 = 1 and x 1 = In each case the mutual information measure was calculated for the raw and for its residuals from the fitted ARMA model. First, stationarity was verified using the Augmented Dicey-Fuller test. The null hypothesis of a unit root was strongly rejected for the all investigated data, except M5s. Thus, instead of M5s, the of its first differences M5s_diff was chosen for further research. In Table 1 the ARMA models fitted to analyzed are presented 4. Table 1. ARMA models for the generated Series ARMA model Series ARMA model M1 White noise (EX=0.648) M1s White noise (EX=0.649) M2 White noise (EX=0.034) M2s White noise (EX=0.067) M3 White noise (EX= 0.007) M3s White noise (EX= 0.033) M4 White noise (EX= 0.011) M4s White noise (EX= 0.018) M5 ARMA(1,1) M5s_diff MA(1) x, with rectangular partitions and calculating frequencies of points in each partition. Next, Equation 1 is used, i.e. the calculated frequencies are estimators of the probability density functions and the integration is carried out numerically. Next, the Ljung-Box test was applied to test if the residual are white noise. The test confirmed that no investigated residuals contain linear dependencies. To estimate the mutual information measure the method proposed by Fraser and Swinney (1986) was used 5. This method is based on an analysis of the twodimensional histogram. Briefly speaing, it consists in covering the twodimensional plane containing pairs ( ) t y t Let i denotes an estimated value of the mutual information measure between variables X t and X t. Due to a purpose of the research, the ey tas is to verify the hypothesis of mutual information measure s insignificance (i.e the hypothesis of independence). To this end, for each investigated and for 4 The models were selected based on the Schwarz criterion. 5 In the calculations the m-file created by A. Leontitsis was used.

5 Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 101 each =1,2,..., 10, the p-value was evaluated through bootstraping 6 with repetitions 7. In Tables 2-6 the calculated values of i and the corresponding p-values (at the bottom) are summarized. The p-values not larger than are bolded 8. Table 2. Values of i for M1s and M1 M1s M Table 3. Values of i for M2s and M2 M2s M Table 4. Values of i for M3s and M3 M3s M Table 5. Values of i for M4s and M4 M4s M Bootstrap without replacement (i.e. permutation) was performed. Bootstrapped p-values correspond to a one-sided test. 7 In this way, for each of the filtered an expected distribution of MI(1) (i.e. the MI measure with =1) was determined. Next, this distribution has led to evaluation of the p-value for each =1,2,...,10. 8 Note that the rejection of the null of i insignificance for at least one =1,2,...,10 implies the rejection of the hypothesis of serial independence. Therefore, adopting the value for each implies that the probability for a type I error (in the test of serial independence) is approximately 5%.

6 102 Witold Orzeszo Table 6. Values of i for M5s and M5 M5s M5s_ diff M5s_ diffma M5 M5ARMA In Tables 7-8 the results of nonlinearity detection carried out by the MI measure are summarized. Table 7. Results of nonlinearity detection for the long Series Serial dependencies Nonlinearity M1 YES YES M2 YES YES M3 YES YES M4 YES YES M5 YES NO Table 8. Results of nonlinearity detection for the short Series Serial dependencies Nonlinearity M1s YES YES M2s YES YES M3s NO NO M4s NO NO M5s_ diff YES NO As it is clearly seen, the MI measure correctly identified each of the investigated long. In an application to the short it led to erroneous conclusions in the case of M3s and M4s. The obtained result is consistent with studies by other authors, i.e. it indicates that histogram-based estimators may be unreliable in a case of a small number of observations (e.g. Dionisio, Menezes, Mendes, 2003) Stoc Maret Indices In this section the indices and the sector sub-indices of the Warsaw Stoc Exchange from (2078 observations) were analyzed. For the each index, the three time were investigated: daily, residuals from their ARMA and ARMA-GARCH models. Investigation of the residuals from the ARMA model gives information, if dependencies are nonlinear. If so, the standardized residuals from the ARMA-GARCH model were ana-

7 Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 103 lyzed, to verify if this class of processes can capture nonlinear dynamics found in the investigated data 9. The results of this analysis are presented in Tables Table 9. Values of i for the WIG index MA(1) MA(1)- GARCH(3,1) Table 10. Values of i for the WIG20 index MA(1) MA(1)- GARCH(3,1) Table 11. Values of i for the mwig40 index AR(3) AR(3)- GARCH(1,2) Table 12. Values of i for the swig80 index ARMA(1.2) ARMA(1,2) The fit of all estimated models was positively verified using the Box-Ljung and the Engle tests.

8 Table 13. Values of i for the WIG-Baning index MA(1) MA(1)- GARCH(1,2) Table 14. Values of i for the WIG-Construction index ARMA(2,1) ARMA(2,1) Table 15. Values of i for the WIG-Developers index ARMA(1,1) ARMA(1,1)- GARCH(1,2) Table 16. Values of i for the WIG-Food index ARMA(1,1) ARMA(1,1)

9 Measuring Nonlinear Serial Dependencies Using the Mutual Information Coefficient 105 Table 17. Values of i for the WIG-IT index AR(1) AR(1) Table 18. Values of i for the WIG-Media index MA(1) MA(1) Table 19. Values of i for the WIG-Oil&Gas index AR(2) AR(2) Table 20. Values of i for the WIG-Telecom index GARCH(1.3) The results summarized in Tables 9-20 indicate that evidence of serial dependencies was found for the most investigated indices 10. The same conclusion may be drawn for the residuals from the ARMA models, which means that the detected dependencies are nonlinear. In most cases the estimated ARMA- GARCH models were able to capture these nonlinearities. Only in the case 10 The exception is the WIG-Oil&Gas index. In this case the obtained result is rather unusual, i.e. filtering data by the ARMA model caused the appearance of significance of the MI measure.

10 106 Witold Orzeszo of WIG and mwig40 indices there are reasons to believe that identified nonlinearity is not caused by an ARCH effect. References Barnett W. A., Gallant A. R., Hinich M. J., Jungeilges J. A., Kaplan D., Jensen M. J. (1998), A Single-blind Controlled Competition among Tests for Nonlinearity and Chaos, Journal of Econometrics, 82.1, Bruzda J. (2004), Miary zależności nieliniowej w identyfiacji nieliniowych procesów eonomicznych (Measures of nonlinear relationship in identification of nonlinear economic processes), Acta Universitatis Nicolai Copernici, 34, Darbellay G.A, Wuertz D. (2000), The entropy as a tool for analysing statistical dependencies in financial time, Physica A, 287, Dionisio A., Menezes R., Mendes D.A. (2003), Mutual Information: a Dependence Measure for Nonlinear Time Series, Woring Paper, / pdf ( ). Fonseca N., Crovella M., Salamatian K. (2008), Long Range Mutual Information, Proceedings of the First Worshop on Hot Topics in Measurement and Modeling of Computer Systems (Hotmetrics 08), Annapolis. Fraser A.M., Swinney H.L. (1986), Independent Coordinates for Strange Attractors from Mutual Information, Physical Review A, 33.2, Granger C. W. J., Terasvirta T. (1993), Modelling Nonlinear Economic Relationship, Oxford University Press, Oxford. Granger C. W. J., Lin J-L. (1994), Using the Mutual Information Coefficient to Identify Lags in Nonlinear Models, Journal of Time Series Analysis, 15, Maasoumi E., Racine J. (2002), Entropy and Predictability of Stoc Maret Returns, Journal of Econometric, 107, Orzeszo W. (2010), Detection of Nonlinear Autodependencies Using Hiemstra-Jones Test, Financial Marets. Principles of Modeling, Forecasting and Decision-Maing, eds. Milo W., Szafrańsi G., Wdowińsi P., Współczynni informacji wzajemnej jao miara zależności nieliniowych w szeregach czasowych Z a r y s t r e ś c i. W artyule scharateryzowano onstrucję, estymację oraz możliwości zastosowania współczynnia informacji wzajemnej. Przedstawiono wynii symulacji, prowadzących do weryfiacji jego przydatności w procesie identyfiacji zależności nieliniowych w szeregach czasowych. Ponadto zaprezentowano wynii zastosowania tego współczynnia do analizy indesów Giełdy Papierów Wartościowych w Warszawie. S ł o w a l u c z o w e: nieliniowość, współczynni informacji wzajemnej, mutual information, identyfiacja zależności.