Goodness-of-Fit Tests for Time Series Models: A Score-Marked Empirical Process Approach

Goodness-of-Fit Tests for Time Series Models: A Score-Marked Empirical Process Approach By Shiqing Ling Department of Mathematics Hong Kong University of Science and Technology Let {y t : t = 0, ±1, ±2, } be a sequence of real time series which is defined on the probability space (Ω, F, P ), and F t be the σ field generated by {y t, y t 1, }. In the last two decades, many models have been proposed for the time series {y t }. The main concerns in these models are the conditional mean function µ and the conditional variance function h, given {y t 1, y t 2, }. More precisely, almost all these models can be written in the following form: (0.1) y t = µ t (θ) + η t h t (θ), where µ t (θ) = µ(θ, y t 1, y t 2 ), h t (θ) = h(θ, y t 1, y t 2 ) ω 0 (a positive constant) almost surely (a.s.), {η t, t = 0, ±1, } is a sequence of independent and identically distributed (i.i.d.) random variables with a common distribution F, zero mean and variance 1, and θ is an unknown p dimensional parameter vector and θ 0 is its true value. Furthermore, assume that µ(θ, ) and h(θ, ) belong to the class of functions, M = {[ µ(θ, ), h(θ, ) ] : θ Θ }, where Θ R p is a suitable parameter space. Much of the literature has focused on estimating the parameter θ in model (0.1). But there has been relatively little attention to the goodness-of-fit test for the model. Koul and Ling (2005) proposed a weighted empirical residual procedure for testing 1

the distribution of η t. However, a more important issue is to test the null hypothesis: H 0 : [ µ(θ 0, ), h(θ 0, ) ] M for some θ 0 Θ. The classical method for this is to test the correlations of the residuals and squaredresiduals from the estimated model, which are based on the portmanteau tests proposed by Box and Pierce (1970) and Ljung and Box (1978) and by McLeod and Li (1983) and Li and Mak (1994), respectively. The recent approaches based on a generalized spectrum were proposed by Hong (1999) and Hong and Lee (2003). This paper proposes a new goodness-of-fit test for the null H 0 against non-specific alternatives. Our proposal solves two difficult problems raised in the literature. The first problem is to test the null against the alternative hypothesis, H 1 : y t = µ t (θ 0 ) + µ t (θ 1 )I{y t 1 x} + η t h t (θ 0 ) + h t (θ 1 )I{y t 1 x}, where θ 0 θ 1. Under H 1, y t is called the threshold model. The original papers that considered this problem were by Chan (1990, 1991) and Chan and Tong (1990) for the AR model (a special case of model (0.1)), see also Tsay (1989, 1998). Testing the AR-ARCH model against threshold AR-ARCH models was studied by Wong and Li (1997). The Wald test was studied by Hansen (1996) for the AR model. Testing a threshold in the nonstationary AR model or in the cointegration model was investigated by Caner and Hansen (2001) and Hansen and Seo (2002). The likelihood ratio test for the linear MA model against the threshold MA models was studied by Ling and Tong (2005). Under suitable conditions and under H 0, the likelihood ratio, Wald- and scoretests for H 0 against H 1 can be approximated generally by S n (x) T n(x, ˆθ n )(Σ x Σ x Σ 1 Σ x ) 1 T n (x, ˆθ n ) = T n(x, θ 0 )(Σ x Σ x Σ 1 Σ x ) 1 T n (x, θ 0 ) + o p (1), uniformly in x R, where o p (1) stands for converging to zero in probability as 2

n, ˆθ n is a suitable estimator of θ 0, (0.2) (0.3) T n (x, θ) = 1 n D t (θ)i{y t 1 x}, n T n (x, θ 0 ) = T n (x, θ 0 ) Σ xσ 1 n n D t (θ 0 ), D t (θ) is a score-type function under H 0, Σ = Σ, and Σ x = E[D t (θ 0 )D t(θ 0 )I{y t 1 x}]. If x is known, then S n (x) is asymptotically χ 2. But x is usually unknown. A natural way for testing H 0 in this case is to maximize S n (x) over a finite interval [a, b] in R, i.e., max x [a,b] S n (x). The difficulty here is to find its critical values. Except for the AR model in Chan (1990), we have to use the simulation method to find the critical values case by case, see Wong and Li (1997), Hansen and Seo (2002) and Ling and Tong (2005). This greatly prevents the research progress in this direction. The second problem is in the model checking approach. This approach was proposed by Stute (1997) for regression models, and was extended by Koul and Stute (1999) for a general AR(1) model, (0.4) y t = µ(θ, y t 1 ) + ση t, where σ > 0 is a constant. It is based on the residual-marked empirical process, U n (x, ˆθ n ) = 1 n ε t (ˆθ n )I{y t 1 x}, n where ε t (θ) = y t µ(θ, y t 1 ). Using the martingale transformation technique, they construct a test statistic that converges to a maxima of the standard Brownian motion. The most attractive point of this approach is that the null H 0 includes a large family of functions and its alternatives are not necessarily specified. Thus, it can serve as a portmanteau test for the null model. However, as they pointed out, it is difficult to extend this method for high-lag models, even for the simple AR(2) model. Only very few time series data sets are generated by a linear or nonlinear 3

AR(1) model as above. This method excludes many interesting applications, such as ARMA, long memory ARIMA, and ARCH-type models, among others. To overcome the previous two difficulties, we first note that, when model (0.1) reduces to the general AR (1) model (0.4), T n (x, θ) is simply T n (x, θ) = 1 n n µ t (θ) ε t (θ)i{y t 1 x}, θ if η t is normal. Thus, T n (x, θ) can be considered as some sort of the randomly weighted residual-marked empirical process in Koul and Stute (1999). We call this process the score-marked empirical process since the weight is a score function. In the spirit of Stute (1997), we can use T n (x, θ) instead of U n (x, θ) to construct a portmanteau test for the null H 0. Furthermore, in order to avoid the problem raised when we use S n (x), this paper will use a linear transformation of T n (x, θ) to construct the test statistic. We show that the new test statistic converges to a maxima of a standard Brownian motion under the null hypothesis and has nontrivial asymptotic local power. This approach avoids the martingale transformation technique as in Koul and Stute (1999) and provides a general but easy way to test the null H 0 against its threshold models in H 1. When the alternative is not specified, it serves as a goodness-of-fit test. Our weight scheme is akin to the weighted empirical process approach proposed by Bai (1996) for testing the change-point in regression models. The conditions for this are verified for the ARMA model and the GARCH model, respectively. Simulation results show that the test has a satisfactory size and is more powerful than the Ljung-Box and the Li-Mak tests. Technically, we establish weak convergence of a general marked empirical process, (0.5) W n (x) = 1 n Z t I{ξ t r x}, n where x R and {(Z t, ξ t r )} is a stationary sequence and r is an interger. Some special versions of {W n (x)} have been investigated in the literature. For an AR(p) 4

model, Chan (1990) and Hansen (2000) established weak convergence of {W n (x) : x R} with Z t = y t 1 η t. They require that EZt 4 < and (Z t, ξ t ) is ρ mixing. We note that it is difficult to verify the ρ mixing condition for a given model. Except for the AR and MA, it seems that a ρ mixing condition is not available for other models. Koul and Stute (1999) established weak convergence of {W n (x) : x [, ]} for a strictly stationary (one-step) Markov process {Z t } with ξ t r = y t 1 and E(Z 4 t y t 1 1+ι ) <. The weak convergence of {W n (x)} in this paper does not need any mixing assumption or Markovian property and its moment condition is almost minimal. To see the significance of our weak convergence, we like to review some related results on the empirical processes for dependent data. Such processes have been extensively investigated in the general setup [see for example, Philipp (1982) and Massart (1987)] and are much more complicated than those for independent data. The main difficulty is that the symmetrization argument for the independent case cannot be directly used for the dependent data. Based on the independent block technique, Yu (1994) only obtained the rates of convergence of the empirical processes of the stationary α mixing and β mixing sequences. To remove the assumption of independence, Nishiyama (2000) used the partitioning entropy technique to establish the weak convergence of martingale differences taking values in a general space. It seems that this is the best and the most general theory in the field for dependent data. However, for the marked empirical process (0.5), it is not clear how to find a pseudo-metric ρ such that his [P E ] condition holds, see his Theorem 3.4 and Corollary 4.4 Using the bracketing argument, Andrews and Pollard (1994) established the weak convergence of an empirical process indexed by a uniformly bounded class of real-value functions in terms of an α mixing triangular array. For the nonuniform bounded class of functions, except for Nishiyama (2000), it seems that only very few nice results are available. The detail reviews on this can be found in An- 5

drews and Pollard (1994, p.126-128). The corresponding class of real-value functions in the marked empirical process (0.5) is not uniformly bounded. The arguments in Chan (1990) and Hansen (2000) heavily depend on the moment inequality for the ρ mixing sequence in Peligrad (1982). Their method can not be applied for α mixing sequences since such a moment inequality for the α mixing sequences is not available in the literature. We note that Shao and Yu (1996) obtained some moment inequalities for the α mixing sequences. But these seem not enough to support the arguments in Chan (1990) and Hansen (2000). Our proof directly uses a chaining argument similar to that in Bass (1985), see also Ossiander (1987). Related discussions can be founded in Pollard (1984) and Van der Vaart and Wellner (1996). REFERENCES Andrews, D.W.K and Pollard, D. (1994) An introduction to functional central limit theorems for dependent stochastic-processes. International Statistical Review 62 119-133. Bai, J. (1996) Testing for parameter constancy in linear regressions: an empirical distribution function approach. Econometrica 64, 597-622. Bass, R.F. (1985) Law of the iterated logarithm for set-indexed partial sum processes with finite variance. Z. Wahrsch. Verw. Gebiete 70, 591-608. Box, G.E.P. and Pierce, D.A. (1970) Distribution of the residual autocorrelations in autoregressive integrated moving average time series models. J. Amer. Statist. Assoc. 65, 1509-1526. Caner, M. and Hansen, B.E. (2001) Threshold autoregression with a unit root. Econometrica 69 1555-1596. Chan, K.S. (1990) Testing for threshold autoregression. Ann. Statist. 18, 1886-1893. 6

Chan, K.S. (1991) Percentage points of likelihood ratio tests for threshold autoregression. J. Roy. Statist. Soc. Ser. B 53, 691-696. Chan, K.S. and Tong, H. (1990) On likelihood ratio tests for threshold autoregression. J. Roy. Statist. Soc. Ser. B 52, 469-476. Hansen, B.E. (1996) Inference when a nuisance parameter is not identified under the null hypothesis. Econometrica 64, 413-430. Hansen, B.E. and Seo, B. (2002) Testing for two-regime threshold cointegration in vector error-correction models. J. Econometrics 110, 293-318. Hong, Y. (1999) Hypothesis testing in time series via the empirical characteristic function: A generalized spectral density approach. Journal of the American Statistical Association 84, 12011220. Hong, Y. and Lee, T.H. (2003) Diagnostic checking for the adequacy of nonlinear time series models. Econometric Theory 19, 1065-1121. Koul, H.L. and Stute, W. (1999) Nonparametric model checks for time series. Ann. Statist. 27, 204-236. Koul, H.L. and Ling, S. (2003) Fitting an error distribution in some heteroscedastic time series models. Working Paper in HKUST, Hong Kong. Li, W.K. and Mak, T.K. (1994) On the squared residual autocorrelations in non-linear time series with conditional heteroskedasticity. Journal of Time Series Analysis 15, 627-636. Ling, S. and Tong, H. (2005) Testing a linear MA model against threshold MA models. Accepted by Ann. of Statist.. Ljung, G.M. and Box, G.E.P. (1978) On a measure of lack of fit in time series models. Biometrika 65, 297-303. 7

Massart, P. (1987) Invariance principles for empirical processes: the weakly dependent case. Chapter 1B of Ph.D. dissertation, University of Paris-South, Orsay. McLeod, A.I. and Li, W.K. (1983) Diagnostic checking ARMA time series models using squared-residual autocorrelations. J. Time Ser. Anal. 4, 269-73. Nishiyama, Y. (2000) Weak convergence of some classes of martingales with jumps. Ann. Probab. 28, 685-712. Ossiander, M. (1987) A central limit theorem under metric entropy with L 2 bracketing. Ann. Probab. 15, 897-919. Peligrad, M. (1982) Invariance principles for mixing sequences of random variables. Ann. Probab. 10, 968-981. Philipp, W. (1982) Invariance principles for sums of mixing random elements and the multivariate empirical process. Colloq. Math. Soc. Jnos Bolyai, 36, 843-872. Pollard, D. (1984)Convergence of stochastic processes. Springer Series in Statistics. Springer-Verlag, New York. Shao, Q.M. and Yu, H. (1996) Weak convergence for weighted empirical processes of dependent sequences. Ann. Probab. 24, 2098-2127. Stute, W. (1997) Nonparametric model checks for regression. Ann. Statist. 25, 613-641. Tsay, R.S. (1989) Testing and modeling threshold autoregressive processes. J. Amer. Statist. Assoc. 84, 231-240. Tsay, R.S. (1998) Testing and modeling multivariate threshold models. J. Amer. Statist. Assoc. 93, 1188-1202. 8

Van der Vaart, A.W. and Wellner, J.A. (1996) Weak convergence and empirical processes. With applications to statistics. Springer Series in Statistics. Springer-Verlag, New York. Wong, C.S. and Li, W.K. (1997) Testing for threshold autoregression with conditional heteroscedasticity. Biometrika 84, 407-418. Yu, B. (1994) Rates of convergence for empirical processes of stationary mixing sequences. Ann. Probab. 22, 94-116. 9