Department of Econometrics and Business Statistics

Transcription

1 ISSN X Australia Department of Econometrics and Business Statistics Nonlinear Regression with Harris Recurrent Markov Chains Degui Li, Dag Tjøstheim and Jiti Gao July 202 Working Paper 4/2

2 Nonlinear Regression with Harris Recurrent Markov Chains By Degui Li, Dag Tjøstheim and Jiti Gao Monash University and University of Bergen Abstract In this paper, we study parametric nonlinear regression under the Harris recurrent Markov chain framework. We first consider the nonlinear least squares estimators of the parameters in the homoskedastic case, and establish asymptotic theory for the proposed estimators. Our results show that the convergence rates for the estimators rely not only on the properties of the nonlinear regression function, but also on the number of regenerations for the Harris recurrent Markov chain. We also discuss the estimation of the parameter vector in a conditional volatility function and its asymptotic theory. Furthermore, we apply our results to the nonlinear regression with I) processes and establish an asymptotic distribution theory which is comparable to that obtained by Park and Phillips 200). Some simulation studies are provided to illustrate the proposed approaches and results. Keywords: Asymptotic distribution, asymptotically homogeneous functions, β-null recurrent Markov chains, Harris recurrence, integrable functions, least squares estimation, nonlinear regression. JEL Classifications: C3, C22. The first authors research was supported by an Australian Research Council Discovery Early Career Researcher Award DE20030) and the third author s research was supported by an Australian Research Council Discovery Grant DP096374). This work was started when the first and third authors visited the second author at Department of Mathematics, University of Bergen in 20. The authors acknowledge the financial support from the Norwegian Research Council.

3 Introduction In this paper, we consider a parametric nonlinear regression model defined by Y t = gx t, θ 0, θ 02,, θ 0d ) + e t =: gx t, θ 0 ) + e t, t =, 2,, n,.) where θ 0 is the true value of the d-dimensional parameter vector such that θ 0 = θ 0, θ 02,, θ 0d ) τ Θ R d and g, ) : R d+ R is assumed to be known. Throughout this paper, we assume that Θ is a compact set. How to construct a consistent estimator for the parameter vector θ 0 and derive an asymptotic theory are an important issue in modern statistics and econometrics. When the observations Y t, X t ) satisfy stationarity and weak dependence conditions, there is an extensive literature on the asymptotic theory of the parametric nonlinear estimator of θ 0, see, for example, Jennrich 969), Malinvaud 970) and Wu 98) for some early references. As pointed out in the literature, assuming stationarity is too restrictive and unrealistic in many practical applications. When tackling economic and financial issues from a time perspective, we often deal with nonstationary components. For instance, neither the consumer price index nor the share price index, nor the exchange rates constitute a stationary process. A traditional method to handle such data is to take the first order difference to eliminate possible stochastic or deterministic trends involved in the data, and then do the estimation for a stationary model. However, such differencing may lead to loss of useful information. Thus, the development of a modeling technique that takes both nonstationary and nonlinear phenomena into account in time series analysis is crucial. Without taking differences, Park and Phillips 200) hereafter PP) study the nonlinear regression.) with the regressor {X t } satisfying a unit root or I)) structure, and prove that the rates of convergence of the nonlinear least squares NLS) estimator of θ 0 depend on the properties of g, ). For 2

4 an integrable g, ), the rate of convergence is as slow as n /4, and for an asymptotically homogeneous g, ), the rate of convergence can achieve the n-rate and even super-n-rate of convergence. As also pointed out in a recent paper by Myklebust et al 202), the null recurrent Markov process is a nonlinear generalization of the linear unit root process. In fact, under the framework of null recurrent Markov chains, there has been an extensive literature on nonparametric and semiparametric estimation Karlsen and Tjøstheim 200; Karlsen et al 2007, 200; Lin et al 2009; Gao et al 20a; Schienle 20; Chen et al 202), by using the technique of the split chain Nummelin 984; Meyn and Tweedie 2009), and the generalized ergodic theorem and functional limit theorem developed in Karlsen and Tjøstheim 200). As far as we know, however, there is virtually no work on the parametric estimation of the nonlinear regression model.) when the regressor {X t } is generated by a class of Harris recurrent Markov processes that includes both stationary and nonstationary cases. This paper aims to fill this gap. If the function g, ) is integrable, we can directly use some existing results for functions of Harris recurrent Markov processes to develop an asymptotic theory for the estimator of θ 0. The case that g, ) belongs to a class of asymptotically homogeneous functions is much more challenging, as in this case the function g, ) is no longer bounded. In nonparametric or semiparametric estimation theory, we do not have such problems because the kernel function is usually assumed to be bounded and have a compact support. Unfortunately, most of the existing results for the asymptotic theory of the Harris recurrent Markov process focus on the case that g, ) is bounded and integrable see, for example, Chen 999, 2000). Hence, in this paper, we first modify the conventional NLS estimator for the asymptotically homogeneous g, ), and then use a novel method to establish asymptotic distribution as well as rates of convergence for the modified parametric estimator. Our results show that the rates of convergence for the parameter vector in nonlinear cointegrating models rely not only on the properties of the function g, ), 3

5 but also on the magnitude of the regeneration number for the Harris recurrent Markov chain. In addition, we also study two important issues, which are closely related to nonlinear regression with Harris recurrent Markov chains. The first one is to study the estimation of the parameter vector in a conditional volatility function and its asymptotic theory. As the estimation method is based on the log-transformation, the rates of convergence for the proposed estimator would depend on the property of the derivative of the log-transformed volatility function. Meanwhile, we also discuss the nonlinear regression with I) processes when g, ) is asymptotically homogeneous. By using Theorem 3.2 in Section 3, we obtain asymptotic normality for the parametric estimator with a stochastic normalized rate, which is comparable to Theorem 5.3 in PP. However, our derivation is done under Markov perspective, which carries with it the potential of extending the theory to nonlinear and nonstationary autoregressive processes. The rest of this paper is organised as follows. Some preliminary results about Markov theory especially Harris recurrent Markov chain) and function classes are introduced in Section 2. The main results of this paper are given in Sections 3 5. Some simulation studies are carried out in Section 6. Section 7 concludes this paper. The proofs of the main results and some useful lemmas are provided in Appendices A and B, respectively. 2 Preliminary results To make the paper self-contained, in this section, we first provide some basic definitions and preliminary results for Harris recurrent Markov process {X t }, and then define function classes in a way similar to those introduced in PP. 4

6 2. Markov theory Let {X t, t 0} be a ϕ-irreducible Markov chain on the state space E, E) with transition probability P. This means that for any set A E with ϕa) > 0, we have P t x, A) > 0 for x E. We further assume that the ϕ-irreducible Markov chain t= {X t } is Harris recurrent. Definition 2.. A Markov chain {X t } is Harris recurrent if, given a neighborhood B v of v with ϕb v ) > 0, {X t } returns to B v with probability one, v E. The Harris recurrence allows one to construct a split chain, which decomposes the partial sum of functions of {X t } into blocks of independent and identically distributed i.i.d.) parts and two asymptotically negligible remaining parts. Let τ k be the regeneration times, n the number of observations and the number of regenerations as in Karlsen and Tjøstheim 200), where they use the notation T n) instead of. For the process {GX t ) : t 0}, defining Z k = τ 0 t=0 τ k GX t ), k = 0, t=τ k + n t=τ + GX t ), k, GX t ), k = +, where G ) is a real function defined on R, then we have S n G) = n GX t ) = Z 0 + t=0 k= Z k + Z +. 2.) From Nummelin 984), we know that {Z k, k } is a sequence of i.i.d. random variables, and Z 0 and Z n) converge to zero almost surely a.s.) when they are divided by the number of regenerations using Lemma 3.2 in Karlsen and Tjøstheim 200). The general Harris recurrence only yields stochastic rates of convergence in asymptotic theory of the parametric and nonparametric estimators see, for example, Theorems 3. and 3.2 below), where distribution and size of the number of regenerations 5

7 have no a priori known structure but fully depend on the underlying process {X t }. To obtain a specific rate of in our asymptotic theory, we next impose some restrictions on the tail behavior of the distribution of the recurrence times of the Markov chain. Definition 2.2. A Markov chain {X t } is β-null recurrent if there exist a small nonnegative function f, an initial measure λ, a constant β 0, ), and a slowly varying function L f ) such that E λ n fx t )) i= Γ + β) nβ L f n), 2.2) where E λ stands for the expectation with initial distribution λ and Γ + β) is the Gamma function with parameter + β. The definition of a small function f in the above definition can be found in some existing literature see, for example, pp.5 in Nummelin 984). Assuming β-null recurrence restricts the tail behavior of the recurrence time of the process to be a regularly varying function. Following a result in Karlsen and Tjøstheim 200), the regeneration number of the β-null recurrent Markov chain {X t } has the following asymptotic distribution n β L s n) d M β ), 2.3) where L s = L f /π s f), π s ) is an invariant measure of the Markov chain {X t }, and M β ) is the Mittag-Leffler distribution with parameter β cf., Kasahara 984). Since < n a.s. for the null recurrent case by 2.3), the rates of convergence for the nonparametric kernel estimators are slower than those for the stationary time series cased see, for example, Karlsen et al 2007, Gao et al 20a). However, this is not necessarily the case for the parametric estimator in our model.). In Section 3 below, we will show that our rate of convergence is slower than that for the stationary time series for integrable g, ) and may be faster than that for the stationary time series case for asymptotically homogeneous g, ). In addition, our rates of convergence also 6

8 depend on the magnitude of β, which measures the recurrence times of the Markov chain {X t }. 2.2 Examples of β-null recurrent Markov chains For a stationary or positive recurrent process, β =. We next give several examples of β-null recurrent Markov chains. Example 2.. /2-null recurrent Markov chain). i) Let a random walk process be defined as X t = X t + x t, t =, 2,, X 0 = 0, 2.4) where {x t } is a sequence of i.i.d. random variables. Kallianpur and Robbins 954) show that {X t } defined by 2.4) is a /2-null recurrent Markov chain under weak conditions on the distribution of x t. ii) Consider a parametric threshold autoregressive TAR) model of the form X t = α X t I X t S ) + α 2 X t I X t S c) + u t, X 0 = 0, 2.5) where S is a compact subset of R, S c is the complement of S, α 2 =, < α <, {u t } is assumed to be i.i.d. with E[u ] = 0, 0 < E[u 2 ] < and E[u 4 ] <, and the distribution of {u t } is absolutely continuous with respect to Lebesgue measure with f u ) being the density function satisfying inf x S f u x) > 0 for all compact sets S. Recently, Gao et al 20b) have shown that such TAR process {X t } is a /2-null recurrent Markov chain. Furthermore, we can generalize the TAR model 2.5) to X t = HX t, ζ)i X t S ) + X t I X t S c) + u t, where X 0 = 0, sup x S Hx, ζ) < and ζ is a parameter vector. According to Teräsvirta et al 200), the above autoregressive process is also a /2-null recurrent Markov chain. 7

9 Example 2.2. β-null recurrent Markov chain with β /2). Let {x t } be a sequence of i.i.d. random variables taking positive values, and {X t } be defined as X t, X t >, X t = x t, X t [0, ], for t, and X 0 = C 0 for some positive constant C 0. Myklebust et al 202) prove that {X t, t 0} is β-null recurrent if and only if P[x ] > n) n β l n), 0 < β <, where [ ] is the integer function and l ) is a slowly varying and positive function. From the above examples, the β-null recurrent Markov chain framework is not restricted to linear processes see Example 2. ii)). Furthermore, such a null recurrent class has the invariance property that if {X t } is β-null recurrent, then for a one-to-one transformation T ), {T X t )} is also β-null recurrent see, for example, Teräsvirta et al 200). Such invariance property does not hold for the I) processes. For other examples of the β-null recurrent Markov chain, we refer to Example in Schienle 20). For some general conditions on diffusion processes to ensure the Harris recurrence is satisfied, we refer to Höpfner and Löcherbach 2003) and Bandi and Phillips 2009). 2.3 Function classes Similarly to Park and Phillips 999, 200), we consider two classes of parametric nonlinear functions: integrable functions and asymptotically homogeneous functions, which include many commonly-used functions in nonlinear regression. Let A = q i= q j= a2 ij for A = a ij) q q, and a be Euclidean norm of the vector a. A function hx) : R R d is π s -integrable if hx) π s dx) <, R where π s ) is the invariant measure of the Harris recurrent Markov chain {X t }. When π s ) is differentiable such that π s dx) = p s x)dx, hx) is π s -integrable if and only 8

10 if hx)p s x) is integrable. For the random walk case as in Example 2. i), the π s - integrability reduces to the conventional integrability as π s dx) = dx. Definition 2.3. A d-dimensional vector function hx, θ) is said to be integrable on Θ if there exists a θ Θ such that hx, θ ) is π s -integrable, and for each θ Θ, there exist a neighborhood B θ of θ and M : R R bounded and π s -integrable such that hx, θ ) hx, θ) θ θ Mx) for any θ B θ. The above definition is comparable to Definition 3.3 in PP. However, in our definition, we do not need condition b) in Definition 3.3 of their paper, which makes the integrable function family in this paper slightly more general. We next introduce a class of asymptotically homogeneous functions. Definition 2.4. For a d-dimensional vector function hx, θ), let hλx, θ) = κλ, θ)hx, θ) + Rx, λ, θ), where κ, ) is nonzero. hλx, θ) is said to be asymptotically homogeneous on Θ if the following two conditions are satisfied: i) H, θ) is locally bounded uniformly for any θ Θ and continuous with respect to θ; ii) the remainder term Rx, λ, θ) is of order smaller than κλ, θ) as λ for any θ Θ. As in PP, κ, ) is the asymptotic order of h, ) and H, ) is the limit homogeneous function. The above definition is quite similar to that of an H-regular function in PP except that the regularity condition a) in Definition 3.5 of PP is replaced by the local boundness condition i) in Definition 2.4. Following Definition 3.4 in PP, as Rx, λ, θ) is of order smaller than κ, ), we have either R x, λ, θ) = aλ, θ)a R x, θ) 2.6) or R x, λ, θ) = bλ, θ)a R x, θ) B R λx, θ), 2.7) where aλ, θ) = oκλ, θ)), bλ, θ) = Oκλ, θ)) as λ, sup θ Θ A R, θ) is locally bounded, and sup θ Θ B R, θ) is bounded and vanishes at infinity. The conditions 2.6) and 2.7) will be used in the proof of Lemma B.3 in Appendix B. 9

11 Note that the above two definitions can be similarly generalized to the case that h, ) is a d d matrix of functions. Details are omitted here to save space. 3 Main results In this section, we establish some asymptotic results for the parametric estimators of θ 0 when g, ) belongs to the two classes of functions introduced in Section Integrable function on Θ We first consider estimating model.) by the nonlinear least squares NLS) approach, which is also used by PP in the unit root framework. Define the loss function by n L n θ) = Y t gx t, θ)) 2. 3.) t= We can obtain the resulting estimator θ n by minimizing L n θ) over θ Θ, i.e., For θ = θ,, θ d ) τ, let θ n = arg min θ Θ L nθ). 3.2) ) ) gx, θ) 2 gx, θ) ġx, θ) =, gx, θ) = θ j d θ i θ j. d d Before deriving the asymptotic properties of θ n when the derivatives of g ) are integrable on Θ, we give some regularity conditions. Assumption 3.. i) {X t } is a Harris recurrent Markov chain with invariant measure π s ). ii) {e t } is a sequence of martingale differences satisfying Ee t F n,t ) = 0 and Ee 2 t F n,t ) = σ 2 <, a.s., where F n,t = σ X s, e s2, s n, s 2 t). 0

12 Assumption 3.2. Both ġx, θ) and gx, θ) are integrable on Θ. Furthermore, Lθ 0 ) := ġx, θ 0 )ġ τ x, θ 0 )π s dx) is a positive definite matrix. Remark 3.. In Assumption 3. i), {X t } is assumed to be Harris recurrent, which includes both positive and null recurrent Markov chains. For the positive recurrent case, the derivative function of the invariant measure reduces to the density function if it exists. For more discussion of Harris recurrence as well as its application, we refer to Karlsen and Tjøstheim 200), Karlsen et al 2007) and Meyn and Tweedie 2009). In Assumption 3. ii), we relax the i.i.d. condition on {e t }, and instead assume a martingale difference structure. And {e t } can also be allowed to be stationary and α mixing dependent at the cost of more tedious proofs to deal with the correlation structure. Furthermore, the homoskedasticity on the error term can also be relaxed. For instance, we can allow the existence of the heteroskedasticity structure defined by Ee 2 t F n,t ) = σ 2 X t, γ 0 ) a.s., 3.3) where σ 2, γ 0 ) is a nonlinear conditional variance function indexed by a p dimensional parameter γ 0. See Section 4 for more discussion. Assumption 3.2 is quite standard, see, for example, conditions b) and c) in Theorem 5. in PP. We next give the asymptotic properties of θ n. The following theorem is applicable for both stationary positive recurrent) and nonstationary null recurrent) time series. Theorem 3.. Let Assumptions 3. and 3.2 hold. a) There exists a solution θ n to minimize the loss function L n θ) defined in 3.), and θ n θ 0 = o P ). 3.4) b) The estimator θ n has an asymptotic normal distribution of the form: θn θ 0 ) ) d N 0 d, σ 2 L θ 0 ), 3.5)

13 where 0 d is a d dimensional null vector. Remark 3.2. As we assume that {e t } satisfies the martingale difference structure defined in Assumption 3. ii), θ n is asymptotically normal with stochastic convergence rate for both the stationary and nonstationary cases. The number of regenerations plays an important role in constructing the consistent estimators of σ 2 and Lθ 0 ). However, as is not directly observable, we need to link it with a directly observable hitting time. Indeed, if C E and I C has a ϕ positive support, the number of times that the process visits C up to the time n is defined by N C n) = n t= I CX t ). By Lemma 3.2 in Karlsen and Tjøstheim 200), we have N C n) π si C a.s. 3.6) From 3.5) in Theorem 3. and 2.3) in Section 2. above, we have the following corollary. Corollary 3.. Suppose that the conditions in Theorem 3. are satisfied. Furthermore, {X t } is a β-null recurrent Markov chain with 0 < β <. Then we have ) θ n θ 0 = O P, 3.7) nβ L s n) where L s n) is defined below 2.3). Remark 3.3. As β < and L s n) is a slowly varying positive function, for the integrable case, the rate of convergence of θ n is slower than n, the rate of convergence of the parametric NLS estimator in the stationary time series case. Roughly speaking, the main reason for the reduction in the rate of convergence is because in the β-null recurrent case, the amount of time spent by the Markov chain around any particular point is of order n β L s n), which is less than n for the stationary positive recurrent) time series case. Corollary 3. complements the existing results on the rates of convergence of nonparametric estimators in β-null recurrent Markov processes see, for example, Karlsen et al 2007; Gao et al 20a). For the random walk case, which 2

14 corresponds to /2-null recurrent Markov chain, the rate of convergence is n /4, which is similar to a result obtained by PP for {X t } processes that are of I) type. 3.2 Asymptotically homogeneous function on Θ We next establish an asymptotic theory for a parametric estimator of θ 0 when the derivatives of g, ) belong to a class of asymptotically homogeneous functions. For a unit root process {X t }, PP establish the asymptotic consistency and limit distribution of the NLS estimator θ n by using the local time technique. Their method relies on the linear framework of the unit root process, the functional limit theorem of the partial sum process, and the continuous mapping theorem. The Harris recurrent Markov chain is a general process and allows for a possibly nonlinear framework, however. In particular, the null recurrent Markov chain can be seen as a nonlinear generalisation of the linear unit root process. Hence, the techniques used by PP for establishing the asymptotic theory is not applicable in such a possibly nonlinear Markov chain framework. Meanwhile, as mentioned in Section, the methods used to prove Theorem 3. cannot be applied here directly because the asymptotically homogeneous functions usually are not bounded and integrable. This leads to the violation of the conditions in the ergodic theorem of the Harris recurrent Markov process. Hence, it is quite challenging to establish an asymptotic theory for the NLS estimator for the case of asymptotically homogeneous functions. To address the above concerns, we have to modify the NLS estimator θ n. Let be a positive and increasing sequence which satisfies as n, but is dominated by certain polynomial rate. We define the modified loss function by Q n,g θ) = n [Y t gx t, θ)] 2 I X t ). 3.8) t= The modified NLS MNLS) estimator θ n can be obtained by minimizing Q n,g θ) over θ Θ, θ n = arg min θ Θ Q n,gθ). 3.9) 3

15 As, for the integrable case discussed in Section 3., we can easily show that θ n has the same asymptotic distribution as θ n under some regularity conditions. In Example 6. below, we compare the finite sample performance of these two estimators, and find that they are quite similar. Let B i ) = [i, i), i =, 2,, [ ], B [Mn]+) = [[ ], ], B i 2) = [ i, i + ), i =, 2,, [ ], B [Mn ]+2) = [, [ ]], where [x] x is the largest integer part of x. It is easy to check that B i k), i =, 2,, [ ] +, k =, 2, are disjoint, and [, ] = 2 k= []+ i= B i k). Let Λ n = [ ] ḣ g i, θ 0 M )ḣτ g i, θ 0 )π s B i+ )) n i=0 [ ] + i=0 ḣ g i, θ 0 )ḣτ g i, θ 0 )π s B i+ 2)), where ḣg, ) will be defined in Assumption 3.3 i) below. Some additional assumptions are introduced below to establish asymptotic properties of θ n. Assumption 3.3. i) ġx, θ) is asymptotically homogeneous on Θ with asymptotic order κ g, ) and limit homogeneous function ḣg, ). gx, θ) is also asymptotically homogeneous on Θ with asymptotic order κ g, ) and limit homogeneous function ḧg, ). ii) Both ḣg, θ) and ḧg, θ) are continuous on the interval [, ] for θ in a neighborhood of θ 0. There exist a positive definite matrix Λ g and a sequence of positive numbers ζ g ) such that lim n ζ g ) Λ n = Λ g. iii) For all θ Θ, κ g i, θ) and κ g i, θ) are positive and nondecreasing such that κ g n, θ) and κ g n, θ) = O κ g n, θ)) as n. 4

16 iv) For each x [, ], N x ) := {y : x < y < x + } is a small set. Remark 3.4. Assumption 3.3 i) is quite standard, see, for example, condition b) in Theorem 5.2 in PP. The explicit forms of Λ g and ζ ) in Assumption 3.3 ii) can be derived for some particular cases. For example, when {X t } is generated by a positive recurrent Markov chain, we have Mn Λ n = + o)) ḣ g x, θ 0 M )ḣτ g x, θ 0 )π s dx) n + o)) = κ 2 ġx, θ 0 )ġ τ x, θ 0 )π s dx). g, θ 0 ) By letting ζ g ) = κ g 2, θ 0 ) and assuming that ġx, θ 0 )ġ τ x, θ 0 )π s dx) = ġx, θ 0 )ġ τ x, θ 0 )p s x)dx is positive definite, we can verify the second condition in Assumption 3.3 ii) for the positive recurrent case, where ζ g ) 0 as n. When {X t } is generated by a null recurrent Markov chain with π s dx) = ζ g )π s dx), we then have Mn Λ n = + o)) ḣ g x, θ 0 M )ḣτ g x, θ 0 )π s dx) n = + o)) = + o))ζ g ) ḣ g x, θ 0 )ḣτ gx, θ 0 )π s dx) ḣ g x, θ 0 )ḣτ gx, θ 0 )π s dx). By assuming that ḣgx, θ 0 )ḣτ gx, θ 0 )π s dx) is positive definite, we can verify the second condition in Assumption 3.3 ii) for the null recurrent case. when {X t } is generated by a random walk process, we have Λ n = + o)) [M n] i= [] ḣ g i, θ 0 )ḣτ g i, θ 0 ) = + o)) ḣ g x, θ 0 )ḣτ gx, θ 0 )π s dx), In particular, which implies that ζ g ) = in the second condition in Assumption 3.3 ii). In fact, for the random walk case, by modifying the proof of Lemma B.3 in Appendix B, 5

17 we need only to assume that there exists a δ > 0 such that δ δ ḣ g x, θ 0 )ḣτ gx, θ 0 )π s dx) = δ δ ḣ g x, θ 0 )ḣτ gx, θ 0 )dx is positive definite, which is much weaker than condition d) in Theorem 5.2 in PP where δ δ ḣgx, θ 0 )ḣτ gx, θ 0 )dx is assumed to be positive definite for all δ > 0. Define J g n, θ 0 ) = κ 2 g, θ 0 )ζ g )Λ g. We next establish an asymptotic theory for θ n. As for Theorem 3., the following theorem is also applicable for both the stationary and nonstationary time series cases. Theorem 3.2. Let Assumptions 3. and 3.3 hold. a) There exists a solution θ n to minimize the loss function Q n,g θ), and θ n θ 0 = o P ). 3.0) b) The estimator θ n has the asymptotic normal distribution, N /2 n)j /2 g n, θ 0 ) θ n θ 0 ) d N 0, σ 2 I d ), 3.) where I d is a d d identity matrix. Remark 3.5. From Theorem 3.2, the asymptotic distribution of θ n for the asymptotically homogeneous case is quite different from that of θ n and θ n ) for integrable case. Such finding is comparable to those in PP. When {X t } is positive recurrent, J g n, θ 0 ) is bounded away from zero and infinity, and the role of is the same as that of the sample size, which implies that the root-n consistency for the parametric estimator in the stationary time series case can be derived from our general asymptotic distribution theory in 3.). The choice of will be discussed in Corollaries 3.2 and 3.3 below. From 3.) in Theorem 3.2 and 2.3) in Section 2 above, we have the following two corollaries. The rate of convergence in 3.2) below is quite general for β-null 6

18 recurrent Markov processes. When β = /2, it is the same as the convergence rate in Theorem 5.2 of PP. Corollary 3.2. Suppose that the conditions in Theorem 3.2 are satisfied. Let {X t } be a β-null recurrent Markov chain with 0 < β < and bounded invariant density function p s ). Taking = M 0 n β L s n) for some positive constant M 0, we have θ n θ 0 = O P n κ 2 g, θ 0 )) /2). 3.2) Corollary 3.3. Suppose that the conditions in Theorem 3.2 are satisfied. Let gx, θ 0 ) = xθ, {X t } be a random walk process and = M 0 n /2 for some positive constant M 0. Then we have θ n θ = O P n ), 3.3) where θ n is the modified NLS estimator of θ. Furthermore, M0 3 n ) 3/2 d θ n θ N 0, 3σ 2 /2 ). 3.4) Remark 3.6. In the above two corollaries, we give the choice of for some special cases. In fact, for the random walk process {X t } defined as in Example 2., where {x t } is a sequence of i.i.d. random variables with zero mean and finite variance, we have X [ nr] = [nr] x i = Br), n n where Br) is a standard Brownian motion and = denotes the weak convergence. Furthermore, by the continuous mapping theorem see, for example, Billingsley 968), sup 0 r i= X [nr] = sup Br), n 0 r which implies that it is reasonable to let = C α n /2, where C α may be chosen such that α = P sup 0 r ) ) Br) > C α = P B) > C α = 2 Φ C α )), 3.5) 7

19 where Φx) = x 2π e u2 2 du. This implies that C α can be obtained when α is given, such as α = If {X t } is stationary and positive recurrent and its distribution has a compact support, say [ M 0, M 0 ], we can let M 0. In addition, for the simple linear regression model with regressors generated by a random walk process, 3.2) and 3.3) imply the existence of super consistency. Corollaries 3. and 3.2 imply that the rates of convergence for the parametric estimator in nonlinear cointegrating models rely not only on the properties of the function g, ), but also on the magnitude of β. 4 Nonlinear heteroskedastic regression In this section, we aim to relax the homoskedastic assumption on model.) and extend our methodology to the heteroskedastic case. We will introduce an estimation method for a parameter vector involved in the conditional variance function. As mentioned in Remark 3., we consider the following case Ee 2 t F n,t ) = σ 2 X t, γ 0 ) a.s. for γ 0 Υ R p, 4.) where σ 2, ) : R p+ R is positive, and γ 0 is the true value of the p-dimensional parameter vector involved in the conditional variance function. Estimation of the parametric nonlinear variance function defined in 4.) is important in empirical applications as many scientific studies depend on understanding the variability of the data. However, so far as we know, there is very little literature on the asymptotic analysis of nonlinear and nonstationary volatility regression PP only consider the parametric nonlinear regression with unit root regressors for the homoskedastic case). An interesting question is whether there exists a super-consistency result for the nonlinear heteroskedastic regression with null recurrent Markov chains. We will provide a partial answer to this question below. 8

20 For simplicity, we focus on the nonlinear and nonstationary heteroskedastic regression defined by Y t = σx t, γ 0 )e t, t =,, n, 4.2) where {e t } is a sequence satisfying Ee t F n,t ) = 0, Ee 2 t F n,t ) = a.s., 4.3) F nt = σ X s, e s2, s n, s 2 t). Letting ϖ 0 be a positive number such that E [ loge 2 t ) F n,t ] = logϖ0 ) a.s., we have logy 2 t ) = log σ 2 X t, γ 0 ) ) + loge 2 t ) = log σ 2 X t, γ 0 ) ) + logϖ 0 ) + loge 2 t ) logϖ 0 ) =: log ϖ 0 σ 2 X t, γ 0 ) ) + ζ t. 4.4) It is easy to see that Eζ t Fn,t ) = 0 a.s. Since our main interest lies in the discussion of the asymptotic theory for the estimator of γ 0, we first assume that ϖ 0 is known to simplify our discussion. Model 4.4) can be seen as another nonlinear regression model with parameter vector γ 0 to be estimated. The log-transformation would make data less skewed, and thus the resulting volatility estimator may be more efficient in terms of dealing with heavy-tailed {e t }. Such transformation has been commonly used to estimate the variability of the data in the stationary time series case see, for example, Peng and Yao 2003; Gao 2007; Chen et al 2009). However, any extension to Harris recurrent Markov chains which may be nonstationary) has not been done in the literature. Our estimation method will be constructed based on 4.4). Noting that ϖ 0 is assumed to be known, define σ 2 X t, γ 0 ) = ϖ 0 σ 2 X t, γ 0 ), g X t, γ 0 ) = logσ 2 X t, γ 0 )). Case i). If the derivatives of g X t, γ) are integrable on Υ, the log-transformed nonlinear least squares LNLS) estimator γ n can be obtained by minimizing L n,σ γ) 9

21 over γ Υ, where L n,σ γ) = n t= [ logy 2 t ) g X t, γ) ] ) Letting Assumptions 3. and 3.2 be satisfied with e t and g, ) replaced by ζ t and g, ), respectively, then the asymptotic results developed in Section 3. still hold for γ n. Case ii). If the derivatives of g X t, γ) are asymptotically homogeneous on Υ, the log-transformed modified nonlinear least squares LMNLS) estimator γ n can be obtained by minimizing Q n,σ γ) over γ Υ, where Q n,σ γ) = n t= [ logy 2 t ) log σ 2 X t, γ) )] 2 I Xt ), 4.6) where is defined as in Section 3.2. Then, the asymptotic results developed in Section 3.2 still hold for γ n under some regularity conditions such as the slightly modified version of Assumptions 3. and 3.3. Hence, it is possible to achieve the super-consistency result for γ n in this case. In particular, if g x, γ) γ = σ2 x, γ) σ 2 x, γ) equals a constant vector, where σ 2 x, γ) = σ2 x,γ), we can show that the n-rate can γ be achieved without the process itself being stationary. We next give some examples to show that different conditions on σ 2, ) would lead to different properties of g, ) in the above two cases. Example 4.. Let us consider a class of popular volatility functions of the form σ 2 x, γ) = x 2 + γ) q with q, γ > 0. Recall that σ 2 x, γ) = σ2 x,γ) γ and note that g x, γ) γ = σ2 x, γ) σ 2 x, γ) = q x 2 + γ is integrable. Hence, we have to use the estimation methodology introduced in Case i), although σ 2 x, γ) = x 2 + γ) q belongs to a class of asymptotically homogeneous functions when q > 0. 20

22 Example 4.2. Let σ 2 x, γ) = e x 2 γ) q with q, γ > 0, which is another class of popular volatility functions with an exponential rate of decaying to zero when x. In this case, we have g x, γ) γ = qx 2q γ q, which is asymptotically homogeneous. Hence, we have to use the estimation methodology introduced in Case ii), although σ 2 x, γ) = e x 2 γ) q belongs to a class of integrable functions. Similarly, if σ 2 x, γ) = e x 2 γ) q with q >, which belongs to a class of explosive functions see Park and Phillips 999 for definitions), we can also show that g x,γ) γ = qx 2q γ q is asymptotically homogeneous, which indicates that we need to apply the estimation methodology discussed in Case ii). In practice, however, ϖ 0 is usually unknown and needs to be estimated. We next briefly tackle this issue. By 4.4), define the loss function by Q n γ, ϖ) = n [ loge 2 t ) log ϖσ 2 X t, γ) )] 2 I Xt ). t= Then, the estimators γ n and ϖ n can be obtained by minimizing Q n γ, ϖ) over γ Υ and ϖ R +. 5 Nonlinear regression with I) processes As mentioned before, PP consider the nonlinear regression.) with the regressors {X t } generated by X t = X t + x t, x t = ϕ j ε t j, 5.) where {ε j } is a sequence of i.i.d. random variables and {ϕ j } satisfies some summability conditions. For simplicity, we assume that X 0 = 0 throughout this section. PP establish a suite of asymptotic results for the NLS estimator of the parameter θ 0 involved in.) when {X t } is defined by 5.). An open problem is how to establish some corresponding results by using the β-null recurrent Markov chain framework. 2 j=0

23 This is quite challenging as {X t } defined by 5.) is no longer a Markov process except for some special cases for example, ϕ j = 0 for j ). The aim of this section is to solve this open problem when g, ) is asymptotically homogeneous on Θ and derive an asymptotic theory for θ n by using Theorem 3.2. The discussion for the integrable case is more complicated, and will be considered in a future study. We first introduce some regularity conditions for the establishment of the asymptotic theory. Assumption 5.. i) Let {ϕ j } satisfy the conditions ϕ j <, ϕ = j=0 for some δ > 0. ϕ j 0, j=0 j=s+ ϕ j ) 2 = Os +δ) ) 5.2) ii) {ε t } is a sequence of i.i.d. random variables with zero mean and finite variance. iii) The error process {e t } involved in model.) is a sequence satisfying Ee t G n,t ) = 0, Ee 2 t G n,t ) = σ 2 < a.s., where G n,t = σ ε s, e s2, s n, s 2 t). iv) N x ) is a small set for the random walk process {X t } defined by X t = ϕ t s= ε s, x [, ]. Assumption 5.2. i) Assumptions 3.3 i) and iii) are satisfied. Furthermore, both ḣ g, θ) and ḧg, θ) are continuous on the interval [, ] for θ in a neighborhood of θ 0, and the matrix ḣgx, θ 0 )ḣτ gx, θ 0 )dx is positive definite ii) Uniformly for θ Θ, we have ġx + y, θ) ġx, θ) Bġx, θ)y 5.3) and gx + y, θ) gx, θ) B g x, θ)y 5.4) 22

24 where Bġx, θ) and B g x, θ) are asymptotically homogeneous with asymptotic orders κ B, ) and κ B2, ), respectively. Let κ B, ) and κ B2, ) satisfy κ 2 B, θ) ) κ 2 g, θ) = O n 2+δ, κ 2 B 2, θ) ) κ 2 g, θ) = O n 2+δ 5.5) uniformly for θ Θ, where κ g, ) is defined in Assumption 3.3 i). Remark 5.. Assumption 5. contains some mild conditions on the linear process {x t } as well as on the error term {e t }. The conditions in 5.2) ensure that {x t } is shortrange dependent see, for example, Wang and Phillips 2009, Wang and Chan 20), and it can be satisfied by some popular time series models such as an autoregressive model. Assumption 5.2 states some smoothness conditions on g, ) as well as its derivatives. In particular, when ϕ j decays with an exponential rate, δ can be taken arbitrarily large, which implies that condition 5.5) is quite weak. Define J g n, θ 0 ) = κ 2 g, θ 0 ) ) ḣ g x, θ 0 )ḣτ gx, θ 0 )dx. 5.6) We next give some asymptotic results for θ n in the case where {X t } is a unit root process. Theorem 5. Let Assumptions 5. and 5.2 hold, and n 22+δ) Mn. a) There exists a solution θ n to minimize the loss function Q n,g θ) and θ n θ 0 = o P ). 5.7) b) The estimator θ n has the asymptotic normal distribution, Nε /2 n)j /2 g n, θ 0 ) ) d θ n θ 0 N ) 0, σ 2 I d, 5.8) where N ε n) is the number of regenerations for the random walk process {Xt }. Remark 5.2. Theorem 5. establishes an asymptotic theory for θ n for the case where {X t } is a unit root process. Our results are comparable with Theorems 5.2 and 5.3 in PP. However, we establish asymptotic normality in 5.7) with stochastic rate 23

25 N /2 ε n)j /2 g n, θ 0 ), and PP establish their asymptotic mixed normal distribution theory with a deterministic rate. As N /2 ε n)j /2 g n, θ 0 ) nj /2 g n, θ 0 ) in probability, if we take = M 0 n as in Corollary 3.3, we will find that our rate of convergence of θ n is the same as that derived by PP. 6 Simulated examples In this section, we provide some simulation studies to compare the finite sample performance of various parametric estimation methods introduced in Sections 3 5 and to illustrate the developed asymptotic theory. Example 6.. Consider the generalised linear model defined by Y t = exp{ θ 0 Xt 2 } + e t, θ 0 =, t =, 2,, n, 6.) where {X t } is generated by one of the three Markov processes: i) AR) process: X t = 0.5X t + x t, where X 0 = 0 and {x t } is a sequence of i.i.d. standard normal random variables; ii) Random walk process: X t = X t + x t, where X 0 = 0 and {x t } is a sequence of i.i.d. standard normal random variables; iii) TAR) process: X t = 0.5X t I X t ) + X t I X t > ) + x t, where X 0 = 0 and {x t } is a sequence of i.i.d. standard normal random variables. The error process {e t } is a sequence of i.i.d. N0, ) random variables and independent of {x t }. In this simulation study, we compare the finite sample behavior of the NLS estimator θ n with that of the MNLS estimator θ n, and the sample size n is chosen to be 500, 000 and The aim of this example is to illustrate the asymptotic theory developed in Section 3. as the regression function in 6.) is integrable when θ 0 0. Following the discussion in Section 2.2, the AR) process defined in i) is positive recurrent, and the random process defined in ii) and the TAR) process defined in iii) are /2-null recurrent. 24

26 We generate 500 replicated samples for this simulation study, and calculate the means and standard errors for both of the parametric estimators in 500 simulations. In the MNLS estimation procedure, we choose = C α n β with α = 0.0, where C α is defined in 3.5), β = for case i), and β = /2 for cases ii) and iii). It is easy to find that C 0.0 = The simulation results are reported in Table 6., where the numbers in the parentheses are the standard errors of the NLS or MNLS) estimator in 500 replications. From Table 6., we have the following interesting findings. a) The parametric estimators perform better in the stationary case i) than in the nonstationary cases ii) and iii). This is consistent with the asymptotic results obtained in Section 3. such as Theorem 3. and Corollary 3., which indicate that the convergence rates of the parametric estimators can achieve O P n) in the stationary case, but only O P n /4 ) in the /2-null recurrent case. b) The finite sample behavior of the MNLS estimator is exactly the same as that of NLS estimator since α = 0.0 means little sample information is lost. c) Both of the two parametric estimators improve as the sample size increases. d) In addition, for case i), the ratio of the standard errors between 500 and 2000 is.9633 close to the theoretical ratio 4 = 2); for case iii), the ratio of the standard errors between 500 and 2000 is.3005 close to the theoretical ratio 4 /4 =.442). Hence, this again confirms that our asymptotic theory is valid. 25

27 Table 6.. Means and standard errors for the estimators in Example 6. sample size The regressor X t is generated in case i) NLS ) ) ) MNLS ) ) ) The regressor X t is generated in case ii) NLS ) ) ) MNLS ) ) ) The regressor X t is generated in case iii) NLS ) ) ) MNLS ) ) ) Example 6.2. Consider the linear model defined by Y t = + Xt 2 ) θ 0 + e t, θ 0 = 0.5, t =, 2,, n, 6.2) where {X t } is generated either by one of the three Markov processes introduced in Example 6., or by iv) the unit root process: X t = X t + x t, x t = 0.2x t + v t, in which X 0 = x 0 = 0, {v t } is a sequence of i.i.d. N0, 0.75) random variables, and the error process {e t } is defined as in Example 6.. In this simulation study, we are interested in the finite sample behavior of the MNLS estimator to illustrate the asymptotic theory developed in Sections 3.2 and 5 as the regression function in 6.2) is asymptotically homogeneous. For the comparison purpose, we also investigate the finite sample behavior of the NLS estimation, although we do not establish the related asymptotic theory under the framework of null recurrent Markov chains. The sample size n is chosen to be 500, 000 and 2000 as in Example 6. and the replication number is R = 500. In the MNLS estimation procedure, as in the previous example, we choose = 2.58n β, where β = for case i), and β = /2 for cases ii) iv). 26

28 Table 6.2. Means and standard errors for the estimators in Example 6.2 sample size The regressor X t is generated in case i) NLS ) ) ) MNLS ) ) ) The regressor X t is generated in case ii) NLS ) ) ) MNLS ) ) ) The regressor X t is generated in case iii) NLS ) ) ) MNLS ) ) ) The regressor X t is generated in case iv) NLS ) ) ) MNLS ) ) ) The simulation results are reported in Table 6.2, from which, we have the following conclusions. a) For the nonlinear cointegration model with asymptotically homogeneous regression function, the parametric estimators perform better in the nonstationary cases ii) and iii) than in the stationary case i). This finding is consistent with the asymptotic results obtained in Section 3.2 such as Theorem 3.2 and Corollaries 3.2 and 3.3, which indicate that the convergence rates of the parametric estimators can achieve O P n) in the /2-null recurrent case, faster than the corresponding rates in the stationary case. b) For the case of asymptotically homogeneous regression function, our MNLS estimator performs as well as the NLS estimator. c) Both the NLS estimation procedure proposed by PP and our MNLS estimation procedure perform poorly for the non-markov unit root process iv), which needs to be further studied in future research. 27

29 Example 6.3. Consider a heteroskedastic model of the form: Y t = σx t, γ 0 ) e t, t =, 2,, n, 6.3) where we consider the following two forms: Form i): σx, γ 0 ) = x 2 + γ 0 with γ 0 = 0.2; x Form ii): σx, γ 0 ) = e 2 γ 0 with γ0 = 0.2, {X t } is generated by one of the three Markov processes introduced in Example 6., e t = ν t η t, {η t } is a sequence of i.i.d. random variables with logη 2 t ) U, ), {ν t } is also i.i.d. and independent of {η t } with Pν t = ) = Pν t = ) = /2. It is easy to see that E[e t ] = 0 and logϖ 0 ) = 0, where ϖ 0 is defined as in Section 4. In this simulation study, as discussed in Section 4, we are interested in the finite sample behaviors of the LNLS estimator γ n for Form i) and the LMNLS estimator γ n for Form ii). The sample size n is chosen to be 500, 000 and 2000 as in the above two examples and the replication number is R = 500. Following the analysis in Example 6.2, in the parametric estimation procedure, we choose = 2.58n β. Table 6.3. Means and standard errors for the estimators in Example 6.3 Estimation n = 500 n = 000 n = 2000 The regressor X t is generated in case i) LNLS γ n ) ) ) LMNLS γ n ) ) ) The regressor X t is generated in case ii) LNLS γ n ) ) ) LMNLS γ n ) ) ) The regressor X t is generated in case iii) LNLS γ n ) ) ) LMNLS γ n ) ) ) 28

30 The simulation results are reported in Table 6.3, from which we can obtain the following conclusions. a) When the conditional volatility function is defined as in Form i), the LNLS estimator γ n performs much better in stationary case i), as there is no super-consistency result in nonstationary cases ii) and iii) see Example 4. for detailed discussion). In addition, for the nonstationary cases ii) and iii), the standard errors of γ n are much smaller than those of γ n. b) When the conditional volatility function is defined as in Form ii), the LWNLS estimator γ n performs much better in the nonstationary case because of the existence of the super-consistency result discussed in Section 4. 7 Conclusion In this paper, we have systematically studied the nonlinear regression under the general Harris recurrent Markov chain framework, which includes both the stationary and nonstationary cases. Note that the nonstationary null recurrent process considered in this paper is under Markov perspective, which indicates that our methodology has the potential of being extended to the nonlinear autoregressive case. A recent paper by Chan and Wang 202) also consider a nonlinear cointegrating regression with nonstationary regressors. However, when the regressors are Harris recurrent, they cannot derive the asymptotic distribution theory for the asymptotically homogeneous case. In this paper, we not only develop asymptotic theory for the estimator of θ 0 when g, ) is integrable, but also modify the conventional NLS estimator for the asymptotically homogeneous g, ) and use a novel method to establish asymptotic theory for the proposed modified parametric estimator. Furthermore, by using the log-transformation, we discuss the estimation of the parameter vector in a conditional volatility function and establish its asymptotic property. Finally, we apply our results to the nonlinear regression with I) processes which may be non-markovian, and establish an asymptotic distribution theory which is comparable to that obtained by 29

31 PP. Some simulation studies are provided to illustrate our approaches and results. A Proofs of the main results In this appendix, we give the detailed proofs of the main results. Proof of Theorem 3.. For a), it suffices to prove that for ϵ > 0, ) lim P inf L nθ) L n θ 0 ) =, A.) n θ S θ0 ϵ) where S θ0 ϵ) is a circle centered at θ 0 with radius ϵ. In fact, A.) implies that there exists a minimum θ n in the interior of S θ0 ϵ), and ) P θ n θ 0 ϵ, which leads to 3.4). By a second order Taylor expansion of L n θ), we have L n θ) = L n θ 0 ) + L τ nθ 0 )θ θ 0 ) + 2 θ θ 0) τ Ln θ )θ θ 0 ), A.2) where θ lies in the line segment connecting θ and θ 0, n L n θ) = 2 ġx t, θ) [ Y t gx t, θ) ], t= { n L n θ) = 2 ġx t, θ)ġ τ X t, θ) t= t= n gx t, θ) [ Y t gx t, θ) ]}. By B.4) in Lemma B.2 in Appendix B, using the martingale difference structure on {e t } in Assumption 3. ii) and the integrability condition in Assumption 3.2, we can show that By B.3) in Lemma B.2, we have L n θ 0 ) = O P ) = o P ). n ġx t, θ )ġ τ X t, θ ) = t= ġx, θ )ġ τ x, θ )π s dx) + o P ). On the other hand, by Definition 2.3 and Lemma B.2, we also have n gx t, θ ) [ Y t gx t, θ ) ] = OP ϵ), t= A.3) A.4) A.5) 30

32 where ϵ is the same as that in A.). Let λ min > 0 be the smallest eigenvalue of the positive definite matrix Lθ 0 ), where Lθ 0 ) = ġx, θ 0 )ġ τ x, θ 0 )π s dx) is defined as in Assumption 3.2. By A.4) and A.5), for θ located on the circle S θ0 ϵ), we have P inf θ θ 0) τ Ln θ )θ θ 0 ) > θ S θ0 ϵ) 2 λ minϵ 2) =. A.6) Then A.2), A.3) and A.6) imply that A.) holds. We next turn to the proof of part b) in Theorem 3.. By some elementary calculations as above, we have θ n θ 0 = L + n θ 0 ) L n θ 0 ) + o P )), A.7) where L + n θ 0 ) is the Moore-Penrose inverse matrix of L n θ 0 ). By B.3) and B.4) in Lemma B.2, we have 2 L n θ 0 ) = Lθ 0 ) + o P ). A.8) By A.7) and A.8), to prove b), we need only to prove that 2 d L n θ 0 ) N 0, Σ θ 0 )), A.9) where Σ θ 0 ) = σ 2 Lθ0 ). By the so-called Cramér-Wold device, to prove A.9), it suffices to prove 2 ατ d Ln θ 0 ) N 0, α τ Σ θ 0 )α), A.0) where α = α,, α d ) τ. Letting Z nt = α τ ġx t, θ 0 )e t, F nt = σx s, e s, s n, s 2 t) in Lemma B., it is easy to check that n E ZntI Z 2 ) nt > ϵ) F n,t t= n [ ατ ġx t, θ 0 ) ] 2 E e 2 t I { e 2 t > ϵ [ α τ ġx t, θ 0 ) ] 2} ) t= = o P n [ ατ ġx t, θ 0 ) ] ) 2 = o P ), t= A.) as α τ ġx, θ)) 2 is π s -integrable. 3