Nonparametric regression with rescaled time series errors

Transcription

1 Nonparametric regression with rescaled time series errors José E. Figueroa-López Michael Levine Abstract We consider a heteroscedastic nonparametric regression model with an autoregressive error process of finite known order p. he heteroscedasticity is incorporated using a scaling function defined at uniformly spaced design points on an interval [0,]. We provide an innovative nonparametric estimator of the variance function and establish its consistency and asymptotic normality. We also propose a semiparametric estimator for the vector of autoregressive error process coefficients that is consistent and asymptotically normal for a sample size. Explicit asymptotic variance covariance matrix is obtained as well. Finally, the finite sample performance of the proposed method is tested in simulations. Keywords and phrases: autoregressive error process; heteroscedastic; semiparametric estimators; difference-based estimation approach. Introduction In this manuscript, we consider the estimation of a time series process with a time-dependent conditional variance function and serially dependent errors. More precisely, we assume that there are observations {(x t, y t )} t {,..., } available that have been generated by the following model: y t = σ t v t, σ t = σ(x t ), (.) p v t = φ j v t j + ε t, (.) j= where {ε t : < t < } are independent identically distributed with mean 0 and variance, while the autoregressive order is a fixed known integer p > 0. We also assume that x t s form an increasing equally spaced sequence on the interval [0, ]. hen, the model (.)-(.) can be viewed as a nonparametric regression model with the mean function identically equal to zero and scaled autoregressive time series errors v t. he simplest case of p = was treated earlier in Dahl and Levine(006). Another interpretation of this model would be as a type of functional autoregressive models (FAR) that were first introduced in Chen and say (993). Indeed, the process y t can be re-expressed as y t = p j= φ j σ t σ t j y t j + σ t ε t, (.3) with functional coefficients being φ j σ t σt j, j =,..., p. Nonparametric regression models with autoregressive time series errors have a long history. Hall and Van Keilegom (003) and, more recently, Shao and Yang (0) considered a general nonparametric model of the form y t = µ t + σ t v t, (.4)

2 with a regression mean process µ t given by µ t = g(x t ) for a smooth trend function g, an error process {v t : < t < } given as in (.), and an error scaling function σ t given by σ t σ, for a (possibly unknown) constant σ. Note that our model has a more general correlation structure since it rescales the AR(p) error process by the conditional variance of the unobserved process σ(x t ). Phillips and Xu (005) also considered the model (.4) with a general scaling function σ t = σ(x t ), but with the autoregressive mean process structure µ t = θ 0 + θ y t θ q y t q and with zero mean martingale difference sequence as an error process {v t : < t < }. As mentioned above, the model considered in this manuscript is an extension of that studied in Dahl and Levine (006), who treated the particular case p =. In the present manuscript, we provide an innovative nonparametric estimator of the variance function σ(x) and establish its consistency and asymptotic normality. We also propose a semiparametric estimator for the autoregressive error process coefficients (φ,..., φ p ), which is consistent and asymptotically normal. he estimators of the error covariance structure (determined by σ(x t ) and (φ,..., φ p )) are needed, for example, in order to estimate the variance of the regression mean function µ t. Another possible application is in using bootstrap methods when constructing confidence bands for the regression mean. For some discussion on this subject see, for example, Hall and Van Keilegom (003). Our results can also be used for testing the martingale difference sequence hypothesis H 0 : φ = = φ p = 0, that is often uncritically adopted in financial time series, against a fairly general alternative. Our estimation approach is based on the two-lag difference statistics (pseudoresiduals): η t = y t y t. (.5) Both Hall and Van Keilegom (003) and Dahl and Levine (006) also use two-lag difference statistics but the method in the former paper does not apply in the presence of a non-constant scaling function σ(x t ) (see Remark 3.5 below), while the method in the second paper relies heavily on the autocorrelation properties of an AR() process and, thus, cannot deal with higher order autocorrelation models for {v t : < t < } (see justification in the paragraph after Eq. (.4)). For simplicity, we mostly focus on the case of a regression mean µ t which is identically equal to zero since our main goal is the estimation of the covariance structure of the process (.)-(.). Nevertheless, we also analyze the effect of a nonzero mean function and propose a natural correction method in this case (see end of Section 3 for more details). We also show that the addition of a sufficiently smooth non-zero mean function has no effect on the asymptotic properties of ˆφ. his fact is explained in more details in Corollary 3.6; see also Hall and Van Keilegom (003) for some additional heuristics on this subject. he rest of the paper is organized as follows. In Section, we present our estimation approach. he consistency and central limit theorem for estimators of autoregressive coefficients φ and φ are given in Section 3. he analogous results for the estimator of the variance function σ(x) are presented in Section 4. A Monte Carlo simulation study and a real data application of our estimators are then presented in Section 5. In Section 6, we conclude the paper with a discussion section about extensions of our method and some interesting open problems. he proofs of main results are given in the Appendix section. Estimation method based on two-lag differences In this section, we present our estimation approach. For simplicity, we first illustrate our method for p = ; however, it will work for any finite order p as we explain further at the end of the present section. As it was mentioned in the introduction, Dahl and Levine (006) proposed an estimation method for the model (.)-(.) in the simple case of p = based on the two-lag difference statistics η t defined in (.5). In order to explain in simple terms the intuition behind our method, we recall the approach therein.

3 Suppose for now that the scaling function σ t is constant; i.e. σ t σ. We denote γ k the autocovariance of the error process v t at lag k. hen, note that ηt ( = σ v t + vt v ) tv t and, therefore, because, under the AR() specification, E η t = σ (γ 0 γ ) σ, (.) γ = φ γ = φ φ. It is now intuitive that η t can be used to develop a consistent estimator for a non-constant function σ t as well. Indeed, in the case of non-constant σ t and under a fixed design on the unit interval (i.e., x t := t/ for t =,..., ), we have E η t = ( σ t γ 0 + σ t γ 0 σ t σ t γ ). Simple heuristics suggest that the above expression can be accurately approximated by σ t in large samples for sufficiently large. his, in turn, suggests turning the original problem (.) into a non-parametric regression η t = σ (x t ) + ε t, (.) where { ε t } t=,..., are approximately centered random errors. Dahl and Levine (006) used a local linear estimator ˆσ t to estimate σ t := σ (x t ), while the parameter φ was estimated using a weighted least square estimator (WLSE). More specifically, noting that (.3) with φ = = φ p = 0 implies it follows that a natural estimator for φ is given by σt y t = φ σt y t + ε t, t =,...,, (.3) ˆφ := arg min φ (,) ( = ˆσ t y t t= t= ) ( (ˆσ t y t φ ˆσ t y ) t t= ) ˆσ t ˆσ t y ty t, (.4) where above {ˆσ t } t=,..., is the local linear estimator of σ (x t ) based on non-parametric model (.). he approach described in the previous paragraph will not work for any p > because, in that case, γ 0 γ and, therefore, even the constant variance function σ cannot be estimated consistently. A first natural idea to extend the method in Dahl and Levine (006) is to look for a linear statistic η t := m a i y t i, (.5) i=0 such that E η t σ t, for sufficiently large sample size. he following result shows that this is essentially impossible even for the simplest AR() case. he impossibility for a general AR(p) model will immediately follow since obviously the AR() model can be seen as a degenerate case of the general AR(p) model. he proof of the following result is deferred to the appendix section. 3

4 Proposition.. Consider again the case where σt σ is constant and the error process is AR() (i.e., φ = = φ p = 0 in (.)-(.)). hen, if Eηt = σ for any φ (, ), there exists a 0 k m such that a k = ±, a k+ =, a i = 0, i k, k +. he previous result shows that the only linear statistic (.5) with a 0 0 that can result in Eη t being independent of φ is the two-lag difference statistic η t = y t y t. Now we show that the expectation of ηt is indeed independent of φ and introduce our new method to estimate the model (.)-(.) with an AR() error process. Recall that (see, for example, Brockwell and Davis (99)) for a stationary AR() process v t, its variance and autocovariances are: φ γ 0 := Var(v k ) = ( + φ )(( φ ) φ (.6) ), γ := Cov(v k, v k+ ) = φ γ 0, φ (.7) γ j := Cov(v k, v k+j ) = φ γ j + φ γ j, for any k N, j. (.8) herefore, the mean of squared pseudoresidual of the nd order, η t, can be simplified as follows under the assumption of constant variance function σ t σ: E η t = σ (γ 0 γ ) = σ + φ. Indeed, using (.6)-(.7), ( ) φ γ = φ γ + φ γ 0 = + φ γ 0 = φ + ( φ )φ γ 0, φ φ ( γ 0 γ = γ 0 φ + ( φ ) ( )φ ( φ ) φ ) = γ 0 φ φ ( ) ( φ ( φ ) φ ) = ( + φ )(( φ ) φ ) =. φ + φ As before, for a general smooth enough function σt and under a fixed design on the unit interval (x t = t/, t =,..., ) with large enough, we expect that E η t σ t + φ, and, hence, we expect to estimate correctly σt up to a constant. It turns out that this will suffice to estimate the parameters φ and φ via weighted least squares (WLSE). Indeed, suppose for now that we know the variance function σt and let ȳ t := σt y t. In light of the relationship (.3), it would then be possible to estimate (φ, φ ) by the WLSE: ( φ, φ ) := arg min φ,φ (ȳ t φ ȳ t φ ȳ t ). 4

5 Basic differentiation leads to the following system of normal equations ȳ t ȳ t + φ ȳ t ȳ t + φ ȳ t + φ ȳ t y t + φ ȳ t ȳ t = 0, ȳ t = 0. Ignoring negligible edge effects (so that ȳtȳ t ȳt ȳ t and ȳ t write the above system as with We finally obtain Ā := Ā φ + B φ B = 0, B φ + Ā φ C = 0, ȳt, B := ȳ t ȳ t, C := ȳ t ȳ t. ȳ t ), we can φ := (Ā B ) (Ā C B ), φ = Ā B( ˆφ ). (.9) Obviously, these estimators are not feasible since σ t is unknown. However, we note that these estimators will not change if instead of σ t in the definition of ȳ t, we use cσ t where c is an arbitrary constant that is independent of t. his fact suggests the following two-step estimation method for the scaling function σ t = σ(x t ) and the autocorrelation coefficients (φ, φ ): Algorithm. (AR() case).. First, estimate the function σ,bias t := σ (x t ) + φ, (.0) by a non-parametric smoothing method (e.g. local linear regression) applied to the two-lag difference statistics η t defined in (.5). Let σ t be the resulting estimator.. Standardize the observations, ỹ t := σ t y t, and, then, estimate (φ, φ ) via the WLSE: ˆφ := (A B ) (AC B ), ˆφ = A B( ˆφ ). (.) with 3. Estimate σ t := σ (x t ) by A := ỹt, B := ỹ t ỹ t, C := ỹ t ỹ t. ˆσ t := ( + ˆφ ) σ t. (.) he same method can be easily extended to the case of an arbitrary autoregressive error process AR(p) with p >. Indeed, let φ,..., φ p be the coefficients of the AR(p) error process. It can be shown that, for large, the expectation of the squared pseudoresidual of order, η t, is approximately equal to the scaled value of σ t : E η t Ψ σ t, where the scaling constant Ψ is an explicit function Ψ Ψ(φ,..., φ p ) of φ,..., φ p. As with the case of p =, since the scaling constant does not depend on the variance function, the following natural extended procedure follows: 5

6 Algorithm. (General AR(p) case).. Obtain an estimate of the scaled variance function σ,bias t := σ,bias (x t ) := Ψ(φ,..., φ p )σ (x t ), by using a non-parametric smoothing method (e.g., local linear regression) applied to η t. As before, let σ t = σ (x t ) be the resulting estimator.. Standardize the observations ỹ t := σ t y t and then estimate (φ,..., φ p ) using the weighted least squares (WLSE): ( ˆφ,..., ˆφ p ) := arg min φ,...,φ p t=p+ (ỹ t φ ỹ t... φ p ỹ t p ). 3. Estimate σ t := σ (x t ) by ˆσ t := Ψ( ˆφ,..., ˆφ p ) σ t. (.3) In the next section, we will give a detailed analysis of the consistency and asymptotic properties of the proposed estimators. 3 Asymptotics Let us now consider the asymptotic properties of the estimation procedure described at the end of the previous section in the context of the general heteroscedastic process (.-.). We will use σ t to denote the inconsistent estimator of σt that is obtained by applying local linear regression to the squared-pseudoresiduals ηt. As explained above, such an estimator is inconsistent since, e.g., even for the homoscedastic model (i.e., σt σ ), Eηt = σ Ψ(φ,..., φ p ). However, note that it is expected to be a consistent estimator of the quantity σ,bias t = σt Ψ(φ,..., φ p ), as it will be formally proved in Section 4 below. hroughout, we denote σ = (σ,..., σ ), σ = ( σ,..., σ ), σ bias = (σ bias,..., σ bias ). (3.) As explained before, it seems reasonable to estimate the coefficients φ, φ,..., φ p using an inconsistent estimator σ t first and, then, correct it to obtain the asymptotically consistent estimator: ˆσ t = Ψ( ˆφ,..., ˆφ p ) σ t. he following detailed algorithm illustrates our approach to obtaining the asymptotic properties of the proposed estimators:. Using the functional autoregression form of the model (.), define the least squares estimator ˆφ := ( ˆφ,..., ˆφ p ) of φ := (φ,..., φ p ) and establish its consistency (i.e. ˆφ P φ as ), under additional conditions.. Define an asymptotically consistent estimator ˆσ t and establish its consistency and asymptotic normality. 6

7 For convenience, we recall here the functional autoregressive form of (.): σt y t = φ σt y t φ p σt p y t p + ε t. Next, for any ϑ = (ϑ,..., ϑ ) and ϕ = (ϕ,..., ϕ p ), let m k (ϑ; ϕ) := m k,t (ϑ; ϕ), (k =,,..., p), (3.) t=p where m k,t (ϑ; ϕ) := ϑ t k y t k[ϑ t y t ϑ t ϕ y t... ϑ t p ϕ py t p ], (3.3) where (y t ) is generated by the model (.) with true parameters (σ t, φ). Denote Note that and, therefore, none of them depends on {σ t }. least-square estimator ˆφ of φ are given by m t (ϑ; ϕ) := (m,t (ϑ; ϕ);... ; m p,t (ϑ; ϕ)). (3.4) m k,t (σ; φ) = v t k ε t, (k =,..., p), (3.5) hen, the first order conditions that determine the m k ( σ; ˆφ) = 0, (k =,..., p). (3.6) A few preliminary results are needed to establish consistency for ˆφ. hroughout, we assume that the data generating process (.)-(.) satisfies the following conditions:. {ε t } are independent identically distributed (i.i.d.) errors with mean zero and variance.. E ε t 4+γ < for some small γ > σ ( ) F := C [0, ], the class of continuous functions on [0, ] with continuous second-derivatives in (0, ). We denote Θ 0 the set of φ = (φ,..., φ p ) such that the roots of the characteristic equation φ z... φ p z p = 0 are greater than + δ in absolute value for some δ > 0. Such a condition on φ guarantees causality and stationarity of the AR(p) error process v t defined in (.); moreover, it also implies that v t can be represented as the MA( ) process v t = ψ i ε t i, (3.7) i=0 where all of the coefficients {ψ i } i 0 satisfy K ρ i ψ i K ρ i, (3.8) for some finite ρ such that +δ < ρ <, and positive constants K and K (see, e.g., Brockwell and Davis (99), Chapter 3). In particular, (3.8) implies that the series {ψ i } i 0 is absolutely converging: i=0 ψ i <. 7

8 Remark 3.. he coefficients {ψ i } i 0 satisfy the recursive system of equations ψ j φ k ψ j k = 0, (3.9) 0<k j with ψ 0 and φ k = 0 for k > p. Note that this implies (by induction) that ψ j is a continuously differentiable function of φ for any j 0. Our first task is to show the weak consistency of the least-square estimator ˆφ. As pointed earlier in Dahl and Levine (006), the estimator ˆφ is an example of a MINPIN semiparametric estimator (i.e., an estimator that minimizes a criterion function that may depend on a Preliminary Infinite Dimensional Nuisance Parameter estimator). MINPIN estimators have been discussed in great generality in Andrews (994). We first establish the following uniform weak law of large numbers (LLN) for m t (σ; φ) (see Andrews (987), Appendix A for the definition). Lemma 3.. Suppose that Θ Θ 0 is a compact set with non-empty interior. hen, as, P sup m t (σ; φ) 0. φ Θ t=p We are ready to show our main consistency result. he proof is presented in Appendix A. heorem 3.3. Let Θ be a compact subset of Θ 0 with non-empty interior. hen, the least squares estimator ˆφ converges to the true φ in probability. Our next task is to establish asymptotic normality of ˆφ. heorem 3.4. Let all of the assumptions of heorem 3.3 hold and, in addition,. m t (ϑ; φ) is twice continuously differentiable in φ for any fixed ϑ;. he matrix M lim t=p E m t (σ; φ) φ exists uniformly over F Θ and is continuous at (σ bias, φ) with respect to any pseudo-metric on F Θ for which ( σ, ˆφ) (σ bias, φ). Furthermore, the matrix M is invertible. hen, (ˆφ φ) d N(0, V ) where V := Γ with Γ := [γ i j ] i,j=,...,p = γ 0 γ γ... γ p γ p γ γ 0 γ... γ p 3 γ p γ p γ p γ p 3... γ γ 0. (3.0) For simplicity, the proof, that can be found in the Appendix A, is only shown for the case p = ; in that case, the covariance matrix takes the form: ( ) ( V V V := φ := ) φ ( + φ ) V V φ ( + φ ) φ. (3.) 8

9 Remark 3.5. he following observations are appropriate at this point:. he matrix Γ of (3.0) is non-singular if γ 0 > 0 and γ l 0 as l (see, e.g. Brockwell and Davis (99), Proposition 5..). his means that it is non-singular for any causal AR(p) process.. In the more general model (.4) with smooth mean function µ t = g(x t ) and constant scaling function σ t σ (possibly unknown), Hall and Van Keilegom (003) and Shao and Yang (0) proposed estimators for the autocorrelation coefficients that are / consistent with the same asymptotic variance covariance matrix as in (3.0). Hence, despite the fact that the scaling factor σ(x t ) may be non-constant in our setting, our estimators of autoregressive coefficients possess the parametric rate of convergence as if the conditional variance was known. 3. It is also important to emphasize that the methods in Hall and Van Keilegom (003) won t apply in the presence of a varying scaling function σ t = σ(x t ). Indeed, the method therein first provides the following estimates for the autocovariances of the error process: ˆγ 0 := ˆγ j := ˆγ 0 m m + ( j) m ( m) m=m i=j+ i=m+ {(D m y) i }, (3.) {(D j y) i }, (3.3) where (D j y) i := y i y i j and, as before, is the number of observation. Here, m, m are positive integer-valued sequences converging to in such a way that m m, m / log, and m = o( / ). hen, the method proceeds to estimate the coefficients φ := (φ,..., φ p ) using Yule-Walker equations Γφ = γ (see Section 8. in Brockwell and Davis (99)): ( ˆφ,..., ˆφ p ) := ˆΓ (ˆγ,..., ˆγ p ), where ˆΓ is the p p matrix having ˆγ i j as its (i, j) th entry (i, j =,..., p). Such an approach clearly will not work if the scaling function σ t is not constant. 4. Shao and Yang s (0) approach consists of first estimating the unknown mean function g(x t ) using a B-spline smoother ĝ(x t ) and, then, constructing estimators for the coefficients ( ˆφ,..., ˆφ p ) using the estimated residuals r t := y t ĝ(x t ), via the Yule-Walker equations as in Hall and Van Keilegom (003). he autocovariances γ j are estimated using the sample autocovariances ˆγ j := k t= r tr t+k. Again, the Yule-Walker estimation approach will not work if σ t is not constant. It is interesting to ask ourselves as to what happens if we consider the more general model (.4) with a non-trivial mean function µ t = g(x t ). In this case, in order to estimate φ, the following algorithm is an option: Algorithm 3... Estimate the mean function µ t using, say, a local linear regression method applied to the observations {y t } t {,..., } (see Section 4 below for more details on this method). Let ˆµ t be the resulting estimator.. Compute the centered observations ŷ t = y t ˆµ t and define the new pseudoresiduals ˆη t = ŷt ŷ t. 3. Again, obtain a biased estimator of σ t applying local linear regression to ˆη t and then follow (.) and (.). 9

10 It turns out that, if the mean function µ t can be estimated consistently, φ can be estimated at the same rate of convergence as before. Note that the mean function µ t = g( t ) here depends on the number of observations. his helps us separate trend, or a low-frequency component, represented by the µ t, from the zero-mean error process, that plays the role of the high-frequency component. A more precise formulation is given in the following result, whose proof is outlined in the Appendix A. Corollary 3.6. Let us assume that g ( ) exists and is continuous on (0, ). Also, let K be a kernel function with bounded support that integrates up to (i.e. it is a proper density) and whose first moment is equal to 0. Select a sequence h := h such that h and h 0, and let ˆµ t be the corresponding sequence of local linear estimators that use the kernel Kh (x) := h K( h x). Note that it is h that plays the role of bandwidth here and not just h. hen, the estimator ˆφ obtained using the Algorithm 3. still satisfies the two assertions of heorem Variance function estimation Estimating the variance function σ t (x) is very similar to how it was done in Dahl and Levine (006). For simplicity, we will illustrate the idea only for p =. As a reminder, the first step of our proposed method is estimating not σ but rather σ,bias (x) = σ (x) + φ, by the local linear regression applied to the squared-pseudoresiduals η t. As in Dahl and Levine (006), we assume that the kernel K(u) is a two-sided proper density second order kernel on the interval [, ]; this means that. K(u) 0 and K(u) du = 0. µ = uk(u) du = 0 and µ σ K = u K(u) du 0. We also denote R K = K (u) du. hen, the inconsistent estimator σ (x) of σ (x) is defined as the value â solving the local least squares problem (â, ˆb) = arg min a,b t=3 (η t a b(x t x)) K m (x t x), where as usual K m (x) := m K(x/m). Since σ (x) estimates σ,bias (x) consistently, at the next step we define a consistent estimator of σ (x) as follows: ˆσ (x) = σ (x)( + ˆφ ), (4.) where ˆφ = ( ˆφ, ˆφ ) is the least-squares estimator defined in (.). he following lemma can be proved by following almost verbatim heorem 3 of Dahl and Levine (006) and is omitted for brevity. Below, we will use Dσ (x) and D σ (x) to denote the first and the second-order derivatives of the function σ (x), respectively. Lemma 4.. Under the above assumptions ()-() on the kernel and assumptions ()-(3) on the data generating process (.) introduced in Section 3, σ (x) is a consistent estimator of σ,bias (x) Moreover, σ (x) σ,bias (x) B(φ, x) V / (φ, x) 0 d N(0, ),

11 where the bias B(φ, x) and variance V (φ, x) of σ (x) are such that { m σ [ K B(φ, x) = D σ (x)/4 γ (Dσ (x)) /σ (x) ] } + o(m ) + O( ) V (φ, x) = R K C(φ, φ )σ 4 (x)( m) + o( m ). and the above constant C(φ, φ ) depends only on φ and φ. Now we are ready to state the main result of this section. heorem 4.. Under the same assumptions as in Lemma 4., the estimator ˆσ (x) introduced in (4.) is an asymptotically consistent estimator of σ (x) that is also asymptotically normal with the bias ( + φ )B(φ, x) and the variance ( + φ ) V (φ, x). Proof. By the Slutsky s theorem, we have σ (x) σ,bias (x) B(φ,x) ( + ˆφ ) d ( + φ )ζ with ζ N(0, ). V (φ,x) his means that ˆσ is a consistent estimator of σt ( + φ ) V (φ, x). with the bias ( + φ )B(φ, x) and the variance 5 Numerical results 5. Simulation study In this part we review the finite-sample performance of the proposed estimators. In order to do this, we consider three model specifications given in able of Appendix B. he variance function specifications are the same as those in Dahl and Levine (006). he specification of σ t in Model is a leading example in econometrics/statistics and can generate ARCH effects if x t = y t. Model is adapted from Fan and Yao (998). In particular, the choice of σ t is identical to the variance function in their Example. he variance function in Model 3 is from Härdle and sybakov (997). We take a fixed design x t = t/ for t = 0,..., and compute the WLSE estimators ˆφ and ˆφ of (.) for the previously mentioned variance function specifications and three different samples sizes, = 00, = 000, and = 000. In order to assess the performance of the estimator (.), we compute the Mean Squared Error (MSE) defined by MSE(ˆσ) := M M i= (ˆσ t,i σt ), t= where ˆσ t,i is the estimated variance function at x t in the i th simulation and M is the number of simulations. We use a local linear estimator σ t for estimating the biased variance function σ,bias t of (.0), as in the step of the Algorithm. in Section. he selection of the method s bandwidth was carried out by a 0-fold cross-validation method (see, e.g., Section in Fan and Yao (003) for further information). able in Appendix B provides the MSE for the above three model specifications and sample sizes, while able 3 shows the sampling mean and standard errors for the estimators ˆφ and ˆφ. In the able 3 of Appendix B, Mn(Sd) stand for Mean(Standard Deviation). We also consider three different parameter settings: () (φ, φ ) = (0.6, 0.3), () (φ, φ ) = (0.6, 0.6), (3) (φ, φ ) = (0.98, 0.6).

12 For the true parameter values (φ, φ ) = (0.6, 0.3), the asymptotic standard deviation and covariance given in (3.0) take the values: V = 0.953, V = In particular, the above standard error should be compared with the asymptotic theoretical standard deviation V / from heorem 3.4. For the sample sizes 00, 000, and 000, V / takes the values , 0.030, and 0.03, which match the sampling standard deviations of able 3 in Appendix B. he results show clear improvement for increasing sample sizes; Models and 3 seem to be a little easier to estimate than Model. Finally, again for φ = 0.6 and φ = 0.3, Figure below shows the sampling densities for ˆφ and ˆφ corresponding to each of the three models and three sample sizes. No severe small sample biases seem to be present in any of the pictures. Model Model Rel. Freq =000 =000 =00 Normal Rel. Freq =000 =000 =00 Normal φ φ Model Model Rel. Freq =000 =000 =00 Normal Rel. Freq =000 =000 =00 Normal φ φ Model 3 Model 3 Rel. Freq =000 =000 =00 Normal Rel. Freq =000 =000 =00 Normal φ φ Figure : Sampling densities in comparison with the standard normal density under the three alternative variance function specifications of able of Appendix B. he true parameter values are φ = 0.6 and φ = 0.3. he number of Monte Carlo replications is 000.

13 5. Real data application We now apply our method to the first differences of annual global surface air temperatures in Celsius from 880 through 985. his data set has been extensively analyzed in the literature using different versions of the general nonparametric regression model (.4). For instance, Hall and Van Keilegom (003) fixed σ t and estimated the coefficients of both the AR() and AR() models for {v t } t {,..., }, directly from the observations {y t } t {,..., } before estimating the trend function µ t = g(x t ) (see Remark 3.5 above for further information about their method). hey reported point estimates of ˆφ = 0.44 and ˆφ = Assuming again σ t as in Hall and Van Keilegom (003), Shao and Yang (0) first used linear B-splines to estimate the trend function g, and then applied a Yule-Walker type estimation method to the residuals ŷ t = y t ˆµ t, under an AR() model specification (see Remark 3.5). hey found a point estimate of ˆφ = with a standard error he left panel of Figure shows the scatter plot of the temperature differences against time and the estimated mean ˆµ t = ĝ(x t ) using a simple local linear estimator applied to the observations (y t ). We then apply our estimation procedure to the differentials ŷ t = y t ˆµ t, assuming an AR() model specification for the error process {v t } t {,..., }. he point estimates for φ and φ are respectively ˆφ = and ˆφ = with a standard error of 0.08 (see (3.)). he resulting estimated variance function is shown in the right panel of Figure. As observed there, ˆσ t exhibits an interesting V-shaped pattern with minimum around the year 945. Also, the estimated values ˆφ and ˆφ are consistent with those reported by Hall and Van Keilegom (003). emperture difference (in CeLsius) Estimated variance function ime (in years) ime (in years) Figure : Differences of annual global surface air temperatures in Celsius from 880 through Discussion In this manuscript, we propose a method for estimation of the variance structure of the scaled autoregressive process unknown coefficients and scale variance function σ t. his method is being proposed to extend earlier results of Dahl and Levine (006), where the analogous problem for the specific case of error autoregressive process of order was solved. he direct generalization of the method of Dahl and Levine (006) does not seem to be possible in the case of the autoregressive process of order more than ; thus, the method proposed in this manuscript represents a qualitatively new procedure. his data set was obtained from the web site 3

14 here is a number of interesting issues left unanswered here that we plan to address in the future research on this subject. Although we only examined the model (.) with the conditional mean equal to zero, an important practical issue is often the robust estimation of the parametrically specified conditional mean when the error process is conditionally heteroscedastic. As an example, an autoregressive conditional mean of order l is commonly assumed. Although standard regression procedures can be made robust in this situation by using the heteroscedasticity-consistent (HC) covariance matrix estimates as suggested in Eicker (963) and White (980), there may be advantages in considering alternative methods that take a specific covariance structure into account. Such methods are likely to provide more efficient estimators. his has been done in, for example, Phillips and Xu (005) who addressed that issue by conducting asymptotic analysis of least squares estimates of the conditional mean coefficients µ t, t =,..., (see (.4)) under the assumption of strongly mixing martingale difference error process and a non-constant variance function. Our setting is not a special case of Phillips and Xu (005) since for our error process E(v t F t ) 0 (where F t = σ(v s, s t) is the natural filtration). his case, to the best of our knowledge, has not been considered in the literature before. We believe, therefore, that the asymptotic analysis of the least squares estimates of the coefficients µ t under the same assumptions on the error process as in (.) is an important topic for future research. Also, being able to estimate the exact heteroscedasticity structure is important in econometrics in order to design unit-root tests that are robust to violations of the homoscedasticity assumption. he size and power properties of the standard unit-root test can be affected significantly depending on the pattern of variance changes and when they occur in the sample; an extensive study of possible heteroscedasticity effects on unit-root can be found in Cavaliere (004). We also intend to consider the design of robust unit-root tests under the serial innovations specification as part of our future research. Finally, yet another interesting topic of future research is a possible extension of these results to the case of a more general ARMA(p,q) error process. One of the possibilities may be using the difference-based pseudoresiduals again to construct an inconsistent estimator of the variance function σt first. Indeed, since the scaling constant will only be dependent on the coefficients of the ARMA (p,q) error process, the MINPIN estimators of the coefficients of the error process based on such an inconsistent variance estimator will be unaffected. herefore, estimation of the coefficients of the error process will proceed in the same way as for usual ARMA processes. he final correction of the nonparametric variance estimator also appears to be straightforward. Acknowledgements José E. Figueroa-López has been partially supported by the NSF grant DMS Michael Levine has been partially supported by the NSF grant DMS he authors gratefully acknowledge the constructive comments provided by two anonymous referees, which significantly contributed to improve the quality of the manuscript. A Proofs Proof of Proposition.. It is easy to see that ηt = σ γ 0 m a j + j=0 i= j=0 m m i a j a j+i γ i. 4

15 Recalling that for an AR() time series, it follows that φ γ 0 = φ, and γ i = φ i γ 0, m j=0 a j + φ his can be written as the following polynomial of φ : m j=0 m m i a j a j+i φ i =. i= j=0 m a j + φ ( + a j a j+ ) + j=0 hen, we get the following system of equations: (i) (iii) m j=0 m i=,i j=0 m a j =, (ii) + a j a j+ = 0, m i a j a j+i = 0, j=0 j=0 i {, 3,..., m}. m i a j a j+i φ i = 0. Suppose that a 0 0. hen, equation (iii) for i = m, implies that a 0 a m = 0 and, hence, a m = 0. Equation (iii) for i = m yields a 0 a m + a a m = 0 and thus a m = 0. By induction, it follows that a m = a m = = a 3 = 0. Plugging in (i-iii), which admits as unique solution a 0 + a + a =, + a 0 a = 0, a 0 a + a a = 0, a 0 = ±, a = 0, a =. If a 0 = 0, but a 0, one can similarly prove that the only solution is a = ±, a 3 =, a i = 0, otherwise. he statement of the proposition can be obtained by induction in k. Proof of Lemma 3.. his can be done by appealing to heorem in Andrews (987). First, using representations (3.5)-(3.7), we define W t := (ε t, ε t,... ) R N, q t,k (W t, φ) := ε t ψ i ε t k i. i=0 It remains to verify the assumptions A-A3 of heorem in Andrews (987). As stated in Corollary of Andrews (987), one can check its condition A4 therein instead of condition A3 since A4 implies A3. We now state these three conditions: A. Θ is a compact set. 5

16 A. Let B(φ 0 ; ρ) Θ be an open ball around φ 0 of radius ρ and let m k,t (σ; ρ) = sup{m k,t(σ; φ) : φ B(φ 0 ; ρ)}, m k,t (σ; ρ) = inf{m k,t (σ; φ) : φ B(φ 0 ; ρ)}. (A.) (A.) hen, the following two conditions are satisfied: (a) All of m k,t (σ; φ), m k,t (σ; ρ), and m k,t (σ; ρ) are random variables for any φ Θ, any t and any sufficiently small ρ; (b) Both m k,t (σ; ρ) and m k,t (σ; ρ) satisfy pointwise weak laws of large numbers for any sufficiently small ρ. A4. For each φ Θ there is a constant τ > 0 such that d( φ, φ) τ implies m t (σ; φ) m t (σ; φ) B t h(d( φ, φ)) where B t is a nonnegative random variable (that may depend on φ) and lim t= EB t <, while h : R + R + is a nonrandom function such that h(y) h(0) = 0 as y 0. Above, denotes the standard Euclidean norm in R p. Since condition A above is assumed as a hypothesis, we only need to work with conditions A and A4. he verification of these is done through the following two steps:. Let ρ > 0 small enough such that B(φ 0 ; ρ) Θ. hen, recalling the representation (3.5) and (3.7), m,t (σ; φ) = ε t ψ i ε t i, i=0 (A.3) we find that the supremum of m,t (σ; φ), taken over φ B(φ 0 ; ρ), also exists and is a random variable. Indeed, each of the summands in (A.3) is a continuous function of φ as we already established earlier, the convergence in mean squared to m,t (σ; φ) is uniform in φ due to (3.8) and, therefore, m,t (σ; φ) is continuous in φ as well. hat, in turn, implies the existence of m,t (σ; ρ). he existence of m,t (σ; ρ) is established in exactly the same way. Moreover, the pointwise WLLNs for both sup φ B m k,t (σ; φ) and inf φ B m k,t (σ; φ) are also clearly satisfied since E m k,t (σ; φ) 0 for any φ B(φ; ρ) Θ.. We now show condition A4 above for m,t (σ; φ) (m,t (σ; φ) can be treated analogously). Denote m,t (σ; φ) = ε t i=0 ψ i ε t i where ψ i correspond to the MA( ) representation of the AR() series with the parameter vector φ = (φ, φ ) Θ. For the sake of brevity we will use m and m for m,t (σ; φ) and m,t(σ; φ), respectively. Note that Let B t := i=0 ψ i ε t i ε t and m m = ε t ψi ε t i ε t ψ i ε t i i=0 i=0 ψ i ε t i ε t i=0 ( ) ψ i ψ i. i=0 ψ i d(φ, φ) := ( ) ψ i ψ i. ψ i=0 i 6

17 hen, sup t= EB t sup ( E t= i=0 ψ i ε t i ε t ) = ψi <. i=0 Now we need to treat the second multiplicative term. First, recalling (3.9), it is easy to conclude, by induction, that the coefficients ψ j are continuously differentiable functions of φ = (φ, φ ) for any φ B(φ, ρ) Θ. herefore, using (3.8), one can easily establish that lim φ φ d(φ, φ) = 0. Note that the quantity ( ) ψ i ψ i i=0 ψ i is not a metric and, therefore, the verification of Assumption A4 seems in doubt at first sight. However (see Andrews (99)), the fact that Assumption A4 implies Assumption A3 does not need the argument d( φ, φ) of the function h to be a proper metric but only that d( φ, φ) 0 as φ φ. Proof of heorem 3.3. Our proof of consistency will rely on the heorem A of Andrews (994) with W t = (y t, y t, y t ). We will simply verify that the sufficient conditions of heorem A are true. he first assumption C(a) follows from Lemma 3. taking m t (σ; φ) 0. It remains to show the other conditions therein. he first part of Assumption C(b) of the heorem A is immediately satisfied because m(σ; φ) 0 for any φ and σ. Since σ t is a local linear regression estimator, it is clear that it is twice continuously differentiable as long as the kernel function K() is twice continuously differentiable; thus, the second part of of the Assumption C(b) is also true. he Assumption C(c) is true if the Euclidean norm of m(σ; φ) = lim t= E m t(σ; φ) is finite. Hereafter, we denote the euclidian norm by. In our case, since both martingale difference sequences v t ε t and v t ε t have mean zero, clearly sup Θ F m(σ; φ) = 0 < and the Assumption C(c) is satisfied. he Assumption C(d) is true because Θ is compact, the functional d t = m t(σ; φ)m t (σ; φ)/, is continuous in φ and the Hessian matrix d t φ allows us to conclude that the weak consistency holds: ˆφ P φ. is positive definite (can be verified). All of the above Proof of heorem 3.4. Recall that σ t is the inconsistent estimator of σ t which, nevertheless, estimates the quantity σt bias = σt +φ consistently (see Lemma 4. above); it is corrected to obtain ˆσ t = σ t ( + ˆφ ). We also recall the notation (3.) and define, for some generic argument ϑ = (ϑ,..., ϑ ) and ϕ = (ϕ, ϕ ), m (ϑ; ϕ) := m t (ϑ; ϕ), t= with m t (ϑ, ϕ) given as in (3.4). Since the vector valued function m t (ϑ t ; ϕ) is twice continuously differentiable, we can use a aylor expansion around ( σ; φ) to get m ( σ; ˆφ) = m ( σ; φ) + φ m ( σ; φ ) (ˆφ φ), (A.4) 7

18 for some φ that lies on the straight line connecting ˆφ and φ. Note that, due to the first order conditions (3.6), the left-hand side of the equation (A.4) cancels out. Hence, using the consistency of ˆφ for φ (heorem 3.3) and the assumption in the statement of the theorem, we obtain that (ˆφ φ) = [ M + o p ()] m ( σ t ; φ), provided that the matrix M = lim t= E m t φ (σbias ; φ) exists and is invertible. In order to verify the latter condition, note that in our case, ( m,t ) m m,t t φ (σbias φ ; φ) = φ In light of (.6), it follows that M = lim m,t φ m,t φ t= which, in particular, is clearly invertible. (σ bias,φ) = ( + φ ) ( v t v t v t v t v t v t E m ( t φ (σbias ; φ) = ( + φ ) γ0 γ γ γ 0 Next, let m (ϑ; ϕ) = t= E m t(ϑ; ϕ) and define the empirical process ν (ϑ) = [ m (ϑ; φ) m (ϑ; φ)] ), ), (A.5) (A.6) Clearly, Now, using (3.5), m ( σ; φ) = m (σ bias ; φ) + ν ( σ) ν (σ bias ) + m ( σ; φ) m (σ bias ; φ) = t= ( m t (σ bias ; φ) = + φ t= v t ε t, t= v t ε t ) Hence, using the CL for martingale difference sequences (see Billingsley (96)), m (σ bias ; φ) is asymptotically normal N(0, ( + φ ) S), where ( Ev S := t ε t Ev t v t ε ) ( ) t γ0 γ Ev t v t ε t Evt =. ε t γ γ 0 Note that M = ( + φ ) S. hen, if we can show that our task is over and we can say that (ˆφ φ) ν ( σ) ν (σ bias ) P 0 and m ( σ; φ) P 0, (A.7) d N(0, V ) with V = M (( + φ ) S)( M ) = ( ( + φ ) S) (( + φ ) S)( ( + φ ) S ) ( ) = S γ0 γ = γ0. γ γ γ 0 A simple computation using (.6)-(.7) leads to (3.0). We show (A.7) through the following two steps:. 8

19 () First, denoting the indicator function of an event A by A, we have, for any ε > 0, m ( σ; φ) = m ( σ; φ) {ρ( σ,σ bias )>ε} + m ( σ; φ) {ρ( σ,σ bias ) ε}. he first term on the right is o p () due to consistency of σ as an estimator of σ bias. Similarly, the second term therein is o p () due to the fact that E m t (σ bias ; φ) = 0 and Em t (ϑ; ϕ) is uniformly continuous in ϑ. We then conclude that m ( σ; φ) P 0 () We can show that ν ( σ) ν (σ bias ) P 0 using the argument of Andrews (994), pp , that requires, first, establishing stochastic equicontinuity of ν (ϑ) at σ bias and then deducing the required convergence in probability. o do this, note first that / t= ( v t E vt ) = Op (), / v t k v t k = O p (), t= (A.8) for k = 0,. he empirical process ν (ϑ) has its values in R ; examining its first coordinate ν (ϑ), one easily obtains ν (ϑ) = / t= S tτ t (ϑ), where S t = (y t y t, φ y t, φ y t y t ), τ t (ϑ) = (ϑ t ϑ t, ϑ t, ϑ t ϑ t ). hen, denoting the standard Euclidean norm in R 3 by, for η > 0 and δ > 0, we have ( ) lim sup P = lim sup P lim sup P sup ρ( σ,σ bias )<δ ( ( sup ρ( σ,σ bias )<δ sup ρ( σ,σ bias )<δ ν ( σ) ν (σ bias ) > η ) / (S t E S t)( τ t ( σ) τ t (σ bias )) > η t= ) / (S t E S t ) > η < ε, δ t= for δ > 0 small-enough, since / t= (S t E S t ) = O p () due to (A.8). Using exactly the same argument, one easily obtains stochastic equicontinuity of the second coordinate ν (ϑ) at σbias ; therefore, needed convergence in probability for ν ( σ) ν (σ bias ) follows from the convergence in both coordinates separately and the Slutsky s theorem. Proof of Corollary 3.6. he following is a sketch of the proof for the case of p =. he conditions given in the corollary are not the least restrictive; they guarantee consistency of the estimator ˆµ t g(x t ) and the order O((h/ ) ) for the bias; see heorem 6. in Fan and Yao (003) for details. Note that the model (.4) can be represented in the functional autoregressive form σt y t = f t (φ) + φ σt y t + φ σt y t + ε t where the new mean function is f t (φ) = σt g(x t ) φ σt g(x t ) φ σt g(x t ). Recall that functions m k,t (φ) v t k ε t, k =, do not depend on σ t ; therefore, (3.5) is still true and the Lemma 3. is also true as well. his implies that the heorem 3.3 is true as well. Finally, when trying to prove heorem 3.4, one quickly discovers that the matrix mt φ (φ) is now different; for example, m,t φ = γ 0 g(x t )σt v t. Nevertheless, since the second term of the above has the mean zero, its expectation is the same as before and the same is true of the other four elements of the matrix; thus, matrix M stays the same. Finally, the vector S t = ((y t µ t (y t µ t ), φ (y t µ t ), φ (y t µ t )(y t µ t )) is now different; nevertheless, it is easy to verify that / t= (S t E S t ) = O p () again; thus the final result is still valid. 9

20 References Andrews, D.W.K. (987) Consistency in nonlinear econometric models: a generic uniform law of large numbers, Econometrica 55, Andrews, D.W.K. (994) Asymptotics for semiparametric econometric models via stochastic equicontinuity, Econometrica 6, Billingsley, P. (96) he Lindeberg-Levy heorem for Martingales, in Proceedings of American Mathematical Society, Vol, Brockwell, P., and Davis, R. (99) ime series: theory and methods. Springer. Cavaliere, G. (004) Unit root tests under time-varying variance shifts, Econometric Reviews 3, Chen, R. and say, R.S. (993) Functional coefficient autoregressive models, Journal of American Statistical Association, 88, Dahl, C. and Levine, M. (006) Nonparametric estimation of volatility models with serially dependent innovations, Statistics and Probability Letters, 76(8), Eicker, B. (963) Limit theorems for regression with unequal and dependant errors, Annals of Mathematics and Statistics 34, Fan, J. and Yao, Q. (998) Efficient estimation of conditional variance functions in stochastic regression, Biometrika, 85, Fan, J. and Yao, Q. (003) Nonlinear time series - nonparametric and parametric methods. Springer. Hall, P. and Van Keilegom, I. (003) Using difference-based methods for inference in nonparametric regression with time series errors, Journal of the Royal Statistical Society (B), 85(), Härdle, W. and sybakov, A. (997) Local polynomial estimators of the volatility function in nonparametric autoregression, Journal of Econometrics, 8, 3-4. Phillips, P.C.B., and Xu, K.-L. (005) Inference in Autoregression under heteroscedasticity, Journal of ime Series Analysis 7, Shao, Q. and Yang, L. (0) Autoregressive coefficient estimation in nonparametric analysis, Journal of ime Series Analysis 3, White, H. (980) A heteroscedasticity-consistent covariance matrix estimator and a direct test for heteroscedasticity, Econometrica 48,

21 B ables Model Specifications σt = 0.5x t + 0. σt = 0.4 exp( x t ) σt = ϕ(x t +.) +.5ϕ(x t.) able : Alternative data generating processes. ϕ( ) denotes the standard normal probability density. φ = 0.6, φ = 0.3 φ = 0.6, φ = 0.6 φ = 0.98, φ = 0.6 Model =00 =000 =000 =00 =000 =000 =00 =000 = able : Mean Squared Errors (MSE) of ˆσ (x) under the variance function specifications of able with 000 Monte Carlo replications and a 0-fold cross-validation for bandwidth selection. φ = 0.6, φ = 0.3 Model = 00 = 000 = 000 Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ 0.559(0.4) 0.98(0.0) 0.594(0.09) 0.30(0.030) 0.595(0.0) 0.30(0.0) 0.565(0.05) 0.96(0.098) 0.594(0.09) 0.30(0.030) 0.598(0.0) 0.99(0.0) (0.03) 0.94(0.099) 0.598(0.030) 0.98(0.030) 0.597(0.0) 0.300(0.0) φ = 0.6, φ = 0.6 Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ 0.554(0.) (0.04) 0.595(0.06) -0.59(0.04) 0.597( 0.07) (0.07) 0.56(0.00) -0.54(0.099) 0.597( 0.05) (0.05) 0.596(0.07) (0.08) ( 0.00) (0.097) 0.596(0.05) (0.05) 0.597(0.07) (0.07) φ = 0.98, φ = 0.6 Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ Mn(Sd) ˆφ 0.894(0.40) -0.57(0.8) 0.97(0.06) -0.59(0.05) 0.975(0.07) ( 0.08) 0.99(0.08) (0.099) 0.973(0.05) -0.59(0.05) 0.976(0.07) (0.07) ( 0.0) (0.098) 0.974(0.06) ( 0.06) 0.976(0.07) (0.08) able 3: Sampling Means and Standard Deviations under under the variance function specifications of able with 000 Monte Carlo replications. Department of Statistics, 50 N. University St., Department of Statistics, Purdue University, West Lafayette, IN figueroa@purdue.edu Department of Statistics, 50 N. University St., Department of Statistics, Purdue University, West Lafayette, IN mlevins@purdue.edu.