SIEVE INFERENCE ON SEMI-NONPARAMETRIC TIME SERIES MODELS. Xiaohong Chen, Zhipeng Liao and Yixiao Sun. February 2012

Transcription

1 SIEVE INFERENCE ON SEMI-NONPARAMERIC IME SERIES MODELS By Xiaohong Chen, Zhipeng Liao and Yixiao Sun February COWLES FOUNDAION DISCUSSION PAPER NO. 849 COWLES FOUNDAION FOR RESEARCH IN ECONOMICS YALE UNIVERSIY Box 88 New Haven, Connecticut

2 Sieve Inference on Semi-nonparametric ime Series Models Xiaohong Chen y, Zhipeng Liao z, and Yixiao Sun x First Draft: October 9; his draft: February Abstract he method of sieves has been widely used in estimating semiparametric and nonparametric models. In this paper, we rst provide a general theory on the asymptotic normality of plug-in sieve M estimators of possibly irregular functionals of semi/nonparametric time series models. Next, we establish a surprising result that the asymptotic variances of plug-in sieve M estimators of irregular (i.e., slower than root- estimable) functionals do not depend on temporal dependence. Nevertheless, ignoring the temporal dependence in small samples may not lead to accurate inference. We then propose an easy-to-compute and more accurate inference procedure based on a pre-asymptotic sieve variance estimator that captures temporal dependence. We construct a pre-asymptotic Wald statistic using an orthonormal series long run variance (OS-LRV) estimator. For sieve M estimators of both regular (i.e., root- estimable) and irregular functionals, a scaled pre-asymptotic Wald statistic is asymptotically F distributed when the series number of terms in the OS-LRV estimator is held xed. Simulations indicate that our scaled pre-asymptotic Wald test with F critical values has more accurate size in nite samples than the usual Wald test with chi-square critical values. Keywords: Weak Dependence; Sieve M Estimation; Sieve Riesz Representor; Irregular Functional; Misspeci cation; Pre-asymptotic Variance; Orthogonal Series Long Run Variance Estimation; F Distribution We acknowledge useful comments from. Christensen, J. Hahn, L. Hansen, J. Hidalgo, M. Jansson, D. Kaplan, O. Linton, P. Phillips, D. Pouzo, J. Powell, H. White, and other participants at Econometrics Society Australasian Meeting in Adelaide, the Pre-conference Workshop of Asian Meeting of the Econometric Society in Seoul, econometrics workshops at Yale, UC Berkeley, UCLA, UCSD and Stanford. Chen and Sun acknowledge nancial support from the National Science Foundation under the respective grant numbers: SES-8386 and SES Any errors are the responsibility of the authors. y Department of Economics, Yale University, 3 Hillhouse, Box 88, New Haven, C xiaohong.chen@yale.edu z Department of Economics, UC Los Angeles, 8379 Bunche Hall, Mail Stop: 4773, Los Angeles, CA zhipeng.liao@econ.ucla.edu x Department of Economics, UC San Diego, 95 Gilman Drive, La Jolla, CA yisun@ucsd.edu

3 Introduction Many economic and nancial time series (and panel time series) are nonlinear and non-gaussian; see, e.g., Granger (3). For policy and welfare analysis, it is important to uncover complicated nonlinear economic relations in dynamic structural models. Unfortunately, it is di cult to correctly parameterize nonlinear dynamic functional relations. Even if the nonlinear functional relation among the observed variables is correctly speci ed by economic theory or by chance, misspecifying distributions of nonseparable latent variables could lead to inconsistent estimates of structural parameters of interest. hese reasons, coupled with the availability of larger data sets, motivate the growing popularity of semiparametric and nonparametric models and methods in economics and nance. he method of sieves (Grenander, 98) is a general procedure for estimating semiparametric and nonparametric models, and has been widely used in economics, nance, statistics and other disciplines. In particular, the method of sieve extremum estimation optimizes a random criterion function over a sequence of approximating parameter spaces, sieves, that becomes dense in the original in nite dimensional parameter space as the complexity of the sieves grows to in nity with the sample size. See, e.g., Chen (7, ) for detailed reviews of some well-known empirical applications of the method and existing theoretical properties of sieve extremum estimators. In this paper, we consider inference on possibly misspeci ed semi-nonparametric time series models via the method of sieves. We focus on sieve M estimation, which optimizes a sample average of a criterion function over a sequence of nite dimensional sieves whose complexity grows to in nity with the sample size. Prime examples include sieve quasi maximum likelihood, sieve (nonlinear) least squares, sieve generalized least squares, and sieve quantile regression. For general sieve M estimators with weakly dependent data, White and Wooldridge (99) establish the consistency, and Chen and Shen (998) establish the convergence rate and the p asymptotic normality of plug-in sieve M estimators of regular (i.e., p estimable) functionals. o the best of our knowledge, there is no published work on the limiting distributions of plug-in sieve M estimators of irregular (i.e., slower than p estimable) functionals. here is also no published inferential result for general sieve M estimators of regular or irregular functionals for possibly misspeci ed semi-nonparametric time series models. We rst provide a general theory on the asymptotic normality of plug-in sieve M estimators of possibly irregular functionals in semi/nonparametric time series models. his result extends that of Chen and Shen (998) for sieve M estimators of regular functionals to sieve M estimators of irregular functionals. It also extends that of Chen and Liao (8) for sieve M estimators of irregular functionals with iid data to time series settings. he asymptotic normality result is rate-adaptive in the sense that researchers do not need to know a priori whether the functional of interest is p estimable or not. For weakly dependent data and for regular functionals, it is known that the asymptotic variance expression depends on the temporal dependence and is usually equal to the long run variance (LRV) of

4 a scaled moment (or score) process. It is often believed that this result would also hold for sieve estimators of irregular functionals such as the evaluation functionals and weighted integration functionals. Contrary to this common belief, we show that under some general conditions the asymptotic variance of the plug-in sieve estimator for weakly dependent data is the same as that for iid data. his is a very surprising result, as sieve estimators are often regarded as global estimators, and hence autocorrelation is not expected to vanish in the limit (as! ). Our asymptotic theory suggests that, for weakly dependent time series data with a large sample size, temporal dependence could be ignored in making inference on irregular functionals via the method of sieves. his resembles the earlier well-known asymptotic results for time series density and regression functions estimated via kernel and local polynomial regression methods. See, e.g., Robinson (983), Fan and Yao (3), Li and Racine (7), Gao (7) and the references therein. However, simulation studies indicate that inference procedures based on asymptotic variance estimates ignoring autocorrelation may not perform well when the sample size is small (relatively to the degree of temporal dependence). See, e.g., Conley, Hansen and Liu (997) and Pritsker (998) for earlier discussion of this problem with kernel density estimation for interest rate data sets. In this paper, for both regular and irregular functionals of semi-nonparametric time series models, we propose computationally simple, accurate and robust inference procedures based on estimates of preasymptotic sieve variances capturing temporal dependence. hat is, we treat the underlying triangular array sieve score process as a generic time series and ignore the fact that it becomes less temporally dependent when the sieve number of terms in approximating unknown functions grows to in nity as goes to in nity. his pre-asymptotic approach enables us to conduct easy-to-compute and accurate inference on semi-nonparametric time series models by adopting any existing autocorrelation robust inference procedures for (misspeci ed) parametric time series models. For semi-nonparametric time series models, we could compute various pre-asymptotic Wald statistics using various existing LRV estimators for regular functionals of (misspeci ed) parametric time series models, such as the kernel LRV estimators considered by Newey and West (987), Andrews (99), Jansson (4), Kiefer and Vogelsang (5), Sun (b) and others. Nevertheless, to be consistent with our focus on the method of sieves and to derive a simple and accurate asymptotic approximation, we compute a pre-asymptotic Wald statistic using an orthonormal series LRV (OS- LRV) estimator. he OS-LRV estimator has already been used in constructing autocorrelation robust inference on regular functionals of parametric time series models; see, e.g., Phillips (5), Müller (7), Sun (a), and the references therein. We extend these results to robust inference on both regular and irregular functionals of semi-nonparametric time series models. For both regular and irregular functionals, we show that the pre-asymptotic t statistic and a scaled We thank Peter Phillips for suggesting that we consider autocorrelation robust inference for semi-nonparametric time series models.

5 Wald statistic converge to the standard t distribution and F distribution respectively when the series number of terms in the OS-LRV estimator is held xed; and that the t distribution and F distribution approach the standard normal and chi-square distributions respectively when the series number of terms in the OS-LRV estimator goes to in nity. Our pre-asymptotic t and F approximations achieve triple robustness in the following sense: they are asymptotically valid regardless of () whether the functional is regular or not; () whether there is temporal dependence or not; and (3) whether the series number of terms in the OS-LRV estimator is held xed or not. o facilitate the practical use of our inference procedure, we show that, in nite samples and for linear sieve M estimators, our pre-asymptotic sieve test statistics (i.e. t statistic and Wald statistic) for semi-nonparametric time series models are numerically equivalent to the corresponding test statistics one would obtain if the models are treated as if they were parametric. hese results are of much use to applied researchers, and demonstrate the advantage of the sieve method for inference on semi-nonparametric time series models. o investigate the nite sample performance of our proposed pre-asymptotic robust inference procedures on semi-nonparametric time series models, we conduct a detailed simulation study using a partially linear regression model. For both regular and irregular functionals, we nd that our test using the pre-asymptotic scaled Wald statistic and F critical values has more accurate size than the pre-asymptotic Wald test using chi-square critical values. For irregular functionals, we nd that they both perform better than the Wald test using a consistent estimate of the asymptotic variance ignoring autocorrelation. hese are especially true when the time series (with moderate sample size) has strong temporal dependence and the number of joint hypotheses being tested is large. Based on our simulation studies, we recommend the use of the pre-asymptotic scaled Wald statistic using an OS-LRV estimator and F approximation in empirical applications. he rest of the paper is organized as follows. of functionals of interest and gives two illustrative examples. Section presents the plug-in sieve M estimator normality of the plug-in sieve M estimators of possibly irregular functionals. Section 3 establishes the asymptotic Section 4 shows the surprising result that the asymptotic variances of plug-in sieve M estimators of irregular functionals for weakly dependent time series data are the same as if they were for i.i.d. data. Section 5 presents the pre-asymptotic OS-LRV estimator and F approximation. Section 6 proves the numerical equivalence result. Section 7 reports the simulation evidence, and the last section brie y concludes. Appendix A contains all the proofs, and Appendix B discusses the properties of the hidden delta functions associated with sieve M estimation of evaluation functionals. Notation. In this paper, we denote f A (a) (F A (a)) as the marginal probability density (cdf) of Here we slightly abuse terminology and de ne a parametric model to be a model with a xed nite number of unknown parameters of interest, although the model may contain in nite dimensional nuisance parameters that are not needed to be estimated, such as Hansen (98) s GMM models. 3

6 a random variable A evaluated at a and f AB (a; b) (F AB (a; b)) the joint density (cdf) of the random variables A and B. We use to introduce de nitions. For any vector-valued A, we let A denote its transpose and jjajj E confusion. p A A, although sometimes we also use jaj = p A A without too much Denote L p (; d), p <, as a space of measurable functions with jjgjj L p (;d) f R jg(t)jp d(t)g =p <, where is the support of the sigma- nite positive measure d (sometimes L p () and jjgjj L p () are used when d is the Lebesgue measure). For any (possibly random) positive sequences fa g = and fb g =, a = O p (b ) means that lim c! lim sup Pr (a =b > c) = ; a = o p (b ) means that for all " >, lim! Pr (a =b > ") = ; and a b means that there exist two constants < c c < such that c a b c a. We use A A k, H H k and V V k to denote various sieve spaces. o simplify the presentation, we assume that dim(v ) = dim(a ) dim(h ) k, all of which grow to in nity with the sample size. Sieve M Estimation and Examples. Basic Setting We assume that the data fz t = (Yt ; Xt) g t= is from a strictly stationary and weakly dependent process de ned on an underlying complete probability space. Let the support of Z t be Z R dz ; d z <, and let Y and X be the supports of Y and X respectively. Let (A; d) denote an in nite dimensional metric space. Let ` : Z A! R be a measurable function and E[`(Z; )] be a population criterion. For simplicity we assume that there is a unique (A; d) such that E[`(Z; )] > E[`(Z; )] for all (A; d) with d(; ) >. Di erent models in economics correspond to di erent choices of the criterion functions E[`(Z; )] and the parameter spaces (A; d). A model does not need to be correctly speci ed and could be a pseudo-true parameter. Let f : (A; d)! R be a known measurable mapping. In this paper we are interested in estimation of and inference on f( ) via the method of sieves. Let A be a sieve space for the whole parameter space (A; d). hen there is an element A such that d ( ; )! as dim(a )! (with ). An approximate sieve M estimator b A of solves X t= `(Z t ; b ) sup A X `(Z t ; ) O p (" ); (.) where the term O p (" ) = o p( ) denotes the maximization error when b maximizer over the sieve space. We call f(b ) the plug-in sieve M estimator of f( ). t= fails to be the exact Under very mild conditions ( see, e.g., Chen (7, heorem 3.) and White and Wooldridge (99)), the sieve M estimator b is consistent for : d(b ; ) = O p fmax [d(b ; ); d ( ; )]g = o p (). 4

7 . Examples he method of sieve M estimation includes many special cases. Di erent choices of criterion functions `(Z t ; ) and di erent choices of sieves A lead to di erent examples of sieve M estimation. As an illustration, we provide two examples below. See, e.g., Chen (7, ) for additional applications. Example. (Partially linear ARX regression) Suppose that the time series data fy t g t= is generated by Y t = X ;t + h (X ;t ) + u t ; E [u t jx ;t ; X ;t ] = ; (.) where X ;t and X ;t are d and d dimensional random vectors respectively, and X ;t could include nitely many lagged Y t s. Let R d and h H a function space. Let = ( ; h ) A = H. Examples of functionals of interest could be f( ) = or h (x ) where R d some point in the support of X ;t. and x is Let X j be the support of X j for j = ;. For simplicity we assume that X is a convex and bounded subset of R d. For the sake of concreteness we let H = s (X ) (a Hölder space): ( s (X ) = h C [s] (X ) : sup sup h (j) h ([s]) (x) h ([s]) (x ) ) (x) < ; sup j[s] xx x;x X jx x j s [s] < ; where [s] is the largest integer that is strictly smaller than s. he Hölder space s (X ) (with s > :5d ) is a smooth function space that is widely assumed in the semi-nonparametric literature. We can then approximate H = s (X ) by various linear sieve spaces: 8 9 < kx = H = : h () : h () = j p j () = P k (); R k ; ; (.3) j= where the known sieve basis P k () could be tensor-products of splines, wavelets, Fourier series and others; see, e.g., Newey (997) and Chen (7). Let `(Z t ; ) = Y t X ;t h (X ;t) =4 with Zt = (Y t ; X ;t ; X ;t ) and = ( ; h) A = H. Let A = H be a sieve for A. We can estimate = ( ; h ) A by the sieve least squares (LS) (a special case of sieve M estimation): b ( b ; b h ) = arg max (;h)h X `(Z t ; ; h). (.4) t= A functional of interest f( ) (such as or h (x )) is then estimated by the plug-in sieve LS estimator f(b ) (such as b or b h (x )). his example is very similar to example in Chen and Shen (998) and example 4..3 in Chen (7). One can slightly modify their proofs to get the convergence rate of b and the p -asymptotic 5

8 normality of b. But neither paper provides a variance estimator for b. he results in our paper immediately lead to the asymptotic normality of f(b ) for possibly irregular functionals f( ) and provide simple, accurate inference on f( ). Example. (Possibly misspeci ed copula-based time series model) Suppose that fy t g t= is a sample of strictly stationary rst order Markov process generated from (F Y ; C (; )), where F Y is the true unknown continuous marginal distribution, and C (; ) is the true unknown copula for (Y t ; Y t ) that captures all the temporal and tail dependence of fy t g. he -th conditional quantile of Y t given Y t = (Y t ; :::; Y ) is: Q Y (y) = F Y C j [jf Y (y)] ; where C C (u; ) is the conditional distribution of U t F Y (Y t ) given U t = u, and C j [ju] is its -th conditional quantile. he conditional density function of Y t given Y t is p (jy t ) = f Y ()c (F Y (Y t ); F Y ()) ; where f Y () and c (; ) are the density functions of F Y () and C (; ) respectively. A researcher speci es a parametric form fc(; ; ) : g for the copula density function, but it could be misspeci ed in the sense c (; ) = fc(; ; ) : g. Let be the pseudo true copula dependence parameter: = arg max Z Z c(u; v; )c (u; v)dudv. Let ( ; f Y ) be the parameters of interest. Examples of functionals of interest could be, f Y (y), F Y (y) or Q Y : (y) = F Y C j [jf Y (y); ] for any R d and some y supp(y t ). We could estimate ( ; f Y ) by the method of sieve quasi ML using di erent parameterizations and di erent sieves for f Y. For example, let h = p f Y and = ( ; h ) be the (pseudo) true unknown parameters. hen f Y () = h () = R h (y) dy, and h L (R). For the identi cation of h, we can assume that h H: 8 < X H = : h () = p () + j p j () : j= 9 X = j < ; ; (.5) where fp j g j= is a complete orthonormal basis functions in L (R), such as Hermite polynomials, wavelets and other orthonormal basis functions. Here we normalize the coe cient of the rst basis function p () to be in order to achieve the identi cation of h (). Other normalization could also be used. It is now obvious that h H could be approximated by functions in the following sieve space: 8 9 < H = : h () = p Xk = () + j p j () = p () + P k () : R k ; : (.6) j= j= 6

9 Let Zt = (Y t ; Y t ), = ( ; h) A = H and ( ) ( h Z (Y t ) Yt `(Z t ; ) = log R + log c h (y) dy h (y) R h (x) dx dy; Z Yt!) h (y) R h (x) dx dy; : (.7) hen = ( ; h ) A = H could be estimated by the sieve quasi MLE b = ( b ; b h ) A = H that solves: ( ( )) X h (Y ) sup `(Z t ; ) + log R H t= h (y) dy O p (" ): (.8) A functional of interest f ( ) (such as, f Y (y) = h (y) = R h (y) dy, F Y (y) or Q Y : (y)) is then estimated by the plug-in sieve quasi MLE f (b ) (such as, b fy b (y) = b h (y) = R b h (y) dy, bf Y (y) = R y f b Y (y)dy or Q by : (y) = F b Y (C j [j F b Y (y); ])). b Under correct speci cation, Chen, Wu and Yi (9) establish the rate of convergence of the sieve MLE b and provide a sieve likelihood-ratio inference for regular functionals including f ( ) = or F Y (y) or Q Y : (y). Under misspeci ed copulas, by applying Chen and Shen (998), we can still derive the convergence rate of the sieve quasi MLE b and the p asymptotic normality of f(b ) for regular functionals. However, the sieve likelihood ratio inference given in Chen, Wu and Yi (9) is no longer valid under misspeci cation. he results in this paper immediately lead to the asymptotic normality of f(b ) (such as b f Y (y) = b h (y) = R b h (y) dy) for any possibly irregular functional f( ) (such as f Y (y)) as well as valid inferences under potential misspeci cation. 3 Asymptotic Normality of Sieve M-Estimators In this section, we establish the asymptotic normality of plug-in sieve M estimators of possibly irregular functionals of semi-nonparametric time series models. We also give a closed-form expression for the sieve Riesz representor that appears in our asymptotic normality result. 3. Local Geometry Given the existing consistency result (d(b ; ) = o p ()), we can restrict our attention to a shrinking d-neighborhood of. We equip A with an inner product induced norm k k that is weaker than d(; ) (i.e., k k cd(; ) for a constant c), and is locally equivalent to p E[`(Z t ; ) `(Z t ; )] in a shrinking d-neighborhood of. For strictly stationary weakly dependent data, Chen and Shen (998) establish the convergence rate kb k = O p ( ) = o =4. he convergence rate result implies that b B B with probability approaching one, where B f A : k k C log(log( ))g; B B \ A : (3.) Hence, we can now regard B as the e ective parameter space and B as its sieve space. 7

10 De ne Let V clsp (B ) ; arg min B jj jj: (3.) f ; g, where clsp (B ) denotes the closed linear span of B under kk. hen V is a nite dimensional Hilbert space under kk. Similarly the space V clsp (B ) f g is a Hilbert space under kk. Moreover, V is dense in V under kk. o simplify the presentation, we assume that dim(v ) = dim(a ) k, all of which grow to in nity with. By de nition we have h ; ; v i = for all v V. As demonstrated in Chen and Shen (998) and Chen (7), there is lots of freedom to choose such a norm k k that is weaker than d(; ) and is locally equivalent to p E[`(Z; ) `(Z; )]. In some parts of this paper, for the sake of concreteness, we present results for a speci c choice of the norm kk. We suppose that for all in a shrinking d-neighborhood of, `(Z; ) `(Z; ) can be approximated by (Z; )[ ] such that (Z; )[ ] is linear in. Denote the remainder of the approximation as: r(z; )[ ; ] f`(z; ) `(Z; ) (Z; )[ ]g : (3.3) When lim! [(`(Z; + [ ]) `(Z; ))=] is well de ned, we could let (Z; )[ ] = lim! [(`(Z; + [ ]) `(Z; ))=], which is called the directional derivative of `(Z; ) at in the direction [ ]. De ne with the corresponding inner product h; i k k = p E ( r(z; )[ ; ]) (3.4) h ; i = E f r(z; )[ ; ]g (3.5) for any ; in the shrinking d-neighborhood of. In general this norm de ned in (3.4) is weaker than d (; ). Since is the unique maximizer of E[`(Z; )] on A, under mild conditions k de ned in (3.4) is locally equivalent to p E[`(Z; ) `(Z; )]. For any v V; we de [v] to be the pathwise (directional) derivative of the functional f () at and in the direction of v = V [v] + k for any v V: (3.6) = For any v = ; V ; we [v ] [ [ ; ]: (3.7) [] is also a linear functional on V : 8

11 Note that V is a nite dimensional Hilbert space. As any linear functional on a nite dimensional Hilbert space is bounded, we can invoke the Riesz representation theorem to deduce that there is a v V such that and [v] = hv ; vi for all v V [v ] = kv k = sup vv ;v6= [v]j kvk (3.9) We call v the sieve Riesz representor of the [] on V. We emphasize that the sieve Riesz representation (3.8) (3.9) of the linear [] on V always exists regardless of [] is bounded on the in nite dimensional space V or not. [] is bounded on the in nite dimensional Hilbert space V, i.e. kv k sup vv;v6= ) [v]j= then kv k = O () (in fact kv k % kv k < and kv < ; (3.) v k! as! ); we say that f () is regular (at = ). In this case, we [v] = hv ; vi for all v V, and v is the Riesz representor of the [] on V. [] is unbounded on the in nite dimensional Hilbert space V, i.e. sup vv;v6= ) [v]j= then kv k % as! ; and we say that f () is irregular (at = ). = ; (3.) As it will become clear later, the convergence rate of f(b ) f ( ) depends on the order of kv k. 3. Asymptotic Normality o establish the asymptotic normality of f(b ) for possibly irregular nonlinear functionals, we assume: Assumption 3. (local behavior of functional) (i) sup B f( ) [ ] = o kv k ; [ ; ] = o kv k : Assumption 3..(i) controls the linear approximation error of possibly nonlinear functional f (). It is automatically satis ed when f () is a linear functional, but it may rule out some highly nonlinear functionals. Assumption 3..(ii) controls the bias part due to the nite dimensional sieve approximation 9

12 of ; to. It is a condition imposed on the growth rate of the sieve dimension dim(a ), and requires that the sieve approximation error rate is of smaller order than kv k. When f () is a regular functional, we have kv k % kv k <, and since h ; ; v i = (by de nition of ; ), we [ ; ] = jhv ; ; ij = jhv v ; ; ij kv v k k ; k ; thus Assumption 3..(ii) is satis ed if kv v k k ; k = o( = ) when f () is regular, (3.) which is similar to condition 4.(ii)(iii) imposed in Chen (7, p. 56) for regular functionals. Next, we make an assumption on the relationship between kv k and the asymptotic standard deviation of f(b ) f( ; ): It will be shown that the asymptotic standard deviation is the limit of the standard deviation (sd) norm kv k sd of v, de ned as kv k sd V ar p X t= (Z t ; )[v ]! : (3.3) Note that kv k sd is the nite dimensional sieve version of the long run variance of the score process (Z t ; )[v ]: Since v V, the sd norm kv k sd depends on the sieve dimension dim(a ) that grows with the sample size. Assumption 3. (sieve variance) kv k = kv k sd = O () : By de nition of kv k given in (3.9), < kv k is non-decreasing in dim(v ), and hence is nondecreasing in. Assumption 3. then implies that lim inf! kv k sd >. De ne u v v (3.4) sd to be the normalized version of v. hen Assumption 3. implies that ku k = O(). Let fg (Z)g P t= [g (Z t) Eg (Z t )] denote the centered empirical process indexed by the function g. Let " = o( = ): For notational economy, we use the same " as that in (.). Assumption 3.3 (local behavior of criterion) (i) f(z; ) [v]g is linear in v V; (iii) (ii) sup f`(z; " u ) `(Z; ) (Z; )[" u ]g = O p (" ); B E[`(Z t; ) `(Z t ; " u jj " u jj jj jj )] = O(" ): sup B

13 Assumptions 3.3.(ii) and (iii) are essentially the same as conditions 4. and 4.3 of Chen (7, p. 56) respectively. In particular, the stochastic equicontinuity assumption 3.3.(ii) can be easily veri ed by applying Lemma 4. of Chen (7). Assumption 3.4 (CL) p f(z; ) [u ]g! d N(; ), where N(; ) is a standard normal distribution. Assumption 3.4 is a very mild one, which e ectively combines conditions 4.4 and 4.5 of Chen (7, p. 56). his can be easily veri ed by applying any existing triangular array CL for weakly dependent data (see, e.g., White (4) for references). We are now ready to state the asymptotic normality theorem for the plug-in sieve M estimator. heorem 3. Let Assumptions 3..(i), 3. and 3.3 hold. hen p f(b ) f( ; ) = p f(z; ) [u ]g + o p () ; (3.5) sd v If further Assumptions 3..(ii) and 3.4 hold, then p f(b ) f( ) v sd = p f(z; ) [u ]g + o p ()! d N(; ): (3.6) In light of heorem 3., we call kv k sd de ned in (3.3) the pre-asymptotic sieve variance of the estimator f(b ). When the functional f( ) is regular (i.e., kv k = O()), we have kv k sd kv k = O() typically; so f(b ) converges to f( ) at the parametric rate of = p. When the functional f( ) is irregular (i.e., kv k! ), we have kv k sd! (under Assumption 3.); so the convergence rate of f(b ) becomes slower than = p. Regardless of whether the pre-asymptotic sieve variance kv k sd stays bounded asymptotically (i.e., as! ) or not, it always captures whatever true temporal dependence exists in nite samples. Note that kv k sd = V ar ((Z; )[v ]) if either the score process f(z t; )[v ]g t is a martingale di erence array or if data fz t g t= is iid. herefore, heorem 3. recovers the asymptotic normality result in Chen and Liao (8) for sieve M estimators of possibly irregular functionals with iid data. For regular functionals of semi-nonparametric time series models, Chen and Shen (998) and Chen (7, heorem 4.3) establish that p (f(b ) f( ))! d N(; v ) with! v = lim! V ar p X t= (Z t ; )[v ] = lim! kv k sd (; ): (3.7) Our heorem 3. is a natural extension of their results to allow for irregular functionals as well.

14 3.3 Sieve Riesz Representor o apply the asymptotic normality heorem 3. one needs to verify Assumptions Once we compute the sieve Riesz representor v V, Assumptions 3. and 3. can be easily checked, while Assumptions 3.3 and 3.4 are standard ones and can be veri ed in the same ways as those in Chen and Shen (998) and Chen (7) for regular functionals of semi-nonparametric models. Although it may be di cult to compute the Riesz representor v V in a closed form for a regular functional on the in nite dimensional space V (see e.g., Ai and Chen (3) for discussions), we can always compute the sieve Riesz representor v V de ned in (3.8) and (3.9) explicitly. herefore, heorem 3. is easily applicable to a large class of semi-nonparametric time series models, regardless of whether the functionals of interest are p estimable or not Sieve Riesz representors for general functionals For the sake of concreteness, in this subsection we focus on a large class of semi-nonparametric models where the population criterion E[`(Z t ; ; h ())] is maximized at = ( ; h ()) A = H, is a compact subset in R d, H is a class of real valued continuous functions (of a subset of Z t ) belonging to a Hölder, Sobolev or Besov space, and A = H is a nite dimensional sieve space. he general cases with multiple unknown functions require only more complicated notation. Let kk be the norm de ned in (3.4) and V = R d fv h () = P k () : R k g be dense in the in nite dimensional Hilbert space (V; kk). By de nition, the sieve Riesz representor v = (v; ; v h; ()) = (v; ; P k () ) V ) [] solves the following [v ] = kv v = sup [v h()] v=(v ;v h) E ( r (Z V ;v6= t ; ; h ()) [v; v]) F k Fk = sup =(v ; ) R d +k ;6= R k ; (3.8) where is a (d + k ) vector, 3 and F [P k () ] (3.9) R k E ( r (Z t ; ; h ()) [v; v]) for all v = v ; P k () V ; (3.) with I I R k = ; I ; I ; and R I k := I I I (3.) 3 [] applies to a vector (matrix), it stands for element-wise (column-wise) operations. We follow the same convention for other operators such as (Z t; ) [] and r (Z t; ) [; ] throughout the paper.

15 being (d +k )(d +k ) positive de nite matrices. For example if the criterion function `(z; ; h ()) is h twice continuously pathwise di erentiable with respect to (; h ()), then we have I = E h i h i I ; = t; ;h [P k (); P k () ], I ; = t; ;h [P k ()] and I ; I;. he sieve Riesz representation (3.8) becomes: for all v = (v ; P k () ) t; ;h [v] = F k = hv ; vi = R k for all = (v ; ) R d +k : (3.) It is obvious that the optimal solution of in (3.8) or in (3.) has a closed-form expression: = v he sieve Riesz representor is then given by Consequently, ; ; v = v ; ; v h; () = v ; ; P k () V : which is nite for each sample size but may grow with. hus Finally the score process can be expressed as = R k F k : (3.3) kv k = R k = F k R k F k ; (3.4) (Z t ; )[v ] = (Z t ; ; h ()) ; h (Z t ; ; h ())[P k () ] S k (Z t ) : V ar ((Z t ; )[v ]) = E S k (Z t )S k (Z t ) (3.5) and kv k sd = V ar p P t= S k (Z t ) : o verify Assumptions 3. and 3. for irregular functionals, it is handy to know the exact speed of divergence of kv k. We assume Assumption 3.5 he smallest and largest eigenvalues of R k bounded away from zero uniformly for all k. de ned in (3.) are bounded and Assumption 3.5 imposes some regularity conditions on the sieve basis functions, which is a typical assumption in the linear sieve (or series) literature. For example, Newey (997) makes similar assumptions in his paper on series LS regression. Remark 3. Assumption 3.5 implies that jjv jj jj jj E jjf k jj E = ) jj E + ) [P k ()]jj E. hen: f() is regular at = if lim k [P k ()]jj E < ; f() is irregular at = if lim k [P k ()]jj E =. 3

16 3.3. Examples We rst consider three typical linear functionals of semi-nonparametric models. For the Euclidean parameter functional f() =, we have F k = ( ; k ) with k = [; :::; ] k, and hence v = (v ; ; P k () ) V with v; = I, = I, and kv k = F k R k F k = I : If the largest eigenvalue of I, max(i ), is bounded above by a nite constant uniformly in k ; then kv k max (I ) < uniformly in, and the functional f() = is regular. For the evaluation functional f() = h(x) for x X, we have F k = ( d ; P k (x) ), and hence v = (v ; ; P k () ) V with v ; = I P k (x), = I P k (x), and kv k = F k R k F k = P k (x)i P k (x) : So if the smallest eigenvalue of I, min(i ), is bounded away from zero uniformly in k, then kv k min (I )jjp k (x)jj E! ; and the functional f() = h(x) is irregular. For the weighted integration functional f() = R X w(x)h(x)dx for a weighting function w(x), we have F k = ( d ; R X w(x)p k (x) dx), and hence v = (v ; ; P k () ) with v; = R I X w(x)p k (x)dx, = R I X w(x)p k (x)dx, and Z Z kv k = Fk R k F k = w(x)p k (x)dx I w(x)p k (x)dx: Suppose that the smallest and largest eigenvalues of I X X are bounded and bounded away from zero uniformly for all k. hen jjv jj jj R X w(x)p k (x)dxjj E. hus f() = R X w(x)h(x)dx is regular if lim k jj R X w(x)p k (x)dxjj E < ; is irregular if lim k jj R X w(x)p k (x)dxjj E =. We nally consider an example of nonlinear functionals that arises in Example. when the parameter of interest is = ( ; h ) with h = f Y being the true marginal density of Y t. Consider the functional f() = h (y) = R h (y) dy. Note that f( ) = f Y (y) = h (y) and h () is approximated by the linear sieve H given in (.6). hen F k = d [P k () ] [P k ()] = h (y) P k (y) and hence v = (v ; ; P k () ) V with v; = I kv k = F k R k F k ) Z h (y) h (y) P k (y)dy ) [P k () ]I [P k ()], = [P k [P k ()], and So if the smallest eigenvalue of I is bounded away from zero uniformly in k, then kv k const: [P k ()]jj E! ; and the functional f () = h (y) = R h (y) dy is irregular at = : 4

17 4 Asymptotic Variance of Sieve Estimators of Irregular Functionals In this section, we derive the asymptotic expression of the pre-asymptotic sieve variance kv k sd for irregular functionals. We provide general su cient conditions under which the asymptotic variance does not depend on the temporal dependence. We also show that evaluation functionals and some weighted integrals satisfy these conditions. 4. Exact Form of the Asymptotic Variance By de nition of the pre-asymptotic sieve variance jjv jj sd fz t g t=, we have: jjv jj sd = V ar ((Z; X )[v ]) + t= " X = V ar ((Z; )[v ]) + t= and the strict stationarity of the data t E ((Z ; )[v ](Z t+ ; )[v ]) (4.) # t (t) ; where f (t)g is the autocorrelation coe cient of the triangular array f(z t; )[v ]g t : (t) E ((Z ; )[v ](Z t+; )[v ]) V ar (Z; )[v : (4.) ] Loosely speaking, one could say that the triangular array f(z t ; )[v ]g t is weakly dependent if X t= hen we have jjv jj sd = O fv ar ((Z; )[v ])g. t (t) = O(): (4.3) When f() is irregular, we have kv k! as dim(v )! (as! ). his and Assumption 3. imply that kv k sd! ; and so V ar ((Z; )[v ])! under (4.3) as! for irregular functionals. In this section we provide some su cient conditions to ensure that, as!, although the variance term blows up (i.e., V ar ((Z; )[v ])! ), the individual autocovariance term stays bounded or diverges at a slower rate, and hence the sum of autocorrelation coe cients becomes asymptotically negligible (i.e., P t= (t) = o()). In the following we denote C sup je f(z ; )[v ](Z t+ ; )[v ]gj : t[; ) Assumption 4. (i) kv k! as!, and kv k =V ar ((Z; )[v ]) = O(); (ii) here is an increasing integer sequence fd [; )g such that d C X (a) V ar (Z; )[v = o() and (b) ] t=d t (t) = o(): 5

18 More primitive su cient conditions for Assumption 4. are given in the next subsection. heorem 4. Let Assumption 4. hold. hen: kv k sd V ar((z; )[v ]) = o (); If further Assumptions 3., 3.3 and 3.4 hold, then p [f(b ) f( )] qv! ar (Z; )[v ] d N (; ) : (4.4) heorem 4. shows that when the functional f() is irregular (i.e., kv k! ), time series dependence does not a ect the asymptotic variance of a general sieve M estimator f(b ). Similar results have been proved for nonparametric kernel and local polynomial estimators of evaluation functionals of conditional mean and density functions. See for example, Robinson (983) and Masry and Fan (997). However, whether this is the case for general sieve M estimators of unknown functionals has been a long standing question. heorem 4. gives a positive answer. his may seem surprising at rst sight as sieve estimators are often regarded as global estimators while kernel estimators are regarded as local estimators. One may conclude from heorem 4. that the results and inference procedures for sieve estimators carry over from iid data to the time series case without modi cations. However, this is true only when the sample size is large. Whether the sample size is large enough so that we can ignore the temporal dependence depends on the functional of interest, the strength of the temporal dependence, and the sieve basis functions employed. So it is ultimately an empirical question. In any nite sample, the temporal dependence does a ect the sampling distribution of the sieve estimator. In the next section, we design an inference procedure that is easy to use and at the same time captures the time series dependence in nite samples. 4. Su cient Conditions for Assumption 4. In this subsection, we rst provide su cient conditions for Assumption 4. for sieve M estimation of irregular functionals f( ) of general semi-nonparametric models. We then present additional low-level su cient conditions for sieve M estimation of real-valued functionals of purely nonparametric models. We show that these su cient conditions are satis ed for sieve M estimation of the evaluation and the weighted integration functionals. 4.. Irregular functionals of general semi-nonparametric models Given the closed-form expressions of kv k and V ar ((Z; )[v ]) in Subsection 3.3, it is easy to see that the following assumption implies Assumption 4..(i). Assumption 4. (i) Assumption 3.5 holds and lim k [P k ()]jj E = ; (ii) he smallest eigenvalue of E [S k (Z t )S k (Z t ) ] in (3.5) is bounded away from zero uniformly for all k. 6

19 Next, we provide some su cient conditions for Assumption 4..(ii). Let f Z ;Z t (; ) be the joint density of (Z ; Z t ) and f Z () be the marginal density of Z. Let p [; ). De ne k(z; )[v ]k p (E fj(z; )[v ]j p g) =p : (4.5) By de nition, k(z; )[v ]k = V ar ((Z; )[v ]). he following assumption implies Assumption 4..(ii)(a). Assumption 4.3 (i) sup t sup (z;z )ZZ jf Z ;Z t (z; z ) = [f Z (z) f Zt (z )]j C for some constant C > ; (ii) k(z; )[v ]k = k(z; )[v ]k = o(). Assumption 4.3.(i) is mild. When Z t is a continuous random variable, it is equivalent to assuming that the bivariate copula density of (Z ; Z t ) is bounded uniformly in t. For irregular functionals (i.e., kv k % ), the L (f Z ) norm k(z; )[v ]k diverges (under Assumption 4..(i) or Assumption 4.), Assumption 4.3.(ii) requires that the L (f Z ) norm k(z; )[v ]k diverge at a slower rate than the L (f Z ) norm k(z; )[v ]k as k!. In many applications the L (f Z ) norm k(z; )[v ]k actually remains bounded as k! and hence Assumption 4.3.(ii) is trivially satis ed. he following assumption implies Assumption 4..(ii)(b). Assumption 4.4 (i) he process fz t g t= is strictly stationary strong-mixing with mixing coe cients (t) satisfying P t= t [ (t)] + < for some > and > ; (ii) As k! ; k(z; )[v ]k k(z; )[v ]k + (Z; )[v ] + = o () : he -mixing condition in Assumption 4.4.(i) with > + becomes Condition.(iii) in Masry and Fan (997) for the pointwise asymptotic normality of their local polynomial estimator of a conditional mean function. See also Fan and Yao (3, Condition.(iii) in section 6.6.). In the next subsection, we illustrate that > + is also su cient for sieve M estimation of evaluation functionals of nonparametric time series models to satisfy Assumption 4.4.(ii). Instead of the strong mixing condition, we could also use other notions of weak dependence, such as the near epoch dependence used in Lu and Linton (7) for the pointwise asymptotic normality of their local linear estimation of a conditional mean function. Proposition 4. Let Assumptions 4., 4.3 and 4.4 hold. hen: P t= j (t)j = o() and Assumption 4. holds. 4.. Irregular functionals of purely nonparametric models In this subsection, we provide additional low-level su cient conditions for Assumptions 4..(i), 4.3.(ii) and 4.4.(ii) for purely nonparametric models where the true unknown parameter is a real-valued func- 7

20 tion h () that solves sup hh E[`(Z t ; h(x t ))]. his includes as a special case the nonparametric conditional mean model: Y t = h (X t ) + u t with E[u t jx t ] =. Our results can be easily generalized to more general settings with only some notational changes. Let = h () H and let f() : H! R be any functional of interest. By the results in Subsection 3.3, f(h ) has its sieve Riesz representor given by: v () = P k () V with = ) [P k ()]; where R k is such that R k = E r (Z t ; h ) [ P k ; Pk ] = E er (Z t ; h (X t )) P k (X t )P k (X t ) for all R k. Also, the score process can be expressed as (Z t ; h )[v ] = (Z e t ; h (X t ))v (X t ) = (Z e t ; h (X t ))P k (X t ) : Here the notations (Z e t ; h (X t )) and er (Z t ; h (X t )) indicate the standard rst-order and secondorder derivatives of `(Z t ; h(x t )) instead of functional pathwise derivatives (for example, we have er (Z t ; h (X t )) = and (Z e t ; h (X t )) = [Y t h (X t )] = in the nonparametric conditional mean model). hus, kv k = E E[ er (Z; h (X)) jx](v (X)) = R k [P k () ) [P k ()]; n V ar ((Z; h )[v ]) = E E([ (Z; e h (X))] jx)(v (X)) o : It is then obvious that Assumption 4..(i) is implied by the following condition. Assumption 4.5 (i) inf xx E[ er (Z; h (X)) jx = x] c > ; (ii) sup xx E[ er (Z; h (X)) jx = x] c < ; (iii) the smallest and largest eigenvalues of E fp k (X)P k (X) g are bounded and bounded away from zero uniformly for all k, and lim k [P k ()]jj E = ; (iv) inf xx E([ e (Z; h (X))] jx = x) c 3 >. It is easy to see that Assumptions 4.3.(ii) and 4.4.(ii) are implied by the following assumption. Assumption 4.6 (i) E fjv (X)jg = O(); (ii) sup xx E (Z; e h (X)) + jx = x c 4 < ; (iii) (+)(+)= Efjv g (X)j Efjv (X)j + g = o(). It actually su ces to use ess-inf x (or ess-sup x ) instead of inf x (or sup x ) in Assumptions 4.5 and 4.6. We immediately obtain the following results. 8

21 Remark 4.3 () Let Assumptions 4.3.(i), 4.4.(i), 4.5 and 4.6 hold. hen: X j kv (t)j = o() and k sd V ar (Z; )[v ] = o (). t= () Assumptions 4.5 and 4.6.(ii) imply that V ar ((Z; )[v ]) E (v (X)) kv k jj jj E [P k ()]jj E! ; hence Assumption 4.6.(iii) is satis ed if EfjP k (X) j+ g=jj jj(+)(+) E = o(). Assumptions 4.3.(i), 4.4.(i), 4.5 and 4.6.(ii) are all very standard low level su cient conditions. In the following, we illustrate that Assumptions 4.6.(i) and (iii) are easily satis ed by two typical functionals of nonparametric models: the evaluation functional and the weighted integration functional. Evaluation functionals. For the evaluation functional f(h ) = h (x) with x X, we [P k ()] = P k (x), v () = P k () = P k () R k P k (x). hen kv k = Pk (x)r k P k (x) = v (x), and kv k jjp k (x)jj E! under Assumption 4.5.(i)(ii)(iii). We rst verify Assumption 4.6.(i): R xx jv (x)j f X (x) dx = O(): For the evaluation functional, we have, for any v V : where v (x) = hh ; v i = E fe[ er (Z; h (X)) jx]v (X)v (X)g Z v (x) (x; x) dx; (4.6) xx (x; x) = E[ er (Z; h (X)) jx = x]v (x) f X (x) (4.7) = E[ er (Z; h (X)) jx = x]p k (x)r k P k (x)f X (x) : By equation (4.6) (x; x) has the reproducing property on V, so it behaves like the Dirac delta function (x x) on V : See Appendix B for further discussions about the properties of (x; x). A direct implication is that v (x) concentrates in a neighborhood around x = x and maintains the same positive sign in this neighborhood. Using the de nition of (; ) in (4.7), we have Z Z jv sign (v (x)j f X (x) dx = (x)) E[ er (Z; h (X)) jx = x] (x; x) dx; xx xx where sign(v (x)) = if v (x) > and sign(v (x)) = if v (x) : Denote b (x) sign(v (x)) E[ er(z;h (X))jX=x]. hen sup xx jb (x)j c < under Assumption 4.5.(i) and R xx jv (x)j f X (x) dx = R xx b (x) (x; x) dx. 9

22 If b (x) V ; then we have, using equation (4.6): Z jv sign (v (x)j f X (x) dx = b (x) = (x)) E[ er (Z; h (X)) jx = x] c = O () : xx If b (x) = V but can be approximated by a bounded function ~v (x) V such that Z [b (x) ~v (x)] (x; x) dx = o(); xx then, also using equation (4.6), we obtain: Z Z Z jv (x)j f X (x) dx = ~v (x) (x; x) dx + xx hus Assumption 4.6.(i) is satis ed. xx = ~v (x) + o() = O () : xx [b (x) ~v (x)] (x; x) dx Next, we verify Assumption 4.6.(iii). Using the de nition of (; ) in (4.7), we have n E jv (X)j +o Z = xx jv (x)j+ sign (v (x)) E[ er (Z; h (X)) jx = x] (x; x) dx: Using the same argument for proving R xx jv (x)j f X (x) dx = O(); we can show that under mild conditions: On the other hand, n E jv (X)j o Z = herefore if + n E jv (X)j +o xx jv (x)j+ E[ er (Z; h (X)) jx = x] ( + o ()) = O jv (x)j + : Z jv (x)j f X (x) dx = xx E njv (X)j o (+)(+)= n E jv (X)j +o jv (x)j + v (x) E[ er (Z; h (X)) jx = x] (x; x) dx v (x): (+)(+)= = o() ( + )( + )= < ; which is equivalent to > =( + ): hat is, when > =( + ); Assumption 4.6.(iii) holds. he above arguments employ the properties of delta sequences, i.e. sequences of functions that converge to the delta distribution. It follows from (4.6) that b h (x) = R xx b h (x) (x; x) dx: When the sample size is large, the sieve estimator of the evaluation functional e ectively entails taking a weighted average of observations with the weights given by a delta sequence viz. (x; x) : he average is taken over a small neighborhood around x in the domain of X where there is no time series dependence. he observations X t that fall in this neighborhood are not necessarily close to each other in time. herefore this subset of observations has low dependence, and the contribution of their joint dependence to the asymptotic variance is asymptotically negligible.

23 Weighted integration functionals. For the weighted integration functional f(h ) = R X w(x)h (x)dx for a weighting function w(x), we [P k ()] = R X w(x)p k (x)dx, v () = P k () = P k () R R k X w(x)p k (x)dx. Suppose that the smallest and largest eigenvalues of R k are bounded and bounded away from zero uniformly for all k. hen kv k jj R X w(x)p k (x)dxjj E ; thus f(h ) = R X w(x)h (x)dx is irregular if lim k jj R X w(x)p k (x)dxjj E =. For the weighted integration functional, we have, for any h V : Z w(a)h (a) da = hh ; v i = E fe[ er (Z; h (X)) jx]h (X)v (X)g X Z Z h (x) f w(a) (a; x) dagdx; where hus, Z where xx xx X (a; x) = E[ er (Z; h (X)) jx = x]p k (a)r k P k (x)f X (x) : Z jv (x)j f X (x) dx = Z = Z = X Z f Z ax ax Z X w(a)pk (a)dagr k P k (x) f X (x) dx xx xx w(a)sign fw(a) (a; x)g b(a; x) (a; x) dadx; b(a; x) w (a) sign fw(a) (a; x)g E[ er (Z; h (X)) jx = x] : If b(; x) V ; then, under Assumption 4.5.(i), Z Z Z jv (x)j f X (x) dx = b(x; x)dx C xx xx (a; x) E[ er (Z; h (X)) jx = x] dadx xx jw (x)j dx for some constant C: If b(; x) = V ; then under some mild conditions, it can be approximated by ~w (; x) V with j ~w (a; x)j C jb(a; x)j for some constant C and Z Z [b(a; x) ~w (a; x)] (a; x) dadx = o(): In this case, we also have Z xx ax xx Z jv (x)j f X (x) dx xx Z ~w (x; x)dx C xx jw (x)j dx: So if R xx jw (x)j dx < ; we have R xx jv (x)j f X (x) dx = O () : Hence Assumption 4.6.(i) holds. It remains to verify Assumption 4.6.(iii). Note that EfjP k (X) j+ g E kp k (X)k + E jj jj(+)(+) E = O 4 jj jj(+)(+) E E k k+ E kp k (X)k + E R xx w (x) P k (x) dx (+) E E kp k (X)k + E = 3 jj jj(+) E 5 = o()

24 for su ciently large >, as jj R xx w (x) P k (x) dxjj E! : he minimum value of may depend on the weighting function w (x) : If sup xx kp k (x)k E = O (k ) ; which holds for many basis functions, and jj R xx w (x) P k (x) dxjj E k ; then EfjP k (X) j+ g=jj jj(+)(+) E = o() for any > : It follows from Remark 4.3 that Assumption 4.6.(iii) holds for the weighted integration functional. 5 Autocorrelation Robust Inference In order to apply the asymptotic normality heorem 3., we need an estimator of the sieve variance kv k sd. In this section we propose a simple estimator of kv k sd and establish the asymptotic distributions of the associated t statistic and Wald statistic. he theoretical sieve Riesz representor v is not known and has to be estimated. Let kk denote the empirical norm induced by the following empirical inner product hv ; v i = X r(z t ; b )[v ; v ]; (5.) t= for any v ; v V. We de ne an empirical sieve Riesz representor bv ; of the [] with respect to the empirical norm kk, i.e. [bv ] = sup vv ;v6= [v]j kvk < [v] = hv; bv i (5.3) for any v V. We next show that the theoretical sieve Riesz representor v can be consistently estimated by the empirical sieve Riesz representor bv under the norm kk. In the following we denote W fv V : kvk = g. Assumption 5. Let f g be a positive sequence such that = o(). (i) sup B ;v ;v W Efr(Z; )[v ; v ] r(z; )[v ; v ]g = O( ); (ii) sup B ;v ;v W fr(z; )[v ; v ]g = O p ( ); (iii) sup [v] = O( ): Assumption 5..(i) is a smoothness condition on the second derivative of the criterion function with respect to. In the nonparametric LS regression model, we have r(z; )[v ; v ] = r(z; )[v ; v ] for all and v ; v. Hence Assumption 5..(i) is trivially satis ed. Assumption 5..(ii) is a stochastic equicontinuity condition on the empirical process P t= r(z t; )[v ; v ] indexed by in the shrinking neighborhood B uniformly in v ; v W. Assumption 5..(iii) puts some smoothness condition on [v] with respect to in the shrinking neighborhood B uniformly in v W.