Efficient estimation using the Characteristic Function

Transcription

1 013s- Efficient estimation using the Characteristic Function Marine Carrasco, Rachidi Kotchoni Série Scientifique Scientific Series Montréal Juillet Marine Carrasco, Rachidi Kotchoni. ous droits réservés. All rights reserved. Reproduction partielle permise avec citation du document source, incluant la notice. Short sections may be quoted without explicit permission, if full credit, including notice, is given to the source.

2 CIRANO Le CIRANO est un organisme sans but lucratif constitué en vertu de la Loi des compagnies du Québec. Le financement de son infrastructure et de ses activités de recherche provient des cotisations de ses organisations-membres, d une subvention d infrastructure du Ministère du Développement économique et régional et de la Recherche, de même que des subventions et mandats obtenus par ses équipes de recherche. CIRANO is a private non-profit organization incorporated under the Québec Companies Act. Its infrastructure and research activities are funded through fees paid by member organizations, an infrastructure grant from the Ministère du Développement économique et régional et de la Recherche, and grants and research mandates obtained by its research teams. Les partenaires du CIRANO Partenaire majeur Ministère de l'enseignement supérieur, de la Recherche, de la Science et de la echnologie Partenaires corporatifs Autorité des marchés financiers Banque de développement du Canada Banque du Canada Banque Laurentienne du Canada Banque Nationale du Canada Banque Scotia Bell Canada BMO Groupe financier Caisse de dépôt et placement du Québec Fédération des caisses Desjardins du Québec Financière Sun Life, Québec Gaz Métro Hydro-Québec Industrie Canada Investissements PSP Ministère des Finances et de l Économie Power Corporation du Canada Rio into Alcan State Street Global Advisors ransat A.. Ville de Montréal Partenaires universitaires École Polytechnique de Montréal École de technologie supérieure ÉS HEC Montréal Institut national de la recherche scientifique INRS McGill University Université Concordia Université de Montréal Université de Sherbrooke Université du Québec Université du Québec à Montréal Université Laval Le CIRANO collabore avec de nombreux centres et chaires de recherche universitaires dont on peut consulter la liste sur son site web. Les cahiers de la série scientifique CS visent à rendre accessibles des résultats de recherche effectuée au CIRANO afin de susciter échanges et commentaires. Ces cahiers sont écrits dans le style des publications scientifiques. Les idées et les opinions émises sont sous l unique responsabilité des auteurs et ne représentent pas nécessairement les positions du CIRANO ou de ses partenaires. his paper presents research carried out at CIRANO and aims at encouraging discussion and comment. he observations and viewpoints expressed are the sole responsibility of the authors. hey do not necessarily represent positions of CIRANO or its partners. ISSN Partenaire financier

3 Efficient estimation using the Characteristic Function * Marine Carrasco, Rachidi Kotchoni Résumé / Abstract he method of moments proposed by Carrasco and Florens 000 permits to fully exploit the information contained in the characteristic function and yields an estimator which is asymptotically as efficient as the maximum likelihood estimator. However, this estimation procedure depends on a regularization or tuning parameter that needs to be selected. he aim of the present paper is to provide a way to optimally choose by minimizing the approximate mean square error AMSE of the estimator. Following an approach similar to that of Newey and Smith 004, we derive a higherorder expansion of the estimator from which we characterize the finite sample dependence of the AMSE on. We provide a data-driven procedure for selecting the regularization parameter that relies on parametric bootstrap. We show that this procedure delivers a root consistent estimator of Moreover, the data-driven selection of the regularization parameter preserves the consistency, asymptotic normality and efficiency of the CGMM estimator. Simulation experiments based on a CIR model show the relevance of the proposed approach. Mots clés/keywords : Conditional moment restriction, continuum of moment conditions, generalized method of moments, mean square error, stochastic expansion, ikhonov regularization. Codes JEL : C00, C13, C15. * An earlier version of this work was joint with Jean-Pierre Florens. We are grateful for his support. his paper has been presented at various conferences and seminars and we thank the participants for their comments, especially discussant Atsushi Inoue. Partial financial support from CRSH is gratefully acknowledged. University of Montreal, CIREQ, CIRANO, marine.carrasco@umontreal.ca. Corresponding author. University of Montreal, CIREQ, rachidi.kotchoni@umontreal.ca.

4 1 Introduction here is a one-to-one relationship between the characteristic function henceforth, CF and the probability distribution function of a random variable, the former being the Fourier transform of the latter. his implies that an inference procedure based on the empirical CF has the potential to be as effi cient as another one that exploits the likelihood function. Paulson et al used a weighted modulus of the difference between the theoretical CF and its empirical counterpart to estimate the parameters of the stable law. Feuerverger and Mureika 1977 studied the convergence properties of the empirical CF and suggested that "it may be a useful tool in numerous statistical problems". Since then, many interesting applications have been proposed, including Feuerverger and McDunnough 1981b,c, Koutrouvelis 1980, Carrasco and Florens 000, Chacko and Viceira 003 and Carrasco, Chernov, Florens, and Ghysels 007 henceforth, CCFG 007. For a quite comprehensive review of empirical CF-based estimation methods, see Yu 004. he CF provides a good alternative to econometricians when the likelihood function is not available in closed form. For example, some distributions in the α-stable family are naturally specified via their CFs while their densities are known in closed form only at isolated points of the parameter space see e.g. Nolan, 009. he density of the Variance-Gamma model of Madan and Seneta 1990 has an integral representation whereas its CF has a simple closed form expression. he transition density of a discretely sampled continuous time process is not available in closed form, except when its parameterization coincides with that of a square-root diffusion Singleton, 001. Even in this special case, the transition density takes the form of an infinite mixture of Gamma densities with Poisson weights. he same type of density arises in the autoregressive Gamma model studied in Gouriéroux and Jasiak 005. Ait-Sahalia and Kimmel 006 propose closed form approximations for the log-likelihood function of various continuous-time stochastic volatility models. But their method cannot be applied to other situations without solving a complicated Kolmogorov forward and backward equation. Interestingly, the conditional CF can be derived in closed form for all continuous-time stochastic volatility models. he CF, ϕ τ, θ, of a random vector x t R p t = 1,..., is nothing but the expectation of e iτ x t with respect the distribution of x t, where θ is the parameter that characterizes the distribution of x t, τ R p is the Fourier index and i is the imaginary number such that i =. Hence, a candidate moment condition for the estimation of θ 0 i.e., the true value of θ is given by h t τ, θ = e iτ x t Ee iτ x t. his moment condition is valid for all τ R p and hence, h t τ, θ is a moment function or a continuum of moment conditions. Feuerverger and McDunnough 1981b propose an estimation procedure that consists of minimizing a norm of the sample average of the moment function. heir objective function involves an optimal weighting function that depends on the true unknown likelihood function. Feuerverger and McDunnough 1981c apply the Generalized Method of Moments GMM to a discrete set of moment conditions obtained by restricting the continuous index τ R p to a discrete grid τ τ 1, τ,...τ N. hey show that the asymptotic variance of the resulting estimator can be made arbitrarily close to the Cramer-Rao bound by selecting the grid for τ suffi ciently fine and extended. Similar discretization approaches are used in Singleton 001 and Chacko and Viceira

5 003. However, the number of points in the grid for τ must not be larger than the sample size for the covariance matrix of the discrete set of moment conditions to be invertible. In particular, the first order optimality conditions associated with the discrete GMM procedure becomes ill-posed as soon as the grid τ 1, τ,...τ N is too refined or too extended. Intuitively, the discrete set of moment conditions {h t τ i, θ} N i=1 converges to the moment function τ h tτ, θ, τ R as this grid is refined and extended. As a result, it is necessary to apply operator methods in a suitable Hilbert space to be able to handle the estimation procedure at the limit. Carrasco and Florens 000 proposed a Continuum GMM henceforth, CGMM that permits to effi ciently use the whole continuum of moment conditions. Similarly to the classical GMM, the CGMM is a two-step procedure that delivers a consistent estimator at the first step and an effi cient estimator at the second step. he ideal unfeasible objective function of the second step CGMM is a quadratic form in an suitably defined Hilbert space with metrics K, where K is the asymptotic covariance operator associated with the moment function h t τ, θ. o obtain a feasible effi cient CGMM estimator, one replaces the operator K by an estimator K obtained from a finite sample. However, the latter empirical operator is degenerate and not invertible while its theoretical counterpart is invertible only on a dense subset of the reference space. o circumvent these diffi culties, Carrasco and Florens 000 resorted to a ikhonov-type regularized inverse of K, e.g. K α = K + αi K, where I is the identity operator and α is a regularization parameter. he CGMM estimator is root- consistent and asymptotically normal for any fixed and reasonably small value of α. However, asymptotic effi ciency is obtained only by letting α 1/ go to infinity and α go to zero as goes to infinity. he main objective of this paper is to characterize the optimal rate of convergence for α as goes to infinity. o this end, we derive a Nagar 1959 type stochastic expansion of the CGMM estimator. his type of expansion has been used in Newey and Smith 004 to study the higher order properties of empirical likelihood estimators. We use our expansion to find the convergence rates of the higher order terms of the MSE of the CGMM estimator. hese rates depend on both α and. We find that the higher order bias of the CGMM estimator is dominated by two higher order variance terms. By equating the rates of these dominant term, we find an expression of the form α = c θ 0 gβ, where c θ 0 does not depend on and g β inherits some properties from the covariance operator K. o implement the optimal selection of α empirically, we advocate a naive estimator of α obtained by minimizing an approximate MSE of the CGMM estimator obtained by parametric bootstrap. Even though the CGMM estimator is consistent, there is a concern that its variance be infinite in finite sample for certain data generating processes. his concern seems unfounded for the CIR model on which our Monte Carlo simulations are based. If applicable, this diffi culty is avoided by truncating the AMSE similarly as in Andrews he remainder of the paper is organized as follows. In Section, we review the properties of the CGMM estimator in IID and Markov cases. In Section 3, we derive a higher-order expansion for the MSE of the CGMM estimator and use this expansion to obtain the optimal rate of convergence for the regularization parameter α. In Section 4, we describe a simulation-based method to estimate α and show the consistency of the resulting estimator. Section 5 presents a simulation study based on the 3

6 CIR term structure model and Section 6 concludes. he proofs are collected in appendix. Overview of the CGMM his section is essentially a summary of known results about the CGMM estimator. he first subsection present a general framework for implementing the CF-based CGMM procedure whilst the second subsection presents the basic properties of the resulting estimator..1 he CGMM Based on Characteristic function Let x t R p be a random vector process whose distribution is indexed by a finite dimensional parameter θ with true value θ 0. When the process x t is IID, Carrasco and Florens 000 propose to estimate θ 0 based on the moment function given by: h t τ, θ = e iτ x t+1 ϕτ, θ, 1 where ϕτ, θ = E θ e iτ x t+1 is the CF of x t and E θ is the expectation operator with respect to the data generating process indexed by θ. CCFG 007 extend the scope of the CGMM procedure to Markov and weakly dependent models. In this paper, we restrict our attention to IID and Markov cases. he moment function used in CCFG 007 for the Markov case is: h t τ, θ = e is x t+1 ϕ t s, θ e ir x t. where ϕ t s, θ = E θ e is x t+1 x t is the conditional CF of x t and τ = s, r R p. In equation, the set of basis functions {e ir x t } is being used as instruments. CCFG 007 show that these instruments are optimal given the Markovian structure of the model. Moment conditions defined by 1 are IID whereas equation describes a martingale difference sequence. Note that a standard conditional moment restriction i.e., non CF-based can be converted into a continuum of moment unconditional moment restrictions featuring. In this case, the CGMM estimator may be viewed as an alternative to the estimator proposed by Dominguez and Lobato 004 and to the smooth minimum distance estimator of Lavergne and Patilea 008. Subsequently, we use the generic notation h t τ, θ, τ R d to denote a moment function defined by either 1 or, where d = p for 1 and d = p for. Let π be a probability distribution function on R d and L π be the Hilbert space of complex valued functions that are square integrable with respect to π, i.e.: L π = {f : R d C fτfτπτdτ < }, 3 where fτ denotes the complex conjugate of fτ. As h t., θ for all θ Θ, the function h t., θ 4

7 belongs to L π for all θ Θ and for any finite measure π. Hence, we consider the following scalar product on L π L π: f, g = fτgτπτdτ. 4 Based on this notation, the effi cient CGMM estimator is given by θ = arg min K ĥ., θ, ĥ., θ. θ where K is the asymptotic covariance operator associated with the moment conditions. K is an integral operator and satisfies: Kf τ 1 = where kτ 1, τ is the kernel given by: kτ 1, τf τ π τ dτ, for any f L π, 5 kτ 1, τ = E h t τ 1, θh t τ, θ. 6 Some basic properties of the operator K are discussed in Appendix A. With a sample of size and a consistent first step estimator θ 1 in hand, one estimates kτ 1, τ by: k τ 1, τ, θ 1 = 1 h t τ 1, θ 1 h t τ, θ 1. 7 t=1 In the specific case of IID data, an estimator of the kernel that does not use a first step estimator is given by: k τ 1, τ = 1 t=1 e iτ 1 xt ϕ τ 1 e iτ xt ϕ τ. 8 where ϕ τ 1 = 1 t=1 eiτ 1 xt. Unfortunately, an empirical covariance operator K with kernel function given by either 7 or 8 is degenerate and not invertible. Indeed, the inversion of K raises a problem similar to one of the Fourier inversion of an empirical characteristic function. his problem is worsened by the fact that the inverse of K which K is aimed at estimating exists only on a dense subset of L π. Moreover, when K f = g exists for a given function f, a small perturbation in f may give rise to a large variation in g. o circumvent these diffi culties, we consider estimating K by: K α = K + αi K, where the hyperparameter α plays two roles. First, it is a smoothing parameter as it allows K α f to exist for all f in L π. Second, it is a regularization parameter as it dampens the sensitivity of K α f to perturbations in the input f. For any function f in the range of K and any consistent estimator 5

8 f of f, K f α converges to K f as goes to infinity and α goes to zero at appropriate rate. he expression for K α uses a ikhonov regularization, also called ridge regularization. Other forms of regularization could have been used, see e.g. Carrasco, Florens and Renault 007. he feasible CGMM estimator is given by: θ α = arg min θ Q α, θ, 9 where Q α, θ = K α ĥ., θ, ĥ., θ. An expression of the objective function Q α, θ in matrix form is given in CCFG 007a, Section 3.3. An alternative expression and a numerical algorithm for the numerical evaluation of this objective function based on Gauss-Hermite quadratures is described in Appendix D.. Consistency and Asymptotic Normality In order to study the properties of a CGMM estimator obtained within the framework described previously, the following assumptions are posited: Assumption 1: he probability density function π is strictly positive on R d and admits all its moments. Assumption : he equation E θ 0 h t τ, θ = 0 for all τ R d, π almost everywhere, has a unique solution θ 0 which is an interior point of a compact set Θ. Assumption 3: h t τ, θ is three times continuously differentiable with respect to θ. Furthermore, the first two derivatives satisfy: for all j, k and. 1 V ar t=1 h t τ, θ 1 < and V ar θ j t=1 h t τ, θ <, θ j θ k Assumption 4: E θ 0 h., θ Φ β for all θ Θ and for some β 1, and the first two derivatives of E θ 0 h., θ w.r.t. θ belong to Φ β for all θ in a neighborhood of θ 0 and for the same β as previously, where: Φ β = { f L π such that K β f } < Assumption 5: he random variable x t is stationary Markov and satisfies x t = r x t, θ 0, ε t where r x t, θ 0, ε t is three times continuously differentiable with respect to θ 0 and ε t is a IID white noise whose distribution is known and does not depend on θ 0. Assumption 1 and are quite standard and they have been used in Carrasco and Florens 000. he first part of Assumption 3 ensures some smoothness properties for θ α while the second part is always satisfied for IID models. 10 he largest real β such that f Φ β in Assumption 4 may be 6

9 called the level of regularity of f with respect to K: the larger β is, the better f is approximated by a linear combination of the eigenfunctions of K associated with the highest eigenvalues. Because Kf. involve a d-dimensional integration, β may be affected by both the dimensionality of the index τ and the smoothness of f. CCFG 007 have shown that we always have β 1 if f = E θ 0 h t τ, θ. Assumption 5 implies that the data can be simulated upon knowing how to draw from the distribution of ε t. It is satisfied for all random variables that can be written as a location parameter plus a scale parameter time a standardized representative of the family of distribution. Examples include the exponential family and the stable distribution. he IID case is a special case of Assumption 5 where r x t, θ 0, ε t takes the simpler form r θ 0, ε t. Further discussions on this type of model can be found in Gourieroux, Monfort, and Renault 1993 in the indirect inference context. Note that the function r x t, θ 0, ε t may not be available in analytical form. In particular, the relation x t = r x t, θ 0, ε t can be the numerical solution of a general equilibrium asset pricing model e.g., as in Duffi e and Singleton, We have the following results: heorem 1 Under Assumptions 1 to 5, the CGMM estimator is consistent and satisfies: 1/ θ α θ 0 L N0, I θ 0. as and α 1/ go to infinity and α goes to zero, where I θ 0 Matrix. denotes the inverse of the Fisher Information See Proposition 3. of CCFG 007 for a more general statement of the consistency and asymptotic normality result. A nice feature about the CGMM estimator is that its asymptotic distribution does not depend on the probability density function π. 3 Stochastic expansion of the CGMM estimator he conditions required for the asymptotic effi ciency result stated by heorem 1 allow for a wide range of convergence rates for α. Indeed, any sequence of type α = c a with c > 0 satisfies these conditions as soon as 0 < a < 1/. Among the admissible convergence rates, we would like to find the one that minimizes the mean square error of the CGMM estimator for a given sample size. o achieve this, we consider deriving the stochastic expansion of the CGMM estimator. he higher order properties of GMM-type estimators have been studied by Rothenberg 1983, 1984, Koenker et al. 1994, Rilstone et al and Newey and Smith 004. For estimators derived in the linear simultaneous equation framework, examples include Nagar 1959, Buse 199 and Donald and Newey 001. he approach followed here is similar to Nagar 1959 and Newey and Smith 004, which tries to approximate the MSE of an estimator analytically based on the leading terms of its stochastic expansion. wo diffi culties arise when analyzing the terms of the expansion of the CGMM estimator. First, when the rate of α as a function of is unknown, it is not always possible to write the terms of the 7

10 expansion in decreasing order. he second diffi culty stems from a result that dramatically differs from the case with a finite number of moment conditions. Indeed, when the number of moment conditions is finite, the quadratic form ĥ θ 0 K ĥ θ 0 is O p 1 and follows asymptotically a chi-square distribution with degrees of freedom given by the number of moment conditions. However, the analogue of the previous quadratic form, K / ĥ θ 0, is not well defined in the presence of a continuum of moment conditions. Its regularized version, K / ĥ θ 0, exists but diverges as goes to infinity and α goes to zero. Indeed, we have Kα / ĥ θ 0 α K + αi /4 K + αi /4 K 1/ ĥ θ 0 }{{}}{{}}{{} α /4 = O p α /4. 1 =O p1 11 he expansion that we derive for θ α θ 0 is of the same form for both the IID and Markov cases. Namely: θ α θ 0 = o p α β min1, + o p α / 1 where 1 = O p / β min1,, = O p α / and 3 = O p α. Appendix B provides details about the above expansion whose validity is ensured by the consistency result of heorem 1. In deriving the expansion above, we wish to find the rate of convergence of the α which minimizes the leading terms of the MSE: [ ] MSE α, θ 0 = E θ α θ 0 θ α θ 0 13 We have the following results on the higher order MSE matrix and on the optimal convergence rate for the regularization parameter. heorem Assume that Assumptions 1 to 5 hold. hen we have: i he approximate MSE matrix of θ α up to order O α / henceforth, AMSE is decomposed as the sum of the squared bias and variance: where as, α and α 0. AMSE α, θ 0 = Bias Bias + V ar Bias Bias = O α, V ar = I θ 0 + O β min, α + O α /. 8

11 ii he α that minimizes the trace of AMSE α, θ 0, denoted α α θ 0, satisfies: Remarks. α = O max 1 6, 1 β We have the usual trade-off between a term that is decreasing in α and another that is increasing in α. Interestingly, the squared bias term is dominated by two higher order variance terms whose rates are equated to obtain the optimal rate for the regularization parameter. he same situation happens for the Limited Information Maximum Likelihood estimator for which the bias is also dominated by variance terms see Donald and Newey, he rate for the O β min, α variance term does not improve for β >.5. his is due to a property of ikhonov regularization that is well documented in the literature on inverse problems, see e.g. Carrasco, Florens and Renault 007. he use of another regularization such as spectral cut-off or Landweber-Fridman would permit to improve the rate of convergence for large values of β. However, this improvement comes at the cost of a greater complexity in the proofs e.g. in the spectral cut-off, we lose the differentiability of the estimator with respect to α. 3. Our expansion is consistent with the condition of heorem 1, since the optimal regularization parameter α satisfies α. form: 4. It follows from heorem that the optimal regularization parameter α is necessarily of the α = c θ 0 gβ, 14 for some positive function c θ 0 that does not depend on and a positive function g β that satisfies max 1 6, 1 β+1 g β < 1/. An expression of the form 14 is often used as starting point for optimal bandwidth selection in nonparametric density estimation. Examples in the semiparametric context include Linton 00 and Jacho-Chavez Estimation of the Optimal Regularization parameter Our purpose is to select the regularization parameter α so as to minimize the trace of the MSE matrix of θ α for a given sample of size, i.e.: α θ 0 = arg min α [0,1] Σ α, θ 0, where Σ α, θ 0 = E θ α θ 0. his raises at least three problems. First, the MSE Σ α, θ 0 might be infinite in finite samples even though θ α is consistent. 1 Second, the true parameter value θ 0 is unknown. hird, the finite sample distribution of θ α θ 0 is not known even 1 his is due to the fact that θ α is a GMM-type estimator. he large sample properties of such estimators are well-known whilst their finite sample properties can be established only a special cases. 9

12 when θ 0 is known. Each of these problems is examined below. he variance of θ α may be infinite for some data generating processes. o hedge against such situations, one may consider a truncated MSE of the form: Σ α, θ 0, ν = E [ξ α, θ 0 ξ α, θ 0 < n ν ], 15 where ξ α, θ θ α θ and n ν satisfies ν = Pr ξ α, θ 0 > n ν. A similar approach has been used in Andrews 1991, p. 86. Given that the finite sample distribution of θ α is unknown in practice, it is convenient to first select the probability of truncation ν e.g., ν = 1% and then deduce the corresponding quantile n ν by simulation. o account for the possibility of the pair ν, n ν depending on α, one may consider instead: Σ α, θ 0, ν = 1 ν E [ξ α, θ 0 ξ α, θ 0 < n ν ] + νn ν, 16 which accounts for the probability mass at the truncation boundary. Note that the truncation will play no role if the MSE of θ α is finite. In this case, we simply let: Σ α, θ 0, 0 Σ α, θ 0 = E [ξ α, θ 0 ]. 17 As θ α is asymptotically normal, its second moment exists for large enough. Hence, the truncation disappears i.e., each of the expressions 15 and 16 converges to 17 if one let ν go to zero as goes to infinity. We define the optimal regularization parameter as: where Σ α, θ 0, ν is given by either 15, 16 or 17. α θ 0 = arg min α [0,1] Σ α, θ 0, ν, 18 Our strategy for estimating α θ 0 relies on approximating the unknown MSE by parametric bootstrap. Let θ 1 be the CGMM estimator of θ 0 obtained by replacing the covariance operator with the identity operator. his estimator is consistent and asymptotically normal albeit ineffi cient. We use θ 1 to simulate M independent samples of size, denoted X j θ 1 for j = 1,,..., M. It should be emphasized that we have adopted a fully parametric approach from the beginning by assuming that the model of interest is fully specified. Indeed, it would not be possible to obtain MLE effi ciency otherwise. he model can be simulated by exploiting Assumption 5, which stipulates that the data generating process satisfies x t = r x t, θ, ε t. o start with, one first generates M IID draws ε j t for j = 1,..., M and t = 1,..., from the known distribution of the errors. Next, M time-series of size are obtained by applying the recursion x j t = r x j t, θ 1, ε j t, t = 1,...,, from M arbitrary starting values x j 0. Using the simulated samples, one computes M IID copies of the CGMM estimator for any given As n ν as ν 0, one might be concerned by the limiting behavior of νn ν as ν 0 when Σ α, θ 0 is infinite. However, this is not an issue as long as n ν is finite for all finite. 10

13 α. We let θ j α, θ 1 denote the CGMM estimator computed from the j th sample. he truncated MSE given by 15 is estimated by: Σ M α, θ 1, ν = 1 ν M M ξ j, α, θ 1 1 ξ j, α, θ 1 n ν, 19 where ξ j, α, θ 1 = θ j α, θ 1 θ 1, ν is a probability selected by the econometrician and n ν satisfies: 1 M M 1 ξ j, α, θ 1 n ν = 1 ν, he truncated MSE based on the alternative Formula 16 is estimated by: Σ M α, θ 1, ν = M M ξ j, α, θ 1 1 ξ j, α, θ 1 n ν + ν n ν. 0 With no truncation, 1 and 19 are identical to the naive MSE estimator given by: Σ M α, θ 1 = M M ξ j, α, θ 1, 1 which is aimed at estimating 17. Finally, we select the optimal regularization parameter according to: where Σ M α, θ 1, ν Let Σ α, θ 1, ν and define: α M θ1 = arg min Σ M α, θ 1, ν, α [0,1] is either 19, 0 or 1. be the limit of Σ M α, θ 1, ν as the number of replications M goes to infinity α θ1 = arg minσ α, θ 1, ν. α [0,1] Note that α θ1 is a deterministic function of a stochastic argument while α M θ1 is doubly random, being a stochastic function of a stochastic argument. he estimator α θ1 is not feasible. However, its properties are the key ingredients for establishing the consistency of its feasible counterpart α M θ1. o pursue, we need the following assumption: Assumption 6: he regularization parameter α that minimizes the possibly truncated criterion Σ α, θ 0, ν is of the form α θ 0 = c θ 0 gβ, for some continuous positive function c θ 0 that does not depend on and a positive function g β that satisfies max 1 6, 1 β+1 g β < 1/. Basically, Assumption 6 requires that the optimal rate found for the regularization parameter at 14 11

14 be insensitive to the MSE truncation scheme. his assumption ensures that α θ1 = c θ1 gβ and is necessarily satisfied as goes to infinity and ν goes to zero. he following result can further be proved. heorem 3 Let θ 1 be a consistent estimator of θ 0. hen under Assumptions 1 to 5, α θ 1 α θ 0 1 converges in probability to zero as goes to infinity. In heorem 3, the function α. is deterministic and continuous but the argument θ 1 is stochastic. As goes to infinity, θ 1 gets closer and closer to θ 0, but at the same time α θ 0 converges to zero at some rate that depends on. his prevents us from claiming without caution that α θ 1 α θ 0 1 = o p1 since the denominator is not bounded away from zero. he next theorem characterizes the rate of convergence of α M θ 0 α θ 0. heorem 4 Under assumptions 1 to 5, α M θ 0 α θ 0 M goes to infinity and is fixed. 1 converges in probability to zero at rate M / as In heorem 4, α M θ 0 is the minimum of the empirical MSE simulated with the true θ 0. In the proof, one first shows that the conditions of the uniform convergence in probability of the empirical MSE are satisfied. Next, one uses heorem.1 of Newey and McFadden 1994 and the fact that α θ 0 is bounded away from zero for any finite to establish the consistency of α M θ 0 α θ 0. In the next theorem, we revisit the previous results when θ 0 is replaced by a consistent estimator θ 1. heorem 5 Let θ 1 be a consistent estimator of θ 0. hen under assumptions 1 to 5, α M θ 1 α θ 0 = O p / + O p M / as M goes to infinity first and goes to infinity second. he result of heorem 5 is obtained by using a sequential limit in M and, which is needed here because heorem 4 has been derived for fixed. Such sequential approach is often used in panel data econometrics, see for instance Phillips and Moon It is also used implicitly in the theoretical analysis of bootstrap. 3 heorem 5 implies that α M θ 1 benefits from an increase in both M and. he last theorem compares the feasible CGMM estimator based on α M to the unfeasible estimator θ α, where α is defined in 14. heorem 6 Let α M = α M θ 1 defined in. hen: θ α M θ α = O p gβ, provided that M. 3 he properties of a bootstrap estimator are usually derived using its bootstrap distribution, hence letting M go to infinity before. 1

15 Hence, theorem 6 implies that the distribution of θ α M θ 0 is the same as the distribution of θ α θ 0 to the order gβ. his ensures that replacing α by a consistent estimator α M such that α M α 1 = o p 1 does not affect the consistency, asymptotic normality and effi ciency of the final CGMM estimator θ α M. he proof of this theorem relies mainly on the fact that θ α is continuously differentiable with respect to α whilst the optimal α is bounded away from zero for any finite. Overall, our selection procedure for the regularization parameter is optimal and adaptive as it does not require the a priori knowledge of the regularity parameter β. 5 Monte Carlo Simulations he aim of this simulation study is to investigate the properties of the MSE function Σ M α, θ 1, ν as the regularization parameter α, the sample size and the number of replications M vary. For this purpose, we consider estimating the parameters of a square-root diffusion also known as the CIR diffusion by CGMM. Below, the first subsection describes the simulation design whilst the second subsection presents the simulation results. 5.1 Simulation Design A continuous time process r t is said to follows a CIR diffusion if it obeys the following stochastic differential equation: dr t = κ β r t dt + σ r t dw t 3 where the parameter κ > 0 is the strength of the mean reversion in the process, β > 0 is the long run mean and σ > 0 controls the volatility of r t. his model has been widely used in the asset pricing literature, see e.g Heston 1993 or Singleton 001. It is shown in Feller 1951 that Equation 3 admits a unique and positive fundamental solution if σ κβ. We assume that r t is observed at regularly spaced discrete times t 1, t,..., t such that t i t i =. he conditional distribution of r t given r t is a noncentered chi-square with possibly fractional order. Its transition density is a Bessel function of type I, which can be represented as an infinite mixture of Gamma densities with Poisson weights: f r t r t = j=0 r j+q k t c j+q p j exp cr t Γ j + q κ where c = σ 1 e κ, q = κβ and p σ j = ce κ r t j exp ce κ r t j!. o implement a likelihood based inference for this model, one has to truncate the expression of f r t r t. However, the conditional CF of r t has a simple closed form expression given by: ϕ t s, θ E e isrt r t = 1 is q ise κ V t exp c 1 is c 4 13

16 with θ = κ, β, σ. of θ is: o start, we simulate one sample of size from the CIR process assuming = 1 and the true value θ 0 = κ 0, β 0, σ 0 = 0.4, 6.0, 0.3. hese parameter values are taken from Singleton 001. We refer the reader to DeVroye 1986 and Zhou 001 for details on how to simulate a CIR process. We treat this simulated sample as the actual data available to the econometrician and use it to estimate the first step CGMM estimator θ 1 as: θ1 = arg min ĥ τ, θĥ τ, θe τ τ dτ, θ R where ĥ τ, θ = 1 i= e iτ 1 r ti ϕ t s, θ e iτ r ti, τ = τ 1, τ 1 R and ϕ t s, θ is given by 4. Next, we simulate M samples of size using θ 1 as pseudo-true parameter value. Each simulated samples is used to compute the second step CGMM estimator θ,j α as: θ,j α = arg min K θ α ĥ τ, θ ĥ τ, θe τ τ dτ 5 R he objective function 5 is evaluated using a Gauss-Hermite quadrature with ten points. regularization parameter α is selected on a thirty points grid that lies between 10 0 and 10, that is: α [10 0, , , , ,..., 1 10 ] For each α in this grid, we compute the MSE using Equation 1 i.e., no truncation of the distribution of θ,j α θ Simulations results able 1 shows the simulations = 51, 501, 751 and 1001 for two different values of M. he For a given sample size, the scenarios with M = 500 and M = 1000 use common random numbers i.e., the results for M = 500 are based on the first 500 replications of the scenarios with M = Curiously enough, the estimate of α M θ 1 is consistently equal to across all scenarios except = 51, M = 1000 and = 1001, M = 500. his result might be suggesting that the grid on which α is selected is not refined enough. Indeed, the value that is immediately smaller than on that grid is , which is selected for the scenario = 1001, M = 500. Arguably, the results suggest that the rate of convergence of α θ 0 to zero is quite slow for this particular data generating process. Overall, seems a reasonable choice for the regularization parameter for all sample sizes for this data generating process. Note that our simulations results do not allow us to infer the behavior of α θ 0 as θ 0 vary in the parameter space. Figure 1 presents the simulated MSE curves. For all eight scenarios, these curves are convex and 14

17 have one minimum. he hump-shaped left tail of the MSE curves for = 51 stems to the fact that the approximating matrix of the covariance operator see Appendix D is severely ill-posed. Hence, the shape of the MSE curve reflects the distortions inflicted to the eigenvalues of the regularized inverse of this approximating matrix as α varies. A smaller number of quadrature points should be used for smaller sample sizes in order to mitigate this ill-posedness and obtain perfectly convex MSE curves. able 1: Estimation of α for different sample size. M = 500 M = 1000 α M θ 1 1 M α M θ 1 1 M = = = = Figure 1: MSE curves of the CGMM estimator for different M and. he vertical axis shows 1 Σ M = 1 M M ξ j, α, θ 1 and the horizontal axis is scaled as log α Sample size: = 51 M=500 reps M=1000 reps Sample size: = 501 M=500 reps M=1000 reps Mean square error Mean square error logα/ logα/ x 10 3 Sample size: = M=500 reps M=1000 reps 6.8 x 10 3 Sample size: = M=500 reps M=1000 reps Mean square error 7. 7 Mean square error logα/ logα/10 6 Conclusion he objective of this paper is to provide a method to optimally select the regularization parameter denoted α in the CGMM estimation. First, we derive a higher order expansion of the CGMM estimator that sheds light on how the finite sample MSE depends on the regularization parameter. We obtain the 15

18 convergence rate for the optimal regularization parameter α by equating the rates of two higher order variance terms. We find an expression of the form α = c θ 0 gβ, where c θ 0 does not depend of the sample size and 0 g β 1/, where β is the regularity of the moment function with respect to the covariance operator see Assumption 4. Next, we propose an estimation procedure for α that relies on the minimization of an approximate MSE criterion obtained by Monte Carlo simulations. he proposed estimator, α M, is indexed by the sample size and the number of Monte Carlo replications M. o hedge against situations where the MSE is not finite, we propose to base the selection of α on a truncated MSE that is always finite. Under the assumption that the truncation scheme does not alter the rate of α, α M is consistent for α as and M increase to infinity. Our simulation-based selection procedure has the advantage to be easily applicable to other estimators, for instance it could be used to select the number of polynomial terms in the effi cient method of moments procedure of Gallant and auchen he optimal selection of the regularization parameter permits to devise a fully feasible CGMM estimator that is a real alternative to the maximum likelihood estimator. 16

19 References [1] Ait-Sahalia, Y. and R. Kimmel 007, Maximum likelihood estimation of stochastic volatility models, Journal of Financial Economics 83, [] Andews, D. W. K. 1991, "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, 593: [3] Buse, A. 199, he Bias of Instrumental Variable Estimators, Econometrica 60:1, [4] Carrasco, M. 01, A regularization approach to the many instruments problem, Journal of Econometrics, 170:, [5] Carrasco, M., Chernov, M., Florens J. P. and E. Ghysels 007, Effi cient estimation of general dynamic models with a continuum of moment conditions, Journal of Econometrics, 140, [6] Carrasco, M. and J. P. Florens 000, Generalization of GMM to a continuum of moment conditions, Econometric heory, 16, [7] Carrasco, M., J. P. Florens, and E. Renault 007, Linear Inverse Problems in Structural Econometrics: Estimation based on spectral decomposition and regularization, in the Handbook of Econometrics, Vol. 6, edited by J.J. Heckman and E.E. Leamer. [8] Chacko, G. and L. Viceira 003, Spectral GMM estimation of continuous-time processes, Journal of Econometrics, 116, [9] Devroye, L. 1986, Non-Uniform Random Variate Generation, Spinger-Verlag. [10] Dominguez, M. A. and I. N. Lobato 004, Consistent Estimation of Models Defined by Conditional Moment Restrictions, Econometrica 7:5, [11] Donald, S. and W. Newey 001, Choosing the number of instruments, Econometrica, 69, [1] Duffi e, D. and K. Singleton 1993, Simulated Moments Estimation of Markov Models of Asset Prices, Econometrica, 61, [13] Feller, W. 1951, wo Singular Diffusion Problems, Annals of Mathematics, 54, [14] Feuerverger, A. 1990, An effi ciency result for the empirical characteristic function in stationary time-series models, Canadian Journal of Statistics, 18, [15] Feuerverger, A. and P. McDunnough 1981a, On Effi cient Inference in Symmetry Stable Laws and Processes, Csorgo, M., ed. Statistics and Related opics. New York: North Holland, [16] Feuerverger, A. and P. McDunnough 1981b, On some Fourier Methods for Inference, J. R. Statist. Assoc. 76:

20 [17] Feuerverger, A. and P. McDunnough 1981c, On the Effi ciency of Empirical Characteristic Function Procedures, J. R. Statist. Soc. B, 43, 0-7. [18] Feuerverger, A. and R. Mureika 1977, he empirical characteristic function and its applications, he annals of statistics, 5, No 1, [19] Gallant, A.R. and G. auchen 1996, Which Moments to Match? Econometric heory, 1, [0] Gouriéroux, C. and A. Monfort 1996, Simulation Based Econometric Methods, CORE Lectures, Oxford University Press, New York. [1] Gouriéroux, C., A. Monfort and E. Renault 1993, Indirect Inference, Journal of Applied Econometrics 8, S85-S118. [] Hansen, L. 198, Large Sample Properties of Generalized Method of Moments Estimators, Econometrica, 50, [3] Heston, S. 1993, A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options, he Review of Financial Studies, 6:, [4] Jacho-Chavez, D.. 010, Optimal Bandwidth Choice for Estimation of Inverse Conditional Density Weighted Expectations, Econometric heory, 6, [5] Jiang, G. and J. Knight 00, Estimation of Continuous ime Processes via the Empirical Characteristic Function, Journal of Business and Economic Statistics, 0, [6] Koenker, R., Machado, J. A. F., Skeels, C. L. and A. H. I. Welsh 1994, Momentary Lapses: Moment Expansions and the Robustness of Minimum Distance Estimators, Econometric heory, 10, [7] Koutrouvelis, I. A. 1980, Regression-type estimation of the parameters of stable laws, J. Am. Statist. Assoc. 75:37, [8] Lavergne, P., and V. Patilea 008, Smooth minimum distance estimation and testing with conditional estimating equations: uniform in bandwidth theory, Working paper available at ideas.repec.org. [9] Linton, O. 00 Edgeworth Approximation for Semiparametric Instrumental Variable and est Statistics, Journal of Econometrics, 106, [30] Liu, Q. and D. A. Pierce 1994, A Note on Gauss-Hermite Quadrature, Biometrika, 81:3, [31] Madan, D. B. and E. Senata 1990, he Variance Gamma Model for Share Market Returns, Journal of Business, 63:4,

21 [3] Nagar, A. L. 1959: he Bias and Moment Matrix of the General k-class Estimators of the Parameters in Simultaneous Equations, Econometrica, 7, [33] Newey, W. K. and D. McFadden 1994, Large Sample Estimation and Hypotheses esting, Handbook of Econometrics, Vol IV, ed. by R.F. Engle and D.L. McFadden. [34] Newey, W K. and R. J. Smith 004, Higher Order Properties of GMM and Generalized Empirical Likelihood Estimators, Econometrica, 7:1, [35] Nolan, J. P. 009, Stable Distributions: Models for Heavy ailed Data, Birkhauser, Boston,. In preparation. [36] Paulson, A. S., Holcomb W. E. and R. A. Leitch 1975, he estimation of the parameters of the stable laws, Biometrika, 6:1, [37] Phillips, P. C. B. and H. R. Moon 1999, Linear Regression Limit heory for Nonstationary Panel Data, Econometrica, 67:5, [38] Rilstone, P., Srivastava, V. K., and A. Ullah 1996, he Second-Order Bias and Mean-Squared Error of Nonlinear Estimators, Journal of Econometrics, 75, [39] Rothenberg,. J. 1983, Asymptotic properties of Some Estimators in Structural Models, Studies in Econometrics, ime Series and Multivariate Statistics, edited by S. Karlin,. Amemiya and L. A. Goodman, New York: Academic Press. [40] Rothenberg,. J. 1984, Approximating the Distributions of Econometric Estimators and est Statistics, Handbook of Econometrics, Vol., ed. by Z. Griliches and M. D. Intriligator. New York: North-Holland. [41] Singleton, K. 001, Estimation of Affi ne Pricing Models Using the Empirical Characteristic Function, Journal of Econometrics, 10, [4] Yu, J. 004, Empirical Characteristic Function Estimation and Its Applications, Econometric Reviews, 3:, [43] Zhou, H. 001, Finite Sample Properties of EMM, GMM, QMLE, and MLE for a Square-Root Interest Rate Diffusion Model, Mimeo, available at 19

22 Appendix A Some basic properties of the covariance operator For more formal proofs of the results mentioned in this appendix, see Carrasco, Florens and Renault 007. Let K be the covariance operator defined in 5 and 6, and ĥt τ, θ the moment function defined in 1 and. Finally, let Φ β be the subset of L π defined in Assumption 4. Definition 7 he range of K denoted RK is the set of functions g such that Kf = g for some f in L π. Proposition 8 RK is a subspace of L π. Note that the kernel functions ks,. and k., r are elements of L π because ks, r = [ ] E h t θ, sh t θ, r 4, s, r R p 1 hus for any f L π, we have Kf s = ks, rf r π r dr 4 f r π r dr <, ks, rf r π r dr implying Kf = Kf s π s ds < Kf L π. Definition 9 he null space of K denoted NK is the set of functions f in L π such that Kf = 0. he covariance operator K associated with a moment function based on the CF is such that NK = {0}. See CCFG 007. Definition 10 φ is an eigenfunction of K associated with eigenvalue µ if and only if Kφ = µφ. Proposition 11 Suppose µ 1 µ... µ j... are the eigenvalues of K. hen the sequence { µ j } satisfies: i µ j > 0 for all j, ii µ 1 < and lim j µ j = 0. Remark. compact. he covariance operator associated with the CF-based moment function is necessarily Proposition 1 Every f L π can be decomposed as: f = f, φj φj. As a consequence, Kf = f, φj Kφj = f, φj µj φ j. 0

23 Proposition 13 If 0 < β 1 β, then Φ β Φ β1. We recall that Φ β is the set of functions such that K β f <. In fact, f RK β K β f exist and K β f = µ β j f, φ j <. hus if f RK β, we have: K β 1f = µ β β 1 j µ β j f, φ j β µ β 1 1 µ β j f, φ j < K β 1 f exist f RK β 1. his means RK RK 1/ so that the function K / f is defined on a wider subset of L π compared to K f. When f Φ 1, K / f, K / f = K f, f. But when f Φ β for 1/ β < 1, the quadratic form K / f, K / f is well defined while K f, f is not. B Expansion of the MSE and proofs of heorems 1 and B.1 Preliminary results and proof of heorem 1 Lemma 14 Let Kα = K + αi K and assume that f Φ β for some β > 1. hen as α goes to zero and n goes to infinity, we have: K α K α K α K α K = O p α 3/ /, f = Op α /, 3 β min1, α, 4 α K f = O f, f = O α K K α min1, β. 5 Proof of Lemma 14. Subsequently, φ j, j = 1,..., denote the eigenfunctions of the covariance operator K associated respectively with the eigenvalues µ j, j = 1,...,. We first consider. By the triangular inequality: K + αi K K + αi K K + αi K K + K + αi K K + αi K K + αi K K + [ K }{{}}{{} + αi K + αi ] K, α =O p / where K K = O p / follows from Proposition 3.3 i of CCFG 007. We have: [ K + αi K + αi ] K = K + αi K K K + αi K K + αi K K K + αi / K + αi / K }{{}}{{}}{{}}{{} α =O p / 1 α / 1

24 his proves. he difference between and 3 is that in 3 we exploit the fact that f Φ β with β > 1, hence K f <. We can rewrite 3 as We have K α K α f = K α K α KK f K α K α K K f. K α K α K = K + αi K K K + αi K he term 6 can be bounded in the following manner = K + αi K K K 6 + [ K + αi K + αi ] K. 7 K + αi K K K K + αi K K K }{{}}{{} α = O p α /. =O p / For the term 7, we use the fact that A / B / = A / B 1/ A 1/ B /. It follows that [ K + αi K + αi ] K = K + αi K K K + αi K K + αi K K K + αi K = O p α /. }{{}}{{}}{{} α =O p / 1 his proves 3. Now we turn our attention toward equation 4. We can write K + αi Kf K f = [ µj α + µ j 1 µ j ] f, φj φj = µ j α + µ j 1 f, φj µ j φ j. We now take the norm: 4 = K + αi Kf K f µ = j f, φj α + µ 1 j µ j µ = µ β 1/ j f, φj f, φ j α + µ 1 j j µ β j µ β j 1/ 1/ sup µ β j 1 j α α + µ. j

25 Recall that as K is a compact operator, its largest eigenvalue µ 1 is bounded. We need to find an equivalent to sup µ β α 0 µ µ 1 α + µ j = sup λ β λ µ α/λ α β// x x β// 1+x 8. An Case where 1 β 3: We apply another change of variables x = α/λ: sup x 0 equivalent to 8 is α β// 1 provided that x x β// 1+x is bounded on R +. Note that g x x3 β/ 1+x is continuous and therefore bounded on any interval of 0, +. It goes to 0 at + and its limit at 0 also equals 0 for 1 β < 3. For β = 3, we have: g x 1 1+x. hen g x goes to 1 at x = 0 and to 0 at +. Case where β > 3: We rewrite the left hand side of 8 as µ β j α α + µ j = αµ β 3 j µ j α + µ j }{{} 0,1 αµ β 3 1 = O α. β min1, o summarize, we have for f Φ β : 4= O α. Finally, we consider 5. We have: 5 = j = j 1 µ j f, µ j µ j + α φj = 1 µ j f, φj µ j j + α µ j µ β j 1 µ j f, φj µ j + α µ β f, φj j j µ β sup j µ µ 1 µ β α µ + α. For β 3/, we have: sup µ µ1 µ β variables x = α/µ and obtain sup x 0 R + β min1,. Finally: 5= O α. Lemma 15 α µ +α αµβ 3 x 1+x 1 = O α. For β < 3/, we apply the change of α β x = O α β, as f x = x β 1+xx is bounded on Suppose we have a particular function fθ Φ β for some β > 1, and a sequence of functions f θ Φ β such that sup f θ fθ = O p /. hen as α goes to zero, we have θ Θ with sup K / α θ Θ Proof of Lemma 15. f θ K / fθ = O p α / + O sup K α f θ K fθ B 1 + B, θ Θ θ Θ β min1, α. B 1 = sup K α f θ K α fθ and B = sup K α K fθ. θ Θ 3