Studies in Nonlinear Dynamics and Econometrics

Transcription

1 Studies in Nonlinear Dynamics and Econometrics Quarterly Journal Volume 4, Number 4 The MIT Press Studies in Nonlinear Dynamics and Econometrics (ISSN ) is a quarterly journal published electronically on the Internet by The MIT Press, Cambridge, Massachusetts, Subscriptions and address changes should be addressed to MIT Press Journals, Five Cambridge Center, Cambridge, MA ; (617) ; journals-orders@mit.edu. Subscription rates are: Individuals $4., Institutions $135.. Canadians add additional 7% GST. Submit all claims to: journals-claims@mit.edu. Prices subject to change without notice. Permission to photocopy articles for internal or personal use, or the internal or personal use of specific clients, is granted by the copyright owner for users registered with the Copyright Clearance Center (CCC) Transactional Reporting Service, provided that the per-copy fee of $1. per article is paid directly to the CCC, 222 Rosewood Drive, Danvers, MA The fee code for users of the Transactional Reporting Service is / $1.. For those organizations that have been granted a photocopy license with CCC, a separate system of payment has been arranged. Address all other inquiries to the Subsidiary Rights Manager, MIT Press Journals, Five Cambridge Center, Cambridge, MA ; (617) ; journals-rights@mit.edu. c 21 by the Massachusetts Institute of Technology

2 Efficient Estimation of Dynamical Systems Stefano Maria Iacus Department of Economics Faculty of Political Sciences University of Milan Abstract. The aim of this article is to show a simple way to construct asymptotic minimax lower bounds for risks based on different types of quadratic loss functions in semiparametric inference problems. For the sake of clarity, we consider the simple case of the state estimation of a dynamical system with small noise. The proofs are based on the van Trees inequality, namely, an integral-type Cramér-Rao inequality. Keywords. asymptotic efficiency, diffusion process, dynamical systems, integrated mean square error, quadratic risk, semiparametric estimation Acknowledgments. I thank Prof. Yury A. Kutoyants, who proposed that I study this statistical problem during the preparation of my Ph.D. thesis. I would like also to thank Prof. Prakasa Rao, who brought his work of 1992 to my attention, and an anonymous referee for valuable comments. 1 Introduction The Cramér-Rao lower bound for the mean square error of an estimator is well known, and its use in statistical inference has been extensively discussed. Extensions of Cramér-Rao-type inequalities are relatively new. These extensions consider integral versions of the mean square error (MSE). The first and most famous one is known as the van Trees inequality, obtained independently by Shutzenberger (1959) and van Trees (1968). Other generalizations of the Cramér-Rao bound and detailed discussions of their features have been considered by Borovkov and Sakhanenko (198), Borovkov (1984), Shemyakim (1987), Bobrovsky, Mayer-Wolf, and Zakai (1987), Prakasa Rao (199), and recently by Gill and Levit (1995). A first application of the van Trees inequality in the minimax context has been considered by Prakasa Rao (1992, 1996) for the construction of local asymptotic minimax lower bounds in different parametric inference problems. Gill and Levit (1995) apply this tool to determine the rate of convergence in some nonregular semiparametric problems without providing the exact lower bounds. They also obtain a lower bound for the integrated MSE for estimators of the distribution function in the independent and identically distributed case. Kutoyants (1998) also used the van Trees inequality to derive the so-called Pinsker s constant in the problem of nonparametric estimation of the intensity function of a Poisson process. Here I would like to show some fruitful applications of the van Trees inequality in the construction of local asymptotic minimax lower bounds for the quadratic risk and for the L 2 -norm risk in a semiparametric inference problem. The aim of this article is to present a clear and easy way to obtain local asymptotic lower bounds via the van Trees inequality. The reader will see that no knowledge on LAN (local asymptotic normality) and Hájek-Le Cam theory is needed in the proofs of the bounds, though LAN can simplify the necessary c 21 by the Massachusetts Institute of Technology Studies in Nonlinear Dynamics and Econometrics, 21, 4(4):

3 calculations. For this reason I explain the technique developed here in detail and for a simple and widely known model. In this note, I will consider the simple case of state estimation for dynamic systems with small noise. This problem is in the framework of statistical inference for diffusion processes with small dispersion asymptotic (small noise), which is a well-developed domain of mathematical statistics. For parametric models in continuous time one can look, for example, at Kutoyants 1984 and 1994; for discretely observed processes, to Bibby and Sørensen For probabilistic aspects, complete treatises are Freidlin and Wentzell 1984, Azencott 1982, and Freidlin Here I consider a semiparametric inference problem for this model. For the definition of semiparametric models we refer to Bickel et al and van der Vaart The simplicity of the model considered allows the reader to extend the technique to general models. In Section 2 I recall the form of the van Trees inequality for parametric diffusion processes with small dispersion and under what conditions it holds for this model. In Section 3 I show how to construct local asymptotic minimax lower bounds (for risks based on quadratic loss functions) in semiparametric models applying the van Trees inequality. I also give the exact constants, not only the rates of convergence. This result, for generic polynomial loss functions, has been obtained previously using Hájek-Le Cam inequality (see Iacus 1998) but not yet published. In Section 4 I give some extensions of the previous semiparametric result by considering maximum and global (L 2 ) versions of the minimax bounds given in Section 3. These results are presented the first time here. I want to stress that this technique is an alternative to the classical approach of using the Hájek-Le Cam inequality and does not require the use of LAN theory. This means that one does not need to ask for too many conditions of regularity for the model involved, as in the Hájek-Le Cam theory, which sometimes are difficult to check. Moreover, Hájek-Le Cam bounds for L 2 -norm risk cannot be obtained using the standard Hájek-Le Cam theorem but must be solved via the abstract version of the same theorem, as explained in Millar 1979, where, to establish the result, one has to construct abstract Wiener spaces and prove convergence of experiments, a somewhat technical method compared with the elementary van Trees inequality. It is also important to point out that the technology based on the van Trees inequality gives an easy way to establish lower bounds but lacks the power of the general Hájek-Le Cam theory of characterizing asymptotically efficient estimators. In any case, nowadays the L 2 analysis is also the preferred way to study the second-order efficiency of estimators, and van Trees inequality seems to be a natural tool of investigation, as pointed out by Golubev and Levit (1996). 2 The van Trees Inequality Let us consider a diffusion process of small noise type (i.e. the diffusion coefficient is small) satisfying the following stochastic differential equation: { dx t S θ (t, X t )dt + εσ(t, X t ) dw t, t [, T ], ε (, 1] X x (1) where {W t, t T } is the Wiener process, S is the drift coefficient assumed to be known up to a parameter θ, where θ is in some set [α, β], with <α<β<+. The diffusion coefficient ε 2 σ 2 is assumed to be completely known, with σ> a smooth function and ε (, 1]. The initial condition x is a deterministic value. For the existence and the uniqueness of the solution of Equation (1) it is sufficient (see for example Liptser and Shiryayev 1977) to require that the functions S θ :[, T ] R R and σ :[, T ] R R are measurable functions and that there exists a positive constant K such that for any x, y R S θ (t, x) S θ (t, y) + σ(t, x) σ(t, y) K x y S θ (t, x) + σ(t, x) K (1 + x ) (2) 214 Estimation of Dynamical Systems

4 uniformly in t [, T ] and θ. In order to avoid heavy notation I use P θ instead of P (ε) θ to denote the law of the process in Equation (1), and with E θ I indicate the mathematical expectation with respect to this measure. The Fisher information for this model is given by (see, e.g., Kutoyants 1994) I ε (θ) ε 2 Ṡ θ (t, X t ) 2 E θ σ(t, X t ) dt, θ 2 where the dot indicates the derivative with respect to θ. Suppose that we want to estimate a function ψ(θ) of the unknown parameter θ and we have some prior statistical knowledge on the parameter to be estimated. Regardless of which estimator ψ ε of ψ(θ) we use, we have the following result on the quadratic risk. Theorem 1 (van Trees inequality). Let p(θ), θ [α, β], be an absolute continuous probability density such that p(α) p(β). Let the information quantity associated with p(θ) be ṗ(θ) 2 I(p) p(θ) dθ< Let the Fisher information I ε (θ) < be continuous in θ and pose that T S θ (t, X t ) T σ(t, X t ) dw Ṡ θ (t, X t ) t σ(t, X t ) dw t (3) Then for any estimator ψ ε of the function ψ(θ), with ψ(θ) absolutely continuous in θ, we have ( E θ ψ ε ψ(θ) ) ( 2 ψ(θ) p(θ) dθ p(θ) dθ ) 2 I (4) ε(θ) p(θ) dθ + I(p) Proof. In the derivation of Equation (4), we follow Gill and Levit (1995) with adaptation to diffusion processes. Recall that, for this model, the likelihood ratio has the form L(θ, X ) dp θ (X ) dp { 1 exp ε 2 S θ (t, X t ) σ(t, X t ) dx 2 t 1 2ε 2 ( ) Sθ (t, X t ) 2 dt} σ(t, X t ) where P is the law of the process dx t εσ(t, X t )dw t. Set H (θ, X ) L(θ, X )p(θ) and let continuous functions on [, T ]. Taking into account the equalities β H (θ, X ) dθ L(θ, X ) p(θ) and we can write ( ψ(θ)p(θ) dθ) 2 ψ(θ) H (θ, X ) dθ { { α ψ(θ)h (θ, X ) dθ ( ψ ε ψ(θ) ) } 2 H (θ, X ) dθ dp (X ) ( ψ ε ψ(θ) ) ln H (θ, X ) } 2 dp θ (X ) p(θ) dθ ( E θ ψ ε ψ(θ) ) 2 p(θ) dθ ( ) ln H (θ, X ) 2 dp θ (X ) p(θ) dθ be the space of Stefano Maria Iacus 215

5 using Cauchy-Schwarz inequality. Summarizing, we have ( E θ ψ ε ψ(θ) ) ( 2 ψ(θ)p(θ) p(θ) dθ dθ ) 2 ) 2 p(θ) dθ Recall that Thus, we can write ( ) ln H (θ, X ) 2 E θ p(θ) dθ E θ ( ln H (θ,x ) dp θ (X ) L(θ, X )dp (X ). ( ) ln H (θ, X ) 2 E θ p(θ) dθ { ( ) } 2 ln(l(θ, X )p(θ)) dp θ (X ) p(θ) dθ ( ) 2 ( L(θ, X )p(θ) + ṗ(θ)l(θ, X )) dp θ p(θ) dθ L(θ, X )p(θ) ( L(θ, X ) L(θ, X ) + ṗ(θ) ) 2 L(θ, X ) p(θ) dθ dp (X ) p(θ) ( ) 2 ln L(θ, X ) H (θ, X ) dθ dp (X ) ( ) + I(p) + 2 E θ ln L(θ, X ) ṗ(θ) dθ ( ln L(θ, X ) ) 2 H (θ, X ) dθ dp (X ) + I(p) where we have used the fact that, under the condition imposed, the score function has expected value identically zero. Further, and by Equation (3) ( ) 2 ln L(θ, X ) H (θ, X ) dθ dp (X ) { ( ) 2 ln L(θ, X ) dp θ (X )} p(θ) dθ ( ) 2 E θ ln L(θ, X ) p(θ) dθ I ε (θ) p(θ) dθ ( ) 2 ( 1 T Ṡ θ (t, X t ) E θ ln L(θ, X ) E θ ε σ(t, X t ) dw t ( 1 T Ṡ θ (t, X t ) 2 ) E θ ε 2 σ(t, X t ) dt 2 I ε (θ) ) 2 Concluding, we have ( ) ln H (θ, X ) 2 E θ p(θ) dθ I ε (θ) p(θ) dθ + I(p) and this completes the proof. 216 Estimation of Dynamical Systems

6 Remark 1. It should be stressed that the right-hand side of the van Trees inequality (Equation (4)) does not depend on the properties of the estimators, as, for example, in the Cramér-Rao case, but of course it does depend on the prior knowledge on θ. Anyway, this bound is uniform with respect to all the estimators and all the values of the parameter. The van Trees inequality, as pointed out by Gill and Levit (1995), is a powerful tool that allows one to give simplified proofs of classical results concerning the variance of the limiting distribution of uniformly regular estimators (see, e.g., Bickel et al for the definition). In particular, it can be shown that for this class of estimators a particular version of Hájek s (197) convolution theorem holds under minimal regularity conditions. 3 Asymptotic Efficiency in a Semiparametric Problem Let us now consider the problem of observations from a process that satisfies the following stochastic differential equation { dx t S(t, X t )dt + εσ(t, X t ) dw t, t [, T ], ε (, 1] X x (5) where, in this case, the function S is posed to be unknown up to an infinite dimensional parameter, so we have a nonparametric model. The functions S and σ are posed to be sufficiently smooth such that Equation (2) holds. The diffusion process {X t, t T } converges under Equation (2), as ε, to the deterministic function {x t, t T }, the solution of the following ordinary differential equation (see, for example, Kutoyants 1994): dx t S(t, x t ), t [, T ] dt (6) x The function x {x t, t T } constitutes a nonlinear dynamic system, and the value x τ of the function x at the point τ [, T ] is usually called the state of the dynamic system at time τ. Estimation of the state of a dynamic system is of statistical interest. Although the model is nonparametric, the statistical problem we wish to consider is semiparametric, because it is that of estimating the function x at a single point only. This case is very similar to the one of pointwise estimation of the distribution function in the i.i.d. case. The problem of the estimation of the whole function has been considered by Apoyan and Kutoyants (1988; see as well Kutoyants 1994, Section 4.3). We now establish a local asymptotic minimax lower bound on the quadratic risk in the problem presented above by applying the result of Theorem 1. This way of deriving information bounds for estimates was first introduced by Stein (1956) and was formalized by Levit (1973) and Koshevnik and Levit (1976). Roughly speaking the idea is to consider an arbitrary parametric family of regular models passing through the point P assumed to be the true underlying model. Then, for each parametric family, the construction of information bounds is considered. Finally, the worst parametric model with maximal information bound (e.g., minimal Fisher information) is chosen as the best achievable semiparametric information bound (see also Bickel and Ritov 199). Here we apply this methodology to the state estimation of the dynamical system considered above. Definition 1. Let us take a positive constant K and define the set G(K ) of measurable functions f (t, x): [, T ] R R with the following properties: f (t, x) belongs to C 1 (R) with respect to the variable x. Moreover, for all t [, T ] and x R, f (t, x) K (1 + x ), f (t, x) K where by f (t, x) we denote the partial derivative of the function f (t, x) with respect to the second variable. Stefano Maria Iacus 217

7 If we pose that S belongs to the class G(K ) and we take σ>in the class G(K ), the Lipschitz condition (Equation (2)) is fulfilled with K 2K, and we have the uniqueness and existence of the solution of the stochastic differential equation (Equation (5)). Fix a function S in G(K /2) and define the neighborhood of functions V δ (S ) of S as follows: { } V δ (S ) S G(K ): t T S(t, x) S (t, x) <δ x [ x,x max ] where x max is the maximum value achievable by the dynamic system in the time interval [, T ] that depends only on the constants K, T, and x and not on the particular choice of the function S; that is, x t x max ( x +KT)e KT t T This can be proved using Gronwall s inequality and the properties of the coefficient S. Recall that {x t x t (S), t T } is a functional of S. Finally, let τ { τ } στ 2 (S) σ 2 (s, x s (S)) exp 2 S (u, x u (S))du ds s Let us introduce the quadratic risk associated with any estimator ˆx τ ε of x τ R ε τ ( ˆx ε τ, x τ (S)) ε 2 E S (( ˆx ε τ x τ (S) )) 2 where with E S we indicate the expectation with respect to the measure P S induced by Equation (5). Theorem 2. Suppose that S belongs to the class G(K ) and <σ τ (S )<. Let ˆx ε τ be any estimator of x τ. Then lim lim δ ε inf ˆx ε τ S V δ (S ) where the inf is taken over all the estimators ˆx ε τ of x τ. R ε τ ( ˆx ε τ, x τ (S)) σ 2 τ (S ) Proof. First of all we need to construct a parametric vicinity of the model. We will write x t for x t (S ). Let us consider the functions S θ (t, x) S (t, x) + (θ θ )χ {t τ} ϕ(t)σ (t, x t ) where, for some γ γ(δ)>, ϕ such that ϕ(t) < K /2δ γ, t T, and θ [θ γ,θ + γ ], are such that S θ V δ (S ). In particular, t T S θ (t, x) S (t, x) K 2 γ (1 + 2x max ) (7) x [ x,x max ] 2δ because t T σ(t, x t ) K (1 + 2x max ). Thus Equation (7) is less than δ if and only if γ γ(δ)<δ 4 (2/K 2 (1 + 2x max )) 2. For such S θ we have a family of parametric diffusion processes {X θ t, t T } θ satisfying the stochastic differential equations dx θ t S θ (t, Xt θ θ ) dt + εσ(t, Xt ) dw t, t T, θ (8) with the initial condition X θ x. We denote by {P θ,θ } the family of measures induced by the processes solution of Equation (8) and by E θ the expectation with respect to this measure. The Fisher information for this family is given by τ Iε ϕ (θ) ε 2 E θ ϕ 2 (t) σ 2 (t, x t (S )) dt σ 2 (t, X t ) 218 Estimation of Dynamical Systems

8 Under P θ, X t x t (S θ ) + o ε (1), so lim ε E θ τ ϕ 2 (t) σ 2 (t, x t (S )) σ 2 (t, X t ) dt θθ τ ϕ 2 (t) dt The limit deterministic systems associated with Equation (8) are of the following form: dx θ t dt It is clear that the following relationship holds: S (t, x θ t ) + (θ θ )χ {t τ} ϕ(t)σ (t, x t (S )), x θ x Rτ ε ( ˆx τ ε, x τ (S)) S V δ (S ) θ θ <γ Rτ ε ( ˆx τ ε, x τ (S θ )) ε 2 E θ (( ˆx τ ε x τ θ ))2 θ because S θ V δ (S ). Let us now introduce a prior density p(θ) on the set satisfying the hypotheses of Theorem 1. By the properties of the remum we have further Rτ ε ( ˆx τ ε, x τ (S θ )) Rτ ε ( ˆx τ ε, x τ (S θ )) p(θ) dθ ε 2 E θ (( ˆx τ ε x τ θ ))2 p(θ) dθ θ For the last term we can apply the van Trees inequality (Equation (4)) because Equation (3) is satisfied for the model (Equation (8)). Thus we obtain ( ε 2 E θ (( ˆx τ ε x τ θ ))2 p(θ) dθ ẋ τ θ p(θ) dθ) 2 (9) Iϕ ε (θ) p(θ) dθ + I(p)ε 2 This term does not depend on a particular estimator ˆx ε τ of x τ, and so we can apply the infimum over the class of all the estimators. Now putting all together and letting ε, we have lim ε inf ˆx ε τ S V δ (S ) R ε τ ( ˆx ε τ, x τ (S)) E θ ( ẋ τ θ p(θ) dθ) 2 τ ϕ2 (t) σ 2 (t,x t (S )) σ 2 (t,x t (S τ θ )) dt p(θ) dθ (1) The last step is to show that the second term of Equation (1) converges to στ 2(S ) as δ. In fact, by direct calculation, we have that τ ẋτ θ e τ s S (u,xθ u )du ϕ(s)σ (s, x s (S )) ds Moreover, when δ, the set shrinks to {θ } so, by absolute continuity, what remains in the end is just So it is clear that taking lim lim δ ε inf ˆx ε τ ϕ(t) e S V δ (S ) τ t R ε τ ( ˆx ε τ, x τ (S)) (ẋ θ τ )2 τ ϕ2 (t) dt S (u,x u(s ))du σ(t, x t (S )), t <τ (11) we have τ ϕ 2 (t) dt ẋ θ τ σ 2 τ (S ) which gives us the statement of the theorem by Equation (11). It remains to be checked that ϕ < K /2δ γ. In fact, we have that ϕ(t) e LK L(1 + 2x max )στ 2 and this is less than K /2δ γ if and only if γ<στ 4/2δ2 (1 + 2x max ) 2. Taking into account the other constraints on γ, and keeping in mind that in the proof we consider the limit as δ, it is clear that there exists a δ such that if δ<δ the value of γ(δ)<δ 4 (2/K 2 (1 + 2x max )) 2 should be chosen. And this completes the proof. Stefano Maria Iacus 219

9 Remark 2. In the proof of Theorem 2 the idea of finding the worst parametric approximation is present, even if a little bit masked. One can see that the particular choice of ϕ that we made corresponds to the equality in the Cauchy-Schwarz inequality used in the proof of the van Trees lower bound. Thus the lower bound given by στ 2(S ) is the highest for this semiparametric model. Another way of thinking about this is that this particular choice of ϕ is such that the derivative of the functional we want to estimate, ẋτ θ, should be asymptotically the same (as θ θ ) as the Fisher information for the model with this particular choice of ϕ. As before this gives us the equality in the lower bound of van Trees. This is a constructive way of finding the worst ϕ. One can even characterize these facts or find in another way the good form of ϕ, for example, by direct maximization of Equation (11) with respect to ϕ. Remark 3. As usual when a lower bound is presented it must be checked to determine that it is not too sharp. For the class of polynomial loss functions it is known (Iacus 1998) that the value Xτ ε is an efficient estimator of x τ in the sense of this bound. Further, we have the following: Proposition 1. The estimator Xτ ε is asymptotically normal (in probability) and consistent uniformly on the class of unknown function S G(K ) and uniformly on t [, T ]. More precisely it is consistent uniformly on S and on t [, T ], that is, for any ν> lim ε S t T P S { X ε t x t ν } asymptotically normal (in probability) uniformly on S and on t [, T ], that is, for any ν> the following probability converges to zero, where ζ t S t t T σ(s, x s )e P S { ε 1 ( X ε t x t ) ζt ν } t s S (u,x u )du dw s, t [, T ] is a Gaussian random variable with zero mean and variance σt 2 (S) and satisfying the stochastic differential equation dζ t S (t, x t )ζ t dt + σ(t, x t ) dw t, t [, T ], ζ The results in Proposition 1 have been obtained (see Iacus 1998) by constructing parametric subfamilies of models with the LAN property and applying the Hájek-Le Cam inequality, a somewhat more difficult approach to the solution of the same problem, but of course one with more general risks than the quadratic one. It is evident that all the lower bounds for semiparametric models, obtained using the approach based on the Hájek-Le Cam inequality, can be obtained easily for the MSE case by means of the van Trees inequality. Of course this is just a way of finding asymptotic lower bounds and does not have the power of the Hájek-Le Cam s general theory, as, for example, in characterizing asymptotically efficient estimators. 4 Different Types of Risk In this section we will use the results obtained in the previous one to obtain an asymptotic lower bound for the risk of the form E S (ε 1 ( ˆx t ε x t (S))) 2 t T 22 Estimation of Dynamical Systems

10 By applying a multidimensional version of the van Trees inequality, we will also construct the lower bound for the risk of the form E S (ε 1 ( ˆx t ε x t (S))) 2 dt (12) We will refer to the first of these as maximum risk, and the second will be called global risk, using the terminology given in Gill and Levit Sometimes the latter is also called MISE (mean integrated square error). The results that we are going to present are likely to hold in general for semiparametric models. In particular the same technique has been applied in the problem of distribution function estimation of the invariant law of an ergodic diffusion process (see Kutoyants and Negri forthcoming). It is easy to show that, under regularity conditions, such as the one of the set G(K ), we have the convergence of the stochastic function {Xt ε, t T } to the deterministic one {x t, t T }. This convergence is uniform in t and S, as we recalled in Remark 3. Thus, it is natural to see what happens to the asymptotic lower bound if we consider some kind of uniformity with respect to t. Theorem 3. Suppose that S G(K ). Let ˆx ε be any estimator of x. Then lim lim δ ε S V δ (S ) t T E S (ε 1 ( ˆx ε t x t (S))) 2 σ 2 (S ) where σ 2 (S ) σt 2 (S ) t T Proof. We observe that E S (ε 1 ( ˆx t ε x t (S))) 2 E S (ε 1 ( ˆx t ε x t (S))) 2 t T Then by applying all the operators (the minimax and limits) we can obtain the following result lim lim δ ε S V δ (S ) t T E S (ε 1 ( ˆx ε t x t (S))) 2 σ 2 t (S ) from Theorem 2. Now it is sufficient to apply the remum on t to both sides of the inequality to obtain the result. It can be shown that the trajectory of the process {X t, t T } is an efficient estimator in the sense of the uniform bound given by Theorem 3 (see Iacus 1998 for details). More interesting is the problem of the global risk. For the sake of simplicity let us introduce the MISE in a compact form. Given any estimator ˆx ε of the function x we introduce the integrated risk as follows { ( IR ε ( ˆx ε, x(s)) ε 2 E S ˆx ε t x t (S) ) } 2 dt Proposition 2. The random variables ζ ε t given by ζ ε t X t ε are uniformly integrable and convergent in probability, as ε, to the random variable ζ t defined in Remark 3. x t ε Stefano Maria Iacus 221

11 Proof. We show that, for any q 1, it holds that P (ε) S { X t x t ν} K 2q ( ε ν ) 2q (13) where the constant K 2q > depends only on K, T, x, and q but not on S or t. In fact, it is well known (see, e.g., Freidlin 1996) that t X t x t εe KT σ(s, X s ) dw s t T t T So for any ν>we have to consider { P (ε) S t T t σ(s, X s ) dw s ν ε e KT } Recall (see, e.g., Kutoyants 1994) that, for any q 1, we have that E (ε) S X t 2q < C 2q, C 2q (K, T, x )> and also { } E (ε) S σ(t, X t ) 2q dt 2q < +, 2q (K, T, x )> Let us consider the quantity Y t t σ(s, X s ) dw s, t [, T ] which is a continuous martingale. Now using in sequence Chebyshev s inequality and the Burkholder-Davis-Grundy maximal inequality, we have that { P (ε) S Y t ν } t T ε e KT ( ε ν K 2q ( ε ν ) ( 2q e 2qKT E (ε) S ) 2q t T ) 2q Y t where K 2q K 2q (K, T, x )>, and we also have the inequality in Equation (13). Now, with the help of Equation (13), we can show the uniform integrability. Let us define Further, for r > 1 E S ζ ε t r 1 F ε (x) P (ε) S { ζt ε > x} K 2qx 2q 1 1 u r df ε (u) u r d (1 F ε (u)) u r df ε (u) 1 [ u r (1 F ε (u)) ] + r u r 1 (1 F 1 ε (u)) du 1 K 2q [ u r 2q ] 1 + rk 2q 1 1 u r 2q 1 du < C < Choosing q > r/2 we obtain the last inequality and the uniform integrability. 222 Estimation of Dynamical Systems

12 From Proposition 2, it follows that the asymptotic risk for the plug-in estimator X ε is given by { ( lim ε IRε (X ε, x(s)) lim ε 2 E S X ε t x t (S) ) } 2 dt ε { } E S (ζ t (S)) 2 dt E S (ζ t (S)) 2 dt σt 2 (S) dt where ζ t (S) N (,σt 2 (S)) is as has been defined in Remark 3. We are now ready to state and prove our final result. As usual, consider a fixed function S and introduce, as before, the same δ-vicinity of the model V δ (S ). Put IR(S ) lim ε IR ε (X ε, x(s )). Suppose now that the function {x t, t T } and any estimator {xt ε, t T } are elements of the Hilbert space L 2 ([, T ], dt). We can then write the following developments: and x t (S) ˆx ε t + i1 + i1 λ i (t)c i (S), c i (S) λ i (t)c ε,i, c ε,i where {λ i } is a system of orthonormal functions of L 2 ([, T ], dt): { T 1 i j λ i (t)λ j (t) dt i j We then have the following result: x t (S)λ i (t) dt ˆx ε t λ i(t) dt Theorem 4. Suppose that S G(K ). Let ˆx ε be any estimator of x. Then Proof. Suppose that lim lim δ ε inf ˆx ε S V δ (S ) IR ε ( ˆx ε, x(s)) IR(S ). IR ε ( ˆx ε, x(s)) < S V δ (S ) (otherwise the proof is trivial). By Parseval s identity, we can write { } + IR ε ( ˆx ε, x(s)) ε 2 E (ε) S (c ε,i c i (S)) 2 i1 Consider now the quantity (14) r δ (S ) lim inf ε ˆx ε Further, take the functions ϕ i, i 1, 2,..., of the form ϕ i (s) e t s For any i 1, 2,..., ϕ i ( ) L 2 ([, T ], dt), in fact ϕ 2 i (s) ds S V δ (S ) R ε ( ˆx ε, x(s)) S (u,x u)du χ {s t} σ(s, x s (S ))λ i (t) dt σ 2 s (S ) ds IR(S )< Stefano Maria Iacus 223

13 So there exists a subsequence {ϕ i,j ( )}, j 1, 2,..., such that lim j (ϕ i,j (s) ϕ i (s)) 2 ds (15) Let us fix a k 1, 2,... and choose a γ γ(δ)> such that γ(δ) when δ. Thus, for every j 1, 2,..., consider the k-dimensional cubes j [θ γ /j,θ + γ /j] k R k such that j shrinks to {θ } when γ orj. Consider a k-dimensional vector θ (θ (1),θ (2),...,θ (k) ) in the interior of j such that for any fixed value of k and every value of j it is possibile to write the parametrization S j θ (s, x) S (s, x) + k (θ (i) θ )ϕ i,j (s)σ (s, x s (S )). i1 The value of γ can be chosen in such a way that S j θ V δ(s ) for every k 1, 2,... and j 1, 2,... (this can be done with the same arguments as those used in the proof of Theorem 2). For such a parametric model, the Fisher information is of the following form: I ϕ ε (θ) ε 2 E θ k i1 ϕ 2 i,j (t)σ 2 (t, x t (S )) σ 2 (t, X t ) where E θ is the expectation with respect to the law P θ of the process satisfying the stochastic differential equation Under P θ, X t x t (S θ ) + o(ε). Thus, we have lim ε ε2 I ϕ ε (θ) E θ dx t S j θ (t, X t) dt + εσ(t, X t ) dw t. k i1 dt, ϕi,j 2 (t) σ 2 (t, x t (S )) dt, j 1, 2,... (16) σ 2 (t, x t (S θ )) Let us take a set strictly contained in j and a prior distribution p(θ), θ, satisfying the multidimensional van Trees inequality (see, e.g., Iacus 1999). Then for every k and j we can write r δ (S ) lim inf ε ˆx ε S V δ (S ) IR ε ( ˆx ε, x(s)) lim inf IR ε ( ˆx ε, x(s j ε ˆx ε θ )) θ j lim inf IR ε ( ˆx ε, x(s j ε ˆx ε θ )) θ lim inf ε ˆx ε IR ε ( ˆx ε, x(s j θ )) p(θ) dθ by the properties of the remum. Using now Equation (14) we obtain { k r δ (S ) lim inf ε 2 E (ε) ε ˆx ε θ ( ε 2 k i1 lim ε ( k i1 E θ } (c ε,i c i (S j θ ))2 p(θ) dθ i1 c i (S j θ ) (i) ) 2 p(θ) dθ ε 2 ε2 I ϕ ε (θ) p(θ) dθ + I(p) c i (S j θ ) ) 2 p(θ) dθ (i) k, j 1, 2,... i1 ϕ2 i,j(t) σ 2 (t,x t (S )) σ 2 (t,x t dt p(θ) dθ (S θ )) 224 Estimation of Dynamical Systems

14 where we have used the van Trees inequality. Let us consider now the term c i (S j θ ) (i) xt θ (i) λ i(t) dt e ϕ i,j (s) t s S (u,xθ u )du ϕ i,j (s)χ {s t} σ(s, x s (S )) ds λ i (t) dt e t s S (u,xθ u )du χ {s t} σ(s, x s (S ))λ i (t) dt ds Remember that if j, then j {θ } and x t (S θ ) x t (S θ ) x t (S ). Thus, in virtue of Equations (15) and (16) and the absolute continuity of p(θ), we have, for every k, and thus r δ (S ) r δ (S ) k i1 i1 ϕi 2 (s) ds ϕi 2 (s) ds By Parseval s identity we obtain, for every δ, and this concludes the proof. r δ (S ) σ 2 t (S ) dt Remark 4. The first part of the proof of Theorem 4 is in principle similar to the one proposed by Gill and Levit 1995 in the problem of distribution function estimation in the i.i.d. framework, but the two proofs end with substantially different arguments. In fact, the crucial hypothesis in Gill and Levit 1995 is that the nonparametric family of distributions, corresponding to the unknown model, should be rich enough, inthe sense of the definition given in Levit 1978 and Millar 1979, that the lower bound is attainable. Here the regularity of the model is sufficient to have the result. References Apoyan, G. T., and Y. A. Kutoyants (1988). On the state estimation of a dynamical system perturbed by small noise. In Probability Theory and Mathematical Statistics with Applications. Dordrecht: Reidel, pp Azencott, R. (1982). Formule de Taylor stochastique et développement asymptotique d integrales de Feynmann (Stochastic Taylor s formula and asymptotic development of the Feynmann integral). In Séminaire de Probabilités XVI, Supplement: Géometrie Différentielle Stochastique. Lecture Notes in Math no Berlin: Springer Verlag, Bibby, B. M., and M. Sørensen (1996). On estimation for discretely observed diffusions: A review. Theory of Stochastic Processes, 2(18): Bickel, P. J., C. A. J. Klaassen, Y. Ritov, and J. A. Wellner (1993). Efficient and Adaptive Estimation for Semiparametric Models. Baltimore: Johns Hopkins University Press. Bickel, P. J., and Y. Ritov (199). Achieving information bounds in non and semiparametric models. Annals of Statistics 2(18): Bobrovsky, B. Z., E. Mayer-Wolf, and M. Zakai (1987). Some classes of global Cramér-Rao bounds. Annals of Statistics 15(4): Borovkov, A. A. (1984). Mathematical Statistics. Moscow: Nauka. Borovkov, A. A., and A. I. Sakhanenko (198). On estimates for the average quadratic risk. Probability and Mathematical Statistics, 1: (in Russian). Stefano Maria Iacus 225

15 Freidlin, M. (1996). Markov Processes and Differential Equations: Asymptotic Problems. Basel: Birkhäuser Verlag. Freidlin, M., and A. D. Wentzell (1984). Random Perturbations of Dynamical Systems. New York: Springer-Verlag. Gill, R. D., and B. Y. Levit (1995). Applications of the van Trees inequality: A Bayesian Cramér-Rao bound. Bernoulli, 1/2: Golubev, G. K., and B. Y. Levit (1996). On the second order minimax estimation of distribution functions. Mathematical Methods of Statistics, 1: Hájek, J. (197). A characterization of limiting distributions of regular estimates. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 14: Iacus, S. M. (1998). Semiparametric estimation of the state of a dynamical system with small noise. Prépublication Le Mans: Université du Maine. Iacus, S. M. (1999). Statistique semiparamétrique pour un processus de diffusion avec coefficient de diffusion petit. Ph.D. thesis, Le Mans: Université du Maine. Kutoyants, Y. A. (1984). Parameter Estimation for Stochastic Processes. Berlin: Heldermann Verlag. Kutoyants, Y. A. (1994). Identification of Dynamical Systems with Small Noise. Dordrecht: Kluwer. Kutoyants, Y. A. (1998). Statistical Inference for Spatial Poisson Processes. Lectures Notes in Statistics no New York: Springer. Kutoyants, Y. A., and I. Negri (forthcoming). On L 2 efficiency of empiric distribution for diffusion processes. Theory of Probability and Its Applications. Levit, B. Y. (1973). On the optimality of some statistical estimates. In J. Hájek (ed.), Proceedings of the Prague Symposium on Asymptotic Statistics, vol. 2. Prague: Charles University Press, Levit, B. Y. (1978). Infinite-dimensional information inequalities. Theory of Probability and Its Applications, 23: Liptser, R. S., and A. N. Shiryayev (1977). Statistics of Random Processes, vol. 1. New York: Springer. Millar, P. W. (1979). Asymptotic minimax theorems for the sample distribution function. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 48: Prakasa Rao, B. L. S. (199). On Cramér-Rao type integral inequalities. Calcutta Statistical Association Bulletin, 4 (H. K. Nandi Memorial Special Volume, ): , Prakasa Rao, B. L. S. (1992). Cramér-Rao type integral inequalities for estimators of functions of multidimensional parameter. Sankhyā A, no. 54: Prakasa Rao, B. L. S. (1996). Remarks on Cramér-Rao type integral inequalities for randomly censored data. Institute of Mathematical Statistics, Lectures Notes, 27: Shemyakin, A. E. (1987). Rao-Cramér type integral inequalities for estimates of a vector parameter. Theory of Probability and Its Application, 32: Shutzenberger, M. P. (1959). A generalization of the Frechet-Cramér inequality to the case of Bayes estimation. Bulletin of the American Mathematical Society, 63: Stein, C. (1956). Efficient nonparametric testing and estimation. In Proceedings of the Third Berkeley Symposium on Mathematics, Statistics, and Probability, vol. 1. Berkeley and Los Angeles: University of California Press, pp van der Vaart, A. (1995). Semiparametric models: An evaluation. Statistica Neerlandica, 49(1): van Trees, H. L. (1968). Detection, Estimation and Modulation Theory, part 1. New York: Wiley. 226 Estimation of Dynamical Systems