Statistical inference on Lévy processes

Transcription

1 Alberto Coca Cabrero University of Cambridge - CCA Supervisors: Dr. Richard Nickl and Professor L.C.G.Rogers Funded by Fundación Mutua Madrileña and EPSRC MASDOC/CCA student workshop th March

2 Outline Motivation and preliminaries 1 Motivation and preliminaries Motivation Non-parametric statistics Lévy processes 2 3 Main result Extensions 4

3 Motivation Motivation and preliminaries Motivation Non-parametric statistics Lévy processes Lévy processes form the fundamental building block for stochastic continuous-time models with jumps. They are highly useful models in many applications. However, in these applications the data corresponds to discrete-time observations of a stochastic process. Recall that an estimator is any measurable function of the data, and it is consistent if it converges in probability to the true value.

4 Motivation Motivation and preliminaries Motivation Non-parametric statistics Lévy processes Therefore, finding consistent estimators of the underlying continuous-time process and functionals of it is of vital importance for these applications. It is also important to understand how fast these estimations approach the true values as the number of observations grows. These theoretical questions proved to be technically challenging which, together with the current growing demand for this type of results, explains why there were hardly any important results in the area before the last decade.

5 Example 1: regression Motivation Non-parametric statistics Lévy processes Suppose we are given n pairs of observations (X 1, Y 1 ),..., (X n, Y n ) and the response variable Y is related to the covariate X by a functional relationship Y i = m(x i ) + ε i i = 1,... n, where m : R R is some unknown function and the ε i represent i.i.d. errors independent of the X i.

6 Motivation Non-parametric statistics Lévy processes

7 Example 1: regression Motivation Non-parametric statistics Lévy processes Simple linear regression: assume m(x i ) = a + bx i for some unknown a, b R. The most famous unbiased and consistent estimator of the parameters is ordinary least squares. Non-parametric regression: as few assumptions on m as possible. In general, it is assumed to live in a metric functional space (Σ, d). E.g. (Σ, d) = L 2. Therefore, many estimators are constructed using bases of these spaces, such as wavelets, splines of polynomials. Kernel based estimators are also important. E.g. the Nadaraya-Watson estimator.

8 Example 2: distribution function Motivation Non-parametric statistics Lévy processes Let X 1,... X n be i.i.d. real random variables with distribution function F. Then it belongs to the functional space Σ = {G : R [0, 1] : lim G(x) = 0, lim G(x) = 1, x x G increasing and right continuous.} with the L metric. The natural estimator of F is F n (x) = 1 n n 1 (,x] (X i ), x R. i=1

9 Motivation Non-parametric statistics Lévy processes

10 Example 3: density function Motivation Non-parametric statistics Lévy processes Let X 1,... X n be i.i.d. real random variables with density function f. Then f belongs to the functional space Σ = {g : R [0, ) : g(x)dx = 1} with the L 1 norm. Here there is no natural estimator of f, but a typical one is the kernel density estimator f n (x, h) = 1 h K( /h) P n = 1 nh R n ( x Xi K h where K is a non-negative kernel that integrates to one, h is a bandwidth and P n = 1 n n i=1 δ X i is the empirical measure. i=1 ),

11 Motivation Non-parametric statistics Lévy processes Figure: Grey: true density (standard normal); Green: KDE with h=2; Black: h=0.337; Red: h=0.05.

12 Minimax rate of estimation Motivation Non-parametric statistics Lévy processes Let f be a function in a normed functional space (Σ, d). Question: what is the best rate in terms of risk under d-loss at which f can be estimated by an estimator f n? Definition Under the setting above, the minimax rate of the problem is the sequence r n of positive real numbers such that inf f n sup rn 1 E f [d(f, f n )] f Σ is bounded away from zero and infinity as n.

13 Minimax rate of estimation Motivation Non-parametric statistics Lévy processes Vaguely speaking, the minimax rate for a metric space (Σ, d) is the best rate at which an unknown function f in Σ can be estimated. It depends both on the complexity of Σ and on the metric d. Examples: Distribution function: the minimax rate in distribution function estimation is r n = 1/ n and, by Donsker s theorem (more details later), this rate is achieved by the estimator given by the empirical distribution function. Density function: denote by C s any Hölder ball of regularity s > 0 and let f C s Σ equipped with the L norm. Then it is well known that, up to logarithmic factors, the minimax rate is r n = n s/(2s+1) > 1/ n.

14 Minimax rate of estimation Motivation Non-parametric statistics Lévy processes In parametric models, where Σ R k for some k N and d may be the Euclidean norm, the minimax rate is always r n = 1/ n. A maximum likelihood-type of estimator is usually available and it achieves the 1/ n rate of convergence in view of the central limit theorem. In non-parametric models r n is in general larger than 1/ n (i.e. the best rate of convergence is slower than in the parametric case) and it is equal to it only in some cases. In these particular cases many times we also have distributional limits as in the parametric case.

15 Minimax rate of estimation Motivation Non-parametric statistics Lévy processes Therefore, we can divide statistical problems into two: 1 Those in which (Σ, d) admits n-consistent estimators ( statistically finite-dimensional models ); 2 And those in which (Σ, d) does not. Type 1. includes interesting probabilistic problems. The statistical interpretation of the results often proceeds along the lines of the classical parametric theory. Statistical inference (tests, confidence sets, etc.) can usually be made without serious difficulty. Type 2. is statistically more involved. The rate of estimation typically depends on specific geometric/analytic properties of Σ. Honest statistical inference is a highly nontrivial problem.

16 Definition Motivation and preliminaries Motivation Non-parametric statistics Lévy processes A Lévy process L = (L t ) t 0 is a continuous-time stochastic process satisfying the following properties: 1 L 0 = 0 a.s. 2 For any 0 t 0 t 1... t n, the increments are independent. L t0, L t1 L t0,..., L tn L tn 1 3 For any 0 s t L t L s is equal in distribution to L t s. 4 The sample paths of L are a.s. right continuous with left limits.

17 Characteristic function Motivation Non-parametric statistics Lévy processes A Lévy process is fully characterised by its characteristic function ϕ t (u) := E[exp(iuL t )]. It satisfies that ϕ t (u) = exp(tψ(u)), where ψ(u) := iγu σ2 ( 2 u2 + e iux ) 1 iux1 x <1 ν(dx), u R. R The function ψ is determined by the Lévy triplet (γ, σ, ν). It comprises: a drift-like part γ R; a volatility parameter σ 0; and the so-called Lévy measure ν, which is non-negative, σ-finite and satisfies R ( 1 x 2 ) ν(dx) <.

18 Interpretation of a Lévy process Motivation Non-parametric statistics Lévy processes Informally, any Lévy process L can be decomposed into independent components L t = σb t + γt + J t, where J is a pure jump process whose jumps are determined by the Lévy measure ν.

19 Motivation Non-parametric statistics Lévy processes Estimating L through its characteristic function The Lévy triplet comprises the characteristics that uniquely determine the Lévy process. Therefore we will consider the problem of estimating the Lévy triplet of a Lévy process or functionals of it from time-discrete observations of the process. As one may already be expecting, the challenges arise when dealing with the jump measure. Since there is only a moment assumption on it, this is a problem of non-parametric statistics.

20 Motivation Non-parametric statistics Lévy processes Assumptions on the observations of L In the following let us assume that we have n observations L ti of L at equidistant time steps t i = i, i = 0, 1,..., n, for some > 0 fixed. Thus, by the definition of a Lévy process and defining X i := L ti L ti 1, i = 1,..., n, we equivalently observe n independent realisations of a random variable X with characteristic function ϕ(u) = exp( ψ(u)), where ψ is as before.

21 Neumann and Reiß (2009) constructed a very general consistent estimator for the Lévy characteristics. The estimator concerning the Lévy measure does not estimate the measure itself but functionals of it, which is generally what is used in applications. They also find the minimax rates of the problem for a rich set of distances d, depending on the decay of the characteristic function of the Lévy process (this is directly related to ν and hence different decays give different Σ spaces from previous section). They show that the estimator achieves these optimal rates.

22 Main theorem Theorem (Neumann and Reiß, 2009) Let (ˆν σ ) n be the estimator of ν σ (dx) = σ 2 δ 0 (dx) + x 2 ν(dx) and let f be such that (1 + u ) s Ff (u) du 1, where F denotes the Fourier transform. Then if r n denotes the rate of convergence of f (ˆνσ ) n to f ν σ it holds that (log n) s/2 if σ > 0 : Re(log(ϕ(u))) σ 2 u 2 /2, r n = (log n) s if α > 0 : Re(log(ϕ(u))) α u, n s/(2β) n 1/2 if C, β > 0 : ϕ(u) C(1 + u ) β, and it is optimal for the (Σ, d) spaces they consider.

23 Drawbacks of the result This result is highly relevant as it informs us of the best rates of convergence we should expect when estimating functionals of the Lévy measure. In particular, we should expect to be in the n world when the characteristic function as polynomial decay. However it does not include any distributional limits. Hence no interesting statistical inference procedures such has confidence bands or tests can be derived from it; and it does not apply to estimating the cummulative function of the Lévy measure, which is a very natural object to estimate.

24 Definitions Motivation and preliminaries Main result Extensions A cummulative-type of function of the Lévy measure can be defined as (t is a space variable, not time) N(t) = t t ν(dx) if t < 0, ν(dx) if t > 0. Note that N(0) is not defined in general. Let us also define L ζ := L ((, ζ], [ζ, )) for some ζ > 0 fixed and { 1 g t (x) = x 1 (,t](x) if t < 0, 1 x 1 [t, )(x) if t > 0.

25 Main result Motivation and preliminaries Main result Extensions Motivated by the results of Neumann and Reiß (2009) and by its drawbacks, Nickl and Reiß (2012) constructed a natural estimator N n for N and proved a Donsker-type of theorem for the cummulative Lévy measure under the assumption of the characteristic function ϕ having polynomial decay.

26 Main result Motivation and preliminaries Main result Extensions Theorem (Nickl and Reiß, 2012) Assume ϕ has polynomial decay, ν has 2 + ε-bounded moment (not necessary) and xν is of bounded variation and has bounded Lebesgue density. Then ) n( Nn N G ϕ as n in distribution in L ζ, where Gϕ is a centered Gaussian random variable on L ζ with covariance structure Σ t,s = 1 ( ) 2 F 1 [ϕ 1 ( )] ( g t ( )) (u) R ( ) F 1 [ϕ 1 ( )] ( g s ( )) (u)p(du) t, s ζ and P is the distribution of X (the increment of the Lévy process).

27 Main result Extensions

28 Main result Extensions Consequences and difficulties of the theorem Thanks to the distributional character of this result, Kolmogorov-Smirnov-type of tests and confidence bands for N can be derived. There are two main difficulties in the proof of the theorem: one of them concerns understanding the pseudo-differential operator given by the convolution with F 1 [ϕ 1 ( )], and the other one has to do with overcoming the unboundedness of the estimator of N.

29 Main result Extensions Difficulties: the pseudo-differential operator 1 The operator f F 1 [ϕ 1 ( )] f is a pseudo differential operator which fractionally differentiates f. 2 When applied to f ( ) = 1 (,t] ( ), t < 0, it induces an integrable singularity at t. 3 In general it appears integrated against P which has an integrable singularity at the origin. 4 However, if we let t be arbitrarily close to 0 the singularity becomes non integrable. This roughly explains why the process is defined in L ζ with ζ > 0, although it also brings other difficulties in the proofs.

30 Main result Extensions Difficulties: the unboundedness of the estimator In order to understand the second difficulty let us derive the estimator N n they use. The assumption of the polynomial decay of ϕ automatically restrics the Lévy processes to those which have local finite variation. This implies that the Lévy exponent takes the form ψ(u) = iγu + (e iux 1)ν(dx). Suppose γ = 0 (it is possible to prove γ can be neglected wlog). Differentiating and using that ψ(u) = log(ϕ(u)) 1 ϕ (u) i ϕ(u) = iψ (u) = e iux xν(dx) = F[xν](u) R

31 Main result Extensions Difficulties: the unboundedness of the estimator Inverting the Fourier transform, diving the LHS and RHS by x and integrating from to t < 0 with respect to x gives 1 t 1 i x F 1[ ϕ ] t (x) dx = ν(dx). ϕ Thefore we can estimate the RHS by the empirical version of the LHS. Namely 1 t 1 i x F 1[ ϕ ] n (x) dx, ϕ n where ϕ n (u) = 1 n n e iux k, and ϕ n(u) = 1 n k=1 n i X k e iux k. k=1

32 Main result Extensions Difficulties: the unboundedness of the estimator However, the Fourier inverse of ϕ n/ϕ n may not be well-definedso we induce a smooth spectral cutoff by multiplying this ratio by F[K h ], K h ( ) = K( /h)/h, where K is a differentiable kernel integrating to one, decaying sufficiently fast and whose Fourier transform has compact support. Then the estimator is defined as N n (t) = 1 i R g t (x) F 1[ ϕ ] n F[K h ] (x) dx, t > 0. ϕ n

33 Main result Extensions Difficulties: the unboundedness of the estimator 1 The spectral cutoff guarantees that N n is well-defined with probability tending to one. 2 Nevertheless, for n finite, n( N n N) may not be bounded and therefore we cannot apply the classical empirical process theory to conclude the Donsker-type of theorem. 3 Giné and Nickl (2008) developed the theory of smoothed empirical processes which shows that, under certain conditions, smoothed empirical processes may converge in situations where the unsmoothed process does not. 4 This is in fact what happens here and it is the first application of this novel theory.

34 Main result Motivation and preliminaries Main result Extensions Theorem (Nickl and Reiß, 2012) Assume ϕ has polynomial decay, ν has 2 + ε-bounded moment (not necessary) and xν is of bounded variation and has bounded Lebesgue density. Then ) n( Nn N G ϕ as n in distribution in L ζ, where Gϕ is a centered Gaussian random variable on L ζ with covariance structure Σ t,s = 1 ( ) 2 F 1 [ϕ 1 ( )] ( g t ( )) (u) R ( ) F 1 [ϕ 1 ( )] ( g s ( )) (u)p(du) t, s ζ and P is the distribution of X (the increment of the Lévy process).

35 Tests and confidence bands Main result Extensions As already mentioned, because of the distributional nature of the result, valuable statistical inference procedures such as goodness of fit tests and confidence bands can be derived from it. Constructing these confidence bands follows directly from the theorem. However, they have to be constructed for t ζ > 0, so we cannot probabilistically bound the Lévy measure around the origin which is of great importance for applications. For this reason we are trying to understand how n( Nn (t n ) N(t n )) behaves as n when t n 0. Let us go into more details of the latter.

36 Main result Extensions Understanding the Lévy measure at the origin In general, N n and N have a discontinuity at the origin. To compensate this we have to multiply the process by a positive power of t n. Namely n t ρ n ( N n (t n ) N(t n )). From the proofs in Nickl and Reiß (2012) it is not hard to prove that ρ 1 necessarily. Furthermore, there is strong evidence that ρ depends on the Lévy process to be estimated, which also brings the problem of estimating it from the data. We still do not fully understand this and it again involves, among other things, understanding the effect of convolving with F 1 [ϕ 1 ( )].

37 Main result Extensions Understanding the Lévy measure at the origin In addition to the confidence band for N around the origin, this result would inform us of something else. The speed at which we approach the origin when constructing this band is given by t n, so it is arbitrarily fast. However, the term n t ρ n gives us the speed at which the random variable n t ρ n ( N n (t n ) N(t n )) approximates a normal distribution (this is not evident from what we have said). Hence we observe a trade off between the speed and the accuracy of approximation very relevant for applications.

38 Other extensions Motivation and preliminaries Main result Extensions A question of particular interest in the area of statistics for stochastic processes is whether one can allow for high-frequency observation regimes n 0. This will be published soon by Nickl et al. The estimator N n of course attains the minimax rate for the problem but, is it assymptotically efficient? I.e. is the limiting variance for each t the smallest as possible? It is in fact the case that it asymptotically achieves the Cramer-Rao bound and this will soon be published by Nickl et al. This result is only for one functional of the Lévy measure, namely the cummulative function of it, but Donsker-type of theorems for more general functionals of ν are highly relevant extensions of the result.

39 In order of appearance: Neumann and Reiß, Nonparametric estimation for Lévy processes from low-frequency observations. Bernoulli 15(1), Nickl and Reiß, J.Funct. Anal., Giné and Nickl, Uniform central limit theorems for kernel density estimators. Probab. Theory Related Fields 141,

40 Thanks for your attention!