Corrections to the Central Limit Theorem for Heavy-tailed Probability Densities

Transcription

1 Corrections to the Central Limit Theorem for Heavy-tailed Probability Densities Henry Lam and Jose Blanchet Harvard University and Columbia University Damian Burch and Martin Bazant Massachusetts Institute of Technology Abstract Classical Edgeworth expansions provide asymptotic correction terms to the Central Limit Theorem (CLT up to an order that depends on the number of moments available. In this paper, we provide subsequent correction terms beyond those given by a standard Edgeworth expansion in the general case of regularly varying distributions with diverging moments (beyond the second. The subsequent terms can be expressed in a simple closed form in terms of certain special functions (Dawson s integral and parabolic cylinder functions, and there are qualitative differences depending on whether the number of moments available is even, odd or not an integer, and whether the distributions are symmetric or not. If the increments have an even number of moments, then additional logarithmic corrections must also be incorporated in the expansion parameter. An interesting feature of our correction terms for the CLT is that they become dominant outside the central region and blend naturally with known large-deviation asymptotics when these are applied formally to the spatial scales of the CLT. Introduction Let (X k : k be a sequence of independent and identically distributed (iid random variables (rv s with mean zero and finite variance σ. If S n = X +...+X n, then the standard Central Limit Theorem (CLT says that for each fixed x R, ( P S n /n / x = P ( N (, σ x + o ( as n. Edgeworth expansions provide additional information about the previous error term, but only if higher moments exist. In particular, if E X k m < for m 3 and assuming that the X k s are continuous random variables (or simply non-lattice an Edgeworth expansion of order m takes the form P ( S n /n / x = P ( N (, σ x m ( + n j/ G j (x + o n m / ( as n, where G j ( is appropriately defined in terms of Hermite polynomials and involves coefficients that depend on EX j+ k (see Section below. As a consequence, a traditional Edgeworth expansion cannot continue beyond the m -th correction term in a situation where j=

2 ( E X k m +3 =. Our focus here is precisely on this situation, building on the preliminary results of Bazant (6. In particular, our goal in this paper is to provide additional correction terms to the Edgeworth expansion and further reduce the order of the error term as n in the presence of regularly varying distributions (which are basically power laws; see Equation (. As we shall discuss in detail below, an interesting feature of our development is the fact that the additional correction terms are qualitatively different (giving rise to expressions involving log (n in some cases depending on whether the power-law decaying rate of the distribution is an integer number or not, or, when such rate is an integer, on its parity. After this paper was completed we learned that a related set of expansion is given in Chapter of Vinogradov (994. Our results here appear to be more general than those expansions. In particular, our results apply to regularly varying tail of any α >, while Vinogradov considers only power-law tail with integer α. Also, our analysis allows us to obtain explicit expressions for the coefficients in our Edgeworth expansion in terms of special functions, an important interest in this paper that was not pursued by Vinogradov. Our results can be applied in the context of random walks (Hughes (995; Weiss (994, where corrections to the CLT are sometimes called Gram-Charlier expansions (and extended to vector random variables. As is typical for such expansions, our asymptotic formulae can be very accurate for small n, but unlike standard Gram-Charlier expansions, our corrections also become dominant at the edge of the central region and beyond, as illustrated by numerical examples in Bazant (6 (see also Theorem 3 in Section 4. Random walks with heavy-tailed distributions (or synonymously fat-tailed distributions in physics and other literatures, in particular those that are regularly varying, are increasingly used to describe stochastic processes in diverse fields, ranging from physics (Bouchaud and Georges (99; Metzler and Klafter ( to economics (Bouchaud and Potters (; Embrechts, Kluppelberg and Mikosch (997. Edgeworth expansions are also often used in statistics and applied probability. For instance, in statistics these types of expansions have enjoyed successful applications in statistical analysis involving relatively small samples (Field and Ronchetti (99. In bootstrapping, where data are resampled to calculate quantities related to the estimator, Edgeworth expansion can reduce the order of the approximation error (Hall (99. In applied probability settings such as queueing or insurance one is often interested in approximating performance measures of a stochastic system, for instance the tail of the delay in a queueing system or the ruin probability in insurance, by means of a CLT. In some cases, one can derive corrections to such approximations (Blanchet and Glynn (6. These corrections are often called corrected diffusion approximations and are similar in spirit to Edgeworth expansions. In fact, our ideas here can be applied to developing higher order corrections to a class of queueing models in heavy-traffic; such development will be explored in future work. Other types of approximation that are often applied to the distribution of S n include so-called large deviations or moderate deviations asymptotics. In contrast to Edgeworth expansions, which approximate the distribution of S n in spatial scales of order O ( n /, large deviations results typically provide approximations in spatial scales of order O (n, whereas moderate deviations are derived in spatial scales of order O ( n /+ε for ε (, /. In situations where the increment distribution has finite logarithmic moment generating function, standard results provide an exponential decay rate to the tail of S n in large deviations or moderate deviations scales, say for quantiles that are of the form nx. Using the fact that typically the rate function is twice continuously differentiable at the origin (as a function of x, one can relate (or transition from large, moderate and

3 3 CLT approximations smoothly. On the other hand, when the increment distributions are regularly varying, the large deviations rate, as we shall recall in future sections, turns out to be basically polynomial. Given that further terms in an Edgeworth expansion reduce the polynomial (in powers of n / error rate of the tail estimate, one may wonder how Edgeworth expansions relate to large deviations rates as additional correction terms are added. As we shall see, perhaps surprisingly, our additional terms blend smoothly with large and moderate deviations rates when these rates are formally applied to spatial scales of order n / only when the power of the regularly varying tail is non-even. These formal connections to large deviations asymptotics are further explored in Section 4; there we would discuss our asymptotics in relation to the classical uniform large deviations result by Rozovskii (989, in which the sum of iid regularly varying rv can be approximated by a sum of Gaussian distribution and heavy-tail asymptotic. We will also show that under restrictive conditions our asymptotic provides a refinement of this result in the moderate deviations region. As we indicated earlier, our development gives rise to qualitative differences in the form of the correction terms depending on the decay rate at infinity of the increment distributions. In order to further explain such qualitative differences, let us introduce some notation. In addition to the assumptions given at the beginning of this introduction, suppose that the X k s have a positive density f X ( of the form f X (x = ( + x (α+ L ( x, where L ( is a slowly varying function (i.e. L (xt /L (x as x for every t > (our development also deals with the non-symmetric case. Denote also η(x as the standard normal density. Theorem in Section yields that the density of S n /n / behaves asymptotically as f Sn/n /(x η(x + 3 j<α G j (x + n j/ { L(n / x G n α/ α (x when α is not even ζ L (n / x G n α/ α (x when α is even as n, where ζ L (x is a suitable slowly varying function (see Equation (6 and G α (x is written in terms of special functions whose qualitative behaviors depend on the parity of α (see Equation (. Our approach to developing the Edgeworth expansions is standard. First we develop an appropriate Gram-Charlier series for the Fourier transform and then apply inversion. An interesting element of the analysis in our situation, however, involves the contribution of a non-analytic component in the Fourier transform. We shall build the elements of our expansion starting from the symmetric case and then extend the construction of our approximation to more general regularly varying densities. Finally, we note that our approach can be used to develop asymptotic expansion for stable limits, which will be reported in future research. The paper is organized as follows. First, we discuss our main results in Section. Then we discuss in detail the symmetric case in Section 3. The formal connection to large deviations is given in Section 4. Section 5 studies the case of non-symmetric density.

4 4 Main Results Let (X k : k be a sequence of iid rv s with mean zero, unit variance and with a regularly varying density. More precisely, we assume that the density f ( of X k satisfies f (x = ( + x (+β L + (x I (x + ( x (+γ L (x I (x <, ( where β, γ > and L + (, L ( are slowly varying functions at infinity (i.e. L ± (±cb /L ± (±b as b. We shall write φ ( = E exp (ix k for the characteristic function of X k and put ψ ( = log φ (. Finally, we let S n = X X n, n, be the random walk generated by the X k s, set φ n ( = E exp ( S n /n / = φ n ( /n / and define ψ n ( = log φ n ( = nψ ( /n /. As we indicated in the introduction, our goal is to extend the standard Edgeworth expansion given by approximation (, which is obtained from a formal expansion of the Fourier transform of S n /n /. In particular, since ψ ( = we have nψ (/n / = κ i 3 3 3!n / κ i m m ( m m m!n m/ + o if m < min (β, γ. Therefore, ( E exp S n /n / ( = exp nψ (/n / = exp ( / m exp j=3 = exp ( / ( + κ 3 (i 3 κ j (i j n j/ n / +... n m/ ( m + o n m/,. (3 Recall that the inverse Fourier transform of (i k exp ( / equals d k η (x /dx k, where η (x = exp ( x / / π. In turn, we have that d k dx k η (x = η (x ( k H k (x, where H k (x is the k-th Hermite polynomial. In particular, H (x = x, H (x = x and H 3 (x = x 3 3x. Therefore, taking inverse Fourier transforms in both sides of (3 we obtain, at least formally, that the density f Sn/n / ( of S n/n / satisfies f Sn/n / (x = η (x + η (x m k=3 ( n k/+ P k (x + o n m/+, (4 where P k ( is a polynomial of degree k that depends only on the first k cumulants of X j. In particular, we have that P 3 = κ 3 H 3 /3!, P 4 = κ 3 H 3/7 + (κ 4 3 H 4 /4!. Our goal is to understand the contribution of the error term o ( n m/+ in expansion (4 and the problem is that subsequent P k s (for k > min (β, γ involve moments (or cumulants that are not defined for X j. The idea is to write ψ ( = χ (+ξ ( where χ ( is analytic in a neighborhood of the origin while ξ ( is a non-analytic function. Dealing with the analytic component, χ, gives

5 5 rise to the standard Edgeworth expansion as in (4, so the interesting part involves the analysis of ξ ( which, as we shall see, yields Theorem and that extends (4. Throughout this paper we assume the following differentiation condition on L ± (for simplicity we write L below: Assumption. L is differentiable and satisfies ( L(x L (x = o x as x Assumption controls the order of the derivative of L. Examples of slowly varying function that satisfy (5 include log x, (log x etc. We also introduce the function ζ L (x defined as Note that Assumption implies ζ L (x = x (5 L(u du (6 u L(x = o(ζ L (x (7 as x. Both Assumption and this condition will be used in our proof of the main result. Next let us define Dawson s integral D(z as z D(z = e z e t dt (8 and the classical parabolic cylinder function D ν (z with parameter ν, where Re(ν >, as /4 D ν (z = π ez e t / t ν cos(zt νπ dt (9 We state our main theorem: Theorem. Assume (X k : k are iid rv s with zero mean, unit variance and density following ( with L + and L satisfying Assumption. We have G j (x /(x η(x + + F (x, n f Sn/n n j/ 3 j<min(β,γ as n, where G j (x, j < min(β, γ are the ordinary Edgeworth coefficients and F (x, n is defined according to the following cases: Case : β = γ = α F (x, n = [ e x / Γ(α+n α/ π H α (x ( ζ L+ (n / x + ζ L ( n / x d α dx α D( x ( L + (n / x L ( n / x ] [ e x / Γ(α+n α/ π H α (x ( ζ L+ (n / x ζ L ( n / x d α dx α D( x ( L + (n / x + L ( n / x ] for even α for odd α π e x /4 [ Dα (xl Γ(α+ sin(απn α/ + (n / x + D α ( xl ( n / x ] for non-integer α

6 6 Case : β < γ Case 3: β > γ F (x, n = F (x, n = H β (xe x / πγ(β+ ζ L+ (n / x n β/ π D β (xe x /4 L + (n / x Γ(β+ sin(βπ n β/ H γ(xe x / πγ(γ+ ζ L ( n / x n γ/ Hγ(xe x / πγ(γ+ ζ L ( n / x n γ/ π D γ( xe x /4 L ( n / x Γ(γ+ sin(γπ n γ/ for integer β for non-integer β for even γ for odd γ for non-integer γ Here H k (z, D(z and D ν (z are Hermite polynomial of order k, Dawson s integral and classical parabolic cylinder function with parameter ν respectively. This result can be obtained by first considering a symmetric density and then splitting the non-symmetric density into odd and even functions which can be tackled separately using result for the symmetric case. For the symmetric case, i.e. β = γ = α and L L + = L, we have a neater representation: Theorem. Assume (X k : k are iid rv s with unit variance and symmetric density f(x = ( + x (α+ L( x with L satisfying Assumption and α >. Then f Sn/n /(x η(x + { G j (x L(n / x n j/ + G n α/ α (x for non-even α ζ L (n / x G n α/ α (x for even α 3 j<α as n, where ζ L is defined in (6. Here G j (x, j < α are the ordinary Edgeworth coefficients, while G α (x is defined as d α Γ(α+ dx D( α x for odd α G α (x = π e x /4 Γ(α+ sin(απ [D α(x + D α ( x] for non-integer α ( e x / π Γ(α+ H α(x for even α where D(z, D ν (z and H k (z are Dawson s integral, classical parabolic cylinder function with parameter ν and Hermite polynomial of order k respectively. As we indicated before, the analysis behind the previous result involves understanding the behavior of the non-analytic component ξ. In fact, we obtain the following decomposition of ψ: Proposition. Assume a symmetric density as in Theorem. We have ψ( = χ( + ξ( + o(ξ( (

7 7 where ξ( = χ( Γ(α+ sin απ j<α is the ordinary Taylor series expansion up to the largest moment and { π α L( ( α/ Γ(α+ α ζ L ( is the non-analytic component. Here ζ L is defined in (6. ( j/ κ j j ( j! for non-even α for even α (3 With this decomposition, we can perform similar derivation as Equation (3 to obtain E exp (S n /n / exp ( / ( + κ 3 (i 3 ( n / nξ n /. Dawson s integral and the other special functions then arise as the inverse Fourier transform of e / α appearing in the non-analytic term of the above expression. The rest of the paper provides the necessary details behind the analysis of ξ and the Fourier inversion required to obtain Theorem and then Theorem ; the proof of Proposition follows directly from Proposition 3, which is given in Section 3. As we indicated in the introduction, in Section 4 we shall discuss the formal connection between our expansion and that of Rozovskii (989 in the context of large deviations. It turns out that the extra term in our expansion, which comes from the non-analytic component of the characteristic function, blends smoothly with the heavy-tail asymptotic, at least in some restrictive cases. When this holds, there exists a smooth transition from central limit to large deviations region. 3 Symmetric Density Our goal in this section is to prove Theorem. As noted before, we adopt the approach of first expanding the characteristic function of a symmetric regularly varying density, and hence its cumulant generating function, followed by an inverse Fourier transform to get the stated result. In order to obtain the characteristic function for regularly varying density, it helps to first consider in detail the nature of the approximation in a simplified context, namely, that of a symmetric Pareto density. 3. Characteristic Function 3.. Pareto Density In this section we shall assume that f (x = a f, x (,, (4 + x +α where a f is the normalizing constant that makes f (x a well defined density. For example, a f = π/ if α = 3. With this we have the simplest version of our expansion:

8 8 Proposition. For rv following (4, the characteristic function takes the form φ( j<α ( j/ m j j + j! { πa f Γ(α+ sin απ α for non-even α ( α/ a f Γ(α+ α log for even α (5 where m j is the j-th moment of the rv. The proof of this theorem is divided into four cases: α is odd, even, and non-integer with the integral part being odd and even. Each case involves finding the asymptotic of an integral as below. Also note that since the density is symmetric, the characteristic function is real and symmetric. Thus we can consider only > without loss of generality. Lemma. For >, we have the four asymptotics: Case : α is odd x α (e ix + x +α dx π (6 Case : α is even x α e ix dx i log (7 + x +α For non-integer α, denote q = α as the integral part of α. Then Case 3: Case 4: q is even q is odd x q (e ix ( α + q + x +α dx α q Γ( α + q cos π x q e ix ( α + q + x +α dx iα q Γ( α + q sin π Proof. We prove the lemma case by case: Case : α is odd Since α is odd, the integrand is a symmetric function, and we can write x α (e ix + x +α dx = = x α (cos(x dx + x+α u α +α (cos u du + u+α

9 9 by a change of variable u = x in the second equality. Since u α +α (cos u + u+α cos u u which is integrable, by dominated convergence theorem we get as. Hence Case is proved. Case : α is even u α +α (cos u du + u+α Note that the integrand is an odd function, and x α e ix dx = i + x +α = i cos u u du = π x α sin(x + x +α dx u α sin u +α du + u+α again by a change of variable u = x. Note that as, the integral blows up at u =. So for small ɛ, we have u α sin u ɛ u α sin u +α du + u+α +α du + u+α ɛ u α u Hence Case is proved. = ɛ Case 3 and 4: α is non-integer with q even and odd +α + u u α +α du + u+α +α du = + α log(+α + u +α log The proof is similar to the last two cases. For even q, we write x q (e ix dx = + x +α ɛ x q (cos(x dx + x+α = α q u q +α (cos u du + u+α α q cos u du u+α q and for odd q, we write x q e ix x q sin(x dx = i dx + x +α + x+α = i α q u q sin u +α + u i α q sin u du u+α q +α du

10 by using a change of variable u = x and dominated convergence theorem. Note that we have used the fact that to get and u z (e iu du = Γ(ze izπ for < Re(z < The lemma is then immediate. cos u + q du = Γ( α + q cos( α π u+α q sin u + q du = Γ( α + q sin( α π u+α q We are now ready to prove Proposition. The proof basically reduces (5 into expressions in terms of the above four integrals through multiple use of L Hospital s rule. Proof of Proposition. It suffices to consider >. Again we consider case by case: Case : α is odd Applying L Hospital s rule successively for α times, we have lim = lim = lim. = lim = ( (α / a f α! = ( (α+/ πa f α! a f e ix dx ( j/ m j + x +α j<α j! j α a f ixe ix + x +α dx j<α α α a f i x e ix dx + x +α j<α α(α α ( j/ m j (j! j ( j/ m j (j! j a f i α x α e ix dx ( (α / m + x +α α α! lim x α (e ix dx + x +α where the last step follows from (6. Note that the integral in each step above is finite since the moments exist up to order α. The result for odd α thus follows.

11 Case : α is even Using the fact that d m dk m kα log k = α(α (α m + k α m log k + γ α m k α m for m =,..., α, where γ α m are constants, and applying L Hospital s rule for α times, we get lim = lim = a f i α α! a f e ix dx ( j/ m j + x +α j<α j! j α log a f i α x α e ix dx + x +α α! log + γ lim = ( α/ a f α! where the last equality follows from (7. The result for even α thus follows. x α e ix + x +α dx log Case 3 and 4: α is non-integer with q even and odd Following the same line of proof as the previous two cases, we get, for even q, and for odd q, lim = lim a f e ix dx ( j/ m j + x +α j<α j! j i q a f α x q (e ix dx + x +α α(α (α q + α q = ( q/ a f Γ(α q + Γ(α + = ( q/ πa f cos( α+q π Γ(α + sin(( α + qπ ( q/ πa f = Γ(α + sin( α q π ( q/ πa f = Γ(α + sin απ πa f = Γ(α + sin απ cos qπ Γ( α + q cos( α + q π

12 lim = lim a f e ix dx ( j/ m j + x +α j<α j! j i q a f α x q e ix dx + x +α α(α (α q + α q = ( (q+/ a f Γ(α q + Γ(α + = ( (q+/ πa f sin( α+q π Γ(α + sin(( α + qπ ( (q+/ πa f = Γ(α + cos( α q π ( (q+/ πa f = Γ(α + sin απ πa f = Γ(α + sin απ sin qπ Γ( α + q sin( α + q π by using the reflection property of gamma function and also the double-angle and sum-of-angle trigonometric identities. We have proven the proposition. 3.. Regularly Varying Density We are going to extend the result in Proposition to density with regularly varying tail in the form f (x = L( x, x (, (8 + x +α where L is a slowly regularly varying function and satisfies the differentiation condition in Assumption. Proposition 3. For rv following (8 with L satisfying Assumption, the characteristic function takes the form ( φ( ( j/ m π j j Γ(α+ sin + απ α L for non-even α ( (9 j! ( α/ Γ(α+ α ζ L for even α j<α where m j is the j-th moment of the rv and ζ L is defined in (6. To prove Proposition 3, we need the following property of slowly varying function:

13 3 Lemma. Let L be a slowly varying function. Then for any ρ, ρ >, we can pick an η > such that for < < η, L( x { Cx L( ρ for x > Cx ρ for x where C is a positive constant. Proof. Karamata s representation (see for example, Resnick, 987 states that for a slowly varying function L : R + R +, we have { x } L(x = c(x exp t ɛ(tdt where c : R + R + and ɛ : R R such that So we have lim c(x = c (, x lim ɛ(t = t L( x L( = c( x { x } c( exp t ɛ(tdt Note that c(x is bounded from above and c(/ > c > for small enough. So c( x c( C for some C > for small. Also by a change of variable t = s/, we have { x } { x exp t ɛ(tdt = exp s ɛ( s } ds Suppose x >. For any given ρ, if we pick small enough, we have ɛ( s < ρ for all s >, and { x exp s ɛ( s } { x } ds exp s ρ ds = x ρ The result for x > follows. Similarly, for x, if we pick small enough we have ɛ( s > ρ, and { x exp s ɛ( s } { ds = exp s ɛ( s } { } x ds exp s ρ ds x ρ x Thus the lemma is proved. We now state the correspondence of Lemma for regularly varying density:

14 4 Lemma 3. For >, we have the four asymptotics: Case : α is odd Case : α is even x α L( x (e ix ( + x +α dx πl x α L( x e ix + x +α dx iζ L ( where ζ L is defined in (6. For non-integer α, denote q = α as the integral part of α. Then Case 3: Case 4: q is even q is odd x q L( x (e ix ( ( α + q + x +α dx α q L Γ( α + q cos π x q L( x e ix ( ( α + q + x +α dx iα q L Γ( α + q sin π Proof. Again we prove the lemma case by case. Case : α is odd We follow similar line of proof as in lemma. Using a change of variables u = x we obtain x α L( x (e ix u α L( u + x +α dx = +α (cos u du + u+α = L( u α +α + u +α L( u (cos u du Now using lemma, for small enough, we can pick < ρ, ρ < such that u α +α + u +α L( u L( C u +ρ (cos u (cos u I(u + C u ρ L( (cos u I(u > for some C >. Note that the majorizing function is integrable. Also note that by the definition of slow variation we have L( u L( as for all u >. So by dominated convergence theorem, u α +α + u +α L( u L( and the asymptotic follows. (cos u du (cos u du u

15 5 Case : α is even We write x α L( x e ix + x +α dx = i = i [ = i x α L(x sin(x + x +α dx u α L( u sin u +α + u +α du u α L( u sin u +α du + + u+α u α L( u sin u ] +α + u +α du For the second integral, if we choose < ρ < in Lemma, we have the following bound u α L( u sin u ( +α + u +α du L u α Cu ρ ( +α + u +α du M L ( for some C, M >. For the first integral, we write where u α L( u sin u u α L( u +α du = + u+α +α du + R( + u+α R( M L for some M, M 3 > and < ρ <. ( Now do a change of variable x = u/ back, we get u α+ u ρ +α + u +α du M 3L ( ( u α L( u +α du = + u+α x α L(x dx + x+α Combining (, ( and (, we obtain the result for even α. L(x dx ( x Case 3 and 4: α is non-integer with q even and odd Using the same line of proof as Lemma and Case here taking < ρ < ρ < α + q, we can get the stated result. < α q and The proof of Proposition 3 is similar to that of Pareto density, and hence we set aside the details of the proof. Proposition then follows immediately by letting ψ = log φ and applying a Taylor expansion. In doing so one just has to verify that the non-analytic term in φ is the only term of that order in the expansion of ψ to conclude the result.

16 6 Outline of Proof of Proposition 3. Again it suffices to consider >, and we use L Hospital s rule successively. For odd α, recognizing that L ( = o(l( as by Assumption, we have lim = lim = lim. = ( (α / α! = ( (α+/ π α! L(xe ix dx ( j/ m j + x +α j<α j! j α L( L(xixe ix dx + x +α j<α α α L( α L ( L(xixe ix dx + x +α j<α α α L( lim ( j/ m j (j! j ( j/ m j (j! j L(xx α (e ix dx + x +α L( The case for even α can be proved similarly by using (7 that states that L( = o(ζ L(, while the case for non-integer α is proved similarly with Assumption. 3. Beyond Edgeworth Expansion We will now prove the main theorem for the symmetric case, namely Theorem. First of all we need to evaluate the following integrals, which corresponds to a Fourier inversion of the non-analytic component of the expansion studied in Section 3.: Lemma 4. ( (α / dα e ix / α dx D( α x for odd α d = π απ sec e x /4 [D α (x + D α ( x] for non-integer α ( α/ πe x / H α (x for even α where D(z, D ν (z and H k (z are Dawson s integral, classical parabolic cylinder function with parameter ν and Hermite polynomial of order k respectively. Proof. Consider the following cases: Case : α is odd

17 7 Observe that e ix / α d = cos(xe / α d (α / dα = ( dx α (α / dα = ( dx α Im (α / dα = ( dx α Im by letting q = ix. We evaluate the integral ix ix e q / dq sin(xe / d [ ] e ix / d [ e x / ix ix ] e q / dq by forming a closed anti-clockwise rectangular contour from ix to R ix to R to to ix. The contour integral is zero by analyticity. Note that π e q / dq = The integral from R ix to R tends to zero as R. By a change of variable t = q/i, the integral along the segment from to ix is ix e q / dq = i x/ e t dt = i e x / D( x where D(z is the Dawson s integral. Therefore we have and the result follows. ix ix e q / dq = π + i e x / D( x Case : α is non-integer Again we have e ix / α d = cos(xe / α d Using sum-of-angle formula and (9, we can write the classical parabolic cylinder function as απ /4 D α (x = cos π ex e t / t α απ /4 cos(txdt + sin π ex e t / t α sin(txdt Since the first term is even while the second term is odd in x, we have απ /4 D α (x + D α ( x = cos π ex e t / t α cos(txdt The result follows.

18 8 Case 3: α is even Note that the integral now becomes analytic and the result is obvious by the discussion in the beginning of Section. We are now ready to prove Theorem : Proof of Theorem. We follow the proof of Evans and Swartz (. express the characteristic function First note that we can ψ( = χ( + ξ( + R( where χ and ξ are as defined in ( and (3, and R is the error term. Also let λ( = χ( + + ξ( be the expansion of characteristic function but trimming the first term and the error term. Let α/ (nλ(/ n k w n ( = k! and q n ( be the expansion of w n ( up to order /n α/. In other words, 4 π q n ( = + κ 4 4! n + + α L(n / / Γ(α+ sin απ for non-even α n α/ for even α k= ( α/ Γ(α+ α ζ L (n / / n α/ Also let r n ( = w n ( q n ( be the error of the trimming. It is easy to verify through direct expansion that { Pα( + for non-even α r n ( = n α/ P α( + for even α n α/ where P α ( are polynomials in, and the higher order terms may possibly involve L(n / / or ζ L (n / /. We first show that e ix / q n ( d π is a good approximation to f Sn/ n(x = e ix+nψ(/ n d/π. After that we will verify using Lemma 4 that this approximation is the one shown in Theorem. Consider δ n δ n [ e ix+nψ(/ n e ix / q n ( e nψ(/ n e / q n ( d π + / ( δ n,δ n e / q n ( d π + ] d π / ( δ n,δ n ( n φ d n π

19 9 for any δ. Later we are going to choose the value of δ to bound our error. We first consider the third integral. By non-lattice assumption, given any δ, we can find τ δ such that φ(y τ δ for any y > δ. Hence ( n ( φ d n π τ n δ φ d n π = o(n p / ( δ n,δ n / ( δ n,δ n for any p >. The last equation follows from our assumption that φ is integrable. Now consider the second integral. Note that for any β, e / β d = o(n p / ( δ n,δ n for any p >. Also, for any α > and slowly varying function U, ( / ( δ n,δ e / α n / U d C q n / ( δ n,δ e / α q n q/ d = o(n p n for any q > and C q a positive constant depending on q. Again p > is arbitrary. By plugging in the expansion of q n, together with these observations, we can conclude easily that the second integral is o(n p for any p >. Finally consider the first integral δ n δ n δ n δ n e nψ(/ n e / q n ( d π e / e nψ(/ n+ / w n ( d π + δ n δ n e / r n ( d π By our observation on r n (, we can verify easily that ( δ n δ e / r n ( d n π = O ( O n α/ n α/ for non-even α for even α Now consider the first part. We use the estimate γ β j eα j! eα e β γ + β j eβ j! j= j= ( α β + β γ+ e max{α,β} (γ +! where we use mean value theorem in the last inequality. Also, we can find δ small enough so that for [ δ, δ], we have ψ( + and λ( 4 4 which implies that for [ δ n, δ n], we get ( nψ n + 4 and ( nλ n 4

20 Using the above estimates, Now since δ n e nψ(/ n e / q n ( d π ( nr d n π + δ n δ n δ e /4 n δ n δ e /4 n ( ( P ( nλ n = o n α/ +p nλ(/ n ( α/ +! α/ + d for [ δ n, δ n] and any p >, and P ( is a polynomial in, it is then easy to see that δ n e /4 nλ(/ n α/ + ( d ( α/ +! π = o n α/ δ n On the other hand, we have R( = o ( o ( α L ( ( α ζ L for non-even α for even α Suppose that α is non-even. For given ɛ, we can choose δ small enough such that for [ δ, δ], ( R( ɛ α L which implies that for [ δ n, δ n], ( ( nr n ɛ α n α/ L n / and δ n δ e /4 n ( nr d n π ɛ δ n δ e /4 n ( α n α/ L Note that by Lemma we can pick M > and any ρ, ρ > such that L( n/ { C L(n / ρ for < C ρ for for n > M. So by dominated convergence theorem and that as n for all nonzero, we have δ n = δ e /4 n δ n δ n n / d π L( n/ L(n / (3 ( α n α/ L e /4 α L(n/ / L(n / e /4 α d π L(n / n α/ n / d π d π L(n / n α/ π

21 Hence Since ɛ is arbitrary, we get δ n δ e /4 n lim sup n δ n δ n e /4 nr L(n / /n α/ ( ( nr d n π = o ( n d π ɛc ( L(n / n α/ = o L(n / x n α/ for any fixed x. The case for even α is the same except L is replaced by ζ L that is also a slowly varying function. All the above estimates hold once we choose δ sufficiently small. We can therefore conclude that [ ] d e ix+nψ(/ n e ix / q n ( π = We now evaluate e ix / q n ( d π. Note that e ix / q n ( d π = η(x + so it suffices to focus on the last term 3 j<α o o ( L(n / x n α/ ( ζl (n / x n α/ G j (x + n j/ for non-even α for even α ( d e ix / nξ n / π = ( d e ix / nξ n / π π Γ(α+ sin απ e ix / α L( n/ e ix / α ζ L ( n/ ( α/ Γ(α+ d π d π n α/ n α/ for non-even α for even α By dominated convergence theorem and (3, we have for odd α, ( π Γ(α + sin απ e ix / α n / L π = Γ(α + sin απ π Γ(α + sin απ = α! d α dx α D( x L(n/ x n α/ n/ L( e ix / α L(n / e ix / α d π d π d π L(n / x n α/ n α/ L(n / n α/ by using Lemma 4 in the last equality. For non-integral and even α essentially the same proof follows, again noting that ζ L (x is also a slowly varying function for proving the even α case.

22 4 Beyond Central Limit Region Although this paper concerns expansions in the central limit region, we devote this section to an interesting connection of our extra term with the large deviations of regularly varying rv s. Note that by local analysis (see for example, Bender and Orszag (999 as x and for non-integral ν, D ν (z So for odd and non-integral α, d α dx α D( x ( α α! x +α { z ν e z /4 as z + π Γ( ν ez /4 z ν G α (x x (+α as z as x. The result for non-integral α follows from the observation that the negative side of D α in G α (x dominates and the reflection property of gamma function. Thus when α is non-even, we get G α (x L(n/ x n α/ L(n/ x +α n α/ (4 for x. On the other hand, Rozovskii (989 proved a result for regularly varying rv on the whole axis: [ ( ] x P (S n > x = Φ n + np (X k > x ( + o( (5 uniformly for x > n. In particular, the result implies that for x > (α n log n, the large deviations asymptotic (the second term on the right hand side dominates i.e. P (S n > x np (X k > x. If we assume the existence of density, this can be written as f Sn/ n(x nf(x n n for x > (α log n, which is exactly (4. In other words, the non-analytic term matches exactly with the large deviations asymptotic. However, for even α our extra term resembles standard Edgeworth expansion, and there is no region on the real axis that this can coincide with the large deviation asymptotic. To rigorously show the validity of the connection would require knowledge about the secondorder variation of the slowly varying function L i.e. how fast L(xt/L(x for given t. This will be a direction for future research, but in this paper we are satisfied that at least in special cases this connection is easily seen to be valid: Theorem 3. Assume (X k : k are iid rv s with unit variance and symmetric Pareto density f(x = a f /(b f + x α+ where α is non-even, and a f and b f are constants. Then we have the asymptotic f Sn/n /(x η(x + 3 j l(x G j (x n j/ + G α(x n α/ (6

23 3 for x (α log n, where G j (x are defined as in Theorem and { l(x = max j < α : η(x G ( } j(x n j/ = Ω Gα (x n α/ In other words, the refined moderate deviations over the range x (α log n is the ordinary Edgeworth expansion up to the term that is asymptotically at least as large as our non-analytic term, and then followed by the non-analytic term. Proof. The proof is a simple adaptation from the proof of Theorem. Note that in that proof we bound uniformly the error of Edgeworth expansion (over the real axis, but this error can be too coarse for x outside the central limit region. It suffices, therefore, to refine our error estimate so that it always has smaller order than our Edgeworth expansion over x (α log n. We will prove the result for the odd α case. The proof for non-integer α is similar. An easy examination of our proof of Theorem would reveal that for Pareto rv, the only error that needs further refinement is that caused by δ n δ e /4 n ( nr d n π For this we need a finer estimate of R(, which in turn needs a finer version of Case in Lemma. Note that a f x α (e ix b f + x +α dx = a f = a f = a f x α (cos(x b f + x +α dx u α (cos u b f +α du + u+α cos u u du a f b f +α The first part becomes a f π, while the second part is a f b f +α = a f b f +α [ ɛ (cos u u (b f +α + u +α du (cos u u (b f +α + u +α du + ɛ (cos u u (b f +α + u +α du ] (cos u u (b f +α + u +α du where ɛ is a small number. Note that for < u < ɛ we have (cos u /u bounded around. Hence ɛ (cos u u (b f +α + u +α du ɛ C b f +α du + u+α = C α ɛ C α dx b f + x+α

24 4 where C are positive constants. Obviously Therefore ɛ (cos u u (b f +α + u +α du = O( a f x α (e ix b f + x +α dx = a f π + O( and hence R( = O( +α. This implies easily that δ n ( ( e /4 nr d n π = O n α/ / δ n We then note that this leads to a uniform estimate of our error in the Edgeworth expansion that always has order less than G j (x/n α/ over x (α log n. So we are done. Similarly for non-integer α we have { O( R( = α + when α is even O( α when α is odd and the same result holds. 5 Non-Symmetric Density With the result for symmetric density in hand, we can generalize our results to the non-symmetric counterpart, namely Theorem. The main idea is to write a non-symmetric density as where is a symmetric function, and f(x = r(x + s(x r(x = [f(x + f( x] (7 s(x = [f(x f( x] (8 is an odd function. The symmetric function r can be easily handled using the results in the last section, and the odd function s can also be dealt with by slight modification. Indeed we have the following proposition: Proposition 4. For rv following ( with L + and L satisfying Assumption, we can decompose its density as f(x = r(x + s(x where r and s are defined by (7 and (8 respectively. Then r and s must be regularly varying and have the same order α min(β, γ, and we can write r(x = L r(x + x +α s(x = L s(x, < x < (9 + x +α

25 5 where L r is even and L s is odd, and they are both slowly varying functions satisfying Assumption. Also define m (r n = x n r(xdx m (s n = x n s(xdx for n α as the moments of r and s. The cumulant generating function takes the form ψ( = χ( + ξ( where χ( j<α κ j (i j j! is the ordinary Taylor series expansion up to the largest moment, and i [ α Γ(α+ α ζ Lr ( + i π L s( ] for even α i ξ( = [ α Γ(α+ α ζ Ls ( + i π L r( ] for odd α π [ Γ(α+ sin(απ α cos απ L r( απ i sin L s( ] for non-integer α (3 is the non-analytic component. Proof. That r and s are regularly varying, have the same order and can be written as (9 satisfying Assumption is obvious. Now define φ r ( and φ s ( as the Fourier transforms, or characteristic functions of r and s. Since r is symmetric, by the same argument as Proposition 3, we get φ r ( j<α m (r (i j j j! { π α ( L r + α ( ζ Lr Γ(α+ sin απ ( α/ Γ(α+ for non-even α for even α Now consider the odd function s. Note that the role played by odd and even α (or q is reversed because of odd instead of even function. Also note that φ s ( = φ s (, so it suffices to prove the result for > and the < case follows. Following similar line of proof as Proposition 3, we get φ s ( j<α j odd m (s (i j j j! + { iπ Γ(α+ cos απ i α Γ(α+ α ζ Ls ( α ( L s for non-odd α for odd α Adding (3 and (3, and noting that the representation can be carried over to cumulant generating function, we obtain the stated result. (3 (3 We also need a correspondence of Lemma 4 for odd function, the proof of which is very similar and is thus omitted:

26 6 Lemma 5. iα+ dα e ix / (± α dx D( α x for even α d = πe x / i α H α (x for odd α i π απ csc e x /4 [D α (x D α ( x] for non-integer α (33 where D(z, H k (z and D ν (z are Dawson integral, Hermite polynomial of order k and classical parabolic cylinder function with parameter ν respectively. Here the ± sign corresponds to > and <. The proof of Theorem follows easily from Proposition 4 and Lemma 5: Proof of Theorem. By ( we can write L r and L s as L r (x = [L +( x + L ( x ] L s (x = ± [L +( x L ( x ] when β = γ L r (x L +( x L s (x ± L +( x when β < γ L r (x L ( x L s (x L ( x when β > γ (34 where the ± and signs correspond to x and x <. We can proceed similarly as in the proof of Theorem by substituting (34 into L r and L s, and using Lemma 5. Note the use of (7 in obtaining the β < γ and β > γ cases. 6 References. Bazant, M. Z. (6, Random Walks and Diffusion, Lecture Notes, MIT OpenCourse- Ware, (See the notes and supplementary material for lecture 7.. Bender, C. M., and Orszag, S. A. (999, Advanced Mathematical Methods for Scientists and Engineers: Asymptotic Methods and Perturbation Theory, nd edition, Springer-Verlag. 3. Bouchaud, J. P. and Georges, M. (99, Anomalous diffusion in disordered media: Statistical mechanisms, models, and physical applications, Physics Reports 95 (4-5, Bouchaud, J. P., and Potters, M. (, Theory of Financial Risks, Cambridge, UK: Cambridge University Press. 5. Embrechts, P., Kluppelberg C., and Mikosch, T. (997, Modelling Extremal Events for Insurance and Finance, Springer. 6. Evans, M., and Swartz, T. (, Approximating Integrals via Monte Carlo and Deterministic Methods, Oxford Statistical Science Series. 7. Field, C. A., and Ronchetti, E. (99, Small Sample Asymptotics, Institute of Mathematical Statistics, Lecture Notes - Monograph Series. 8. Hall, P. (99, The Bootstrap and Edgeworth Expansion, Springer-Verlag.

27 7 9. Hughes, B. (996, Random Walks and Random Environments, Vol., Oxford, UK: Clarendon Press.. Metzler, R. and Klafter, J. (, The random walk s guide to anomalous diffusion: a fractional dynamics approach, Physics Reports 339, Resnick, S. I. (987, Extreme Values, Regular Variation, and Point Processes, Springer- Verlag.. Rozovskii, L. V. (989, Probabilities of large deviations of sums of independent random variables with common distribution function in the domain of attraction of the normal law, Theory Probab. Appl. 34(4, Vinogradov, V. (994, Refined Large Deviation Limit Theorems, Longman Scientific & Technical. 4. Weiss, G. H. (994, Aspects and Applications of the Random Walk, North-Holland.