Fourier Transforms of Measures

Transcription

1 Fourier Transforms of Measures Sums of Independent andom Variables We begin our study of sums of independent random variables, S n = X 1 + X n. If each X i is square integrable, with mean µ i = EX i and variance σi = E[(X i µ i ), then S n is square integrable too with mean ES n = µ n = i n µ i and variance VS n = σ n = i n σ i. But what about the actual probability distribution? If the X i have density functions f i (x i ) then so does S n ; for example, with n =, S = X 1 + X has CDF F (s) and pdf f(s) = F (s) given by P[S s = F (s) = f 1 (x 1 )f (x ) dx 1 dx = f(s) = F (s) = = x 1 +x s s x f 1 (x 1 )f (x ) dx 1 dx f 1 (s x )f (x ) dx f 1 (x 1 )f (s x 1 ) dx 1, the convolution of f 1 (x 1 ) and f (x ). Even if the distributions aren t absolutely continuous, so no pdf s exist, S has a distribution measure µ given by µ(ds) = µ 1(dx 1 )µ (ds x 1 ). There is an analogous formula for n = 3, but it is quite messy; things get worse and worse as n increases, so this is not a promising approach for studying the distribution of sums S n for large n. If CDF s and pdf s of sums of independent V s are not simple, is there some other feature of the distributions that is? The answer is Yes. What is simple about independent random variables is calculating expectations of products of the X i, or products of any functions of the X i ; the exponential function will let us turn the partial sums S n into products e S n = e X i or, more generally, e zs n = e zx i for any real or complex number z. Thus for independent V s X i and any number z we can use independence to compute the expectation Ee zs n = n Ee zx i, often called the moment generating function and denoted M X (z) = Ee zx for any random variable X. For real z the function e zx becomes huge if X becomes very large (for positive z) or very negative (if z < ), so that even for integrable or square-integrable random variables X the expectation M(z) = Ee zx may be infinite. Here are a few examples of Ee zx for some familiar distributions: Binomial: Bi(n, p) [1 + p(e z 1) N z C Neg Bin: NB(α, p) [1 (p/q)(e z 1) α z C i=1 Poisson Po(λ) e λ(ez 1) Normal: No(µ, σ ) e zµ+z σ / z C z C Gamma: Ga(α, λ) (1 λz) α (z) < 1/λ Cauchy: a π(a + (x b) ) e zb a z (z) =

2 Aside from the problem that M(z) = Ee zx may fail to exist for some z C, the approach is promising: we can identify the probability distribution from M(z), and we can even find important features about the distribution directly from M: if we can justify interchanging the limits implicit in differentiation and integration, then M (z) = E[Xe zx and M (z) = E[X e zx, so (upon taking z = ) M () = EX = µ and M () = EX = σ + µ, so we can calculate the mean and variance (and other moments EX k = M (k) ()) from derivatives of M(z) at zero. We have two problems to overcome: discovering how to infer the distribution of X from M X (z) = Ee zx, and what to do about distributions for which M(z) doesn t exist. Characteristic Functions For complex numbers z = x + iy the exponential e z can be given in terms of familiar real-valued transcendental functions as e x+iy = e x cos(y) + ie x sin(y). Since both sin(y) and cos(y) are bounded by one, for any real-valued random variable X and real number ω the real and imaginary parts of the complex-valued random variable e iωx are bounded and hence integrable; thus it always makes sense to define the characteristic function φ X (ω) = Ee iωx = e iωx µ X (dx). Of course this is just φ X (ω) = M X (iω) when M X exists, but φ X (ω) exists even when M X does not; on the chart above you ll notice that only the real part of z posed problems, and (z) = was always OK, even for the Cauchy. Binomial: Bi(n, p) [1 + p(e iω 1) N Uniqueness Neg Bin: NB(α, p) [1 (p/q)(e iω 1) α Poisson Po(λ) e λ(eiω 1) Normal: No(µ, σ ) e iωµ ω σ / Gamma: Ga(α, λ) (1 iλω) α Cauchy: a/π a + (x b) e iωb a ω Suppose that two probability distributions µ 1 (A) = P[X 1 A and µ (A) = P[X A have the same Fourier transforms ˆµ j (ω) = E[e iωx j = eiωx µ j (dx); does it follow that X 1 and X have the same probability distributions, i.e., that µ 1 = µ? The answer is yes; in fact, one can recover the measure µ explicitly from the function ˆµ(ω). Thus we regard uniqueness as a corollary of the much stronger result, the Fourier Inversion Theorem. esnick has lots of interesting results about characteristic functions in Chapter 9, Grimmett and Stirzaker discuss related results in their Chapter 5, and Billingsley proves several versions of this theorem in his Section 6; I m going to take a different approach, and stress the two special cases in which µ is discrete or has a density function, trying to make some connections with other encounters you might have had with Fourier transforms. Page

3 Inversion: Integer-valued Discrete Case Notice that the integer-valued discrete distributions always satisfy φ(ω + π) = φ(ω) (and in particular are not integrable over ), while the continuous ones satisfy φ(ω) as ω ±. For integer-valued random variables X we can recover p k = Pr[X = k by inverting the Fourier series: φ(ω) = E[e iωx = p k e ikω, so p k = 1 π π π Inversion: Continuous andom Variables e ikω φ(ω) dω. Now let s turn to the case of a distribution with a density function; first two preliminaries. For any real or complex numbers a, b, c it is easy to compute (by completing the square) that π e a bx cx /4c dx = (1) c e a+b if c has positive real part, and otherwise the integral is infinite; in particular, for any ɛ > the function γ ɛ (x) 1 πɛ e x /ɛ satisfies γ ɛ (x) dx = 1 (it s just the normal probability density with mean and variance ɛ). Let µ(dx) = f(x)dx be any probability distribution with density function f(x) and ch.f. φ(ω) = ˆµ(ω) = e iωx f(x) dx. Then φ(ω) 1 so for any ɛ > the function e iyω ɛω / φ(ω) is integrable and we can compute 1 e iyω ɛω / φ(ω) dω = 1 π π = 1 π = 1 π = 1 π = 1 πɛ [ e iyω ɛω / e ixω f(x) dx dω e i(x y)ω ɛω / f(x) dx dω [ e i(x y)ω ɛω / dω f(x) dx () [ π ɛ e (x y) /ɛ e (x y) /ɛ f(x) dx = γ ɛ f(y) = γ ɛ µ(y) f(x) dx (3) (where the interchange of orders of integration in () is justified by Fubini s theorem and the calculation in (3) by equation (1)), the convolution of the normal kernel γ ɛ ( ) with f(y). This converges uniformly to f(y) as ɛ if f( ) is bounded and continuous (the most common case), pointwise to f(y )+f(y+) if f(x) has a jump discontinuity at x = y, and to infinity if µ({y}) >, i.e., if Pr[X = y >. This is the Fourier Inversion Formula for f(x): we can recover the density f(x) from its Fourier transform φ(ω) = ˆµ(ω) by f(x) = 1 π e iωx φ(ω) dω, if that integral exists, or otherwise as the 1 limit f(x) = lim ɛ π e iωx ɛω / φ(ω) dω. Page 3

4 There are several interesting connections between the density function f(x) and characteristic function φ(ω). If φ(ω) wiggles with rate approximately ξ, i.e., if φ(ω) a cos(ωξ) + b sin(ωξ) + c, then f(x) will have a spike at x = ξ and X will have a high probability of being close to ξ; if φ(ω) is very smooth (i.e., has well-behaved continuous derivatives of high order) then it does not have high-frequency wiggles and f(x) falls off quickly for large x, so E[ X p < for large p. If φ(ω) falls off quickly as ω ± then φ(ω) doesn t have large low-frequency components and f(x) must be rather tame, without any spikes. Thus φ and f both capture information about the distribution, but from different perspectives. This is often useful, for the vague descriptions of this paragraph can be made precise: Theorem 1. If ˆµ(ω) dω < then µ ɛ µ γ ɛ converges a.s. to an L 1 function f(x), ˆµ ɛ (ω) converges uniformly to ˆf(ω), and µ(a) = 1 f(x) dx. Also f(x) = A π e iωx ˆµ(ω) dω for almostevery x. Theorem. For any µ and any a < b, µ ( (a, b) ) + 1 / µ ( {a, b} ) = lim T T T e iωa e iωb πiω ˆµ(ω) dω. Theorem 3. If x k µ(dx) < for an integer k > then ˆµ(ω) has continuous derivatives of order k given by ˆµ (k) (ω) = (ix) k e iωx µ(dx) Conversely, if ˆµ(ω) has a derivative of finite even order k at ω =, then x k µ(dx) < and EX k = xk µ(dx) = ( 1) k/ ˆµ (k) (). By Theorem 3 the first few moments of the distribution, if they exist, can be determined from derivatives of the characteristic function or its logarithm at zero: φ() = 1, φ () = ie[x, φ () = E[X, so Mean: [log φ () = φ ()/φ() = ie[x = iµ Variance: [log φ () = φ ()φ() (φ ()) φ() = E[X + E[X = σ Etc.: [log φ () = O(E[ X 3 ), so by Taylor s theorem, we have log φ(ω) = + iµω σ ω / + O(ω 3 ) φ(ω) e iµω σ ω /+O(ω 3 ) Limits of Partial Sums We ll need to re-center and re-scale the distribution of S n = n i=1 X i before we can hope to make sense of S n s distribution for large n, so we ll need some facts about characteristic functions of linear combinations of independent V s: for independent X and Y, and real numbers α, β, γ, φ α+βx+γy (ω) = Ee iω(α+βx+γy ) = Ee iωα Ee iωβx Ee iωγy = e iωα φ X (ωβ) φ Y (ωγ) In particular, for i.i.d. L random variables X i with characteristic function φ(t), the normalized sum [S n nµ/ nσ has characteristic function φ S (ω) = n [ φ(ω/ nσ )e iωµ/ nσ j=1 = [ φ(s)e isµ n, where s = ω/ nσ n[log φ(s) isµ = e Page 4

5 with logarithm log φ S (ω) = n[log φ(s) isµ = n[ + iµs σ s / + O(s 3 ) nisµ = nσ (ω /nσ )/ + O(n 1/ ) = ω / + O(n 1/ ), so φ S (ω) e ω / for all ω and hence [S n nµ/ nσ Theorem. N(, 1), the Central Limit Note: We assumed X i were i.i.d. with finite third moment; the method of proof really only requires E[X <, and can be extended to the non-identically-distributed case (and even independence can be weakened), but S n cannot converge to a normally-distributed limit if E[X = ; ask for details (or read Glivenko & Kolmogorov) if you re interested. Generalized Poisson Distributions Let X j have independent Poisson distributions with means ν j and let u j ; then the ch.f. for Y u j X j is φ Y (ω) = exp [ ν j (e iωu j 1) = exp [ (e iωu j 1)ν j = exp [ (e iωu 1)ν(du) for the discrete measure ν(du) that assigns mass ν j to each point u j ; evidently we could take a limit using a sequence of discrete measures that converges to a continuous measure ν(du) so long as the integral makes sense, i.e. eiωu 1 ν(du) < ; this will follow from the requirement that (1 u )ν(du) <. Such a distribution is called Generalized Poisson; we ll now see that it includes an astonishingly large set of distributions. Distribution Poisson Po(λ) λ(e iω 1) = Gamma: Ga(α, λ) α log(1 iλω) = Log Characteristic Function (e iωu 1) λδ 1 (du) (e iωu 1) αe λu u 1 du Normal: No(, σ ) ω σ / = lim ɛ (e iωu 1) σ ɛ δ ɛ(du) iωσ ɛ Neg Bin: NB(α, p) α log[1 p q (eiω 1) = (e iωu αp k 1)ν(du), ν(du) = k δ k(du) k=1 Cauchy: Ca(a, ) a ω = (e iωu 1)ν(du), ν(du) = a u du Stable: St(α, β, γ) ω α [1 iβ tan πα sgn(ω) = (e iωu 1)ν(du), ν(du) = c α γu 1 α du, where c α = π Γ(1+α) sin πα (must adjust for β ). Try to verify the measures ν(du) for the Negative Binomial and Cauchy distributions. All these distributions share the property called Page 5

6 infinite divisibility, that each can be written as a sum of n independent identically distributed pieces for every number n; independently the French probabilist Paul Lévy and ussian probabilist A. Ya. Khinchine discovered that every distribution with this property must have a ch.f. of a very slightly more general form than that given above, log φ(ω) = iaω σ ω + [e iωu 1 iω h(u)ν(du), where h(u) is any bounded continuous function that acts like u for u close to zero (for example, h(u) = arctan(u) or h(u) = sin(u) or h(u) = u/(1 + u )). The measure ν(du) need not quite be finite, but we must have u integrable near zero and 1 integrable away from zero... one way to write this is to require that (1 u ) ν(du) <, another is to require u 1+u ν(du) <. Some authors consider the finite measure κ(du) = u 1+u ν(du) and write log φ(ω) = iωa + [e iωu 1 iω h(u) 1 + u u κ(du), where now the Gaussian component σ ω arises from a point mass for κ(du) of size σ at u =. If u is locally integrable, i.e. if ɛ u ν(du) < for some (and hence every) ɛ >, then the ɛ term iω h(u) is unnecessary (it can be absorbed into iωa). This always happens if ν( ) =, i.e. if ν is concentrated on the positive half-line. Every increasing stationary independent-increment stochastic process X t has increments which are infinitely divisible with ν concentrated on the positive half-line and no Gaussian component (σ = ), so have the representation log φ(ω) = iωa + [e iωu 1ν(du). for some a and some measure ν on + satisfying (1 u) ν(du) <. In the generalized Poisson example, ν(du) = ν j δ uj (du) was the sum of point masses of size ν j at the possi- ble jump magnitudes u j. This interpretation extends to help us understand all ID distributions: every ID random variable X may be viewed as the sum of a constant, a Gaussian random variable, and a generalized Poisson random variable, the sum of independent Poisson jumps of sizes u E with rates ν(e). Stable Limit Laws Let S n = X X n be the partial sum of iid random variables. IF the random variables are all square integrable, then the Central Limit Theorem applies and necessarily S n nσ µ = No(, 1). But what if each X n is not square integrable? Denote by F (x) = P[X n x the common CDF of the {X n }. Theorem (Stable Limit Law). There exist constants A n > and B n and a distribution µ for which the S n A n B n = µ if and only if there are constants < α, M, and M +, with M + M + >, such that as x the following limits hold for every ξ > : F ( x) 1. 1 F (x) M M + ; Page 6

7 . M + > 1 F (xξ) 1 F (x) ξ α M > F ( xξ) F ( x) ξ α. In this case the limit is the Stable Distribution with index α, with characteristic function E[e iωy = e iδω γ ω α [1 iβ tan πα sgn(ω), M where β = + and γ = (M + M + ). Note that the Cauchy distribution is the special M +M + case of (α, β, γ, δ) = (1,, 1, ) and the Normal distribution is the special case of (α, β, γ, δ) = (,, σ /, µ). Although each Stable distribution has an absolutely continuous distribution with continuous probability density function f(y), these two cases and the inverse Gaussian distribution with α = 1/ and β = ±1 are the only ones where the p.d.f. can be given in closed form. Moments are easy enough to compute; for α < the Stable distribution only has finite moments of order p < α and, in particular, none of them has a finite variance. The Cauchy has finite moments of order p < 1 but does not have a well-defined mean. Condition. says that each tail must be fall off like a power (sometimes called Pareto tails), and the powers must be identical; Condition 1. gives the tail ratio. In a common special case, M = ; for example, random variables X n with the Pareto distribution (often used to model income) given by P [X n > t = (k/t) α for t k will have a stable limit for their partial sums if α <, and (by CLT) a normal limit if α. You can find out more details reading Chapter 9 of Breiman s book. Page 7