Chapter 6. Characteristic functions

Transcription

1 Chapter 6 Characteristic functions We define the characteristic function of a random variable X by ' X (t) E e x for t R. Note that ' X (t) R e x P X (dx). So if X and Y have the same law, they have the same characteristic function. Also, if the law of X has a densy, that is, P X (dx) f X (x) dx, then' X (t) R e x f X (x) dx, sointhiscasethe characteristic function is the same as (one definion of) the Fourier transform of f X. Proposion 6. '(), '(t) apple, '( continuous. t) '(t), and ' is uniformly Proof. Since e x apple, everything follows immediately from the definions except the uniform continuy. For that we wre '(t + h) '(t) E e i(t+h)x E e X applee e X (e ihx ) E e ihx. e ihx tends to almost surely as h!, so the right hand side tends to by dominated convergence. Note that the right hand side is independent of t. Proposion 6. ' ax (t) ' X (at) and ' X+b (t) e b ' X (t), 47

2 48 CHAPTER 6. CHARACTERISTIC FUNCTIONS Proof. The first follows from E e (ax) E e i(at)x,andthesecondissimilar. Proposion 6.3 If X and Y are independent, then ' X+Y (t) ' X (t)' Y (t). Proof. From the multiplication theorem, E e (X+Y ) E e X e Y E e X E e Y. Note that if X and X are independent and identically distributed, then ' X X (t) ' X (t)' X (t) ' X (t)' X ( t) ' X (t)' X (t) ' X (t). Let us look at some examples of characteristic functions. (a) Bernoulli: Bydirectcomputation,thisispe +( p) p( e ). (b) Coin flip: (i.e., P(X +)P(X ) /) We have e + e cost. (c) Poisson: E e X X e k e k k k! e X ( e ) k k! e e e e (e ). (d) Point mass at a: E e X e a. Note that when a,then'. (e) Binomial: WreX as the sum of n independent Bernoulli r.v.s B i.so ' X (t) ny ' Bi (t) [' Bi (t)] n [ p( e )] n. i (f) Geometric: '(t) X p( p) k e k p X (( p)e ) k k p ( p)e.

3 6.. INVERSION FORMULA 49 (g) Uniform on [a, b]: '(t) b a b a e a e x dx eb (b a). Note that when a (h) Exponential: b this reduces to sin(bt)/bt. e x e x dx e ( )x dx. (i) Standard normal: '(t) p e x e x / dx. This can be done by completing the square and then doing a contour integration. Alternately, ' (t) (/ p ) R ixex e x / dx. (do the real and imaginary parts separately, and use the dominated convergence theorem to justify taking the derivative inside.) Integrating by parts (do the real and imaginary parts separately), this is equal to t'(t). The only solution to ' (t) t'(t) wh'() is '(t) e t /. (j) Normal wh mean µ and variance :WringX + µ, where is a standard normal, then ' X (t) e iµt ' ( t) e iµt t /. (k) Cauchy: Wehave '(t) e x +x dx. This is a standard exercise in contour integration in complex analysis. The answer is e t. 6. Inversion formula We need a preliminary real variable lemma, and then we can proceed to the inversion formula, which gives a formula for the distribution function in terms of the characteristic function.

4 5 CHAPTER 6. CHARACTERISTIC FUNCTIONS Lemma 6.4 (a) R N (sin(ax)/x) dx! sgn (A) / as N!. (b) sup a R a (sin(ax)/x) dx <. Proof. If A,thisisclear. ThecaseA<reducestothecaseA> by the fact that sin is an odd function. By a change of variables y Ax, we reduce to the case A. Part(a)isastandardresultincontourintegration, and part (b) comes from the fact that the integral can be wrten as an alternating series. An alternate proof of (a) is the following. {(x, y); < x < a, <y<}. So a sin x x dx a a e xy sin xdydx e xy sin xdxdy h e xy i a y + ( y sin x cos x) dy h e ay o y + ( y sin a cos a) sin a ye ay y + dy e xy sin x is integrable on cos a i dy y + e ay y + dy. The last two integrals tend to as a!since the integrand is bounded by ( + y)e y if a. Theorem 6.5 (Inversion formula) Let µ be a probabily measure and let '(t) R e x µ(dx). Ifa<b, then lim T! T T e a e b '(t) dt µ(a, b)+ µ({a})+ µ({b}). The example where µ is point mass at, so '(t),showsthatone needs to take a lim, since the integrand in this case is sin t/t, whichisnot integrable.

5 6.. INVERSION FORMULA 5 Proof. By Fubini, T T e a e b '(t) dt T T T e a T e a e b e b e x µ(dx) dt e x dt µ(dx). To justify this, we bound the integrand by the mean value theorem Expanding e b and e a using Euler s formula, and using the fact that cos is an even function and sin is odd, we are left wh h T sin(t(x t a)) dt T sin(t(x t Using Lemma 6.4 and dominated convergence, this tends to [ sgn (x a) sgn (x b)]µ(dx). b)) i dt µ(dx). Theorem 6.6 If R '(t) dt <, then µ has a bounded densy f and f(y) e y '(t) dt. Proof. µ(a, b)+ µ({a})+ µ({b}) lim T! apple b a T T e a e a '(t) dt. e b Letting b! a shows that µ has no point masses. e b '(t) dt '(t) dt

6 5 CHAPTER 6. CHARACTERISTIC FUNCTIONS We now wre µ(x, x + h) e x e (x+h) '(t)dt x+h e y dy '(t) dt x x+h e y '(y) dt dy. x So µ has densy (/ ) R e y '(t) dt. As in the proof of Proposion 6., we see f is continuous. Acorollarytotheinversionformulaistheuniquenesstheorem. Theorem 6.7 If ' X ' Y, then P X P Y. Proof. Let a and b be such that P(X a) P(X b),andthesame when X is replaced by Y.Wehave F X (b) F X (a) P(a <Xapple b) P X ((a, b]) P X ((a, b)), and the same when X is replaced by Y.Bytheinversiontheorem,P X ((a, b)) P Y ((a, b)). Now let a! along a sequence of points that are continuy points for F X and F Y. We obtain F X (b) F Y (b) ifb is a continuy point for X and Y. Since F X and F Y are right continuous, F X (x) F Y (x) forall x. The following proposion can be proved directly, but the proof using characteristic functions is much easier. Proposion 6.8 (a) If X and Y are independent, X is a normal wh mean a and variance b, and Y is a normal wh mean c and variance d, then X+Y is normal wh mean a + c and variance b + d. (b) If X and Y are independent, X is Poisson wh parameter, and Y is Poisson wh parameter, then X + Y is Poisson wh parameter +. (c) If X i are i.i.d. Cauchy, then S n /n is Cauchy.

7 6.. CONTINUITY THEOREM 53 Proof. For (a), ' X+Y (t) ' X (t)' Y (t) e iat b t / e ict c t / e i(a+c)t (b +d )t /. Now use the uniqueness theorem. Parts (b) and (c) are proved similarly. 6. Continuy theorem Lemma 6.9 Suppose ' is the characteristic function of a probabily µ. Then Proof. Note T µ([ A, A]) A T T /A /A '(t) dt. '(t) dt T e x µ(dx) dt T T T [ T,T](t)e x dt µ(dx) sin Tx Tx µ(dx). Since sin(tx) apple, then (sin(tx))/t x apple /TA if x A. Since (sin(tx))/t x apple, we then have sin Tx µ(dx) apple µ([ A, A]) + Tx [ A,A] TA µ(dx) c µ([ A, A]) + ( µ([ A, A]) TA TA + µ([ A, A]). TA Setting T /A, A /A /A Now multiply both sides by. '(t) dt apple + µ([ A, A]).

8 54 CHAPTER 6. CHARACTERISTIC FUNCTIONS Proposion 6. If µ n converges weakly to µ, then ' n converges to ' uniformly on every fine interval. Proof. Let ">andchoosem large so that µ([ M,M] c ) <". Define f to R be on [ M,M], on [ M,M +] c, and linear in between. Since fdµn! R fdµ,thenifnis large enough, ( f) dµ n apple ". We have ' n (t + h) ' n (t) apple e ihx µ n (dx) apple ( f) dµ n + h apple " + h(m +). x f(x) µ n (dx) So for n large enough and h apple"/(m +),wehave ' n (t + h) ' n (t) apple3", which implies that the ' n are equicontinuous. Therefore the convergence is uniform on fine intervals. The interesting result of this section is the converse, Lévy s continuy theorem. Theorem 6. Suppose µ n are probabilies, ' n (t) converges to a function '(t) for each t, and ' is continuous at. Then ' is the characteristic function of a probabily µ and µ n converges weakly to µ. Proof. Let ">. Since ' is continuous at, choose '(t) dt <". small so that Using the dominated convergence theorem, choose N such that ' n (t) '(t) dt < "

9 6.. CONTINUITY THEOREM 55 if n N. Soifn N, ' n (t) dt ". '(t) dt ' n (t) '(t) dt By Lemma 6.9 wh A /,forsuchn, This shows the µ n are tight. µ n [ /, / ] ( ") 4". Let n j be a subsequence such that µ nj converges weakly, say to µ. Then ' nj (t)! ' µ (t), hence '(t) ' µ (t), or ' is the characteristic function of aprobabilyµ. If µ is any subsequential weak lim point of µ n, then ' µ (t) '(t) ' µ (t); so µ must equal µ. Hence µ n converges weakly to µ. We need the following estimate on moments. Proposion 6. If E X k < for an integer k, then ' X has a continuous derivative of order k and ' (k) (t) (ix) k e x P X (dx). In particular, ' (k) () i k E X k. Proof. Wre '(t + h) h '(t) e i(t+h)x h e x P(dx). The integrand is bounded by x. Soif R x P X (dx) <, wecanusedominated convergence to obtain the desired formula for ' (t). As in the proof of Proposion 6., we see ' (t) is continuous. We do the case of general k by induction. Evaluating ' (k) at gives the particular case. Here is a converse.

10 56 CHAPTER 6. CHARACTERISTIC FUNCTIONS Proposion 6.3 If ' is the characteristic function of a random variable X and ' () exists, then E X <. Proof. Note and ( e ihx +e ihx cos hx apple h h cos hx)/h converges to x as h!. So by Fatou s lemma, cos hx x P X (dx) apple liminf P h! h X (dx) '(h) '() + '( h) lim sup ' () <. h! h One nice application of the continuy theorem is a proof of the weak law of large numbers. Its proof is very similar to the proof of the central lim theorem, which we give in the next section. Another nice use of characteristic functions and martingales is the following. Proposion 6.4 Suppose X i is a sequence of independent r.v.s and S n converges weakly. Then S n converges almost surely. Proof. We first rule out the possibily that S n!wh posive probabily. If this happens wh posive probabily ", givenm there exists N depending on M such that if n N, thenp( S n >M) >".Thenthelim law of P Sn,say,P,willhaveP ([ M,M] c ) " for all M, acontradiction. Let N be the set of! for which S n (!)!. Suppose S n converges weakly to W. Then ' Sn (t)! ' W (t) uniformly on compact sets by Proposion 6.. Since ' W () and ' W is continuous, there exists such that ' W (t) < / if t <.Sofornlarge, ' Sn (t) /4 if t <. Note E he Sn X,...X n i e S n E [e Xn X,...,X n ]e S n ' Xn (t).

11 6.. CONTINUITY THEOREM 57 Since ' Sn (t) Q ' Xi (t), follows that e Sn /' Sn (t) isamartingale. Therefore for t < and n large, e Sn /' Sn (t) isaboundedmartingale, and hence converges almost surely. Since ' Sn (t)! ' W (t) 6,thene Sn converges almost surely if t <. Let A {(!, t) (, ) : e Sn(!) does not converge}. For each t, we have almost sure convergence, so R A (!, t) P(d!). Therefore R R A dp dt,andbyfubini, RR A dt dp. Hence almost surely, R A (!, t) dt. Thismeans,thereexistsasetN wh P(N ),andif!/ N,thene Sn(!) converges for almost every t (, ). Call the lim, when exists, L(t). Fix!/ N [ N. Suppose one subsequence of S n (!) convergestoq and another subsequence to R. Then L(a) e iar e iaq for a.e. a small, which implies that R Q. Suppose one subsequence converges to R 6 andanotherto ±. By integrating e Sn and using dominated convergence, we have a e Sn dt eiasn is n converges. We then obtain e iar ir for a.e. a small, which implies R. Finally suppose one subsequence converges to and another to ±. Since (e iasn j )/is nj! a, weobtaina fora.e.a small, which is impossible. We conclude S n must converge.