:23 /4/2000 TOPIC Characteristic functions This lecture begins our study of the characteristic function φ X (t) := Ee itx = E cos(tx)+ie sin(tx) (t R) of a real random variable X Characteristic functions are of interest for a variety of reasons, among them: φ X uniquely determines the distribution of X; random variables X n converge in distribution to X iff φ Xn (t) φ X (t) for each t R; and the characteristic function of a sum of independent random variables is the product of the characteristic functions of the summands These properties make characteristic functions an ideal tool for proving limit theorems for sums of independent random variables The integral of a complex-valued function Let C := {u + iv : u, v R} (i = ) be the complex plane Let be a sample space A mapping Z = U + iv from to C is said to be a complex-valued random variable on if the mappings U and V are real-valued random variables Z is said to be integrable with respect to a measure µ on, written Z L(µ), if U and V are each integrable with respect to µ The integral of Z L(µ) is Z dµ := U dµ + i V dµ () Here, eg, U dµ would be or when u(x)f(x) dx n= u(n)f(n) µ has density f with respect to Lebesgue measure on = R, when µ places mass f(n) at each point n = N, 2 Z = U + iv is integrable U and V are integrable or E ( U(X) ) In general, since when µ is the distribution of a -valued random variable X max( U, V ) Z := U 2 + V 2 U + V, it follows that Z is integrable Z is integrable (2) Generating functions Let X be a real-valued random variable defined on a sample space and let P be a probability measure on Consider Z := e zx for z := u + iv in C We ask: When is Z integrable with respect to P? Since e zx = e ux e ivx = e ux( cos(vx) + i sin(vx) ), we have Z = e ux e ivx = e ux = e ux and thus E( Z ) < M X (u) := E(e ux ) < (3) Here M X is the (real) moment generating function of X; it was studied in the previous section M X maps R into [0, ] By Theorem 2, B X := { u R : M X (u) < } (4) is a (possibly degenerate) interval containing 0 According to (3), 2 2
0 B X := { u R : M X (u) := E(e ux ) < } Z = e zx is integrable if and only if z B X, where BX := { z C : R(z) B X } (5) The function G X : BX C defined by BX 0 B X G X (z) = E(e zx ) (6) is called the complex moment generating function of X, and the function φ X : R C defined by φ X (t) := G X (it) = Ee itx = E cos(tx) + ie sin(tx) (7) is called the characteristic function of X We are primarily interested in characteristic functions; the real and complex generating functions will serve as convenient means to obtain the characteristic function in the case where B X is nondegenerate Example (a) Suppose X N(0, ) By Example 2 (a), M X (u) = e u2 /2 for all u R and B X = R We see later on (see Example 4 (a)) that (8) implies that G X (z) = e z2 /2 for z BX = C, and thus that φ X (t) = e t2 /2 for t R (b) Suppose next that X G(r, ) By Example 2 (b), { /( u) r, if < u < M X (u) =, if u < and B X = (, ) We will see later on (see Example 4 (b)) that (9) implies that G X (z) = /( z) r for z BX = { z C : R(z) < }, and thus that φ X (t) = /( it) r for t R 2 3 (8) (9) Properties of the integral Return now to the general complexvalued random variable Z = U + iv on a sample space, and recall that the integral of Z with respect to a measure µ on is defined as Z dµ = U dµ + i V dµ (0) provided Z is integrable The following theorem presents the basic properties of Z dµ Theorem The complex integral in (0) has the following properties (I) is linear: if Z and Z 2 are complex-valued integrable random variables and c and c 2 are complex numbers, then the complexvalued random variable c Z + c 2 Z 2 is integrable and (c Z + c 2 Z 2 ) dµ = c Z dµ + c 2 Z 2 dµ () (I2) decreases absolute values: if Z is an integrable complexvalued random variable, then Z dµ Z dµ (2) (I3) the Dominated Convergence Theorem (DCT) holds for : if Z and Z, Z 2, are complex-valued random variables such that (i) Z n tends to Z pointwise (or just µ-almosteverywhere) on as n and (ii) there exists an integrable real-valued random variable D on such that Z n D for all n, then each Z n is integrable, Z is integrable, and Z n dµ Z dµ as n (3) Proof Properties (I) and (I3) follow easily from the corresponding properties of the integral for real-valued random variables For (I2), 2 4
:23 /4/2000 write Z as U + iv If U and V each take only finitely many values, then they can be put in the form U = k u ki Ak and V = k v ki Ak for finitely many disjoint sets A k In this case Z = k (u k + iv k )I Ak and Z = k u k + iv k I Ak, whence Z dµ = (u k + iv k )µ(a k ) k u k + iv k µ(a k ) = Z dµ k by the triangle inequality in C For the general case, one writes U = lim n U n and V = lim n V n as pointwise limits of finitely-valued functions U n and V n, each of which is dominated by Z (see Exercise 3) Then Z n := U n +iv n converges pointwise to Z and Z n 2 Z for all n Thus Z dµ = lim Z n dµ lim Z n dµ = Z dµ n n by two applications of the DCT (the first for complex-valued random variables, the second for real-valued ones) Example 2 Let φ(t) = E(e itx ) be the characteristic function of a real-valued random variable X By (I2) φ(t) = E(e itx ) E( e itx ) = E = ; (4) thus φ is bounded by Moreover φ is continuous (5) To see this, suppose t n t R Then Z n := e itnx Z := e itx pointwise on, and Z n D := for all n Since E(D) = <, the complex DCT implies that φ(t n ) = E(Z n ) E(Z) = φ(t) Fubini s Theorem also holds for complex-valued random variables The following result is a special case 2 5 Theorem 2 Let Z and Z, Z 2, be complex-valued random variables on a sample space equipped with a measure µ Suppose that Z = ( ) Z n and Z n dµ < (6) Then each Z n is integrable, Z is integrable, and Z dµ = ( ) Z n dµ (7) Proof Each Z n is integrable because ( ) ( ) Z n dµ = Z n dµ < Put S = Z and S n = n k=0 Z k for n = 0,, Then S n converges pointwise to S as n tends to, and S n D := k=0 Z k for all n Since D is integrable, the complex DCT implies that each S n is integrable, S = Z is integrable, and S n dµ S dµ (7) follows since S n dµ = n k=0 Z k dµ Analyticity of the complex generating function Suppose X is a random variable such that the region B X where M X is finite contains a nonempty open interval Let B X be the largest such interval; B X is all of B X except possibly for the endpoints Put D X = {u + iv C : u B X }; (8) D X is the largest open subset of C contained in BX We are going to show that the complex generating function G X has a power series expansion about each point of D X In the language of complex variables, this says that G X is analytic on D X For complex numbers z and real numbers r > 0 let B(z, r) := { ζ C : ζ z < r } be the open ball about z of radius r The following result generalizes Theorems 22, 23, and part of 25 2 6
Theorem 3 Let X be real random variable such that the interior D X of the domain B X of G X is nonempty (i) Let z D X and set ρ = sup{ r > 0 : B(z, r) D X } Then X n e zx is integrable for n = 0,, 2,, and G X (ζ) = E(Xn e zx ) (ζ z)n D X ρ z ( ) B X (9) for each ζ D X such that ζ z < ρ (ii) G X is infinitely differentiable in D X and G (n) X (z) = E(Xn e zx ) for all z D X and n N (20) Proof (i) We have G X (ζ) = E(e ζx ) with and e ζx = e zx e (ζ z)x = ezx Xn (ζ z) n e zx Xn (ζ z) n e zx = e ux e r X e (u r)x + e (u+r)x := D, X(ζ z) n where u = R(z) and r = ζ z D is integrable because u r and u + r each lie in B X (9) now follows from Theorem 2 (ii) (20) follows by differentiating both sides of (9) Example 3 Suppose X N(0, ) We have seen that E(X n ) = 0 if n is odd, whereas E(X 2k ) = (2k)!/(2 k k!) for k = 0,, Since D X = C, it follows from (9) with z = 0 that G X (ζ) = for all ζ C k=0 (2k)! 2 k k! ζ 2k (2k)! = 2 7 k=0 (ζ 2 /2) k k! = e ζ2 /2 (2) The substitution principle The formula G X (z) = E(Xn ) zn (22) can be used to compute the complex generating function if B X = R, as in the preceding example, but not otherwise We describe an approach that works whenever B X has nonempty interior The technique is based on the following uniqueness theorem from complex variables Theorem 4 Let D be an open connected subset of C and let f and g be differentiable complex-valued functions defined on D Then f(z) = g(z) for all z D provided f(z) = g(z) for all z in some subset of D having a limit point in D By definition, z is a limit point of if there exists an infinite sequence of points z n such that z n z as n, and, moreover, z n z for all n Theorem 5 Let X be a real random variable such that B X has nonempty interior Suppose that H is a differentiable complex-valued function on D X such that H(u) = M X (u) for all u in some subset of BX having a limit point in B X (in particular, for all u in B X ) Then G X (z) = H(z) for all z D X Proof This follows from Theorem 4 by taking D = D X, f(z) = H(z), and g(z) = G X (z) This is called the substitution principle, because it states, in effect, that you can get G X (z) by substituting z for u in M X (u) Of course, it is necessary to check that the resulting function of z is differentiable The formulas d dz zn = nz n d, dz ez = e z, d dz f(g(z)) = f (g(z))g (z) are helpful in this regard 2 8 d dz log(z) = z,
:23 /4/2000 Example 4 (a) Suppose X N(0, ) X has real moment generating function M X (u) = e u2 /2 and D X = C The function H: D X C defined by H(z) = e z2 /2 is differentiable (the derivative is ze z2 /2 ) and coincides with M X on BX = R According to the substitution principle, G X (z) = e z2 /2 for all z C (b) Suppose X G(r, ) The real moment generating function M X of X is finite in B X = (, ), and has there the value M X (u) = /( u) r = e r log( u) Let H mapping D X = { z C : R(z) < } to C be defined by H(z) = e r log( z) Here ζ := z = ρe iθ with ρ > 0 and π/2 < θ < π/2, and log(ζ) = log(ρ) + iθ is the so-called principal value of the logarithm of ζ H is differentiable on D X (the derivative is rh(z)/( z)) and agrees with M X on BX = B X According to the substitution principle, G X (z) = e r log( z) = /( z) r for all z D X The substitution principle can be used (verify this!) to obtain all the entries in the table on the next page except for lines 9 and 2 The exercises in this section ask you to derive the entries on lines 5, 8, 0, 3, and 4 The remaining formulas will be obtained in the next section Table Characteristic Functions of Some Distributions Distributions 5 (respectively, 6 4) have the indicated density with respect to counting measure (respectively, Lebesgue measure) on the indicated support On lines 3 and 5, q = p Distribution Density Support Cf Degenerate {} e it 2 Symmetric /2 {, } cos(t) Bernoulli n 3 Binomial p x q n x {0,,, n} (pe it + q) n x 4 Poisson e λ λ x 5 Negative binomial 6 Normal 7 Gamma 8 9 0 Two-sided exponential Symmetric Cauchy Symmetric uniform Triangular 2 3 Inverse triangular Double exponential 4 Logistic r x ( q) x p r {0,, } x! e (x µ)2 /(2σ 2 ) σ 2π r x r e x Γ(r) {0,, } exp(λ(e it )) r p qe it (, ) exp(itµ σ 2 t 2 /2) (0, ) e x /2 (, ) π( 2 + x 2 ) 2 x (, ) [, ] [, ] cos(x) πx 2 (, ) ( it/) r + t 2 / 2 e t sin(t) t 2( cos(t)) 2 t 2 t e e x e x (, ) Γ( it) + e x ( + e x ) 2 (, ) Γ( + it)γ( it) 2 9 2 0
Exercise Evaluate the integral cos(tx) e x2 /2 dx without calculation ; here t R Exercise 2 Theorem Write out complete proofs of properties I and I3 of Exercise 3 Let U be a real random variable Show that there exists an infinite sequence U, U 2, of real random variables such that: (i) lim n U n (ω) = U(ω) for all sample points ω and (ii) U n (ω) U(ω) for n N and all sample points ω [Hint: If 0 U(ω) n, let U n (ω) be the result of rounding U(ω) down to the nearest multiple of /2 n Exercise 4 Prove the following version of Fubini s theorem Let f = u+iv be a complex-valued mapping from R 2 to C; here u(x, y) = R ( f(x, y) ) and v(x, y) = I ( f(x, y) ) for real numbers x and y Show that if f(x, y) dx dy <, then [ f(x, y) dx dy == [ = ] f(x, y) dy dx ] f(x, y) dx dy; (23) here the first integral is a double integral, and the next two integrals are iterated ones Exercise 5 Suppose that Z is a discrete complex-valued random variable, taking the values z, z 2, with probabilities p, p 2, Show that Z is integrable iff k z k p k <, in which case EZ = k p n kz k := lim n k= p kz k Use this formula to verify the characteristic functions on lines 5 of Table Exercise 6 (The fundamental theorem of calculus) Suppose that Z is a continuously differentiable complex-valued function defined on a 2 subinterval [a, b] of R Show that Z(b) Z(a) = b a Z (t) dt (24) [Hint: apply the FTC for real-valued functions to the real and imaginary parts of Z] Use this formula to verify the characteristic function on line 0 in Table Exercise 7 Deduce the results on lines 5 and 0 in Table from the substitution principle Exercise 8 Use the substitution principle to deduce the result on line 8 in Table Exercise 9 (On the complex gamma function) Suppose z C with R(z) > 0 Show that the function f(z, x) := x z e x is integrable in x with respect to dx on (0, ), and thus that Γ(z) := 0 is defined Show further that Γ (k) (z) := dk dz k Γ(z) = x z e x dx (25) 0 x z (log(x)) k e x dx (26) for all z C with R(z) > 0 and k =, 2, (The case k = and z real was used in Exericse 88) [Hint: For (26) you can use the DCT and induction, or mimic the argument used in Theorem 3 to establish the analyticity of complex moment generating functions However, it is easiest to make use of that result, by showing that for R(z) > 0 one has Γ(z) = G Y (z ), where Y = log(x) for a standard exponential random variable X and G Y is the complex generating function of Y ] 2 2
:23 /4/2000 Exercise 0 Deduce the last two lines in Table [Hint: this can be done using the substitution principle and the differentiability of the complex Gamma function; there are other other ways to do it as well] Theorem 5 can be used to obtain the characteristic function of a random variable X from its MGF when 0 is in the interior of B X The case that 0 B X is covered by the following exercise Exercise Let X be a random variable such that B X has nonempty interior BX, but 0 / B X The imaginary axis I := { iv : v R } of C is thus a (vertical) edge of region BX in which the complex generating function G X is defined (a) Show that G X is continuous at each point z I and differentiable at each point z in the interior D X of BX (b) Suppose that H: B X C is continuous at each point of I and differentiable at each point of D X, and that G X (z) = H(z) for each point z in a subset of D X having a limit point in D X Show that G X (z) = H(z) for all z BX, and deduce that φ X (t) = H(it) for all t R Exercise 2 Let Y be a random variable with a chisquare distribution with degree of freedom Put X = /Y Using a martingale argument, one can show that E(e sx ) = exp ( 2s ) (27) for s 0 Use the result of the preceding exercise to show that X has characteristic function φ X (t) = exp ( 2 t ( i) ) (28) 2 3