18.175: Lecture 15 Characteristic functions and central limit theorem

Size: px

Start display at page:

Download "18.175: Lecture 15 Characteristic functions and central limit theorem"

Curtis Gaines
5 years ago
Views:

1 18.175: Lecture 15 Characteristic functions and central limit theorem Scott Sheffield MIT

2 Outline Characteristic functions

3 Outline Characteristic functions

4 Characteristic functions Let X be a random variable.

5 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ].

6 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ]. Recall that by definition e it = cos(t) + i sin(t).

7 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ]. Recall that by definition e it = cos(t) + i sin(t). Characteristic function φ X similar to moment generating function M X.

8 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ]. Recall that by definition e it = cos(t) + i sin(t). Characteristic function φ X similar to moment generating function M X. φ X +Y = φ X φ Y, just as M X +Y = M X M Y, if X and Y are independent.

9 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ]. Recall that by definition e it = cos(t) + i sin(t). Characteristic function φ X similar to moment generating function M X. φ X +Y = φ X φ Y, just as M X +Y = M X M Y, if X and Y are independent. And φ ax (t) = φ X (at) just as M ax (t) = M X (at).

10 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ]. Recall that by definition e it = cos(t) + i sin(t). Characteristic function φ X similar to moment generating function M X. φ X +Y = φ X φ Y, just as M X +Y = M X M Y, if X and Y are independent. And φ ax (t) = φ X (at) just as M ax (t) = M X (at). And if X has an mth moment then E[X m ] = i m φ (m) X (0).

11 Characteristic functions Let X be a random variable. The characteristic function of X is defined by φ(t) = φ X (t) := E[e itx ]. Recall that by definition e it = cos(t) + i sin(t). Characteristic function φ X similar to moment generating function M X. φ X +Y = φ X φ Y, just as M X +Y = M X M Y, if X and Y are independent. And φ ax (t) = φ X (at) just as M ax (t) = M X (at). And if X has an mth moment then E[X m ] = i m φ (m) X (0). Characteristic functions are well defined at all t for all random variables X.

12 Characteristic function properties φ(0) = 1

13 Characteristic function properties φ(0) = 1 φ( t) = φ(t)

14 Characteristic function properties φ(0) = 1 φ( t) = φ(t) φ(t) = Ee itx E e itx = 1.

15 Characteristic function properties φ(0) = 1 φ( t) = φ(t) φ(t) = Ee itx E e itx = 1. φ(t + h) φ(t) E e ihx 1, so φ(t) uniformly continuous on (, )

16 Characteristic function properties φ(0) = 1 φ( t) = φ(t) φ(t) = Ee itx E e itx = 1. φ(t + h) φ(t) E e ihx 1, so φ(t) uniformly continuous on (, ) Ee it(ax +b) = e itb φ(at)

17 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t.

18 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer?

19 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer? Poisson: If X is Poisson with parameter λ then φ X (t) = k=0 e λ λk e itk k! = exp(λ(e it 1)).

20 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer? Poisson: If X is Poisson with parameter λ then φ X (t) = k=0 e λ λk e itk k! = exp(λ(e it 1)). Why does doubling λ amount to squaring φ X?

21 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer? Poisson: If X is Poisson with parameter λ then φ X (t) = k=0 e λ λk e itk k! = exp(λ(e it 1)). Why does doubling λ amount to squaring φ X? Normal: If X is standard normal, then φ X (t) = e t2 /2.

22 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer? Poisson: If X is Poisson with parameter λ then φ X (t) = k=0 e λ λk e itk k! = exp(λ(e it 1)). Why does doubling λ amount to squaring φ X? Normal: If X is standard normal, then φ X (t) = e t2 /2. Is φ X always real when the law of X is symmetric about zero?

23 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer? Poisson: If X is Poisson with parameter λ then φ X (t) = k=0 e λ λk e itk k! = exp(λ(e it 1)). Why does doubling λ amount to squaring φ X? Normal: If X is standard normal, then φ X (t) = e t2 /2. Is φ X always real when the law of X is symmetric about zero? Exponential: If X is standard exponential (density e x on (0, )) then φ X (t) = 1/(1 it).

24 Characteristic function examples Coin: If P(X = 1) = P(X = 1) = 1/2 then φ X (t) = (e it + e it )/2 = cos t. That s periodic. Do we always have periodicity if X is a random integer? Poisson: If X is Poisson with parameter λ then φ X (t) = k=0 e λ λk e itk k! = exp(λ(e it 1)). Why does doubling λ amount to squaring φ X? Normal: If X is standard normal, then φ X (t) = e t2 /2. Is φ X always real when the law of X is symmetric about zero? Exponential: If X is standard exponential (density e x on (0, )) then φ X (t) = 1/(1 it). Bilateral exponential: if f X (t) = e x /2 on R then φ X (t) = 1/(1 + t 2 ). Use linearity of f X φ X.

25 Fourier inversion formula If f : R C is in L 1, write ˆf (t) := f (x)e itx dx.

26 Fourier inversion formula If f : R C is in L 1, write ˆf (t) := f (x)e itx dx. Fourier inversion: If f is nice: f (x) = 1 2π ˆf (t)e itx dt.

27 Fourier inversion formula If f : R C is in L 1, write ˆf (t) := f (x)e itx dx. Fourier inversion: If f is nice: f (x) = 1 2π ˆf (t)e itx dt. Easy to check this when f is density function of a Gaussian. Use linearity of f ˆf to extend to linear combinations of Gaussians, or to convolutions with Gaussians.

28 Fourier inversion formula If f : R C is in L 1, write ˆf (t) := f (x)e itx dx. Fourier inversion: If f is nice: f (x) = 1 2π ˆf (t)e itx dt. Easy to check this when f is density function of a Gaussian. Use linearity of f ˆf to extend to linear combinations of Gaussians, or to convolutions with Gaussians. Show f ˆf is an isometry of Schwartz space (endowed with L 2 norm). Extend definition to L 2 completion.

29 Fourier inversion formula If f : R C is in L 1, write ˆf (t) := f (x)e itx dx. Fourier inversion: If f is nice: f (x) = 1 2π ˆf (t)e itx dt. Easy to check this when f is density function of a Gaussian. Use linearity of f ˆf to extend to linear combinations of Gaussians, or to convolutions with Gaussians. Show f ˆf is an isometry of Schwartz space (endowed with L 2 norm). Extend definition to L 2 completion. Convolution theorem: If then h(x) = (f g)(x) = ĥ(t) = ˆf (t)ĝ(t). f (y)g(x y)dy,

30 Fourier inversion formula If f : R C is in L 1, write ˆf (t) := f (x)e itx dx. Fourier inversion: If f is nice: f (x) = 1 2π ˆf (t)e itx dt. Easy to check this when f is density function of a Gaussian. Use linearity of f ˆf to extend to linear combinations of Gaussians, or to convolutions with Gaussians. Show f ˆf is an isometry of Schwartz space (endowed with L 2 norm). Extend definition to L 2 completion. Convolution theorem: If then h(x) = (f g)(x) = ĥ(t) = ˆf (t)ĝ(t). f (y)g(x y)dy, Possible application? 1 [a,b] (x)f (x)dx = (1 [a,b] f )(0)=(ˆf 1 [a,b] )(0)= ˆf (t) 1 [a,b] ( t)dx.

31 Characteristic function inversion formula If the map µ X φ X is linear, is the map φ µ[a, b] (for some fixed [a, b]) a linear map? How do we recover µ[a, b] from φ?

32 Characteristic function inversion formula If the map µ X φ X is linear, is the map φ µ[a, b] (for some fixed [a, b]) a linear map? How do we recover µ[a, b] from φ? Say φ(t) = e itx µ(x).

33 Characteristic function inversion formula If the map µ X φ X is linear, is the map φ µ[a, b] (for some fixed [a, b]) a linear map? How do we recover µ[a, b] from φ? Say φ(t) = e itx µ(x). Inversion theorem: lim T (2π) 1 T T e ita e itb it φ(t)dt = µ(a, b) + 1 µ({a, b}) 2

34 Characteristic function inversion formula If the map µ X φ X is linear, is the map φ µ[a, b] (for some fixed [a, b]) a linear map? How do we recover µ[a, b] from φ? Say φ(t) = e itx µ(x). Inversion theorem: lim T (2π) 1 T T e ita e itb Main ideas of proof: Write e ita e itb I T = φ(t)dt = it it φ(t)dt = µ(a, b) + 1 µ({a, b}) 2 T T e ita e itb e itx µ(x)dt. it

35 Characteristic function inversion formula If the map µ X φ X is linear, is the map φ µ[a, b] (for some fixed [a, b]) a linear map? How do we recover µ[a, b] from φ? Say φ(t) = e itx µ(x). Inversion theorem: lim T (2π) 1 T T e ita e itb Main ideas of proof: Write e ita e itb I T = φ(t)dt = it it φ(t)dt = µ(a, b) + 1 µ({a, b}) 2 T T e ita e itb e itx µ(x)dt. it Observe that e ita e itb by b a. it = b a e ity dy has modulus bounded

36 Characteristic function inversion formula If the map µ X φ X is linear, is the map φ µ[a, b] (for some fixed [a, b]) a linear map? How do we recover µ[a, b] from φ? Say φ(t) = e itx µ(x). Inversion theorem: lim T (2π) 1 T T e ita e itb Main ideas of proof: Write e ita e itb I T = φ(t)dt = it Observe that e ita e itb by b a. it it φ(t)dt = µ(a, b) + 1 µ({a, b}) 2 T T e ita e itb e itx µ(x)dt. it = b a e ity dy has modulus bounded That means we can use Fubini to compute I T.

37 Bochner s theorem Given any function φ and any points x 1,..., x n, we can consider the matrix with i, j entry given by φ(x i x j ). Call φ positive definite if this matrix is always positive semidefinite Hermitian.

38 Bochner s theorem Given any function φ and any points x 1,..., x n, we can consider the matrix with i, j entry given by φ(x i x j ). Call φ positive definite if this matrix is always positive semidefinite Hermitian. Bochner s theorem: a continuous function from R to R with φ(1) = 1 is a characteristic function of a some probability measure on R if and only if it is positive definite.

39 Bochner s theorem Given any function φ and any points x 1,..., x n, we can consider the matrix with i, j entry given by φ(x i x j ). Call φ positive definite if this matrix is always positive semidefinite Hermitian. Bochner s theorem: a continuous function from R to R with φ(1) = 1 is a characteristic function of a some probability measure on R if and only if it is positive definite. Positive definiteness kind of comes from fact that variances of random variables are non-negative.

40 Bochner s theorem Given any function φ and any points x 1,..., x n, we can consider the matrix with i, j entry given by φ(x i x j ). Call φ positive definite if this matrix is always positive semidefinite Hermitian. Bochner s theorem: a continuous function from R to R with φ(1) = 1 is a characteristic function of a some probability measure on R if and only if it is positive definite. Positive definiteness kind of comes from fact that variances of random variables are non-negative. The set of all possible characteristic functions is a pretty nice set.

41 Continuity theorems Lévy s continuity theorem: if lim φ X n n (t) = φ X (t) for all t, then X n converge in law to X.

42 Continuity theorems Lévy s continuity theorem: if lim φ X n n (t) = φ X (t) for all t, then X n converge in law to X. Slightly stronger theorem: If µ n = µ then φ n (t) φ (t) for all t. Conversely, if φ n (t) converges to a limit that is continuous at 0, then the associated sequence of distributions µ n is tight and converges weakly to measure µ with characteristic function φ.

43 Continuity theorems Lévy s continuity theorem: if lim φ X n n (t) = φ X (t) for all t, then X n converge in law to X. Slightly stronger theorem: If µ n = µ then φ n (t) φ (t) for all t. Conversely, if φ n (t) converges to a limit that is continuous at 0, then the associated sequence of distributions µ n is tight and converges weakly to measure µ with characteristic function φ. Proof ideas: First statement easy (since X n = X implies Eg(X n ) Eg(X ) for any bounded continuous g). To get second statement, first play around with Fubini and establish tightness of the µ n. Then note that any subsequential limit of the µ n must be equal to µ. Use this to argue that fdµ n converges to fdµ for every bounded continuous f.

44 Moments, derivatives, CLT If x n µ(x) < then the characteristic function φ of µ has a continuous derivative of order n given by φ (n) (t) = (ix) n e itx µ(dx).

45 Moments, derivatives, CLT If x n µ(x) < then the characteristic function φ of µ has a continuous derivative of order n given by φ (n) (t) = (ix) n e itx µ(dx). Indeed, if E X 2 < and EX = 0 then φ(t) = 1 t 2 E(X 2 )/2o(t 2 ).

46 Moments, derivatives, CLT If x n µ(x) < then the characteristic function φ of µ has a continuous derivative of order n given by φ (n) (t) = (ix) n e itx µ(dx). Indeed, if E X 2 < and EX = 0 then φ(t) = 1 t 2 E(X 2 )/2o(t 2 ). This and the continuity theorem together imply the central limit theorem.

47 Moments, derivatives, CLT If x n µ(x) < then the characteristic function φ of µ has a continuous derivative of order n given by φ (n) (t) = (ix) n e itx µ(dx). Indeed, if E X 2 < and EX = 0 then φ(t) = 1 t 2 E(X 2 )/2o(t 2 ). This and the continuity theorem together imply the central limit theorem. Theorem: Let X 1, X 2,... by i.i.d. with EX i = µ, Var(X i ) = σ 2 (0, ). If S n = X X n then (S n nµ)/(σn 1/2 ) converges in law to a standard normal.

18.175: Lecture 8 Weak laws and moment-generating/characteristic functions

18.175: Lecture 8 Weak laws and moment-generating/characteristic functions Scott Sheffield MIT 18.175 Lecture 8 1 Outline Moment generating functions Weak law of large numbers: Markov/Chebyshev approach