The Multivariate Normal Distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

Transcription

1 The Multivariate Normal Distribution Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

2 The Moment Generating Function (MGF) of a random vector X is given by M X (t) = E(e t X ) provided h > 0 E(e t X ) exists t = (t 1,..., t n ) t i ( h, h) i = 1,..., n. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

3 Result 5.1: If the MGFs of two random vectors X 1 and X 2 exist in an open rectangle R that includes the origin, then the cumulative distribution functions (CDFs) of X 1 and X 2 are identical iff M X1 (t) = M X2 (t) t R. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

4 A random variable Z with MGF M Z (t) = E(e tz ) = e t2 /2 is said to have a standard normal distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

5 Show that a random variable Z with density f Z (z) = 1 2π e z2 /2 has a standard normal distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

6 Suppose Z has a standard normal distribution. Then E(Z) = M Z(t) t t=0 = et2 /2 t t=0 = e t2 /2 (t) t=0 = 0. E(Z 2 ) = 2 M Z (t) t 2 t=0 = e t2 /2 + t 2 e t2 /2 t=0 = 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

7 Thus, E(Z) = 0 and Var(Z) = 1. If Z is standard normal, then Y = µ + σz has mean E(Y) = E(µ + σz) = µ + σe(z) = µ and variance Var(Y) = Var(µ + σz) = Var(σZ) = σ 2 Var(Z) = σ 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

8 Furthermore, the MGF of Y is M Y (t) = E(e ty ) = E(e t(µ+σz) ) = e tµ E(e tσz ) = e tµ M Z (tσ) = e tµ e t2 σ 2 /2 = e tµ+t2 σ 2 /2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

9 A random variable Y with MGF M Y (t) = e tµ+t2 σ 2 /2 is said to have a normal distribution with mean µ and variance σ 2. We denote the distribution of Y by N(µ, σ 2 ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

10 If Y N(µ, σ 2 ), then Thus, the density of Y is P(Y y) y P(Y y) = P(µ + σz y) = P(Z y µ σ ). P(Z y µ σ ) = y ( y µ = f Z σ ) 1 σ = 1 2πσ 2 e 1 2( y µ σ ) 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

11 That is, f Y (y) = = 1 2πσ 2 e 1 2( y µ σ ) 2 1 2πσ 2 e 1 2σ 2 (y µ)2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

12 Suppose Z 1,..., Z p i.i.d. N(0, 1). Then Z = Z 1. Z p is said to have a standard multivariate normal distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

13 E(Z) = 0 Var(Z) = I. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

14 Find the Moment Generating Function of a standard multivariate normal random vector Z. p 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

15 A p-dimensional random vector Y has the Multivariate Normal Distribution with mean µ and variance Σ (Y N(µ, Σ)) iff the MGF of Y is M Y (t) = e t µ+t Σt/2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

16 Suppose Z N(0, I). Show that Y = µ + AZ has a multivariate normal distribution. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

17 Note that if rank( q p A ) < q, then will be singular. Var(Y) = Var(µ + AZ) = AA In this case, the support of the q 1 random vector Y will lie within a rank(a)(< q)- dimensional vector space. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

18 Give a specific example of a singular multivariate normal distribution (Y N(µ, Σ), Σ singular). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

19 Result 5.3: If X N(µ, Σ) and Y = a + BX, then Y N(a + Bµ, BΣB ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

20 Proof of Result 5.3: E(e t Y ) = E(e t a+t BX ) = e t a E(e t BX ) = e t a M X (B t) = e t a e t Bµ+t BΣB t/2 = e t (a+bµ)+t BΣB t/2. Thus, Y N(a + Bµ, BΣB ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

21 Corollary 5.1: If X is multivariate normal (MVN), then the joint distribution of any p 1 subvector of X is MVN. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

22 Corollary 5.2: If X N(µ, Σ) and Σ is nonsingular, then p 1 (a) a nonsingular matrix A Σ = AA, (b) A 1 (X µ) N(0, I), and (c) The probability density function of X is f X (t) = (2π) p/2 Σ 1/2 e 1/2(t µ) Σ 1 (t µ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

23 Proof of Corollary 5.2: (a) We can take A = Σ 1/2 because Σ is symmetric and positive definite. Because Σ positive definite, (Σ 1/2 ) 1 = Σ 1/2 exists. (b) By Result 5.3, A 1 (X µ) N(A 1 µ A 1 µ, A 1 Σ(A 1 ) ), with A 1 µ A 1 µ = 0 and A 1 Σ(A 1 ) = Σ 1/2 ΣΣ 1/2 = I. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

24 (c) Homework problem. You may wish to use the multivariate change of variables result on page 185 of Casella and Berger. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

25 Result 5.2: Suppose the MGF of X i is M Xi (t i ) i = 1,..., p. Let X = [X 1, X 2,..., X p] and t = [t 1, t 2,..., t p]. Suppose X has MGF M X (t). Then X 1,..., X p are mutually independent iff p M X (t) = M Xi (t i ) i=1 t in an open rectangle that includes 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

26 Result 5.4: If X N(µ, Σ) and we partition X = X 1. X p, µ = µ 1. µ p, Σ = Then X 1,..., X p are mutually independent iff Σ 11 Σ 1p..... Σ p1 Σ pp. Σ ij = 0 i j. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36

27 Corollary 5.3: Suppose X N(µ, Σ) Y 1 = a 1 + B 1 X, Y 2 = a 2 + B 2 X. and Then Y 1 and Y 2 are independent iff B 1 ΣB 2 = 0. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36