7.3 The Chi-square, F and t-distributions

Transcription

1 7.3 The Chi-square, F and t-distributions Ulrich Hoensch Monday, March 25, 2013

2 The Chi-square Distribution Recall that a random variable X has a gamma probability distribution (X Gamma(r, λ)) with parameters r > 0 and λ > 0 if it has the PDF Definition f (x) = λr Γ(r) x r 1 e λx, x > 0. A random variable X has a chi-square distribution with n degrees of freedom if X Gamma(n/2, 1/2). We write X χ 2 (n) in this situation.

3 The Chi-Square Distribution Theorem Let Z 1, Z 2,..., Z n be independent random variables with Z i N(0, 1). Then the sum V = n i=1 Z i 2 has the distribution V χ 2 (n). Proof. It is sufficient to show that if Z N(0, 1), then Z 2 χ 2 (1) = Gamma(1/2, 1/2). This is since the Zi 2 are also independent and it follows from Theorem that the sum of n independent random variables Zi 2 Gamma(1/2, 1/2) has again a gamma distribution with parameters r = n/2 and λ = 1/2.

4 The Chi-Square Distribution Let Z N(0, 1), i.e. f Z (z) = (1/ 2π)e z2 /2. Then, for U = Z 2, and u > 0 the CDF is F U (u) = P(Z 2 u) = P( u Z u) = = u u 2 u 2π 1 2π e z2 /2 dz 0 e z2 /2 dz. Differentiating (and using the chain rule) gives f U (u) = 2 2π 1 2 u e u/2 = (1/2)1/2 Γ(1/2) u(1/2) 1 e (1/2)u, where we used that Γ(1/2) = 0 e t t (1/2) 1 dt = π.

5 Distribution of the Sample Variance Theorem Let X 1, X 2,..., X n be a random sample of size n from a normally distributed population with mean µ and variance σ 2. Let S 2 = (1/(n 1)) n i=1 (X i X ) 2 be the sample variance. Then, (a) X and S 2 are independent random variables. (n 1)S 2 (b) σ 2 χ 2 (n 1). Remarks. The Central Limit Theorem and Theorem give us information about the distribution of the sample mean and the sample variance of a random sample taken from a normally distributed population. You can use χ 2 cdf (a, b, n) on a TI-83/84 calculator to find the probability that a chi-square distributed random variable with n degrees of freedom is between a and b.

6 Proof of Theorem Fisher s Lemma. Let Z 1, Z 2,..., Z n be iid random variables with Z i N(0, 1), and let A be an orthogonal n n-matrix (i.e. AA T = I). Let (U 1, U 2,..., U n ) = A(Z 1, Z 2,..., Z n ). Then U 1, U 2,..., U n are iid random variables with U i N(0, 1) Proof of Fisher s lemma. We have that for U = (U 1, U 2,..., U n ) and Z = (Z 1, Z 2,..., Z n ), P(U D) = P(AZ D) = P(Z A 1 D) = f Z (z) dz. A 1 D

7 Proof of Theorem Using the change of coordinates formula, the facts that det A = ±1 and Ax = x, and independence, we get: P(U D) = f Z (z) dz A 1 D = f Z (A 1 u) det A 1 du D = f Z (A 1 u) du D ( ) 1 n = exp( (1/2) A 1 u 2 ) du D 2π ( ) 1 n = exp( (1/2) u 2 ) du. 2π D Thus, U 1, U 2,..., U n are independent and U i N(0, 1).

8 Proof of Theorem Now, let A be an orthogonal n n-matrix whose last row is (1/ n, 1/ n,..., 1/ n). (It is possible to construct such a matrix using the Gram-Schmidt method.) Also, let Z i N(0, 1) be iid. The last row of A was chosen so that U n = (1/ n)z 1 + (1/ n)z (1/ n)z n = ( n)z. Thus, since n i=1 U2 i = n i=1 Z 2 i, n 1 Ui 2 + nz 2 = i=1 n Ui 2 = i=1 n i=1 Z 2 i = n (Z i Z) 2 + nz 2. i=1 Thus, n 1 i=1 U2 i = n i=1 (Z i Z) 2 and nz 2 (and hence Z) is independent of n i=1 (Z i Z) 2.

9 Proof of Theorem Also, n n 1 (Z i Z) 2 = Ui 2 χ 2 (n 1) i=1 from theorem i=1 The general result follows from normalization: X = σz + µ, n i=1 (X i X ) 2 = σ 2 n i=1 (Z i Z) 2.

10 Example Suppose X 1, X 2,..., X 10 is a random sample taken from an N(12, 25)-distributed population. Find the range for the middle 90% of the distribution of the sample mean X.

11 Example Find the probability that the sample variance S 2 is between 20 and 30.

12 Example Find the range for the middle 90% of the distribution of the sample variance.

13 F -Distribution Definition Suppose U and V are independent random variables, U χ 2 (n 1 ) and V χ 2 (n 2 ). Then, F = (U/n 1 )/(V /n 2 ) has a (Fisher) F -distribution with n 1 numerator and n 2 denominator degrees of freedom. We write F F (n 1, n 2 ) in this situation. Remarks. The PDF of F F (n 1, n 2 ) is zero if x 0 and for x > 0 it is equal to f (x) = Γ((n 1 + n 2 )/2) Γ(n 1 /2)Γ(n 2 /2) ( n1 n 2 ) n1 /2 ( x (n 1/2) n ) (n1 +n 2 )/2 1 x. n 2 The F -distribution is skewed to the right, and if F α (n 1, n 2 ) is chosen so that P(F > F α (n 1, n 2 )) = α, then F α (n 1, n 2 )F 1 α (n 2, n 1 ) = 1.

14 F -Distribution Theorem Suppose a random sample of size n 1 is drawn from a normally distributed population with variance σ1 2, and (independently) another random sample of size n 2 is drawn from a normally distributed population with variance σ2 2. If S 1 2 is the variance of the first sample, and S2 2 is the variance of the second sample, then F = S 2 1 /σ2 1 S 2 2 /σ2 2 F (n 1 1, n 2 1). Proof: By Theorem 7.3.2, U = (n 1 1)S 2 1 /σ2 1 χ2 (n 1 1) and V = (n 2 1)S 2 2 /σ2 2 χ2 (n 2 1). Note that U and V are independent. By Definition 7.3.2, F = (U/(n 1 1))/(V /(n 2 1)) F (n 1 1, n 2 1). But now F = U/(n 1 1) V /(n 2 1) = S 1 2/σ2 1 S2 2. /σ2 2

15 Example Two random samples are taken from two different normally distributed populations. It is assumed that these populations have the same variance σ 2 = σ1 2 = σ2 2. Suppose the size of the first sample is n 1 = 20 and the size of the second sample is n 2 = 30. Find the probability that the ratio of the sample variances S1 2/S 2 2 is between 0.5 and 1.5.

16 Example Find two values a and b so that P(a S1 2/S 2 2 b) = 0.90.

17 Student t-distribution Definition Suppose U and Z are independent random variables, U χ 2 (n) and Z N(0, 1). Then, T = Z/ U/n has a (Student) t-distribution with n degrees of freedom. We write T t(n) in this situation. Remarks. The PDF of T t(n) is symmetric about the line t = 0 and equal to f (t) = ( ) (n+1)/2 Γ((n + 1)/2) 1 + t2. πnγ(n/2) n If T t(n), then E(T ) = 0 and Var(T ) = n/(n 2). For n 30, we have that a t(n)-distribution can be well-approximated by a N(0, 1)-distribution.

18 Student t-distribution n 2 n 5 n 10 n 30 4 The PDF of T t(n) for n = 2, 5, 10, 30.

19 Student t-distribution Theorem Suppose X is the mean and S 2 is the variance of a random sample of size n taken from a normally distributed population with mean µ and variance σ 2. Then the random variable T = X µ S/ n t(n 1). Proof: By the Central Limit Theorem, X N(µ, σ 2 /n), so Z = (X µ)/(σ/ n) N(0, 1). By Theorem 7.3.2, U = (n 1)S 2 /σ 2 χ 2 (n 1). Since X and S 2 are independent by Theorem 7.3.2, so are Z and U. Thus, by Definition 7.3.3, T = Z/ U/(n 1) t(n 1). But now T = Z U/(n 1) = X µ σ/ n (n 1)S 2 σ 2 (n 1) = X µ σ/ n S/σ = X µ S/ n.

20 Homework Problems for Section 7.3 (Points) p.394: (2), (2), (2), (2). Homework problems are due at the beginning of the class on Wednesday, April 3, 2013.