1/24/2008. Review of Statistical Inference. C.1 A Sample of Data. C.2 An Econometric Model. C.4 Estimating the Population Variance and Other Moments

Transcription

1 /4/008 Review of Statistical Inference Prepared by Vera Tabakova, East Carolina University C. A Sample of Data C. An Econometric Model C.3 Estimating the Mean of a Population C.4 Estimating the Population Variance and Other Moments C.5 Interval Estimation Slide C- C.6 Hypothesis Tests About a Population Mean C.7 Some Other Useful Tests C.8 Introduction to Maximum Likelihood Estimation C.9 Algebraic Supplements Slide C-3

2 /4/008 Slide C-4 Figure C. Histogram of Hip Sizes Slide C-5 EY [ ] =μ (C.) var( Y) = E[ Y E( Y)] = E[ Y μ ] =σ (C.) Slide C-6

3 /4/008 y = yi (C.3) Y = Yi / i= (C.4) Slide C-7 y = yi (C.3) Y = Yi / i= (C.4) Slide C-8 Slide C-9 3

4 /4/008 Y Y = Yi = Y+ Y Y i= (C.5) E[ Y] = E Y + E Y E Y = EY [ ] + EY [ ] EY [ ] = μ+ μ μ =μ Slide C-0 Y var ( Y) = var Y+ Y Y = var Y var Y... var Y = σ + σ σ σ = ( Y ) + ( Y ) + + ( Y ) (C.6) Slide C- Y Figure C. Increasing Sample Size and Sampling Distribution of Y Slide C- 4

5 /4/008 Y Y μ P μ Y μ+ = P σ/ σ/ σ/ = P Z.5.5 = P Z =.9544 [ ] If we draw a random sample of size = 40 from a normal population with variance 0, the least squares estimator will provide an estimate within inch of the true value about 95% of the time. If = 80 the probability that Y is within inch of μ increases to Slide C-3 Central Limit Theorem: If Y,,Y are independent and identically distributed random variables with mean μ and variance σ Y μ, and Y = Yi /, then Z = σ has a probability distribution that converges to the standard normal (0,) as. Slide C-4 f ( y) y 0< y< = 0 otherwise Y /3 Z = /8 Slide C-5 5

6 /4/008 Figure C.3 Central Limit Theorem Slide C-6 A powerful finding about the estimator of the population mean is that it is the best of all possible estimators that are both linear and unbiased. A linear estimator is simply one that is a weighted average of the Y i s, such as Y = ay, where the a i i i are constants. Best means that it is the linear unbiased estimator with the smallest possible variance. Slide C-7 μ r = E Y ( μ) r ( ) ( ) μ = μ = μ = E Y E Y 0 ( Y ) μ = E μ =σ μ 3 = E μ 4 = E ( Y μ) ( Y μ) 3 4 Slide C-8 6

7 /4/008 var ( Y) =σ = E[ Y μ] σ = ( Yi Y) ˆ σ = ( Y ) i Y (C.7) Slide C-9 ( ) var Y =σˆ (C.8) se Y = var Y =σˆ / ( ) ( ) (C.9) Slide C-0 μ r = E Y ( μ) r In statistics the Law of Large umbers says that sample means converge to population averages (expected values) as the sample size. 3 4 ( i ) ( i ) ( i ) μ = Y Y =σ μ = μ = 3 Y Y 4 Y Y Slide C- 7

8 /4/008 μ = = σ 3 skewness S 3 4 kurtosis K μ = = σ 4 Slide C- ( yi y) ( yi ) σ ˆ = = = = σˆ var ( Y ) = = = ( Y) se =σ ˆ =.556 Slide C-3 ( Y ) i Y σ= = = 3 4 ( i ) μ = Y Y = ( i ) μ = Y Y = Slide C-4 8

9 /4/008 Y μ 8 μ PY ( > 8) = P > σ σ Y y > 8 > = >.4659 =.307 ˆ σ.8070 P ( Y ) P P ( Z ) Y y y* 7.58 y* 7.58 PY ( y* ) P P Z.95 ˆ = = σ y * 7.58 =.645 y * = Slide C-5 C.5. Interval Estimation: σ Known Y = Y i i= ( ) Y ~ μ, σ Z Y μ = = σ σ Y μ ~ 0, ( ) (C.0) P[ Z z] =Φ( z) Slide C-6 Figure C.4 Critical Values for the (0,) Distribution Slide C-7 9

10 /4/008 [ ] PZ [ ] PZ.96 =.96 =.05 [ ] P.96 Z.96 =.05 =.95 (C.) P Y.96 Y.96 σ μ + σ =.95 Slide C-8 σ σ P Y zc μ Y + zc = α (C.) Y ± z c σ (C.3) Slide C-9 Slide C-30 0

11 /4/008 Slide C-3 Any one interval estimate may or may not contain the true population parameter value. If many samples of size are obtained, and intervals are constructed using (C.3) with ( α) =.95, then 95% of them will contain the true parameter value. A 95% level of confidence is the probability that the interval estimator will provide an interval containing the true parameter value. Our confidence is in the procedure, not in any one interval estimate. Slide C-3 When σ is unknown it is natural to replace it with its estimator σˆ. ( Y ) i Y i= σ ˆ = Y μ t = t σ ˆ ( ) (C.4) Slide C-33

12 /4/008 Y μ P tc tc = α σˆ σˆ σˆ P Y tc μ Y + tc = α σˆ Y ± tc or Y ± tcse Y ( ) (C.5) Slide C-34 Remark: The confidence interval (C.5) is based upon the assumption that the population is normally distributed, so that Y is normally distributed. If the population p is not normal, then we invoke the central limit theorem, and say that Y is approximately normal in large samples, which from Figure C.3 you can see might be as few as 30 observations. In this case we can use (C.5), recognizing that there is an approximation error introduced in smaller samples. Slide C-35 Slide C-36

13 /4/008 Given a random sample of size = 50 we estimated the mean U.S. hip width to be = 7.58 inches. ˆ 3.65 therefore ˆ.807 σ = σ= σ ˆ = =.556 σˆ.807 y± t c = 7.58 ±.0 = , [ ] Slide C-37 Components of Hypothesis Tests A null hypothesis, H 0 An alternative hypothesis, H A test statistic A rejection region A conclusion Slide C-38 The ull Hypothesis The null hypothesis, which is denoted H 0 (H-naught), specifies a value c for a parameter. We write the null hypothesis as H : μ = c. A null hypothesis is the belief we 0 will maintain until we are convinced by the sample evidence that it is not true, in which case we reject the null hypothesis. Slide C-39 3

14 /4/008 The Alternative Hypothesis H : μ > c If we reject the null hypothesis that μ = c, we accept the alternative that μ is greater than c. H : μ < c If we reject the null hypothesis that μ = c, we accept the alternative that μ is less than c. H : μ c If we reject the null hypothesis that μ = c, we accept the alternative that μ takes a value other than (not equal to) c. Slide C-40 The Test Statistic A test statistic s probability distribution is completely known when the null hypothesis is true, and it has some other distribution if the null hypothesis is not true. Y μ t = ~ t( ) σ ˆ If H : μ = c is true then 0 Y c t = ~ t( ) σ ˆ (C.6) Slide C-4 Remark: The test statistic distribution in (C.6) is based on an assumption that the population is normally distributed. If the population is not normal, then we invoke the central limit theorem, and say that Y is approximately normal in large samples. We can use (C.6), recognizing that there is an approximation error introduced if our sample is small. Slide C-4 4

15 /4/008 The Rejection Region If a value of the test statistic is obtained that falls in a region of low probability, then it is unlikely that the test statistic has the assumed distribution, and thus it is unlikely that the null hypothesis is true. If the alternative hypothesis is true, then values of the test statistic will tend to be unusually large or unusually small, determined by choosing a probability α, called the level of significance of the test. The level of significance of the test α is usually chosen to be.0,.05 or.0. Slide C-43 A Conclusion When you have completed a hypothesis test you should state your conclusion, whether you reject, or do not reject, the null hypothesis. Say what the conclusion means in the economic context of the problem you are working on, i.e., interpret the results in a meaningful way. Slide C-44 Figure C.5 The rejection region for the one-tail test of H : μ = c against H : μ > c Slide C-45 5

16 /4/008 Figure C.6 The rejection region for the one-tail test of H : μ = c against H : μ < c Slide C-46 Figure C.7 The rejection region for a test of H : μ = c against H : μ c Slide C-47 The null hypothesis is H : μ = The alternative hypothesis is H : μ > 6.5. Y 6.5 The test statistic t = ~ t ˆ hypothesis is true. σ ( ) if the null The level of significance α=.05. = t = tc (.95,49).6766 Slide C-48 6

17 /4/008 The value of the test statistic is t = = Conclusion: Since t =.5756 >.68 we reject the null hypothesis. The sample information we have is incompatible with the hypothesis that μ = 6.5. We accept the alternative that the population mean hip size is greater than 6.5 inches, at the α=.05 level of significance. Slide C-49 The null hypothesis is H : μ = 7. 0 The alternative hypothesis is H : μ 7. Y 7 The test statistic t = ~ t ˆ hypothesis is true. σ ( ) if the null The level of significance α=.05, therefore = t = tc (.975,49).0 α =.05. Slide C-50 The value of the test statistic is t = = Conclusion: Since.0 < t =.69<.0 we do not reject the null hypothesis. The sample information we have is compatible with the hypothesis that the population mean hip size μ = 7. Slide C-5 7

18 /4/008 Warning: Care must be taken here in interpreting the outcome of a statistical test. One of the basic precepts of hypothesis testing is that finding a sample value of the test statistic in the non-rejection region does not make the null hypothesis true! The weaker statements we do not reject the null hypothesis, or we fail to reject the null hypothesis, do not send a misleading message. Slide C-5 p-value rule: Reject the null hypothesis when the p- value is less than, or equal to, the level of significance α. That is, if p α then reject H 0. If p > α then do not reject H 0 Slide C-53 How the p-value is computed depends on the alternative. If t is the calculated value [not the critical value t c ] of the t- statistic with degrees of freedom, then: if H : μ > c, p = probability to the right of t if H : μ < c, p = probability to the left of t if H : μ c, p = sum of probabilities to the right of t and to the left of t Slide C-54 8

19 /4/008 Figure C.8 The p-value for a right-tail test Slide C-55 Figure C.9 The p-value for a two-tailed test Slide C-56 A statistical test procedure cannot prove the truth of a null hypothesis. When we fail to reject a null hypothesis, all the hypothesis test can establish is that the information in a sample of data is compatible with the null hypothesis. On the other hand, a statistical test can lead us to reject the null hypothesis, with only a small probability, α, of rejecting the null hypothesis when it is actually true. Thus rejecting a null hypothesis is a stronger conclusion than failing to reject it. Slide C-57 9

20 /4/008 Correct Decisions The null hypothesis is false and we decide to reject it. The null hypothesis is true and we decide not to reject it. Incorrect Decisions The null hypothesis is true and we decide to reject it (a Type I error) The null hypothesis is false and we decide not to reject it (a Type II error) Slide C-58 The probability of a Type II error varies inversely with the level of significance of the test, α, which is the probability of a Type I error. If you choose to make α smaller, the probability of a Type II error increases. If the null hypothesis is μ = c, and if the true (unknown) value of μ is close to c, then the probability of a Type II error is high. The larger the sample size, the lower the probability of a Type II error, given a level of Type I error α. Slide C-59 H : μ= c 0 H : μ c If we fail to reject the null hypothesis at the α level of significance, then the value c will fall within a 00( α)% confidence interval estimate of μ. If we reject the null hypothesis, then c will fall outside the 00( α)% confidence interval estimate of μ. Slide C-60 0

21 /4/008 We fail to reject the null hypothesis when when Y c tc t σˆ c t t t c, c or σˆ Y tc c Y + t c σˆ Slide C-6 C.7. Testing the population variance ( ) Y ~ μσ,, Y =Yi σ ˆ = ( Yi Y) ( ) H : σ =σ 0 0 ( )ˆ σ V = ~ χ σ 0 ( ) Slide C-6 If H : σ >σ, then the null hypothesis is rejected if V χ 0. (.95, ) If H: σ σ0, then we carry out a two tail test, and the null hypothesis is rejected if V χ(.975, ) or if V χ(.05, ). Slide C-63

22 /4/008 Case : Population variances are equal p ( ) ˆ ( ) σ + σˆ p + σ ˆ = σ =σ =σ If the null hypothesis H : μ μ = c is true then t = ( ) 0 Y Y c ~ t σ ˆ p + ( + ) Slide C-64 Case : Population variances are unequal df ( ) Y Y c * t = σˆ ˆ σ + ( σ ˆ +σˆ ) ( σˆ ) ( σˆ ) = + Slide C-65 ( ) σˆ σ ( ) σˆ σ ~ F (, ) ( ˆ ) σ σ σˆ σ ( ) F = = Slide C-66

23 /4/008 The normal distribution is symmetric, and has a bell-shape with a peakedness and tail-thickness leading to a kurtosis of 3. We can test for departures from normality by checking the skewness and kurtosis from a sample of data. μ skewness = S = σ μ 4 kurtosis = K = σ Slide C-67 The Jarque-Bera test statistic allows a joint test of these two characteristics, ( K ) JB = 6 S If we reject the null hypothesis then we know the data have nonnormal characteristics, but we do not know what distribution the population might have. Slide C-68 For the Hip data, ( K ) ( ) JB = S + = (.038) + = p= P χ().935 =.673 Slide C-69 3

24 /4/008 Figure C.0 Wheel of Fortune Game Slide C-70 For wheel A, with p=/4, the probability of observing WI, WI, LOSS is = 64 = For wheel B, with p=3/4, the probability of observing WI, WI, LOSS is = = Slide C-7 If we had to choose wheel A or B based on the available data, we would choose wheel B because it has a higher probability of having produced the observed data. It is more lk likelyl that wheel B was spun than wheel A, and p ˆ = 34is called the maximum likelihood estimate of p. The maximum likelihood principle seeks the parameter values that maximize the probability, or likelihood, of observing the outcomes actually obtained. Slide C-7 4

25 /4/008 Suppose p can be any probability between zero and one. The probability of observing WI, WI, LOSS is the likelihood L, and is 3 L( p) = p p ( p) = p p (C.7) We would like to find the value of p that maximizes the likelihood of observing the outcomes actually obtained. Slide C-73 Figure C. A Likelihood Function Slide C-74 ( ) dl p = p 3p dp ( ) p 3 p = 0 p 3 p = 0 = = There are two solutions to this equation, p=0 or p=/3. The value that maximizes L(p) is p ˆ = 3, which is the maximum likelihood estimate. Slide C-75 5

26 /4/008 Let us define the random variable X that takes the values x= (WI) and x=0 (LOSS) with probabilities p and p. x [ ] f ( p ) p ( p ) x P X = x = f x p = p p, x = 0, (,, ) = ( ) ( ) f x x p f x p f x p = p xi ( p) (,, x ) = L p x xi (C.8) Slide C-76 Figure C. A Log-Likelihood Function Slide C-77 ( ) = ( i ) ln L p ln f x p i= i i i= i= ( ) ( ) = x ln p + x ln p (C.9) ( ) dln L p xi xi = dp p p Slide C-78 6

27 /4/008 xi xi = 0 pˆ pˆ ( ) i ( i) p ˆ x p ˆ x = 0 xi pˆ = = x (C.0) Slide C-79 ( θ ) = ( i θ) ln L ln f x i= ( θv) θˆ ~ a, (C.) θ ˆ c a t = ~ t( ) se ( θˆ ) (C.) Slide C-80 REMARK: The asymptotic results in (C.) and (C.) hold only in large samples. The distribution of the test statistic can be approximated by a t-distribution with degrees of freedom. If is truly large then the t (-) distribution converges to the standard normal distribution (0,). When the sample size may not be large, we prefer using the t-distribution critical values, which are adjusted for small samples by the degrees of freedom correction, when obtaining interval estimates and carrying out hypothesis tests. Slide C-8 7

28 /4/008 ( ˆ d ln L V = var θ ) = E dθ ( θ) (C.3) Slide C-8 Figure C.3 Two Log-Likelihood Functions Slide C-83 ( ) ( ) d ln L p xi xi = dp p p (C.4) ( ) ( ) 0 ( 0) 0 ( ) E x = P x = + P x = = p+ p = p i i i Slide C-84 8

29 /4/008 ( ) ( i) ( i) = ( ) E d ln L p E x E x dp p p p p = p = p ( p) ( p) Slide C-85 d ln L p p p ( ) ( ) V = var ( pˆ ) = E = dp ( ) a p p pˆ ~ p, Slide C-86 ( pˆ) ˆ pˆ V = se ( pˆ ) = Vˆ = pˆ ( pˆ) Slide C-87 9

30 /4/008 ( pˆ) pˆ se ( pˆ ) = = = pˆ t = = =.7303 se.034 ( pˆ ) ( pˆ) ( ) [ ] pˆ ±.96 se =.375 ± =.3075,.445 Slide C-88 C.8.4a The likelihood ratio (LR) test The likelihood ratio statistic which is twice the difference between L ( θˆ ) L ( c ) ln and ln. ( ˆ ) ( ) LR= lnl θ lnl c (C.5) Slide C-89 Figure C.4 The Likelihood Ratio Test Slide C-90 30

31 /4/008 Figure C.5 Critical Value for a Chi-Square Distribution Slide C-9 ln L( pˆ) = x ln ˆ ln( ˆ i p+ xi p) i= i= ( ) = pˆln pˆ + pˆ ln( pˆ) ( ) = pˆln pˆ + pˆ ln( pˆ) Slide C-9 For the cereal box problem p ˆ =.375 and = 00. [ ] ln L( p ˆ) = ln(.375) + (.375)ln(.375) = 3.36 Slide C-93 3

32 /4/008 The value of the log-likelihood function assuming is true is: ln L (.4) = xi ln(.4) + xi ln(.4) i= i= = 75 ln(.4) + (00 75) ln(.6) = H : p=.4 0 Slide C-94 The problem is to assess whether 3.36 is significantly different from The LR test statistic (C.5) is: ( ) LR = [ln L( pˆ ) ln L(.4)] = 3.36 ( 3.575) =.547 The critical value is χ = (.95,) Since.547 < 3.84 we do not reject the null hypothesis. Slide C-95 Figure C.6 The Wald Statistic Slide C-96 3

33 /4/008 ( ˆ c) d ln L W = θ dθ ( θ) (C.6) If the null hypothesis is true then the Wald statistic (C.6) has a distribution, and we reject the null hypothesis if W χ χ(). ( α,) Slide C-97 ( θ) d ln L I ( θ ) = E = V dθ (C.7) ( ˆ ) ( ) W = θ c I θ (C.8) ( ˆ ) ( ˆ ) W = θ c V = θ c V (C.9) Slide C-98 ˆ V ( ˆ ) = I θ (C.30) θ ˆ c θ ˆ c W = = = t Vˆ se ( θˆ ) Slide C-99 33

34 /4/008 In the blue box-green box example: 00 ( ˆ ) ˆ I p = V = = = pˆ ( pˆ ).375 (.375 ) ( ) ( ) ( ) W = pˆ c I pˆ = =.5333 Slide C-00 Figure C.7 Motivating the Lagrange multiplier test Slide C-0 dln L s( θ ) = dθ ( θ) (C.3) ( ) ( θ) s c LM = = s( c) I ( θ) I (C.3) ( ) ( ) LM = s c I c ( ˆ ) ( ˆ) W = θ c I θ Slide C-0 34

35 /4/008 In the blue box-green box example: xi xi s(.4) = = = c c I (.4) = = = c ( c).4(.4) ( ) ( ) [ ] [ ] LM = s.4 I.4 = =.508 Slide C-03 C.9. Derivation of Least Squares Estimator S = ( y μ) i= i d = ( y μ) i i d = ( y μ) i i Slide C-04 ( μ ) = i = ( i μ) S d y i= i= ( μ ) = i μ i + μ = 0 μ+ μ S y y a a a i= i= a = y = , a = y = , a = = 50 0 i i Slide C-05 35

36 /4/008 Figure C.8 The Sum of Squares Parabola For the Hip Data Slide C-06 ( ) ds μ = a + a μ dμ a + a μ= ˆ 0 a a i i= μ= ˆ = = Y y i i= μ= ˆ = Y y Slide C-07 For the hip data in Table C. yi i= μ= ˆ = = Thus we estimate that the average hip size in the population is 7.58 inches. Slide C-08 36

37 /4/008 Y = Yi / = Y+ Y Y i= = ay + ay ay = ay i i i= Slide C-09 a iyi i= Y = a i = ai + ci = + ci Slide C-0 Y = a Y = + c Y i i i i i= i= = Y + cy i i i i= i= = Y + cy i i i= Slide C- 37

38 /4/008 E Y = E Y + cy =μ+ ce Y [ ] i i i i i= i= =μ+μc i i= Slide C- i i i i i i i= i= i= var( Y ) = var a Y = var + c Y = + c var( Y) ci c i ci ci ci i= i= i= i= =σ + =σ + + =σ + + ci i= =σ +σ (since c = 0) i= i = var( Y) +σ ci i= Slide C-3 alternative hypothesis asymptotic distribution BLUE central limit theorem central moments estimate estimator experimental design information measure interval estimate Lagrange multiplier test Law of large numbers level of significance likelihood function likelihood ratio test linear estimator log likelihood function maximum likelihood estimation null hypothesis point estimate population parameter p-value random sample rejection region sample mean sample variance sampling distribution sampling variation standard error standard error of the mean standard error of the estimate statistical inference test statistic two-tail tests Type I error Type II error unbiased estimators Wald test Slide C-4 38