4.5.1 The use of 2 log Λ when θ is scalar

Transcription

1 4.5. ASYMPTOTIC FORM OF THE G.L.R.T The use of 2 log Λ when θ is scalar Suppose we wish to test the hypothesis NH : θ = θ where θ is a given value against the alternative AH : θ θ on the basis of the values x 1, x 2,...,x n of a large random sample X 1,X 2,...,X n from a distribution with p.d/m.f. fx θ, where θ is a scalar parameter with θ Θ. The likelihood for the given sample values is n Lθ = fx i θ and since the sample is large we can test the hypothesis using the g.l.r.t. with test statistic 2 log Λx = 2 log L θ log Lθ = 2l θ lθ where θ is the m.l.e. of θ. This test statistic is approximately χ 2 1 distributed when NH is true. There will be no statistical evidence against NH i.e. we do not reject NH at α-level of significance if Dθ = 2 l θ lθ χ 2 1,α 4.22 Criterion 4.22 says do not reject NH if the value of the function Dθ at θ = θ is smaller than χ 2 1,α where Dθ = 2 l θ lθ. Definition: The function Dθ = 2 l θ lθ is called the Deviance function. Since the m.l.e. θ maximizes the log-likelihood function lθ, the deviance function enjoys the properties 1. Dθ 0 for all θ Θ, 2. D θ = 0, 3. Dθ exhibits a minimum at θ = θ. One can then plot the Deviance function Dθ for all θ Θ using a mathematical package such as Matlab or Maple and superimpose on the plot

2 98 CHAPTER 4. MAXIMUM LIKELIHOOD ESTIMATION Deviance function Dθ 2 χ 1,α 0 θ L θ U θ Figure 4.10: The deviance function and its use in constructing confidence intervals a horizontal line at level χ 2 1,α as shown in fig4.10. If at θ = θ the curve of Dθ is above the horizontal line i.e. Dθ > χ 2 1,α then the hypothesis NH : θ = θ is rejected. Otherwise it is not. Equivalently we could argue that Dθ 0 = 2 l θ lθ 0 χ 2 1 if θ 0 is the true value of the parameter, i.e. Pr Dθ 0 χ 2 1,α = 1 α 4.23 Notice that the position of the curve Dθ is random since θ is random/uncertain prior to taking the observations Equation 4.23 says there is probability 1001 α% that the curve will position itself so that at the true value θ 0, Dθ 0 is below the level χ 2 1,α i.e. there is probability 1001 α% that the curve will position itself so that the true value θ 0 θ L,θ U Prθ L θ 0 θ U where θ L,θ U are the roots of the equation Dθ χ 2 1,α = 0. Thus the interval θ L,θ U = {θ : Dθ χ 2 1,α}

3 4.5. ASYMPTOTIC FORM OF THE G.L.R.T. 99 is a 1001 α% confidence interval for θ. θ L,θ U may found by plotting Dθ if need be using Matlab or Maple etc and finding the points at which it intersects the constant line χ 2 1,α. Example Let X 1,X 2,...,X 50 be a random sample from the Poisson distribution with parameter θ. The observed values x 1,x 2,...,x 50 gave a total value 50 = 200. Test the hypothesis θ = 5 against θ 5 at the 5% level of significance and find a 95% confidence interval for θ. Solution The likelihood is Lθ = and the log-likelihood is Thus 50 e θ θ x i x i! = e 50θ θ 200.Const = e 50θ θ 50 x i 50 lθ = 50θ log θ + Constant dlθ dθ = θ = 0 so that the m.l.estimate θ = 4. Hence the deviance function is Dθ = 2 l θ lθ = log θ 200 log θ = 250θ log θ 4 = 100θ log θ 4 Fig4.11 shows the plot of this deviance function together with the constant line at χ 2 1;0.05 = As can be seen from the figure at θ = 5, Dθ = which is above the constant line which leads us to the conclusion that there is evidence at the 5% level of significance to reject the hypothesis θ = 5. Further, from the plot we see that deviance function crosses the horizontal line at θ L = 3.48 and θ U = 4.58 Thus a 95% confidence interval for the Poisson mean θ is 3.48, x i!

4 100 CHAPTER 4. MAXIMUM LIKELIHOOD ESTIMATION Deviance function Dθ θ Figure 4.11: The deviance function and confidence intervals interval for Poisson mean θ The Pearson Chi-squared statistic in the Multinomial test The statistic 2 o i log oi in the multinomial test can be approximated by the statistic e i X 2 = o i e i 2 e i known as the Pearson chi-squared statistic, which is also, for large sample size n, approximately χ 2 k 1 s distributed when the null hypothesis NH is true. The null hypothesis NH is therefore rejected if at α-level of significance if X 2 > χ 2 k 1 s; α The approximation can be shown as follows. Recall that

5 4.5. ASYMPTOTIC FORM OF THE G.L.R.T The m.l.e. of θ i is θ i = x i n and is consistent for θ i, i = 1, 2,...,k. 2. When NH : θ i = π i β ı = 1, 2,...,k is true, the m.l.e π i β is a consistent estimator of θ i, i = 1, 2,...,k. Hence, when NH is true x i and π n i β are close to each other with high probability i.e. x i n π i β = δ i is small with high probability. Hence when NH is true we can write x i = n π i β + δ i and for some small δ i. Consequently 2 o i log oi e i = 2 = 2n 2n n x i nπ i β = 1 + δ i π i β xi x i log nπ i β π i β + δ i log 1 + δ i π i β π i β + δ i δ i π i β δ 2 i π i β = 2πi 2 β x i nπ i β 2 o i e i 2 nπ i β = e i where the approximation was obtained by taking a Taylor expansion of the logarithm about 1 and retaining terms of order δ 2 i or less. We also made use of the fact that k δ i = Goodness of fit of a distribution The multinomial test can be used to determine whether a random sample of values y 1,y 2,...,y n come from a hypothesized distribution with p.m/d.f. fy β, y R Y as follows. δ2 i

6 102 CHAPTER 4. MAXIMUM LIKELIHOOD ESTIMATION Partition R Y into intervals I i, i = 1, 2,...,k if the hypothesized distribution is continuous and into set I i, i = 1, 2,...,k if the hypothesized distribution is discrete. For example, if the hypothesized distribution is the Normal Nµ,σ 2, β = µ,σ 2 distribution then we can take I 1 =,a 1, I 2 = a 1,a 2 I k 1 = a k 2,a k 1, I k = a k 1, whereas if for example the hypothesized distribution is Poisson we can take I 1 = {0}, I 2 = {1}, I k 1 = {k 1}, I k = {l : l k} The number k of intervals/sets should be taken such that there are at least five y j values falling in each interval/set. Let x i be the number of y j sample values that fall in I i, i = 1, 2,...,k. Take π i β = fy βdy if hypothesized dist n is continuous I i = y I i fy β if hypothesized dist n is discrete and proceed to perform a multinomial test to test the hypothesis NH : PrI i = π i β, i = 1, 2,...,k. If this hypothesis is not rejected then it can be assumed that the hypothesized distribution with p.m/d.f. fy β is the true distribution of the y j s. The estimator β under NH can be taken the value of β that maximizes the likelihood of the original sample y 1,y 2,...,y n L y β = n fy j β j=1 instead of the value that maximises the likelihood of the x i values i.e. the multinomial probability. L x β = n! n x i! n π i β x i