1/15. Over or under dispersion Problem

Transcription

1 1/15 Over or under dispersion Problem

2 2/15 Example 1: dogs and owners data set In the dogs and owners example, we had some concerns about the dependence among the measurements from each individual. Let Y ij = 1 if the j-th quiz question was answered correctly by the i-th person. In the data set we collected, i = 1,, 27 and j = 1,, 12. It is reasonable to assume that Y ij s (j = 1,, 12) are dependent to each other.

3 3/15 Example 1 continued To model the dependence among Y ij s (j = 1,, 12), we could assume that Y ij Bernoulli(ν i ) where ν i Beta(α 1, α 2 ) is a random variable. Using the property of Beta distribution, E(ν i ) = α 1 α 1 + α 2 and Var(ν i ) = α 1 α 1 + α 2 α 2 α 1 + α 2 1 α 1 + α For convenience, define p i = α 1 /(α 1 + α 2 ) and φ as an additional parameter. Then E(ν i ) = p i and Var(ν i ) = φp i (1 p i ).

4 4/15 Example 1 continued By using the above model, we know that Cov(Y ij, Y ij ) = E(Y ij Y ij ) E(Y ij )E(Y ij ) = E(ν 2 i ) E 2 (ν i ) = Var(ν i ) = φp i (1 p i ). If φ = 0, then Y ij and Y ij are uncorrelated. This also implies that ν i is a constant degenerated to p i. If φ > 0, then Y ij and Y ij are dependent. This corresponds to the overdispersion case.

5 Example 1 continued Let S i = Y i1 + + Y ini. Then we have and E(S i ) = E{E(S i ν i )} = E(ν i ) = n i p i, n i j=1 Var(S i ) = E{Var(S i ν i )} + Var{E(S i ν i )} = E{n i ν i (1 ν i )} + Var{n i ν i } = n i (p i φp i (1 p i ) pi 2 ) + n2 i φp i(1 p i ) = n i p i (1 p i )[1 + (n i 1)φ]. If φ = 0, no dispersion. If φ > 0, over dispersion; If φ < 0, under dispersion. 5/15

6 6/15 Over and under dispersion problems In a common logistic regression model, S i Binomial(n i, p i ) and p i = exp(x T i β) 1 + exp(x T i β). If one assumes that the model for p i is correctly specified but the observed variance of S i is larger or smaller than expected variance n i p i (1 p i ), then we have the so-called under or over dispersion problems.

7 7/15 Detection of over or under dispersion problem If the usual logistic regression model is correct, then the deviance D follows a chi-square distribution with m p degrees of freedom. If D > m p = E(χ 2 m p), it could be an indicator of the over dispersion problem. If D < m p = E(χ 2 m p), it could be an indicator of the under dispersion problem. But D is away from m p could also be the result of (1): under or over fitting; (2): wrong link function; (3): existence of outliers; (4): binary data or n i small.

8 8/15 Possible reasons for dispersion Variation among success probabilities. Correlation among binary responses.

9 Over or under dispersion logistic regression model Let S i be the number of successes among n i trials. An over or under dispersion logistic regression model assumes that E(S i ) = n i p i and Var(S i ) = φn i p i (1 p i ). Moreover, p i = exp(x i T β) 1 + exp(xi T β). Here φ is called dispersion parameter. 9/15

10 Quasi-likelihood Recall that, in a usual logistic regression model, S i Binomial(n i, p i ) and p i = exp(x T i The log-likelihood for β is l(β) = S i Xi T β The score function for β is β)/{1 + exp(xi T β)}. n i log{1 + exp(xi T β)} + Constant. l(β) β m = X i (S i n i p i ) = S i n i p i n i p i n i p i (1 p i ) β = β where V (µ) = µ(n i µ)/n i and µ i = n i p i. µi S i S i µ V (µ) dµ 10/15

11 11/15 Quasi-likelihood for over or under dispersion models Define the log quasi-likelihood for β as Q(β) = Q i = µi S i µ S i φv (µ) dµ. The maximum quasi-likelihood estimator of β is ˆβ = arg max Q(β). The above estimator ˆβ is the same as the MLE of β in a usual logistic regression model. Because φ is not useful in the quasi-score function.

12 12/15 Estimation of dispersion parameter Define the Pearson χ 2 statistic as χ 2 = (S i n i ˆp i ) 2 n i ˆp i (1 ˆp i ). It can be shown that E(χ 2 ) (m p)φ. Then we can estimate φ by ˆφ = χ 2 /(m p).

13 13/15 Deviance The deviance for the over or under dispersion logistic regression model is defined as D = 2φQ = 2 µi S i S i µ V (µ) dµ. The deviance is the same as the usual logistic regression model without dispersion parameter.

14 14/15 Wald type inference for β For over or under dispersion logistic regression model, ˆβ β N(0, φ(x T VX) 1 ) where V = diag(n 1 p 1 (1 p 1 ),, n m p m (1 p m )) and φ is the dispersion parameter. In performing the inference, we need to take the dispersion parameter into consideration.

15 15/15 Likelihood ratio type inference for β Suppose we consider comparing the following two models Model Deviance Covariates 1 D 1 x 1,, x l 2 D 2 x 1,, x l, x l+1,, x p for p > l. This corresponds to testing H 0 : β l+1 = = β p = 0 vs H 1 : not H 0. The test statistic for testing above hypothesis is F n = (D 1 D 2 )/(p l), ˆφ which follows an F p l,m p distribution under H 0.