GENLOG Multinomial Loglinear and Logit Models

Transcription

1 GENLOG Multinomial Loglinea and Logit Models Notation This chapte descibes the algoithms used to calculate maximum-likelihood estimates fo the multinomial loglinea model and the multinomial logit model. This algoithm is applicable only to aggegated data. The following notation is used thoughout this chapte unless othewise stated: A Geneic categoical independent (explanatoy) vaiable. Its categoies ae indexed by an aay of integes. B Geneic categoical dependent (esponse) vaiable. Its categoies ae indexed by an aay of integes. Numbe of categoies of B. c Numbe of categoies of A. p Numbe of nonedundant (nonaliased) paametes. i Geneic index fo the categoy of B. j Geneic index fo the categoies of A. k Geneic index fo the paamete. n Obseved count in the ith esponse of B and the jth setting of A. N j N m Maginal total count at the jth setting of A. It is equal to n. Total obseved count. It is equal to n. i Expected count.

2 GENLOG Multinomial Loglinea and Logit Models π z α j β k β x k Pobability of having an obsevation in the ith esponse of B and the jth setting of A. 0 π and π. i Cell stuctue value. jth nomalizing constant. kth nonedundant paamete A vecto of β, K, β p An element in the ith ow and the kth column of the design matix fo the j setting. The same notation is used fo both loglinea and logit models so that the methods ae pesented in a unified way. Conceptually, one can conside a loglinea model as a special case of a logit model whee the explanatoy vaiable has only one level (that is, c ). Components of the Model Thee ae two components in a loglinea model: the andom component and the systematic component. Random Component The andom component descibes the joint distibution of the counts. B at the jth setting of A have the multinomial 3N j, π j, K, π j 8 distibution. The counts n j, K, nj The counts n and n ae independent if j j.

3 GENLOG Multinomial Loglinea and Logit Models 3 The joint pobability distibution of n B is the poduct of these c independent multinomial distibutions. The pobability density function is c N j! n π () n! The expected count is E3n8 m Njπ. The covaiance is 4 9 &K 4 9 N if cov n n jπ δ j j, ii πi j 'K 0 if j j whee δ if a band δ 0 if a b. ab ab Systematic Component The systematic component descibes the linkage function between the expected counts and the paametes. The expected counts ae themselves functions of othe paametes. Explicitly, fo i, K, and j, K, c, m whee &K 'K α j z e v if z 0 if z 0 > 0 p v xk β k k SPSS does not conside α to α c as paametes, but as nomalizing constants.

4 4 GENLOG Multinomial Loglinea and Logit Models Nomalizing Constants N j α j log j, K, c () v ze i Cell Stuctue Values The cell stuctue values play two oles in SPSS loglinea pocedues, depending on thei signs. If z > 0, it is a usual weight fo the coesponding cell and log3z 8is sometimes called the offset. If z 0, a stuctual zeo is imposed on the cell 0B i, A j5. Contingency tables containing at least one stuctual zeo ae called incomplete tables. If n 0 but z > 0, the cell 0B i, A j5 contains a sampling zeo. Although SPSS still consides a stuctual zeo pat of the contingency table, it is not used in fitting the model. Cellwise statistics ae not computed fo stuctual zeos. Maximum-Likelihood Estimation The multinomial log-likelihood is L β L β, K, β p constant n log m (3) Likelihood Equations It can be shown that L β k 3n m8xk fo k, K, p

5 GENLOG Multinomial Loglinea and Logit Models Let g β g β g β, K, p be the p 6 gk β L β k 0 5 gadient vecto with The maximum-likelihood estimates $ β $ β,, $ K β p the vecto of likelihood equations: t 4 9 ae egaded as a solution to g6 β 0 (4) Hessian Matix The likelihood equations ae nonlinea functions of β. Solving them fo $ β equies an iteative method. The Newton-Raphson method is used. It can be shown that L β β k t 3 k jk83 l jl8 m x θ x θ whee θ jk m xk j c k p N, K, and, K, (5) j Let H6 β be the p p infomation matix, whee H6 β is the Hessian matix of (3). The elements of H6 β ae L hkl6 β k, K, pand l, K, p (6) β k β l Note: H6 β is a symmetic positive-definite matix. The asymptotic covaiance matix of $ β is estimated by H β 6.

6 6 GENLOG Multinomial Loglinea and Logit Models Newton-Raphson Method 05 Let β s denote the sth appoximation fo the solution to (4). By the Newton- Raphson method, 0 s 5 05 s 05 s 05 s β β H β g β Define q6 β H6 β β g6 β. Using (5) again, the kth element of q6 β is qk β η xk θ jk (7) whee &K 'K 3 8 if and m η v n m z > 0 m > 0 0 othewise Then H β s β s q β s (8) 05 Thus, given β s, the 0s 5th appoximation β s of equations in (8). 0 5 is found by solving the system

7 GENLOG Multinomial Loglinea and Logit Models 7 Initial Values 05 SPSS uses the β 0, which coesponds to a satuated model as the initial value fo β. Then the initial estimates fo the expected cell counts ae ' 05 0 n if z > 0 m & 0 if z 0 whee 0 is a constant. Note: Fo satuated models, SPSS adds to n if z > 0. This is done to avoid numeical poblems in case some obseved counts ae 0. We advise uses to set to 0 wheneve all obseved counts (othe than stuctual zeos) ae positive. The initial values fo othe quantities ae (9) θ jk m xk N (0) j and &K 'K m η log m / z n m if z and m > 0 > 0 othewise () Stopping Citeia SPSS checks the following conditions fo convegence:. max 0s 5 05 s 05 s, m m / m < ε povided that m s 05 > 0 49 < s. max, m ε s m < p 3. g $ k β / p k ε

8 8 GENLOG Multinomial Loglinea and Logit Models The iteation is said to be conveged if eithe conditions and 3 o conditions and 3 ae satisfied. If p 0, then condition 3 will be automatically satisfied. The iteation is said to be not conveged if neithe pai of conditions is satisfied within the maximum numbe of iteations. Algoithm The iteation pocess uses the following steps: Calculate m using (9), θ jk. Set s Calculate H β s s (7) evaluated at n n 4. Solve fo β s 05 using (0), and n using (6) evaluated at m m using (8). 0s 5 p 0s 5 5. Calculate v xkβ k k and v0s 5 s v0 5 0s 5 N z e z e z m j / tj > &K if 0 t 0 if z 0 'K 05 using () ; calculate q β s s 4 9 using 6. Check whethe the stopping citeia ae satisfied. If yes, stop iteation and declae convegence. Othewise continue. 7. Incease s by and check whethe the maximum iteation has been eached. If yes, stop iteation and declae the pocess not conveged. Othewise epeat steps 3-7.

9 GENLOG Multinomial Loglinea and Logit Models 9 Estimated Nomalizing Constants Using (), the maximum-likelihood estimate fo α j is N j $ α j log j, K, c v$ ze whee p v$ x $ k β k k Estimated Cell Counts The estimated expected count is m$ &K 'K Nj v$ $ vtj ze / ztje t if z > 0 0 if z 0 Goodness-of-Fit Statistics The Peason chi-squae statistic is X X

10 0 GENLOG Multinomial Loglinea and Logit Models whee X & K3 8 if and ' K n m$ / m$ z > 0, n > 0, m$ > 0 SYSMIS if z > 0, n > 0, and m$ 0 0 if z 0 o n m$ If any X is system missing, then X is also system missing. The likelihood-atio chi-squae statistic is G G whee G & K ' K if and n log n / m$ z > 0, n > 0 m$ > 0 SYSMIS if z > 0, n > 0 and m$ 0 0 if z > 0, n 0, and m$ 0; z 0 o n m$ If any G is system missing, then G is also system missing. Degees of Feedom 0 5, whee E The degees of feedom fo each statistic is defined as a p E is the numbe of cells with z 0 o m$ 0. Significance Level The significance level (o the p value) fo the Peason chi-squae statistic is 4 9 and that fo the likelihood-atio chi-squae statistic is Pob χ a > X Pob4 χ a > G 9. In both cases, χ a is the cental chi-squae distibution with a degees of feedom.

11 GENLOG Multinomial Loglinea and Logit Models Analysis of Dispesion (Logit Models Only) SPSS povides the analysis of dispesion based on two types of dispesion: entopy and concentation. The following definitions ae used: S(A) S(B A) S(B) RS(A)/S(B) Dispesion due to the model Dispesion due to esiduals Total dispesion Measue of association whee S A SBA SB. Also define $ π i c m$ c N j $ π m$ N j The bounds ae 0 $ πi and 0 $ πi j. Entopy SB NSi B whee 0 5 & 6 Si B i i < i ' $ π log π if 0 $ π 0 if $ π i 0

12 GENLOG Multinomial Loglinea and Logit Models and 6 6 SBA NjS BA Concentation whee $ log $ $ SB A6 3 8 < &K $ 'K π π if 0 π 0 if π 0 05 Degees of Feedom SB N $π i c SBA 6 N j $ π i j Souce of Dispesion Measue Degees of Feedom Due to model S(A) f Due to esiduals S(B A) N f Total S(B) N whee f equals p minus the numbe of nonedundant columns (in the design matix) associated with the main effects of the dependent factos.

13 GENLOG Multinomial Loglinea and Logit Models 3 Residuals Goodness-of-fit statistics povide only boad summaies of how models fit data. The patten of lack of fit is evealed in cell-by-cell compaisons of obseved and fitted cell counts. Simple Residuals The simple esidual of the (i,j)th cell is & ' n m$ if z > 0 SYSMIS if z 0 Standadized Residuals The standadized esidual fo the (i,j)th cell is & K SYSMIS ' K n m$ / m$ m$ / N if z > 0 and 0 < m$ < N 0 if z > 0 and n m$ j j S othewise The standadized esiduals ae also known as Peason esiduals even though 4 S 9 X. Although the standadized esiduals ae asymptotically i nomal, thei asymptotic vaiances ae less than. Adjusted Residuals The adjusted esidual is the simple esidual divided by its estimated standad eo. Its definition and applications fist appeaed in Habeman (973) and eappeaed on page 454 of Habeman (979). This statistic fo the (i,j)th cell is

14 4 GENLOG Multinomial Loglinea and Logit Models K3 8 if and ' n m$ / s z > 0 m$ > 0 A & 0 if z > 0 and n m$ KSYSMIS othewise whee p p m$ kl s m$ m$ xk $ jk xl $ jl h N 4 θ 94 θ 9 j k l h kl is the (k,l)th element of H 49. $ β The adjusted esiduals ae asymptotically standad nomal. Deviance Residuals Piece and Schafe (986) and McCullagh and Nelde (989) define the signed squae oot of the individual contibution to the G statistic as the deviance esidual. This statistic fo the (i,j)th cell is 3 8 D sign n m$ d whee d & K ' K if and n log n / m$ n m$ z > 0, m$ > 0, n > 0 m$ if z > 0, m$ 0, and n 0 0 if z > 0 and n m$ SYSMIS othewise Fo multinomial sampling, the individual contibution to the G statistic is only n log 3n / m$ 8, but this is negative when n < m$. Thus, an exta tem 3 8 is added to it so that d > 0 fo all i and j. Howeve, we still have n m$ 4 D 9 G. i

15 GENLOG Multinomial Loglinea and Logit Models 5 Genealized Residual Conside a linea combination of the cell counts dn, whee d ae i eal numbes. The estimated expected value is d m$ The simple esidual fo this linea combination is d n m$ 3 8 The standadized esidual fo this linea combination is c c d n m$ d m$ dm Nj / 3 8 The adjusted esidual fo this linea combination is, as given on page 40 of Habeman (979), 3 8 d n m $ i V

16 6 GENLOG Multinomial Loglinea and Logit Models whee p p j i j j i k l V d m $ dm N $ f f h k l kl fk dm$ xk θ ik 3 8 Genealized Log-Odds Ratio Conside a linea combination of the natual logaithm of cell counts d log3m8 () whee d ae eal numbes with the estiction d j, K, c The quantity in () is estimated by p d log m$ d log z d x $ k k k β (3)

17 GENLOG Multinomial Loglinea and Logit Models 7 The vaiance of (3) is p p kl 3 8 (4) k l va d log m$ wkwlh whee wk d xk k, K, p Wald Statistic The null hypothesis is H0: d log3m8 0 The Wald statistic is W c p k 3 8 d log m$ p kl wwh k l l Unde H 0, W asymptotically distibutes as a chi-squae distibution with degee of feedom. The significance level is Pob4χ W9. Note: W will be system missing if (4) is 0.

18 8 GENLOG Multinomial Loglinea and Logit Models Asymptotic Confidence Inteval The asymptotic α confidence inteval fo () is p p kl d log 3m$ 8± zα / wkwlh k l whee z α / is the uppe α / point of the standad nomal distibution. The default value of α is Aggegated Data This section shows how data ae aggegated fo a multinomial distibution. The following notation is used in this section: 0 5 and A j 0j c5 v Numbe of SPSS cases fo B i i, K,, K, n s sth SPSS caseweight fo B iand A j s, K, v i 6 x s Covaiate z s Cell weight c s GRESID coefficient e s GLOR coefficient v Numbe of positive z s (cell weights) fo s v

19 GENLOG Multinomial Loglinea and Logit Models 9 The cell count is n &K 'K * ns if v > 0 s v 0 if v 0 o v 0 whee & ' ns if ns > 0 and zs > 0 ns 0 if ns 0 and zs > 0 and z * means summation ove the ange of s with the tems z s > 0. s v The cell weight value is & K ' K * nszs / n if n > 0 and v > 0 s v * zs / v if n 0 and v > 0 s v 0 if v if v 0 0 If no vaiable is specified as the cell weight vaiable, then all cases have unit cell weights by default.

20 0 GENLOG Multinomial Loglinea and Logit Models The cell covaiate value is x & K ' K * s v * s v s s n x / n if n > 0 and v > 0 x / v if n 0 and v > 0 s 0 if v 0 o v 0 The cell GRESID coefficient is c & K ' K * s v * s v s s n c / n if n > 0 and v > 0 c / v if n 0 and v > 0 s 0 if v o v 0 Thee ae no defaults fo the GRESID coefficients. The cell GLOR coefficient is e & K ' K * s v * s v s s n e / n if n > 0 and v > 0 e / v if n 0 and v > 0 s 0 if v 0 o v 0 Thee ae no defaults fo the GLOR coefficients.

21 GENLOG Multinomial Loglinea and Logit Models Refeences Agesti, A Categoical data analysis. New Yok: John Wiley & Sons, Inc. Chistensen, R Log-linea models. New Yok: Spinge-Velag. Habeman, S. J The analysis of esiduals in coss-classified tables. Biometics, 9: Habeman, S. J Analysis of qualitative data, Volume. New Yok: Academic Pess. McCullagh, P., and Nelde, J. A Genealized linea models. nd ed. London: Chapman and Hall. Piece, D. A., and Schafe, D. W Residuals in genealized linea models. Jounal of the Ameican Statistical Association, 8: