Cohen s s Kappa and Log-linear Models

Transcription

1 Cohen s s Kappa and Log-linear Models HRP /03/ am

2 1. Cohen s Kappa Actual agreement = sum of the proportions found on the diagonals. π ii Cohen: Compare the actual agreement with the chance agreement (which depends on the marginals). π ii π i + π + i Normalize by its maximum possible value. 1 π ii π i + π π i + π + i + i

3 Example: rater agreement Rating by supervisor 2 Rating by supervisor 1 Authoritarian Democratic Permissive Totals Authoritarian Democratic Permissive Totals

4 Null hypothesis: Kappa=0 (no agreement beyond chance) Example: student teacher ratings Kˆ = 17 ( = = % CI = ( 13 ) ( * * * * * * 72 * * * 21 ) * 21 Interpretation: achieved 36.2% of maximum possible improvement over that expected by chance alone ) **See class handout for the formula for the asymptotic (large sample) variance of Kappa.

5 2. Log-Linear Linear Models for Multi-way Contingency Tables 1. GLM for Poisson-distributed data with log-link (see Agresti chapter 4). 2. Recall: log µ = α x µ = e α ( e β)x A one-unit increase in X has a multiplicative impact of e β on µ. 3. General idea: predict the expected frequency (count) in each cell by a product of effects main effects and interactions. 4. (Take logs to linearize).

6 Log-linear vs. logistic 1. The expected distribution of the categorical variables is Poisson, not binomial. 2. The link function is the log, not the logit. 3. Predictions are estimates of the cell counts in a contingency table, not the logit of y.

7 Log-linear vs. logistic The variables investigated by log linear models are all treated as response variables. Therefore, loglinear models only demonstrate association between variables (like chi-square or correlation coefficient). If clear explanatory and response variables exist, then logistic regression should be used instead. Also, if the variables are continuous and cannot be broken down into discrete categories, logistic regression is preferable.

8 Example: 3-way 3 contingency Heart Disease Total Body Weight Sex Yes No Not over weight Male Fe Total Over weight Male Fe Total Source: Angela Jeansonne

9 In class exercise: Analyze these data using methods we have already learned. Is gender related to heart disease and is this effect modified or confounded by weight? What s the relationship between eight and gender (controlled for ) and eight and heart disease (controlled for gender)?

10 Example: sex, weight, and heart disease Model 1 (main effects only): Log (counts) = α eight ismale HeartDisease proc genmod data=loglinear; model total = Overweight IsMale HeartDis / dist=poisson link=log pred ; run;

11 df = cells parameters in model χ 2 = Independence model: goodness-of of-fitfit Cells Observed Pred light//disease light//no disease light/fe/disease light/fe/no disease heavy//disease Suggests independen ce model is a poor fit!! heavy//no disease heavy/fe/disease heavy/fe/no disease

12 Independence model: parameters Standard Wald 95% Chi- Parameter DF Estimate Error Confidence Limits Square Intercept Overweight IsMale HeartDis Model 1: Parameter Pr > ChiSq Intercept <.0001 Overweight IsMale <.0001 HeartDis Log (counts) = (weight) 1.1 () -.30 (heart disease)

13 Interpretation of Parameters Model 1: Log (counts) = (weight) 1.1 () -.30 (heart disease) e -.41 = the (marginal) odds of being eight =.66= 80/120 e -1.1 = the odds of being =.33 = 50/150 e -0.3 = the odds of having disease=.74 = 85/115

14 Model with Interaction: Model 2 (main effects + some interactions): This model corresponds to case when heart disease and eight are conditionally independent (conditioned on gender). Log (counts) = α eight ismale HeartDisease + β ismale *β HeartDisease + β ismale * β eight proc genmod data=loglinear; model total = Overweight IsMale HeartDis ismale*heartdis ismale*overweight/ dist=poisson pred ; run; link=log

15 Analysis Of Parameter Estimates Model 2: Standard Wald 95% Parameter DF Estimate Error Confidence Limits Intercept Overweight IsMale HeartDis IsMale*HeartDis Overweight*IsMale Analysis Of Parameter Estimates Chi- Parameter Square Pr > ChiSq Intercept <.0001 Overweight <.0001 IsMale <.0001 HeartDis <.0001 IsMale*HeartDis <.0001 Overweight*IsMale Log (counts) = (weight) 2.4 () -.69 (heart disease) 1.54 (if and heartdis) (if eight and )

16 Interpretation of Parameters, Model 2: Model 2 Log (counts) = (weight) 2.4 () -.69 (heart disease) log( OR k = β k * ( α = β OR * 1.54 (if and heartdis) (if eight and ) µ 11µ 22 ) = log( ) = log µ 11 + log µ 22 log µ 12 log µ 21 = µ µ = eight :( α = * 1.54 = e = 12 = not eight :( α e β 21 * 4.66 ) ( α * ) + ( α) ( α * ) * ) ( α ) + ( α ) ) =

17 OR estimate from predicted counts Cells Observed Pred light//disease light//no disease 5 6 light/fe/disease light/fe/no disease heavy//disease heavy//no disease 10 9 χ 2 = heavy/fe/disease heavy/fe/no disease OR( k OR( k 14*66.6 = light) = = * *33.3 = heavy) = = *16.6 OR - is not confounded by weight

18 Male and Overweight Model 2: Log (counts) = (weight) 2.4 () -.69 (heart disease) log( 1.54 (if and heartdis) (if eight and ) µ 11µ 22 OR.) = log( ) = log µ 11 + log µ 22 log µ 12 log µ 21 = µ µ k = no :( α = β * k = ( α = β * OR = e :( α β * 1.1 = e = 12 over * * ) ( α ) + ( α) ( α * * ) ) ( α ) + ( α ) over ) =

19 OR estimate from predicted counts Cells Observed Pred light//disease light//no disease 5 6 light/fe/disease light/fe/no disease heavy//disease heavy//no disease 10 9 heavy/fe/disease heavy/fe/no disease OR( k OR( k 21*33.3 = ) = = *16.6 9*66.6 = no ) = = *33.3 OR -eight is not confounded by

20 Interpretation: Model 2 Overweight and heart-disease are independent when you condition on gender. Heart Disease Men Yes No Overweight 21 9 normal 14 6 OR=21*6/14*9 =1.0 Women Overweight normal OR=16.6*33.3/33.3*33.3 =1.0

21 Model 3: only and are related Model 2 (main effects + single interaction): This model corresponds to case when heart disease and eight and gender and eight are conditionally independent. Log (counts) = α eight ismale HeartDisease + Output Model 3: β ismale *β HeartDisease Log (counts) = (weight) 1.9 () -.69 (heart disease) 1.54 (if and heartdis)

22 OR: Male and CHD Model 3: Log (counts) = (weight) 1.9 () -.69 (heart disease) 1.54 (if and heartdis) µ 11µ 22 log( OR ) = log( ) = log µ 11 + log µ µ µ k = no eight :( α = β * k = eight :( α ( α = β * OR = e β * 1.54 = e = ) ( α 4.66 ) * 22 log µ log µ ) + ( α) ( α * 12 ) + ( α 21 ) ( α ) = ) =

23 Model 3: only and are related Cells Observed Pred light//disease light//no disease 5 9 light/fe/disease light/fe/no disease heavy//disease heavy//no disease 10 6 heavy/fe/disease heavy/fe/no disease 40 40

24 Collapses to Male Fe CHD No CHD OR = 35*100 50*15 = 4.66

25 And heart disease and eight are independent, regardless of gender Overweight light CHD No CHD OR = 34*69 46*51 = 1.00

26 And eight and gender are independent, regardless of disease Overweight light Male Fe OR = 20*90 60*30 = 1.00

27 proc genmod data=loglinear; model total = Overweight IsMale HeartDis ismale*heartdis ismale*overweight Overweight*HeartDis / dist=poisson link=log pred ; run; M4: All pair-wise interactions Model 4 (main effects +all pairwise interactions): No pair of variables is conditionally independent. Log (counts) = α eight ismale β ismale *β HeartDisease + β ismale * β eight + HeartDisease β HeartDis * β eight

28 Model 4: Standard Wald 95% Parameter DF Estimate Error Confidence Limits Intercept Overweight IsMale HeartDis IsMale*HeartDis Overweight*IsMale Overweight*HeartDis Analysis Of Parameter Estimates Chi- Parameter Square Pr > ChiSq Intercept <.0001 Overweight IsMale <.0001 HeartDis IsMale*HeartDis <.0001 Overweight*IsMale Overweight*HeartDis Log (counts) = (weight) 2.7 () -.45 (heart disease) 1.8 (if and heartdis) (if eight and )-.82 (if over and heartdis)

29 OR: Male and CHD Model 4: Log (counts) = (weight) 2.7 () -.45 (heart disease) 1.8 (if and heartdis) (if eight and )-.82 (if over and heartdis) log( OR = β * ( α = β OR * = e µ 11µ ) = log( µ µ k = eight :( α * 1.8 = e = 12 * k = not eight :( α β ) = log µ 11 ) ( α + log µ 22 * log µ log µ ) + ( α) ( α * 12 * * ) 21 = ) ( α * ) = ) + ( α Corresponds to the M-H summary OR, stratified by eight )

30 Corresponds to the M-H summary OR, stratified by gender OR: CHD and eight Model 4: Log (counts) = (weight) 2.7 () -.45 (heart disease) 1.8 (if and heartdis) (if eight and )-.82 (if over and heartdis) OR = e β * eight = e.82 =.42

31 OR: and eight Model 4: Log (counts) = (weight) 2.7 () -.45 (heart disease) 1.8 (if and heartdis) (if eight and )-.82 (if over and heartdis) OR β * = e eight = e 1.4 = 4.1 Corresponds to the M-H summary OR, stratified by

32 OR estimate from predicted counts Cells Observed Pred light//disease light//no disease 5 4 light/fe/disease light/fe/no disease heavy//disease heavy//no disease heavy/fe/disease heavy/fe/no disease χ 2 = 1.571

33 The saturated model Model 5 (saturated): Log (counts) = α eight ismale β ismale *β HeartDisease + β ismale * β eight + β HeartDis * β eight + HeartDisease β ismale *β HeartDisease * β eight Perfect fit no degrees of freedom.