Introduction to General and Generalized Linear Models

Transcription

1 Introduction to General and Generalized Linear Models Generalized Linear Models - part III Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby October 2010 Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

2 Today Test for model reduction Inference on individual parameters Confidence intervals Example Odds ratio Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

3 Test for model reduction Test for model reduction The principles for model reduction in generalized linear models are essentially the same as the principles for classical GLM s. In classical GLM s the deviance is calculated as a (weighted) sum of squares, and in generalized linear models the deviance is calculated using the expression for the unit deviance. Besides this, the major difference is that instead of the exact F -tests used for classical GLM s the tests in generalized linear models are only approximate tests using the χ 2 -distribution. In particular does the principles of successive testing in hypotheses chains using a type I, or type III partition of the deviance carry over to generalized linear models. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

4 Test of individual parameters β j Test of individual parameters β j Theorem (Test of individual parameters β j - Wald test) A hypothesis H : β j = βj 0 related to specific values of the parameters is tested by means of the test statistic u j = ˆβ j β 0 j σ 2 σ jj, where σ 2 indicates the estimated dispersion parameter (if relevant), and σ jj denotes the j th diagonal element in Σ. Under the hypothesis is u j approximately distributed as a standardized normal distribution. The test statistic is compared with quantiles of a standardized normal distribution (some software packages use a t(n k) distribution). The hypothesis is rejected for large values of u j. The p-value is found as p = 2 ( 1 Φ( u j ) ). Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

5 Test of individual parameters β j Test of individual parameters β j Theorem (Test of individual parameters β j - Wald test) In particular is the test statistic for the hypothesis H : β j = 0 u j = ˆβ j σ 2 σ jj. An equivalent test is obtained by considering the test statistic z j = u 2 j and reject the hypothesis for for z j > χ 2 1 α (1). Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

6 Confidence intervals Confidence intervals Wald - interval for individual parameters An approximate 100(1 α) % Wald-type confidence interval is obtained as β j ± u 1 α/2 σ 2 σ jj Confidence intervals for fitted values An approximate 100(1 α)% confidence interval for the linear prediction is obtained as η i ± u 1 α/2 σ 2 σ ii with σ ii denoting the i th diagonal element in XΣX T. The corresponding interval for the fitted value µ i is obtained by applying the inverse link transformation g 1 ( ) to the confidence limits. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

7 Example: Link functions for binary response regression An experiment testing the insulation effect of a gas (SF 6 ) was conducted. In the experiment a gaseous insulation was subjected to 100 high voltage pulses with a specified voltage, and it was recorded whether the insulation broke down (spark), or not. After each pulse the insulation was reestablished. The experiment was repeated at twelve voltage levels from 1065 kv to 1135 kv. Voltage (kv) Breakdowns Trials Voltage (kv) Breakdowns Trials Table: The insulation effect of a gas (SF 6 ) Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

8 Example: Link functions for binary response regression As the insulation was restored after each voltage application it seems reasonable to assume that the trials were independent. At each trial the response is binary (Breakdown/Not), and therefore it seems appropriate to use a binomial distribution model for the experiment. We shall assume that the data are stored in an R object dat with the variables Volt, Breakd, Trials Let Z i denote the number of breakdowns at the i th trial at the voltage x i. We shall then use the model Z i B(n i, p i ) with n i = 100, and p i = p(x i ), where p(x) is some suitable dose-response function. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

9 Logit transformation - logistic regression The logistic regression is of the form ( ) p g(p) = η = ln = β 1 + β 2 x 1 p p(x) = exp(η) 1 + exp(η) = exp(β 1 + β 2 x) 1 + exp(β 1 + β 2 x) We use the following commands to fit the model: dat$resp<-cbind(breakd,(trials-breakd)) fit1<-glm(resp~volt,family=binomial(link=logit),data=dat) Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

10 Logit link > summary(fit1) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e e <2e-16 *** Volt 1.155e e <2e-16 *** --- (Dispersion parameter for binomial family taken to be 1) Null deviance: on 11 degrees of freedom Residual deviance: on 10 degrees of freedom AIC: Number of Fisher Scoring iterations: 4 Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

11 Logit link From the output we can make a deviance table: Source f Deviance Mean deviance Model H M Residual (Error) Corrected total The p-value corresponding to the goodness of fit statistic D(y; µ( β)) = is assessed by calculating > pval <- 1- pchisq( ,10) leading to pval = Thus, H logist is rejected at any significance level greater than 2 %. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

12 Logit link Also, a look at the deviance residuals: residuals(logist.glm) They indicate underestimation in the tails, and overestimation in the central part of the curve. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

13 The probit link The transformation g(p) = η = Φ 1 (p) = β 1 + β 2 x p(x) = Φ(η) = Φ(β 1 + β 2 x) with Φ( ) denoting the cumulative distribution function for the standardized normal distribution is termed the probit-transformation. The function tends towards 0 and 1 for x, respectively. The convergence is faster than for the logistic transformation. There is a long tradition in biomedical literature for using the probit transformation. We fit the model with: fit2<-glm(resp~volt,family=binomial(link=probit),data=dat) Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

14 The probit link > summary(fit2) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) <2e-16 *** Volt <2e-16 *** --- (Dispersion parameter for binomial family taken to be 1) Null deviance: on 11 degrees of freedom Residual deviance: on 10 degrees of freedom AIC: Number of Fisher Scoring iterations: 5 Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

15 The probit link From the output we can make a deviance table: Source f Deviance Mean deviance Model H M Residual (Error) Corrected total The p-value corresponding to the goodness of fit statistic D(y; µ( β)) = is assessed by calculating > pval <- 1- pchisq(26.215,10) leading to pval = Thus, H probit is rejected at any significance level greater than 0.3 %. The fit is not satisfactory. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

16 The probit link Again, a look at the deviance residuals: residuals(prob.glm) They indicate systematic underestimation in both tails, and overestimation in the central part. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

17 Complementary log-log link The transformation g(p) = η = ln( ln(1 p)) = β 0 + β 1 x p(x) = 1 exp[ exp(β 0 + β 1 x)] is termed the complementary log-log transformation. The response function is asymmetrical. It increases slowly away from 0, whereas it approaches 1 in a rather steep manner. We fit the model with: fit3<-glm(resp~volt,family=binomial(link=cloglog),data=dat) Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

18 Complementary log-log link > summary(fit3) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) <2e-16 *** Volt <2e-16 *** --- (Dispersion parameter for binomial family taken to be 1) Null deviance: on 11 degrees of freedom Residual deviance: on 10 degrees of freedom AIC: Number of Fisher Scoring iterations: 4 Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

19 Complementary log-log link From the output we can make a deviance table: Source f Deviance Mean deviance Model H M Residual (Error) Corrected total The p-value corresponding to the goodness of fit statistic D(y; µ( β)) = is assessed by calculating pval <- 1- pchisq(5.671,10), leading to pval = Thus, data do not provide any evidence against the cloglog model. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

20 Complementary log-log link This is further supported by the deviance residuals: residuals(clog.glm) There is no systematic pattern in the residuals, and all residuals are in the interval ±2. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22

21 Logit/probit/cloglog p(x) Logit Probit cloglog Figure: Probability of breakdown for an insulator as function of applied pulse voltage. The curves correspond to different assumptions on the functional form of the relation. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22 Volt

22 Odds Ratio Odds ratio If an event occurs with probability p, then the odds in favor of the event is Odds = p 1 p A comparison between two events can be made by computing the odds ratio: OR = p 1/(1 p 1 ) p 2 /(1 p 2 ) An odds ratio larger than 1 is an indication the event is more likely in the first group than in the second group. Henrik Madsen Poul Thyregod (IMM-DTU) Chapman & Hall October / 22