Generalized Linear Models

Transcription

1 Generalized Linear Models

2 Assumptions of Linear Model Homoskedasticity Model variance No error in X variables Errors in variables No missing data Missing data model Normally distributed error Error in Y variables is measurement error Observations are independent GLM

3 Generalized Linear Models Retains linear function Allows for alternate PDFs to be used in likelihood However, with many non-normal PDFs the range of the model parameters does not allow a linear function to be used safely Pois(l): l > 0 Binom(n,q) 0 < q < 1 Typically a link function is used to relate linear model to PDF

4 Link Functions Canonical Link Functions Distribution Link Name Link Function Mean Function Xb = m m = Xb Normal Identity Exponential Inverse Xb = m-1 m = (Xb)-1 Gamma Xb = ln(m) m = exp(xb) Poisson Log exp Xb Binomial = Logit Xb=ln Multinomial 1 exp Xb 1 Can use most any function as a link function but may only be valid over a restricted range Most are technically nonlinear functions

5 Poisson Regression Most common link is log(l) y~poisson log = X Commonly used to model count data Especially for low counts were poor normal approx Easily generalized to Negative Binomial regression (not canned) Flexible in alternative link functions (l > 0) Lab 4: cone counts

6

7 Likelihood in R Option 1 glm(y ~ x,family=poisson(link= log )) Option 2 lnl = function(beta){ -sum(dpois(y,exp(beta[0] + beta[1]*x),log=t)) }

8 DAIC

9 Poisson Regression y ~Pois exp X X Y Data Model b Process Model B 0,Vb Parameter Model

10 Bayesian Poisson Regression y~poisson log = X ~N B0, V B model{ for(i in 1:2) { beta[i] ~ dnorm(0,0.001)} ## no sigma for(i in 1:n){ log(mu[i]) <- beta[1]+beta[2]*x[i] y[i] ~ dpois(mu[i]) } }

11

12 Logistic Regression Common model for the analysis of boolean data (0/1, True/False, Present/Absent) Assumes a Bernoulli likelihood Bern(q) = Binom(1,q) Likelihood specification y~bern logit = X Data Model ~N B0, V B Parameter Model Process Model Bayesian

13 Logit Xb=ln 1 Interpretation: Log of the ODDS RATIO logit(0.5) = 0.0

14 Logistic Regression y ~Binom 1, logit X 1 X Y Data Model b Process Model B 0,Vb Parameter Model

15

16

17 Logistic Regression in R Option 1 built in function glm(y ~ x, family = binomial(link= logit )) Option 2 homebrew lnl = function(beta){ -dbinom(y,1,ilogit(beta[0] + beta[1]*x),log=t) }

18

19 Call: glm(formula = y ~ x, family = binomial()) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e-15 *** x < 2e-16 *** --Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 249 degrees of freedom Residual deviance: on 248 degrees of freedom AIC:

20

21 Alternative link functions probit Normal CDF cauchit - Cauchy CDF log -- cloglog - Complimentary log-log =exp X Asymmetric, often used for high or low probabilities =1 exp exp X If you code yourself, any function that projects from Real to (0,1)

22

23 Bayesian Logistic Regression model { ## priors for(i in 1:2) { beta[i] ~ dnorm(0,0.001)} for(i in 1:n){ logit(mu[i]) <- beta[1]+beta[2]*x[i] y[i] ~ dbern(mu[i]) } }

24

25 Logistic + Exponential: Tree Mortality Logistic model of annual survival probability logit = X Bernoulli likelihood t Normal prior Y ~Bern ~N B 0, V 0 Metropolis MCMC 50K steps, 6-20K burn-in, thin 1/3 Dietze and Moorcroft 2011 Global Change Biology

26 Distribution of Eastern Forest

27 Mortality Covariates Climate Landscape/abiotic Precipitation Elevation Summer Max Temp Slope Winter Min Temp Radiation index Topographic Moisture Air Pollutants Stand scale biotic NO3 deposition SO4 deposition DBH Ozone Stand Basal Area Stand Age

28 Annual Mortality Probability Binned Data Mean Binned Data CI Model w/ average covariates Binned model w/ full covariates Annual Precipitation (meters)

29

30

31 Survival Covariate Effects Climate PFT Early HW N Mid HW S Mid HW Late HW N.Pine S. Pine Mid Con Late Con Evergreen Hydric int Precip Tmin Pollution Tmax no ns ns + ns ns + so4 Stand O3 age ns ns ns - - ns ns BA + - Topography slope rad elev TCI - ns + - ns - ns ns + - ns - ns - ns - ns ns ns - ns ns + - ns ns - ns ns + ns ns ns ns - n 78, , ,437 78,635 18, ,476 10, ,397 7,891 22,352

32 Parameter Sensitivity Climate PFT Precip Tmin Early HW N Mid HW 1.82 S Mid HW Pollution Tmax Stand Topography no3 so4 NxS O3 age BA dia slope rad elev TCI MEAN Late HW N.Pine S. Pine Mid Con Late Con Evergreen Hydric MEAN i S i =1000 SD Y pred X

33 Logit-Normal Model Bernoulli likelihood only accounts for sampling error Can add additional extrabinomial variation to account for the fact that the covariates do not full account for the mean risk y~bern logit = X 2 ~N 0, Fairly sensitive to prior on sigma

34 Logit-Normal Regression model <- function(){ for(i in 1:2) { beta[i] ~ dnorm(0,0.01)} sigma ~ dgamma(1,0.1) for(i in 1:n){ Ey[i] <- beta[1]+beta[2]*x[i] mu[i] ~ dnorm(ey[i],sigma) logit(theta[i]) <- mu[i] y[i] ~ dbern(theta[i]) } }

35

36 Multinomial Regression Categorical variable as a function of covariates in a linear model Probability of falling within each group Categories can be ordered (ordinal) or unordered (nominal) Multivariate extension of the logistic regression to >2 categories cumulative logit model End up with K-1 regression models for K classes

37 Logistic Regression n y~binom 1, = y y n y 1 logit =log =X 1 Multinomial Regression n y~multinom 1, 1,, K = y y 1 K th For the k class: 1 k log = X k 1 1 k K k=1 yk k

38 Inverting the link function for the FIRST class exp X 1 1= 1 exp X 1 For the middle classes k 1 exp X k k = j 1 exp X k j=1 For the LAST class K 1 K =1 j j=1

39 Additional Restrictions Because the regression model for each of the K-1 categories is expressed in terms of the cumulative probability of all categories up to that one, these curves cannot cross each other Slopes and intercepts must be ordered b0,k < b0,k+1 and b1,k < b1,k+1 for all k Range restrictions can be accommodated in the prior using an indicator function I(A) I(A) = 1 if A is true and I(A) = 0 if A is false

40 Multinomial Regression Priors { { N b0,1, V b I 0,1 0,2 k=1 0, k ~ N b0, k, V b I 0, k 1 0, k 0, k 1 1 k K 1 N b0, K 1, V b I 0, K 2 0, K 1 k =K 1 N b1,1,v b I 1,1 1,2 k=1 1, k ~ N b1, k, V b I 1, k 1 1, k 1, k 1 1 k K 1 N b1, K 1, V b I 1, K 2 1, K 1 k=k 1

41

42 Multinomial JAGS code model{ beta[1, 1] ~ dnorm(0.0, 0.001) T(,beta[2, 1]) beta[2, 1] ~ dnorm(0.0, 0.001) beta[1, 2] ~ dnorm(0.0, 0.001) T(,beta[2, 2]) beta[2, 2] ~ dnorm(0.0, 0.001) for (i in 1:n) { logit(mu[i, 1]) <- beta[1, 1] + beta[1, 2] * x[i] logit(cmu2[i]) <- beta[2, 1] + beta[2, 2] * x[i] mu[i, 2] <- cmu2[i] - mu[i, 1] mu[i, 3] <- 1 - cmu2[i] y[i, ] ~ dmulti(mu[i, ], 1) } }

43

44

45 Assumptions of Linear Model Homoskedasticity Model variance No error in X variables Errors in variables No missing data Missing data model Normally distributed error GLM Error in Y variables is measurement error Observations are independent

46 Example Are banana slugs susceptible to soil mercury contamination? Data: Y = slug counts, n= 35 XO = soil mercury concentration ( mg/kg) Lost samples 9, 13, 27 Manufacture reports a 10% accuracy Further digging reveals this refers to a 95% CI which was calibrated on n=150 Graph? JAGS Code?