Chapter 1. Modeling Basics

Transcription

1 Chapter 1. Modeling Basics What is a model? Model equation and probability distribution Types of model effects Writing models in matrix form Summary 1

2 What is a statistical model? A model is a mathematical description of the processes we think give rise to the observations in a set data. Minimal elements for statistical model Observation Systematic part describes the presumed impact of explanatory variables (observation mean). Random part describes the probability distributions associated with aspects of the process we assume to be characterized by random variation (the distribution of observations). 2

3 Explanatory variable Fixed effect - Parameter of interest (mean, beta coefficient) Random effect - repeated measurement - multi-level design Response Gaussian data Non-Gaussian data - Categorical : binomial/multinomial - Count : Poisson/negative binomial - Continuous : lognormal, proportion, beta - Time to event 3

4 From Linear model to generalized linear mixed model 4

5 Acronym LM: linear model GLM: generalized linear model LMM: linear mixed model GLMM: generalized linear mixed model SAS Procedure Response Gaussian Non-Gaussian Fixed Only LM PROC GLM + All others GLM PROC GENMOD PROC GLIMMIX Model Effects Mixed LMM PROC MIXED PROC GLIMMIX GLMM PROC GLIMMIX 5

6 Example I Cell means model y e ij i ij where - i = 1,2 denotes treatment - j = 1,, n i denotes the number of observations on the i-th treatment - y ij denotes the j-th observation on the i-th treatment - μ i denotes the mean of the i-th treatment - e ij denotes random error / random variation μ i : observation mean - systematic part e ij : random part typically we assumes iid N(0, σ 2 ). 6

7 Example II Linear regression y X e ij 0 1 i ij where - β 0 : the intercept - β 1 : the slope - X i : the value of predictor β 0 + β 1 X i : observation mean - systematic part e ij : random part typically we assumes iid N(0, σ 2 ). 7

8 Remarks George Box All models are wrong but some are useful. i. There is much we do not know about each subject or we choose not to pursue. ii. Alternatively, we settle for approximating variation among these subjects using the Gaussian probability distribution. It is wrong but good enough for many situations. 8

9 Two model forms Model equation Basic form: observation = systematic part + random part See comparison of two mean (example I) and regression (example II) Cell mean vs effect model Cell mean model: ij i ij y e Effect model: ij i ij y e where 0 i Probability distribution form As long as we assume a Gaussian distribution, example I can be expressed as y N 2 ~ (, ) ij i i 9

10 Weakness of the model equation form Example We observe N ij number of 0/1 or success / failure for y ij. Assumption: y ~ B( N, ) ij ij i where π i : the probability of a success It cannot be modeled by model equation approach 10

11 Analysis with linear regression - p y N X e e N 2 /, ~ (0, ) ij ij ij 0 1 i ij ij i. Histograms of simulated data from repeated sampling of a binomial distribution with N = 100 and π between 0.1 and 0.9 are virtually indistinguishable from the normal distribution, and in such a case, estimates of β from this approach are close to the coefficient of logistic regression. ii. However for binomial distribution the variance of response variable should depend on the probability of success but the linear regression always generates the same variance. iii. ˆ and ˆ If X 0 1 i is 0, pˆ (it cannot be negative!!) i 11

12 - Transformation with sin -1 ( / y N ) ij ij i. If we know we have a binomial response variable, we would do better to deal with it as such and not try to force it to be normal when we know it is not. ii. Therefore we prefer the probability distribution form. 12

13 SAS program for linear regression - SAS program data example1; ; input obs x N propotion=y/n; cards; run; proc genmod data=example1; run; model proportion = x /dist=normal link=identity; - SAS output 13

14 Modeling binomial data with probability distribution form Inspect the probability distribution Example: logistic regression (generalized linear model) (i) y ~ B( N, ) : random component ij ij ij log X 1 ij (ii) 0 1 ij i : link function & systematic component For logistic regression, ˆ 4.109, ˆ What is ˆ if X i = 10? cor(observed,logit) = cor(observed,p_glm) = Fit with logistic regression is better!! 14

15 SAS Program for logistic regression - SAS Program data example2; set example1; success=y; run; proc genmod data=example2; run; model success/n = x / dist=bin link=logit; - SAS Output 15

16 Define a plausible process Example : probit regression We never see the process and we only see its consequences (liability threshold model). (i) If the process exceeds a certain value, we observe a failure, and otherwise a success. We denote the boundary between success and failure by η : η = β 0 + β 1 X. (ii) We assume that the unobservable process has a standard normal distribution. The probability of success at X can be modeled by ( X ) i ( i ) 0 1 X i i 16

17 Generalized linear model - Identify the probability distribution of the observed data. - Focus on modeling the expected value, E( y ) N. If N ij is known, we focus only on π i. ij ij i - State the linear predictor - Identify the function that connects the expected value to the linear predictor (link function). In this lecture, we focus on the probability distribution model. 17

18 Provide the probability distribution model for linear regression (example II) Consider the two-treatment paired comparison. The response variable is continuous and can be assumed to have a Gaussian distribution. Consider the two-treatment paired comparison. For the i-th treatment (i = 1,2) on the j-th pair (j = 1,, p) at the k-th time point (k=1,2), N ijk observations are taken and each observation has either a favorable or unfavorable outcome. Denote y ijk as the number of favorable outcomes observed on the ij-th pair at the k-th time point. 18

19 Types of model effects Classification variable Classification variable: in the two-treatment mean comparison model (Model I), predictor variable is a treatment and it is an example of a classification variable. Direct variable: if is a function of X i i as was in linear regression (Model 2), predictor variable is a direct variable. 19

20 Example We assume that levels of X are observed at multiple locations or for multiple batches, and there is only one observation per level of X. There are 11 levels of X in varying intervals from 0 to 48. For each of four batches, a continuous variable (Y) and a binomial variable are observed at each level of X. Batch1 Batch3 Global mean Batch4 Batch2 20

21 Assuming separate linear regressions by batch: X (i = 1,, 4) can be alternatively expressed as b ( b ) X where β 0i 1i ij 0 0i 1 1i ij 0 and β 1 are the overall intercept and slope. - Random effect: batches could represent a sample of a larger population of batches and we could have sampled ay four batches. Then b N and 2 ~ (0, ) 0i 0 b N. 2 ~ (0, ) 1i 1 - Fixed effect: four batches in this data set could be the entire population, and batch i really means supplier i. For a linear regression (example II), state the mixed effects model. For a multi batch data with binomial distribution, state the final model. 21

22 Fixed/random effects model Definition - Fixed effects model: models that contain only fixed effects - Mixed effects model: models that contain both fixed and random effects. If we assumed random batches effect, you must state assumed probability distributions. For instance, b 2 0i 0 01 ~ MVN 0, 2 b 1i 01 1 Remarks Both random effect and response variable can have any plausible distribution. However computational methods for mixed models with Gaussian random effects are better developed, and we focus on the Gaussian case in this lecture. 22

23 Complete description of models discussed in this chapter TABLE 1.3: Complete Description of Models Discussed in This Chapter Type of Model Distribution Linear Predictor Link 1. LM ) or equivalently Identity: 2. LM Identity: 3. GLM or equivalently Logit: Or probit: = 4. GLM Logit or probit 5. LM Identity: 6. GLM Logit or probit 7. LMM Identity: 8. GLMM Logit or probit 23

24 Writing models in matrix form Matrix form is important for the theoretical development of estimation, inference, and statistical computing programs. Fixed-effect models We consider the two-treatment LM and suppose there are three observations on each treatment , , , , If we define (, 1, 2) t and X ( X, X ), define each term. X matrix is often called a design matrix. It is also called derivative matrix because X η η η

25 Consider the linear regression over levels of X ij = 0,, 10. We assume that 1 0, 1 0 1,, Define the design matrix and the parameter vector. Consider the fixed effects model for multi-batch models with ij 0 b0 i ( 1 b1 i) Xij and find a matrix form. 25

26 Mixed-effect models Consider the mixed effects model for multi-batch models with ij 0 b0 i ( 1 b1 i) Xij and find a matrix form. 26

27 Summary Typology of linear models Type of Model GLMM LMM GLM LM Observations Link Linear Predictor Mean Model 1 1 y b ~ G(, R) g( b) X Zb, b ~ N(0, G) ˆ g ( ˆ ) g ( X ˆ Zbˆ ) y b ~ N(, R) X Zb, b ~ N(0, G) ˆ X ˆ Zbˆ 1 1 y ~ G(, R) g( ) X ˆ g ( ˆ ) g ( X ˆ ) y ~ N(, R) X ˆ X ˆ GLMM : generalized linear mixed model, LMM : linear mixed model GLM : generalized linear model, LM : linear model LM, GLM, and LMM are all special cases of the GLMM 27

28 SAS procedure PROC GLM : General linear model (not generalized linear model) PROC ANOVA : Analysis of variance PROC LMM (extension of ANOVA) : Linear mixed model PROC LOGISTIC/GENMOD/CATMOD : Generalized linear model PROC GLIMMIX : Generalized linear mixed model CATMOD vs GENMOD vs LOGISTIC vs PROBIT - CATMOD: provides maximum likelihood estimation for logistic regression, including the analysis of logits for dichotomous outcomes and the analysis of generalized logits for 28

29 polychotomous outcomes. It provides weighted least squares estimation of many other response functions, such as means, cumulative logits, and proportions, and you can also compute and analyze other response functions that can be formed from the proportions corresponding to the rows of a contingency table. In addition, a user can input and analyze a set of response functions and user-supplied covariance matrix with weighted least squares. With the CATMOD procedure, by default, all explanatory (independent) variables are treated as classification variables. - GENMOD: a general statistical modeling tool which fits generalized linear models to data: it fits several useful models to categorical data including logistic regression, the proportional odds model, and Poisson regression. The GENMOD procedures also provides a facility for fitting generalized estimating equations to correlated response data that are categorical, such as repeated dichotomous outcomes. The GENMOD procedure fits models using maximum likelihood estimation, and you include classification variables in your models with a CLASS 29

30 statement. PROC GENMOD can perform type I and type III tests, and it provides predicted values and residuals. - LOGISTIC: specifically designed for logistic regression. For dichotomous outcomes, it performs the usual logistic regression and for ordinal outcomes, it fits the proportional odds model. Note that any polychotomous response variable will be treated as an ordinal outcome by PROC LOGISTIC. This procedure has capabilities for a variety of model-building techniques, including stepwise, forward, and backwards selection. It produces predicted values and can create output data sets containing these values and other statistics including ROC, and it produces a number of regression diagnostics. - PROBIT: designed for quantal assay or other discrete event data. It performs logistic regression. This procedure includes a CLASS statement. 30

31 31