WU Weiterbildung. Linear Mixed Models
|
|
- Bruce Golden
- 5 years ago
- Views:
Transcription
1 Linear Mixed Effects Models WU Weiterbildung SLIDE 1
2 Outline 1 Estimation: ML vs. REML 2 Special Models On Two Levels Mixed ANOVA Or Random ANOVA Random Intercept Model Random Coefficients Model Intercept-and-Slopes-as-Outcomes Model Hierarchical Linear Model 3 Variance Structures 4 Testing Hypotheses for LMM 5 Predicting Values With LMM Best Linear Unbiased Estimators (BLUE) Best Linear Unbiased Predictors (BLUP) 6 Centering 7 Generalisations SLIDE 2
3 What Are Mixed Effect Models? Models with mixed effects, i.e. fixed and random effects Very similar mixed effects models are known under many different names Mixed (effect) models (statistics esp. biostatistics) Multilevel models (sociology) Hierarchical models (psychology) Random coefficient models (econometrics)... SLIDE 3
4 Types Of Mixed Effect Models We can distinguish between Linear mixed effect models (LMM) Linear models for metric variables with normal error structure t-test, ANOVA, Regression, ANCOVA,... Generalized linear mixed effect models (GLMM) Linear models for expected values of exponential families Linear models, logistic models, poisson models, gamma models,... Nonlinear mixed effect models Nonlinear models with nonlinear influence of the random effects GLMM, growth curve models We will only look at the first class of models. SLIDE 4
5 Linear Mixed Effect Models We will consider models with the following properties Normal error structure (i.e. the errors are normally distributed) The dependent variable is a linear combination of the explanatory variables The explanatory variables can have fixed or random effects or both More than one source of variation is present They are for data that need not be i.i.d. sampled from the same population SLIDE 5
6 Types Of Effects - I Fixed effects are the classic type of effects They influence the mean of the dependent variable They are usually of interest, hence inference is done for them They are controlled All levels or realisations that are of interest are included Usually the substantive theory refers to them Small number of levels for factors All levels/realisations under consideration are represented in the coding Examples: Experimental conditions (treatment/control), causal variables (education on income), control factors (age), uncontrollable conditions in quasi-experimental designs (gender),... SLIDE 6
7 Types Of Effects - II Random effects are the classy type of effects They influence the covariance structure of the dependent variable They are effects associated with the sampling We have only a random sample of all possible levels/realisations (in our coding) They influence how accurate our fixed effect model predictions are We are usually interested in their variance, not their actual values Usually a large number of possible levels/realisations Examples: Error, experimental conditions that are not fully realised (all chairs of a company), confounding variables, mediators, individuality of subjects,... If an effect is random or fixed is due to the study design! SLIDE 7
8 Nested Random Effects In Models (Hierarchically) Nested random effects arise if we have some type of clustering structure, either by design or because of our interest. The idea is that observations within a cluster are not independent Each random effect level is some kind of subpopulation Independence holds between clusters and their nested structues (i.e. subjects, realisations or other cluster) Can be multilevel, i.e. a cluster in a cluster in a cluster Very often the case in social science Can correct sampling errors to a certain degree Should be the method of choice for quasi-experiments We will mostly focus on this type of random effects. SLIDE 8
9 Crossed Random Effects In Models Crossed random effects (non-hierarchically nested) arise if there is no hierarchically nested structure, either by design or because of our interest. We are interested in a fixed effect but want to correct for possible sources of bias (or random fluctuations) Subjects belong to more than one factor that are (factorially) crossed If nesting is not strict but dynamic, crossed effects also arise (cross-over designs for example) Subjects may be cross-classified (partially crossed) Rarely the case in social science but potentially very useful SLIDE 9
10 Types Of Grouped Data Some examples of natural hierarchical data Longitudinal data Multicenter data Meta analysis Questionnaires Representative surveys data Block designs Panels Matched pairs SLIDE 10
11 Conceptual Differences LMM Vs. LM Classic Linear Models They have one source of randomness, the error The error structure is usually specified as independent identically distributed Sampling has to be totally random for all subjects Mixed Models They have more than one source of randomness, the random effects (including error) The errors no longer need to be independent identically distributed Sampling can be less restrictive They can account for heteroscedasticity SLIDE 11
12 Model Formulation - LM In matrix notation, the linear model looks like this with y = Xβ + ɛ, ɛ N (0, Σ). Here y is the dependent variable vector, X is a n p design matrix ɛ the vector of errors and β the fixed effect parameter vector of interest. Often, Σ is assumed to be σ 2 I with I being the n n indentity matrix. The distribution of y is then normal y N (Xβ, Σ) SLIDE 12
13 Model Formulation - LMM In matrix notation, the linear mixed model looks like this with y = Xβ + Zu + ɛ, ɛ N (0, Σ), u N (0, Ω) The notation is as before, and Z is a n q design matrix for the random effects, u is the vector of random effect parameters and Ω is the variance-covariance matrix of the random effects (in which we are primarily interested). The random vectors u and ɛ are independent. This leads to the distribution of y to be with V = ZΩZ + Σ. SLIDE 13 y N (Xβ, V)
14 Software For LMM - General SPSS offers the MIXED procedure to fit models as described above. In R we have a number of packages that can calculate these models, but lme4 or nlme are recommended. These procedures can Fit variance components, random coefficients, slopes-as-outcomes etc. models They need data to be in the long format (see below) Additional variables code the nesting structure SLIDE 14
15 Software For LMM - Data SPSS or R need a certain data structure. The data restructure wizard can do that for us in SPSS, in R the package reshape helps. Let us look at the didactic examples: Example 1 (firms): Growth of firm size measured at 9 occasions (longitudinal data) Example 2 (groceries): Amount spent on grocery items for different retailers (clustered data) Example 3 (testmarket): Coupon campaign of a retailer (multilevel clusters and longitudinal) Example 4 (bdf): Data on language scores of pupils (multilevel clusters and cross-section/longitudinal structure with many explanatory variables) Please try to restructure firmwide from wide to long format. You may want ot look at groceries to see how that should look. SLIDE 15
16 First Examples - Grocery Sales The groceries data are amount spent on grocery sales in different stores. 60 stores, 351 customers Various explanatory variables Let us calculate an overall mean model and an ANOVA with/without random effect. SLIDE 16
17 First Examples - Grocery Sales Amount spent Store ID SLIDE 17
18 First Examples - Employees In Firms Let us look at a regression model of the number of employees explained by the revenue for the firms data. Measured at 9 occasions Not necessarily measured at the same time/revenue level Revenue is centered Let us calculate a regression of employees on revenue with/without a random effect over the firms. SLIDE 18
19 Firm Employees - LM revenue employees SLIDE 19
20 Firm Employees - LM employees revenue SLIDE 20
21 Firm Employees - LMM employees revenue SLIDE 21
22 Estimation One can estimate LMM parameters (fixed effects, error, variances and covariances) in many different ways, either frequentist or Bayesian. Frequentist approaches Full Information Maximum Likelihood Restricted Maximum Likelihood Least Squares (ordinary or weighted) Bayesian MCMC approaches The two ML procedures are most popular. SLIDE 22
23 Estimation - FIML It uses the whole information of the joint likelihood of β, Ω and Σ and includes a penalty term for the magnitude of the random effects. It is a shrinkage estimator. The problem is that those estimates are biased since the estimates for the fixed effects have to be plugged in to calculate variance components and vice versa. This will reduce the degrees of freedom available but that is not accounted for. Intuitively: The FIML fixed effects are WLS estimates with the FIML variance components used for weighting. SLIDE 23
24 Estimation - REML This approach restricts the likelihood to error contrasts and maximizes that restricted (residual) likelihood, a function of Ω and Σ. This takes the loss of degrees of freedom into account and has therefore smaller bias. It can be shown that it is the FIML with an additional bias correction. The problem is that the fixed effects no longer occur in the restricted likelihood and model comparison with likelihood ratio tests can only be done if two models have the same fixed effect specification. Intuitively: The REML fixed effects are WLS estimates with the REML variance components used for weighting. SLIDE 24
25 ANOVA With Random Effect - I This is an ANOVA for which there is more than one random effect, the within group error and between group error. There are j = 1,..., J groups. y ij = µ + u j + ɛ ij It can be easier to specify such models on each level separately Level 1: y ij = β j + ɛ ij Level 2: β j = µ + u j This model is very useful for calculating the amount of variance that can be explained by higher level covariates (via the identity Var(y ij ) = Var(u j + ɛ ij ) = ω + σ 2 ) and the intraclass correlation as ω/(ω + σ 2 ). SLIDE 25
26 ANOVA With Random Effect- II We already calculated an example, the mean amount spent on groceries. Let us also calculate the IC correlation. SLIDE 26
27 Random Intercept Model - I This model adds a metric predictor at the first level. Level 1: y ij = β 0j + β 1j X ij + ɛ ij, That predictor s coefficient is fixed over all groups. Level 2: β 0j = µ + u j β 1j = γ 10 These methods are very popular especially for panel data. SLIDE 27
28 Random Intercept Model - I We already calculated an example, the number of employees in firms. SLIDE 28
29 Random Coefficients Model - I This model generalizes the idea of groupwise random variation to all coefficients. Level 1: y ij = β 0j + β 1j X ij + ɛ ij, This model specifies the metric predictor s coefficient as randomly varying over groups too. Level 2: β 0j = µ + u 0j β 1j = γ 10 + u 1j SLIDE 29
30 Random Coefficients Model - II Let us fit a random coefficient model for the firm data. SLIDE 30
31 Intercept and Slopes as Outcomes - I This is a very flexible generalisation. The level 1 model remains as before. Level 1: y ij = β 0j + β 1j X ij + ɛ ij, On the second level however, we explain the random variation in the coefficients by second level covariates. Level 2: β 0j = γ 00 + γ 01 W j + u 0j β 1j = γ 10 + γ 11 W j + u 1j SLIDE 31
32 Intercept-and-Slopes-as-Outcomes - II A company ran a promotion campaign over a number of weeks at different stores. At level 1 we have repeated sales and an explanatory factor. At level 2 we have the store location and the store location age as an explanatory variable. Let us calculate an intercept and slopes as outcomes model for the testmarket data. SLIDE 32
33 Multilevel Model Also known as hierarchical linear model. Maximum flexibility of these specification can be achieved by allowing for more levels and multiple explanatory variables on all levels. This is easiest by specifying each level model separately, but becomes notationally tedious. The matrix formulation y = Xβ + Zu + ɛ, is most parsimonous. However this does not really show what is happening. SLIDE 33
34 A Three Level RC Model Let us continue with the testmarket. We have three levels, the stores sales nested in the stores nested in a market. The Level 1 model is: y ijk = β 0jk + β 1jk X ijk + ɛ ijk On Level 2 we specify random slope and intercepts: The Level 3 model is then: β 0jk = γ 00k + u 0jk β 1jk = γ 10k + u 1jk γ 00k = δ u 00k γ 10k = δ u 10k SLIDE 34
35 A Three Level IaSaO model Now we additonally want to include predictors on levels 2 and 3. On level 2 it is store location age and on level 3 it is market size. Let us first specify the level two and level three models (level 1 remains unchanged) Fit it with the software of your choice SLIDE 35
36 Variance Structure for Ω So far we assumed the structure of Ω to be completely free. This means an awful lot of parameters to be estimated (the upper/lower triangular matrix of Ω, p + p(p 1)/2 parameters). We can however restrict entries of Ω to be zero or have certain structure. Popular choices are Identity: All covariances between the random effects are zero AR process: Auto regressive structures based on response ARMA: Auto regressive moving average structures Spatial: Different spatial correlation structures Certain structures are useful for certain data (e.g. panel data, spatial data) SLIDE 36
37 Variance Structure for Σ - I Usually Σ is the standard σ 2 I. This can be too restrictive, especially for repeated measurements. Hence we can specify different error assumptions, similar to classic linear models Identity: All covariances between the random effects are zero Compound symmetry: Constant variance and within-group correlations AR process: Auto regressive structures MA process: Moving average structure ARMA: Auto regressive moving average structures Spatial: Different spatial correlation structures SLIDE 37
38 Variance Structure For Σ - II Let us now specify an autoregressive first order error process (AR(1)) for the residuals for the testmarket data (or the firm data) In SPSS you can do that in the first tab, in R you must use lme with a corstruct object. SLIDE 38
39 Testing Hypotheses For LMM The best thing is that testing works exactly as in all other maximum likelihood models, with the likelihood ratio test: 2log L 0 L A χ 2 p a p 0 Fixed effects can also be tested with F -Tests If REML has been used inference can only be done with the same fixed effect specification Tests on variance components are conservative (boundary solution) SLIDE 39
40 Best Linear Unbiased Estimators (BLUE) Estimators of fixed effects β are best linear unbiased estimators, BLUE. This holds of course if Ω is known (it is the GLS then) The MLE for the fixed effects is unbiased in small (and certainly in large) samples regardless of ˆΩ The MLE for the fixed effects is BLUE in most relevant cases (estimating variances of u Under randomness, estimators for β are asymptotically consistent, efficient and normal Not always identifiable We can predict a persons value based on fixed effects alone if we want, but they will not be subject specific. SLIDE 40
41 Best Linear Unbiased Predictors (BLUP) Estimators of random effects β are not really estimators. Since they are random, we call them predictors. This holds if ˆβ is the GLS estimator (Ω known) Under randomness, estimators for u are asymptotically consistent, efficient and normal The MLE for the random effects is BLUP in most practical cases when estimating variances of u It is a shrinkage estimator and is highly Bayesian It can also be represented as a posterior mode (or mean) We can make predictions of a persons value as the some of fixed and random effect. Such we can even make individual cluster predictions if the number of observations for the cluster is less than the number of fixed effects. SLIDE 41
42 Grand Mean Centering Centering of explanatory variables on the lowest level can become important for a good interpretation of especially the intercepts. This applies even more when we have longitudinal data. Grand mean centering: This is centering around the overall predictor mean This will make the intercept the grand mean This will avoid high negative correlations between intercept and coefficient Regression coefficients are the average change over the whole level 1 support Such a translation can be compensated by the regression coefficients, hence there is invariance of the expected values (for non-zero regression coefficients) There is also invariance of covariance matrix parameters SLIDE 42
43 Group Mean Centering This is centering the mean per level 2 group Necessary if there is systematic variability between the level 1 covariates Invariance as before does not hold We cannot compensate for group based translations by changing the coefficients and covariance matrix It is a different model compared to the non-centered one Including group means as predictors is recommended SLIDE 43
44 Generalized Here we fit a GLM on level 1, everything else remains as before. The GLMM is defined as E(y i ) = µ i = g 1 (Xβ + Zu) or g(µ i ) = Xβ + Zu where g( ) is a link function and the distribution of y i is an exponential family Metric data (normal distribution) Binary data (binomial distribution) Count data (Poisson distribution) Survival data (gamma distribution) In R via glmer and in SPSS via GLMMIXED. SLIDE 44
45 Thank you for your Attention Thomas Rusch Department of Finance, Accounting and Statistics Institute for Statistics and Mathematics URL: WU Wirtschaftsuniversität Wien Augasse 2 6, A-1090 Wien SLIDE 45
Linear Mixed Models. One-way layout REML. Likelihood. Another perspective. Relationship to classical ideas. Drawbacks.
Linear Mixed Models One-way layout Y = Xβ + Zb + ɛ where X and Z are specified design matrices, β is a vector of fixed effect coefficients, b and ɛ are random, mean zero, Gaussian if needed. Usually think
More informationGeneralized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science.
Texts in Statistical Science Generalized Linear Mixed Models Modern Concepts, Methods and Applications Walter W. Stroup CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint
More informationNow consider the case where E(Y) = µ = Xβ and V (Y) = σ 2 G, where G is diagonal, but unknown.
Weighting We have seen that if E(Y) = Xβ and V (Y) = σ 2 G, where G is known, the model can be rewritten as a linear model. This is known as generalized least squares or, if G is diagonal, with trace(g)
More informationLongitudinal and Panel Data: Analysis and Applications for the Social Sciences. Table of Contents
Longitudinal and Panel Data Preface / i Longitudinal and Panel Data: Analysis and Applications for the Social Sciences Table of Contents August, 2003 Table of Contents Preface i vi 1. Introduction 1.1
More informationChapter 4 Multi-factor Treatment Designs with Multiple Error Terms 93
Contents Preface ix Chapter 1 Introduction 1 1.1 Types of Models That Produce Data 1 1.2 Statistical Models 2 1.3 Fixed and Random Effects 4 1.4 Mixed Models 6 1.5 Typical Studies and the Modeling Issues
More informationHierarchical Generalized Linear Models. ERSH 8990 REMS Seminar on HLM Last Lecture!
Hierarchical Generalized Linear Models ERSH 8990 REMS Seminar on HLM Last Lecture! Hierarchical Generalized Linear Models Introduction to generalized models Models for binary outcomes Interpreting parameter
More informationAn Introduction to Multilevel Models. PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012
An Introduction to Multilevel Models PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012 Today s Class Concepts in Longitudinal Modeling Between-Person vs. +Within-Person
More information36-720: Linear Mixed Models
36-720: Linear Mixed Models Brian Junker October 8, 2007 Review: Linear Mixed Models (LMM s) Bayesian Analogues Facilities in R Computational Notes Predictors and Residuals Examples [Related to Christensen
More informationLongitudinal Modeling with Logistic Regression
Newsom 1 Longitudinal Modeling with Logistic Regression Longitudinal designs involve repeated measurements of the same individuals over time There are two general classes of analyses that correspond to
More informationIntroduction to Within-Person Analysis and RM ANOVA
Introduction to Within-Person Analysis and RM ANOVA Today s Class: From between-person to within-person ANOVAs for longitudinal data Variance model comparisons using 2 LL CLP 944: Lecture 3 1 The Two Sides
More informationReview of CLDP 944: Multilevel Models for Longitudinal Data
Review of CLDP 944: Multilevel Models for Longitudinal Data Topics: Review of general MLM concepts and terminology Model comparisons and significance testing Fixed and random effects of time Significance
More informationNon-maximum likelihood estimation and statistical inference for linear and nonlinear mixed models
Optimum Design for Mixed Effects Non-Linear and generalized Linear Models Cambridge, August 9-12, 2011 Non-maximum likelihood estimation and statistical inference for linear and nonlinear mixed models
More informationIntroduction to Regression Analysis. Dr. Devlina Chatterjee 11 th August, 2017
Introduction to Regression Analysis Dr. Devlina Chatterjee 11 th August, 2017 What is regression analysis? Regression analysis is a statistical technique for studying linear relationships. One dependent
More informationFor more information about how to cite these materials visit
Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
More informationLISA Short Course Series Generalized Linear Models (GLMs) & Categorical Data Analysis (CDA) in R. Liang (Sally) Shan Nov. 4, 2014
LISA Short Course Series Generalized Linear Models (GLMs) & Categorical Data Analysis (CDA) in R Liang (Sally) Shan Nov. 4, 2014 L Laboratory for Interdisciplinary Statistical Analysis LISA helps VT researchers
More informationCourse Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model
Course Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model EPSY 905: Multivariate Analysis Lecture 1 20 January 2016 EPSY 905: Lecture 1 -
More informationCourse Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model
Course Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 1: August 22, 2012
More informationGeneralized Linear Models for Non-Normal Data
Generalized Linear Models for Non-Normal Data Today s Class: 3 parts of a generalized model Models for binary outcomes Complications for generalized multivariate or multilevel models SPLH 861: Lecture
More informationRon Heck, Fall Week 8: Introducing Generalized Linear Models: Logistic Regression 1 (Replaces prior revision dated October 20, 2011)
Ron Heck, Fall 2011 1 EDEP 768E: Seminar in Multilevel Modeling rev. January 3, 2012 (see footnote) Week 8: Introducing Generalized Linear Models: Logistic Regression 1 (Replaces prior revision dated October
More informationStat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010
1 Linear models Y = Xβ + ɛ with ɛ N (0, σ 2 e) or Y N (Xβ, σ 2 e) where the model matrix X contains the information on predictors and β includes all coefficients (intercept, slope(s) etc.). 1. Number of
More informationRepeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models
Repeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models EPSY 905: Multivariate Analysis Spring 2016 Lecture #12 April 20, 2016 EPSY 905: RM ANOVA, MANOVA, and Mixed Models
More informationDescribing Change over Time: Adding Linear Trends
Describing Change over Time: Adding Linear Trends Longitudinal Data Analysis Workshop Section 7 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section
More informationMultilevel Statistical Models: 3 rd edition, 2003 Contents
Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction
More informationTime-Invariant Predictors in Longitudinal Models
Time-Invariant Predictors in Longitudinal Models Today s Class (or 3): Summary of steps in building unconditional models for time What happens to missing predictors Effects of time-invariant predictors
More informationUnivariate analysis. Simple and Multiple Regression. Univariate analysis. Simple Regression How best to summarise the data?
Univariate analysis Example - linear regression equation: y = ax + c Least squares criteria ( yobs ycalc ) = yobs ( ax + c) = minimum Simple and + = xa xc xy xa + nc = y Solve for a and c Univariate analysis
More informationPS 271B: Quantitative Methods II Lecture Notes
PS 271B: Quantitative Methods II Lecture Notes (Part 6: Panel/Longitudinal Data; Multilevel/Mixed Effects models) Langche Zeng zeng@ucsd.edu Panel/Longitudinal Data; Multilevel Modeling; Mixed effects
More informationTime Invariant Predictors in Longitudinal Models
Time Invariant Predictors in Longitudinal Models Longitudinal Data Analysis Workshop Section 9 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section
More informationModeling the Covariance
Modeling the Covariance Jamie Monogan University of Georgia February 3, 2016 Jamie Monogan (UGA) Modeling the Covariance February 3, 2016 1 / 16 Objectives By the end of this meeting, participants should
More informationEstimation and Centering
Estimation and Centering PSYED 3486 Feifei Ye University of Pittsburgh Main Topics Estimating the level-1 coefficients for a particular unit Reading: R&B, Chapter 3 (p85-94) Centering-Location of X Reading
More informationCategorical Predictor Variables
Categorical Predictor Variables We often wish to use categorical (or qualitative) variables as covariates in a regression model. For binary variables (taking on only 2 values, e.g. sex), it is relatively
More informationIntroducing Generalized Linear Models: Logistic Regression
Ron Heck, Summer 2012 Seminars 1 Multilevel Regression Models and Their Applications Seminar Introducing Generalized Linear Models: Logistic Regression The generalized linear model (GLM) represents and
More informationGeneral Regression Model
Scott S. Emerson, M.D., Ph.D. Department of Biostatistics, University of Washington, Seattle, WA 98195, USA January 5, 2015 Abstract Regression analysis can be viewed as an extension of two sample statistical
More informationTopic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model
Topic 17 - Single Factor Analysis of Variance - Fall 2013 One way ANOVA Cell means model Factor effects model Outline Topic 17 2 One-way ANOVA Response variable Y is continuous Explanatory variable is
More informationBayesian Linear Regression
Bayesian Linear Regression Sudipto Banerjee 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. September 15, 2010 1 Linear regression models: a Bayesian perspective
More informationStatistical Distribution Assumptions of General Linear Models
Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions
More informationIntroduction to Random Effects of Time and Model Estimation
Introduction to Random Effects of Time and Model Estimation Today s Class: The Big Picture Multilevel model notation Fixed vs. random effects of time Random intercept vs. random slope models How MLM =
More informationDescribing Nonlinear Change Over Time
Describing Nonlinear Change Over Time Longitudinal Data Analysis Workshop Section 8 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section 8: Describing
More informationModeling the Mean: Response Profiles v. Parametric Curves
Modeling the Mean: Response Profiles v. Parametric Curves Jamie Monogan University of Georgia Escuela de Invierno en Métodos y Análisis de Datos Universidad Católica del Uruguay Jamie Monogan (UGA) Modeling
More informationLinear Regression Models P8111
Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started
More informationLinear, Generalized Linear, and Mixed-Effects Models in R. Linear and Generalized Linear Models in R Topics
Linear, Generalized Linear, and Mixed-Effects Models in R John Fox McMaster University ICPSR 2018 John Fox (McMaster University) Statistical Models in R ICPSR 2018 1 / 19 Linear and Generalized Linear
More informationLogistic regression. 11 Nov Logistic regression (EPFL) Applied Statistics 11 Nov / 20
Logistic regression 11 Nov 2010 Logistic regression (EPFL) Applied Statistics 11 Nov 2010 1 / 20 Modeling overview Want to capture important features of the relationship between a (set of) variable(s)
More informationLinear Regression With Special Variables
Linear Regression With Special Variables Junhui Qian December 21, 2014 Outline Standardized Scores Quadratic Terms Interaction Terms Binary Explanatory Variables Binary Choice Models Standardized Scores:
More informationMultivariate Survival Analysis
Multivariate Survival Analysis Previously we have assumed that either (X i, δ i ) or (X i, δ i, Z i ), i = 1,..., n, are i.i.d.. This may not always be the case. Multivariate survival data can arise in
More informationA (Brief) Introduction to Crossed Random Effects Models for Repeated Measures Data
A (Brief) Introduction to Crossed Random Effects Models for Repeated Measures Data Today s Class: Review of concepts in multivariate data Introduction to random intercepts Crossed random effects models
More informationChapter 1. Modeling Basics
Chapter 1. Modeling Basics What is a model? Model equation and probability distribution Types of model effects Writing models in matrix form Summary 1 What is a statistical model? A model is a mathematical
More informationH-LIKELIHOOD ESTIMATION METHOOD FOR VARYING CLUSTERED BINARY MIXED EFFECTS MODEL
H-LIKELIHOOD ESTIMATION METHOOD FOR VARYING CLUSTERED BINARY MIXED EFFECTS MODEL Intesar N. El-Saeiti Department of Statistics, Faculty of Science, University of Bengahzi-Libya. entesar.el-saeiti@uob.edu.ly
More information36-720: The Rasch Model
36-720: The Rasch Model Brian Junker October 15, 2007 Multivariate Binary Response Data Rasch Model Rasch Marginal Likelihood as a GLMM Rasch Marginal Likelihood as a Log-Linear Model Example For more
More informationClassification. Chapter Introduction. 6.2 The Bayes classifier
Chapter 6 Classification 6.1 Introduction Often encountered in applications is the situation where the response variable Y takes values in a finite set of labels. For example, the response Y could encode
More informationClass Notes: Week 8. Probit versus Logit Link Functions and Count Data
Ronald Heck Class Notes: Week 8 1 Class Notes: Week 8 Probit versus Logit Link Functions and Count Data This week we ll take up a couple of issues. The first is working with a probit link function. While
More informationLongitudinal Data Analysis Using SAS Paul D. Allison, Ph.D. Upcoming Seminar: October 13-14, 2017, Boston, Massachusetts
Longitudinal Data Analysis Using SAS Paul D. Allison, Ph.D. Upcoming Seminar: October 13-14, 217, Boston, Massachusetts Outline 1. Opportunities and challenges of panel data. a. Data requirements b. Control
More informationMixed- Model Analysis of Variance. Sohad Murrar & Markus Brauer. University of Wisconsin- Madison. Target Word Count: Actual Word Count: 2755
Mixed- Model Analysis of Variance Sohad Murrar & Markus Brauer University of Wisconsin- Madison The SAGE Encyclopedia of Educational Research, Measurement and Evaluation Target Word Count: 3000 - Actual
More informationMulti-level Models: Idea
Review of 140.656 Review Introduction to multi-level models The two-stage normal-normal model Two-stage linear models with random effects Three-stage linear models Two-stage logistic regression with random
More informationEPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7
Introduction to Generalized Univariate Models: Models for Binary Outcomes EPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7 EPSY 905: Intro to Generalized In This Lecture A short review
More informationHW 2 due March 6 random effects and mixed effects models ELM Ch. 8 R Studio Cheatsheets In the News: homeopathic vaccines
Today HW 2 due March 6 random effects and mixed effects models ELM Ch. 8 R Studio Cheatsheets In the News: homeopathic vaccines STA 2201: Applied Statistics II March 4, 2015 1/35 A general framework y
More informationStatistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
More informationMultilevel Modeling: A Second Course
Multilevel Modeling: A Second Course Kristopher Preacher, Ph.D. Upcoming Seminar: February 2-3, 2017, Ft. Myers, Florida What this workshop will accomplish I will review the basics of multilevel modeling
More informationMultilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Today s Lecture Linear models from a matrix perspective An example of how to do
More informationImpact of serial correlation structures on random effect misspecification with the linear mixed model.
Impact of serial correlation structures on random effect misspecification with the linear mixed model. Brandon LeBeau University of Iowa file:///c:/users/bleb/onedrive%20 %20University%20of%20Iowa%201/JournalArticlesInProgress/Diss/Study2/Pres/pres.html#(2)
More informationISQS 5349 Spring 2013 Final Exam
ISQS 5349 Spring 2013 Final Exam Name: General Instructions: Closed books, notes, no electronic devices. Points (out of 200) are in parentheses. Put written answers on separate paper; multiple choices
More informationTime-Invariant Predictors in Longitudinal Models
Time-Invariant Predictors in Longitudinal Models Topics: Summary of building unconditional models for time Missing predictors in MLM Effects of time-invariant predictors Fixed, systematically varying,
More informationChapter 1 Statistical Inference
Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations
More informationLongitudinal Data Analysis
Longitudinal Data Analysis Mike Allerhand This document has been produced for the CCACE short course: Longitudinal Data Analysis. No part of this document may be reproduced, in any form or by any means,
More informationReview. Timothy Hanson. Department of Statistics, University of South Carolina. Stat 770: Categorical Data Analysis
Review Timothy Hanson Department of Statistics, University of South Carolina Stat 770: Categorical Data Analysis 1 / 22 Chapter 1: background Nominal, ordinal, interval data. Distributions: Poisson, binomial,
More informationLinear Regression 9/23/17. Simple linear regression. Advertising sales: Variance changes based on # of TVs. Advertising sales: Normal error?
Simple linear regression Linear Regression Nicole Beckage y " = β % + β ' x " + ε so y* " = β+ % + β+ ' x " Method to assess and evaluate the correlation between two (continuous) variables. The slope of
More informationIntermediate Econometrics
Intermediate Econometrics Heteroskedasticity Text: Wooldridge, 8 July 17, 2011 Heteroskedasticity Assumption of homoskedasticity, Var(u i x i1,..., x ik ) = E(u 2 i x i1,..., x ik ) = σ 2. That is, the
More informationRegression With a Categorical Independent Variable
Regression With a Independent Variable Lecture 10 November 5, 2008 ERSH 8320 Lecture #10-11/5/2008 Slide 1 of 54 Today s Lecture Today s Lecture Chapter 11: Regression with a single categorical independent
More informationLinear Regression. Junhui Qian. October 27, 2014
Linear Regression Junhui Qian October 27, 2014 Outline The Model Estimation Ordinary Least Square Method of Moments Maximum Likelihood Estimation Properties of OLS Estimator Unbiasedness Consistency Efficiency
More informationAn R # Statistic for Fixed Effects in the Linear Mixed Model and Extension to the GLMM
An R Statistic for Fixed Effects in the Linear Mixed Model and Extension to the GLMM Lloyd J. Edwards, Ph.D. UNC-CH Department of Biostatistics email: Lloyd_Edwards@unc.edu Presented to the Department
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models Mixed effects models - Part II Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby
More informationEMERGING MARKETS - Lecture 2: Methodology refresher
EMERGING MARKETS - Lecture 2: Methodology refresher Maria Perrotta April 4, 2013 SITE http://www.hhs.se/site/pages/default.aspx My contact: maria.perrotta@hhs.se Aim of this class There are many different
More informationCorrelation in Linear Regression
Vrije Universiteit Amsterdam Research Paper Correlation in Linear Regression Author: Yura Perugachi-Diaz Student nr.: 2566305 Supervisor: Dr. Bartek Knapik May 29, 2017 Faculty of Sciences Research Paper
More informationGLS and FGLS. Econ 671. Purdue University. Justin L. Tobias (Purdue) GLS and FGLS 1 / 22
GLS and FGLS Econ 671 Purdue University Justin L. Tobias (Purdue) GLS and FGLS 1 / 22 In this lecture we continue to discuss properties associated with the GLS estimator. In addition we discuss the practical
More informationGeneralized Linear Models 1
Generalized Linear Models 1 STA 2101/442: Fall 2012 1 See last slide for copyright information. 1 / 24 Suggested Reading: Davison s Statistical models Exponential families of distributions Sec. 5.2 Chapter
More informationLongitudinal Data Analysis of Health Outcomes
Longitudinal Data Analysis of Health Outcomes Longitudinal Data Analysis Workshop Running Example: Days 2 and 3 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development
More information1 Mixed effect models and longitudinal data analysis
1 Mixed effect models and longitudinal data analysis Mixed effects models provide a flexible approach to any situation where data have a grouping structure which introduces some kind of correlation between
More informationRelated Concepts: Lecture 9 SEM, Statistical Modeling, AI, and Data Mining. I. Terminology of SEM
Lecture 9 SEM, Statistical Modeling, AI, and Data Mining I. Terminology of SEM Related Concepts: Causal Modeling Path Analysis Structural Equation Modeling Latent variables (Factors measurable, but thru
More informationCoping with Additional Sources of Variation: ANCOVA and Random Effects
Coping with Additional Sources of Variation: ANCOVA and Random Effects 1/49 More Noise in Experiments & Observations Your fixed coefficients are not always so fixed Continuous variation between samples
More informationIntroduction and Background to Multilevel Analysis
Introduction and Background to Multilevel Analysis Dr. J. Kyle Roberts Southern Methodist University Simmons School of Education and Human Development Department of Teaching and Learning Background and
More informationRegression in R. Seth Margolis GradQuant May 31,
Regression in R Seth Margolis GradQuant May 31, 2018 1 GPA What is Regression Good For? Assessing relationships between variables This probably covers most of what you do 4 3.8 3.6 3.4 Person Intelligence
More informationmultilevel modeling: concepts, applications and interpretations
multilevel modeling: concepts, applications and interpretations lynne c. messer 27 october 2010 warning social and reproductive / perinatal epidemiologist concepts why context matters multilevel models
More informationGeneralized Linear Models (GLZ)
Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) are an extension of the linear modeling process that allows models to be fit to data that follow probability distributions other than the
More informationLinear Methods for Prediction
Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we
More informationTime-Invariant Predictors in Longitudinal Models
Time-Invariant Predictors in Longitudinal Models Today s Topics: What happens to missing predictors Effects of time-invariant predictors Fixed vs. systematically varying vs. random effects Model building
More informationEconometrics. Week 4. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague
Econometrics Week 4 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 23 Recommended Reading For the today Serial correlation and heteroskedasticity in
More informationReview: what is a linear model. Y = β 0 + β 1 X 1 + β 2 X 2 + A model of the following form:
Outline for today What is a generalized linear model Linear predictors and link functions Example: fit a constant (the proportion) Analysis of deviance table Example: fit dose-response data using logistic
More informationLecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011
Lecture 2: Linear Models Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector
More informationFitting Multidimensional Latent Variable Models using an Efficient Laplace Approximation
Fitting Multidimensional Latent Variable Models using an Efficient Laplace Approximation Dimitris Rizopoulos Department of Biostatistics, Erasmus University Medical Center, the Netherlands d.rizopoulos@erasmusmc.nl
More informationRegression With a Categorical Independent Variable
Regression With a Categorical Independent Variable Lecture 15 March 17, 2005 Applied Regression Analysis Lecture #15-3/17/2005 Slide 1 of 29 Today s Lecture» Today s Lecture» Midterm Note» Example Regression
More informationStat 579: Generalized Linear Models and Extensions
Stat 579: Generalized Linear Models and Extensions Linear Mixed Models for Longitudinal Data Yan Lu April, 2018, week 15 1 / 38 Data structure t1 t2 tn i 1st subject y 11 y 12 y 1n1 Experimental 2nd subject
More informationProblem Set #6: OLS. Economics 835: Econometrics. Fall 2012
Problem Set #6: OLS Economics 835: Econometrics Fall 202 A preliminary result Suppose we have a random sample of size n on the scalar random variables (x, y) with finite means, variances, and covariance.
More information20. REML Estimation of Variance Components. Copyright c 2018 (Iowa State University) 20. Statistics / 36
20. REML Estimation of Variance Components Copyright c 2018 (Iowa State University) 20. Statistics 510 1 / 36 Consider the General Linear Model y = Xβ + ɛ, where ɛ N(0, Σ) and Σ is an n n positive definite
More informationTime-Invariant Predictors in Longitudinal Models
Time-Invariant Predictors in Longitudinal Models Topics: What happens to missing predictors Effects of time-invariant predictors Fixed vs. systematically varying vs. random effects Model building strategies
More informationPart 8: GLMs and Hierarchical LMs and GLMs
Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course
More informationECON3150/4150 Spring 2015
ECON3150/4150 Spring 2015 Lecture 3&4 - The linear regression model Siv-Elisabeth Skjelbred University of Oslo January 29, 2015 1 / 67 Chapter 4 in S&W Section 17.1 in S&W (extended OLS assumptions) 2
More informationExperimental Design and Data Analysis for Biologists
Experimental Design and Data Analysis for Biologists Gerry P. Quinn Monash University Michael J. Keough University of Melbourne CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv I I Introduction 1 1.1
More informationSpecifying Latent Curve and Other Growth Models Using Mplus. (Revised )
Ronald H. Heck 1 University of Hawai i at Mānoa Handout #20 Specifying Latent Curve and Other Growth Models Using Mplus (Revised 12-1-2014) The SEM approach offers a contrasting framework for use in analyzing
More informationNormal distribution We have a random sample from N(m, υ). The sample mean is Ȳ and the corrected sum of squares is S yy. After some simplification,
Likelihood Let P (D H) be the probability an experiment produces data D, given hypothesis H. Usually H is regarded as fixed and D variable. Before the experiment, the data D are unknown, and the probability
More informationModels for Clustered Data
Models for Clustered Data Edps/Psych/Soc 589 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline Notation NELS88 data Fixed Effects ANOVA
More informationSTAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression
STAT 135 Lab 11 Tests for Categorical Data (Fisher s Exact test, χ 2 tests for Homogeneity and Independence) and Linear Regression Rebecca Barter April 20, 2015 Fisher s Exact Test Fisher s Exact Test
More informationModels for Clustered Data
Models for Clustered Data Edps/Psych/Stat 587 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Notation NELS88 data Fixed Effects ANOVA
More information