Multilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
|
|
- Daniela Long
- 5 years ago
- Views:
Transcription
1 Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
2 Today s Lecture Linear models from a matrix perspective An example of how to do MANOVA in linear mixed models A more modern twist on the classic technique
3 THE MULTIVARIATE NORMAL DISTRIBUTION
4 Multivariate Normal Distribution The generalization of the univariate normal distribution to multiple variables is called the multivariate normal distribution (MVN) Many multivariate techniques rely on this distribution in some manner Multilevel/Mixed models
5 Univariate Normal Distribution The univariate normal distribution function is: f x = 1 2 x μ exp 2πσ2 2σ 2 The mean is μ The variance is σ 2 Standard notation for random variables x following normal distributions is x N μ, σ 2
6 Univariate Normal Distribution
7 Multivariate Normal The multivariate normal distribution function is: 1 f x = 2π p 2 Σ 1 exp x μ Σ 1 x μ T 2 2 The mean vector is μ The covariance matrix is Σ Standard notation for the MVN distribution of p variables is N p μ, Σ
8 Picturing the Multivariate Normal
9 Contour Plot (View From Above)
10 Another Multivariate Normal Plot
11 MVN Properties The MVN distribution has some convenient properties for mixed models If x has a multivariate normal distribution, then: 1. Linear combinations of x are normally distributed. 2. All subsets of the components of x have an MVN distribution. 3. Zero covariance implies that the corresponding components are independently distributed. 4. The conditional distributions of the components are MVN. Especially important for our models
12 LINEAR MODELS IN MATRICES
13 Linear Models with Matrices Recall our basic linear model (here a regression model) for observation i (of N): Y i = β 0 + β 1 X i1 + + β k X ik + e i The equation above can be expressed more compactly by a set of matrices Y = Xβ + e Y is of size (N x 1) X is of size (N x (1 + k)) β is of size ((1+k) x 1) e is of size (N x 1)
14 Unpacking the Equation Y 1 Y 2 Y NY (N x 1) = 1 X 11 X 1k 1 X 21 X 2k 1 X N1 X Nk X (N x (1 + k)) β 0 β 1 β k β ((1 + k) x 1) + e 1 e 2 e Ne (N x 1) For the first observation: Y 1 = β 0 + β 1 X β k X 1k + e 1
15 Notes on Matrices The use of matrices allows for a compact form of the model equation All observations are included The matrix of predictors, X, has the first column containing all ones Corresponds (multiplies) the intercept β 0 Shows how design matrices can be used for in linear models Think about categorical predictors (dummy coding/effect coding)
16 Linear Model Assumptions Recall that we assumed that the error terms were assumed to be Independent Normally distributed e i N(0, σ e 2 ) With matrices, we can now talk about the joint distribution of error terms (for everyone) e N N 0, σ e 2 I N
17 Error Covariance Matrix The fixed effects linear model assumes the following structure for the errors: 2 σ e 0 σ 2 e I N = 2 0 σ e In multilevel analyses, this assumption is not valid so our models introduce terms to relax this assumption
18 Estimation in Linear Models Regression estimates are typically found via least squares (called L 2 estimates) In least squares regression, the estimates are found by minimizing the sum of squared errors: N i=1 N e i 2 = Y i Y i 2 i=1 N = Y i β 0 β 1 X i1 β k X ik 2 i=1 As you could guess, we could do this matrices: N i=1 N e i 2 = Y i x i T β 2 i=1 = Y Xβ T Y Xβ = e T e
19 The Estimator The equation for β that minimizes e T e is: β = X T X 1 X T Y The nice thing about this equation is that simultaneously is the MLE (maximizes the likelihood function under normality assumptions)
20 Model Assumptions The conditional distribution of Y has a multivariate normal distribution: Mean vector is the predicted values of Y Covariance matrix is error covariance matrix f Y X N N Xβ, σ e 2 I N
21 Variance of Estimates The covariance matrix of β contains useful information regarding the standard errors of the estimates (which are found along the diagonal) Under the linear model, this is given by: Var β = σ e 2 X T X 1
22 MULTILEVEL MODELS IN MATRICES (GENERAL LINEAR MIXED MODELS)
23 Multilevel (Mixed) Models The general linear mixed model is given by: Y = Xβ + Zγ + e Y is of size (N x 1) X is of size (N x (1+k)) β is of size ((1+k) x 1) Z is of size (N x r*g) (r random effects; g groups) γ is of size (r*g x 1) e is of size (N x 1)
24 The New Terms The Z matrix is analogous to the X matrix it contains the predictors of the random effects (i.e., random intercepts, slopes, etc ) The γ matrix contains the random effects for each observation Because of the size of the observations, these matrices are rather large Can be notated differently, though
25 Z and γ for a Random Intercept For a model with a random intercept, this is how Z and γ appear: Rows Represent Observations Z = Columns Represent Groups ; γ = Columns Represent Type of Effect γ 10 γ 20 γ 30 Rows Represent Group Effect Values
26 Multilevel (Mixed) Model Assumptions Assumptions in multilevel (mixed) models involve the random effects and the error terms Random effect assumptions: Multivariate Normal (across r random effects) Mean Vector 0; Covariance Matrix G (block diagonal within a group) γ N r 0, G Error term assumptions Multivariate Normal (within a group) Mean Vector 0; Covariance Matrix R e N N (0, R)
27 Model Assumptions The conditional distribution of Y has a multivariate normal distribution: Mean vector is the predicted values of Y Covariance matrix is combination of random effect and error term covariance matrices Allows for correlated observations f Y X, Z N N Xβ, ZGZ T + R
28 New Covariance Matrix Because of the grouping structure of data, the new covariance matrix is block-diagonal Blocks represent the covariance matrix for a group/cluster of observations
29 Model Estimation Because of the inclusion of random effects (which are not directly observable), the model no longer has a single estimation equation Rather, we now must use an iterative process to estimate model parameters Two estimators are commonly used: maximum likelihood (ML) and residual maximum likelihood (REML) I will introduce ML first then REML
30 ML Estimation of Mixed Models The goal in ML estimation is to pick a set of parameters that maximize the likelihood function Typically the log-likelihood is used Here, we have to know β, γ, G, R γ isn t a part of the function below The log-likelihood function is the log of the model-assumed MVN: N N Xβ, V = ZGZ T + R
31 Simplifying Things Because of the wonders of math, we can use a technique called estimated generalized least squares Use some method to find G and R: G and R Given G and R, we can find β Here, we will define V = ZGZ T + R Specifically: β = X T VX 1 X T V 1 Y
32 The ML Log Likelihood The goal is to pick G and R and then substitute them into the log likelihood function, producing a log likelihood value Picking G and R can be done using Newton-Raphson (as is done in SAS) The function value is: Where: l G, R = 1 2 log V 1 2 rt V 1 r n 2 log(2π) r = Y Xβ = Y X X T VX 1 X T V 1 Y
33 Issues with ML Estimates ML estimation is a common choice and performs well when sample sizes are large However, estimates of the variances will be biased Similar to basic statistics phenomena of using N versus N-1 in the variance/standard deviation Therefore, the residual ML estimator was developed Called REML
34 REML Estimator The REML estimator maximizes the likelihood of the residuals The likelihood function comes from stating the likelihood of the data as a function of the likelihood of the estimated fixed effects and the residuals Here, we take the estimated residuals to be e = Y Xβ Where β = X T V 1 X 1 X T V 1 Y
35 Deriving REML Because Y is multivariate normal, β and e are linear functions of Y that are: Normally distributed (see properties of MVN) Independent Therefore, with independence we can re-express the likelihood of Y as a product of β and e L Y V = L β V L e V
36 More Deriving REML Further, due to the consistency of the estimates, we know that β N β, X T V 1 X 1 Therefore, it is now our goal to maximize the log-likelihood of the residuals, or L(e V)
37 Step 1: Taking the Log We now take the log of our original likelihood function: L Y V = L β V L e V Yielding: log L Y V = log L β V + log L e V Which gives us: log L e V = log L Y V log L β V
38 Step 2: We know that Y N N Xβ, V = ZGZ T + R and β N β, X T V 1 X 1 We can then put the MVN associated with each into our log likelihood of the residual log L e V = log L Y V log L β V
39 Even More log L e V = log L Y V log L β V = 1 2 log XT V 1 X + log V + Y Xβ T V 1 Y Xβ β β T X T V 1 X β β Here: Y Xβ T V 1 Y Xβ = Y Xβ T V 1 Y Xβ + β β T X T V 1 X β β Meaning we can cancel the last term.
40 The REML Log Likelihood After all the slides before, we can now present the REML log likelihood: log L e V = 1 2 log XT V 1 X + log V + e T n p e 2 log (2π)
41 Uses of ML and REML ML can be used for deviance tests when the fixed effects are the same or are different REML can be used for deviance tests when the fixed effects are the same only Residuals change when the fixed effects change
42 Demonstrating Through an Example REVISITING MANOVA FROM A LINEAR MODELS PERSPECTIVE
43 MANOVA Revisited The classical MANOVA model can be rephrased so as to fit into a multilevel or mixed-effects model framework The new framework can allow for: A different (smaller) set of covariances to be estimated Useful for approximating a full matrix when you do not have a lot of data Predictor variables that vary by outcome Useful for repeated measures designs Synchronization with more modern methods Multilevel models The new framework does not provide an overall MANOVA hypothesis test (i.e., Wilks Lambda)
44 Rearranging Data The first step to using a linear model framework is to convert our data from wide to long Here we take two scores and put both into one column We also must add two dummy-coded variables indicating which score is represented by a row of the data Wide Data Long Data
45 The Analysis - MANOVA Previously, we used MANOVA to test the multivariate hypothesis that the mean vectors were the same across all conditions:
46 More From MANOVA The Error SSCP Matrix:
47 Converting the Error SSCP Matrix to an Error Covariance Matrix Because the Error SSCP matrix is not a covariance matrix, we can obtain the covariance matrix by dividing the Error SSCP matrix by the degrees of freedom (here 109): S e = 1 E = df e =
48 Univariate Results from GLM Once we rejected our null hypothesis we then became interested in univariate ANOVAs for each outcome variable:
49 Using the MIXED Procedure However, we can now do *most* of the univariate procedures from our MANOVA within proc mixed
50 MIXED: The Error Covariance Matrix
51 MIXED: The Univariate Hypothesis Tests
52 Secondary Phrasing: *almost* MANOVA
53 Final Thoughts Today we discussed the matrix form of linear models with mixed effects Multilevel models The matrix form can be useful for reading about these models in papers and presentations This class was meant to be an introduction to the technical side of the modeling framework Much more time can be spent on just this alone
54 Next Time (Friday) Lab: Meet in Helen Newberry We ll discuss how to do MANOVA and discriminant analysis in SAS
Statistical Distribution Assumptions of General Linear Models
Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions
More informationRepeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models
Repeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models EPSY 905: Multivariate Analysis Spring 2016 Lecture #12 April 20, 2016 EPSY 905: RM ANOVA, MANOVA, and Mixed Models
More informationMaximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood
Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood PRE 906: Structural Equation Modeling Lecture #3 February 4, 2015 PRE 906, SEM: Estimation Today s Class An
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationCourse Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model
Course Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 1: August 22, 2012
More informationMultivariate Regression (Chapter 10)
Multivariate Regression (Chapter 10) This week we ll cover multivariate regression and maybe a bit of canonical correlation. Today we ll mostly review univariate multivariate regression. With multivariate
More informationGeneralized Linear Models for Non-Normal Data
Generalized Linear Models for Non-Normal Data Today s Class: 3 parts of a generalized model Models for binary outcomes Complications for generalized multivariate or multilevel models SPLH 861: Lecture
More informationCourse Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model
Course Introduction and Overview Descriptive Statistics Conceptualizations of Variance Review of the General Linear Model EPSY 905: Multivariate Analysis Lecture 1 20 January 2016 EPSY 905: Lecture 1 -
More informationApplied Multivariate Statistical Modeling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur
Applied Multivariate Statistical Modeling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 29 Multivariate Linear Regression- Model
More informationA Re-Introduction to General Linear Models (GLM)
A Re-Introduction to General Linear Models (GLM) Today s Class: You do know the GLM Estimation (where the numbers in the output come from): From least squares to restricted maximum likelihood (REML) Reviewing
More informationA Introduction to Matrix Algebra and the Multivariate Normal Distribution
A Introduction to Matrix Algebra and the Multivariate Normal Distribution PRE 905: Multivariate Analysis Spring 2014 Lecture 6 PRE 905: Lecture 7 Matrix Algebra and the MVN Distribution Today s Class An
More informationIntroduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation. EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016
Introduction to Bayesian Statistics and Markov Chain Monte Carlo Estimation EPSY 905: Multivariate Analysis Spring 2016 Lecture #10: April 6, 2016 EPSY 905: Intro to Bayesian and MCMC Today s Class An
More informationProfile Analysis Multivariate Regression
Lecture 8 October 12, 2005 Analysis Lecture #8-10/12/2005 Slide 1 of 68 Today s Lecture Profile analysis Today s Lecture Schedule : regression review multiple regression is due Thursday, October 27th,
More informationAn Introduction to Mplus and Path Analysis
An Introduction to Mplus and Path Analysis PSYC 943: Fundamentals of Multivariate Modeling Lecture 10: October 30, 2013 PSYC 943: Lecture 10 Today s Lecture Path analysis starting with multivariate regression
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationIntroduction to Matrix Algebra and the Multivariate Normal Distribution
Introduction to Matrix Algebra and the Multivariate Normal Distribution Introduction to Structural Equation Modeling Lecture #2 January 18, 2012 ERSH 8750: Lecture 2 Motivation for Learning the Multivariate
More informationLongitudinal Data Analysis of Health Outcomes
Longitudinal Data Analysis of Health Outcomes Longitudinal Data Analysis Workshop Running Example: Days 2 and 3 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development
More informationModel Estimation Example
Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationCategorical Predictor Variables
Categorical Predictor Variables We often wish to use categorical (or qualitative) variables as covariates in a regression model. For binary variables (taking on only 2 values, e.g. sex), it is relatively
More informationEPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7
Introduction to Generalized Univariate Models: Models for Binary Outcomes EPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7 EPSY 905: Intro to Generalized In This Lecture A short review
More informationUnivariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation
Univariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation PRE 905: Multivariate Analysis Spring 2014 Lecture 4 Today s Class The building blocks: The basics of mathematical
More informationNormal distribution We have a random sample from N(m, υ). The sample mean is Ȳ and the corrected sum of squares is S yy. After some simplification,
Likelihood Let P (D H) be the probability an experiment produces data D, given hypothesis H. Usually H is regarded as fixed and D variable. Before the experiment, the data D are unknown, and the probability
More informationMultivariate analysis of variance and covariance
Introduction Multivariate analysis of variance and covariance Univariate ANOVA: have observations from several groups, numerical dependent variable. Ask whether dependent variable has same mean for each
More informationIntroduction to Random Effects of Time and Model Estimation
Introduction to Random Effects of Time and Model Estimation Today s Class: The Big Picture Multilevel model notation Fixed vs. random effects of time Random intercept vs. random slope models How MLM =
More informationAn Introduction to Path Analysis
An Introduction to Path Analysis PRE 905: Multivariate Analysis Lecture 10: April 15, 2014 PRE 905: Lecture 10 Path Analysis Today s Lecture Path analysis starting with multivariate regression then arriving
More informationMultivariate Linear Models
Multivariate Linear Models Stanley Sawyer Washington University November 7, 2001 1. Introduction. Suppose that we have n observations, each of which has d components. For example, we may have d measurements
More informationPath Analysis. PRE 906: Structural Equation Modeling Lecture #5 February 18, PRE 906, SEM: Lecture 5 - Path Analysis
Path Analysis PRE 906: Structural Equation Modeling Lecture #5 February 18, 2015 PRE 906, SEM: Lecture 5 - Path Analysis Key Questions for Today s Lecture What distinguishes path models from multivariate
More informationAdditional Notes: Investigating a Random Slope. When we have fixed level-1 predictors at level 2 we show them like this:
Ron Heck, Summer 01 Seminars 1 Multilevel Regression Models and Their Applications Seminar Additional Notes: Investigating a Random Slope We can begin with Model 3 and add a Random slope parameter. If
More informationSTA441: Spring Multiple Regression. This slide show is a free open source document. See the last slide for copyright information.
STA441: Spring 2018 Multiple Regression This slide show is a free open source document. See the last slide for copyright information. 1 Least Squares Plane 2 Statistical MODEL There are p-1 explanatory
More informationSTAT 501 EXAM I NAME Spring 1999
STAT 501 EXAM I NAME Spring 1999 Instructions: You may use only your calculator and the attached tables and formula sheet. You can detach the tables and formula sheet from the rest of this exam. Show your
More informationGeneral Linear Models. with General Linear Hypothesis Tests and Likelihood Ratio Tests
General Linear Models with General Linear Hypothesis Tests and Likelihood Ratio Tests 1 Background Linear combinations of Normals are Normal XX nn ~ NN μμ, ΣΣ AAAA ~ NN AAμμ, AAAAAA A sum of squared, standardized
More informationA Re-Introduction to General Linear Models
A Re-Introduction to General Linear Models Today s Class: Big picture overview Why we are using restricted maximum likelihood within MIXED instead of least squares within GLM Linear model interpretation
More informationLecture 2. The Simple Linear Regression Model: Matrix Approach
Lecture 2 The Simple Linear Regression Model: Matrix Approach Matrix algebra Matrix representation of simple linear regression model 1 Vectors and Matrices Where it is necessary to consider a distribution
More informationRegression With a Categorical Independent Variable
Regression With a Independent Variable Lecture 10 November 5, 2008 ERSH 8320 Lecture #10-11/5/2008 Slide 1 of 54 Today s Lecture Today s Lecture Chapter 11: Regression with a single categorical independent
More informationIntroduction to Within-Person Analysis and RM ANOVA
Introduction to Within-Person Analysis and RM ANOVA Today s Class: From between-person to within-person ANOVAs for longitudinal data Variance model comparisons using 2 LL CLP 944: Lecture 3 1 The Two Sides
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Canonical Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Canonical Slide
More informationRon Heck, Fall Week 3: Notes Building a Two-Level Model
Ron Heck, Fall 2011 1 EDEP 768E: Seminar on Multilevel Modeling rev. 9/6/2011@11:27pm Week 3: Notes Building a Two-Level Model We will build a model to explain student math achievement using student-level
More informationHierarchical Generalized Linear Models. ERSH 8990 REMS Seminar on HLM Last Lecture!
Hierarchical Generalized Linear Models ERSH 8990 REMS Seminar on HLM Last Lecture! Hierarchical Generalized Linear Models Introduction to generalized models Models for binary outcomes Interpreting parameter
More informationRegression With a Categorical Independent Variable
Regression With a Categorical Independent Variable Lecture 15 March 17, 2005 Applied Regression Analysis Lecture #15-3/17/2005 Slide 1 of 29 Today s Lecture» Today s Lecture» Midterm Note» Example Regression
More informationChapter 7, continued: MANOVA
Chapter 7, continued: MANOVA The Multivariate Analysis of Variance (MANOVA) technique extends Hotelling T 2 test that compares two mean vectors to the setting in which there are m 2 groups. We wish to
More informationReview of the General Linear Model
Review of the General Linear Model EPSY 905: Multivariate Analysis Online Lecture #2 Learning Objectives Types of distributions: Ø Conditional distributions The General Linear Model Ø Regression Ø Analysis
More informationInteractions among Continuous Predictors
Interactions among Continuous Predictors Today s Class: Simple main effects within two-way interactions Conquering TEST/ESTIMATE/LINCOM statements Regions of significance Three-way interactions (and beyond
More informationLinear Regression Models P8111
Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started
More informationLecture 16 Solving GLMs via IRWLS
Lecture 16 Solving GLMs via IRWLS 09 November 2015 Taylor B. Arnold Yale Statistics STAT 312/612 Notes problem set 5 posted; due next class problem set 6, November 18th Goals for today fixed PCA example
More informationAppendix A: Review of the General Linear Model
Appendix A: Review of the General Linear Model The generallinear modelis an important toolin many fmri data analyses. As the name general suggests, this model can be used for many different types of analyses,
More informationOther hypotheses of interest (cont d)
Other hypotheses of interest (cont d) In addition to the simple null hypothesis of no treatment effects, we might wish to test other hypothesis of the general form (examples follow): H 0 : C k g β g p
More informationMixed models in R using the lme4 package Part 4: Theory of linear mixed models
Mixed models in R using the lme4 package Part 4: Theory of linear mixed models Douglas Bates 8 th International Amsterdam Conference on Multilevel Analysis 2011-03-16 Douglas Bates
More informationNotes on the Multivariate Normal and Related Topics
Version: July 10, 2013 Notes on the Multivariate Normal and Related Topics Let me refresh your memory about the distinctions between population and sample; parameters and statistics; population distributions
More informationLinear Regression. In this lecture we will study a particular type of regression model: the linear regression model
1 Linear Regression 2 Linear Regression In this lecture we will study a particular type of regression model: the linear regression model We will first consider the case of the model with one predictor
More informationANOVA Longitudinal Models for the Practice Effects Data: via GLM
Psyc 943 Lecture 25 page 1 ANOVA Longitudinal Models for the Practice Effects Data: via GLM Model 1. Saturated Means Model for Session, E-only Variances Model (BP) Variances Model: NO correlation, EQUAL
More informationDescribing Change over Time: Adding Linear Trends
Describing Change over Time: Adding Linear Trends Longitudinal Data Analysis Workshop Section 7 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section
More information20. REML Estimation of Variance Components. Copyright c 2018 (Iowa State University) 20. Statistics / 36
20. REML Estimation of Variance Components Copyright c 2018 (Iowa State University) 20. Statistics 510 1 / 36 Consider the General Linear Model y = Xβ + ɛ, where ɛ N(0, Σ) and Σ is an n n positive definite
More informationUnivariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation
Univariate Normal Distribution; GLM with the Univariate Normal; Least Squares Estimation EPSY 905: Multivariate Analysis Spring 2016 Lecture 3: February 10, 2016 Today s Class The building blocks: The
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Mixed Effects Estimation, Residuals Diagnostics Week 11, Lecture 1
MA 575 Linear Models: Cedric E Ginestet, Boston University Mixed Effects Estimation, Residuals Diagnostics Week 11, Lecture 1 1 Within-group Correlation Let us recall the simple two-level hierarchical
More informationDescribing Within-Person Fluctuation over Time using Alternative Covariance Structures
Describing Within-Person Fluctuation over Time using Alternative Covariance Structures Today s Class: The Big Picture ACS models using the R matrix only Introducing the G, Z, and V matrices ACS models
More informationCorrelation and Regression
Correlation and Regression October 25, 2017 STAT 151 Class 9 Slide 1 Outline of Topics 1 Associations 2 Scatter plot 3 Correlation 4 Regression 5 Testing and estimation 6 Goodness-of-fit STAT 151 Class
More informationEconomics 240A, Section 3: Short and Long Regression (Ch. 17) and the Multivariate Normal Distribution (Ch. 18)
Economics 240A, Section 3: Short and Long Regression (Ch. 17) and the Multivariate Normal Distribution (Ch. 18) MichaelR.Roberts Department of Economics and Department of Statistics University of California
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationMANOVA is an extension of the univariate ANOVA as it involves more than one Dependent Variable (DV). The following are assumptions for using MANOVA:
MULTIVARIATE ANALYSIS OF VARIANCE MANOVA is an extension of the univariate ANOVA as it involves more than one Dependent Variable (DV). The following are assumptions for using MANOVA: 1. Cell sizes : o
More informationGeneralized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science.
Texts in Statistical Science Generalized Linear Mixed Models Modern Concepts, Methods and Applications Walter W. Stroup CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint
More informationIntroduction to Factor Analysis
to Factor Analysis Lecture 10 August 2, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Lecture #10-8/3/2011 Slide 1 of 55 Today s Lecture Factor Analysis Today s Lecture Exploratory
More informationDesigning Multilevel Models Using SPSS 11.5 Mixed Model. John Painter, Ph.D.
Designing Multilevel Models Using SPSS 11.5 Mixed Model John Painter, Ph.D. Jordan Institute for Families School of Social Work University of North Carolina at Chapel Hill 1 Creating Multilevel Models
More informationRestricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model
Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives
More informationGeneral Principles Within-Cases Factors Only Within and Between. Within Cases ANOVA. Part One
Within Cases ANOVA Part One 1 / 25 Within Cases A case contributes a DV value for every value of a categorical IV It is natural to expect data from the same case to be correlated - NOT independent For
More informationRejection regions for the bivariate case
Rejection regions for the bivariate case The rejection region for the T 2 test (and similarly for Z 2 when Σ is known) is the region outside of an ellipse, for which there is a (1-α)% chance that the test
More informationMa 3/103: Lecture 24 Linear Regression I: Estimation
Ma 3/103: Lecture 24 Linear Regression I: Estimation March 3, 2017 KC Border Linear Regression I March 3, 2017 1 / 32 Regression analysis Regression analysis Estimate and test E(Y X) = f (X). f is the
More informationRegression With a Categorical Independent Variable
Regression ith a Independent Variable ERSH 8320 Slide 1 of 34 Today s Lecture Regression with a single categorical independent variable. Today s Lecture Coding procedures for analysis. Dummy coding. Relationship
More informationAGEC 621 Lecture 16 David Bessler
AGEC 621 Lecture 16 David Bessler This is a RATS output for the dummy variable problem given in GHJ page 422; the beer expenditure lecture (last time). I do not expect you to know RATS but this will give
More informationHypothesis Testing for Var-Cov Components
Hypothesis Testing for Var-Cov Components When the specification of coefficients as fixed, random or non-randomly varying is considered, a null hypothesis of the form is considered, where Additional output
More informationLecture 9: Linear Regression
Lecture 9: Linear Regression Goals Develop basic concepts of linear regression from a probabilistic framework Estimating parameters and hypothesis testing with linear models Linear regression in R Regression
More informationPart 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 12: Frequentist properties of estimators (v4) Ramesh Johari ramesh.johari@stanford.edu 1 / 39 Frequentist inference 2 / 39 Thinking like a frequentist Suppose that for some
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Addressing ourliers 1 Addressing ourliers 2 Outliers in Multivariate samples (1) For
More informationEstimation: Problems & Solutions
Estimation: Problems & Solutions Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline 1. Introduction: Estimation of
More informationApplication of Ghosh, Grizzle and Sen s Nonparametric Methods in. Longitudinal Studies Using SAS PROC GLM
Application of Ghosh, Grizzle and Sen s Nonparametric Methods in Longitudinal Studies Using SAS PROC GLM Chan Zeng and Gary O. Zerbe Department of Preventive Medicine and Biometrics University of Colorado
More informationThe Gaussian distribution
The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a
More informationCovariance Structure Approach to Within-Cases
Covariance Structure Approach to Within-Cases Remember how the data file grapefruit1.data looks: Store sales1 sales2 sales3 1 62.1 61.3 60.8 2 58.2 57.9 55.1 3 51.6 49.2 46.2 4 53.7 51.5 48.3 5 61.4 58.7
More information4.1 Computing section Example: Bivariate measurements on plants Post hoc analysis... 7
Master of Applied Statistics ST116: Chemometrics and Multivariate Statistical data Analysis Per Bruun Brockhoff Module 4: Computing 4.1 Computing section.................................. 1 4.1.1 Example:
More informationMean Vector Inferences
Mean Vector Inferences Lecture 5 September 21, 2005 Multivariate Analysis Lecture #5-9/21/2005 Slide 1 of 34 Today s Lecture Inferences about a Mean Vector (Chapter 5). Univariate versions of mean vector
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationChapter 6. Logistic Regression. 6.1 A linear model for the log odds
Chapter 6 Logistic Regression In logistic regression, there is a categorical response variables, often coded 1=Yes and 0=No. Many important phenomena fit this framework. The patient survives the operation,
More informationUNIVERSITY OF TORONTO Faculty of Arts and Science
UNIVERSITY OF TORONTO Faculty of Arts and Science December 2013 Final Examination STA442H1F/2101HF Methods of Applied Statistics Jerry Brunner Duration - 3 hours Aids: Calculator Model(s): Any calculator
More informationCAS MA575 Linear Models
CAS MA575 Linear Models Boston University, Fall 2013 Midterm Exam (Correction) Instructor: Cedric Ginestet Date: 22 Oct 2013. Maximal Score: 200pts. Please Note: You will only be graded on work and answers
More informationReview of CLDP 944: Multilevel Models for Longitudinal Data
Review of CLDP 944: Multilevel Models for Longitudinal Data Topics: Review of general MLM concepts and terminology Model comparisons and significance testing Fixed and random effects of time Significance
More informationAn Introduction to Multilevel Models. PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012
An Introduction to Multilevel Models PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012 Today s Class Concepts in Longitudinal Modeling Between-Person vs. +Within-Person
More informationChapter 9. Multivariate and Within-cases Analysis. 9.1 Multivariate Analysis of Variance
Chapter 9 Multivariate and Within-cases Analysis 9.1 Multivariate Analysis of Variance Multivariate means more than one response variable at once. Why do it? Primarily because if you do parallel analyses
More informationLecture 3: Linear Models. Bruce Walsh lecture notes Uppsala EQG course version 28 Jan 2012
Lecture 3: Linear Models Bruce Walsh lecture notes Uppsala EQG course version 28 Jan 2012 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector of observed
More informationSTAT5044: Regression and Anova
STAT5044: Regression and Anova Inyoung Kim 1 / 15 Outline 1 Fitting GLMs 2 / 15 Fitting GLMS We study how to find the maxlimum likelihood estimator ˆβ of GLM parameters The likelihood equaions are usually
More informationLecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
More informationMATH5745 Multivariate Methods Lecture 07
MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction
More informationLecture 6: Single-classification multivariate ANOVA (k-group( MANOVA)
Lecture 6: Single-classification multivariate ANOVA (k-group( MANOVA) Rationale and MANOVA test statistics underlying principles MANOVA assumptions Univariate ANOVA Planned and unplanned Multivariate ANOVA
More informationA Practitioner s Guide to Generalized Linear Models
A Practitioners Guide to Generalized Linear Models Background The classical linear models and most of the minimum bias procedures are special cases of generalized linear models (GLMs). GLMs are more technically
More informationMultivariate Time Series: VAR(p) Processes and Models
Multivariate Time Series: VAR(p) Processes and Models A VAR(p) model, for p > 0 is X t = φ 0 + Φ 1 X t 1 + + Φ p X t p + A t, where X t, φ 0, and X t i are k-vectors, Φ 1,..., Φ p are k k matrices, with
More informationDescribing Within-Person Change over Time
Describing Within-Person Change over Time Topics: Multilevel modeling notation and terminology Fixed and random effects of linear time Predicted variances and covariances from random slopes Dependency
More informationGeneralized Linear Models 1
Generalized Linear Models 1 STA 2101/442: Fall 2012 1 See last slide for copyright information. 1 / 24 Suggested Reading: Davison s Statistical models Exponential families of distributions Sec. 5.2 Chapter
More information1 Mixed effect models and longitudinal data analysis
1 Mixed effect models and longitudinal data analysis Mixed effects models provide a flexible approach to any situation where data have a grouping structure which introduces some kind of correlation between
More informationLinear Methods for Prediction
Chapter 5 Linear Methods for Prediction 5.1 Introduction We now revisit the classification problem and focus on linear methods. Since our prediction Ĝ(x) will always take values in the discrete set G we
More informationMultinomial Logistic Regression Models
Stat 544, Lecture 19 1 Multinomial Logistic Regression Models Polytomous responses. Logistic regression can be extended to handle responses that are polytomous, i.e. taking r>2 categories. (Note: The word
More information