8// The General Linear Model Monday, Lecture Jeanette Mumford University of Wisconsin - Madison How we re approaching the GLM Regression for behavioral data Without using matrices Understand least squares Using matrices With more than regressor, you need this What you ll get out of this What is least squares? What is a residual? How do you multiply a matrix and a vector? What are degrees of freedom? How do you obtain the estimates for the GLM using matrix math including the variance
Reaction Time (s) 8// Do you remember the equation for a line? Do you remember the equation for a line? y=b+mx Do you remember the equation for a line? RT i + + Age i Age
Reaction Time (s) Reaction Time (s) 8// Do you remember the equation for a line? population mean RT i = + Age i Age Do you remember the equation for a line? RT i = + Age i fit isn t perfect, so we must account for error Age The Model For the i th observational unit : The dependent (random) variable : Independent variable (not random), : Model parameters : Random error, how the observation deviates from the population mean
8// Fixed: Mean of, ( ) Random: Variability of It follows that the variance of is Fixed: Mean of, ( ) Random: Variability of It follows that the variance of is Fixed: Mean of, ( ) Random: Variability of It follows that the variance of is
8// Simple summary mean(y i ) var(y i ) Fitting the Model Reaction Time (s) Q: Which line fits the data best? Age Fitting the Model Reaction Time (s) Minimize the distance between the data and the line (error). Absolute distance? squared distance? Error term Age 5
8// Least Squares Minimize squared differences Minimize Least Squares Minimize squared differences Minimize Works out nicely distribution-wise You can use calculus to get the estimates Bias and Variance
8// Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance
8// Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance Property of least squares Gauss Markov Assumptions error has mean things aren t correlated variance is the same for all observations Unbiased and have lowest variance among all unbiased estimators Property of least squares Gauss Markov Assumptions error has mean things aren t correlated variance is the same for all observations Unbiased and have lowest variance among all unbiased estimators 8
8// What about the variance? We also need an estimate for Start with the sums of squared error Divide by the appropriate degrees of freedom # of independent pieces of information - # parameters in model What about the variance? We also need an estimate for Start with the sums of squared error Divide by the appropriate degrees of freedom # of independent pieces of information - # parameters in model Take away up to this point We use typically use least squares estimation to estimate the betas in regression Gauss Markov Minimum variance among all unbiased estimators 9
8// You don t need to do regression this way Anybody ever hear of using absolute error instead of squared error? Do you know the context?? Anybody ever hear of purposely biasing (!) an estimate in order to reduce variability? Do you know the context? Multiple Linear Regression Add more parameters to the model Time for linear algebra! is a x matrix Matrices Row index Column index
8// Matrices Square matrix- Same # of rows and columns Vector- column(row) vector has column(row) Matrices Transpose: or. Swap columns and rows. Element-wise addition and subtraction Matrices Multiplication: Trickier Number of columns of first matrix must match number of rows of second matrix
8// Matrices Multiplication Matrices Multiplication x+ Matrices Multiplication x+x=
8// Matrices Multiplication x+x= Multiplication Matrices You try it out B @ C A =??
8// You try it out B B @ C C A You try it out B B @ C C A B B @ C C A You try it out B B @ C C A
8// Matrix Inverse Denoted Only for square matrices Only exists if matrix is full rank All columns (rows) are linearly independent, but I ll spare the details Rank Deficient Matrices *column=column column+column=column Rank Deficient Matrices *column=column column+column=column SPM can handle rank deficiency, if the contrasts are specified properly 5
8// Can you find the rank deficiency?? Inverting rectangular matrix If the columns *only* are linearly independent, then is invertible Pseudoinverse: Inverting a rank-deficient matrix I m not going to get into the nitty gritty pinv() in MATLAB does it You *must* be careful if you go this route on your own Could accidentally do something silly, but SPM seems to have built in controls so you don t
8// Inverting a rank-deficient matrix Back to linear regression (nx) (nx) (x) (nx) Back to linear regression (nx) (nx) (x) (nx)
8// Back to linear regression (nx) (nx) (x) (nx) Back to linear regression (nx) (nx) (x) (nx) Back to linear regression (nx) (nx) (x) (nx) 8
8// Viewing the Design Matrix Look at the actual numbers M F age Viewing the Design Matrix Look at in image representation Darker=smaller # M F age Multiple Linear Regression The distribution of Y is a multivariate Normal 9
8// Multiple Linear Regression is really easy to derive Multiple Linear Regression is really easy to derive Same as least squares, but much easier to understand and write code for thanks linear algebra! Multiple Linear Regression where N=length(Y) p=length( )
8// Multiple Linear Regression where N=length(Y) p=length( ) Or Rank(X) Statistical Properties So the estimate is unbiased But we don t know Take away Matrix algebra makes GLM estimation waaay easier Make sure you re comfortable multiplying a matrix and a vector Handy to know how to estimate the parameters
8// Ask me some questions Recall GLM is flexible One Sample T Test Two sample T Test Paired T test ANOVA ANCOVA What do the models look like? Do you know the answers? What is least squares? What is a residual? How do you multiply a matrix and a vector? What are degrees of freedom? How do you obtain the estimates for the GLM using matrix math including the variance
8// Let s set up some simple models - sample t- test - sample t- test With contrasts! - sample t- test Y = + Y Y. Y N - sample t- test 5 =. 5 +
8// Y Y. - sample t- test 5 =. 5 + Y N MulBply out the right hand side Y Y. - sample t- test 5 = Y N MulBply out the right hand side. 5 + Y Y. - sample t- test 5 = Y N MulBply out the right hand side. 5 +
8// But why is it the mean and not something else? But why is it the mean and not something else? Because we re using least squares! I m going to write this out 5
8// Two- sample t- test There are at least ways I can think of parameterizing this! Start with the easiest a person is either in group or in group Y i = {sub i in group?} + {sub i in group?} + Two- sample t- test There are at least ways I can think of parameterizing this! Start with the easiest a person is either in group or in group Y i = G i + G i + group indicator variables Two- sample t- test Group Group Y Y Y Y Y 5 Y = 5 apple 5 +
8// Two- sample t- test Y Y Y Y Y 5 Y = 5 + 5 Two- sample t- test Y Y Y Y Y 5 Y = 5 + 5 mean for Group mean for Group Two- sample t- test (another way) Now you do it. Unwrap what this means Y i = + {subject i in Group } +
8// Contrasts Contrasts are vectors that pull out what we d like to test Using the two sample t- test from the first example, we might test Is the mean of G larger than? Is the mean of G larger than? Is the mean of G > G? General idea Take your contrast statement and get it to look like something > Figure out the vector, c, such that cb = something Y Y Y Y Y 5 Is group >? = 5 Y We ve already established the first beta represents group s mean c = [, ] pulls out the first beta apple 5 + 8
8// Y Y Y Y Y 5 Is group >? = 5 Y We ve already established the second beta represents group s mean c = [, ] pulls out the first beta apple 5 + Is group > group? Y First, get something > group group > c = [, - ] Y Y Y Y Y 5 = 5 apple 5 = apple + Is group > group? Y First, get something > group group > c = [, - ] Y Y Y Y Y 5 = 5 apple 5 = apple + 9
8// Is group > group? Y First, get something > group group > c = [, - ] Y Y Y Y Y 5 = 5 apple 5 = apple + Is group > group? Y First, get something > group group > c = [, - ] Y Y Y Y Y 5 = 5 apple 5 = apple + Can you do this for the second setup of the - sample t- test?
8// Take away Did you feel preey confident with the last example? Yes = Yay! No = Ask quesbons! That s it! I m guessing we need to at least stretch our legs right now