The General Linear Model. Monday, Lecture 2 Jeanette Mumford University of Wisconsin - Madison

Transcription

1 The General Linear Model Monday, Lecture 2 Jeanette Mumford University of Wisconsin - Madison

2 How we re approaching the GLM Regression for behavioral data Without using matrices Understand least squares Using matrices With more than 1 regressor, you need this

3 What you ll get out of this What is least squares? What is a residual? How do you multiply a matrix and a vector? What are degrees of freedom? How do you obtain the estimates for the GLM using matrix math including the variance

4 Do you remember the equation for a line?

5 Do you remember the equation for a line? y=b+mx

6 Reaction Time (s) Do you remember the equation for a line? RT i Age i 1 Age

7 Reaction Time (s) Do you remember the equation for a line? population mean RT i = 0 + Age i 1 Age

8 Reaction Time (s) Do you remember the equation for a line? RT i = 0 + Age i 1 fit isn t perfect, so we must account for error Age

9 The Model For the i th observational unit : The dependent (random) variable : Independent variable (not random), : Model parameters : Random error, how the observation deviates from the population mean

10 Simple summary mean(y i ) var(y i )

11 Reaction Time (s) Fitting the Model Q: Which line fits the data best? Age

12 Fitting the Model Reaction Time (s) Minimize the distance between the data and the line (error). Absolute distance? squared distance? Error term Age

13 Least Squares Minimize squared differences Minimize

14 Least Squares Minimize squared differences Minimize Works out nicely distribution-wise Easy minimization problem

15 Bias and Variance

16 Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance

17 Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance

18 Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance

19 Bias and Variance high bias / low variance low bias / high variance high bias / high variance low bias / low variance

20 Property of least squares Gauss Markov Assumptions error has mean 0 things aren t correlated variance is the same for all observations Unbiased and have lowest variance among all unbiased estimators

21 Property of least squares Gauss Markov Assumptions error has mean 0 things aren t correlated variance is the same for all observations Unbiased and have lowest variance among all unbiased estimators

22 What about the variance? We also need an estimate for Start with the sums of squared error Divide by the appropriate degrees of freedom # of independent pieces of information - # parameters in model

23 What about the variance? We also need an estimate for Start with the sums of squared error Divide by the appropriate degrees of freedom # of independent pieces of information - # parameters in model

24 Take away up to this point We use typically use least squares estimation to estimate the betas in regression Gauss Markov Minimum variance among all unbiased estimators

25 You don t need to do regression this way Anybody ever hear of using absolute error instead of squared error? Do you know the context?? Anybody ever hear of purposely biasing (!) an estimate in order to reduce variability? Do you know the context?

26 Multiple Linear Regression Add more parameters to the model Time for linear algebra!

27 Matrices is a 2x3 matrix Row index Column index

28 Matrices Square matrix- Same # of rows and columns Vector- column(row) vector has 1 column(row)

29 Matrices Transpose: or. Swap columns and rows. Element-wise addition and subtraction

30 Matrices Multiplication: Trickier Number of columns of first matrix must match number of rows of second matrix

31 Multiplication Matrices

32 Matrices Multiplication 1x4+

33 Matrices Multiplication 1x4+2x1=6

34 Matrices Multiplication 1x2+2x4=10

35 Multiplication Matrices

36 You try it out C A =??

37 You try it out C A

38 You try it out C A C A

39 You try it out C A 0 1

40 Matrix Inverse Denoted Only for square matrices Only exists if matrix is full rank All columns (rows) are linearly independent, but I ll spare the details

41 Rank Deficient Matrices 2*column1=column3 column1+column2=column3

42 Rank Deficient Matrices 2*column1=column3 column1+column2=column3 SPM can handle rank deficiency, if the contrasts are specified properly

43 Can you find the rank deficiency??

44 Inverting rectangular matrix If the columns *only* are linearly independent, then is invertible Pseudoinverse:

45 Back to linear regression (nx1) (nx4) (4x1) (nx1)

46 Back to linear regression (nx1) (nx4) (4x1) (nx1)

47 Back to linear regression (nx1) (nx4) (4x1) (nx1)

48 Back to linear regression (nx1) (nx4) (4x1) (nx1)

49 Back to linear regression (nx1) (nx4) (4x1) (nx1)

50 Viewing the Design Matrix Look at the actual numbers M F age

51 Viewing the Design Matrix Look at in image representation Darker=smaller # M F age

52 Multiple Linear Regression The distribution of Y is a multivariate Normal

53 Multiple Linear Regression is really easy to derive

54 Multiple Linear Regression is really easy to derive Same as least squares, but much easier to understand and write code for thanks linear algebra!

55 Multiple Linear Regression where N=length(Y) p=length( )

56 Multiple Linear Regression where N=length(Y) p=length( ) Or Rank(X)

57 Statistical Properties So the estimate is unbiased But we don t know

58 Take away Matrix algebra makes GLM estimation waaay easier Make sure you re comfortable multiplying a matrix and a vector Handy to know how to estimate the parameters

59 Ask me some questions

60 Do you know the answers? What is least squares? What is a residual? How do you multiply a matrix and a vector? What are degrees of freedom? How do you obtain the estimates for the GLM using matrix math including the variance

61 Questions??