Data Skills 08: General Linear Model

Transcription

1 Data Skills 08: General Linear Model Dale J. Barr University of Glasgow Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 1 / 21

2 What is the General Linear Model (GLM)? Definition (General Linear Model or GLM) A general mathematical framework for expressing relationships among variables Differs from the cookbook approach to statistics t-test, ANOVA, ANCOVA, χ 2 test, regression, correlation, etc. Can express/test linear relationships between a numerical dependent variable and any combination of independent variables (categorical or continuous) Can even be generalized to categorial dependent variables (through Generalized Linear Models ; NB: advanced) Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 2 / 21

3 ANOVA, Regression, ANCOVA Fig 1a. Cholesterol levels by Fig 1a. Cholesterol levels by ethnic group and gender age. (male=sqr, female=tri). Fig 1a. Cholesterol levels by age and gender. Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 3 / 21

4 How the GLM represents relationships Component of GLM Notation DV Y Grand Average µ mu Main Effects A, B, C,... Interactions AB, AC, BC, ABC,... Random Error S(Group) Score = Grand Avg. + Main Effects + Interactions + Error Y = µ + A + B + C AB + AC + BC + ABC S(Group) Components of the model are estimated from the observed data Tests are performed ( F ) to see whether its variability is too large to be introduced by chance Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 4 / 21

5 An example: Simple Linear Regression Y i = µ + b X i + e i Score i = Baseline + Slope Hours i + Error i Y i = X i + e i e i N(µ = 0, σ 2 = 10) Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 5 / 21

6 Making comparisons across groups Example (Spelling) You wish to compare the benefits of three different spelling programs. Do these programs yield differences in spelling performance? H 0 : µ 1 = µ 2 = µ 3 Factors and Levels Factor: a categorical variable that is used to divide subjects into groups, usually to draw some comparison. Factors are composed of different levels. Do not confuse factors with levels! Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 6 / 21

7 Means, Variability, and Deviation Scores A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 7 / 21

8 Means, Variability, and Deviation Scores Y.. = ij Y ij N A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 8 / 21

9 Means, Variability, and Deviation Scores grand mean Y.. = SD Y = ij(y ij Y..) 2 ij Y ij N deviation score: Y ij Y.. N A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 9 / 21

10 GLM for One-Factor ANOVA Y ij = µ A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 10 / 21

11 GLM for One-Factor ANOVA Y ij = µ + A i A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 10 / 21

12 GLM for One-Factor ANOVA Y ij = µ + A i + S(A) ij A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 10 / 21

13 GLM for One-Factor ANOVA Y ij = µ + A i + S(A) ij Estimation Equations ˆµ = Y..  i = Y i. ˆµ A B Ŝ(A) ij = Y ij ˆµ Âi Note that i  i = 0 and ij Ŝ(A) ij = 0 Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 10 / 21

14 Sources of Variance Y ij = µ + A i + S(A) ij Y ij µ = A i + S(A) ij individual = group + random Sum of Squares (SS) A measure of variability consisting of the sum of squared deviation scores, where a deviation score is a score minus a mean. SS A = (Y i. µ) 2 A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 11 / 21

15 Decomposition Matrix ˆµ =  1 = = 20  2 = 97 = 3  3 = 83 = 17 A B Y ij = ˆµ +  i + Ŝ(A) ij 124 = = = = = = = = = = = = SS = = Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 12 / 21

16 Logic of ANOVA Compare two estimates of the variability, the between-group estimate (SS_{between}) and the within-group estimate (SS_{within}) If H 0 : µ 1 = µ 2 = µ 3 is true, then these two measures estimate the same quantity. The extent to which the between-group variability exceeds the within-group variability gives evidence against H 0 : µ 1 = µ 2 = µ 3. A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 13 / 21

17 Calculating SS_{between} and SS_{within} Y ij = ˆµ +  i + Ŝ(A) ij 124 = = = = = = = = = = = = SS = = check your math A B SS Y = SS µ + SS A + SS S(A) Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 14 / 21

18 H 0 and Sums of Squares Y ij µ = A i + S(A) ij A SS A = 2792 SS S(A) = 526 SS A + SS S(A) = 3318 B SS A = 266 SS S(A) = 3052 SS A + SS S(A) = 3318 A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 15 / 21

19 Mean Square and Degrees of Freedom A B Degrees of Freedom (df) The number of observations that are free to vary. df A = K 1 df S(A) = N K where N is the number of subjects and K is the number of groups. Mean Square (MS) A sum of squares divided by its degrees of freedom. MS A = SS A df A = = 1396 MS S(A) = SS S(A) df = 526 S(A) 9 = 58.4 Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 16 / 21

20 The F -ratio F density function Relative Freq df(num,denom)=2,9 F ratio A ratio of mean squares, with df_{numerator} and df_{denominator} degrees of freedom. F A = MS A MS = 1396 S(A) 58.4 = F_crit= F value If F obs > F crit, then reject H 0 Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 17 / 21

21 Density/Quantile functions for F-distribution name function pf(x, df1, df2, lower.tail = FALSE) density (get p given F obs ) qf(p, df1, df2, lower.tail = FALSE) quantile (get F crit given p) Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 18 / 21

22 Summary Table A Source df SS MS F p Error µ <.001 S(A) A <.001 S(A) S(A) Total B Source df SS MS F p Error µ <.001 S(A) A S(A) S(A) Total A B Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 19 / 21

23 Overview of One-Way ANOVA 1 Write the GLM: Y ij = µ + A i + S(A) ij 2 Write down the estimating equations: ˆµ = Y.. Âi = Y i. ˆµ Ŝ(A)ij = Y ij ˆµ Âi 3 Compute estimates for all terms in model. 4 Create decomposition matrix. 5 Compute SS, MS, df. dfµ = 1 df A = K 1 df S(A) = N K MS = SS/df 6 Construct a summary ANOVA table. 7 Compare F_{obs} with F_{crit}. R use the aov() function, e.g.: 1 spelling$a <- factor(spelling$a) 2 mod <- aov(y ~ A, data = spelling) 3 summary(mod) onefactoranova.html#sec-3-2 Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 20 / 21

24 Other GLMs one-sample t-test Y i c = β 0 + e i two-sample t-test Y i = β 0 + β 1 X i + e i where X i (0, 1) paired-samples t-test Y 1i Y 2i = µ + e i simple linear regression Y i = β 0 + β 1 X i + e i multiple regression Y i = β 0 + β 1 X 1i + β 2 X 2i + e i ANCOVA Y i = β 0 + β 1 X 1i + β 2 X 2i + β 3 X 1i X 2i + e i where X 1i (0, 1) and X 2i R Dale J. Barr Data Skills 08: General Linear Model University of Glasgow 21 / 21