Topic 7: Analysis of Variance

Transcription

1 Topc 7: Analyss of Varance

2 Outlne Parttonng sums of squares Breakdown the degrees of freedom Expected mean squares (EMS) F test ANOVA table General lnear test Pearson Correlaton / R 2

3 Analyss of Varance Organzes results arthmetcally Total sum of squares n Y s 2 SSTO Y Y Partton SST nto two sources Model (SS explaned by regresson lne) Error (unexplaned SS / varaton about means) Y Y Y Yˆ Yˆ Y

4 Total Sum of Squares SSTO/(n-1) s the usual estmate of the varance of Y f there s no explanatory varable SAS uses the term Corrected Total for ths source The term corrected means that we subtract off the sample mean Y before squarng Uncorrected SS s just the sum of squared observatons (ΣY 2 )

5 Sum of Squares Partton 2 Y Y Y ˆ ˆ Y Y Y SSTO = SSR + SSE Y ˆ Y 2 Yˆ ˆ ˆ Y Y Y Y Y Y ˆ Y Y ˆ Y Usng propertes of ftted regresson lne (pgs 23-24),ths term s zero

6 Model Sum of Squares ˆ 2 SSR= Y Y df R = 1 (due to the addton of the slope) MSR = SSR/df R = SSR KNNL uses regresson for what SAS calls model So SSR (KNNL) s the same as SS Model

7 Error Sum of Squares ˆ 2 SSE= Y -Y df E = n-2 (adjustng for slope and ntercept) MSE = SSE/df E MSE s an estmate of the varance of Y takng nto account (or condtonng on) the explanatory varable MSE=s 2 (our estmate of the Var(e))

8 ANOVA Table Source df SS MS 2 Regresson 1 Ŷ Y SSR/df R 2 Error n-2 Y -Y ˆ SSE/df E 2 Total n-1 Y -Y SSTO/df T

9 Expected Mean Squares MSR, MSE are random varables because they are functons of the Y s E(MSR) X X 1 2 E(MSE) When H 0 : β 1 = 0 s true E(MSR) =E(MSE) Both are unbased estmators of s 2

10 F test When H 0 : β 1 =0 s false, MSR tends to be larger than MSE F*=MSR/MSE ~ F(df R, df E ) = F(1, n-2) See KNNL pgs We reject H 0 when F* s large If F* F(1-α, df R, df E ) = F(.95, 1, n-2) In practce we use P-values to assess level of evdence aganst null

11 F test When H 0 : β 1 =0 s false, F* has a noncentral F dstrbuton Ths can be used to calculate power Recall t* = b 1 /s(b 1 ) tests H 0 : β 1 =0 It can be shown that (t*) 2 = F* (pg 71) Two approaches gve same P-value n smple lnear regresson

12 ANOVA Table Source df SS MS F P Model 1 SSM MSM MSM/MSE 0.## Error n-2 SSE MSE Total n-1 **Note: Model nstead of Regresson used here. More smlar to SAS

13 Examples Tower of Psa study (n=13 cases) proc reg data=a1; model lean=year; run; Toluca lot sze study (n=25 cases) proc reg data=toluca; model hours=lotsze; run;

14 Psa Output Number of Observatons Read 13 Number of Observatons Used 13 Analyss of Varance Source DF Sum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

15 Psa Output Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var (30.07) 2 =904.2 (roundng error) Parameter Estmates Varable DF Parameter Estmate Standard Error t Value Pr > t Intercept year <.0001

16 Toluca Output Number of Observatons Read 25 Number of Observatons Used 25 Analyss of Varance Source DF Sum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

17 Toluca Output Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var (10.29) 2 = Parameter Estmates Varable DF Parameter Estmate Standard Error t Value Pr > t Intercept lotsze <.0001

18 General Lnear Test A dfferent vew of the same problem Very useful n multple regresson Here, we want to compare two models Y = β 0 + β 1 X + e (full model) Y = β 0 + e (reduced model) Compare the two models usng ther error sum of squares better model wll have smaller mean square error

19 General Lnear Test Let SSE(F) = SSE for full model SSE(R) = SSE for reduced model F ( SSE(R)-SSE(F) ) ( df df ) SSE(F) df Compare wth F(1-α,df R -df F,df F ) F R F

20 Smple Lnear Regresson SSE(R) Y b Y Y SSTO SSE(F) SSE df R =n-1, df F =n-2, df R -df F =1 F=(SSTO-SSE)/MSE=SSR/MSE Same test as before Ths approach s more general and used frequently n multple regresson

21 Pearson Correlaton r s the usual correlaton coeffcent It s a number between 1 and +1 and measures the strength of the lnear relatonshp between two varables r ( X X)( Y Y) ( X X) ( Y Y) 2 2

22 Pearson Correlaton r does not dstngush between the response and predctor varables r ( X X)( Y Y) ( X X) ( Y Y) Can swtch X and Y varables and get same correlaton..not true f you swtch Y and X n regresson model 2 2

23 Pearson Correlaton Notce that r b ( ) X X 2 1 ( Y Y) 2 b ( s s ) 1 X Y Test H 0 : β 1 =0 smlar to H 0 : ρ=0

24 R 2 and r 2 r ( ) X X 2 2 b 2 1 ( Y Y) 2 SSR SSTO Rato of explaned and total varaton

25 R 2 and r 2 We use R 2 when the number of explanatory varables s arbtrary (smple and multple regresson) r 2 =R 2 only for smple regresson R 2 s often multpled by 100 and thereby expressed as a percent

26 R 2 and r 2 R 2 always ncreases when addtonal explanatory varables are added to the model Adjusted R 2 penalzes larger models so that t doesn t necessarly get larger We wll dscuss ths n detal later

27 Psa Output Analyss of Varance Source DF Sum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total R-Square (SAS) = SSM/SSTO = 15804/15997 =

28 Toluca Output Analyss of Varance Source DF Sum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total R-Square (SAS) = SSM/SSTO = / =

29 Background Readng May fnd 2.10 and 2.11 nterestng 2.10 provdes cautonary remarks Wll dscuss these as they arse 2.11 dscusses bvarate Normal dst Smlartes and dfferences Confdence nterval for r Program topc7.sas has the code to generate the ANOVA output Read Chapter 3