tatstcal Models Lecture nalyss of Varance wo-factor model Overall mean Man effect of factor at level Man effect of factor at level Y µ + α + β + γ + ε Eε f (, ( l, Cov( ε, ε ) lmr f (, nteracton effect of factor and at pectve levels and f γ 0, the two-factor model s called addtve Rats rat experment from the lterature. our types of medcaton and three types of poson; 48 rats actor one: type of medcaton, four levels (,,C or D) actor two: type of poson, three levels (, and ) here are 4 3 dfferent ments ow were these ments assgned to the 48 avalable rats? Each ment to 4 plots (four replcates) Completely randomzed factoral desgn M<-rep(c("","","","","","","C","C","C","D","D","D"),tmes4) P<-rep(c("", "", ""),tmes) desmat<-cbnd(m,p) num<-sample(:48,48) desgn<-desmat[nu] Least qua estmaton n LM two-factor model nalogous to one-factor dervaton: J α β ssume J γ γ 0 {,... J}, {,... } ˆ µ Y ˆ α Y Y,,,..., ˆ β Y Y,,,..., J ˆ γ Y Y Y + Y,,,..., and,,..., J Y ˆ µ + ˆ α + ˆ β + ˆ γ + e magne matrces and gve degrees of freedom JK + ( ) + ( J ) + ( )( J ) + J ( K ) 48 rats Rat example n R two-factor model > data(rats) > rats tme poson medc 0.3 0.8 3 0.43 C 4 0.4 D 0.4 M M M M 4 0. C 48 0.33 D > plot(tme~medc+poson,datarats) > nteracton.plot(rats$medc,rats$poson,rats$tme) > nteracton.plot(rats$poson,rats$medc,rats$tme) our medcaton types (,,C and D) hree types of poson (, and ) urvval tme n tens of wees our replcates of 3 4 factoral experment Plot of data nteracton plots tme mean of rats$tme Rat example n R two-factor model 0. 0.4 0. 0.8.0. 0. 0.4 0. 0.8 C D C D rats$ rats$poson tme mean of rats$tme 0. 0.4 0. 0.8.0. 0. 0.4 0. 0.8 poson rats$poson rats$ D C
Rat example two-factor model > rataov<-aov(tme~poson*medc,datarats) > model.tables(rataov) poson 0.38 0.0-0.03 rep.0000.000.0000 C D -0. 0.93-0.088 0.049 rep.0000.0000.00000.00000 poson: poson C D -0.040 0.0 0.03-0.0-0.09 0.03-0.08 0.08 0.099-0.39 0.04-0.00 re the nteracton effects needed n the model? man effects poson man effects ment nteracton effects Rat example two-factor model re the nteracton effects needed n the model? ddtve model can also be ftted: > rataddaov<-aov(tme~poson+,datarats) > model.tables(rataddaov) poson poson 0.383 0.000-0.033 C D -0. 0.99-0.088 0.049 + nstead of * man effects poson man effects ment nalyss of Varance ntetng questons n NOV models formulaton of hypotheses to be tested; full and trcted models Normal dstrbuton theory multvarate normal-, (non-) central χ -, tudent s t- and sher s (non-) central dstrbutons nalyss of varance n one- and two factor model test for absence of effect n one-factor model, for man effects n addtve two-factor model and for absence of nteractons n general two-factor model relevant NOV tables (summarzng test ults) nalyss of Varance ntetng hypotheses to test re the nteracton effects n the two-factor model zero or not? s there no dfference between n expected yeld dependng on the varous ments or s there? re the ment effects all zero or not? ormulate null and alternatve hypotheses n the model and perform statstcal test. n NOV: ult of test s summarzed n NOV table ypothess testng n one factor model ypothess testng n two factor model Y µ + α + ε,,...,,,..., J Eε : f (, ) (, Cov( ε, ε l ) f (, ) (, 0 : α α L α 0 Y µ + ε,,...,,,..., J Eε : f (, ) (, Cov( ε ε l ) f (, ) (, : not allα : : α Y µ + α + β + γ + ε Eε f (, ( l, Cov( ε, ε ) lmr f (, : β : γ,...,,..., J, n addtve model, no man effects n general two-factor model: no nteracton effects
estng hypotheses n NOV We have: general models () models under ntetng null hypotheses () n order to test the null hypothess we need: test statstc qualtatve shape of the reecton regon dstrbuton of the test statstc under the null hypothess ome normal dstrbuton theory ntermezzo: normal dstrbuton theory Defnton: random vector n R has a (multvarate) normal dstrbuton f and only f for each a n R Y a a s (unvarate) normally dstrbuted,.e., Y s a degenerate random varable or has probablty densty functon f Y ( y) exp y σ σ π ( ( µ ) ) Notaton: ~ ( µ, Σ) mean vector N covarance matrx ntermezzo: normal dstrbuton theory f ~ ( µ, Σ), then N ( µ, Σ ) N ~ f,, K, are stochastcally ndependent and N ( µ, σ ) then (, K, ) ~ N ( µ,dag( σ )) f ~ ( µ, Σ) and s a p matrx, then N ~ N ( µ, Σ ) p ntermezzo: normal dstrbuton theory Defnton: random varable has a central ch-square dstrbuton wth degrees of freedom f D where the are ndependent standard normal random varables. f ~ N ( µ,), then has a noncentral ch-square dstrbuton wth degrees of freedom and noncentralty parameter ν µ R.. sher 890-9 Notaton: χ or ~ ( ν ) ~ χ ntermezzo: normal dstrbuton theory f ~ χ ( ν ), then E +ν and var + 4ν f ~ N (0,) and, ndependent of ths, ~ χ then a student-t ~ t dstrbuton wth degrees of freedom uppose ~ N ( µ, σ ) and s a symmetrc, dempotent matrx wth ran r. hen wth ν µ µ σ ~ χ ( ν ) σ Lemma. n lecture notes ntermezzo: normal dstrbuton theory Defnton: random varable has a non-central dstrbuton wth degrees of freedom and and non-centralty parameter ν f D and and are ndependent. f ν0, has a central dstrbuton Notaton: or ( ) ~, where χ ~ ( ), ~ ~ ν, ν χ R.. sher 890-9 3
Resdual sum of squa towards test statstc and null hypothess dstrbuton Recall the general lnear model, now wth normalty assumpton on the error varables: hen the dual sum of squa of the L (ML) estmator satsfes: ˆ) Y β + ε ( 0, ) ε ~ N n σ n ˆ) n ( β σ ( Y β ( Y β σ ~ χ nran ( ) ˆ) ypothess testng n one factor model test statstc dea: Model ft under null hypothess wll have larger dual sum of squa than the ft under the general model. f the ft under the general model s much better, then the null hypothess s probably not true and should be reected Defne: : dual sum of squa under model : dual sum of squa under null hypothess model ( ) ( ) ~, n ( n ) under null hypothess (Lemma. n lecture notes) ypothess testng n one factor model test statstc Note that J ( Y ) Y - J J ( Y - ) {( ) ( ) } Y Y -Y + Y -Y + ( ) ( ) J Y -Y wthn groups sum of squa - J Y -Y between groups sum of squa Large value of means bg varaton between groups compared to varaton wthn groups ypothess testng n one factor model summarzng the test: NOV table factor Df dual n ow to do t n R? tr tr M ( ) ( n ) M tr M > summary(coaov) Df um q Mean q value Pr(>) det 3 8.0.0 3. 4.8e-0 *** Resduals 0.0. > model.tables(coaov) det C D -3 4-3 rep 4 8 > par(mfrowc(,)) > plot(coaov) p ypothess testng n one factor model Resduals - -4-0 4 Resduals vs tted 3 4 8 tted values tandardzed duals - - 0 Normal Q-Q plot - - 0 heoretcal Quantles dagnostcs n R ypothess testng n two factor model Y µ + α + β + γ + ε Eε : f (, ( l, Cov( ε, ε ) lmr f (, cale-locaton plot Coo's dstance plot tandardzed duals 0.0 0..0. 3 4 8 Coo's dstance 0.00 0.0 0.0 0 0 : α,,..., : β, : γ,,,..., J n addtve model, no man effect of factor n general two-factor model: no nteracton effects (model s addtve) tted values Obs. number 4
estng n addtve two factor model : Y µ + α + β + ε Eε f (, ( l, Cov( ε, ε ) lmr f (, : α,,..., est statstc agan based on comparson of two sums of squa: : dual sum of squa under model : dual sum of squa under null hypothess estng n addtve two factor model why -dstrbuton of test statstc? Usng smlar arguments as n the one-factor model: σ σ ~ χ ( - ) n-( + J - ) σ ~ χ - Under null hypothess ( ) ( ) ( ) ~, n( + J ) ( n ( + J )) ( n ( + J )) ( ) ( ) ( ) ~, n( + J ( n ( + J )) ( n ( + J )) ) estng n addtve two factor model summarzng the test: NOV table estng n addtve two factor model table of parameter estmates factor factor Df J dual n ( + J ) ow to do t n R? M ( ) ( J ) ( n ( + J )) M M M M > rataddaov<-aov(tme~poson+medc,datarats) > summary(rataddaov) Df um q Mean q value Pr(>) poson.03 0. 0.43.04e-0 *** medc 3 0.9 0.0.3.9e-0 *** Resduals 4.008 0.00 p > rataddaov<-aov(tme~poson+medc,datarats) > model.tables(rataddaov) poson 0.383 0.000-0.033 medc C D -0. 0.99-0.088 0.049 estng n addtve two factor model dagnostcs n R? Resduals -0. 0.0 0. 0.4 Resduals vs tted 0. 0. 0.3 0.4 0. 0. 0. 0.8 4 tted values tandardzed duals - 0 3 Normal Q-Q plot - - 0 heoretcal Quantles 4 ypothess testng n two factor model Y µ + α + β + γ + ε Eε : f (, ( l, Cov( ε, ε ) lmr f (, tandardzed duals 0.0 0..0. cale-locaton plot 4 Coo's dstance 0.00 0.0 0.0 0. Coo's dstance plot 4 : α,,..., : β, : γ,,,..., J n addtve model, no man effect of factor n general two-factor model: no nteracton effects (model s addtve) 0. 0. 0.3 0.4 0. 0. 0. 0.8 0 0 0 40 tted values Obs. number
estng for nteracton n two factor model : Y µ + α + β + γ + ε Eε f (, ( l, Cov( ε, ε ) lmr f (, : γ,, est statstc agan based on comparson of two sums of squa: : dual sum of squa under model : dual sum of squa under null hypothess Under null hypothess ( J J ) ( )( ) ( )( ) ~ ( )( J ), nj ( n J )) ( n J ) estng n addtve two factor model summarzng the test: NOV table factor factor nteracton ( )( J ) dual Df J n J ow to do t n R? M ( ) ( J ) ( )( J ) M ( n J) M M M M M > rataov<-aov(tme~poson*medc,datarats) > summary(rataov) Df um q Mean q value Pr(>) poson.03 0. 3. 3.33e-0 *** medc 3 0.9 0.0 3.80 3.e-0 *** poson:medc 0.04 0.049.843 0.3 Resduals 3 0.8003 0.04 p estng n addtve two factor model > model.tables(rataov) poson 0.38 0.0-0.03 rep.0000.000.0000 medc C D -0. 0.93-0.088 0.049 rep.0000.0000.00000.00000 poson:medc medc poson C D -0.040 0.0 0.03-0.0-0.09 0.03-0.08 0.08 0.099-0.39 0.04-0.00 table of parameter estmates nalyss of Varance summary Dfference n expected yeld two sample t-test and equvalent test L estmaton n the general lnear model and NOV models ncdence matrx, categorcal explanatory varables varables, normal equatons, lnear constrants Normal dstrbuton theory multvarate normal-, (non-) central χ -, tudent s t- and sher s (non-) central dstrbutons nalyss of varance n one- and two factor model test for absence of effect n one-factor model, for man effects n addtve two-factor model and for absence of nteractons n general two-factor model nalyss of Varance possble extensons Random effect models when effects are assumed to vary wthn groups and the conclusons are to be vald for the whole group effect-parameters modeled as random varables Mxed effect models fxed- and random effect factors