Chapter Analyss of Covarance Any scentfc experment s performed to know somethng that s unknown about a group of treatments and to test certan hypothess about the correspondng treatment effect When varablty of expermental unts s small relatve to the treatment dfferences and the expermenter do not wshes to use expermental desgn, then ust take large number of observatons on each treatment effect and compute ts mean he varaton around mean can be made as small as desred by takng more observatons When there s consderable varaton among observatons on the same treatment and t s not possble to take an unlmted number of observatons, the technues used for reducng the varaton are ( use of proper expermental desgn and ( use of concomtant varables he use of concomtant varables s accomplshed through the technue of analyss of covarance If both the technues fal to control the expermental varablty then the number of replcatons of dfferent treatments (n other words, the number of expermental unts are needed to be ncreased to a pont where adeuate control of varablty s aaned Introducton to analyss of covarance model In the lnear model Y = Xβ+ Xβ + + X p β p + ε, f the explanatory varables are uanatve varables as well as ndcator varables, e, some of them are ualtatve and some are uanatve, then the lnear model s termed as analyss of covarance (ANCOVA model Note that the ndcator varables do not provde as much nformaton as the uanatve varables For example, the uanatve observatons on age can be converted nto ndcator varable Let an ndctor varable be Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur
f age 7 years D = 0 f age < 7 years Now the followng uanatve values of age can be changed nto ndcator varables Ages (years Ages 4 0 5 0 6 0 7 0 In many real applcaton, some varables may be uanatve and others may be ualtatve In such cases, ANCOVA provdes a way out It helps s reducng the sum of suares due to error whch n turn reflects the beer model adeuacy dagnostcs See how does ths work: In one way model : Y = µ + α + ε, we have SS = SSA + SS In two way model : Y = µ + α + β + ε, we have SS = SSA + SSB + SS In three way model : Y = µ + α + β + γ + ε, we have SS = SSA + SSB + SSγ + SS k k If we have a gven data set, then deally SS = SS = SS3 SSA = SSA = SSA3; SSB = SSB 3 So SS SS SS3 3 3 3 3 3 Note that n the constructon of F -statstcs, we use So F -statstc essentally depends on the SSs Smaller SS large F more chance of reecton SS( effects/ df SS / df Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur
Snce SSA, SSB etc here are based on dummy varables, so obvously f SSA, SSB, etc are based on uanatve varables, they wll provde more nformaton Such deas are used n ANCOVA models and we construct the model by ncorporatng the uanatve explanatory varables n ANOVA models In another example, suppose our nterest s to compare several dfferent knds of feed for ther ablty to put weght on anmals If we use ANOVA, then we use the fnal weghts at the end of experment However, fnal weghts of the anmals depend upon the ntal weght of the anmals at the begnnng of the experment as well as upon the dfference n feeds Use of ANCOVA models enables us to adust or correct these ntal dfferences ANCOVA s useful for mprovng the precson of an experment Suppose response Y s lnearly related to covarate X (or concomtant varable Suppose expermenter cannot control X but can observe t ANCOVA nvolves adustng Y for the effect of X If such an adustment s not made, then the X can nflate the error mean suare and makes the true dfferences s Y due to treatment harder to detect If, for a gven expermental materal, the use of proper expermental desgn cannot control the expermental varaton, the use of concomtant varables (whch are related to expermental materal may be effectve n reducng the varablty Consder the one way classfcaton model as Y ( = β =,, p, =,, N, Var Y ( = σ If usual analyss of varance for testng the hypothess of eualty of treatment effects shows a hghly sgnfcant dfference n the treatment effects due to some factors affectng the experment, then consder the model whch takes nto account ths effect Y ( = β + γt =,, p, =,, N, Var( Y = σ where t are the observatons on concomtant varables (whch are related to X and γ s the regresson coeffcent assocated wth t Wth ths model, the varablty of treatment effects can be consderably reduced Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 3
For example, n any agrcultural expermental, f the expermental unts are plots of land then, t can be measure of fertlty characterstc of the th plot recevng th treatment and X can be yeld In another example, f expermental unts are anmals and suppose the obectve s to compare the growth rates of groups of anmals recevng dfferent dets Note that the observed dfferences n growth rates can be arbuted to det only f all the anmals are smlar n some observable characterstcs lke weght, age etc whch nfluence the growth rates In the absence of smlarty, use t whch s the weght or age of th anmal recevng th treatment If we consder the uadratc regresson n t then n Y t t p n ( = β + γ + γ, =,,, =,,, ( = σ Var Y ANCOVA n ths case s the same as ANCOVA wth two concomtant varables t and t In two way classfcaton wth one observaton per cell, Y ( = µ + α + β + γt, =,, I, =,, J or Y ( = µ + α + β + γ t + γ w wth α = 0, β = 0, then ( y, t or ( y, t, w are the observatons n (, cell and t, w are the concomtment varables th he concomtant varables can be fxed on random We consder the case of fxed concomtant varables only Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 4
One way classfcaton Let Y ( = n, = p be a random sample of sze n from th normal populatons wth mean µ = Y ( = β + γt Var( Y = σ where β, γ and a concomtant varable σ are the unknown parameters, t are known constants whch are the observatons on he null hypothess s H0 : β = = βp Let y = y ; y = y, y = y n p n o o t = t ; t = t, t = t n p n o o n= n Under the whole parametrc space ( π Ω, use lkelhd rato test for whch we obtan the ˆ β ' s and ˆ γ usng the least suares prncple or maxmum lkelhd estmaton as follows: Mnmze S = ( y S = 0 for fxed γ β β = y γt o o = ( y β γt S Put β n S and mnmze the functon by = 0, γ e,mnmze y yo γ( t to wth respect to γ gves ( y yo ( t to ˆ= γ ( t t hus ˆ β ˆ = yo γto ˆ µ = ˆ β + ˆ γt o Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 5
Snce y ˆ µ = y ˆ β ˆ γt = y y ˆ( γ t t, o o we have ( y y ( t t o o ( y ˆ = ( y yo ( t to Under H0 : β = = βp = β (say, consder w = β γ and mnmze w S y t S under sample space ( π w, Sw = 0, β Sw = 0 γ ˆ β = y ˆ γt ( y y( t t ˆ γ = ( t t ˆ µ = ˆ β + ˆ γt Hence ( y y ( t t ( y ˆ = ( y y ( t t and ( ( ( ˆ( ˆ µ ˆ ˆ ˆ = y y + γ t to γ t t he lkelhd rato test statstc n ths case s gven by Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 6
max L( βγσ,, w λ = max L( βγσ,, = Ω ( ˆ µ ˆ µ ( y ˆ µ Now we use the followng theorems: heorem : Let Y ( Y, Y,, Y n = follow a multvarate normal dstrbuton N ( µ, Σ wth mean vector µ and postve defnte covarance matrx Σ hen Y AY follows a noncentral ch-suare dstrbuton wth p degrees of freedom and noncentralty parameter µ Aµ, e, χ ( p, µ Aµ f and only f Σ A s an dempotent matrx of rank p heorem : Let Y = ( Y, Y,, Y n follows a multvarate normal dstrbuton N ( µ, wth mean vector µ and postve defnte covarance matrx Σ Let YAY follows χ ( p, µ Aµ and YAY follows χ ( p, µ A µ hen YAY and YAY are ndependently dstrbuted f AΣ A = 0 heorem 3: Let Y = ( Y, Y,, Y n follows a multvarate normal dstrbuton N (, I µσ, then the maxmum lkelhd (or least suares estmator L ˆ β of estmable lnear parametrc functon s ndependently dstrbuted of ˆ σ ; L ˆ β follow β and nσˆ σ N L, L( XX L follows χ ( n p where rank( X = p Usng these theorems on the ndependence of uadratc forms and dvdng the numerator and denomnator by respectve degrees of freedom, we have ( ˆ µ ˆ n p F = p ( y ˆ µ ~ F( p, n p under H So reect H 0 whenever F F α ( p, n p at α level of sgnfcance he terms nvolved n λ can be smplfed for computatonal convenence follows: 0 We can wrte Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 7
( y ˆ µ = y ˆ ˆ β γ t = ( y ˆ y γ ( t t = ( y ˆ( ˆ( ˆ( ˆ y γ t t + γ t to γ t to = ( ˆ( ˆ y yo γ t to = ( y ˆ( ˆ y + γ t to γ( t t y ˆ ˆ ˆ µ µ µ = ( + ( For computatonal convenence ( ˆ µ ˆ ( y ˆ λ = = where = ( y y = ( t t = ( y y ( t t = ( y y o = ( t t o = ( y y ( t t o o Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 8
Analyss of covarance table for one way classfcaton s as follows: Source of varaton Populaton rror Degrees of freedom p n p Sum of products P ( = P ( = P ( = Adusted sum of suares Degress of feedom Sum of suares p = 0 n p = F n p p otal n n 0 = If H 0 s reected, employ multple comprses methods to determne whch of the contrasts n β are responsble for ths For any estmable lnear parametrc contrast p ϕ = Cβ wth C = 0, = = p p p ˆ ϕ = C ˆ β = Cy ˆ γ C t = = = p Var( ˆ γ = σ ( t t Var = + Ct C ( ˆ ϕ σ n ( t t Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 9
wo way classfcaton (wth one observatons per cell Consder the case of two way classfcaton wth one observaton per cell Let ~ (, y N µ σ be ndependently dstrbuted wth ( y = µ + α + β + γt, = I, = J V( y = σ where µ : Grand mean α : ffect of β : ffect of I th level of A satsfyng α = 0 J th level of B satsfyng β = 0 t : observaton (known on concomtant varable he null hypothess under consderaton are H0 α : α = α = = αi = 0 H : β = β = = β = 0 0β J Dmenson of whole parametrc space ( π Ω :I + J Dmenson of sample space ( π wα : J + under 0 H α Dmenson of sample space ( π wβ : I + under 0 wth respectve alternatve hypotheses as H α : At least one par of α 's s not eual H β : At least one par of β 's s not eual Consder the estmaton of parameters under the whole parametrc space ( π Ω H β Fnd mnmum value of ( y under π Ω o do ths, mnmze ( y α β γ t For fxed γ, whch gves on solvng the least suares estmates (or the maxmum lkelhd estmates of the respectve parameters as Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 0
µ = y γt o α = y y γ( t t ( o β = y y γ( t t o o Under these values of µα, and β, the sum of suares ( y µ α β γt reduces to y yo yo + y + γ ( t to to + t ( Now mnmzaton of ( wth respect to γ gves ˆ γ = I J = = ( y y y + y ( t t t + t o o o o I J = = Usng ˆ, γ we get from ( ˆ µ = y ˆ γt ˆ α = ( y y ˆ γ( t t o o ( t t t + t o o ˆ β = ( y y ˆ γ( t t o o Hence where ( y ˆ µ o o o o = ( y yo yo y + ( t to to + t = = ( y y y + y o o = ( y y y + y ( t t t + t o o o o = ( t t t + t o o ( y y y + y ( t t t + t Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur
Case ( : est of H0 α Mnmze ( y β γ t wth respect to, µβ and γ gves the least suares estmates (or the maxmum lkelhd estmates of respectve parameters as ˆ µ = y ˆ γt ˆ β = y y ˆ( γ t t o o ( y yo ( t to ˆ γ = (3 ( t t ˆ µ = ˆ µ + ˆ β + ˆ γt Subsutng these estmates n (3 we get o ( y y ( t t o o ( y ˆ = ( y y ( t to where = + A A = J( y y o A = J( t t o + A + A A = J( y y ( t t o o = ( y y y + y o o = ( t t t + t o o = ( y y y + y ( t t t + t o o o o hus the lkelhd rato test statstc for testng H0 α s λ ( y ˆ µ ( y ˆ µ = ( y ˆ Adustng wth degrees of freedom and usng the earler result for the ndependence of two uadratc forms and ther dstrbuton Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur
( IJ I J ( I ( y ˆ µ F ( y ˆ µ ( y ˆ µ = So the decson rule s to reect H α whenever F > F ( I, IJ I J o α ~ F( I, IJ I J under Ho α Case b: est of H0 β Mnmze ( y α γ t wth respect to, µα and γ gves the least suares estmates (or maxmum lkelhd estmates of respectve parameters as ˆ µ = y γt α = y y γ( t t γ = o o µ = µ + α + γ From (4, we get ( y y ( t t o o ( t t o o o ( y µ = ( y yo ( t to B = I( y y o where B = I( t t o = + B = ( o ( o B I y y t t + B B hus the lkelhd rato test statstc for testng H0 β s (4 ( y y ( t t ( y ( ˆ µ y µ ( IJ I J F = ~ F( J, IJ I J ( J ( y ˆ µ So the decson rule s to reect H0 β whenever F F α ( J, IJ I J under Ho β Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 3
If Ho α s reected, use multple comparson methods to determne whch of the contrasts responsble for ths reecton he same s true for Ho β α are he analyss of covarance table for two way classfcaton s as follows: Source of varaton Degrees of freedom Sum of products F Between evels of A I A A A I = 0 3 F = IJ I J I 0 Between levels of B J B B B J = 4 F = IJ I J J rror ( I ( J IJ I J = otal IJ IJ rror + levels of A IJ J ( A + 3 = ( A + A + rror + levels of B IJ I ( B + 4 = ( B + B + Analyss of Varance Chapter Analyss of Covarance Shalabh, II Kanpur 4