STS 414 ANALYSIS OF VARIANCE (ANOVA) REVIEW OF SIMPLEREGRESSION

Transcription

1 STS 44 ANALSIS OF VARIANCE (ANOVA) REVIEW OF SIMPLEREGRESSION Modelg refer to the developmet of mathematcal expreo thatdecrbe ome ee the behavor of a radom varable of teret. Thvarable ma be the prce of wheat the world maret, the umber ofdeath from lug cacer, the rate of growth of a partcular tpe of tumor,or the tele tregth of metal wre. I all cae, th varable called thedepedet varable ad deoted wth. A ubcrpt o detfe thepartcular ut from whch the obervato wa tae, the tme at whchthe prce wa recorded, the cout whch the death were recorded, theexpermetal ut o whch the tumor growth wa recorded, ad o forth.mot commol the modelg amed at decrbg how the mea of thedepedet varable E( ) chage wth chagg codto; the varaceof the depedet varable aumed to be uaffected b the chaggcodto. Other varable whch are thought to provde formato o the behavour of the depedet varable are corporated to the model a predctor orexplaator varable. Thee varable are called the depedet varablead are deoted b wth ubcrpt a eeded to detf dfferetdepedet varable. Addtoal ubcrpt deote the obervatoal utfrom whch the data were tae. The are aumed to be ow cotat. I addto to the, all model volve uow cotat, calledparameter, whch cotrol the behavor of the model. Thee parameterare deoted b Gree letter ad are to be etmated from the data. The mathematcal complext of the model ad the degree to whcht a realtc model deped o how much ow about the procebeg tuded ad o the purpoe of the modelg exerce. I prelmartude of a proce or cae where predcto the prmar obectve,the model uuall fall to the cla of model that are lear theparameter. That, the parameter eter the model a mple coeffceto the depedet varable or fucto of the depedet varable.such model are referred to looel a lear model. The more realtcmodel, o the other had, are ofte olear the parameter. Motgrowth model, for example, are olear model. Nolear model fallto two categore: trcall lear model, whch ca be learzedb a approprate traformato o the depedet varable, ad thoethat caot be o traformed.

2 The Lear Model ad Aumpto The mplet lear model volve ol oe depedet varable ad tate that the true mea of the depedet varable chage at a cotat ratea the value of the depedet varable creae or decreae. Thu, thefuctoal relatohp betwee the true mea of, deoted b E( ), ad the equato of a traght le: E( ) = β + β. β the tercept, the value of E( ) whe =, ad β the lope of the le, the rate of chage E( ) per ut chage. The obervato o the depedet varable are aumed to be radom obervato from populato of radom varable wth the mea of eachpopulato gve b E( ). The devato of a obervato from tpopulato mea E() tae to accout b addg a radom error e to gve the tattcal model = β + β + e. The ubcrpt dcate the partcular obervatoal ut, =,,...,.The are the obervato o the depedet varable ad are aumedto be meaured wthout error. That, the oberved value of are aumedto be a et of ow cotat. The ad are pared obervato; bothare meaured o ever obervatoal ut. The radom error e have zero mea ad are aumed to have commovarace σ ad to be parwe depedet. Sce the ol radom elemet the model e, thee aumpto mpl that the alo have commovarace σ ad are parwe depedet. For purpoe of mag tetof gfcace, the radom error are aumed to be ormall dtrbuted,whch mple that the are alo ormall dtrbuted. The radom erroraumpto are frequetl tated a e NID(, σ ), where NID tad for ormall ad depedetl dtrbuted. The quatte parethee deote the mea ad the varace, repectvel, of theormal dtrbuto. Leat Square Etmato The mple lear model ha two parameterβ ad β, whch are to beetmated from the data. If there were o radom error, a two datapot could be ued to olve explctl for

3 the value of the parameter.the radom varato, however, caue each par of oberved datapot to gve dfferet reult. (All etmate would be detcal ol f theoberved data fell exactl o the traght le.) A method eeded thatwll combe all the formato to gve oe oluto whch bet bome crtero. The leat quare etmato procedure ue the crtero that the oluto mut gve the mallet poble um of quared devato of the oberved from the etmate of ther true mea provded b the oluto. Let β ad β be umercal etmate of the parameter β ad β, repectvel, ad let = + be the etmated mea of for each, =,...,. Note that obtaed b ubttutg the etmate for the parameter the fuctoal form of the model relatg E() to. The leat quare prcplechooe ad that mmze the um of quare of the redual,ss(re): SS(Re ) = e ( ) wheree = ) the oberved redual for the th obervato. Theummato dcated b ( over all obervato the data et a dcated b the dex of ummato, = to. (The dex of ummato omtted whe the lmt of ummato are clear from the cotext.) The etmator for β ad β are obtaed b ug calculu to fd thevalue that mmze SS(Re). The dervatve of SS(Re) wth repect to ad tur are et equal to zero. Th gve two equato twouow called the ormal equato: ( ) ( ) ) ( ) (. 3

4 4 Solvg the ormal equato multaeoul for ad gve the etmate ofβ ad β a ) ( ) )( ( x x. Note that x = ( ) ad = ( ) deote obervato expreeda devato from ther ample mea ad, repectvel. The morecoveet form for had computato of um of quare ad um ofproduct are x ) ( x ) )( ( Thu, the computatoal formula for the lope ) ( ) )( ( Thee etmate of the parameter gve the regreo equato ANALSIS OF VARIANCE (ANOVA) Aal of Varace (ANOVA) wa troduced b Sr Roald Fher ad eetall a arthmetc proce for parttog a total um of quare to compoet aocated wth recogzed ource of varato. It ha bee ued to advatage all feld of reearch where data are meaured quattatvel. Suppoe a dutral expermet that a egeer tereted how the mea aborpto of moture cocrete vare amog 5 dfferet cocrete aggregate.

5 The ample are expoed to moture for 48 hour. It decded that 6 teted. The data are preeted Table. The model for th tuato codered a follow. There are 6 obervato tae from each of 5 populato wth mea,...,, repectvel. We ma wh to tet H H, 5 :... 5, : At leat two of the mea are ot equal. Aggregate: Total ,854 Mea I addto, we ma be tereted mag dvdual comparo amog thee 5 populato mea. Two Source of Varablt the Data I the ANOVA procedure, t aumed that whatever varato ext betwee the aggregate average attrbuted to () varato aborpto amog obervato wth aggregate tpe, ad () varato due to aggregate tpe, that, dueto dfferece the chemcal compoto of the aggregate. The wth aggregate varato, of coure, brought about b varou caue. Perhap humdt ad temperature codto were ept etrel cotat throughout the expermet. It poble that there wa a certa amout of heterogeet the batche of raw materal that were ued. At a rate, we hall coder the wth ample varato to be chace or radom varato, ad part of the goal of the ANOVA to determe f the dfferece amog the 5 ample mea are what we would expect due to radom varato aloe. Ma poted queto appear at th tage cocerg the precedg problem. For example, how ma ample mut be teted for each aggregate? Th a queto that cotuall haut 5

6 the practtoer. I addto, what f the wth ample varato o large that t dffcult for a tattcal procedure to detect the tematc dfferece? Ca we tematcall cotrol extraeou ource of varato ad thu remove them from porto we call radom varato? We hall attempt to awer thee ad other queto th coure. Completel Radomzed Deg (Oe-Wa ANOVA) Radom ample of ze are elected from each of populato. The dfferet populato are clafed o the ba of a gle crtero uch a dfferet treatmet or group. Toda the term treatmet ued geerall to refer to the varou clafcato, whether the are dfferet aggregate, dfferet aalt, dfferet fertlzer, or dfferet rego of the coutr. Aumpto ad Hpothee Oe-Wa ANOVA It aumed that the populato are depedet ad ormall dtrbuted wth mea,,..., ad commo varace. Thee aumpto are made more palatable b radomzato. We wh to derve approprate method for tetg the hpothe Let deote the H :... H : At leat two of the mea are ot equal. th obervato from the. the total of all obervato the ample from the obervato the ample from the the mea of all obervato. th treatmet ad arrage the data a Table. Here, th treatmet,. the mea of all th treatmet,.. the total of all obervato, ad.. Model for Oe-Wa ANOVA Each obervato ma be wrtte the form, where meaure the devato of the th obervato of the th ample from the correpodg treatmet mea. The -term repreet radom error ad pla the ame role a the error term the regreo model. A alteratve ad 6

7 Table. Radom Sample Treatmet Total. Mea Preferred form of th equato obtaed b ubttutg, ubect to the cotrat. Hece we ma wrte, Where ut the grad mea of all the '; that,, ad called the effect of the th treatmet. The ull hpothe that the populato mea are equal agat the alteratve that at leat two of the mea are uequal ma ow be replaced b the equvalet hpothe. H :..., H : At leat two of the are ot equal zero. Reoluto of Total Varablt to Compoet Our tet wll be baed o a comparo of two depedet etmate of the commo populato varace. Thee etmate wll be obtaed b parttog the total varablt of our data, degated b the double ummato ' 7

8 ( ).. to two compoet. Theorem 3.: Sum-of-quare Idett.. (. ) ( ) (... ) It wll be coveet what follow to detf the term of the um-of-quare detf b the followg otato: Three Importat Meaure of Varablt SST ( total um of quare,.. ) SSA SSE ( treatmet um of quare,...) ( ). error um of quare, The um-of quare dett ca the be repreeted mbolcall b the equato SST SSA SSE. F-Rato for Tetg Equalt of Mea Whe H true, the rato f a value of the radom varable F havg the F- dtrbuto wth - ad (-) degree of freedom. Sce overetmate whe H fale, we have a oe-taled tet wth the crtcal rego etrel the rght tal of the dtrbuto. The ull hpothe H reected at the level of gfcace whe 8

9 f f [, ( )]. Aother approach, the P-value approach, ugget that the evdece favour of or agat H P P[ f [, ( )] f ]. The computato for ANOVA problem are uuall ummarzed tabular form a how Table 3. ANOVA for the Oe-Wa ANOVA Source of Sum of Square Degree of Freedom Mea Square Computed f Varato Treatmet SSA - Error SSE (-) Total SST K- SSA SSE ( ) Example. Tet the hpothe... 5 at the.5 level of gfcace for the data of Table o aborpto of moture b varou tpe of cemet aggregate. Soluto: H..., : 5 H : At leat two of the mea are ot equal..5 Crtcal rego: f. 76 wth v 4 ad v 5 degree of freedom. The um of quare computato gve SST=9,377 SSA=85,356 SSE=4,. 9

10 Source of Sum of Square Degree of Freedom Mea Square Computed f Varato Treatmet Error Total Deco: Reect H ad coclude that the aggregate do ot have the ame mea aborpto. Radomzed Complete Bloc Deg A tpcal laout for the radomzed complete bloc deg (RCB) ug 3 meauremet 4 bloc a follow: Bloc Bloc Bloc 3 Bloc 4 t t t 3 t t t 3 t t t 3 t t T 3 The t deote the agmet to bloc of each of the 3 treatmet. Of coure, the true allocato of treatmet to ut wth bloc doe at radom. Oce the expermet ha bee completed, the data ca be recorded a the followg Treatmet Bloc

11 where repreet the repoe obtaed b ug treatmet bloc, repreet the repoe obtaed b ug treatmet bloc,... ad 34 repreet the repoe obtaed b ug treatmet 3 bloc4. Let u ow geeralze ad coder the cae of treatmet aged to bbloc. The data ma ummarzed a how the b rectagular arra of Table 4. It wll be aumed that the, =,,, ad =,,,b, are value of depedet radom varable havg ormal dtrbuto wth mea ad commo varace. Table 4. b Arra for the RCB Deg Bloc Treatmet B Total Mea b T b T I b T K Total. T b T... T.... b T. T. T..... Mea b.. Let. repreet the average (rather tha the total) of the b populato mea for the thtreatmet. That,

12 b.. b Smlarl, the average of the populato mea for the th bloc,., defed b., ad the average of the bpopulato mea defed b b b. To determe f part of the varato our obervato due to dfferece amog the treatmet, we coder the tet H....., H : The are ot all equal.' Model for the RCB Deg Each obervato ma be wrtte the form, where meaure the devato of the oberved value preferred form of th equato obtaed b ubttutg from the populato mea. The, where, a before, the effect of the th treatmet ad the effect of the th bloc. It aumed that the treatmet ad bloc effect are addtve. Hece we ma wrte.

13 The bac cocept much le that of the oe wa clafcato except that we mut accout the aal for the addtoal effect due to bloc, ce we are ow tematcall cotrollg varato two drecto. ANOVA for the Radomzed Complete Bloc Deg Source Varato of Sum of Square Degree of Freedom Mea Square Computed f Treatmet SSA - Bloc SSB b- Error SSE (-)(b-) Total SST b- SSA SSA b SSE ( -)(b-) f Example.Four dfferet mache, M, M, M 3 ad M 4, are beg codered for the aemblg of a partcular product. It decded that 6 dfferet operator are to be ued a radomzed bloc expermet to compare the mache. The mache are aged a radom order to each operator. The operato of the mache requre phcal dextert, ad t atcpated that there wll be a dfferece amog the operator the peed wth whch the operate the mache (Table 5). The amout of tme (ecod) were recorded for aemblg the product: Tet the hpothe H, at the.5 level gfcace, that the mache perform at the ame mea rate of peed. Table 4: Tme, Secod, to Aemble Product Operator Mache Total

14 Total Soluto: H (mache effect are zero), : 3 4 H : At leat oe of the ot equal to zero Table5. ANOVA Table for Table 4 ' Source of Sum of Degree of Mea Computed varato Square Freedom quare Mache Operator Error Total f ug 5% a at leat a approxmate ardtc, we coclude that the mache do ot perform at the ame mea rate of peed. Lat Square The radomzed bloc deg ver effectve for reducg expermetal error b removg oe ource of varato.aother degthat partcular ueful cotrollg two ource of varato, whle reducg the requred umber of treatmet combato, called the Lat quare.suppoe that we are tereted the eld of 4 varete of wheat ug 4 dfferet fertlzer over a perod of 4 ear. The total umber of treatmet combato for a completel radomzed deg would be 64. B electg the ame umber of categore for all three crtera of clafcato, we ma elect a Lat quare deg ad perform the aal of varace ug the reult of ol 6 treatmet combato. A tpcal Lat quare, elected at radom from all poble 4 4 quare, the followg: Colum 4

15 Row 3 4 A B C D D A B C 3 C D A B 4 B C D A The four letter, A, B,C, ad D, repreet the 4 varete of wheat that are referred to a the treatmet. The row ad colum, repreeted b the 4 fertlzer ad the ear, repectvel, are the two ource of varato that we wh to cotrol. We ow ee that each treatmet occur exactl oce each row ad each colum. Wth uch a balaced arragemet the aal of varace eable oe to eparate the varato due to the dfferet fertlzer ad dfferet ear from the error um of quare ad thereb obta a more accurate tet for dfferece the eldg capablte of the 4 varete of wheat. Whe there teracto preet betwee a of the ource of varato, the f-value the aal of varace are o loger vald. I that cae, the Lat quare deg would be approprate. Geeralzato to the Lat Square We ow geeralzed ad coder a r r Lat quarewhere deote a obervato the th row ad th colum correpodg to th letter. Note that oce ad are pecfed for a partcular Lat quare, we automatcall ow the letter gve b. For example, whe ad 3 the 4 4 Lat quare, we have B. Hece a fucto of ad. If ad are the effect of the th row ad th colum, the effect of the th treatmet, the the grad mea, ad the radom error, the we ca wrte =, where we mpoe the retrcto. 5

16 A before, the dtrbuto wth mea are aumed to be value of depedet radom varable havg ormal Ad commo varace. The hpothe to be teted a follow: H :... r H : At leat oe of the ' ot equal to zero. The ANOVA (Table 6) dcate the approprate F-tet for treatmet. Table 4. ANOVA for a r r Lat Square Source of Sum of Degree of Mea Computed Varato Square Freedom Square f Row SSR r- Colum SSC r- SSR r SSC r Treatmet SSTr r- 3 3 f SSTr r Error SSE (r-)(r-) Total SST r - SSE ( r )( r ) To llutrate the aal of a Lat quare deg, let u retur to the expermet where the letter A, B, C ad D repreet 4 repreet 4 varete of wheat; the row repreet 4 dfferet fertlzer; ad the colum accout for 4 dfferet ear. The data Table 5 are the eld for the 4 varete of wheat, meaured logram per plot. It aumed that the varou ource varato do ot teract. Ug a.5 level of gfcace, tet the hpothe H : There o dfferece the average eld of the 4 varete of wheat. 6

17 Table7. eld of Wheat (logram per plot) Fertlzer Treatmet t A:7 B:75 C:68 D:8 t D:66 A:59 B:55 C:63 t 3 C:59 D:66 A:39 B:4 t 4 B:4 C:57 D:39 D:55 Soluto: H :... r H : At leat oe of the ' ot equal to zero. Table 8. ANOVA for the Data of Table 7 Source of Sum of Degree of Mea Computed Varato Square Freedom Square f Fertlzer ear Treatmet Error Total 5 5 We therefore, coclude that wheat varete gfcatl affect wheat eld. 7