INSY Reference : Chapter 3 of Montgomery(8e)

Transcription

1 11 INSY Referece : Chpter 3 of Motgomery(8e) Mghsoodloo Sigle Fctor Experimets The objective is to determie if sigle fctor (or iput, or the bsciss) hs sigifict effect o respose vrible Y (the output, or the ordite). If the iput is foud to hve sigifict impct o the output Y, the the d objective is to idetify which level of the fctor (or iput) will optimize the respose Y. For exmple see Tble 3.1 p. 67 of Motgomery s 8 th editio, where the fctor is the rdio frequecy power (RFP) settig (i wtts) d the output is Y Etch rte of tool mesured i A /mi (Agstrom/miute). I this exmple (the sme s Exmple 3.1 of Motgomery p. 75), the experimeter kows tht the rge of the iput is from 160 0W d uses 4 equispced levels of this fctor, mely 160, 180, 00, d 0W. The experimetl lyout i Tble 3.1 d the rdomiztio order re give o pge 75 of Motgomery(8e), where 5 replictios per tretmet d the N 0 totl observtios re tke completely t rdom order. Such experimet is clled Completely Rdomized Desig (CRD). See lso pp of Motgomery s 8 th editio. A procedure clled Alysis of Vrice (ANOVA) is used to test the equlity of 3 or more process mes simulteously W/O ffectig the pre ssiged LOS,, of the test. To develop the ANOVA procedure, we strt with the followig idetity y + i i + y + ( i ) + (y i ) + i + (1) This lst idetity (1) is clled the lier dditive model (LAM) for the oe fctor CRD (completely rdomized desig). A fctor is fixed if its levels re purposely desiged (or set) by the experimeter d ot selected t rdom from popultio of levels. I such cse, the LAM (1) is clled the fixed effects model. If the levels of the fctor re selected t rdom from popultio of levels, the ech i i (i 1,, ) is rdom vrible d the LAM (1) is clled the rdom effects model. For the fixed effects model, i1 μ /, while for the i rdom effects model E( i ). 11

2 ANOVA ASSUMPTIONS 1 Fixed Effects: The prmeters d i re fixed whether the desig is blced (i.e., ll i ) or ot, d s result we lwys hve i 0. However, the rvs (rdom vribles) re ssumed NID(0, ), where NID stds for ormlly d idepedetly distributed. Heceforth, we will lso use i lieu of. I the cse of fixed effects, coclusios from ANOVA perti oly to the tretmets tht hve bee studied d o other levels. Rdom effects: Oly is prmeter, i s ~ NID (0, ) d idepedet of which re N(0, ). Coclusios from ANOVA perti to the etire popultio of tretmets from which oly rdom smple of size hs bee studied. THE FISHER S F TEST FOR TESTING H 0 : 1 3, ( ) versus the ltertive H 1 : At lest two i s re sigifictly differet. First, we replce ech prmeter i Eq. (1) by its poit lest squres estimtor s show below. y y.. (yi. y.. ) (y y i. ) y.. ˆ i e, () where ˆ i... i (y y ) is clled the effect of the i th tretmet d e (y y i. ) is clled the residul of the () th cell. Exercise 1. Show tht for fixed effects the e s ~ N[0, ( 1) /], d for ublced desig i ˆi 0. Further, the V(y ), while the V(e ) (1) /. We ow rewrite the idetity () by trsposig y.. to the LHS (left hd side) of the equtio: y y.. (yi. y.. ) (y y i. ) ˆ i e (3) 1

3 13 Next we squre both sides of Eq. (3), the sum both sides over i d j, d use the fct tht the resultig sum of cross product term vishes: i i i (y y.. ) (yi. y.. ) (y y i. ) j1 j1 j1 i e j1 j1 i i SS Totl ˆ SS Tretmets + SS Residuls, (4) where SS RES SS Residuls. Idetity (4) is fudmetl to ANOVA becuse it breks dow the totl i sum of squres, [SS Totl SS T (y y.. ) ], o the LHS ito dditive compoets: (1) j1 the 1 st term o the RHS of Eq. (4) gives the SS (sum of squres) mog (or due to) tretmets, d () the d term o the RHS of Eq. (4) gives the SS withi tretmets (or experimetl error SS). Further, both SS s o the RHS hve idepedet ( ) distributios uder H 0 : i i 0. I fct, SS Error / hs lso distributio uder H 1. As result the ull distributio of (SS / ) / ( 1) Tretmet Error (SS / ) / ( N ) MS Tretmets / MS Error is the Fisher s F with 1 1 df for the umertor d N df for the deomitor. Note tht Motgomery uses the ottio SS E for SS Error. Lettig N i, the the df (degrees of freedom) for the idetity (4) re s follows: df: N 1 ( 1) + Error df df of Error N ; If the desig is blced, i.e., i for ll i, the Error or Residul df (i 1) ( 1). Exercise. Show tht uder the LAM (1) d blced desig E(SS Error ) ( 1) A ubised estimtor of is give by MS Error SS Error /[( 1)]. Further, E(SS ) E(SS Tretmets ) i + ( 1) E(MS Tretmets ) + i /( 1). Thus, if H 0 : zero tretmet effects is true, i.e., ll i s re 0, the MS Tretmets MS is lso ubised estimtor of. 13

4 From Exercise bove we deduce tht the rtio MS(Tretmets)/MS(Error) hs the Fisher F distributio with 1 1 d N df. Sice E(MS Tretmets ) E(MS Error ) 14 + /( 1), this costitutes right tiled test for the sttistic F 0 MS(Tretmets)/ i i MS(Error). For exmple, H 0 : ll i s 0 must be rejected t the 5% LOS if F 0 exceeds the 5 percetge poit of F 1,N, deoted by F 0.05, 1,N. Exercise 3. Show tht the computtiol formuls for SS T SS(Totl) d Tretmet SS re give by SS Totl i y CF SS T d SS(Tretmets) SS j1 (y i i. y).. i1 i1 i. i (y / ) CF, where CF y.. /N. RESIDUALS For ll sttisticl models, residul e is the differece betwee the ctul observed vlue d the correspodig vlue estimted by the model, i.e., e y ŷ. Uder the LAM (1), the fitted vlue ŷ ˆ ˆi ˆ y.. (yi. y.. ) 0 yi., d hece e y ŷ y y i.. Residuls ply importt role i model dequcy checkig (study pp of Motgomery). However, the equtio (3.18) of Motgomery s 8 th editio i the middle of pge 8 is ot ccurte becuse the estimted V(e ) MS Error but is equl to v(e ) ( 1)MS Error / d hece (3.18) must be revised to r d e ( 1)MS / Error (Revised 3.18, p. 8) As result the lrgest Studetized residul is r 3 d 3 5.6/( ) , ot 1.40 s reported o p. 8 of Motgomery. Further, there seems to exist some discrepcies i Figure 3.6 top p. 83 of Motgomery s 8 th editio. 14

5 For testig equlity of vrices withi tretmets, Study pp of Motgomery o Brtlett s test d verify tht ideed q 0 whe ll S i re equl to the sme vlue S. If residul lysis revels tht ormlity ssumptio of e s is violted, the dt should be trsformed ccordig to pp of Motgomery. However, the coclusios from the ANOVA Tble perti oly to the trsformed dt y * y. If the fctor is foud to hve sigifict impct o the respose Y, the regressio c be used to fid the best model tht describes the reltioship betwee y d the fctor withi the rge of the fctor spce. The regressio models o pges of Motgomery(8e) re vlid oly for 160 x 0 W, where y represets the etch rte of tool. 15 POST ANOVA For The FIXED EFFECTS MODEL If the F test from the ANOVA tble rejects H 0 : i 0 for ll i t the 5% level, the the objective will be to idetify which pirs of mes re sigifictly differet. If the experimeter decides prior to dt gtherig which tretmets should be compred, the s/he should use orthogol cotrsts; otherwise, fter experimettio there re severl post ANOVA multiple compriso procedures tht c be pplied. Motgomery presets the Scheffe s Method (pp for cotrsts), Tukey s procedure (pp ), d Duett s test for compriso with cotrol. All these 3 procedures cotrol the overll type I error rte. Emphsis will be oly o Tukey s Studetized Rge Test d Duett s test becuse most sttisticl pckges (Miitb) provide these procedures. The expltio i the text (pp ) is cler, d short time will be spet i clss to review Tukey s multiple compriso s procedure for the dt of Motgomery s plsm etchig experimet of Exmple 3.1 give i my otes uder ANOVA Logic. ORTHOGONAL CONTRASTS (ORCs) The umber of orthogol cotrsts possible is lwys equl to the df of SS(Tretmets), or tretmet SS. For the Problem 3.10 o pge 13 of Motgomery s 8 th editio, we c defie mximum of 4 ORCs prior to dt gtherig. Sy, we wish to compre 1 vs, 3 vs 5, ( 1 15

6 + )/ vs 4, d ( )/3 vs ( )/. The the 4 correspodig cotrsts will be give s follows: C 1 y 1. y. 8, or C 1 y1. y. C y 3. y 5. 34, or C y3. y5. C 3 y 1. + y. y 4. 90, or C 3 y1. y. y4. C 4 y 1. +y. 3y 3. + y 4. 3y 5. 4, or C 4 y1. y. 3y3. y4. 3y5. 16 where y 1. 49, y. 77, y 3. 88, y d y Ech cotrst crries exctly 1 df d their SS s is defied s SS(C k ) k i ik, k 1,,, 1, c i ik (C ) / c 0, d to gurtee orthogolity it is ecessry tht c c i ik im 0, for ll k m, d s result we will hve SS(C k ) SS(Tretmets). Note tht Motgomery defies cotrst, C, o pge 9 i terms of y i., while my defiitio is i terms of y i. ; therefore, my bove formuls d Motgomery s top pge 89 re the sme. We ow compute the SS s of the bove 4 cotrsts. SS(C 1 ) ( 8) /[5(1+1)] 78.4, SS(C ) 34 /[5(1+1)] 115.6, SS(C 3 ) ( 90) /[5(1+1+4)] 70, d SS(C 4 ) 4 /[5( )] Note tht SS(C k ) SS(Tretmets) , which is i complete greemet with the k1 ANOVA of Tble 3.4 t the bottom of pge 71 of Motgomery s 5 th editio. It seems tht if cotrsts re defied i terms of tretmet mes, the the orthogolity coditio should be revised to cikc im /i 0. Further, the cotrst SS is give by SS(C k ) k ik i (C ) / (c / ). Further, s Motgomery poits out i his Eq. (3.6, p.9), the vrice of cotrst, such s C 3 bove, is computed s V(C 3 ) V(y 1. + y. y 4. ) V(y 1. ) + V(y. ) + 4V(y 4. ) ( c i ik ), i.e., k i ik (C ) / ( c ) Z d hece, k Error i ik (C ) /(MS c ) t F. 1, 16

7 17 THE RANDOM EFFECTS MODEL This model is vlid oly whe the levels of fctor re rdomly selected from popultio of levels (e.g., selectig 6 opertors t rdom from popultio of 100 opertors workig i plt). Opertors would the form rdom qulittive fctor d the coclusios drw from the ANOVA Tble will perti to the etire popultio of 100 opertors. Note tht there will ot be y iterest whtsoever i determiig which of the ctully selected 6 tretmets (or opertors) re sigifictly differet (i.e., Tukey s SRT d ORCs will ot be pplicble) but rther the objective is to determie if there is sigifict vritio i the etire popultio of tretmets (or opertors). Therefore, our ull hypothesis is H 0 : 0 versus H 1 : > 0, which gi is clerly right tiled test. The ANOVA Tble is obtied exctly s i the cse of fixed effects, but the post ANOVA procedure is quite differet. I the cse of rdom effects model, if the F test rejects the ull hypothesis tht 0, i.e., F 0 exceeds the threshold vlue of F, 1,, the it is essetil to estimte the compoets of vrice i the LAM : y + i +, where E(y ) for ll i 1,,...,, d j 1,,...,. Note tht we re cosiderig oly the simpler blced cse (i.e., i for ll i) for the rdom effects model. Further, i 's re ssumed NID(0, ) so tht the costrit i 0 o loger pplies to the rdom effects model, but E( i ) 0 for ll i, which implies tht E( i ). The experimetl errors, 's, re s before NID(0, ) d idepedet of i 's. This leds to V(y ) +, which shows tht there re compoets of vrice. For exmple of obtiig the ANOVA Tble for rdom effects model, see the Exmple 3.11, pp of Motgomery s 8 th editio. I order to estimte the compoets of vrice, d, we must derive E(MS E ) d E(MS Tretmets ). As i the cse of fixed 17

8 effects, it c esily be verified tht E(MS E ). Therefore, for the Exmple 3.11 of 18 Motgomery, ˆ ˆ MS(Error) MS PE MS E, where PE stds for pure error. We ow prove tht for the rdom effects model, uder the LAM (1), E(MS Tretmets ) E(MS ) +. Proof. E(MS Tretmets ) E (yi. -y.. ) /( 1) i1j1 E (y y ) i... /( 1) i 1 E ( ii. ) (.. ) 1 E ( i ) ( i... ) 1 E ( ) E ( ) i i... 1 ( 1) ( 1) 1 + E(MS ). As before ubised estimtor of is clerly MS E, d the bove proof shows tht E(MS Tretmets ). Isertig E(MS E ) for i this lst equtio results i E(MS Tretmets ) E(MS E ), or E[(MS Tretmets MS E )/]. This lst equlity clerly shows tht ubised estimtor of is ideed (MS Tretmets MS E )/. You should ow be i good positio to uderstd the poit estimtes obtied er the bottom of p. 119 of Motgomery(8e) for the dt of Exmple 3.11, where MS(Tretmets) MS ˆ Tretmets ˆ MS Error MS MS Tretmets RES ˆ + ˆ ˆ The % of vritio ttributed to ll the looms i the textile compy is estimted to be pproximtely 78.59%. ˆ y respectively. Exercise 4. Work problems 3.30 & 3.31 o pge 135, codig the dt by 3 d 480, 18

9 GENERAL REGRESSION SIGNIFICANCE TEST APPROACH TO ANOVA (GRST) If DOX (desig of experimet or experimets) ivolves or more fctors d is ublced ( i s re differet t differet fctor level combitios), the the procedures preseted so fr for computig SS s will o loger be vlid. I fct, whe there re or more fctors, some SS s will lwys become egtive, if the stdrd ANOVA procedure for SS s computtios is used! Thus, i the cse of or more fctors impctig the respose, resort must be mde to the GRST pproch which is very complicted d cumbersome t best. Fortutely, most sttisticl pckges (such s SAS, Miitb d SPSS) hve built i routie to obti the ANOVA tble for ublced fctoril experimets ivolvig more th oe fctor (such s Miitb s GLM Geerl Lier Models). Before presetig the GRST pproch, for 19 the reders uderstdig, we should show tht SS Error j1 y R(, ), where R(, ) is clled the reductio of the USS j1 y due to fittig the LAM + i + to the y s. Exercise 5. Show tht the R(, ) ˆ y.. + ˆ iyi.. Hit: SS Error j1 (y y ) i. j1 (y ˆ y ) i.., where we hve mde use of the fct tht ˆ i yi. y... Thus, SS Error [(y y.. ) ˆ i)] SS Totl ˆ iyi. + j1 y.. ˆ i, but the ˆ i 0 so tht the desired result ow follows. If the i s re ot equl, the the costrit chges to ˆ i i 0. The coclusios tht you must drw from this exercise is tht the SS(Totl) SS Error + ˆ iyi., which clerly shows tht SS(Tretmets) SS ˆ iyi.. Further, SS Error USS CF ˆ iyi. USS ˆ y.. ˆ iyi., which shows tht R(, ) 19

10 0 ˆ y.. + ˆ iyi.. THE GRST APPROACH Our first objective is to estimte the prmeters d the vector [ 1 ]' such tht the lest squres fuctio (LSF) LSF L(, i ) (y i) j1 j1 is miimized wrt the ( +1) ukow prmeters d. To ccomplish this tsk, we must require tht the prtil derivtives of the LSF wrt the (+1) prmeters is zero. (LSF) (y i )( 1) j1i1 (LSF) (y ) i k j1 k i1 (y kj k )( 1) j1 set to set to 0 0, for k 1,,,. Ech prtil derivtive bove whe set equl to 0 leds to exctly oe LS (lest squres) orml equtio so tht we hve system of (+1) equtios with (+1) ukows ˆ d ˆ i (i 1,,, ), which re listed o pges of Motgomery s 8 th editio. As Motgomery otes, the (+1) equtios re ot idepedet becuse the sum of the lst equtios give rise to the 1st. However, for blced desig we hve the costrit tht ˆ i 0, d this leds to set of uique solutios ˆ y.. d ˆ i yi. y.., i 1,,,. From the Exercise 5 o the previous pge, it ow follows tht the cotributio of the tretmet i to SS(Tretmets) is simply the product of ˆ i by the correspodig subtotl y i. o the RHS of the (i +1) th LS orml equtio. Before providig exmple to illustrte the GRST pproch, we should emphsize tht there is geerlly simpler method of obtiig the LSNEs for most ANOVA models. For the blced LAM (1), to obti the LSNE for, simply dd both sides of Eq. (1) over ll i d j 0

11 d use the ssumptios tht i 0, 0 d plce ht o the remiig prmeters. This leds to y ( + i + ) N ˆ y.. j1 1 which is the 1 st LSNE o pge 15 of Motgomery. To obti the LSNE for ˆ i, keep the idex i fixed d sum both sides of the LAM(1) from j 1 to j : y j1 ( + i+ ) ˆ + i j1 ˆ y i., which for i 1,,,, gives the lst LSNEs o pges of Motgomery. To obti the F test, 1 st fit the LAM: + i + to the y s; the the reductio of the USS j1 y resultig from fittig this model (s ws show i Exercise 5 of my otes) is give by R(, ) ˆ y.. + ˆ iyi.. Secod, hypothesize tht H 0 : i 0 for ll i, which reduces the LAM to y +. Now fit this reduced model + to the dt. Summig the reduced LAM over both i d j results i oe LSNE: y.. N ˆ. Therefore, the reductio of the USS j1 y due to fittig the reduced model y + is R() ˆ y.. y.. y.. y.. /N the CF. Cosequetly, the et reductio of USS due to ddig the i s to the reduced model y + must be R(, ) R() ˆ iyi. (yi. y.. )yi. (yi. yi. y.. y i. ) ( y i. /) y.. y.. i. CF SS. ( y /) As exmple of the GRST pproch, we work PROBLEM 3.7 o pge 135 of Motgomery, d to mke clcultios less cumbersome, we chge the vlue of y 33 from 55.4 to y ; further, i order to ese computtios we code the dt by subtrctig 50 from ll N 16 observtios. This leds to the coded y , y , y. 3.1, y , y , USS , d the CF 7 /

12 To pply the GRST procedure to this ublced oe fctor experimet ( 1 5, 4 4 d 3 3), we 1 st sum the LAM: + i + y over the 4 tretmets d ll the observtios withi tretmets, ledig to the LSNE for : 16 ˆ + 5 ˆ ˆ + 3 ˆ ˆ 4 y.. Next, i order to obti the LSNE for ˆ 1, we put i 1 i the LAM d sum over j 1 to j 5, d ssume 5 ˆ 1j 0. (Note tht j y 1j ) j1 1 : 5 ˆ + 5 ˆ 1 y 1. To obti the LSNE for, we put i i the LAM d sum from j 1 to j 4. : 4 ˆ + 4 ˆ y. I similr mer we obti the equtios for 3 d 4 : 3 : 3 ˆ + 3 ˆ 3 y 3., 4 : 4 ˆ + 4 ˆ 4 y 4. The bove system of 5 equtios with 5 ukows re summrized below: 16 ˆ + 5 ˆ ˆ + 3 ˆ ˆ ˆ + 5 ˆ ˆ + 4 ˆ ˆ + 3 ˆ ˆ + 4 ˆ But the bove heterogeeous system hs determit of 0 becuse the sum of the lst 4 equtios give rise to the 1 st d hece there is o uique solutio. I order to obti uique solutio, we c impose either the costrit 4 ˆ i 0, or the costrit 4 iˆ i 0. Both costrits led to the sme exct vlue of R(, ) but differet prmeter estimtes! We rbitrrily select the costrit 4 ˆ 0 d i the ext exercise you will use the other i costrit. Therefore, we ow hve system of 5 idepedet equtios with 5 ukows s listed below: ˆ 1 + ˆ + ˆ 3 + ˆ 4 0

13 5 ˆ + 5 ˆ ˆ + 4 ˆ ˆ + 3 ˆ ˆ + 4 ˆ I wrote simple Mtlb progrm to solve the bove system (or you my use MS Excel by ivertig the mtrix of the system d multiplyig it by the colum vector [ ]). The Mtlb codes use Crmer s Rule to obti the solutio which re ˆ 4.75, 1.65, ˆ 1.50, ˆ , d ˆ Thus, R(, ) ˆ y.. + ˆ iyi SS(Error) USS R(, ) , which is i exct greemet with the SS(Error) of the Miitb progrm output tht I used to verify the bove results. Note tht if we first squre the idetity (), y y.. ˆ i e give o pge 11, d the sum from j 1 to i d i 1 to, we obti USS i y j1 i (y.. ˆ i e ) j1 i i i (y..) ˆ i e j1 j1 j1 becuse ll 3 cross product terms vish. Therefore, the mgitude of USS is first due to the correctio fctor.. CF N(y ), ext is due to the i ˆi j1 i ˆ i (y i. y.. ) i ˆiyi. ˆ iyi. j1 j1 SS Tretmets, d the rest is due to SS RES i e j1. Filly, SS Tretmets c lso be obtied by first hypothesizig tht H 0 : i 0 for ll i d usig the model y + to obti the LS estimte of. This yields y.. N ˆ, or ˆ y , resultig i R() y.. y Thus, SS Tr R(, ) R() , s before. 4 Exercise 6. Repet the bove lysis usig the d costrit iˆ i 0. 3

14 Errt for Chpter 3 of Motgomery s 8 th editio 4 1. Pge 77, i the middle the pge chge the Eq. (3.18) to d e ( 1)MS / E d chge d 1 to d 3 d e 1 to e 3.. Pge 83, i Figure 3.6 the bsciss vlue should be , d should be Pge 86, i Tble 3.7, chge S to 1.19, S 3 to , d S 4 to Pge 94, If cotrsts re defied i terms of tretmet mes isted of subtotls (y i. s), the the orthogolity coditio er the bottom of pge 94 of Motgomery s 8 th editio must be chged to i i i ( cd / ) Pge 15, lie 4 from the bottom of the pge chge the lest squres orml equtio 0, i 1,, (y ˆ ˆ ) 0 to (y ˆ ˆ ) j1 i i j1,. 4