Chapter 2 General Linear Hypothesis and Analysis of Variance
|
|
- Madison Byrd
- 6 years ago
- Views:
Transcription
1 Chater Geeral Lear Hyothess ad Aalyss of Varace Regresso model for the geeral lear hyothess Let Y, Y,..., Y be a seuece of deedet radom varables assocated wth resoses. The we ca wrte t as EY ( ) = β x, =,,...,, j=,,..., Var Y j j j= ( ) =. Ths s the lear model the exectato form where β, β,..., β are the ukow arameters ad x s are the kow values of deedet covarates j X, X,..., X. Alteratvely, the lear model ca be exressed as Y = β x, + ε, =,,..., ; j =,,..., j j j= where ε s are detcally ad deedetly dstrbuted radom error comoet wth mea ad varace,.e., E( ε ) = Var( ε ) = ad Cov( ε, ε ) = ( j). j I matrx otatos, the lear model ca be exressed as Y = Xβ + ε where Y ( Y, Y,..., Y )' s a vector of observatos o resose varable, = the matrx X X X... X X X... X = s a matrx of observatos o deedet X X... X covarates X, X,..., X, β = ( β, β,..., β ) s a vector of ukow regresso arameters (or regresso coeffcets) β, β,..., β assocated wth X, X,..., X, resectvely ad ε = ( ε, ε,..., ε ) s a vector of radom errors or dsturbaces. We assume that E( ε ) =, covarace matrx V = E = I rak X =. ( ε ) ( εε '), ( ) Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
2 I the cotext of aalyss of varace ad desg of exermets, the matrx X s termed as desg matrx; ukow β, β,..., β are termed as effects, the covarates X, X,..., X, are couter varables or dcator varables where x couts j the umber of tmes the effect β occurs the th observato x j. xj mostly takes the values or but ot always. The value x j = dcates the resece of effect x j = dcates the absece of effect β x. j β x j ad Note that the lear regresso model, the covarates are usually cotuous varables. Whe some of the covarates are couter varables ad rest are cotuous varables, the the model s called as mxed model ad s used the aalyss of covarace. Relatosh betwee the regresso model ad aalyss of varace model The same lear model s used the lear regresso aalyss as well as the aalyss of varace. So t s mortat to uderstad the role of lear model the cotext of lear regresso aalyss ad aalyss of varace. Cosder the multle lear model Y = β + X β + X β + + X β + ε.... I the case of aalyss of varace model, the oe-way classfcato cosders oly oe covarate, two-way classfcato model cosders two covarates, three-way classfcato model cosders three covarates ad so o. If βγ, ad δ deote the effects assocated wth the covarates X, Z ad W whch are the couter varables, the Oe-way model: Y = α + Xβ + ε Two-way model: Y = α + Xβ + Zγ + ε Three-way model : Y = α + Xβ + Zγ + Wδ + ε ad so o. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
3 Cosder a examle of agrcultural yeld. The study varable Y deotes the yeld whch deeds o varous covarates X, X,..., X. I case of regresso aalyss, the covarates X, X,..., X are the dfferet varables lke temerature, uatty of fertlzer, amout of rrgato etc. Now cosder the case of oe-way model ad try to uderstad ts terretato terms of multle regresso model. The covarate X s ow measured at dfferet levels, e.g., f X s the uatty of fertlzer the suose there are ossble values, say Kg., Kg.,,..., Kg. the X, X,..., X deotes these values the followg way. The lear model ow ca be exressed as Y = β + β X + β X + + β X + ε by defg. o... X X X f effect of Kg.fertlzer s reset = f effect of Kg.fertlzer s abset f effect of Kg.fertlzer s reset = f effect of Kg.fertlzer s abset f effect of Kg.fertlzer s reset = f effect of Kg.fertlzer s abset. If effect of Kg. of fertlzer s reset, the other effects wll obvously be abset ad the lear model s exressble as Y = β + β ( X = ) + β ( X = ) β ( X = ) + ε = β + β+ ε If effect of Kg. of fertlzer s reset the Y = β + β ( X = ) + β ( X = ) β ( X = ) + ε = β + β + ε If effect of Kg. of fertlzer s reset the Y = β + β ( X = ) + β ( X = ) β ( X = ) + ε = β + β + ε ad so o. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 3
4 If the exermet wth Kg. of fertlzer s reeated umber of tmes the observato o resose varables are recorded whch ca be rereseted as Y Y = β + β. + β β. + ε = β + β. + β β. + ε Y = β + β. + β β. + ε If X = s reeated tmes, the o the same les umber of tmes the observato o resose varables are recorded whch ca be rereseted as Y Y Y = β + β. + β β. + ε = β + β. + β β. + ε = β + β. + β β. + ε The exermet s cotued ad f X = s reeated tmes, the o the same les Y Y Y = β + β. + β β.+ ε P = β + β. + β β.+ ε P = β + β. + β β.+ ε All these,,.., observatos ca be rereseted as y ε y ε y ε y ε y β ε β = + y ε β y ε y ε y ε or Y = Xβ + ε. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 4
5 I the two-way aalyss of varace model, there are two covarates ad the lear model s exressble as Y = β + β X + β X β X + γ Z + γ Z γ Z + ε where X, X,..., X deotes, e.g., the levels of uatty of fertlzer, say Kg., Kg.,..., Kg. ad Z, Z,..., Z deotes, e.g., the levels of level of rrgato, say Cms., Cms., Cms. etc. The levels X, X,..., X, Z, Z,..., Z are the couter varable dcatg the resece or absece of the effect as the earler case. If the effect of X ad Z are reset,.e., Kg of fertlzer ad Cms. of rrgato s used the the lear model s wrtte as Y = β + β. + β β. + γ. + γ γ. + ε = β + β + γ + ε. If X = ad Z = s used, the the model s Y = β + β + γ + ε. The desg matrx ca be wrtte accordgly as the oe-way aalyss of varace case. I the three-way aalyss of varace model Y = α + β X β X + γ Z γ Z + δw δ W + ε r r The regresso arameters β 's ca be fxed or radom. If all β 's are ukow costats, they are called as arameters of the model ad the model s called as a fxed effect model or model I. The objectve ths case s to make fereces about the arameters ad the error varace. If for some j, x j = for all =,,..., the β s termed as addtve costat. I ths case, j occurs wth every observato ad so t s also called as geeral mea effect. If all β s are observable radom varables excet the addtve costat, the the lear model s termed as radom effect model, model II or varace comoets model. The objectve ths case s to make fereces about the varaces of varace ad/or certa fuctos of them.. β 's,.e., β j,,..., ad error β β β If some arameters are fxed ad some are radom varables, the the model s called as mxed effect model or model III. I mxed effect model, at least oe β j s costat ad at least oe β j s radom varable. The objectve s to make ferece about the fxed effect arameters, varace of radom effects ad error varace. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 5
6 Aalyss of varace Aalyss of varace s a body of statstcal methods of aalyzg the measuremets assumed to be structured as y = βx + βx βx + ε, =,,..., where x are tegers, geerally or dcatg usually the absece or resece of effects j β j ; ad ε s are assumed to be detcally ad deedetly dstrbuted wth mea ad varace. It may be oted that the ε s ca be assumed addtoally to follow a ormal dstrbuto N(, ). It s eeded for the maxmum lkelhood estmato of arameters from the begg of aalyss but the least suares estmato, t s eeded oly whe coductg the tests of hyothess ad the cofdece terval estmato of arameters. The least suares method does ot reure ay kowledge of dstrbuto lke ormal uto the stage of estmato of arameters. We eed some basc cocets to develo the tools. Least suares estmate of β : Let y, y,..., y be a samle of observatos o Y, Y,..., Y. The least suares estmate of β s the values ˆβ of β for whch the sum of suares due to errors,.e., = S = ε = εε ' = ( yxβ)( yxβ) = yy X' y+ β XX β s mmum where y = ( y, y,..., y ). Dfferetatg S wth resect to β ad substtutg t to be zero, the ormal euatos are obtaed as ds = XX β Xy = dβ or XX β = Xy. If X has full rak, the ( XX ) has a uue verse ad the uue least suares estmate of β s ˆ β = ( XX) Xy whch s the best lear ubased estmator of β the sese of havg mmum varace the class of lear ad ubased estmator. If rak of X s ot full, the geeralzed verse s used for fdg the verse of ( XX ). Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 6
7 If L β s a lear arametrc fucto where L = (,,..., ) s a o-ull vector, the the least suares estmate of L β s L ˆ. β A uesto arses that what are the codtos uder whch a lear arametrc fucto L β admts a uue least suares estmate the geeral case. The cocet of estmable fucto s eeded to fd such codtos. Estmable fuctos: A lear fucto λβ of the arameters wth kow λ s sad to be a estmable arametrc fucto (or estmable) f there exsts a lear fucto LY of Y such that b E( LY ) = λβ for all β R. Note that ot all arametrc fuctos are estmable. Followg results wll be useful uderstadg the further tocs. Theorem : A lear arametrc fucto L β admts a uue least suares estmate f ad oly f L β s estmable. Theorem (Gauss Markoff theorem): If the lear arametrc fucto L β s estmable the the lear estmator L ˆ β where ˆβ s a soluto of XX ˆ β = XY s the best lear ubased estmator of L ' β the sese of havg mmum varace the class of all lear ad ubased estmators of L β. ' ' ' Theorem 3: If the lear arametrc fucto φ = lβ, φ = l β,..., φ = l β are estmable, the ay lear combato of φ, φ,..., φ k s also estmable. k k Theorem 4: All lear arametrc fuctos β are estmable f ad oly f X has full rak. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 7
8 If X s ot of full rak, the some lear arametrc fuctos do ot admt the ubased lear estmators ad othg ca be ferred about them. The lear arametrc fuctos whch are ot estmable are sad to be cofouded. A ossble soluto to ths roblem s to add lear restrctos o β so as to reduce the lear model to a full rak. Theorem 5: Let ' Lβ ad ther least suares estmators. The ' ˆ ' Var( Lβ) = L( X X ) L ' ˆ ' ˆ ' Cov( L β, L β) = L ( X X ) L ' Lβ be two estmable arametrc fuctos ad let ' ˆ L β ad ' ˆ L β be assumg that X s a full rak matrx. If ot, the geeralzed verse of XX ca be used lace of uue verse. Estmator of based o least suares estmato: Cosder a estmator of as, ˆ β ˆ β ˆ = ( yx )( yx ) [ ( ) ' ][ ( ) y X XX X y Y X XX Xy ] = = y'[ IX( XX ) X'][ IX( XX ) X ] y = y'[ I X( XX ) X ] y where the hat matrx s a demotet matrx wth ts trace as [ I X( XX) X ] = tr( X X ) X X (usg the result tr( AB) = tr( BA)) tr[ I X ( X X ) ] = tri trx ( X X ) X = tr I =. Note that usg E( y ' Ay) = µ ' Aµ + tr( AΣ ), we have E( ˆ ) = tr[ I X ( X X ) X ] = ad so ˆ s a ubased estmator of. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 8
9 Maxmum Lkelhood Estmato The least suares method does ot uses ay dstrbuto of the radom varables case of the estmato of arameters. We eed the dstrbutoal assumto case of least suares oly whle costructg the tests for hyothess ad the cofdece tervals. For maxmum lkelhood estmato, we eed the dstrbutoal assumto from the begg. Suose y, y,..., y are deedetly ad detcally dstrbuted followg a ormal dstrbuto wth mea E( y ) = β x ad varace Var( y ) = ( =,,, ). The the lkelhood fucto j j j= of y, y,..., y s L y y X y X ( π) ( ) ( β, ) = ex ( )( ) β β where y = ( y, y,..., y ). The L= L y = yx y X l ( β, ) log π log ( β) ( β). Dfferetatg the log lkelhood wth resect to β L = XX β = Xy β L = = yx β yx β ( )( ) ad, we have Assumg the full rak of X, the ormal euatos are solved ad the maxmum lkelhood estmators are obtaed as β = ( XX ) Xy = ( yx β )( yx β ) = y I X( XX ) X y. The secod order dfferetato codtos ca be checked ad they are satsfed for ˆβ ad to be the maxmum lkelhood estmators. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 9
10 Note that the maxmum lkelhood estmator β s same as the least suares estmator β ad β s a ubased estmator of β,.e., E( β) = β ulke the least suares estmator but s ot a ubased estmator of,.e., E( ) = estmator. lke the least suares Now we use the followg theorems for develog the test of hyothess. Theorem 6: Let Y ( Y, Y,..., Y ) = follow a multvarate ormal dstrbuto N ( µ, Σ ) wth mea vector µ ad ostve defte covarace matrx Σ. The Y AY follows a ocetral ch-suare dstrbuto wth degrees of freedom ad ocetralty arameter µ Aµ,.e., oly f Σ A s a demotet matrx of rak. χ (, µ Aµ ) f ad Theorem 7: Let Y ( Y, Y,..., Y ) = follows a multvarate ormal dstrbuto N ( µ, ) wth mea vector µ ad ostve defte covarace matrx Σ. Let YAY follows χ (, µ Aµ ) ad YAY follows χ (, µ A µ ). The YAY ad YAY are deedetly dstrbuted f AΣ A =. Theorem 8: Let Y = ( Y, Y,..., Y ) follows a multvarate ormal dstrbuto N (, I) µ, the the maxmum lkelhood (or least suares) estmator L ˆ β of estmable lear arametrc fucto s deedetly dstrbuted of ˆ ; L ˆ β follow ad ˆ N Lβ, L( XX) L follows χ ( ) where rak( X ) =. Proof: Cosder ˆ ( ) β = XX XY, the E Lˆ β = L XX XE Y ( ) ( ) ( ) = L( XX) XXβ = Lβ Var( L ˆ β) = L Var( ˆ β) L = LE ( ˆ β β)( ˆ β β) L = L ( XX ) L. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
11 Sce ˆβ s a lear fucto of y ad L ˆ β s a lear fucto of ˆβ, so L ˆ β follows a ormal dstrbuto. Let A= I X( XX ) X ad N Lβ, L( XX) L B L XX X = ( ), the ˆ L β = L ( XX ) XY = BY ad ˆ ad = ( Y Xβ)' ( ) I X XX X ( Y Xβ) = Y' AY. So, usg Theorem 6 wth rak(a) = -, = ( ) ( ) ( ) BA L XX X L XX XX XX X =. ˆ follows a χ ( ). Also So usg Theorem 7, Y ' AY ad BY are deedetly dstrbuted. Tests of Hyothess the Lear Regresso Model Frst we dscuss the develomet of the tests of hyothess cocerg the arameters of a lear regresso model. These tests of hyothess wll be used later the develomet of tests based o the aalyss of varace. Aalyss of Varace The techue the aalyss of varace volves the breakg dow of total varato to orthogoal comoets. Each orthogoal factor reresets the varato due to a artcular factor cotrbutg the total varato. Model Let Y, Y,..., Y be deedetly dstrbuted followg a ormal dstrbuto wth mea EY ( ) = β x ad varace. Deotg Y = ( Y, Y,..., Y ) a colum vector, such j j j= assumto ca be exressed the form of a lear regresso model Y = Xβ + ε where X s a matrx, β s a vector ad ε s a vector of dsturbaces wth E ( ε ) = Cov( ε) = I ad ε follows a ormal dstrbuto. Ths mles that EY ( ) = Xβ Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
12 E Y X Y X = I ( β)( β). Now we cosder four dfferet tyes of tests of hyothess. I the frst two cases, we develo the lkelhood rato test for the ull hyothess related to the aalyss of varace. Note that, later we wll derve the same test o the bass of least suares rcle also. A mortat dea behd the develomet of ths test s to demostrate that the test used the aalyss of varace ca be derved usg least suares rcle as well as lkelhood rato test. Case : Cosder the ull hyothess for testg H β = β : where β = ( β, β,..., β ), β = ( β, β,..., β )' s secfed ad s ukow. Ths ull hyothess s euvalet to H : β = β, β = β,..., β = β. Assume that all β ' s are estmable,.e., rak( X ) = (full colum rak). We ow develo the lkelhood rato test. The ( + ) dmesoal arametrc sace Ω s a collecto of ots such that { β β } Ω= (, ); < <, >, =,,.... Uder H, all β s s are kow ad eual, say β dmesoal sace gve by {(, ); } ω = β >. The lkelhood fucto of y, y,..., y s = L( y β, ) ex ( y Xβ )( y Xβ ) π all are kow ad the Ω reduces to oe The lkelhood fucto s maxmum over Ω whe β ad are substtuted wth ther maxmum lkelhood estmators,.e., ˆ β = ( XX ) Xy ˆ β ˆ β ˆ = ( yx )( yx ). Substtutg ˆβ ad ˆ Ly ( β, ) gves Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
13 ( β, ) = ex ( ˆ β)( ˆ β) Max L y y X y X Ω πˆ ˆ = ex. π( y X ˆ β)( y X ˆ β) Uder H, the maxmum lkelhood estmator of s The maxmum value of the lkelhood fucto uder H s ( β, ) = ex ( β )( β) Max L y y X y X ω πˆ ˆ The lkelhood rato test statstc s (, ) Max L y β ω λ = Max L y β Ω (, ) ( yx ˆ β)( yx ˆ β) = ( yxβ )( yxβ ) = ex π( yxβ )( yxβ ) ˆ ˆ ( yxβ)( yxβ) = ' ˆ ˆ ˆ ˆ ( y Xβ) + ( Xβ Xβ ) ( y Xβ) + ( Xβ Xβ ) ( ˆ β β )' XX ( ˆ β β ) = + ( y X ˆ β)( y X ˆ β) = + where ˆ ˆ = ( yxβ) ( yxβ) = ˆ β β XX ˆ β β ad ( ) ( ). β β ˆ = ( yx )( yx ). The exresso of ad ca be further smlfed as follows. Cosder Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 3
14 = ˆ β β XX ˆ β β ( ) ( ) = ( XX) Xy β XX ( XX ) Xy β = ( XX) X ( y Xβ ) XX ( XX ) X ( yxβ ) = ( yxβ ) X( XX ) XX ( XX ) X ( yxβ ) = β ( y X ) X( XX) X ( y X ) = ( yx ˆ β)( yx ˆ β) = y X( XX) Xy yx( XX ) Xy = y I X( XX ) X y = y Xβ + Xβ I X XX X y Xβ + Xβ [( ) ] [ ( ) '][( ) ] = yxβ I X XX X yxβ ( )[ ( ) ]( ) β Other two terms become zero usg = [ I X( XX) X ] X I order to fd out the decso rule for H based o λ, frst we eed to fd f λ s a mootoc creasg or decreasg fucto of. So we roceed as follows: Let g =, so that λ = + = ( + g) the dλ = dg ( + g ) + So as g creases, λ decreases. Thus λ s a mootoc decreasg fucto of. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur The decso rule s to reject H f λ λ where λ s a costat to be determed o the bass of sze of the test α. Let us smlfy ths our cotext. 4
15 λ λ or or or or + λo ( + g ) λ o ( + ) λ g g λ or g C where C s a costat to be determed by the sze α codto of the test. So reject H wheever Note that the statstc C. ca also be obtaed by the least suares method as follows. The least suares methodology wll also be dscussed further lectures. ˆ ˆ = ( β β ) XX ( β β ) = M( yxβ)( yxβ) M( yxβ)( yxβ) ω sum of suares sum of suares sum of suares due to due to Ho due to error devato from Ho OR OR sum of suares due Totalsum of suares to β It wll be see later that the test statstc wll be based o the rato. I order to fd a arorate dstrbuto of, we use the followg theorem: Ω Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 5
16 Theorem 9: Let Q The Z = Y Xβ Q = ZX XX X Z ( ) ' Q = Z [ I X( XX ) X ] Z.. Q ad are deedetly dstrbuted. Further, whe Q H s true, the ~ χ ( ) Q ad ~ χ ( ) where χ ( m ) deotes the Proof: Uder H, ( ) EZ = Xβ Xβ = ( ) ( ). Var Z = Var Y = I χ dstrbuto wth m degrees of freedom. Further Z s a lear fucto of Y ad Y follows a ormal dstrbuto. So Z N I ~ (, ) The matrces X( XX) X ad are demotet matrces. So [ I X( XX) X ] tr X X X X = tr X X X X = tr I = tr[ I X ( X X ) X ] = tr I tr[ X ( X X ) X ] = [ ( ) ] [( ) ] ( ) So usg theorem 6, we ca wrte that uder H Q ~ χ ( ) Q ad ~ χ ( ) where the degrees of freedom ad ( ) are obtaed by the trace of X( XX) X ad trace of I X( XX) X, resectvely. Sce I X XX X X XX X = ( ) ( ), So usg theorem 7, the uadratc forms Q ad Q are deedet uder H. Hece the theorem s roved. Sce Q ad Q are deedetly dstrbuted, so uder H Q / Q /( ) follows a cetral F dstrbuto,.e. Q F(, ). Q Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 6
17 Hece the costat C the lkelhood rato test statstc λ s gve by C= F α (, ) where F α (, ) deotes the uer α % ots of F-dstrbuto wth ad degrees of freedom. The comutatos of ths test of hyothess ca be rereseted the form of a aalyss of varace table. ANOVA for testg Source of Degrees Sum of Mea F-value varato of freedom suares suares Due to β Error ( ) Total β β ( y X )( y X ) Case : Test of a subset of arameters H β = β k = r < whe βr+, βr+,..., β ad are ukow. :,,,.., k k I case, the test of hyothess was develoed whe all β s are cosdered the sese that we test for each β = β, =,,...,. Now cosder aother stuato, whch the terest s to test oly a subset of β, β,..., β,.e., ot all but oly few arameters. Ths tye of test of hyothess ca be used, e.g., the followg stuato. Suose fve levels of voltage are aled to check the rotatos er mute (rm) of a fa at 6 volts, 8 volts, volts, volts ad 4 volts. It ca be realzed ractce that whe the voltage s low, the the rm at 6, 8 ad volts ca be observed easly. At ad 4 volts, the fa rotates at the full seed ad there s ot much dfferece the rotatos er mute at these voltages. So the terest of the exermeter les testg the hyothess related to oly frst three effects, vz., β, for 6 volts, β for 8 volts ad β 3 for volts. The ull hyothess ths case ca be wrtte as: Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 7
18 H : β = β, β = β, β = β 3 3 whe β4, β 5 ad are ukow. Note that uder case, the ull hyothess wll be H : β = β, β = β, β = β, β = β, β = β Let β, β,..., β be the arameters. We ca dvde them to two arts such that out of β, β,..., βr, βr,..., β ad we are terested + testg a hyothess of a subset of t. Suose, We wat to test the ull hyothess H β = β k = r < whe βr+, βr+,..., β ad are ukow. : k k,,,.., The alteratve hyothess uder cosderato s H : β β for at least oe k =,,.., r <. I order to develo a test for such a hyothess, the lear model Y = Xβ + ε uder the usual assumtos ca be rewrtte as follows: Partto X = ( X X ), where () β() β = β() β = ( β, β,..., β ), β() = ( β, β,..., β ) r r+ r+ wth order as X : r, X : ( r), β() : r ad β : ( r). () k k The model ca be rewrtte as Y = Xβ + ε β() = ( X X) + ε β() = X β + X β + ε () () The ull hyothess of terest s ow H : β() = β() = ( β, β,..., β r ) where β () ad are ukow. The comlete arametrc sace s {( β, ); β,,,,..., } Ω= < < > = ad samle sace uder H s { () () r r } ω = ( β, β, ); < β <, >, = +, +,...,. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 8
19 The lkelhood fucto s ( β, ) = ex ( β)( β) L y y X y X π. The maxmum value of lkelhood fucto uder Ω s obtaed by substtutg the maxmum lkelhood estmates of β ad,.e., ˆ β = ( XX) Xy = ˆ β ˆ β ˆ ( y X )( y X ) as ( β, ) = ex ( ˆ β)( ˆ β) MaxL y y X y X Ω πˆ ˆ = ex. ˆ ' π( y Xβ)( y X ˆ β) Now we fd the maxmum value of lkelhood fucto uder H. The model uder H becomes Y = X β + X β + ε. The lkelhood fucto uder H s () = () () () () L( y β, ) ex ( y X β X β )( y X β X β ) π ex ( * y X β() )( y* X β() ) = π where () y* = y X β(). Note that () β ad are the ukow arameters. Ths lkelhood fucto looks lke as f t s wrtte for y N X * ~ ( β(), ). Ths hels s wrtg the maxmum lkelhood estmators of β () ad drectly as ˆ β = ( X X ) X y* ' ' () ˆ β ˆ β ˆ = ( y* X ())( y* X ()). Note that. ' ' XX s a rcal mor of XX. Sce XX s a ostve defte matrx, so XX s ' also ostve defte. Thus ( X X ) exsts ad s uue. Thus the maxmum value of lkelhood fucto uder H s obtaed as Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 9
20 ˆ = ˆ ˆ () () MaxL( y* β, ) ex ( y* X β )( y* X β ) ω πˆ ˆ = ex ( y* X ˆ ˆ π β() )'( y* X β() ) The lkelhood rato test statstc for H : β = β s () () λ = max Ly ( β, ) ω max Ly ( β, ) Ω ( yx ˆ β)( yx ˆ β) = ( * ˆ ˆ y X β() )( y* X β() ) - ( y* X ˆ β )( y* X ˆ β ) ( yx ˆ β)( y X ˆ β) + ( yx ˆ β)( yx ˆ β) ( y X ˆ β)( y X ˆ β) () () = - ( y* X ˆ ˆ ˆ ˆ β() )( y* X β() ) ( yxβ)( yxβ) = + ( y X ˆ β)( y X ˆ β) - = + = ( y* X ˆ β )( y* X ˆ β ) ( yx ˆ β)( yx ˆ β) ad ˆ ˆ = ( yxβ)( yx β). where () () Now we smlfy ad. Cosder ( y* X ˆ β )( y* X ˆ β ) = () () ' ' ' ' ' ' *' ( ) * = ( y* X ( X X ) X y*) ( y* X ( X X ) X y*) = y I X X X X y ' ' ' = ( yx β() X β() ) + X β () I X( XX) X ( yx β() X β() ) + X β () ' ' = ( y X β() X β() ) I X ( X X ) X ( y X β() X β() ). The other terms becomes zero usg the result X ' ' ' I X ( X X ) X =. Note that uder H, X β + X β ca be exressed as () () ( X X )( β β )', () () Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
21 Cosder ( yx ˆ β)( y X ˆ β) = = ( y X( X ' X) X ' y)( y X( X ' X) X ' y) y I X XX X y = ( ) ' ˆ = ( y X β() X β() ) X β ˆ () X β() ) I X( X ' X) X ( y X β() X β() ) X β() X β() ) ( yxβ() X β() ) IX( XX ) X ( yx β() X β() ) = ( yx β() X β() )' I X( XX ) X ( yx β() X β() ) ad other term becomes zero usg the result X I X XX X = ' ( ). Note that uder H, the term X β + X β ca be exressed as () () ( X X )( β β )'. () () Thus = ( y* X ˆ β )( y* X ˆ β ) ( yx ˆ β)( yx ˆ β) () () = y*' I X( X X) X y* y' I X( XX ) X y ' ' = ( yx β() X β() ) I X( X X) X ( yx β() X β() ) ( yx β() X β() )' IX( XX ) X ( yx β X β () () ' ' = ( yx β() X β() ) X( XX ) X X( X X) X ( yx β() X β() ) ) = ( yx ˆ β)( yx ˆ β) [ ( ) ] = y I X XX X y ' [ ] = ( yx β() X β() ) + ( X β() + X β() ) IX( XX) X ( yx β() X β() ) + ( X β() + X β() ) = ( yx β() X β() )' I X( XX ) X ( yx β() X β() ). Other terms become zero. Note that smlfyg the terms ad, we tred to wrte them the uadratc form wth same varable ( yx β X β ). () () Usg the same argumet as the case, we ca say that sce λ s a mootoc decreasg fucto of, so the lkelhood rato test rejects H wheever Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
22 > C where C s a costat to be determed by the sze α of the test. The lkelhood rato test statstc ca also be obtaed through least suares method as follows: ( + ) : Mmum value of ( yxβ)( yx β) whe : Sum of suares due to H H : β = β holds true. () () : Sum of suares due to error. : Sum of suares due to the devato from H or sum of suares due to β adjusted () for β. () If β () = the = ( yx ˆ β() ) '( yx ˆ ˆ ˆ β() ) ( yxβ) '( yxβ) ˆ' ' = ( yy β ˆ () Xy) ( yy β Xy ) ˆ ˆ ' ' = β Xy β X y. Reducto sum of suares or sum of suares due to β () sum of suares due to β() gorg β () Now we have the followg theorem based o the Theorems 6 ad 7. Theorem : Let Z = Y X β X β Q Q = Z AZ = Z BZ () () where A= X( XX ) X' X ( X X ) X B= I X XX X ( ) '. ' ' Q The ad Q Q are deedetly dstrbuted. Further ~ () Q χ r ad ~ χ ( ). Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur
23 Thus uder H, Q / r Q = Q /( ) r Q follow a F-dstrbuto Fr (, ). Hece the costat C λ s C= F α ( r, ), where F (, r ) deotes the uer α % ots o F-dstrbuto wth r ad ( ) degrees of freedom. α The aalyss of varace table for ths ull hyothess s as follows: ANOVA for testg H : β = β () () Source of Degrees Sum of Mea F-value Varato of suares suares Freedom Due to β r () Error Total ( r) + r ( ) r Case 3: Test of H : Lβ = δ Let us cosder the test of hyothess related to a lear arametrc fucto. Assumg that the lear arameter fucto L β s estmable where L = (,,..., ) s a vector of kow costats ad β = ( β, β,..., β ). The ull hyothess of terest s H : Lβ = δ. where δ s some secfed costat. Cosder the set u of lear model Y = Xβ + ε where Y = ( Y, Y,..., Y ) follows N( Xβ, I). The maxmum lkelhood estmators of β ad are ˆ β = ( XX ) Xy ad ˆ = ( yx ˆ β)( y X ˆ β) resectvely. The maxmum lkelhood estmate of estmable L β s L ˆ β, wth Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 3
24 EL ( ' ˆ β) = L β ( ˆ β) = ( ) ˆ β ~ β, ( ) Cov L L X X L L N L L XX L ad ˆ χ ~ ( ) assumg X to be the full colum rak matrx. Further, dstrbuted. Uder H : Lβ δ, = the statstc L ˆ β ad ˆ are also deedetly t = ( )( L ˆ β δ) ˆ ( ) L XX L follows a t-dstrbuto wth ( ) degrees of freedom. So the test for H : Lβ = δ agast H : Lβ δ rejects H wheever t t α ( ) where t α ( ) deotes the uer α % ots o t-dstrbuto wth degrees of freedom. Case 4: Test of H : φ = δ, φ = δ,..., φk = δk Now we develo the test of hyothess related to more tha oe lear arametrc fuctos. Let the th estmable lear arametrc fucto s φ = L β ad there are k such fuctos wth L ad β both beg vectors as the Case 3. ' Our terest s to test the hyothess H : φ = δ, φ = δ,..., φk = δk where δ, δ,..., δ k are the kow costats. Let φ = ( φ, φ,..., φ ) k ad δ = ( δ, δ,.., δ ) k. The H s exressble as H : φ = L β = δ where L s a k matrx of costats assocated wth L, L,..., L k. The maxmum lkelhood estmator of φ s : The ˆ φ = ( ˆ φ ˆ ˆ ˆ, φ,..., φk ) = Lβ. Also E( ˆ φ) = φ ˆ φ L ˆ = β where ˆ ( ) β = XX Xy. ' Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 4
25 ˆ Cov( φ) = V where V L XX L where ' = (( ( ) j)) L XX L s the (, j ) th elemet of V. Thus ' ( ( ) j) ˆ φ φ ˆ φ φ ( ) V ( ) follows a χ dstrbuto wth k degrees of freedom ad ˆ follows a χ dstrbuto wth ( ) degrees of freedom where ˆ ( ˆ)( ˆ = yxβ y Xβ) s the maxmum lkelhood estmator of. Further ˆ φ φ ˆ φ φ ( ) V ( ) Thus uder H :φ = δ or ˆ V k ˆ ( φ δ) ( φ δ) ˆ ( ) ad ˆ φδ V ˆ φδ k ˆ ( ) ( ) ˆ are also deedetly dstrbuted. follows F- dstrbuto wth k ad ( ) degrees of freedom. So the hyothess H : φ = δ s rejected agast H : At least oe φ δ for = ì,,..., k wheever F F α ( k, ) where F α ( k, ) deotes the α% ots o F-dstrbuto wth k ad ( ) degrees of freedom. Oe-way classfcato wth fxed effect lear models of full rak: The objectve the oe-way classfcato s to test the hyothess about the eualty of meas o the bass of several samles whch have bee draw from uvarate ormal oulatos wth dfferet meas but the same varaces. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 5
26 Let there be uvarate ormal oulatos ad samles of dfferet szes are draw from each of the oulato. Let y ( j =,,..., ) be a radom samle from the th ormal oulato wth mea β ad varace j, =,,...,,.e. j ~ ( β, ), =,,..., ; =,,...,. Y N j The radom samles from dfferet oulato are assumed to be deedet of each other. These observatos follow the set u of lear model Y = Xβ + ε where Y = ( Y, Y,..., Y, Y,..., Y,..., Y, Y,..., Y )' y = ( y, y,..., y, y,..., y,..., y, y,..., y )' β = ( β, β,..., β ) ε = ( ε, ε,..., ε, ε,..., ε,..., ε, ε,..., ε )' X... values... values = values... x th f β occurs the j observato = or f effect β s reset x f effect β s abset x j j j =. = x j So X s a matrx of order, β s fxed ad - frst rows of ε are ε ' = (,,,...,), ' - ext rows of ε are ε = (,,,...,) Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 6
27 - ad smlarly the last rows of ε are ε = (,,...,,). ' Obvously, ( ), ( ) rak X = E Y = X β ad Cov Y ( ) = I. Ths comletes the reresetato of a fxed effect lear model of full rak. The ull hyothess of terest s H : β = β =... = β = β (say) ad H : At least oe β β ( j) j where β ad are ukow. We would develo here the lkelhood rato test. It may be oted that the same test ca also be derved through the least suares method. Ths wll be demostrated the ext module. Ths way the readers wll uderstad both the methods. We already have develoed the lkelhood rato for the hyothess H : β = β =... = β the case. The whole arametrc sace Ω s a ( + ) dmesoal sace {( β, ) : β,,,,..., } Ω = < < > =. Note that there are ( + ) arameters are β, β,..., β ad. Uder H, Ω reduces to two dmesoal sace ω as { } ω= ( β, ); < β<, >.. The lkelhood fucto uder Ω s = y j β π = j= Ly ( β, ) ex ( ) L= l Ly ( β, ) = l ( π ) ( y β ) L ˆ = β = y = y β L j = j= j o j= = = ˆ ( yj yo ). = j= Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 7
28 The dot sg ( o ) y o dcates that the average has bee take over the secod subscrt j. The Hessa matrx of secod order artal dervato of l L wth resect to β ad s egatve defte at β = values. y ad = ˆ whch esures that the lkelhood fucto s maxmzed at these o Thus the maxmum value of Lyβ (, ) over Ω s ˆ = y j β πˆ = j= Max L( y β, ) ex ( ) Ω The lkelhood fucto uder ω s / = ex. π ( yj yo ) = j= = y j β π = j= Ly ( β, ) ex ( ) l Ly ( β, ) l( π ) ( β ) = yj = j= The ormal euatos ad the least suares are obtaed as follows: l Ly ( β, ) β = ˆ β = yj = y = j= l Ly ( β, ) = ˆ = yj yoo = j= ( ). oo The maxmum value of the lkelhood fucto over ω uder H s ˆ = j ˆ y β π ˆ = j= / Max L( y β, ) ex ( ) ω = ex. π ( yj yoo) = j= The lkelhood rato test statstc s Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 8
29 Max L y ω λ = Max L y Ω ( β, ) ( β, ) / ( yj yo ) = j= = ( yj yoo) = j= We have that ( y ) j yoo = ( yj yo ) + ( yo yoo) = j= = j= ( yj yo ) ( yo yoo) = j= = = + Thus ( yj y ) + ( yo yoo) = = = λ = j I ( yj yo ) = j= = + where = ( yo y oo), ad = = ( y j y o). = j= Note that f the least suares rcal s used, the : sum of suares due to devatos from H or the betwee oulato sum of suares, : sum of suares due to error or the wth oulato sum of suares, + : sum of suares due to H or the total sum of suares. Usg the theorems 6 ad 7, let = ( o oo), = = = Q Y Y Q S where Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 9
30 = ( j o) = S Y Y Y Y oo o = = = j= j= Y j Y the uder H j Q Q ~ χ ( ) ~ χ ( ) Q Q ad ad are deedetly dstrbuted. Thus uder H Q ~ F(, ). Q The lkelhood rato test reject H wheever > C where the costat C = F (, ). α The aalyss of varace table for the oe-way classfcato fxed effect model s Source of Degrees of Sum of Mea sum F Varato freedom suares of suares Betwee Poulato Wth Poulato Total +. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 3
31 Note that Q = E ( β β) Q = E = + ; = β β = Case of rejecto of H If F > F α (, ), the H : β = β =... = β s rejected. Ths meas that at least oe β s dfferet from other whch s resosble for the rejecto. So objectve s to vestgate ad fd out such β ad dvde the oulato to grou such that the meas of oulatos wth the grous are same. Ths ca be doe by arwse testg of β '. s Test H : ( ) β = β k agast H : β β. k Ths ca be tested usg followg t-statstc t = Yo Yko s + k whch follows the t dstrbuto wth ( ) degrees of freedom uder H ad k s = Thus. the decso rule s to reject H at level α f the observed dfferece ( y y ) t s k o ko > α +, The uatty t α, Thus followg stes are followed : s + s called the crtcal dfferece. k. Comute all ossble crtcal dffereces arsg out of all ossble ar ( β, βk), k =,,...,.. Comare them wth ther observed dffereces 3. Dvde the oulatos to dfferet grous such that the oulatos the same grou have same meas. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 3
32 The comutato are smlfed f (CCD) s = for all. I such a case, the commo crtcal dfferece CCD = t α, s ad the observed dfferece ( y y ), k are comared wth CCD. If y y > CCD o ko o ko the the corresodg effects/meas y o ad y ko are comg from oulatos wth the dfferet meas. Note: I geeral we say that f there are three effects β, β, β 3 the H : β = β ( deote as evet A) s acceted f ad f H : β = β3( deote as evet B) s acceted the H3 : β = β( deote as evet C) wll be acceted. The uesto arses here that what sese do we coclude such statemet about the accetace of H 3. The reaso s as follows: Sce evet A B C, so PA ( B) PC ( ) I ths sese f the robablty of a evet s hgher tha the tersecto of the evets,.e., the robablty that H 3 s acceted s hgher tha the robablty of accetace of H ad H both, so we coclude, geeral, that the accetace of H ad H mly the accetace of H 3. Multle comarso test: Oe terest the aalyss of varace s to decde whether oulato meas are eual or ot. If the hyothess of eual meas s rejected the oe would lke to dvde the oulatos to subgrous such that all oulatos wth same meas come to the same subgrou. Ths ca be acheved by the multle comarso tests. A multle comarso test rocedure coducts the test of hyothess for all the ars of effects ad comare them at a sgfcace level α.e., t works o er comarso bass. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 3
33 Ths s based maly o the t-statstc. If we wat to esure that the sgfcace level α smultaeously for all grou comarso of terest, the aroxmate multle test rocedure s oe that cotrols the error rate er exermet bass. There are varous avalable multle comarso tests. We wll dscuss some of them the cotext of oe-way classfcato. I two-way or hgher classfcato, they ca be used o smlar les.. Studetzed rage test: It s assumed the Studetzed rage test that the samles, each of sze, have bee draw from ormal oulatos. Let ther samle meas be y,..., o, y o y These meas are raked ad arraged o a ascedg order as y, y,..., y where * * * y M y ad * = o y = Max y, =,,...,. * o Fd the rage as R= y y. * * The Studetzed rage s defed as, = R s where α,, γ avalable. s the uer α % ot of Studetzed rage whe γ =. The tables for α,, γ are The testg rocedure volves the comarso of, γ wth the usual way as-,, β = β =... = β f, < the coclude that. α,, α γ f, > the all α,, β s the grou are ot the same.. Studet-Newma-Keuls test: The Studet-Newma-Keuls test s smlar to Studetzed rage test the sese that the rage s comared wth α % ots o crtcal Studetzed rage W P gve by W s = α,, γ. The observed rage R= y y s ow comared wth W. * * If R< W the sto the rocess of comarso ad coclude that β = β =... = β. If R> W the Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 33
34 () dvde the raked meas * * * y, y,..., y to two subgrous cotag - ( y, y,..., y ) * * * ad ( y, y,..., y ) * * * () Comute the rages R = y y ad * * R = y y. The comare the rages R ad R * * wth W. If ether rage ( R or R ) s smaller tha W, the the meas (or β s) each of the grous are eual. If R ad/or R are greater the W, the the ( ) meas (or β s) the grou cocered are dvded to two grous of ( ) meas (or β s) each ad comare the rage of the two grous wth W. Cotue wth ths rocedure utl a grou of meas (or β s) s foud whose rage does ot exceed W. By ths method, the dfferece betwee ay two meas uder test s sgfcat whe the rage of the observed meas of each ad every subgrou cotag the two meas uder test s sgfcat accordg to the studetzed crtcal rage. Ths rocedure ca be easly uderstood by the followg flow chart. Arrage y o ' s creasg order y y... y * * * Comute Comare wth W R= y y * * = α,, γ s If R< W Sto ad coclude β = β =... = β If R> W cotue Dvde raked mea grous ( y,..., y ) ad ( y,..., y ) * * * *,. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 34
35 Comute R = y y * * R = y y * * Comare R adr wth W 4 ossbltes of R ad R wth W R R < W < W β = β =... = β ad 3 β = β =... = β β = β =... = β R R β = β =... = β 3 3 β β, j =,,..., < W > W ad at least oe whch s β j oe subgrou s ( β, β,..., β ) ad aother grou has oly β R R β = β =... = β β β, j =,3,..., > W < W ad at least oe whch s β j ( β, β,..., β ) Oe subgrou has ad aother has oly β R R. y,..., y > W > W Dvde raked meas four grous * * 3. y,..., y * * 3. y,..., y (same as ) * * 4. y,..., y * * Comute R = y y * * 3 3 R = y y * * 4 R = y y * * 5 ad comare wth W Cotue tll we get subgrous wth same β 's Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 35
36 3. Duca s multle comarso test: The test rocedure Duca s multle comarso test s the same as the Studet-Newma- Keuls test excet the observed rages are comared wth Duca s α % crtcal rage D = where * α,, γ s α = ( α), α γ deotes the uer α % ots of the Studetzed rage based o Duca s rage. *,, Tables for Duca s rage are avalable. Duca feels that ths test s better tha the Studet-Newma-Keuls test for comarg the dffereces betwee ay two raked meas. Duca regarded that the Studet-Newma-Keuls method s too strget the sese that the true dffereces betwee the meas wll ted to be mssed too ofte. Duca otes that testg the eualty of a subset k,( k ) meas through ull hyothess, we are fact testg whether ( ) orthogoal cotrasts betwee the β 's dffer from zero or ot. If these cotrasts were tested searate deedet exermets, each at level α, the robablty of correctly rejectg the ull hyothess would be ( α). So Duca roosed to use ( α) lace of α the Studet-Newma-Keuls test. [Referece: Cotrbutos to order statstcs, Wley 96, Chater 9 (Multle decso ad multle comarsos, H.A. Davd, ages 47-48)]. Case of ueual samle szes: Whe samle meas are ot based o the same umber of observatos, the rocedures based o Studetzed rage, Studet-Newma-Keuls test ad Duca s test are ot alcable. Kramer roosed that Duca s method, f a set of meas s to be tested for eualty, the relace * s * * by * s α,, γ α,, γ + U L where U ad L are the umber of observatos corresodg to the largest ad smallest meas the data. Ths rocedure s oly a aroxmate rocedure but wll ted to be coservatve, sce meas based o a small umber of observatos wll ted to be overrereseted the extreme grous of meas. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 36
37 Aother oto s to relace by the harmoc mea of,,...,,.e., 4. The Least Sgfcat Dfferece (LSD): =. I the usual testg of H : β = β agast H :, β β the t-statstc k k t = y o y ko Var ( y y ) o ko s used whch follows a t-dstrbuto, say wth degrees of freedom ' df '. Thus H s rejected wheever t > t α df, ad t s cocluded that β ad β are sgfcatly dfferet. The eualty t > t ca be α df, euvaletly wrtte as y o yko > t α Var( yo yko)., df If every ar of samle for whch y o yko exceeds t α Var( yo yko), df the ths wll dcate that the dfferece betwee β ad β k s sgfcatly dfferet. So accordg to ths, the uatty t α Var( yo yko) would be the least dfferece of y o ad, df y ko for whch t wll be declared that the dfferece betwee β ad ooled varace of the two samle ( ) defed as o ko βk s sgfcat. Based o ths dea, we use the Var y y as s ad the Least Sgfcat Dfferece (LSD) s LSD = t s + α df, k. If = =, the s LSD = t. Now all α df, ( ) ars of y o ad y, ( k =,,..., ) are comared wth LSD. Use of LSD ko crtero may ot lead to good results f t s used for comarsos suggested by the data Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 37
38 (largest/smallest samle mea) or f all arwse comarsos are doe wthout correcto of the test level. If LSD s used for all the arwse comarsos the these tests are ot deedet. Such correcto for test levels was cororated Duca s test. 5. Tukey s Hoestly sgfcat Dfferece (HSD) I ths rocedure, the Studetzed rak values α,, γ are used lace of t-uatles ad the stadard error of the dfferece of ooled mea s used lace of stadard error of mea commo crtcal dfferece for testg H : β = βk agast H : β βk ad Tukey s Hoestly Sgfcat Dfferece s comuted as HSD MS error = α.,, γ assumg all samles are of the same sze. All If yo yko > HSD the β ad ( ) β k are sgfcatly dfferet. ars yo yko are comared wth HSD. We otce that all the multle comarso test rocedure dscussed u to ow are based o the testg of hyothess. There s oe-to-oe relatosh betwee the testg of hyothess ad the cofdece terval estmato. So the cofdece terval ca also be used for such comarsos. Sce H : β = βk s same as H : β β = k so frst we establsh the relatosh ad the descrbe the Tukey s ad Scheffe s rocedures for multle comarso test whch are based o the cofdece terval. We eed the followg cocets. Cotrast: A lear arametrc fucto L= l 'β = β where β = ( β, β,..., β P ) ad = (,,..., ) are = the vectors of arameters ad costats resectvely s sad to be a cotrast whe = =. For examle β β =, β+ β β3 β =, β+ β 3β3 = etc. are cotrast whereas β + β =, β + β + β + β =, β β 3β = etc. are ot cotrasts Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 38
39 Orthogoal cotrast: If L = 'β = β ad = 'β = L m m β are cotrasts such that m = or = = = m = the L ad L are called orthogoal cotrasts. For examle, L = β+ β β3 β4 ad L = β β + β3 β4 are cotrasts. They are also the orthogoal cotrasts. The codto m = esures that L ad L are deedet the sese that = (, ) = m =. = Cov L L Mutually orthogoal cotrasts: If there are more tha two cotrasts the they are sad to be mutually orthogoal, f they are arwse orthogoal. It may be oted that the umber of mutually orthogoal cotrasts s the umber of degrees of freedom. Comg back to the multle comarso test, f the ull hyothess of eualty of all effects s rejected the t s reasoable to look for the cotrasts whch are resosble for the rejecto. I terms of cotrasts, t s desrable to have a rocedure () that ermts the selecto of the cotrasts after the data s avalable. () wth whch a kow level of sgfcace s assocated. Such rocedures are Tukey s ad Scheffe s rocedures. Before dscussg these rocedure, let us cosder the followg examle whch llustrates the relatosh betwee the testg of hyothess ad cofdece tervals. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 39
40 Examle: Cosder the test of hyothess for H : β = β j ( j =,,..., ) or H : β β j = or H : cotrast = or H : L=. The test statstc for H : β = β s j ( ˆ β ˆ ˆ β j) ( β β j) L L t = = Var ( y y ) Var ( Lˆ ) o ko where ˆβ deotes the maxmum lkelhood (or least suares) estmator of β ad t follows a t- dstrbuto wth df degrees of freedom. Ths statstc, fact, ca be exteded to ay lear cotrast, say e.g., L= β+ β β ˆ ˆ ˆ ˆ ˆ 3 β4, L = β+ β β3 β4. The decso rule s reject H : L= agast H : L. If Lˆ > t Var ( Lˆ). df The ( α) % cofdece terval of L s obtaed as Lˆ L P tdf t df = α Var ( Lˆ ) or P Lˆt ( ˆ) ˆ ( ˆ df Var L L L + tdf Var L) = α so that the ( α) % cofdece terval of L s Lˆ t ( ˆ), ˆ ( ˆ df Var L L + tdf Var L) ad Lˆt Var ( Lˆ) L Lˆ+ t Var ( Lˆ) df df If ths terval cludes L = betwee lower ad uer cofdece lmts, the H : L= s acceted. Our objectve s to kow f the cofdece terval cotas zero or ot. Suose for some gve data the cofdece tervals for β β ad β β3 are obtaed as 3 ββ ad β β3 4. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 4
41 Thus we fd that the terval of β β cludes zero whch mles that H : β β= s acceted. Thus β = β. O the other had terval of β β 3 does ot clude zero ad so H : β β = s ot acceted. Thus β β. 3 3 If the terval of β β 3 s ββ3 the H : β = β3 s acceted. If both H : β = β ad H : β = β3 are acceted the we ca coclude that β = β = β3. Tukey s rocedure for multle comarso (T-method) The T-method uses the dstrbuto of studetzed rage statstc. (The S-method (dscussed ext) utlzes the F-dstrbuto). The T-method ca be used to make the smultaeous cofdece statemets about cotrasts ( β β ) error varace f certa restrctos are satsfed. amog a set of arameters j {,,..., } β β β ad a estmate s of These restrctos have to be vewed accordg to the gve codtos. For examle, oe of the restrctos s that all ˆ β ' s have eual varaces. I the setu of oe-way classfcato, ˆβ has ts mea Y ad ts varace s. Ths reduces to a smle codto that all ' s are same,.e., = for all. so that all the varaces are same. Aother assumto s to assume that ˆ β ˆ ˆ, β,..., β are statstcally deedet ad the oly cotrasts cosdered are the ( ) We make the followg assumtos: dffereces { β β j, j,,..., } =. () The ˆ β ˆ ˆ, β,..., β are statstcally deedet () ˆ ~ (, N a ),,,...,, a β β = > s a kow costat. () s s a deedet estmate of wth γ degrees of freedom (here γ = ),.e., γ s ~ χ ( γ ) ad (v) s s statstcally deedet of ˆ β ˆ ˆ, β,..., β. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 4
42 The statemet of T-method s as follows: Uder the assumtos ()-(v), the robablty s ( α) that the values of cotrasts L= Cβ ( C = ) smultaeously satsfy = = ˆ ˆ L Ts C L L + Ts C = = where Lˆ = Cβˆ, ˆ β s the maxmum lkelhood (or least suares) estmate of β, T a,,, = = α γ wth α,, γ beg the uer α ot of the dstrbuto of studetzed rage. Note that f L s a cotrast lke β β j( j) the a = ad the terval smlfes to ( ˆ β ˆ β ) Ts β β ( ˆ β ˆ β ) + Ts j j j C = ad the varace s so that = where T = α,, γ. Thus the maxmum lkelhood (or least suares) estmate Lˆ = ˆ β ˆ β of L = β β s sad to be sgfcatly dfferet from zero accordg to T-crtero f the terval j ( ˆ β ˆ β Ts, ˆ β ˆ β + Ts) does ot cover β β =,.e., j j f ˆ β ˆ β > j Ts or more geeral f ˆ L > Ts C. = The stes volved the testg ow volve the followg stes: - Comute L ( ˆ β ˆ β j) ˆ or. - Comute all ossble arwse dffereces. - Comare all the dffereces wth j j α,, γ. s = C - If ˆ or ( ˆ ˆ L β β j) > Ts C = the ˆβ ad ˆβ j are sgfcatly dfferet where Tables for T are avalable. T = α,, γ. Whe samle szes are ot eual, the Tukey-Kramer Procedure suggests to comare ˆL wth Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 4
43 α,, γ s + C j = or T + C. j = The Scheffe s method (S-method) of multle comarso S-method geerally gves shorter cofdece tervals the T-method. It ca be used a umber of stuatos where T-method s ot alcable, e.g., whe the samle szes are ot eual. A set L of estmable fuctos { ψ } s called a -dmesoal sace of estmable fuctos f there exsts learly deedet estmable fuctos ( ψ, ψ,..., ψ ) such that every ψ L s of the form ψ = Cy where C, C,..., C are kow costats. I other word, L s the set of all = lear combatos of ψ, ψ,..., ψ. Uder the assumto that the arametrc sace Ω s Y ~ N( Xβ, I) wth rak( X ) =, β = ( β,..., β ), X s matrx, cosder a -dmesoal sace L of estmable fuctos geerated by a set of learly deedet estmable fuctos { ψ, ψ,..., ψ }. For ay ψ L, Let ψˆ = Cy be ts least suares (or maxmum lkelhood) estmator, Var( ψˆ ) = = C = ψ ( say) = ad ˆ ψ = s C = where s s the mea suare due to error wth ( ) degrees of freedom. The statemet of S-method s as follows: Uder the arametrc sace Ω, the robablty s ( α) that smultaeously for all ψ L ψˆ S ˆ ψ ψˆ + S ˆ where the costat S = F (, )., ψˆ ψˆ Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur α 43
44 Method: For a gve sace L of estmable fuctos ad cofdece coeffcet ( α), the least suare (or maxmum lkelhood) estmate ˆ ψ of ψ L wll be sad to be sgfcatly dfferet from zero accordg to S-crtero f the cofdece terval ( ψˆ S ˆ ψ ψˆ + S ˆ ) ψˆ does ot cover ψ =,.e., f ψˆ > S ˆ. ψˆ ˆψ The S-method s less sestve to the volato of assumtos of ormalty ad homogeety of varaces. Comarso of Tukey s ad Scheffe s methods:. Tukey s method ca be used oly wth eual samle sze for all factor level but S-method s alcable whether the samle szes are eual or ot.. Although, Tukey s method s alcable for ay geeral cotrast, the rocedure s more owerful whe comarg smle arwse dffereces ad ot makg more comlex comarsos. 3. It oly arwse comarsos are of terest, ad all factor levels have eual samle szes, Tukey s method gves shorter cofdece terval ad thus s more owerful. 4. I the case of comarsos volvg geeral cotrasts, Scheffe s method teds to gve marrower cofdece terval ad rovdes a more owerful test. 5. Scheffe s method s less sestve to the volatos of assumtos of ormal dstrbuto ad homogeety of varaces. Aalyss of Varace Chater Geeral Lear Hyothess ad Aova Shalabh, IIT Kaur 44
Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure
More informationChapter 3 Experimental Design Models
Chater 3 Exermetal Desg Models We cosder the models whch are used desgg a exermet. The exermetal codtos, exermetal setu ad the obectve of the study essetally determe that what tye of desg s to be used
More informationTraining Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ
Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 3.
More informationMultiple Linear Regression Analysis
LINEA EGESSION ANALYSIS MODULE III Lecture - 4 Multple Lear egresso Aalyss Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Cofdece terval estmato The cofdece tervals multple
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More information2SLS Estimates ECON In this case, begin with the assumption that E[ i
SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll
More informationChapter 8. Inferences about More Than Two Population Central Values
Chapter 8. Ifereces about More Tha Two Populato Cetral Values Case tudy: Effect of Tmg of the Treatmet of Port-We tas wth Lasers ) To vestgate whether treatmet at a youg age would yeld better results tha
More informationBASIC PRINCIPLES OF STATISTICS
BASIC PRINCIPLES OF STATISTICS PROBABILITY DENSITY DISTRIBUTIONS DISCRETE VARIABLES BINOMIAL DISTRIBUTION ~ B 0 0 umber of successes trals Pr E [ ] Var[ ] ; BINOMIAL DISTRIBUTION B7 0. B30 0.3 B50 0.5
More information2. Independence and Bernoulli Trials
. Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationCHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and
CHAPTR 6 Secto 6.. a. We use the samle mea, to estmate the oulato mea µ. Σ 9.80 µ 8.407 7 ~ 7. b. We use the samle meda, 7 (the mddle observato whe arraged ascedg order. c. We use the samle stadard devato,
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationChapter 3 Sampling For Proportions and Percentages
Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationLINEAR REGRESSION ANALYSIS
LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationAnalysis of Variance with Weibull Data
Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad
More informationLecture 9. Some Useful Discrete Distributions. Some Useful Discrete Distributions. The observations generated by different experiments have
NM 7 Lecture 9 Some Useful Dscrete Dstrbutos Some Useful Dscrete Dstrbutos The observatos geerated by dfferet eermets have the same geeral tye of behavor. Cosequetly, radom varables assocated wth these
More informationStatistics MINITAB - Lab 5
Statstcs 10010 MINITAB - Lab 5 PART I: The Correlato Coeffcet Qute ofte statstcs we are preseted wth data that suggests that a lear relatoshp exsts betwee two varables. For example the plot below s of
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationChapter 5 Properties of a Random Sample
Lecture 3 o BST 63: Statstcal Theory I Ku Zhag, /6/006 Revew for the revous lecture Cocets: radom samle, samle mea, samle varace Theorems: roertes of a radom samle, samle mea, samle varace Examles: how
More informationChapter -2 Simple Random Sampling
Chapter - Smple Radom Samplg Smple radom samplg (SRS) s a method of selecto of a sample comprsg of umber of samplg uts out of the populato havg umber of samplg uts such that every samplg ut has a equal
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationQualifying Exam Statistical Theory Problem Solutions August 2005
Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More information12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
More informationUnit 9. The Tangent Bundle
Ut 9. The Taget Budle ========================================================================================== ---------- The taget sace of a submafold of R, detfcato of taget vectors wth dervatos at
More informationMean is only appropriate for interval or ratio scales, not ordinal or nominal.
Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More information22 Nonparametric Methods.
22 oparametrc Methods. I parametrc models oe assumes apror that the dstrbutos have a specfc form wth oe or more ukow parameters ad oe tres to fd the best or atleast reasoably effcet procedures that aswer
More informationChapter 11 Systematic Sampling
Chapter stematc amplg The sstematc samplg techue s operatoall more coveet tha the smple radom samplg. It also esures at the same tme that each ut has eual probablt of cluso the sample. I ths method of
More informationA New Family of Transformations for Lifetime Data
Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several
More information4. Standard Regression Model and Spatial Dependence Tests
4. Stadard Regresso Model ad Spatal Depedece Tests Stadard regresso aalss fals the presece of spatal effects. I case of spatal depedeces ad/or spatal heterogeet a stadard regresso model wll be msspecfed.
More informationChapter -2 Simple Random Sampling
Chapter - Smple Radom Samplg Smple radom samplg (SRS) s a method of selecto of a sample comprsg of umber of samplg uts out of the populato havg umber of samplg uts such that every samplg ut has a equal
More informationSTK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
More informationRandom Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois
Radom Varables ECE 313 Probablty wth Egeerg Alcatos Lecture 8 Professor Rav K. Iyer Uversty of Illos Iyer - Lecture 8 ECE 313 Fall 013 Today s Tocs Revew o Radom Varables Cumulatve Dstrbuto Fucto (CDF
More informationChapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance
Chapter, Part A Aalyss of Varace ad Epermetal Desg Itroducto to Aalyss of Varace Aalyss of Varace: Testg for the Equalty of Populato Meas Multple Comparso Procedures Itroducto to Aalyss of Varace Aalyss
More informationMS exam problems Fall 2012
MS exam problems Fall 01 (From: Rya Mart) 1. (Stat 401) Cosder the followg game wth a box that cotas te balls two red, three blue, ad fve gree. A player selects two balls from the box at radom, wthout
More informationFactorization of Finite Abelian Groups
Iteratoal Joural of Algebra, Vol 6, 0, o 3, 0-07 Factorzato of Fte Abela Grous Khald Am Uversty of Bahra Deartmet of Mathematcs PO Box 3038 Sakhr, Bahra kamee@uobedubh Abstract If G s a fte abela grou
More information1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
More informationWu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1
Wu-Hausma Test: Detectg Falure of E( ε X ) Caot drectly test ths assumpto because lack ubased estmator of ε ad the OLS resduals wll be orthogoal to X, by costructo as ca be see from the momet codto X'
More informationSTA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
More informationChannel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory
Chael Models wth Memory Chael Models wth Memory Hayder radha Electrcal ad Comuter Egeerg Mchga State Uversty I may ractcal etworkg scearos (cludg the Iteret ad wreless etworks), the uderlyg chaels are
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER II STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for
More informationENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
More informationCS 2750 Machine Learning Lecture 5. Density estimation. Density estimation
CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationChapter 11 The Analysis of Variance
Chapter The Aalyss of Varace. Oe Factor Aalyss of Varace. Radomzed Bloc Desgs (ot for ths course) NIPRL . Oe Factor Aalyss of Varace.. Oe Factor Layouts (/4) Suppose that a expermeter s terested populatos
More informationLECTURE - 4 SIMPLE RANDOM SAMPLING DR. SHALABH DEPARTMENT OF MATHEMATICS AND STATISTICS INDIAN INSTITUTE OF TECHNOLOGY KANPUR
amplg Theory MODULE II LECTURE - 4 IMPLE RADOM AMPLIG DR. HALABH DEPARTMET OF MATHEMATIC AD TATITIC IDIA ITITUTE OF TECHOLOGY KAPUR Estmato of populato mea ad populato varace Oe of the ma objectves after
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationTHE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE
THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE PAPER I STATISTICAL THEORY The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the
More informationSTA 105-M BASIC STATISTICS (This is a multiple choice paper.)
DCDM BUSINESS SCHOOL September Mock Eamatos STA 0-M BASIC STATISTICS (Ths s a multple choce paper.) Tme: hours 0 mutes INSTRUCTIONS TO CANDIDATES Do ot ope ths questo paper utl you have bee told to do
More informationChapter 13 Student Lecture Notes 13-1
Chapter 3 Studet Lecture Notes 3- Basc Busess Statstcs (9 th Edto) Chapter 3 Smple Lear Regresso 4 Pretce-Hall, Ic. Chap 3- Chapter Topcs Types of Regresso Models Determg the Smple Lear Regresso Equato
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationThe number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter
LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s
More informationModule 7. Lecture 7: Statistical parameter estimation
Lecture 7: Statstcal parameter estmato Parameter Estmato Methods of Parameter Estmato 1) Method of Matchg Pots ) Method of Momets 3) Mamum Lkelhood method Populato Parameter Sample Parameter Ubased estmato
More informationMultiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades
STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos
More informationMedian as a Weighted Arithmetic Mean of All Sample Observations
Meda as a Weghted Arthmetc Mea of All Sample Observatos SK Mshra Dept. of Ecoomcs NEHU, Shllog (Ida). Itroducto: Iumerably may textbooks Statstcs explctly meto that oe of the weakesses (or propertes) of
More informationOn the characteristics of partial differential equations
Sur les caractérstques des équatos au dérvées artelles Bull Soc Math Frace 5 (897) 8- O the characterstcs of artal dfferetal equatos By JULES BEUDON Traslated by D H Delhech I a ote that was reseted to
More informationSTRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING
Joural of tatstcs: Advaces Theory ad Alcatos Volume 5, Number, 6, Pages 3- Avalable at htt://scetfcadvaces.co. DOI: htt://d.do.org/.864/jsata_7678 TRONG CONITENCY FOR IMPLE LINEAR EV MODEL WITH v/ -MIXING
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More informationEntropy, Relative Entropy and Mutual Information
Etro Relatve Etro ad Mutual Iformato rof. Ja-Lg Wu Deartmet of Comuter Scece ad Iformato Egeerg Natoal Tawa Uverst Defto: The Etro of a dscrete radom varable s defed b : base : 0 0 0 as bts 0 : addg terms
More informationIntroduction to Econometrics (3 rd Updated Edition, Global Edition) Solutions to Odd-Numbered End-of-Chapter Exercises: Chapter 9
Itroducto to Ecoometrcs (3 rd Udated Edto, Global Edto) by James H. Stock ad Mark W. Watso Solutos to Odd-Numbered Ed-of-Chater Exercses: Chater 9 (Ths verso August 7, 04) 05 Pearso Educato, Ltd. Stock/Watso
More informationExample: Multiple linear regression. Least squares regression. Repetition: Simple linear regression. Tron Anders Moger
Example: Multple lear regresso 5000,00 4000,00 Tro Aders Moger 0.0.007 brthweght 3000,00 000,00 000,00 0,00 50,00 00,00 50,00 00,00 50,00 weght pouds Repetto: Smple lear regresso We defe a model Y = β0
More informationProbability and Statistics. What is probability? What is statistics?
robablt ad Statstcs What s robablt? What s statstcs? robablt ad Statstcs robablt Formall defed usg a set of aoms Seeks to determe the lkelhood that a gve evet or observato or measuremet wll or has haeed
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationCHAPTER 3 POSTERIOR DISTRIBUTIONS
CHAPTER 3 POSTERIOR DISTRIBUTIONS If scece caot measure the degree of probablt volved, so much the worse for scece. The practcal ma wll stck to hs apprecatve methods utl t does, or wll accept the results
More informationSampling Theory MODULE V LECTURE - 14 RATIO AND PRODUCT METHODS OF ESTIMATION
Samplg Theor MODULE V LECTUE - 4 ATIO AND PODUCT METHODS OF ESTIMATION D. SHALABH DEPATMENT OF MATHEMATICS AND STATISTICS INDIAN INSTITUTE OF TECHNOLOG KANPU A mportat objectve a statstcal estmato procedure
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationLecture Outline. Biost 517 Applied Biostatistics I. Comparing Independent Proportions. Summary Measures. Comparing Independent Proportions
Bost 57 Aled Bostatstcs I Scott S. Emerso, M.D., Ph.D. Professor of Bostatstcs Uversty of Washgto Lecture 4: Two Samle Iferece About Ideedet Proortos Lecture Outle Comarg Ideedet Proortos Large Samles
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationThe equation is sometimes presented in form Y = a + b x. This is reasonable, but it s not the notation we use.
INTRODUCTORY NOTE ON LINEAR REGREION We have data of the form (x y ) (x y ) (x y ) These wll most ofte be preseted to us as two colum of a spreadsheet As the topc develops we wll see both upper case ad
More informationParameter Estimation
arameter Estmato robabltes Notatoal Coveto Mass dscrete fucto: catal letters Desty cotuous fucto: small letters Vector vs. scalar Scalar: la Vector: bold D: small Hgher dmeso: catal Notes a cotuous state
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationChapter 8: Statistical Analysis of Simulated Data
Marquette Uversty MSCS600 Chapter 8: Statstcal Aalyss of Smulated Data Dael B. Rowe, Ph.D. Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 08 by Marquette Uversty MSCS600 Ageda 8. The Sample
More information