1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands

Transcription

1 Content. Inference on Regresson Parameters a. Fndng Mean, s.d and covarance amongst estmates.. Confdence Intervals and Workng Hotellng Bands 3. Cochran s Theorem 4. General Lnear Testng 5. Measures of Assocaton

2 Inference on the regresson parameters: In the model: Y = We learnt how to estmate the parameters 0 and. If ths was all we wanted to do we could use least square estmates and not have to make dstrbutonal assumptons. However, n most cases ths s not enough and we want to make certan nferences on the parameters of regresson:. We are nterested n testng for the slope. Testng for the ntercept 3. Confdence nterval for slope 4. Confdence nterval for ntercept 5. Confdence ntervals and testng on the mean of Y 6. Predcton ntervals For all ths t s mportant to make the assumpton that the error s dstrbuted normally wth mean 0 and standard devaton.

3 At frst let us deal wth testng and confdence nterval for the slope. Frst we need to talk about the samplng dstrbuton of the estmate of. k y k y y y y y y S Sy Usng Algebra t s easy to verfy these results on k : S k k k, 0,.

4 We need to fnd the samplng dstrbuton of the rv k y, when we assume:. The y are uncorrelated. y ~ Normal Ey =, Vary =. k y Snce,, s a lnear functon of the rv y, t also follows a normal dstrbuton wth E E k y ke y k 0 Var Var k y k Var y k S

5 Inference about : For testng: H : 0 H 0 : 0 Test statstc: t 0 MSE/S Reject f Observed t > t/,n- Interpretaton: Under H 0 we are assumng no lnear relatonshp between and Y. Under H 0 : EY = = 0 does not depend on at all For the normal regresson model the condton =0 mples that all the probablty dstrbutons of Y are the same, Y ~ N 0, for all values of X. No assocaton of any form between X and Y snce the Y s reman dentcal for all values of X.

6 For testng H : H 0 : * * Test statstc: * t MSE/S Reject f Observed t > t/,n- Smlarly one-sded tests can be performed. Confdence ntervals: A 00-% confdence nterval for s gven by: t /, n MSE S

7 Inference about 0 : Ths s done only when the scope of the model ncludes X=0. Samplng dstrbuton of 0 y c y, c k. n s Usng results smlar to that for, we have, 0 normally dstrbuted wth E 0 0, Var 0 { n X S } Ths Varance s estmated by: Var MSE{ n 0 X S Hence the test statstc s gven by: 0 0 * t X MSE{ } n S follows a t dstrbuton wth n- degrees of freedom. A 00-% confdence nterval for 0 s gven by: 0 t /, n } MSE n X S

8 Some consderatons on makng nference concernng and :. Effects of departures from normalty If the dstrbuton of Y s not eactly normal but does not depart serously from normalty the samplng dstrbutons of the estmates wll be appromately normal.. Interpretaton of confdence coeffcents and rsks for errors: Snce t s assumed that the X s are constant the confdence ntervals are nterpreted wth respect to repeated observatons at the same level of X. 3. Spacng of the X levels: From the Varance formula for the estmates, t s evdent that f and n are held constant the larger the spacng among the X s the lower the varance.

9 Samplng dstrbuton of Y h Yh 0 h Propertes: E Y h Var Y h 0 h MSE{ n E Y h X h - X S } A 00-% confdence nterval for EY h s gven by: Y h t /, n MSE{ n X h - X S }

10 Comments:. Here too we assume that the X s are known constants. Confdence ntervals are nterpreted the same way as n the case of 0 and.. Smallest varance or mamum precson when X h s X. 3. Usual relatonshp wth confdence nterval and testng. 4. Confdence ntervals are not senstve to moderate departures form normalty. 5. The confdence ntervals can be etended to several X s.

11 Predcton of new observatons: Commonly asked queston: What s the dfference between EY h and Ŷ? hnew The dfference s basc: EY h s the mean of the dstrbuton of Y at a gven value of X and s a parameter. We estmate the populaton mean, EY h, by the sample mean Ŷ. For Ŷ hnew we want to predct a new value of Y for a specfed value of X. Ths s not a parameter. We use the estmated regresson functon to predct ths. h

12 new Samplng dstrbuton of Y h Y h new 0 h Y h A 00-% confdence nterval for Y hnew s gven by: h t /, n MSE{ n Y new X h - X S } Predcton ntervals for m observatons: Sometmes we may be nterested n predctng m new observatons at a partcular value of X, X h. Our predcton nterval would be gven by: h t /, n MSE{ m n Y new X h - X S }

13 Confdence Bands for regresson lne: Sometmes t may be of nterest to obtan a confdence band for the entre regresson lne, EY =. Ths band allows us to see the regon n whch the regresson lne les. The Workng-Hotellng -a00% confdence band for the dscussed regresson model s gven by: Y h W where W MSE{ n X h - X S F-,,n - Ths s a Scheffe Type multple comparson nterval. }

14 Notes and Comments on confdence bands:. The boundary values defne a hyperbola and hence the further X h s from X the wder s the band.. Often the confdence band boundares are not too much wder than the ndvdual confdence nterval boundares. Eample: t.975, 34 = W = The confdence bands apply to all real X s -,. We only nterpret the values of X that are kept at the same level as our study. 4. There are qute a few other alternatves to the Workng- Hotellng s confdence bands. These are the smplest.

15 ANOVA table for testng =0 Source df SS MS EMS F Regresson Ŷ Y SSR + S MSR MSE Error n- SSE Y - Ŷ Total n- Y - Y n Cochran s Theorem: Let Y ~ N =,,n and Sum of Squares Total SSTO be decomposed nto k sums of squares, SSr each wth degrees of freedom df r. Then SSr/df r are all ndependent wth degrees of freedom dfr. Also df TO = df r.

16 Constructon of Decson Rule: From Cochran s Theorem we have under H 0 : =0 SSR SSE and are ndependent. Hence, F * MSR MSE n- SSR / σ SSE /n σ / / n follows a F dstrbuton wth and n- degrees of freedom. Equvalence of F and t tests: F * F * SSR/ SSE/n - MSE / S S MSE MSE / S 0 MSE / S t Ths s ONLY true for F dstrbuton wth degree of freedom.

17 General Lnear Testng: It has 3 basc steps. Very useful testng procedure when dealng wth comple testng problems. However t s just as useful for the smple ones. Step - Full model: The full or unrestrcted model s the normal regresson model Y = + For ths model fnd the Error Sum of Squares, SSEFull = Y Y Ŷ SSE 0 Step Reduced Model: Now wrte out the model under the null hypothess. For eample for testng =0 the reduced model s Y =. Hence the Sum of Squares for the reduced model s: SSERed = Y Y Y SSTO Now t s logcal that SSEFull SSERed. 0 Ths s because the full model eplans the model n more detal and should have a smaller error compared to the reduced model.

18 Step 3 - The Test statstc: The statstc s always a F statstc of form: F* SSERed - SSEFull df df red full SSEFull df full In our case ths s: SSTO-SSE SSE F* df df df TO E E SSR/ SSE/n - Whch s dentcal to the ANOVA F-statstc.

19 Descrptve Measures of Assocaton:. Coeffcent of Determnaton, R : Defned by = R SSTO-SSE SSTO - SSE SSTO SSR SSTO Interpreted as the proporton of the total varaton n Y that has been eplaned by the model. 0 R. Correlaton Coeffcent, r: For smple lnear regresson For the tree data: R = r = 0.8 r R

20 Lmtatons of R and r : Common Msunderstandngs:. Hgh Correlaton ndcates useful predctons can be made.. Hgh correlaton ndcates that the estmated regresson lne s a good ft. 3. Correlaton coeffcent close to zero ndcates X and Y are not related.

21 Some Consderatons n Applyng Regresson:. Predctons about the future: Have to assume that the basc causal condtons reman the same over tme.. Predctng Y when X s tself predcted: Ths s condtonal regresson of Y gven X whch s consdered stochastc. Here t s assumed that Y and X are dstrbuted n a bvarate normal dstrbuton and we are makng condtonal nferences on Y, at a fed value of X. Also sometmes called correlaton models*. 3. Predctng outsde the range of the data: Ths s EXTRAPOLATION. Etreme cauton needs to be used. 4. Implcaton of Test 0: no CAUSAL relatonshp can be mpled.

22 Correlaton Models: Ths s gven n. n your book. Essentally the phlosophy s dfferent, snce here Y and Y are both random varables. But practcally the procedure s smlar snce we conduct our nference for a FIXED value of Y, and we can proceed wth usual technques lke least squares.. The observatons of Y s are consdered ndependent. The Y observatons when Y s consdered fed follows a normal dstrbuton wth EY Y = =, wth constant varance where = And And