17 - LINEAR REGRESSION II

Transcription

1 Topc 7 Lnear Regresson II 7- Topc 7 - LINEAR REGRESSION II Testng and Estmaton Inferences about β Recall that we estmate Yˆ ˆ β + ˆ βx. 0 μ Y X x β0 + βx usng To estmate σ σ squared error Y X x ε s ε we use MSE, the mean SSE MSE n Note that t has n degrees of freedom because Ŷ nvolves two estmates ( 0 ˆβ and βˆ ). a) Samplng Dstrbuton for β Under the assumptons of the lnear regresson model, we have that ˆ β s Normally dstrbuted wth mean μ β βˆ and

2 Topc 7 Lnear Regresson II 7- σ ε standard devaton σ ˆ β n ( x x) σ S ε xx 3 thngs to note: ) ˆ s an unbased estmator of β β ) an estmator of σ βˆ s SE ˆ β s S ε xx n MSE ( x x) (NOTE: s x s the standard devaton of X whereas S xx s the numerator of s x). 3) Snce t s a normally dstrbuted estmator we can calculate a CI and do a T-Test!!! b) Confdence Interval Estmaton of β A (-α)00% CI of β s ˆ β ± t SE. α, n ˆ β Example: ntolerant peas. A 95% CI s

3 Topc 7 Lnear Regresson II 7-3 n 0, ˆβ 0.05, SSE.3894, df 8, and Sxx ( x x) So, f we were dong ths by hand, we would now fnd SE β ˆ SS( resd) /( n ) n ( x x).3894 / And for 95% CI and df8, we need t (α/, n-) t (0.05,8).306. So, a 95% Confdence nterval estmate of β s ˆ β ± t α,( n ) SE ˆ β 0.05 ±.306( ) ( , 0.354) So, we are 95% confdent that the true slope of the relatonshp between dsease ndex (Y) and crop nterval (X) s contaned wthn the nterval.3087 to.354. See the SAS output as well. Parameter Estmates Parameter Standard Varable DF Estmate Error t Value Pr> t 95% Confdence Lmts Intercept < Interval

4 Topc 7 Lnear Regresson II 7-4 Although t s possble, one usually has no nterest n testng the ntercept. If you should decde you want to, the test s very smlar to that for the slope: t s a t-test and you can calculate a CI as well. c) Hypothess Tests of β Hypotheses a. Is there a relatonshp between Y and X? H o : β 0 versus H A : β 0 b. Is the relatonshp postve (as X, Y )? H o : β 0 versus H A : β > 0 c. Is the relatonshp negatve (as X, Y )? H o : β 0 versus H A : β < 0 Test Statstc: t obs β SE ˆ 0 ˆ β Conduct the test as you do any t-test,.e. ) choose the hypothess, ) choose α, ) calculate t obs and ts degrees of freedom (df n ) v) calculate the p-value, and v) draw a concluson.

5 Topc 7 Lnear Regresson II 7-5

6 Topc 7 Lnear Regresson II 7-6 Example: Pea Intolerance. Testng Parameter Estmates H 0 : β 0 0 H A : β 0 0 Parameter Standard Varable DF Estmate Error t Value Pr> t Intercept <.000 Interval Testng H 0 : β 0 H A : β 0 d) ANOVA table and R Decomposton of the SUMS of SQUARES Let s start by gnorng any possble relatonshp between X the explanatory varable and Y response varable. When gnorng X, we can look at the emprcal dstrbuton of Y:

7 Topc 7 Lnear Regresson II 7-7 e.g. for Yndex n the ntolerant peas study: P e r c 0 e n t Index The total sum of squares (TSS) s a measure of the varablty of Y around the sample mean Y (t s the numerator of s y ): n TSS ( y y) S yy ˆ A 4 ˆ A A I n A d 3 ˆƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒAƒƒƒƒƒƒƒƒAƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ e A x A ˆ A A ˆ Šƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒ Y Interval

8 Topc 7 Lnear Regresson II 7-8 Now, nstead of comparng the data to the overall mean, let s look at the regresson relatonshp. Recall that SSE s a measure of the varablty of the ponts around the regresson lne. We also call t the unexplaned varablty because t s the nose of the observatons around the predcted lne: SSE Y Y ) ( Y [ β 0 + ( ˆ ˆ ˆ β X ]) For the ntolerant peas study, SSE I ndex I nt er val N 0 Rsq Adj Rsq RMSE Y ˆ Y Interval It turns out that the total sum of squares can be broken nto two parts:

9 Topc 7 Lnear Regresson II 7-9 ( Y Y ) TSS ( Y SSR Yˆ ) + ( Y + SSE Yˆ ) where SSR s the regresson sum of squares ( explaned varablty) SSE s the error sum of squares ( unexplaned varablty) Plot of yhat*interval. Legend: A obs, B obs, etc. yhat ˆ A ˆ 4.5 ˆ ˆ ˆ A ˆ ˆ C 3.0 ˆƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ.790 ˆ A.568 ˆ C.346 ˆ.40 ˆ.900 ˆ.6799 ˆ.4579 ˆ A Šƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒˆƒƒ Y Interval Now, the degrees of freedom can also be decomposed. As we know, the df for TSS are (n ) (because f we wanted

10 Topc 7 Lnear Regresson II 7-0 to calculate the sample varance for Y, t would be TSS /( n ) ). So, the decomposton of the df s s Y TSS SSR + SSE ( n ) () + ( n ) All of ths can be put nto a table called the Analyss of Varance table: e.g. ntolerant peas: MSRSSR/ SSR Analyss of Varance F obs MSR/MSE Sum of Mean Source DF Squares Square F Value Pr > F Model Error CTotal SSE TSS MSESSE/(n-) p-value for F-test of H 0 : β 0 H A : β 0 The Mean squares are the SS dvded by ther degrees of freedom. The F-test tests whether a relatonshp exsts. It cannot be used to test drecton as can be done wth the t-test we just learned.

11 Topc 7 Lnear Regresson II 7- SSR / MSR The F-statstc s F obs and t has a SSE /( n ) MSE dstrbuton known as the F-dstrbuton. Further lke the t-dstrbuton, the F-dstrbuton depends on degrees of freedom for shape. In fact, the F-dstrbuton depends on both the numerator and denomnator degrees of freedom. It s denoted as F (num df, den df). Tables n your book are avalable for lookng up the crtcal values for the F-test for gven values of α. obs t obs (Note: for a sngle explanatory varable, F ). A measure of the amount of explaned varablty s R-squared, also called the coeffcent of determnaton, SSE r TSS e.g. ntolerant peas Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var The value of the coeffcent of determnaton s SSE.389 r TSS 7.449

12 Topc 7 Lnear Regresson II 7- Snce r s a proporton we have that 0 r no relatonshp between Y and X perfect relatonshp (no varablty left over!) NOTE: r assumes that f a relatonshp exsts t s a lnear relatonshp So, r mples that the regresson relatonshp explans approxmately 8% of the observed varablty n Y (ndex). SAS code used n ths example: data peas; nput ndex datalnes; ; proc reg datapeas; model ndex nterval /clb; *clb 95% confdence ntervals for the betas; plot ndex*nterval; * plot the response Y aganst X; plot r.*p.; * plot the resduals aganst the predcted values; output outresds pyhat rresd; * output the predcted values and resduals nto a fle named "resds"; qut; proc prnt dataresds; var nterval ndex yhat resd; qut; proc plot datapeas hpercent60 vpercent60; plot ndex*nterval / vref.99; * plot the observed Y aganst X wth a reference lne at ybar.99; qut; proc plot dataresds hpercent60 vpercent60; plot yhat*nterval / vref.99; * plot the predcted values aganst X wth a reference lne at ybar.99; qut;

13 Topc 7 Lnear Regresson II 7-3 Example: a scentst s nterested n plasma polyamne levels n chldren as a functon of age. He collected polyamne levels n chldren between the ages of 0 and 4: Age Plasma Log0(Plasma level) Level

14 Topc 7 Lnear Regresson II 7-4 A plot shows possble non-lnearty: Pl asmalevel Age The correlatons: Pearson Correlaton Coeffcents, N 5 Prob > r under H0: Rho0 Plasma log0_ Level plasma_ Age Age <.000 <.000 Spearman Correlaton Coeffcents, N 5 Prob > r under H0: Rho0 Plasma log0_ Level plasma_ Age Age <.000 <.000

15 Topc 7 Lnear Regresson II 7-5 We dd a regresson analyss: Model: MODEL Dependent Varable: log0_plasma_ log0(plasma) Number of Observatons Read 5 Number of Observatons Used 5 Analyss of Varance Sum of Mean Source DF Squares Square F Value Pr > F Model <.000 Error C Total Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estmates Parameter Standard Varable Label DF Estmate Error t Value Pr > t Intercept Intercept <.000 Age Age <.000 Parameter Estmates Varable Label DF 95% Confdence Lmts Intercept Intercept Age Age

16 Topc 7 Lnear Regresson II 7-6 l og0_pl asma_ Age. 3.. N 5 Rsq Adj Rsq RMSE Age l og0_pl asma_ Age N 5 Rsq Adj Rsq RMSE Pr ed ct ed Val ue

17 Topc 7 Lnear Regresson II 7-7 SAS code: PROC IMPORT OUT WORK.plasma DATAFILE "C:\Documents and Settngs\mcxman\My Documents\Classes\STA 666 Fall 007\ Plasma Polyamne by Age.xls" DBMSEXCEL REPLACE; SHEET"Plasma_Polyamne_by_Age"; GETNAMESYES; MIXEDNO; SCANTEXTYES; USEDATEYES; SCANTIMEYES; RUN; proc gplot data plasma; plot plasmalevel*age log0_plasma_*age; run; qut; proc corr dataplasma pearson spearman; var plasmalevel log0_plasma_; wth age; qut; proc reg dataplasma; model log0_plasma_ age /clb; plot log0_plasma_*age; plot r.*p.; output outresds pyhat rresd; qut; proc reg dataplasma; model plasmalevel age /clb; plot plasmalevel*age; plot r.*p.; output outresds pyhat rresd; qut;