Regresson Smple Lnear Regresson Model Least Squares Method Coeffcent of Determnaton Model Assumptons Testng for Sgnfcance Usng the Estmated Regresson Equaton for Estmaton and Predcton Resdual Analss: Valdatng Model Assumptons Outlers and Influental Oservatons The Smple Lnear Regresson Model Smple Lnear Regresson Model β 0 + β x + ε Smple Lnear Regresson Equaton E( β 0 + β x ^ Estmated Smple Lnear Regresson Equaton 0 + x
Least Squares Method Least Squares Crteron mn ( $ oserved value of the dependent varale ^ for the th oservaton estmated value of the dependent varale for the th oservaton The Least Squares Method Slope for the Estmated Regresson Equaton x ( x / n x ( x / n -Intercept for the Estmated Regresson Equaton 0 - x x value of ndependent varale for th oservaton value of dependent varale for th oservaton _ x mean value for ndependent varale _ mean value for dependent varale n total numer of oservatons
The Coeffcent of Determnaton Relatonshp Among SST, SSR, SSE SST SSR + SSE ( ( ^ + ( ^ Coeffcent of Determnaton r SSR/SST SST total sum of squares SSR sum of squares due to regresson SSE sum of squares due to error Model Assumptons Assumptons Aout the Error Term ε The error ε s a random varale wth mean of zero. The varance of ε, denoted σ, s the same for all values of the ndependent varale. The values of ε are ndependent. The error ε s a normall dstruted random varale.
Testng for Sgnfcance To test for a sgnfcant regresson relatonshp, we must conduct a hpothess test to determne whether the value of β s zero. Two tests are commonl used t Test F Test Both tests requre an estmate of σ, the varance of ε n the regresson model. An Estmate of σ Testng for Sgnfcance The mean square error (MSE provdes the estmate of σ, and the notaton s s also used. s MSE SSE/(n- SSE ( ˆ ( 0 x An Estmate of σ To estmate σ we take the square root of σ. The resultng s s called the standard error of the estmate. SSE s MSE n
Testng for Sgnfcance: t Test Hpotheses Test Statstc Rejecton Rule H 0 : β 0 H a : β 0 t s Reject H 0 f t < -t α/ or t > t α/ where t α/ s ased on a t dstruton wth n - degrees of freedom. Confdence Interval for β We can use a 95% confdence nterval for β to test the hpotheses just used n the t test. H 0 s rejected f the hpotheszed value of β s not ncluded n the confdence nterval for β. The form of a confdence nterval for β s: ± t α / where s the pont estmate t α / s s the margn of error t α / s the t value provdng an area of α/ n the upper tal of a t dstruton wth n - degrees of freedom s
Testng for Sgnfcance: F Test Hpotheses Test Statstc Rejecton Rule H 0 : β 0 H a : β 0 F MSR/MSE Reject H 0 f F > F α where F α s ased on an F dstruton wth d.f. n the numerator and n - d.f. n the denomnator. Some Cautons aout the Interpretaton of Sgnfcance Tests Rejectng H 0 : β 0 and concludng that the relatonshp etween x and s sgnfcant does not enale us to conclude that a cause-and-effect relatonshp s present etween x and. Just ecause we are ale to reject H 0 : β 0 and demonstrate statstcal sgnfcance does not enale us to conclude that there s a lnear relatonshp etween x and.
Usng the Estmated Regresson Equaton for Estmaton and Predcton Confdence Interval Estmate of E( p $ p ± t α / s $ p Predcton Interval Estmate of p p + t α/ s nd where the confdence coeffcent s - α and t α/ s ased on a t dstruton wth n - d.f. Resdual Analss If the assumptons aout the error term ε appear questonale, the hpothess tests aout the sgnfcance of the regresson relatonshp and the nterval estmaton results ma not e vald. The resduals provde the est nformaton aout ε. Much of the resdual analss s ased on an examnaton of graphcal plots.
Resdual Plot Aganst x If the assumpton that the varance of ε s the same for all values of x s vald, and the assumed regresson model s an adequate representaton of the relatonshp etween the varales: The resdual plot should gve an overall mpresson of a horzontal and of ponts Standardzed Resdual for Oservaton Standardzed Resduals ˆ s ˆ ˆ s s h ˆ ( x x h + n ( x x The standardzed resdual plot can provde nsght aout the assumpton that the error term ε has a normal dstruton. If ths assumpton s satsfed, the dstruton of the standardzed resduals should appear to come from a standard normal proalt dstruton.
Outlers and Influental Oservatons Detectng Outlers An outler s an oservaton that s unusual n comparson wth the other data. Mnta classfes an oservaton as an outler f ts standardzed resdual value s < - or > +. Ths standardzed resdual rule sometmes fals to dentf an unusuall large oservaton as eng an outler. Ths rule s shortcomng can e crcumvented usng studentzed deleted resduals. The th studentzed deleted resdual wll e larger than the th standardzed resdual.