Simple Linear Regression Analysis

Size: px

Start display at page:

Download "Simple Linear Regression Analysis"

Randall Brown
5 years ago
Views:

1 LINEAR REGREION ANALYSIS MODULE II Lecture - 5 Smple Lear Regreo Aaly Dr Shalabh Departmet of Mathematc Stattc Ida Ittute of Techology Kapur

2 Jot cofdece rego for A jot cofdece rego for ca alo be foud Such rego wll provde a 00( α% cofdece that both the etmate of are correct Coder the cetered vero of the lear regreo model β where β = β + β x 0 0 The leat quare etmator of β 0 β are b0 = y b = repectvely Ug the reult that Eb ( = β, Whe β 0 β β 0 y ( = β0 + β x x + ε 0 0 Eb ( = β, Var( b0 =, Var( b = kow, the the tattc β β 0 b β 0 0 ~ N(0, b β ~ N(0,

3 Moreover, both the tattc are depedetly dtrbuted Thu b b β 0 0 β ~ χ ~ χ 3 are alo depedetly dtrbuted becaue b ( β ( b β 0 o + ~ χ b b are depedetly dtrbuted Coequetly um of thee two 0 Sce re ~ χ re depedetly dtrbuted of b ( β ( b β re ( b ~ F b 0,, o the rato

4 4 Subttutg b = b + bx β = β + β, we get 0 0 Q f re where 0 0 x f = 0 β0 + 0 β β + β = = Q b ( x( b ( b x( b Sce Q P f re F, = α hold true for all value of β β, o the 00( α% cofdece rego for β β 0 0 Q f re F, ; α Th cofdece rego a ellpe whch gve the 00 ( α % probablty that β β are cotaed multaeouly th ellpe 0

5 Aaly of varace 5 The techque of aaly of varace uually ued for tetg the hypothe related to equalty of more tha oe parameter, lke populato mea or lope parameter It more meagful cae of multple regreo model whe there are more tha oe lope parameter Th techque dcued llutrated here to udert the related bac cocept fudametal whch wll be ued developg the aaly of varace the ext module multple lear regreo model where the explaatory varable are more tha two A tet tattc for tetg H0 : β = 0 ca alo be formulated ug the aaly of varace techque a follow O the ba of the detty y yˆ = ( y y ( yˆ y, the um of quared redual Further coder Thu we have The term Sb ( = ( y yˆ = = or total um of quare deoted a ( y ( ˆ ( ( ˆ y y y y y y y = = = = + ( y y( yˆ y = ( y y b( x x = = ( = = ( yˆ y = ( y ( ˆ ( ˆ y = y y + y y = = = = b x x ( y y called the um of quare about the mea or corrected um of quare of y (e, corrected

6 6 The term = wherea the term ( y yˆ = decrbe the devato: obervato mu predcted value, vz, the redual um of quare, e: ( yˆ y re g = ( y yˆ re = decrbe the proporto of varablty explaed by regreo, ( ˆ = = y y If all obervato y are located o a traght le, the th cae ( y ˆ y = 0 thu corrected = r e g = Note that reg completely determed by b o ha oly oe degree of freedom The total um of quare = ( y y = ha ( - degree of freedom due to cotrat ( y y = 0 re ha ( - degree of freedom a t deped o b 0 b = All um of quare are mutually depedet dtrbuted a dtrbuted The mea quare due to regreo MS re g= re g mea quare due to redual χ df wth df degree of freedom f the error are ormally The tet tattc for tetg re MSE = H : β = 0 0 F MS r e g 0 = MSE

7 7 If H0 : β = 0 true, the e MSE are depedetly dtrbuted thu The deco rule for at α MS r g F > F α H to reject H 0 f : β 0 0, ; level of gfcace The tet procedure ca be decrbed a Aaly of Varace table Aaly of varace for tetg H0 : β = 0 Source of varato Sum of quare Degree of freedom Mea Square F 0 F, ~ Regreo reg MS reg re Redual - MSE Total - Some other form of reg, re ca be derved a follow: The ample correlato coeffcet the may be wrtte a r = Moreover, we have b The etmator of = = r = e = th cae may be expreed a = re

8 8 Varou alteratve formulato for re are ue a well: = [ y ( b + b x ] re 0 = = [( y y b( x x] = = + b b = b = ( Ug th reult, we fd that corrected = = r e g re ( = = b = b

9 9 Goode of ft of regreo It ca be oted that a ftted model ca be ad to be good whe redual are mall Sce re baed o redual, o a meaure of qualty of ftted model ca be baed o S re Whe tercept term preet the model, a meaure of goode of ft of the model gve by R = = re g re Th kow a the coeffcet of determato Th meaure baed o the cocept that how much varato y tated by explaable by reg how much uexplaable part cotaed re The rato reg / decrbe the proporto of varablty that explaed by regreo relato to the total varablty of y The rato re / decrbe the proporto of varablty that ot covered by the regreo It ca be ee that R = r where r the mple correlato coeffcet betwee x y Clearly dcate the better ft value of R cloer to zero dcate the poor ft 0 R, o a value of R cloer to oe

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION I lear regreo, we coder the frequecy dtrbuto of oe varable (Y) at each of everal level of a ecod varable (X). Y kow a the depedet varable. The