ECONOMETRIC THEORY MODULE IV Lecture - 16 Predictions in Liner Regression Model Dr. Shlbh Deprtent of Mthetics nd Sttistics Indin Institute of Technology Knpur
Prediction of vlues of study vrible An iportnt use of liner regression odeling is to predict the verge nd ctul vlues of study vrible. The ter prediction of vlue of study vrible corresponds to knowing the vlue of E(y) (in cse of verge vlue) nd vlue of y (in cse of ctul vlue) for given vlue of explntory vrible. We consider both the cses. The prediction of vlues consists of two steps. In the first step, the regression coefficients re estited on the bsis of given observtions. In the second step, these estitors re then used to construct the predictor which provides the prediction of ctul or verge vlues of study vribles. Bsed on this pproch of construction of predictors, there re two situtions in which the ctul nd verge vlues of study vrible cn be predicted- within sple prediction nd outside sple prediction. We describe the prediction in both the situtions.` Within sple prediction in siple liner regression odel Consider the liner regression odel Bsed on sple of n sets of pired observtions N(, σ ). The preters β nd β1 re estited using the ordinry lest squres estition s b of β nd b1 of β1 s where The fitted odel is y= b + bx. y = β + β + ε 1 x. ( x, y ) ( i 1,,..., n) following y = β + β x + ε where εi s re identiclly nd independently distributed following i i = i 1 i i b = y bx 1 sxy b1 = s 1 1 s x x y y s x x x x y y n n n n xy = ( i )( i ), = ( i ), = i, = i. i= 1 i= 1 n i= 1 n i= 1 1
Cse 1: Prediction of verge vlue of y 3 Suppose we wnt to predict the vlue of E(y) for given vlue of x = x. Then the predictor is given by p = b + bx 1 Here stnds for en vlue.. Predictive bis The prediction error is given s p E( y) = b + bx E( β + β x + ε) Then the prediction bis is given s 1 1 = b + bx ( β + β x ) 1 1 = ( b β ) + ( b β ) x. 1 1 [ ( )] = ( β) + ( 1 β1) E p E y Eb Eb x Thus the predictor p = + =. is n unbised predictor of E(y). Predictive vrince The predictive vrince of p is PV ( p ) = Vr( b + b x ) 1 [ ( )] = Vr y + b x x 1 = Vr( y) + ( x x) Vr( b ) + ( x x) Cov( y, b ) σ σ ( x x) = + + n s 1 ( x x) = σ + n s 1 1.
Estite of predictive vrince 4 The predictive vrince cn be estited by substituting 1 ( x x) PV ( p ) ˆ = σ + n s 1 ( x x) = MSE + n s. ˆ σ by σ = MSE s Prediction intervl estition The 1(1 α)% prediction intervl for E(y) is obtined s follows: The predictor p is liner cobintion of norlly distributed rndo vribles, so it is lso norlly distributed s ( β+ β1 ( ) ) p ~ N x, PV p. So if σ is known, then the distribution of p E( y) PV ( p ) is N(,1). So the 1(1 α)% prediction intervl is obtined s p E( y) P z z = α α 1 α PV ( p) which gives the prediction intervl for E(y) s 1 ( x x) 1 ( x x) p zα σ +, p + zα σ +. n s n s
5 σ ˆ σ = MSE When is unknown, it is replced by nd in this cse, the spling distribution of p E( y) 1 ( x x) MSE + n s is t-distribution with (n - ) degrees of freedo, i.e., t n -. The 1(1 α)% prediction intervl in this cse is p E( y) P t t = MSE + n s α α 1 α, n, n 1 ( x x) which gives the prediction intervl s 1 ( x x) 1 ( x x) p tα MSE +, p + tα MSE +., n, n n s n s Note tht the width of prediction intervl E(y) is function of x. The intervl width is iniu for x = x nd widens s x increses. This is expected lso s the best estites of y to be de t x-vlues lie ner the center of the dt x nd the precision of estition to deteriorte s we ove to the boundry of the x-spce.
6 Cse : Prediction of ctul vlue If x is the vlue of the explntory vrible, then the ctul vlue predictor for y is p = b + bx 1. Here ens ctul. Note tht the for of predictor is se s tht of verge vlue predictor but its predictive error nd other properties re different. This is the dul nture of predictor. Predictive bis The predictive error of p is given by p y = b + bx ( β + β x + ε) 1 1 = ( b β ) + ( b β ) x ε. 1 1 Thus, we find tht E( p y) = Eb ( β ) + Eb ( β ) x E( ε) 1 1 = + + = which iplies tht p is n unbised predictor of y.
Predictive vrince 7 Becuse the future observtion y is independent of p, the predictive vrince of p is PV ( p ) = E( p y ) = E[( b β ) + ( x x)( b β ) + ( b β ) x ε ] 1 1 1 1 = Vr( b ) + ( x x) Vr( b ) + x Vr( b ) + Vr( ε ) + ( x x) Cov( b, b ) + xcov( b, b ) + ( x x) Vr( b ) 1 1 1 1 1 [rest of the ters re ssuing the independence of ε with ε, ε,..., ε ] 1 = Vr( b ) + [( x x) + x + ( x x)] Vr( b ) + Vr( ε ) + [( x x) + x] Cov( b, b ) 1 1 = Vr( b ) + x Vr( b ) + Vr( ε ) + x Cov( b, b ) 1 1 1 x = σ + + σ + x σ x n s s s 1 n ( x x) s = σ 1+ +. xσ n Estite of predictive vrince The estite of predictive vrince cn be obtined by replcing by its estite ˆ σ = MSE s 1 ( x x) PV ( p ) ˆ = σ 1+ + n s 1 ( x x) = MSE + + n s 1. σ
Prediction intervl 8 If σ is known, then the distribution of p E( p ) PV ( p ) is N(,1). So the 1(1 α)% prediction intervl is obtined s p E( p ) P z z = α α 1 α PV ( p ) ŷ s which gives the prediction intervl for When σ 1 ( x x) 1 ( x x) p zα σ 1 + +, p + zα σ 1 + +. n s n s is unknown, then p E( p) PV ( p ) follows t-distribution with (n - ) degrees of freedo. The 1(1 α)% prediction intervl for in this cse is obtined ŷ s p E( p ) P t t = 1, n, n PV ( p ) α α α which gives the prediction intervl 1 ( x x) 1 ( x x) p tα MSE 1 + +, p + tα MSE 1 + +., n, n n s n s
9 The prediction intervl is of iniu width t x x nd widens s x x increses. = The prediction intervl for p is wider thn the prediction intervl for p becuse the prediction intervl for p depends on both the error fro the fitted odel s well s the error ssocited with the future observtions. Within sple prediction in ultiple liner regression odel Consider the ultiple regression odel with k explntory vribles s y = Xβ + ε, where y = ( y1, y,..., y n )' is n 1 vector of n observtion on study vrible, X x x x x x x x x x 11 1 1k 1 k = n1 n nk is n k trix of n observtions on ech of the k explntory vribles, = (,,..., )' is k 1 vector of β β β β k 1 ε = ( ε1, ε,..., ε n )' n 1 (, σ ). regression coefficients nd is vector of rndo error coponents or disturbnce ter N I n (1,1,...,1)' following. If intercept ter is present, tke first colun of X to be. β 1 = p = Xb Let the preter be estited by its ordinry lest squres estitor b ( X ' X) X ' y. Then the predictor is which cn be used for predicting the ctul nd verge vlues of study vrible. This is the dul nture of predictor.
1 Cse 1: Prediction of verge vlue of y When the objective is to predict the verge vlue of y, i.e., E(y) then the estition error is given by p E( y) = Xb X β = Xb ( β ) = X X X = Hε where H = X X X X 1 ( ' ) '. X 1 ( ' ) ' ε Then [ ] E p E( y) = Xβ Xβ = which proves tht the predictor p = Xb provides unbised prediction for verge vlue. The predictive vrince of p is The predictive vrince cn be estited by PV ( p) = ˆ σ k where ˆ σ = MSE is obtined fro nlysis of vrince bsed on OLSE. = E = E { } { } PV ( p) = E p E( y) ' p E( y) [ ε' HH. ε] ( ε' Hε) = σ tr H = σ k.
Cse : Prediction of ctul vlue of y 11 When the predictor p = Xb is used for predicting the ctul vlue of study vrible y, then its prediction error is given by p y = Xb X β ε = Xb ( β) ε 1 = X( X ' X) X ' ε ε = I X X X X 1 ( ' ) ' The predictive vrince in this cse is ε = Hε. Thus E( p y) = which shows tht p provides unbised predictions for the ctul vlues of study vrible. = E = E ( ) PV ( p) = E p y) '( p y ( ε' HH. ε) ( ε' Hε) = σ trh σ ( n k). = The predictive vrince cn be estited by where ˆ σ = MSE is obtined fro nlysis of vrince bsed on OLSE. Copring the perfornces of p to predict ctul nd verge vlues, we find tht p in better predictor for predicting the verge vlue in coprison to ctul vlue when or PV ( p) = ˆ σ ( n k) PV < PV ( p) k< ( n k) or k < n, i.e., when the totl nuber of observtions re ore thn twice the nuber of explntory vribles.