Chapter 9 Autocorrelation

Transcription

1 Chaper 9 Aocorrelaio Oe of he basic assmpios i liear regressio model is ha he radom error compoes or disrbaces are ideically ad idepedely disribed So i he model y = Xβ +, i is assmed ha σ if s = E (, s) = if s ie, he correlaio bewee he sccessive disrbaces is zero I his assmpio, whe E (, s) = σ, s= is violaed, ie, he variace of disrbace erm does o remais cosa, he problem of heeroskedasiciy arises Whe E (, s) =, s is violaed, ie, he variace of disrbace erm remais cosa hogh he sccessive disrbace erms are correlaed, he sch problem is ermed as problem of aocorrelaio Whe aocorrelaio is prese, some or all off diagoal elemes i E( ') are ozero Someimes he sdy ad explaaory variables have a aral seqece order over ime, ie, he daa is colleced wih respec o ime Sch daa is ermed as ime series daa The disrbace erms i ime series daa are serially correlaed The aocovariace a lag s is defied as γ = E (, ); s=, ±, ±, s s A zero lag, we have cosa variace, ie, γ E ( ) = = σ The aocorrelaio coefficie a lag s is defied as Assme E( s) γ s ρs = = ; s =, ±, ±, Var( ) Var( ) γ s ρ s ad γ s are symmerical i s, ie, hese coefficies are cosa over ime ad deped oly o legh of lag s The aocorrelaio bewee he sccessive erms ( ad ), 3 gives he aocorrelaio of order oe, ie, ( ad ),,( ad ) ρ Similarly, he aocorrelaio Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr

2 bewee he sccessive erms ( 3 ad ),( 4 ad )( ad ) gives he aocorrelaio of order wo, ie, ρ Sorce of aocorrelaio Some of he possible reasos for he irodcio of aocorrelaio i he daa are as follows: Carryover of effec, aleas i par, is a impora sorce of aocorrelaio For example, he mohly daa o expedire o hosehold is ifleced by he expedire of precedig moh The aocorrelaio is prese i cross-secio daa as well as ime series daa I he cross-secio daa, he eighborig is ed o be similar wih respec o he characerisic der sdy I ime series daa, he ime is he facor ha prodces aocorrelaio Wheever some orderig of samplig is is prese, he aocorrelaio may arise Aoher sorce of aocorrelaio is he effec of deleio of some variables I regressio modelig, i is o possible o iclde all he variables i he model There ca be varios reasos for his, eg, some variable may be qaliaive, someimes direc observaios may o be available o he variable ec The joi effec of sch deleed variables gives rise o aocorrelaio i he daa 3 The misspecificaio of he form of relaioship ca also irodce aocorrelaio i he daa I is assmed ha he form of relaioship bewee sdy ad explaaory variables is liear If here are log or expoeial erms prese i he model so ha he lieariy of he model is qesioable he his also gives rise o aocorrelaio i he daa 4 The differece bewee he observed ad re vales of variable is called measreme error or errors i-variable The presece of measreme errors o he depede variable may also irodce he aocorrelaio i he daa Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr

3 Srcre of disrbace erm: Cosider he siaio where he disrbaces are aocorrelaed, γ γ γ γ γ γ E( ') = γ γ γ ρ ρ ρ ρ γ = ρ ρ ρ ρ ρ ρ = σ ρ ρ Observe ha ow here are ( + k) parameers- β, β,, β, σ, ρ, ρ,, ρ These ( + k) parameers are k o be esimaed o he basis of available observaios Sice he mber of parameers are more ha he mber of observaios, so he siaio is o good from he saisical poi of view I order o hadle he siaio, some special form ad he srcre of he disrbace erm is eeded o be assmed so ha he mber of parameers i he covariace marix of disrbace erm ca be redced The followig srcres are poplar i aocorrelaio: Aoregressive (AR) process Movig average (MA) process 3 Joi aoregressio movig average (ARMA) process Aoregressive (AR) process The srcre of disrbace erm i aoregressive process (AR) is assmed as = φ + φ + + φq q+, ie, he crre disrbace erm depeds o he q lagged disrbaces ad φ, φ,, φ k are he parameers (coefficies) associaed wih,,, q respecively A addiioal disrbace erm is irodced i which is assmed o saisfy he followig codiios: Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 3

4 E E ( ) = ( ) s σ if s = = if s This process is ermed as AR ( q ) process I pracice, he ( ) AR process is more poplar Movig average (MA) process: The srcre of disrbace erm i he movig average (MA) process is = + θ + + θp p, ie, he prese disrbace erm depeds o he p lagged vales The coefficies θ, θ,, θ p are he parameers ad are associaed wih,,, process p respecively This process is ermed as MA ( p ) 3 Joi aoregressive movig average (ARMA) process: The srcre of disrbace erm i he joi aoregressive movig average (ARMA) process is = φ + + φ + + θ + + θ q q p p This is ermed as ARMA( q, p ) process The mehod of correlogram is sed o check ha he daa is followig which of he processes The correlogram is a wo dimesioal graph bewee he lag s ad aocorrelaio coefficie ploed as lag s o X -axis ad ρ s o y -axis ρ s which is I MA () process = + θ θ for s = ρ s = + θ for s ρ = ρ ρ = i =,3, i So here is o aocorrelaio bewee he disrbaces ha are more ha oe period apar Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 4

5 I ARMA (,) process = φ + + θ ( + φθ )( φ + θ ) for s = ρ ( s = + θ + φθ ) φρ s for s + θ+ φθ σ = σ φ The aocorrelaio fcio begis a some poi deermied by boh he AR ad MA compoes b hereafer, declies geomerically a a rae deermied by he AR compoe I geeral, he aocorrelaio fcio - is ozero b is geomerically damped for AR process - becomes zero afer a fiie mber of periods for MA process The ARMA process combies boh hese feares The resls of ay lower order of process are o applicable i higher order schemes As he order of he process icreases, he difficly i hadlig hem mahemaically also icreases Esimaio der he firs order aoregressive process: Cosider a simple liear regressio model y = β + β X +, =,,, Assme ' s follow a firs order aoregressive scheme defied as i = ρ + where ρ <, E( ) =, σ if s E(, s) = + = if s for all =,,, where ρ is he firs order aocorrelaio bewee ad,,,, = Now = ρ + = ρ( + ) + = = + ρ + ρ + = r= r ρ r Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 5

6 E ( ) = E ( ) = E( ) + ρ E( ) + ρ E( ) + 4 = + ρ + ρ + σ s 4 ' ( ) ( are serially idepede) σ = = ρ ( ) σ for all E Similarly, I geeral, ( ρ ρ ) ( ρ ρ ) E( ) = E E = ρσ { ρ ( ρ )}{ ρ } = E ( ) = ρe + ρ + ( ) = ρσ E( ) = ρσ s s ρ ρ ρ ρ ρ ρ E( ') =Ω= σ ρ ρ ρ 3 ρ ρ ρ 3 Noe ha he disrbace erms are o more idepede ad ospherical E ( ') σ I The disrbace are Coseqeces of aocorrelaed disrbaces: Cosider he model wih firs order aoregressive disrbaces y = X β + k k = ρ +, =,,, wih assmpios E ( ) =, E ( ') = Ω σ if s E( ), E( s) = = + = if s where Ω is a posiive defiie marix Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 6

7 The ordiary leas sqares esimaor of β is b= ( X ' X) X ' y = X X X Xβ + ( ' ) '( ) b β = ( X ' X) X ' Eb ( β ) = So OLSE remais biased der aocorrelaed disrbaces The covariace marix of b is Vb ( ) = Eb ( β)( b β)' = ( X ' X) X ' E( ') X( X ' X) = ( X ' X) X ' ΩX( X ' X) σ ( X ' X) The residal vecor is Sice so e = y Xb = Hy = H e' e = y ' Hy = ' H Eee ( ' ) = E ( ' ) E ' X( X' X) X' = σ r( X ' X ) X ' ΩX s ee =, so ' σ E( s ) = r( X ' X ) X ' ΩX, s is a biased esimaor of σ I fac, s has dowward bias Applicaio of OLS fails i case of aocorrelaio i he daa ad leads o serios coseqeces as overly opimisic view from arrow cofidece ierval R sal -raio ad F raio ess provide misleadig resls predicio may have large variaces Sice disrbaces are ospherical, so geeralized leas sqares esimae of β yields more efficie esimaes ha OLSE Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 7

8 The GLSE of β is ˆ β = ( ' Ω ) ' Ω E( ˆ β) = β V ˆ = X Ω X X X X y ( β) σ ( ' ) The GLSE is bes liear biased esimaor of β Tess for aocorrelaio: Drbi Waso es: The Drbi-Waso (D-W) es is sed for esig he hypohesis of lack of firs order aocorrelaio i he disrbace erm The ll hypohesis is H : ρ = Use OLS o esimae β i y = Xβ + ad obai residal vecor e = y Xb = Hy where b= X X X y H = I X X X X ( ' ) ', ( ' ) ' The D-W es saisic is d = = ( e e ) = e e e ee = = = e e e = = = = + For large, d + r d ( r) where r is he sample aocorrelaio coefficie from residals based o OLSE ad ca be regarded as he regressio coefficie of e o e Here Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 8

9 posiive aocorrelaio of egaive aocorrelaio of e s d < e s d > zero aocorrelaio of As < r <, so e s d if < r<, he < d< 4 ad if < r<, he < d< So d lies bewee ad 4 Sice e depeds o X, so for differe daa ses, differe vales of d are obaied So he samplig disribio of d depeds o X Coseqely exac criical vales of d cao be ablaed owig o heir depedece o X Drbi ad Waso herefore obaied wo saisics d ad d sch ha d< d< d ad heir samplig disribios do o deped po X Cosiderig he disribio of d ad d, hey ablaed he criical vales as d L ad d U respecively They prepared he ables of criical vales for 5< < ad k 5 Now ables are available for 6 < < ad k The es procedre is as follows: H : Nare of H Rejec H whe Reai H whe The es is icoclsive whe H : d < dl d > du dl < d < du H : d > d L d < (4 d U ) ( 4 du) < d < (4 dl) H : d < dl du < d < (4 du) dl < d < du d > (4 dl) or (4 du) < d < (4 dl) Vales of d L ad d U are obaied from ables Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 9

10 Limiaios of D-W es If d falls i he icoclsive zoe, he o coclsive iferece ca be draw This zoe becomes fairly larger for low degrees of freedom Oe solio is o rejec H if he es is icoclsive A beer solios is o modify he es as Rejec H whe Accep H whe d< d U d d U This es gives saisfacory solio whe vales of x i s chage slowly, eg, price, expedire ec The D-W es is o applicable whe iercep erm is abse i he model I sch a case, oe ca se aoher criical vales, say d M i place of d L The ables for criical vales d M are available 3 The es is o valid whe lagged depede variables appear as explaaory variables For example, y = β y + β y + + β y + β x + + β x +, r r r+ k k, r = ρ + I sch case, Drbi s h es is sed which is give as follows Drbi s h-es Apply OLS o y = β y + β y + + β y + β x + + β x +, r r r+ k k, r = ρ + ad fid OLSE b of β Le is variace be Var( b ) ad is esimaor is Var ( b ) The he Dbi s h - saisic is h= r Var ( b ) which is asympoically disribed as N (,) ad r = = = ee e Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr

11 This es is applicable whe is large Whe followig es procedre ca be adoped Irodce a ew variable o = ρ + The Var ( b ) <, he es breaks dow I sch cases, he e = δρ + y Now apply OLS o his model ad es H A : δ = verss H A : δ sig -es I H A is acceped he accep H : ρ = If H A : δ = is rejeced, he rejec H : ρ = 4 If H : ρ = is rejeced by D-W es, i does o ecessarily mea he presece of firs order aocorrelaio i he disrbaces I cold happe becase of oher reasos also, eg, disribio may follows higher order AR process some impora variables are omied dyamics of model is misspecified fcioal erm of model is icorrec Esimaio procedres wih aocorrelaed errors whe aocorrelaio coefficie is kow Cosider he esimaio of regressio coefficie der firs order aoregressive disrbaces ad aocorrelaio coefficie is kow The model is y = Xβ +, = ρ + ad assme ha E = E = ψ σ I E = E = σ I ) ( ), ( '), ( ), ( ') The OLSE of β is biased b o, i geeral, efficie ad esimae of σ is biased So we se geeralized leas sqares esimaio procedre ad GLSE of β is where ˆ β = ( ' ψ ) ' ψ X X X y Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr

12 ψ ρ ρ ρ ρ + ρ + ρ = + ρ ρ ρ To employ his, we proceed as follows: Fid a marix P sch ha PP ' = ψ I his case ρ ρ ρ P = ρ Trasform he variables as y* = Py, X* = PX, * = P Sch rasformaio yields ρ y ρ ρ x ρ x k y ρ y ρ x ρx xk ρx k y* = y3 ρ y, X* ρ x3 ρx x3k ρx = k y ρ y ρ x ρx,, x ρx Noe ha he firs observaio is reaed differely ha oher observaios For he firs observaio, ( ρ ) ( ) ( ) = ρ ' β + ρ whereas for oher observaios where y x ( ) y = ρy = x ρx )' β + ( ρ ; =,3,, ' x is a row vecor of X Also, ρ ad ( ρ) hese wo errors o be correlaed ad homoscedasic have same properies So we expec Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr

13 If firs colm of X is a vecor of oes, he firs colm of X * is o cosa Is firs eleme is ρ Now employ OLSE wih observaios y * ad X *, he he OLSE of β is β * = ( X *' X*) X *' y*, is covariace marix is V( ˆ β) = σ ( X *' X*) ad is esimaor is where = σ ( X ' ψ X) Vˆ ( ˆ β) = ˆ σ ( X ' ψ X) y X ˆ y X ˆ k ( β)' ψ ( β) ˆ σ = Esimaio procedres wih aocorrelaed errors whe aocorrelaio coefficie is kow Several procedre have bee sggesed o esimae he regressio coefficies whe aocorrelaio coefficie is kow The feasible GLSE of β is ˆ ( ' ˆ ) ' ˆ β F = X Ω X X Ω y where ˆ Ω is he Ψ marix wih ρ replaced by is esimaor ˆρ Use of sample correlaio coefficie Mos commo mehod is o se he sample correlaio coefficie r bewee sccessive residals as he aral esimaor of ρ The sample correlaio ca be esimaed sig he residals i place of disrbaces as where r = = = ee e e = y xb, =,,, ad b is OLSE of β ' Two modificaios are sggesed for r which ca be sed i place of r Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 3

14 k r* = r is he Theil s esimaor d r ** = for large where d is he Drbi Waso saisic for H : ρ = Drbi procedre: I Drbi procedre, he model is expressed as y ρy = β ( ρ) + β( x ρx ) +, =,3,, y = β ( ρ) + ρ y + β x ρβ x + = β + ρy + βx + β x +, =,3,, (*) * * where β = β ( ρ ), β = ρβ * * Now r regressio sig OLS o model (*) ad esimae r * as he esimaed coefficie of Aoher possibiliy is ha sice ρ (,), so search for a siable ρ which has smaller error sm of sqares 3 Cochrae-Orc procedre: This procedre ilizes P marix defied while esimaig β whe ρ is kow I has followig seps: (i) Apply OLS o y = β + βx + ad obai residal vecor e y (ii) (iii) Esimae ρ by r = = = ee e Noe ha r is a cosise esimaor of ρ Replace ρ by r is y ρy = β ( ρ) + β( x ρx ) + ad apply OLS o rasformed model y ry = β + β( x rx ) + disrbace erm * * ad obai esimaors of β ad β as ˆ β * ad ˆ β respecively This is Cochrae-Orc procedre Sice wo sccessive applicaios of OLS are ivolved, so i is also called as wo-sep procedre Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 4

15 This applicaio ca be repeaed i he procedre as follows: * (I) P ˆ β ad ˆ β i origial model (II) Calclae he residal sm of sqares (III) Calclae ρ by r = = = ee e ad sbsie i i he model (IV) y ρy = β ( ρ) + β( x ρx ) + ad agai obai he rasformed model Apply OLS o his model ad calclae he regressio coefficies This procedre is repeaed il covergece is achieved, ie, ierae he process ill he wo sccessive esimaes are early same so ha sabiliy of esimaor is achieved This is a ieraive procedre ad is merically coverge procedre Sch esimaes are asympoically efficie ad here is a loss of oe observaio 4 Hildreh-L procedre or Grid-search procedre: The Hilreh-L procedre has followig seps: (i) Apply OLS o ( y ρy ) = β ( ρ) + β( x ρx ) +, =,3,, (ii) (iii) sig differe vales of ρ( ρ ) sch as ρ = ±,, Calclae residal sm of sqares i each case Selec ha vale of ρ for which residal sm of sqares is smalles Sppose we ge ρ = 4 Now choose a fier grid For example, choose ρ sch ha 3 < ρ < 5 ad cosider ρ = 3, 3,, 49 ad pick p ha ρ wih smalles residal sm of sqares Sch ieraio ca be repeaed il a siable vale of ρ correspodig o miimm residal sm of sqares is obaied The seleced fial vale of ρ ca be sed ad for rasformig he model as i he case of Cocharae-Orc procedre The esimaors obaied wih his procedre are as efficie as obaied by Cochrae-Orc procedre ad here is a loss of oe observaio Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 5

16 5 Prais-Wiso procedre This is also a ieraive procedre based o wo sep rasformaio (i) Esimae ρ by ˆ ρ = = = 3 ee e where e s are residals based o OLSE (ii) Replace ρ by ˆρ is he model as i Cochrae-Orc procedre ( ˆ ρ ) ( ) ( ) ( ) = ˆ ρ β + β ˆ ρ ˆ + ρ y x y ˆ ρy = ( ˆ ρβ ) + β( x ˆ ρx ) + ( ˆ ρ ), =,3,, (iii) Use OLS for esimaig he parameers The esimaors obaied wih his procedre are asympoically as efficie as bes liear biased esimaors There is o loss of ay observaio (6) Maximm likelihood procedre Assmig ha y ~ N( Xβσψ, ), he likelihood fcio for β, ρ ad σ is L= y X y X ψ ( πσ ) exp ( )' ( ) β ψ β σ Igorig he cosa ad sig ψ =, he log-likelihood is ρ l L= l L( βσ,, ρ) = lσ + l( ρ) ( y Xβ) ' ψ ( y Xβ) σ The maximm likelihood esimaors of β, ρ ad σ ca be obaied by solvig he ormal eqaios l L l L l L =, =, = β ρ σ There ormal eqaios r o o be oliear i parameers ad ca o be easily solved Oe solio is o - firs derive he maximm likelihood esimaor of σ Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 6

17 Ths - Sbsie i back io he likelihood fcio ad obai he likelihood fcio as he fcio of β ad ρ - Maximize his likelihood fcio wih respec o β ad ρ l L σ σ σ = + ( y Xβ)' ψ ( y Xβ) = is he esimaor of σ σ = β ψ β ˆ ( y X )' ( y X ) Sbsiig σ ˆ i place of σ i he log-likelihood fcio yields L = L β ρ = y Xβ ψ y Xβ + ρ = l {( y Xβ) ' ψ ( y Xβ) } l( ρ ) k + l * l *(, ) l ( ) ' ( ) l( ) = k where k = l ( β)' ψ ( β) l y X y X ( ρ ) Maximizaio of l L * is eqivale o miimizig he fcio y Xβ ψ y Xβ ( )' ( ) ( ρ ) Usig opimizaio echiqes of o-liear regressio, his fcio ca be miimized ad esimaes of β ad ρ ca be obaied If is large ad ρ is o oo close o oe, he he erm ( ) / ρ is egligible ad he esimaes of β will be same as obaied by oliear leas sqares esimaio Ecoomerics Chaper 9 Aocorrelaio Shalabh, IIT Kapr 7