Chapter 16 Measurement Error Models

Transcription

1 Chapter 6 Measreet Error Models A fdaetal asspto all the statstcal aalyss s that all the obseratos are correctly easred I the cotet of ltple regresso odel, t s assed that the obseratos o stdy ad eplaatory arables are obsered wthot ay error I ay statos, ths basc asspto s olated There ca be seeral reasos for sch olato For eaple, the arables ay ot be easrable, eg, taste, clatc codtos, tellgece, edcato, ablty etc I sch cases, the dy arables are sed ad the obseratos ca be recorded ters of ales of dy arables Soetes the arables are clearly defed bt t s hard to take correct obseratos For eaple, the age s geerally reported coplete years or ltple of fe Soetes the arable s coceptally well defed bt t s ot possble to take correct obserato o t Istead, the obseratos are obtaed o closely related proy arables, eg, the leel of edcato s easred by the ber of years of schoolg Soetes the arable s well derstood bt t s qaltate s atre For eaple, tellgece s easred by tellgece qotet (IQ) scores I all sch cases, the tre ale of arable ca ot be recorded Istead, t s obsered wth soe error The dfferece betwee the obsered ad tre ales of the arable s called as easreet error or errors-arables Dfferece betwee dstrbaces ad easreet errors: The dstrbaces the lear regresso odel arse de to factors lke predctable eleet of radoess, lack of deterstc relatoshp, easreet error stdy arable etc The dstrbace ter s geerally thoght of as represetg the flece of aros eplaatory arables that hae ot actally bee clded the relato The easreet errors arse de to the se of a perfect easre of tre arables Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr

2 Large ad sall easreet errors If the agtde of easreet errors s sall, the they ca be assed to be erged the dstrbace ter ad they wll ot affect the statstcal fereces ch O the other had, f they are large agtde, the they wll lead to correct ad ald statstcal fereces For eaple, the cotet of lear regresso odel, the ordary least sqares estator (OLSE) s the best lear based estator of regresso coeffcet whe easreet errors are abset Whe the easreet errors are preset the data, the sae OLSE becoes based as well as cosstet estator of regresso coeffcets Coseqeces of easreet errors: We frst descrbe the easreet error odel Let the tre relatoshp betwee correctly obsered stdy ad eplaatory arables be y X where y s a ( ) ector of tre obserato o stdy arable, X s a ( k) atr of tre obseratos o eplaatory arables ad s a ( k ) ector of regresso coeffcets The ale y ad X are ot obserable de to the presece of easreet errors Istead, the ales of addte easreet errors as y y + X X + V y ad X are obsered wth where y s a ( ) ector of obsered ales of stdy arables whch are obsered wth ( ) easreet error ector Slarly, X s a ( k ) atr of obsered ales of eplaatory arables whch are obsered wth ( k) atr V of easreet errors X I sch a case, the sal dstrbace ter ca be assed to be sbsed wthot loss of geeralty Sce or a s to see the pact of easreet errors, so t s ot cosdered separately the preset case Alterately, the sae setp ca be epressed as y X + X X + V where t ca be assed that oly X s easred wth easreet errors V ad ca be cosdered as the sal dstrbace ter the odel Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr

3 I case, soe of the eplaatory arables are easred wthot ay easreet error the the correspodg ales V wll be set to zero We asse that E ( ) 0, E ( ') I EV ( ) 0, EVV ( ' ) Ω, EV ( ' ) 0 The followg set of eqatos descrbes the easreet error odel y X y y + X X + V whch ca be re-epressed as y y + X + ( X V) + X + ( V) X + ω where ω V s called as the coposte dstrbace ter Ths odel reseble lke a sal lear regresso odel A basc asspto lear regresso odel s that the eplaatory arables ad dstrbaces are correlated Let s erfy ths asspto the odel y X + w as follows: { ( )} '{ ω ( ω) } [ '( ) ] EV [ ' ] EVV [ ' ] E X EX E EV V 0 Ω Ω 0 Ths X ad ω are correlated So OLS wll ot prode effcet reslt Sppose we gore the easreet errors ad obta the OLSE Note that gorg the easreet errors the data does ot ea that they are ot preset We ow obsere the propertes of sch a OLSE der the setp of easreet error odel Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 3

4 The OLSE s ( ' ) b X X X ' y ( ' ) '( ) b X X X X + ω ( X ' X) X ' ω ( ) ( ' ) E b E X X X ' ω ( X X) ' X ' E( ω) 0 as X s a rado atr whch s correlated wth ω So b becoes a based estator of Now we check the cosstecy property of OLSE Asse pl X ' X Σ pl VV ' Σ pl XV ' 0 pl V ' 0 The X ' X X ' ω pl( b ) pl X ' X ( X + V) '( X + V) X ' X + XV ' + V' X + VV ' pl X ' X pl X ' X + pl X ' V + pl V ' X + pl V ' V Σ Σ Σ +Σ X ' ω X ' ω+ V ' ω X '( V) + V '( V) Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 4

5 pl X' ω pl X ' pl XV ' + pl V ' pl VV ' Σ X ' X X ' ω pl( b ) pl pl 0 ( ) - Σ +Σ Σ Ths b s a cosstet estator of Sch cosstecy arses essetally de to correlato betwee X ad ω Note: It shold ot be sderstood that the OLSE ( ) ( )'( ) b X ' X X ' y s obtaed by zg S ωω ' y X y X the odel y X + ω I fact ωω ' caot be zed as the case of sal lear regresso, becase the coposte error ω V s tself a fcto of To see the atre of cosstecy, cosder the sple lear regresso odel wth easreet error as y +,,,, 0 y y + + Now 0 0 X, X, V 0 ad assg that pl µ we hae pl ( µ ), Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 5

6 Also, Now Σ pl X ' X pl µ µ + µ Σ pl VV ' pl ( b ) ( ) Σ +Σ Σ b0 0 µ pl b µ + µ + 0 ( ) 0 + µ + µ + µ µ + + Ths we fd that the OLSEs of 0 ad are based ad cosstet So f a arable s sbjected to easreet errors, t ot oly affects ts ow paraeter estate bt also affect other estator of paraeter that are assocated wth those arable whch are easred wthot ay error So the presece of easreet errors ee a sgle arable ot oly akes the OLSE of ts ow paraeter cosstet bt also akes the estates of other regresso coeffcets cosstet whch are easred wthot ay error Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 6

7 Fors of easreet error odel: Based o the asspto abot the tre ales of eplaatory arable, there are three fors of easreet error odel Cosder the odel y 0 +,,,, y y + + Fctoal for: Whe the ' s are kow costats (fed), the the easreet error odel s sad to be ts fctoal for Strctral for: Whe the ' s are detcally ad depedetly dstrbted rado arables, say, wth ea µ ad arace ( 0) >, the easreet error odel s sad to be the strctral for Note that case of fctoal for, 0 3 Ultrastrctral for: Whe the ' s are depedetly dstrbted rado arables wth dfferet eas, say µ ad arace ( 0) >, the the odel s sad to be the ltrastrctral for Ths for s a sythess of fcto ad strctral fors the sese that both the fors are partclar cases of ltrastrctral for Methods for cosstet estato of : The OLSE of whch s the best lear based estator becoes based ad cosstet the presece of easreet errors A portat objecte easreet error odels s how to obta the cosstet estators of regresso coeffcets The stretal arable estato ad ethod of a lkelhood (or ethod of oets) are tlzed to obta the cosstet estates of the paraeters Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 7

8 Istretal arable estato: The stretal arable ethod prodes the cosstet estate of regresso coeffcets lear regresso odel whe the eplaatory arables ad dstrbace ters are correlated Sce easreet error odel, the eplaatory arables ad dstrbace are correlated, so ths ethod helps The stretal arable ethod cossts of fdg a set of arables whch are correlated wth the eplaatory arables the odel bt correlated wth the coposte dstrbaces, at least asyptotcally, to esre cosstecy Let Z, Z,, Z k be the k stretal arables I the cotet of the odel y X + ω, ω V, let Z be the k atr of k stretal arables Z, Z,, Z k, each hag obseratos sch that Z ad X are correlated, atleast asyptotcally ad So we hae Z ad ω are correlated, atleast asyptotcally pl Z' X Σ pl Z ' ω 0 ZX The stretal arable estator of s ge by ( Z X) Z y ( Z' X) Z' ( X ω) ( Z X) Z ˆ ' ' IV + ˆ ' ' ω IV ( ˆ IV ) pl pl Z' X pl Z' ω ΣZX 0 0 So ˆIV s cosstet estator of Ay stret that flflls the reqreet of beg correlated wth the coposte dstrbace ter ad correlated wth eplaatory arables wll reslt a cosstet estate of paraeter Howeer, there ca be aros sets of arables whch satsfy these codtos to becoe stretal arables Dfferet Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 8

9 choces of strets ge dfferet cosstet estators It s dffclt to assert that whch choce of strets wll ge a stretal arable estator hag asyptotc arace Moreoer, t s also dffclt to decde that whch choce of the stretal arable s better ad ore approprate coparso to other A addtoal dffclty s to check whether the chose strets are deed correlated wth the dstrbace ter or ot Choce of stret: We dscss soe poplar choces of strets a arate easreet error odel Cosder the odel y + + ω, ω,,,, 0 A arable that s lkely to satsfy the two reqreets of a stretal arable s the dscrete gropg arable The Wald s, Bartlett s ad Drb s ethods are based o dfferet choces of dscrete gropg arables Wald s ethod Fd the eda of the ge obseratos,,, Now classfy the obseratos by defg a stretal arable Z sch that Z I ths case, f > eda (,,, ) f < eda (,,, ) Z Z Z, X Z Now for two grops of obseratos as follows Oe grop wth those ' s below the eda of,,, Fd the eas of y ' s ad y ad, respectely ths grop ' s, say Aother grop wth those ' s aboe the eda of,,, Fd the eas of y ' s ad ' s, say y ad, respectely ths grop Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 9

10 Now we fd the stretal arable estator der ths set p as follows Let ( Z X) ˆ IV ' ' Z y Z' X 0 ( ) Z Z y y Z' y ( y y ) Zy ˆ y 0IV ˆ 0 ( ) ( y y) IV y y y ( ) 0 y y y y y, y y ˆ y IV y ˆ y y ˆ 0IV y y IV If s odd, the the ddle obseratos ca be deleted Uder farly geeral codtos, the estators are cosstet bt are lkely to hae large saplg arace Ths s the ltato of ths ethod Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 0

11 Bartlett s ethod: Let,,, be the obseratos Rak these obserato ad order the a creasg or decreasg order Now three grops ca be fored, each cotag /3 obseratos Defe the stretal arable as Z f obserato s the top grop 0 f obserato s the ddle grop f obserato s the botto grop Now dscard the obseratos the ddle grop ad copte the eas of y ' s ad ' s - botto grop, say y ad ad - top grop, say y 3 ad 3 Sbstttg the ales of X ad Z ( ) ˆ IV y y 3 3 ˆ y ˆ 0IV IV ˆ IV Z' X Z' y ad o solg, we get These estators are cosstet No coclse edeces are aalable to copare the Bartlett s ethod ad Wald s ethod bt three gropg ethod geerally prodes ore effcet estates tha two gropg ethod s ay cases 3 Drb s ethod Let,,, be the obseratos Arrage these obseratos a ascedg order Defe the stretal arable Z as the rak of The sbstttg the stable ales of Z ad X ( ) ˆ IV Z' X Z' y, we get the stretal arable estators ( ) ˆ IV Z y y ( ) Z Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr

12 Whe there are ore tha oe eplaatory arables, oe ay choose the stret as the rak of that partclar arable Sce the estator ses ore forato, t s beleed to be speror effcecy to other gropg ethods Howeer, othg defte s kow abot the effcecy of ths ethod I geeral, the stretal arable estators ay hae farly large stadard errors coparso to ordary least sqare estators whch s the prce pad for cosstecy Howeer, cosstet estators hae lttle appeal Ma lkelhood estato strctral for Cosder the a lkelhood estato of paraeters the sple easreet error odel ge by y 0 +,,,, y y + + Here (, y ) are obserable ad (, ) Asse ( ) 0, E( j) E ( ) 0, E( j) E ( j) f j 0 f j, f j 0 f j, y are obserable E V 0 for all,,, ; j,,, For the applcato of ethod of a lkelhood, we asse the oral dstrbto for ad We cosder the estato of paraeters the strctral for of the odel whch So asse ( ) ~ N µ, ' s are stochastc ad ' s are depedet of ad Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr

13 Ths E ( ) µ Var( ) E ( ) µ ( ) [ ( ) ] E[ + µ ] Var E E ( ) + E( ) E y ( µ ) ( ) ( µ ) 0 + µ 0 E + E + ( ) [ ( ) ] E[ + + µ ] Var y E y E y 0 0 ( ) ( ) ( ) E µ + E E µ + So (, ) { ( ) }{ ( ) } E { + µ }{ + + µ } Co y E E y E y 0 0 ( ) ( ) ( ) ( ) E µ + E µ + E µ + E y 0 + µ + ~ N, µ + The lkelhood fcto s the jot probablty desty fcto of ad,,,, as Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 3

14 (,,,,,, ) L f / ep ep π / ( ) ( y 0 ) ep ep π The log-lkelhood s ( ) ( y ) L* l L costat ( l + l ) 0 The oral eqatos are obtaed by eqatg the partal dfferetatos eqals to zero as L * () ( y 0 ) 0 0 L * () ( y 0 ) 0 L * (3) ( ) + ( y ) 0,,,, (4) (5) 0 L* + 4 L* + 4 ( ) ( y ) 0 These are ( + 4) eqatos ( 4) eqato (4), we get whch s desrable + paraeters bt sg eqato (3) oer,,, ad sg These eqatos ca be sed to estate the two eas ( µ ad µ ) coarace The s paraeters 0 relatos dered fro these oral eqatos +, two araces ad oe 0 µ,,,, ad ca be estated fro the followg fe strctral Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 4

15 ( ) ( ) ( ) ( ) ( ) µ y + µ yy y ( ) where, y y,, ( y y) ad ( )( y y) yy y These eqatos ca be dered drectly sg the sffcecy property of the paraeters barate oral dstrbto sg the defto of strctral relatoshp as E ( ) µ E( y) + µ 0 Var( ) + Var( y) + Co y (, ) We obsere that there are s paraeters,, µ,, ad to be estated based o fe strctral 0 eqatos ()-() So o qe solto ests Oly µ ca be qely detered whle reag paraeters ca ot be qely detered So oly µ s detfable ad reag paraeters are detfable Ths s called as the proble of detfcato Oe relato s short to obta a qe solto, so addtoal a pror restrctos relatg ay of the s paraeters s reqred Note: The sae eqatos ()-() ca also be dered sg the ethod of oets The strctral eqatos are dered by eqatg the saple ad poplato oets The asspto of oral dstrbto for, ad s ot eeded case of ethod of oets Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 5

16 Addtoal forato for the cosstet estato of paraeters: The paraeters the odel ca be cosstetly estated oly whe soe addtoal forato abot the odel s aalable Fro eqatos () ad (), we hae ˆ µ ad so µ s clearly estated Frther ˆ y ˆ 0 s estated f ˆ s qely detered So we cosder the estato of,, ad oly Soe addtoal forato s reqred for the qe deterato of these paraeters We cosder ow aros type of addtoal forato whch are sed for estatg the paraeters qely s kow: Sppose s kow a pror Now the reag paraeters ca be estated as follows: + ˆ y ˆ y + ˆ ˆ ˆ yy yy ˆ y yy Note that ca be egate becase that ˆ > 0 ad redefe ˆ y ; > s kow ad s based po saple So we asse Slarly, ˆ s also assed to be poste der stable codto All the estators ˆ ˆ ˆ, ad are the cosstet estators of, ad respectely Note that ˆ looks lke as f the drect regresso estator of has bee adjsted by for ts cosstecy So t s tered as adjsted estator also Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 6

17 s kow Sppose s kow a pror The sg y we ca rewrte, yy + + y ˆ yy ; yy > y ˆ ˆ y ˆ ˆ The estators ˆ ˆ, ad ˆ are the cosstet estators of, ad respectely Note that ˆ looks lke as f the reerse regresso estator of s adjsted by for ts cosstecy So t s tered as adjsted estator also 3 λ s kow Sppose the rato of the easreet error araces s kow, so let s kow λ Cosder yy + + λ y y ( ) + λ y y + λ (sg ()) (sg ()) (sg ) ( ) 0 ( 0) ( ) 0 + λ y y yy + λ λ y yy y Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 7

18 Solg ths qadratc eqato Sce ˆ y ˆ 0 y ˆ U y ( ) ( ) λ ± λ + 4λ 0 0 yy yy y U, say y ad sce y 0, so U st be oegate y Ths ples that the poste sg U has to be cosdered ad so ( ) ( ) λ + λ + 4λ yy yy y ˆ Other estates are ˆ ˆ + ˆ s λ+ ˆ ˆ y ˆ yy y y Note that the sae estator ˆ of ca be obtaed by orthogoal regresso Ths aots to trasfor by / ad y by y / ad se the orthogoal regresso estato wth trasfored arables 4 Relablty rato s kow The relablty rato assocated wth eplaatory arable s defed as the rato of araces of tre ad obsered ales of eplaatory arables, so K Var( ) ;0 K Var( ) + s the relablty rato Note that K, whe eplaatory arable ad K 0, eas 0 whch eas that there s o easreet error the 0 whch eas eplaatory arable s fed Hgher ale Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 8

19 of K s obtaed whe s sall, e, the pact of easreet errors s sall The relablty rato s a poplar easre psychoetrcs Let K be kow a pror The y + y K + ˆ y K y ˆ Note that K + ˆ ( K ) ˆ K b where b s the ordary least sqares estator b y 5 0 s kow Sppose 0 s kow a pror ad E ( ) 0 The y + µ ˆ 0 y 0 ˆ µ y 0 y ˆ ˆ ˆ ˆ yy y ˆ y ˆ Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 9

20 6 Both ad are kow Ths case leads to oer-detfcato the sese that the ber of paraeters to be estated are saller tha the ber of strctral relatoshps bdg the So o qe soltos are obtaed ths case Note: I each of the cases -6, ote that the for of the estate depeds o the type of aalable forato whch s eeded for the cosstet estator of the paraeters Sch forato ca be aalable fro aros sorces, eg, log assocato of the epereter wth the eperet, slar type of stdes codcted the part, soe etraeos sorce etc Estato of paraeters fcto for : I the fctoal for of the easreet error odel, ' s are assed to be fed Ths asspto s realstc the sese that whe ' s are obserable ad kow, t s dffclt to kow f they are fed or ot Ths ca ot be esred ee repeated saplg that the sae ale s repeated All that ca be sad ths case s that the forato ths case s codtoal po codtoally kow So the odel s y y y + the y ~ N, 0 The lkelhood fcto s ( y 0 ) ( ) L ep ep π π The log-lkelhood s ( y ) L* l L costat l l 0 ( ) ' s So asse that ' s are Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr 0

21 The oral eqatos are obtaed by partally dfferetatg L * ad eqatg to zero as L * ( I) 0 ( y 0 ) 0 0 L * ( II) 0 ( y 0 ) 0 L* ( III) 0 + ( y 0 ) L* ( IV ) 0 + ( ) 0 L* ( V ) 0 y + ( ) 0 ( ) 0 Sqarg ad sg eqato (V), we get ( y 0 ) ( ) or ( y 0 ) 4 ( ) Usg the left had sde of eqato (III) ad rght had sde of eqato (IV), we get whch s acceptable becase ca be egate also I the preset case, as > 0 ad > 0, so wll always be poste Ths the a lkelhood breaks dow becase of sffcet forato the odel Icreasg the saple sze does ot sole the prpose If the restrctos lke kow, kow or kow are corporated, the the a lkelhood estato s slar to as the case of strctral for ad the slar estates ay be obtaed For eaple, f λ / s kow, the sbsttte t the lkelhood fcto ad aze t The sae solto as the case of strctral for are obtaed Ecooetrcs Chapter 6 Measreet Error Models Shalabh, IIT Kapr