Semiparameric ad Noparameric Mehods i Poliical Sciece Lecure : Semiparameric Esimaio Michael Peress, Uiversiy of Rocheser ad Yale Uiversiy
Lecure : Semiparameric Mehods Page 2 Overview of Semi ad Noparameric Models Parameric Model: saisical model characerized by fiie dimesioal ukow parameer 2 y ~ N ( µ, σ ) 2 y ~ N( β ' x, σ ) (ormal-liear model) Noparameric Model: saisical model characerized by ifiie dimesioal ukow parameer y ~ f where f is ukow (desiy esimaio) y = g( x ) + ε, E[ ε x ] = 0 (oparameric regressio)
Lecure : Semiparameric Mehods Page 3 Overview of Semi ad Noparameric Models Semiparameric Model: saisical model characerized by fiie dimesioal parameer of ieres ad ifiie dimesioal uisace parameer y = β ' x + ε, ε x ~ F( ε x) wih E[ ε x] = 0 (semiparameric liear model) β is parameer of ieres F is a uisace parameer y = β ' x + ε, ( x, ε ) is saioary ad ergodic ad E[ ε x ] = 0 β is parameer of ieres Sochasic process characerizig ( x, ε ) is a uisace parameer
Lecure : Semiparameric Mehods Page 4 Overview of Semi ad Noparameric Models More Semiparameric/Noparameric Models: y = g( β ' x) + ε, ε x ~ F( ε x) wih E[ ε x] = 0 (liear idex model) y = g( x) + β ' z + ε, ε x ~ F( ε x) wih E[ ε x] = 0 (parially liear model) Pr( y = ) = G( β ' x ) (semiparameric biary choice)
Lecure : Semiparameric Mehods Page 5 Overview of Semi ad Noparameric Models Parameric Models: MLE is efficie if parameric model is correc MLE is ofe icosise if parameric model is icorrec N -covergece rae Noparameric Models: More geeraliy, bu heory more difficul Implemeaio difficul Slower covergece (slower ha parameric rae of N ) Efficiecy loss (relaive o MLE if parameric model is corr.)
Lecure : Semiparameric Mehods Page 6 Overview of Semi ad Noparameric Models Semiparameric Models: More geeraliy, ad Ofe, N -covergece for parameer of ieres Ofe, easy o impleme Ofe, lile efficiecy loss heory ca be very hard, bu some impora cases are sufficiely worked ou so ha we do have o worry abou i Lecure will focus o easy bu powerful semiparameric esimaors Lecure 2 will focus o basics of oparameric esimaio Lecure 3 will focus o applicaios of oparameric esimaors ad more advaced semiparameric esimaors
Lecure : Semiparameric Mehods Page 7 Overview of Semi ad Noparameric Models Examples of Easy Semiparameric Esimaors: OLS w/ robus se s - semiparameric because OLS is cosise eve if error erms are o-ormal ad heeroskedasic Poisso regressio w/ robus se s - semiparameric because esimaor is cosise whe depede variable is o Poisso disribued Liear-oliear models w/ Newey-Wes se s semiparameric because OLS/MLE are cosise eve whe depede variable exhibis ime series depedece Shor paels wih clusered sadard errors semiparameric because OLS/MLE are cosise eve whe depede variable exhibis group correlaio/ime series depedece
Lecure : Semiparameric Mehods Page 8 Overview of Semi ad Noparameric Models Why Use hese Semiparameric Esimaors? Easy o apply: Some parameric aleraives are VERY compuaioally iesive Skip he specificaio sep (which is someimes ear impossible) Modelig heeroskedasiciy Selecig ARMA srucure (w/ ime series ad pael daa) Selecig bewee egaive biomial, zero-iflaed, zerorucaed, ec., i cou models
Lecure : Semiparameric Mehods Page 9 Overview of Semi ad Noparameric Models Drawbacks: Efficiecy loss relaive o parameric model However: Parameric model may be wrog! Usually, semiparameric esimaors achieve semiparameric efficiecy bouds (hey are efficie uder maiaied assumpios) Ofe, o much efficiecy loss Ofe, hese semiparameric esimaors give robusess pracically for free sice we do have o esimae he uisace parameers
Lecure : Semiparameric Mehods Page 0 Heeroskedasiciy i he Liear Model Parameric Liear Model:. y = β ' 0 x + ε (lieariy) 2. ( x, ε ) are idepede (idepedece) 3. Exx [ '] has full rak (ideificaio) 4. E[ ε x ] = 0 5. 2 ε ~ N(0, σ ) (homoskedasiciy ad ormaliy) Uder -5, OLS is MLE; OLS is ubiased, ormally disribued, cosise, ad asympoically ormal; he iformaio equaliy holds; ad OLS is efficie
Lecure : Semiparameric Mehods Page Heeroskedasiciy i he Liear Model Semiparameric Liear Model:. y = β ' 0 x + ε (lieariy) 2. ( x, ε ) are idepede (idepedece) 3. Exx [ '] has full rak (ideificaio) 4. E[ ε x ] = 0 5. 2 ε ~ N(0, σ ) (homoskedasiciy ad ormaliy) Cosider OLS as semiparameric esimaor Uder -4, OLS is ubiased, ormally disribued, cosise, ad asympoically ormal; he iformaio equaliy holds; ad OLS is efficie
Lecure : Semiparameric Mehods Page 2 Heeroskedasiciy i he Liear Model Properies of OLS as semiparameric esimaor: N N N N N xx N x y 0 N xx N x ˆ β = ' = β + ' ε = = = ( β0 ' x+ ε) = = By law of ieraed expecaios, OLS is ubiased: Ex [ ε ] = Ex [ E[ ε x]] = 0 = 0 N N [ ˆ ] 0 N ' N [ ] = = = 0 E β x = β + x x E x ε = β0
Lecure : Semiparameric Mehods Page 3 Heeroskedasiciy i he Liear Model OLS is cosise: N N = x ε E[ x ε ] = 0 prob. = 0 N N N prob. 0 N xx N x 0 N xx = = = prob. 0 ˆ β = β + ' ε β + ' 0= β Normaliy/homoskedasiciy o eeded for hese resuls 0
Lecure : Semiparameric Mehods Page 4 Heeroskedasiciy i he Liear Model OLS is asympoically ormal: N N = 0 N N = = N( ˆ β β ) x x ' x ε LLN CL [ '] N(0, Var( x )) Exx N ˆ N E x x Var x E x x dis. ( β β0) (0, [ '] ( ε) [ '] ) bread mea bread Esimae asympoic disribuio usig: N [ '] N xx ' = E xx N 2 2 ( ε) = [ ε '] N ε ' = Var x E x x x x ε
Lecure : Semiparameric Mehods Page 5 Heeroskedasiciy i he Liear Model Implemeaio: I saa, regress y x x2, robus I r, use sadwich package: lm <- lm(y ~ X + X2) sw <- sadwich(lm)
Lecure : Semiparameric Mehods Page 6 Heeroskedasiciy i he Liear Model Overview: Apply OLS whe homoskedasiciy/ormaliy do o hold Beefi: robusess Drawback: less efficiecy
Lecure : Semiparameric Mehods Page 7 Heeroskedasiciy i he Liear Model Example: OLS is MLE whe errors are ormal ad homoskedasic LAD is MLE whe errors are double expoeial ad homoskedasic OLS will be more efficie ha LAD whe errors are ormal ad homoskedasic, robus se s will be correc for boh esimaors LAD will be more efficie ha OLS whe errors are double expoeial ad homoskedasic, robus se s will be correc for boh esimaors
Lecure : Semiparameric Mehods Page 8 Heeroskedasiciy i he Liear Model Example Coiued: Geerae 000 Moe Carlo daa ses wih N=500, X~N(0,), X2~N(0,), Bea=(-.5,.5,-.0), ad errors eiher N(0,) or DExp(0,) DGP = Normal Liear DGP = DExp Liear Bea Bea2 Bea3 Bea Bea2 Bea3 Rel. Eff. OLS/LAD 0.8 0.80 0.82.3.37.28 OLS Overcofidece.06.04.05 0.97.02.02 LAD Overcofidece.04.04.02 0.98 0.98.04 OLS se / LAD se 0.89 0.89 0.89.5.5.5
Lecure : Semiparameric Mehods Page 9 Noproporioal Dispersio i Cou Models Parameric Poisso Model: 0. y ~ Poisso( λ ), ' x λ = e β 2. ( y, x ) are iid Noice ha E y x Var y x e β 0 [ ] ( ) ' x = = λ = We ca derive he log-likelihood fucio: N β ' x (, ; β) = β' log! = l y x y x e y
Lecure : Semiparameric Mehods Page 20 Noproporioal Dispersio i Cou Models Semiparameric Poisso Model: 0. y ~ Poisso( λ ), ' x λ = e β 0 [ ] ' x E y x = e β 2. ( y, x ) are iid Semiparameric esimaor defied by: N ˆ β ' x = arg max N yβ' x e log y! β = β
Lecure : Semiparameric Mehods Page 2 Noproporioal Dispersio i Cou Models Cosisecy of semiparameric Poisso regressio: N ˆ β ' x = arg max N yβ' x e log y! β = β Firs order codiio: I large samples: N ˆ ' x 0 = N xk, ( y e β ) = ˆ ˆ ˆ β' ' 0 ' ' 0 [ ( x β )] [ [( x β ) ]] [ ( x β = E x y e = E x E y e x = E x e e x ) x ]] k, k, k,. Hece, ˆ prob β β0 as log as codiioal mea is correcly specified (eve if Poisso assumpio does o hold)
Lecure : Semiparameric Mehods Page 22 Noproporioal Dispersio i Cou Models Wha abou sadard errors? ' Defie ψ ( y, x ; β) = y β' x e β x log y! aylor expasio argume: ˆ dis. N( β β ) N(0, Q VQ ) where, Q= E[ ψ ( y, x ; β )], ββ If MLE assumpios hold, Q 0 0 = V V = Var( ψ ( y, x ; β )) If o, mus use sadwich esimaor, i.e. robus se s β 0
Lecure : Semiparameric Mehods Page 23 Noproporioal Dispersio i Cou Models Implemeaio: I saa: poisso y x x2, robus I r, use sadwich package: pm <- glm(y ~ X + X2,family= poisso ) sw <- sadwich(pm)
Lecure : Semiparameric Mehods Page 24 Noproporioal Dispersio i Cou Models Wha we ge: Robusess o overdispersio Robusess o zero-iflaio, zero-rucaio, oe iflaio, ec. All we eed is correcly specified codiioal mea Wha we do ge Efficiecy (MLE is more efficie if parameric model is correc) Prediced values easily geeraed, bu o prediced disribuio (sice disribuio is o Poisso) If we wa prediced values, we ca use procedures discussed i lecure 3
Lecure : Semiparameric Mehods Page 25 Noproporioal Dispersio i Cou Models Same pricipal exeds o egaive biomial model Same pricipal exeds o oher expoeial family models (i.e. cosisecy holds as log as codiioal mea is correcly specified), robus does o provide ay beefi for logi, probi, ordered logi, muliomial logi, ec.: hese models are oly correc if parameric model is correc If parameric model is correc, se s = robus se s i large samples
Lecure : Semiparameric Mehods Page 26 ime Series Depedece i he Liear Model Semiparameric ime Series Liear Model:. y = β ' 0 x + ε (lieariy) 2. ( x, ε ) are idepede ( x, ε ) are saioary ad ergodic 3. Exx [ '] has full rak (ideificaio) 4. E[ ε x ] = 0 5. 2 ε ~ N(0, σ ) (homoskedasiciy ad ormaliy) Cosider OLS as semiparameric esimaor Uder -4, OLS is ubiased, ormally disribued, cosise, ad asympoically ormal; he iformaio equaliy holds; OLS is efficie
Lecure : Semiparameric Mehods Page 27 ime Series Depedece i he Liear Model OLS esimaor: OLS is ubiased: = + 0 xx ' x = = ˆ β β ε = = = 0 ˆ E[ β x] = β0 + x x ' E[ xε ] = 0 Uder saioariy ad ergodiciy, Hece, OLS is cosise = xε E[ xε ] = 0 prob.
Lecure : Semiparameric Mehods Page 28 ime Series Depedece i he Liear Model Large sample disribuio: = 0 = = ( ˆ β β ) x x ' xε LLN CL Exx [ '] N 0,limVar x Q = = = V ε ˆ dis. ( β β0) N( Q VQ )
Lecure : Semiparameric Mehods Page 29 ime Series Depedece i he Liear Model ricky par is esimaig V = limvar xε = If x ε are idepede, he 2 2 Var [ '] ' x ε = E ε x x ε x x = = = If x ε o idepede, he covariace erms make i hard 2 Var ( ) (, ) x ε = Var x ε + Cov x ε xsε s = = s< Newey-Wes (ad relaed procedures) provide a way o esimae limvar x ε =
Lecure : Semiparameric Mehods Page 30 ime Series Depedece i he Liear Model Defie, ad esimae, ˆ( s) = s sxx s' = s+ γ ε ε Selec m such ha m Vˆ = ˆ γ(0) + ( )[ ˆ γ( s) + ˆ γ( s)'] s= s m + m as Auomaic procedures for choosig m efficiely are available Rule of humb is ( ) 2/9 = 4 00 m Newey-Wes is special case of specral desiy approach o covariace marix esimaio (w/ a Barle Kerel)
Lecure : Semiparameric Mehods Page 3 ime Series Depedece i he Liear Model Wha we ge: Robusess o heeroskedasiciy ad auocorrelaio No eed o selec appropriae ARMA model (here is some badwidh selecio goig o i he backgroud, bu his par is largely auomaed) Wha we lose: Efficiecy: If correc ARMA model is seleced, he MLE will be more efficie
Lecure : Semiparameric Mehods Page 32 ime Series Depedece i Noliear Models Newey-Wes sadard errors ca be used o correc for ime series depedece i early ay oliear model For may oliear models, icorporaig ime series depedece is exremely difficul Parameric ime-series versios of sadard esimaors cao be esimaed i mos (or eve all?) saisical packages ime series logi, ime series probi, ime series cou, ec.
Lecure : Semiparameric Mehods Page 33 ime Series Depedece i Noliear Models Biomial Probi wih AR errors (parameric model). y * = β ' x + ε, y = { y * 0} (probi model) 2. ε = ρε + u, u ~ N (0,), u are iid (AR errors) MLE ivolves complicaed -dimesioal iegral Defie Ay (,..., y) = { x: x [0, ) ify =, x (,0] ify = 0} µ ( x; β) = β' x, s ρ Ω s, ( ρ) = ρ Pr( y,..., y ; β, ρ) = φ( ϑ ; µ ( x ; β), Ω( ρ)) d ϑ ϑ A( y,..., y ) Really hard o compue! (saa/r do do i righ ow)
Lecure : Semiparameric Mehods Page 34 ime Series Depedece i Noliear Models Aleraive approach: semiparameric esimaio Claim (Poirier ad Ruud, 986): Probi is sill cosise whe observaios are depede Why? MLE is ˆ θ = arg max log f( y ; θ ) θ = MSE is cosise because log f( y; θ ) E[log f( y; θ )] Iformaio iequaliy implies E[log f( y ; θ )] is miimized a θ 0, he rue parameer value Iformaio iequaliy will coiue o hold for all models ha have he same margials =
Lecure : Semiparameric Mehods Page 35 ime Series Depedece i Noliear Models Hece, we ca apply Newey-Wes sadard errors o probi o obai cosise esimaes wih correcs sadard errors Same resul holds for oher models: Parameric Poisso models w/ ime series depedece are difficul o obai I Poisso case, usig Newey Wes sadard errors give esimaor ha is robus o over/uder dispersio, zero-iflaio, ad ime series depedece
Lecure : Semiparameric Mehods Page 36 ime Series Depedece i Noliear Models Implemeaio (liear model): I saa, ewey y x x2, lag(#) I r, sadwich package: lm <- lm(y ~ X + X2) sw <- NeweyWes(lm) Implemeaio (oliear models): I saa, usig wes package I r, sadwich package: glm <- glm(y ~ X + X2,family= poisso ) sw <- NeweyWes (glm)
Lecure : Semiparameric Mehods Page 37 ime/group Depedece i Pael Daa Semiparameric Liear Pael Daa Model (shor paels wih may idividuals): y = β ' x + ε (liear model).,,, E[ ε x ] = 0 2.,, E[ x x '] has full rak (ideificaio) 3.,, ( ε,..., ε ) are idepede over (idepedece) 4.,, OLS esimaor: N N 0 N x, x, ' N x,, = = = = ˆ β = β + ε N N N N 0 N x, x, ' N x,, 0 N z N = = = = = = = z = ω = β + ε = β + ω
Lecure : Semiparameric Mehods Page 38 ime/group Depedece i Pael Daa OLS is ubiased: E[ ˆ β x] = β0 OLS is cosise sice, N N = ω prob. If x, are idepede over ad, he sadwich esimaor provides correc sadard errors Oherwise, N N N 2 Var,, (,, ) (,,,,, ) N N N s s x ε = Co Var x ε + v x ε x ε = = = = = s< 0
Lecure : Semiparameric Mehods Page 39 ime/group Depedece i Pael Daa Aleraively, N N = 0 N z N = = N( ˆ β β ) ω (0, [ ] ( ω) [ ] ) [ ] (0, ( ω)) N Ez Var Ez Ez N Var N N Var( ω ) N ωω' = N x, ε, x, ε, ' = = = = Noice ha his will o work wih small-log paels sice LLN i N will o kick i As log as ω are idepede, variace esimaor is accurae (does o require ay assumpio abou ime-series depedece) Cluserig will o corol for a commo ime effec
Lecure : Semiparameric Mehods Page 40 ime/group Depedece i Pael Daa Same pricipal holds if wo-way srucure is o idividuals/ime, bu idividuals/groups (e.g. couries, saes) G Ig G Ig 0 G I x,, ' g gix gi G I x g gi, gi, g= i= g= i= ˆ β = β + ε G Ig G Ig 0 G Ig gi, gi, G Ig gi, gi, g= i= g= i= ( ˆ G β β ) = x x ' x ε
Lecure : Semiparameric Mehods Page 4 ime/group Depedece i Pael Daa Suppose here is a group effec, ε gi, = ug+ ξgi,, u g ad ξ gi, are iid ad idepede of each oher (i.e. idividuals i differe couries, u g represeig a coury effec, possibly due o omied coury variables) If group (e.g. coury) fixed effecs are excluded, mus cluser If group fixed effecs are icluded, o eed o cluser If coury fixed effecs are omied, he cluserig deals wih wihi coury correlaio (as log as x gi, ad u g are o depede, i which case OLS w/ou fixed effecs is icosise)
Lecure : Semiparameric Mehods Page 42 ime/group Depedece i Pael Daa Implemeaio: I saa, regress y x x2, cluser(id)
Lecure : Semiparameric Mehods Page 43 Example: Mohly error Aacks i Israel Number of Israelis Killed: 40 20 00 80 60 40 20 0 00 0 02 03 04 05 06 07 08 09 KILLED
Lecure : Semiparameric Mehods Page 44 Example: Mohly error Aacks i Israel Number of Israelis Killed: 70 60 50 40 30 20 0 0 0 20 40 60 80 00 20 Series: KILLED Sample 2000M0 200M03 Observaios 23 Mea 0.0083 Media 5.000000 Maximum 30.0000 Miimum 0.000000 Sd. Dev. 5.73265 Skewess 4.6009 Kurosis 29.29346 Jarque-Bera 3897.928 Probabiliy 0.000000
Lecure : Semiparameric Mehods Page 45 Example: Mohly error Aacks i Israel Corol for: Elecio period (3 mohs leadig up o Israeli elecio) Pos peace summi (6 mohs followig peace summi) Righ-wig Israeli prime-miiser
Lecure : Semiparameric Mehods Page 46 Example: Mohly error Aacks i Israel Liear Model i saa: Calculae by had, m = 4 Naïve sadard errors, regress killed killed_m elec possummi righpm Robus sadard errors, regress killed killed_m elec possummi righpm, robus Newey-Wes sadard errors ewey killed killed_m elec possummi righpm, lag(4)
Lecure : Semiparameric Mehods Page 47 Example: Mohly error Aacks i Israel Poisso Model i r: m calculaed auomaically mod <- glm(killed ~ killed_m + elec + possummi + righpm,family="poisso",daa=xls) coef <- summary(mod)$coefficies[:5,] se <- summary(mod)$coefficies[:5,2] se2 <- sqr(diag(sadwich(mod))) se3 <- sqr(diag(neweywes(mod)))