ON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS 1

Transcription

1 Teory of Stocastic Processes Vol2 28, o3-4, 2006, pp*-* SILVELYN ZWANZIG ON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS Local liear metods are applied to a oparametric regressio model wit ormal errors i te variables ad uiform distributio of te variables Te local eigborood is determied wit elp of decovolutio kerels Two differet liear estimatio metod are used: te aive estimator ad te total least squares estimator Bot local liear estimators are cosistet But oly te local aive estimator delivers a estimatio of te taget Itroductio Errors-i-variables models are essetially more complicated as ordiary regressio models Te desig poits are observed wit errors oly, suc tat te models iclude a icreasig umber of uisace parameters Neverteless it is wated to estimate a regressio fuctio belogig to some smootess class Fa ad Truog 993 applied te decovolutio tecique of desity estimatio to oparametric regressio wit errors i variables Te mai idea was to use kerel estimators wit decovolutio kerels I ordiary regressio a metod of bias reducig is te local polyomial regressio, see [] Local liear regressio as two mai aspects - Aroud te wated x a local eigborood is defied - Te regressio fuctio is approximated by its taget at x Tis taget is estimated by usual metods wit observatios comig from te eigborood Te defiitio of te eigborood of x is ot trivial i errors-i-variables models, because observatios of desig poits ear te wated x ca come from a desig poit lyig far away Ivited lecture 2000 Matematics Subject Classificatios: 62G08, 62G05, 62G20 Key words ad prases: error i variable, measuremet errors, oparametric regressio, decovolutio, kerel metods, local liear regressio, aive estimatio, total least squares

2 2 SILVELYN ZWANZIG J Staudemayer ad D Ruppert 2004 applied te local polyomial approac to errors-i-variables models Tey derived asymptotic results for decreasig error variaces Te local eigborood is described by weigts basig o ordiary kerels at te observed variables Te taget or te best polyomial was estimated by a weigted aive estimator I tis paper we are iterested i cosistet local liear regressio, were te cosistecy is based o icreasig sample size Ufortuately te best possible rates are slow For ormal errors of te variables te best possible rate is O P log I tis paper we describe te local eviromet by decovolutio kerels Tat is a adjustmet of te errors i variables As estimatio metods for te taget two differet estimators from te teory of te liear errors-ivariables model are take: a weigted liear aive estimator ad a weigted total least squares estimator Te uweigted aive estimator as a bias i usual liear errors-i-variables models Te uweigted total least squares estimator is cosistet ad is te maximum likeliood estimator i liear errors-i-variables models wit ormal error distributios Te mai result is tat bot procedures deliver cosistet estimators, but oly te weigted aive estimator is a estimator of te taget Te weigted total least squares estimator estimates sometig else Te limit value of te slope is derived A euristic iterpretatio of tis effect may be, tat te weigts basig o te decovolutio kerels ad te total least squares estimatio priciple are two idepedet adjustmets for te same tig 2 Te model ad metods Let te observatios x, y, x i, y i, x, y be idepedetly distributed, geerated by y i = g ξ i + ε i x i = ξ i + δ i 2 Te error variables ε i, δ i, i =,, are mutually idepedet wit expectatio zero ad bouded fourt momets, wit V ar ε i = σ 2 ad V ar ε 2 i µ 4 ε Te errors i te errors-i-variables equatio 2 are ormal distributed δ i N0, σ 2 iid, σ 2 > 0 3 Te uobserved desig poits come from a uiform distributio ξ i [0, ], iid U[0, ] 4 ad ξ i, ε i, δ i are mutually idepedet fuctio g C[0,] 2 wit We assume a smoot regressio

3 LOCAL LINEAR ESTIMATION 3 g 2 x g 2 x + δ Lδ 5 Te aim is to estimate gx for a give x 0, Furter we itroduce kerel fuctios K wit Kudu =, Ku = K u, Ku 2 du = µ K < 6 ad wit compact supported Fourier trasform Φ K t = expitukudu, Φ K t = 0 for t < a, t > b 7 Furtermore we assume tat te secod derivative of te kerel exists ad tat for some costats µ K, µ K, c K, c K K u 2 du = µ K < ; K u 2 du = µ K <, 8 u 4 Ku c K, u 4 K u c K 9 Examples for kerels fulfillig tese coditios 6-9 are K 2 u = 48 cosu 5 44 siu 2 5 ad K πu 4 u 2 πu 5 u 2 3 u = 3 8π si u 4 4 Note u 4 te sic kerel K u = siu does ot fulfill te coditio 9 π u We set te badwidt = Clog, C sufficietly large Tat is te optimal badwidt i te model - 3, see [2] I ordiary local liear regressio commo local weigts are w i x = K ξ i x K 0 ξ i x I errors-i-variables models te desig poits ξ i are ukow, tat is wy we ave cose wi x = K xi x K xi, x were K is te decovolutio kerel of K defied by K u = exp itu + t2 σ2 Φ 2 2 K t dt 2 Note, tat te domiators i 0 ad are positive wit icreasig probability, because tey are cosistet desity estimators, compare also 22

4 4 SILVELYN ZWANZIG For weigted meas ad weigted quadratic forms wit weigts 0 we use a aalog otatio as i [3]: x w, ξ w, m ξξ, m yx,, m gg If te weigts are used, te we write x w, ξ w, m ξξ, m yx,, m gg Tus x w = w i xx i, x w = w i xx i ad m xy = w i xx i x w y i y w ad m xy = w i xx i x wy i y w ad so o Deote te local liear approximatio of gξ by tξ = β 0 + β ξ x Te te local aive estimator is defied as ĝ aive x = y w m xy x m w x 3 xx Note tat ĝ aive x = t aive 0 = β 0,aive, were β 0,aive, β,aive T = arg mi β 0,β wi x y i tx i 2 Te local total least squares estimator is defied for m xy 0 as ĝ tls x = y w β,tls x w x 4 wit β,tls = m yy m xx + m yy m xx m xy 5 2m xy Note tat ĝ tls x = t tls 0 = β 0,tls, were β 0,tls, β,tls T = arg mi β 0,β w i x mi ξ [ yi tξ 2 + x i ξ 2] 3 Mai result I tis sectio te mai teorems are proved Te proofs are based o auxiliary results sow i Sectio 4 Te followig teorem states te cosistece of te local aive estimator Furter it is sow tat te aive estimator is really a estimator of te taget Teorem I te model - 5 ad for kerels wit 6-9 it olds β,aive = g x + O P log 2 ĝ aive x = gx + O P log Proof

5 Usig 37 ad 38 we get β,aive = m xy m xx LOCAL LINEAR ESTIMATION 5 = σ2 g x σ 2 + O p = g x + O p 2 Remember 3 Applyig 35, 36 ad tat β,aive is stocastically bouded, we obtai te statemet Te followig teorem delivers te results for te local total least squares estimator Te estimator is cosistet but does ot yield a cosistet of te taget Teorem I te model - 3 ad for kerels wit 6-9 it olds for g x 0 β,tls = + + g g x x 2 + O P log 2 ĝ tls x = gx + O P log Proof Itroduce F x for m xy > 0 F x = udefied for m xy = 0 F 2 x for m xy < 0, 6 were F x = x x2 + 4 ad F 2 x = x 2 x2 + 4 Bot fuctios are icreasig F is covex ad F 2 is cocave Te first derivatives are bouded by for all x It olds m β,tls = F yy m xx m xy Cosider z = m yy m xx From 37, 38 ad 39 it follows for g x m xy 0 tat z = z + O p, were z = 2 Deote g x β lim = + + g g x x 2 It olds for g x < 0 tat F z = β lim ad for g x > 0 tat F 2 z = β lim Suppose g x > 0, te from 37 follows tat lim P m xy > 0 = 0 We ave for T > 0 P β,tls β lim > T = P β,tls β lim > T /m xy < 0 +o

6 6 SILVELYN ZWANZIG Furter P F 2 z β lim > T = P F 2 z F 2 z > T = o for T Assume g x < 0, te lim P m xy < 0 = 0 For T > 0 P β,tls β lim > T = P F z F z > T + o Hece lim lim sup P β,tls β lim > T = 0 T 2 Remember 4 Applyig 35, 36 ad tat β,tls is stocastically bouded, we obtai te statemet 3 Auxiliary results I [2] it is sow tat E x K x a ξ a = K Furtermore we ave te followig itegral equatios Lemma Uder x = ξ + δ, δ N0, σ 2 ad for kerels wit 6-9 it olds for all a tat te expectatio wit respect to δ is 2 E x/ξ K x a σ2 x ξ = K ξ a 7 E x/ξ K x a x ξ2 = σ4 K 2 ξ a + σ2 K ξ a 8 Proof Note tat Usig exp it x a K u = it exp itu Φ K t dt 9 = exp it ξ a x ξ exp it ad 2 we get E x/ξ K x a x ξ = exp it ξ a Φ K t Φ t I 2 t dt δ were Φ δ t = exp σ2 t 2 ad I 2 t = u exp i t u ϕ δ u dx wit ϕ δ u = exp u2 Usig te quadratic decompositio we get 2 exp i t u ϕ δ u = Φ δ t ϕ it σ 2 u, 20,σ2

7 LOCAL LINEAR ESTIMATION 7 were ϕ it σ2,σ2 u = σ exp 2 u + it σ2 2σ 2 Tus I 2 t = Φ δ t u ϕ it σ2,σ2 udu = it σ2 t Φ δ Summarizig ad applyig 9 we obtai E x/ξ K x a σ2 x ξ = = σ2 K it exp it ξ a Φ K t dt ξ a 2 Note tat K u = t 2 exp itu Φ K t dt 2 I te same way as above we get E x/ξ K x a x ξ2 = exp it ξ a Φ K t Φ t I 3 t dt, δ wit I 3 t = u 2 exp i t uϕ δ u du Applyig 20 we obtai I 3 t = Φ δ t u 2 ϕ it σ 2 u du =,σ2 Summarizig ad usig 2 we get σ 2 σ 4 t 2 Φ δ t E x/ξ K x a x ξ2 = exp it ξ a t 2 Φ K t σ 2 σ 4 dt = σ4 ξ a ξ a 2 K + σ 2 K Lemma I te model - 5 ad for kerels wit 6-9 ad = C log, C sufficietly large, it olds tat: K xi x = + O P log 22

8 8 SILVELYN ZWANZIG 2 K xi x x i = x + O P log K xi x x 2 i = σ 2 + O P log ξi x K ξ 2 i K xi x y i = K xi x = ξi x K ξi x K gξ i +O P log 25 y 2 i 26 gξi 2 + σ 2 + O P log 2 6 K xi x x i y i = σ 2 g x + O P log 3 2 ξi x K ξ i gξ i 27 Proof Te proofs are based o a sum of iid radom variables of te form S = Z i For V = V ar Z i we ave S = ES + O p V 2 28 Here oly te mai steps are give, but all details are sow i [5] First we cosider S = Z,i wit Z,i = K for te expectatio wit respect to δ i E xi /ξ i Z,i = E xi /ξ i K xi x ξi x = K xi x We ave

9 LOCAL LINEAR ESTIMATION 9 Furter E 2 K xi x VK wit V K = Ku 2 du From te Parseval equality ad 2 it follows tat V K = ΦK t 2 dt = expσ 2 t2 2 Φ K t 2 dt For kerels K wit compactly supported Fourier trasform, 7, we get t2 V K maxexpσ2 t [a,b] Φ 2 K t 2 cost expc Tus for =, V ars = V K cost expc 0 2 l l < cost Te expectatio of S wit respect to all δ i is te ordiary desity estimator of p ξ : ES = p ξ x wit p ξ x = ξi x K Usig te model assumptio 3 we get for more details see te report [5] p ξ x = + O P log 30 2 Here S 2 = Z 2,i wit Z 2,i = K xi x xi From 7 it follows tat E xi /ξ i Z 2,i = σ2 ξi x ξi x K + K ξ i 3 Furter E Z 2,i 2 E 2 K xi x xi c2 ξ i V K Tus usig a similar argumetatio as i 29 we get tat te V ar S 2 < cost Te coditioal expectatio of S 2 ca be iterpreted as a sum of a ordiary estimator of p ξ ad of te expectatio of a ordiary kerel estimator fx = ξi x K x i 32 of fx = x Usig te model assumptio 3 we get for more details see te report [5] for = ES 2 = x + O P 3 + = x + O P log

10 0 SILVELYN ZWANZIG 3 Now we ave a sum of idepedet rv S 3 = Z 3,i wit Z 3,i = x 2 i Applyig 7, 8 ad 3, we obtai E xi /ξ i Z 3,i = K xi x K ξ i x σ 2 + ξ 2 σ 2 i + 2ξi K ξi x + σ4 2 K ξ i x Furter we apply similar argumetatio as above ad use p ξ x = 0 + O P + ad σ 2 K ξi x ξ i = σ 2 + O P We estimate V ar S 3 < cost 4 Ca be sow similarly, see [5] We use E xi /ξ i Z 4,i = K ξ i x g ξi i a similar way as above 5 Ca be sow similarly, see [5] We use E xi /ξ i Z 5,i = K ξ i x g ξi 2 + σ 2 6 Ca be sow similarly, see [5] wit Z 6,i = K xi x yi x i We use E xi /ξ i Z 6,i = E K xi x x i g ξ i ξi x = K ξ i + σ2 ξi x K g ξ i ad tat σ 2 K ξi x g ξ i = g x + O P Now we summarize te results for meas ad quadratic forms wit weigts wi Lemma I te model - 5 ad for kerels wit 6-9 ad = C log, C sufficietly large, it olds tat: m ξξ = O P log 34 x w = x + O P log 35 y w = gx + O P log 36 m xx = σ 2 + O P log 37 m xy = σ 2 g x + O P log 38 m yy = σ 2 + O P log 39

11 LOCAL LINEAR ESTIMATION Proof We decompose m ξξ = w i x ξ i x 2 ξ w x 2 tus mξξ w i x ξ i x 2 Cosider K ξ i x ξi x 2 as S = Z i We get from 9 tat x EZ i 2 0 x x K u u 2 du 2 c K Te variace term is estimated by E K ξ x Remember 30 we get w i x ξ i x 2 = O p 0 x u 2 du = O3 2 ξ x 2 = O 2 Applyig 22 ad 23 we get x w = x + Op 3 + O p = x + Op 3 Aalogously we estimate y w by usig 22 25, 30 y w = K ξ i x g ξi + O p + O p = g w p ξ x + O p = g w + O p Furtermore it olds g w = gx+o p 2 +, compare for istace Teorem 4 i [5] Tus y w = gx + O p 4 Usig te decompositio m xx = wi x 2 i x w 2 Because of 22, 24 it olds tat wi x 2 i = K xi x + O p x 2 i = σ 2 + ξi x K ξ 2 i + O p Applyig 30 we obtai w i x 2 i = w i ξ 2 i σ 2 + O p Tus m xx = m ξξ σ 2 +O p Te te statemet 37 follows from 34 5 Usig te decompositio m xy = wi x i y i y w x w ad 22, 27, ad 30 we get wi x i y i = ξi x K ξ + O p i gξ i σ 2 g x + O p = w i ξ i gξ i σ 2 g x + O p

12 2 SILVELYN ZWANZIG Uder 5 we ca sow compare Teorem 4 i [5] tat m ξg = g xm ξξ + O p 2 + Te 38 follows from 34, 35, ad 36 6 We cosider ow m yy = w i y 2 i y w 2 From 22 ad 26 we get Tus by 30 w i y 2 i = ξi x K gξ i 2 + σ O p wi yi 2 = w i gξ i 2 + σ 2 + O p Uder 5 we ca sow compare Teorem 4 i [5] tat m gg = g x 2 m ξξ + O p Te 39 follows from 34, 35 ad 36 Bibliograpy J Fa ad I Gjibels 996, Local Polyomial Modelig ad Its Applicatio, Lodo: Capma ad Hall 2 J Fa ad Y Truog 993, Noparametric regressio wit errors i variables A Statist WA Fuller 987, Measuremet Errors Models Wiley, New York 4 J Staudemayer ad D Ruppert 2004, Local polyomial regressio ad simulatio extrapolatio JRSocB 66, Part, S Zwazig 2006, O local liear estimatio i oparametric errors-ivariables models UUDM Report Uppsala Uiversity Departmet of Matematics, Uppsala Uiversity, SE-7506 Uppsala, Box 480 Swede zwazig@matuuse