SMOOTHING QUANTILE REGRESSIONS
|
|
- Basil Strickland
- 5 years ago
- Views:
Transcription
1 SMOOTHING QUANTILE REGRESSIONS Emmauel Guerre Marcelo Ferades Eduardo Horta Scool of Ecoomics ad Fiace Scool of Ecoomics ad Fiace Departmet of Statistics Quee Mary Uiversity of Lodo Quee Mary Uiversity of Lodo UFRGS ABSTRACT: Te paper proposes a ew smooted versio of te quatile regressio estimator. We sow tat smootig te objective fuctio improves te efficiecy of quatile regressio estimatio relative to smootig oly te ceck fuctio. Its secod-order mea squared error is studied ad sow to be smaller ta te oe of Koeker ad Bassett 978 quatile regressio estimator we a optimal badwidt is used. Te paper proposes a data-drive coice of te badwidt troug cross-validatio. A simulatio experimet reveals tat te improvemet ca be substatial wit a Mea Squared Error reductio as ig as 4% compared to te usual quatile regressio estimator of Koeker ad Bassett 978. JEL CLASSIFICATION: C4. KEYWORDS: Baadur approximatio, data-drive badwidt, kerel estimatio, quatile regressio. ACKNOWLEDGEMENTS:
2 . INTRODUCTION 2. THE OTHER SMOOTHED QUANTILE REGRESSOR ESTIMATOR I wat follows. we restrict attetio to te media estimatio. Tis is just for simplicity i tat it is straigtforward to exted te discussio to ay oter quatile. Defie te media estimator as te miimizer of H b Y i b K Yi b, 2 were K t t Ku du. Te first ad secod derivatives of te are H b H 2 b { Y i b K Y i b + K Yi b }, 2 { Yi b 2 K Yi b } + 2 K Y i b. 2.. Te secod-order term. Observe tat t k K t dt k t k Kt dt for ay iteger k as log as bot itegrals exist. It te olds tat y b y b E H2 b 2 K fy dy + 2 K y b fy dy t K t fb + z dz + 2 Kz fb + z dz fb + O s + 2 fb + O s fb + O s Te first-order term. Te variace of te first-order term satisfies var H b var K Yi b It turs out tat var K Yi b + var Y i b K Y i b + 2 cov Y i b K Y i b, K Yi b { F b F b 2 } K z K z dz + o, 2
3 wereas var Y i b K Y i b fb cov Y i b K Y i b, K Yi b fb u 2 K 2 u du + o, zkzk zfb + z dz K z K z dz + o give tat z Kz K z dz K z K z dz. It te follows tat { } var H 2 b F b F b 2 K z K z dz + o + 2 fb K z K z dz + fb u 2 K 2 u du + o. 3. SMOOTHING QUANTILE REGRESSIONS Cosider a radom sample {Y i, X i, i,..., } of Y, X R R d + suc tat Y i X iβu i X i β U i + + X id β d U i, were U i is a, -uiform radom variable idepedet of X i. Let F y x P Y y X x deote te coditioal cumulative distributio fuctio of Y give X. Assumig tat te mappigs u β j u for j,..., d are cotiuous ad icreasig over te uit iterval, te coditioal quatile fuctio is te uiquely defied by Qτ x F τ x x βτ. Koeker ad Bassett 978 estimate te quatile regressio by βτ argmi ρ τ Yi X b R d ib, 2 wit ρ τ v v τ Iv <. Te stadard approac i te literature is to smoot te ceck fuctio ρ τ by replacig te idicator fuctio I v < wit a kerel-based couterpart see Amemiya, 982; Horowitz, 998. We take a differet route by usig a smooted estimator of te cumulative distributio of deviatio V i b Y i X i b rater ta te empirical distributio i 2. We sow tat tis etails a more efficiet smootig of te objective fuctio ta just smootig te ceck fuctio. 3
4 Let F t; b IV ib t ad F t; b t f v; b dv, were f v; b is te kerel desity estimator, wit badwidt >, give by f v; b K v t d F t; b wit Kt dt. Itegratio by parts gives way to ρ τ V i b ρ τ v d F v; b τ τ F v; b dv + τ K v V i b, v d F v; b + τ F v; b dv. We propose smootig te objective fuctio by replacig F t; b wit F t; b, amely, Rb; τ, τ F v; b dv + τ Te resultig smooted quatile regressio estimator te is F v; b dv. v d F v; b βτ; arg mi b R d R b; τ,. 3 It is easier to appreciate te motivatio for smootig directly te objective fuctio by lookig at te first-order coditio, i.e., te score fuctio R b; τ, b Rb; τ,. It follows from te defiitios of F ad f tat R b; τ, τ τ F t; b dt τ F t; b dt b b X i K t V i b dt τ X i K t V i b dt { } X X i τk i b Y i X τ K i b Y i X X i K i b Y i τ. If d ad X i, solvig R b; τ, reduces to τ F b τ F b, 4 wit F b K b Yi or, equivaletly, F b τ. Tis meas tat te solutio of 4 is te well-kow Nadaraya s 964 smooted quatile estimator. Tis also illustrates te fact tat R b; τ, is 4
5 aalogous to a smooted cumulative distributio fuctio. Tis is i cotrast wit te smootig procedure put fort by Horowitz 998, wose first-order coditio ivolves a quatity aalogous to a kerel-based estimator of te probability desity fuctio. As a result, te secod-order derivative of our smooted quatile regressio estimator R 2 b; τ, is similar to a kerel desity estimator, wereas Horowitz s 998 ivolves terms tat are similar to a kerel estimator of te derivative of a probability desity fuctio. Tis esures tat our estimator as better ig-order properties ta Horowitz s 998 give tat te kerel desity estimator coverges at a faster rate relative to te kerel derivative estimator. Our mai assumptios are as follows. Deote te coditioal probability desity fuctio of Y give X x by fy x d x j β j Q y x j ad let f j y x deote its jt derivative j y j fy x. Let also s s deote te lower iteger part of ay positive real umber s, tat is to say, te uique iteger umber satisfyig s s < s. Defie X ad Y R as te port of X ad Y. Assumptio Q Te coditioal probability desity fuctio fy x is cotiuous over X Y, wit fy x > for all x, y X Y. Te coditioal quatile fuctio y Qy x is strictly icreasig for all x X. Assumptio D Tere exist some s, L > suc tat x,y f j y x L, wit lim y ± f j y x for j,..., s, ad f s y x f s y x L y y s s for all x X ad y, y Y. Assumptio X Te covariates X X,..., X d are almost surely positive, bouded ad suc tat D EXX as full rak. Assumptio K Te kerel K is symmetric, cotiuous ad twice differetiable wit Ky dy <, Ky dy, K y dy <, K 2 y dy <, ad < For s as i Assumptio Q, y s+ Ky dy < ad yky dy K y K y dy <. y s+ Ky dy. 5
6 Assumptio H Te badwidt is i te iterval, wit / o/ l ad o. Remarks a Commets o Assumptio D ad te Laplace distributio 2 exp x for wic s eve if tere is o derivative. b Ay symmetric o-egative kerel automatically meets te coditio K y K y dy > i Assumptio K. Te latter is oterwise fairly geeral, requirig oly tat te kerel fuctio is moderately large i te tails so as to esure tat K y remais iferior to oe. Before establisig te mai results, it is ecessary to itroduce some furter otatio. Let ad deote te cadidate limit of R 2 b; τ, by R 2 b; τ, 2 Rb; τ, b b D τ E XX f X βτ X. Assumptios Q, D ad X te esure tat D τ exists for all τ. Let deote te Euclidea orm of a vector or a matrix, amely, V trv V. 4. MAIN RESULTS Altoug tis paper ougt to illustrate te positive aspects of smootig, ay reader familiar wit oparametric approaces is probably already aware tat te beefits expected from tis teciques comes wit potetially importat drawbacks. Ideed, te smooted quatile regressio estimator β τ; sould ot be viewed as a estimator of βτ, a approac wic would amout to igore te impact of smootig. It is more oest to iterpret β τ; as a estimator of β τ; arg mi E Rb; τ,, b R d a poit of view tat ackowledges tat β τ; ca be a biased estimator of βτ. Te ext result studies te order of te bias term β τ; βτ. THEOREM BiasQ. Give Assumptios Q, X, K, β τ; is uiquely defied for all τ τ, τ ad satisfies, uiformly wit respect to τ, τ, τ,, βτ; βτ + O L s+. 6
7 Additioally, if s is a iteger umber ad y fy x is st cotiuously differetiable for all x, te followig expasio olds βτ; βτ s+ Bτ { + o}, 5 were Bτ z fx βτ x xx s+ Kz dz fx dx s +! E Xf s X βτ X. Let us ow tur to te positive improvemets iduced by smootig. Te ext teorem deals wit Baadur represetatios for βτ; βτ;, tat is, te approximatio of tis quatity by a stadardized sum. THEOREM 2 Baa. Give Assumptios K, Q ad X, βτ; is uique for τ, τ, τ, wit probability tedig to ad satisfies βτ; βτ; R2 βτ; ; τ, R βτ; ; τ, l + O P 3/2 + s /2 D τ R βτ; ; τ, +O P l 3/2 + s /2 + l /2 + L s 6 7 uiformly wit respect to τ, τ, τ,, were R2 τ, τ,τ, βτ; ; τ, ad τ, τ,τ, R βτ; ; τ, are bot of order O P. Tese liear approximatios for βτ; βτ; differ oly because tey employ distict stadardizatios. Te latter is te radom sum R 2 βτ; ; τ, i 6, wereas 7 uses istead its limitig couterpart D τ. As a cosequece, te remaider term of 7 icludes a additioal term l / /2 + s relative to 6, correspodig to te order of R2 τ τ,τ βτ; ; τ, D τ. As usual, tere is a factor l because of te uiform ature of te result. Note tat bot approximatios ivolve te term R βτ; ; τ,, wic is cetered give tat E R βτ; ; τ, b E bβτ; Rb; τ,. 7
8 Koeker 25 discusses similar approximatios for te regressio quatile estimator βτ βτ. See page 23 as well as refereces terei. It is well kow tat te best possible error for te correspodig remaider term is /4 up to iessetial logaritmic factors. Tis cotrasts wit te remaider term of 6 wic is at worst of order /4 provided tat s > /2 ad remais larger ta l /3 / /2. Ideed, te remaider term 6 acieves te order /2 wic is typical of estimators wic miimize a smoot criterio fuctio provided s ad assumig tat te order of remais larger or equal ta l / /3. Te study of te remaider term of 7 is more complex due to te additioal term l / /2 + s. Coosig a badwidt proportioal to /2s+, wic is optimal i view of te criterio cosidered i Teorem 3 below, gives tat te order of l / /2 + s is l / s/2s+ wic is always larger ta /2 but is smaller ta /4 provided s > /2. Teorem 2 is also a crucial step for te study of te asymptotic properties of te smooted quatile regressio estimator. Te liear approximatio 7 sows tat te asymptotic properties of β; τ are give by te leadig term βτ; D τ R βτ; ; τ,. I wat follows, we focus o te optimal coice of te badwidt we estimatig a liear combiatio λ βτ. Tis icludes i particular estimatio of eac of te coefficiets β j τ by a proper coice of te vector of λ. Defie te Asymptotic Mea Square Error of λ βτ as AMSEλ β; τ E λ βτ; D τ 2 R βτ; ; τ, βτ. Tis quatity is a proxy for te mea square error MSEλ β; τ E λ β; τ βτ 2. Studyig te AMSE istead of te MSE amouts to eglectig te remaider term of 7, wic sriks to. Te ext result describes a optimal badwidt coice wit respect to te AMSE criterio. For te quatile estimator 4, Ralescu 996 gives a liear approximatio wit a remaider of order /4 up to a logaritmic term witout restrictio o s > but provided tat te badwidt is larger ta l l / /4, suggestig tat te order of 6 is too pessimistic for small s. Ideed our proof teciques are cetered o larger values of s for wic a liear approximatio ca old wit a better order ta /4. 8
9 THEOREM 3 VarQ. Give Assumptios Q, D, X, K, ad H we ave uiformly wit respect to τ, τ, τ,, D var τ R βτ; ; τ, τ τd τd D τ { } 2 K y K y dy + O +s. 8 If te expasio i 5 olds, AMSEλ β; τ is asymptotically miimal if + oopt, were opt as log as λ Bτ. I tis case, λ D τd D τλτ τ K y K y dy s + λ Bτ 2 2s+ AMSEλ λ β; D τ τd D τλτ τ { 2s + } 2 opt K y K y dy + o 2s 2s+. 2s + 2 A key poit i Teorem 3 is te fact tat te variace expasio i 8 icludes a egative term, amely, 2 K y K y dy. Tis implies tat te asymptotic variace of te smooted βτ; is smaller ta te oe of te quatile regressio estimator βτ. As already oticed by Azzalii 98 for stadard quatile estimatio, it te follows tat te AMSE of λ βτ; ca be made smaller ta te oe of λ βτ. I oter words, smootig te loss fuctio ρ τ as i 3 improves te stadard regressio quatile estimator. I te case of quatile estimatio, a fudametal reaso for suc a improvemet is tat te smoot estimator of te cumulative distributio fuctio domiates te sample cumulative distributio fuctio as sow i Reiss 98. Uder Assumptios Q ad D, te optimal badwidt is of order /2s+ as te oe wic would be used to estimate te ucoditioal probability desity fuctio of Y. Te order of te secod-order variace term wit is /2s+ / 2s+2/2s+ wic is close to te first-order variace term λ D τd D τλτ τ/ we s is reasoably large, beig for istace 6/5 if s 2. Teorem 3 gives more detailed expasio i te case were y fy x is s-t times cotiuously differetiable. Te expressio of te optimal badwidt opt resembles te oe give i Lio ad Padgett 99 for estimatio of te cumulative distributio fuctio, see also Reiss 98. 9
10 5. A DATA-DRIVEN BANDWIDTH CHOICE Defie β λ τ; λ βτ; ad Bλ β λ τ; β λ τ; for λ R d. For a badwidt b, let D X ix i, D R 2 βτ; b; τ, b, D τ τ D D D, ad D λ λ Dλ. Let also H {,,..., K }, wit k k. Cosider te data-drive badwidt ĥ arg mi H Q;, wit Q; B 2 λ 4 Dλ K y K y dy, for. We propose to compute te badwidt by ĥ ĥ+, leadig to te fully data-drive estimator β λ τ β λ τ; ĥ. Let argmi H Q;, were Q; B 2 λ 2 D λ τ β λ τ; λ βτ;, ad B λ β λ τ; β λ τ;. K y K y dy, Assumptio B: Te badwidt b is suc tat b o ad /b o/ l, wereas te badwidts ad K respectively satisfy C /2 l ad K o, wit Kl. I additio, > is suc tat wit meetig te followig coditios. l l l. Tere also exist ukow s > /2 ad L > L > B B λ / Ls + s, L s+ B λ, ad is uiquely defied i te limit wit /2s++. B2 For all ɛ >, lim if mi 2s/2s++ H; / ɛ Q; Q ; >. THEOREM 4 DD. Uder Assumptios B ad K, te data-drive smooted quatile regressio estimator is suc tat, for +, βτ βτ Z + o P, were EZ ad varz { 2 } K y K y dy τ τd τd D τ. Note tat tere is o bias term above because βτ; βτ o P.
11 Lettig Rb; τ, E Rb; τ, yields 6. PROOFS Rb; τ Rb; τ, E ρ τ Y X b x b τ F v x dv + τ wose first- ad secod-order derivatives are respectively x b F v x dv fx dx, R b; τ R 2 b; τ F x b x τ x fx dx, fx b xxx fx dx. Let also deote S b,τ, te set R d,, to wic b, τ, belog. 6.. Prelimiary results. LEMMA BiasRF. Assumptios K, D ad X esure tat R 2 b; τ, R 2 b; τ b,τ, S b,τ, L s O, R b; τ, R b; τ b,τ, S b,τ, L s+ O, Rb; τ, Rb; τ b,τ, S b,τ, maxl, s+ O, R 2 max j j 2 b + δ; τ, R 2 j j 2 b; τ, j,j 2 d b,τ, S b,τ, L δ s O. Proof of Lemma. We ave Rb; τ, τ + τ τ + τ { v E { v E K t Y X b } dt E K t Y X b } dt dv, dv { } v+x b E E K t Y X dt dv { E v+x b K t Y X } dt dv.
12 It te follows from te Lebesgue Domiated Covergece teorem tat { R b; τ, E τ XE K v + X b Y X dv τ E τ X b X K v Y dv τ X b X K v Y dv R 2 b; τ, τe XX K X b Y + τe XX K X b Y E XX K X b Y. XE K v + X b Y X Assume from ow o tat s give tat te case s is trivial. Uder Assumptio D, a Taylor expasio wit itegral remaider gives s fv + z x f l v x l zl l! + zs s! Usig a cage of variables y t + x b + z yields uder Assumptio K E K t Y X b X x ft + x b x f s v + wz x w s dw. K t y x bfy x dy ft + x b x Kzft + x b + z x ft + x b x dy, } dv It te follows from Assumptio D tat E K X b Y { f s t + x b + wz x f s t + x b x z s fx b xfxdx C s z s Kz dz. s! Kz d Tis yields te first stated result. We ow tur our attetio to te stated results for R ad R. Itegratig 9 wit b yields x b { } E K t Y X x ft x dt { } z f s x b + wz x f s x b x s Kz dz w s dw s! { } z f s x b + vwz x f s x s+ b x dv Kz dz w w s dw s! C L s+, 2
13 wereas v+x b { } E K t Y X x ft x dt { } z f s 2 v + x b + wz x f s 2 v + x s b x s! Kz dz w s dw C maxl, s+. Tis implies tat { R b; τ, τf x b x τ F x b x } x fx dx CLs+, + } { Rb; τ, τ F v + x b x dv + τ F v + x b x dv fx dx CL s+ for all τ,. Observe ow tat R 2 j j 2 b; τ, E X j X j2 K X b Y x x 2 Kz fx b + z x fx dx dz. It te follows tat R 2 j j 2 b + δ; τ, R 2 j j 2 b; τ, uiformly i b,, δ, τ, j ad j 2. L x x 2 Kz fx b + x δ + z x fx b + z x fx dx dz x x 2 Kz x δ s fx dx dz CL δ s, LEMMA 2 StocR. Give Assumptios K ad X, max b,τ, R d,, b,τ, R d,, j,j 2 d b,τ, R d,, were, for te latter R b; τ, R b; τ, O P, R2 b; τ, R 2 b; τ, l / l l O P, 2 R j j 2 b; τ, R 2 j j 2 b; τ, δ l / l l O P, R 2 j j 2 b; τ, R 2 j j 2 b + δ; τ, R 2 j j 2 b; τ,, R 2 j j 2 b; τ, R 2 j j 2 b + δ; τ, R 2 j j 2 b; τ,. Proof of Lemma 2. Recall tat 3
14 R b; τ, { τ t Yi X i X i K b } dt τ X X i K i b Y i τ. + { t Yi X i X i K b } dt Cosider te classes of fuctios { x F j x, y x j K b y ; b, R d, }, j,..., d. Sice K < ad X j is bouded, Lemma 22-ii i Nola ad Pollard 987 esures tat F j is Euclidea wit a square itegrable evelope C X j. Te te remark below Defiitio 8 i Nola ad Pollard 987, Propositio ad iequality p. 39 i Eimal ad Maso 25 imply tat { } X X ji K i b Y i X E X ji K i b Y i O P. b, R d, Te te expressio of R b; τ, give above sows tat R b; τ, R b; τ, O P. b,τ, R d,, Te proof of te result for R 2 follows te same steps ta te proof of Teorem i Eimal ad Maso 25, usig K < ad Assumptio K2. Te proof of te result for R 2 follows te same steps, usig K < ad X j X j2 X b + δ Y K δ X b Y K to boud te summads i R 2 ad to compute teir variace X j X j2 X δ X K b + wδ Y dw δ C X K b + wδ Y dw 6.2. Proof of Teorem. Let βτ; be ay miimizer of Rb; τ,. Lemma sows, sice β τ arg mi b R b; τ R βτ; ; τ R βτ; τ if b Rb; τ, mi R βτ; τ + R βτ; ; τ R βτ; ; τ, O max L, s+, 4 b
15 uiformly i ad τ. Sice R βτ; τ, te Taylor formula wit itegral remaider gives R βτ; ; τ R βτ; τ βτ; βτ R 2 βτ + w βτ; βτ ; τ dw βτ; β τ. Sice R 2 b; τ f x b x xx fxdx, Assumptio Q ad X esures tat te miimal eigevalue of R 2 βτ + w βτ; βτ ; τ dw is larger ta C > for all τ τ, τ ad,. Hece substitutig i te iequalities above gives βτ; βτ max L, s+ /2 O.,τ, τ,τ We ow sow tat βτ; is uique, tat is Rb; τ, as a uique miimizer, provided tat is small eoug. Cosider te compact set C C ɛ { } b R d ; b j β j τ ɛ, β j τ + ɛ, j,..., p, ɛ > recallig tat β j icreases wit τ uder Q. Takig ɛ small eoug esures via Q ad X tat te miimal eigevalue of te defiite positive R 2 b; τ is larger ta a C > for all b C, τ τ, τ. Hece Lemma esures tat te maps b C Rb; τ,, τ τ, τ ad,, are all strictly covex provided is small eoug. Sice sows tat all te cadidate miimizers β τ;, τ τ, τ ad,, are i C for is small eoug, te strict covexity of te correspodig Rb; τ, esures tat all tese βτ; are uiquely defied. We ow improve. Sice Lemma gives tat R βτ; ; τ, adr βτ; τ, τ,,, τ,,, R β τ; ; τ R βτ; τ L s+ R β τ; ; τ, R βτ; ; τ L s+ O. 5
16 Now te Taylor formula, te expressio of R 2, Assumptio Q ad give R βτ; ; τ R βτ; τ R 2 βτ + w βτ; βτ ; τ dw βτ; βτ R 2 βτ; τ + O L βτ; βτ s β τ; βτ R 2 βτ; τ + O L max L, s+ s /2 βτ; βτ uiformly i τ,,,. Sice te LHS is O L s+, tis gives uiformly i τ,,, ad provided tat is small eoug, ad βτ; βτ + O L s+ is prove. Te proof of 5 similarly uses te expasios { } βτ; βτ + R 2 βτ; τ O L s+ R βτ; τ, R βτ; ; τ, R βτ; τ, { R 2 βτ; τ + O C} βτ; βτ, ad, usig s s +, ad w 2 w s dw/2 /ss + s + 2 R z s+ Kzdz βτ; τ, + o f s x βτ x fxdx, s +! so tat 5 olds Proof of Teorem 2. Lemmas ad 2 give sice τ,,, τ,,, b,τ, R d,, + o P. Sice τ,,, β R τ; ; τ R βτ; τ β R τ; ; τ R b; τ R b; τ, + R βτ; ; τ, R βτ; ; τ R βτ; τ b,τ, R d,, R b; τ, R b; τ, R 2 βτ + w βτ; βτ ; τ dw βτ; βτ 6
17 ad because te miimal eigevalue of R2 βτ + w b βτ ; τ dw is larger ta C > for all b ad τ τ, τ uder Assumptios Q ad X, tis gives tat τ, τ,τ, β τ; βτ o P, ad by Teorem, τ, τ,τ, β τ; βτ; o P. 2 Note te former sows tat it is possible to fid a compact B suc tat { βτ; }, τ, τ, τ, B wit a probability tedig to. Because Lemmas ad 2 imply tat all te fuctios b B R b; τ, are strictly covex wit a probability tedig to, it follows tat all te βτ;, τ, τ, τ, are uiquely defied wit a probability goig to. Sice R β τ; ; τ,, Lemmas 2, ad Teorem, we ave uiformly i τ, τ, τ,, R βτ; ; τ, R β τ; ; τ, R β τ; ; τ, R 2 β τ; + w βτ; β τ; ; τ, dw β τ; βτ; R 2 l /2 βτ; ; τ, + O P βτ; β τ; + O L β τ; βτ; s βτ; βτ;, 3 were te eigevalues of R 2 β τ; ; τ, are bouded away from ad ifiity wit a probability tedig to. Sice R βτ; ; τ,, Lemma 2 sows tat R βτ; ; τ, O P / uiformly. Hece applyig 3 ad 2 give, sice o/ l, 3/2 βτ; βτ; OP τ, τ,τ, l /2 ad, reapplyig 3, Applyig 3 agai gives βτ; β τ; τ, τ,τ, β τ; βτ; O P R 2 l /2 β τ; ; τ, + O P 3/2 + s /2 R βτ; ; τ, R2 βτ; ; τ, R l /2 βτ; ; τ, + O P 3/2 + s /2 7
18 uiformly wit respect to τ, τ, τ,. Tis gives te first Baadur represetatio 6 i Teorem 2. Te Baadur represetatio 7 te follows from Lemmas ad Proof of Teorem 3. We ave, sice E R βτ; ; τ,, var R β τ; ; τ, { X var X K β τ; Y E } X τ E XX K βτ; Y 2 τ X XX K β τ; Y 2 X 2τE XX K βτ; Y + τ 2 E XX. Asuumptios D ad K give, itegratig by parts ad usig Teorem, X E K βτ; Y X x x K βτ; y x fy x dy K βτ; y F y x dy F x βτ; x + F x βτ; z x F x β τ; x Kz dz F x βτ x + O s+ + O s+ τ + O s+, sice x βτ F τ x ad arguig as i Lemma.For te term ivolvig K 2, defie Kz 2KzK z d dz K z 2, wic is suc tat Kz dz lim z + K z 2. 8
19 Arguig as above ow gives X E K βτ; Y 2 X x x K βτ; y F y x dy τ + O s+ + F x βτ; z x F x βτ; x Kz dz τ f x βτ; x + O s zkz dz + O s+ τ f x βτ x + O +s s + O s zkz dz + O s+ τ f x βτ x zkz dz + O +s. Substitutig gives te variace expasio sice K z K z, zkz dz 2 zkzk zdz zd K z zd K z + K z 2 dz + + K z K z 2 dz K z dz 2 Te variace expasio ad 5 te gives AMSE λ +s βτ; g + o were g Sice g s+2, + K z K z dz. τ τ λ K z K D z dz τd D τλ + 2s+2 λ Bτ 2. K z K z dz τ τ λ D τd D solvig g gives te desired opt ad te correspodig AMSE expasio. τλ +2s + 2 2s+ λ Bτ 2, 7. WORK IN PROGRESS Defie for λ R d β λ τ; λ βτ;, Bλ β λ τ; β λ τ;. 9
20 Cosider a badwidt b. Let D D τ τ X i X i, D R 2 β τ; b ; τ, b, D D D, Dλ λ Dλ. Let H {,,..., K } were k k. A first data-drive badwidt is, for ad, Te proposed optimal badwidt is ĥ arg mi H Q ; were Q ; B 2 λ 4 Dλ ĥ ĥ+ + K y dy. leadig to te fully data-drive estimator β λ τ β λ τ; ĥ. 7.. Proof i progress. Set β λ τ; λ β τ;, B λ β λ τ; β λ τ; ad Q ; B 2 λ 4 D λ τ arg mi Q ;. H + K y dy, Assumptio B: Te badwidts, K ad K satisfy l /2 C, K o adk/ l. > is suc tat wit lim L > L > suc tat l l l +. Tere are some ukow s > /2 ad B: B λ / L s + s, L s+ B λ ad is uiquely defied asymptotically wit /2s++ ; B2: lim if 2s/2s++ mi H; / ɛ {Q ; Q ; } > for all ɛ >. Routie modificatios of Arcoes 996 give l βλ τ; β λ τ λ D τ R /4 βτ; τ, + O P, 2
21 see also He ad Sao 996, Remark 3.4. Hece 7 gives β λ τ; β /2 λ τ; B λ τ; λ were +O P l /3 3/4 + l/2 3/2 + /2 λ λ D τ R βτ; ; τ, R βτ; τ, λ D τ s /2+ + B λ /2 { X X i K i βτ; Y i I Y i X iβτ }. We ave E λ ad X E K β τ; Y I Y X βτ 2 X x + x βτ + + x βτ x K βτ; y x βτ βτ;/ x βτ βτ;/ + f x βτ x { x K βτ; y 2 fy x dy 2 fy x dy K u 2 f x β τ; u x du K u 2 f x βτ; u x du K u 2 du + + } 2 K u du uiformly i, ad x uder Assumptios B ad X. Hece, uiformly i,, { + } 2 var λ K u 2 du + K u du λ D τe XX f X βτ X D τλ. Te arguig as i Eimal ad Maso 25 gives λ τ, max O P. l /, 2
22 It follows tat, uiformly wit respect to H, β λ τ; β l / λ τ; B λ τ; + o P O P 3/4 +O P l /3 + l/2 3/2. /2 Observe ow tat te coditios o, ad K i Assumptio B esure tat { } { l / max H max l } { max exp l H H\{ } O exp l l l K l max H\{ } 3/4 l /3 / l/2 3/2 { 3/4 /2 } l /3 mi H\{ } max H\{ } l /3 l l l o K o, } 3/4 3/2 2 l /3 /2 l /6 O, 3/4 /2 /2 /2. l Tis gives uiformly wit respect to H B λ τ; β λ τ; β /2 λ τ; B λ τ; + o P + o P. Sice ad D λ D λ + o P, it follows tat max H max H B 2 λ τ; + / Q ; + / O Observe ow tat, sice /K o/ l ad usig B, Q ; Q ; Q ; + / o P. 4 C Q ; mi H {L 2 2s+2 C } C + o. Tis implies i particular tat Q ; Q ; + o P. Cosider ow some ɛ,. Uder B, we ave for all ɛ C Q ; L2 2s+2 C C ad 4 gives mi Q ; mi Q ; + o P Q ;. H; ɛ H; ɛ 22
23 It follows tat, uder B, B2 ad sice Q ; Q ; + o P, mi Q ; > mi Q ; H; ɛ H; / <ε wit a probability tedig to. Cosider ow + ɛ. Set κ 4D λ τ + K y dy so tat Q; Bλ 2 ; τ κ /. We ave for tose suc tat Q; κ/2 / Q ; Q ;. κ If Q; < κ/2 /, te B 2 λ ; τ 3κ/2 / so tat B gives 3κ/2L 2 /2s++ implyig tat Q; CQ ;. Hece 4 gives mi Q ; + op mi Q ; + o P Q ;, H; ɛ H; ɛ wic togeter wit Q ; Q ; + o P ad B2 gives mi Q ; > mi Q ;. H; +ɛ H; / <ε Hece ĥ/ + o P wic also sows tat ĥ+ + o P. We ow tur to βτ β τ; ĥ, ĥ ĥ+. 7 gives, for η ĥ+ / + were + β λ τ β λ τ; + / + + /2 E λ η l /2 +O P + /2 E λ η λ D τ R β τ; η + { λ D τ X i 3/2 + ; τ, η + X K i β τ; η + s /2+ + η + + /2 s+ R β τ; + Yi ; τ, + X K i β τ; + + } Yi. 23
24 We first study te icremets of E λ. Assumptio B gives for all η, η 2 /2, 2, { E X + K β τ; η + Y X η + K β τ; η 2 + } 2 Y η 2 + X x η {K u K u + x β τ; η + η β τ; η 2 + η 2 } 2 f x β τ; η + η + u x du x η 2 βτ;η + η βτ;η 2 + /η 2 Cη Ku + v dv 2 du C η 2 β τ; η + η β τ; η C η2 β τ; η + β τ; η2 + + η2 η β τ; η2 + 2 C η2 η 2. Hece va der Vaart ad Weller Corollary 2.2.5, 996 gives tat for ay ɛ > 2 E /2 E λ η 2 E λ η η,η 2 /2,2; η 2 η ɛ Cɛ /2. Tis gives E λ η O P ĥ + + /2 o P, ad, sice + l / /2, β λ τ β λ τ; + op + + /2 o P. /2 l /2 + O P + 3/2 + s / /2 s+ Tis gives te desired result. REFERENCES AMEMIYA, T. 982: Two stage least absolute deviatios estimators, Ecoometrica, 5, AZZALINI, A. 98: A ote i te estimatio of a distributio ad quatiles by a kerel metod, Biometrika, 68, HOROWITZ, J. L. 998: Bootstrap metods for media regressio models, Ecoometrica, 66, KOENKER, R. 25: Quatile Regressio. Cambridge Uiversity Press, Cambridge. KOENKER, R., AND G. BASSETT 978: Regressio quatiles, Ecoometrica, 46, LIO, Y. L., AND W. J. PADGETT 99: A ote o te asymptotically optimal badwidt for Nadaraya s quatile estimator, Statistics & Probability Letters,, NADARAYA, E. A. 964: Some ew estimates for distributio fuctio, Teory of Probability ad its Applicatios, 9, RALESCU, S. S. 996: A Baadur-Kiefer law for te Nadaraya empiric-quatile processes, Teory of Probability ad its Applicatios, 4, REISS, R.-D. 98: Noparametric estimatio of smoot distributio fuctios, Scadiavia Joural of Statistics, 8,
LECTURE 2 LEAST SQUARES CROSS-VALIDATION FOR KERNEL DENSITY ESTIMATION
Jauary 3 07 LECTURE LEAST SQUARES CROSS-VALIDATION FOR ERNEL DENSITY ESTIMATION Noparametric kerel estimatio is extremely sesitive to te coice of badwidt as larger values of result i averagig over more
More information4 Conditional Distribution Estimation
4 Coditioal Distributio Estimatio 4. Estimators Te coditioal distributio (CDF) of y i give X i = x is F (y j x) = P (y i y j X i = x) = E ( (y i y) j X i = x) : Tis is te coditioal mea of te radom variable
More informationNonparametric regression: minimax upper and lower bounds
Capter 4 Noparametric regressio: miimax upper ad lower bouds 4. Itroductio We cosider oe of te two te most classical o-parametric problems i tis example: estimatig a regressio fuctio o a subset of te real
More informationNotes On Nonparametric Density Estimation. James L. Powell Department of Economics University of California, Berkeley
Notes O Noparametric Desity Estimatio James L Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Uivariate Desity Estimatio via Numerical Derivatives Cosider te problem of estimatig te desity
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationLecture 9: Regression: Regressogram and Kernel Regression
STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 9: Regressio: Regressogram ad erel Regressio Istructor: Ye-Ci Ce Referece: Capter 5 of All of oparametric statistics 9 Itroductio Let X,
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS 1
Teory of Stocastic Processes Vol2 28, o3-4, 2006, pp*-* SILVELYN ZWANZIG ON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS Local liear metods are applied to a oparametric regressio
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationSemiparametric Mixtures of Nonparametric Regressions
Semiparametric Mixtures of Noparametric Regressios Sijia Xiag, Weixi Yao Proofs I tis sectio, te coditios required by Teorems, 2, 3 ad 4 are listed. Tey are ot te weakest sufficiet coditios, but could
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationRank tests and regression rank scores tests in measurement error models
Rak tests ad regressio rak scores tests i measuremet error models J. Jurečková ad A.K.Md.E. Saleh Charles Uiversity i Prague ad Carleto Uiversity i Ottawa Abstract The rak ad regressio rak score tests
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationUniformly Consistency of the Cauchy-Transformation Kernel Density Estimation Underlying Strong Mixing
Appl. Mat. If. Sci. 7, No. L, 5-9 (203) 5 Applied Matematics & Iformatio Scieces A Iteratioal Joural c 203 NSP Natural Scieces Publisig Cor. Uiformly Cosistecy of te Caucy-Trasformatio Kerel Desity Estimatio
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationDEGENERACY AND ALL THAT
DEGENERACY AND ALL THAT Te Nature of Termodyamics, Statistical Mecaics ad Classical Mecaics Termodyamics Te study of te equilibrium bulk properties of matter witi te cotext of four laws or facts of experiece
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationLecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
More informationNotes On Median and Quantile Regression. James L. Powell Department of Economics University of California, Berkeley
Notes O Media ad Quatile Regressio James L. Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Coditioal Media Restrictios ad Least Absolute Deviatios It is well-kow that the expected value
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationTR/46 OCTOBER THE ZEROS OF PARTIAL SUMS OF A MACLAURIN EXPANSION A. TALBOT
TR/46 OCTOBER 974 THE ZEROS OF PARTIAL SUMS OF A MACLAURIN EXPANSION by A. TALBOT .. Itroductio. A problem i approximatio theory o which I have recetly worked [] required for its solutio a proof that the
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationDetailed proofs of Propositions 3.1 and 3.2
Detailed proofs of Propositios 3. ad 3. Proof of Propositio 3. NB: itegratio sets are geerally omitted for itegrals defied over a uit hypercube [0, s with ay s d. We first give four lemmas. The proof of
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationStudy the bias (due to the nite dimensional approximation) and variance of the estimators
2 Series Methods 2. Geeral Approach A model has parameters (; ) where is ite-dimesioal ad is oparametric. (Sometimes, there is o :) We will focus o regressio. The fuctio is approximated by a series a ite
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationALLOCATING SAMPLE TO STRATA PROPORTIONAL TO AGGREGATE MEASURE OF SIZE WITH BOTH UPPER AND LOWER BOUNDS ON THE NUMBER OF UNITS IN EACH STRATUM
ALLOCATING SAPLE TO STRATA PROPORTIONAL TO AGGREGATE EASURE OF SIZE WIT BOT UPPER AND LOWER BOUNDS ON TE NUBER OF UNITS IN EAC STRATU Lawrece R. Erst ad Cristoper J. Guciardo Erst_L@bls.gov, Guciardo_C@bls.gov
More informationJournal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula
Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials
More informationA survey on penalized empirical risk minimization Sara A. van de Geer
A survey o pealized empirical risk miimizatio Sara A. va de Geer We address the questio how to choose the pealty i empirical risk miimizatio. Roughly speakig, this pealty should be a good boud for the
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationRates of Convergence by Moduli of Continuity
Rates of Covergece by Moduli of Cotiuity Joh Duchi: Notes for Statistics 300b March, 017 1 Itroductio I this ote, we give a presetatio showig the importace, ad relatioship betwee, the modulis of cotiuity
More informationBrief Review of Functions of Several Variables
Brief Review of Fuctios of Several Variables Differetiatio Differetiatio Recall, a fuctio f : R R is differetiable at x R if ( ) ( ) lim f x f x 0 exists df ( x) Whe this limit exists we call it or f(
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informations = and t = with C ij = A i B j F. (i) Note that cs = M and so ca i µ(a i ) I E (cs) = = c a i µ(a i ) = ci E (s). (ii) Note that s + t = M and so
3 From the otes we see that the parts of Theorem 4. that cocer us are: Let s ad t be two simple o-egative F-measurable fuctios o X, F, µ ad E, F F. The i I E cs ci E s for all c R, ii I E s + t I E s +
More informationCONCENTRATION INEQUALITIES
CONCENTRATION INEQUALITIES MAXIM RAGINSKY I te previous lecture, te followig result was stated witout proof. If X 1,..., X are idepedet Beroulliθ radom variables represetig te outcomes of a sequece of
More informationAppendix to: Hypothesis Testing for Multiple Mean and Correlation Curves with Functional Data
Appedix to: Hypothesis Testig for Multiple Mea ad Correlatio Curves with Fuctioal Data Ao Yua 1, Hog-Bi Fag 1, Haiou Li 1, Coli O. Wu, Mig T. Ta 1, 1 Departmet of Biostatistics, Bioiformatics ad Biomathematics,
More informationMath 341 Lecture #31 6.5: Power Series
Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationREGRESSION WITH QUADRATIC LOSS
REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationLecture 7 Testing Nonlinear Inequality Restrictions 1
Eco 75 Lecture 7 Testig Noliear Iequality Restrictios I Lecture 6, we discussed te testig problems were te ull ypotesis is de ed by oliear equality restrictios: H : ( ) = versus H : ( ) 6= : () We sowed
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationChapter 2 The Monte Carlo Method
Chapter 2 The Mote Carlo Method The Mote Carlo Method stads for a broad class of computatioal algorithms that rely o radom sampligs. It is ofte used i physical ad mathematical problems ad is most useful
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationINEQUALITIES BJORN POONEN
INEQUALITIES BJORN POONEN 1 The AM-GM iequality The most basic arithmetic mea-geometric mea (AM-GM) iequality states simply that if x ad y are oegative real umbers, the (x + y)/2 xy, with equality if ad
More informationRandom Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.
Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)
More informationLecture 27: Optimal Estimators and Functional Delta Method
Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationChapter 8. Euler s Gamma function
Chapter 8 Euler s Gamma fuctio The Gamma fuctio plays a importat role i the fuctioal equatio for ζ(s that we will derive i the ext chapter. I the preset chapter we have collected some properties of the
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More informationChapter 8. Euler s Gamma function
Chapter 8 Euler s Gamma fuctio The Gamma fuctio plays a importat role i the fuctioal equatio for ζ(s) that we will derive i the ext chapter. I the preset chapter we have collected some properties of the
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 12
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract I this lecture we derive risk bouds for kerel methods. We will start by showig that Soft Margi kerel SVM correspods to miimizig
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationINFINITE SEQUENCES AND SERIES
INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES I geeral, it is difficult to fid the exact sum of a series. We were able to accomplish this for geometric series ad the series /[(+)]. This is
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationSOLUTIONS TO EXAM 3. Solution: Note that this defines two convergent geometric series with respective radii r 1 = 2/5 < 1 and r 2 = 1/5 < 1.
SOLUTIONS TO EXAM 3 Problem Fid the sum of the followig series 2 + ( ) 5 5 2 5 3 25 2 2 This series diverges Solutio: Note that this defies two coverget geometric series with respective radii r 2/5 < ad
More informationMatrix Representation of Data in Experiment
Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y
More informationEstimation of the essential supremum of a regression function
Estimatio of the essetial supremum of a regressio fuctio Michael ohler, Adam rzyżak 2, ad Harro Walk 3 Fachbereich Mathematik, Techische Uiversität Darmstadt, Schlossgartestr. 7, 64289 Darmstadt, Germay,
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationChapter 10: Power Series
Chapter : Power Series 57 Chapter Overview: Power Series The reaso series are part of a Calculus course is that there are fuctios which caot be itegrated. All power series, though, ca be itegrated because
More informationx x x 2x x N ( ) p NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS By Newton-Raphson formula
NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS. If g( is cotiuous i [a,b], te uder wat coditio te iterative (or iteratio metod = g( as a uique solutio i [a,b]? '( i [a,b].. Wat
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationWeb-based Supplementary Materials for A Modified Partial Likelihood Score Method for Cox Regression with Covariate Error Under the Internal
Web-based Supplemetary Materials for A Modified Partial Likelihood Score Method for Cox Regressio with Covariate Error Uder the Iteral Validatio Desig by David M. Zucker, Xi Zhou, Xiaomei Liao, Yi Li,
More information2 Banach spaces and Hilbert spaces
2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationEstimating the Population Mean using Stratified Double Ranked Set Sample
Estimatig te Populatio Mea usig Stratified Double Raked Set Sample Mamoud Syam * Kamarulzama Ibraim Amer Ibraim Al-Omari Qatar Uiversity Foudatio Program Departmet of Mat ad Computer P.O.Box (7) Doa State
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationQuantile regression with multilayer perceptrons.
Quatile regressio with multilayer perceptros. S.-F. Dimby ad J. Rykiewicz Uiversite Paris 1 - SAMM 90 Rue de Tolbiac, 75013 Paris - Frace Abstract. We cosider oliear quatile regressio ivolvig multilayer
More informationElements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More information