Computation Of Asymptotic Distribution For Semiparametric GMM Estimators

Size: px
Start display at page:

Download "Computation Of Asymptotic Distribution For Semiparametric GMM Estimators"

Transcription

1 Computatio Of Asymptotic Distributio For Semiparametric GMM Estimators Hideiko Icimura Departmet of Ecoomics Uiversity College Lodo Cemmap UCL ad IFS April 9, 2004

2 Abstract A set of su ciet coditios for computig te asymptotic distributio of estimators wic are de ed via momet coditios wit i ite dimesioal parameters are preseted. We te coditios old, te mai teorem reduces te computatio of te asymptotic distributio to computig limits of a few momets.

3 Itroductio Adrews (994), Newey (994), Serma (994) ad Ai ad Ce (2003) ave exteded te ite dimesioal asymptotic aalysis to iclude i ite dimesioal parameters ad clari ed te structure of te computatio of asymptotic aalysis greatly. However, we give a estimator, eiter teir framework is limited or te coditios put forward are ot ecessarily easy to verify. Tis paper presets a set of su ciet coditios for computig te asymptotic distributio of estimators wic are de ed via momet coditios wit i ite dimesioal parameters. Te coditios are oped to be easy to verify i may applicatios. We te coditios old, te mai teorem reduces te computatio of te asymptotic distributio to computig limits of a few momets. 2 Model Let R p, Z R k, X R d, ad for eac, (; ) be a fuctio from X ito R`. We cosider a mappig g (z; ; (; )) from Z (; (; )) 2 ito R m. Geerally we cosider a Baac space of fuctios o R d wit some properties suc as give degree of di eretiability ad assume tat g is well de ed over Z. Te orm o is deoted by kk. We te domai of suc fuctios are restricted to X, we deote it by (X ). We assume for eac 2, (; ) 2 (X ). For brevity we sometimes write istead of (; ) ad, we z is evaluated at z i, write g i (; ) istead of g (z i ; ; (; )). Ofte g i (; ) = g (z i ; ; ( 2 (z i ; ) ; )) for a ukow fuctio ito R` ad a kow fuctio 2 so tat g ca be regarded as a fuctio from Z R` ito R m. Te added geerality is useful to adle applicatios were is a coditioal expectatio of a ukow variable wic eeds to be estimated, for example. Te geerality is also useful i applicatios were idividuals decisios deped o te etire distributio of a variable, wic i tur is estimated. Tis is te case for idividual decisios i auctio models, for example, or more geerally ay decisio uder explicitly stated expectatio wic is to be estimated. Let G (; ) = g i (; ) :

4 Tis paper cosiders a set of su ciet coditios wic imply asymptotic ormality of a ite dimesioal compoet at te rate te square root of te sample size i a class of te geeralized metod of momet (GMM) estimator wic is de ed as a solutio to te followig problem: if G (; ^ ) T ^AG (; ^ ) 2 were ^ is a estimator of ad ^A is a m m matrix wic coverges i probability to a positive de ite matrix A. 3 Asymptotic Distributio Our approac is a direct applicatio of te stadard aalysis of te GMM estimators. Like te stadard aalysis, te basic result appeals to te Taylor s series expasio. Let B be a Baac space equipped wit orm kk B ad let kk be a orm o R K. First we state a Taylor s series expasio teorem for a geeral mappig F from a ope subset of space B ito R K. To state tis teorem, we rst eed to de e te cocept of Frécet di eretiability of a mappig from a ope subset O of a ormed space X ito aoter ormed space Y. Let kk X ad kk Y be te orms of X ad Y, respectively. De itio (Frécet Di eretiability) A mappig f : O! Y is Frécet di eretiable if ad oly if at x 2 O tere is a cotiuous liear operator L x suc tat for ay " > 0 tere exists " > 0 suc tat for ay kk X < " te followig iequality olds: kf (x + ) f (x) L x k Y " kk X : We write tis as f (x + ) f (x) L x = o () : Next we discuss secod order di eretiability of mappig f or di eretiability of L x. Note tat L x ca be regarded as a mappig from X ito L (X; Y ), a space of liear operators from X ito Y. Because X is a ormed space ad L (X; Y ) is a ormed space, we ca discuss Frécet di eretiability of tis mappig L x we f is Frécet di eretiable over O. 2 We L x See for example Kolmogorov ad Fomi (957), Fourt editio Capter 0. 2 For ay L 2 L (X; Y ) te orm of L (X; Y ) is de ed by sup klk Y : kk X 2

5 is Frécet di eretiable, we ave L x+ L x Q x = o () for some liear operator Q x. We regard tis liear operator as te secod derivative of f. Note tat Q x is a elemet of L (X; L (X; Y )). Because L (X; L (X; Y )) ca be ideti ed wit a space of biliear operators B X 2 ; Y via B (x ; x 2 ) = A (x ) x 2 ; were A 2 L (X; L (X; Y )) ad B 2 B X 2 ; Y, we will regard Q x as a elemet of B X 2 ; Y. Aalogously oe ca de e te t order derivative of mappig f ad will regard tem as a elemet of te space of te rt order liear operators R (X r ; Y ). From ow o, we will deote te derivative of f (x) by f 0 (x), te secod derivative by f 00 (x), te rt derivative by f (r) (x). As discussed above, for eac x, f 0 (x) is a liear operator, f 00 (x) is a biliear operator, ad i geeral f (r) (x) is te rt order liear operator ito Y. Tus for ay elemet 2 X, f 0 (x) (), f 00 (x) (; ), ad i geeral f (r) (x) (; :::; ) are all well de ed ad take values i Y. Usig tese otatios, we ca state te Taylor s series expasio teorem: See Kolmogorov ad Fomi (976). 3 Teorem 2 (Taylor s Series Expasio) Let F be a mappig from B ito R K ad let F be de ed over a ope subset O of B. If F (r) (x) exists for ay x 2 O ad is uiformly cotiuous, te F (x + ) = F (x)+f 0 (x) ()+ 2! F 00 (x) (; )+ + r! F (r) (x) (; :::; )+! (x; ) were k! (x; )k B = o (kk r B ). If te rt derivative satis es te Lipscitz coditio wit expoet > 0, te k! (x; )k B = O kk r+ B : We prove asymptotic distributio of te semiparametric GMM estimator uder te followig assumptios. Coditio 3 fz i g are idepedet ad idetically distributed. Coditio 4 ( 0 ; 0 (; ) is a iterior poit of f(; )g 2. 3 Capter 0, Teorem 2. 3

6 Coditio 5 g (z; ; ) is Frécet di eretiable wit respect to (; ) i ad te Frécet derivatives satis es te Lipscitz cotiuity coditios: for C j (z) > 0 E fc j (z)g < (j = ; 2; 3; (z; ; ) z; 0 ; 0 =@ R mp C (z) 0 R p + C 2 (z) (z; ; ) z; 0 ; 0 =@ L( ;L( ;R m )) C 3 (z) 0 R p + C 4 (z) 0 We sall deote te Frécet derivative wit respect to iclusive of te e ect of o by rg (z; ; (; )). Note tat rg (z; ; (; )) (z; ; (; )) =@+@g (z; ; (; )) (; ) =@: Coditio 6 sup 2 k@g (z; ; (; )) =@k L( ;L( ;R m )) +sup 2 k@g (z; ; (; )) =@k R mp C 0 (z) ad E fc 0 (z)g <. Coditio 7 E frg (z; 0 ; 0 (; )g rg is ite ad as full rak. Coditio 8 plim! ^A = A were A is symmetric ad positive de ite. Coditio 9 7! 0 (; ) as a mappig from ito 0. is cotiuous at We de e te cocept of asymptotic liearity. Let deote a sample size ad fr g be a determiistic sequece wic coverges to 0. We cosider a geeral estimator ^ of a elemet 0 i a Baac space B wit orm kk B. De itio 0 A statistic ^ i B is asymptotically liear for 0 i B wit te residual rate r if tere exist a stocastic sequece f i g wit i 2 B ad E ( i ) = 0 ad a determiistic sequece fb g wit b 2 B suc tat ^ 0 i b = o p (r ) : B I our applicatio, for eac i, typically i is a fuctio of some argumets. Coditio sup 2 k^ (; ) 0 (; )k = o p () ad tat ^ (; ) is asymptotically liear for 0 (; ) i wit rate =2. We impose te followig coditio as well: 4

7 Coditio 2 plim! 3=2 i ( 0 ; =@ 0 i = 0. Coditio 3 plim! =2 i ( 0 ; =@ 0 b = g b. Typically we will d coditios uder wic g b = 0. Uder te coditios, te term ca be bouded: i ( 0 ; =@ 0 b R C 0 (z i ) p kb k : m Tus if p kb k = o (), te g b = 0. We also assume tat te oparametric estimator is smoot ad te derivative beaves as expected. Coditio 4 ^ (; ) is cotiuously di eretiable ad sup 2 k@^ (; ) 0 (; = o p (). For asymptotic ormality, te two coditios above are eeded oly i te eigborood of 0. For cosistecy, owever, tat is ot eoug, but peraps ot as strog as te coditio above. Let i = i E f i g. Coditio 5 E f i g 2 o =4. ad tat P i + ke f i gk = Let H = (rg) T A (rg) ad deote te expectatio coditioal o z i by E fjz i g. Let = V ar g (z ; 0 ; + (z ; 0 ; =@ 0 2jz + (z 2 ; 0 ; =@ 0 jz ad for ay c 2 R p, c = lim! c 0 H (rg) T A ArGH c. Teorem 6 Suppose ^ is cosistet to 0. Uder te coditios above, for ay c 2 R p for wic c is positive ad ite, p c ^ 0 0 coverges i distributio to a ormal radom variable wit mea 0 ad variace c. Te proof makes use of te followig two lemmas. rg ( 0 ; 0 (; ) = o p () i te eigbor- Lemma 7 rg (; (; )) ood 0 ad 0 (;. 5

8 Proof. To see tis, just ote tat krg (; (; )) rg ( 0 ; 0 (; )k R mp krg (; (; )) rg ( 0 ; (; ))k R mp + krg ( 0 ; (; )) rg ( 0 ; 0 (; )k R mp [C (z i ) k 0 k R p + C 3 (z i ) k@ (; ) =@k k 0 k R p] + C 2 (z i ) [k (; ) 0 (; )k + k 0 (; ) 0 (; k ] + C 4 (z i ) [k (; ) 0 (; )k + k 0 (; ) 0 (; k ] k@ (; ) =@k + C 0 (z i ) [k@ (; ) 0 (; + k@ 0 (; ) 0 ] Tis implies te result. Lemma 8 p G ( 0 ; ^ (; ) is asymptotically equivalet (zi ; p 0 ; g (z i ; 0 ; ( ) + (zj ; 0 0 j jz i + 0 i jz i Proof. By te Taylor s series expasio teorem for some R G ( 0 ; ^ (; ) = G ( 0 ; 0 (; ) i ( 0 ; 0 (; 0 (^ (; 0 (; ) + R : We te Frécet derivative satis es te Lipscitz coditio wit expoet, te last term ca be bouded: jr j C 4 (z i ) k^ (; (; k 2 : Tus p jr j coverges to zero uder te coditios. Usig te asymptotic liearity of te oparametric estimator ^, te rst term of te rigt-ad side equals i ( 0 ; j= j + b A 6 +g b

9 wic, by exploitig te liearity of te Frécet derivative equals j= i ( 0 0 i ( 0 0 i ( 0 0 b : Te secod term coverges to zero after multiplied by p uder te coditio. Te tird term multiplied by p coverges to g b uder te coditio. By te U-statistics cetral limit teorem, te rst term coverges wit te rate te square root of te sample size. To obtai te asymptotic variace formula, we compute te projectio: First by symmetrizatio we ave ( ) = times 2 ( ) j>i Te projectio is (for j 6= i) 2 (zi ; 0 0 j (z j; 0 0 i (zi ; 0 ; E 0 j (z j; 0 0 (zi ; 0 0 j jz i + E =2jz (zj ; 0 0 i jz i : Tus combiig wit te rst term, we obtai te result. Tus te asymptotic distributio is drive (z ; 0 ; = V ar g (z ; 0 ; 0 (; ) + (z2 ; jz + 0 jz : Now we tur to a proof of te mai teorem. Proof. Let rg (; ^ ) (; ^ ) =@ (; ^ ) () =@ (; ) =@ deotes te Frécet derivative of G (; ) wit respect usig te orm kk. Te te rst order coditio solves 0 = rg ^; ^ ; ^ i T ^AG ^; ^ ; ^ : 7

10 We cosider te expasio of G ^; ^ ; ^ at = 0 : By te stadard Taylor s series expasio teorem, G ^; ^ ; ^ = G ( 0 ; ^ (; ) + rg ( 0 ; ^ (; ) ^ 0 + rg ; ^ ; rg ( 0 ; ^ (; ) ^ 0 for some wic lies o a lie coectig ^ ad 0. After substitutio tis expressio ito te rst order coditio ad rearragig, we ave [rg ( 0 ; (; )] T ^A [rg ( 0 ; ^ (; )] ^ 0 = [rg ( 0 ; (; )] T ^AG ( 0 ; ^ (; ) + T + T2 + T3 were T = T2 = T3 = rg ^; ^ ; ^ T rg ( 0 ; 0 (; )i ^AG ( 0 ; ^ (; ) ; rg ^; ^ ; ^ T rg ( 0 ; 0 (; )i ArG ( 0 ; ^ (; ) ^ rg ^; ^ ; ^ i T A rg ; ^ ; rg ( 0 ; ^ (; ) ^ 0 ; ad 0 : Uder te coditio, clearly rg ( 0 ; 0 (; ) coverges to a full rak matrix rg. Te limit ( 0 ; rg = E 0 (; ) i ( 0 ; 0 0 (; Note tat equals ^ 0 o = rg ( 0 ; (; ) T ^ArG ( 0 ; ^ (; ) i rg ( 0 ; (; ) T ^AG ( 0 ; ^ (; ) + T + T2 + T3 Earlier lemma implies rg ^; ^ ; ^ rg ( 0 ; 0 (; ) = o p () ; rg ; ^ ; rg ( 0 ; ^ (; ) = o p () ; ad G ( 0 ; ^ (; ) rg ( 0 ; 0 (; ) = o p () : Tese also imply tat rg ( 0 ; (; ) T ^ArG ( 0 ; ^ (; ) coverges i probability to a ivertible matrix. 8

11 Multiply bot sides by p = + p R ^ 0 ad take te orm of p te left-ad side. Suppose p R ^ 0 diverges wit positive probability. Te wit te positive probability, te left-ad side coverges to. p Te rigt-ad side owever coverges to zero from te earlier lemma. Tus p R ^ 0 = O p (). p Next multiply bot sides wit p c 2 R p ad applyig te coditio implies te result. Tese calculatios clarify wat we eed to kow to compute te asymptotic distributio of a semiparametric GMM estimator. Tey are rg ad. 4 Applicatios To carry out tese computatios, we eed to d out te relevat Frécet derivatives ad kow wat te asymptotic liear expressios are for te oparametric estimators used i te estimatio. For te kerel desity estimators te followig are te expressios: i = d K zi z E d K zi z ad b = E d K zi z f (z) : To cotrol te bias, so tat te asymptotic liearity coditio olds wit rate =2, a certai type of kerel fuctio eeds to be used. Te followig iger order kerel by Bartlett (963) is a stadard device i te literature. Let j0 = if j = 0 ad 0 for ay oter iteger value j. De itio 9 K`, ` is te class of symmetric fuctios k : R! R aroud zero suc tat R tj k (t) dt = j0 for j = 0; ; :::; ` ad for some " > 0 lim k (t) = + jtj`++" < : jtj! Dimesio d kerel fuctio K of order ` is costructed by K (t ; :::; t d ) = k (t ) k (t d ) for k 2 K`. I order to improve te order of bias by te iger order kerel, te uderlyig desity is required to be smoot accordigly. Te followig otio of smootess is used by Robiso (988). Let [] deote te largest iteger ot equal or larger ta. 9

12 De itio 20 G, > 0, > 0, is te class of fuctios g : R d! R satisfyig: g is []-times partially di eretiable for all z 2 R d ; for some > 0, sup y2fky zkr d <g jg (y) g (z) Q (y; z)j = ky zk R d (z) for all z; Q = 0 we [] = 0; Q is a []-t degree omogeeous polyomial i (y z) wit coe ciets te partial derivatives of g at z of orders troug [] we [] ; ad g (z), its partial derivatives of order [] ad less, ad (z) ave ite t momets. Bouded fuctios are deoted by G. Let K be a iger order kerel costructed as above. Robiso (988) as sow te followig result: E Lemma 2 (Robiso) E d K ((z 2 we f 2 G for some > 0 ad k 2 K []+. z ) =) jz f (z ) 2 o = O 2 Lemma 22 (Robiso) E (g (z2 ) g (z )) d K ((z 2 z ) =) = O mi(;+;+) we f 2 G, g 2 G, ad k 2 K []+[]+. Tese results are useful to examie estimators we rg ad g are liear i. Usig tese results we will examie various examples. Te followig estimator ^ of E ffg is examied by Amad (976): Example 23 0 = i ^f (zi ) : I tis applicatio g (z; ; ) = (z). Tree aspects of tis applicatio makes it particularly easy to directly verify te coclusios of te lemmas: tat does ot deped o, rg (z; ; ) =, ad tat g (z; ; ) is liear i. Te rst two imply tat te coclusio of te rst lemma olds witout ay furter assumptios. Te tird implies tat tere is o approximatio error to be cocered, so we just eed to (z ; 0 ; V ar g (z ; 0 ; 0 (; ) + (z2 ; jz + 0 jz : Frécet derivative wit respect to ca be directly computed as mius te liear mappig from ito R wic evaluates a give fuctio at a poit g 0

13 is evaluated so (z ; (z 2 ; 0 0 = (z ; 0 ; 0 2 (z2 ; 0 ; 0 jz d K d K = 0 = E z2 z z z 2 d K + E + E z z 2 (z ; 0 ; V ar g (z ; 0 ; 0 (; ) jz = V ar 0 0 (z ) E d K z z 2 jz + E! 4E [ 0 0 (z )] 2o : d K z2 z jz d K z z 2 jz 2 : jz + E d K z z (z2 ; 0 ; + E z z 2 d K Also, Robiso s result allows us to d coditios uder wic g b = 0. Aoter example is te partial liear regressio model of Cosslett (984), Sciller (984) ad Waba (984). Example 24 For x 2 R K, y 2 R, w 2 R d te model is y = x T 0 + (w) + " were E ("jw; x) = 0. equatios: 0 = Cosider a estimator wic solves te followig y i x 0 i^ ^E (yjwi ) + ^E x 0 jw i ^i ^Ii x i were ^I i = I ^f (wi ) > b ad I is te idicator fuctio. Te followig lemma is useful. Let I i = I (f (w i ) > b). Lemma 25 Pr at least oe of ^I i I i 6= 0! 0 we f 2 G, for some > 0, k 2 K []+, jk (0)j <, b is positive ad bouded, d b 2 = log!, b=!, ad we tere is o positive probability tat f (w i ) = 0 jz

14 Note tat b is ot ecessarily required to coverge to zero. Tis result allows us to cosider 0 = y i x 0 i^ ^E (yjwi ) + ^E x 0 i jw i ^ I i x i istead of te feasible GMM. Proof. Te probability is bouded by P o ^Ii Pr I i 6= 0. Note tat o Pr ^I I 6= 0 oo = E Pr ^I I 6= 0jw = E Pr ^f (w ) > bjw o ( I ) o o o + E Pr ^f (w ) < bjw I : Let ~ b = b d K (0) [( ) =] E d K ((w 2 w ) =) jw i : Te by Berstei s iequality for some positive umbers C ad C 2, o Pr ^f (w ) > bjw ( I ) ( = Pr d X ) wi w wi w K E K jw > ~ b jw ( I ) i=2 ( ) d ~ b 2 exp I (f (w C + C 2 ~ ) < b) b ad tat o Pr ^f (w ) < bjw I ( = Pr d X ) wi w wi w K + E K jw > ~ b jw I i=2 ( ) d ~ b 2 exp I (f (w C 2 ~ ) > b) : b C Te a applicatio of Lebesgue domiatig covergece teorem implies tat P o ^Ii Pr I i 6= 0 coverges to zero we all te rates coditios old. 2

15 For te kerel regressio estimators of g (x) = E (Y jx = x), deotig " = Y g (X), te liear approximatio of (^g g) I ^f > b takes te followig form: i = " i d K ((x i x) =) I (f (x) > b) ad f (x) b = E I (f (x) > b) (g (x i) g (x)) xi x K Let z = (w; x; y). I tis example, g (z; ; ) = I x so tat d rg (z; ; ) = I x [x 2 (w)] T : =f (x) : y x T (w) 2 (w) T Sice rg (z; ; ) is liear i ad does ot deped o, te direct veri catio of te lemma is easier. Oe ca verify o o rg = E I x [x E (xjw)] T! E x [x E (xjw)] T we b! 0 x we E [x E (xjw)] T K2o <. To examie te asymptotic distributio ote tat g (z; ; ) is liear i so tat direct calculatio is simpler. Te Frécet derivative of g wit respect to () = (w) 2 (w) T 0 x so tat writig u = y E (yjw) ad v = x E (xjw) ad " = y x T 0 (z ; (z 2 ; 0 0 = u 2 v2 T 0 d K ((w 2 w ) =) I (f (w ) > b) x f (w ) u v T 0 d K ((w w 2 ) =) I (f (w 2 ) > b) x 2 : f (w 2 ) Tus otig tat u = y E (xjw) T 0 (w) = " + v T (z ; 0 ; 0 2 jz = (z2 ; 0 ; E d K ((w w 2 ) 0 jz = " E I (f (w 2 ) > b) x 2 jw f (w 2 ) so (z ; 0 ; g (z ; 0 ; 0 (; ) I + E V 0 2 jz = V ar " d K ((w w 2 ) =) x I E I (f (w 2 ) > b) x 2 jw f (w 2 )! V ar [" [x E (x jw )]] as b! 0: 3 (z2 ; 0 ; + 0 jz

16 We ext cosider trimmig fuctio suitable to adle idex models. I tis part of te paper let ^I i = I if ^f w2br(wi ) (w) > b ad I i = I if w2br(wi ) f (w) > b Te followig lemma is useful. Lemma 26 Pr at least oe of ^I i I i 6= 0! 0 we f 2 G, for some > 0, k 2 K []+, jk (0)j <, b is positive ad bouded, d b 2 = log!, b=!, ad we tere is o positive probability tat f (w i ) = b. Proof. Te probability is bouded by P o ^Ii Pr I i 6= 0. Note tat o Pr ^I I 6= 0 oo = E Pr ^I I 6= 0jw = E Pr if ^f (w) > bjw ( I ) + E Pr if ^f (w) < bjw I : w2b r(w ) w2b r(w ) Let ~ b = b d K (0) E d K ((w 2 w ) =) jw. Te by Berstei s iequality for some positive umbers C ad C 2, o Pr ^f (w ) > bjw ( I ) ( = Pr d X ) wi w wi w K E K jw > ~ b jw ( I ) i=2 ( ) d ~ b 2 exp I (f (w C + C 2 ~ ) < b) b ad tat o Pr ^f (w ) < bjw I ( = Pr d X ) wi w wi w K + E K jw > ~ b jw I i=2 ( ) d ~ b 2 exp I (f (w C 2 ~ ) > b) : b C Te a applicatio of Lebesgue domiatig covergece teorem implies tat P o ^Ii Pr I i 6= 0 coverges to zero we all te rates coditios old. 4

17 Refereces Ai, C. ad X. Ce (2003) E ciet Estimatio of Models wit Coditioal Momet Restrictios Cotaiig Ukow Fuctios, Ecoometrica, 7, Amad, I. A. (976) O Asymptotic Properties Of A Estimate Of A Fuctioal OF A Probability Desity, Scadiavia Actuarial Joural, Adrews, D. (994) Asymptotics for Semiparametric Ecoometric Models via Stocastic Equicotiuity, Ecoometrica, 62, Bartlett, M. S. (963) Statistical Estimatio Of Desity Fuctios, Sakya, Series A, 25, Cosslett, S. J. (984) Distributio Free Estimator Of A Regressio Model Wit Sample Selectivity, mauscript. Kolmogorov, A. H. ad C. B. Fomi (976) Teory Of Fuctios Ad Foudatios Of Fuctioal Aalysis 4t Editio Moskow (Japaese Traslatio 979). Newey, W.K. (994) Te Asymptotic Variace of Semiparametric Estimators, Ecoometrica, 62, Serma, R. (994) U-processes i te aalysis of a geeralized semiparametric regressio estimator, Ecoometric Teory, 0, Robiso, P. M. (988) Root-N-Cosistet Semiparametric Regressio, Ecoometrica 56, Sciller, R. J. (984) Smootess Priors ad Noliear Regressio, Joural of te America Statistical Associatio, 72, Waba, G. (984) Partial Splie Models for te Semi-Parametric Estimatio of Fuctios of Several Variables, i Statistical Aalysis of Time Series. Tokyo: Istitute of Statistical Matematics,

Lecture 7 Testing Nonlinear Inequality Restrictions 1

Lecture 7 Testing Nonlinear Inequality Restrictions 1 Eco 75 Lecture 7 Testig Noliear Iequality Restrictios I Lecture 6, we discussed te testig problems were te ull ypotesis is de ed by oliear equality restrictios: H : ( ) = versus H : ( ) 6= : () We sowed

More information

Study the bias (due to the nite dimensional approximation) and variance of the estimators

Study the bias (due to the nite dimensional approximation) and variance of the estimators 2 Series Methods 2. Geeral Approach A model has parameters (; ) where is ite-dimesioal ad is oparametric. (Sometimes, there is o :) We will focus o regressio. The fuctio is approximated by a series a ite

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator

Economics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters

More information

Nonparametric regression: minimax upper and lower bounds

Nonparametric regression: minimax upper and lower bounds Capter 4 Noparametric regressio: miimax upper ad lower bouds 4. Itroductio We cosider oe of te two te most classical o-parametric problems i tis example: estimatig a regressio fuctio o a subset of te real

More information

ON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS 1

ON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS 1 Teory of Stocastic Processes Vol2 28, o3-4, 2006, pp*-* SILVELYN ZWANZIG ON LOCAL LINEAR ESTIMATION IN NONPARAMETRIC ERRORS-IN-VARIABLES MODELS Local liear metods are applied to a oparametric regressio

More information

4 Conditional Distribution Estimation

4 Conditional Distribution Estimation 4 Coditioal Distributio Estimatio 4. Estimators Te coditioal distributio (CDF) of y i give X i = x is F (y j x) = P (y i y j X i = x) = E ( (y i y) j X i = x) : Tis is te coditioal mea of te radom variable

More information

Uniformly Consistency of the Cauchy-Transformation Kernel Density Estimation Underlying Strong Mixing

Uniformly Consistency of the Cauchy-Transformation Kernel Density Estimation Underlying Strong Mixing Appl. Mat. If. Sci. 7, No. L, 5-9 (203) 5 Applied Matematics & Iformatio Scieces A Iteratioal Joural c 203 NSP Natural Scieces Publisig Cor. Uiformly Cosistecy of te Caucy-Trasformatio Kerel Desity Estimatio

More information

LECTURE 2 LEAST SQUARES CROSS-VALIDATION FOR KERNEL DENSITY ESTIMATION

LECTURE 2 LEAST SQUARES CROSS-VALIDATION FOR KERNEL DENSITY ESTIMATION Jauary 3 07 LECTURE LEAST SQUARES CROSS-VALIDATION FOR ERNEL DENSITY ESTIMATION Noparametric kerel estimatio is extremely sesitive to te coice of badwidt as larger values of result i averagig over more

More information

Regression with an Evaporating Logarithmic Trend

Regression with an Evaporating Logarithmic Trend Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,

More information

Notes On Nonparametric Density Estimation. James L. Powell Department of Economics University of California, Berkeley

Notes On Nonparametric Density Estimation. James L. Powell Department of Economics University of California, Berkeley Notes O Noparametric Desity Estimatio James L Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Uivariate Desity Estimatio via Numerical Derivatives Cosider te problem of estimatig te desity

More information

Semiparametric Mixtures of Nonparametric Regressions

Semiparametric Mixtures of Nonparametric Regressions Semiparametric Mixtures of Noparametric Regressios Sijia Xiag, Weixi Yao Proofs I tis sectio, te coditios required by Teorems, 2, 3 ad 4 are listed. Tey are ot te weakest sufficiet coditios, but could

More information

Computation Of Asymptotic Distribution. For Semiparametric GMM Estimators. Hidehiko Ichimura. Graduate School of Public Policy

Computation Of Asymptotic Distribution. For Semiparametric GMM Estimators. Hidehiko Ichimura. Graduate School of Public Policy Computation Of Asymptotic Distribution For Semiparametric GMM Estimators Hidehiko Ichimura Graduate School of Public Policy and Graduate School of Economics University of Tokyo A Conference in honor of

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors

ECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic

More information

Kernel density estimator

Kernel density estimator Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I

More information

ALLOCATING SAMPLE TO STRATA PROPORTIONAL TO AGGREGATE MEASURE OF SIZE WITH BOTH UPPER AND LOWER BOUNDS ON THE NUMBER OF UNITS IN EACH STRATUM

ALLOCATING SAMPLE TO STRATA PROPORTIONAL TO AGGREGATE MEASURE OF SIZE WITH BOTH UPPER AND LOWER BOUNDS ON THE NUMBER OF UNITS IN EACH STRATUM ALLOCATING SAPLE TO STRATA PROPORTIONAL TO AGGREGATE EASURE OF SIZE WIT BOT UPPER AND LOWER BOUNDS ON TE NUBER OF UNITS IN EAC STRATU Lawrece R. Erst ad Cristoper J. Guciardo Erst_L@bls.gov, Guciardo_C@bls.gov

More information

SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker

SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker CHAPTER 9. POINT ESTIMATION 9. Covergece i Probability. The bases of poit estimatio have already bee laid out i previous chapters. I chapter 5

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Week 10. f2 j=2 2 j k ; j; k 2 Zg is an orthonormal basis for L 2 (R). This function is called mother wavelet, which can be often constructed

Week 10. f2 j=2 2 j k ; j; k 2 Zg is an orthonormal basis for L 2 (R). This function is called mother wavelet, which can be often constructed Wee 0 A Itroductio to Wavelet regressio. De itio: Wavelet is a fuctio such that f j= j ; j; Zg is a orthoormal basis for L (R). This fuctio is called mother wavelet, which ca be ofte costructed from father

More information

1 Covariance Estimation

1 Covariance Estimation Eco 75 Lecture 5 Covariace Estimatio ad Optimal Weightig Matrices I this lecture, we cosider estimatio of the asymptotic covariace matrix B B of the extremum estimator b : Covariace Estimatio Lemma 4.

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

LECTURE 11 LINEAR PROCESSES III: ASYMPTOTIC RESULTS

LECTURE 11 LINEAR PROCESSES III: ASYMPTOTIC RESULTS PRIL 7, 9 where LECTURE LINER PROCESSES III: SYMPTOTIC RESULTS (Phillips ad Solo (99) ad Phillips Lecture Notes o Statioary ad Nostatioary Time Series) I this lecture, we discuss the LLN ad CLT for a liear

More information

LECTURE 8: ASYMPTOTICS I

LECTURE 8: ASYMPTOTICS I LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece

More information

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials

More information

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f. Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

McGill University Math 354: Honors Analysis 3 Fall 2012 Solutions to selected problems

McGill University Math 354: Honors Analysis 3 Fall 2012 Solutions to selected problems McGill Uiversity Math 354: Hoors Aalysis 3 Fall 212 Assigmet 3 Solutios to selected problems Problem 1. Lipschitz fuctios. Let Lip K be the set of all fuctios cotiuous fuctios o [, 1] satisfyig a Lipschitz

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Notes On Median and Quantile Regression. James L. Powell Department of Economics University of California, Berkeley

Notes On Median and Quantile Regression. James L. Powell Department of Economics University of California, Berkeley Notes O Media ad Quatile Regressio James L. Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Coditioal Media Restrictios ad Least Absolute Deviatios It is well-kow that the expected value

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

x x x 2x x N ( ) p NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS By Newton-Raphson formula

x x x 2x x N ( ) p NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS By Newton-Raphson formula NUMERICAL METHODS UNIT-I-SOLUTION OF EQUATIONS AND EIGENVALUE PROBLEMS. If g( is cotiuous i [a,b], te uder wat coditio te iterative (or iteratio metod = g( as a uique solutio i [a,b]? '( i [a,b].. Wat

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

Math Solutions to homework 6

Math Solutions to homework 6 Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there

More information

Lecture 9: Regression: Regressogram and Kernel Regression

Lecture 9: Regression: Regressogram and Kernel Regression STAT 425: Itroductio to Noparametric Statistics Witer 208 Lecture 9: Regressio: Regressogram ad erel Regressio Istructor: Ye-Ci Ce Referece: Capter 5 of All of oparametric statistics 9 Itroductio Let X,

More information

Bull. Korean Math. Soc. 36 (1999), No. 3, pp. 451{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Seung Hoe Choi and Hae Kyung

Bull. Korean Math. Soc. 36 (1999), No. 3, pp. 451{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Seung Hoe Choi and Hae Kyung Bull. Korea Math. Soc. 36 (999), No. 3, pp. 45{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Abstract. This paper provides suciet coditios which esure the strog cosistecy of regressio

More information

Stability analysis of numerical methods for stochastic systems with additive noise

Stability analysis of numerical methods for stochastic systems with additive noise Stability aalysis of umerical metods for stoctic systems wit additive oise Yosiiro SAITO Abstract Stoctic differetial equatios (SDEs) represet pysical peomea domiated by stoctic processes As for determiistic

More information

Supplemental Material: Proofs

Supplemental Material: Proofs Proof to Theorem Supplemetal Material: Proofs Proof. Let be the miimal umber of traiig items to esure a uique solutio θ. First cosider the case. It happes if ad oly if θ ad Rak(A) d, which is a special

More information

Week 6. Intuition: Let p (x, y) be the joint density of (X, Y ) and p (x) be the marginal density of X. Then. h dy =

Week 6. Intuition: Let p (x, y) be the joint density of (X, Y ) and p (x) be the marginal density of X. Then. h dy = Week 6 Lecture Kerel regressio ad miimax rates Model: Observe Y i = f (X i ) + ξ i were ξ i, i =,,,, iid wit Eξ i = 0 We ofte assume X i are iid or X i = i/ Nadaraya Watso estimator Let w i (x) = K ( X

More information

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

More information

Estimating the Population Mean using Stratified Double Ranked Set Sample

Estimating the Population Mean using Stratified Double Ranked Set Sample Estimatig te Populatio Mea usig Stratified Double Raked Set Sample Mamoud Syam * Kamarulzama Ibraim Amer Ibraim Al-Omari Qatar Uiversity Foudatio Program Departmet of Mat ad Computer P.O.Box (7) Doa State

More information

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4. 4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad

More information

A remark on p-summing norms of operators

A remark on p-summing norms of operators A remark o p-summig orms of operators Artem Zvavitch Abstract. I this paper we improve a result of W. B. Johso ad G. Schechtma by provig that the p-summig orm of ay operator with -dimesioal domai ca be

More information

Random Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices

Random Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

Lecture 33: Bootstrap

Lecture 33: Bootstrap Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece

More information

Estimation of the essential supremum of a regression function

Estimation of the essential supremum of a regression function Estimatio of the essetial supremum of a regressio fuctio Michael ohler, Adam rzyżak 2, ad Harro Walk 3 Fachbereich Mathematik, Techische Uiversität Darmstadt, Schlossgartestr. 7, 64289 Darmstadt, Germay,

More information

Lecture 27: Optimal Estimators and Functional Delta Method

Lecture 27: Optimal Estimators and Functional Delta Method Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

11 THE GMM ESTIMATION

11 THE GMM ESTIMATION Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................

More information

Lecture 24: Variable selection in linear models

Lecture 24: Variable selection in linear models Lecture 24: Variable selectio i liear models Cosider liear model X = Z β + ε, β R p ad Varε = σ 2 I. Like the LSE, the ridge regressio estimator does ot give 0 estimate to a compoet of β eve if that compoet

More information

Asymptotic distribution of products of sums of independent random variables

Asymptotic distribution of products of sums of independent random variables Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege

More information

Solution to Chapter 2 Analytical Exercises

Solution to Chapter 2 Analytical Exercises Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =

More information

It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable.

It should be unbiased, or approximately unbiased. Variance of the variance estimator should be small. That is, the variance estimator is stable. Chapter 10 Variace Estimatio 10.1 Itroductio Variace estimatio is a importat practical problem i survey samplig. Variace estimates are used i two purposes. Oe is the aalytic purpose such as costructig

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo

More information

Basics of Probability Theory (for Theory of Computation courses)

Basics of Probability Theory (for Theory of Computation courses) Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

A survey on penalized empirical risk minimization Sara A. van de Geer

A survey on penalized empirical risk minimization Sara A. van de Geer A survey o pealized empirical risk miimizatio Sara A. va de Geer We address the questio how to choose the pealty i empirical risk miimizatio. Roughly speakig, this pealty should be a good boud for the

More information

Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution

Large Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,

More information

Assignment 5: Solutions

Assignment 5: Solutions McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece

More information

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting

Lecture 6 Chi Square Distribution (χ 2 ) and Least Squares Fitting Lecture 6 Chi Square Distributio (χ ) ad Least Squares Fittig Chi Square Distributio (χ ) Suppose: We have a set of measuremets {x 1, x, x }. We kow the true value of each x i (x t1, x t, x t ). We would

More information

Solution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1

Solution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1 Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity

More information

Sieve Estimators: Consistency and Rates of Convergence

Sieve Estimators: Consistency and Rates of Convergence EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes

More information

Advanced Stochastic Processes.

Advanced Stochastic Processes. Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.

More information

Summary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector

Summary and Discussion on Simultaneous Analysis of Lasso and Dantzig Selector Summary ad Discussio o Simultaeous Aalysis of Lasso ad Datzig Selector STAT732, Sprig 28 Duzhe Wag May 4, 28 Abstract This is a discussio o the work i Bickel, Ritov ad Tsybakov (29). We begi with a short

More information

A Pseudo Spline Methods for Solving an Initial Value Problem of Ordinary Differential Equation

A Pseudo Spline Methods for Solving an Initial Value Problem of Ordinary Differential Equation Joural of Matematics ad Statistics 4 (: 7-, 008 ISSN 549-3644 008 Sciece Publicatios A Pseudo Splie Metods for Solvig a Iitial Value Problem of Ordiary Differetial Equatio B.S. Ogudare ad G.E. Okeca Departmet

More information

EXPLICIT FORMULAE FOR THE BRENTI S POLYNOMIALS γ a1,...,a r

EXPLICIT FORMULAE FOR THE BRENTI S POLYNOMIALS γ a1,...,a r EXPLICIT FORMULAE FOR THE BRENTI S POLYNOMIALS γ,,a r Jia-yi Si Departmet of Matematics, East Cia Normal Uiversity, Sagai, 0006, P R Cia Astract We itroduce two kids of geeralized Catala umers a a, Z ad

More information

An Introduction to Asymptotic Theory

An Introduction to Asymptotic Theory A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu

More information

Conditional-Sum-of-Squares Estimation of Models for Stationary Time Series with Long Memory

Conditional-Sum-of-Squares Estimation of Models for Stationary Time Series with Long Memory Coditioal-Sum-of-Squares Estimatio of Models for Statioary Time Series with Log Memory.M. Robiso Lodo School of Ecoomics The Sutory Cetre Sutory ad Toyota Iteratioal Cetres for Ecoomics ad Related Disciplies

More information

Econometrica Supplementary Material

Econometrica Supplementary Material Ecoometrica Supplemetary Material SUPPLEMENT TO NONPARAMETRIC INSTRUMENTAL VARIABLES ESTIMATION OF A QUANTILE REGRESSION MODEL : MATHEMATICAL APPENDIX (Ecoometrica, Vol. 75, No. 4, July 27, 1191 128) BY

More information

MAT1026 Calculus II Basic Convergence Tests for Series

MAT1026 Calculus II Basic Convergence Tests for Series MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real

More information

Machine Learning Brett Bernstein

Machine Learning Brett Bernstein Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete

More information

PRELIM PROBLEM SOLUTIONS

PRELIM PROBLEM SOLUTIONS PRELIM PROBLEM SOLUTIONS THE GRAD STUDENTS + KEN Cotets. Complex Aalysis Practice Problems 2. 2. Real Aalysis Practice Problems 2. 4 3. Algebra Practice Problems 2. 8. Complex Aalysis Practice Problems

More information

Linear Elliptic PDE s Elliptic partial differential equations frequently arise out of conservation statements of the form

Linear Elliptic PDE s Elliptic partial differential equations frequently arise out of conservation statements of the form Liear Elliptic PDE s Elliptic partial differetial equatios frequetly arise out of coservatio statemets of the form B F d B Sdx B cotaied i bouded ope set U R. Here F, S deote respectively, the flux desity

More information

This section is optional.

This section is optional. 4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore

More information

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002 ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom

More information

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we

More information

32 estimating the cumulative distribution function

32 estimating the cumulative distribution function 32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio

More information

A Central Limit Theorem for Belief Functions

A Central Limit Theorem for Belief Functions A Cetral Limit Theorem for Belief Fuctios Larry G. Estei Kyougwo Seo November 7, 2. CLT for Belief Fuctios The urose of this Note is to rove a form of CLT (Theorem.4) that is used i Estei ad Seo (2). More

More information

Asymptotic Results for the Linear Regression Model

Asymptotic Results for the Linear Regression Model Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is

More information

Equivalence Between An Approximate Version Of Brouwer s Fixed Point Theorem And Sperner s Lemma: A Constructive Analysis

Equivalence Between An Approximate Version Of Brouwer s Fixed Point Theorem And Sperner s Lemma: A Constructive Analysis Applied Mathematics E-Notes, 11(2011), 238 243 c ISSN 1607-2510 Available free at mirror sites of http://www.math.thu.edu.tw/ame/ Equivalece Betwee A Approximate Versio Of Brouwer s Fixed Poit Theorem

More information

Computation of Hahn Moments for Large Size Images

Computation of Hahn Moments for Large Size Images Joural of Computer Sciece 6 (9): 37-4, ISSN 549-3636 Sciece Publicatios Computatio of Ha Momets for Large Size Images A. Vekataramaa ad P. Aat Raj Departmet of Electroics ad Commuicatio Egieerig, Quli

More information

MOMENT-METHOD ESTIMATION BASED ON CENSORED SAMPLE

MOMENT-METHOD ESTIMATION BASED ON CENSORED SAMPLE Vol. 8 o. Joural of Systems Sciece ad Complexity Apr., 5 MOMET-METHOD ESTIMATIO BASED O CESORED SAMPLE I Zhogxi Departmet of Mathematics, East Chia Uiversity of Sciece ad Techology, Shaghai 37, Chia. Email:

More information

Law of the sum of Bernoulli random variables

Law of the sum of Bernoulli random variables Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible

More information

TRIGONOMETRIC POLYNOMIALS WITH MANY REAL ZEROS AND A LITTLEWOOD-TYPE PROBLEM. Peter Borwein and Tamás Erdélyi. 1. Introduction

TRIGONOMETRIC POLYNOMIALS WITH MANY REAL ZEROS AND A LITTLEWOOD-TYPE PROBLEM. Peter Borwein and Tamás Erdélyi. 1. Introduction TRIGONOMETRIC POLYNOMIALS WITH MANY REAL ZEROS AND A LITTLEWOOD-TYPE PROBLEM Peter Borwei ad Tamás Erdélyi Abstract. We examie the size of a real trigoometric polyomial of degree at most havig at least

More information

Properties and Hypothesis Testing

Properties and Hypothesis Testing Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.

More information

Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities

Chapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other

More information

Chapter 7 Isoperimetric problem

Chapter 7 Isoperimetric problem Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated

More information

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS

A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS J. Japa Statist. Soc. Vol. 41 No. 1 2011 67 73 A RANK STATISTIC FOR NON-PARAMETRIC K-SAMPLE AND CHANGE POINT PROBLEMS Yoichi Nishiyama* We cosider k-sample ad chage poit problems for idepedet data i a

More information

Lecture 3 : Random variables and their distributions

Lecture 3 : Random variables and their distributions Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}

More information

Empirical Processes: Glivenko Cantelli Theorems

Empirical Processes: Glivenko Cantelli Theorems Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3

More information

Problem Set 2 Solutions

Problem Set 2 Solutions CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S

More information

UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA

UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA Ecoometric Teory, 24, 2008, 726 748+ Prited i te Uited States of America+ doi: 10+10170S0266466608080304 UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA BRUCE E. HANSEN Uiversity of

More information

Application to Random Graphs

Application to Random Graphs A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let

More information

Singular Continuous Measures by Michael Pejic 5/14/10

Singular Continuous Measures by Michael Pejic 5/14/10 Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable

More information