Econometrica Supplementary Material
|
|
- Cora Carson
- 5 years ago
- Views:
Transcription
1 Ecoometrica Supplemetary Material SUPPLEMENT TO NONPARAMETRIC INSTRUMENTAL VARIABLES ESTIMATION OF A QUANTILE REGRESSION MODEL : MATHEMATICAL APPENDIX (Ecoometrica, Vol. 75, No. 4, July 27, ) BY JOEL L. HOROWITZ AND SOKBAE LEE THIS APPENDIX provides proofs of Theorems 1 3. Theorems 4 ad 5 ca be proved by followig the same steps after coditioig o Z. A.1. PROOF OF THEOREM 1 The proof is a modificatio of the proof of Theorem 2 i Bissatz, Hohage, ad Muk (24). By (2.5), (A1) ˆT (ĝ) q ˆ f W 2 + a ĝ 2 ˆT (g) q ˆ f W 2 + a g 2 I additio, E ˆT (g) T (g) 2 = O(δ ) ad E ˆ f W f W 2 = O(δ ). Therefore, by Assumptio 3, E ˆT (g) q ˆ f W 2 2E ˆT (g) T (g) 2 + 2qE ˆ f W f W 2 = O(δ ) Combiig this result with (A1) gives E ˆT (ĝ) q ˆ f W 2 + a E ĝ 2 Cδ + a g 2 for some costat C< ad all sufficietly large. Therefore, by Assumptio 3, lim sup E ĝ 2 g 2 Note, i additio, that E ˆT (ĝ) qf ˆ W 2 as. Moreover, Assumptios 1(b) ad 2 imply that T is weakly closed. Cosistecy ow follows from argumets idetical to those used to prove Theorem 2 of Bissatz, Hohage, ad Muk (24, p. 1777). Q.E.D. A.2. PROOF OF THEOREM 2 Assumptios 1, 2, ad 4 7 hold throughout this sectio. Let deote the ier product i L 2 [ 1].Defieω g = (T g T g ) 1 T g g ad (A2) g = g a (T g T g + a I) 1 T g ω g 1
2 2 J. L. HOROWITZ AND S. LEE where I is the idetity operator. Observe that by (3.4), (A3) L ω g < 1 Let ˆr ad r be Taylor series remaider terms with the properties that (A4) T (ĝ) = qf W + T g (ĝ g) + ˆr ad (A5) T ( g) = qf W + T g ( g g) + r where qf W = T (g). By (3.3), ˆr (L/2) ĝ g 2 ad r (L/2) g g 2. LEMMA A.1: For ay g G, (1 L ω g ) ĝ g 2 PROOF: By (A5), g g 2 + a 1 ˆT ( g) T ( g) + r + qf W qf ˆ W 2 + 2T g ( g g) + a ω g ω g +a 1 T g( g g) qf W qf ˆ W ω g +2a 1 r + qf W qf ˆ W T g ( g g) + 2 ˆT (ĝ) T (ĝ) ω g +2a 1 ˆT ( g) T ( g) T g ( g g) (A6) ˆT ( g) q ˆ f W 2 = ˆT ( g) T ( g) + r + qf W q ˆ f W 2 + T g ( g g) r + qf W q ˆ f W T g ( g g) +2 ˆT ( g) T ( g) T g ( g g) Also, (A7) ˆT (ĝ) q ˆ f W 2 = ˆT (ĝ) q ˆ f W + a ω g 2 + a 2 ω g 2 2a ˆT (ĝ) T (ĝ) ω g 2a T (ĝ) qf W ω g 2a qf W qf ˆ W ω g 2a 2 ω g 2 Moreover, (A8) g g g = g g T g ω g = T g ( g g) ω g By (A4), (A9) ĝ g g = ĝ g T g ω g
3 NONPARAMETRIC ESTIMATION OF REGRESSION MODEL 3 = T g (ĝ g) ω g = T (ĝ) qf W ω g ˆr ω g By (2.5), (A1) ˆT (ĝ) q ˆ f W 2 + a ĝ 2 ˆT ( g) q ˆ f W 2 + a g 2 Rearragig ad expadig terms i (A1) gives (A11) ĝ g 2 [ a 1 ˆT ( g) qf ˆ W 2 ˆT (ĝ) qf ˆ W 2] + g g g g g 2 ĝ g g Combiig (A11) with (A6) (A9) gives (A12) ĝ g 2 2 ˆr ω g a 1 ˆT (ĝ) qf ˆ W + a ω g 2 + g g 2 + a 1 ˆT ( g) T ( g) + r + qf W qf ˆ W 2 + 2T g ( g g) + a ω g ω g +a 1 T g( g g) qf W qf ˆ W ω g +2a 1 r + qf W qf ˆ W T g ( g g) + 2 ˆT (ĝ) T (ĝ) ω g +2a 1 ˆT ( g) T ( g) T g ( g g) The lemma follows by otig that the secod term o the right-had side of (A12) is opositive ad that 2 ˆr ω g L ω g ĝ g 2. Q.E.D. LEMMA A.2: For ay g G, (A13) (A14) (A15) a 1 ˆT ( g) T ( g) + r + qf W qf ˆ W 2 4a 1 ( ˆT ( g) T ( g) 2 + L2 4 g g 4 + qf W q ˆ f W 2 ) 2T g ( g g) + a ω g ω g +a 1 T g( g g) 2 = a 3 (T gt + a g I) 1 ω g 2 2 qf W qf ˆ W ω g +2a 1 r + qf W qf ˆ W T g ( g g) L ω g g g 2 + 2a qf W q ˆ f W (T g T g + a I) 1 ω g + La g g 2 (T g T g + a I) 1 ω g
4 4 J. L. HOROWITZ AND S. LEE ad (A16) 2 ˆT (ĝ) T (ĝ) ω g +2a 1 ˆT ( g) T ( g) T g ( g g) 2a ˆT ( g) T ( g) (T g T + a g I) 1 ω g + 2 ω g [ ˆT (ĝ) T (ĝ)] [ˆT (g) T (g)] + 2 ω g [ ˆT ( g) T ( g)] [ˆT (g) T (g)] PROOF: Iequality (A13) follows from (A5) ad the relatio A + B + C 2 4( A 2 + C 2 + C 2 ) for ay fuctios A, B,adC. To show (A14), ote that (A17) a (T g T g + a I) 1 = I T g (T g T g + a I) 1 T g By (A2), (A18) T( g g) = a T g (T g T g + a I) 1 T g ω g It follows from (A17) ad (A18) that (A19) a ω g + T( g g) = a 2 (T gt g + a I) 1 ω g Takig the squares of the orms of both sides of (A19) ad expadig the term o the left-had side yields (A2) a 2 ω g 2 + T g ( g g) a ω g T g ( g g) = a 4 (T gt g + a I) 1 ω g 2 The (A14) follows by dividig both sides of (A2) by a. We ow tur to (A15). First ote that (A21) r + qf W q ˆ f W T g ( g g) = r + qf W q ˆ f W T g ( g g) + a ω g r + qf W q ˆ f W a ω g It follows from (A19) ad (A21) that (A22) 2 qf W qf ˆ W ω g +2a 1 r + qf W qf ˆ W T g ( g g) + a ω g = 2a r + qf W q ˆ f W (T g T g + a I) 1 ω g 2 r ω g
5 NONPARAMETRIC ESTIMATION OF REGRESSION MODEL 5 The (A15) follows by applyig the Cauchy Schwarz ad triagle iequalities to (A22). Now we prove (A16). Observe that by (A19) ad algebra like that yieldig (A21), 2 ˆT (ĝ) T (ĝ) ω g +2a 1 ˆT ( g) T ( g) T g ( g g) = 2a ˆT ( g) T ( g) (T g T + a g I) 1 ω g + 2 [ ˆT (ĝ) T (ĝ)] [ˆT ( g) T ( g)] ω g which yields (A16) by the Cauchy Schwarz ad triagle iequalities. Q.E.D. LEMMA A.3: The followig relatios hold uiformly over H H. (a) g g 2 = O[ (2β 1)/(2β+α) ]; (b) a 1 g g 4 = O[ (2β 1)/(2β+α) ]; (c) a 3 (T gt + a g I) 1 ω g 2 = O[ (2β 1)/(2β+α) ]; (d) a g g 2 (T g T + a g I) 1 ω g =O[ (2β 1)/(2β+α) ]; (e) a qf W qf ˆ W (T g T + a g I) 1 ω g =O p [ (2β 1)/(2β+α) ]; (f) a ˆT ( g) T ( g) (T g T + a g I) 1 ω g =O p [ (2β 1)/(2β+α) ]; (g) there are radom variables = O p [ (β 1/2)/(2β+α) ] ad Ɣ = o p (1) such that [ ˆT (ĝ) T (ĝ)] [ˆT (g) T (g)] ĝ g +Ɣ ĝ g 2 ; (h) [ ˆT ( g) T ( g)] [ˆT (g) T (g)] = O p [ (2β 1)/(2β+α) ]. PROOF: To prove (a), ote that by (A2) ad T g ω g = g, Therefore, g(x) g(x) = a g g 2 = a 2 = a 1 λ j + a φ j (x) φ j T g ω g b j λ j + a φ j (x) b 2 j (λ j + a ) 2 = O[ (2β 1)/(2β+α) ] where the secod lie follows from argumets idetical to those used to prove equatio (6.4) of Hall ad Horowitz (25). This proves (a). It follows from (a) that a 1 g g 4 = O[ (2β 1)/(2β+α) ]
6 6 J. L. HOROWITZ AND S. LEE wheever α < 2β 1, thereby provig (b). We ow tur to (c). Defie ψ j = T g φ j / T g φ j.useω g = (T g T g ) 1 T g g ad the sigular value decompositio T ψ g j = λ 1/2 j φ j to obtai (T g T + a g I) 1 ω g = = = = 1 λ j (λ j + a ) ψ j ψ j T g g 1 λ j (λ j + a ) ψ j T g ψ j g 1 λ j (λ j + a ) ψ j λ 1/2 j φ j g b j λ 1/2 j (λ j + a ) ψ j Therefore, (A23) (T g T g + a I)ω g 2 = b 2 j λ j (λ j + a ) 2 = O[a (2β 3α 1)/α ] by argumets like those used to prove equatio (6.4) of Hall ad Horowitz (25). Therefore, (c) follows from a = C a α/(2β+α). To prove (d), ote that by (A23), (A24) a (T g T g + a I) 1 ω g =O[a (2β α 1)/(2α) ] = O[ (2β α 1)/(4β+2α) ] Therefore, (d) follows from (A24) ad (a), because α<2β 1. Now by Assumptios 2 ad 5(b), ˆ f W f W 2 = O p [ (2β 1+α)/(2β+α) ] Moreover, because T is a desity oly with respect to its w argumet ad ca be estimated with the oe-dimesioal oparametric rate of covergece, we have ˆT T 2 = O p [ (2β 1+α)/(2β+α) ] uiformly over H. Therefore, (e) ad (f) follow from (A24).
7 NONPARAMETRIC ESTIMATION OF REGRESSION MODEL 7 We ow tur to (g). By the mea value theorem, { [ ˆT (ĝ) T (ĝ)] [ˆT (g) T (g)] } (w) = { ˆ fyxw [ḡ(x) x w] f YXW [ḡ(x) x w] } [ĝ(x) g(x)] dx where ḡ is betwee ĝ ad g. The by the Cauchy Schwarz iequality, [ ˆT (ĝ) T (ĝ)] [ˆT (g) T (g)] 2 = ( { ˆ fyxw [ḡ(x) x w] f YXW [ḡ(x) x w] } [ĝ(x) g(x)] dx) 2 dw But = ( { fyxw ˆ [ḡ(x) x w] f YXW [ḡ(x) x w] }2 dx ) [ĝ(x) g(x)] 2 dx dw { ˆ fyxw [ḡ(x) x w] f YXW [ḡ(x) x w] }2 dxdw ĝ g 2 { fyxw ˆ [ḡ(x) x w] f YXW [ḡ(x) x w] }2 dxdw = O p [ (2β 1)/(2β+α) ]+ ĝ g 2 o p (1) by Assumptios 2 ad 5(b), thereby yieldig (g). Fially, (h) ca be proved by combiig (a) with argumets similar to those used to prove (g). The lemma is ow proved because the foregoig argumets hold uiformly over H H. Q.E.D. PROOF OF THEOREM 2: The theorem follows by combiig the results of Lemmas A.1 A.3 with L ω g < 1. Q.E.D. A.3. PROOF OF THEOREM 3 It suffices to fid a sequece of fiite-dimesioal models {g } H for which lim if P H[ g g 2 >D (2β 1)/(2β+α) ] >
8 8 J. L. HOROWITZ AND S. LEE To this ed, let m deote the iteger part of 1/(2β+α) ad let f XW deote the desity of (X W ).Letr w = (2β + α 1)/2. Assume that f XW (x w) C for all (x w) [ 1] 2 ad some costat C<.Let Y = g (X) + U where U is idepedet of (X W ) ad P(U ) = q. LetF U ad f U,respectively, deote the distributio fuctio ad desity of U. Assume that f U () > ad that F U is twice cotiuously differetiable everywhere with F (u) <M for all u ad some M<. Defie the operator Q o L U 2[ 1] by (Πg)(x) = for ay g L 2 [ 1],where π(x z)g(z)dz π(x z) = f U () 2 f XW (x w)f XW (z w)dw Let {λ j φ j : j = 1 2 } deote the orthoormal eigevalues ad eigevectors of Π ordered so that λ 1 λ 2 >. Assume that j α λ j is bouded away from ad1forallj.set g (x) = θ j β φ j (x) j=m for some fiite, costat θ>. The for ay h L 2 [ 1], (T h)(w) = ad the Fréchet derivative of T at g is (T g h)(w) = f U () F U [h(x) g (x)]f XW (x w)dx f XW (x w)[h(x) g (x)] dx Assumptio 6 is satisfied with L = MC wheever θ> is sufficietly small. Now let ˆθ be a estimator of θ. The ĝ(x) ˆθ j β φ j (x) j=m is a estimator of g (x). Moreover, (A25) ĝ g 2 = ( ˆθ θ) 2 R
9 NONPARAMETRIC ESTIMATION OF REGRESSION MODEL 9 where R = j=m j 2β. Note that (2β 1)/(2β+α) R is bouded away from ad 1 as. I additio, f W is estimated by ˆ f w (w) = q 1 (T ĝ)(w) Defie ψ j = T g φ j / T g φ j. The a Taylor series approximatio ad sigular value expasio give Now ˆ f W (w) f W (w) = q 1 ( ˆθ θ) = q 1 ( ˆθ θ) j β (T g φ j )(w) + ( ˆθ θ) 2 O[ (2β 1)/(2β+α) ] j=m j=m (2β+α 1)/(2β+α) j=m j β λ 1/2 j ψ j (w) + ( ˆθ θ) 2 O[ (2β 1)/(2β+α) ] j β λ 1/2 j ψ j 2 is bouded away from ad as. Therefore, there is a fiite costat C θ > such that (A26) ( ˆθ θ) 2 C θ (2β+α 1)/(2β+α) ˆ f W f W 2 Combiig (A25) ad (A26) shows that there is a fiite costat C g > such that (A27) (2β 1)/(2β+α) ĝ g 2 C g (2β+α 1)/(2β+α) ˆ f W f W 2 The theorem ow follows from (A27) ad the observatio that with r w = (2β + α 1)/2 O p [ (2β+α 1)/(2β+α) ] is the fastest possible miimax rate of covergece of f ˆ W f W 2. Q.E.D. Rewrite (3.4) as A.4. PROOF OF (3.5) (A28) ( L< b 2 j λ j ) 1
10 1 J. L. HOROWITZ AND S. LEE By a Taylor series expasio, T (g 1 ) T (g 2 ) T g2 (g 1 g 2 ) = 5 f YXW [ḡ(x) x w] [g 1 (x) g 2 (x)] 2 dx y where ḡ is betwee g 1 ad g 2. Therefore, it follows from (3.3) that (A29) L<sup f YXW (y x w) y y x w It follows from (A28) ad (A29) that (3.5) is a sufficiet coditio for (3.4). Q.E.D.
11 NONPARAMETRIC ESTIMATION OF REGRESSION MODEL 11 A.5. AN ADDITIONAL FIGURE FOR MONTE CARLO EXPERIMENTS FIGURE A1. Mote Carlo results for = 8.
12 12 J. L. HOROWITZ AND S. LEE Dept. of Ecoomics, Northwester Uiversity, 21 Sherida Road, Evasto, IL 628, U.S.A.; ad Dept. of Ecoomics, Uiversity College Lodo, Gower Street, Lodo WC1E 6BT, U.K.;
Convergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationMa 4121: Introduction to Lebesgue Integration Solutions to Homework Assignment 5
Ma 42: Itroductio to Lebesgue Itegratio Solutios to Homework Assigmet 5 Prof. Wickerhauser Due Thursday, April th, 23 Please retur your solutios to the istructor by the ed of class o the due date. You
More informationReview Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =
Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:
More informationPRELIM PROBLEM SOLUTIONS
PRELIM PROBLEM SOLUTIONS THE GRAD STUDENTS + KEN Cotets. Complex Aalysis Practice Problems 2. 2. Real Aalysis Practice Problems 2. 4 3. Algebra Practice Problems 2. 8. Complex Aalysis Practice Problems
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationREGRESSION WITH QUADRATIC LOSS
REGRESSION WITH QUADRATIC LOSS MAXIM RAGINSKY Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X, Y ), where, as before, X is a R d
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationRandom Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.
Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationMINIMAX RATES OF CONVERGENCE AND OPTIMALITY OF BAYES FACTOR WAVELET REGRESSION ESTIMATORS UNDER POINTWISE RISKS
Statistica Siica 19 (2009) 1389 1406: Supplemet S1 S17 1 MINIMAX RATES OF ONVERGENE AND OPTIMALITY OF BAYES FATOR WAVELET REGRESSION ESTIMATORS UNDER POINTWISE RISKS Natalia Bochkia ad Theofais Sapatias
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More informationS1 Notation and Assumptions
Statistica Siica: Supplemet Robust-BD Estimatio ad Iferece for Varyig-Dimesioal Geeral Liear Models Chumig Zhag Xiao Guo Che Cheg Zhegju Zhag Uiversity of Wiscosi-Madiso Supplemetary Material S Notatio
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationNotes for Lecture 11
U.C. Berkeley CS78: Computatioal Complexity Hadout N Professor Luca Trevisa 3/4/008 Notes for Lecture Eigevalues, Expasio, ad Radom Walks As usual by ow, let G = (V, E) be a udirected d-regular graph with
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationRegression with quadratic loss
Regressio with quadratic loss Maxim Ragisky October 13, 2015 Regressio with quadratic loss is aother basic problem studied i statistical learig theory. We have a radom couple Z = X,Y, where, as before,
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationTHE SPECTRAL RADII AND NORMS OF LARGE DIMENSIONAL NON-CENTRAL RANDOM MATRICES
COMMUN. STATIST.-STOCHASTIC MODELS, 0(3), 525-532 (994) THE SPECTRAL RADII AND NORMS OF LARGE DIMENSIONAL NON-CENTRAL RANDOM MATRICES Jack W. Silverstei Departmet of Mathematics, Box 8205 North Carolia
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationAnalytic Continuation
Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationA survey on penalized empirical risk minimization Sara A. van de Geer
A survey o pealized empirical risk miimizatio Sara A. va de Geer We address the questio how to choose the pealty i empirical risk miimizatio. Roughly speakig, this pealty should be a good boud for the
More informationINFINITE SEQUENCES AND SERIES
11 INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES 11.4 The Compariso Tests I this sectio, we will lear: How to fid the value of a series by comparig it with a kow series. COMPARISON TESTS
More informationON THE CHOICE OF m IN THE n OUT OF n BOOTSTRAP AND CONFIDENCE BOUNDS FOR EXTREMA
Statistica Siica 182008: Supplemet, S47 S57 ON THE CHOICE OF m IN THE OUT OF BOOTSTRAP AND CONFIDENCE BOUNDS FOR EXTREMA Peter J. Bickel ad Aat Sakov Uiversity of Califoria, Berkeley ad Taor Group 2. Appedix
More informationGlivenko-Cantelli Classes
CS28B/Stat24B (Sprig 2008 Statistical Learig Theory Lecture: 4 Gliveko-Catelli Classes Lecturer: Peter Bartlett Scribe: Michelle Besi Itroductio This lecture will cover Gliveko-Catelli (GC classes ad itroduce
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationRates of Convergence for Quicksort
Rates of Covergece for Quicksort Ralph Neiiger School of Computer Sciece McGill Uiversity 480 Uiversity Street Motreal, HA 2K6 Caada Ludger Rüschedorf Istitut für Mathematische Stochastik Uiversität Freiburg
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationSequences and Limits
Chapter Sequeces ad Limits Let { a } be a sequece of real or complex umbers A ecessary ad sufficiet coditio for the sequece to coverge is that for ay ɛ > 0 there exists a iteger N > 0 such that a p a q
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationTESTING FOR THE BUFFERED AUTOREGRESSIVE PROCESSES (SUPPLEMENTARY MATERIAL)
TESTING FOR THE BUFFERED AUTOREGRESSIVE PROCESSES SUPPLEMENTARY MATERIAL) By Ke Zhu, Philip L.H. Yu ad Wai Keug Li Chiese Academy of Scieces ad Uiversity of Hog Kog APPENDIX: PROOFS I this appedix, we
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationMath 341 Lecture #31 6.5: Power Series
Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationModern Discrete Probability Spectral Techniques
Moder Discrete Probability VI - Spectral Techiques Backgroud Sébastie Roch UW Madiso Mathematics December 1, 2014 1 Review 2 3 4 Mixig time I Theorem (Covergece to statioarity) Cosider a fiite state space
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationLecture 9: Expanders Part 2, Extractors
Lecture 9: Expaders Part, Extractors Topics i Complexity Theory ad Pseudoradomess Sprig 013 Rutgers Uiversity Swastik Kopparty Scribes: Jaso Perry, Joh Kim I this lecture, we will discuss further the pseudoradomess
More informationThe natural exponential function
The atural expoetial fuctio Attila Máté Brookly College of the City Uiversity of New York December, 205 Cotets The atural expoetial fuctio for real x. Beroulli s iequality.....................................2
More informationTechnical Proofs for Homogeneity Pursuit
Techical Proofs for Homogeeity Pursuit bstract This is the supplemetal material for the article Homogeeity Pursuit, submitted for publicatio i Joural of the merica Statistical ssociatio. B Proofs B. Proof
More informationMath 128A: Homework 1 Solutions
Math 8A: Homework Solutios Due: Jue. Determie the limits of the followig sequeces as. a) a = +. lim a + = lim =. b) a = + ). c) a = si4 +6) +. lim a = lim = lim + ) [ + ) ] = [ e ] = e 6. Observe that
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationf(x)g(x) dx is an inner product on D.
Ark9: Exercises for MAT2400 Fourier series The exercises o this sheet cover the sectios 4.9 to 4.13. They are iteded for the groups o Thursday, April 12 ad Friday, March 30 ad April 13. NB: No group o
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationSupplementary Materials for Statistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting
Supplemetary Materials for Statistical-Computatioal Phase Trasitios i Plated Models: The High-Dimesioal Settig Yudog Che The Uiversity of Califoria, Berkeley yudog.che@eecs.berkeley.edu Jiamig Xu Uiversity
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationDynamic Policy Programming with Function Approximation: Supplementary Material
Dyamic Policy Programmig with Fuctio pproximatio: Supplemetary Material Mohammad Gheshlaghi zar Radboud Uiversity Nijmege Geert Grooteplei Noord 21 6525 EZ Nijmege Netherlads m.azar@sciece.ru.l Viceç Gómez
More informationSieve Estimators: Consistency and Rates of Convergence
EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes
More informationReal Numbers R ) - LUB(B) may or may not belong to B. (Ex; B= { y: y = 1 x, - Note that A B LUB( A) LUB( B)
Real Numbers The least upper boud - Let B be ay subset of R B is bouded above if there is a k R such that x k for all x B - A real umber, k R is a uique least upper boud of B, ie k = LUB(B), if () k is
More informationAdvanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology
Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationTopics in Probability Theory and Stochastic Processes Steven R. Dunbar. Stirling s Formula Derived from the Gamma Function
Steve R. Dubar Departmet of Mathematics 23 Avery Hall Uiversity of Nebraska-Licol Licol, NE 68588-3 http://www.math.ul.edu Voice: 42-472-373 Fax: 42-472-8466 Topics i Probability Theory ad Stochastic Processes
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationn=1 a n is the sequence (s n ) n 1 n=1 a n converges to s. We write a n = s, n=1 n=1 a n
Series. Defiitios ad first properties A series is a ifiite sum a + a + a +..., deoted i short by a. The sequece of partial sums of the series a is the sequece s ) defied by s = a k = a +... + a,. k= Defiitio
More informationThe log-behavior of n p(n) and n p(n)/n
Ramauja J. 44 017, 81-99 The log-behavior of p ad p/ William Y.C. Che 1 ad Ke Y. Zheg 1 Ceter for Applied Mathematics Tiaji Uiversity Tiaji 0007, P. R. Chia Ceter for Combiatorics, LPMC Nakai Uivercity
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationj=1 dz Res(f, z j ) = 1 d k 1 dz k 1 (z c)k f(z) Res(f, c) = lim z c (k 1)! Res g, c = f(c) g (c)
Problem. Compute the itegrals C r d for Z, where C r = ad r >. Recall that C r has the couter-clockwise orietatio. Solutio: We will use the idue Theorem to solve this oe. We could istead use other (perhaps
More informationMath 299 Supplement: Real Analysis Nov 2013
Math 299 Supplemet: Real Aalysis Nov 203 Algebra Axioms. I Real Aalysis, we work withi the axiomatic system of real umbers: the set R alog with the additio ad multiplicatio operatios +,, ad the iequality
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationMixingales. Chapter 7
Chapter 7 Mixigales I this sectio we prove some of the results stated i the previous sectios usig mixigales. We first defie a mixigale, otig that the defiitio we give is ot the most geeral defiitio. Defiitio
More informationSolutions to home assignments (sketches)
Matematiska Istitutioe Peter Kumli 26th May 2004 TMA401 Fuctioal Aalysis MAN670 Applied Fuctioal Aalysis 4th quarter 2003/2004 All documet cocerig the course ca be foud o the course home page: http://www.math.chalmers.se/math/grudutb/cth/tma401/
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationDifferentiable Convex Functions
Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationComplex Analysis Spring 2001 Homework I Solution
Complex Aalysis Sprig 2001 Homework I Solutio 1. Coway, Chapter 1, sectio 3, problem 3. Describe the set of poits satisfyig the equatio z a z + a = 2c, where c > 0 ad a R. To begi, we see from the triagle
More informationA Hadamard-type lower bound for symmetric diagonally dominant positive matrices
A Hadamard-type lower boud for symmetric diagoally domiat positive matrices Christopher J. Hillar, Adre Wibisoo Uiversity of Califoria, Berkeley Jauary 7, 205 Abstract We prove a ew lower-boud form of
More informationOrthogonal Dirichlet Polynomials with Arctangent Density
Orthogoal Dirichlet Polyomials with Arctaget Desity Doro S. Lubisky School of Mathematics, Georgia Istitute of Techology, Atlata, GA 3033-060 USA. Abstract Let { j } j= be a strictly icreasig sequece of
More informationON POINTWISE BINOMIAL APPROXIMATION
Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 57-66 ON POINTWISE BINOMIAL APPROXIMATION BY w-functions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece
More informationThe central limit theorem for Student s distribution. Problem Karim M. Abadir and Jan R. Magnus. Econometric Theory, 19, 1195 (2003)
The cetral limit theorem for Studet s distributio Problem 03.6.1 Karim M. Abadir ad Ja R. Magus Ecoometric Theory, 19, 1195 (003) Z Ecoometric Theory, 19, 003, 1195 1198+ Prited i the Uited States of America+
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationDetailed proofs of Propositions 3.1 and 3.2
Detailed proofs of Propositios 3. ad 3. Proof of Propositio 3. NB: itegratio sets are geerally omitted for itegrals defied over a uit hypercube [0, s with ay s d. We first give four lemmas. The proof of
More informationMATH 324 Summer 2006 Elementary Number Theory Solutions to Assignment 2 Due: Thursday July 27, 2006
MATH 34 Summer 006 Elemetary Number Theory Solutios to Assigmet Due: Thursday July 7, 006 Departmet of Mathematical ad Statistical Scieces Uiversity of Alberta Questio [p 74 #6] Show that o iteger of the
More informationSolutions to Tutorial 5 (Week 6)
The Uiversity of Sydey School of Mathematics ad Statistics Solutios to Tutorial 5 (Wee 6 MATH2962: Real ad Complex Aalysis (Advaced Semester, 207 Web Page: http://www.maths.usyd.edu.au/u/ug/im/math2962/
More information