EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS
|
|
- Cameron Cole
- 5 years ago
- Views:
Transcription
1 EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS Ryszard Zieliński Ist Math Polish Acad Sc POBox 21, Warszawa 10, Polad ABSTRACT Weak laws of large umbers (W LLN), strog laws of large umbers (SLLN), ad cetral limit theorems (CLT ) i statistical models differ from those i probability theory i that they should hold uiformly i the family of distributios specified by the model If a limit law states that for every ε > 0 there exists N such that for all > N the iequalities ξ < ε are satisfied ad N = N(ε) is explicitly give tha we call the law effective It is trivial to obtai the effective statistical versio of W LLN i the Beroulli scheme, to get SLLN takes a little while, but CLT does ot hold uiformly Other statistical schemes are also cosidered Mathematics Subject Classificatio: 62E12, 62E20, 60F05, 60F15 Key words ad phrases: weak laws of large umbers, strog laws of large umbers, cetral limit theorems, statistical models, effective limit laws, uiform limit laws, Beroulli scheme, expoetial distributio, quatiles 1
2 1 THE BERNOULLI SCHEME Let X, X 1, X 2,, X, be iid radom variables with P θ X = 1 = P θ X = 0 = θ, θ (0, 1) ad let S = i=1 X i W LLN states that, uder every fixed θ (0, 1), S / θ i probability, which ca be writte i the form θ (0, 1) ε>0 η >0 N N P θ S θ > ε < η A appropriate N is give by the formula N = θ(1 θ)/ηε 2 I the related statistical model all what we kow about θ is that θ (0, 1) so that the above result is of o use: the statistical versio may be formulated as follows: propositio 1 (A) ε>0 η >0 N N θ (0, 1) P θ S θ > ε < η (B) the appropriate N = N(ε, η) = 1 4ηε 2 The formula is useful for example for costructig the cofidece iterval for a ukow θ, with a a priori postulated accuracy ad cofidece level Here ad further o Part (A) states the uiform covergece ad Part (B) makes the law effective Part (B) may be improved by the argumet used i the proof of Propositio 2 below (Berstei iequality) SLLN states that, uder every fixed θ (0, 1), S / θ as Usig the fact that ξ 0 as iff ε > 0 lim N P =N X > ε = 0 iff ε > 0 η > 0 N P =N X > ε < η, a appropriate effective statistical versio of the law takes o the form 2
3 propositio 2 (A) ε>0 η >0 N θ (0, 1) P θ =N S θ > ε < η; (B) the appropriate N =N(ε, η)=mi proof 4 ( η ( )) ε 2 log 1 e ε2 /4 1, 2 4ηε 2 By a rather crude estimatio oe obtais P θ S θ > ε < =N =N P θ S θ > ε ad the by the Berstei iequality for the Beroulli scheme (Serflig 1980, Jakubowski et al 2001) i the form P θ S θ > ε 2 /4 2e ε the followig estimatio holds P θ =N S θ > ε < 2e Nε2 /4 1 e ε2 /4 which eables us to explicitly fix N as ay iteger such that N > 4 ε 2 log ( η 2 ( 1 e ε2 /4 )) Table 1 (first lie) exhibits N = N(ε, η) for some ε ad η Aother formula for N may be costructed as follows (Weso lowski 2002) Defie Y i = X i θ, T k = 1 k k i=1 Y i, ad G k = σ(t k, T k+1, ) The (T k, G k+1 ) k=1,2, is a iverse martigale: E(T k G k+1 ) = E(T k T k+1 ) = 1 k k E(Y i T k+1 ) = T k+1, k = 1, 2, i=1 The maximal iequality for iverse martigales gives us P max T k a N k m V ar(t N ) a 2 = 3 θ(1 θ) Na 2 1 4Na 2
4 ad i cosequece P θ =N S θ > ε = P sup k N = lim m P T k ε max T k ε N k m 1 lim m 4Nε 2 = 1 4Nε 2 Now propositio 2(A) holds for ay N 1 4ε 2 η (secod lie i Table 1) Table 1 N(ε, η) η ε ,596 4,517 5, ,500 25, , , ,902 25, ,000 2,500, CLT for the Beroulli scheme holds for every θ (0, 1) separately, eve i a stroger versio ( uiformly i x ): θ (0, 1) Pθ sup S x Φ x ( x θ θ(1 θ) ) 0, as The classical CLT for the Beroulli scheme may be writte i the form θ x ε N = N(θ, x, ε) N S θ Pθ x θ(1 θ) Φ(x) ε 4
5 What statisticias eed is x ε N = N(x, ε) N θ S θ Pθ x Φ(x) ε θ(1 θ) or eve i a stroger form: uiformly i x The latter is however ot true To see that oe should prove that S θ x ε N N θ P θ x Φ(x) > ε θ(1 θ) It is sufficiet to prove that S θ x ε θ P θ x Φ(x) > ε θ(1 θ) To this ed take x = 0 ad ε = 1/4 The LHS = P θ S θ 1/2 If for ay fixed oe takes θ such that θ < 1 ad (1 θ) > 3/4, the P θ S θ = P θ S = 0 = (1 θ) > 3/4 ad LHS > ε It follows that CLT does ot hold uiformly i the statistical model with θ (0, 1) It is iterestig to observe that similar result holds i the iverse Biomial scheme (egative Biomial distributio) Let Y be the umber of experimets eeded to observe first success: P θ Y = y = (1 θ) y 1 θ, E θ Y = 1 θ, V ar θy = 1 θ θ 2 If Y, Y 1, Y 2, are iid ad T = i=1 Y i the P θ T θ 1 θ θ 2 x Φ(x) x=0 = P θ T Φ(0) θ > P θ T Φ(0) = θ 1 2 which teds to 1/2 as θ 1 Oe may coclude that typical difficulties i costructig cofidece itervals for θ (eg Brow et al 2001), based o ormal approximatio, arises from the fact that CLT does ot hold uiformly 5
6 2 EXPONENTIAL DISTRIBUTION If X 1, X 2, are iid radom variables with probability desity fuctio 1 e x/, x > 0, > 0, ad S = i=1 X i the the SLLN S / as does ot hold uiformly i > 0 ad the CLT x S P x Φ(x) 0 holds uiformly To prove the former it is eough to observe that for some fixed ε > 0, η > 0, ad for each, oe ca fid > 0 such that S P < ε < η which, by the fact that S / has gamma distributio Γ(, ) with the shape parameter ad the scale parameter /, easily follows from the followig estimatio S P < ε = 1 Γ() (1+ε/) (1 ε/) t 1 e t dt < 2ε 1 2π A stroger versio of the secod statemet may be formulated as the followig effective propositio 3 6
7 Before statig the theorem let us defie +x 1 t R(x, ) = 1 e t dt Φ(x) if x > Γ() 0 0 elsewhere propositio 3 If X 1, X 2, are iid radom variables with probability distributio fuctio 1 e x/ ad S = i=1 X i the (A) ε > 0 N = N(ε) > 0 sup P x S x Φ(x) < ε ad (B) a appropriate N = N(ε) is give umerically as a N such that max R(x, ) ε x proof To prove part (A) of the propositio it is eough to observe that P S x = P 1 S 1 + x which, due to the fact that (1/)(S /) is distributed as Γ(, 1/), does ot deped o : P S x = 1 Γ() +x 0 t 1 e t dt To prove part (B) observe that R(x, ) = P S x Φ(x) Fuctio R(x, ) is cotiuous ad bouded; two examples are exhibited i Fig 1 7
8 R(x, ) = =20 x -002 Fig1 Fuctio R(x,) Some values of R(x, ) preseted i Tab 2 below eable us to choose a proper N for typical values of ε; here x = arg mi x R(x, ) Tab 2 R(x, ) R(x, ) x , , , , , , , , , , ,132 17, , ,540 70, , ,224 Explicite formulas either for x or for R(x, ) are kow to the author 8
9 3 QUANTILES It is well kow (eg Serflig 1980) that if x q = x q (F ) is the uique quatile of order q of the distributio F ad k()/ q, the X k(): x q as Here X k: is the k-th order statistic from the sample X 1, X 2,, X The covergece is however ot uiform: for each ε, for each η, ad for every oe ca fid a distributio F with the uique quatile x q such that Xk(): P F x q > ε 1 η A ecessary ad sufficiet coditio for uiform covergece has bee give i Zieliński (1998) A effective uiform asymptotic theorem for a smaller class of model distributios may be stated as follows For a fixed q (0, 1), cosider the class F(q, ϑ) of all distributios F such that the desities f at the qth quatile x q exist ad they satisfy f(x q ) ϑ > 0 propositio 4 (A) ad (B) ε > 0 η > 0 N = N(ε, η) F F(q, ϑ) P F sup Xk(): x q > ε < η N ( ) 1 ( 1 8 log 1 exp 2 8 ϑ2 ε 2) η N(ϑ, ε, η) ϑ 2 ε 2 proof If δ = if F F miq F (x q ε), F (x q + ε) q for a class F of distributios, the for every F F P F sup Xk(): x q > ε < 2τ N N 1 τ with τ = exp δ 2 /2 (Serflig 1980) I the class F(q, ϑ) we have F (x q + t) q lim 0<t 0 t q F (x q t) = lim 0<t 0 t 9 = ϑ
10 so that there exists t 0 > 0 such that for all t < t 0 F (x q + t) q 1 2 ϑt ad q F (x q t) 1 2 ϑt ad i cosequece, for all sufficietly small ε (for ε < t 0 ) δ = miq F (x q ε), F (x q + ε) q 1 2 ϑε Now τ = exp δ 2 /2 exp 1 8 ϑ2 ε 2 Solvig, with respect to N, the equatio we obtai the result 2τ N 1 τ = η Table 3 below gives us a isight i how large samples are eeded to get the prescribed accuracy of the asymptotic Table 3 N(ϑ, ε, η) ε η ϑ ,398 35, ,414 7, ,745 1, ,871 42, ,782 9, ,587 2, SOME NON EFFECTIVE UNIFORM ASYMPTOTIC RESULTS Cosider the problem as i the previous Sectio As a o effective asymptotic theorem we have the followig Corollary (Zieliński 1998): if F is a cotiuous ad strictly icreasig distributio fuctio ad k()/ q the X k(): x q as uiformly i the family of distributios F θ (x) = F (x θ), < θ < 10
11 Two more geeral theorems cocerig the covergece of s (θ) = a(x i, θ) where a(x, θ) = ( a 1 (X, θ),, a m (X, θ) ) is a give vector-valued fuctio, are take from Borovkov (1998) To state the theorems recall that a itegral ψ(x, θ)p θ (dx) is said to be coverget i Θ uiformly with respect to θ if sup ψ(x, θ) P θ (dx) 0, as N θ Θ ψ(x,θ) >N theorem 1 (uiform law of large umbers) coverges uiformly i θ Θ, the P θ s (θ) a(θ) > ε 0 If a(θ) = a(x, θ)p θ (dx) as uiformly i θ To state the cetral limit theorem assume that a(θ) = 0 (or take a (X, θ) = a(x, θ) a(θ) istead of a(x, θ)) theorem 2 (uiform cetral limit theorem) If a 2 j (x, θ)p θ(dx), j = 1,, m, coverge uiformly i θ, the s (θ)/ ( ) coverges to a ormal radom variable N 0, σ 2 (θ) uiformly with respect to θ, where σ 2 (θ) = E θ ( a T (X, θ)a(x, θ) ) 5 COMMENTS Though of great importace for statistical iferece, the literature o uiform asymptotic theorems i statistical models, ad especially o effective limit laws, is extremely scarce Perhaps the oly two examples of specific theorems for statistical models are the above result o sample quatiles ad a geeral result o uiform cosistecy of maximum likelihood estimators (Borovkow 1998, Ibragimov et al 1981) Other uiform versios of asymptotic theorems are mostly costructed as follows: take a probability asymptotic theorem which states that if a distributio uder 11
12 cosideratio satisfies a coditio C the W LLN (or SLLN, or CLT ) holds The formulate the statistical theorem: if the coditio C is satisfied uiformly i a give statistical model the W LLN (or, respectively, SLLN, or CLT ) holds uiformly (Ibragimov et al 1981) If a distributio-free statistic i a model uder cosideratio is available, the problem of uiform limit laws is automatically solved, but costructig a effective limit law may be difficult As a example cosider the Kolmogov statistic D = sup x F (x) F (x) i a statistical model with F cotiuous; here F (x) is the empirical distributio fuctio It is well kow that the distributio of D does ot deped o the specific distributio F so that the stochastic covergece P D > ε 0 for every ε > 0 holds uiformly That meas that for every ε > 0 ad for every η > 0 there exists N = N(ε, η) such that for all F cotiuous ad for all > N, P D > ε < η I Birbaum (1952) oe reads that N(015, 01) = 65 ad N(005, 001) = 1, 060 The values were obtaied umerically ad o explicite formula for N(ε, η) is kow 12
13 REFERENCES Birbaum, ZW (1952): Numerical tabulatio of the distributio of Kolmogorov s statistic for fiite sample size, JASA 47, Boratyńska, A ad Zieliński, R (1997): Asymptotic behavior of sample media i a parametric model, Aales UMCS, Sectio A, Vol LI1,2, Borovkov, AA (1998): Mathematical statistics, Gordo ad Breach Brow, LD, Cai, TT ad DasGupta, A (2001): Iterval estimatio for a Biomial Proportio, Statistical Sciece 16, 2, Ibragimov, IA ad Has miskii (1981): Statistical estimatio Asymptotic theory Spriger Jakubowski, J ad Sztecel, R (2001): Wste,p do teorii prawdopodobieństwa Wyd II Script, Warszawa Serflig, RJ (1980): Approximatio theorems of mathematical statistics Wiley Weso lowski, J (2002): Private commuicatio Zieliński, R (1998): Uiform strog cosistecy of sample quatiles, Statist Probab Lett 37,
This section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationINFINITE SEQUENCES AND SERIES
INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES I geeral, it is difficult to fid the exact sum of a series. We were able to accomplish this for geometric series ad the series /[(+)]. This is
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information6. Uniform distribution mod 1
6. Uiform distributio mod 1 6.1 Uiform distributio ad Weyl s criterio Let x be a seuece of real umbers. We may decompose x as the sum of its iteger part [x ] = sup{m Z m x } (i.e. the largest iteger which
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationBerry-Esseen bounds for self-normalized martingales
Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationMATH4822E FOURIER ANALYSIS AND ITS APPLICATIONS
MATH48E FOURIER ANALYSIS AND ITS APPLICATIONS 7.. Cesàro summability. 7. Summability methods Arithmetic meas. The followig idea is due to the Italia geometer Eresto Cesàro (859-96). He shows that eve if
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationMATH 413 FINAL EXAM. f(x) f(y) M x y. x + 1 n
MATH 43 FINAL EXAM Math 43 fial exam, 3 May 28. The exam starts at 9: am ad you have 5 miutes. No textbooks or calculators may be used durig the exam. This exam is prited o both sides of the paper. Good
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationDetailed proofs of Propositions 3.1 and 3.2
Detailed proofs of Propositios 3. ad 3. Proof of Propositio 3. NB: itegratio sets are geerally omitted for itegrals defied over a uit hypercube [0, s with ay s d. We first give four lemmas. The proof of
More informationA Note on the Kolmogorov-Feller Weak Law of Large Numbers
Joural of Mathematical Research with Applicatios Mar., 015, Vol. 35, No., pp. 3 8 DOI:10.3770/j.iss:095-651.015.0.013 Http://jmre.dlut.edu.c A Note o the Kolmogorov-Feller Weak Law of Large Numbers Yachu
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationStatistical Theory MT 2008 Problems 1: Solution sketches
Statistical Theory MT 008 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. a) Let 0 < θ < ad put fx, θ) = θ)θ x ; x = 0,,,... b) c) where α
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationMath 113 Exam 4 Practice
Math Exam 4 Practice Exam 4 will cover.-.. This sheet has three sectios. The first sectio will remid you about techiques ad formulas that you should kow. The secod gives a umber of practice questios for
More informationINFINITE SEQUENCES AND SERIES
11 INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES 11.4 The Compariso Tests I this sectio, we will lear: How to fid the value of a series by comparig it with a kow series. COMPARISON TESTS
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationJANE PROFESSOR WW Prob Lib1 Summer 2000
JANE PROFESSOR WW Prob Lib Summer 000 Sample WeBWorK problems. WeBWorK assigmet Series6CompTests due /6/06 at :00 AM..( pt) Test each of the followig series for covergece by either the Compariso Test or
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationTR/46 OCTOBER THE ZEROS OF PARTIAL SUMS OF A MACLAURIN EXPANSION A. TALBOT
TR/46 OCTOBER 974 THE ZEROS OF PARTIAL SUMS OF A MACLAURIN EXPANSION by A. TALBOT .. Itroductio. A problem i approximatio theory o which I have recetly worked [] required for its solutio a proof that the
More informationLimit distributions for products of sums
Statistics & Probability Letters 62 (23) 93 Limit distributios for products of sums Yogcheg Qi Departmet of Mathematics ad Statistics, Uiversity of Miesota-Duluth, Campus Ceter 4, 7 Uiversity Drive, Duluth,
More informationSummary. Recap ... Last Lecture. Summary. Theorem
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 23 Hyu Mi Kag April 11th, 2013 What is p-value? What is the advatage of p-value compared to hypothesis testig procedure with size α? How ca
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationCentral limit theorem and almost sure central limit theorem for the product of some partial sums
Proc. Idia Acad. Sci. Math. Sci. Vol. 8, No. 2, May 2008, pp. 289 294. Prited i Idia Cetral it theorem ad almost sure cetral it theorem for the product of some partial sums YU MIAO College of Mathematics
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationJournal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula
Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationRates of Convergence by Moduli of Continuity
Rates of Covergece by Moduli of Cotiuity Joh Duchi: Notes for Statistics 300b March, 017 1 Itroductio I this ote, we give a presetatio showig the importace, ad relatioship betwee, the modulis of cotiuity
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam 4 will cover.-., 0. ad 0.. Note that eve though. was tested i exam, questios from that sectios may also be o this exam. For practice problems o., refer to the last review. This
More informationMetric Space Properties
Metric Space Properties Math 40 Fial Project Preseted by: Michael Brow, Alex Cordova, ad Alyssa Sachez We have already poited out ad will recogize throughout this book the importace of compact sets. All
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationLast Lecture. Unbiased Test
Last Lecture Biostatistics 6 - Statistical Iferece Lecture Uiformly Most Powerful Test Hyu Mi Kag March 8th, 3 What are the typical steps for costructig a likelihood ratio test? Is LRT statistic based
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationUniversity of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!
Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationAn alternative proof of a theorem of Aldous. concerning convergence in distribution for martingales.
A alterative proof of a theorem of Aldous cocerig covergece i distributio for martigales. Maurizio Pratelli Dipartimeto di Matematica, Uiversità di Pisa. Via Buoarroti 2. I-56127 Pisa, Italy e-mail: pratelli@dm.uipi.it
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationB Supplemental Notes 2 Hypergeometric, Binomial, Poisson and Multinomial Random Variables and Borel Sets
B671-672 Supplemetal otes 2 Hypergeometric, Biomial, Poisso ad Multiomial Radom Variables ad Borel Sets 1 Biomial Approximatio to the Hypergeometric Recall that the Hypergeometric istributio is fx = x
More information5.1 A mutual information bound based on metric entropy
Chapter 5 Global Fao Method I this chapter, we exted the techiques of Chapter 2.4 o Fao s method the local Fao method) to a more global costructio. I particular, we show that, rather tha costructig a local
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationEnumerative & Asymptotic Combinatorics
C50 Eumerative & Asymptotic Combiatorics Stirlig ad Lagrage Sprig 2003 This sectio of the otes cotais proofs of Stirlig s formula ad the Lagrage Iversio Formula. Stirlig s formula Theorem 1 (Stirlig s
More information