An Introduction to Asymptotic Theory
|
|
- Anthony Booker
- 5 years ago
- Views:
Transcription
1 A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20
2 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu (HKU) Asymptotic Theory 2 / 20
3 Five Weapos i Asymptotic Theory Five Weapos The weak law of large umbers (WLLN, or LLN) The cetral limit theorem (CLT) The cotiuous mappig theorem (CMT) Slutsky s theorem The Delta method Notatios: - I oliear (i parameter) models, the capital letters such as X deote radom variables or radom vectors ad the correspodig lower case letters such as x deote the potetial values they may take. - Geeric otatio for a parameter i oliear eviromets (e.g., oliear models or oliear costraits) is θ, while i liear eviromets is β. Pig Yu (HKU) Asymptotic Theory 3 / 20
4 Five Weapos i Asymptotic Theory The WLLN Defiitio A radom vector Z coverges i probability to Z as!, deoted as Z for ay δ > 0, lim P(kZ Z k > δ ) = 0.! p! Z, if Although the limit Z ca be radom, it is usually costat. [ituitio] p The probability limit of Z is ofte deoted as plim(z ). If Z! 0, we deote Z = o p (1). Whe a estimator coverges i probability to the true value as the sample size diverges, we say that the estimator is cosistet. Cosistecy is a importat prelimiary step i establishig other importat asymptotic approximatios. Theorem (WLLN) Suppose X 1,,X, are i.i.d. radom vectors, ad E [kxk] < ; the as!, X 1 p X i! E [X ]. Pig Yu (HKU) Asymptotic Theory 4 / 20
5 Five Weapos i Asymptotic Theory The CLT Defiitio A radom k vector Z coverges i distributio to Z as!, deoted as Z d! Z, if lim F (z) = F (z),! at all z where F () is cotiuous, where F is the cdf of Z ad F is the cdf of Z. Usually, Z is ormally distributed, so all z 2 R k are cotiuity poits of F. If Z coverges i distributio to Z, the Z is stochastically bouded ad we deote Z = O p (1). Rigorously, Z = O p (1) if 8ε > 0, 9M ε < such that P(kZ k > M ε ) < ε for ay. If Z = o p (1), the Z = O p (1). We ca show that o p (1) + o p (1) = o p (1), o p (1) + O p (1) = O p (1), O p (1) + O p (1) = O p (1), o p (1)o p (1) = o p (1), o p (1)O p (1) = o p (1), ad O p (1)O p (1) = O p (1). Theorem (CLT) suppose X 1,,X, are i.i.d. radom k vectors, E [X ] = µ, ad Var(X ) = Σ; the p X µ d! N(0,Σ). Pig Yu (HKU) Asymptotic Theory 5 / 20
6 Five Weapos i Asymptotic Theory Compariso Betwe the WLLN ad CLT The CLT tells more tha the WLLN. p X µ d! N(0,Σ) implies X p! µ, so the CLT is stroger tha the WLLN. X p! µ meas X µ = o p (1), but does ot provide ay iformatio about p X µ. The CLT tells that p X µ = O p (1) or X µ = O p ( 1/2 ). But the WLLN does ot require the secod momet fiite; that is, a stroger result is ot free. Pig Yu (HKU) Asymptotic Theory 6 / 20
7 Five Weapos i Asymptotic Theory The CMT Theorem (CMT) Suppose X 1,,X, are radom k vectors, ad g is a cotiuous fuctio o the support of X (to R l ) a.s. P X ; the X p! X =) g(x ) p! g(x ); X d! X =) g(x ) d! g(x ). The CMT allows the fuctio g to be discotiuous but the probability of beig at a discotiuity poit is zero. For example, the fuctio g(u) = u 1 is discotiuous at u = 0, but if d X! X N(0,1) the P(X = 0) = 0 so X 1 d! X 1. Pig Yu (HKU) Asymptotic Theory 7 / 20
8 Five Weapos i Asymptotic Theory Slutsky s Theorem I the CMT, X coverges to X joitly i various modes of covergece. For the covergece i probability ( p!), margial covergece implies joit covergece, so there is o problem if we substitute joit covergece by margial covergece. But for the covergece i distributio ( d d d!), X! X, Y! Y does ot imply X d X!. Y Y Nevertheless, there is a special case where this result holds, which is Slutsky s theorem. Theorem (Slutsky s Theorem) d d p X If X! X, Y! c () Y! c, where c is a costat, the Y d! X c This implies X + Y d! X + c, Y X d! cx, Y 1 X d! c 1 X whe c 6= 0. Here X,Y,X,c ca be uderstood as vectors or matrices as log as the operatios are compatible.. Pig Yu (HKU) Asymptotic Theory 8 / 20
9 Five Weapos i Asymptotic Theory Applicatios of the CMT ad Slutsky s Theorem Example Suppose X d! N(0,Σ), ad Y p! Σ; the Y 1/2 X d! Σ 1/2 N(0,Σ) = N(0,I), where I is the idetity matrix. (why?) Example Suppose X d! N(0,Σ), ad Y p! Σ; the X 0 Y 1 X d! χ 2 k, where k is the dimesio of X. (why?) Aother importat applicatio of Slutsky s theorem is the Delta method. Pig Yu (HKU) Asymptotic Theory 9 / 20
10 Five Weapos i Asymptotic Theory The Delta Method Theorem Suppose p (Z c) d! Z N(0,Σ), c 2 R k, ad g(z) : R k! R. If dg(z) dz is 0 cotiuous at c, the p (g(z ) g(c)) d! dg(c) dz Z. 0 Proof. p (g(z ) g(c)) = p dg(c) dz 0 (Z c), where c is betwee Z ad c. p (Z c) d p! Z implies that Z! c, so by the CMT, dg(c) p dg(c) dz! 0 dz. By Slutsky s theorem, p (g(z 0 ) g(c)) has the asymptotic distributio dg(c) dz Z. 0 The Delta method implies that asymptotically, the radomess i a trasformatio of Z is completely cotrolled by that i Z. Pig Yu (HKU) Asymptotic Theory 10 / 20
11 Asymptotics for the MoM Estimator Asymptotics for the MoM Estimator Pig Yu (HKU) Asymptotic Theory 11 / 20
12 Asymptotics for the MoM Estimator The MoM Estimator Recall that the MoM estimator is defied as the solutio to 1 m(x i jθ) = 0. We ca prove the MoM estimator is cosistet ad asymptotically ormal (CAN) uder some regularity coditios. Specifically, the asymptotic distributio of the MoM estimator is p bθ θ 0 d! N 0,M 1 ΩM 0 1, where M = de[m(xjθ 0)] dθ 0 ad Ω = E [m(xjθ 0 )m(xjθ 0 ) 0 ]. The asymptotic variace takes a sadwich form ad ca be estimated by its sample aalog. Pig Yu (HKU) Asymptotic Theory 12 / 20
13 Asymptotics for the MoM Estimator Derivatio of the Asymptotic Distributio of the MoM Estimator 1 =) 1 =) p bθ m(x i j b θ) = 0 m(x i jθ 0 ) + 1 θ 0 = d!? 1 dm(x i jθ) dθ 0 bθ θ 0 = 0 M 1 N (0,Ω) dm(x i jθ) dθ 0 p 1 1 m(x i jθ 0 ) p bθ θ 0 p 1 M 1 m(x i jθ 0 ), so M 1 m(x i jθ 0 ) is called the ifluece fuctio. h i We use de[m(xjθ 0)] dm(xjθ istead of E 0 ) because E [m(xjθ)] is more smooth dθ 0 dθ 0 tha m(xjθ) ad ca be applied to such situatios as quatile estimatio where m(xjθ) is ot differetiable at θ 0. I this course, we will ot meet such cases. Pig Yu (HKU) Asymptotic Theory 13 / 20
14 Asymptotics for the MoM Estimator Ituitio for the Asymptotic Distributio of the MoM Estimator Suppose E [X ] = g(θ 0 ) with g 2 C (1) i a eighborhood of θ 0 ; the θ 0 = g 1 (E [X ]) h(e [X ]). (what are m,m ad Ω here?) The MoM estimator of θ is to set X = g(θ), so b θ = h(x ). By the WLLN, X p! E [X ]; the by the CMT, θ b p! h(e [X ]) = θ 0 sice h() is cotiuous. Now, p bθ θ 0 = p h(x ) h(e [X ]) = p h 0 X X E [X ] = h 0 X p X theorem (MVT). E [X ], where the secod equality is from the mea value Because X is betwee X ad E [X ] ad X p! E [X ], X p! E [X ]. By the CMT, h 0 X p! h 0 (E [X ]). By the CLT, p X E [X ] d! N(0,Var(X )). The by Slutsky s theorem, p bθ d! θ 0 h 0 (E [X ])N(0,Var(X )) = N 0,h 0 (E [X ]) 2?= Var(X ) N 0, Var(X ) g 0 (θ 0 ) 2. Pig Yu (HKU) Asymptotic Theory 14 / 20
15 Asymptotics for the MoM Estimator cotiue... The larger g 0 (θ 0 ) is, the smaller the asymptotic variace of θ b is. o Cosider a more specific example. Suppose the desity of X is θ 2 x exp x 2 θ, θ > 0, x > 0, that is, X follows the Weibull (2,θ) distributio. p We ca show E [X ] = g(θ) = π 2 θ 1/2 π, ad Var (X ) = θ 1 4. So p bθ d! θ(1 θ N 0, π 4 ) pπ θ 1/2 2! = N 0,16θ 2 1 π 1 4. Figure 1 shows E [X ] ad the asymptotic variace of p bθ θ. θ as a fuctio of Ituitively, the larger the derivative of E[X ] with respect to θ, the easier to idetify θ from X, so the smaller the asymptotic variace. Pig Yu (HKU) Asymptotic Theory 15 / 20
16 Asymptotics for the MoM Estimator Figure: E [X ] ad Asymptotic Variace as a Fuctio of θ Pig Yu (HKU) Asymptotic Theory 16 / 20
17 Asymptotics for the MoM Estimator A Example Suppose the momet coditios are X µ E (X µ) 2 σ 2 The the sample aalog is 0 so the solutio is 1 X i µ (X i µ) 2 σ 2 = 0. 1 C A = 0, bµ = X bσ 2 = 1 X i X 2 = X 2 X 2. Pig Yu (HKU) Asymptotic Theory 17 / 20
18 Asymptotics for the MoM Estimator cotiue... Cosistecy: bµ = X p! µ, bσ 2 = X 2 X 2 p! 1 0 Asymptotic Normality: M = E 2(X µ) 1 " Ω = E = µ 2 + σ 2 µ 2 = σ 2. = (X µ) 2 (X µ) 3 σ 2 (X µ) (X µ) 3 σ 2 (X µ) (X µ) 4 2σ 2 (X µ) 2 + σ 4 σ 2 E h(x µ) 3i 1 h E (X µ) 3i h E (X µ) 4i A, σ 4,!# so p bµ µ bσ 2 σ 2 If X N µ,σ 2, the what is Ω? d! N (0,Ω). Pig Yu (HKU) Asymptotic Theory 18 / 20
19 Asymptotics for the MoM Estimator Aother Example: Empirical Distributio Fuctio Suppose we wat to estimate θ = F (x) for a fixed x, where F () is the cdf of a radom variable X. A ituitive estimator is the ratio of samples below x, 1 1(X i x), which is called the empirical distributio fuctio (EDF), while it is a MoM estimator. Why? ote that the momet coditio for this problem is Its sample aalog is so E [1(X x) F (x)] = 0. 1 (1(X i x) F (x)) = 0, bf (x) = 1 1(X i x). By the WLLN, it is cosistet. By the CLT, p bf (x) F (x) d! N (0,F (x) (1 F (x))).(why?) A iterestig pheomeo is that the asymptotic variace reaches its maximum at the media of the distributio of X. Pig Yu (HKU) Asymptotic Theory 19 / 20
20 Asymptotics for the MoM Estimator Figure: Empirical Distributio Fuctios: 10 samples from N(0, 1) with sample size = 50 Pig Yu (HKU) Asymptotic Theory 20 / 20
Lecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationIntroductory statistics
CM9S: Machie Learig for Bioiformatics Lecture - 03/3/06 Itroductory statistics Lecturer: Sriram Sakararama Scribe: Sriram Sakararama We will provide a overview of statistical iferece focussig o the key
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationAsymptotics. Hypothesis Testing UMP. Asymptotic Tests and p-values
of the secod half Biostatistics 6 - Statistical Iferece Lecture 6 Fial Exam & Practice Problems for the Fial Hyu Mi Kag Apil 3rd, 3 Hyu Mi Kag Biostatistics 6 - Lecture 6 Apil 3rd, 3 / 3 Rao-Blackwell
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More information11 THE GMM ESTIMATION
Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationLecture 15: Strong, Conditional, & Joint Typicality
EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationLast Lecture. Biostatistics Statistical Inference Lecture 16 Evaluation of Bayes Estimator. Recap - Example. Recap - Bayes Estimator
Last Lecture Biostatistics 60 - Statistical Iferece Lecture 16 Evaluatio of Bayes Estimator Hyu Mi Kag March 14th, 013 What is a Bayes Estimator? Is a Bayes Estimator the best ubiased estimator? Compared
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationPoint Estimation: properties of estimators 1 FINITE-SAMPLE PROPERTIES. finite-sample properties (CB 7.3) large-sample properties (CB 10.
Poit Estimatio: properties of estimators fiite-sample properties CB 7.3) large-sample properties CB 10.1) 1 FINITE-SAMPLE PROPERTIES How a estimator performs for fiite umber of observatios. Estimator:
More informationNotes On Median and Quantile Regression. James L. Powell Department of Economics University of California, Berkeley
Notes O Media ad Quatile Regressio James L. Powell Departmet of Ecoomics Uiversity of Califoria, Berkeley Coditioal Media Restrictios ad Least Absolute Deviatios It is well-kow that the expected value
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationElements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationLecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More information4. Basic probability theory
Cotets Basic cocepts Discrete radom variables Discrete distributios (br distributios) Cotiuous radom variables Cotiuous distributios (time distributios) Other radom variables Lect04.ppt S-38.45 - Itroductio
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationTAMS24: Notations and Formulas
TAMS4: Notatios ad Formulas Basic otatios ad defiitios X: radom variable stokastiska variabel Mea Vätevärde: µ = X = by Xiagfeg Yag kpx k, if X is discrete, xf Xxdx, if X is cotiuous Variace Varias: =
More informationLeast Squares Estimation- Large-Sample Properties
Least Squares Estimatio- Large-Sample Properties Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Large-Sample 1 / 63 1 Asymptotics for the LSE 2 Covariace Matrix Estimators 3 Fuctios
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationSTATISTICAL METHODS FOR BUSINESS
STATISTICAL METHODS FOR BUSINESS UNIT 5. Joit aalysis ad limit theorems. 5.1.- -dimesio distributios. Margial ad coditioal distributios 5.2.- Sequeces of idepedet radom variables. Properties 5.3.- Sums
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationCEU Department of Economics Econometrics 1, Problem Set 1 - Solutions
CEU Departmet of Ecoomics Ecoometrics, Problem Set - Solutios Part A. Exogeeity - edogeeity The liear coditioal expectatio (CE) model has the followig form: We would like to estimate the effect of some
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete
More informationLecture Note 8 Point Estimators and Point Estimation Methods. MIT Spring 2006 Herman Bennett
Lecture Note 8 Poit Estimators ad Poit Estimatio Methods MIT 14.30 Sprig 2006 Herma Beett Give a parameter with ukow value, the goal of poit estimatio is to use a sample to compute a umber that represets
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationRank tests and regression rank scores tests in measurement error models
Rak tests ad regressio rak scores tests i measuremet error models J. Jurečková ad A.K.Md.E. Saleh Charles Uiversity i Prague ad Carleto Uiversity i Ottawa Abstract The rak ad regressio rak score tests
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More informationRates of Convergence by Moduli of Continuity
Rates of Covergece by Moduli of Cotiuity Joh Duchi: Notes for Statistics 300b March, 017 1 Itroductio I this ote, we give a presetatio showig the importace, ad relatioship betwee, the modulis of cotiuity
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationSummary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square
More informationBull. Korean Math. Soc. 36 (1999), No. 3, pp. 451{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Seung Hoe Choi and Hae Kyung
Bull. Korea Math. Soc. 36 (999), No. 3, pp. 45{457 THE STRONG CONSISTENCY OF NONLINEAR REGRESSION QUANTILES ESTIMATORS Abstract. This paper provides suciet coditios which esure the strog cosistecy of regressio
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationStatistical Theory MT 2009 Problems 1: Solution sketches
Statistical Theory MT 009 Problems : Solutio sketches. Which of the followig desities are withi a expoetial family? Explai your reasoig. (a) Let 0 < θ < ad put f(x, θ) = ( θ)θ x ; x = 0,,,... (b) (c) where
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture 27: Optimal Estimators and Functional Delta Method
Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationStudy the bias (due to the nite dimensional approximation) and variance of the estimators
2 Series Methods 2. Geeral Approach A model has parameters (; ) where is ite-dimesioal ad is oparametric. (Sometimes, there is o :) We will focus o regressio. The fuctio is approximated by a series a ite
More information1.010 Uncertainty in Engineering Fall 2008
MIT OpeCourseWare http://ocw.mit.edu.00 Ucertaity i Egieerig Fall 2008 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu.terms. .00 - Brief Notes # 9 Poit ad Iterval
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationEFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS
EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS Ryszard Zieliński Ist Math Polish Acad Sc POBox 21, 00-956 Warszawa 10, Polad e-mail: rziel@impagovpl ABSTRACT Weak laws of large umbers (W LLN), strog
More information