Notes on Asymptotic Theory: Convergence in Probability and Distribution Introduction to Econometric Theory Econ. 770
|
|
- Phillip Andrew Carroll
- 6 years ago
- Views:
Transcription
1 Notes on Asymptotic Theory: Convergence in Probability and Distribution Introduction to Econometric Theory Econ. 770 Jonathan B. Hill Dept. of Economics University of North Carolina - Chapel Hill November 9, 0 Introduction Let ( F ) be a probability space. Throughout is a parameter of interest like the mean, variance, correlation, or distribution parameters like Poisson, Binomial, or exponential. Throughout f^ g is a sequence of estimators of based on a sample of data f g = with sample size. Assume ^ is F-measurable for any. Unless otherwise noted, assume the 0 have the same mean and variance:» ( ). If appropriate, we may have a bivariate sample f g = where» ( ) and» ( ). Examples include the sample mean, variance, or correlation: Sample Mean : ¹ := X = Sample Variance # : := Sample Variance # : ^ := X ( ) = X ¹ = Sample Correlation : ^ := P = ¹ ¹ ^ ^ Similarly, we may estimate a probability by using a sample relative frequency: ^ () = X ( ) the sample percentage of =
2 Notice ^ () estimates ( ). We will look at estimator properties: what ^ is on average for any sample size; and what ^ becomes as the sample size grows. In every case above the estimator is a variant of a straight average (e.g. P = ¹ ¹ is a straight average of ¹ ¹ ), or a function of a straight average (e.g. ^ := ( P = ¹ ), the square root of the average ¹ ). We therefore pay particular attention to the sample mean. Unbiasedness Defn. We say ^ is an unbiased estimator of if [^ ] =. De ne bias as ³^ B := [^ ] An unbiased estimator has zero bias: B(^ ) = 0. If we had an in nite number of samples of size, then the average estimate ^ across all samples would be. An asymptotically unbiased estimator satis es B(^ )! 0 as!. Claim (Weighted Average): Let have a common mean := [ ]. Then the weighted average ^ := P P = is an unbiased estimator of := [] if = =. " X # = = X [ ] = = X = QED. Corollary (Straight Average): The sample mean ¹ := P = is a weighted average with at or uniform weights = hence trivially P = = hence [ ] ¹ = The problem then arises as to which weighted average P = may be preferred in practice since any with unit summed weights is unbiased. We will discuss below the concept of e ciency below, but the minimum mean-squared-error unbiased estimator has uniform weights if» ( ). That is: Claim (Sample Mean is Best): Let» ( ). Then ¹ is the best linear unbiased estimator of (i.e. it is BLUE). The Lagrange is = We want to solve à X! min subject to = X = = à X! à L ( ) := + =! X =
3 where by independence ( P = ) = P =, hence X L ( ) := Ã + =! X = The rst order conditions are L ( ) = = 0 and L ( ) = X = 0 = Therefore = ( ) is a constant that sums to P = =. Write = ( ) =:. Since P = = P = = = it follows = =. QED. Remark: As in many cases here and below, independence can be substituted for uncorrelatedness since the same proof applies: [ ] = [ ][ ] for all 6=. We can also substitute uncorrelatedness with a condition that restricts the total correlation across all and for 6=, but such generality is typically only exploited in time series settings (where is at a di erent time period). Claim (Sample Variance): Let» ( ). The estimator is unbiased and ^ is negatively biased but asymptotically unbiased. Notice = ^ = X ¹ X = + ¹ = = = X ( ) + X ¹ X + ( ) ¹ = = = = X ( ) + ¹ ¹ X ( ) = = = X ( ) + ¹ ¹ ¹ = = X ( ) ¹ = By the iid assumption and the fact that ¹ is unbiased Ã! ¹ X = = X ( ) = = = = Further, by de nition := [( ) ] hence " # X ( ) = X h ( ) i = X = = 3 = =
4 Therefore = ^ = = This implies each claim: = ( is unbiased), ^ = ( ) (^ is negatively biased), and ^ = ( )! (^ is asymptotically unbiased). QED. Example: We simulate 00 samples of» (754) with sample size = 0. In Figure we plot ¹ for each sample. The simulation average of all ¹ is and the simulation variance of all ¹ is In Figure we plot ^ = P = for each sample with weights = P =. The simulation average of all ^ is and the simulation variance of all ^ is Thus, both display the same property of unbiasedness, but ¹ exhibits less dispersion across samples Figure : ¹ Figure : ^ 3 Convergence in Mean-Square or L -Convergence Defn. We say ^ R converges to in mean-square if We also write If ^ is unbiased for then MSE(^ ) := ³^! 0 ^! and ^! in mean-square. MSE(^ ) = ³^ h^ i = h^ i Convergence in mean-square certainly does not require unbiasedness. In the, MSE is i i MSE(^ ) = ³^ = ³^ h^ + h^ i ³ i i ³ i = ³^ h^ + h^ + ³^ h^ h^ i ³ i = ³^ h^ + h^ 4
5 i i i since h^ is just a constant and ³^ h^ = [^ ] h^ = 0. Hence MSE is the variance plus bias squared: i ³ i i ³ ³^ MSE(^ ) = ³^ h^ + h^ = h^ + B If ^ R then we write MSE(^ ) := ³^ ³^ 0! 0 hence component wise convergence. We may similarly write convergence in -norm ³^ ³^ 0 0 X! 0 where kk X = = or convergence in matrix (spectral) norm: ³^ ³^ 0! 0 where kk is the largest eigenvalue of. A Both imply convergence with respect to each element ³^! 0. Defn. We say ^ R has the property of -convergence, or convergence in -norm, to if for 0 ^! 0 Clearly -convergence and mean-square convergence are equivalent. Claim (Sample Mean): Let» ( ). Then ¹! in mean square. ( ¹ ) = [ ¹ ] =! 0 QED. We only require uncorrelatedness since [ ¹ ] = still holds. Claim (Sample Mean): mean square. Let» ( ) be uncorrelated. Then ¹! in ( ¹ ) = [ ¹ ] =! 0 QED. In fact, we only need all cross covariances to not be too large as the sample size grows. Claim (Sample Mean): Let» ( ) satisfy P ( )! 0. Then ¹! in mean square. ( ¹ ) = [ ¹ ] = + P ( )! 0 QED. Remark: In micro-economic contexts involving cross-sectional data this type of correlatedness is evidently rarely or never entertained. Typically we assume the 0 are uncorrelated. It is, however, profoundly popular in macroeconomic and 5
6 nance contexts where data are time series. A very large class of time series random variables satis es both ( ) 6= 0 8 6= and P ( )! 0, and therefore exhibits ¹! in mean square. If» ( ) then ¹! in -norm for any (] but proving the result for non-integer ( ) is quite a bit more di cult. There are many types of "maximal inequalities", however, that can be used to prove X for ( ) where 0 is a nite constant. = Claim (Sample Mean): any ( ). X = since QED. Let» ( ) be iid. Then ¹! in -norm for = X f g = =! 0 Example: We simulate» (7400) with sample sizes = In Figure 3 we plot ¹ and [ ] ¹ = 400 over sample size. Notice the high volatility for small. Figure 3: ¹ and [ ¹] 4 Convergence in Probability : WLLN Defn. We say ^ converges in probability to if lim ³ ^ = ()! We variously write ^! and ^! 6
7 and we say ^ is a consistent estimator of. Since probability convergence is convergence in the sequence f(j^ j )g =, by the de nition of a limit it follows for every 0 there exists 0 such that ³ ^ 8 That is, for a large enough sample size ^ is guaranteed to be as close to as we choose (i.e. the ) with as a great a probability as we choose (i.e. ). Claim (Law of Large Numbers = LLN): If» ( ) then ¹!. By Chebyshev s inequality and independence, for any 0 ¹ ¹ =! 0 QED Remark : We call this a Weak Law of Large Numbers [WLLN] since convergence is in probability. A Strong LLN based on a stronger form of convergence is given below. Remark : We only need uncorrelatedness to get ¹ =! 0. The WLLN, however extends to many forms of dependent random variables. Remark 3: In the iid case we only need j j, although the proof is substantially more complicated. Even for non-iid data we typically only need j j + for in nitessimal 0 (pay close attention to scholarly articles you read, and to your own assumptions: usually far stronger assumptions are imposed than are actually required). The weighted average P = is also consistent as long as the weights decay with the sample size. Thus we write the weight as. Claim: If» ( ) then P =! if P = = and P =! 0. By Chebyshev s inequality, independence and P = =, for any 0! Ã X X Ã X Ã! = f g = which proves the claim. QED. = X = = = h( ) i X =! 0 An example is ¹ with =, but also the weights = P Figure. = =! used in Example: We simulate» (75 0) with sample sizes = In Figures 4 and 5 we plot ¹ and ^ = P = over sample size. Notice the high volatility for small. 7
8 Figure 4 : ¹ Figure 5 : ^ Sample Size n Sample Size n Claim (Slutsky Theorem): Let ^ R. If ^! and : R! R is continuous (except possibly with countably many discontinuity points) then (^ )! (). Corollary: Let ^!, =. Then ^ ^!, ^ ^!, and if 6= 0 and liminf! j^ j 0 then ^ ^!. Claim: If» ( ) and [ 4] then!. Note = X = ¹ = X ( ) ¹ By LLN ¹!, therefore by the Slutsky Theorem ¹! 0. By [ 4 ] it follows ( ) is iid with a nite variance, hence it satis es the LLN: P = ( )! [( ) ] =. QED. Claim: = If» ( ) and [ j then the sample correlation ^! the population correlation. Example: We simulate» (7 400) and» (0900) and construct = The true correlation is = [ ] [ ] [ ] = 43 [ ] + 7 ( ) 0 p = ( ) 0 p = 8 We estimate correlation for samples with size = Figure 6 demonstrates consistency and therefore the Slutsky Theorem. 8
9 .00 Figure 6: Correlation Sample Size n 5 Almost Sure Convergence : SLLN Defn. We say ^ converges almost surely to if ³ ^ = = lim! This is identical to µ lim max ^ = 0 8 0! We variously write and we say ^ is strongly consistent for. We have the following relationships. Claim: ^! and ^! ^! implies ^! ;. ^! implies ^!. (j^ j ) (^ ) by Chebyshev s inequality. If (^ )! 0 (i.e. ^! ) then (j^ j )! 0 where 0 is arbitrary. Therefore ^!. (j^ j ) (sup j^ j ) since sup j^ j j^ j. Therefore if (sup j^ j )! (i.e. ^! ) then (j^ j )! (i.e. ^! ). QED. If ^ is bounded wp then ^! if and only if [^ ]! which is asymptotic unbiasedness (see Bierens). By the Slutsky Theorem ^! implies (^ )! 0 hence [(^ ) ]! 0: convergence in probability implies convergence in mean-square. This proves the following (and gives almost sure convergence as the "strongest" form: the one that implies all the rest). Claim (a.s. =) i.p. =) m.s.): Let ^ be bounded wp: (j^ j ) = 9
10 for nite 0. Then ^! implies ^! implies asymptotic unbiasedness and ^!. Claim (Strong Law of Large Numbers = SLLN):!. Remark: Example: If» ( ) then ¹ The Slutsky Theorem carries over to strong convergence. Let» ( ) and de ne ^ := + ¹ Then (j^ j ) =. Moreover, under the iid assumption ¹! by the SLLN, hence by the Slutsky Theorem ^! + Therefore and [^ ]! = ( + ) and ^! + ³^! 0 6 Convergence in Distribution : CLT Defn. We say ^ converges in distribution to a distribution, or to a random variable with distribution, if lim ³^ = () for every on the support.! Thus, while ^ may itself not be distributed, asymptotically it is. We write ^! or ^! where». The notation ^! is a bit awkward, because characterizes in nitely many random variables. We are therefore saying there is some random draw from that ^ is becoming. Which random draw is not speci ed. 6. Central Limit Theorem By far the most famous result concerns the sample mean ¹. Convergence of some estimator ^ in a monumentally large number of cases reduces to convergence of a sample mean of something, call it. This carries over to the sample correlation, regression model estimation methods like Ordinary Least Squares, GMM, and Maximum Likelihood, as well as non-parametric estimation, and on and on. 0
11 As usual, we limit ourselves to the iid case. The following substantially carries over to non-iid data, and based on a rarely cited obscure fact does not even require a nite variance (I challenge you to nd a proof of this, or to ever discover any econometrics textbook that accurately states this). Claim (Central Limit Theorem = CLT): := p ¹! (0 ) If» ( ) then Remark : This is famously cites as the Lindeberg Lévy CLT. Historically, however, the proof arose in di erent camps sometime between (covering Lindeberg, Lévy, Chebyshev, Markov and Lyapunov). Remark : Notice by construction := p ¹ is a standardized sample mean because [ ] ¹ = by identical distributedness and [ ] ¹ = by independence and identical distributedness. Thus := p ¹ ¹ = p = ¹ [ ] ¹ [ ] ¹ Therefore p ¹ has mean 0 and variance : " # p ¹ p = ¹ = 0 " # p ¹ = ¹ = = Thus, even as! the random variable» (0). Although this is a long way from proving has a de nable distribution, even in the limit, it does help to point out that the term p! is necessary to stabilize, ¹ for otherwise we simply have ¹! 0. Remark 3: Asymptotically := p ¹ has a standard normal density () expf g. De ne := ( ), hence := p ¹ X = p We will show the characteristic function [ ]!. The latter is the characteristic function of a standard normal, while characteristic functions and distributions have a unique correspondence: only standard normals have a characteristic function like. =
12 By independence and identical distributednessnow expand around = 0 by a second order Taylor expansion: " i h = h i # Y = = () = Y = = i ³ h i h = = +! +! + = +!! + where is a remainder term that is a function of. Now take the expectations as in (), and note [ ] = [( )] = 0 and [ ] = [( ) ] = = : i h = + [ ]!! + [ ] = + where := [ ] It is easy to prove is a bounded random variable, in particular j j wp (see Bierens) so even if does not have higher moments we know j j. Further! 0 because [ ]!. Now take the -power in (): by the Binomial expansion ³ i h = = µ µ The rst term satis es µ + + = X = X µ µ µ =0 µ! because the sequence f( + ) g converges: ( )! (simply put = ). For the second term notice for large enough we have j j hence X µ µ X µ X µ = ( + ) = = =0
13 See Bierens for details that verify ( + )! 0. QED. Example (Bernoulli): The most striking way to demonstrate the CLT is to begin with the least normal of data, a Bernoulli random variable which is discrete and takes only two nite values, and show p ¹! (0 ), a continuous random variable with in nite support. We simulate» () for = 5, , 0000 and compute ¹ ¹ ¹ := p = p p = p 8 In order to show the small sample distribution of we need a sample of, 0 so we repeat the simulation 000 times. We plot the relative frequencies of the sample of 0 for each. Let f g 000 = be the simulated sample of 0. The relative frequencies are the percentage 000 P 000 = ( + ) for interval endpoints = [ ]. See Figure 7. For the sake of comparison in Figure 8 we plot the relative frequencies for one sample of 000 iid standard normal random variables» (0 ). Another way to see how becomes a standard normal random variable is to compute the quantile such that ( ) = 975. A standard normal satis es ( 96) = 975. We call an empirical quantile since it is based on a simulated set of samples. We simulate 0,000 samples for each size = , 5005 and compute. See Figure 9. As increases! 96. Figure 7 Standardized Means for Bernoulli , = , = , = , =
14 Figure 8 Standard Normal Standard Normal Figure 9 - Empirical Quantiles q Sample Size n 4
Notes on Mathematical Expectations and Classes of Distributions Introduction to Econometric Theory Econ. 770
Notes on Mathematical Expectations and Classes of Distributions Introduction to Econometric Theory Econ. 77 Jonathan B. Hill Dept. of Economics University of North Carolina - Chapel Hill October 4, 2 MATHEMATICAL
More informationEconomics 241B Review of Limit Theorems for Sequences of Random Variables
Economics 241B Review of Limit Theorems for Sequences of Random Variables Convergence in Distribution The previous de nitions of convergence focus on the outcome sequences of a random variable. Convergence
More informationEconomics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 9: Asymptotics III(MLE) 1 / 20 Jensen
More informationEconomics 583: Econometric Theory I A Primer on Asymptotics
Economics 583: Econometric Theory I A Primer on Asymptotics Eric Zivot January 14, 2013 The two main concepts in asymptotic theory that we will use are Consistency Asymptotic Normality Intuition consistency:
More informationLecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
More informationChapter 1. GMM: Basic Concepts
Chapter 1. GMM: Basic Concepts Contents 1 Motivating Examples 1 1.1 Instrumental variable estimator....................... 1 1.2 Estimating parameters in monetary policy rules.............. 2 1.3 Estimating
More informationA Course on Advanced Econometrics
A Course on Advanced Econometrics Yongmiao Hong The Ernest S. Liu Professor of Economics & International Studies Cornell University Course Introduction: Modern economies are full of uncertainties and risk.
More informationEconomics 620, Lecture 8: Asymptotics I
Economics 620, Lecture 8: Asymptotics I Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 1 / 17 We are interested in the properties of estimators
More informationEstimation, Inference, and Hypothesis Testing
Chapter 2 Estimation, Inference, and Hypothesis Testing Note: The primary reference for these notes is Ch. 7 and 8 of Casella & Berger 2. This text may be challenging if new to this topic and Ch. 7 of
More informationFunction Approximation
1 Function Approximation This is page i Printer: Opaque this 1.1 Introduction In this chapter we discuss approximating functional forms. Both in econometric and in numerical problems, the need for an approximating
More informationRegression #3: Properties of OLS Estimator
Regression #3: Properties of OLS Estimator Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #3 1 / 20 Introduction In this lecture, we establish some desirable properties associated with
More informationSTAT 7032 Probability. Wlodek Bryc
STAT 7032 Probability Wlodek Bryc Revised for Spring 2019 Printed: January 14, 2019 File: Grad-Prob-2019.TEX Department of Mathematical Sciences, University of Cincinnati, Cincinnati, OH 45221 E-mail address:
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationSIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER. Donald W. K. Andrews. August 2011
SIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER By Donald W. K. Andrews August 2011 COWLES FOUNDATION DISCUSSION PAPER NO. 1815 COWLES FOUNDATION FOR RESEARCH IN ECONOMICS
More informationEconomics 620, Lecture 20: Generalized Method of Moment (GMM)
Economics 620, Lecture 20: Generalized Method of Moment (GMM) Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 20: GMM 1 / 16 Key: Set sample moments equal to theoretical
More informationLarge Sample Theory. Consider a sequence of random variables Z 1, Z 2,..., Z n. Convergence in probability: Z n
Large Sample Theory In statistics, we are interested in the properties of particular random variables (or estimators ), which are functions of our data. In ymptotic analysis, we focus on describing the
More informationLimiting Distributions
Limiting Distributions We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the
More informationA General Overview of Parametric Estimation and Inference Techniques.
A General Overview of Parametric Estimation and Inference Techniques. Moulinath Banerjee University of Michigan September 11, 2012 The object of statistical inference is to glean information about an underlying
More informationLECTURE 12 UNIT ROOT, WEAK CONVERGENCE, FUNCTIONAL CLT
MARCH 29, 26 LECTURE 2 UNIT ROOT, WEAK CONVERGENCE, FUNCTIONAL CLT (Davidson (2), Chapter 4; Phillips Lectures on Unit Roots, Cointegration and Nonstationarity; White (999), Chapter 7) Unit root processes
More informationP n. This is called the law of large numbers but it comes in two forms: Strong and Weak.
Large Sample Theory Large Sample Theory is a name given to the search for approximations to the behaviour of statistical procedures which are derived by computing limits as the sample size, n, tends to
More information1 Appendix A: Matrix Algebra
Appendix A: Matrix Algebra. Definitions Matrix A =[ ]=[A] Symmetric matrix: = for all and Diagonal matrix: 6=0if = but =0if 6= Scalar matrix: the diagonal matrix of = Identity matrix: the scalar matrix
More informationLimiting Distributions
We introduce the mode of convergence for a sequence of random variables, and discuss the convergence in probability and in distribution. The concept of convergence leads us to the two fundamental results
More informationThe properties of L p -GMM estimators
The properties of L p -GMM estimators Robert de Jong and Chirok Han Michigan State University February 2000 Abstract This paper considers Generalized Method of Moment-type estimators for which a criterion
More informationTime Series Models and Inference. James L. Powell Department of Economics University of California, Berkeley
Time Series Models and Inference James L. Powell Department of Economics University of California, Berkeley Overview In contrast to the classical linear regression model, in which the components of the
More informationV. Properties of estimators {Parts C, D & E in this file}
A. Definitions & Desiderata. model. estimator V. Properties of estimators {Parts C, D & E in this file}. sampling errors and sampling distribution 4. unbiasedness 5. low sampling variance 6. low mean squared
More informationSTATISTICS/ECONOMETRICS PREP COURSE PROF. MASSIMO GUIDOLIN
Massimo Guidolin Massimo.Guidolin@unibocconi.it Dept. of Finance STATISTICS/ECONOMETRICS PREP COURSE PROF. MASSIMO GUIDOLIN SECOND PART, LECTURE 2: MODES OF CONVERGENCE AND POINT ESTIMATION Lecture 2:
More informationIntroduction to the Mathematical and Statistical Foundations of Econometrics Herman J. Bierens Pennsylvania State University
Introduction to the Mathematical and Statistical Foundations of Econometrics 1 Herman J. Bierens Pennsylvania State University November 13, 2003 Revised: March 15, 2004 2 Contents Preface Chapter 1: Probability
More information11. Bootstrap Methods
11. Bootstrap Methods c A. Colin Cameron & Pravin K. Trivedi 2006 These transparencies were prepared in 20043. They can be used as an adjunct to Chapter 11 of our subsequent book Microeconometrics: Methods
More informationRobust Estimation and Inference for Extremal Dependence in Time Series. Appendix C: Omitted Proofs and Supporting Lemmata
Robust Estimation and Inference for Extremal Dependence in Time Series Appendix C: Omitted Proofs and Supporting Lemmata Jonathan B. Hill Dept. of Economics University of North Carolina - Chapel Hill January
More informationChapter 6. Convergence. Probability Theory. Four different convergence concepts. Four different convergence concepts. Convergence in probability
Probability Theory Chapter 6 Convergence Four different convergence concepts Let X 1, X 2, be a sequence of (usually dependent) random variables Definition 1.1. X n converges almost surely (a.s.), or with
More informationProbability for Statistics and Machine Learning
~Springer Anirban DasGupta Probability for Statistics and Machine Learning Fundamentals and Advanced Topics Contents Suggested Courses with Diffe~ent Themes........................... xix 1 Review of Univariate
More informationFinite Population Sampling and Inference
Finite Population Sampling and Inference A Prediction Approach RICHARD VALLIANT ALAN H. DORFMAN RICHARD M. ROYALL A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane
More informationParametric Inference on Strong Dependence
Parametric Inference on Strong Dependence Peter M. Robinson London School of Economics Based on joint work with Javier Hualde: Javier Hualde and Peter M. Robinson: Gaussian Pseudo-Maximum Likelihood Estimation
More informationRegression #4: Properties of OLS Estimator (Part 2)
Regression #4: Properties of OLS Estimator (Part 2) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #4 1 / 24 Introduction In this lecture, we continue investigating properties associated
More informationSTAT 7032 Probability Spring Wlodek Bryc
STAT 7032 Probability Spring 2018 Wlodek Bryc Created: Friday, Jan 2, 2014 Revised for Spring 2018 Printed: January 9, 2018 File: Grad-Prob-2018.TEX Department of Mathematical Sciences, University of Cincinnati,
More informationElements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the
More information6.1 Moment Generating and Characteristic Functions
Chapter 6 Limit Theorems The power statistics can mostly be seen when there is a large collection of data points and we are interested in understanding the macro state of the system, e.g., the average,
More informationIntroduction: structural econometrics. Jean-Marc Robin
Introduction: structural econometrics Jean-Marc Robin Abstract 1. Descriptive vs structural models 2. Correlation is not causality a. Simultaneity b. Heterogeneity c. Selectivity Descriptive models Consider
More informationGMM-based inference in the AR(1) panel data model for parameter values where local identi cation fails
GMM-based inference in the AR() panel data model for parameter values where local identi cation fails Edith Madsen entre for Applied Microeconometrics (AM) Department of Economics, University of openhagen,
More informationδ -method and M-estimation
Econ 2110, fall 2016, Part IVb Asymptotic Theory: δ -method and M-estimation Maximilian Kasy Department of Economics, Harvard University 1 / 40 Example Suppose we estimate the average effect of class size
More informationProduct measure and Fubini s theorem
Chapter 7 Product measure and Fubini s theorem This is based on [Billingsley, Section 18]. 1. Product spaces Suppose (Ω 1, F 1 ) and (Ω 2, F 2 ) are two probability spaces. In a product space Ω = Ω 1 Ω
More informationProblem set 1 - Solutions
EMPIRICAL FINANCE AND FINANCIAL ECONOMETRICS - MODULE (8448) Problem set 1 - Solutions Exercise 1 -Solutions 1. The correct answer is (a). In fact, the process generating daily prices is usually assumed
More informationQuantitative Techniques - Lecture 8: Estimation
Quantitative Techniques - Lecture 8: Estimation Key words: Estimation, hypothesis testing, bias, e ciency, least squares Hypothesis testing when the population variance is not known roperties of estimates
More informationSampling Distributions and Asymptotics Part II
Sampling Distributions and Asymptotics Part II Insan TUNALI Econ 511 - Econometrics I Koç University 18 October 2018 I. Tunal (Koç University) Week 5 18 October 2018 1 / 29 Lecture Outline v Sampling Distributions
More informationSTATISTICS SYLLABUS UNIT I
STATISTICS SYLLABUS UNIT I (Probability Theory) Definition Classical and axiomatic approaches.laws of total and compound probability, conditional probability, Bayes Theorem. Random variable and its distribution
More informationPanel Data. March 2, () Applied Economoetrics: Topic 6 March 2, / 43
Panel Data March 2, 212 () Applied Economoetrics: Topic March 2, 212 1 / 43 Overview Many economic applications involve panel data. Panel data has both cross-sectional and time series aspects. Regression
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationGraduate Econometrics I: Maximum Likelihood I
Graduate Econometrics I: Maximum Likelihood I Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
More informationStatistical Methods in HYDROLOGY CHARLES T. HAAN. The Iowa State University Press / Ames
Statistical Methods in HYDROLOGY CHARLES T. HAAN The Iowa State University Press / Ames Univariate BASIC Table of Contents PREFACE xiii ACKNOWLEDGEMENTS xv 1 INTRODUCTION 1 2 PROBABILITY AND PROBABILITY
More informationEconomics 241B Estimation with Instruments
Economics 241B Estimation with Instruments Measurement Error Measurement error is de ned as the error resulting from the measurement of a variable. At some level, every variable is measured with error.
More informationNonconcave Penalized Likelihood with A Diverging Number of Parameters
Nonconcave Penalized Likelihood with A Diverging Number of Parameters Jianqing Fan and Heng Peng Presenter: Jiale Xu March 12, 2010 Jianqing Fan and Heng Peng Presenter: JialeNonconcave Xu () Penalized
More informationSIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER. Donald W. K. Andrews. August 2011 Revised March 2012
SIMILAR-ON-THE-BOUNDARY TESTS FOR MOMENT INEQUALITIES EXIST, BUT HAVE POOR POWER By Donald W. K. Andrews August 2011 Revised March 2012 COWLES FOUNDATION DISCUSSION PAPER NO. 1815R COWLES FOUNDATION FOR
More informationTwo hours. To be supplied by the Examinations Office: Mathematical Formula Tables THE UNIVERSITY OF MANCHESTER. 21 June :45 11:45
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS 21 June 2010 9:45 11:45 Answer any FOUR of the questions. University-approved
More informationA Primer on Asymptotics
A Primer on Asymptotics Eric Zivot Department of Economics University of Washington September 30, 2003 Revised: October 7, 2009 Introduction The two main concepts in asymptotic theory covered in these
More informationECON 616: Lecture 1: Time Series Basics
ECON 616: Lecture 1: Time Series Basics ED HERBST August 30, 2017 References Overview: Chapters 1-3 from Hamilton (1994). Technical Details: Chapters 2-3 from Brockwell and Davis (1987). Intuition: Chapters
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More informationThe main results about probability measures are the following two facts:
Chapter 2 Probability measures The main results about probability measures are the following two facts: Theorem 2.1 (extension). If P is a (continuous) probability measure on a field F 0 then it has a
More informationChapter 2. Dynamic panel data models
Chapter 2. Dynamic panel data models School of Economics and Management - University of Geneva Christophe Hurlin, Université of Orléans University of Orléans April 2018 C. Hurlin (University of Orléans)
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationEcon 424 Time Series Concepts
Econ 424 Time Series Concepts Eric Zivot January 20 2015 Time Series Processes Stochastic (Random) Process { 1 2 +1 } = { } = sequence of random variables indexed by time Observed time series of length
More informationEconometrics I, Estimation
Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the
More informationSTA205 Probability: Week 8 R. Wolpert
INFINITE COIN-TOSS AND THE LAWS OF LARGE NUMBERS The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and
More informationThe Central Limit Theorem: More of the Story
The Central Limit Theorem: More of the Story Steven Janke November 2015 Steven Janke (Seminar) The Central Limit Theorem:More of the Story November 2015 1 / 33 Central Limit Theorem Theorem (Central Limit
More informationMarkov-Switching Models with Endogenous Explanatory Variables. Chang-Jin Kim 1
Markov-Switching Models with Endogenous Explanatory Variables by Chang-Jin Kim 1 Dept. of Economics, Korea University and Dept. of Economics, University of Washington First draft: August, 2002 This version:
More informationBusiness Economics BUSINESS ECONOMICS. PAPER No. : 8, FUNDAMENTALS OF ECONOMETRICS MODULE No. : 3, GAUSS MARKOV THEOREM
Subject Business Economics Paper No and Title Module No and Title Module Tag 8, Fundamentals of Econometrics 3, The gauss Markov theorem BSE_P8_M3 1 TABLE OF CONTENTS 1. INTRODUCTION 2. ASSUMPTIONS OF
More information1 Correlation between an independent variable and the error
Chapter 7 outline, Econometrics Instrumental variables and model estimation 1 Correlation between an independent variable and the error Recall that one of the assumptions that we make when proving the
More informationECO Class 6 Nonparametric Econometrics
ECO 523 - Class 6 Nonparametric Econometrics Carolina Caetano Contents 1 Nonparametric instrumental variable regression 1 2 Nonparametric Estimation of Average Treatment Effects 3 2.1 Asymptotic results................................
More informationStochastic Convergence, Delta Method & Moment Estimators
Stochastic Convergence, Delta Method & Moment Estimators Seminar on Asymptotic Statistics Daniel Hoffmann University of Kaiserslautern Department of Mathematics February 13, 2015 Daniel Hoffmann (TU KL)
More informationLecture 18: Central Limit Theorem. Lisa Yan August 6, 2018
Lecture 18: Central Limit Theorem Lisa Yan August 6, 2018 Announcements PS5 due today Pain poll PS6 out today Due next Monday 8/13 (1:30pm) (will not be accepted after Wed 8/15) Programming part: Java,
More informationStatistical inference
Statistical inference Contents 1. Main definitions 2. Estimation 3. Testing L. Trapani MSc Induction - Statistical inference 1 1 Introduction: definition and preliminary theory In this chapter, we shall
More information1 of 7 7/16/2009 6:12 AM Virtual Laboratories > 7. Point Estimation > 1 2 3 4 5 6 1. Estimators The Basic Statistical Model As usual, our starting point is a random experiment with an underlying sample
More informationMax. Likelihood Estimation. Outline. Econometrics II. Ricardo Mora. Notes. Notes
Maximum Likelihood Estimation Econometrics II Department of Economics Universidad Carlos III de Madrid Máster Universitario en Desarrollo y Crecimiento Económico Outline 1 3 4 General Approaches to Parameter
More informationMeasuring robustness
Measuring robustness 1 Introduction While in the classical approach to statistics one aims at estimates which have desirable properties at an exactly speci ed model, the aim of robust methods is loosely
More information1. (Regular) Exponential Family
1. (Regular) Exponential Family The density function of a regular exponential family is: [ ] Example. Poisson(θ) [ ] Example. Normal. (both unknown). ) [ ] [ ] [ ] [ ] 2. Theorem (Exponential family &
More informationGMM estimation of spatial panels
MRA Munich ersonal ReEc Archive GMM estimation of spatial panels Francesco Moscone and Elisa Tosetti Brunel University 7. April 009 Online at http://mpra.ub.uni-muenchen.de/637/ MRA aper No. 637, posted
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Stochastic Convergence Barnabás Póczos Motivation 2 What have we seen so far? Several algorithms that seem to work fine on training datasets: Linear regression
More informationMarch 1, Florida State University. Concentration Inequalities: Martingale. Approach and Entropy Method. Lizhe Sun and Boning Yang.
Florida State University March 1, 2018 Framework 1. (Lizhe) Basic inequalities Chernoff bounding Review for STA 6448 2. (Lizhe) Discrete-time martingales inequalities via martingale approach 3. (Boning)
More information1 The Multiple Regression Model: Freeing Up the Classical Assumptions
1 The Multiple Regression Model: Freeing Up the Classical Assumptions Some or all of classical assumptions were crucial for many of the derivations of the previous chapters. Derivation of the OLS estimator
More informationSTAT 302 Introduction to Probability Learning Outcomes. Textbook: A First Course in Probability by Sheldon Ross, 8 th ed.
STAT 302 Introduction to Probability Learning Outcomes Textbook: A First Course in Probability by Sheldon Ross, 8 th ed. Chapter 1: Combinatorial Analysis Demonstrate the ability to solve combinatorial
More informationSpring 2017 Econ 574 Roger Koenker. Lecture 14 GEE-GMM
University of Illinois Department of Economics Spring 2017 Econ 574 Roger Koenker Lecture 14 GEE-GMM Throughout the course we have emphasized methods of estimation and inference based on the principle
More informationMath Camp II. Calculus. Yiqing Xu. August 27, 2014 MIT
Math Camp II Calculus Yiqing Xu MIT August 27, 2014 1 Sequence and Limit 2 Derivatives 3 OLS Asymptotics 4 Integrals Sequence Definition A sequence {y n } = {y 1, y 2, y 3,..., y n } is an ordered set
More informationLECTURE 2 LINEAR REGRESSION MODEL AND OLS
SEPTEMBER 29, 2014 LECTURE 2 LINEAR REGRESSION MODEL AND OLS Definitions A common question in econometrics is to study the effect of one group of variables X i, usually called the regressors, on another
More informationStochastic Processes
Stochastic Processes A very simple introduction Péter Medvegyev 2009, January Medvegyev (CEU) Stochastic Processes 2009, January 1 / 54 Summary from measure theory De nition (X, A) is a measurable space
More informationEconomics 620, Lecture 18: Nonlinear Models
Economics 620, Lecture 18: Nonlinear Models Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 18: Nonlinear Models 1 / 18 The basic point is that smooth nonlinear
More informationCOMP2610/COMP Information Theory
COMP2610/COMP6261 - Information Theory Lecture 9: Probabilistic Inequalities Mark Reid and Aditya Menon Research School of Computer Science The Australian National University August 19th, 2014 Mark Reid
More informationECONOMETRICS FIELD EXAM Michigan State University May 9, 2008
ECONOMETRICS FIELD EXAM Michigan State University May 9, 2008 Instructions: Answer all four (4) questions. Point totals for each question are given in parenthesis; there are 00 points possible. Within
More informationStatistical Data Analysis
DS-GA 0 Lecture notes 8 Fall 016 1 Descriptive statistics Statistical Data Analysis In this section we consider the problem of analyzing a set of data. We describe several techniques for visualizing the
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationBTRY 4090: Spring 2009 Theory of Statistics
BTRY 4090: Spring 2009 Theory of Statistics Guozhang Wang September 25, 2010 1 Review of Probability We begin with a real example of using probability to solve computationally intensive (or infeasible)
More information1. The Multivariate Classical Linear Regression Model
Business School, Brunel University MSc. EC550/5509 Modelling Financial Decisions and Markets/Introduction to Quantitative Methods Prof. Menelaos Karanasos (Room SS69, Tel. 08956584) Lecture Notes 5. The
More information13 Endogeneity and Nonparametric IV
13 Endogeneity and Nonparametric IV 13.1 Nonparametric Endogeneity A nonparametric IV equation is Y i = g (X i ) + e i (1) E (e i j i ) = 0 In this model, some elements of X i are potentially endogenous,
More informationLecture Notes on Measurement Error
Steve Pischke Spring 2000 Lecture Notes on Measurement Error These notes summarize a variety of simple results on measurement error which I nd useful. They also provide some references where more complete
More informationStatistics and Econometrics I
Statistics and Econometrics I Point Estimation Shiu-Sheng Chen Department of Economics National Taiwan University September 13, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I September 13,
More informationCS 5014: Research Methods in Computer Science. Bernoulli Distribution. Binomial Distribution. Poisson Distribution. Clifford A. Shaffer.
Department of Computer Science Virginia Tech Blacksburg, Virginia Copyright c 2015 by Clifford A. Shaffer Computer Science Title page Computer Science Clifford A. Shaffer Fall 2015 Clifford A. Shaffer
More informationChapter 3. Point Estimation. 3.1 Introduction
Chapter 3 Point Estimation Let (Ω, A, P θ ), P θ P = {P θ θ Θ}be probability space, X 1, X 2,..., X n : (Ω, A) (IR k, B k ) random variables (X, B X ) sample space γ : Θ IR k measurable function, i.e.
More informationRegression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
More informationEcon 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE
Econ 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE Eric Zivot Winter 013 1 Wald, LR and LM statistics based on generalized method of moments estimation Let 1 be an iid sample
More informationAsymptotic Statistics-VI. Changliang Zou
Asymptotic Statistics-VI Changliang Zou Kolmogorov-Smirnov distance Example (Kolmogorov-Smirnov confidence intervals) We know given α (0, 1), there is a well-defined d = d α,n such that, for any continuous
More informationECE531 Lecture 10b: Maximum Likelihood Estimation
ECE531 Lecture 10b: Maximum Likelihood Estimation D. Richard Brown III Worcester Polytechnic Institute 05-Apr-2011 Worcester Polytechnic Institute D. Richard Brown III 05-Apr-2011 1 / 23 Introduction So
More informationStochastic Processes (Master degree in Engineering) Franco Flandoli
Stochastic Processes (Master degree in Engineering) Franco Flandoli Contents Preface v Chapter. Preliminaries of Probability. Transformation of densities. About covariance matrices 3 3. Gaussian vectors
More information