Economics 620, Lecture 8: Asymptotics I

Transcription

1 Economics 620, Lecture 8: Asymptotics I Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 1 / 17

2 We are interested in the properties of estimators as n! 1: Consider a sequence of random variables fx n ; n 1g: Often X n is an estimator such as a sample mean or c n Often it is convenient to center the sequence: f c n g and sometimes to scale f( c n )= n g Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 2 / 17

3 Plim De nition: (convergence in probability) A sequence of random variables fx n ; n 1g is said to converge weakly to a constant c if lim n!1 P(jX n cj > ") = 0 for every given " > 0. This is written p lim X n = c or X n p! c Some properties of plim: 1. plim XY = plim X plim Y 2. plim (X + Y ) = plim X + plim Y 3. Slutsky s theorem: If the function g is continuous at plim X, then plim g(x ) = g(plim X ). Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 3 / 17

4 A.S. convergence De nition: (Strong convergence) A sequence of random variables is said to converge strongly to a constant c if P( lim X n = c) = 1 n!1 or lim P(sup jx n cj > ") = 0: N!1 n>n Strong convergence is also called almost sure convergence or convergence a:s: with probability one and is written X n! c w.p. 1 or X n! c. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 4 / 17

5 Di erence betwen convergence a.s. and plim plim involves probabilities on each element of the sequence, and limits of these probabilities. lim n!1 P(jX n cj > ") = 0 Strong convergence involves probabilities on the entire sequence. P(lim n!1 X n = c) = 1 Sequence of marginal probabilities vs. joint probability over in nite sequences. Note a.s. convergence implies plim. Di erence usually doesn t matter in applications and plim is easier to establish. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 5 / 17

6 Laws of Large Numbers: Let fx n ; n 1g be observations and suppose we look at the sequence X n = P n i=1 X i =n when does X n p! where is some parameter? Weak Law of Large Numbers: (WLLN) Let E(X i ) =, V (X i ) = 2, cov(x i X j ) = 0. Then X n! 0 in probability. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 6 / 17

7 Proof of WLLN Lemma: Chebyshev s Inequality: P(jX j k) 2 =k 2 where E(X ) = and V (X ) = 2. Proof of Chebshev s inequality Z 2 = (x ) 2 df = R k 1 (x ) 2 +k R df + (x ) 2 df + 1R +k k (x ) 2 df Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 7 / 17

8 Proof of WLLN (cont d) Put in the largest value of x in the rst and smallest in the last integral, and drop the middle to get: 2 k 2 P(jx j k) Proof of WLLN: Since we are interested in X n, note that E(X n ) = and V (X n ) = 2 =n. Consequently, lim n!1 P(j X n j > ") 5 lim n!1 2 =n" 2 = 0. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 8 / 17

9 Notes: 1. E(X i ) = i is okay. Consider X n n with n = n 1 P i : 2. V (X i ) = 2 i is okay. As long as lim P 2 i =n2 = 0, our proof applies. 3. Existence of 2 can be dropped if we assume independent and identically distributed observations. In this case, the proof is di erent and is based on Markov s inequality P(jX j k) EjX j=k from which Chebyshev s inequality follows. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 9 / 17

10 Strong Law of Large Numbers: If X i are independent with E(X i ) = i, V (X i ) = 2 i and P 2 i =i 2 < 1. Then X n n! 0 almost surely (a.s.). We can drop the existence of 2 i distributed observations. if we assume independent and identically Example (Shiryayev): let the probability space be [0,1) with Lebesgue measure (length of intervals). To each element! of [0,1), there is a sequence fx i gwhere x i is the ith element in the dyadic expansion of!;i.e.!=0:x 1 x 2 :::; x i 2 f0; 1g: Then P(X 1 = x 1 ; :::; X n = x n ) = P(x 1 =2 + x 2 =2 2 + :::x n =2 n! < x 1 =2 + x 2 =2 2 + :::x n =2 n + 1=2 n ) = 1=2 n :Thus P(X P = 1) = P(X = 0) = 1=2 and the obs are iid. By the SLLN, Xi =n! 1=2: Interpretation? Borel result on normal numbers. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 10 / 17

11 Weak Convergence in Distribution De nition: (Convergence in distribution): A sequence of random variables fx n ; n 1g with distribution functions ff n (x) = P(X n x); n 1g is said to converge in distribution to a random variable X with distribution function F (x) if and only if lim n!1 F n (x) = F (x) at all points of continuity of F (x). Notation: X n D! X. plim is a special case in which F is a degenerate distribution. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 11 / 17

12 More on Convergence in Distribution An equivalent characterization is: Ef (X n )! Ef (X ) for all bounded continuous functions f.another is for all sets B with P(@B) = 0: We have X n as!) X n p!) Xn d! P(X n 2 B)! P(X 2 B) Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 12 / 17

13 Continuous Mapping Theorem Convergence in distribution is used to approximate the distribution of estimators. If an estimator is consistent (plim=true value), studying the limiting distribution nontrivially requires norming. CMT: Let g(x) be continuous on a set which has probability one. Then d X n! X ) g(xn )! d g(x ) p X n! X ) g(xn )! p g(x ) as X n! X ) g(x n )! as g(x ) The CMT is extremely useful. Why? Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 13 / 17

14 Some properties of convergence in probability (plim) and convergence in distribution: 1. X n and Y n are random variable sequences. If plim(x n Y n ) = 0 D D and Y n! Y, then Xn! Y as well. This is an extremely useful device. 2. D If Y n! Y and Xn! c in probability (i.e., plimx n = c), then a. D X n + Y n! c + Y b. D X n Y n! cy c. D Y n =X n! Y =c, c 6= 0. Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 14 / 17

15 "Big O and little o" This notation is used to denote relative orders of magnitude of sequences in the limit. Sequences fx i g; fb i g (nonstochastic, for now) x n = O(b n ) ) lim n!1 x n=b n = 1 < c < 1 x n = o(b n ) ) lim n!1 x n=b n = 0 Thus x n = o(1) ) x n! 0; x n = o(n) ) x n =n! 0 x n = O(1) ) x n! c; x n = O(n) ) x n =n! c Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 15 / 17

16 Stochastic Versions For stochastic sequences fx i g we have X n = O p (b n ) ) 89C such that lim n!1 P(jX n=b n j < C) > 1 This says that the ratio remains bounded in probability. Also X n = o p (b n ) ) p lim X n =b n = 0 Thus for example (using results above) if X n Y n = o p (1) and X n d! X Then Y n d! X Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 16 / 17

17 Further Properties of O p and o p o p (1) + o p (1) = o p (1) o p (1) + O p (1) = O p (1) O p (1)o p (1) = o p (1) (1 + o p (1)) 1 = O p (1) o p (b n ) = b n o p (1) O p (b n ) = b n O p (1) Professor N. M. Kiefer (Cornell University) Lecture 8: Asymptotics I 17 / 17