Graduate Econometrics I: Asymptotic Theory

Transcription

1 Graduate Econometrics I: Asymptotic Theory Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Asymptotic Theory 1/42

2 Outline Yves Dominicy Graduate Econometrics I: Asymptotic Theory 2/42

3 Outline Yves Dominicy Graduate Econometrics I: Asymptotic Theory 3/42

4 Asymptotic theory, or large sample theory, has to do with the study of random variables as n. Convergence is seen in two different ways : Towards which limiting value does a trajectory of random variables converge? Towards which limiting distribution does the distribution of a random variable converge? Yves Dominicy Graduate Econometrics I: Asymptotic Theory 4/42

5 Outline Yves Dominicy Graduate Econometrics I: Asymptotic Theory 5/42

6 Convergence in Probability The random variable Y n converges in probability to a constant c if lim P( Yn c > ε) = 0 ε > 0. n That is the probability that the difference between Y n and c is larger than any ɛ goes to zero as n becomes bigger. Or put differently, values that Y n may take that are not close to c become increasingly unlikely as n increases. It is also written as : plimy n = c or Y n p c. Remark : If X n is an estimator, for instance the sample mean, and if plimx n = θ, we say that X n is a consistent estimator of θ. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 6/42

7 Convergence in Mean Square A random variable Y n converges in mean square (or in quadratic mean) to a constant c if : lim n E[(Yn c)2 ] = 0 or Y n m.s. c. The flutuations of Y n around c goes to 0 as n. The distribution of Y n collapses to a spike. Property Convergence in mean square implies convergence in probability : Y n m.s. c Y n p c. The sample mean converges to a constant, since its variance converges to 0. Remark : One can as well have convergence in mean r, for r 1 : lim n E[(Yn c)r ] = 0. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 7/42

8 Convergence in Mean Square PROOF If Y n m.s. c then E[Y n] = µ n c V [Y n] = σ 2 n 0 The Chebychev s inequality says : Then : P( Y n c > ε) E[(Yn c)2 ] ε 2. lim P( Yn c > ε) n E[(Y n c) 2 ] lim n ε 2 lim P( Y n c > ε) n 0 plimy n = c. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 8/42

9 Almost Sure Convergence A random variable Y n converges almost surely to a constant c if lim P( Y i c > ε i n) = 0 ε > 0 or Y n a.s. c. n Intuitively : once the sequence Y n becomes close to c, it stays close to c. Instead of almost surely, the terminologies everywhere or with probability 1 are often used, and it can as well be defined in the following way : P( lim n Y n = c) = 1. Almost sure convergence Convergence is probability Mean Square convergence Convergence is probability Convergence in probability is the weaker type and it is often called weak convergence. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 9/42

10 Modes of Convergence Convergence in probability means. c Yves Dominicy Graduate Econometrics I: Asymptotic Theory 10/42

11 Modes of Convergence Convergence in mean square means. c Yves Dominicy Graduate Econometrics I: Asymptotic Theory 11/42

12 Modes of Convergence And almost sure convergence means. c Yves Dominicy Graduate Econometrics I: Asymptotic Theory 12/42

13 Modes of Convergence Simulation of 1000 draws from a N (0, 1) with size 100, 1000 and This plot shows the 1000 empirical means for the 3 sample sizes. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 13/42

14 Convergence Definition An estimator ˆθ n of a parameter θ is a consistent estimator of θ iff plimˆθ n = θ. Theorem The empirical mean ˆµ of a random sample from any population with finite mean µ and finite variance σ 2 is a consistent estimator of µ. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 14/42

15 Convergence Corollary For any function g(y ), if E[g(Y )] and V [g(y )] are finite constants, plim 1 n n g(y i ) = E[g(Y )]. i=1 Theorem For a continuous function g(y n) : plimg(y n) = g(plimy n). Yves Dominicy Graduate Econometrics I: Asymptotic Theory 15/42

16 Convergence Theorem Rules for probability limits If Y n and X n are two random variables with plimy n = c and plimx n = d : i) plim(y n + X n) = c + d ii) plimy nx n = cd iii) plim Yn X n = c if d 0. d If W n is a matrix whose elements are random variables and if plimw n = Ω, then plimwn 1 = Ω 1. If Y n and X n are random matrices with plimy n = A and plimx n = B, then plimy nx n = AB. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 16/42

17 Outline Yves Dominicy Graduate Econometrics I: Asymptotic Theory 17/42

18 Y n is a random variable it also has a probability distribution F n(y ). Definition Y n converges in distribution to a random variable Y with probability distribution F(Y ) if : lim Fn(Y ) F (Y ) = 0. n Notice that Y n may converge to some distribution but it does not imply that Y n itself converges. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 18/42

19 Definition If Y n converges in distribution to Y then F(Y ) is the limiting distribution of Y n, where F(Y ) is the cumulative function of Y : Y n d Y. It tells us that the distribution of Y n is well approximated by the distribution of Y for large sample size n. Example Consider Y to be a sample of size n The test statistic for the mean (Y, N(0, 1) n ). t n 1 = ȳ, s 2 n 1 Yves Dominicy Graduate Econometrics I: Asymptotic Theory 19/42

20 : Example cont. where ȳ = 1 n n i=1 y i and s 2 = 1 n 1 n (y i ȳ) 2, follows a standardized Student t distribution with n 1 degrees of freedom. The density is : with µ = 0 and σ 2 = n 1 n 3. As n, t d N(0, 1). i=1 f (t n 1 ) = Γ( n ) 2 Γ( n 1 ) ((n 1 1)π) 2 [1 + t ] 2 n 2 n 1 2 Yves Dominicy Graduate Econometrics I: Asymptotic Theory 20/42

21 Definition The limiting mean and variance of a random variable are the mean and the variance of the limiting distribution. Example In the previous example the limiting mean is 0 and the limiting variance is : n 1 lim n n 3 = 1. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 21/42

22 Theorem Some rules : 1 If Y n d Y and plimx n = c, then Y nx n d cy, the limiting distribution of Y nx n is the distribution of cy Y n + X n d Y + c if c 0. Y n Y X n d c 2 If Y n d Y and g( ) is continuous, then g(y n) d g(y ). 3 If Y n has a limiting distribution and plim(y n X n) = 0, then X n has the same limiting. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 22/42

23 Example of rule 2 Let Y n be a Student t random variable with n degrees of freedom. We know that as n, Y n d N(0, 1). Therefore Y 2 n d χ 2 1 as n. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 23/42

24 Central Limit Theorems An important relation is : If plimy n = Y, then Y n d Y. This relation says that the limiting distribution of Y n is a spike, which is not very informative. To avoid the degeneration (a degenerated random variables is a random with zero variance), we transform the random variable such that the resulting new random variables has a well-defined limiting distribution. This transformation, so-called stabilizing transformation, is a function of n. Very often it is n : Factor that stabilizes the distribution (compensates for plimy n = Y ). It is also a rate of convergence. It tells us how fast Y n converges to the limiting distribution. If n = n 1 2, the "2" tells us the rate of convergence. In general n 1 α. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 24/42

25 Central Limit Theorems Therefore whereas : we often find that : plimy n = Y, Z n = n(y n Y ) d F(Z ) where F(Z ) is a well defined distribution with mean and a positive variance. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 25/42

26 Central Limit Theorems : Lindberg-Levy CLT Theorem If Y 1,..., Y n is a random sample from a probability distribution with finite µ and finite variance σ 2 and Ȳn = 1 n n i=1 y i : n( Ȳ n µ) d N(0, σ 2 ). This result is very remarkable as it hold regardless of the form of the parent distribution. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 26/42

27 Central Limit Theorems : Lindberg-Feller CLT Theorem Let Y 1,..., Y n be a random sample with finite means µ 1,..., µ n and finite positive variances σ 2 1,..., σ 2 n. Let σ 2 n = 1 n (σ2 1 + σ ), lim n max σ i nσ n = 0, lim n σ 2 n = σ 2. Then n( Ȳ n µ n) d N(0, σ 2 ). This theorem states that sums of random variables, regardless of their form, will tend to be normally distributed. And it does not require the variables in the sum to come from the same underlying distribution. The only requirement is that the mean be a mixture of many random variables, none of which is large compared with the sum. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 27/42

28 Central Limit Theorems Example 1 The least squares estimator in the case of one regressor is ˆβ OLS = ( n i=1 x i x i ) 1 n x i y i. i=1 ˆβ OLS can as well be seen as a weighted average : ˆβ OLS = ( 1 n n i=1 x i x i ) 1 1 n n x i y i. By the CLT 1 n n i=1 x iy i converges in distribution to a Gaussian, regardless of the distribution of (y 1,..., y n), and so does ˆβ OLS. i=1 Yves Dominicy Graduate Econometrics I: Asymptotic Theory 28/42

29 Central Limit Theorems Example 2 Let Y 1,..., Y n be a random sample with finite means µ 1,..., µ n and finite positive variances σ 2 1,..., σ 2 n. Let the empirical mean : with variance : Var(Ȳ ) = Var ( 1 n Ȳ = 1 n n y i, i=1 ) n y i = 1 n Var(y n 2 i ). i=1 i=1 Yves Dominicy Graduate Econometrics I: Asymptotic Theory 29/42

30 Central Limit Theorems Stabilizing by n : Stabilizing by n : Var( ( nȳ ) = Var 1 n Var(nȲ ) = Var ( n i=1 ) n y i = 1 n i=1 y i ) = 1 1 Stabilizing by n 1/4 : ( ) Var(n 1/4 1 n Ȳ ) = Var y n 3/4 i Note that 1 n 3/2 < 1 n < 1 1. i=1 n Var(y i ). i=1 n Var(y i ). i=1 = 1 n 3/2 n Var(y i ). i=1 Yves Dominicy Graduate Econometrics I: Asymptotic Theory 30/42

31 Distribution of 1000 empirical means times n. Data are simulated from a N (0, 1) and for 3 sample sizes. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 31/42

32 Distribution of 1000 empirical means times n. Data are simulated from a N (0, 1) and for 3 sample sizes. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 32/42

33 Distribution of 1000 empirical means times n 1/4. Data are simulated from a N (0, 1) and for 3 sample sizes. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 33/42

34 Delta Method We now put toghether three results : 1 plim g(y n) = g(plimy n) 2 Y n d Y, g(y n) d = g(y ) 3 CLT that leads to a useful result, known as the Delta Method. Theorem If n(y n c) d N(0, σ 2 ) and if g(y n) is continuous, then : ( n(g(yn) g(c)) d N 0, g(c) ) g(c) c σ2. c Yves Dominicy Graduate Econometrics I: Asymptotic Theory 34/42

35 Outline Yves Dominicy Graduate Econometrics I: Asymptotic Theory 35/42

36 Law of Large Numbers Theorem Let {Y i }, i = 1,..., n be a random sample (i.i.d.) and assume that E[Y i ] = µ a finite constant. Then : Ȳ = 1 n Y i p µ. n Notice that there are no restriction on the variances. i=1 Theorem Let {Y i }, i = 1,..., n be a random sample (i.i.d.) and assume that E[Y i ] = µ and V [Y i ] = σ 2 are finite constants. Then : Ȳ = 1 n n Y i a.s. µ. i=1 Yves Dominicy Graduate Econometrics I: Asymptotic Theory 36/42

37 Outline Yves Dominicy Graduate Econometrics I: Asymptotic Theory 37/42

38 Definition The sequence {a n} is at most of order n k, and write it as a n = O(n k ), if there exists a real number δ such that a n lim δ n. n n k A related concept is : Definition The sequence {a n} is of smaller order than n k, and write it as a n = o(n k ), if lim n n k a n = 0. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 38/42

39 Example : X X lim n n then X X = o(n 2 ) because : X X lim n n 2 = Q X X = O(n), Q = lim n n = 0. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 39/42

40 Theorem : Rules Consider the sequences {a n} and {b n} such that a n = O(n k ) and b n = O(n j ). Then : 1 a nb n = O(n k+j ) 2 a n s = O(n ks ), s > 0 3 a n + b n = O(max{n k, n j }). The same if O is replaced by o. In addition : 4 If a n = O(n k ) a n = o(n k+j ), j > 0 5 If a n = O(n k ) and b n = o(n j ) a nb n = o(n k+j ). Yves Dominicy Graduate Econometrics I: Asymptotic Theory 40/42

41 All these orders of magnitude are from nonstochastic sequences (in the examples X X is a nonstochastic matrix). However, in econometrics we deal with random variables. Associated with sequences of random variables and probability limits we have a similar concept known as order in probability. Definition The sequence of random variables Y 1,..., Y n is at most of order in probability n k, and we write Y n = O p(n k ), if, for every δ > 0, there exists a real number ν such that : P[n k Y n ν] δ n. It reads : The probability of being larger than ν goes to zero. n k Y n does not scape to. It remains bounded. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 41/42

42 Definition The sequence of random variables Y 1,..., Y n is of smaller order in probability than n k, and write Y n = o p(n k ), if : plim n k Y n = lim n P[ n k Y n > ν] = 0. Theorem : Rules The same as for O and o. Yves Dominicy Graduate Econometrics I: Asymptotic Theory 42/42