Lecture 1: Review of Basic Asymptotic Theory

Transcription

1 Lecture 1: Instructor: Department of Economics Stanfor University Prepare by Wenbo Zhou, Renmin University

2 Basic Probability Theory Takeshi Amemiya, Avance Econometrics, 1985, Harvar University Press. George Casella an Roger Berger, Statistical Inference, Duxbury Avance Series. Patrick Billingsley, Probability an Measure, John Wiley an Sons James Davison, Stochastic Limit Theory, An Introuction for Econometricians. Davi Pollar, Convergence of Stochastic Processes.

3 Probability Space Probability Space: (Ω, F, P), if A F = A c F. A 1, A 2,..., F = i=1 A i F. F. Show Ω F an i=1 A i F. The events lim sup an lim inf are efine as: lim sup A n = n lim inf n A n = n=1 m n n=1 m n A m = {A n, i.o.} A m = {A n, e.v.} where i.o. enotes infinitely often an e.v. eventually.

4 Stochastic Convergence a.s. Convergence almost surely: X n 0 if ( ) P lim X n (ω) = 0 = 1 ɛ > 0, P ( X n (ω) > ɛ, i.o.) = 0. n L p convergence: X n L p 0 if Convergence in probability: X n lim E X n p = 0. n p 0 if ɛ > 0, lim n P ( X n (ω) ɛ) = 1. Convergence in istribution: X n 0 if lim P (X n x) = P (X x) n for every continuity point x in the istribution of X.

5 Relations: a.s. p X n 0 = X n 0. ( ) Note that P m n { X m (ω) ɛ} P ( X n (ω) ɛ). X n L p p 0, p > 0 = X n 0. Note that P ( X n > ɛ) ɛ p E X n p by the Markov Inequality. X n p 0 X n 0 However, X n X X n X p 0 unless X is egenerate.

6 Borel-Cantelli Lemma (BC) BC: Examples: P (E n ) < = P (E n, i.o.) = 0 n=1 i.i. a.s. X i Uniform(0, 1), then min 1 i n X i 0. Amemiya P88 But BC may not be neccessary for a.s. convergence: e.g. ω i.i. Uniform(0, 1) an efine X n (ω) = n if ω 1 n an X n (ω) = 0 if ω > 1 n. The above example also imply X n a.s. L X X p n X.

7 Uniform Integrability (UI) Conitions ensuring E( lim n X n) = lim n E(X n)? Monotone Convergence Theorem(MON): If X n a.s. X an X n is increasing a.s., then lim n EX n = EX. Dominate Convergence Theorem(DOM): If X n a.s. X an E (sup n X n (ω) ) <, then lim n EX n = EX. This also applies to the Lebesgue measure. Definition: X n is U.I., if lim sup E ( X n 1 ( X n > M)) = 0. M n a.s. Theorem: If X n is U.I. an if X n 0 then lim E X n = 0. Hence lim EX n = 0.

8 Stochastic Orer X n = o p (1) if X n X n = O p (1) if In particular, if X n Facts p 0. lim M lim sup P ( X n > M) = 0. n X, X n = O p (1). X n = O p (a n ) means a 1 n X n = O p (1). O p (1) o p (1) = o p (1). O p (a n ) O p (b n ) = O p (a n b n ). O p (a n ) + O p (b n ) = O p (a n + b n ) = O p (max (a n, b n )).

9 Continuity mapping Continuity mapping: Let X n X. Let g ( ) be a function such that its set of iscontinuity points E is close an P (X E) = 0, then g (X n ) g (X ). Slutsky is a special case of continuous mapping: p If X n X an Y n α, then X n + Y n X n Y n X n /Y n X + α. αx X /α if α 0.

10 Law of Large Numbers Weak Law of Large Numbers (WLLN): X t, t = 1,.... Let X n = 1 n n X t, unerwhat conitions oes X n E X n p 0? Sufficient to show E X n E X n p 0, say p = 2 but other p > 0 also works. An easy WLLN: X t uncorrelate with mean 0 an Var (X t ) = σ 2. WLLN for inepenent noninentically istribute X t : Let X t be inepenent. If σt 2 t 2 E ( ) 2 X n E X n 0. < then

11 Strong Law of Large Numbers (SLLN): Uner what conitions oes a.s. X n E X n 0. Kolmogorov SLLN 1: Let X t inepenent with finite variance, if then X a.s. n EX n 0. Kolmogorov SLLN 2: If X t ii with finite mean u, then X n u a.s. 0. σ 2 t t 2 <,

12 Characteristic Functions Definition: φ X (λ) = Ee iλx = E [cos (λx ) + i sin (λx )]. Examples: X N (0, 1), φ X (λ) = e λ2 /2. Cauchy f (x) = 1 π(1+x 2 ), then φ X (λ) = e λ. Properties: φ ax +b (λ) = Ee iλ(ax +b) = e ibλ φ X (aλ). Let X t be i.i., an let S n = n X t, then φ Sn (λ) = [φ Xt (λ)] n.

13 Examples If X i i.i.. normal N (0, 1), let S n = 1 n n X t, then φ Sn (λ) = [ φ Xt ( λ/ n )] n = [exp Still N (0, 1) If X t ii Cauchy, let S n = 1 n n X t, then ( ( λ/ n ) )] ) 2 n = exp ( λ2. 2 φ Sn (λ) = [φ Xt (λ/n)] n = [exp ( λ/n )] n = exp ( λ ). Still Cauchy. So No LLN for Cauchy ranom variables, because it has no mean.

14 Central Limit Theory (CLT) Triangular Array: {{X nt, t = 1,..., n}, n = 1,..., }. Example, given X t, t = 1,..., inepenent an mean 0, let σt 2 = Var (X t ) an Cn 2 = n σ2 t. Then X nt = Xt C n is an triangular array of ranom variables, for t = 1,..., n, n = 1,...,. An Var ( n X n nt) = Var (X nt) = 1. Sequence Version: Uner what conitions oes S n = ( Var ( X n )) 1/2 ( X n E X n ) N(0, 1). Triangular Array Version: The CLT looks for conitions uner which n S n = N (0, 1). X nt Remember that by efinition Var (S n ) = 1.

15 Lineberg Conition (Sufficient Conition for CLT): Sequence Version: lim n 1 C 2 n Array Version: lim n n E [ Xt 2 1 ( X t > ɛc n ) ] = 0. ɛ > 0. n E [ Xnt1 2 ( X nt > ɛ) ] = 0. ɛ > 0. Liapounov Conition (= Lineberg conition): Sequence Version: Array Version: ( [ ] n ) 1/3 lim C 2 1/2 n E X t 3 = 0. n δ > 0, s.t. lim n n E X nt 2+δ = 0.

16 Liapounov conition implies the Lineberg conition Proof(use the array version): ( ) E X nt 2+δ E 1 ( X nt > ɛ) Xnt 2+δ Therefore n lim E ( Xnt1 2 ( X nt > ɛ) ) 1 ɛ δ n ɛ δ E ( 1 ( X nt > ɛ) Xnt 2 ) lim n n E X nt 2+δ = 0.

17 Lineberg-Levy CLT: If X t ii ( 0, σ 2 ), then the CLT hols. A sufficient conition for Lineberg conition: Let σ 2 t 1 for all t an nonecreasing, if X 2 t σ 2 t is U.I. an sup n max 1 t n σt 2 n n σ2 t <, then the Lineberg conition hols.

18 Proof: 1 C 2 n n E [ Xt 2 1 ( X t > ɛc n ) ] 1 n max ( 1 C 2 n 0. ) n max 1 t n σ2 t sup E t C 2 n 1 t n σ2 t E ( X 2 t σ 2 t ) ( 1 X t ɛ > C n σ t [ (Xt ) ( 2 (Xt ) 2 ε 2 1 > σ t σ t max 1 t n σt 2 C 2 n )] σ t )

19 Consistency Moel 1: y t = x tβ + u t, where u t is uncorrelate with mean 0, Eu 2 t = σ 2 for all t, then least square coefficients ˆβ = (X X ) 1 X y. Consistency of ˆβ: If λ s (X X ), then ˆβ p β. λ s (A) enote the smallest eigenvalues of A. Consistency of ˆσ 2 : Assume u t ii in Moel 1, then ˆσ 2 p σ 2, where ˆσ 2 = T 1 û û = T 1 u u T 1 u Pu for P = X (X X ) 1 X.

20 Asymptotic Normality Moel 1, u t ii ( 0, σ 2), max x t is a scalar, if lim 1 t T xt 2 T T = 0, then x2 t σ 1 ( x x ) ( ) 1/2 ˆβ β N (0, 1). x t is a k-imensional vector, for each i = 1,..., k, ( ) lim x 1 i x i max x ti 2 = 0. T 1 t T ( Let S = iag (x i x i) 1/2), Z = XS 1, lim T Z Z = R nonsigular, then S ( ˆβ β) N ( 0, σ 2 R 1).

21 Proof for the ( scalar ) case. Note first σ 1 (x x) 1/2 T ˆβ β = xtut 1 T T = T xtut σ x2 t Var( 1 T so T xtut), sufficient to check Lineberg conition for x t u t, which is 1 T σ 2 T x E (x tu t) 2 1 x tu t > σ T x 2 t 2 t [ ( 1 T T )] = σ 2 T x E (x tu t) 2 1 (x tu t) 2 > σ 2 x t 2 t 2 [ ( 1 T )] = σ 2 T x Ex 2 t 2 t ut 2 1 ut 2 > σ2 T x t 2 x 2 t [ ( 1 )] T max σ E u11 2 u 2 t 2 1 > σ2 t x t 2 1 t T xt 2 ( = 1 )] T [u σ E 11 2 u > σ2 t x t 2 0. max 1 t T xt 2 where the convergence follows from Eu1 2 an the state assumption that σ 2 t T x2 t max 1 t T x 2 t.

22 Proof ( for ) the vector case S ˆβ β = S (X X ) 1 X u = S (X X ) 1 SS 1 X u = (Z Z) 1 Z u. Use Cramer-Rao Device(Amemiya p93 Thm 3.3.8) sufficient to show that for c 0, c (Z Z) 1 Z u N ( 0, σ 2 c R 1 c ). But c (Z Z) 1 Z u LD = γ Z u for γ = c R 1 by Slutsky. So take x t = γ z t in Thm an check its conition: max 1 t T (γ z t ) 2 lim T γ Z Zγ k γ γ [λ s (Z Z)] γ γ i=1 (γ γ) max 1 t T z lim tz t T γ Z Zγ max 1 t T x 2 ti T x 2 ti 0. 1st inequality by Cauchy-Schwartz, 2n inequality by efinition of smallest eigenvalue an by efinition of z it, convergence by λ s boune away from 0 for large sample an the last term goes to 0 for each i = 1,..., k by assumption. Note that Z Z is the sample var-cov matrix for the regressors.