Paul Schrimpf. January 23, UBC Economics 326. Statistics and Inference. Paul Schrimpf. Properties of estimators. Finite sample inference

Size: px

Start display at page:

Download "Paul Schrimpf. January 23, UBC Economics 326. Statistics and Inference. Paul Schrimpf. Properties of estimators. Finite sample inference"

Diane Phelps
5 years ago
Views:

1 UBC Economics 326 January 23, 2018

3 Wooldridge (2013) appendix C Stock and Watson (2009) chapter 3 Angrist and Pischke (2014) chapter 1 appendix Diez, Barr, and Cetinkaya-Rundel (2012) chapters 4, 5, & 6 Linton (2017) Part II

4 Statistics Interested in parameters of population, e.g. Moments, functions of moments Conditional expectation functions Distribution functions Learn about parameters by observing sample from the population Use sample to form estimates of parameters

5 Section 1

6 Setup: Sample {y 1,..., y n } Parameter θ Estimator W, some function of sample The bias of W is Bias(W) = E[W] θ W is unbiased if E[W] = θ : Sample mean: ȳ = 1 n n i=1 y i is unbiased Sample variance: s 2 y = 1 n n i=1 (y i ȳ) 2 is biased

7 The variance of W is Var(W) The standard error of W is Var(W) If W 1 and W 2 is are two unbiased, then W 1 is relatively efficient if Var(W 1 ) Var(W 2 ) The mean square error of W is MSE(W) = E[(W θ) 2 ] MSE(W) = Bias(W) 2 + Var(W) Prefer with that are relatively efficient and/or have low MSE

8 Sample mean: ȳ = 1 n n i=1 y i sample mean: ( ) 1 n Var(ȳ) =Var y i n i=1 ( 1 n =Cov y i, 1 n n = 1 n 2 sample mean n i=1 i=1 ) n y i i=1 n Cov(y i, y j ) j=1 assume independent observations = 1 n n 2 Var(y i ) i=1 = 1 n Var(Y) Standard error of sample mean: SE(ȳ) = Var(Y)/n

9 Section 2

10 : (roughly) how close are our sample estimates to the population parameters Example: N independent observations distributed Bernoulli(p), X i = 1 if success, else 0 Joint f(x 1,..., x n ) = n i=1 px i (1 p) 1 x i = p x i (1 p) N x i Could use to compute how surprising sample x that we observe is if the true p = p 0, e.g. compute P ( 1 n n i=1 X i p 0 x p0 ) Example: x i normally distributed, can do on sample mean using t-tests Problem: usually do not know of sample and/or of estimator intractable

11 Section 3

12 Idea: use limit of of estimator as N to approximate finite sample of estimator Notation: Sequence of samples of increasing size n, S n = {y 1,..., y n } Estimator for each sample W n : Wooldridge (2013) appendix C Menzel (2009) especially VI-IX

13 W n converges in to θ if for every ε > 0, lim P ( W n θ > ε) = 0 n p denote by plim W n = θ or W n θ Show using law of large numbers: if y 1,..., y n are i.i.d. with mean µ, or if y 1,..., y n have finite expectations and Var(y i ) i=1 i is finite, then ȳ p µ 2 Properties: plim g(w n ) = g(plim W n ) if g is continuous (continuous mapping theorem (CMT)) p p If W n ω and Zn ζ, then (Slutsky s lemma) p W n + Z n ω + ζ p W n Z n ωζ Wn Z n p ω ζ W n is a consistent estimate of θ if W n p θ

14 1 n < seq ( 1, ) ## sample s i z e s Demonstration of LLN 2 sims < 10 ## number of simulations f o r each sample s i z e 3 rand < function (N) { return ( r u n i f (N) ) } 4 ## simulation 5 samplemean < matrix ( 0, nrow= length ( n ), ncol = sims ) 6 f o r ( i in 1 : length ( n ) ) { 7 N = n [ i ] ; 8 x < matrix ( rand (N * sims ), nrow=n, ncol = sims ) 9 samplemean [ i, ] = colmeans ( x ) 10 } 11 ## p l o t sample means 12 l i b r a r y ( ggplot2 ) 13 l i b r a r y ( reshape ) 14 df < data. frame ( samplemean ) 15 df $n < n 16 df < melt ( df, i d = n ) 17 l l n P l o t < ggplot ( data = df, aes ( x =n, y= value, colour = v a r i a b 18 geom_ l i n e ( ) + s c a l e _ colour _ brewer ( p a l e t t e = Paired ) + 19 s c a l e _ y _ continuous (name= sample mean ) + 20 guides ( colour = FALSE ) 21 l l n P l o t

15 Demonstration of LLN 0.75 sample mean n

16 ȳ p E[y] by LLN Sample variance: : examples ˆσ 2 = 1 n n (y i ȳ) 2 i=1 = 1 n y 2 i ȳ 2 n }{{} i=1 p }{{} E[y] 2 by LLN and CMT p E[y 2 ] by LLN p E[y 2 ] E[y] 2 = Var(y)

17 : examples Mean divided by variance: n i=1 y 1 n i n i=1 (y i ȳ) 2 = n i=1 y i n i=1 (y i ȳ) 2 1 n = ȳ ˆσ 2 p E[y] by above and Slutsky s lemma Vary

18 Sample correlation: : examples ˆρ = 1 n 1 n n i=1 (x i x)(y i ȳ) n i=1 (x i x) 2 1 n n i=1 (y i ȳ) 2 1 n n i=1 (x i x) 2 p Var(X) and 1 n n i=1 (y i ȳ) 2 p Var(Y) as above 1 n n i=1 (x i x) 2 1 n n i=1 (y i ȳ) 2 p Var(X)Var(Y) by Slutsky s lemma 1 n n i=1 (x i x) 2 1 n n i=1 (y i ȳ) 2 p Var(X)Var(Y) by CMT 1 n n i=1 (x i x)(y i ȳ) for sample variance ˆρ p Corr(X, Y) by Slutsky s lemma p Cov(X, Y) by similar reasoning as

19 Let F n be the CDF of W n and W be a random variable with CDF F d W n converges in to W, written W n W, if lim n F n (x) = F(x) for all x where F is continuous Central limit theorem: Let {y 1,..., y n } be i.i.d. with mean µ and variance σ 2 then Z n = n ȳn µ σ converges in to a standard normal random variable As with the LLN, the i.i.d. condition can be relaxed if additional moment conditions are added Demonstration Properties: d If W n W, then g(wn ) d g(w) for continuous g (continuous mapping theorem (CMT)) Slutsky s theorem: If W n d W and Zn p c, then (i) W n + Z n d W + c, (ii) Wn Z n d cw, and (iii) Wn /Z n d W/c

20 1 N < c ( 2, 5, 1 0, ) 2 simulations < 5000 Demonstration of CLT 3 means < matrix ( 0, nrow= simulations, ncol = length (N) ) 4 f o r ( i in 1 : length (N) ) { 5 n < N[ i ] 6 dat < matrix ( rbeta ( n * simulations, 7 shape1 = 0. 5, shape2 = 0. 5 ), 8 nrow= simulations, ncol =n ) 9 means [, i ] < ( apply ( dat, 1, mean) 0. 5 ) * s q r t ( n ) 10 } # P l o t t i n g 13 df < data. frame ( means ) 14 df $n < N 15 df < melt ( df ) 16 c l t P l o t < ggplot ( data = df, aes ( x = value, f i l l = v a r i a b l e ) ) + 17 geom_ histogram ( alpha = 0. 8, p o s i t i o n = i d e n t i t y ) + 18 s c a l e _ x _ continuous (name= expression ( s q r t ( n ) ( hat (mu) mu) ) 19 s c a l e _ f i l l _ brewer ( type = d i v, p a l e t t e = RdYlGn, 20 name= N, l a b e l =N) + 21 l a b s ( t i t l e = Histogram of sample mean f o r Beta ( 0. 5, 0. 5 ) d i 22 f a c e t _ g r i d ( v a r i a b l e ~., l a b e l = function ( x ) { } ) + theme_ mi

21 600 Demonstration of CLT Histogram of sample mean for Beta(0.5,0.5) 400 count N n(µ^ µ)

22 CLT examples Mean: n ȳ µ d σ N(0, 1) by CLT What if σ estimated instead of known? n ȳ µ ˆσ ˆσ p σ from before n(ȳ µ) d N(0, σ 2 ) by CLT Slutsky s theorem with W n = n(ȳ µ), Z n = ˆσ gives n ȳ µ ˆσ d N(0, 1)

23 CLT examples Variance: ˆσ 2 = 1 n n (y i ȳ) 2 i=1 = 1 n (y i µ + µ ȳ) 2 n i=1 [ ] 1 n = (y i µ) 2 (ȳ µ) 2 n i=1 So n(ˆσ 2 σ 2 ) = n [ 1 n ] n (y i µ) 2 σ 2 n(ȳ µ)(ȳ µ) i=1

24 CLT examples n(ȳ µ) d N(0, σ 2 ), (ȳ µ) p 0, so n(ȳ µ)(ȳ µ) d 0 CLT implies n [ 1 n n i=1 (y i µ) 2 σ 2] d N(0, V)

25 Section 4

26 Null hypothesis, H 0 p-value: if null hypothesis what is the of a sample as or more extreme than what is observed From 325: if y i N(µ, σ 2 ), then n ȳ µ ˆσ 2 t(n 1)

27 Example: mean From 325: If y i N(µ, σ 2 ), then n ȳ µ ˆσ t with n 1 degrees of freedom p-value for H 0 : µ = µ 0 vs H a : µ µ 0 is ( 1 n n i=1 p =P y ) i µ 0 s y ȳ µ 0 ˆσ ) =2 F t ( ȳ µ 0 ˆσ ; n 1 }{{} CDF of t What if of y i unknown? Use CLT to approximate of t-statistic

28 Example: one mean From earlier slide, n ȳ µ ˆσ d N(0, 1) So, ( 1 n n i=1 P y ) i µ 0 s y ȳ µ 0 ˆσ ) ȳ µ 0 ˆσ }{{} standard normal CDF ( 2 Φ

29 Data on deworming treatment and school participation from Miguel and Kremer (2003) Treatment Control Sample size Infection School attendance Only the sample size for observations of infections. The real data has more observations of school participation, but for illustration we will pretend this is the sample size.

30 Example: one mean Test H 0 : E[infected control] = 0.5 againt H a : E[infected control] 0.5 Let I i = 1 if infected, else 0 What is sample variance of I i? ) ˆσ 2 I = 1 n Test statistic n (I i Ī) 2 = i=1 = ( 1 n ( 1 n n i=1 I 2 i Ī 2 ) n I i Ī 2 (I 2 i = I i ) i=1 =Ī Ī 2 = Ī(1 Ī) z = n Ī 0.5 Ī(1 Ī) P-value = 2(1 Φ(z)) = = = (1 0.54)

31 Example: two means Often want to test difference between two means H 0 : treatment has no effect H 0 treatment mean = control mean Let µ T = treatment mean, µ C = control mean H 0 : µ T = µ C CLT implies nt (Ī T µ T ) d N(0, σ 2 T ) and n C (Ī C µ C ) d N(0, σ 2 C) i.e. Ī T is approximately N(µ T, σ 2 T /n T) Sum of two normals is normal If treatment and control groups independent, then Ī T Ī C is approximately N(0, σ 2 T /n T + σ 2 C /n C) formally we could show Ī T Ī C d ˆσ 2 T /n T + ˆσ C 2/n N(0, 1) C

32 Example: two means Test H 0 : E[infected control] = E[infected treatment] against H a : E[infected control] > E[infected treatment] Test statistic: Ī T Ī C z = 2 ˆσ T /n T + ˆσ C 2/n C = = (1 0.32)/ (1 0.54)/352 P-value = Φ(z) = Test H 0 : E[school control] = E[school treatment] against H a : E[school control] < E[school treatment] Test statistic: S T S C z = 2 ˆσ T /n T + ˆσ C 2/n C = = ( )/ ( )/352 P-value = 1 Φ(z) =

33 Example: RAND health insurance experiment

34 Example: RAND health insurance experiment

35 Example: TRUMP health insurance experiment? 1 New Yorker daily cartoon, January 5th 2017,

36 p-value pitfalls tests and are a tool for quantifying uncertainty, but not the only tool Significance thresholds often over-emphasized Difference in significance is not necessarily a significant difference A statistically significantly different from 0 and B not statistically significantly different from 0 does not imply that A B is statistically significant When many hypotheses, likely to find significant results by chance alone Researcher degrees of freedom and garden of forking paths can introduce many tests inadvertently

37 CLT: under H 0, or equivalently, z n = nȳ µ ˆσ d N(0, 1) lim P(z n x) = Φ(x) n P-values are only correct asymptotically (for large samples) Small finite sample will be somewhat off With exact, under H 0, from distributed U[0, 1], under H 0, only U[0, 1] in large samples

38 Convergence of P(z n x) N 2 5 CDF infty z

39 Convergence of actual size 0.50 N infty nominal size

40 Code for graphs

41 Angrist, Joshua D and Jörn-Steffen Pischke Mastering Metrics: The Path from Cause to Effect. Princeton University Press. Diez, David M, Christopher D Barr, and Mine Cetinkaya-Rundel OpenIntro Statistics. OpenIntro. URL Linton, Oliver B Probability, Econometrics. Academic Press. URL Probability%2C%20statistics%20and%20econometrics.

42 Menzel, Konrad Introduction to Statistical Methods in Economics. Massachusetts Institute of Technology: MIT OpenCourseWare. URL introduction-to-statistical-methods-in-econom lecture-notes/. Miguel, E. and M. Kremer Worms: identifying impacts on education and health in the presence of treatment externalities. Econometrica 72 (1): URL x/abstract. Stock, J.H. and M.W. Watson Introduction to Econometrics, 2/E. Addison-Wesley. Wooldridge, J.M Introductory econometrics: A modern approach. South-Western.

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables. Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel