STAT Homework X - Solutions

Transcription

1 STAT Homework X - Solutions Fall 201 November 12, 201 Tis contains solutions for Homework 4. Please note tat we ave included several additional comments and approaces to te problems to give you better insigt. Problem 1. Let Y 1,..., Y n N(θ, 1). Let p θ = (1/ 2 )e (x θ)2 /2 be te density function. Since tis is a parametric model, we can estimate te density by using te estimator ˆp = p ˆθ were ˆθ is te mle. Find, approximately, te bias and variance of ˆp. Sow tat, for large n, te risk is smaller tan te risk of te kernel density estimator (assuming te model is correct). Hint: Let θ 0 denote te true value of θ. We can write p θ p θ0 + (θ θ 0 )p θ 0 were Solution 1. Recall p θ = p θ. θ ˆθ mle = Y N (θ 0, 1 n ) Ten by te int, E[p ˆθ ] E[p θ 0 ] + E[( ˆθ θ 0 )p θ 0 ] = p θ0 + p θ 0 E[( ˆθ θ 0 )] = p θ0 bias(p ˆθ ) = E[p ˆθ p θ 0 ] = 0 And Var(p ˆθ ) (p θ 0 ) 2 Var( ˆθ) Tus = (p θ 0 ) 2 1 n R( ˆθ, θ 0 ) = bias(p ˆθ )2 + Var(p ˆθ ) = (p θ 0 ) 2 1 n = O( 1 n ) By Lecture ) Notes 20 p7, te minimum risk of kernel density estimator is O( 1 > O( 1 n 4/5 n ).

2 stat omework x - solutions 2 Problem 2. Let ˆp be te istogram estimator using bins of size. Suppose tat p = 0. Sow tat te bias is O( 2 ) and te variance is O(1/n) and ence te risk is O( 4 + 1/(n)). Solution 2. Similar to te derivation in Lecture Notes 20, we ave E ( ) ˆθ j = θj = p(u) du = p + p (u x) + = p + p (k) p (k) (u x) k du by Taylor expansion (u x) k du since p = 0 Tus, ( ) ˆθ j E( ˆp) = E = p + 1 p (k) (u x) k du b = E( ˆp) p = 1 p (k) (u x) k du 1 p(k) k du p(k) k du = = = O( 2 ) p(k) k+1 p(k) k Since Hence, θ j = p + = O(), p (k) (u x) k du Var( ˆp) = θ j(1 θ j ) n 2 = O( 1 n ) R( ˆp) = O( n )

3 stat omework x - solutions 3 Problem 3. Let (X 1, Y 1 ),..., (X n, Y n ) P. Suppose tat X j = j/n. Consider bins B 1,..., B m eac of lengt /m. Let ˆm be te regressogram estimator given by ˆm = Y j were Y j is te average of te Y i s in bin. Bound te bias and variance of ˆm. Solution 3. Ten E( ˆm) E(Y i ) E(m(X i ) + ɛ) m(x i ) b( ˆm) = E( ˆm) m 1 m(x i ) m m (z i )(X i x) 1 C for some z i by te Mean Value Teorem assuming m C = C Var( ˆm) 2 Var(Y i ) 2 σ 2 = σ2 n

4 stat omework x - solutions 4 Problem 4. Let (X 1, Y 1 ),..., (X n, Y n ) were X i {0, 1}. Suppose tat te assignment of te X i s is randomized and tat P(X i ) = and P(X i = 0). Define ˆθ n i Y i X i 1 n i Y i (1 X i ) 1. Sow tat ˆθ is a consistent estimator of te causal effect θ. Solution 4. Our solution is detailed below: Proof. We utilize key results from Lecture 22 1 from our course. 1 Larry Wasserman. CMU STAT Notes, Lecture 22: Causal Inference, 201. URL: ttp:// ~larry/=stat700/lecture22.pdf Specifically using Teorem 3 we ave tat if X is randomly assigned (as in our case) ten we ave θ = α (note tat (0, 1)). Were: α = E(Y X ) E(Y X = 0) Furter by Teorem 3 we also ave tat: α = E(XY) E(Y(1 X)) E(X) E(1 X) = E(XY) E(Y(1 X)) 1 Now we ave our estimator: ˆθ = ˆα n Y n i X i }{{} := A n := A n B n 1 n n Y i (1 X i ) 1 }{{} := B n Now by WLLN we ave tat: ( ) p XY A n E = E(XY) ( ) p Y(1 X) B n E E(Y(1 X)) = 1 So finally we ave combining all of te above tat: ˆθ = ˆα = A n B n p E(XY) E(Y(1 X)) 1 = α = θ

5 stat omework x - solutions 5 Problem 5. Suppose we ave variables (X, Y 0, Y 1 ) and Y = XY 1 + (1 X)Y 0 were X {0, 1}, Y 0 {0, 1}, Y 1 {0, 1}. Construct a joint distribution p(x, y 0, y 1 ) suc tat te casual effect θ = E[Y 1 ] E[Y 0 ] = 0 but α = E[Y X ] E[Y X = 0]. Solution 5. We derive eac of te results below: (a) We claim tat E(X) = µ. Proof. Te essence of tis question is to prove (in a counterfactual setting) te famous statement tat association does not necessarily imply causation. Te following example is taken directly from Example 16.2 te course textbook 2 (wit minor canges in notation per tis questions requirements). Suppose tat te entire population is given by: 2 Larry Wasserman. All of statistics: a concise course in statistical inference. Springer Science & Business Media, 2013 X Y Y 0 Y 1 Te asterix denote unobserved values. We note tat Y 0 = Y 1 for every subject and tus tat te treatment as no effect. More formally we can see tat tis is given by: θ = E(Y 1 ) E(Y 0 ) Y 1i Y 0i = = 0 So te average causal effect is 0 i.e. θ = Tus we can also estimate te association directly as follows: α = E(Y X ) E(Y X = 0) So te required joint distribution p(x, y 0, y 1 ) suc tat te casual effect is not equal to te associative effect i.e. θ = α as been constructed.

6 stat omework x - solutions 6 References Larry Wasserman. All of statistics: a concise course in statistical inference. Springer Science & Business Media, Larry Wasserman. CMU STAT Notes, Lecture 22: Causal Inference, 201. URL: ttp:// Lecture22.pdf.