When is the OLSE the BLUE? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

Transcription

1 When is the OLSE the BLUE? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

2 When is the Ordinary Least Squares Estimator (OLSE) the Best Linear Unbiased Estimator (BLUE)? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

3 We already know the OLSE is the BLUE under the GMM, but are there other situations where the OLSE is the BLUE? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

4 Consider an experiment involving 4 plants. Two leaves are randomly selected from each plant. One leaf from each plant is randomly selected for treatment with treatment 1. The other leaf from each plant receives treatment 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

5 Let y ij = the response for the treatment i leaf from plant j (i = 1, 2; j = 1, 2, 3, 4). Suppose y ij = µ i + p j + e ij, where p 1,..., p 4, e 11,..., e 24 are uncorrelated, E(p j ) = E(e ij ) = 0, Var(p j ) = σ 2 p, Var(e ij ) = σ 2 i = 1, 2; j = 1,..., 4. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

6 Suppose σ 2 p/σ 2 is known to be equal to 2 and y = (y 11, y 21, y 12, y 22, y 13, y 23, y 14, y 24 ). Show that the AM holds. Find OLSE pf µ 1 µ 2 and the BLUE of µ 1 µ 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

7 E(y) = Xβ, where X = and β = [ µ1 µ 2 ]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

8 Var(y ij ) = Var(µ i + p j + e ij ) = Var(p j + e ij ) = Var(p j ) + Var(e ij ) = σp 2 + σ 2 = σ 2 (σp/σ ) = 3σ 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

9 Cov(y 1j, y 2j ) = Cov(µ 1 + p j + e 1j, µ 2 + p j + e 2j ) = Cov(p j + e 1j, p j + e 2j ) = Cov(p j, p j ) + Cov(p j, e 2j ) + Cov(p j, e 1j ) + Cov(e 1j, e 2j ) = Cov(p j, p j ) = Var(p j ) = σp 2 = σ 2 (σp/σ 2 2 ) = 2σ 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

10 Cov(y ij, y i j ) = 0 if j j because = Cov(p j + e ij, p j + e i j ) = Cov(p j, p j ) + Cov(p j, e i j ) + Cov(p j, e ij ) + Cov(e ij, e i j ) = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

11 Thus, Var(y) = σ 2 V, where V = Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

12 We can write the model as y = Xβ + ε, where p 1 + e 11 p 1 + e 21 p 2 + e 12 p 2 + e 22 ε =, E(ε) = 0 and Var(ε) = σ 2 V. p 3 + e 13 p 3 + e 23 p 4 + e 14 p 4 + e 24 opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

13 Note that I 2 2 I 2 2 X = I 2 2 I 2 2 = 1 I Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

14 Thus, I X I X = [I, I, I, I] I I = 4I 2 2 (X X) 1 = 1/4I. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

15 X y = [I, I, I, I]y = [ y1 y 2 ]. Thus, ˆβ OLS = 1 4 I [ y1 y 2 ] = [ȳ1 ȳ 2 ]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

16 Thus, the OLSE of µ 1 µ 2 is [1, 1]ˆβ OLS = ȳ 1 ȳ 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

17 To find [ the] GLSE, which we know is the BLUE for this AM, let 3 2 A =. Then 2 3 A A 0 0 V = 0 0 A A = I A. 4 4 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

18 V 1 = I A 1 = 4 4 A A A A 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

19 It follows that X V 1 X = [1 I][I A 1 ][1 I] A I 0 A I = [I, I, I, I] 0 0 A 1 0 I A 1 I = 4A 1. Thus, (X V 1 X) 1 = 1 4 A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

20 A X V 1 0 A y = [I, I, I, I] 0 0 A 1 y A 1 = [A 1, A 1, A 1, A 1 ]y = A 1 [I, I, I, I]y. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

21 Thus, ˆβ GLS = 1 4 AA 1 [I, I, I, I]y = 1 [I, I, I, I]y 4 ] = [ȳ1 ȳ 2 = ˆβ OLS. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

22 Thus, the GLSE of µ 1 µ 2 and the BLUE of µ 1 µ 2 is [1, 1]ˆβ GLS = ȳ 1 ȳ 2. Thus, OLSE = GLSE = BLUE in this case. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

23 Although we assumed that σ 2 p/σ 2 = 2 in our example, that assumption was not needed to find the GLSE. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

24 We have looked at one specific example where the OLSE = GLSE = BLUE. What general conditions must be satisfied for this to hold? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

25 Result 4.3: Suppose the AM holds. The estimator t y is the BLUE for E(t y) t y is uncorrelated with all linear unbiased estimators of zero. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

26 Proof: ( =) Let h y be an arbitrary linear unbiased estimator of E(t y). Then E((h t) y) = E(h y t y) = E(h y) E(t y) = 0. Thus, (h t) y is linear unbiased for zero. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

27 It follows that Cov(t y, (h t) y) = 0. Var(h y) = Var(h y t y + t y) Var(h y) Var(t y) with equality iff = Var((h t) y) + Var(t y). Var((h t) y) = 0 h = t. t y is the BLUE of E(t y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

28 (= ) Suppose h y is a linear unbiased estimator of zero. If h = 0, then Cov(t y, h y) = Cov(t y, 0) = 0. Now suppose h 0. Let c = Cov(t y, h y) and d = Var(h y) > 0. We need to show c = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

29 Now consider a y = t y (c/d)h y. E(a y) = E(t y) (c/d)e(h y) = E(t y). Thus, a y is a linear unbiased estimator of E(t y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

30 Var(a y) = Var(t y (c/d)h y) = Var(t y) + c2 d 2 Var(h y) 2Cov(t y, (c/d)h y) = Var(t y) + c2 d 2 d 2(c/d)c = Var(t y) c2 d. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

31 Now Var(a y) = Var(t y) c2 d c = 0 t y has lowest variance among all linear unbiased estimator of E(t y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

32 Corollary 4.1: Under the AM, the estimator t y is the BLUE of E(t y) Vt C(X). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

33 Proof of Corollary 4.1: First note that h y a linear unbiased estimator of zero is equivalent to E(h y) = 0 β R p h Xβ = 0 β R p h X = 0 X h = 0 h N (X ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

34 Thus, by Result 4.3, t y BLUE for E(t y) Cov(h y, t y) = 0 h N (X ) σ 2 h Vt = 0 h N (X ) h Vt = 0 h N (X ) Vt N (X ) = C(X). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

35 Result 4.4: Under the AM, the OLSE of c β is the BLUE of c β estimable c β a matrix Q VX = XQ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

36 Proof of Result 4.4: ( =) Suppose c β is estimable. Let t = c (X X) X. Then VX = XQ VX[(X X) ] c = XQ[(X X) ] c Vt C(X), which by Cor. 4.1, t y is BLUE of E(t y) c ˆβOLS is BLUE of c β. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

37 (= ) By Corollary 4.1, c ˆβOLS is BLUE for any estimable c β. VX[(X X) ] c C(X) c C(X ) VX[(X X) ] X a C(X) a R n VP X a C(X) a R n q i VP X x i = Xq i i = 1,..., p, where x i denotes the i th column of X. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

38 VP X [x 1,..., x p ] = X[q 1,..., q p ] VP X X = XQ, where Q = [q 1,..., q p ] VX = XQ for Q = [q 1,..., q p ]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

39 Show that Q VX = XQ in our previous example. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40

40 VX = A A A A I I I I = A A A A = I I I I A = XA = XQ, where Q = A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 40