When is the OLSE the BLUE? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 40
When is the Ordinary Least Squares Estimator (OLSE) the Best Linear Unbiased Estimator (BLUE)? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 2 / 40
We already know the OLSE is the BLUE under the GMM, but are there other situations where the OLSE is the BLUE? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 3 / 40
Consider an experiment involving 4 plants. Two leaves are randomly selected from each plant. One leaf from each plant is randomly selected for treatment with treatment 1. The other leaf from each plant receives treatment 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 4 / 40
Let y ij = the response for the treatment i leaf from plant j (i = 1, 2; j = 1, 2, 3, 4). Suppose y ij = µ i + p j + e ij, where p 1,..., p 4, e 11,..., e 24 are uncorrelated, E(p j ) = E(e ij ) = 0, Var(p j ) = σ 2 p, Var(e ij ) = σ 2 i = 1, 2; j = 1,..., 4. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 5 / 40
Suppose σ 2 p/σ 2 is known to be equal to 2 and y = (y 11, y 21, y 12, y 22, y 13, y 23, y 14, y 24 ). Show that the AM holds. Find OLSE pf µ 1 µ 2 and the BLUE of µ 1 µ 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 6 / 40
E(y) = Xβ, where 1 0 0 1 1 0 0 1 X = 1 0 0 1 1 0 0 1 and β = [ µ1 µ 2 ]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 7 / 40
Var(y ij ) = Var(µ i + p j + e ij ) = Var(p j + e ij ) = Var(p j ) + Var(e ij ) = σp 2 + σ 2 = σ 2 (σp/σ 2 2 + 1) = 3σ 2. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 8 / 40
Cov(y 1j, y 2j ) = Cov(µ 1 + p j + e 1j, µ 2 + p j + e 2j ) = Cov(p j + e 1j, p j + e 2j ) = Cov(p j, p j ) + Cov(p j, e 2j ) + Cov(p j, e 1j ) + Cov(e 1j, e 2j ) = Cov(p j, p j ) = Var(p j ) = σp 2 = σ 2 (σp/σ 2 2 ) = 2σ 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 9 / 40
Cov(y ij, y i j ) = 0 if j j because = Cov(p j + e ij, p j + e i j ) = Cov(p j, p j ) + Cov(p j, e i j ) + Cov(p j, e ij ) + Cov(e ij, e i j ) = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 10 / 40
Thus, Var(y) = σ 2 V, where 3 2 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 2 3 0 0 0 0 V =. 0 0 0 0 3 2 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 2 3 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 11 / 40
We can write the model as y = Xβ + ε, where p 1 + e 11 p 1 + e 21 p 2 + e 12 p 2 + e 22 ε =, E(ε) = 0 and Var(ε) = σ 2 V. p 3 + e 13 p 3 + e 23 p 4 + e 14 p 4 + e 24 opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 12 / 40
Note that I 2 2 I 2 2 X = I 2 2 I 2 2 = 1 I. 4 1 2 2 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 13 / 40
Thus, I X I X = [I, I, I, I] I I = 4I 2 2 (X X) 1 = 1/4I. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 14 / 40
X y = [I, I, I, I]y = [ y1 y 2 ]. Thus, ˆβ OLS = 1 4 I [ y1 y 2 ] = [ȳ1 ȳ 2 ]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 15 / 40
Thus, the OLSE of µ 1 µ 2 is [1, 1]ˆβ OLS = ȳ 1 ȳ 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 16 / 40
To find [ the] GLSE, which we know is the BLUE for this AM, let 3 2 A =. Then 2 3 A 0 0 0 0 A 0 0 V = 0 0 A 0 0 0 0 A = I A. 4 4 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 17 / 40
V 1 = I A 1 = 4 4 A 1 0 0 0 0 A 1 0 0 0 0 A 1 0 0 0 0 A 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 18 / 40
It follows that X V 1 X = [1 I][I A 1 ][1 I] A 1 0 0 0 I 0 A 1 0 0 I = [I, I, I, I] 0 0 A 1 0 I 0 0 0 A 1 I = 4A 1. Thus, (X V 1 X) 1 = 1 4 A. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 19 / 40
A 1 0 0 0 X V 1 0 A 1 0 0 y = [I, I, I, I] 0 0 A 1 y 0 0 0 0 A 1 = [A 1, A 1, A 1, A 1 ]y = A 1 [I, I, I, I]y. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 20 / 40
Thus, ˆβ GLS = 1 4 AA 1 [I, I, I, I]y = 1 [I, I, I, I]y 4 ] = [ȳ1 ȳ 2 = ˆβ OLS. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 21 / 40
Thus, the GLSE of µ 1 µ 2 and the BLUE of µ 1 µ 2 is [1, 1]ˆβ GLS = ȳ 1 ȳ 2. Thus, OLSE = GLSE = BLUE in this case. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 22 / 40
Although we assumed that σ 2 p/σ 2 = 2 in our example, that assumption was not needed to find the GLSE. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 23 / 40
We have looked at one specific example where the OLSE = GLSE = BLUE. What general conditions must be satisfied for this to hold? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 24 / 40
Result 4.3: Suppose the AM holds. The estimator t y is the BLUE for E(t y) t y is uncorrelated with all linear unbiased estimators of zero. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 25 / 40
Proof: ( =) Let h y be an arbitrary linear unbiased estimator of E(t y). Then E((h t) y) = E(h y t y) = E(h y) E(t y) = 0. Thus, (h t) y is linear unbiased for zero. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 26 / 40
It follows that Cov(t y, (h t) y) = 0. Var(h y) = Var(h y t y + t y) Var(h y) Var(t y) with equality iff = Var((h t) y) + Var(t y). Var((h t) y) = 0 h = t. t y is the BLUE of E(t y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 27 / 40
(= ) Suppose h y is a linear unbiased estimator of zero. If h = 0, then Cov(t y, h y) = Cov(t y, 0) = 0. Now suppose h 0. Let c = Cov(t y, h y) and d = Var(h y) > 0. We need to show c = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 28 / 40
Now consider a y = t y (c/d)h y. E(a y) = E(t y) (c/d)e(h y) = E(t y). Thus, a y is a linear unbiased estimator of E(t y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 29 / 40
Var(a y) = Var(t y (c/d)h y) = Var(t y) + c2 d 2 Var(h y) 2Cov(t y, (c/d)h y) = Var(t y) + c2 d 2 d 2(c/d)c = Var(t y) c2 d. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 30 / 40
Now Var(a y) = Var(t y) c2 d c = 0 t y has lowest variance among all linear unbiased estimator of E(t y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 31 / 40
Corollary 4.1: Under the AM, the estimator t y is the BLUE of E(t y) Vt C(X). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 32 / 40
Proof of Corollary 4.1: First note that h y a linear unbiased estimator of zero is equivalent to E(h y) = 0 β R p h Xβ = 0 β R p h X = 0 X h = 0 h N (X ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 33 / 40
Thus, by Result 4.3, t y BLUE for E(t y) Cov(h y, t y) = 0 h N (X ) σ 2 h Vt = 0 h N (X ) h Vt = 0 h N (X ) Vt N (X ) = C(X). opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 34 / 40
Result 4.4: Under the AM, the OLSE of c β is the BLUE of c β estimable c β a matrix Q VX = XQ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 35 / 40
Proof of Result 4.4: ( =) Suppose c β is estimable. Let t = c (X X) X. Then VX = XQ VX[(X X) ] c = XQ[(X X) ] c Vt C(X), which by Cor. 4.1, t y is BLUE of E(t y) c ˆβOLS is BLUE of c β. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 36 / 40
(= ) By Corollary 4.1, c ˆβOLS is BLUE for any estimable c β. VX[(X X) ] c C(X) c C(X ) VX[(X X) ] X a C(X) a R n VP X a C(X) a R n q i VP X x i = Xq i i = 1,..., p, where x i denotes the i th column of X. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 37 / 40
VP X [x 1,..., x p ] = X[q 1,..., q p ] VP X X = XQ, where Q = [q 1,..., q p ] VX = XQ for Q = [q 1,..., q p ]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 38 / 40
Show that Q VX = XQ in our previous example. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 39 / 40
VX = A 0 0 0 0 A 0 0 0 0 A 0 0 0 0 A I I I I = A A A A = I I I I A = XA = XQ, where Q = A. opyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 40 / 40