Estimable Functions and Their Least Squares Estimators. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

Transcription

1 Estimable Functions and Their Least Squares Estimators Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

2 Consider the GLM y = n p X β + ε, where E(ε) = 0. p 1 n 1 n 1 Suppose we wish to estimate c β for some fixed and known c R p. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

3 An estimator t(y) is an unbiased estimator of the function c β iff E[t(y)] = c β β R p. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

4 An estimator t(y) is a linear estimator in y iff t(y) = d + a y for some known constants d, a 1,..., a n. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

5 A function c β is linearly estimable iff a linear estimator that is an unbiased estimator of c β. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

6 Henceforth, we will use estimable as a synonym for linearly estimable. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

7 A function c β is said to be nonestimable if there does not exist a linear estimator that is an unbiased estimator of c β. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

8 Result 3.1: Under the GLM, c β is estimable iff the following equivalent conditions hold: (i) a E(a y) = c β β R p (ii) a c = a X (X a = c) (iii) c C(X ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

9 Show conditions (i), (ii), and (iii) are equivalent. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

10 (i) (ii) (iii): a E(a y) = c β β R p a a E(y) = c β β R p a a Xβ = c β β R p a a X = c a X a = c c C(X ). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

11 Show that any of the equivalent conditions is equivalent to c β estimable. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

12 c β estimable d, a E(d + a y) = c β β R p d, a d + a Xβ = c β β R p a a Xβ = c β β R p a a X = c. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

13 Example: Suppose that when team i competes against team j, the expected margin of victory for team i over team j is µ i µ j, where µ 1,..., µ 5 are unknown parameters. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

14 Suppose we observe the following outcomes. Team 1 beats Team 2 by 7 points Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

15 Determine y, X, β. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

16 , , µ 1 µ 2 µ 3 µ 4 µ 5 y, X, β Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

17 Is µ 1 µ 2 is estimable? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

18 Yes, µ 1 µ 2 is estimable. µ 1 µ 2 = c β, where c = [1, 1, 0, 0, 0] = [1, 0, 0, 0, 0, 0] = a X. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

19 Thus, a y = [1, 0, 0, 0, 0, 0]y = y 1 is an unbiased estimator of c β = µ 1 µ 2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

20 Is µ 1 µ 3 is estimable? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

21 Yes, µ 1 µ 3 is estimable. µ 1 µ 3 = c β, where c = [1, 0, 1, 0, 0] = [0, 1, 0, 0, 0, 0] = a X. y 2 is unbiased estimator of µ 1 µ 3. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

22 Is µ 1 µ 5 is estimable? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

23 Yes, µ 1 µ 5 is estimable. µ 1 µ 5 = c β, where c = [1, 0, 0, 0, 1] = [0, 1, 0, 1, 0, 0] = a X. y 4 y 2 is unbiased estimator of µ 1 µ 5. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

24 Is µ 1 estimable? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

25 µ 1 = c β, where c = [1, 0, 0, 0, 0]. So, does a a X = [1, 0, 0, 0, 0]? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

26 a X = [a 1, a 2, a 3, a 4, a 5, a 6 ] = [a 1 a 2 a 6, a 1 a 3, a 2 + a 3 + a 4, a 5 + a 6, a 4 a 5 ]? = [1, 0, 0, 0, 0]. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

27 a 1 a 2 a 6 = 1 (1) a 1 a 3 = 0 (2) a 2 + a 3 + a 4 = 0 (3) a 5 + a 6 = 0 (4) a 4 a 5 = 0 (5) (1) and (2) imply a 2 a 3 a 6 = 1. (4) and (5) imply a 4 = a 6, which together with (3) implies a 2 + a 3 + a 6 = 0. There does not exist a a X = [1, 0, 0, 0, 0] µ 1 is nonestimable. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

28 Result 3.1 tells us that c β is estimable iff a c β = a Xβ β R p. Recall that E(y) = Xβ. Thus, c β is estimable iff it is a LC of the elements of E(y). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

29 This leads to Method 3.1: LCs of expected values of observations are estimable. c β is estimable iff c β is a LC of the elements of E(y); i.e., c β = n a i E(y i ) for some a 1,..., a n. i=1 opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

30 Use Method 3.1 to show that µ 2 µ 4 is estimable in our previous example. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

31 µ 1 µ 2 µ 3 µ 1 E(y) = Xβ = µ 3 µ 2. µ 3 µ 5 µ 4 µ 5 µ 4 µ 1 µ 2 µ 4 = (µ 1 µ 2 ) (µ 4 µ 1 ) = E(y 1 ) E(y 6 ). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

32 Method 3.2: c β is estimable iff c C(X ). Thus, find a basis for C(X ), say {v 1,..., v r }, and determine if c = r d i v i for some d 1,..., d r. i=1 opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

33 Method 3.3: By Result A.5, we know that C(X ) and N (X) are orthogonal complements in R p. Thus, c C(X ) iff c d = 0 d N (X), which is equivalent to Xd = 0 c d = 0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

34 Reconsider our previous example. Use Method 3.3 to show that µ 1 is nonestimable. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

35 Xd = 0 d 1 d 2 = 0 d 3 d 1 = 0 d 3 d 2 = 0 d 3 d 5 = 0 d 4 d 5 = 0 d 4 d 1 = 0. d 1 = d 2 = d 3 = d 4 = d 5. For example, X1 = 0. Note [1, 0, 0, 0, 0]1 = 1 0. Thus, µ 1 is not estimable. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

36 Now use method 3.3 to establish that c β = c 1 µ 1 + c 2 µ 2 + c 3 µ 3 + c 4 µ 4 + c 5 µ 5 is estimable iff 5 c i = 0. i=1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

37 {d R 5 : Xd = 0} = {d1 : d R}. Thus 5 1 c d = 0 d N (X) c (d1) = 0 dc 1 = 0 d 5 c i = 0 i=1 5 c i = 0. i=1 d R d R d R Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

38 The least squares estimator of an estimable function c β is c ˆβ, where ˆβ is any solution to the NE (X Xb = X y). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

39 Result 3.2: If c β is estimable, then c ˆβ is the same for all solutions ˆβ to the NE. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

40 Proof of Result 3.2: Suppose ˆβ 1, and ˆβ 2 are any two solutions to the NE. From Corollary 2.3, we know Xˆβ 1 = Xˆβ 2. Now c β is estimable a c = a X. Thus, c ˆβ1 = a Xˆβ 1 = a Xˆβ 2 = c ˆβ2. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

41 Result 3.3: The least squares estimator of an estimable function c β is a linear unbiased estimator of c β. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

42 Proof of Result 3.3: From Result 3.2, we know c ˆβ is the same solution to NE. We know (X X) X y is a solution to NE. Thus, c ˆβ = c (X X) X y. This is a linear estimator. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

43 Furthermore, c β estimable a c = a X. Thus, E(c ˆβ) = E(c (X X) X y) = c (X X) X E(y) = c (X X) X Xβ = a X(X X) X Xβ = a Xβ = c β. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

44 Consider again our previous example. Recall that y 1 is a linear unbiased estimator of µ 1 µ 2. Is this the least squares estimator? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

45 It can be shown that the least squares estimator of µ 1 µ 2 is 7 11 y y y y y y 6. Based on the observed y, the least squares estimate is 8.0. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

46 One of infinitely many solutions to the NE is ˆβ = (X X) X y = Which team is best? Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

47 Suppose y = Xβ + ε, where E(ε) = 0 and rank( n p X) = p. Show that c β is estimable c R p. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

48 Proof: rank( n p X) = p X has p LI columns and each column is an element of R p. Thus, by Fact V4, p LI columns of X form a basis for R p. C(X ) = R p. This also follows from Result A.7: C(X ) R p, dim(c(x )) = dim(r p ) = p C(X ) = R p. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

49 Now from Result 3.1, we know c β is estimable iff c C(X ). C(X ) = R p, c β is estimable c R p. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

50 Alternatively, we can prove the result by noting the following rank( n p X) = p rank(x X) = p by Corollary 2.2. Now X X is p p of rank p and (X X) 1 exists. Thus, ˆβ = (X X) 1 X y is the unique solution to NE X Xb = X y. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51

51 Note that c R p E(c ˆβ) = c (X X) 1 XE(y) = c (X X) 1 X Xβ = c β. Thus, c ˆβ = c (X X) 1 X y is a linear unbiased estimator of c β c R p. c β is estimable c R p. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 51