1 Solutons Homework 4 March 5, 018 Soluton to Exercse 5.1.8: Let a IR be a translaton and c > 0 be a re-scalng. ˆb1 (cx + a) cx n + a (cx 1 + a) c x n x 1 cˆb 1 (x), whch shows ˆb 1 s locaton nvarant and scale equvarant. It s not clear why anyone would want to compute ths. Turnng to the others: ˆb (cx + a) (cx (n) + a) (cx (1) + a) c(x (n) x (1) ) cˆb (x). Note that snce c > 0, f we defne y cx+a then the order statstcs (sorted values) of y satsfy y () cx () + a, a fact that was used n the frst step of the computaton above. Note that the statstc analyzed here s the sample range, whch s used often for varous reasons. ˆb3 (cx + a) 1 n(n 1) cˆb 3 (x). ˆb4 (cx + a) n c (cx () + a) 1 n n c c x () + a c 1 1 n c c x () 1 cˆb 4 (x). (cx + b) c(x j + b) j In the last verfcaton, we used the comment about the relaton between the order statstcs of y and x at the frst step, and the assumpton that c 0 n the second to last step.
Soluton to Exercse 5..9: These are all exponental famles, so we wll apply Proposton 5..3. For part (a), f θ (x) 1 x! θx e θ exp[x log θ θ]h(x). Cleary η(θ) log θ s dfferentable and the dervatve 1/θ has rank 1 for all θ (.e., s nonzero). The formula for the Fsher Informaton n equaton (5.40) s I(θ) (1/θ) Var θ (X) 1/θ. Note as an asde that X s an unbased estmator of θ and Var θ (X) θ 1/I(θ), so X s UMVUE for θ. Turnng to part (b), n f θ (x) θ x (1 θ) n x exp[x log(θ/(1 θ)) + n log(1 θ)]h(x). x The dervatve of the natural parameter functon s η (θ) d dθ log(θ/(1 θ)) 1 θ + 1 1 θ 1 θ(1 θ). As ths clearly exsts for 0 < θ < 1, we conclude that Proposton 5..3 apples. Now, the Fsher Informaton s I(θ) η (θ) Var θ [X] nθ(1 θ) [θ(1 θ)] n θ(1 θ). Agan, as an asde, we can easly see that n 1 X s an unbased estmator of θ whose varance equals the lower bound, so t s UMVUE. Fnally, for part (c), the pmf s m + x 1 f θ (x) θ m (1 θ) x exp[x log(1 θ) + m log(θ)]h(x). m 1 Clearly, η (θ) d 1 log(1 θ) dθ 1 θ exsts. One can use Proposton.3.1(b) (on the moments of the suffcent statstc n an exponental famly; we have to substute n θ 1 e η nto m log(θ)) to derve that Var θ (X) m(1 θ) θ.
3 Therefore, I(θ) 1 m(1 θ) 1 θ θ m θ (1 θ). Soluton to Exercse 5..16: If f( x) f(x), then ψ(x) f (x)/f(x) satsfes ψ( x) ψ(x) (snce f ( x) f (x)), so ψ ( x) ψ (x), and therefore I 1 (a, b) 1 a xψ (x)f(x) dx 0, snce x s an odd functon and ψ (x)f(x) s an even functon. nformaton matrx s dagonal: I(a, b) 1 a [ I11 (1, 0) 0 0 I (1, 0) ], Thus, the and thus I 1 (a, b) a [ I 1 11 (1, 0) 0 0 I 1 (1, 0) Now t s clear that a I 11 (1, 0) (respectvely a I (1, 0)) are the Informatons for locaton estmaton when scale s known (respectvely, scale estmaton when locaton s known) for a sngle sample, so ths shows parts (a) and (b). For part (c), smply note that all the p.d.f. s mentoned there are symmetrc, so the result apples to them. Soluton to Exercse 5.3.: For part (a), the p.d.f. w.r.t. countng measure on IN {0, 1,,...} s whch s an exponental famly wth f µ (x) exp [x log µ µ] 1 x!, ]. T (x) x, η(µ) log(µ). Note that x s not a.s. constant (.e., does not satsfy a lnear constrant n one dmenson) and η ranges over IR as µ ranges over (0, ), so the famly s full rank, and X s complete and suffcent for a sngle sample, and T X s complete and suffcent for n..d. observatons. We know from e.g. a m.g.f. argument that T s P osson(nµ).
4 g(µ) s U-estmable f and only f there s a δ : IN IR such that g(µ) k0 δ(k) (nµ)k e nµ, µ > 0. k! Multplyng through by e nµ we see that e nµ g(µ) has a power seres that converges for all µ > 0, so t must n fact converge for all real µ (ndeed, for all complex µ), and I hope that you at least learned n calculus that f a power seres centered at 0 (a.k.a. a McLaurn seres) converges for some x, then t converges for any y wth y < x. Furthermore, such power seres representatons are unque and are gven by the classcal Taylor formula: e nµ g(µ) δ(k)n k 1 k0 k! µk µ f and only f k dk δ(k) n dµ k [enµ g(µ)] µ0. In concluson, g(µ) s U-estmable f and only f t has a power seres expanson (centered at 0) vald for all real numbers (otherwse sad, s an entre analytc functon), and then the UMVUE s δ(t ) where δ s gven by formula above. For part (c), t s sometmes easer to use ad hoc methods rather than the formula above to fnd the UMVUE. () g(µ) µ k s already gven as a power seres expanson, and t s easy to see that power seres multply, so the power seres we want s e nµ µ k µ k n j µ j. n k 1 j!! n µ! ( k)! j0 k Matchng the coeffcents, t s clear that the UMVUE s δ(t ) where In partcular, δ(t) { 0 f t < k, n k t!/(t k)! f t k. UMVUE for µ n 1 T X
5 and UMVUE for µ { 0 f T 0 or T 1 T (T 1)/n f T. T (T 1)/n Note that f we wrte t!/(t k)! k 1 0 (t ), then the UMVUE for µ k may be wrtten more smply as k 1 δ(t) n k (t ). 0 () Of course, Thus, g(µ) P µ [X 1 k] µk k! e µ. e nµ g(µ) µk k! e(n 1)µ µk k! k j0 1 j! (n 1)j µ j! ( k)!k! n (n 1) k n 1! µ, and we see that the desred UMVUE s t δ(t) n t (n 1) t k. k Note that the bnomal coeffcent our UMVUE of P µ [X 1 0] s ( t k δ(t) n t (n 1) t (1 1/n) t. ) s 0 unless k t. In partcular,
6 () Note that g(µ) log µ does not have the requred Taylor seres expanson (n partcular, t doesn t have a fnte value and dervates at µ 0). No unbased estmate exsts. (v) Pluggng n our formula, (I am changng the estmand to g(µ) e aµ ) e nµ e aµ e (n+a)µ n + a k n k 1 k0 n k! µk, from whch we can read off the UMVUE as δ(t) n + a t (1 + a/n) t. n (v) Pluggng n agan, e nµ e µ + µ k0(nµ ) k 1 k! 1 k k n j µ j+(k j) k! j k0 k0 k j0 (by the bnomal formula) 1 k k n j µ k j k0 k! j j0 k 1 k n k µ k! k 0 k /! ( k)!(k!) n(k ) 1! n µ. In the above, / denotes the smallest nteger /,.e. / f s even and ( + 1)/ f s odd. From ths, we can read off the UMVUE as T T! δ(t ) (T k)!(k T!) n(k T ) k T/ (v) Ths one s easy: e µ s at µ 0, so t can t have the requste Taylor seres expanson.
Soluton to Exercse 5.3.6: We have from the fact that σ n 1 (X X) has a χ n 1 dstrbuton that E[ˆσ (a)] (n 1)aσ Var[ˆσ (a)] (n 1)a σ 4. Usng the formula MSE Bas + Varance, we obtan σ 4 MSE[ˆσ (a)] [a(n 1) 1] + (n 1)a. Thus, wth a lttle algebra we obtan σ 4 { MSE[ˆσ (a)] MSE[ˆσ (1/(n 1))] } (n 1)a (n 1)a + 1 /(n 1) (n 1)[a 1/(n 1)][a 1/(n + 3)]. Note that the r.h.s. s a quadratc functon of a whose graph s an upward openng parabola wth roots at a 1/(n 1) and a 1/(n+3). Thus, ˆσ (a) has smaller MSE than the UMVUE ˆσ (1/(n 1)) for (n+3) 1 < a < (n 1) 1. Soluton to Exercse 5.3.8: (a) The lkelhood s [ f(y; a, b, σ ) (πσ ) n/ exp 1 ] n (y σ (ax + b)) 1 [ (π) n/ exp 1 y σ + a x σ y + b σ 1 (ax σ + b) n ] log σ As long as we have at least two dstnct values of the x, then the famly wll be dentfable, as dfferent values of a and b wll gve dfferent means. Ths assumpton was apparently forgotten. Thus, the suffcent statstc vector T ( x Y, Y, Y ) does not satsfy any lnear constrants. As (a, b, σ ) range over IR IR (0, ), the natural parameter vector ranges over (, 0) IR IR, whch s an open set, so has nonempty nteror. Thus, the famly s full rank, hence T s complete and suffcent. (b) Clearly Ȳ n 1 T s a functon of T. Also, y 7 n x Ỹ 1 x Y x Y Ȳ x
8 s a functon of T. Thus, we only need to check that â s unbased for a. Note that E[Ȳ ] a x + b. Thus, E (a,b,σ )[â] 1 x x E[Ỹ] 1 x x [(ax + b) (a x + b)] a. 1 x a x (x x ) Thus, â s a functon of the complete and suffcent statstcs whch s unbased for a, so t s the UMVUE of a. We see mmedately that ˆb s also a functon of T (snce Ȳ and â are), so we need only check that t s expectaton s always b. Thus, (c) Clearly ˆσ E (a,b,σ )[ˆb] E[Ȳ ] E[â] x a x + b a x b. 1 [ Y n + (âx + ˆb) â x Y ˆb ] Y, whch s a functon of T. Now we want to check that ts expectaton of σ. It s almost always easer to do these types of calculatons f one subtracts and adds E[Y ] n the squared quantty: (n )E[ˆσ ] [ (Y E E[Y ] + E[Y ] âx ˆb ) ] [ (ɛ E + (a â)x + (b ˆb) ) ] (snce Y E[Y ] + ɛ and E[Y ] ax + b) â a Ỹ x x a Ỹ Y Ȳ ax + b + ɛ (a x + b + ɛ) a x + ɛ (where ɛ ɛ ɛ)
9 â a (a x + ɛ ) x x a Put ɛ x x ˆb b Ȳ â x b a x + b + ɛ â x b (a â) x + ɛ (â a)x + (ˆb b) (â a) x + ɛ j ɛ j x j x j x + ɛ j Note that w x ( j x j ) 1/. w 1, w 0, where the latter follows because x (x x) 0. Thus, we have (n )E[ˆσ ] E ɛ w w j ɛ j ɛ j E ɛ w w j ɛ j j E ɛ ɛ w w j ɛ j + w w j ɛ j j j E ɛ w j ɛ j. j Now [ ] E ɛ [ ] E (ɛ ɛ) (n 1)σ.
10 Also, E j w j ɛ j E j E j w j (ɛ j ɛ) w j ɛ j (snce wj 0) σ j w j σ. Thus, we get n the end that E[ˆσ ] σ, as desred.