Exercise 7 / page 356 Noe ha X i are ii from Beroulli(θ where 0 θ a Meho of momes: Sice here is oly oe parameer o be esimae we ee oly oe equaio where we equae he rs sample mome wih he rs populaio mome, m µ, a we solve i erms of θ m X a µ E(X, where E(X xp (X x P (X θ x0 Hece, he meho of momes esimaor of θ is ˆθ MM X Noe hf more ha half ou of he observaios are equal o, he X > a ˆθ MM ges a value which is ou of he rage of parameer θ Recall ha 0 θ MLE of θ: We have o eermie he likelihoo fucio rs: L(θ x P (X i x i θ θ x i ( θ x i θ x i ( θ x i a for θ (0, ] he log-likelihoo is of he form: log L(θ x x i log θ + ( x i log( θ (Noe ha θ [0, ] bu he log-likelihoo is o ee for θ 0 So, we may ee o suy he fucio L(θ x a his specic poi separaely The rs erivaive of log L(θ x wih respec o θ is log L(θ x θ θ x i θ ( θ x i θ( x i θ( θ ( x θ θ( θ For θ log L(θ x 0 we ge θ x The seco erivaive of log L(θ x wih respec o θ is ( log L(θ x [ ] x i θ θ θ θ θ θ ( θ < 0 Therefore, θ x is global mamum poi (a we o o ee o suy L(θ x a θ 0 Noe ha θ [0, ], ie we ee o he global mamum of log L(θ x wihi his rage of θ values Therefore, we coclue ha: ˆθ MLE { X if X if X > or ˆθ MLE mi { X, }
(Observe ha for θ < x, log L(θ x > 0, ie log L(θ x a subsequely L(θ x are icreasig, θ a for θ > x, log L(θ x < 0, ie log L(θ x a subsequely L(θ x are ecreasig b θ MSE(ˆθ MM [ Bias(ˆθ MM ] + V ar(ˆθmm, where Bias(ˆθ MM E(ˆθ MM θ a ˆθ MM X Moreover, E(ˆθ MM E( X E(X θ Bias(ˆθ MM 0, a sice X i are ii from Beroulli(θ Thus, V ar( X V ar(x θ ( θ MSE(ˆθ MM θ ( θ For MSE(ˆθ MLE i is preferable o work wih he eiio of MSE, ie: ( { } ( { } Xi MSE(ˆθ MLE E (ˆθMLE θ E mi X, θ E mi, θ Sice X i Beroulli(θ, 0 θ, he X i Bi(, θ wih 0 θ Le T { } X i The mi Xi, is a fucio of he raom variable T where T Bi(, θ Thus: ( { } T ( { } ( MSE(ˆθ MLE E mi, θ mi, θ θ ( θ 0 Noe hf is eve he mi {, } { for 0,, / for (/ +,, a if is o he mi {, } Le [/] { / ( / { if eve if o, he for 0,, ( / for ( + /,, { } { mi, for 0,, [/] for [/] +,, Thus, he expecaio above ca be wrie as MSE(ˆθ MLE [/] 0 ( ( θ θ ( θ + [/]+ ( θ ( θ ( θ
c We will base our aswer o he quesio which of he wo esimaors is prefere o he MSE crierio To compare he MSE of he wo esimaors i woul help if we express MSE(ˆθ MM i a similar fashio as we i for MSE(ˆθ MLE Thus, MSE(ˆθ MM E ( X θ E ( T θ [/] 0 ( ( θ 0 θ ( θ + ( ( θ [/]+ θ ( θ ( ( θ θ ( θ Sice > for [/] +,,, he ( θ < ( θ for all [/] +,, a [/]+ ( θ ( θ ( θ < [/]+ ( ( θ θ ( θ for all θ (0, ] Hece, MSE(ˆθ MLE < MSE(ˆθ MM for all θ (0, ] If θ 0 he MSE(ˆθ MLE MSE(ˆθ MM Thus, ˆθ MLE is preferre base o he MSE crierio Oe coul hik as follows as well: Sice ˆθ MM X a ˆθ MLE mi { X, } he ˆθMLE ˆθ MM Therefore, (ˆθMLE θ (ˆθMM θ a E (ˆθMLE θ E (ˆθMM θ (see Theorem 5, page 57, ie MSE(ˆθMLE MSE(ˆθ MM Thus, we woul prefer ˆθ MLE agai o be o he safe sie bu i woul o be clear i which cases equaliy hols where boh esimaors perform equally well base o he MSE crierio Exercise 738 / page 356 To aswer he quesio of he exercise we use Corollary 735 a I orer o use he corollary we have o check rs wheher f(x θ saises he coiio of Cramér-Rao iequaliy, ie wheher he ierchage of iegraio a iereiaio is allowe Noice ha f(x θ θx θ is a expoeial family isribuio (i ca be wrie i he form f(x θ h(xc(θ exp ( k w k(θ k (x Thus, he requireme of Cramér-Rao iequaliy hols So, ow we ca apply he corollary The likelihoo fucio is of he form: L(θ x f(x θ f(x i θ θx θ i ( θ θ x i a The log L(θ x log θ + (θ log x i ( θ log L(θ x θ + log x i log x i θ, 3
ie log L(θ x ca be facorize i he form log L(θ x a(θ [W (X τ(θ] Hece, accorig o Corollary 735, log X i θ θ is a bes ubiase esimaor of θ b f(x θ log θ θ θx is a expoeial family isribuio a herefore he requireme of Cramér- Rao iequaliy hols Thus, we ca apply Corollary 735 The likelihoo fucio is of he form: a The L(θ x f(x θ f(x i θ ( log θ θ θx i log θ θ x i θ ( log L(θ x log (log θ log (θ + x i log θ log L(θ x θ θ log θ θ + θ x i θ ( ( θ x θ log θ ie log L(θ x ca be facorize i he form log L(θ x a(θ [W (X τ(θ] Hece, accorig o Corollary 735, X is a bes ubiase esimaor of θ θ θ θ log θ, Exercise 740/ p 36 Oe soluio coul be usig Cramér-Rao iequaliy & Corollary 730 & Lemma 73: Firs of all, we check ha X is a ubiase esimaor of p Sice X i are ii from Beroulli(p, i,,, we ge E( X E(X p We also compue he variace of X i orer o compare i wih he Cramér-Rao bou, V ar( X V ar(x p( p Sice Beroulli(p is a expoeial family isribuio, he ierchage of iegraio a iffereiaio is allowe a V ar( X <, so he requiremes of Cramér-Rao iequaliy hol Thus, ( p E( X V ar( X [ ( ], E log f(x p pθ [ ( where E( X ] [ ( ] p a E log f(x p E log f(x p sice X p p p p i are ii (see [ ( ] ( Corollary 730 Furhermore, E log f(x p E log f(x p because Beroulli(p p p is a expoeial family isribuio a Lemma 73 hols To compue he eomiaor: f(x p p x ( p x log f(x p x log p + ( x log( p p log f(x p x ( x p p p log f(x p x ( x p ( p 4
( ( E log f(x p E X ( X E(X + E(X p p ( p p ( p p + p p( p Thus, he Cramér-Rao lower bou is ( E( X p [ ( ] E log f(x p pθ ( E log f(x p p p( p p( p which is acually equal o V ar( X Hece, X is a bes ubiase esimaor of p Base o Theorem 739 we coclue h is he bes ubiase esimaor of p Aleraively, we coul use Corollary 735: X i are ii from Beroulli(p which saises he coiio of he Cramér-Rao Theorem sice Beroulli(p is a expoeial family isribuio The likelihoo fucio is of he form: L(p x f(x p f(x i p p x i ( p x i p x i ( p x i a he log-likelihoo is log L(p x ( log p + ( x i log( p a he rs erivaive of he log-likelihoo wih respec o θ is log L(p x p p x i p p p( p ( x p p( p Sice log L(p x ca be facorize i he form log L(p x a(p [W (x τ(p], accorig o p p Corollary 735, X is a ubiase esimaor of p a i aais he Cramér-Rao Lower bou Hece, X is a bes ubiase esimaor of p a base o Theorem 739 i is he bes ubiase esimaor of p Exercise 74/ p 363 X,, X is a raom sample where E(X i µ a V ar(x i σ a ( E X i E(X i µ µ If, he E ( X i µ, ie a ubiase esimaor of µ b Firsly, we eermie he variace of a esimaor of ype X i where : ( V ar X i a i V ar(x i a i σ σ a i 5
To he esimaor wih he miimum variace we ca see he above as a fucio of 's, i,, ha we wa o miimize uer he coiio Equivalely, we ca miimize he fucio: ( f(a, a,, a a i + (Observe ha he laer is a fucio of 's, i,,, a a is expresse as a base o he resricio ha The parial erivaives of f(a, a,, a are f(a, a,, a a a ( a f(a, a,, a a ( ( f(a, a,, a a a ie f(a, a,, a ( for i,, Seig all parial erivaives equal o 0 we ge a a a ie, for i,, Sice he righ ha sie i all equaios is he same, i hols ha se a a a k Replacig i ay of he above equaios he 's wih k we ge: ( k k k ( k k k The seco parial erivaives are f(a, a,, a 4 > 0, i,, 6
a j f(a, a,, a > 0, i, j,, a i j Thus, f(a, a,, a reaches is global miimum a he poi (a, a,, a (,,, Sice a, a his poi a akes he value: a Hece, i he case of X i variables wih he same mea a variace, amog all liear ubiase esimaors of ype X i for µ, he oe wih all 's equal o /, ie X, has he miimum variace Is variace is V ar( X σ V ar(x 7