1. Parameter estimation point estimation and interval estimation. 2. Hypothesis testing methods to help decision making.

Transcription

1 Chapter 7 Parameter Estimatio 7.1 Itroductio Statistical Iferece Statistical iferece helps us i estimatig the characteristics of the etire populatio based upo the data collected from (or the evidece 0produced by) a sample. Statistical iferece may be divided ito two major areas: 1. Parameter estimatio poit estimatio ad iterval estimatio. 2. Hypothesis testig methods to help decisio makig. 7.2 Poit Estimatio I the followig, we summarize the parameters of several importat distributios: B(, p) p (proportio); G(p) p (proportio); NB(r, p) p (proportio); P (λ) λ (arrivig rate); Exp(λ) λ (failure rate); Γ(α, λ) α, λ; N(µ, σ 2 ) µ (mea), σ 2 (variace); U(α, β) α (miimum), β (maximum). The major goal of poit estimatio is to estimate the parameter(s) for a populatio. Def 1 Ay statistic ˆθ ˆθ(X 1, X 2,..., X ) used to estimate the value of a parameter θ of the populatio is called a estimator of θ. A observed value of statistic ˆθ ˆθ(x 1, x 2,..., x ) is kow as a estimate of θ. 1

2 Example 1 To fid the mea lifetime for a iexpesive brad of ball-poit pe. Suppose the lifetime is ormally distributed with parameters µ ad σ 2. A special mechie is used to test 10 such pes. The observed lifetimes are: x , x , x , x , x , x , x , x , x , x Please suggest some methods to estimate µ. 1. ˆµ X ˆµ x x i / ˆµ X ˆµ x ( )/ ˆµ [mi(x i ) + max(x i )]/2 ˆµ ( )/ ˆµ X tr(10) ˆµ x tr(10) The Criteria for a Good Estimator 1. Ubiasedess; 2. Efficiecy; 3. Cosistecy; 4. Sufficiecy. Ubiasedess Def 2 A statistic ˆΘ ˆΘ(X 1,..., X ) is said to costitude a ubiased estimator of parameter θ provided E[ ˆΘ] θ. If ˆΘ is ot ubiased, the differece E[ ˆΘ] θ is called the bias of Θ. 2

3 Example 2 Let X is a biomial radom variable with parameters ad p. Show that the sample proportio ˆp is a ubiased estimator of p. pf) Sice X B(, p), we have E[X] p. Thus, E[ˆp] E[X/] p/ p. This cocludes that ˆp is a ubiased estimator of p. Example 3 Suppose that the reactio time to a certai stimulus has a uiform distributio o the iterval from 0 to a ukow upper limit θ. Let X 1,..., X be the reactio times i a sample. Let ˆΘ 1 max(x i ), ˆΘ ˆθ ˆΘ 3 2X. max(x i), be three differet estimators for parameter θ. Discuss the ubiasedess or biasedess for each estimator. 1. F ˆΘ1 (x) P ( ˆΘ 1 x) P (X 1 x, X 2 x,..., X x) P (X 1 x)p (X 2 x) P (X x) ( ) x θ f ˆΘ1 (x) x 1 E[ ˆΘ 1 ] θ θ ( x x 0 θ + 1 θ 0 x θ ) 1 θ dx Hece, ˆΘ 1 is a biased estimator for θ with bias θ/( + 1). 3

4 2. Clearly, ˆΘ 1 is a ubiased estimator for θ. 3. Efficiecy E[ ˆΘ 3 ] E[2X] Hece, ˆΘ 3 is a ubiased estimator for θ. 2E[X] θ Def 3 If statistic ˆΘ is employed as a poit estimator of parameter θ, the the mea-square error associated with ˆΘ is E[( ˆΘ θ) 2 ] Facts Def 4 Give two estimators of θ, say ˆΘ 1 ad ˆΘ 2, ˆΘ 1 is called a better estimator, or more efficiet estimator, tha ˆΘ 2 if Facts 1. E[( ˆΘ 1 θ) 2 ] < E[( ˆΘ 2 θ) 2 ] E[( ˆΘ θ) 2 ] E[ ˆΘ 2 ] 2E[ ˆΘθ] + E[θ 2 ] E[ ˆΘ 2 ] E[ ˆΘ] 2 + E[ ˆΘ] 2 2θE[ ˆΘ] + θ 2 V ar[ ˆΘ] + (E[ˆθ] θ) 2 2. If ˆΘ is a ubiased estimator, the E[( ˆΘ θ) 2 ] V ar[ ˆΘ] 3. Hece, attetio is restricted to ubiased estimators, the ˆΘ 1 is more efficiet tha ˆΘ 2 if ad oly if V ar[ ˆΘ 1 ] < V ar[ ˆΘ 2 ] Example 4 Compare the efficiecy for estimators ˆΘ 1, ˆΘ 2 ad ˆΘ 3 i Example 3. 4

5 E[ ˆΘ 1 ] + 1 θ θ ) 1 θ dx E[ ˆΘ ( x 2 1] x 2 0 θ + 2 θ2 E[( ˆΘ 1 θ) 2 ] E[ ˆΘ 2 1] 2θE[ ˆΘ 1 ] + θ 2 2θ 2 ( + 1)( + 2) E[( ˆΘ 2 θ) 2 ] V ar[ ˆΘ 2 ] E[ ˆΘ 2 2] (E[ ˆΘ 2 ]) 2 ( ) E[ ˆΘ 2 1 ] θ 2 θ 2 ( + 2) E[( ˆΘ 3 θ) 2 ] V ar[ ˆΘ] V ar[2x] 4V ar[x] θ2 3 Amog them, ˆΘ 2 is the best, ˆΘ 1 is secod, ad ˆΘ 3 is the worst. Def 5 A estimator ˆΘ of parameter θ is said to be cosistet if ˆΘ coverges i probability to θ, i.e., lim P ( ˆΘ θ ɛ) 0 Example 5 Show that ˆΘ 1 i Example 3 is a cosistet estimator pf) By Chebyshev s iequality, Facts: P ( ˆΘ µ Θ ɛ) σ2ˆθ ɛ 2 5

6 1. lim µ ˆΘ lim E[ ˆΘ 1 ] θ 2. σ 2ˆΘ E[ ˆΘ 2 ] (E[ ˆΘ]) 2 ( + 2)( + 1) 2 θ2 lim σ 2ˆΘ 0 Hece, lim P ( ˆΘ θ ɛ) lim P ( ˆΘ µ Θ ɛ) σ 2ˆΘ lim ɛ 0 2 Theorem 1 If ˆΘ is a ubiased estimator of θ for which the ˆΘ is a cosistet estimator of θ. pf) (By Chebyshev s iequality) lim V ar[ ˆΘ] 0 Example 6 Let X 1, X 2,..., X be a sample from a populatio with mea µ ad variace σ 2. Cosider the two ubiased estimators: ˆµ 1 X X X ˆµ 2 X 1 + X 2 Show that ˆµ 1 is a cosistet estimator os µ, while ˆµ 2 is ot. pf) 1. For ˆµ 1 V ar[ˆµ 1 ] σ2 lim V ar[ˆµ 1 ] 0 6

7 Hece, ˆµ 1 is cosistet. 2. For ˆµ 2 V ar[ˆµ 2 ] σ2 4 lim V ar[ˆµ 1 ] 0 Hece, ˆµ 2 is ot cosistet. Sufficiecy Def 6 Let X 1, X 2,..., X be a sample from a populatio with pdf f(x; θ). If the jpdf f(x 1, x 2,..., x ; θ) f(x 1 ; θ)f(x 2 ; θ) f(x ; θ) ca be decomposed ito the product of f(x 1, x 2,..., x ; θ) g(ˆθ, θ)h(x 1, x 2,..., x ) where h(x 1, x 2,..., x ) is idepedet of θ, the ˆθ is a sufficiet estimator (statistic) of θ. Example 7 Let X 1, X 2,..., X be a sample from a populatio with pdf f(x; β) 1 β e x/β, x > Show that ˆβ X is a ubiased estimator of λ. 2. Show that ˆλ is a sufficiet estimator of λ. pf) 1. Oe ca easily verify that E[X i ] β. 2. Hece, E[ ˆβ] β. This cocludes that ˆβ is a ubiased estimator of β. f(x 1, x 2,..., x ; β) f(x 1 ; β)f(x 2 ; β) f(x ; β) 7 e x 1/β β e i x i/β β e X/β β e x 2/β e x/β β β

8 Let The, g( ˆβ, β) e X/β β h(x 1, x 2,..., x ) 1. f(x 1, x 2,..., x ; β) g( ˆβ, β)h(x 1, x 2,..., x ), x i > 0 This cocludes the sufficiecy of ˆβ. Example 8 Let X 1 B(m, p) ad X 2 B(, p). 1. Show that ˆp X 1 + X 2 m + 2. Show that ˆp is sufficiet. is a ubiased estimator of p. pf) This cocludes the ubiasedess of ˆp. Let E[ˆp] [ ] X1 + X 2 E m + 1 m + E[X 1 + X 2 ] 1 m + (E[X 1] + E[X 2 ]) (m + )p m + p f(x 1, x 2 ; p) f(x 1 ; p)f(x 2 ; p) ( ) ( ) m p x 1 (1 p) m x 1 p x 2 (1 p) x 2 x 1 x 2 ( )( ) m p x 1+x 2 (1 p) (m+) (x 1+x 2 ) x 1 x 2 ( )( ) m p (m+)ˆp (1 p) (m+)(1 ˆp). x 1 x 2 g(ˆp, p) p (m+)ˆp (1 p) (m+)(1 ˆp) ( )( ) m h(x 1, x 2 ). x 1 x 2 8

9 The, f(x 1, x 2 ; p) g(ˆp, p)h(x 1, x 2 ) This cocludes the sufficiecy of ˆp. 7.3 Miimum-Variace Ubiased Estimator (MVUE) Def 7 A ubiased estimator ˆθ is said to be a miimum-variace ubiased estimator of θ is, for ay ubiased estimator ˆθ of θ, V ar[ˆθ] V ar[ˆθ]. ˆθ is also called a best ubiased estimator of θ. Theorem 2 [Cramer-Rao Iequality] If ˆθ is a ubiased estimator of the parameter θ i the desity of radom variable, the V ar[ˆθ] 1 [ ( E log f(x; θ θ)) 2 ] where is the size of the radom variable. Example 9 Show that the sample mea is a MVUE for the mea of a ormally distributed radom variable. f(x; µ) 1 e 1 2( x µ σ ) 2 2πσ ( x µ ) 2 log f(x; µ) log(σ 2π) 1 2 σ x µ log f(x; µ) µ σ 2 ( ) 2 E log f(x; µ) σ 4 E[(X µ) 2 ] σ 2 µ 9

10 By Cramer-Rao Iequality V ar[x] 1 E[ ] σ2 So, X is a MVUE of µ. Def 8 Let X 1, X 2,..., X be a radom sample from a populatio. Cosider the weighted average of the sample variables, say Y a 1 X 1 + a 2 X a X where a 1 +a 2 + +a 1. Such a liear combiatio of the radom variables X 1, X 2,, X is called a covex combiatio. Facts 1. The sample mea X 1 (X 1 + X X ) is a special covex combiatio of radom sample X 1, X 2,..., X. 2. Ay covex combiatio of radom variables X 1, X 2,..., X is a ubiased estimator of the correspodig populatio-mea. Theorem 3 Amog all covex combiatios of radom variables X 1, X 2,..., X, X is a ubiased estimator of the mea with miimum variace, i.e., the sample mea is a best liear ubiased estimator (BLUE) of the mea. pf) X is udoubtedly a ubiased estimator of the mea µ. We ow show that it possesses the miimum variace amog all the liear combiatios. Let ˆµ a 1 X 1 + a 2 X a X where a 1 + a a 1. The, V ar[ˆµ] E[ˆµ 2 ] (E[ˆµ]) 2 (a a a 2 )σ 2 µ 2 Cosider the eergy fuctio: where λ. E(a 1, a 2,..., a ) (a a a 2 ) + λ(a 1 + a a 1) 2 With a little ivestigatio, oe ca see that the miimizatio of E correspods to the miimizatio of V ar[ˆµ]. 10

11 By settig all get Remark The BLUE is ot always a MVUE. E a i 0, ad solvig the correspodig system of liear equatios, we a 1 a 2 a 1 Example 10 Suppose the errors i a measuremet process are uiformly distributed betwee -1/2 ad 1/2 grams. Compare the efficiecy of the followig two ubiased estimators: ˆµ 1 X ˆµ 2 mi(x i) + max(x i ) 2 For ˆµ 1 For ˆµ 2 We ca show that fˆµ2 (x) E[ˆµ 2 ] 0 1/2 V ar[ˆµ 1 ] ( x) ( 1 2 x) 1 x ( x ) 1 dx + 1 2( + 1)( + 2) V ar[ˆµ] E[ˆµ 2 ] (E[ˆµ]) < x < 0 0 x 1 2 1/ ( + 1)( + 2) Sice V ar[ˆµ 2 ] < V ar[ˆµ 1 ] for 3, ˆµ 2 is more efficiet tha ˆµ Methods of Poit Estimatio The Method of Momets x ( 1 2 x ) 1 dx Def 9 Let X 1, X 2,..., X be a sample from a pmf or pdf f(x). For k 1, 2, 3,..., the kth populatio momet µ k ad the kth sample momet are defied by: µ k E[X k ] M k 1 11 Xi k i1

12 Remark 1. M 1 X; 2. M 2 The mothod ( 1)S2 + X 2. Let X 1, X 2,..., X be a radom sample from a distributio with pmf or pdf f(x; θ 1,..., θ m ), where θ 1,..., θ m are parameters whose values are ukow. The, the estimators ˆθ 1,..., ˆθ 2 ca be foud as follows: 1. List the followig equatios: 2. Solve the above equatios such that 3. Simply let M 1 µ 1 g 1 (θ 1, θ 2,..., θ m ) M 2 µ 2 g 2 (θ 1, θ 2,..., θ m ). M m µ m g m (θ 1, θ 2,..., θ m ) θ 1 h 1 (M 1, M 2,..., M m) θ 2 h 2 (M 1, M 2,..., M m). θ m h 1 (M 1, M 2,..., M m) ˆθ 1 h 1 (M 1, M 2,..., M m) ˆθ 2 h 2 (M 1, M 2,..., M m). θˆ m h m (M 1, M 2,..., M m) Example 11 Let X 1, X 2,..., X represet a radom sample of service times of customers at a certai facility, where the uderlyig distributio is assumed expoetial with parameter λ. Use the method of memet to fid ˆλ. M 1 E[X] 1 λ 12

13 Solve it. We have λ 1 M 1 1 X Hece, ˆλ 1 X Example 12 Let X deote the mai memory requiremet of a job as a fractio of the total user-allocatable mai memory of a computig ceter. We suspect that the desity fuctio of X has the form: { (k + 1)x k 0 < x < 1, k > 0 f(x) 0 otherwise Use the method of momets to fid ˆk. Let Solve it, we have Hece, µ (x1)x k xdx k + 1 k + 2 M 1 µ 1 k + 1 k + 2 k 2M M 1 ˆk 2X 1 1 X 2X 1 1 X Example 13 Assume that the repair time of a computer system has a gamma distributio with parameters λ ad α. Use the method of momets to fid ˆλ ad ˆα. Clearly, µ 1 α λ ad µ 2 α λ 2 + ( α λ ) 2. 13

14 Let M 1 α λ ) 2 M 2 α λ 2 + ( α λ Solve it. We have λ α M 1 X M 2 M S 2 M 1 2 X 2 M 2 M S 2 Hece, ˆλ ˆα X 1 S 2 M 1 2 M 2 M The Maximum-Likelihood Estimatio Def 10 Let X 1, X 2,..., X be idepedet radom variables take from a probability distributio represeted by the pmf or pdf f(x; θ), where θ (θ 1,..., θ m ). Now L(x 1, x 2,..., x ; θ) f(x 1, x 2,..., x ; θ) f(x 1 ; θ)f(x 2 ; θ) f(x ; θ) is the joit distributio of the radom variables. This is ofte referred to as the likelihood fuctio. The estimators ˆθ (ˆθ 1,..., ˆθ m ) of θ satisfig L(x 1, x 2,..., x ; ˆθ) L(x 1, x 2,..., x ; ˆθ) for all ˆθ (ˆθ1,..., ˆθm ) are called the maximum likelihood estimators (MLE) of θ. The Method 1. List the likelihood fuctio of a populatio. 2. Use a coveiet way (e.g., differetiatio if possible) to fid ˆθ so as to maximize the likelihood fuctio. Example 14 Fid the MLE of parameter p for a Beroulli distributio 14

15 The Likelihood fuctio for Beroulli distributio is L(x 1,..., x ; p) p i x i (1 p) i x i ( ) ( l L x i l p + i i ) x i l(1 p) To maximize L is equivalet to maximize l L. Let p 0 p l L i x i p i x i i x i 1 p This cocludes that the MLE of p is ˆp i X i Example 15 Fid the MLE of µ ad σ 2 for a ormal populatio N(µ, σ 2 ). The Likelihood fuctio for ormal distributio is L(x 1,..., x ; µ, σ 2 ) i1 1 e (x 1 µ)2 2 2πσ 1 (2π) /2 (σ 2 ) 1 e 2σ /2 2 i1 (x i µ) 2 l L 2 l 2π 2 l σ2 1 2σ 2 (x i µ) 2 i1 To maximize L is equivalet to maximize l L. Let 0 µ l L 1 σ 2 i1 x i ˆµ x 15 (x i µ) i1

16 Let ˆσ 2 0 σ 2 l L 2 (σ2 ) (σ2 ) 2 (x i x) 2 i1 (x i µ) 2 i1 Remark From the above example, oe the see that a MLE is ot ecessary a ubiased estimator. Example 16 Each of specimes is to be weighted twice o the same scale. Let X i ad Y i deote the two observed weights for the ith specime. Suppose X i ad Y i are idepedet of oe aother, each ormally distributed with mea value µ i (the true weighted of specime i) ad variace σ Show that the MLE of σ 2 is ˆσ 2 (X i Y i ) 2 4 i1 2. Is the MLE a ubiased estimator of σ 2? If ot, fid a ubiased estimator of σ The Likelihood fuctio will be Takig L(x 1, y 1, x 2, y 2,..., x, y ; µ 1, µ 2,..., µ, σ 2 ) ( ) ( ) 1 e (x i µ i )2 1 2σ 2 e (y i µ i )2 2σ 2 2πσ 2πσ i1 1 i1 2πσ 2 e ( 1 2πσ 2 (x i µ i ) 2 +(y i µ i ) 2 2σ 2 ) e 1 2σ 2 [(xi µ i ) 2 +(y i µ i ) 2 ] l L l 2πσ 2 1 2σ 2 [(xi µ i ) 2 + (y i µ i ) 2 ] µ i l L 0, gives Substibutig ˆµ i ito l L gives ˆµ i x i + y i 2 l L l(2πσ 2 ) 1 [(xi x i + y i ) 2 + (y 2σ 2 i x i + y i ) 2 ] 2 2 l(2πσ 2 ) 1 (xi y 4σ 2 i ) 2 16

17 2. Remark By settig l L 0, we the have σ2 ˆσ 2 (X i Y i ) 2 i1 4 [ E[ˆσ 2 (X i Y i ) 2 ] ] E i1 4 1 E[(X i Y i ) 2 ] 4 i1 E[(X i Y i ) 2 ] V ar[x i Y i ] + (E[X i Y i ]) 2 2σ 2 E[ˆσ 2 ] σ2 2 Hece, MLE ˆσ 2 is ot ubiased. However, is a ubiased estimator of σ 2. 2ˆσ 2 (X i Y i ) 2 i1 1. A MLE may be ot ubiased ad icosistet. 2. Sometimes, we caot use derivative to maximize a likelihood fuctio. 2 Example 17 Cosider the uiform distributio over the iterval [α, β], where α ad β are ukow. Fie the MLE s for α ad β. The likelihood fuctio will be L(x 1,..., x ; α, β) i1 1 β α, α < x i < β for all i. We caot use differetiatio to maximize L. However, to maximize L, we ca simply miimize its deomiator. So, we ca try to miimize ˆβ ad maximize ˆα. Cosider ˆβ. If ˆβ < x i for some i, we have L 0, which certaily does ot maximize L. Thus, the MLE of beta must larger tha all of the sample values. This tells us that the MLE of β will be ˆβ max{x 1, X 2,..., X } 17

18 Likewise, oe ca easily see that the MLE for α will be ˆα mi{x 1, X 2,..., X } Example 18 To estimate the size of a wildlife populatio i a certai evioromet, M aimals is captured, each of these aimals is tagged, ad the aimals are the retured to the populatio. After allowig eough time for the tagged idividuals to mix ito the populatio, aother sample of size is captured. With X the umber of tagged aimals i the secod sample, fid the MLE for the populatio size N. The Likelihood fuctio correspodig to the secod sample is ( ) ( ) M N M x x L(x; N) p(x; N) ( ) N Because the iteger-valued ature of N, it we ca ot take the devivative of L. Now cosider To maximize p(x; N), we set The, Hece, the MLE for N is Bayes Estimator p(x; N) p(x; N 1) (N M) (N ) N(N M + x) (N M) (N ) N(N M + x) 1. N 2 N MN + M N 2 MN N + Nx M Nx N MN x ˆN MN x The classical methods of estimatio are based solely o the iformatio provided by the radom sample. These methods essetially iterpret probabilities as relative frequecies. Probability estimated i the frequecy sese are called Objective probabilities. The Bayesia approach combies sample iformatio with other available prior iformatio that may appear to be pertiet. The probabilities associated with the prior iformatio are called Subjective probabilities. 18

19 Basic Cocept 1. Risk fuctio Give estimator ˆθ ˆθ(x 1,..., x ) of θ, the true value of the parameter. The correspodig risk fuctio is defied by R(θ, ˆθ) E[(θ ˆθ) 2 ] (θ ˆθ) 2 p(x 1,..., x ) discrete case x 1 x + + (θ ˆθ) 2 dx 1 dx p(x 1,..., x ) cotiuous case 2. Bayes Risk Suppose that Θ is a radom variable possessig the prior desity π(θ). The, the risk fuctio ow is writte as R(Θ, ˆθ) Eˆθ[(Θ ˆθ) 2 ]. Hece, R(Θ, ˆθ) is a fuctio of θ. The Bayes risk ow is defied by B(ˆθ) E[R(Θ, ˆθ)] R(Θ, ˆθ)π(θ) θ + R(Θ, ˆθ)π(θ)dθ discrete case cotiuous case 3. Bayes Estimator For a give prior distributio π(θ), the estimator ˆθ 0 is called the Bayes estimator i the class C of estimators if B(ˆθ 0 ) mi B(ˆθ) ˆθ C 4. Cosider the followig pdf s (i cotiuous case): f(x 1,..., x θ) the jpdf of the sample give that Θ θ. Clearly, f(x 1,..., x θ) f(x i θ) i1 f(x i, θ) i1 π(θ) f(x 1,..., x, θ) the jpdf of the sample ad Θ. Accordig to the Bayes rule, we have f(x 1,..., x, θ) f(x 1,..., x θ)π(θ) f(θ x 1,..., x ) the posterior desity of Θ give a paricular sample. Accordig to the Bayes rule, we have where f(x 1,..., x ) f(θ x 1,..., x ) f(x 1,..., x, θ) f(x 1,..., x ) f(x 1,..., x, θ) θ f(x 1,..., x, θ)dθ 19 discrete case cotiuous case

20 5. B(ˆθ) + + R(θ, ˆθ)π(θ)dθ Eˆθ[(θ ˆθ(X 1,..., X )) 2 ]π(θ)dθ + [ E[(Θ ˆθ(X 1,..., X )) 2 ] [θ ˆθ(X 1,..., X )] 2 f(x 1,..., x θ)dx 1 dx ] π(θ)dθ [θ ˆθ(X 1,..., X )] 2 f(x 1,..., x, θ)dx 1 dx dθ Theorem 4 Suppose that X is a radom variable possessig desity f(x θ), where θ is cosidered to be a value of the radom parameter possessig prior desity π(θ). If X 1,..., X is a radom sample, the the Bayes estimator ralative to the prior distributio π(θ) is give by the coditioal expectatio ˆθ E[Θ X 1,..., X ] + θf(θ x 1,..., x )dθ Example 19 Suppose that the proportio p of defective uits resultig from a maufacturig process teds to possesses a beta distributio with parameters α ad β. Fid the Bayes estimaor for p The followig probabilities are kow Hece, π(p) pα 1 (1 p) β 1 B(α, β) f(x p) p x (1 p) 1 x x 0, 1 f(x 1,..., x, p) π(θ) f(x k p) k1 f(x 1,..., x ) px+α 1 (1 p) x+β 1 B(α, β) + 1 B(α, β) f(x 1,..., x, p)dp p x+α 1 (1 p) x+β 1 dp

21 Accordigly, we have B(x + α, x + β) B(α, β) f(p x 1,..., x ) f(x 1,..., x, p) f(x 1,..., x ) So, the Bayes estimator of p is px+α 1 (1 p) x+β 1 B(x + α, x + β) ˆp E[P x 1,..., x ] + x + α + α + β pf(p x 1,..., x )dp ˆp X + α 1 + α+β Example 20 Suppose i the estimatio of a proportio that the experimeter lacks ay credible prior iformatio regardig the state of ature. Hece, i Example 19, P is assumed to be uiform distributed betwee 0 ad 1, i.e., π(p) { 1 0 p 0 0 otherwise. Fid the Bayes estimator based o such a prior distributio. The desity goverig a sigle observatio is f(x p) p x (1 p) 1 x x 0 or 1, 0 p 1 Hece, f(x 1,..., x, p) π(θ) f(x k p) f(x 1,..., x ) k1 p x (1 p) x f(x 1,..., x, p)dp p x (1 p) x dp B(x + 1, x + 1) 21

22 f(p x 1,..., x ) f(x 1,..., x, p) f(x 1,..., x ) p x (1 p) x B(x + 1, x + 1) Accordigly, we have So, the Bayes estimator of p is ˆp E[P x 1,..., x ] + x x pf(p x 1,..., x )dp ˆp X

23 7.5 Exercises 1. Let X 1, X 2,..., X be a sample from a populatio with mea µ ad variace σ 2. Defie ˆµ i1 ix i i1 i (a) Show that ˆµ is a ubiased estimator of µ. (b) Fid V ar[ˆµ]. (c) Show that ˆµ is a cosistet estimator of µ. (d) Let ˆµ ˆµ. i1 X i be the other estimator of µ. Compare the efficiecy for ˆµ ad 2. Let X 1, X 2,..., X be a sample from a ormal populatio N(µ, σ 2 ). Suppose that σ 2 is kow. Show that X is a sufficiet estimator of µ. (Hit: (xi µ) 2 (xi X) 2 + (X µ) 2.) 3. Let X ad Y be idepedet radom variables with E[X] 1, E[Y ] 2, V ar[x] V ar[y ] σ 2, for what value of K is K(X 2 Y 2 ) + Y 2 is a ubiased estimator of σ Cosider a Poisso distributio with parameter λ. (a) Show that the sample mea X is a complete sufficiet statistic for λ. (b) Show that X is a MVUE of λ. 5. Suppose ˆµ 1 ad ˆµ 2 are sample meas correspodig to idepedet radom samples of sizes 1 ad 2, respectively, arisig from a ormally distributed radom variable, possessig mea µ ad variace σ 2. Show that, for ay costat a betwee 0 ad 1, ˆµ a ˆµ 1 + (1 a) ˆµ 2 is a ubiased estimator. Show that this estimator possesses miimum variace whe a 1 /( ). 6. Let X 1, X 2,..., X be a sample from a Poisso populatio P (λ). Show that ˆλ X is a sufficiet estimator of λ. 7. Let X 1, X 2,..., X be a sample from a Poisso populatio P (λ). (a) Fid the MLE ˆλ for λ. (b) Fid the expected value ad variace of ˆλ. (c) Show the cosistecy of ˆλ. 8. Cosider a radom sample from the shifted expoetial pdf. f(x; λ, θ) { λe λ(x θ) x θ 0 otherwise Takig θ 0 gives the pdf of the expoetial distributio. (a) Obtai the MLE of θ ad λ. 23

24 (b) If 10 observatios are made, resultig i the values 3.11, 0.64, 2.55, 2,20, 5.44, 3.42, 10.39, 8.93, 17.82, ad Calculate the estimators of θ ad λ. 9. Cosider a radom sample X 1, X 2,..., X from the shifted expoetial pdf Obtai the MLE s of θ ad λ. f(x; λ, θ) { λe λ(x θ) x θ 0 otherwise 10. Let X 1, X 2,..., X be a radom sample from the distributio Fid the MLE of θ. f(x; θ) θ x θ + 1, x > Let the lifetime Y (hours) of a compoet have the pdf (Rayleigh desity): where θ > 0. (a) Fid the MLE ˆθ of θ. f Y (y; θ) (b) Fid the distributio of ˆθ as. { 2 θ ye y2 /θ y > 0 0 otherwise 12. There are N + 1 attedaces i a coferece. The attedat cards for the attedaces are umbered from 0 to N + 1. To estimate the umber of attedace, 6 attedat cards are radomly chose, ad their umbers are recorded as follows: (a) Use MLE to estimate N. 15, 40, 45, 48, 25, 26 (b) Use the method of momets to estimate N. (c) Discuss the biasedess or ubiasedess of above estimators. 13. Let X 1,..., X be a radom sample from the populatio distributed with the desity fuctio f(x; θ) θx θ 1, 0 < x < 1, 0 < θ < Derive the poit estimators of parmeter θ by (a) the method of momets; ad (b) the method of maximum likelihood. 14. The maager of a appliace service ceter believes that the proportio P of dissatisfied customers possesses a beta distributio with α 2 ad β 10. (a) Fid the mea ad variace of prior distributio. 24

25 (b) What is the posterior desity correspodig to a radom sample of 10 customers? (c) What are the mea ad variace of the posterior distributio? 15. Suppose X possesses a expoetial distributio with radom parameter B that is uiformly distributed over the iterval [0, 1]. Fid the Bayes estimator of the parameter B based o a radom sample of size. 16. Suppose X possesses a expoetial distributio with radom parameter B that itself possesses a expoetial distributio with parameter c. Fid the Bayes estimator of the parameter B based o a radom sample of size. 17. If jobs arrivig for service at a statio satisfy the coditios of a Poisso experimet ad possess arrival rate q per hour, the the umber X of jobs arrivig i a hour possesses a Poisso distributio with parameter q. Now, suppose the arrival rate is ot costat, but the operatig egieer believes it to be a gamma-distributed radom variable Q possessig a prior desity with parameter α ad β. (a) show that the posterior desity relative to a sigle observatio (of jobs arrivig i a give hour) is gamma distributed. (b) Fid the Bayes estimator of the parameter Q based o a sigle observatio of the process. 25