LINKÖPINGS UNIVERSITET Institutionen för datavetenskap Statistik, ANd 73A36 THEORY OF STATISTICS, 6 CDTS Master s program in Statistics and Data Mining Fall semester Written exam Suggested solutions to written exam Jan 7, Task a) The mean of the Gamma distribution is x x 3 λ 4 3! e x/λ dx 4λ x x 4 λ 5 4! e x/λ dx 4λ since the latter integral integrates a new Gamma density from zero to infinity. The moment estimator of λ satisfies the equation EX) x which gives the moment estimate ˆλ MM x 4 b) Since the Gamma distribution satisfies normal regularity properties the Cramér-Rao inequality applies and hence the answer is yes. { } d lλ; x) n ) ) x 3 Iλ) E dλ lλ; x) ln i λ 4 e x i λ 3! Task Thus, and giving and the lower bound is 3 ln x i 4n ln λ n ln 3! λ dlλ; x) dλ d lλ; x) dλ Iλ) 4n λ + λ 3 4n λ + λ 4n λ λ 3 x i x i x i EX i ) 4n λ + 4n 3 n 4λ λ λ I λ) λ 4n λ 4 An estimate of this lower bound is found by replacing λ in I λ) by its moment estimate from a) giving I λ) x/4) 4 fx; θ) e x ln θ θ ln x! θx x! e θ Hence, it is the Poisson distribution.
a) Use the result that the MLE of θ has an asymptotic normal distributions with mean θ and variance I θ) lθ; x) x i ln θ nθ ln x i This can be easily derived from the first expression of the density above. Using the exponential family representation with the natural parameterization φ ln θ the MLE of φ is found by solving ) x i E X i n θ n e φ i giving ˆθ MLE x due to the invariance property of MLEs). Now i and that gives Thus, Iθ) dlθ; x) dθ d lθ; x) dθ n EX i) θ ˆθ MLE N n x i n θ n x i θ nθ θ θ, θ ) n n θ A 95% approximate confidence interval for θ can be found by solving for θ.96 < ˆθ MLE θ θ/n <.96 However, it is simpler and still good enough use the interval ˆθMLE 3 ˆθ MLE ±.96 3 ±.96 3 ±.5.5, 3.5) n 5 b) The likelihood can be written Lθ; x) 3 θ x i x i! e θ PrX < ) θ 4+3+3 e 3θ e θ + θe θ ) θ e 4θ + θ) and the log-likelihood lθ; x) Constant + ln θ 4θ + ln + θ)
Task 3 Hence, dlθ; x) dθ θ 4 + + θ The score equation dlθ; x) dθ gives upon simplification θ 7 4 θ 4 with the solution θ 7 + 9 8 since θ is known to be >. The second derivative which is negative for all θ> ). Hence, a) The likelihood function is d lθ; x) dθ θ + θ) ˆθ MLE 7 + 9 8 La; x) n x a i x i ) Ba, ) n x i) a n x i) Ba, )) n The best test satisfies Taking natural logarithms gives La ; x) La ; x) A la ; x) la ; x) ln A B a ) ln x i + ln x i ) n ln Ba, ) a ) ln x i ln x i )+n ln Ba, ) B a a ) Since a a < we get as the best test. ln x i B + nln Ba, ) ln Ba, )) ln x i C 3
b) No, it is not UMP since the inequality above changes direction if a > a c) The Central Limit Theorem can be used on ln X i provided we know Eln X i H ) and Varln X i H ). Now, Eln X i ) Ba, ) Ba, ) ln x) xa x) Ba, ) { [ ln x) xa a { [ x a a ] ] dx Ba, ) ] x a [ln a dx x xa+ + a + [ ] } x a+ + a + ) Ba, ) Ba, ) Γa) Γ) Γa + ) Under H we have a a 3 which is an integer. ln x) x a ln x) x a )dx } x a a + dx a + ) a ) B3, ) Γ3) Γ) Γ5)!! 4! and Further, Eln X i H ) 4 ) 3 7 78.4 Eln X i ) ) { [ ln x) Ba, ) xa a ln x) xa x) dx Ba, ) ] Ba, ) ln x) x a ln x) x a )dx ] ln x) [ln xa a dx x) xa+ + a + / a ln x) x a when x since xln x) / { ] [ ] [ln x) Ba, ) xa x a a + a dx + x a+ ln x) a + ) [ ] x a [ ] ) x a+ Ba, ) ) + a 3 + a + ) 3 ) Ba, ) a 3 a + ) 3 and this gives Eln X i ) H ) 4 3 3 ) 4 3 37 736 ln x } x a a + dx } x a a + )
and V arln X H ) 37 736 7 78 ).8 The Central limit theorwm now gives that 5 ln X i N5.4; 5.8) ifh istrue Pr 5 Hence, with α 5% we ll get ) ln X i C ) C 5.4 Φ 5.8 C 5.4 z.5 5.8 /z.5.6449/.9 Task 4 Repetaed coin tossing is a binomial experiment. Thus the likelihood function for the experimental data in this case is Lπ; x) ) x π x π) x a) The Minimax estimator coincides with the Bayes estimator when the risk function is constant. The conjugate prior to the binomial likelihood is the beta distribution with parameters α and β. The Bayes estimator with quatratic loss is the posterior mean, i.e. ˆπ B For this estimator the risk function is α + x α + β + n α + x α + β + Rπ; ˆπ) E X π [ ˆπ π) ] Varˆπ) + Biasˆπ)) Var ) [ ) α + x α + x + E π] α + β + α + β + /VarX) π π) and EX) π/ ) π π) α + π α + β + ) + α + β + π [α + π πα + β + )] + π π) α + β + ) [α πα + β)] + π π) α + β + ) For this risk function to be independent of π we require the coeffcients of π and π in the numerator to be zero. This gives α + β) and αα + β) which is satisfied by α β. The whole derivation for a general n is made in the textbook on page, unfortunately with a small error stating that the common value should be n/, but the correct value is n/). 5
Thus, with these values on α and β the Bayes estimator and also the minimax estimator becomes / + x ˆπ + b) This loss function is a zero-one loss function {, ˆπ π <.5 L S π, ˆπ), ˆπ π.5 With zero-one loss the Bayes estimator is the mode of the posterior distribution, which with a beta prior α,β) is ˆπ B α + x α + β + n /n / α + x α + β The prior to be used here is the one that was derived in a), i.e. a beta with α β which gives / + x / + ˆπ B /x /.85, Task 5 a) We would like to test H : The suspect is the writer of the signature against H : The owner of the signature is the writer of the signature The study of the samples of handwriting gives us approximative) likelihoods of the two hypotheses: LH ; x) PrCharacteristics H ) / LH ; x) PrCharacteristics H ) / Since the two hypothese are both simple, the Bayes factor is B / / 5 Now, since the prior odds for H are nine to one on Q 9) the posterior odds are or 45 to on. Q B Q 5 9 45 6
b) In this case we test H above against the composite hypothesis H : One of the owner and the third person is the writer of the signature The likelihood for the third person being the writer is analogously to the previous likelihoods) PrCharacteristics Third person) / Further, we also have that PrOwner H ) PrThird person H ) PrOwner H ) /3 and PrThird person H ) /3 The Bayes factor then becomes B PrCharacteristics H ) PrCharacteristics Owner) /3) + PrCharacteristics Third person) /3) and the posterior odds become / /) /3) + /) /3).5 Q.5 9.5 Task 6 a) Compute the difference in response time between engina A and engine B: 3 4 5 6 7 8 9 resp.time A 4. 5.4.7. 4.4 7.8.5.9 5. 7.7 resp.time B 5.6 6..5.5 4.3 8..4.9 6. 8.5 difference.5.6..5..4.9..8 Discard the pair with equal response times. AMong the remaining nine, seven differences are negative. Under the null hypothesis of generally equally long response times median difference ) the number of negative differences, X, would follow a Bi9,.5)-distribution. Hence, the P -value is PrX 7) PrX ) 9 ) + ) 9 + )) 9.5 9.9 b) Rank the absolute differences in a) discard the zero difference) and compute the rank sum of the absolute differences origin in negative differences. 3 4 5 6 7 8 9 resp.time A 4. 5.4.7. 4.4 7.8.5.9 5. 7.7 resp.time B 5.6 6..5.5 4.3 8..4.9 6. 8.5 difference.5.6..5..4.9..8 abs. difference.5.6..5..4.9..8 rank 9 5 4 3 7-8 6 7
The rank sum of the originally negative differences becomes W 9 + 5 + 4 + 3 + 7 + 8 + 6 4 Under the assumption the response times are generally equally long for the two engines ) nn + ) nn + )n + ) W N ; /n 9/ N.5; 7.5) 4 4 Hence, the P -value is approximately ) 4.5 PrW 4) Φ Φ.3). 7/4 8