Math 70 Homework 8 Michael Downs. and, given n iid observations, has log-likelihood function: k i n (2) = 1 n

Transcription

1 1. (a) The poisson distribution has density function f(k; λ) = λk e λ k! and, given n iid observations, has log-likelihood function: l(λ; k 1,..., k n ) = ln(λ) k i nλ ln(k i!) (1) and score equation: dl dλ = 1 λ k i n (2) which gives the maximum likelihood estimator ˆλ = 1 n n k i = k. The fisher information can be determined from taking the variance of the score equation for only one observation k: I(λ) = var( k var(k) 1) = = 1 (3) λ λ 2 λ so the inverse Fisher information is I(λ) 1 = λ. Thus the Wald test for the hypothesis H 0 : λ = λ 0 given n iid observations is: n(ˆλ λ0 ) N (0, 1) (4) λ0 For the Likelihood Ratio test, the restricted maximum likelihood value is: l(λ 0 ; k) = ln(λ 0 ) k i nλ 0 ln(k i!) (5) and the maximum likelihood value is: l(ˆλ; k) = ln(ˆλ) k i nˆλ ln(k i!) (6) ( ) n λ Their difference is ln 0 ˆλ k i + n(ˆλ λ 0 ) giving the likelihood ratio test for the hypothesis H 0 : λ = λ 0 : ( ( ) ) λ0 2 ln k i + n(ˆλ λ 0 ) χ ˆλ 2 (1) (7) (b) Code: 1 hw8 = f u n c t i o n ( job =1.2, lambda = 10, nexp = 1000, alpha =. 0 5 ) { 3 i f ( job ==1.2) { 5 n = c ( 2, 5, 1 0, 2 0 ) ln = l e n g t h ( n ) 7 alphaexp=matrix ( ncol =2,nrow=ln ) f o r ( i in 1 : ln ) 9 { 1

2 pvwald=pvlr=rep (NA, nexp) 11 f o r ( j in 1 : nexp) { 13 obs = r p o i s ( n [ i ], lambda = lambda ) mle = mean( obs ) 15 waldstat = s q r t ( n [ i ] ) ( mle lambda ) / s q r t ( lambda ) pvwald[ j ] = 2 (1 pnorm ( abs ( waldstat ) ) ) 17 LRstat = 2 ( l o g ( lambda/mle ) sum( obs ) + n [ i ] ( mle lambda ) ) 19 pvlr [ j ] = 1 pchisq ( LRstat, df =1) 21 alphaexp [ i,1]= mean(pvwald<alpha ) alphaexp [ i,2]= mean(pvlr<alpha, na. rm=t) 23 matplot (n, alphaexp, ylim=c ( 0,. 2 ), type= b, xlab= Sample s i z e, n, ylab = S i g n i f i c a n c e l e v e l ) 25 segments ( 1, alpha, 1 0 0, alpha, lwd=3) t i t l e ( Wald vs LR f o r o b t a i n i n g e m p i r i c a l s i g n i f i c a n c e l e v e l ) 27 legend ( 1 0,. 2, c ( 1=Wald Z t e s t, 2=LR chi square 1 t e s t ), l t y =1:2, c o l =1:2) r eturn ( cbind (n, alphaexp ) ) 29 > hw8(1.2, 10, ,.05) n [1,] [2,] [3,] [4,] Plot: Wald vs LR for obtaining empirical significance level Significance level Sample size, n 1=Wald Z test 2=LR chi square 1 te (c) Code: 2

3 obs = c ( 9, 1 4, 1 7, 1 2, 1 0 ) 2 mle = mean( obs ) n u l l = n = l e n g t h ( obs ) 6 waldstat = s q r t ( n ) ( mle n u l l ) / s q r t ( n u l l ) pwald = 2 (1 pnorm ( abs ( waldstat ) ) ) 8 LRstat = 2 ( l o g ( n u l l /mle ) sum( obs ) + n ( mle n u l l ) ) 10 p l r = 1 pchisq ( LRstat, df =1) 12 p r i n t ( p value f o r wald t e s t : ) p r i n t ( pwald ) 14 p r i n t ( p value f o r l r t e s t : ) p r i n t ( p l r ) 16 p r i n t ( in both c a s e s we cannot r e j e c t the n u l l ) > hw8(1.3) [1] "p-value for wald test: " [1] [1] "p-value for lr test: " [1] [1] "in both cases we cannot reject the null" 2. (a) Using the derivation for the Wald test from the lecture notes, n(ˆp p0 ) p0 (1 p 0 ) N (0, 1) where ˆp is the MLE m where m is the number of successes in n trials we can derive n (using a method similar to the one from the lecture notes) the two power functions under null H 0 : p = p 0 at the α significance level: ( ) ( ) n(p Power 1 (p) = Φ Z 1 α p0 ) n(p + 1 Φ Z 2 1 α 2 p0 (1 p 0 ) p0 ) (8) p0 (1 p 0 ) Power 2 (p) = Φ ( Z 1 α 2 n(p p0 ) p(1 p) ) + 1 Φ ( ) n(p Z 1 α p0 ) 2 p(1 p) (9) These gives the approximate probability of rejecting the null when the alternative hypothesis is correct. (b) Code: 3

4 hw8 = f u n c t i o n ( job =1.2, lambda = 10, nexp = 1000, alpha =. 0 5, p0 =. 2 5, n = 10) 2 { i f ( job ==2.2) 4 { z = qnorm (1 alpha / 2) 6 power1 = f u n c t i o n ( pr ) 8 { d = s q r t ( n ) ( pr p0 ) / s q r t ( p0 (1 p0 ) ) 10 r eturn ( pnorm( z d ) + 1 pnorm ( z d ) ) 12 power2 = f u n c t i o n ( pr ) 14 { d = s q r t ( n ) ( pr p0 ) / s q r t ( pr (1 pr ) ) r eturn ( pnorm( z d ) + 1 pnorm ( z d ) ) 20 x = seq ( from =.001, to =. 9 9, l ength = 100) y1 = power1 ( x ) 22 y2 = power2 ( x ) p l o t ( x, y1, type = l, c o l = 2, xlab = a l t e r n a t i v e p, ylab = p r o b a b i l i t y o f r e j e c t i n g n u l l ) 24 l i n e s ( x, y2, type = l, c o l = 3) 26 emp. pow1=rep ( 0, l e n g t h ( x ) ) emp. pow2=rep ( 0, l e n g t h ( x ) ) 28 counter = 1 f o r ( i in x ) 30 { obs = rbinom (nexp, n, i ) 32 obs = obs /n s = s q r t ( n ) ( obs p0 ) / s q r t ( p0 (1 p0 ) ) 34 emp. pow1 [ counter ] = mean( abs ( s )>z ) s2 = s q r t ( n ) ( obs p0 ) / s q r t ( i (1 i ) ) 36 emp. pow2 [ counter ] = mean( abs ( s2 )>z ) 38 counter = counter l i n e s ( x, emp. pow1, type = l, c o l = 4) l i n e s ( x, emp. pow2, type = l, c o l = 5) 42 segments ( 100, alpha, 1 0 0, alpha, l t y =2, c o l =2) segments ( p0, 1,p0, 1, c o l =3) 44 t i t l e ( Four power f u n c t i o n s ) 46 legend (. 6,. 5, c ( I at n u l l, I at a l t e r n a t i v e, Empirical n u l l, Empirical a l t ), l t y = 1, c o l =2:5) 48 p r i n t ( sum o f d i f f e r e n c e o f squares f o r e m p i r i c a l 1 and power1 : ) p r i n t (sum ( ( emp. pow1 y1 ) ˆ2) ) 50 p r i n t ( sum o f d i f f e r e n c e o f squares f o r e m p i r i c a l 2 and power2 : ) 4

5 52 p r i n t (sum ( ( emp. pow2 y2 ) ˆ2) ) p r i n t ( power f u n c t i o n s at p=.35 ) 54 p r i n t ( power1 (. 3 5 ) ) p r i n t ( power2 (. 3 5 ) ) 56 p r i n t ( e m p i r i c a l value 1 at p=.35 ) obs = rbinom (nexp, n,. 3 5 ) 58 obs = obs /n s = s q r t ( n ) ( obs p0 ) / s q r t ( p0 (1 p0 ) ) 60 emp. pow = mean( abs ( s )>z ) p r i n t (emp. pow) 62 p r i n t ( e m p i r i c a l value 2 at p=.35 ) 64 s = s q r t ( n ) ( obs p0 ) / s q r t (. 3 5 (1.35) ) emp. pow = mean( abs ( s )>z ) 66 p r i n t (emp. pow) > hw8(2.2, nexp = , n =10) [1] "sum of difference of squares for empirical1 and power1: " [1] [1] "sum of difference of squares for empirical2 and power2: " [1] [1] "power functions at p=.35" [1] [1] [1] "empirical value 1 at p=.35" [1] [1] "empirical value 2 at p=.35" [1]

6 Four power functions probability of rejecting null I at null I at alternative Empirical null Empirical alt alternative p The version where the Fisher information is evaluated at the alternative hypothesis appears to be better because it expresses the fact that p can only takes values from 0 to 1. (c) The derivation for the formula is similar to the derivation from the lecture notes. Code: p =. 4 2 p0 =. 2 5 d e l t a = p p0 4 z = qnorm ( / 2) z2 = qnorm (. 1 ) 6 n1 = p0 (1 p0 ) / d e l t a ˆ2 ( z z2 ) ˆ2 8 n2 = p (1 p ) / d e l t a ˆ2 ( z z2 ) ˆ2 10 p r i n t ( number o f mice r e q u i r e d using n u l l f i s h e r power f u n c t i o n ) p r i n t ( c e i l i n g ( n1 ) ) 12 p r i n t ( number o f mice r e q u i r e d using a l t f i s h e r power f u n c t i o n ) p r i n t ( c e i l i n g ( n2 ) ) [1] "number of mice required using null fisher power function" [1] 88 [1] "number of mice required using alt fisher power function" 6

7 [1] 113 7