18.05 Problem Set 6, Spring 2014 Solutions

Transcription

1 8.5 Problem Set 6, Spring 4 Solutions Problem. pts.) a) Throughout this problem we will let x be the data of 4 heads out of 5 tosses. We have 4/5 =.56. Computing the likelihoods: 5 5 px H )=.5) 5 px H )=.56) 4.44) 4 4 whih yields Bayes fator px H ).5) 5 = =.6458, px H ).56) 4.44) Atually, we omputed the log Bayes fator sine it is numerially more stable. Then we exponentiated to get the Bayes fator.) Sine we hose the probability 4/5 of H to exatly math the data it is not surprising that the probability of the data given H is muh greater than the probability given H. Said differently, the data will pull our prior towards one entered at 4/5. b) Here are the plots of the five priors. The vertial dashed red line is at =.5. The R ode is posted alongside these solutions Beta,) Beta,) Beta5,5 Beta5,5) Beta3,7) A priori I would want my prior entered at.5. This rules out Beta3,7). Beta5,5) seems too narrow. Beta,) doesn t really math my experiene with oins, but I might go with it and just let the data speak for itself. Both Beta,) and Beta5,5) seem plausible. Even if they re wrong they aren t so strong that they would ause us to ignore the evidene in the data. ) The prior probability of a bias in favor of heads is P >/). Looking at the plots of the prior pdf s in part b) we see that i)-iv) are symmetri about.5, therefore they predit the probability of heads is /. That is they are all unbiased. v) has most of it s probability below.5. So it is strongly biased against heads. Thus, the ranking in order of bias from least to greatest is v) followed by a four-way tie between i)-iv). d) All of the prior pdf s are beta distributions, so they have the form f) = a ) b.

2 8.5 Problem Set 6, Spring 4 Solutions For a fixed hypothesis the likelihood funtion given the data x) is 5 px ) = 4 ). 4 Thus the posterior pdf is f x) = 4+a ) +b beta4 + a, + b). So the five posterior distributions i)-v) are beta4, ), beta5, ), beta9, 6), beta64, 6), and beta7, 8). Here are the plots of the five posteriors Beta,) Beta,) Beta5,5 Beta5,5) Beta3,7) Eah prior is entered on a value of. The sharpness of the peak is a measure of the prior ommitment to this value. So prior iv) is strongly ommitted to =.5, but prior ii) is only weakly ommitted and i) is essentially unommitted. The effet of the data is to pull the enter of the prior towards the data mean of.56. That is, it averages the enter of the prior and the data mean. The stronger the prior belief the less the data pulls the enter towards.56. So prior iv) is only moved a little and prior i) is moved almost all the way to.56. Priors ii) and iii) are intermediate. Prior v) is entered at =.3. The data moves the enter a long way towards.56. But, sine it starts so muh farther from.56 than the other priors, the posterior is still entered the farthest from.56. e) For eah of the five posterior distributions, we ompute p.5 x) : P i) >.5 x) =- pbeta.5, 4,) =.97 P ii) >.5 x) =- pbeta.5, 5,) =.9664 P iii) >.5 x) =- pbeta.5, 9,6) =.9459 P iv) >.5 x) =- pbeta.5, 64,6) =.8 P v) >.5 x) =- pbeta.5, 7,8) =.963 This is onsistent with the plot in d), as the posterior omputed from the uniform prior has the most density past.5 while the posterior omputed from prior v) has the least.

3 8.5 Problem Set 6, Spring 4 Solutions f) Step. Sine the intervals are small we an use the relation probability density Δ. So P i) H x) =P i).49.5 x) f i).5 x). and P i) H x)p i.55.57) f i).56).. So the the posterior odds using prior i)) of H versus H are approximately P i) H x) f P i) H x) i).56) f i).5) = dbeta.56,4,) dbeta.5,4,) =.56)4.44).5) 4. 5) By similar reasoning, the posterior odds using prior iv)) of H versus H is approximately.437. Problem. pts.) Let A be the event that Alie is olleting tikets and B the event that Bob is olleting tikets. Denoting our data as D, we have the likelihoods e 5 P D A) =!!!4!! e 75 P D B) =.!!!4!! P A) Moreover, we are given prior odds, OA) = P B) =. Thus, our posterior odds are P D A) 48 OA D) = OA) = e 5 5 P D B) 5.48 Note that the Bayes fator is about 5. Problem 3. pts.) a) We have a flat prior pdf f) =. For a single data value x, our likelihood funtion is: { if <x fx ) = if x Thus our table is prior likelihood unnormalized posterior hyp. f) fx ) posterior f x) <x d d x d d Tot. T The normalizing onstant must make the total posterior probability, so d x = =. lnx) Note that sine x, we have = / lnx) >. 3

4 8.5 Problem Set 6, Spring 4 Solutions Here are plots for x =. =.6) and x =.5 =.443). 3.5 f x =.) 3 f x =.5) b) Notie that annot be less than any one of x,...,x n. So the likelihood funtion is given by { if <max { x,...,x n } fx,...,x n ) = if max {x n,...,x n }. Let x M =max{x,...,x n }. So our table is prior likelihood unnormalized posterior hyp. f) fdata ) posterior f data) <x M d d x M d n n d n Tot. T The normalizing onstant must make the total posterior probability, so d x M n = = n. x n M The posterior pdf depends only on n and x M, therefore the data.,.5) and.5,.5) have the same posteriors. Here are the plots of the posteriors for the given data. 8 data =.,.5) or.5,.5) data =.,...,.5)

5 8.5 Problem Set 6, Spring 4 Solutions ) We now have x M =.5 so from part b) the posterior density is { if <.5 f x,...,x 5 )= if.5, 5 where = = 4 5. Now, let x be amount by whih Jane is late for the sixth lass. The likelihood is { if <x fx ) = if x We have the posterior preditive probability fx x,...,x 5 )= fx )f x,...,x 5 ) d.5 x = d = = 4 75 if x<.5 4 d = x = 4 75x 5 ) if.5 x Thus the posterior preditive probability that x.5 is P x.5 x,...,x 5 )= fx x,...,x n ) dx = dx = 6 75 = d) The graphs or the formula in part a) show that f x) is dereasing for x, sothe MAP is when = x. e) Extra redit: 5 points i) From part a) we have the posterior pdf f x). The onditional expetation is E x) = f x) d = = x) = x lnx). x ii) After observing x, we know that x, and as a result the onditional expetation E[ x] x. So the onditional expetation estimator is always at least as big as the MAP estimator. However, the MAP estimator is preisely x, the amount by whih Jane is late on the first lass. In this ontext, the MAP is not reasonable as it suggests that on the first lass, Jane arrived as late as possible and that in the future, she will arrive less than x hours late. 5

6 8.5 Problem Set 6, Spring 4 Solutions MAP Cond. expet x Problem 4. pts.) a) Leaving the sale fators as letters our table is prior likelihood posterior hyp. f) N5, 9) fx ) N, 4) f x) e 5) /8 6 ) /8 d e 5) 6 ) exp 8 8 Tot. All we need is some algebrai manipulations of the exponent in the posterior: 5) 6 ) = ) ) = 36 ) 74/3) = + k 36/3 where k is a onstant. Thus the posterior ) 74/3) f x) exp 36/3 This has the form of a pdf for N 74 3, 36 3). QED b) We have μ prior =5,σprior =9, x =6,σ =4,n= 4 So we have a =, b =, a+ b = 9 9 μ post = 5/9+6 =5.9, σ post = =.9. /9 /9 6

7 8.5 Problem Set 6, Spring 4 Solutions.5.4 Prior N5,9) Posterior N5.9,.9) After observing x,...,x 4, we see that the posterior mean is lose to x and the posterior variane is muh smaller than the prior variane. The data has made us more ertain about the loation of. ) As more data is reeived n inreases, so b inreases, so the mean of the posterior is loser to the data mean and the variane of the posterior dereases. Sine the variane goes down, we gain more ertainty about the true value of. d) With no new data we are given the prior f) N, 5 ). For data x = sore on the IQ test we have the likelihood fx ) N, ). Using the update formulas we have μ prior =, σprior =5, σ =, n =. Soa =/5, b =/ and a 8 i) Randall, x = 8: μ post = + b =86.5 a + b a ii) Mary, x = 5: μ post = + b 5 = 34.6 a + b Regression towards the mean! e) Extra redit: 5 points. This is essentially the same manipulation as in part a). First suppose we have one data value x then prior likelihood posterior hyp. f) Nμ prior,σprior ) fx ) N, σ ) f x ) ) e μ prior) /σprior d e x ) /σ μ ) prior x ) exp σ σ Tot. All we need is some algebrai manipulations of the exponent in the posterior. Whenever prior 7

8 8.5 Problem Set 6, Spring 4 Solutions we get a term not involving we just absorb it into the onstant k, k et. μprior) σprior x ) σ = μ prior + μ prior σprior + x + x σ = a + b) ) aμ prior + bx ) + k Thus the posterior = f x ) exp aμ prior +bx ) aμprior+bx + k ) aμprior +bx This has the form of a pdf for N,. This proves the formulas ) when n =. Theformulaswhenn> are a simple onsequene of updating one data point at a time using the formulas when n =. Problem 5. pts.) Censored data. We note that we assume that, given a partiular die, the rolls are independent. Let x be the ensored value on one roll. The Bayes fator for x is { 3/4 px 4-sided) Bayes fator = px 6-sided) = 5/6 =9/ if x = /4 /6 =3/ if x = Starting from the prior odds of, we multiply by the appropriate Bayes fator and get the posterior odds after rolls -5 are 3 =.5, 7 =.35, 8 4 =.5, 43 8 =3.375, 79 = b) In part a) we saw the Bayes fator when x = is 3/. Sine this is more than it is evidene in favor of the 4-sided die. When x = the Bayes fator is 9/, whih is evidene in favor of the 6-sided die. We saw this in part a) beause after every value of the odds for the 4-sided die went up and after the value of the odds went down. 8

9 MIT OpenCourseWare Introdution to Probability and Statistis Spring 4 For information about iting these materials or our Terms of Use, visit: