Unobservable Parameter. Observed Random Sample. Calculate Posterior. Choosing Prior. Conjugate prior. population proportion, p prior:

Transcription

1 Pi Priors Unobservable Parameter population proportion, p prior: π ( p) Conjugate prior π ( p) ~ Beta( a, b) same PDF family exponential family only Posterior π ( p y) ~ Beta( a + y, b + n y) Observed Random Sample In n Bernoulli trials, let Y be the number of successes. Likelihood f ( y1, y, K, yn p) Calculate Posterior Prior*Likelihood Choosing Prior Uniform prior π ( p ) ~ Uniform (0,1) no information Posterior p n π ( p y) 1 (0,1) p y (1 p) n y y ~ Beta( y + 1, n y + 1) ab5 (a, b) are both shape parameter (9)

2 Priors Pi () Jeffreys Prior Invariance property for prior d ~ π ( ) ψ h ( ) ~ π ( h 1 ( ψ )) dψ Uniform prior is not invariant Invariant Prior for the parameter, π ( ) I( ) Fisher Information I() Score statistics d log L( y; ) U ( ) d E(U)0, V(U)Fisher Information d log L( y; ) log ( ; ) ( ) [( ) d L y I E ] E( ) d d In binomial proportion case (show that) Choosing Prior Before looking at data Conjugate prior is recommended even when vague; referring the shape of Beta PDF by matching the location and scale; solving the following two equations π ( p) Beta( a, b) a mean( p0) a + b ab p (1 ) ( ) 0 p std σ 0 0 ( a + b) ( a + b + 1) ( a + b + 1) Preparation Graph your guess for Beta(a, b) Calculate the equivalent sample size of the prior. variance of sample proportion prior variance π ( ) ~ Beta(1/,1/ ) y p (1 ) ( ˆ) ( ) 0 p V p V 0 n neq n eq a + b +11 ab ( a + b) ( a + b + 1) (30)

3 Example: priors Example want to know the proportion of residents who are in favor of a new building observed: n100, y6 (favor) priors Bart 3 different priors Bart: conjugate prior, Beta(a, b) mean0., std0.08 > a4.8, b19. equivalent sample size n5 Anna: no information, Uniform prior Beta(1, 1) equivalent sample size n3 Chris discrete weight, not beta prior sum of weights shouldn t be 1. Chris 0 p, 0 p 0. 1 π ( p) 0., 0.1 p p, 0.3 p 0.5 Anna (31)

4 Posteriors Posterior Anna ~ Beta(4.8+6, ) Bart ~ Beta(1+6, 1+74) Chris ~ Beta in the interval Bart Chris Anna (3)

5 Wrapping up Effect of the Prior more data, less effect of our priors (in the previous example, n100) The data is said to swamp the prior. Estimating the proportion MSE (mean squared error) of Estimator MSE ( pˆ) V ( pˆ) + B( pˆ) Summarizing the Posterior, Beta (α, β) not confidence ce interval, credible intervals; why? h? exact PDF measure of location posterior mode posterior median posterior mean measure of spread variance, std percentiles Bayesian Estimator? squared error loss function L(, ˆ) ( ˆ) absolute error loss function L(, ˆ) ˆ Bayesian Risk : Expectation of Loss function Bayes Risk (, ˆ) R (, ˆ) π ( ) d L(, ˆ) f ( y ) dyπ ( ) d Posterior mean for the squared error loss function Posterior median for the absolute error loss fn. (Theorem 10.3., G. Casella and R. Berger, Statistical inference, 1 st edition) π k I 1 k ( α, β ) k 100 π ( p y) dπ (33)

6 Wrapping up () Bayesian Credible Interval Using percentiles of Posterior not confidence intervals π ( p y) Exercise Exercise 1 do the same things with n87 and y6. do the same things with non-information prior *) how to do numerically calculation Credible Type Prior Posterior Mean Median STD IOR Lower Upper Anna No prior Beta(7, 75) Bart Beta(4.8, 19.) Beta(30.8, 93.) Chris Discrete Weight Numerical* (34)

7 Comparing Bayesian and Frequentist tit Using PDF? Bayesian: Posterior of p Frequentist: sampling dist. of the estimator Likelihood function for updating the prior for calculating the more likely value of parameter Frequentist q (pre-data analysis) Precision of Estimator: MSE MSE ( ˆ) E( ˆ E( ˆ)) B( ˆ) + V ( ˆ) Find MVUE (point estimator) RCRB gives the lower bound for the variance of estimator. Find MLE ( CSS) by Factorization Theorem or exponential family, more likely parameter value what if this actual data is obtained R-B Theorem: V(E(UE SS)) V(UE) Calculate CI with Sampling distribution of MVUE all possible random variable Comparing Estimators Frequentist Bayesian pˆ f y n MSE( pˆ f ) V ( pˆ f ) + B 0 conjugate prior; Beta(a, b) observed y out of n posterior; Beta (a+y, b+n-y) p(1 p) n y + a y a pˆ b + n + a + b n + a + b n + a + b MSE( pˆ b) V ( pˆ b) + B( pˆ b) 1 ( ) a ap bp np(1 p) + ( ) n + a + b n + a + b (35)

8 Comparing Bayesian and Frequentist tit () Example Frequentist: solid black line no information, Uniform Beta(1,1) prior: blue line conjugate prior: Beta (,3); dot black prior is rightly skewed, that is why MSE(Bayesian) is the smallest when the parameter is small (36)

9 Comparing Bayesian and Frequentist t (3) Interval estimation Frequentist: confidence interval pre-data concept using the sampling dist. of the best estimator ˆ 1 ± F 1 α / or large sample theory s ( ˆ) Hypothesis Testing statistical hypothesis: null, alternative the Best Rejection Region by Neyman-Lemma Lemma(LR): UMPT LR Test: function of SS, -ln(lr) ~ approximately Chi-Square power function: significant level, power ˆ E( ˆ) ~ N(0,1) s( ˆ) Frequentist e pˆ(1 pˆ) pˆ ± z α / n previous example: n100, y6, 95% confidence interval (0.174, 0.346) Bayesian: credible bound post-data using the posterior dist of, π( y) previous example: e: see the slide of page 34 One-to-one relationship between CI and Testing by chance or sig. moved. proof by contradiction. Null hypothesis remains valid? (1) Set up Hypotheses. () Calculate the test statistic. (4) Calculate the p-value. (5) Get the conclusion Bayesian one-sided: d using posterior, like Frequentist t two-sided: using Credible Interval ˆ (37)

10 Comparing Bayesian and Frequentist tit (4) Example in One-sided Hypothesis Testing Better Treatment effect? old p0.6, New treatment is effective? 8 patients are cured out of random 10 patients. (sig. level10%) Frequentist F t Hypothesis: null: p0.6, alternative: p>0.6 Test statistic: y# of successes ~ sampling distribution (n10, p0.6) p-valuep(y 8 y~b(10,0.6))0.167 can t reject Ho Bayesian uniform prior Posterior is Beta(9,3). Example in Two-sided Hypothesis Testing A A coin is fair? n15, y# of heads (10) Frequentist Hypothesis: null: p0.5, alternative: p 0.5 Test statistic: y# of successes ~ sampling distribution (n15, p0.5) p-valuep(y 10 y~b(15,0.5)) + P(y 5 y~b(15,0.5)) 0.30 can t reject Ho Bayesian Using Credible Interval. why? Pr(p 0 )0 Posterior using Uniform prior: Beta(11,6) using normal approximation (0.46, 0.868) p value p 0 π ( p 0 y) dp (38)