The Bayesian Choice. Christian P. Robert. From Decision-Theoretic Foundations to Computational Implementation. Second Edition.

Transcription

1 Christian P. Robert The Bayesian Choice From Decision-Theoretic Foundations to Computational Implementation Second Edition With 23 Illustrations ^Springer"

2 Contents Preface to the Second Edition Preface to the First Edition List of Tables List of Figures vii xi xxi xxiii 1 Introduction Statistical problems and statistical models The Bayesian paradigm as a duality principle Likelihood Principle and Sufficiency Principle Sufficiency The Likelihood Principle Derivation of the Likelihood Principle Implementation of the Likelihood Principle Maximum likelihood estimation Prior and posterior distributions Improper prior distributions The Bayesian choice Exercises Notes 45 2 Decision-Theoretic Foundations Evaluating estimators Existence of a utility function Utility and loss Two optimalities: minimaxity and admissibility Randomized estimators Minimaxity Existence of minimax rules and maximin strategy Admissibility Usual loss functions The quadratic loss The absolute error loss The 0-1 loss Intrinsic losses 81

3 Criticisms Exercises Notes and Contents alternatives From Prior Information to Prior Distributions The difficulty in selecting a prior distribution Subjective determination and approximations Existence Approximations to the prior distribution Maximum entropy priors Parametric approximations Other techniques Conjugate priors Introduction Justifications Exponential families Conjugate distributions for exponential families Criticisms and extensions Noninformative prior distributions Laplace's prior Invariant priors The Jeffreys prior Reference priors Matching priors Other approaches Posterior validation and robustness Exercises Notes 158 Bayesian Point Estimation Bayesian inference Introduction MAP estimator Likelihood Principle Restricted parameter space Precision of the Bayes estimators Prediction Back to Decision Theory Bayesian Decision Theory Bayes estimators Conjugate priors Loss estimation Sampling models Laplace succession rule The tramcar problem Capture-recapture models 182

4 Contents xvii 4.4 The particular case of the normal model Introduction Estimation of variance Linear models and G-priors Dynamic models Introduction The AR model The MA model The ARMA model Exercises Notes 216 Tests and Confidence Regions Introduction A first approach to testing theory Decision-theoretic testing The Bayes factor Modification of the prior Point-null hypotheses Improper priors Pseudo-Bayes factors Comparisons with the classical approach UMP and UMPU tests Least favorable prior distributions Criticisms The p-values Least favorable Bayesian answers The one-sided case A second decision-theoretic approach Confidence regions Credible intervals Classical confidence intervals Decision-theoretic evaluation of confidence sets Exercises Notes 279 Bayesian Calculations Implementation difficulties Classical approximation methods Numerical integration Monte Carlo methods Laplace analytic approximation Markov chain Monte Carlo methods MCMC in practice Metropolis-Hastings algorithms The Gibbs sampler 307

5 ii Contents Rao-Blackwellization The general Gibbs sampler The slice sampler The impact on Bayesian Statistics An application to mixture estimation Exercises Notes 334 Model Choice Introduction Choice between models Model choice: motives and uses Standard framework " Prior modeling for model choice Bayes factors Schwartz's criterion Bayesian deviance Monte Carlo and MCMC computations Importance sampling Bridge sampling MCMC methods Reversible jump MCMC, Model averaging Model projections Goodness-of-fit Exercises Notes 386 Admissibility and Complete Classes Introduction Admissibility of Bayes estimators General characterizations Boundary conditions Inadmissible generalized Bayes estimators Differential representations Recurrence conditions Necessary and sufficient admissibility conditions Continuous risks Blyth's sufficient condition Stein's necessary and sufficient condition Another limit theorem Complete classes Necessary admissibility conditions Exercises Notes 425

6 Contents xix 9 Invariance, Haar Measures, and Equivariant Estimators Invariance principles The particular case of location parameters Invariant decision problems Best equivariant noninformative distributions The Hunt-Stein theorem The role of invariance in Bayesian Statistics Exercises Notes Hierarchical and Empirical Bayes Extensions Incompletely Specified Priors Hierarchical Bayes analysis Hierarchical models Justifications Conditional decompositions Computational issues Hierarchical extensions for the normal model Optimality of hierarchical Bayes estimators The empirical Bayes alternative Nonparametric empirical Bayes Parametric empirical Bayes Empirical Bayes justifications of the Stein effect Point estimation Variance evaluation Confidence regions Comments Exercises Notes A Defense of the Bayesian Choice 507 A Probability Distributions 519 A.I Normal distribution, Af p (8, E) 519 A.2 Gamma distribution, Q(a, (3) 519 A.3 Beta distribution, Be(a, /3) 519 A.4 Student's t-distribution, T p {y, 9, E) 520 A.5 Fisher's F-distribution, T{v, g) 520 A.6 Inverse gamma distribution, IQ(a,/3) 520 A. 7 Noncentral chi-squared distribution, xtw 520 A.8 Dirichlet distribution, T>k{a\,, a/t) 521 A.9 Pareto distribution,-l?a(a,xo) 521 A.10 Binomial distribution, B(n,p). 521 A.ll Multinomial distribution, Mk(n;pi,..,Pk) 521 A. 12 Poisson distribution, V{\) 521 A.13 Negative Binomial distribution, Afeg(n,p) 522

7 xx Contents A. 14Hypergeometric distribution, "Hyp(N;n;p) 522 B Usual Pseudo-random Generators 523 B.I Normal distribution, W(0,1) 523 B.2 Exponential distribution, xp(x) 523 B.3 Student's t-distribution, T{v, 0,1) 524 B.4 Gamma distribution, Q(a, 1) 524 B.5 Binomial distribution, B(n,p) 525 B.6 Poisson distribution, V(X) 525 C Notations 527 C.I Mathematical 527 C.2 Probabilistic. 528 C.3 Distributional 528 C.4 Decisional 529 C.5 Statistical 529 C.6 Markov chains 530 D References 531 Author Index 581 Subject Index 589