PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

Transcription

1 Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability Classical definition of probability Statistical definition of probability Bayesian understanding of probability 10 2 Basic definitions for frequentist statistics and Bayesian inference Definition of mean, moments and marginal distribution Worked example: The three-urn problem Frequentist statistics versus Bayesian inference 28 3 Bayesian inference Propositions Selected examples Ockham s razor 43 4 Combinatorics Preliminaries Partitions, binomial and multinomial distributions Occupation number problems Geometric and hypergeometric distributions The negative binomial distribution 66 5 Random walks First return First lead Random walk with absorbing wall 80 6 Limit theorems Stirling s formula de Moivre Laplace theorem/local limit theorem Bernoulli s law of large numbers Poisson s law 87

2 Table of vi 7 Continuous distributions Continuous propositions Distribution function and probability density functions Application in statistical physics Definitions for continuous distributions Common probability distributions Order statistic Transformation of random variables Characteristic function Error propagation Helmert transformation The central limit theorem The theorem Stable distributions Proof of the central limit theorem Markov chain Monte Carlo (MCMC) The multivariate case Poisson processes and waiting times Stochastic processes Three ways to generate Poisson points Waiting time paradox Order statistic of Poisson processes Various examples 157 PART II ASSIGNING PROBABILITIES Prior probabilities by transformation invariance Bertrand s paradox revisited Prior for scale variables The prior for a location variable Hyperplane priors The invariant Riemann measure (Jeffreys prior) Testable information and maximum entropy Discrete case Properties of the Shannon entropy Maximum entropy for continuous distributions Quantified maximum entropy The entropic prior Derivation of the entropic prior Saddle-point approximation for the normalization Posterior probability density 204

3 Table of vii 12.5 Regularization and good data A technical trick Application to ill-posed inversion problems Global smoothness A primer on cubic splines Second derivative prior First derivative prior Fisher information prior 221 PART III PARAMETER ESTIMATION Bayesian parameter estimation The estimation problem Loss and risk function Confidence intervals Examples Frequentist parameter estimation Unbiased estimators The maximum likelihood estimator Examples Stopping criteria for experiments Is unbiasedness desirable at all? Least-squares fitting The Cramer Rao inequality Lower bound on the variance Examples Admissibility of the Cramer Rao limit 251 PART IV TESTING HYPOTHESES The Bayesian way Some illustrative examples Independent measurements with Gaussian noise The frequentist approach Introduction Neyman Pearson lemma Sampling distributions Mean and median of i.i.d. random variables Mean and variance of Gaussian samples z-statistic Student s t-statistic Fisher Snedecor F -statistic 302

4 Table of viii 19.6 Chi-squared in case of missing parameters Common hypothesis tests Comparison of Bayesian and frequentist hypothesis tests Prior knowledge is prior data Dependence on the stopping criterion 325 PART V REAL-WORLD APPLICATIONS Regression Linear regression Models with nonlinear parameter dependence Errors in all variables Consistent inference on inconsistent data Erroneously measured uncertainties Combining incompatible measurements Unrecognized signal contributions The nuclear fission cross-section 239 Pu (n, f ) Electron temperature in a tokamak edge plasma Signal background separation Change point problems The Bayesian change point problem Change points in a binary image Neural network modelling Thin film growth detected by Auger analysis Function estimation Deriving trends from observations Density estimation Integral equations Abel s integral equation The Laplace transform The Kramers Kronig relations Noisy kernels Deconvolution Model selection Inelastic electron scattering Signal background separation Spectral line broadening Adaptive choice of pivots Mass spectrometry 484

5 Table of ix 28 Bayesian experimental design Overview of the Bayesian approach Optimality criteria and utility functions Examples N-step-ahead designs Experimental design: Perspective 504 PART VI PROBABILISTIC NUMERICAL TECHNIQUES Numerical integration The deterministic approach Monte Carlo integration Beyond the Gaussian approximation Monte Carlo methods Simple sampling Variance reduction Markov chain Monte Carlo Expectation value of the sample mean Equilibration Variance of the sample mean Taming rugged PDFs by tempering Evidence integral and partition function Nested sampling Motivation The theory behind nested sampling Application to the classical ideal gas Statistical uncertainty Concluding remarks 594 Appendix A Mathematical compendium 595 Appendix B Selected proofs and derivations 611 Appendix C Symbols and notation 619 References 620 Index 631