Irr. Statistical Methods in Experimental Physics. 2nd Edition. Frederick James. World Scientific. CERN, Switzerland

Transcription

1 Frederick James CERN, Switzerland Statistical Methods in Experimental Physics 2nd Edition r i Irr 1- r ri Ibn World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI

2 CONTENTS Preface to the Second Edition Preface to the First Edition v vii Chapter 1. Introduction Outline Language Two Philosophies Notation 4 Chapter 2. Basic Concepts in Probability Definitions of Probability Mathematical probability Frequentist probability Bayesian probability Properties of Probability Addition law for sets of elementary events Conditional probability and independence Example of the addition law: scanning efficiency Bayes theorem for discrete events Bayesian use of Bayes theorem Random variable Continuous Random Variables Probability density function Change of variable 20 xi

3 xii Statistical Methods in Experimental Physics - 2nd Edn Cumulative, marginal and conditional distributions Bayes theorem for continuous variables Bayesian use of Bayes theorem for continuous variables Properties of Distributions Expectation, mean and variance Covariance and correlation Linear functions of random variables Ratio of random variables Approximate variance formulae Moments Characteristic Function Definition and properties Cumulants Probability generating function Sums of a random number of random variables Invariant measures 41 Chapter 3. Convergence and the Law of Large Numbers The Tchebycheff Theorem and Its Corollary Tchebycheff theorem Bienaym6 Tchebycheff inequality Convergence Convergence in distribution The Paul Levy theorem Convergence in probability Stronger types of convergence The Law of Large Numbers Monte Carlo integration The Central Limit theorem Example: Gaussian (Normal) random number generator 51 Chapter 4. Probability Distributions Discrete Distributions Binomial distribution Multinomial distribution Poisson distribution Compound Poisson distribution Geometric distribution 62

4 Contents xiii Negative binomial distribution Continuous Distributions Normal one-dimensional (univariate Gaussian) Normal many-dimensional (multivariate Gaussian) Chi-square distribution Student's t-distribution Fisher Snedecor F and Z distributions Uniform distribution Triangular distribution Beta distribution Exponential distribution Gamma distribution Cauchy, or Breit Wigner, distribution Log-Normal distribution Extreme value distribution Weibull distribution Double exponential distribution Asymptotic relationships between distributions Handling of Real Life Distributions General applicability of the Normal distribution Johnson empirical distributions Truncation Experimental resolution Examples of variable experimental resolution 95 Chapter 5. Information Basic Concepts Likelihood function Statistic Information of R.A. Fisher Definition of information Properties of information Sufficient Statistics Sufficiency Examples Minimal sufficient statistics Darmois theorem Information and Sufficiency 108

5 xiv Statistical Methods in Experimental Physics - 2nd Edn Example of Experimental Design 109 Chapter 6. Decision Theory Basic Concepts in Decision Theory Subjective probability, Bayesian approach Definitions and terminology Choice of Decision Rules Classical choice: pre-ordering rules Bayesian choice Minimax decisions Decision-theoretic Approach to Classical Problems Point estimation Int erval estimation Tests of hypotheses Examples: Adjustment of an Apparatus Adjustment given an estimate of the apparatus performance Adjustment with estimation of the optimum adjustment Conclusion: Indeterminacy in Classical and Bayesian Decisions 124 Chapter 7. Theory of Estimators Basic Concepts in Estimation Consistency and convergence Bias and consistency Usual Methods of Constructing Consistent Estimators The moments method Implicitly defined estimators The maximum likelihood method Least squares methods Asymptotic Distributions of Estimates Asymptotic Normality Asymptotic expansion of moments of estimates Asymptotic bias and variance of the usual estimators Information and the Precision of an Estimator Lower bounds for the variance Cram6r Rao inequality Efficiency and minimum variance Cram6r Rao inequality for several parameters The Gauss Markov theorem Asymptotic efficiency 153

6 Contents xv 7.5. Bayesian Inference Choice of prior density Bayesian inference about the Poisson parameter Priors closed under sampling Bayesian inference about the mean, when the variance is known Bayesian inference about the variance, when the mean is known Bayesian inference about the mean and the variance Summary of Bayesian inference for Normal parameters 162 Chapter 8. Point Estimation in Practice Choice of Estimator Desirable properties of estimators Compromise between statistical merits Cures to obtain simplicity Economic considerations The Method of Moments Orthogonal functions Comparison of likelihood and moments methods The Maximum Likelihood Method Summary of properties of maximum likelihood Example: determination of the lifetime of a particle in a restricted volume Academic example of a poor maximum likelihood estimate Constrained parameters in maximum likelihood The Least Squares Method (Chi-Square) The linear model The polynomial model Constrained parameters in the linear model Normally distributed data in nonlinear models Estimation from histograms; comparison of likelihood and least squares methods Weights and Detection Efficiency Ideal method maximum likelihood Approximate method for handling weights Exclusion of events with large weight Least squares method 201

7 xvi Statistical Methods in Experimental Physics - 2nd Edn Reduction of Bias Exact distribution of the estimate known Exact distribution of the estimate unknown Robust (Distribution-free) Estimation Robust estimation of the centre of a distribution Trimming and Winsorization Generalized pth-power norms Estimates of location for asymmetric distributions 213 Chapter 9. Interval Estimation Normally distributed data Confidence intervals for the mean Confidence intervals for several parameters Interpretation of the covariance matrix The General Case in One Dimension Confidence intervals and belts Upper limits, lower limits and flip-flopping Unphysical values and empty intervals The unified approach Confidence intervals for discrete data Use of the Likelihood Function Parabolic log-likelihood function Non-parabolic log-likelihood functions Profile likelihood regions in many parameters Use of Asymptotic Approximations Asymptotic Normality of the maximum likelihood estimate Asymptotic Normality of Oln LIDO ,100 confidence regions in many parameters Finite sample behaviour of three general methods of interval estimation Summary: Confidence Intervals and the Ensemble The Bayesian Approach Confidence intervals and credible intervals Summary: Bayesian or frequentist intervals? 250 Chapter 10. Test of Hypotheses Formulation of a Test Basic concepts in testing 254

8 Contents xvii Example: Separation of two classes of events Comparison of Tests Power Consistency Bias Choice of tests Test of Simple Hypotheses The Neyman Pearson test Example: Normal theory test versus sign test Tests of Composite Hypotheses Existence of a uniformly most powerful test for the exponential family One- and two-sided tests Maximizing local power Likelihood Ratio Test Test statistic Asymptotic distribution for continuous families of hypotheses Asymptotic power for continuous families of hypotheses Examples Small sample behaviour Example of separate families of hypotheses General methods for testing separate families Tests and Decision Theory Bayesian choice between families of distributions Sequential tests for Optimum number of observations Sequential probability ratio test for a continuous family of hypotheses Summary of Optimal Tests 298 Chapter 11. Goodness-of-Fit Tests GOF Testing: From the Test Statistic to the P-value Pearson's Chi-square Test for Histograms Moments of the Pearson statistic Chi-square test with estimation of parameters Choosing optimal bin size Other Tests an Binned Data Runs test 308

9 xviii Statistical Methods in Experimental Physics - 2nd Edn Empty cell test, order statistics Neyman Barton smooth test Tests Free of Binning Smirnov Cramer von Mises test Kolmogorov test More refined tests based an the EDF Use of the likelihood function Applications Observation of a fine structure Combining independent estimates Comparing distributions Combining Independent Tests Independence of tests Significance level of the combined test 331 References 335 Subject Index 341