Confidence is epistemic probability

Transcription

1 Confidence is epistemic probability Tore Schweder Dept of Economics University of Oslo Oslo Workshop, 11. May 2015 Oslo Workshop, 11. May /

2 Overview 1662 Confidence is epistemic probability Confidence distributions Confidence distributions do not obey the laws of probability Neyman-Pearson for confidence distributions Example: Fieller confidence Example: Combining two independent posteriors for climate sensitivity Example: What to add to nothing? A mathematical theory of confidence? Confidence distribution - the Fisherian synthesis Oslo Workshop, 11. May /

3 1662 Probability from the Latin probabilitas, a measure of the authority worthy of approbation (Hacking 1975). Wahrscheinlichkeit: Gewissheit von Vorhersagen, Sannsynlig: what seems true, earlier from what the authorities claim. By authority: before 1662 By (mathematical) reason after 1662 Oslo Workshop, 11. May /

4 Epistemic and aleatory probability Liber de ludo aleae (Book on games of chance, Girolamo Cardano 1663) An aleatory probability is a property of frequency in nature or society Epistemic: according to knowledge. The 95% confidence interval for the Newtonian gravitational constant G based on the CODATA 2010 is (6.6723, ) in appropriate units (Milyukov and Fan, 2012). is an epistemic probability. P( G ) = 0.95 Oslo Workshop, 11. May /

5 Confidence distributions (CD) Definition (Dimension 1) C(θ; X ) is cdf of a CD when C(θ; x) is a cdf in θ for all x and C(θ 0 ; X ) U(0, 1) when X f θ0 confidence curve cc(θ) = 2C(θ) 1 : cc(θ 0, X ) U Definition (Dimension 1) cc(θ; X ) is a confidence curve of a CD when min(cc(θ)) = 0, cc(θ; x) : θ [0, 1] for all x and cc(θ 0 ; X ) U(0, 1) when X f θ0 the level curve at β, {θ : cc(θ) β} is a confidence region of level β the mutinormal CD has many different confidence curves: ellipsoidic K p ((µ ˆµ) t Σ 1 (µ ˆµ)), rectangular in eigen-directions,... Oslo Workshop, 11. May /

6 Confidence curves for the probability of baby being small. Smoking mothers full line, non-smoking red dashed. Oslo Workshop, 11. May /

7 Loss, risk and optimality of CDs The less dispersed a CD is the more informative it is. Measure dispersion of C(ψ, y) by convex functions Γ(ψ) 0, Γ(0) = 0: loss: lo(ψ, C, x) = Γ(ψ ψ)c(dψ, x) risk: R(ψ, C) = E[lo(ψ, C, X )] Theorem (Neyman-Pearson for confidence distributions) A confidence distribution C based on a sufficient statistic in which the likelihood ratio is everywhere increasing, is uniformly optimal: P(lo(ψ, C, X ) lo(psi, C, X )) = 1 for all C and all Γ Lemma (Otimality of conditional CD in exponential families) When l(θ) = θ t s + k(θ), θ t = (ψ, θ 2,, θ d ) the conditional CD based on S 1 given S 2,, S d is uniformly optimal. Oslo Workshop, 11. May /

8 Confidence distributions do not in general obey Kolmogorov s laws of probability Length problem, d = 1: X N(µ, 1), C(µ) = Φ(µ X ). For ψ = µ, the derived distribution F (ψ; X ) = Φ(ψ X ) Φ( ψ X ) is not a CD since e.g. F (1; X ) is not uniformly distributed when X N(1, 1). length problem d > 1: X N d (µ, I ). Then N d (X, I ) is the confidence distribution for µ and K d (ψ; ncp = X 2 ) is the derived distribution for ψ = µ 2, the non-central chi-sq. K d (ψ, X 2 ) < U(0, 1) stochastically. Distributions for derived parameters ψ = g(µ, σ) are obtained from the natural confidence distribution from a normal sample for (µ, σ) by integration. Only for linear parameters ψ = aµ + bσ are these confidence distributions (Pedersen 1978). Oslo Workshop, 11. May /

9 Example: The Fieller confidence distribution for a ratio â N(a, σ1 2), ˆb N(b, σ2 2 ). The profile deviance for the ratio ψ = a/b is D = (â ψˆb) 2, V (ψ) = σ1 2 + ψ 2 σ2 2 : V (ψ) cc(ψ) = K(D(ψ); df = 1) For â = 1.333, ˆb =.333, ˆψ = 4.003, σ 1 = σ 2 = 1 Oslo Workshop, 11. May /

10 Example: Combining two independent posterior distributions of the climate sensitivity Nicholas Lewis 2015 The climate sensitivity ψ is the increase in global surface temperature resulting from doubling the amount of carbon dioxide in the atmosphere. IPCC Fifth Assessment Report gives Bayesian posteriors, p 1 (ψ) based on paleo-data and p 2 (ψ) based on direct measurements. Independent! ψ is really a ratio a/b. If estimated from independent normals, the posterior (flat priors for a, b): p(ψ) = ˆbσ âσ2 2 ψ V (ψ) 3/2 φ( â ψˆb V (ψ) 1/2 ) Lewis found this to fit both the paleo posterior and the instrument posterior (4 parameters). Oslo Workshop, 11. May /

11 Figure 10-20b in IPCC-AR5, posteriors for climate sensitivity Oslo Workshop, 11. May /

12 Profile deviances and combined confidence curve The recovered profile deviances are D i (ψ) = (â i ψˆb i ) 2, i = paleo, instr. ˆV i (ψ) This is also the implied likelihood of Efron (1993), L = exp( 1 2 (Φ 1 (C(ψ))) 2 ). Combined deviance D comb (ψ) = D paleo + D instr min (D paleo + D instr ) resulting in the confidence curve cc(ψ) = K 1 (D comb (ψ)). Oslo Workshop, 11. May /

13 Example: More heart attacks when using an antidiabetic drug? Nissen and Wolski (2007) considered 48 independent trials with Y i,treated deaths out of m i,treated and Y i,control deaths out of m i,control, i = 1,, 48. Binomial model, p i,control p i,treated θ i = log( ), θ i + ψ = log( ) 1 p i,control 1 p i,treated Exponential family model with Optimal CD S 1 = Y i,treated, S i+1 = Y i,treated + Y i,control. C(ψ) = P ψ (S 1 > s 1,obs s 2,obs,, s 49,obs )+ 1 2 P ψ(s 1 = s 1,obs s 2,obs,, s 49,obs 8 of the 48 triasl had Y treated = Y control = 0. They drop out of the CD. You should add nothing to nothing (Sweeting et. al (2004))! Oslo Workshop, 11. May /

14 40 individual CDs for ψ and an optimally combined CD Oslo Workshop, 11. May /

15 A mathematical theory of confidence (fiducial probability)? Two attempts Pitman (1939,1957) suggested to restrict the sets over which a confidence distribution can be integrated for the result to be a confidence Hacking (1965) suggested two principles The Frequency Principle: support for a statement is the probability of the statement being true. The Principle of Irrelevance: irrelevant information should not alter the support for a proposition. Implying that only models that can be transformed to location form can yield a confidence distribution. Oslo Workshop, 11. May /

16 Conclusion Confidence distribution is the appropriate concept of epistemic probability for empirical science. It is the Fisherian compromise between Bayesianisme and frequentisme. Oslo Workshop, 11. May /

17 Conclusion Confidence distribution is the appropriate concept of epistemic probability for empirical science. It is the Fisherian compromise between Bayesianisme and frequentisme. Efron (1998): My actual guess is that the old Fisher will have a very good 21st century. The world of applied statistics seems to need an effective compromise between Bayesian and frequentist ideas, and right now there is no substitute in sight for the Fisherian synthesis. Oslo Workshop, 11. May /

18 Conclusion Confidence distribution is the appropriate concept of epistemic probability for empirical science. It is the Fisherian compromise between Bayesianisme and frequentisme. Efron (1998): My actual guess is that the old Fisher will have a very good 21st century. The world of applied statistics seems to need an effective compromise between Bayesian and frequentist ideas, and right now there is no substitute in sight for the Fisherian synthesis. But a mathematical theory for epistemic probability calculus for empirical science is needed. The math of epistemic probability has been neglected! (Fine 1977 when reviewing Shafer 1977). Oslo Workshop, 11. May /

19 References Hacking, I. M.(1975, 2006). The Emergence of Probability: A Philosophical Study of Early Ideas About Probability, Induction and Statistical Inference. Cambridge University Press, Cambridge. This is the third edition of the book, with an extended preface. Efron, B. (1998). R.A. Fisher in the 21st century [with discussion and a rejoinder]. Statistical Science, 13: Fine, T. L. (1977). Book review of Shafer: A mathematical theory of evidence. Bulletin of the American Statistical Society, 83: Hacking, I. M. (1965). Logic of Statistical Inference. Cambridge University Press, Cambridge. Lewis, N. (2015). Objectively combining instrumental period and paleoclimate sensitivity evidence. Unpublished Milyukov, V. and Fan, S.-H. (2012). The Newtonian gravitational constant: Modern status of measurement and the new CODATA value. Gravitation and Cosmology, 18: Nissen, S. E. and Wolski, K. (2007). Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. The New England Journal of Medicine, 356: Pedersen, J. G. (1978). Fiducial inference. International Statistical Review, 146: Pitman, E. J. G. (1939). The estimation of location and scale parameters of a continuous population of any given form. Biometrika, 30: Pitman, E. J. G. (1957). Statistics and science. Journal of the American Statistical Association, 52: Schweder, T. and Hjort, N. L. (2015). Confidence, Likelihood, Probability: Statistical inference with confidence distributions. Cambridge University Press. Forthcoming Sweeting, M. J., Sutton, A. J., and Lambert, P. C. (2004). What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Statistics in Medicine, 23: Oslo Workshop, 11. May 2015 /