Lies, damned lies, and statistics in Astronomy

Transcription

1 Lies, damned lies, and statistics in Astronomy Bayes Theorem For proposition A and evidence B,! P(A), the prior probability, is the initial degree of belief in A. P(A B), the conditional probability of A given B. P(B A), the conditional probability of B given A P(B) the probability of B

2 First Example! A couple has two children, the (older of which / at least one of which) is a girl. If the probabilities of having a boy or girl are both 0.5, what is the probability that the couple has two girls? A : both children are girls B : older child is a girl C: at least one child is a girl! P(A) = 0.5 x 0.5 = 0.25 P(B A)=1 (older child must be girl if both girls) P(B) = 0.5 P(C) = =0.75 P(C A) = 1 P(A B) = P(A) x P(B A) / P(B) = 0.25 x 1 / 0.5 = 1/2 P(A C) = P(A) x P(C A) / P(C) = 0.25 x 1 / 0.75 = 1/3!

3 BB GG BG GB BB X GG BG X GB BB X GG BG GB All possibilities Oldest is Girl One is a Girl

4 A few Probability Rules Product Rule: P(A,B)=P(B A)P(A)=P(A B)P(B) Sum Rule: P(A)= Σ all B P(A,B)=Σ all B P(A B)P(B) Independence: P(A,B)=P(A)P(B) (not always true!)

5 Test for Tuberculosis The General population is screened for TB. Test comes back positive for 98% of people with TB Test comes back negative for 99% of people without TB The rate of TB in 1971 was 1.7 in You are tested and come back positive, should you immediately be quarantined? What are the chances you have TB, but came back negative?

6 DISEASE {T or F} TEST {T or F} P(TEST=T DISEASE=T)=0.98 P(TEST=F DISEASE=T)=0.02 P(TEST=T DISEASE=F)=0.01 P(TEST=F DISEASE=F)=0.99 P(DISEASE=T)=1.7x10-4 P (DISEASE = T TEST = T) = P (TEST = T) = X DISEASE P(TEST = T DISEASE = T)P(DISEASE = T) P(TEST = T) P (TEST = T DISEASE)P (DISEASE) Sum rule = P (TEST = T DISEASE = T)P (DISEASE = T) + P(TEST = T DISEASE = F)P(DISEASE = F)) P (DISEASE = T TEST = T )= =0.0164

7 Modelling data Common problem in science, social sciences, finance, engineering... fitting a model with several parameters to data Is the model a good fit, and if so, what are the range of values of the parameters, and their uncertainty.

8 For example SN distances and redshifts What are the values of the cosmological parameters such as q0 or ΩM, Ω Λ or κ and their uncertainties one can derive from the data sets?

9 Least squares for data which has a gaussian distribution, minimising the residual squared between the model and the data provides the best estimate of the model - this is the least squares method. Usually pretty good for most data Also possible to use a maximum likelihood where you minimise a different function

10 probability and uncertainty Uncertainty of a model is most generally described as the probability of different values of each parameter per unit of each parameter. e.g. P(q0)dq0 or P(ΩM, Ω Λ )dωm, dω Λ 1σ, 2σ, 3σ contain 68.3%, 95.4%, and 99.73% of the probability respectively in a normal distribution - but we often use the same terms for any distribution

11 Probabilities of Data given P (D )P ( ) P ( D) = = P (D) Models P Q D Q D P (D )P ( ) / P (D )P ( ) P (D )P ( ) We want the Probability of the parameters to a model, θ, given the data, D" comparing data to model provides Π P(D θ) P(θ) is any prior information we have about the parameters And we need to normalise by the sum over all parameters and data to make Probability sum to 1

12 Let s try something Astronomical I observe a patch of sky (known as an aperture) with a photomultiplier tube. I integrate for a minute, and I detect 2 photons I do it again and I detect 6 photons. What is my best estimate for the average arrival rate of photons into my detector per minute?

13 Poisson Distribution The detection rate for k discrete events with expectation rate of detection λ is given by k e P (k, )= k! P (,k =2,k = 6) = P (k =2, )P (k =6, ) P P (k =2, )P (k =6, ) So for each value of λ, calculate the probability of counting 2 and 6, and multiply their probability. Normalise by sum of all probabilities as a function of λ excel spreadsheet

14 Poisson and Normal statistics for a randomly sampled set of objects, the actual number is uncertain due to random variations as described by a poisson distribution.! N = p N For a normal distribution, mean = p N

15 Adding uncertainties Normal statistics add in quadrature.. e.g.! 12 = q More generally, if I have two quantities (which are independent), then! e.g. f(a, b) =a b 2 f = b 2 2 a + a 2 2 b a =6± 1,b=8± 1 (a b) = p = 10

16 Fitting data with Gaussian Errors Imagine I have a model f(θ) with parameters θ (θ1, θ2,..θn) and data D (x1,y1,σ1, x2,y2,σ2, xi,yi,σι,) which has gaussian distributed uncertainties Q P (D )P ( ) P (D )P ( ) D P ( D) = = P Q P (D) P (D )P ( ) P (D i )P ( ) = P (D )P ( ) / exp D apple 1 (D(yi p 2 2 exp ) f(, x i )) " X i (D(y i ) f(, x i )) i # / P (D )P ( ) =exp " 1 2 X i 2 i #

17 χ 2 Chi 2 is the square of the model-data/uncertainty 2 i = apple (D(yi ) f(, x i )) Chi 2 is often used in Astronomy to model normally distributed data using a frequentist approach. As one varies a single parameter, and calculates chi 2, the parameter confidence interval can be shown to be where Δχ 2 is defined as change of χ 2 compared to best fit. Multiple parameters the confidence interval can be calculated. 2 = i 2 i 2

18 Estimating Uncertainty If errors are normally distributed and model is a realistic description of data, then Values of delta chi^2 as a function of probability number of parameters fit % % % % % % = 2 2 min

19 Bayesian Approach Calculate ΣΔχ 2 for each set of parameters 2 convert to relative probability via P (D )P ( ) / exp i 2 i where P(θ) is any prior you might have on the parameters. Normalise the probability by integrating over the entire space of parameters and ensuring the integral = 1 by applying a common factor to all relative probabilities calculate confidence intervals by integrating over volumes of Probability > than the value necessary to make these integrals equal to the confidence you are interested in. " 1 X # P ( )

20 Marginalising oftentimes you are not interested in the joint probability of all of the parameters, rather just the probability of one of the parameters. Use the Sum Rule: P(A)= Σ all B P(A,B) and integrate (marginalise) over the variables you are not interested to find the probability just of the one you are interested in.

21 Fitting a Line Uniform Spacing of lines of slope between 0 and10! So in practice a flat prior on slope is probably not what you want. We ll leave that for another day.! Worked example in excel

22 Doing this for n-variables Brute force, n dimensional space, calculate P(n) on a grid, and integrate over the grid (100) 4 => for 100 grid points in 4 dimensions. Use minimiser to find most probable value in n- dimensions. Then expand grid around that point to cover relevant probability space Markov Chain Monte Carlo - the super-hero of Bayesian analysis -