Numerical Methods for Data Analysis

Transcription

1 Michael O. Distler Bosen (Saar), August 29 - September 3, 2010 Fundamentals Probability distributions Expectation values, error propagation Parameter estimation Regression analysis Maximum likelihood Linear Regression Advanced topics

2 Some statistics books, papers, etc. Volker Blobel und Erich Lohrmann: Statistische und numerische Methoden der Datenanalyse, Teubner Verlag (1998) Siegmund Brandt: Datenanalyse, BI Wissenschaftsverlag (1999) Philip R. Bevington: Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill (1969) Roger J. Barlow: Statistics, John Wiley & Sons (1993) Glen Cowan: Statistical Data Analysis, Oxford University Press (1998) Frederick James: Statistical Methods in Experimental Physics, 2nd Edition, World Scientific, 2006 Wes Metzger s lecture notes: Glen Cowan s lecture notes: Particle Physics Booklet:

3 Introduction Data analysis in nuclear and particle physics Observe events of a certain type Measure characteristics of each event Theories predict distributions of these properties up to free parameters Some tasks of data analysis: Estimate (measure) the parameters; Quantify the uncertainty of the parameter estimates; Test the extent to which the predictions of a theory are in agreement with the data.

4 Introduction Philosophy of Science Karl R. Popper (* 28. Juli 1902 in Vienna, Austria; 17. September 1994 in London, England) coined the term critical rationalism. At the heart of his philosophy of science lies the account of the logical asymmetry between verification and falsifiability. Logik der Forschung, Existence of a true value of measured quantities and derived values.

5 Theory of probability Probability theory, mathematics: Kolmogorov axioms Classical interpretation, frequentist probability: Pragmatical definition of probability: n p(e) = lim N N n(e) = number of events E N = number of trials (experiments) Experiments have to be repeatable (in principle). Disadvantage: Strictly speaking one cannot make statements on the probability of any true value. Only upper and lower limits are possible given a certain confidence level.

6 Theory of probability Probability theory, mathematics Classical interpretation, frequentist probability Bayesian statistics, subjective probability: Prior subjective assumptions enter into the calculation of probabilities of a hypotheses H. p(h) = degree of belief that H is true Metaphorically speaking: Probabilities are the ratio of the (maximum) wager and the anticipated prize in a bet.

7 Theory of probability Probability theory, mathematics Classical interpretation, frequentist probability Bayesian statistics, subjective probability: Suppose there is a town with green and yellow taxicabs. In a hit-and-run accident a man was hurt and a witness saw a green cab. In court the lawer of the taxi company impeaches the credibility of the witness, because of the lighting conditions. A test showed that under similar conditions 10% of the witnesses confuse the color of the cabs. Would you believe the witness?

8 Theory of probability Probability theory, mathematics Classical interpretation, frequentist probability Bayesian statistics, subjective probability: Suppose there is a town with green and yellow taxicabs. In a hit-and-run accident a man was hurt and a witness saw a green cab. In court the lawer of the taxi company impeaches the credibility of the witness, because of the lighting conditions. A test showed that under similar conditions 10% of the witnesses confuse the color of the cabs. Would you believe the witness? What if there were 20 times more yellow cabs than green cabs? Would you still believe the witness?

9 Theory of probability Probability theory, mathematics Classical interpretation, frequentist probability Bayesian statistics, subjective probability: Prior subjective assumptions enter into the calculation of probabilities of a hypotheses H. taxicabs witness sees... statement is yellow 180 yellow 20 green 20/29 = 69% wrong 10 green 9 green 9/29 = 31% true 1 yellow

10 Theory of probability Probability theory, mathematics Classical interpretation, frequentist probability Bayesian statistics, subjective probability: Prior subjective assumptions enter into the calculation of probabilities of a hypotheses H. Disadvantage: Prior hypotheses influence the probability. Advantages for rare and one-time events, like noisy signals or catastrophe modeling.

11 Theory of probability Probability theory, mathematics Classical interpretation, frequentist probability Bayesian statistics, subjective probability: Prior subjective assumptions enter into the calculation of probabilities of a hypotheses H. Disadvantage: Prior hypotheses influence the probability. Advantages for rare and one-time events, like noisy signals or catastrophe modeling. In this lecture we will focus on the classical statistics, e.g. error estimates have to be understood as confidence regions.

12 Combining probabilities Two kinds of events are given: A and B. The probability of A is p(a) (B: p(b)). Then the probability of A or B is: p(a or B) = p(a) + p(b) p(a and B) If A and B are mutually exclusive then p(a and B) = 0 Example: Drawing from a deck of German Skat cards. p(ace or spades) = = Special case: B = Ā (A will NOT occur). p(a and Ā) = p(a) + p(ā) = 1

13 Combining probabilities Joint probability of A and B occuring simultaniously: p(a and B) = p(a) p(b A), p(b A) is called condional probability. If A and B are independent one gets p(b A) = p(b), respectively p(a and B) = p(a) p(b)

14 Death in the mountains In a book on mountaineering achievements of Reinhold Messner one reads the following: If you consider that the probability of dying in a expedition to an eight-thousander is 3,4%, then Messner had a probability of 3,4% 29 = 99% to be killed during his 29 expeditions.

15 Death in the mountains In a book on mountaineering achievements of Reinhold Messner one reads the following: If you consider that the probability of dying in a expedition to an eight-thousander is 3,4%, then Messner had a probability of 3,4% 29 = 99% to be killed during his 29 expeditions. That may not be true. What if Messner sets off to a 30th expedition?

16 Death in the mountains In a book on mountaineering achievements of Reinhold Messner one reads the following: If you consider that the probability of dying in a expedition to an eight-thousander is 3,4%, then Messner had a probability of 3,4% 29 = 99% to be killed during his 29 expeditions. That may not be true. What if Messner sets off to a 30th expedition? The probability to survive an expedition is obviously 1 0,034 = 0,966. If one assumes that the various expeditions represent independent events, the probability of surviving all 29 expeditions is: P = 0, = 0,367.

17 Definitions probability mass function (pmf) probability density function (pdf) of a measured value (=random variable) f(n) n f(x) f (n) discrete f (x) continuous Normalization: f (n) 0 f (n) = 1 f (x) 0 f (x) dx = 1 Probability: n p(n 1 n n 2 ) = n 2 x n 1 f (n) p(x 1 x x 2 ) = x2 x 1 f (x)dx

18 Definitions Cumulative distribution function (CDF): F(x) = x f (x )dx, F( ) = 0, F( ) = 1 Example: Decay time t of a radioactive nucleus with mean life time τ: f (t) = 1 τ e t/τ F(t) = 1 e t/τ f(t)*12s F(t) t/s

19 Expectation values and moments Mean: A random variable X takes on the values X 1, X 2,..., X n with probability p(x i ), then the expected value of X ( mean ) is X = X = n X i p(x i ) i=1 The expected value of an arbitrary funktion h(x) for a continuous random variable is: E[h(x)] = The mean ist the expected value of x: E[x] = x = h(x) f (x)dx x f (x)dx

20 Expectation values and moments standard deviation = {mean (deviation from x) 2 } 1/2 σ 2 = (x x) 2 = = (x x) 2 f (x)dx (x 2 2x x + x 2 ) f (x)dx = x 2 2 x x + x 2 = x 2 x 2 σ 2 = Variance, σ = Standard deviation Discrete distributions: ( x 2 ( x) 2 ) σ 2 = 1 N N Attention: This is the definition of the variance! To get a bias free estimation of the variance, 1 1 N will be replaced by N 1.

21 Expectation values and moments Moments are the expected value of x n and of (x x ) n. They are called nth algebraic moment µ n and nth central moment µ n, respectivly. Skewness v(x) is a measure of the asymmetry of the probability distribution of a random variable x: v = µ 3 σ 3 = E[(x E[x])3 ] σ 3 Kurtosis is a measure of the peakedness of the probability distribution of a random variable x. γ 2 = µ 4 σ 4 3 = E[(x E[x])4 ] σ 4 3

22 Binomial distribution The binomial distribution is the discrete probability distribution of the number of successes r in a sequence of n independent yes/no experiments, each of which yields success with probability p (Bernoulli experiment). P(r) = ( n r ) p r (1 p) n r P(r) is normalized. Proof: Binomial theorem with q = 1 p. The mean of r is: n r = E[r] = rp(r)= np The varianz σ 2 is V [r] = E[(r r ) 2 ] = r=0 n (r r ) 2 P(r)= np(1 p) r=0

23 Poisson distribution The Poisson distribution ist given by: The mean is: The variance is: P(r) = µr e µ r! r = µ V [r] = σ 2 = np = µ µ = µ = µ = µ =

24 Law of large numbers The law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed. We perform n independent experiments (Bernoulli trials) where the result j occurs n j times. p j = E[h j ] = E[n j /n] The variance of a Binomial distribution is: V [h j ] = σ 2 (h j ) = σ 2 (n j /n) = 1 n 2 σ2 (n j ) = 1 n 2 np j(1 p j ) From the product p j (1 p j ) which is 1 4, we can deduce the law of large numbers: σ 2 (h j ) < 1/n

25 The central limit theorem The central limit theorem (CLT) states conditions under which the mean of a sufficiently large number of independent random variables, each with finite mean and variance, will be approximately normally distributed. Let x i be a sequence of n independent and identically distributed random variables each having finite values of expectation µ and variance σ 2 > 0. In the limit n the random variable w = n i=1 x i will be normally distributed with mean w = n x and variance V [w] = nσ 2.

26 Illustration: The central limit theorem N=1 0.4 Gauss 0.4 N= N= N= The sum of uniformly distributed random variables and the standard normal distribution.

27 Special probability densities Uniform distribution: This probability distribution is constant in between the limits x = a and x = b: f (x) = Mean and variance: { 1 b a a x < b 0 otherwise x = E[x] = a + b 2 V [x] = σ 2 = (b a)2 12

28 Gaussian distribution The most important probability distribution - also called normal distribution: f (x) = 1 e (x µ)2 2σ 2 2πσ The Gaussian distribution has two parameters, the mean µ and the variance σ 2. The probability distribution with mean µ = 0 and variance σ 2 = 1 is named standard normal distribution or short N(0, 1). The Gaussian distribution can be derived from the binomial distribution for large values of n and r and similarly from the Poisson distribution for large values of Werte von µ.

29 Gaussian distribution dx N(0, 1) = 0,6827 = (1 0,3173) dx N(0, 1) = 0,9545 = (1 0,0455) dx N(0, 1) = 0,9973 = (1 0,0027) FWHM: useful to estimate the standard deviation: FWHM = 2σ 2ln2 = 2,355σ

30 Gaussian distribution Left side: The binomial distribution for n = 10 and p = 0,6 in comparison to the Gaussian distribution for µ = np = 6 and σ = np(1 p) = 2,4. Right side: The Poisson distribution for µ = 6 and σ = 6 in comparison to the Gaussian distribution.