Probability Theory. Alea iacta est!

Transcription

1 Probability Theory Alea iacta est!

2 "Scientific theories which involve the properties of large aggregates of individuals [...] are liable to misinterpretation as soon as the statistical nature of the argument is lost sight of. " Ronald Aylmer Fisher ( )

3 An event is said to occur at random if its outcome is variable and unpredictable.

4 The classical interpretation The probability of an event is the ratio of the favourable outcomes to the possible outcomes. Pierre-Simon Laplace ( ) Example: A die is thrown with the result that "the count is five". There are six possible outcomes with only one being favourable. Thus, the probability of "the count is five" is one sixth.

5 The frequency interpretation The probability of an event in a sequence of events is the limit of the relative frequency of that event. John Venn ( ) Example: Throw a die several times and record the counts. The relative frequency of "the count is five" is about one sixth. The limit of the relative frequency is exactly one sixth.

6 The subjective interpretation The probability is a measure of the degree of belief. Frank P. Ramsey ( ) Example: suppose you bet that the score would be five; you bet a dollar and, if you win, you will receive six dollars: this is a fair bet.

7 The axiomatic interpretation The probability is whatever fulfils the axioms of the theory of probability. Andrei N. Kolmogorov ( )

8

9 Kolmogorov's Probability Calculus Let Ω be a non-empty set and let be a family of subsets of Ω that has Ω as a member, and that is closed under complementation (with respect to Ω) and union. Let P be a function from to the real numbers that fulfils 1. Non-negativity P(A) 0 for all A 2. Normalization P(Ω) = 1 3. Additivity P(A 1 A 2...) = P(A 1 )+P(A 2 )+... for all A 1, A 2,... with A i A j = Ø P is called a 'probability' and (Ω,, P) is called a 'probability space'. Foundations of the Theory of Probability (1933)

10 Game of Dice P=6/6 P=4/6 P=2/6 {1,2,3,4,5,6} Ω {1,2,3,4} {1,2,3,5} {1,2,3,6} {1,2,4,5} {1,2,4,6} {1,2,5,6} {1,3,4,5} {1,3,4,6} {1,3,5,6} {1,4,5,6} {2,3,4,5} {2,3,4,6} {2,3,5,6} {2,4,5,6} {3,4,5,6} {1,2} {1,3} {1,4} {1,5} {1,6} {2,3} {2,4} {2,5} {2,6} {3,4} {3,5} {3,6} {4,5} {4,6} {5,6} P=5/6 P=3/6 P=1/6 {1,2,3,4,5} {1,2,3,4,6} {1,2,3,5,6} {1,2,4,5,6} {1,3,4,5,6} {2,3,4,6,5} {1,2,3} {4,5,6} {1,2,4} {3,5,6} {1,2,5} {3,4,6} {1,2,6} {3,4,5} {1,3,4} {2,5,6} {1,3,5} {2,4,6} {1,3,6} {2,4,5} {1,4,5} {2,3,6} {1,4,6} {2,3,5} {1,5,6} {2,3,4} {1} {2} {3} {4} {5} {6} P=0/6 Ø

11 Measure Theoretical View of Probability Ω Ω A B A A C A B: Either event A, or event B, or both occur. A B: Both event A and event B occur simultaneously. A C : The complement, or opposite, of event A occurs. P(A B) = P(A)+P(B)-P(A B) P(A)+P(A C ) = P(A A C ) = P(Ω) = 1 P(A C ) = 1-P(A)

12 Independence of Events Two events A and B are called 'independent' if P(A B) = P(A) P(B)

13 Blood Pressure and Blood Lipids Some 25% of adult US Americans have high blood pressure. (A) Some 20% of adult US Americans have high blood lipid levels. (B) Some 17% of adult US Americans are both hypertensive and hyperlipidemic. (A B) P(A B) = 0.17 > 0.05 = = P(A) P(B)

14 Random Variables... map complex random events from real life onto a simple (most often numerical) scale X: number of times the first five dice show 6 X = 2 a 'realisation' of X

15 Discrete Random Variables X: number of times the first five dice show 6 X = 2 a 'realisation' of X Probability Function of X f(a)=p(x=a) for all possible values a of X

16 Binomial Distribution Bin(n,π) Model: n independent replications of a random experiment with binary outcome ("success","failure") and a constant success probability π in each replication X: number of successes f(k) = P(X = k) = n k π k (1 π) n k n k = n k (n k) = n! k!(n k)! 'binomial coefficient'

17 Coin Tossing n=5, k=3 π (1-π) π (1-π) π=π 3 (1-π) 2 (1-π) π π (1-π) π=π 3 (1-π) In how many different ways can 3 positions be chosen from 5 positions? = 10 = = 5! 3! 2!

18 Effectiveness of Antibiotics An antibiotic is effective in 85% of patients with a particular disease. What is the probability that 8 or more out of 10 patients given the drug will be cured? n=10, π=0.85, k=8, 9 or 10 P(X 8) = f(8) + f(9) + f(10) = = = = 0.820

19 Probability Function 0.4 Bin(10,0.85) 0.3 f(k) k

20 Continuous Random Variables X = 22.5 Distribution Function of X F(b)=P(X b) for real numbers b

21 Distribution Function 1,0 0,8 F(b) 0,6 0,4 0,2 0, b 0 F(b) 1 F(b) is monotonously increasing

22 Continuous Random Variable y b f(x) 'density' x F(b) = b f(x)dx y P(a < X b) = F(b) F(a) a b x

23 Independence of Random Variables Two random variables X and Y are called 'stochastically independent' if P(X a,y b) = P(X a) P(Y b) for any choice of real numbers a and b.

24 Independence of Random Variables Examples Independent X: body height Y: time of consultation X: gender Y: hair colour Not Independent X: body mass index Y: age X: blood pressure Y: blood lipid level

25 Expected Value E(X) The expected value (or expectation or population mean) of a random variable indicates its average or central value. It is a useful summary value of the variable's distribution. discrete continuous E(X) = a P(X = a a) E(X) + = x f(x) dx

26 Game of Dice X: number of points in a single throw E(X) = 1 1 / / / / / / 6 = 3.5 Y: sum of points in two rolls E(Y) = 2 1 / / / / / / / / / / / 36 = 7

27 Law of Large Numbers X 1, X 2,, X n independent and identically distributed with E(X 1 ) =... = E(X n ) = µ X = X n + X n µ when n gets large

28 Game of Dice X i : count of a single throw (i=1,..,n) X : average count of n throws replicates X n=10 n=100 n=500

29 Variance Var(X) The (population) variance of a random variable is a nonnegative number which gives an idea of how widely spread the values of the random variable are likely to be; the larger the variance, the more scattered the observations are on average. Var(X) = E([X-E(X)] 2 ) Var(X) 'standard deviation'

30 Variance Var(X) E(X 1 ) E(X 2 ) Var(X 1 ) < Var(X 2 )

31 Game of Dice X: number of points in a single roll Var(X) = (1-3.5) 2 1 / 6 + (2-3.5) 2 1 / 6 + (3-3.5) 2 1 / 6 + (4-3.5) 2 1 / 6 + (5-3.5) 2 1 / 6 + (6-3.5) 2 1 / 6 = 2.9 Y: sum of points in two rolls Var(Y) = (2-7) 2 1 / 36 + (3-7) 2 2 / 36 + (4-7) 2 3 / 36 + (5-7) 2 4 / 36 + (6-7) 2 5 / 36 + (7-7) 2 6 / 36 +(8-7) 2 5 / 36 + (9-7) 2 4 / 36 + (10-7) 2 3 / 36 + (11-7) 2 2 / 36 + (12-7) 2 1 / 36 = 5.8

32 Some Computational Rules E(X+Y) = E(X) + E(Y) E(α X) = α E(X) Var(α X) = α 2 Var(X) if X and Y are independent E(X Y) = E(X) E(Y) Var(X+Y) = Var(X) + Var(Y)

33 Normal Distribution N(µ,σ 2 ) f(x) = 1 e (x µ) 2 2σ 2 σ 2π µ = E(X), σ 2 = Var(X)

34 Standard Normal Distribution N(0,1) z P(Z z)=φ(z)

35 Standardization of N(µ,σ 2 ) If X has a normal distribution with expected value µ and variance σ 2, then the random variable Z = X µ σ has a standard normal distribution. The distribution function F(b) of X fulfils F(b) = b Φ( µ σ )

36 Blood Pressure If the diastolic blood pressure of healthy individuals has a normal distribution with expected value µ=80 mmhg and standard deviation σ=10 mmhg, what is the probability that a randomly chosen individual has a blood pressure between 70 mmhg and 85 mmhg? P(70 X 85) = F(85) F(70) = = Φ( ) Φ( ) Φ( 0.5) Φ( 1) = =0.5328

37 Standard Normal Distribution N(0,1) P(-1.00 Z +1.00) = 0.68 P(µ-σ X µ+σ) = 0.68

38 Standard Normal Distribution N(0,1) P(Z 0.00) = 0.50= P(Z 0.00) P(X µ) = 0.50 = P(X µ)

39 Standard Normal Distribution N(0,1) P(-1.96 Z +1.96) = 0.95 P(µ-1.96σ X µ+1.96σ) = 0.95

40 Standard Normal Distribution N(0,1) P(Z 1.65) = 0.95 = P(Z -1.65) P(X µ+1.65σ) = 0.95 = P(X µ-1.65σ)

41 Normal Distribution N(µ,σ 2 ) N(0,1) N(1,1) N(0,4) N(0,0.25)

42 Central Limit Theorem X 1, X 2,, X n independent and identically distributed with expected value µ and variance σ 2 n σ X X n µ n Z with a standard normal Z, when n gets large

43 Galton Board A board which contains several rows of staggered but equally spaced nails, named after its inventor, Francis Galton ( )

44 Game of Dice 3.7 X : average count of n throws 3.6 X n=500 Frequency replicates 0 X

45 The (Almost) Universal Nature of Normality length oxygen consumption fecundity bill length weight

46 Summary -Probability theory, as a scientific discipline, is based upon Kolmogorov's axiomatic definition of probability. -Random variablesmap complex (real) events onto simple numerical scales; they can be discrete or continuous. -The distribution of a random variable is characterised by its probability function (discrete) or density (continuous). -Random variables are independent if the joint distribution equals the product of their individual distributions. -Important summary values of the distribution of random variables include the expected value and the variance. -The normal distribution is a universal, large-scale approximation of the "average" of other random variables.