Chapter 1 Axioms of Probability. Wen-Guey Tzeng Computer Science Department National Chiao University

Transcription

1 Chapter 1 Axioms of Probability Wen-Guey Tzeng Computer Science Department National Chiao University

2 What is probability? A branch of mathematics that deals with calculating the likelihood of a given event s occurrence, which is expressed as a number between 0 and Fall 2

3 History Luca Paccioli( ), Studies of chances of events Niccolo Tartaglia( ) Girolamo Cardano( ) Galileo Galielei( ) Blaise Pascal( ) French Pierre de Fermat( ) Christian Huygens( ) 1657, first book On Calculations in Games of Chance James Bernoulli( ) 2017 Fall 3

4 Abraham de Moivre( ) Pierre-Simon Laplace( ) Simeon Denis Poisson( ) Karl Friedrich Gauss( ) Pafnuty Chebyshev( ) Andrei Markov( ) Aleksandr Lyapunov( ) Emile Borel( ) Serge Bernstein( ) Richard von Mises( ) 2017 Fall 4

5 1900 David Hilbert ( ) pointed out the problem of the axiomatic treatment of the theory of probability 1933 Andrei Kolmogorov ( ), Russian successfully axiomatized the theory of probability 2017 Fall 5

6 Experiment with uncertain results Experiments Toss a coin Toss a die Pick a person from a group Lifetime of a TV set Arrival time of a customer to a store A stock goes up or down tomorrow Outcomes (or samples): results of an experiment 2017 Fall 6

7 Sample space and Event For an experiment Sample space = {all outcomes} An event is a subset of the sample space Note: a subset of a sample space is not necessarily an event 2017 Fall 7

8 Experiment: flip a coin once Outcomes: H, T Sample space S = {H, T} Events: {}, {H}, {T}, {H,T} Experiment: flip two coins Sample space? 2017 Fall 8

9 Experiment: toss a die( 骰子 ) Sample space: which one? S = {1, 2, 3, 4, 5, 6} or S = {even, odd} or S = {red, black} or S = { (1, red), (2, black), } You can use whatever appropriate to your concerns 2017 Fall 9

10 Experiment: flip a coin and if the outcome is T, toss a die, else flip a coin again S={T1, T2, T3, T4, T5, T6, HT, HH} Events:? 2017 Fall 10

11 Experiment: Roll two dice at the same time Sample space: Events: The event that the first die is greater than the second die = The event that the sum of two dice is greater than 10 = 2017 Fall 11

12 Experiment: Put three balls of green, red and blue into two boxes randomly Sample space: Events: The event that green and red balls are in different boxes. = The event that red and blue balls are in the same boxes = 2017 Fall 12

13 Countable and uncountable S Countable number of outcomes Experiment: toss a coin until the head appears Sample space S = {H, TH, TTH, TTTH, } Uncountable number of outcomes Experiment: choose a number between 0 and 1 Lifetime of a TV set Sample space S = {x: x is real, 0x1} 2017 Fall 13

14 Event operations E and F are events over sample space S. E 1, E 2, are events Fall 14 E S E F E F E F E EF F E c,,,, i i i i i n i i n i E E E E ,,,

15 Associative laws: EU(FUG)=(EUF)UG E(FG)=(EF)G Distributive laws: (EF)UH=(EUH)(FUH) (EUF)H=(EH)U(FH) De Morgan s laws (E U F) c = E c F c (EF) c = E c U F c E = ES = E(FUF c ) = EF U EF c 2017 Fall 15

16 Event occurrence Event E has occurred if the outcome of an experiment belongs to E. Experiment: roll a die S = {1, 2, 3, 4, 5, 6} E 1 = {1, 3, 5} E 2 = {5, 6} E 3 = {1, 2, 3} If we rolled a die and got outcome=5, then events E 1 and E 2 occurred, but E 3 did not occur Fall 16

17 Experiment: observe a TV set s lifetime in days Sample space S: Events E1: the event that the lifetime is greater than a week E2: the event that the lifetime is less than 3 days You bought a TV and it was broken after 2 week. Then, event E 1 occurred and E 2 did not occur Fall 17

18 Probability function P A function from events to reals between 0 and 1 For every event A, 0P(A) 1 P(A): the probability (odd) that event E occurs. Questions What is a legitimate P for a sample space S? Complied with our intuition about probability Derive useful probability laws and theorems No conflicts between event probabilities For an experiment, how to define/deduce an appropriate probability function? 2017 Fall 18

19 Types of sample spaces Countable Finite Infinite Uncountable 2017 Fall 19

20 Finite S Example: toss a die S={1, 2, 3, 4, 5, 6} For equally likely samples For an event A, let # of elemtns in A P A = # of elements in S Example: roll a fair die. What is the probability that the result is even? What is the probability that the result is greater than 2? Question: what if outcomes are not equally likely? (The die is a fake.) 2017 Fall 20

21 Infinite and countable S Example, Toss a coin till a head appears. Sample space S={H, TH, TTH, TTTH, } Let P( {T i H} ) = 1/2 i+1 and P( {o 1, o 2,, o m } ) = P({o 1 }) + P({o 2 }) + + P({o m }) What is the probability that the number of tosses is less than or equal to 3? What is the probability that the number of tosses is odd? It cannot be all outcomes occur equally likely Fall 21

22 Uncountable S Example: choose a real number between 0 and 2. Sample space S={x : x is real and 0x2} However, not every subset is an event. How to define P? P( {0.3} )? P( {x: 0.1x0.2} )? P( {0.3} {x: 0.1x0.2} )? 2017 Fall 22

23 Two ways to define P Frequentist probability Axiomatic probability 2017 Fall 23

24 Frequentist probability For an event A, do n times of independent experiments, # of occurrences of event A P A = lim n n Rationale Constructively define P(A) Capture our intuition about probability Problems: This limitation may not converge. Even if it converges, it is hard to compute the exact value. Some events occur only one, such as, tomorrow s weather. It is not possible to repeat experiments 2017 Fall 24

25 Axiomatic probability Find a minimum set of axioms for S and P to satisfy. These axioms are valid and consistent. From this set of axioms, we can derive all useful probability theorems and laws. Then, (S, P) is a legal probability model Fall 25

26 Probability axioms Definition. (Probability Axioms) S: a sample space A: an event of S P is a probability function for S if P satisfies Axiom 1: P(A) 0 Axiom 2: P(S) = 1 Axiom 3: If A 1, A 2, A 3, is an infinite sequence of mutually exclusive events, then P( --infinite additivity A ) i1 i i1 P( A i ) 2017 Fall 26

27 Rationale: It focuses on valid operations on probability values rather than on the assignment of values to P(A). If P satisfies the three axioms, all fundamental probability theorems and laws can be derived from the axioms Fall 27

28 Problems It does not tell you how to assign values to P(A). We need to find a way to assign probability values to some P(A). For example, For finite S and all samples in S are equally likely, then P(A) = #(A)/#(S). If some P(A) s are known, we use these P(A) s and the derived theorems to compute P(B) of other events B. The appropriateness of the assignment to P(A) for the experiment needs to be validated Fall 28

29 Derive Theorems by Axioms Theorem: P() = 0 Proof: 2017 Fall 29

30 Theorem: If A 1, A 2, A 3,, A n are mutually exclusive, P( n ) n i1 Ai i 1 P( A i ) 2017 Fall 30

31 Theorem: P(A c ) = 1 P(A) Proof: 2017 Fall 31

32 Theorem: If A B, P(B-A) = P(BA c ) = P(B)-P(A) Proof: Corollary: If A B, P(A) <= P(B) 2017 Fall 32

33 Theorem: P(AUB) = P(A) + P(B) - P(AB) Proof: 2017 Fall 33

34 Inclusion-Exclusion Principle )... ( 1) (... ) ( ) ( ) ( )... ( n n k j i j i i n A A A P A A A P A A P A P A A A P 2017 Fall 34

35 Deduce P by Axioms Experiment: toss a fair die - all outcomes are equally likely S= {1, 2, 3, 4, 5, 6} P(S) = P({1, 2, 3, 4, 5, 6}) = P({1})+P({2})+ +P({6}) Since the die is Fair: P({1}) = P({2}) = = P({6}) P({1}) = P({2}) = = P({6}) = 1/ Fall 35

36 Experiment: toss a bias coin The probability that the head appears is twice as much as the tail S = {H, T} Probability function P? 2017 Fall 36

37 Experiment: sexes of three children in a family Count the number of boys and girls Sample space S1 = {bbb, ggg, bbg, bgg} Probability function P? List from older to younger Sample space S2 = {bbb, bbg, bgg, bgb, ggg, ggb, gbb, gbg} Probability function P? 2017 Fall 37

38 Use Theorems to compute P(B) Problem: In a community of 400 adults, 300 bike or swim or do both, 160 swim, and 120 swim and bike. What is the probability that an adult, selected at random from this community, bikes? Sol: S = { (A 1, B, NS), (A 2, B, S),, (A 400, NB, S) } A: event that a person swims B: event that a person bikes Goal: compute P(B) P(AUB)=300/400, P(A)=160/400, P(B)=P(AUB)+P(AB)-P(A) P(AB)=120/400 = 300/ / /400=260/400= Fall 38

39 Problem: a number is chosen at random from the set {1, 2, 3,, 1000}. What is the probability that it is divisible by 3 or 5, that is, either 3 or 5 or both? Sol: S ={1, 2, 3,, 1000} A: event that the outcome is divisible by 3 B: event that the outcome is divisible by 5 Goal: Compute P(A U B) P(AUB) =P(A)+P(B)-P(AB) =333/ / /1000 =467/ Fall 39

40 Problem: in a community, 32% of the population are male smokers, 27% are female smokers. Randomly choose a person from the community. What is the probability that this person smokes? Sol: Sample space S = {all persons in the community} = { (M 1, S), (M 2, N), (M 3, N),, (F 1, N), (F 2, S), } Probability function P: all outcomes are equally likely Event A: the chosen person smokes Event B: the chosen person is male Goal: Compute P(A) P(A) = P(AB) + P(AB c ) = = 0.59 What is the probability that the person is male? P(B)=? 2017 Fall 40

41 Probabilities 0 and 1 When S is uncountable Not every subset of S is an event. For P(E) = 1, it does not mean E=S. For P(F) = 0, it does not mean F=. Thus, P(F)=0 does not mean event F will never happen. Event F can happen, but with probability 0. Example: selecting a random point from (0,1) A={1/3, 2/3}, P(A)=0 B=(0,1)-A, P(B)= Fall 41