Chapter 1 Axioms of Probability Wen-Guey Tzeng Computer Science Department National Chiao University
What is probability? A branch of mathematics that deals with calculating the likelihood of a given event s occurrence, which is expressed as a number between 0 and 1. 2017 Fall 2
History Luca Paccioli(1445-1514), Studies of chances of events Niccolo Tartaglia(1499-1557) Girolamo Cardano(1501-1576) Galileo Galielei(1564-1642) Blaise Pascal(1623-1662) French Pierre de Fermat(1601-1665) Christian Huygens(1629-1695) 1657, first book On Calculations in Games of Chance James Bernoulli(1654-1705) 2017 Fall 3
Abraham de Moivre(1667-1754) Pierre-Simon Laplace(1749-1827) Simeon Denis Poisson(1781-1840) Karl Friedrich Gauss(1777-1855) Pafnuty Chebyshev(1821-1894) Andrei Markov(1856-1922) Aleksandr Lyapunov(1857-1918) Emile Borel(1871-1956) Serge Bernstein(1880-1968) Richard von Mises(1883-1953) 2017 Fall 4
1900 David Hilbert (1862-1943) pointed out the problem of the axiomatic treatment of the theory of probability 1933 Andrei Kolmogorov (1903-1987), Russian successfully axiomatized the theory of probability 2017 Fall 5
Experiment with uncertain results Experiments Toss a coin Toss a die Pick a person from a group Lifetime of a TV set Arrival time of a customer to a store A stock goes up or down tomorrow Outcomes (or samples): results of an experiment 2017 Fall 6
Sample space and Event For an experiment Sample space = {all outcomes} An event is a subset of the sample space Note: a subset of a sample space is not necessarily an event 2017 Fall 7
Experiment: flip a coin once Outcomes: H, T Sample space S = {H, T} Events: {}, {H}, {T}, {H,T} Experiment: flip two coins Sample space? 2017 Fall 8
Experiment: toss a die( 骰子 ) Sample space: which one? S = {1, 2, 3, 4, 5, 6} or S = {even, odd} or S = {red, black} or S = { (1, red), (2, black), } You can use whatever appropriate to your concerns 2017 Fall 9
Experiment: flip a coin and if the outcome is T, toss a die, else flip a coin again S={T1, T2, T3, T4, T5, T6, HT, HH} Events:? 2017 Fall 10
Experiment: Roll two dice at the same time Sample space: Events: The event that the first die is greater than the second die = The event that the sum of two dice is greater than 10 = 2017 Fall 11
Experiment: Put three balls of green, red and blue into two boxes randomly Sample space: Events: The event that green and red balls are in different boxes. = The event that red and blue balls are in the same boxes = 2017 Fall 12
Countable and uncountable S Countable number of outcomes Experiment: toss a coin until the head appears Sample space S = {H, TH, TTH, TTTH, } Uncountable number of outcomes Experiment: choose a number between 0 and 1 Lifetime of a TV set Sample space S = {x: x is real, 0x1} 2017 Fall 13
Event operations E and F are events over sample space S. E 1, E 2, are events. 2017 Fall 14 E S E F E F E F E EF F E c,,,, i i i i i n i i n i E E E E 1 1 1 1,,,
Associative laws: EU(FUG)=(EUF)UG E(FG)=(EF)G Distributive laws: (EF)UH=(EUH)(FUH) (EUF)H=(EH)U(FH) De Morgan s laws (E U F) c = E c F c (EF) c = E c U F c E = ES = E(FUF c ) = EF U EF c 2017 Fall 15
Event occurrence Event E has occurred if the outcome of an experiment belongs to E. Experiment: roll a die S = {1, 2, 3, 4, 5, 6} E 1 = {1, 3, 5} E 2 = {5, 6} E 3 = {1, 2, 3} If we rolled a die and got outcome=5, then events E 1 and E 2 occurred, but E 3 did not occur. 2017 Fall 16
Experiment: observe a TV set s lifetime in days Sample space S: Events E1: the event that the lifetime is greater than a week E2: the event that the lifetime is less than 3 days You bought a TV and it was broken after 2 week. Then, event E 1 occurred and E 2 did not occur. 2017 Fall 17
Probability function P A function from events to reals between 0 and 1 For every event A, 0P(A) 1 P(A): the probability (odd) that event E occurs. Questions What is a legitimate P for a sample space S? Complied with our intuition about probability Derive useful probability laws and theorems No conflicts between event probabilities For an experiment, how to define/deduce an appropriate probability function? 2017 Fall 18
Types of sample spaces Countable Finite Infinite Uncountable 2017 Fall 19
Finite S Example: toss a die S={1, 2, 3, 4, 5, 6} For equally likely samples For an event A, let # of elemtns in A P A = # of elements in S Example: roll a fair die. What is the probability that the result is even? What is the probability that the result is greater than 2? Question: what if outcomes are not equally likely? (The die is a fake.) 2017 Fall 20
Infinite and countable S Example, Toss a coin till a head appears. Sample space S={H, TH, TTH, TTTH, } Let P( {T i H} ) = 1/2 i+1 and P( {o 1, o 2,, o m } ) = P({o 1 }) + P({o 2 }) + + P({o m }) What is the probability that the number of tosses is less than or equal to 3? What is the probability that the number of tosses is odd? It cannot be all outcomes occur equally likely. 2017 Fall 21
Uncountable S Example: choose a real number between 0 and 2. Sample space S={x : x is real and 0x2} However, not every subset is an event. How to define P? P( {0.3} )? P( {x: 0.1x0.2} )? P( {0.3} {x: 0.1x0.2} )? 2017 Fall 22
Two ways to define P Frequentist probability Axiomatic probability 2017 Fall 23
Frequentist probability For an event A, do n times of independent experiments, # of occurrences of event A P A = lim n n Rationale Constructively define P(A) Capture our intuition about probability Problems: This limitation may not converge. Even if it converges, it is hard to compute the exact value. Some events occur only one, such as, tomorrow s weather. It is not possible to repeat experiments 2017 Fall 24
Axiomatic probability Find a minimum set of axioms for S and P to satisfy. These axioms are valid and consistent. From this set of axioms, we can derive all useful probability theorems and laws. Then, (S, P) is a legal probability model. 2017 Fall 25
Probability axioms Definition. (Probability Axioms) S: a sample space A: an event of S P is a probability function for S if P satisfies Axiom 1: P(A) 0 Axiom 2: P(S) = 1 Axiom 3: If A 1, A 2, A 3, is an infinite sequence of mutually exclusive events, then P( --infinite additivity A ) i1 i i1 P( A i ) 2017 Fall 26
Rationale: It focuses on valid operations on probability values rather than on the assignment of values to P(A). If P satisfies the three axioms, all fundamental probability theorems and laws can be derived from the axioms. 2017 Fall 27
Problems It does not tell you how to assign values to P(A). We need to find a way to assign probability values to some P(A). For example, For finite S and all samples in S are equally likely, then P(A) = #(A)/#(S). If some P(A) s are known, we use these P(A) s and the derived theorems to compute P(B) of other events B. The appropriateness of the assignment to P(A) for the experiment needs to be validated. 2017 Fall 28
Derive Theorems by Axioms Theorem: P() = 0 Proof: 2017 Fall 29
Theorem: If A 1, A 2, A 3,, A n are mutually exclusive, P( n ) n i1 Ai i 1 P( A i ) 2017 Fall 30
Theorem: P(A c ) = 1 P(A) Proof: 2017 Fall 31
Theorem: If A B, P(B-A) = P(BA c ) = P(B)-P(A) Proof: Corollary: If A B, P(A) <= P(B) 2017 Fall 32
Theorem: P(AUB) = P(A) + P(B) - P(AB) Proof: 2017 Fall 33
Inclusion-Exclusion Principle )... ( 1) (... ) ( ) ( ) ( )... ( 2 1 1 2 1 n n k j i j i i n A A A P A A A P A A P A P A A A P 2017 Fall 34
Deduce P by Axioms Experiment: toss a fair die - all outcomes are equally likely S= {1, 2, 3, 4, 5, 6} P(S) = P({1, 2, 3, 4, 5, 6}) = P({1})+P({2})+ +P({6}) Since the die is Fair: P({1}) = P({2}) = = P({6}) P({1}) = P({2}) = = P({6}) = 1/6 2017 Fall 35
Experiment: toss a bias coin The probability that the head appears is twice as much as the tail S = {H, T} Probability function P? 2017 Fall 36
Experiment: sexes of three children in a family Count the number of boys and girls Sample space S1 = {bbb, ggg, bbg, bgg} Probability function P? List from older to younger Sample space S2 = {bbb, bbg, bgg, bgb, ggg, ggb, gbb, gbg} Probability function P? 2017 Fall 37
Use Theorems to compute P(B) Problem: In a community of 400 adults, 300 bike or swim or do both, 160 swim, and 120 swim and bike. What is the probability that an adult, selected at random from this community, bikes? Sol: S = { (A 1, B, NS), (A 2, B, S),, (A 400, NB, S) } A: event that a person swims B: event that a person bikes Goal: compute P(B) P(AUB)=300/400, P(A)=160/400, P(B)=P(AUB)+P(AB)-P(A) P(AB)=120/400 = 300/400+120/400-160/400=260/400= 0.65 2017 Fall 38
Problem: a number is chosen at random from the set {1, 2, 3,, 1000}. What is the probability that it is divisible by 3 or 5, that is, either 3 or 5 or both? Sol: S ={1, 2, 3,, 1000} A: event that the outcome is divisible by 3 B: event that the outcome is divisible by 5 Goal: Compute P(A U B) P(AUB) =P(A)+P(B)-P(AB) =333/1000+200/1000-66/1000 =467/1000 2017 Fall 39
Problem: in a community, 32% of the population are male smokers, 27% are female smokers. Randomly choose a person from the community. What is the probability that this person smokes? Sol: Sample space S = {all persons in the community} = { (M 1, S), (M 2, N), (M 3, N),, (F 1, N), (F 2, S), } Probability function P: all outcomes are equally likely Event A: the chosen person smokes Event B: the chosen person is male Goal: Compute P(A) P(A) = P(AB) + P(AB c ) = 0.32 + 0.27 = 0.59 What is the probability that the person is male? P(B)=? 2017 Fall 40
Probabilities 0 and 1 When S is uncountable Not every subset of S is an event. For P(E) = 1, it does not mean E=S. For P(F) = 0, it does not mean F=. Thus, P(F)=0 does not mean event F will never happen. Event F can happen, but with probability 0. Example: selecting a random point from (0,1) A={1/3, 2/3}, P(A)=0 B=(0,1)-A, P(B)=1 2017 Fall 41