Chapter 1 Axioms of Probability Wen-Guey Tzeng Computer Science Department National Chiao University
Introduction Luca Paccioli(1445-1514), Studies of chances of events Niccolo Tartaglia(1499-1557) Girolamo Cardano(1501-1576) Galileo Galielei(1564-1642) Blaise Pascal(1623-1662) French Pierre de Fermat(1601-1665) Christian Huygens(1629-1695) 1657, first book On Calculations in Games of Chance James Bernoulli(1654-1705) 2015 Fall 2
Abraham de Moivre(1667-1754) Pierre-Simon Laplace(1749-1827) Simeon Denis Poisson(1781-1840) Karl Friedrich Gauss(1777-1855) Pafnuty Chebyshev(1821-1894) Andrei Markov(1856-1922) Aleksandr Lyapunov(1857-1918) Emile Borel(1871-1956) Serge Bernstein(1880-1968) Richard von Mises(1883-1953) 2015 Fall 3
1900 David Hilbert (1862-1943) pointed out the problem of the axiomatic treatment of the theory of probability 1933 Andrei Kolmogorov (1903-1987), Russian successfully axiomatized the theory of probability 2015 Fall 4
Sample space and events Experiments toss a coin/die pick a person from a group lifetime of a TV set arrival time of a customer to a store Outcomes (or samples, or sample points) Sample space = {all outcomes} An event is a subset of the sample space Note: a subset of a sample space is not necessarily an event 2015 Fall 5
Experiment: flip a coin once Outcomes: H, T Sample space S = {H, T} Events: {}, {H}, {T}, {H,T} Experiment: flip two coins Sample space? 2015 Fall 6
Experiment: toss a die( 骰子 ) Sample space: which one? S = {1, 2, 3, 4, 5, 6} or S = {even, odd} or S = {red, black} or S = { (1, red), (2, black), } You can use whatever appropriate to your concerns 2015 Fall 7
Experiment: flip a coin and if the outcome is T, toss a die, else flip a coin again S={T1, T2, T3, T4, T5, T6, HT, HH} Events:? 2015 Fall 8
Experiment: Roll two dice at the same time Sample space: Events: The event that the first die is larger than the second die = The event that the sum of two dice is greater than 10 = 2015 Fall 9
Countable and uncountable S Countable number of sample points Experiment: toss a coin until the head appears Sample space S = {H, TH, TTH, TTTH, } Uncountable number of sample points Experiment: choose a number between 0 and 1 Sample space S = {x: x is real, 0x1} 2015 Fall 10
Event operations E and F are events over sample space S. E 1, E 2, are events. 2015 Fall 11 E S E F E F E F E EF F E c,,,, i i i i i n i i n i E E E E 1 1 1 1,,,
Associative laws: EU(FUG)=(EUF)UG E(FG)=(EF)G Distributive laws: (EF)UH=(EUH)(FUH) (EUF)H=(EH)U(FH) De Morgan s laws (E U F) c = E c F c (EF) c = E c U F c E = ES = E(FUF c ) = EF U EF c 2015 Fall 12
Event occurrence Event E has occurred if the outcome of an experiment belongs to E. Experiment: roll a die S = {1, 2, 3, 4, 5, 6} E 1 = {1, 3, 5} E 2 = {5, 6} E 3 = {1, 2, 3} If we rolled a die and got outcome=5, then events E 1 and E 2 occurred, but E3 did not occur. 2015 Fall 13
Experiment: observe a TV set s lifetime in days Sample space S: Events E1: the event that the lifetime is greater than a week E2: the event that the lifetime is less than 3 days You bought a TV and it was broken after 2 week. Then, event E 1 occurred and E 2 did not occur. 2015 Fall 14
Probability function P A function from events to reals between 0 and 1 For every event E, 0P(A) 1 P(A): the probability (odd) that event E occurs. Questions: What is a legitimate P? Complied with our intuition about probability Derive useful probability laws and theorems No conflicts between event probabilities For an experiment, how to define/deduce an appropriate probability function? 2015 Fall 15
Finite S and equally likely samples S is finite and all outcomes occur equally likely. For an event A, let # of elemtns in A P A = # of elements in S Example: roll a die. What is the probability that the result is even? What is the probability that the result is greater than 2? Question: what if outcomes are not equally likely? 2015 Fall 16
Infinite and countable S It cannot be all outcomes occur equally likely. Example, Toss a coin till a head appears. Sample space S={H, TH, TTH, TTTH, } Let P( {HT i } ) = 1/2 i+1 and P( {o 1, o 2,, o m } ) = P({o 1 }) + P({o 2 }) + + P({o m }) What is the probability that the number of tosses is less than or equal to 3? What is the probability that the number of tosses is odd? 2015 Fall 17
Uncountable S Example: choose a real number between 0 and 2. Sample space S={x : 0x2} How to define P? P( {0.3} )? P( {x: 0.1x0.2} )? P( {0.3} {x: 0.1x0.2} )? Note: not every subset is an event. 2015 Fall 18
Frequentist probability For an event A, do n times of independent experiments, # of occurrences of event A P A = lim n n Constructively define P(A) Capture our intuition about probability Problems: This limitation may not converge. Even if it converges, it is hard to compute it. Some events occur only one, such as, tomorrow s weather 2015 Fall 19
Axioms of probability Definition. (Probability Axioms) S: a sample space A: an event over S P: a function from A to a real number P(A) is the probability of event A if P satisfies Axiom 1: P(A) 0 Axiom 2: P(S) = 1 Axiom 3: If A 1, A 2, A 3, is an infinite sequence of mutually exclusive events, then P( A ) i1 i i1 P( A i ) 2015 Fall 20
It focuses on valid operations on probability values rather than on the initial assignment of values Problems: We need to find a way to assign initial probability values to some events. Whether P s initial assignment is appropriate for the experiment is another story. 2015 Fall 21
Basic Theorems All fundamental probability theorems can be derived from Axioms. Theorem: P() = 0 Proof: 2015 Fall 22
Theorem: If A 1, A 2, A 3,, A n are mutually exclusive, P( n ) n i1 Ai i 1 P( A i ) 2015 Fall 23
Theorem: P(A c ) = 1 P(A) Proof: 2015 Fall 24
Theorem: If A B, P(B-A) = P(BA c ) = P(B)-P(A) Proof: Corollary: If A B, P(A) <= P(B) 2015 Fall 25
Theorem: P(AUB) = P(A) + P(B) - P(AB) Proof: 2015 Fall 26
Inclusion-Exclusion Principle )... ( 1) (... ) ( ) ( ) ( )... ( 2 1 1 2 1 n n k j i j i i n A A A P A A A P A A P A P A A A P 2015 Fall 27
Deduce P by Axioms Experiment: toss a fair die S= {1, 2, 3, 4, 5, 6} P({1, 2, 3, 4, 5, 6}) = P({1})+P({2})+ +P({6}) = 1 Since the die is Fair: P({1}) = P({2}) = = P({6}) P({1}) = P({2}) = = P({6}) = 1/6 2015 Fall 28
Experiment: toss a bias coin The probability that the head appears is twice as much as the tail S = {H, T} Probability function P? 2015 Fall 29
Examples Experiment: sexes of three children in a family Count the number of boys and girls Sample space S1 = {bbb, ggg, bbg, bgg} Probability function P? List from older to younger Sample space S2 = {bbb, bbg, bgg, bgb, ggg, ggb, gbb, gbg} Probability function P? 2015 Fall 30
Problem: In a community of 400 adults, 300 bike or swim or do both, 160 swim, and 120 swim and bike. What is the probability that an adult, selected at random from this community, bikes? Sol: S = { (A 1, B, NS), (A 2, B, S),, (A 400, NB, S) } A: event that a person swims B: event that a person bikes Goal: compute P(B) P(AUB)=300/400, P(A)=160/400, P(AB)=120/400 P(B)=P(AUB)+P(AB)-P(A) = 300/400+120/400-160/400=260/400= 0.65 2015 Fall 31
Problem: a number is chosen at random from the set {1, 2, 3,, 1000}. What is the probability that it is divisible by 3 or 5, that is, either 3 or 5 or both? Sol: S ={1, 2, 3,, 1000} A: event that the outcome is divisible by 3 B: event that the outcome is divisible by 5 Goal: Compute P(A U B) P(AUB) =P(A)+P(B)-P(AB) =333/1000+200/1000-66/1000 =467/1000 2015 Fall 32
Problem: in a community, 32% of the population are male smokers, 27% are female smokers. Randomly choose a person from the community. What is the probability that this person smokes? Sol: Sample space S = {all persons in the community} = { (M 1, S), (M 2, N), (M 3, N),, (F 1, N), (F 2, S), } Probability function P: all outcomes are equally likely Event A: the chosen person smokes Event B: the chosen person is male Goal: Compute P(A) P(A) = P(AB) + P(AB c ) = 0.32 + 0.27 = 0.59 What is the probability that the person is male? P(B)=? 2015 Fall 33
Probabilities 0 and 1 When sample space is uncountable Not every subset is an event. If E and F are events with probabilities 1 and 0, it is not correct to say that E is the sample space S, or F is the empty set. Example: selecting a random point from (0,1) A={1/3, 2/3}, P(A)=0 B=(0,1)-A, P(B)=1 2015 Fall 34