Chapter 1 Axioms of Probability. Wen-Guey Tzeng Computer Science Department National Chiao University

Transcription

1 Chapter 1 Axioms of Probability Wen-Guey Tzeng Computer Science Department National Chiao University

2 Introduction Luca Paccioli( ), Studies of chances of events Niccolo Tartaglia( ) Girolamo Cardano( ) Galileo Galielei( ) Blaise Pascal( ) French Pierre de Fermat( ) Christian Huygens( ) 1657, first book On Calculations in Games of Chance James Bernoulli( ) 2015 Fall 2

3 Abraham de Moivre( ) Pierre-Simon Laplace( ) Simeon Denis Poisson( ) Karl Friedrich Gauss( ) Pafnuty Chebyshev( ) Andrei Markov( ) Aleksandr Lyapunov( ) Emile Borel( ) Serge Bernstein( ) Richard von Mises( ) 2015 Fall 3

4 1900 David Hilbert ( ) pointed out the problem of the axiomatic treatment of the theory of probability 1933 Andrei Kolmogorov ( ), Russian successfully axiomatized the theory of probability 2015 Fall 4

5 Sample space and events Experiments toss a coin/die pick a person from a group lifetime of a TV set arrival time of a customer to a store Outcomes (or samples, or sample points) Sample space = {all outcomes} An event is a subset of the sample space Note: a subset of a sample space is not necessarily an event 2015 Fall 5

6 Experiment: flip a coin once Outcomes: H, T Sample space S = {H, T} Events: {}, {H}, {T}, {H,T} Experiment: flip two coins Sample space? 2015 Fall 6

7 Experiment: toss a die( 骰子 ) Sample space: which one? S = {1, 2, 3, 4, 5, 6} or S = {even, odd} or S = {red, black} or S = { (1, red), (2, black), } You can use whatever appropriate to your concerns 2015 Fall 7

8 Experiment: flip a coin and if the outcome is T, toss a die, else flip a coin again S={T1, T2, T3, T4, T5, T6, HT, HH} Events:? 2015 Fall 8

9 Experiment: Roll two dice at the same time Sample space: Events: The event that the first die is larger than the second die = The event that the sum of two dice is greater than 10 = 2015 Fall 9

10 Countable and uncountable S Countable number of sample points Experiment: toss a coin until the head appears Sample space S = {H, TH, TTH, TTTH, } Uncountable number of sample points Experiment: choose a number between 0 and 1 Sample space S = {x: x is real, 0x1} 2015 Fall 10

11 Event operations E and F are events over sample space S. E 1, E 2, are events Fall 11 E S E F E F E F E EF F E c,,,, i i i i i n i i n i E E E E ,,,

12 Associative laws: EU(FUG)=(EUF)UG E(FG)=(EF)G Distributive laws: (EF)UH=(EUH)(FUH) (EUF)H=(EH)U(FH) De Morgan s laws (E U F) c = E c F c (EF) c = E c U F c E = ES = E(FUF c ) = EF U EF c 2015 Fall 12

13 Event occurrence Event E has occurred if the outcome of an experiment belongs to E. Experiment: roll a die S = {1, 2, 3, 4, 5, 6} E 1 = {1, 3, 5} E 2 = {5, 6} E 3 = {1, 2, 3} If we rolled a die and got outcome=5, then events E 1 and E 2 occurred, but E3 did not occur Fall 13

14 Experiment: observe a TV set s lifetime in days Sample space S: Events E1: the event that the lifetime is greater than a week E2: the event that the lifetime is less than 3 days You bought a TV and it was broken after 2 week. Then, event E 1 occurred and E 2 did not occur Fall 14

15 Probability function P A function from events to reals between 0 and 1 For every event E, 0P(A) 1 P(A): the probability (odd) that event E occurs. Questions: What is a legitimate P? Complied with our intuition about probability Derive useful probability laws and theorems No conflicts between event probabilities For an experiment, how to define/deduce an appropriate probability function? 2015 Fall 15

16 Finite S and equally likely samples S is finite and all outcomes occur equally likely. For an event A, let # of elemtns in A P A = # of elements in S Example: roll a die. What is the probability that the result is even? What is the probability that the result is greater than 2? Question: what if outcomes are not equally likely? 2015 Fall 16

17 Infinite and countable S It cannot be all outcomes occur equally likely. Example, Toss a coin till a head appears. Sample space S={H, TH, TTH, TTTH, } Let P( {HT i } ) = 1/2 i+1 and P( {o 1, o 2,, o m } ) = P({o 1 }) + P({o 2 }) + + P({o m }) What is the probability that the number of tosses is less than or equal to 3? What is the probability that the number of tosses is odd? 2015 Fall 17

18 Uncountable S Example: choose a real number between 0 and 2. Sample space S={x : 0x2} How to define P? P( {0.3} )? P( {x: 0.1x0.2} )? P( {0.3} {x: 0.1x0.2} )? Note: not every subset is an event Fall 18

19 Frequentist probability For an event A, do n times of independent experiments, # of occurrences of event A P A = lim n n Constructively define P(A) Capture our intuition about probability Problems: This limitation may not converge. Even if it converges, it is hard to compute it. Some events occur only one, such as, tomorrow s weather 2015 Fall 19

20 Axioms of probability Definition. (Probability Axioms) S: a sample space A: an event over S P: a function from A to a real number P(A) is the probability of event A if P satisfies Axiom 1: P(A) 0 Axiom 2: P(S) = 1 Axiom 3: If A 1, A 2, A 3, is an infinite sequence of mutually exclusive events, then P( A ) i1 i i1 P( A i ) 2015 Fall 20

21 It focuses on valid operations on probability values rather than on the initial assignment of values Problems: We need to find a way to assign initial probability values to some events. Whether P s initial assignment is appropriate for the experiment is another story Fall 21

22 Basic Theorems All fundamental probability theorems can be derived from Axioms. Theorem: P() = 0 Proof: 2015 Fall 22

23 Theorem: If A 1, A 2, A 3,, A n are mutually exclusive, P( n ) n i1 Ai i 1 P( A i ) 2015 Fall 23

24 Theorem: P(A c ) = 1 P(A) Proof: 2015 Fall 24

25 Theorem: If A B, P(B-A) = P(BA c ) = P(B)-P(A) Proof: Corollary: If A B, P(A) <= P(B) 2015 Fall 25

26 Theorem: P(AUB) = P(A) + P(B) - P(AB) Proof: 2015 Fall 26

27 Inclusion-Exclusion Principle )... ( 1) (... ) ( ) ( ) ( )... ( n n k j i j i i n A A A P A A A P A A P A P A A A P 2015 Fall 27

28 Deduce P by Axioms Experiment: toss a fair die S= {1, 2, 3, 4, 5, 6} P({1, 2, 3, 4, 5, 6}) = P({1})+P({2})+ +P({6}) = 1 Since the die is Fair: P({1}) = P({2}) = = P({6}) P({1}) = P({2}) = = P({6}) = 1/ Fall 28

29 Experiment: toss a bias coin The probability that the head appears is twice as much as the tail S = {H, T} Probability function P? 2015 Fall 29

30 Examples Experiment: sexes of three children in a family Count the number of boys and girls Sample space S1 = {bbb, ggg, bbg, bgg} Probability function P? List from older to younger Sample space S2 = {bbb, bbg, bgg, bgb, ggg, ggb, gbb, gbg} Probability function P? 2015 Fall 30

31 Problem: In a community of 400 adults, 300 bike or swim or do both, 160 swim, and 120 swim and bike. What is the probability that an adult, selected at random from this community, bikes? Sol: S = { (A 1, B, NS), (A 2, B, S),, (A 400, NB, S) } A: event that a person swims B: event that a person bikes Goal: compute P(B) P(AUB)=300/400, P(A)=160/400, P(AB)=120/400 P(B)=P(AUB)+P(AB)-P(A) = 300/ / /400=260/400= Fall 31

32 Problem: a number is chosen at random from the set {1, 2, 3,, 1000}. What is the probability that it is divisible by 3 or 5, that is, either 3 or 5 or both? Sol: S ={1, 2, 3,, 1000} A: event that the outcome is divisible by 3 B: event that the outcome is divisible by 5 Goal: Compute P(A U B) P(AUB) =P(A)+P(B)-P(AB) =333/ / /1000 =467/ Fall 32

33 Problem: in a community, 32% of the population are male smokers, 27% are female smokers. Randomly choose a person from the community. What is the probability that this person smokes? Sol: Sample space S = {all persons in the community} = { (M 1, S), (M 2, N), (M 3, N),, (F 1, N), (F 2, S), } Probability function P: all outcomes are equally likely Event A: the chosen person smokes Event B: the chosen person is male Goal: Compute P(A) P(A) = P(AB) + P(AB c ) = = 0.59 What is the probability that the person is male? P(B)=? 2015 Fall 33

34 Probabilities 0 and 1 When sample space is uncountable Not every subset is an event. If E and F are events with probabilities 1 and 0, it is not correct to say that E is the sample space S, or F is the empty set. Example: selecting a random point from (0,1) A={1/3, 2/3}, P(A)=0 B=(0,1)-A, P(B)= Fall 34