THEORY OF PROBABILITY VLADIMIR KOBZAR

Transcription

1 THEORY OF PROBABILITY VLADIMIR KOBZAR Lecture 3 - Axioms of Probability I Sample Space and Events. In probability theory, an experiment is any situation which has several possible outcomes, exactly one of which then happens. This is a technical use of the term. It may include situations which we wouldn t call experiments in everyday speech. For example, Finding out sex of a newborn child; Order of finish in a 7-horse race; Flipping a pair of coins; and Measuring the lifetime of a transistor. Probability theory teaches us how to calculate the probabilities of different things that can happen in an experiment. Our mathematical description of the experiment is based on set theory. The sample space is the set of all possible outcomes. Denoted by S or, in other books, often Ω. Conversely an outcome is always an element of S. Sex of a newborn child: S tb, gu (can also include intersex variations); Order of finish in a 7-horse race: S tall orderings of p1, 2,..., 7qu; Flipping a pair of coins: S thh, HT, T H, T T u; and Measuring the lifetime of a transistor: S treal numbers ě 0u. The sample space is not the same thing as the experiment. An experiment is something that happens in the real world. Then a sample space is a choice that we make about how to model that experiment. There can be more than one valid choice. The important thing is to make a good choice and then stick to it. Date: July 9,

2 2 VLADIMIR KOBZAR Example: Suppose that in a horse race, we care only about who wins, not the order of the other horses. Then we could use as before, or we could use S tall orderings of p1, 2,..., 7qu S t1, 2,..., 7u (possible numbers of the horse that wins) The second is simpler. It is adequate to describe the outcomes only if we don t care who comes in second, third, etc. Having chosen the sample space, we will need to discuss different events that can occur. To do so formally and carefully, we observe that specifying an event is equivalent to specifying the outcomes for which that event occurs. Thus: An event is a subset of the sample space S. Consequence: two events are the same if they consist of the same outcomes, even if they are described in two different ways. Examples: Sex of a newborn child: E tgu tchild is a girlu. 7-Horse race: E tall orderings of p1, 2,..., 7q starting with 3u thorse 3 winsu. Flipping a pair of coins: E thh, HT u tfirst coin lands headsu. Transistor lifetime: E tx : 0 ď x ď 5u ttransistor does not last longer than 5 hoursu. Non-example: if flipping a pair of coins, then thu, th, T u, tfirst coinu are not events of the sample space. They all have something to do with the experiment, but the information is of the wrong kind to describe an event. In the first two examples, we see only possible results from a single coin, whereas the outcomes of the experiment depend on a pair of coins. In the third, it seems we are choosing one of the coins itself, which isn t the same as flipping the coins. Logic and operations with events: Thinking about events as sets clarifies various relations between then. The union of events E YF is the event in which either E occurs, F occurs, or both. As a set, it is the set of all outcomes which are in either E or F or both.

3 THEORY OF PROBABILITY 3 The intersection of events E X F or just EF is the event in which both E and F occur. As a set, it is the set of all outcomes which are in both E and F. If E and F are events, then F Ă E, or E contains F, if every outcome in E is also in F. This means that if E occurs, then F necessarily also occurs. Example: Flipping two coins. Let Then E thh, HT u tfirst coins lands headsu F thh, T Hu tsecond coins lands headsu. E Y F thh, HT, T Hu tat least one of the coins lands headsu E X F thhu tboth coins land headsu. The null event, denoted H, is the event that does not contain any outcomes. Also called the empty set. Roughly, the null event is to sample spaces as 0 is to counting. Two events E and F are mutually exclusive if E X F H, that is if there are no outcomes in both. More informally: if E and F cannot both occur. Also called disjoint. Several events E 1, E 2,... are mutually exclusive if every pair of them is mutually exclusive: that is, if at most one of them can occur at a time. If E is an event, then its complement E c is the event E does not occur : that is, it consists of exactly those outcomes which are not in E. Theorem 1. De Morgan s Laws If E 1, E 2 is a finite or infinite sequence of events, then and 1. pe 1 Y E 2 Y...q c E c 1 X E c 2 X pe 1 X E 2 X...q c E c 1 Y E c 2 Y... Proof. (remember what it means for two events to be the same).

4 4 VLADIMIR KOBZAR 1. pñq For all i, pðq For all i, x P pye i q c ñx R E i ñx P E c i ñx P XE c i x P XE c i ñx P E c i ñx R E i ñx R YE i ñx P pye i q c 2. By (1) and then taking complements of both sides pye c i q c XpE c i q c XE i ñ YE c i pxe i q c The axioms of probability. Now consider an experiment, and let S be its sample space. In probability theory, we assume that each event E Ă S has a probability value P(E), also referred to as probability measure or just probability. These values are supposed to put a number on how likely that event is. Although the idea of likelihood is very intuitive, it is very hard to say exactly what it means. The modern approach to probability theory is to avoid this question. Instead, we just agree on some basic axioms that these numbers should follow, and derive the theory from those. The interpretation can be decided later, and may vary with the context. It doesn t matter for building the theory as long as we agree on the axioms. For example, we are not assuming that the probability is the relative frequency of the event E occurring, i.e., we are NOT assuming that the probability is given by P peq lim nñ8 npeq n where npeq is the number of times in the first n repetitions of an experiment, the event E occurs. Instead we will prove from the axioms that in some sense a limiting behavior exists, e.g. the Law of Large

5 Numbers and Central Limit Theorem. Axioms of Probability (1) 0 ď P peq ď 1 (2) P psq 1 THEORY OF PROBABILITY 5 (3) (Countable additivity) For any sequence of mutually exclusive events E i (that is events for which E i X E j H when i j), 8ď 8ÿ P p E i q P pe i q Axiom 3 is needed to ensure the convergence of the series on the right-hand side when S has an infinite number of points. The modern approach is to assume the existence of a real-valued set function (measure) defined on the events of S and satisfying the axioms. In a graduate course, we would get very precise on which subsets of S are allowed to be events (so called measurable sets). In this course, we won t worry about this point. Fact 1 P phq 0 (Ross, p 27) Proof Let E 1 S and E i H for i ě 2. Since E 1 Ş H H, by Axiom 3, 8ď 8ÿ 8ÿ P psq P p E i q P pe i q P psq ` P phq i 0 ñ P phq 0 i 0 i 2 Fact 2 (Monotonicity) If E Ă F, then P peq ď P pf q. Proof E Ă F ñ F E Y E c X F E Y F ze

6 6 VLADIMIR KOBZAR Since E X F ze H, by Axiom 3, P pf q P peq ` pf zeq ñ P peq ď P pf q Theorem 2. (Equally likely outcomes) The axioms of probability are satisfied with respect to a finite set S, such that for A Ă S, P paq A S Proof. Axioms 1 and 2 are satisfied by construction of P(A). Let A i be a collection of disjoint subsets of S. 8ď kď P p A i q P p A i q (S is finite) Ť k A i S kÿ A i S (defn of P) (addition principle: A Y B A ` B q kÿ P pa i q (defn of P) Note that since S is finite there could not be infinitely many disjoint nonempty subsets, so there exists k, s.t. A i H for j ą k. Review Problems (1) For any A Ă S derive an expression for P pa c q in terms of P paq. (2) (Finite Additivity) For any n events A 1,..., A n Ă S with A i X A j H when i j), prove nď nÿ P p A i q P pa i q (3) For any events E, F prove that P pe Y F q P peq ` P pf q P pe X Eq

7 THEORY OF PROBABILITY 7 (4) (Countable Subadditivity) For any set of events E 1... not necessarily disjoint, prove 8ď 8ÿ P p E i q ď P pe i q Hint: Construct a sequence of pairwise disjoint (mutually exclusive) sets from E 1... with the same union as Ť 8 E i. Solutions (1) Since E X E c H and E Y E c S, by Axioms 2 and 3, 1 P psq P pe Y E c q P peq ` P pe c q ñp pe c q ď 1 P peq (2) Define A n`1 H, A n`2 H, etc. and apply countable additivity. (3) Represent EYF EYpF XE c q as a union of mutually exclusive sets. By (2), P pe Y F q P peq ` P pf X E c q Similarly, we can represent F pf XEqYpF XE c q as a union of mutually exclusive sets. By (2), P pf q P pf XEq`P pf XE c q. Combining the above, we get P peq`p pf q P pe YF q`p pf X Eq. (4) We can form a sequence of mutually exclusive sets F i that have the same union as E i s. F 1 E 1 Since F i are disjoint, F 2 E 2 ze 1 F 3 E 3 zpe 1 Y E 2 q... F n E n z Y n 1 k 1 E k P pye i q P pyf i q ř P pf i q ď ř P pe i q (since F i Ă E i q

8 8 VLADIMIR KOBZAR References [1] Austin, Theory of Probability lecture notes, tim/tofp [2] Bernstein, Theory of Probability lecture notes, brettb/probsum2015/index.html [3] Ross, A First Course in Probability (9th ed., 2014)