ECE 302: Chapter 02 Probability Model

Transcription

1 ECE 302: Chapter 02 Probability Model Fall 2018 Prof Stanley Chan School of Electrical and Computer Engineering Purdue University 1 / 35

2 1. Probability Model 2 / 35

3 What is Probability? It is a number. Always between 0 and 1. Always the probability of an event. Example. The probability of getting a Head when tossing a coin: P( H ) = 3 / 35

4 Three Elements of a Probability Model 1. Sample Space 2. Event 3. Probability Law 4 / 35

5 2. Sample Space 5 / 35

6 Definition (Sample Space) A sample space Ω is We denote ω as an element in Ω. Example. Coin flip: Ω = Throw a dice: Ω = Waiting time for a bus in West Lafayette: Ω = 6 / 35

7 3. Event Space 7 / 35

8 Definition (Event) An event F is a in the sample space Ω. Outcome VS Event: Example. Throw a dice. Let Ω = {1, 2, 3, 4, 5, 6}. F 1 = {even numbers} = {2, 4, 6}. F 2 = {less than 3} = {1, 2}. Example. Wait a bus. Let Ω = {0 t 30}. F 1 = {0 t < 10} F 2 = {0 t < 5} {20 < t 30}. 8 / 35

9 Definition (Event Space) The collection of all possible events is called the Event Space or σ-field denoted as F. F satisfies the following two properties: If F F, then If F 1, F 2,... F, then Example. Ω = {H, T }, the event space is 9 / 35

10 4. Probability Law 10 / 35

11 Definition (Probability Law) A probability law is a function P : F [0, 1] that maps an event A to a real number in [0, 1]. The function must satisfy three axioms known as Probability Axioms. I. Non-negativity: II. Normalization: 11 / 35

12 III. Additivity: For any disjoint subsets {A 1, A 2,...}, it holds that [ ] P A n = P[A n ]. n=1 n=1 12 / 35

13 Properties of Probability 1. P[A c ] = 1 P[A]. 2. For any A Ω, P[A] P[ ] = / 35

14 Properties of Probability 4. For any A and B, P[A B] = P[A] + P[B] P[A B]. 14 / 35

15 Properties of Probability 5. (Union Bound) For any A and B, P[A B] P[A] + P[B]. 15 / 35

16 Properties of Probability 6. If A B, then P[A] P[B] Example. A = {t 5}, and B = {t 10}, then P[A] P[B]. 16 / 35

17 Example Let the events A and B have P[A] = x, P[B] = y and P[A B] = z. Find the following probabilities. (a) P[A B] (b) P[A c B c ] 17 / 35

18 Example (c) P[A c B c ] (d) P[A B c ] 18 / 35

19 5. Conditional Probability 19 / 35

20 Definition (Conditional Probability) Assume P[B] 0. The conditional probability of A given B is Difference: P[A B] = P[A B] P[B] and P[A B] = P[A B]. P[Ω] 20 / 35

21 Figure: Illustration of conditional probability and its comparison with P[A B]. 21 / 35

22 Example. Let A = {Eat 2 burgers} and B = {Win a football game}. Example. Throw a dice. Let A = {Get 3} and B = {odd numbers}. 22 / 35

23 Theorem (Bayes Theorem) For any two events A and B such that P[A] > 0 and P[B] > 0, it holds that P[B A] P[A] P[A B] =. P[B] 23 / 35

24 Theorem (Law of Total Probability) Let {A 1, A 2,..., A n } be a partition of Ω, i.e., A 1,..., A n are disjoint and Ω = A 1 A 2... A n. Then, for any B Ω, P[B] = n P[B A i ] P[A i ]. i=1 24 / 35

25 Figure: Law of total probability decomposes the probability P[B] into multiple conditional probabilities P[B A i ]. The probability of obtaining each P[B A i ] is P[A i ]. 25 / 35

26 Example 1: Tennis Tournament Consider a tennis tournament. Your probability of winning the game is 0.3 against against against 1 4 of the players (Event A). of the players (Event B). of the players (Event C). What is the probability of winning the game? 26 / 35

27 Example 2: Communication Channel Consider a communication channel shown below. The probability of sending a 1 is p and the probability of sending a 0 is 1 p. Given that 1 is sent, the probability of receiving 1 is 1 η. Given that 0 is sent, the probability of receiving 0 is 1 ε. Find P[Receive 1] Find P[Send 1] Find P[Receive 1 Send 1] Find P[Send 1 Receive 1] 27 / 35

28 Properties of Conditional Probability Theorem Let P[B] > 0. The conditional probability P[A B] satisfies Axiom I to Axiom III. Proof. Axiom I: Axiom II: Axiom III: 28 / 35

29 6. Independence 29 / 35

30 Independence Definition Two events A and B are statistically independent if Disjoint VS Independent. 30 / 35

31 Examples of Independence Example 1. Throw a dice twice. Let A = {1st dice is 3} and B = {2nd dice is 4}. Are A and B independent? 31 / 35

32 Examples of Independence Example 2. Throw a dice twice. Let A = {1st dice is 1} and B = {sum is 7}. Are A and B independent? 32 / 35

33 Examples of Independence Example 3. Throw a dice twice. Let Are A and B independent? A = {max is 2} and B = {min is 2}. 33 / 35

34 Independence Via Conditional Probability Recall that P[A B] = P[A B] P[B]. If A and B are independent, then P[A B] = P[A] P[B] Therefore, Interpretation. P[A B] = P[A B] P[B] = P[A] P[B] P[B] = P[A]. 34 / 35

35 Prisoner s Dilemma Three Prisoners: A, B, C. The King decides to release 2 and kill 1. You were A. Your chance of release is 2/3. Suppose you know the guard well. You can ask him about which of B or C will be released. But if you find out B (or C) is released, your chance becomes 1/2. How come!! 35 / 35