Probability Review I

Transcription

1 Probability Review I Harvard Math Camp - Econometrics Ashesh Rambachan Summer 2018

2 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

3 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

4 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

5 The sample space and events We wish to model a random experiment - an experiment/process whose outcome cannot be predicted beforehand. What are the building blocks? The sample space Ω is the set of all possible outcomes of a random experiment. We denote an outcome as ω Ω. An event A is a subset of the sample space, A Ω. Let A denote the family of all events.

6 Simple examples Example: Suppose we survey 10 randomly selected people on their employment status and count how many are unemployed. Ω = {0, 1, 2,..., 10} A is the event that more than 30% of those surveyed are unemployed. A = {4, 5, 6,..., 10} Example: Suppose we ask a random person what is their income. Ω = R + A is the event that the person earns between $30, 000 and $40, 000. A = [30, 000, 40, 000]

7 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

8 Putting structure on the set of events To be able to sensibly define probabilities, we need to place some additional structure on the set of events, A. Let Ω be a set and A 2 Ω be a family of its subsets. A is a σ-algebra if and only if it satisfies the following 1. Ω A. 2. A is closed under complements: A A implies that A C = Ω A A. 3. A is closed under countable union: If A n A for n = 1, 2,..., then n A n A. = We assume that A is a σ-algebra. (Ω, A) is a measurable space and A A is measurable with respect to A.

9 Properties of a σ-algebra If A is a σ-algebra, then A. 2. A is closed under countable intersection i.e, if A n A for n = 1, 2,..., then n A n A. Why? 1. This one s simple. 2. Hint: DeMorgan s Law - (A B) C = A C B C.

10 What is probability? We re now ready to finally define what is probability! We will provide the mathematical definition. Not defined directly as a long-run frequency or subjective beliefs. But it will capture all of the properties associated with these. Let (Ω, A) be a measurable space. A measure is a function, µ : A R such that 1. µ( ) = µ(a) 0 for all A A. 3. If A n A for n = 1, 2,... with A i A j = for i j, then µ(u n A n ) = n µ(a n ) If µ(ω) = 1, µ is a probability measure, denoted as P : A [0, 1].

11 Putting it all together So, we model a random experiment as a probability space, (Ω, A, P). 1. Ω - set of outcomes. 2. A - σ-algebra on the set of outcomes. 3. P - a probability measure defined on the σ-algebra.

12 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

13 Basic probability rules We can prove all of the usual probability rules from this. Consider a probability space (Ω, A, P). The following hold: 1. For all A A, P(A C ) = 1 P(A). 2. P(Ω) = If A 1, A 2 A with A 1 A 2, then P(A 1 ) P(A 2 ). 4. For all A A, 0 P(A) P(1). 5. If A 1, A 2 A, then P(A 1 A 2 ) = P(A 1 ) + P(A 2 ) P(A 1 A 2 )

14 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

15 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

16 Conditional Probability Given a random experiment and the information that event B has occurred, what is the probability that the outcome also belongs to event A? Let A, B A with P(B) > 0. The conditional probability of A given B is P(A B) P(A B) = P(B) P(A B) is a probability measure so all the usual probability rules apply! We use conditioning to describe the partial information that an event B gives about another event A. Implies that P(A B) = P(A B)P(B).

17 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

18 Multiplication Rule P( n i=1a i ) = P(A 1 )P(A 2 A 1 )P(A 3 A 2 A 1 )... P(A n n 1 i=1 A i) Proof?

19 The Law of Total Probability Consider K disjoint events C k that partition Ω. That is, C i C j = for all i j and K i=1 C i = Ω. Let C be some event. P(C) = K P(C C i )P(C i ) i=1 Proof?

20 Bayes Rule Bayes Rule: P(B A) = P(A B)P(B) P(A B)P(B) + P(A B C )P(B C ) Proof? Definitely the most important probability rule out there...

21 Outline Random Experiments The sample space and events σ-algebra and measures Basic probability rules Conditional Probability Definition Bayes rule and more Independence

22 Independence What if event B has no information about event A? Two events A, B are independent if P(A B) = P(A) Equivalently, or P(B A) = P(B) P(A B) = P(A)P(B).

23 Independence Let E 1,..., E n be events. E 1,..., E n are jointly independent if for any i 1,..., i k P(E i1 E i2... E ik ) = E i1 Given an event C, events A, B are conditionally independent if P(A B C) = P(A C)P(B C).