Statistics and Econometrics I

Transcription

1 Statistics and Econometrics I Probability Model Shiu-Sheng Chen Department of Economics National Taiwan University October 4, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

2 Part I Set Theory Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

3 Set Theory Review of Set Theory Definition (Set) A set is a collection of elements. Example: A = {1, 2, 3}. Definition (Subset) B is a subset of A if every element of B is also in A. It is denoted by B A Example: Let A = {1, 2, 3}, then B = {1, 3} is a subset of A. A = B if A B and B A Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

4 Set Theory Complement A c = {x x A} Intersection A B = {x x A and x B} Set Operations Operations Union A B = {x x A or x B} Relative Complement A B = A B c = {x x A and x B c } = {x x A and x B} Can be shown by the Venn Diagram Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

5 Set Theory Set Operations De Morgan s laws: (A B) c = A c B c (A B) c = A c B c Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

6 Set Theory Set Operations De Morgan s laws: (A B) c = A c B c (A B) c = A c B c Distributivity laws: B (A 1 A 2 ) = (B A 1 ) (B A 2 ) B (A 1 A 2 ) = (B A 1 ) (B A 2 ) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

7 Set Theory Some Important Definitions Definition (Empty Set) An empty set is a set consisting of no elements. It is denoted by = {} Definition (Disjoint Sets) Two sets A, B are said to be disjoint if A B = Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

8 Set Theory Some Important Definitions Definition (Partition) A 1,..., A n Ω is said to be a partition of Ω if 1 A i A j = for i j (pairwise disjoint) 2 A 1 A 2 A n = Ω Example: a trivial partition: (A, A c ) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

9 Part II Probability Theory Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

10 Probability Theory Probability Theory Basic concepts: Sample spaces Events Probability measure Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

11 Probability Theory Probability Theory Definition (Sample Space) The set, Ω, of all possible outcomes of a particular experiment is called the sample space for the experiment. Points ω Ω are called sample outcomes. Subsets of Ω are called events. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

12 Probability Theory Examples of Sample Space Roll a die (discrete sample space: finite) Ω = {1, 2, 3, 4, 5, 6} Flip a coin until a head appears (discrete sample space: infinitely countable) Ω = {H, T H, T T H, T T T H, T T T T H,...} Draw at random a point in the interval [0, 1] (continuous sample space) Ω = [0, 1] Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

13 Probability Theory Probability Theory Definition (Event) An event, denoted by E, is just a subset of Ω. Examples: Roll a die E = the event of odd numbers = {1, 3, 5} Ω Flip a coin until a head appears E = the event of at most two tails = {H, T H, T T H} Ω Draw at random a point in the interval [0, 1] E = the event that a point will fall into [ 0, 1 2] = { x x [ 0, 1 2]} Ω Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

14 Probability Theory What is Probability? Probability is a mathematical language for quantifying uncertainty. To answer the question how likely is it...? Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

15 Probability Theory What is Probability? Probability is a mathematical language for quantifying uncertainty. To answer the question how likely is it...? To put it loosely, probability is a number between 0 and 1, where: a number close to 0 means not likely a number close to 1 means quite likely Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

16 Probability Theory What is Probability? Probability is a mathematical language for quantifying uncertainty. To answer the question how likely is it...? To put it loosely, probability is a number between 0 and 1, where: a number close to 0 means not likely a number close to 1 means quite likely How to assign probability to events? (A) The classical approach (B) The relative frequency approach (C) The subjective approach Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

17 Probability Theory (A) The Classical Approach Principle of Indifference: every outcome is equally likely to occur. Examples: P (A) = card(a) card(ω) Draw one card at random from a standard deck of 52 cards Roll a six-side die It is the interpretation identified with the works of Jacob Bernoulli and Pierre-Simon Laplace. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

18 Probability Theory (B) The Relative Frequency Approach Some people argue that we need to further justify the assumption that every outcome is equally likely to occur by experience. 1 The relative frequency approach involves taking the follow three steps in order to determine P (A), the probability of an event A: Perform an experiment N times. Count the number of times the event A of interest occurs, call the number N(A). Then, the probability of event A is: N(A) P (A) = lim N N 1 Such as Richard von Mises. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

19 Probability Theory (B) The Relative Frequency Approach Example: Flip a coin. Let X = # of Heads Total Flips Consider total flips = 50, 100, and 1000 If the coin is fair, it is expected that the fraction of heads is close to 1 2 after many flips. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

20 Probability Theory 50 flips X Total Flips Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

21 Probability Theory 100 flips X Total Flips Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

22 Probability Theory 1000 flips X Total Flips Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

23 Probability Theory (C) Subjective Approach The subjective approach is simply a personal opinion. I think there is an 80% chance of rain today. I think there is an 50% chance that I will get an A+ in this course It is also called personal probability Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

24 Probability Theory Probability Models: Kolmogorov Axioms Now we present a probability model using the axioms of probability. This axiomatic approach to probability is developed by a Soviet mathematician, Andrey Kolmogorov ( ). Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

25 Probability Theory Probability Model Definition (Probability Measure) A real-value function P is called a probability measure on the sample space Ω if all events A Ω are assigned numbers P (A) satisfying (a) P (Ω) = 1 (b) P (A) 0 for all A Ω (c) For all disjoint A, B Ω, P (A B) = P (A) + P (B) The pair (Ω, P ) is called a probability model Axiom (c) can be extended to a finite union of disjoint events: A 1, A 2,..., A k a countably infinite union of disjoint events: A1, A 2,... Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

26 Probability Theory Example Suppose that a Econ 2014 class contains 100 students, such that Status Sophomores Juniors Seniors Count Proportion 50/100 30/100 20/100 Randomly select one student from the Econ 2014 class. Defining the following events: So = the event that a Sophomore is selected Ju = the event that a Junior is selected Se = the event that a Senior is selected The sample space is Ω = {So, Ju, Se}. Use the classical approach to assign probability: P (So) = 0.5, P (Ju) = 0.3, P (Se) = 0.2 Check if each of the three axioms of probability are satisfied. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

27 Probability Theory Some Useful Rules Theorem (Some Useful Rules) P (A) + P (A c ) = 1 P (A) 1 B A implies that P (B) P (A) P (A B) = P (A) + P (B) P (A B) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

28 Probability Theory 可能性的評估 Linda 今年 31 歲, 單身, 個性率直且聰穎 她在大學時主修哲學 當她在學時, 她十分熱衷於性別歧視與社會正義等議題 此外, 她也參與了反核四的示威運動 根據以上的資料, 請對以下八種對於 Linda 的描述, 根據其可能性大小 ( 機率大小 ) 排列 1 代表可能性最大, 2 則為其次, 依此類推, 8 代表最不可能 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

29 Probability Theory A. Linda 是一個小學老師 B. Linda 在書店工作且在下班後參與瑜珈的課程 C. Linda 熱衷於參與女性主義運動 D. Linda 是一個為人精神治療的社工 E. Linda 是一個銀行行員 F. Linda 是哲學學會的會員 G. Linda 是一個保險業務代表 H. Linda 是一個熱衷於女性主義運動的銀行行員 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

30 Probability Theory 可能性的評估 根據心理學家 Daniel Kahneman 與 Amos Tversky 的研究, 在 88 個受試者中, 有 78 個給了錯誤的答案 (89%) Daniel Kahneman (1934 ), Israeli American psychologist, Nobel Laureate (2002) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

31 Probability Theory 可能性的評估 本班結果 76 位同學中, 23 位答錯 (30%) 2015 年 97 位同學中, 32 位答錯 (33%) 2014 年 67 位同學中, 25 位答錯 (37%) 2013 年 65 位同學中, 30 位答錯 (46%) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

32 Conditional Probability Conditional Probability Taking into account the available additional knowledge could result in revised (and more accurate) probabilities about the events of interest to us. Fir instance, The risk manager of a bank observed that over the past 10 years, 5% of corporate loans defaulted Additionally, during recession periods, 11% of corporate loans defaulted over the past 10 years We call these revised probabilities conditional probabilities Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

33 Conditional Probability Conditional Probability Definition The conditional probability of an event A given that an event B has occurred is P (A B) = P (A B) P (B) whenever P (B) 0. In the previous example, P (default) = 5% < 11% = P (default recession) One more example: Die Roll P ({1} Odd) = P ({1} {1, 3, 5}) = 1/3 > P ({1}) = 1/6 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

34 Conditional Probability Properties of Conditional Probability Since conditional probability is just a probability, it satisfies the three axioms of probability. Hence, as long as P (B) > 0, P (A B) 0 P (B B) = 1 If A1 and A 2 are disjoint, then P (A 1 A 2 B) = P (A 1 B) + P (A 2 B) likewise for finite and countably infinite unions. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

35 Conditional Probability Joint and Marginal Probabilities Y N Democrat Republican Y N Democrat Republican Joint Probability: e.g., P (Democrat Y) Marginal Probability: e.g., P (Democrat) Conditional Probability: e.g., P (Democrat Y) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

36 Conditional Probability Multiplication Rule Simple Rule P (A B) = P (A B)P (B) = P (B A)P (A) Example: A box contains 6 green balls and 4 blue balls. We randomly (and without replacement) draw two balls from the box. What is the probability that the second ball selected is blue? Extended Rule P (A B C) = P ((A B) C) = P (C A B)P (A B) = P (C A B)P (B A)P (A) Example: Three cards are dealt successively at random and without replacement from a standard deck of 52 playing cards. What is the probability of receiving, in order, a king, a queen, and a jack? Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

37 Independence Independent Events Definition Two events A, B Ω are said to be independent if P (A B) = P (A) Hence, by the definition of conditional probability, P (A B) = P (A B)P (B) = P (A)P (B) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

38 Independence Independent Events: More Theorems Theorem If A and B are independent, then A and B c are independent If A and B are independent, then A c and B c are independent Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

39 Independence Independence vs. Disjoint Disjoint P (A B) = 0 Independent P (A B) = P (A)P (B) Assume P (A) 0, P (B) 0 A, B independent A, B NOT disjoint A, B disjoint A, B NOT independent Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

40 Independence Mutual Independence Definition Three events A 1, A 2, and A 3 are mutually independent if (1) The events are pairwise independent. That is, P (A 1 A 2 ) = P (A 1 )P (A 2 ) P (A 1 A 3 ) = P (A 1 )P (A 3 ) P (A 2 A 3 ) = P (A 2 )P (A 3 ) (2) Moreover, P (A 1 A 2 A 3 ) = P (A 1 )P (A 2 )P (A 3 ) The idea of mutual independence can be extended to four or more events each pair, triple, quartet, and so on, must satisfy the above type of multiplication rule. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

41 Independence An Example Toss a fair coin twice: Ω = {HH, HT, T H, T T } Let A = {H on first toss} = {HH, HT } B = {H on second toss} = {HH, T H} C = {Both tosses the same} = {HH, T T } P (A) = P (B) = P (C) =? P (A B); P (A C); P (B C)? P (A B C)? This example shows the events are pairwise independent but they are NOT independent. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

42 Part III Bayes Theorem Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

43 Bayes Theorem Bayes Theorem: Motivation An iphone was found to be defective (D). There are three factories (A, B, C) where such smartphones are manufactured. A Quality Control Manager (QCM) is responsible for investigating the source of found defects. This is what the QCM knows about the company s iphone production and the possible source of defects: Factory % of total production Probability of defective product A 0.35 = P (A) = P (D A) B 0.35 = P (B) = P (D B) C 0.30 = P (C) = P (D C) The QCM would like to answer the following question: If a randomly selected iphone is defective, what is the probability that the iphone was manufactured in factory C? That is, P (C D) =? Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

44 Bayes Theorem Bayes Theorem Law of Total Probability ( 總機率法則 ) Bayes Theorem ( 貝氏定理 ) Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

45 Bayes Theorem Thomas Bayes British mathematician ( ) He is credited with inventing Bayes Theorem Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

46 Bayes Theorem Law of Total Probability Theorem Let A 1, A 2,..., A n Ω be a partition of Ω, and T Ω with P (T ) > 0. The Law of Total Probability says that n P (T ) = P (T A j )P (A j ), j=1 Note P (Ai ) is called prior probability P (T Ai ) is called sample probability Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

47 Bayes Theorem Law of Total Probability A 1 A 2 A 3.. ) ( ) ( ) ( A P A T P A T P ) ( ) ( ) ( A P A T P A T P ) ( ) ( ) ( n n n A P A T P A T P Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

48 Bayes Theorem Bayes Theorem Theorem P (A i T ) = P (A i T ) P (T ) where P (A i T ) is called posterior probability = P (T A i )P (A i ) n j=1 P (T A j)p (A j ), In the iphone example, P (D C)P (C) P (C D) = P (D A)P (A) + P (D B)P (B) + P (D C)P (C) = = Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

49 Bayes Theorem Example (Monty Hall Problem) There are three closed doors, Door A, Door B and Door C. Behind one of these doors is a car; behind the other two are goats. The contestant does not know where the car is, but Monty Hall does. The contestant picks a door and Monty opens one of the remaining doors, one he knows doesn t hide the car. If the contestant has already chosen the correct door, Monty opens either of the two remaining doors (with equal probabilities). Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

50 Bayes Theorem Example (Monty Hall Problem) After Monty has shown a goat behind the door that he opens, the contestant is always given the option to switch doors. What is the probability of winning the car if she stays with her first choice? What if she decides to switch? The contestant, of course, wants to maximizing his chance of winning the car. Let s play the game Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

51 Bayes Theorem Example (Monty Hall Problem) Think about the following argument. After you named Door A, suppose Monty showed you Door B. According to Bayes Rule, we can get and P (A B c ) = P (A Bc ) P (B c ) P (C B c ) = P (C Bc ) P (B c ) = P (A) P (B c ) = 1/3 2/3 = 1/2 = P (C) P (B c ) = 1/3 2/3 = 1/2 That means it doesn t matter if you switch or not. Notice that we use the fact that A B c = A when A, B disjoint. This is a paradox since indeed you are much more likely to win a car by switching. Why? Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

52 Bayes Theorem Example (Monty Hall Problem) Monty Shows Door A Door B Door C Car Door A 0 1/6 1/6 1/3 Behind Door B 0 0 1/3 1/3 Door C 0 1/3 0 1/3 0 1/2 1/2 P (A SB) = P (C SB) = P (A SB) P (SB) P (C SB) P (SB) = 1/6 1/2 = 1/3 = 1/3 1/2 = 2/3 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

53 Part IV Problems with Probability Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

54 Problems with Probability Problems with Probability Statistical discrimination (statistical screening) 馬路三寶 : 未看先猜三寶 見警即逃, 非奸即盜 Assuming events are independent when they are not A 2-engine airplane has 0.01 probability of an engine failure in general. So the probability of an airplane crash is only ?! Not understanding when events are independent Law of small numbers (Amos Tversky and Daniel Kahneman); Gambler s Fallacy Clusters happen The Prosecutor s Fallacy Context surrounding statistical evidence is neglected Reversion or regression to the mean Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

55 Problems with Probability Law of Small Numbers A man tosses a fair coin eight times. Which of the following sequences of coin tosses is the man more likely to get a head (H) on his next toss? This one: Or this one: T T T T T T T T H H T H T T H H Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

56 Problems with Probability Law of Small Numbers A man tosses a fair coin eight times. Which of the following sequences of coin tosses is the man more likely to get a head (H) on his next toss? This one: Or this one: T T T T T T T T H H T H T T H H Answer: neither Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52

57 Problems with Probability Law of Small Numbers A man tosses a fair coin eight times. Which of the following sequences of coin tosses is the man more likely to get a head (H) on his next toss? This one: Or this one: T T T T T T T T H H T H T T H H Answer: neither Law of small numbers: a person mistakenly assumes that a departure from what occurs on average in the long term will be corrected in the short term. Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 4, / 52