Statistical Methods for the Social Sciences, Autumn 2012

Size: px

Start display at page:

Download "Statistical Methods for the Social Sciences, Autumn 2012"

Ruth Holt
5 years ago
Views:

1 Statistical Methods for the Social Sciences, Autumn 2012 Review Session 3: Probability. Exercises Ch.4. More on Stata TA: Anastasia Aladysheva Office hours: Mon 10:00-12:00 Rigot 10 October 10, / 39

2 Plan for the RS3: 1. Some basic probability concepts: follow-up on the lecture 3; 2. Exercises Chapter 4 from the textbook; 3. More on Stata: PS2 2 / 39

3 Basic Concepts of Probability Theory Define: P : Probability Function Ω : Sample space A : Event and a subset A Ω A probability space is a pair (Ω P) consisting of a set Ω and a function P which assigns to each subset A of Ω a real number P(A) in the interval [0 1]. For an event A, the real number P(A) is called the probability of A. Axiom: P(Ω) = 1 Example: throw a dice Consider the set Ω = { }. For each subset A of Ω, define: number of elements in A P(A) =. Then the pair (Ω P) is a probability space. One can view 6 this probability space as a model for the situation throw a dice. 3 / 39

4 Enumerate the sample space: examples Example 1: flip a coin 4 / 39

5 Enumerate the sample space: examples Example 1: flip a coin ANSWER: Ω = { heads tails } 5 / 39

6 Enumerate the sample space: examples Example 1: flip a coin ANSWER: Ω = { heads tails } Example 2: pick a ball An urn contains a red ball, a blue ball and a yellow ball. I pick one ball from the urn. 6 / 39

7 Enumerate the sample space: examples Example 1: flip a coin ANSWER: Ω = { heads tails } Example 2: pick a ball An urn contains a red ball, a blue ball and a yellow ball. I pick one ball from the urn. ANSWER: Ω = { blue red yellow } 7 / 39

8 Enumerate the sample space: examples Example 1: flip a coin ANSWER: Ω = { heads tails } Example 2: pick a ball An urn contains a red ball, a blue ball and a yellow ball. I pick one ball from the urn. ANSWER: Ω = { blue red yellow } Example 3: pick a ball twice, without replacement 8 / 39

9 Enumerate the sample space: examples Example 1: flip a coin ANSWER: Ω = { heads tails } Example 2: pick a ball An urn contains a red ball, a blue ball and a yellow ball. I pick one ball from the urn. ANSWER: Ω = { blue red yellow } Example 3: pick a ball twice, without replacement ANSWER: Order matters! Ω = { blue red }, { red blue }, { blue yellow }, { yellow blue }, { red yellow }, { yellow red } 9 / 39

10 More on Probability 0 P(A) 1 P(A) = 0: the event will never occur; P(A) = 1: the event will definitely (100%) occur Full group of events: P(A 1 ) + P(A 2 ) P(A n) = 1 Two opposite events (tails/heads of a coin): P(A) + P(A) = 1 10 / 39

11 Calculate probability: simple examples Example 1: The probability of rolling an even number on a fair die is: P(rolling a even number) = number of even number faces number of faces = 3 6 = / 39

12 Calculate probability: simple examples Example 1: The probability of rolling an even number on a fair die is: P(rolling a even number) = number of even number faces number of faces = 3 6 = 1 2 Example 2: Suppose there is 5 choices of answer for a question in your exam. What is the probability that you choose the right answer by chance? P(right answer) = number of right answers number of answers = / 39

13 Calculate probability: simple examples Example 1: The probability of rolling an even number on a fair die is: P(rolling a even number) = number of even number faces number of faces = 3 6 = 1 2 Example 2: Suppose there is 5 choices of answer for a question in your exam. What is the probability that you choose the right answer by chance? P(right answer) = number of right answers number of answers = 1 5 Example 3: What is the probability of picking a jack in a normal 52-card deck? P(jack) = number of jacks number of cards = / 39

14 More on Probability: when can we sum up and when can we multiply the probabilities? 1. We can sum up disjoint events: P(A + B) = P(A) + P(B). Example: There are 30 balls in the box: 10 red, 5 blue and 15 white. What s the probability of picking a coloured one? Answer: P(coloured ball) = = / 39

15 More on Probability: when can we sum up and when can we multiply the probabilities? 1. We can sum up disjoint events: P(A + B) = P(A) + P(B). Example: There are 30 balls in the box: 10 red, 5 blue and 15 white. What s the probability of picking a coloured one? Answer: P(coloured ball) = = 1 2 This is an example of the probability of alternative occurrences/events. It happens when we want to know the probability that something, or another thing, happens. 15 / 39

16 More on Probability: when can we sum up and when can we multiply the probabilities? 1. We can sum up disjoint events: P(A + B) = P(A) + P(B). Example: There are 30 balls in the box: 10 red, 5 blue and 15 white. What s the probability of picking a coloured one? Answer: P(coloured ball) = = 1 2 This is an example of the probability of alternative occurrences/events. It happens when we want to know the probability that something, or another thing, happens. 2. Product of two events A B: joint realization of two events. Example 1: A - the cell phone has no defect, B - the cell phone is white, A B - the cell phone has no defect and is white. 16 / 39

17 More on Probability: when can we sum up and when can we multiply the probabilities? 1. We can sum up disjoint events: P(A + B) = P(A) + P(B). Example: There are 30 balls in the box: 10 red, 5 blue and 15 white. What s the probability of picking a coloured one? Answer: P(coloured ball) = = 1 2 This is an example of the probability of alternative occurrences/events. It happens when we want to know the probability that something, or another thing, happens. 2. Product of two events A B: joint realization of two events. Example 1: A - the cell phone has no defect, B - the cell phone is white, A B - the cell phone has no defect and is white. Example 2: What is the probability of obtaining two sixes when rolling two fair dice? Answer: P(having two sixes) = = / 39

18 Independent Events Note: in the previous examples 2 a) and 2 b) we considered independent events! Two events A and B are called independent, if P(A B) = P(A) P(B) Remember Two events are independent if the probability of one of them is not affected by knowing whether the other has occurred or not. P(A B) is called joint probability. Example: Throwing two dice (see 2 b): P(having two sixes) = P(rolling 6) P(rolling 6) 18 / 39

19 Another example on independent events An urn contains 20 red balls, and 10 blue balls. What is the probability that, when we pick two balls without replacement, we obtain exactly one red and one blue? 19 / 39

20 Another example on independent events An urn contains 20 red balls, and 10 blue balls. What is the probability that, when we pick two balls without replacement, we obtain exactly one red and one blue? ANSWER: Obviously, there are two ways of obtaining this outcome, i.e. R&B and B&R. The order is important, since we pick balls without replacement. Now, these two ways of obtaining exactly one red and one blue balls are just alternative events. Thus, P(1 red and 1 blue) = P(R&B) + P(B&R) = Here events R&B and B&R are independent. However, P(R&B) P(R) P(B) Why? P(B&R) P(B) P(R) 20 / 39

21 Conditional probability Conditional probability is very important for cases in which events are not independent. In such cases, the probability of an event depends on the occurrence of some other event. In general, for two events A and B, we say that the probability of A given B is: How to compute? P(A B) Bayes formula P(A B) = P(A B) P(B) Refer to this page 21 / 39

22 Compare two examples: What is the probability of drawing two queens in a normal 52-card deck (a) with replacement? (b) without replacement? 22 / 39

23 Compare two examples: What is the probability of drawing two queens in a normal 52-card deck (a) with replacement? (b) without replacement? ANSWER: (a) With replacement: P(two queens) = / 39

24 Compare two examples: What is the probability of drawing two queens in a normal 52-card deck (a) with replacement? (b) without replacement? ANSWER: (a) With replacement: P(two queens) = (b) Without replacement: P(two queens) = P(Q on 1st draw) P(Q on 2nd draw Q on 1st draw) = / 39

25 What is a Random variable? Formal definition: Consider a probability space (Ω P). A random variable is a map X from Ω into the set of real numbers R. A random variable is characterised by a distribution function, expected value, variance, standard deviation. Probability distribution tells you how likely each possible outcome is to be the value of the variable. 25 / 39

26 Example from the Lecture 3 (slide 7) Toss a fair coin twice y is my random variable defined as the number of heads that I got. There are 4 possible outcomes. Let s define the sample space: Sample y (T, T ) 0 (T, H) 1 (H, T ) 1 (H, H) 2 It s probability distribution is defined in such as way: Sample y 0 1/4 1 1/2 2 1/4 26 / 39

27 Draw the distribution: discrete y in flipping a coin twice is an example of a discrete random variable We can draw it s probability distribution in such a way: This is an example of a binomial distribution. 27 / 39

28 Draw the distribution: continuous What if y is a continuous random variable, i.e. can take on any real value? We can draw it s probability distribution in such a way: This is an example of a standard normal distribution. The graph above represents the probability density function (pdf). The coloured area is the cumulative probability (cdf). To get it, if we know that a random variable is standard normally distributed, we need to know the Z-score. 28 / 39

29 What is the difference between normal and standard normal distributions? Recall, that each probability distribution is characterized by mean (µ) and standard deviation (σ). Above that, we know the law, or the pdf, of that distribution. Normal distribution For the normal we have the pdf: f (y) = 1 σ 2π e (y µ) 2 /2σ 2 This function exactly has this bell shape and is symmetric, like we saw on the pictures in the lecture. Standard Normal distribution For the standard normal, plug in µ = 0 and σ = 1 in the pdf above. The resulted function (I don t present it here) will also has the bell shape, but will be symmetric around 0 (see our previous picture). 29 / 39

30 What is the difference between normal and standard normal distributions? 30 / 39

31 Q. How can we calculate the probability (coloured area under the curve) of y if we do not have any software in hands? 1. Standardize y! How? Convert y to a Z-score using: Z = y µ σ If y was normally distributed, Z is now standard normally distributed! (Proof: algebra, using the mathematical properties of expected value and variance) 2. Use a table to look for the value of a probability! 3. Table A p. 592 in the textbook, or use a handout table. Be careful: the handout gives the values which represent the area to the left of the Z-score, while in the textbook you have the values to the right! (If you look both values will sum up to 1) 31 / 39

32 Why we care about normal distribution? Central Limit Theorem provides an answer! Central Limit Theorem For random sampling with a large sample size n, the sampling distribution of the sample mean y is approximately a normal distribution. 32 / 39

33 Why we care about normal distribution? Central Limit Theorem provides an answer! Central Limit Theorem For random sampling with a large sample size n, the sampling distribution of the sample mean y is approximately a normal distribution. In other words, If we repeatedly select samples of size n from the population, and each time form a particular statistic (in this case sample means y), we ll get some variation and a mean of this statistic. If we then draw the probability distribution of y, the graph will resemble of a normal distribution pdf. 33 / 39

34 Why we care about normal distribution? Central Limit Theorem provides an answer! Central Limit Theorem For random sampling with a large sample size n, the sampling distribution of the sample mean y is approximately a normal distribution. In other words, If we repeatedly select samples of size n from the population, and each time form a particular statistic (in this case sample means y), we ll get some variation and a mean of this statistic. If we then draw the probability distribution of y, the graph will resemble of a normal distribution pdf. Why the result of CLT is useful? We can use the normal distribution to find probabilities about y, other statistics, and point estimates. 34 / 39

35 Resources Textbook, Chapter 4 Wikipedia Other references (more advanced): Hogg and Craig, Greene, Wooldridge appendix, Stock and Watson 35 / 39

36 Appendix: Bayes formula (n events case) I Formula of total probability Theorem Let event A may occur only if one of the events B 1, B 2,...,B n, that constitute a full group, occurs. What s the probability that A occurs? P(A) = P(B 1 ) P B1 (A) + P(B 2 ) P B2 (A) P(B n) P Bn (A) (computation of probability by division into possible cases) Example: In the French Open final, Federer plays the winner of the semifinal between Djokovic and Nadal. A bookmaker estimates that probability of Djokovic winning the semifinal is 75%. The probability that Federer can beat Djokovic is estimated to be 51%, whereas the probability that Federer can beat Nadal is estimated to be 80%. The bookmaker therefore computes the probability that Federer wins the French Open, using division into possible cases, as follows: P(Federer wins the final) = / 39

37 Appendix: Bayes formula (n events case) II Derivation of Bayes formula We have: P(A) = P(B 1 ) P B1 (A) + P(B 2 ) P B2 (A) P(B n) P Bn (A) Suppose that event A indeed occurred. How would we change our hypotheses about P A (B 1 ), P A (B 2 ),...,P A (B n)? Let s get P A (B 1 ). We know that the product of two events is equal: P(A B 1 ) = P(A) P A (B 1 ) = P(B 1 ) P B1 (A). Therefore, P A (B 1 ) = P(B 1) P B1 (A) or P(A) P(B 1 ) P B1 (A) P A (B 1 ) = P(B 1 ) P B1 (A) + P(B 2 ) P B2 (A) P(B n) P Bn (A) 37 / 39

38 Appendix: Bayes formula (n events case) III Application of Bayes formula Example: Suppose that Bob can decide to go to work by one of three modes of transportation, car, bus, or commuter train. Because of high traffic, if he decides to go by car, there is a 50% chance he will be late. If he goes by bus, which has special reserved lanes but is sometimes overcrowded, the probability of being late is only 20%. The commuter train is almost never late, with a probability of only 1%, but is more expensive than the bus. Suppose that Bob is late one day, and his boss wishes to estimate the probability that he drove to work that day by car. Since he does not know which mode of transportation Bob usually uses, he gives a prior probability of 1 to each of the three 3 possibilities. What is the boss estimate of the probability that Bob drove to work? 38 / 39

39 Appendix: Bayes formula (n events case) IV Solution P(bus) = P(car) = P(train) = 1 3 P(late) car = 0.5 P(late) train = 0.01 P(late) bus = 0.2 We want to calculate P(car) late. By Bayes formula, this is: P(car) late = P(late) car P(car) P(late) car P(car) + P(late) bus P(bus) + P(late) train P(train) = / 39

Mathematical Foundations of Computer Science Lecture Outline October 18, 2018

Mathematical Foundations of Computer Science Lecture Outline October 18, 2018 The Total Probability Theorem. Consider events E and F. Consider a sample point ω E. Observe that ω belongs to either F or