TA Qinru Shi: Based on poll result, the first Programming Boot Camp will be: this Sunday 5 Feb, 7-8pm Gates 114.

Size: px

Start display at page:

Download "TA Qinru Shi: Based on poll result, the first Programming Boot Camp will be: this Sunday 5 Feb, 7-8pm Gates 114."

Vanessa Beasley
5 years ago
Views:

1 TA Qinru Shi: Based on poll result, the first Programming Boot Camp will be: this Sunday 5 Feb, 7-8pm Gates 114. Prob Set 1: to be posted tomorrow. due in roughly a week

2 Finite probability space S 1) a set S 2) a function p : S! [0, 1] such that p(s) > 0(8s 2 S) A and P s2s p(s) = 1. S is the sample space, subsets of S are events, and p is the probability distribution. Probability of event A S is p(a) = P a2a p(a). (And p(;) = 0.) Two events are disjoint if their intersection is empty. In general we have p(a [ B) =p(a)+p(b) p(a \ B)

3 What is the chance of rolling a die one time and getting a 6? 1/6 Now, what is the chance of rolling a die twice and getting at least one 6? THINK: 1/6 + 1/6 = 2/6 = 1/3 (From my nephew s fourth grade math text X... except the two events have non-zero intersection.) Instead use any of: a) of the 36 possibilities, enumerate 11 with at least one six, gives 11/36, b) p(a [ B) =p(a)+p(b) p(a \ B) =1/6+1/6 1/36 = 11/36, c) probability of no sixes is (5/6)(5/6), so at least one six is 1 25/36 = 11/36.

4 Example: a) What is the probability that in a group of n people, at least two have the same birthday? (Simplifications: assume no leap years, and assume that all birthdays are equally likely.) Again consider the complement problem, the probability that no two birthdays coincide: Total number of possibilities with no coincidences is (366 n) (i.e., n factors each successive one with one fewer choice of day). Total number of possibilities for n choices of birthdays is 365 n, so the probability of no coincidences is (366 n)/365 n. The probability that at least two coincide is therefore (366 n)/365 n. This probability is rapidly increasing as a function of n and turns out to be greater than.5 for n =23

5 ! Red (upper): The probability 1 that at least two birthdays coincide within a (365 n)!365 n group of n people, as function of n. Green (lower): The probability 1 ( 364 n 1 365) of a birthday coinciding with yours within a group of n people including you.

6 For small n, we can estimate the probability as n(n 1) since n(n 1) 2 = ( n 2) is the number of pairs, and 1/365 is the probability that any pair has a coincident birthday. (Once n is too large, the pairs can no longer be considered to be roughly independent, and starts deviating by around n 15.)

7 b) In a group of 23 people, what it the probability that at least one person has a birthday coincident specifically with yours? First calculate probability that none of the 22 others (again under the above simplifications) has a birthday coincident with a given day: (364/365) 22 Probability that at least one coincides with that day is therefore 1 (364/365) , so a roughly 6% chance. This probability increases more slowly as a function of the size of the group. Note that 1 (364/365) n is well approximated by 1 exp( n/365) for all n.

8 365! Red (upper): The probability 1 that at least two birthdays coincide within a (365 n)!365 n group of n people, as function of n. Green (lower): The probability 1 ( 364 n 1 365) of a birthday coinciding with yours within a group of n people including you /e

9 Conditional Probability Suppose we know that one event has happened and we wish to ask about another. For two events A and B, thejoint probability of A and B is defined as p(a, B) =p(a B) the probability of the intersection of events A and B in the sample space, equivalently the probability that events A and B both occur The conditional probability of A relative to B is p(a B) =p(a B)/p(B) the probability of A given B

10 A B) A B p(a B) =p(a B)/p(B) the probability of A given B

11 Example: Flip a fair coin 3 times. B = event that we have at least one H A = event of getting exactly 2 Hs What is the probability of A given B? In this case, (A B) =A, p(a) =3/8, p(b) =7/8, and therefore p(a B) =3/7. B HHH HHT HTH HTT THH THT TTH TTT

12 \ ; Find probability of events by breaking sample space into disjoint pieces: Then for any event A, p(a) = P i p(a S i)p(s i )= P i p(a \ S i). If S = S 1 [ S 2...[ S n all pairs S i, S j are disjoint: S i \ S j = ; S S A

13 P P Example: Flip a fair coin twice. S HT HH S 1 =firstflipish S 2 =firstflipist TT TH What is the probability of A =getting2hs? p(a) =p(a \ S 1 )p(s 1 )+p(a \ S 2 )p(s 2 )=(1/2)(1/2) + (0)(1/2) = 1/4.

14 S T (Coin) H (Die) S S S

15 Two events A and B are independent if p(a \ B) =p(a)p(b). Since p(a \ B) =p(a B)p(B): A and B are independent i p(a B) =p(a). If p(a \ B) >p(a)p(b) then A and B are said to be positively correlated. (equivalently, p(a B) >p(a)). If p(a \ B) <p(a)p(b) then A and B are said to be negatively correlated. \ Q (p(a B) <p(a)) Alternate notation for joint probability: P (A, B) =P (A \ B) Note that it is symmetric: p(a, B) =p(b,a).

16 Example: flip 3 coins Recall B =atleastoneh A =exactly2hs p(a) =3/8, p(b) =7/8, and p(a B) =3/7 p(a B) 6= p(a), so the two events are not independent (Since p(a)p(b) =(3/8)(7/8) < 3/8 =p(a, B), they re positively correlated.) HHH HHT HTH HTT THH THT TTH TTT B

$Example: flip 3 coins (cont d) C =atleastoneh and at least one T. C D D =atmostoneh HHH HHT HTH THH HTT THT TTH TTT p(c) =6/8, p(d) =4/8, and p(c \ D) =3/8.$

17 Example: flip 3 coins (cont d) C =atleastoneh and at least one T. C D D =atmostoneh HHH HHT HTH THH HTT THT TTH TTT p(c) =6/8, p(d) =4/8, and p(c \ D) =3/8. Therefore events C and D are independent. B D HHH HHT HTH HTT Whereas p(b)p(d) =(7/8)(1/2) > 3/8 =p(b,d) THH THT TTH TTT so the events B and D are negatively correlated (not surprising for at least one H and atmostoneh ).

18 Example: 4bitnumber E = at least two consecutive 0 s F = first bit is 0. (E \ F = { }) p(e \ F )=5/16, p(f )=8/16, p(e F )=(5/16) (1/2) = 5/8.

19 Example: twochildren E= 2boys,F = at least one boy. p(e F )=1/3 (E =BB,F =BBBGGB). Are the events independent? p(e) =1/4, p(f )=3/4, p(e,f)=1/4 6= 3/16, so they are positively correlated. Example: now3children E= at least one of each sex, F = at most one boy p(e) =6/8, p(f )=4/8, p(e,f)=3/8, so they re independent: p(e F )=p(e) =3/4

20 134 Total for grade 1 Feb 2017 Agriculture and Life Sciences: info: 42, non: 13 Arts and Sciences: info: 27, non: 34 Engineering: info: 0, non: 6 O: info: 4, non: 8 p(info major) = 73 / 134 =.54 Agriculture and Life Sciences [5, 22, 19, 9] Arts and Sciences [0, 28, 21, 12] Engineering [0, 2, 2, 2] O [0, 5, 5, 2] 57 yr=2 (42%) 47 yr=3 (35%) 25 yr=4 (19%) 5 yr=1 (4%) 61 Arts and Sciences (45%) 55 Agriculture and Life Sciences (41%) 11 Business (8%) 6 Engineering (4%) 1 Architecture, Art and Planning (1%) p(info major CALS) = 42 / 55 p(info major A&S) = p(info major ENG) = p(cals info major) =? (calculate directly or via Bayes thm)

21 60 Information Science (44%) 25 Undeclared (19%) 7 Applied Economics and Mgmt (5%) 5 Computer Science (4%) 4 Applied Economics and Mgmt, Information Science (3%) 3 Economics, Information Science (2%) 3 Communication (2%) 2 Psychology (1%) 2 Envr/Sustainability Sciences (1%) 2 Biometry & Statistics (1%) 2 Economics (1%) 2 Communication, Information Science (1%) 2 Oper Research & Engineering (1%) 1 Linguistics, Information Science (1%) 1 Animal Science (1%) 1 Science of Nat & Envir Sys (1%) 1 Philosophy (1%) 1 French (1%) 1 Interdisciplinary Study in ALS (1%) 1 Science & Technology Studies, Information Science (1%) 1 Biology & Society (1%) 1 Architecture (1%) 1 Biological Sciences (1%) 1 Biological Engineering (1%) 1 Electrical and Computer Engr (1%) 1 Agricultural Sciences (1%) 1 Information Science, Psychology (1%) 1 China Asia-Pacific Studies, Information Science (1%)

22 Example: flip a coin 3 times A= 1st flip is H, B = at least two H, C= at least two T Verify that p(a) =p(b) =p(c) =1/2 but the probability 1/2 events can be correlated or uncorrelated p(a, B) =3/8 so A, B positively correlated (makes sense, since 1st being H makes more likely there are at least two H). p(a, C) =1/8 so A, C negatively correlated (again makes sense, since 1st being H makes less likely there are at least two T). p(b,c) = 0, disjoint events (maximally negatively correlated, can t have both two T and two H in three rolls)

23 The notions of disjoint and independent events are very di erent. Two events A, B are disjoint if their intersection is empty, whereas they are independent if p(a, B) =p(a)p(b). Two events that are disjoint necessarily have p(a, B) =p(a \ B) =0 so if their independent probabilities are non-zero, they are necessarily negatively correlated (p(a, B) <p(a)p(b))

24 Bayes Rule A simple formula follows from the above definitions and symmetry of the joint probability: p(a B)p(B) =p(a, B) =p(b,a) =p(b A)p(A): p(a B) = p(b A)p(A) p(b) Called Bayes theorem or Bayes rule connects inductive and deductive inference (Rev. Thomas Bayes (1763), Pierre-Simon Laplace (1812), Sir Harold Je reys (1939)) For mutually disjoint sets A i with S n i=1 A i = S, Bayes rule takes the form p(a i B) = p(b A i )p(a i ) p(b A 1 )p(a 1 )+...+ p(b A n )p(a n ).

25 Example 1: Consider a casino with loaded and unloaded dice. For a loaded die (L), probability of rolling a 6 is 50%: p(6 L) =1/2, and p(i L) =1/10 (i =1,...,5) For a fair die (L), the probabilities are p(i L) =1/6 (i =1,...,6). Suppose there s a 1% probability of choosing a loaded die: p(l) =1/100. If we select a die at random and roll three consecutive 6 s with it, what is the posterior probability, P (L 6, 6, 6), that it was loaded?

26 The probability of the die being loaded, given 3 consecutive 6 s, is p(l 6, 6, 6) = p(6, 6, 6 L)p(L) p(6, 6, 6) = p(6 L) 3 p(l) p(6 L) 3 p(l)+p(6 L) 3 p(l) = = (1/2) 3 (1/100) (1/2) 3 (1/100) + (1/6) 3 (99/100) 1 1+(1/3) 3 99 = /3 = , so only a roughly 21% chance that it was loaded. (Note that the Bayesian prior in the above is p(l) =1/100, giving the expected probability before collecting the data from actual rolls, and significantly a ects the inferred posterior probability.)

1 INFO 2950, 2 4 Feb 10

1 INFO 2950, 2 4 Feb 10 First a few paragraphs of review from previous lectures: A finite probability space is a set S and a function p : S [0, 1] such that p(s) > 0 ( s S) and s S p(s) 1. We refer to S as the sample space, subsets