Pr[A B] > Pr[A]Pr[B]. Pr[A B C] = Pr[(A B) C] = Pr[A]Pr[B A]Pr[C A B].

Transcription

1 CS70: Jean Walrand: Lecture 25. Product Rule Product Rule Causality, Independence, Collisions and Collecting 1. Product Rule 2. Correlation and Causality 3. Independence Birthdays 6. Checksums 7. Collecting Coupons Recall the definition: Hence, Consequently, Pr[B A] = Pr[A B]. Pr[A] Pr[A B] = Pr[A]Pr[B A]. Pr[A B C] = Pr[(A B) C] = Pr[A B]Pr[C A B] = Pr[A]Pr[B A]Pr[C A B]. Theorem Product Rule Let A 1,A 2,...,A n be events. Then Pr[A 1 A n ] = Pr[A 1 ]Pr[A 2 A 1 ] Pr[A n A 1 A n 1 ]. Proof: By induction. Assume the result is true for n. (It holds for n = 2.) Then, Pr[A 1 A n A n+1 ] = Pr[A 1 A n ]Pr[A n+1 A 1 A n ] = Pr[A 1 ]Pr[A 2 A 1 ] Pr[A n A 1 A n 1 ]Pr[A n+1 A 1 A n ], so that the result holds for n + 1. Correlation An example. Random experiment: Pick a person at random. Event A: the person has lung cancer. Event B: the person is a heavy smoker. Fact: Conclusion: Pr[A B] = 1.17 Pr[A]. Smoking increases the probability of lung cancer by 17%. Smoking causes lung cancer. Correlation Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A B] = 1.17 Pr[A]. A second look. Note that Pr[A B] = 1.17 Pr[A] Conclusion: Pr[A B] = 1.17 Pr[A] Pr[B] Pr[A B] = 1.17 Pr[A]Pr[B] Pr[B A] = 1.17 Pr[B]. Lung cancer increases the probability of smoking by 17%. Lung cancer causes smoking. Really? Causality vs. Correlation Events A and B are positively correlated if Pr[A B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples: Tesla owners are more likely to be rich. That does not mean that poor people should buy a Tesla to get rich. People who go to the opera are more likely to have a good career. That does not mean that going to the opera will improve your career. Rabbits eat more carrots and do not wear glasses. Are carrots good for eyesight?

2 Proving Causality Proving causality is generally difficult. One has to eliminate external causes of correlation and be able to test the cause/effect relationship (e.g., randomized clinical trials). Some difficulties: A and B may be positively correlated because they have a common cause. (E.g., being a rabbit.) If B precedes A, then B is more likely to be the cause. (E.g., smoking.) However, they could have a common cause that induces B before A. (E.g., smart, CS70, Tesla.) More about such questions later. For fun, check N. Taleb: Fooled by randomness. Example 2 Independence Recall : A and B are independent Pr[A B] = Pr[A]Pr[B] Pr[A B] = Pr[A]. The intuition is that A does not say anything about B. This intuition is a bit misleading. See next slide. Mutual Independence Example 1 Flip two fair coins. Let A = first coin is H = {HT,HH}; B = second coin is H = {TH,HH}; C = the two coins are different = {TH,HT }. A, C are independent; B, C are independent; A B,C are not independent. (Pr[A B C] = 0 Pr[A B]Pr[C].) If A did not say anything about C and B did not say anything about C, then A B would not say anything about C. Mutual Independence Flip a fair coin 5 times. Let A n = coin n is H, for n = 1,...,5. Then, A m,a n are independent for all m n. Also, A 1 and A 3 A 5 are independent. Indeed, Pr[A 1 (A 3 A 5 )] = 1 8 = Pr[A 1]Pr[A 3 A 5 ]. Similarly, A 1 A 2 and A 3 A 4 A 5 are independent. This leads to a definition... Definition Mutual Independence (a) The events A 1,...,A 5 are mutually independent if Pr[ k K A k ] = Π k K Pr[A k ], for all K {1,...,5}. (b) More generally, the events {A j,j J} are mutually independent if Pr[ k K A k ] = Π k K Pr[A k ], for all finitek J. Example: Flip a fair coin forever. Let A n = coin n is H. Then the events A n are mutually independent. Theorem (a) If the events {A j,j J} are mutually independent and if K 1 and K 2 are disjoint finite subsets of J, then k K1 A k and k K2 A k are independent. (b) More generally, if the K n are pairwise disjoint finite subsets of J, then the events k Kn A k are mutually independent. (c) Also, the same is true if we replace some of the A k by Āk. Proof: See homework.

3 One throws m balls into n > m bins. One throws m balls into n > m bins. Theorem: Pr[no collision] exp{ m2 }, for large enough n. Theorem: Pr[no collision] exp{ m2 }, for large enough n. Theorem: Pr[no collision] exp{ m2 }, for large enough n. In particular, Pr[no collision] 1/2 for m 2 /() ln(2), i.e., E.g., m 2ln(2)n 1.2 n. Roughly, Pr[collision] 1/2 for m = n. (e ) The Calculation. A i = no collision when ith ball is placed in a bin. Pr[A i A i 1 A 1 ] = (1 i 1 n ). no collision = A 1 A m. Product rule: Pr[A 1 A m ] = Pr[A 1 ]Pr[A 2 A 1 ] Pr[A m A 1 A m 1 ] ( Pr[no collision] = 1 1 ) ( 1 m 1 ). n n Hence, ln(pr[no collision]) = m 1 k=1 = 1 n ( ) We used ln(1 ε) ε for ε 1. ( ) m 1 = (m 1)m/2. ln(1 k m 1 n ) ( k n ) ( ) k=1 ( ) m(m 1) m2 2 Approximation exp{ x} = 1 x + 1 2! x x, for x 1. Hence, x ln(1 x) for x 1.

4 Sum of consecutive integers Today s your birthday, it s my birthday too.. Checksums! Recall this useful fact: Probability two of n people have the same birthday? With n = 365, one finds Pr[collision] 1/2 if m If m = 60, we find that Pr[not collision] exp{ m2 602 } = exp{ } Given two random files, What are the odds they have same checksum? Let n = 2 b be the number of checksums. Let m be the number of files. How big should b be to avoid any collisions? If m = 366, then Pr[no collision] = 0. (No approximation here!) Checksum For b-bit checksums for m files, we claim that Pr[collision] 1 if b 2.9lnm E.g., for m = files, a 103-bit checksum suffices! Derivation: b bits n = 2 b bins. Pr[collision] 2 10 Pr[no collision] exp{ m2 } m m m n m b m b 2log 2 (m) + 9 = ( ) 2lnm/(ln2) lnm + 9. An aside. Secure checksums. Random plus security. Want: cannot find two files that hash to the same bucket. Checksum of virus program should checksum of real program. MD5: broke. SHA-1: broke. SHA-2: so far... CS161 Coupon Collector Problem. n different baseball cards. (Brian Wilson, Jackie Robinson, Roger Hornsby,...) One random baseball card in each cereal box. How many cereal boxes do you have to buy to get Brian Wilson card... with probability at least 1 2? ( ) log 2 (x) = ln(x)/ln(2). Indeed: ln(x) = log 2 (x)ln(2) since e log 2 (x)ln(2) = [e ln(2) ] log 2 (x) = 2 log 2 (x) = x.

5 Coupon Collector Problem. Event A m = fail to get Brian Wilson in m cereal boxes Fail the first time: (1 1 n ) Fail the second time: (1 1 n ) And so on... for m times. Hence, Pr[A m ] = (1 1 n ) (1 1 n ) = (1 1 n )m. Analyze expression. Pr[A m ] = (1 1 n )m When is p m = Pr[A m ] 1 2? Taylor s formula: when x is small. Hence, e x = 1 x + x 2 2! x 3 3! 1 x. p m = (1 1 n )m (e 1 n ) m = e m n = e ln(1/p m). After m = n ln 1 p m cards, we fail to get a Brian Wilson card with probability p m. For p m = 1 2, we need around n ln2 0.69n boxes. Collect all cards? Experiment: Choose m cards at random with replacement. Events: E k = fail to get player k, for k = 1,..., n Probability of failing to get at least one of these n players: p := Pr[E 1 E 2 E n ] How does one estimate p? Union Bound: p = Pr[E 1 E 2 E n ] Pr[E 1 ] + Pr[E 2 ] Pr[E n ]. Plug in and get Pr[E k ] e m n,k = 1,...,n. p ne m n. Collect all cards? Bittorrent. Using codes! Thus, Pr[missing at least one card] ne m n. Hence, Pr[missing at least one card] p when m n ln( n p ). To get p = 1/2, set m = n ln(). E.g., n = 10 2 m = 530;n = 10 3 m = If file is split into n pieces. Each server has random piece. Ask n ln() servers to get file with probability 1 2. Use Reed-Solomon codes (or Tornado codes 1 ). Encode n pieces into pieces. Any n pieces ok (n + n pieces ok with Tornado codes.) How many requests to get n different pieces with failure probability at most p? More complicated. Random variables, distributions. But at most: (ln2) + O( n) is good enough. Much better than n ln(). E.g., for n = 100, around a factor of 4 better! 1 Linear time decoding!

6 Summary. Causality, Independence, Collisions and Collecting Main results: Product Rule Correlation Causality : m balls into n > m bins. Pr[no collisions] exp{ m2 } Coupon Collection: n items. Buy m cereal boxes. Pr[miss one specific item] e m n ; Pr[miss any one of the items] ne m n. Key ideas: ln(1 ε) ε;e ε 1 ε; product rule; union bound.