CS 330 Discussion - Probability

Transcription

1 CS 330 Discussio - Probability March Fudametals of Probability 11 Radom Variables ad Evets A radom variable X is oe whose value is o-determiistic For example, suppose we flip a coi ad set X = 1 if we get heads, ad X = 0 if we get tails The, X is a radom variable sice its value is ot kow prior to the flip The set of values X ca take ad their associated probabilities is referred to as a probabilistic distributio Variables ca be discrete (eg the result of a die) or cotiuous (eg the distace betwee two objects) For this course, we focus o discrete distributios We ca defie a evet E as somethig that may or may ot happe, ad the discuss the probability it happes as P r[e] Whe we discuss radom variables, we may describe the probability they take o certai values usig this otatio For example, P r[x = x] sigifies the probability of the evet X = x We ca defie evets which are fuctios of other evets as well For example, give two evets A, B the evet where both A ad B happe ca be writte as A B (the itersectio, or logical AND) The evet where at least oe of A, B happes is A B (the uio, or logical OR) We ca also defie the complemet of a evet, A C A C is the evet where A does ot happe Note that: P r[a C ] = 1 P r[a] (A B) C = A C B C (A B) C = A C B C P r[a B] P r[a] + P r[b] (this is the uio boud, ad geeralized to multiple evets) P r[a B] = P r[a] + P r[b] P r[a B] (ca be prove pictorially via Ve diagram) We ca also defie evets coditioally o other evets already havig occurred For example, give that we rolled a eve umber o a die, what s the 1

2 probability we rolled a 6? The coditioal probability of B give A is writte as P r[b A] More formally, P r[b A] = P r[b A] P r[a] Some coditioal probabilities ca be very clealy defied i terms of each other Note that P r[b A]P r[a] == P r[a B] = P r[a B]P r[b] The, P r[a B]P r[b] P r[b A] = P r[a] This is Bayes rule For example, if we kow the probability a drug-usig idividual tests positively for drugs, if a test has false positives determiig the probability someoe who tests positive has actually take drugs is t easy However, if we also kow the probability a idividual tests positive overall ad the probability a idividual has take drugs, Bayes rule lets us relate the two quatities easily 12 Mutual Exclusio ad Idepedece Two evets are said to be mutually exclusive if oe happeig meas the other caot happe For example, rollig a 1 ad rollig a eve umber o a die are mutually exclusive evets Formally, A ad B are mutually exclusive if P r[a B] = 0 Two evets are idepedet if kowig whether oe happeed or ot does ot give us ay additioal kowledge as to whether the other happeed or ot For example, while kowig that I rolled a eve umber o a die gives us more ifo about whether I rolled a 6 or ot, kowig that I flipped a coi ad got heads tells me othig about whether I rolled a 6 or ot Formally, A ad B are idepedet if P r[a B] = P r[a] or equivaletly, P r[a B] = P r[a]p r[b] The latter equatio exteds to multiple idepedet evets That is, P r[e 1 E 2 E ] = i P r[e i] if all E i are idepedet I geeral, two evets which have o effect o each other are idepedet So, if we have may evets we kow occur idepedetly, we ca calculate the probability of the joit set of evets occurrig by just multiplyig each of the idividual evets probabilities of occurrig 13 Expected Values For a radom variable X, the expected value ca be deoted as E[X] The expected value ca be thought of as the average outcome Formally, E[X] = x P r[x = x] x For example, if I flip a fair coi ad set X = 1 if the coi gets heads ad X = 0 otherwise, E[X] = 5 We ca also write expected values coditioally - for example, the expected value of a die roll give I got a eve umber The expected value of X coditioed o A happeig is E[X A] For some set of outcomes of some evet E 1 E ad some associated radom variable, E[X] = i P r[e i]e[x E i ] Some radom variables ca be writte as the sum of other radom variables For example, suppose I flip cois, ad deote X 1, X 2 X to be 1 if the ith coi lads heads ad 0 if it lads tails Suppose X is a radom variable which couts the umber of heads I flip I ca write X = X 1 + X 2 X Whe a radom variable is the sum of other radom variables, its expected value is the 2

3 sum of their expected values ie X = i X i E[X] = i E[X i] This is kow as liearity of expectatio It holds eve whe the X i are depedet o each other, makig it a very strog property 2 The Geometric Distributio Suppose we flip a coi that lads heads with probability p ad tails otherwise, util it lads heads How may flips will this take? If X is the umber of flips it takes, the i order for X to equal i, we eed to flip i 1 tails (if we flip heads before the ith flip, we stop) ad the 1 heads So, P r[x = i] = (1 p) i 1 p for all positive itegers i A radom variable with this distributio is said to be geometrically distributed - the set of probabilities is referred to as a geometric distributio Formally, X Geo(p) The geometric distributio is useful i that it lets us predict how may times we may have to repeat some trial util it succeeds For example, i radomized algorithms we might ru a while loop util some coditio is If the chace of that coditio beig met o each iteratio is costat, the umber of iteratios the while loop takes is geometrically distributed Note that P r[x > b] = (1 p) b (we make more tha b flips if the first b flips produce tails) The expected value of the geometric distributio is 1 p Ituitively this makes sese, but we ca also prove it quite simply With probability p, o the first flip, we get heads ad so X = 1 Otherwise, if we flip tails, the umber of remaiig flips is still E[X], so ow X s coditioal expected value is (1 + E[X])) I other words, E[X] = P r[x = 1] E[X X = 1] + P r[x > 1] E[X X > 1] = p 1 + (1 p)(1 + E[X]) This is solved by E[X] = 1 p 3 Probability Problems 31 Coupo Collector We re drawig coupos from a machie, ad there are types of coupos I oe draw, a coupo is picked at radom ad each type of coupo has the same probability to be picked What is the expected umber of draws to get all types of coupos? Let X i be the radom variable represetig the umber of draws to get the i the distict coupo after we have got (i 1) distict coupos We have X i follows a geometric distributio ad the probability of success for each trial is: The expectatio of X i is p i = i + 1 E[X i ] = 1 p i = i + 1 3

4 Let X be the umber of draws to get all coupos The we have 32 Birthday Paradox X = X 1 + X 2 + X E[X] = E[X i ] i=1 E[X] = [ ] Θ( log ) There are k people ad each perso has a radom birthday from days How large should k be such that the chace that two people have the same birthday is at least α Let X be the evet that every perso has a distict birthday The ith perso has ( i+1) optios to have a distict birthday Thus, the probability to have a distict birthday is i+1 The: P r[x] = 1 2 k + 1 P r[x] = 1(1 1 )(1 2 ) (1 k 1 ) Calculus shows us e x 1 x for small x The: P r[x] e ( k 1 + ) P r[x] e k(k 1) The the miimum k eeded is roughly that which solves e k(k 1) = 1 α 33 Balls i Bis Suppose we have balls ad m bis, ad we toss the balls uiformly at radom ito the bis This is called a balls ad bis process, ad is a popular model i practice For example, hashig algorithms ca use a balls ad bis process to aalyze collisios For this sectio, let X i be the umber of balls i ith bi after all balls are throw 331 Balls to fill all bis How large should be before all bis have a ball i expectatio? This is a restatemet of the coupo collector problem, so we get = Θ(m log m) 4

5 332 E[X i ] What is E[X i ]? Note that sice all bis are equivalet from the perspective of throwig balls, E[X i ] is the same for all i The, ad thus m E[X i ] = i=1 E[X i ] = m 333 Most filled bi How large is the most filled bi i expectatio? That is, what is E[max i X i ]? This problem is particularly iterestig for hashig i that it asks what s the expected worst case rutime of checkig a bucket i the hashtable For simplicity, assume there are also bis (that is, m = ) I additio, we will oly try to upper boud the expected value The probability a fixed bi has more tha k balls is the probability that some set of k balls all laded i this bi There are ( k) such sets, so by uio boud, this is ( ) P r(x i k) ( 1 k )k The operatio ( k) is the choose operatio It computes the umber of subsets of size k we ca choose from a set of size ( )! = k k!( k)! Where i! = 1 2 i A useful iequality kow as Stirlig s approximatio bouds ( ) k ( ( ) k )k ( e k k )k The: P r(x i k) ( e k )k ( 1 )k = ( e k )k The, max i X i is at least k if ay X i is at least k: By uio boud: P r(max i X i k) = P r( i=1x i k) P r(max i X i k) ( e k )k 5

6 We kow that max i X i The, we ca upper boud E[max i X i ] as follows: E[max i X i ] k P r(max i X i < k) + P r(max i X i k) 2 ( e k )k + k The first iequality is arrived at by takig the expected value formula ad icreasig all values less tha k to k, ad all other values to The secod is arrived at by boudig the first probability by 1, ad the secod as show above We will the choose k i order to remove the first half of the sum Asymptotically, this will be the same as if we had optimized for k 2 ( e k )k 1 ( e k )k 1 2 ( kk e k ) 2 k k poly() k log k = O(log ) This happes to be solved by k = O( log log log ) k log k = log log log log( log log log ) = log (log log log log log) log log log log log is vaishigly small, so: The, sice 2 ( e k )k 1: log k log k = Θ( log log ) = Θ(log ) log log E[max i X i ] k P r(max i X i k)+ P r(max i X i k) 2 ( e log k )k +k = 1+O( log log ) = O( log log log ) 6