UC Berkeley, CS 174: Combinatorics and Discrete Probability (Fall 010 Solutions to Problem Set 4 1. (MU 5.4 In a lecture hall containing 100 people, you consider whether or not there are three people in the room who share the same birthday. Explain how to calculate this probability exactly, using the same assumptions as in our previous analysis. There are a number of ways to count this value, so several solutions were accepted. One such solution is given here. Let A 0 be the event that everyone in the room has a distinct birthday. Let A i be the event that i pairs of people share a birthday. Then the probability we are interested in is P[at least 3 people share a birthday = 1 P[A 0 P[A i (1 where there are n people in the room. First, we note that P[A 0 is in the book and is P[A 0 = (365 n n! 365. Second, we wish to compute P[A n i. For i = 1, we have ( n 365 1 P[A 1 = 365 ( n (n! For i = we have ( 365 ( n n ( 365 P[A = ( n 4 (n 4! This generalizes for i pairs of people sharing i distinct birthdays P[A i = ( 365 n ( i ( n j+ n/ ( 365 i n i (n i! Plugging everything into Equation 1 and putting in n = 100, we can compute a value for this probability. We get P[at least 3 people share a birthday 0.6459.. (MU 5.8 Our analysis of Bucket sort in Section 5.. assumed that n elements were chosen independently and uniformly at random from the range [0, k. Suppose instead that n elements are chosen independently from the range [0, k according to a distribution with the property that any number x [0, k is chosen with probability at most a/ k for some fixed constant a > 0. Show that, under these conditions, Bucket sort still requires linear expected time. Recall from the proof in section 5.. that the expected sorting time for all the buckets is at most n n E c(x j = c E[Xj. We will now use the probability that any bin is chosen with probability at most a/ k. The number of elements any bin are upper bounded by a random variable Y Bin(n, a/n. To upper bound 1
the expected time for sorting, we see that c n E[Xj cne[y [ ( a ( a = cn n(n 1 + n (by p48 of book n n [( = cn 1 1 a + a n cn(a + a. This is linear-time sorting, since c and a + a are both constants. 3. (MU 5.9, a and b Consider the probability that every bin receives exactly one ball when n balls are thrown uniformly at random into n bins. Let X i be the event that the i th bin has one ball, and let X be the event that all the bins have a single ball: X = n X i. (a Using a Poisson approximation, we find an upper bound. variable for bin i, and recall that m = n Let Y i be a Poisson random P[Y i = j = e n/n ( n n j j! P[Y i = 1 = 1 e Applying the constant factor for the Poisson approximation (Cor. 5.9, we can bound the probability of X using Y = n {Y i = 1}. P[X e np[y = e ( 1 n n. e (b Now, we determine the exact probability of this event. There are n n ways to throw n balls into n bins. There is only one way, given a specific ordering of the balls, to put a single ball into each bin. And there are n! possible orderings of the balls. Thus the exact probability of n balls in n bins is: P[X = n! n n. 4. (MU 5.11 The following problem models a simple distributed system wherein agents contend for resources but back off in the face of contention. Balls represent agents, bins represent resources. The system evolves over rounds. Every round, balls are thrown independently and uniformly at random into n bins. Any ball that lands in a bin by itself is served and removed from consideration. The remaining balls are thrown again in the next round. We begin with n balls in the first round, and we finish when every ball is served.
(a If there are b balls in a stage, the probability that a particular ball lands in a bin by itself is (1 1/n b 1. By linearity of expectation, the expected number of balls that get served is b(1 1/n b 1, and the expected number of those that remain is b b(1 1/n b 1. (b Suppose that every round the number of balls served was exactly the expected number of balls to be served. Show that all the balls would be served in O(loglogn rounds. We use the previous part together with the assumption that in each round the expected number of balls are served. To obtain the second line below, notice also that (1 x k 1 kx for k 0. x j+1 = x j (1 (1 1/n x j 1 x j (1 (1 (x j 1/n = x j (x j 1/n x j/n. Notice from the exact formula in part (a that a constant number of rounds i suffices to reduce n balls to n/ balls. By recursively expanding the above relation, we see that x i+t x t i /n t 1 (n/ t /n t 1 n/ t. This value is 1 when t = O(log log n. Since i is constant, the total number of rounds is i + t = O(log log n. 5. (MU 5.13, a and b Let Z be a Poisson random variable with mean µ where µ 1 is an integer. (a Show that P[Z = µ + h P[Z = µ h 1 for 0 h µ 1. P[Z = µ + h P[Z = µ h 1 e µ µ µ+h (µ + h! µ µ+h (µ + h! e µ µ µ h 1 (µ h 1! µ µ h 1 (µ h 1! µ h+1 (µ + h! (µ h 1! µ h+1 (µ h(µ h + 1...(µ 1µ(µ + 1...(µ + h 1(µ + h Now multiply pairs of the coefficients in the product on the right hand side: (µ h(µ + h,..., (µ 1(µ+1. We see that there are h of these new coefficients (µ (h i(µ+(h i = (µ (h i, and certainly we have the inequality (µ (h i µ. So we see that the inequality does hold for h 0 since µ h+1 (µ h (µ (h 1...(µ 1µ µ h (µ h (µ (h 1...(µ 1. 3
(b Using part (a, we can see that P[Z µ 1/. P[Z µ = = P[Z = h h=µ P[Z = µ + h µ 1 P[Z = µ + h µ 1 P[Z = µ h 1 = P[Z < µ by part (a 6. (MU 5.15 Since P[Z < µ + P[Z µ = 1 and P[Z µ P[Z < µ from above, we have P[Z µ 1/. We consider another way to obtain Chernoff-like bounds in the setting of balls and bins without using Theorem 5.7. Consider n balls thrown randomly into n bins. Let X i = 1 if the ith bin is empty and 0 otherwise. Let X = n X i. Let Y i, i = 1,..., n, be independent Bernoulli random variables that are 1 with probability p = (1 1/n n. Let Y = n Y i. (a Show that E[X 1 X X k E[Y 1 Y Y k for any k 1. For the X i s, the product is 1 only if the first k bins are empty. So there are (n k n ways to throw n balls into n k bins. And there are n n ways to throw n balls into n bins. So, we have (n kn E[X 1 X X k = n n = (1 k/n n. For the Y i s, since they are independent, we simply have the product of k binomial random variables with p = (1 1/n n. This is E[Y 1 Y Y k = (1 1/n kn. From the book, we know that 1 k/n (1 1/n k for all positive integers n, k. So we have E[X 1 X X k E[Y 1 Y Y k. (b Show that E[e tx E[e ty for all t 0. (Hint: use the expansion for e x and compare E[X k to E[Y k. Using the Taylor series for e x and linearity of expectation, we can write the MGF of X and Y as: E [ e tx = E [ e ty = 4 i=0 i=0 t i i! E[Xi t i i! E[Y i
Now, we want to compare term-wise, the MGF of X and the MGF of Y. This means that we want to show E[X i E[Y i. Let Z = n Z i be an indicator random variable which is the sum of independent and identically distributed indicators Z i. For example, both X and Y satisfy the properties of Z. Then ( n j E[Z j = E Z i. After expanding out the jth power, we get a sum of terms where each term is a product of various Z i s to powers C i, i.e. Z c i 1 Zc Zcn n. Since the Z i are indicators, each term simplifies to n k Z c i 1 Zc Zcn n = Z I{c i 0} i = Z i, where I{c i 0} indicates the event that c i is positive, and k = n I{c i ne0}. Notice that the last equality follows from the Z i being independent and identically distributed. Notice from this and by application of part (a to each term, we can write ( n j [ n E[X j = E X i = E = [ k E X i [ k E Y i = E[Y j. c=(c 1,...,c n X c i i This proves that for a particular term, i, in the MGF of X and the MGF of Y, the inequality holds. Which means we have shown that E[e tx E[e ty. (c Derive a Chernoff bound for P [X (1 + δe[x. Using Markov s inequality and part (b, we get P[X x = P[e tx e tx E[etX e tx E[et Y e tx. The problem ask for us to set x = (1 + δe[x, and we know that the MGF of Y is upperbounded by e E[Y (et 1 (page 64. Now, we have P[X (1 + δe[x ee[y (et 1 e t(1+δe[x By the problem set-up, we know that E[X = E[Y, since E[X = E[X i = (1 1/n n = E[Y. Because of this, we write Setting t = ln(1 + δ we get ( e (et 1 P[X (1 + δe[x e t(1+δ E[Y ( e δ E[Y P [X (1 + δe[x (1 + δ (1+δ. 5