CS Homework Week 2 ( 2.25, 3.22, 4.9)

Transcription

1 CS Homework Week 2 ( 2.25, 3.22, 4.9) Dan Li, Xiaohui Kong, Hammad Ibqal and Ihsan A. Qazi Deparmen of Compuer Science, Universiy of Pisburgh, Pisburgh, PA Inelligen Sysems Program, Universiy of Pisburgh, Pisburgh, PA January 20, 2006 Conens 1 Problem Problem Problem This was wrien by Dan Li 1

2 1 PROBLEM Problem 2.25 A blood es is being performed on n individuals. Each person can be esed separaely, bu his is expensive. Pooling can decrease he cos. The blood sample of k people can be pooled and analyzed ogeher. If he es is negaive, his one es suffices for he group of k in es suffices for he group of k individuals. If he es is posiive, hen each of he k person mus be esed separaely and hus k+1 oal ess are required for he k people. Suppose ha we creae n/k disjoin groups of k people(where k divides n) and use he pooling mehod. Assume ha each person has a posiive resul on he es independenly wih probabiliy p. (a) Wha is he probabiliy ha he es for a pooled sample of k people will be posiive? Answer: The resul of he pooled sample is posiive means ha a leas one of he k esed samples has posiive resul, which probabiliy is: 1 - P(all of he k people have negaive sample). Since we assume ha each person has a posiive resul on he es independenly wih probabiliy p, so ha he probabiliy ha each person has negaive resul is 1 p, and he probabiliy ha all of he k persons have negaive resuls is (1 p) k. Finally we have he probabiliy ha he es for he pooled sample of k people is posiive is: (b) Wha is he expeced number of ess necessary? 1 (1 p) k (1) Answer: A leas one es is needed no maer wha he es resul of he pooled sample is. When he resul is posiive, k exra ess are needed. The probabiliy ha k exra ess are needed is: so ha he expeced number of ess for each group of k people is And here are n/k groups, so he oal number of ess is: 1 (1 p) k (2) 1+k [1 (1 p) k ] = 1+k k (1 p) k (3) N(n,k) = n k [1+k k (1 p)k ] = n (1+ 1 k (1 p)k ) (4) (c) Describe how o find he bes value of k. Answer: In order o find he bes value of k, we need o find such a value of k ha he number from (b) reach is minimum value. This is done by he following: N(n, k) k = k n [1+ 1 k (1 p)k ] = n [ 1 k 2 (1 p)k ln(1 p)] = 0 (5) 2

3 1 PROBLEM 2.25 which gives: k 2 (1 p) k 1 = ln(1 p) The value of k can no be solved in close form. (6) (d) Give an inequaliy ha shows for wha values of p pooling is beer han jus esing every individual. Answer: When he number of ess using pooled sample is less ha he number of ess for esing every individual, he pooling mehod is beer. This is obained by having: n[1+ 1 k (1 p)k ] < n 1+ 1 k (1 p)k < 1 1 < (1 p) k k 1 k > ( 1 p )k k 1 k > 1 1 p 1 p > ( 1 k ) 1 k p < 1 ( 1 k ) 1 k (7) 3

4 2 PROBLEM Problem 3.22 Suppose ha we flip coin n imes o obain n random bis. Consider all m = ( n 2) pairs of hese bis in some order. Le Y i be he exclusive-or of he ih pair of bis, and le Y = m Y i be he number of Y i ha equal 1. (a) Show ha each Y i is 0 wih probabiliy 1/2 and 1 wih probabiliy 1/2. Answer: Each Y i is he exclusive-or of wo bis. Assume Y i = x j x k, hen P(Y i = 1) = P((x j = 0 x k = 1) (x j = 1 x k = 0)) = P(x j = 0 x k = 1)+P(x j = 1 x k = 0) = P(x j = 0) P(x k = 1)+P(x j = 1) P(x k = 0) = = 1 2 (8) and P(Y i = 0) = P((x j = 0 x k = 0) (x j = 1 x k = 1)) = P(x j = 0 x k = 0)+P(x j = 1 x k = 1) = P(x j = 0) P(x k = 0)+P(x j = 1) P(x k = 1) = = 1 2 (9) (b) Show ha he Y i are no muually independen. Answer: Muually independen means for every subse, he probabiliy Pr(Y i Y j Y r ) = Pr(Y i ) P(Y j ) P(Y r ) If we choose such a subse ha hose Y i s have facors in common, for example, we choose Y i = x a x b, Y j = x a x c and Y k = x b x c, hen P(Y i = 1 Y j = 1 Y k = 1) = 0 bu P(Y i = 1)P(Y j = 1)P(Y k = 1) = 1 8 They are no equal. So he Y i s are no muually independen. (c) Show ha he Y i saisfy he propery ha E[Y i Y j ] = E[Y i ]E[Y j ]. 4

5 2 PROBLEM 3.22 Answer: E[Y i Y j ] = Pr(Y i Y j = 1) = Pr(Y i = 1 Y j = 1). If Y i and Y j do no have facor in common, i.e. Y i = x a x b and Y j = x c x d, hen Pr(Y i = 1 Y j = 1) = Pr((x a x b = 1) (x c x d = 1)) = Pr(x a = 0)Pr(x b = 1) P(x c x d = 1) +Pr(x a = 1)Pr(x b = 0) P(x c x d = 1) = 1 4 Pr(x c x d = 1)+ 1 4 Pr(x c x d = 1) = 1 2 Pr(x c x d = 1) = 1 4 (10) If Y i and Y j have one facor in common, i.e. Y i = x a x b and Y j = x b x c, hen Pr(Y i = 1 Y j = 1) = Pr((x a x b = 1) (x b x c = 1)) = Pr(x a = 0)Pr(x b = 1)P(x c = 0) +Pr(x a = 1)Pr(x b = 0)P(x c = 1) = = 1 4 (11) While, for any i, E[Y i ] = Pr(Y i = 1) = Pr(x a = 0 x b = 1)+Pr(x a = 1 x b = 0) = = 1 2 (12) So ha E[Y i ]E[Y j ] = 1 4. In any case, he equaliy E[Y i Y j ] = E[Y i ]E[Y j ] holds. (d) Using Exercise 3.15, find Var[Y]. Answer: Using Exercise 3.15, since he above eualiy holds, and Y = m Y i, Var[Y] = m Var[Y i ] = Y 2 i (Yi) 2 Var[Y i ] = Pr(Y i = 1) (Pr(Y i = 1)) 2 = 1 2 (1 2 )2 = 1 4 (13) 5

6 2 PROBLEM 3.22 So, Var[Y] = m/4. (e) Using Chebyshev s inequaliy, prove a bound on Pr( Y E[Y] n). Answer: Using Chebyshev s inequaliy, Pr( Y E[Y] n) Var[Y] n 2 = m/4 n 2 = n 1 8n = 1 8 (1 1 n ) (14) 6

7 3 PROBLEM Problem 4.9 Suppose ha we can obain independen samples X 1, X 2, of a random variable X and ha we wan o use hese samples o esimae E[X]. Using samples, we use ( X i)/ for esimae of E[X]. We wan he esimae o be wihin εe[x] from he rue value of E[X] wih probabiliy a leas 1-δ. We may no be able o use Chernoff s bound direcly o bound how good our esimae is if X is no a 0-1 random variable, and we do no know is momen generaing funcion. We develop an alernaive approach ha requires only having a bound on he variance of X. Le r = Var[X]/E(X). (a) Show using Chebyshev s inequaliy ha O(r 2 /ε 2 δ) samples are sufficien o solve he problem. Answer: Pr( X i / (1+ε)E[X]) = 1 Pr( = 1 Pr( X i / > (1+ε)E[X]) X i > (1+ε)E[X]) (15) and E( Var( Using Chebyshev s Inequaliy, and wrie Y = X i, Pr( X i ) = E(X i ) = E(X) (16) X i ) = Var(X i ) = Var(X) (17) X i > (1+ε)E[X]) = Pr(Y > E(Y)+εE(Y)) = Pr(Y E(Y) > εe(y)) Pr( Y E(Y) > εe(y)) Var(Y) (εe(y)) 2 = Var( X i) (εe(x)) 2 = Var(X) 2 ε 2 E(X) 2 = r2 ε 2 (18) 7

8 3 PROBLEM 4.9 As long as r 2 /(ε 2 δ), we have Pr( X i / (1+ε)E[X]) = 1 Pr( X i > (1+ε)E[X]) = 1 r2 ε 2 1 δ (19) So he number of esimaes needed is: r 2 /(ε 2 δ) = O(r 2 /ε 2 δ). Bu, if we have O(r 2 /ε 2 δ) samples, i does no guaranee he probabiliy of 1 δ. (b) Suppose ha we need only a weak esimae ha is wihin εe[x] of E[X] wih probabiliy a leas 3/4. Argue ha O(r 2 /ε 2 ) samples are enough for his weak esimae. Answer: Probabiliy of 3/4 means δ = 1/4. By seing δ = 1/4 in O(r 2 /ε 2 δ), we have O(4r 2 /ε 2 ) = O(r 2 /ε 2 ). (c) Show ha, by aking he median of O(log(1/δ)) weak esimaes, we can obain an esimae wihin εe[x] of E[X] wih probabiliy a leas 1-δ. Conclude ha we need only O((r 2 log(1/δ))/ε 2 ) samples. Answer: If he median of he weak esimaes saisfies he condiion, i means less han half of he weak esimaes are no wihin εe[x] of he rue value of E[X]. Le s use a new random variable X i : { 1 if he ih weak esimae fall above εe(x) of E(X) X i = 0 if he ih weak esimae fall below εe(x) of E(X) X i follows binomial disribuion wih probabiliy of 1/4 or more o be 1 and 3/4 or less o be 0. For simpliciy, we use 1/4 in his problem. Lower probabiliy will need lower number of esimaes. If we use X = m X i o represen how many weak esimaes fall above (1+ε)E(X), we will be able o use Chernoff bound o for he value of m so ha Pr(X >= m/2) < δ. Chernoff bound gives: where E(X) = m/4. Use δ = 1, we have Pr(X (1+δ )E(X)) e E(X)δ 2 /3 Pr(X m/2) e m/12 By using m = 12 log(1/δ), we have Pr(X m/2) δ, so ha he probabiliy ha he median of weak esimaes gives resul wihin εe(x) is a leas 1 δ. Each weak esimae uses O(r 2 /ε 2 ) samples, and here are O(log(1/δ)) weak esimaes so ha he oal number of samples is O(r 2 log(1/δ)/ε 2 ). 8