ECE 302: Probabilistic Methods in Engineering

Transcription

1 Purdue University School of Electrical and Computer Engineering ECE 32: Probabilistic Methods in Engineering Fall 28 - Final Exam SOLUTION Monday, December 5, 28 Prof. Sanghavi s Section Score: Name: No calculators/computers/handhelds of any sort are allowed. No books/notes are allowed except four A-4 size sheets (both sides) is allowed. Note: after the exam, you may want to save this sheet. Please show all of your work. Answers without justification are subject to receiving no credit. If you need extra space, you may use the back of the previous exam page. The exam is 2 minutes long. Good luck!. (6 pts.) 2. (26 pts.) 3. (2 pts.) 4. (3 pts.) 5. (8 pts.) Total: ( pts.)

2 Problem. (6 points) This question is multiple-choice, and you do not need to justify your answers. There will be a negative one point penalty for incorrect answers; blank answers receive no points; correct answsers receive two points apiece. True False P(A B) max{p(a), P(B)}. True: Since A A B and B A B P(A B) min{p(a), P(B)}. False: Consider two nonempty disjoint sets A and B. Then P(A B). P(A B) + P(A B c ). False: P(B A) + P(B c A) P(A B)P(B) + P(A c B)P(B) P(A). False: P(A B)P(B) + P(A c B)P(B) P(A B) + P(A c B) P(B) If P(A) P(B) then P(A B) P(B A). True: P(A B) P(B A)P(A)/P(B) P(B A) If P(A B) P(B A) then P(A) P(B). True: By Bayes rule P(A) P(B)P(A B)/P(B A) P(B). If P(A B) P(A) then P(B c A) P(B). True: P(A B) P(A) A and B are independent P(B c A) P(B c ) P(B) P(A B) + P(A B) P(A) + P(B). True: Since P(A B) P(A) + P(B) P(A B) 2

3 Problem 2. (26 points) The joint PDF of two random variables X and Y is given by { x + y, if < x < and < y < f X,Y (x, y), else. (a) (3 points) Find the marginal PDF f X (x). Be sure to specify its value for all x. Solution: We have that f X (x) f XY (x, y)dy. Thus, f X (x) is zero for x and x. For x (, ), we have f X (x) (x + y)dy x + 2. (b) (4 points) What is E[X]? What is V ar[x]? Solution: E[X] E[X 2 ] xf X (x)dx x 2 f X (x)dx ( x x + ) dx V ar[x] E[X 2 ] (E[X]) ( x 2 x + ) dx (c) (3 points) Find the conditional PDF f Y X (y x). Be sure to specify its value for all x and y. Solution: We have that f Y X (y x) f XY (x, y) f XY (x, y)dy By symmetry, we may use part (a) to obtain f XY (x, y)dy x + 2. Thus, f Y X (y x) x + y x + /2, for x, y (, ), and zero otherwise. 3

4 (d) (6 points) Find the MMSE estimate, ŷ MMSE (x), for all real numbers x such that < x <. Solution: ŷ MMSE (x) E[Y X] (e) (5 points) Find Cov(X, Y ). symmetry.) yf Y X (y x)dy y(x + y)dy x + /2 x/2 + /3 x + /2 3x + 2 6x + 3 (Hint: note that E[X] E[Y ] by Solution: We have tha Cov(X, Y ) E[XY ] E[X]E[Y ], where E[X] E[Y ] was found in part (b). We then compute E[XY ] xyf XY (x, y)dxdy xy(x + y)dxdy ( y 3 + y ) dy Therefore Cov(X, Y ) ( ) (f) (5 points) What is the LMMSE estimator ŷ LMMSE (x), for all real numbers < x <? Solution: We have that ŷ LMMSE (x) E[Y ] + Cov(X, Y ) (X E[X]) V ar[x] 7 ( ) 2 ( 44 )(X 7 2 ) 7 X 44 4

5 Problem 3. (2 points) A bag contains n pairs of shoes; each pair is a different style. (a) You pick two random shoes from the bag (Note: this is sampling without replacement). i) (4 points) What is the probability that the two shoes you picked out are a pair; i.e. left and right of the same style. Solution: You pick any shoe, followed by the only remaining shoe that forms a pair: probability is ()( ) 2n ii) (4 points) What is the probability you picked one left shoe and one right shoe? Solution: You pick any shoe, followed by any of the n remaining shoes which is for the opposite foot: probability is ()( n ). 2n (b) You pick a third shoe without returning the first two. i) (6 points) What is the probability of having a pair of shoes among the three you picked? Solution: Let us first compute the probability, P(no pair), of not having a pair among the three shoes. In this case, you pick a shoe, leaving 2n 2 of the 2n shoes which won t give a pair. You draw one of these, and then must in turn draw one of the 2n 4 of the 2n 2 remaining shoes which don t give a pair for either shoe you have. Thus, P(no pair) ()( 2n 2 2n )(2n 4) 2n 4, and 2n 2 2n P(pair) P(no pair) 2n 4 2n 3 2n. We may also solve by using the results of (a)i. Since (a)i showed the probability of obtaining a pair in two picks, we use this to show the probability of obtaining a pair in three. We have that given that no pair was obtain on the first pick (happens with probability ) that we get 2n a pair on the 3rd pick with probability 2 (since there 2n 2 are 2 shoes out of the 2n 2 remaining which would yeild a pair). We have then that ( P(pair in 3) 2n + ) 2 2n 2n 2 3 2n 5

6 ii) (6 points) What is the probability there is at least one left and at least one right shoe among the three? Solution: We again compute the probability of the compliment. To have all three shoes be for the same foot, we would draw any shoe, followed by one of the n of the 2n shoes which are for the same foot. Finally, draw one of the n 2 of the 2n 2 which are for the same foot. i.e. P(same foot) ()( n 2n )( n 2 2n 2 ) n 2 4n 2 ( ) n 2 P(one left or one right) P(same foot) 4n 2 3n 4n 2. We may also solve this using (a)ii. Since (a)ii showed the probability of obtaining a left and right in two picks, we use this to show the probability of obtaining a left and right in three. We have that given that we two right or two left after the first pick (happens with probability n ) that we get a left and right on the 3rd pick with 2n probability n (since there are 2 shoes out of the 2n 2 2n 2 remaining which would yeild the match we need). We have then that n P(one left or one right) 2n + ( 3n ( n n (2n )(2n 2) ) n 2n 2n 2 ) 3n 4n 2 6

7 Problem 4. (3 points) n couples attend a morally ambiguous party. They are separated and then randomly paired up; i.e. each man is now paired with a randomly chosen lady. As a result, it is possible some couples rejoin each other. For each couple, i, let X i be a random variable such that X i if the i-th couple rejoins, and X i otherwise. Note that: i) X, X 2,...,X n are not independent; ii) E[X i ] probability that the i-th couple re-forms. (a) (3 points) Find E[X i ]. Show your reasoning. Solution: As in the hint, we have E[X i ] P(i re-forms). The man from i may be paired with any of the n ladies, so that E[X i ] n. (b) (3 points) Let X X + X X n be the total number of reformed couples. Find E[X]. Solution: [ ] E[X] E X i E[X i ] n (c) (6 points) Using the result in part (b), find an upper bound on P(X > 7). Solution: Since P(X i < ), the Markov inequality applies. By Markov, we have P(X > 7) P(X 8) 8 E[X] 8 (d) (6 points) Find E[X i X j ]. Note that X i and X j are not independent. (Hint: X i X j if both couples i and j are rejoined, and otherwise). Solution: We have that E[X i X j ] P(i, j re-join)+ P(i or j do not re-join) P(i, j re-join). i and j re-join if first (order is arbitrary) couple i is paired together with probability leaving n couples left to n 7

8 form, so that then couple j is re-joined with probability n. Therefore E[X i X j ] P(i, j re-join) n(n ) (e) (6 points) Find V ar[x]. Note that X i and X j are not independent. (Hint: (X X n ) 2 i X2 i + 2 n i j X ix j ) Solution: V ar[x] E[X 2 ] (E[X]) 2 E[X 2 ], so that we need solve for E[X 2 ]. We have ( ) 2 [( E[X 2 ] E X i E X i)( j E[X i X j ]. j X j )] Now, we note that E[X i X j ] as above only if i j. n(n ) If i j, then E[X i X j ] E[Xi 2]. Thus, we have n E[X 2 ] E[Xi 2 ] + n + E[X i X j ] j i j i n(n ) 2. Therefore, V ar[x] E[X 2 ]. (f) (6 points) Using the results in parts (b) and (e), find a better upper bound on P(X > 7). (Hint: P(X > 7) P( X > 6)). Solution: We have by the Chebyshev inequality that P(X > 7) P( X > 6] V ar[x]

9 Problem 5. (8 points) X is an exponential random variable. Under hypothesis H, it has parameter λ 5 and under hypothesis H, it has parameter λ. The prior probabilities for the two hypotheses are P λ (λ ) 3, P λ(λ ) 2 3. (a) (4 points) What is the Maximum Likelihood (ML) decision rule for this problem? Solution: The two liklihood functions f X H (x H ) and f X H (x H ) are plotted below. We see, then, that f X H (x H ) and f X H (x H ) H 6 p(x H i ) H.386 x intersect at the point where 5e 5x e x x ln(2)/5. Thus the ML decision rule is to decide H if x > ln(2)/5 and H if x ln(2)/5. (b) (4 points) What are the probabilities of Type-I and Type-II errors for the ML rule of part (a)? Solution: By definition, a Type-I error is deciding H when H is true, while Type-II is the reverse. Thus, the probability of a Type-I error is defined as P(decide H H ) P(X ln(2)/5 H ) ln(2)/5 5e 5x dx e 5ln(2)/5 2, while the probability of a Type-II error is P(decide H H ) P(X > ln(2)/5 H ) ln(2)/5 e x dx e ln(2)/5 4. 9

10 (c) (4 points) What is the MAP decision rule? Solution: The MAP estimation rule reduces to choosing the larger of f X H (x H )P(H ) and f X H (x H )P(H ). We see that f X H (x H )P(H ) f X H (x H )P(H ) 3 5e 5x 2 3 e x x ln(4)/5. Thus we decide H if x ln(4)/5 and H if x < ln(4)/5. (d) (6 points) What is the overall probability of error P(err) under the MAP rule? Note that you CANNOT re-use the Type-I and Type-II errors you calculated for the ML rule. Solution: We have that the overall probability of error follows P(err) P(H )P(decide H H ) + P(H )P(decide H H ) 3 ln(4)/5 5e 5x dx ln(4)/5 3 ( e 5ln(4)/5 ) (e ln(4)/5 ) ( ) ( ) e x dx