Problem Set 2 Due Sept, 21

Transcription

1 EE6: Rando Processes in Sstes Lecturer: Jean C. Walrand Proble Set Due Sept, Fall 6 GSI: Assane Guee This proble set essentiall reviews notions of conditional epectation, conditional distribution, and Jointl Gaussian rando variables. Not all eercises are to be turned in. Onl those with the sign are due on Thursda, Septeber at the beginning of the class. Although the reaining eercises are not graded, ou are encouraged to go through the. We will discuss soe of the eercises during discussion sections. Please feel free to point out errors and notions that need to be clarified. Conditional Epectation and distribution Eercise.. A lost tourist arrives at a point with 3 roads. The first road brings hi back to the sae point after hours of walk. The second road brings hi back to the sae point after 6 hours of travel. The last road leads to the cit after hours of walk. There are no signs on the roads. Assuing that the tourist chooses a road equall likel at all ties. What is the ean tie until the tourist arrives to the cit. This eercise uses conditional epectation. Let T the tie it takes to reach the cit, and D i the event the tourist chooses door i for i,, 3. To copute E[T ] we will condition on the door scalper picks at the first tie: E[T ] E[T D ]P (D + E[T D ]P (D + E[T D 3 ]P (D 3 3 (E[T D ] + E[T D ] + E[T D 3 ] ( + E[T ] E[T ] + 3 (. (. where in. we use the fact that after each return to the point, the tourist does not keep eor of its past chooses. Rearranging the ters in the last equation, we obtain Eercise.. E[T ] 9 -

2 EE6 Proble Set Due Sept, Fall 6 Eercise.3. On the da before the ea, each student entering the GSI s office will ask a question that will coe out for the ea with probabilit p. The nuber of student going to office hours that da is Poisson distributed with ean λ. What is the probabilit that the GSI does not have to answer an ea question? The ke idea in this eercise is again conditioning. Let N be the nuber of students who enter the GSI s office on that da (abe net Monda!, and let A be the event that the GSI does not have to answer an ea question. P (A n P (A N np (N n ( p n λn n! ep( λ n ep( λ (( pλ n n! n ep( λ ep(( pλ ep( λp Eercise.4. X, Y, and Z are jointl defined rando variables.. Show that E[X] E[E[X Y ]].. Show that E[Y Z] E[E[Y X, Z] Z]. E[X] E[E[X Y ]] First note E[X Y ] is a function of Y E[E[X Y ]] all integrals are finite E[X] - E[X ]f Y (d ( f X ( d f Y (d ( f X ( f Y (d d ( f X,Y (, d d f X (d

3 EE6 Proble Set Due Sept, Fall 6. E[Y Z] E[E[Y X, Z] Z] First note that E[Y X, Z] is a function of (X, Z. Now Eercise.5. Eercise.6. E[E[Y X, Z] Z] E[h(X, Y Z] f Y X,Z (, zd h(x, Z all integral finite E[Y Z] h(, Zf X Z ( Zd ( f Y X,Z (, Zd f X Z ( Zd ( f Y X,Z (, Zf X Z ( Zd d ( f Y,X Z (, Zd d f Y Z ( Zd Eercise.7. A scalper is considering buing tickets for the net cal football gae. The price of the tickets is $, and the scalper will sell the at $4. However, if he cant sell the at $4, he wont sell the at all. Given that the deand for tickets is a binoial rando variable with paraeters n and p /, how an tickets should he bu in order to aiize his epected profit? Let N be the nuber of tickets that the scalper bus, D be the deand. If S is the total nuber of tickets sold, we have S in(n, D and the scalper s profit R(N is: R(N 4S N Thus E[R(N] E[4S N] 4E[S] N. Note that here we have assued that N is know (we will var it later. So the onl rando ter is S (through D. Let s first copute E[S]. We consider that N because it is obvious that buing ore -3

4 EE6 Proble Set Due Sept, Fall 6 than the aiu deand cannot be optial. So we have: Thus E[S] E[D D N]P (D N + E[b D > N]P (D > N N ( ( ( ( i + N i i i in+ ( ( N ( ( i + N i i r(n E[R(N] 4 i in+ ( ( N ( i i i + N in+ ( N i Intuitivel, r(n increases in N and then decreases. Thus, to find when r(n is aial, we find the first value of N for which r(n + r(n. We will plot r(n and find N in the net version of this handout. Eercise.8.. Jointl Gaussian Rando Variables Eercise.9. Eercise.. Eercise.. John Gae uses his high-speed ode to pla network gaes over the internet. The ode transits zeros and ones b sending signals and +, respectivel. We assue that an given bit has probabilit p of being a zero. The telephone line introduces additive zero-ean Gaussian (noral noise with variance (so, the receiver at the other end receives a signal which is the su of the transitted signal and the channel noise. The value of the noise is assued to be independent of the encoded signal value.. Let a be a constant between and. The receiver at the other end decides that the signal (respectivel, + was transitted if the value it receives is less (respectivel, ore than a. Find a forula for the probabilit of aking an error.. Find a nuerical answer for the question of part (a assuing that p /5, a / and /4. Solution This eercise considered a detection proble. You will learn ore about detection in the net hoeworks. -4

5 EE6 Proble Set Due Sept, Fall 6 An error occurs whenever was transitted and the received signal is ore than a so + N > a, or whenever a + was sent and the received signal is less than a or + N < a where N is the noise. The probabilit of error thus given b: where P (error P (error send P (send + P (error send P (send P ( + N > ap + P ( + N < a( p P (N > a + p + P (N < a ( p P (N > a + p + ( P (N a ( p ( N P > a + ( N p + ( P a ( p ( ( a + a Q p + ( Q ( p ( ( a + a Q p + ( Q ( p ( ( a + a Q p + Q ( p Q( π ep ( z is the copleentar cuulative distribution function (CCDF of a standard noral rando variable (that ou will stud in the net hoework! There is no closed for for Q(, but it is a standard function that ou will be using quite often. We can plug in the values and look at a table to find the error. Will give the values in net version. Eercise.. The Cauch distribution has a for f X ( π( + < <. Show that this arises fro the ratio X \X where X and X are independent zeroean gaussian rando variable with variance.. Show that the oents of X do not eist. To show (, we will copute the pdf of the rando variable Z X X and observe that it -5

6 EE6 Proble Set Due Sept, Fall 6 has a Cauch distribution. We have: ( X F Z (z P (Z z P z X ( X P z f X ( d P (X < z f X ( d Using independence F X (z f X ( d Now we can copute the pdf of Z b taking the derivative of the last epression with respect to z. f Z (z F X (z f X ( d z Since the integral is finite (and everthing is nice! Wh? and z is not involved in the integration, we can safel interchange the order of the derivative and the integral to obtain: f Z (z f X (z f X ( d z π e (z e ( d π e +z d π e +z d π π [ + z + z e ] +z which is the pdf of a Cauch rando variable. To show that the oents do not eist, we will show that k. First observe that But k + for k > and + d li + T log( + T k + d does not eist for an -6

7 EE6 Proble Set Due Sept, Fall 6 Now Thus the functions Eercise.3. Eercise.4. Let k + d d, k + C st + + d k + are not integrable for all k. Hence the oents do not eist. Z Y + Y Z Y Z 3 Y Y for soe independent standard noral rando variables Y and Y.. Is Z (Z, Z, Z 3 T JG?. If es copute its correlation atri. Does it have an inverse? 3. Now ou are given a JG rando vector W with correlation atri: K W Find atri M such that W MV for JG V with detk V. Hint: Use eigenvalue decoposition Since the Z i s are linear cobinations of independent standard noral rando variables, the vector Z is JG. Its correlation atri is: K Z Since deterinant is zero, K Z does not adit an inverse. We will solve (3 for a general K W R n n and let ou verif our result using atlab or an other software. The idea of this eercise is to write an JG rando vector W, with K W, as a function of another JG rando Z that has non-singular correlation atri. If we succeed to do so we can use Z to copute things like conditional ean and pdf given Y, since we can alwas go -7

8 EE6 Proble Set Due Sept, Fall 6 fro Z to W. Using E.V.D, we can write K W QΛQ T for soe orthogonal Q, and diagonal atri ( Λ Λ for soe < n, Λ R, Λ >. Set M Q ( Λ Now we will solve W MZ for Z R. ( Λ W Q Z ( Λ Q T W Z B defining B : (Λ (Λ Q T W (Λ (Λ Q T W Z ( Λ Z I Z Q T, we have Z BW and W MZ. Also K Z E[BW W T B T ] (Λ Thus K Z, and also Z is JG. ( Λ Q T QΛQ T Q I Eercise.5. Let X and X be two independent standard noral rando variables and let Y X + X Y X X Y 3 3X + X Copute the conditional pdf f Y (Y,Y 3 (, 3 of Y given (Y, Y 3 and copute its ean. Hint: Argue that Y, Y, and Y 3 are jointl gaussian and show (Y, Y 3 T AZ and (Y, Z is JG, for soe atri A, and Z with det(k Z. Calculate f Y Z(, K Y,Z, and K Z Reark: In this eercise I did not choose good coefficients to ake the proble interesting. As I proised in discussion sections, I will present another eaple that illustrates better the point I wanted ou to get. -8

9 EE6 Proble Set Due Sept, Fall 6 Eercise.6. Gallager s note: Eercise. (a Note that there is a one-to-one apping between Y and Y 3 eaning that given Y 3 we can uniquel deduce Y. Thus the distribution of X reains the sae given that we have observed Y or Y 3. We can now use forula (.4 of the reader f X Y ( (we replace b 3 because we observe Y 3. f X Y 3( v ( ep ( ρ( / 3 v π( ρ ( ρ (b This part is a little harder because we no longer have the one-to-one apping. However we can appl the technique in part (a if we first condition on the sign of Y : f X Y ( v f X Y ( v, Y > P (Y > Y v + f X Y ( v, Y P (Y Y v The event {Y > } is independent of Y (convince ourself!, so P (Y > Y v P (Y > /. For v >, the events {Y v, Y > } and {Y v} are the sae, so f X Y ( v, Y > ( ep ( ρ( / v π( ρ ( ρ Siilarl, for v the events {Y v, Y } and {Y v} are the sae, so f X Y ( v, Y > ( ep ( + ρ( / v π( ρ ( ρ Thus f X Y ( v ( ep ( ( ρ( / v + ep ( ( + ρ( / v π( ρ ( ρ ( ρ Eercise.7. Gallager s note: Eercise.3 (a For Y AW, E[W Y T ] E[W W T A T ] E[W W T ]A T A T Thus in this case writing K [E[W Y T ] A T, we see that A K T cross-covariance. (b We proceed as in part (a and copute achieves the desired E[W Z T ] E[W W T ]A T K Z B T -9

10 EE6 Proble Set Due Sept, Fall 6 Thus B K T K Z will achieve the desired cross-correlation. (c Fro part (b, we know that we are looking for B such that K Z B T K. Assuing that B T [B... B ] and K [K... K ], let us rewrite K Z B T K in the following for K Z [B... B ] [K... K ] This can be decoposed into a set of linear sstes of equations K Z B i K i, i,...,. Now observe that K Z QΛQ T where Q is orthogonal and Λ has the for λ λ Λ r K Z B i K i can be written as ΛQ T B i Q T K i or in a ore detailed for: λ Q T B i Q T K i.. λ r Q T r B i Q T r K i Q T r+k i.. Q T nk i This tells us that K i, i,..., is orthogonal to Q j, j r +,..., n. In other words, each colun of K is orthogonal to all the eigenvectors that have a zero eigenvalue. This is equivalent to saing that each colun of K belongs V span(q,..., Q r. In conclusion, for B to satisf the desired cross-correlation atri, each colun of K ust belongs to the space spanned b the eigenvectors that correspond to the non-zero eigenvalues of K Z. Copleent.. Proof of the sufficient statistics propert In class we have stated that if g(y is a sufficient statistic for observation Y, then X is independent to Y given g(y. We give a proof of this fact below. The proof is given for X discrete but it can easil be generalized to the continuous case. Consider a detection proble where the observation Y is used to decide on the value of X. We assue that X lies in a discrete set I,..., n and its prior probabilit is given b P (X π(. The conditional distribution of the observation Y given X is given b -

11 EE6 Proble Set Due Sept, Fall 6 f Y X (. fact: If g(y is a sufficient statistic for Y, then X is independent to Y given g(y. Proof: To prove the fact, we will show that f Y (X,g(Y (, g( does not depend on X. First note that since g(y is a sufficient statistic, the conditional distribution of Y given X can be written in the for: f Y X ( F (g(, h( (.3 where F (. depends on Y onl through g(. and h(. is onl a function of. Now we have: f Y (X,g(Y (, g( g f (Y,X,g(Y (,, g( g f (X,g(Y (, g( g (.4 π(f Y X ( [g(g] π( f Y X ( [g( g] F (g(, h( [g(g] F (g(, h( [g( g] F (g, h( [g(g] F (g, h( [g( g] (.5 (.6 (.7 h( [g(g] h( [g( g] (.8 where we go fro.4 to.5 b observing that f (Y,X,g(Y (,, g( g is for an such that g( g and is otherwise equal to f Y X (. In.6, we have used.3 and canceled out the π( s. The ter F (. in the denoinator is not involved in the suation, so we take it out in.7 and it cancels out the corresponding ter in the nuerator and we see that f Y (X,g(Y ( does not depend on... Reake of Eerise.5 You have all noticed that eercise.5 was too obvious. In fact there was an error in the choice of the coefficients and the eercise did not illustrate what it was supposed to illustrate. We give here an eaple that illustrates better the notion. Eaple: Let (X, Y, Y T is a zero-ean jointl gaussian rando vector with correlation atri K Find E[X Y]. In this eaple we see that K Y and we cannot appl the forula given in class E[X Y] K XY K Y Y -

12 EE6 Proble Set Due Sept, Fall 6 The hint in eercise (.5 was to find Z such that Y AZ, (X, Z is JG, and K Z. In eercise (.4 part (3, we have seen one wa to copute A and Z. Here we will give a less foral procedure. First notice that the second row of K Y is a ultiple of the first row. Let s stud the rando variable T Y 3Y. E[T ] E[Y ] 3E[Y ] V ar(t E[(Y 3Y ] E[Y ] + 9E[Y ] 6E[Y Y ] T has zero ean and zero variance thus it is deterinistic and equal to. So we have that Y 3Y 3. Hence Y carries all the inforation in Y (Y, Y (or is a sufficient statistic. Thus E[X Y] E[X Y ] It is not hard to convince ourself that (X, Y is JG and K Y Y. Thus E[X Y ] XY Y Y 3 Y -