Problem Set 3 Due Oct, 5

EE6: Random Processes in Systems Lecturer: Jean C. Walrand Problem Set 3 Due Oct, 5 Fall 6 GSI: Assane Gueye This problem set essentially reviews detection theory. Not all eercises are to be turned in. Only those with the sign are due on Thursday, October 5 th at the beginning of the class. Although the remaining eercises are not graded, you are encouraged to go through them. We will discuss some of the eercises during discussion sections. Please feel free to point out errors and notions that need to be clarified. Eercise 3.. We have seen in class that a convenient way to write the error probability in Gaussian detection is the use of the Q-function defined as: Q ep{ t π }dt In this eercise we will study some properties of the Q-function: a Show that Q Q b Show that e π e Q π Hint: rewrite the integrand multiply and divide by t and use integration by parts. Solution: awe will use the definition and a change of variable. Q π change of variable e t dt π π e t + dt + e t dt π e t dt + + π Q e t dt + 3-

EE6 Problem Set 3 Due Oct, 5 Fall 6 bfor the upper and lower bounds we will use integration by parts. We will drop the π term in the calculations. e t t t t e [ u t, v te t e t ] + t t t e dt 3. second term e For the upper bound we will perform another integration by parts in the second term of equation 3.. e t e u t, 3 v te t e + t t t e ] t [ e + t 3 t dt + t t 4 e dt 3 rd term e e 3 Eercise 3.. The output of a system is normal N.,. if the input X is equal to. When the input X is equal to, the output is normal N.,.. Observing the output of the system, we would like to compute our best estimate of X i or estimate of i. Find the best estimate of i, given the output of the system best in term of maimizing the probability of X i given the system output. We assume that the input is X with probability.6. The solution is given in the 6 notes Eample 8.6. Eercise 3.3. The Binary Phase-shift Keying BPSK and Quadrature Phase-shift Keying QPSK modulation schemes are shown in figure 3.. We consider that in both cases, the symbols S are sent over an additive gaussian channel with zero mean and variance σ. Assuming that the symbols are equally likely, compute the average error probability for each scheme. Which one is better? Note that the comparison is not fair because the two schemes do not have the same rate eggs and apples!. But let us compute the error probabilities and compare them. The BPSK error probability is the one given in eample 3. of Gallager s notes ML detection with bgal a and agal a. Thus the error probability is given by P bpsk e Q a σ 3-

EE6 Problem Set 3 Due Oct, 5 Fall 6 QPSK BPSK S s -a a S S -a a S S3 Figure 3.. BPSK and QPSK modulations For QPSK, consider that signal S was sent and observe that error occurs if the received signal does not fall in the first quartan. By considering the probability of detecting any other signal, we can see that the error probability is equal to P qpsk e Q a σ + Qa σ For a large enough we have P qpsk e P bpsk e Eercise 3.4. Given X i, i,, Z is an eponential random variable with mean λ i. Give the MAP decision rule of X given the observation Z. What is the probability of error? For the MAP assume that X with probability p. Solution given in 6 notes eample 8.6. Eercise 3.5. The conditional pmf of Y given X is given by the table below. X Y + +.3.7.6.4 Find the MAP and MLE of X given Y. For the MAP assume that X with probability p. Solution given in 6 notes eample 8.6.7 Eercise 3.6. The purpose of a radar system is to detect the presence of a target and etract useful information about the target. Suppose that in such a system, hypothesis H is that there is no target present, so that the observation is s w, where w is a normal random variable with zero mean and variance σ. Hypothesis H corresponds to the presence of the 3-3

EE6 Problem Set 3 Due Oct, 5 Fall 6 target and the observation is s a + w where a is an echo produced by the target and is assumed to be known. Evaluate: The probability of false alarm defined as the probability that the receiver decides a target is present when it is not. The probability of miss detection defined as the probability that the receiver decides a target is not present when it is. Write down the decision rule: H if y < a/ and H otherwise and compute the errors. Eercise 3.7. Suppose that we want to send at the same time two independent binary symbols X {, } and X {, } over an additive Gaussian channel, where and + are equally likely. A clever communication engineer proposes a transmission scheme that uses twice the channel to send at each time a linear combination of X and X. The channel input output relation is given by: Y h h X Z + h h Y where Y and Y are the channel outputs, the h i, i, are known real constant, and Z i, i, are iid standard normal random variables. We are interested in detecting both X and X i.e. the vector X, X T. a Show that the vector detection problem can be decomposed into two scalar detection problems. Hint: observe that the rows of the matri H are orthogonal. b Give the decision rule for detecting X and write the error probability using the Q- function. Solution: The scheme presented in this eercise in the Alamouti scheme well known in communication systems mainly in multi-antenna communications. a Observing that the rows and columns of the matri H are orthogonal, let us project the received signal on the directions H h, h T and H h, h T. We have: X Z Y H T Y, h + h Y h + h + h z + h z Y H T Y h + h, Y h + h + h z h z Thus we have written the vector detection problem into two scalar detection problems Y i α i + w i, i, where w i N, h + h. To be sure that the problem can be decoupled, 3-4

EE6 Problem Set 3 Due Oct, 5 Fall 6 we need to check that the w i s are uncorrelated. [E[w w ] E[h z + h z h z h z ] h h E[z ] h E[z z ] + h E[z z ] h h E[z ] h h + h h Thus w and w are uncorrelated. Since they are gaussian, they are also independent. Hence the vector detection problem can be decoupled into independent scalar detection problems. b We can use the same technique as in eample 3. of the lecture notes. We decide ˆ if y < and ˆ if y >. Since the two symbols are equally likely, the error probability is equal to the probability of error given that was sent. Given was sent, Y N h + h, h + h P e P N h + h, h + h > P N, > h + h Q h + h Eercise 3.8. In this eercise we consider a binary detection problem like in eample 3. where n independent observations Y i a+z i are available. Assume that the Z i, i,..., n are iid N, σ. Hypothesis H and H are mapped to a and a + respectively. Find the decision rule for the ML and the MAP detectors. Hint: find a sufficient statistic first. Convince yourself that the sample mean Ȳ n n i Y i is a sufficient statistic and that the decision rule is given by: H if Ȳ < and H otherwise. Now compute the distribution of Ȳ and deduce the error probabilities. Eercise 3.9. We observe n independent realizations Y,..., Y n of a random variable Y which distribution depends on another variable X. If X resp., then Y is an eponential random variable with mean resp.. We assume that X takes the value with probability p,. Compute the optimal detector of X given the Y i s optimal in terms of maimizing the probability of X given the observation Y n Y,..., Y n. What is this detector when p /. Solution given in eample 8.6.8 of 6 notes. 3-5

EE6 Problem Set 3 Due Oct, 5 Fall 6 Eercise 3.. Consider a modified version of Eample 3. of the course note Gallager where the source output is no longer or but takes values in the set S {,,..., I}. If the source output is i S, the modulator produces the real vector a i a i,..., a in. Given that the source output is i the observation is Y a i + Z where the noise is assumed to be N, I n. a Show that the ML decision rule is given by î argmin i Y a i where. is the Euclidean distance in R n. b For n, use a one dimensional sufficient statistics to compute the decision rule. c Compare your decision rule to the one in a. Eplain. Solution: a The ML decision point is the one that maimizes the likelihood or conditional distribution. It is given by î argma i f Y H y i argma i ep π n Y a iy a i T, By def. of the cond. dist argma i Y a iy a i T, Taking log... argmin i Y a i Y a i T argmin i Y a i Thus the decision point is the one that minimizes the euclidian distance between the receive signal Y and the data symbols a i. b A one-dimensional sufficient statistic when n is Θ a a T Y. This is the inner product between Y and the direction of the signal. Given that S i, i, was sent, Θ N a a T a i, a a. Now we can compute the LLRΘ which is equal to LLRΘ Θ a a T a + a a a T y a a T a + a a a T y a + a We will then decide that Ŝ S whenever LLRΘ. To understand this decision rule, note first y a +a is just changing the origin from, vector to a +a to simplify we assume that a a. Think of the aes to be changed also from the usual ones to a a the direction of the signal and a a the direction perpendicular to the signal. Let y αa a + βa a in the new coordinate system. 3-6

EE6 Problem Set 3 Due Oct, 5 Fall 6 Taking the inner product a a T y a a T αa a + βa a αa a T a a + βa a T a a α Our decision rule becomes the simple test of whether α is greater then a a T a +a. Geometrically, we have reduced our vector detection problem to a one-dimensional detection problem where the detection is along the direction of the signal. Important remarks Our decomposition was possible because the noise was gaussian, uncorrelated and unit variance. If the noise is still gaussian but correlated with arbitrary non singular covariance matri, we need to re-scale the received signal before the projection. This is the job of the term K in the general sufficient statistic a a K Z Y. 3 The case where the noise is gaussian with singular covariance matri is discussed in Problem 3. 4 For a general distribution of the noise, there is no such result as far as I know!...this gives an idea why Gaussian s are important for us. c It is not hard to verify that the two decision rules are the same they have to be!. For that, note that in part a we decide that S S if y a < y a y a T y + a < y a T y + a a a T y a a < a a T y a a T a + a < a a T y a + a which is the decision in part b. The method in part b is quite intuitive and brings us back to the simple scalar detection case but it is applicable only for binary detection. If there are more than hypothesis, only the method in part a can be used. Eercise 3.. Gallager s note: Eercise 3. This eercise was just verification of results presented in the Galalger s note. a Given H, an error occurs if LLR Y. Write the error probability to get 3.3. b Similar to part a. c E[U] E[ b a T Y ] b a T E[ Y ] 3-7 <

EE6 Problem Set 3 Due Oct, 5 Fall 6 Thus given H E[U] b a T a and given H E[U] b a T b The variances are given by: varu H σ b a varu H Now using the decision rule given by 3.7, we easily get 3.3. d Again U is a Gaussian random variable with mean and variance given above. Just write the ratio of the conditional distribution and take the log. e Since U is one dimensional, we can now apply 3.8 and get 3.3 again. The whole point of this eercise was to study properties of the sufficient statistic. The message to be taken is that the sufficient statistic is easier to work with since it is usually of lower dimension than the original variable, it has the same LLR and leads to the same decision rule as the observation. Eercise 3.. Gallager s note: Eercise 3. Parts a, b, c are very similar to what we did in the previous problem. The only tricky part is d. d As it has been noticed in previous homework, if the correlation matri K Z is singular, then the noise lies in a lower dimensional space. If b a is entirely in that space, then the problem can be reduced to a lower dimensional estimation problem. If some component of b a is not in the space, then that component will be received without noise. By observing that particular component, we can always detect without error. Eercise 3.3. Gallager s note: Eercise 3.5 a Observe that given H, Y N a, AAT and given H, Y N b, AA T. The log-likelihood ratio is then given by: LLRY [ b a T A T A Y + aaa T a baa T b ] [ b a T A T v + aaa T a baa T b ] which shows that the dependency of the LLR to Y, is only through A Y v. Thus v is a sufficient statistic. b Observe that given H, v N A a, I and given H, v N A b, I. Now compute the LLR of verify that it is equal to the one computed in a. c Note that the noise is uncorrelated and we can apply the techniques used in eample 3. 3-8

EE6 Problem Set 3 Due Oct, 5 Fall 6 of Gallager s notes. After some calculation steps, we obtain: P re H Q logη γ + γ where γ A b a Note: The goal of this eercise was to study the whitening filter. Whenever the noise covariance matri can be written as K Z AA T with A a nonsingular matri this is always the case if K Z is not singular, then the detection problem Y a i + Z is equivalent to Ỹ A a i + W where Ỹ A Y and W is a zero mean, uncorrelated N, I in case of gaussian noise. The process of multiplying by A is called the whitening filter. You will see more of it later in the course. Eercise 3.4. Gallager s note: Eercise 3.9 Solution: Conditioned on H, Y, Y and Y 3, Y 4 are independent. Each of these twodimensional distributions is circularly symmetric in the plane, so the conditional density of Y, Y, Y 3, Y 4 given H must depend on Y, Y, Y 3, Y 4 only through Y + Y, and Y3 +Y4. Since this is also true for the conditional density given H, Y +Y, Y3 +Y4 V, V must be sufficient statistic. By the symmetry of the transmitted signal and the noise, f V,V Hv, v f V,V Hv v We also epect that f V,V Hv, v > f V,V Hv, v whenever v > v proving this is technical, and not required. Then the decision rule must be the one stated. 3-9

EE6 Problem Set 3 Due Oct, 5 Fall 6 Technical stuff follows: f Y H y π π π f Y,φ H y, θ dθ f Y H y f φ θdθ π ep y acosθ + y asinθ + y 3 + y 4 π π σ ep 4 σ π σ ep y + y + y3 + y4 + a π 4 σ π σ ep y + y + y3 + y4 + a 4 σ π a y + y π σ 4 ep π σ 4 ep cosθ arctany σ /y y + y + y3 + y4 + a π σ y + y + y 3 + y 4 + a σ dθ π ep ay cosθ + y sinθ/σ dθ dθ π ep a y + y cosψ dψ σ I a y + y σ where I is the modified Bessel function of the first kind and th order. Similarly, f Y H y π σ ep y + y + y3 + y4 + a I 4 σ a y3 + y4 σ Since I z is an increasing function of z, the ML is of the stated form. Another way to be convince yourself is to compute the LLR. We know that given H i, i,, Y N, Σ i where: Σ a + σ a + σ σ σ The pdf of Y given H is given by: f Y H y Σ π σ a + σ e σ σ a + σ a + σ y +y a +σ + y 3 +y 4 σ We can also write f Y H y and see that it depends only on v and v. Now we can write the LLR and find that it is LLRY a σ a + σ v v 3-

EE6 Problem Set 3 Due Oct, 5 Fall 6 We will thus choose H whenever v > v and H otherwise. b Given H, V Y + Y, the distribution of which we found in Eercise.: f V Hv Similarly V Y + Y has distribution. a + σ ep v/a+ σ, v f V Hv σ ep v/σ, v c Let U V V compute P U > u H. Conditioning on V we have P U > u H V ind. V { a +σ P V > u + v H, V vf V Hv P V > u + v H f V Hv ep σ +a a +σ u σ ep u, σ +a σ u < Thus the pdf is given by: f U H u { ep u, u σ +a a +σ ep u, u < σ +a σ d Given H, an error occurs whenever U <. The corresponding probability is P U < H P U H u σ + a ep σ udu σ σ + a + a /σ e By symmetry of the transmitted signals and the noise, this is also the overall error probability. 3-