Designing Information Devices and Systems I Spring 2018 Homework 11

Transcription

1 EECS 6A Designing Information Devices and Systems I Spring 28 Homework This homework is due April 8, 28, at 23:59. Self-grades are due April 2, 28, at 23:59. Submission Format Your homework submission should consist of two files. hw.pdf: A single PDF file that contains all of your answers (any handwritten answers should be scanned) as well as your IPython notebook saved as a PDF. If you do not attach a PDF of your IPython notebook, you will not receive credit for problems that involve coding. Make sure that your results and your plots are visible. hw.ipynb: A single IPython notebook with all of your code in it. In order to receive credit for your IPython notebook, you must submit both a printout and the code itself. Submit each file to its respective assignment on Gradescope.. Cauchy-Schwarz Inequality The Cauchy-Schwarz inequality states that for two vectors x, y R n : x, y = x T y x y Use the Cauchy-Schwarz inequality to verify (i.e. prove or derive) the triangle inequality: Hint: Start with x + y 2. x + y x + y We consider the Euclidean 2-norm here. By Cauchy-Schwarz inequality, we have x, y x 2 y 2 Therefore, y, x y 2 x 2 x + y 2 2 = ( x + y) T ( x + y) = x y x, y + y, x x y y 2 x 2 = ( x 2 + y 2 ) 2 Taking square root on both sides, we get x + y 2 x 2 + y 2 EECS 6A, Spring 28, Homework, Last Updated: :5

2 2. Mechanical Correlation (a) Calculate and plot the autocorrelation (the inner products of one period of the signal with all the possible shifts of one period of the same signal) of each of the above signals. Each signal is periodic with a period of (one period is shown). You may use ipython for this question, please use the notebook provided to you for all work Autocorrelation of s : EECS 6A, Spring 28, Homework, Last Updated: :5 2

3 Autocorrelation of s 2 : (b) Calculate and plot the cross-correlation (the inner products of one period of the first signal with all possible shifts of one period of the second signal) of the two signals. Each signal is periodic with a period of (one period is shown) s with s 2 : EECS 6A, Spring 28, Homework, Last Updated: :5 3

4 s 2 with s : Audio File Matching Lots of different quantities we interact with every day can be expressed as vectors. For example, an audio clip can be thought of as a vector. The series of numbers in the clip determines the sounds we hear. An audio segment or a sound wave is a continuous function of time, but this can be sampled at regular intervals to make a discrete sequence of numbers that can be represented as a vector. This problem explores using inner products to measure similarity. The ideas here are similar to the themes of the Acoustic Positioning System in the lab. EECS 6A, Spring 28, Homework, Last Updated: :5 4

5 Let us consider a very simplified model for an audio signal, one that is just composed of two tones. One is represented by and the other by +. A vector of length N makes up the audio file. (a) Say we want to compare two audio files of the same length N to decide how similar they are. First, consider two vectors that are exactly identical x = [ and x2 = [. What is the dot product of these two vectors? What if x = [ and x2 = [ (where the length of the vector is an even number)? Can you come up with an idea to compare two general vectors of length N now? The dot product u v = N i= u iv i is the same thing as the Euclidean Inner Product. The dot product of x = [ and x2 = [ is x x 2 = N. The dot product of x = [ and x2 = [ is x x 2 = when the vector length is even. To compare two vectors of length N composed of and, we first take the dot product of the two vectors. The larger the magnitude of the dot product, the more similar the two vectors are. The smaller the magnitude of the dot product, the more dissimilar the two vectors are. This is related to the ideas of correlation that you will see later on. In many circumstances, a dot product with a very large negative value would mean the vectors are very different, but it turns out that humans are unable to perceive the sign of sound, so two sounds vectors x and x sound exactly the same. As a result, for this problem we are interested in is the absolute value of the dot product, but in many other problems, we will interpret a large negative dot product as very different vectors. Don t take off points in parts (a), (b), or (c) if you didn t mention the absolute value. (b) Next suppose we want to find a short audio clip in a longer one. We might want to do this for an application like Shazam, which is able to identify a song from a signature tune. Consider the vector of length 8, x = [. Let us label the elements of x so that x = [ x x 2 x 3 x 4 x 5 x 6 x 7 x 8. We want to find the short segment y = [ in the longer vector, i.e. we want to find i, such that the sequence represented by [ x i x i+ x i+2 is the closest to y. How can we find this? Applying the same technique, what i gives the best match for y = [? For each sub-sequence of length 3 of x, say x i = [ x i x i+ x i+2 (the sub-sequence which starts at position i), take the dot product of x i and y. The sub-sequences x i with the largest dot product magnitude with y gives us the closest match with y. For y = [, the list of i with maximum dot product between x i and y are {2,5} with each dot product being 3. For y = [, every dot product is, so there is no good match. We can also implement the 6 dot products in one matrix multiplication. For instance, if x = [ x x 2... x 8 and y = [ y y 2 y 3, the dot products (or correlation) zi = x i y can be represented as: x y y 2 y 3 x 2 z y y 2 y 3 x 3 z 2 y y 2 y 3 x 4 y y 2 y 3 x 5 = z 3 z 4 y y 2 y 3 x 6 z 5 y y 2 y 3 x 7 z 6 x 8 EECS 6A, Spring 28, Homework, Last Updated: :5 5

6 We then pick the z i with the largest magnitude (there can be multiple as we ve just seen) and the corresponding x i gives us the required substrings. (c) Now suppose our vector is represented using integers and not just by and. Say we want to locate the sequence closest to y = [ 2 3 in x = [ What happens if you apply the technique of part (b)? How would you modify this technique for the problem here? Applying the technique in part (b), we get the best match to be [ as this has the largest dot product with y = [ 2 3. One way to modify the previous approach is by considering how close the directions of [ 2 3 and any sub-sequence of length 3 of x are. The unit vector in the direction of [ 2 3 is yu = [ ] T 4 2 3, and the unit vector in the direction of any sub-sequence of length 3 xi = [ x i x i+ x i+2 is then given by u i = [ x i xi x i+ x i+2. The unit vector ui, which has the largest dot product magnitude with y u (i.e. u i y u ), is the one most aligned with the vector y. Thus, in our example for x = [ [ and y = 2 3, the unit vector of the sub-sequence x = [ 2 3 has the largest dot product ( u y u = ). What are other interesting ways of achieving this? (d) Answer part (d) in the provided IPython notebook. See sol.ipynb. (e) Answer part (e) in the provided IPython notebook. See sol.ipynb. 4. Finding Signals in Noise Disclaimer: This problem looks long. However, almost all of the parts involve only running the provided IPython code and commenting on its output. In this problem, we will explore how to use correlation and least squares to find signals, even in the presence of noise and other interfering signals. (a) Suppose that there is a transmitter sending a known signal s, which is periodic with length N =. Say s is chosen to be a random {+, } vector (for example, by tossing a coin N times and replacing every heads with + and every tails with ). For convenience, let us also normalize s, so that s has a norm of. That is, the vector s looks like: where the ± entries are chosen randomly. s = n [ + +, We claim that such a vector s is approximately orthogonal to circular shifts of itself. That is, if s ( j) denotes circularly shifting s by j, then for all j : s, s ( j) ε for some small epsilon. EECS 6A, Spring 28, Homework, Last Updated: :5 6

7 Run the provided IPython code to generate a random s and plot its autocorrelation (all inner products with shifted versions of itself). Run this a few times, for different random vectors s. Around how small are the largest inner products s, s ( j), j? Recall that we normalized s, so s, s =. After running the provided code a few times, you should have observed that the individual inner products s, s ( j), j are usually less than.6 and that the maximum magnitude of the inner product s, s ( j) for all j is usually around.. (You could have done this by looking at the maximum of the autocorrelation plot at indices j.) Therefore, we can let Let us also define ε =. ε 2 =.6 as a bound on the deviation of an individual inner product. (It is fine to just use ε as a bound instead of ε 2, but we will carry ε 2 through to be slightly more precise.) Note that s, s ( j) for j is much smaller than s, s =. (b) Suppose we receive a signal y, which is s delayed by an unknown amount. That is, for some unknown shift j. To find the delay, we can choose the shift j with the largest inner product s ( j)., y Run the provided IPython code to plot the cross-correlation between y and s (i.e. all the shifted inner products). Can you identify the delay? Briefly comment on why this works using the findings from the previous part. Hint: What does s (k), y look like, for k = j? For k j? Running the provided code, we see that the autocorrelation peaks at index k =. That is, the inner product s (k), y is maximum for the shift k =. Thus, we identify the delay as j =. This works because the inner product s (k), y = s (k) j) will usually be small (at most ε) if k j (from the previous part) but will be exactly if k = j. Thus, the maximum index k will usually identify the correct shift. Note: We used the word usually here as a technicality since it is possible, though unlikely, that our random choice of s happens to have high correlation with its shifted versions. For example, it is possible that our random choice resulted in s = [ ] N, i.e. the normalized all-ones vector. However, this is unlikely to occur random vectors are usually almost orthogonal to their shifted selfs, as you saw in the previous part. We will omit the usually for the rest of the solutions for the sake of readability. (c) Now suppose that we receive a slightly noisy signal: +. n, where the noise source n is chosen to be a random normalized vector, just like s. EECS 6A, Spring 28, Homework, Last Updated: :5 7

8 Run the provided IPython code to compute s, n. Run this a few times for different random choices of s,n. Around how small is s, n? How does this compare to s, s ( j) from your answer in part (a)? After running the provided code a few times, you should have observed that s, n is usually around.3 and almost always less than.6 = ε 2. This is similar to how the inner products s, s ( j), j looked in the previous parts. That is, the inner product of a random signal with its shifted self is roughly the same as the inner product of a signal with random noise. (d) Can we identify the delay from this noisy reception? In this case, we do not know the noise n, and we do not know the delay j. (But, as before, we know that the signal s being transmitted). Run the provided IPython code to plot the cross-correlation between y and s. Briefly comment on why this works to find the delay using the findings from the previous part. Yes, we can identify the delay in this case, as demonstrated by the provided code. This works roughly because, for a fixed k j, the inner product s (k), y = s (k) j) +. n = s (k) j) + (.) s (k), n ε 2 ± (.)ε 2 is small. In fact, all the inner products for k j are small; the maximum over all k j is roughly bounded by: max s (k), y = max s (k) j) +. n k j k j ( ) = max s (k) j) + (.) s (k), n k j max s (k) j) + (.)max s (k), n k j k j ε + (.)ε, where we used our findings from the previous parts in the last step. On the other hand, for k = j, the inner product s (k), y = s ( j) j) +. n j) = + (.) is large. (e) What if the noise is higher? For example: s ( j) ± (.)ε 2 + n s ( j), n Does cross-correlation still work to find the delay? (Use the provided IPython notebook). EECS 6A, Spring 28, Homework, Last Updated: :5 8

9 What about very high noise? + n Does cross-correlation still work to find the delay? If not, can you explain why? (use findings from previous parts). Noise with the same amplitude as the signal ( + n) still works, as seen in the IPython notebook. We expect this since, similar to the rough calculations from the previous part, the maximum correlation for all k j: is still small compared to max s (k), y ε + ()ε.2 k j s ( j), y. However, very high noise ( y = s ( j) + n) no longer works there is no clear autocorrelation peak. Again, we expect this because the maximum correlation for k j is now roughly: max k j s (k), y ε., which is larger than s ( j), y. Thus, it is likely that some inner product than the correct inner product s ( j)., y In other words, the noise masks the signal in this case. s (k), y,k j is greater (f) Now suppose that there are two transmitters, sending known signals s and s 2 at two unknown delays. That is, we receive + s(k) 2 for unknown shifts j,k. Both signals s and s 2 are chosen as random normalized vectors, as before. We can try to find the first signal delay by cross-correlating y with s (as in the previous parts). Similarly, we can try to find the second signal delay by cross-correlating y with s 2. Run the provided IPython code to estimate the delays j,k. Does this method work to find both delays? Briefly comment on why or why not. Yes, this method works in this case, as demonstrated by the provided code. We may expect this because when we try to find the delay of s by computing inner products s (k),, y the signal s 2 looks like noise. In fact, s 2 is chosen as an independent random normalized vector, just like n in the previous part. Thus, this works for exactly the same reason that the medium noise case ( + n) worked in the previous part. (g) Now, suppose the second transmitter is very weak, so we receive: +. s(k) 2 Does the method of the previous part work reliably to find signal s? What about s 2? (Run the provided code a few times to test for different choices of random signals). Briefly justify why or why not. Hint: s looks like noise when we are trying to find s 2. Based on the previous parts, would you expect to be able to find s 2 under such high noise? EECS 6A, Spring 28, Homework, Last Updated: :5 9

10 This still works to find s since the signal s 2 looks like random noise with a very low amplitude. However, this no longer works reliably to find s 2 for the same reason that the very high noise case in part (e) failed. That is, when we are trying to find s 2, the signal s appears as random noise, at times the amplitude of s 2. Thus, as in part (e), we may expect that the noise masks the signal in this case. (h) To address the problem of the previous part, suppose that we use the following strategy: First, crosscorrelate to find the delay j of the strongest signal (say, s ). Then, subtract this out from the received y, to get a new signal y = y s ( j). Then cross-correlate to find the second signal in y. Run the provided IPython code to test this strategy for the setup of the previous part (with a strong and weak transmitter). Does it work? Briefly comment on why or why not. Yes, this new strategy works to find both signals. The received signal is: +. s(k) 2 If we know the coefficient of s exactly ( in this case) and we find the shift of s correctly, then when we subtract the contribution of s, we find y = y s ( j) =. s (k) 2, which only contains signal s 2. Thus, we can cross-correlate it to find its shift. 5. Homework Process and Study Group Who else did you work with on this homework? List names and student ID s. (In case of homework party, you can also just describe the group.) How did you work on this homework? I worked on this homework with... I first worked by myself for 2 hours, but got stuck on problem 5, so I went to office hours on... Then I went to homework party for a few hours, where I finished the homework. EECS 6A, Spring 28, Homework, Last Updated: :5