Last Updated: 218-11-19 21:23 1 EECS 1A Fall 218 Designing Information Devices and Systems I Discussion 13A 1. Ohm s Law With Noise We are trying to measure the resistance of a black box. We apply various i test currents and measure the ouput voltage v test. Sometimes, we are quite fortunate to get nice numbers. Oftentimes, our measurement tools are a little bit noisy, and the values we get out of them are not accurate. However, if the noise is completely random, then the effect of it can be averaged out over many samples. So we repeat our test many times: Test i test (ma) v test (V) 1 1 21 2 3 7 3 1 2 4 5 8 5 8 15 5 11 (a) Plot the measured voltage as a function of the current. v test i test Notice that these points do not lie on a line! UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 1
Last Updated: 218-11-19 21:23 2 1 21 3 7 (b) Suppose we stack the currents and voltages to get I = 1 5 and V = 2 8. Is there a unique 8 15 5 11 solution for R? What conditions must I and V satisfy in order for us to solve for R uniquely? We cannot find the unique solution for R because V is not a scalar multiple of I. In general, we need V to be a scalar multiple of I to be able to solve for R exactly (another linear algebraic way of saying this is that V is in the span of I). We know that the physical reason we are not able to solve for R is that we have imperfect observations of the voltage across the terminals, V. Therefore, now that we know we cannot solve for R directly, a very pertinent goal would be to find a value of R that approximates the relationship between I and V as closely as possible. Let s move on and see how we do this. (c) Ideally, we would like to find R such that V = IR. If we cannot do this, we d like to find a value of R that is the best solution possible, in the sense that IR is as close to V as possible. We are defining the sum of squared errors as a cost function. In this case the cost function for any value of R quantifies the difference between each component of V (i.e. v j ) and each component of IR (i.e. i j R) and sum up the squares of these differences as follows: cost(r) = (v j i j R) 2 Do you think this is a good cost function? Why or why not? For each point (i j,v j ), we want v j i j R to be as small as possible. We can call this term the individual error term for this point. One way of looking at the aggregate error in our fit is to add up the squares of the individual errors, so that all errors add up. This is precisely what we ve done in the cost function. If we did not square the differences, then a positive difference and a negative difference would cancel each other out. (d) Show that you can also express the above cost function in vector form, that is, cost(r) = ( V IR),( V IR) Hint: a, b = a T b = i a i b i Let s define the error vector as e = V IR. Then, we observe that e j = v j i j R. UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 2
Last Updated: 218-11-19 21:23 3 Therefore, cost(r) = = e 2 j (v j i j R) 2 = e 2 2 = e, e = ( V IR),( V IR) (e) Find ˆR, which is defined as the optimal value of R that minimizes cost(r). Hint: Use calculus. The optimal ˆR makes dcost( ˆR) dr = First, note that For R = ˆR, we will have dcost(r) dr which will ultimately give us In our particular example, I, V dcost(r) dr = 2 =. This means that 2 ˆR = i jv j i2 j i j (v j i j R) i j (v j i j ˆR) =, = I, V = 448 and I 2 = 224. Therefore, we will get ˆR = 2kΩ. 1 21 3 7 Using the equation for least squares estimate with A = 1 5 and b = 2 8 I 2 8 5 ˆR = (A T A) 1 A T b I, V 15 11, we would have: ˆR = I, I I, V ˆR = I 2, which gives us the same expression as before! UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 3
Last Updated: 218-11-19 21:23 4 (f) On your original IV plot, also plot the line v test = ˆRi test. Can you visually see why this line fits the data well? How well would we have done if we had guessed R = 3kΩ? What about R = 1kΩ? Calculate the cost functions for each of these choices of R to validate your answer. When ˆR = 2kΩ, we have When ˆR = 3kΩ, we have When ˆR = 1kΩ, we have cost(2k) = (21 2 1) 2 + (7 2 3) 2 + ( 2 2 ( 1)) 2 + (8 2 5) 2 + ( 15 2 ( 8)) 2 + ( 11 2 ( 5)) 2 = 8. cost(3k) = (21 3 1) 2 + (7 3 3) 2 + ( 2 3 ( 1)) 2 + (8 3 5) 2 + ( 15 3 ( 8)) 2 + ( 11 3 ( 5)) 2 = 232. cost(1k) = (21 1 1) 2 + (7 1 3) 2 + ( 2 1 ( 1)) 2 + (8 1 5) 2 + ( 15 1 ( 8)) 2 + ( 11 1 ( 5)) 2 = 232. (g) Now, suppose that we add a new data point: i 7 = 2mA, v 7 = 4V. Will ˆR increase, decrease, or remain the same? Why? What does that say about the line v test = ˆRi test? We can qualitatively see that ˆR will remain 2. This is because we already obtained ˆR to fit our previous data in the best way. Now, you should notice that this new piece of data (i 7,v 7 ) also lies exactly on the line v test = ˆRi test! Therefore, you have no reason to change ˆR. It is the best fit for the old data and will fit the new data anyway. (h) Let s add another data point: i 8 = 4mA,v 8 = 11V. Will ˆR increase, decrease, or remain the same? Why? What does that say about the line v test = ˆRi test? We can qualitatively see that ˆR should be something greater than or equal to 2. This is because you have already obtained ˆR to fit your previous data in the best way. Now, you notice that this new piece of data (i 8,v 8 ) also lies above the line v test = ˆRi test! Therefore, if you decreased ˆR, it would be a worse fit for the old data and the new data. You would increase ˆR to find a better fit. (i) Now your mischievous friend has hidden the black box. You want to predict what output voltage across the terminals if you applied 5.5mA through the black box. What would your best guess be? Hopefully, by now, it makes sense to the class that you will estimate ˆV = 5.5mA ˆR = 5.5mA 2kΩ = 11 V. This is an example of estimation from machine learning! You have learned what is going on inside the black box, that is, ˆR, by making observations of I and V. Now, you are using what you have learned, ˆR, to estimate ˆV for new values of I. 2. Correlation Revisited We are given the following two signals, s 1 and s 2 respectively: UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 4
Last Updated: 218-11-19 21:23 5 Signal 1 Signal 2 3 4 2 3 2 1 1 1 2 1 2 We have find their cross-correlation for three different scenarios: Periodic linear cross-correlation Linear cross-correlation when the signals are not periodic Circular cross-correlation Note that periodic linear cross correlation and circular cross correlation are essentially the same while periodic linear cross correlation looks at linear shifts of an infinite periodic signal, circular correlation looks at circular shifts of one period of the signal. (a) Find the periodic linear cross correlations, corr N=3 ( s 1, s 2 ) and corr N=3 ( s 2, s 1 ) for s 1 and s 2, assuming they are periodic with period N = 3, where periodic linear cross correlation is defined as: N 1 corr N ( x, y)[k] = i= x[i]y[i k] The signals continue to + and, however, since they are periodic we can focus on just one period of the signal. Calculate the periodic linear cross-correlation assuming the signals are periodic as below (The first one is already done for you!): corr N=3 ( s 1, s 2 ) s 2 [n] 2 4 3 s 1, s 2 [n] 2 + 8 + 9 = 19 corr N=3 ( s 2, s 1 ) s 1 [n] s 2, s 1 [n] + + = s 2 [n 1] s 1, s 2 [n 1] + + = s 1 [n 1] s 2, s 1 [n 1] + + = s 2 [n 2] s 1, s 2 [n 2] + + = s 1 [n 2] s 2, s 1 [n 2] + + = UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 5
Last Updated: 218-11-19 21:23 corr N=3 ( s 1, s 2 ) s 2 [n] 2 4 3 s 1, s 2 [n] 2 + 8 + 9 = 19 corr N=3 ( s 2, s 1 ) s 1 [n] 1 2 3 s 2, s 1 [n] 2 + 8 + 9 = 19 s 2 [n 1] 3 2 4 s 1, s 2 [n 1] 3 + 4 + 12 = 19 s 1 [n 1] 3 1 2 s 2, s 1 [n 1] + 4 + = 1 s 2 [n 2] 4 3 2 s 1, s 2 [n 2] 4 + + = 1 s 1 [n 2] 2 3 1 s 2, s 1 [n 2] 4 + 12 + 3 = 19 2 Periodic Cross-correlation of s 1 and s 2 2 Periodic Cross-correlation of s 2 and s 1 15 15 1 1 5 5 1 2 1 2 It is evident from the results that corr N=3 ( s 1, s 2 )[k] = corr N=3 ( s 2, s 1 )[ k] = corr N=3 ( s 2, s 1 )[N k]. (b) Find the two linear cross correlations, corr( s 1, s 2 ) and corr( s 2, s 1 ) for s 1 and s 2, assuming they are not periodic, where corr( x, y)[k] = i= x[i]y[i k] corr( s 1, s 2 ) s 2 [n + 2] s 1, s 2 [n + 2] + + + + + + = s 2 [n + 1] s 1, s 2 [n + 1] + + + + + + = s 2 [n] s 1, s 2 [n] + + + + + + = UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission.
Last Updated: 218-11-19 21:23 7 s 2 [n 1] s 1, s 2 [n 1] + + + + + + = s 2 [n 2] s 1, s 2 [n 2] + + + + + + = corr( s 2, s 1 ) s 2 2 4 3 s 1 [n + 2] s 2, s 1 [n + 2] + + + + + + = s 2 2 4 3 s 1 [n + 1] s 2, s 1 [n + 1] + + + + + + = s 2 2 4 3 s 1 [n] s 2, s 1 [n] + + + + + + = s 2 2 4 3 s 1 [n 1] s 2, s 1 [n 1] + + + + + + = s 2 2 4 3 s 1 [n 2] s 2, s 1 [n 2] + + + + + + = In this case, s 1 [n] and s 2 [n] are not periodic, so they contain only zeros outside the range: n 2. The linear cross-correlation is calculated by shifting the second signal both forward and backward until there is no overlap between the signals. When there is no overlap, the cross-correlation goes to zero. Both of these cross-correlations should have only zeros outside the range: 2 n 2. corr( s 1, s 2 ) s 2 [n + 2] 2 4 3 s 1, s 2 [n + 2] + + 3 + + + + = 3 s 2 [n + 1] 2 4 3 s 1, s 2 [n + 1] + + 4 + + + + = 1 UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 7
Last Updated: 218-11-19 21:23 8 s 1, s 2 [n] + + 2 + 8 + 9 + + = 19 s 2 [n 1] 2 4 3 s 1, s 2 [n 1] + + + 4 + 12 + + = 1 s 2 [n 2] 2 4 3 s 1, s 2 [n 2] + + + + + + = corr( s 2, s 1 ) s 1 [n + 2] 1 2 3 s 2, s 1 [n + 2] + + + + + + = s 1 [n + 1] 1 2 3 s 2, s 1 [n + 1] + + 4 + 12 + + + = 1 s 1 [n] 1 2 3 s 2, s 1 [n] + + 2 + 8 + 9 + + = 19 s 2 [n 1] 1 2 3 s 2, s 1 [n 1] + + + 4 + + + = 1 s 2 [n 2] 1 2 3 s 2, s 1 [n 2] + + + + 3 + + = 3 UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 8
Last Updated: 218-11-19 21:23 9 2 Linear Cross-correlation of s 1 and s 2 2 Linear Cross-correlation of s 2 and s 1 15 15 1 1 5 5 2 1 1 2 2 1 1 2 Notice that corr( s 1, s 2 )[k] = corr( s 2, s 1 )[ k], i.e. changing the order of the signals reveres the crosscorrelation sequence. (c) Find the circular cross correlation, circcorr( s 1, s 2 ) and circcorr( s 2, s 1 ) for s 1 and s 2, where N 1 circcorr( x, y)[k] = i= x[i]y[i k] N circcorr( s 1, s 2 ) s 2 [n] 2 4 3 s 1, s 2 [n] 2 + 8 + 9 = 19 circcorr( s 2, s 1 ) s 1 [n] 1 2 3 s 2, s 1 [n] 2 + 8 + 9 = 19 s 2 [n 1] 3 2 4 s 1, s 2 [n 1] 3 + 4 + 12 = 19 s 1 [n 1] 3 1 2 s 2, s 1 [n 1] + 4 + = 1 s 2 [n 2] 4 3 2 s 1, s 2 [n 2] 4 + + = 1 Alternatively, we can utilize the circulant matrix to compute: 2 4 3 1 19 circcorr( s 1, s 2 ) = 3 2 4 2 = 19 4 3 2 3 1 1 2 3 2 19 circcorr( s 2, s 1 ) = 3 1 2 4 = 1 2 3 1 3 19 s 1 [n 2] 2 3 1 s 2, s 1 [n 2] 4 + 12 + 3 = 19 You can easily verify that these results are the same as the periodic cross-correlation results. UCB EECS 1A, Fall 218, Discussion 13A, All Rights Reserved. This may not be publicly shared without explicit permission. 9