Detection and Estimation Theory

Transcription

1 ESE 54 Detection and Estimation heory Joseph A. O Sullivan Samuel C. Sachs Professor Electronic Systems and Signals Research Laboratory Electrical and Systems Engineering Washington University Urbauer Hall (Lynda answers) jao@wustl.edu J. A. O'S. ESE 54, Lecture, 4//9

2 Announcements Review class on Friday April 3 First eam is April 7, in class Other announcements? Questions? April : more on random processes April 9: detection theory for random processes April 4: M-ary detection theory April 6: estimation theory for random processes April : estimation theory and Cramer-Rao bounds April 3: more estimation theory April 4 (Friday): Wrap-up and review (last class) J. A. O'S. ESE 54, Lecture, 4//9

3 rh arhunen-loeve Epansions Orthogonal epansions Complete orthonormal (CON) sets Fourier series as an eample, Parseval s equality, Plancherel s equality Introduction to arhunen-loeve epansions, Gauassian distribution covariance functions, Eigenfunctions and eigenvalues, CON sets Mercer s theorem Implications, eamples J. A. O'S. ESE 54, Lecture, 4//9 3

4 Series Representations { φ < } φ φ = δ, Complete Orthonormal (CON) set,,,. Finite energy signal t ( ), t. Analysis equation: * =, φ = ( ) φ( ) a t t dt Synthesis equation: t () = aφ (), t t = l l Signal space is vector space of coefficients Inner products in function space equal inner products in signal space Coefficient computation via correlator or matched filter Parseval's equality: = E = a, = Plancherel's equality: *, =, where =, = J. A. O'S. ESE 54, Lecture, 4//9 y ab b y φ 4

5 Complete Orthonormal r (CON) Sets he Fourier series is a standard eample of a CON set Let the basis functions be for the Fourier series for period, φ () t =. e j π t π j t a =, φ = ( t) e dt J. A. O'S. ESE 54, Lecture, 4//9 5

6 Covariance Function for Gaussian Random Processes Assume t ( ) is a Gaussian random process (GRP) on [, ]. Et [ ( )] = Et [ ( ) *( u)] = ( tu, ), ( tu, )* = ( ut, ), Recall that ψ *() t dd () *() ( tu, ) ψ( ududt ) = E t ψ t dt, with equality if and only if t () ψ *() tdt = with probability. (,) t t = E[ () t ] < E () t dt < t ( ) is mean-square continuous lim E t () t ( + τ ) = (, tu) is continuous in (, tu ). τ J. A. O'S. ESE 54, Lecture, 4//9 6

7 rh arhunen-loeve Epansion he set of functions n n n { φ t } ( ) that solve λ φ () t = (, t u) φ ( u) du, t { λ } are eigenfunctions of ( t, u). he are called eigenvalues. ( t, u) is nonnegative definite λ,. { φ t } ( ) form an orthonormal set, a CON set if ( t, u) is positive definite. it Epand the random process () t using φ () t to get: Analysis equation: ( λ ) a =, φ Comment: a N,, independen t Synthesis equation: t () = l.i.m. aφ (), t t N N = { } J. A. O'S. ESE 54, Lecture, 4//9 7

8 Mercer s heorem he mean energy in the error is N * N = N = λφ n n φn n= E () t E () t () t (,) t t () t () t Note that N N N * * * E, N = E an ( t) φn( t) dt = E anan = λn. n= n= n= heorem: If ( t, u) is continuous on [, ] [, ], is nonnegative definite, ( tu, ) dudtd <, and is Hermitian symmetric, then N * (, ) = lim λ nφ n () φ n ( ) N n = t u t u uniformly in ( tu, ). For us, the covariance function satisfies these properties, p so N ( t) = l.i.m. anφn( t) on [, ]. N n= J. A. O'S. ESE 54, Lecture, 4//9 8

9 arhunen-loeve Epansion: Eample Suppose that the covariance function equals the outer product of a fied real-valued signal. t () = gs (), t ss, =, g N (, σ ) t u s t s u (, ) = σ () ( ) hen the only eigenfunction with nonzero eigenvalue is st ( ) : ( tu, ) φ( udu ) = σ stsu ( ) ( ) φ( udu ) = σ st ( ) su ( ) φ( udu ) φ = λ = σ () t s (), t and. If a complete orthonormal set is needed, then the orthonormal set of eigenfunctions is augmented with additional functions orthogonal π t to the eigenfunctions. For eample, if st ( ) = cos, then additional functions to get a CON set could be the set πt πt πt J. A. O'S. ESE 54, Lecture, 4//9, sin, cos, sin,. 9

10 Eample: White Gaussian Noise ( t) is (real-valued) white Gaussian noise (WGN) N Et [ ( )] =, Etu [ ( ) ( )] = δ ( t u), j π fτ N S( f) = ( τ) e dτ = Find the eigenfunctions: N N t u u du t u u du t (, ) φ( ) = δ( ) φ( ) = φ( ) his holds for any function. hus the eigenfunctions are N an arbitrary CON set. All eigenvalues equal. WGN violates many of our assumptions! J. A. O'S. ESE 54, Lecture, 4//9

11 Eample: Stationary Process t ( ) is stationary. Observation time is -,. Et [ ( )] =, Ett [ ( ) ( τ)] = ( τ), / jπ fτ S ( f) = ( τ) e d / Find the eigenfunctions: / / τ ( t u) φ( u) du = λφ( t) Let the observation time get large (stationary process with a large observation time (SPLO)). ae the Fourier transform of both sides Φ( f) S ( f) λ Φ( f) f More carefully, use the Fourier series basis, then n λn S, samples of the power spectrum. J. A. O'S. ESE 54, Lecture, 4//9

12 Eample: Prolate Spheroidal Wave Functions ae WGN and pass it through a band-limited filter, bandlimited to [-W,W] hertz Denote the band-limiting operation by B Multiplication by a rectangular filter in the frequency (spectral) domain Convolution with a sinc function in the time (covariance) domain Observe a time-limited version of the output, time-limited to [-/,/] seconds Denote the time-limiting operation by D Multiplication by a rectangular window in the time domain Convolution with sinc function in the freqency (spectral) domain Compute the covariance functions and power spectra at all points in order to gain insight Eigenfunctions are time-limited; band-limiting then timelimiting an eigenfunction returns a scaled eigenfunction prolate spheroidal wave functions J. A. O'S. ESE 54, Lecture, 4//9

13 Eample: Prolate Spheroidal Wave Functions Prolate spheroidal wave functions form a complete orthonormal set. he sum of the eigenvalues equals the integral of the covariance function equals the timebandwidth product, W. he Fourier transforms of the prolate spheroidal wave functions form an orthonormal set on (-, ) and an orthogonal set on [-W,W]. Scaling them yields a CON set in the frequency domain on [-W,W]; again a set of prolate spheroidal wave functions. Additional notes on board. J. A. O'S. ESE 54, Lecture, 4//9 3

14 Matlab Code function [eigenfn,eigenval]=prolate(w,) eigenval] oversampfactor=; twow=*w; twopi=*pi; pi; nt=floor(twow**oversampfactor)+; tstep=/(nt-); t=:nt; tt=t*tstep; tt=ones(nt,)*tt; tt=tt'-tt; y=(twow)*sinc(w*tt); [eigenfn,eigenval]=eig(y); figure plot(tt,eigenfn(:,:)') eigenval=diag(eigenval); J. A. O'S. ESE 54, Lecture, 4//9 4

15 Binary Detection ti wo hypotheses (M-ary case later) Signal plus noise for hypothesis Noise only for hypothesis (null hypothesis) Can handle two hypotheses of signal plus noise Start with arbitrary GRP, study AWGN, return to general case H : r ( t ) = Es ( t ) + n ( t ), t H : rt ( ) = nt ( ), t ss, =, nt ( ) st ( ) [ ] E n() t n( u) = (, t u) J. A. O'S. ESE 54, Lecture, 4//9 5

16 Binary Detection ti Use the eigenfunctions for the noise covariance function to approimate r(t) using a finite number of terms H : rt ( ) = Est ( ) + nt ( ), t H : r ( t ) = n ( t ), t ss, =, nt ( ) st ( ) [ ] E n() t n( u) = (, t u) λφ () t = (, t u) φ ( u) du r n n n E s, φ + n, H = r, φ = n, H N, λ independent GRVs ( ) r () t = rφ () t = J. A. O'S. ESE 54, Lecture, 4//9 6

17 Binary Detection ti Find the optimal decision rule (log-lielihood ratio test) using the finite number of random variables that determine the approimation H : r = s + n, =,,..., H : r = n, =,,..., ( λ ) n N, independent GRVs, E s, φ = s ep ( r s) πλ λ l() r = ln = ep r πλ λ l () r = r s s = λ J. A. O'S. ESE 54, Lecture, 4//9 Sufficient statistic is the weighted sum of the product of the data coefficients and the signal coefficients 7

18 Binary Detection ti he optimal decision rule for the random process is found by letting the number of terms in the approimation go to infinity (get large). l() r = r s ( s) = λ r s = r() t Es() t φ() t dt s = Es() t φ () t dt l ( r ) = r () t Es () t φ () t φ ( u ) Es ( u ) dtdu = λ l () r l() r = r() t Es() t Q(, t u) Es() u dtdu if φ( t) φ( u) Q( t, u) λ = J. A. O'S. ESE 54, Lecture, 4//9 Sufficient statistic is the weighted inner product between the measurement and the signal, where the weighting uses the inverse of the covariance matri if it eists (more later). he log-lielihood lielihood ratio is a Gaussian random variable under either hypothesis. he variance of the log- lielihood ratio is the same under the two hypotheses (it is determined only by the noise component). Performance is determined by the signal to noise ratio (signal energy to noise 8 power of the LLR)

19 Binary Detection: ti AWGN Case H : ( ) ( ) ( ), rt = Est + wt t H : rt () = wt (), t ss, =, N E [ w () t wu ( )] = δ ( t u ) s w, H r = r, φ = + w, H N w E s s N, i.i.d.,, φ = r () t = rφ () t = () r = = N l rs s J. A. O'S. ESE 54, Lecture, 4//9 Any CON set can be used Sufficient statistic in signal space is simply the inner product between the coefficients for the measurement and the coefficients for the signal 9

20 Binary Detection: ti AWGN Case s + w, H r, he log-lielihood ratio = r φ = w, H converges as the number N of terms gets large w N, i.i.d.,, E s φ = s he inner product between the measurement and the r() t = rφ () t signal is a sufficient = statistic l = () r rs s l() r Matched filter or correlator N = receiver structure is E E E l() r = E rs, ss, rs, optimal = N N N he performance is easily E E found to be determined by H : l N, N the ratio of the signal N E d = energy to the noise power E E N H : l N, A simpler derivation is N N J. A. O'S. ESE 54, Lecture, 4//9 possible

21 Binary Detection: AWGN Case Revisited it H : r( t) Es( t) w( t), t H : r( t) = w( t), t = + used N ss, =, Ewtwu [ ( ) ( )] = δ ( t u) Let φ ( t) = s( t), and let the remaining φ () t be arbitrary (but orthogonal). E + w, H r = r, s = w, H w, H r = r, φ = w, H l () r = r E E = rs E E, N N Any CON set can be Select the first function in the CON set to be s(t) and let the rest be arbitrary All coefficients ecept the first are equal under the two hypotheses. hus the first coefficient i is the sufficient statistic. J. A. O'S. ESE 54, Lecture, 4//9