On Preprocessing for Mismatched Classification of Gaussian Signals

Transcription

1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 3, MARCH [6] P. A. Voois, A theorem on the asymptotic eigenvalue distribution of Toeplitz-bloc-Toeplitz matrices, IEEE Trans. Signal Processing, vol. 44, pp , July 996. [7] N. K. Bose and K. J. Boo, Asymptotic eigenvalue distribution of bloc- Toeplitz matrices, IEEE Trans. Inform. Theory, vol. 44, pp , Mar [8] S. Serra, On the extreme eigenvalues of Hermitian (bloc) Toeplitz matrices, Linear Algebra Its Applic., vol. 270, pp , 998. [9] P. Tilli, On the asymptotic spectrum of Hermitian bloc Toeplitz matrices with Toeplitz blocs, Math. Comput., vol. 6, no. 29, pp , 997. [20] L. Tong, G. Xu, and T. Kailath, Blind identification and equalization based on second-order statistics: A time domain approach, IEEE Trans. Inform. Theory, vol. 40, pp , Mar [2] I. Fijalow, A. Touzni, and J. R. Treichler, Fractionally spaced equalization using CMA: Robustness to channel noise and lac of disparity, IEEE Trans. Signal Processing, vol. 45, pp , Jan [22] A. P. Liavas, P. A. Regalia, and J. P. Delmas, Blind channel approximation: Effective channel order determination, IEEE Trans. Signal Processing, vol. 47, pp , Dec [23] P. Ciblat and P. Loubaton, Second order blind equalization: The bandlimited case, in Proc. IEEE ICASSP, 998. On Preprocessing for Mismatched Classification of Gaussian Signals Yariv Ephraim, Fellow, IEEE, and William J. J. Roberts, Member, IEEE Abstract The optimal linear preprocessor for classifying two zero-mean Gaussian discrete-time signals which have been corrupted by additive zero-mean Gaussian noise is studied. Conditions for existence of the optimal linear preprocessor that achieves the performance of the lielihood ratio test for the noisy signals are given and the preprocessor is explicitly derived. Index Terms Hypothesis testing, mismatched classification, preprocessing. I. INTRODUCTION We study a binary hypothesis testing problem in which a classifier was designed for clean signals but the observed signals are noisy. We assume zero-mean Gaussian discrete-time signals and an additive statistically independent zero-mean Gaussian discrete-time noise process. These mismatched conditions may result in significant performance degradation especially at low signal-to-noise ratio (SNR). The optimal classifier for the clean signals compares the lielihood ratio of these signals to a threshold that depends on the optimality criterion. When the observed signals are noisy, the classifier must use the lielihood ratio for the noisy signals and a different threshold may be required. Under the Gaussian regime considered here, the lielihood Manuscript received July, 999; revised September 7, Y. Ephraim is with the Department of Electrical and Computer Engineering, George Mason University, Fairfax, VA USA ( yephraim@gmu. edu). W. J. J. Roberts was with the School of Information Technology and Engineering, George Mason University, Fairfax, VA USA. He is now with the Information Technology Division, Defense Science Technology Organisation, Salisbury, SA 508, Australia ( william.roberts@dsto.defence.gov.au). Communicated by S. R. Kularni, Associate Editor for Nonparametric Estimation, Classification, and Neural Networs. Publisher Item Identifier S (0) ratio for the noisy signals can be obtained from the original lielihood ratio by replacing the covariance matrices of the clean signals by the covariance matrices of the noisy signals. In some applications, however, it is not desirable to modify the original classifier. Instead, the original lielihood ratio test is supplemented by a preprocessor that provides an estimate of the clean signal from the observed noisy signal. This approach may be chosen, for example, in designing smart antennas where the preprocessor and the classifier may be in separate locations. In this correspondence, we study the hooup of a linear preprocessor with the original lielihood ratio test which provides the desired lielihood ratio test for the noisy signals. Optimal classifications in the sense of minimum probability of error and in the Neyman Pearson sense [20] are considered. We provide conditions for the linear preprocessing approach to be optimal and give the explicit form of the optimal preprocessor. When the Gaussian signals and noise have circulant covariance matrices [7], the optimal preprocessor is proportional to the geometric mean of the Wiener filters for the two hypotheses. For independent and identically distributed (i.i.d.) signals and noise, we calculate the probability of error and compare the optimal preprocessor for classification with the optimal linear preprocessor for estimation of the signal in the minimum mean-square error (MMSE) sense. This is the Wiener estimator for the mixture covariance of the signals under the two hypotheses. It is demonstrated that the optimal linear preprocessor for classification can substantially outperform the optimal linear preprocessor for estimation especially at low SNRs. Other signal estimation preprocessors that adaptively estimate the clean signals from the noisy signals are often used but will not be considered here [7]. The optimal preprocessor for classification derived here under Gaussian assumptions may also be useful in approximating the optimal linear preprocessor when the signals and noise are not strictly Gaussian and compensation of the lielihood ratio for the clean signals is not trivial. In such situations, only second-order statistics of the signals and noise are required for designing the linear preprocessor for classification. The preprocessing approach is motivated by the infamous Kailath Duncan theorem [0], []. This theorem draws an analogy between the lielihood ratio functions for two binary detection problems. In one problem, the signal to be detected is deterministically nown while in the other problem, it is a sample function of a finite-energy random process that is not necessarily Gaussian. In both cases, the signals and noise are continuous-time processes, and the noise is assumed to be a zero-mean Gaussian white process. The signal and noise are assumed statistically independent. The theorem shows that the lielihood ratio for detecting the random signal has a similar form as that of the lielihood ratio for detecting the deterministic signal. The former lielihood ratio can formally be obtained from the latter by replacing the deterministic signal with the MMSE causal estimator of the signal random process, and by interpreting the correlator integral as an Ito integral. Thus, this theorem shows that the optimal detector for the signal random process is an estimator correlator receiver in which the signal is first estimated from the observed process and then the estimated signal is applied to the correlation detector as if it were the nown deterministic signal. An excellent review of this and related results can be found in [3]. Weaer conditions for the theorem are also given in [3]. The Kailath Duncan theorem deals with a detection problem that is different from the mismatched classification problem we study here. Nevertheless, it is often cited as the rationale for replacing unavailable clean signals by their estimates in detection and classifications problems. Furthermore, the Kailath Duncan theorem applies to continuous-time signals only. No analogous theorem for discrete-time sig /0$ IEEE

2 252 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 3, MARCH 200 Fig.. The linear preprocessing classification system. nals is nown primarily since there is no explicit lielihood ratio formula for discrete-time signals [4], [9], [8]. We also see here a single preprocessor for the two hypotheses rather than a single estimator for each hypothesis as in [0], [], and [2]. The remainder of this correspondence is organized as follows. Our main results are presented in Section II. The i.i.d. case is studied in Section III. Comments are given in Section IV. II. MAIN RESULTS Let Y denote a -dimensional zero-mean Gaussian vector in the Euclidean space R. Let and 2 denote two hypotheses for the covariance matrix of Y. Let R i = EfYY 0 j ig denote the positive-definite covariance matrix of Y under i ;i = ; 2; where ( ) 0 denotes matrix transpose. Let y denote a realization of Y. Let p(yj i) denote the density of Y under hypothesis i, i = ; 2. Consider optimal classifications of y in the minimum probability of error sense or in the Neyman Pearson sense. In both cases, the optimal partition of R is obtained by comparing the lielihood ratio L Y (y) =p(yj )=p(yj 2 ) to a threshold. When optimality is in the minimum probability of classification error sense, the threshold is given by P 2=P where P denotes the probability of and P 2 =0P denotes the probability of 2. In the Neyman Pearson formulation the threshold is determined by the probability of false alarm. Let = fy: logl Y (y) 0 log >0g; 2 = C () denote the optimal partition of R where C is the complement of. We shall use natural logarithms throughout the correspondence unless otherwise specified. For the Gaussian densities of the problem where log L Y (y) 0 log = 0 2 y0 R 4 y (2) = log jjdenotes a determinant, and R0 =2 R 2 (3) R 4 = R 0 0 R 0 2 : (4) We assume that =is not an eigenvalue of R 0 R 2 and thus R 4 is nonsingular. Suppose that Y is nonobservable and that a corrupted version Z = Y + W is observed where W is a zero-mean -dimensional Gaussian vector with positive-definite covariance matrix R w. Let z 2R denote a realization of Z. Let p(zj i) denote the density of Z under the hypothesis i, i =; 2. These Gaussian densities differ in their covariance matrices given by ~R i = EfZZ 0 j i g = R i + R w ; i =; 2: The optimal classifier of z compares the lielihood ratio L Z (z) = p(zj )=p(zj 2 ) to a threshold ~ that depends on the optimality criterion. The optimal partition of the Euclidean space of the noisy signals fzg is given by where and ~ = fz: log L Z (z) 0 log ~ >0g; ~ 2 = ~ C (5) log L Z (z) 0 log ~ =~ 0 2 z0 ~ R 4 z (6) ~ = log R ~ ~ 0 =2 ~R 2 (7) ~R 4 = ~ R 0 0 ~ R 0 2 : (8) We assume that ~ =is not an eigenvalue of R ~ 0 ~R 2 and, thus, R ~ 4 is nonsingular. Clearly, the lielihood ratio test for the noisy signal can be obtained from the lielihood ratio test for the clean signal simply by replacing the covariance matrices of Y by those of Z and the threshold by ~. Suppose instead that we choose to continue using the lielihood ratio test for the clean signal when the input signal is noisy. To compensate for the input noise, however, we preprocess the input signal by a linear estimator H, as shown in Fig., where > 0 is some constant. Substituting Hz for y in (2) we obtain log L Y (Hz) 0 log = z0 [H 0 R 4 H]z: (9) Since this lielihood is compared to zero, we may as well use 2 [log LY (Hz) 0 log ] = z0 [H 0 R 4 H]z (0) instead of (9). For the estimator H to be optimal, and H must satisfy 2 [log L Y (Hz) 0 log ] = log L Z (z) 0 log ~: () Comparing (0) to (6) we find that the following equations must be satisfied: 2 log = ~ = jr0 R 2j =2 log j ~ (2) R 0 ~ ~R 2 j =2 H 0 R 4 H = ~ R 4 : (3) The right-hand side of (2) is not guaranteed to be positive and hence the linear preprocessing approach does not always provide the optimal solution. It is optimal for those parameters fr ;R 2 ;R w ;;~g for which =~ > 0. Equation (3) is a degenerate form of the algebraic Ricatti equation [4]. A necessary condition for (3) to have a solution, not necessarily nonsingular, is that R 4 and ~ R 4 have the same inertia [5, Theorem 5.5.3]. The inertia of a symmetric matrix R, denoted by In R, is a triplet

3 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 3, MARCH of integers indicating the number of positive, zero, and negative eigenvalues of R, counted with their multiplicities. To show that R 4 and ~R 4 have the same inertia we first find in the Appendix that ~R 4 = W 0 2R 4 W (4) where W i = R ~ i R 0 is the Wiener filter for Y given Z under hypothesis i, i =; 2. Note that in general W i is nonsymmetric. It can be i expressed as W i = R =2 w Q i6i(6i + I) 0 Q 0 ir 0=2 w (5) where Q i is the orthogonal matrix of eigenvectors and 6 i is the diagonal matrix of eigenvalues of the symmetric matrix Rw 0=2 R i Rw 0=2. The matrix Rw =2 is the positive-definite square root of R w and Rw 0=2 = (Rw =2 ) 0. This form details the implementation of the Wiener filter by first whitening the noise using Rw 0=2. Next we use a theorem due to Sylvester [5, Exercise 6, p. 450] which shows that In R = InAR for any Hermitian matrix R and positive-definite matrix A. We have the following sequence of equalities: In ~ R 4 =In ~ R 2 ~ R 4 =In(R 2R 4 R ) ~ R 0 =In(R 2 R 4 R ) = In (R 2 R 4 R )R 0 =InR 2R 4 =InR 4 (6) where we have used (4) for the second equality and the symmetry of R 2R 4 R for the third equality. Having established (6) we can now find a solution of (3) [6]. Let R 4 = U 3U 0 where U is the orthogonal matrix of eigenvectors and 3 is the diagonal matrix of eigenvalues of R 4. Let 0 denote a diagonal matrix with the square root of the absolute value of the eigenvalues of R 4 on its main diagonal. Let J denote a diagonal matrix with 6 s on its main diagonal indicating the signs of the eigenvalues of R 4.We have that 3=0J0. Similarly, let ~ R 4 = ~ U ~ 3 ~ U 0 where ~ U is the orthogonal matrix of eigenvectors and ~ 3 is the diagonal matrix of eigenvalues of ~ R 4. Let ~ 3= ~ 0 ~ J ~ 0 where ~ 0 is a diagonal matrix with the square root of the absolute value of the eigenvalues of ~ R 4 on its main diagonal and ~J denotes a diagonal matrix with the signs of the eigenvalues of ~ R 4 on its main diagonal. From (6), there exists a perturbation matrix P such that P 0 JP = ~ J. A permutation matrix is orthogonal and it is obtained from the identity matrix by interchanges of rows or columns [5, p. 64]. Hence (3) can now be written as and (H 0 U 0P ) ~ J(P 0 0U 0 H)= ~ U ~ 0 ~ J ~ 0 ~ U 0 (7) H 0 = ~ U ~ 0 (U 0P ) 0 = ~ U ~ 0P U 0 : (8) Clearly, (3) does not have a unique solution. Other solutions are obtained as follows. Let Q be any orthogonal matrix such that Q 0 JQ ~ = J. ~ The matrix Q may be a permutation matrix that results in interchanging diagonal entries of J ~ which have the same sign. Substituting this relation in the left-hand side of (7) we find that H 0 = U ~ 0QP ~ U 0 is also a solution of (3). More explicit solutions for H are possible under additional assumptions on the matrices R, R 2, and R w. For example, assume that fr ig, i =; 2 or i = w, are positive definite circulant matrices. In that case R i = D6 i D # where D = fd nm g, d nm = 0=2 expfj2nm=g for n; m = 0; ;0, 6 i is a diagonal matrix of eigenvalues of R i, and ( ) # denotes conjugate transpose. From (8) it is easy to see that H is a positive-definite circulant matrix that is given by H =[W W 2] =2. Thus, the optimal normalized (by ) preprocessor is the geometric mean of the two Wiener filters for the two hypotheses. III. THE i.i.d. CASE When the signals under the two hypotheses are i.i.d. sources, and the noise is white, the probability of error can be explicitly written, thereby enabling its minimization over the class of all linear estimators. This is, of course, a special case of the more general problem studied in Section II. It is of interest, however, since an explicit expression for the probability of error can be used to compare the optimal preprocessor for classification derived in Section II with other suboptimal preprocessors. Of particular interest is a comparison with the preprocessor that linearly estimates the clean signal y from the noisy signal z in the MMSE sense. Linear preprocessor for estimating the signals have often been used in classifying clean signals which have been degraded by noise, see, e.g., [3], [7] and the references therein. This comparison will demonstrate the usefulness of designing preprocessors for discriminating the signals rather than for linearly estimating them in the MMSE sense. Since the density of the clean signal is given by p(y) =P p(yj )+P 2p(yj 2) (9) where p(yj i ) denotes the Gaussian density with zero mean and covariance matrix R i, the optimal linear MMSE estimator of the signal Y given the noisy signal Z is given by ~H =(P R + P 2 R 2 )(P R + P 2 R 2 + R w ) 0 : (20) For i.i.d. signals and noise, R i = r i I, i = ; 2 or i = w, and ~R i =~rii, ~r i = ri +rw, i =; 2, where I is the 2 identity matrix. We assume without loss of generality that r < r 2 and thus r 4 = (=r 0 =r 2 ) > 0. Since in this section we shall only be concerned with minimum probability of error classification, it is clear that =~. From (2) and (8), the optimal preprocessor for classification is given by h dis I where h dis is the positive square root of h 2 dis = r r + r w r 2 r 2 + r w log log r r ~r ~r =2 =2 : (2) The linear MMSE signal estimator (20) is given by h sigi where P r + P 2r 2 h sig = : (22) P r + P 2r 2 + r w Suppose that a linear estimator gi, not necessarily the optimal estimator h dis I, is used in the preprocessing scheme of Fig.. From (9), the statistic used for classifying the noisy signals is given by x = 0 2 g2 r 4 i= z 2 j (23) where z j denotes the jth component of the vector z. The sum of the squares of zero-mean equal variance Gaussian random variables fz 2 j g in (23) is gamma distributed [22, eq. (4.68)]. Hence, the conditional density of the almost surely positive random variable is given by where p(vj i)= 2 i 0 2 V = 2 g2 r 4 j= Z 2 j v 2 0 e 0 v U (v); i =; 2 (24) i = ~r i g 2 r 4 ; i =; 2 (25)

4 254 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 3, MARCH 200 Fig. 2. Minimum probability of error as a function of the ratio of the signal variances at three different SNRs. This is also the probability of error of the linear preprocessor for classification h I. and U (v) =if v 0 and U (v) =0otherwise. The probability of error in classifying z using this system is given by P e (g) =P 0 0 p(xj ) dx + P 2 0 p(xj 2 ) dx (26) where p(xj i) denotes the density of the random variable X in (23) given the hypothesis i, i =; 2. This density is easily obtained from (24). Thus, taing into account the fact that V is an almost surely positive random variable, we have P e(g) =P p(vj ) dv + P 2 p(vj 2) dv (27) provided that > 0. The case of 0 is of no interest since with probability one the random variable V cannot be smaller than a nonpositive. Thus, the hypothesis 2 is always chosen and the probability of error is P independently of the estimator gi. We shall, therefore, proceed with the assumption that > 0. We shall also assume that is even so that the integrals in (27) have finite series representations. On substituting (24) in (27) and evaluating the integrals using [6, eqs. (3.38.3), ( ), (3.38.), (8.352.), (8.339.)] we obtain the desired expression for the probability of error when an arbitrary linear estimator is used in the preprocessing classification system P e(g) =Pe 0 20 m=0 ( ) m + P 2 0 e 0 m! 0 20 m=0 ( 2 ) m (28) Setting the derivative of (28) with respect to g 2 to zero gives the optimal preprocessor (2). For the probability of error in classifying Gaussian signals that are not necessarily i.i.d., see [8]. The expression for the probability of error (28) can now be evaluated for the optimal preprocessor for classification h dis in (2) and for m! : the linear MMSE signal estimation preprocessor h sig in (22). The optimal preprocessor for classification is also linear for the given problem. Let P e (h dis ) denote the probability of error when the preprocessor h dis is used. This is also the minimum probability of error for the given problem. Let P e(hsig) denote the probability of error obtained when h sig is used. We have compared the two preprocessors in classifying two equally liely zero mean i.i.d. Gaussian vectors of dimension = 20and variances r = and : r 2=r 5. Three SNRs conditions given by 0, 5, and 0 db were assumed for the observed signal. The SNR is defined as 0 log 0 (r =r w ). Figs. 2 and 3 depict, respectively, P e(hdis ) and P e(hsig) as a function of r 2 =r for the three input SNRs. Fig. 4 depicts the ratio P e (h sig )=P e (h dis ) for the same range of r 2 =r and input SNRs. When r 2=r is smaller than a certain value 6:2, and the input SNR is not too low, the performance of the classifier using either preprocessor is similar. In that range of r 2=r, the ratio P e(hsig)=p e(hdis ) varies in a nonmonotonic fashion for each SNR, since P e(hsig) decreases with r 2 =r at a varying rate. As the ratio of r 2 =r increases beyond that value of 6:2, the mixture statistic used by the linear MMSE signal estimator becomes less reliable for the given hypothesis, and the ratio P e (h sig )=P e (h dis ) increases monotonically for each input SNR. When r is substantially different from r 2 and the input SNR is smaller than 0 db, the optimal preprocessor for classification provides significantly lower probability of error than that obtained with the suboptimal linear MMSE signal estimator. For example, when r 2 =r = 5and SNR = 0 db, we have P e (h dis )=0:00076, P e (h sig )=0:0064, and P e (h sig )=P e (h dis )=8:4022. IV. COMMENTS A linear preprocessing approach for classification of two zero-mean Gaussian signals which have been degraded by statistically indepen-

5 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 3, MARCH Fig. 3. Probability of error obtained with the linear MMSE signal estimation preprocessor as a function of the ratio of the signal variances at three different SNRs. Fig. 4. Ratio of the error probabilities obtained using the suboptimal linear MMSE signal estimation preprocessor and the optimal preprocessor for classification as a function of the ratio of the signal variances at three different SNRs.

6 256 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 3, MARCH 200 dent additive zero-mean Gaussian noise was studied. The linear preprocessing approach was shown to be optimal in the minimum probability of error sense and in the Neyman Pearson sense for a subset of the parameter set of the problem. When the Gaussian processes have circulant covariance matrices, the optimal preprocessor for classification is proportional to the geometric mean of the Wiener filters for the two processes. For i.i.d. signals and noise, the probability of error was explicitly calculated and used in comparing the optimal preprocessor for classification with the optimal preprocessor for linear MMSE estimation of the signal. This preprocessor is the Wiener filter for the mixture of the two Gaussian processes. It was demonstrated that the optimal preprocessor for classification can substantially outperform the linear MMSE preprocessor for signal estimation, especially at low SNRs. The general preprocessing approach for optimal classification of non-gaussian signals and noise forms an interesting and challenging problem for which no solution is yet nown. Of particular interest are signals with mixtures of Gaussian densities or hidden Marov processes [9]. Such models are extensively used in automatic speech recognition [9] and in many other classification problems [2]. In all of these examples, redesign of the recognition system for clean signals to accommodate for input noise is expensive and highly undesirable. The design of a preprocessor for classification of such signals is extremely difficult. The approach used in this wor does not seem extendable to the general case, and alternative approaches in which the linear preprocessor is designed by minimizing bounds on the probability of error [5], [], or by minimizing asymptotic forms of the probability of error, over the space of all possible linear preprocessors, did not result in tractable optimization problems even for the simple Gaussian case considered here. It is evident from the results of this wor, however, that the commonly used linear MMSE signal estimation preprocessing approach in applications such as automatic speech recognition (see. e.g., [3]) may be far from optimal. We finally note that while effective linear preprocessing is desirable for its simplicity and ease of implementation, nonlinear preprocessors may be easier to derive as the optimization problem is less constrained. APPENDIX DERIVATION OF (4) Since R and R2 are symmetric matrices R 0 0 R 0 2 = U 3U 0 (A) where 3 is a diagonal matrix and U is an orthogonal matrix. Thus, R 0 = R U 3U 0 : (A2) Applying the Woodbury Schur matrix inversion formula [2] gives R = R2 0 R2U (3 0 + U 0 R2U ) 0 U 0 R2: Adding the noise covariance Rw to (A3) gives (A3) ~R = ~ R2 0 R2U (3 0 + U 0 R2U ) 0 U 0 R2: (A4) Applying again the matrix inversion formula to (A4) we obtain ~R 0 = R ~ W 0 2 R 2 + R W2R2 W 2: (A5) Using some basic matrix manipulations it can be shown that R2 + R W2R2 = R4 WW 0 2 : (A6) The result (4) is obtained by substituting (A6) into (A5). ACKNOWLEDGMENT The authors wish to than Dr. Y. Steinberg and Dr. H. Lev-Ari for helpful discussions during this wor. They also than Dr. N. Merhav, Dr. S. Shamai, and the anonymous referees for reading the correspondence and for their useful comments. REFERENCES [] R. E. Blahut, Principles and Practice of Information Theory. Reading, MA: Addison-Wesley, 988. [2] Y. Ephraim, Hidden Marov models, in The Encyclopedia of Operations Research, I. Gass and C. M. Harris, Eds. Norwell: Kluwer Academic, 200. [3] Y. Ephraim and M. Rahim, On second order statistics and linear estimation of cepstral coefficients, IEEE Trans. Speech Audio Processing, vol. 7, pp , Mar [4] R. Esposito, On the relation between detection and estimation in decision theory, Inform. Contr., no. 2, pp. 6 20, 968. [5] R. G. Gallager, Information Theory and Reliable Communication. New Yor: Wiley, 968. [6] I. S. Gradshteyn and I. M. Ryzhi, Table of Integrals, Series, and Products. New Yor: Academic, 979. [7] R. M. Gray. (977, revised 998) Toeplitz and circulant matrices: II. Stanford Electron. Lab., Tech. Rep [Online] Available: people/gray/compression.html. [8] A. G. Jaffer and S. C. Gupta, On the relation between detection and estimation of discrete time processes, Inform. Contr., no. 20, pp , 972. [9] T. Kailath, A note on least squares estimates from lielihood ratios, Inform. Contr., no. 3, pp , 968. [0], A general lielihood-ratio formula for random signals in Gaussian noise, IEEE Trans. Inform. Theory, vol. IT-5, pp , May 969. [], A further note on a general lielihood formula for random signals in Gaussian noise, IEEE Trans. Inform. Theory, vol. IT-6, pp , July 970. [2], Linear Systems. Englewoods Cliffs, NJ: Prentice Hall, 980. [3] T. Kailath and H. V. Poor, Detection of stochastic processes, IEEE Trans. Inform. Theory, vol. 44, pp , Oct [4] H. Kwaernaa and R. Sivan, Linear Optimal Control Systems. New Yor: Wiley-Interscience, 972. [5] P. Lancaster and M. Tismenetsy, The Theory of Matrices. New Yor: Academic, 985. [6] H. Lev-Ari, Nonstationary lattice-filter modeling, Ph.D. dissertation, Stanford Univ., Stanford, CA, Dec [7] J. S. Lim and A. V. Oppenheim, Enhancement and bandwidth compression of noisy speech, in Proc. IEEE, vol. 67, Dec. 979, pp [8] Y. S. Natovich, Error probability in binary classification for Gaussian distributions with unequal covariance matrices, Soviet-Automat.-Contr., vol. 0, no. 2, pp. 66 2, 977. [9] L. R. Rabiner, A tutorial on hidden Marov models and selected applications in speech recognition, in Proc. IEEE, vol. 77, Feb. 989, pp [20] H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I. New Yor: Wiley, 969. [2], Detection, Estimation, and Modulation Theory, Part III. New Yor: Wiley, 97. [22] A. D. Whalen, Detection of Signals in Noise. New Yor: Academic, 97.