Equalization. Chapter 3. x k. y k. z k. ( t ) mod#

Transcription

1 Contents 3 Equalization 6 3. Intersymbol Interference and Receivers for Successive Message ransmission ransmission of Successive Messages Bandlimited Channels he ISI-Channel Model Basics of the Receiver-generated Equivalent AWGN Receiver Signal-to-Noise Ratio Receiver Biases he Matched-Filter Bound Nyquist s Criterion Vestigial Symmetry Raised Cosine Pulses Square-Root Splitting of the Nyquist Pulse Linear Zero-Forcing Equalization Performance Analysis of the ZFE Noise Enhancement Minimum Mean-Square Error Linear Equalization Optimization of the Linear Equalizer Performance of the MMSE-LE Examples Revisited Fractionally Spaced Equalization Decision Feedback Equalization Minimum-Mean-Square-Error Decision Feedback Equalizer (MMSE-DFE) Performance Analysis of the MMSE-DFE Zero-Forcing DFE Examples Revisited Finite Length Equalizers FIR MMSE-LE FIR ZFE example FIR MMSE-DFE An Alternative Approach to the DFE he Stanford DFE Program Error Propagation in the DFE Look-Ahead Precoding he omlinson Precoder Partial Response Channel Models Classes of Partial Response Simple Precoding General Precoding Quadrature PR

2 3.9 Diversity Equalization Multiple Received Signals and the RAKE Infinite-length MMSE Equalization Structures Finite-length Multidimensional Equalizers DFE RAKE Program Multichannel ransmission MIMO/Matrix Equalizers he MIMO Channel Model Matrix SNRs he MIMO MMSE DFE and Equalizers MIMO Precoding MIMO Zero-Forcing Adaptive Equalization he LMS algorithm for Adaptive Equalization Vector-MIMO Generalization of LMS Exercises - Chapter A Useful Results in Linear Minimum Mean-Square Estimation, Linear Algebra, and Filter Realization 37 A. he Orthogonality Principle A. Spectral Factorization and Scalar Filter Realization A.. Some ransform Basics: D-ransform and Laplace ransform A.. Autocorrelation and Non-Negative Spectra Magnitudes A..3 he Binlinear ransform and Spectral Factorization A..4 he Paley-Wiener Criterion A..5 Linear Prediction A..6 Cholesky Factorization A.3 MIMO Spectral Factorization A.3. ransforms A.3. Autocorrelation and Power Spectra for vector sequences A.3.3 Factorizability for MIMO Processes A.3.4 MIMO Paley Wiener Criterion and Matrix Filter Realization A.4 he Matrix Inversion Lemma B Equalization for Partial Response 339 B. Controlled ISI with the DFE B.. ZF-DFE and the Optimum Sequence Detector B.. MMSE-DFE and sequence detection B. Equalization with Fixed Partial Response B(D) B.. he Partial Response Linear Equalization Case B.. he Partial-Response Decision Feedback Equalizer

3 Chapter 3 Equalization he main focus of Chapter was a single use of the channel, signal set, and detector to transmit one of M messages, commonly referred to as one-shot transmission. In practice, the transmission system sends a sequence of messages, one after another, in which case the analysis of Chapter applies only if the channel is memoryless: that is, for one-shot analysis to apply, these successive transmissions must not interfere with one another. In practice, successive transmissions do often interfere with one another, especially as they are spaced more closely together to increase the data transmission rate. he interference between successive transmissions is called intersymbol interference (ISI). ISI can severely complicate the implementation of an optimum detector. Chapter 3 will focus on the situation where L x =, that is no MIMO, while MIMO will be re-introduced in Chapters 4 and 5. In practice the equalization methods of this chapter, while they could be generalized to MIMO, are only used in scalar situations. A more general generalized theory of equalization will evolve in Chapters 4 and 5 that allow a practical and also optimum way of dealing with the MIMO and non-mimo cases. Figure 3. illustrates a receiver for detection of a succession of messages. he matched filter outputs are processed by the receiver, which outputs samples, z k, that estimate the symbol transmitted at time k, ˆx k. Each receiver output sample is the input to the same one-shot detector that would be used on an AWGN channel without ISI. his symbol-by-symbol (SBS) detector, while optimum for the AWGN channel, will not be a maximum-likelihood estimator for the sequence of messages. Nonetheless, if the receiver is well-designed, the combination of receiver and detector may work nearly as well as an optimum detector with far less complexity. he objective of the receiver will be to improve the performance of this simple SBS detector. More sophisticated and complex sequence detectors are the topics of Chapters 4, 5 and 9. Equalization methods are used by communication engineers to mitigate the e ects of the intersym- x k input# symbol#at# <me#k# mod# x ( t ) h(t)# Bandlimited# channel# n (t ) +# y k Matched# filter# R# sampler# receiver# z k SBS# detector# xˆ k es<mate#of#input# symbol#at# <me#k# Equivalent#AWGN# x k y k SNR = at#<me#k# E{ e } k +# z k Ε x Figure 3.: he band-limited channel with receiver and SBS detector. 6

4 bol interference. An equalizer is essentially the content of Figure 3. s receiver box. his chapter studies both intersymbol interference and several equalization methods, which amount to di erent structures for the receiver box. he methods presented in this chapter are not optimal for detection, but rather are widely used sub-optimal cost-e ective methods that reduce the ISI. hese equalization methods try to convert a bandlimited channel with ISI into one that appears memoryless, hopefully synthesizing a new AWGN-like channel at the receiver output. he designer can then analyze the resulting memoryless, equalized channel using the methods of Chapter. With an appropriate choice of transmit signals, one of the methods in this chapter - the Decision Feedback Equalizer of Section 3.6 can be generalized into a canonical receiver (e ectively achieves the highest possible transmission rate even though not an optimum receiver), which is discussed further in Chapter 5 and occurs only when special conditions are met. Section 3. models linear intersymbol interference between successive transmissions, thereby both illustrating and measuring the problem. In practice, as shown by a simple example, distortion from overlapping symbols can be unacceptable, suggesting that some corrective action must be taken. Section 3. also refines the concept of signal-to-noise ratio, which is the method used in this text to quantify receiver performance. he SNR concept will be used consistently throughout the remainder of this text as a quick and accurate means of quantifying transmission performance, as opposed to probability of error, which can be more di cult to compute, especially for suboptimum designs. As Figure 3. shows, the objective of the receiver will be to convert the channel into an equivalent AWGN at each time k, independent of all other times k. An AWGN detector may then be applied to the derived channel, and performance computed readily using the gap approximation or other known formulae of Chapters and with the SNR of the derived AWGN channel. here may be loss of optimality in creating such an equivalent AWGN, which will be measured by the SNR of the equivalent AWGN with respect to the best value that might be expected otherwise for an optimum detector. Section 3.3 discusses some desired types of channel responses that exhibit no intersymbol interference, specifically introducing the Nyquist Criterion for a linear channel (equalized or otherwise) to be free of intersymbol interference. Section 3.4 illustrates the basic concept of equalization through the zero-forcing equalizer (ZFE), which is simple to understand but often of limited e ectiveness. he more widely used and higher performance, minimum mean square error linear (MMSE-LE) and decision-feedback equalizers (MMSE-DFE) are discussed in Sections 3.5 and 3.6. Section 3.7 discusses the design of finite-length equalizers. Section 3.8 discusses precoding, a method for eliminating error propagation in decision feedback equalizers and the related concept of partial-response channels and precoding. Section 3.9 generalizes the equalization concepts to systems that have one input, but several outputs, such as wireless transmission systems with multiple receive antennas called diversity. Section 3.0 generalizes the developments to MIMO channel equalization, while Section 3. discusses adaptive algorithms that are used in practice to e ect the various analytical solutions found throughout this chapter. Appendix A provides significant results on minimum mean-square estimation, including the orthogonality principle, scalar and MIMO forms of the Paley Weiner Criterion, results on linear prediction, Cholesky Factorization, and both scalar and MIMO canonical factorization results. Appendix B generalizes equalization to partial-response channels. he area of transmit optimization for scalar equalization, is yet a higher performance form of equalization for any channel with linear ISI, as discussed in Chapters 4 and 5. 6

5 3. Intersymbol Interference and Receivers for Successive Message ransmission Intersymbol interference is a common practical impairment found in many transmission and storage systems, including voiceband modems, digital subscriber loop data transmission, storage disks, digital mobile radio channels, digital microwave channels, and even fiber-optic (where dispersion-limited) cables. his section introduces a model for intersymbol interference. his section then continues and revisits the equivalent AWGN of Figure 3. in view of various receiver corrective actions for ISI. 3.. ransmission of Successive Messages Most communication systems re-use the channel to transmit several messages in succession. From Section., the message transmissions are separated by units in time, where is called the symbol period, and / is called the symbol rate. he data rate of Chapter for a communication system that sends one of M possible messages every time units is R = log (M) = b. (3.) o increase the data rate in a design, either b can be increased (which requires more signal energy to maintain P e ) or can be decreased. Decreasing narrows the time between message transmissions and thus increases intersymbol interference on any band-limited channel. he transmitted signal x(t) corresponding to K successive transmissions is x(t) = KX k=0 x k (t k). (3.) Equation (3.) slightly abuses previous notation in that the subscript k on x k (t k) referstotheindex associated with the k th successive transmission. he K successive transmissions could be considered an aggregate or block symbol, x(t), conveying one of M K possible messages. he receiver could attempt to implement MAP or ML detection for this new transmission system with M K messages. A Gram- Schmidt decomposition on the set of M K signals would then be performed and an optimum detector designed accordingly. Such an approach has complexity that grows exponentially (in proportion to M K ) with the block message length K. hat is, the optimal detector might need M K matched filters, one for each possible transmitted block symbol. As K!, the complexity can become too large for practical implementation. Chapter 9 addresses such sequence detectors in detail, and it may be possible to compute the à posteriori probability function with less than exponentially growing complexity. An alternative (suboptimal) receiver can detect each of the successive K messages independently. Such detection is called symbol-by-symbol (SBS) detection. Figure 3. contrasts the SBS detector with the block detector of Chapter. he bank of matched filters, presumably found by Gram-Schmitt decomposition of the set of (noiseless) channel output waveforms (of which it can be shown K dimensions are su cient only if N =, complex or real), precedes a block detector that determines the K-dimensional vector symbol transmitted. he complexity would become large or infinite as K becomes large or infinite for the block detector. he lower system in Figure 3. has a single matched filter to the channel, with output sampled K times, followed by a receiver and an SBS detector. he symbol rate is sometimes also called the baud rate, although abuse of the term baud (by equating it with data rate even when M 6= )hasrenderedthetermarchaicamongcommunicationengineers,andtheterm baud usuallynow only appears in trade journals and advertisements. 63

6 φ ( K t) y p (t) φ ( ) K t.!.!.! ( K t) φ K Block! Detector! ˆx t = K y p (t) ( t) ϕ p R! SBS! detector! ˆx k k = 0,...,K t = k, k = 0,..., K y p ( t) Figure 3.: Comparison of Block and SBS detectors for successive transmission of K messages. he later system has fixed (and lower) complexity per symbol/sample, but may not be optimum. Interference between successive transmissions, or intersymbol interference (ISI), can degrade the performance of symbol-by-symbol detection. his performance degradation increases as decreases (or the symbol rate increases) in most communication channels. he designer mathematically analyzes ISI by rewriting (3.) as x(t) = KX k=0 n= NX x kn ' n (t k), (3.3) where the transmissions x k (t) are decomposed using a common orthonormal basis set {' n (t)}. In (3.3), ' n (t k) and ' m (t l) may be non-orthogonal when k 6= l. In some cases, translates of the basis functions are orthogonal. For instance, in QAM, the two bandlimited basis functions r m t t ' (t) = cos sinc (3.4) r m t t ' (t) = sin sinc, (3.5) or from Chapter, the baseband equivalent '(t) = p t sinc. (3.6) (with m a positive integer) are orthogonal for all integer-multiple-of- time translations. In this case, the successive transmissions, when sampled at time instants k, are free of ISI, and transmission is equivalent to a succession of one-shot uses of the channel. In this case symbol-by-symbol detection is optimal, and the MAP detector for the entire block of messages is the same as a MAP detector used separately for each of the K independent transmissions. Signal sets for data transmission are usually designed to be orthogonal for any translation by an integer multiple of symbol periods. Most linear AWGN channels, however, are more accurately modeled by a filtered AWGN channel as discussed in Section.7 and Chapter. he filtering of the channel alters the basis functions so that at the channel output the filtered basis functions are no longer orthogonal. he channel thus introduces ISI. 64

7 3.. Bandlimited Channels ( ) x p t x h(t)# +! ( t ) x ( t ) x p t n x ϕ n (t)# h(t)# +! kn n p t x p n (t)# +! kn ( t ) y p ( t ) n p ( t ) ( ) ( t ) y p n p ( t ) x p ( ) ( t ) y p Figure 3.3: he bandlimited channel, and equivalent forms with pulse response. he bandlimited linear ISI channel, shown in Figure 3.3, is the same as the filtered AWGN channel discussed in Section.7. his channel is used, however, for successive transmission of data symbols. he (noise-free) channel output, x p (t), in Figure 3.3 is given by x p (t) = = KX k=0 n= KX NX k=0 n= NX x kn ' n (t k) h(t) (3.7) x kn p n (t k) (3.8) where p n (t) = ' n (t) h(t). When h(t) 6= (t), the functions p n (t k) do not necessarily form an orthonormal basis, nor are they even necessarily orthogonal. An optimum (MAP) detector would need to search a signal set of size M K, which is often too complex for implementation as K gets large. When N = (or N = with complex signals), there is only one pulse response p(t). Equalization methods apply a processor, the equalizer, to the channel output to try to convert {p n (t k)} to an orthogonal set of functions. Symbol-by-symbol detection can then be used on the equalized channel output. Further discussion of such equalization filters is deferred to Section 3.4. he remainder of this chapter also presumes that the channel-input symbol sequence x k is independent and identically distributed at each point in time. his presumption will be relaxed in later Chapters. 65

8 While a very general theory of ISI could be undertaken for any N, such a theory would unnecessarily complicate the present development. his chapter handles the ISI-equalizer case for N =. Using the baseband-equivalent systems of Chapter, this chapter s (Chapter 3 s) analysis will also apply to quadrature modulated systems modeled as complex (equivalent to two-dimensional real) channels. In this way, the developed theory of ISI and equalization will apply equally well to any one-dimensional (e.g. PAM) or two-dimensional (e.g. QAM or hexagonal) constellation. his was the main motivation for the introduction of bandpass analysis in Chapter. he pulse response for the transmitter/channel fundamentally quantifies ISI: Definition 3.. (Pulse Response) he pulse response of a bandlimited channel is defined by p(t) ='(t) h(t). (3.9) For the complex QAM case, p(t), '(t), and h(t) can be complex time functions. he one-dimensional noiseless channel output x p (t) is x p (t) = = KX k=0 KX k=0 x k '(t k) h(t) (3.0) x k p(t k). (3.) he signal in (3.) is real for a one-dimensional system and complex for a baseband equivalent quadrature modulated system. he pulse response energy kpk is not necessarily equal to, and this text introduces the normalized pulse response: ' p (t) = p(t) kpk, (3.) where kpk = Z p(t)p (t)dt = hp(t),p(t)i. (3.3) he subscript p on ' p (t) indicates that ' p (t) is a normalized version of p(t). Using (3.), Equation (3.) becomes where x p (t) = KX k=0 x p,k ' p (t k), (3.4) x p,k = x k kpk. (3.5) x p,k absorbs the channel gain/attenuation kpk into the definition of the input symbol, and thus has energy E p = E x p,k = Ex kpk. While the functions ' p (t k) are normalized, they are not necessarily orthogonal, so symbol-by-symbol detection is not necessarily optimal for the signal in (3.4). EXAMPLE 3.. (Intersymbol interference and the pulse response) As an example of intersymbol interference, consider the pulse response p(t) = +t and two successive 4 transmissions of opposite polarity ( followed by +) through the corresponding channel. Chapter 4 considers multidimensional signals and intersymbol interference, while Section 3.7 considers diversity receivers that may have several observations a single or multiple channel output(s). 66

9 Example of intersymbol interference # 0.4 amplitude sum# '# time Figure 3.4: Illustration of intersymbol interference for p(t) = +t 4 with =. Figure 3.4 illustrates the two isolated pulses with correct polarity and also the waveform corresponding to the two transmissions separated by unit in time. Clearly the peaks of the pulses have been displaced in time and also significantly reduced in amplitude. Higher transmission rates would force successive transmissions to be closer and closer together. Figure 3.5 illustrates the 67

10 0.8 Example of pulses moved closer together p(t)= +t 4 =.0 =0.5 =0. 0. amplitude time Figure 3.5: ISI with increasing symbol rate. resultant sum of the two waveforms for spacings of unit in time,.5 units in time, and. units in time. Clearly, ISI has the e ect of severely reducing pulse strength, thereby reducing immunity to noise. EXAMPLE 3.. (Pulse Response Orthogonality - Modified Duobinary) A PAM modulated signal using rectangular pulses is he channel introduces ISI, for example, according to he resulting pulse response is and the normalized pulse response is '(t) = p (u(t) u(t )). (3.6) h(t) = (t)+ (t ). (3.7) p(t) = p (u(t) u(t )) (3.8) ' p (t) = p (u(t) u(t )). (3.9) he pulse-response translates ' p (t) and ' p (t '(t ) were originally orthonormal. ) are not orthogonal, even though '(t) and 68

11 Dubinary Channel Magnitud magnitude normalized radian frequency Figure 3.6: Duobinary Frequency Response. Noise Equivalent Pulse Response Figure 3.7 models a channel with additive Gaussian noise that is not white, which often occurs in practice. he power spectral density of the noise is N0 S n(f). σ S n ( f ) (AGN)& x P f +! k ( ) S n ( f ) reversible& filter& receiver& White4noise&equivalent&channel&(with&reversible&receiver& noise4whitening &filter&viewed&as&part&of&pulse&response)& (AWGN)& σ P( f ) x k S f +! n ( ) o& matched& filter&...& equivalent&pulse&response& Figure 3.7: White Noise Equivalent Channel. When S n (f) 6= 0 (noise is never exactly zero at any frequency in practice), the noise psd has an invertible square root as in Section.7. he invertible square-root can be realized as a filter in the beginning of a receiver. Since this filter is invertible, by the reversibility theorem of Chapter, no information is lost. he designer can then construe this filter as being pushed back into, and thus a 69

12 part of, the channel as shown in the lower part of Figure 3.7. he noise equivalent pulse response then has Fourier ransform P (f)/sn / (f) for an equivalent filtered-awgn channel. he concept of noise equivalence allows an analysis for AWGN to be valid (using the equivalent pulse response instead of the original pulse response). hen also colored noise is equivalent in its e ect to ISI, and furthermore the compensating equalizers that are developed later in this chapter can also be very useful on channels that originally have no ISI, but that do have colored noise. An AWGN channel with a notch in H(f) at some frequency is thus equivalent to a flat channel with H(f) =, but with narrow-band Gaussian noise at the same frequency as the notch, as illustrated in Figure Flat channel with narrowband noise Notched channel with white noise Equivalent channels magnitude 5 magnitude normalized radian frequency normalized radian frequency 3..3 he ISI-Channel Model Figure 3.8: wo noise-equivalent channels. n p ( t) pulse response p(t) x p, x ( t) k xk p ϕ p ( t) y p ( t) Σ! p * ϕ ( ) p t q * ( t) = ϕ ( t) ϕ ( t) p p y( t) yk s!a!m!p! l!e! a! t!!t!i!m!e!s! k!! d!i!s!c!r!e!t!e!! t!i!m!e!! r!e!c!e!i!v!e!r! xˆ k Figure 3.9: he ISI-Channel model. A model for linear ISI channels is shown in Figure 3.9. In this model, x k is scaled by kpk to form x p,k so that Ex p = Ex kpk. he additive noise is white Gaussian, although correlated Gaussian noise can be included by transforming the correlated-gaussian-noise channel into an equivalent white Gaussian noise channel using the methods in the previous subsection and illustrated in Figure 3.7. he channel output y p (t) is passed through a matched filter ' p( t) to generate y(t). hen, y(t) is sampled at the symbol rate and subsequently processed by a discrete time receiver. he following theorem illustrates that there is no loss in performance that is incurred via the matched-filter/sampler combination. heorem 3.. (ISI-Channel Model Su ciency) he discrete-time signal samples y k = y(k) in Figure 3.9 are su cient to represent the continuous-time ISI-model channel output y(t), if0 < kpk <. (i.e., a receiver with minimum P e can be designed that uses only the samples y k ). Sketch of Proof: Define ' p,k (t) = ' p (t k), (3.0) 70

13 where {' p,k (t)} k(,) is a linearly independent set of functions. he set {' p,k (t)} k(,) is related to a set of orthogonal basis functions { p,k (t)} k(,) by an invertible transformation (use Gram-Schmidt an infinite number of times). he transformation and its inverse are written { p,k (t)} k(,) = ({' p,k (t)} k(,) ) (3.) {' p,k (t)} k(,) = ({ p,k (t)} k(,) ), (3.) where is the invertible transformation. In Figure 3.0, the transformation outputs are the filter samples y(k). he infinite set of filters { p,k ( t)} k(,) followed by is o equivalent to an infinite set of matched filters to n' p,k ( t). By (3.0) this last set k(,) is equivalent to a single matched filter ' p( t), whose output is sampled at t = k to produce y(k). Since the set { p,k (t)} k(,) is orthonormal, the set of sampled filter outputs in Figure 3.0 are su cient to represent y p (t). Since is invertible (inverse is ), then by the theorem of reversibility in Chapter, the sampled matched filter output y(k) isasu cient representation of the ISI-channel output y p (t). QED.. ( ) * φ p, t y p ( t). ( ) * φ p, t Γ { y( k )} k (, ) φ * p, k ( t). t = 0 Figure 3.0: Equivalent diagram of ISI-channel model matched-filter/sampler Referring to Figure 3.9, y(t) = X k kpk x k q(t k)+n p (t) ' p( t), (3.3) 7

14 q(t) time in symbol periods Figure 3.: ISI in q(t) where q(t) = ' p (t) ' p( t) = p(t) p ( t) kpk. (3.4) he deterministic autocorrelation function q(t) is Hermitian (q ( t) =q(t)). Also, q(0) =, so the symbol x k passes at time k to the output with amplitude scaling kpk. he function q(t) can also exhibit ISI, as illustrated in Figure 3.. he q plotted q(t) corresponds to q k =[ ] or, equivalently, to the channel p(t) = (sinc(t/ )+.5 sinc((t )/ ).5 sinc((t )/ )), or P (D) = p ( +.5D)(.5D) (the notation P (D) is defined in Appendix A.). (he values for q k can be confirmed by convolving p(t) with its time reverse, normalizing, and sampling.) For notational brevity, let y k = y(k), q k = q(k), n k = n(k) wheren(t) = n p (t) ' p( y k = kpk x k + n k + kpk X x {z } {z} m q k m m6=k scaled input (desired) noise {z } ISI t). hus. (3.5) he output y k consists of the scaled input, noise and ISI. he scaled input is the desired informationbearing signal. he ISI and noise are unwanted signals that act to distort the information being transmitted. he ISI represents a new distortion component not previously considered in the analysis of Chapters and for a suboptimum SBS detector. his SBS detector is the same detector as in Chapters and, except used under the (false) assumption that the ISI is just additional AWGN. Such a receiver can be decidedly suboptimum when the ISI is nonzero. Using D-transform notation, (3.5) becomes Y (D) =X(D) kpk Q(D)+N(D) (3.6) 7

15 where Y (D) = P k= y k D k. If the receiver uses symbol-by-symbol detection on the sampled output y k, then the noise sample n k of (one-dimensional) variance N0 at the matched-filter output combines with ISI from the sample times m (m 6= k) in corrupting kpk x k. here are two common measures of ISI distortion. he first is Peak Distortion, which only has meaning for real-valued q(t): 3 Definition 3.. (Peak Distortion Criterion) If x max is the maximum value for x k, then the peak distortion is: D p = x max kpk X q m. (3.7) For q(t) in Figure 3. with x max = 3, D p =3 kpk( ) 3 p he peak distortion represents a worst-case loss in minimum distance between signal points in the signal constellation for x k, or equivalently 3 P e apple N e Q 4 kpk d min D p 5, (3.8) for symbol-by-symbol detection. Consider two matched filter outputs y k and yk 0 that the receiver attempts to distinguish by suboptimally using symbol-by-symbol detection. hese outputs are generated by two di erent sequences {x k } and {x 0 k }. Without loss of generality, assume y k >yk 0, and consider the di erence 3 m6=0 y k yk 0 = kpk 4(x k x 0 k)+ m6=k(x X m x 0 m)q k m 5 +ñ (3.9) he summation term inside the brackets in (3.9) represents the change in distance between y k and y 0 k caused by ISI. Without ISI the distance is while with ISI the distance can decrease to y k y 0 k kpk y k y 0 k kpk d min, (3.30) 4d min 3 X x max q m 5. (3.3) Implicitly, the distance interpretation in (3.3) assumes D p applekpkd min. 4 While peak distortion represents the worst-case ISI, this worst case might not occur very often in practice. For instance, with an input alphabet size M = 4 and a q(t) that spans 5 symbol periods, the probability of occurrence of the worst-case value (worst level occurring in all 4 ISI contributors) is 4 4 = , well below typical channel P e s in data transmission. Nevertheless, there may be other ISI patterns of nearly just as bad interference that can also occur. Rather than separately compute each possible combination s reduction of minimum distance, its probability of occurrence, and the resulting error probability, data transmission engineers more often use a measure of ISI called Mean-Square Distortion (valid for or dimensions): Definition 3..3 (Mean-Square Distortion) he Mean-Square Distortion is defined by: 8 9 < D ms = E : X = x p,m q k m (3.3) ; m6=k = Ex kpk X q m, (3.33) 3 In the real case, the magnitudes correspond to actual values. However, for complex-valued terms, the ISI is characterized by both its magnitude and phase. So, addition of the magnitudes of the symbols ignores the phase components, which may significantly change the ISI term. 4 On channels for which D p kpkd min,theworst-caseisioccurswhen D p kpkd min is maximum. m6=0 m6=0 73

16 where (3.33) is valid when the successive data symbols are independent and identically distributed with zero mean. In the example of Figure 3., the mean-square distortion (with Ex = 5) is D ms =5kpk ( ) 5(.078) he fact p.588 =.767 <.99 illustrates that D ms appledp.(he proof of this fact is left as an exercise to the reader.) he mean-square distortion criterion assumes (erroneously 5 ) that D ms is the variance of an uncorrelated Gaussian noise that is added to n k. With this assumption, P e is approximated by " # kpkdmin P e N e Q p + D. (3.34) ms One way to visualize ISI is through the eye diagram, some examples of which are shown in Figures 3. and 3.3. he eye diagram is similar to what would be observed on an oscilloscope, when the trigger is synchronized to the symbol rate. he eye diagram is produced by overlaying several successive symbol intervals of the modulated and filtered continuous-time waveform (except Figures 3. and 3.3 do not include noise). he Lorentzian pulse response p(t) =/( + (3t/ ) ) is used in both plots. For binary transmission on this channel, there is a significant opening in the eye in the center of the plot in Figure 3.. With 4-level PAM transmission, the openings are much smaller, leading to less noise immunity. he ISI causes the spread among the path traces; more ISI results in a narrower eye opening. Clearly increasing M reduces the eye opening. Figure 3.: Binary eye diagram for a Lorentzian pulse response. 5 his assumption is only true when x k is Gaussian. In very well-designed data transmission systems, x k is approximately i.i.d. and Gaussian, see Chapter 6, so that this approximation of Gaussian ISI becomes accurate. 74

17 Figure 3.3: 4-Level eye diagram for a Lorentzian pulse response. 75

18 3. Basics of the Receiver-generated Equivalent AWGN Figure 3.4 focuses upon the receiver and specifically the device shown generally as R. When channels have ISI, such a receiving device is inserted at the sampler output. he purpose of the receiver is to attempt to convert the channel into an equivalent AWGN that is also shown below the dashed line. Such an AWGN is not always exactly achieved, but nonetheless any deviation between the receiver output z k and the channel input symbol x k is viewed as additive white Gaussian noise. An SNR, as in Subsection 3.. can be used then to analyze the performance of the symbol-by-symbol detector that follows the receiver R. Usually, smaller deviation from the transmitted symbol means better performance, although not exactly so as Subsection 3.. discusses. y p ( t) matched filter * p (- t) y( t) sample at times k(/l) yk Receiver R z k SBS detector ˆx k - e k Equivalent AWGN x k + z { x SNR = } at time k k E zk - x k E Figure 3.4: Use of possibly suboptimal receiver to approximate/create an equivalent AWGN. Subsection 3..3 finishes this section with a discussion of the highest possible SNR that a designer could expect for any filtered AWGN channel, the so-called matched-filter-bound SNR, SNR MFB.his section shall not be specific as to the content of the box shown as R, but later sections will allow both linear and slightly nonlinear structures that may often be good choices because their performance can be close to SNR MFB. 3.. Receiver Signal-to-Noise Ratio Definition 3.. (Receiver SNR) he receiver SNR, SNR R for any receiver R with (pre-decision) output z k, and decision regions based on x k (see Figure 3.4) is SNR R = E x E e k, (3.35) where e k = x k z k is the receiver error. he denominator of (3.35) is the mean-square error MSE=E e k. When E [z k x k ]=x k, the receiver is unbiased (otherwise biased) with respect to the decision regions for x k. he concept of a receiver SNR facilitates evaluation of the performance of data transmission systems with various compensation methods (i.e. equalizers) for ISI. Use of SNR as a performance measure builds upon the simplifications of considering mean-square distortion, that is both noise and ISI are jointly considered in a single measure. he two right-most terms in (3.5) have normalized mean-square value + D ms. he SNR for the matched filter output y k in Figure 3.4 is the ratio of channel output sample energy Ēxkpk to the mean-square distortion + D ms. his SNR is often directly related to probability of error and is a function of both the receiver and the decision regions for the SBS detector. his text uses SNR consistently, replacing probability of error as a measure of comparative performance. 76

19 SNR is easier to compute than P e, independent of M at constant Ēx, and a generally good measure of performance: higher SNR means lower probability of error. he probability of error is di cult to compute exactly because the distribution of the ISI-plus-noise is not known or is di cult to compute. he SNR is easier to compute and this text assumes that the insertion of the appropriately scaled SNR (see Chapter - Sections.4 -.6)) into the argument of the Q-function approximates the probability of error for the suboptimum SBS detector. Even when this insertion into the Q-function is not su ciently accurate, comparison of SNR s for di erent receivers usually relates which receiver is better. 3.. Receiver Biases Receiver' (has'bias)' unbiased'receiver' k ( x u ) z =α + k SNR = SNRU + k /α z = x + u U, k k k-' SNR U = SNR u k Detector' based'on' x" (unbiased)' k = rm xk m + fm nk m m 0 m z so E & $% # = x x k k!" Figure 3.5: Receiver SNR concept. Figure 3.5 illustrates a receiver that somehow has tried to reduce the combination of ISI and noise. Any time-invariant receiver s output samples, z k, satisfy z k = (x k + u k ) (3.36) where is some positive scale factor that may have been introduced by the receiver and u k is an uncorrelated distortion u k = X r m x k m + X f m n k m. (3.37) m6=0 m he coe cients for residual intersymbol interference r m and the coe cients of the filtered noise f k will depend on the receiver and generally determine the level of mean-square distortion. he uncorrelated distortion has no remnant of the current symbol being decided by the SBS detector, so that E [u k /x k ] = 0. However, the receiver may have found that by scaling (reducing) the x k component in z k by that the SNR improves (small signal loss in exchange for larger uncorrelated distortion reduction). When E [z k /x k ]= x k 6= x k, the decision regions in the SBS detector are biased. Removal of the bias is easily achieved by scaling by / as also in Figure 3.5. If the distortion is assumed to be Gaussian noise, as is the assumption with the SBS detector, then removal of bias by scaling by / improves the probability of error of such a detector as in Chapter. (Even when the noise is not Gaussian as is the case with the ISI component, scaling the signal correctly improves the probability of error on the average if the input constellation has zero mean.) he following theorem relates the SNR s of the unbiased and biased decision rules for any receiver R: heorem 3.. (Unconstrained and Unbiased Receivers) Given an unbiased receiver R for a decision rule based on a signal constellation corresponding to x k, the maximum 77

20 unconstrained SNR corresponding to that same receiver with any biased decision rule is SNR R = SNR R,U +, (3.38) where SNR R,U is the SNR using the unbiased decision rule. Proof: From Figure 3.5, the SNR after scaling is easily SNR R,U = Ēx u. (3.39) he maximum SNR for the biased signal z k prior to the scaling occurs when is chosen to maximize the unconstrained SNR Ex SNR R = u + Ex. (3.40) Allowing for complex with phase and magnitude, the SNR maximization over alpha is equivalent to minimizing cos( )+ ( + ). (3.4) SNR R,U Clearly = 0 for a minimum and di erentiating with respect to yields + ( + SNR R,U )=0 or opt =/( + (SNR R,U ) ). Substitution of this value into the expression for SNR R finds SNR R = SNR R,U +. (3.4) hus, a receiver R and a corresponding SBS detector that have zero bias will not correspond to a maximum SNR the SNR can be improved by scaling (reducing) the receiver output by opt. Conversely, a receiver designed for maximum SNR can be altered slightly through simple output scaling by / opt to a related receiver that has no bias and has SNR thereby reduced to SNR R,U = SNR R. QED. o illustrate the relationship of unbiased and biased receiver SNRs, suppose an ISI-free AWGN channel has an SNR=0 with Ex = and u = N0 =.. hen, a receiver could scale the channel output 0 by = 0/. he resultant new error signal is e k = x k ( ) 0 n k, which has MSE=E[ e k ]= + 00 (.) =, and SNR=. Clearly, the biased SNR is equal to the unbiased SNR plus. he scaling has done nothing to improve the system, and the appearance of an improved SNR is an artifact of the SNR definition, which allows noise to be scaled down without taking into account the fact that actual signal power after scaling has also been reduced. Removing the bias corresponds to using the actual signal power, and the corresponding performance-characterizing SNR can always be found by subtracting from the biased SNR. A natural question is then Why compute the biased SNR? he answer is that the biased receiver corresponds directly to minimizing the mean-square distortion, and the SNR for the MMSE case will often be easier to compute. Figure 3.30 in Section 3.5 illustrates the usual situation of removing a bias (and consequently reducing SNR, but not improving P e since the SBS detector works best when there is no bias) from a receiver that minimizes mean-square distortion (or error) to get an unbiased decision. he bias from a receiver that maximizes SNR by equivalently minimizing mean-square error can then be removed by simple scaling and the resultant more accurate SNR is thus found for the unbiased receiver by subtracting from the more easily computed biased receiver. his concept will be very useful in evaluating equalizer performance in later sections of this chapter, and is formalized in heorem 3.. below. heorem 3.. (Unbiased MMSE Receiver heorem) Let R be any allowed class of receivers R producing outputs z k, and let R opt be the receiver that achieves the maximum signal-to-noise ratio SNR(R opt ) over all R Rwith an unconstrained decision rule. hen the receiver that achieves the maximum SNR with an unbiased decision rule is also R opt,and max SNR R,U = SNR(R opt ). (3.43) RR 78

21 Proof. From heorem 3.., for any R R, the relation between the signal-to-noise ratios of unbiased and unconstrained decision rules is SNR R,U = SNR R, so max [SNR R,U] = max [SNR R] = SNR Ropt. (3.44) RR RR QED. his theorem implies that the optimum unbiased receiver and the optimum biased receiver settings are identical except for any scaling to remove bias; only the SNR measures are di erent. For any SBS detector, SNR R,U is the SNR that corresponds to best P e. he quantity SNR R,U + is artificially high because of the bias inherent in the general SNR definition he Matched-Filter Bound he Matched-Filter Bound (MFB), also called the one-shot bound, specifies an upper SNR limit on the performance of data transmission systems with ISI. Lemma 3.. (Matched-Filter Bound SNR) he SNR MFB is the SNR that characterizes the best achievable performance for a given pulse response p(t) and signal constellation (on an AWGN channel) if the channel is used to transmit only one message. his SNR is SNR MFB = Ēxkpk N 0 (3.45) MFB denotes the square of the argument to the Q-function that arises in the equivalent one-shot analysis of the channel. Proof: Given a channel with pulse response p(t) and isolated input x 0, the maximum output sample of the matched filter is kpk x 0. he normalized average energy of this sample is kpk Ēx, whilethe corresponding noise sample energy is N0, SNR MFB = Ēxkpk N 0. QED. he probability of error, measured after the matched filter and prior to the symbol-by-symbol detector, satisfies P e N e Q( p MFB). When Ēx equals (d min /4)/apple, then MFB equals SNR MFB apple. Ine ectthe MFB forces no ISI by disallowing preceding or successive transmitted symbols. An optimum detector is used for this one-shot case. he performance is tacitly a function of the transmitter basis functions, implying performance is also a function of the symbol rate /. No other (for the same input constellation) receiver for continuous transmission could have better performance, if x k is an i.i.d. sequence, since the sequence must incur some level of ISI. he possibility of correlating the input sequence {x k } to take advantage of the channel correlation will be considered in Chapters 4 and 5. he following example illustrates computation of the MFB for several cases of practical interest: EXAMPLE 3.. (Binary PAM) For binary PAM, x p (t) = X k x k p(t k), (3.46) where x k = ± p Ex. he minimum distance at the matched-filter output is kpk d min = kpk d = kpk pex, soex = d min 4 and apple =. hen, MFB = SNR MFB. (3.47) hus for a binary PAM channel, the MFB (in db) is just the channel-output SNR, SNR MFB. If the transmitter symbols x k are equal to ± (Ex = ), then MFB = kpk, (3.48) where, again, = N0. he binary-pam P e is then bounded by P e Q( p SNR MFB ). (3.49) 79

22 EXAMPLE 3.. (M-ary PAM) For M-ary PAM, x k = ± d, ± 3d (M )d,...,± so apple =3/(M ). hus, MFB = r d = and 3E x M, (3.50) 3 M SNR MFB, (3.5) for M. If the transmitter symbols x k are equal to ±, ±3,,...,±(M ), then MFB = kpk. (3.5) Equation (3.5) is the same result as (3.48), which should be expected since the minimum distance is the same at the transmitter, and thus also at the channel output, for both (3.48) and (3.5). he M ary-pam P e is then bounded by r 3 SNRMFB P e ( /M ) Q( M ). (3.53) EXAMPLE 3..3 (QPSK) For QPSK, x k = ± d ± d, and d =p Ēx, soapple =. hus MFB = SNR MFB. (3.54) hus, for a QPSK (or 4SQ QAM) channel, MFB (in db) equals the channel output SNR. If the transmitter symbols x k are ± ±, then he best QPSK P e is then approximated by MFB = kpk. (3.55) P e Q( p SNR MFB ). (3.56) EXAMPLE 3..4 (M-ary QAM Square) For M-ary QAM, <{x k } = ± d, ± 3d,...,± (p M )d, ={x k } = ± d, ± 3d,..., ± (p M )d, (recall that < and = denote real and imaginary parts, respectively) and so apple =3/(M ). hus d = s 3Ēx M, (3.57) MFB = 3 M SNR MFB, (3.58) for M 4. If the real and imaginary components of the transmitter symbols x k equal ±, ±3,,...,±( p M ), then he best M ary QAM P e is then approximated by In general for square QAM constellations, MFB = kpk. (3.59) P e ( / p 3 SNRMFB M) Q(r ). (3.60) M MFB = 3 4 b SNR MFB. (3.6) 80

23 For the QAM Cross constellations, MFB = Ēx kpk 3 48 M 3 = b 3 SNR MFB. (3.6) For the suboptimum receivers to come in later sections, SNR U apple SNR MFB. As SNR U! SNR MFB, then the receiver is approaching the bound on performance. It is not always possible to design a receiver that attains SNR MFB, even with infinite complexity, unless one allows co-design of the input symbols x k in a channel-dependent way (see Chapters 4 and 5). he loss with respect to matched filter bound will be determined for any receiver by SNR u /SNR MFB apple, in e ect determining a loss in signal power because successive transmissions interfere with one another it may well be that the loss in signal power is an acceptable exchange for a higher rate of transmission. 8

24 3.3 Nyquist s Criterion Nyquist s Criterion specifies the conditions on q(t) =' p (t) ' p( t) for an ISI-free channel on which a symbol-by-symbol detector is optimal. his section first reviews some fundamental relationships between q(t) and its samples q k = q(k) in the frequency domain. q(k) = = = = Z X n= X n= Z Q(!)e!k d! (3.63) Z (n+) (n ) Z Q(!)e!k d! (3.64) Q(! + n where Q eq (!), the equivalent frequency response becomes Q eq (!) = )e (!+ n )k d! (3.65) Q eq (!)e!k d!, (3.66) X n= Q(! + n ). (3.67) he function Q eq (!) isperiodicin! with period. his function is also known as the folded or aliased spectrum of Q(!) because the sampling process causes the frequency response outside of the fundamental interval (, ) to be added (i.e. folded in ). Writing the Fourier ransform of the sequence q k as Q(e! )= P k= q ke!k leads to Q eq(!) =Q(e! ) = X k= q k e!k, (3.68) a well-known relation between the discrete-time and continuous-time representations of any waveform in digital signal processing. It is now straightforward to specify Nyquist s Criterion: heorem 3.3. (Nyquist s Criterion) A channel specified by pulse response p(t) (and resulting in q(t) = p(t) p ( t) kpk )isisi-freeifandonlyif Q(e! )= X n= Q(! + n )=. (3.69) Proof: By definition the channel is ISI-free if and only if q k = 0 for all k 6= 0 (recall q 0 =by definition). he proof follows directly by substitution of q k = k into (3.68). QED. Functions that satisfy (3.69) are called Nyquist pulses. One function that satisfies Nyquist s Criterion is t q(t) =sinc, (3.70) which corresponds to normalized pulse response ' p (t) = p t sinc. (3.7) he function q(k) =sinc(k) = k satisfies the ISI-free condition. One feature of sinc(t/ ) is that it has minimum bandwidth for no ISI. 8

25 sinc(t/) time (in units of ) Figure 3.6: he sinc(t/) function. No other function has this same minimum bandwidth and also satisfies the Nyquist Criterion. (Proof is left as an exercise to the reader.) he sinc(t/ ) function is plotted in Figure 3.6 for 0 apple t apple 0. he frequency (/ the symbol rate) is often construed as the maximum frequency of a sampled signal that can be represented by samples at the sampling rate. In terms of positive-frequencies, / represents a minimum bandwidth necessary to satisfy the Nyquist Criterion, and thus has a special name in data transmission: Definition 3.3. (Nyquist Frequency) he frequency! = or f = Nyquist frequency. 6 is called the 3.3. Vestigial Symmetry In addition to the sinc(t/ ) Nyquist pulse, data-transmission engineers use responses with up to twice the minimum bandwidth. For all these pulses, Q(!) = 0 for! >. hese wider bandwidth responses will provide more immunity to timing errors in sampling as follows: he sinc(t/ ) function decays in amplitude only linearly with time. hus, any sampling-phase error in the sampling process of Figure 3.9 introduces residual ISI with amplitude that only decays linearly with time. In fact for q(t) = sinc(t/ ), the ISI term P k6=0 q( +k) with a sampling timing error of 6= 0 is not absolutely summable, resulting in infinite peak distortion. he envelope of the time domain response decays more rapidly if the frequency response is smooth (i.e. continuously di erentiable). o meet this smoothness condition and also satisfy Nyquist s Criterion, the response must occupy a larger than minimum bandwidth that is between / and /. Aq(t) with higher bandwidth can exhibit significantly faster decay as t increases, thus reducing sensitivity to timing phase errors. Of course, any increase in bandwidth should be as small as possible, while still meeting other system requirements. he percent excess bandwidth 7, or percent roll-o, is a measure of the extra bandwidth. Definition 3.3. (Percent Excess Bandwidth) he percent excess bandwidth is determined from a strictly bandlimited Q(!) by finding the highest frequency in Q(!) for 6 his text distinguishes the Nyquist Frequency from the Nyquist Rate in sampling theory, where the latter is twice the highest frequency of a signal to be sampled and is not the same as the Nyquist Frequency here. 7 he quantity alpha used here is not a bias factor, and similarly Q( ) isameasureofisiandnottheintegralofa unit-variance Gaussian function uses should be clear to reasonable readers who ll understand that sometimes symbols are re-used in obviously di erenct contexts. 83

26 which there is nonzero energy transfer. hat is nonzero! apple( + ) Q(!) = 0! > ( + ).. (3.7) hus, if =.5 (a typical value), the pulse q(t) is said to have 5% excess bandwidth. Usually, data transmission systems have 0 apple apple. In this case in equation (3.69), only the terms n =, 0, + contribute to the folded spectrum and the Nyquist Criterion becomes = Q(e! ) (3.73) = Q! + + Q (!)+Q! apple! apple. (3.74) Further, recalling that q(t) =' p (t) ' p( t) is Hermitian and has the properties of an autocorrelation function, then Q(!) is real and Q(!) 0. For the region 0 apple! apple (for real signals), (3.74) reduces to = Q(e! ) (3.75) = Q (!)+Q!. (3.76) For complex signals, the negative frequency region ( apple! apple 0) should also have = Q(e! ) (3.77) = Q (!)+Q! +. (3.78) sinc(t/) squared time (in units of ) Figure 3.7: he sinc (t/ ) Function time-domain Any Q(!) satisfying (3.76) (and (3.78) in the complex case) is said to be vestigially symmetric with respect to the Nyquist Frequency. An example of a vestigially symmetric response with 00% excess bandwidth is q(t) =sinc (t/ ), which is shown in Figures 3.7 and

27 Q( ω)! Q e jω ( )! π π π π ω" π π ω" Figure 3.8: he sinc (t/ ) function frequency-domain 0%, 50%, and 00% raised cosine pulses % 50% 00% time (in units of ) Figure 3.9: Raised cosine pulse shapes time-domain 85

28 3.3. Raised Cosine Pulses 0%, 50%, and 00% raised cosine pulses % 50% 00% time (in units of ) Figure 3.0: Raised cosine pulse shapes time-domain he most widely used set of functions that satisfy the Nyquist Criterion are the raised-cosine pulse shapes: Definition (Raised-Cosine Pulse Shapes) he raised cosine family of pulse shapes (indexed by 0 apple apple ) is given by " # t cos t q(t) =sinc t, (3.79) and have Fourier ransforms 8 <! apple ( ) Q(!) = : sin (! ) ( ) apple! apple ( + ) 0 ( + ) apple!. (3.80) 86

29 Q( f ) Frequency (units of /) 0% 50% 00% Figure 3.: Raised Cosine Pulse Shapes frequency-domain he raised cosine pulse shapes are shown in Figure 3.0 (time-domain) and Figure 3. (frequencydomain) for =0,.5, and. When = 0, the raised cosine reduces to a sinc function, which decays asymptotically as t for t!, while for 6= 0, the function decays as t 3 for t!. Raised Cosine Derviation Many texts simply provide the form of the raised-cosine function, but for the intrigued student, this text does the inverse transform q RCR (t) = Z Q RCR (!) e!t d!. Before inserting the expression for Q RCR (!), the inverse transform of this even function simplifies to q RCR (t) = Z 0 Q RCR (!) cos (!t) d!, which is separated into integrals over successive positive frequency ranges when the formula from Section is inserted as q RCR (t) = Z ( ) / cos (!t) d! + Z (+ ) / apple 0 ( ) / sin (! / ) cos (!t) d!. (3.8) he first integral in Equation (3.8) easily is performed to obtain q RCR, (t) =( apple ( ) sinc )t, Equation (3.3. ) also can be rewritten using sin(a B) = sin(a) cos(b) cos(a)sin(b) as h i sin ( ) t q RCR, (t) = = t = sinc pit apple t sin t cos cos t t 87 cos t t t t cos sin t sin,

30 which will be a convenient form later in this derviation. he second integral in Equation (3.8) follows as the sum of two more integrals for the constant term (a) and the sinusoidal term (b) as q RCR,a (t) = t apple sin ( + ) t ( sin ) t Using the formula cos(a)sin(b) = [sin(a + B) sin(a B)], then q RCR,a(t) becomes: It follows then that q RCR,a (t) = cos t/ t sin. and t t q RCR, (t)+q RCR,a (t) =sinc cos. q RCR,b (t) = Z (+ ) / apple sin! cos(!t) d!. t ( ) / Using the formula sin(a) cos(b) = [sin(a + B)+sin(A B)], q RCR,b (t) = Z (+ ) / 4 ( ) / Z (+ ) / apple sin sin (! + t + t apple! apple / )+!t +sin then q RCR,b(t) becomes: apple +sin! (! / )!t d! = d! 4 ( ) / ( = apple apple ( + ) 4 + t cos ( ) + t cos + apple apple ( + ) t cos ( ) t t cos ( = apple apple ( + ) t 4 + t sin ( ) t + t sin + apple ( + ) t t sin + apple ) ( ) t t sin ( = t t + t sin cos + ) t t t sin cos ( t t ) t t = sin cos t = apple 8 t t t 4 t sin cos = apple 4 t t t t 4 (t/ ) sin cos apple 4 t t t = 4 (t/ ) sinc cos. hen, the sum is the RCR so q RCR (t) = q RCR, (t)+q RCR,a (t)+q RCR,b (t) apple 4 t t t = + 4 (t/ ) sinc cos 88 t + t t )

31 = = " sinc t t cos 4 (t/ ) " sinc t t cos ( t ) # #, which is the desired result Square-Root Splitting of the Nyquist Pulse he optimum ISI-free transmission system that this section has described has transmit-and-channel filtering ' p (t) and receiver matched filter ' p( t) such that q(t) =' p (t) ' p( t) or equivalently p(!) =Q / (!) (3.8) so that the matched filter and transmit/channel filters are square-roots of the Nyquist pulse. When h(t) = (t) or equivalently ' p (t) ='(t), the transmit filter (and the receiver matched filters) are squareroots of a Nyquist pulse shape. Such square-root transmit filtering is often used even when ' p (t) 6= '(t) and the pulse response is the convolution of h(t) with the square-root filter. he raised-cosine pulse shapes are then often used in square-root form, which is 8 p ><! apple p q ( ) Q(!) = > : sin (! ) / ( ) apple! apple ( + ). (3.83) 0 ( + ) apple! his expression can be inverted to the time-domain via use of the identity sin ( ) =.5( in Problem 3.6, to obtain cos( )), as ' p (t) = 4 p cos [ + ] t t sin([ ] ) + 4 t 4 t. (3.84) Another simpler form for seeing the value at time 0 (which is [/ p ] ( +4 / )) is h i ( ) sinc ( )t 4 t cos (+ ) t ' p (t) = p ( 4 t ) + p t ( 4 t ). (3.85) However, Equation (3.84) seems best to see the limiting value at t = ± 4, where the denominator is readily seen as zero. However, the numerator is also zero if one recalls that cos(x) =sin( / x). he value at these two times (through use of L Hopital s rule) is ' SR (± 4 )= p apple + sin + cos. (3.86) 4 4 he reader is reminded that the square-root of a Nyquist-satisfying pulse shape does not need to be Nyquist by itself, only in combination with its matched square-root filter. hus, this pulse is not zero at integer sampling instants. Square-Root Raised Cosine Derviation transform ' SR (t) = Z he square-root of a raised-cosine function has inverse p QRCR (!) e!t d!. Before inserting the expression for Q RCR (!), the inverse transform of this even function simplifies to ' SR (t) = Z 0 p QRCR (!) cos (!t) d!, 89

32 which is separated into integrals over successive positive frequency ranges when the formula from Section is inserted as ' SR (t) = Z ( ) / p Z s (+ ) / apple cos (!t) d! + 0 ( ) / sin (! / ) cos (!t) d!. (3.87) he first integral in Equation (3.87) easily is performed to obtain ' SR, (t) = ( ) p apple ( sinc )t. (3.88) Using the half-angle formula sin(x) = cos(x /) = sin x, the nd integral in Equation (3.87) expands into integrable terms as p Z (+ ) / h i ' SR, (t) = sin! cos(!t)d! ( ) / 4 4 p Z (+ ) / h i = sin! cos(!t)d! ( ) / 4 p Z (+ ) / apple ( + ) = sin! cos(!t)d!. 4 ( ) / 4 A basic trig formula is sin(a) cos(b) = [sin(a + B)+sin(A B)], so ' SR, (t) = = = = p p p ( + Z (+ ) / ( ) / Z (+ ) / ( ) / 4 4 t + cos sin 4 apple sin! apple 4 4 t cos! 4 4 p (4 t + ) 4 p + ( 4 t) cos cos apple! t + 4 apple! t + 4 ( + ) ( + ) 4 ( + ) 4 ( + ) t 4 apple ( + ) apple ( + ) ( 4 +!t +sin 4 apple +sin! (+ ) / ( ) / (+ ) / ( ) / (t + 4 ) ( + ) 4 cos t) ) ( + ) 4 cos 4 apple! apple ( apple ( ( + ) ( + ) t 4 )!t d! d! (t + 4 ) ( + ) 4 ) ( 4 t) ( + ) 4 Continuing with algebra at this point: p ( + ) t ( ) t ' SR, (t) = cos +sin (4 t + ) + p ( + ) t ( ) t cos sin ( 4 t) = p apple 4 t + + ( + ) t cos 4 t + p apple 4 t + + ( ) t sin 4 t = p apple ( + ) t (4 t) cos 90

33 = = + p apple 8 t ( ) t (4 t) sin 4 t cos (+ ) t 6 t p sin ( p t ( 4 t ) + t ( 4 t ) (+ ) t 4 t cos p t ( 4 t ) + 6 t / sinc ( ) t )t ( 4 t ). he first term from Equation (3.87) now is added so that apple ( ) ( )t 4 t cos (+ ) t 6 t / sinc ' SR (t) = p sinc + p t ( 4 t ) + ( )t ( 4 t ). Combining the sinc terms provides one convenient form of the square-root raised cosine of h i ( ) sinc ( )t 4 t cos (+ ) t ' SR (t) = p ( 4 t ) + p t ( 4 t ), from which limiting results like ' SR (0) = /sqrt follow easily. he preferred form for most engineering texts follows by combining the two fractional terms and simplifying. ' SR (t) =' p (t) = 4 p cos [ + ] t t sin([ ] ) + 4 t 4 t. 9

34 3.4 Linear Zero-Forcing Equalization his section examines the Zero-Forcing Equalizer (ZFE), which is the easiest type of equalizer to analyze and understand, but has inferior performance to some other equalizers to be introduced in later sections. he ISI model in Figure 3.9 is used to describe the zero-forcing version of the discrete-time receiver. he ZFE is often a first non-trivial attempt at a receiver R in Figure 3.4 of Section 3.. he ZFE sets R equal to a linear time-invariant filter with discrete impulse response w k that acts on y k to produce z k, which is an estimate of x k. Ideally, for symbol-by-symbol detection to be optimal, q k = k by Nyquist s Criterion. he equalizer tries to restore this Nyquist Pulse character to the channel. In so doing, the ZFE ignores the noise and shapes the signal y k so that it is free of ISI. From the ISI-channel model of Section 3. and Figure 3.9, y k = kpk x k q k + n k. (3.89) In the ZFE case, n k is initially viewed as being zero. he D-transform of y k is (See Appendix A.) he ZFE output, z k, has ransform Y (D) =kpk X(D) Q(D). (3.90) Z(D) =W (D) Y (D) =W (D) Q(D) kpk X(D), (3.9) and will be free of ISI if Z(D) = X(D), leaving the ZFE filter characteristic: Definition 3.4. (Zero-Forcing Equalizer) he ZFE transfer characteristic is W (D) = Q(D) kpk. (3.9) his discrete-time filter processes the discrete-time sequence corresponding to the matched-filter output. he ZFE is so named because ISI is forced to zero at all sampling instants k except k = 0. he receiver uses symbol-by-symbol detection, based on decision regions for the constellation defined by x k, on the output of the ZFE Performance Analysis of the ZFE he variance of the noise, which is not zero in practice, at the output of the ZFE is important in determining performance, even if ignored in the ZFE design of Equation (3.9). As this noise is produced by a linear filter acting on the discrete Gaussian noise process n k, it is also Gaussian. he designer can compute the discrete autocorrelation function (the bar denotes normalized to one dimension, so N = for the complex QAM case) for the noise n k as or more simply r nn,k = E n l n l k /N (3.93) = = = N 0 = N 0 Z Z Z Z Z Z E n p (t)n p(s) ' p(t l)' p (s (l k) )dtds (3.94) N 0 (t s)' p(t l)' p (s (l k) )dtds (3.95) ' p(t l)' p (t (l k) )dt (3.96) ' p(u)' p (u + k)du (letting u = t l) (3.97) = N 0 q k = N 0 q k, (3.98) R nn (D) = N 0 Q(D), (3.99) 9

35 where the analysis uses the substitution u = t l in going from (3.96) to (3.97), and assumes baseband equivalent signals for the complex case. he complex baseband-equivalent noise, has (one-dimensional) sample variance N0 at the normalized matched filter output. he noise at the output of the ZFE, n ZFE,k, has autocorrelation function he D-ransform of r ZFE,k is then R ZFE (D) = N 0 r ZFE,k = r nn,k w k w k. (3.00) Q(D) kpk Q (D) = N0 kpk Q(D). (3.0) he power spectral density of the noise samples n ZFE,k is then R ZFE (e! ). he (per-dimensional) variance of the noise samples at the output of the ZFE is the (per-dimensional) mean-square error between the desired x k and the ZFE output z k.since then where ZFE is computed as ZFE = Z R ZFE (e! )d! = ZFE = z k = x k + n ZFE,k, (3.0) Z Z N 0 kpk Q(e! ) d! = N0 kpk ZFE = N 0 kpk w 0, (3.03) Q(e! ) d! = w 0 kpk. (3.04) he center tap of a linear equalizer is w 0, or if the ZFE is explicity used, then w ZFE,0. his tap is easily shown to always have the largest magnitude for a ZFE and thus spotted readily when plotting the impulse response of any linear equalizer. From (3.0) and the fact that E [n ZFE,k ] = 0, E [z k /x k ]=x k, so that the ZFE is unbiased for a detection rule based on x k. he SNR at the output of the ZFE is SNR ZFE = Ēx ZFE = SNR MFB ZFE. (3.05) Computation of d min over the signal set corresponding to x k leads to a relation between Ēx, d min, and M, and the NNUB on error probability for a symbol-by-symbol detector at the ZFE output is (please, do not confuse the Q function with the channel autocorrelation Q(D)) P ZFE,e N e Q dmin ZFE. (3.06) Since symbol-by-symbol detection can never have performance that exceeds the MFB, apple ZFE kpk = Z d! Q(e! ) = ZFE (3.07) SNR MFB SNR ZFE (3.08) ZFE, (3.09) with equality if and only if Q(D) =. Finally, the ZFE equalization loss, / ZFE, defines the SNR reduction from the MFB: Definition 3.4. (ZFE Equalization Loss) he ZFE Equalization Loss, ZFE in decibels is SNRMFB kpk ZFE = 0 log 0 = 0 log ZFE SNR 0 = 0 log 0 kpk w ZFE,0, (3.0) ZFE and is a measure of the loss (in db) with respect to the MFB for the ZFE. he sign of the loss is often ignored since it is always a negative (or 0) number and only its magnitude is of concern. Equation (3.0) always thus provides a non-negative number. 93

36 3.4. Noise Enhancement he design of W (D) for the ZFE ignored the e ects of noise. his oversight can lead to noise enhancement, a that is unacceptably large, and the consequent poor performance. ZFE Q ( e jω ) W ZFE e jω ( ) Noise enhancement 0! π ω 0! π ω Figure 3.: Noise Enhancement Figure 3. hypothetically illustrates an example of a lowpass channel with a notch at the Nyquist Frequency, that is, Q(e! ) is zero at! =. Since W (e! )=/(kpk Q(e! )), then any noise energy near the Nyquist Frequency is enhanced (increased in power or energy), in this case so much that ZFE!. Even when there is no channel notch at any frequency, ZFE can be finite, but large, leading to unacceptable performance degradation. In actual computation of examples, the author has found that the table in able 3. is useful in recalling basic digital signal processing equivalences related to scale factors between continuous time convolution and discrete-time convolution. he following example clearly illustrates the noise-enhancement e ect: 94

37 time frequency ( t) P( ω) p continuous-time p k = p ω j ( ) ( ) k P e = n= ) P' ω + ( πn & $ % discrete-time y ( t) = x( t) p( t) Y( ω) = X( ω) P( ω) y ( k ) = x k p k Y ( e jω ) = X ( e jω ) P ( e jω ) Y jω ( e ) = X ( ω) P( ω) continuous-time discrete-time sampling theorem satisfied p p ( t) dt = P( ω ) = p dω = k = p k π = π π π jω ( ) P e dω continuous-time discrete-time (sampling thm satisfied) able 3.: Conversion factors of. 95

38 0 5 0 magnitude in db normalized frequency Figure 3.3: Magnitude of Fourier transform of (sampled) pulse response. EXAMPLE 3.4. (+.9D -ZFE)Suppose the pulse response of a (strictly bandlimited) channel used with binary PAM is P (!) = p +.9e!! apple 0! >. (3.) hen kpk is computed as kpk = = Z Z P (!) d! (3.) [ cos(!)] d! (3.3) =.8 =.8 = +.9. (3.4) he magnitude of the Fourier transform of the pulse response appears in Figure 3.3. hus, with p (!) as the Fourier transform of {' p (t)}, ( q p(!) =.8 +.9e!! apple. (3.5) 0! > 96

39 hen, (using the bandlimited property of P (!) in this example) hen Q(e! ) = p(!) (3.6) =.8 +.9e! (3.7) = cos(!).8 (3.8) Q(D) =.9D D.8. (3.9) W (D) = p.8d.9+.8d +.9D. (3.0) he magnitude of the Fourier transform of the ZFE response appears in Figure magnitude in db magnitude in in db db normalized frequency normalized frequency Figure 3.4: ZFE magnitude for example. he time-domain sample response of the equalizer is shown in Figure

40 8 6 4 response of of equalizer time in symbol periods time in symbol periods Figure 3.5: ZFE time-domain response. Computation of ZFE allows performance evaluation, 8 R 8 From a table of integrals ZFE = = = N 0 Z Z N 0 Q(e! d! (3.) ) kpk (.8/.8) N cos(!) Z = ( N 0 ) " = ( N 0 ) 4 p.8 p a+b cos u du = p an a b tan( u ) a b a+b d! (3.) du (3.3) cos u p.8.8 an p.8.8 tan u (3.4) (3.5) = ( N 0 ) p.8.8 (3.6) = N 0 (5.6). (3.7). 98

41 he ZFE-output SNR is SNR ZFE = Ēx 5.6 N0. (3.8) his SNR ZFE is also the argument of the Q-function for a binary detector at the ZFE output. Assuming the two transmitted levels are x k = ±, then we get leaving MFB = kpk =.8, (3.9) ZFE = 0 log 0 (.8 5.6) 9.8dB (3.30) for any value of the noise variance. his is very poor performance on this channel. No matter what b is used, the loss is almost 0 db from best performance. hus, noise enhancement can be a serious problem. Chapter 9 demonstrates a means by which to ensure that there is no loss with respect to the matched filter bound for this channel. With alteration of the signal constellation, Chapter 0 describes a means that ensures an error rate of 0 5 on this channel, with no information rate loss, which is far below the error rate achievable even when the MFB is attained. thus, there are good solutions, but the simple concept of a ZFE is not a good solution. Another example illustrates the generalization of the above procedure when the pulse response is complex (corresponding to a QAM channel): EXAMPLE 3.4. (QAM:.5D +(+.5 ).5 D Channel) Given a baseband equivalent channel p(t) = p t + sinc + + t t sinc 4 sinc, (3.3) the discrete-time channel samples are p k = p apple +,, 4. (3.3) his channel has the transfer function of Figure

42 6 complex baseband channel pulse response magnitude 4 magnitude in db normalized frequency f Figure 3.6: Fourier transform magnitude for pulse response for complex channel example. he pulse response norm (squared) is kpk = ( ) =.565. (3.33) hen q k is given by or q k = q(k) = ' p,k ' p, k =.565 Q(D) = apple.565 hen, Q(D) factors into 4 D ( + )D apple 4, 5 8 ( + ),.565, 5 8 ( + ), 4 (3.34) 5 8 ( + )D + 4 D (3.35) Q(D) =.565 (.5 D).5D +.5 D (.5D), (3.36) and W ZFE (D) = Q(D)kpk (3.37) 00

43 or W ZFE (D) = p.565 (.5 D +.65( + )D ( + )D +.5 D ). (3.38) he Fourier transform of the ZFE is in Figure magnitude in db magnitude in db normalized frequency normalized frequency Figure 3.7: Fourier transform magnitude of the ZFE for the complex baseband channel example. he real and imaginary parts of the equalizer response in the time domain are shown in Figure 3.8 0

44 Figure 3.8: ime-domain equalizer coe cients for complex channel. hen or ZFE = Z ZFE = N 0 N 0 kpk Q(e! d! ), (3.39) w 0 kpk, (3.40) where w 0 is the zero (center) coe cient of the ZFE. his coe cient can be extracted from W ZFE (D) = p apple D ( 4 ).565 {(D + )(D ) (D.5) (D +.5 )} = p apple A.565 D + B D + + terms not contributing to w 0, where A and B are coe cients in the partial-fraction expansion (the residuez and residue commands in Matlab can be very useful here): Finally and A = B = 4( 4 ) ( + )( +.5 )(.5) = = (3.4) 4( 4 ) (.5)( )(.5 ) = = (3.4) W ZFE (D) = p apple A(.5) B(.5 ) terms, (3.43).5D (.5 D) w 0 = p.565 [(.5)( ).5 ( )] (3.44) he ZFE loss can be shown to be = p.565(.57) =.96 (3.45) ZFE = 0 log 0 (w 0 kpk) =3.9 db, (3.46) which is better than the last channel because the frequency spectrum is not as near zero in this complex example as it was earlier on the PAM example. Nevertheless, considerably better performance is also possible on this complex channel, but not with the ZFE. 0

45 o compute P e for 4QAM, the designer calculates p P e Q SNRMFB 3.9 db. (3.47) If SNR MFB = 0dB, then P e. 0. If SNR MFB = 7.4dB, then P e If SNR MFB = 3.4dB for 6QAM, then P e

46 3.5 Minimum Mean-Square Error Linear Equalization he Minimum Mean-Square Error Linear Equalizer (MMSE-LE) balances a reduction in ISI with noise enhancement. he MMSE-LE always performs as well as, or better than, the ZFE and is of the same complexity of implementation. Nevertheless, it is slightly more complicated to describe and analyze than is the ZFE. he MMSE-LE uses a linear time-invariant filter w k for R, but the choice of filter impulse response w k is di erent than the ZFE. he MMSE-LE is a linear filter w k that acts on y k to form an output sequence z k that is the best MMSE estimate of x k. hat is, the filter w k minimizes the Mean Square Error (MSE): Definition 3.5. (Mean Square Error (for the LE)) he LE error signal is given by e k = x k w k y k = x k z k. (3.48) he Minimum Mean Square Error (MMSE) for the linear equalizer is defined by MMSE-LE =min w k E x k z k. (3.49) he MSE criteria for filter design does not ignore noise enhancement because the optimization of this filter compromises between eliminating ISI and increasing noise power. Instead, the filter output is as close as possible, in the Minimum MSE sense, to the data symbol x k Optimization of the Linear Equalizer Using D-transforms, E(D) =X(D) W (D)Y (D) (3.50) By the orthogonality principle in Appendix A, at any time k, the error sample e k must be uncorrelated with any equalizer input signal y m.succinctly, Evaluating (3.5), using (3.50), yields where (N = for PAM, N = for Quadrature Modulation) 9 E E(D)Y (D ) =0. (3.5) 0= R xy (D) W (D) R yy (D), (3.5) R xy (D) = E X(D)Y (D ) /N = kpkq(d)ēx R yy (D) = E Y (D)Y (D ) /N = kpk Q (D)Ēx + N 0 Q(D) =Q(D) kpk Q(D)Ēx + N 0. hen the MMSE-LE becomes W (D) = R xy (D) R yy (D) = kpk (Q(D)+/SNR MFB ). (3.53) he MMSE-LE di ers from the ZFE only in the additive positive term in the denominator of (3.53). he transfer function for the equalizer W (e! ) is also real and positive for all finite signal-to-noise ratios. his small positive term prevents the denominator from ever becoming zero, and thus makes the MMSE-LE well defined even when the channel (or pulse response) is zero for some frequencies or frequency bands. Also W (D) =W (D ). 9 he expression R xx(d) = E X(D)X (D ) is used in a symbolic sense, since the terms of X(D)X (D )areof P P the form k x kx k j,sothatweareimplyingtheadditionaloperationlim K![/(K +)] on the sum in KapplekappleK such terms. his is permissable for stationary (and ergodic) discrete-time sequences. 04

47 Q j ( e ω ) W ZFE e jω ( ) π!ω!!ω! 0! 0! SNR p W MMSE e LE jω ( ) π Figure 3.9: Example of MMSE-LE versus ZFE. Figure 3.9 repeats Figure 3. with addition of the MMSE-LE transfer characteristic. he MMSE- LE transfer function has magnitude Ēx/ at! =, while the ZFE becomes infinite at this same frequency. his MMSE-LE leads to better performance, as the next subsection computes Performance of the MMSE-LE he MMSE is the time-0 coe cient of the error autocorrelation sequence R ee (D) = E E(D)E (D ) /N (3.54) = Ēx W (D ) R xy (D) W (D) R xy(d )+W (D) R yy (D)W (D ) (3.55) = Ēx W (D) R yy (D)W (D ) (3.56) = Ēx = Ēx = he third equality follows from Q(D) kpk Q(D)Ēx + N0 kpk (Q(D)+/SNR MFB ) (3.57) ĒxQ(D) (Q(D)+/SNR MFB ) N 0 kpk (Q(D)+/SNR MFB ) W (D) R yy (D)W (D ) = W (D) R yy (D) R yy (D) R xy (D ) = W (D) R xy(d ), (3.58) (3.59) and that W (D) R yy (D)W (D ) = W (D) R yy (D)W (D ). he MMSE then becomes MMSE-LE = Z R ee (e! )d! = Z N 0 d! kpk (Q(e! )+/SNR MFB ) By recognizing that W (e! ) is multiplied by the constant N0 /kpk in (3.60), then. (3.60) From comparison of (3.60) and (3.03), that MMSE-LE = w 0 N 0 kpk. (3.6) MMSE-LE apple ZFE, (3.6) 05

48 with equality if and only if SNR MFB!. Furthermore, since the ZFE is unbiased, and SNR MMSE LE = SNR MMSE LE,U + and SNR MMSE LE,U are the maximum-snr corresponding to unconstrained and unbiased linear equalizers, respectively, SNR ZFE apple SNR MMSE LE,U = Ēx MMSE-LE apple SNR MFB. (3.63) One confirms that the MMSE-LE is a biased receiver by writing the following expression for the equalizer output Z(D) = W (D)Y (D) (3.64) = (Q(D)kpkX(D)+N(D)) kpk (Q(D)+/SNR MFB ) (3.65) = X(D) /SNR MFB N(D) X(D)+ Q(D)+/SNR MFB kpk (Q(D)+/SNR MFB ), (3.66) for which the x k -dependent residual ISI term contains a component signal-basis term = /SNR MFB w 0 kpk x k (3.67) = /SNR MFB MMSE-LE kpk N 0 x k (3.68) = MMSE-LE x k Ēx (3.69) = SNR MMSE x k LE. (3.70) (3.7) So z k = SNR MMSE x k LE e 0 k where e0 k is the error for unbiased detection and R. he optimum unbiased receiver with decision regions scaled by SNR MMSE LE (see Section 3..) has the signal energy given by Ēx. (3.7) SNR MMSE A new error for the scaled decision regions is e 0 k =( /SNR MMSE LE) x k z k = e k SNR x k, MMSE LE which is also the old error with the x k dependent term removed. Since e 0 k and x k are then independent, then e = MMSE-LE = e + Ēx, (3.73) 0 leaving e 0 = MMSE-LE SNR MMSE LE LE SNR MMSE LE Ēx = SNR MMSE LE MMSE-LE SNR MMSE he SNR for the unbiased MMSE-LE then becomes (taking the ratio of (3.7) to e 0) LE Ēx = Ēx (SNR MMSE LE ) SNR MMSE LE (3.74). SNR MMSE LE,U = (SNR MMSE LE ) SNR Ēx MMSE LE Ēx (SNR MMSE LE ) SNR MMSE LE = SNR MMSE LE, (3.75) which corroborates the earlier result on the relation of the optimum biased and unbiased SNR s for any particular receiver structure (at least for the LE structure). he unbiased SNR is SNR MMSE LE,U = Ēx MMSE-LE, (3.76) 06

49 which is the performance level that this text always uses because it corresponds to the best error probability for an SBS detector, as was discussed earlier in Section 3.. Figure 3.30 illustrates the concept with the e ect of scaling on a 4QAM signal shown explicitly. Again, MMSE receivers (of which the MMSE-LE is one) reduce noise power at the expense of introducing a bias, the scaling up removes the bias, and ensures the detector achieves the best P e it can. MMSE$ Receiver$ (has$bias)$ z = α x + e k z E ' k $ = α x x k %& k "# k k SNR = SNRU + /α z U, k = x k + e U,-$ k SNR U = SNR Detector$ based$on$ x" (unbiased)$ unbiased$output,$snr U $ α α $ biased$output,$snr$ Figure 3.30: Illustration of bias and its removal. Since the ZFE is also an unbiased receiver, (3.63) must hold. he first inequality in (3.63) tends to equality if the ratio Ēx!or if the channel is free of ISI (Q(D) = ). he second inequality tends to equality if Ēx! 0 or if the channel is free of ISI (Q(D) = ). he unbiased MMSE-LE loss with respect to the MFB is SNR MFB kpk MMSE LE = = MMSE-LE SNR MMSE LE,U Ēx Ēx MMSE-LE. (3.77) he kpk MMSE-LE in (3.77) is the increase in noise variance of the MMSE-LE, while the term Ēx term represents the loss in signal power at the equalizer output that accrues to lower Ēx MMSE-LE noise enhancement. he MMSE-LE also requires no additional complexity to implement and should always be used in place of the ZFE when the receiver uses symbol-by-symbol detection on the equalizer output. he error is not necessarily Gaussian in distribution. Nevertheless, engineers commonly make this assumption in practice, with a good degree of accuracy, despite the potential non-gaussian residual ISI component in MMSE-LE. his text also follows this practice. hus, papplesnrmmse P e N e Q, (3.78) LE,U where apple depends on the relation of Ēx to d min for the particular constellation of interest, for instance apple =3/(M ) for Square QAM. he reader may recall that the symbol Q is used in two separate ways in these notes, for the Q-function, and for the transform of the matched-filter-pulse-response cascade. he actual meaning should always be obvious in context.) 07

50 3.5.3 Examples Revisited his section returns to the earlier ZFE examples to compute the improvement of the MMSE-LE on these same channels. EXAMPLE 3.5. (PAM - MMSE-LE) he pulse response of a channel used with binary PAM is again given by P (!) = p +.9e!! apple 0! >. (3.79) Let us suppose SNR MFB = 0dB (= Ēxkpk / N0 ) and that E x =. he equalizer is W (D) =.9 kpk.8 D D. (3.80) Figure 3.3 shows the frequency response of the equalizer for both the ZFE and MMSE-LE. Clearly the MMSE-LE has a lower magnitude in its response. 50 Equalizers for real-channel example amplitude in db normalized frequency Figure 3.3: Comparison of equalizer frequency-domain responses for MMSE-LE and ZFE on baseband example. he MMSE-LE is computed as MMSE-LE = Z N 0 d! (3.8) cos(!)+.8/0 08

51 = N 0 p (3.8).99.8 = N 0 (.75), (3.83) which is considerably smaller than ZFE. he SNR for the MMSE-LE is SNR MMSE LE,U =.75(.8).75(.8) = 3.7 (5.7dB). (3.84) he loss with respect to the MFB is 0dB - 5.7dB = 4.3dB. his is 5.5dB better than the ZFE (9.8dB - 4.3dB = 5.5dB), but still not good for this channel. Figure 3.3 compares the frequency-domain responses of the equalized channel. 5! 0! Frequency-domain comparison of equalized channel responses! W ZFE p Q -5! W MMSE LE p Q amplitude in db! -0! -5! -0! -5! -30! -0.5! -0.4! -0.3! -0.! -0.! 0! 0.! 0.! 0.3! 0.4! 0.5! normalized frequency! Figure 3.3: Comparison of equalized frequency-domain responses for MMSE-LE and ZFE on baseband example. Figure 3.33 compares the time-domain responses of the equalized channel. he MMSE-LE clearly does not have an ISI-free response, but mean-square error/distortion is minimized and the MMSE-LE has better performance than the ZFE. 09

52 time-domain comparison of equalized channel response, MMSE-LE vs ZFE!.!! W ZFE p Q 0.8! amplitude! 0.6! 0.4! W MMSE LE p Q 0.! 0! -0.! -5! -0! -5! 0! 5! 0! time in symbol periods! Figure 3.33: Comparison of equalized time-domain responses for MMSE-LE and ZFE on baseband example. he relatively small energy near the Nyquist Frequency in this example is the reason for the poor performance of both linear equalizers (ZFE and MMSE-LE) in this example. o improve performance further, decision-assisted equalization is examined in Section 3.6 and in Chapter 5 (or codes combined with equalization, as discussed in Chapters 0 and ). Another yet better method appears in Chapter 4. he complex example also follows in a straightforward manner. EXAMPLE 3.5. (QAM - MMSE-LE) Recall that the equivalent baseband pulse response samples were given by p k = p apple +,, 4 Again, SNR MFB = 0dB (= Ēxkpk / N0 ), Ēx =, and thus. (3.85) MMSE-LE = w 0 N 0 kpk. (3.86) he equalizer is, noting that N0 W (D) = = Ēx kpk SNR MFB =.565/0 =.565, (Q(D)+/SNR MFB ) kpk (3.87) 0

53 or = p D +.65( + )D +.565( +.).65( + )D +.5 D. (3.88) Figure 3.34 compares the MMSE-LE and ZFE equalizer responses for this same channel. he MMSE-LE high response values are considerably more limited than the ZFE values. 5! MMSE%LE!for!Complex!Channel! 0! W ZFE amplitude in db! 5! 0! -5! -0! WMMSE LE -5! -0.5! -0.4! -0.3! -0.! -0.! 0! 0.! 0.! 0.3! 0.4! 0.5! normalized frequency! Figure 3.34: Comparison of MMSE-LE and ZFE for complex channel example. he roots of Q(D)+/SNR MFB, or of the denominator in (3.88), are.63 D =.76 = (3.89).06 D =.76 = (3.90).63 D =.456 = (3.9).06 D =.456 = , (3.9) and (3.88) becomes p.565d /(.5 ) W (D) = (D.6.63 )(D.6.06 )(D )(D (3.93) ) or A W (D) =.63 D + B , (3.94).6 D.6 where the second expression (3.94) has ignored any terms in the partial fraction expansion that will not contribute to w 0.hen, p.565( 4 )( ) A =.63 ( )( )( ) = p.565( ) = (3.95) p.565( 4 )( ) B =.06 ( )( )( ) = p.565(.0.805) = (3.96)

54 hen, from (3.94), hen w 0 = A( )+B( )= p.565(.5). (3.97) MMSE-LE =.5 N 0 =.5(.565) =.758, (3.98) [.5(.565)] SNR MMSE LE,U = =4.69 ( 6.7 db ), (3.99).5(.565) which is 3.3 db below the matched filter bound SNR of 0dB, or equivalently, MMSE LE = 0 log 0 (0/4.69) = 0 log 0 (.3) = 3.3 db, (3.00) but.6 db better than the ZFE. It is also possible to do considerably better on this channel, but structures not yet introduced will be required. Nevertheless, the MMSE-LE is one of the most commonly used equalization structures in practice. Figure 3.34 compares the equalized channel responses for both the MMSE-LE and ZFE. While the MMSE-LE performs better, there is a significant deviation from the ideal flat response. his deviation is good and gains.6 db improvement. Also, because of biasing, the MMSE-LE output is everywhere slightly lower than the ZFE output (which is unbiased). his bias can be removed by scaling by 5.69/ Fractionally Spaced Equalization o this point in the study of the discrete time equalizer, a matched filter ' p( t) precedes the sampler, as was shown in Figure 3.9. While this may be simple from an analytical viewpoint, there are several practical problems with the use of the matched filter. First, ' p( t) is a continuous time filter and may be much more di cult to design accurately than an equivalent digital filter. Secondly, the precise sampling frequency and especially its phase, must be known so that the signal can be sampled when it is at maximum strength. hird, the channel pulse response may not be accurately known at the receiver, and an adaptive equalizer (see Chapter 7), is instead used. It may be di cult to design an adaptive, analog, matched filter. xk p x p, k ϕ p ( t) n p ( t) x p ( t) y p ( t) +! anti-alias! filter! gain! y( t) sample! at times! k(/l)! y k Frac'onally,spaced!equalizer! w k zk Figure 3.35: Fractionally spaced equalization. For these reasons, sophisticated data transmission systems often replace the matched-filter/sampler/equalizer system of Figure 3.9 with the Fractionally Spaced Equalizer (FSE) of Figure Basically, the sampler and the matched filter have been interchanged with respect to Figure 3.9. Also the sampling rate has been increased by some rational number l (l >). he new sampling rate is chosen su ciently high so as to be greater than twice the highest frequency of x p (t). hen, the matched filtering operation and the equalization filtering are performed at rate l/, and the cascade of the two filters is realized as a single filter in practice. An anti-alias or noise-rejection filter always precedes the sampling device (usually an analog-to-digital converter). his text assumes this filter is an ideal lowpass filter with gain p transfer

55 over the frequency range l apple! apple l. he variance of the noise samples at the filter output will then be N0 l per dimension. In practical systems, the four most commonly found values for l are 4 3,, 3, and 4. he major drawback of the FSE is that to span the same interval in time (when implemented as an FIR filter, as is typical in practice), it requires a factor of l more coe cients, leading to an increase in memory by a factor of l. he FSE outputs may also appear to be computed l times more often in real time. However, only th l of the output samples need be computed (that is, those at the symbol rate, as that is when we need them to make a decision), so computation is approximately l times that of symbol-spaced equalization, corresponding to l times as many coe cients to span the same time interval, or equivalently, to l times as many input samples per symbol. he FSE will digitally realize the cascade of the matched filter and the equalizer, eliminating the need for an analog matched filter. (he entire filter for the FSE can also be easily implemented adaptively, see Chapter 7.) he FSE can also exhibit a significant improvement in sensitivity to sampling-phase errors. o investigate this improvement briefly, in the original symbol-spaced equalizers (ZFE or MMSE-LE) may have the sampling-phase on the sampler in error by some small o set t 0. hen, the sampler will sample the matched filter output at times k + t 0, instead of times k. hen, y(k + t 0 )= X m x m kpk q(k m + t 0 )+n k, (3.0) which corresponds to q(t)! q(t + t 0 ) or Q(!)! Q(!)e!t0. For the system with sampling o set, Q(e! )! X Q(! n n n (! )e )t0, (3.0) which is no longer nonnegative real across the entire frequency band. In fact, it is now possible (for certain nonzero timing o sets t 0, and at frequencies just below the Nyquist Frequency) that the two aliased frequency characteristics Q(!)e!t0 (! and Q(! )e )t0 add to approximately zero, thus producing a notch within the critical frequency range (-, ). hen, the best performance of the ensuing symbol-spaced ZFE and/or MMSE-LE can be significantly reduced, because of noise enhancement. he resulting noise enhancement can be a major problem in practice, and the loss in performance can be several db for reasonable timing o sets. When the anti-alias filter output is sampled at greater than twice the highest frequency in x p (t), then information about the entire signal waveform is retained. Equivalently, the FSE can synthesize, via its transfer characteristic, a phase adjustment (e ectively interpolating to the correct phase) so as to correct the timing o set, t 0, in the sampling device. he symbol-spaced equalizer cannot interpolate to the correct phase, because no interpolation is correctly performed at the symbol rate. Equivalently, information has been lost about the signal by sampling at a speed that is too low in the symbol-spaced equalizer without matched filter. his possible notch is an example of information loss at some frequency; this loss cannot be recovered in a symbol-spaced equalizer without a matched filter. In e ect, the FSE equalizes before it aliases (aliasing does occur at the output of the equalizer where it decimates by l for symbol-by-symbol detection), whereas the symbol-spaced equalizer aliases before it equalizes; the former alternative is often the one of choice in practical system implementation, if the extra memory and computation can be accommodated. E ectively, with an FSE, the sampling device need only be locked to the symbol rate, but can otherwise provide any sampling phase. he phase is tacitly corrected to the optimum phase inside the linear filter implementing the FSE. he sensitivity to sampling phase is channel dependent: In particular, there is usually significant channel energy near the Nyquist Frequency in applications that exhibit a significant improvement of the FSE with respect to the symbol-spaced equalizer. In channels with little energy near the Nyquist Frequency, the FSE is avoided, because it provides little performance gain, and is significantly more complex to implement (more parameters and higher sampling rate). Infinite-Length FSE Settings o derive the settings for the FSE, this section assumes that l, the oversampling factor, is an integer. he sampled output of the anti-alias filter can be decomposed into l 3

56 sampled-at-rate-/ interleaved sequences with D-transforms Y 0 (D), Y (D),..., Y l corresponds to the sample sequence y[k i/l]. hen, (D), where Y l (D) Y i (D) =P i (D) X(D)+N i (D) (3.03) where P i (D) is the transform of the symbol-rate-spaced-i th -phase-of-p(t) sequencep[k (i )/l], and similarly N i (D) is the transform of a symbol-rate sampled white noise sequence with autocorrelation function R nn (D) =l N0 per dimension, and these noise sequences are also independent of one another. A column vector transform is 6 Y (D) = 4 Y 0 (D) = P (D)X(D)+N(D). (3.04) Y l (D) Also P (D) = 6 4 P 0 (D) P l. (D) and N(D) = 4 N 0 (D) N l. (D) (3.05) By considering the FSE output at sampling rate /, the interleaved coe cients of the FSE can also be written in a row vector W (D) =[W (D),..., W l (D)]. hus the FSE output is Z(D) =W (D)Y (D). (3.06) Again, the orthogonality condition says that E(D) = X(D) which can be written in vector form as Z(D) should be orthogonal to Y (D), E E(D)Y (D ) = R xy (D) W (D)R YY (D) =0, (3.07) where MMSE-FSE filter setting is then R xy (D) = E X(D)Y (D ) = ĒxP (D ) (3.08) R YY (D) = E Y (D)Y (D ) = ĒxP (D)P (D )+l N0 I. (3.09) W (D) =R xy (D)R YY (D) =P (D ) P (D)P (D )+l/snr he corresponding error sequence has autocorrelation function. (3.0) R ee (D) =Ēx R xy (D)R YY (D)R Y x (D) = l N0 P (D )P (D)+l/SNR. (3.) he MMSE is then computed as MMSE MMSE FSE = Z l N0 d! kp (e! )k + l/snr =MMSE MMSE LE, (3.) where kp (e! )k = P l i= P i(e! ). he SNR s, biased and unbiased, are also then exactly the same as given for the MMSE-LE, as long as the sampling rate exceeds twice the highest frequency of P (f). he reader is cautioned against letting N0! 0 to get the ZF-FSE. his is because the matrix R YY (D) will often be singular when this occurs. o avoid problems with singularity, an appropriate pseudoinverse, which zeros the FSE characteristic when P (!) = 0 is recommended. 4

57 Passband Equalization his chapter so far assumed for the complex baseband equalizer that the passband signal has been demodulated according to the methods described in Chapter before filtering and sampling. An alternative method, sometimes used in practice when! c is not too high (or intermediate-frequency up/down conversion is used), is to commute the demodulation and filtering functions. Sometimes this type of system is also called direct conversion, meaning that no carrier demodulation occurs prior to sampling. his commuting can be done with either symbol-spaced or fractionally spaced equalizers. he main reason for the interchange of filtering and demodulation is the recovery of the carrier frequency in practice. Postponing the carrier demodulation to the equalizer output can lead to significant improvement in the tolerance of the system to any error in estimating the carrier frequency, as Chapter 6 investigates. In the present (perfect carrier phase lock) development, passband equalization and baseband equalization are exactly equivalent and the settings for the passband equalizer are identical to those of the corresponding baseband equalizer, other than a translation in frequency by! c radians/sec. Passband equalization is best suited to CAP implementations (See Section 4 of Chapter ) where the complex channel is the analytic equivalent. he complex equalizer then acts on the analytic equivalent channel and signals to eliminate ISI in a MMSE sense. When used with CAP, the final rotation shown in Figure 3.36 is not necessary such a rotation is only necessary with a QAM transmit signal. x p ( t) n p ( t) ( t) Σ! y p jω t ( ) phase! split! ϕ t e * c jωc k p wk e /! j c e ω k!! Decision! Figure 3.36: Passband equalization, direct synthesis of analytic-equivalent equalizer. he passband equalizer is illustrated in Figure he phase splitting is the forming of the analytic equivalent with the use of the Hilbert ransform, as discussed in Chapter. he matched filtering and equalization are then performed on the passband signal with digital demodulation to baseband deferred to the filter output. he filter w k can be a ZFE, a MMSE-LE, or any other desired setting (see Chapter 4 and Sections 3.5 and 3.6). In practice, the equalizer can again be realized as a fractionally spaced equalizer by interchanging the positions of the matched filter and sampler, increasing the sampling rate, and absorbing the matched filtering operation into the filter w k, which would then be identical to the (modulated) baseband-equivalent FSE with output sample rate decimated appropriately to produce outputs only at the symbol sampling instants. Often in practice, the cross-coupled nature of the complex equalizer W (D) is avoided by sampling the FSE at a rate that exceeds twice the highest frequency of the passband signal y(t) prior to demodulation. (If intermediate frequency (IF) demodulation is used, then twice the highest frequency after the IF.) In this case the phase splitter (contains Hilbert transform in parallel with a unit gain) is also absorbed into the equalizer, and the same sampled sequence is applied independently to two equalizers, one of which estimates the real part, and one the imaginary part, of the analytic representation of the data sequence x k e!ck. he two filters can sometimes (depending on the sampling rate) be more cost e ective to implement than the single complex filter, especially when one considers that the Hilbert transform is incorporated into the equalizer. his approach is often taken with adaptive implementations of the FSE, as the Hilbert transform is then implemented more exactly adaptively than is possible with fixed design. 0 his is often called Nyquist Inband Equalization. A particularly common variant of this type of equalization is to sample at rate / both of the outputs of a continuous-time phase splitter, one stream of samples staggered by /4 with respect to the other. he corresponding / complex 0 However, this structure also slows convergence of many adaptive implementations. 5

58 equalizer can then be implemented adaptively with four independent adaptive equalizers (rather than the two filters, real and imaginary part, that nominally characterize complex convolution). he adaptive filters will then correct for any imperfections in the phase-splitting process, whereas the two filters with fixed conjugate symmetry could not. 6

59 x ( ) k p t n p ( t) x p ( t) y p ( t) * +# ( ) ϕ p t W (D) s#a#m#p#l#e## a#t# #t#i#m#e#s# # k## Whitened#matched#filter# F#e#e#d#f#o#r#w#a#r#d# # f#i#l#t#e#r# z k +# R# ' z k B( D) F#e#e#d#b#a#c#k# # f#i#l#t#e#r# (any#bias# removal# included)# SBS# xˆk Figure 3.37: Decision feedback equalization. 3.6 Decision Feedback Equalization Decision Feedback Equalization makes use of previous decisions in attempting to estimate the current symbol (with an SBS detector). Any trailing intersymbol interference caused by previous symbols is reconstructed and then subtracted. he DFE is inherently a nonlinear receiver. However, it can be analyzed using linear techniques, if one assumes all previous decisions are correct. here is both a MMSE and a zero-forcing version of decision feedback, and as was true with linear equalization in Sections 3.4 and 3.5, the zero-forcing solution will be a special case of the least-squares solution with the SNR!. hus, this section derives the MMSE solution, which subsumes the zero-forcing case when SNR!. he Decision Feedback Equalizer (DFE) is shown in Figure he configuration contains a linear feedforward equalizer, W (D), (the settings for this linear equalizer are not necessarily the same as those for the ZFE or MMSE-LE), augmented by a linear, causal, feedback filter, B(D), where b 0 =. he feedback filter accepts as input the decision from the previous symbol period; thus, the name decision feedback. he output of the feedforward filter is denoted Z(D), and the input to the decision element Z 0 (D). he feedforward filter will try to shape the channel output signal so that it is a causal signal. he feedback section will then subtract (without noise enhancement) any trailing ISI. Any bias removal in Figure 3.37 is absorbed into the SBS. his section assumes that previous decisions are correct. In practice, this may not be true, and can be a significant weakness of decision-feedback that cannot be overlooked. Nevertheless, the analysis becomes intractable if it includes errors in the decision feedback section. o date, the most e cient way to specify the e ect of feedback errors has often been via measurement. Section 3.7 provides an exact errorpropagation analysis for finite-length DFE s that can (unfortunately) require enormous computation on a digital computer. Section 3.8. shows how to eliminate error propagation with precoders Minimum-Mean-Square-Error Decision Feedback Equalizer (MMSE- DFE) he Minimum-Mean-Square Error Decision Feedback Equalizer (MMSE-DFE) jointly optimizes the settings of both the feedforward filter w k and the feedback filter k b k to minimize the MSE: Definition 3.6. (Mean Square Error (for DFE)) he MMSE-DFE error signal is he MMSE for the MMSE-DFE is he error sequence can be written as e k = x k z 0 k. (3.3) MMSE-DFE =min w k,b k E x k z 0 k. (3.4) E(D) =X(D) W (D) Y (D) [ B(D)]X(D) =B(D) X(D) W (D) Y (D). (3.5) 7

60 For any fixed B(D), E [E(D)Y (D )] = 0 to minimize MSE, which leads us to the relation hus, B(D) R xy (D) W (D) R yy (D) =0. (3.6) B(D) W (D) = = B(D) W MMSE-LE (D), (3.7) kpk Q(D)+ SNR MFB for any B(D) withb 0 =. (Also, W (D) =W MMSE-LE (D) B(D), a consequence of the linearity of the MMSE estimate, so that E(D) =B(D) E MMSE-LE (D)) he autocorrelation function for the error sequence with arbitrary monic B(D) is R ee (D) = B(D)ĒxB (D ) < B(D) R xy (D)W (D ) + W (D) R yy (D)W (D )(3.8) = B(D)ĒxB (D ) W (D) R yy (D)W (D ) (3.9) = B(D)ĒxB (D ) R xy (D) B(D) B (D ) kpk Q(D)+ SNR MFB (3.0) = B(D)R MMSE LE (D)B (D ) (3.) where R MMSE LE (D) = N 0 kpk /(Q(D) +/SNR MFB ) is the autocorrelation function for the error sequence of a MMSE linear equalizer. he solution for B(D) is then the forward prediction filter associated with this error sequence as discussed in the Appendix. he linear prediction approach is developed more in what is called the linear prediction DFE, in an exercise where the MMSE-DFE is the concatenation of a MMSE-LE and a linear-predictor that whitens the error sequence. In more detail on B(D), the (scaled) inverse autocorrelation has spectral factorization: Q(D)+ SNR MFB = 0 G(D) G (D ), (3.) where 0 is a positive real number and G(D) is a canonical filter response. A filter response G(D) is called canonical if it is causal (g k = 0 for k<0), monic (g 0 = ), and minimum-phase (all its poles are outside the unit circle, and all its zeroes are on or outside the unit circle). If G(D) is canonical, then G (D )isanti-canonical i.e., anti-causal, monic, and maximum-phase (all poles inside the unit circle, and all zeros in or on the unit circle). Using this factorization, R ee (D) = B(D) B (D N ) 0 Q(D)+/SNR MFB kpk (3.3) = B(D) G(D) B (D ) G (D ) N 0 0kpk (3.4) N 0 r ee,0 0kpk, (3.5) with equality if and only if B(D) =G(D). hus, the MMSE will then be MMSE-DFE = N 0. he 0kpk feedforward filter then becomes W (D) = G(D) kpk 0 G(D) G (D ) = kpk 0 G (D ) he last step in (3.5) follows from the observations that r ee,0 = k B N 0 G k. (3.6) 0 kpk, (3.7) the fractional polynomial inside the squared norm is necessary monic and causal, and therefore the squared norm has a minimum value of. B(D) and W (D) specifythemmse-dfe: 8

61 Lemma 3.6. (MMSE-DFE) he MMSE-DFE has feedforward section W (D) = kpk 0 G (D ) (3.8) (realized with delay, as it is strictly noncausal) and feedback section B(D) =G(D) (3.9) where G(D) is the unique canonical factor of the following equation: Q(D)+ SNR MFB = 0 G(D) G (D ). (3.30) his text also calls the joint matched-filter/sampler/w (D) combination in the forward path of the DFE the Mean-Square Whitened Matched Filter (MS-WMF). hese settings for the MMSE-DFE minimize the MSE as was shown above Performance Analysis of the MMSE-DFE Again, the autocorrelation function for the error sequence is R ee (D) = N 0 0 kpk. (3.3) his last result states that the error sequence for the MMSE-DFE is white when minimized (since N 0 R ee (D) is a constant) and has MMSE or average energy (per real dimension) kpk 0. Also, Z ln Q(e! )+ SNR MFB d! = ln( 0 )+ Z ln G(e! )G (e! ) d!(3.3) = ln( 0 ). (3.33) (he last line follows from ln G(e! ) being a periodic function integrated over one period of its fundamental frequency, and similarly for ln G (e! ).) his last result leads to a famous expression for MMSE-DFE, which was first derived by Salz in 973, MMSE-DFE = N 0 kpk e he SNR for the MMSE-DFE can now be easily computed as R ln Q(e! )+ SNR d! MFB. (3.34) Ēx SNR MMSE DFE = = 0 SNR MFB (3.35) MMSE-DFE R ln Q(e! )+ = SNR MFB e SNR d! MFB. (3.36) From the k = 0 term in the defining spectral factorization, 0 can also be written 0 = +/SNR MFB kgk = +/SNR MFB + P i= g i. (3.37) From this expression, if G(D) = (no ISI), then SNR MMSE DFE = SNR MFB +, so that the SNR would be higher than the matched filter bound. he reason for this apparent anomaly is the artificial 9

62 signal power introduced by biased decision regions. his bias is noted by writing Z 0 (D) = X(D) E(D) (3.38) = X(D) G(D) X(D)+ kpk 0 G (D Y (D) ) (3.39) = X(D) G(D)X(D)+ Q(D) 0G (D ) X(D)+N(D) kpk 0 G (D ) (3.40) = X(D) /SNR MFB 0 G (D ) X(D)+N(D) kpk 0 G (D ). (3.4) he current sample, x k, corresponds to the time zero sample of /SNR MFB 0 G (D ), and is equal to ( SNR )x k. hus z 0 MMSE DF E k contains a signal component of x k that is reduced in magnitude. hus, again using the result from Section 3.., the SNR corresponding to the lowest probability of error corresponds to the same MMSE-DFE receiver with output scaled to remove the bias. his leads to the more informative SNR If G(D) =, then SNR MMSE MMSE DFE = SNR MMSE DFE,U = SNR MMSE DFE = 0 SNR MFB. (3.4) DFE,U = SNR MFB. Also, SNR MFB SNR MMSE DFE,U = SNR MFB 0 SNR MFB =. (3.43) 0 /SNR MFB Rather than scale the decision input, the receiver can scale (up) the feedforward output by. SNR MMSE DF E his will remove the bias, but also increase MSE by the square of the same factor. he SNR will then be SNR MMSE DFE,U. his result is verified by writing the MS-WMF output Z(D) =[X(D) kpk Q(D)+N(D)] kpk 0 G (D ) where N(D), again, has autocorrelation R nn (D) = N0 Q(D). Z(D) expands to (3.44) Z(D) = [X(D) kpk Q(D)+N(D)] kpk 0 G (D (3.45) ) = X(D) 0 G(D) G (D ) /SNR MFB 0 G (D + N(D) ) kpk 0 G (D (3.46) ) X(D) = X(D) G(D) SNR MFB 0 G (D ) + N 0 (D) (3.47) apple apple = X(D) G(D) + SNR MMSE DFE G (D X(D)+N(3.48) 0 ) = X(D) apple G(D) SNR MMSE DFE SNR MMSE DFE + E 0 (D), (3.49) where N 0 (D) is the filtered noise at the MS-WMF output, and has autocorrelation function apple and R n 0 n 0(D) = N 0 Q(D) kpk 0 G(D) G (D ) = Ēx SNR MMSE e 0 k = e k + DFE /SNR MFB Q(D)+/SNR MFB, (3.50) SNR MMSE DFE x k. (3.5) he error e 0 k is not a white sequence in general. Since x k and e 0 k are uncorrelated, e = MMSE-DFE = e + Ēx. If one defines an unbiased monic, causal polynomial 0 SNR MMSE DF E G U (D) = SNR MMSE DFE SNR MMSE DFE,U apple G(D) 0 SNR MMSE DFE, (3.5)

63 y k W( D) z k z U, k Σ! ' zu,k U!n!b!i!a!s!e!d!! D!e!c!i!s!i!o!n! xˆ k F!e!e!d!f!o!r!w!a!r!d!! f!i!l!t!e!r! SNR SNR MMSE DFE MMSE DFE, U G U ( D) F!e!e!d!b!a!c!k!! f!i!l!t!e!r! Figure 3.38: Decision feedback equalization with unbiased feedback filter. then the MMSE-DFE output is Z U (D) removes the scaling Z(D) = SNR MMSE DFE,U X(D)G U (D)+E 0 (D). (3.53) SNR MMSE DFE Z U (D) =Z(D) SNR MMSE DFE = X(D)G U (D)+E U (D), (3.54) SNR MMSE DFE,U where E U (D) = SNR MMSE DF E SNR E 0 (D) and has power MMSE DF E,U e U = MMSE-DFE Ēx SNR MMSE = SNR MMSE DFE MMSE-DFE SNR MMSE DFE DFE Ēx! SNRMMSE DFE SNR MMSE SNR MMSE SNR MMSE DFE,U DFE DFE,U (3.55) (3.56) = Ēx[(SNR MMSE DFE /Ēx)SNR MMSE DFE MMSE-DFE ] SNR MMSE DFE,U (3.57) = Ēx[SNR MMSE DFE ] SNR MMSE DFE,U (3.58) = Ēx SNR MMSE DFE,U. (3.59) SNR for this scaled and unbiased MMSE-DFE is thus verified to be Ēx Ēx /SNR = SNR MMSE DFE,U. MMSE DF E,U he feedback section of this new unbiased MMSE-DFE becomes the polynomial G U (D), if the scaling is performed before the summing junction, but is G(D) if scaling is performed after the summing junction. he alternative unbiased MMSE-DFE is shown in Figure All the results on fractional spacing for the LE still apply to the feedforward section of the DFE thus this text does not reconsider them for the DFE, other than to state that the characteristic of the feedforward section is di erent than the LE characteristic in all but trivial channels. Again, the realization of any characteristic can be done with a matched filter and symbol-spaced sampling or with an anti-alias filter and fractionally spaced sampling, as in Section Zero-Forcing DFE he ZF-DFE feedforward and feedback filters are found by setting SNR MFB!in the expressions for the MMSE-DFE. he spectral factorization (assuming ln(q(e! ) is integrable over (, ), see Appendix A) Q(D) = 0 P c (D) Pc (D ) (3.60)

64 then determines the ZF-DFE filters as W (D) = B(D) =P c (D), (3.6) 0 kpk P c (D ). (3.6) P c is sometimes called a canonical pulse response for the channel. Since q 0 == 0 kp c k,then 0 = + P i= p c,i. (3.63) Equation (3.63) shows that there is a signal power loss of the ratio of the squared first tap magnitude in the canonical pulse response to the squared norm of all the taps. his loss ratio is minimized for a minimum phase polynomial, and P c (D) is the minimum-phase equivalent of the channel pulse response. he noise at the output of the feedforward filter has (white) autocorrelation function N0 /(kpk 0 ), so that ZFDFE = he corresponding SNR is then Examples Revisited N 0 N 0 kpk = 0 kpk e SNR ZF DFE = 0 SNR MFB = R ln Q(e! ) d!. (3.64) + P i= p c,i SNR MFB. (3.65) EXAMPLE 3.6. (PAM - DFE) he pulse response of a channel used with binary PAM is again given by Again, the SNR MFB is 0dB and Ex =. P (!) = p +.9e!! apple 0! > hen Q(D) = Q(D)+/SNR MFB is computed as. (3.66) Q(e! ) = Q(D) = = cos(!)+.8/0.8 (3.67).8 ( +.9D)( +.9D )+. (3.68).8.9D D. (3.69) he roots of Q(D) are.633 and.58. he function Q(D) has canonical factorization Q(D) =.785( +.633D)( +.633D ). (3.70) hen, 0 =.785. he feedforward section is W (D) = p.8(.785)( +.633D ) = D, (3.7) and the feedforward filter transfer function appears in Figure he feedback section is B(D) =G(D) =+.633D, (3.7) with magnitude in Figure he MS-WMF filter transfer function appears in Figure 3.4, illustrating the near all-pass character of this filter. he MMSE is MMSE-DFE = N 0.8 kpk =. =.74 (3.73)

65 0 FF filter magnitude 5 magnitude in db normalized frequency Figure 3.39: Feedforward filter transfer function for real-channel example. 3

66 5 Feedback filter magnitude 0 magnitude in db normalized frequency Figure 3.40: Feedback filter transfer function for real-channel example. 4

67 0 MS-WMF filter magnitude -5 magnitude in db normalized frequency Figure 3.4: MS-WMF transfer function for real-channel example. 5

68 and SNR MMSE DFE,U =.74 =6.85 (8.4dB). (3.74).74 hus, the MMSE-DFE is only.6db below the MFB on this channel. However, the ZF- DFE for this same channel would produce 0 =/kp c k =.555. In this case ZFDFE =.8/(.555).8 =.8 and the loss with respect to the MFB would be 0 =.6 db,db lower than the MMSE-DFE for this channel. It will in fact be possible to do yet better on this channel, using sequence detection, as discussed in Chapter 9, and/or by codes (Chapters 0 and ). However, what originally appeared as a a stunning loss of 9.8 db with the ZFE has now been reduced to a much smaller.6 db. he QAM example is also revisited here: EXAMPLE 3.6. (QAM - DFE) Recall that the equivalent baseband pulse response samples were given by p k = p apple +. (3.75) 4 he SNR MFB is again 0 db. hen, or Q =.5 D +.65( + )D +.565( +.).65( + )D +.5 D.565. (3.76) Q = ( D)( D)( D )( D ) ( )( (3.77) ) from which one extracts 0 =.7866 and G(D) =.46(+ )D+.034 D. he feedforward and feedback sections can be computed in a straightforward manner, as and he MSE is B(D) =G(D) =.46( + )D D (3.78) W (D) = and the corresponding SNR is.07.46( )D.034 D. (3.79) MMSE-DFE =(.565) =.7. (3.80).7866(.565) SNR MMSE DFE,U =.7 =8.4dB. (3.8) he loss is (also coincidentally).6 db with respect to the MFB and is.7 db better than the MMSE-LE on this channel. For the ZF-DFE, Q(D) = (.56 / D)(.5D)(.56 / D )(.5D ) (3.8) (3.83) (.56 / )(.5) and thus 0 =.6400 and P c (D) =.5( + )D +.5 D. he feedforward and feedback sections can be computed in a straightforward manner, as B(D) =P c (D) =.5( + )D +.5 D (3.84) 6

69 and he output noise variance is and the corresponding SNR is W (D) =.5.5( )D.5 D. (3.85) ZFDFE =(.565) =.563. (3.86).6400(.565) SNR ZFDFE = =6.4 =8.0dB. (3.87).563 he loss is.0 db with respect to the MFB and is.4 db worse than the MMSE-DFE. 7

70 3.7 Finite Length Equalizers he previous developments of discrete-time equalization structures presumed that the equalizer could exist over an infinite interval in time. Equalization filters, w k, are almost exclusively realized as finiteimpulse-response (FIR) filters in practice. Usually these structures have better numerical properties than IIR structures. Even more importantly, adaptive equalizers (see Section 3.8 and Chapter 3) are often implemented with FIR structures for w k (and also for b k, in the case of the adaptive DFE). his section studies the design of and the performance analysis of FIR equalizers. Both the LE and the DFE cases are examined for both zero-forcing and least-squares criteria. his section describes design for the MMSE situation and then lets SNR! to get the zero-forcing special case. Because of the finite length, the FIR zero-forcing equalizer cannot completely eliminate ISI in general FIR MMSE-LE Returning to the FSE in Figure 3.35, the matched filtering operation will be performed digitally (at sampling rate l/ ) and the FIR MMSE-LE will then incorporate both matched filtering and equalization. Perfect anti-alias filtering with gain p precedes the sampler and the combined pulse-response/anti-alias filter is p(t). he assumption that l is an integer simplifies the specification of matrices. One way to view the oversampled channel output is as a set of l parallel -spaced subchannels whose pulse responses are o set by /l from each other as in Figure 3.4. p( k) x k p ( k l).!!.!!.! ( k [ l ] l) p y ( k i l) Figure 3.4: Polyphase subchannel representation of the channel pulse response. Each subchannel produces one of the l phases per symbol period of the output set of samples at sampling rate l/. Mathematically, it is convenient to represent such a system with vectors as shown below. he channel output y(t) is y(t) = X m x m p(t m )+n(t), (3.88) which, if sampled at time instants t = k y(k i l )= X m= i l, i =0,...,l x m p(k i l, becomes m )+n(k i l ). (3.89) he (per-dimensional) variance of the noise samples is N0 l because the gain of the anti-alias filter, p, 8

71 is absorbed into p(t). he l phases per symbol period of the oversampled y(t) are contained in he vector y k can be written as where y k = p k = 6 4 X m= y k = 6 4 y(k) y(k l ). y(k l l ) x m p k m + n k = p(k) p(k l ). p(k l l ) 3 X m= and n k = (3.90) x k m p m + n k, (3.9) n(k) n(k l ). n(k l l ) (3.9) he response p(t) is assumed to extend only over a finite time interval. In practice, this assumption requires any nonzero component of p(t) outside of this time interval to be negligible. his time interval is 0 apple t apple. hus,p k = 0 for k<0, and for k>. he sum in (3.89) becomes y k =[p 0 p,..., p ] 6 4 x k x k x k n k. (3.93) 7 5 Each row in (3.93) corresponds to a sample at the output of one of the filters in Figure 3.4. More generally, for N f successive l-tuples of samples of y(t), Y k = = y k y k. y k Nf p 0 p... p p 0 p p p 0 p... p 6 4 n k n k. n k Nf + 3 x k x k.. x k Nf (3.94) 7 5. (3.95) his text uses P to denote the large (N f l) (N f + ) matrix in (3.95), while X k denotes the data vector, and N denotes the noise vector. hen, the oversampled vector representation of the channel is Y k = PX k + N k. (3.96) 9

72 When l = n/m (a rational fraction), then (3.96) still holds with P an [N f n m ] (N f + ) matrix that does not follow the form in (3.95), with each row possibly having a set of coe cients unrelated to the other rows. An equalizer is applied to the sampled channel output vector Y k by taking the inner product of an N f l-dimensional row vector of equalizer coe cients, w, and Y k,soz(d) =W (D)Y (D) can be written z k = wy k. (3.97) For causality, the designer picks a channel-equalizer system delay symbol periods, + N f, (3.98) with the exact value being of little consequence unless the equalizer length N f is very short. he delay is important in finite-length design because a non-causal filter cannot be implemented, and the delay allows time for the transmit data symbol to reach the receiver. With infinite-length filters, the need for such a delay does not enter the mathematics because the infinite-length filters are not realizable in any case, so that infinite-length analysis simply provides best bounds on performance. is approximately the sum of the channel and equalizer delays in symbol periods. he equalizer output error is then and the corresponding MSE is e k = x k z k, (3.99) MMSE-LE = E e k = E {e k e k} = E (x k z k )(x k z k ). (3.300) Using the Orthogonality Principle of Appendix A, the MSE in (3.300) is minimized when E {e k Y k} =0. (3.30) In other words, the equalizer error signal will be minimized (in the mean square sense) when this error is uncorrelated with any channel output sample in the delay line of the FIR MMSE-LE. hus E {x k Y k} we {Y k Y k} =0. (3.30) he two statistical correlation quantities in (3.30) have a very special significance (both y(t) and x k are stationary): Definition 3.7. (Autocorrelation and Cross-Correlation Matrices) he FIR MMSE autocorrelation matrix is defined as R YY = E {Y k Y k} /N, (3.303) while the FIR MMSE-LE cross-correlation vector is defined by R Y x = E Y k x k /N, (3.304) where N =for real and N =for complex. Note R xy = R Y,andR x YY function of. With the above definitions in (3.30), the designer obtains the MMSE-LE: is not a Definition 3.7. (FIR MMSE-LE) he FIR MMSE-LE for sampling rate l/, delay, and of length N f symbol periods has coe cients w = R Y x R YY = R xy R YY, (3.305) or equivalently, w = R YY R Y x. (3.306) In this case, (3.88) is used to compute each row at the appropriate sampling instants. N f n should also be an m integer so that P is constant. Otherwise, P becomes a time-varying matrix P k. 30

73 In general, it may be of interest to derive more specific expressions for R YY and R xy. R xy = E {x k Y k} = E {x k X k}p + E {x k N k} (3.307) = Ēx P + 0 (3.308) = Ēx [ p... p ] (3.309) R YY = E {Y k Y k} = P E {X k X k}p + E {N k N k} (3.30) = ĒxPP + l N0 R nn, (3.3) where the (l N0 -normalized) noise autocorrelation matrix R nn is equal to I when the noise is white. A convenient expression is [0...0 p...p ] = P, (3.3) so that is an N f + -vector of 0 s and a in the ( +) th position. hen the FIR MMSE-LE becomes (in terms of P, SNR and ) w = P apple PP + SNR R nn apple = P RnnP + l SNR I P Rnn. (3.33) he matrix inversion lemma of Appendix B is used in deriving the above equation. he position of the nonzero elements in (3.309) depend on the choice of. he MMSE, MMSE-LE, can be found by evaluating (3.300) with (3.305): MMSE-LE = E x k x k x k Y kw wy k x k + wy k Y kw (3.34) With algebra, (3.38) is the same as = Ēx R xy w wr Y x + wr YY w (3.35) = Ēx wr YY w (3.36) = Ēx R xy R YY R Y x (3.37) = Ēx wr Y x, (3.38) MMSE-LE = l N0 kpk P RnnP l kpk + I SNR MFB = l N0 kpk Q, (3.39) so that the best value of (the position of the in the vectors above) corresponds to the smallest diagonal element of the inverted ( Q-tilde ) matrix in (3.39) this means that only one matrix need be inverted to obtain the correct value as well as compute the equalizer settings. A homework problem develops some interesting relationships for w in terms of P and SNR. hus, the SNR for the FIR MMSE-LE is Ēx SNR MMSE LE =, (3.30) MMSE-LE and the corresponding unbiased SNR is he loss with respect to the MFB is, again, SNR MMSE LE,U = SNR MMSE LE. (3.3) MMSE LE = SNR MFB SNR MMSE LE,U. (3.3) 3

74 3.7. FIR ZFE One obtains the FIR ZFE by letting the SNR! in the FIR MMSE, which alters R YY to R YY = PP Ēx (3.33) and then w remains as w = R xy R YY. (3.34) Because the FIR equalizer may not be su ciently long to cancel all ISI, the FIR ZFE may still have nonzero residual ISI. his ISI power is given by MMSE ZFE = Ēx wr Y x. (3.35) However, (3.35) still ignores the enhanced noise at the FIR ZFE output. he power of this noise is easily found to be FIR ZFE noise power = N 0 l kwk Rnn, (3.36) making the SNR at the ZFE output SNR ZFE = Ēx he ZFE is still unbiased in the finite-length case: 3 Ēx wr Y x + N0 lkwk Rnn. (3.37) E [wy k /x k ] = R xy R YY {P E [X k/x k ]+E[N k ]} (3.38) = [ ]E P (PP ) PX k /x k (3.39) 8 x k x k.. >< = [ ]P (PP ) P E /x k x >: k Nf 5 don t care 3 9 >= >; (3.330) = x k. (3.33) Equation (3.33) is true if is a practical value and the finite-length ZFE has enough taps. he loss is the ratio of the SNR to SNR ZFE, example ZFE = SNR MFB SNR ZFE. (3.33) For the earlier PAM example, one notes that sampling with l = is su cient to represent all signals. First, choose N f = 3 and note that =. hen P = (3.333) With a choice of =, then he notation kwk Rnn means wrnnw. R Y x = ĒxP = (3.334) 5, (3.335) 3 he matrix P (PP ) P is a projection matrix and PP is full rank; therefore the entry of x k passes directly (or is zeored, in which case needs to be changed). 3

75 and R YY = Ēx = 4 PP + SNR I (3.336) 5, (3.337) he FIR MMSE is w = R YY R Y x = = (3.338) hen he SNR is computed to be 0 MMSE-LE [.3.5.] 4 SNR MMSE LE,U = A =.94 (3.339) =.4 (3.8 db). (3.340) which is 6.dB below MFB performance and.9db worse than the infinite length MMSE-LE for this channel. With su ciently large number of taps for this channel, the infinite length performance level can be attained. his performance is plotted as versus the number of equalizer taps in Figure 33

76 Figure 3.43: FIR equalizer performance for +.9D versus number of equalizer taps Clearly, 5 taps are su cient for infinite-length performance. EXAMPLE 3.7. For the earlier complex QAM example, sampling with l = issu cient to represent all signals. First, choose N f = 4 and note that =. hen /4 / P = /4 / /4 / 0 5. (3.34) /4 / he matrix (P P I) has the same smallest element.3578 for both =, 3, so choosing = will not unnecessarily increase system delay. With a choice of =, then R Y x = ĒxP 6 0 (3.34) = 6 + / , (3.343) 0 34

77 and R YY = Ēx = 6 4 PP + SNR I / /4 / / (3.344) he FIR MMSE is w = R YY R Y x = (3.345) hen MMSE LE = ( wr Y x ) (3.346) =.. (3.347) he SNR is computed to be SNR MMSE LE,U =. = 3.74 (5.7 db), (3.348) which is 4.3 db below MFB performance and. db worse than the infinite-length MMSE-LE for this channel. With su ciently large number of taps for this channel, the infinite-length performance level can be attained FIR MMSE-DFE he FIR DFE case is similar to the feed-forward equalizers just discussed, except that we now augment the fractionally spaced feed-forward section with a symbol-spaced feedback section. he MSE for the DFE case, as long as w and b are su ciently long, is MSE = E x k wy k + bx k, (3.349) where b is the vector of coe cients for the feedback FIR filter b =[b b... b Nb ], (3.350) and x k is the vector of data symbols in the feedback path. It is mathematically convenient to define an augmented response vector for the DFE as apple w = w. b, (3.35) and a corresponding augmented DFE input vector as apple Y Ỹ k = k x k. (3.35) hen, (3.349) becomes n MSE = E x k w Y k o. (3.353) he solution, paralleling (3.304) - (3.306), is determined using the following two definitions: 35

78 Definition (FIR MMSE-DFE Autocorrelation and Cross-Correlation Matrices) he FIR MMSE-DFE autocorrelation matrix is defined as o RỸ nỹ Ỹ = E k Ỹ k /N (3.354) apple R = YY E[Y k x k ] E[x k Y (3.355) k] ĒxI Nb " = Ēx PP + l R # nn SNR ĒxP J ], (3.356) ĒxJ P ĒxI Nb where J is an (N f + ) N b matrix of 0 s and s, which has the upper + rows zeroed and an identity matrix of dimension min(n b,n f + ) with zeros to the right (when N f + <N b ), zeros below (when N f + >N b ), or no zeros to the right or below exactly fitting in the bottom of J (when N f + =N b ). he corresponding FIR MMSE-DFE cross-correlation vector is o RỸ = E nỹ k x x k /N (3.357) = = apple RY x 0 apple ĒxP 0 (3.358), (3.359) where, again, N =for real signals and N =for complex signals. he FIR MMSE-DFE for sampling rate l/, delay, and of length N f and N b has coe cients Equation (3.360) can be rewritten in detail as w = R xỹ R Ỹ Ỹ. (3.360) apple apple PP w. b Ēx + l SNR R nn P J J P = I Nb which reduces to the pair of equations w PP + l SNR R nn apple Ēx P.0, (3.36) bj P = P (3.36) wp J b = 0. (3.363) hen and thus or hen b = wp J, (3.364) w PP P J J P + l SNR R nn = P (3.365) w = P PP P J J P + l SNR R nn. (3.366) b = P PP P J J P + l SNR R nn P J. (3.367) 36

79 he MMSE is then MMSE DFE = Ēx wrỹ x (3.368) = Ēx wr Y x (3.369) = Ēx P PP P J J P + l! SNR R nn P, (3.370) which is a function to be minimized over. hus, the SNR for the unbiased FIR MMSE-DFE is SNR MMSE DFE,U = Ēx MMSE DFE = wrỹ x Ēx wrỹ x, (3.37) and the loss is again MMSE DFE = SNR MFB SNR MMSE DFE,U. (3.37) EXAMPLE 3.7. (MMSE-DFEs, PAM and QAM) For the earlier example with l =, N f =, and N b = and =, we will also choose =. hen P = J = = apple , (3.373) 4 0 5, (3.374) , (3.375) 0 and hen w = [0 0] 4 R YY = apple.99.9 = [.9] apple apple (3.376) apple 0 [0 ] (3.377) (3.378) = [ ] (3.379) b = wp J (3.380) 3 apple = [ ] (3.38) 0.9 = (3.38) hen he SNR is computed to be apple MMSE DFE = [.6.76].9 =.57. (3.383) SNR MMSE DFE,U = =5.4 (7.3dB), (3.384) 37

80 which is.7 db below MFB performance, but.8 db better than the infinite-length MMSE- LE for this channel! he loss with respect to the infinite-length MMSE-DFE is. db. A picture of the FIR MMSE-DFE is shown in Figure y k D! 6!.!4!.6!.76! 5!.!4! x! dec! xˆk Δ z k +! +! -! D!.76! Figure 3.44: FIR MMSE-DFE Example. With su ciently large number of taps for this channel, the infinite-length performance level can be attained. his performance is plotted versus the number of feed-forward taps (only one feedback tap is necessary for infinite-length performance) in Figure In this case, 7 feed-forward and feedback taps are necessary for infinite-length performance. hus, the finite-length DFE not only outperforms the finite or infinite-length LE, it uses less taps (less complexity) also. For the QAM example l =, N f =, N b =, and =, we also choose =. Actually this channel will need more taps to do well with the DFE structure, but we can still choose these values and proceed. hen apple.5 + /4 / 0 P =, (3.385) /4 / J = , (3.386) 0 and hen w = [0 0 0] 6 4 apple = apple R YY = /4.5 / /4 0 / 3 7 5, (3.387) (3.388) 7 5 (3.389) apple /4 /8 / /8+ /.35 38

81 apple = [ /.5] (3.390) = [ ] (3.39) b = wp J apple = [ ].5 + /4 / /4 / (3.39) 7 5 (3.393) = [ ]. (3.394) hen he SNR is computed to be MMSE DFE =.97. (3.395) SNR MMSE DFE,U = =6.3dB, (3.396) which is 3.77 db below MFB performance, and not very good on this channel! he reader may evaluate various filter lengths and delays to find a best use of 3, 4, 5, and 6 parameters on this channel. Figure 3.45: FIR MMSE-DFE performance for +.9D versus number of feed-forward equalizer taps. 39

82 FIR ZF-DFE alters RỸ Ỹ to One obtains the FIR ZF-DFE by letting the SNR! in the FIR MMSE-DFE, which RỸ Ỹ = apple Ēx P P ĒxP J ĒxJ P Ēx I Nb (3.397) and then w remains as w = R R xỹ Ỹ Ỹ. (3.398) Because the FIR equalizer may not be su ciently long to cancel all ISI, the FIR ZF-DFE may still have nonzero residual ISI. his ISI power is given by MMSE-DFE = Ēx wrỹ x. (3.399) However, (3.399) still ignores the enhanced noise at the w k filter output. he power of this noise is easily found to be FIR ZFDFE noise variance = N 0 l kwk, (3.400) making the SNR at the FIR ZF-DFE output SNR ZF DFE = Ēx Ēx wrỹ x + Ēx SNR lkwk. (3.40) he filter is w in (3.400) and (3.40), not w because only the feed-forward section filters noise. he loss is: ZF-DFE = SNR MFB. (3.40) SNR ZF DFE An Alternative Approach to the DFE While the above approach directly computes the settings for the FIR DFE, it yields less insight into the internal structure of the DFE than did the infinite-length structures investigated in Section 3.6, particularly the spectral (canonical) factorization into causal and anti-causal ISI components is not explicit. his subsection provides such an alternative caused by the finite-length filters. his is because finite-length filters inherently correspond to non-stationary processing. 4 he vector of inputs at time k to the feed-forward filter is again Y k, and the corresponding vector of filter coe cients is w, so the feed-forward filter output is z k = wy k. Continuing in the same fashion, the DFE error signal is then described by e k = bx k ( ) wy k, (3.403) where x k:k N b = x k... x k N b, and b is now slightly altered to be monic and causal b =[b b... b Nb ]. (3.404) he mean-square of this error is to be minimized over b and w. Signals will again be presumed to be complex, but all developments here simplify to the one-dimensional real case directly, although it is important to remember to divide any complex signal s variance by to get the energy per real dimension. N0 he SNR equals Ēx/ = E x/n 0 in either case. 4 Amoreperfectanalogybetweenfinite-lengthandinfinite-lengthDFE soccursinchapter5,wherethebestfinite-length DFE s are actually periodic over a packet period corresponding to the length of the feed-forward filter. 40

83 Optimizing the feed-forward and feedback filters For any fixed b in (3.403), the cross-correlation between the error and the vector of channel outputs Y k should be zero to minimize MSE, E [e k Y k]=0. (3.405) So, and wr YY = br XY ( ), (3.406) n h i R XY ( ) = E X k ( ) x k x k... x k N f + P o (3.407) = Ēx J P, (3.408) where the matrix J has the first columns all zeros, then an (up to) N b N b identity matrix at the top of the up to N b columns, and zeros in any row entries below that identity, and possibly zeroed columns following the identity if N f + >N b. he following matlab commands produce J,using for instance N f = 8, N b = 5, = 3, and = 5, >> size=min(delta,nb); >> Jdelta=[ zeros(size,size) eye(size) zeros(size, max(nf+nu-*size,0)) zeros(max(nb-delta,0),nf+nu)] Jdelta = or for N f = 8, N b =, = 3, and = 5, >> >> size=min(delta,nb) size = >> Jdelta=[ zeros(size,size) eye(size) zeros(size, max(nf+nu-*size,0)), zeros(max(nb-delta,0),nf+nu)] Jdelta = he MSE for this fixed value of b then becomes ( ) = b ĒxI R XY ( )R YY R YX ( ) b (3.409) = b R? X/Y ( )b (3.40) = b ĒxI Nb J P PP + l SNR R nn P J ( ) b (3.4) = l N0 b J P R nnp + l SNR I P J ( ) J b, (3.4) and R? X/Y ( ) = ĒxI R XY ( )R YY R YX ( ), the autocorrelation matrix corresponding to the MMSE vector in estimating X( ) from Y k. his expression is the equivalent of (3.). hat is R? X/Y ( ) is the autocorrelation matrix for the error sequence of length N b that is associated with a 4

84 matrix MMSE-LE. he solution then requires factorization of the inner matrix into canonical factors, which is executed with Cholesky factorization for finite matrices. By defining Q( ) = ( J P P + l SNR I J ), (3.43) this matrix is equivalent to (3.), except for the annoying J matrices that become identities as N f and N b become infinite, then leaving Q as essentially the autocorrelation of the channel output. he J factors, however, cannot be ignored in the finite-length case. Canonical factorization of Q proceeds according to ( ) = bg S G b, (3.44) which is minimized when b = g( ), (3.45) the top row of the upper-triangular matrix G. he MMSE is thus obtained by computing Cholesky factorizations of Q for all reasonable values of and then setting b = g( ). hen ( ) = S o ( ). (3.46) From previous developments, as the lengths of filters go to infinity, any value of N S 0! 0 0 /Ēx to ensure the infinite-length MMSE-DFE solution of Section 3.6. he feed-forward filter then becomes works and also w = br XY ( )R YY (3.47) = g( ) J P PP + l SNR I (3.48) = g( ) J P P + l SNR I {z} P, (3.49) {z } matched filter feedforward filter which can be interpreted as a matched filter followed by a feed-forward filter that becomes /G (D ) as its length goes to infinity. However, the result that the feed-forward filter is an anti-causal factor of the canonical factorization does not follow for finite length. Chapter 5 will find a situation that is an exact match and for which the feed-forward filter is an inverse of a canonical factor, but this requires the DFE filters to become periodic in a period equal to the number of taps of the feed-forward filter (plus an excess bandwidth factor). he SNR is as always SNR MMSE DFE,U = Ēx. (3.40) S 0 ( ) Bias can be removed by scaling the decision-element input by the ratio of SNR MMSE DFE /SNR MMSE DFE,U, thus increasing its variance by (SNR MMSE DFE /SNR MMSE DFE,U ). Finite-length Noise-Predictive DFE Problem 3.8 introduces the noise-predictive form of the DFE, which is repeated here in Figure

85 y p ( t) * ( ) ϕ p t Y ( D ) U( D) Z ( D) +! Z' ( D) +!! +! B ( D) +!! [h] Figure 3.46: Noise-Predictive DFE In this figure, the error sequence is feedback instead of decisions. he filter essentially tries to predict the noise in the feedforward filter output and then cancel this noise, whence the name noise-predictive DFE. Correct solution to the infinite-length MMSE filter problem 3.8 will produce that the filter B(D) remains equal to the same G(D) found for the infinite-length MMSE-DFE. he filter U(D) becomes the MMSE-LE so that Z(D) has no ISI, but a strongly correlated (and enhanced) noise. he preditor then reduces the noise to a white error sequence. If W (D) = 0 kpk G (D ) of the normal MMSE-DFE, then it can be shown also (see Problem 3.8) that U(D) = W (D) G(D). (3.4) he MMSE, all SNR s, and biasing/unbiasing remain the same. An analogous situation occurs for the finite-length case, and it will be convenient notationally to say that the number of taps in the feedforward filter u is (N f )l. Clearly if is fixed as always, then any number of taps (greater than can be investigated without loss of generality with this early notational abuse. Clearly by abusing N f (which is not the number of taps), ANY positive number of taps in u can be constructed without loss of generality for any value of 0. In this case, the error signal can be written as e k = b k X k ( ) bz (3.4) where Z k = 6 4 z ḳ (3.43) hen, where and By defining the (N f )l-tap filter z k Z k = UY k (3.44) 3 u u... 0 U = u Y k = 6 4 y ḳ. y k Nf + 3 (3.45) 7 5. (3.46) w = bu, (3.47) 43

86 then the error becomes e k = b k X k ( ) wy, (3.48) which is the same error as in the alternate viewpoint of the finite-length DFE earlier in this section. hus, solving for the b and w of the conventional finite-length MMSE-DFE can be followed by the step of solving Equation (3.47) for U when b and w are known. However, it follows directly then from Equation (3.406) that u = R XY ( )R YY, (3.49) so following the infinite-case form, u is the finite-length MMSE-LE with (N f )l taps. he only di erence from the infinite-length case is the change in length he Stanford DFE Program A matlab program has been created, used, and refined by the students of EE379A over a period of many years. he program has the following call command: function (SNR, wt) = dfe ( l,p,n, nbb, delay, Ex, noise); where the inputs and outputs are listed as l = oversampling factor p = pulse response, oversampled at l (size) n = number of feed-forward taps nbb = number of feedback taps delay delay of system N + length of p - nbb ; if delay = -, then choose the best delay Ex = average energy of signals noise = autocorrelation vector (size l*n ) (NOE: noise is assumed to be stationary). For white noise, this vector is simply [ 0...0]. SNR = equalizer SNR, unbiased and in db wt = equalizer coe cients he student may use this program to a substantial advantage in avoiding tedious matrix calculations. he program has come to be used throughout the industry to compute/project equalizer performance (setting nbb=0 also provides a linear equalizer). he reader is cautioned against the use of a number of rules of thumb (like DFE SNR is the SNR(f) at the middle of the band ) used by various who call themselves experts and often over-estimate the DFE performance using such formulas. Di cult transmission channels may require large numbers of taps and considerable experimentation to find the best settings. function [dfsesnr,w_t,opt_delay]=dfsecolorsnr(l,p,nff,nbb,delay,ex,noise); % % [dfsesnr,w_t] = dfecolor(l,p,nff,nbb,delay,ex,noise); % l = oversampling factor % p = pulse response, oversampled at l (size) % nff = number of feed-forward taps % nbb = number of feedback taps % delay = delay of system <= nff+length of p - - nbb % if delay = -, then choose best delay % Ex = average energy of signals % noise = noise autocorrelation vector (size l*nff) % NOE: noise is assumed to be stationary % % outputs: 44

87 % dfsesnr = equalizer SNR, unbiased in db % w_t = equalizer coefficients [w -b] % opt_delay = optimal delay found if delay =- option used. % otherwise, returns delay value passed to function % created 4/96; % size = length(p); nu = ceil(size/l)-; p = [p zeros(,(nu+)*l-size)]; % error check if nff<=0 error( number of feed-forward taps > 0 ); end if delay > (nff+nu--nbb) error( delay must be <= (nff+(length of p)--nbb) ); elseif delay < - error( delay must be >= - ); elseif delay == - delay = 0::nff+nu--nbb; end %form ptmp = [p_0 p_... p_nu] where p_i=[p(i*l) p(i*l-)... p((i-)*l+) ptmp(:l,) = [p(); zeros(l-,)]; for i=:nu ptmp(:l,i+) = conj((p(i*l+:-:(i-)*l+)) ); end %form matrix P, vector channel matrix P = zeros(nff*l+nbb,nff+nu); for i=:nff, P(((i-)*l+):(i*l),i:(i+nu)) = ptmp; end %precompute Rn matrix - constant for all delays Rn = zeros(nff*l+nbb); Rn(:nff*l,:nff*l) = toeplitz(noise); dfsesnr = -00; P_init = P; %loop over all possible delays for d = delay, P = P_init; P(nff*l+:nff*l+nbb,d+:d++nbb) = eye(nbb); %P temp= zeros(,nff+nu); temp(d+)=; %construct matrices Ry = Ex*P*P + Rn; Rxy = Ex*temp*P ; new_w_t = Rxy*inv(Ry); sigma_dfse = Ex - real(new_w_t*rxy ); new_dfsesnr = 0*log0(Ex/sigma_dfse - ); 45

88 end %save setting of this delay if best performance so far if new_dfsesnr >= dfsesnr w_t = new_w_t; dfsesnr = new_dfsesnr; opt_delay = d; end Error Propagation in the DFE o this point, this chapter has ignored the e ect of decision errors in the feedback section of the DFE. At low error rates, say 0 5 or below, this is reasonable. here is, however, an accurate way to compute the e ect of decision feedback errors, although enormous amounts of computational power may be necessary (much more than that required to simulate the DFE and measure error rate increase). At high error rates of 0 3 and above, error propagation can lead to several db of loss. In coded systems (see Chapters 7 and 8), the inner channel (DFE) may have an error rate that is unacceptably high that is later reduced in a decoder for the applied code. hus, it is common for low decision-error rates to occur in a coded DFE system. he Precoders of Section 3.8 can be used to eliminate the error propagation problem, but this requires that the channel be known in the transmitter which is not always possible, especially in transmission systems with significant channel variation, i.e, digital mobile telephony. An analysis for the infinite-length equalizer has not yet been invented, thus the analysis here applies only to FIR DFE s with finite N b. By again denoting the equalized pulse response as v k (after removal of any bias), and assuming that any error signal, e k, in the DFE output can is uncorrelated with the symbol of interest, x k, this error signal can be decomposed into 4 constituent signals. precursor ISI - e pr,k = P i= v ix k i. postcursor ISI - e po,k = P i=n b + v ix k 3. filtered noise - e n,k = P i= w in k i i 4. feedback errors - e f,k = P N b i= v i (x k i ˆx k i ) he sum of the first 3 signals constitute the MMSE with power according to previous results. he last signal is often written MMSE-DFE, which can be computed XN b e f,k = v i k i (3.430) i= where k = x k ˆx k. his last distortion component has a discrete distribution with (M ) N b possible points. he error event vector, k,is k =[ k Nb + k Nb +... k ] (3.43) Given k, there are only M possible values for k. Equivalently, the evolution of an error event, can be described by a finite-state machine with (M ) N b states, each corresponding to one of the (M ) N b possible length-n b error events. Such a finite state machine is shown in 46

89 ! 0! 0! -! 0 0! -! 0 -! - 0! - -! Figure 3.47: Finite State Machine for N b = and M = in evaluating DFE Error Propagation Figure he probability that k takes on a specific value, or that the state transition diagram goes to the corresponding state at time k, is (denoting the corresponding new entry in k as k ) apple dmin e f,k ( k ) P k / k = Q. (3.43) MMSE-DFE here are M such values for each of the (M ) N b states. denotes a square (M ) N b (M ) N b matrix of transition probabilities where the i, j th element is the probability of entering state i, given the DFE is in state j. he column sums are thus all unity. For a matrix of non-negative entries, there is a famous Peron-Frobenious lemma from linear algebra that states that there is a unique eigenvector of all nonnegative entries that satisfies the equation =. (3.433) he solution is called the stationary state distribution or Markov distribution for the state transition diagram. he i th entry, i, is the steady-state probability of being in state i. Of course, P (M ) N b i= i =. By denoting the set of states for which k 6= 0 as E, one determines the probability of error as P e = X i. (3.434) ie hus, the DFE will have an accurate estimate of the error probability in the presence of error propagation. A larger state-transition diagram, corresponding to the explicit consideration of residual ISI as a discrete probability mass function, would yield a yet more accurate estimate of the error probability for the equalized channel - however, the relatively large magnitude of the error propagation samples usually makes their explicit consideration more important than the (usually) much smaller residual ISI. he number of states can be very large for reasonable values of M and N b, so that the calculation of the stationary distribution could exceed the computation required for a direct measurement of SNR with a DFE simulation. here are a number of methods that can be used to reduce the number of states in the finite-state machine, most of which will reduce the accuracy of the probability of error estimate. In the usual case, the constellation for x k is symmetric with respect to the origin, and there is essentially no di erence between k and k, so that the analysis may merge the two corresponding 47

90 states and only consider one of the error vectors. his can be done for almost half 5 the states in the state transition diagram, leading to a new state transition diagram with M N b states. Further analysis then proceeds as above, finding the stationary distribution and adding over states in E. here is essentially no di erence in this P e estimate with respect to the one estimated using all (M ) N b states; however, the number of states can remain unacceptably large. At a loss in P e accuracy, we may ignore error magnitudes and signs, and compute error statistics for binary error event vectors, which we now denote " =[",..., " Nb ], of the type (for N b = 3) [000], [00], [00], [0]. (3.435) his reduces the number of states to N b, but unfortunately the state transition probabilities no longer depend only on the previous state. hus, we must try to find an upper bound on these probabilities that depends only on the previous states. In so doing, the sum of the stationary probabilities corresponding to states in E will also upper bound the probability of error. E corresponds to those states with a nonzero entry in the first position (the odd states, " = ). o get an upperbound on each of the transition probabilities we write X P ",k =/" k = P ",k=/" k,",k(i)p ",k(i)} (3.436) {i ",k (i)is allowed transition from " k apple X i ",k (i)is allowed transition from " k max ",k (i) P ",k =/" k,",k (i)p ",k (i) (3.437) = max ",k (i) P ",k =/" k,",k (i). (3.438) Explicit computation of the maximum probability in (3.438) occurs by noting that this error probability corresponds to a worst-case signal o set of XN b max(") =(M )d v i ",k i, (3.439) which the reader will note is analogous to peak distortion (the distortion is understood to be the worst of the two QAM dimensions, which are assumed to be rotated so that d min lies along either or both of the dimensions ). As long as this quantity is less than the minimum distance between constellation points, the corresponding error probability is then upper bounded as 3 d min P ek /" k apple Q 4 q max(") 5. (3.440) MMSE-DFE Now, with the desired state-dependent (only) transition probabilities, the upper bound for P e with error propagation is 3 P e apple X d min Q 4 q max(") 5 P". (3.44) " MMSE-DFE Even in this case, the number of states N b can be too large. A further reduction to N b + states is possible, by grouping the N b states into groups that are classified only by the number of leading in " k ; thus, state i = corresponds to any state of the form [0 "], while state i = corresponds to [0 0 "...], etc. he upperbound on probability of error for transitions into any state, i, thenusesa max (i) given by 5 here is no reduction for zero entries. max(i) =(M )d i= XN b i=i+ v i, (3.44) 48

91 and max (N b ) = 0. Finally, a trivial bound that corresponds to noting that after an error is made, we have M N b possible following error event vectors that can correspond to error propagation (only the all zeros error event vector corresponds to no additional errors within the time-span of the feedback path). he probability of occurrence of these error event vectors is each no greater than the initial error probability, so they can all be considered as nearest neighbors. hus adding these to the original probability of the first error, P e (errorprop) apple M N b P e (first). (3.443) It should be obvious that this bound gives useful results only if N b is small (that is a probability of error bound of.5 for i.i.d. input data may be lower than this bound even for reasonable values of M and N b ). hat is suppose, the first probability of error is 0 5, and M = 8 and N b = 8, then this last (easily computed) bound gives P e apple 00! Look-Ahead ν M (" +" smallest ˆ E U, k * ( A ) k z U, k Δ (" B( A k Δ ) +" SBS' ν M ν M Memory' ν M A" D" D" R" E" S" S" A min' address xˆk Δ [ x x x ] k Δ = k Δ k Δ k Δ ν Figure 3.48: Look-Ahead mitigation of error propagation in DFEs. Figure 3.48 illustrates a look-ahead mechanism for reducing error propagation. Instead of using the decision, M possible decision vectors are retained. he vector is of dimension and can be viewed as an address A k to the memory. he possible output for each of the M is computed and subtracted from z U,k. M decisions of the symbol-by-symbol detector can then be computed and compared in terms of the distance from a potential symbol value, namely smallest E U,k. he smallest such distance is used to select the decision for ˆx k. his method is called look-ahead decoding basically because all possible previous decisions ISI are precomputed and stored, in some sense looking ahead in the calculation. If M calculations (or memory locations) is too complex, then the largest 0 < taps can be used (or typically the most recent 0 and the rest subtracted in typical DFE fashion for whatever the decisions previous to the 0 tap interval in a linear filter. Look-ahead leads to the Maximum Likelihood Sequence detection (MLSD) methods of Chapter 9 that are no longer symbol-by-symbol based. Lookahead methods can never exceed SNR MMSE DFE,U in terms of performance, but can come very close since error propagation can be very small. (MLSD methods can exceed SNR MMSE DFE,U.) 49

92 3.8 Precoding his section discusses solutions to the error-propagation problem of DFE s. he first is precoding, which essentially moves the feedback section of the DFE to the transmitter with a minimal (but nonzero) transmit-power-increase penalty, but with no reduction in DFE SNR. he second approach or partial response channels (which have trivial precoders) has no transmit power penalty, but may have an SNR loss in the DFE because the feedback section is fixed to a desirable preset value for B(D) rather than optimized value. his preset value (usually with all integer coe cients) leads to simplification of the ZF-DFE structure he omlinson Precoder Error propagation in the DFE can be a major concern in practical application of this receiver structure, especially if constellation-expanding codes, or convolutional codes (see Chapter 0), are used in concatenation with the DFE (because the error rate on the inner DFE is lower (worse) prior to the decoder). Error propagation is the result of an incorrect decision in the feedback section of the DFE that produces additional errors that would not have occurred if that first decision had been correct. he omlinson Precoder (PC), more recently known as a omlinson-harashima Precoder, is a device used to M!-!a!r!y!! I!n!p!u!t! x k ẑ k +! x~k Γ [ ~ M x k ] modulo!operator! ' x k a!p!p!r!o!x!i!m!a!t!e!l!y!! u!n!i!f!o!r!m!!i!n!!d!i!s!t!r!i!b!u!t!i!o!n!! o!v!e!r! [! Md /, Md / ) a!n!d!!i!.!i!.!d!.! B( D) a!)!.!!i!m!p!l!e!m!e!n!t!a!t!i!o!n!!f!o!r!!r!e!a!l!!b!a!s!e!b!a!n!d!!s!i!g!n!a!l!s! x,k x,k z ˆ,k +! z ˆ, k +! ~ x,k ~ x,k Γ Γ [ ~ ] [ ~ ] M x,k M x,k e!n!c!l!o!s!e!d!!b!o!x!!e!l!e!m!e!n!t!s!!c!a!n!!b!e!!i!m!p!l!e!m!e!n!t!e!d!! a!s!!t!w!o!-!d!i!m!e!n!s!i!o!n!a!l!!m!o!d!u!l!o! ' x, k ' x, k u!n!i!f!o!r!m!!o!v!e!r!! s!q!u!a!r!e!!r!e!g!i!o!n! B( D) c!o!m!p!l!e!x!! f!i!l!t!e!r! b!)!.!!i!m!p!l!e!m!e!n!t!a!t!i!o!n!!f!o!r!!c!o!m!p!l!e!x!!(!q!u!a!d!r!a!t!u!r!e!)!!s!i!g!n!a!l!s! Figure 3.49: he omlinson Precoder. eliminate error propagation and is shown in Figure Figure 3.49(a) illustrates the case for real 50

93 one-dimensional signals, while Figure 3.49(b) illustrates a generalization for complex signals. In the second complex case, the two real sums and two one-dimensional modulo operators can be generalized to a two-dimensional modulo where arithmetic is modulo a two-dimensional region that tesselates twodimensional space (for example, a hexagon, or a square). he omlinson precoder appears in the transmitter as a preprocessor to the modulator. he omlinson Precoder maps the data symbol x k into another data symbol x 0 k, which is in turn applied to the modulator (not shown in Figure 3.49). he basic idea is to move the DFE feedback section to the transmitter where decision errors are impossible. However, straightforward moving of the filter /B(D) to the transmitter could result in significant transmit-power increase. o prevent most of this power increase, modulo arithmetic is employed to bound the value of x 0 k : Definition 3.8. (Modulo Operator) he modulo operator M (x) is a nonlinear function, defined on an M-ary (PAM or QAM square) input constellation with uniform spacing d, such that M (x) =x Mdb x + Md Md c (3.444) where byc means the largest integer that is less than or equal to y. M (x) need not be an integer. his text denotes modulo M addition and subtraction by M and M respectively. hat is x M y = M [x + y] (3.445) and x M y = M [x y]. (3.446) For complex QAM, each dimension is treated modulo p M independently. Figure 3.50 illustrates modulo arithmetic for M = 4 PAM signals with d =. +!4! +!3! +!! 0! -!! -!3! -!4! +!3! +!4! -!4! -!3! +!! -!! 0! x! k! Γ!(! x!!! k!!)! +!3! +!3! -!3! -!3! -!4!.!5!+!3!.!5! -!6! +!! +!6!.!! -!!.!9! Figure 3.50: Illustration of modulo arithmetic operator. he following lemma notes that the modulo operation distributes over addition: Lemma 3.8. (Distribution of M (x) over addition) he modulo operator can be distributed over a sum in the following manner: M [x + y] = M (x) M M (y) (3.447) M [x y] = M (x) M M (y). (3.448) 5

94 he proof is trivial. he omlinson Precoder generates an internal signal where x k = x k x 0 k = M [ x k ]= M " X i= x k b i x 0 k i (3.449) X i= b i x 0 k i #. (3.450) he scaled-by-snr MMSE DFE /SNR MMSE DFE,U output of the MS-WMF in the receiver is an optimal unbiased MMSE approximation to X(D) G U (D). hat is apple SNRMMSE DFE E z k /[x k x k,...,] = SNR MMSE DFE,U X g U,i x k i. (3.45) hus, B(D) =G U (D). From Equation (3.54) with x 0 (D) as the new input, the scaled feedforward filter output with the omlinson precoder is! X z U,k = x 0 k + g U,i x 0 k i + e U,k. (3.45) M [z U,k ] is determined as i= X X M [z U,k ] = M [ M (x k g U,i x 0 k i)+ g U,i x 0 k i + e U,k ] (3.453) i= i= i= i=0 X X = M [x k g U,i x 0 k i + g U,i x 0 k i + e U,k ] (3.454) i= = M [x k + e U,k ] (3.455) = x k M M [e U,k ] (3.456) = x k + e 0 U,k. (3.457) Since the probability that the error e U,k being larger than Md/ in magnitude is small in a well-designed system, one can assume that the error sequence is of the same distribution and correlation properties after the modulo operation. hus, the omlinson Precoder has allowed reproduction of the input sequence at the (scaled) MS-WMF output, without ISI. he original omlinson work was done for the ZF-DFE, which is a special case of the theory here, with G U (D) =P c (D). he receiver corresponding to omlinson precoding y p ( t) * ϕ ( ) p t! y k w! k z k SNR SNR MMSE DFE MMSE DFE, U U k!! z, ( ) Γ M d!e!c!i!s!i!o!n! xˆk M!S!-!W!M! Γ M ( ) d!e!c!i!s!i!o!n! (!c!o!m!p!l!e!x!!c!a!s!e!!h!a!s!!q!u!a!d!r!a!t!u!r!e!! c!o!m!p!o!n!e!n!t!!i!n!!d!o!t!t!e!d!!l!i!n!e!s!)! xˆ k Figure 3.5: Receiver for omlinson precoded MMSE-DFE implementation. 5

95 is shown in Figure 3.5. he noise power at the feed-forward output is thus almost exactly the same as that of the corresponding MMSE-DFE and with no error progagation because there is no longer any need for the feedback section of the DFE. As stated here without proof, there is only a small price to pay in increased transmitter power when the PC is used. heorem 3.8. (omlinson Precoder Output) he omlinson Precoder output, when the input is an i.i.d. sequence, is also approximately i.i.d., and futhermore the output sequence is approximately uniform in distribution over the interval [ Md/,Md/). here is no explicit proof of this theorem for finite M, although it can be proved exactly as M!. his proof notes that the unbiased and biased receivers are identical as M!, because the SNR must also be infinite. hen, the modulo element is not really necessary, and the sequence x 0 k can be shown to be equivalent to a prediction-error or innovations sequence, which is known in the estimation literature to be i.i.d. he i.i.d. part of the theorem appears to be valid for almost any M. he distribution and autocorrelation properties of the PC, in closed form, remain an unsolved problem at present. Using the uniform distribution assumption, the increase in transmit power for the PC is from the nominal value of (M )d to the value for a continuous uniform random variable over the output interval [ Md/,Md/), which is M d, leading to an input power increase of M M (3.458) for PAM and correspondingly M M (3.459) for QAM. When the input is not a square constellation, the omlinson Precoder Power loss is usually larger, but never more than a few db. he number of nearest neighbors also increases to N e = for all constellations. hese losses can be eliminated (and actually a gain is possible) using techniques called rellis Precoding and/or shell mapping (See Section 0.6). Laroia or Flexible Precoder Figure 3.5 shows the Laroia precoder, which is a variation on omlinson precoding introduced by Rajiv Laroia, mainly to reduce the transmit-power loss of the omlinson precoder. he Laroia precoder largely preserves the shape of the transmitted constellation. he equivalent circuit is also shown in Figure 3.5 where the input is considered to be the di erence between the actual input symbol value x k and the decision output k. he decision device finds the closest point in the infinite extension of the constellation 6. he extension of the constellation is the set of points that continue to be spaced by d min from the points on the edges of the original constellation and along the same dimensions as the original constellation. m k is therefore a small error signal that is uniform in distribution over ( d/,d/), thus having variance d / << E x. he energy increase is therefore Ēx + d. (3.460) Ēx For PAM, this increase would be M 3 M 3 while similarly it would be for SQ QAM: d 4 d 4 + d 4 = + 3 M = M M, (3.46) M M, (3.46) 6 he maximum range of such an infinite extension is actually the sum P i= < or =g U,i R ee(d) or=x max. 53

96 Cross constellations take a little more e ort but follow the same basic idea above. Because of the combination of channel and feedforward equalizer filters, the feedforward filter output is Z U (D) =[X(D)+M(D)] G U (D)+E U (D) =X(D) (D)+E U (D). (3.463) Processing Z U (D) by a decision operation leaves X(D) to recover the input sequence X(D), the receiver forms (D), which essentially is the decision. However, Z 0 (D) = X(D) (D) = X(D)+M(D). (3.464) G U (D) Since M(D) is uniform and has magnitude always less than d/, then M(D) is removed by a second truncation (which is not really a decision, but operates using essentially the same logic as the first decision). x k +# +# m k +# +# +# %# λ k & $ $ $ % infinite extension for x k SBS# or up to largest value #!!! " G U ( D) x k + m k Γ d ( ) Laroia#precoder# xk λ k / G U ( D) x + m k k equivalent#circuit# y k W( D) SNR SNR MMSE DFE MMSE DFE, U x x k + m k λk k SBS# / G U ( D) SBS# all possible ( ) & $ % x k λ k #! " for x k xˆ k Receiver#for#flexible#precoding# Figure 3.5: Flexible Precoding. 54

97 Example and error propagation in flexible precoding x k +# +# flexible#precoder# x k + m k +# m k +# +# (# λ k ± Infinite#extension#not# needed#in#this#example# >0#:#+# <0#:#(# D# (.75# Receiver# y k D +# 0# (# xk λ k +.75D x k + m k >0#:#+# <0#:#(# xˆk Non(causal,# Realize#with#delay# out + in k =.75 outk k Figure 3.53: Flexible Precoding Example. EXAMPLE 3.8. (+.9D system with flexible precoding) he flexible precoder and corresponding receiver for the +.9D example appear in Figure he bias removal factor has been absorbed into all filters (7.85/6.85 multiples.633 to get.75 in feedback section, and multiplies.9469 to get.085 in feedforward section). he decision device is binary in this case since binary antipodal transmission is used. Note the IIR filter in the receiver. If an error is made in the first SBS in the receiver, then the magnitude of the error is = + ( ) = (+). Such an error is like an impulse of magnitude added to the input of the IIR filter, which has impulse response (.75) k u k, producing a contribution to the correct sequence of (.75) k u k. his produces an additional error half the time and an additional errors /4 the time. Longer strings of errors will not occur. hus, the bit and symbol error rate in this case increase by a factor of.5. (less than. db loss). A minor point is that the first decision in the receiver has an increased nearest neighbor coe cient of N e =.5. Generally speaking, when an error is made in the receiver for the flexible precoder, (x k k ) decision(x k k )= k (3.465) where could be complex for QAM. hen the designer needs to compute for each type of such error, its probability of occurence and then investigate the string (with (/b) k being the impulse response of the post-first-decision filter in the receiver) (/b) k << d min /. (3.466) For some k, this relation will hold and that is the maximum burst length possible for the particular type of error. Given the filter b is monic and minimum phase, this string should not be too long as long as the 55

98 roots are not too close to the unit circle. he single error may cause additional errors, but none of these additional errors in turn cause yet more errors (unlike a DFE where second and subsequent errors can lead to some small probability of infinite-length bursts), thus the probaility of an infinitely long burst is zero for the flexible precoder situation (or indeed any burst longer than the k that solves the above equation Partial Response Channel Models Partial-response methods are a special case of precoding design where the ISI is forced to some known well-defined pattern. Receiver detectors are then designed for such partial-response channels directly, exploiting the known nature of the ISI rather than attempting to eliminate this ISI. Classic uses of partial response abound in data transmission, some of which found very early use. For instance, the earliest use of telephone wires for data transmission inevitably found large intersymbol interference because of a transformer that was used to isolate DC currents at one end of a phone line from those at the other (see Prob 3.35). he transformer did not pass low frequencies (i.e., DC), thus inevitably leading to non-nyquist 7 pulse shapes even if the phone line otherwise introduced no signficant intersymbol interference. Equalization may be too complex or otherwise undesirable as a solution, so a receiver can use a detector design that instead presumes the presence of a known fixed ISI. Another example is magnetic recording (see Problem 3.36) where only flux changes on a recording surface can be sensed by a read-head, and thus D.C. will not pass through the read channel, again inevitably leading to ISI. Straightforward equalization is often too expensive at the very high speeds of magnetic-disk recording systems. Partial-Response (PR) channels have non-nyquist pulse responses that is, PR channels allow intersymbol interference over a few (and finite number of) sampling periods. A unit-valued sample of the response occurs at time zero and the remaining non-zero response samples at subsequent sampling times, thus the name partial response. he study of PR channels is facilitated by mathematically presuming the tacit existence of a whitened matched filter, as will be described shortly. hen, a number of common PR channels can be easily addressed. 7 hat is, they do not satisfy Nyquist s condition for nonzero ISI - See Chapter 3, Section 3. 56

99 x k p( t) n p ( t) x p ( t) y p ( t) * +! ( ) ϕ p t * * p η H ( D ) s!a!m!p!l!e!! a!t!!t!i!m!e!s!! k!! 0 y k Whitened!matched!filter! (a)!isi4channel!model! n k xk hk y~k +! yk (b)!par:al4response!channel! Figure 3.54: Minimum Phase Equivalent Channel. Figure 3.54(a) illustrates the whitened-matched-filter. he minimum-phase equivalent of Figure 3.54(b) exists if Q(D) = 0 H(D) H (D ) is factorizable. 8 his chapter focuses on the discrete-time channel and presumes the WMF s presence without explicitly showing or considering it. he signal output of the discrete-time channel is or y k = X m= h m x k m + n k, (3.467) Y (D) =H(D) X(D)+N(D), (3.468) where n k is sampled Gaussian noise with autocorrelation r nn,k = kpk 0 N0 k or R nn (D) = kpk 0 N0 when the whitened-matched filter is used, and just denoted pr otherwise. In both cases, this noise is exactly AWGN with mean-square sample value pr, which is conveniently abbreviated for the duration of this chapter. Definition 3.8. (Partial-Response Channel) A partial-response (PR) channel is any discrete-time channel with input/output relation described in (3.467) or (3.468) that also satisfies the following properties:. h k is finite-length and causal; h k =0 8 k<0 or k>, where < and is often called the constraint length of the partial-response channel,. h k is monic; h 0 =, 3. h k is minimum phase; H(D) has all roots on or outside the unit circle, 4. h k has all integer coe cients; h k Z8k 6= 0(where Z denotes the set of integers). 8 H(D) =P c(d) insection3.6.3onzf-dfe. 57

100 More generally, a discrete finite-length channel satisfying all properties above except the last restriction to all integer coe cients is known as a controlled intersymbol-interference channel. Controlled ISI and PR channels are a special subcase of all ISI channels, for which h k =08 k>. is the constraint length of the controlled ISI channel. hus, e ectively, the channel can be modeled as an FIR filter, and h k is the minimum-phase equivalent of the sampled pulse response of that FIR filter. he constraint length defines the span in time of the non-zero samples of this FIR channel model. he controlled ISI polynomial (D-transform) 9 for the channel simplifies to H(D) = X h m D m, (3.469) m=0 where H(D) is always monic, causal, minimum-phase, and an all-zero (FIR) polynomial. If the receiver processes the channel output of an ISI-channel with the same whitened-matched filter that occurs in the ZF-DFE of Section 3.6, and if P c (D) (the resulting discrete-time minimum-phase equivalent channel polynomial when it exists) is of finite degree, then the channel is a controlled intersymbol interference channel with H(D) =P c (D) and = N0 kpk 0. Any controlled intersymbol-interference channel is in the form that the omlinson Precoder of Section 3.8. could be used to implement symbol-bysymbol detection on the channel output. As noted in Section 3.8., a (usually small) transmit symbol energy increase occurs when the omlinson Precoder is used. Section shows that this loss can be avoided for the special class of polynomials H(D) that are partial-response channels. Equalizers for the Partial-response channel. his small subsection serves to simplify and review equalization structure on the partial-response channel. he combination of integer coe cients and minimum-phase constraints of partial-response channels allows a very simple implementation of the ZF-DFE as in Figure Since the PR channel is already discrete time, no sampling is necessary and the minimum-phase characteristic of H(D) causes the WMF of the ZF-DFE to simplify to a combined transfer function of (that is, there is no feedforward section because the channel is already minimum phase). he ZF-DFE is then just the feedback section shown, which easily consists of the feedback coe cients h m for the delayed decisions corresponding to ˆx k m,m=,...,. he loss with respect to the matched filter bound is trivially /khk, which is easily computed as 3 db for H(D) =± D (any finite ) with simple ZF-DFE operation z k = y k ˆx k and 6 db for H(D) =+D D D 3 with simple ZF-DFE operation z k = y k ˆx k +ˆx k +ˆx k 3. 9 he D-ransform of FIR channels ( <) isoftencalledthe channelpolynomial, ratherthanits D-ransform in partial-response theory. his text uses these two terms interchangeably. 58

101 Simple"DFE"operaons" " H for" " for" " H ( D) = D, zk = yk xk ˆ 3 ( D) = + D D D, z = ˆ ˆ ˆ k yk xk + xk + xk 3 y k " +" SBS" z k whitened:matched"filter" h xˆk h ν h D" " D" D" Figure 3.55: Simple DFE for some partial-response channels. he ZF-LE does not exist if the channel has a zero on the unit circle, which all partial-response channels do. A MMSE-LE could be expected to have signficant noise enhancement while existing. he MMSE-DFE will perform slightly better than the ZF-DFE, but is not so easy to compute as the simple DFE. omlinson or Flexible precoding could be applied to eliminate error propagation for a small increase in transmit power ( M M ) Classes of Partial Response A particularly important and widely used class of partial response channels are those with H(D) given by H(D) =(+D) l ( D) n, (3.470) where l and n are nonnegative integers. For illustration, Let l = and n = 0 in (3.470), then H(D) =+D, (3.47) which is sometimes called a duobinary channel (introduced by Lender in 960). he Fourier transform of the duobinary channel is! H(e! )=H(D) D=e! =+e! =e!/ cos. (3.47) he transfer function in (3.47) has a notch at the Nyquist Frequency and is generally lowpass in shape, as is shown in Figure

102 !# 0# # # 3# Figure 3.56: ransfer Characteristic for Duobinary signaling. A discrete-time ZFE operating on this channel would produce infinite noise enhancement. If SNR MFB = 6 db for this channel, then Q(e! )+ he MMSE-LE will have performance (observe that N0 =.05) MMSE-LE = N 0 Z SNR MFB =(+/40) + cos!. (3.473) kpk (.05 + cos!) d! = N 0 / p.05 =.N 0, (3.474) he SNR MMSE LE,U is easily computed to be 8 (9dB), so the equalizer loss is 7 db in this case. For the MMSE-DFE, Q(D)+ SNR =.05 MFB.64 ( +.8D)( +.8D ), so that 0 =.05/.64 =.65, and thus MMSE-DFE =.db. For the ZF-DFE, 0 =, and thus ZF-DFE = 3dB. It is possible to achieve MFB performance on this channel with complexity far less than any equalizer studied earlier in this chapter, as will be shown in Chapter 9. It is also possible to use a precoder with no transmit power increase to eliminate the error-propagation-prone feedback section of the ZF-DFE. here are several specific channels that are used in practice for partial-response detection: EXAMPLE 3.8. (Duobinary + D) he duobinary channel (as we have already seen) has H(D) =+D. (3.475) he frequency response was already plotted in Figure his response goes to zero at the Nyquist Frequency, thus modeling a lowpass-like channel. For a binary input of x k = ±, the channel output (with zero noise) takes on values ± with probability /4 each and 0 with probability /. In general, for M-level inputs (± ± 3 ± 5... ± (M )), there are M possible output levels, M +,..., 0,..., M. hese output values are all possible sums of pairs of input symbols. 60

103 EXAMPLE (DC Notch D) he DC Notch channel has so that l = 0 and n = in (3.470). he frequency response is H(D) = D, (3.476) H(e! )= e! = e! sin!. (3.477) he response goes to zero at the DC (! = 0), thus modeling a highpass-like channel. For a binary input of x k = ±, the channel output (with zero noise) takes on values ± with probability /4 each and 0 with probability /. In general for M-level inputs (± ± 3 ± 5... ± (M )), there are M possible output levels, M +,...,0,...,M. When the D shaping is imposed in the modulator itself, rather than by a channel, the corresponding modulation is known as AMI (Alternate Mark Inversion) if a di erential encoder is also used as shown later in this section. AMI modulation prevents charge (DC) from accumulating and is sometimes also called bipolar coding, although the use of the latter term is often confusing because bipolar transmission may have other meanings for some communications engineers. AMI coding, and closely related methods are used in multiplexed (.544 Mbps DS or ANSI.403 ) and E (.048 Mbps or IU- G.703 ) speed digital data transmission on twisted pairs or coaxial links. hese signals were once prevalent in telephone-company non-fiber central-o ce transmission of data between switch elements. EXAMPLE (Modified Duobinary D ) he modified duobinary channel has H(D) = D =(+D)( D), (3.478) so l = n = in (3.470). Modified Duobinary is sometimes also called Partial Response Class IV or PR4 or PRIV in the literature. he frequency response is H(e! )= e! = e! sin(!). (3.479) he response goes to zero at the DC (! = 0) and at the Nyquist frequency (! = / ), thus modeling a bandpass-like channel. For a binary input of x k = ±, the channel output (with zero noise) takes on values ± with probability /4 each and 0 with probability /. In general, for M-level inputs (± ± 3 ± 5... ± (M )), there are M possible output levels. Modified duobinary is equivalent to two interleaved D channels, each independently acting on the inputs corresponding to even (odd) time samples, respectively. Many commercial disk drives use PR4. 6

104 magnitude EPR4% EPR6% 3% % % 0% -% -% -3% -% 0% % % 3% 4% 5% 6% time sample% EPR6% normalized frequency time sample EPR4% Figure 3.57: ransfer Characteristic for EPR4 and EPR6 EXAMPLE (Extended Partial Response 4 and 6 ( + D) l ( channel has l = and n = or D) n ) he EPR4 H(D) =(+D) ( D) =+D D D 3. (3.480) his channel is called EPR4 because it has 4 non-zero samples (hapar). he frequency response is H(e! )=(+e! ) ( e! ). (3.48) he EPR6 channel has l = 4 and n = (6 nonzero samples) H(D) =(+D) 4 ( D) =+3D +D D 3 3D 4 D 5. (3.48) he frequency response is H(e! )=(+e! ) 4 ( e! ). (3.483) hese channels, along with PR4, are often used to model disk storage channels in magnetic disk or tape recording. he response goes to zero at DC (! = 0) and at the Nyquist frequency in both EPR4 and EPR6, thus modeling bandpass-like channels. he magnitude of these two frequency characteristics are shown in Figure hese are increasingly used in commercial disk storage read detectors. he higher the l, the more lowpass in nature that the EPR channel becomes, and the more appropriate as bit density increases on any given disk. For partial-response channels, the use of omlinson Precoding permits symbol-by-symbol detection, but also incurs an M /(M ) signal energy loss for PAM (and M/(M ) for QAM). A simpler method for PR channels, that also has no transmit energy penalty, is known simply as a precoder Simple Precoding he simplest form of precoding for the duobinary, DC-notch, and modified duobinary partial-response channels is the so-called di erential encoder. he message at time k, m k, is assumed to take on values m =0,,...,M. he di erential encoder for the case of M = is most simply described as 6

105 the device that observes the input bit stream, and changes its output if the input is and repeats the last output if the input is 0. hus, the di erential encoder input, m k, represents the di erence (or sum) between adjacent di erential encoder output ( m k and m k ) messages 0 : Definition (Di erential Encoder) Di erential encoders for PAM or QAM modulation obey one of the two following relationships: m k = m k m k (3.484) m k = m k m k, (3.485) where represents subtraction modulo M (and represents addition modulo M). For SQ QAM, the modulo addition and subtraction are performed independently on each of the two dimensions, with p M replacing M. (A di erential phase encoder is also often used for QAM and is discussed later in this section.) A di erential encoder is shown on the left side of Figure As an example if M = 4 the corresponding inputs and outputs are given in the following table: m k m k k With either PAM or QAM constellations, the dimensions of the encoder output m k are converted into channel input symbols (± d, ± 3d...) according to x k =[ m k (M )] d. (3.486) n k m k!" m k d [ mk ( M ) ] x k y~k + D +" y k 6 D" M = 6 0 M = 4 Figure 3.58: Precoded Partial Response Channel. Decoding"rules"(d=)" Figure 3.58 illustrates the duobinary channel H(D) = + D, augmented by the precoding operation of di erential encoding as defined in (3.484). he noiseless minimum-phase-equivalent channel output ỹ k is ỹ k = x k + x k = ( m k + m k ) d (M ) d (3.487) 0 his operation is also very useful even on channels without ISI, as an unknown inversion in the channel (for instance, an odd number of amplifiers) will cause all bits to be in error if (di erential) precoding is not used. 63

106 so ỹ k d +(M ) = m k + m k (3.488) appleỹk +(M ) = m k m k (3.489) d M appleỹk +(M ) = m k (3.490) d M where the last relation (3.490) follows from the definition in (3.484). All operations are integer mod-m. Equation (3.490) shows that a decision on ỹ k about m k can be made without concern for preceding or succeeding ỹ k, because of the action of the precoder. he decision boundaries are simply the obvious regions symmetrically placed around each point for a memoryless ML detector of inputs and the decoding rules for M = and M = 4 are shown in Figure In practice, the decoder observes y k, not ỹ k,so that y k must first be quantized to the closest noise-free level of y k. hat is, the decoder first quantizes ỹ k to one of the values of ( M + )(d/),..., (M )(d/). hen, the minimum distance between outputs is d min = d, so that d min pr = d pr,whichis3dbbelowthe p MFB = p (d/) for the + D pr channel because kpk =. (Again, pr = kpk 0 N0 with a WMF and otherwise is just pr.) his loss is identical to the loss in the ZF-DFE for this channel. One may consider the feedback section of the ZF-DFE as having been pushed through the linear channel back to the transmitter, where it becomes the precoder. With some algebra, one can show that the PC, while also e ective, would produce a 4-level output with.3 db higher average transmit symbol energy for binary inputs. he output levels are assumed equally likely in determining the decision boundaries (half way between the levels) even though these levels are not equally likely. he precoded partial-response system eliminates error propagation, and thus has lower P e than the ZF-DFE. his elimination of error propagation can be understood by investigating the nearest neighbor coe cient for the precoded situation in general. For the + D channel, the (noiseless) channel output levels are (M ) (M 4) (M 4) (M ) with probabilities of occurrence M M... M M... M M, assuming a uniform channel-input distribution. Only the two outer-most levels have one nearest neighbor, all the rest have nearest neighbors. hus, N e = M () + M M () = For the ZF-DFE, the input to the decision device z k as which can be rewritten M. (3.49) z k = x k + x k + n k ˆx k, (3.49) z k = x k +(x k ˆx k )+n k. (3.493) Equation becomes z k = x k + n k if the previous decision was correct on the previous symbol. However, if the previous decision was incorrect, say + was decided (binary case) instead of the correct, then z k = x k +n k, (3.494) which will lead to a next-symbol error immediately following the first almost surely if x k = (and no error almost surely if x k = ). he possibility of z k = x k ++n k is just as likely to occur and follows an identical analysis with signs reversed. For either case, the probability of a second error propagating is /. he other half of the time, only error occurs. Half the times that errors occur, a third error also occurs, and so on, e ectively increasing the error coe cient from N e =to N e = = ( ) + ( 4 ) + 3( 8 ) + 4( 6 )+... = X k (.5) k.5 = (.5). (3.495) ( ) M means the quantity is computed in M-level arithmetic, for instance, (5) 4 =. Alsonotethat M (x) 6= (x) M, and therefore is di erent from the M of Section 3.5. he functions (x) M and have strictly integer inputs and have possible outputs 0,...,M only. k= 64

107 (In general, the formula P k= k rk = r ( r),r < may be useful in error propagation analysis.) he error propagation can be worse in multilevel PAM transmission, when the probability of a second error is (M )/M, leading to N e (error prop) N e (no error prop) = M + M M X k M = k M k= M +3 M M M +... (3.496) M (3.497) = M. (3.498) Precoding eliminates this type of error propagation, although N e increases by a factor of ( + /M ) with respect to the case where no error propagation occurred. For the ZF-DFE system, Pe = (M )Q( d pr ), while for the precoded partial-response system, P e = M Q( d pr ). For M, the precoded system always has the same or fewer nearest neighbors, and the advantage becomes particularly pronounced for large M. Using a rule-of-thumb that a factor of increase in nearest neighbors is equivalent to an SNR loss of. db (which holds at reasonable error rates in the 0 5 to 0 6 range), the advantage of precoding is almost. db for M = 4. For M = 8, the advantage is about.6 db, and for M = 64, almost. db. Precoding can be simpler to implement than a ZF-DFE because the integer partial-response channel coe cients translate readily into easily realized finite-field operations in the precoder, while they represent full-precision add (and shift) operations in feedback section of the ZF-DFE. Precoding the DC Notch or Modified Duobinary Channels he m k = m k m k di erential encoder works for the + D channel. For the D channel, the equivalent precoder is m k = m k m k, (3.499) which is sometimes also called NRZI (non-return-to-zero inverted) precoding, especially by storagechannel engineers. In the D case, the channel output is ỹ k = x k x k = ( m k m k ) d (3.500) ỹ k d ỹk d M ỹk d M = m k m k (3.50) = m k m k (3.50) = m k. (3.503) he minimum distance and number of nearest neighbors are otherwise identical to the + D case just studied, as is the improvement over the ZF-DFE. he D case is identical to the D case, on two interleaved D channels at half the rate. he overall precoder for this situation is m k = m k m k, (3.504) and the decision rule is ỹk = m k. (3.505) d M he combination of precoding with the D channel is often called alternate mark inversion (AMI) because each successive transmitted bit value causes a nonzero channel output amplitude of polarity oppositie to the last nonzero channel output amplitude, while a 0 bit always produces a 0 level at the channel output. he ratio of ( /M )withprecodingto( /M )form-arypamwithnoerrorpropagatione ectsincluded 65

108 Precoding EPR4 An example of precoding for the extended Partial Response class is EPR4, which has l = and n = in (3.480), or EPR4. hen, ỹ k d ỹk d M ỹk d where the precoder, from (3.50) and (3.509), is ỹ k = x k + x k x k x k 3 (3.506) ỹ k = d( m k + m k m k m k 3 ) (3.507) M = m k + m k m k m k 3 (3.508) = m k m k m k m k 3 (3.509) = m k (3.50) m k = m k m k m k m k 3. (3.5) he minimum distance at the channel output is still d in this case, so P e apple N e Q(d/ pr ), but the MFB = ( d ), which is 6dB higher. he same 6dB loss that would occur with an error-propagation-free pr ZF-DFE on this channel. For the + D D D 3 channel, the (noiseless) channel output levels are (4M 4) (4M 6) (4M 6) (4M 4). Only the two outer-most levels have one nearest neighbor, all the rest have nearest neighbors. hus, N e = M 4 () + M 4 M 4 () = M 4. (3.5) hus the probability of error for the precoded system is P e = M Q( ). he number of nearest 4 pr neighbors for the ZF-DFE, due to error propagation, is di cult to compute, but clearly will be worse General Precoding he general partial response precoder can be extrapolated from previous results: Definition (he Partial-Response Precoder) he partial-response precoder for a channel with partial-response polynomial H(D) is defined by m k = m k M ( h i ) m k i. (3.53) i= he notation L means a mod-m summation, and the multiplication can be performed without ambiguity because the h i and m k i are always integers. he corresponding memoryless decision at the channel output is X! ˆm k = ŷk M d + h i i=0 M. (3.54) he reader should be aware that while the relationship in (3.53) is general for partial-response channels, the relationship can often simplify in specific instances, for instance the precoder for EPR5, H(D) = ( + D) 3 ( D) simplifiesto m k = m k m k 4 when M =. In a slight abuse of notation, engineers often simplify the representation of the precoder by simply writing it as P(D) = (3.55) H(D) 66

109 where P(D) is a polynomial in D that is used to describe the modulo-m filtering in (3.53). Of course, this notation is symbolic. Furthermore, one should recognize that the D means unit delay in a finite field, and is therefore a delay operator only one cannot compute the Fourier transform by inserting D = e! into P(D); Nevertheless, engineers commonly to refer to the NRZI precoder as a /( D) precoder. Lemma 3.8. (Memoryless Decisions for the Partial-Response Precoder) he partialresponse precoder, often abbreviated by P(D) =/H(D), enables symbol-by-symbol decoding on the partial-response channel H(D). An upper bound on the performance of such symbolby-symbol decoding is P e apple apple M + Q d pr, (3.56) where d is the minimum distance of the constellation that is output to the channel. Proof: he proof follows by simply inserting (3.53) into the expression for ỹ k and simplifying to cancel all terms with h i, i>0. he nearest-neighbor coe cient and d/ pr follow trivially from inspection of the output. Adjacent levels can be no closer than d on a partialresponse channel, and if all such adjacent levels are assumed to occur for an upper bound on probability of error, then the P e bound in (3.56) holds. QED Quadrature PR Di erential encoding of the type specified earlier is not often used with QAM systems, because QAM constellations usually exhibit 90 o symmetry. hus a 90 o o set in carrier recovery would make the constellation appear exactly as the original constellation. o eliminate the ambiguity, the bit assignment for QAM constellations usually uses a precoder that uses the four possibilities of the most-significant bits that represent each symbol to specify a phase rotation of 0 o, 90 o, 80 o, or 70 o with respect to the last symbol transmitted. For instance, the sequence 0,, 0, 00 would produce (assuming an initial phase of 0 o, the sequence of subsequent phases 90 o,0 o, 80 o, and 80 o. By comparing adjacent decisions and their phase di erence, these two bits can be resolved without ambiguity even in the presence of unknown phase shifts of multiples of 90 o. his type of encoding is known as di erential phase encoding and the remaining bits are assigned to points in large M QAM constellations so that they are the same for points that are just 90 o rotations of one another. (Similar methods could easily be derived using 3 or more bits for constellations with even greater symmetry, like 8PSK.) hus the simple precoder for the + D and D (or D ) given earlier really only is practical for PAM systems. he following type of precoder is more practical for these channels: 67

110 ω" c" #"π" " ω" c" ω" c" +"π" " ("0","")"="" ("","")"="3" " #"" " #"" ("0","0")"="0" ("","0")"="" +" D" "+"D" ("0","0")"="0" ("","0")"="" ("0","0")"="0" ("0","0")"="0" " ("0","")"="" ("0","")"="" ("","")"="3" #"" 0" #"" ("","0")"="" " ("0","0")"="0" Figure 3.59: Quadrature Partial Response Example with + D. A Quadrature Partial Response (QPR) situation is specifically illustrated in Figure 3.59 for M = 4 or b = bit per dimension. he previous di erential encoder could be applied individually to both dimensions of this channel, and it could be decoded without error. All previous analysis is correct, individually, for each dimension. here is, however, one practical problem with this approach: If the channel were to somehow rotate the phase by ±90 o (that is, the carrier recovery system locked on the wrong phase because of the symmetry in the output), then there would be an ambiguity as to which part was real and which was imaginary. Figure 3.59 illustrates the ambiguity: the two messages (0, ) = and (, 0) = commute if the channel has an unknown phase shift of ±90 o. No ambiguity exists for either the message (0, 0) = 0 or the message (, ) = 3. o eliminate the ambiguity,the precoder encodes the and signals into a di erence between the last (or ) that was transmitted. his precoder thus specifies that an input of (or ) maps to no change with respect to the last input of (or ), while an input of maps to a change with respect to the last input of or. A precoding rule that will eliminate the ambiguity is then Rule 3.8. (Complex Precoding for the +D Channel) if m k =(m i,k,m q,k )=(0, 0) or (, ) then m i,k = m i,k m i,k (3.57) m q,k = m q,k m q,k (3.58) else if m k =(m i,k,m q,k )=(0, ) or (, 0), check the last m =(0, ) or (, 0) transmitted, call it m 90,(thatis,was m 90 =or?). If m 90 =, the precoder leaves m k unchanged prior to di erential encoding according to (3.57) and (3.58). he operations and are the same in binary arithmetic. If m 90 =, then the precoder changes m k from to (or from to ) prior to encoding according to (3.57) and (3.58). 68

111 he corresponding decoding rule is (keeping a similar state ŷ 90 =, at the decoder) 8 (0, 0) ŷ k =[± ± ] >< (, ) ŷ ˆm k = k =[00]. (3.59) (0, ) (ŷ k =[0 ± ] or ŷ k =[±0 ]) and 6 ŷ k 6 ŷ 90 =0 >: (, 0) (ŷ k =[0 ± ] or ŷ k =[±0 ]) and 6 ŷ k 6 ŷ 90 6=0 he probability of error and minimum distance are the same as was demonstrated earlier for this type of precoder, which only resolves the 90 o ambiguity, but is otherwise equivalent to a di erential encoder. here will however be limited error propagation in that one detection error on the or message points leads to two decoded symbol errors on the decoder output. 69

112 3.9 Diversity Equalization Diversity in transmission occurs when there are multiple channels from a single message source to several receivers, as illustrated in Figure 3.60, and as detailed in Subsection Optimally, the principles of Chapter apply directly where the channel s conditional probability distribution p y/x typically has a larger channel-output dimensionality for y than for the input, x, N y >N x. his diversity often leads to a lower probability of error for the same message, mainly because a greater channel-output minimum distance between possible (noiseless) output data symbols can be achieved with a larger number of channel output dimensions. However, intersymbol interference between successive transmissions, along with interference between the diversity dimensions, again can lead to a potentially complex optimum receiver and detector. hus, equalization again allows productive use of suboptimal SBS detectors with diversity. he diversity equalizers become matrix equivalents of those studied in Sections Multiple Received Signals and the RAKE x k p 0 ( t ) p ( t ) p L ( t ) n 0 ( t ) +" n ( t ) +" n L ( t ) +" y 0 ( t ) y ( t ) y L ( t ) Figure 3.60: Basic diversity channel model. Figure 3.60 illustrates the basic diversity channel. Channel outputs caused by the same channel input have labels y l (t), l =0,...,L. hese channel outputs can be created intentionally by retransmission of the same data symbols at di erent times and/or (center) frequencies. Spatial diversity often occurs in wireless transmission where L spatially separated antennas may all receive the same transmitted signal, but possibly with di erent filtering and with noises that are at least partially independent. With each channel output following the model y p,l (t) = X k x k p l (t k)+n l (t), (3.50) 70

113 a corresponding L vector channel description is y p (t) = X k x k p(t k)+n p (t), (3.5) where y p (t) = 6 4 y 0 (t) y (t) y L. (t) p(t) = 6 4 p 0 (t) p (t) p L. (t) and n p(t) = 6 4 n 0 (t) n (t) n L. (t) (3.5) From Section..5, generalization of an inner product < x(t), y(t) >= X i Z x i (t)y i (t)dt. (3.53) Without loss of generality, the noise can be considered to be white on each of the L diversity channels, independent of the other diversity channels, and with equal power spectral densities N0.3 A single transmission of x 0 corresponds to a vector signal y p (t) =x 0 p(t)+n(t) =x 0 kpk ' p (t)+n(t). (3.54) his situation generalizes slightly that considered in Chapter, where matched-filter demodulators there combined all time instants through integration, a generalization of the inner product s usual sum of products. A matched filter in general simply combines the signal components from all dimensions that have independent (pre-whitened if necessary) noise. Here, the inner product includes also the components corresponding to each of the diversity channels so that all signal contributions are summed to create maximum signal-to-noise ratio. he relative weighting of the di erent diversity channels is thus maintained through L unnormalized parallel matched filters each corresponding to one of the diversity channels. When several copies are combined across several diversity channels or new dimensions (whether created in frequency, long delays in time, or space), the combination is known as the RAKE matched filter of Figure 3.6: Definition 3.9. (RAKE matched filter) A RAKE matched filter is a set of parallel matched filters each operating on one of the diversity channels in a diversity transmission system that is followed by a summing device as shown in Figure 3.6. Mathematically, the operation is denoted by LX y p (t) = p l ( t) y l (t). (3.55) l=0 he RAKE was originally so named by Green and Price in 958 because of the analogy of the various matched filters being the fingers of a garden rake and the sum corresponding to the collection of the fingers at the rake s pole handle. he RAKE is sometimes also called a diversity combiner, although the latter term also applies to other lower-performance suboptimal combining methods that do not maximize overall signal-to-noise strength through matched filter. One structure, often called maximal combining, applies a matched filter only to the strongest of the L diversity paths to save complexity. he equivalent channel for this situation is then the channel corresponding to this maximum-strength individual path. he original RAKE concept was conceived in connection with a spread-spectrum transmission method that achieves diversity essentially in frequency (but more precisely in a codedivision dimension to be discussed in the appendix of Chapter 5), but the matched filtering implied is 3 In practice, the noises may be correlated with each other on di erent subchannels and not white with covariance matrix R n(t) andpowerspectraldensitymatrixr n(f) = N 0 Rn / (f)rn / (f). By prefiltering the vector channel output by the matrix filter Rn / (f), the noise will be whitened and the noise equivalent matrix channel becomes P (f)! Rn / (f)p (f). Anaylsis with the equivalent channel can then proceed as if the noise where white, independent on the diversity channels, and of the same variance N 0 on all. 7

114 easily generalized. Some of those who later studied diversity combining were not aware of the connection to the RAKE and thus the multiple names for the same structure, although diversity combining is a more accurate name for the method. his text also defines r l (t) =p l (t) p l ( t) and for an equivalent RAKE-output equivalent channel and the norm hen, the normalized equivalent channel q(t) is defined through he sampled RAKE output has D-transform LX r(t) = r l (t) (3.56) l=0 LX kpk = kp l k. (3.57) l=0 r(t) =kpk q(t). (3.58) Y (D) =X(D) kpk Q(D)+N(D), (3.59) which is essentially the same as the early channel models used without diversity except for the additional scale factor of kpk, which also occurs in the noise autcorrelation, which is R nn (D) = N 0 kpk Q(D). (3.530) An SNR MFB = Ēxkpk N 0 and all other detector and receiver principles previously developed in this text now apply directly. y ( t ) 0 y ( t ) y L ( t ) ( ) * p t 0 ( ) * p t +" p * (- t) L y p ( t ) Figure 3.6: he basic RAKE matched filter combiner. If there were no ISI, or only one-shot transmission, the RAKE plus symbol-by-symbol detection would be an optimum ML/MAP detector. However, from Subsection 3..3, the set of samples created by this set of matched filters is su cient. hus, a single equalizer can be applied to the sum of the matched filter outputs without loss of generality as in Subsection However, if the matched filters are absorbed into a fractionally-spaced and/or FIR equalizer, then the equalizers become distinct set of coe cients for each rail before being summed and input to a symbol-by-symbol detector as in Subsection

115 3.9. Infinite-length MMSE Equalization Structures he sampled RAKE output can be scaled by the factor kpk to obtain a model identical to that found earlier in Section 3. in Equation (3.6) with Q(D) and kpk as defined in Subsection hus, the MMSE-DFE, MMSE-LE, and ZF-LE/DFE all follow exactly as in Sections he matched-filter bound SNR and each of the equalization structures tend to work better with diversity because kpk is typically larger on the equivalent channel created by the RAKE. Indeed the RAKE will work better than any of the individual channels, or any subset of the diversity channels, with each of the equalizer structures. Often while one diversity channel has severe characteristics, like an inband notch or poor transmission characteristic, a second channel is better. hus, diversity systems tend to be more robust. EXAMPLE 3.9. (wo ISI channels in parallel) Figure 3.6 illustrates two diversity channels with the same input and di erent intersymbol interference. he first upper channel has a sampled time equivalent of +.9D with noise variance per sample of.8 (and thus could be the channel consistently examined throughout this Chapter so Ēx = ). his channel is in e ect anti-causal (or in reality, the.9 comes first). A second channel has causal response +.8D with noise variance.64 per sample and is independent of the noise in the first channel. he ISI e ectively spans 3 symbol periods among the two channels at a common receiver that will decide whether x k = ± has been transmitted. he SNR MFB for this channel remains SNR MFB = Ēx kpk N 0, but it remains to compute this quantity correctly. First the noise needs to be whitened. While the two noises are independent, they do not have the same variance per sample, so a pre-whitening matrix is " 0 and so then the energy quantified by kpk is hen 0 q.8.64 kpk = kp k + k p k =.8 + # (3.53).8.64 = (.8). (3.53).64 SNR MFB = (.8) = 3 db. (3.533).8 Because of diversity, this channel has a higher potential performance than the single channel alone. Clearly having a second look at the input through another channel can t hurt (even if there is more ISI now). he ISI is characterized as always by Q(D), which in this case is apple Q(D) = ( +.9D )( +.9D)+.8 (.8).64 ( +.8D)( +.8D ) (3.534) =.49D ++.49D (3.535) =.589 ( +.835D) ( +.835D ) (3.536) Q(D) =.49D +(+/0) +.49D (3.537) =.708 ( +.695D) ( +.695D ) (3.538) hus, the SNR of a MMSE-DFE would be SNR MMSE DFE,U =.708(0) = 3.6 (.5 db). (3.539) he improvement of diversity with respect to a single channel is about.8 db in this case. he receiver is a MMSE-DFE essentially designed for the +.839D ISI channel after adding the matched-filter outputs. he loss with respect to the MFB is about.8 db. 73

116 .8" x k +.9D +".64" +.8D +" & $ % 0 0 #.8!.64 " +.9D +.8D +" ( ) (.587) ( +.839D ).8 x" dec" +" -".839" ˆ x k D" Figure 3.6: wo channel example Finite-length Multidimensional Equalizers he case of finite-length diversity equalization becomes more complex because the matched filters are implemented within the (possibly fractionally spaced) equalizers associated with each of the diversity subchannels. here may be thus many coe cients in such a diversity equalizer. 4 In the case of finite-length equalizers shown in figure 3.63, each of the matched filters of the RAKE is replaced by a lowpass filter of wider bandwidth (usually l times wider as in Section 3.7), a sampling device at rate l/, and a fractionally spaced equalizer prior to the summing device. With vectors p k in Equation (3.93) now becoming l L-tuples, p k = p(k), (3.540) as do the channel output vectors y k and the noise vector n k, the channel input/output relationship (in Eq (3.95)) again becomes Y k = = y k y k. y k Nf p 0 p... p p 0 p p p 0 p... p 6 4 n k n k. n k Nf + 3 x k x k.. x k Nf (3.54) 7 5. (3.54) he rest of Section 3.7 then directly applies with the matrix P changing to include the larger l L-tuples corresponding to L diversity channels, and the corresponding equalizer W having its L coe cients corresponding to w 0... w Nf. Each coe cient thus contains l values for each of the L equalizers. he astute reader will note that the diversity equalizer is the same in principle as a fractionally spaced equalizer except that the oversampling that creates diversity in the FSE generalizes to simply any type of additional dimensions per symbol in the diversity equalizer. he dfecolor program can be used where the oversampling factor is simply l L and the vector of the impulse response is appropriately organized to have l L phases per entry. Often, l =, so there are just L antennas or lines of samples per symbol period entry in that input vector. 4 Maximal combiners that select only one (the best) of the diversity channels for equalization are popular because they reduce the equalization complexity by at least a factor of L andperhapsmorewhenthebestsubchannelneedsless equalization. 74

117 y 0 ( t) anti-alias! filter! gain! sample! at times! k(/l)! y 0, k W z U, k Σ! ' zu,k U!n!b!i!a!s!e!d!! D!e!c!i!s!i!o!n! xˆ k y L ( t) anti-alias! filter! gain! sample! at times! k(/l)! y L, k Frac&onally+spaced!! Diversity!equalizer! SNR SNR MMSE DFE MMSE DFE, U ( ) G U D F!e!e!d!b!a!c!k!! f!i!l!t!e!r! DFE RAKE Program Figure 3.63: Fractionally spaced RAKE MMSE-DFE. A DFE RAKE program similar to the DFERAKE matlab program has again been written by the students (and instructor debugging!) over the past several years. It is listed here and is somewhat self-explanatory if the reader is already using the DFECOLOR program. %DFE design program for RAKE receiver %Prepared by: Debarag Banerjee, edited by Olutsin OLAUNBOSUN % and Yun-Hsuan Sung to add colored and spatially correlated noise %Significant Corrections by J. Cioffi and above to get correct results -- %March 005 % function [dfsesnr,w,b]=dfsecolorsnr(l,p,nff,nbb,delay,ex,noise); % %**** only computes SNR **** % l = oversampling factor % L = No. of fingers in RAKE % p = pulse response matrix, oversampled at l (size), each row corresponding to a diversity path % nff = number of feedforward taps for each RAKE finger % nbb = number of feedback taps % delay = delay of system <= nff+length of p - - nbb % if delay = -, then choose best delay % Ex = average energy of signals % noise = noise autocorrelation vector (size L x l*nff) % NOE: noise is assumed to be stationary, but may be spatially correlated % outputs: % dfsesnr = equalizer SNR, unbiased in db % siz = size(p,); L=size(p,); nu = ceil(siz/l)-; p = [p zeros(l,(nu+)*l-siz)]; % error check if nff<=0 error( number of feedforward taps must be > 0 ); 75

118 end if delay > (nff+nu--nbb) error( delay must be <= (nff+(length of p)--nbb) ); end if delay < - error( delay must be >= 0 ); end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %if length(noise) ~= L*l*nff % error( Length of noise autocorrelation vector must be L*l*nff ); %end if size(noise,) ~= l*nff size(noise,) ~= L error( Size of noise autocorrelation matrix must be L x l*nff ); end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %form ptmp = [p_0 p_... p_nu] where p_i=[p(i*l) p(i*l-)... p((i-)*l+) for m=:l ptmp((m-)*l+:m*l,) = [p(m,); zeros((l-),)]; end for k=:nu for m=:l ptmp((m-)*l+:m*l,k+) = conj((p(m,k*l+:-:(k-)*l+)) ); end end ptmp; % form matrix P, vector channel matrix P = zeros(nff*l*l+nbb,nff+nu); %First construct the P matrix as in MMSE-LE for k=:nff, P(((k-)*l*L+):(k*l*L),k:(k+nu)) = ptmp; end %Add in part needed for the feedback P(nff*l*L+:nff*l*L+nbb,delay+:delay++nbb) = eye(nbb); temp= zeros(,nff+nu); temp(delay+)=; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Rn = zeros(nff*l*l+nbb); for i = :L n_t = toeplitz(noise(i,:)); end for j = :l:l*nff for k = :l:l*nff Rn((i-)*l+(j-)*L+:i*l+(j-)*L, (i-)*l+(k-)*l+:i*l+(k-)*l) = n_t(j:j+l-,k:k+l-); end end 76

119 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Ex*P*P ; Ry = Ex*P*P + Rn; Rxy = Ex*temp*P ; IRy=inv(Ry); w_t = Rxy*IRy; %Reshape the w_t matrix into the RAKE filter bank and feedback matrices ww=reshape(w_t(:nff*l*l),l*l,nff); for m=:l W(m,:)=reshape(ww((m-)*l+:m*l,:nff),,l*nff); end b=-w_t(nff*l*l+:nff*l*l+nbb); sigma_dfse = Ex - w_t*rxy ; dfsesnr = 0*log0(Ex/sigma_dfse - ); For the previous example, some command strings that work are (with whitened-noise equivalent channel first): >> p p = >> [snr,w,b] = dferake(,p,6,,5,,[.8 zeros(,5) ;.8 zeros(,5)]) snr =.465 W = b = 0.70 or with unequal-variance noise directly inserted >> p p = >> [snr,w,b] = dferake(,p,6,,5,,[.8 zeros(,5) ;.64 zeros(,5)]) snr =.486 W = b = 0.70 wo outputs are not quite the same because of the finite number of taps Multichannel ransmission Multichannel transmission in Chapters 4 and 5 is the logical extension of diversity when many inputs may share a transmission channel. In the multichannel case, each of many inputs may a ect each of many outputs, logically extending the concept of intersymbol interference to other types of overlap than simple ISI. Interference from other transmissions is more generally called crosstalk. Di erent inputs may for instance occupy di erent frequency bands that may or may not overlap, or they may be transmitted from di erent antennas in a wireless system and so thus have di erent channels to some common receiver or set of receivers. Code division systems (See Chapter 5 Appendix) use di erent codes for the di erent sources. A diversity equalizer can be designed for some set of channel outputs for each and every of 77

120 the input sequences, leading to multichannel transmission. In e ect the set of equalizers attempts to diagonalize the channels so that no input symbol from any source interferes with any other source at the output of each of the equalizers. From an equalizer perspective, the situation is simply multiple instances of the diversity equalizer already discussed in this section. However, when the transmit signals can be optimized also, there can be considerable improvement in the performance of the set of equalizers. Chapters 4 and 5 (EE379C) develop these concepts. However, the reader may note that a system that divides the transmission band into several di erent frequency bands may indeed benefit from a reduced need for equalization within each band. Ideally, if each band is su ciently narrow to be viewed as a flat channel, no equalizer is necessary. he SNR of each of these sub-channels relates (via the gap approximation) how many bits can be transferred with QAM on each. By allocating energy intelligently, such a system can be simpler (avoiding equalization complexity) and actually perform better than equalized wider-band QAM systems. While this chapter has developed in depth the concept of equalization because there are many wide-band QAM and PAM systems that are in use and thus benefit from equalization, an intuition that for the longest time transmission engineers might have been better o never needing the equalizer and simply transmitting in separate disjoint bands is well-founded. Chapters 4 and 5 support this intuition. Progress of understanding in the field of transmission should ultimately make equalization methods obsolete. As in many technical fields, this has become a line of confrontation between those resisting change and those with vision who see a better way. 78

121 3.0 MIMO/Matrix Equalizers he extension of finite-length equalizers to MIMO was pioneered in this text and in the author s entrepreneurial endeavors. Chapters 4 and 5 detail these concepts, while later Chapters (3 and 4) use these concepts to solve many common multiple-user problems of interest. his section instead extends Chapter 3 s basic infinite-length equalization designs to MIMO systems that are for a single user he MIMO Channel Model A MIMO channel is (L y N) (L x N) in general from Chapter, but this text has viewed transmissions as sequences of -dimensional real or -dimensional-complex symbols. Coded systems (see Chapters 9-) simply use concatenated sets of these -dimensional-real and/or -dimensional-complex symbols to form codewords. hus the theory developed here in Chapter 3 remains applicable also to coded systems. Further, the complex case contains within it the real case, so one theory applied throughout this chapter. hus, the theory need no long carry the notational burden of Chapter s NL doubly-indexed basis functions. he MIMO theory here can then focus on up to L x di erent basis functions without further concern for the value of N (which will be for real baseband and for complex baseband). he two theories of equalization can separate into space-time (this Section) as well as time-frequency (previous sections and some parts of Chapter 4) as the remainder of this text will consistently show. he space-time MIMO channel (without loss of generality) can be viewed as being L y L x on each symbol, if the input is viewed as a vector sequence of L x complex-sample inputs that that then modulate the basis-function column vectors of L x L x matrix (t): 3 ' Lx,L x (t) ' Lx,L x (t)... ' Lx, ' Lx,L x (t) ' Lx,L x (t)... ' Lx, (t) = = ' Lx (t) ' Lx (t)... ' (t). (3.543) ',Lx ',Lx... ', (t) (t) s columns ' l (t) for l =,...L x are basis-vector functions (one for each of the L x components of the input vector x k, and indeed each such basis vector could have L x di erent components for each dimension, creating the L x L x matrix in (3.543).. Orthonormality of the column basis vectors may be simply ensured because they correspond to di erent (non-mutually interfering, or non crosstalking ) channel paths. Equivalently stated, the column vectors need not be orthogonal to one another in time-frequency if they are non overlapping in space. However, some channels may introduce crosstalk (which is really intrasymbol interference among the spatial dimensions that at least entered the channel free of such crosstalk), which is the spatial equivalent of intersymbol interference - just between the MIMO spatial dimensions instead of between successive symbols. here can still also be ISI between the successive overlapping vector symbols transmitted. Each transmit column vector though will have unit norm. A simple way of modeling such spatially orthogonal basis-function vectors is for (t) tobe diagonal, probably with the same unit-norm basis function '(t) along the constant diagonal. However, there is nothing that forces such diagonal structure in a more general modulator, and as later chapters show, it can be helpful to be non-diagonal if the ensuing channel is not a vector AWGN 5. With (t) havingl x normalized column basis-vector functions, the vector continuous-time transmit signal is x(t) = X (t k) x k, (3.544) k consistent with the scalar successive-transmission development earlier in Section 3. of this chapter 6. he L x -dimensional channel input simply has a component for each channel input dimension instead of 5 AWGN means white Gaussian noise on each dimension, but also of equal power and independent of all other L y dimensions for every dimension. 6 he use of a in(3.544)simplydenotesmultiplication. hisdotnotationisusedforappearanceofresultsto separate quantities that might otherwise notationally confused - it can mean matrix multiply or scalar multiply in this text, depending on context. 79

122 being a complex scalar (L x = ). he L y -dimensional channel output is y p (t) =H(t) x(t)+n p (t), (3.545) where y p (t) isl y, x(t) isl x, H(t) isl y L x, and n p (t) is an L y random-process vector of noise. Diversity in Section 3.9 is a special case with L x =butl y >. Indeed, MIMO is essentially L x independent diversity channels each of size L y. In e ect, the diversity analysis of Section 3. could just be applied to each input independently. However, the treatment here is more compact and allows some insights into the sum of the data rates of all these diversity channels as a direct calculation and as a direct interpretation of the MIMO-channel performance for a single user. his MIMO system then has matrix pulse response P (t) =H(t) (t), (3.546) and so the vector channel output (see Figure 3.65) is y p (t) = X k P (t k) x k + n(t). (3.547) A MIMO one-shot would occur when only the single vector symbol x 0 is sent. However, an L y - dimensional channel output consequentially occurs with possible crosstalk between the output dimensions. An ML MIMO symbol-by-symbol detector is not a simple slicer on each dimension unless the channel is crosstalk free (or the inputs have been carefully designed, see Chapters 4 and 5). Nonetheless, a constant (operating within the symbol only on crosstalk) MMSE equalizer of L x L y coe cients could be designed for this one-shot system. he matched-filter-bound (MFB) measures if an equalizer s performance is close to an upper bound (that may not always be attainable). his MFB concept makes more sense for MIMO if each dimension of a simple slicer were used in each output dimension (following a MIMO matched filter of size L x L y ). hus, the MFB is really a concept to be individually applied to each of these L x dimensions. hus, an L x L x diagonal matrix with the norms of individual matched-filter-output scalings on the diagonal will also be useful and is defined as k P 3 Lx k k P Lx k... 0 kp k = (3.548) k P k Each norm on the diagonal is the sum of the integrals of each of the L x columns of the pulse-response matrix P (t), so L k P X y Z l k = p i,l dt, (3.549) where P l (t) = 6 4 i= p Ly,l(t). p,l (t) l =,...,L x. (3.550) Notational convenience considers an unscaled L x L y MIMO matched-filter matrix P ( here this unnormalized matched filter output will instead be defined as: t), so instead y p (t)! X k P (t k) x k + n p (t). (3.55) he MIMO stationary noise vector has L y L y dimensional autocorrelation matrix function R n (t). Ideally, Rnn(t) = N0 I (t) so that the noise is white in all time dimensions and also uncorrelated between the di erent spatial dimensions, with the same variance N0 on all dimensions. Such a noise autocorrelation matrix function will have a matrix Fourier transform Sn(f) = N0 I L y (see Appendix 80

123 A.3). When Sn(f) satisfies the MIMO Paley-Wiener criterion of Appendix A.3 (and all practical noises will), Z ln [S n (f)] +f df <, (3.55) then a factorization of Sn(f) into a causal and causally invertible matrix filter exists (through matrix spectral factorization exists of Appendix A.3): S / n Sn(f) = N 0 S/ n (f) S / n ( f). (3.553) (f) is the causal and causally invertible filter, and here to specify uniquely one such filter, this text will take as upper triangular 7 such matrix. his Sn / (f) is the causal, MIMO, noise-whitening matrix filter, and the MIMO white-noise equivalent channel becomes P (f) =S / n (f) P (f), (3.554) which has inverse transform P (t). hen the noise-whitened channel output becomes y p (t) = X k P (t k) x k + n p (t), (3.555) where n p (t) is now the desired white noise with variance N0 he result of the matched-filter matrix operation then is in each and every (real) dimension. y(t) = P ( t) y(t). (3.556) he unnormalized L x L x ISI-crosstalk characterizing matrix function will then be defined as R p (t) = P ( t) P (t). (3.557) hen, finally the sampled channel output vector is (with R p,k R p,k = Rp, k or R p(d) =Rp(D )) = R p (k) and noting therefore that y k = X m R p,k m x m + n k. (3.558) he vector D-ransform (see Appendix A.3) is correspondingly Y (D) =R p (D) X(D)+N(D). (3.559) here is no intersymbol interference (nor crosstalk) when R p,k = k P k k or R p (D) = k P k, basically a MIMO Nyquist criterion. In this Nyquist-satisfying case, the pulse response is such that symbol-by-symbol detection is optimum, and each dimension has an he corresponding diagonal matrix is SNR MFB (l) =k P l k Ē l N 0 l =,...,L x. (3.560) hese matrix SNRs are explored further in the next subsection. SNR M4B = k P k Ēx N 0. (3.56) 7 Since the dimensions are spatial, the upper triangularity is not necessary, but it specifies a unique choice on the square-root matrix choices. Appendix A.3 considers such factorizations and separates a diagonal Cholesky-factor matrix that has its square root absorbed into the noise-whitening filter matrices of this section. 8

124 3.0. Matrix SNRs he SNR concept (for single user) MIMO/vector transmission systems remains a scalar for which the data rate (and/or bits/symbol) can be computed by the gap approximation, but is the ratio of the determinant of the transmitted symbol s (in this chapter constant diagonal) autocorrelation matrix to the determinant of the MSE matrix of Appendix A.3. Figure 3.64 expands Figure 3.4 of Section 3. to the MIMO case. x k!p ( t ) n P + ( t ) y P matched filter ( t ) y t ( )!P * t sample at times k(/l) Vector/Matrix Receiver ( ) y z k SBS k ˆx R detector k Equivalent MIMO AWGN at time k x k e k + z k SNR( R) = R xx R ee Figure 3.64: Use of possibly suboptimal MIMO receiver to approximate/create an equivalent MIMO AWGN. Chapter 3 s transmit symbol vector has equal energy per dimension Ēx in each and every dimension so that Rxx(D) =Ēx I. IfRee(D) is also constant diagonal, then (with numerator and denominator autocorrelation matrices both normalized consistently to the same number of dimensions, for real or for complex): SNR(R) = R xx(d) Ree(D) = Q Lx l= Q Ēx Lx Se,0(l) = YL x l= l= SNR l. (3.56) If Ree(D) is not diagonal, the symbol-by-symbol detector of Figure 3.64 will still independently detect each of the L x dimensions, so (3.56) has consistent practical meaning, but then the dimensional SNR s simply use the diagonal elements of Ree(D = 0) as the entries of Se,0(l), so diag {Ree(0)} = Se,0(l). As in Appendix A.3, the individual optimization problems are separable if the receiver uses a MMSE criterion, and each dimension is an equalizer with a bias that can be removed, so SNR U,l = SNR l (3.563) define the unbiased SNR s that can be obtained in the usual way with bias removal in MMSE equalization. Each SNR corresponds to one of the L x MIMO dimensions so the data rate is the sum of the individual data rates for the entire L x -dimensional symbol 8. he over all number of bits per symbol is then ( l = for real baseband and l = for complex baseband with gap = 0 db) b = XL x l= b l = XL x l= l log ( + SNR U,l ) (3.564) = l YL x log ( + SNR U,l ) (3.565) l= = l log apple R xx(d) Ree(D), (3.566) 8 Or L x real dimensions if the system is complex baseband, but consists of L x complex dimensional channels. 8

125 which also validates the utility of the ratio of determinants he MIMO MMSE DFE and Equalizers Figure 3.65 redraws Figure 3.37 for the case of MIMO filters. All vector processes are shown in boldface, while the matrix filters are also in boldface. Dimensions have been included to facilitate understanding. his section will follow the earlier infinite-length scalar DFE development, starting immediately with MMSE. ( )!P t ( ) x k x P t n P L x L y L x L y ( t ) y P ( t ) + W D L x L y ( )!P * t sam pl e at tim es k Whitened matched filter feedforward matrix ( ) L x L x z k + ' z k feedback matrix ( ) I B D (any bias removal included) L x SBS ˆx k L x B M4 DFE W M4 DFE ( ) = G( D) D ( D) = S e G * D * ( ) B M4 LE W M4 LE ( ) = I D ( D) =!P Q( D)+ SNR M4B!P R L x L x Figure 3.65: MIMO decision feedback equalization. he upper triangular feedback section eliminates intersymbol interference from previous L x -dimensional symbols, but also eliminates some crosstalk from earlier decided dimensions. Specifically, a first decision is made on dimension after all previous ISI corresponding to x k n for all n can be made by subtracting ISI from that sole dimension (since the feedback section is also upper triangular and then has only one non-zero coe cient in the bottom row). hat first dimension s current decision, plus all its previous decisions, can be used respectively to subtract crosstalk and ISI from the bottom dimension into the next dimension up. his process continues up the triangular feedback section where the top row uses the current decisions from all lower dimensions to eliminate current crosstalk along with all previous-decision dimensions crosstalk and ISI from its dimension. hus, B(D) mustbeupper triangular, causal, and monic ( s on the diagonal of B 0 ). he MIMO MMSE Decision Feedback Equalizer (M4-DFE) jointly optimizes the settings of both the infinite-length L x L y matrix-sequence feedforward filter W k and the causal infinite-length L x L x upper triangular monic matrix-sequence feedback filter k I Lx B k to minimize the MSE. Appendix A. develops the orthogonality principal for vector processes and MIMO. Definition 3.0. (Minimum Mean Square Error Vector (for M4-DFE)) he m4- DFE error signal is e k = x k z 0 k. (3.567) he MMSE for the M4-DFE is he L x vector error sequence can be written as: MMSE-DFE = min E { Ree(D) }. (3.568) W (D),B(D) E(D) =X(D) W (D) Y (D) [I B(D)] X(D) =B(D) X(D) W (D) Y (D). (3.569) For any fixed B(D), E [E(D)Y (D )] = 0 to minimize MSE, which leads to the relation B(D) Rxy(D) W (D) Ryy(D) =0. (3.570) 83

126 wo correlation matrices of interest are (assuming the input has equal energy in all dimensions, Ēx) Rxy(D) = Ēx R p (D) (3.57) Ryy(D) = Ēx R p(d)+ N 0 R p(d), (3.57) he MMSE-MIMO LE (M4-LE) matrix filter then readily becomes W M4 LE (D) = Rxy(D) Ryy (D) (3.573) apple = Ēx R p (D) Ēx Rp(D)+ N 0 R p(d) (3.574) apple = Ēx R p (D) Ēx R p (D)+ N 0 I R p(d) (3.575) apple = Ēx R p (D) Rp (D) Ēx R p (D)+ N 0 I (3.576) = [R p (D)+SNR], (3.577) where SNR = Ēx N0 I. he corresponding MMSE MIMO (M4-LE) matrix is (with B(D) =I) S M4 LE = min E Re M4 LE e M4 LE (D) (3.578) W (D) M4 LE = = Rxx(D) Rxy(D) Ryy(D) Ryx(D) (3.579) apple = Ēx I Ēx R p (D) Ēx Rp(D)+ N 0 R p R p (D)Ēx (3.580) = Ēx I Rp (D)+SNR Rp (D) Ēx (3.58) n = Ēx I Rp (D)+SNR Rp (D)o (3.58) = Ēx R p (D)+SNR Rp (D)+SNR R p (D) (3.583) = N 0 R p (D)+SNR (3.584) N 0 R p (D)+SNR. (3.585) A form similar to the scalar MMSE-LE version would define Q(D) through R p (D) =k P k Q(D) k P k, and then W M4 LE (D) =k P k Q(D)+SNR M4B k P k, (3.586) where the trailing factor k P k could be absorbed into the preceding matrix-matched filter to normalize it in Figure 3.65, and thus this development would exactly then parallel the scalar result (noting that multiplication by a non-constant diagonal matrix does not in general commute). Similarly, S M4 LE = N 0 k P k Q(D)+SNR M4B k P k M4 LE = (3.587) N 0 k P k Q(D)+SNR. (3.588) By linearity of MMSE estimates, then the FF filter matrix for any B(D) becomes W (D) = B(D) Rxy(D) Ryy (D) (3.589) = B(D) [R p (D)+SNR] = B(D) k P k Q(D)+SNR M4B k P k 84 (3.590), (3.59)

127 for any upper triangular, monic, and causal B(D). Again, since the MMSE estimate is linear then E(D) =B(D) E M4-LE (D). (3.59) he autocorrelation function for the error sequence with arbitrary monic B(D) is Ree(D) =B(D) Re M4 LE e M4 LE (D) B (D ), (3.593) which has a form very similar to the scalar quantity. In the scalar case, a canonical factorization of the scalar Q(D) was used and was proportional to the (inverse of the) autocorrelation function of the MMSE of the LE, but not exactly equal. Here because matrices do not usually commute, the MIMO canonical factorization will be directly of Re M4 LE e M4 LE (D), or via (3.587) of N0 k P k Q(D)+SNR M4B k P k, so Re M4 LE e M4 LE (D) = G(D) S e G (D ) or (3.594) Q(D)+SNR M4B = k P k G(D) S e G (D )k P k, (3.595) where G(D) is monic (diag {G(0)} = I), upper triangular, causal, and minimum-phase (causally invertible); S e is diagonal with all positive elements. Substitution of the inverse factorization into (3.593) produces Ree(D) = N 0 B(D) G(D) S e G (D ) B (D ). (3.596) he determinant Ree(D) is the product of ( N0 with the determinants of the 5 matrices above, )Lx each of which is the product of the elements on the diagonal. Since B(D) and G(D) are both upper triangular, their product is also upper triangular (and they are both monic, which means that diagonal with D = 0 is all ones), and that product F = B(D) G(D) has a causal monic form I +F D+F D +... he product F (D)S e F (D ) will have a weighted sum of positive terms with I as one of them (time zero-o set term) and will have all other terms as positive weights on Fk and thus is minimized when when F k =08k>0, so B(D) =G(D) is the MMSE solution. Further the minimum value is Ree(D) =S Ree(D) = S e (3.597) M4 DFE = Se = Se,0. (3.598) Lemma 3.0. (MMSE-DFE) he MMSE-DFE has feedforward matrix filter W (D) =W (D) =S e G (D ) (3.599) (realized with delay, as it is strictly noncausal) and feedback section B(D) =G(D) (3.600) where G(D) is the unique canonical factor of the following equation: N 0 k P k Q(D)+SNR M4B k P k = G(D) S e G (D ). (3.60) his text also calls the joint matched-filter/sampler/w (D) combination in the forward path of the DFE the MIMO Mean-Square Whitened Matched Filter (M4-WMF). hese settings for the M4-DFE minimize the MSE as was shown above in (3.598). he matrix signal to noise ratio is SNR M4 DFE = R xx Se (Ēx) Lx = Q Lx l= S e(l) = 85 e. (3.60) (3.603) YL x SNR M4 DFE (l) (3.604) l=

128 and correspondingly each such individual SNR is biased and can be written as while also SNR M4 DFE (l) = SNR M4 DFE,U (l)+, (3.605) SNR M4 DFE = SNR M4 DFE,U + I. (3.606) he overall bits per symbol (with zero-db gap) is written as (with l = when the signals are real baseband, and l = when complex baseband) ( ) b = l Rxx log S (3.607) and with non-zero gap, b = l XL x log [ + SNR M4 DFE,U (l)] (3.608) l= l= b = l XL x apple log + SNR M4 DFE,U(l). (3.609) o normalize to the number of dimensions, b = b L x. he process of current symbol previous-order crosstalk subtraction can be explicitly written by defining the ISI-free (but not current crosstalk-free) output z 0 k = z k X l= G l ˆx k l. (3.60) Any coe cient matrix of G(D) can be written G k with its (m, n) th element counting up and to the left from the bottom right corner where m is the row index and n is the column index, and m =0,...,L x and also n =0,...,L x. hen, the current decisions can be written as (with sbs denoting simple one- (or two- if complex) dimensional slicing detection): MIMO Precoding ˆx 0,k = sbs 0 z 0 0,k (3.6) ˆx,k = sbs z 0,k G 0 (, 0) ˆx 0,k (3.6) ˆx,k = sbs z 0,k G 0 (, ) ˆx,k G 0 (, 0) ˆx 0,k (3.63). =. (3.64)! LX x ˆx Lx,k = sbs Lx zl 0 x,k G 0 (L x,l) ˆx l,k. (3.65) omlinson Precoders follow exactly as they did in the scalar case with the sbs above in Equations (3.6) - (3.65) being replaced by the modulo element in the transmitter. he signal constellation on each dimension may vary so the modulo device corresponding to that dimension corresponds to the signal constellation used. Similar Laroia/Flexible precoders also operate in the same way with each spatial dimension have the correct constellation. o form these systems, an unbiased feedback matrix filter is necessary, which is found by computing the new unbiased feedback matrix filter G U (D) =[SNR M4 DFE I] l=0 SNR M4 DFE G(D) SNR M4 DFE. (3.66) G U (D) may then replace G(D) in feedback section with the feedforward matrix-filter output in Figure 3.65 scaled up by [SNR M4 DFE I] SNR M4 DFE. However, the precoder will move the feedback section using G U (D) to the transmitter in the same way as the scalar case, just for each dimension. 86

129 3.0.5 MIMO Zero-Forcing he MIMO-ZF equalizers can be found from the M4 Equalizers by letting SNR!in all formulas and figures so far. However, this may lead to infinite noise enhancement for the MIMO-ZFE or to a unfactorable Ree(D) in the ZF-DFE cases. When these events occur, ZF solutions do not exist as they correspond to channel and/or input singularity. Singularity is otherwise handed more carefully in Chapter 5. In e ect the MIMO Channel must be factorizable for these solutions to exist, or equivalently the factorization P k Q(D) k P k = P c (D) S c P c(d ), (3.67) exists. his means basically that Q(D) satisfies the MIMO Paley-Weiner Criterion in Appendix A.3. 87

130 3. Adaptive Equalization In practice, digital receivers that use equalizers are usually adaptive. An adaptive equalizer automatically adjusts its internal settings (the values for w in Section 3.7) so as to realize the MMSE settings as closely as possible. he adaptive equalizer performs this self optimization by measuring its own performance and then recursively in time incrementing its tap settings so as to improve performance on the average. It is presumed that the designer does not know the channel pulse response, so that a good equalizer cannot be predesigned, and that the channel response may be subject to variation with time or from use to use. here are several adaptive algorithms in use in sophisticated equalization systems today. By far, the most commonly used is (a variant of) the Widrow-Ho (960) LMS algorithm, which is an example of the engineering use of what is known in statistics as a stochastic-gradient algorithm, see Robins-Munro (95). For numerical reasons, this algorithm is altered slightly in practice. Other algorithms in use are Lucky s Zero-Forcing algorithm (966), Recursive Least-Squares methods (Godard), and frequencydomain algorithms. 3.. he LMS algorithm for Adaptive Equalization he Least Mean Square (LMS) algorithm is used with both w and b in Section 3.7 as slowly time-varying FIR filters. A matched filter is usually not used (only a front-end anti-alias filter), and the adaptive equalizer is usually realized as a fractionally spaced equalizer, but is also sometimes implemented in the suboptimal configuration of no matched filter and symbol-rate sampling. he LMS algorithm attempts to minimize the MSE by incrementing the equalizer parameters in the direction of the negative gradient of the MSE with respect to w (and also b for the DFE). his direction is the direction of steepest descent (or most reduction in MSE). Recall the MSE is E e k, so the idea is to update the filter parameters w by w k+ = w k µ r w k E e k, (3.68) where r w k E e k is the gradient of the MSE with respect to the filter coe cient vector w k. In practice, the channel statistics are not (exactly) known so that the gradient term,r w k E e k, in (3.68) is unknown. It is, instead, estimated by the stochastic gradient r w k e k, which is easily computed as r w k e k = e ky k (3.69) where y k is (again) the vector of inputs to the FIR filter(s) in the adaptive equalizer. he LMS algorithm then becomes e k = x k w ky k (3.60) w k+ = w k + µe ky k. (3.6) here are a variety of proofs, under various mild assumptions and approximations, that the LMS algorithm converges to the MMSE settings if µ is su ciently small and numerical precision is adequate (see Chapter 4). hus the exact channel pulse response and noise statistics need not be known in advance. Usually, the input sequence x k is known to the receiver for a short training time in advance of unknown data transmission, so that (3.60) can be computed in the receiver, and updating allowed. After training for a time period su cient for the adaptive equalizer to converge, the known data sequence must be replaced by the decisions in the receiver for adaptation to continue. his type of adaptive equalization is called decision directed. For stable operation of the LMS adaptive algorithm, 0 apple µ apple N f Ey + N b Ex. (3.6) ypically, the upper limit is avoided and the step-size µ is usually several orders of magnitude below the upper stability limit, to ensure good noise averaging and performance close to the MMSE level. 88

131 In practice, the LMS update must be implemented in finite-precision, and a variant of it, known as the ap-leakage Algorithm, or as Leaky LMS, must be used: w k+ = w k + µe ky k (3.63) where is close to, but slightly less than unity. his is equivalent to minimizing (via stochastic gradient) the sum of the MSE and ( )k w k k, thus preventing an arbitrary growth of w caused by accumulated round-o error in practical implementation. Zero-Forcing Algorithm A lesser used, but simpler, adaptive algorithm is the zero-forcing algorithm, which produces the zero forcing settings for the adaptive equalizer. his algorithm is derived by noting that the equalizer error, and the input data vector should be uncorrelated when the equalizer has the zero-forcing setting. his algorithm is written w k+ = w k + µe kx k (3.64) where x k is a vector of training (or decision-directed-estimated) data symbol values. Since the symbol values are often integer valued, the multiply in (3.64) is easily implemented with addition and possible minor shifting, thus making the zero-forcing algorithm somewhat simpler than the LMS. 9 In practice, this simplification can be significant at very high-speeds, where multiplication can be di cult to implement cost e ectively. Signed LMS Algorithm Perhaps, more common than the zero-forcing algorithm is the signed LMS algorithm w k+ = w k + µ e k sgn [y k ] (3.65) where the sgn function eliminates the need for multiplication. his algorithm also converges to the MMSE solution in practice, although usually more slowly than the LMS. 3.. Vector-MIMO Generalization of LMS When the error is a vector, LMS generalizes in the obvious way to (with W =[W B] e k = x k W ky k (3.66) W k+ = W k + µe ky k. (3.67) Di erent values of µ may be used for di erent dimensions (as well as for feedback and feedforward components within W. he ZF and Signed LMS both also generalize in exactly the same way for the MIMO case. 9 he step-size µ is usually implemented as a power of, and thus also corresponds only to minor shifting. 89

132 Exercises - Chapter 3 3. Single Sideband (SSB) Data ransmission with no ISI - 8 pts Consider the quadrature modulator shown in Figure 3.66, H ( f ) cos( ω c t) x( t) +" s( t) H ( f ) sin( ω c t) Figure 3.66: Quadrature modulator. a. What conditions must be imposed on H (f) and H (f) for the output signal s(t) tohaveno spectral components between f c and f c? Assume that f c is larger than the bandwidth of H (f) and H (f). ( pts.) b. With the input signal in the form, x(t) = X n= a n (t n ), what conditions must be imposed on H (f) and H (f) if the real part of the demodulated signal is to have no ISI. (3 pts.) c. Find the impulse responses h (t) and h (t) corresponding to the minimum bandwidth H (f) and H (f) that simultaneously satisfy Parts a and b. he answer can be in terms of (t). (3 pts.) 3. Sampling ime and Eye Patterns - 9 pts he received signal for a binary transmission system is, X y(t) = a n q(t n )+n(t) n= where a n {, +} and Q(f) is a triangular, i.e. q(t) =sinc ( t ). he received signal is sampled at t = k + t 0, where k is an integer and t 0 is the sampling phase, t 0 <. a. Neglecting the noise for the moment, find the peak distortion as a function of t 0. Hint: Use Parseval s relation. (3 pts.) b. For the following four binary sequences {u n }, {v n }, {w n }, {x n } : u n = 8n, v n = + 8n, +, for n =0 w n =, otherwise, and, for n =0 x n = +, otherwise, use the result of (a) to find expressions for the 4 outlines of the corresponding binary eye pattern. Sketch the eye pattern for these four sequences over two symbol periods apple t apple.(pts.) 90

133 c. Find the eye pattern s width (horizontal) at its widest opening. ( pts.) d. If the noise variance is, find a worst case bound (using peak distortion) on the probability of error as a function of t 0.(pts.) 3.3 he Stanford EE379 Channel Model - pts For the ISI-model shown in Figure 3.9 with PAM and symbol period, '(t) = p sinc( t ) and h(t) = (t) (t ): a. Determine p(t), the pulse response. ( pt) b. Find p and ' p (t). ( pts.) c. ( pts.) Find q(t), the function that characterizes how one symbol interferes with other symbols. Confirm that q(0) = and that q(t) is hermitian (conjugate symmetric). A useful observation is t + m t + n t +(m + n) sinc sinc = sinc. d. Use Matlab to plot q(t). his plot may use = 0 and show q(t) for integer values of t. With y(t) sampled at t = k for some integer k, how many symbols are distorted by a given symbol? How will a given positive symbol a ect the distorted symbols? Specifically, will they be increased or decreased? ( pts.) e. he remainder of this problem considers 8 PAM with d =. Determine the peak distortion, D p. ( pts.) f. Determine the MSE distortion, D mse and compare with the peak distortion. ( pts.) g. Find an approximation to the probability of error (using the MSE distortion). he answer will be in terms of.( pt.) 3.4 Bias and SNR - 8 pts his problem uses the results and specifications of Problem 3.3 with 8 PAM and d = and sets N 0 = =0.. he detector first scales the sampled output y k by and chooses the closest x to y k as illustrated in Figure 3.5. Please express your SNR s below both as a ratio of powers and in db. a. For which value of is the receiver unbiased? ( pt.) b. For the value of found in (a), find the receiver signal-to-noise ratio, SNR(R). ( pts.) c. Find the value of that maximizes SNR R and the corresponding SNR(R). ( pts.) d. Show that the receiver found in the previous part is biased. ( pt.) e. Find the matched filter bound on signal-to-noise ratio, SNR MFB.(pt.) f. Discuss the ordering of the three SNR s you have found in this problem. Which inequalities will always be true? ( pt.) 3.5 Raised Cosine Pulses with Matlab - 8 pts his problem uses the.m files from the instructors EE379A web page at /ee379a/homework.html. 9

134 a. he formula for the raised cosine pulse is in Equation (3.79) and repeated here as: " # t cos t q(t) =sinc t here are three values of t that would cause a program, such as Matlab, di culty because of a division by zero. Identify these trouble spots and evaluate what q(t) should be for these values. ( pts) b. Having identified those three trouble spots, a function to generate raised-cosine pulses is a straightforward implementation of the above equation and is in mk rcpulse.m from the web site. Executing q=mkrcpulse(a) will generate a raised-cosine pulse with = a. he pulses generated by mk rcpulse assume = 0 and are truncated to 75 points. ( pts) Use mk rcpulse to generate raised cosine pulses with 0%, 50%, and 00% excess bandwith and plot them with plt pls lin. he syntax is plt pls lin(q 50, title ) where q 50 is the vector of pulse samples generated by mk rcpulse(0.5) (50% excess bandwidth). Show results and discuss any deviations from the ideal expected frequency-domain behavior. c. Use plt pls db(q, title ) to plot the same three raised cosine pulses as in part (b). Show the results. Which of these three pulses would you provides the smallest band of frequencies with energy above -40dB? Explain this unexpected result. (3 pts) d. he function plt qk (q, title ) plots q(k) for sampling at the optimal time and for sampling that is o by 4 samples (i.e., q(k) and q(k + 4) where = 0). Use plt qk to plot q k for 0%, 50%, and 00% excess bandwidth raised cosine pulses. Discuss how excess bandwidth a ects sensitivity of ISI performance to sampling at the correct instant. (3 pts) 3.6 Noise Enhancement: MMSE-LE vs ZFE - 3 pts his problem explores the channel with kpk =+aa =+ a Q(D) = a D 0 apple a <. +kpk +ad kpk a. ( pts) Find the zero forcing and minimum mean square error linear equalizers W ZFE (D) and W MMSE-LE (D). Use the variable b = kpk + SNR in your expression for W MFB MMSE-LE (D). b. (6 pts) By substituting e jw = D (with = ) and using SNR MFB = 0 kpk, use Matlab to plot (lots of samples of) W (e jw ) for both ZFE and MMSE-LE for a =.5 and a =.9. Discuss the di erences between the plots. c. (3 pts) Find the roots r,r of the polynomial ad + bd + a. Show that b 4 a is always a real positive number (for a 6= ). Hint: Consider the case where SNR MFB = 0. Let r be the root for which r < r. Show that r r =. d. ( pts) Use the previous results to show that for the MMSE-LE W (D) = kpk a e. ( pts) Show that for the MMSE-LE, w 0 = ZFE, w 0 = kpk a. D (D r )(D r ) = kpk a(r r ) r D r r D r kpk p b 4 a. By taking SNR MFB 9. (3.68) = 0, show that for the

135 f. (4 pts) For Ēx = and =0. find expressions for ZFE, mlse, ZFE, and MMSE-LE. g. (4 pts) Find ZFE and MMSE-LE in terms of the parameter a and calculate for a =0, 0.5,. Sketch ZFE and MMSE-LE for 0 apple a<. 3.7 DFE is Even Better - 8 pts a. ( pts) For the channel of Problem 3.6, show that the canonical factorization is Q(D)+ SNR MFB = 0 ( r D )( r D). What is 0 in terms of a and b? Please don t do this from scratch; use results of Problem 3.6. b. ( pts) Find B(D) and W (D) for the MMSE DFE. c. (4 pts) Give an expression for MMSE-DFE. Compute its values for a =0,.5, for the Ēx and of problem 3.6. Sketch MMSE-DFE as in problem 3.6. Compare with your sketches from Problem Noise Predictive DFE - pts y p ( t) * ( ) ϕ p t Y ( D ) U( D) Z ( D) +! Z' ( D) +!! +! B ( D) +!! Figure 3.67: Noise-Predictive DFE With the equalizer shown in Figure 3.8 with B(D) restricted to be causal and monic and all decisions presumed correct, use a MMSE criterion to choose U(D) and B(D) tominimizee[ x k z 0 k ]. (x k is the channel input.) a. (5 pts) Show this equalizer is equivalent to a MMSE-DFE and find U(D) and B(D) in terms of G(D), Q(D), kpk, Ēx, SNR MFB and 0. b. ( pts) Relate U NPDFE (D) tow MMSE-LE (D) and to W MMSE-DFE (D). c. ( pt) Without feedback (B(D) = ), what does this equalizer become? d. ( pt) Interpret the name noise-predictive DFE by explaining what the feedback section is doing. e. ( pts) Is the NPDFE biased? If so, show how to remove the bias. 3.9 Receiver SNR Relationships - pts 93

136 a. (3 pts) Recall that: Show that: Q(D)+ SNR MFB = 0 G(D)G (D ) + SNR MFB = 0 kgk b. (3 pts) Show that: kgk with equality if and only if Q(D) = ( i.e. if the channel is flat) and therefore: 0 apple + SNR MFB c. (3 pts) Let x 0 denote the time-zero value of the sequence X. Show that: Q(D)+ 0 SNR MFB and therefore: SNR MMSE-LE,U apple SNR MMSE-DFE,U (with equality if and only if Q(D) =, that is Q(!) has vestigial symmetry.) d. ( pts) Use the results of parts b) and c) to show: SNR MMSE-LE,U apple SNR DFE,U apple SNR MFB 3.0 Bias and Probability of Error - 7 pts his problem illustrates that the best unbiased receiver has a lower P e than the (biased) MMSE receiver using the 3-point PAM constellation in Figure !! 0!! Figure 3.68: A three point PAM constellation. he AWGN n k has N 0 = =0.. he inputs are independent and identically distributed uniformly over the three possible values. Also, n k has zero mean and is independent of x k. a. ( pt) Find the mean square error between y k = x k + n k and x k. (easy) b. ( pt) Find the exact P e for the ML detector of the unbiased values y k. 94

137 c. ( pts) Find the scale factor that will minimize the mean square error E[e k]=e[(x k y k ) ]. Prove that this does in fact provide a minimum by taking the appropriate second derivative. d. ( pts) Find E[e k ] and P e for the scaled output y k. When computing P e, use the decision regions for the unbiased ML detector of Part b. he biased receiver of part (c) has a P e that is higher than the (unbiased) ML detector of Part b, even though it has a smaller squared error. e. ( pt) Where are the optimal decision boundaries for detecting x k from y k (for the MMSE ) in Part c? What is the probability of error for these decision boundaries? 3. Bias and the DFE - 4 pts For the DFE of Problem 3.7, the canonical factorization s scale factor was found as: 0 = b + p b 4 a ( + a ). a. ( pts) Find G u in terms of a, b, and r. Use the same SNR MFB as in Prob 3.7. b. ( pt) Draw a DFE block diagram with scaling after the feedback summation. c. ( pt) Draw a DFE block diagram with scaling before the feedback summation. 3. Zero-Forcing DFE - 7 pts his problem again uses the general channel model of Problems 3.6 and 3.7 with Ēx = and =0.. a. ( pts) Find 0 and P c (D) so that b. ( pts) Find B(D) and W (D) for the ZF-DFE. c. ( pt) Find ZF DFE Q(D) = 0 P c (D)P c (D ). d. ( pts) Find the loss with respect to SNR MFB for a = 0,.5,. Sketch the loss for 0 apple a<. 3.3 Complex Baseband Channel - 6 pts he pulse response of a channel is band-limited to and has p 0 = p = p ( +.j) p ( j) where p k = p(k) and SNR MFB = 5 db. a. ( pts) Find kpk and a so that p k = 0 for k 6= 0, Q(D) = a D + kpk + ad kpk b. ( pts) Find SNR MMSE LE,U and SNR MMSE DFE,U for this channel. Use the results of Problems 3.6 and 3.7 wherever appropriate. Since kpk 6=+ a, some care should be exercised in using the results of Problems 3.6 and

138 c. ( pts) Use the SNR s from Part b to compute the NNUB on P e for 4-QAM with the MMSE-LE and the MMSE-DFE. 3.4 omlinson Precoding - 3 pts his problem uses a + 0.9D channel (i.e. the + ad channel that we have been exploring at length with a =0.9) with = 0 and with the 4-PAM constellation having x k { 3,,, 3}. a. (5 pts) Design a omlinson precoder and its associated receiver for this system. his system will be fairly simple. How would your design change if noise were present in the system? b. (5 points) Implement the precoder and its associated receiver using any method you would like with no noise for the following input sequence: {3, 3,,, 3, 3, } Also compute the corresponding output of the receiver, assuming that the symbol x 0 = sent just prior to the input sequence. 3 was c. (3 pts) Again with zero noise, remove the modulo operations from the precoder and the receiver. For this modified system, compute the output of the precoder, the output of the channel, and the output of your receiver for the same inputs as in the previous part. Did the system still work? What changed? What purpose do the modulo operators serve? 3.5 Flexible Precoding - 3 pts his problem uses a + 0.9D channel (i.e. the + ad channel that we have been exploring at length with a =0.9) with = 0 and with the 4-PAM constellation having x k { 3,,, 3}. a. (5 pts) Design a Flexible precoder and its associated receiver for this system. his system will be fairly simple. How would your design change if noise were present in the system? b. (5 points) Implement the precoder and its associated receiver using any method you would like with no noise for the following input sequence: {3, 3,,, 3, 3, } Also compute the corresponding output of the receiver, assuming that the symbol x 0 = sent just prior to the input sequence. 3 was c. (3 pts) Again with zero noise, remove the modulo operations from the precoder and the receiver. For this modified system, compute the output of the precoder, the output of the channel, and the output of your receiver for the same inputs as in the previous part. Did the system still work? What changed? What purpose do the modulo operators serve? 3.6 Finite-Length Equalization and Matched Filtering - 5 pts his problem explores the optimal FIR MMSE-LE without assuming an explicit matched filter. However, there will be a matched filter anyway as a result pf this problem. his problem uses a system whose pulse response is band limited to! < that is sampled at the symbol rate after passing through an anti-alias filter with gain p. a. (3 pts) Show that where w = R xy R YY =(0,...,0,, 0,...,0) p kpk ˆQ + l I. SNR MFB ˆQ = PP kpk 96

139 b. ( pts) Which terms in Part a correspond to the matched filter? Which terms correspond to the infinite length W MMSE LE? 3.7 Finite-Length Equalization and MALAB - 5 pts he + 0.5D channel with =. has an input with Ēx =. a. ( pt) Find SNR MMSE LE,U for the infinite-length filter. b. ( pts) his problem uses the MALAB program mmsele. his program is interactive, so just type mmsele and answer the questions. When it asks for the pulse response, you can type [ 0.5] or p if you have defined p = [ 0.5]. Use mmsele to find the best and the associated SNR MMSE LE,U for a 5-tap linear equalizer. Compare with the value from Part a. How sensitive is performance to for this system? c. ( pts) Plot P (e (jw) and P (e (jw) W (e (jw) for w [0, ]. Discuss the plots briefly. 3.8 Computing Finite-Length equalizers - pts his problem uses the following system description: Ē x = N 0 = 8 (t) = p sinc( t ) h(t) = (t) 0.5 (t ) l = Matlab may assist any matrix manipulations in completing this problem. Answers may be validated with dfecolor.m. However, this problem s objective is to explore the calculations in detail. a. ( pts) he anti-alias filter is perfect (flat) with gain p. Find p(t) =( (t) h(t)) and k pk, corresponding to the discrete-time channel Also, find k P (D)k = P k p k. b. ( pt) Compute SNR MFB for this channel. c. ( pts) Design a 3 tap FIR MMSE-LE for = 0. d. ( pt) Find the LE for the equalizer of the previous part. e. ( pts) Design a 3 tap FIR ZF-LE for = 0. f. ( pt) Find the associated ZFE. y k = x k.5 x k + n k. (3.69) g. ( pts) Design an MMSE-DFE which has feedforward taps and feedback tap. Again, assume that = 0. h. ( pts) Compute the unbiased SNR s for the MMSE-LE and MMSE-DFE. Compare these two SNR s with each other and the SNR MFB. 3.9 Equalizer for wo-band Channel with Notch - 0 pts n!(!t!)! s!a!m!p!l!e!!a!t!!k!! x k p!(!t!)! +!!.!5! i!d!e!a!l!!a!n!t!i!-!a!l!i!a!s!! f!i!l!t!e!r! y k Figure 3.69: Channel for Problem

140 Figure 3.69 has AWGN channel with pulse response p(t) = p sinc t sinc t 4. he receiver anti-alias filter has gain p over the entire Nyquist frequency band, / <f</, and zero outside this band. he filter is followed by a / rate sampler so that the sampled output has D-ransform y(d) =( D 4 ) x(d)+n(d). (3.630) x(d) isthed-ransform of the sequence of M-ary PAM input symbols, and n(d) isthed-ransform of the Gaussian noise sample sequence. he noise autocorrelation function is R nn (D) = N0. Further equalization of y(d) is in discrete time where (in this case) the matched filter and equalizer discrete responses (i.e., D-ransforms) can be combined into a single discrete-time response. he target P e is 0 3. Let N0 = -00 dbm/hz (0 dbm = milliwatt =.00 Watt) and let / = MHz. a. Sketch P (e! ).(pts) b. What kind of equalizer should be used on this channel? ( pts) c. For a ZF-DFE, find the transmit symbol mean-square value (i.e., the transmit energy) necessary to achieve a data rate of 6 Mbps using PAM and assuming a probability of symbol error equal to 0 3.(4pts) d. For the transmit energy in Part c, how would a MMSE-DFE perform on this channel? ( pts) 3.0 FIR Equalizer Design - 4 pts A symbol-spaced FIR equalizer (l = ) is applied to a stationary sequence y k at the sampled output of an anti-alias filter, which produces a discrete-time IIR channel with response given by y k = a y k + b x k + n k, (3.63) where n k is white Gaussian noise. he SNR (ratio of mean square x to mean square n) is Ēx N0 = 0 db. a <, and both a and b are real. For all parts of this question, choose the best where appropriate. a. Design a -tap FIR ZFE. (3 pts) b. Compute the SNR zfe for part a. ( pts) c. Compare the answer in Part b with that of the infinite-length ZFE. ( pt) d. Let a =.9 and b =. Find the -tap FIR MMSE-LE. (4 pts) e. Find the SNR MMSE LE,U for Part d. ( pts) f. Find the SNR ZFE for an infinite-length ZF-DFE. How does the SNR compare to the matched-filter bound if there is no information loss incurred in the symbol-spaced anti-alias filter? Use a =.9 and b =. ( pts) 3. ISI quantification - 8 pts For the channel P (!) = p ( +.9 e! ) 8! < / studied repeatedly in this chapter, use binary PAM with Ēx = and SNR MFB = 0 db. Remember that q 0 =. a. Find the peak distortion, D p.(pt) b. Find the peak-distortion bound on P e.(pts) c. Find the mean-square distortion, D MS.(pt) d. Approximate P e using the D MS of Part c. ( pts) 98

141 e. ZFE: Compare P e in part d with the P e for the ZFE. Compute SNR di erence in db between the SNR based on mean-square distortion implied in Parts c and d and SNR ZFE. (hint, see example in this chapter for SNR ZFE )(pts) 3. Precoding - pts For an ISI channel with and Ēx N0 = 00. Q(D)+ a. Find SNR MFB, kp, and SNR MMSE DFE.(pts) h =.8 SNR MFB D +.5 b. Find G(D) and G U (D) for the MMSE-DFE. ( pt) i D, c. Draw a omlinson-harashima precoder, showing from x k through the decision device in the receiver in your diagram (any M). ( pts) d. Let M = 4 in the precoder of Part c. Find P e.(pts) e. Design (show/draw) a Flexible precoder, showing from x k through the decision device in the receiver in your diagram (any M). ( pts) f. Let M = 4 the precoder of Part e. Find P e.(pts) 3.3 Finite-Delay ree Search - 7 pts A channel with multipath fading has one reflecting path with gain (voltage) 90% of the main path with binary input. he relative delay on this path is approximately seconds, but the carrier sees a phase-shift of 60 o that is constant on the second path. Use the model p +ae!! < / P (!) = 0 elsewhere to approximate this channel with N0 =.08 and Ēx =. a. Find a. ( pt) b. Find SNR MFB.(pt) c. Find W (D) and SNR MMSE LE,U for the MMSE-LE. (3 pts) d. Find W (D), B(D), and SNR MMSE DFE,U for the MMSE-DFE. (3 pts) e. Design a simple method to compute SNR ZF-DFE ( pts). f. A finite-delay tree search detector at time k decides ˆx k is shown in Figure 3.70 and chooses ˆx k to minimize min z k ˆx k,ˆx k ˆx k aˆx k + z k ˆx k aˆx k, where ˆx k = x k by assumption. 99

142 xk p x p, k ϕ p ( t) n p ( t) x p ( t) y p ( t) * +! ( ) ϕ p t W (D) s!a!m!p!l!e!! a!t!!t!i!m!e!s!! k!! feedforward! filter!of!dfe! z k D!e!c!i!s!i!o!n!! (!F!D!!S!)! D! z k D! ˆ x k Whitened!matched!filter! ˆ x k Figure 3.70: Finite-delay tree search. How does this FDS compare with the ZF-DFE (better or worse)? ( pt) g. Find an SNR fdts for this channel. (3 pts) h. Could you generalize SNR in part g for FIR channels with taps? (3 pts) 3.4 Peak Distortion - 5 pts Peak Distortion can be generalized to channels without reciever matched filtering (that is ' p ( a lowpass anti-alias filter). A channel has t) is y(d) =x(d) (.5D)+N(D) (3.63) after sampling, where N(D) is discrete AWGN. P (D) =.5D. a. Write y k in terms of x k, x k and n k. (hint, this is easy. pt) b. he peak distortion for such a channel is D p = x max Pm6=0 p m. FindthisD p for this channel if x max =. ( pts) c. Suppose N0, the mean-square of the sample noise, is.05 and Ēx =. What is P e for symbol-bysymbol detection on y k with PAM and M =? ( pts) 3.5 Equalization of Phase Distortion - 3 pts An ISI channel has p(t) =/ p [sinc(t/ ) sinc[(t )/ ]], Ēx =, and N0 =.05. a. What is SNR MFB?(pt) b. Find the ZFE. ( pts) c. Find the MMSE-LE. ( pts) d. Find the ZF-DFE. ( pts) e. Find the MMSE-DFE. ( pts) f. Draw a diagram illustrating the MMSE-DFE implementation with omlinson precoding. ( pts) g. For P e < 0 6 and using square or cross QAM, choose a design and find the largest data rate you can transmit using one of the equalizers above. ( pts) 3.6 Basic QAM Systems Revisted with Equalization Background - 6 pts An AWGN channel has Ēx = 40 dbm/hz and N0 = 66 dbm/hz. he symbol rate is initially fixed at 5 MHz, and the desired P e = 0 6. Only QAM CR or SQ constellations may be used. 300

143 a. What is the maximum data rate? ( pt) b. What is the maximum data rate if the desired P e < 0 7?(pt) c. What is the maximum data rate if the symbol rate can be varied (but transmit power is fixed at the same level as was used with a 5 MHz symbol rate)? ( pt) d. Suppose the pulse response of the channel changes from p(t) = p sinc t to p sinc t +.9 p sinc t+, and Question a is revisited. With respect to your answer in Question a, compare the data rate for a ZF-DFE applied to this channel where / = 5 MHz. ( pt) e. Under the exact same change of Question d, compare the data rate with respect to your answer in Question c. ( pt) f. Suppose an optimum ML detector is used (observes all transmissions and makes one large decision on the entire transmitted message) with any pulse response that is a one-to-one transformation on all possible QAM inputs with kpk =. Compare the probability of error with respect to that of the correct response to Question a. ( pt) 3.7 Expert Understanding - 0 pts H ( f ) f Figure 3.7: Channel for Problem 3.7. A channel baseband transfer function for PAM transmission is shown in Figure 3.7. PAM transmission with sinc basis function is used on this channel with a transmit power level of 0 mw =E x /.he one-sided AWGN psd is -00 dbm/hz. a. Let the symbol rate be 0 MHz - find SNR MFB.( pt) b. For the same symbol rate as Part a, what is SNR MMSE LE?(pt) c. For the equalizer in Part b, what is the data rate for PAM if P e apple 0 6?(pt) d. Let the symbol rate be 40 MHz - find the new SNR MFB.( pts) e. Draw a the combined shape of the matched-filter and feedforward filter for a ZF-DFE corresponding to the new symbol rate of Part d. ( pt) 30

144 f. Estimate SNR MMSE DFE,U for the new symbol rate, assuming a MMSE-DFE receiver is used. ( pts) (Hint: ythe relationship of this transfer function to the channel is often used as an example in Chapter 3). g. What is the new data rate for this system at the same probability of error as Part c - compare with the data rate of part c. ( pt) h. What can you conclude about incurring ISI if the transmitter is allowed to vary its bandwidth? ( pts) 3.8 Infinite-Length EQ - 0 pts An ISI channel with PAM transmission has channel correlation function given by Q(D) =.9 ( +.9D)( +.9D ), with SNR MFB =0 db, Ex =, and kpk =.9. a. (3 pts) Find W ZFE (D), ZFE, and SNR ZFE. b. ( pt) Find W MMSE LE (D) c. (3 pts) Find W ZF DFE (D), B ZF DFE (D), and SNR ZF DFE. d. (3 pts) Find W MMSE DFE (D), B MMSE DFE (D), and SNR MMSE DFE,U. 3.9 Finite-Length EQ - 9 pts A baseband channel is given by P (f) = p.7e f f <.5 0 f.5 (3.633) and finite-length equalization is used with anti-alias perfect LPR with gain p followed by symbol-rate sampling. After the sampling, a maximum complexity of 3 taps OAL over a feedforward filter and a feedback filter can be used. Ex = and N0 =.0 for a symbol rate of / = 00kHz is used with PAM transmission. Given the complexity constraint, find the highest data rate achievable with PAM transmission when the corresponding probability of symbol error must be less than Infinite-length Channel and Equalizer - 0 pts he pulse response on a channel with additive white Gaussian noise is p(t) =e at u(t), where u(t) is the unit step response. a>0 and the two symbol inputs per dimension are ±. Only QAM CR or SQ constellations allowed. a. What is the argument to the Q-function in P e = Q(x) at very low symbol rates? ( pt) b. For P e 0 6, N0 =.05, and very low symbol period, what is the value of a =? ( pt) For the remainder of this problem, let =. c. Find Q(D). ( pt) d. Find D ms.(pt) e. Find the receiver normalized matched filter prior to any symbol-rate sampling device. ( pt) f. Find SNR MFB. Let SNR = Ēx/ N0.(pt) g. Find the loss of the ZF-DFE with respect to the MFB. ( pt) 30

145 h. Find SNR MMSE DFE,U for a = and SNR = 0 db. (3 pts) i. What would an increase in a> with respect to Part h do to the gap in performance between the ZF-DFE and the MMSE-DFE? ( pts) 3.3 Diversity Channel - 6 pts his problem compares the channel studied throughout this text with the same input energy Ēx = and noise PSD of N0 =.8 versus almost the same channel except that two receivers independently receive: an undistorted delayed-by- signal (that is P (D) = D), and a signal that is not delayed, but reduced to 90% of its amplitude (that is P (D) =.9). On both paths, the channel passes nonzero energy only between zero frequency and the Nyquist frequency. a. Find Q(D), kpk, and SNR MFB for the later diversity channel. (3 pts) b. Find the performance of the MMSE-LE, MMSE-DFE for the later diversity channel and compare to that on the original one-receiver channel and with respect to the matched filter bound. ( pts) c. Find the probability of bit error on the diversity channel for the case of bit/dimension transmission. ( pt) 3.3 Do we understand basic detection - 9 pts A filtered AWGN channel with = has pulse response p(t) =sinc(t)+sinc(t ) with SNR MFB = 4. db. a. Find the probability of error for a ZF-DFE if a binary symbol is transmitted ( pts). b. he omlinson precoder in this special case of a monic pulse response with all unity coe cients can be replaced by a simpler precoder whose output is x 0 k = x 0 k if x k = x 0 k if x k = (3.634) Find the possible (noise-free) channel outputs and determine an SBS decoder rule. What is the performance (probability of error) for this decoder? ( pts) c. Suppose this channel (without precoder) is used one time for the transmission of one of the 8 4-dimensional messages that are defined by [+ ± ± ±], which produce 8 possible 5-dimensional outputs. he decoder will use the assumption that the last symbol transmitted before these 4 was -. ML detection is now used for this multidimensional -shot channel. What are the new minimum distance and number of nearest neighbors at this minimum distance, and how do they relate to those for the system in part a? What is the new number of bits per output dimension? (3 pts) d. Does the probability of error change significantly if the input is extended to arbitrarily long block length N with now the first sample fixed at + and the last sample fixed at, and all other inputs being equally likely + or in between? What happens to the number of bits per output dimension in this case? ( pts) Equalization of a 3-tap channel - 0 pts PAM transmission wiht Ēx = is used on a filtered AWGN channel with N0 =.0, =, and pulse repsonse p(t) =sinc(t)+.8sinc(t ) +.8sinc(t ). he desired probability of symbol error is 0 6. a. Find kpk, SNR MFB, AND Q(D) for this channel. ( pts) 303

146 b. Find the MMSE-LE and corresponding unbiased SNR MMSE LE,U for this channel. ( pts) c. Find the ZF-DFE detection SNR and loss with respect to SNR MFB.(pt) d. Find the MMSE-DFE. Also compute the MMSE-DFE SNR and maximum data rate in bits/dimension using the gap approximation. (3 pts) e. Draw the flexible (Laroia) precoder for the MMSE-DFE and draw the system from transmit symbol to detector in the receiver using this precoder. Implement this precoder for the largest number of integer bits that can be transmitted according to your answer in Part d. ( pts) 3.34 Finite-Length Equalizer Design - Lorentzian Pulse Shape- pts A filtered AWGN channel has the Lorentzian impulse response h(t) = + 07 t 3, (3.635) with Fourier ransform H(f) = 3 0 e f. (3.636) his channel is used in the transmission system of Figure 3.7. QAM transmission with / = MHz and carrier frequency f c = 600kHz are used on this channel. he AWGN psd is N0 = 86.5 dbm/hz. he transmit power is E x = mw. he oversampling factor for the equalizer design is chosen as l =. Square-root raised cosine filtering with 0% excess bandwidth is applied as a transmit filter, and an ideal filter is used for anti-alias filtering at the receiver. A matlab subroutine at the course web site may be #85(dBm/Hz( E x x k ( = mw Sq#rt#rcr:(α=.0( h(t)% +( y(t)% p~ ( t) Figure 3.7: ransmission system for Problem useful in computing responses in the frequency domain. a. Find so that an FIR pulse response approximates this pulse response so that less than 5% error 30 in kpk.(3pts) 30 5% error in approximation does not mean the SNR is limited to /0 or 3 db. 5% error in modeling a type of pulse response for analysis simply means that the designer is estimating the performance of an eventual adaptive system that will adjust its equalizer parameters to the best MMSE settings for whatever channel. hus, 5% modeling error is often su cient for the analysis step to get basic estimates of the SNR performance and consequent achievable data rates. 304

147 b. Calculate SNR MFB. Determine a reasonable set of parameters and settings for an FIR MMSE-LE and the corresponding data rate at P e = 0 7. Calculate the data rate using both an integer and a possibly non-integer number of bits/symbol. (4 pts) c. Repeat Part b for an FIR MMSE-DFE design and draw receiver. Calculate the data rate using both an integer and a possibly non-integer number of bits/symbol. (3 pts) d. Can you find a way to improve the data rate on this channel by changing the symbol rate and or carrier frequency? ( pts) 3.35 elephone-line ransmission with - 4 pts Digital transmission on telephone lines necessarily must pass through two isolation transformers as illustrated in Figure hese transformers prevent large D.C. voltages accidentally placed on the line from unintentionally harming the telephone line or the internet equipment attached to it, and they provide immunity to earth currents, noises, and ground loops. hese transformers also introduce ISI. modem& transformer& phone&line& transformer& modem& Figure 3.73: Illustration of a telephone-line data transmission for Problem a. he derivative taking combined characteristic of the transformers can be approximately modeled at a sampling rate of / =.544 MHz as successive di erences between channel input symbols. For su ciently short transmission lines, the rest of the line can be modeled as distortionless. What is a reasonable partial-response model for the channel H(D)? Sketch the channel transfer function. Is there ISI? (3 pts) b. How would a zero-forcing decision-feedback equalizer generally perform on this channel with respect to the case where channel output energy was the same but there are no transformers? ( pts) c. What are some of the drawbacks of a ZF-DFE on this channel? ( pts) d. Suppose a omlinson precoder were used on the channel with M =, how much is transmit energy increased generally? Can you reduce this increase by good choice of initial condition for the omlinson Precoder? (3 pts) e. Show how a binary precoder and corresponding decoder can significantly simplify the implementation of a detector. What is the loss with respect to optimum MFB performance on this channel with your precoder and detector? ( 3 pts) f. Suppose the channel were not exactly a PR channel as in Part a, but were relatively close. Characterize the loss in performance that you would expect to see for your detector. ( pt) 3.36 Magnetic Recording Channel - 4 pts Digital magnetic information storage (i.e., disks) and retrieval makes use of the storage of magnetic fluexes on a magnetic disk. he disk spins under a read head and by Maxwell s laws the read-head wire senses flus changes in the moving magnetic field. his read head thus generates a read current that is translated into a voltage through amplifiers succeeding the read head. Change in flux is often encoded to mean a was stored and no change means a 0 was stored. he read head also has finite-band limitations. 305

148 a. Pick a partial-resonse channel with = that models the read-back channel. Sketch the magnitude characteristic (versus frequency) for your channel and justify its use. ( pts) b. How would a zero-forcing decision-feedback equalizer generally perform on this channel with respect to the best case where all read-head channel output energy conveyed either a positive or negative polarity for each pulse? ( pt) c. What are some of the drawbacks of a ZF-DFE on this channel? ( pts) d. Suppose a omlinson Precoder were used on the channel (ignore the practical fact that magnetic saturation might not allow a omlinson nor any type of precoder) with M =, how much is transmit energy increased generally? Can you reduce this increase by good choice of initial condition for the omlinson Precoder? ( pts) e. Show how a binary precoder and corresponding decoder can significantly simplify the implementation of a detector. What is the loss with respect to optimum MFB performance on this channel with your precoder and detector? (3 pts) f. Suppose the channel were not exactly a PR channel as in part a, but were relatively close. Characterize the loss in performance that you would expect to see for your detector. ( pt) g. Suppose the density (bits per linear inch) of a disk is to be increased so one can store more files on it. What new partial response might apply with the same read-channel electronics, but with a correspondingly faster symbol rate? (3 pts) 3.37 omlinson Preocoding and Simple Precoding - 8 pts Section 3.8 derives a simple precoder for H(D) = + D. a. Design the omlinson precoder corresponding to a ZF-DFE for this channel with the possible binary inputs to the precoder being ±. (4 pts) b. How many distinct outputs are produced by the omlinson Precoder assuming an initial state (feedback D element contents) of zero for the precoder. ( pt) c. Compute the average energy of the omlinson precoder output. ( pt) d. How many possible outputs are produced by the simple precoder with binary inputs? ( pt) e. Compute the average energy of the channel input for the simple precoder with the input constellation of Part a. ( pt) 3.38 Flexible Precoding and Simple Precoding - 8 pts Section 3.8 derives a simple precoder for H(D) = + D. a. Design the Flexible precoder corresponding to a ZF-DFE for this channel with the possible binary inputs to the precoder being ±. (4 pts) b. How many distinct outputs are produced by the Flexible Precoder assuming an initial state (feedback D element contents) of zero for the precoder. ( pt) c. Compute the average energy of the Flexible precoder output. ( pt) d. How many possible outputs are produced by the simple precoder with binary inputs? ( pt) e. Compute the average energy of the channel input for the simple precoder with the input constellation of Part a. ( pt) 3.39 Partial Response Precoding and the ZF-DFE - 0 pts An AWGN has response H(D) =( D) with noise variance and one-dimensional real input x k = ±. 306

149 a. Determine a partial-response (PR) precoder for the channel, as well as the decoding rule for the noiseless channel output. ( pts) b. What are the possible noiseless outputs and their probabilities? From these, determine the P e for the precoded channel. (4 pts) c. If the partial-response precoder is used with symbol-by-symbol detection, what is the loss with respect to the MFB? Ignore nearest neighbor terms for this calculation since the MFB concerns only the argument of the Q-function. ( pt) d. If a ZF-DFE is used instead of a precoder for this channel, so that P c (D) = D + D, what is 0? Determine also the SNR loss with respect to the SNR MFB ( pts) e. Compare this with the performance of the precoder, ignoring nearest neighbor calculations. ( pt) 3.40 Error propagation and nearest neighbors - 0 pts A partial-response channel has H(D) = D channel with AWGN noise variance and d = and 4-level PAM transmission. a. State the precoding rule and the noiseless decoding rule. ( pts) b. Find the possible noiseless outputs and their probabilities. Find also N e and P e with the use of precoding. (4 pts) c. Suppose a ZF-DFE is used on this system and that at time k = 0 an incorrect decision x 0 ˆx 0 = occurs. his incorrect decision a ects z k at time k =. Find the N e (taking the error at k =0 into account) for the ZF-DFE. From this, determine the P e with the e ect of error propagation included. (4 pts) d. Compare the P e in part (c) with that of the use of the precoder in Part a. ( pt) 3.4 Forcing Partial Response - 6 pts Consider a H(D) =+.9D channel with AWGN noise variance to a + D channel. he objective is to convert this a. Design an equalizer that will convert the channel to a + D channel. ( pts) b. he received signal is y k = x k +.9 x k + n k where n k is the AWGN. Find the autocorrelation of the noise after going through the receiver designed in Part a. Evaluate r 0, r ±, and r ±.Isthe noise white? (3 pts) c. Would the noise terms would be more or less correlated if the conversion were instead of a +.D channel to a + D channel? You need only discuss briefly. ( pt) 3.4 Equalization of a General Single-Pole Channel - pts PAM transmission on a filtered AWGN channel uses basis function '(t) = p sinc t with = and undergoes channel impulse response with Fourier transform ( < ) H(!) = +a e!! apple 0! > (3.637) and SNR = Ēx = 8 db. a. Find the Fourier ransform of the pulse response,p (!) =? ( pt) b. Find kpk.(pt) c. Find Q(D), the function characterizing ISI. ( pts) 307

150 d. Find the filters and sketch the block diagram of receiver for the MMSE-DFE on this channel for a =.9. (3 pts) e. Estimate the data rate for uncoded PAM transmission and P e < 0 6 that is achievable with your answer in part d. ( pts) f. Draw a diagram of the better precoder s (omlinson or Laroia), transmitter and receiver, implementations with d = in the transmitted constellation. ( pts) 3.43 Finite Equalization of a causal complex channel 5 pts A channel sampled at symbol rate = (after anti-alias filtering) has output samples given by y k = p k x k + n k, where p k = k +( +0.5 ) k+, E[n k n k l ]= l 50 a. Compute SNR MFB.(pt) and ex =. b. For a channel equalizer with 3 feedforward taps and no feedback, write the P convolution matrix. Is this channel causal? If not, is it somehow equivalent to causal for the use of equalization programs like DFE color? ( pts) c. For the re-indexing of time to correspond to your answer in Part b, find the best delay for an MMSE-LE when N f = 3. ( pt) d. Find W MMSE LE for N ff = 3 for the delay in Part c. ( pt) e. Convolve the equalizer for Part d with the channel and show the result. ( pt). f. Compute SNR MMSE LE,U and the corresponding MMSE. (both per-dimension and per symbol). ( pts). g. Compute the loss w.r.t. MFB for the 3-tap linear equalizer. ( pt). h. Find a better linear equalizer using up to 0 taps and corresponding SNR MMSE LE,U improvement. ( pts) i. For your answer in Part h, what is P e for 6 QAM transmission? ( pt) j. Design an MMSE-DFE with no more than 3 total taps that performs better than anything above. (3 pts) 3.44 Diversity Concept and COFDM - 7 pts An additive white Gaussian noise channel supports QAM transmission with a symbol rate of / = 00 khz (with 0 % excess bandwidth) anywhere in a total baseband-equivalent bandwidth of 0 MHz transmission. Each 00 khz wide QAM signal can be nominally received with an SNR of 4.5 db without equalization. However, this 0 MHz wide channel has a 00 khz wide band that is attenuated by 0 db with respect to the rest of the band, but the location of the frequency of this notch is not known in advance to the designer. he transmitters are collocated. a. What is the data rate sustainable at probability of bit error P b apple 0 7 in the nominal condition? ( pt) b. What is the maximum number of simultaneous QAM users can share this channel at the performance level of Part a if the notch does not exist? ( pt) c. What is the worst-case probability of error for the number of users in Part b if the notch does exist? ( pt) 308

151 d. Suppose now (unlike Part b) that the notch does exist, and the designer decides to send each QAM signal in two distinct frequency bands. (his is a simple example of what is often called Coded Orthogonal Frequency Division Modulation or COFDM.) What is the worst-case probability of error for a diversity-equalization system applied to this channel? ( pt) e. For the same system as Part c, what is the best SNR that a diversity equalizer system can achieve? ( pt) f. For the system of Part c, how many users can now share the band all with probability of bit error less than 0 7? Can you think of a way to improve this number closer to the level of the Part b? ( pts) 3.45 Hybrid ARQ (HARQ) Diversity - pts A time-varying wireless channel sends long packets of binary information that are decoded by a receiver that uses a symbol-by-symbol decision device. he channel has pulse response in the familiar form: P (!) = p +ai e!! apple 0! > (3.638) where a i may vary from packet to packet (but not within a packet). here is also AWGN with constant power spectral density N0 =.. QPSK (4QAM) is sent on this channel with symbol energy. a. Find the best DFE SNR and corresponding probability of symbol error for the two cases or states when a =.9 and a =.5. Which is better and why? ( pts) Now, suppose the transmitter can resend the same packet of symbols to the receiver, which can delay the channel output packet from the first transmission until the symbols from the new packet arrive. Further suppose that the second transmission occurs at a time when the channel state is known to have changed. However, a symbol-by-symbol detector is still to be used by the receiver jointly on both outputs. his is a variant of what is called Automatic Repeat request or ARQ. (Usually ARQ only resends when it is determined by the receiver that the decision made on the first transmission is unreliable and discarded. Hybrid ARQ uses both channel outputs.) For the remainder of this problem, assume the two a i values are equally likely. b. Explain why this is a diversity situation. What is the approximate cost in data rate for this retransmission in time if the same symbol constellation (same b) is used for each transmission? ( pt) c. Find a new SNR MFB for this situation with a diversity receiver. ( pt) d. Find the new function Q(D) for this diversity situation. ( pts) e. Determine the performance of the best DFE this time (SNR and P e ) and compare to the earlier single-instance receivers. ( pts) f. Compare the P e of Part e with the product of the two P e s found in Part a. Does this make sense? Comment. ( pts) g. Draw the receiver with RAKE illustrated as well as DFE, unbiasing, and decision device. (phase splitter can be ignored in diagram and all quantities can be assumed complex.) ( pts) Precoder Diversity - 7 pts A system with 4 PAM transmits over two discrete-time channels shown with the two independent AWGN channels shown in Figure (Ēx = ) 309

152 σ =.0 (+D) ((D)" +" y,k y,k σ =.0 (D (3 " "D ( "+"D ( "+" +" y,k Figure 3.74: ransmission system for Problem a. Find the RAKE matched filter(s) for this system? ( pt) b. Find the single feedforward filter for a ZF-DFE? ( pt) c. Show the best precoder for the system created in Part b. ( pt) d. Find the SNR at the detector for this precoded system? ( pt) e. Find the loss in SNR with respect to the of either single channel. ( pt) f. Find the P e for this precoded diversity system. ( pt) g. Find the probability that a packet of 540 bytes contains one or more errors. What might you do to improve? ( pt) 3.47 Equalizer Performance and Means - 0 pts Recall that arithmetic mean, geometric mean, and harmonic mean are of the form P n n i= x i, ( Q n i= x i) n P n,, n i= x i respectively. Furthermore, they satisfy the following inequalities: n nx i= x i with equality when x = x = = x n n Y i= x i! n n nx x i= i!, a. Express SNR ZFE, SNR ZF DFE, SNR MFB in terms of SNR MFB and frequency response of the autocorrelation function q k. Prove that SNR ZFE apple SNR ZF DFE apple SNR MFB using above inequalities. When does equality hold? hint: use R b a x(t)dt =lim P n n! k= x( b a n k + a) b a n. (4 pts) b. Similarly, prove that SNR MMSE LE,U apple SNR MMSE DFE,U apple SNR MFB using above inequalities. When does equality hold? (4 pts) c. Compare SNR ZFE and SNR MMSE LE. Which scheme has better performance? ( pts) 30

153 3.48 Unequal Channels and Basic Loading Malkin- 6 pts Use the gap approximation for this problem. A sequence of 6-QAM symbols with in-phase component a k and quadrature component b k at time k is transmitted on a passband channel by the modulated signal x(t) = p (" # " # ) X X a k '(t k) cos(! c t) b k '(t k) sin(! c t), (3.639) k where = 40 ns, f c =.4 GHz, and the transmit power is 700 µw. he transmit filter/basic function is '(t) '(t) = p t sinc, (3.640) and the baseband channel response is a frequency translation given by H(f) =e j5 f. he received signal is thus y(t) =h(t) x(t)+n(t) (3.64) where n(t) is an additive white Gaussian noise random process with two-sided PSD of -00 dbm/hz. a. What is the data rate of this system? ( pts) b. What are E x, Ēx and d min for this constellation? ( pts) c. What is x bb (t)? (.5 pts) d. What is the ISI characterizing function q(t)? what is q k? (.5 pts) e. What are P e and P e for an optimal ML detector? ( pts) For the remainder of this problem: he implementation of the baseband demodulator is faulty and the additive Gaussian noise power in the quadrature component is = 4 times what it should be. hat is, the noise variance of the quadrature component is N0 =N 0 while the noise variance in the in-phase dimension is N0. Answer the following questions for this situation. f. What is the receiver SNR? Is this SNR meaningful any longer in terms of the gap formula? (.5 pts) g. What are P e and P e?(3pts) h. Instead of using QAM modulation as above, two independent 4-PAM constellations (with half the total energy allocated for each constellation) are now transmitted on the in-phase and quadrature channels, respectively. What is P e averaged over both of the PAM constellations? (.5 pts) i. he system design may allocate di erent energies for the in-phase and quadrature channels (but this design still uses QAM modulation with the same number of bits on each of the in-phase and quadrature channels, and the transmit power remains at 700 µw ). Use di erent inphase and quadrature energies to improve the P e performance from Part (g). What is the best energy allocation for the in-phase and quadrature channels? For the given P e = 0 6, what is b and the achievable data rate? (3 pts) j. Using the results from part (i) find the minimal increase in transmit power needed to guarantee P e = 0 6 for 6-QAM transmission? (.5 pts) k he design may now reduce losses. Given this increased noise in the quadrature dimension, tthe design may use only the in-phase dimension for transmission (but subject to the original transmit power constraint of 700 µw ). 3

154 k. X What data rate can be then supported that gives P e = 0 6? (.5 pts) l. Using a fair comparison, compare the QAM transmission system from part (i) to the singledimensional scheme (N = ) from part (k). Which scheme is better? (3 pts) m. Design a new transmission scheme that uses both the in-phase and quadrature channels to get a higher data rate than part (i), subject to the same P e = 0 6, and original symbol rate and power constraint of 700 µw.(3pts) 3.49 A Specific One-Pole Channel (9 pts) A transmission system uses the basis function of the channel impulse response is: (t) =sinc(t) with = Hz. he Fourier transform H(f) = 0.5j e j f, for f apple 0, for f >. (3.64) and SNR = "x = 0 db. a. Find p and Q(D). ( pt) b. Find W ZFE (D) and W MMSE LE (D). ( pt) c. Find W (D) and B(D) for ZF-DFE. ( pt) d. Find W (D) and B(D) for MMSE-DFE. ( pts) e. What is SNR ZF DFE? (0.5 pt) f. Design a omlinson precoder based on the ZF-DFE. Show both the precoder and the corresponding receiver (for any M). (.5 pts) g. Design a Laroia precoder based on the ZF-DFE. Assume the system uses 4-QAM modulation. Show both the precoder and the corresponding receiver. he kind of constellation used at each SBS detector should be shown. ( pts) 3.50 Infinite Diversity - Malkin (7 pts) A receiver has access to an infinite number of diversity channels where the output of channel i at time k is y k,i = x k +0.9 x k+ + n k,i, i =0,,,... where n k,i is a Gaussian noise process independent across time and across all the channels, and with variance i =. i 0.8. Also, Ēx =. a. Find SNR MFB,i,theSNR MFB for channel i. (pt) b. What is the matched filter for each diversity channel? ( pts) c. What is the resulting Q(D) for this diversity system? (.5 pts) d. What is the detector SNR for this system for a MMSE-DFE receiver? (.5 pts) 3.5 Infinite Feedforward and Finite Feedback - Malkin (9 pts) A symbol-rate sampled MMSE-DFE structure, where the received signal is filtered by a continuous time matched filter, uses an infinite-length feedforward filter. However, the MMSE-DFE feedback filter has finite length N b. 3

155 a. Formulate the optimization problem that solves to find the optimal feedforward and feedback coe - cients to minimize the mean squared error (Hint: Don t approach this as a finite-length equalization problem as in Section 3.7). Assume that N0 =.8, E x =, and P (D) =+.9 D. (and SNR MFB = 0 db). ( pts) b. Using only feedback tap (N b = ), determine the optimal feedforward and feedback filters (Hint: his is easy - you have already seen the solution before). ( pts) c. Now consider the ZF-DFE version of this problem. State the optimization problem for this situation (analogous to Part a). ( pts) Now, change the channel to the infinite-impulse response of P (D) = +0.9D (and kpk = 0.9 ). 0.9 = d. Determine the optimal feedforward and feedback filters for the ZF-DFE formulation using only feedback tap (N b = ). (3 pts) 3.5 Packet Processing - Malkin (0 pts) Suppose that in a packet-based transmission system, a receiver makes a decision from the set of transmit symbols x,x,x 3 based on the channel outputs y,y,y 3,where 4 y 3 y y 3 5 = 4 p p p 3 0 p p p x 3 x x n 3 n n 3 5 = x 3 x x n 3 n n 3 5 (3.643) and where the noise autocorrelation matrix is given by 0 3 n 3 R n = 4 n 5 n 3 n n A = 4 n (3.644) he inverse of the channel matrix is given by = (3.645) he transmit symbols are i.i.d with E x = E x = E x3 = 0. a. What is the 3-tap zero-forcing equalizer for x based on the observation y,y,y 3? Repeat for x and x 3.(.5 pts) b. What is the matrix ZFE for detecting x,x, and x 3? (Hint: his is easy - all the work occurred in the previous part) ( pt) c. What is the detection SNR for x using a ZFE? Answer the same for x and x 3. (.5 pts) d. An engineer now realizes that the performance could be improved through use previous decisions when detecting the current symbol. If the receiver starts by detecting x,thenx, and finally x 3, describe how this modified ZF detector would work? ( pt) e. Assuming previous decisions are correct, what is now the detection SNR for each symbol? (.5 pts) f. he approach above does not necessarily minimize the MMSE. How would a packet MMSE-DFE work? How would you describe the feedback structure? (Don t solve for the optimal matrices - just describe the signal processing that would take place). ( pt) 33

156 g. Can the HP also be implemented for this packet based system? Describe how this would be done ( a description is su cient response). ( pt) 3.53 DFEcolor Program with Nontrivial Complex Channel 9 pts Use of the dfecolor.m program in Matlab produces: >> p = [ i i i i] >> [snr,w]=dfecolor(,p,7,3,6,,.03*[ zeros(,6)]) snr = w = Columns through i i i i i Columns 6 through i i i i i a. What is equalizer output (before bias removal), E [z k /x k ], for the equalizer as output above? ( pt) b. If the noise vector in the matlab command above becomes.0*[ *i], what will happen to DFE performance (improve/degrade)? ( pt) c. Find B U (D) for the original (white) noise. ( pt) d. What is the maximum data rate if the symbol rate is 00 MHz and P e = 0 6? (Assume no error propagation and arbitrary constellations with SBS decisions in the MMSE-DFE.) ( pt) e. What is the maximum data rate that can be achieved if a omlinson precoder is used? ( pt) f. If the carrier frequency is 75 MHz, what are possible pulse-response samples for the actual channel? ( pt) g. Assuming the channel has the minimum bandwidth for the pulse response given, what is the loss with respect to the matched filter bound? ( pt) h. When bias is absorbed into the feedforward equalizer, what is the magnitude of the center tap? ( pt) i. For a flexible precoder in the transmitter, what is the maximum number of points in the first two-dimensional decision device used in the receiver? ( pt) 3.54 Finite-Length Equalizer Design with Diversity 7 pts A receiver designer has two options for the two-path channel described below. he designer may use only a single antenna (Parts a to c) that receives both paths outputs added together with a single AWGN power spectral density of -0 dbm/hz. Or as this problem progresses in Parts d and beyond, the designer may instead use two antennas that are directional with one receiver only on the first path with AWGN at -3 dbm/hz and the second receiver only only the second path with AWGN also at -3 dbm/hz (the noise that was present on the channel is presumably split evenly in this case). he transmit power is P x = E x apple 0 dbm. he symbol rate is MHz. he target probability of error is 0 6 and only integer number of bits/symbol are allowed. he system is sampled at instants k/ =k 0 for integer k 0 he two paths sampled (at pulse responses are: where µs= 0 6 s. p (t 0 ) = e 06t u(t) p (t) = e (t 0.5µs) u(t 0.5µs), 34

157 a. Is there aliasing even with a sampling rate of /? Find by continuous integration kp k, kp k, and kp + p k and comment on the latter with respect to the sum of the former. Repeat this exercise for kp (k 0 )k, kkp (k 0 )k and kp (k 0 )+p (k 0 )k, and comment on their size relative to the true norms found by continuous integration relative to the choice of symbol rate (and thus also sampling rate). (4 pts) b. Find and the FIR pulse response that approximates the actual response so that less than.5 db error 3 in kpk (that is, the norm square of the di erence between the actual p and the truncated p is less than (0.05 ) kpk ). Now find the Ex input for the dfecolor program, the vector p input that would be input to the DFECOLOR matlab program, and the CORRESPONDING noise vector that would be input. Please find these presuming that the anti-alias filter for / 0 sampling is normalized. (3 pts) c. Design an FIR DFE that achieves within. db of the infinite-length performance for Part b s inputs. Record the smallest numbers of feedforward and feedback taps for which the equalizer can achieve this performance and the corresponding delay. An acceptable response is the equalizer coe cient vector from Matlab decomposed separately into feedforward and feedback taps. Find the SNR and corresponding data rate for the design (3 pts). d. Continue Part b except now find the two FIR pulse responses that approximates the pulse response so that less than.5 db error in the overall kpk occurs, and provide the larger value of. (pts) e. Now find the Ex input for the dfecolor program, the vector (rows) of p input that would be input to the DFERAKE matlab program, and the CORRESPONDING noise vector that would be input. Please find these presuming that the anti-alias filter for / 0 sampling is normalized on both antennas. ( pts) f. Design an FIR DFE that achieves within. db of the infinite-length performance for Part d s pulse responses. Record the smallest numbers of feedforward and feedback taps that can achieve this performance and the corresponding delay. An acceptable response is the equalizer coe cient vector from Matlab. Find the SNR and corresponding data rate for your design. (3 pts) Optimizing the DFE ransmit Filter 3 pts A discrete-time channel is described by y(d) = H(D) X(D) + N(D), where X(D) is the transmit symbol sequence, E[ and N(D) is AWGN with variance N0. hese notes so far have always assumed that the transmit sequence x k s white (messages are independent). Suppose the transmitter could filter the symbol sequence before transmission, though the filtered symbol sequence must have the same power Ex. Call the magnitude squared of the transmit filter kph(e! ). a. Given a MMSE-DFE receiver, find the transmit filter (squared magnitude) that maximizes the MMSE-DFE detection SNR. ( pts) b. Find the best transmit filter (squared magnitude) for the ZF-DFE ( pt) Multi-User Scenario pts A single receiver receives transmissions from two independent users with symbol sequences X (D) and X (D), respectively. he two received signals are (with a and b real). Y (D) = (+a D)) X (D) X (D)+N (D) Y (D) = X (D)+( b D ) X (D)+( b D ) N (D)+b D Y (D), 3 Readers should note that.5 db accuracy is only about 6% of the energy; however, this is close enough to estimate equalizer behavior of the actual system. In practice, an adaptive implementation would tune the equalizer to get roughly the same performance as is derived here for the truncated approximation to the response p(k/) over all time. 35

158 where N (D) is AWGN with variance N0, and similarly N (D) is AWGN with variance N0. he quantites a and b are real. he following are the energies E X (D)X (D = E E X (D)X (D = E. he noises N (D) and N (D) are independent of one another and all mutually independent of the two transmitted data sequences X (D) and X (D). For this problem treat all ISI and crosstalk interference as if it came from a normal distribution, unless specifically treated otherwise as in Parts f and beyond. Matlab may be useful in this problem, particularly the routines conv roots, and integral. he receiver will use both Y (D) and Y (D) for detection in all parts. a. he receiver here will use both Y (D) and Y (D) for optimum detection of user (X (D)), but treat X (D) as noise. Find the noise-equivalent channel P (D) and the resulting Q(D) for the case when E = N0. he Matlab Function chol may be useful in finding a noise-whitening filter. (3 pts) b. For E = 0, E =, N0 =, a =, and b =0.7, find Q(D), SNR MFB, Q(D), and the feed-forward and feedback filters for an infinite length MMSE-DFE for Part a s. he integral function may be convenient to compute the norm of channel elements. he roots and conv commands may also reduce labor to solve. (5 pts) c. Show a omlinson precoder implementation of the structure in part b if M-PAM is transmitted. ( pt) d. Suppose, instead, the receiver instead first detects user s signal X (D) based on both Y (D) and y (D) by using a ZF-DFE receiver. Please specify all the filters and the SNR ZF DFE, given that E =, E = 30, N0 =, a =, and b = 7. Noise whitening may need to presume a triangular form as in Part a except that now the elements are functions of D - just solve it, 3 equations in 3 unknown functions of D - not that hard for first two elements, 3rd is ratio of 3rd order polynomials he integral and conv Matlab functions are again useful. (4 pts) e. Show a Laroia implementation of the structure in Part d if M-PAM is transmitted. ( pts) f. his sub question investigates the system in Part b if x,k is known and its e ects removed at the receiver? Suppose a structure like that found in Parts b and c to detect x,k reliably. (Since E = 0 is large, the data rate for the channel from x,k to the receiver may be very low relative to the correct response for Part b.) Show a receiver for x,k that makes use of a known x,k.does this receiver look familiar in some way? What is roughly the new gap-based data rate achievable for x?(3pts) g. Repeat Part f for the numbers in Part d except that x,k is detected and its a ects removed. ( pts) h. Suppose time-sharing of the two systems (E = 0 or E = 30) is allowed (long blocks are assumed so any beginning or end e ects are negligible). What data rate pairs for x and x might be possible (think of plot in two dimensions)? ( pts) 36

159 Appendix A Useful Results in Linear Minimum Mean-Square Estimation, Linear Algebra, and Filter Realization his appendix derives and/or lists some key results from linear Minimum Mean-Square-Error estimation. Section A. is on the Orthogonality Principal that this text uses throughout to minimize mean square errors, both for scalar and vector processes. Section A. addresses scalar spectrum factorization and filter realization that is very important in solving many MSE problems also, especially for data transmission and noise whitening. he famed (scalar) Paley-Weiner Criterion is explained and derived in this Section. Section A.3 address a vector/matrix generalization of Paley Wiener heory that this text calls MIMO Paley-Weiner, and which provides elegant solution generalizations to many MIMO situations of increasing interest. Section A.4 is short and lists the well known matrix inversion lemma. A linear MMSE estimate of some random variable x, given observations of some other random sequences {y k }, chooses a set of parameters w k (with the index k having one distinct value for each observation of y k that is used) to minimize E e (A.) where e = x NX k=0 w k y k. (A.) N!without di culty. he term linear MMSE estimate is derived from the linear combination of the random variables y k in (A.). More generally, it can be shown that the MMSE estimate is E [x/{y k }], the conditional expectation of x, given y 0,...,y N. which will be linear for jointly Gaussian x and {y k }. his text is interested only in linear MMSE estimates. In this text s developments, y i will usually be successive samples of some channel output sequence, and x will be the current channel-input symbol that the receiver attempts to estimate. When the quantity being estimated is an L x -dimensional vector x, Equation (A.)) generalizes to E kek, (A.3) where e = x NX k=0 w k y k. (A.4) his sum of L x mean squared errors in (A.3) has each squared error within looking like (A.), while the set of parameters generalizes also to an L x -dimensional vector w k, each component of which multiplies the As Chapters 8 and show, the best codes for all single/multi-user AWGNs have Guassian input codes, so linear will indeed be best for such channels and optimized coded inputs. 37

160 same input y k. Further generalization in the same way allows the input y to become an L y -dimensional vector y k. In this case, each w k becomes an L x L y matrix W k. he vector-case MSE s are variables separable into individual scalar MMSE problems that can each be solved separately and the corresponding filter values stacked into the solution w or W (the latter of which may be a tensor or sequence of matrices). Minimizing the MSE in (A.3) is also equivalent to minimizing the trace of the matrix Ree = E [e k e k], (A.5) or equivalently minimizing the sum of the eigenvalues of Ree = Q e Q where is a diagonal matrix of non-negative real eigenvalues (see Matlab eig command) and QQ = Q Q = I. Since multiplication by a Q matrix (unitary) does not change the length of a vector, then the MMSE problem could be also solved by minimizing the mean square of ẽ = Qe and so the solution then would be W = QW, (A.6) which is now separable in the ẽ components. Given the separability, then the product, as well as the sum, of the eigenvalues has been minimized also. his means the determinant Ree has also been minimized because the product of the eigenvalues is the determinant (as should be obvious from observing Q = ). hus, often MMSE vector problems are written in terms of minimizing the determinant of the error autocorrelation matrix, but this is the same as minimizing the sum of component MSE s. A. he Orthogonality Principle While (A.) can be minimized directly by di erentiation in each case or problem of interest, it is often far more convenient to use a well-known principle of linear MMSE estimates - that is that the error signal e must always be uncorrelated with all the observed random variables in order for the MSE to be minimized. his must be true for each error-vector component as well. heorem A.. (Orthogonality Principle) he MSE is minimized if and only if the following condition is met E [e y k]=0 8 k =0,...,N (A.7) Proof: Writing e =[<(e)] +[=(e)] allows di erentiation of the MSE with respect to both the real and imaginary parts of w k for each k of interest. he pertinent parts of the real and imaginary errors are (realizing that all other w i, i 6= k, will drop from the corresponding derivatives) e r = x r w r,k y r,k + w i,k y i,k (A.8) e i = x i w i,k y r,k w r,k y i,k, (A.9) where subscripts of r and i denote real and imaginary part in the obvious manner. hen, to optimize over w r,k and w i = e r +e i = (e r y r,k + e i y i,k ) = 0 i = e r +e i =(e r y i,k e i y r,k )=0. i,k he desired result is found by taking expectations and rewriting the series of results above in vector form. Since the MSE is positive-semi-definite quadratic in the parameters w r,k and w i,k, this setting must be a global minimum. If the MSE is strictly positive-definite, then this minimum is unique. QED. 38

161 A. Spectral Factorization and Scalar Filter Realization his section provides results on the realization of desired filter-magnitude characteristics. he realized filters will be causal, causally invertible (minimum phase), and monic. Such realization invokes the so-called Paley-Wiener Criterion (PWC) that is constructively developed and proven as part of the realization process, basing the proof on discrete sequences but covering also continuous signals (in both cases applicable to deterministic magnitude-squared functions or to stationary random processes with consequently non-negative real power spectra). Section A.3 generalizes these filters and/or process and Paley Wiener criterion to MIMO (matrix filters and/or processes), which will require understanding Cholesky Factorization of simple matrices first from Subsection A..6 that ends this Section A. and precedes A.3. A.. Some ransform Basics: D-ransform and Laplace ransform his appendix continues use of D-transform notation for sequences : Definition A.. (D- ransforms) A sequence x k 8 integer k (, ) has D-ransform X(D) = P k= x k D k for all D D x, where D x is the region of convergence of complex D values for which the sum X(D) converges, and X(D) is analytic 3 in D x. he inverse transform is ahclockwise line/contour integral around any closed circle in the region of convergence x k = DD x X(D) D k dd. 4 he sequence x k can be complex. he symbol rate will be presumed = in this appendix 5. he sequence x k has a D-ransform X (D )= P k= x k D k. When the region of convergence includes the unit circle, the discrete-time sequence s Fourier transform exists as X(e! )=X(D) D=e!, and such sequences are considered to be stable or realizable (non-causal sequencies become realizable with su cient delay, or infinite delay in some limiting situations). A su cient condition for the discrete-time sequence s Fourier transform to exist (the sum X(D) converges) is that the sequence itself be absolutely summable, meaning X k= x k <, (A.) or equivalently the sequence belongs to the space of sequences L. Another su the sequence belongs to L or has finite energy according to X k= cient condition is that x k <. (A.3) he similarity Rof the form of transform and inverse then allows equivalently that the inverse Fourier ransform ( X (e! ) e!k d!) existsif: Z X(e! )) d! <, (A.4) or if Z X(e! )) d! <. (A.5) So readers can insert Z for D if they need to relate to other developments where Z transforms are used 3 Analytic here means the function and all its derivatives exist and are bounded - not to be confused with Chapter s analytic-equivalent signals. 4 ypically the inverse transform is implemented by partial fractions, which often arise in communication problems and in most approaches to contour integration anyway, whenever the D-transform is a rational fraction of polynomials in D. 5 he generalization for 6= isaddressedinthespecifictextsectionsofthischapterandlaterchapterswhennecessary. For instance, see able 3. for the generalization of transforms under sampling for all.samplingshouldnotbeconfused with the bi-linear transform: the former corresponds to conversion of a continuous waveform to discrete samples, while the latter maps filters or functions from/to continuous to/from discrete time and thus allows use of continuous- (discrete-) time filter realizations in the other discrete- (continuous-) time domain. 39

162 While x k L or x k L are su cient conditions, this book has already used non-l -nor-l functions that can have transforms. hese generalized functions include the Dirac Delta function (t) or (!), so that for instance cos(! 0 k) has Fourier ransform [ (!! 0 )+ (! +! 0 )], or the series x k = has transform (!), even though neither of the sums above in Equations (A.) and (A.3) converge for these functions. hese types of generalized-function-assisted Fourier ransforms are on the stability margin where values of D in a region of convergence arbitrarily close to the unit circle (outside, but not on) will converge so the criteria are considered satisfied in a limiting or generalized sense. A continuous-time sequence x(t) has a Lapace ransform X(s) defined over a region of convergence S x as Definition R A.. (Laplace ransform) A function x(t) has Lapace-ransform X(s) = x(t)e st dt, which converges/exists for all s S x, where S x is the region of convergence of complex s values. he inverse transform is H X(s) e st ds when the closed contour of integration is in S x. he time function x(t) can be complex. he function x ( t) has a Lapaceransform R x ( t)e st dt = X ( s ). When the region of convergence includes the! axis, the Fourier transform exists as X(!) = X(s) s=!, and such functions are considered to be stable or realizable (non-causal functions become realizable with su cient delay, or infinite delay in some limiting situations). A su cient condition for the continuous Fourier transform to exist (the integral converges) is that the function be absolutely integrable, meaning Z x(t) dt <, or equivalently (A.6) Z X(!) d! <, (A.7) or equivalently that the function x(t) belongs to the space of continuous functions L. Another su condition is that function x(t) belongs to L or has finite energy according to Z cient x(t) dt <, or equivalently (A.8) Z X(!) d! < (A.9) Similar to the discrete-time D-ransform, generalized functions complete the capability to handle continuous Fourier ransforms that are on the stability margin where values of s in a region of convergence arbitrarily close to the! axis (left of, but not on this axis) will converge so the criteria are considered satisfied in a limiting or generalized sense. A.. Autocorrelation and Non-Negative Spectra Magnitudes Of particular interest in this text, and generally in digital communication, are the autocorrelation functions and associated power spectra for stationary and wide-sense stationary processes. (See also Appendix C of Chapter for continuous time.) hese concepts are revisited briefly here for discrete processes before returning to filter realization of a given specified non-negative Fourier ransform magnitude. Definition A..3 (Autocorrelation and Power Spectrum for Sequences) If x k is any stationary complex sequence, its autocorrelation function is defined as the sequence R xx (D) whose terms are r xx,j = E[x k x k j ];symbolically6 R xx (D) = E X(D) X (D ). (A.0) 6 he expression R xx(d) = E X(D)X (D ) is used in a symbolic sense, since the terms of X(D)X (D )areof the form P k x kx k P j,implyingtheadditionaloperationlim N![/(N +)] on the sum in such terms. NapplekappleN 30

163 By stationarity, r xx,j = r xx, j and R xx(d) = R xx(d ). he power spectrum of a stationary sequence is the Fourier transform of its autocorrelation function, which is written as R xx (e! )=R xx (D) D=e!, <!apple, (A.) which is real and nonnegative for all!. Conversely, any complex sequence R(D) for which R(D) =R (D ) is an autocorrelation function, and any function R(e! ) that is real and nonnegative over the interval { <!apple } is a power spectrum. he quantity E x k is Ex, or Ēx per dimension, and can be determined from either the autocorrelation function or the power spectrum as follows: Ex = E x k = r xx,0 = Z R xx (e! )d!. (A.) If the sequence in question (or a filter for instance) is deterministic, then the averages are not necessary above. he power spectra is then essentially the magnitude squared of the Fourier ransform R(e! ) = X(e! ) 0 for discrete time. hese Fourier ransforms magnitudes can be thought of also as power spectra, and the corresponding inverse transforms as autocorrelation functions in this text. Definition A..4 (Autocorrelation and Power Spectrum for Continuous-time Functions) If x(t) is any stationary (or WSS, see Chapter Appendices) complex function, its autocorrelation function is defined as the Laplace ransform R xcx c (s) whose terms are r xcx c (t) =E[x c (u)x c(u t)]; symbolically 7 R xcx c (s) = E [X(s) X ( s )]. (A.3) By stationarity, r xcx c (t) =r x cx c ( t) and R xcx c (s) =R x cx c ( s ).hepower spectrum of a stationary continuous-time process is the Fourier transform of its autocorrelation function, which is written as R xcx c (! c )=R xcx c (s) s=!c, <! c apple, (A.4) which is real and nonnegative for all! c. Conversely, any complex sequence R(s) for which R(s) = R ( s ) is an autocorrelation function, and any function R(!) that is real and nonnegative over the interval { <! c apple } is a power spectrum. he quantity E x c (t) is P x, or the power of the random continuous-time process, and can be determined from either the autocorrelation function or the power spectrum as follows: P x = E x c (t) = r xcx c (0) = Z R xcx c (! c ) d! c. (A.5) If the sequence in question (or a filter for instance) is deterministic, then the averages are not necessary above. he power spectra is then essentially the magnitude squared of the Fourier ransform R(! c ) = X c (! c ) 0 for continuous time. hese Fourier ransforms magnitudes can be thought of also as power spectra, and the corresponding inverse transforms as autocorrelation functions in this text. A..3 he Binlinear ransform and Spectral Factorization his section denotes a continuous-time function s Fourier transform radian frequency by! c while the discrete-time sequence s Fourier ransform variable will be! (with no subscript of c). Similarly all continuous-time quantities will use a subscript of c to avoid confusion with discrete time. For the transforms, if a transform X(D) or X c (s) exists in their respective regions of convergence, then the 7 he R expression R xcxc (s) = E [X(s)X ( s )] is used in a symbolic sense, since R the terms of X(s)X ( s ) are of the form E[xc(u)x c(u t)]du, implyingtheadditionaloperationlim! [/( )] on the integral in such terms. applevapple 3

164 transforms e X(D) and e Xc(s) also exist in that same region of convergence 8. Similarly then, ln(x(d)) and ln(x c (s)) also have the same regions of convergence. A filter-design technique for discrete-time filters uses what is known as the bi-linear transform to map a filter designed in continuous time into a discrete-time filter (or vice versa): Definition A..5 (Binlinear ransforms) he bilinear transform maps between discretetime D ransforms and continuous-time Laplace ransforms according to and conversely s = D +D D = s +s (A.6). (A.7) he bilinear transform can also relate discrete-time and continuous-time Fourier ransforms by inserting D = e! and s =! c.he! c (complex) axis from 0 to ± corresponds to mapping D = e! along the unit circle (centered at origin of D plane) from the (real, imaginary) D-plane point [, 0] of 0 radians to the point of radians (or [, 0]) clockwise for positive! c (counter clockwise for negative! c ). his blinear transform scales or compresses the infinite range of frequencies! c (, ) for the continuous-time Fourier ransform to the finite range of frequencies! [, ] for the discretetime Fourier ransform. he bilinear transform does not correspond to sampling (and why = here to avoid confusion) to go from continuous time to discrete time. he bilinear transform is a filterdesign/synthesis method that maps a design from continuous time to/from discrete time. he designer needs to pre-warp filter cut-o frequencies with this compression/scaling in mind. A stable design in continuous time corresponds to all poles on/in the left-half plane of s (or the region of convergence includes, perhaps in limit, the! c axis). hese poles will map (with the warping of frequency scale) into corresponding points through the bilinear transform in the region outside (or in limiting sense on) the unit circle D =, and vice-versa. Similarly a minimum-phase design (all poles and zeros in LHP) will map to all poles/zeros outside the unit circle, and vice-versa. For the Fourier ransform, the blinear transformation maps frequency according to! c = e! +e! (A.8)! = tan (A.9)! = arctan(! c ) (A.30) d! c d! = +! c. (A.3) he spectral factorization of a discrete-time autocorrelation function s D-ransform is: Definition A..6 (Factorizability for Sequences) An autocorrelation function R xx (D), or equivalently any non-negative real R xx (e! ) so that r k = r k,willbecalledfactorizable if it can be written in the form R xx (D) =S x,0 G x (D) G x(d ), (A.3) where S x,0 is a finite positive real number and G x (D) is a canonical filter response. A filter response G x (D) is called canonical if it is causal (g x,k =0for k<0), monic (g x,0 = ), and minimum-phase (all of its poles and zeros are outside or on the unit circle). If G x (D) is canonical, then G x(d ) is anticanonical; i.e., anticausal, monic, and maximum-phase (all poles and zeros inside or on the unit circle). 8 his follows from the derivative of e f(x) for any function f(x) ise f(x) f 0 (x) soifthefunctionexistedatx, then e f(x) also exists at all argument values, and then since f 0 (x) alsoexits,thensodoesthederivative. hisargumentcanbe recursively applied to all successive derivatives that of course exist for f(x) initsdomainofconvergence. 3

165 he region of convergence for factorizable R xx (D) clearly includes the unit circle, as do the regions for both G x (D) and G x(d ). If G x (D) is a canonical response, then kg x k = P j g x,k, with equality if and only if G x (D) =, since G x (D) is monic. Further, the inverse also factorizes similarly into Rxx (D) =(/S x,0 ) Gx (D) Gx (D ). (A.33) Clearly if R xx (D) is a ratio of finite-degree polynomials in D, then it is factorizable (simply group poles/zeros together for inside and outside of the circle - any on unit circle will also appear in conjugate pairs so easily separated). For the situation in which R xx (D) is not already such a polynomial, the next section generalizes through the Paley-Wiener Criterion. Also, if R xx (D) is factorizable, then the corresponding R xcx c (s) =R xx s +s is also factorizable into R xcx c (s) =S xc,0 G xc (s) G x c ( s ). (A.34) Definition A..7 (Factorizability for Continuous Functions) An autocorrelation function R xcx c (s), or equivalently any non-negative real power spectrum R xcx c (!) so that r(t) = r ( t), willbecalledfactorizable if it can be written in the form R xcx c (s) =S xc,0 G xc (s) G x c ( s ), (A.35) where S xc,0 is a finite positive real number and G xc (s) is a canonical filter response. A filter response G xc (s) is called canonical if it is causal (g xc (t) =0for t<0), monic (g xc (0) = ), andminimum-phase (all of its poles and zeros are in the left half plane). If G xc (s) is canonical, then G x c ( s ) is anticanonical; i.e., anticausal, monic, and maximumphase (all poles and zeros inside in the right half plane). he region of convergence for factorizable R xcx c (s) clearly includes the! c axis, as do the regions for both G xc (s) and G x c ( s ). If G xc (s) is a canonical response, then kg xc k = R g x c (t), with equality if and only if G xc (s) =, since G xc (s) is monic. Further, the inverse also factorizes similarly into R x cx c (s) =(/S xc,0) G x c (s) G x c ( s ). (A.36) Clearly if R xx (s) is a ratio of finite-degree polynomials in s, then it is factorizable (simply group poles/zeros together for left and right half planes - any on imaginary axis will also appear in conjugate pairs, and so are easily separated). When not already such a polynomial, the next section generalizes this situation through the Paley-Wiener Criterion. A..4 he Paley-Wiener Criterion Minimum-phase signals or filters are of interest in data transmission not only because they are causal and admit causal invertible inverses (one of the reasons for their study more broadly in digital signal processing) but because they allow best results with Decision Feedback as in Section 3.6. hese minimum-phase filters are also useful in noise whitening. Calculation of S x,0 for a factorizable D-ransform follows Equations (3.3) to (3.33) in Section 3.6 as R S x,0 = e ln[rxx(e! )] d! or (A.37) ln (S x,0 ) = Z ln [R xx (e! )] d!, (A.38) which has components corresponding to the last two terms in (A.3) integrating to zero because they are periodic with no constant components and being integrated over one period of the fundamental frequency. he integral in (A.38) must be finite for the exponent in (A.3) to be finite (also, any exponent of a real function is a real positive number, so S x,0 > 0 and real, consistent with the power spectral density). 33

166 hat integral of the natural log of a power-spectral density is fundamental in filter realization and in the Paley Wiener Criterion to follow. Again, D x is the same for R xx (D) as for ln [R xx (D)] and includes the unit circle; this also means the region of convergence for ln [G x (D)] also is the same as for G x (D) and includes the unit circle. Further the region of convergence for Gx (D) also includes the unit circle and is the same as for ln Gx (D). he calculation of Sx,0 has a very similar form to that of S x,0: S x,0 = e ln S x,0 = R Z ln[rxx(e! )] d! or (A.39) ln [R xx (e! )] d!, (A.40) Because (A.38) and (A.34) are similar, just di ering in sign, and because any functions of G x (including in particular ln or ) are all periodic, factorizability also implies Z ln [R xx (e! )] d! < (A.4) (or the function ln [R xx (D)] exists because the autocorrelation in L is absolutely integrable). Essentially the finite nature of this integral corresponding to factorizable R xx (D) means that the sequence s Fourier ransform has no frequencies (except point frequencies of non-zero measure) at which it can be either zero or infinite. he non-infinite is consistent with the basic criterion (A.) to be absolutely integrable, but the the non-zero portion corresponds intuitively to saying any filter that has some kind of dead band is singular. Any energy in these singular bands would be linear combinations of energy at other frequencies. Singular bands thus carry no new information, and can be viewed as useless: A signal with such a dead band is wasting energy on components transmitted already at other frequencies that exactly cancel. A filter with such a dead band would block any information transmitted in that band, making reliable data-detection/communication impossible (kind of an inverse to the reversibility concept and theorem in Chapter ). Such a filter would not be reversible (causally or otherwise). Both situations should be avoided. Chapter 5 will deal with such singularity far more precisely. he following Paley-Wiener heorem from discrete-time spectral factorization theory formalizes when an autocorrelation function is factorizable. he ensuing development will essentially prove the theorem while developing a way to produce the factors G x (D) and thus G x(d ) of the previous subsection. his development also finds a useful way to handle the continuous-time case, which can be useful in noise-whitening. he reader is again reminded that any non-negative (real) spectrum and corresponding inverse transform is a candidate for spectral factorization. heorem A.. (MIMO Paley Wiener Criterion) If R xx (e! ) is any power spectrum such that both R xx (e! ) and ln R xx (e! ) are absolutely integrable over <!apple, and R xx (D) is the corresponding autocorrelation function, then there is a canonical discrete-time response G x (D) that satisfies the equation where the finite constant S x,0 is given by R xx (D) =S x,0 G x (D) G x(d ), ln S x,0 = Z ln R xx (e! )d!. (A.4) (A.43) For S x,0 to be finite, R xx (e! ) must satisfy the discrete-time Paley-Wiener Criterion (PWC) Z ln R xx (e! ) d! <. (A.44) he continuous-time equivalent of this PWC is that the Fourier ransform of the continuoustime autocorrelation function is factorizable R xcx c (s) =S xc,0 G xc (s) G x c ( s ), (A.45) 34

167 where G xc (s) is minimum phase (all poles and zeros in the left half plane or on axis in limiting sense) and monic g xc (t) t=0 =, whenever Z ln R xcx c (! c ) +! c d! c <. (A.46) Constructive Proof: he equivalence of the two PW criteria in (A.44) and (A.46) (discreteand continuous-time) follows directly from Equations (A.8) to (A.3). However, it remains to show that the condition is necessary and su cient for the factorization to exist. he necessity of the criterion followed previously when it was shown that factorizability lead to the PWC being satisfied. he su ciency proof will be constructive from the criterion itself. he desired non-negative real in (A.33) and (A.38) frequency has a (positive or zero) real square root Rxx / (e! ) at each frequency, and this function in turn has a natural log h i A(e! ) =ln Rxx / (e! ). (A.47) A(e! ) itself is periodic and by the PWC integral equation is absolutely integrable and so has a corresponding Fourier representation A(e! ) = a k = X k= Z a k e!k A(e! ) e! d!. (A.48) (A.49) Because the Fourier ransform A(e! ) is purely real, then a k = a k, and the D-ransform simplifies to X X A(D) =a 0 + a k D k + a l D l, (A.50) and then by letting k = k= l in the second sum, k= l= X X A(D) = a 0 + a k D k + a k D k = a 0 + k= (A.5) X [a k + a k ] D k (A.5) k= = a 0 + X < [a k ] D k, (A.53) k= h i which defines a causal sequence a k that corresponds to ln Rxx / (D). he sequence is causally h i invertible because ln Rxx / can be handled in the same way following A.44. So, R xx (D) =e A(D) e A (D ). (A.54) hen, R S x,0 = e ln[rxx(e! )] d! (A.55) G x (D) = ea(d) p Sx,0. (A.56) 35

168 he corresponding continuous-time spectrum factorization then would be found with R xcx c (s) = s s R xx +s and thus A c (s) =A +s.hen,withs!! c S xc,0 = e G xc (s) = eac(s) p Sxc,0 R ln[rxcxc (!c)] +! c d! c (A.57). (A.58) If the original desired spectra were defined in continuous time, then it could be mapped into discrete time through! c! tan(! ) and then proceeding with that discrete-time mapped equivalent through the process above, ultimately leading to Equations (A.5) and (A.53). Su ciency has thus been established in both discrete- and continuous-time. QED. Minimum-phase functions and sequences have several interesting properties that can help understand their utility. If a phasor diagram were drawn from each pole and zero to a point on the unit circle (or imaginary axis for continuous time), the magnitude is the ratio of the zero-phasor-length products to the pole-phasor-length products while the phase is the sum of the zero phases minus the sum of the pole phases. For the D-ransform the phasor angle is measured from a horizontal line to the left (while for Laplace it is measured from a horizontal line to the right). hese phase contributions are always the smallest with respect to the other choice of the zero/pole from the maximum-phase factor. Whence the name minimum phase. Perhaps more importantly, one can see as frequency increases the rate of change of the angle (the magnitude of delay) is smallest for this same choice. Equivalently, each frequency for the particular magnitude spectrum of interest is delayed the smallest possible amount. With respect to all other pole/zero choices, the energy is maximally concentrated towards zero for any length of time (among all waveforms with the same spectrum). In Section 3.6, the feedback section of the DFE thus has largest ratio of first tap to sum of rest of taps, so smallest loss with respect to matched filter bound. Such minimal-energy delay allows inversion of the function with the also-minimum-phase/delay that is actually the negative of the first delay. In e ect, this only occurs when the function is causal and causally invertible. A..5 Linear Prediction he inverse of R xx (D) is also an autocorrelation function and can be factored when R xx (D) also satisfies the PW criterion with finite x,0. In this case, as with the MMSE-DFE in Section 3.6, the inverse autocorrelation factors as Rxx (D) =Sx,0 Ḡ(D) Ḡ (D ), (A.59) where Ḡ(D) =Gx (D). If A(D) is any causal and monic sequence, then A(D) is a strictly causal sequence that may be used as a prediction filter, and the prediction error sequence E(D) is given by E(D) =X(D) X(D) [ A(D)] = X(D) A(D). (A.60) he autocorrelation function of the prediction error sequence is R ee (D) =R xx (D) A(D) A (D )= A(D) A (D ) S x,0 Ḡ(D) Ḡ (D ), (A.6) so its average energy satisfies E e = S 0 k/g ak S 0 (since A(D) is monic), with equality if and only if Ḡ(D) X(D) A(D) is chosen as the whitening filter A(D) =Ḡ(D). he process X(D) Ḡ(D) = G x(d) is often called the innovations of the process X(D), which has mean square value S x,0 =/S 0.hus,S x,0 of the direct spectral factorization is the mean-square value of the innovations process or equivalent of the MMSE in linear prediction. X(D) can be viewed as being generated by inputting a white innovations process U(D) =Ḡ(D) X(D) with mean square value S x,0 into a filter G x (D) so that X(D) =G x (D) U(D). he factorization of the inverse and resultant interpretation of the factor Ḡ(D) as a linear-prediction filter helps develop an interest interpretation of the MMSE-DFE in Section 3.6 where the sequence X(D) there is replaced by the sequence at the output of the MS-WMF. 36

169 A..6 Cholesky Factorization Cholesky factorization is the finite-length equivalent of spectral factorization for a symbol/packet of N samples. In this finite-length case, there are really two factorizations, both of which converge to the infinite-length spectral factorization when the process is stationary and the symbol length becomes large, N!. Cholesky Form - Forward Prediction Cholesky factorization of a positive-definite (nonsingular) 9 N N matrix R xx (N) producesauniqueupper triangular monic (ones along the diagonal) matrix G x (N) and a unique diagonal positive-semidefinite matrix S x (N) of Cholesky factors such that (dots are used in this subsection to help notation, even though they here may correspond to matrix multiplication - it just makes it easier to read here) R xx (N) =G x (N) S x (N) G x(n). (A.6) It is convenient in transmission theory to think of the matrix R xx (N) as an autocorrelation matrix for N samples of some random vector process x k with ordering x N 3 6 X N = 4.. x (A.63) A corresponding order of G x (N) s and S x (N) s elements is then 3 g N s N g N G x (N) = and S 0 s N... 0 x(n) = s 0 g (A.64) Since G x (N) is monic, it is convenient to write g i = 0 N i g i, (A.65) where 0 j in general is a column vector with j zeros in it, and g 0 = ; or g 0 =. he determinant of R N is easily found as S x,0 = R xx (N) = NY n=0 s n. (A.66) (or ln S x,0 =ln R xx (N) for readers taking limits as N!). A convenient recursion description of R xx (N) s components is then (with r n = E x N and r N = E [X n x N ]) R xx (N) = apple rn r N r N R xx (N ). (A.67) he submatrix R xx (N ) also has a Cholesky Decomposition R xx (N ) = G x (N ) S x (N ) G x(n ), (A.68) which because of the 0 entries in the triangular and diagonal matrices shows the recursion inherent in Cholesky decomposition (that is the G x (N ) matrix is the lower right (N ) (N ) submatrix of G x (N), which is also upper triangular). hus, the corresponding recursion description for G x is G x (N) = apple gn 0 N G x (N ). (A.69) 9 his is the equivalent of ln R xx(n) < where R xx(n) is the determinant of R xx(n). hus the determinant cannot be infinite nor can it be zero, eliminating singularity. he determinant is the product of the eigenvalues of R xx so the ln transforms that product into a sum of the log eigenvalues - analogous to the integral in the PWC summing the log transform values. he Fourier transform values are the eigenvalues at each frequency for infinite time extent. 37

170 he inverse of R xx (N) has a Cholesky factorization R xx (N) =G x (N) S x (N) G x (N), (A.70) where Gx (N) is also upper triangular and monic with ordering 3 G x (N) = 6 4 ḡ N ḡ N. ḡ (A.7) Also, because it is monic, linear prediction : g i = 0 N i ḡ i. (A.7) A straightforward algorithm for computing Cholesky factorization derives from a linear-prediction interpretation. he innovations, U N, of the N samples of X N are defined by X N = G x (N) U N (A.73) where E [U N U N ]=S N, and the individual innovations are independent (or uncorrelated if not Gaussian), E u i u j = ij.hen 3 u N 6 U N = 4.. u (A.74) Also, he cross-correlation between X N and U N is which is lower triangular. hus, U N = G x (N) X N. (A.75) Rux = S x (N) G x(n), (A.76) E u k x k i =0 8 i. (A.77) Since X N = G x (N ) U N shows a reversible mapping from U N to X N, then (A.77) relates that the sequence u k is a set of growing-order MMSE prediction errors for x k in terms of x k... x 0 (i.e., (A.77) is the orthogonality principle for linear prediction). hus, u N = x N r R xx (N ) X N (A.78) since r is the cross-correlation between x N and X N of (A.73) and (A.64), rewrites as in (A.67). Equation (A.78), using the top row x N = u N + r Rxx (N ) X N (A.79) = U N + r Gx (N ) Sx (N ) {z } {z} Gx (N ) g N X N (A.80) {z } g N U N = g N U N, (A.8) so g N = r g N S x (N ). (A.8) 38

171 hen, from (A.75), (A.78), and (A.80), ḡ N = g N G x (N ). (A.83) Finally, the mean-square error recursion is s N = E u N u N (A.84) = E x N x N r N Rxx (N ) r N (A.85) = r N g N S x (N ) g N. (A.86) Forward [Upper riangular] Cholesky Algorithm: For nonsingular R N : Set g 0 = G x () = Gx () = ḡ 0 =, S x () = s 0 = E x 0, and r i the last i upper row entries in R xx (i + ) as per (A.67). For n =... N: a. g n = r n Gx (n ) Sx (n ). apple gn b. G x (n) = 0 G x (n ). apple c. Gx gn (n) = Gx (n ) 0 Gx (n ) d. S x (n) =r n g n S x (n ) g n.. A singular R N means that s n = 0 for at least one index n = i, which is equivalent to u i = 0, meaning that x i can be exactly predicted from the samples x i... x 0 or equivalently can be exactly constructed from u i... u 0. In this case, Cholesky factorization is not unique. Chapter 5 introduces a generalized Cholesky factorization for singular situations that essentially corresponds to doing Cholesky factorization for the nonsingular process, and then generating the deterministic parts that are singular and depend entirely on the nonsingular parts from those nonsingular parts. Backward [Lower riangular] Cholesky Algorithm Backward Cholesky essentially corresponds to time-order reversal for the finite group of N samples (for infinite-length sequences, this corresponds to G x (D )). ime reversal is achieved simply by x N J N X N where J N is the N N matrix with ones on the anti-diagonal and zeros everywhere else. Note that JN = J N, and JN = I. For this time reversal, the R xx (N) J N R xx (N) J N (A.87) So the operation in Equation (A.87) is the autocorrelation matrix corresponding to the time reversal of the components. his operation reversal basically flips the matrix about it s antidiagonal 0. For a REAL oeplitz matrix (stationary sequence), this flipping does not change the matrix; however for a complex oeplitz matrix, the new matrix is the conjugate of the original matrix. Further, the operation J N G x (N) J N converts G x (N) from upper triangular to lower triangular, with the ones down the diagonal (monic) retained. he operation J N R xx (N) J N =[J N G x (N) J N ] {z } {z} [J N S x (N) J N ] {z } G x (N) S x(n) {z} [J N G x(n) J N ] {z } G x(n), (A.88) which is the desired lower-diagonal-upper or Backward-Cholesky factorization. hus, the backward algorithm can start with the forward algorithm, and then just use the tilded quantities defined in (A.88) as the backward Cholesky factorization (including G x (N)! J N Gx (N) J N ). 0 Flip is like transpose but around the anti-diagonal. 39

172 Infinite-length convergence Extension to infinite-length stationary sequences takes the limit as N!ineitherforward or backward Cholesky factorization. In this case, the matrix R xx (and therefore G x and S x ) must be nonsingular to satisify the Paley-Weiner Criterion. he equivalence to spectral factorization is evident from the two linear-prediction interpretations for finite-length and infinite length series of samples from random processes earlier in this section. For the stationary case, the concepts of forward and backward prediction are the same so that the backward predictor is just the time reverse (and conjugated when complex) of the coe cients in the forward predictor (and vice versa). his is the equivalent (with re-index of time 0) of G x(d )being the reverse of G x (D) with conjugate coe cients. hus, the inverse autocorrelation function factors as R xx (D) =S x,0 G x (D) G x (D ), (A.89) where Gx (D) is the forward prediction polynomial (and its time reverse specified by G (D )isthe backward prediction polynomial). he series of matrices {R xx (n)} n=: formed from the coe cients of R xx (D) creates a series of linear predictors {G x (N)} N=: with D-transforms G x,n (D). In the limit as N!for a stationary nonsingular series, Similarly, lim G x,n (D) =G x (D). (A.90) N! lim N! G x,n (D) =G x(d ). (A.9) As N!, the prediction-error variances S N, should tend to a constant, namely S x,0. Finally, defining the geometric-average determinants as S x,0 (N) = R xx /N and Sx,0 (N) = Rxx /N n o lim S x,0(n) = S x,0 = e N! lim x,0 (N) = S x,0 = e N! n R ln(rxx(e R ln(rxx(e! ))d! o! ))d! (A.9). (A.93) he convergence to these limits implies that the series of filters converges or that the bottom row (last column) of the Cholesky factors tends to a constant repeated row/column. Interestingly, a Generalized Cholesky factorization of a singular process exists only for finite lengths. Using the modifications to Cholesky factorization suggested above with restarting, it becomes obvious why such a process cannot converge to a constant limit, and so only nonsingular processes are considered in spectral factorization. Factorization of a singular process at infinite-length involves separating that process into a sum of subprocesses, each of which is resampled at a new sampling rate so that the PW criterion (nonsingular needs PW) is satisfied over each of the subbands associated with these processes. his is equivalent to the restarting of Cholesky at infinite length. For more, see Chapter 5. A.3 MIMO Spectral Factorization A.3. ransforms he extension of a D-ransform to an L-dimensional vector sequence x k is X(D) = X x k D k, (A.94) k= H for all scalar complex D Dx. he inverse D-ransform is given by x k = DD x X(D) D k dd. (here is not a separate D value for each dimension of the vector x k - just one scalar D.) When the 330

173 unit circle is in the region of convergence D Dx, then a vector Fourier ransform exists and is X(e! )= X x k e!k. (A.95) k= Su cient conditions for convergence of the Fourier ransform generalize to (with x k = max l x l,k, the maximum absolute value over the L x components of x k ) or X x k <, (A.96) k= X kx k k <. k= (A.97) Vector D-ransforms with poles on the unit circle are handled again in the same limiting sense of approaching arbitrarily closely from outside the circle, which essentially allows generalized functions to be used in the frequency domain, and the vector sequence s Fourier ransform then also exists. he similarity of the form of transform and inverse then allows equivalently that the inverse Fourier ransform ( or if R X (e! ) e!k d!) existsif(where again means maximum component absolute value): Z X(e! )) d! <, (A.98) Z kx(e! ))k d! <. (A.99) Rather, than repeat all for a continuous-time vector function s Laplace ransform, the Vector Laplace ransform for continuous-time vector process x(t) is X(s) = Z x(t) e st dt, (A.00) with convergence region s Sx. When the imaginary axis is in the convergence region Sx, the Fourier ransform is X(!) = Z x(t) e!t dt. Convergence conditions on the vector continuous-time process are then Z (A.0) x(t) dt <, or equivalently (A.0) Z X(!) d! <, (A.03) his means that the vector function x(t) belongs to the space of continuous functions L. Another su cient condition is that vector function x(t) belongs to L or has finite energy according to Z kx(t)k dt <, or equivalently (A.04) Z kx(!)k d! <. (A.05) Similarly generalized functions complete the capability to handle continuous Fourier ransforms that are on the stability margin where values of s in a region of convergence arbitrarily close to the! axis (left of, but not on this axis) will converge so the criteria are considered satisfied in a limiting or generalized sense. 33

174 Similarly a matrix can have a D-ransform (Laplace ransform), which is basically the D-ransform (Laplace ransform) of each of the vector rows/columns stacked in a new matrix. here is only one D variable for all the elements of the matrix. he region of convergence will be the intersection of the individual column vector convergence regions. When that overall region of convergence includes the unit circle (complex axis), the Fourier ransform also exists and the inversion formula are all the obvious extensions. Square matrices can be simplified somewhat in terms of the su ciency criteria in that the Fourier ransform of a square matrix sequence R k exists if X k= R k < (A.06) where the summed entities are the determinants of the sequence s element matrices (another su cient condition could use the trace in place of determinant). Basically the sum of the element norms for transforms (or integral for Laplace) has to be finite for a su cient condition for any valid norm defined on the elements, so in this case the chosen norm is the determinant. he bilinear transform is unchanged with respect to scalar processes. A.3. Autocorrelation and Power Spectra for vector sequences his subsection generalizes factorization of scalar D-ransforms to autocorrelation functions and associated power spectra for stationary and wide-sense stationary vector processes. Definition A.3. (Autocorrelation and Power Spectrum for Vector Sequences) If x k is any stationary complex vector sequence, its autocorrelation matrix is defined as the sequence Rxx(D) whose discrete-time matrix terms are rxx,j = E[x k x k j ];symbolically Rxx(D) = E X(D) X (D ). (A.07) By stationarity, rxx,j = rxx, j and R xx(d) =Rxx(D ).hepower spectrum matrix of a stationary vector sequence is the Fourier transform of its autocorrelation function, which is written as Rxx(e! )=Rxx(D) D=e!, <!apple, (A.08) which has real and nonnegative determinant for all!, namely Rxx(e! ) 0 8! (, ). (A.09) (It is not necessarily true that a non-negative determinant implies an autocorrelation matrix, unlike the scalar case, but conjugate time symmetry of rxx,j = rxx, j does imply autocorrelation for the matrix/vector case.) he use of the determinant both incorporates the autocorrelation vector s positive-matrix spectra, but also condenses the MIMO vector autocorrelation in a single scalar measure that is useful for characterizing MIMO vector communication systems. Rxx(e! ) is real and nonnegative for all!. Conversely, any complex matrix function Rxx(D) for which Rxx(D) =Rxx(D ) is an autocorrelation matrix function, and any function Rxx(e! ) that is real and nonnegative over the interval { <!apple } is a power-spectrum matrix. he quantity E x k is Ex, and can be determined from either the autocorrelation matrix or the power spectrum matrix as follows: Ex = E x k = trace {rxx,0} = Z trace Rxx(e! )d!. (A.0) Anormassignsapositive(orzeroforthezeroelement)realvaluetoeachelementinasequence(function),and satisfies the triangle property x + y apple x + y and for scalar, then x = x. he expression Rxx(D) = E X(D)X (D ) is used in a symbolic sense, since the terms of X(D)X (D )areof the form P k x kx k P j,implyingtheadditionaloperationlim N![/(N +)] on the sum in such terms. NapplekappleN 33

175 If a matrix sequence R k in question is deterministic, then the averages are not necessary above. he power spectra matrix is then the positive semi-definite matrix R k R k for discrete time or R(t)R ) t) for continuous time. hese Fourier ransforms magnitudes can be thought of also as power spectra matrices, and the corresponding inverse transforms as autocorrelation functions in this text. Definition A.3. (Autocorrelation & Power Spectra for Continuous-ime Vector Functions) If x(t) is any stationary (or WSS, see Chapter Appendices) complex vector function, its autocorrelation matrix is defined as the Laplace ransform Rx cx c (s) whose terms are rx cx c (t) =E[x c (u)x c(u t)]; symbolically 3 Rx cx c (s) = E [X(s) X ( s )]. (A.) By stationarity, rx cx c (t) =r x cx c ( t) and Rx cx c (s) =R x cx c ( s ). he power spectrum matrix of a stationary continuous-time vector process is the Fourier transform of its autocorrelation matrix function, which is written as Rx cx c (! c )=Rx cx c (s) s=!c, <! c apple, (A.) which has real and nonnegative determinant for all! c, namely Rx cx c (! c ) 0 <! c <. (A.3) (It is not necessarily true that a non-negative determinant implies an autocorrelation matrix, unlike the scalar case, but conjugate symmetry rx cx c (t) = rxx( t) does imply an autocorellation matrix.) he use of the determinant both incorporates the autocorrelation vector s positive-matrix spectra, but also condenses the MIMO vector autocorrelation in a single scalar measure that is useful for characterizing vector communication systems. Conversely, any complex sequence R(s) for which R(s) =R ( s ) is an autocorrelation matrix function, and any function R(! c ) that is real and nonnegative definite over the interval { <! c apple } is a power-spectrum matrix. he quantity E x c (t) is the power P x and can be determined from either the autocorrelation function or the power spectrum as follows: P x = E x c (t) = trace {rx cx c (0)} = Z trace Rx cx c (! c ) d! c. (A.4) A.3.3 Factorizability for MIMO Processes Matrix factorizability has some flexibility in terms of definition, but this text will combine Cholesky factorization and infinite-length factorization to create a MIMO theory of factorization with unique factors in a form that are useful elsewhere in this text. he spectral factorization of a discrete-time vector autocorrelation function s D-ransform is: Definition A.3.3 (Factorizability for Vector Sequences) An autocorrelation function Rxx(D), or equivalently any non-negative definite real Rxx(e! ) so that rxx,k = r xx, k, will be called factorizable if it can be written in the form Rxx(D) =Gx(D) Sx,0 G x(d ), (A.5) where Sx,0 is a constant diagonal matrix with all positive entries on the diagonal and where Gx(D) is a canonical matrix filter response. A matrix filter response Gx(D) is called canonical if it is causal (Gx,k = 0 for k<0), monic ( diag {Gx(0) = I} ), upper triangular, and minimum-phase (all of its poles and zeros are outside or on the unit circle). If Gx(D) is canonical, then G x(d )isanticanonical; i.e., anticausal, monic, lower triangular, and maximum-phase (all poles and zeros inside or on the unit circle). 3 he R expression Rx cx c (s) = E [X(s)X ( s )] is used in a symbolic sense, since R the terms of X(s)X ( s ) are of the form E[xc(u)x c(u t)]du, implyingtheadditionaloperationlim! [/( )] on the integral in such terms. applevapple 333

176 he region of convergence for a factorizable Rxx(D) clearly includes the unit circle, as do the regions for both Gx(D) and G x(d ). If Gx(D) is a canonical response, and if the squared matrix norm is defined as the sum of the squared eigenvalue magnitudes, then kgxk N with equality if and only if G x (D) =I, sinceg x (D) is monic. Further, the inverse also factorizes similarly into Rxx (D) =Gx (D) S x,0 G x (D ). (A.6) he determinant Rxx(D) will capture all poles and zeros in any and all terms of the factorization (and hidden cancellations will not be important). If R xx (D) is a ratio of finite-degree polynomials in D, then Rxx(D) is factorizable (group poles/zeros together for inside and outside of the circle - any poles/zeros on the unit circle will also appear in conjugate pairs and so are easily separated). Handling of autocorrelation matrices whose determinant is not already such a polynomial ratio appears in the next section through the MIMO Paley-Wiener Criterion. Also, if Rxx(D) is factorizable, then the corresponding Rx cx c (s) =Rxx into s +s is also factorizable Rx cx c (s) =Gx c (s) Sx c G x c ( s ). (A.7) Definition A.3.4 (Factorizability for Continuous Vector Functions) An autocorrelation matrix Rx cx c (s), or equivalently any non-negative definite Rx cx c (!) so that rx cx c (t) = rx cx c ( t), willbecalledfactorizable if it can be written in the form Rx cx c (s) =Gx c (s) Sx c,0 G x c ( s ), (A.8) where Sx c,0 is a finite positive real diagonal matrix and Gx c (s) is a canonical filter response. A filter response Gx c (s) is called canonical if it is causal (Gx c (t) =0for t<0), monic ( diag {Gx c (0)} = I ), upper triangular, and minimum-phase (all of its poles and zeros are in the left half plane). If Gx c (s) is canonical, then G x c ( s )isanticanonical; i.e., anticausal, monic, lower triangular, and maximum-phase (all poles and zeros inside in the right half plane). Further, S x,0 = Sx,0. (A.9) he region of convergence for factorizable Rx cx c (s) clearly includes the! c axis, as do the regions for both Gx c (s) and G x c ( s ). n R o If Gx c (s) is a canonical response, then kgx c k = trace g x c (t) N, with equality if and only if Gx c (s) =I, sincegx c (s) is monic. Further, the inverse also factorizes similarly into R x cx c (s) =G x c (s) S x c,0 G x c ( s ). (A.0) he determinant of Rxx(s) will capture all poles and zeros in any and all terms of the factorization (and hidden cancellations will not be important). Clearly if R xx (s) is a ratio of finite-degree polynomials in s, then it is factorizable (simply group poles/zeros together for left and right half planes - any on imaginary axis will also appear in conjugate pairs so easily separated). When not already such a polynomial, the next section generalizes through the Paley-Wiener Criterion. A.3.4 MIMO Paley Wiener Criterion and Matrix Filter Realization Minimum-phase vector signals and matrix filters are of interest in data transmission not only because they are causal and triangular and admit causal invertible triangular inverses (one of the reasons for their study more broadly in digital signal processing) but because they allow best results with MIMO Decision Feedback as in Section 3.0. hese minimum-phase matrix filters are also useful in noise whitening. 334

177 Analytic Functions of Matrices Analytic functions, like ln(x) and e x as used here, have convergent power-series representations like e x = +x + x + x (A.) = X x m m (A.) m=0 (x ) (x ) ln(x) = (x ) + X ( ) m (x ) m = m m= (x ) (A.3) 3 (A.4) for all values of x. When the argument of the function is a square matrix R, the value of the corresponding output matrix of the same dimension can be found by insertion of this matrix into the power series, so 4 e R = ln(r) = X R m m m=0 X ( ) m (R I) m m= m (A.5). (A.6) With some care on aversion of commuting matrices, the following will hold: Necessity and the Sum-Rate Equivalent ln(r R ) = ln(r )+ln(r ) (A.7) e R+R = e R e R. (A.8) Calculation of Sx,0 and S x,0 = Sx,0 for a factorizable D-ransform autocorrelation matrix generalizes Equations (3.3) to (3.33) in Section 3.6 as Sx,0 = e ln (Sx,0) = S x,0 = e ln (S x,0 ) = R Z R Z ln[r xx (e! )] d! or ln [Rxx(e! )] d! ln R xx (e! ) d! or (A.9) (A.30) (A.3) ln Rxx(e! ) d!, (A.3) which has components corresponding to the first and last terms of the factorization in ( A.5 ) integrating to zero because they are periodic with zero constant term ( see A.3) for ln(x + )) and being integrated over one period of the fundamental frequency. he integral in (A.3) must be finite for the exponent in (A.3) to be finite (also, any exponent of a real function is a real positive number, so S x,0 > 0 and real, consistent with the power spectral density). hat integral of the natural log of a power-spectral density is fundamental in filter realization and in the MIMO Paley Wiener Criterion. Again, Dx is the same for Rxx(D) as for ln [Rxx(D)] and includes the unit circle; this also means the region of convergence for ln [Gx(D)] also is the same as for Gx(D) and includes the unit circle. Further the region of convergence for Gx (D) also includes the unit circle and is the same as for ln Gx (D). 4 For non-square matrices, it is usually possible to achieve desired results by forming a square matrix RR and applying the power series of the function to that squared matrix, and then finding the positive square root through Cholesky Factorization on the result and absorbing the square root of the diagonal matrix of Cholesky factors into each of the triangular matrices. 335

178 he calculation of Sx,0 has a very similar form to that of S x,0: S x,0 = e ln S x,0 = R Z ln Rxx(e! ) d! or (A.33) ln R xx (e! ) d!, (A.34) Because (A.3) and (A.34) are similar, just di ering in sign, and because any functions of G x (including in particular ln or ) are all periodic, factorizability also implies Z ln [R xx (e! )] d! < (A.35) (or the function ln [R xx (D)] exists because this log-autocorrelation s determinant is absolutely integrable). Essentially the finite nature of this integral corresponding to factorizable Rxx(D) means that the sequence s Fourier ransform has no frequencies (except point frequencies of non-zero measure) at which it can be either zero or infinite. he non-infinite nature is consistent with the basic criterion for the norm to be absolutely integrable, but the the non-zero portion corresponds intuitively to saying any non-satisfying matrix filter has some singular null components. Any energy in these null components would be linear combinations of energy of other components. Null components thus carry no new information, and can be viewed as useless: A signal with such a null components is wasting energy on these components that exist elsewhere already. A matrix filter with such a null space would block any information transmitted in that corresponding null space, making reliable data-detection/communication impossible (kind of a MIMO inverse to the reversibility concept and theorem in Chapter ). Such a filter would not be reversible (causally or otherwise). Both infinite size and singular situations should be avoided. Chapter 5 will deal with such singularity and null spaces far more precisely. he following MIMO Paley-Wiener heorem from discrete-time spectral factorization theory formalizes when an autocorrelation function is factorizable. he ensuing development will essentially prove the theorem while developing a way to produce the factors Gx(D) and thus G x(d ) of the previous subsection. his development also finds a useful way to handle the continuous-time case, which can be useful in noise-whitening. he reader is again reminded that any non-negative-definite matrix and corresponding inverse transform is a candidate for spectral factorization. heorem A.3. (Paley Wiener Criterion) If Rxx(e! ) is any power spectrum such that both Rxx(e! ) and ln Rxx(e! ) are absolutely integrable over <!apple, and Rxx(D) is the corresponding autocorrelation matrix, then there is a canonical discrete-time response Gx(D) that satisfies the equation Rxx(D) =Gx(D) Sx,0 G x(d ), (A.36) where the diagonal matrix of all positive elements Sx,0 is given by ln [Sx,0] = ln Sx,0 = Z Z ln Rxx(e! )d!. ln Rxx (e! )d!. (A.37) (A.38) For Sx,0 to be finite, Rxx(e! ) must satisfy the discrete-time MIMO Paley-Wiener Criterion (PWC) S x,0 = Z ln Rxx(e! ) d! <. (A.39) he continuous-time equivalent of this MIMO PWC is that the Fourier ransform of the continuous-time autocorrelation function is factorizable Rx cx c (s) =Gx c (s) Sx c,0 G x c ( s ), (A.40) 336

179 where Gx c (s) is minimum phase (all poles and zeros in the left half plane or on axis in limiting sense), upper triangular, and monic Gx c (s) s=0 =, whenever Z ln Rx cx c (! c ) +! c d! c <. (A.4) Constructive Proof: he equivalence of the two PW criteria in (A.39) and (A.4) (discrete- and continuous-time) follows directly from Equations (A.8) to (A.3). However, it remains to show that the condition is necessary and su cient for the factorization to exist. he necessity of the criterion followed previously when it was shown that factorizability lead to the PWC being satisfied. he su ciency proof will be constructive from the criterion itself. he desired non-negative positive-definite matrix is an upper-triangular positive-definite square root Rxx / (e! ) that can, for instance, be found by Cholesky Factorization (and absorbing positive diagonal square-root equally into the two upper and lower factors), and this function in turn has a natural-log upper-triangular real matrix h i A(e! ) =ln Rxx / (e! ). (A.4) A(e! ) itself is periodic and by the MIMO PWC integral equation is absolutely integrable and so has a corresponding Fourier representation A(e! ) = A k = X k= Z A k e!k A(e! ) e! d!. (A.43) (A.44) Because the Fourier ransform A(e! ) is positive real, then A k = A k, and the D-ransform simplifies to X X A(D) =A 0 + A k D k + A l D l, (A.45) and then by letting k = k= l in the second sum, k= l= X X A(D) = A 0 + A k D k + A k D k = A 0 + k= (A.46) X [A k + A k ] D k (A.47) k= = a 0 + X < [A k ] D k, (A.48) k= h which defines a causal sequence A k that corresponds to ln Rxx i. / (D) A(D) remains upper triangular through the above constructive process. So, Rxx(D) =e A(D) e A (D ). (A.49) hen, R Sx,0 = e ln[r xx (e! )] d! (A.50) Gx(D) = e A(D) S / x,0. (A.5) 337

180 he corresponding continuous-time spectrum factorization then would be found with R xcx c (s) = s s R xx +s and thus A c (s) =A +s.hen,withs!! c R Sx c,0 = e ln[r x cx c (!c)] +! c d! c (A.5) Gx c (s) = e Ac(s) S / x c,0. (A.53) If the original desired spectra were defined in continuous time, then it could be mapped into discrete time through! c! tan(! ) and then proceeding with that discrete-time mapped equivalent through the process above, ultimately leading to Equations (A.5) and (A.53). Su ciency has thus been established in both discrete- and continuous-time. QED. A.4 he Matrix Inversion Lemma he matrix inversion lemma: [A + BCD] = A A B C + DA B DA, (A.54) is often useful in simplifying matrix expressions. his can be proofed through straightforward matrix multiplication. 338

181 Appendix B Equalization for Partial Response Just as practical channels are often not free of intersymbol interference, such channels are rarely equal to some desirable partial-response (or even controlled-isi) polynomial. hus, an equalizer may be used to adjust the channel s shape to that of the desired partial-response polynomial. his equalizer then enables the use of a partial-response sequence detector or a symbol-by-symbol detector at its output (with partial-response precoder). he performance of either of these types of detectors is particularly sensitive to errors in estimating the channel, and so the equalizer can be crucial to achieving the highest levels of performance in the partial-response communication channel. here are a variety of ways in which equalization can be used in conjunction with either sequence detection and/or symbol-by-symbol detection. his section introduces some equalization methods for partial-response signaling. Chapter 9 studies sequence detection so terms related to it like MLSD (maximum likelihood sequence detection) and Viterbi detectors/algorithm are pertinent to readers already familiar with Chapter 9 contents and can be viewed simply as finite-real-time-complexity methods to implement a full maximumlikelihood detector for an ISI channel by other readers of this Appendix. B. Controlled ISI with the DFE Section 3.8. showed that a minimum-phase channel polynomial H(D) can be derived from the feedback section of a DFE. his polynomial is often a good controlled intersymbol interference model of the channel when H(D) has finite degree. When H(D) is of larger degree or infinite degree, the first coe cients of H(D) form a controlled intersymbol interference channel. hus, a detector on the output of the feedforward filter of a DFE can be designed using the controlled ISI polynomial H(D) or approximations to it. B.. ZF-DFE and the Optimum Sequence Detector Section 3. showed that the sampled outputs, y k, of the receiver matched filter form a su cient statistic for the underlying symbol sequence. hus a maximum likelihood (or MAP) detector can be designed that uses the sequence y k to estimate the input symbol sequence x k without performance loss. he feedforward filter /( 0 kpkh (D )) of the ZF-DFE is invertible when Q(D) is factorizable. he reversibility theorem of Chapter then states that a maximum likelihood detector that observes the output of this invertible ZF-DFE-feedforward filter to estimate x k also has no performance loss with respect to optimum. he feedforward filter output has D-transform Z(D) =X(D) H(D)+N 0 (D). (B.) he noise sequence n 0 k is exactly white Gaussian for the ZF-DFE, so the ZF-DFE produces an equivalent channel H(D). If H(D) is of finite degree, then a maximum-likelihood (full sequence of transmitted signals is considered to be a one-shot transmission, which would nominally require infinite delay and complexity for an infinite-length input, but Chapter 9 shows how to implement such a detector recursively 339

182 with finite delay and finite-real-time complexity) detector can be designed based on the controlled-isi polynomial H(D) and this detector has minimum probability of error. Figure B. shows such an optimum receiver. Figure B.: he partial response ZF equalizer, decomposed as the cascade of the WMF (the feedforward section of the ZF-DFE) and the desired partial response channel B(D). When H(D) has larger degree than some desired determined by complexity constraints, then the first feedback taps of H(D) determine a controlled-isi channel. Figure H 0 (D) =+h D h D, (B.) Figure B.: Limiting the number of states with ZF-DFE and MLSD. B. illustrates the use of the ZF-DFE and a sequence detector. he second feedback section contains all the channel coe cients that are not used by the ML (sequence, see Chapter 9) detector. hese coe cients have delay greater than. When this second feedback section has zero coe cients, then the configuration shown in Figure B. is an optimum detector. When the additional feedback section is not zero, then this structure is intermediate in performance between optimum and the ZF-DFE with symbol-by-symbol detection. he inside feedback section is replaced by a modulo symbol-by-symbol detector when precoding is used. Increase of causes the minimum distance to increase, or at worst, remain the same. hus, the ZF-DFE with sequence detector in Figure B. defines a series of increasingly complex receivers whose performance approach optimum as!. A property of a minimum-phase H(D) is that X h i = kh 0 k (B.3) 0 i=0 340