II - Baseband pulse transmission

II - Baseband pulse transmission 1 Introduction We discuss how to transmit digital data symbols, which have to be converted into material form before they are sent or stored. In the sequel, we associate the symbols with pulses, and a sequence of these pulses are added up to form a pulse train that carries the entire message. Let the symbol duration be s. he reciprocal 1/ s is called the baud rate of the transmission. For a symbol representing n bits, the symbol duration and the bit duration are related by s = n b and the bit rate in bits/second is 1/ b = n/ s. If the pulse train itself or a similar waveform is transmitted, the communication system is said to be baseband. he simplest way to generate a pulse train is to assume that the pulses last only as long as the data symbols and do not overlap with succeeding pulses. he shape and relations among pulses are called their format. A format is sometimes called a line code. he next not so-easy way to generate a pulse train is to let the pulses overlap and explore the conditions under which the amplitudes of individual pulses may still be observed from samples of the entire pulse train; these are called Nyquist pulses. Finally, we consider orthogonal pulses which overlap more seriously, but in a way that makes all but one pulse in the train invisible to a properly designed detector. hese last two pulse classes introduce two basic detectors: the sampling receiver and the linear receiver. he former works with Nyquist pulses by simply observing the data symbol values at the right moments in the pulse train. he latter observes the entire signal and depends on the orthogonality property to separate out the data symbols [1]. 2 Power spectral density of PAM signals Consider the pulse amplitude modulation (PAM) signal FDNunes, IS 213. 16

2 POWER SPECRAL DENSIY OF PAM SIGNALS 17 x(t) = k= A k p(t k s ) (1) where A k is a real discrete random variable that models the digital source and p(t) is the signaling pulse. Recall that the power spectrum is defined as G x (f) = lim E{ X (f) 2 } where X (f) is the Fourier transform of the truncated PAM signal x (t) in the interval of duration = (2L + 1) s, i.e., We get L x (t) = A k p(t k s ) k= L G x (f) = P (f) 2 lim L 1 (2L + 1) s L L k= L m= L R A (k m)e j2πf(k m)s where P (f) is the Fourier transform of p(t), R A (k m) = E{A k A m } is the autocorrelation of the time sequence of random variables, A k, assumed to be wide-sense stationary. Finally doing L we obtain G x (f) = P (f) 2 s n= R A (n)e j2πfn s If we consider that the data symbols are uncorrelated, that is we obtain R A (n) = { { A 2 k, n = σ 2 A k A k+n, n = A + m 2 A, n = m 2 A, n G x (f) = σa 2 P (f) 2 + m2 a P (f) 2 s s Using now the Poisson s sum formula n= e j2πfns (2) leads to e ±j2πnf/u = U δ(f mu) n= m= G x (f) = σa 2 P (f) 2 + m2 A s s 2 ( ) m P 2 ( δ f m ) m= s s (3)

2 POWER SPECRAL DENSIY OF PAM SIGNALS 18 he first part of the right-hand member is the continuous spectrum and the second part is the discrete spectrum, which is null if m A = and/or P (m/ s ) =, m [2]. In that case G x (f) = σa 2 P (f) 2 (4) s he previous result can be used to compute the power spectra of line codes. Examples of line codes are shown in Fig. 1 where NRZ stands for nonreturn-to-zero and RZ denotes return-to-zero. bit= bit=1 data = 1 1 1 b b polar NRZ b b polar RZ b b unipolar NRZ b b bipolar RZ b b Manchester Figure 1: Examples of binary line codes based on the square pulse. he pulses corresponding to bits and 1 appear at left In the polar NRZ line code, symbols 1 and are represented by transmitting pulses of amplitudes +A and A, respectively. his code is relatively easy to generate but its disadvantage is that its power spectrum is large near zero frequency. In the unipolar NRZ line code symbol 1 is represented by transmitting a pulse of amplitude A for the duration of the symbol and symbol is represented by switching off the pulse (on-off signaling). Disadvantages of the on-off signaling are the waste of power due to the transmitted DC level and the fact that the power spectrum does not approach zero at zero frequency. he bipolar RZ line code uses three amplitude levels where positive and negative pulses are used alternatively for symbol 1, with each pulse having half-symbol wide, and no pulse is always used for symbol. his line code is also called alternate mark inversion (AMI) signaling. he power spectrum of the transmitted signal has no DC component and small low-frequency components when symbols and 1 occur

2 POWER SPECRAL DENSIY OF PAM SIGNALS 19 with equal probabilities. In the Manchester code symbol 1 is represented by a positive pulse of amplitude A followed by a negative pulse of amplitude A. For symbol, the polarities of these pulses are reversed. he Manchester code suppresses the DC component and has relatively small low-frequency components, regardless of the signal statistics [3]. Example: Determine the power spectral density (PSD) of the Manchester code for independent and equally likely bits and 1. We consider the basic pulse p(t) of Fig. 2. 1 p(t) b t -1 Figure 2: Basic pulse of the Manchester code he digital source is formed by symbols A k = A and A k = A. he pdf of the r.v. A k is with mean and variance p Ak (a) = 1 2 δ(a + A) + 1 δ(a A) 2 m A = A k = ap Ak (a) da = σa 2 = E{(A k m A ) 2 } = he Fourier transform of p(t) is (a m A ) 2 p Ak (a) da = A 2 P (f) = b /2 e j2πft dt = j 2 πf e jπf b sin 2 b b /2 ( πfb 2 e j2πft dt )

3 MACHED FILER 2 leading to where sinc(x) = sin(πx)/(πx) and finally ( ) ( ) P (f) 2 = b 2 sinc 2 fb sin 2 πfb 2 2 ( ) ( ) G x (f) = A 2 b sinc 2 fb sin 2 πfb 2 2 he power spectrum has no discrete part as shown in Fig. 3. Manchester code PSD.25.2 A 2 b.15.1.5 5 2.5 2.5 5 f b Figure 3: Power spectral density of the Manchester code with independent and equally likely symbols his spectrum has a null at frequency zero which may be useful for certain channels, as in digital recording using magnetic tapes. 3 Matched filter A basic problem that often arises is that of detecting a pulse transmitted over a channel that is corrupted by channel noise. Consider the receiver model shown in Fig. 4, involving a linear time-invariant filter of impulse response h(t). he filter input consists of a pulse signal g(t) corrupted by additive channel noise w(t) with x(t) = g(t) + w(t), t

3 MACHED FILER 21 where is an arbitrary observation interval and w(t) is the sample function of a zeromean white noise process with power spectral density G w (t) = N /2. he goal of the receiver is to detect the pulse signal g(t) in an optimum manner, given the received signal x(t). hus, we have to optimize the design of the filter in order to minimize the effects of noise at the filter output in some statistical sense. Since the filter is linear, the output may be expressed as y(t) = g o (t) + n(t) where g o (t) and n(t) are the filter responses to the signal and noise, respectively. he filter output is sampled at the optimal time t = where the peak pulse signal-to-noise ratio η = g o( ) 2 E[n 2 (t)] is maximized. he quantity g o ( ) 2 is the instantaneous power of the output signal and E[n 2 (t)] is the average output noise power. We wish to specify the filter s impulse response h(t) such that the signal-to-noise ratio η is maximized. signal g(t) + linear time-invariant filter of impulse response h(t) y(t) y() sample at time t= white noise w(t) hus Figure 4: Linear receiver he output signal g o (t) may be written as the inverse Fourier transform g o (t) = g o ( ) 2 = H(f)G(f)exp(j2πft) df H(f)G(f) exp(j2πf ) dt and we may re-write the expression for the peak pulse signal-to-noise ratio as η = H(f)G(f) exp(j2πf ) df 2 N (5) 2 H(f) 2 df he problem is to find, for a given G(f), the particular form of the frequency response H(f) that makes maximizes η. o solve this problem we resort to the Schwarz s inequality 2

3 MACHED FILER 22 2 H(f)G(f) exp(j2πf ) df H(f) 2 df G(f) exp(j2πf ) 2 df he equality holds if, and only, if H(f) = kg (f) exp( j2πf ) (6) where k is an arbitrary constant. Replacing (2) in (5) leads to η max = 2 G(f) 2 df = 2E g N N where E g is the energy of signal g(t) and the optimum value of H(f) is H opt (f) = kg (f) exp( j2πf ) he corresponding impulse response of the optimum filter is given by h opt (t) = kg( t) which is, except for the scaling factor k, the time-reversed and delayed version of the input signal g(t); that is, the impulse response is matched to the input signal [3]. Example: Matched filter for rectangular pulse Let g(t) = A ( ) t /2 he matched filter for additive white noise is h opt (t) = kg( t) and the matched filter output for signal g(t) is = ka ( ) /2 t = ka ( ) t /2 g o (t) = g(t) h opt (t) = = ka 2 h opt (λ)g(t λ) dλ ( λ /2 ) ( ) λ t + /2 dλ

3 MACHED FILER 23 or g (t) = ka 2 = ka 2 ( t ( ) λ t + /2 dλ he matched filter output is shown in Fig. 5. In the presence of noise with PSD G w (f) = N /2 the peak signal-to-noise ratio is ) η max = 2E g N = 2A2 N 3.1 integrate-and-dump circuit For the special case of a rectangular pulse, the matched filter can be implemented using an integrate-and-dump circuit, which is shown in Fig. 6. he integrator output is sampled at t =, where is the pulse duration. Immediately after t =, the integrator is restored to its initial condition; hence the name of the circuit. A g(t) t ka 2 g (t) o 2 t Figure 5: Rectangular pulse and matched filter response At the sampling time the integrator output is y( ) = [g(t) + w(t)] dt = A + n where n is a zero-mean random variable (r.v.) given by n = w(t) dt

3 MACHED FILER 24 noise w(t) t= g(t) rectangular pulse + (.)dt y() Figure 6: Integrate-and-dump receiver he variance of n is σ 2 n = E w(λ)w(α) dλdα = E{w(λ)w(α) dλdα yielding = N 2 δ(λ α) dλdα σ 2 n = N 2 hus, at the sampling time t = the signal-to-noise ratio is y 2 ( ) σ 2 n = 2A2 N which is equal to η max. herefore, at the sampling time the matched filter and the integrate-and-dump circuit are equivalent. 3.2 correlator circuit his result can be readily to non-rectangular pulses. In fact, the matched filter can be implemented using a correlator circuit, as shown in Fig. 7, where k is an arbitrary constant. At the sampling time the integrator output is y( ) = k g 2 (t) dt + k g(t)w(t) dt = ke g + n

4 NYQUIS PULSES 25 where n is a zero-mean r.v. with variance σ 2 n = k 2 g(λ)g(α)e{w(λ)w(α)} dλdα = N 2 k2 g 2 (α) dα = N 2 k2 E g and the signal-to-noise ratio at the sampling time is y 2 ( ) σ 2 n = k 2 E 2 g (N /2)k 2 E g = 2E g N which proves that the correlator is equivalent to the matched filter at the optimal sampling time. Note that the integrate-and-dump receiver is a particular case of the correlator receiver where g(t) is a rectangular pulse. noise w(t) t= g(t) pulse + (.)dt y() + kg(t) Figure 7: Correlator receiver 4 Nyquist pulses Although the previous line codes, based on the square pulse, are simple to implement, the small duration of the pulses and their discontinuities cause the line codes to have very large bandwidths. wo ways to reduce the bandwidth of any pulse are to round off its corners and transitions and to increase the pulse duration. Overlapping pulses interfere with each other. However, several kinds of interference still allow an effective detector to be built. he first class of such pulses obeys a zero-crossing criterion called the Nyquist pulse criterion. Let the basic pulse p(t) be centered at time zero. A pulse p(t) satisfies the Nyquist pulse criterion if it passes through at time t = n, n = ±1, ±2,... but not a t = p(n ) = { p(), n =, n (7)

4 NYQUIS PULSES 26 For Nyquist pulses the symbol detector (called sample detector) is quite simple: a sample at time n directly gives the value of the n.th transmission symbol. Consider that the PAM signal y(t) = is sampled at t = n. he result is y(n ) = k= k= A k p(t k ) A k p[(n k) ] = A n p() which is proportional to the transmitted symbol A n. Assume now that the sampling time is affected by a synchronization error ɛ with ɛ <. hen y(n + ɛ) = k= k= A k p[(n k) + ɛ] = A n p(ɛ) + k= k n A k p[(n k) + ɛ] } {{ } ISI he term ISI is the intersymbol interference. We can obtain an equivalent condition to (7) for the elimination of ISI in the frequency domain. Let P (f) be the Fourier transform of p(t). hen, the Poisson sum formula yields n= P ( f n ) = p(m )e j2πmf m= = p() (8) hus, the pulse p(t) satisfies the Nyquist pulse criterion (7) if and only if (Nyquist s first criterion) n= P ( f n ) = p() = constant (9) If P (f) is nonzero outside the Nyquist interval 1/(2 ) f 1/(2 ), many classes of pulses satisfy (9). hus, the Nyquist criterion does not uniquely specify the frequency response P (f). On the contrary, if P (f) is limited to an interval smaller than Nyquist s, it is impossible for (9) to hold [4] and ISI cannot be removed from the received signal.

4 NYQUIS PULSES 27 If P (f) is exactly bandlimited in the Nyquist interval 1/(2 ) f 1/(2 ), (9) requires that P (f) = { p(), f 1/(2 ), elsewhere hat is, the only pulse satisfying the Nyquist criterion is the pulse with rectangular spectrum and in the time domain P (f) = p() ( ) f = constant 1/ ( ) t p(t) = p() sinc here are two serious problems with this solution. First, it is not physically realizable because of its instantaneous jump to at f = ±1/(2 ). he second problem comes from the fact that even small synchronization errors of sampling times t = n would lead to the appearance of ISI. For this reason, it is convenient to use pulses with wider bandwidths to reduce sensitivity to inaccuracies in the sampling times. he most common example in practice is the raised cosine pulse, defined in frequency by, f { [ ( )]} 1 α 2 P (f) = 2 1 cos π α f 1+α 2, 1 α 1+α f < (1) 2 2, f 1+α 2 he spectra of the raised cosine pulse are plotted in Fig. 8 for different values of α. he parameter α is called the rolloff factor or excess bandwidth factor, since the bandwidth of p(t) is (1 + α)/(2 ), while the narrowest possible bandwidth (corresponding to p(t) = sinc(t/ )) is 1/(2 ). In the time domain the pulse is defined by ( ) t cos(απt/ ) p(t) = sinc 1 (2αt/ ) 2 and decays asymptotically as 1/ t 3 for t. he raised cosine pulses are shown in Fig. 9 for different values of α. Consider the PAM signal in (1) where the signalling pulse p(t) is raised cosine and A k = ±1. Assume that x(t) is sampled at t = ɛ, where ɛ is the synchronization error, with ɛ < ( is the symbol duration). We obtain y(ɛ) = y (ɛ) + y ISI (ɛ)

4 NYQUIS PULSES 28 spectrum of the raised cosine pulse, 1.8.6.4.2 α= α=.2 α=.5 α=1. 1.5.5 1 normalized frequency, f Figure 8: Raised cosine spectra for different values of the rolloff factor with and ( ) ɛ cos(απɛ/ ) y (ɛ) = A sinc 1 (2αɛ/ ) 2 y ISI (ɛ) = k= k A k sinc (ɛ/ k) cos (απ (ɛ/ k)) 1 (2α (ɛ/ k)) 2 he signal-to-noise ratio at the sampling time t = ɛ, due to ISI, is SNR(ɛ) = E{y2 (ɛ)} E{y 2 ISI(ɛ)} = sinc2 (ɛ/ ) cos 2 (απɛ/ ) [1 (2αɛ/ ) 2 ] 2 E{y 2 ISI(ɛ)} (11) Fig. 1 shows the signal-to-noise ratios versus the rolloff factor α for different normalized synchronization errors ɛ/. he plots were computed by Monte Carlo simulation assuming that the sequence of symbols {A k } is random. Notice that, for a constant value of ɛ, the signal-to-noise ratio grows with α; thus, the degradation due to ISI increases as α diminishes.

5 ORHOGONAL PULSES 29 1.8 α= α=.2 α=.5 α=1..6 raised cosine pulse, p(t).4.2.2.4 6 4 2 2 4 6 normalized time, t/ Figure 9: Raised cosine pulses for different values of the rolloff factor 5 Orthogonal pulses So far, we have designed a class of pulse trains with relatively narrow bandwidth whose underlying symbols are easy to extract using the Nyquist criterion. he problem is that a simple receiver can have poor error probability in the presence of channel additive noise. he key to solving this problem is to make the pulses orthogonal. A pulse p(t) is orthogonal if p(t)p(t n ) dt =, n = ±1 ± 2,... where is the symbol interval. Consequently, a correlation of the pulse train s(t) = m a m p(t m ) with p(t n ) gives the symbol a n [ ] a m p(t m ) p(t n ) dt = a n p 2 (t n ) dt (12) m where E p is the energy of p(t). = a n E p

5 ORHOGONAL PULSES 3 35 raised cosine pulse with ISI 3 signal to noise ratio, db 25 2 15 ε/=.5 ε/=.1 ε/=.15 1.1.2.3.4.5.6.7.8.9 1 rolloff factor, α Figure 1: Signal-to-noise ratio due to ISI at the sampling time of a raised cosine pulse versus the rolloff factor for different normalized synchronization errors ɛ/ he correlation in (12) may be realized by a simple linear filter. We may rewrite (12) as s(t)p(t n ) dt = s(λ)p[ (n λ)] dt = s(t) p( t) = a n E p t=n hus, the desired correlation is the value of the convolution of the pulse train with a filter with impulse response h r (t) = p( t) at time t = n. We can implement this by applying the train to a filter with transfer function H r (f) = P (f) and sampling the output at time n. he corresponding communication system is sketched in Fig. 11. In the detector, the signal passes through the filter H r (f) and gets sampled at time t = n. If there is no noise, the sample gives the transmission symbol a n, thus eliminating the ISI. Otherwise, the sample is compared to the noise-free symbol values in a threshold (V th ) comparator. his detector circuit is known as linear receiver. Note that the Nyquist s first criterion (equation (9)) has to be verified for the pulse p(t) p( t) whose Fourier transform is P (f)p (f) = P (f) 2. hus, the Nyquist criterion for ISI elimination is now P n= ( f n ) 2 = constant (13) In the case the pulse p(t) is symmetric the spectrum P (f) is real and condition (13)

5 ORHOGONAL PULSES 31 may be written as n= ( P 2 f n ) = constant Probably the most common used pulse in sophisticated systems is the root raisedcosine pulse [1]. he pulse p(t) is symmetric in time, so expression (5) may be applied. he root raised cosine pulse spectrum, P (f), is such that P 2 (f) is expressed by (1). he root raised cosine pulse satisfies the orthogonality constraint and has the same excess bandwidth specified by the rolloff factor α. noise w(t) x(t)= Σ a m δ( t-m) P(f) + s(t)=σa m p(t-m) P *(f) r(t) t=n + - comparator binary decision V th receiver Figure 11: Communication system for orthogonal pulses and linear receiver Consider now a more complex communication system displayed in Fig. 12 and modeled as the cascade of a modulator having the ideal impulse δ(t) as its basic waveform, and of a shaping filter with frequency response S(f). he number of symbols to be transmitted per second is 1/. he channel is represented by a time-invariant linear system having known transfer function C(f) and a generator of additive noise w(t). We aim to design a receiver having the form of a linear filter followed by a sampler. After linear filtering, the received signal is sampled every seconds and the resulting sequence is sent to the detector. he detector makes decisions on a sampleby-sample basis. he design criterion concerns the elimination of ISI regarding the cascade Q(f) = S(f)C(f)U(f), leaving open the choice how to partition the overall transfer function between the transmitter and the receiver, i.e., how to choose S(f) and U(f) once the product S(f)C(f)U(f) has been specified. Note that the pulse corresponding to Q(f) must be a Nyquist pulse. he freedom to chose U(f) permits to impose one further condition, that is, the minimization of the error probability (in fact, in the absence of ISI, errors are caused only by additive noise). he average noise power at the receiving filter output is σ 2 n = G w (f) U(f) 2 df Since the overall channel frequency response is the fixed function Q(f), the signal power spectral density at the shaping filter output is (see equation (4))

5 ORHOGONAL PULSES 32 r(t) x(t) x k source modulator S(f) C(f) + U(f) sampler transmitter A k A k δ(t-k) w(t) channel detector receiver A ^ k Figure 12: ransmission system for linearly modulated data over a time-dispersive channel with sampling receiver and the corresponding signal power is σa 2 S(f) 2 = σ2 A Q(f) 2 C(f)U(f) 2 P = σ2 A Q(f) 2 df (14) C(f)U(f) 2 Minimization of σ 2 n under the constraint (14) can be performed using the Lagrange multipliers. he minimizing U(f) can be shown to be given by U(f) = Q(f) 1/2 and the corresponding shaping filter is obtained through S(f) = G 1/4 w (f) C(f) 1/2 (15) Q(f) C(f)U(f) In (15) and (16) it is assumed that Q(f) = at those frequencies for which the denominators are zero. In the special case of white noise and C(f) constant, we obtain where γ is a nonzero factor. hus U(f) = γ Q(f) 1/2 (16) U(f) = γ S(f) 1/2 U(f) 1/2 or U(f) = (γ ) 2 S(f) where γ is a nonzero factor. So, only one design has to be implemented for the shaping filter and the receiving filter.

6 ZERO-FORCING EQUALIZAION 33 6 Zero-forcing equalization he theory previously developed devoted to the design of an optimum receiver in the presence of channel distortion was based on the assumption of a linear channel and of the exact knowledge of its impulse response (or transfer function). While the former assumption is reasonable in many situations, the latter is often unrealistic. For instance, the channel may be static but selected randomly from an ensemble, as happens with telephone lines. Or, the channel may be vary randomly with time due to fading. hus, the receiver designed to cope with the effects of ISI and additive noise should be selfoptimizing or adaptive. hat is, its parameters should be automatically adjusted to an optimum operating point. wo philosophies can be adopted to design an adaptive receiver. he first, assumes that the relevant channel parameters are first estimated, then fed to a detector which is (approximately) optimum for those parameters. his can be, for example, a Viterbi detector, which requires the knowledge of the channel impulse response samples. he other approach consists of using an equalizer to compensate for the unwanted channel features, and feeds the detector with a sequence of samples that have been cleaned from ISI. Fig. 13 shows a transversal equalizer with 2N + 1 taps and total delay 2ND. he distorted pulse shape p(t) at the input to the equalizer is assumed to have its peak at t = and ISI on both sides. he equalized output pulse will be or doing t = t k (k + N)D p eq (t k ) = p eq (t) = N n= N N n= N c n p(t nd ND) c n p(kd nd) = N n= N c n p k n (17) with p k n = p[(k n)d]. Ideally, we would like the equalizer to eliminate all ISI, resulting in p eq (t k ) = { 1, k =, k But this cannot be achieved, in general, because the 2N + 1 tap gains are the only variables at our disposal. We may instead chose the tap gains such that p eq (t k ) = { 1, k =, k = ±1, ±2,..., ±N (18)

6 ZERO-FORCING EQUALIZAION 34 total delay 2ND p(t) D D... D + + + + c c c c c -N -N+1 -N+2 N-1 N + Σ p (t) eq Figure 13: ransversal equalizer with 2N + 1 taps thereby forcing N zero values on each side of the peak of p eq (t). he corresponding tap gains are computed from (17) and (18) combined in the matrix equation p p 2N c N....... p N 1 p N 1 c 1 p N p N c = 1 p N+1 p N+1 c 1....... p 2N p c N Equation (19) describes a zero-forcing equalizer. he equalized pulse will have a maximum value for k = and is forced to be zero for the N preceding and the N following decision instants, thus the name zero-forcing equalizer. his equalizer removes the ISI in 2N sampling instants. For an ideal equalizer N would have to be very large. Practical equalizers have taps in the range of 3 to several hundreds [5]. his equalization strategy is optimum in the sense that it minimizes the peak intersymbol interference, and it has the added advantage of simplicity [6]. Example: Consider that a three-tap zero forcing equalizer is to be designed for the distorted pulse p(t) plotted in Fig. 14 (solid line). Inserting the values of p k into (19) with N = 1, leads to (19)

7 MEAN SQUARE ERROR MINIMIZAION 35 herefore, 1..1. c 1.2 1..1 c = 1.1.2 1. c 1 c 1 =.96, c =.96, c 1 =.2 he corresponding samples of p eq (t) are plotted in Fig. 14 (dashed line). As expected, there is one zero on each side of the peak. However, zero forcing has produced some small ISI at points further out where the unequalized pulse was zero. 1.2 1 1. 1. p(t) p eq (t).8.6.4.2.1.1.1.6.2.2.2 5 4 3 2 1 1 2 3 4 5 time, D Figure 14: Distorted and equalized pulse he zero-forcing equalizer is relatively easy to implement because it ignores the effect of channel noise w(t). A serious consequence of this simplification is that it leads to overall performance degradation due to noise enhancement at frequencies where the equalizer gain is large. A more refined approach for the receiver design is to use the mean-square error criterion, which provides a balanced solution to the problem of reducing the effects of both channel noise and ISI. 7 Mean square error minimization We have thus far treated the following two channel conditions separately: - Channel noise acting alone, which led to the formulation of the matched filter receiver.

7 MEAN SQUARE ERROR MINIMIZAION 36 - Intersymbol interference (ISI) acting alone, which led to the formulation of the pulse-shaping transmit filter so as to realize the Nyquist channel. In a real-life situation, however, channel noise and ISI act together, affecting the behavior of a data transmission system in a combined manner. In the sequel, consider again the communication system of Fig. 12 and assume that U(f) must be chosen so as to minimize the effects of additive noise at the detector output, and hence to minimize the probability of error for the transmission system under the condition of no ISI. We shall consider the minimum mean-square error (MMSE) criterion for the system optimization; this choice allows ISI and noise to be taken jointly into account, and in most practical situations leads to values of error probability very close to their minimum [4]. Instead of constraining the noiseless samples to be equal to the transmitted symbols in Fig. 12, we can take into account the presence of additive noise and try to minimize the mean-squared difference between the sequence of transmitted symbols {A k } and the sampler outputs {x k }. By allowing for a channel delay of D symbol intervals, we want to determine the shaping filter S(f) and the receiving filter U(f) so that the mean-square value of ɛ k x k A k D is minimized. his will result in a system that, although not specifically designed for optimum error performance, should provide a satisfactory performance even in terms of error probability. For the special case of a channel bandlimited to the Nyquist interval [ 1/(2 ), 1/(2 )], it can be proved [4] that the transfer function of the optimum receiving filter is given by where U opt (f) = σ 2 AP (f) G w (f) + (σ 2 A/ ) P (f) 2 e j2πfd (2) σ 2 A P (f) 2 is the power spectral density of the received digital signal (see equation (4)), D is the channel delay in symbol intervals and P (f) S(f)C(f). Equation (2) shows that, in the absence of noise, the optimum receiving filter is simply the inverse of P (f). his results from having an overall transfer function that is constant in the Nyquist band, then verifying the Nyquist s first criterion for zero ISI. However, when G w (f), elimination of ISI does not provide the best solution. On the contrary, for spectral regions where the denominator of the right-hand side of (2) is dominated by G w (f), U opt (f) approaches the matched filter frequency response P (f)/g w (f).

REFERENCES 37 More generally, for a channel with nonzero transfer function outside the Nyquist interval, the transfer function of the optimum receiving filter is [4], [3] U opt (f) = P (f) Γ(f) (21) G w (f) where Γ(f) is a periodic function with period 1/ given by Γ(f) = σ2 Ae j2πfd 1 + σ 2 AL(f), L(f) = 1 k= P (f + k/ ) 2 G w (f + k/ ) Note that in (21) P (f)/g w (f) is the transfer function of a filter matched to the impulse response p(t) of the cascade of the shaping filter and the channel. Also, Γ(f), being a periodic transfer function with period 1/, can be thought of as the transfer function of a transversal filter whose taps are spaced seconds apart. hus, we can conclude that the optimum receiving filter is the cascade of a matched filter and a transversal equalizer, as shown in Fig. 15. he former reduces the noise effects and the latter reduces ISI. o implement (21) exactly we need an equalizer of infinite length. In practice, we may approximate the optimum solution by using an equalizer with a finite set of coefficients c k, N k N, provided N is large enough [3]. received signal matched filter p(-t) x(t) transversal equalizer... + + + + N + c c c c c -N -N+1 -N+2 N-1 Σ y(t) y(n) Figure 15: Optimum linear receiver consisting of the cascade connection of matched filter and transversal equalizer References [1] John B. Anderson, Digital ransmission Engineering, IEEE Press, N. York, 1999.

REFERENCES 38 [2] Leon W. Couch II, Digital and Analog Communication Systems, Macmillan, N. York, 199. [3] Simon Haykin, Communication Systems, 4.th edition, Wiley, N. York, 21. [4] Sergio Benedetto and Ezio Biglieri, Principles of Digital ransmission with Wireless Applications, Kluwer, N. York, 1999. [5] Kamilo Feher, Digital Communications: microwave applications, Prentice-Hall, Englewood Cliffs, NJ, 1981. [6] A. Bruce Carlson, Communication Systems. An Introduction to Signals and Noise in Electrical Communication, McGraw-Hill, N. York, NJ, 1986.