Performance Analysis and Optimal Filter Design for Sigma-Delta Modulation via Duality with DPCM

Size: px

Start display at page:

Download "Performance Analysis and Optimal Filter Design for Sigma-Delta Modulation via Duality with DPCM"

Justin Johnson
6 years ago
Views:

1 Performace Aalysis ad Optimal Filter esig for Sigma-elta Modulatio via uality with PCM Or Ordetlich Tel Aviv Uiversity Uri Erez Tel Aviv Uiversity Abstract Samplig above the Nyquist-rate is at the heart of sigma-delta modulatio, where the icrease i samplig rate is traslated to a reductio i the overall miimum mea-squarederror recostructio distortio. This is attaied by usig a feedback filter at the ecoder, i cojuctio with a low-pass filter at the decoder. The goal of this work is to characterize the optimal trade-off betwee the per-sample quatizatio rate ad the resultig mea-squared-error distortio, uder various restrictios o the feedback filter. To this ed, we establish a duality relatio betwee the performace of sigma-delta modulatio, ad that of differetial pulse-code modulatio whe applied to discrete-time bad-limited iputs. As the optimal trade-off for the latter scheme is fully uderstood, the full characterizatio for sigma-delta modulatio, as well as the optimal feedback filters, immediately follow. I. INTROUCTION Aalog-to-digital A/ ad digital-to-aalog /A covertors are a itegral part of almost all electric devices i use. Ofte, the same A/ or /A compoet is applied to a variety of sigals with distict characterizatios. For this reaso, it is desirable to desig the data-coverter to be robust to the characteristics of its iput sigal. Oe assumptio that caot be avoided is kowledge of the badwidth of the sigal to be coverted or at least a upper boud o the badwidth, which dictates the miimal samplig rate, accordig to Nyquist s Theorem. Beyod the badwidth, however, oe would like to assume as little as possible about the iput sigal. A reasoable model for the iput sigal is therefore a stochastic oe, where the iput sigal is assumed to be a statioary Gaussia process with a give variace ad a arbitrary ukow power spectral desity PS withi the assumed badwidth, ad zero otherwise. I this paper, we adopt this compoud model which is rich eough to iclude a wide variety of processes. The robustess requiremet from the A/ or /A covertor traslates to requirig that it iduces a small average distortio simultaeously for all processes withi our compoud model. Sigma-delta modulatio is a widely used techique for A/ as well as /A coversio. The mai advatage offered by this type of modulatio is the ability to trade-off the samplig rate ad the umber of bits per sample required The work of O. Ordetlich was supported by the Adams Fellowship Program of the Israel Academy of Scieces ad Humaities, a fellowship from The Yitzhak ad Chaya Weistei Research Istitute for Sigal Processig at Tel Aviv Uiversity ad the Feder Family Award. The work of U. Erez was supported by by the ISF uder Grat 557/3. for achievig a target mea-squared error MSE distortio. The iput to the sigma-delta modulator is a sigal sampled at times the Nyquist rate >. This over-sampled sigal is the quatized usig a R-bit quatizer, where the task of exploitig the beefits of oversamplig is performed by a feedback filter whose role is to push the quatizatio oise ito high frequecies which are evetually filtered out at the receiver, see Figure. Aother techique for compressig sources with memory is differetial pulse-code modulatio PCM. I PCM, a predictio filter is applied to the quatized sigal. The output of this filter is the subtracted from the source ad the result is fed to the quatizer, see Figure 2. At the decoder, the quatized sigal is simply passed through the iverse of the predictio filter. The coectio betwee PCM ad sigma-delta modulatio, as two istaces of predictive codig, was kow from the outset. I fact, both paradigms emerged from two Bell- abs patets authored by C. C. Cutler i 952 ad 954. Nevertheless, the techiques used for the performace aalysis of PCM ad sigma-delta modulatio are quite differet. Oe explaatio for the divergece i the aalysis methods is that PCM was developed as a predictio scheme for a stochastic sigal, whereas sigma-delta modulatio was origially iveted as a oise-shapig techique aimed at achievig a more desirable oise spectrum, rather tha reducig compressio rate. However, through the years the most promiet use of sigma-delta modulatio has become reducig compressio rate at the expese of icreasig samplig rate, as discussed above. Clearly, oe ca pursue the same goal by applyig the PCM scheme to a over-sampled sigal. The performace of PCM uder the assumptio of highresolutio quatizatio is well uderstood sice as early as the mid 60 s. Uder this assumptio, the predictio filter should be chose as the optimal liear miimum mea-squarederror MMSE predictio filter of the source process from its past [], ad the effect of the filtered quatizatio oise ca be eglected i the predictio process. While i most cases where PCM is traditioally used, the high resolutio assumptio is well justified, it totally breaks dow for the class of badlimited processes, which icludes the iput sigals to sigmadelta modulators. Fortuately, the high-resolutio assumptio i PCM aalysis has bee overcome i [2], where it was show that

2 for ay distortio level ad ay statioary Gaussia source, the PCM architecture iduces a rate-distortio optimal test chael, provided that the predictio filter is chose as the optimal filter for predictig the source from its quatized past, ad i additio water-fillig pre- ad post-filters are applied. The aalysis of [2], which takes ito accout the effect of the quatizatio oise, ca therefore be used to obtai the optimal feedback filter ad its correspodig performace for a PCM system applied to a over-sampled statioary Gaussia source. Our mai result, derived i Sectio II, is that for oversampled bad-limited statioary Gaussia processes, the test chael iduced by the sigma-delta modulator Figure achieves precisely the same rate-distortio fuctio as that of the PCM test chael Figure 2 with a Gaussia statioary iput with the same variace, whose spectrum is flat withi the same frequecy bad. More specifically, for such processes, for ay choice of σpcm 2 ad predictio filter CZ i the test chael of Figure 2, the same choice of CZ together with the choice σ 2 = σ 2 PCM / / C 2 d i Figure, yields the same compressio rate ad the same distortio. While this result is simple to derive, it has a very pleasig cosequece: the problem of optimizig the filter CZ i sigma-delta modulatio, uder ay set of costraits, ca be cast as a equivalet problem of optimizig the PCM predictio filter uder the same set of costraits. Usig results from liear time-ivariat predictio theory, we ca the easily fid the optimal filter for sigma-delta modulatio uder costraits for which a explicit solutio was lackig i the literature, or was cumbersome to derive. Fially, i Sectio III we show that the rate-distortio tradeoff derived for sigma-delta modulatio i Sectio II, which is based o aalyzig the test-chael from Figure, remais valid for a sigma-delta modulator with a scalar uiform quatizer of fiite support. Applyig such a scalar quatizer icurs a costat additive rate pealty, whose purpose is to esure that a overload evet, which jeopardizes the stability of the system, occurs with low probability. Our treatmet tackles the issue of stability, which is treated rather heuristically i much of the sigma-delta literature, i a systematic ad rigourous maer, ad the trade-off betwee the rate pealty ad the overload probability is aalytically determied. II. MAIN RESUT For a discrete sigal {c }, the Z-trasform CZ ad the discrete-time Fourier trasform C are defied i the usual maer. For a discrete statioary process {X } with zeromea ad autocorrelatio fuctio R X [k] EX +k X we defie the power-spectral desity PS as the Fourier trasform of the autocorrelatio fuctio S X R X [k]e jk. k= The PS of a cotiuous statioary process is defied i a aalogous maer. Assume X t is a cotiuous statioary bad-limited Gaussia process with zero mea ad variace σx 2, whose PS is zero for all frequecies f > f max, but is otherwise ukow. The Nyquist samplig rate for this process is 2f max samples per secod. Sice our focus here is o quatizatio of over-sampled sigals, we assume that X t is sampled uiformly with rate of 2f max samples per secod for some >. The obtaied sampled process {X } is therefore a discrete statioary Gaussia process with zero mea ad variace σx 2 whose PS is zero for all / [/,/], but is otherwise ukow. Our goal is to characterize the rate-distortio trade-off obtaied by a sigma-delta modulator, modeled as the test chael from Figure, whose iput is {X }. To that ed, we establish a equivalece betwee the performace obtaied by this test chael for ay statioary bad-limited Gaussia process with variace σx 2 ad the performace obtaied by the test chael from Figure 2, which models a PCM compressio system, for a statioary flat bad-limited Gaussia process with variace σx 2. The performace of the latter is ow well uderstood [2], ad, as we shall show, ca be traslated to a simple characterizatio of the sigma-delta modulatio performace. The test chaels i Figure ad Figure 2 model a sigmadelta modulator ad a PCM system, respectively, where i both systems the filter CZ is assumed strictly causal ad the quatizer was replaced by a AWGN chael. We aalyze the distortios attaied by the test chaels ad the scalar mutual iformatio IU ;U + N betwee the iput ad output of the additive white Gaussia oise AWGN chaels embedded withi the two test chaels. The test chaels i Figure ad Figure 2 do ot immediately iduce a output distributio from which a radom quatizatio codebook with rate IU ;U + N ad MSE distortio ca be draw. The reaso for this is the sequetial ature of the compressio, which seems to coflict with the eed of usig highdimesioal quatizers, as required for attaiig a quatizatio error distributed asn with compressio rate IU ;U +N. Fortuately, this difficulty, which is also preset i decisio feedback equalizatio for itersymbol iterferece chaels, ca be overcome with the help of a iterleaver [2] [4] see discussio i [5, Sectio II.B]. Thus, the scalar mutual iformatio IU ;U +N ca ideed be iterpreted as the compressio rate eeded to achieve the distortio attaied by the test chaels i Figure ad Figure 2. Moreover, i Sectio III we show that IU ;U +N is closely related to the required quatizatio rate i a sigma-delta modulator that applies a uiform scalar quatizer of fiite support. The proofs of the followig two propositios are straightforward ad ca be foud i [5]. Propositio : For a Gaussia statioary process {X } with variaceσx 2 whose PS is zero for all / [/,/],

3 X U N N 0,σ 2 U +N CZ N Fig.. The test chael correspodig to the sigma-delta modulatio architecture, with the sigma-delta quatizer replaced by a AWGN chael. X PCM N PCM U PCM N 0,σPCM 2 U PCM +N PCM + CZ V PCM H H Fig. 2. The test chael correspodig to the PCM architecture, with the PCM quatizer replaced by a AWGN chael. ˆX ˆX PCM the test chael from Figure achieves MSE distortio = σ 2 / / C 2 d, ad its scalar mutual iformatio satisfies IU ;U +N = 2 log + C 2 d + σ2 X σ 2. Propositio 2: For a Gaussia statioary process {X PCM } with variace σx 2 ad PS { SX PCM σx 2 for / = 0 for / < <, 2 the test chael from Figure 2 achieves MSE distortio = σ2 PCM ad its scalar mutual iformatio satisfies IU PCM ;U PCM +N PCM = 2 log + C 2 d + σ2 X / σpcm 2 C 2 d. / Remark : I Propositios ad 2 we derived the scalar mutual iformatio betwee the iput ad output of the AWGN test chaels embedded i Figures ad 2, respectively. As will become clear i Sectio III, the scalar mutual iformatio is closely related to the required quatizatio rate whe a scalar memoryless quatizer is used withi the sigmadelta or PCM modulator. I [2], [4], the directed iformatio was show to be related to the required quatizatio rate whe the quatizer is followed by a etropy coder. Here, we do ot cosider applyig etropy codig to the quatizer s output as we require that the desiged modulator be robust to the statistics of the iput process, whereas etropy codig heavily relies o the statistics of the process. Furthermore, etropy codig is udesirable i A/ coverters. All logarithms i this paper are take with base 2. Our mai result ow follows immediately from Propositios ad 2. Theorem : et {X } be a Gaussia statioary process with variaceσx 2 whose PS is zero for all / [/,/], let {X PCM } be a Gaussia statioary process with PS as i 2, ad let CZ be a strictly causal filter. The test chael from Figure with σ 2 = / / C 2 d, ad the test chael from Figure 2 with σpcm 2 = both achieve MSE distortio ad their scalar mutual iformatio satisfy IU ;U +N = 2 log + + σ2 X = IU PCM / ;U PCM C 2 d / +N PCM C 2 d. This theorem idicates that for ay statioary bad-limited Gaussia process with variace σx 2, the sigma-delta test chael from Figure achieves precisely the same ratedistortio trade-off as that of the PCM test chael from Figure 2 with a statioary flat bad-limited Gaussia iput with the same variace, provided that the AWGN variaces are scaled accordig to. Thus, Theorem provides a uified framework for aalyzig the performace of sigmadelta modulatio ad PCM. A great advatage afforded by such a uified framework, is that ay result kow for PCM ca be traslated to a correspodig result for sigma-delta modulatio, ad vice versa. Theorems 2 ad 3 below costitute two importat examples of such results. Theorem 2: et {X } be a Gaussia statioary process with variace σx 2 whose PS is zero for all / [/,/] ad let C be a family of strictly causal filters. efie the virtual process {S } as a Gaussia statioary process with PS as i 2, ad the virtual process {W } as a Gaussia i.i.d. radom process statistically idepedet of {S } with

4 variace, > 0. et C σ 2 = mi CZ C ES c S +W 2 Z = argmies c S +W 2. CZ C If the filter CZ i the sigma-delta test chael from Figure belogs to C ad the MSE distortio attaied by this test chael is, the IU ;U +N 2 log + σ 2, 3 with equality if CZ = C Z. Theorem 2 states that for a target distortio, the sigmadelta filter which miimizes the required compressio rate is the optimal liear time-ivariat MMSE estimator, withi the class of costraits C, for S from the past of the oisy process {S +W }. For example, if C cosists of all strictly causal fiite-impulse respose FIR filters of legth p, the optimal filter CZ is the optimal predictor of S from the samples {S + W,...,S p + W p }, which ca be easily calculated i closed-form. The optimal sigma-delta filter desig problem was studied by several authors, uder various assumptios [6] [9]. However, to the best of our kowledge, the simple expressio from Theorem 2 for the optimal filter as the optimal predictor of S from the past of {S +W } is ovel. Proof of Theorem 2. By Propositio, if the test chael from Figure achieves MSE distortio, we must have σ 2 = / / C 2 d. By Theorem, the correspodig mutual iformatio IU ;U + N is equal to the mutual iformatio IU PCM ;U PCM +N PCM i the PCM test chael from Figure 2 with X PCM = S ad N PCM = W. It is show i [2], [5] that IU PCM ;U PCM +N PCM = PCM 2 log EU 2 + ad that U PCM Therefore, we have = X PCM c X PCM σ 2 PCM +N PCM. IU ;U +N = 2 log + ES c S +W 2. 4 It follows that amog all filters i C, the filter that miimizes 4 is C Z, ad that it attais 3 with equality. It is iterestig to ote [2] that sice {W } is a i.i.d. process with variace ad CZ is strictly causal, the mutual iformatio 4 ca also be writte as IU ;U +N = 2 log ES +W c S +W 2. 5 Thus, the optimal predictor of S from the past of {S +W } is idetical to the optimal predictor of S +W from its past samples. Whe CZ is take as the uique ifiite order optimal oe-step predictio filter of S + W from its past samples, the predictio error variace is the etropy power of the process {S +W } [0], which equals 2 / log +SSd = + σ2 X. 6 Moreover, the ifiite order predictio error E pred S +W c S +W is i this case a white process. This, together with 6 implies that for the optimal ucostraied sigma-delta filter CZ we must have S E pred C 2 +S S / = + σ2 X, [, 7 Combiig 5, 6, ad 7 yields the followig theorem. Theorem 3: et {X } be a Gaussia statioary process with variaceσx 2 whose PS is zero for all / [/,/]. If the test chael from Figure attais MSE distortio, the IU ;U +N 2 log + σ2 X. 8 with equality if ad oly if CZ is a strictly causal filter satisfyig 2 / + σ2 X C 2 [ =, ] / 9 + σ2 X / [, ], ad σ 2 = / / C 2 d = /. + σ2 X Remark 2: The output of the test chael from Figure as well as that from Figure 2 is of the form ˆX = X + E, where E has zero mea ad variace, ad is statistically idepedet of X. This estimate ca be further improved by applyig scalar MMSE estimatio for X from ˆX. I this case the mutual iformatio from 8 is further reduced to 2 log σ 2 X which is the optimal ratedistortio fuctio for a statioary Gaussia source {X } with PS as i 2. It follows that the sigma-delta test chael from Figure with CZ ad σ 2 as specified i Theorem 3 is miimax optimal for the class of all statioary Gaussia sources with variace σx 2 ad PS that equals zero for all / [/,/], i.e., o other system ca achieve MSE distortio with a smaller compressio rate, uiversally for all sources i this class. 2 The existece of a strictly causal filter CZ which satisfies 9 is guarateed by Wieer s spectral-factorizatio theorem.

5 X CZ N U Z + Q R,σ 2 U +N H ˆX Fig. 3. A sigma-delta modulator with a dithered scalar uiform quatizer. III. SIGMA-ETA MOUATION WITH A SCAAR UNIFORM QUANTIZER The sigma-delta modulatio architecture is maily used for A/ ad /A coversio. I such applicatios, vector quatizatio is typically out of the questio ad simple scalar quatizers of fiite support are used istead. For such quatizers, the quatizatio error is composed of two mai factors []: graular errors that correspods to the quatizatio error i the case where the iput sigal falls withi the quatizer s support, ad overload errors that correspod to the case where the iput sigal falls outside the quatizer s support. ue to the feedback loop, iheret to the sigma-delta modulator, errors of the latter kid, whose magitude is ot bouded, may have a disastrous effect as they jeopardize the system s stability. I order to avoid such errors, the support of the quatizer has to be large eough, which traslates to a costrait o the quatizer rate. We shall show that, give that overload errors did ot occur, the quatizatio oise ca be modeled as a additive oise. Thus, the test chael from Figure accurately predicts the total distortio icurred by a sigma-delta A/ or /A i this case. Further, the overload probability ca be cotrolled by takig the quatizatio rate greater thaiu ;U +N. et Q R,σ 2 be a uiform mid-riser quatizer [] with quatizatio step 2σ 2 ad 2 R quatizatio levels, such that the quatizer support is [Γ/2,Γ/2, where Γ 2 R 2σ 2. Our goal is to aalyze the distortio ad overload probability P ol attaied by a sigma-delta modulator that uses a Q R,σ 2 quatizer, as a fuctio of R ad σ 2. Clearly, if we employ the scalar sigma-delta modulator o a log eough iput sequece, a overload evet will evetually occur. As discussed above, the effects of overload errors ca be amplified due to the feedback loop, ad i this case the average MSE may sigificatly grow. We therefore split the iput sequece ito fiite blocks of legthn, ad iitialize the memory of the filter CZ with zeros before the begiig of each ew block. This makes sure that the effect of a overload error i the origial system is restricted to the block where it occurs. The aalysis is made much simpler by itroducig a subtractive dither []. Namely, let {Z } be a sequece of i.i.d. radom variables uiformly distributed over the iterval [ 2σ 2 /2, 2σ 2 /2. I order to quatize a real umber U, we add Z to it before applyig the quatizer, ad subtract Z afterwards, such that the obtaied result is Q R,σ 2 U +Z Z. The followig theorem, whose proof ca be foud i [5], characterizes the trade-off betwee the distortio, quatizatio rate ad overload probability achieved by the scalar sigmadelta modulator depicted i Figure 3 i terms of the scalar mutual iformatio betwee the iput ad output of the AWGN chael from Figure. Theorem 4: et be the MSE distortio attaied by the test chael i Figure with a filter CZ of fiite legth, ad IU ;U +N the scalar mutual iformatio betwee the iput ad output of the AWGN chael i the same figure. For ay 0 < P ol < the scalar sigma-delta modulator from Figure 3 applied o a sequece of N cosecutive source samples with quatizatio rater = IU ;U +N +δp ol attais MSE distortio smaller tha +o N, P ol with probability greater tha P ol, where o N 0 as N icreases, ad δp ol 2 log 2 3 l P ol. 2N REFERENCES [] N. S. Jayat ad P. Noll, igital codig of waveforms: priciples ad applicatios to speech ad video. Pretice-Hall, 984. [2] R. Zamir, Y. Kochma, ad U. Erez, Achievig the Gaussia ratedistortio fuctio by predictio, IEEE Tras. If. Theory, vol. 54, o. 7, pp , July [3] T. Guess ad M. K. Varaasi, A iformatio-theoretic framework for derivig caoical decisio-feedback receivers i Gaussia chaels, IEEE Tras. If. Theory, vol. IT-5, pp , Ja [4] J. Østergaard ad R. Zamir, Multiple-descriptio codig by dithered delta-sigma quatizatio, IEEE Tras. If. Theory, vol. 55, o. 0, pp , Oct [5] O. Ordetlich ad U. Erez, Performace aalysis ad optimal filter desig for sigma-delta modulatio via duality with PCM, 205. [Olie]. Available: [6] H. Spag III ad P. Schultheiss, Reductio of quatizig oise by use of feedback, IRE Tra. Comm. Systems, vol. 0, o. 4, pp , ec 962. [7] P. Noll, O predictive quatizig schemes, The Bell System Techical Joural, vol. 57, o. 5, pp , May 978. [8] M. erpich, E. Silva,. Quevedo, ad G. Goodwi, O optimal perfect recostructio feedback quatizers, IEEE Tras. Sigal Processig, vol. 56, o. 8, pp , Aug [9] M. erpich ad J. Østergaard, Improved upper bouds to the causal quadratic rate-distortio fuctio for Gaussia statioary sources, IEEE Tras. If. Theory, vol. 58, o. 5, pp , May 202. [0] T. Berger, Rate distortio theory: A mathematical basis for data compressio. Pretice-Hall, 97. [] R. Zamir, attice Codig for Sigals ad Networks. Cambridge Uiversity Press, 204.

Performance Analysis and Optimal Filter Design for Sigma-Delta Modulation via Duality with DPCM

Performance Analysis and Optimal Filter Design for Sigma-Delta Modulation via Duality with DPCM Performace Aalysis ad Optimal Filter Desig for Sigma-Delta Modulatio via Duality with Or Ordetlich ad Uri Erez, Member, IEEE Abstract Samplig above the Nyquist rate is at the heart of sigma-delta modulatio,