Series Expansion of Wide-Sense Stationary Random Processes

Transcription

1 792 IEEE TRANSACTIONS ON INFORMATION THEORY. VOL. 14, NO. 6, NOVEMUER 1968 Serie Expanion of Wide-Sene Stationary Ro Procee ELIAS MASRY, MEMBER, IEEE, BEDE LIU, MEMBER, IEEE, AND KENNETH STEIGLITZ, MEMBER, IEEE Abfracf-Thi paper preent a general approach to the derivation of erie expanion of econd-order wide-ene tationary ean-quare continuou ro proce valid over an infinite-tie interval. The coefficient of the expanion are orthogonal convergence i in the ean-quare ene. The ethod of derivation i baed on the integral repreentation of uch procee. It cover both the periodic the aperiodic cae. A contructive procedure i preented to obtain an explicit expanion for a given pectral ditribution. I. INTRODUCTION ERIES expanion of ro procee are ueful in variou area of counication theory alo provide oe inight into the tructure of ro procee. Quite generally we write x(t) = c r&&(t) (1) the convergence i uually taken to be in the tochatic ean. Variou contraint ay be ipoed on x(t), p,(t), the coefficient r,. In the failiar Karhunen-Loeve expanion, {p,(t) ) i a et of quare integrable orthogonal function t,he coefficient r,, are orthogonal. It i known that +~~(t) i a olution of the linear integral equation 1 P(t) = x Wt, -T MT) dt; Itl < T (2) R(t, 7) i the covariance function of x(t). The Karhunen-Loeve expanion ay fail when the interval (- T, T) i infinite if the kernel R(t, T) doe not behave properly. For exaple, for R(t, T) = e- - the pectru for the integral equation T p(t) = X e- - (p(t) --T dt (3) i not dicrete [l]. In fact, eiel i a olution for the eigenvalue X = fr (1 + (Y ). Thu every X > 3 i an eigenvalue. Expanion other than the Karhunen-Loeve are alo known under different auption. If the proce i bliited, we have the well-known apling theore Manucript received Noveber 13, 1967; revied April 4, Thi work wa upported in part by the National Science Foundation under Grant GK-1439 by the Ary Reearch Office, Durha, N. C., under Contract DA ARO-D-292. E. Mary wa with the Departent of Electrical Engineering, Princeton Univerity, Princeton, N. J. He i now with the Departent of Applied Electrophyic, Univerity of California, La Jolla, Calif. B. Liu K. Steiglitz are with the Departent of Electrical Engineering, Princeton Univerity, Princeton, N. J. x(t) = 2 x(nl ), ;,~ ; - e WC the pectral denity (X) vanihe outide of the interval [- w,, 0~1. The coefficient x(n T) are orthogonal if only if the (X) i contant over [-w,, w,]. Papouli [2] derived an intereting expanion for nonperiodic procee with the p*&(t) being trigonoetric function the r, orthogonal. Recently Capbell [3] gave an expanion for ro procee whoe pectral denitie vanih outide of the interval [-a, a]. It include the apling theore a a pecial cae. Thi paper preent a general approach to the derivation of erie expanion of econd-order wide-ene tationary ean-quare continuou ro proce valid over an infinite-tie interval. The coefficient (r,) will be orthogonal convergence will be in the ean-quare ene. The ethod of derivation i baed on the integral repreentation of uch procee. It cover both the periodic aperiodic cae. A contructive procedure i preented to obtain an explicit expanion for a given pectral ditribution. Capbell reult, which wa derived in a different anner, ay be regarded a a pecial cae of our expanion. II. GENERAL DERIVATION OFTHEEXPANSION Let x(t, w), t E R, be a econd-order wide-ene tationary ean-quare continuou ro proce with autocorrelation function R(T). Then [4] x(t, w) adit the pectral repreentation x(t, w) = eith d{ (X, w), - E Idi- (A, w> I:-= ds 04 (5) if only if (4) R(t) = [I eith ds (A). (6) Let H(z) be the Hilbert pace panned by the et of ro variable {xt(w); t E RI) with inner product defined by (E(w), V(W)) = E{~v*}, t, rl E H(x). Furtherore, let L2 = L (dx (A); - a, a) denote the pace of all coplexvalued function f(x) atifying i Fro the publihed abtract of Capbell paper [3], it appeared that the expanion i for bliited procee only. However, upon receiving hi full anucript, it becae clear that the interval [-a, a] need not be finite.

2 MASRY et al. : WIDE-SENSE STATIONARY RANDOM PROCESSES It can be hown that [4] the Hilbert pace H(x) L2 are ioorphic. For each t, ei x(t, w) are correponding eleent in L2 H(x), repectively. Theoyenz 2 Let x(t, W) be a econd-order wide-ene tationary ean-quare continuou ro proce with autocorrelation function R(7). Let the et of function (f%(x)] be orthonoral coplete in L. Then x(t, w) adit an orthonoral erie expanion x(4 u) = C an(tbn(w) (7) xv-here r,(w) = fnod ds- 0, w); - E[r&] = Bnk (8) 40 = j-1 e /:(A) dx (A). (9) ProoJ: The pace H(z) i eparable ince the proce i ean-quare continuou [4]. Hence by the ioorphi, L i eparable. For each t, the function eitx adit the erie expanion e itx = c a&m~> a,,(t) = S_:, e fn*(a) ds (A) dx (A) = C ja,(t)]. - The ioorphi between L H(a) then iplie (10) (11) x(4 w> = C a,(t)(~) (12) cc r,(w) = fnw &- (A, ~1 - (13) the et (Y,(W) ) i orthonoral coplete in H(x). Q.E.D. Define the proce y(t, a) a a linear tranforation 44 ~1 by dt, ~1 = [l ei f&> di+ (A, ~1. (14) The ro variable T,(W) can then be written a r,(w) = Y,(O, u> (15) which ay be interpreted forally a the output at tie t = 0 of a linear yte with tranfer function f%(x) ubjected to the input x(t, u). 2 By an ioorphi we ean a one-to-one onto inner product preerving tranforation. of 111. EXPANSION IN THE REPROINJCING KERNEL RILBEIW SPICE For every non-negative-definite kernel K(t, ) defined on R X R, there exit a reproducing kernel Hilbert pace H(K) uch that K(., ) E H(K); for all E R ; (16.1) For each f E EI(K), we have the inner product 793 (fc.1, KC., )) = I(). (16.2) It follow that the et of function (K(., ); E R } pan H(K). Denote by H(R*) the reproducing kernel Hilbert pace with the reproducing kernel R*(t, T). Then H(R*) i ioorphic to H(z) [5] with R*(., 7) E H(R*) ~(7, w) E W(x) correponding eleent for each 7. It can be hown fro thi ioorphi (16.1) that the et of function (a*,(t) } i orthonoral coplete in H(R*). Hence R(t, T) adit a erie repreentation By putting R*(t - T) = c an(t)a (17.1) 7 = 0 in (17.1) we have R(t) = C a*,(o)a,(o. IV. DERIVATION OF COMPLETE ORTTIOSORMM, SETS IN L2 Fro the reult of previou ection, we can obtain an explicit erie expanion of a ro proce provided an orthonoral bai in L2 can be contructed for the given pectral ditribution. Let the pectral ditribution X(h) be noralized o that X(X+) = S(X). By the pectral decopoition of S(X) SW = SlO) + X20) X,(h) i a tep function which include all the jup of S(X) X,(X) i the continuou part that i the u of the abolutely continuou coponent the continuou ingular coponent of X(X). Correpondingly, there exit a decopoition of the proce x(t, w) into the utually orthogonal procee x(t, cd) = Xl& cd) + x2(t) w). The repreentation of x1(& 0) by an orthonoral erie can be carried out a follow. Denote the point of jup of X(X) by {A,} let elk = X(X,) - X(X,-). Then [6] -3 1 T = a* 1.i.. -- x(t, u)eeithr dt. T-e 2T --T (18) (19) We conider now the expanion of the proce x2(t, w). If X,(X) i contant over a eaurable ubet A C R, we can define the Hilbert pace, ioorphic to H(x2), a L (dx (A); R - A). Let E be the et over which X,(X) i trictly increaing.

3 794 IEEE TRANSACTIONS ON INFORMATION THEORY, NOVEMBER 1968 Theore 2? Let the pectral ditribution X,(X) of the proce x2(4 w) be continuou trictly increaing over et E. Then the et of function defined by fn(x) = $==== einfcx ; A d& (0) F(X) = 2a inf BR ; 2 n = 0, fl, f2,... (20) R, = [ ds, (0) (21) E i orthonoral coplete in L (ds, (X); E). Proof: For orthonorality, let y = F(X). Then L fn@)fz(x) ds, (A) = $ 1 eicnvk) dy = L. (2% For copletene, let f(x) be an arbitrary function in L2(dSz (X); E) fad - conider -& the error c C.e. ~2 ds, (X). (23) Let y = F(X). Then there exit an invere function X = F- (y), which i alo continuou trictly increaing. Subtituting in (23) reult in 1 2n 1 dz f@ - (y)) - c CneinV12 5, dy. (24) Since f(f- (y)) belong to L2(dy; 0, 2~) the et of function {ein j i coplete in L (dy; 0, 27r), the integral (24) can be ade a all a deired by proper choice of C, (ee alo reference [3] [7]). Q.E.D. For bliited procee, we ay ue a f=(x) the et of function obtained by orthonoralizing the baic polynoial 1, X, h2,.... It i known [8] that thee are coplete in L (ds (X); a, b). We illutrate A. The Periodic Cae V. EXAMPLES the theore with oe exaple. Let the proce x(t, w) be periodic with period 2a. The pectral ditribution S(X) i then given by the tep function S(X) = c a,. VL<lXl The et of function defined by (25) i orthonoral coplete in L (ds (A); -, a). Hence, by theore 1, x(t, w) adit an orthonoral erie expanion (26) T,(W) i given by the tochatic integral lr r,(w) = x(t) $$ I --li R dt a,(t) i given by u,(t) = cr;+eint. Therefore expanion (26) can be written x(t, w) = C c&,(w)eint. n=-co The expanion of the autocorrelation by in the for R(t) i clearly given (28) In the following, x(t, w) i aued to be real with pectral denity (X). B. Low-Pa Procee The et {fn(x) ) can be taken to be the polynoial obtained by orthonoralixing 1, X, X2,. 1. with repect to the weight function (X). For the particular cae (X) = 4; 1x1 I 1 10; otherwie f%(x) i the nth Legendre polynoial fro (9) u,(t) = drr(n + 3) (-i)tijn+$(t) (29) n = O,l,.a. (30) J (x) i the vth order Beel function of the firt kind. The ro variable T,(W) i given by the tochatic integral r74-4 = x(t, w)u,(t) dt. - (31) It i eay to verify that integral (31) converge in the ean to the ro variable defined by (8). Note that the apling repreentation for the above bliited proce can be obtained fro theore 1 with f%(h) = einrx; n = 0, fl, ~2,.... C. Nonbliited Procee Thi i the ore intereting Define (X) = e-l. cae. Aue firt 5 -j$ [e- X ]u(x) ; 7220 f&9 =. (32) 1 f-n-1(- A) ; n < 0.

4 emasry et d. : WIDE-SENSE STATIONARY RANDOM PROCESSES 795 a the nth Laguerre polynoial. The a,(t) can be calculated eaily. _I. (4) (1- ityl n>o dt) = (33). (it)- - n < 0. I (1 + it>-* Since f%(x) i a polynoial, r,(o) can be written a a linear cobination of the derivative of (t, w). T,(W) = 2 Cn,kXCk)(O, cd). (34) k=o Again, (34) i equal to Y,,(W) defined by (8) in the ean. The exitence of the derivative i guaranteed by the analyticity of R(7). A a lat exaple, uppoe R(T) = 8 ; Then by theore 2 we obtain (X) = &. L(X) = (-l) (p$ n = 0, Al,... a,(t) i given by --Iti. e, n=o i (35) (36) (37) VI. THE DISCRETE CASE Let (Z,,(W)} be a econd-order wide-ene tationary dicrete-tie ro proce. Then (Z,(W)) adit the pectral repreentation T z,(w) = eina d( (X, w), -?r if only if E Id{ (A, w) j2 = dp (A) ; n = 0, fl, =t2,... R(n) = _: einx dp (X) R(n) i the nth correlation coefficient P(h) i the pectral ditribution of the proce. It i evident that the Hilbert pace H(x) panned by the proce i eparable o i the ioorphic pace L (dp (A); --a, T). Let the et of function (Hk(X) )k,- be orthonoral coplete in L, (dp (A); -in, r). Then the proce (X,(W) ) adit an orthonoral erie expanion x?b) = c %I,kTk(4 (40) k %ak = T einxh$ (A) dp (A) (41) --r LLal (x) i the kth generalized Laguerre polynoial of order a! [9]. Note that a,(t) i bounded by one, uniforly continuou, belong to L (dt; - ~0, a). However, the function a,(t) are not orthonoral. The ro variable r,(o) i given by the tochatic integral 40,4; n=o T,(W) =.,x(0, I W) - 2 lrn x(t, w)e-tll?l(2t) dt; n21 0 x( - t, w)eetl1y-,(2t) dt; n 5-1. (38) It i een that for n 2 1, T,(W) i evaluated by a linear tranforation of the future of the proce ~(1, w). The convere i true for n 5-1. Uing the fact that u,(t), n # 0 i a one-ided function we can write 4t, w) = (40, w)e- u(t) + g r,,(~)u,(t)) (3% the ter in the firt bracket repreent x(t, w) for 2 0 thoe in the econd bracket repreent z(t, w) for t < 0. Expreion (35) i equal to r,(w) defined by (S) in the ean. Thi exaple can alo be derived fro Capbell reult [3]. A in the continuou-tie cae, the ro variable r,(w) can be interpreted forally a the output at intant k = 0 of a linear tie-invariant digital filter with x- tranfor H,(X), z = eix ubjected to the input (x,(w) ). Given any pectral ditribution P(A), it i known [lo] that the et (ei x)~~-_ i cloed in L (dp (X); -T, n). Orthonoralizing thi et reult in a coplete orthonora1 yte (H,(h)} in L (dp (X); -g,?r), which ay be ued in (40) (41) for the expanion. Suppoe the equence of ro variable (x,(w)) i orthonoralized, o that Tk(4 = c bk,iq(4 (43) i or, in atrix for, r = Bx. The invere relationhip i, forally, x = B- r. (44) If the gra-schidt procedure i applied to the equence {x,(w)} the et {ei }~~-, in the ae anner, then the reulting expanion given by (40) (44) can be hown to be identical. Thu for the orthonoral et obtained fro ( eina ),y=+ expanion (40) ay be regarded a a ethod of inverting the atrix B. A oewhat different approach to generating {H,(X) ) i to orthonoralize the polynoial 1, X, X2,. *.. The reulting et i coplete orthonoral in L (dp (A); -r, 7r).

5 796 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 14, NO. 6, NOVEMBER 1968 ACKNOWLEDGMENT The author wih to expre their thank to the reviewer whoe uggetion have iproved the original anucript. REFERENCES [I] R. Courant D. Hilbert, Method of Matheatical Phyic, vol. 1. New York: Intercience, [2] A. Papouli, Probability, Ro Variable, Stochatic Procee. New York: McGraw-Hill, 19G5, p [3] L. L. Capbell, A erie expanion for ro procee," IEEE Tran. Inforation Theory (Ablracl), vol. IT-12, p. 271, April [4] H. Craer M. R. Leadbetter, Stationary Related Stochatic Procee. New York: Wiley, [5] E. Parzen, Stati&ical inference on tie erie by Hilbert pace ethod, I? Applied Matheatic Statitic Lab., y;anford. I Uverlty, Stanford, Calif., Tech. Rept. 23, January [6] J. L. Doob, Stochatic Procee. New York: Wiley, [7] E. Mary, K. St,eiglitz, B. Liu, Bae in Hilbert pace related to the repreentation of tationary operator, J. SIAM, vol. 16, pp , May [8] N. I. Achieer I. M. G&an, Theory of Linear Operator in Hilbert Space, vol. 1. New York: Ungar, 1961, ec. 12. [9] Erdelyi, Higher Trancendental Function, vol. 2. New York: McGraw-Hill, [lo] N. I. Achieer, Theory of Approziation. New York: Ungar, 1956, p Correlation Function Etiation by a Polarity Method Uing Stochatic Reference Signal HELMUT BERNDT, MEMBER, IEEE Abfract-Intead of etiating correlation function by conventional ean, a pecific polarity chee ay be ued on bounded procee. The ethod i baed on a rather iple relationhip between the correlation function before after infinite clipping, provided that tochatic reference ignal of unifor ditribution are added to the proce. Thi correlation technique ha been known for oe tie. Becaue of the apparent coputational advantage, it application to the etiation of correlation function fro dicrete or apled data i being exained. A general derivation of the appropriate oent relationhip i given a coplete ean-quare error analyi of etiate i provided under the auption of white-noie-type reference ignal. It i hown that correlation function etiate obtained by thi polarity ethod poe a ean-quare error that differ fro the error of conventional etiate only by a ter proportional to l/n, N i the aple ize. Thi ter ay be ade arbitrarily all. Thu, only all degradation in the accuracy of etiate have to be expected when uing the polarity approach. I. INTR~DUOTION MONG the correlation technique correlator deign reported in the literature of recent year, there i one ethod jutifying pecial attention. It appear to be very well uited for digital proceing at high data rate becaue the coputational effort ay be coniderably reduced in coparion to ore conventional ethod. In thi polarity correlation chee, tochatic reference ignal are added to the ignal of interet Manucript received Noveber 9, 1967; revied May 6, An earlier verion of thi paper wa preented a part, of a hort paper entitled Polarity correlation pectral denity etiation uing tochatic reference ignal at the 1966 IEEE Internat l Sypoiu on Inforation Theory, Univerity of California, Lo Angele, January 31-February 2, The author wa with Bell Telephone Laboratorie, Inc., Whippany, N. J. He i now with Sieen Aktiengeellchaft, Zentral- Laboratoriu fiir Nachrichtentechnik, Munich, Gerany. infinite clipping i perfored before correlation. Thu, iilar advantage ay be gained a with the ore coon polarity-coincidence correlation of Gauian procee. However, no inherent property of the ignal i aued thee two ethod hould be well ditinguihed. In thi paper, we hall be concerned with oe of the theoretical apect of polarity correlation eaureent uing tochatic reference ignal. Intruentational coputational quetion will only be tated to the extent needed in the theoretical developent. Thi correlation technique ha been known for well over five year, but a coplete error analyi had yet to be given. We hall how that under uitable condition, eaily et in ot cae of phyical reality, an application of thi ethod caue only a all degradation in the accuracy of the eaured correlation function a copared to a conventional etiate. The error ay even be ade arbitrarily all by increaing the aple ize vanihe, in the liit, for continuou procee. Thi general reult agree with an earlier known approxiation for large lag value. Only the autocorrelation cae will be conidered. While lightly ore coplex or, better, becaue ore care i needed, the dicuion of an autocorrelation function eaureent offer a better inight into the procedure a a whole. The pecialization to the cro-correlation cae ay thu be left to the reader. II. POLARITY CORRELATION If a tationary Gauian proce t(t) i ubjected to infinite clipping, the autocorrelation function of the reulting quare wave i given by 2 BgE(7) = ;in- p&7) (1)