Prc. 36th Asilmar Cnf. n Signals, Systems, and Cmputers, Pacific Grve, CA, Nvember 2002 Least Squares Optimal Filtering with Multirate Observatins Charles W. herrien and Anthny H. Hawes Department f Electrical and Cmputer Engineering Cde EC/i Naval Pstgraduate Schl Mnterey, CA 93940 el: 831.656.3347 FAX: 831.656.2760 e-mail: therrien@nps.navy.mil Abstract his paper addresses the prblem f ptimal filtering frm a least squares perspective when multiple bservatin sequences are available with differing sampling rates. In such cases the prcesses are jintly cyclstatinary and the resulting linear ptimal filters are peridically time-varying. he data matrices fr this prblem have interesting structure and we develp the frm f the resulting least squares multirate Wiener-Hpf equatins. Filtering results are illustrated fr a typical example and issues f cmputatin and amunt f training data needed are investigated. 1: Intrductin In sme prblems f interest multiple independent sets f bservatins are available, but the bservatins are nt necessarily taen with identical sampling rates. Fr example, we may have a vice transmissin available simultaneusly ver tw channels: a high rate channel with pr signal t nise rati and a lw rate channel with much higher signal t nise rati. In an image prcessing scenari we may have a suite f sensrs c-lcated s that IR and visible light are bserved simultaneusly, but with differing reslutins [1]. In all f these prblems, we may want t perfrm varius tass such as filtering (estimatin) r detectin. In [2] and [3], we reprted early results in the prblem f multirate filtering fr bservatins whse sampling rates differ by a factr f tw. If the underlying signal and nise prcesses are wide-sense statinary, then the bservatin sequences are jintly cyclstatinary (in the wide sense) and the ptimal linear filter is peridically time-varying (LPV) [4]. In the previus wr the structure f the theretical crrelatin matrices and the Wiener-Hpf equatins fr ptimal filtering were develped in detail. In the present paper we develp the ptimal filter frm a least squares pint f view. We present a generic algrithm fr cnstructing and partitining the relevant data matrices with tw bservatin sequences differing by a sampling rate K. We frmulate the slutin in terms f SVD f the data matrices. Results f multirate filtering are cmpared t crrespnding results fr single-channel single-rate filtering. 2: he Multirate Wiener Filter Let us assume the bservatins cnsist f a high rate sequence x[ n ] and a lw rate sequence ym [ ] and that the rati between the sampling rates is K. We will cnsider three pssible structures fr the ptimal filter shwn in Figs. 1, 2, and 3. x[n] h [n] (LPV) g [m] (LPV) Figure 1: Direct Frm d ^ [n]
x[n] (LI) h 0 [n] ^ d [n] n = 0 1 2 3 4 5 d[n] H [n] (LPV) n = 0 1 2 3 4 5 x[n] h 2 h 1 h 0 _ g [m] m = 0 1 2 (LPV) Figure 2: Innvatins Frm g 1 g 0 Figure 4: Filter in the ime Dmain m=n/2 x[n] K h I [n] h[n] g[n] Figure 3: Interplatin Frm d ^ [n] In the first frm (Fig. 1), the ptimal estimate is frmed frm tw branches with linear peridically time-varying (LPV) filters. he utput is frmed as the sum f these filter utputs. In the secnd case (Fig. 2) the filter in the tp branch is time-invariant and represents the filter that wuld be used if the lw rate data were nt present. An auxiliary filter H is used t predict the lw rate data frm the high rate data and the predictin errr r innvatins sequence is applied t the filter in the lwer branch. hus the utput f the lwer branch represents the additinal infrmatin that is prvided by the lw rate bservatins. Bth filters in the lwer branch are peridically time-varying. In the last case (Fig. 3) the lw rate data is merely interplated t the high data rate and bth sequences are prcessed by time-invariant filters. Under suitable cnditins discussed belw this prcedure is als legitimate. Direct Frm. In this paper we cnsider a least-squares apprach t designing the filters. he filtering f the data fr a high rate filter f rder P and a lw rate filter f rder Q is depicted in Fig. 4. (In that figure the rati f the sampling rates is taen t be K 2.) We can write the filtered estimate in general as P1 Q1 (1) i0 j0 dn ˆ[ ] h [ ixn ] [ i ] g [ jym ] [ j ] fr n Km and 0,1,, K 1. he subscript n the filter cefficients indicates that they are peridically time-varying with perid K. We can dente the set f time-varying cefficients by the vectrs h and g fr 0,1,, K 1. If we further define the vectrs f bservatins as x [ n ] x [ n ] x [ n P 1] (2a) and y [ m ] y [ m ] y [ m Q 1] (2b) then (1) can be written as ˆ[ ] [ ] dnhxngy [ m ] (3) cnsider a least squares slutin fr the filter cefficients, define the data vectr d dn [ ] dn [ N1] (4) where n is the starting pint and N is the number f samples. Als define the crrespnding data matrices x [ n] y [ m] X Y (5) x [ n N 1] y [ m M 1]
where n and m are crrespnding pints in the bservatin sequences, and N KM. he filter vectrs can be fund by slving a set f K least squares equatins f the frm ls X Y d 0,1,, K 1 h (6) g he vectr d is frmed frm d by extracting every K th element beginning with element 1, and X is liewise frmed frm X by extracting every K th rw beginning with rw 1. he ntatin ls in (6) means that a least squares slutin is sught that minimizes the squared nrm f the errr between the left and right sides f the equatin. he slutin is given by g h X Y d 0,1,, K 1 (7) where dentes the Mre-Penrse pseudinverse. By the rthgnality principle f linear mean square estimatin, the mean-squared errr is given by: N 1 1 MSE ˆ [ ] [ ] ˆ d n d n d[ n] (8) N n0 which can be reduced t: K 1 ˆ 1 * * * MSE N d d d X h d Yg (9) 0 Innvatins Frm. derive the innvatins frm f the filter (Fig. 2) we use sme results frm the thery f generalized inverses [5] t write the pseudinverse matrix in partitined frm. In particular, given that the rws f the data matrix are independent, (7) can be written as h X X YC X Y d d g C Xd XYCd Cd (10) where C=(IXX )Y. Frm (8), it can be seen that if we define H = XYthen h =h H g (11) where h = X d is the ptimal filter fr estimating the data sequence using nly the x[ n ] bservatin sequence. In a stchastic prcess framewr, this filter wuld nt be a functin f due t the statinarity [2]. In this least squares framewr the filter des indeed depend n, but all f the h cnverge t a cmmn value fr a lng data sequence. By applying the last equatin t (3), the filter utput can be written as ˆ[ ] [ ] [ ] h x ghx gy [ ] h x[ n] g y[ m] Hx[ n] dn n n m (12) which yields the frm depicted in Fig. 2. Interplatin Frm. his realizatin appears t be the simplest since the branch filters are all time-invariant. It can be shwn that this structure is valid as lng as ym [ ] is bandlimited t / K and the interplatin filter is bandlimited t the same band (the cnditin fr statinarity f the utput [4]). he ideal interplatin filter is nt causal hwever, and a causal apprximatin has t be used fr this realizatin. 4: Cmputatin and raining he three frms f the ptimal multirate filter can be cmpared in terms f the number f parameters needed t specify the filter and the number f cmputatins needed t be perfrmed at each pint in time. Nte that a peridically time-varying impulse respnse functin such as h [ n ] represents a ttal f KP real scalar parameters that need t be estimated. In general, estimatin f a larger number f parameters requires a larger number f time samples. (We refer t this as training data. ) he table belw cmpares the three frms f filters: Frm N. filter N. peratins/ parameters unit time Direct (PQ)K PQ Innvatins PPQKKQ PPQQ Interplatin PQ PQI r PQK PQKI able 1: Parameters fr Frms f Optimal Filter he number f parameters fr mst frms f the filter has a direct dependence n K. he number f arithmetic peratins (multiplicatins and/r additins) that need t be perfrmed fr each time step fr the direct frm is
simply equal t P Q and des nt depend n K. hus althugh the filter needs t cycle thrugh K different sets f parameters, the cmputatinal requirement is the same as a filter with a fixed set f parameters. he innvatins frm, thugh appealing frm an analysis pint f view, has the largest number f parameters and the largest cmputatinal requirements. Frm an implementatin pint f view, this filter is nt liely t be used because f the extra cmputatin cmpared with the direct frm. w results are listed fr the interplatin frm f the filter. Frm a naïve pint f view, the filter has just P Q parameters since the parameters f the interplatin filter are cnstant and d nt depend n the input statistics. hus the parameter estimatin prblem is mitigated. he number f peratins hwever depends n the length I f the interplatin filter, which can becme the dminate part f the cmputatin. Frm a mre realistic pint f view hwever, a great advantage f the multirate filter is t l further bac in time alng the lw rate prcess. btain equivalent perfrmance, ne wuld expect that the filter fr the lw-rate prcess after interplatin wuld have t be f rder KQ rather than just rder Q. hus the number f parameters and cmputatinal requirements are wuld be cnsiderably larger as shwn in the last rw f the table. 5: Perfrmance Results Mean squared-errr (MSE) is a natural chice fr perfrmance criterin in this situatin. Hwever the MSE fr any simulatin f this prcess is affected by the parameters P, Q, and K, the channel signal-t-nise ratis, and the length f data used t calculate the filter impulse respnses. In this paper a few parameters have been varied in the fllwing results. Simulatins which cmpare 1) the perfrmance f the time-varying and time invariant filters, and 2) the amunt f training data needed t slve fr the filter cefficients in each case are given belw. 5.1: ime-variant vs. ime-invariant It is useful t cmpare the perfrmance f the direct frm t using a single channel ptimal filter n either the high r lw-rate data sequences separately. gain insight int this matter, the rder f the lw-rate filter was varied and the resulting MSE was calculated. w specific cases are shwn here. Fr bth cases, P 30, K 10, and the SNRs f the high and lw-rate data sequences are 0 and 10 db, respectively. Figure 6 shws the signal t be estimated, and the crrespnding highrate bservatins x[ n ], and lw-rate bservatins ym [ ]: Figure 5: Signal Estimatin: (a) desired signal (b) high-rate data (0 db SNR) (c) lw-rate data (10 db) he length f the training and test sequences was 25000 pints each. Results fr MSE were averaged ver 100 trials f randm channel nise. he table belw summarizes the results fr Q 3 : Sensr raining Set (db) est Set (db) High-rate 9.84 9.86 Lw-rate 13.59 13.53 Bth 7.65 7.64 able 2: Mean Squared-Errr fr Q = 3 (100 trial average) In this case the lw-rate filter uses data ging bac in time t the same pint used by the high-rate filter. As anther example, Q 10 is chsen and the results are tabulated belw: Sensr raining Set (db) est Set (db) High-rate 9.84 9.86 Lw-rate 10.67 10.53 Bth 5.96 5.77 able 3: Mean Squared-Errr fr Q = 10 (100 trial average) In this case the lw-rate filter uses data ging much further bac in time than the high-rate filter. In bth cases, these results clearly indicate a maredly lwer
MSE while using bth data sequences cmpared t results using either high-rate r lw-rate data sequences alne. 5.2: raining Data Required Anther imprtant factr in implementing the direct frm is the length f training data required t accurately estimate the filter cefficients. Since the number f filter cefficients is equal t ( P Q) K, cncern arises fr large ratis f the sampling rates. Fr example: when K 10 the number f parameters is 10 times the number f parameters that wuld be required if the filter were nt time-varying. Fr the direct frm t be practical, slving fr the parameters cannt require an inrdinate length f training data. investigate this matter, the length f training data used t design the filter was varied frm 500 t 25000 pints in 500 pint intervals. All ther factrs are as stated abve. he resulting MSE n the training and test sequences were mnitred. he length f training data at which the relative errr n the training and test sets settles t within ne percent is taen as the criterin fr an adequate length in slving fr the cefficients. Results were averaged ver 100 trials. Figure 6 illustrates the cnvergence in MSE when using high-rate data alne versus bth data sets. he cnvergence pints differ by a factr f 2, but nt by a factr f 10 r mre as might be expected. Figure 6: MSE vs. Length f raining Data he results summarized in able 4 indicate that the amunt f training data needed t estimate the filter cefficients is cmparable t that when using timeinvariant filters n either sequence alne. Parameter Lw High Bth Q = 3 10000 4500 7000 Q = 10 >25000 4500 4500 able 4: Length f training data required t achieve relative errr f 1% (100 trial average) 6: Cnclusins In sme prblems it is desired t cmbine bservatins taen at different sampling rates t estimate sme underlying signal f interest. If the signal and assciated channel nises are wide sense statinary, then the bservatins are jintly cyclstatinary. Optimal multirate filtering f such bservatins requires linear peridically time-varying filters. In this paper we cnsidered a least squares apprach t designing the ptimal peridically time-varying filters. We cmpared varius frms f the filter in terms f cmputatin and number f parameters. We then investigated perfrmance f the direct frm experimentally. Experimental results shw that multirate prcessing f sensr measurements sampled at different rates can be beneficial. Specifically, it was shwn that ptimal multirate filtering f bservatins samples at different rates is superir t using a single-channel filter n either bservatins sequence alne. Als, the amunt f training data needed fr the multirate prcessing is cmparable t that when using a single-channel filter. 7: References 1. A. M. Waxman et al, Prgress n clr night visin: Visible/IR fusin, perceptin and search, and lw-light CCD imaging. In Enhanced and Synthetic Visin 1996, Vl. 2736, pp. 96 107, SPIE he Internatinal Sciety fr Optical Engineering, 1996. 2. Dimitris Kupatsiaris, Analysis f Multirate Randm Signals, hese fr the M.S. and E.E. degree, Naval Pstgraduate Schl, December 2000. 3. R. Cristi, D. Kupatsiaris, and C. W. herrien, Multirate filtering and estimatin: the multirate Wiener filter, Prc. 34 th Asilmar Cnf. On Signals, Systems, and Cmputers, Octber 2000, pp. 450 454, Pacific Grve, CA. 4. C. W. herrien, Issues in multirate statistical signal prcessing, Prc. 35 th Asilmar Cnf. On Signals, Systems, and Cmputers, Nvember 2001, pp. 573 576, Pacific Grve, CA. 5..L. Bullin and P.L. O Dell, Generalized Inverse Matrices, Jhn Wiley, 1971.