Least Squares Optimal Filtering with Multirate Observations

Transcription

1 Prc. 36th Asilmar Cnf. n Signals, Systems, and Cmputers, Pacific Grve, CA, Nvember 2002 Least Squares Optimal Filtering with Multirate Observatins Charles W. herrien and Anthny H. Hawes Department f Electrical and Cmputer Engineering Cde EC/i Naval Pstgraduate Schl Mnterey, CA el: FAX: therrien@nps.navy.mil Abstract his paper addresses the prblem f ptimal filtering frm a least squares perspective when multiple bservatin sequences are available with differing sampling rates. In such cases the prcesses are jintly cyclstatinary and the resulting linear ptimal filters are peridically time-varying. he data matrices fr this prblem have interesting structure and we develp the frm f the resulting least squares multirate Wiener-Hpf equatins. Filtering results are illustrated fr a typical example and issues f cmputatin and amunt f training data needed are investigated. 1: Intrductin In sme prblems f interest multiple independent sets f bservatins are available, but the bservatins are nt necessarily taen with identical sampling rates. Fr example, we may have a vice transmissin available simultaneusly ver tw channels: a high rate channel with pr signal t nise rati and a lw rate channel with much higher signal t nise rati. In an image prcessing scenari we may have a suite f sensrs c-lcated s that IR and visible light are bserved simultaneusly, but with differing reslutins [1]. In all f these prblems, we may want t perfrm varius tass such as filtering (estimatin) r detectin. In [2] and [3], we reprted early results in the prblem f multirate filtering fr bservatins whse sampling rates differ by a factr f tw. If the underlying signal and nise prcesses are wide-sense statinary, then the bservatin sequences are jintly cyclstatinary (in the wide sense) and the ptimal linear filter is peridically time-varying (LPV) [4]. In the previus wr the structure f the theretical crrelatin matrices and the Wiener-Hpf equatins fr ptimal filtering were develped in detail. In the present paper we develp the ptimal filter frm a least squares pint f view. We present a generic algrithm fr cnstructing and partitining the relevant data matrices with tw bservatin sequences differing by a sampling rate K. We frmulate the slutin in terms f SVD f the data matrices. Results f multirate filtering are cmpared t crrespnding results fr single-channel single-rate filtering. 2: he Multirate Wiener Filter Let us assume the bservatins cnsist f a high rate sequence x[ n ] and a lw rate sequence ym [ ] and that the rati between the sampling rates is K. We will cnsider three pssible structures fr the ptimal filter shwn in Figs. 1, 2, and 3. x[n] h [n] (LPV) g [m] (LPV) Figure 1: Direct Frm d ^ [n]

2 x[n] (LI) h 0 [n] ^ d [n] n = d[n] H [n] (LPV) n = x[n] h 2 h 1 h 0 _ g [m] m = (LPV) Figure 2: Innvatins Frm g 1 g 0 Figure 4: Filter in the ime Dmain m=n/2 x[n] K h I [n] h[n] g[n] Figure 3: Interplatin Frm d ^ [n] In the first frm (Fig. 1), the ptimal estimate is frmed frm tw branches with linear peridically time-varying (LPV) filters. he utput is frmed as the sum f these filter utputs. In the secnd case (Fig. 2) the filter in the tp branch is time-invariant and represents the filter that wuld be used if the lw rate data were nt present. An auxiliary filter H is used t predict the lw rate data frm the high rate data and the predictin errr r innvatins sequence is applied t the filter in the lwer branch. hus the utput f the lwer branch represents the additinal infrmatin that is prvided by the lw rate bservatins. Bth filters in the lwer branch are peridically time-varying. In the last case (Fig. 3) the lw rate data is merely interplated t the high data rate and bth sequences are prcessed by time-invariant filters. Under suitable cnditins discussed belw this prcedure is als legitimate. Direct Frm. In this paper we cnsider a least-squares apprach t designing the filters. he filtering f the data fr a high rate filter f rder P and a lw rate filter f rder Q is depicted in Fig. 4. (In that figure the rati f the sampling rates is taen t be K 2.) We can write the filtered estimate in general as P1 Q1 (1) i0 j0 dn ˆ[ ] h [ ixn ] [ i ] g [ jym ] [ j ] fr n Km and 0,1,, K 1. he subscript n the filter cefficients indicates that they are peridically time-varying with perid K. We can dente the set f time-varying cefficients by the vectrs h and g fr 0,1,, K 1. If we further define the vectrs f bservatins as x [ n ] x [ n ] x [ n P 1] (2a) and y [ m ] y [ m ] y [ m Q 1] (2b) then (1) can be written as ˆ[ ] [ ] dnhxngy [ m ] (3) cnsider a least squares slutin fr the filter cefficients, define the data vectr d dn [ ] dn [ N1] (4) where n is the starting pint and N is the number f samples. Als define the crrespnding data matrices x [ n] y [ m] X Y (5) x [ n N 1] y [ m M 1]

3 where n and m are crrespnding pints in the bservatin sequences, and N KM. he filter vectrs can be fund by slving a set f K least squares equatins f the frm ls X Y d 0,1,, K 1 h (6) g he vectr d is frmed frm d by extracting every K th element beginning with element 1, and X is liewise frmed frm X by extracting every K th rw beginning with rw 1. he ntatin ls in (6) means that a least squares slutin is sught that minimizes the squared nrm f the errr between the left and right sides f the equatin. he slutin is given by g h X Y d 0,1,, K 1 (7) where dentes the Mre-Penrse pseudinverse. By the rthgnality principle f linear mean square estimatin, the mean-squared errr is given by: N 1 1 MSE ˆ [ ] [ ] ˆ d n d n d[ n] (8) N n0 which can be reduced t: K 1 ˆ 1 * * * MSE N d d d X h d Yg (9) 0 Innvatins Frm. derive the innvatins frm f the filter (Fig. 2) we use sme results frm the thery f generalized inverses [5] t write the pseudinverse matrix in partitined frm. In particular, given that the rws f the data matrix are independent, (7) can be written as h X X YC X Y d d g C Xd XYCd Cd (10) where C=(IXX )Y. Frm (8), it can be seen that if we define H = XYthen h =h H g (11) where h = X d is the ptimal filter fr estimating the data sequence using nly the x[ n ] bservatin sequence. In a stchastic prcess framewr, this filter wuld nt be a functin f due t the statinarity [2]. In this least squares framewr the filter des indeed depend n, but all f the h cnverge t a cmmn value fr a lng data sequence. By applying the last equatin t (3), the filter utput can be written as ˆ[ ] [ ] [ ] h x ghx gy [ ] h x[ n] g y[ m] Hx[ n] dn n n m (12) which yields the frm depicted in Fig. 2. Interplatin Frm. his realizatin appears t be the simplest since the branch filters are all time-invariant. It can be shwn that this structure is valid as lng as ym [ ] is bandlimited t / K and the interplatin filter is bandlimited t the same band (the cnditin fr statinarity f the utput [4]). he ideal interplatin filter is nt causal hwever, and a causal apprximatin has t be used fr this realizatin. 4: Cmputatin and raining he three frms f the ptimal multirate filter can be cmpared in terms f the number f parameters needed t specify the filter and the number f cmputatins needed t be perfrmed at each pint in time. Nte that a peridically time-varying impulse respnse functin such as h [ n ] represents a ttal f KP real scalar parameters that need t be estimated. In general, estimatin f a larger number f parameters requires a larger number f time samples. (We refer t this as training data. ) he table belw cmpares the three frms f filters: Frm N. filter N. peratins/ parameters unit time Direct (PQ)K PQ Innvatins PPQKKQ PPQQ Interplatin PQ PQI r PQK PQKI able 1: Parameters fr Frms f Optimal Filter he number f parameters fr mst frms f the filter has a direct dependence n K. he number f arithmetic peratins (multiplicatins and/r additins) that need t be perfrmed fr each time step fr the direct frm is

4 simply equal t P Q and des nt depend n K. hus althugh the filter needs t cycle thrugh K different sets f parameters, the cmputatinal requirement is the same as a filter with a fixed set f parameters. he innvatins frm, thugh appealing frm an analysis pint f view, has the largest number f parameters and the largest cmputatinal requirements. Frm an implementatin pint f view, this filter is nt liely t be used because f the extra cmputatin cmpared with the direct frm. w results are listed fr the interplatin frm f the filter. Frm a naïve pint f view, the filter has just P Q parameters since the parameters f the interplatin filter are cnstant and d nt depend n the input statistics. hus the parameter estimatin prblem is mitigated. he number f peratins hwever depends n the length I f the interplatin filter, which can becme the dminate part f the cmputatin. Frm a mre realistic pint f view hwever, a great advantage f the multirate filter is t l further bac in time alng the lw rate prcess. btain equivalent perfrmance, ne wuld expect that the filter fr the lw-rate prcess after interplatin wuld have t be f rder KQ rather than just rder Q. hus the number f parameters and cmputatinal requirements are wuld be cnsiderably larger as shwn in the last rw f the table. 5: Perfrmance Results Mean squared-errr (MSE) is a natural chice fr perfrmance criterin in this situatin. Hwever the MSE fr any simulatin f this prcess is affected by the parameters P, Q, and K, the channel signal-t-nise ratis, and the length f data used t calculate the filter impulse respnses. In this paper a few parameters have been varied in the fllwing results. Simulatins which cmpare 1) the perfrmance f the time-varying and time invariant filters, and 2) the amunt f training data needed t slve fr the filter cefficients in each case are given belw. 5.1: ime-variant vs. ime-invariant It is useful t cmpare the perfrmance f the direct frm t using a single channel ptimal filter n either the high r lw-rate data sequences separately. gain insight int this matter, the rder f the lw-rate filter was varied and the resulting MSE was calculated. w specific cases are shwn here. Fr bth cases, P 30, K 10, and the SNRs f the high and lw-rate data sequences are 0 and 10 db, respectively. Figure 6 shws the signal t be estimated, and the crrespnding highrate bservatins x[ n ], and lw-rate bservatins ym [ ]: Figure 5: Signal Estimatin: (a) desired signal (b) high-rate data (0 db SNR) (c) lw-rate data (10 db) he length f the training and test sequences was pints each. Results fr MSE were averaged ver 100 trials f randm channel nise. he table belw summarizes the results fr Q 3 : Sensr raining Set (db) est Set (db) High-rate Lw-rate Bth able 2: Mean Squared-Errr fr Q = 3 (100 trial average) In this case the lw-rate filter uses data ging bac in time t the same pint used by the high-rate filter. As anther example, Q 10 is chsen and the results are tabulated belw: Sensr raining Set (db) est Set (db) High-rate Lw-rate Bth able 3: Mean Squared-Errr fr Q = 10 (100 trial average) In this case the lw-rate filter uses data ging much further bac in time than the high-rate filter. In bth cases, these results clearly indicate a maredly lwer

5 MSE while using bth data sequences cmpared t results using either high-rate r lw-rate data sequences alne. 5.2: raining Data Required Anther imprtant factr in implementing the direct frm is the length f training data required t accurately estimate the filter cefficients. Since the number f filter cefficients is equal t ( P Q) K, cncern arises fr large ratis f the sampling rates. Fr example: when K 10 the number f parameters is 10 times the number f parameters that wuld be required if the filter were nt time-varying. Fr the direct frm t be practical, slving fr the parameters cannt require an inrdinate length f training data. investigate this matter, the length f training data used t design the filter was varied frm 500 t pints in 500 pint intervals. All ther factrs are as stated abve. he resulting MSE n the training and test sequences were mnitred. he length f training data at which the relative errr n the training and test sets settles t within ne percent is taen as the criterin fr an adequate length in slving fr the cefficients. Results were averaged ver 100 trials. Figure 6 illustrates the cnvergence in MSE when using high-rate data alne versus bth data sets. he cnvergence pints differ by a factr f 2, but nt by a factr f 10 r mre as might be expected. Figure 6: MSE vs. Length f raining Data he results summarized in able 4 indicate that the amunt f training data needed t estimate the filter cefficients is cmparable t that when using timeinvariant filters n either sequence alne. Parameter Lw High Bth Q = Q = 10 > able 4: Length f training data required t achieve relative errr f 1% (100 trial average) 6: Cnclusins In sme prblems it is desired t cmbine bservatins taen at different sampling rates t estimate sme underlying signal f interest. If the signal and assciated channel nises are wide sense statinary, then the bservatins are jintly cyclstatinary. Optimal multirate filtering f such bservatins requires linear peridically time-varying filters. In this paper we cnsidered a least squares apprach t designing the ptimal peridically time-varying filters. We cmpared varius frms f the filter in terms f cmputatin and number f parameters. We then investigated perfrmance f the direct frm experimentally. Experimental results shw that multirate prcessing f sensr measurements sampled at different rates can be beneficial. Specifically, it was shwn that ptimal multirate filtering f bservatins samples at different rates is superir t using a single-channel filter n either bservatins sequence alne. Als, the amunt f training data needed fr the multirate prcessing is cmparable t that when using a single-channel filter. 7: References 1. A. M. Waxman et al, Prgress n clr night visin: Visible/IR fusin, perceptin and search, and lw-light CCD imaging. In Enhanced and Synthetic Visin 1996, Vl. 2736, pp , SPIE he Internatinal Sciety fr Optical Engineering, Dimitris Kupatsiaris, Analysis f Multirate Randm Signals, hese fr the M.S. and E.E. degree, Naval Pstgraduate Schl, December R. Cristi, D. Kupatsiaris, and C. W. herrien, Multirate filtering and estimatin: the multirate Wiener filter, Prc. 34 th Asilmar Cnf. On Signals, Systems, and Cmputers, Octber 2000, pp , Pacific Grve, CA. 4. C. W. herrien, Issues in multirate statistical signal prcessing, Prc. 35 th Asilmar Cnf. On Signals, Systems, and Cmputers, Nvember 2001, pp , Pacific Grve, CA. 5..L. Bullin and P.L. O Dell, Generalized Inverse Matrices, Jhn Wiley, 1971.