[db]

Size: px

Start display at page:

Download "[db]"

Rosa Bates
5 years ago
Views:

1 Blind Source Separation based on Second-Order Statistics wit Asymptotically Optimal Weigting Arie Yeredor Department of EE-Systems, el-aviv University P.O.Box 3900, el-aviv 69978, Israel Abstract Blind Source Separation (BSS) addresses te reconstruction of statistically independent source signals from (instantaneous) linear combinations tereof. Second-order statistics can be used for BSS wenever te source signals ave distinct spectra. e Second-Order Blind Identication (SOBI) algoritm, proposed by Beloucrani et al. in 997, uses approximate joint diagonalization of estimated correlation matrices at several lags to estimate te unknown mixing matrix. In tis paper we sow ow te SOBI algoritm can be substantially improved wen te joint diagonalization is transformed into a properly weigted Nonlinear Least Squares (NLS) problem. We propose an iterative solution for tis NLS problem, and derive te optimal weigts under te assumption tat te source signals are Gaussian. Under weak ergodicity conditions tese weigt matrices can be estimated consistently from te observed data, rendering te blind algoritm asymptotically optimal. e improvement is demonstrated by simulations, and sown (empirically) to be robust wit respect to te Gaussianity assumption. keywords: Blind Source Separation, Second-Order Statistics, Least Squares, Optimal Weigting Introduction Blind Source Separation (BSS) addresses te reconstruction of N statistically independent source signals from M linear combinations tereof. In te static mixture framework, te observation model is were s[t] = signals, x[t] = x[t] = As[t] t = ; 2; : : : ; () s[t] s2[t] s N [t]i are te source x[t] x2[t] x M [t]i are te observations and A 2 C MN is te unknown mixing matrix. e term 'blind' ascribes lack of any additional information regarding te signals or A. In [], Beloucrani et al. proposed te "Second- Order Blind Identication" (SOBI) algoritm for stationary signals wit distinct spectra. SOBI is based on te joint diagonalization property of te observations' correlation matrices Rx[] = E x[t + ]x H [t]. Spcically, tese matrices satisfy Rx[] = ARs[]A H 8 (2) were Rs[] = i E s[t + ]s H [t] are te source signals' (unknown) diagonal correlation matrices. us, An is a joint diagonalizer of o any set of K matrices, Rx[]; Rx[2]; : : : Rx[ K ]. It can be sown tat if all te source signals ave distinct spectra (diering by more tan scale), ten a set of lags can be found suc tat te joint diagonalizer is unique, up to irrelevant scaling and permutation of columns. It is terefore proposed in [] to estimate A as te joint diagonalizer n of a set of estimated o correlation matrices ^Rx []; ^Rx [2]; : : : ^Rx [ K ]. However, wile te set of true correlation matrices admits exact diagonalization, it is almost surely impossible to jointly diagonalize te set of estimated matrices. It is still possible, owever, to obtain consistent estimators for A by resorting to approximate joint diagonalization, attained in [] in two pases: In te rst pase a "witening" matrix ^W is found, suc tat ^W ^Rx[0] ^W H = I (identity matrix). All te oter matrices are ten similarly transformed, ~^R[ k ] = ^W ^Rx[ k ] ^W H k = ; 2; : : : K: (3) In te second pase te unitary approximate joint diagonalizer ^U of te transformed set is found, using successive Jacobi rotations, wic iteratively minimize te o-diagonal entries of te transformed matrices ([, 3]). e desired estimate is ten given by ^A = ^W ] ^U (were ^W ] denotes te pseudo-inverse of ^W ).

2 It can be easily observed (see also [6]) tat te second pase optimizes a Least-Squares (LS) t of te ~^R[ k ]-s wit respect to ^U. However, tis LS criterion is not optimized wit respect to ^A, since te nonunitary part ^W is cosen to attain exact diagonalization of ^Rx [0], possibly at te expense of poor diagonalization of te oter matrices. It as been noted in [2] tat suc "ard-witening" operation bounds te attainable performance, since errors incurred by te estimated ^Rx [0] and inicted upon ^W can no longer be compensated for by te oter matrices, wic only aect ^U. Furtermore, wile te "ard-witening" approac imposes severely unbalanced weigting, it may be desired to not only balance te weigts (e.g., use unweigted LS), but rater to seek te optimal weigting. In fact, since te errors in estimating te correlation values are temselves strongly correlated, it can be expected tat substantial improvement in performance may be attained by using an optimally weigted LS (WLS) criterion, wic can account for tese correlations by te use of a non-diagonal weigt matrix. e weigting approac as also been proposed in [], but not pursued furter in tere. In tis paper we address tese two sortcomings of te SOBI algoritm. First, we reformulate te approximate diagonalization problem as a non-linear WLS problem. In tat framework, several iterative algoritm can be considered for minimization wit respect to an arbitrary (not necessarily unitary) matrix ^A. We ten proceed to nd te optimal weigt matrix. In order to evaluate te correlation between te estimated correlation values, we assume tat te source signals are Gaussian wit nite-lengt correlations (i.e. Moving Average (MA) Processes). is assumption enables to express te optimal weigt matrix in terms of te estimated correlation matrices, since all te required fourt-order moments can be expressed in terms of te available (estimated) secondorder moments. In conclusion we demonstrate (wit simulations results) substantial improvement over te SOBI algoritm. o capture te essence of our proposal in tis limited-lengt exposition, we focus on te case of M = N = 2 real-valued signals wit a real-valued mixing matrix. Extension to complex signals (and mixing) is relatively straigtforward. Our algoritm is given te acronym WASOBI (Weigts-Adjusted SOBI). 2 Formulation as a Weigted LS Problem e formulation of a LS model requires te description of available (inaccurate) measurements in terms of te parameters of interest. We use te elements of te estimated correlation matrices at various (nonnegative) lags as te set of raw measurements. For reasons tat will become clear in te next section, we assume tat all te estimated correlation matrices ^Rx [ k ] are calculated using te same number of data points. Specically, ^Rx[ k ] = X x[t]x [t + k ]: () t= Note tat tis assumes (implicitly) tat + k samples are available for te estimation of Rx[ k ], wic means tat te actual number of available samples is + max were max is te fartest lag used. is is somewat wasteful in te sense tat not all available data points are used for smaller lags. However, tis wastefulness is negligible wen is large relative to max, and tis assumption simplies te derivation of optimal weigts in te next section. We seek a 2 2 matrix A and K diagonal matrices ; 2 : : : K suc tat ^Rx [ k ] are "best tted" by A k A for k = ; 2; : : : K. us, tere are four parameters of interest, denoted a = vecfag = (;) (2;) (;2) (2;2) [A A A A ], and 2K nuisance parameters, wic are te K 2 vectors k = diagf k g k = ; 2; : : : K. However, due to te inerent scaling ambiguity (wic enables to commute scales between A and k ), we may arbitrarily x e.g., reducing te true number of nuisance parameters to 2(K ). Note tat te estimated ^Rx [ k ] are not necessarily symmetric (for k 6= 0), in contrast to A k A. We sall tus attempt to t eac A k A to a symmetric variant of te respective ^Rx [ k ], obtained by substituting its o-diagonal terms wit teir aritmetic average. We terefore dene ^rk = vecf ^Rx[ k ]g and y k = C^rk k = ; 2 : : : K; (5) were C is a constant transformation matrix, 2 C = ; (6) are te actual measurements of te LS model. e

3 desired t for eac k can ten be written as were te matrix G(a) is given by y k G(a)k: (7) 2 G(a) = a2 a 2 3 aa2 a3a5 : (8) a 2 a 2 Concatenating all y k into y = [y y 2 y K ], we get y [IK G(a)] = ~ G(a) (9) were IK denotes te K K identity matrix, denotes Kronecker's product, and = [ 2 K] is te concatenation of k. We also dene = [ 2 3 K] ; (0) te vector of free parameters in. Given any 3K 3K symmetric weigt matrix W, we may now dene te WLS criterion as C W LS (a; ) = [y ~ G(a)] W [y ~ G(a)] () to be minimized wit respect to (w.r.t.) a and, wit set arbitrarily. Wile linear (quadratic) in, tis WLS criterion is nonlinear in a. Several metods for minimizing C W LS can be considered. For example, Gauss iterations (see e.g. [7]) can be used. However, o exploit te linear part (w.r.t. ), te Gauss iterations may be restricted to te nonlinear minimization w.r.t. a wit xed. us, C W LS can be minimized by alternating between linear (closed-form) minimization w.r.t. wit a xed, and vice-versa. Anoter appealing approac would be to interlace minimizations w.r.t. wit te Gauss iterations. e SOBI estimate may be used as an initial value for te iterations. e minimization of C W LS would often be computationally more intensive tan te SOBI minimization (tis is obviously te case if te SOBI estimate is used as an initial guess). Note, owever, tat te computational load of te minimization depends only on K, and is independent of te number of observations. us, asymptotically, te mean computational load per sample of te two metods is equal, since it is dominated by te calculation of te estimated correlation, rater tan by te jointdiagonalization / LS-minimization. 3 Optimal Weigting e LS criterion presented above allows te use of any (arbitrary) weigt matrix W. Naturally, we would 3 like to use te optimal weigt matrix, wic is wellknown (e.g. [7]) to be te inverse of te measurements' covariance matrix. us we need te covariance matrix of y, denoted. Assuming Gaussian signals, we ave from () i E ^R(i;j) x [ k ] ^R(m;n) x [ l ] X = x i [t]x j [t + k ]x m [s]x n [s + l ] X 2 E = t= s= (i;j) R x [ (m;n) k ]R x [ l ] + X ( jpj )R(i;m) (j;n) x [p]r x [p + l k ] + p= ( ) X ( jpj )R(i;n) x [p + (j;m) l ]R x [p k ] p= ( ) (2) wic implies tat te covariance of ^R(i;j) [ k ] and (m;n) ^R x [ l ] is given by sum of te last two terms. We now furter assume tat te source signals are MA of orders Q, wereas te selected lags are k = k ; k = ; 2; : : : Q +. e summation over p can ten be reduced to Q to Q for k; l K = Q +, wic implies tat estimating te correlation matrices up to lag Q is also sucient for consistently estimating. Wit sligt manipulations (2) can be reformulated in matrix form, suc tat Cov[^rk; ^rl] = QX ( jqj )R x[p + l k ] Rx[p] + p= Q QX ( jqj )R x[p k ] Rx[p l ])P p= Q x i (3) were P is a permutation matrix tat swaps te second and tird columns of te matrix to its left. Recalling te linear transformation (7) from ^rk to y k we conclude tat te (k; l)-t 3 3 block of is given by k;l = Cov[yk ; y l ] = CCov[^rk; ^rl]c : () e optimal weigt matrix is ten given by Wopt =. In practice, estimated correlations would replace true correlations in (3), providing a consistent estimate of Wopt. us te resulting weigts are asymptotically optimal.

4 Simulations Results Fig. presents some simulations results in terms of te mean Interference to Signal Ratio (ISR) for bot SOBI and WASOBI, vs. te observation lengt. e source signals used were MA() and MA(3) processes: s(t) is an MA() process wit zeros at 0:8e j 2, 0:7e j 3 (and teir reciprocals); s2(t) is an MA(3) process wit zeros at 0:7, 0:3e j 2 (and teir reciprocals). Bot algoritms used te same data. Eac simulation point represents an average of 000 trials. e mixing matrix used was A = 2 3. Interestingly, owever, it turns out tat performance (in terms of ISRs) does not depend on A for neiter SOBI nor WASOBI. e A-invariance of SOBI agrees wit []; for WASOBI it is more subtle to conclude from te derivation - note tat suc invariance is not attained wit arbitrary W! o put te simulations results in context, we also present (in solid lines, superimposed on simulations results) te teoretically predicted performance: Since te "measurements" y are unbiased (teir expected value are te true correlation values), te estimated parameters are also unbiased, under a smallerrors assumption (regardless of te weigting used). Using te derivative of te LS criterion () wit respect to all te parameters, as well as te measurements' covariance, standard tools can be used (e.g. [7]) to obtain te (approximate) error covariance in estimating a, te elements of A. is covariance can in turn be translated to te mean ISR obtained wen te estimated A is used for reconstruction of te source signals. See [5] for an explicit derivation. e resulting expressions are general, and can be used wit any weigt matrix W. For example, for te SOBI (unweigted) algoritm, W would be set to WSOBI = diagfi3; I3(K )g (in te matrices-to-matrix sense), were >> Q (we used = 00) is a large constant reecting SOBI's obligatory witening pase, wic attributes innite weigt to warrant exact diagonalization of ^Rx [0]. For WASOBI we used te optimal weigt, Wopt. Note tat wile te true Wopt was used for calculating te predicted performance, te WA- SOBI algoritm naturally used only te estimated Wopt (based on te available data). It is seen tat as expected, te predictions are approaced as increases, wen te estimated Wopt approaces te true Wopt, and wen te small-errors assumption prevails. In addition to nominal-setup simulations we also demonstrate te relative robustness of te algoritm to te Gaussianity assumptions. e two MA signals wit zeros as specied above were generated once wit a Gaussian driving-noise (rendering tem Gaussian), and once wit non-gaussian (Uniformly distributed) Gaussian noise. e ISR results for bot cases are denoted by te symbols 'o' and '*', respectively. e similarity of results is evident, and te advantage of WASOBI over SOBI is also maintained wen te underlying Gaussianity assumption is somewat violated. 5 Summary We presented an improved version of te SOBI algoritm, wic is given te acronym WASOBI. It is based on optimal weigting of te LS criterion used in te joint diagonalization of te estimated correlation matrices. e optimal weigts are can be obtained from te estimated correlations due to te Gaussianity assumption, combined wit te nite correlation lengts of te source signals. Note tat since te correlation estimated are asymptotically Gaussian, te LS criterion wit asymptotically optimal weigting realizes an asymptotic maximum-likeliood (ML) criterion wit respect to te estimated matrices. Note furter, tat since te estimated matrices are sucient statistics wit respect to te observed data for MA Gaussian signals, te WASOBI algoritm actually yields (asymptotically) te ML estimate of te mixing matrix, wose mean squared error coincides (asymptotically) wit te Cram'er-Rao lower Bound. References [] A. Beloucrani, K. Abed-Meraim, J.-F. Cardoso and E. Moulines, \A blind source separation tecnique using second-order statistics" IEEE ran. Signal Processing, vol. 5 no. 2, pp. 3{, 997. [2] J.-F. Cardoso, \On te Performance of Ortogonal Source Separation Algoritms" Proceedings of EUSIPCO'9, pp. 776{779, 99. [3] J.-F. Cardoso, \Jacobi angles for simultaneous diagonalization" SIAM Journal on Matrix Analysis and Applications, vol. 7 no., pp. 6{6, 996. [] H. Salin, \Blind Signal Separation by Second Order Statistics" PD esis, Scool of Electrical and Computer Engineering, Calmers University of ecnology, Sweden 998.

5 ISR ISR Analytic Gaussian Sim. Non-Gaussian Sim Analytic Gaussian Sim. Non-Gaussian Sim [db] -30 [db] Figure : Simulations results for Gaussian ('o') and non-gaussian signals ('*') (and teoretically predicted results, solid lines) for SOBI and WASOBI, in terms of ISR, vs. te observation lengt. Source signals are MA() and MA(3) processes. Bot algoritms used te same data. Eac simulation point represents an average of 000 trials. [5] A. Yeredor, \Blind Separation of Gaussian Sources via Second-Order Statistics wit Asymptotically Optimal Weigting", accepted for publication in IEEE Signal Processing Letters, [6] M. Wax and J. Seinvald, \A least-squares approac to joint diagonalization", IEEE Signal Processing Letters, vol. no. 2, pp. 52{53, 997. [7] H. W. Sorenson, Parameter Estimation Marcel- Dekker, 980.

Symmetry Labeling of Molecular Energies

Capter 7. Symmetry Labeling of Molecular Energies Notes: Most of te material presented in tis capter is taken from Bunker and Jensen 1998, Cap. 6, and Bunker and Jensen 2005, Cap. 7. 7.1 Hamiltonian Symmetry