BLIND SOURCES SEPARATION BY SIMULTANEOUS GENERALIZED REFERENCED CONTRASTS DIAGONALIZATION Abdellah ADIB, Eric MOREAU and Driss ABOUTAJDINE Institut Scientifique, av. Ibn Battouta, BP. 73, Rabat, Maroc. adib@israbat.ac.ma SIS-TD, ISITV, av. G. Pompidou, BP 56, F-836 La Valette du Var, France moreau@univ-tln.fr GSCM-Faculté des Sciences, av. Ibn Battouta, BP. 4, Rabat, Maroc. aboutaj@fsr.ac.ma ABSTRACT In this contribution we generalize some links between contrasts functions and considering of a reference signal in the field of source separation. This yields a new contrasts that allows us to show that a function proposed in [4] is also a contrast, and frees us from the constraints on the introduced reference signal []. Associated optimization criteria is shown to have a close relation to a joint-diagonalization criterion of a matrices set. Moreover, the algorithm is of the same kind as JADE algorithm. Also, the number of matrices to be joint diagonalized is reduced in relation to SSARS algorithm [4] and to JADE one []. Simulations studies are used to show that the convergence properties of the new contrasts, even in real environment, are much improved upon those of the conventional algorithms.. INTRODUCTION The signal processing problem of separating observed mixtures, is known as blind source separation (BSS or Independent Components Analysis (ICA. BSS seeks to extract salient features and structure from a dataset which is assumed to be a linear mixture of independent underlying features. The goal of BSS is to unmix the dataset and recover these features. The problem is addressed extensively in the literature [3], [], [8], and different algorithms for a wide range of applications in speech processing, wireless communications, biomedical signal processing exist and recently in cosmic microwave background imaging. In statistical signal processing, the contrast functions are chosen with respect to statistical measures of independencies, and their optimization corresponds to separating states [3], [5], []. Recently, many authors have proposed methods to generalize some existing contrasts to separate mix- That work has been accomplished when Abdellah Adib was as invited professor at the SIS-SD laboratory, during October. tures of statistically independent signals [], [7]. In [7] the results leading to the most popular JADE and STOTD algorithms are extended to cumulants of any order R 3 what unveiled, in first, a unified framework for the above known results and secondly a generalization of some links between contrasts and joint-diagonalization criteria. A new family of contrasts [] baptized R-contrast was introduced by hiring an auxiliary system, Reference system in the unmixing scheme. This procedure permits to generalize a great number of the existing contrasts and requires thus less prior information than the preceding methods. In this last decade, the introduction of of a reference system both in source separation and blind equalization schemes has reveal marvellous results in signal processing problems [4], []. In this context, the use of higher order statistics, between the observations and the introduced reference, forces the statistical independence of the separated components. The one developed in [4] and called Blind Source Separation Algorithm with Reference System (SSARS is derived in order to maximize the same criterion as in JADE in the sense that it is based on simultaneous diagonalization of some matrices based on crosscumulants built from the output data of the separating system and the reference; but no link with contrast functions was evidenced. The R-contrast, proposed in [] I f R,z i (y i = f( Cum{y i,y i,z i,...,z }{{} i } with i {,...,N }, had R times been adaptively maximized over the space of white random signal y i (n and for given reference signal z i (n. This technique, a deflation approach intends to extract estimator of one of sources with each stage i. This method globally works better both in convergence speed and in steady state in comparison with the conventional ones. In this contribution we are mainly interested to constitutes unified framework for the JADE algorithm. This is achieved by the proposition of a new family of R-contrasts
for an arbitrary order R 3 of cross-cumulants and with any restriction made on the reference system. The innovation in our technique consists in using less prior information and debases the computational cost of the preceding methods; the number of matrices to be joint diagonalized is reduced in relation to SSARS algorithm [4] and to JADE one []. Also the proposed family yields a new R-contrast that allows us to show that a function proposed as a separating criteria in [4] is also a contrast, and frees us from the constraints on the introduced reference signal [4], [].. SYSTEM MODEL We consider the classical model that navigates instantaneous BSS task, N source signals, pass through an unknown N-input, N-output linear time-invariant mixing system G to yield the M mixed signals x i (n. Defining the random vectors with zero mean and finite covariance, a(n = [a (n...a N (n] T vector of sources with components being of unit power and statistically independent, x(n = [x (n...x N (n] T vector of observations and b(n = [b (n...b N (n] T vector of additive noises, the assumption of linear model becomes x(n =Ga(n+b(n ( The source signals are moreover assumed to possess non zero cumulants of order under consideration. The goal of the BSS task is to calculate a square demixing system H such that the outputs of this system given in vector form by y(n =Hx(n ( with y(n = [y (n...y N (n] T contain estimates of the N source signal sequences in a(n. Because of the only assumed independence property of the source signals some indeterminacies occur and sources are said separated if vector y(n is equal to a(n up to the multiplication of a permutation matrix and to the multiplication of an unitary diagonal matrix. In this framework it is useful to define the global mixing matrix S as y(n =Sa(n (3 and thus S = HG is a N N matrix. In the above sense, the separation holds when S = DP where P is a permutation matrix and D an unitary diagonal matrix. 3. NEW R-CONTRAST Let us consider z(n as the output of another separating matrix, cf Fig., defined according to z(n =H r x(n =Ta(n (4 a a.. a N G x(n H H r Fig.. Mixing/Unmixing scheme. y(n z(n where the unitary matrix T = H r G which corresponds to a second global system is assumed not equal to S. Following [4], the signal z(n is called the reference signal and H r the reference matrix. Using our notations, the following objective function was proposed in [4] J z (y = Cum{y i,yi,z j,zj }. (5 i,j= We here propose a generalization of this objective function as J R,z (y = C R {y i,z j }. (6 where R 3 and i,j= C R {y i,z j } = Cum{y i,yi,z j,zj,...}. (7 }{{} R terms In the above cumulant definition, the terms corresponding to z i are considered alternatively with no conjugation and with a conjugation. Clearly for R =4, J 4,z (y J z (y. We can now propose the following first result. Proposition For a given signal z(n, the function J R,z (y with R 3 satisfies the following inequality J R,z (y J R,z (a. (8 Proof. Let us first introduce the following function where K R,z (y = i,i,j= C R {y i,i,z j } (9 C R {y i,i,z j } = Cum{y i,yi,z j,zj,...}. ( }{{} R terms According to (, (4, the multilinearity property of cumulants and the independence of source signals, we have C R {y i,i,z j } = S i,lsi,lw j,l C R {a l,a l } ( l=
where W j,l = T j,l R if R is even and W j,l = T j,l R 3 T j,l if R is odd. Using ( in (9, we have K R,z (y = l,l = N i= S i,l Si,l ( N j= W j,l Wj,l C R {a l,a l } C R {a l,a l }. ( Because S is a unitary matrix, i= S i,l Si,l = δ l,l where δ l,l =if l = l and otherwise. Hence K R,z (y = l,j= W j,lc R {a l,a l } = l,j= C R{a l,z j } (3 = J R,z (a. Finally, as it is clear that J R,z (y K R,z (y then using the result in (3 in this inequality leads to proposed result. Now it is easy to see that J R,z (DPy =J R,z (y where P is any permutation matrix and D any unitary diagonal matrix. Hence according to the above proposition, the function J R,z (y has to be (globally maximized to get separation. However an important question subsists that concerns the characterization of the maxima. We have the following result. Proposition For a given signal z(n, the equality in (8 holds if and only if S = DP where P is any permutation matrix and D any unitary diagonal matrix, when. R =3or. R 4 and R even under the supplementary assumptions that all source signals possess R-th order cumulant of the same sign. Proof. Using ( with i = i = i, the equality in (8 can easily be written as l,l = ( N i= S i,l Si,l ( N δl,l j= W j,l W j,l C R {a l,a l }C R {a l,a l } =. (4 When R =3, then W j,l = T j,l and j= W j,l Wj,l = j= T j,l Tj,l = δ l,l because T is a unitary matrix. Thus, in that case, relation (4 becomes ( S i,l 4 C 3 {a l,a l } =. (5 l= i= Because S is a unitary matrix and because at most one C 3 {a l,a l }, l =,...,N is potentially zero then relation in (5 holds if and only if S = DP with D and P as specified in the proposition (for more details see e.g. [5]. Now when R 4 and R even, then the cumulants C R {a l,a l }, l =,...,N are reals. Moreover if they all have the same sign then their two by two products are positive. Also in such a case W j,l = T j,l R and the term j= W j,l Wj,l in (4 is positive. Hence the equality (4 holds if and only if N i= S i,l Si,l = δl,l that is if and only if S = DP with D and P as specified in the proposition. Notice that in the real case and using strictly the same derivation as above, the constraint that R is even in the second point of proposition can be relaxed. Notice also that one can consider in the second point of proposition that at most one R-th order cumulant can be null. Thus according to the two above propositions the function J R,z (y can be claimed to be a contrast. For R =3this is a general result while for R 4 we need an additional assumption which is fortunately not too restricting. Indeed for example in digital telecommunication problems, sources signals are often with a negative fourth-order cumulant. A general result for R 4 is again an open problem. 4. LINKS WITH JOINT DIAGONALIZATION Because of the usefulness of the joint diagonalization technique in the field of the source separation, one shows the close link between the criterion L R,z (y and this technique. Definition Considering a set of M square (N N matrices M(i,i =,...,M denoted by M. A jointdiagonalizer of this set is an unitary matrix that maximizes the function D(H, M = M diag(hm(ih T (6 i= Proposition 3 Let M x R,z be the set of N matrices C R,zi (x =C l,l (i with R 3 and defined as C l,l (i =Cum[x l,x l,z i,...,z }{{} i ] (7 R times then if H is an unitary matrix, we have D(H, M x R,z =L R,z (Hx (8 Proof. According to y = Ha with H =(h l,j and because of the multi-linearity of cumulants, we have Cum[y i,y i,z i...,z i ]= h i,l }{{} h i,l C l,l (i l R times,l (9 Hence combining (9 and (5 we get L R,z (Hx = i,i l,l j,j h i,l h i,l ( h i,j h i,j C l,l (i C j,j (i
H being unitary, the summation in the later equation can be reduced to L R,z (Hx = h i,l h i,l C l,l (i i i l,l = ( { HCR,zi [x]h T } i,i i i = i diag ( HC R,zi (x]h T = D(H, M x R,z ( 5. NUMERICAL RESULTS Computer simulations were carried out to illustrate the effectiveness of the proposed approach. In our computer experiment, we consider two case of sources with normalized unit power that are:. Two synthetic signals a 4-QAM and a 6-QAM constellations.. Three real audio speech sources with 89884 samples of female voices, all sampled at 44kHz The mixing matrices are randomly chosen, complex for the first case an real for the second one, and we transform them to an orthonormal ones in such a way that the pre-whitening stage can be dropped. The proposed approach presents good performance in recovering all the original sources for all the considered cases. We only take into consideration cumulants of order 3 and 4. The different estimated source signals are plotted on the four given figures. They reveal the success of the separation. Note that, the restitution is always achieved by diagonalizing N cumulant matrices and that is not directly the case for the two classical algorithms JADE and SSARS. For the case of QAM signals, in Fig.( we only consider fourth order cumulants, i.e. R =4. For the case of real audio speech, we use respectively: only fourth order cumulants in Fig.(3, only third order cumulants in Fig.(4 and both third and fourth order cumulants in Fig.(5. The global mixing/unmixig system takes respectively the following values For case of sources considering fourth order cumulants (.6.9984 S =.9984. For case of sources considering third order cumulants.4.989.3 S =.9893.5..3.3.9994 For case of sources considering fourth order cumulants S =......... For case of sources considering third and fourth order cumulants S =.988.8..3.5.999.7.9877.6 6. CONCLUSION We have generalized some links between contrast functions and a reference based approaches in the source separation domain. This allowed us to generalize some of the existing algorithms [], [4] and [5]. The main advantages of the proposed method is to yield a joint-diagonalization criterion of the same number of matrices (that is N whatever the order of the considered cumulants and frees us from the constraints on the introduction of reference signal [4], []. Simulations are used to illustrate the effectiveness of our method for the different types of sources and scripts. Fig.. First row, the original sources a 4-QAM and a 6- QAM, second row the observations and the third row represent the recovred sources
.5.5.5.5.5.5.5.5 3 3 x 5 x 5.5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5 3 x 5.5 3 x 5.5 3 x 5.5 3 x 5.5 3 x 5 Fig. 3. First row, the original sources real audio speech the recovred sources, by considering third order cumulants. Fig. 5. First row, the original sources real audio speech the recovred sources, by considering a combination of third and fourth order cumulants. 7. REFERENCES [] A. Adib, E. Moreau and D. Aboutajdine, A combined contrast and Reference Signal Based Blind Source Separation by a Deflation Approach, nd IEEE Int. Symp. on Sig. Proc. and Info. Techn., Marrakesh, Morocco, Dec.. [] J.-F. Cardoso and A. Souloumiac, Blind beamforming for non gaussian signals, IEE Proceedings-F, Vol. 4, pp. 36-37, 993. [3] P. Comon, Independent component analysis, a new concept?, Signal Processing, Vol. 36, No. 3, pp. 87 34, Apr. 994. Special issue on Higher Order Statistics..5.5.5 [4] M. Feng and K.D.Kammeyer, Blind source separation for communication signals using antenna arrays, in Proc. ICUPC 98, pp. 665 669, Florence, Oct. 998..5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5.5 3 x 5.5 [5] E. Moreau, Criteria for complex sources separation, in Proc. European Signal Processing Conference, (EUSIPCO 96, Trieste, Italy, pp 93-934, Sept. 996. [6] E. Moreau and N. Thirion-Moreau, Nonsymmetrical contrasts for source separation, IEEE Transactions on Signal Processing, Vol. 47, No. 8, pp 4-5, August 999. [7] E. Moreau, A generalization of joint-diagonalization criteria for source separation, IEEE Transactions on Signal Processing, Vol. 49, No. 3, pp 53-54, March..5 3 x 5.5 3 x 5.5 3 x 5 [8] C.B. Papadias, Equivariant adaptive source separation, IEEE Trans. Signal Processing, Vol. 48, No., pp 358-359, December. Fig. 4. First row, the original sources real audio speech the recovred sources, by considering fourth order cumulants.