IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 11, NOVEMBER Blind MIMO Identification Using the Second Characteristic Function

Transcription

1 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER Blind MIMO Identification Using the Second Characteristic Function Eran Eidinger and Arie Yeredor, Senior Member, IEEE Abstract We propose a new approach for the blind identification of a multi-input-multi-output (MIMO) system As a substitute to using classical high-order statistics (HOS) in the form of time-lagged joint cumulants, or polyspectra, we use the estimated Hessian matrices of the second joint generalized characteristic function of time-lagged observations, evaluated at several preselected processing-points These matrices admit straightforward consistent estimates, whose statistical stability can be finely tuned (by proper selection of the processing-points) in contrast to classical HOS Transforming the obtained matrix sequence into the frequency-domain, we obtain (and solve) a sequence of frequency-dependent joint diagonalization problems This yields a set of estimated frequency response matrices, which are transformed back into the time domain after resolving frequency-dependent phase and permutation ambiguities The performance of the proposed algorithm depends on the choice of processing-points, yet compares favorably with other algorithms, especially at moderate signal-to-noise ratio conditions, as we demonstrate in simulation results Index Terms Blind MIMO Deconvolution, blind MIMO identification, characteristic function, convolutive blind source separation, joint diagonalization, permutation ambiguity, phase ambiguity I INTRODUCTION BLIND identification of a Single-Input Single-Output (SISO) system is concerned with identifying an unknown linear, time-invariant mixing system, driven by an unobserved source, based on the system s observed output, and on the presumed independent, identically distributed (iid) time-structure of the source This paper is concerned with blind identification of a Multi-Input-Multi-Output (MIMO) system, which is denoted by the matrix coefficients and driven by unknown independent (usually non-gaussian) sources The identification is based on the observed outputs and exploits both the mutual independence of the sources and their individual iid time-structure The goal of blind MIMO system identification is usually to reverse the effect of the mixing system in order to recover the independent sources The term blind implies that no further information, such as the statistical distributions of the sources, is available Manuscript received August 25, 2004; revised January 12, 2005 Parts of this work were presented at the International Conference on Independent Component Analysis, ICA2004, Granada, Spain, September 2004 The work of E Eidinger was supported in part by the Yitzhak and Chaya Weinstein Institute for Research in Signal Processing The associate editor coordinating the review of this manuscript and approving it for publication was Dr Athina Petropulu The authors are with the School of Electrical Engineering, Tel-Aviv University, Tel-Aviv Israel ( erane@engtauacil; arie@engtauacil) Digital Object Identifier /TSP In the degenerate case of a memoryless system, the problem reduces to an instantaneous linear mixture, known as the basic, most simple version of a Blind Source Separation (BSS) model Therefore, blind MIMO identification can be seen as a two-fold extension of two different problems: BSS of instantaneous mixtures and/or blind SISO identification This problem is thus dually referred to as convolutive BSS and has applications in many fields, such as separating multiple speakers in a reverberant environment, where an array of microphones represents the various receivers, and the acoustics of the room are represented by the convolutive nature of each channel Additional fields in which convolutive BSS is of interest are communications, geophysics, acoustics, biomedical engineering [electroencephalography (EEG) signals separation, for example] etc Many of the existing BSS algorithms can be classified as either taking an inverse or a direct approach The inverse approach is based on trying to estimate the inverse system (applied to the observed signals) by gradually adapting its parameters, such that its outputs (estimated sources) maximize (empirically) some selected independence criteria (termed contrasts by Comon and often used by others [1] [4]) Such approaches are taken, eg, in Hyvärinen s FastICA family of algorithms [5] in the context of static BSS or in Tugnait s Inverse Filter criteria [6] (shown valid also for nonlinear input sources in [4]) in the context of convolutive BSS Conversely, direct approaches are aimed at direct estimation of the mixing system, based on collecting some set of statistics of the observed mixtures Many of these direct algorithms rely on a joint eigen-structure inherent in the set and use Singular-Value-Decomposition (SVD) or Joint-Diagonalization (JD) to estimate the mixing system In the context of static mixtures, such approaches are taken, eg, by Cardoso and Souloumiac in the Joint Approximate Diagonalization of Eigenmatrices (JADE) algorithm [7] based on fourth-order cumulants, by Belouchrani et al in the Second-Order Blind Identification (SOBI) algorithm [8] based on correlations at different lags, or by Yeredor in the CHaracteristic-function Enabled Source Separation (CHESS) [9] algorithm, based on Hessians of the second characteristic function In the context of convolutive mixtures, such approaches are taken, eg, by Comon and Moreau [10], Comon and Rota [11] (see also [12]) in the time-domain or by Chen and Petropulu [13], Shamsunder and Giannakis [14], and Rahbar and Reilly [15] in the frequency domain The approach proposed in this paper belongs to the family of direct approaches and works in the frequency-domain X/$ IEEE

2 4068 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER 2005 Many of the existing approaches to the Convolutive BSS problem (working either in the time- or frequency-domains) are based on high-order cumulants or polyspectra (eg, [16], [17] and [13]), which often (but not always) exhibit a relatively large estimation variance Cumulants are well-known to be the high-order derivatives (at the origin) of the second Generalized Characteristic Function (GCF) of the observations joint distribution (see the definition in the sequel) An interesting alternative to the use of high-order derivatives at the origin is the use of second-order derivatives at selected off-origin points Second (and higher) order off-origin derivatives of the second characteristic function 1 were first considered by Gürelli and Nikias in [18] (where they were termed Generalized Cumulants ) in the contexts of blind SISO identification and direction-of-arrival estimation However, the associated algorithms involved numerical differentiation for estimating the derivatives; thus, their appeal was probably somewhat overlooked ever since More recently, stemming from a different perspective, Yeredor proposed the use of second-order derivatives of the second GCF at selected off-origin points (termed processing-points ) in the contexts of static BSS and blind SISO identification (in [9] and in [19], respectively) It was shown in [9] that these derivatives actually admit straightforward consistent estimates, which, somewhat surprisingly, take the appealing form of specially weighted empirical covariance matrices (thereby numerical differentiation is not required) Thus, in this paper, we extend the GCF-based SISO identification algorithm of [19] into a MIMO algorithm called CHAracteristic function MIMO Blind Identification (CHAMBI) CHAMBI uses the empirical Hessian matrices of the second GCF (SGCF) and attains the required diversity of these raw statistics by choosing the processing-points from a continuous set rather than choosing discrete orders of derivatives (as higher order cumulants) Moreover, classical HOS-based algorithms must resort to (at least) fourth-order statistics whenever the sources are symmetrically distributed (or have otherwise null third-order cumulants) No such restriction applies to the CHAMBI framework With an educated choice of its processing-points, CHAMBI can be shown to outperform polyspectra based algorithms (PSA), especially at moderate Signal-to-Noise Ratio (SNR) conditions The paper is structured as follows Following the problem formulation in Section II, we discuss the SGCF and its derivatives in Section III and present a consistent estimator for the Hessian In Section IV, we transform the problem into the frequency-domain, outlining a JD-based solution along with procedures to resolve the associated frequency-dependent permutation and phase ambiguities A closed-form summary of the suggested CHAMBI algorithm is given in Section V, along with analysis of its computational complexity The CHAMBI and PSA algorithms were applied to exponentially distributed data in Monte Carlo simulation for different data lengths and SNR conditions, and the results are presented in Section VI Concluding remarks are summarized in Section VII 1 See Section III for a discussion of the difference between the Characteristic Function and the Generalized Characteristic Function II PROBLEM FORMULATION The following blind MIMO Finite Impulse Response (FIR) model of order is considered: where,, and are the sources, observations, and noise vectors, respectively is the mixing matrix at lag so that can be viewed as the impulse response of length from source to sensor The sources are assumed to be zero-mean and iid in time and mutually independent in space The noise is assumed to be zero-mean, spectrally white Gaussian with an arbitrary spatial covariance matrix, which is assumed to be known In many situations, can be estimated off-line when no sources are present In other cases, is sometimes known to have a special parametric structure, eg, when the noise is spatially white and In such cases, the noise parameters, such as, can sometimes be estimated from the data, eg, when more sensors than sources are available However, we will not pursue such approaches in this work, but rather assume that is known The number of observations is assumed equal to the number of sources for simplicity of the exposition, yet extension to other cases can be obtained through modification of some of the joint diagonalization stages (to be discussed later) The goal is to estimate, from observations, III SECOND GENERALIZED CHARACTERISTIC FUNCTION A Definition of the GCF and Its Extended Version The GCF of any random vector at a processing-point vector, which is denoted,isdefined as whenever this mean exists Note that the classical definition of the characteristic function (eg, [20, Sec 5-5]) uses a realvalued argument in a complex exponent, defining, which exists everywhere in The term generalized in this context is meant to allow a complex-valued argument, and therefore, is omitted from the exponent Existence of the GCF everywhere in is guaranteed for finite-support distributions, whereas for infinite-support distributions, the GCF may exist only in part of The GCF is sometimes also called the Moments Generating Function, but that term usually implies a real-valued argument ; hence, we prefer to use the term GCF The second GCF (SGCF) is then defined as We now assume that while the true channel length is not known, an upper bound is available (whose validity can be partly verified following the identification stage by checking (1) (2) (3)

3 EIDINGER AND YEREDOR: BLIND MIMO IDENTIFICATION USING THE SECOND CHARACTERISTIC FUNCTION 4069 that the trailing elements of the estimated system appear negligible) Let us define, which is an augmented version of of length, where : and its associated augmented processing-point vector is also of length : (4), which where is a sequence of selected processing points This augmented processing points vector is thus comprised of the concatenated vectors, where each is associated with,, respectively We now observe the augmented GCF (and the associated SGCF) as the joint GCF (joint SGCF) of consecutive samples of : (5) (6a) (6b) Consider the noiseless case first (we will address the noisy case later, essentially showing that the main result is unaffected by the presence of independent white Gaussian noise) Using, for convenience, an infinite sum in (1) (under the convention that for ), we have so that using, we obtain where can be interpreted as the response (at time ) of the MIMO system to the input sequence : (7) (8) The first (second) extended GCF-s is thus seen to be a product (sum) of GCF-s (SGCF-s) of the source vector at processing points, which is dependent on the unknown MIMO system, as well as on the selected processing points B Derivatives of the SGCF Differentiating once with respect to (wrt), and again wrt, yields (11a) (11b) where the vector and the matrix denote the first and second derivatives, respectively, of wrt at A key property to be exploited in the algorithm is the diagonality of the Hessian matrix of the sources SGCF In fact, the mutual independence of the sources implies that is diagonal for any, as shown in [9] The diagonality of stems from the fact that for independent sources the joint SGCF is the sum of marginal SGCFs of the individual sources at the respective elements of Thus, the second derivative with respect to any two different elements of vanishes Note that for any random vector, the first and second derivatives of the SGCF at the origin, namely, and equal the vector s mean and covariance, respectively Therefore, an educated choice of can lead to first and second derivatives that for most values of are equal to the sources mean and covariance, respectively Specifically, if so that (12) where is an arbitrary processing point (selected such that the GCF exists) then, by (9) (9) ow (13) From (6a), (6b), and (8), we obtain, due to the iid timestructure of the sources (and dropping the time index in the subscript, due to stationarity) which leads to ow (14a) ow (14b) (10a) (10b) Since, for our choice of, and depend only on, the subscript will, from now on, be replaced with Equation (11a) is thus reduced to (15)

4 4070 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER 2005 Moving on to, note that the expression in (11b) is reminiscent of the equation describing the relation between (the correlation sequence of the system s output) and (the correlation matrix of the white input) In fact, defining the diagonal matrices, (11b) can be restated as follows: (16) Concentrating on the second derivative (Hessian) with different choices of, we would obtain different matrix-sequences, all satisfying a similar multiplicative relation in the frequency-domain with different diagonal matrices This hints that a least-squares (LS) Approximate Joint Diagonalization (A-JD) approach may be pursued in the frequency-domain in order to identify the MIMO system (further discussion is deferred to Section IV-C) In order to pursue this approach, we will first propose consistent estimators for in the following subsection C Estimating the Hessian Matrices Luckily, a straightforward consistent estimate of at (for, ) exists under mild ergodicity conditions The Hessian of the SGCF (at any ) is given for all by (17) Further defining all ), we have (for (18) where the vector (21) which is the main result to be exploited by the proposed algorithm Before proceeding, we now consider the validity of (18) in the noisy case It is easily verified that in the presence of additive white Gaussian noise (AWGN), independent of the sources, (17) becomes (for all ) (19) The presence of the right-most additive term is due to the independence of the noise from the sources The fact that this term is invariant in stems directly from the Gaussianity of the noise: Since, for the zero-mean multivariate Gaussian distribution, the GCF is also Gaussian, its log (the SGCF) is quadratic, and the Hessian is always constant (and equals the covariance) Thus, the Hessian does not depend on the processing points Moreover, the output correlation matrix is similarly contaminated, so that (16) becomes and the matrix (22a) (22b) denote first and second derivatives, respectively, of the GCF By substituting the GCF and its first and second derivatives in (21) with the following empirical averaging estimators, the Hessian of the SGCF (for ) is consistently estimated (assuming ) (23a) (23b) (20) Therefore, the constant noise contamination matrix is cancelled when is subtracted from, and (18) holds for the noisy model as well As can be seen from (11a) and (18), both the first and second derivatives, respectively, exhibit a structure of convolution that would translate into multiplication in the frequency domain (23c) While not necessarily optimal, these are evidently consistent estimators (under mild ergodicity conditions), implying consistent estimation of the Hessian

5 EIDINGER AND YEREDOR: BLIND MIMO IDENTIFICATION USING THE SECOND CHARACTERISTIC FUNCTION 4071 An interesting observation is that the proposed estimator can be expressed as a specially weighted empirical covariance matrix following some straightforward manipulations (see [9]) Likewise, defining (28) (24) where are the weights, and are the weighted means For, the weights are guaranteed to be real-valued and positive, thus obeying the intuitive notion of weights For complex-valued, the weighting may become intuitively obscure, yet the expressions remains valid IV FREQUENCY-DOMAIN SEPARATION A Transforming Into the Frequency Domain To motivate the transformation into the frequency-domain, we now introduce the concept of joint diagonalization (JD) Consider a set of some target matrices The JD problem consists in finding a single matrix and a set of diagonal matrices, such that (25) Of course, there is no guarantee that such a set and such a matrix exist Often, while a set is known to admit such a structure, only an estimate of this set is accessible Approximate Joint Diagonalization (A-JD) is then the problem of finding an approximate joint diagonalizer and a set of diagonal matrices, such that (25) holds (for the estimated counterparts) as closely as possible Several A-JD approaches have been recently proposed in the literature (eg, [21] [25]), where each is associated with different criteria for closeness to the state of exact JD While the structure of the matrices in (18) is reminiscent of the JD framework, the multiplication is replaced with a two-dimensional (2-D) convolution Thus, to establish a JD framework, (18) may be transformed into the frequency-domain, eg, by applying an -point 2-D Discrete Fourier Transform (DFT) to for, (equal to the 2-D Discrete-Time Fourier Transform (DTFT) sampled at the respective Fourier frequencies, ) Using the following frequency-domain notation for the matrices involved (the observations spectral matrix), we have, in the noisy model (29) B Prewhitening One of the most popular (although not necessarily statistically optimal) approaches for A-JD involves a whitening stage, in which a whitening matrix is found (using the eigenvalue decomposition of one of the matrices in the target set), and the entire set is transformed accordingly, leaving only an unresolved unitary matrix factor This unitary factor is in turn identified by some iterative unitary A-JD algorithm, eg, the one proposed by Cardoso and Souloumiac [21], based on sequential Jacobi rotations Due to its computational appeal, this A-JD approach will be pursued in this paper as well, applying whitening and unitary A-JD separately at each frequency Noting that only the Fourier frequencies are of interest (since points in the frequency domain are sufficient for determining the time-domain matrix coefficients), from here on, the argument is dropped and replaced with a discrete index Define such that (for the noisy model) (30) is easily associated with the eigenvalue decomposition of the matrix We thus have is evidently a spatial whitening matrix (at each frequency ) and, considering the working assumption that all sources have unit variance (namely that ), the matrix is unitary (at each frequency ) Defining the following transformed matrices: (31) it then follows from (27) that the following relation holds (for all ): (32) it can be readily verified that (18) transforms into (26a) (26b) (26c) C Formulation as a JD Problem Essentially, (32) is reminiscent of a unitary JD problem for all Note, however, that in the right-hand expression, we have, rather than Therefore, some further manipulation is required, which is also aimed at mitigating frequency-dependent phase and permutation ambiguities Choosing as some integer, we define (at each frequency ) a set of matrices, each related to a possibly different processingpoint : (27) (33)

6 4072 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER 2005 where the last equality dwells on the unitarity of,as Note that is a diagonal real-valued matrix We now have, at each frequency, a set of target matrices corresponding to different choices of, which are jointly diagonalizable by a unitary matrix The aforementioned unitary A-JD algorithm can now be applied to estimates of the set Note that consistent estimates of the target set would lead to consistent estimates of (up to the inherent permutation and phase ambiguities) under regularity conditions that are almost always satisfied (see [13]) D Inherent Ambiguities In theory, by finding the joint diagonalizer, we can reconstruct the frequency response at each frequency and, thus, the overall system response However, note that if is a unitary joint diagonalizer, then due to the structure of (25), so is (34) where is a permutation matrix, a real-valued diagonal phase matrix, and a diagonal matrix whose diagonal elements are the exponentials of the diagonal of There usually also exists a scaling ambiguity that is inherent in the problem of BSS, which is resolved here implicitly by the prewhitening (ie, assuming the sources to be of unit variance) This means that for the set of (at frequency ), the estimated joint diagonalizer can be used to estimate the original system response matrix up to frequency-dependent permutation and phase ambiguities as follows (the scaling ambiguity still exists but is not frequency-dependent): (35) where is the estimated whitening matrix that is obtained from (30) by replacing with its consistent estimate The subscript in implies the existence of frequency-dependent ambiguities E Resolving the Permutation and Phase Ambiguities 1) Permutation Ambiguity: The permutation ambiguity can be easily resolved thanks to the structure of the diagonal matrices in (33), which evidently do not depend on, but only on the choice of (which is identical for all frequencies) Thus, by ordering the diagonals in the same hierarchy, eg, in increasing order, one can impose the same permutation matrix at all frequencies, and becomes simply Such a frequency-independent permutation is acceptable because it merely implies reordering of the sources A word of caution is in order While ordering the rows of a matrix at each frequency by the order of the diagonal of one specific diagonal matrix (for a specific and ) is enough for the exact JD problem, when using A-JD, the elements of the diagonals are distorted by estimation errors, and relying on just one choice of and for ordering the rows of may result in faulty permutations Thus, in an A-JD problem, it is best to rely on several (or even all) of the diagonal matrices resulting from the A-JD algorithm For example, we used the following method that proved quite sufficient (working separately at each frequency ): Each diagonal matrix (corresponding to choices of and ) was used to produce a possibly different permutation matrix, as described above Then, the element-wise average of the resulting permutation matrices was rounded to the nearest permutation matrix (in the LS sense) Note that in the 2 2 case, this operation reduces to a simple majority vote (at each frequency) between the diagonal matrices that warrant a swap and those that do not (and under good enough conditions, these votes are usually decided by an overwhelming majority at each frequency) 2) Phase Ambiguity: Once the permutation ambiguity is resolved, (35) can be rewritten as, and the phase ambiguity can be resolved, once again due to the fact that the diagonal matrices are frequency invariant The ambiguity is resolved in a fashion similar to the one in [13] For any integer, consider the following diagonal matrix, whose diagonality is implied by (27) and by the property : (36) Bearing in mind that is a diagonal phase matrix, we define phase matrices for the other diagonal elements in (36) (37a) (37b) where denotes the phase of, taking values in the range The phase of (36) can then be rewritten as (38) where the special notation, denotes equality up to possible addition of some (possibly different) integer multiples of to each element Summing (38) over, the sums of and cancel each other, and we have which can be rewritten as (39) (40) where is some diagonal matrix of integers Since we have access to, this means that can be estimated from the observations, up to an ambiguity term of Its ambiguity-prone estimator is denoted as the diagonal matrix (41) Define now, for each diagonal element,, vectors and, which are comprised of phases at all nonzero

7 EIDINGER AND YEREDOR: BLIND MIMO IDENTIFICATION USING THE SECOND CHARACTERISTIC FUNCTION 4073 frequencies, extracted from the diagonals of and, respectively, for : (42a) (42b) Note that (37a) implies the following relation (exploiting the -periodicity in of the sequences and ): (43) where is an full-rank matrix, constructed by removing the last column and last row of the circular Toeplitz matrix whose first column is and where denotes an all-zeros vector (of the appropriate dimensions) with a 1 at the th location denotes an all-ones vector This relation implies that the above mentioned ambiguityprone estimate of can be used to construct ambiguityprone estimates of The following proposition suggests such estimates, implying that the only remaining phase ambiguity is a tolerable linear-phase Proposition 1: Let and be co-prime, and let denote the ambiguity-prone estimate of obtained from (41) Let be given (for )by Then (44) (45) where is some integer Proof: See the Appendix According to Proposition 1, by choosing such that it is co-prime with, the phase matrices can be reconstructed (from the respective estimates ) up to some phase linear in and a constant phase (and an irrelevant integer multiple of ) Therefore, if denotes the diagonal matrices resulting from applying Proposition 1 to the estimates of and, then the final reconstruction of the system is (46) where is some diagonal matrix of integers, containing the implied (integer) circular time shifts F Inverting Back Into the Time Domain When an inverse DFT is applied to, the estimate of is obtained up to an overall permutation ambiguity, an integer circular time shift, and a constant phase shift for each channel The integer circular time shift, while tolerable for purposes of system identification, may induce distortion (other than a simple time-shift) of the respective separated sources This happens whenever the circular time shift splits the respective impulse responses into two sections, where the first section appears at the end of the resulting buffer, and the trailing section appears at the beginning of that buffer Such a split can be mitigated by observing the extended estimated channel of length and identifying a silent interval, where the estimated elements nearly vanish A corrective circular time shift may then be applied in order to properly align the estimated channel such that the silent interval is moved to the end of the resulting buffer It has to be noted that since the diagonal phase-matrix multiplies on the right, the constant phases and circular timeshifts are identical within each column of the identified system matrix Namely, while ambiguous relative time shifts between sources are nondistinguishable, relative time shifts of the same source between different sensors are not ambiguous Consequently, the procedure described above should be applied such that one common corrective time shift is determined for each column of V CHAMBI ALGORITHM SUMMARY We now summarize our proposed CHAMBI algorithm We assume that a number and a set of processing-point vectors have been selected 1) Obtain consistent estimates and (eg, using a -windowed correlogram) of the observations auto-correlation and spectrum, respectively Compute,, which is the frequency-dependent spatial whitening matrix, by EVD, according to (30) 2) For the set of processing-point vectors, evaluate (for each ) by (24), and by subtracting, for, Note that since the true are known to be zero for, there is no need to compute for such values of and, as these estimates should be set to zero 3) Obtain by (26a) 4) Using and, compute the transformed matrices (for, ) according to (31) 5) For each and for each, obtain an average predicted diagonal matrix as follows: Average all the diagonal matrices obtained from the SVD-s of for (due to the uniqueness of the SVD, these would be consistent estimates of ) 6) Grade the matrices (obtained for each pair and ) based on the distinctness of their diagonals (eg, the grade each matrix obtains may be the minimum absolute distance between the elements on its diagonal) 7) Choose a subset of best graded (with the highest grade) pairs of,

8 4074 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER ) At each frequency,define the target set of matrices for A-JD ( ) by (33) Apply a unitary JD scheme to the set and, thus, obtain estimates for the unitary factor, leading to estimates, as in (35) 9) Resolve the permutation ambiguity by an LS fitting to the nearest index-invariant permutation matrix, as described in Section IV-E-1 10) Choose some, and evaluate by (36) (retaining its diagonal elements only) Choose some co-prime with, construct the matrix, and evaluate the phase ambiguity as in (44), yielding the estimate, which is free from any intolerable frequency-dependent ambiguities (namely, it has the form of (46)) 11) Apply an inverse DFT to finally obtain, which is an estimate of the MIMO system (possibly scaled, circularly shifted, and multiplied by a constant phase) We supplement this description with a rudimentary complexity analysis of each stage of the proposed algorithm (recall that denotes the number of sources/sensors, denotes the number of available time samples, denotes the presumed upper bound on the system s length, and denotes the number of selected processing-points): Straightforward estimation of the correlation matrix at each time lag requires a total of operations, as does the estimation of the Hessian matrices (since it is equivalent to a weighted empirical covariance matrix ), requiring operations Applying DFT to both Hessians and correlation matrices requires, using the FFT algorithm on each element, of each matrix There are Hessian matrices so that a total of operations are required Computing involves matrix multiplication of matrices, thus taking Calculating the SVD to obtain the diagonals requires Grading the diagonal matrices is a simple sort of operation with a complexity of Applying Unitary JD takes Using the method suggested in Section IV-E1 for resolving the permutation ambiguity takes, although other methods may be used The suggested method for resolving the phase ambiguity takes Finally, an inverse DFT is applied, requiring another operations The overall complexity can thus be approximated by VI SIMULATION RESULTS 2 nonmin- We ran 200 Monte Carlo trials, identifying a 2 imum-phase system (48) (where ), which is the same system used in [13] In the first experiment, we used zero-mean, unit variance sources with one-sided exponential distributions in order to enable a fair comparison with the results of the polyspectra slices algorithm (PSA) reported in [13] (using the same setup), which was chosen for comparison because it is relatively widely used Additive, spatially and spectrally white Gaussian noise was applied to each sensor, and SNR was measured as the ratio between the average sensor power and the noise variance We chose (so that ) and processing points, spread over half a circle such that with Out of possible matrices at each frequency, the best graded 2 matrices were chosen for the JD matrix set For resolving the phase ambiguity, we chose (co-prime with ) Assuming an unknown channel length, for performance analysis, we used an estimated channel length of, thus truncating the impulse response of length (obtained from the inverse DFT) to length The filters were artificially aligned before truncation The performance measure per channel is the Normalized Mean Square Error (NMSE), which is defined as the total square absolute estimation error over all taps (truncated to length ), normalized by the total energy of the taps Results are presented in terms of the Overall NMSE (ONMSE), which is the NMSE averaged over all channels (and all Monte Carlo trials) Fig 1 depicts the performance of the CHAMBI algorithm in the frequency domain, demonstrating the mean and standard deviation of the magnitude of the estimated system response Fig 2 shows the same results, in a boxed format in the time domain, where possible phase discrepancies are factored in (as opposed to Fig 1) Fig 3 compares the performance of CHAMBI to PSA in terms of ONMSE versus the SNR 3 CHAMBI is seen Asymptotically, as computational complexity of (47), the dominant term implies 2 The grade was computed as grade(d) =min j log(d =D )j, which in our case reduces to j log(d =D )j 3 It is to be noted that the results obtained in our implementation of the PSA algorithm are slightly better than those reported in [13] at the respective SNRs this may be due to the fact that some of the implementation details not reported in [13] were slightly different in our implementation Anyway, since all variations are in favor of PSA, we did not further explore these slight differences

9 EIDINGER AND YEREDOR: BLIND MIMO IDENTIFICATION USING THE SECOND CHARACTERISTIC FUNCTION 4075 Fig 1 Estimation of the system in (48) by CHAMBI, with SNR = 20 db, using T = 8192 The magnitude of the frequency response is presented True: Dashed CHAMBI empirical mean: Solid CHAMBI empirical standard-deviation: Light fill Based on 200 Monte Carlo trials Fig 2 Same setting as Fig 1 The truncated impulse response is presented True: dots CHAMBI: filled boxes, centered around the empirical mean, and demonstrating the empirical standard deviation by their height Based on 200 Monte Carlo trials to outperform PSA and suffers less degradation than the polyspectra algorithm at the lower SNR It should also be noted that while the third-order PSA ([13]) cannot accommodate symmetric distributions, CHAMBI has no such restrictions (namely,

10 4076 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER 2005 Fig 3 Dependence of the ONMSE (over 150 Monte Carlo trials) of the CHAMBI and PSA algorithms on the SNR of the data is shown here in the same setting as Fig 1 (SNR = 20 db, T = 8192) CHAMBI ONMSE: Triangles PSA ONMSE: Circles Both algorithms were applied to the same data Fig 5 CHAMBI ONMSE for symmetric (Laplacian) sources versus the imaginary-valued radius of the processing-points semi-circle Results are presented for two cases: one using the raw complex-valued estimates C [k; `] and one using only their real part Averaged over 300 Monte Carlo trials, in each trial, the different radii and different matrices were used with the same data (noiseless case T = ) Fig 4 CHAMBI ONMSE (over 500 Monte Carlo experiments) dependence on the radius of the semi-circle on which the processing-point vectors are spread (SNR =30dB, T = 4096) the nonsymmetric distribution was merely used in order to enable comparison to third-order PSA) For symmetric sources, PSA would be forced to resort to higher order polyspectra, which generally (but not always) admit less accurate estimates The performance of CHAMBI with symmetrically distributed sources will be presented shortly Fig 4 demonstrates the sensitivity of the CHAMBI algorithm to the choice of processing points (for the same system, with SNR db and ) Spreading all processing-points evenly on half a sphere around the origin, the radius of that sphere was varied to observe the effects on performance Intuitively, as the radius grows and processing-points depart from the origin (in ), the variance in the estimation of the Hessian at these points increases Conversely, when the radius is decreased and all the processing-points collapse to the origin, the accuracy in the Hessian estimates improves, but the information content in the set of Hessians decreases, as they become quite similar (evaluated at a clustered set of points), and diversity is lost Hence, an optimal trade-off is sought between the diversity of the A-JD set of equations and the variance in estimating the Hessians In our specific scenario, the optimal radius is seen to be reached around 0065 In a second experiment, we tested the algorithm with the same system, but with symmetrically-distributed sources: a case that cannot be handled by the third-order based PSA approach The specific sources distribution was unit-variance Laplacian (double-sided exponential) For this experiment, we used eight processing points that were evenly spread on half a sphere with an imaginary-valued radius, ranging from to, and the ONMSE is presented versus that radius (for the noiseless case) in Fig 5 Note that performance in this case is presented for a longer observation interval, and it is seen to be generally worse than that obtained with the asymmetric sources It is interesting to note that dramatic improvement in performance can be attained in the symmetric-sources case by employing the following observation: When the processing-points are imaginary valued, the true set of matrices can be easily shown to be real valued whenever all the sources have a symmetric distribution Consequently, in such cases, provided that the sources are known to be symmetric, we may take just the real part of the estimates thus eliminating the variance induced by the futile estimation of the imaginary part and improving the overall performance The resulting performance (versus the imaginary radius length) is also presented in Fig 5 Note, however, that these improved results are only applicable when the additional knowledge as to the sources symmetry is available, which slightly breeches the blindness framework This improvement cannot be attained when not all sources are symmetric, since in such cases, taking just the real part of the estimated matrices would cause a severe bias, as the true matrices would generally not be real valued (of course, only when complex-valued or imaginary-valued processing-points are used)

11 EIDINGER AND YEREDOR: BLIND MIMO IDENTIFICATION USING THE SECOND CHARACTERISTIC FUNCTION 4077 TABLE I RESULTS FOR RANDOMIZED SYSTEMS (BASED ON 250 TRIALS): PERCENTAGE OF SUCCESSFUL TRIALS (WITH RESULTING ONMSE SMALLER THAN 02), AND AVERAGE ONMSE AMONG SUCCESSFUL TRIALS In the third experiment, we tested CHAMBI with randomized systems to verify that reasonable performance is also attained with the same processing points for systems other than the one previously tested (48) Thus, in each trial, a 2 2 system of order was generated by independently drawing the five taps of each subsystem from a zero-mean uniform distribution and normalizing the energy to 6 so that the power of each observation signal would roughly resemble the power obtained with the system in (48) Asymmetric (zero-mean, unit variance, one-sided Exponential) sources were also randomly generated at each trial CHAMBI was operated with the same parameters as in the first experiment In presenting the results, we must consider the fact that the performance often exhibits a threshold effect, whereby a small degradation in the statistical stability of the intermediate estimates can cause significant degradation in the overall ONMSE performance This is mainly due to the fact that any error in resolving the permutation ambiguity, even at a single frequency, can have a devastating effect on the time-domain estimates, thus significantly increasing the ONMSE Similar degradation can be caused by errors in resolving the phase ambiguity Consequently, when the processing points are predetermined, and the system is randomized, cases of severe outliers are inevitable, yet their occurrence becomes more rare as the observation length increases We chose to define successful estimation as cases where the ONMSE is smaller than 02 In Table I, we present the percentage of cases of successful estimation (based on 250 trials) versus different values of the observation length, as well as the average ONMSE among the successful estimations (only) VII CONCLUSIONS We proposed a new algorithm for blind MIMO identification/bss of convolutive mixtures, based on the Hessians of the second GCF, evaluated at different user-specified processingpoints With a specific family of processing points, the straightforward estimates of the Hessians, consistent under commonly met conditions, turn out to be specially weighted covariance matrices at different lags Following Fourier transformation of the estimated matrix sequences, joint diagonalization is applied in the frequency-domain in order to estimate the unknown system, up to (possibly) frequency-dependent permutation and phase ambiguities By exploiting the frequency invariance of the diagonal matrices involved, we proposed algorithms for resolving these ambiguities Performance naturally depends on the choice of processing points, and with proper selection thereof, the proposed algorithm was shown to outperform a polyspectra based algorithm, especially at moderate SNR conditions Somewhat unduly, the fact that this new approach offers extended freedom in selecting arbitrary processing points (away from the origin) constitutes its main drawback: Proper (let alone optimal) selection of the processing points is yet obscure Currently, we can only provide rough guidelines: When the power of all the observed mixtures is approximately equal (otherwise, they may be normalized first), spreading the processing-points on a sphere seems like a reasonable option As a rough rule of thumb, the radius of that sphere should be selected so that its product with the observations mean absolute value be in the order of 1 In the absence of a specific selection strategy, the sensitivity to the selection of processing points can be somewhat restrained by applying a proper weighting scheme that would outweigh bad processing points Thus, further potential directions for enhanced exploitation of this new tool may include a better-specified data-adaptive selection of processing points; possible weighting of the collected statistics; possible use of first-order derivatives; and better optimized joint diagonalization schemes (eg, rather than solve individual joint diagonalization problems at each frequency bin, apply global joint diagonalization by exploiting the frequency-invariance of the diagonal matrices, thus improving the estimation accuracy and inherently mitigating the frequency-dependent ambiguities) APPENDIX Proof of Proposition 1: Recall that (38) can be reformulated, for each ( th) diagonal element, as (49) where is an full-rank matrix, constructed by removing the last column and last row of the circular Toeplitz matrix whose first column is, and where denotes an all-zeros vector (of the appropriate dimensions) with a 1 at the th location As an example of the structure of, observe the case of and : (50) Eliminating the dot equality from (49), we may rewrite the same relation as where ) (51) is some vector of integers (possibly different for each

12 4078 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL 53, NO 11, NOVEMBER 2005 Note that is a nonsingular matrix whenever and are co-prime, and its inverse can be shown to be a binary matrix, containing only ones and zeros [26] Therefore, substituting (41), (40), and (44) into (51), and multiplying both sides on the left by, yields Consequently, (52) reduces to which satisfies Proposition 1 (with ) (55) REFERENCES (52) where and are defined in (41) and (44), respectively Observing the last two terms in (52), note that since is a binary matrix, comprised of zeros and ones, we have, where is some other vector of integers Further, note that since by simple summation over the rows of, it is easily observed that Furthermore, consider the following Lemma Lemma 1: There exists some positive integer satisfying (53) Proof of Lemma 1: It is enough to show that there exists some such that the implied vector satisfies the relation Indeed, observing the structure of,we have (54) A fundamental theorem of Number Theory states that the set of positive integers smaller than and co-prime with is a group under mod multiplication Therefore, has an inverse under mod multiplication, namely, there exists some such that This proves the Lemma [1] P Comon, Independent component analysis, A new concept?, Signal Process Special Issue on Higher Order Statistics, vol 36, no 3, pp , 1994 [2], Contrasts for multichannel blind deconvolution, IEEE Signal Process Lett, vol 3, no 7, pp , Jul 1996 [3] E Moreau and J-C Pesquet, Generalized contrasts for multichannel blind deconvolution of linear systems, IEEE Signal Process Lett, vol 4, no 6, pp , Jun 1997 [4] C Simone, P Loubaton, and C Jutten, Separation of a class of convolutive mixtures: a contrast function approach, Signal Process, vol 81, pp , 2001 [5] A Hyvärinen, Fast and robust fixed-point algorithms for independent component analysis, IEEE Trans Neural Networks, vol 10, no 3, pp , May 1999 [6] J K Tugnait, Indentification and deconvolution of multichannel linear non-gaussian processes using higher order statistics and inverse filter criteria, IEEE Trans Signal Process, vol 45, no 3, pp , Mar 1997 [7] J-F Cardoso and A Souloumiac, Blind beamforming for nongaussian signals, Proc Inst Elect Eng F, vol 140, no 6, pp , 1993 [8] A Belouchrani, K Abed-Meraim, J-F Cardoso, and E Moulines, A blind source separation technique based on second order statistics, IEEE Trans Signal Process, vol 45, no 2, pp , Feb 1997 [9] A Yeredor, Blind source separation via the second characteristic function, Signal Process, vol 80, no 5, pp , 2000 [10] P Comon and E Moreau, Blind MIMO equalization and joint diagonalization criteria, in Proc ICASSP, vol 5, Salt Lake City, UT, 2001 [11] P Comon and L Rota, Blind source separation: a pertial joint diagonalization (pajod) approach, in Proc Third Workshop Physics Signal Image Processing, Grenoble, France, 2003 [12] P Comon, E Moreau, and L Rota, Blind separation of convolutive mixtures: a contrast based joint diagonalization approach, in Proc Third Int Conf Independent Component Analysis Blind Source Separation, 2001, pp [13] B Chen and A P Petropulu, Frequency domain blind MIMO system identification based on second- and higher order statistics, IEEE Trans Signal Process, vol 49, no 8, pp , Aug 2001 [14] S Shamsunder and G B Giannakis, Multichannel blind signal separation and reconstruction, IEEE Trans Speech Audio Process, vol 5, no Nov, pp , 1997 [15] K Rahbar, J P Reilly, and J H Manton, Blind identification of mimo fir systems driven by quasistationary sources using second-order statistics: a frequency domain approach, IEEE Trans Signal Process, vol 52, no 2, pp , Feb 2004 [16] D Yellin and E Weinstein, Criteria for multichannel signal separation, IEEE Trans Signal Process, vol 42, no 8, pp , Aug 1994 [17], Multichannel signal separation: methods and analysis, IEEE Trans Signal Process, vol 44, no 1, pp , Jan 1996 [18] M I Gürelli and C L Nikias, Blind identification and array processing applications of generalized higher-order statistics, in Proc Military Commun Conf, vol 45, 1996, pp [19] A Yeredor, Blind channel estimation using first and second derivatives of the characteristic function, IEEE Signal Process Lett, vol 9, no 3, pp , Mar 2002 [20] A Papoulis and S U Pillai, Probability, Random Variables and Stochastic Processes, Fourth ed Boston, MA: McGraw-Hill, 2002 [21] J-F Cardoso and A Souloumiac, Jacobi angles for simultaneous diagonalization, SIAM J Matrix Anal Appl, vol 17, pp , 1996 [22] A Yeredor, Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation, IEEE Trans Signal Process, vol 50, no 7, pp , Jul 2002 [23] D-T Pham, Joint approximate diagonalization of positive definite matrices, SIAM J Matrix Anal Appl, vol 22, no 4, pp , 2001 [24] A Ziehe, P Laskov, K R Muller, and G Nolte, A linear least-squares algorithm for joint diagonalization, in Fourth Int Symp Independent Component Anal Blind Signal Separation, Nara, Japan, 2003

13 EIDINGER AND YEREDOR: BLIND MIMO IDENTIFICATION USING THE SECOND CHARACTERISTIC FUNCTION 4079 [25] M Joho and H Mathis, Joint diagonalization of correlation matrices by using gradient methods with application to blind signal separation, in Proc IEEE Sensor Array Multichannel Signal Processing Workshop, 2002, pp [26] H Pozidis and A Petropulu, System reconstruction based on selected regions of discretized higher-order spectra, IEEE Trans Signal Process, vol 46, pp , Dec 1998 Eran Eidinger was born in Tel-Aviv, Israel, in 1979 He received the BSc degree in electrical engineering and computer science and the MSc degree in electrical engineering in 2001 and 2005, respectively (both cum laude) from Tel-Aviv University (TAU) His research interests include statistical signal processing and (convolutive) blind source separation Mr Eidinger received the Cellcom Communications Research Scholarship in 2002 and the Weinstein Prize, also in 2002, from the Yitzhak and Chaya Weinstein Institute for Research in Signal Processing Arie Yeredor (M 99 SM 02) was born in Haifa, Israel, in 1963 He received the BSc (summa cum laude) and PhD degrees in electrical engineering from Tel-Aviv University (TAU), Tel Aviv, Israel, in 1984 and 1997, respectively He is currently a Senior Lecturer with the Department of Electrical Engineering Systems, at TAU s School of Electrical Engineering, where he teaches courses in statistical and digital signal processing He also holds a consulting position with NICE Systems Inc, Ra anana, Israel, in the fields of speech and audio processing, video processing, and emitter location algorithms His research interests include estimation theory, statistical signal processing, and blind source separation Dr Yeredor is an Associate Editor for IEEE SIGNAL PROCESSING LETTERS and a member of the Signal Processing Society s Signal Processing Theory and Methods Technical Committee He has been awarded the Best Lecturer of the Faculty of Engineering Award for three consecutive years