Signal Processing. Data fusion over localized sensor networks for parallel waveform enhancement based on 3-D tensor representations

Transcription

1 Signal Processing 141 (2017) Contents lists available at ScienceDirect Signal Processing journal homepage: Data fusion over localized sensor networks for parallel waveform enhancement based on 3-D tensor representations Renjie Tong a, b, c, Zhongfu Ye a, b, c, a University of Science and Technology of China, Hefei, China b Department of Electronic Engineering and Information Science, Hefei, China c National Engineering Laboratory for Speech and Language Information Processing, China a r t i c l e i n f o a b s t r a c t Article history: Received 17 January 2017 Revised 16 May 2017 Accepted 13 June 2017 Available online 15 June 2017 Keywords: Parallel waveform enhancement Multi-sensor fusion technology 3-D tensor Transforming and filtering Multidimensional filtering Spatially white/colored noise In this paper the problem of parallel waveform enhancement via the multi-sensor fusion technology is carefully studied. Through representing the observed multiple noisy observations as a 3-D tensor, we propose two novel approaches in the time domain, i.e. the transforming and filtering (TAF) approach and the direct multidimensional filtering (DMF) approach, for parallel waveform recovery and interference suppression. The term parallel indicates the system can produce an estimate of the clean waveform in each sensor channel simultaneously. Specifically, the TAF approach transforms the observed tensor into a different domain where the noise can then be filtered by discarding the insignificant coefficients. The DMF approach directly reduces the noise level by applying multidimensional filtering on the observed tensor. Both DMF and TAF are blind because they do not require precise frequency responses between the desired source and distributed sensors. Simulations show that TAF is capable of yielding satisfactory performances for spatially white noise, while DMF can produce satisfactory results on spatially colored noise. Besides, both TAF and DMF can work well in complex real environments Elsevier B.V. All rights reserved. 1. Introduction Blind multi-sensor fusion aims to achieve parameter estimation or waveform reconstruction by combining multichannel recordings of a particular source obtained by distributed sensors with unknown array configurations. The resulting information generally has less uncertainty than would be possible when these sensors were used individually. This technology has found applications in many research areas such as image, radar, sonar, acoustic and seismic signal processing, which respectively use an array of cameras, antennas, hydrophones, microphones and seismographs to yield more accurate and more dependable results [1 3,25]. Consider the situation where these sensors are used individually for waveform estimation. Perhaps Wiener filtering (WF) is the most well-known method available, which can provide a minimum mean square error (MMSE) estimate of the clean waveform from its corrupted observations. Nevertheless, the flexibility of standard This work was supported by the National Natural Science Foundation of China under Grant Corresponding author at: Department of Electronic Engineering and Information Science Technology Building in West Campus, No. 443, Huangshan Road, Shushan Dis Hefei, Anhui China. addresses: tongren@mail.ustc.edu.cn (R. Tong), yezf@ustc.edu.cn (Z. Ye). WF is profoundly restricted because it does not distinguish between the noise residual and the signal distortion. The subspace approach (SA) was proposed and analysed in [4,5] to improve WF. In single-channel SA, a low-rank model for the clean signal and a white noise model for the interference are first assumed. Then the noisy data is transformed into the Karhunen-Loeve transform (KLT) domain which has uncorrelated coefficients. The enhanced signal is obtained from the inverse KLT of the altered coefficients. Many adapted algorithms were later proposed to deal with colored noise [6 9]. In [10,11], further studies on noise reduction in the KLT domain were presented. By manipulating eigenvalues of the signal and noise covariance matrix, SA is able to trade-off between the signal distortion and the noise residual. Another subspace filtering technique features the use of singular value decomposition (SVD), where a matrix is constructed from the noisy data in various ways. The SVD methodology was first proposed in [12] and then used for waveform estimation and data reconstruction in [13,14]. The clean signal matrix is estimated from the noisy data matrix by solving a low-rank approximation problem, whose solution can be directly given by the well-known Eckart-Young theorem [33]. Although SA and SVD respectively perform low-rank decomposition in the filter domain and the waveform domain, they actually both reveal the sparse and structured nature of most naturally occurring signals. The success of SVD / 2017 Elsevier B.V. All rights reserved.

2 250 R. Tong, Z. Ye / Signal Processing 141 (2017) mainly lies in three aspects: (1) information on the underlying data is mostly contained in a low-dimensional subspace that can be determined from the SVD of the noisy data matrix; (2) the complexity of this model can be estimated from dominant singular values; (3) SVD is expected to have a certain noise reduction effect by discarding insignificant singular values [15]. SVD may also bring serious signal distortion when the noise level is high. After all, information provided by an individual sensor is rather limited, which is a major cause of multi-sensor fusion technology (MFT) for signal and waveform enhancement. Recently, MFT-based waveform enhancement has become increasingly popular in many hardware systems. For example, today s smart phones often adopt multiple microphones to realize highquality audio acquisition. In a typical MFT system, it is generally assumed that each sensor node can receive not only its own measurement but also its neighboring sensors measurements according to the interconnection topology. Based on this assumption, a novel approach to filter design for TS fuzzy discrete-time systems with time-varying delay was proposed in [44]. The problem of distributed fuzzy filter design with time-varying delays and multiple probabilistic packet losses was considered in [45]. The design of the desired filter to guarantee an induced l 2 performance for the filtering error system was considered in [46]. The problem of reliable filter design with strict dissipativity was investigated in [47]. To simplify the problem, in the rest of this paper, we will only consider MFT systems whose sensor nodes can only receive their own measurements and afterwards all the measurements are sent to a fusion center to produce final enhanced waveforms. Also, the position of the target source is assumed to be fixed and thus the delay of samples remains time-invariant during the whole fusion process. Perhaps the most famous algorithms for MFT are beamforming techniques including the Frost beamformer [16] and generalized sidelobe canceler [17], whose basic idea is to steer a beam towards the target while suppressing interferences coming from other directions. Another fusion technique is the multichannel WF (MWF) which provides an MMSE estimate of the clean waveform by filtering a concatenated vector obtained by stacking different sensors frames successively [2]. It was demonstrated that MWF theoretically includes a linear beamforming process and a post-filtering process [43]. The extension from MWF to multichannel SA (MSA) [18 20] considers a fixed parameter to trade-off between the noise residual and the signal distortion. More details of MWF and MSA can be seen in the next section. A spatial-temporal prediction (STP) approach was proposed in [21] which assumes a linear relationship between different channel frames to account for their spatial dependencies. Its flexibility is a little limited because it mainly intends to minimize the distortion while the noise residual is not considered. A detailed analysis and comparison of MWF, MSA and STP, which feature the concatenated vector-based fusion technique, is presented in [22]. Such a technique is attractive mainly because it is blind, i.e., it relies on neither the geometry of sensor distributions nor the frequency response of sensors. This is an advantage over beamforming algorithms, which need an exact array manifold and will suffer from performance degradation when sensors location errors and amplitude-phase errors occur obviously [34,37,42]. Besides, the technique is parallel, i.e., it can produce an estimate for the clean waveform in each channel and thus can serve as a pre-filtering method for beamforming approaches. Nevertheless, in vector-based fusion, the concatenated vector generally has a high dimension and the curse of dimensionality may cause additional numerical difficulty such as increasing computational complexity and instability. Besides, although MWF and MSA theoretically include linear spatial filtering and single-channel temporal post-filtering, it remains unclear whether the information in ob- served data has been fully considered [43], because the observed data, as can be seen in the next section, can be naturally represented as a 3-D tensor rather than a 1-D extended vector. Actually, the authors already demonstrated that the vectorbased fusion strategy may not be the best choice for a blind and parallel MFT-based system [26]. Earlier than that, a parallel tensorbased fusion strategy, i.e. the multidimensional WF, was proposed for reduction of spatially white noise (SWN) in seismic signals [23]. Specifically, it represents the multichannel data as a 3-D tensor rather than a concatenated 1-D vector to preserve its original data structure. Through this, it can take full advantage of spatial and temporal prediction to improve the signal enhancement performance. Nevertheless, the flexibility of multidimensional WF is a little limited as it does not allow the trade-off between signal distortion and noise residual. Besides, it suffers from serious performance degradation on spatially colored noise (SCN). The main contributions of this paper are in two aspects. Firstly, we present an overview and comparison of two mainstream fusion strategies, i.e. the vector-based fusion and the tensor-based fusion. Secondly, we develop the tensor-based fusion strategy and propose two novel approaches for blind and parallel MFT-based waveform enhancement, i.e. the transforming and filtering (TAF) approach and the direct multidimensional filtering (DMF) approach. The motivation of this paper is to provide an experimental study of tensorbased fusion algorithms and show their advantages over vectorbased fusion algorithms. The remainder of this paper is organized as follows. In Section 2, prior art about two fusion strategies is introduced. In Section 3 and Section 4 we present the details of TAF and DMF, respectively. In Section 5 experiments are conducted to demonstrate performances of TAF and DMF. Conclusions are given in Section Evaluations and strategies In this section we will first define some useful measures to evaluate the performance of different algorithms. Then we will briefly introduce two popular fusion strategies, i.e. the vectorbased fusion strategy and the tensor-based fusion strategy, for parallel and blind MFT-based waveform enhancement Some evaluation standards Based on signal-to-noise ratio (SNR), we define some objective measures including the noise reduction factor (NRF), the signal distortion index (SDI), input SNR (ISNR) and output SNR (OSNR) to evaluate the performance of different fusion algorithms. Consider a linear or multi-linear fusion system f : R a b c R a b c that takes tensor as input and outputs f ( ). Note that may be vectors or matrices as well in this framework. Due to the linearity or multi-linearity of system f, we have f ( ) = f ( 1 ) + f ( 2 ) (1) where 1 and 2 respectively represent the clean signal and the noise. Note that some fusion systems may include nonlinear operations such as the thresholding process which can be linearized using a masking matrix or tensor. With above, the ISNR is defined as 10log (E[ 1 2 ]/E[ 2 2 ]) where 2 denotes the quadratic sum of elements in a tensor and E[ ] denotes the expectation operator. The primary issue is to determine how much noise is actually attenuated. Based on Eq. (1), errors in the fused result can be decomposed into two parts, i.e. the signal distortion 1 - f ( 1 ) and the noise residual f ( 2 ). The NRF is defined as ξ nr ( f ) = 10 log ( E[ 2 2 ] / E[ f ( 2 ) 2 ] ) which should be lower bounded by 0. The larger the value of ξ nr ( f ), the more is the noise reduction. Almost all fusion methods will distort the clean signal. In order to quantify this distortion, the definition of SDI is given as

3 R. Tong, Z. Ye / Signal Processing 141 (2017) ν sd ( f ) = 10 log ( E[ 1 f ( 1 ) 2 ] / E[ 1 2 ] ) which should be upper bounded by 0. The higher the value of ν sd ( f ), the more the underlying waveform is distorted. In order to know whether a fusion system f improves the SNR, we define OSNR as OSNR ( f ) = 10 log ( E[ f ( 1 ) 2 ] / E[ f ( 2 ) 2 ] ). (2) Sometimes OSNR is also defined as OSNR ( f ) = 10 log ( E[ 1 2 ] / E[ 1 f ( ) 2 ] ). (3) The purpose of MFT-based waveform enhancement is to find a fusion system f such that OSNR( f ) > ISNR. This is possible with a judicious choice of f, since the clean signal is generally more predictable than the irregular noise. Although SNR is a reliable objective measure, maximizing OSNR( f ) may not be the optimal thing to do because the distortion should be minimized as well The vector-based parallel fusion strategy The waveform reconstruction is generally carried out in a frame-by-frame way to decompose long-term non-stationary signals into short segments when their first-order and second-order statistics are approximately the same. The goal of conventional SA, which uses an individual sensor for waveform estimation, is to find a linear filter that can minimize a linear combination of the noise residual and the signal distortion [4]. Let 1-D vector y ( k ) be one of the observed frames with frame length L and frame index k = 1, 2,...., K (where K is the number of frames) in a certain stationary segment. Obviously y ( k ) is composed of a desired signal frame x ( k ) and an unwanted noise frame n ( k ). Specifically, the single-channel SA estimates filter F by: F = arg mine ( Fx ( k ) x (k ) 2 ) + η E( Fn ( k ) 2 ) (4) where terms on the right side respectively denote the signal distortion and the noise residual. The solution can be directly obtained as R x ( R x + ηr n ) 1 where R x = E(x (k ) x (k ) ) and R n = E(n (k ) n (k ) ) are respectively the signal and noise covariance matrix. Here ( ) denotes the vector/matrix transposition. Suppose that we have to fuse N observations obtained from N locally distributed sensors and the k -th observed frame of the n -th sensor/channel can be expressed as: y n (k ) = x n (k ) + n n (k ), n = 1, 2,..., N (5) where y n (k ) = [ y n ( f k ) y n ( f k + 1)... y n ( f k + L 1) ], f k equals 1 + S(k 1) with step length S, and vectors x n ( k ) and n n ( k ) are defined in a similar way to y n ( k ). In vector-based fusion, we estimate the clean waveform x n ( k ) by applying a linear transformation to the concatenation of N sensor frames. Specifically, the following fusion model n z = N (k) = F n y n (k) = Fy (k) = F [ x (k) + n (k)] (6) n =1 is adopted with F = [ F 1 F 2... F N ] y (k ) = [ y 1 (k ) y 2 (k )... y N (k ) ] x (k) = [ x 1 (k) x 2 (k)... x N (k)] n (k ) = [ n 1 (k ) n 2 (k )... n N (k ) ] and F n, n = 1, 2,..., N being filtering matrices of size L L. For example, MSA solves the following problem: F = arg mine ( F x ( k ) x n (k ) 2 ) + η E( F n ( k ) 2 ) (7) to estimate F and obtain x n (k ), n = 1, 2,, N. When η approaches 1, MWF can be derived from (7). The vector-based fusion model Fig. 1. Mode-1 fibers and frontal slices of a 3-D tensor. (6) can generally yield an acceptable performance for both directional and nondirectional noise. However, it is worth exploring whether (6) is the ideal model to estimate x n ( k ) since y ( k ) is essentially a stack of y n ( k ). Although MWF and MSA use spatial and temporal correlations to a certain degree, the information may have not been fully considered The tensor-based parallel fusion strategy Tensor-based fusion has been considered for blind and parallel signal enhancement in [26]. Specifically, a 3-D tensor Y = X + N R L K N is constructed from N observations to preserve the original data structure. Here tensors X and N represent the clean signal and the unwanted noise, respectively. For convenience, 1-D vectors Y (:, k, n ), Y ( l, :, n ) and Y ( l, k, :) will respectively be named as mode-1, mode-2 and mode-3 fibers of Y. 2-D matrices Y ( l, :, : ), Y (:, k, : ) and Y (:, :, n ) will respectively be named as horizontal, lateral and frontal slices of Y. Then the observed tensor can be uniquely determined by its lateral slices as Y (:, k, :) = [ y 1 (k ) y 2 (k )... y N (k ) ], k = 1, 2,..., K. More exactly, Y ( l, k, n ) denotes the l th sample of the k th frame in the n th channel. A sketch of mode-1 fibers and frontal slices of a 3-D tensor is shown in Fig. 1. The goal of tensor-based fusion is to recover the underlying signal tensor X, which contains x n (k ), n = 1, 2,, N in each channel. Traditionally, two approaches are considered for the tensorbased fusion strategy. One is based on the Tucker decomposition and the other is based on multidimensional filtering. The Tucker decomposition of Y has the following form: Y = 1 U 1 2 U 2 3 U 3 (8) where { U 1 R L L, U 2 R K K, U 3 R N N } are the adaptive bases and R L K N is the core tensor. Some additional constraints, such as the low-rank constraint, the diagonal constraint and the orthogonal constraint, can be enforced on Eq. (8) to recover X from Y. In this paper, we consider the well-known sparsity constraint and derive the TAF approach. On the other hand, multidimensional filtering is performed on Y according to the following multi-linear filtering model: Z = Y 1 H 1 2 H 2 3 H 3. (9) Above i, i = 1, 2, 3 denotes the mode- i tensor-matrix product defined in [27], H 1 R L L performs temporal intraframe fusion on mode-1 fibers, H 2 R K K performs temporal interframe fusion on

4 252 R. Tong, Z. Ye / Signal Processing 141 (2017) mode-2 fibers and H 3 R N N performs spatial fusion on mode- 3 fibers of Y. Some criterions like MMSE can be considered for optimization of H 1, H 2 and H 3. Previous work generally does not distinguish between the signal distortion and the noise residual in the fusion process [23,28]. In this paper, we propose the DMF approach which can minimize the waveform distortion by restricting the noise residual to an acceptable level. In the following, we will present the details of the proposed TAF approach and the DMF approach. 3. The TAF approach In the first part, we will reinvestigate the matrix SVD for singlesensor waveform enhancement and then introduce Tucker s decomposition model of higher-order tensors for parallel MFT-based enhancement. In the second part, we will give a detailed description of the TAF approach for MFT-based signal enhancement From matrix SVD to Tucker s decomposition model Suppose a matrix Y = X + N R L K whose columns represent observed noisy frames in the received single-sensor data. Note that Y may also denote a noisy batch in image denoising [24]. The single-channel SVD-based waveform recovery [40] attempts to estimate X representing clean signals by solving a low-rank optimization problem: X = arg min Y X 2 s. t. rank( X ) r 0 (10) where r 0 is the maximum rank known a priori. Suppose that the SVD of matrix Y can be written as Y = U y y V y where U y R L L and V y R K K are orthogonal and y R L K is diagonal. Similarly we define the SVD of X as U x x V x. The well-known Eckart-Young theorem [33] states that the optimal solution for Eq. (10) is X = U y V y (11) where is obtained from y by setting to zero all but its r 0 largest singular values. The traditional SVD-based waveform estimation mainly intends to seek for the best least squares approximation of lower rank r 0 to the noisy matrix. It has implicitly assumed that X and Y are generated by the same SVD bases { U y, V y }. Hence X can be obtained by merely manipulating the singular values of matrix Y, which is a natural consequence of the low-rank constraint. However, it is obvious that matrices X and Y generally have different SVD bases because { U y, V y } result from the joint effects of the SVD bases of X and N. In the area of image signal processing, authors of [24] have already demonstrated that the following solution X = U x V x (12) can yield a much better image denoising performance than solution (11). Above is not diagonal and can be obtained from U x Y V x by setting elements with amplitude below 2 σ 2 log ( LK ) to zero. Bases { U x, V x } are assumed to be known and are calculated from clean data. We take the estimation of a sine wave from white Gaussian noise (ISNR = 5 db) as an example and their results are presented in Figs. 2 and 3. We set L = 40 and K = 50; OSNR is calculated according to Eq. (3). One can see that solution (12) outperforms solution (11) by 4 db and perfectly recovers the underlying sine wave. The only difference is that solution (11) uses the noisy SVD bases and diagonal, while solution (12) uses clean SVD bases and nondiagonal. The advantage of solution (12) actually tells us that it deserves higher priority to design proper orthogonal bases than to design a diagonal coefficient matrix, and the SVD denoiser with Fig. 2. Solution (11) for sine waveform estimation. Fig. 3. Solution (12) for sine waveform estimation. properly designed orthogonal bases can work remarkably well for waveform estimation. Motivated by this, we may naturally wonder whether the SVD denoiser can be extended to the higher-order cases, where the bases are adaptively determined from multiple noisy observations, for parallel and blind MFT. The Tucker s decomposition model, i.e. Eq. (8), is often referred to as the extension of SVD to higher-order tensors. Thus it is interesting to make a connection with the matrix SVD. Two important features characterize the SVD of matrix Y : (1) bases { U y, V y } have orthogonal columns; (2) y is diagonal. However, this is not the case for decomposition of higher-order tensors. It is clear that if the number of free parameters in the right-hand side of (8) is smaller than the number of equations, there will be generally no solution. This happens to be the case if bases { U 1, U 2, U 3 } are orthogonal and core tensor is diagonal [35]. In the quest for existence, we have to choose: either is diagonal but we have to relax the orthogonal constraint on bases, or we keep the orthogonal constraint but allow to be non-diagonal. The former results

5 R. Tong, Z. Ye / Signal Processing 141 (2017) Table 1 Algorithm flow of HOSVD. Input : noisy tensor Y, L, K, N Output : core tensor, orthogonal bases { U 1, U 2, U 3 } Step 1: 1) Compute the SVD of Y (1) as U D 1 1 V 1 and design U as ( ) 1 U = U 1 1 :, 1 : L. 2) Compute the SVD of Y (2) as U D 2 2 V 2 and design U as ( ) 2 U = U 2 2 :, 1 : K. 3) Compute the SVD of Y (3) as U D 3 3 V 3 and design U as ( ) 3 U = U 3 3 :, 1 : N. Step 2: Calculate tensor as = Y 1 U 1 2 U 2 3 U 3. Table 2 Algorithm flow of the TAF approach. Input : noisy tensor Y, threshold (or regularization) parameter λ Output : core tensor, square and orthogonal bases { U, 1 U, 2 U } 3 Step 1: For r = 1, 2, 3, compute the SVD of Y ( r ) as U D r r V r and thus U r is adaptively learned. Step 2: For fixed bases { U, 1 U, 2 U }, calculate 3 by: (a) = arg min Y 1 U 1 2 U 2 3 U 3 2, s. t. p 0 or (b) = arg min Y 1 U 1 2 U 2 3 U 3 2, s. t. q. 1 in canonical polyadic decomposition (CPD) [27] and the latter results in higher-order SVD (HOSVD) [36] Details of the TAF approach As mentioned above, to use Tucker s decomposition for parallel MFT, we have to choose between the orthogonality of bases and the diagonality of the core tensor. For single-channel waveform estimation using matrix SVD, the orthogonal bases deserve higher priority than the diagonal coefficient matrix. Then we may naturally ask: what is the case for MFT-based signal enhancement using Tucker s decomposition? In the following, we will answer this question and then develop the TAF approach. In the presence of noise, Eq. (8) does not strictly hold, which is different from the matrix SVD. Thus the underlying signal tensor X is often estimated by imposing various kinds of regularization terms or additional constraints on (8). For instance, CPD uses a diagonal constraint and learns nonorthogonal bases by solving the following problem: (, U 1, U 2, U 3 ) = arg min Y 1 U 1 2 U 2 3 U 3 2, s. t. L = K = N = R, is diagonal (13) where R represents a possible rank of the underlying tensor X. Eq. (13) can be efficiently tackled by the alternating least square algorithm [27]. On the contrary, HOSVD uses an orthogonal constraint and learns adaptive bases by approximately solving (, U 1, U 2, U 3 ) = arg min Y 1 U 1 2 U 2 3 U 3 2, s. t. U 1 U 1 = I L, U 2 U 2 = I K, U 3 U 3 = I N (14) with L L, K K, N N. More details about HOSVD can be found in Table 1. At step 1, SVD is performed respectively on Y ( r ) for r = 1, 2, 3 and the singular values are in descending order. Here Y ( r ) is the r -mode unfolding matrix of Y and can be obtained by properly relocating the r -mode fibers [27]. Then U r is adaptively designed by reserving only the leading singular vectors of matrix Y ( r ). The core tensor is calculated at step 2. Tensor X is finally estimated as 1 U 1 2 U 2 3 U 3. A more exact solution of (14) can be given by the higher order orthogonal iteration (HOOI) algorithm [39], which often adopts HOSVD to provide an initial value. We have omitted the details of CPD and HOOI since our contribution, i.e. the TAF approach, is mainly based on HOSVD. Intuitively, CPD aims at finding a lowrank approximation to the observed tensor because clean signals generally have a certain structure and their information is mainly contained in a low-rank tensor. HOSVD and HOOI aim to compress the noisy data into the core tensor of a smaller size as clean signals are considered to be structured and thus more compressible than the irregular noise. One such proof is that none of CPD, HOSVD and HOOI can produce satisfactory results for highly structured noise. Besides, CPD and HOOI are generally much more time-consuming than HOSVD because they are iterative algorithms in essence. Based on HOSVD, the proposed TAF approach introduces an additional sparsity regularization term and approximately solves: (, U 1, U 2, ) U 3 = arg min Y 1 U 1 2 U 2 3 U 3 2, s. t. 0 p, U 1 U 1 = I L, U 2 U 2 = I K, U 3 U 3 = I N. (15) The TAF approach differs with HOSVD in three aspects, although they both use adaptively learned orthogonal bases. Firstly, bases { U 1, U 2, U 3 } are constrained to square in TAF. Secondly, an exact core tensor can be found for each group of orthogonal bases. Thirdly, the core tensor is constrained to having no more than p nonzero entries. The l 0 -norm, which is a pseudo norm counting nonzero elements in a tensor, is introduced as a sparsity regularization term to avoid over-fitting. Without this term, all the information about Y, including the noise information, will likely be contained in. Considering that l 0 -norm can be approximated by l 1 -norm [29], we recast (15) as: (, U 1, U 2, ) U 3 = arg min Y 1 U 1 2 U 2 3 U 3 2, s. t. 1 q, U 1 U 1 = I L, U 2 U 2 = I K, U 3 U 3 = I N. Above problem can be further converted to the following: (16) (, U 1, U 2, U 3 ) = arg min 1 2 Y 1 U 1 2 U 2 3 U λ 1, s. t. U 1 U 1 = I L, U 2 U 2 = I K, U 3 U 3 = I N (17) where parameter λ controls the trade-off between the decomposition error and the sparsity regularization term. Eqs. (15) (17), at first glance, seem intractable because can interact with the bases. In order to obtain an exact solution, we have to iterate between the optimization of and { U 1, U 2, U 3 }. However, as can be seen later, HOOI bases do not produce better results than HOSVD bases, although they provide a more exact solution to Eq. (14). Therefore, for Eqs. (15) (17), we can roughly use HOSVD bases to find an approximate solution. Details of the TAF approach can be found in Table 2. At step 1, we obtain desired HOSVD bases as in Table 1. At step 2, we obtain the core tensor according to either ( a ) or ( b ), whose solutions can be respectively written as l 0 TAF : and l 1 TAF : = HARD λ( Y 1 U 1 2 U 2 3 U ) 3 (18) = SOF T λ( Y 1 U 1 2 U 2 3 U ) 3 (19) where the hard shrinkage operator HARD λ and the soft shrinkage operator SOFT λ are respectively defined as HARD λ(x ) = and { x, x λ 0, x < λ (20) { x λ sgn (x), x λ SOF T λ(x ) =. (21) 0, x < λ

6 254 R. Tong, Z. Ye / Signal Processing 141 (2017) Fig. 4. Spectrum of a chirp signal. Note that both HARDλ and SOFTλ have been extended to tensors by applying them to each element. The clean signal tensor X is finally estimated as 1 U1 2 U2 3 U3. The clean signal waveforms can be estimated from X (:, :, n), n {1, 2,, N} using the overlap-add method. From expression (19) one can clearly see that the sparsity regularization parameter λ appears as a threshold parameter in the final solution. The TAF mainly intends to transform the noisy tensor into the HOSVD domain that preserves all the energy and information about the observed data. In the HOSVD domain, insignificant entries of, whose absolute values are below λ, are considered to be noise components and penalized to zero. Intuitively, a larger λ will result in more signal distortion and a smaller λ will result in more noise residual. Similar to solution (12), the threshold is adaptively computed as g 2σ 2 log (NLK ) with g 1 which is considered to be statistically optimal for orthogonal bases and white Gaussian noise with deviation σ [38]. We conducted a simulation based on a chirp signal (Fig. 4) and Gaussian SWN to compare different algorithms. We try to blindly fuse N = 8 time-aligned noisy observations of the chirp signal, which are independently corrupted by white Gaussian noise with ISNR = 5 db. The performance is evaluated using OSNR calculated by Eq. (3). We set R = 10 for CPD, L = 10, K = 10 and N = 2 for HOSVD and HOOI, and g = 1 for TAF to give nearly the best results. The spectra of the noisy and fused signals are presented in Figs Results show that despite of their different time complexity, HOSVD can yield slightly better performances than CPD and HOOI. Besides, both l0 -TAF and l1 -TAF can apparently outperform CPD by about 3 db, which shows the advantage of the orthogonal and sparse constraint over the diagonal constraint. This is consistent with the conclusion of matrix SVD at part A. 4. The DMF approach The TAF approach proposed at above section works well when the distributed sensors are independently corrupted by Gaussian SWN. That is, the noise field is spatially uncorrelated among the sensors. In this section we will propose the DMF approach, which can work well for SCN as well Integration of the spatial and temporal fusion Following definitions of Eq. (9) and utilizing the MMSE criterion, we can calculate the average signal distortion as Fig. 5. The noisy version (ISNR = 5 db). Fig. 6. CPD (OSNR = 17.8 db). Jasd = E( X X 1 H1 2 H2 3 H3 ) and the average noise residual 2 as Janr = E( N 1 H1 2 H2 3 H3 ). Similar to the single-sensor SA, three filters are estimated as 2 (H1, H2, H3 ) = arg min Jasd, s.t. Janr α. (22) That is, the distortion is minimized while the noise residual is restricted to an acceptable level. As there is no direct solution for above equation, a block-wise coordinate descent approach is adopted to jointly update these filters. For example, at the ith iteration, filter H3 is updated as 2 Hi3 = arg min E( X X 1 Hi1 2 Hi2 3 H3 ), s.t. 2 E( N 1 Hi1 2 Hi2 3 H3 ) α (23) and similarly we can update H1 and H2. Above process is iterated among the three filters until convergence.

7 R. Tong, Z. Ye / Signal Processing 141 (2017) Fig. 7. HOSVD (OSNR = 18.1 db). Fig. 9. l1 -TAF (OSNR = 20.6 db). Fig. 8. HOOI (OSNR = 18.0 db). Fig. 10. l0 -TAF (OSNR = 22.4 db). In order to solve (23), we define two auxiliary matrices S = Hi2 Hi1, T = S S (24) where denotes the Kronecker product. Utilizing the property X Hi Hi H = H3 X ( 3 ) S 1 1i 2 2i 3 3 (3) N 1 H 1 2 H 2 3 H 3 (3 ) = H3 N ( 3 ) S (25) we can expand problem (23) as min E( X(3) H3 X(3) S 2 ), s.t.e( H3 N(3) S 2 ) α. According to the KKT conditions, the optimal H3 should satisfy the following constraints: 2 primal feasibility : E( H3 N(3) S ) α dual feasibility : ξ 0 2 complementary slackness : E(ξ H 3 N2(3) S ) = ξ α 2 stationarity : H3 E( X(3) H3 X(3) S + ξ H3 N(3) S ) = 0 (27) (26) The optimal solution of (26) must satisfy the Karush Kuhn Tucker (KKT) conditions. The KKT point is also the optimal solution because problem (26) is strictly convex. where ξ is the Lagrangian multiplier. It is obvious that we cannot minimize the distortion and noise residual simultaneously. Therefore it makes sense to assume a positive ξ, which indicates that the inequality constraint of (26) is always tight.

8 256 R. Tong, Z. Ye / Signal Processing 141 (2017) For simplicity we define R SX = E( X (3) S X (3)) as the S-weighted covariance matrix of X (3). Similarly we define ( ) ( ) ( ) R TX = E X (3) T X (3), R TN = E N (3) T N (3), R SN = E N (3) S N (3), ( ) ( ) R SY = E Y (3) S Y R SX + R SN, R TY = E Y (3) T Y R TX + R TN (3) (3) (28) where we used the fact that desired signal waveforms are generally uncorrelated with noise waveforms. By the stationarity condition of (27), we have the following H i 3 = R SX ( R TX + ξ R TN ) 1. (29) By ξ > 0 and the complementary slackness, we have {( ) α = tr R TX R 1 TN R TX + 2 ξ R TX + ξ 2 1 } R TN R SX R SX (30) where tr{ } is the trace operator. Let C be the unitary eigenvector matrix of E( X (3) X (3) ). Then X (3) can be expressed as X (3) = CO where E( OO ) is the diagonal eigenvalue matrix of E( X (3) X (3) ). Similarly S = E( OSO ) and T = E( OTO ) are respectively the diagonal eigenvalue matrix of R SX and R TX, because elements of O are in general mutually uncorrelated [23,28]. Combining the above, we rewrite equation (29) as H i 3 = C S ( T + ξ C R TN C ) 1 C (31) which shares the same form with the single-sensor SA Details of the DMF approach Above S and T in solution (31) can be calculated respectively from eigenvalue matrices of R SY and R TY obtained by local averaging in practice. For example, let P R L I N ( I is much larger than K ) be the overall noisy tensor and Y t = P (:, t C : t + C, :) with C = (K 1) / 2 be the tensor block processed at time index t. Then we approximately have R S Y t j= t+ D j= t D Y j (3) S Y j (3) / (2 D + 1) and R T Y t j= t+ D j= t D Y j (3) T Y j (3) / (2 D + 1). Integers C and D are restricted by stationary time of signals. Let Ɣ S and Ɣ T be the diagonal eigenvalue matrix of R SY and R TY, respectively. In practice R SY is only approximately symmetric because samples for calculation of the mathematical expectation are finite in number. Thus its symmetric component R sym SY = ( R SY + R SY ) / 2 (32) is adopted to ensure that Ɣ S and S are real and diagonal. When the SWN condition is met, entries of N are mutually uncorrelated. Thus R SN and R TN are respectively equal to tr (S ) σ 2 I N and tr (T ) σ 2 I N, where σ is the noise deviation and I N is the N N identity matrix. Utilizing the following rules: R SY R SX + R SN, R TY R TX + R TN, (33) matrices S and T can be respectively estimated as: S = g(ɣ S tr (S ) σ 2 I N ), T = g(ɣ T tr (T ) σ 2 I N ) (34) where function g ( ) maps negative entries to zero and preserves nonnegative entries. With above, solution (31) can be written as DMF _ 1 : H i 3 = C S ( T + σ 2 ξ tr (T ) I N ) 1 C (35) When the SWN assumption is not met, however, DMF _ 1 will suffer from performance degradation. This is because R SN and R TN are not diagonal anymore for SCN. Fortunately, in our experiments, we find the off-diagonal entries of C R TN C in (31) generally have negligible absolute values. To see this, we define the diagonality measure (DM) of a square matrix A R N N as: D (n ) = i = n i =1 A (i, i ), n = 1,..., N. (36) i =1 A (i, j) j= n i = n j=1 Table 3 Algorithm flow of the DMF approach. Input : noisy tensor Y, maximum iteration number I max, threshold value ε min, weighting parameter η Initialization : identity matrices H 0 n, n = 1, 2, 3, i = 0, E 0 = inf, X 0 = inf Block-wise coordinate descent for DMF Repeat 1-4 until i = I max or E i 2 < ε min 1. i i Solve problem (22) and update H i 1, H i 2 and H i 3. Solutions (35) and (37) are respectively for SWN and SCN 3. Estimate signal tensor X i by (9). 4. Update the relative improvement as E i = X i X i 1. Output: X = Y 1 H i 1 2 H i 2 3 H i 3 Table 4 DM values of diagonal matrices and C R TN C. D ( n ) D (2) D (3) D (4) D (5) D (6) Diagonal Matrices C R TN C The larger DM is, the more A looks like a diagonal matrix. DM values of diagonal matrices and C R TN C are presented in Table 4. A 6- element uniform linear array is used for simulation and the testing data contains a wideband speech signal from the TIMIT database [30] and directional babble noise from the noisex-92 database [31]. From Table 4 one can see C R TN C is close to a diagonal matrix, whose DM is always equal to one. This inspires us to set its offdiagonal elements to zero and convert (31) into: DMF _ 2 : H i 3 = C S ( T + ξ diag( C R TN C ) ) 1 C (37) where diag ( ) clears the off-diagonal elements of a matrix. Similarly we can update H i 1 and H i. In experiments this simple numer- 2 ical trick can greatly improve performances on SCN. The rational between this trick is that an orthogonal matrix C applied on the weighted noise covariance matrix R TN makes C R TN C close to a diagonal matrix, which is a known fact when R TN is a Toeplitz matrix [48]. Actually, the best choice for C is a matrix that jointly transforms R SX, R TX, R TN in solution (29) to a diagonal matrix. However, to the best of our knowledge, such a matrix has not been found and currently we could only propose the suboptimal solution (37). As a result, for Gaussian SWN, DMF _ 2 produces similar results with DMF _ 1 ; for SCN, DMF _ 2 can always yield better performances than DMF _ 1. Details of DMF _ 1 and DMF _ 2 can be found in Table 3. As Eq. (22) is non-convex, after a few iterations, a locally optimal solution may be found for Eq. (22). Fortunately this local optima can in general yield satisfactory performances as well. By substituting this local optima into Eq. (9), we can estimate the signal tensor X t from Y t. Then the overall signal tensor Q R L I N is synthesized from X t using the overlap-add method, which is used similarly in the TAF approach. 5. Experiment In this section we try to fulfill the task of high-quality audio acquisition using an array of microphones. As a typical representative of localized sensor networks, a uniform or nonuniform linear array (LA) with a synchronized clock is adopted. We adopt some wideband audio data to conduct simulations and real experiments to compare different fusion algorithms. All the testing data is sampled to Hz. The beginning speech-absent segments are adopted to estimate noise statistics. For fair comparison, we implement MSA, STP, the time-domain multichannel Wiener filter (TMW) and the frequency-domain multichannel wiener (FMW) filter. Their parameters are all carefully

9 R. Tong, Z. Ye / Signal Processing 141 (2017) Fig. 11. SDI with changing ISNR. Fig. 12. NRF with changing ISNR. tuned to produce nearly optimal results. The frame length L is set to 25 for TMW, STP and MSA and 64 for FMW. We set L = 40 and K = 50 for TAF. For DMF, we set L = 40, C = D = 2, I max = 10 and ε min = 0.1. The forgetting factor is set to for TMW, MSA, STP and FMW. The frame overlapping ratio is 0% for DMF, 80% for TMW, MSA and STP, and 75% for FMW. As both TAF and DMF are parallel, the final output is generated from Q (:, :, 1) using the overlap-add method. The OSNR is calculated according to Eq. (2) Performances of different algorithms on Gaussian SWN In this part, the simulated data is generated by a uniform LA. The source signals (5 male and 5 female TIMIT sentences) are positioned at the LA s perpendicular direction and thus added to each sensor without relative time delays. The SWN is generated by adding white Gaussian noise to each sensor independently. The results are averaged over 10 speech sentences. The performance curves of different algorithms with 8 microphones and changing ISNR are plotted in Figs As shown in Fig. 11, η of MSA, ξ of DMF and g of TAF are carefully tuned so that they produce similar SDI with TMW, FMW and STP. From Figs. 12 and 13, one can see that the TAF approach produces the highest NRF and OSNR of all, followed by the DMF approach. As mentioned, DMF _ 1 and DMF _ 2 can yield the same SDI, NRF and OSNR for Gaussian SWN. STP is able to yield the least signal distortion. Nevertheless, it also produces lower NRF and OSNR. In general, tensor-based fusion algorithms have obviously outperformed vector-based fusion algorithms. The performance curves of DMF _ 1 with varying number of microphones and changing ISNR are plotted in Figs As expected, higher OSNR ( Fig. 14 ), lower SDI Fig. 15 ) and higher NRI ( Fig. 16 ) can be obtained by adopting more microphones in the array Performances of different algorithms on directional SCN In this part, the simulated data is generated by an 8-element uniform or non-uniform LA. The source signals, which contain 5 male and 5 female TIMIT sentences, are positioned at the LA s perpendicular direction. The directional SCN, which contains babble, destroyer, f16, pink and volvo from the noisex-92 database, is positioned at the LA s endfire (or axis) direction. The ISNR is set to 5 Fig. 13. OSNR with changing ISNR. db. The results are averaged over 10 speech sentences. The speed of sound is set to 340 meters per second. We tune all the algorithms so that they can produce nearly the best results. The singlechannel SA for colored noise [9] is also included to demonstrate the effectiveness of using multiple microphones. In this paragraph, we adopt an 8-element uniform LA with inter-element spacing 5 cm for simulation. At Table 5 we present simulation results of different algorithms on pure directional SCN. One can see that DMF _ 2 has an apparent advantage over the other algorithms (including DMF _ 1 ) in terms of SDI, NRF and OSNR. That is, the proposed DMF _ 2 has the potential to yield the highest noise reduction while causing the least signal distortion. Besides, DMF _ 2 has obviously outperformed DMF _ 1, which means the diagonalization operation in Eq. (31) can significantly improve the performance. In this paragraph, we adopt an 8-element non-uniform LA with inter-element spacing (5, 10, 5, 10, 5, 10, 5) centimeters. At Table 6 we present simulation results of different algorithms on pure directional SCN. One can see that DMF _ 2 still outperforms the other

10 258 R. Tong, Z. Ye / Signal Processing 141 (2017) Table 5 Comparison of different fusion algorithms on a uniform linear array. The ISNR is set to 5 db. Noise Type babble destroyer f16 pink volvo Metrics SDI NRF OSNR SDI NRF OSNR SDI NRF OSNR SDI NRF OSNR SDI NRF OSNR SA FMW STP MSA DMF _ DMF _ l 0 -TAF Table 6 Comparison of different fusion algorithms on a non-uniform array. The ISNR is set to 5 db. Noise Type babble destroyer f16 pink volvo Metrics SDI NRF OSNR SDI NRF OSNR SDI NRF OSNR SDI NRF OSNR SDI NRF OSNR SA FMW STP MSA DMF _ DMF _ l 0 -TAF Fig. 14. OSNR with changing ISNR. Fig. 15. SDI with changing ISNR. algorithms (including DMF _ 1 ) in terms of SDI, NRF and OSNR. This indicates the proposed algorithms do not rely on the exact array manifold. Actually, during the whole fusion process, the information of sensor positions are unknown to the system. The average time costs of different algorithms (on 1-s audio data) are listed below. The adopted i5-5200u CPU has 2 cores and 4 logical processors. Among all the algorithms, DMF _ 2 is the most time-consuming one (11.67 s), followed by DMF _ 1 (4.56 s), MSA (3.67 s), STP (3.45 s), SA (0.88 s), FMW (0.77 s) and l 0 -TAF (0.44 s) Performances of different algorithms in the real environment Pure directional noise does not exist in the real world, because there always exists sensor noise and channel fluctuations, which can be roughly modelled as Gaussian SWN. In the following, we will show that the proposed algorithms can work in the real environment as well. The measured data is generated by an 8-element uniform LA with inter-element spacing 5 cm in a real recording studio. The signal source is obtained by splicing above mentioned wideband audio data (10 TIMIT sentences). The signal source and the noise source are played through two mobile phones linked to two loudspeakers respectively positioned at the 8-element LA s perpendicular direction and axis direction. The reference signal is obtained by playing the signal source only. The recording studio has five walls covered by sound-absorbing materials and the floor is left unprocessed. Both the signal and noise source are placed in the far field and a segment is cut out for evaluation. The results are evaluated using the improved segmental SNR (SSNR) and improved perceptual evaluation of speech quality (PESQ) (ITU-T P.862) [32]. From Table 7, one can see that the proposed tensor-based fusion algorithms indeed have an advantage over conventional vector-based fusion algorithms in the real environment. The results also indicate l 1 -TAF has better robustness

11 R. Tong, Z. Ye / Signal Processing 141 (2017) Table 7 Comparison of different algorithms for multichannel fusion in a semi-anechoic chamber. Noise Type babble destroyer pink f16 volvo Metrics PESQ SSNR PESQ SSNR PESQ SSNR PESQ SSNR PESQ SSNR FMW STP MSA DMF _ l 0 -TAF l 1 -TAF form LA. From Fig. 17, one can see that DMF generally converges in two or three iterations. Also, we notice that a larger ξ converges slightly faster than a smaller ξ. 6. Conclusions In this paper we have proposed two fusion algorithms, i.e. TAF and DMF, for parallel tensor-based MFT. Simulation results show that the TAF approach is capable of yielding satisfactory performances for SWN, while the DMF approach can produce satisfactory results on SCN. Both simulations and real experiments imply that the proposed algorithms can work well for MFT-based systems. In the future, we will try to explore tensor-based data fusion over more complicated networks. Acknowledgment Fig. 16. NRF with changing ISNR. The authors would like to thank Dr. Sorber, Prof. Barel and Prof. Lathauwer for sharing software Tensorlab [41] which has brought much efficiency to our research. The authors would also like to thank the editor and reviewers for their valuable suggestions to improve the work. References Fig. 17. Energy curves of relative improvement as a function of iteration numbers. than l 0 -TAF and yields better performances in the real environment Discussion on the convergence property of DMF In Fig. 17, we describe the energy value of relative improvement ( E i = X i X i 1 ) as a function of iteration numbers i. In the simulation, we set ISNR = 5 db for Gaussian SWN and N = 8 for the uni- [1] B.G. Ferguson, K.W. Lo, Passive and active sonar signal processing methods for port infrastructure protection and harbor security, J. Acoust. Soc. Am. 140 (4) (2016) [2] J. Benesty, J. Chen, Y. Huang, Microphone Array Signal Processing, Springer, Berlin, Germany, [3] J.E. Palmer, H.A. Harms, S.J. Searle, et al., DVB-t passive radar signal processing, IEEE T. Signal Process. 61 (8) (2013) [4] Y. Ephraim, H.L.V. Trees, A signal subspace approach for speech enhancement, IEEE T. Speech Audio Process. 3 (1995) [5] H. Kris, P. Wambacq, A review of signal subspace speech enhancement and its application to noise robust speech recognition, EURASIP J. Appl. Signal Process (20 07) [6] A. Rezayee, S. Gazor, An adaptive KLT approach for speech enhancement, IEEE T. Speech Audio Process. 9 (2001) [7] U. Mittal, N. Phamdo, Signal/noise KLT based approach for enhancing speech degraded by colored noise, IEEE T. Speech Audio Process. 8 (20 0 0) [8] H. Lev-Ari, Y. Ephraim, Extension of the signal subspace speech enhancement approach to colored noise, IEEE Signal Process. Lett. 10 (4) (2003) [9] Y. Hu, P.C. Loizou, A subspace approach for enhancing speech corrupted by colored noise, IEEE Signal Process. Lett. 9 (6) (2002) [10] J. Benesty, J. Chen, Y.A. Huang, On noise reduction in the Karhunen-Love expansion domain, in: ICASSP, Taipei, China, 2009, pp [11] J. Chen, J. Benesty, Y.A. Huang, Study of the noise-reduction problem in the Karhunen-Love expansion domain, IEEE T. Audio Speech Lang. Process. 17 (4) (2009) [12] K. Iizuka, P. Kmtky, Data adaptive signal estimation by singular value decomposition of data matrix, Proc. IEEE 70 (6) (1982). [13] S. Bakamidis, M. Dendrinos, G. Carayannis, SVD analysis by synthesis of harmonic signals, IEEE T. Signal Process. 39 (2) (1991) [14] D.W. Tufts, A. Shah, Estimation of a signal waveform from noisy data using low-rank approximation to a data matrix, IEEE T. Signal Process. 41 (4) (1993) [15] B. De Moor, The singular value decomposition and long and short spaces of noisy matrices, IEEE T. Signal Process. 41 (9) (1993) [16] O.L. Frost III, An algorithm for linearly constrained adaptive array processing, Proc. IEEE 60 (8) (1972) [17] L.J. Griffiths, C.W. Jim, An alternative approach to linearly constrained adaptive beamforming, IEEE T. Antenn. Propag. 30 (1) (1982)