Comparison of Fast ICA and Gradient Algorithms of Independent Component Analysis for Separation of Speech Signals

Transcription

1 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) Comparison of Fast ICA and Gradient Algorithms of Independent Component Analsis for Separation of Speech Signals K. Mohanaprasad #1, P. Arulmozhivarman # # School of Electronics Engineering, VI Universit Vellore, amil Nadu, India 1 kmohanaprasad@vit.ac.in parulmozhivarman@vit.ac.in Abstract Voice plas a vital role in distant communications like video conferencing, teleconferencing and hands free mobile conversion etc. Here, the qualit of speech is degraded b the Cocktail part problem. Cocktail part problem is described as combination of various sources of speech signal received b a microphone. Solution for the above problem can be obtained b using Independent component Analsis (ICA), which has the abilit to separate multiple speech signals into individual ones. his paper deals with application of principle of negentrop from maximization of non-gaussianit technique of ICA using Gradient and Fast ICA algorithm. he results in Matlab show that Fast ICA provides better execution time compared with gradient with minimum number of iteration. Keword - ICA, Negentrop, Fast ICA, Gradient, Maximization of non-gaussianit. I. INRODUCION Imagine that two people speaking simultaneousl are recorded using two microphones placed in different positions of the room. he microphones give two recorded signals x 1 (t) and x (t) with x 1 and x amplitudes, and t the time index. Each of the recorded signal is the linear combination of the two speech signals emitted b the speaker is denoted b s 1 (t) and s (t) [1]. So we could express this linear equation as x 1 (t) = a 11 s 1 +a 1 s (1) x (t) = a 1 s 1 +a s () Where a 11, a 1, a 1, a are the parameters that depend on the distances of the microphones from the speakers. Here the source signals s 1 and s is estimated from the mixed signals x 1 and x using Independent component analsis. his is known as blind source separation. In this process the mixed signals are obtained from statisticall independent and non-gaussian source signals. For simplicit we assume the unknown mixing matrix A, as the square matrix. he estimated source signals could be obtained up to their permutation, sign, and amplitude onl that is their order and variance cannot be obtained with independent component analsis. In recent ears, Researchers had proposed man criterions, Minimization of Mutual information have been used to estimate source signals using Independent component analsis. In those maximization of non- Guassianit gives the better performance. here are two techniques in maximizing non-guasianit, the are using kurtosis and negentrop. In which negentrop is more reliable as kurtosis is most sensitive to outliers and computationall robust process. In this paper, we estimated the source signals using Independent component analsis [] b maximizing negentrop. he maximization of negentrop can be done using two algorithms (Fast ICA and gradient). o estimate the source signals the demixing matrix is estimated. he fundamental restriction in ICA [3] is that the independent components are non-guassian in nature. o see wh gaussian variables make ICA impossible, assume that the signals are Gaussian and mixing matrix is orthogonal. hen x 1 and x are Gaussian, uncorrelated, and of unit variance. heir joint densit is given b 1 x (, ) exp( 1 + x px 1 x = π ) (3) his distribution is illustrated in Fig 1. he Figure 1 shows that the densit is completel smmetric. So, it does not contain an information on the directions of the mixing matrix. ISSN : Vol 5 No 4 Aug-Sep

2 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) Fig. 1. Multivariate distribution of two independent Gaussian variables o estimate one of the independent components, consider a linear combination of the x i let us denote this b w x wx t t t = = (4) where w is a vector to be determined. If w were one of the rows of the inverse of A, then the linear combination will equal one of the independent components. So determine such a w (i.e inverse of A) without knowledge of A matrix is not practical, but we can find an estimator that gives good approximation. o see how this leads to the basic principle of ICA estimation, let us make change of variables, defining z = A w (5) = was= zs is thus a linear combination of s i, with weights given b z i. Since a sum of even two independent random variables is more gaussian than the original variables, zs is more gaussian than an of the s i and becomes least gaussian when it in fact equals one of the s i. In this case, onl one of the elements z i of z is nonzero. herefore, we could take as w a vector that maximizes the non gaussianit of wx [4]. Such a vector would necessaril correspond to a z which has onl one nonzero component. his means that wx = zs equals one of the independent components. Maximizing the non gaussianit of wx thus gives us one of the independent components. o find several independent components, we need to find all the local maxima. Its not difficult, because different independent components are uncorrelated. his corresponds to orthogonalization in a suitabl transformed (i.e. whitened) space. II. EVALUAION OF INDEPENDEN COMPONENS BY MAXIMIZING A QUANIAIVE MEASURE OF NON- GAUSSIANIY wo quantitative measures of non-gaussianit are used in ICA estimation are kurtosis and negentrop. A. Negentrop Negentrop is based on the information-theoretic quantit [5] of differential entrop, which we here call simpl entrop. he more random, i.e., unpredictable and unstructured the variable is, the larger its entrop. he (differential) entrop H of a random vector with densit ( ) p η is defined as H ( ) = p ( η)log p ( η) dη (6) Gaussian variable has the largest entrop among all random variables. his means, entrop could be used as a measure of nongaussianit. Negentrop J is defined as follows J( ) = H( ) H( ) (7) gauss Where gauss is a Gaussian random vector of the same covariance matrix as. Negentrop, or negative normalized entrop, is alwas non-negative, and is zero if and onl if has a Gaussian distribution. Negentrop can be done using two algorithms as stated above. o make computation eas we center the data to make mean zero and then we go for whitening process to make uncorrelated data with variance one. he whitening process is done b eigen value decomposition method. ~ 1 x ED E x = (8) ISSN : Vol 5 No 4 Aug-Sep

3 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) he estimation of negentrop is difficult, as mentioned above, and therefore this contrast function remains mainl a theoretical one. he classical method of approximating negentrop is using higher-order moments, J( ) E{ } + kurt( ) (9) 1 48 he random variable is assumed to be of zero mean and unit variance. In particular, these approximations suffer from the no robustness encountered with kurtosis. o avoid the problems encountered with the preceding approximations, new approximations were developed. hese approximations were based on the maximumentrop principle. In general we obtain the following approximation p J( ) ki[ E{ Gi( )} E{ Gi( v)] (10) i= 1 Where ki are positive constants, andvis a gaussian variable of zero mean and unit variance B. Negentrop based fixed point algorithm A much faster method for maximizing negentrop [6] is done using fixed-point algorithm. he resulting FastICA algorithm finds a direction, i.e., a unit vector w, such that the projection w z maximizes nongaussianit. Non-gaussianit is here measured b the approximation of negentrop. FastICA is based on a fixed-point iteration for finding a maximum of the nongaussianit of w z. he FastICA algorithm using negentrop combines preferable statistical properties due to negentrop. he fixed point iteration can be approximated as follows: w E{ zg( w z)} (11) he above iteration does not have the good convergence properties of the FastICA using kurtosis, because the non polnomial moments do not have the same nice algebraic properties as real cumulants like kurtosis. So the modified iteration process can be as below, { ( w Ezgwz)} (1 + α) w= Ezgwz { ( )} + αw (1) Due to the subsequent normalization of w to unit norm, the latter equation gives a fixed-point iteration that has the same fixed points. So choice of α is more useful, it ma be possible to obtain an algorithm that converges as fast as the fixed-point algorithm using kurtosis. So the algorithm can be further simplified as Step wise procedure for Fast ICA Negentrop [7] 1. Center the data to make its mean zero.. Whiten the data to give z. 3. Choose an initial vector w of unit norm. { ( w Ezgwz)} Eg { '( wz)} w (13) 4. Let w Ezgwz { ( )} Eg { '( wz)} w, where g is defined as g1( ) = tanh( ) g( ) = exp( ) 5. Let w w/ w 6. If not converged, go back to step 4. C. Negentrop based gradient algorithm[8] A simple gradient algorithm can be derived as, aking the gradient of the approximation of negentrop with respect to w, and taking the normalization E{( w z) } = w = I into account, we can obtain the following algorithm, Δw γ E{ zg( w z)} (14) w w/ w (15) Where γ = [ EGwz { ( )} EGv { ( )}], v being an Gaussian random variable with zero mean and unit varience. he normalization is necessar to project w on the unit sphere to keep the variance of w z constant. ISSN : Vol 5 No 4 Aug-Sep

4 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) he parameter γ, which gives the algorithm a kind of self-adaptation qualit, can be easil estimated as follows Δγ [ EGwz { ( )} EGv { ( )}] γ (16) Step wise procedure for gradient negentrop: 1. Center the data to make its mean zero.. Whiten the data to give z. 3. Choose an initial vector w of unit norm, and an initial value for γ. 4. Update Δw γ zg( w z), where g is defined as in above algorithm. 5. Normalize w w/ w 6. If the sign of γ is not known a priori, update Δγ [ EGwz { ( )} EGv { ( )}] γ. If not converged, go back to step 4. D. Deflamationar orthogonalisation B the above process we will estimate onl one independent component [9] and to estimate all the components we have to run the process several times which is not reliable one so we use an algorithm known as deflamationar orthogonalisation which works on the propert of orthogonalisation. Orthogonalit is described as Non overlapping or uncorrelated. So, b this propert we will find out the orthogonal demixing matrices and with these matrices we will estimate the corresponding independent components. Deflamationar orthogonalisataion means finding w matrix which are orthogonal to each other. After estimating the w matrix using one unit algorithm for the first time, we have to run the whole one unit algorithm for estimating the other w matrix which is orthogonal to the first estimated w matrix. III. SIMULAION A. Results In this simulation two source signals Male and Female voices which are recorded from external sources are used. hen the signal S is produced b adding the two source signals. Now this signal is multiplied with random matrix to get a mixed signal X. he whitened signal is obtained when the mixed signal is done through the whitening process. he sample length of mixed signal X and estimated independent components are both of same order in the simulation. Fig. and Fig. 3 are the two source signals (male and female voices respectivel). Fig. 4 is the mixed signal X, Fig. 5 is the whitened signal. Now the demixing matrix is found b using an one of the one unit algorithm as explained above. After the completion of the one unit algorithm we get one of the source signal as separated signal. Fig.. S1-Male voice signal Fig. 3. S-Female voice signal ISSN : Vol 5 No 4 Aug-Sep

5 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) Fig. 4. S- Mixed voice signal Fig. 5. Y- Whitened voice signal Fig. 6. S Separated Female voice signal from Mixed voice signal B using deflamationar orthogonalisation S1 separated male voice signals are estimated after one unit algorithm Fig. 7. Separated signal male voice signal from mixed voice signal In the next simulation two standard signals are used as source signals, such as chirp and gong. hen the signal S is produced b adding the two source signals. Now this signal is multiplied with random matrix to get a mixed signal X. he whitened signal is obtained when the mixed signal is done through the whitening process. he sample length of mixed signal X and estimated independent components are both of same order in the simulation. ISSN : Vol 5 No 4 Aug-Sep

6 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) Fig.8 and Fig. 9, are the two source signals (chirp and gong signals respectivel). Fig.10 is the mixed signal X. Now the demixing matrix is found b using an one of the one unit algorithm as explained above. After the completion of the one unit algorithm we get one of the source signal as separated signal which as shown in Fig.11 and Fig.1. Fig.8 P1-Standard Chirp signal Fig. 9 P-Standard Gong signal Fig. 10. P- Mixed of Chirp and Gong signal Fig. 11. P Separated Gong signal from Mixed signal P B deflamationar orthogonalisation other signals are estimated after one unit algorithm. Fig. 1. P1 Separated Chirp signal from Mixed signal P ISSN : Vol 5 No 4 Aug-Sep

7 K. Mohanaprasad et.al / International Journal of Engineering and echnolog (IJE) B. Comparison between Fast ICA and Gradient Negentrop Here we have calculated the execution time and the amount of error signal in terms of correlation coefficient present in the separated signal (i.e., the amount of the other source signal present in the separated signal when one source signal is separated) for man number of observations and took an average of the observations and compared for Gradient and Fast ICA algorithms [10] coefficient can be calculated using the equation Corrcoef ( X, Y ) = Co variance( X, Y ) / Co variance( X, X ) Co variance( Y, Y ) (17) When the correlation coefficient reaches 1 then the two signals are highl correlated. When the value of correlation coefficient reaches nearl zero then there is no correlation between the two signals. Algorithm for negentrop Male voice (S1) separated from the mixture S between S1&S Female voice (S) separated from the mixture S between S1&S FAS ICA GRADIEN able I. Performance of Male and Female voice separation in Fast ICA and Gradient Algorithm Algorithm for negentrop Chirp signal (P1) separated from the mixture P between P1&P Gong signal (P) separated from the mixture P between P1&P FAS ICA GRADIEN able II. Performance of Chirp and Gong signal separation in Fast ICA and Gradient Algorithm From the above able I and able II we observed the Fast ICA provides better execution time compared to gradient with minimum no of iteration. Gradient ICA provides lesser values for correlation coefficients, which indicates that there is no correlation between separated signal with other signal IV. CONCLUSION his paper shows that Gradient based negentrop algorithm provides higher efficienc in separating speech signals compared with Fast ICA based negentrop algorithm. Fast ICA needs less execution time as compared to gradient based negentrop with minimum number of. REFERENCES [1] Hakin, Simon, and Zhe Chen. "he cocktail part problem." Neural computation 17.9 (005): [] Hvärinen, A., J. Karhunen and E. Oja, Independent Component Analsis. New York: Wile,(001) pp: [3] P. Comon, Independent Component Analsis-A new concept? Signal Processing, vol. 36, pp , [4] Hvärinen, Aapo, and Erkki Oja. "Independent component analsis: algorithms and applications." Neural networks 13.4 (000): [5] Hvarinen, Aapo. "Fast and robust fixed-point algorithms for independent component analsis." IEEE ransactions on Neural Networks 10.3 (1999): [6]. Chien and B.C. Chen, A new independent component analsis for speech recognition and separation. IEEE rans. Speech Audio Process (006): [7] K. Mohanaprasad and P. Arulmozhivarman. Comparison of Independent component analsis techniques for Acoustic Echo Cancellation during Double alk scenario. Australian Journal of Basic and Applied Sciences (013): [8] R. Ganesh, and K. Dinesh, An overview of independent component analsis and its application Informatica., 011: pp: [9] H.M.M. Joho and G. Moschtz, Combined blind / non blind source separation based on the natural gradient, IEEE signal process. Lett (001): [10] S. Miabe,. akatani, H. Saruwatari, K.Shikano, and Y. atekura. Barge-in and noise-free spoken dialogue interface based on sound field control and semi-blind source separation. In proc. Eur. Signal Process. Conf., 007:3-36. ISSN : Vol 5 No 4 Aug-Sep