DIMENSIONALITY REDUCTION METHODS IN INDEPENDENT SUBSPACE ANALYSIS FOR SIGNAL DETECTION. Mijail Guillemard, Armin Iske, Sara Krause-Solberg

Transcription

1 DIMENSIONALIY EDUCION MEHODS IN INDEPENDEN SUBSPACE ANALYSIS FO SIGNAL DEECION Mijail Guillemard, Armin Iske, Sara Krause-Solberg Department of Mathematics, University of Hamburg, {guillemard, iske, ABSAC In the last few years an important family of methods for singlechannel signal separation has been developed using tools from time-frequency analysis. Given a mixture of signals s = i s i, the task is to estimate the components s i with some assumptions on their time-frequency or statistical characteristics. A well known strategy, sometimes denominated independent subspace analysis (ISA), is to reduce the embedding dimension of the time-frequency representation of s, prior to the application of independent component analysis (ICA). In these methods, a standard strategy for dimensionality reduction is principal component analysis (PCA), but other nonlinear methods have also been proposed over the last few years. In this paper, we compare different dimensionality reduction methods for single channel signal separation in the context of ISA. Our focus is on signals with transitory components, and the objective is to detect the time behavior of each individual signal s i. Keywords Dimensionality eduction, Wavelet, Gabor transforms, SF, Signal Separation, Independent Subspace Analysis. 1. INODUCION Signal separation is a central topic in many engineering fields and its modern development depends on new experimental and empirical insights together with modern mathematical tools In the last decade, different approaches have been proposed for dealing with the blind source separation of single channel signals. A strategy that has benn proposed in [2, 3] integrates the well known method of independent component analysis (ICA) with time-frequency transforms. A crucial step in this framework is to reduce the dimensionality of the data prior to the application of ICA. In recent developments of data analysis new strategies for dimensionality reduction have been inspired by geometrical and topological concepts [4]. New algorithms based on differential geometry are Whitney embedding based methods, Isomap, LSA, Laplacian eigenmaps, iemannian normal coordinates, to mention but a few. In parallel developments, probabilistic conditions and numerical algorithms (e.g. persistent homology) have provided new tools for reconstructing the homology of a manifold M n from a finite dataset X = {x i } m i=1 hese techniques have also delived new strategies for cluster analysis of point cloud data. he objective of this paper is to use these new developments in the framework of time-frequency signal separation based on independent subspace analysis. Our main contributions are the evaluation of different dimensionality reduction techniques, together with cluster techniques for improving the quality of the signal separation. he outline of this paper is as follows. In Section 4, we describe the basic elements of time-frequency independent subspace analysis. In Section 2, we describe the basic elements of our modulation map framework based on dimensionality reduction. An important component of our work are efficient interpolation techniques based on radial basis functions that we discuss in Section??. 2. DIMENSIONALIY EDUCION AND SIGNAL POCESSING Due to the high dimensionality of the time-frequency data, it is of interest to work with analysis techniques that combine signal processing transforms with dimensionality reduction methods. In this case, the basic objects are the manifold M, the data samples X = {x i } m i=1 taken from M, and a diffeomorphism A : Ω M, where Ω is the low-dimensional copy of M to be reconstructed via dimensionality reduction. Here, the only algorithmic input is the dataset X, but with the assumption that we can reconstruct topological information of M with X (see for instance [5]). he other basic object in our scheme is a signal processing map : M M, which may be based on Fourier analysis, wavelet transforms, or convolution filters, together with the resulting set M := { (p), p M} of transformed data. he following diagram shows the basic situation. Ω d Ω d A X M n (X) M n he main objective is to find an approximation of Ω, denoted Ω = (M ), by using a suitable dimensionality reduction map. Some properties of Ω and Ω may differ depending on the dimensionality reduction technique, but the target is to

2 construct Ω, so that geometrical and topological properties of Ω are recovered. In Section??, we use a particular modulation map A and we study the geometrical effects being incurred by several dimensionality reduction maps : M Ω. Dim ed Since real life data is multifaceted and complex a given data set remains even after discretization (on a set of certain observations) very high dimensional. Analysing and interpreting this kind of datasets pose some mathematical and computational challenges and might cause the failure of traditional statistical methods. We observe that in many cases not all information contained in the data points are relevant for understanding the underlying characteristics or properties of the data. Also low dimensional datasets are much easier to operate with in case of classification, visualization or compression. As a consequence we would like to reduce the dimensionality of the data. his is where dimensionality reduction comes in. Dimensionality reduction is an embedding of the data into a significant manifold of fewer dimension within the higher dimensional space in order to encode important information of the dataset. his lower dimension should ideally correspond to the intrinsic dimensionality of the data [?]. Mathematically the above problem can be formulated as follows: Let X = {x i } n i=1 D be a dataset of dimensionality D represented as a n D matrix. Furthermore the data is assumed to lie on or nearby a (smooth) manifold M of dimension d embedded in a D-dimensional space, with the intrinsic dimensionality of the data d being d D. hen there exists a (non-)linear mapping from X to the manifold his transformation maps the dataset X with dimensionality D into a new dataset Y with dimensionality d preserving the main structure of the data. In this setting usually neither the parameter d nor the manifold M are known. here are two major types of dimensionality reduction methods: linear and non-linear ones. In this context linearity refers to the idea that each data point on the manifold is a linear combination of the original data points, i.e. we assume the manifold M to be linear [?]. Non-linear techniques mainly base on at least one of the following qualities [?]: 1. Preservation of global properties or structures of the dataset in the low dimensional dataset 2. Preservation of local properties or structures 3. Composition of linear techniques he choice of a technique depends seriously on the concrete problem setting. ICA he input of the ICA algorithm is a Point cloud data, defined as a finite sequence of vector values, written in matrix form as X = (x 1... x m ) n m. he objective is to find a sequence of source signals S = (s 1... s m ) n m, assuming a linear dependence between X and S. By denoting the mixing matrix as W n n, this property can be expressed as: X = W S with X = (x 1... x m ), and S = (s 1... s m ). In this equation, the mixing matrix W and the source signals S are the unknown variables to be found with the ICA procedure. he second core assumption of the ICA algorithm is the statistical independence of the signals {s i } n i=1. In order to resolve this problem, a general strategy can be described with the following measure for a set of random variables Y = {y i } n i=1 : I(Y ) = D(P Y, i P Yi ) with D(f, g) = f(x)log ( ) f(x) dx. g(x) he measure I allows to compute the degree of statistical independence by comparing the joint distribution P Y, and the marginal distributions P Yi. he comparison function D, used in the measure I, is the Kullback-Leibler distance, also known as relative entropy. he function I allows to express the ICA algorithm as an optimization problem, where the solution space is the General Linear Group, defined as the set of n n invertible matrices: GL(n, ) = {A n n, det(a) 0}, with f(a) := I(A 1 X), and X = (x 1... x m ) n m : min f(a) with A GL(n, ) 3. IME-FEQUENCY ISA In our problem, we consider a bandlimited signal f L 2 () and a segmentation of its domain in such a way that small consecutive signal patches are analyzed, as routinely performed in SF or wavelet analysis. For instance, the set of signal patches can be defined as a dataset X f = {x f i }m i=1, x f i = (f(t k(i 1)+j)) n 1 j=0 n, for k N a fixed hop-size. Here, the regular sampling grid {t l } km k+n 1 l=0 is constructed when considering the Nyquist-Shannon theorem for f. he fundamental problem of signal separation has been described in different applications. A particular example are cocktail party effect problems, where f = g + h is a mixture of two signals g and h, and the objective is to separate g and h from f Our concrete acoustical example is a one-channel signal f composed of two different instruments (represented by g and h). It is reasonable to obtain sample patches x g X g and x h X h, but due to their complex frequency characteristics, an accurate separation of f, specially when g and h are played simultaneously, is a challenging problem. In the particular case of noise reduction, power spectral subtraction is a fundamental strategy which removes the noise signal g from f = g+h by subtracting the frequency content ˆf k ĥk at each frequency bin k [?]. A basic hypothesis is that the noise and clear signal vectors are orthogonal to each other. But this assumption is usually wrong,

3 and a generalized approach takes into account a more accurate geometrical relation between the noise and signal vectors [?]. In our framework we use this generalized scenario but considering point cloud data structures instead of single frequency bins. Given a signal s = s i contaning a mixture of signals s i, the accurate extraction of signals is the denominated signal separation problem. here are different variations including cocktail party effect problem, X g (X g ) X h (X h ) (X g ) (X h )) Ω g h Independent subspace analysis (ISA) is a natural generalization of independent component analysis (ICA). ecall than in ICA, we have a 4. COMPUAIONAL EXPEIMENS Fig. 2. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of Fig. 1. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of Laplacian LSA PCA 5. EFEENCES [1] DS Broomhead and Kirby. A new approach to dimensionality reduction: heory and algorithms. SIAM Journal on Applied Mathematics, 60(6): , Fig. 3. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of [2] A. Casey and A. Westner. Separation of mixed audio sources by independent subspace analysis. In Proceedings of the International Computer Music Conference, [3] D. FitzGerald, E. Coyle, and B. Lawlor. Independent subspace analysis using locally linear embedding. In Proc. DAFx, pages Citeseer, [4] J.A. Lee and Verleysen. Nonlinear dimensionality reduction. Springer, 2007.

4 Fig. 4. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of Fig. 6. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of Fig. 5. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of [5] P. Niyogi, S. Smale, and S. Weinberger. Finding the homology of submanifolds with high confidence from random samples. Discrete and Computational Geometry, 39(1): , Fig. 7. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of

5 Fig. 8. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of Fig. 10. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of Fig. 9. he PCA 3D projection of the frequency content of he Isomap 3D projection of the frequency content of