Undercomplete Independent Component. Analysis for Signal Separation and. Dimension Reduction. Category: Algorithms and Architectures.

Size: px
Start display at page:

Download "Undercomplete Independent Component. Analysis for Signal Separation and. Dimension Reduction. Category: Algorithms and Architectures."

Transcription

1 Undercomplete Independent Component Analysis for Signal Separation and Dimension Reduction John Porrill and James V Stone Psychology Department, Sheeld University, Sheeld, S10 2UR, England. Tel: Fax: , J.Porrill, J.V.Stone@shef.ac.uk Category: Algorithms and Architectures. Author for correspondence: JV Stone. Abstract We introduce undercomplete independent component analysis (uica), a method for extracting K signals from M K mixtures of N M source signals. The mixtures x = (x 1 ; : : :; x M ) T are formed from a linear combination of the independent source signals s = (s 1 ; : : :; s N ) T using an M N mixing matrix A, so that x = As. For the case N = K = M, Bell and Sejnowski [Bell and Sejnowski, 1995] showed that a square N N unmixing matrix W can be found by maximising the joint entropy of M signals (Y 1 ; : : :; Y N ) T = (W x), where is a monotonic, non-linear function. Using a similar approach, we show that a K M unmixing matrix W can be used to recover K M signals (s 1 ; : : :; s K ) T by maximising the joint entropy of K signals Y = (W x). The matrix W is essentially a pseudo-inverse of the M N mixing matrix A. A dierent and widely used method for reducing the size of W is to perform principal component analysis (PCA) on the data set, and to use only the L principal components with the largest eigenvalues as input to ICA. This results in an L L unmixing matrix W. However, there is no a priori reason to assume that independent components exist in only the L-D subspace dened by the L principal components with largest eigenvectors. Thus, discarding some eigenvectors may also corrupt or discard independent components. In contrast, uica does not discard any independent components in the data set, and can extract between 1 and M signals from x. The method is demonstrated on mixtures of high kurtosis (speech and music) and Gaussian signals.

2 Introduction We present a method for extracting K signals from M mixtures of N sources, where K M N. This is a generalisation of the method described by Bell and Sejnowski (B&S) [Bell and Sejnowski, 1995], who showed how K signals could be extracted from M mixtures of N source signals for K = M = N. Both methods can be described informally as follows. The amplitudes of N source signals can be represented as a point in an N-dimensional space, and, considered over all times, they dene a distribution of points in this space. If the signals are from dierent sources then they tend to be statistically independent of each other. A key observation is that, if a signal s has a cumulative density function (cdf) then the distribution of (s) has maximum entropy. Similarly, if N signals each have cdf then the joint distribution of (s) = ((s 1 ); : : :; (s N )) T has maximum entropy. For a set of signal mixtures x = As, a linear unmixing matrix exists such that s = W x. Given that (s) has maximum entropy, s can be recovered by nding a matrix W that maximises the entropy of Y = (W x), at which point W x = s. Why Extract Fewer Sources Than Mxtures? Given a temporal sequence x of M (P P ) images, ICA can be used to extract M spatially independent components (IC) with an (M M) unmixing matrix W. However, such a matrix may be large. In such cases, we would like to be able to extract fewer sources than mixtures. A common method for reducing the size of the unmixing matrix W is to perform principal component analysis (PCA) on the data matrix x, and then to retain the L < M principal components (PCs) with the largest eigenvalues. One property of ICA is that it is insensitive to the RMS amplitude of the ICs it extracts from the mixtures x. However, for a given data set x, there is no a priori reason to suppose that ICs should reside only within the subspace dened by the L PCs with the largest eigenvalues. Thus, discarding a subspace using PCA removes ICs that exist within it, and partially destroys any IC with a non-zero projection onto that subspace. For example, in analysing fmri data, an IC with very small variance was found to be associated with the form of the 'on-o' experimental protocol used [McKeown et al., 1998]. Additionally, the reduced dimensionality of the input space dened by the L retained PCs may contain more ICs than PCs. That is, if the original M-D input space x contains N ICs then, as the number L of retained PCs is reduced, there is an increasing likelihood that L < N. If L < N then a linear decomposition of the L eigenvectors into K <= N sources s does not exist, and therefore ICA cannot work in this case 1. Signal Separation Using Entropy Maximisation Suppose that the outputs x = (x 1 ; : : :; x M ) T of M measurement devices are a linear mixture of N independent signal sources s = (s 1 ; : : :; s N ) T, x = As, where A is an M N mixing matrix. We wish to nd a K M unmixing matrix W such that the K recovered components y = W x are a subset of the original signals s (i.e. K N). 1 Thanks to Martin McKeown for pointing this out

3 Signal Separation For Equal Numbers of Sources and Source Mixtures In the case K = M = N, B&S showed that the unmixing matrix W can be found by maximising the entropy H(Y) of the joint distribution Y = fy 1 ; : : :; Y N g = f 1 (y 1 ); : : :; N (y N )g, where y i = W x i. The correct i have the same form as the cdfs of the input signals x i. However, in many cases it is sucient to approximate these cdfs by sigmoids 2 Y i = tanh(y i ). The output entropy H(Y) can be shown to be related to the entropy of the input H(x) by H(Y) = H(x) + E [ log jjj ] (1) where E denotes expected value, and jjj is the absolute value of the determinant of the Jacobian We can evaluate @y are Jacobian matrices. Equation (1) yields H(Y) = H(x) + E Y N = i(y 0 i )jw j (2) " NX Substituting Equation (2) in + log jw j: (3) The term H(x) is constant, P and can therefore be ignored in the maximisation of H(Y). The term E [ log i 0 ] can be estimated given n samples from the distribution dened by y: E " NX 1 n nx NX j=1 (y(j) i ) (4) Ignoring H(x), and substituting Equation (4) in (3) yields a new function that diers from H(Y) by a constant (= H(x)) h(w ) = 1 n nx NX j=1 (y(j) i ) + log jw j (5) B&S showed how maximising this function with respect to the matrix W can be used to recover linear mixtures of signals. Signal Separation For Unequal Numbers of Sources and Source Mixtures The assumption K = M = N is very restrictive. If the number of sources is unknown one might like to reduce the dimensionality of the problem by looking for a small subset of the source signals, so that in general K < M. This means that a rectangular unmixing matrix is required. This can be determined using the criterion of maximum output entropy which ensures unmixing of those independent variables with cdfs that best t the functional forms of i (this follows from the maximumlikelihood interpretation of the maximum entropy criterion, see [Amari et al., 1996]). Note that if K < M then the mere independence of the outputs y is no longer sucient to guarantee that we have recovered a subset of the input variables because all linear combinations of disjoint subsets of inputs are independent. For 2 In fact sources s i normalised so that E[s i tanh s i] = 1=2 can be separated using tanh sigmoids if and only if the pairwise conditions i j > 1 are satised, where i = 2E[s 2 i ]E[sech 2 s i].

4 example, given four signal mixtures of four source signals, it is possible to extract two independent signals (y 1 and y 1 ) that are linear combinations of disjoint pairs of source signals, y 1 = w 1 s 1 + w 2 s 2 and y 2 = w 3 s 3 + w 4 s 4 ; where the w's are elements in a 2 2 unmixing matrix W. Note that, because y 1 and y 2 are mixtures, they are approximately Gaussian. The criterion that the recovered variables have cdf's approximated by i is thus of much greater importance in the case K < M than if K = M. Thus, to simultaneously optimise both W and the i would be counter-productive for K < M. Equation (1) cannot be used when K < M, so we replace it by H(Y) = H(y) + log j@y=@yj: H(Y) = H(y) + E " KX In general, the entropy H(y) is dicult to calculate. We can approximate it by the entropy of a multi-dimensional Gaussian, which is given by H(y) 1=2 log jcj + K=2 (1 + log 2), where C = Cov[y]. However, as the algorithm converges, this approximation becomes less accurate. This is because the projection of the input data onto the subspace dened by the rows of W denes an increasingly non-gaussian distribution as the algorithm converges. In practice, this approximation seems to work adequately (see Results). More accurate approximations involving higher moments of the distribution than the covariance can be derived if required (for example [Amari et al., 1996]). C can be re-written as C = E[yy T ] = W SW T where S = Cov[x]. In maximising H(Y), we can ignore the constant K=2 (1 + log 2). We can now dene a new function which is an approximation to H(Y), and which diers from it by this constant: h(w ) = 1 2 log jw SW T j + E " KX This can be maximised using its derivative, which can be shown to be: ij = W T ij + E [ 00 i =0 i x j] (8) where W = (SW T )(W SW T )?1, which is the pseudo-inverse of W with respect to the positive denite matrix S. Note that, if K = N then W = W?1. If i = tanh then this evaluates to: r W h = W T? 2 E y T x (9) In our experiments, this gradient was used to maximise equation (7), using a BFGS quasi-newton method. Results Extracting K Source Signals from M K Mixtures Using uica: The method has been tested by extracting signals from mixtures of natural sounds and Gaussian noise. The program was stopped when the correlation r between each of the K outputs and exactly one of the N input signals s was greater than 0:95, or when the number of iterations (evaluations of h(y)) exceeded All signals were normalised to have zero mean and unit variance. The N signals were then linearly combined using a random M N mixing matrix (with normally distributed independent entries) to produce M signal mixtures x, which were used as input to the method. In experiments reported here N = 6. The signals (s 1 ; s 2 ; s 3 ) were a gong, Handel's Messiah, and laughter, respectively, obtained from the MatLab

5 software package. Each of the signals s 4 to s 6 was a dierent random sample of Gaussian noise. Each source signal consisted of a random sample of 20,000 points from each signal. The rst task consists of extracting K = 1; 3 and 4 dierent sound signals from M = 6 linear mixtures of N = 6 signals. For K = 3 all and only the three non- Gaussian sources were recovered, despite dierent initial values for W. For K = 4, these three sources were always recovered, with the highest correlation between one output and a Gaussian source being around 0:8. For K = 1, the algorithm was run three times with dierent random number seeds. On each occasion, s 1 was recovered; this source having the distribution with most kurtosis, so that its cdf is a good match to the tanh non-linearity. The results are displayed in Table 1. These experiments were repeated with M = 12 and 24, with K = 3 and 1. In each case, the required number of sources was recovered, each source was recovered once only, and these sources did not include one of the Gaussian sources. Extracting K Source Signals from M K Mixtures Using PCA/ICA: We compared the results obtained with uica with those obtained using a conventional dimension reduction method (PCA) to preprocess the set of M mixtures. The results presented here involve the three sound sources described in the previous section, plus three other sound sources (obtained from MatLab). Each source consisted of 10,000 samples due to the short length of one source signal. All signals were normalised and mixed together as described in the previous section. First, we ran uica with six mixtures, and set the number of required sources to K = 4. The four extracted signals each had a correlation of jrj > 0:9 with exactly one of the source signals (see Table 2). This result is consistent with those reported in the previous section. Next, we ran ICA after preprocessing with PCA to obtain four eigenvectors with the largest eigenvalues. These were then used as input to ICA using a 4 4 unmixing matrix. From Table 3, only three sources (1,3 and 6) can reasonably considered to have been extracted, with jrj > 0:9. The remaining sources (2, 4 and 5) have maximum correlations with extracted signals of 0.56, 0.59 and 0.50, respectively. Thus, one IC had been eectively discarded along with the two eigenvectors with smallest eigenvalues. For completeness, all six eigenvectors and a 6 6 unmixing matrix were used. In this case, each source signal had a correlation jrj > 0:95 with exactly one extracted signal and jrj < 0:25 with the remaining extracted signals. These results demonstrate that using PCA to reduce the number of signal mixtures used as input to ICA can compromise the ability of ICA to extract source signals. This is because the source signals had non-zero projections on to the discarded eigenvectors. In contrast, uica does not require PCA, and can therefore extract exactly K N source signals from M N signal mixtures. Discussion Two alternative approaches to the problem of dimension reduction are: 1) reduce the dimension of the mixture space from M to K before separating signals, which risks corrupting ICs, or, 2) nd K = M > N components (which could be a large number) using the B&S's method, and then apply a separate method to identify the K < M most important components (see [Cichocki and Kasprzak, 1996]). With respect to 1, discarding PCs cannot also inadvertently discard ICs if only PCs with

6 zero eigenvalues are discarded. However, zero eigenvalues are rarely encountered with noisy data, so that one is forced to risk corrupting ICs when using PCA to reduce the dimensionality of data used as input to ICA. The method we have described unies approaches 1 and 2, without compromising the ability of ICA to extract ICs. A logical modication to our algorithm would be to extract ICs one at a time, as in [Girolami and Fyfe, 1996] using projection pursuit indices. This involves extracting one IC from the M-dimensional space of signal mixtures x, 'removing' the one-dimensional subspace corresponding to that IC (using Gramm-Schmidt orthonormalisation), and then extracting the next IC. This operation is repeated until all the sources have been extracted. This method has been tested on the data described above, and results are not noticably dierent from those reported here. However, we conjecture that sequential extraction of sources does not provide similar results to ICA in general. Consider image data which is a mixture of two spatial ICs each of which has exactly one region (A and B, respectively) with non-zero grey-levels, and the same grey-levels in an overlapping region C of the IC images. The independence criterion implict in ICA would force it to identify three ICs, corresponding to regions (A? C), (B? C) and C. In contrast, the `cdf-matching' criterion implicit in projection pursuit methods (and uica for K = 1) would ensure that an IC corresponding to A would be extracted rst, followed by B 3. Conclusion Using an undercomplete basis set to extract ICs is useful when the number of signal mixtures is larger than the number of source signals. This commonly occurs in high dimensional data sets in which each signal is an entire image, or even a sequence of images. The obvious strategy of discarding PCs of the data set that have small eigenvalues can compromise ICA's ability to extract ICs for two reasons. First, ICs may have non-zero projections onto the subspace dened by these discarded PCs; so partially destroying these ICs. Second, ICA is only possible if the number of source signals is equal to or less than the number of signal mixtures. Using PCA to eectively reduce the number of mixtures therefore increases the probability that the number of source signals is greater than the number of mixtures. In contrast, uica extracts a specied number of sources signals from the original data set, and therefore precludes the problems associated with preprocessing with PCA. Acknowledgements: Thanks to members of the Computational Neurobiology Laboratory at the Salk Institute, and to Tony Bell, for comments on this work. Thanks to Stephen Isard for comments on a previous draft of this paper. J Stone is supported by a Mathematical Biology Wellcome Fellowship (Grant number ). References [Amari et al., 1996] Amari, S., Cichocki, A., and Yang, H. (1996). A new learning algorithm for blind signal separation. In Touretzky, D., Mozer, M., and Hasselmo, M., editors, Advances in Neural Information Processing Systems 8. MIT Press, Cambridge MA (In Press). [Bell and Sejnowski, 1995] Bell, A. and Sejnowski, T. (1995). An informationmaximization approach to blind separation and blind deconvolution. Neural Computation, 7:1129{ Thanks for Martin McKeown for pointing this out

7 [Cichocki and Kasprzak, 1996] Cichocki, A. and Kasprzak, W. (1996). Local adapative learning algorithms for blind separation of natural images. Neural Network World, 6(4):515{523. [Girolami and Fyfe, 1996] Girolami, M. and Fyfe, C. (1996). Negentropy and kurtosis as projection pursuit indices provide generalised ica algorithms. NIPS96 Blind Signal Separation Workshop. [McKeown et al., 1998] McKeown, M., Makeig, S., Brown, G., Jung, T., Kindermann, S., and Sejnowski, T. (1998). Spatially independent activity patterns in functional magnetic resonance imaging data during the stroop color-naming task. Proceedings of the National Academy of Sciences USA., 95:803{810. Sources (N) Mixtures (M) Required (K) Extracted Iterations Table 1: Performance for N=6 signals s = (s 1 ; : : :; s 6 ). (s 1 ; s 2 ; s 3 ) are a gong, Handel's Messiah, and laughter, respectively, and (s 4 ; s 5 ; s 6 ) are three Gaussian noise signals. The method was tested with dierent numbers M of signal mixtures and dierent numbers K of required outputs. Iterations denotes the number of function evaluations of h(w ) required for convergence (see text). Src 1 Src 2 Src 3 Src 4 Src 5 Src Table 2: Using uica to extract 4 signals from a mixture of 6 sound signals. Each cell species the absolute value of the correlation jrj between a source signal (columns) and a signal extracted by uica (rows). Src 1 Src 2 Src 3 Src 4 Src 5 Src Table 3: Using PCA to preprocess six mixtures to obtain four PCs. Each cell species the absolute value of the correlation jrj between a source signal (columns) and an extracted signal (rows). The four PCs were used as input to ICA. Only three of the source signals (in bold typeface) can be considered to have been extracted, with correlations jrj > 0:90.

File: ica tutorial2.tex. James V Stone and John Porrill, Psychology Department, Sheeld University, Tel: Fax:

File: ica tutorial2.tex. James V Stone and John Porrill, Psychology Department, Sheeld University, Tel: Fax: File: ica tutorial2.tex Independent Component Analysis and Projection Pursuit: A Tutorial Introduction James V Stone and John Porrill, Psychology Department, Sheeld University, Sheeld, S 2UR, England.

More information

Independent Component Analysis

Independent Component Analysis Independent Component Analysis James V. Stone November 4, 24 Sheffield University, Sheffield, UK Keywords: independent component analysis, independence, blind source separation, projection pursuit, complexity

More information

ICA [6] ICA) [7, 8] ICA ICA ICA [9, 10] J-F. Cardoso. [13] Matlab ICA. Comon[3], Amari & Cardoso[4] ICA ICA

ICA [6] ICA) [7, 8] ICA ICA ICA [9, 10] J-F. Cardoso. [13] Matlab ICA. Comon[3], Amari & Cardoso[4] ICA ICA 16 1 (Independent Component Analysis: ICA) 198 9 ICA ICA ICA 1 ICA 198 Jutten Herault Comon[3], Amari & Cardoso[4] ICA Comon (PCA) projection persuit projection persuit ICA ICA ICA 1 [1] [] ICA ICA EEG

More information

Natural Gradient Learning for Over- and Under-Complete Bases in ICA

Natural Gradient Learning for Over- and Under-Complete Bases in ICA NOTE Communicated by Jean-François Cardoso Natural Gradient Learning for Over- and Under-Complete Bases in ICA Shun-ichi Amari RIKEN Brain Science Institute, Wako-shi, Hirosawa, Saitama 351-01, Japan Independent

More information

Independent Component Analysis

Independent Component Analysis A Short Introduction to Independent Component Analysis Aapo Hyvärinen Helsinki Institute for Information Technology and Depts of Computer Science and Psychology University of Helsinki Problem of blind

More information

1 Introduction Independent component analysis (ICA) [10] is a statistical technique whose main applications are blind source separation, blind deconvo

1 Introduction Independent component analysis (ICA) [10] is a statistical technique whose main applications are blind source separation, blind deconvo The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis Aapo Hyvarinen Helsinki University of Technology Laboratory of Computer and Information Science P.O.Box 5400,

More information

Independent component analysis: an introduction

Independent component analysis: an introduction Research Update 59 Techniques & Applications Independent component analysis: an introduction James V. Stone Independent component analysis (ICA) is a method for automatically identifying the underlying

More information

A Constrained EM Algorithm for Independent Component Analysis

A Constrained EM Algorithm for Independent Component Analysis LETTER Communicated by Hagai Attias A Constrained EM Algorithm for Independent Component Analysis Max Welling Markus Weber California Institute of Technology, Pasadena, CA 91125, U.S.A. We introduce a

More information

Fundamentals of Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Independent Vector Analysis (IVA)

Fundamentals of Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Independent Vector Analysis (IVA) Fundamentals of Principal Component Analysis (PCA),, and Independent Vector Analysis (IVA) Dr Mohsen Naqvi Lecturer in Signal and Information Processing, School of Electrical and Electronic Engineering,

More information

HST.582J/6.555J/16.456J

HST.582J/6.555J/16.456J Blind Source Separation: PCA & ICA HST.582J/6.555J/16.456J Gari D. Clifford gari [at] mit. edu http://www.mit.edu/~gari G. D. Clifford 2005-2009 What is BSS? Assume an observation (signal) is a linear

More information

Independent Component Analysis of Incomplete Data

Independent Component Analysis of Incomplete Data Independent Component Analysis of Incomplete Data Max Welling Markus Weber California Institute of Technology 136-93 Pasadena, CA 91125 fwelling,rmwg@vision.caltech.edu Keywords: EM, Missing Data, ICA

More information

CIFAR Lectures: Non-Gaussian statistics and natural images

CIFAR Lectures: Non-Gaussian statistics and natural images CIFAR Lectures: Non-Gaussian statistics and natural images Dept of Computer Science University of Helsinki, Finland Outline Part I: Theory of ICA Definition and difference to PCA Importance of non-gaussianity

More information

below, kernel PCA Eigenvectors, and linear combinations thereof. For the cases where the pre-image does exist, we can provide a means of constructing

below, kernel PCA Eigenvectors, and linear combinations thereof. For the cases where the pre-image does exist, we can provide a means of constructing Kernel PCA Pattern Reconstruction via Approximate Pre-Images Bernhard Scholkopf, Sebastian Mika, Alex Smola, Gunnar Ratsch, & Klaus-Robert Muller GMD FIRST, Rudower Chaussee 5, 12489 Berlin, Germany fbs,

More information

Independent Component Analysis

Independent Component Analysis 1 Independent Component Analysis Background paper: http://www-stat.stanford.edu/ hastie/papers/ica.pdf 2 ICA Problem X = AS where X is a random p-vector representing multivariate input measurements. S

More information

Principal Component Analysis

Principal Component Analysis Principal Component Analysis Introduction Consider a zero mean random vector R n with autocorrelation matri R = E( T ). R has eigenvectors q(1),,q(n) and associated eigenvalues λ(1) λ(n). Let Q = [ q(1)

More information

Gatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts I-II

Gatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts I-II Gatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts I-II Gatsby Unit University College London 27 Feb 2017 Outline Part I: Theory of ICA Definition and difference

More information

Independent Component Analysis (ICA) Bhaskar D Rao University of California, San Diego

Independent Component Analysis (ICA) Bhaskar D Rao University of California, San Diego Independent Component Analysis (ICA) Bhaskar D Rao University of California, San Diego Email: brao@ucsdedu References 1 Hyvarinen, A, Karhunen, J, & Oja, E (2004) Independent component analysis (Vol 46)

More information

Independent Component Analysis

Independent Component Analysis A Short Introduction to Independent Component Analysis with Some Recent Advances Aapo Hyvärinen Dept of Computer Science Dept of Mathematics and Statistics University of Helsinki Problem of blind source

More information

Independent Component Analysis and Its Applications. By Qing Xue, 10/15/2004

Independent Component Analysis and Its Applications. By Qing Xue, 10/15/2004 Independent Component Analysis and Its Applications By Qing Xue, 10/15/2004 Outline Motivation of ICA Applications of ICA Principles of ICA estimation Algorithms for ICA Extensions of basic ICA framework

More information

PCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani

PCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani PCA & ICA CE-717: Machine Learning Sharif University of Technology Spring 2015 Soleymani Dimensionality Reduction: Feature Selection vs. Feature Extraction Feature selection Select a subset of a given

More information

Advanced Introduction to Machine Learning CMU-10715

Advanced Introduction to Machine Learning CMU-10715 Advanced Introduction to Machine Learning CMU-10715 Independent Component Analysis Barnabás Póczos Independent Component Analysis 2 Independent Component Analysis Model original signals Observations (Mixtures)

More information

PROPERTIES OF THE EMPIRICAL CHARACTERISTIC FUNCTION AND ITS APPLICATION TO TESTING FOR INDEPENDENCE. Noboru Murata

PROPERTIES OF THE EMPIRICAL CHARACTERISTIC FUNCTION AND ITS APPLICATION TO TESTING FOR INDEPENDENCE. Noboru Murata ' / PROPERTIES OF THE EMPIRICAL CHARACTERISTIC FUNCTION AND ITS APPLICATION TO TESTING FOR INDEPENDENCE Noboru Murata Waseda University Department of Electrical Electronics and Computer Engineering 3--

More information

HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007

HST.582J / 6.555J / J Biomedical Signal and Image Processing Spring 2007 MIT OpenCourseWare http://ocw.mit.edu HST.582J / 6.555J / 16.456J Biomedical Signal and Image Processing Spring 2007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

More information

Gaussian process for nonstationary time series prediction

Gaussian process for nonstationary time series prediction Computational Statistics & Data Analysis 47 (2004) 705 712 www.elsevier.com/locate/csda Gaussian process for nonstationary time series prediction Soane Brahim-Belhouari, Amine Bermak EEE Department, Hong

More information

Independent Component Analysis. Contents

Independent Component Analysis. Contents Contents Preface xvii 1 Introduction 1 1.1 Linear representation of multivariate data 1 1.1.1 The general statistical setting 1 1.1.2 Dimension reduction methods 2 1.1.3 Independence as a guiding principle

More information

Linear Regression and Its Applications

Linear Regression and Its Applications Linear Regression and Its Applications Predrag Radivojac October 13, 2014 Given a data set D = {(x i, y i )} n the objective is to learn the relationship between features and the target. We usually start

More information

1 Introduction Blind source separation (BSS) is a fundamental problem which is encountered in a variety of signal processing problems where multiple s

1 Introduction Blind source separation (BSS) is a fundamental problem which is encountered in a variety of signal processing problems where multiple s Blind Separation of Nonstationary Sources in Noisy Mixtures Seungjin CHOI x1 and Andrzej CICHOCKI y x Department of Electrical Engineering Chungbuk National University 48 Kaeshin-dong, Cheongju Chungbuk

More information

Cheng Soon Ong & Christian Walder. Canberra February June 2018

Cheng Soon Ong & Christian Walder. Canberra February June 2018 Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 Outlines Overview Introduction Linear Algebra Probability Linear Regression

More information

An Improved Cumulant Based Method for Independent Component Analysis

An Improved Cumulant Based Method for Independent Component Analysis An Improved Cumulant Based Method for Independent Component Analysis Tobias Blaschke and Laurenz Wiskott Institute for Theoretical Biology Humboldt University Berlin Invalidenstraße 43 D - 0 5 Berlin Germany

More information

Non-Euclidean Independent Component Analysis and Oja's Learning

Non-Euclidean Independent Component Analysis and Oja's Learning Non-Euclidean Independent Component Analysis and Oja's Learning M. Lange 1, M. Biehl 2, and T. Villmann 1 1- University of Appl. Sciences Mittweida - Dept. of Mathematics Mittweida, Saxonia - Germany 2-

More information

ICA. Independent Component Analysis. Zakariás Mátyás

ICA. Independent Component Analysis. Zakariás Mátyás ICA Independent Component Analysis Zakariás Mátyás Contents Definitions Introduction History Algorithms Code Uses of ICA Definitions ICA Miture Separation Signals typical signals Multivariate statistics

More information

Independent Component Analysis (ICA)

Independent Component Analysis (ICA) Independent Component Analysis (ICA) Université catholique de Louvain (Belgium) Machine Learning Group http://www.dice.ucl ucl.ac.be/.ac.be/mlg/ 1 Overview Uncorrelation vs Independence Blind source separation

More information

Statistical Analysis of fmrl Data

Statistical Analysis of fmrl Data Statistical Analysis of fmrl Data F. Gregory Ashby The MIT Press Cambridge, Massachusetts London, England Preface xi Acronyms xv 1 Introduction 1 What Is fmri? 2 The Scanning Session 4 Experimental Design

More information

Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources

Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources LETTER Communicated by Jean-François Cardoso Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources Te-Won Lee Howard Hughes Medical Institute,

More information

Artificial Intelligence Module 2. Feature Selection. Andrea Torsello

Artificial Intelligence Module 2. Feature Selection. Andrea Torsello Artificial Intelligence Module 2 Feature Selection Andrea Torsello We have seen that high dimensional data is hard to classify (curse of dimensionality) Often however, the data does not fill all the space

More information

Introduction to Independent Component Analysis. Jingmei Lu and Xixi Lu. Abstract

Introduction to Independent Component Analysis. Jingmei Lu and Xixi Lu. Abstract Final Project 2//25 Introduction to Independent Component Analysis Abstract Independent Component Analysis (ICA) can be used to solve blind signal separation problem. In this article, we introduce definition

More information

Massoud BABAIE-ZADEH. Blind Source Separation (BSS) and Independent Componen Analysis (ICA) p.1/39

Massoud BABAIE-ZADEH. Blind Source Separation (BSS) and Independent Componen Analysis (ICA) p.1/39 Blind Source Separation (BSS) and Independent Componen Analysis (ICA) Massoud BABAIE-ZADEH Blind Source Separation (BSS) and Independent Componen Analysis (ICA) p.1/39 Outline Part I Part II Introduction

More information

To appear in Proceedings of the ICA'99, Aussois, France, A 2 R mn is an unknown mixture matrix of full rank, v(t) is the vector of noises. The

To appear in Proceedings of the ICA'99, Aussois, France, A 2 R mn is an unknown mixture matrix of full rank, v(t) is the vector of noises. The To appear in Proceedings of the ICA'99, Aussois, France, 1999 1 NATURAL GRADIENT APPROACH TO BLIND SEPARATION OF OVER- AND UNDER-COMPLETE MIXTURES L.-Q. Zhang, S. Amari and A. Cichocki Brain-style Information

More information

ADAPTIVE LATERAL INHIBITION FOR NON-NEGATIVE ICA. Mark Plumbley

ADAPTIVE LATERAL INHIBITION FOR NON-NEGATIVE ICA. Mark Plumbley Submitteed to the International Conference on Independent Component Analysis and Blind Signal Separation (ICA2) ADAPTIVE LATERAL INHIBITION FOR NON-NEGATIVE ICA Mark Plumbley Audio & Music Lab Department

More information

Learning with Ensembles: How. over-tting can be useful. Anders Krogh Copenhagen, Denmark. Abstract

Learning with Ensembles: How. over-tting can be useful. Anders Krogh Copenhagen, Denmark. Abstract Published in: Advances in Neural Information Processing Systems 8, D S Touretzky, M C Mozer, and M E Hasselmo (eds.), MIT Press, Cambridge, MA, pages 190-196, 1996. Learning with Ensembles: How over-tting

More information

Separation of the EEG Signal using Improved FastICA Based on Kurtosis Contrast Function

Separation of the EEG Signal using Improved FastICA Based on Kurtosis Contrast Function Australian Journal of Basic and Applied Sciences, 5(9): 2152-2156, 211 ISSN 1991-8178 Separation of the EEG Signal using Improved FastICA Based on Kurtosis Contrast Function 1 Tahir Ahmad, 2 Hjh.Norma

More information

Single Channel Signal Separation Using MAP-based Subspace Decomposition

Single Channel Signal Separation Using MAP-based Subspace Decomposition Single Channel Signal Separation Using MAP-based Subspace Decomposition Gil-Jin Jang, Te-Won Lee, and Yung-Hwan Oh 1 Spoken Language Laboratory, Department of Computer Science, KAIST 373-1 Gusong-dong,

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning Christoph Lampert Spring Semester 2015/2016 // Lecture 12 1 / 36 Unsupervised Learning Dimensionality Reduction 2 / 36 Dimensionality Reduction Given: data X = {x 1,..., x

More information

ICA and ISA Using Schweizer-Wolff Measure of Dependence

ICA and ISA Using Schweizer-Wolff Measure of Dependence Keywords: independent component analysis, independent subspace analysis, copula, non-parametric estimation of dependence Abstract We propose a new algorithm for independent component and independent subspace

More information

Plan of Class 4. Radial Basis Functions with moving centers. Projection Pursuit Regression and ridge. Principal Component Analysis: basic ideas

Plan of Class 4. Radial Basis Functions with moving centers. Projection Pursuit Regression and ridge. Principal Component Analysis: basic ideas Plan of Class 4 Radial Basis Functions with moving centers Multilayer Perceptrons Projection Pursuit Regression and ridge functions approximation Principal Component Analysis: basic ideas Radial Basis

More information

Independent Component Analysis

Independent Component Analysis Independent Component Analysis Seungjin Choi Department of Computer Science Pohang University of Science and Technology, Korea seungjin@postech.ac.kr March 4, 2009 1 / 78 Outline Theory and Preliminaries

More information

Unsupervised learning: beyond simple clustering and PCA

Unsupervised learning: beyond simple clustering and PCA Unsupervised learning: beyond simple clustering and PCA Liza Rebrova Self organizing maps (SOM) Goal: approximate data points in R p by a low-dimensional manifold Unlike PCA, the manifold does not have

More information

Independent Components Analysis

Independent Components Analysis CS229 Lecture notes Andrew Ng Part XII Independent Components Analysis Our next topic is Independent Components Analysis (ICA). Similar to PCA, this will find a new basis in which to represent our data.

More information

Blind Machine Separation Te-Won Lee

Blind Machine Separation Te-Won Lee Blind Machine Separation Te-Won Lee University of California, San Diego Institute for Neural Computation Blind Machine Separation Problem we want to solve: Single microphone blind source separation & deconvolution

More information

Rigid Structure from Motion from a Blind Source Separation Perspective

Rigid Structure from Motion from a Blind Source Separation Perspective Noname manuscript No. (will be inserted by the editor) Rigid Structure from Motion from a Blind Source Separation Perspective Jeff Fortuna Aleix M. Martinez Received: date / Accepted: date Abstract We

More information

Soft-LOST: EM on a Mixture of Oriented Lines

Soft-LOST: EM on a Mixture of Oriented Lines Soft-LOST: EM on a Mixture of Oriented Lines Paul D. O Grady and Barak A. Pearlmutter Hamilton Institute National University of Ireland Maynooth Co. Kildare Ireland paul.ogrady@may.ie barak@cs.may.ie Abstract.

More information

Gaussian Processes for Regression. Carl Edward Rasmussen. Department of Computer Science. Toronto, ONT, M5S 1A4, Canada.

Gaussian Processes for Regression. Carl Edward Rasmussen. Department of Computer Science. Toronto, ONT, M5S 1A4, Canada. In Advances in Neural Information Processing Systems 8 eds. D. S. Touretzky, M. C. Mozer, M. E. Hasselmo, MIT Press, 1996. Gaussian Processes for Regression Christopher K. I. Williams Neural Computing

More information

An Introduction to Independent Components Analysis (ICA)

An Introduction to Independent Components Analysis (ICA) An Introduction to Independent Components Analysis (ICA) Anish R. Shah, CFA Northfield Information Services Anish@northinfo.com Newport Jun 6, 2008 1 Overview of Talk Review principal components Introduce

More information

On Information Maximization and Blind Signal Deconvolution

On Information Maximization and Blind Signal Deconvolution On Information Maximization and Blind Signal Deconvolution A Röbel Technical University of Berlin, Institute of Communication Sciences email: roebel@kgwtu-berlinde Abstract: In the following paper we investigate

More information

Remaining energy on log scale Number of linear PCA components

Remaining energy on log scale Number of linear PCA components NONLINEAR INDEPENDENT COMPONENT ANALYSIS USING ENSEMBLE LEARNING: EXPERIMENTS AND DISCUSSION Harri Lappalainen, Xavier Giannakopoulos, Antti Honkela, and Juha Karhunen Helsinki University of Technology,

More information

Robust extraction of specific signals with temporal structure

Robust extraction of specific signals with temporal structure Robust extraction of specific signals with temporal structure Zhi-Lin Zhang, Zhang Yi Computational Intelligence Laboratory, School of Computer Science and Engineering, University of Electronic Science

More information

c Springer, Reprinted with permission.

c Springer, Reprinted with permission. Zhijian Yuan and Erkki Oja. A FastICA Algorithm for Non-negative Independent Component Analysis. In Puntonet, Carlos G.; Prieto, Alberto (Eds.), Proceedings of the Fifth International Symposium on Independent

More information

2 Tikhonov Regularization and ERM

2 Tikhonov Regularization and ERM Introduction Here we discusses how a class of regularization methods originally designed to solve ill-posed inverse problems give rise to regularized learning algorithms. These algorithms are kernel methods

More information

Recursive Generalized Eigendecomposition for Independent Component Analysis

Recursive Generalized Eigendecomposition for Independent Component Analysis Recursive Generalized Eigendecomposition for Independent Component Analysis Umut Ozertem 1, Deniz Erdogmus 1,, ian Lan 1 CSEE Department, OGI, Oregon Health & Science University, Portland, OR, USA. {ozertemu,deniz}@csee.ogi.edu

More information

Chapter 15 - BLIND SOURCE SEPARATION:

Chapter 15 - BLIND SOURCE SEPARATION: HST-582J/6.555J/16.456J Biomedical Signal and Image Processing Spr ing 2005 Chapter 15 - BLIND SOURCE SEPARATION: Principal & Independent Component Analysis c G.D. Clifford 2005 Introduction In this chapter

More information

Multi-Class Linear Dimension Reduction by. Weighted Pairwise Fisher Criteria

Multi-Class Linear Dimension Reduction by. Weighted Pairwise Fisher Criteria Multi-Class Linear Dimension Reduction by Weighted Pairwise Fisher Criteria M. Loog 1,R.P.W.Duin 2,andR.Haeb-Umbach 3 1 Image Sciences Institute University Medical Center Utrecht P.O. Box 85500 3508 GA

More information

CONVOLUTIVE NON-NEGATIVE MATRIX FACTORISATION WITH SPARSENESS CONSTRAINT

CONVOLUTIVE NON-NEGATIVE MATRIX FACTORISATION WITH SPARSENESS CONSTRAINT CONOLUTIE NON-NEGATIE MATRIX FACTORISATION WITH SPARSENESS CONSTRAINT Paul D. O Grady Barak A. Pearlmutter Hamilton Institute National University of Ireland, Maynooth Co. Kildare, Ireland. ABSTRACT Discovering

More information

A Canonical Genetic Algorithm for Blind Inversion of Linear Channels

A Canonical Genetic Algorithm for Blind Inversion of Linear Channels A Canonical Genetic Algorithm for Blind Inversion of Linear Channels Fernando Rojas, Jordi Solé-Casals, Enric Monte-Moreno 3, Carlos G. Puntonet and Alberto Prieto Computer Architecture and Technology

More information

Extraction of Sleep-Spindles from the Electroencephalogram (EEG)

Extraction of Sleep-Spindles from the Electroencephalogram (EEG) Extraction of Sleep-Spindles from the Electroencephalogram (EEG) Allan Kardec Barros Bio-mimetic Control Research Center, RIKEN, 2271-13 Anagahora, Shimoshidami, Moriyama-ku, Nagoya 463, Japan Roman Rosipal,

More information

Different Estimation Methods for the Basic Independent Component Analysis Model

Different Estimation Methods for the Basic Independent Component Analysis Model Washington University in St. Louis Washington University Open Scholarship Arts & Sciences Electronic Theses and Dissertations Arts & Sciences Winter 12-2018 Different Estimation Methods for the Basic Independent

More information

CS281 Section 4: Factor Analysis and PCA

CS281 Section 4: Factor Analysis and PCA CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we

More information

Lecture 10: Dimension Reduction Techniques

Lecture 10: Dimension Reduction Techniques Lecture 10: Dimension Reduction Techniques Radu Balan Department of Mathematics, AMSC, CSCAMM and NWC University of Maryland, College Park, MD April 17, 2018 Input Data It is assumed that there is a set

More information

where A 2 IR m n is the mixing matrix, s(t) is the n-dimensional source vector (n» m), and v(t) is additive white noise that is statistically independ

where A 2 IR m n is the mixing matrix, s(t) is the n-dimensional source vector (n» m), and v(t) is additive white noise that is statistically independ BLIND SEPARATION OF NONSTATIONARY AND TEMPORALLY CORRELATED SOURCES FROM NOISY MIXTURES Seungjin CHOI x and Andrzej CICHOCKI y x Department of Electrical Engineering Chungbuk National University, KOREA

More information

Matching the dimensionality of maps with that of the data

Matching the dimensionality of maps with that of the data Matching the dimensionality of maps with that of the data COLIN FYFE Applied Computational Intelligence Research Unit, The University of Paisley, Paisley, PA 2BE SCOTLAND. Abstract Topographic maps are

More information

Tutorial on Blind Source Separation and Independent Component Analysis

Tutorial on Blind Source Separation and Independent Component Analysis Tutorial on Blind Source Separation and Independent Component Analysis Lucas Parra Adaptive Image & Signal Processing Group Sarnoff Corporation February 09, 2002 Linear Mixtures... problem statement...

More information

Undercomplete Blind Subspace Deconvolution via Linear Prediction

Undercomplete Blind Subspace Deconvolution via Linear Prediction Undercomplete Blind Subspace Deconvolution via Linear Prediction Zoltán Szabó, Barnabás Póczos, and András L rincz Department of Information Systems, Eötvös Loránd University, Pázmány P. sétány 1/C, Budapest

More information

(a)

(a) Chapter 8 Subspace Methods 8. Introduction Principal Component Analysis (PCA) is applied to the analysis of time series data. In this context we discuss measures of complexity and subspace methods for

More information

LECTURE :ICA. Rita Osadchy. Based on Lecture Notes by A. Ng

LECTURE :ICA. Rita Osadchy. Based on Lecture Notes by A. Ng LECURE :ICA Rita Osadchy Based on Lecture Notes by A. Ng Cocktail Party Person 1 2 s 1 Mike 2 s 3 Person 3 1 Mike 1 s 2 Person 2 3 Mike 3 microphone signals are mied speech signals 1 2 3 ( t) ( t) ( t)

More information

Independent component analysis: algorithms and applications

Independent component analysis: algorithms and applications PERGAMON Neural Networks 13 (2000) 411 430 Invited article Independent component analysis: algorithms and applications A. Hyvärinen, E. Oja* Neural Networks Research Centre, Helsinki University of Technology,

More information

Final Report For Undergraduate Research Opportunities Project Name: Biomedical Signal Processing in EEG. Zhang Chuoyao 1 and Xu Jianxin 2

Final Report For Undergraduate Research Opportunities Project Name: Biomedical Signal Processing in EEG. Zhang Chuoyao 1 and Xu Jianxin 2 ABSTRACT Final Report For Undergraduate Research Opportunities Project Name: Biomedical Signal Processing in EEG Zhang Chuoyao 1 and Xu Jianxin 2 Department of Electrical and Computer Engineering, National

More information

Independent Component Analysis and Its Application on Accelerator Physics

Independent Component Analysis and Its Application on Accelerator Physics Independent Component Analysis and Its Application on Accelerator Physics Xiaoying Pang LA-UR-12-20069 ICA and PCA Similarities: Blind source separation method (BSS) no model Observed signals are linear

More information

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017

COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 COMS 4721: Machine Learning for Data Science Lecture 19, 4/6/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University PRINCIPAL COMPONENT ANALYSIS DIMENSIONALITY

More information

Outline Introduction: Problem Description Diculties Algebraic Structure: Algebraic Varieties Rank Decient Toeplitz Matrices Constructing Lower Rank St

Outline Introduction: Problem Description Diculties Algebraic Structure: Algebraic Varieties Rank Decient Toeplitz Matrices Constructing Lower Rank St Structured Lower Rank Approximation by Moody T. Chu (NCSU) joint with Robert E. Funderlic (NCSU) and Robert J. Plemmons (Wake Forest) March 5, 1998 Outline Introduction: Problem Description Diculties Algebraic

More information

Comparative Performance Analysis of Three Algorithms for Principal Component Analysis

Comparative Performance Analysis of Three Algorithms for Principal Component Analysis 84 R. LANDQVIST, A. MOHAMMED, COMPARATIVE PERFORMANCE ANALYSIS OF THR ALGORITHMS Comparative Performance Analysis of Three Algorithms for Principal Component Analysis Ronnie LANDQVIST, Abbas MOHAMMED Dept.

More information

One-unit Learning Rules for Independent Component Analysis

One-unit Learning Rules for Independent Component Analysis One-unit Learning Rules for Independent Component Analysis Aapo Hyvarinen and Erkki Oja Helsinki University of Technology Laboratory of Computer and Information Science Rakentajanaukio 2 C, FIN-02150 Espoo,

More information

MULTICHANNEL BLIND SEPARATION AND. Scott C. Douglas 1, Andrzej Cichocki 2, and Shun-ichi Amari 2

MULTICHANNEL BLIND SEPARATION AND. Scott C. Douglas 1, Andrzej Cichocki 2, and Shun-ichi Amari 2 MULTICHANNEL BLIND SEPARATION AND DECONVOLUTION OF SOURCES WITH ARBITRARY DISTRIBUTIONS Scott C. Douglas 1, Andrzej Cichoci, and Shun-ichi Amari 1 Department of Electrical Engineering, University of Utah

More information

An Adaptive Bayesian Network for Low-Level Image Processing

An Adaptive Bayesian Network for Low-Level Image Processing An Adaptive Bayesian Network for Low-Level Image Processing S P Luttrell Defence Research Agency, Malvern, Worcs, WR14 3PS, UK. I. INTRODUCTION Probability calculus, based on the axioms of inference, Cox

More information

On the INFOMAX algorithm for blind signal separation

On the INFOMAX algorithm for blind signal separation University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 2000 On the INFOMAX algorithm for blind signal separation Jiangtao Xi

More information

Blind Source Separation Using Artificial immune system

Blind Source Separation Using Artificial immune system American Journal of Engineering Research (AJER) e-issn : 2320-0847 p-issn : 2320-0936 Volume-03, Issue-02, pp-240-247 www.ajer.org Research Paper Open Access Blind Source Separation Using Artificial immune

More information

Blind Extraction of Singularly Mixed Source Signals

Blind Extraction of Singularly Mixed Source Signals IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL 11, NO 6, NOVEMBER 2000 1413 Blind Extraction of Singularly Mixed Source Signals Yuanqing Li, Jun Wang, Senior Member, IEEE, and Jacek M Zurada, Fellow, IEEE Abstract

More information

Entropy Manipulation of Arbitrary Non I inear Map pings

Entropy Manipulation of Arbitrary Non I inear Map pings Entropy Manipulation of Arbitrary Non I inear Map pings John W. Fisher I11 JosC C. Principe Computational NeuroEngineering Laboratory EB, #33, PO Box 116130 University of Floridaa Gainesville, FL 326 1

More information

Fast blind separation based on information theory.

Fast blind separation based on information theory. Proceedings 1 YY5 Internatiotlal Symposium un Non-1 i neat- Thec~r-y and Applications 1 :43-47. (I ti press) Fast blind separation based on information theory. Anthony J. Bell & Terrence J. Sejnowski Computational

More information

Mark Gales October y (x) x 1. x 2 y (x) Inputs. Outputs. x d. y (x) Second Output layer layer. layer.

Mark Gales October y (x) x 1. x 2 y (x) Inputs. Outputs. x d. y (x) Second Output layer layer. layer. University of Cambridge Engineering Part IIB & EIST Part II Paper I0: Advanced Pattern Processing Handouts 4 & 5: Multi-Layer Perceptron: Introduction and Training x y (x) Inputs x 2 y (x) 2 Outputs x

More information

Localization of Multiple Deep Epileptic Sources in a Realistic Head Model via Independent Component Analysis

Localization of Multiple Deep Epileptic Sources in a Realistic Head Model via Independent Component Analysis Localization of Multiple Deep Epileptic Sources in a Realistic Head Model via Independent Component Analysis David Weinstein, Leonid Zhukov, Geoffrey Potts Email: dmw@cs.utah.edu, zhukov@cs.utah.edu, gpotts@rice.edu

More information

PCA, Kernel PCA, ICA

PCA, Kernel PCA, ICA PCA, Kernel PCA, ICA Learning Representations. Dimensionality Reduction. Maria-Florina Balcan 04/08/2015 Big & High-Dimensional Data High-Dimensions = Lot of Features Document classification Features per

More information

BLIND SEPARATION OF POSITIVE SOURCES USING NON-NEGATIVE PCA

BLIND SEPARATION OF POSITIVE SOURCES USING NON-NEGATIVE PCA BLIND SEPARATION OF POSITIVE SOURCES USING NON-NEGATIVE PCA Erkki Oja Neural Networks Research Centre Helsinki University of Technology P.O.Box 54, 215 HUT, Finland erkki.oja@hut.fi Mark Plumbley Department

More information

ARTEFACT DETECTION IN ASTROPHYSICAL IMAGE DATA USING INDEPENDENT COMPONENT ANALYSIS. Maria Funaro, Erkki Oja, and Harri Valpola

ARTEFACT DETECTION IN ASTROPHYSICAL IMAGE DATA USING INDEPENDENT COMPONENT ANALYSIS. Maria Funaro, Erkki Oja, and Harri Valpola ARTEFACT DETECTION IN ASTROPHYSICAL IMAGE DATA USING INDEPENDENT COMPONENT ANALYSIS Maria Funaro, Erkki Oja, and Harri Valpola Neural Networks Research Centre, Helsinki University of Technology P.O.Box

More information

Introduction to Machine Learning

Introduction to Machine Learning 10-701 Introduction to Machine Learning PCA Slides based on 18-661 Fall 2018 PCA Raw data can be Complex, High-dimensional To understand a phenomenon we measure various related quantities If we knew what

More information

Viewpoint invariant face recognition using independent component analysis and attractor networks

Viewpoint invariant face recognition using independent component analysis and attractor networks Viewpoint invariant face recognition using independent component analysis and attractor networks Marian Stewart Bartlett University of California San Diego The Salk Institute La Jolla, CA 92037 marni@salk.edu

More information

Ch 4. Linear Models for Classification

Ch 4. Linear Models for Classification Ch 4. Linear Models for Classification Pattern Recognition and Machine Learning, C. M. Bishop, 2006. Department of Computer Science and Engineering Pohang University of Science and echnology 77 Cheongam-ro,

More information

By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

By choosing to view this document, you agree to all provisions of the copyright laws protecting it. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services. Internal

More information

New Machine Learning Methods for Neuroimaging

New Machine Learning Methods for Neuroimaging New Machine Learning Methods for Neuroimaging Gatsby Computational Neuroscience Unit University College London, UK Dept of Computer Science University of Helsinki, Finland Outline Resting-state networks

More information

CPSC 340: Machine Learning and Data Mining. More PCA Fall 2017

CPSC 340: Machine Learning and Data Mining. More PCA Fall 2017 CPSC 340: Machine Learning and Data Mining More PCA Fall 2017 Admin Assignment 4: Due Friday of next week. No class Monday due to holiday. There will be tutorials next week on MAP/PCA (except Monday).

More information

NONLINEAR INDEPENDENT FACTOR ANALYSIS BY HIERARCHICAL MODELS

NONLINEAR INDEPENDENT FACTOR ANALYSIS BY HIERARCHICAL MODELS NONLINEAR INDEPENDENT FACTOR ANALYSIS BY HIERARCHICAL MODELS Harri Valpola, Tomas Östman and Juha Karhunen Helsinki University of Technology, Neural Networks Research Centre P.O. Box 5400, FIN-02015 HUT,

More information