Independent Component Analysis and Its Application on Accelerator Physics Xiaoying Pang LA-UR-12-20069
ICA and PCA Similarities: Blind source separation method (BSS) no model Observed signals are linear combinations of the source signals x i = a i1 s 1 + a i2 s 2 + + a in s n, i = 1,, m Raw data matrix: BPM 1 BPM 2 BPM M turn 1 turn2 turn N Same goal: find the demixing matrix W such that s= Wx Trying to find the physical source for each individual source mode is always the difficult part
ICA and PCA Differences: Source signals are : Uncorrelated (PCA) Cov(x,y) = 0 Independent (ICA) Cov(f(x), f(y)) = 0 ICA is more robust to coupling, noise and nonlinearity
Principal Component Analysis (PCA) Identifying patterns in data, compressing high-dimensional data by reducing redundant dimensions without much loss of information. Find the underlying structures and make them the principal components (PCs) X : raw data matrix, S: source data matrix, s = WW uncorrelated ss T = Σ WWx T W T = Σ xx T = W T ΣΣ XX T and X T X are symmetric positive definite diagonalizable with an orthogonal matrix Information is contained in variance maximize Correlations are contained in covariance minimize Modes are ranked by variances Singular Value Decomposition (SVD)
Singular Value Decomposition SVD is looking for an orthonormal base in the row space of a matrix X that goes over into an orthonormal base in the column space of X. X mxn R n V1 Row Space(X) V2 X*V1 = σ 1 U1 X*V2 = σ 2 U2 R m U1 Column space (X) U2 N(X) N(X T ) X v 1 v 2 v r = [u 1 u 2 u r ] σ 1 0 0 0 σ 2 0 0 0
Singular Value Decomposition m n matrix X, it can be decomposed into X = U m m D m n V T n n Columns of U m m are eigenvector of (XX T ) m m Columns of V n n are eigenvector of (X T X) n n Eigenvalues of (XX T ) m m and (X T X) n n are the same, they are the diagonal elements of the diagonal matrix D m n i.e. If m<n, D m n = d 11 0 0 0 0 d 22 0.0 0.. 0 0.0 d mm 0 0 n m 6
Independent Component Analysis using time structure Autocovariance: covariance btw signals at two different time points, i.e. cov(x(t), y(t+τ)) Independence of source signals o cov(s i (t), s j (t+ τ)) = 0 o Autocovariance matrix of the source signals should be diagonal In order to find the unique set of fundamental modes, the autocovariance matrix of source signals should be non-degenerate unique autocovariances (eigenvalues) x1(1) x2(1) x = : xm (1).... :.. x (1 + τ ) x x 1 2 M (1 + τ ) : (1 + τ ).... :.. x ( N) x x 1 2 M ( N) : ( N)...... x ( N + τ ).. x x 1 2 M ( N + τ ).. : ( N : + τ ).. x1( L) x2( L) : x ( L) M X(0) X(τ)
ICA preprocessing Before ICA, we first preprocess the raw data using PCA to get rid of some of the noises. o o Compute the m by m equal time covariance matrix: Perform SVD to obtain where o Construct the whitened matrix z by z = YY, Y = Λ 1/2 1 U T 1, zz T = I After ICA finds the demixing W, the mixing matrix A and source s are: s = WW, A = Y 1 W T
AMUSE (Algorithm for Multiple Unknown Signals Extraction) Consider the whitened data z(t), with the separating matrix W, the source signals s(t) can be found as: s t = WW t, s t + τ = WW t + τ time-lagged autocovariance matrix for z is: C z τ = z t z(t + τ) T C z = 1 2 C z τ + C z T τ = 1 2 WT s t s(t + τ) T + s t + τ s(t) T W = W T C sw Cons: Since W has to be uniquely defined. This requires that C s has to be non-degenerate. But this may not be possible! We need to search for a particular τ that can satisfy this condition. Better way: Instead of using a single time lag, we can use multiple time lags at a time and generate multiple covariance matrices. Then find a way to joint diagonalize these covariance matrices simultaneously. So searching for one particular choice of time lag would be unnecessary.
ICA using multiple time lags Since the covariance matrix of the source signals is diagonal. If VC z V T is diagonal then W = V, and VC z V T = C s Using multiple time lags, our goal is to find the single demixing matrix W or V that will simultaneously joint diagonalize all the autocovariance matrices. Minimize the joint diagonality: f V = K τ ooo(vc l l=1 z V T ) Define V as a 2x2 rotational matrix, diagonalize all the 2x2 submatrices τ of C i z along its diagonal line by adjusting the rotational angles. cooθ sssθ c s V = = sssθ cccθ s c A 2x2 submatrix of C z τ i along its diagonal line can be τ (C i z ) ii = a ii a ii a jj a jj
ICA using multiple time lags Unitary transformation preserves the sum of squares of elements and also the trace of a matrix τ V(C i z ) ii V T = a ii a ij c s a ii a ii c s a ji a = jj s c a jj a jj s c So, a ii 2 + a jj 2 + a ij 2 + a jj 2 = a ii 2 + a jj 2 + a ij 2 + a jj 2 and maximize minimize a ii + a jj = a ii + a jj Since a ii 2 + a jj 2 = 1 2 [ a ii + a jj 2 + a ii a jj 2 ] maximize maximize For all the autocovariance matrices with different time lags, maximize Q, where Q ii = (a ii τ l a jj τ l ) 2 K l=1
ICA using multiple time lags Since a ii a jj = a ii a jj c 2 s 2 + 222 a ii + a ji = ccccθ a ii a jj + ssssθ a ij + a ji Q ii = q T q, q = a ii τ 1 a jj τ 1 a ii τ 2 a jj τ 2 a ii τ K a jj τ K = τ a 1 τ ii a 1 jj τ a 2 τ ii a 2 jj a ii τ K a jj τ K τ a 1 τ ij + a 1 ji τ a 2 τ ii + a 2 jj a ii τ K + a jj τ K coooθ ssssθ = gt u To maximize Q ii = u T gg T u, u = coooθ need to be the eigenvector of ssssθ G = gg T corresponding to the largest eigenvalue. If G = G 11 G 11, then G 22 ttt 2θ = G 22 +G 11 G 11 G 22 + (G 11 G 22 ) 2 +(G 12 +G 21 ) 2 G 22
Application to Simple Sinusoidal Motion Raw data matrix element for simple sinusoidal motion x = AAAA υ x φ i x ii = AAAA 2πυ x M + 22υ x j 1, i = 1, 2,, M, j = 1, 2,, N PCA result :
Application to Betatron Motion Now PCA result:
Application to Betatron Motion
Application to Betatron Motion
Application to Nonlinear Motion Put sextupoles into the AGS lattice and look at the 2υ x mode. Hill s eqn: For a short sextupole, Floquet transformation: Solution:
Application to Nonlinear Motion Closed Orbit= x-x β -x 2ν Simple betatron oscillation!
Application to Nonlinear Motion PCA cannot recover The nonlinear motion!
References A. Belouchrani et al. A Blind Source Separation Technique Using Second- Order Statistics, IEEE Transactions on signal processing, vol. 45, no. 2, 1997 A. Hyvarinen, J. Karhunen, E. Oja, Independent Component Analysis, John Wiley & Sons, Inc. 2001 X. Pang, S. Y. Lee Independent Component Analysis fro beam measurement, J. of App. Phys., 106, 074902, 2009