Filtering via Rank-Reduced Hankel Matrix CEE 690, ME 555 System Identification Fall, 2013 c H.P. Gavin, September 24, 2013 This method goes by the terms Structured Total Least Squares and Singular Spectrum Analysis and finds application to a very wide range of problems [4]. For noise-filtering applications, a discrete-time signal, y i, i = 1, N, is broken up into n time-shifted segments and the segments are arranged as columns in a Hankel matrix, Y R m n, m > n, Y ij = y i+j 1. Y = y 1 y 2 y n y 2 y 3 y n+1... y m y m+1 y m+n 1 This is a Hankel matrix because values along the anti-diagonals are all equal to one another. The SVD of Y is Y = UΣV T, and a reduced-rank version of Y can be reconstructed from the first r dyads of the SVD. Y r = U r Σ r V T r where U r R m r, Σ r R r r, and V r R r n. are the first r singular vectors of U and V and the largest r singular values. This reduced-rank matrix Y r is the rank-r matrix that is closest to Y in the sense of minimizing the Frobeneous norm of their difference, Y Y r 2 F. In general Y r will not have the same Hankel structure as Y, but a matrix with a Hankel structure, Ȳ r, can be obtained from Y in a number of ways. Singular Spectrum Analysis [2]. In SSA, Y r is computed as above, and elements along each anti-diagonal are replaced by the average of the anti-diagonal. The resulting matrix Ȳ r will not have rank-r and will not be the closest matrix to Y in the sense of Frobeneous, but it will have a Hankel structure. Cadzow s algorithm [1]. In Cadzow s algorithm, the SSA averaging is repeatedly applied. After each anti-diagonal averaging step, the matrix grows in rank, so a new SVD can be computed, a new rank-r matrix can be constructed, and the anti-diagonals of the new reduced rank matrix can be averaged. Structured low-rank approximation [3]. These methods solve the constrained optimization problem: min Y Ȳ r 2 F such that rank(ȳ r ) = r and Ȳ r has a desired structure. These methods are iterative, but apply more rigorous methods to determining Ȳ r. The low-rank (filtered signal) can be recovered from the first row and last column of Ȳ. The examples below apply SSA to recoving signals from noisy data. If a signal can be represented by a few components of the SVD of Y, it will be clear from the plot of the singular values of
2 CEE 690, ME 555 System Identification Duke University Fall 2013 H.P. Gavin Y. This is the case in the first example (a good application of SSA in which the signal-to-noise ratio can even be less than 1), but not in the second. SSA is a type of Principal Component Analysis (PCA). 1 Recover a reduced basis from a very noisy measurement (PCA) signal is sum of sines: y i = j sin(2πf j t i ) noisy measurements: ỹ i = y i + dn y n y is a unit white noise process d = σ y /(SNR t) SNR : signal-to-noise ratio σ i / σ 1 10-1 10 1 10 2 10 3 i noisy 10-4 PSD 10-6 10-8 10-10 true filtered 10-12 10-1 10 1 frequency, Hz 3 2 1 signals 0-1 -2-3 10 12 14 16 18 20 time, s
SVD filtering 3 2 Filter a noisy measurement of a broad-band response noise-driven linear dynamics: ẋ = ax + bn x n x is a unit white noise process true output: y = cx noisy measurements: ỹ = y + dn y n y is a unit white noise process parameter values: a = 0.5; b = 1; c = 1; d = σ y /(SNR t) SNR : signal-to-noise ratio σ i / σ 1 10-1 10 1 10 2 10 3 i 10-1 10-3 noisy 10-4 PSD 10-5 10-6 10-7 10-8 10-9 true filtered 10-10 10-1 10 1 frequency, Hz 4 3 2 signals 1 0-1 -2-3 10 12 14 16 18 20 time, s
4 CEE 690, ME 555 System Identification Duke University Fall 2013 H.P. Gavin SVD filter.m 1 function y = SVD_filter (y, m, sv_ratio ) 2 % y = S V D f i l t e r (u, m, s v r a t i o ) 3 % Use t h e SVD to f i l t e r out l e a s t s i g n i f i c a n t ( noisy ) p o r t i o n s o f a s i g n a l 4 % 5 % INPUTS DESCRIPTION 6 % ========== =============== 7 % y s i g n a l to be f i l t e r e d, 1 x N 8 % m rows in t h e Hankel matrix o f y, Y m > N/2+1 9 % s v r a t i o remove components o f SVD o f Y with s i < s v r a t i o s 1 10 % 11 % OUTPUT DESCRIPTION 12 % ========== =============== 13 % y re c o n s t r u c t i o n o f y from low rank approximation o f Y 14 15 [l, N] = size ( y); 16 17 % put s i g n a l i n t o a Hankel Matrix. o f dimension m x n ; m > n ; m+n = N+1 18 i f ( m < N /2) error ( SVD_filter : m should be greater than N/2 ); end 19 n = N+1 - m; % number o f columns o f t h e Hankel matrix 20 21 Y = zeros (m, n); 22 for k =1: m 23 Y ( k, : ) = y ( k :k+n -1 ); 24 end 25 26 [U,S, V] = svd ( Y, economy ); % make economical SVD o f t h e Hankel Matrix. 27 28 K = max( find (diag(s)/s(1,1) > sv_ratio )) % f i n d t h e most s i g n i f i c a n t p a r t 29 30 figure (3) 31 loglog (diag(s)/s(1,1), o, [1 K],[1 1]* sv_ratio, k, [K K],[ sv_ratio 1], k ) 32 ylabel ( \ sigma_i / \ sigma_1 ) 33 xlabel ( i ) 34 print ( SVD_filter_svd. eps, -color, -solid, -F :28 ); 35 36 37 % b u i l d a new rank K matrix from t h e f i r s t K dyads o f t h e SVD 38 Y = U (:,1: K)* diag(diag(s )(1: K ))* V (:,1: K) ; 39 40 % Average anti d i a g o n a l components to make t h e lower rank matrix a Hankel matrix 41 % E x t r a c t t h e f i l t e r e d s i g n a l from t h e f i r s t column and l a s t row o f t h e 42 % lower rank Hankel matrix. 43 44 y = zeros (1,N); 45 y (1) = Y (1,1); 46 47 for k =2: m % f i r s t column o f Hankel matrix 48 min_kn = min(k, n); 49 y(k) = sum(diag(y(k : -1:1,1: min_kn ))) / min_kn ; 50 end 51 for k =2: n % l a s t row o f Hankel matrix 52 y(m+k -1) = sum(diag(y(m: -1:m-n+k,k:n ))) / (n-k +1); 53 end 54 55 % S V D f i l t e r HP Gavin 2013 09 10 56 % System I d e n t i f i c a t i o n, Duke U n i v e r s i t y, F a l l 2013
SVD filtering 5 1 % S V D f i l t e r t e s t 2 % t e s t t h e use o f SVD o f s i g n a l Hankel matrix f o r f i l t e r i n g 3 4 % use SVD to remove n o i s e 5 % m = number o f rows in Hankel matrix ; m >= N/2+1 ; 6 % s m a l l e r m : : s l o w e r SVD : : l e s s e x t r a c t i o n 7 % s m a l l e r m : : sharper SVD knee : : l e s s n o i s e in p r i n c i p a l components 8 10 9 % HP Gavin, CEE 699, System I d e n t i f i c a t i o n, F a l l 2013 11 epsplots = 1; formatplot ( epsplots ); 12 13 Example = 1; 14 15 randn( seed,2); % i n i t i a l i z e random number g e n e r a t o r 16 N = 2048; % number o f data p o i n t s 17 dt = 0.05; % time s t e p increment 18 t = [1: N]* dt; % time v a l u e s 19 20 i f ( Example == 1) % sum o f harmonic s i g n a l s 21 22 freq = [ ( sqrt (5) -1)/2 1 2/( sqrt (5) -1) e pi ] ; % s e t o f f r e q u e n c i e s 23 yt = sum( sin (2* pi * freq *t)) / length( freq ); % t r u e s i g n a l 24 25 SNR = 0.5; % works with very poor s i g n a l to n o i s e r a t i o 26 m = c e i l (0.6* N + 1 ) 27 sv_ratio = 0.60; % s i n g u l a r v a l u e r a t i o c l o s e r to 1 : : more f i l t e r i n g 28 29 end 30 31 i f ( Example == 2) % dynamical system d r i v e n by u n i t white n o i s e 32 33 yt = lsim ( -0.5,1,1,0,randn(1,N)/ sqrt (dt),t,0); % t r u e s i g n a l 34 35 SNR = 2.0; % needs b e t t e r s i g n a l to n o i s e r a t i o 36 m = c e i l (0.9* N + 1 ) 37 sv_ratio = 0.15; % s m a l l e r s i n g u l a r v a l u e r a t i o : : l e s s f i l t e r i n g 38 39 end 40 41 % add measurement n o i s e 42 yn = yt + randn(1,n)/ sqrt (dt) * sqrt (yt*yt /N) / ( SNR * sqrt (dt )); 43 44 45 46 yf = SVD_filter ( yn, m, sv_ratio ); % remove random components 47 48 49 50 yf_yt_err = norm(yf -yt )/norm(yt) % compare f i l t e r e d to t r u e 51 yf_yn_err = norm(yf -yn )/norm(yt) % compare f i l t e r e d to noisy 52 53 nfft = 512; 54 [ PSDyt,f] = psd (yt,1/ dt, nfft ); 55 [ PSDyn,f] = psd (yn,1/ dt, nfft ); 56 [ PSDyf,f] = psd (yf,1/ dt, nfft ); 57 58 % 59 60 % P l o t t i n g 61 62 figure (1); 63 c l f 64 plot (t,yt, t,yn, t,yf) 65 axis ([10 20]) 66 ylabel ( signals ) 67 xlabel ( time, s )
6 CEE 690, ME 555 System Identification Duke University Fall 2013 H.P. Gavin 68 i f epsplots, print ( sprintf ( SVD_filter_ % d_1. eps, Example ), -color, -solid, -F :28 ); end 69 70 figure (2) 71 c l f 72 idx = [4: nfft /2]; 73 loglog (f( idx ), PSDyt ( idx ), f( idx ), PSDyn ( idx ), f( idx ), PSDyf ( idx )) 74 xlabel ( frequency, Hz ) 75 ylabel ( PSD ) 76 text (f( nfft /2), PSDyt ( nfft /2), true ) 77 text (f( nfft /2), PSDyn ( nfft /2), noisy ) 78 text (f( nfft /2), PSDyf ( nfft /2), filtered ) 79 i f epsplots, print ( sprintf ( SVD_filter_ % d_2. eps, Example ), -color, -solid, -F :28 ); end References [1] Cadzow, J., Signal Enhancement: a composite property mapping algorithm, IEEE Trans. Acoustics, Speech, and Signal Processing, 36(2):49-82 (1988). [2] Golyandina et. al., Analysis of Time Series Structure: SSA and related techniques, Champman-Hall, CRC 2001 [3] Lemmerling, Philippe, Structured Total Least Squares: Analysis, Algorithms, and Applications Ph.D. Dissertation, Leuven, 1999. [4] Markovsky, Ivan Structured low-rank approximation and its application, Automatica 44: 891-909 (2008).