Filtering via Rank-Reduced Hankel Matrix CEE 690, ME 555 System Identification Fall, 2013 c H.P. Gavin, September 24, 2013

Transcription

1 Filtering via Rank-Reduced Hankel Matrix CEE 690, ME 555 System Identification Fall, 2013 c H.P. Gavin, September 24, 2013 This method goes by the terms Structured Total Least Squares and Singular Spectrum Analysis and finds application to a very wide range of problems [4]. For noise-filtering applications, a discrete-time signal, y i, i = 1, N, is broken up into n time-shifted segments and the segments are arranged as columns in a Hankel matrix, Y R m n, m > n, Y ij = y i+j 1. Y = y 1 y 2 y n y 2 y 3 y n+1... y m y m+1 y m+n 1 This is a Hankel matrix because values along the anti-diagonals are all equal to one another. The SVD of Y is Y = UΣV T, and a reduced-rank version of Y can be reconstructed from the first r dyads of the SVD. Y r = U r Σ r V T r where U r R m r, Σ r R r r, and V r R r n. are the first r singular vectors of U and V and the largest r singular values. This reduced-rank matrix Y r is the rank-r matrix that is closest to Y in the sense of minimizing the Frobeneous norm of their difference, Y Y r 2 F. In general Y r will not have the same Hankel structure as Y, but a matrix with a Hankel structure, Ȳ r, can be obtained from Y in a number of ways. Singular Spectrum Analysis [2]. In SSA, Y r is computed as above, and elements along each anti-diagonal are replaced by the average of the anti-diagonal. The resulting matrix Ȳ r will not have rank-r and will not be the closest matrix to Y in the sense of Frobeneous, but it will have a Hankel structure. Cadzow s algorithm [1]. In Cadzow s algorithm, the SSA averaging is repeatedly applied. After each anti-diagonal averaging step, the matrix grows in rank, so a new SVD can be computed, a new rank-r matrix can be constructed, and the anti-diagonals of the new reduced rank matrix can be averaged. Structured low-rank approximation [3]. These methods solve the constrained optimization problem: min Y Ȳ r 2 F such that rank(ȳ r ) = r and Ȳ r has a desired structure. These methods are iterative, but apply more rigorous methods to determining Ȳ r. The low-rank (filtered signal) can be recovered from the first row and last column of Ȳ. The examples below apply SSA to recoving signals from noisy data. If a signal can be represented by a few components of the SVD of Y, it will be clear from the plot of the singular values of

2 2 CEE 690, ME 555 System Identification Duke University Fall 2013 H.P. Gavin Y. This is the case in the first example (a good application of SSA in which the signal-to-noise ratio can even be less than 1), but not in the second. SSA is a type of Principal Component Analysis (PCA). 1 Recover a reduced basis from a very noisy measurement (PCA) signal is sum of sines: y i = j sin(2πf j t i ) noisy measurements: ỹ i = y i + dn y n y is a unit white noise process d = σ y /(SNR t) SNR : signal-to-noise ratio σ i / σ i noisy 10-4 PSD true filtered frequency, Hz signals time, s

3 SVD filtering 3 2 Filter a noisy measurement of a broad-band response noise-driven linear dynamics: ẋ = ax + bn x n x is a unit white noise process true output: y = cx noisy measurements: ỹ = y + dn y n y is a unit white noise process parameter values: a = 0.5; b = 1; c = 1; d = σ y /(SNR t) SNR : signal-to-noise ratio σ i / σ i noisy 10-4 PSD true filtered frequency, Hz signals time, s

4 4 CEE 690, ME 555 System Identification Duke University Fall 2013 H.P. Gavin SVD filter.m 1 function y = SVD_filter (y, m, sv_ratio ) 2 % y = S V D f i l t e r (u, m, s v r a t i o ) 3 % Use t h e SVD to f i l t e r out l e a s t s i g n i f i c a n t ( noisy ) p o r t i o n s o f a s i g n a l 4 % 5 % INPUTS DESCRIPTION 6 % ========== =============== 7 % y s i g n a l to be f i l t e r e d, 1 x N 8 % m rows in t h e Hankel matrix o f y, Y m > N/2+1 9 % s v r a t i o remove components o f SVD o f Y with s i < s v r a t i o s 1 10 % 11 % OUTPUT DESCRIPTION 12 % ========== =============== 13 % y re c o n s t r u c t i o n o f y from low rank approximation o f Y [l, N] = size ( y); % put s i g n a l i n t o a Hankel Matrix. o f dimension m x n ; m > n ; m+n = N+1 18 i f ( m < N /2) error ( SVD_filter : m should be greater than N/2 ); end 19 n = N+1 - m; % number o f columns o f t h e Hankel matrix Y = zeros (m, n); 22 for k =1: m 23 Y ( k, : ) = y ( k :k+n -1 ); 24 end [U,S, V] = svd ( Y, economy ); % make economical SVD o f t h e Hankel Matrix K = max( find (diag(s)/s(1,1) > sv_ratio )) % f i n d t h e most s i g n i f i c a n t p a r t figure (3) 31 loglog (diag(s)/s(1,1), o, [1 K],[1 1]* sv_ratio, k, [K K],[ sv_ratio 1], k ) 32 ylabel ( \ sigma_i / \ sigma_1 ) 33 xlabel ( i ) 34 print ( SVD_filter_svd. eps, -color, -solid, -F :28 ); % b u i l d a new rank K matrix from t h e f i r s t K dyads o f t h e SVD 38 Y = U (:,1: K)* diag(diag(s )(1: K ))* V (:,1: K) ; % Average anti d i a g o n a l components to make t h e lower rank matrix a Hankel matrix 41 % E x t r a c t t h e f i l t e r e d s i g n a l from t h e f i r s t column and l a s t row o f t h e 42 % lower rank Hankel matrix y = zeros (1,N); 45 y (1) = Y (1,1); for k =2: m % f i r s t column o f Hankel matrix 48 min_kn = min(k, n); 49 y(k) = sum(diag(y(k : -1:1,1: min_kn ))) / min_kn ; 50 end 51 for k =2: n % l a s t row o f Hankel matrix 52 y(m+k -1) = sum(diag(y(m: -1:m-n+k,k:n ))) / (n-k +1); 53 end % S V D f i l t e r HP Gavin % System I d e n t i f i c a t i o n, Duke U n i v e r s i t y, F a l l 2013

5 SVD filtering 5 1 % S V D f i l t e r t e s t 2 % t e s t t h e use o f SVD o f s i g n a l Hankel matrix f o r f i l t e r i n g 3 4 % use SVD to remove n o i s e 5 % m = number o f rows in Hankel matrix ; m >= N/2+1 ; 6 % s m a l l e r m : : s l o w e r SVD : : l e s s e x t r a c t i o n 7 % s m a l l e r m : : sharper SVD knee : : l e s s n o i s e in p r i n c i p a l components % HP Gavin, CEE 699, System I d e n t i f i c a t i o n, F a l l epsplots = 1; formatplot ( epsplots ); Example = 1; randn( seed,2); % i n i t i a l i z e random number g e n e r a t o r 16 N = 2048; % number o f data p o i n t s 17 dt = 0.05; % time s t e p increment 18 t = [1: N]* dt; % time v a l u e s i f ( Example == 1) % sum o f harmonic s i g n a l s freq = [ ( sqrt (5) -1)/2 1 2/( sqrt (5) -1) e pi ] ; % s e t o f f r e q u e n c i e s 23 yt = sum( sin (2* pi * freq *t)) / length( freq ); % t r u e s i g n a l SNR = 0.5; % works with very poor s i g n a l to n o i s e r a t i o 26 m = c e i l (0.6* N + 1 ) 27 sv_ratio = 0.60; % s i n g u l a r v a l u e r a t i o c l o s e r to 1 : : more f i l t e r i n g end i f ( Example == 2) % dynamical system d r i v e n by u n i t white n o i s e yt = lsim ( -0.5,1,1,0,randn(1,N)/ sqrt (dt),t,0); % t r u e s i g n a l SNR = 2.0; % needs b e t t e r s i g n a l to n o i s e r a t i o 36 m = c e i l (0.9* N + 1 ) 37 sv_ratio = 0.15; % s m a l l e r s i n g u l a r v a l u e r a t i o : : l e s s f i l t e r i n g end % add measurement n o i s e 42 yn = yt + randn(1,n)/ sqrt (dt) * sqrt (yt*yt /N) / ( SNR * sqrt (dt )); yf = SVD_filter ( yn, m, sv_ratio ); % remove random components yf_yt_err = norm(yf -yt )/norm(yt) % compare f i l t e r e d to t r u e 51 yf_yn_err = norm(yf -yn )/norm(yt) % compare f i l t e r e d to noisy nfft = 512; 54 [ PSDyt,f] = psd (yt,1/ dt, nfft ); 55 [ PSDyn,f] = psd (yn,1/ dt, nfft ); 56 [ PSDyf,f] = psd (yf,1/ dt, nfft ); % % P l o t t i n g figure (1); 63 c l f 64 plot (t,yt, t,yn, t,yf) 65 axis ([10 20]) 66 ylabel ( signals ) 67 xlabel ( time, s )

6 6 CEE 690, ME 555 System Identification Duke University Fall 2013 H.P. Gavin 68 i f epsplots, print ( sprintf ( SVD_filter_ % d_1. eps, Example ), -color, -solid, -F :28 ); end figure (2) 71 c l f 72 idx = [4: nfft /2]; 73 loglog (f( idx ), PSDyt ( idx ), f( idx ), PSDyn ( idx ), f( idx ), PSDyf ( idx )) 74 xlabel ( frequency, Hz ) 75 ylabel ( PSD ) 76 text (f( nfft /2), PSDyt ( nfft /2), true ) 77 text (f( nfft /2), PSDyn ( nfft /2), noisy ) 78 text (f( nfft /2), PSDyf ( nfft /2), filtered ) 79 i f epsplots, print ( sprintf ( SVD_filter_ % d_2. eps, Example ), -color, -solid, -F :28 ); end References [1] Cadzow, J., Signal Enhancement: a composite property mapping algorithm, IEEE Trans. Acoustics, Speech, and Signal Processing, 36(2):49-82 (1988). [2] Golyandina et. al., Analysis of Time Series Structure: SSA and related techniques, Champman-Hall, CRC 2001 [3] Lemmerling, Philippe, Structured Total Least Squares: Analysis, Algorithms, and Applications Ph.D. Dissertation, Leuven, [4] Markovsky, Ivan Structured low-rank approximation and its application, Automatica 44: (2008).