Shift-Variance and Nonstationarity of Generalized Sampling-Reconstruction Processes Runyi Yu Eastern Mediterranean University Gazimagusa, North Cyprus Web: faraday.ee.emu.edu.tr/yu Emails: runyi.yu@emu.edu.tr and yu@ieee.org 23 September 2015 Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 1 / 28
Overview 1 Background and Problems 2 µ-shift-invariance 3 Shift-Variance Analysis of Generalized Sampling Processes (GSPs) 4 Shift-Variance Analysis of Generalized Sampling and Reconstruction Processes (GSRPs) 5 Nonstationarity Analysis of Random Processes 6 Conclusions Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 2 / 28
Shift-Invariance of System H Definition System H : x y is shift-invariant if for any x and τ, it holds y( ) = H x( ) = y( τ) = H x( τ). Descriptions: convolution CT : y(t) = h(t) x(t) = h(t τ)x(τ)dτ DT : y[n] = h[n] x[n] = k h[n k]x[k] where h = H δ is the impulse response. Much desired in applications (e.g. detection, classification, pattern recognition) Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 3 / 28
Shift-variant Systems: Examples Common building blocks Modulation: y(t) = h(t) x(t) or y[n] = h[n] x[n]. Sampling (unless h is bandlimited) y[n] = h(nt ), x( ) = h(nt τ)x(τ)dτ In ideal sampling, h is the Dirac impulse, thus y[n] = x(nt ). M-downsampling: y[n] = x[mn]. { x[n/l], n = 0, ±L, ±2L,..., L-upsampling: y[n] = 0, otherwise Useful signal processing systems Short-time Fourier Transforms Discrete-time Wavelet Transforms Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 4 / 28
Shift-variance Comparison: an illustration Responses of the B-spline GSRPs to shifted pulses All are shift-variant; yet some (those of high orders) are nearly shift-invariant. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 5 / 28
The Problems 1 How to quantify shift-variance of a systems? 2 How to compare systems in terms of their shift-variance? 3 How to characterize system performance in terms of the shift-variance? How to study nonstationarity of random processes? [a closely related problem] Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 6 / 28
µ-shift-invariance 1 Definition System H : x y is µ-shift-invariant if for any x and k, it holds y[ ] = H x[ ] = y[ αk] = H x[ k]. Examples L-upsampler is µ-shift-invariant with α = L. M-downsampler is µ-shift-invariant with α = 1/M for signals bandlimited in [ π/m, π/m). (M, L)-rate converter is µ-shift-invariant with α = L/M for signals bandlimited in [ π/m, π/m). Reflector y = x is µ-shift-invariant with α = 1. 1 µɛρlκσς, a Greek word, means partial. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 7 / 28
Generalized Sampling Processing (GSP) Mathematical Description H : y[n] = h(nt ), x( ) = h(nt τ)x(τ)dτ H : ŷ(e jt ω ) = 1 T ĥ(ω 2kπ/T ) x(ω 2kπ/T ) k Z The Induced -Norm (L 2 l 2 ) H = sup{ Sx l 2 x 0 x L 2 H = 1 h = 1 sup T T ω [0,2π/T ){( ĥ(ω + 2kπ/T ) 2 ) 1/2 } k Z Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 8 / 28 }
µ-shift-invariance Analysis of GSPs Commutators DT fraction shift-operator D τ/t B : û(e jω ) e jωτ/t û(e jω ), ω B where B is an admissible defining band (see the right figures for examples). H is µ-shift-invariant K τ,b = 0 for all τ R. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 9 / 28
Mathematical Model of Commutators Commutators Description: ŷ(ω) = 2 T e jωτ/t e jkπτ sin(kπτ/t )ĥ((ω 2kπ)/T ) x((ω 2kπ)/T ) k Z Norm: K τ,b 2 T sup ω B/T ) { [ } sin(kπτ/t )ĥ(ω 2kπ/T ) 2] 1/2 k Z Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 10 / 28
Determination of Shift-Variance of GSPs Shift-variance level SVL(H) = 2 T inf B where Σ τ = diag{sin(τkπ)} k Z. { sup [0,T /2) { Στ ĥ } } Shift-variance index SVI(H) = inf B { sup τ (0,T /2] { Σ τ ĥ }} ĥ If T = 1, then SVL(H) inf ω 0 R { sup τ (0,1/2] { sup ω [ω 0,ω 0 +2π) { [ 2 sin(kπτ)ĥ(ω 2kπ) 2] 1/2} }} k Z Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 11 / 28
Shift-Variance of Short-Time Fourier Transforms Window* SVL(H) SVI(H) Gaussian 0.9658 0.5709 Blackman 1.0000 0.5899 Hanning 1.000 0.5000 Hamming 0.9200 0.4259 Rectangular 1.0578 0.2644 *The support is [ 1, 1]. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 12 / 28
Shift-Variance of Wavelet Transforms Wavelet SVL(H) SVI(H) Shannon 0 0 Meryer 0.5642 0.7072 Maxican hat 0.3086 0.0965 Hermitian hat 0.0881 0.0509 Complex Morlet 0.0109 0.0072 Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 13 / 28
Shift-variance of Wavelets Transform: an illustration SVI = 0.0965(Mexican hat), 0.7072(Meyer), 0.0072(Complex Morlet), 0.0509(Hermitian hat). Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 14 / 28
Generalized Sampling-Reconstruction Processes x(t) 1 t u [n] v [n] 1 ϕ 1( ) q [n] ( t ) T T T ϕ y(t) 2 T t=nt Descriptions Sampling: û(e jω ) = 1 T k Z ϕ 1(ω 2πk) x ( (ω 2πk)/T ) Filtering : v(e jω ) = q(e jω )û(e jω ) Reconstr: ŷ(ω) = T ϕ 2 (T ω) v(e jt ω ), (y(t) = 1 v[k] ϕ 2 (t/t k) ) T Thus k Z H : ŷ(ω) = k Z ĥ k ((ω 2πk)/T ) x((ω 2πk)/T ) where ĥk(ω) = ϕ 2 (T ω + 2πk) q(e jt ω ) ϕ 1 (T ω). Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 15 / 28
Shift-Variance of GSRPs: perspectives In Relation to the LSI subspace: Distance and Angle 1 d(h, B 0 ) = inf G B 0 H G 2 1 H, G θ(h, B 0 ) = sup cos 0 G B 0 H G HD τ 0 θ H D τ 0 H K HSV H0 τ0 B0 Via commutator: Shift-variance level SVL 2 (H) = E τ { HD τ D τ H 2 } For particular input: Average shift-variance ASV 2 (H, x) = 1 T T SV2( H, x( s) ) ds and SV 2 (H, x) = E τ { K τ x 2 2 } Implications Var τ { HD τ x 2 } SV 2 (H, x) and Var τ { HD τ x 2 } ASV 2 (H, x) Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 16 / 28
Shift-variance of GSRPs: Results Shift-variance kernel SVK H (ξ) = ϕ 1 (T ξ) q(e jt ξ ) 2 ϕ 2 (T ξ 2πn) 2 n 0 Formulas ( 1 SVL(H) = 2 R SVK H(ω) dω 2π ) 1/2 2 d(h, B 0 ) = SVL(H)/ 2 and θ(h, B 0 ) = sin 1 (SVI(H) ) ( 3 ASV(H, x) = 2 ) 1/2. R SVK H(ω) x(ω) 2 dω 2π Implications H is shift-invariant. ϕ 1 (ξ) ϕ 2 (ξ + 2πn) = 0 for all n 0. ϕ 1 and ϕ 2 have the identical admissible band. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 17 / 28
Shift-variance and Approximation Error x (t) 1 t ϕ 1( ) T T u[n] v[n] 1 q [n] ( t ) 2 t=nt T ϕ y(t) T The average of approximation error e 2 (x) = 1 H e [x( s)] 2 2 ds T T e 2 (x) = (I H 0 )x 2 2 + 1 2 ASV2 (H, x) where H 0 is the nearest LSI system (whose frequency response is ϕ 1 (ω) q(ejω ) ϕ 2 (ω)). The variance of approximation error Var τ { H e D τ x } ASV 2 (H, x). Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 18 / 28
Examples: B-spline GSRPs (ϕ 1(t) = β n 1 (t) and ϕ 2(t) = β n 2 (t)) n 2 n 1 GSRP d(h, B 0) θ(h, B 0) SVI(H) ASV(H, x) e(x) σ e 0 0 ORTH 0.5774 35.27 0.5774 0.4159 0.3542 0.0548 1 1 ORTH 0.3547 20.78 0.3547 0.2106 0.1926 0.0699 0 REG 0.4379 28.24 0.4731 0.1657 0.2269 0.0577 0 CON 0.5590 31.03 0.5155 0.2144 0.1939 0.0703 2 2 ORTH 0.2864 16.64 0.2864 0.1554 0.1619 0.0674 0 REG 0.4348 27.87 0.4675 0.1284 0.2093 0.0527 0 CON 0.5387 29.95 0.4992 0.1604 0.1666 0.0694 3 3 ORTH 0.2485 14.39 0.2485 0.1292 0.1510 0.0548 1 REG 0.3079 18.11 0.3109 0.1314 0.1546 0.0561 1 CON 0.3192 18.42 0.3160 0.1360 0.1530 0.0555 4 4 ORTH 0.2227 12.87 0.2227 0.1131 0.1458 0.0452 2 REG 0.2580 15.00 0.2589 0.1175 0.1471 0.0459 2 CON 0.2619 15.13 0.2610 0.1192 0.1468 0.0455 T = 1 and x(t) = 1/(2πa) 1/4 e t2 /(4a) with a = 2 ln(2)/π 2. SVL(H REG) SVL(H CON), SVL(H REG, x) SVL(H CON, x) Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 19 / 28
Examples: B-spline GSRPs under shifted Gaussian inputs Outputs of GSRPs (n 1 = 0, n 2 = 1) The Minimax Regret GSRP The Consistent GSRP (a) (b) Variations of approximation error (Orthogonal, n 1 = n 2 = n) 0.4 Estimated Approximation Error Approximation Error 0.2 (a) 0 0 25 50 Order n Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 20 / 28
Nonstationarity Analysis of Random Processes Let y : R C be a zero-mean random process. Denote its autocorrelation function as r y (t, s) = E y { y(t) 2 } <. Recall that if r y (t, s) is independent of time t, then y is wide-sense stationary (WSS). Nonstationarity of y It can be characterized by shift-variance of autocorrelation Operator R y : NSt(y) = SVI(R y ) Autocorrelation Operator R y is a deterministic linear system specified by the impulse response r y (t, s). Note that y is WSS (NSt(y) = 0) if and only if R y is LSI (SVI(R y ) = 0). Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 21 / 28
Determination of NSt(y) Fourier series representation For T -WSCS random processes, r y (t, s) = r y (t + T, s), then r y (t, s) = r y,k (s) e j2kπt/t k Z Nonstationarity of y NSt(y) = ( 1 R r y,0(s) 2 ds R r y,n(s) 2 ds n Z The specific formulas for the GSRP can then be derived. ) 1/2 Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 22 / 28
where J and J 0 are the MSEs of optimal linear filtering and optimal LSI filtering respectively, and σ is the signal-noise ratio. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 23 / 28 Applications of Nonstationarity Detection of Weak WSCS random processes Let z(t) = w(t) or y(t) + w(t). Then max{d 2 (t)} max{d 2 0 (t)} max{d 2 (t)} where deflection d 2 (t) = is the performance for the phase randomization. = Nst 2 (y) Ez {z(t) y(t) present} E z {z(t) y(t) absent} 2 Var z {z(t) y(t) absent} and d 0 (t) Denoising of PAM signals Let y(t) = n Z v n g(t nt ), z(t) = y(t) + w(t). Then σ σ + 1 NSt2 (y) J 0 J J σ NSt 2 (y).
Conclusions We conducted a systematic analysis on shift-variance and presented answers to: For generalized sampling processes How to deal with the time-frame mismatch? [µ-shift-variance] How to exploit the flexibility in defining discrete-time fractional delays? [admissible defining band] how to determine the maximal (relative) error between responses to shift-inputs and the shifted responses? [SVL and SVI via commutators] For generalized sampling-reconstruction processes How far is a GSRP away from the subspace of shift-invariant system? [distance and angle] How much is the possible maximal error between responses to shift-inputs and the shifted responses? [SVL and SVI via commutators] For a particular input, how much is the possible average error between responses to shift-inputs and the shifted responses? [ASV, Var τ ] How is the reconstruction error related to the average shift-variance? Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 24 / 28
Applications of Shift-Variance Measures Interpolation Given a sampling process, find the optimal amount of shift in the reconstruction kernel so that the average error is minimized; thus improving interpolation result. Superresolution Given a sampling process, find the optimal admission band for the fractional delay filter so that the commutators (error systems) yields the minimal error, thus producing good high resolution signals. To images/videos Both interpolation and superposition can be applied to n-dimensional signals/systems. Design of nearly shift-invariant transforms/systems Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 25 / 28
Conclusions ctd We also presented a systematic analysis on nonstationarity in terms of shift-variance of the autocorrelation operator. NSt(y) = SVI(R y ) We showed that Nst can be used to characterize performance loss in: Detection of Weak signals Denoising of PAM signals Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 26 / 28
References T. Aach (2007), Comparative analysis of shift variance and cyclostationarity in multirate filter banks, IEEE Transactions on Circuits & Systems I: Regular Papers. T. Aach and H. Führ (2009), On bounds of shift variance in two-channel mutirate filter banks, IEEE Transactions on Signal Processing. T. Aach and H. Führ (2012), Shift variance measures for multirate LPSV filter banks with random input signals, IEEE Transactions on Signal Processing. R. Yu (2009), A new shift-invariance of discrete-time systems and its application to discrete wavelet transform analysis, IEEE Transactions on Signal Processing. R. Yu (2011), Shift-variance measure of multichannel multirate systems, IEEE Transactions on Signal Processing. R. Yu (2012), Shift-variance analysis of generalized sampling processes, IEEE Transactions on Signal Processing. B. Sadeghi and R. Yu (2015), Shift-Variance and nonstationarity of linear periodically shift-variant systems and applications to generalized sampling-reconstruction processes, IEEE Transactions on Signal Processing, submitted. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 27 / 28
Thank you. Runyi Yu (EMU) SV and NSt of GSRPs 23 September 2015 28 / 28