µ-shift-invariance: Theory and Applications

Transcription

1 µ-shift-invariance: Theory and Applications Runyi Yu Department of Electrical and Electronic Engineering Eastern Mediterranean University Famagusta, North Cyprus Homepage: faraday.ee.emu.edu.tr/yu The 8th International Conference on Signal Processing Systems AUT University, Auckland, New Zealand 22 November 216

2 Overview of the talk 1 Motivation and Background 2 Research Problems 3 µ-shift-invariance: concept, characterization, and examples 4 Shift-Variance Analysis: generalized sampling processes 5 Applications to DWT and STFT 6 Concluding Remarks R. Yu (EMU) µ-shift-invariance 2 / 25

3 Motivation and background Shift-invariance is crucial to the success of many signal processing systems. But... The issue of shift-variance problem provide an ficient repfor many Test Signal x(n) = δ (n 6) Test Signal x(n) = δ(n 64) gnals that 1 1 in practice.8.8 ell matched rier basis,.6.6 ally meant.4.4 signals. In.2.2 avelets proimal reprefor many d(,n), Real DWT, Energy = d(,n), Real DWT, Energy = aining sinjumps and archetypal.1.1 ng a pieceh function f low-order.1.1 s separated.2.2 ontinuities. representaally sparse d(,n), Complex WT, Energy = d(,n), Complex WT, Energy = als, requirr of magni coefficients rier basis to.2.2 within the The key to is that since.1.1 llate locally,.5.5 ts overlaplarity have let coeffither coeffiall. impulse [FIG2] The wavelet coefficients of a signal x(n) are very sensitive to translations of the signal. For two Wavelet signals x(n) = δ(n 6) transforms and x(n) = δ(n 64) (a), we plot the of wavelet impulses coefficients d(j,n) at a fixed scale j (b) and (c). (b) shows the real coefficients computed using the conventional real discrete sity of the wavelet transform (DWT, with Daubechies length-14 filters). (c) shows the magnitude of the complex fficients of coefficients computed using the dual-tree complex discrete wavelet transform (CWT with length-14 filters orld signals from [58]). For the dual-tree CWT the total energy at scale j is nearly constant, in contrast to the real DWT. (Selesnick-B-K-25) optimal sigg based on simple thresholding (keep the large coefill the small ones), the core of a host of powerful often through zero, we see that the conventional wisdom that sin- very challenging [22]. Moreover, since an oscillating function passes ession (JPEG2 [98]), denoising, approximation, gularities yield large wavelet coefficients is overstated. Indeed, as we istic and statistical signal and image algorithms. see in Figure 1, it is quite possible for a wavelet overlapping a singularity to have a small or even zero wavelet coefficient. PARADISE: FOUR PROBLEMS WAVELETS PROBLEM 2: SHIFT VARIANCE is not the end of the story. In spite of its efficient comorithm and sparse representation, the wavelet transcient oscillation pattern around singularities (see Figure 2). A small shift of the signal greatly perturbs the wavelet coeffiom four fundamental, intertwined shortcomings. Shift variance also complicates wavelet-domain processing; Responses of B-spline reconstructions to shifted pulses Then, search for nearly shift-invariant algorithms (DT-CWT, Gabor, SIFT...) R. Yu (EMU) µ-shift-invariance 3 / 25

4 Problems under consideration Theory Questions: What does it mean for a shift-variant system to be nearly shift-invariant? How to quantify shift-variance of a system? How to evaluate system performance via shift-variance measures? Answers: µ-shift-invariance and generalized commutators The shift-invariance level and shift-invariance index What performance? Applications Shift-variance analysis of important systems (DWT, STFT...) Characterization of nonstationarity of random processes Design of signal processing algorithms robust to input shift... R. Yu (EMU) µ-shift-invariance 4 / 25

5 Shift-Invariance of System H Definition (SI) System S : x y is shift-invariant if for any x and τ, it holds y( ) = S x( ) = y( τ) = S x( τ) Examples: linear SI systems via convolution CT : y(t) = h(t) x(t) = h(t τ)x(τ)dτ DT : y[n] = h[n] x[n] = k h[n k]x[k] where h = S δ is the impulse response. R. Yu (EMU) µ-shift-invariance 5 / 25

6 Shift-variant Systems: Examples But many practical systems are shift-variant. Common building blocks Modulation: y(t) = h(t) x(t) or y[n] = h[n] x[n] Sampling: y[n] = x(nt ) (ideal), y[n] = h(nt τ)x(τ)dτ (generalized), or M-downsampling: y[n] = x[mn] Time-scaling: y(t) = x(αt), or { x[n/l], n =, ±L, ±2L,..., L-upsampling: y[n] =, otherwise Useful signal processing systems Discrete-time wavelet transforms Short-time Fourier transforms Discrete Gabor transforms Then, how to design these systems that are nearly shift-invariant? R. Yu (EMU) µ-shift-invariance 6 / 25

7 µ-shift-invariance: 1 concept Note the change of time scale from input to output... Definition (µ-si) System H : x y is µ-shift-invariant if for any x, it holds CT: y( ) = S x( ) = y( ατ) = S x( τ), τ R or DT: y[ ] = S x[ ] = y[ αk] = S x[ k], k Z Examples L-upsampler is µ-shift-invariant with α = L. Interpolator y(t) = n x[n]h(t nt ) is µ-shift-invariant with α = T. Reflector y(t) = x( t) is µ-shift-invariant with α = 1. 1 µɛρlκσς means partial in Greek. R. Yu (EMU) µ-shift-invariance 7 / 25

8 µ-shift-invariance Properties and characterization Shifting input does not change the magnitude spectrum of the output, thus energy is preserved. Shifting in input result in only a linear phase shift ( (Y (ω)) ατω). Let S (L 2 L 2 ): x y be linear. S is µ-shift-invariant iff x and y are related in the Fourier domain as Y (ω) = H(ω)X (αω), ω B where B is an admissible defining band. That is, S is a cascade connection of LSI system H with an time-rate converter. Examples Decimator h( M) is µ-shift-invariant (with α = 1/M) if h is band-limited by 2π/M. Generalized sampler h( T ) is µ-shift-invariant (with α = 1/T ) if h is band-limited by 2π/T. R. Yu (EMU) µ-shift-invariance 8 / 25

9 Dual-tree Discrete-time Wavelet Transforms (DT-DWT) Filterbank implementation H (1) 1 2 x H (1) 2 H H ( 2 ) 1 ( 2 ) 2 yh ( 3 ) 2 H 2 1 y jy x h g ( 3) 2 (2) (1) ( 3) 2 (2) (1) 3 H (2 ω) H (2 ω) H ( ω) + jg (2 ω) G (2 ω) G ( ω) G G (1) 1 (1) 2 G ( 2 ) G H G ( 3 ) 2 ( 2 ) ( 3 ) y g H (l) 1 (ω) = H (l) (ω + π) G (l) 1 (ω) = G (l) (ω + π) G ( 3 ) 2 µ-si characterizations The DT-DWT is µ-shift-invariant at level L >, if H (l) and G (l) are bandlimited by 2π/3, l = 1,..., L. The µ-shift-invariance can be achieved if the scaling filters satisfy the half-delay condition [G (l) (ω) = e j ω 2 H (l) (ω), ω [ π, π).] R. Yu (EMU) µ-shift-invariance 9 / 25

10 Generalized sampling processes Mathematical descriptions S : y[n] = h(nt τ)x(τ)dτ S : ŷ(e jt ω ) = 1 ĥ(ω 2kπ/T ) x(ω 2kπ/T ) T k Z The kernel h characterizes the acquisition device or the transform The -norm.4 Meyer { Sx l 2 S = sup x x L 2 S = 1 T sup ω [,2π/T ) } (maximal energy gain) {( 1/2} ĥ(ω + 2kπ/T ) 2) k Z ω/π Hermitian hat ω/π R. Yu (EMU) µ-shift-invariance 1 / 25

11 µ-shift-variance Analysis of GSPs Commutators DT fractional shift operators D τ/t B : x(e jω ) e jωτ/t x(e jω ), ω B where B is the admissible defining band (e.g., [ π, π], [ 3π/2, π/2] [π/2, 3π/2], [ π/2, 3π/2]). (Example: τ = 1/2, x[n] x[n 1/2].) Characterization S is µ-shift-invariant K τ,b = for all τ R ĥ is supported in B/T = {ω/t : ω B} R. Yu (EMU) µ-shift-invariance 11 / 25

12 Generalized Commutators Description ŷ(ω) = 2 T e jωτ/t e jkπτ sin(kπτ/t )ĥ((ω 2kπ)/T ) x((ω 2kπ)/T ), ω B k Z Norm K τ,b 2 T sup ω B/T ) dependent on shift τ and band B { [ } sin(kπτ/t )ĥ(ω 2kπ/T ) 2] 1/2 k Z R. Yu (EMU) µ-shift-invariance 12 / 25

13 Quantifying shift-variance Shift-variance level { SVL(S) = inf sup B B τ R K τ,b } (Best B and worst τ) { SVL(S) inf ω R sup τ (,1/2] { sup ω [ω,ω +2π) { [ 2 sin(kπτ)ĥ(ω ] 1 }}} 2kπ) 2 2 k Z Shift-variance Index SVI(S) = SVL(S) 2 S SVI(S) 1, from µ-shift-invariant to maximally shift-variant; SVI(S) if S is nearly shift-invariant. R. Yu (EMU) µ-shift-invariance 13 / 25

14 Discrete-time wavelet transforms DWT as sampling processes x(t) { x, ψ m,n } m,n Z where ψ m,n(t) = a m/2 ψ(a m t bn), a, b > ; ψ is the wavelet function. Sampling process at scale m Z h(t) = ψ m,( t), and T = a m b. S m : x(t) { x, ψ m,n } n Z Scale-invariance of DWT SVL(S m ) = a m/2 SVL(S ) and SVI(S m ) = SVI(S ) Thus the SVL and SVI of S describes shift-variance of the DWT. R. Yu (EMU) µ-shift-invariance 14 / 25

15 Examples: six wavelet transforms Wavelet functions Wavelet Shannon Meyer Mexican hat Hermitian hat ψ(ω) { e jω/2, ω [π, 2π), otherwise (2π) 1/2 e jω/2 sin ( π 2 v( 3 2π ω 1)), ω [2π/3, 4π/3) (2π) 1/2 e jω/2 cos ( π 2 v( 3 4π ω 1)), ω (4π/3, 8π/3], otherwise where v(s) = s 4 (35 84s + 7s 2 2s 3 ), s [, 1] 8 3 π1/4 ω 2 e ω2 /2 2 π 1/4 ω(1 + ω)e ω2 /2 5 Complex Morlet π 1/4 [e j(ω 5)2 e (ω { )/2 ] e Complex Shannon jω/2, ω [π, 3π), otherwise R. Yu (EMU) µ-shift-invariance 15 / 25

16 Determination of system norms Frequency responses Meyer 1.6 Mexican hat ω/π Hermitian hat ω/π Complex Morlet ω/π ω/π Magnitude spectra ψ(ω) (solid line) and ψ(ω) (dashed line) for four wavelets (note that the overlapping) The plots also indicate how a good admission band B could be selected. R. Yu (EMU) µ-shift-invariance 16 / 25

17 Examples: shift-variance levels Observations Shift-variance level K τ,b for three complex wavelets The shift-variance levels are small for small τ <.5. With properly selected the defining band B, K τ,b are small for any value of τ, indicating nearly-shift-invariance. R. Yu (EMU) µ-shift-invariance 17 / 25

18 Examples: results Shift-variance analysis results Wavelet ψ Norm S Band B SVL(S) SVI(S)(%) Shannon 1. [ 2π, π) [π, 2π) Meyer.3989 [ 2π, π) [π, 2π) Mexican hat [ π, π) Hermitian hat.8651 [.9574π, 1.426π) Complex Morlet.7511 [.5916π, π) Complex Shannon 1. [π, 3π) Observations The two Shannon wavelets are µ-shift-invariant. Complex Morlet wavelet are nearly µ-shift-invariant. Meyer hat is most shift-variant ( 71%). Complex wavelets are generally less µ-shift-invariant than real wavelets. R. Yu (EMU) µ-shift-invariance 18 / 25

19 Shift-variance of Wavelets Transform: an illustration SVI =.965(Mexican hat),.772(meyer),.72(complex Morlet),.59(Hermitian hat). R. Yu (EMU) µ-shift-invariance 19 / 25

20 Shift-Variance of Short-Time Fourier Transforms Window* SVL(S) SVI(H) Gaussian Blackman Hanning 1..5 Hamming Rectangular *The support is [ 1, 1]. R. Yu (EMU) µ-shift-invariance 2 / 25

21 Concluding Remarks We provided answers to Q1: What does it mean for a shift-variant system to be nearly shift-invariant? Q2: How can one quantify shift-variance of a system? We provided shift-variance analysis for Generalized sampling processes, in particular for Discrete wavelet transforms and short-time Fourier transforms R. Yu (EMU) µ-shift-invariance 21 / 25

22 Further Applications Deterministic setting Design of transforms that are nearly shift-invariant Shift-variance and approximation error of generalized sampling and reconstruction processes Optimal kernel shifting towards orthogonal projection for mimimal error Superresolution algorithm based on tunable fractional shift operators Random setting Nonstationarity analysis of random process as shift-variance of its autocorrelation Detection of Weak signals Denoising of PAM signals R. Yu (EMU) µ-shift-invariance 22 / 25

23 References T. Aach (27), Comparative analysis of shift variance and cyclostationarity in multirate filter banks, IEEE Transactions on Circuits & Systems I: Regular Papers. T. Aach and H. Führ (29), On bounds of shift variance in two-channel mutirate filter banks, IEEE Transactions on Signal Processing. T. Aach and H. Führ (212), Shift variance measures for multirate LPSV filter banks with random input signals, IEEE Transactions on Signal Processing. R. Yu (29), A new shift-invariance of discrete-time systems and its application to discrete wavelet transform analysis, IEEE Transactions on Signal Processing. R. Yu (211), Shift-variance measure of multichannel multirate systems, IEEE Transactions on Signal Processing. R. Yu (212), Shift-variance analysis of generalized sampling processes, IEEE Transactions on Signal Processing. B. Sadeghi and R. Yu (216), Shift-Variance and nonstationarity of linear periodically shift-variant systems and applications to generalized sampling-reconstruction processes, IEEE Transactions on Signal Processing. R. Yu (EMU) µ-shift-invariance 23 / 25

24 Thank you. R. Yu (EMU) µ-shift-invariance 24 / 25

25 Thank you. R. Yu (EMU) µ-shift-invariance 25 / 25