Sampling Signals of Finite Rate of Innovation*

Transcription

1 Sampling Signals of Finite Rate of Innovation* Martin Vetterli EPFL & UCBerkeley *Joint work with T. Blu, I. Maravic (EPFL), and P. Marziliano,(NTU) Sampling FRI- 1

2 Outline 1. Motivation 2. Signals of Finite Rate of Innovation 3. The Periodic Case - Diracs, non-uniform splines and piecewise polynomials 4. Finite Length Signal Case - gaussian and sinc kernels 5. Applications 6. Multidimensional case - 2D Diracs - Radon transform 7. Wideband communications - CDMA and UWB - channel estimation 8. Conclusions Sampling FRI- 2

3 1 Motivation Signal Processors love bandlimited Signals... Then : { fnt ( )}, n Z, T = π ω m is a sufficient representation, since where ft () = fnt ( ) sinct ( T n) n Z sinct () sin( πt) = I[ ππ, ] πt (1) Sampling FRI- 3

4 Motivation(2) But what if f(t) F (ω) F t ω just one discontinuity and no more sampling theorem... Often, one does not have access to the signal itself, but to a measurement Example: neural spikes measured in non invasive manner ;) x(t) f(t) t Measuring device t t Sampling FRI- 4

5 Example: photographing stars Motivation(3) "Optical" Telescope Can we sample such signals that we see through an imperfect measuring device? There are many parametric signals which are far from bandlimited Example: CDMA Note: rate of transition is finite, given by the chip rate symbol rate much slower Sampling FRI- 5

6 Example: Woodcut pictures Motivation (4) Sampling FRI- 6

7 2 Signals of Finite Rate of Innovation What is so special about a signal f(t) bandlimited to [ ω m, ω m ]? With a sampling interval of T = π ω m the signal f(t) is specified by ρ = 1 T = ω m π degrees of freedom per unit of time. By the interpolation formula (1), any bandlimited signal can be generated as Sampling FRI- 7

8 Signals of finite (2) Definition: The number of degrees of freedom per unit of time is called the rate of innovation ρ. Rate of innovation Assume a class of signals having a parametric representation Consider one signal x from the class Call C x ( t, t 1 ) the number of degrees of freedom in ρ Then ρ = lim 1 -- τ τ C τ, τ x If ρ<, we call x a signal of finite rate of innovation Sampling FRI- 8

9 Example: Poisson process ρ Signals of finite (3) Interarrival times: i.i.d., pdf µe µt Expected interarrival time: 1 µ { t i } ρ = is a sufficient description of a realization = µ E( int. time) Sampling FRI- 9

10 Aquisition Model, Notation where x() t : signal ht (): sampling kernel yt (): filtered version of x() t y n : samples Sampling FRI- 1

11 Natural questions Signals of finite (5) 1. What are interesting classes of signals with finite 2. For which of these classes can we find unique representations through sampling (in particular uniformly) that is: ρ ρ x ht () Sampling y n = ht ( nt), xt () such that x y n just like in the bandlimited case 3. What are good kernels h() t? 4. What are the algorithms to find xt () from y n? Sampling FRI- 11

12 1. Classic, subspace case. Given known fct ϕ() t : xt () = c n ϕ t T -- n n Z Space: Span ϕ t T -- n This is a well studied case (sampling, non-uniform sampling, reconstruction). It is a linear problem. Example: Bandlimited signals [ w m, w m ], ϕ() t = sinc(t) Basis: Ex: Example: Uniform, B-splines, Basis: Ex: Sampling FRI- 12

13 2.Arbitrary shifts, known ϕ() t : xt () = c n ϕ t T -- τ n n Z This is not a subspace! Example: Non-uniform splines Sampling FRI- 13

14 3. Arbitrary shifts, set of known fcts ϕ r () t : R xt () c nr ϕ t = τ n T n Z r= Example: Non-uniform piecewise polynomials Sampling FRI- 14

15 1 Periodic signal infinite kernel The 4 cases of interest: 2 Finite signal infinite kernel 3 Infinite signal finite kernel 4 Infinite signal infinite kernel Note: 1, 2 and 3 lead to finite dimensional problems Sampling FRI- 15

16 3 The periodic case Fourier series xt () = j2πmt Xm [ ] e τ m Z Sampling FRI- 16

17 3.A Periodic stream of Diracs K Diracs per τ : 2K degrees of freedom ρ = 2K τ K 1 K 1 j2πm( t t k ) xt () = c n δ( t t n ) = c k -- e τ c k δ( t t k nτ) = τ n Z n Z k = k = m Z or Xm [ ] K 1 2πmt j k = -- c τ k e τ m Z k =! Xm [ ] is a weighted sum of K exponentials j2πt k m τ e Sampling FRI- 17

18 Consider: j2πt K Az ( ) Am [ ]z m K k τ 1 e z 1 = = m= k = Now, note that j2πt k j2πt k j2πt τ k j4πt τ τ k τ [ 1, e ] [, e, 1, e, e, ] is zero, from which follows that Am [ ] X[ m] = Equivalently, in time domain at () = Az ( ) = z = e 2πt j k τ K 1 k = j2π( t k t) τ 1 e has zeros at t = t k k =,, K 1, thus at () xt () = Az ( ) is called an annihilating filter, since it kills xt () ECC: error locator polynomial Sampling FRI- 18

19 Theorem 1: Consider a periodic stream of K Diracs, of period τ, weights { c k } and locations { t k }. Take a sampling kernel h sinc ˆ = I[ ππ, ] β () t = βsinc( βt) where β 2K = > ρ τ Pick N = 2K + 1 and T = τ N Then y n = h β ( t nt), xt (), n =,, N 1 is a sufficient characterization of x(t). Sampling FRI- 19

20 Proof 1. y n is a sufficient characterization of Xm [ ], m = K K Either use Poisson Am [ ]z m, or graphically: 2. Finding Am [ ] s.t. Am [ ] X[ m] = A[ ] = 1, solve for m = 1 K. This leads to a Toeplitz system, e.g. K = 3 X[ ] X[ 1] X[ 2] X[ 1] X[ ] X[ 1] X[ 2] X[ 1] X[ ] A[ 1] A[ 2] A[ 3] = X[ 1] X[ 2] X[ 3] Classic Yule-Walker system Unique solution for distinct Dirac locations Sampling FRI- 2

21 K 1 3. Factorisation of Az ( ): Az ( ) = ( 1 u k z 1 ) k = j2πt where k u τ, thus is found k = { t e k } K 1 k= 4. Finding the weights c k. Given { t k }, K values of X[ k] are given, for ex. for K = 3 X[ ] X[ 1] X[ 2] = 1 -- τ u u 1 u u u1 u2 c c 1 c 2 which is a Vandermonde system, having always a solution given distinct t k s. Sampling FRI- 21

22 K=8 xt () ht () yt ()y, n 2 2 x Time Freq Sampling FRI- 22

23 Diracs Real(A) Imag(A) Sampling FRI

24 Interpretation The projection of xt () onto the lowpass space BL K2π, K2π τ τ is one-to-one for a periodic stream of K Diracs Corollary 1: Given Am [ ], m = K and Xm [ ], m = K K one can recover the entire spectrum as K Xm [ ] = Ak [ ]Xm [ k], m = K + 1 k = 1 Proof: left to the reader Notes: 1. annihilating filter known in sinusoidal retrieval from noise 2. same filter used in error correction coding, and called error locator polyn 3. recursive spectrum extrapolation known as Berlekamp-Massey algo. in ECC 2,3 over finite fields... Sampling FRI- 24

25 3.B Non-uniform splines A signal xt () is a periodic non-uniform spline of degree R with K knots at { t k } K 1 iff its ( R + 1) th derivative is periodic of the k= form ( R + 1) x () t = c m δ( t t m ) m Z where t m + k = t m + τ Clearly, the Fourier series satisfy ( R + 1) j2πm X [ m] = R + 1 Xm [ ] (*) τ Thus Sampling FRI- 25

26 Theorem 2: Consider a periodic non-uniform spline of max degree R and period τ. Take h β () t as sampling kernel, with β 2K + 1 τ = and T = --- N = 2K + 1 τ N Then y n = h β ( t nt), xt () n = N 1 uniquely defines xt (). ( R + 1) Proof: similar to Thm 1 to get Xm [ ]. Then X [ m] follows from (*), to which we apply Thm 1. X[ ] is added at the end Sampling FRI- 26

27 xt () ht () yt ()y, n.6 1 x Time Freq Sampling FRI- 27

28 3.C Derivatives of Diracs δ () r () t : ft ()δ () r ( t t ) d t = ( 1) r f () r ( t ) where f is r-times differentiable Then a periodic stream of differentiated Diracs is R m 1 xt () = c δ () r ( t t mr m ) m Z r= K 1 There are: K locations, K = R k weights. k = K+ K Thus: ρ = τ It can be verified that: Xm [ ] K 1 R 1 m 1 -- c j2πm τ r j2πmt k = e τ kr τ k = r= Sampling FRI- 28

29 The annihilating filter now requires multiples zeros, since ( 1 u k z 1 ) R annihilates m R 1 k u m. Thus Az ( ) becomes K 1 Az ( ) = ( 1 u k z 1 ) k = Then: Am [ ] X[ m] =, therefore, one can show: R k Sampling FRI- 29

30 Theorem 3: Consider a periodic stream of differentiated Diracs as above. Take as sampling kernel h β () t = βsinc( βt) with β = ρ+ 1 τ and sample h β Sx at N points t = nτ N where n = N 1 and N = K+ K + 1. Then y n h β t n τ = N ---, xt () n = N 1 is a sufficient characterization of xt (). Proof: Similarly to Thm 1, we first get Xm [ ] from y n. Then we solve for the location { t k } Am [ ] X[ m] = and finally for the coefficients { c kr }. The latter calls for a generalized Vandermonde system which is non-singular for t i t j i j. Sampling FRI- 3

31 3.D Piecewise Polynomials A periodic piecewise polynomial xt () with K pieces of degree max R has an ( R + 1) th derivative which is a stream of differentiated Diracs, or R m 1 xt () = c δ () r ( t t mr m ) m Z r= There are: K locations, K = ( R + 1)K weights ρ = ( R + 2)K τ Sampling FRI- 31

32 Then: Theorem 4: A signal defined by its derivatives as in (**) can be recovered after convolution by h β () t, where β= ρ+ 1 τ and sampling at t = nτ N with N = ( R + 2)K + 1, that is y n h β t n τ = N ---, xt () n = N 1 uniquely specifies xt () Proof: left to the reader, along Theorem 1, 2 and 3. Sampling FRI- 32

33 xt () ht () yt ()y, n.15 2 x Time Freq Sampling FRI- 33

34 4 Finite Length Signals A finite length signal with finite degrees of freedom. ρ clearly has a finite # of The question of interest is: given a sampling kernel with a infinite support (like the sinc or the gaussian), is there a finite set of samples that uniquely specifies the signal? Sampling FRI- 34

35 4.1 Gaussian Kernel Consider the same signal as in (4.1), now using a gaussian kernel ht () = e t2 ( 2σ 2 ) Then, the sample values are t -- n 2 ( 2σ 2 ) y n = xt ()e, T y n = K 1 k = t k c k e --- T n 2 ( 2σ 2 ) Sampling FRI- 35

36 Expanding (4.1) y n = t k c k et 2 2σ 2 k nt k et2σ 2 n e2σ 2 (4.2) Introduce Y n = n et2σ 2 y n Thus Y n = k t k c k et 2 2σ 2 a k t k n et2σ 2 n u k Y n = K 1 n a k u k (4.4) k = that is... a linear combination of exponentials! Sampling FRI- 36

37 Therefore, use the usual method of the good old annihilating filter A*Y = and factor it such as to find From u k : { u } k k = K 1 t k = 2σ 2 Tlnu k From u k and t k and K values of Y n, we can solve for c k in (4.2). Thus Sampling FRI- 37

38 Theorem 5: Given a finite stream of K Diracs and a gaussian kernel ht () = e t2 ( 2σ 2 ), then N samples y n = xt ()h, t T -- n where N 2K, are sufficient to reconstruct the signal. Note: Similar remarks as for Theorem 3... But: Here, unlike in the sinc case, we have an almost local reconstruction because of the exponential decay of ht ()! Sampling FRI- 38

39 4.2 Sinc kernel (Thierry s tour de force) Consider a finite sequence of spikes K 1 xt () = C k δ( t t k ) k = (4.5) and a kernel sinc( t T) The samples y n = xt (), sinc t are T -- n K 1 y n C k c t k sin --- n ( 1) n K 1 sin( πt k T) = = C T k k = k = π t k --- n T (4.6) Sampling FRI- 39

40 Introduce the following interpolators: K 1 t K k Pu ( ) = ---, deg. T u = p k u k K k = k = t k P l ( u) = ---, deg. T u K 1 k l Then, consider the following Y n ( 1) n K 1 1 = Pn ( )yn ( ) = -- C π k sin( ( πt k ) T)P k ( n) k = Y = A C Now (key insight!) Y n is of degree K 1 Thus K Y n = n = K N 1 V p = N K K Note: ~ k similar to annihilating filter (4.7) (4.8) Sampling FRI- 4

41 So, as long as N K K, one can use (4.4) to solve for P k from y. This leads to { t n, t 1,, t K 1 }. Using this in (4.6) allows to solve for { c i }. Thus: Theorem 6: Given a finite stream of K Diracs and a sinct ( T) kernel, N samples y n = xt (), sinc t where T -- n n = N 1 N 2K, are sufficient to reconstruct the signal. Note: the result does not depend on T! of course, it shows up in the conditionning of linear system!! Sampling FRI- 41

42 The steps to reconstruct the signal are 1. Solve a linear system KxK { y i } { p i }, i = K 1 ( p k = 1) 2. Factor Pu ( ) { t i }, i = K 1 3. Solve linear system { c i } This method can be extended to piecewise polynomials, similarly to Theorem 4. Also, there is an obvious equivalent for discrete-time signals from l 2 ( Z) and discrete-time sinc kernels. Sampling FRI- 42

43 Sinc Kernel, finite length signals log 1 condition number log 1 (condition number) T Conditioning on location T Conditioning on weights Sampling FRI- 43

44 5. Applications We show 2 direct applications of the results shown above. 5.1 Piecewise Bandlimited Signals Consider a signal that is the sum x = x BL + x PP where x BL is bandlimited and x PP is piecewise polynomial. Assume x BL is specified by its frequency component X BL [ k], k [ M, M] while x PP has 2K degrees of freedom. Sampling FRI- 44

45 Then, consider the spectrum of X[ k], k [ M 2K, M + 2K]. First, using Xk [ ], k [ M+ 1, M + 2K] and the technique of Proposition 1 or Theorem 1, we can recover x PP. Substracting X PP from X, we can then recover X BL. Sampling FRI- 45

46 Piecewise Bandlimited Signal Sampling FRI- 46

47 Thus: Proposition 3: Given a piecewise BL signal of length N, with 2M + 2K degrees of freedom. Pick Q a divisor of N and ϕ[ n] = IDTFS( I[ 2K M, M + 2K] ). Then yl [] = xn [ ], ϕ[ n lq] circ uniquely specify xn [ ] if N > M + 2K 2Q The proof follows from earlier results with adjustements Sampling FRI- 47

48 5.2 Filtered Piecewise Polynomials Consider a stream of K Diracs convolved with a known filter gt () g t t Thus: xt () = gt () * dt () where g is known and d() t = α i δ( t t i ) i Clearly, if gn [ ] Gk [ ] is invertible over 2K frequency values, then we can use Proposition 1. Sampling FRI- 48

49 Example: 1 Stream of Diracs 1 Filtered stream of Diracs 1.5 Filter Sampling FRI- 49

50 In particular: Propositon 4: Assume xn [ ] with K Diracs and a filter G[ k], k [ K, K]. The signal we observe is xn [ ] * gn [ ]. Using ϕ[ n] IDTFS ( I [ K, K] ) N = and M such that > K, M a divisor of N 2M Then yl [] = xn [ ], ϕ[ n lm] is a sufficient representation of xn [ ]. A more difficult case appears when finite ρ... gn [ ] is unknown but of Sampling FRI- 5

51 6 Multidimensional Case 2D Poisson: K Diracs on R 2 T Various approaches non separability is the key! Xm [ 1, m 2 ], m i K is sufficient OK ( 2 ) samples Xm [ 1, m 1 ], m 1 K is sufficient OK ( ) samples --> 2D root finding (...) or spectral extrapolation Extension: lines simples objects Goal: #samples ~ #deg. of freedom of object Sampling FRI- 51

52 Example o f a 2 D g a u s si a n k e r n el: Finite set of M=17 weighted Diracs Gaussian sampling kernel y 1 x 5 y Convolution with the sampling kernel x Reconstructed signal y x y 5 x Sampling FRI- 52

53 Radon Transform 2D methods based on projections fxy (, ) F( θ, t) What about finite complexity objects? Projections are finite rate of innovation! θ n θ o θ n θ Sampling FRI- 53

54 y y Result: Set of K Diracs can be perfectly reconstrucded from K+1 bandlimited projections with 2K samples x x See [Maravic] ICASSP-22 Sampling FRI- 54

55 7 Communications Applications Many communication systems use wideband signalling CDMA: chip rate >> symbol rate UWB: pulse position modulation In both cases rate of innovation << bandwidth But: Noise! Solution: oversample subspace methods, SVD Sampling FRI- 55

56 7.1 Solving for sinusoids in noise Idea: Solve for longer filter: M+1xM+1 x( ) x( 1) x( M) x( 1) x( ) x( 2) xm ( ) x( ) a 1 a 2 a M 1 = x( 1) xm ( + 1) using 2M+1 samples > 2K oversample Now: The noiseless Toeplitz matrix has rank K (# of sinusoids) with A = a a 1 a K 1 where a i = e jω i M 1 e jω i M T Sampling FRI- 56

57 we can write the Toeplitz matrix as T = A α α 1 A M + N α K 1 where N is the noise Toeplitz matrix Thus: If the sinusoids dominate the noise (M large enough), a K-dimensional subspace idendifies the sinusoids Then: 1. Compute SVD of T 2. Approximate by K largest singular value: T Tˆ 3. Solve Tˆ a = x on subspace 4. Find roots closest to U.C. best approximation of sinusoids Sampling FRI- 57

58 Note: Many alternative available well studied problem time versus correlation domain Example: - MUSIC - ESPRIT - NL Sampling FRI- 58

59 7.2 Multiuser Communication Direct Sequence Code Division Mult. Access (DS-CDMA) Model: - User i has a signature sequence S i - each bit is spread into this signature Ex: chip length signature or symbol length L Thus: {Ui } L S i D/A channel Clearly: rate of innovation is symbol rate Usually: sampling done at chip rate or faster Now: chip rate > symbole rate! (e.g. L=511) Sampling FRI- 59

60 But: - multiaccess scheme - multipath environment Multiaccess: signature are orthogonal Multipath: small number of dominant pulses p () l User i: p i () t = β i δ( t t k ) k = 1 Two phases 1. Channel estimation: Using training sequences, { p i () l } i = 1.. K is estimated 2. Detection: Based on the channel estimate, various detectors (e.g. MMSE) can be applied Question: For a digital receiver, Should one run: - channel estimation - detection at symbol rate or chip rate? Sampling FRI- 6

61 {U L S 1 P1 1} Detect.1 {U 2 } L S 2 P 2 Detect Chann. est {U K } L S K P K Detect.K K Users Signatures Channels MAC Chann. Est. Detection S i S j analog Sampling FRI- 61

62 Channels: - K users - P multiple paths Degrees of freedom But: users can use training sequences of length K Result: Solving K linear systems of O(M) with M 2P, is sufficient for channel estimation Sampling FRI- 62

63 7.3 Ultrawideband communications Very low signal to noise ratio (-15 db) Used for communications in unlicensed spectrum and for ranging applications Bandwidth: several GHz Very difficult to design digital receivers.25 Transmitted sequence of UWB pulses & Received signal t Results: Finding one dominant eigenvalue can be sufficient! Sampling FRI- 63

64 8 Conclusions We have seen: Many signals that look unsampleable actually can be sampled at their rate of innovation! Methods: give me an exponential and I will annihilate it! Structured linear systems with fast algorithms OK ( 2 ) Can be generalized (rotational, 2D) But: There are many more signals with finite rate of innovation Conjecture: They can be sampled at or above their rate of innovation! Sampling FRI- 64

65 Outlook Many other parametric classes are of interest (piecewise trigonom.) Often, there is a low degree of freedom explanation This is not necessarily a subspace (e.g. manifold) Super-resolution signal processing for appropriate models (channels, images, etc...) has great potential Occam s Razor for sampling! Sampling FRI- 65

66 References M. Vetterli Sampling of piecewise polynomial signals, LCAV Technical Report, EPFL, Switzerland, Dec M.Vetterli, P.Marziliano, T.Blu, A Sampling Theorem for Periodic Piecewise Polynomial Signals, ICASSP-21 M.Vetterli, P.Marziliano, T.Blu, Sampling Piecewise Bandlimited Signal, SAMP- TA-21 M.Vetterli. P.Marziliano, T.Blu, Sampling Signals with Finite Rate of Innovation, IEEE Tr. on SP, June 22 P. Marziliano, Sampling Innovation, PhD Thesis, EPFL 21. J. Kusuma, A. Ridolfi, M. Vetterli, Sampling signals with bandwidth expansion, ICC 22. I. Maravic, M. Vetterli, Sampling Results for Classes of Non-Bandlimited 2-D Signals Trans. on Signal Processing, accepted for publication. I. Maravic, M. Vetterli, Digital DS-CDMA Receivers Working Below the Chip Rate: Theory and Design, submitted to the IEEE Trans. on Signal Processing. I. Maravic, M. Vetterli, A Sampling Theorem for the Radon Transform of Finite Complexity Objects, in Proc. ICASSP, May 22 Sampling FRI- 66