Distributed Noise Shaping of Signal Quantization

Size: px

Start display at page:

Download "Distributed Noise Shaping of Signal Quantization"

Melvyn Richards
5 years ago
Views:

1 1 / 37 Distributed Noise Shaping of Signal Quantization Kung-Ching Lin Norbert Wiener Center Department of Mathematics University of Maryland, College Park September 18, 2017

2 2 / 37 Overview 1 Introduction Pulse Coding Modulation (PCM) Quantization Alternative Scheme: Σ and Noise Shaping 2 Adaptation to Finite Dimensional Space Quantization on Frame Setting Modification on Dual Frame 3 Distributed Noise Shaping: Beta Dual Main Result Setting of Distributed Noise Shaping Beta Dual for Unitarily Generated Frames 4 Reference

3 3 / 37 Introduction 1 Introduction Pulse Coding Modulation (PCM) Quantization Alternative Scheme: Σ and Noise Shaping 2 Adaptation to Finite Dimensional Space Quantization on Frame Setting Modification on Dual Frame 3 Distributed Noise Shaping: Beta Dual Main Result Setting of Distributed Noise Shaping Beta Dual for Unitarily Generated Frames 4 Reference

4 Introduction A/D Conversion for Signals Theorem (Classical Sampling Theorem) Given f PW [ 1/2,1/2], i.e., f,ˆf L 2 (R), and supp(ˆf ) [ 1/2, 1/2]. Then for any g satisfying ĝ(ω) = 1 on [ 1/2, 1/2] ĝ(ω) = 0 for ω 1/2 + ɛ, and for any T (0, 1 2ɛ), t R, f (t) = T n Z f (nt )g(t nt ) (1) where the convergence is both uniform on compact sets and in L 2. Remark f has a continuous representative, so it makes sense to evaluate f at points. 4 / 37

5 5 / 37 Introduction 2.5 Frequency Domain Plot for f and g Figure: Black: Signal f, Red: Reconstruction Kernel g In particular, it is only necessary to store {f (nt )} n Z to reconstruct f. Computers cannot store real numbers, so instead {q n } n Z A is considered where A is a finite subset of R.

6 6 / 37 Introduction Pulse Coding Modulation (PCM) Quantization Naive Approach: Pulse Coding Modulation (PCM) Given a finite alphabet A R, define Q : R A by Q(x) = arg min x q (2) q A For a bandlimited function f PW [ 1/2,1/2], its reconstructed function via PCM will be f (t) = T Q(f (nt ))g(t nt ) (3) n Z Remark In practice, mid-rise uniform quantizer is often used. That is, A = {(k + 1/2)δ : k = N,..., N 1} (4)

7 7 / 37 Introduction Pulse Coding Modulation (PCM) Quantization Reconstruction Error Estimate of PCM For the mid-rise uniform quantizer, the reconstruction error is f (t) f (t) = T ( ) f (nt ) Q(f (nt )) g(t nt ) n Z δ T n Z g(t nt ) (5) = C g,t δ Remark C g,t g 1 as T 0 +, so oversampling doesn t improve the reconstruction significantly.

8 Introduction Pulse Coding Modulation (PCM) Quantization Caveat of PCM: Imperfect Quantizers Consider the following imperfect base quantizer 0 if x 1 2 ɛ Q 1 (x) = 1 if x ɛ (6) 0 or 1 if x ( 1 2 ɛ, ɛ) For x (0, 1),Let Q k (x) = k n=1 ( where Q n (x) = Q 1 2 n 1 (x n 1 Q s(x) s=1 2 ) ). s Q n (x) 2 n (7) 8 / 37

9 Introduction Pulse Coding Modulation (PCM) Quantization 0.7 Approximation with Imperfect Quantizer Approximation Value Input Correct Approximation Imperfect Quantizer Approximation Level Figure: Illustration of Imperfect Quantization Remark For x ( 1 2 ɛ, ɛ), Qk (x) x ɛ in worst case scenario, and we cannot improve such issue by adding more bits. 9 / 37

10 Introduction Alternative Scheme: Σ and Noise Shaping Alternative Option: Σ Quantization Introduce auxiliary variable {u n } n Z and the recursive equation where q n = Q(u n + f (nt )) for each n. u n+1 = u n + f (nt ) q n (8) Proposition (Uniform Boundedness of {u n }) With the choice of mid-rise uniform quantizer A = {(k + 1/2)δ : k = N,..., N 1}, u < δ if sup f (nt ) (N 1)δ. We call such scheme a stable Σ quantization scheme. 10 / 37

11 Introduction Alternative Scheme: Σ and Noise Shaping Reconstruction Error for Σ Quantization Consider the reconstructed signal f (t) = T q n g(t nt ) (9) n Z Then the reconstruction error is f (t) f (t) = T (nt ) q n)g(t nt ) n Z(f = T n+1 u n)g(t nt ) n Z(u = T n Z u n(g(t nt ) g(t (n 1)T )) (10) = T n Z nt u n g (t u) du (n 1)T T u g 1 0 as T 0 Note that g is independent of T. 11 / 37

12 Introduction Alternative Scheme: Σ and Noise Shaping Robustness of Σ Against Imperfect Quantizer For Σ quantization, the scheme is u n+1 = u n + f (nt ) q n (11) Imperfect quantizer Q gives a larger sup-norm for {u n }. In particular, it now changes to u δ + ɛ. However, the scheme can still be stable, and the reconstruction error is still f f T u g 1 (12) 12 / 37

13 Introduction Alternative Scheme: Σ and Noise Shaping r-th Order Σ Quantization It is now a natural step to consider the following scheme: y q = r u (13) Existence of a stable scheme of such kind is proven in [6, Daubechies & DeVore (2003)]. Proposition (Error decay for high order Σ ) Let f and f as before, except that {q n } now comes from (13). Suppose the kernel g C r, then f f T r u g (r) (14) Again, both u and g (r) are independent of sampling period T. 13 / 37

14 Introduction Alternative Scheme: Σ and Noise Shaping Noise Shaping Feature of Σ Quantization normalized frequency (cycles/sample) Figure 1: Illustration of classical noise shaping via SD modulation: The superimposed Fourier spectra of a bandlimited signal Figure: (in black), and Classical the quantization noise errorshaping signals usingvia MSQ Σ (in red), modulation[5, 1st order SD modulation Chou, (ingunturk, magenta), and 2nd order SD modulation (in blue). Krahmer, Saab, Yilmaz (2015)] pulse-code modulation (PCM). If each sample is quantized with error no more than d, i.e., ky qk apple d, then the error signal e(t) := x(t) (Yq)(t)=t Â n2z y n q n y(t nt) (4) obeys the bound kek L apple Cd where C is independent of d. This is essentially the best error bound one can expect for Nyquist-rate converters. Because setting d very small is costly, Nyquist-rate converters are not very suitable for signals that require high-fidelity such as audio signals. Oversampling converters are designed to take advantage of the redundancy in the representation (1) when t < t crit. In this case, the interpolation operator Y has a kernel which gets bigger as t! 0. Indeed, let by(x )=0 for x > W 0. Black Fourier spectra of a bandlimited signal Red Quantization error signals using PCM Pink Error signal for 1st order Σ quantization Blue Error signal for 2nd order Σ quantization 14 / 37

15 15 / 37 Introduction Alternative Scheme: Σ and Noise Shaping Generalization: Noise Shaping Quantization Instead of difference operator, consider the following scheme: y q = h u (15) where 1 h = {h n } n N has h 0 = 1, and 2 (h u) n = m=0 h mu n m [8, Gunturk, 2003] constructed a family of h to achieve exponential decay, with sub-optimal exponent.

16 16 / 37 Adaptation to Finite Dimensional Space 1 Introduction Pulse Coding Modulation (PCM) Quantization Alternative Scheme: Σ and Noise Shaping 2 Adaptation to Finite Dimensional Space Quantization on Frame Setting Modification on Dual Frame 3 Distributed Noise Shaping: Beta Dual Main Result Setting of Distributed Noise Shaping Beta Dual for Unitarily Generated Frames 4 Reference

17 17 / 37 Adaptation to Finite Dimensional Space Quantization on Frame Setting Finite Frame and Quantization For a given space F k where F = R or C, suppose {e n } m n=1 Fk is a spanning set and let the rows of E F m k be {e n}, the conjugate transpose of {e n }. Then for any dual F F k m, we have FE = I k (16) In particular, if F = (f 1 f m ), then for any x F k, m x = < x, e n > f n (17) n=1 where f n F k. Then the quantized version x shall be m x = q n f n (18) n=1

18 18 / 37 Adaptation to Finite Dimensional Space Quantization on Frame Setting First Order Σ Quantization for Finite Frames Consider the following scheme: y q = u (19) Then the reconstruction error x x 2 is m x x 2 = (< x, e n > q n)f n 2 = = n=1 m ((u n u n 1 )f n) 2 n=1 m u n(f n f n+1 ) + u mf m 2 n=1 m u ( f n f n f m 2 ) n=1 (20)

19 19 / 37 Adaptation to Finite Dimensional Space Quantization on Frame Setting Frame Analogy of Σ Quantization[1] Definition (Frame Variation) Let E = {e n } m n=1 be a finite frame for Rk, and p a permutation of {1, 2,..., N}. The variation of the frame E with respect to p is σ(e, p) := m 1 n=1 e p(n) e p(n+1) 2 (21) Theorem ([1], Benedetto, Powell, Yilmaz, 2006) Suppose E = {e n } m n=1 is a zero-sum FUNTF with frame bound m/k. Then the reconstruction error x x 2 satisfies x x 2 { δk 2m δk σ(e, p) if m is even 2m (σ(e, p) + 1) if m is odd (22)

20 Adaptation to Finite Dimensional Space Modification on Dual Frame Noise Shaping Scheme Definition Let B be the unit ball centered around origin of F m. A map Q : B A m is a stable noise shaping scheme if H: lower triangular, {u n } uniformly bounded by δ such that y q = Hu (23) where q = Q(y) and we use x = Fq as the reconstruction vector for x. Stability: [4, Chou, Gunturk, 2016] gives a sufficient condition for such scheme to work Recursiveness: Requires H to be lower triangular 20 / 37

21 21 / 37 Adaptation to Finite Dimensional Space Modification on Dual Frame Reconstruction Frame/ Kernel F Many choices. Depends on H in quantization scheme ( x Fq = F(y q) = FHu ) Canonical dual: E V-dual: (VE) V such that VE is still a frame. 1 Sobolev dual [2]: ( 1 E) 1 achieves minimum 2-norm for F. 2 Alternative dual: (H 1 E) H 1 3 Beta dual: (V β E) V β, V β to be specified later. How does the choice of V affect our reconstruction?

22 22 / 37 Adaptation to Finite Dimensional Space Modification on Dual Frame De-Noising Aspect of Dual Frame im(h 1 E) im(ve) Alt. Dual Noise Shaping F k H 1 E F m H F m V F p Noise u Figure: Error Cutoff Procedures for Different Quantization Schemes For M F l k injective, ker(m ) = ( M(F k ) ).

23 next section. In Fig. 2, we illustrate a beta dual of a certain roots-of-unity frame along with the Sobolev duals of order 0 (the Adaptation to Finite Dimensional Space canonical dual), 1, and 2. Modification on Dual Frame canonical dual 1st order Sobolev dual 2nd order Sobolev dual beta dual with β=1.6 and p=3 Figure: Different alternative duals for the 15th roots-of-unity frame in re 2: Comparative R 2 [5] illustration of the various alternative duals described in this paper: Each plot depicts the original e in R 2 consisting of the 15th roots-of-unity along with one of its duals (scaled up by a factor of two for visual 23 / 37

24 24 / 37 Distributed Noise Shaping: Beta Dual 1 Introduction Pulse Coding Modulation (PCM) Quantization Alternative Scheme: Σ and Noise Shaping 2 Adaptation to Finite Dimensional Space Quantization on Frame Setting Modification on Dual Frame 3 Distributed Noise Shaping: Beta Dual Main Result Setting of Distributed Noise Shaping Beta Dual for Unitarily Generated Frames 4 Reference

25 25 / 37 Distributed Noise Shaping: Beta Dual Main Result Theorem (Chou, Gunturk) Given a unitarily generated frame Φ with generator Ω, a k k Hermitian matrix with {v s } k s=1 being a basis of orthonormal eigenvectors and φ 0 F k. Suppose the eigenvalues are all distinct modulo l where l k, then we have ( ) { m 2 L m/l if F = C x F V q 2 < 7e + 1 c(φ 0 ) l L m/l if F = R (24) where c(φ 0 ) := ( min < φ 0, v s > ) 1 (25) 1 s k

26 Distributed Noise Shaping: Beta Dual Setting of Distributed Noise Shaping Setup of Distributed Noise Shaping (DNS) Definition (V-dual) Let E R m k be a frame, m > k. F V R k m is a V-dual of E if F V = (VE) V (26) where V R p m such that VE is still a frame. Recall that a stable noise shaping scheme has y q = Hu (27) 26 / 37

27 27 / 37 Distributed Noise Shaping: Beta Dual Setting of Distributed Noise Shaping Setup of DNS (Cont d) In the setting of DNS, V R p m and H R m m are block matrices V 1 V 2 V = V 3... H 1 H 2, H = H 3... V l where V i R p i m i, H i R m i m i with p i = p, m i = m. H l (28)

28 28 / 37 Distributed Noise Shaping: Beta Dual Setting of Distributed Noise Shaping β-dual Definition (β-dual) A β-dual F V = (VE) V has V = V β,m, where β = [β 1,..., β l ] t and m = [m 1,..., m l ] t, is a l-by-m block matrix such that V i = [β 1 i, β 2 i,..., β m i i ] R 1 m i, i.e. l = p. {β i } satisfies β i > 1 for every 1 i l, whose choice is limited by a technical lemma. Under this setting, each H i is chosen to be a m i m i matrix with unit diagonal entries and β i sub-diagonal entries.

29 29 / 37 Distributed Noise Shaping: Beta Dual Setting of Distributed Noise Shaping β-dual (Cont d) V i H i = [β 1 i β 2 i β m i i ] = ( β m ) i i 1 β i 1 0 β i β i 1 where β i > 1, so it effectively reduced the size of error. (29)

30 30 / 37 Distributed Noise Shaping: Beta Dual Setting of Distributed Noise Shaping Reconstruction Error Estimate for β-dual Lemma (Chou, Gunturk, 2016) Given a β-dual V, suppose VE is a frame, then the reconstruction error is x F V q 2 = F V Hu 2 F V H 2 u u 1 σ min (VE) VH 2 l σ min (VE) δβ m/l < E 2 e(1 + m/l ) l L ( m/l +1) σ min (VE) (30)

31 31 / 37 Distributed Noise Shaping: Beta Dual Beta Dual for Unitarily Generated Frames Proof of Theorem[3] Assume that m/l N for simplicity. Assumption: Eigenvalues {λ s } are distinct modulo l. σ min (VE) can be controlled uniformly for all m in such setting. E 2 is easily seen to be uniformly bounded. Given a k k Hermitian matrix Ω and φ 0 F k, consider U t := e 2πıΩt, φ n = U n m φ 0, n = 0,..., m 1 (31) Set E to be the collection of such elements, that is, E = φ 0. φ m 1 = E 1. E l (32)

32 32 / 37 Distributed Noise Shaping: Beta Dual Beta Dual for Unitarily Generated Frames Proof (Cont d) Then, where VE = m/l V 1 E 1. (33) V l E l (V j E j ) = β n φ (j 1)m/l+n F k (34) n=1

33 33 / 37 Distributed Noise Shaping: Beta Dual Beta Dual for Unitarily Generated Frames Now, let {v s } be an ONB of eigenvectors with respect to Ω with eigenvalue {λ s }. Then where m/l < (V j E j ), v s > = β n < U (j 1)m/l+n φ 0, v s > m n=1 m/l = β n < φ 0, U (j 1)m/l n v s > m n=1 m/l = β n (j 1)m/l+n 2πı e m λ s < φ 0, v s > n=1 (j 1) 2πı = e l λ s w s < φ 0, v s > m/l ( w s = β 1 e 2πıλs/m) n n=1 (35) (36)

34 34 / 37 Distributed Noise Shaping: Beta Dual Beta Dual for Unitarily Generated Frames Remark m/l ( w s = β 1 e 2πıλs/m) n 1 β m/l e 2πıλs/l = 1 β 1 e 2πıλs/m 1 β β 1 n=1 (37)

35 35 / 37 Distributed Noise Shaping: Beta Dual Beta Dual for Unitarily Generated Frames Then, for any x F k, VEx 2 2 = = = = l l V j E j x 2 j=1 l k < (V j E j ), v s >< x, v s > 2 j=1 k s=1 s=1 t=1 k l < x, v s >< v t, x >< φ 0, v s >< v s, φ 0 > w s w t k < x, v s > 2 < φ 0, v s > 2 w s 2 s=1 ( ) 1 β 1 2 l min 1 + β < φ 0, v 1 s > 2 x s k if λ s λ t are integers and nonzero modulo l. j=1 e 2πı(j 1)(λ t λ s)/l (38)

36 Reference Reference I J. J. Benedetto, A. M. Powell, and O. Yilmaz. Sigma-delta quantization and finite frames. IEEE Transactions on Information Theory, 52(5): , J. Blum, M. Lammers, A. M. Powell, and Ö. Yılmaz. Sobolev duals in frame theory and sigma-delta quantization. Journal of Fourier Analysis and Applications, 16(3): , E. Chou and C. S. Güntürk. Distributed noise-shaping quantization: Ii. classical frames. Not yet published,. E. Chou and C. S. Güntürk. Distributed noise-shaping quantization: I. beta duals of finite frames and near-optimal quantization of random measurements. Constructive Approximation, 44(1):1 22, E. Chou, C. S. Güntürk, F. Krahmer, R. Saab, and Ö. Yılmaz. Noise-shaping quantization methods for frame-based and compressive sampling systems. Number Springer, I. Daubechies and R. DeVore. Approximating a bandlimited function using very coarsely quantized data: A family of stable sigma-delta modulators of arbitrary order. Annals of mathematics, 158(2): , / 37

37 Reference Reference II I. Daubechies, R. A. DeVore, C. S. Gunturk, and V. A. Vaishampayan. A/d conversion with imperfect quantizers. IEEE Transactions on Information Theory, 52(3): , C. S. Güntürk. One-bit sigma-delta quantization with exponential accuracy. Communications on Pure and Applied Mathematics, 56(11): , / 37

Finite Frame Quantization

Finite Frame Quantization Liam Fowl University of Maryland August 21, 2018 1 / 38 Overview 1 Motivation 2 Background 3 PCM 4 First order Σ quantization 5 Higher order Σ quantization 6 Alternative Dual