ABSTRACT NUMBER THEORETIC ASPECTS OF REFINING QUANTIZATION ERROR. Aram Tangboondouangjit Doctor of Philosophy, Department of Mathematics

Transcription

1 ABSTRACT Title of dissertation: SIGMA-DELTA QUATIZATIO: UMBER THEORETIC ASPECTS OF REFIIG QUATIZATIO ERROR Aram Tangboondouangjit Doctor of Philosophy, 006 Dissertation directed by: Professor John J. Benedetto Department of Mathematics The linear reconstruction phase of analog-to-digital (A/D) conversion in signal processing is analyzed in quantizing finite frame expansions for R d. The specific setting is a K-level first order Sigma-Delta (Σ ) quantization with step size δ. Based on basic analysis, the d-dimensional Euclidean -norm of quantization error of Σ quantization with input of elements in R d decays like O(/) as the frame size approaches infinity; while the L norm of quantization error of Σ quantization with input of bandlimited functions decays like O(T ) as the sampling ratio T approaches zero. It has been, however, observed via numerical simulation that, with input of bandlimited functions, the mean square error norm of quantization error seems to decay like O(T 3/ ) as T approaches zero. Since the frame size can be taken to correspond to the reciprocal of the sampling ratio T, this belief suggests that the corresponding behavior of quantization error, namely O(/ 3/ ), holds in the setting of finite frame expansions in R d as well. A number theoretic technique involving uniform distribution of sequences of real numbers and approxi-

2 mation of exponential sums is introduced to derive a better quantization error than O(/),. This estimate is signal dependent.

3 SIGMA-DELTA QUATIZATIO: UMBER THEORETIC ASPECTS OF REFIIG QUATIZATIO ERROR by Aram Tangboondouangjit Dissertation submitted to the Faculty of the Graduate School of the University of Maryland, College Park in partial fulfillment of the requirements for the degree of Doctor of Philosophy 006 Advisory Commmittee: Professor John J. Benedetto, Chair/Advisor Professor Rama Chellappa Professor Rebecca A. Herb Professor Raymond L. Johnson Professor Lawrence C. Washington

4 c Copyright by Aram Tangboondouangjit 006

5 TABLE OF COTETS Introduction. Quantization of signals Overview of the thesis and main results ew results Definitions and notation Frame Theory 9. Overview Bessel sequences Frames in Hilbert spaces Harmonic frames for R d Sigma-Delta (Σ ) Quantization Overview Pulse Code Modulation (PCM) Sigma-Delta (Σ ) quantization Uniform Distribution and Discrepancy Uniform distribution mod The Weyl criterion Approximation of exponential sums Discrepancy umber Theoretic Approximation Theorem 8 5. Statement of the main theorem Güntürk s theorem Proof of the main theorem Examples Bibliography 05 ii

6 Chapter Introduction. Quantization of signals In signal processing, transmitted signals in analog form need to be converted into digital form for storing, coding, and recovering purposes. This process of analogto-digital (A/D) conversion consists of two main steps: sampling and quantization. In the sampling step, a given signal x is expressed as a linear combination over an at most countable dictionary {e n } n Λ with real or complex coefficients, i.e., x = n Λ x ne n (x n C or R). The expansion is said to be redundant if the choice of the coefficient sequence {x n } n Λ is not unique. We shall refer to a coefficient sequence {x n } n Λ as a sampling sequence. In order to be able to process the signal, one needs to reduce the continuous range of the sampling sequence consisting of real or complex numbers to a finite set. This step of signal processing is called quantization. More precisely, quantization is a mapping process with a map Q such that Q : x x = n Λ q ne n, where, for each n Λ, q n is an element from a finite set A called the quantization alphabet. The map Q is naturally called a quantizer. We see that Q replaces the sampling sequence {x n } n Λ with {q n } n Λ in a linear manner; so we refer to this manner of mapping as linear reconstruction. The natural question arises: how different is the new expansion x = n Λ q ne n from the signal x? This difference occurring in the quantization step is called quantization error, and it

7 is measured by computing x x, where is a suitable norm in the space of signals. An optimal quantizer is the one that minimizes the quantization error norm. evertheless, finding a good quantizer has been proved to be a nontrivial, yet challenging, problem to the engineering community involved in signal processing. For reasons of applicability, an audio signal f of interest is usually modelled as a bandlimited function. This means that f is an L function on R whose Fourier transform f (as a distribution) is compactly supported. For each 0 < T <, the function f can be reconstructed from the sampling sequence {f(nt )} n Z as follows: f(t) = T n Z f(nt )g(t nt ), (..) where g is an appropriate smoothing kernel or sampling function. Applying a first order Σ scheme on f yields a function f T such that f T (t) = T n Z q T n g(t nt ), (..) where each q T n {, }. Standard analysis (see, e.g., [3],[7]) has shown that for some absolute constant C > 0, f f T L CT. However, numerical experiments suggest a better bound than T. More precisely, it has been conjectured that there exists an absolute constant C > 0, independent of f, such that lim K K t K f(t) f T (t) dt CT 3. (..3) This means the approximation error decays on average like T 3/ [7]. We shall see later that the basic bound T corresponds to the basic bound / in Euclidean norm

8 in the setting of finite frames for R d where is the frame size. This correspondence suggests that there should be a better bound for the setting of finite frames as well. We shall assume that the signal of interest is an element of the Euclidean space R d, and that the sampling coefficients are real numbers. We shall also focus on structured dictionaries called frames.. Overview of the thesis and main results We begin Chapter by discussing material on frame theory. We discuss the definition of frames in Hilbert spaces and prove some properties of frames in this setting. Then we focus on finite frames for Euclidean space R d. Some interesting results dealing with finite unit norm tight frames are analyzed, based on the works by Benedetto and Fickus [4] and by Zimmermann [0]. We pay attention to a specific infinite family of frames called the harmonic frames. This family of frames provides substantive structure, and it is used in Chapter 5 to provide examples to illustrate the results on quantization error. The notion of the first order frame variation, σ(f, p), is introduced, and it is generalized to define the nth order frame variation, σ n (F, p). We derive a general formula of σ n (F, p) for harmonic frames. We shall see that frame variation plays an important role in the basic quantization error as it relates the dependency of the Σ scheme with the properties of frames. In Chapter 3, we discuss a classic quantization scheme called Pulse Code Modulation (PCM) and derive quantization error estimate associated to this scheme for finite frames for R d. We then provide the setting of this thesis, viz., the first order 3

9 K-level Σ scheme with step size δ. The quantizer map is defined algorithmically. However, this makes it inconvenient to program numerical experiments using MAT- LAB so we derive the general formula for this quantizer. Then we derive a basic quantization error estimate based on the Σ scheme. This is done in [6], where it is proved that if F is a unit norm tight frame for R d of cardinality d, then the K-level Σ scheme with quantization step size δ gives quantization error x x δd (σ(f, p) + ), where x is a given signal, x is the quantized signal, and is the d-dimensional Euclidean -norm. In Chapter 4, we first provide the background material from the theory of uniform distribution of sequences of real numbers [7]. In particular, we define uniform distribution modulo, and state examples of real sequences with this property. We then discuss the notion of discrepancy of a finite sequence and prove some basic results on the bound of discrepancy. We provide two inequalities that improve the bound of discrepancy and emphasize one of them, viz., the Erdös-Turán Inequality which states the following: For any finite sequence x,..., x of real numbers and any positive integer m, we have D 6 m π m h= ( h ) m + e πihxn, where D is the discrepancy of the sequence x,..., x. This inequality plays an important role in our analysis of quantization error as it approximates discrepancy in terms of an exponential sum which will be approximated further by a theorem of van der Corput. This latter theorem states the following: If a and b are integers 4

10 with a < b, and if f is a twice differentiable function on [a, b] with f (x) ρ > 0 for all x [a, b] or f (x) ρ < 0 for all x [a, b], then b n=a ( 4 ) e πif(n) ( f (b) f (a) + ) ρ + 3. In Chapter 5, we collect all the ingredients to give the detailed proof of the theorem on improving the quantization error stated in [6]. We provide a new construction which corrects errors observed in the original proof. One ingredient we need in the proof is a result by Güntürk [7, 8]. This theorem allows us to construct an analytic function with certain properties such that the values at the natural numbers correspond to the terms of a given real sequence. We prove the special case of this theorem, and give an explicit bound for the inequality not given in the original theorem. The quantization error obtained in this chapter is an improvement from the basic error estimate obtained in Chapter 3. In fact, our improvement goes from order / to one of order / 5/4 ɛ where denotes the cardinality of frame. (ɛ > 0) for certain choices of frames, We show further that with a certain natural assumption, the order of the quantization error estimate can be improved to / 4/3. This order is better than the order obtained by Güntürk in [7] in the setting of bandlimited functions. There he obtained a bound of order / 4/3 ɛ for ɛ > 0. On the other hand, the bounds we obtain for these improved estimates depend on the given signals. One of the main goals of our future research is to dispense with this restriction. The last section of Chapter 5 is devoted to examples to justify the results of the theorems we have proved. We show various graphs of quantization error norms 5

11 which are plotted against the cardinality of the frames. We analyze some interesting phenomena concerning the periodic pattern occurring in the shapes of these graphs..3 ew results In this section we specifically describe our own contributions. In Chapter, we generalize the notion of the first order frame variation σ(f, p ) to the nth order frame variation σ n (F, p ), where F is a given frame and p is a permutation of the set {,..., }. We then prove the explicit formulae of σ n (H d, p) for the harmonic frame Hd with respect to the identity permutation p (Theorem.4.5). Such formulae can be used in refined quantization error estimates. We also prove a result (Theorem.4.6), which is a consequence of the proof of Theorem.4.5, which gives relatively sharp inequalities for some new trigonometric binomial sums. In Theorem 3.3. of Chapter 3, we prove the general formula of the quantizer associated with first order Σ quantization. Theorem 3.3. is crucial in programming numerical experiments using MATLAB. In Chapters and 4, we give details for difficult issues concerning frames, uniform distribution, and discrepancy, which are not readily available in the literature. For example see Proposition.3.6, Examples 4.., 4..8, 4.4.9, Theorem In particular, we proved in Example 4.. that the following 6

12 sequence is u.d. mod : 0, 0,, 0 3, 3, 3,..., 0 n, n,..., n n,.... In Chapter 5, noting that there was a gap in the original proof of Güntürk s theorem in [7] we provided a complete proof for an important special case. Independently, Güntürk has given a complete proof in [8] and in a private communication, the latter after seeing our work. We also compute an explicit bound of the inequality occurring in the theorem, which will be useful in evaluating quantization error independent of signal. We also correct the proof of Theorem 5.. from the original one by providing a new intricate construction. Finally, we have constructed a new class of examples of quantization error plots, showing and giving preliminary analysis of various periodic patterns of the shape of graphs of the quantization error as a function of the frame size..4 Definitions and notation We shall use the following definitions and notation. The Fourier transform is formally defined by f(γ) = f(t)e πitγ dt. We denote the characteristic function of a set E by E, i.e., if x E, E (x) = 0 otherwise. 7

13 For x R, we denote by x the floor function of x, which is the largest integer that is not greater than x; and we denote by {x} the fractional part x x [0, ) of x. We also denote by x the ceiling function of x, which is the smallest integer that is at least x. 8

14 Chapter Frame Theory. Overview The necessary condition for a sequence {e n } of unit norm vectors to be an orthonormal basis (OB) for a Hilbert space H is that it satisfies Parseval s equation, that is, x, e n = x for all x H. (..) A relaxation of this condition (specified later) leads to a generalization of the notion of OB, namely frames. If a sequence {e n } of vectors is a frame for a Hilbert space H then it spans H and yet it is not necessarily linearly independent. In other words, in the case of frames, for each x H, there exists a sequence {x n } of real or complex numbers such that x = x n e n. (..) Because the frame elements are allowed to be linearly dependent, the coefficients {x n } are not necessarily unique. We usually referred to this property as the redundancy of frames and it is one of the main reasons why frames have been extensively used in signal processing. The notion of frames was introduced by Duffin and Schaeffer in their 95 paper [8]. The main subject of their study is nonharmonic Fourier series, i.e., sequences of the type {e iλnx } n Z, where {λ n } n Z is 9

15 a family of real or complex numbers satisfying a uniform density condition. However, the potential of frames was not realized until 34 years later during the era of wavelet theory, in a paper by Daubechies, Grossman and Meyer [9] (986). Using frames, they expanded functions f L (R) in a similar manner as using orthonormal bases. The mathematical framework of signal processing was set rigorously by assuming the signal of interest referred by the authors as incoming information to be an element of a Hilbert space H, particularly of H = L (R). Parts of the following materials on frames in Hilbert spaces are adapted from Chapters 3 and 5 of Christensen s book [].. Bessel sequences Definition... A sequence {e n } in H is said to be a Bessel sequence if there exists a constant B > 0 such that x H, x, e n B x. (..) A number B satisfying condition (..) is called a Bessel bound for {e n }. Lemma... Let {e n } be a Bessel sequence in a Hilbert space H. Define the associated Bessel map L : H l () by x { x, e n }. Then L is a bounded (continuous) linear operator. Moreover, the corresponding adjoint operator L : l () H is given by {a n } a n e n. 0

16 Proof. We see that L is well defined since {e n } is a Bessel sequence. Let B be a Bessel bound for the sequence {e n }. Then for each x H, Lx = x, e n B x. So Lx B x. This shows that L is bounded. Let c l () and define S = c[n]e n for each. Then for all integers, M with > M, S S M = sup x = sup x = B n=m+ ( n=m+ n=m+ c[n]. c[n] e n, x c[n] )( n=m+ e n, x ) The first equality is an equivalent way of expressing the norm in a Hilbert space, see Remark..3. The second inequality follows from the Hölder Inequality. ow, since { } n c l (), it follows that the sequence k= c[k] is Cauchy. We therefore see from the above calculation that the sequence {S n } is Cauchy, and hence converges in H. To find the formula for the adjoint operator L, we let c l (), and let x H. Then x, L c = Lx, c = = (Lx)[n]c[n] = x, e n c[n] x, c[n]e n = x, c[n]e n. (..) The last equality follows from the continuity of the inner product, see Remark..3. Since this is true for all x H, it follows from the Hahn-Banach Theorem that L c = c[n]e n.

17 Remark..3. In the proof of Lemma.., we have used an equivalent way of expressing the norm in a Hilbert space. This can be shown as follows. Let y H. Then the map ψ y defined by ψ y x = x, y is a bounded linear operator. From Cauchy-Schwarz Inequality we have ψ y x = x, y y x. Since the equality holds if and only if x = ay for some scalar a, we see that ψ y = y. Since ψ y = sup x = ψ y x = sup x = x, y, it follows that y = sup x = x, y. The fact that ψ y is bounded, and therefore continuous, allows the final equality in (..). We note that the proof of Lemma.. remains the same if the order of the sequence {e n } has been changed. Hence we have the following corollary. Corollary..4. If {e n } is a Bessel sequence in H, then c[n]e n converges unconditionally for all c l (). By Corollary..4 we see that it does not matter what index set we use to index the series c[n]e n since each reordering of the sequence {c[n]e n } will have the series converge to the same element. Hence we can use natural numbers as the standard index set..3 Frames in Hilbert spaces We are now in a position to state the definition of frames. Definition.3.. A sequence {e n } of elements in a Hilbert space H is said to

18 be a frame for H if there exist constants 0 < A B < such that A x x, e n B x for all x H. (.3.) The numbers A and B are called frame bounds. The optimal upper frame bound is the infimum over all upper frame bounds and the optimal lower frame bound is the supremum over all lower frame bounds. We note that the optimal bounds are actually frame bounds. A frame is said to be A-tight if A = B, and is said to be unit norm if e n = for all n. A frame is said to be exact if it ceases to be a frame when one of the elements is removed from the sequence {e n }. Some basic examples of frames are as follows: Example.3.. Let {e n } be an orthonormal basis for H. (i) By repeating each element in {e n } twice, we obtain {f n } = {e, e, e, e,... } which is a -tight frame. In fact, for each x H we have x, f n = x, e n + x, e n = x + x = x. (ii) By repeating only e, we obtain {f n } = {e, e, e, e 3,... } which is a frame with frame bounds A = and B =. In fact, for each x H 3

19 we have x = x, e n x, e + x, e n = x, f n x, e n + x, e n = x + x = x. (iii) Let {f n } = {e, e, e, 3 e 3, 3 e 3, } e 3, This is the sequence where each vector n e n is repeated n times. As such, it is a -tight frame. In fact, for each x H we have x, f n = n x, n e n = x. (iv) Let v = (, 0), v = ( / 5, ), v 3 = (4/ 5, ). By a direct computation, one can show that {v, v, v 3 } is a 5-tight frame for R. In fact, letting v = (a, b) be a vector in R, we have 3 v, v n = a + ( 5 a + b) + ( 4 5 a + b) = 5(a + b ) = 5 v. Let {e n } be a frame for a Hilbert space H. We define an operator S : H H by Sx = L Lx = x, e n e n. We see that since {e n } is a Bessel sequence, Corollary..4 implies that S is a well-defined operator. The operator S is called the frame operator for {e n }. We prove some properties of the frame operator S in the following lemma. 4

20 Lemma.3.3. Let {e n } be a frame with frame bounds A, B. Then the following hold: (i) The frame operator S is bounded, invertible, self-adjoint, and positive. (ii) The sequence {S e n } is a frame with bounds B and A ; if A, B are the optimal bounds for {e n }, then the bounds B, A are optimal for {S e n }. The frame operator for {S e n } is S. The sequence {S e n } is called the (canonical) dual frame of {e n}. Before proving the lemma, we state two classical results from operator theory. Lemma.3.4 (eumann Theorem). Let X be a Banach space and let U : X X be a bounded operator. If I U <, then U is invertible with U = (I U) n. Furthermore, U ( I U ). Lemma.3.5. Let H be a Hilbert space and let U j : H H (j =,, 3) be selfadjoint operators with U 3 0. If U U and U 3 commutes with U and U, then U U 3 U U 3. (By definition, two self-adjoint operators U W if Ux, x W x, x for all x H.) We are now ready to prove Lemma.3.3. Proof of Lemma.3.3. (i) Since L and L are bounded operator, the frame operator S being the composition of these two operators is also bounded. ow since S = (L L) = L (L ) = L L = S, the operator S is self-adjoint. By direct calculation 5

21 we see that for each x H, Sx, x = x, e n. So we can rewrite the frame condition (.3.) in terms of S as AI S BI (.3.) This shows that for each x H, Sx, x A x 0. So S is positive. By subtracting BI and multiplying by B through the inequality (.3.), we obtain that 0 I B S B A I. Therefore B I B S = sup (I B S)x, x B A x = B <, which, by Lemma.3.4, shows that S is invertible. (ii) We first show that {S e n } is a Bessel sequence. Indeed, for each x H, x, S e n = S x, e n B S x B S x. Hence the frame operator for {S e n } is well defined. This frame operator acts on x H by x, S e n S e n = S S x, e n e n = S SS x = S x. This shows that the frame operator of {S e n } is S. ow since the operator S commutes with both S and I, we can apply Lemma.3.5 and obtain that, upon multiplying the inequality (.3.) with S, B I S A I. This means for all x H, B x S x, x = x, S e n A x. 6

22 Thus {S e n } is a frame for H with frame bounds B and A. ow suppose that A, B are optimal bounds for the frame {e n }. Let C be the optimal upper bound for the frame {S e n } and assume that C < /A. Then since S is the frame operator for {S e n } it follows that the frame {(S ) S e n } = {e n } has lower bound /C > A. This is a contradiction since A is the optimal lower bound for {e n }. Hence C = /A. We can show similarly that the optimal lower bound for {S e n } is /B. Proposition.3.6. Let {e n } be a frame for a Hilbert space H with frame bounds A, B and with frame operator S. Then the following inequalities hold: A x Sx B x for all x H. Proof. Let x H. We shall prove the leftmost inequality first. By definition of operator S, we have Sx, x = x, e n. It follows from the Cauchy-Schwarz Inequality and the frame condition (.3.) that ( This implies x, e n ) = Sx, x Sx x Sx A A x, e n Sx. x, e n. From the frame condition (.3.) we have x, e n A x, and so A x Sx. Hence the leftmost inequality follows. ow we show the rightmost inequality. Let c l () and recall that L c = c[n]e n. By an equivalent definition of norm in 7

23 a Hilbert space, see Remark..3 and the Hölder Inequality, it follows that L c = sup L c, y = sup y = y = c[n]e n, y = sup c[n] e n, y sup c[n] e n, y y = ( B c. c[n] ) / sup y = y = ( ) e n, y / Hence L B. ow since S = L L it follows from a property of the adjoint operator that S = L L = L B. Thus Sx S x B x, which is the rightmost inequality and hence the proof is complete. ow we arrive at the main elementary theorem in frame theory. All applications of frames start with this so-called frame decomposition which shows that every element in a Hilbert space can be represented as an infinite linear combination of the frame elements. Theorem.3.7 (Frame Decomposition). Let {e n } be a frame for a Hilbert space H with corresponding frame operator S. Then x = x, S e n e n = x, e n S e n for all x H. (.3.3) Both of the series converge unconditionally for all x H. 8

24 Proof. Let x H. Then we have from properties of the frame operator in Lemma.3.3 that x = SS x = S x, e n e n = x, S e n e n. The last equality follows from the fact that S is self-adjoint. ow since {e n } is a Bessel sequence and { x, S e n } l (), it follows from Corollary..4 that the series converges unconditionally. Similarly, by composing S with S we have another way to represent the element x, that is, x = S Sx = Sx, S e n S e n = x, SS e n S e n = x, e n S e n. The penultimate equality follows from the fact that S is self-adjoint. Since {S e n } is a Bessel sequence and { x, e n } l (), it follows from Corollary..4 that the series converges unconditionally. Hence the proof is complete..4 Harmonic frames for R d The atomic decompositions in (.3.3) are the first step towards a digital representation. If the frame is tight with frame bound A, then from (.3.) we have the frame operator S = AI, and therefore we see that both of the frame expansions in (.3.3) are equivalent, i.e., for each x H, x = A x, e n e n. For convenience, we let K = R or K = C. When the Hilbert space H is K d and the cardinality of frame is finite, the frame is referred to as a finite frame for H. In this case, there is a systematic method to check whether an arbitrary finite set 9

25 of vectors is a tight frame. Let {v n } be a set of vectors in Kd We define the associated matrix L to be the d matrix whose rows are the v n. The following lemma, found in [0], allows us to determine whether {v n } forms a tight frame for K d. Lemma.4.. A set of vectors {v n } in Kd is a tight frame with frame bound A if and only if its associated matrix L satisfies L L = AI d, where L is the conjugate transpose of L, and I d is the d d identity matrix. Moreover the frame {v n } is unit norm if and only if the diagonal of LL equals (,...,). Proof. Let x = (x,..., x d ) K d. Then by a straightforward calculation, we obtain Lx = ( x, v,..., x, v ) (.4.) From (.4.) we obtain x, v n = (Lx) (Lx) = x (L L)x. (.4.) A set {v n } is an A-tight frame for Kd if and only if x, v n = A x = x (AI d )x for all x K d. From (.4.) this is true if and only if x (AI d )x = x (L L)x for all x K d ; and this in turn is true if and only if AI d = L L. To prove the second part we observe that the frame {v n } is unit norm if and only if = v n = v n, v n = d j= v n(j)v n (j) for each n. We see that the last sum is exactly the nth diagonal element of the matrix LL for each n. Hence the result follows. 0

26 The following lemma determines the frame bound for a finite unit norm tight frame in K d. The first proof can be found in [0] where the author uses matrix properties, and the second in [4] where the authors use the definition of an orthonormal basis. Lemma.4.. A unit norm tight frame for K d with elements has frame bound A = /d. First proof. We denote the trace of a matrix M by Tr(M). It is straightforward to show that Tr(M) = Tr(M) for all matrices M, that can be multiplied. Using this property and Lemma.4., we have A = d Tr(L L) = d Tr(LL ) = d. Second proof. Let {v n } be a unit norm tight frame for Kd with frame bound A. Let {e j } d j= be an orthonormal basis for Kd. Then Ad = d A e n = j= d d e j, v n = e j, v n = v n =. j= j= ow we introduce harmonic frames for R d. This family of frames has a Fourierbased structure, and it provides good examples that we shall use later in Chapter 5. The definition of the harmonic frame H d = {e n} n=0, > d, depends on whether the dimension d is even or odd. If d is even, let e n = [ d cos πn, sin πn, cos πn, sin πn,..., cos π dn, sin π dn ] (.4.3) for n = 0,,...,.

27 If d > is odd, let e n = [ d, cos πn for n = 0,,...,., sin πn, cos πn, sin πn d π,..., cos n, sin π d n ] (.4.4) We shall now show that H d, as defined above, is a unit norm tight frame for R d. From the identity cos θ + sin θ =, it follows immediately that e n is unit norm for each 0 n. To verify that H d is a tight frame, we have options either to apply Lemma.4. or to verify the definition directly. In this case it turns out that the latter option is easier. We verify only the case when d is even. The case when d is odd will be similar. So let d be even, let > d, and take x = (a, b,..., a d, b d ) R d. We want to show that We have n=0 x, e n = d x. (.4.5) d x, e n = n=0 = = ( d/ n=0 j= ( d/ n=0 n=0 j= + d/ j= a j cos πnj a j + b j sin ( πnj (a j + b j) sin ( πnj n=0 j k + b j sin πnj ) + φ j + φ j ) ) ) ( πnj a j + b j a k + b k sin + φ j ) ( πnk ) sin + φ k. ow by using the identities sin θ = ( cos θ)/ and sin θ sin ψ = cos(θ ψ) cos(θ + ψ)

28 and interchanging the sums, the right-hand side quantity equals d/ j= (a j + b j) + j k j k n=0 a j + b j ( ( πnj cos a k + b k n=0 + φ j ) ) ( πn ) cos (j k) + φ j φ k a j + b j a k + ( πn ) b k cos (j + k) + φ j + φ k. n=0 By using the identity n=0 ( πnj ) cos + α = 0, which holds for each integer j that is not divisible by and for each α R, the sums above simplify to Since this is equal to d d/ j= (a j + b j) = x. n=0 x, e n, we obtain (.4.5). Definition.4.3. Let k, d, and be integers such that k < and d <. Let F = {e n } be a frame for R d. Let p be a permutation of {,,..., }. We define the variation of order k of the frame F with respect to p as σ k (F, p ) := k k e p (n), where k denotes the kth order difference defined recursively by e p (n) = e p (n) e p (n+) and k e p (n) = ( k e p (n)) for all k. Frame variation is the quantity that reflects the interdependencies among frame elements. More precisely, if a frame F has low variation with respect to a permutation p, then the frame elements will not oscillate too much in that ordering 3

29 [5]. We shall see in Chapter 3 that the notion of frame variation plays an important role in refining quantization error. Families of frames that have bounded frame variation will result in a better quantization error. Harmonic frames are an example of such a family of frames. In fact, one can compute the frame variation of harmonic frames explicitly. Lemma.4.4. Let a sequence {e n } of vectors in R be defined by e n = (cos nθ, sin nθ) for some θ [0, π]. We have, for each integer k, k e n = ( sin θ ) k. Proof. By induction one can show that the kth order difference is equivalent to the following: k =,,..., k e n = k ( ) k ( ) j e n+j. (.4.6) j j=0 With another application of induction, one can show that k e n = k j=0 ( ) j ( k k + j ) cos jθ ( ) k. (.4.7) k ow we shall use induction to show that the last step is equal to k ( cos θ) k. It is easy to verify the formula for k =. Assume the formula holds for some k >, i.e., k j=0 ( ) j ( k k + j ) cos jθ ( ) k = k ( cos θ) k. (.4.8) k We want to show that the formula holds for k +, i.e., k+ j=0 ( ) j ( k + k + + j ) cos jθ ( ) k + = k+ ( cos θ) k+. (.4.9) k + 4

30 The left side of (.4.9) is k+ [( ) ( )] ( ) ( ) k + k + k + k + ( ) j + cos jθ k + j k + j + k + k j=0 k+ [( ) ( ) ( ) ( )] k k k k = ( ) j cos jθ k + j k + j k + j + k + j j=0 ( ) ( ) ( ) ( ) k k k k k k + k k k+ ( ) ( ) k k k+ ( ) k = ( ) j cos jθ + ( ) j cos jθ k + j k k + j j=0 j=0 k+ ( ) ( ) ( ) k k k + ( ) j cos jθ k + j + k + k j=0 k ( ) k = k+ ( cos θ) k ( ) j cos(j + )θ k + j j= k+ ( ) ( ) k k ( ) j cos(j )θ k + j k + j= k ( ) k = k+ ( cos θ) k cos θ ( ) j cos jθ k + j + sin θ j=0 j=0 k ( ) k k+ ( ) j sin jθ cos θ k + j k+ sin θ j= ( ) j ( k k + j ) sin jθ j= ( ) k = k+ ( cos θ) k S k cos θ S k cos θ + cos θ k ( k ( ) ) k where S k = ( ) j cos jθ k + j j=0 ( ( )) k = k+ ( cos θ) k cos θ S k k = k+ ( cos θ) k cos θ k ( cos θ) k = k+ ( cos θ) k+ ( ) j ( k k + j ) cos jθ and this is the right side of (.4.9). ow using the identity that cos θ = 5

31 sin (θ/), we have the result. Theorem.4.5. For each integer k, and each integer >, if p is the identity permutation of {,..., } then ( π σ k (H, p) = ( k) k sin k ( ) ( / ( k) σ k (H, d d p) = ( ( k) d ), A(k,, d) d+ ( k k ) ) / ) ( / + A(k,, d ) d k ) ( ) / k if d is even, if d is odd, where A(k,, d) = k j=0 ( ) k ( ) j sin k + j ( (d + )πj ) ( πj ) csc. Proof. From Lemma.4.4 we have σ k (H, p) = k n=0 k e n = k n=0 ( sin ( π ) ) k ( π ) = ( k) k sin k. To prove the second formula for d even, we let e n be defined as in (.4.3). Then from formulae (.4.6) (.4.8), we obtain [ d/ d ( k ( ) k e n k = ( ) j cos(n + j) πl ) ( k ( ) k + ( ) j sin(n + j) πl ) ] j j j=0 j=0 l= d/ ( πl ) d/ = k sin k = k l= l= [ d/ k ( k = ( ) j k + j = l= k j=0 j=0 ( ) j ( k k + j = A(k,, d) ) d/ l= k ( k ( ) j j=0 = A(k,, d) d + ( k k ( πl ) sin k ) cos πjl ( ) ] k k cos πjl d ( ) k k ) d ( ) k k + j k ). (.4.0) 6

32 The last two equalities follow from two well-known summing identities. Combining equation (.4.0) with the definition of σ k (H d, p), we have the result. For the case when d is odd we use the definition of e n defined in (.4.4) and proceed similarly as in the above argument. Also, by using the same method as in the proof of Theorem.4.5, we have the following theorem. Theorem.4.6. Let θ be a real number and let, d be positive integers. Then we have the following: (i) ( ) n+ ( + n ) sin(d + )nθ sin nθ ( d + ) ( ), (ii) ( ) ( nθ ) ( ) ( dθ ) ( ) n+ sin(ndθ) cot d + sin + n ( ) d + (dθ), (iii) (d+) / n=0 ( + n / )+( ) d+ n=0 ( ) + n + = 4 d ( + d+ ) ( ), (iv) (v) / n=0 ( ) = n / n=0 ( ) + n + = 4, ( ), 7

33 (vi) n=0 ( ) = [ n ( )]. Proof. By retracing the steps of proof of Theorem.4.5 beginning from the third equality of (.4.0), we have [ d ( lθ ) d ( ) ( ) ] sin = ( ) n cos(nlθ) + n l= l= n=0 ( ) d ( ) = ( ) n cos(nlθ) d + n n=0 l= ( ) [ = ( ) n sin(d + )nθ ] ( ) d + n sin nθ n=0 ( ) sin(d + = ( ) n )nθ ( ) + (d + ) + n sin nθ ( d + ) ( ) ( ) sin(d + = ( ) n )nθ ( + d + ) ( ). (.4.) + n sin nθ By letting θ = π and noting that we obtain (iii). We observe that d l= πl sin = d, d 4d + d/ d + d + 4d +. Thus, lim d d/ d + = 4. Dividing both sides of (iii) by d + and letting d we obtain (iv). To obtain (v), we substitute (iv) back in (iii) and solve for (v). The equality (vi) is obtained 8

34 by adding (iv) and (v). Inequality (i) follows by noting that the left-hand side of (.4.) is nonnegative. We obtain inequality (ii) by expanding and using.4.8. ( sin d + ) nθ = sin(dnθ) cos nθ nθ + cos(ndθ) sin, Remark.4.7. (a) We remark that the inequalities (i) and (ii) in Theorem.4.6 are quite sharp for certain choices of θ, d, and. For example, for θ = π/7, d =, and = 6, the left side of inequality (i) is about , while the right side is 386. With the same values of θ, d, and the left side of inequality (ii) is about , while the right side is about (b) The combinatoric sum identities (iii)-(vi) also have a direct proof using some properties of binomial coefficients. 9

35 Chapter 3 Sigma-Delta (Σ ) Quantization 3. Overview In this chapter, we shall discuss two schemes of quantization. The first one, called Pulse Code Modulation or PCM, is considered perhaps the most basic scheme of quantization. This scheme quantizes a signal of interest by replacing each coefficient of the signal expansion with the element of a given discrete set (alphabet) that is closest in distance to the coefficient. We shall discuss the PCM of finite frame expansions of signals in R d and shall derive a quantization error associated to this technique [6]. The second scheme of quantization, called Sigma-Delta (Σ ) quantization, was introduced by Inose, Yasuda and Murakami in 96 [3]. This scheme is widely used in quantizing signals because of its robustness against circuit imperfections, and it can provide high accuracy A/D conversion [3, 9, 5, 6]. We shall see that this scheme uses feedback loops in the sense that the elements of a quantized sequence keep being fed back into the scheme to produce new quantized coefficients. This exploitation of feedback loops generates a quantized signal that oscillates between levels, keeping its average equal to the average input [4]. We shall use basic analysis to derive the quantization error associated to the Σ quantization. As such, we shall see that, in the setting of redundant signal expansions, this quantization scheme outperforms PCM with respect to faster decay in quan- 30

36 tization error. However, if the signal is expanded over an orthonormal basis, then PCM turns out to be the optimal quantizer since it minimizes the Euclidean norm of quantization error. More precisely, let x be a signal of interest in R d, and let {e n } be an orthonormal basis for Rd. (It follows necessarily that d =.) Then there exist unique c,..., c R such that x = c n e n. Let q,..., q be the quantized coefficients obtained from PCM algorithms and let x = q ne n. Then by a property of orthonomal bases, we have x x = (c n q n ). Since PCM determines q n, an element of the given alphabet, in such a way that c n q n is the minimum for each n, we see that x x is the minimum as well. 3. Pulse Code Modulation (PCM) Let {e n } be a unit norm tight frame for Rd. Then from Chapter, we have the expansion for each x R d by x = d x n e n, x n = x, e n. (3..) Definition 3... Let δ > 0. The /δ -level PCM quantizer with step size δ 3

37 replaces each x n R in the frame expansion (3..) with δ ( x n /δ / ) if x n <, q n = q n (x) = δ ( /δ / ) if x n, (3..) δ ( /δ / ) if x n. Proposition 3... Let δ > 0, and let be the d-dimensional Euclidean -norm. Let x R d and let x be the quantized expansion given by /δ -level PCM. If x then the quantization error x x satisfies x x ( d ) δ. Proof. First we write x = d q n e n, (3..3) where q n is obtained from PCM for each n. Then, from the Cauchy-Schwarz Inequality, we have for each n, that x n = x, e n x e n. For each n we have x n /δ x n /δ < x n /δ +, so that δ ( = x xn ) n δ δ + + δ < x n q n = x n δ Hence, for each n, xn δ + δ x n δ xn δ + δ = δ. x n q n δ. (3..4) From (3..3) and (3..4) we have x x = d ( δ )( d ) (x n q n )e n e n = Thus, the proof is complete. 3 ( d ) δ.

38 3.3 Sigma-Delta (Σ ) quantization When first introduced, Sigma-Delta (Σ ) Quantization was used to quantize oversampled bandlimited functions; so, before we define the definition of this quantization scheme in the setting of finite frame expansion, we should understand the definition in its original setting [9]. Let f be a bandlimited function on R with bandwidth Ω > 0 and assume that f takes value in the interval [, ]. We recall from Chapter that this means that f is an L function on R whose Fourier transform f (as a distribution) vanishes outside [ Ω, Ω]. Then from the classical sampling theorem [3, 7], for each 0 < T <, the function f can be reconstructed from the sampling sequence {f(nt )} n Z as follows: f(t) = T n Z f(nt )g(t nt ), (3.3.) where g is an appropriate smoothing kernel or sampling function, that is, ĝ C and for ξ Ω, ĝ(ξ) = 0 for ξ > Ω/T. Then the first order Σ modulator uses {f(nt )} n Z as inputs to generate the se- 33

39 quence {q T (n)} n Z as follows: n k= f(kt ) for n, F T (n) = 0 for n = 0, (3.3.) 0 k=n+ f(kt ) for n < 0, Q T (n) = F T (n), (3.3.3) q T (n) = Q T (n) Q T (n ), (3.3.4) From (3.3.) we see that F T (n) F T (n ) = f(nt ) for all n Z. Since f takes value in the interval [, ] we can check that q T (n) takes either the value or. In fact, for each n Z, we have F T (n) < F T (n) F T (n) and similarly F T (n ) F T (n ) < F T (n ) +. Adding these inequalities yields 0 f(nt ) = F T (n) F T (n ) < q T (n) < F T (n) F T (n ) + = f(nt ) + 3. Since q T (n) is an integer, from the above chain of inequalities we have either q T (n) = or q T (n) =. We note that the equations (3.3.) and (3.3.4) correspond to Σ and, respectively; hence the name of the modulator. Since F T and Q T will accumulate into large numbers as time elapses, neither can be calculated in a circuit. Thus one introduces the auxiliary variable u T = F T Q T = {F T } [0, ). Then u T satisfies the recursive relation: u T (n) u T (n ) = F (nt ) q T (n). (3.3.5) Since u T (n) [0, ), from (3.3.5) we have the relation q T (n) = f(nt ) + u T (n ). (3.3.6) 34

40 To see this, we observe from (3.3.5) that q T (n)+u T (n) = f(nt )+u T (n ), so that q T (n) + u T (n) = q T (n) + u T (n) = f(nt ) + u T (n ). Since u T (n) [0, ), it follows that u T (n) = 0 and hence (3.3.6) follows. Using the auxiliary variable u T, now we can translate the procedure in (3.3.) (3.3.4) into the following equivalent procedure: u T (n) = u T (n ) + F (nt ) q T (n) with u T (0) = 0, (3.3.7) if f(nt ) + u T (n ) <, q T (n) = (3.3.8) if f(nt ) + u T (n ). Formulae (3.3.7) and (3.3.8) motivate the definition of the Σ quantization for finite frame expansions in R d. Let K and δ > 0. We define the midrise quantization alphabet A δ K to be the set of K numbers in arithmetic progression with common difference δ, and the first number ( K + /)δ. Thus, A δ K = {( K + /)δ, ( K + 3/)δ,..., ( /)δ, (/)δ,..., (K /)δ}. We define the K-level midrise uniform scalar quantizer with stepsize δ by Q(u) = arg min u q. q A δ K In words, Q(u) denotes the element q in A δ K which is closest in distance to the element u. By convention, if there are two elements in A δ K which are equally closest to u, then Q(u) will be chosen to be the larger of these two elements. In order to be able to do numerical simulation with Σ quantization, we need an explicit formula for the quantizer Q. The following theorem will let us do just that. 35

41 Theorem The quantizer Q as defined above has the following formula: sgn(u) ( K ) δ if u Kδ, Q(u) = ( + (3.3.9) ) u δ if u < Kδ. δ Proof. Let a real number u be given. If u Kδ then the formula is easily verified. If (K )δ u < Kδ then by definition Q(u) = (K /)δ. Since K u δ < K, we have u/δ = K. So ( u ) ( ) ( + δ = δ + K δ = K ) δ = Q(u). Similarly for the case ( Kδ < u K + ) δ. ow assume that ( K + ) ( δ < u < K ) δ and that for some m = 0,..., K ( Q(u) = K + ) δ + mδ. Then by definition of Q(u) we see that δ u Q(u) < δ. 36

42 Replacing Q(u) with ( K + /)δ + mδ, we have 0 u δ + K m <. Hence u δ + K m = 0. ow using the property that x + n = x + n for all integers n and real numbers x, we obtain that m = u/δ + K. Replacing this value of m in Q(u), we get ( Q(u) = K + ) ( u ) ( u ) δ + + K δ = δ + δ. δ A δ y(u) = u 5 9 δ Q(u ) u δ δ u Q(u ) 9 δ u Figure 3.: Picture diagram of the quantizer Q. Definition Let {x n } R, and let p be a permutation of the set {,,..., }. Then the K level first-order Σ quantizer with step size δ is defined recursively by u n = u n + x p(n) q n, (3.3.0) q n = Q(u n + x p(n) ), (3.3.) 37

43 where u 0 is a specified constant. We usually refer to this definition as the first-order Σ quantizer for short. We see that Σ quantizer produces two sequences: {u n } n=0 and {q n}. We shall refer to {q n } as the quantized sequence and refer to {u n} n=0 as the auxiliary sequence of state variables. We shall refer to the permutation p as the quantization order. The following proposition shows that the Σ quantizer defined above is stable, that is, the auxiliary sequence {u n } n=0 is uniformly bounded provided that the input sequence {x n } is appropriately uniformly bounded. Proposition Let K be a positive integer, let δ > 0, and consider the Σ quantizer defined by (3.3.0)-(3.3.). If u 0 δ/ and for all integers n, x n ( K ) δ, then for all integers 0 n, u n δ. Proof. We may assume without loss of generality that p is the identity permutation. We shall proceed by induction. The base step, u 0 δ/, holds by assumption. ext, we suppose that u j δ/ for some j. We want to show that u j δ/. We have u j + x j u j + x j Kδ. We also note from the definition of Q that if u Kδ, then 0 Q(u) u δ/. Combining this with (3.3.0)-(3.3.), we have u j = (u j + x j ) Q(u j + x j ) δ. 38

44 The following Theorem is one of the main results found in [5]. It states the basic quantization error estimate associated to the first-order Σ quantization. We begin with the setup. Let K and δ > 0. Let F = {e n } be a frame for Rd, let p be a permutation of {,..., }, and let x R d be the input. We form the quantized expansion x = q n S e p(n) (3.3.) from the frame expansion x = x p(n) S e p(n), x p(n) = x, e p(n). (3.3.3) Here, {q n } is the quantized sequence which is calculated using the recurrence relations (3.3.0)-(3.3.). We want to calculate how well (3.3.) approximates (3.3.3). Theorem Given the Σ quantization as above. Let F = {e n } be a finite unit norm frame for R d, p a permutation of {,,..., }, u 0 δ/. If x R d satisfies x (K /)δ, then we have the following quantization error: ( x x S σ(f, p) δ ) + u + u 0, where S is the inverse frame operator for F and σ(f, p), the frame variation with respect to p, is defined by (see Chapter, Section.4) σ(f, p) = e p(n) e p(n+). 39

45 Proof. x x = = = (x p(n) q n )S e p(n) (u n u n )S e p(n) u n S (e p(n) e p(n+) ) + u S e p() u 0 S e p(). Since x (K /)δ it follows from Cauchy-Schwarz Inequality that n, x n = x, e n x e n (K /)δ. Thus, combining with the stability result of the sequence {u n } n=0 from Proposition 3.3.3, x x = S δ S ep(n) e p(n+) + u S + u0 S ( σ(f, p) δ ) + u 0 + u. For the purpose of applicability we usually use unit norm tight frame for R d to expand a signal of interest. In this case the bound of the quantization error derived in Theorem can be adjusted according to the properties of the frame operator S. More precisely, from Lemma.4. and the condition (.3.), we have S = I, (3.3.4) d so that S = d/ and we have the following corollary. Corollary Given the Σ scheme of Definition Let F = {e n } be a unit norm tight frame for R d with frame bound A = /d, let p be a permutation 40

46 of {,,..., }, let u 0 δ/, and let x R d satisfy x (K /)δ. Then the quantization error x x satisfies x x d ( σ(f, p) δ ) + u + u 0. If we apply the stability result of the auxiliary sequence {u n } from Proposition then we obtain x x δd ( ) σ(f, p) +. Since from Definition 3.3. the only restriction of the initial variable u 0 is that u 0 δ/, to lower the bound of quantization error, we set the initial variable u 0 to be 0. Moreover the bound of quantization error can be improved if one knows more information about the variable u. An example of frame that allows us to characterize the variable u based on the parity of the cardinality of frame is the zero sum frame which is a type of frame for which the sum of all frame elements is equal to 0. Theorem Given the Σ scheme of Definition Let F = {e n } be a unit norm tight frame for R d with frame bound A = /d, and assume that F satisfies the zero sum condition Additionally, set u 0 = 0. Then e n = 0. (3.3.5) 0 if is even, u = δ/ if is odd. (3.3.6) 4

47 Proof. From (3.3.0) we have u n u n = x p(n) q n, so that u = u u 0 = u n u n = Since e n = 0, we have x n = x p(n) x, e n = x, q n = x n e n = x, 0 = 0. q n. Therefore q n = u. Since for each n, q n is an odd integer multiple of δ/, we consider two cases. The first case is that is an odd integer. Then q n is an odd integer multiple of δ/ and since from stability result u δ/, it follows that u = δ. The second case is that is an even integer. Then q n is an integer multiple of δ and since from stability result u δ/, it follows that u = 0. Combining this theorem with Corollary 3.3.5, we have the following corollary. Corollary Given the Σ scheme of Definition Let F = {e n } be a unit norm tight frame for R d with frame bound A = /d, and assume that F satisfies the zero sum condition (3.3.5). Let p be a permutation of {,,..., }, let u 0 δ/, and let x R d satisfy x (K /)δ. Then the quantization error 4

48 x x satisfies δd σ(f, p) if is even, x x ( δd σ(f, p) + ) if is odd. 43

49 Chapter 4 Uniform Distribution and Discrepancy In this chapter, we develop the theory of uniform distribution of sequences of real numbers. Some classic examples are discussed including the sequence {nθ}, where θ is irrational. The second part of this chapter deals with the question of how well a given sequence is distributed over an interval of finite length. The notion used to measure the distribution of a sequence is called discrepancy. The larger the discrepancy the worse the sequence is distributed. One tries to approximate discrepancy rather than compute it directly. Erdös-Turán Inequality is one of the major tools that are used to approximate discrepancies. We shall see that this inequality approximate discrepancies in terms of exponential sum, that is, sum of the form b n=a eπif(n), for some real valued function f. The following materials on the theory of uniform distribution and discrepancy are adapted from Chapters and of [7]. 4. Uniform distribution mod We begin by setting up some notations. Let I = [0, ) be the unit interval. We recall that the fractional part of each real number lies in I. Let ω = {x n } be a given sequence of real numbers. For a positive integer and a subset E of I, let the counting function A(E; ; ω) be defined as the number of terms x n ( n ) 44

50 for which {x n } E. We shall sometimes write A(E; ) instead of A(E; ; ω) if the sequence ω is understood from the context. Definition 4... The sequence ω = {x n } of real numbers is said to be uniformly distributed modulo (abbreviated u.d. mod ) if for every pair a, b of real numbers with 0 a < b we have A([a, b); ; ω) lim = b a. (4..) Descriptively the definition says that a real sequence is uniformly distributed modulo if the share of the fractional parts of the terms of the sequence with the terms in each half open subinterval of I is finally equal to the length of each such subinterval. Example 4... The following sequence is u.d. mod : { 0, 0,, 0 3, 3, 3,..., 0 n, n,..., n } n,.... Proof. Let and 0 a < b. We try to compute A([a, b); ). We observe that there exists a unique integer k such that k (k + ) < (k + )(k + ). (4..) We write the first terms of the sequence in question in terms of k as follows: 0, 0,, 0 3, 3, 3,..., 0,,..., k 0, k k k k +,..., k (k + )/. k + ow we partition this finite sequence into k + blocks by letting the jth block ( j k ) consist of 0 j, j, j,..., j j 45

51 and letting the (k + )th block consist of 0 k +, k +, k +,..., k (k + )/. k + We now count the number of elements in the jth block ( j k ) that are in [a, b), i.e., we count the integers m in [0, j ) such that a m/j < b or ja m < jb. We see that this number equals j(b a) + θ j, where θ j <. We also see that the number of elements in the (k + )th block that are in [a, b) is not greater than k + and is at least 0. Using these ingredients we can approximate A([a, b); ) as follows: k j= j(b a) + θ j A([a, b); ) k j= j(b a) + θ j + k +. Since θ j < and k j= j = k (k + )/, we can approximate further that (b a)k (k + ) k < A([a, b); ) < k (k + ) + k + k +. By (4..) we have (b a)( (k + )) k < A([a, b); ) < (b a) + k + or (b a) b a k ( + b a) < A([a, b); ) < b a + k +. (4..3) Since by (4..), k (k + )/, we have (k /)(k / + /)/ / so that lim k / = 0. Hence inequalities (4..3) imply that A([a, b); ) lim = b a. Since a, b were arbitrary in I we have that the sequence in question is u.d. mod. 46