distortion and, usually, "many" cells, and "large" rate. Later we'll see Question: What "gross" characteristics" distinguish different highresolution

Transcription

1 High-Resolution Analysis of Quantizer Distortion For fied-rate, memoryless VQ, there are two principal results of high-resolution analysis: Bennett's Integral A formula for the mean-squared error distortion of a "high-resolution" VQ in terms of its "gross" characteristics. A "high-resolution" VQ is one with "small" cells, so it has "small" distortion and, usually, "many" cells, and "large" rate. Later we'll see roughly how "small", how "many" and how "large" are adequate. Question: What "gross" characteristics" distinguish different highresolution quantizers? Zador's formula An approimation to the OPTA function δ(k,r). It applies when R is large. Question: We'd much rather have small rate, than large rate. Why should we be interested in high-resolution formulas, which apply when rate is larger? Answer: We'll see that these formulas are accurate when R 3, and accurate enough even when R 2 to provide ecellent insight. /7/05 Bennett- Eamples of High-Resolution Quantizers: k= (scalar quantizers) /7/05 Bennett-2

2 2 k=2 2 /7/05 Bennett-3 /7/05 Bennett-4

3 /7/05 Bennett-5 The Key Gross Characteristics of a High-Resolution Quantizer: Dimension k Size The distribution or density of points/cells over R k Some way of characterizing the shapes of the cells as a function of location in R k. /7/05 Bennett-6

4 Case : Bennett's Integral for Quantizers with Congruent Cells Theorem The SE of a k-dimensional VQ with size and "small" congruent cells, or at least most cells must be approimately congruent, is approimately D 2/k m λ 2/k () f X() d where m = quantity, to be defined later, that depends only on the shape of the cells, but not on their size, nor on the source density λ() = "quantization density function". This is a function that characterizes, approimately, the density of cells/points in the vicinity of. It is also called the "point density" or "cell density" function. The formula above is called "Bennett's integral". It was derived by Bennett for scalar quantizers, and then etended to VQ's with congruent cells by Gersho 2. Our derivation comes later. Eamples of VQ's with congruent cells can be found on pages 4 and 5. Later we drop the congruent cell restriction. mostly means: the contribution to distortion due to cells that are not approimately congruent is small. Among other things, this requires the overload region to have small probability. W. Bennett, "Spectra of quantized signals," Bell Syst. Tech. J., vol. 27, pp , July A. Gersho, "Asymptotically optimal block quantizers," IEEE Trans. Inform. Thy., vol. 25, pp , 979. /7/05 Bennett-7 Quantization Density λ() The quantization density is a function λ() that characterizes the distribution of quantization cells and points, by having the following properties:. λ() 0, λ() d = 2. A λ() d fraction of codevectors (or cells) in region A # cells/points in A 3. if A is small, then λ() A, ( A = vol of A) (assuming A is much larger than cells in the vicinity of ) 4. λ() is a smooth or piecewise smooth function. 5. λ() S i when S i or equivalently, S i λ() Why 5? In a small region A containing, one epects most cells to have approimately the same volume. Therefore, A # cells in A cell vol, and by 3, λ() A λ() S i # cells in A = A /(cell vol) /7/05 Bennett-8

5 Eamples of Point Densities k= /7/05 Bennett-9 Optimal Quantizer for a 2-dimensional IID Gaussian random vector and its point density /7/05 Bennett-0

6 Derivation of Bennett's Integral D = k E X-Q(X) 2 = k k fx (w i ) i= Si i= Si -w i 2 f X () d -w i 2 d because f X () f X (w i ) when S i f X () f X (w i ) when S i is a valid approimation because most S i 's are small, and because f X () is ordinarily either continuous, in which case it changes little or over S i and can be approimated by its value at w i, or piecewise continuous, in which case it is continuous on most cells and the previous argument applies. = fx (w i ) i= k (S i,w i ) where (S,w) = -w 2 d = "moment of inertia" (mi) of S about w. S For future reference, we note that the key approimation made here is Si -w i 2 f X () d f X (w i ) Si -w i 2 d /7/05 Bennett- Normalized oment of Inertia (nmi) Let us now separate the effect on (S,w) of the shape of S from its size. k (S,w) = S -w 2 d k S +2/k S +2/k, where S = vol of S = S d = m(s,w) S +2/k where m(s,w) = S -w 2 d k S +2/k = "normalized moment of inertia" (nmi) of S about w. /7/05 Bennett-2

7 Fact: m(s,w) is not affected by scaling nor translation. Thus, it is determined only by its shape, but not its size nor position. Proof: First, consider scaling by a factor a>0: S as = {z = a : S}; w aw m(as,aw) = = as -aw 2 d k as +2/k = S az-aw 2 a k dz k as +2/k where az =, a k dz = d S a2 z-w 2 a k dz k S +2/k (a k ) +2/k because as = a k S = S z-w 2 dz k S +2/k = m(s,w) Net, consider translating by a vector v: S S+v = {z = +v : S}; w w+v m(s+v,w+v) = S+v -w-v 2 d k S+v +2/k = = m(s,w) S z-w 2 dz k S +2/k where z = -v, dz = d /7/05 Bennett-3 Completion of the Derivation of Bennett's Integral D fx (w i ) i= k (S i,w i ) derived earlier = fx (w i ) m(s i,w i ) S i +2/k by the definition of nmi i= = m(s o,w o ) fx (w i ) S i +2/k since all S i 's are congruent to S o i= Note: For the last equality to hold, each w i must be in the same relative position within S i that w o is in S o. We define "congruence" so as to imply this. Note: We can now see how cell sizes and shape separately affect distortion. Now recall: S i Therefore, D λ(w i ), which implies S i +2/k = 2/k 2/k m(s o,w o ) fx (w i ) i= λ 2/k (w i ) S i 2/k m(s o,w o ) f () λ 2/k () d = Bennett's integral λ 2/k (w i ) S i where the last " " is by the definition of an integral and the fact that most S i 's are small. /7/05 Bennett-4

8 Special case: Scalar quantization (k=) Cells are intervals. If codepoints (levels) are in the centers of the cells, then it is easy to show that Then m(interval) = 2 D 2 2 f X () - λ 2 () d This is what Bennett originally derived. /7/05 Bennett-5 Special case: Uniform scalar quantizer. Consider a quantizer that is uniform over the interval [a,b], in the sense that (a) the partition divides [a,b] into cells of width = b-a (c) the codepoints (levels) are in the centers of the cells (i.e. they are uniformly spaced, apart) a b is called the "quantizer stepsize". Suppose is small. Then λ() = b-a Suppose also that Pr(a X b). Then D 2 2 b f X () a /(b-a) 2 d (b-a)2 2 2 b fx () d 2 a 2 This is a formula worth remembering. /7/05 Bennett-6

9 Bennett's Integral for Vector Quantizers -- General Case Theorem 3 : Under the high-resolution conditions stated below, the SE distortion of a k-dimensional VQ with size applied to random vector X with pdf f X () can be approimated by D m() 2/k λ 2/k () f X() d High-resolution conditions: + ost cells are small enough that the prob. density can be approimated as being constant on each. (Union of cells for which prob. density cannot be so approimated has very small probability. The overload distortion is negligible.) + Neighboring cells have similar sizes and shapes, i.e. cell size & shape change slowly, if at all, with. ("similar shape" also refers to placement of codevectors within cells.) + The quantization density is approimately λ(). + The inertial profile is approimately m(), where the inertial profile of a VQ is a function such that m() NI of cell containing = m(s i,w i ) if S i We also require m() 0, all. 3S. Na and D. Neuhoff, "Bennett's integral for vector quantizers," IEEE Trans. Inform. Thy., vol. 4, pp , July 995. /7/05 Bennett-7 Derivation of Bennett's Integral -- General Case D = k E X-Q(X) 2 i=si = k -w i 2 f X () d k fx (w i ) i= Si -w i 2 d by same argument as in congruent case = fx (w i ) i= k (S i,w i ) (S i,w i ) is the I of S i about w i = fx (w i ) m(s i,w i ) S i +2/k recall: (S i,w i ) = m(s i,w i ) k S i +2/k i= 2/k fx (w i ) m(w i) i= λ 2/k (w i ) S i recall: S i λ(w i ), m(w i ) m(s i,w i ) 2/k m() λ 2/k () f X() d by the definition of an integral /7/05 Bennett-8

10 Notes on Bennett's Integral Bennett's integral identifies point density and inertial profile as key characteristics of VQ's, in addition to k and. When is large, both left and righthand sides of the Bennett integral relation are approimately zero; so when does a careful derivation, what really needs to be show (and what has been shown) is D m() 2/k λ 2/k () f X() d /7/05 Bennett-9 Bennett's integral shows that distortion decreases as 2/k, assuming point density and inertial profile stay the same. To see that this makes sense, consider what happens when doubles, while maintaining the same point density function and the same inertial profile. Doubling cuts the volumes of cells in a given region in half. Assuming the cell shapes stay the same, this decreases the linear dimensions of the cells in any region by the factor /2 /k, and causes the average squared distance between points and the codevector of the cell in which lies to decrease on the average by /2 2/k. This suggests that mean-squared error decreases as / 2/k. Equivalently, SNR increases 6 db for each one bit increase of rate. 2/k = 2-2R D 2-2R m() λ 2/k () f X() d SNR = 0 log 0 σ2 D = 0 log 0 2 2R + 0 log 0 = 6.02 R + 0 log 0 m() λ -2/k () f X () d This is called the "6 db per bit rule". /7/05 Bennett-20 σ 2 σ 2 m() λ -2/k () f X () d

11 Usually, we don't employ a quantization density or inertial profile to describe a VQ unless most cells are small, where "small" means that the probability density changes little across the cell and "most" means that the probability of the cells that are small is large. Usually, λ and m are fairly smooth functions that do not convey the detailed locations of codevectors and cells. VQ's with the same quantization density can differ in the number of points, in the eact placement of codepoints and in the shapes of the cells. VQ's with the same inertial profile can differ in the number and placement of codepoints Quantization density and inertial profile are, generally, idealizations or models. Often we pick a target quantization density and/or target inertial profile and try to make our quantizer approimate it. Later we'll find what quantization densities and inertial profiles are desirable. /7/05 Bennett-2 We don't use the following as definitions because if we did, a quantizer would almost never have a specified quantization density or inertial profile: λ() = S i when S i, m() = m(s i,w i ) if S i Sketch of why Property 5 on p. 8 implies Property 2 λ() d = λ() d A i= S i A i= because vol(s i A) vol(s i ) = vol(s i A) i= vol(s i ) S i A vol(s i ) d # cells in A, if S i A = 0, if S i A = φ, and because most S i 's are small <, otherwise When, as usual, λ() is smooth, Prop. 5, p. 8 implies neighboring cells have similar sizes; e.g. it rules out quantizers with alternating large and small cells. When, as usual, m() is smooth, the defining property of m implies neighboring cells mostly have similar NI; e.g. it rules out quantizers whose cell shapes change rapidly. We see from Bennett's integral that to make D small, we want larger quantization density where f X () is larger. Small inertial profile is desired, everywhere. /7/05 Bennett-22

12 Properties and Eamples of Normalized oment of Inertia (NI) Definition: NI of S about point w is m(s) = m(s,w) = -w 2 d S k vol +2/k (S) /7/05 Bennett-23 NI of a k-dimensional cube is /2, for all k. This is why the definition of m(s) includes /k. Defn: k-dimen'l cube = { : a i b, i =,,k} for some a < b} Proof: Let S = { : -/2 i /2, i =,,k} and w = (0,,0). Then vol(s) = and m(s) = /2 k vol(s)+2/k -/2 = k = k = k = k /2 -/2 -/2 /2 -/2 /2 /2 -/2 -/2 /2 -/2 i= k 2 i d d k k 2 i + 3 i=2 3 /2 -/2 d 2 d k /2 k 2 -/2 i=2 /2 -/2 /2 k /2 -/2 i=3 i + 2 d 2 d k k 2 i /2 i=4 = k 2 = 2 i= k 2 i /2 -/2 + 2 d 3 d k -/ d 4 d k /7/05 Bennett-24

13 The NI of various cell shapes cell shape dimension NI 2 rectangle 2.04 cube any.0833 = 2 heagon = circle = 4π sphere = (4π/3)-2/3 5 sphere k (k+2)(v k ) 2/k sphere.0585 = 2πe 7 s s 2 s k rectangle k \f(,2) kσ k i= (s i) 2 ( k i= (s i) ) 2 /k = 2 arith mean of sides 2 geom mean of sides 2 where V k = volume of k-dimensional sphere with radius /7/05 Bennett-25 Shapes that tend to make NI smaller + Spheroidal rather than oblong s. + ore finely faceted vs. (many sides rather than few) + Higher rather than lower dimension A sphere has the lowest NI of any cell of a given dimension. NI of a sphere decreases with dimension to the limit /2πe =.0585 /7/05 Bennett-26

14 V k = Volume of k-dimensional sphere (from Wozencraft & Jacobs, p. 357) vol. of k-dimensional sphere with radius = From Stirling's approimation, n! = 2πn n n ε e e N n e n 2πn π k/2 (k/2)!, k even 2 k π (k-)/2 ( k- 2 )!, k odd k! where 0 < ε n < 2n, which somewhat underestimates n!, one can show V k N 2πe πk k k/2. From the above, k V k ep 2 (ln(2πe) - ln(k)) 0 as k because -k ln k - /7/05 Bennett-27 It follows that m(k-dim'l sphere) = (k+2)(v k ) 2/k = (πk) /k k k+2 2πe 2πe as k. 2πe k/2-2/k k+2 πk k = ep{ k ln(πk)} k k+2 2πe /7/05 Bennett-28

15 Formal Statement of the Validity of Bennett's Integral Theorem 4 : lim 2/k m() D(Q ) = λ 2/k () f X() d Assuming +Q, Q 2,... is sequence of k-dim'l VQ's, with Q having size, partition S + λ () λ() in probability as where λ () = vol(cell containing ) = specific point denstiy of Q +m () m() as in prob. where m () = m(s,y ) = specific inertial profile of Q +{ 2/k -Q () 2 } has uniformly absolutely continous integrals + diam(cell of S containing X) 0 in prob. +f X () is piecewise continuous + Bennett's integral is finite "Sequence approach" first used by Bucklew & Wise (982), for scalar quantizers. 4S. Na and D. Neuhoff, "Bennett's integral for vector quantizers," IEEE Trans. Inform. Thy., vol. 4, pp , July 995. /7/05 Bennett-29 Eample: Polar Quantization Quantize X = (X,X 2 ) by independently scalar quantizing its magnitude and phase: agnitude: Phase: X X 2 A Φ bits scalar quantizer A, Q A scalar quantizer Φ, Q Φ bits A = X = X+X = magnitude/amplitude, A 0 Φ = X = angle or phase of X = (X,X 2 ), 0 Φ < 2π = X 2 A^ Φ^ ^ X ^ X 2 = A ^ cos Φ ^ = A ^ sin Φ^ - tan X, 2 0 tan - X 2 X +π, 2 <0, (assuming 0 tan - z < π) agnitude quantizer: ^A = QA (A), C A = {w,...,w A }, S A = {S,...,S A } Phase quantizer: ^Φ = QΦ (Φ), C Φ = {v,...,v }, T Φ = {T,...,T Φ } /7/05 Bennett-30

16 Polar Quantization is a kind of 2-dimensional VQ. Dimension: k = 2 Size: = A Φ X X 2 A Φ scalar quantizer, Q A A scalar quantizer, Q Φ Φ bits bits A^ Φ^ ^ X ^ X 2 = A ^ cos Φ^ = A ^ sin Φ^ Codebook: C = { w i,j }, where w i,j = (w i cos v j, w i sin v j ), i =,..., A, j =,..., Φ Partition: S = { S i,j }, where S i,j = { : S i, X T j }, i =,..., A, j =,..., Φ } Quantization rule: Q() = (Q A ( cos Q Φ ( ), Q A ( sin Q Φ ( ) ) /7/05 Bennett-3 Distortion analysis via Bennett's integral Assumptions: A, Φ are large, magnitude quantizer has point density λ A (a), phase quantizer is uniform with step size = 2π Φ Consequence: cells of polar quant cells are, approimately, small rectangles: W 2πw i Φ Volume of cell near w i H 2π A λ A (w i ) HW A Φ λ A ( ) Inertial profile of the VQ: m() = 2 2 (H2 +W 2 ) Point density of the VQ: λ() = \f(, H W) = λ A ( ) 2π H2 W 2 = Φ 24 2π A λ A ( ) + 2π A λ A ( ) Φ /7/05 Bennett-32

17 Substituting: λ() and m() into Bennett's integral and simplifying gives: D m() λ() f X() d where = = A λ A (a) 2 f A(a) da + π2 A 6 L 2 0 λ A (a) 2 f A(a) da + π2 6 L2 2 a fa (a) da a 2 fa (a) da L = A = A = magnitude level allocation Φ (Recall: = A Φ ) Notice that D does not depend on the probability distribution of the angle. This is because the angle quantizer is uniform. So it does not favor some angles over others. /7/05 Bennett-33 Optimizing Polar Quantization Given (large), choose L and λ A (the key characteristics) to minimize distortion D L 2 0 λ A (a) 2 f A (a) da + π2 6 L2 2 a fa (a) da 0 Approach : For given choice of λ A, find best L by equating to zero the derivative of D with respect to L. Then find best λ A by calculus of variations. Approach 2: For given choice of L, find best λ A by calculus of variations or Holder's inequality. Then find best L by equating to zero the derivative wrt L of the resulting epression for distortion. The result: λ A (a) = c p A (a) /3 (c chosen to make λ A (a) integrate to one) L 2 = A = Φ 2π EA 2 /3 f A (a) da -/2 0 D 2 2π E A2 /3 f 0 /7/05 Bennett-34 A (a) da 3/2 We'll review calculus of variations shortly.

18 Prime Eample: X,X 2 IID Gaussian f X () = 2/2 2π e- and f A (a) = a e -a2 /2, a 0 (Rayleigh density) EA = π/2, EA 2 = E(X+X 2 2) 2 = 2 EX 2 = 2, σa 2 π = 2-2 =.429 For optimized polar quantization λ A (a) = c p A (a) /3 = a 3 e-a 2/6, a 0 L 2 = A = Φ 2π EA 2 /3 f 0 A (a) da -/2 =.376 A Φ =.63 (more phase levels than amplitude levels) D 2 σ2 X 29.7 Later we'll show that for optimal scalar quantization applies directly to X,X 2 D 2 σ2 X 32.6 Gain of polar quantization over conventional optimal scalar quantization 0 log =.4 db Gain of optimal two-dimensional VQ over scalar quantization =.30 db /7/05 Bennett-35 Comments on Polar Quantization Polar quantization would seem to be "especially well suited" to quantizing X when f X () is circularly symmetric; i.e. when f X () depends only on. equivalently, A and Φ are indep., and Φ is uniformly distributed between 0 and 2π. equivalently, f X () = 2π f A( ), where f A (a) is pdf of amplitude a because f A ( ) Pr( A + ) = Pr( X + ) 2π f X () On the other hand, we found that distortion does not depend on the angle distribution. The eplanation: The opta function for a noncircularly symmetric density is less than the opta function for a circularly symmetric density with the same magnitude density. Therefore, it's not that polar quantization attains less distortion for a density that is circularly symmetric than for a density that is not. Rather, for a circularly symmetric density, polar quantization is closer to being an optimal 2-dimensional VQ than for a noncircularly symmetric pdf. /7/05 Bennett-36

19 Calculus of Variations Fi and L. Let J(λ A ) be the functional defined by the formula for the distortion of a polar quantizer. Calculus of variations finds an equation that λ A must solve. If λ A is the optimal point density, i.e the one that makes J(λ A ) smallest among all nonnegative functions λ that integrate to one, then for any function 5 g such that 0 g(a) da = 0 and any ε > 0, λ A (a)+εg(a) cannot be better than J(λ A ), i.e. J(λ A +εg) has a local minimum with respect to ε when ε = 0 Therefore, d dε J(λ A+εg) ε=o = 0 and this must be true for every function g such that 0 g(a) da = 0. Therefore, we take the derivative of J(λ A +εg) with respect to ε. We set ε = 0 and we equate the derivative at ε=0 to zero. This gives an equation that λ Α must satisfy for every function g such that 0 g(a) da = 0. Solving this equation gives the answer 6. Typically, the equation is something like g(a) (λ A (a)-h(a)) da = 0 for all functions g such that g(a) da = 0 which implies that λ A (a) = h(a) + c, where c is some constant. 5 To be perfectly rigorous, we need to restrict attention to functions g such that g(a) = 0 wherever λ A (a) = 0. 6 One should check that g(a) = 0 wherever λ A (a) = 0. /7/05 Bennett-37 Generalizing Bennett's Integral to Quantizers Whose Neighboring Cells Do Not Have Similar Sizes or Shapes 7 Can we apply Bennett s integral to the two-dimensional quantizer shown to the right in which cells come in identical groups of three? For concreteness and tractability, assume X is uniformly distributed on the support of this quantizer. As a first try, the point density would appear to be 3 λ() = 2 2 because there are 3 cells with in each 2 rectangular group, and the inertial profile would appear to be m() = = because half the area is contained in squares with nmi /2, and half is in 2 rectangles with nmi Substituting these into Bennett s integral gives D m() 2/k λ 2/k () f X() d = /2 (3/(2 2 )) 2/2 f X () d = This section is optional reading. /7/05 Bennett-38

20 As a check, let us do a direct computation of distortion. When X lies in a square cell, the distortion is the same as when the quantizer consists entirely of square cells, namely, 2 /2, which is the distortion of a uniform scalar quantizer with step size. Similarly, when X lies in a 2 rectangular cell, the distortion will be the same as a quantizer consisting entirely of such cells. Such a quantizer is equivalent to using a uniform scalar quantizer with step size on X and a uniform scalar quantizer with step size /2 on X 2. The distortion for X is 2 /2, the distortion for X 2 is ( /2) 2 /2, and these average to give distortion (5/96) 2 for X in a 2 cell. Since X is in square or rectangular cells with probability /2, the overall distortion is D = = = This differs considerably from the value computed with Bennett s integral. Why does Bennett s integral give the wrong answer? Notice the cells within each group do not all have the same, or even similar, sizes and shapes. This invalidates the high-resolution conditions, used in deriving Bennett s integral. /7/05 Bennett-39 To see eplicitly where these conditions were used, let us eamine the three approimations in the derivation of Bennett's integral, which is repeated below: D = k () i= Si -w i 2 f X () d fx (w i ) i= k (S i,w i ) = fx (w i ) m(s i,w i ) S i +2/k i= (2) i= fx (w i ) m(w i ) 2/k λ 2/k (w i ) S i this approimation is OK; it requires only that the cells be small this approimation requires S i λ(w i ) and m(s i,w i ) m(w i ) ( *** ) (3) m() 2/k λ 2/k () f X() d this approimation is OK; it requires only that the S i s form a partition with small cells and w i S i The problem is that the approimations in ( *** ) are not valid because λ() and m() are, generally, slowly varying smooth functions, but cell size and shape change abruptly from cell to cell. /7/05 Bennett-40

21 To rectify this problem, we need to consider groups of three cells as one unit. Previously, we approimated distortion one cell at a time: k Si -w i 2 f X () d f X (w i ) k (S i,w i ) = f X (w i ) m(s i,w i ) S i +2/k m(w f X (w i ) i ) 2/k λ 2/k (w i ) S i Now, instead, we compute the distortion of the three cells as group. Let S n,,s n,2,s n,3 denote the cells in the nth group, let w n,, w n,2, w n,3 denote the corresponding codevectors, let ~ S n = S n, S n,2 S n,3, and let ~ w n be some point in ~ S n. Point density: With a macroscopic view, we take the point density to be a function such that λ() 3 S ~ = n when ~ S n /7/05 Bennett-4 Inertial profile: Let m(s ~ n,, w n,, S n,2, w n,, S n,3, w n, ) denote the following average nmi of the group of three cells: ~ m(s n,, w n,, S n,2, w n,, S n,3, w n, ) = k 3 ((S n,,w n, )+(S n,2,w n,2 )+(S n,3,w n,3 )) 3 ( S n, + S n,2 + S n,3 ) +2/k and let m() ~ denote the corresponding inertial profile: ~ m() m(s ~ n,, w n,, S n,2, w n,, S n,3, w n, ) when S ~ n One can straightforwardly check that ~ m is invariant to the scale and position of the group of cells. Thus, it depends only the shapes and relative sizes of the cells in the group. Since ~ m does not depend on, we may assume = and compute ~ m = 2 3 ( ) 3 ( ) 2 = 3 28 = 0.02 which lies between the nmi of a square (0.0833),,and that of a 2 rectangle (0.04)) /7/05 Bennett-42

22 Then the distortion due to the group of three cells is k 3 -w n,i 2 f X () d f X ( w ~ 3 n ) k (S n,,w n,i ) since S ~ n is also small i= Sn,i i= = f X ( ~ w n ) 3 ~ m(s n,, w n,, S n,2, w n,2, S n,3, w n,3 ) ( 3 )+2/k ~ S n +2/k = f X ( ~ w n ) ~ m(s n,, w n,, S n,2, w n,2, S n,3, w n,3 ) ( 3 ~ S n ) 2/k ~ S n f X ( ~ w n ) ~ m( ~ w n ) 2/k λ 2/k ( ~ w n ) ~ S n where we have used the approimation 3 ~ S n λ( w ~ n ) Now, summing the approimate distortions of all groups gives /3 D = n= k 3 i= Sn,i -w n,i 2 /3 f X () d fx ( w ~ n ) n= ~ m( ~ w n ) 2/k λ 2/k ( ~ w n ) ~ S n m() ~ 2/k λ 2/k () f X() d This is just like the original Bennett's integral, but with a definition of inertial profile that averages over cells in a group. /7/05 Bennett-43 As a check, substituting for λ() and ~ m() gives D 3/28 3/(2 2 ) f X() d = = , which is the correct answer. /7/05 Bennett-44

23 ore generally if a partition comes in groups of o cells, then the Bennett integral approimation is: where D λ() m() ~ 2/k λ 2/k () f X() d o o Sn,i i= when S n,... S n,o ~ m() ~ m(s n,, w n,,..., S n,o, w n,o ) when S n,... S n,o ~ m(s n,, w n,,..., S n,o, w n,o ) = k o(sn,i,w n,i ) o i= o ( S n, S n,o ) +2/k /7/05 Bennett-45 Distortion Density and Local Distortion 8 Let G be some subset of R k. If one repeats the derivation of Bennett's integral, but for the distortion computed only X in the set G, then one obtains where G -Q() 2 m() f X () d G 2/k λ 2/k () f X() d = ρ() d G m() ρ() = 2/k λ 2/k () f X() Since ρ is a function that when integrated over a region G yields the distortion in region G, it can be called the distortion density of the quantizer. One may also rewrite the distortion in G as where G -Q() 2 f X () d G d() = m() 2/k λ 2/k () d() f() d is considered the local distortion of the quantizer at. 8This section is optional reading. /7/05 Bennett-46

24 Notice that the distortion in region G is not the same as the conditional distortion given region G, which is E [ X-Q(X) 2 X G] = G G = G -Q() 2 f X() Pr(X G) d ρ() Pr(X G) d d() f X () Pr(X G) d This shows that conditional distortion is the average of local distortion. When G is small and G, approimating d and f as being constant on G yields E [ X-Q(X) 2 X G] ρ() Pr(X G) G ρ() f X () = d(). Thus for a small region, the conditional distortion approimately equals the local distortion in that region. /7/05 Bennett-47