Least Distortion of Fixed-Rate Vector Quantizers. High-Resolution Analysis of. Best Inertial Profile. Zador's Formula Z-1 Z-2

Transcription

1 High-Resolution Analysis of Least Distortion of Fixe-Rate Vector Quantizers Begin with Bennett's Integral D 1 M 2/k Fin best inertial profile Zaor's Formula m(x) λ 2/k (x) f X(x) x Fin best point ensity Substitute into Bennett's integral to obtain high-resolution approximation to the OPTA function of k-imensional VQ. Z-1 Best Inertial Profile For any imension k an probability istribution, the best inertial profile is a constant. The constant is inepenent of the source probability istribution. Sketch of the proof: (no rigorous proof exists) - Claim 1: When the source ranom vector is uniformly istribute on a k- imensional cube, the best inertial profile is a constant. That is the inertial profile of an optimal quantizer is, approximately, a constant. Let m k * enote the constant. (Actually, one can replace "cube" with "sphere" or any other sufficiently nice convex region.) Z-2

2 - Claim 2: For any k-imensional probability ensity function, the best inertial profile is the constant m * k. That is, an optimal quantizer with many points has m(x) = m * k. Z-3 Gersho's Conjecture: When M is large, most cells of an optimal, k-imensional, M-point, fixe-rate quantizer are approximately congruent to some basic cell shape. The basic cell shape is the k-imensional tesselating polyheron with least NMI. In most small regions, the quantizer partition is a "tesselation" base on this basic cell shape. A cell "tesselates" or "tiles" if there exists a partition of R k with each cell being a translation or rotation (but not a scaling) of the basic cell shape. m k * = least nmi of any tesselating k-imensional polyheron = Gersho's constant Notes: Since the optimal quantizer will orinarily have cells of ifferent sizes, its partition will not be a tesselation. However, in small regions the tesselation will be apparent, i.e. locally it is approximately a tesselation. The tesselation generate by the basic cell shape might even be a "lattice", meaning that all cells of the tesselation are translations (rather than rotations) of one another. We will assume Gersho's conjecture is correct when we nee the value of m * k. If this is not exactly correct, it is not off by much. Z-4

3 Example: A two-imensional VQ with M = 256 esigne by the LBG algorithm for an IID Gaussian source. Z-5 Best Point Density Assuming the best inertial profile is a constant, we nee to fin the point ensity minimizing the remaining part of Bennett's integral. λ 2/k (x) x (*) It coul be foun with calculus of variations, but we will use Holer's inequality. Z-6

4 Holer's inequality: Given function's f an g, then for any q,r > 0 such that 1 q + 1 r = 1, f(x) g(x) x f(x) q x 1/q g(x) r x 1/r with equality iff for some c, f(x) q = c g(x) r, all x Strategy: Choose f, g, q an r so that f(x) q = f X(x) λ(x) 2/k an g(x) r x = 1. Then λ(x) 2/k x = f(x) q x f(x) g(x) x q g(x) r x -q/r = f(x) g(x) x q with equality iff there is constant c such that f(x) q = c g(x) r. If it turns out that the last integral oes not epen on λ, then we have a lower boun to the integral we are minimizing. An we can minimize the integral by choosing λ to satisfy the conition that gives equality in the lower boun. Z-7 Our choices: q = k+2 k, r = k q + 1 r = k k k+2 = 1 f(x) = λ 2/k (x) k/(k+2) Then as esire, f(x) q = f X(x) λ 2/k (x) Therefore, an g(x) = λ 2/(k+2) (x) an g(x) r x q/r = λ(x) x q/r = 1 λ 2/k (x) x f(x) g(x) x q = f k/(k+2) X (x) x (k+2)/k where, fortunately, the right-han sie oes not epen on λ, an where equality hols iff there is a constant c such that λ 2/k (x) = c λ(x), i.e. λ(x) = c' f k/(k+2) X (x) where c' is chosen to make λ(x) integrate to one. We conclue that the integral (*) is minimize by the point ensity λ * k(x) = f k/(k+2) X (x) f k/(k+2) X (x') x' an the resulting minimum value is f k/(k+2) X (x) x (k+2)/k Z-8

5 Substituting λ * k an m * k into Bennett's integral yiels: Zaor's Theorem (Zaor, 1963) When M is large, the least istortion of fixe-rate, k-im'l VQ with M points is δ(k,m) σ 2 β k m * 1 k M 2/k = Z(k,M) where Z(k,M) is calle Zaor's function σ 2 = source variance = 1 k k variance(xi ) i=1 β k = 1 ( σ 2 k/(k+2) x ) (k+2)/k = "Zaor's factor" epens on "shape" of ; invariant to a scaling) m * k = least NMI of k-im'l tesselating polytopes (we assume) Equivalent Statements When R is large, the best fixe-rate, k-imensional VQ's with rate R have MSE δ(k,r) σ 2 β k m k * 2-2R = Z(k,R) When R is large, the best fixe-rate, k-imensional VQ's with rate R have SNR σ S(k,R) 2 10 log 10 Zk(R) = 6.02 R - 10 log 10 m k * βk Note: SNR increases at 6 B per bit for optimal quantizers. Z-9 How large must M or R be in orer for the formulas to be accurate? Example: Gauss-Markov Source, corr. coeff. ρ =.9 25 k=2 VQ's esigne by LBG algorithm. 20 k=4 k=1 Straight lines are Zaor's funtion Z(k,R). SNR, B m * 4 is estimate Rate Z-10

6 Data for the previous plot Gauss-Markov Source, ρ =.9, SNR's in B k = 1 k = 2 k = 4 Rate Act'l Pre' Diff. Act'l Pre' Diff. Act'l Pre' Diff The preicte value at R = 0 is -10 log 10 m * k β k Z-11 Example 2: IID Gaussian Source 20 k=2 VQ's esigne by LBG algorithm. 15 k=4 k=1 Straight lines are from Zaor's function Z(k,R). SNR, B 10 m * 4 is estimate Rate Z-12

7 Data for the previous plot IID Gaussian Source, SNR's in B k = 1 k = 2 k = 4 Rate Act'l Pre' Diff. Act'l Pre' Diff. Act'l Pre' Diff The preicte value at R = 0 is -10 log10 m * k β k Z-13