Least Distortion of Fixed-Rate Vector Quantizers. High-Resolution Analysis of. Best Inertial Profile. Zador's Formula Z-1 Z-2

Similar documents
PROOF OF ZADOR-GERSHO THEOREM

Convergence of Random Walks

PDE Notes, Lecture #11

REAL ANALYSIS I HOMEWORK 5

Topic 7: Convergence of Random Variables

Problem set 2: Solutions Math 207B, Winter 2016

Summary of Shannon Rate-Distortion Theory

LECTURE NOTES ON DVORETZKY S THEOREM

Robust Forward Algorithms via PAC-Bayes and Laplace Distributions. ω Q. Pr (y(ω x) < 0) = Pr A k

distortion and, usually, "many" cells, and "large" rate. Later we'll see Question: What "gross" characteristics" distinguish different highresolution

1. Aufgabenblatt zur Vorlesung Probability Theory

Introduction to the Vlasov-Poisson system

Lecture 5. Symmetric Shearer s Lemma

The derivative of a function f(x) is another function, defined in terms of a limiting expression: f(x + δx) f(x)

Acute sets in Euclidean spaces

θ x = f ( x,t) could be written as

Discrete Mathematics

Function Spaces. 1 Hilbert Spaces

Time-of-Arrival Estimation in Non-Line-Of-Sight Environments

Schrödinger s equation.

BIRS Ron Peled (Tel Aviv University) Portions joint with Ohad N. Feldheim (Tel Aviv University)

Calculus of Variations

A Unified Theorem on SDP Rank Reduction

A note on asymptotic formulae for one-dimensional network flow problems Carlos F. Daganzo and Karen R. Smilowitz

arxiv: v1 [cs.it] 21 Aug 2017

Math 300 Winter 2011 Advanced Boundary Value Problems I. Bessel s Equation and Bessel Functions

arxiv: v1 [math.mg] 10 Apr 2018

Lecture Introduction. 2 Examples of Measure Concentration. 3 The Johnson-Lindenstrauss Lemma. CS-621 Theory Gems November 28, 2012

Least-Squares Regression on Sparse Spaces

Lower Bounds for the Smoothed Number of Pareto optimal Solutions

Monte Carlo Methods with Reduced Error

Lecture 6 : Dimensionality Reduction

Tractability results for weighted Banach spaces of smooth functions

An extension of Alexandrov s theorem on second derivatives of convex functions

A LIMIT THEOREM FOR RANDOM FIELDS WITH A SINGULARITY IN THE SPECTRUM

6 General properties of an autonomous system of two first order ODE

1 Math 285 Homework Problem List for S2016

Monotonicity for excited random walk in high dimensions

ON THE OPTIMAL CONVERGENCE RATE OF UNIVERSAL AND NON-UNIVERSAL ALGORITHMS FOR MULTIVARIATE INTEGRATION AND APPROXIMATION

A Weak First Digit Law for a Class of Sequences

A simple tranformation of copulas

Floating Body, Illumination Body, and Polytopal Approximation

Tutorial on Maximum Likelyhood Estimation: Parametric Density Estimation

A Sketch of Menshikov s Theorem

Computing Exact Confidence Coefficients of Simultaneous Confidence Intervals for Multinomial Proportions and their Functions

Lectures - Week 10 Introduction to Ordinary Differential Equations (ODES) First Order Linear ODEs

arxiv: v2 [cond-mat.stat-mech] 11 Nov 2016

Semi-continuous challenge

Entanglement is not very useful for estimating multiple phases

SHARP BOUNDS FOR EXPECTATIONS OF SPACINGS FROM DECREASING DENSITY AND FAILURE RATE FAMILIES

Leaving Randomness to Nature: d-dimensional Product Codes through the lens of Generalized-LDPC codes

Assignment 1. g i (x 1,..., x n ) dx i = 0. i=1

Monotonicity of facet numbers of random convex hulls

Lower Bounds for k-distance Approximation

Final Exam Study Guide and Practice Problems Solutions

Physics 505 Electricity and Magnetism Fall 2003 Prof. G. Raithel. Problem Set 3. 2 (x x ) 2 + (y y ) 2 + (z + z ) 2

Center of Gravity and Center of Mass

Qubit channels that achieve capacity with two states

An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback

Jointly continuous distributions and the multivariate Normal

FLUCTUATIONS IN THE NUMBER OF POINTS ON SMOOTH PLANE CURVES OVER FINITE FIELDS. 1. Introduction

Math 342 Partial Differential Equations «Viktor Grigoryan

Analysis IV, Assignment 4

Iterated Point-Line Configurations Grow Doubly-Exponentially

Fall 2016: Calculus I Final

NOTES ON EULER-BOOLE SUMMATION (1) f (l 1) (n) f (l 1) (m) + ( 1)k 1 k! B k (y) f (k) (y) dy,

Companding of Memoryless Sources. Peter W. Moo and David L. Neuho. Department of Electrical Engineering and Computer Science

Levy Process and Infinitely Divisible Law

Some Examples. Uniform motion. Poisson processes on the real line

Logarithmic spurious regressions

Quantum Mechanics in Three Dimensions

arxiv: v1 [stat.co] 23 Jun 2012

Extreme Values by Resnick

On combinatorial approaches to compressed sensing

Large Triangles in the d-dimensional Unit-Cube (Extended Abstract)

2.5 SOME APPLICATIONS OF THE CHAIN RULE

Bayesian Estimation of the Entropy of the Multivariate Gaussian

Characterizing Real-Valued Multivariate Complex Polynomials and Their Symmetric Tensor Representations

Relative Entropy and Score Function: New Information Estimation Relationships through Arbitrary Additive Perturbation

Wiener Deconvolution: Theoretical Basis

1. At time t = 0, the wave function of a free particle moving in a one-dimension is given by, ψ(x,0) = N

CHAPTER 1 : DIFFERENTIABLE MANIFOLDS. 1.1 The definition of a differentiable manifold

GCD of Random Linear Combinations

12.11 Laplace s Equation in Cylindrical and

arxiv: v4 [math.pr] 27 Jul 2016

arxiv: v2 [math.ca] 8 Oct 2014

Lagrangian and Hamiltonian Mechanics

Separation of Variables

MAE 210A FINAL EXAM SOLUTIONS

Second order differentiation formula on RCD(K, N) spaces

Large Cells in Poisson-Delaunay Tessellations

( 1 k "information" I(X;Y) given by Y about X)

Bandwidth expansion in joint source- channel coding and twisted modulation

The Press-Schechter mass function

The Exact Form and General Integrating Factors

1 dx. where is a large constant, i.e., 1, (7.6) and Px is of the order of unity. Indeed, if px is given by (7.5), the inequality (7.

HITTING TIMES FOR RANDOM WALKS WITH RESTARTS

Goal of this chapter is to learn what is Capacitance, its role in electronic circuit, and the role of dielectrics.

Unit vectors with non-negative inner products

A FURTHER REFINEMENT OF MORDELL S BOUND ON EXPONENTIAL SUMS

Transcription:

High-Resolution Analysis of Least Distortion of Fixe-Rate Vector Quantizers Begin with Bennett's Integral D 1 M 2/k Fin best inertial profile Zaor's Formula m(x) λ 2/k (x) f X(x) x Fin best point ensity Substitute into Bennett's integral to obtain high-resolution approximation to the OPTA function of k-imensional VQ. Z-1 Best Inertial Profile For any imension k an probability istribution, the best inertial profile is a constant. The constant is inepenent of the source probability istribution. Sketch of the proof: (no rigorous proof exists) - Claim 1: When the source ranom vector is uniformly istribute on a k- imensional cube, the best inertial profile is a constant. That is the inertial profile of an optimal quantizer is, approximately, a constant. Let m k * enote the constant. (Actually, one can replace "cube" with "sphere" or any other sufficiently nice convex region.) Z-2

- Claim 2: For any k-imensional probability ensity function, the best inertial profile is the constant m * k. That is, an optimal quantizer with many points has m(x) = m * k. Z-3 Gersho's Conjecture: When M is large, most cells of an optimal, k-imensional, M-point, fixe-rate quantizer are approximately congruent to some basic cell shape. The basic cell shape is the k-imensional tesselating polyheron with least NMI. In most small regions, the quantizer partition is a "tesselation" base on this basic cell shape. A cell "tesselates" or "tiles" if there exists a partition of R k with each cell being a translation or rotation (but not a scaling) of the basic cell shape. m k * = least nmi of any tesselating k-imensional polyheron = Gersho's constant Notes: Since the optimal quantizer will orinarily have cells of ifferent sizes, its partition will not be a tesselation. However, in small regions the tesselation will be apparent, i.e. locally it is approximately a tesselation. The tesselation generate by the basic cell shape might even be a "lattice", meaning that all cells of the tesselation are translations (rather than rotations) of one another. We will assume Gersho's conjecture is correct when we nee the value of m * k. If this is not exactly correct, it is not off by much. Z-4

Example: A two-imensional VQ with M = 256 esigne by the LBG algorithm for an IID Gaussian source. Z-5 Best Point Density Assuming the best inertial profile is a constant, we nee to fin the point ensity minimizing the remaining part of Bennett's integral. λ 2/k (x) x (*) It coul be foun with calculus of variations, but we will use Holer's inequality. Z-6

Holer's inequality: Given function's f an g, then for any q,r > 0 such that 1 q + 1 r = 1, f(x) g(x) x f(x) q x 1/q g(x) r x 1/r with equality iff for some c, f(x) q = c g(x) r, all x Strategy: Choose f, g, q an r so that f(x) q = f X(x) λ(x) 2/k an g(x) r x = 1. Then λ(x) 2/k x = f(x) q x f(x) g(x) x q g(x) r x -q/r = f(x) g(x) x q with equality iff there is constant c such that f(x) q = c g(x) r. If it turns out that the last integral oes not epen on λ, then we have a lower boun to the integral we are minimizing. An we can minimize the integral by choosing λ to satisfy the conition that gives equality in the lower boun. Z-7 Our choices: q = k+2 k, r = k+2 2 1 q + 1 r = k k+2 + 2 k+2 = 1 f(x) = λ 2/k (x) k/(k+2) Then as esire, f(x) q = f X(x) λ 2/k (x) Therefore, an g(x) = λ 2/(k+2) (x) an g(x) r x q/r = λ(x) x q/r = 1 λ 2/k (x) x f(x) g(x) x q = f k/(k+2) X (x) x (k+2)/k where, fortunately, the right-han sie oes not epen on λ, an where equality hols iff there is a constant c such that λ 2/k (x) = c λ(x), i.e. λ(x) = c' f k/(k+2) X (x) where c' is chosen to make λ(x) integrate to one. We conclue that the integral (*) is minimize by the point ensity λ * k(x) = f k/(k+2) X (x) f k/(k+2) X (x') x' an the resulting minimum value is f k/(k+2) X (x) x (k+2)/k Z-8

Substituting λ * k an m * k into Bennett's integral yiels: Zaor's Theorem (Zaor, 1963) When M is large, the least istortion of fixe-rate, k-im'l VQ with M points is δ(k,m) σ 2 β k m * 1 k M 2/k = Z(k,M) where Z(k,M) is calle Zaor's function σ 2 = source variance = 1 k k variance(xi ) i=1 β k = 1 ( σ 2 k/(k+2) x ) (k+2)/k = "Zaor's factor" epens on "shape" of ; invariant to a scaling) m * k = least NMI of k-im'l tesselating polytopes (we assume) Equivalent Statements When R is large, the best fixe-rate, k-imensional VQ's with rate R have MSE δ(k,r) σ 2 β k m k * 2-2R = Z(k,R) When R is large, the best fixe-rate, k-imensional VQ's with rate R have SNR σ S(k,R) 2 10 log 10 Zk(R) = 6.02 R - 10 log 10 m k * βk Note: SNR increases at 6 B per bit for optimal quantizers. Z-9 How large must M or R be in orer for the formulas to be accurate? Example: Gauss-Markov Source, corr. coeff. ρ =.9 25 k=2 VQ's esigne by LBG algorithm. 20 k=4 k=1 Straight lines are Zaor's funtion Z(k,R). SNR, B 15 10 m * 4 is estimate. 5 0 0 1 2 3 4 Rate Z-10

Data for the previous plot Gauss-Markov Source, ρ =.9, SNR's in B k = 1 k = 2 k = 4 Rate Act'l Pre' Diff. Act'l Pre' Diff. Act'l Pre' Diff. 0.0 0.0-4.35 4.35 0.0 0.6-0.56 0.00 3.33-3.33 0.5 4.0 3.6 0.47 6.55 6.34 0.21 1.0 1.68 4.4 2.72 7.9 6.6 1.35 10.22 9.35 0.87 1.5 10.8 9.6 1.21 13.04 12.36 0.68 2.0 7.70 9.3 1.60 13.5 12.6 0.92 15.81 15.37 0.44 2.5 16.3 15.6 0.65 18.66 18.38 0.27 3.0 13.72 14.62 0.90 19.0 18.6 0.41 3.5 21.9 21.6 0.26 4.0 19.74 20.22 0.48 24.8 24.6 0.16 The preicte value at R = 0 is -10 log 10 m * k β k Z-11 Example 2: IID Gaussian Source 20 k=2 VQ's esigne by LBG algorithm. 15 k=4 k=1 Straight lines are from Zaor's function Z(k,R). SNR, B 10 m * 4 is estimate. 5 0 0 1 2 3 4 Rate Z-12

Data for the previous plot IID Gaussian Source, SNR's in B k = 1 k = 2 k = 4 Rate Act'l Pre' Diff. Act'l Pre' Diff. Act'l Pre' Diff. 0.0 0.0-4.35 4.35 0.00-2.95 2.95 0.00-2.01 2.01 0.5 1.66 0.06 1.60 1.89 1.00 0.89 1.0 1.68 4.4 2.72 4.39 3.07 1.32 4.60 4.01 0.60 1.5 6.96 6.08 0.88 7.34 7.02 0.32 2.0 7.70 9.3 1.60 9.64 9.09 0.55 10.18 10.03 0.15 2.5 12.42 12.10 0.32 13.21 13.04 0.17 3.0 13.72 14.62 0.90 15.27 15.11 0.16 16.18 16.05 0.14 3.5 18.17 18.12 0.05 4.0 19.74 20.22 0.48 21.12 21.13-0.01 The preicte value at R = 0 is -10 log10 m * k β k Z-13