Quantization. Introduction. Roadmap. Optimal Quantizer Uniform Quantizer Non Uniform Quantizer Rate Distorsion Theory. Source coding.

Transcription

1 Roadmap Quantization Optimal Quantizer Uniform Quantizer Non Uniform Quantizer Rate Distorsion Theory Source coding 2 Introduction 4 1

2 Lossy coding Original source is discrete Lossless coding: bit rate >= entropy rate One can further quantize source samples to reach a lower rate General description Scalar Quantization Original source is continuous Lossless coding will require an infinite bit rate! One must quantize source samples to reach a finite bit rate Lossy coding rate is bounded by the mutual information between the original source and the quantized source that satisfy a distortion criterion Quantization methods Scalar quantization Vector quantization Uniform quantization Non-uniform quantization Lloyd algorithm 5 6 Example of quantization Quantization Assume a coding image intensity value F pixels can be assigned values in [0, 255]: 8 bits are use to code each value. There are 4 bins in this uniform quantizer 7 8 2

3 Quantization Quantization 9 10 SQ as Line Partition Uniform quantization Mapping f(n) g(n) f(n) continuous-value or discrete-value signal g(n) discrete-value signal 12 3

4 SQ Function Representation Mapping onto quantization index and reconstruction : f(n) i, i g(n)=gi. Decision thresholds Uniform quantization The distance between reconstruction values which are neighbours on the amplitude scale, are quantized intervals of constant width Δ Each reconstruction g i value is positioned at the center of a quantization interval All continuous-amplitude signal values within this interval are mapped into g i Uniform Quantization For positive only signal Quantization Reconstruction q = quantization step-size L = number of quantization levels q = B/L = B*2^(-R)

5 Example Distortion measure General measure for a scalar quantizer (N = 1, i.e. one sample is quantized at the same time): Mean distortion incurred in region B l Distorsion measure Uniform Quantization Mean Square Error: Minimize the mean squared error, MSE = Expected value of (f-q(f)) 2 given the number of quantization levels L. Assume that the density function p f (f) is known (or can be approximated by a normalized histogram). Note that for images, f=image intensity. p f (f) is the image intensity ditribution

6 Demonstration Demonstration Optimum reconstruction Uniform Optimal Quantizer (1) Optimal transition levels lie halfway between the optimum reconstruction levels. (2) Optimum reconstruction levels lie at the center of mass of the probabality density in between the transition levels. (3) A and B are simultaneous non-linear equations (in general) Closed form solutions normally does not exist use numerical techniques

7 Uniform Optimal Quantizer Non Uniform Quantization 25 a dead-zone quantizer: (the values around 0 are enlarged; beyond the deadzone, the quantizer characteristics is of uniform step 26 size) Useful when? Non Uniform Quantization Non Uniform Quantization - Optimization If the input signal f(n) has a non-uniform PDF. Quantizers of non-uniform step sizes can be used to implement nonlinear amplitude-mappings as a by-product of quantization. If quantization errors are perceived as more severe at low amplitude ranges, where more accurate quantization should be performed

8 Optimization Optimization Unlike uniform quantization, the optimal reconstruction value is no longer at the center position of the interval, but it is the mean value of all signal values which are located in this interval or the mass centroid of the respective PDF partition Steinhaus/Lloyd Approach for avoiding non uniform quantization Iterative algorithms for determining Minimum Mean Square Error (MMSE) quantizer parameters Can be based on a pdf or training data Iterate between centroid condition and nearest neighbor condition Originally developed by Steinhaus (1955) and Lloyd (1957). Variously rediscovered as Forgey s algorithm, k-means [MacQueen (1967)] and Isodata [Ball and Hall (1965)]

9 Vector Quantization Source Coding Theorem [ Shannon, 1959] Rate-Distortion function R(D): For the encoding of a discrete, memoryless source, when a distortion smaller or equal to D shall be allowed, a block code of minimum rate R = R(D) + ε with ε> 0 exists. To approach R(D), the block length of the code must be sufficiently large. Distortion-Rate function D(R): 33 If a rate R is available for encoding of a discrete, memoryless source, the distortion can never become lower than D(R). 34 Rate Distortions Bounds for Gaussian Sources For most sources, determining R(D) for signals of arbitrary PDF is in quite complex and only possible by numeric approximations. Rate-Distortion Characterization of Lossy Coding Operational R(D) function of a quantizer: Relates rate and distortion: R(D) -> Minimum rate R needed to describe the source with distortion <=D RD optimal quantizer: Minimize D for given R or vice versa Alternatively:D(R)

10 RD Optimization Entropy Constrained Quantization