The information loss in quantization

Size: px

Start display at page:

Download "The information loss in quantization"

Melinda Chambers
5 years ago
Views:

1 The information loss in quantization The rough meaning of quantization in the frame of coding is representing numerical quantities with a finite set of symbols. The mapping between numbers, which are normally real values, and the corresponding symbols from a finite set is an irreversable operation because it is a many-to-one mapping. A symbol at the quantizer output can be generated with many inputs which have similar values. In this scheme, the exact information of x(t) is lost, and only an approximation of x(t) can be recovered from y(t) using an inverse operation of the quantizer. The amount of loss due to the quantizer is represented by where d(.) is a distance measure. Usually or. We will call this difference quantization error. A very common class of quantizers are known as "scalar quantizers". Scalar quantizers have a very easy interpretation in the form of algebraic analysis. First, let's start with the well known rounding operator as a quantization operation:. The operator yields integer numbers which are nearest to the number, x. Obviously, we cannot totally recover x from q(x). More generally, we describe a scalar quantizer by A list of output levels (will be called codebook or reproduction values ) A set of disjoint intervals Whenever a value is numerically in an interval the quantizer output becomes. Quantizer block, therefore, can be analyzed in two components: Encoder: and or an equivalent set of of binary vectors. Decoder: Overall Quantizer: and the reproduction codebook is

. The coding efficiency of the quantizer comes from the fact that, if you have a total of M quantization levels, then each symbol can be represented by at most bits (usually less bits per symbol can

Attention: After the binary representation of the symbol the decompressor, which tries to reconstruct the data from the compressed bit-stream, should also know all of the quantization values (or the

2 . The coding efficiency of the quantizer comes from the fact that, if you have a total of M quantization levels, then each symbol can be represented by at most bits (usually less bits per symbol can be obtained using other compression techniques). Normally, a high precision real valued number should be represented with infinite (or very high number of) bits, therefore there is a compression. Attention: After the binary representation of the symbol the decompressor, which tries to reconstruct the data from the compressed bit-stream, should also know all of the quantization values (or the codebook ). This is true for most of the cases since the encoders and decoders both behave on a pre-determined compression algorithm. The number of these levels or intervals determine the precision of the quantizer. Let us assume that i=1..m, so there will be M output levels. Formally, the overall relation can be written as for "Obviously, the biggest issue is to select the intervals and reproduction levels the best way to minimize the quantization error at a given number of quantization levels, M." The simplest quantizer is the uniform scalar quantizer. The rounding operator that we have mentioned above is a special uniform scalar quantizer with the intervals and the output levels. Consider the figure below. The illustrated quantizer quantizes the input signal x(n) using uniform quantization steps to produce y(n). As you see, y(n) is only an approximate versron of the original x(n), and some of the precision is lost. The input-output caracteristics of such a scalar quantizer can be illustrated as:

3 The above figure shows a uniform scalar quantizer with M=8 levels and interval width of. If a large interval around zero is quantized to value zero, the quantizer is called a deadzone quantizer. An example is: The design issue for a quantizer is to minimize the reproduction error. We will come back to the design concepts later, however you should start thinking about ways to make the information loss as small as possible. Using the above error notation, we can model the quantization error as an additive noise on top of the signal x, therefore the quantizer output is. The graphical chart is:

4 The optimization of the quantizer by minimizing the error generally produces a nonuniform quantizer. In the non-uniform case, the intervals are not the same for all i. Let us denote the general i-th interval as. In this notation are the thresholds which form an increasing sequence. We can say that the width of the i-th interval is. Let us also denote the output of the quantizer to an input in the i- th interval as. A general non-uniform quantizer may look like: Notice that the first interval starts from -infinity and the last interval ends at +infinity. The quality of the quantizer is measured by the amount of error where d(.) is a distance measure as described above. The average error (which will be called distortion) can be written as (Eq. 1) using the fact that the input signal is treated as a random signal. The E{.} operator means the expected value of the argument. Attention: If you are uncomfortable with expected values and random variables, please refer to any book about probability and random processes. You can find a very brief review with basic properties about random processes and random signals here. From the definition of expected value, we can write: (Eq. 2) where is the Probability Distribution Function (PDF) of the random variable x. Minimizing the distortion D will maximize the reproduction quality.

5 In practice, you may not know the PDF of the input, and you may only observe L samples from the data. In that case, the distortion can be approximated by. The distortion expression in Eq. 1 and Eq. 2 can be formed into an addition of individual distortions in separate intervals: (Eq. 3) where, distortion measure is (MSE). is the probability of being in the i-th interval. If the then the distortion is called the Mean Squared Error Exercise 1: Find the MSE of the level uniform quantizer formed by selecting the interval widths as and the quantization values equal to the mid-points of the intervals. Assume that a random signal with uniform PDF in the range is fed to the quantizer. (This is also known as R-bit uniform quantizer). Answer: The MSE is Eq. 4 This is an important result, and the MSE found for the above example approximately holds for uniform quantizers applied to signals with different PDF's if the quantization intervals are taken small. When the intervals are small enough, one can assume that the PDF is almost uniform within that small range, and the above distortion expression holds. Homework 1: In the above example, the uniform quantizer was acting on a random

6 signal whose PDF is uniform between -X max and X max. The quantization interval was and the quantiaztion output was the midpoint of each interval : If input was between (i-1) and i (corresponding to the ith interval) then the quantizer output was. We saw that the quantization error (, when ) has a power (called MSE) of. The uniform PDF is illustrated at the left part of the figure below. For the homework, consider the same quantizer, but this time the input signal has a PDF as shown at the right part of the figure below (the triangle shaped). The quantizer again uses interval width of and output levels as the mid point of the interval (just like before). What is the MSE of the quantizer to this input? Another important consequence of the above result is about the relation between the amount of bits (R) and distortion D. Clearly, D is proportional to and is inversely proportional to R. Precisely,. This relation implies that if you increase R, the distortion is decreased. This is a famous result, know as the rate-distortion function in the information theory. More precisely, we define which is the minimum attainable distortion at a given rate R. The plot of versus R is known as the rate-distortion curve. Practically, a quantizer distortion at a given R is above this curve. A good quantizer produces distortions at given rates R as near to the rate-distortion curve as possible. An alternative definition of is and the minimum is over all codebooks (or reproduction levels and all interval partitions. The typical characteristics of a rate-distortion curve and a practical quantizer performance is illustrated as follows:

Exercise 2: A sinusoidal signal with peak-to-peak amplitude 5V is quantized by a R- bit Analog-to-Digital Convertor (ADC). Assuming the quantization intervals are uniform, determine: 1.

Answer: The quantiaztion step size is: We know that the signal power fo a sinusoid with amplitude A is given as the above step size, the SQNR (in decibels) is given as:.

7 Exercise 2: A sinusoidal signal with peak-to-peak amplitude 5V is quantized by a R- bit Analog-to-Digital Convertor (ADC). Assuming the quantization intervals are uniform, determine: 1. The quantization step size. 2. The root-mean-squared (RMS) signal to noise (due to quantization) ratio SQNR. 3. The SQNR for a 16 bit ADC. Answer: The quantiaztion step size is: We know that the signal power fo a sinusoid with amplitude A is given as the above step size, the SQNR (in decibels) is given as:.using which indicates that each aditional bit adds to the SQNR performance by approximately 6dB - known as the 6dB per bit rule. Using this result, the 16 bit ADC gives SQNR of 6.02x =98dB (CD quality). In the following sections we will elaborate more on quantization, its techniques and variations. However, I encourage you to keep in mind where in the coding framawork we are this week. The basic steps of a typical coder was given in the last week's introduction. This week, we are covering the typical middle block of the coding scheme, called quantization, where the loss and bit reductions occur. Before the quantization, there is usually a predictive coder or a transform coder block and after the quantization, there is usually an entropy coder to compress the bits produced by the quantizer.

8 Now that we have an understanding about how much information loss we have for a quantizer at a given rate, we can proceed to the quantizer design techniques.

at Some sort of quantization is necessary to represent continuous signals in digital form

Quantization at Some sort of quantization is necessary to represent continuous signals in digital form x(n 1,n ) x(t 1,tt ) D Sampler Quantizer x q (n 1,nn ) Digitizer (A/D) Quantization is also used for