Audio /Video Signal Processing Lecture 2, Quantization, SNR Gerald Schuller, TU Ilmenau Quantization Signal to Noise Ratio (SNR). Assume we have a A/D converter with a quantizer with a certain number of bits (say N bits), what is the resulting Signal to Noise Ratio (SNR) of this quantizer? The SNR is defined as the ratio of the expectation of the signal power to the expectation of the noise power. In our case, the expectation of the noise power is the expectation of the quantization error power. We already have the expectation of the quantization error power as Δ 2 /12. So what we still need for the SNR is the expectation of the signal power. How do we obtain this? Basically we can take the same approach as we did for the expectation of the power of the quantization error (which is basically the second moment of the distribution of the quantization error). So what we need to know from our signal is its probability distribution. For the quantization error it was a uniform distribution between Δ /2 and + Δ/2. A very simple case would be a uniformly distributed signal with amplitude A, which has values between A/2 up to +A/2. pdf(x)
1/A -A/2 So here we have the same type of signal, and the resulting expectation of the power (its second moment) is obtained by using our previous formula and replace Δ by A. The resulting power is: A 2 12 +A/2 Which signals have this property? One example is uniformly distributed random values (basically like our quantization error). Matlab or Octave produces this kind of signal if we use the command: rand 0.5 (the 0.5 is to make the distribution centered around 0). Observe: Speech has a non uniform pdf, it is usually modeled by a Laplacian distribution or a gausssian mixture model, so it doesn't apply to this case! Another example: a triangular wave: +A/2 f(t) -A/2 Time t What is its pdf? One can imagine the vertical axis (the function value) covered by small intervals, and each interval is then passed in the same time span. This means that the resulting pdf is also uniform!
A further example: A sawtooth wave: f(t) t Again we can make the same argument, each small interval of our function value is covered in the same time span, hence we obtain a uniform distribution. We now have seen a few examples which fulfil our assumption of a uniform distribution (realistic examples), and we know: their expectation of their power is A²/12. So what does this then mean for the SNR? The SNR is just the ratio SNR= A2 /12 Δ 2 /12 = A2 Δ 2 If we assume our signal is full range, meaning the maximum values of our A/D converter is A/2 and +A/2 (the signal goes to the maximum), we can compute the step size Δ if we know the number of bits of the converter, and if we assume uniform quantization step sizes. Assume we have N bits in our converter, how big is Δ? Δ= A 2 N Plug this in the SNR equation, and we obtain:
SNR= A2 Δ 2= A2 ( A/2 N ) 2=22N This is now quite a simple result! But usually, the SNR is given in db, so lets convert it into db: SNR db =10 log 10 (2 2N )=10 2N log 10 (2) 10 2N 0.301dB=N 6.02 db This is now our famous rule of thumb, that each bit more gives you about 6 db more SNR. But observe that the above formula only holds for uniformly distributed full range signals, and for uniform quantizers! (the signal is between A/2 and +A/2, using all possible values of our converter) What happens if the signal is not full range? What is the SNR if we have a signal with reduced range? Assume our signal has an amplitude of A/c, with a factor c>1. We can then simply plug this into our equation: SNR= ( A/c)2 Δ 2 in db: = ( A/c)2 22N ( A/2 N ) 2= c 2 SNR db =10 log 10 ( 22N c 2 )=10 2N log 10(2) 20 log 10 (c) 10 2N 0.301 db 20 log 10 (c)= =N 6.02 db 20 log 10 (c) The last term, the 20 log 10(c), is the number of db which we are below our full range with our signal. This means
we reduce our SNR by this number of db which we are below full range! Example: We have a 16 bit quantiser, then the SNR for uniformly distributed full range signals would be SNR=6.02 16 db=96.32 db Now assume we have the same signal, but 20dB below full range. Then the resulting SNR would be only 96.32 20=76.32 db! This is considerably less. This also shows why it is important not to make the safety margin to full range too big! So for instance our sound engineer should keep the signal as big as possible, without ever reaching full range to avoid clipping the signal. This is also the reason for using 24 or 32 bit/sample A/D converters, they offer a higher savety margin for our SNR, here we could be much below full range with our signal and still have an SNR which delivers inaudible quantization errors. The other assumption we made concerned the type of signal we quantize. What if we don't have a uniformly distributed signal? As we saw, speech signals are best modeled by a Laplacian distribution or a gaussian mixture model, and similar for audio signals. Or even a simple sine wave does not fulfil this requirement. What is the probability density function (pdf) of a simple sine wave? sin(t) t
What is our SNR if we have a sinusoidal signal? What is its pdf? If we look at the signal, and try to see how probable it is for the signal to be in a certain small interval on the y axis, we see that the signal stays longest around +1 and 1, because there the signal slowly turns around. Hence we would expect a pdf, which has peaks at +1 and 1. If you calculate the pdf of a sine wave, x=sin(t), with t being continuous and with a range larger than 2pi, then the result is 1 p(x)= π 1 x 2 pdf(sin)
This results from the derivative of the inverse sine function (arcsin). This derivation can be found for instance on Wikipedia. Here we can see that p(x) indeed becomes infinite at x=+/ 1! We could now use the same approach as before tp obtain the expectation of the power, multiplying it with x^2 and integrating it. But this seems to be somewhat tedious. But since we now have a deterministic signal, we can also try an alternative solution. We can simply directly compute the power of our sine signal over t, and then take the average over at least one period of the sine function. E (x 2 )= 1 2π t=0 2 π sin 2 (t)dt= 1 2π t=0 x 2 π (1 cos(2t))/2 dt = 1 2 2π π 1 1/2 dt= t=0 2 π π=1 2 What do we get for a sinusoid with a different amplitude, say A/2 sin (t)? The expected power is So this leads to an SNR of E (x 2 )= A2 8 SNR= A2 /8 Δ 2 /12 = 3 A2 2 Δ 2
Now assume again we have a A/D converter with N bits, and the sinusoid is at full range for this converter. Then A=2 N Δ We can plug in this result into the above equation, and get SNR= 3 22N Δ 2 =1.5 2 2N 2 Δ 2 In db this will now be 10 log 10 (SNR)=10 log 10 (1.5)+ N 20 log 10 (2)= =1.76 db+ N 6.02 db Here we can see now, that using a sinusoidal signal instead of a uniformly distributed signal gives us a boost of 1.76 db in SNR. This is because it is more likely to have larger values! Companding This is a synonym for compression and expanding. Uniform quantization can be seen as a quantization value which is constant on the absolute scale. Non uniform quantization, using companding, can be seen as having step sizes which stay constant relative to the amplitude of the signal, their step size grows with the amplitude. Range of (index) values is compressed, smaller values become larger, large values become smaller:
(From : http://w w w.dspguide.com /ch22/5.htm, also below ) This type of non linear quantizer is usually used for speech coding, for instance in the mentioned ITU G.711 speech coding standard. It has the effect the the SNR becomes less dependent on the amplitude of the signal, since the quantisation step size is roughly proportional to the signal amplitude. In the decoder, the inverse function is used for the expansion, which in this case is an exponential function as the inverse of the log function above.
Lloyd Max Quantizer This is a type of non uniform quantizer, which is adapted to the signals pdf. It basically minimizes the expectation of the quanization power (its second moment), given the pdf of the signal to quantize. It can be imagined as having small quantisation intervals at signal values which are more probable, and larger step sizes for values which are less likely, such that the average error is reduced. Let's call our Quantisation function Q(x) (this is quantization followed by inverse quantization). You can also think of nonlinear quantization as first applying this non linear function and then to use uniform quantization. Then the expectation of our quantization power is D=E ((x Q(x)) 2 ) Observe that we use the square here, and not for instance the magnitude of the error, because the square leads to an easier solution for minimum, which we would like to find.