Overview Analog capturing device (camera, microphone) Sampling Fine Quantization A/D CONVERTER PCM encoded or raw signal ( wav, bmp, ) Transform Quantizer VLC encoding Compressed bit stream (mp3, jpg, ) COMPRESSION/SOURCE CODING Encipher Error protect. Channel bit stream CHANNEL CODING -1-
Reduce the #Amplitudes 24 bit (16777200 different colors) -2-
Reduce the #Amplitudes 8 bit (256 different colors) CF 3-3-
Reduce the #Amplitudes 6 bit (64 different colors) CF 4-4-
Reduce the #Amplitudes 4 bit (16 different colors) CF 6-5-
Reduce the #Amplitudes 1 bit (2 different colors) CF 24-6-
Reduce the #Amplitudes 8 bit (256 different gray values) CF 3-7-
Reduce the #Amplitudes 4 bit (16 different gray values) CF 6-8-
Reduce the #Amplitudes 3 bit (8 different gray values) CF 8-9-
Reduce the #Amplitudes 2 bit (4 different gray values) CF 12-10-
But there are More Cleaver Ways JPEG CF 15-11-
Why can Signals be Compressed? Because infinite accuracy of signal amplitudes is (perceptually) irrelevant 24 bit (16777200 different colors) 8 bit (256 different colors) Compression factor 3 Rate-distortion theory, scalar/vector quantization -12-
Why can Signals be Compressed? Because signal amplitudes are statistically redundant Information theory, Huffman coding -13-
Why can Signals be Compressed? Because signal amplitudes are mutually dependent 250 200 150 S 100 50 0 500 1000 1500 2000 2500 3000 3500 4000 4500 200 150 O 100 0 1000 2000 3000 4000 5000 Rate-distortion theory, transform coding -14-
Why can Signals be Compressed? Example of signal with no dependencies between successive amplitudes (Gaussian uncorrelated noise) Indeed noise compresses badly -15-
System Overview Analog capturing device (camera, microphone) Sampling Fine Quantization A/D CONVERTER PCM encoded or raw signal ( wav, bmp, ) Transform Quantizer VLC encoding Compressed bit stream (mp3, jpg, ) COMPRESSION/SOURCE CODING Encipher Error protect. Channel bit stream CHANNEL CODING -16-
Quantization Even Uniform Odd Uniform p(x) Q(x) Even Non-uniform Odd Non-uniform Quantizer level more probable -17-
Information and Entropy More Probable Less information -18-
Huffman Code y 4 0.51 0.51 (0) 0 y 3 0.20 0.29 (0) 0.49 (1) 11 y 2 0.14 0.15 (0) 0.20 (1) 101 y 5 0.08 0.08 (0) 0.14 (1) 1000 y 6 0.04 0.04 (0) 0.07 (1) 10010 y 1 0.02 0.02 (0) 0.03 (1) 100110 y 7 0.005 (0) 0.01 (1) 1001110 y 0 0.005 (1) 1001111-19-
Run Length Coding Representing 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1 1 1 1 0 0... Runs : 8 ( zeros ) 2 ( ones ) 6 ( zeros ) 5 ( ones ) The run lengths are also encoded (e.g. with Huffman coding) Efficient transforms (like DCT) used in compression produce A lot of zero values And a few (significant) non-zero values Typical symbol sequences to be coded 5 1 0 0 0 0 0 0 0 3 0 0 6 0 0 0 0 1 0 0 0 0.. will be done by {zero-run, non-zero symbol/0} pairs Here: {0,5},{0,1},{7,3},{2,6},{4,1},.. The pairs will now be assigned a Huffman code This is used in JPEG -20-
General Compression System Analog capturing device (camera, microphone) Sampling Fine Quantization A/D CONVERTER PCM encoded or raw signal ( wav, bmp, ) Transform Quantizer VLC encoding Compressed bit stream (mp3, jpg, ) COMPRESSION/SOURCE CODING Encipher Error protect. Channel bit stream CHANNEL CODING -21-
Correlation in Signals - I Meaningful signals are often highly predictable: 250 4 100 Δx(n)=x(n)-x(n-1) 200 150 0 100 50 0 500 1000 1500 2000 2500 3000 3500 4000 4500 100 500 1000 1500 2000 2500 3000 3500 4000 4500 (Linear) Predictability has something to do with the autocorrelation function n -22-
Principle of Differential PCM x(n) Δx(n) Δx * (n) 001010010 xˆ ( n) - Q VLC Predict x(n) Previous signal values ( past ) x(n-1), x(n-2), x(n-3),. -23-
Works Really Good - I Variance = 3240-24-
Works Really Good - II Variance = 315-25-
DPCM Signal to be encoded Prediction difference/error Predicted signal Reconstructed signal -26-
What Linear Predictor to Use? Examples: PCM xn $( ) = 0 Simple differences xn $( ) = ~ xn ( 1) Average last two samples General linear predictor xn $( ) = hxn ~ ( 1 ) + hxn ~ ( 2 ) N k= 1 1 2 xn $( ) = hxn ~ ( k) k -27-
DPCM on Images Same principle as 1-D Definition of Past and Future in Images: Predictions: horizontal (scan line) vertical (column) 2-dimensional -28-
General Compression System Analog capturing device (camera, microphone) Sampling Fine Quantization A/D CONVERTER PCM encoded or raw signal ( wav, bmp, ) Transform Quantizer VLC encoding Compressed bit stream (mp3, jpg, ) COMPRESSION/SOURCE CODING Encipher Error protect. Channel bit stream CHANNEL CODING -29-
Transform Coding x(n) x θ Vectorize channel T $ θ T -1 $x xn $( ) Decorrelating Transform Q Q -1 Correlating Transform Inverse Vectorize Which transform used? -30-
Removing Correlation - I 100 50 0 50 100 x x 1 2 0 200 400 600 800 1000 x 2 40 20 x 1 40 20 0 20 40 20 40-31-
Removing Correlation - II 100 50 0 50 100 x x 1 2 0 200 400 600 800 1000 x 2 40 2 σ 2 20 2 σ 1 θ 1 x 1 40 20 0 20 40 Rotate the coordinates 20 θ 2 40-32-
Removing Correlation - III 50 0 50 0 100 200 300 400 500 600 700 800 900 x x 1 2 x 2 40 20 θ 1 x 1 40 20 0 20 40 20 θ 2 40-33-
θ 1 t11, L t1, θ = = M M O M θ t L t N t 1 N N, 1 NN, N Decomposition x 1 M = Tx x N x = θk t k = θ1 t1 + θ2 t 2+ L + θnt k = 1 θ 0.707 0.25-0.125 0.0 0.0 0.063 N = t x k = 12,, K, N k k, n n n = 1 N x = θ t n = 12,, K, N n k n, k k = 1 x N -0.702 3.377 0.638 1.020 4.516 2.266 1.718 3.165 (Example) t 8-0.2-0.025 θ -34-
Discrete Cosine Transform N=8 t 1 t 2 0.354 0.354 0.49 0.416 0.462 0.191 0.416 0.098 0.354 0.354 0.278 0.49 0.191 0.462 0.098 0.278 t 3 t 4 T t = 0.354 0.354 0.354 0.354 0.278 0.098 0.098 0.278 0.191 0.462 0.462 0.191 0.49 0.278 0.278 0.49 0.354 0.354 0.354 0.354 0.098 0.416 0.416 0.098 0.462 0.191 0.191 0.462 0.416 0.49 0.49 0.416 t 5 0.354 0.354 0.416 0.49 0.191 0.462 0.098 0.416 0.354 0.354 0.49 0.278 0.462 0.191 0.278 0.098 t 6 t 7 (All picture compression standards implement this matrix in a clever way) t 8-35-
Image Transforms - I 6 0 4 6 = 1 1 1-1 1 1 4 1-1 2 1 1 1-1 -1-1 1-1 -1 1 x mn, t 11 t 12 t 21 t 22 θ 11 θ 12 θ 21 θ 22-36-
Basis Images for 2-D DCT -37-
Discrete Cosine Transform (DCT) (1) 8 rows 8 pixels -38-
Discrete Cosine Transform (DCT) (2) 64 pixels Cosine patterns/dct basis functions -39-
Discrete Cosine Transform (DCT) (3) 89 8 8 1 0-1 1 0 2 3 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 64 pixels 2 0 0 1 0 1 0 0 0 0-1 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 Cosine patterns/dct basis functions -40-
Discrete Cosine Transform (DCT) (4)! -41-
2-D DCT of Picture - I -42-
2-D DCT of Image - II A DCT coefficients is a weight of particular DCT basis function. -43- High High frequencies frequencies Low Low frequencies frequencies
Grouped DCT Coefficients -44-
JPEG Compression System -45-
DCT Weight Matrix and Quantization Quantization: $ θ = Q[ θ ] = kl, kl, round θ kl, ' kl, QN Recommended JPEG normalization matrix N kl, = 16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99-46-
User Controllable Quality User has control over a quality parameter that runs from 100 ( perfect ) to 0 ( extremely poor ) Parameter used to scale the normalization matrix Q' 6 5 4 3 2 1 0 0 10 20 30 40 50 60 70 80 90 100 Q Increasing quality -47-
Entropy coding Huffman coded Runlength coding then huffman or arithmetic coding -48-
Example average * 8 f = kl, θ = kl, 139 144 149 153 155 155 155 155 144 151 153 156 159 156 156 156 150 155 160 163 158 156 156 156 159 161 162 160 160 159 159 159 159 160 161 162 162 155 155 155 161 161 161 161 160 157 157 157 162 162 161 163 162 157 157 157 162 162 161 161 163 158 158 158 1260 1 12 5 2 2 3 1 23 17 6 3 3 0 0 1 11 9 2 2 0 1 1 0 7 2 0 1 1 0 0 0 1 1 1 2 0 1 1 1 2 0 2 0 1 1 1 1 1 0 0 1 0 2 1 1 3 2 4 2 2 1 1 0 DCT transform is exactly defined in JPEG standard -49-
Example $ θ = kl, 79 0 1 0 0 0 0 0 2 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Quantization using Q = 1 DC: Difference with quantized DC coefficient of previous block is Huffman encoded AC: Zig-zag scan coefficients, and convert to (zero runlength, amplitude) combinations: (79) 0-2 -1-1 -1 0 0-1 EOB {1,-2}{0,-1}{0,-1}{0,-1} {2,-1} EOB -50-
VLC Coding of AC Coefficients The (zero run-length, amplitudes) are put into categories Category AC Coefficient Range 1-1,1 2-3,-2,2,3 3-7,...,-4,4,...,7 4-15,...,-8,8,...,15 5-31,...,-16,16,...,31 6-63,...,-32,32,...,63 7-127,...,-64,64,...,127 8-255,...,-128,128,...,255 9-511,...,-256,256,...,511 10-1023,...,-512,512,...,1023 The (zero run-length, categories) are Huffman encoded The sign and offset into a category are FLC encoded (required #bits = category number) -51-
JPEG AC Huffman Table Zero Run Category Code length Codeword 0 1 2 00 0 2 2 01 0 3 3 100 0 4 4 1011 0 5 5 11010 0 6 6 111000 0 7 7 1111000....... 1 1 4 1100 1 2 6 111001 1 3 7 1111001 1 4 9 111110110........ 2 1 5 11011 2 2 8 11111000........ 3 1 6 111010 3 2 9 111110111........ 4 1 6 111011 5 1 7 1111010 6 1 7 1111011 7 1 8 11111001 8 1 8 11111010 9 1 9 111111000 10 1 9 111111001 11 1 9 111111010........ EOB 4 1010-52-
Example - III The series {1,-2}{0,-1}{0,-1}{0,-1} {2,-1} EOB now becomes 111001 01/00 0/00 0/00 0/11011 0/1010 Bit rate for AC coefficients in this DCT block 27 bits/64 pixels = 0.42 bit/pixel -53-
Ordering of Coefficient -54-
Sequential Encoding -55-
Acknowledgement Prof. Inald Lagendijk, Delft University of Technology Also try the software VcDemo http://ict.ewi.tudelft.nl/index.php?itemid=124-56-