Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE

Transcription

1 General e Image Coder Structure Motion Video x(s 1,s 2,t) or x(s 1,s 2 ) Natural Image Sampling A form of data compression; usually lossless, but can be lossy Redundancy Removal Lossless compression: predictive coding transform coding subband coding Quantization Lossy compression; typically removes less significant or irrelevant information Run-length & Entropy Coding Lossless compression; variable length coding Channel coding (adds redundancy for error correction) Perform inverse operations at the receiver

2 Lossless Entropy Coding: Huffman Coding Exp. L=4 Huffman Coding Algorithm Given L symbols to be coded: 1. Arrange the symbol probability p i in descending order and consider them as leaf nodes of a tree 2. While there is more than one node Merge the two nodes with smallest probability to form a new node whose probability = sum of the two merged nodes Arbitrarily assign 1 or 0 to each pair of branches merging into a node (e.g. 0 to left branch, 1 to right branch or vice versa) 3. Read sequentially from root node to leaf where symbol is located s 1 : p 1 =0.1; s 2 : p 2 =0.2; s 3 : p 3 =0.4; s 4 : p 4 = s 3 s s 2 s 1 0.1

3 Lossless Entropy Coding: Huffman Coding Huffman optimal in the sense that its average bit-rate (B) does not exceed the average bit-rate of any other code Assumption: each symbol coded separately and assigned a codeword with integer length Huffman codes are also nown as prefix codes since they have a unique prefix attribute: shorter codewords cannot be prefix of longer ones can be uniquely decoded without nowing length of codeword in advance Huffman codes not unique but all optimal (in the above sense) Disadvantage: Each symbol coded using integer number of bits Minimum possible codeword length is 1 Cannot obtain less than 1 bit per pixel with pure entropy coding Solutions: Combine symbols together and code jointly Run-length coding used before entropy coding

4 Lossless Entropy Coding: Huffman Coding Average bit-rate of Huffman coding is within one bit of entropy (depending on p i s) H B H+1 B = H = lower bound if p i are powers of ½

5 Lossless Coding: Run-Length Coding Run-Length Coding Example: Consider binary document (white & blac) A SU Text document 0 white 1 blac Give pixel value and number of pixels in a given row (0,22), (1,10), (0,108) - (value, number of pixels with value) Then, one can possibly Huffman code these pairs

6 Arithmetic coding Basic idea: Map input sequence of symbols into one single codeword Symbol blocing not needed Codeword is determined and updated incrementally with each new symbol (symbol-by-symbol coding) At any time, the determined codeword uniquely represents all the past occurring symbols Codeword is represented by a half-open subinterval [L c,h c ) [0,1) The half-open subinterval gives the set of all codewords that can be used to encode the input symbol sequence, which consists of all past input symbols any real number within the subinterval [L c,h c ) can be assigned as the codeword representing the past occurring symbols

7 The selected real-valued codeword is transmitted in binary form (fractional binary representation) With every new symbol, the subinterval is updated by finding new subinterval [L c,h c) [L c,h c ) The codeword d subinterval has length equal to the probability bilit of occurrence of the corresponding encoded input sequence less probable sequence shorter interval more precision bits required longer codeword Summary: The arithmetic encoding procedure constructs, in a hierarchical manner, a code subinterval which uniquely represents a sequence of successive symbols

8 Let S = { s 0,, s (N-1) } source alphabet p = P(s ) probability of symbol s, 0 ( N 1 ) [ L, H ) s s - interval assigned to symbol s, where p = H L s s The subinterval limits can be computed as: L H s 1 = p = 1 ; P i= 0 = s i= 0 p = P ; 0 0 ( N 1 ) ( N 1)

9 1. Coding begins by dividing interval [0,1) into N non-overlapping i t l ith l th l t b l b biliti intervals with lengths equal to symbol probabilities p Source symbol Probability Symbol Subinterval L, H s 2. L c = 0; H c = 1 p [ ) s s [0,0.1) s [0.1,0.4) s [0.4,0.8) s [0.8,1) s

10 3. Calculate code subinterval length: length = H c L c 4. Get next input symbol s 5. Update the code subinterval L c = L c + length L s H c = L c + length H s 6. Repeat from Step 3 until all the input sequence has been encoded s s s 2 s Input sequence: s 1 s 0 s 2 s 3 s 3 code interval

11 Iteration ti # Encoded d symbol Code Subinterval I s [ L c, H c ) 1 s 1 [ 0.1, 0.4 ) 2 s 0 [ 0.1, 0.13 ) 3 s 2 [ 0.112, ) 4 s 3 [ , 0.124) 5 s 3 [ , ) Incremental coding Begin outputing the leading bit of the result as soon as it can be determined Example: At second iteration, the leading part 0.1 can be output since it is not going to be changed by future encoding steps

12 Decoding procedure: 1. L c = 0 ; H c = 1 2. Calculate code subinterval length: length = H c L c 3. Find symbol subinterval L, H, 0 ( N 1)such that [ ) s s L codeword length H c L s < s 4. Output symbol s 5. Update the code subinterval L c = L c + length L s H c = L c + length H s 6. Repeat from Step 2 until last symbol is decoded. In order to determine when to stop the decoding: A special end-of-sequence symbol can be added to the source alphabet. If fixed-length bloc of symbols are encoded, the decoder can simply eep a count of the number of decoded symbols.