Source Coding Techniques

Transcription

1 Source Coding Techniques. Huffman Code. 2. Two-pass Huffman Code. 3. Lemple-Ziv Code. 4. Fano code. 5. Shannon Code. 6. Arithmetic Code.

2 Source Coding Techniques. Huffman Code. 2. Two-path Huffman Code. 3. Lemple-Ziv Code. 4. Fano Code. 5. Shannon Code. 6. Arithmetic Code.

3 Source Coding Techniques. Huffman Code. With the Huffman code in the binary case the two least probable source output symbols are joined together, resulting in a new message alphabet with one less symbol take together smallest probabilites: P(i) + P(j) 2 replace symbol i and j by new symbol 3 go to - until end Application examples: JPEG, MPEG, MP3

4 . Huffman Code. ADVANTAGES: uniquely decodable code smallest average codeword length DISADVANTAGES: LARGE tables give complexity sensitive to channel errors

5 . Huffman Code. Huffman is not universal! it is only valid for one particular type of source! For COMPUTER DATA data reduction is lossless no errors at reproduction universal effective for different types of data

6 Huffman Coding: Example Compute the Huffman Code for the source shown H( S ) = ( 04log. ) ( 02log. ) ( 0log. ) 2 0. = L Source Symbol s k 0. s 0 s 0.2 s s s 4 Symbol Probability p k

7 Solution A Source Symbol s k s 2 s s 3 s 0 s 4 Stage I

8 Solution A Source Symbol s k s 2 s s 3 s 0 s 4 Stage I Stage II

9 Solution A Source Symbol s k Stage I Stage II Stage III s s s s s 4 0.

10 Solution A Source Symbol s k Stage I Stage II Stage III Stage IV s s s s s 4 0.

11 Solution A Source Symbol Stage I Stage II Stage III Stage IV s s k s s 3 s s 4 0.

12 Solution A Source Symbol Stage I Stage II Stage III Stage IV Code s s k s s 3 s s

13 Source Symbol Symbol Probability p k Solution A Cont d s s s s s 4 Code word c k s k ( ) = H S L = = 22. ( S ) L ( S ) H < H + THIS IS NOT THE ONLY SOLUTION!

14 Another Solution B Source Symbol Stage I Stage II Stage III Stage IV s s k 0 Code 0.6 s s 3 s s

15 Another Solution B Cont d Source Symbol Symbol Probability p k s s s s s 4 Code word c k s k ( ) = H S L = = 22. ( S ) L ( S ) H < H +

16 What is the difference between the two solutions? They have the same average length They differ in the variance of the average code length Solution A s 2 =0.6 Solution B s 2 =.36 σ 2 K ( ) 2 p l L k k k = 0 =

17 Source Coding Techniques. Huffman Code. 2. Two-pass Huffman Code. 3. Lemple-Ziv Code. 4. Fano Code. 5. Shannon Code. 6. Arithmetic Code.

18 Source Coding Techniques 2. Two-pass Huffman Code. This method is used when the probability of symbols in the information source is unknown. So we first can estimate this probability by calculating the number of occurrence of the symbols in the given message then we can find the possible Huffman codes. This can be summarized by the following two passes. Pass : Measure the occurrence possibility of each character in the message Pass 2 : Make possible Huffman codes

19 Source Coding Techniques 2. Two-pass Huffman Code. Example Consider the message: ABABABABABACADABACADABACADABACAD 0

20 Source Coding Techniques. Huffman Code. 2. Two-path Huffman Code. 3. Lemple-Ziv Code. 4. Fano Code. 5. Shannon Code. 6. Arithmetic Code.

21 Lempel-Ziv Coding Huffman coding requires knowledge of a probabilistic model of the source This is not necessarily always feasible Lempel-Ziv code is an adaptive coding technique that does not require prior knowledge of symbol probabilities Lempel-Ziv coding is the basis of well-known ZIP for data compression

22 Lempel-Ziv Coding History GIF, TIFF, V.42bis modem compression standard, PostScript Level published by Abraham Lempel and Jakob Ziv 984 LZ-Welch algorithm published in IEEE Computer Sperry patent transferred to Unisys (986) GIF file format Required use of LZW algorithm Universal Lossless

23 Lempel-Ziv Coding Example Codebook Index Subsequence 0 Representation Encoding

24 Lempel-Ziv Coding Example Codebook Index Subsequence 0 00 Representation Encoding

25 Lempel-Ziv Coding Example Codebook Index Subsequence Representation Encoding

26 Lempel-Ziv Coding Example Codebook Index Subsequence Representation Encoding

27 Lempel-Ziv Coding Example Codebook Index Subsequence Representation Encoding

28 Lempel-Ziv Coding Example Codebook Index Subsequence Representation Encoding

29 Lempel-Ziv Coding Example Codebook Index Subsequence Representation Encoding

30 Lempel-Ziv Coding Example Codebook Index Subsequence Representation Encoding

31 Lempel-Ziv Coding Example Information bits Source encoded bits Codebook Index Subsequence Representation Source Code

32 How Come this is Compression?! The hope is: If the bit sequence is long enough, eventually the fixed length code words will be shorter than the length of subsequences they represent. When applied to English text Lempel-Ziv achieves approximately 55% Huffman coding achieves approximately 43%

33 Encoding idea Lempel Ziv Welch-LZW Assume we have just read a segment w from the text. a is the next symbol. w a If wa is not in the dictionary,?write the index of w in the output file.?add wa to the dictionary, and set w a.?if wa is in the dictionary,?process the next symbol with segment wa. a

34 LZ Encoding example address 0: a address : b address 2: c Input string: a a b a a c a b c a b c b a a b a a c a b c a b c b output update a a a a b a a b a a a b a a c aa not in dictionry, output 0 add aa to dictionary continue with a, store ab in dictionary a a b a a c a a a b a a c a b c a a b a a c a b c a b continue with b, store ba in dictionary aa in dictionary, aac not, 0 aa 3 0 ab 4 ba 5 3 aac 6 2 ca 7 4 abc 8 7 cab 9 aabaacabcabcb LZ Encoder

35 UNIVERSAL (LZW) (decoder). Start with basic symbol set 2. Read a code c from the compressed file. - The address c in the dictionary determines the segment w. - write w in the output file. 3. Add wa to the dictionary: a is the first letter of the next segment

36 LZ Decoding example address 0: a address : b address 2: c Output String: Input update a? Output a 0 a a! a a b. output a determines? = a, update aa output determines!=b, update ab 0 aa 3 ab 4 a a b a a. a a b a a c. a a b a a c a b. a a b a a c a b c a ba 5 aac 6 ca 7 abc LZ Decoder aabaacabcabcb

37 Exercise. Find Huffman code for the following source Symbol h e l o w r d Probability Find LZ code for the following input

38 Source Coding Techniques. Huffman Code. 2. Two-path Huffman Code. 3. Lemple-Ziv Code. 4. Fano Code. 5. Shannon Code. 6. Arithmetic Code.

39 4. Fano Code. The Fano code is performed as follows:. arrange the information source symbols in order of decreasing probability 2. divide the symbols into two equally probable groups, as possible as you can 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol 4. repeat steps 2 and 3 per group as many times as this is possible. 5. stop when no more groups to divide

40 4. Fano Code. Example :. arrange the information source symbols in order of decreasing probability Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

41 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

42 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code 0 0

43 4. Fano Code. Example : 4. repeat steps 2 and 3 per group as many times as this is possible. Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code 0 0

44 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code 0 0

45 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code 0 0 0

46 4. Fano Code. Example : Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code 0 0 0

47 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code 0 0 0

48 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

49 4. Fano Code. Example : 4. repeat steps 2 and 3 per group as many times as this is possible. Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

50 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

51 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

52 Example : 4. Fano Code. J I H G /6 F /6 E /8 D /8 C /4 B /4 A Fano Code Probability Symbol

53 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

54 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

55 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

56 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

57 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

58 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

59 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

60 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

61 4. Fano Code. Example : 2. divide the symbols into two equally probable groups, as possible as you can Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

62 4. Fano Code. Example : 3. each group receives one of the binary symbols (i.e. 0 or ) as the first symbol Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

63 4. Fano Code. Example : 5. stop when no more groups to divide Symbol A B C D E F G H I J Probability /4 /4 /8 /8 /6 /6 Fano Code

64 4. Fano Code. Note that: If it was not possible to divide precisely the probabilities into equally probable groups, we should try to make the division as good as possible, as we can see from the following example. Example 2: Symbol T U V W X Probability /3 /3 /9 /9 /9 Fano Code

65 Source Coding Techniques. Huffman Code. 2. Two-path Huffman Code. 3. Lemple-Ziv Code. 4. Fano Code. 5. Shannon Code. 6. Arithmetic Code.

66 5. Shannon Code. The Shannon code is performed as follows:. calculate a series of cumulative probabilities = n q k p i i=, k=,2,,n 2. calculate the code length for each symbol using log( \p i ) = l i < log ( \p i ) + 3. write q k in the form c c c li 2 -li where each c i is either 0 or

67 5. Shannon Code. Example 3:. calculate a series of cumulative probabilities Symbol Probability q k Length l i Shannon Code A /4 + 0 B /4 + /4 C /8 + /2 D /8 + 5/8 E /6 + 3/4 F /6 + 3/6 G + 7/8 H I /32 5/6 J 3

68 5. Shannon Code. Example 3: 2. calculate the code length for each symbol using log( \p i ) = l i < log ( \p i ) + Symbol Probability q k Length l i Shannon Code A /4 0 B /4 /4 C /8 /2 D /8 5/8 E /6 3/4 F /6 3/6 G 7/8 H 29/32 I 5/6 J 3

69 5. Shannon Code. Example 3: 2. calculate the code length for each symbol using log( \p i ) = l i < log ( \p i ) + Symbol A Probability /4 q k 0 Length l i Shannon Code Log(/(/4)) = l < Log(/(/4)) + B /4 /4 2 = l < 2 + C /8 /2 D /8 5/8 E /6 3/4 F /6 3/6 G 7/8 H 29/32 I 5/6 J 3

70 5. Shannon Code. Example 3: 2. calculate the code length for each symbol using log( \p i ) = l i < log ( \p i ) + Symbol A Probability /4 q k 0 Length l i 2 Log(/(/4)) = l < Log(/(/4)) + Shannon Code 2 = l < 2 + B C /4 /8 /4 /2 l = 2 D /8 5/8 E F /6 /6 3/4 3/6 G 7/8 H I J 29/32 5/6 3

71 5. Shannon Code. Example 3: 2. calculate the code length for each symbol using log( \p i ) = l i < log ( \p i ) + Symbol Probability q k Length l i Shannon Code A B C /4 /4 /8 0 /4 / D /8 5/8 3 E F /6 /6 3/4 3/6 4 4 G 7/8 5 H I J 29/32 5/

72 5. Shannon Code. Example 3: 3. write q k in the form c c c li 2 -li where each c i is either 0 or Symbol Probability q k Length l i Shannon Code A B C /4 /4 /8 0 /4 / D /8 5/8 3 E F /6 /6 3/4 3/6 4 4 G 7/8 5 H I J 29/32 5/

73 5. Shannon Code. Example 3: 3. write q k in the form c c c li 2 -li where each c i is either 0 or Symbol Probability q k Length l i q k = c c Shannon Code A B C /4 /4 /8 0 /4 / D /8 5/8 3 E F /6 /6 3/4 3/6 4 4 G 7/8 5 H I J 29/32 5/

74 5. Shannon Code. Example 3: 3. write q k in the form c c c li 2 -li where each c i is either 0 or Symbol Probability q k Length l i q k = c c Shannon Code A B C /4 /4 /8 0 /4 / = c c D /8 5/8 3 E F /6 /6 3/4 3/6 4 4 G 7/8 5 H I J 29/32 5/

75 5. Shannon Code. Example 3: 3. write q k in the form c c c li 2 -li where each c i is either 0 or Symbol A B C Probability /4 /4 /8 q k 0 /4 /2 Length l i q k = c c Shannon Code 0 = c c c = 0, c 2 = 0 D /8 5/8 3 E F /6 /6 3/4 3/6 4 4 G 7/8 5 H I J 29/32 5/

76 5. Shannon Code. Example 3: 3. write q k in the form c c c li 2 -li where each c i is either 0 or Symbol Probability q k Length l i Shannon Code A B C D E F G /4 /4 /8 /8 /6 /6 0 /4 /2 5/8 3/4 3/6 7/ q k = c c = c c c = 0, c 2 = 0 H I J 29/32 5/

77 5. Shannon Code. Example 3: 3. write q k in the form c c c li 2 -li where each c i is either 0 or Symbol Probability q k Length l i Shannon Code A B C /4 /4 /8 0 /4 / D /8 5/8 3 0 E F /6 /6 3/4 3/ G H I J 7/8 29/32 5/

78 5. Shannon Code. Example 3: Symbol Probability q k Length l i Shannon Code A B C /4 /4 /8 0 /4 / D /8 5/8 3 0 E F /6 /6 3/4 3/ G H I J 7/8 29/32 5/

79 5. Shannon Code. Note that: from examples and 3 one may conclude that Fano coding and Shannon coding produce the same code, however this is not true in general as we can see from the following example. Example Symbol Probability q k Length l i Shannon Code Fano code W X Y Z

80 Source Coding Techniques. Huffman Code. 2. Two-path Huffman Code. 3. Lemple-Ziv Code. 4. Shannon Code. 5. Fano Code. 6. Arithmetic Code.

81 6. Arithmetic Code. Coding In arithmetic coding a message is encoded as a number from the interval [0, ). The number is found by expanding it according to the probability of the currently processed letter of the message being encoded. This is done by using a set of interval ranges IR determined by the probabilities of the information source as follows: IR ={ [0, p ), [p, p + p 2 ), [p + p 2, p + p 2 + p 3 ), [p + + p n-, p + + p n ) } q j p i j Putting = we can write IR = { [0, q ), [q, q 2 ), [q n-, ) } i= In arithmetic coding these subintervals also determine the proportional division of any other interval [L, R) contained in [0, ) into subintervals IR [L,R] as follows:

82 6. Arithmetic Code. Coding In arithmetic coding these subintervals also determine the proportional division of any other interval [L, R) contained in [0, ) into subintervals IR [L,R] as follows: IR [L,R] = { [L, L+(R-L) q ), [L+(R-L) q, L+(R-L) q 2 ), [L+(R-L) q 2, L+(R-L) q 3 ),, [L+(R-L) P n-, L+(R-L) ) } Using these definitions the arithmetic encoding is determined by the Following algorithm: ArithmeticEncoding ( Message ). CurrentInterval = [0, ); While the end of message is not reached 2. Read letter x i from the message; 3. Divid CurrentInterval into subintervals IR CurrentInterval ; Output any number from the CurrentInterval (usually its left boundary); This output number uniquely encoding the input message.

83 6. Arithmetic Code. Coding Example Consider the information source A B C # Then the input message ABBC# has the unique encoding number As we will see the explanation In the next slides

84 6. Arithmetic Code. Coding 2. Read X i Example input message: A B B C #. CurrentInterval = [0, ); X i Current interval A [0, ) Subintervals

85 6. Arithmetic Code. Coding 2. Read X i Example input message: A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval A [0, ) Subintervals IR [0,) = { [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) j } q = j p i i= [L+(R-L) q i, L+(R-L) q i+ )

86 6. Arithmetic Code. Coding 2. Read X i Example input message: A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval A [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) IR [0,) = { [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) }

87 6. Arithmetic Code. Coding Example input message: No. A B B C # A B C # X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4)

88 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) B [0, 0.4) IR [0,0.4) = { [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) j } q = j p i i= [L+(R-L) q i, L+(R-L) q i+ )

89 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) B [0, 0.4) [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) IR [0,0.4) = { [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) }

90 6. Arithmetic Code. Coding Example input message: No. 2 A B B C # A B C # X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) B [0, 0.4) [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) [0.6, 0.28)

91 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) B [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.6, 0.28)

92 6. Arithmetic Code. Coding Example input message: No. 2 A B B C # A B C # X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) B [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.6, 0.28) [0.208, 0.244)

93 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0.6, 0.28) C B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.208, 0.244) [0.208, ), [0.2224, ), [0.2332, ), [0.2368, 0.244)

94 6. Arithmetic Code. Coding Example input message: A B B C # No. 3 A B C # X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0.6, 0.28) C B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.208, 0.244) [0.208, ), [0.2224, ), [0.2332, ), [0.2368, 0.244) [0.2332, )

95 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # 3. Divid CurrentInterval into subintervals IR CurrentInterval ; X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0.6, 0.28) C B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) # [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.208, 0.244) [0.208, ), [0.2224, ), [0.2332, ), [0.2368, 0.244) [0.2332, ) [0.2332, ), [ , ), [ , ), [ , )

96 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # No. 3 A B C # X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0.6, 0.28) C B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.208, 0.244) [0.208, ), [0.2224, ), [0.2332, ), [0.2368, 0.244) # [0.2332, ) [0.2332, ), [ , ), [ , ), [ , ) [ , )

97 6. Arithmetic Code. Coding Example input message: 2. Read X i A B B C # # is the end of input message Stop Return current interval [ , ) X i Current interval Subintervals A [0, ) [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) [0, 0.4) B [0.6, 0.28) C B [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) [0.208, 0.244) [0.208, ), [0.2224, ), [0.2332, ), [0.2368, 0.244) # [0.2332, ) [0.2332, ), [ , ), [ , ), [ , ) [ , )

98 6. Arithmetic Code. Coding Example input message: A B B C # # is the end of input message Stop Return current interval [ , ) Return the lower bound of the currentinterval as the codeword of the input message Input message ABBC# Codeword

99 6. Arithmetic Code. Decoding Arithmetic decoding can be determined by the following algorithm: ArithmeticDecoding ( Codeword ) 0. CurrentInterval = [0, ); While(). Divid CurrentInterval into subintervals IR CurrentInterval ; 2. Determine the subinterval i of CurrentInterval to which Codeword belongs; 3. Output letter x i corresponding to this subinterval; 4. If x i is the symbol # Return; 5. CurrentInterval = subinterval i in IR CurrentInterval ;

100 6. Arithmetic Code. Decoding Example Consider the information source Symbol A B C # Probability Then the input code word can be decoded to the message ABBC# As we will see the explanation In the next slides

101 6. Arithmetic Code. Decoding Example input codeword: CurrentInterval = [0, ); Current interval [0, ) Subintervals Output

102 6. Arithmetic Code. Decoding Example input codeword: Divid CurrentInterval into subintervals IR CurrentInterval ; Current interval [0, ) Subintervals Output IR[0,)= { [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) } q = j p i i= [L+(R-L) q i, L+(R-L) q i+ ) j

103 6. Arithmetic Code. Decoding Example input codeword: Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output IR [0,) = { [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) }

104 6. Arithmetic Code. Decoding Example input codeword: Determine the subinterval i of CurrentInterval to which Codeword belongs; 0 = < 0.4 Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output

105 6. Arithmetic Code. Decoding Example input codeword: Determine the subinterval i of CurrentInterval to which Codeword belongs; 0 = < 0.4 Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output

106 6. Arithmetic Code. Decoding Example input codeword: Output letter x i corresponding to this subinterval; No. A B C # No Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output A

107 6. Arithmetic Code. Decoding Example input codeword: If x i is the symbol # Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output A

108 6. Arithmetic Code. Decoding Example input codeword: If x i is the symbol # NO Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output A

109 6. Arithmetic Code. Decoding Example input codeword: CurrentInterval = subinterval i in IR CurrentInterval ; Current interval [0, ) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output A

110 6. Arithmetic Code. Decoding Example input codeword: CurrentInterval = subinterval i in IR CurrentInterval ; Current interval [0, ) [0, 0.4) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output A

111 6. Arithmetic Code. Decoding Example input codeword: Similarly we repeat the algorithm steps to 5 until the output symbol = # Current interval [0, ) [0, 0.4) [0.6, 0.28) Subintervals [0, 0.4), [0.4, 0.7), [0.7, 0.8), [0.8, ) Output [0, 0.6), [0.6, 0.28), [0.28, 0.32), [0.32, 0.4) B [ 0.6, 0.208), [0.208, 0.244), [0.244, 0.256), [0.256, 0.28) B [0.208, 0.244) [0.208, ), [0.2224, ), [0.2332, ), [0.2368, 0.244) [0.2332, ) [0.2332, ), [ , ), [ , ), [ , ) A C # 4. If x i is the symbol # Yes Stop Return the output message: A B B C #