Chapter7 Image Compression

Chapter7 Image Compression Preview 7I 7. Introduction 7.2 Fundamental concepts and theories 7.3 Basic coding methods 7.4 Predictive coding 7.5 Transform coding 7.6 Introduction to international standards 23 Digital Image Processing

Preview General communication model f( x, y) n n 2 n 3 Soure encoder Channel encoder Channel Channel Soure decoder decoder ^ (, ) f xy n >n 2 n 3 >n 2 n > n 3 >n 2 23 Digital Image Processing 2

Preview Lossless compression 8.7Kbyte 3.46Kbyte 2.62Kbyte winzip G4 23 Digital Image Processing 3

Preview Lossy compression 92kbyte.2Kbyte PSNR=36.97dB 23 Digital Image Processing 2.5kbyte PSNR=25.dB 4

7.. why code data? To reduce storage volume To reduce transmission time One colour image 76 by 58 pixels 3 channels, each 8 bits.3 Mbyte Video data same resolution 25 frames per second 33 Mbyte/second 7. Introduction 7..2 Classification Lossless compression: reversible Lossy compression: irreversible 23 Digital Image Processing 5

7..3 Compression methods 7. Introduction Entropy coding: Huffman coding, arithmetic coding Run length coding: G3, G4 LZW coding: arj(dos), zip(windows), i compress( UNIX) Predictive coding: linear prediction, non-linear prediction Transform coding: DCT, KLT, Walsh-T, TWT( WT(wavelet) Others: vector quantization (VQ), Fractal coding the second generation coding 23 Digital Image Processing 6

7. Introduction 7..3 Compression methods: Fractal coding Julia Set Koch curve Mandelbrot Set 23 Digital Image Processing 7

7. Introduction 7..3 Compression methods: the second generation coding Region of interest (ROI) original ii ROI.3 bpp Saliency map 23 Digital Image Processing 8

7.2 Fundamental concepts and theories 7.2. Data redundancy If n denote the original data number, and n 2 denote the compressed data number, then the data redundancy can be defined as RD = C R where C R, commonly called the compression ratio, is C R = n n 2 For example n 2 =n R D = n n C, R 2 R D 23 Digital Image Processing 9

7.2 Fundamental concepts and theories 7.2. Data redundancy Other expression of compression ratio C R n n 2 = or n2 8 bits / pixel L = bits / pixel, CR = M N L Three basic image redundancy Coding redundancy Interpixle ep eredundancy d psychovisual redundancy 23 Digital Image Processing

7.2 Fundamental concepts and theories 7.2. Data redundancy: coding redundancy If r k represents the gray levels of an image and each r k occurs with probability p r (r k ), then nk pr( rk) =, k =,, L n where L is the number of gray levels, n k is number of times that the kth gray level appears in the image, and n is the total number of pixels in the image 23 Digital Image Processing

7.2 Fundamental concepts and theories 7.2. Data redundancy: coding redundancy If the number of bits used to represent each value of r k is l(r k ),then the average number of bits required to represent each pixel is For example L L = lr ( ) p( r) avg k r k k = r k P r (r k ) Code l (r k ) Code2 l 2 (r k ) r.9 3 2 r.25 3 2 r 2.2 3 2 r 3.6 3 3 r 4.8 3 4 r 5.6 3 5 R 6.3 3 6 r 7.2 3 7 23 Digital Image Processing 2

7.2 Fundamental concepts and theories 7.2. Data redundancy: coding redundancy 7 L = l ( r ) p ( r ) avg 2 k r k k = = 2(.9) + 2(.25) + 2(.2) + 3(.6) + 4(.8) + 5(.6) + 6(.3) + 6(.2) = 2.7bits C = 3/27 3/2.7=. R R D = = 99.99. 23 Digital Image Processing 3

7.2 Fundamental concepts and theories 7.2. Data redundancy: interpixel redundancy 23 Digital Image Processing 4

7.2 Fundamental concepts and theories 7.2. Data redundancy: interpixel redundancy Autocorrelation coefficients where γ ( Δ n) = A( Δn) A() N Δ n A( Δ n) = f( x, y) f( x, y+δn) N Δn y= 23 Digital Image Processing 5

7.2 Fundamental concepts and theories 7.2. Data redundancy: interpixel redundancy 23 Digital Image Processing 6

7.2 Fundamental concepts and theories 7.2. Data redundancy: psychovisual redundancy.psychovisual redundancy is associated with real or quantifiable visual information. The information itself is not essential for normal visual processing 8bits/pixel 4bits/pixel 2bits/pixel 23 Digital Image Processing 7

7.2 Fundamental concepts and theories 7.2.2 Fidelity criteria: objective Lt ^ Let f ( x, y ) represent an input timage and f ( xy, ) denote the Decompressed image. For any value x and y, the error e(x,y) bewteen f ( x, y) and f ^ ( xy, ) can be defined as ^ exy (, ) = f( xy, ) f( xy, ) So that the total error between the two images is M N ^ x= y= [ f ( x, y ) f ( x, y)] 23 Digital Image Processing 8

7.2 Fundamental concepts and theories 7.2.2 Fidelity criteria: objective The root-mean-square error, e ems, between and f xy is M N ^ erms = { [ f( x, y) f( x, y)] } MN x= y= f( x, y) 2 /2 The mean-square signal noise ratio is defined as SNR = lg M N ^ x= y= rms M N ^ x= y= f ( x, y) 2 [ f ( x, y) f( x, y)] 2 ^ (, ) 23 Digital Image Processing 9

7.2 Fundamental concepts and theories 7.2.2 Fidelity criteria: objective PSNR The signal noise ratio is defined as SNR = lg M N ^ f ( x, y ) f ( x, y ) x= y= M N ^ x= y= [ f ( x, y ) f ( x, y )] The peak signal noise ratio is defined as M N 2 fmax 2 x= y= fmax lg M N ^ 2 2 erms = lg = [ f ( x, y) f( x, y )] x= y= 23 Digital Image Processing 2 2 2

7.2 Fundamental concepts and theories 7.2.2 Fidelity criteria: subjective Side-by-side comparisons 23 Digital Image Processing 2

两幅图象有相同的 PSNR==3.7db,, 同样数量的矩形噪声在左图中加到了平坦的天空区域, 而在右图却加在了纹理比较丰富的地面岩石区域, 噪声在左图中很易被察觉到, 而在右图中确很难被发现 ( 这就是所谓的掩蔽效应 ), 两幅图象主观上质量有很大的差别, 而 PSNR 却不能反映这一区别 23 Digital Image Processing 22 22

23 Digital Image Processing 23

7.2 Fundamental concepts and theories 7.2.3 image compression models Source encoder model f ( x, y) decorrelation quantizer Symbol encoder ^ (, ) f xy Remove Interpixel redundancy Remove psychovisual redundancy Remove coding redundancy 23 Digital Image Processing 24

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: terms 948, Cloude Shannon, A mathematical theory of communication News:data information: contents Information source: symbols Memoryless source Markov source 23 Digital Image Processing 25

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Self-information A random event E that occurs with probability is said to contain I ( E ) units of information. I( E) = log log P( E) PE ( ) = P( E) For example PE ( ) =, Unit: I( E ) = PE ( ) =, I( E ) = Base=2, bits ( binary digits) Base=e, nats ( nature digits) Base=, Hartly 23 Digital Image Processing 26

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Entropy of the source definition Suppose X={x, x, x N- }is a discrete random variable, and the probability of x j is P(x j ),namely X x x x N = PX ( ) p ( x x x ) p( ) p( ) N N then H ( X) = p( xj )log = p( xj)log p( xj) j= px ( j ) j= 23 Digital Image Processing 27

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Entropy of the source properties () H ( X ) (2) H( X) = H(,, ) = H(,, ) = = H(,, ) = (3) H( X) = H( x, x x ) logn N When P(x j )=/N for all j,, H(X)=logN example X={2345678} X={,2,3,4,5,6,7,8 }, p j =/8 for each j 8 H ( X ) = log2 8= 3 bits / symbol 8 j= 8 23 Digital Image Processing 28

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Conditional entropy definitions H( X) = p( xi )log i p ( xi ) H( X) = p( xi) p( xi xi )log p ( x x ) i i i i H ( X ) = px ( ) ( )log 2 i pxi xi x i 2 p ( x x x ) i i i i i 2 Hm( X) = px ( i) px ( i xi xi m)log px ( x x ) i i i i i m property H = = H < H < < H < H < H m m 2 23 Digital Image Processing 29

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: using information theory Asimple8 8-bits image: Zero-order entropy: 2 2 2 95 69 243 243 243 2 2 2 95 69 243 243 243 2 2 2 95 69 243 243 243 2 2 2 95 69 243 243 243 Probability of the source symbols: Gray-levelGa Count probability 2 2 3/8 95 4 /8 69 4 /8 243 2 3/8 H( X) = p( xi )log p ( x ) i = 8.8 bits / pixel i 23 Digital Image Processing 3

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: using information theory Relative frequency of pairs of pixels: Gray-level Count probability (2,2) Paira 8 2/8 (2,95) 4 /8 (95,69) 4 /8 (69,243) 4 /8 (243,243) 8 2/8 (243,2) 4 /8 H( X ) = p( xi )log = 2.52 bits / string px ( ) i i =.25 bits / pixel 23 Digital Image Processing 3

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Noiseless coding theorem Also called Shannon first theorem. Defines the minimum average code word length per source symbol that can be achieved H ( X) Lavg For memoryless (zero-memory) source: H( X) = p( xi )log p ( x ) i But, the image data usually are Markov source, how to code? i 23 Digital Image Processing 32

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Noiseless coding theorem Coding steps marcov X decorrelation Y memoryless entropy coding entropy coding L X L Y H(X)=H m (X)<H (X), L X = H (X) H(Y)=H (Y), L Y = H (Y) H(X)=H(Y) ( ) L Y= H m m( (X)<L X 23 Digital Image Processing 33

7.2 Fundamental concepts and theories 7.2.4 Elements of information theory: Noise coding theorem For a given rate distortion D, the rate distortion function R(D) is less than H(X), and R(D) is the minimum of coding length L avg 23 Digital Image Processing 34

7.3. Huffman coding Huffman, 952 73Basic 7.3 coding methods code the low probabilities symbols with long word code the high probabilities symbols with short word smallest possible number of code symbols per source symbol the source symbols must be coded one at a time 23 Digital Image Processing 35

7.3. Huffman coding 73Basic 7.3 coding methods step:create a series of source reduction by ordering the probabilities step2:combine the lowest probability symbols into a single symbol that replaces them in the next source reduction step3:repeat step2 until the lowest probability is For example Symbol Probability 2 3 4 A B C D E F.4.3...6.4.4.3....4.3.2..4.3.3.6.4 23 Digital Image Processing 36

7.3. Huffman coding 73Basic 7.3 coding methods step4:code each reduced d source with and, staring with the smallest source and working back to the original source Symbol Probability 2 3 4 A B C D E F.4.3.4.3.4 3.3...2....66..4 L =.4 +.3 2 +. 3+. 4 avg +.6 5 +.4 5 = 2.2 bits / symbol L ( ) avg > H X 23 Digital Image Processing.4 3.3.3 6 i==.6 4.4 H( A) = p( ai)log p( ai) = 2.4 bits / symbol 37

7.3 Basic coding methods 7.3.2 Arithmetic coding An entire sequence of source symbols( or message) is assigned as a single arithmetic code word the code word itself defines an interval of real numbers between and The encoding process of an arithmetic coding can be explained through the following example 23 Digital Image Processing 38

7.3 Basic coding methods 7.3.2 Arithmetic coding: example encoding Let us assume that the source symbols are {a a 2 a 3 a 4 } and the probabilities of these symbols are {.2, {2.2, 2.4, 4.2} 2} To encode a message of a sequence: a a 2 a 3 a 3 a 4 The interval l[) [,) can be divided id das four sub-intervals: [.,.2), [.2,.4), [.4,.8), [.8,.), symbols probabilites Initial intervals a.2 [.,.2) a 2.2 [.2,.4) a 3.4 [.4,.8) a 4 2.2 [8) [.8,.) 23 Digital Image Processing 39

7.3 Basic coding methods 7.3.2 Arithmetic coding: example We take the first symbol a from the message and find its encoding range is [.,.2). The second symbol a2is encodedby taking the 2 th -4 th of interval [.,.2) as the new interval [.2,.4). And so on. Visually, we can use the figure: Finally, choose a number from the interval of [.688,.6752) 688 6752) as the output:.68 23 Digital Image Processing 4

7.3 Basic coding methods 7.3.2 Arithmetic coding : example decoding <.688688 <.2 a.4<.688 <.8 a 2.56< 56<.688688 <.7272 a 3.624<.688 <.688 a 3.6752<.688 <.688 a 4 23 Digital Image Processing 4

73Basic 7.3 coding methods 7.3.4 Binary image coding: constant area coding all white all black mixed intensity The most probable category is coded with -bit code word The other two category is coded with 2-bit code word and The code assigned to the mixed intensity category is used as a prefix 23 Digital Image Processing 42

73Basic 7.3 coding methods 7.3.4 Binary image coding: run length coding Represent each row of a image by a sequence of lengths that describe runs of black and white pixels the standard compression approach in facsimile (FAX) coding. CCITT, G3,G4. (,8) (,3) (,4) (,4) (, 28) (,3) (,23) (,3) (,29) (, 3) (,28) (,3) (,3) (,4) (,25) (,4) (,38) (,4) 8 3 4 4 28 3 23 3 29 3 28 9 3 3 4 25 4 38 4 23 Digital Image Processing 43

73Basic 7.3 coding methods 7.3.4 Binary image coding: contour tracing and coding Δ ' is the difference between the starting coordinates of the front contours adjacent lines Δ '' is the difference between the front-to-back contour lengths contours adjacent lines code: new start, Δ ' and Δ '' 23 Digital Image Processing 44

But for gray image 62 6 59 6 62 6 58 56 56 6 62 6 59 6 62 6 58 56 56 6 63 58 59 59 6 58 55 55 56 58 59 57 59 56 59 59 54 52 55 53 55 57 56 56 58 57 56 55 54 56 56 57 55 5 57 56 55 56 56 54 57 56 56 56 56 56 54 56 55 55 58 57 55 55 56 55 55 55 55 55 56 55 56 53 56 55 56 55 54 56 55 55 57 54 57 55 57 58 58 58 23 Digital Image Processing 45

7.4. lossless coding encoder 74Predictive 7.4 coding e = f fˆ n n n Huffman coding or Arithmetic coding decoderd f = e + fˆ n n n 23 Digital Image Processing 46

74Predictive 7.4 coding 7.4. lossless coding In most case, the prediction is formed by a linear combination of m previous pixels. That is, m ˆ fˆn = round α i f n i i= where m is the order of fthe linear predictor α i are prediction coefficients For example, -D linear predictive can be written m fˆ( xy, ) = round αi f ( xy, i ) i= 23 Digital Image Processing 47

74Predictive 7.4 coding 7.4. lossless coding: Experimental result m=,α= fˆ ( xy, ) = round[ f( xy, ) ] Original image 23 Digital Image Processing Residual image (28 represents ) 48

74Predictive 7.4 coding 7.4. lossless coding: Experimental result m=, α= 2 e e p () = σ e e e 2σ e Histogram of original image Histogram of residual image 23 Digital Image Processing 49

74Predictive 7.4 coding 7.4. lossless coding: JPEG lossless compression standard Prediction schemes c b a x Location of pixels Prediction Prediction Scheme value Null a 2 b 3 c 4 a+b-c 5 a+(b-c)/2 6 b+(a-c)/2 7 (a+b)/2 23 Digital Image Processing 5

7.4.2 lossy coding: model 74Predictive 7.4 coding f = e + fˆ encoder n n n decoder 23 Digital Image Processing 5

74Predictive 7.4 coding 7.4.2 lossy coding: Optimal predictors differential pulse code modulation( (DPCM) Minimizes the encoder s mean-square prediction error E e E f f ˆ 2 2 { n} = {[ n n] } Subject to the constraint that fˆ m = α f n i n i i= E e E f f m 2 { = n } f n α i f n i= 2 23 Digital Image Processing 52

74Predictive 7.4 coding 7.4.2 lossy coding: Optimal predictors Experiment: four predictors: () f ˆ( x, y ) =.97 f ( x, y ) (2) f ˆ( x, y) =.5 f( x, y ) +.5 f( x, y) f ˆ( x, y) =.75 f( x, y ) +.75 f( x, y).5 f( x, y ) (3).97 (, ) if (4) fˆ ( xy, ).97 f( x, y ) if Δ h Δ v =.97 f( x, y) otherwise where Δ h= f( x, y) f( x, y ) f(x-,y-) f(x-,y) f(x-,y+) Δ v= f( x, y ) f( x, y ) f(x,y-) f(x,y) 23 Digital Image Processing 53

74Predictive 7.4 coding 7.4.2 lossy coding: Optimal predictors Experiment original 23 Digital Image Processing 54

74Predictive 7.4 coding 7.4.2 lossy coding: Optimal predictors Conclusion:Error decreases as the order of the predictor increases 23 Digital Image Processing 55

74Predictive 7.4 coding 7.4.2 lossy coding: Optimal quantization The staircase quantization function t=q(s) is shown as 23 Digital Image Processing 56

75Transform 7.5 Coding 7.5. Review of image transform: definitions X T Y Direct transform Inverse transform Y = TX X = Vector expression T Y M N ymn, = xi, jφ(, i j, m, n) i= j= polynomial expression Question: why transformation can compress data? 23 Digital Image Processing 57

75Transform 7.5 Coding 7.5. review of image transform: properties Entropy keeping: Energy keeping: decorrelation: Energy re-assigned: H(X)=H(Y) M N M N 2 2 X ( mn, ) = Ykl (, ) m= n= k= l= H = = H < H < < H < H < H m m 2 H(Y)=H (Y) H(X)=H m (X) H (X)>H (Y) original DFT DCT 23 Digital Image Processing 58

7.5.2 Transform coding system 75Transform 7.5 Coding f( x, y) Construct n*n subimage Forward transform encoder quantizer Symbol encoder Bits stream Bits Symbol Inverse stream encoder transform Merge n*n subimage ^ (, ) f xy decoder 23 Digital Image Processing 59

7.5.3 Transform selection Information packing ability: Computational complexity: 7.5.4 sub-image size selection Computational complexity increase as the subimage size increase Correlation decrease as the subimage size increase The most popular subimage size are 8*8, and 6*6 75Transform 7.5 Coding KLT>DCT>DFT>WHT WHT<DCT<DFT<KLT 23 Digital Image Processing 6

75Transform Coding 7.5 Transform Coding 7.5.5 bit allocation: zonal coding transform coefficients of maximum variance carry the most information and should be retained 8 7 7 6 6 5 4 3 4 3 2 2 6 4 3 5 4 4 3 3 3 3 3 2 3 2 3 2 3 2 3 2 Zonal mask Bit allocation 23 Digital Image Processing 6

75Transform Coding 7.5 Transform Coding 7.5.5 bit allocation: threshold coding Zig-zag g g threshold mask 23 Digital Image Processing 62

75Transform 7.5 Coding 7.5.5 bit allocation: threshold coding There are three basic ways to threshold a transformed coefficients A single global threshold can be applied to all subimages A different threshold can be used for each subimage The threshold can be varied as a function of the location of each coefficient within the subimage where Tuv (, ) Tˆ( u, v) = round Zuv (, ) T(u,v): transform coefficients Z(u,v): quantization matrix 23 Digital Image Processing 63

75Transform 7.5 Coding 7.5.5 bit allocation: threshold coding Z(u,v) is assigned a particular value c Z(u,v) used in JPEG stardard 23 Digital Image Processing 64

75Transform 7.5 Coding 7.5.5 bit allocation: threshold coding Experiment results Left column: quantize with Z(u,v) (,) Right column: quantize with 4*Z(u,v) 23 Digital Image Processing 65

75Transform 7.5 Coding 7.5.3 JEPG lossy compression standard DCT Zig-zag quantise run length code entropy code 23 Digital Image Processing.. 66

Original image 39 44 49 53 44 5 53 56 5 55 6 63 59 6 62 6 DCT 26 - -2-5 -23-7 -6-3 - -9-2 2-7 -2 quantise 79-2 - - - - 79 Zig-zag 79 - -2 - - - -2 run - length - code - Huffman 2 - code. 23 Digital Image Processing 67

JPEG Decoding DCT Zig-zag quantise run length code entropy code 23 Digital Image Processing.. 68

Result of Coding and Decoding 39 44 49 53 44 46 49 52 44 5 53 56 48 5 52 54 5 55 6 63 55 56 57 58 59 6 62 6 6 6 6 62 Original block Reconstructed block -5-2 -4 2-5 - 3 5 - -2 errors 23 Digital Image Processing 69

75Transform 7.5 Coding 7.5.3 JEPG lossy compression standard f( x, y) Construct 8*8 subimage DCT encoder quantizer Zig-zag Question: why there are mosaics in JEPG images? Huffman coding Bits stream 23 Digital Image Processing 7

75Transform 7.5 Coding 7.5.3 JEPG 2 lossy compression standard 23 Digital Image Processing 7

23 Digital Image Processing 72

23 Digital Image Processing 73

7.6 Introduction to international standards Image compression standards: -JPEG, -JPEG2 Video compression standards: -MPEG-,MPEG-2,MPEG-4, -H.26,H.263,H.263+,H.264 H 263 H 263+ H 264 -H.264/AVC 23 Digital Image Processing 74

Video compression standards 23 Digital Image Processing 75

Homework 6.7(-4) () 请说明是否能用变长变码法压缩 幅已直方图均衡化的具有 2 n 级灰度的图? (2) 这样的图像中包含像素间冗余吗? 6.(-22) 已知符号 a,e,i,o,u,? 的出现概率分别是.2,.3,.,.2,.,.,.23355 进行解码, 解码长度为 6 6.(-22) 已知符号 a,e,i,o,u,?,,,,, 的出现概率分别是.2,.3,.,.2,.,,,,,,.,.23355 进行解码, 解码长度为 6 23 Digital Image Processing 76

THE END THE END 科大西区 West campus of USTC 23 Digital Image Processing 77