IMAGE COMPRESSION IMAGE COMPRESSION-II. Coding Redundancy (contd.) Data Redundancy. Predictive coding. General Model

Transcription

1 IMAGE COMRESSIO IMAGE COMRESSIO-II Data redundancy Self-information and Entropy Error-free and lossy compression Huffman coding redictive coding Transform coding Week IX 3/6/23 Image Compression-II 3/6/23 Image Compression-II 2 Data Redundancy Coding Redundancy (contd) CODIG: Fewer bits to represent frequent symbols ITERIXEL / ITERFRAME: eighboring pixels have similar values Consider equation (A): It makes sense to assign fewer bits to those r k for which p r (r k ) are large in order to reduce the sum this achieves data compression and results in a variable length code More probable gray levels will have fewer # of bits SYCHOVISUAL: Human visual system can not simultaneously distinguish all colors L Lavg = l( rk ) pr( rk) ( A) k = 3/6/23 Image Compression-II 3 3/6/23 Image Compression-II General Model General compression model Source Source Mapper Channel Channel Quantizer Channel Symbol Source g(x,y) To channel redictive coding To reduce / eliminate interpixel redundancies Lossless predictive coding: ECODER Image Σ Q redictor - Symbol Image rediction error e = f fˆ n n n 3/6/23 Image Compression-II 5 3/6/23 Image Compression-II 6

2 Decoder Example symbol Σ original redictor rediction error: = -f n is coded using a variable length code (symbol ) = m ˆ Example : f n=int αi fn i i= Linear predictor ; m = order of predictor m Example 2: f ˆ n(x,y)=int αi f( x, y i) i= In 2- D X ast information W xy Current pixel f (x,y) = Σ α (x, y ) f (x, y) x y 3/6/23 Image Compression-II 7 (x, y ) ε W xy 3/6/23 Image Compression-II 8 An example of first order predictor f ˆ( x, y) = Int[ f( x, y ) rediction error Lossy Compression (Section 85) Lossy compression: uses a quantizer to compress further thumber of bits required to encode the error First consider this: e Σ Q Enc f n e e! n f" ~ n fn Dec ~ Σ Histograms of original and error s 3/6/23 Image Compression-II 9 otice that, unlike in the case of loss-less prediction, in lossy prediction the predictors see different inputs at the and 3/6/23 Image Compression-II Quantization error This results in a gradual buildup of error which is due to the quantization error at the site In order to minimize this buildup of error due to quantization we should ensure that s have the same input in both the cases = - = = = /6/23 Image Compression-II redictive Coding With Feedback - Q Symbol = symbol 3/6/23 Image Compression-II 2 = This feedback loop prevents error building uncompressed

3 Example Example Example: f"! n = α fn en e! n = ξ > and ξ en < f! e! f" n = n n = e! f! n α n < α < prediction coefficient with feedback = 2 3 = _ 2 2 = _ 2 2 = 2 2 f n = _ Q ote: The quantizer used here is-- floor ( /2)*2 This is different from the one used in the earlier example ote that this would result in a worse response if used without Feedback (output will be flat at ) 3/6/23 Image Compression-II 3 3/6/23 Image Compression-II Another example {, 5,, 5, 3, 5, 5,, 2, 26, 27, 28, 27, 27, 29, 37, 37, 62, 75, 77, 78, 79, 8, 8, 8, 82, 82} A comparison (Fig 823) 3/6/23 Image Compression-II 5 3/6/23 Image Compression-II 6 Four linear predictors Transform Coding Feb 28, 22 3/6/23 Image Compression-II 7 3/6/23 Image Compression-II 8

4 Transform coding Transform Selection Construct n X n subs X s Forward transform Quantizer Symbol DFT Discrete Cosine Transform (DCT) Wavelet transform Karhunen-Loeve Transform (KLT) Symbol Inverse transform Merge n X n subs Uncompressed Blocking artifact: boundaries between subs become visible 3/6/23 Image Compression-II 9 3/6/23 Image Compression-II 2 Fig 83 Discrete Cosine Transform DFT, Hadamard and DCT Ahmed, atarajan, and Rao, IEEE T-Computers, pp 9-93, 97 3/6/23 Image Compression-II 2 3/6/23 Image Compression-II 22 -D Case: Extended 2 oint Sequence DCT & DFT Consider - D first; Let x(n) be a point sequence n - 2- point x(n) y(n) DFT 2 point point Y(u) C(u) xn ( ), n yn ( ) = xn ( ) x( 2 n) = x( 2 n), n 2 x(n) y(n) /6/23 Image Compression-II π Y( u) = y( n)exp j un n= 2 2 2π 2π = xn ( )exp j un x(2 n)exp j un n= 2 n= 2 2π 2π = xn ( )exp j un xm ( )exp j u(2 m) n= 2 m= 2 π π 2π = exp j u x( n) exp j u j un 2 n= 2 2 π π 2π exp j u x( n) exp j u j un 2 n= 2 2 π π = exp j u 2 x( n) cos u(2n ) 2 n= 2 3/6/23 Image Compression-II 2

5 DCT The - point DCTof x(n), C(u), is given by Rexp F π - j I u - Cu ( ) uyu ( ), = S H 2 K otherwise T The unitary DCT transformations are: - π Fu ( ) = α( u) Â f( n)cos ( 2 where n ) u, u -, H 2 K n= 2 α( ) =, α( u) = for k - The inverse transformation is - π f( n) = Âα( u) F( u)cos ( 2 n ) u, u - H 2 K n= F F I I Discrete Cosine Transform in 2-D 2x ) uπ Cuv (,) = α() uα() v f(, xy)cos cos for uv, =,,2,,, where α( u) = R S T 2 x= y= for u = for u =,2,, - and 2x ) uπ f( x, y) = α( u) α( v) C( u, v)cos cos u= v= for x, y =,,2,, ( ( 2y ) vπ 2 2 ( ( 2y ) vπ 2 2 3/6/23 Image Compression-II 25 3/6/23 Image Compression-II 26 DCT Basis functions Implicit eriodicity-dft vs DCT (Fig 832) 3/6/23 Image Compression-II 27 3/6/23 Image Compression-II 28 Why DCT? Blocking artifacts less pronounced in DCT than in DFT Good approximation to the Karhunen-Loeve Transform (KLT) but with basis vectors fixed DCT is used in JEG compression standard Karhunen-Loeve Transform READ pp 76 and Section Text (if you are working on the face recognition project, you must!) Also called the Hotelling Transform Transform is data dependent Let X denote the random data, and let C be the covariance matrix of X, ie C = E{(X-m)(X-m) T } where m is the mean vector of the data The matrix C is real and symmetric, and hence can be diagonalized using its eigenvectors 3/6/23 Image Compression-II 29 3/6/23 Image Compression-II 3

6 What are eigenvectors? T Let C =Ε[( x mx)( x mx) ] where C is the covariance matrix, x is an dimensional vector, and mx is the mean vector of the samples The eigenvectors ei of C are given by Cei = λiei where λi are the corresponding eigenvalues ow consider a matrix A whose columns correspond to the eigenvectors of C, arranged such that the first column corresponds to the eigenvector with the largest eigenvalue, and second with the second largest and so on Then, consider a transformation on the vectors x such that T y = A ( x mx) ote that y is zero mean, and its covariance matrix is T T T T Cy =Ε ( yy ) =Ε[ A ( x mx)( x mx) A] = A CA=Λ = diagnoal matrix of eigenvalues, in decreasing order ote that the elements of the transformed vector are uncorrelated (non-diagnoal elements are zero) 3/6/23 Image Compression-II 3 What is KLT The transformation from X to Y is called the KLT In computing the transformation matrix A, we assume that the columns are made up of orthonormal eigenvectors (ie, the inverse of A is also its transpose) Thus the basis vectors of the KLT are the orthonormal eigenvectors of the covriance matrix C The KLT yields uncorrelated coefficients For compression, we only keep the top K coefficients corresponding to the K largest eigenvectors Then the mean squared error in reconstruction is given by j= K 3/6/23 Image Compression-II 32 MSE = λ j Sub- size selection Different sub- sizes 3/6/23 Image Compression-II 33 3/6/23 Image Compression-II 3 Bit Allocation/Threshold Coding # of coefficients to keep Threshold coding How to quantize them Zonal coding Threshold coding For each sub i Arrange the transform coefficients in decreasing order of magnitude Keep only the top X% of the coefficients and discard rest Code the retained coefficient using variable length code 3/6/23 Image Compression-II 35 Zonal Coding Zonal coding:!!!!!!!!!! i th coefficient in each sub () Compute the variance of each of the transform coeff; use the subs to compute this (2) Keep X% of their coeff which have maximum variance (3) Variable length coding (proportional to variance) Bit allocation: In general, let thumber of bits allocated be made proportional to the variance of the coefficients Suppose the total number of bits per block is B Let thumber of retained coefficients be M Let v(i) be variance of the i-th coefficient Then M B bi () = M 2log 2vi () 2M log 2vi () 3/6/23 Image Compression-II 36 i=

7 Zonal Mask & bit allocation: an example Typical Masks (Fig 836) /6/23 Image Compression-II 37 3/6/23 Image Compression-II 38 Image Approximations The JEG standard The following is mostly from Tekalp s book Digital Video rocessing by M Tekalp (rentice Hall) MEG Video compression standard, edited by Mitchell, ennebaker, Fogg and LeGall (Chapman and Hall) For thew JEG-2 check out the web site wwwjpegorg 3/6/23 Image Compression-II 39 3/6/23 Image Compression-II JEG (contd) JEG-baseline JEG is a lossy compression standard using DCT Activities started in 986 and the ISO in 992 Four modes of operation: Sequential (basline), hierarchical, progressive, and lossless Arbitrary sizes; DCT mode 8-2 bits/sample Luminance and chrominance channels are separately encoded We will only discuss the baseline method DCT: The is divided into 8x8 blocks Each pixel is level shifted by 2 n- where 2 n is the maximum number of gray levels in the Thus for 8 bit s, you subtract 28 Then the 2-D DCT of each block is computed For the baseline system, the input and output data precision is restricted to 8 bits and the DCT values are restricted to bits Quantization: the DCT coefficients are threshold coded using a quantization matrix, and then reordered using zig-zag scanning to form a -D sequence Thon-zero AC coefficients are Huffman coded The DC coefficients of each block are DCM coded relative to the DC coefficient of the previous block 3/6/23 Image Compression-II 3/6/23 Image Compression-II 2

8 JEG -color JEG quantization matrices RGB to Y-Cr-Cb space Y = 3R 6G B Cr = 5 (B-Y) 5 Cb = (/6) (R Y) 5 Chrominance samples are sub-sampled by 2 in both directions Y Y5 Y9 Y3 Y2 Y6 Y Y Y3 Y7 Y Y5 Y Y8 Y2 Y6 Cr Cr3 Cr2 Cr Cb Cb3 Cb2 Cb on-interleaved Scan :Y,Y2,,Y6 Scan 2: Cr, Cr2, Cr3, Cr Scan 3: Cb, Cb2, Cb3, Cb Check out the matlab workspace (dctexmat) Quantization table for the luminance channel Quantization table for the chrominance channel JEG baseline method Consider the 8x8 (matlab: array s) Level shifted (s-28=sd) 2d-DCT: dct2(sd)= dcts After dividing by quantiization matrix qmat: dctshat Zigzag scan as in threshold coding [2, 5, -3, -, -2,-3,,, -, -,,,, 2, 3, -2,,,,,,,,,,,,, EOB] Interleaved: Y, Y2, Y3, Y, Cr, Cb,Y5,Y6,Y7,Y8,Cr2,Cb2, 3/6/23 Image Compression-II 3 3/6/23 Image Compression-II An 8x8 sub- (s) 2D DCT (dcts) and the quantization matrix (qmat) s = (8x8block) sd =(level shifted) dcts= qmat= /6/23 Image Compression-II 5 3/6/23 Image Compression-II 6 Division by qmat (dctshat)=dcts/qmat Zig-zag scan of dcthat dcthat= dcts= dcthat= Zigzag scan as in threshold coding [2, 5, -3, -, -2,-3,,, -, -,,,, 2, 3, -2,,,,,,,,,,,,, EOB] /6/23 Image Compression-II 7 3/6/23 Image Compression-II 8

9 Threshold coding -revisited Zig-zag scanning of the coefficients JEG baseline method example Zigzag scan as in threshold coding [2, 5, -3, -, -2,-3,,, -, -,,,, 2, 3, -2,,,,,,,,,,,,, EOB] The coefficients along the zig-zag scan lines are mapped into [run,level] where the level is the value oon-zero coefficient, and run is thumber of zero coeff preceding it The DC coefficients are usually coded separately from the rest The DC coefficient is DCM coded (difference between the DC coefficient of the previous block and the current block) The AC coef are mapped to run-level pairs (,5), (,-3), (, -), (,-2),(,-3),(,),(,),(,-),(,-), (2,), (,2), (,3), (,- 2),(,),(,),(6,),(,),(,),EOB These are then Huffman coded (codes are specified in the JEG scheme) The follows an inverse sequence of operations The received coefficients are first multiplied by the same quantization matrix (recddctshat=dctshat*qmat) Compute the inverse 2-D dct (recdsd=idct2(recddctshat); add 28 back (recds=recdsd28) 3/6/23 Image Compression-II 9 3/6/23 Image Compression-II 5 Decoder Received signal Recddcthat=dcthat*qmat Recdsd= Reconstructed S= s = (8x8block) /6/23 Image Compression-II 5 3/6/23 Image Compression-II 52 Example Image Compression: Summary Data redundancy Self-information and Entropy Error-free compression Huffman coding, Arithmetic coding, LZW coding, Run-length encoding redictive coding Lossy coding techniques redictive coing (Lossy) Transform coding DCT, DFT, KLT, JEG compression standard 3/6/23 Image Compression-II 53 3/6/23 Image Compression-II 5