Review of Quantization. Quantization. Bring in Probability Distribution. L-level Quantization. Uniform partition

Transcription

1 Review of Quantization UMCP ENEE631 Slides (created by M.Wu 004) Quantization UMCP ENEE631 Slides (created by M.Wu 001/004) L-level Quantization Minimize errors for this lossy process What L values to use? Map what range of continous values to each of L values? Uniform partition Maximum errors = ( t max -t min ) / L = A / L (t max t max )/L dynamic range A t k t k1 quantization Best solution? error Consider minimizing maximum absolute error (min-max) vs. MSE what if the value between [a, b] is more likely than other intervals? t min t min t max t max ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [1] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [] Bring in Probability Distribution Minimize error in a statistical sense MMSE (minimum mean square error) t L1 r 1 p.d.f p u (x) Allocate more reconstruct. values in more probable ranges assign high penalty to large error and to likely occurring values squared error gives convenience in math.: differential, etc. An optimization problem What {t k } and {r k } to use? Necessary conditions: by setting partial differentials to zero r L UMCP ENEE631 Slides (created by M.Wu 001/004) MMSE Quantizer (Lloyd-Max) Reconstruction and decision levels need to satisfy A nonlinear problem to solve Solve iteratively Choose initial values of {t k } (0), compute {r k } (0) Evaluate MSE Compute new values {t k } (1), and {r k } (1) For large number of quantization levels Approx. constant pdf within t [t k, t k1 ), i.e. p(t) = p(t k ) for t k =(t k t k1 )/ Reference: S.P. Lloyd: Least Squares Quantization in PCM, IEEE Trans. Info. Theory, vol.it-8, March 198, pp ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [3] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [4] 1

2 UMCP ENEE631 Slides (created by M.Wu 001/004) MMSE Quantizer for Uniform Distribution Uniform quantizer t L1 Optimal for uniform distributed r.v. in MMSE sense MSE = q / 1 with q = A / L SNR of uniform quantizer Variance of uniform distributed r.v. = A / 1 SNR = 10 log 10 (A / q ) = 0 log 10 L (db) If L = B, SNR = (0 log 10 )*B = 6B (db) 1 bit is worth 6 db. Rate-Distortion tradeoff p.d.f. of uniform distribution 1/A t L1 A ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [5] UMCP ENEE739M Slides (created by M.Wu 00) Quantization A Lossy Step in Source Coding Quantizer achieves compression in a lossy way Lloyd-Max quantizer minimizes MSE distortion with a given rate Need at least how many # bits for certain amount of error? (information-theoretic) Rate-Distortion theory Rate distortion function of a r.v. Minimum average rate R D bits/sample required to represent R this r.v. while allowing a fixed distortion D D For Gaussian r.v. and MSE log ( σ / D), D σ 1 R D = 0, D > σ (just use the mean) D 1bit more cuts down distortion to ¼ => 6dB See Info. Theory course/books σ for detailed proof of R-D theorem ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [6] Think More About Uniform Quantizer Compandor Uniformly quantizing [0, 55] to 16 levels Compare relative changes x1 = 100 [96, 11) quantize to 104 ~ 4% change x = 1 [0, 16) quantize to 8 ~ 33% change Large relative changes could be easily noticeable by eyes! u f(.) w Uniform y g(.) u compressor quantizer expander Compressor-Expander Uniform quantizer proceeded and succeeded by nonlinear transformations nonlinear function amplifies or suppresses particular ranges Why use compandor? Inputs of smaller values suffer higher percentage of distortion under uniform quantizer Nonlinearity in perceived luminance small difference in low luminance is more visible Overall companding system may approximate Lloyd-Max quantizer ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [7] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [8]

3 Compandor (cont d) Nonlinear transformation functions To make overall system approximate Lloyd-Max quantizer Without iterative process to determine parameters UMCP ENEE408G Slides (created by M.Wu & R.Liu 00) Quantization with Prediction Consider: high correlation between successive samples Predictive coding Basic principle: Remove redundancy between successive pixels and only encode residual between actual and predicted Residue usually has much smaller dynamic range Allow fewer quantization levels for the same MSE => get compression Compression efficiency depends on intersample redundancy First try: u(n) e(n) Quantizer _ Any problem with this codec? u P (n) = f[u(n-1)] Encoder (Jain s Fig.4.19) u P (n) = f[u Q (n-1)] Decoder ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [9] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [10] UMCP ENEE408G Slides (created by M.Wu & R.Liu 00) Predictive Coding (cont d) Problem with 1 st try Input to predictor are different at encoder and decoder decoder doesn t know u(n)! Mismatch error could propagate to future reconstructed samples Solution: Differential PCM (DPCM) Use quantized sequence for prediction at both encoder and decoder Prediction error e(n) Quantized prediction error e Q (n) Distortion d(n) = e(n) e Q (n) u(n) _ e(n) u P (n) = f[u Q (n-1)] Encoder Quantizer u P (n)=f[u Q (n-1)] Decoder Note: contains one-step buffer as input to the prediction More on Causality required for coding purpose Can t use the samples that decoder hasn t got as reference Use last sample u q (n-1) Equiv. to coding the difference p th order auto-regressive (AR) model p Use a linear predictor from past samples u ( n ) = a i u ( n i) i = 1 Determining the predictor coeff. (discuss more later) Line-by-line DPCM predict from the past samples in the same line -D DPCM predict from past samples in the same line and from previous lines e( n ) ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [11] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [1] 3

4 Comparison and Further Improvement From Jain s Fig.11.1 Adaptive technique (brief) Adapting the quantizer and predictor to variations in local image statistics Improve subjective image quality especially on boundary of different regions Ref. Jain s pp495 See more in ENEE634 Adaptive Signal Processing Vector Quantization Encode a set of values together Find the representative combinations Encode the indices of combinations Stages Codebook design Encoder Decoder Scalar vs. Vector quantization VQ allows flexible partition of coding cells VQ could naturally explore the correlation between elements SQ is simpler in implementation vector quantization of elements scalar quantization of elements From Bovik s Handbook Sec.5.3 ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [13] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [14] UMCP ENEE631 Slides (created by M.Wu 001/004) Outline of Core Parts in VQ Design codebook Optimization formulation is similar to MMSE scalar quantizer Given a set of representative points Nearest neighbor rule to determine partition boundaries Given a set of partition boundaries Probability centroid rule to determine representative points that minimizes mean distortion in each cell Search for codeword at encoder Tedious exhaustive search Design codebook with special structures to speed up encoding E.g., tree-structured VQ vector quantization of elements Reference: Wang s book Section 8.6 A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Publisher. R. M. Gray, ``Vector Quantization,'' IEEE ASSP Magazine, pp. 4--9, April UMCP ENEE631 Slides (created by M.Wu 004) Summary: List of Compression Tools Lossless encoding tools Entropy coding: Huffman, Lemple-Ziv, and others (Arithmetic coding) Run-length coding Lossy tools for reducing bit rate Quantization: scalar quantizer vs. vector quantizer Truncations: discard unimportant parts of data Facilitating compression via Prediction Convert the full signal to prediction residue with smaller dynamic range Encode prediction parameters and residues with less bits Facilitating compression via Transforms Transform into a domain with improved energy compaction ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [15] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [16] 4

5 UMCP ENEE631 Slides (created by M.Wu 004) Summary of Today s s Lecture Quantization Uniform vs non-uniform quantization (Lloyd-Max, companding) Quantization in predictive coding (careful in designing decoder) Scalar vs Vector quantization Next time Compacting signal energy via transforms Readings Jain s book ; (or) Gonzalez s book 8.3.1; 8.4; Wang s book 8.4; 8.5 Reference [Copy] Compandor design of Waveform coding book [Copy] Transform coding article in Sig.Proc. Magzine [Copy] Bovik s book Chapt.5.1 (Lossless coding) huffman, lempel-ziv, etc. [links of E-copy] Wallace s JPEG paper [VQ] A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. R. M. Gray, ``Vector Quantization,'' IEEE ASSP Magazine, pp. 4--9, April Y. Linde, A. Buzo, and R. M. Gray, ``An Algorithm for Vector Quantizer Design,'' IEEE Transactions on Communications, pp , January Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 3) ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [17] ENEE631 Digital Image Processing (Spring'06) Lec9 Basics on Compression [18] 5