Fundamentals of Image Compression

Transcription

1 Fundaentals of Iage Copression Iage Copression reduce the size of iage data file while retaining necessary inforation Original uncopressed Iage Copression (encoding) Decopression (decoding) Copressed File Decopressed Iage 1 Fundaentals of Iage Copression Fundaentals of Iage Copression Copression ratio the ration of the original, uncopressed iage file and the copressed file Uncopressed File Size SIZE Copressio n Ratio = = Copressed File Size SIZE U C 2 Fundaentals of Iage Copression

2 Iage Copression (cont.) Bits per pixel (bpp) denote the nuber of bits required to represent an iage pixel Nuber of Bits 8(Nuber of Bytes) bpp = = Nuber of Pixels N N Data the pixel gray-level values that correspond to the brightness of a pixel at a point in space Inforation an interpretation of the data in a eaningful way 3 Fundaentals of Iage Copression Iage Copression Techniques Inforation preserving (lossless copression) No data are lost original iage can be recreated exactly fro copressed data Copression ratio 2:1 to 3:1 Useful in iage archiving (such as legal or edical iages) Inforation lossy (lossy copression) Allow a loss in the actual iage data original iage cannot be recovered exactly fro copressed data Copression ratio 10:1 to 20:1or ore Useful for broadcast television, video-conferencing, and facsiile transission, etc. 4 Fundaentals of Iage Copression

3 Iage/Video Copression Techniques Iage/Video Copression Predictive Coding Transfor Coding Iportance Oriented Coding Hybrid Coding Delta odulation DCT Bit allocation JPEG DPCM KL Sub-sapling H.26X ME/MC Subband Wavelet Quantization Scalar Vector MPEG-X 5 Fundaentals of Iage Copression How To Do Iage Copression Basis - redundancy exists in iage data and can be reoved Types of iage data redundancy: Coding redundancy (statistical redundancy) -variablelength code (RLC, Huffan Coding, Arithetic Coding, LZW) Interpixel redundancy (spatial redundancy) apper (DPCM, DCT, Subband, Wavelet) Psychovisual redundancy (perceptual redundancy) quantization (SQ, VQ, fractal) Teporal redundancy (interfrae redundancy), for video) DPCM, MC/ME 6 Fundaentals of Iage Copression

4 Interpixel Redundancy The gray levels of adjacent pixels in noral iages are highly correlated, resulting in interpixel redundancy The gray level of a given pixel can be predicted by its neighbors and the difference is used to represent the iage; this type of transforation is called apping Run-length coding can also be eployed to utilize interpixel redundancy in iage copression Reoving interpixel redundancy is lossless 7 Fundaentals of Iage Copression Interpixel Redundancy (cont.) In run-length coding, each run R i is represented by a pair (g i, r i ) with g i = gray level of R i r i = length of R i 8 Fundaentals of Iage Copression

5 Psychovisual Redundancy Certain inforation are ore iportant than other inforation to the huan visual syste Huans perceives spatial frequencies below about 50 cycles per degree so that higher-frequency inforation is less iportant Requantization using less bits can eliinate psychovisual redundancy, but false contour will appear For color iages, color subsapling can be used to reduce the size of color coponents since huan eyes are less sensitive to variations in color than variations in light intensity Reoving psychovisual redundancy is inforation lossy 9 Fundaentals of Iage Copression Requantization Requantization the process of liiting the value of a function at any saple to one of a predeterined nuber of perissible values A general usage or gray-scale reduction is to reduce the nuber of bits per pixel (bpp) False contouring as the nuber of gray levels decreases, false edges or lines will appear in the iage The false contouring effect can be visually iproved upon by using an IGS (Iproved Gray-Scale) quantization adding a sall rando nuber to each pixel before quantization 10 Fundaentals of Iage Copression

6 Requantization (cont.) 8 bits 7 bits 6 bits 5 bits 4 bits 3 bits 2 bits 1 bits 11 Fundaentals of Iage Copression Requantization (cont.) Types of Quantizer Scalar Quantizers» Unifor Scalar Quantizer» Non-unifor Scalar Quantizer» The Lloyd-Max Quantizer» Entropy-constrained Quantizer Vector Quantizer 12 Fundaentals of Iage Copression

7 Unifor Scalar Quantizer Unifor scalar quantizer the siplest quantizer Masking the lower bits via an AND operation Exaple reduce 8-bit pixel (256 levels) to be 3-bit pixel (8 levels) by ANDing with the bit string Fundaentals of Iage Copression Unifor Scalar Quantizer Input range 0-31 Output code 000 Reconstruction value (0) MSE Reconstruction value (16) MSE (32) (48) (64) (80) (96) (112) (128) (144) (160) (176) (192) (208) (224) (240) 14 Fundaentals of Iage Copression

8 IGS quantization IGS quantization procedure: 1. Set initial SUM = if ost significant 4 bits of current pixel A = 1111 new_sum = A else new_sum = A + least significant 4 bits of old SUM Pixel Gray Level Su IGS Code i - 1 N/A N/A i i i i Fundaentals of Iage Copression IGS quantization (cont.) 16 Fundaentals of Iage Copression

9 Non-unifor Scalar Quantizer Idea - the quantization levels are finely spaced in the range that gray levels occur frequently, while coarsely spaced outside of it 17 Fundaentals of Iage Copression The Lloyd-Max Quantizer Optial quantization A quantization function can be regarded as a staircase, in which s i is called decision levels and t i reconstruction levels s is apped to t i if it lies in the half-open interval (s i -1, s i ] 18 Fundaentals of Iage Copression

10 The Lloyd-Max Quantizer (cont.) 19 Fundaentals of Iage Copression The Lloyd-Max Quantizer (cont.) How to select s i and t i? The Llooyd-Max quantizer Assuing the criterion is MSE, E{(s - t i ) 2 }, and p(s) is even The condition for inial error are si s i 1 ( s ti ) p( s) ds = 0 i = 1,2,..., L/2 0 i = 0 si = ( ti + ti +1)/2 i = 1, 2,..., L/2 1 and i = L/2 s -i = -s i, t -i = -t i» Each t i is the centroid of the area under p(s) over two s i» s i is halfway between two t i 20 Fundaentals of Iage Copression

11 Entropy-constrained Quantizers Ai at transitting the iage with as few bits as possible rather than looking for a quantizer that provides the lowest distortion for a given nuber of output levels Entropy-constrained quantizer (ECQ) a quantizer that iniizes distortion for a given entropy easured at the quantizer output The quantizer output is variable-length encoded In iage copression, the unifor quantizer followed by variable-length coding will produce the results close to the optiu ECQ 21 Fundaentals of Iage Copression Color Quantization 16,777,216 colors 256 colors 22 Fundaentals of Iage Copression

12 Color subsapling Interpolation Fundaentals of Iage Copression Fidelity Criteria Fidelity criteria is used to easure inforation loss Two classes: objective fidelity criteria - easured atheatically about the aount of error in the reconstructed data subjective fidelity criteria - easured by huan observation 24 Fundaentals of Iage Copression

13 Fidelity Criteria (cont.) Subjective fidelity criteria: Ipairent test how bad the iages are Quality test how good the iages are Coparison test Ipairent 5 - Iperceptible 4 Perceptible, not annoying 3 Soewhat annoying 2 Severely annoying 1 - Unusable Quality A - Excellent B - Good C - Fair D - Poor E - Bad Coparison +2 uch better +1 better 0 the sae -1 worse -2 uch worse 25 Fundaentals of Iage Copression Fidelity Criteria (cont.) Objective criteria root-ean-square (rs) error: e RMS e 1 ˆ M 1 N 1 2 RMS = [ I ( r,c) I( r,c)] MN r = 0 c= 0 26 Fundaentals of Iage Copression

14 Fidelity Criteria (cont.) Objective criteria (cont.) root-ean-square signal-to-noise ratio: SNR RMS SNR RMS r= 0 c= 0 = M 1 N 1 r= 0 c= 0 M 1 N 1 Iˆ( r, c) [ Iˆ( r, c) I( r, c)] Fundaentals of Iage Copression Fidelity Criteria (cont.) Objective criteria (cont.) peak signal-to-noise ratio: SNR PEAK SNR PEAK = 10 log10 1 M 1 MN where L is the axiu N 1 r= 0 c= 0 saple ( L 1) 2 [ Iˆ( r, c) I( r, c)] value 2 28 Fundaentals of Iage Copression

15 Iage Copression Model I(r, c) Preprocessing Encoder Copression (encoding) Copressed File Î(r, c) Postprocessing Decoder Decopression (decoding) The encoder reoves input redundancies (interpixel, psychovisual, and coding redundancy) 29 Fundaentals of Iage Copression Encoder and Decoder I(r, c) Î(r, c) Mapper Inverse apper Quantizer encoder decoder Sybol encoder Sybol decoder Copressed File apper - reduce interpixel redundancy, reversible (predictive coding, run-length coding, Transfor coding) quantizer - reduce psychovisual redundancy, irreversible (unifor quantization, nonunifor quantization) sybol encoder - reduce coding redundancy, reversible (Huffan coding, arithetic coding, LZW) 30 Fundaentals of Iage Copression

16 Predictive Coding 31 Fundaentals of Iage Copression Predictive Coding + + Quantization Bit Allocation Predictor Fundaentals of Iage Copression

17 Predictive Coding (cont.) Input data I n + Ĩ n - e n Predictor Encoder Quantizer Î n + + ê n Sybol encoder Copressed data Decoder Copressed data Sybol decoder ê n + + Î n Decopressed data Ĩ n Predictor 33 Fundaentals of Iage Copression Predictive Coding (cont.) In general, the prediction is fored by a linear cobination of previous saples: Iˆ n = round [ α I i = 1 n i where is the order of the linear predictor, round is a function to denote the rounding or nearest integer operation, and the α i are prediction coefficients i ] 34 Fundaentals of Iage Copression

18 Predictive Coding (cont.) For 2-D iages, the prediction is a function of the previous pixels in a left-to-right, top-to-botto scan of an iage I(r-1, c-1) I(r-1, c) I(r-1, c+1) I(r, c-1) I(r, c) 35 Fundaentals of Iage Copression Predictor (cont.) I(r-1, c-1) I(r-1, c) Exaples: First-order predictor: ˆ (, ) ( - I(r, c-1) I(r, c) I r c = I r,c 1) Second-order predictor: I ˆ( r, c) = 1/ 2[ I ( r,c -1) + I ( r -1, c)] Third-order predictor:» I ˆ( r, c) = I ( r,c -1) - I ( r -1,c -1) + I ( r -1, c) I(r-1, c+1)» I ˆ( r, c) = 0.75 I ( r,c -1) I ( r -1,c -1) I ( r -1, c) 36 Fundaentals of Iage Copression

19 Predictive Coding (cont.) In 3-D case, the prediction is based on the above pixels and the previous pixels of preceding fraes 37 Fundaentals of Iage Copression Differential Pulse Code Modulation (DPCM) Differential pulse code odulation (DPCM) a for of predictive coding suitable for use with Markov sources Use the fact that the intensity value of a pixel is likely to be siilar to that of the surrounding pixels Eliinating interpixel redundancies Instead of coding an intensity value, we predict the value fro the values of nearby pixels The difference between the actual and predicted value of that pixel is coded There is a high probability that the difference will be sall 38 Fundaentals of Iage Copression

20 DPCM (cont.) The predictive coding syste consists of an encoder and a decoder, each containing an identical predictor The predictor transfors a set of high-entropy but correlated values into a set of low-entropy but less correlated values The aount of inforation is reduced Optial predictor We shall iniize the variance of e = x i.e., σ e = E{ ( x xˆ ) } = E x α i 0 = i xˆ, 2 xi 39 Fundaentals of Iage Copression DPCM (cont.) To find the inial value, we partial differentiate the following and let it be zero : E 2 [ ( x xˆ ) ] α E = i = -2E 2 {[ x ( α x + α x + L + α x )] } α 1 1 [( x xˆ ) x ] = 0 i = 0, 1, L,-1 (1) i i 40 Fundaentals of Iage Copression

21 DPCM (cont.) Define the autocorrelation as R = E Fro (1) : E R { x x } = E{ xˆ x } i i = E ij ( xi x j { α x x + α x x + L + α x x } = α R 0 0 0i 0 i i + α R 1 = E ( 1 1i the equations in (2) 1 i 1 i= 0 + L + α αixi ) xi -1 R 1 ( 1 )i 1 i = 0, 1, L, -1 (2) α, i = 0,1, L, 1can be obtained by solving i i ) 41 Fundaentals of Iage Copression DPCM (cont.) If xˆ i is obtained by the predictor that uses α thus obtained, then 2 e But E σ R σ σ 2 e 2 e = E 2 {( x xˆ ) } = E{ ( x ˆ } { ˆ ˆ x ) x -E ( x x ) x} {( x ˆ ) ˆ x x} = 0 fro Eq. (1) E{ ( x xˆ ) x } = = E = R : can : can 2 [ x ]- E[ xˆ x ] 0 ( α R 0 be considered + α R be considered 1 1 as the + L + α as the Thus, 1 variance R variance the optial 1 of of ) the (3) error original signal signal 42 Fundaentals of Iage Copression

22 DPCM (cont.) Eq.(3) verifies that variance of saller than that of the original signal! the error signal is Thus, it worths DPCM! 43 Fundaentals of Iage Copression DPCM (cont.) 44 Fundaentals of Iage Copression

23 Adaptive Predictor Adaptive predictor several predictor are predefined, select the best predictor according to the characteristics of the data Try out a nuber of predictors and deterine which is the best, then tell the decoder which predictor has been used Consider blocks of pixels and choose aong n predictors, the overhead is log 2 n bits per block, or (log 2 n)/ bpp Deterine the choice of predictor fro the data values in[ I( r,c-1), I( r -1,c )] Iˆ( r, c) = ax[ I( r,c-1), I( r -1,c )] I( r -1,c) + I( r,c-1)- I( r -1,c- 1) if I( r -1,c- 1) ax[ I( r,c-1), I( r -1,c)] if I( r -1,c- 1) in[ I( r,c-1), I( r -1,c)] otherwise 45 Fundaentals of Iage Copression Results (cont.) DPCM at 1.0 bpp 46 Fundaentals of Iage Copression

24 Results (cont.) DPCM at 2.0 bpp 47 Fundaentals of Iage Copression Results (cont.) DPCM at 3.0 bpp 48 Fundaentals of Iage Copression

25 Results (cont.) DPCM at 2.0 bpp 49 Fundaentals of Iage Copression Conclusions Lossless JPEG 0 1 Prediction No A 前一行... B C D A X B C A+B-C 5 A-(B-C)/2 6 B-(A-C)/2 7 (A+B)/2 50 Fundaentals of Iage Copression

26 Conclusions (cont.) Post processor JPEG: transfored dc coponents MPEG-4: transfored dc, ac 51 Fundaentals of Iage Copression Entropy Coding 52 Fundaentals of Iage Copression

27 Entropy Coding Inforation Theory Huffan Coding Arithetic Coding 53 Fundaentals of Iage Copression Inforation Theory Inforation theory provides the atheatical fraework to deterine the iniu aount of data that is sufficient to describe copletely an iage without loss of inforation A event E that occurs with probability P(E) will contain 1 I( E) = log = log P( E) P( E) unit of inforation, I(E) is called the self-inforation of E 54 Fundaentals of Iage Copression

28 Inforation Theory (cont.) I(E) is a easure of uncertainty/unpredictability and is inversely related to P(E) If P(E) = 1, I(E) = 0: no inforation (surprise) is contained On the other hand, if the probabilities are all very different, then when a sybol with a low probability arrives, you feel ore surprised, get ore inforation, than when a sybol with a higher probability arrives If the base r logarith is used, the easureent is said to be in r-ary unit; if r = 2, the resulting is called a bit 55 Fundaentals of Iage Copression Inforation Theory (cont.) If we get I(s i ) units of inforation when we receive the sybol s i, how uch do we get on the average? A source alphabet S for a set of n sybols {s 1, s 2,..., s n } with the sybol probabilities {P(s 1 ), P(s 2 ),..., P(s n )}, define H ( S) = p( s ) I( s ) + p( s ) I( s ) + L+ p( s = n i= p( s ) I( s ) = i i 2 n i= 1 2 p( s )logp( s ) i i n ) I( s n ) H(S) is called the uncertainty or entropy of the source The entropy defines the average aount of inforation obtained by observing a single source output 56 Fundaentals of Iage Copression

29 Inforation Theory (cont.) If the source sybols are equally probable, the entropy is axiized and the source provides the greatest possible average inforation per source sybol H(P) P 57 Fundaentals of Iage Copression Inforation Theory (cont.) Exaple: S p(s i ) I(s i ) p(s i ) I(s i ) p(s i ) I(s i ) s s s s H(S) Fundaentals of Iage Copression

30 Inforation Theory (cont.) Instantaneous Codes : a code is said to be instantaneous if when a coplete sybol is received, the receiver iediately knows this, and do not have to look further before deciding what essage sybol is received. No encoded sybol of this code is a prefix of any other sybol 59 Fundaentals of Iage Copression Inforation Theory (cont.) Exaple : s 1 = 0 s 1 = 0 s 2 = 10 s 2 = 01 s 3 = 110 s 3 = 011 s 4 = 111 s 4 = 111 instantaneous non-instantaneous s 2 s 3 s 1 s s 3 s 4 s 4 No code word s i which is a prefix of another code word, s j 60 Fundaentals of Iage Copression

31 Inforation Theory (cont.) Shannon s noiseless source coding theore for a uniquely decodable code, the average nuber of bits per sybol used by the code ust be at least equal to the entropy of the source H ( S) L avg = q i= 1 p l The entropy H(S) provides the iniu aount of data that is sufficient to describe data without loss of inforation i i 61 Fundaentals of Iage Copression Inforation Theory (cont.) Exaple: S p(s i ) I(s i ) Code (length) p(s i ) I(s i ) Code (length) s (1) (1) s (2) (2) s (3) (3) s (3) (3) Average bits Fundaentals of Iage Copression

32 Variable-Length Coding Idea - assigning the shortest possible code words to the ost probable sybols The source sybols ay be the gray levels of an iage or a gray-level apping operation (pixel differences, run-lengths, etc.) Two widely used variable-length coding ethod Huffan coding generate an optial code in ters of the nuber of code sybols subject to the constraint that the source sybols are coded one at a tie» Adopted in JPEG, MPEG Arithetic coding an entire sequence of sybols is assigned a single arithetic codeword 63 Fundaentals of Iage Copression Huffan Coding Encoding steps: Initialization: Given the probability distribution of the source sybols Step 1: source reduction source reduction» Order the probabilities of the source sybols» Cobine two lowest probability sybols into a single sybol that replaces the in the next source reduction Step 2: codeword construction codeword construction» Code each reduced source starting with the sallest source and work back to the original source 64 Fundaentals of Iage Copression

33 Huffan Coding (cont.) Source reduction process: H ( z) = P( ai )logp( ai ) = [ 0.4log(0.4) + 0.3log(0.3) log(0.1) log(0.06) log(0.04) ] = 2.14 n i= 1 65 Fundaentals of Iage Copression Huffan Coding (cont.) Codeword construction process: L L 1 avg = k= 0 l( a ) p( k a k ) = 1 (0.4) + 2 (0.3) + 3 (0.1) + 4 (0.1) + 5 (0.06) + 5 (0.04) = Fundaentals of Iage Copression

34 Huffan Decoding Concept reconstruct the binary coding tree to the decoder fro the sybol to codeword table Decoding steps: Step 1: Read the input copressed strea bit by bit and traverse the tree until a leaf is reached Step 2: Discard each input bits Step 3: When a leaf node is reached, output the sybol at the leaf node, this will coplete the decoding for this sybol; go to Step 1 for the decoding of next sybol 67 Fundaentals of Iage Copression Modified Huffan Codes An Exaple: assue the source has a range of 510 to +511 S p(s i ) I(s i ) Code Bits All other values cobined plus value (10 bits) Fundaentals of Iage Copression

35 Arithetic Coding Huffan coding encodes individual sybols relies heavily on accurate knowledge of sybol statistics disadvantage of Huffan coding the length of Huffan codes have to be an integer, while the entropy value of a sybol ay be a faction nuber Theoretical possible copressed essage cannot be achieved For exaple, if p(s) = 0.9, then I(s) = log(0.9) = 0.15 bits 69 Fundaentals of Iage Copression Arithetic Coding (cont.) Arithetic coding An entire sequence of sybols is assigned a single arithetic codeword The codeword defines an interval of real nubers between 0 and 1 Theoretically achieve the axiu copression 70 Fundaentals of Iage Copression

36 Arithetic Coding (cont.) Exaple: S i P i subinterval k 0.05 [0.00,0.05) l 0.20 [0.05,0.25) 0.70 u 0.10 [0.25,0.35) w 0.05 [0.35,0.40) 0.40 e 0.30 [0.40,0.70) 0.35 r 0.20 [0.70,0.90) 0.25? 0.10 [0.90,1.00) Alphabet (k, l, u, w, e, r,?)? : end-of-essage ? r e w u l k 71 Fundaentals of Iage Copression Arithetic Coding (cont.) l l u u r e? ???????? r r r r r r r r e e e e e e e e w w w w w w w w u u u u u u u u l l l l l l l l k k k k k k k k Fundaentals of Iage Copression

37 Arithetic Coding (cont.) Encoding process Low = 0.0 ; high =1.0 ; while not EOF do range = high - low ; read(c) ; high = low + range high_range(c) ; low = low + range low_range(c) ; endofwhile output(low); 73 Fundaentals of Iage Copression Arithetic Coding (cont.) New character Low value high value l l u u r e ? Fundaentals of Iage Copression

38 Arithetic Coding (cont.) Decoding is the inverse process Since falls between 0.05 and 0.25, the first character ust be L. Reoving the effect of L fro by first subtracting the low value of L, 0.05: = Then divided by the range of L, 0.2: /0.2 = Then calculate where that lands, which is in the range of the character L The process repeats until 0 or the known length of the essage is reached 75 Fundaentals of Iage Copression Arithetic Coding (cont.) Decoding algorith r = input_code repeat search c such that r falls in its range ; output(c) ; r = r - low_range(c) ; r = r/(high_range(c) - low_range(c)); until EOF or the length of the essage is reached 76 Fundaentals of Iage Copression

39 Arithetic Coding (cont.) r c Low High range L L U U R E ? Fundaentals of Iage Copression Arithetic Coding (cont.) In suary, the encoding process is siply one of narrowing the range of possible nubers with every new sybol The new range is proportional to the predefined probability attached to that sybol Decoding is the inverse procedure, in which the range is expanded in proportion to the probability of each sybol as it is extracted 78 Fundaentals of Iage Copression

40 Arithetic Coding (cont.) Coding rate approaches high-order entropy theoretically Not so popular as Huffan coding because, are needed. Average bits/byte on 14 files (progra, object, text, and etc.) Huff. Arithetic Fundaentals of Iage Copression Conclusions JPEG, MPEG-1/2 uses Huffan and arithetic coding preprocessed by DPCM JPEG2000, MPEG-4 uses arithetic coding only 80 Fundaentals of Iage Copression