Vector Quantization Encoder Decoder Original Form image Minimize distortion Table Channel Image Vectors Look-up (X, X i ) X may be a block of l

Transcription

1 Vector Quantization Encoder Decoder Original Image Form image Vectors X Minimize distortion k k Table X^ k Channel d(x, X^ Look-up i ) X may be a block of l m image or X=( r, g, b ), or a block of DCT coeff. X^ Codebook i, i=1,..., N c X^ Codebook i, i=1,..., N c dxxˆ (, i) = J 1 -- x J ( j xˆij ) 2 j = 1 d w ( X, Xˆ i) = J 1 -- w J j ( x j xˆij ) 2 j = 1 ( more useful in the transform domain )

2 Advantages Can achieve rate-distortion bound as vector dimension approaches infinite Support fractional bit allocation Very simple decoder: table look-up

3 Disadvantages As the vector dimension increases, the codebook size grows up exponentially, thus making codebook storage and searching impractical Local codebook has better performance, but requires the training and transmission of codebook Global codebook: inferior performance Optimal codebook is hard to obtain For low bit rate image coding, edges are blurred

4 Codebook Generation Sub-optimal codebook obtained by iteration. This is called the generalized Lloyd-Max algorithm Initial codebook assignment: LBG approach, other heuristic solutions Initial codebook assignment is very important to the quality of final codebook, due to local minimum in the quadratic optimization problem behind the Lloyd iteration

5 Lloyd Iteration for Training Data (a) Given a codebook C m = { y 1,..., y N }, partition the training set into cluster sets R i using the nearest neighbor condition: R i = { x A: d(x,y i ) d(x, y j ); all j i } (b) Using the centroid condition, compute the centroids for the cluster sets just found to obtain the new codebook: C m+1 = { cent( R i ) } If an empty cell was generated in (a), an alternate code vector assignment is made.

6 Codebook Generation Example x = training vectors o = codewords P i = region encoded into codeword i

7 Codebook Generation Example J = lm = 1 2 = 2, N c = 8, R = log 2 Nc / J = 1.5 bpp LENA as training vectors, ε = 0.001, MSE = 72.53

8 Codebook Initialization Random codes PNN ( Pairwise Nearest Neighbor ) clustering Start with N > N c clusters each containing one training vector. Merge two closest vectors to form N-1 clusters Repeat the process until N N c. The centroid of these are then used as the initial codebook Splitting

9 Codebook Design Full Search O( JN c ) both in computational as well as storage Tree Search Image Vector X Example: 3 level binary tree, each level uses LBG, N c = 8 X^ 1 (1) X^ 1 (2) X^ 1 (3) X^ 2 (3) X^ 2 (2) X^ 3 (3) X^ 4 (3) X^ 3 (2) X^ 5 (3) X^ 2 (1) X^ 6 (3) X^ 4 (2) X^ 7 (3) X^ 8 (3) Intermediate Intermediate Final Codevectors Codevectors Codevectors ( Level 1 ) ( Level 2 ) ( Level 3 ) No. of searches = 6, storage 14 codevectors. For full search, they are 8 and 8 respectively. In general, if each node has m branches and there are p = log m N c levels, the computational cost is O( Jmlog m N c ). However, the storage cost increase to Jm(N c -1)/(m-1). Tree search performs worse than full search!

10 Tree-structure Trade-off ( N c = 64 ) Technique Branches per node Number of nodes Computations Storage Bi-tree 2 6 O( 12J ) 126J Quad-tree 4 3 O( 12J ) 84J Oct-tree 8 2 O( 16J ) 72J Full search 64 1 O( 64J ) 64J Recall that R = (log 2 N c ) / J N c = 2 RJ J = lm = vector size Thus N c increases exponentially with J for a constant R. The solution is to use product codes

11 Product Codebook Codebook is formed as Cartesian product of several smaller codebooks Suitable for vector that are characterized by independent features, e.g., magnitude and orientation Final codeword is the concatenation of all different encoder outputs Effective size: N = N 1 N 2, while storage and computational complexity are proportional to N 1 + N 2 Suboptimal than full search for same codebook size

12 Mean / Residual VQ Many image vectors exhibit similar variations about different mean levels By removing mean, fewer codevectors are required to represent residuals Mean can be obtained within each vector and then scalar quantized Remove the quantized mean and code residual using VQ Can also remove the subsampled interpolation, then code residual using VQ ( Interpolative / Residual )

13 Mean-Removed VQ Blockdiagram Independent Quantizer Structure Alternative Structure

14 Interpolative / Residual VQ ( I/RVQ ) Blockdiagram Image 2-D Scalar Subsampling Q l:1 l:1 + Σ - 2-D Bilinear Interpolation 1:l 1:l Form Vectors VQ

15 Classified VQ Classify codebook by features, e.g., horizontal edge, vertical edge, uniform area, etc A classifier is needed to find features A set of small codebooks can provide comparable image quality, with lower search complexity Can also be used on the residual vectors

16 Classified VQ Blockdiagram VQ 1 size N 1 Image Form Vectors Block (Vector) Classifier VQ 2 size N 2 N c = M N i i = 1 No overhead bits are necessary since all possible codevectors are labelled from 1 to N c... VQ M size N M

17 VQ Results Both Mean/Residual and Interpolative/Residual are tree coded The codebook has 8 branches with 5 levels (size = 8 5 = 2 15 ) The size of vector 4 4 = 16 A level 0, the mean value for M/RTVQ is quantized at 8 bits, thus R 0 = 8 / (4 4) = 0.5 bpp For I/RTVQ, the image is subsampled 8:1 in 2-D and then quantized by 8 bits at level 0, R 0 = 8 / (8 8) = bpp At each level, the incremental bit rate is R = log 2 8 / (4 4) = bpp Training set consists of images with various characteristics

18 VQ Results Level Technique: M/RTVQ (Include) Bit rate bit/pixel RMSE (0~255) SNR (db) Level Technique: M/RTVQ (Exclude) Bit rate bit/pixel RMSE (0~255) SNR (db)

19 VQ Results Level Technique: I/RTVQ (Include) Bit rate bit/pixel RMSE (0~255) SNR (db) Level Technique: I/RTVQ (Exclude) Bit rate bit/pixel RMSE (0~255) SNR (db)

20 Residual Codebook for I/RTVQ Level 0 Level 1

21 I/RTVQ at 0.13 bit/pixel

22 I/RTVQ at 0.31 bit/pixel

23 I/RTVQ at 0.50 bit/pixel

24 Residual Image for I/RTVQ at 0.50 bit/pixel Original Residual Encoded Residual

25 I/RTVQ at 0.50 bit/pixel (magnified) Original Encoded

26 I/RTVQ at 0.69 bit/pixel

27 I/RTVQ at 0.87 bit/pixel

28 I/RTVQ at 1.06 bit/pixel

29 New Codebook Design Algorithm Motivation: The higher correlation between symbols, the lower bit rate can be achieved. Correlation between spatial vectors is usually high. VQ usually used in tandem with an entropy encoder Objective: Design vector quantizer to generate indices with higher inter-index correlation without sacrificing the fidelity.

30 New Codebook Design Algorithm (continued) Similarity Measure Potential: Definition: k P( x) = x 2 i i = 1, where k is the dimension of x Justification: P( x) P( xˆ ) P( x xˆ ) Solution: Order the codewords monotonically according to their potentials.

31 New Codebook Design Algorithm MLBG Algorithm After completion of the conventional LBG algorithm, insert an ordering process that orders the codewords just obtained, either ascendingly or descendingly, on the basis of their potentials.

32 Effects of MLBG Algorithm 2-D Index-Map LBG-codebook MLBG-codebook 2-D index map of Lena ( , outside the training set). Codebook size is 256 and vector dimensionality is 4 4.

33 Effects of MLBG Algorithm Distance Relationship P ( x xˆ ) P ( x xˆ ) codeword index codeword index LBG-codebook MLBG-codebook Distances between a typical vector to each codeword in a codebook.

34 Effects of MLBG Better Prediction LBG-codebook MLBG-codebook Histograms of indices (after horizontal DPCM) of Lena (512x512).

35 Performance of VQ with MLBG-Codebook Lena (512x512,outside training set) Goldhill (512x512,inside training set) PSNR-Rate plots for Lena and Goldhill.

36 Existing Fast Vector Quantization Search Algorithms Category 1 structurally constrained VQ Utilizes the special structure imposed on the codebook The results are generally inferior to full search. Category 2 premature exit or half-way stop technique. Intensively uses triangular inequality The results are the same as full search. Gain computation savings from distortion evaluation. A good prediction is crucial.

37 Window-based Fast Search (WBFS) Algorithm Two properties of a MLBG-codebook The distance tends to increase as the index moves away from the optimal index in either direction. Better prediction is achievable. Threshold: Take care of the rare cases when prediction is not accurate enough. Control the overall performance of the WBFS. P( x) P( xˆ ) > threshold Px ( xˆ ) > threshold Empirical threshold decision formula threshold = d 255 e = psnr 20 10

38 Window-based Fast Search (WBFS) Algorithm Step 1: Initialization. (get threshold and window size (assume 2w+1)). Step 2: Apply full search for the first vector. Step 3: Build / update the search window around predicted index. Search Window = [I pre - w, I pre + w]. Step 4: Search the window. If d wmin < threshold, go to Step 6. Step 5: Do full search. Step 6: Switch to next vector and go to step 3.

39 Speed-up of WBFS Algorithm Best window size is: min(max(speed-up) = 1.625, computation saving = 38.5% max(max(speed-up) = 2.19, computation saving = 52.3%

40 Rate-Distortion Performance of WBFS Algorithm Lena (512x512,outside training set) Goldhill (512x512,inside training set) Rate-distortion plots of WBFS for Lena and Goldhill.

41 Additional References 1. R. M. Gray, Vector Quantization, IEEE ASSP Magazine, pp 4-29, April A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Publishers, G. B. Shen and M. L. Liou, An efficient codebook post-processing techniques and a window-based fast search algorithm for image vector quantization, to appear, IEEE Trans. Circuits and Systems for Video Technology