CHAPTER 3. Implementation of Transformation, Quantization, Inverse Transformation, Inverse Quantization and CAVLC for H.

Size: px

Start display at page:

Download "CHAPTER 3. Implementation of Transformation, Quantization, Inverse Transformation, Inverse Quantization and CAVLC for H."

Richard Fox
5 years ago
Views:

1 CHAPTER 3 Implementation of Transformation, Quantization, Inverse Transformation, Inverse Quantization and CAVLC for H.264 Video Encoder 3.1 Introduction The basics of video processing in H.264 Encoder are presented in detail in Chapter One. This Chapter presents the mathematical equations and theoretical approach to implement various blocks of Advanced Video Encoder namely Transformation, Quantization, Inverse Transformation and Inverse Quantization CAVLC. The algorithm for nine intra prediction modes of Encoder in MATLAB is presented in the next Chapter. The block diagram of H.264 Video Encoder is shown in Fig.3.1. Figure 3.1: Modules implemented in H.264 Encoder Block Diagram The modules designed in MATLAB for functional verification is shown as shaded in the block diagram. H.264 encoder has a forward path and a reconstruction path. The forward path is used to encode a video frame by using intra and inter 42

2 predictions and to create the bit stream. The reconstruction path is used to decode the encoded frame and to reconstruct the decoded frame. Since a decoder never gets original images, but rather works on the decoded frames, reconstruction path in the encoder ensures that both encoder and decoder use identical reference frames for intra and inter prediction. This avoids possible encoder decoder mismatches [1, 3, 4]. Forward path starts with partitioning the input frame into MBs. Each MB is encoded in intra or inter mode depending on the mode decision. In both intra and inter modes, the current MB is predicted from the reconstructed frame. Intra mode generates the predicted MB based on spatial redundancy, whereas inter mode, generates the predicted MB based on temporal redundancy. In both cases, the predicted MB is subtracted from the current MB to generate the residual MB. Residual MB is transformed using 4x4 and 2x2 integer transforms. Transformed residual data is quantized. The quantized and transformed coefficients are re-ordered in a zig-zag scan order. The reordered quantized transform coefficients are entropy encoded. The entropy-encoded coefficients together with header information, such as MB prediction mode and quantization step size, form the compressed bit stream. The compressed bit stream is passed to network abstraction layer (NAL) for storage or transmission [1, 3, 4]. Reconstruction path begins with inverse quantization and inverse transform operations. The quantized, transformed coefficients are inverse quantized and inverse transformed to generate the reconstructed residual data. Since quantization is a lossy process, inverse quantized and inverse transformed coefficients are not identical to the original residual data. The reconstructed residual data are added to the predicted pixels in order to create the reconstructed frame. A de-blocking filter is applied to reduce the effects of blocking artifacts in the reconstructed frame [1, 3, 4]. The intra prediction algorithm used in H.264 reduces spatial redundancies by exploiting the spatial correlation between adjacent blocks in a given picture. Each picture is divided into pixel MBs and each MB is composed of luma and chroma components. Intra prediction module predicts the pixels in a MB using the pixels in the available neighbouring blocks. For the luma component of a MB, a 16x16 predicted luma block is formed either by performing intra predictions for each 4x4 luma block in the MB or by performing intra prediction for the 16x16 MB. There are nine prediction modes for each 4x4 luma block and four prediction modes for a 16x16 luma block. For the chroma components of a MB, a predicted 8x8 chroma block is formed for each 8x8 chroma component by performing intra prediction for 43

3 the MB. There are four prediction modes for each chroma component. In the present work all nine Intra prediction modes for 4x4 luma block are implemented in MATLAB to confirm the functionality of prediction block before taking up Verilog realization. The following sections explain the process of developing the Transformation Quantization, Inverse Transformation and Inverse Quantization algorithms in step wise. 3.2 H.264 Transformation and Quantization The basic transform coding process in H.264 is similar to that of previous standards. The process includes a forward transform and quantization followed by zigzag ordering and entropy coding. The transform coded residual data is also reconstructed. The reconstruction process includes an inverse quantization and inverse transform followed by motion compensation. The reconstructed data before deblocking filter is used for intra prediction [7,8] in current frame, and the reconstructed data after de-blocking filter is used for motion estimation in future entropy coding and reconstruction process. The complete process is as shown in Fig Prediction Transformation & Quantization Entropy Encoding Input Video Out put Coded sequence Entropy encode Rescale & Inverse Transformation Reconstruct Input Coded sequence Output Video Figure 3.2: Transformation and quantization in H.264 CODEC The transformation and quantization algorithms process the video input in macro blocks and send the resulted data to entropy coding. The reconstruction process order is also shown in Fig H.264 uses three transforms depending on the type of residual data that is to be coded, a transform for the 4x4 array of luma DC coefficients in intra macro blocks (predicted in 16x16 mode), a transform for the 2x2 44

4 array of chroma DC coefficients (in any macro block) and a transform for all other 4x4 blocks in the residual data Developing forward Transformation and Quantization process The Transform and quantization process are structured such that computational complexity is minimized. This is achieved by reorganizing the process into a core part and scaling part as given in Fig.3.3. Consider a block of pixel data X is to be processed by a two dimensional DCT and followed by quantization. The complete process is shown in Fig.3.3. The steps of developing forward Transformation and Quantization followed are explained below. X Discrete Cosine Transform (DCT) Round 1 (Qstep) X Forward Core Transform (C f ) Forward Scaling Factor (S f ) Round 1 (Qstep) X Forward Core Transform (C f ) Forward Scaling Factor (S f ) 2 15 Qstep 2 15 Qstep X Forward Core Transform (C f ) Multiplication factor(m f ) 2 15 Qstep Figure 3.3: Development of Forward Transform and Quantization Developing C f and S f Consider a 4X4 two dimensional DCT of block X = A. X. A T (3.1) Where. Indicates matrix multiplication A = (3.2) 45

5 Where = ½ b = cos = c = cos = The Calculation of Equation 3.1 on practical processor requires approximation of the irrational numbers b and c. A fixed point approximation is equivalent to scaling each row of A and rounding to the nearest integer. Choosing the multiplication factor 2.5 and rounding presents C f. C f = (3.3) The implementation of C f requires only additions and binary shifts, which minimizes the complexity. To restore the orthogonal matrix A, multiply all the values C ij in row by 1/ A 1 = C f. R f (3.4). Indicates element-by-element multiplication R f is a matrix given by R f = 1/2 1/2 1/2 1/2 1/ 10 1/ 10 1/ 10 1/ 10 1/2 1/2 1/2 1/2 1/ 10 1/ 10 1/ 10 1/ 10 (3.5) The two dimentional transform becomes = A 1. X. A 1 T (3.6) Substitute for A1 in the above equation = [ C f. R f ]. X. [ C f T. R f T ] (3.7) Rearrange the equation = [ C f. C f T ]. X. [ R f. R f T ] (3.8) = [ C f. X. C f T ]. S f 46

6 Where S f = R f. R f T S f =R f. R f T = 1/4 1/2 10 1/4 1/2 10 1/2 10 1/10 1/2 10 1/10 1/4 1/2 10 1/4 1/2 10 1/2 10 1/10 1/2 10 1/10 (3.9) Scale the quantization process by a constant (2 15 ) and compensate by dividing and rounding the final result. Combine S f and the quantization process in to M f M f =! ". $ %& ' ()*+ (3.10) Developing Inverse Transformation and Inverse Quantization The following Fig.3.4 explains the Inverse Transformation and Inverse quantization operation. IDCT process is rearranged as Core transformation (C i ) and scaling Matrix (S i ) Scale the rescaling process by constant 2 6 and compensate by dividing and rounding the final result. The steps to obtain C i and S i are as follows. Q step IDCT Z Q step S i C i Z Q step.2 6 S i C i Z Q step.2 6 C i Z Figure 3.4: Development of Rescaling and Inverse Transform Process 47

7 3.2.4 Developing C i and S i Consider a 4X4 two dimensional IDCT of block Z= A T.. A (3.11) Where. indicates matrix multiplication and matrix A is given below A = (3.12) Where = ½ b = cos = c = cos = Choose a particular approximation by scaling each row of A and rounding to the nearest 0.5, which gives C i C i = /2 1/ / /2 (3.13) The rows of C i are orthogonal but have non-unit norms, to restore orthonormality multiply all the row C ij by 1/ Let A 2 = C i. R i R i = 1/2 1/2 1/2 1/2 2/5 2/5 2/5 2/5 1/2 1/2 1/2 1/2 2/5 2/5 2/5 2/5 The two dimensional Inverse transform becomes (3.14) = A 2.. A 2 T = [ C i T. R i T ].. [ C i. R i ] (3.15) Rearrange the equation Z = [ C i T ]. [. R i T. R i ]. [Ci] Z = [ C T i ]. [. S i ]. [ Ci ] (3.16) 48

8 Where S i = R it. R i S i =R i T. R i = 1/4 1/ 10 1/4 1/ 10 1/ 10 2/5 1/ 10 2/5 1/4 1/ 10 1/4 1/ 10 1/ 10 2/5 1/ 10 2/5 (3.17) The core Inverse transforms C i and the rescaling matrix V i are defined in H.264 standard. The development of rescaling matrix is as follows Developing V i (Rescaling Matrix) The Rescaling Matrix is given by V i = S i. Qstep.2 6 H.264 supports a range of quantization step sizes Qstep. The precise step sizes are not defined in the standard, rather the scaling Matrix V i is specified. Qstep values corresponding to the entries in V i are shown below Table 3.1. Table 3.1: Qstep values corresponding to V i QP Qstep QP Qstep QP Qstep QP Qstep The ratio between successive Q step values is chosen to be 2 = , so that Q step doubles in size when QP increases by 6. Any value of Q step can be derived from first 6 values in the table (QP0- QP5). Q step (QP) = Q step (QP%6). 2 floor(qp/6) for higher values of QP the corresponding values in V i are doubled. There are only three unique values in each matrix V i. These three values are defined as a table of values V in the H.264 standard for QP=0 to QP=5. 49

9 P V(r,0) V i Positions (0,0), (0,2),(2,0), (2,2) Table 3.2: Values of V V(r,1) V i Positions (1,1), 1,3),(3,1), (3,3) V(r,2) Remaining V i Positions Hence for QP values from 0 to 5, V i is obtained as V i = V(QP, 0) V(QP, 2) V(QP, 0) V(QP, 2) V(QP, 2) V(QP, 1) V(QP, 2) V(QP, 1) V(QP, 0) V(QP, 2) V(QP, 0) V(QP, 2) V(QP,2) V(QP,1) V(QP,2) V(QP,1) (3.18) In general Vi =V(QP, n) For larger values of QP (QP>5) V i = V(QP%6, n). 2 floor(qp/6) The complete inverse transformation and scaling process Z = round ([C i T ]. [. V (QP %6,n). 2 floor (qp/6) ]. [C i ]. ½ 6 ] (3.19) The Multiplication factor M f M f S i. S f V i Substitute for S i S f and V i M f = round S i. S f V i The multiplication factor table is as given below 50

10 QP Table 3.3: Multiplication factor Positions (0,0), (2,0), (2,2), (0,2) MF Positions (1,1), (1,3), (3,1), (3,3) MF Other positions MF The complete forward transform scaling and Quantization process is =round ([C f ]. []. [C T f ].M(QP%6,n)/2 floor(qp/6). ½ 15 ) (3.20) The complete process of Transformation, Quantization, Inverse Transformation and Inverse Quantization is summarized as shown in Fig.3.5 Z W ij Input block X Forward transform Cf Post-scaling and quantisation Encoder output / Decoder input Rescale and pre-scaling Inverse transform Ci Output residual 2x2 or 4x4 DC transform 2x2 or 4x4 DC transform Steps of Implementation Encoding: Figure 3.5: Complete process of TQIQIT Step1. Input: 4X4 residual samples X Step2. Forward Core transformation: = C f X C T f E f Step3. Post Scaling and Quantization: Decoding: PF Z = W. Qstep.2 Step4. Re-scaling (Incorporating inverse transform pre-scaling): W ij= Z ij x Qstep x PF x 64 ' T ' Step5. Inverse core transform: X = C W C ' '' X Step6. Post scaling X = round 64 '' Output 4X4 residual sample X i i qbits 51

11 3.3 Context-Based Adaptive Variable Length Coding (CAVLC) This is the method used to encode residual, zig-zag ordered 4x4 (and 2x2) blocks of transform coefficients. CAVLC is designed to take advantage of several characteristics of quantized 4x4 blocks: 1. After prediction, transformation and quantisation blocks are typically sparse (containing mostly zeros). CAVLC uses run-level coding to represent strings of zeros compactly. 2. The highest nonzero coefficients after the zig-zag scan are often sequences of ±1 and CAVLC signals the number of high-frequency ±1 coefficients ( Trailing Ones ) in a compact way. 3. The number of nonzero coefficients in neighbouring blocks is correlated. The number of coefficients is encoded using a look-up table and the choice of look-up table depends on the number of nonzero coefficients in neighbouring blocks. 4. The level (magnitude) of nonzero coefficients tends to be larger at the start of the reordered array (near the DC coefficient) and smaller towards the higher frequencies. CAVLC takes advantage of this by adapting the choice of VLC lookup table for the level parameter depending on recently-coded level magnitudes. This section presents an encoding example Consider 4 4 sublock Reordered block: 0, 3, 0, 1, -1, -1, 0, 1, 0, 0 TotalCoeffs = 5 (indexed from highest frequency, 4, to lowest frequency, 0) total zeros = 3 TrailingOnes = 3 (in fact there are four trailing ones but only three can be encoded as a special case ) 52

12 Encoding: Element Value Code coeff token TotalCoeffs = 5 TrailingOnes= 3 (from Table 1, Table 1 is given in annexture) TrailingOne sign (4) + 0 TrailingOne sign (3) - 1 TrailingOne sign (2) - 1 Level (1) +1 (use suffixlength = 0) 1 (prefix) Level (0) +3 (use suffixlength = 1) 001 (prefix) 0 (suffix) total zeros run before(4) ZerosLeft = 3; run before 10 =1 run before(3) run before(2) run before(1) ZerosLeft = 2; run before =0 ZerosLeft = 2; run before =0 ZerosLeft = 2; run before =1 run before(0) ZerosLeft = 1; run before = No code required; last coefficient. The transmitted bitstream for this block is

13 References [1] ITU-T Recommendation H.264 and ISO/IEC (MPEG-4) AVC, Advanced Video Coding for Generic Audiovisual Services, Version 3: [2] Advanced Video coding for generic audio-visual services,iso/iec [3] ITU-T (1993), Video Codec for Audio visual Services at px 64 Kbits, ITU- T Recommendation H.261, Version 2. [4] ITU-T (2000), Video coding for low bit rate communication, ITU-T Recommendation H.263, Version 1, Nov. 1995: Version 3. [5] MPEG-4 Overview, ISO/IEC JTC 1/SC29/WG11 N4668. [6] MPEG website: [7] JVT website: ftp://standards.polycom.com [8] Sadiqullah Khan and Gulistan Raja. (2004), Integer cosine transform and Its application in Image/Video Compression, ICSEA, Conference proceeding Islamabad, pp [9] Liu Ling-zhi, Qiu Lin, Rong Meng-tian and Jiang Li. (2004), A 2-D forward/inverse integer transform processor of h.264 based on highly parallel architecture, proceedings of the 4 th IEEE International workshop on System-on-Chip for Real-time applications. [10] Qiang Peng and Jin Jing.(2003), H.264 system on chip design and verification, The IEEE workshop on Signal Processing systems. [11] Iain E. G. Richardson, H.264 and Video Compression, John Wiley And Sons, [12] Lu u, Sijia Chen and Jianpeng Wang. (2009), Overview of AVS video coding standards, Elsevier, Signal Processing: Image Communication 24, pp [13] Thomas Wiegand and Gary J. Sullivan, (2003), Overview of the H.264/AVC Video Coding Standard, IEEE Transactions on Circuits and Systems for Video Technology, pp [14] LeGall D. (1991), MPEG: A video compression standard for multimedia application, Communication, ACM, Vol.34, pp [15] Chan ul Park and Nam Ik Cho (2005), A fast algorithm for the conversion of DCT coefficients to H.264 transform coefficients, IEEE conference. [16] Xin J., Verto A. and H. Sun. (2004), Converting DCT coefficients to H.264/AVC Transform Coefficients, Technical Report of Mitsubishi Electric Research Lab. 54

14 [17] Cham W. K. (1989), Development of integer cosine transforms by the Principle of dyadic symmetry, Proceedings, Vol.136, No.4. [18] Cheung K.M Pllara F. and Shahshahani M (1991), Integer Cosine Transform For image compression, the Telecommunications and data Acquisition Progress Report, jet Propulsion laboratory, Pasadena, California, pp [19] Ci-Xun Zhang, Jian Lou, Lu u, Jie Dong, Wai-Kuen Cham. (2005), The technique of pre-scaled integer transform, IEEE International Symposium on Circuits and Systems. [20] Cixun Zhang, Lu u, Jian Lou, Wai-kuen Cham and Jie Dong. (2008), The technique of pre scaled integer transform: concept, design and applications, IEEE Transactions on Circuits and Systems for Video Technology 18,pp [21] Malvar H.S., Hallapuro A. and Kerofsky L. (2003), Low Complexity Transform and Quantization in H.264/AVC, IEEE Transactions on Circuits and Systems for Video Technology, Vol.13, No.7, pp [22] Karewicz M. (2004), Transform and Quantization in H.264/AVC, IEEE Transactions on Circuits and Systems for Video Technology, vol.11, No. 3, pp [23] Cham W.K (1989), Development of integer cosine transforms by the principle of dyadic symmetry, Proceedings, IEEE,pp [24] TuChih Wang. (2003), Parallel 4 4 2D Transform and inverse Transform architecture for MPEG-4 AVC/H.264, Proceeding of IEEE International Symposium on Circuits and Systems, Bangkok, Thailand, pp [25] eong-kang Lai, Chih-Chung Chou and Tu-Chieh Hung. (2005), A Simple and Cost Effective Video Encoder with Memory-Reducing CAVLC, pp IEEE. [26] ong Ho Moon. (2008), An Advanced Total Zeros Decoding Method Based on New Memory Architecture in H.264/AVC CAVLC, IEEE Transactions on Circuits and Systems for Video Technology, Vol.18, No. 9. [27] Li Zhang, Qiang Wang, Ning Zhang, Debin Zhao, Xialin Wu and Wen Gao. (2009), Context-based entropy coding in AVS Video Coding standard, Signal Procession: Image Communication 24, pp [28] Richardson I.E. G.(2002), White Paper H.264/MPEG-4 Part 10: Variable Length Coding. [29] Li Zhang, Xiaolin Wu, Ning Zhang, et.al. (2007) Context-based Arithmetic Coding Reexamined for DCT Video Compression, IEEE International Symposium on Circuits and Systems (ISCAS). [30] Dong J., Lou j. and u L. (2003), Improved entropy coding method, Doc. AVS Working Group (M1214), Beijing, Chaina. 55

15 56

4x4 Transform and Quantization in H.264/AVC

Video compression design, analysis, consulting and research White Paper: 4x4 Transform and Quantization in H.264/AVC Iain Richardson / VCodex Limited Version 1.2 Revised November 2010 H.264 Transform and