Detailed Review of H.264/AVC, Ph.D.. abuhajar@digitavid.net (408) 506-2776 P.O. BOX:720998 San Jose, CA 95172 1
Outline Common Terminologies Color Space Macroblock and Slice Type Slice Block Diagram Intra-Prediction Integer Transform Quantization 2
Common Terminologies Frame is a picture Frame may be divided into Two Fields: Top Field is the Even Field Bottom Field is the Odd Field Progressive Frame: The two fields are captured at the same time (30 frames/ sec) Interlaced Frame: The two fields are captured at two separate times ( 60 frames/sec) 3
Color Space H.264/AVC adopts YCrCb color space contains three components Y is the luma component Cr is a chroma component Cb is a chroma component Most common resolution formats Monochrome 4:2:0 Chroma components have half height and half the width (same as MPEG2 and Analog TV) 4:2:2 chroma components have same height and half the width 4:4:4 full resolution 4
Macroblocks and Slices Each frame is divided into Mamcroblocks 16X16 for luma 8X8 for chroma Frame is divided into SLICES Slice is a sequence of macroblocks Raster scan Flexible Macroblock ordering FMO 5
Types of Slices Types of slice modes I-Slice: All of its macroblocks use Intra coding P-Slice: Using Inter Prediction coding (motion estimation) in one direction B-Slice: Using Binary direction Inter Prediction coding (motion estimation) SP-Slice: Switch to P-Slice SI-Slice: Switch to I-Slice The encoder makes decision the segmentation of slices and the slice mode selection 6
Encoder Block Diagram Block Diagram Current Block _ Intra Prediction Transform Quantize Coefficients Intra Coding Entropy Coding Compressed Data Motion Compensation Inter Coding Inverse Quantization Current Block Motion Estimation Algorithm De-blocking Filter & Inverse Prediction Inverse Transform 7
Intra Prediction Lumina Prediction I_PCM : Skips prediction Intra_16X16 : Applied to 16X16 macroblocks Selects one of four types Intra_4X4 : Applied to 4X4 blocks Selects one of nine types Chroma Prediction Selects one of four types 8
Intra_16X16 Previously coded pixels are known at the coder and decoder Mode 0: Vertical p( x, y) = I( x, 1); x, y 0,1,...,15 The predicted pixel I ( x, y) = I( x, y) p( x, y) = p( x, 1); x 0,1,...15 and y 0,1,...,15 Previously coded Z X 0 X 1 X 14 X 15 Previously coded Y 0 Y 1 Y 2 Y 14 Y 15 16 16 Macroblock 9
Intra_16X16 Mode 1: Horizontal Previously coded p( x, y) = I( 1, y); x, y 0,1,...,15 Mode 2: DC All neighboring pixels exists 15 15 p( x, y) = I ( x, 1) + I( 1, y) + 16 / 32 x= 0 y= 0 Only Ys exists 15 p( x, y) = I ( x, 1) + 8 /16 x= 0 Previously coded Z X 0 X 1 X 14 X 15 Y 0 Y 1 Y 2 16 16 Macroblock Y 14 Y 15 10 Only Xs exist 15 p( x, y) = I ( 1, y) + 8 /16 y= 0
Intra_16X16 Mode 3: Intra_16X6_Plane ([ ] ) p( x, y) = Clip1 a + b( x 7) + c( y 7) + 16 / 32 ( ) a = 16 X + Y 15 15 ( H ) b = 5 + 32 / 64 ( V ) c = 5 + 32 / 64 7 k= 0 ( ) H = ( k + 1) I (8 + k, 1) I (6 k, 1) Previously coded Z X 0 X 1 X 14 X 15 Previously coded Y 0 Y 1 Y 2 16 16 Macroblock Y 14 Y 15 11 7 k= 0 ( ) V = ( k + 1) I ( 1,8 + k) I ( 1,6 k)
Intra 4X4 Prediction Nine prediction modes Mode 0: Vertical Mode 1: Horizontal Mode 2: DC Mode 3: Diagonal Down Left Mode 4: Diagonal Down Right Mode 5: Vertical Right Mode 6: Horizontal Down Mode 7: Vertical Left Mode 8: Horizontal Up Previously coded Previously Coded Z X 0 X 1 X 2 X 3 Y 0 Y 1 Y 2 Y 3 4 4 Block 12
Intra 4X4 Prediction Nine prediction modes Mode 0: Vertical p( x, y) = I( x, 1); x, y 0,1,2,3 Mode 1: Horizontal p( x, y) = I( 1, y); x, y 0,1,2,3 Mode 2: DC Previously coded Previously Coded Z X 0 X 1 X 2 X 3 Y 0 Y 1 Y 2 4 4 Block Y 3 13 3 3 p( x, y) = I( x, 1) + I ( 1, y) + 4 /8 x= 0 y= 0 Refer to section 8.3 of the standard for remaining predictions
Intra 8X8 Chroma Prediction 14 Four prediction modes Mode 0: DC Mode 1: Vertical Mode 2: Horizontal Mode 3: Chroma_Plan prediction Very similar to the 16X16 Lumina prediction. Refer to section 8.3 of the standard for the equations Previously coded Previously Coded Z X 0 X 7 Y 0 Y 7 8 8 Block
Decision is of the best prediction Current Block Prediction Mode 1 Prediction Mode 2 Select Best Prediction Predicted Current Block Prediction Mode N 15
Integer Transform It requires two steps Step One: Applied to 4x4 blocks c00 c01 c02 c03 1 1 1 1 p00 p01 p02 p03 1 2 1 1 c10 c11 c12 c 13 2 1 1 2 p10 p11 p12 p 13 1 1 1 2 = c20 c21 c22 c 23 1 1 1 1 p20 p21 p22 p 23 1 1 1 2 c c c c 1 2 2 1 p p p p 1 2 1 1 30 31 32 33 30 31 32 33 4x4 Coefficients 4x4 Pixels 16
Integer Transform Step Two: Applied to DC Coefficients Intra_16X16 Lumina block Apply Hadamard transform the DC coefficient of each of it 16 4X4 block Intra_8X8 Chroma block Apply 2x2 Hadamard transform to the DC block 16X16 Macroblock DC Coeff. Only DC-Block 4X4 8X8 Macroblock DC-Block 2X2 17 DC Coeff. Lumina 4X4 Block DC Coeff. 4X4 Block Chroma
Integer Transform 4x4 Hadamard Transform ( DC Lumina Coeff) c00 c01 c02 c03 1 1 1 1 p00 p01 p02 p03 1 1 1 1 c10 c11 c12 c 13 1 1 1 1 p10 p11 p12 p 13 1 1 1 1 = c20 c21 c22 c 23 1 1 1 1 p20 p21 p22 p 23 1 1 1 1 c c c c 1 1 1 1 p p p p 1 1 1 1 30 31 32 33 30 31 32 33 4x4 Coefficients 4x4 DC- Coeff 2x2 Hadamard Transform ( DC Lumina Coeff) c00 c 01 1 1 p00 p 01 1 1 c c = 1 1 p p 1 1 10 11 10 11 18 2x2 Coefficients 2x2 DC- Coeff
Inverse Integer Transform Requires two steps Step One: applied to the DC coefficients Step TWO: Applied to 4x4 blocks c00 c01 c02 c03 1 1 1 1/ 2 p00 p01 p02 p03 1 1 1 1 c10 c11 c12 c 13 1 1/ 2 1 1 p10 p11 p12 p 13 1 1/ 2 1 1 = c20 c21 c22 c 23 1 1/ 2 1 1 p20 p21 p22 p 23 1 1 1 1 c c c c 1 1 1 1/2 p p p p 1/2 1 1 1/2 30 31 32 33 30 31 32 33 4x4 Coefficients 4x4 Pixels 19
Quantization The Quantization is given by the following: Where X quantized { X i j } = sgn (, ) X ( i, j) + f ( Q ) Xq(I,j) F(Qs) controls the quantization near zero Qs is the quantization step The division uses integer arithmetic (drops the remaining) The re-construction of quantized coefficients Q s s X(I,j) X ( i, j) = Q X ( i, j) rec s quantized 20
Quantization The standard avoided the division by zeros by { } ( ) X sgn (, ) (, ) 2 L quantized = X i j X i j A Qs + f >> L X = X ( i, j) B( Q) reconst quant A Q B Q G 2 ( ) ( ) 2 L + N Where L=N=20 21
Summary Common Terminologies Color Space Macroblock and Slice Type Slice Block Diagram Intra-Prediction Integer Transform Quantization 22