Detailed Review of H.264/AVC

Size: px

Start display at page:

Download "Detailed Review of H.264/AVC"

Iris Gardner
5 years ago
Views:

1 Detailed Review of H.264/AVC, Ph.D.. (408) P.O. BOX: San Jose, CA

2 Outline Common Terminologies Color Space Macroblock and Slice Type Slice Block Diagram Intra-Prediction Integer Transform Quantization 2

3 Common Terminologies Frame is a picture Frame may be divided into Two Fields: Top Field is the Even Field Bottom Field is the Odd Field Progressive Frame: The two fields are captured at the same time (30 frames/ sec) Interlaced Frame: The two fields are captured at two separate times ( 60 frames/sec) 3

4 Color Space H.264/AVC adopts YCrCb color space contains three components Y is the luma component Cr is a chroma component Cb is a chroma component Most common resolution formats Monochrome 4:2:0 Chroma components have half height and half the width (same as MPEG2 and Analog TV) 4:2:2 chroma components have same height and half the width 4:4:4 full resolution 4

5 Macroblocks and Slices Each frame is divided into Mamcroblocks 16X16 for luma 8X8 for chroma Frame is divided into SLICES Slice is a sequence of macroblocks Raster scan Flexible Macroblock ordering FMO 5

6 Types of Slices Types of slice modes I-Slice: All of its macroblocks use Intra coding P-Slice: Using Inter Prediction coding (motion estimation) in one direction B-Slice: Using Binary direction Inter Prediction coding (motion estimation) SP-Slice: Switch to P-Slice SI-Slice: Switch to I-Slice The encoder makes decision the segmentation of slices and the slice mode selection 6

Encoder Block Diagram Block Diagram Current Block _ Intra Prediction Transform Quantize Coefficients Intra Coding Entropy Coding Compressed Data

7 Encoder Block Diagram Block Diagram Current Block _ Intra Prediction Transform Quantize Coefficients Intra Coding Entropy Coding Compressed Data Motion Compensation Inter Coding Inverse Quantization Current Block Motion Estimation Algorithm De-blocking Filter & Inverse Prediction Inverse Transform 7

8 Intra Prediction Lumina Prediction I_PCM : Skips prediction Intra_16X16 : Applied to 16X16 macroblocks Selects one of four types Intra_4X4 : Applied to 4X4 blocks Selects one of nine types Chroma Prediction Selects one of four types 8

9 Intra_16X16 Previously coded pixels are known at the coder and decoder Mode 0: Vertical p( x, y) = I( x, 1); x, y 0,1,...,15 The predicted pixel I ( x, y) = I( x, y) p( x, y) = p( x, 1); x 0,1,...15 and y 0,1,...,15 Previously coded Z X 0 X 1 X 14 X 15 Previously coded Y 0 Y 1 Y 2 Y 14 Y Macroblock 9

10 Intra_16X16 Mode 1: Horizontal Previously coded p( x, y) = I( 1, y); x, y 0,1,...,15 Mode 2: DC All neighboring pixels exists p( x, y) = I ( x, 1) + I( 1, y) + 16 / 32 x= 0 y= 0 Only Ys exists 15 p( x, y) = I ( x, 1) + 8 /16 x= 0 Previously coded Z X 0 X 1 X 14 X 15 Y 0 Y 1 Y Macroblock Y 14 Y Only Xs exist 15 p( x, y) = I ( 1, y) + 8 /16 y= 0

11 Intra_16X16 Mode 3: Intra_16X6_Plane ([ ] ) p( x, y) = Clip1 a + b( x 7) + c( y 7) + 16 / 32 ( ) a = 16 X + Y ( H ) b = / 64 ( V ) c = / 64 7 k= 0 ( ) H = ( k + 1) I (8 + k, 1) I (6 k, 1) Previously coded Z X 0 X 1 X 14 X 15 Previously coded Y 0 Y 1 Y Macroblock Y 14 Y k= 0 ( ) V = ( k + 1) I ( 1,8 + k) I ( 1,6 k)

12 Intra 4X4 Prediction Nine prediction modes Mode 0: Vertical Mode 1: Horizontal Mode 2: DC Mode 3: Diagonal Down Left Mode 4: Diagonal Down Right Mode 5: Vertical Right Mode 6: Horizontal Down Mode 7: Vertical Left Mode 8: Horizontal Up Previously coded Previously Coded Z X 0 X 1 X 2 X 3 Y 0 Y 1 Y 2 Y Block 12

13 Intra 4X4 Prediction Nine prediction modes Mode 0: Vertical p( x, y) = I( x, 1); x, y 0,1,2,3 Mode 1: Horizontal p( x, y) = I( 1, y); x, y 0,1,2,3 Mode 2: DC Previously coded Previously Coded Z X 0 X 1 X 2 X 3 Y 0 Y 1 Y Block Y p( x, y) = I( x, 1) + I ( 1, y) + 4 /8 x= 0 y= 0 Refer to section 8.3 of the standard for remaining predictions

14 Intra 8X8 Chroma Prediction 14 Four prediction modes Mode 0: DC Mode 1: Vertical Mode 2: Horizontal Mode 3: Chroma_Plan prediction Very similar to the 16X16 Lumina prediction. Refer to section 8.3 of the standard for the equations Previously coded Previously Coded Z X 0 X 7 Y 0 Y Block

15 Decision is of the best prediction Current Block Prediction Mode 1 Prediction Mode 2 Select Best Prediction Predicted Current Block Prediction Mode N 15

16 Integer Transform It requires two steps Step One: Applied to 4x4 blocks c00 c01 c02 c p00 p01 p02 p c10 c11 c12 c p10 p11 p12 p = c20 c21 c22 c p20 p21 p22 p c c c c p p p p x4 Coefficients 4x4 Pixels 16

17 Integer Transform Step Two: Applied to DC Coefficients Intra_16X16 Lumina block Apply Hadamard transform the DC coefficient of each of it 16 4X4 block Intra_8X8 Chroma block Apply 2x2 Hadamard transform to the DC block 16X16 Macroblock DC Coeff. Only DC-Block 4X4 8X8 Macroblock DC-Block 2X2 17 DC Coeff. Lumina 4X4 Block DC Coeff. 4X4 Block Chroma

18 Integer Transform 4x4 Hadamard Transform ( DC Lumina Coeff) c00 c01 c02 c p00 p01 p02 p c10 c11 c12 c p10 p11 p12 p = c20 c21 c22 c p20 p21 p22 p c c c c p p p p x4 Coefficients 4x4 DC- Coeff 2x2 Hadamard Transform ( DC Lumina Coeff) c00 c p00 p c c = 1 1 p p x2 Coefficients 2x2 DC- Coeff

19 Inverse Integer Transform Requires two steps Step One: applied to the DC coefficients Step TWO: Applied to 4x4 blocks c00 c01 c02 c / 2 p00 p01 p02 p c10 c11 c12 c / p10 p11 p12 p / = c20 c21 c22 c / p20 p21 p22 p c c c c /2 p p p p 1/ / x4 Coefficients 4x4 Pixels 19

20 Quantization The Quantization is given by the following: Where X quantized { X i j } = sgn (, ) X ( i, j) + f ( Q ) Xq(I,j) F(Qs) controls the quantization near zero Qs is the quantization step The division uses integer arithmetic (drops the remaining) The re-construction of quantized coefficients Q s s X(I,j) X ( i, j) = Q X ( i, j) rec s quantized 20

21 Quantization The standard avoided the division by zeros by { } ( ) X sgn (, ) (, ) 2 L quantized = X i j X i j A Qs + f >> L X = X ( i, j) B( Q) reconst quant A Q B Q G 2 ( ) ( ) 2 L + N Where L=N=20 21

22 Summary Common Terminologies Color Space Macroblock and Slice Type Slice Block Diagram Intra-Prediction Integer Transform Quantization 22

h 8x8 chroma a b c d Boundary filtering: 16x16 luma H.264 / MPEG-4 Part 10 : Intra Prediction H.264 / MPEG-4 Part 10 White Paper Reconstruction Filter

h 8x8 chroma a b c d Boundary filtering: 16x16 luma H.264 / MPEG-4 Part 10 : Intra Prediction H.264 / MPEG-4 Part 10 White Paper Reconstruction Filter H.264 / MPEG-4 Part 10 White Paper Reconstruction Filter 1. Introduction The Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG are finalising a new standard for the coding (compression) of natural