H.264 (MPEG-4 Part 10)

Size: px

Start display at page:

Download "H.264 (MPEG-4 Part 10)"

Cecily Cummings
6 years ago
Views:

1 H.64 (MPEG-4 Prt 0) Encoded bitstrem Fig. 4.4 H.64 encoder. ELEG550 Video Coding Technology

2 H.64 (MPEG-4 Prt 0) Encoded bitstrem Fig. 4.5 H.64 decoder. ELEG550 Video Coding Technology

3 H.64 (MPEG-4 Prt 0) Coding is bsed on mcroblock structure consisting of 6 6 Y block, 8 8 Cr nd Cb blocks. A slice is group of mcroblocks etc. ELEG550 Video Coding Technology 3

H.64 Coding Tools Intr prediction: 3 modes for Y components (9 for INTRA_4 4 blocks nd 4 for INTRA_6 6 blocks) 4 modes for Cr & Cb components Cn use decoded mcroblocks of the sme imge for prediction

4 H.64 Coding Tools Intr prediction: 3 modes for Y components (9 for INTRA_4 4 blocks nd 4 for INTRA_6 6 blocks) 4 modes for Cr & Cb components Cn use decoded mcroblocks of the sme imge for prediction Motion compensted prediction (MCP): Block sizes of 6 6, 6 8, 8 6 & 8 8 Sub-block sizes of 8 8 cn be further divided into 8 4, 4 8 & 4 4 Multiple reference frmes (MRF) (up to 6 frmes) ¼ pixel motion estimtion ELEG550 Video Coding Technology 4

5 H.64 Coding Tools Prediction mode is chosen by minimizing the rtedistortion function using Lgrngin optimiztion techniques (RDO). 4 4 integer DCT, nd 4 4 or HT Entropy coding: Context-bsed dptive vrible length codes (CAVLC) Context-bsed dptive binry rithmetic coding (CABAC) Adptive deblocking filter ELEG550 Video Coding Technology 5

6 Intr Prediction 0 (verticl) (horizontl) (DC) M I J K L A B C D E F G H M I J K L A B C D E F G H M I J K L A B C D E F G H Men (A, D, I, L) 3 (digonl down-left) 4 (digonl down-right) M I J K A B C D E F G H M I J K A B C D E F G H L L Fig lum prediction modes. ELEG550 Video Coding Technology 6

7 Intr Prediction 5 (verticl-right) 6 (horizontl-down) M I J K A B C D E F G H M I J K A B C D E F G H L L 7 (verticl-left) 8 (horizontl-up) M I J K A B C D E F G H M I J K A B C D E F G H L L Fig lum prediction modes. ELEG550 Video Coding Technology 7

8 Intr Prediction 0 (verticl) (horizontl) (DC) 3 (plne) H H H H V... V... V Men (H+V) V Fig lum prediction modes. ELEG550 Video Coding Technology 8

9 Inter Prediction Fig. 4.8 Mcroblock prtitions: 6 6, 8 6, 6 8, Fig. 4.9 Sub-mcroblock prtitions: 8 8, 4 8, 8 4, 4 4. ELEG550 Video Coding Technology 9

10 Inter Prediction A seprte motion vector (MV) is required for ech mcroblock or sub-mcroblock prtition. Choosing lrge prtition (6 6, 6 8, 8 6) results in smll number of bits for encoding the MVs nd the prtition type, but the MC residul my contin lrge mount of energy. Conversely, smll prtition (8 8, 8 4, 4 8) contins lower MC residul energy but my require more bits to encode the MVs nd the prtition type. In generl, lrge prtition size is suitble for homogeneous regions, wheres smll prtition is beneficil in detiled res. The chrom component (Cb nd Cr) is prtitioned in the sme wy but with hlf the sptil resolution. ELEG550 Video Coding Technology 0

11 Motion Vector Prediction B C A E Fig Current nd neighboring sme-sized prtitions. ELEG550 Video Coding Technology

12 Motion Vector Prediction B 4 8 C 6 8 A 8 8 E 6 6 Fig Current nd neighboring different-sized prtitions. ELEG550 Video Coding Technology

13 Motion Vector Prediction The predicted motion vector MVP for prtition E is generted s follows:. For trnsmitted prtitions excluding 6 8 nd 8 6 prtition sizes, MVP is the medin of the MVs for prtitions A, B nd C.. For 6 8 prtitions, MVP for the upper 6 8 prtition is predicted from B nd MVP for the lower 6 8 prtition is predicted from A. 3. For 8 6 prtitions, MVP for the left 8 6 prtition is predicted from A nd MVP for the right 8 6 prtition is predicted from C. 4. For skipped mcroblocks, 6 6 vector MVP is generted s in cse () bove. 5. In the decoder, MVP is formed in the sme wy nd dded to the decoded vector difference MVD. ELEG550 Video Coding Technology 3

14 Sub-pixel Motion Estimtion Interpoltion for Lum Resolution A B Qurter-pixel ff hh Filters Hlf-pixel Blue: [/3, -5/3, 5/8, 5/8, -5/3, /3] C bb G ee gg D d h n H b c E e f g i j k m p q r s I F cc J Qurter-pixel K dd L White: biliner (horizontl or verticl) Red: biliner (digonl) ELEG550 Video Coding Technology 4

15 Sub-pixel Motion Estimtion Interpoltion for Chrom A B dy dx E 8-dx 8-dy C D Biliner E = [(8 dx) (8 dy) A + dx (8 dy) B + (8 dx) dy C + dx dy D] /64 ELEG550 Video Coding Technology 5

16 Deblocking Filter h0 h h h3 h4 h5 v0 v v v3 6 6 lum v4 v5 8 8 chrom Fig. 4.3 Edge filtering order in mcroblock. Verticl boundry p3 p p p3 p p p0 q0 q q q3 p0 q0 Horizontl boundry q q q3 Fig. 4.3 Smples djcent to verticl nd horizontl boundries. ELEG550 Video Coding Technology 6

17 Deblocking Filter A deblocking filter is pplied to ech decoded mcroblock (MB) to reduce the blocking rtifcts. It is pplied fter the inverse trnsform before reconstruction in both in the encoder nd decoder (see Figs. 4.4 & 4.5). The filtered imge is used for motion compenstion in the encoder nd disply in the decoder. It smooths the block edges, nd therefore improves the coding efficiency in the encoder, nd enhnces the ppernce of the decoded frmes in the disply. ELEG550 Video Coding Technology 7

18 Deblocking Filter Filtering is pplied to the verticl nd horizontl edges of 4 4 blocks in MB (except for edges on slice boundries) in the following order: 4 verticl boundries of lum component (in order of v0, v, v, v3) 4 horizontl boundries of lum component (in order of h0, h, h, h3) verticl boundries of chrom component (in order of v4, v5) horizontl boundries of chrom component (in order of h4, h5) Three smples on either side of the boundry (p0, p, p nd q0, q, q) re involved in the filtering opertion. The mount of filtering depends on the current quntizer, the coding modes of neighboring blocks, nd the grdient of imge smples cross the boundry. ELEG550 Video Coding Technology 8

19 Deblocking Filter Boundry Strength The filtering outcome depends on the boundry strength (BS) nd the grdient of imge smples cross the boundry. The filter is stronger t plces where there is likely to hve higher blocking distortions, such s n intr coded MB boundry, or boundry between blocks with coded coefficients. BS=4 BS=3 BS= BS= BS=0 (no filtering) p nd/or q is intr coded nd boundry is MB boundry p nd q re intr coded nd boundry is not MB boundry neither p or q is intr coded; p nd q contin coded coefficients neither p or q is intr coded; neither p nd q contin coded coefficients; p nd q use different reference pictures or different number of reference pictures or hve MV vlues otherwise ELEG550 Video Coding Technology 9

Deblocking Filter Filter Decision The group of smples (p0, p, p nd q0, q, q) re filtered if the following conditions re stisfied: () BS > 0 nd (b) p0q0 < α nd pp0 < β nd qq0 < β α nd β re thresholds

20 Deblocking Filter Filter Decision The group of smples (p0, p, p nd q0, q, q) re filtered if the following conditions re stisfied: () BS > 0 nd (b) p0q0 < α nd pp0 < β nd qq0 < β α nd β re thresholds defined in the stndrds which increse with the qunitzer prmeter QP of the two blocks p nd q. When QP is smll (with corresponding smll α nd β), lrge grdient cross the block boundry is likely due to nturl fetures in the originl imge rther thn blocking rtifcts, nd should be preserved. On the other hnd, lrge QP results in blocky boundry; higher α nd β vlues will cuse more boundry smples to be filtered. Filtering is performed by 3-tp, 4-tp or 5-tp filters s defined in the stndrd. ELEG550 Video Coding Technology 0

21 Trnsformtion H.64 uses three trnsforms depending on the type of residul dt tht is to be coded: Hdmrd trnsform for 4 4 rry of lum DC coefficients predicted in 6 6 mode; Hdmrd trnsform for rry of chrom DC coefficients; integer DCT for ll other 4 4 blocks of residul dt. Dt within MB re trnsmitted in the order shown in Fig ELEG550 Video Coding Technology

22 Trnsformtion Fig Scnning order of residul blocks within MB. ELEG550 Video Coding Technology

23 Integer DCT The integer DCT opertes on 4 4 blocks of residul dt (lbeled 0-5 nd 8-5 in Fig. 4.33) fter motion-compensted prediction or Intr prediction. It is possible to ensure zero mismtch between encoder nd decoder inverse trnsforms. The core prt of the trnsform cn be implemented using only dditions nd shifts. A scling-multipliction is integrted into the quntizer, reducing the totl number of multiplictions. The inverse quntiztion (scling) nd inverse trnsform opertions cn be crried out using 6-bit integer rithmetic with only single multiply per coefficient, without ny loss of ccurcy. ELEG550 Video Coding Technology 3

24 4 ELEG550 Video Coding Technology Integer DCT The 4 4 DCT is given by where = = c b b c b c c b c b b c b c c b X AXA Y T. 8 3 cos, 8 cos, = = = π π c b

25 5 ELEG550 Video Coding Technology Integer DCT The mtrix multipliction cn be fctorized to the following equivlent form: CXC T is the core D trnsform, E is mtrix of scling fctors, nd the symbol indictes tht ech element of (CXC T ) is multiplied by the scling fctor in E t the corresponding position (sclr multipliction rther thn mtrix multipliction). The constnts nd b re s before nd d is c/b ( 0.44). = = T ) ( b b b b b b b b b b b b d d d d d d d d X E CXC Y

26 Integer DCT To simplify the implementtion of the trnsform, nd to ensure tht trnsform remins orthogonl, the constnts re modified s follows: =, b =, d 5 =. The nd nd 4 th rows of mtrix C nd the nd nd 4 th columns of mtrix C T re scled by nd the mtrix E is post-scled down to compenste, to void multiplictions by / in the core trnsform CXC T. ELEG550 Video Coding Technology 6

27 7 ELEG550 Video Coding Technology Integer DCT The finl forwrd trnsform becomes This trnsform is n pproximtion to the 4 4 DCT, but becuse of the pproximtion to fctor d, the result will not be identicl to the 4 4 DCT. = = ) ( T b b b b b b b b b b b b f f f X E XC C Y

28 8 ELEG550 Video Coding Technology Integer DCT The inverse trnsform is defined s Note tht Y is pre-scled by multiplying ech coefficient by the pproprite weighting fctor from mtrix E. The forwrd nd inverse trnsform re orthogonl, i.e., = = T b b b b b b b b b b b b i i i Y E )C (Y C X. ) ( T X TX T =

29 Quntiztion Forwrd Quntiztion The unscled coefficient W ij is quntized nd scled s follows: Z ij = round W ij PF Q step where PF depends on the position (i, j) of the coefficient, Position (0,0), (,0), (0,) or (,) (,), (,3), (3,) or (3,3) others PF b /4 b/ ELEG550 Video Coding Technology 9

30 Quntiztion nd Q step is determined ccording to the tble below. There re 5 quntizer step sizes nd Q step doubles for every increment of 6 in QP. Tble 4.5 Quntiztion step sizes in H.64 codec. QP Q step QP Q step QP Q step QP Q step ELEG550 Video Coding Technology 30

31 Quntiztion To simplify the computtion, the fctor ( PF Q step ) is implemented s multipliction by fctor MF nd right-shift, voiding ny division opertion: Z ij W MF = round ij qbits where MF qbits = PF Q step nd qbits = 5 + floor ( QP / 6) ELEG550 Video Coding Technology 3

32 Quntiztion The fctor MF cn be computed for different vlues of QP nd PF: QP Positions (0,0), (,0), (0,), (,) Positions (,), (,3), (3,), (3,3) Other positions Exercise: Given QP = 4 nd (i, j) = (0, 0), clculte MF. ELEG550 Video Coding Technology 3

33 Quntiztion Inverse Quntiztion The scling fctor PF for inverse trnsform is incorported in the inverse quntiztion opertion, together with constnt scling fctor or 64 to void rounding error. Hence, the inverse quntized coefficient is given by W = Z Q PF ij ij step 64 where PF depends on the position (i, j) of the coefficient. ELEG550 Video Coding Technology 33

34 Quntiztion Position (0,0), (,0), (0,) or (,) (,), (,3), (3,) or (3,3) others PF b b W ij is scled coefficient which is trnsformed by the core inverse T trnsform C i WC. i The output of the inverse trnsform re divided by 64 to remove the scling fctor (this cn be implemented using only n ddition nd right-shift). ELEG550 Video Coding Technology 34

35 Quntiztion The stndrd does not specify PF nd Q step directly, but define prmeter V ij for ech coefficient position: V ij = Q ( QP 6R) PF 64 step where Q step (m) is the Q step vlue when QP = m from Tble 4.5, nd R = floor(qp/6). The scling opertion thus becomes: W = Z V ij ij ij R The fctor R hs the effect of incresing the scled output by fctor of for every increment of 6 in QP. ELEG550 Video Coding Technology 35

36 Quntiztion R The vlues of the scling fctor Vij for 0 QP 5 re given in the tble below: QP Positions (0,0), (,0), (0,), (,) Positions (,), (,3), (3,), (3,3) Other positions floor( QP / 6) Exercise: Given QP = 9 nd (i, j) = (, ), find V. ij ELEG550 Video Coding Technology 36

37 Entropy Coding Above the slice lyer, the syntx elements re encoded s fixed- or vrible-length codes (VLCs). At the slice lyer nd below, VLCs or context-dptive rithmetic coding (CABAC) re used to encode the elements depending on the entropy encoding mode. [CABAC is similr to the CAE in MPEG-4.] When entropy_encoding_mode = 0, the residul block dt is coded using context-dptive vrible length coding (CAVLC) scheme nd the other vrible-length coded units re coded using Exp-Golomb codes. ELEG550 Video Coding Technology 37

38 Exp-Golomb Entropy Coding Exp-Golomb codes (Exponentil Golomb codes) re constructed in logicl wy: [M zeroes][][info] where INFO is n M-bit field crrying informtion. The vlue of M is function of the index code_num: M = floor(log [code_num + ]) nd INFO = code_num + M The length of ech Exp-Golomb codeword is (M+) bits. ELEG550 Video Coding Technology 38

39 Exp-Golomb Entropy Coding A Exp-Golomb code tble is shown below: code_num Codeword ELEG550 Video Coding Technology 39

Exp-Golomb Entropy Coding A codeword cn be decoded s follows:. Red in M leding zeroes followed by.

) A prmeter k to be encoded is mpped to code_num so s to produce short codewords for

For exmple, coded_block_pttern = 0 (no non-zero blocks) is ssigned code_num = 0 becuse it occurs

40 Exp-Golomb Entropy Coding A codeword cn be decoded s follows:. Red in M leding zeroes followed by.. Red M-bit INFO field. 3. Find code_num = M + INFO. (For codeword 0, INFO nd M re zero.) A prmeter k to be encoded is mpped to code_num so s to produce short codewords for frequently-occurring vlues nd longer codewords for less common ones. For exmple, coded_block_pttern = 0 (no non-zero blocks) is ssigned code_num = 0 becuse it occurs frequently; coded_block_pttern = 3 (chrom DC nd non-zero AC blocks) is ssigned code_num = 6 becuse of its low frequency of occurrence. ELEG550 Video Coding Technology 40

41 Context Adptive Vrible Length Coding (CAVLC) This is the method used to encode the residul, zig-zg ordered 4 4 (nd ) blocks of trnsform coefficients. CAVLC is designed to tke dvntge of severl chrcteristics of the quntized 4 4 blocks:. After prediction, trnsformtion nd quntiztion, the blocks re typiclly sprse (contining mostly zeroes). CAVLC uses run-level coding to represent strings of zeroes compctly.. The highest non-zero coefficients fter the zig-zg scn re often sequences of ± coefficients (Triling Ones) in compct wy. 3. The number of non-zero coefficients in neighboring blocks is correlted. The number of coefficients is encoded using look-up tble nd the choice of look-up tbles depends on the number of non-zero coefficients in neighboring blocks. 4. The level (mgnitude) of non-zero coefficients tends to be lrger t the strt of the reordered rry, nd smller towrds the higher frequencies. CAVLC exploits this by dpting the choice of VLC look-up tble for the level prmeters depending on recently-coded level mgnitudes. ELEG550 Video Coding Technology 4

42 Context Adptive Vrible Length Coding (CAVLC) CAVLC syntx for encoding block of trnsform coefficients is s follows: Coeff_token Triling_ones_sign_flg Level_prefix Level_suffix Totl_zeroes Run_before Encodes the number of non-zero coefficients (Totl_coeffs) nd TrilingOnes ( per block) Sign of TrilingOne vlue ( per TrilingOne) First prt of code for non-zero coefficient ( per coefficient, excluding triling ones) Second prt of code for non-zero coefficient (not lwys present) Encodes totl number of zeroes occurring fter the first non-zero coefficient ( per block) in reverse zig-zg order Encodes the number of zeroes preceding ech non-zero coefficient in reverse zig-zg order ELEG550 Video Coding Technology 4

43 Exmple Context Adptive Vrible Length Coding (CAVLC) Reordered block: 0, 3, 0,,,, 0,, 0,.. Totl_coeffs = 5 (indexed from highest frequency, 3, to lowest frequency, 0) Totl_zeroes = 3 TrilingOnes = 3 (in fct there re 4 triling ones but only mximum of 3 cn be encoded, s indicted in Tble ) ELEG550 Video Coding Technology 43

44 Context Adptive Vrible Length Coding (CAVLC) Encoding Reordered block: 0, 3, 0,,,, 0,, 0,.. Use Tble.0 Use Tble. Use Tble 4. Use Tble 5 The trnsmitted bitstrem is ELEG550 Video Coding Technology 44

45 Decoding Context Adptive Vrible Length Coding (CAVLC) The decoder hs lredy inserted zeroes. Totl_zeroes = 3, so nother zero is inserted before the lowest coefficient. ELEG550 Video Coding Technology 45

46 H.64 Profiles 3 profiles with 5 levels for ech profile Bit rtes from 64 Kbps to 40 Mbps Bseline Error Resilient Tools SI-SP Slices Extended Dt Prtitioning Core Coding Tools Min Interlce Tools B Slices Weighted Prediction CABAC Core Coding Tools: I & P slices, different block sizes, ¼ pixel MC, MRF, deblocking filter, CAVLC, Intr prediction ELEG550 Video Coding Technology 46

47 H.64 Profiles Typicl pplictions: Bseline Min Video conferencing Low-cost/low-complexity pplictions Brodcsting HD-DVD Extended Video streming ELEG550 Video Coding Technology 47

objects Number of profiles 9 3 Compression efficiency Medium High Rectngulr video frmes nd fields Support for video streming Sclble coding Switching

48 Differences between MPEG-4 Visul & H.64 Comprison MPEG-4 Visul H.64 Supported dt types Rectngulr video frmes nd fields, rbitrry-shped video objects, still texture or synthetic-nturl hybrid video objects, D & 3D mesh objects Number of profiles 9 3 Compression efficiency Medium High Rectngulr video frmes nd fields Support for video streming Sclble coding Switching slices MC minimum block size MV ccurcy Hlf or qurter-pixel Qurter-pixel Trnsform 8 8 DCT 4 4 ICT Deblocking filter No Yes Licence pyments required for commercil implementtion Yes Yes (lower thn MPEG-4) ELEG550 Video Coding Technology 48

49 Comprisons For videoconferencing pplictions H.63 Bse H.63 Bseline Profile H.63 CHC H.63 Converstionl High Compression Profile ELEG550 Video Coding Technology 49

50 Comprisons For video streming pplictions H.63 HLP H.63 High Ltency Profile ELEG550 Video Coding Technology 50

51 Comprisons For entertinment-qulity pplictions ELEG550 Video Coding Technology 5

References http://www.vcodex.com http://bs.hhi.de/mpeg-video T. Wiegnd, H. Schwrz, A. Joch nd F. Kossentini, Rte Constrined Coder Control nd Comprison of Video Coding Stndrds, IEEE Trns. CSVT, vol.

52 References T. Wiegnd, H. Schwrz, A. Joch nd F. Kossentini, Rte Constrined Coder Control nd Comprison of Video Coding Stndrds, IEEE Trns. CSVT, vol. 3, no. 7, July 003, pp T. Wiegnd, G.J. Sullivn, G. Bjontegrd nd A. Luthr, Overview of H.64/AVC Video Coding Stndrd, IEEE Trns. CSVT, vol. 3, no. 7, July 003, pp F. Pereir nd T. Ebrhimi (eds), The MPEG-4 Book, Prentice Hll, 00. In E.G. Richrdson, The H.64 Advnced Video Compression Stndrd, Wiley, 00. ELEG550 Video Coding Technology 5

An efficient fractional-pixel motion compensation based on Cubic convolution interpolation

$An efficient fractional-pixel motion compensation based on Cubic convolution interpolation$ Journl of Electricl nd Electronic Engineering 2014; 2(3): 47-54 Published online September 20, 2014 (http://www.sciencepublishinggroup.com/j/jeee) doi: 10.11648/j.jeee.20140203.11 ISSN: 2329-1613 (Print);