Modelling of produced bit rate through the percentage of null quantized transform coefficients ( zeros )

Size: px

Start display at page:

Download "Modelling of produced bit rate through the percentage of null quantized transform coefficients ( zeros )"

Roger Sparks
5 years ago
Views:

1 Rate control strategies in H264 Simone Milani with the collaboration of Università degli Studi di adova ST Microelectronics

2 Summary General scheme of the H.264 encoder Rate control problem over limited bandwidth channels The rate control algorithm JVT-D030 The rate control algorithm JVT-D030 Modelling of produced bit rate through the percentage of null quantized transform coefficients ( zeros ) A rate control algorithm based on the percentage of zeros arametric models for the coefficients statistics Future work Conclusions 2

3 The H.264 standard Macroblock Frame format Size QCIF 76 X 44 CIF 352 X 76 Input signal YUV 4:2:0 8 bit/sample Macroblock blocks smaller blocks 4 4 H.264 standardizes a hybrid transform video coder with spatial and temporal prediction 3

4 Spatial and temporal prediction Intra Inter (MME) The current 4 x 4 block is predicted from the neighboring pixels according to different directions. Motion vector resolution: pixel 4 Image coding through motion vectors. 2/04/2007 iattaforme Riconfigurabili per 4

5 The transform The prediction error, i.e. the difference between the original signal and the predicted one, is coded. The transform is applied on 4 x 4 pixel blocks. The transform is an accurate approximation of the Discrete Cosine Trasform, implemented with sums and shifts, operating with integer values. The transform coefficients are quantized. Transform x: prediction error signal 2 Y = 2 2 x 2 x x x An Hadamard Transform 4 x 4 can be optionally performed over the DC coefficients. x x x x x x x x x x x x Quantization =K 2 Q/6 Q [0,5] The coefficients are uniformly quantized 5 step sizes are defined (Q) There is an exponential relation between Q e 5

6 it stream coding in the H.264 standard The quantized transform coefficients are converted into a sequence of couples (run,length). The encoder syntax elements are converted into a binary stream with two possible coding engines CAVLC CAAC Single scan order CAVLC Context-based Adaptive Variable Length Coding CAAC Context-based Adaptive inary Arithmetic Coding C: current block A,: previously coded blocks Each syntax element is coded according to the information coded in the previous blocks 6

allow the H.264 standard to provide better video quality than MEG4 MEG-4 H.

7 A comparison between H.264 and MEG4 A performing prediction scheme and an adaptive entropy coder allow the H.264 standard to provide better video quality than MEG4 MEG-4 H frames/s GO I 60 frames MEG4-TM5/H264- zeros Standard SNR(d) MEG H Standard SNR(d) MEG H

8 Rate Control Target The transmission of a coded video sequence over a bandlimited channel with minimum distortion. Control parameters Q: quantization parameter Quality parameters SNR: reconstructed video quality ε: relative bit rate error : buffer level SNR = 0 log 0 N i o, i r, i 2 o r, i, i : : original reconstructed The rate control algorithms are designed to keep the coded video bit stream among the bandwidth constraints modifying the quantization parameter Q. The distortion is determined by the quantization parameter, which may vary according to the coded frame or macroblock. 8

9 Different kinds of rate control Constant it Rate algorithms (CR) The produced bit rate is kept nearly constant according to the available bandwidth. Variable it Rate algorithms (VR) Strong variations may affect the number of bits in the coded stream in order to keep a constant quality. Average it Rate algorithms (AR) The bit rate fits the given constraints at constant time intervals Maximum it Rate algorithms (MR) The produced bit stream is upper bounded. 9

10 Some rate control problems The choice of a rate control algorithm is affected by different requirements Rate control precision Application Storage Videocall Videoconferencing Video quality of reconstructed images Variations in reconstructed video quality during the time Coding latency due to pre-analisys Computational complexity Efficiency at low bit rates In video communication applications over mobile devices, rate control techniques that require low complexity and coding latency are required. 0

Rate control algorithms classification Feed Forward algorithms the input signal is

allow to keep the produced bit stream under the channel constraints Feed ackward

11 Rate control algorithms classification Feed Forward algorithms the input signal is pre-analyzed (pre-coded) in order to find the video coding parameter values that allow to keep the produced bit stream under the channel constraints Feed ackward algorithms the input signal is coded without pre-analysis, using the knowledge of the past. E.g. the rate control algorithm based on zeros Hybrid algorithms they are based on a blending between the previous two techniques E.g. the algorithms JVT-D030, JVT-E069.

12 2 The JVT-D030 algorithm It is quite similar to the algorithm TM5 of MEG-2 According to the available number of bits, a target bit size is assigned to the frame After each frame is coded, the transmission buffer and the parameters that characterizes the frame statistics are updated. + = + = + + = R b R b R b I I I F R X K X K N N R T F R X K X K N N R T F R X K X N X K X N R T 8, max 8, max 8, max

The current macroblock is recoded in case the estimated optimal Q does not correspond to the one used in precoding. Q Estim.

13 M rate control for the JVT-D030 algorithm At macroblock level, the quantization parameter is computed according to the buffer level and the image statistics. recoder Each M is precoded and an optimal Q is computed from the produced bit rate and the macroblock activity. The current macroblock is recoded in case the estimated optimal Q does not correspond to the one used in precoding. Q Estim. Coder N d act ( n) = ( n) = d 2 act( n) + act 2 act { I, } { I, } { I, } (0) { I, } + act( n) T + N { I, } M n 3 d{ I, }( n) Qn = N r act ( n) uffer 3

14 Some comments about the JVT-D030 algorithm Accurate rate control Advantages The macroblock precoding requires a computational effort lower than performing the precoding at frame level. Disadvantages High computational load due to the preanalysis (in the worst case, each frame is coded twice ) Great variation of Q (and, as a consequence, video quality ) across the same image Quality losses and low performance with scene changes. Variable coding latency (not deterministic). Improvement The JVT-E069 algorithm 4

15 The JVT-E069 algorithm The algorithm structure is the same of the JVT-D030 The quantization parameter computation and some parameter initialization are different Q 3 d I n = ( {, } ) r n + dq dq act ( n) act = 0 act + act ( n) se se se act 0 < act ( n) 2 act < 2 2 act ( n) act 2 < act ( n) arameter \ Algorithm JVT-D030 JVT-E069 X I 50 R b /5 5 R b /5 X 25 R b /5 5 R b /5 X 8 R b /5 55 R b /5 I d 0 0 r D030 /3 20 r E069 /3 5

16 Comments about the JVT-E069 algorithm Advantages Accurate rate control Smoothness of video quality among different area in the same frame Reduced Q variation at macroblock level Disadvantages The complexity is not reduced Slower adaptation to channel bandwidth variations. Low quality at scene changes N, m ( n) M no. d m m ( n) / r act m (n) act Q (JVT-D030) Q (JVT-E069) act 6

17 it rate modelling through the percentage of null quantized transform coefficients ( zeros ) Many rate control technique approximate the Rate-Distortion curves through a hyperbolic model It is possible to characterize the bit rate produced by the H.264 encoder through the percentage of null quantized transform coefficients (ρ), which are called zeros : the produced bit rate is a linear function of ρ. where Hyperbolic model γ R( ) = γ 0 + D( ) : quantization step for transform coefficients x (a): coefficients distribution for the actual frame Model based on zeros Given ρ = a < p x ( a) the number of bits required to code the whole image can be related to ρ through the equation R( ρ ) = λ ( ρ ) 7

18 Experimental results from the H.264 encoder Rate (kbit) Rate (Kbit) Rate (kbit) Rate (Kbit) ρ Q then it is possible to characterize R(ρ) in a simple way: R (ρ) = m ρ + q [lots from carphone, GO I 60 frames Q=cost. Frame 0(I), 8(), 40()] 8

19 9 A rate control algorithm based on the percentage of zeros The zeros model can be used to control the bit rate produced by the H.264 encoder The whole algorithm has different control levels: GO level frame level macroblock level As for the GO and frame levels, a target number of bits is assigned to each frame according to the image statistics, the available bandwidth and the previous coding errors. I I I N N K K K G K K T + + =,,,,, N N K G K T + =,, N N K G T + =

20 A rate control algorithm based on the percentage of zeros After computing the target bit rate for the current frame, the algorithm has to find the percentage of zeros (ρ) that allows to fit the target number of bits. This value can be related to an average quantization parameter through the coefficients distribution. From the equation R (ρ ) = m ρ + q It is possible to relate the desired bit rate to a target percentage of zeros that can be related to a quantization step or quantization parameter Q through the equation ρ = a < p ( a x ) The parameters m, q and the coefficients distribution are found from the previously coded frames. At macroblock level, the average Q value is corrected according to the actual bit rate and percentage of zeros Available bits ρ k = ρ + Q = Qavg n if k < n δk 20

21 Comments about the zeros algorithm Advantages Sufficiently accurate rate control. Quality smoothness between different areas of the same frame. Limited SNR variations between different frames. Reduced and deterministic coding delay. Good performance at low bit rate. Robustness at scene changes. Fast adaptation to channel variations. Great reuse of H.264 encoder syntax parameters. Disadvantages Less accurate than JVT algorithms. Requires a great memory area and a great number of memory accesses. At high bit rate, the performance are either equal or slightly worse. 2

22 A comparison between different algorithms arameter Value GO I it rate 64 kbit/s Frame rate 30 frame/s No. frames GO 60 Coding engine CAVLC Zeros JVT-E069 SNR (d) (d) Zeri JVT-E069 JVT-D Scene change Frame no. 22

23 23 Introduction of parametric models in the rate control algorithm based on zeros It is possible to reduce the hardware requirements of the zeros algorithm through the use of parametric models that approximate the coefficient distribution. In our investigation we considered two approximations: a generalized gaussian p.d.f. a laplacian+impulsive p.d.f. Generalized gaussian p.d.f. Laplacian+impulsive p.d.f. ( ) α β γ ) ( a x e a f = Γ Γ = α α σ β 3 ( ) α β α γ Γ = 2 with ( ) = Γ 0 dt e t p t p ) ( ) ( a d e b a f a c x δ + = where δ(a) is the Dirac impulse. The l+i model turns out to be the simplest one and provides the lower computational load.

24 A comparison between different algorithms 7 x 04 6 zeros JVT-E i t ti 4 3 RNS Frame no. 26 ``zeros'' JVT-E Frame no. Sequence foreman R b =64000 bit/s F r =30 frame/s GO I 60 frame CAVLC 24

25 A comparison between different algorithms(2) 2.5 x t i JVT-E069 Zeros Frame no Sequenza foreman R b =64000 bit/s F r =30 frame/s GO I 60 frame CAVLC RNS Frame no. 25

26 A comparison between different algorithms(3) RNS zeros JVT-E Rate (kbit/s) Sequence foreman F r =30 frame/s GO I 60 frame CAVLC 26

27 A comparison between different algorithms(4) x 04 zeros JVT-E zeros JVT-E t i RNS Frame no Frame no. Sequenza container R b =64000 bit/s F r =30 frame/s GO I 60 frame CAVLC 27

A comparison between different algorithms(5) 40 39 JVT-E069 Zeros 38 37 RNS 36 35 34 33 32 20 30 40 50 60 70

28 A comparison between different algorithms(5) JVT-E069 Zeros RNS Rate (kbit/s) zeros JVT-E069 Sequenza container R b =64000 bit/s F r =30 frame/s GO I 60 frame CAVLC 28

29 A comparison between different algorithms(6) 7 x 04 6 zeros JVT-E ti 4 3 RNS Frame no. 26 ``zeros'' JVT-E Frame no. Sequence foreman R b =64000 bit/s F r =30 frame/s GO I 60 frame CAVLC 29

30 A comparison between different algorithms(7) 40 JVT-E069 Zeros RNS Frame no. Sequence table R b =64000 bit/s F r =30 frame/s GO I 60 frame CAVLC 30

31 A comparison between different algorithms(8) RNS Rate (kbit/s) zeros JVT-E069 Sequence table F r =30 frame/s GO I 60 frame CAVLC 3

32 A comparison between different algorithms(0) JVT-E069 Zeros Sequence table CIF R b =92000 bit/s F r =30 frame/s GO I 60 frame CAVLC 32

33 A comparison between different algorithms(9) RNS Rate (kbit/s) Sequence table CIF F r =30 frame/s GO I 60 frame CAVLC 33

34 Conclusions The different algorithms presented have similar rate control accuracy. The zeros algorithms performances are better at low bit rates (higher SNR and reduced quality variation) and comparable at high bit rates. The JVT-D030 and JVT-E069 algorithms requires a greater computational load. The zeros algorithms are more robust against scene changes than JVTs. The zeros algorithm is faster and provides an efficient technique for the implementation over time-varying channels. The coding latency of the zeros algorithms is constant. The zeros algorithms reuse many parameters belonging to the H.264 syntax. 34

35 Future work A VR rate control algorithm. The adaptation of the rate control algorithm to the transmission over a packet switched network, taking into account the existing session layer protocols for audio/video transmission (e.g. RT, ). Increased robustness to errors and packet losses through a joint source-channel coding. A layered rate control implementation. Integration of the rate control into a multiple description scheme. 35

Introduction to Video Compression H.261

Introduction to Video Compression H.6 Dirk Farin, Contact address: Dirk Farin University of Mannheim Dept. Computer Science IV L 5,6, 683 Mannheim, Germany farin@uni-mannheim.de D.F. YUV-Colorspace Computer