Video Encoder Control

Transcription

1 Vide Encder Cntrl Thmas Wiegand Digital Image Cmmunicatin 1 / 41

2 Outline Intrductin Encder Cntrl using Lagrange multipliers Lagrangian ptimizatin Lagrangian bit allcatin Lagrangian Optimizatin in Hybrid Vide Encders Mde decisin Mtin estimatin Quantizatin Selectin f Lagrange multiplier Summary f Encding Algrithm Cding Efficiency Summary Thmas Wiegand Digital Image Cmmunicatin 2 / 41

3 Intrductin Encding Prblem Given Bitstream syntax (frmat fr transmitting cding parameters) Decding prcess (algrithm fr recnstructing vide pictures) Cding Efficiency Maximum achievable cding efficiency is determined by set f syntax features and cding tls supprted in bitstream syntax and decding prcess Actual cding efficiency fr a bitstream is determined by encding prcess = Selectin f cding mdes = Selectin f mtin parameters = Selectin f transfrm cefficient levels (quantizatin indexes) Main Encding Prblem Select cding parameters such that the cding efficiency is maximized Have t cnsider encding delay and cmplexity f the algrithm Thmas Wiegand Digital Image Cmmunicatin 3 / 41

4 Intrductin Encding Prblem Encding prblem fr given input vide s v Generate a cnfrming bitstream b B such that the distrtin D ( s v, s v (b)) between the input vide s v and its recnstructin s v (b) is minimized while the bit rate R(b) des nt exceed a given bit rate budget R B b = arg min D ( s v, s v (b)) subject t R(b) R B b B Equivalent prblem Generate a cnfrming bitstream b B such that the bit rate R(b) is minimized while the distrtin D ( s v, s v (b)) between the input vide s v and its recnstructin s v (b) des nt exceed a given maximum distrtin D max b = arg min R(b) subject t D ( s v, s v (b)) D max b B = Impssible t find ptimal slutin (extremely large parameter space) = Split int smaller sub-prblems Thmas Wiegand Digital Image Cmmunicatin 4 / 41

5 Encding Cntrl using Lagrange multipliers Lagrangian Optimizatin Cnstrained Optimizatin Prblem Cnsider set f samples s (blck, picture, r set f pictures) Vectr f cding parameters p P Cnstrained prblem fr given rate budget R B, with D(p) = D ( s, s (p) ) min p P D(p) subject t R(p) R B Varying R B = Optimal cding parameter vectrs {p pt } Uncnstrained Optimizatin Prblem Using Lagrange multipliers λ 0, we btain the uncnstrained prblem p λ = arg min p P D(p) + λ R(p) Cannt find all ptimal slutins {p pt } But: Each slutin p λ is an ptimal slutin, {p λ } {p pt} Thmas Wiegand Digital Image Cmmunicatin 5 / 41

6 Encding Cntrl using Lagrange multipliers Optimality f Lagrangian Apprach Each slutin f uncnstrained prblem is als a slutin f riginal prblem Cnsider slutin p λ fr a particular value f λ, with λ 0 By definitin, we have p P, D(p) + λ R(p) D(p λ) + λ R(p λ) D(p) D(p λ) λ (R(p λ) R(p) ) Cnsider all parameter vectrs p with R(p) R(p λ ) Since λ 0, the abve inequality implies p P : R(p) R(p λ), D(p) D(p λ) Hence, p λ is a slutin f the cnstrained prblem p λ = arg min p P D(p) subject t R(p) R B = R(p λ) Thmas Wiegand Digital Image Cmmunicatin 6 / 41

7 Encding Cntrl using Lagrange multipliers Lagrangian Optimizatin Illustratin D cnvex hull tangent with slpe -λ D+λR cnvex hull D = -λ R d slutin f cnstrained prblem D D+λR d (1+λ 2 ) 0.5 slutin f uncnstrained prblem slutin f uncnstrained prblem tangent parallel t R-axis R B R R Slutins f uncnstrained ptimizatin prblem Subset f slutins f uncnstrained ptimizatin prblem Minimize distance d t lines D = λ R Slutins lie n cnvex hull f area f all pssible rate-distrtin pints Thmas Wiegand Digital Image Cmmunicatin 7 / 41

8 Encding Cntrl using Lagrange multipliers Lagrangian Bit Allcatin Lagrangian Optimizatin fr Independent Subsets Cnsider partitining f s int independent subsets s k Cnsider any additive distrtin measure, D = D k Overall ptimizatin prblem {p 0, p 1, } = arg min p 0 P 0, p 1 P 1, D k (p k ) + λ k k R k (p k ) Can be slved by separate minimizatins k, p k = min p k P k D k (p k ) + λ R k (p k ) Key Advantage f Lagrangian Optimizatin Glbal ptimum prblem can be slved by separate minimizatins Yields ptimal bit allcatin {R 0, R 1, } Thmas Wiegand Digital Image Cmmunicatin 8 / 41

9 Encding Cntrl using Lagrange multipliers Lagrangian Bit Allcatin Illustratin D subset A subset B subset C subset D subset E D cnvex hull nt ptimal ptimal, nt n cnvex hull ptimal, n cnvex hull R R Example: 5 subsets (A,B,C,D,E), each with 6 perating pints Fr entire set: 6 5 = 7776 cding ptins Cnstrained ptimizatin: Evaluate all 7776 cmbinatins Lagrangian apprach: Only 5 6 = 30 cmparisns required Thmas Wiegand Digital Image Cmmunicatin 9 / 41

10 Lagrangian Optimizatin in Hybrid Vide Encders Lagrangian Optimizatin in Vide Encders Decisins fr blcks are nt independent f each ther Intra-picture predictin Mtin-cmpensated predictin Mtin vectr predictin Cnditinal entrpy cding Cncept f Lagrangian ptimizatin is still applicable Partly neglect dependencies between cding decisins Apprach with same cmplexity as the methd fr independent sets min D k (p k p k 1, p k 2, ) + λ R k (p k p k 1, p k 2, ) p k P k = Past decisins {p k 1, p k 2, } are taken int accunt (by using crrect predictrs and cnditinal entrpy cdes) = Impact n decisins fr fllwing blcks is ignred Thmas Wiegand Digital Image Cmmunicatin 10 / 41

11 Lagrangian Optimizatin in Hybrid Vide Encders Decisins n Blck Level Still very large parameter space fr single blck Split decisin prcess fr a blck int smaller prblems = Mde decisin = Mtin estimatin = Quantizatin Use different amunt f simplificatins fr sub-prblems Distrtin measure fr encder decisins Require simple additive distrtin measure D k = s i s k s i s i β β = 2: Sum f squared differences (SSD) = Mde decisin & quantizatin β = 1: Sum f abslute differences (SAD) = Mtin estimatin Alternative measure fr mtin estimatin (sub-sample vectrs): = SAD after Hadamard transfrm Thmas Wiegand Digital Image Cmmunicatin 11 / 41

12 Mde Decisin Lagrangian Mde Decisin Cnsider small set C k f cding mdes fr blck s k Example: C k = {Intra, Inter, Skip, Split} Assciated parameters are determined in advance = Mtin vectrs, intra predictin mdes, transfrm cefficient levels,... Each cding mde c C k is assciated with cding parameters p k (c) = Small subset P Ck = {p k (c) c C k } f the parameter space P k Lagrangian mde decisin Subset P Ck small enugh fr testing all included parameter vectrs with c k = arg min c C k D k (c p k (c), p k 1, ) + λ R k (c p k (c), p k 1, ) D k (c ) SSD between riginal blck s k and its recnstructin s k(p k (c)) R k (c ) Number f bits fr transmitting cding parameters p k (c) Nte: Requires cmplete transfrm cding fr each tested cding mde Thmas Wiegand Digital Image Cmmunicatin 12 / 41

13 Mde Decisin Lagrangian Mde Decisin Example PSNR (Y) [db] Kimn ( , 24 Hz) Lagrangian mde decisin TM5 mde decisin bit rate [Mbit/s] bit-rate saving vs TM5 [%] Entertainment-quality test sequences Cactus (avg. 13.0%) BQTerrace (avg. 5.6%) BasketballDrive (avg. 5.9%) Kimn (avg. 8.6%) PSNR (Y) [db] ParkScene (avg. 9.2%) Example: H.262 MPEG-2 Vide, IPPP cding structure Cmpared with Test Mdel 5 (TM5): Reference encder implementatin Cnsider 4 cding mdes: C k = {Intra, Inter, NCeff, ZerMv} Average bit-rate savings relative t TM5 (fr test set): 8.5% Thmas Wiegand Digital Image Cmmunicatin 13 / 41

14 Mde Decisin Lagrangian Mde Decisin in Mdern Vide Encders Lagrangian mde decisin is typically used fr the fllwing decisins: Decisin between intra and inter cding mdes Determinatin f intra predictin mdes Decisin whether a blck is subdivided int smaller blcks Selectin f transfrm size r subdivisins fr transfrm cding Well-suited fr determining tree-based partitinings Fr each internal nde: = Decide whether blck is split Evaluate blcks in depth-first rder = Crrect predictrs frm neighbring blcks Example: Tw quadtree levels (1) Decide splitting fr A, B, C, D (2) Decide splitting fr entire blck Can be cmbined with fast pruning strategies Thmas Wiegand Digital Image Cmmunicatin 14 / 41

15 Mtin Estimatin Mtin Estimatin Blck Matching Algrithm The measurement windw is cmpared with different shifted blcks in the reference frame and the best match is determined The cnsidered blck f samples in the current frame is selected as a measurement windw Thmas Wiegand Digital Image Cmmunicatin 15 / 41

16 Mtin Estimatin Lagrangian Mtin Estimatin Lagrangian Cst Measure Mde decisin cncept (with transfrm cding f residual) is t cmplex fr hundreds r thusands f mtin vectr candidates = Assume that recnstructed predictin errr signal is equal t zer = Select mtin vectr m in search range M accrding t m k = arg min m M k D k (m) + λ M R k (m) with D k (m) Distrtin between riginal blck s k and predictin signal ŝ k R k (m) Number f bits fr transmitting the mtin vectr m λ M Lagrange multiplier (depends n chsen distrtin measure) Distrtin measure SSD (λ M = λ) r SAD (λ M = λ) = SAD is ften faster t cmpute SAD after Hadamard transfrm (better apprximatin f real RD csts) Thmas Wiegand Digital Image Cmmunicatin 16 / 41

17 Mtin Estimatin Lagrangian Mtin Estimatin Cst Measure bit-rate saving vs TM5-ME [%] Kimn ( , 24 Hz) Full transfrm cding (avg. 17.8%) 30 Hadamard SAD (avg. 12.2%) 25 SAD (avg. 6.6%) SSD (avg. 6.5%) PSNR (Y) [db] Cmpare exhaustive Lagrangian mtin search with TM5 apprach Full transfrm cding (similar t mde decisin) = very cmplex SSD between riginal and predictin signal SAD between riginal and predictin signal Hadamard SAD between riginal and predictin signal Thmas Wiegand Digital Image Cmmunicatin 17 / 41

18 Mtin Estimatin Search Strategy Sub-sample accurate mtin vectrs Sub-sample lcatins: Interplatin Split search int integer-sample search and sub-sample refinement(s) Use simpler distrtin measure fr integer-sample search Fast integer-sample search Reduce number f tested candidates Multiple strategies suggested Examples: Three-step search Lgarithmic search Cnjugate directinal search Enhanced predictive znal search... Thmas Wiegand Digital Image Cmmunicatin 18 / 41

19 Mtin Estimatin Fast Search Strategies: Lgarithmic Search Lgarithmic search [Jain, Jain, 1981] Iterative cmparisn f the cst measures at 5 pints (crners and center) f a diamnd-shaped pattern Mve pattern s that pattern in centered arund best match N mre than 3 new candidates Lgarithmic refinement f search pattern (4 new candidates) if Best match is in center f pattern Or best match is at the brder f the search range Mtin search is terminated if Best match is in center f pattern And smallest pattern size is used Thmas Wiegand Digital Image Cmmunicatin 19 / 41

20 Mtin Estimatin Fast Search Strategies: Diamnd Search Diamnd search [Li, Zeng, Liu, 1994] and [Zhu, Ma, 1997] Iterative search with 9 pints f a diamnd pattern Similar search strategy as lgarithmic search Start with large diamnd pattern at mtin vectr (0,0) r at a predicted vectr If best match is in the center f a large diamnd, prceed with a smaller diamnd If best match des nt lie in the center f the diamnd pattern, center next diamnd pattern at the best match Thmas Wiegand Digital Image Cmmunicatin 20 / 41

21 Mtin Estimatin Fast Search Strategies: Chsing f Start Pint Nn-adaptive chices f start pint Use mtin vectr (0,0) as start pint f mtin search = Suitable fr applicatins like vide cnferencing = Prblematic if large mtins ccur in vide sequence Use mtin vectr predictr as start pint fr mtin search = Typically results in faster terminatin f mtin search Adaptive chice f start pint General idea: Mtin f a blck is similar t at least ne f the neighbring blcks First evaluate the mtin vectrs f the already estimated neighbring blcks Example: Blcks A, B, C and D Candidates can als include a temprally predicted mtin vectr Chse best match amng the candidates as start pint f the mtin search Thmas Wiegand Digital Image Cmmunicatin 21 / 41

22 Mtin Estimatin Mtin Estimatin Search Strategy bit-rate saving vs TM5-ME [%] Kimn ( , 24 Hz) Hadamard SAD (avg. 12.2%) 30 Sub-sample refinement (avg. 10.3%) 25 SAD + HSAD (avg. 9.3%) 20 Fast SAD + HSAD (avg. 9.1%) 15 SAD + SAD (avg. 5.3%) PSNR (Y) [db] TM5 apprach: Integer search with SAD Sub-sample refinement with SAD Cst measure: Distrtin (withut a rate term) Cmpare different search strategies with TM5 apprach Exhaustive search (all sub-sample lcatins) with Hadamard SAD Exh. integer search + sub-sample refinement (bth with Hadamard SAD) Exh. integer search with SAD + sub-sample refinement with Hadamard SAD Exh. integer search with SAD + sub-sample refinement with SAD Fast integer search with SAD + sub-sample refinement with Hadamard SAD Thmas Wiegand Digital Image Cmmunicatin 22 / 41

23 Mtin Estimatin Mtin Estimatin Summary Require significant cmplexity reductin cmpared t mde decisin Neglect dependencies between mtin vectrs and transfrm cefficient levels = Assume residual signal equal t zer during mtin estimatin Split mtin search int integer-sample search and sub-sample refinement Apply fast search strategies Cnfiguratin with reasnable trade-ff between cding efficiency and cmplexity Fast integer-sample search with Lagrangian cst Distrtin: SAD between riginal and predictin signal Rate: Number f bits required fr transmitting mtin vectrs Fast search strategy: HM apprach (cmbinatin f different cncepts) Sub-sample refinement with Lagrangian cst Distrtin: Hadamard SAD between riginal and predictin signal Rate: Number f bits required fr transmitting mtin vectrs Thmas Wiegand Digital Image Cmmunicatin 23 / 41

24 Mtin Estimatin Reference Picture Selectin Typical apprach Determine mtin vectr m r fr each cnsidered reference picture r R Select mtin parameters {r, m r } amng the pre-determined sets Criterin fr reference picture selectin Lagrangian decisin similar t mtin search Chse mtin parameters {r, m r } accrding t r = arg min r R D k (r, m r ) + λ M R k (r, m r ) with D k (r, m r ) Distrtin between riginal blck s and predictin signal ŝ R k (r, m r ) Number f bits fr reference index r and mtin vectr m r = Usually: Same distrtin measure as fr sub-sample search Thmas Wiegand Digital Image Cmmunicatin 24 / 41

25 Quantizatin Quantizatin Vide Cdecs: Transfrm cding with rthgnal blck transfrms Inverse transfrm and frward transfrm are given by s = B t and t = B 1 s = B T s with B being the inverse transfrm matrix SSD distrtin in sample space = SSD distrtin in transfrm dmain D = (s s ) T (s s ) = (t t ) T (B T B)(t t ) = (t t ) T (t t ) Mdern Vide Cdecs: Unifrm recnstructin quantizers (URQs) Inverse quantizer mapping t k = k q k Distrtin fr vectr q f quantizatin indexes is given by D(q) = N 1 k=0 N 1 D k (q k ) = (t k k q k ) 2 Thmas Wiegand Digital Image Cmmunicatin 25 / 41 k=0

26 Quantizatin Quantizatin Lagrangian Optimizatin Simple: Minimize SSD distrtin fr given quantizatin step sizes k Orthgnal transfrms: Independent treatment f transfrm cefficients D(q) = N 1 k=0 N 1 D k (q k ) = (t k k q k ) 2 k=0 SSD distrtin is minimized by simple runding accrding t tk q k = sgn(t k ) + 1 k 2 Lagrangian ptimizatin Imprve cding efficiency by taking int accunt bit rate q = arg min q Q N D(q) + λ R(q) Entrpy cding explits dependencies between transfrm cefficient levels = Transfrm cefficient levels cannt be treated separately Thmas Wiegand Digital Image Cmmunicatin 26 / 41

27 Quantizatin Rate-Distrtin Optimized Quantizatin (RDOQ) Prblem: Evaluatin f prduct space Q N is much t cmplex min q Q N D(q) + λ R(q) Reasnable assumptins Recnstructin vectr t lies inside assciated quantizatin cell Levels with abslute value t k d nt require mre bits than the less prbable levels with an abslute value t k + 1 = Cnsider at mst tw candidate levels per transfrm cefficient tk tk q k,0 = sgn(t k ) and q k,1 = sgn(t k ) + 1 k 2 k Rate-distrtin ptimized quantizatin Cnsider a small number f candidate levels (e.g., 1-2 per cefficient) Perhaps: Neglect sme aspects f the entrpy cding technique Actual algrithm depends n entrpy cding Thmas Wiegand Digital Image Cmmunicatin 27 / 41

28 Quantizatin Entrpy Cding Example: Run-Level Cding Run-Level Cding (e.g., H.262 MPEG-2 Vide) Map scanned sequence f transfrm cefficients t (run,level) pairs run : Number f transfrm cefficient levels equal t zer that precede the next nn-zer transfrm cefficient level level : Value f the next nn-zer transfrm cefficient level Cdewrds are assigned t (run,level) pairs Cde includes an additinal end-f-blck symbl (eb) = Signals that all fllwing transfrm cefficient levels are equal t zer Run-level cding is a practical example f a V2V cde Example: Scanned sequence f 20 transfrm cefficient levels A cnversin int run-level pairs (run,level) yields (0,5) (0, 3) (3,1) (1, 1) (2, 1) (eb) Thmas Wiegand Digital Image Cmmunicatin 28 / 41

29 Quantizatin Example: RDOQ fr Run-Level Cding Cnsider sub-sequences f transfrm cefficient levels (in cding rder) Distrtin D(q k ) fr sub-sequences q k = (q 0, q q,, q k ) D(q k ) = k i=0 (t i i q i ) 2 Number f bits R(q k ) fr sub-sequences q k = (q 0, q q,, q k ) q k 0 = Add up cdewrd lengths fr (run,level) pairs q k = 0 = Rate R(q k ) depends n fllwing levels = Trellis-based apprach (n further simplificatin required) RDOQ algrithm fr run-level cding: Prcess cefficients in scanning rder Cnsider (up t) tw candidate levels fr each cefficient: q k,0 and q k,1 Keep (at mst) ne sub-sequence q k = (q 0, q q,, q k ) with q k 0 Keep (at mst) k + 1 sub-sequences q k = (q 0, q q,, q k ) with q k = 0 (each with a different number f zers at the end) Final decisin at end f blck Thmas Wiegand Digital Image Cmmunicatin 29 / 41

30 Quantizatin Simple Example fr k = 10 and λ = 10 t k q k,i (q 0,, q k ) distrtin D number f bits R D + λ R 36 3 {3} 6 2 = 36 R(0, 3) = 6 96 = discard 4 {4} 4 2 = 16 R(0, 4) = {4, 0} = ? =?? [incmplete] 1 {4, 1} = R(0, 1) = {4, 0, 1} = R(1, 1) = = discard {4, 1, 1} = R(0, 1) = {4, 1, 1, 0} = ? =?? [incmplete] 1 {4, 1, 1, 1} = R(0, 1) = {4, 1, 1, 0, 0} = ? =?? [incmplete] {4, 1, 1, 1, 0} = ? =?? [incmplete] 6 0 {4, 1, 1, 0, 0, 0} = R(eb) = {4, 1, 1, 1, 0, 0} = R(eb) = = chse 1 {4, 1, 1, 0, 0, 1} = R(2, 1) + R(eb) = {4, 1, 1, 1, 0, 1} = R(1, 1) + R(eb) = excerpt f H.262 MPEG-2 Vide cdewrd table fr transfrm cefficient levels (s = sign) run level cdewrd run level cdewrd run level cdewrd 0 ±1 11s 0 ± s 2 ± s 0 ± s 1 ±1 011s eb 10 Thmas Wiegand Digital Image Cmmunicatin 30 / 41

31 Quantizatin Lw-Cmplexity Quantizatin Cmplexity f RDOQ Rather large due t cnsideratin f dependencies in entrpy cding Cmplexity reductin: Neglect dependencies Lw-Cmplexity Quantizatin: General Idea Neglect dependencies between transfrm cefficient levels Use simple rate mdels fr transfrm cefficient levels = Example: R(q) = a + b q Assume that recnstructed cefficient lies inside quantizatin cell = Tw candidate levels and q k,0 = sgn(t k ) t k / k q k,1 = q k,0 + sgn(t k ) (runding twards zer) (runding away frm zer) Thmas Wiegand Digital Image Cmmunicatin 31 / 41

32 Quantizatin Lw-Cmplexity Quantizatin cnsidered range fr t k Δ k decisin threshld d k q k,0 t k q k,0 (t k ) 1 qk,0 t k q k,1 t k = q k,0 t k + 1 q k,0 t k +2 Δ k quantizatin interval fr q k,0 quantizatin interval fr q k,1 Withut lss f generality: Cnsider t k 0 Chse q k = q k,0 if and nly if (t k k q k,0 ) 2 + λ R(q k,0 ) (t k k (q k,0 + 1)) 2 + λ R(q k,0 + 1) which yields q k = q k,0 t k d k (q k,0 ) k with decisin threshld d k (q k,0 ) = q k, λ (R(q 2 2 k,0 + 1) R(q k,0 )) k Thmas Wiegand Digital Image Cmmunicatin 32 / 41

33 Quantizatin Lw-Cmplexity Quantizatin Decisin threshld d k (q k,0 ) = q k, λ 2 2 (R(q k,0 + 1) R(q k,0 )) k Example: Simple rate mdel R(q) = a + b q d k (q k,0 ) = q k, λ b 2 2 k Lagrange parameter is ften set accrding t λ = c 2 d k (q k,0 ) = q k, b c 2 Extending cnsideratins t negative values yields ( tk q k = sgn(t k ) + f k with f k = max 0, 1 k 2 b c ) 2 = Runding with cnstant ffset f k = Runding ffset f k can be determined experimentally Thmas Wiegand Digital Image Cmmunicatin 33 / 41

34 Quantizatin Cmparisn f Quantizatin Methds PSNR (Y) [db] Kimn ( , 24 Hz) RDOQ f k = f k = bit-rate saving [%] Kimn ( , 24 Hz) RDOQ vs f k =0.5 (avg. 20.5%) f k =0.2 vs f k =0.5 (avg. 15.8%) RDOQ vs f k =0.2 (avg. 5.8%) bit rate [Mbit/s] PSNR (Y) [db] Example: H.265 MPEG-H HEVC Simple runding (f k = 0.5) Experimentally ptimized runding ffset = f k = 0.2 Rate-distrtin ptimized quantizatin = Changing f quantizatin ffset f k yields large cding gain (ca. 16%) = Cnsideratin f actual entrpy cding (RDOQ) gives additinal 6% gain Thmas Wiegand Digital Image Cmmunicatin 34 / 41

35 Selectin f Lagrange Multiplier Selectin f Lagrange Multiplier Discussed appraches f Lagrangian ptimizatin Encder peratin pint is determined by Quantizatin parameter QP Lagrange multiplier λ Lagrange multiplier λ M is nt cnsidered as additinal degree f freedm = We always chse λ M = λ (fr SSD) r λ M = λ (fr SAD) Typically, QP can be mdified n a blck basis Fr each λ, there is an ptimal chice f QP values Cnsequent ptimizatin: Chse QP values as part f the encding prcess Culd be incrprated int mde decisin {c k, QP k } = arg min D k (c, QP) + λ R k (c, QP) c C, QP Q = Minimizatin ver prduct space C Q substantially increases cmplexity = Desirable: Deterministic relatinship between λ and QP Thmas Wiegand Digital Image Cmmunicatin 35 / 41

36 Selectin f Lagrange Multiplier Apprximate Relatinship between λ and QP High-rate apprximatin Assume: Strictly cnvex peratinal distrtin rate functin D(R) d d ( D(R) + λ R ) = 0 = λ = d R d R D(R) High-rate apprximatin: D(R) = a e b R = λ = d d R D(R) = a b e br = b D(R) High-rate apprximatin: D( ) = 2 /12 = λ = cnst 2 Relatinship between λ and QP High-rate apprximatins are nt cmpletely realistic fr a vide cdec Nnetheless, indicate strng dependency between λ and QP λ 2 (nte: QP specifies quantizatin step size ) Thmas Wiegand Digital Image Cmmunicatin 36 / 41

37 Selectin f Lagrange Multiplier Experimental Investigatin f λ-qp Relatinship D+λ R (per luma sample) Kimn ( , 24 Hz) minima are marked by circles 60 λ = 1000 λ = λ = 32 λ = λ = quantizatin parameter QP Lagrange multiplier λ Entertainment-quality test sequences BasketballDrive BQTerrace Cactus Kimn ParkScene λ = QP/ quantizatin parameter QP Experiment fr IPPP cding with H.265 MPEG-H HEVC Fix λ and run encdings with all supprted QP values Chse QP that minimizes D + λ R fr given λ Plt btained (λ,qp) pints int diagram (fr multiple test sequences) Regressin yields apprximate relatinship λ = QP/3 = cnfirms λ 2 (nte: 2 QP/6 ) Thmas Wiegand Digital Image Cmmunicatin 37 / 41

38 Selectin f Lagrange Multiplier Experimental Investigatin f λ-qp Relatinship quantizatin Lagrange multiplier λ = f(qp) fr... step size intra pictures inter pictures H.262 MPEG-2 Vide QP λ = 0.6 QP 2 λ = 1.0 QP 2 MPEG-4 Visual QP λ = 0.5 QP 2 λ = 1.0 QP 2 H.263 QP λ = 0.5 QP 2 λ = 0.9 QP 2 H.264 MPEG-4 AVC 2 QP/6 λ = QP/3 4 λ = QP/3 4 H.265 MPEG-H HEVC 2 QP/6 λ = QP/3 4 λ = QP/3 4 Results fr different vide cding standards Similar λ-qp relatinships fr ther vide cding standards Nte: Fr H.264 MPEG-4 AVC and H.265 MPEG-H HEVC, a value f a = 2 QP/6 2 represents apprximately the same quantizatin step size as a = QP fr the ther cnsidered standards Thmas Wiegand Digital Image Cmmunicatin 38 / 41

39 Lagrangian Encder Cntrl Cding Efficiency Lagrangian Encder Cntrl: Summary Trade-ff between cding efficiency and cmplexity Impssible t cnsider all dependencies between cding parameters Neglect impact f certain decisins n selectin f ther cding parameters Chsen degree f simplificatin determines trade-ff between cmplexity and cding efficiency Feasible encding algrithm Select peratin pint using the quantizatin parameter QP Set Lagrange multiplier λ accrding t determined relatinship λ = f(qp) Lagrangian mtin estimatin cnsisting f Fast integer-sample search using SAD as distrtin measure Sub-sample refinement using Hadamard SAD as distrtin measure Rate-distrtin ptimized quantizatin Lagrangian decisin between cding mdes = This algrithm will be used in all fllwing experiments Thmas Wiegand Digital Image Cmmunicatin 39 / 41

40 Lagrangian Encder Cntrl Cding Efficiency Lagrangian Encder Cntrl Cding Efficiency PSNR (Y) [db] Kimn ( , 24 Hz) Exhaustive Opt Opt. MD Opt. MD, ME, Q 38 Opt. MD, ME 37 Test Mdel 5 (TM5) bit rate [Mbit/s] bit-rate saving vs TM5 [%] Kimn ( , 24 Hz) 60 Exhaustive Opt. (avg. 28.7%) 50 Opt. MD, ME, Q (avg. 23.2%) 40 Opt. MD, ME (avg. 16.8%) 30 Opt. MD (avg. 8.5%) PSNR (Y) [db] Experimental results fr IPPP cding with H.262 MPEG-2 Vide Started with Test Mdel 5 (TM5) and successively enabled Lagrangian appraches fr mde decisin, mtin estimatin, and quantizatin Additinal test Exhaustive ptimizatin Transfrm cding with RDOQ fr all pssible mtin vectrs Increases encder run time by mre than a factr f 1000 Thmas Wiegand Digital Image Cmmunicatin 40 / 41

41 Summary Summary Encding prblem Minimize distrtin while nt exceeding given bit rate Minimize bit rate while nt exceeding given distrtin Lagrangian ptimizatin Frmulate cnstrained prblem as uncnstrained prblem (D + λ R) Slutins f uncnstrained prblem are als slutins f riginal prblem Fr independent sets and additive distrtin measures: Glbal ptimum is fund by separate minimizatins Lagrangian encder cntrl Partly neglect dependencies between blcks (ignre impact n future) Partly neglect dependencies between cding parameters fr a blck Split decisin fr a blck int Mde decisin (includes transfrm cding) Mtin estimatin (assumes zer residual signal) Quantizatin (cnsiders actual entrpy cding) Thmas Wiegand Digital Image Cmmunicatin 41 / 41