New rate adaptation method for JPEG2000-based SNR Scalable Video Coding with Integer Linear Programming models

Transcription

1 New rate adaptaton method for JPEG2000-based SNR Scalable Vdeo Codng wth Integer Lnear Programmng models Lvo Lma #1, Renata Mansn, Rccardo Leonard #2 # Sgnal Processng Group Department of Electroncs for Automaton, Unversty of Bresca Va Branze, Bresca, Italy 1 lvo.lma@ng.unbs.t 2 rccardo.leonard@ng.unbs.t Operatons Research Group Department of Electroncs for Automaton, Unversty of Bresca Va Branze, Bresca, Italy renata.mansn@ng.unbs.t Abstract In the last few years scalable vdeo codng emerged as a promsng technology for effcent dstrbuton of vdeos through heterogeneous networks. In a heterogeneous envronment, the vdeo content needs to be adapted n order to meet dfferent end termnal capablty requrements (user adaptaton) or fluctuatons of the avalable bandwdth (network adaptaton). Consequently, the adaptaton problem s a crtcal ssue n scalable vdeo codng desgn. In ths paper we ntroduce a new adaptaton method for a proposed JPEG2000-based SNR scalable codec, that formulates and solves the adaptaton problem as an Integer Lnear Programmng problem. Index Terms Scalable vdeo codng, rate adaptaton, JPEG2000, Integer Lnear Programmng. I. INTRODUCTION In the last years, scalable vdeo codng emerged as a promsng technology for effcent dstrbuton of vdeos through heterogeneous networks, and t has been recently standardzed as scalable extenson of the H.264/AVC standard [1], hereafter ndcated as SVC. An useful overvew of the SVC extenson can be found n [2]. The man advantage of SVC s that t offers codng flexblty to decode dfferent workng ponts n terms of spatal, temporal and qualty resoluton from a unque coded representaton. In a heterogeneous envronment typcally the vdeo content needs to be adapted n order to meet dfferent end termnal capablty requrements (user adaptaton) or fluctuatons of the avalable bandwdth (network or rate adaptaton). In ths work we address only the rate adaptaton problem. The scalablty features gven by the scalable vdeo codng offer a very flexble way to perform the adaptaton, for example reducng the spatal resoluton or the vdeo qualty. In partcular, wth the SVC scalablty features, rate adaptaton can be effcently managed usng qualty scalablty,.e. coarse gran (CGS) or fne gran (FGS) scalablty. Although the ntrnsc support for adaptaton provded wth the scalablty, the problem of the adaptaton for scalable vdeo content s an open research ssue that s stll under nvestgaton. An exhaustve survey on the proposed approaches for solvng the adaptaton problem can be found n [3]. In [3] dfferent classfcaton crtera and propertes of the adaptaton methods are presented. For the purpose of the present work, three aspects have to be consdered when comparng dfferent adaptaton methods: the performance, evaluated n terms of decoded qualty (dependng on a partcular metrcs) of the extracted data. the satsfacton of addtonal constrants on the decoded vdeo sequences. the complexty of the adaptaton process. Among the dfferent approaches presented n [3] the frst and most attractve approach proposed for rate adaptaton wth SVC s that proposed n [4]. Recently, another nterestng approach has been proposed n [5]. In [5] the authors compare ther approach to that n [4] and they show that the two approaches have smlar extracton performance. Snce the mplementaton of the approach n [4] s ncluded n the SVC reference software (JSVM 9, verson 9.14) [6] we decde to use only ths approach as reference for evaluatng the performance of our method, descrbed n the followng sectons. Although SVC s the reference standard for scalable vdeo codng, other scalable vdeo codng approaches has been proposed n lterature. Most of them are based on the wavelet technology, that has natve spatal scalablty features. Furthermore, JPEG2000 s the state-of-the-art n stll mage compresson and t offers very effcent spatal and SNR scalablty. Inspred by the potentaltes of JPEG2000 and the prevous wavelet approaches we have proposed a very smple but effcent new scalable vdeo codng soluton based on JPEG2000. In Secton II we defne the archtecture of the proposed codec and the man features of JPEG2000 that are fundamental to understand

2 Fg. 1. Herarchcal B-pcture decomposton moton nfo ntra-frame MC-resdual moton nfo codng JPEG2000 codng Proposed JPEG2000-based codec archtecture Multplex the proposed adaptaton soluton. At present, the proposed archtecture s stll under nvestgaton relatvely to the spatal scalablty. Thus, n ths work, we only focus on the SNR scalablty, also avodng to consder the temporal scalablty for adaptaton purposes. The man dea behnd the proposed approach s that the rate adaptaton problem for SNR scalable codng can be seen as an optmal resources allocaton problem and thus formulated usng Integer Lnear Programmng (ILP) as follows: mnmze { subject to c T x Ax b x 0 nteger where c T x s the objectve functon and descrbes the target of the adaptaton problem, Ax b represents the constrants gven by the codng system and addtonal constrants on the decoded vdeo sequence and x s a vector of bnary unknowns descrbng how resources are allocated. In Secton III we descrbe the proposed nteger programmng problem for rate adaptaton and underlne some of ts propertes. Although NP-hard, the problem (1) can be effcently solved by means of a commercal software for mxed nteger lnear programmng such as CPLEX and explotng partcular propertes of the model. Fnally, Secton IV s devoted to compare our approach performance wth that of the method n [4], whereas n Secton V conclusons and possble future developments are drawn. II. PROPOSED CODEC SOLUTION The proposed scalable vdeo codng system, that at present only offers temporal and SNR scalablty features, s based on a very smple archtecture shown n Fgure 1. The nput vdeo sequence s temporally decomposed usng a smlar approach to Herarchcal B-pcture decomposton proposed for SVC (see [2] and [7]), that enables closed-loop moton estmaton and natve temporal scalablty. Herarchcal B-pcture decomposton s based on the defnton of key pcture and group of pcture (GOP). The frst pcture of a vdeo sequence s an ntra-coded pcture, whle the other key-pctures could be ntra-coded or nter coded usng the prevous key pctures as reference. Usually key-pctures are coded at regular ntervals, and the dstance between two key-pctures defnes the GOP length. In fact a key pcture and all pctures that are temporally located between the key pcture and the prevous key pcture are consdered to buld a group of pctures. Fgure 2 shows a typcal decomposton structure based on herarchcal B- pcture for a GOP wth length equal to 8. Ths partcular GOP structure enables 4 levels of temporal scalablty. In the frst (1) one, whch we call level 0 and represents the sequence at ts lowest avalable frame rate, we consder only the sequence made of key pctures, whle n a generc level ( 1) we consder the pctures used at lower temporal levels plus the pctures ndcated n Fgure 2 as B level pctures. For each GOP there s only 1 B level 1 pcture and t s predcted usng the key pcture of the prevous GOP for forward predcton and the key pcture of the same GOP for backward predcton. For > 1 the B level pctures are predcted usng the pctures belongng to the lower temporal resoluton. It should be notced that the herarchcal B-pcture structure enables closed-loop codng. In fact, the encodng order s dfferent from the dsplay order. The frst encoded pcture s the frst frame of the sequence, then for every GOP we encode the key pcture before moton estmaton and compensaton for the B level 1 pcture. Then every B level pcture s encoded before moton estmaton and compensaton of the B level + 1 pctures. Ths encodng order ensures that at each temporal level the moton estmaton and compensaton process uses the already encoded reference pctures. In the proposed scalable vdeo codec the herarchcal B-pcture decomposton s adopted wth the only constrant that the keypcture of each GOP can be only ntra-coded. After the temporal decomposton, for each GOP, the keypcture and the moto-compensaton resdual for all the B- pctures wthn a GOP are encoded wth a JPEG2000 framework as a sngle pcture. SNR scalablty s obtaned generatng JPEG2000 codestreams wth multple qualty layers. Snce the proposed work addresses only the SNR scalablty, n the followng we overvew the man features of JPEG2000 that are fundamental to understand the model proposed n Secton III. JPEG2000 s the state-of-the-art n mage compresson and s based on the Dscrete Wavelet Transform (DWT), together wth Embedded Block Codng wth Optmzed Truncaton (EBCOT) [8]. D stages of DWT analyss decompose the mage nto 3D+1 subbands, labeled LH d, HL d, HH d and LL D, for d = 1,..., D. An useful overvew of the man features of JPEG2000 can be found n [9], whle for a complete techncal descrpton the reader s referred to [10]. Each subband s parttoned nto rectangular blocks called code-blocks, each of whch s ndependently coded. Resoluton scalablty s obtaned by dscardng the code-blocks of detal subbands and omttng the fnal DWT synthess stage. Qualty scalablty s obtaned through a qualty layers abstracton. Each layer represents an ncremental contrbuton (possbly empty) from the embedded bt-stream assocated wth each code-block n the mage. Dscardng one or more layers (startng from the hghest one) produces a representaton of the code-block wth lower qualty. JPEG2000 also defnes collectons of spatally adjacent code-blocks as precncts. Each precnct of resoluton level LL d consst of the code-blocks correspondng to the same spatal regon wthn the subbands LH d+1, HL d+1 and HH d+1 f d < D, or wthn the subband LL D f d = D. The data-stream assocated wth each precnct s organzed as a

3 key-pc B-lev-3 B-lev-2 B-lev-3 B-lev-1 B-lev-3 B-lev-2 B-lev-3 key-pc f P f 1 f 2 f 3 f 4 f 5 f 6 f 7 f 8 GOP border GOP border Fg. 2. Herarchcal B-pctures temporal decomposton wth GOP sze equal to 8 collecton of packets, one for each qualty layer. III. RATE EXTRACTION OPTIMIZATION As prevously descrbed, the rate adaptaton problem can be formulated as a problem of optmal allocaton of resources. Snce n the proposed scalable vdeo codec the frames (ntraframes and moto-compensated resdual) are encoded usng JPEG2000, the resources that have to be allocated are the JPEG2000 codng elements. In the proposed nteger model the mnmum addressable codng element s the JPEG2000 packet, whereas we avod to consder for the adaptaton the moton nformaton. In the proposed approach the adaptaton s performed ndependently on each GOP (wth length equal to F frames and F s a power of 2), and consequently the optmal allocaton of the JPEG2000 packets has to be performed n order to mnmze the overall dstorton over the GOP, gven by: F D T OT = D t (2) t=1 where D t s the dstorton for the frame t evaluated as MSE between the orgnal and the decoded frame. The total dstorton (2) represents the objectve functon c T x of the model (1). The budget avalable for the allocaton s gven by the bandwdth or equvalently by the data-rate. Gven the GOP length and the frame-rate of the sequence, t s always possble to determne the amount of bts L that can be used to encode each GOP, where constant bt-rate transmsson s consdered. In secton III-A the expressons for the dstorton of the keypcture and the B-pctures wthn each GOP wll be provded, avodng the mathematcal detals that can be found n [11], whle n secton III-B the ILP model wll be presented. A. Dstorton computaton Frst, we ntroduce the dstorton contrbuton gven by a sngle JPEG2000 packet. Let I be the number of precncts n each frame (supposed to be constant over the frames), K the number of qualty layers ncluded n the data-stream assocated wth each precnct and F the GOP length (n frames). We defne the followng quanttes: P t a generc precnct, = 1,..., I, belongng to frame t, t = 1,..., F, P t,k the decoded verson of P t usng k, k = 1,..., K, qualty layers and Lt,k the sze (n bts) of the frst k packets related to the precnct P t. The dstorton ntroduced approxmatng Pt wth Pt,k s gven by D t,k = P t Pt,k 2. Introducng the rate-dstorton slope S t,n, defned as the rato D/ L related to each qualty layer of each code-block, the dstorton D t,k can be expressed as: D t,k = D t,k + S t,n L t,n (3) n=k+1 where D t,k s the dstorton experenced (possbly equal to 0) f all the qualty layers are consdered, L t,n = L t,n L t,n 1 s the sze (n bts) of the qualty layer n and S t,n L t,n s the dstorton contrbuton gven by the layer n. Unfortunately, the exact calculaton of the dstorton ntroduced for each precnct P t requre the knowledge of the rate-dstorton slopes S t,n. Typcally, n order to reduce the overhead requred to mantan ths slope nformaton, only the rate-dstorton slope threshold values used for the layer generaton are ncluded nto the JPEG2000 codestream header (see [10] for more detals). Ths means that the rate dstorton slope for a partcular layer n s consdered to be constant over the precncts and equal to the threshold Tt n. Usng the rate-dstorton slope thresholds the dstorton (3) can be approxmated as: ˆD t,k = D t,k + n=k+1 T n t L t,n (4) In order to estmate the overall dstorton D T OT, the man assumpton that s consdered hereafter s that the JPEG2000 packets are ndependent, that s approxmately true for the 9/7 tap borthogonal flters typcally used n JPEG2000. Ths leads to an addtve model for the dstorton of a frame so that the sum of all the precncts dstorton contrbuton I =1 Dt, where the sngle contrbuton Dt s estmated usng the equaton (4), can be consdered as a good approxmaton of D t. In order to estmate the dstorton of a generc frame t nsde

4 2 220 R-D Layer R-D Layer 3 18 R-D Layer proposed dstorton model measured dstorton proposed dstorton model measured dstorton proposed dstorton model measured dstorton rate [byte] rate [byte] rate [byte] (a) R-D performance of layer 1 (b) R-D performance of layer 3 (c) R-D performance of layer 7 Fg. 3. Typcal Rate-Dstorton performance of a key-pcture encoded wth 8 qualty layers Drft effect for temporal level 1 Drft effect for temporal level Drft effect temporal level proposed dstorton model measured dstorton 20 proposed dstorton model measured dstorton 60 proposed dstorton model measured dstorton dscarded qualty layers dscarded qualty layers dscarded qualty layers (a) Drft effect on temporal level 1 (b) Drft effect on temporal level 2 (c) Drft effect on temporal level 3 Fg. 4. Analyss of the drft dstorton contrbuton on B-frames at dfferent temporal levels each GOP, dfferent expressons has to be consdered for ntra-frames (key-pctures) and nter-frames (B-pctures). To correctly defne these quanttes n vew of ther use nsde the nteger problem, we need to ntroduce the problem varables as follows. Let x = (x 1,..., x F ) be a vector of bnary varables where x t s gven by: ( ) x t = x t,1 1,..., xt,k 1, x t,1 2,..., xt,k 2,..., x t,1,..., xt,k Each bnary varable x t,k assumes value 1 f the JPEG2000 packet made by the layer k of the precnct P t s consdered n the decodng process and 0 otherwse. Intra-frames are managed as stll pctures, and ths leads to a straghtforward expresson for the dstorton estmaton: ˆD F = D F,K + I =1 k=1 I I (1 x F,k )TF k L F,k (5) where we recall that wth the hrarchcal B-pcture decomposton the key-pcture s always the last pcture of each GOP and consequently t s n poston F, and that D F,K s the key pcture dstorton f all the qualty layers are decoded. The correctness of the expresson (5) has been expermentally valdated, and the results are reported n Fgure 3 whch s referred to a key-pcture encoded wth 8 qualty layers. Fgure 3 compares the dstorton model gven by the equaton (5) to the real dstorton expermentally measured. Fgures 3(a), 3(b) and 3(c) show respectvely the Rate-Dstorton performance for the qualty layer 1,3 and 7. For a partcular layer, each pont n the R-D curve s the contrbuton gven by a precnct,.e. the contrbuton of a packet, to the dstorton reducton of the whole frame. As t can be notced n Fgure 3 the dstorton model gven by equaton (5) s qute accurate, except for the very frst qualty layer. For the purpose of the rate adaptaton, ths dstorton model nconsstency n the lowest qualty layer s neglgble, snce n typcal adaptaton scenaro the lowest qualty layer s always fully consdered n the decodng process, and, as t can be notced from Fgure 3(a), the overall dstorton contrbuton gven by layer 1 s correctly estmated wth the model (5). The expresson of the dstorton for an nter-frame s not easy to derve, snce the herarchcal B-pcture decomposton wth closed-loop moton estmaton ntroduces complex relatonshps between the dstorton on the nter-frames at a partcular level of temporal decomposton and the dstorton on the reference frames of the hgher temporal levels. Brefly, the dstorton on the nter-frames depends on two factors: the dstorton on the moton-compensated resdual related to the nter-frame and the drft effect that depends on the fact that n SNR scalablty dfferent qualty verson of the reference frames could be used for moton compensaton. If the qualty verson that has been used at the encoder for the moton estmaton and compensaton process s not avalable at the decoder, for example because parts of the vdeo btstream are dscarded for rate adaptaton purpose, the drft s ntroduced. The drft problem can be reduced usng a low qualty verson of the reference frames for moton

5 ˆD τ = ntal dstorton y=1 moton-compensated resdual component {}}{{}}{ 2 τ 1 2 τ 1 I D λ,k + (1 x λ,k )Tλ k L λ,k + 2 τ 3 I =1 k=1 y=1 =1 k=1 (1 x F,k )T k F L F,k } {{ } drft component from GOP key-pcture + 2 τ 3 I τ 1 2 z 1 + z=1 w=1 =1 k=1 drft component from lower temporal layers {}}{ [ I 1 ] H 2 τ 2 z (1 x λz,k )Tλ k z L λz,k + (1 x P,k )TP k L P,k }{{} drft component from prevous GOP key-pcture =0 k=1 (6) estmaton and compensaton, but t s well know that ths choce decreases the predcton effcency and consequently the codng performance. Therefore, the common adopted soluton s to use the hghest avalable qualty verson,.e. consderng all the qualty layers, for the reference frames n moton estmaton and compensaton process, and to accept the drft effect. Furthermore, wth the herarchcal B-pcture temporal decomposton, the drft effect also depends on the temporal level. Intutvely, referrng to Fgure 2, t s clear as the drft component of the dstorton ntroduced on the B-level-1 pcture f 4 depends only on the key-pctures f P and f 8, the drft effect ntroduced on the B-level-2 pcture f 2 depends on the the key-pcture f P and the B-level-1 pcture f 4 (that s also affected by f 8 ), and so on. Although we ntentonally avod the mathematcal detals, t can be shown that a good approxmaton of the overall dstorton on the B-level-τ frames, τ = 1,..., log 2 F, s gven by equaton (6), where λ = s + (y 1)δ δ = F 2 τ 1 s = F 2 τ λ z = s z + (w 1)δ z δ z = F 2 z 1 s z = F 2 z It s mportant to note as n expresson (6) t has been consdered that the hghest avalable qualty verson of the reference frames has been used for moton estmaton and compensaton, as prevously descrbed. Ths means that the three drft components are affected by all the mssng qualty layers up to K. Although the proof of the expresson (6) can be found n [11], the expresson has been expermentally valdated, and the results are shown n Fgure 4. Fgure 4 shows the dstorton on the frames that belong to dfferent temporal levels, assumng that the moton-compensated resdual f fully consdered n the decodng process. Consequently, the dstorton s gven by the effect of the three drft components and the ntal dstorton (not equal to 0 snce lossy codng s consdered). In order to show the effect of the drft and ts approxmaton gven by equaton (6), we dscard whole qualty layers of the reference frames n the decodng process, from 0 dscarded layers (no drft) to 5 qualty layers, where the encodng process has been performed generatng 8 qualty layers. As t can be notced n Fgure 4 the dstorton model gven by equaton (6) s accurate assumng to dscard up to 3 qualty layers, a model error of approxmately 10% s evdenced for the fourth dscarded layer whle a greater error s ntroduced from the ffth dscarded layer. Nevertheless, some consderaton has to be done n order to justfy the error ntroduced. Frst, the am of the proposed work s not to gve a quanttatve expresson of the GOP dstorton, but to provde a consstent approxmaton n order to use t for the rate adaptaton model descrbed n the followng secton. Furthermore, the approxmaton error s ntroduced only f we dscard many qualty layers, that means that most of the orgnal vdeo btstream has to be dscarded for rate adaptaton purposes. Nevertheless, the typcal rate adaptaton scenaro that has been consdered n the smulatons performed assumes to dscard from 30% to 50% of the compressed vdeo btstream. It has been verfed that n ths range the proposed dstorton models are accurate. B. The ILP model We now descrbe how modelng the optmal extracton as an ILP problem. The vector x of bnary varables has already been defned. We use bnary varables snce, n order to optmze the decodng performance, we assume to decode only full packets and the stuaton n whch part of packets are forwarded or dscarded s not taken nto account. The objectve functon of the ILP problem s represented by the overall dstorton (2) that, usng the dstorton models ntroduced n the prevous secton, can be approxmated as ˆD T OT = F t=1 log ˆD t = ˆD 2 F F + τ=1 ˆD τ (7) where ˆD F and ˆD τ are respectvely gven by equatons (5) and (6). In the basc ILP model two types of constrants can be dentfed: the qualty layer constrants and the budget constrant. By the progressve layers generaton process of the EBCOT algorthm, the extracton has to verfy the followng condtons: x t,k x t,k+1 0 (8) wth = 1,..., I, k = 1,..., K 1 and t = 1,..., F. As prevously descrbed, the amount of packets that have to be dscarded to perform the adaptaton depends on the avalable bandwdth, that could be converted n terms of bts (L) avalable for each GOP. Consequently, the followng budget constrant has to be consdered: F I t=1 =1 k=1 x t,k L t,k L (9)

6 The exact soluton of the descrbed ILP problem provdes the Rate-Dstorton optmal way to adapt the btstream for each GOP. Addtonally, further constrants could be ntroduced n order to satsfy partcular decodng requrements. For example, n the performed tests the vdeo sequences have been encoded n order to enable near constant decoded qualty f the full vdeo btstream s decoded. Typcally, after the rate adaptaton process ths feature s not mantaned. In order to lmt the decoded qualty fluctuaton potentally ntroduced by the adaptaton, the followng constrants could be ntroduced n the ILP model: β ˆD T OT F ˆD t γ ˆD T OT t = 1,..., F (10) F where β 1 and γ 1. The constrants (10) controls the fluctuaton of the dstorton on the sngle frame wth respect to the mean dstorton over the GOP. Adjustng the values β and γ t s possble to control the level of the dstorton fluctuatons. Nevertheless, t s mportant to note that the ntroducton of the constrant (10) does not guarantee that the ILP problem have a soluton for all the values of β and γ. Furthermore, t s expected that the decodng performance wll be reduced n order to satsfy the constrants. The effect of the constrants (10) on the decodng performance wll be analyzed n secton IV. In contrast to Lnear Programmng (LP) problems, whch can be effcently solved, ILP problems are typcally NPhard thus requrng a computatonal tme whch ncreases exponentally wth the problem sze. The analyzed problem s NP-hard. Nevertheless, t has a partcular structure. The matrx assocated to the qualty layer constrants (8) can be shown to be Totally Unmodular (TUM) (a proof s provded n [11]). ILP problems wth TUM constrant matrx and nteger rghthand-sdes can be solved very effcently, snce the optmal soluton of the related LP problem (obtaned relaxng the constrant that varables x are nteger) corresponds to the optmal nteger soluton. However, t has to be noted that the constrants matrx of our problem s not TUM, snce we have the extra budget constrant (9) and eventually the constrants (10) for the dstorton control. Nevertheless, the constrant (9) s a classcal knapsack constrant, that can be effcently managed by common solvers for mxed nteger lnear programmng problems as CPLEX. Dfferent consderatons has to be done for the constrants (10), that, depends on the values of β and γ, could decrease the model resoluton effcency. IV. EXPERIMENTAL VALIDATION To valdate our method we compare t to the approach proposed n [4]. It s worth notcng that the two approaches are appled on dfferent scalable vdeo codec,.e. SVC and the proposed JPEG2000-based vdeo codec. In order to compare the dfferent codng performance of the two systems Fgures 5(a) and 5(b) show the comparson of the two codecs n sngle layer mode,.e. generatng a non-scalable btstream. As t can be notced, SVC has better compresson performance. A deeper nvestgaton shows that ths dfference depends on the features of the vdeo sequence. Two aspects are evaluated related to the SNR extracton performance: the mean PSNR over the frames obtaned extractng sub-btstream at dfferent data-rate from the full SNR scalable btstream, and the PSNR fluctuatons (evaluated as the PSNR standard devaton) between the frames. The PSNR fluctuatons are evaluated snce the method proposed n [4] operatonally starts by dscardng parts of the full scalable btstream from the lower temporal layers. Ths approach enables to maxmze the mean dstorton over the frames but could ntroduce a fluctuaton of the vdeo qualty that generates annoyng vsual artfacts. As prevously descrbed, due to the dscrepancy between the real dstorton and the proposed dstorton models, the rate adaptaton s performed n order to dscard up to the 50% of the full vdeo btstream. As shown n Fgures 5(c) and 5(d), the proposed extracton model enables comparable performance for Soccer sequence, and better performance for Harbour sequence. However, for both the sequences the codng performance n SNR extracton scenaro ncreases compared to the sngle layer scenaro. Furthermore, Fgures 5(e) and 5(f) show as the proposed approach mantans lmted the PSNR fluctuatons compared to [4], especally at hgher btrates. It has to be noted that at lower btrates the extracton performance of the proposed method decreases compared to SVC. Ths not only depends on the lower codng effcency of our codec compared to SVC, but also by the consderaton that the proposed dstorton models became naccurate when many qualty layers have to be dscarded, as descrbed n secton III-A, generatng a non-optmal extracton. Relatvely to the addtonal constrants (10), n Fgure 6 s shown the effect of the constrants on the extracton performance and on the PSNR fluctuatons, where the dstorton fluctuatons have been controlled settng β = 0 and γ = 1.1. Fgure 6(a) shows how the addtonal constrants decrease the extracton performance of approxmately 0, 5dB, but, as shown n Fgure 6(b), wth a consderable reducton of the PSNR fluctuatons. As a fnal remark, t s mportant to note as the computatonal tme of the proposed adaptaton approach s neglgble. For example, n the test reported n Fgure 5 we used a vdeo sequence at 4CIF resoluton (704x576 pxel), a GOP length equal to 8 frames, and we confgured the JPEG2000 encoder n order to have 5 levels of resoluton, precnct sze equal to pxel and 8 qualty layers. Ths leads to approxmately 3000 JPEG2000 packets (equal to the problem sze) for each GOP. For each GOP the optmal allocaton of the rate, obtaned solvng the ILP problem wth the CPLEX verson 8.1 [12], s performed n fractons of seconds, approxmately 2 or 3 tenths of a second. Ths s manly due the specal structure of the descrbed problem. Smlar computatonal tme can be obtaned also ncreasng the problem sze, for example for vdeo at Hgh Defnton (HD) resoluton. Furthermore, even n the test reported n Fgure 6 when the addtonal constrants (10) have been added to the model, the optmal allocaton s performed n smlar computatonal tme. Ths s manly due to the fact that the dstorton threshold γ = 1.1 s not very

7 σ-y-psnr SNR scalable codng wth adaptaton basc ILP model ILP model wth constrants (10) rate [kb/s] (a) Harbour SNR extracton basc ILP model ILP model wth constrants (10) Y-PSNR standard devaton Verson 4: Sept. 2005, Verson 5 and Verson 6: June 2006, Verson 7: Apr. 2007, Verson 8 (ncludng SVC extenson): Consented n July [2] H. Schwarz, D. Marpe, and T. Wegand, Overvew of the Scalable Vdeo Codng extenson of the H.264/AVC standard, IEEE Transacton on Crcut and System for Vdeo Technology, vol. 17, no. 9, pp , [3] T. Thang, J.-G. Km, J. W. Kang, and J.-J. Yoo, SVC adaptaton: Standard tools and supportng methods, Sgnal Processng: Image Communcaton, [4] I. Amonou, N. Cammas, S. Kervadec, and S. Pateux, Optmzed rate-dstorton extracton wth qualty layers n the Scalable extenson of H.264/AVC, IEEE Transacton on Crcut and System for Vdeo Technology, vol. 17, no. 9, pp , [5] J. Sun, G. Wen, D. Zhao, and W. L, On Rate-Dstorton Modellng and Extracton of H.264/SVC Fne-Granular Scalable Vdeo, IEEE Transacton on Crcut and System for Vdeo Technology, vol. 19, no. 3, pp , [6] ITU-T, JSVM 10 software, jvt-w203, Aprl [7] H. Schwarz, D. Marpe, and T. Wegand, Comparson of MCTF and closed-loop herarchcal B pcs, jvt nput document P059, July [8] D. Taubman, Hgh performance scalable mage compresson wth EBCOT, IEEE Transacton on Image Processng, vol. 9, no. 7, pp , July [9] C. Chrstopoulos, A. Skodras, and T. Ebrahm, The JPEG2000 stll mage codng system: An overvew, IEEE Transacton on Consumer Electroncs, vol. 46, no. 1, pp , November [10] D. Taubman and M. Marcelln, JPEG2000: Image Compresson Fundamentals, Standards and Practce. Kluwer Academc, Boston, MA, USA, [11] L. Lma, Scalablty of vsual nformaton for mproved communcaton, PhD thess, Shortly avalable at lvo.lma/. [12] I. INC., ILOG CPLEX 8.1 reference manual, Inclne Vllage: ILOG Inc., CPLEX Dv., (b) Harbour Y-PSNR standard devaton Fg. 6. Analyss of the effect of the constrants (10) restrctve. Decreasng ts value ncreases the computatonal tme requred to solve the model. V. CONCLUSIONS In ths work we propose an effcent method for SNR scalable vdeo adaptaton based on the formulaton of the adaptaton problem as an Integer Lnear Programmng problem and successfully appled to a JPEG2000-based scalable vdeo codec. The proposed approach shows two very nterestng features. Frst, t provdes a comparable performance wth respect to the adaptaton method used for SVC. Secondly, the TUM property of part of the constrants matrx of the proposed ILP problem can be exploted to fnd effcent approxmated solutons (for nstance, by means of Lagrangan Relaxaton) to more complex adaptaton problems where addtonal constrants such as a further control on the dstorton fluctuaton are ntroduced. REFERENCES [1] Advanced vdeo codng for generc audovsual servces, ITU-T Rec. H.264 and ISO/IEC (MPEG-4 AVC), ITU-T and ISO/IEC JTC 1, Verson 1: May 2003, Verson 2: May 2004, Verson 3: Mar. 2005,

8 42 Sngle layer codng 42 Sngle layer codng SVC proposed codec 36 SVC proposed codec rate [Kb/s] rate [Kb/s] (a) Soccer sngle layer (b) Harbour sngle layer SNR scalable codng wth adaptaton SNR scalable codng wth adaptaton proposed extracton SVC MGS wth QL 39 proposed extracton SVC MGS wth QL rate [kb/s] (c) Soccer SNR extracton rate [kb/s] (d) Harbour SNR extracton 4 Y-PSNR standard devaton 4 Y-PSNR standard devaton 3 3 σ-y-psnr 2 σ-y-psnr 2 1 proposed extracton SVC MGS wth QL 1 proposed extracton SVC MGS wth QL (e) Soccer Y-PSNR standard devaton (f) Harbour Y-PSNR standard devaton Fg. 5. Analyss of the extracton performance