A VC-1 TO H.264/AVC INTRA TRANSCODING USING ENCODING INFORMATION TO REDUCE RE-QUANTIZATION NOISE

Transcription

1 A VC- TO H.6/AVC INTRA TRANSCODING USING ENCODING INFORMATION TO REDUCE RE-QUANTIZATION NOISE Takeshi Yoshitome, Yoshiyuki Nakajima, and Kazuto Kamikura NTT Cyer Space Laoratories, NTT Corporation, Yokosuka-shi, 39-07, JAPAN Shoji Makino and Nouhiko Kitawaki Graduate School of Systems and Information Engineering University of Tsukua Tsukua, Japan, -73, JAPAN ABSTRACT We propose a VC- to H.6/AVC intra transcoding method. This method uses the encoding information from a VC- stream and keeps as many coefficients of the original VC- itstream as possile. Experimental results show that the proposed method improves PSNR y aout db compared with a conventional method. KEY WORDS Transcoding, VC-, H.6, Quantization noise, Encoding information Introduction Recently, it has ecome important to re-compress video contents to decrease itstream size y using a higher performance encoding tool such as H.6[]. In this decade, a numer of video transcoding studies have een reported [, 6, 7] that aim to reduce the H.6 encoding time, which is several times larger than that of old compression tools, such as MPEG-[] and VC-[3]. For example, [, 9] have reported that the use of an MPEG- motion vector and compression mode information helps to reduce the processing time of H.6 re-encoding. A direct mathematical matrix conversion from the x real of MPEG- to the x integer of H.6 has also een reported[0]. However, the image quality otained with these approaches ecomes lower than that of conventional approaches using a directly connected MPEG- or VC- decoder and H.6 encoder. We have focused on the fact that all decisions of the first encoding procedure are repeated during the transcoding procedure to reduce re-quantization noise[], where all decisions means decisions on picture type, macrolock type, motion vectors, type, and so on. This noise suppression mechanism is effective when the compression standards of the first and second encoding are identical, such as for MPEG-/MPEG- or H.6/H.6 conversion. We have tried to expand this approach to apply it to MPEG-/H.6 transcoding for progressive itstreams[, ], ecause the image quality otained with this approach may ecome higher than that of conventional method mentioned aove. This paper reports our attempt to apply this noise suppression mechanism to VC- intra it-streams. First, we discuss the noise suppression mechanism for progressive it-streams. Next, we show the differences etween the VC- and H.6/AVC standards, analyze how they affect the noise reduction, and propose a VC- to H.6/AVC transcoding method. Finally, we descrie simulation results we otained and compare the transcoding noises generated y our method and a conventional method. Noise reduction mechanism using first encoding information Data flow examples of transcoding with and without first VC- encoding information are shown in Fig.. The data flow of Enc Dec Enc Dec is with encoding information, and that of Enc Dec Enc3 Dec3 is without it. For simplicity, the macro-lock (MB) size is x. In the first VC- encoding, an input MB that includes the pixel values (A, B, C, and D) is processed y intra prediction and orthogonal transformation. Then, the transformed coefficients (3, 3,, and 3) are quantized y a quantization step =0. The quantized coefficients (, 3,, and ) are converted to itstream# y variale length coding (VLC). The quantization from (3, 3,, and 3) to (, 3,, and ) adds quantization noise so that the decoded MB pixels (a,, c, and d) differ from the input MB pixels (A, B, C, and D). Until this point, transcoding oth with and without encoding information does the same work. In the transcoding without encoding information, Enc3 searches the est motion vector to minimize motion estimation error (p, q, r, and s ). If the motion search performance of Enc3 is higher than that of Enc, Enc3 can reduce its motion estimation error (p, q, r, and s ) compared with Enc s motion estimation error (A, B, C, and D ). In the same way, if the orthogonal transformation performance of Enc3 is higher than that of Enc, Enc3 can reduce the size of itstream#3 compared with that of itstream#. However, the re-quantization noise occurs when coefficients (3, 33,, and ) are quantized to (3, 3,, and 0). In short, Enc3 adds re-quantization noise even if the itstream ecomes smaller. On the other hand, in the transcoding with encoding information, every type of en

2 Enc Dec Enc Dec Enc3 Dec3 Input MB A C a c a c p r B D d d q s ME (left) re-use ME (left) ME (up) Intra-prediction Error A B C D a c d a c d a c d p q r s p q r s (x) re-use (x) (x) coefficient Q ( =0) re-use Q ( =0) Quantized coefficient No quantization noise Q ( =0) Quantization noise VLC Bit stream # VLD Almost same VLC Bit stream # Transcoding with Enc s info. : Enc+Dec + Enc+Enc Transcoding without Enc s info. : Enc+Dec + Enc3+Dec3 VLD VLC Bit stream #3 VLD Tale. Major differences etween VC- and H.6 for intra encoding VC- H.6 H.6 re-enact Intra prediction x x OK lock size x Direction left directions of intra up without? prediction no-pred. no-pred. Orthogonal unique integer? transformation Size of orthogonal x x(*) OK transformation x Specification of no yes(*) OK quantization matrix Interval of linear logarithmic? quantization step Deadzone in yes/no no? quantization (*) (*): only high-profile (*): depends on encoder setting Figure. Transcoding with and without first VC- encoding information coding information, such as MB type, motion vector, mode, and quantization step, is re-used in Enc. This re-use of encoding information does not produce any significant difference etween motion compensation errors of the first encoding and those of the second encoding, nor does it produce any notale difference etween the coefficients of the two encodings. This means that the re-quantization noise added in the second encoding phase is almost zero. As a result, the same motion estimation error and the same quantization coefficients are otained. Unfortunately, there is no it reduction ecause the sizes of itstream# and # are almost the same when these encodings make use of the same compression tool. But transcoding using different compression tools, such as VC- to H.6 transcoding, may reduce the itstream size ecause the performance of H.6 s entropy coding (CABAC) is higher than that of VC- s entropy coding (VLC). 3 Differences etween VC- and H.6 As depicted in Fig., there is a close resemlance etween the decoded images of the first and second decoders when the compression standards of the first and second encodings are exactly the same. To re-enact the effect of this noise reduction mechanism, we must control H.6 ehavior to imitate VC- ehavior as much as possile when the first encoder is VC- and the second encoder is H.6. The two encoders are similar ut are not upper compatile. The differences etween them are listed in Tale. The ok means that H.6 can recreate the same VC- function in the tale. The? means that H.6 can process the function only in a similar ut not identical way that VC- does. The following four functions are not exactly the same etween VC- and H.6, and are not investigated: () Intra prediction, () Availale quantization step, (3) Orthogonal transformation, () Quantization with and without deadzone. In the next section, we descrie these differences in detail and show a way to overcome them. 3. Intra prediction There are three intra prediction modes, including the no prediction mode, in the VC- specification. Adjoining pixels in the upper or left locks are referred to in this prediction. H.6 has eight prediction angles to increase compression efficiency. The two prediction modes (mode=up and mode=left) in VC- can e converted to the mode=0 and mode= in H.6, as shown in Fig.. But it is impossile to exactly equalize the VC- and H.6 intra locks when VC- selects the no prediction mode ecause H.6 does not have this mode. The est way to imitate this mode is to use the DC prediction mode in H.6. If this is used, the orthogonal transformation will change only the DC coefficient. None of the coefficients except the DC one are affected y the difference etween the VC- and H.6 predictions. Intra prediction mode mapping from VC- to H.6 is shown in Tale. 7

3 pixcels Tale. Left No prediction VC- Up 6 0 H mode :DC prediction Figure. Intra x prediction of VC- and H.6 Intra prediction mode mapping from VC- to H.6 VC- prediction mode H.6 prediction mode Up 0 Left No prediction (DC) 3. Availale quantization step size The derivation manner from the quantization code to the quantization step in VC- is similar to that of MPEG- when the parameter q scale type is equal to zero in MPEG-. The quantization step equals the quantization code in MPEG-. The quantization step equals twice the quantization code in VC-. Additionally, VC- can also use the quantization steps, (3,, 7,..,, 7 ) using its it parameter, HALFQP, which increases the quantization step Tale 3. Availale in VC- and in H.6 H.6 VC- H.6 VC- 0.6 O - 0 O O 0.67 O - O O 0. O - - O 0.7 O - 6 O O O - O O. O - - O. O - 3 O O.37 O O.6 O - O O.7 O O O O 0 O O. O - - O. O - O O.7 O O 3 - O - O 3. O O 3. O - O O O O - O. O - 6 O O O O - O. O O 6 - O 6 - O 6. O - 6 O - 7 O O 7 O - O O 0 O - 9 O O O - 0 O O 0 O - O O O - - O O - 3 O O O - O O 60 O - - O 76 O - 6 O O 0 O O O - O O y one. Thus, there are 39 availale quantization steps in VC-. In contrast, there are availale quantization steps in H.6. The minimum value is 0.6 and the maximum value is. The ratio of the two proceeding quantization steps is an almost constant value in H.6. Availale quantization steps in VC- and in H.6 are shown in Tale3. value of O means there is availaility. There are cases in which VC- can specify and H.6 can not. In these cases, the nearest quantization step of H.6 is used in place of that of VC-. The ratio difference etween the VC- and H.6 quantization steps is from 3.% to.3%. The negative effects caused y this difference will e descried in detail in Section.. The amount of this difference is within the acceptale level. 3.3 Orthogonal transformation In VC- intra compression, the orthogonal transformation which has x size is used. The transformation, and the quantization and de-quantization, are given in the following Eqs. () and () : Y = (V X V t ) N () Y d = Dequant vc ( Quant vc (Y )) () The inverse orthogonal transformation in VC- is descried in Eqs. (3) and (): E = (Y d V + ) >> 3 (3) ˆX = V t E + C () where X is the input image, Y is the transformed coefficient, N is the normalization matrix, the operator V t represents a transposition of matrix V, the operator >> represents an arithmetic right shift performed y entrywiseon a matrix, the operator represents matrix multiplication, the operator is a component-wise multiplication, Y d is the de-quantized transformed coefficient that was quantized, ˆX is the decoded image, V is the transfer matrix, Dequant vc () and Quant vc () are the quantization and de-quantization functions of VC-, V t is the transposed matrix of V, E is an intermediate matrix, C = ( ) t, and is an eight length row vector of ones. The (i,j) component of V is descried in Eq. as follows: V = () The (i,j) component of V is descried as follows: where c= ( 9 N ij = c j c t i ) 9 7

4 0 A VC- B VC- C 0 A VC- B H.6(x) C 0 A VC- B H.6(x) C 0 PSNR(A,C) 0 PSNR(A,C) 0 PSNR(A,C) PSNR(B,C) PSNR(A,B) PSNR(A,B) PSNR(B,C) PSNR(A,B) Step size Figure 3. PSNR of VC- /VC- re-encoding Step size Figure. PSNR of VC-/H.6 (H=x) re-encoding Step size Figure. PSNR of VC-/H.6 (H=x) re-encoding In H.6, if we assume that Z is the integer output and H is the integer matrix, Dequant H6 () and Quant H6 () are the quantization and de-quantization functions of H.6, and Z d is the de-quantized integer coefficient that was quantized. Then the integer, the quantization and de-quantization, and the inverse integer are descried in Eqs. (6), (7), () elow: Z = HXH t (6) Z d = Dequant H6 ( Quant H6 (Z)) (7) H with N= is shown in Eq. (9). H = 0 0 ˆX = H t Z d H () 0 0 H with N= is shown in Eq.(0). a a a a a a a a 3c c 3c 3c 3c c c H = 3c 3c c a a a a a a a a 3c 3c c c 3c 3c c 3c c 3c where a = 3, =, c = 9 (9) (0) To compare the 6 components of Y and Z, an example image is processed y two consecutive stages using mathematical operations. In the first stage, the image is processed y the VC- s transformation, the quantization and de-quantization, and the VC- s inverse transformation in Eq.() - Eq(). In the second stage, the output image of the first stage is processed y following the different functions.. VC-Trans/quantize/de-quantize/VC-InverseTrans (Eq.-Eq., PSNR is shown in Fig. 3). xint-/quantize/de-quantize/xint-i (Eq.6- with N=. PSNR is shown in Fig. ) 3. xint-/quantize/de-quantize/xint-i (Eq.6- with N=. PSNR is shown in Fig. ) In the second stage, the quantization and dequantization use the same quantization step used in the first stage. The quantization and de-quantization without deadzone are used ecause only the difference etween the orthogonal transformations of VC- and H.6 reflects the PSNR results. From Fig. 3 to Fig., A means the input image, B means the decoding image of the first encoding, C means the decoding image of the second encoding, and PSNR (A, B) means the PSNR value etween A and B. Figure 3 shows that PSNR (A, B) equals PSNR (A, C). This indicates that the repetition of orthogonal transformations of VC- does not decrease the PSNR, as mentioned in Section. We omitted PSNR (B, C) from Fig. 3 ecause the re-quantization noise showed the lowest value and PSNR (B, C) rose to over 00 db. When the second encoding is the x integer of H.6 (Eq.(6) - Eq.(), N=), the PSNR (A, B) is slightly higher than the PSNR (B, C) shown in Fig., ecause the amount of quantization noise of the 73

5 second encoding is slightly lower than that of the first encoding. This indicates there is a slight similarity etween the VC- s x transform matrix and the H.6 s x transform matrix. The final PSNR (A, C) is lower than PSNR (A, B) y nearly 3 db ecause the similarity is not large. On the other hand, when the second encoding is the x integer of H.6 (Eq. (6) - Eq. (),N=), PSNR (A, B) < PSNR (B, C) and the difference ecomes larger as ecomes larger, as shown in Fig. 3. This indicates there is a large similarity etween the VC- s x transform matrix and the H.6 s x transform matrix. Figure. also shows that a large decreases the PSNR reduction caused y the difference etween the VC- s x transformation and the H.6 s x transformation. The differences etween PSNR (A, B) and PSNR (A, C) are (.,.,.0, 0., and 0.3) when = (,,, 6, and 3). (a) () X - X = - = X - = -0 X - = X 0 X X = 0 = = X 0 X X = 0 = 0 = Quantized output Quantization in Input Quantized output Quantization in Input Figure 6. Examples of uniform and non-uniform quantization 3. Quantization For intra MB quantization, VC- can use the two different quantization modes shown in Fig. 6. Figure 6 (a) shows a non-uniform quantization and Fig. 6 () shows a uniform quantization. X i means quanization output and i means quanization in. If the input is in i, the quantized output equals X i. In a uniform quantization, the size of quanization in is constant. In a non-uniform quantization, the size of 0 is larger than that of the others. The non-uniform quantization decreases the it-stream size ecause it increases the numer of X 0. The uniform quantization enales accurate quantization, so it is suitale for high itrate encoding. H.6 uses only uniform quantization that has no deadzone. The noise reduction effect, mentioned in Section., decreases when VC- selects the nonuniform quantization. Although the VC- specification allows the use of these two quantization modes in the same intra picture, most VC- it-streams are quantized using the uniform quantization ecause the intra picture needs to e more accurate in terms of encoding quality than the inter picture. The negative effects caused y this uniformity difference etween the first VC- encoding and the second H.6 re-encoding will e descried in detail in Section. The amount of this difference is within the acceptale level. Method The proposed VC- to H.6 conversion rules are listed in Tale. The H.6 high profile enales the use of an x integer that has the same orthogonal transformation size as that of VC-. All coefficients of the H.6 quantization matrix are set to sixteen to equalize the flat VC- s quantization characteristics. The proposed method chooses the H.6 s quantization step whose value is nearest to that of the VC- s quantization step. The intra prediction mode is set to the DC mode to imitate the VC- standard. The H.6 standard can select one of the intra MB modes from among Ix, Ix, and I6x6 and each matrix size Tale. VC-/H.6 intra transcoding rule VC- H.6 Profile simple/main/ high advance Intra up 0 prediction left no pred. (DC) MB type intra Ix Encoder flag setting - transform size flag = Quantization matrix - flat matrix Quantization step - nearest to VC- s step is either x, x, or x. The proposed method uses only the Ix mode ecause its matrix size is the same as that of VC-. Simulation. Conditions We used experimental simulation to compare our proposed method with a conventional method. The conventional method we used is the ordinary VC-/H.6 transcoding method. It does not re-use the first encoding information, and ut does intra MB mode selection. The VC- sample program[] was used for first encoding under the following conditions: The quantization step was fixed in the picture. VC- encoding was done twice for all simulations using uniform and non-uniform quantization. We modified the VC- decoder in ffmpeg[] to extract the first VC- encoding information from the VC- itstream. The jm. was used for the conventional second H.6 encoding under the following conditions: The profile was highprofile, the RD optimization was off, the UseHadamrd was on, and the quantization matrix was the H.6 default setting matrix. The conventional second encoding had no restrictions with regard to selecting MB mode and intra prediction mode, so its output it-streams included the all-mb 7

6 = Input VC-( = ) = Input VC-( = 6) = Input VC-( = 6) = = 6 = 6 Figure 7. PSNR of proposed method and conventional method when VC- uses uniform quantization 3 33 = Input VC-( = ) = Input VC-( = 6) = Input VC-( = 6) = = 6 = 6 Figure. PSNR of proposed method and conventional method when VC- uses non-uniform quantization mode and DC prediction, and also included all availale sizes. We used the modified jm., which can input the first encoding information, for our proposed transcoding method. The quantization step was fixed for the simulations of oth the conventional and proposed methods.. Results for VC- itstreams encoded y uniform quantization The PSNR of transcoding from input VC- itstreams encoded using uniform quantization to H.6 itstreams are shown in Fig.7(a)-7(c). The solid line indicates the proposed method and the dotted line indicates the conventional method. The quantization step of the first encoding,, was set to, 6, or 6. That of the second encoding,, varied from 0.6 to. The x-axis means PSNR and the y- axis means picture its. When input streams were encoded using uniform quantization, Fig. 7 shows that the proposed method showed two peaks when = and =0.. At these points, the PSNR of the proposed method was higher than that of the conventional method from aout db. The PSNR of the proposed method drops when 0. < < and < <.0. This PSNR drop is similar to that reported in MPEG-/MPEG- transcoding. Figure 7(a)-(c) shows that the PSNR of the proposed method is always higher than that of the conven- 7

7 Tale. Comp. ratio and PSNR of proposed method when VC- used uniform quantization Input VC- transcoder Conv. PSNR Image PSNR MB size MB size comp. PSNR PSNR diff. (db) (kb) (kb) ratio (db) (db) (db) Cheer Leader Bus Moile and Calendar Tale 6. Comp. ratio and PSNR of proposed method when VC- used non-uniform quantization Input VC- transcoder Conv. PSNR image PSNR MB size MB size comp. PSNR PSNR diff. (db) (kb) (kb) ratio (db) (db) (db) Cheer Leader Bus Moile and Calendar tional method. As ecomes smaller, the PSNR improvement of the proposed method decreases ecause the noise reduction effect mentioned in Section maximizes when =. As we mentioned in Sec.3., if the first encoding VC- uses the quantization step that is not allowed in H.6, an approximate quantization step must e used in H.6 transcoding. However, the difference etween the true value and the approximation value is as high as.3%. The range in which the proposed method is superior to the conventional method exceeds this.3% error in Fig.7(a) - (c). This indicates that the proposed method is superior to the conventional method even if the approximate quantization step is used in H.6 transcoding. The compression ratio and PSNR of the proposed method when the quantization step is equal to are listed in Tale. The conventional PSNR column is the interpolation value when the conventional method generates the same itstream in terms of size. These interpolation values were calculated using the simulation results of the conventional method. The PSNR of the proposed method is aout db etter than that of the conventional method when the quantization step is equal to. The re-compression ratio of the H.6 itstream size to the VC- itstream size ranges from 0. to Results for VC- itstreams encoded using nonuniform quantization The PSNR values of transcoding from input VC- itstreams encoded using non-uniform non-quantization to H.6 itstreams are shown in Fig.(a)-(c). When the first VC- encoding is done y non-uniform quantization, the PSNR improvement is less than that otained with uniform quantization. Figure shows the PSNR values otained with the proposed method when =θ (θ=.-.3). This shift in the optimum was generated y the difference etween the quantization uniformities of VC- and H.6. This phenomenon was also reported in MPEG- /H.6 inter transcoding research [3]. Despite the difference in quantization uniformity, Fig.(a)-(c) shows that the PSNR of the proposed method is always higher than that of the conventional method. The compression ratio and PSNR of the proposed method when the quantization step is equal to. are listed in Tale.6. At these shifted points, the PSNR of the proposed method is higher than that of the conventional method from aout db. 6 Conclusion We proposed a re-quantization noise reduction method for VC- to H.6 intra transcoding. This method uses encoding information from the VC- stream and keeps as many 76

8 coefficients of the original VC- itstream as possile. We analyzed the difference etween VC- and H.6 regarding the intra prediction and quantization steps and orthogonal transformation to reduce the re-quantization noise in H.6. Experimental results showed that the proposed method improves PSNR y aout db compared with the conventional method. The advantages of the proposed method are that it not only results in high PSNR ut also that it does not require complex calculation to select an MB mode and an intra prediction mode. References [] ISO/IEC IS 3-, ITU-T Recommendation H.6, Generic coding of moving pictures and associated audio information, Nov. 99. [] Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec.H.6/ISO/IEC 6-0 AVC, in Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-G00,003 [] T. Yoshitome, et al, A study of MPEG- to H.6 Intra Transcoding for Progressive Contents ITE Journal, 6,,pp9- (Nov. 00) (in Japanese) [] T. Yoshitome, K. Kamikura, and N. Kitawaki, A Requantization Noise Reduction Method in MPEG- to H.6 Intra Transcoding IEEE Symposium on Industrial Electronics and Application (ISIEA 009), Kuala Lumpur, Malaysia, pp.9-99, Oct -6, 009 [3] T. Yoshitome, et al, An MPEG- to H.6 Transcoding Method Preserving Information for Progressive Contents ITE Journal, 63,6,pp.37-6 (June 009) (in Japanese) [] Intel Integrated Performance Primitives 6. [] [6] Joint Video Team(JCT), Reference Software JM., [7] High vision digital standard image data chart index.html [3] SMPTE, Standard for Television: VC- Compressed Video Bitstream Format and Decoding Process, SMPTE M-006. [] P. Guilotel, et al, Adaptive Encoders: The New Generation of MPEG- Encoders SMPTE journal (April 000) [] J.-B. Lee, and H. Kalva, An Efficient Algorithm for VC- to H.6 Video Transcoding in Progressive Compression, Proc. IEEE International Conf. on Multimedia and Expo, pp.3-6, July 006. [6] M. Pantoja, N. Ling, and W. Shang, Coefficient Conversion for Transform Domain VC- to H.6 Transcoding, Proc. 007 IEEE Workshop on Signal Processing Systems, Shanghai, China, pp. 3-7, Octoer 7-9, 007. [7] M. Pantoja, and N. Ling, Adaptive transform size and frame-field selection for efficient VC- to H.6 high profile transcoding, Proc. 009 IEEE Workshop Circuits and Systems, pp. 7-0, ISCAS 009. [] Xiaoan Lu, et.al Fast mode decision and motion estimation for H.6 with a focus on MPEG-/H.6 transcoding in Proc. of ISCAS 00, 3-6 May 00, pp6-9 Vol. [9] Gao Chen, et.al Efficient lock size selection for MPEG- to H.6 transcoding Proc. of the th annual ACM international conference on Multimedia [0] Joo-Kyong Lee, et.al Quantization/ Conversion Scheme for -domain MPEG- to H.6/AVC Transcoding IEICE Trans. Commun., Vol.E7-B, No.7 July 00 77