International Conference on Image Processing 22 Video Coding with Motion Compensation for Groups of Pictures Markus Flierl Telecommunications Laboratory University of Erlangen-Nuremberg mflierl@stanford.edu Bernd Girod Information Systems Laboratory Stanford University bgirod@stanford.edu
Motivation Today s video coding schemes utilize DPCM with MCP. How about motion-compensated 3-d transform coding? This talk... provides an analysis based on a power spectral model. ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 1
Overview Coding scheme for a group of pictures Model for a group of motion-compensated pictures Model assumptions Performance measure Performance and impact of residual noise Comparison to motion-compensated prediction ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 2
Coding Scheme for a Group of Pictures s c y Intra-frame Encoder z s 1 c 1 y 1 Intra-frame Encoder z 1 MC KLT... s K c K y K Intra-frame Encoder z K ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 3
Motion Compensation for a Group of Pictures s s 1 s 2 s 3 6 6 6 6 Motion Compensation Alignment c c 1 c 2 c 3 6 6 6 6 ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 4
Model for a Group of Motion-Compensated Pictures v n c + n1 y v k model picture k-th displacement error + 1 c 1 y 1 n k k-th noise signal KLT... c k y k k-th motion-compensated signal k-th transform signal n K K c K + y K Codec has the freedom to determine its own reference picture! ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 5
Basic Model Assumptions Model Picture Residual Noise Displacement Error Bandlimited version of a 2- d signal with exponentially decaying and isotropic autocorrelation function. Characterized by the PSD Φ vv (ω) with variance σ 2 v = 1. White noise with variance σ 2 n and PSD Φ nn(ω). Normal distributed and isotropic with variance σ 2 and characteristic function P (ω, σ 2 ). Φ vv (ω) 6 5 4 3 2 1 Φ nn (ω) 1.8.6.4.2 P(ω) 1.8.6.4.2 1.5 ω y /π.5.5 ω x /π.5 1 1.5 ω y /π.5.5 ω x /π.5 1 1.5 ω y /π.5.5 ω x /π.5 1 ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 6
Assumptions about Displacement Errors 2-d stationary normal distribution with variance σ 2 each motion-compensated picture and zero mean for x- and y-components are statistically independent Each displacement error pair is assumed to be jointly Gaussian with no preference among the K motion-compensated signals K K covariance matrix of a displacement error component: C x x = C y y = σ 2 1 ρ ρ ρ 1 ρ...... ρ ρ 1 Covariance matrix is nonnegative definite: 1 1 K ρ 1 K>1 ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 7
Displacement Inaccuracy and Correlation ρ, σ 2 c c 3 Absolute displacement inaccuracy σ 2 σ 2 σ 2 = E { x 2 µ } ρ, σ 2 v ρ, σ 2 σ 2 σ 2 Relative displacement inaccuracy c 1 c 2 ρ, σ 2 σ 2 = E { ( x µ x ν) 2} ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 8
Performance Measure Rate difference for each picture k R k = 1 4π 2 π π π π 1 2 log 2 ( ) Φyk y k (ω) dω Φ ck c k (ω) Measures maximum bit-rate reduction Compared to optimum intra-frame encoding For the same mean squared reconstruction error For Gaussian signals Average rate difference R = 1 K K k= R k ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 9
Performance with Negligible Residual Noise K=2 K=8 K=32 K= rate difference [bit/sample] 2 3 4 Calibration: β = 1 2 log 2(12σ 2 ) Accuracy: β = : IP β = : HP β = 2 : QP 5 6 4 3 2 1 2 displacement inaccuracy β Identical absolute and relative displacement inaccuracy ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 1
Performance with Residual Noise -3 db.5 K=2 K=8 K=32 K= rate difference [bit/sample].5 2 2.5 3 4 3 2 1 2 displacement inaccuracy β Identical absolute and relative displacement inaccuracy ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 11
Motion-Compensated Prediction model picture v n original picture + s e + residual noise residual picture n1 1 + c Wiener Filter displacement error due to inaccurate motion compensation motion-compensated picture ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 12
Comparison to Motion-Compensated Prediction I rate difference [bit/sample] 2 3 4 K=2 K=8 K=32 K= MCP 5 6 4 3 2 1 2 displacement inaccuracy β Absolute and relative displacement inaccuracy are identical, residual noise level -1 db ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 13
Comparison to Motion-Compensated Prediction II rate difference [bit/sample].5.5 2 K=2 K=8 K=32 K= MCP 2.5 3 4 3 2 1 2 displacement inaccuracy β Absolute and relative displacement inaccuracy are identical, residual noise level -3 db ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 14
Summary and Conclusions Model for a group of motion-compensated pictures with correlated displacement error Motion-compensated pictures are decorrelated by the Karhunen-Loeve Transform Results of the analysis: 1. Without residual noise, the slope of the rate difference achieves up to 1 bit/sample and displacement inaccuracy step 2. Residual noise limits the gain by accurate motion compensation Comparison to motion-compensated prediction: 1. The transform model outperforms MCP with optimum Wiener filter by at most.5 bit/sample 2. The performance for a group of K =8pictures is comparable to MCP ICIP 22 Markus Flierl, Video Coding with Motion Compensation for GOP 15