Encoder and Decoder Optimization for Source-Channel Prediction in Error Resilient Video Transmission

Transcription

1 Encoer an Decoer Optmzaton for Source-Channel Precton n Error Reslent Veo Transmsson Hua Yang an Kenneth Rose Sgnal Compresson Lab ECE Department Unversty of Calforna Santa Barbara, USA

2 Outlne Backgroun Source-channel precton (SCP) encoer The matche SCP ecoer Iteratve SCP coec esgn Smulaton results Summary + recent evelopments Oct ICIP

3 Backgroun Exstng error reslent approaches to moton compensate precton Multple precton threas Slcng Long-term memory moton compensaton, mult-frame moton compensaton An more Source-channel efforts manly focuse on moe selecton, unequal error protecton Oct ICIP

4 Backgroun Common to moton compensate precton technques: Do not account for channel errors Encoer reconstructon-base precton Moton estmaton mn mv f n f ˆ 2 + mv ( ) n 1 MB Precton resue e n = f n f ˆ +mv n 1 For pxel n frame n: f n, ˆ f n : Orgnal an encoer reconstructon values e n : Precton error Oct ICIP

5 Source-Channel Precton Propose: Jont source-channel approach Precton base on the expecte ecoer reconstructon Moton estmaton mn mv MB E ( f n f +mv ) 2 n 1 { } Precton resue e n = f n E +mv { f n 1 } For pxel n frame n: f n Decoer reconstructon value (ranom varable for the encoer) Oct ICIP

6 Source-Channel Precton Crucal: accurate estmaton of en-to-en quanttes Unlke more forgvng case of Inter/Intra moe selecton Bul on the technque: Recursve optmal per-pxel estmate (ROPE) [Zhang et al. 00] Orgnally propose for optmal moe ecsons E f {}= n 1 p { } ( ) ( e ˆ n + E f +mv n 1 )+ p E ên Quantze precton error p Packet loss rate Smlarly for secon moments { f n 1 } Oct ICIP

7 Matche SCP Decoer Conventonal ecoer (assume by SCP encoer): If packet receve: If packet lost: ~ f ~ n = ~ f ~ fn = fn 1 + mv n 1 + eˆ n Msmatch! Problem: SCP encoer reconstructon: f ˆ n = E f +mv { n 1 }+ ˆ e n Hence value of quantze resual s compromse Gven p we propose a matche SCP ecoer Oct ICIP

8 Matche SCP Decoer The formulaton: If packet receve: If packet lost: f n = E +mv { f n 1 } + e ˆ n E f { n } = 1 p f n = f n 1 ( ) ˆ E f { n } = 1 p ( ) f n 1 { } { } ( e n + E f +mv ) + p E n 1 f n 1 + p E { f n 1 } Note error concealment effect, cannot reprouce E {} f n : Decoer s emulaton of encoer s E { f n } E f +mv { n 1 } + ˆ e n Oct ICIP

9 Iteratve Desgn We may now revst the SCP encoer, an n turn ecoer Iterate SCP coec optmzaton: Conventonal encoer Conventonal ecoer SCP encoer (1 st ) SCP ecoer (1 st ) SCP encoer (2 n ) SCP ecoer (2 n ) Oct ICIP

10 Iteratve Desgn Unfortunately, complexty grows wth atonal esgn teratons. More correlaton terms nvolve (etals omtte) Results show: One complete roun of SCP optmzaton offers sgnfcant gans. Multple re-optmzaton teratons prove only mnor atonal gans, whle ncurrng a conserable ncrease n complexty. Oct ICIP

11 Smulaton Results Smulaton contons UBC H.263+ coec System performance measure: average lumnance PSNR 50 fferent packet loss patterns Testng scenaros Peroc Intra upatng: an MB s Intra coe once per 1/p frames Optmal Intra upatng: RD optmze Intra moe selecton usng ROPE. Oct ICIP

12 Smulaton Results Teste versons SP coec: conventonal encoer + conventonal ecoer (sourcebase precton) SCP1 enc: SCP encoer + conventonal ecoer SCP1 coec: SCP encoer + SCP matche ecoer SCP2 enc: 2 n -roun optmze SCP encoer + SCP matche ecoer SCP2 coec: 2 n -roun optmze SCP encoer + 2 n -roun matche SCP ecoer Oct ICIP

13 Smulaton Results Peroc Intra SP coec SCP1 enc SCP1 coec SCP2 enc SCP2 coec PSNR (B) Packet loss rate (%) Foreman: PSNR vs. packet loss rate. QCIF, 10f/s, 48kb/s, 1 st 200frames. Oct ICIP

14 Smulaton Results Peroc Intra SP coec SCP1 enc SCP1 coec SCP2 enc SCP2 coec PSNR (B) Packet loss rate (%) Carphone: PSNR vs. packet loss rate. QCIF, 10f/s, 48kb/s, 1 st 200frames. Oct ICIP

15 Smulaton Results Peroc Intra PSNR (B) SP coec SCP1 enc SCP1 coec SCP2 enc SCP2 coec Total bt rate (kb/s) Foreman: PSNR vs. total bt rate. QCIF, 10f/s, p=0.1, 1 st 200frames. Oct ICIP

16 Smulaton Results Peroc Intra PSNR (B) SP coec SCP1 enc SCP1 coec SCP2 enc SCP2 coec Total bt rate (kb/s) Carphone: PSNR vs. total bt rate. QCIF, 10f/s, p=0.1, 1 st 200frames. Oct ICIP

17 Smulaton Result Optmal Intra SP coec SCP1 enc SCP1 coec PSNR (B) Packet loss rate (%) Foreman: PSNR vs. packet loss rate. QCIF, 10f/s, 48kb/s, 1 st 200frames. Oct ICIP

18 Smulaton Result Optmal Intra SP coec SCP1 enc SCP1 coec PSNR (B) Packet loss rate (%) Carphone: PSNR vs. packet loss rate. QCIF, 10f/s, 48kb/s, 1 st 200frames. Oct ICIP

19 Smulaton Results Optmal Intra PSNR (B) SP coec SCP1 enc SCP1 coec Total bt rate (kb/s) Foreman: PSNR vs. total bt rate. QCIF, 10f/s, p=0.1, 1 st 200frames. Oct ICIP

20 Smulaton Results Optmal Intra PSNR (B) SP coec SCP1 enc SCP1 coec Total bt rate (kb/s) Carphone: PSNR vs. total bt rate. QCIF, 10f/s, p=0.1, 1 st 200frames. Oct ICIP

21 Summary Source-channel precton: The ROPE-base SCP encoer an ts matchng ecoer Iteratve SCP esgn One complete teraton of SCP esgn offers sgnfcant gans. Smulatons suggest that multple-roun optmzaton s not cost-effectve - offers mnor atonal gans, an nvolves a conserable complexty ncrease. Oct ICIP

22 Recent Developments There seems to be a way to obtan non trval gans from more esgn teratons at low complexty by changng the conventonal concealment rule at the matche ecoer Work n progress to ncorporate SCP moton compensaton/precton wthn overall rate storton framework at moerate complexty. Oct ICIP

23 Thanks! Oct ICIP

24 The 2 n -Roun Optmze SCP Coec The encoer: assumng the SCP matche ecoer E f {}= n ( 1 p) ˆ E{ E f {} n }= ( 1 p) ( 1 p) ˆ + p ( 1 p) E{ f n 1 }+ p E E ( e n + E{ E f +mv }) + p E n 1 { } [ { }] { f n 1 } ( e n + E{ E f +mv }) + p E E n 1 f n 1 { f n 1 } { } { } { } {} f n s a ranom varable for the encoer. E ( ) 2 E E {} f n nvolves cross-correlaton terms. To estmate them wth tractable complexty, the estmaton accuracy have to be slghtly compromse [Yang 03]. Oct ICIP

25 The 2 n -Roun Optmze SCP Coec The matche ecoer ( ) ˆ If packet receve: E f {} n = 1 p e n {{ f +mv n 1 } } + E E + p E f { n 1 } { } = 1 p E E{} f n + p ( 1 p) E If packet lost: ( ) ( 1 p) ˆ e n { f n 1 } + p E E f n 1 E f { n } = 1 p { } = 1 p E E{} f n + p ( 1 p) E { { f +mv n 1 } } + E E {{ } } ( ) X 2 + p E ( ) ( 1 p) X 2 + p E E { f n 1 } + p E E f n 1 + p E E { f n 1 } { { f n 1 } } {{ } } f { { n 1 } } Oct ICIP

26 The 2 n -Roun Optmze SCP Coec The matche ecoer (cont.) For error concealment of : ˆ e n + E E { f +mv n 1 } { } X 2 = E{ E f } n 1 { f } n Decoer s emulaton of encoer s E E E E {} { f n } { } { } Smlarly, more rouns of SCP optmzaton ncur more complexty, esp. for the encoer, as more cross-correlaton terms wll be nvolve. Oct ICIP