Digital Video Systems ECE 634

Transcription

1 Digital Video Systems ECE 634 egieerig.purdue.edu/~reibma/ece634/idex.html Professor Amy Reibma MSEE 356

2 MulGple descripgo codig Layered codig vs. MDC Layered codig: Base layer is high priority; caot recostruct video without base layer. Ehacemet layer is bous MulGple descripgo codig: either of the two descripgos are OK; both are berer Most) mulgple descripgo codig are ot exactly compagble with stadards ParGGoig the video ad reassemblig ca somegmes) be doe outside the cotext of the stadards

3 CosideraGos for MD codig Compressio efficiecy Decoder complexity Resiliece to losses Flexible rate adaptago Number of descripgos Ca you add redudacy flexibly? CompaGbility with stadards Ease of priorigzago PredicGo structure ad method of parggoig cotrols most of these! 3

4 MulGple DescripGo Codig MDC: Geerate mulgple correlated descripgos Ay descripgo provides low but acceptable quality AddiGoal received descripgos provide icremetal improvemets However: correlago à reduced codig efficiecy AssumpGos: MulGple chaels betwee source ad desgago Idepedet error ad failure evets Probability that all chaels fail simultaeously is low All are reasoable assumpgos for the Iteret ad wireless etworks, provided data are properly packegzed ad iterleaved Desig goal: maximize the robustess to chael errors at a permissible level of redudacy Wag Orchard Reibma 4

5 Geeric Two DescripGo Coder Source Sigal Balaced MDC: R=R2, D=D2 MDC Ecoder S R) S2 R2) Chael Chael 2 Decoder Decoder Decoder 2 MDC Decoder Decoded Sigal from S D) Decoded Sigal from S,S2 D) Decoded Sigal from S2 D2) Wag Orchard Reibma 5

6 Four approaches to MDC DuplicaGo! Iterleaved temporal samplig MulGple DescripGo Scalar QuaGzer Basic idea: Use iterleavig quagzers of a scalar value Pairwise CorrelaGg Trasforms Basic idea: Use NxN liear trasform to geerate two groups of coefficiets with a desired amout of correlago, so that oe group ca be esgmated from the other with a give accuracy. Coceptually simple method: Use a decorrelagg trasform followed by a pairwise correlagg trasform

7 Video Redudacy Codig i H.263+ Code eve frames ad odd frames as separate threads Good video quality at packet loss rates up to 2% High redudacy ~3%) due to reduced predicgo gai because of loger distace betwee frames Hard to vary the redudacy based o chael loss characterisgcs eve frames odd frames Yao Wag, 24 7

8 Three- thread three- pictures- per- thread VRC frame is syc frame; appears i all threads frame is i thread, predicted from frame 2 is i thread 2, predicted from frame 3 is i thread 3, predicted from frame 4 is i thread, predicted from frame 5 is i thread 2, predicted from 2 frame 6 is i thread 3, predicted from 3 frame 7 is syc frame; appears i all threads

9 MulGple descripgo Scalar QuaGzaGo Basic idea: Use iterleavig scalar quagzers a cetral quagzer whe both descripgos received a side quagzer whe oly oe descripgo received cetral quagzer side quagzer side quagzer 2 Very elegat, but their desig is complicated

10 MDSQ Idex assigmet QuaGze oce with desired quagzer The assig each bi a idex i each chael cetral quagzer side quagzer side quagzer idex assigmet

11 Some MDSQ Idex assigmets Less efficiet More robust More efficiet Less robust sigle- chael recostrucgo error ca be quite large)

12 Pairwise correlagg trasforms Basic idea: Geerate two sets of trasform coefficiets, oe for each descripgo, such that each set is ucorrelated withi the set, but correlated pairwise across sets Take output of a decorrelagg trasform KLT), ad recorrelate pairs of coefficiets; assig each coefficiet of a pair to a differet descripgo At decoder: if a coefficiet is missig, esgmate it usig its pair

13 Codig of a Sigle Pair A B Estimator for A,B from C ~ A C ~ Q Chael Q - Forward Iverse Q PCT Chael 2 PCT Q - ~ ~ B D 2 Variable MDC Ecoder Q - Q -!C!D Estimator for A,B from D!A!B!A!B!A!B 2 Variable MDC Decoder ICIP98 3 Wag Orchard Reibma

14 MDC Usig Pairwise CorrelaGg Trasform F F 2 F 3 F 4 F 5 KLT DCT) T T 2 T 3 T 4 Pairig A B A 2 B 2 Forward PCT Forward PCT C D C 2 D 2 Stream T 5 T6 F 6 F 7 F 8 T 7 T 8 Stream 2 2 σ D,max, 5 Wag Orchard Reibma ICIP98 4

15 Let Performace evaluago of MD coders DefiiGos: DistorGo with both descripgos: Average distorgo with oe descripgo: Sigle descripgo coder RD) fucgo: MulGple descripgo coder rate- distorgo: Redudacy Rate- Distor3o : To decrease D oe must itroduce correlago betwee the two descripgos ad thereby reduce the codig efficiecy * * * * ρ = R R, whe D R ) = D R) = D Redudacy Rate - Distortio Fuctio: D D * * * * D R ), D R ) * * ρ D ; D ) or D ρ; D ) D R), D R) ICIP98 5 Wag Orchard Reibma

16 Redudacy Rate DistorGo D Rate-Distortio RD) Fuctio D Redudacy Rate Distortio RRD) Fuctio D * D R ) MDC) D * ρ; D ) ρ D * R) SDC) R * R R ρ Desig criteria for MD coders Miimize D for a give ρ, for fixed R* or D * miimizig the average distortio give chael loss rates, for give total rate) Ca easily vary the ρ vs. D trade-off to match etwork coditios Wag Orchard Reibma 6

17

18

19

20

21 MD video quesgos How should the followig elemets of a hybrid video coder be adapted for a mulgple descripgo video coder? Side iformago MB type) duplicate) MoGo vector iformago duplicate) PredicGo error compared to two- chael recostrucgo) pairwise correlagg trasforms) Mismatch betwee two- ad oe- chael recostrucgos) extra predicgo- error sigals) Wag Orchard Reibma ICIP99 2

22 Core MDTC- based video coder X F DCT Q PCT Chael Chael 2 P IDCT IQ IPCT FM Apply Pairwise CorrelaGg Trasform PCT) to DCT coefficiets iside mai loop of hybrid video coder Mismatch if both chaels are ot received Mismatch for this frame is limited to the orthogoal subspace, but mogo compesago may spread it Wag Orchard Reibma ICIP99 22

23 MulGple DescripGo MoGo CompesaGo Wag ad Li, 2) A descripgo cotais eve or odd) frames oly, but each frame is predicted cetral predictor) from both eve ad odd past frames Code the cetral predicgo error sufficiet if both descripgos are received To avoid mismatch, a side predictor for eve frames predicts oly from the past eve frame, ad the mismatch sigal differece betwee cetral ad side predicgo) is also coded The quagzago of the predictors ad the mismatch error cotrols the redudacy of the coder, ad ca be desiged based o the chael loss characterisgcs Yao Wag, 24 23

24 Yao Wag, Special Case: Two-Tap Predictor, No - leaky predictor :,MV,MV2, ~ ~ Sed : ~ ) ˆ ) ˆ : Mismatch error 2) ~ ) ˆ Side predictor : ~ ) ˆ ) : predictio error Cetral 2) ~ ) ~ ) ˆ predictor : Cetral = = + = = = + = a a a e ) ) e e ) ) q e ) a ) e ) e a a ) ) ~ ~ ) ˆ ) 2)) received have oly oe descriptio is If ) ) ~ ) ˆ ) 2)) ), both descriptios received have both If q e ) ) e q ) e + = + + = + = + = a, MV a 2,MV2 a 3,MV2

25 RRD Performace of VRC ad MDMC Yao Wag, 24 Error Cotrol 25

26 Performace i Packet Lossy Networks Yao Wag, 24 26

27 Sample Recostructed Frames % Radom Packet Loss, MDMC o top, VRC o borom) Yao Wag, 24 Error Cotrol 27

28 Summary: Challeges i Desigig MD Video Coder To achieve high codig efficiecy, the ecoder should retai the temporal predicgo loop PredicGo strategies are key to cotrol trade- off betwee added redudacy ad reduced compressio efficiecy Predict from two- descripgo recostrucgo, or oe? PredicGo based o two- descripgo recostrucgo Higher predicgo efficiecy Mismatch problem at the decoder PredicGo based o sigle- descripgo recostrucgo Lower predicgo efficiecy No mismatch problem Oe desig strategy Predict based o two- descripgo recostrucgo, but explicitly code the mismatch error Yao Wag, 24 28

29 Refereces Y. Wag ad Q. Zhu, Error cotrol i video commuicatios A review, Proc. IEEE, 998 Y. Wag, A. R. Reibma, ad S. Li, Multiple descriptio codig for video delivery, ivited paper, Proc. IEEE, Ja