2(25) Mean / average / expected value of a stochastic variable X: Variance of a stochastic variable X: 1(25)

Transcription

1 Lecture 5: Codig of Aalog Sources Samplig ad Quatizatio Images ad souds are ot origially digital! The are cotiuous sigals i space/time as well as amplitude Typical model of a aalog source: A statioary Gaussia process t The process ca be memoryless (white) or have memory (coloured) See the power spectral desity (5) Basic Probability Theory (Sayood App A) Probability distributio fuctio (pdf) of a stochastic variable X: P ( a X b) f X x b a ( ) d x Mea / average / expected value of a stochastic variable X: E [ X ] µ X xf X x ( ) d x Variace of a stochastic variable X: Var [ X ] E [ X µ X ] ( x µ X ) f X ( x ) d x (5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 The Gaussia (ormal) distributio: X is a Gaussia stochastic variable with mea µ ad stadard deviatio σ (variace σ ): f X ( x ) X ~ N ( µ, σ) ---- e πσ ( x µ ) -- σ The auto correlatio fuctio (acf) of a radom process: r ( τ ) E [ Xt ()Xt ( + τ )] Covolutio: xt () * yt () x ( τ )yt ( τ ) d τ 3(5) Goig to the frequecy domai (Sayood ) The Fourier trasform ats iverse: Xf () F [ xt ()] xt ()e jπft xt () F Xf () [ ] Xf d t ()e jπft d f Power spectral desity (psd): Φ xx () f Xf () F [ r () τ ] Covolutio: F [ xt () * yt ()] F [ xt ()] F [ yt ()] Xf () Yf () 4(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003

2 Back to Codig of Aalog Sources istortio-free codig bits! Thus: The rate R C must be related to the acceptable distortio R C Coder ecoder ^ Rate: The umber of bits per secod istortio: The mea square error E [( xt () xˆ () t ) ] How are R C ad related? 5(5) istortio efiitio of distortio: E [( xt () xˆ () t ) ] Aalog sigal: T T -- ( xt () xˆ () t ) d t Sampled sigal: N --- ( x [ ] xˆ [ ]) Sigal-to-Noise ratio (SNR) Usually measure db: SNR 0 log [db] 0 Logarithm base te! 6(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 Rate/istortio Accordig to Shao, the followig lower limit holds for a white Gaussia process with badwidth W: R C W log [bits/s] -W Φ(f) W f Example: Speech sigal with W 4 khz ad SNR 40 db log 0 4 R C 4000 log [kbit/s] 7(5) Pulse-Code Modulatio (PCM) Basic idea: Sample the sigal -> time discrete sigal Quatize the samples -> digital sigal 3 Variable legth codig -> more efficiet represeatio LP Q VLC ^ R C LP δ( ) Q - VL The distortio samplig + quatizatio 8(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003

3 Samplig x() t x samp () t xt () st () xt -- () δt I the frequecy domai: X(f) W * X samp () f Xf () * Sf () 9(5) Samplig, cot If W <, xt () ca be recostructed without distortio LP LP ~ Apply a low- pass filter removig frequecis higher tha! If the sigal has power outside the distortio is: Φ(f) / samp E ( x x ) [ ] Φ f () d f Without the lowpass-filterig we get aliasig which doubles the distortio! 0(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 Quatizatio From cotiuous to discrete alphabet out i Example: 3 bits give 8 levels, 4 bits give 6 levels esigig a quatizer: Place the recostructio levels, the iterval limits, ad the saturatio level (5) Quatizatio Example Origial sigal Quatized sigal ad the error Coarse quatizatio (5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003

4 Quatizatio istortio For each bit, the amplitude error is halved, ad thus the distortio is decreased with a factor 4, that is: R is also proportioal to the sigal eergy: Thus: c R for some costat c Cosequetly, the rate is: R -- c log 3(5) Quatizatio istortio, cot Rewrite to get the SNR i db: SNR 0 log R c log-- R 0 ( R log 0 0 c c 0log ) 6R c [db] c (ad thus c 0 0log c ) depeds o the distributio ad the type of quatizatio For uiform quatizatio c is approximately 7dB (see slide ) Example: igital telephoy usig 8 bits/sample: SNR 4 db 4(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 Quatizig a Bad-limited White Process Sample at W samples/s to avoid samplig distortio Thus, we get WR bits/s R c log WR W c log R C W c log W c c 0 log Thus SNR 6R is the best that ca be achieved whe quatizig a white Gaussia process 5(5) Practical Quatizers A practical quatizer is represeted by a decisios levels ad recostructio levels r i d d x The error: x r i r r r i istortio cotributio from the ith iterval: ( x r i ) f X ()x x d + Total distortio: quat ( x r i ) f X ()x x d N i + 6(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 W samples/s ad R bits/sample: WR bits/s That is, RC WR TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003

5 Optimal Quatizatio ( pdf-optimized ) δ - 0 δ δ - 0 δr i r i + r ---- i + xf X ()x x d d r i i - + f X ()x x d (Check yourself!) (Ceter of gravity) Numerical solutios by Joel Max i 960 Max quatizatio The table collectio give optimal quatizers for, 4, 8, levels (,, 3, bits) ad the associated distortio for sigals with variace ad differet distributios (Gauss, Laplace, Rayleigh) Note that the step sizes are o-uiform! 7(5) Fie Quatizatio If the umber of levels is very large (R >> ) certai approximatios ca be itroduced to give closed-form solutios We approximate f X (x) f X ( ) (cotat withi the iterval) Thus, the probability that the sigal falls i iterval i is p i i f X ( ) ad the recostructio level should be d r i + i + The distortio cotributio from iterval i becomes + ( x r ) f x i ()x d z f X X ( ) d z p i z d z p i i i i i i 8(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 The total distortio becomes quat p i i, where ( x ) gives the iterval legth as a fuctio of x i ( x )f X ()x x d The umber of itervals is the N d x ( x ) If we represet that with a fixed legth code we get (the usual) R log N bits 9(5) Case : Fie Uiform Quatizatio a max () x x a max x > a max Choose a max so that P( X > a max ) is small! p i i 0(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003

6 Fie Uiform Quatizatio, cot From last slide: Number of levels: N R a -- max a -- max ad R a max ---- R 3 Typically choose a max 4σ σ 6 R 3 SNR 6R 73 [db] (5) Case : Fie Max-quatizatio Also called pdf-optimized quatizatio or source adapted quatizatio Miimizig over all (x) ad keepig N costat yields: () x c f X 3 () x For Gaussia distributio this gives σ π 3 R SNR 6R 434 [db] (5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 Case 3: Quatizatio + Etropy Codig Use a variable legth code (typically Huffma codig or arithmetic codig) for the itervals x[] Q VLC It ca be show that uiform quatizatio is always best regardless of the probability distributio of x! For Gaussia distributios: σ πe 6 R SNR 6R 53 [db] 3(5) Summary Aalog sources are modelled as stochastic processes, for example a Gaussia process with psd (power spectral desity) Φ xx () f The data rate must be related to the allowed distortio R c 3 For a white bad-limited Gaussia process R W log [bits/s] c σ x 4 PCM: Samplig + Quatizatio, samp + quat 5 samp Φ xx () f 6 M quat x r i i ( ) fx ( ) d x 7 Numerical optimizatio of, r i gives Max-quatizatio (source adapted quatizatio, pdf-optimzed quatizatio) 4(5) + TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003 TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003

7 8 Fie quatizatio: ( x )fx ( ) d x, M d x ( x ) 6 Fie uiform quatizatio: σ x SNR 6R - 73 [db] 3 R Fie Max-quatizatio: π σ x 3 R SNR 6R [db] Fie etropy coded q: πe σ R SNR 6R - 53 [db] x 6 Shao limit: σ x R SNR 6R [db] 5(5) TSBK0 Image Codig ad ata Compressio Jörge Ahlberg 00, 003