Flexible Quantization

Transcription

1 wb 06/02/21 1 Flexble Quantzaton Bastaan Klejn KTH School of Electrcal Engneerng Stocholm

2 wb 06/02/21 2 Overvew Motvaton for codng technologes Basc quantzaton and codng Hgh-rate quantzaton theory

3 wb 06/02/21 3 Dgtal Representatons Dgtal representaton of sgnal Sequence of samples wth fnte precson Robust aganst dstorton Facltates processng Basc rates for audo sgnals: 48 Hz audo, 16 bts, stereo: bts/second 8 Hz speech, 16 bts: bts/second

4 wb 06/02/21 4 Hgh Rate s Expensve Transmsson Wred lns Last mle Pacet networs Swtchng vdeo Wreless lns WF Moble telephony: codng was enablng technology Secure communcaton Storage Portable audo/vdeo players Output survellance cameras

5 wb 06/02/21 5 Hgh Qualty Now Less Natural Conventonal crcut-swtched networs Vrtually no bt errors, no loss Moble networs Reasonable cost and delay mples bt errors Pacet networs Reasonable cost and delay mples pacet loss

6 wb 06/02/21 6 Networs More Dverse How t was: Sngle-paradgm networ end-to-end One servce How t s: Many paradgms n one composte networ: Crcut-swtched networ Pacet networ Wreless crcut-swtched networ Wreless pacet networ Many types of servce Range of qualty-cost Streamng, one-on-one communcaton

7 wb 06/02/21 7 How We Desgned Coders New applcaton (partcular networ, storage) appeared Study applcaton requrements Desgn coder for applcaton requrements Have competton between coder desgns for condtons of applcaton Select best coder No vson of ntegrated networ

8 wb 06/02/21 8 More On Desgn Condtons Attrbutes of a coder Rate Qualty (subjectve), ncludes sgnal bandwdth Delay Robustness: bt errors and pacet loss Computatonal complexty Desgns selected for one confguraton of attrbutes Assocated wth one networ paradgm Desgn effort rrelevant

9 wb 06/02/21 9 Adaptng to the New Envronment Implcatons of old-school desgn n new world: Coders mplctly unable to adapt: codeboos Transcodng Performance unclear when appled to other condtons GSM coder appled to pacet networs New-school desgn Goal: coders that can adapt n real-tme to Networ condtons Qualty requrements Near-optmal over large range of condtons Employ hgh-rate quantzaton theory and more modelng

10 wb 06/02/21 10 Overvew Motvaton for codng technologes Basc quantzaton and codng Hgh-rate quantzaton theory

11 wb 06/02/21 11 Quantzaton Quantzaton: non-nvertble mappng from Eucldan space R to a countable set of ponts C = { c } that s a subset of R Quantzaton cell: V = { x R : Q( x) = c} Inverse quantzaton s msnomer

12 wb 06/02/21 12 Example: Scalar Quantzer Q(x) x

13 Example: Vector Quantzer wb 06/02/21 13

14 wb 06/02/21 14 Quantzaton Cells and Centrods R V = { x : Q( x) = c} s a cell Usually assumed convex: regular quantzers c Cell = Vorono regon The quantzaton ndex specfes the cell and the reconstructon pont (often called the centrod) If the set of ndces {} s countable, the quantzaton ndex can generally be transmtted wth a fnte number of bts x encoder networ decoder c = { c : x V} C

15 wb 06/02/21 15 Example: Vector Quantzer

16 wb 06/02/21 16 Codng Prncples Is t smart to smply transmt the ndex? NO! (t s f ndex probablty unform) Apply lossless (entropy) codng to ndces Used to create.zp x encoder decoder c = { c : x V} C lossless encoder w networ w lossless decoder

17 wb 06/02/21 17 Mnmum (Bt) Rate of Index Code: the set of all codewords { w } Unquely decodable code: can always reconstruct ~Mnmum codeword length for unquely decodable code: lw ( ) = log ( p( )) (follows from Kraft nequalty) Entropy of the ndex: 2 I H( I) = pi() log 2( pi()) Is ~mnmum average rate needed for ndex More accurately: H( I) L< H( I) + 1

18 wb 06/02/21 18 Example Index resembles con flps = I 2 I = 2 2 = H() I p () log( p ()) 0.5 log(0.5) 0.5 log(0.5) 1bts Index resembles based con flps H( I ) = 0.25 log (0.25) 0.75 log (0.75) = bts 2 2 = 0.05 log (0.05) 0.95 log (0.95) = bts 2 2

19 wb 06/02/21 19 Now Bac to Quantzaton To quantze we need to now wth respect to what Optmal trade-off dstorton versus number of ndces Constraned-resoluton Assumes codeword length s fxed Generally short delay Consstent wth TDMA and FDMA, crcut-swtched networs The past Optmal trade-off dstorton versus average rate Constraned-entropy Assumes only average codeword length matters Often long delay Consstent wth CDMA and pacet-swtched networs The future!

20 wb 06/02/21 20 Old-School, Any-Rate Quantzaton Standard approach Constraned-resoluton Stored codeboos Codeboos traned wth data (Generalzed) Lloyd algorthm (GLA), Bell Labs, 1958 / K-means algorthm

21 wb 06/02/21 21 Old-School, Any-Rate Quantzaton Is constraned-resoluton x encoder networ decoder C c = { c : x V}

22 wb 06/02/21 22 Lloyd Algorthm Note: Encoder = Partton={Vorono regons} Decoder = codeboo={centrods) Lloyd algorthm: ntal encoder and decoder optmze encoder done? fnal encoder and decoder optmze decoder Optmze: mnmze mean dstorton: Locally optmal E[mn d( X, c )] I

23 Outcome Lloyd for Vector Quantzer wb 06/02/21 23

24 wb 06/02/21 24 Practcal (Dscrete) Lloyd Algorthm Have database { x } m m M Encoder = partton = { V = { x } : UV ={ x } } j I ( j) j I Decoder = codeboo = C = { c } { V j} C = { c } { } Coptmze V j done? { Vj} C optmze C Optmze = mnmze overall dstorton: m M [mn d( x, c )] I m

25 wb 06/02/21 25 Old-School, Any-Rate Quantzaton Is n your cell phone Constraned resoluton (fxed number of cells/centrods) Wors even at low rates Locally optmal Dstorton decreases each step Tranng computatonally expensve: not n real tme Iteratve solutons only Many varants: Mult-stage Tree Constraned-entropy verson (around 1990)

26 wb 06/02/21 26 Hgh-Rate Quantzaton Assume data densty can be assumed constant wthn a cell (Bennett, 1948) Assume that noton densty of centrods s meanngful Problem formulaton Gven data densty, dstorton crteron, constrant Fnd centrod densty ( quantzer ) Advantage of approach Optmal quantzer can be computed analytcally Can be done n real-tme

27 wb 06/02/21 27 Dstorton and Geometry: SQ D = V 2 f X( xd ) ( xqx, ( )) dx fx( x) ( x c) dx V v f X ( x) dx f X ( x) Δ c 24 /2 2 2 Δ 1 = xdx= Δ Δ Δ /2 12 Δ 23 c 23 Scalar = cubc geometry

28 wb 06/02/21 28 Dstorton and Geometry: VQ Mean dstorton n cell, r th power crteron, per dm fx ( x ) d( x, Q( x )) dx 1 D = x c dx r f ( x ) dx V V v X r r+ r 1 = V V x c dx = V C(, r, G()) r V V CrG (,, ()) CrGx (,, ( )) s the nertal profle coeffcent of quantzaton

29 wb 06/02/21 29 Quantzaton and Cell Geometry Scalar case: cubc cells 1 Cr ( = 2, = 1, G= optmal) = Cr ( = 2, = 2, G= optmal) = D: Hexagonal n sets of two dmensons -D: Sphercal cells Cr G e 1 ( = 2, =, = optmal) (2 π ) = VQ has space-fllng advantage; 1.53 db (= 0.25 bt)

30 wb 06/02/21 30 CE Quantzers n 2D Two dmensons: square and hexagonal lattce

31 wb 06/02/21 31 Hgh-Rate Quantzaton What we have done: relate local geometry to local dstorton Next step: to relate dstorton, rate and centrod densty (and local geometry) Centrod densty: number of centrods/unt volume gx ( )

32 wb 06/02/21 32 Remnder: Constraned-Entropy Codng Apply lossless (entropy) codng to ndces Used to create.zp Rate s mean rate of codewords Consstent wth CDMA, pacet networs, the future x encoder decoder C c = { c : x V} lossless encoder w networ w lossless decoder

33 wb 06/02/21 33 Constraned-Entropy Quantzaton I Constrant on ndex entropy Equvalent constrant H( I) = p ()log( p ()) = I Vf ( c)log( Vf ( c)) X X ( ) f ( x ) log( f ( x )) log( g ( x ) dx = hx ( ) + f ( x)log( g ( x)) dx I X X X X X f ( x )log( g ( x ) dx = constant X X

34 wb 06/02/21 34 Constraned-Entropy Quantzaton II r Dstorton: D = p () D = p () V C( r,, G()) Add Lagrange-multpler term: I I CrG (,, ) f ( x) g ( x) dx r = CrG (,, ) f X ( x ) g ( x ) + λ log( g( x )) dx Mnmze; get Euler-Lagrange equaton; solve X r D C(, r, G) f ( x ) g ( x ) dx X r r+ 1 fx ( x ) g ( x ) + λg ( x )) = 0 gx ( ) = constant!!

35 wb 06/02/21 35 Moral of the Story For constraned-entropy quantzaton: smplest quantzer s best All cells are same sze and shape (not proven, that) Facltates low computatonal complexty quantzer Can compute quantzer for gven pdf and dstorton Does not mean entre encoder s low complexty! Somewhat non-ntutve: Infnte number of cells/centrods! Cell sze ndependent of data densty

36 wb 06/02/21 36 CE Quantzers n 2D Two dmensons: square and hexagonal lattce

37 wb 06/02/21 37 Constraned-Entropy Quantzaton IIa Complete soluton: At a gven dstorton level the optmal centrod densty: ncreases wth mean ndex rate decreases wth dfferental entropy of data (= complexty of data) Can adjust coder n real tme! ( HI ) gx ( ) = exp ( ) h(x )

38 wb 06/02/21 38 Dstorton-Rate Relaton Relaton dstorton and rate (per dmenson): D = p ( ) D C( r,, G) f ( x ) g( x ) dx I X r X = CrGgx (,, ) ( ) f ( x) dx r = CrG (,, )exp HI () hx ( ) ( ) r

39 wb 06/02/21 39 The Vector-Quantzaton Advantage Dvde dstortons of SQ and VQ (Gray & Looabough, 1989) D D SQ VQ = ( ( 1 )) Cr (,1, G)exp r HI () hx ( ) r CrG (,, )exp HI () hx ( ) ( ) Cr (,1, G) 1 = r h X h X CrG (,, ) 1 exp ( ) ( ) Space-fllng advantage Memory advantage (due to redundancy) ρ = hx ( ) hx ( ) 1 1

40 wb 06/02/21 40 Constraned-Entropy Quantzaton s Easy Unform quantzer: smple to mplement Small advantage from usng best lattce Somewhat more complcated Lossless codng s not easy: Does not even exst n old-school quantzaton Must now data densty

41 wb 06/02/21 41 Some Notes on Lossless Codng Lossless codng tres to reduce rate to ndex entropy Huffman code: Table w based on probablty dstrbuton Wors on per-varable bass; hgh overhead Smple to mplement Arthmetc code: Computes codewords for sequence of coeffcents Trcy to wrte program Low overhead Requres cumulatve dstrbuton functon (cmf) Often nontrval to obtan cmf Preferred method

42 wb 06/02/21 42 Practcal Hgh-Rate CE Codng No sgnfcant commercal mplementatons as yet Quantzer and arthmetc coder are computed; flexble x estmate pdf hgh-rate encoder arthmetc encoder w, 1, 2, 3, L networ c = { c : x V} C decoder arthmetc decoder

43 wb 06/02/21 43 Example PDF Estmaton Dffcult; smplfy problem: Densty modeled as mxture p ( x ) = p ( m) p ( x ) X M X, m m Interpretaton: data fall n one of set of probabltes Each mxture component s Gaussan (usually) Know how to desgn quantzer for Gaussan Symmetrc Just one desgn procedure needed for cmf computaton Encode whch component you select then use correspondng quantzer

44 wb 06/02/21 44 Gaussan mxture Four components:

45 wb 06/02/21 45 Hgh-Rate Quantzaton Not yet wdely appled Real-tme adaptaton not used Constraned entropy (constrant on average rate)

46 wb 06/02/21 46 What Have We Learned Problem: Have audo or vdeo data (transformed or not) Need to encode effcently Old-School Soluton Good performance / not flexble Constraned resoluton Codeboo (often computatonally expensve) Commonly used New-World Approach Good performance / can adapt n real-tme Constraned entropy; requres lossless coder (arthmetc coder) Quantzer and arthmetc coder computed = flexble Not yet ready

47 wb 06/02/21 47 Quantzaton Conclusons Emphass was on performance Emphass s on flexblty (but no loss of performance)