Bounds for Permutation Rate-Distortion

Transcription

1 Bouds fo Pemutatio Rate-Distotio Fazad Faoud (Hassazadeh) Electical Egieeig Califoia Istitute of Techology Pasadea, CA 95, U.S.A. Moshe Schwatz Electical ad Compute Egieeig Be-Guio Uivesity of the Negev Bee Sheva 84050, Isael Jehoshua Buck Electical Egieeig Califoia Istitute of Techology Pasadea, CA 95, U.S.A. Abstact We study the ate-distotio elatioship i the set of pemutatios edowed with the Kedall τ-metic ad the Chebyshev metic. Ou study is motivated by the applicatio of pemutatio ate-distotio to the aveage-case ad wostcase distotio aalysis of algoithms fo akig with icomplete ifomatio ad appoximate sotig algoithms. Fo the Kedall τ-metic we povide bouds fo small, medium, ad lage distotio egimes, while fo the Chebyshev metic we peset bouds that ae valid fo all distotios ad ae especially accuate fo small distotios. I additio, fo the Chebyshev metic, we povide a costuctio fo coveig codes. I. INTRODUCTION I the aalysis of sotig ad akig algoithms, it is ofte assumed that complete ifomatio is available, that is, the aswe to evey questio of the fom is x > y? ca be foud, eithe by quey o computatio. A stadad ad staightfowad esult i this settig is that, o aveage, oe eeds at least log paiwise compaisos to sot a adomlychose pemutatio of legth. I pactice, howeve, it is usually the case that oly patial ifomatio is available. Oe example is the leaig-to-ak poblem, whee the solutios to paiwise compaisos ae leaed fom data, which may be icomplete, o i big-data settigs, whee the umbe of items may be so lage as to make it impactical to quey evey paiwise compaiso. It may also be the case that oly a appoximately-soted list is equied, ad thus oe does ot seek the solutios to all paiwise compaisos. I such cases, the questio that aises is what is the quality of a akig obtaied fom icomplete data, o a appoximately-soted list. Oe appoach to quatify the quality of a algoithm that aks with icomplete data is to fid the elatioship betwee the umbe of compaisos ad the aveage, o wost-case, quality of the output akig, as measued via a metic o the space of pemutatios. To explai, coside a detemiistic algoithm fo akig items that makes R queies ad outputs a akig of legth. Suppose that the tue akig is π. The ifomatio about π is available to the algoithm oly though the queies it makes. Sice the algoithm is detemiistic, the output, deoted as f (π), is uiquely detemied by π. The distotio of this output ca be measued with a metic d as d(π, f (π)). The goal is to fid the elatioships betwee R ad d(π, f (π)) whe π is chose at adom ad whe it is chose by a advesay. This wok was suppoted i pat by a NSF gat CIF-8005, ad a U.S.-Isael Biatioal Sciece Foudatio (BSF) gat A geeal way to quatify the best possible pefomace by such a algoithm is to use the ate-distotio theoy o the space of pemutatios. I this cotext, the codebook is the set f (π) : π S }, whee S is the set of pemutatios of legth, ad the ate is detemied by the umbe of queies. Fo a give ate, o algoithm ca have smalle distotio tha what is dictated by ate-distotio. With this motivatio, we study ate-distotio i the space of pemutatios ude the Kedall τ-metic ad the Chebyshev metic. Pevious wok o this topic icludes [7], which studies pemutatio ate-distotio with espect to the Kedall τ-metic ad the l -metic of ivesio vectos, ad [7] which cosides Speama s footule. Ou esults o the Kedall τ- metic impove upo those peseted i [7]. I paticula, fo the small distotio egime, as defied late i the pape, we elimiate the gap betwee the lowe boud ad the uppe boud give i [7]; fo the lage distotio egime, we povide a stoge lowe boud; ad fo the medium distotio egime, we povide uppe ad lowe bouds with eo tems. Ou study icludes both wost-case ad aveage-case distotios as both measues ae fequetly used i the aalysis of algoithms. We also ote that pemutatio ate-distotio esults ca also be applied to lossy compessio of pemutatios, e.g., ak-modulatio sigals [8]. Fially, we also peset coveig codes fo the Chebyshev metic, whee coveig codes fo the Kedall τ-metic wee aleady peseted i [7]. The codes ae the coveig aalog of the eo-coectig codes aleady peseted i [], [9], [], [6]. The est of the pape is ogaized as follows. I Sectio II, we peset pelimiaies ad otatio. Sectio III cotais o-asymptotic esults valid fo both metics ude study. Fially, Sectio IV ad Sectio V focus o the Kedall τ- metic ad the Chebyshev metic, espectively. Due to lack of space, some of the poofs ae omitted o shoteed. These ca be foud i the full vesio of the pape, available o axiv [6]. II. PRELIMINARIES AND DEFINITIONS Fo a oegative itege, let [] deote the set,..., }, ad let S deote the set of pemutatios of []. We deote a pemutatio σ S as σ = [σ, σ,..., σ ], whee the pemutatio sets σ(i) = σ i. We also deote the idetity pemutatio by Id = [,,..., ]. The Kedall τ-distace betwee two pemutatios π, σ S is the umbe of taspositios of adjacet elemets eeded to tasfom π ito σ, ad is deoted by d K (π, σ). I cotast, /4/$ IEEE 6

2 the Chebyshev distace betwee π ad σ is defied as d C (π, σ) = max π(i) σ(i). i [] Additioally, let d(π, σ) deote a geeic distace measue betwee π ad σ. Both d K ad d C ae ivaiat; the fome is left-ivaiat ad the latte is ight-ivaiat [5]. I othe wods, fo all f, g, h S, we have d K ( f, g) = d K (h f, hg) ad d C ( f, g) = d C ( f h, gh). Hece, the size of the ball of a give adius i eithe metic does ot deped o its cete. The size of a ball of adius with espect to d K, d C, ad d, is give, espectively, by B K (), B C (), ad B(). The depedece of the size of the ball o is implicit. A code C is a subset C S. Fo a code C ad a pemutatio π S, let d(π, C) = mi σ C d(π, σ) be the (miimal) distace betwee π ad C. We use ˆM(D) to deote the miimum umbe of codewods equied fo a wost-case distotio D. That is, ˆM(D) is the size of the smallest code C such that fo all π S, we have d(π, C) D. Similaly, let M(D) deote the miimum umbe of codewods equied fo a aveage distotio D ude the uifom distibutio o S, that is, the size of the smallest code C such that π S d(π, C) D. Note that M(D) ˆM(D). I what follows, we assume that the distotio D is a itege. Fo wost-case distotio, this assumptio does ot lead to a loss of geeality as the metics ude study ae itege valued. We also defie ˆR(D) = lg ˆM(D), R(D) = lg M(D), Â(D) = ˆM(D) lg, Ā(D) = M(D) lg, whee we use lg as a shothad fo log. It is clea that ˆR(D) = Â(D) + lg /, ad that a simila elatioship holds betwee R(D) ad Ā(D). The easo fo defiig  ad Ā is that they sometimes lead to simple expessios compaed to ˆR ad R. Futhemoe,  (esp. Ā) ca be itepeted as the diffeece betwee the umbe of bits pe symbol equied to idetify a codewod i a code of size ˆM (esp. M) ad the umbe of bits pe symbol equied to idetify a pemutatio i S. Thoughout the pape, fo ˆM, M, Â, Ā, ˆR, ad R, subscipts K ad C deote that the subscipted quatity coespods to the Kedall τ-metic ad the Chebyshev metic, espectively. Lack of subscipts idicates that the esult is valid fo both metics. I the sequel, the followig iequalities [4] will be useful, H(p) ( ) H(p), () 8p( p) p πp( p) whee H( ) is the biay etopy fuctio, ad 0 < p <. f (x) Futhemoe, to deote lim x =, we use g(x) f (x) g(x) as x, o f g if doig so does ot cause ambiguity. III. NON-ASYMPTOTIC BOUNDS I this sectio, we deive o-asymptotic bouds, that is, bouds that ae valid fo all positive iteges ad D. The esults i this sectio apply to both the Kedall τ-distace ad the Chebyshev distace as well as ay othe ivaiat distace o pemutatios. The ext lemma gives two basic lowe bouds fo ˆM(D) ad M(D). Lemma. Fo all, D N, ˆM(D) B(D), M(D) > B(D)(D + ). Poof: Sice the fist iequality is well kow ad its poof is clea, we oly pove the secod oe. Fix ad D. Coside a code C S of size M ad suppose the aveage distotio of this code is at most D. Thee ae at most MB(D) pemutatios π such that d(π, C) D ad at least MB(D) pemutatios π such that d(π, C) D +. Hece, D > (D + )( MB(D)/). The secod iequality the follows. I the ext lemma, we use a simple pobabilistic agumet to give a uppe boud o ˆM(D). Lemma. Fo all, D N, ˆM(D) l /B(D). Poof: Suppose that a sequece of M pemutatios, π,..., π M, is daw by choosig each π i i.i.d. with uifom distibutio ove S. Deote C = π,..., π M } S. The pobability P f that thee exists σ S with d(σ, C) > D is bouded by P f σ S P( i : d(π i, σ) > D) = ( B(D)/) M < e MB(D)/ = e l MB(D)/. Let M = l /B(D) so that P f <. Hece, a code of size M exists with wost-case distotio D. The followig theoem by Stei [5], which ca be used to obtai existece esults fo coveig codes (see, e.g., [4]), eables us to impove the above uppe boud. We use a simplified vesio of this theoem, which is sufficiet fo ou pupose. Theoem 3. [5] Coside a set X, with X = N, ad a family A i } i= N of subsets of X. Suppose thee is a itege Q such that A i = Q fo all i ad that each elemet of X is i Q of the sets A i. The thee is subfamily of A i } i= N, cotaiig at most (N/Q)( + l Q) sets, that coves X. I ou cotext X is S, A i ae the balls of adius D ceteed at each pemutatio, N = ad Q = B(D). Hece, the theoem implies that ˆM(D) ( + l B(D)). B(D) The followig theoem summaizes the esults of this sectio. Theoem 4. Fo all, D N, B(D) ˆM(D) ( + l B(D)), () B(D) 7

3 B(D)(D + ) < M(D) ˆM(D). (3) IV. THE KENDALL τ-metric The goal of this sectio is to coside the ate-distotio elatioship fo the pemutatio space edowed by the Kedall τ-metic. Fist, we fid o-asymptotic uppe ad lowe bouds o the size of the ball i the Kedall τ-metic. The, i the followig subsectios, we coside asymptotic bouds fo small, medium, ad lage distotio egimes. Thoughout this sectio, we assume D < ( ) ad. Note that D is uppe bouded by ( ), ad the case of ( ) D ( ) leads to tivial codes, e.g., Id, [,,..., ]} ad Id}. A. No-asymptotic Results Let X be the set of itege vectos x = x, x,..., x of legth such that 0 x i i fo i []. It is well kow (fo example, see [9]) that thee is a bijectio betwee X ad S such that fo coespodig elemets x X ad π S, we have d K (π, Id) = i= x i. Hece, B K () = x X : x i } (4) i= fo ( ). Thus, the umbe of oegative itege solutios to the equatio i= x i is at least B K (), i.e., ( ) + B K (). (5) Futhemoe, fo δ Q, δ 0, such that δ is a itege, it ca also be show that B K (δ) + δ! + δ +δ, (6) by otig the fact that the ight-had side of (6) couts the elemets of X such that 0 x i i, fo i + δ, 0 x i δ, fo i > + δ, ad that i +δ (i ) + ( + δ ) δ δ δ. Next we fid a lowe boud o B K () with <. Let I (, ) deote the umbe of pemutatios i S that ae at distace fom the idetity. We have [, p. 5] ( ) + I (, ) = + j= ( ) j f j, whee f j = ( + (u j j) ) + ( + u j ), ad u (u j j) u j j = (3j + j)/. It ca be show that ( + ) f 4 (+ ) ad that, fo j, we have f j f j+. Thus, fo <, B K () I (, ) ( + 4 ). (7) I the ext two theoems, we use the afoemetioed bouds o B K () to deive lowe ad uppe bouds o  K (D) ad Ā K (D). Theoem 5. Fo all, D N, ad δ = D/, ( + δ)+δ  K (D) lg δ δ, Ā K (D) lg ( + δ)+δ δ δ lg. Poof: Usig (5), (), ad the fact that δ /, we ca show B K (D) (+δ)h( +δ ). The fist esult the follows fom (). The poof of the secod esult is i essece simila but uses (3). To obtai the ext theoem, the lowe bouds give i (6) ad (7) ae used. We omit a detailed poof. Theoem 6. Assume, D N, ad let δ = D/. We have Ā K (D)  K (D) lg ( + δ)+δ δ δ + 3 lg +, fo δ <, ad Ā K (D)  K (D) lg + δ + ) (e lg +δ l + δ, fo δ. B. Small Distotio I this subsectio, we coside small distotio, that is, D = O(). Fist, suppose D <, o equivaletly, δ = D/ <. The ext lemma follows fom Theoems 5 ad 6. Lemma 7. Fo δ = D/ <, we have that ( ) ( + δ)+δ lg  K (D) = lg δ δ + O, (8) ad that Ā K (D) satisfies the same equatio. The ext theoem, gives the asymptotic size of the ball B K (D) whee D = Θ(). Theoem 8. Let = () = c + O() fo a costat c > 0. The ( ) + B K () K c (9) as,, whee K c is a positive costat that depeds o c. We omit the poof of the theoem due to its legth ad oly metio that it elies o a pobabilistic tasfom, amely, Theoem 3. of []. Oe ca use (9), Theoem 4, ad the defiitios of  K ad Ā K to obtai the followig lemma. Lemma 9. Fo a costat c > 0 ad D = c + O(), we have ( ) ( + c)+c lg  K (c + O()) = lg c c + O. (0) Futhemoe, Ā K (c + O()) satisfies the same equatio. The esults give i (8) ad (0) ae give as lowe bouds i [7, Equatio (4)]. We have thus show that these lowe bouds i fact match the quatity ude study. Futhemoe, we have show that Ā K (D) satisfies the same elatios. 8

4 8 6 4 A Lowe Boud W Lowe boud Uppe boud W Uppe boud c Figue. Bouds o  K (D) + lg fo D = c + O() whee the eo tems ae igoed. The bouds deoted by [W] ae those fom [7]. C. Medium Distotio We ext coside the medium distotio egime, that is, D = c +α + O() fo costats c > 0 ad 0 < α <. Fo this case, fom [7], we have  K (D) lg α. We impove upo this esult by povidig uppe ad lowe bouds with eo tems. Usig Theoems 5 ad 6, oe ca show the followig lemma. Lemma 0. Fo D = c +α + O(), whee α ad c ae costats such that 0 < α < ad c > 0, we have lg (ec α ) + O ( α)  K (D) lg (c α ) + O ( α + α ) D. Lage Distotio I the lage distotio egime, we have D = c + O() ad δ = c + O(). The followig lemma ca be poved usig Theoems 5 ad 6. Lemma. Suppose D = c + O() fo a costat 0 < c <. We have lg (ec) + O( )  K (D) lg (ec) + ( + c) lg e + O( lg ). Fom [7], it follows that lg (ec) + O( lg )  K (D) lg e /(c) + O( lg ). These bouds ad those of Lemma ae compaed i Figue, whee we added the tem lg to emove depedece o. V. THE CHEBYSHEV METRIC We ow tu to coside the ate-distotio fuctio fo the pemutatio space ude the Chebyshev metic. We stat by statig lowe ad uppe bouds o the size of the ball i the Chebyshev metic, ad the costuct coveig codes. A. Bouds Fo a matix A, the pemaet of A = (A i,j ) is defied as, pe(a) = A i,π(i). π S i= It is well kow [0], [4] that B C () ca be expessed as the pemaet of the biay matix A fo which i j A i,j = () 0 othewise. Accodig to Bégma s Theoem (see [3]), fo ay biay matix A with i s i the i-th ow pe(a) i= ( i!) i. Usig this boud we ca state the followig lemma (patially give i [0] ad exteded i [6]). Lemma. [6] Fo all 0, (( + )!) + B C () i=+ (i!) i, () + i=+ (i!) i, Lemma 3. Fo all 0, B C () (+),, ( ) 0, 0,.. Poof: The fist case was aleady poved i [0]. Thus, oly the secod claim equies poof, so suppose that ( )/. The poof follows the same lies as the oe appeaig i [0]. Let A be defied as i (), ad let B be a matix with, i + j, B i,j =, i + j + +, A i,j, othewise. We obseve that B/ is doubly stochastic. It follows that B C () = pe(a) pe(b) ( ) pe(b/) ( ) ( ), whee the last iequality follows fom Va de Waede s Theoem [3]. Theoem 4. Let N, ad let 0 < δ < be a costat atioal umbe such that D = δ is a itege. The lg ˆR C (D) δ + δ lg e + O(lg /), 0 < δ δ lg δ + ( δ) lg e + O(lg /), δ ad ˆR C (D) lg δ + δ + O(lg /), 0 < δ ( δ) + O(lg /), δ Futhemoe, the same bouds also hold fo R C (D). Poof: (outlie) To pove the lowe boud fo ˆR C (D), we fist use Lemma to fid a asymptotic uppe boud o B C (D). The, fom Theoem 4, which states that ˆM C (D) /B C (D), we fid a lowe boud o ˆR C (D) = lg ˆM C (D)/. To pove the uppe boud fo fo ˆR C (D), fom Lemma 3, we fid a lowe boud o B C (D). The esult the follows fom Lemma, statig that ˆM C (D) l /B C (D), ad the fact that ˆR C (D) = lg ˆM C (D)/. 9

5 4 3 R a b c Figue. Rate-distotio i the Chebyshev metic: The lowe ad uppe bouds of Theoem 4, (a) ad (b), ad the ate of the code costuctio, give i Theoem 7, (c). The poof of the lowe boud fo R C (D) is simila to that of ˆR C (D) except that we use M(D) > /(B(D)(D + )) fom Theoem 4. The poof of the uppe boud fo R C (D) follows fom the fact that R C (D) ˆR C (D). B. Code Costuctio Let A = a, a,..., a m } [] be a subset of idices, a < a < < a m. Fo ay pemutatio σ S we defie σ A to be the pemutatio i S m that peseves the elative ode of the sequece σ(a ), σ(a ),..., σ(a m ). Ituitively, to compute σ A we keep oly the coodiates of σ fom A, ad the elabel the eties to [m] while keepig elative ode. I a simila fashio we defie σ A = ( σ A ). Ituitively, to calculate σ A we keep oly the values of σ fom A, ad the elabel the eties to [m] while keepig elative ode. Example 5. Let = 6 ad coside the pemutatio σ = [6,, 3, 5,, 4]. We take A = 3, 5, 6}. We the have σ A = [,, 3], sice we keep positios 3, 5, ad 6, of σ, givig us [3,, 4], ad the elabel these to get [,, 3]. Similaly, we have σ A = [3,, ], sice we keep the values 3, 5, ad 6, of σ, givig us [6, 3, 5], ad the elabel these to get [3,, ]. Costuctio. Let ad d be positive iteges, d. Futhemoe, we defie the sets A i = i(d + ) + j j d + } [], fo all 0 i ( )/(d + ). We ow costuct the code C defied by } C = σ S σ A i = Id fo all i. We ote that this costuctio may be see as a dual of the costuctio give i [7]. Theoem 6. Let ad d be positive iteges, d. The the code C S of Costuctio has coveig adius exactly d ad size M = (d + )! /(d+) ( mod (d + ))!. Poof: Due to lack of space, we oly pove the fact that the code C S of Costuctio has coveig adius (at most) d. Let σ S be ay pemutatio. We let I i deote the idices i which the elemets of A i appea i σ. Let us ow costuct a ew pemutatio σ i which the elemets of A i appea i idices I i, but they soted i ascedig ode. Thus σ A i = Id, fo all i, ad so σ is a codewod i C. We obseve that if σ(j) A i, the σ (j) A i as well. It follows that σ(j) σ (j) d ad so d C (σ, σ ) d. The code costuctio has the followig asymptotic fom: Theoem 7. The code fom Costuctio has the followig asymptotic ate, ( ) R = H δ + δ lg, δ δ δ whee H is the biay etopy fuctio. The bouds give i Theoem 4 ad the ate of the code costuctio, give i Theoem 7, ae show i Figue. REFERENCES [] A. Bag ad A. Mazumda, Codes i pemutatios ad eo coectio fo ak modulatio, IEEE Tas. Ifom. Theoy, vol. 56, o. 7, pp , Jul. 00. [] M. Bóa, Combiatoics of pemutatios. CRC Pess, 0. [3] L. M. Bégma, Some popeties of oegative matices ad thei pemaets, Soviet Math. Dokl., vol. 4, pp , 973. [4] G. Cohe, I. Hokala, S. Litsy, ad A. Lobstei, Coveig Codes. Noth-Hollad, 997. [5] M. Deza ad H. Huag, Metics o pemutatios, a suvey, J. Comb. If. Sys. Sci., vol. 3, pp , 998. [6] F. Faoud, M. Schwatz, ad J. Buck, Rate-distotio fo akig with icomplete ifomatio, axiv pepit: [7] J. Giese, E. Schubeth, ad M. Stojakovi, Appoximate sotig, i LATIN 006: Theoetical Ifomatics, o Spige, Ja. 006, pp [8] A. Jiag, R. Mateescu, M. Schwatz, ad J. Buck, Rak modulatio fo flash memoies, IEEE Tas. Ifom. Theoy, vol. 55, o. 6, pp , Ju [9] A. Jiag, M. Schwatz, ad J. Buck, Coectig chage-costaied eos i the ak-modulatio scheme, IEEE Tas. Ifom. Theoy, vol. 56, o. 5, pp. 0, May 00. [0] T. Kløve, Sphees of pemutatios ude the ifiity om pemutatios with limited displacemet, Uivesity of Bege, Bege, Noway, Tech. Rep. 376, Nov [] A. Mazumda, A. Bag, ad G. Zémo, Costuctios of ak modulatio codes, IEEE Tas. Ifom. Theoy, vol. 59, o., pp , Feb. 03. [] O. Milekovic ad K. J. Compto, Pobabilistic tasfoms fo combiatoial u models, Combiatoics, Pobability, ad Computig, vol. 3, o. 4-5, pp , Jul [3] H. Mic, Pemaets, i Ecyclopedia of Mathematics ad its Applicatios. Cambidge Uivesity Pess, 978, vol. 6. [4] M. Schwatz, Efficietly computig the pemaet ad Hafia of some baded Toeplitz matices, Liea Algeba ad its Applicatios, vol. 430, o. 4, pp , Feb [5] S. Stei, Two combiatoial coveig theoems, Joual of Combiatoial Theoy, Seies A, vol. 6, o. 3, pp , 974. [6] I. Tamo ad M. Schwatz, Coectig limited-magitude eos i the ak-modulatio scheme, IEEE Tas. Ifom. Theoy, vol. 56, o. 6, pp , Ju. 00. [7] D. Wag, A. Mazumda, ad G. W. Woell, A ate distotio theoy fo pemutatio spaces, i Poc. IEEE It. Symp. Ifomatio Theoy (ISIT), Istabul, Tukey, Jul. 03, pp