Icreasg Kolmogorov Complexty Harry Buhrma Lace Fortow Ila Newma Nkola Vereshchag September 7, 2004 classfcato: Kolmogorov complexty, computatoal complexty 1 Itroducto How much do we have to chage a strg to crease ts Kolmogorov complexty. We show that we ca crease the complexty of ay o-radom strg of legth by flppg O bts ad some strgs requre Ω bt flps. For a gve m, we also gve bouds for creasg the complexty of a strg by flppg m bts. By usg costructble expadg graphs we gve a effcet algorthm that gve ay o-radom strg of legth wll gve a small lst of strgs of the same legth, at least oe of whch wll have hgher Kolmogorov complexty. As a applcato, we show that BPP s cotaed P relatve to the set of Kolmogorov radom strgs. Alleder, Buhrma, Koucký, va Melkbeek ad Roeberger [2] buldg o our techques later mproved ths result to show that all of PSPACE reduces to P wth a oracle for the radom strgs. 2 Icreasg Complexty by Flppg Bts Usg the otato of L ad Vtáy, we use C U x to represet the sze of the smallest program p such that Up = x. We fx a uversal referece computer U ad let Cx = C U x. Assume we are gve a bary strg x. By how much we ca crease ts complexty by flppg at most m bts of x? Let N m x deote the set of all strgs wth Hammg dstace at most m from x. Let N m A stad for the uo of N m x over x A. We use the otato O1, c, c 1,... for costats depedg o the referece mache U ad d, d 1,... for absolute costats. The followg, rather geeral theorem, assertg that the complexty of ay typcal strg a set ca be creased by flppg m bts to the expected log N m A s a mmedate mplcato of the cardalty lower boud for Kolmogorov complexty. Theorem 1. Let k, m, a be such that the followg codto hold * for every set A {0, 1} wth A > 2 a, N m A 2 k for k <, or N m A 2 1 1c 2 for k =. The, there are costats c 1, c 2 depedg o the referece computer such that for every strg x of complexty at least Cx a 2Ck, m, a c 1 there s a strg y obtaed from x by flppg at most m bts such that Cy k. Proof. Cosder the followg set B = {x {0, 1} Cy < k for all y N m x}. CWI, Amsterdam Uversty of Chcago. Hafa Uversty Moscow State Uversty, Emal: ver@mccme.ru. The work was doe whle vstg CWI; also supported part by the RFBR grat 02-01-22001. 1
As the Kolmogorov complexty of all strgs N m B s less tha k we have N m B < 2 k. I the case = k we may upper boud N m B better. Recall the followg lower boud for the umber of radom strgs for the proof see [5]: for approprate choce of c 2 for every the umber of strgs y of legth wth Cy s more tha 2 c 2. Therefore the case k = we have N m B < 2 1 1c 2. I both cases we thus obta B 2 a. The set B may be eumerated gve k, m,. Therefore every strg x B ca be descrbed by m,, k ad ts dex B of bt legth a. Thus Cx < a 2Ck, m, a c 1 for all x B, where c 1 s a costat depedg o the referece computer. I other words, for every x such that the last equalty s false there s y N m x wth Cy k. Theorem 1 s rather geeral ad apples to ay graph rather just the Boolea cube, whe we replace flppg bts wth gog to eghbors. Ths wll be dscussed Secto 3. We ow wat to apply Theorem 1. For ths we eed to aalyze the expadg propertes of the Boolea cube. The complete aalyss s gve by the followg theorem. We frst troduce a otato. Let b, l deote the bomal sum: b, l = 0 1 l. Theorem 2 Harper. Let J 2. Take all the strgs wth less tha l oes ad take J l frst strgs wth l oes the lexcographcal order, where l s chose so that b, l 1 < J b, l. The the resultg set has the least N 1 A amog all sets A wth A = J. We wll use the followg corollary of Harper s theorem. Corollary 3. If N m A b, l ad l < the A b, l m ad N m A A > l l m. We ote that the secod boud s very weak ad becomes trval for l > 2. It wll be suffcet though for our applcatos. Proof. It s eough to prove the theorem the case m = 1. For m > 1 we ca use ducto where ductve step s due to the case m = 1. The frst statemet mmedately follows from Harper s theorem. Let us prove the secod oe assumg that l 2. Let J = A. It suffces to establsh the equalty assumg that A s the worst case set defed the Harper s theorem. We have 0 1 l < A = J 1 0 1 for some l. We clam that l < l. Ideed, otherwse A > b, l 1 ad therefore A has a strg wth l oes, thus NA has a strg wth l 1 oes hece NA > b, l, a cotradcto. For the worst case set A we wll prove that A A l l 1 l 1l where A stads for the set of strgs obtaed from strgs A by chagg a 0 to 1 but ot vce verse. Actually A ad NA dffer by oly oe strg, 00... 0. Let B cosst of all strgs wth less tha l oes thus B A. Obvously, A ad B A do ot tersect, as every strg the frst set has at most l oes ad every strg the secod set has l 1 oes. Therefore t suffces to prove that B B l l 1 ad B A B A l l 1. The frst equalty s proved as follows: B s the set of all strgs wth at most l oes except 00... 0, so B = 1 2 l. Ad B = 0 1 l 1. The rato of th term the frst sum ad th term the secod sum s 1 = 1 l 1l l l 1. Let us prove the secod equalty. Let x be a strg wth l oes ad let C x deote the set of all strgs wth l oes that are less tha or equal to x. We clam C x C x s a o-creasg fucto x. To prove ths clam t suffces to show that C x {x } C x s a o-creasg fucto x where x deotes the successor of x. The set C x {x } C x cossts of all strgs obtaed by flppg all zeros x precedg the leadg 1 all other flps result strgs that are already C x. Hece C x {x } C x s equal to the umber of zeros precedg the leadg 1 x. Ad the latter umber does ot creases as x creases. For x equal to the last strg wth l oes we have C x C x = l 1 l = l l 1 so we are doe. l 2
As a result we obta the followg trplets of k, m, a for whch codto * ad hece Theorem 1 hold. Theorem 4. There s a costat c 3, such that for every k, m ad a strg x of complexty at least Cx a 2Cm, a c 3, there s a strg y obtaed from x by flppg at most m bts such that Cy k. Here a = k m log ll where l s the least umber such that 2 k b, l. Proof. Let l be as above ad let c 1 be the costat from Theorem 1. We frst ote that the codtos of Theorem 1 hold for a, k, m. Ideed, assume that N m A < 2 k, the by the defto of l, N m A < 0 1 l ad by Corollary 3 we have A < N m A ll m < 2 k ll m 2 a. Hece, Theorem 1 asserts that for every strg x wth Cx a 2Ck, m, a c 1 there s a strg y obtaed from x by flppg at most m bts such that Cy k. It suffces to prove that Ck, m, a Cm, ao1 log mo1. To ths ed we wll prove that k ca be retreved from m,, a. By defto l s a fucto of, k ad a s a fucto of, k, m. The fucto l, k s o-decreasg k hece the fucto a, k, m = k m1 log ll s also o-decreasg k, as the sum of two o-decreasg fuctos. Moreover, the frst term creases by 1 as k cremets by 1. Ths mples that k ca be retreved from m,, a hece Ck, m, a Cm, a O1. For p 0, 1 let Hp = p log p 1 p log1 p be the Shao Etropy fucto. Note that for every α [0; 1 there are two dfferet β 1, β 2 such that hβ 1 = hβ 2 = α; they are related by the equalty β 1 β 2 = 1. Let H 1 α stad for the least of them. The fucto H 1 α creases the rage 0, 0.5 as so does H. Theorem 5. For all α < 1 ad > 0 there s mα, depedg also o the referece computer such that for all large eough the followg holds: For all x of legth wth Cx α there s y obtaed from x by flppg at most mα, bts such that Cy Cx. For ay fxed there s a postve α such that mα, = 1. Proof. Fx α ad ad let x be such that Cx α ad let k = Cx. Let l be the least umber such that b, l 2 k. We frst prove that l β for some costat β < 12, for large eough. Ths meas that b, β 2 k for some costat β < 1, for large eough. Let β be ay umber the terval H 1 α; 12 As α < 1, the terval s ot empty. The, b, β β 2 Hβ1o1 where the last equalty s stadard, see e.g. [7]. Pluggg the defto of β ca cotue the equalty: b, β 2 Hβ1o1 2 α 2 k for large eough. Defe ow a = k m log ll. Applyg Theorem 4, wth a, k, l as above, we get that for every x there s y obtaed from x by flppg at most m bts such that Cy k, as eeded, provded that Cx a 2Cm, a c 3. 1 To show that 1 holds, ote that Cm, a log m. Pluggg ths, alog wth the defto of a, k, 1 we get that t s eough to show that Cx Cx m log ll 2 log m c 3. Usg that l β ad the approprate boud o β we get that t s eough to have m log1 ββ > 2 log m c 3. Note that the defto of β mples that β < 12 hece 1 β β > 1. Therefore for large eough m we wll have m log1 ββ > 2 log m c 3. Fally, let m = 1. Note that log1 ββ teds to fty as β teds to 0. Therefore for ay fxed there s a postve β such that m log1 ββ > 2 log m c 3. Let α be equal to ay postve real such that Hα < β. Remark 1. We ote that Theorem 5 works for fxed, wth respect to, whle m depeds o ad α for fxed α or could be fxed whe α gets small eough. Oe could ask whether t mght be true that could be a fucto of, e.g, could the followg stregtheg of Theorem 5 be true: For ay α or eve for some α the complexty of a strg x that s bouded by α could be creased to α by chagg oly oe bt. It obvous that we caot expect such a stregtheg for > log, as gve x the complexty of ay y that dffers form t oe place s at most Cx log. Other lower bouds o m vs. the amout of crease complexty, ad the relato to α are developed Theorem 7 ad Theorem 9. 3
Let us estmate how may bts we eed to flp to crease complexty from k 1 to k whe k s close to, say for k =. Theorem 6. For every x wth Cx < by flppg at most c 3 bts of x we ca crease ts complexty by at least 1. Proof. Assume frst that Cx 3. Let k = Cx 1 2 ad m = c 4 for a costat c 4 to be defed later. Apply Theorem 4. As 2 k 2 4 we have l 2 d 2 ad ll 2 d 2 2 d2 1 2d3 2 d4 for large eough. Ths mples that a k c 4 d 4. By Theorem 4 for every x wth Cx k c 4 d 4 2Cm a, c 3 there s y obtaed from x by flppg at most m bts wth Cy k. Obvously Cm a, log c 4 c 5. Therefore f c 4 s large eough we have k c 4 d 4 2Cm a, c 1 k 1 ad we are doe. Assume ow that Cx 2. Let us prove that by flppg O bts we ca crease the complexty of x up to. Ths tme we wll apply Theorem 1 ad Corollary 3 drectly. For some c 3 for l = 2 c 3 we have b, l 2 1 1c 2, where c 2 s the costat from Theorem 1. Let m = c 3 c4, where c4 s chose so that b, l m 2 c5, ad c 5 wll be chose later. Let a = c 5 ad k =. By Corollary 3 the codtos of Theorem 1 are fulflled. As a 2Ck, m c 1 c 5 2 log c 5 c 6 2 f c 5 was chose approprately, we are doe. Now we proceed to the lower bouds of the umber of flpped bts. We wll show that for every m there s α such that the complexty of some strgs of complexty α caot be creased by flppg at most m bts. Ad there are strgs for whch we eed to flp Ω bts. Theorem 7. For every m, k 1 there s a θk, m < 1 such that for every α > θk, m, for almost all there s a strg x of legth such that Cx α ad Cy < Cx k for every strg y obtaed from x by flppg at most m bts. Proof. Let θk, m = H11 2 km, ad let θk, m < α. As k > 0 we ote that 11 2 km < 12. Hece θk, m < 1. Wthout loss of geeralty assume that α < 1. Pck ay β the terval 11 2 km ; H 1 α. Aga by the boud above, ad usg the fact that H s mootoe the terval 0; 0.5, the terval for β s o empty. Let l = β c 2 m for a costat c 2 to be defed later. We frst prove that every strg x havg at most l oes satsfes the equalty Cx < α, for large eough. Ideed, the umber of such strgs s equal to b, l ad hece s at most 2 Hl1o1 [7] as l < 2. Therefore Cx < Hβ1 o1 O1 < α for large eough, where the costat O1 depeds o β, c 2, m ad the referece computer. So we eed to show that there s a strg x havg at most l oes ad satsfyg the secod statemet of the theorem. Assume that ths s ot the case. Let the x 0 be a radom strg havg at most β oes, that s, Cx 0 logb, β. If x 0 satsfes the statemet the we are doe. Otherwse there s x 1 havg at most β m oes such that Cx 1 Cx 0 k. Repeatg ths argumet c 2 tmes we ether fd a strg satsfyg the statemet or obta a strg x c2 wth Cx c2 Cx 0 c 2 k havg at most β c 2 m = l oes. Hece Cx c2 logb, β c 2 k. O the other had, Cx c2 logb, l2cl c 1 logb, l2 log c 2 c 3, where c 3 depeds o k, m, α ad the referece computer. To obta the cotradcto we have to show that the upper boud of Cx c2 s less tha the lower boud. The rato of 0 1 l ad 0 1 β ca be s bouded usg the followg Lemma 8. If j s 0 ad j s 2 the b, j s b, j 1 j s s. j s 1 4
Proof. b, j s b, j 1 max s =1 1 max s =1 j s j s 1 j j s s j s 1 j s 1 s. By Lemma 8 we have that c2m b,l b,β 2 1 β β. Thus, to acheve cotradcto t s eough to choose c2 so 1 c 2 m log1 ββ 2 log c 2 c 3 < c 2 k. 2 Ideed, by the choce of β we have m log1 ββ < k. Hece the left had sde of 2 as a fucto of c 2 grows slowly tha the rght had sde ad for large eough c 2 the equalty holds. We wll show ow that sometmes we eed to flp Ω bts of x to crease ts complexty eve by 1. Theorem 9. There s a costat c such that for almost all there s a strg x of legth ad complexty at most 1, ad such that the followg holds: For every strg y obtaed from x by flppg at most c bts, Cy Cx. Proof. For every c 1 there s c 2 such the set of strgs wth at most 2 c 2 oes has cardalty less tha 2 c1 ad therefore the complexty of every such strg s less tha c 1 2 log c 1 c 3. Pck c 1 so that c 1 2 log c 1 c 3 1. Let x 0 be a radom strg wth at most l = 2 c 2 1 oes. Assume that for some x 1 we have Cy Cx 1 ad x 1 dffers from x 0 at most c bts. I ths case apply the same argumet to x 1 ad so o, c tmes. Ether we wll obta x dfferg from x 0 at most c bts satsfyg the statemet of the theorem, or x c such that Cx c Cx c. I the frst case x has at most 2 c 2 1 = 2 c 2 oes hece Cx 1 ad we are doe. Let us show that the secod case s mpossble. 2 log c c4. By Lemma 8 we ca upper boud the rato l log l 1 l l We have Cx c log l c ad Cxc l by 2 c2 2 = 1 2 c 2 2 c5 for some costat c 5 for large eough. Therefore we wll have a cotradcto f log c 5 2 log cc 4 < c. 3 Icreasg Kolmogorov Complexty va Expaders I ths secto we wll use, place of Boolea cubes, graphs that have stroger expaso propertes. Recall the theorem of Marguls [6] o explct expaders. Theorem 10 Marguls. Let k be a teger ad G = V, E be the graph wth vertces V = {0,..., k 1} 2 where a vertex x, y s adjacet to vertexes x, y, x1, y, x, y 1, x, xy, ad y, x all operatos are mod k. There s a postve ε such that for every A V the set NA of all eghbors of vertexes A has at least 1 ε1 A V A elemets. Let k = 2 l. We wll detfy strgs of legth = 2l ad odes of the Marguls expader G. Let N d u deote the set of all odes at the dstace at most d from u the graph G. Let N d A stad for the uo of N d u over u A. Theorem 11. There s a costat c 2 such that for every ode u G wth Cu < there s a ode v N c2 u wth Cu > Cv. 5
Proof. Let c be a costat to be specfed later. Let c 1 be the costat such that for every the umber of strgs y of legth wth Cy s more tha 2 c 1. Let c 2 be a costat such that 1 εc 1 c2 2 c. Assume that the statemet of the theorem s false for some ode u. Let us exhbt a small set cotag u. Let A = {u V v N u Cv Cu } where = 0,..., c 2. Obvously, A 1 = NA ad therefore we have A 0 A 1 A c2. By defto, all strgs A c2 have Kolmogorov complexty at most Cu <. Therefore we ca upper boud A 0 two ways: A 0 2 Cu 1 ad A 0 2 2 c 1. By expaso property we have A 0 1 εc 1 A 1 1 εc 1 c2 A c2 2 c A c2. Hece A c2 s small, A c2 2 c A 0 2 Cu 1 c. Sce u s A c2 ad A c2 ca be eumerated gve l ad Cu, we ca descrbe u by ts dex the eumerato of A c2 of legth Cu 1 c ad by c ad Cu ca be computed from the legth of the dex ad c. Hece Cu Cu 1 c2 log co1. If c s large the ths s a cotradcto. Usg Theorem 11 we may desg a polyomal tme algorthm that havg access to the oracle R = {x Cx x x } for every eve legth 2l fds a strg R of legth 2l. Theorem 12. There s a algorthm that havg access to the oracle R = {x Cx x x } for every eve legth 2l tme polyl fds a strg R of legth 2l. Proof. We wll fd strgs u 0,..., u l such that u = 2 ad u R. Let u 0 be the empty strg. Certaly u 0 R. To fd u gve u 1 apped frst 00 to u 1 ad let u be the resultg strg. As Cu 1 2 1 2 1 we have Cu 2 2 c 3 for some costat c 3. By Theorem 11 there s a strg v at N c3c2 u such that v R. Makg at most 5 c3c2 queres to the oracle R we fd the frst such v ad let u = v. Remark 2. The same argumet apples as well to every set of the form {x Cx x f x } where f ad f 1 f Olog for all. I ths case we search for v N c3olog c2 u place of N c3c2 u. As N c3olog c2 u stll has polyomal sze the algorthm rus polyomal tme. Note that the algorthm eed o other formato about f tha the costat hdde Olog. Remark 3. The argumet apples also to fd radom strgs of odd legths, but that requres more techcal detals. Gve a strg u of eve legth = 2l wth Cu we eed to fd a strg v of odd legth = 2l 1 wth Cv. To ths ad we ca use Marguls expader for the largest k such that k 2 2 2l1. Obvously k 2 2 2l ad we may detfy strgs of legth 2l 1 edg wth 0 wth the frst 2 2l odes of the graph, ad the other odes wth the frst remag strgs of legth 2l 1. Aga we have Cu0 2l 1 2l 1 c 3 for a costat c 3. For large eough l the dfferece betwee 2 2l1 ad k 2 s less tha 2 2l1 2c 1 where c 1 s a costat such that the umber of radom strgs of legth 2l 1 s at least 2 2l1 c 1. Therefore at least k 2 2c 1 odes the graph are radom ad we ca apply the argumets from the proof of Theorem 11 wth 2c 1 place of c 1. Corollary 13. BPP P R Proof. Let M be a probablstc mache recogzg a laguage A. Let be the legth of put to M. We ca assume that the probablty that M errs o at least oe strg of legth s at most 2. Let d be the legth of radom strgs used by M o puts of legth. Here s the determstc algorthm wth oracle R to recogze A: Fd a strg r R of legth d ad ru M o the put x usg r as the sequece of radom bts for M we use the same strg r for all puts x. The output the result of M. If for some strg of legth the aswer s correct the the strg r falls to a set of cardalty 2 d that s detfed by ad M ad hece Cr d d O1 < d for large eough, whch s a cotradcto. Thus our polyomal tme algorthm wth oracle R s correct for almost all puts. Hardwrg the table of aswers for small puts we obta a polyomal tme algorthm wth oracle R that recogzes A o all puts. 6
Let us tur to the ucodtoal Kolmogorov complexty Cx. Let R = {x Cx x }. We wll show that Theorem 12, the ext two remarks ad Corollary 13 geeralze to R. As to Theorem 11, we ca prove oly a weaker ts verso: Theorem 14. There s a costat c 2 v N c2 log c2 u wth Cu > Cv. such that for every ode u G wth Cu < there s a ode Proof. Essetally the same proof, as for Theorem 11 but ths tme we eed to choose c 2 so that 1 εc 1 c2 log c2 2 c2 log. I place of equalty Cu Cu 1 c 2 log c O1 we have the equalty Cu Cu 1 c 2 log 2 log c 2 log l O1. The term 2 log s eeded as ths tme we have to detfy the legth of u. However, to prove the aalog of Theorem 12 we eed oly to crease Kolmogorov complexty of strgs u wth Cu u O1. For that specal case we have Theorem 15. For every costat c 3 there s a costat c 4 such that for every ode u G wth > Cu c 3 there s a ode v N c4 u wth Cu > Cv. Proof. Aga the same proof but place of equalty Cu Cu 1 c 2 log c O1 we have the equalty Cu Cu 1 c 2 log c O1. Ths tme we ca fd the legth of u from the legth Cu 1 c of the dex of u A c4 ad from c, as Cu ad u are close to each other. Therefore Theorem 12, the ext two remarks ad Corollary 13 geeralze to the ucodtoal Kolmogorov complexty. Refereces [1] Ahlswede, Gács, Körer. Bouds o codtoal probabltes wth applcatos mult-user commucato. Z. Wahrschelchketstheore verw. Gebete 34 1976 157 177. [2] Alleder, Buhrma, Koucký, va Melkbeek ad Roeberger. Power from Radom Strgs. 43rd IEEE Symposum o the Foudatos of Computer Scece 2002 669 678. [3] L.H. Harper. Optmal umbergs ad sopermetrc problems o graphs. J. Combatoral Theory 1 1966 385 393. [4] G.O.H. Katoa. The Hammg-sphere has mmum boudary. Studa Scetarum Mathematcarum Hugarca 10 1975 131 140. [5] M. L, P.M.B. Vtáy. A troducto to Kolmogorov complexty ad ts applcatos. New York, Sprger-Verlag, 1997. [6] G.A. Marguls. Explct costructos of cocetrators. Explct costructo of cocetrators. Probab. Ifo. Tras., 9 1975, 325 332. Traslated to Eglsh from Problemy peredach formats 92 1973 71 80. [7] Rose ed., Hadbook of Dscrete Combatoral Mathematcs, CRC Press, 2000. 7