Entropy and Ergodic Theory Lecture 24: Rokhlin s lemma

Transcription

1 Etropy ad Ergodic Theory Lecture 24: Rokhli s lemma I the remaider of the course, our most substatial results are about the existece of factor maps betwee various pairs of MPSs or sources, sometimes with special extra properties. Our first result of this kid is the other half of the geeralized rate-distortio theory begu i Lecture 23. Let ra Z, µs be a ergodic source, ad assume ow that µ is atomless. For each, let µ be the projectio of µ to ay cosecutive coordiates. Fix δ ą 0. For each, let F pδq be the miimum value of the rate 1 log ΦpA q over all ecoders Φ : A ÝÑ A which sastisfy d px, Φpxqq µ pdxq ď δ. (1) Let R map pδq be the solutio of the ifiitary, statioary-code aalog of this optimizatio problem: R map pδq : if hpϕ µq : ϕ : A Z ÝÑ A Z statioary ad µtϕ 0 pxq x 0 u ď δ (. We showed last time that lim ÝÑ8 F pδq exists, ad that it is bouded above by R map pδ 1 q wheever δ 1 ă δ. (I fact, we showed a slightly stroger coclusio: oe ca use the joiig-versio R joi pδ 1 q i place of R map pδ 1 q here. See Lecture 23.) I this lecture we prove the reverse coclusio: that lim ÝÑ8 F pδq ě R map pδ 1 q wheever δ 1 ą δ. (2) 1

2 To do this, we cosider a large value of ad a ecoder Φ : A ÝÑ A which achieves the rate F pδq, ad tur these ito a statioary code A Z ÝÑ A Z which gives a correspodig boud o the rate R map pδq. The method for doig this rests o a ew idea that is also essetial to the other costructios of factor maps later i the course, ad to a wide variety of other existece results i ergodic theory as well. 1 A special case ivolvig periodic sets Before itroducig the mai ew idea that we eed, let us cosider agai the costructio problem described above. Suppose we have chose a ear-optimal -block ecoder Φ : A ÝÑ A. The a obvious way to costruct a map A Z ÝÑ A Z is to apply copies of the map Φ to cosecutive -blocks withi a strig i A Z, thus: ϕpxq : `..., Φpx,..., x 1 q loooooooomoooooooo output coords,..., 1, Φpx 0,..., x 1 q loooooooomoooooooo output coords 0,..., 1, Φpx,..., x 2 1 q looooooooomooooooooo output coords,..., 2 1,.... The problem: this map is geerally ot equivariat. Ideed, the map S 1 ϕ S differs from ϕ i that it applies Φ to -blocks with differet startig poits: ϕpxq : `..., Φpx `1,..., x 0 q looooooooomooooooooo output coords ` 1,..., 0, Φpx 1,..., x q loooooomoooooo output coords 1,...,, Φpx `1,..., x 2 q looooooooomooooooooo output coords ` 1,..., 2,.... This is the mai problem that we must overcome i order to complete our costructio. There is a simple way forward i case the source ra Z, µs has a very special piece of extra structure: a measurable subset F Ď A Z such that: F, SF,..., S 1 F are disjoit ad µpf Y SF Y Y S 1 F q 1. (3) Observe that this implies µpf q 1{. Give such a set F ad also a iteger i P t1, 2,..., 1u, we have S F X S i F S i`s i F X F H, ad so S F is disjoit from SF Y S 1 F. By the secod part of (3), we therefore have S F Ď F modulo µ, ad hece i fact S F F modulo µ because these sets have equal measure. It follows that, for µ-a.e. x P A Z, the 2

3 orbit ps xq PZ visits the set F exactly every time-steps. So we ca regard this set F as a clock, drive by the source itself, which rigs every time-steps. Equivaletly: The t0, 1u-valued process p1 F ps xqq PZ almost surely produces isolated 1 s separated by itervals of 1 0 s. Now we ca use the times of the 1 s produced by this process to mark the starts of the blocks to which we apply Φ. That is, we defie a ew map ϕ : A Z ÝÑ A Z with referece to the process p1 F S q as follows: x x x 1 x 2 p1 F ps xqq ϕpxq Φpxq Φpx 1 q Φpx 2 q This solves our previous problem: if we shift the iput strig x by S, the we also shift the sequece of block markers p1 F ps xqq, ad so we ed up applyig Φ to the correctly-shifted sequece of -blocks, so that ϕpsxq Sϕpxq. However, it also itroduces a ew problem. I this ew map, we apply Φ to a -block px t,..., x t` 1 q of x P A Z oly whe the start-time t of that block satisfies S t x P F. This meas that the fidelity criterio (1) is o loger eough to imply that µtϕ 0 pxq x 0 u ď δ. Istead we have the followig calculatio. Lemma 1. For Φ, F ad ϕ as above, we have µtϕ 0 pxq x 0 u d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx F q. (4) Proof. The sets S i F for i 0, 1,..., p 1q, are a partitio of A Z ito sets of measure 1{, so the law of total probability gives µtϕ 0 pxq x 0 u 1 For each i, statioarity gives µpϕ 0 pxq x 0 S i F q. µptx : ϕ 0 pxq x 0 u X S i F q µps i tx : ϕ 0 pxq x 0 u X F q µpty : ϕ 0 ps i yq ps i yq 0 u X F q µpty : ϕ i pyq y i u X F q, ad hece µpϕ 0 pxq x 0 S i F q µpϕ i pxq x i F q. 3

4 Therefore µtϕ 0 pxq x 0 u 1 µpϕ i pxq x i F q 1 1 ÿ 1 tϕi pxq x i u µpdx F q d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx F q. Thus, to cotrol the error probability µtϕ 0 pxq x 0 u, we ow eed cotrol over the coditioal itegral i (4), rather tha (1). Happily, this ca easily be obtaied from (1) itself. The poit is simply that, if F satisfies (3), the so do all of its images S i F (exercise!). O the other had, sice the images i (3) form a partitio modulo µ, we have d px, Φpxqq µ pdxq d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdxq, 1 d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx S i F q. Therefore, if we have (1), the at least oe of the images S i F satisfies the aalog of (4). So we ca simply replace F with S i F throughout the above costructio. This essetially completes the proof of (2): the oly poit remaiig is to show that this ew process ϕ has etropy at most F pδq. This is fairly easy, but we set it aside, because a slightly more geeral result will appear later i the lecture. 2 Rokhli s lemma May sources do ot have ay set which satisfies (3), so we caot use the above costructio. However, we ca make do with a slightly weaker approximate versio which does hold for all atomless, ergodic MPSs. This fact is the mai techical result of today s lecture. Defiitio 2. Let px, µ, T q be a MPS, ad let P N ad ε ą 0. A p, εq-rokhli set for px, µ, T q is a measurable set U Ď X such that U, T U,..., T 1 U are disjoit ad µpu Y T U Y Y T 1 Uq ą 1 ε. (5) 4

5 Lemma 3 (The Kakutai Rokhli lemma). A ergodic MPS px, µ, T q o a atomless probability space has p, εq-rokhli sets for every ad ε. Proof. Step 1. First we show that, for ay N P N there is a measurabe subset F with µpf q ą 0 ad such that F, T F,..., T N F are disjoit. To see this, start with ay measurable set G whose measure is positive but less tha 1{pN ` 1q. By the orm ergodic theorem 1 m mÿ 1 G pt i xq 1 mÿ 1 T m i Gpxq ÝÑ µpgq i 1 i 1 i } } 1 as m ÝÑ 8. I particular, this implies that µ-a.e. x visits G ifiitely may times durig its orbit (sice for ay x with oly fiitely may visits, the lefthad side above teds to zero). Therefore Ť iě0 T i G has full µ-measure. Now let F : GzpT G Y Y T N Gq. This has the disjoitess property by costructio; we must oly show that it has positive measure. Suppose ot. The G Ď T G Y Y T N G modulo µ. This implies that T 1 G Ď T 1 pt G Y Y T N Gq G Y Y T N 1 G Ď T G Y Y T N G modulo µ, ad ow a simple iductio gives T i G Ď T G Y Y T N G for all i P N. However, the right-had side has measure at most N{pN `1q, which cotradicts our previous coclusio that Ť iě0 T i G has full measure. Step 2. Let N ě maxt, 1{εu, ad let F be a set as give by Step 1. For each x P X let τ F pxq : mit ě 0 : T x P F u (the first visit time of x to F ). This is well-defied ad fiite for a.e. x, by the same ergodic-theorem argumet as i Step 1. Now defie U : tx : τ F pxq ě ad τ F pxq 0 mod u. 5

6 If τ F pxq ě 1, the τ F pt xq τ F pxq 1. It follows that U, T U,..., T 1 U are disjoit. The set U is also oempty, sice (for example) it cotais T F by the disjoitess of F,..., T N F ad the fact that ď N. Lastly, we have U Y T U Y Y T 1 U tx : τ F pxq ě 1u XzF modulo µ, so this has measure at least 1 1{pN ` 1q ą 1 ε. Give a p, εq-rokhli set U, let be a placeholder symbol, ad defie a t0, 1, u-process pτ q by settig $ & 1 if x P U τ 0 pxq : 0 if x P T U Y Y T 1 U % (some ew placeholder symbol) otherwise. Sice U is p, εq-rokli, this has the followig properties: 1. For µ-a.e. x, the strig pτ pxqq cosists of isolated 1 s, each of them followed by 1 0 s ad the followed by a ukow umber of s. 2. µtτ 0 1u µpuq ą p1 εq{. We call pτ q the p, εq-rokhli process associated to U. Ay process with these two properties arises this way from the Rokhli set tτ 0 1u. 3 The block-to-statioary costructio To fiish the lecture, let us go back over the costructio of Sectio 1, makig the ecessary modificatios to use a Rokhli set istead of a periodic oe. Assume that ra Z, µs is ergodic ad atomless. Thus, fix P N ad let Φ : A ÝÑ A be a -block ecoder. For our ratedistortio applicatio it is chose to achieve the rate F pδq. Let ε ą 0, let U be a p, εq-rokhli set, ad let τ pτ q be the associated t0, 1, u-valued Rokhli process. Also, fix arbitrarily some distiguished letter b P A. We ow costruct a statioary code ϕ : A Z ÝÑ A Z usig Φ ad τ as follows: x x y x 1 y 1 x 2 τpxq ϕpxq Φpxq bb b Φpx 1 q bb b Φpx 2 q looomooo loomoo looomooo loomoo looomooo variable legth 6 variable legth

7 This procedure for costructig a statioary code from a Rokhli process ad a block code is very geeral. We call it the block-to-statioary costructio. Here is the aalog of Lemma 1 for this costructio. Lemma 4. For Φ, U ad ϕ as above, we have µtϕ 0 pxq x 0 u ă d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx Uq ` ε. Proof. Let W : XzpU Y SU Y Y S p 1q Uq. The µpw q ă ε, µps i Uq ď 1{ for each i, ad the sets U, SU,..., S p 1q U form a measurable partitio of X. Therefore µtϕ 0 pxq x 0 u ă 1 1 µpϕ 0 pxq x 0 S i UqµpS i Uq ` µpϕ 0 pxq x 0 W qµpw q ÿ µpϕ 0 pxq x 0 S i Uq ` ε. As i the proof of Lemma 1, statioarity gives so the above becomes µpϕ 0 pxq x 0 S i Uq µpϕ i pxq x i Uq, 1 1 ÿ 1 tϕi pxq x i u µpdx Uq`ε µtϕ 0 pxq x 0 u ă 1 µpϕ i pxq x i Uq`ε d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx Uq ` ε. We also eed to cotrol the etropy of the process produced by a block-tostatioary costructio. Lemma 5. Let ϕ : A Z ÝÑ A Z be produced by the block-to-statioary costructio for some block map Φ : A ÝÑ A ad p, εq-rokhli process τ. The hpϕq : hpϕ µq ď Hpε ` 1{, 1 ε 1{q ` pε ` 1{q log 2 ` 1 log ΦpA q. 7

8 Proof. By the chai rule ad mootoicity for coditioal etropy rates, we have hpϕq ď hpτq ` hpϕ τq. The evet SU Y Y S 1 U has measure greater tha 1 ε 1{, ad o this evet the observable τ 0 equals zero. Therefore Fao s iequality for processes gives hpτq ď Hpε ` 1{, 1 ε 1{q ` pε ` 1{q log 2. O the other had, cosider the coditioal Shao etropy for some large iteger N. It is at most Hpϕ 1,..., ϕ N τ 1,..., τ N q Hpϕ,..., ϕ N τ 1,..., τ N q ` 2 log A, simply by the chai rule. However, if we kow the strig τ 1 pxq,..., τ N pxq, the we kow exactly where the distiguished -blocks of our costructio itersect the discrete iterval t,..., N u. The umber of those -blocks is at most N{, ad for each of them the umber of possible strigs produced by Φ is at most ΦpA q. Outside of those distiguished -blocks, the process ϕ always takes a fixed value b, so has o remaiig radomess i those coordiates. Therefore Hpϕ,..., ϕ N τ 1,..., τ N q! possible strigs ď log ˇ pϕ pxq,..., ϕ N pxqq )ˇˇˇ µpdxq compatible with pτ 1 pxq,..., τ N pxqq ď N log ΦpA q. Dividig by N ad lettig N ÝÑ 8, this completes the proof. Proof of (2). Let P N ad let Φ : A ÝÑ A be a ecoder satisfyig (1). Let ε ą 0, ad let U be a p, εq-rokhli set. It satisfies µpu q ą 1 ε. Let W : XzpU Y Y S p 1q Uq. Sice the sets U,..., S p 1q U ad W are a partitio, we have d px, Φpxqq µ pdxq ` W ě µpuq S i U d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdxq d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdxq d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx S i Uq, 8

9 where we simply drop the itegral over W. Re-arragig, we obtai 1 d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx S i Uq ď 1 µpuq d px, Φpxqq µ pdxq ă δ 1 ε. Therefore, by Markov s iequality, there is a i P t0, 1,..., 1u for which 1 d `px0,..., x 1 q, Φpx 0,..., x 1 q µpdx S i Uq ă δ 1 ε. Replacig U by S i U, this is still a p, εq-rokhli set, so we may ow assume that the above estimate holds with i 0. Havig made this assumptio, let τ be Rokhli process associated to U, ad costruct ϕ from Φ ad τ as above. By Lemma 4, we have Fially, Lemma 5 shows that µtϕ 0 pxq x 0 u ă δ 1 ε ` ε. hpϕ µq ď 1 log ΦpA q ` pε ` 1{q. Takig ε sufficietly small ad lettig ÝÑ 8, this completes the proof. 4 Notes ad remarks See [Hal60, p71] or [EW11, Lemma 2.45] for the stadard proof of Rokhli s lemma. Refereces [EW11] Mafred Eisiedler ad Thomas Ward. Ergodic theory with a view towards umber theory, volume 259 of Graduate Texts i Mathematics. Spriger-Verlag Lodo, Ltd., Lodo, [Hal60] Paul R. Halmos. Lectures o ergodic theory. Chelsea Publishig Co., New York,

10 TIM AUSTIN URL: math.ucla.edu/ tim 10