CHAPTER 7. COMPLEXITY OF ENCODER 187 Ch. 1], the complexity is usually measured by the space requirements and the running time of the program. Encoder

Transcription

1 Chapter 7 Complexity of Encoders 7.1 Complexity criteria There are various criteria that are used to measure the performance and complexity of encoders, and their corresponding decoders. We list here the predominant factors that are usually taken into account while designing rate p : q nite-state encoders. The values of p and q. Typically, the rate p=q is chosen to be as close to cap() as possible, subject to having p and q small enough: The reason for the latter requirement is minimizing the number of outgoing edges, 2 p, from each state in the encoder and keeping to a minimum the number of input{output connections of the encoder. Number of states in an encoder. In both hardware and software implementation of encoders E, we will need dlog jv E je bits in order to represent the current state of E. This motivates an encoder design with a relatively small number of states [Koh78, Ch. 9]. Gate complexity. In addition to representing the state of a nite-state encoder, we need, in hardware implementation, to realize the next-state function and the output function as a gate circuit. Hardware complexity is usually measured in terms of the number of required gates (e.g., NAND gates), and this number also includes the implementation of the memory bit cells that represent the encoder state (each memory bit cell can be realized by a xed number of gates). The number of states in hardware implementation becomes more signicant in applications where we run several encoders in parallel with a common circuit for the next-state and output functions, but with duplicated hardware for representing the state of each encoder. Time and space complexity of a RAM program. When the nite-state encoder is to be implemented as a computer program on a random-access machine (RAM) [AHU74, 186

2 CHAPTER 7. COMPLEXITY OF ENCODER 187 Ch. 1], the complexity is usually measured by the space requirements and the running time of the program. Encoder anticipation. One way to implement a decoder for an encoder E with nite anticipation A(E) is by accumulating the past A(E) symbols (in ( q )) that were generated by E these symbols, with the current symbol, allow the decoder to simulate the state transitions of E and, hence, to reconstruct the sequence of input tags (see ection 4.1). The size of the required buer thus depends on A(E). Window length of sliding-block decodable encoders. A typical decoder of an (m a)- sliding-block decodable encoder consists of a buer that accumulates the past m+a symbols (in ( q )) that were generated by the encoder. A decoding function D : (( q )) (m+a+1)! f0 1 ::: 2 p ;1g is then applied to the current symbol and to the contents of the buer to reconstruct an input tag in f0 1g p (see ection 4.3). From a complexity point-of-view, the window length, m+a+1, determines the size of the required buer. In order to establish a general framework for comparing the complexity of encoders generated by dierent methods of encoder synthesis, we need to set some canonical presentation of the constrained system, in terms of which the complexity will be measured. We adopt the hannon cover of as such a distinguished presentation. 7.2 Number of states in the encoder In this section we present upper and lower bounds on the smallest number of states in any ( n)-encoder for agiven constrained system and integer n. Let G be a deterministic presentation of. As was described in Chapter 5, the statesplitting algorithm [ACH83] starts with an (A G n)-approximate eigenvector x = (x v ) v2vg, which guides the splitting of the states in G until we obtain an ( n)-encoder with at most P v2v G x v = kxk 1 states. Hence, we have the following. Theorem 7.1 Let be a constrained system presented by a deterministic graph G and let n be apositive integer. Assume that cap() log n. Then, there exists an ( n)-encoder E such that jv E j min kxk 1 : x2x (A G n) On the other hand, the following lower bound on the number of states of any ( n)- encoder was obtained in [MR91].

3 CHAPTER 7. COMPLEXITY OF ENCODER 188 Theorem 7.2 Let be a constrained system presented by a deterministic graph G and let n be a positive integer. Assume that cap() log n. Then, for any ( n)-encoder E, jv E j min kxk 1 : x2x (A G n) Proof: Let E be an ( n)-encoder and let = (). The following construction eectively provides an (A G n)-approximate eigenvector x which satises the inequality jv E jkxk 1. (a) Construct a deterministic graph H = H(E) which presents 0 = (E). This can be done using the determinizing graph of ection (b) For an irreducible sink H 0 of H, dene a vector 6= 0 such that A H 0 = n. Let H 0 be an irreducible sink of H. Recall that each state Z 2 V H 0 VH is a subset T E (w v) of states of E that can be reached in E from a given state v 2 V E by paths that generate a given word w. Let Z = jzj denote the number of states of E in Z and let be the positive integer vector dened by =( Z ) Z2VH 0. We now claim that A H 0 = n : Consider a state Z 2 V H 0. ince E has out-degree n, thenumber of edges in E outgoing from the set of states Z V E is njzj. Now, let E a denote the set of edges in E labeled a that start at the states of E in Z and let Z a denote the set of terminal states, in E, of these edges. Note that the sets E a, for a 2, induce a partition on the edges of E outgoing from Z. Clearly, if Z a 6=, there is an edge Z a! Z a in H and, since H 0 is an irreducible sink, this edge is also contained in H 0. We now claim that any state u 2 Z a is accessible in E by exactly one edge labeled a whose initial state is in Z otherwise, if Z = T E (w v), the word wa could be generated in E by two distinct paths which start at v and terminate in u, contradicting the losslessness of E. Hence, je a j = jz a j and, so, the entry of A H 0 corresponding to the state Z in H 0 satises as desired. (A H 0) Z = X Y 2V H 0 = X a2 (A H 0) Z Y Y = X jz a j = X a2 Y 2V H 0 (A H 0) Z Y jy j je a j = njzj = n Z (c) Construct an (A G n)-approximate eigenvector x = x(e) from. As G and H 0 comply with the conditions of Lemma 2.13, each follower set of a state in H 0 is contained in a follower set of some state in G. Let x = (x u ) u2vg be the nonnegative integer vector dened by x u = max f Z : Z 2 V H 0 and F H 0(Z) F G (u) g u 2 V G

4 CHAPTER 7. COMPLEXITY OF ENCODER 189 and denote by Z(u) some particular state Z in H 0 for which the maximum is attained. In case there is no state Z 2 V H 0 such that F H 0(Z) F G (u), dene x u = 0 and Z(u) =. We claim that x is an (A G n)-approximate eigenvector. First, since V H 0 is nonempty, we have x 6= 0. Now, let u be a state in G if x u = 0 then, trivially, (A G x) u nx u and, so, we can assume that x u 6= 0. Let Z a (u) be the terminal state in H 0 for an edge labeled a outgoing from Z(u). ince F H 0 Z(u) F G (u), there exists an edge labeled a in G from u which terminates in some state u a in G and, since G and H 0 are both deterministic, we have F H 0 Z a (u) F G (u a ). Furthermore, by the way x was dened, we have x ua Za(u) and, so, letting Z(u) denote the set of labels of edges in H 0 outgoing from Z(u), we have (A G x) u X a2 Z(u) x ua X a2 Z(u) Za(u) =(A H 0) Z(u) = n Z(u) = nx u where we have used the equality A H 0 = n. Hence, A G x nx. The theorem now follows from the fact that each entry in x is a size of a subset of states of V E. The bound of Theorem 7.2 can be eectively computed by thefranaszek algorithm which was described in ection The upper bound of Theorem 7.1 is at most jv G j times the lower bound of Theorem 7.2, which amounts to an additive term of log jv G j in the number of bits required to represent the current state of the encoder. There are examples of sequences of labeled graphs G for which the lower bound of Theorem 7.2 is exponential in the number of states of G [Ash88], [MR91]. We give here such an example, which appears in [AMR95] and [MR91]. Example 7.1 Let r be a positive integer and let denote the alphabet of size r 2 +r+1 given by fag[fb i g r2 +r;1 i=1 [fcg. Consider the constrained systems k that are presented by the graphs G k of Figure 7.1 (from each state u k there are r 2 +r;1 parallel outgoing edges labeled by the b i 's to state k+u). It is easy to verify that (A Gk )= = r+1 and that every (A Gk )-approximate eigenvector is a multiple of ( 2 ::: k 1 ::: k;1 ) >. Hence, by Theorem 7.2, every ( k r+1)-encoder must have at least (r+1) k = expfo(jv Gk j)g states. On the other hand, note that the vector x =(x u ) u, whose nonzero components are x k = r and x 2k = 1, is an (A Gk r)-approximate eigenvector. Hence, if we can compromise on the rate and construct ( k r)-encoders instead, then the state-splitting algorithm provides such encoders with at most r+1 states. The bound of Theorem 7.2 is based on the existence of an approximate eigenvector x = x(e), where each of the entries in x is a size of a subset of V E. The improvements on this bound, given in [MR91], are obtained by observing that some of these subsets might be disjoint. One such improvement (with proof left to the reader) is as follows.

5 CHAPTER 7. COMPLEXITY OF ENCODER 190 ' $? c a a k;1 a k fb i g c fb i g c fb i g a fb i g c fb i g c & ? w? w? w? w? % k+1 - k+2 - k+3-2k;1-2k c c c Figure 7.1: Labeled graph G k for Example 7.1. Theorem 7.3 Let be a constrained system presented by a deterministic graph G and let n be a positive integer. Assume that cap() log n. Then, for any ( n)-encoder E, X jv E j min x2x (A G n) where the maximum is taken over all subsets U V G such that F G (u)\f G (u 0 )= for every distinct states u and u 0 in U. max U u2u x u In fact, the preceding result can be generalized further to obtain the best general lower bound known on the number of states in any ( n)-encoder, as it is stated in [MR91]. In order to state this result, we need the following denitions. Let be a constrained system presented by a deterministic graph G. For a state u 2 V G and a word w 2F G (u), let G (w u)bethe terminal state of the path in G that starts at u and generates w. (Using the notations of ection 2.2.1, wethus have T G (w u)=f G (w u)g.) For aword w 62F G (u), dene G (w u)=. Let n be a positive integer and x =(x u ) u2vg be an (A G n)-approximate eigenvector. For a word w and a subset U V G, let I G (x w U) denote a state u 2 U such that x G (w u) is maximal (for the case where G (w u)=, we dene x =0). Let U be a subset of V G. A list C of words is U-complete in G, ifevery word in u2u F G (u) either has a prex in C or is a prex of a word in C. Let C G (U) denote the set of all nite U-complete lists in G. For example, the list F m G (U) of all words of length m that can be generated in G from states of U, belongs to C G (U). Finally, given an integer n, an(a G n)-approximate eigenvector x, a subset U of V G, and

6 CHAPTER 7. COMPLEXITY OF ENCODER 191 a list C of words, we dene G (x n U C) by G (x n U C)= X u2u x u ; X w2c (recall that `(w) is the length of the word w). X n ;`(w) x G (w u) u2u;fi G (x w U)g Theorem 7.4 Let be a constrained system presented by a deterministic graph G and let n be a positive integer. Assume that cap() log n. Then, for any ( n)-encoder E, jv E j min In particular, for all U V G and m, max y2x (A G n) UV G jv E j sup G (y n U C) : C2C G (U) min G (y n U F m G (U)) : y2x (A G n) Example 7.2 Figure 4.6 depicts a rate 2:3four-state encoder for the (1 7)-RLL constrained system. The example therein is due to Weathers and Wolf [WW91], whereas the example of an encoder used in practice is due to Adler, Hassner, and Moussouris [AHM82] and has ve states (see also [How89]). Using Theorem 7.4, a lower bound of 4 on the number of states of any such encoder is presented in [MR91]. Thus, the Weathers{Wolf encoder has the smallest possible number of encoder states. Example 7.3 Figure 4.4 depicts a rate 1 : 2 six-state encoder for the (2 7)-RLL constrained system. This encoder is used in practice and is due to Franaszek [Fra72] (see also [EH78], [How89]). On the other hand, the encoder shown in Figure 4.5, which is due to Howell [How89], has only ve states. Using Theorem 7.4, it can be shown that 5 is a lower bound on the number of encoder states for this system thus, the Howell encoder has the smallest possible number of encoder states (note, however, that the anticipation of the Howell encoder is larger than Franaszek's). 7.3 Values of p and q When cap() is a rational number p=q, we can attain the bound of Theorem 4.2 by a rate p : q nite-state encoder for : Taking a deterministic presentation G of, we have in this case an (A q G 2p )-approximate eigenvector which yields, by the state-splitting algorithm, an ( q 2 p )-encoder. On the other hand, when cap() is not a rational number, we cannot attain the bound of Theorem 4.2 by a rate p : q nite-state encoder for. till, we can approach the bound

7 CHAPTER 7. COMPLEXITY OF ENCODER 192 cap() from below by a sequence of rate p m : q m nite-state encoders E m. In fact, as stated in Theorem 4.3, we can approach capacity from below even by block encoders. It can be shown that in this way we obtain a sequence of rate p m : q m block encoders E m for such that p m q m ; cap() for some constant = (G). However, the constant might be very large (e.g., exponential) in terms of the number of states of G. This means that q m might need to be extremely large in order to have rates p m =q m close to capacity. Obviously, for every sequence of rate p m : q m nite-state encoders E m for, the number of edges in E m is increasing exponentially with p m. The question is whether convergence of p m =q m to cap() that is faster than O(1=q m )might force the number of states in E m to blow up as well. The answer is given in the following result, which is proved in [MR91] using Theorem 7.2. q m Theorem 7.5 Let be a constrained system with cap() =log. (a) If = k s=t for some positive integers k, s, and t, then there exists an integer N such that for any two positive integers p, q, where p=q log and t divides q, there is an ( q 2 p )-encoder with at most N states. (b) If is not a rational power of an integer and, in addition, lim m!1 p m q m ; log then for any sequence of ( qm 2 pm )-encoders E m,! lim m!1 jv E m j = 1 : q m =0 ketch of proof. Case (a): Let G be a deterministic presentation of. ince = k s=t, the matrix (A G ) t has an integer largest eigenvalue and an associated integer nonnegativeright eigenvector x. An ( tm k sm )-encoder E m can therefore be obtained by the state-splitting algorithm for every m, with number of states which is at most N = kxk 1. Write q = tm we have 2 p k sm and, so, an ( q 2 p )-encoder can be obtained by deleting excess edges from E m. Case (b): It can be shown that if the values p m =q m approach log faster than O(1=q m ), then the respective ((A G ) qm 2 pm )-approximate eigenvectors x (when scaled to have a xed norm) approach a right eigenvector which must contain an irrational entry. Therefore, the largest components in such approximate eigenvectors tend to innity. The result then follows from Theorem 7.2.

8 CHAPTER 7. COMPLEXITY OF ENCODER 193 If we choose p m and q m to be the continued fraction approximants of log, we get p m ; log q m < for some constant. o, in case (b), the fastest approach to capacity necessarily forces the number of states to grow without bound. q 2 m 7.4 Encoder anticipation Deciding upon existence of encoders with a given anticipation We start with the following theorem, taken from [AMR96], which shows that checking whether there is an ( n)-encoder with anticipation t is a decidable problem. A special case of this theorem, for t =0,was alluded to in ection 4.4. Recall that FG(u) t stands for the set of words of length t that can be generated from a state u in a labeled graph G. Theorem 7.6 Let be an irreducible constrained system with a hannon cover G, let n and t be positive integers, and, for every state u in G, let N(u t) =jf t G(u)j. If there exists an ( n)-encoder with anticipation t, then there exists an ( n) encoder with anticipation t and at most P u2v G (2 N(u t) ; 1) states. By Lemma 2.9, we may assume that there is an irreducible ( n)-encoder E with anticipation at most t. The proof of Theorem 7.6 is carried out by eectively constructing from E an ( n)-encoder E 0 with anticipation t and with a number of states which isatmostthe bound stated in the theorem. We describe the construction of E 0 below, and the theorem will follow from the next two lemmas. For a state u 2 V G and a nonempty subset F of F t G(u), let ;(u F) denote the set of all states v in E for which F E (v) F G (u) and F t E(v) =F. Whenever ;(u F) is nonempty we designate a specic such state v 2 ;(u F) and call it v(u F). By Lemma 2.13, at least one ;(u F) is nonempty. We nowdenethe labeled graph E 0 as follows. The states of E 0 are the pairs (u F) such that ;(u F) is nonempty. We draw an edge (u F) a! (bu b F) in E 0 if and only if there is an edge u a! bu in G and an edge v(u F) a! bv in E for some bv 2 ;(bu b F). Lemma 7.7 For every ` t+1, F E ` (u F) = F ` 0 E v(u F) :

9 CHAPTER 7. COMPLEXITY OF ENCODER 194 Proof. We prove that FÈ0 (u F) FÈ v(u F) by induction on `. reverse inclusion (which is not used here) to the reader. We leave the The result is immediate for ` = 0. Assume now that the result is true for some xed ` t. Let w 0 w 1 :::w` 2 F E `+1 (u F), which implies that there is in E 0 a path of the 0 form (u F) w 0! (u 1 F 1 ) w 1! (u 2 F 2 )! :::! (u` F`)! w` (u`+1 F`+1 ). By the inductive hypothesis, there is a path v(u 1 F 1 ) w 1 w! 2 w` v 2! v 3! :::! v`! v`+1 in E. Therefore, the word w = w 1 w 2 :::w` belongs to F E v(u 1 F 1 ) and, since ` t, we can extend w to form a word ww 0 of length t that belongs to F 1. Now, by denition of the edges in E 0, there is an edge v(u F) w 0! bv in E for some bv 2 ;(u 1 F 1 ). ince ww 0 2 F 1, there is a path labeled w outgoing from bv in E and, so, there is a path labeled w 0 w 1 :::w` outgoing from v(u F) ine. Hence, F E `+1 (u F) F`+1 0 E v(u F), as desired. The next lemma shows that E 0 is an ( n)-encoder with anticipation t. Lemma 7.8 The following three conditions hold: (a) The out-degree of each state in E 0 is n (b) (E 0 ) and (c) E 0 has anticipation t. Proof. Part (a): It suces to show that there is a one-to-one correspondence between the outgoing edges of (u F) in E 0 and those of v(u F) ine. Consider the mapping from outgoing edges of (u F) to outgoing edges of v(u F) dened by (u F)! a (bu F) b = v(u F)! a bv where bv 2 ;(bu b F). To see that is well-dened, observe that since E has anticipation at most t, there cannot be two distinct edges v(u F) a! bv and v(u F) a! bv 0 with bv and bv 0 both belonging to the same ;(bu b F). To see that is onto, rst consider an outgoing edge v(u F) a! bv from v(u F), and note that since F F G (u), there is in G an outgoing edge u a! bu for some bu. Let b F = F t E(bv). We claim that bv 2 ;(bu b F). Of course F t E(bv) = b F and since F E (v(u F)) F G (u) andg is deterministic, F E (bv) F G (bu). Thus, by denition of E 0 there is an edge (u F) a! (bu b F). We thus conclude that is onto. ince u and a determine bu and since bv determines b F, it follows that is 1{1. This completes the proof of (a). Part (b): By denition of E 0, we see that whenever there is a path (u 0 F) w 0! (u 1 F) w 1! (u 2 F)! :::! (u`;1 F) w`;1! (u` F) ine 0 w, there is also a path 0 w u 0! 1 w`;1 u 1! :::! u`;1 u` in G. Thus (E 0 ) (G) =, as desired. Part (c): We must show that the initial edge of any path of length t+1 in E 0 is determined by its label w 0 w 1 :::w t and its initial state (u F). Write the initial edge of as:!

10 CHAPTER 7. COMPLEXITY OF ENCODER 195 (u F) w 0! (bu F). b By Lemma 7.7, there is a path in E with label w 0 w 1 :::w t that begins at state v(u F). ince E has anticipation at most t, the label sequence w 0 w 1 :::w t and v(u F) determine the initial edge v(u F) w 0! bv of this path. o, it suces to show that u, w 0,andbv determine bu and F b for then (u F) andw 0 w 1 :::w t will determine the initial edge of. Indeed, by denition of E 0, there must be an edge u w 0! bu in G such that bv 2 ;(bu F). b ince G is deterministic, u and w 0 determine bu. Furthermore, for any xed bu, the sets ;(bu G) are disjoint for distinct G, and, so, bv determines F. b It follows that u, w 0, and bv determine bu and F, b as desired, thus proving (c). Now, for every state u 2 V G,thenumber of distinct nonempty subsets ;(u F) is bounded from above by2 N(u t) ;1. This yields the desired upper bound of Theorem 7.6 on the number of states of E 0. It follows by Theorem 7.6 that in order to verify whether there exists an ( n)-encoder with anticipation t, we can exhaustively check all irreducible graphs E with labeling over (), with out-degree n, and with number of states jv E j which is at most the bound of Theorem 7.6. Checking that such a labeled graph E is an ( n)-encoder can be done by the following nite procedure: Construct the determinizing graph H of E as in ection ince E is irreducible, the states of any irreducible sink H 0 of H, as subsets of V E, must contain all the states of E. Hence, we must have (E) = (H 0 ). Then, we verify that (G H 0 )=(H 0 ) to this end, it suces, by Lemma 2.9, to check that (H 0 )ispresented by an irreducible (deterministic) component G 0 of G H 0. The equality (H 0 )=(G 0 ), in turn, can be checked by Theorem 2.12, using the Moore algorithm of ection Finally, testing whether E has anticipation t can be done by the ecient algorithm described in ection By the Moore co-form construction of ection 2.2.7, the existence of such an encoder implies the existence of an ( n)-encoder with anticipation exactly t Upper bounds on the anticipation Continuing the discussion of ection 7.4.1, we now obtain more tractable upper and lower bounds on the smallest attainable anticipation of ( n)-encoders in terms of n and a deterministic presentation of the constrained system. Let G be a deterministic presentation of. The anticipation of an encoder obtained by the state-splitting algorithm [ACH83] is bounded from above by the number of splitting rounds. This, in turn, yields the following result, which is, so far, the best general upper bound known for the anticipation obtained by direct application of the state-splitting algorithm. Theorem 7.9 Let be a constrained system presented by a deterministic graph G and let n be apositive integer. Assume that cap() log n. Then, there exists an ( n)-encoder

11 CHAPTER 7. COMPLEXITY OF ENCODER 196 E, obtained by the state-splitting algorithm, such that, A(E) min x2x (A G n) n kxk1 ; w(x) o where w(x) is the number of nonzero components in x. Furthermore, if G has nite memory, then E is (M(G) A(E))-denite. This bound is quite poor, since it may be exponential in jv G j, as is, indeed, the case for the constrained systems of Example 7.1. On the other hand, if G has nite memory, then the encoder E obtained by the state-splitting algorithm is guaranteed to be denite. Now, suppose that G t can be split fully in one round that is, the splitting yields a labeled graph E 1 with out-degree n t at each state. By deleting excess edges, E 1 can be made an ( t n t )-encoder E 2 with anticipation 1 over ( t ). Let E 3 be the Moore co-form of E 2 as in ection Then E 3 is an ( t n t )-encoder with anticipation 2. If we replace the n t outgoing edges from each state in E 3 byann-ary tree of depth t, we obtain an ( n)-encoder E 4 with anticipation 3t;1. Therefore, we have the following. Theorem 7.10 Let be a constrained system presented by a deterministic graph G and let n and t be positive integers. uppose that G t can be split in one round, yielding a labeled graph with minimum out-degree atleast n t. Then, there isan( n)-encoder with anticipation 3t;1. In [Ash87b] and [Ash88], Ashley shows that for t = O(jV G j), G t can be split in one round, yielding a labeled graph with minimum out-degree at least n t moreover, the splitting can be chosen to be x-consistent with respect to any (A G n)-approximate eigenvector x. This provides encoders with anticipation which is at most linear in jv G j. The following theorem is a statement of Ashley's result. Theorem 7.11 Let be a constrained system presented by a deterministic graph G and let n be apositive integer. Assume that cap() log n. Then, there exists an ( n)-encoder E such that, (a) when n = (A G ), A(E) 9jV G j +6dlog n jv G je ; 1 (b) when n<(a G ), A(E) 15jV G j +3dlog n jv G je ; 1 :

12 CHAPTER 7. COMPLEXITY OF ENCODER 197 Note, however, that the encoders obtained by splitting the tth power of G are typically not sliding-block decodable when t > 1, even when G has nite memory. A further improvement on the upper bound of the smallest attainable anticipation is presented in [AMR95], using the stethering method which, in turn, is based on an earlier result by Adler, Goodwyn, and Weiss [AGW77] (see Chapter 6). The following result applies to the case where n (A G ) ; 1. Theorem 7.12 Let be a constrained system presented by a deterministic graph G and let n be a positive integer (A G ) ; 1. Then, there is an ( n)-encoder E, obtained by the (punctured) stethering method, such that A(E) 1+ min x2x (A G n+1) n dlogn+1 kxk 1 e o : Furthermore, if G has nite memory, then E is (M(G) A(E))-denite, and hence any tagged ( n)-encoder based on E is (M(G) A(E))-sliding-block decodable. In particular, when n (A G ) ; 1, there always exists an (A G n+1)-approximate eigenvector x such that kxk 1 (n+1) 2jV Gj [Ash87a], [Ash88]. Hence, we have the following. Corollary 7.13 Let be a constrained system presented by a deterministic graph G and let n be a positive integer (A G ) ; 1. Then, there is an ( n)-encoder E, obtained by the (punctured) stethering method, such that A(E) 2jV G j +1: Furthermore, if G has nite memory, then E is (M(G) 2jV G j+1)-denite, and hence any tagged ( n)-encoder based on E is (M(G) 2jV G j+1)-sliding block decodable. In terms of rate p : q nite-state encoders, the requirement n (A G ) ; 1isimplied by p q cap() ; 1 2 p q log e 2 namely, we need a margin between the rate and capacity which decreases exponentially with p. Applying the stethering method on a power of G, the following result is obtained in [AMR95]. Theorem 7.14 Let be a constrained system presented by a deterministic graph G and let n be a positive integer smaller than (A G ). Then, there is an ( n)-encoder E such that A(E) 12jV G j;1 : Theorem 7.14 improves on Theorem 7.11, but it does not cover the case n = (A G ). Also, the encoders guaranteed by Theorem 7.14 are typically not sliding-block decodable.

13 CHAPTER 7. COMPLEXITY OF ENCODER Lower bounds on the anticipation The next theorem, taken from [MR91], provides a lower bound on the anticipation of any ( n)-encoder. A special case of this bound appears in [Fra89]. Theorem 7.15 Let be aconstrained system presented bya deterministic graph G and let n be a positive integer. Assume that cap() log n. Then, for any ( n)-encoder E, n o A(E) logn kxk 1 : min x2x (A G n) Proof. The theorem trivially holds if A(E) =1, so we assume that E has nite anticipation A. Let x = x(e) =(x u ) u2vg be as in the proof of Theorem 7.2. We recall that by the way x was constructed, each nonzero component of x is asizeof some subset Z = T E (w v) of states in E which are accessible from v 2 V E by paths labeled w. Let T E (w v) be such a subset whose size equals the largest component of x and let ` = `(w) (i.e., the length of w). ince the out-degree of E is n, we have n` paths of length ` starting at v in E and, so, n` jt E (w v)j =max u2v G x u implying ` Therefore, when A`, we are done. n o min logn (max u2vg y u ) : (y u)u2x (A G n) Assume now that ` > A. ince E has nite anticipation A, the rst `;A edges of any path in E labeled w are uniquely determined, once we know the initial state v. It thus follows that the paths from v to T E (w v) labeled w may dier only in their last A edges. Hence, we can have at most n A such paths. Recalling that the number of such paths is jt E (w v)j, we have n A jt E (w v)j = max x u min max y u u2v G u2v G as claimed. (y u)u2x (A G n) There is some resemblance between the lower bound of Theorem 7.15 and the upper bound of Theorem And, indeed, there are many cases where the dierence between these bounds is at most 1. Note, however, that for the constrained systems k = (G k ) of Example 7.1, we obtain, by Theorem 7.12, an upper bound of jv G k j on the smallest anticipation of any ( k r)-encoder, where the lower bound of Theorem 7.15 equals 2. In fact, this lower bound is tight [AMR95]. The following bound, proved in [AMR96] is, in a way, a converse of Theorem 7.10.

14 CHAPTER 7. COMPLEXITY OF ENCODER 199 Theorem 7.16 Let be an irreducible constrained system presented by an irreducible deterministic graph G and let n and t be positive integers. If there is an ( n)-encoder with anticipation t, then G t can be split in one round, yielding a graph with minimum out-degree at least n t. Proof. Let E be an ( n)-encoder with anticipation t, let =(), and let H be the determinizing graph constructed from E as in ection Recall that each state Z 2 V H is a subset T E (w v) of states of E that can be reached in E from a given state v 2 V E by paths that generate a given word w. Let H 0 be an irreducible sink of H and let x =(x u ) u2vg be the nonnegative integer vector dened in the proof of Theorem 7.2: x u = max fjzj : Z 2 V H 0 and F H 0(Z) F G (u) g u 2 V G in case there is no state Z 2 V H 0 proof of Theorem 7.2, x is an (A G n)-approximate eigenvector. such that F H 0(Z) F G (u), dene x u =0. Then, as in the Let Z = T E (w v) be a state in H 0 and suppose that Z contains two distinct states, z and z 0, of E. First, we claim that there is no word w 0 of length t that can be generated in E from both z and z 0. Otherwise, we would have in E two paths of length `(w)+t, starting at the same state v, with the same labeling ww 0, that do not agree in at least one of their rst `(w) edges. This, however, contradicts the fact that E has anticipation t. For w 0 2 FH t 0(Z), denote by Z w 0 the terminal state in H0 of a path labeled w 0 starting at Z. As we have just shown, a word w 0 2FH t 0(Z) can be generated in E from exactly one state z 2 Z. Therefore, the sets FE(z), t z 2 Z, form a partition of FH t 0(Z). Furthermore, by the losslessness of E, the numberofpathsine that start at z 2 Z and generate w 0 2FE(u) t equals jt E (w 0 z)j = jz w 0j. ince E is an ( n)-encoder, we conclude: X jz w 0j = n t for every z 2 Z : (7.1) w 0 2F t E (z) For each state u 2 V G such that x u 6=0,select some Z = Z(u) 2 V H 0 such that jzj = x u and F H 0(Z) F G (u). Now, the partition ffe(z) t :z 2 Zg of FH t 0(Z) may be regarded as a partition of FG(u) t by appending the complement FG(u)nF t H t 0(Z) to one of the atoms F E(z), t z 2 Z. ince G t is deterministic, this denes a partition P G t(u) = fe G t(u z)g z2z(u) of the outgoing edges from u in G t into jz(u)j = x u atoms. For w 0 2 F E (z), let u 0 denote the terminal state of the edge in G t that begins at u and is labeled w 0. Now, if w 00 2F H 0(Z w 0), then w 0 w 00 2 F H 0(Z) F G (u). ince G is deterministic, this implies that w 00 2 F G (u 0 ). Thus F H 0(Z w 0) F G (u 0 ) and, so, jz w 0j x u 0. This, together with Equation (7.1), shows that the splitting of G t dened by the partition P G t(u) satises the following inequality: X x (e) n t for every u 2 V G and z 2 Z(u) : e2e G t (u z) Hence, the split graph has minimum out-degree at least n t.

15 CHAPTER 7. COMPLEXITY OF ENCODER 200 Theorem 7.16 may be regarded as a lower bound on the anticipation of an encoder. This result, together with Theorem 7.10, shows that by one round of splitting of some power of G, one can obtain an encoder whose anticipation is within a constant factor from the smallest anticipation possible. There are examples which show that neither of the lower bounds in Theorems 7.15, 7.16 implies the other. On the other hand, when cap() = log n, we claim that for irreducible constrained systems the lower bound of Theorem 7.16 implies that of Theorem Indeed, let t denote the bound of Theorem Then for each state u 2 V G there is a partition fe G t(u i)g xu i=1 of the outgoing edges from u in G t such that the vector x =(x u ) u is a positive (A G n)-approximate eigenvector and X e2e G t (u i) x t(e) n t for each (u i) : (7.2) We now claim that (7.2) holds in our case with equality for every (u i). Otherwise, the corresponding splitting would yield an irreducible, lossless presentation of t with minimum out-degree at least n t and at least one state with out-degree greater than n t contradicting the equality cap() = log n. Let u max be a state in G for which x umax = kxk 1. Also, let v be a state with an outgoing edge, in G t,tou max. Then any edge e from v to u max in G t belongs to some E G t(v i) and so the equality P e2e G t (v i) x t(e) = n t implies x umax n t i.e., t log n kxk 1 min y2x (AG n) n logn kyk 1 o. We end this section by mentioning without proof the improvements on Theorems 7.15 and 7.16 that have been obtained in [Ru96] and [RuR01]. Recall that Theorem 5.10 in ection provides a necessary and sucient condition for having ( n)-encoders with anticipation t uch acharacterization also implies a lower bound on the anticipation of ( n)-encoders: given and n, the anticipation any ( n)-encoder is at least the smallest nonnegative integer t for which there exists a presentation G of and an (A G n)-approximate eigenvector x that satisfy conditions (a){(e) of Theorem The following is another result obtained in [Ru96] and [RuR01]. Theorem 7.17 Let be an irreducible constraint, let n be a positive integer where cap() log n, and let G be any irreducible deterministic presentation of. uppose there exists some irreducible ( n)-encoder with anticipation t<1. Then there exists an (A G n)- approximate eigenvector x such that the following holds: (a) kxk 1 n t.

16 CHAPTER 7. COMPLEXITY OF ENCODER 201 (b) For every k =1 2 ::: t, the states of G k can be split in one round consistently with the (A k G nk )-approximate eigenvector x, such that the induced approximate eigenvector x 0 satises kx 0 k 1 n t;k, and each of the states in G k is split into no more than n k states. While Theorem 5.10 gives a necessary and sucient condition on the existence of ( n)- encoders with a given anticipation t, Theorem 7.17 gives only a necessary condition on the existence of such encoders. On the other hand, Theorem 7.17 allows to obtain a lower bound on the anticipation using any irreducible deterministic presentation of in particular the hannon cover of. Therefore, it will typically be easier to compute bounds using Theorem Note that Theorem 7.15 is equivalent to Theorem 7.17(a), while Theorem 7.16 is equivalent to Theorem 7.17(b) for the special case k = t. Examples in [RuR01] show that Theorem 7.17 (and hence Theorem 5.10) yields stronger bounds than these two former results. The results in [RuR01] also imply tight lower bounds in certain practical cases. For example, it is shown therein that any rate 2 : 3 nite-state encoder for the (1 7)-RLL constraint must have anticipation at least 2, and the Weathers-Wolf encoder in 4.6 does attain this bound (and so does the encoder of Adler Coppersmith, and Hassner in [ACH83]). imilarly, any rate 1 : 2 encoder for the (2 7)-RLL constraint must have anticipation at least 3, and this bound is attained by the Franaszek encoder in Figure 4.4. A lower bound of 3 applies also to the anticipation of any rate 2 : 5 encoder for the (2 18 2)-RLL constraint (see Figure 1.12) this bound is tight due to the constructions by Weigandt [Weig88] and Hollmann [Holl95]. 7.5 liding-block decodability The following is the analog of Theorem 7.6 for sliding-block decodable encoders. The special case of block decodable encoders was treated in ection 4.4. Theorem 7.18 Let be anirreducible constrained system with a hannon cover G, and let n be a positive integer and m and a be nonnegative integers. For every state u in G, let N(u a) =jfg(u)j a and let P (u m) be the number of words of length m that can be generated in G by paths that terminate in state u. If there exists an (m a)-sliding-block decodable ( n)-encoder, then there exists such an encoder with at most P u2v G P (u m)(2 N(u a) ; 1) states. Proof. The proof is similar to that of Theorem 7.6. In fact, that proof applies almost verbatim to the case of (0 a)-sliding-block decodable encoders, so we assume here that m is strictly positive. Let E be an irreducible (m a)-sliding-block decodable ( n)-encoder. Also,

17 CHAPTER 7. COMPLEXITY OF ENCODER 202 let H 0 be an irreducible sink of the determinizing graph of E obtained by the construction of ection By construction of H 0, for every path from state v to state bv in E that generates a word w, there is a path in H 0 that generates w, starting at a state Z and terminating in a state b Z, such that v 2 Z and bv 2 b Z. Hence, by Lemma 2.13, there also exists a path in G that generates w, starting at a state u and terminating in a state bu, such that F E (v) F G (u) and F E (bv) F G (bu). It thus follows that for every state bv in E and a word w that can be generated in E by a path terminating in bv, there is a path in G that generates w whose terminal state, bu, satises F E (bv) F G (bu). For a state u 2 V G,aword w of length m that can be generated in G by a path terminating in u, and a nonempty subset F F a G(u), we dene ;(u w F) to be the set of all states v in E which are terminal states of paths in E that generate w and such that F E (v) F G (u) and F a E(v) = F. Note that each state of E is contained in some set ;(u w F) and, so, at least one such set is nonempty. A tagged ( n)-encoder E 0 is now dened as follows. In each nonempty set ;(u w F), we designate a state of E and call it v(u w F). The states of E 0 are triples (u w F) for which ;(u w F) is nonempty. Let u and bu be states in G and let w = w 1 w 2 :::wm and bw = bw 1 bw 2 ::: bwm be two words that can be generated by paths in G that terminate in u and bu, respectively. If ;(u w F) and ;(bu bw F) b are nonempty, then we draw a tagged edge (u w F) s=b! (bu bw F)inE b 0 if and only if the following four conditions hold: (a) bw j = w j+1 for j =1 2 ::: m;1 (b) b = bwm (c) there is a tagged edge v(u w F)! s=b bv in E for some bv 2 ;(bu bw F) b (d) there is an edge u b! bu in G. By the proof of Theorem 7.6, it follows that E 0 is, indeed, an ( n)-encoder and that v(u w F). Furthermore, it can be shown by induction that, for F E a+1 (u 0 w F) = FE a+1 every ` m, the paths of length ` in E 0 that terminate in (u w 1 w 2 :::wm F), all have the same labeling wm;`+1wm;`+2 :::wm. ince the `outgoing picture' including tagging from state (u w F) in E 0 is the same as that from state v(u w F) in E, it follows that E 0 is (m a)-sliding-block decodable. The upper bound on jv E 0j is now obtained by counting the number of distinct states (u w F). The upper bound on the number of states in Theorem 7.18 is doubly-exponential in the decoding look-ahead a. In [AM95], a stronger result is obtained where the upper bound on the number of states is singly-exponential. It is still open whether such an improvement is

18 CHAPTER 7. COMPLEXITY OF ENCODER 203 possible also for the doubly-exponential bound of Theorem 7.6. The following bound is easily veried. Proposition 7.19 Let E be an irreducible (m a)-sliding-block decodable encoder. Then, a A(E). Hence, we can apply the lower bounds on the anticipation which were presented in ection 7.4.3, to obtain lower bounds on the attainable look-ahead of sliding-block decodable encoders (but these do not give lower bounds on the decoding window length, m + a +1, since m may be negative). On the other hand, Theorems 7.9 and 7.12 and Corollary 7.13 provide upper bounds on the look-ahead of encoders obtained by constructions that yield sliding-block decodable encoders for nite-type constrained systems. We remark that Theorem 7.18 implies upper bounds on the the size of encoders which are sliding-block decodable also when m is negative: simply apply the theorem with m = 0. And nally we note that, at least for nite-type constrained systems, Hollmann [Holl96] has given a procedure for deciding if there exists a sliding block decodable ( n)-encoder with a given window length here, the window length L = m + a +1,rather than m and a, is specied. Even for L = 1, this is a non-trivial problem, because one must consider the possibility that a = ;m may be arbitrarily large. 7.6 Gate complexity and time{space complexity In this section, we discuss the gate complexity and time-space complexity of some of the encoding schemes that were mentioned in the previous sections. We start with the timespace complexity criterion, assuming that the encoders are to be implemented as a program on a random-access machine (RAM) [AHU74, Ch. 1]. The results on gate complexity will then follow by known results in complexity theory. We dene an encoding scheme as a function (G q n) 7!E(G q n), that maps a deterministic graph G and integers q and n into an ((G q ) n)-encoder E(G q n). The state-splitting algorithm of [ACH83], the method described by Ashley in [Ash88], and the stethering method of [AMR95] are examples of encoding schemes. For a given encoding scheme (G q n) 7!E(G q n), we can formalize the encoding problem as follows: We are to write an encoding program P on a RAM an input instance to P consists of the following entries: a deterministic graph G over an alphabet, an integer q,

19 CHAPTER 7. COMPLEXITY OF ENCODER 204 an integer n (A q G), a state u of E(G q n), an input tag s 2f0 1 ::: n;1g. For any input instance, the program P computes an output q-block over and the next state of the tagged ((G q ) n)-encoder E(G q n), given we are at state u in E(G q n) and the current input tag is s. Note that in order to perform its function, the program P does not necessarily have to generate the whole graph presentation of E(G q n). We denote by Poly() a xed arbitrary multinomial, whose coecients are absolute constants, independent of its arguments. The following was proved in [AMR95] for the stethering coding scheme and for a variation of Ashley's construction [Ash88]. Theorem 7.20 There exists an encoding scheme (G q n) 7!E(G q n) (such as the one presented in [Ash88] or [AMR95]), for which there is an encoding program P on a RAM that solves the encoding problem in time complexity which is at most Poly jv G j q log jj. In particular, if we now x G, q, and n, we obtain an encoding program that simulates E(G q n) with a polynomial time and space complexity. Theorem 7.20 applies to the constructions covered in Theorems 7.11, 7.12, and In contrast, it is not known yet whether a polynomial encoder can be obtained by a direct application of the state-splitting algorithm. For a positive integer `, denote by I` the set of all possible inputs to P of size `, according to some standard representation of the input. Now, if the time complexity of P on each element of I` is polynomial in `, then for any input size `, there exists a circuit C` with Poly(`) =Poly jv G j q log jj gates that implements P for inputs in I`. Furthermore, such circuits C` are `uniform' in the sense that there is a program on a RAM that generates the layouts of C` in time complexity which is Poly(`). This is a consequence of a known result on the equivalence between polynomial circuit complexity and polynomial RAM complexity of decision problems [T93, Theorem 2.3]. By Theorem 7.20, it thus follows that we can have such a polynomial circuit at hand for both Ashley's construction and the stethering construction. We summarize this in the following theorem. Theorem 7.21 For every constrained system over an alphabet, presented by a deterministic graph G, and for any positive integers q and n (A q G), there exists an ( q n)- encoder that can be implemented by a circuit consisting of Poly jv G j q log jj gates and

20 CHAPTER 7. COMPLEXITY OF ENCODER 205 O(jV G j log n) memory bit-cells. Furthermore, there exists a program on a RAM that generates the layout of such an implementation in polynomial-time. Theorems 7.20 and 7.21 apply also to the decoding complexity of the corresponding encoders. Problems Problem 7.1 Prove Theorem 7.3 by modifying the end of the proof of Theorem 7.2. Problem 7.2 Verify the assertion of Example 7.2. Problem 7.3 Verify the assertion of Example 7.3. Problem 7.4 Let be the constrained system presented by the graph G in Figure Is there a positive integer ` for which there exists a deterministic ( 2` 2`)-encoder? If yes, construct such an encoder otherwise, explain. ' a $ b A d C a a B a D Problem 7.5 Let be the constrained system presented by the graph G in Figure d & b c c - W?? %6 Figure 7.2: Graph G for Problem Compute the capacity of. 2. Compute an (A G 2)-approximate eigenvector in which the largest entry is the smallest possible. 3. For every positive integer `, determine the smallest anticipation of any (` 2`)-encoder.

21 CHAPTER 7. COMPLEXITY OF ENCODER 206 Problem 7.6 Let be the constrained system presented by the graph G in Figure Find the smallest anticipation possible of any ( 2)-encoder. 2. Find the smallest number of states of any ( 2 4)-encoder.