Exact Synchronization for Finite-State Sources

Transcription

1 Santa Fe Institute Woring Paper XXX arxiv.org: [nlin.cd] Exact Synchronization for Finite-State Sources Nicholas F. Travers 1, 2, 1, 2, 3, 4, and James P. Crutchfield 1 Complexity Sciences Center 2 Mathematics Department 3 Physics Department University of California at Davis, One Shields Avenue, Davis, CA Santa Fe Institute 1399 Hyde Par Road, Santa Fe, NM (Dated: October 25, 2010) We analyze how an observer synchronizes to the internal state of a finite-state information source, using the ɛ-machine causal representation. Here, we treat the case of exact synchronization, when only a finite number of observations are required. The more difficult case of strictly asymptotic synchronization is treated in a sequel. In both cases, we find that an observer synchronizes to the source exponentially fast and, as a result, the accuracy in an observer s predictions of future source output approaches its optimal level exponentially fast as well. We show how to analytically calculate the synchronization exponent. Additionally, we provide a characterization theorem for exact ɛ-machines and, from it, build a polynomial-time algorithm to test for exactness. PACS numbers: r c Tp Ey I. INTRODUCTION One of the ey questions in time series analysis is prediction: given a sequence of observed time series data X 0, X 1,..., X t 1, how well can one predict the next data point X t? If the X t s are independent, identically distributed and one nows their distributions, then, of course, no amount of past data can aid an observer in this tas. However, generally there are correlations between them and, in this case, the more data an observer has, the better its predictions become. Our wor is ultimately motivated by an interest in the question of prediction, but we approach it from a logically prior problem that of synchronization. Specifically, we study data sources from a class of finite-state hidden Marov models nown as ɛ-machines [1]. We tae the position of an observer that has a correct ɛ-machine model of the source and tries to predict the next symbol it generates using past observations of the source output. That is, we assume the observer sees past output symbols generated by the source, but does not have direct access to its hidden internal states (either current or past). Rather, the observer must infer the internal state (synchronize) only through observations of the output. Since future output is a function of the current state, better nowledge of the current state enables an observer to mae better predictions. The more readily an observer synchronizes, the better it can predict. Synchronization and state estimation for finite-state machines is a central interest in several disciplines. In information theory state estimation for hidden Marov Electronic address: ntravers@math.ucdavis.edu Electronic address: chaos@cse.ucdavis.edu models (HMMs) has been studied extensively; see, for example, Refs. [2] and [3]. However, the output sequences of generic HMMs irretrievably throw away internal state information. They lac, what is called, unifilarity. Without this property, an observer cannot (on average) synchronize completely and, if synchronization is gained, it can be repeatedly lost. And so, much of the analysis there is necessarily different, as we will consider only unifilar HMMs. There has been substantial wor, as well, in symbolic dynamics on synchronizing presentations of sofic shifts [4] and in automata theory on synchronizing sequences and homing sequences for Mealy machines [5]. Much of this wor, though, is of an entirely topological nature, since the machines are often defined with nonprobabilistic transitions. Although, in some cases probabilistic machines have been studied as well [6]. We will consider here, specifically, probabilistic, unifilar, finitestate machines. Our development proceeds as follows. Section II provides the necessary bacground. Definitions of ɛ-machines and synchronization are given, as well as those for the predictive quantities h µ and h µ (L) the entropy rate and its finite-time window approximates [7]. We identify two qualitatively distinct types of synchronization: exact (synchronization driven by finite sequences) and asymptotic (where there are no such finite sequences). The exact case is the subject here; the nonexact case is treated in a sequel [8]. Section III gives results on the synchronization rates for exact ɛ-machines and consequences for the convergence rates of the average state uncertainty H(L) and entropy-rate approximation h µ (L). Section IV characterizes exact ɛ-machines and introduces an algorithmic test for exactness. Finally, Sec. V summarizes our results and indicates several extensions.

2 2 II. BACKGROUND This section introduces the primary objects of study: finite-state ɛ-machines and the processes that they describe. Additionally, we give formal definitions for exact and asymptotic synchronization, the source entropy rate h µ, and its length-l approximation h µ (L). The latter two quantities will be used to monitor an observer s predictive capability. A. Stationary Information Sources Let A be a finite alphabet, and let X 0, X 1,... be the random variables (RVs) for a sequence of observed symbols x t A generated by an information source. We denote the RVs for the sequence of future symbols beginning at time t 0 as X X 0 X 1 X 2..., the bloc of L symbols beginning at time t 0 as X L X 0 X 1...X L 1, and the bloc of L symbols beginning at a given time t as X L t X t X t+1...x t+l 1. A stationary source is one for which Pr( X L t ) Pr( X L 0 ) for all t and all L > 0. We monitor our observer s performance at predicting a stationary source using information-theoretic measures [9]. The first step in this is to review several basic measures for stationary processes. Definition 1. The bloc entropy H(L) for a stationary source is H(L) H[ X L ] { x L } Pr( x L ) log 2 Pr( x L ). (1) The bloc entropy gives the average uncertainty in observing blocs X L. Definition 2. The entropy rate h µ is the asymptotic average entropy per symbol: H(L) h µ lim L L lim L H[X L X L ]. (2) It gives the intrinsic randomness of a source the irreducible uncertainty in future output once all of the correlations and structure of the source are nown. An observer cannot predict the source output any better than an error rate that is a function of h µ [9]. Definition 3. The entropy rate s length-l approximation is h µ (L) H(L) H(L 1) H[X L 1 X L 1 ]. (3) That is, h µ (L) is the observer s average uncertainty in the next symbol to be generated after observing the first L 1 symbols. For any stationary process, h µ (L) monotonically decreases to the limit h µ [9]. However, the form of convergence depends on the process. The lower the value of h µ a source has, the better an observer s predictions of the source output will be asymptotically. The faster h µ (L) converges to h µ, the faster an observer s predictions will reach this optimal asymptotic level. Since we are often interested in maing predictions after observing only finite data, the source s true entropy rate h µ, as well as the rate of convergence of h µ (L) to h µ, are both important properties. B. Hidden Marov Models In what follows we will restrict our attention to an important class of stationary information sources nown as hidden Marov models. For simplicity, we assume the number of states is finite. Definition 4. A finite-state edge-labeled hidden Marov machine (HMM) consists of 1. a finite set of states S {σ 1,..., σ N }, 2. a finite alphabet of symbols A, and 3. a set of N by N symbol-labeled transition matrices T (x), x A, where T (x) ij is the probability of transitioning from state σ i to state σ j on symbol x. The corresponding overall state-to-state transition matrix is denoted T x A T (x). A hidden Marov machine can be depicted as a directed graph with labeled edges. The nodes are the states {σ 1,..., σ N } and for all x, i, j with T (x) ij > 0 there is an edge from state σ i to state σ j labeled p x for the symbol x and transition probability p T (x) ij. We require that the transition matrices T (x) be such that this graph is strongly connected. A hidden Marov machine M generates a stationary process P as follows. Initially, M starts in some state σ i chosen according to the stationary distribution π over machine states the distribution satisfying πt π. It then pics an outgoing edge according to their relative transition probabilities T (x) i j, generates the symbol x labeling this edge, and follows the edge to a new state σ j. The next output symbol and state are consequently chosen in a similar fashion, and the process is repeated indefinitely. We denote S 0, S 1, S 2,... as the RVs for the sequence of machine states visited and X 0, X 1, X 2,... as the RVs for the associated sequence of output symbols generated. The sequence (S L ) L 0 is a Marov chain with transition ernel T. However, the stochastic process we consider is not the sequence of states, but rather the associated sequence of outputs (X L ) L 0, which is not (generally) Marovian. We assume the observer directly observes this sequence of outputs, but does not have direct access to the machine s hidden internal states.

3 3 C. Examples In the following it will be helpful to refer to several example hidden Marov machines that illustrate ey properties and definitions. We introduce four examples, all with a binary alphabet A {0, 1}. 1. Even Process Figure 1 gives a HMM for the Even Process. Its transitions matrices are: T (0) p 0, 0 0 T (1) 0 1 p. (4) 1 0 p 0 1 p 1 σ 1 σ FIG. 1: A hidden Marov machine (the ɛ-machine) for the Even Process. The transitions denote the probability p of generating symbol x as p x. The support for the Even Process consists of all binary sequences in which blocs of uninterrupted 1s are even in length, bounded by 0s. After each even length is reached, there is a probability p of breaing the bloc of 1s by inserting one or more 0s. The hidden Marov machine has two internal states, S {σ 1, σ 2 }, and a single parameter p [0, 1] that controls the transition probabilities. 2. Alternating Biased Coins Figure 2 shows a HMM for the Alternating Biased Coins (ABC) Process. The transitions matrices are: T (0) 0 1 p, 1 q 0 T (1) 0 p. (5) q 0 the biases of the two coins that are flipped alternatively in sequence to generate the process. 3. SNS Process Figure 3 depicts a two-state HMM for the SNS Process which generates long sequences of 1s broen by isolated 0s. Its matrices are: T (0) 0 0, 1 q 0 T (1) p 1 p. (6) 0 q 1 p 1 p 1 σ 1 σ 2 q 0 1 q 1 FIG. 3: An HMM for the SNS Process. Note that the two transitions leaving state σ 1 both emit x Noisy Period-2 Process Finally, Fig. 4 depicts a nonminimal HMM for the Noisy Period-2 (NP2) Process. The transition matrices are: T (0) T (1) p p p p 0 0 0, 1 1 σ 1 σ 2. (7) p 1, 1 p 0 σ 1 σ 2 q 1, 1 q 0 p 1, 1 p 0 p 1, 1 p 0 FIG. 2: A hidden Marov machine (the ɛ-machine) for the Alternating Biased Coins Process. It too has two internal states. There are also two transition probability parameters: p, q [0, 1]. These control σ σ 3 FIG. 4: An HMM for the Noisy Period-2 Process.

4 4 It is clear by inspection that the same process can be captured by a hidden Marov machine with fewer states. Specifically, the distribution over future sequences from states σ 1 and σ 3 are the same, so those two states are redundant and can be merged. The same is also true for states σ 2 and σ 4. D. ɛ-machines We now introduce a class of hidden Marov machines that has a number of desirable properties for analyzing synchronization. Definition 5. A finite-state ɛ-machine is a finite-state edge-labeled hidden Marov machine with the following properties: 1. Unifilarity: For each state σ S and each symbol x A there is at most one outgoing edge from state σ labeled with symbol x. 2. Probabilistically distinct states: For each pair of distinct states σ, σ j S there exists some finite word w x 0 x 1... x L 1 such that Pr( X L w S 0 σ ) Pr( X L w S 0 σ j ). The hidden Marov machines given above for the Even Process and ABC process are both ɛ-machines. The SNS machine of example 3 is not an ɛ-machine, however, since state σ 1 is not unifilar. The NP2 machine of example 4 is also not an ɛ-machine, since it does not have probabilistically distinct states, as noted before. ɛ-machines were originally defined in Ref. [1] as hidden Marov machines whose states, nown as causal states, were the equivalence classes of infinite pasts x with the same probability distribution over futures x. This history ɛ-machine definition is, in fact, equivalent to the generating ɛ-machine definition presented above in the finite-state case. Although, this is not immediately apparent. Formally, it follows from the synchronization results established here and in Ref. [8]. It can also be shown that an ɛ-machine M for a given process P is unique up to isomorphism [1]. That is, there cannot be two different finite-state edge-labeled hidden Marov models with unifilar transitions and probabilistically distinct states that both generate the same process P. Furthermore, ɛ-machines are minimal unifilar generators in the sense that any other unifilar machine M generating the same process P as an ɛ-machine M will have more states than M. Note that uniqueness does not hold if we remove either condition 1 or 2 in the ɛ-machine definition. E. Synchronization Assume now that an observer has a correct model M (ɛ-machine) for a process P, but is not able to directly observe M s hidden internal state. Rather, the observer must infer the internal state by observing the output data that M generates. For a word w of length L generated by M let φ(w) Pr(S w) be the observer s belief distribution as to the current state of the machine after observing the word w. That is, φ(w) Pr(S L σ X L w) Pr(S L σ X L w, S 0 π). (8) Let L(M) be the set of all finite words that M generates, L L (M) be the set of all words of length L it generates, and L (M) be the set of all infinite sequences x x 0 x 1... which it generates. Definition 6. A word w L(M) is a synchronizing word (simply, sync word) for M if H[φ(w)] 0, where H[µ] is the Shannon entropy of a discrete distribution µ. That is, a synchronizing word w is one for which the observer nows exactly which state M is in after seeing w. Let us denote: Sync M : The set of M s synchronizing sequences as SYN(M) { x L (M): there exists L N such that x L is a sync word for M}. Wea Sync M : The set of M s wealy synchronizing sequences as WSYN(M) { x L (M): H[φ( x L )] 0 as L }. Definition 7. An ɛ-machine M is exactly synchronizable (simply, exact) if Pr(SYN(M)) 1. That is, the observer synchronizes exactly to almost every sequence generated by M in finite time. Definition 8. An ɛ-machine M is asymptotically synchronizable if Pr(WSYN(M)) 1. That is, the observer s uncertainty in M s state vanishes asymptotically for almost every sequence generated. The Even Process ɛ-machine, Fig. 1, is an exact machine. Any word containing a 0 is a sync word for this machine, and almost every x generated by it contains at least one 0. The ABC Process ɛ-machine, Fig. 2, is not exactly synchronizable, but it is asymptotically synchronizable. Remar. If w L(M) is a sync word, then by unifilarity so is wu, for all u with wu L(M). Thus, once an observer synchronizes exactly, it remains synchronized exactly for all future times. It follows that any exactly synchronizable machine is also asymptotically synchronizable. Remar. If w L(M) is a sync word then so is uw, for all u with uw L(M). Since any finite word w L(M) will be contained in almost every infinite sequence x the machine generates, it follows that a machine is exactly synchronizable if (and only if) it has some sync word w of finite length.

5 5 Remar. It turns out all finite-state ɛ-machines are asymptotically synchronizable; see Ref. [8]. Hence, there are two disjoint classes to consider: exactly synchronizable machines and asymptotically synchronizable machines that are nonexact. The exact case is the subject of the remainder. Finally, one last important quantity for synchronization is the observer s average uncertainty in the machine state after seeing a length L bloc of data [7]. Definition 9. The observer s average state uncertainty at time L is: H(L) H[S L X L ] Pr( x L ) H[S L X L x L ]. (9) { x L } That is, H(L) is the expected value of an observer s uncertainty in the machine state after observing L symbols. Now, for an ɛ-machine, an observer s prediction of the next output symbol is a direct function of the probability distribution over machine states induced by the previously observed symbols. Specifically, A. Exact Machine Synchronization Theorem Our first theorem states that an observer will synchronize (exactly) to the internal state of any exact ɛ-machine exponentially fast. Theorem 1. For any exact ɛ-machine M, there are constants K > 0 and 0 < α < 1 such that for all L N. Pr(NSYN L ) Kα L, (11) Proof. Let M be an exact machine with sync word w L(M, σ ). Since the graph of M is strongly connected we now that for each state σ j, there is a word v j such that δ(σ j, v j ) σ. Let w j v j w, n max j w j, and p min j Pr(w j σ j ). Then, for all L 0, we have: Pr(w X n+l w X L ) Pr(w X n L w X L ) min j Pr(w X n L S L σ j ) p. (12) Pr(X L x X L x L ) Pr(x σ ) Pr(S L σ X L x L ). (10) Hence, Pr(w X n+l w X L ) 1 p, (13) Hence, the better an observer nows the machine state at the current time, the better it can predict the next symbol generated. And, on average, the closer H(L) is to 0, the closer h µ (L) is to h µ. Therefore, the rate of convergence of h µ (L) to h µ for an ɛ-machine is closely related to the average rate of synchronization. III. EXACT SYNCHRONIZATION RESULTS This section provides our main results on synchronization rates for exact machines and draws out the consequences for the convergence rates of H(L) and h µ (L). The following notation will be used throughout: 1. SYN L {w L L (M) : w is a sync word for M}. 2. NSYN L {w L L (M) : w is not a sync word for M}. 3. SYN L,σ {w L L (M) : w synchronizes the observer to state σ }. 4. L(M, σ ) {w : M generates w starting in state σ }. 5. For a word w L(M, σ ), δ(σ, w) is defined to be w the (unique) state in S such that σ δ(σ, w). 6. For a set of states S S, we define: δ(s, w) {σ j S : σ w σj from some σ S}. for all L 0. And, therefore, for all m N: Pr(NSYN mn ) P r(w X mn ) Pr(w X n ) Pr(W X 2n w X n ) Pr(w X mn w X (m 1)n ) (1 p) (1 p) (1 p) (1 p) m β m, (14) where β 1 p. Hence, for L mn (m N): Pr(NSYN L ) Pr(NSYN mn ) β m α mn α L, (15) where α β 1/n. Since Pr(NSYN L ) is monotonically decreasing, it follows that: Pr(NSYN L ) 1 α n αl Kα L, (16) for all L N (where K 1/α n ). Remar. In the above proof we have implicitly assumed β 0. If β 0, then the conclusion follows trivially.

6 6 B. Synchronization Rate Theorem 1 states that an observer synchronizes (exactly) to any exact ɛ-machine exponentially fast. However, the synchronization rate constant: α lim L Pr(NSYN L) 1/L (17) depends on the machine, and it may often be of practical interest to now what the constant α is. We now we provide a method for computing α analytically. It is based on the construction of an auxiliary machine M. Definition 10. Let M be an ɛ-machine with states S {σ 1,..., σ N }, alphabet A, and transition matrices T (x), x A. The possibility machine M is defined as follows: 1. The alphabet of M is A. 2. The states of M are pairs of the form (σ, S) where σ S and S is a subset of S that contains σ. 3. The transition probabilities are: Pr((σ, S) x (σ, S )) Pr(x σ)i(x, (σ, S), (σ, S )), where I(x, (σ, S), (σ, S )) is the indicator function: I(x,(σ, S), (σ, S )) { 1 if δ(σ, x) σ and δ(s, x) S 0 otherwise. A state of M is said to be initial if it is of the form (σ, S) for some σ S. For simplicity we restrict the M machine to consist of only those states that are accessible from initial states. The other states are irrelevant for the analysis below. The idea is that M states represent states of the joint (machine,observer) system. Causal state σ is the true machine state at the current time, and S is the set of states that the observer believes are currently possible for the machine to be in, after observing all previous symbols. Initially, all states are possible (to the observer), so initial states are those in which the set of possible states is the complete set S. If the current true machine state is σ and then the symbol x is generated, then the new true machine state must be δ(σ, x). Similarly, if the observer believes any of the states in S are currently possible and then the symbol x is generated, then the new (to the observer) set of possible states is δ(s, x). This accounts for transitions in M topologically. The probability of generating a given symbol x from (σ, S) is, of course, governed only by the true state σ of the machine: Pr(x (σ, S)) Pr(x σ). An example of this construction for a 3-state exact ɛ-machine is given in Appendix A. Note that the graph of the M machine there has a single recurrent strongly connected component, which is isomorphic to the original machine M. This is not an accident. It will always be the case, as long as the original machine M is exact. Remar. If M has more than 1 state, the graph of M itself is never strongly connected. So, M is not an ɛ-machine or even an HMM in the sense of Def. 4. However, we still refer to M as a machine. M s states are denoted S {q 1,..., qñ}, its symbol-labeled transition matrices T (x), and its overall state-to-state transition matrix T T x A (x). We assume the states are ordered in such a way that the initial states (σ 1, S),..., (σ N, S) are, respectively, q 1,..., q N. Similarly, the recurrent states (σ 1, {σ 1 }), (σ 2, {σ 2 }),..., (σ N, {σ N }) are, respectively, q n+1, q n+2,..., qñ, where n Ñ N. The ordering of the other states is irrelevant. In this case, the matrix T has the following bloc upper triangular form: B B T, (18) O T where B is a n n matrix with nonnegative entries, B is a n N matrix with non-negative entries, O is a N n matrix of all zeros, and T is the N N state-to-state transition matrix of the original ɛ-machine M. Let π (π 1,..., π N, 0,..., 0) denote the length-ñ row vector whose distribution over the initial states is the same as the stationary distribution π for the ɛ-machine M. Then, the initial probability distribution φ 0 over states of the joint (machine, observer) system is simply: φ 0 π, (19) and the distribution over states of the joint system after the first L symbols is: φ L π T L. (20) If the joint system is in a recurrent state of the form (σ, {σ }), then to the observer the only possible state of the ɛ-machine is the true state. The observer is synchronized. For all other states of M, the observer is not yet synchronized. Hence, the probability the observer is not synchronized after L symbols is simply the combined probability of all nonreccurent states q i in the distribution φ L. Mathematically, we have: n Pr(NSYN L ) ( φ L ) i i1 n ( π T L ) i i1 n (π B B L ) i i1 π B B L 1, (21)

7 7 where π B (π 1,..., π N, 0,..., 0) is the length-n row vector corresponding to the distribution over initial states π. (The third equality follows from the bloc uppertriangular form of T.) Appendix B shows that: lim ( π B B L ) 1/L 1 r, (22) L where r r(b) is the (left) spectral radius of B: r(b) max{ λ : λ is a (left) eigenvalue of B}. (23) Thus, we have, Theorem 2. For any exact ɛ-machine M, α r. C. Consequences We now apply Thm. 1 to show that an observer s average uncertainty H(L) in the machine state and average uncertainty h µ (L) in predictions of future symbols both decay exponentially fast to their respective limits: 0 and h µ. The decay constant α in both cases is essentially bounded by the sync rate constant α from Thm. 2. Proposition 1. For any exact ɛ-machine M, there are constants K > 0 and 0 < α < 1 such that H(L) Kα L, for all L N. Proof. Let M be any exact machine. Then by Thm. 1, there are constants C > 0 and 0 < α < 1 such that Pr(NSYN L ) Cα L, for all L N. Thus, we have: H(L) Pr(w)H[φ(w)] w L L (M) Pr(w)H[φ(w)] + Pr(w)H[φ(w)] w SYN L w NSYN L 0 + Pr(w) log(n) w NSYN L log(n) Cα L Kα L, (24) where N is the number of machine states and K C log(n). Proposition 2. For any exact ɛ-machine M: h µ π H[X next σ ], (25) where X next is the RV for the symbol following σ. And, there are constants K > 0 and 0 < α < 1 such that: for all L N. h µ (L) h µ Kα L, (26) Remar. The h µ formula Eq. (25) has been nown for some time, although in slightly different contexts. Shannon, for example, derived this formula in his original publication [10] for a type of hidden Marov machine that is similar (apparently unifilar) to an ɛ-machine. Proof. Let M be any exact machine. Since we now h µ (L) h µ it suffices to show there are constants K > 0 and 0 < α < 1 such that: h µ(l) π H[X next σ ] KαL, (27) for all L N. This will establish both the value of h µ and the necessary convergence. Now, by Thm. 1, there are constants C > 0 and 0 < α < 1 such that Pr(NSYN L ) Cα L, for all L N. Also, note that for all L and we have: π Pr(w) φ(w) Thus, and w L L (M) w SYN L,σ Pr(w) φ(w) Pr(SYN L,σ ). (28) (π Pr(SYN L,σ )) H[X next σ ] 0, (29) (π Pr(SYN L,σ )) H[X next σ ] Also, clearly, and, (π Pr(SYN L,σ )) log A ( log A π ) Pr(SYN L,σ ) log A (1 Pr(SYN L )) log A Pr(NSYN L ) log A Cα L. (30) w NSYN L Pr(w) H[X next w] 0, (31) w NSYN L Pr(w) H[X next w] log A Pr(NSYN L ) log A Cα L. (32)

8 8 Therefore, we have for all L N: h µ(l + 1) π H[X next σ ] Pr(w)H[X next w] π H[X next σ ] w L L (M) Pr(w)H[X next w] w NSYN L + Pr(w)H[X next w] π H[X next σ ] w SY N L Pr(w)H[X next w] w NSYN L + Pr(w)H[X next w] π H[X next σ ] w SYN L,σ Pr(w)H[X next w] w NSYN L + Pr(SYN L,σ )H[X next σ ] π H[X next σ ] Pr(w)H[X next w] w NSYN L (π Pr(SYN L,σ ))H[X next σ ] C log A α L. (33) The last inequality follows from Eqs. (29)-(32), since x y z for all non-negative real numbers x, y, and z with x z and y z. Finally, since, h µ(l + 1) π H[X next σ ] C log A αl, (34) for all L N, there must be some K C log A /α such that: for all L N. h µ(l) π H[X next σ ] KαL, (35) Remar. For any α > α there exists some K > 0 for which Eq. 11 holds. Hence, by the constructive proofs above, we see that the constant α in Props. 1 and 2 can be chosen arbitrarily close to α. IV. CHARACTERIZATION OF EXACT ɛ-machines In this section we provide a set of necessary and sufficient conditions for exactness and an algorithmic test for exactness based upon these conditions. A. Exact Machine Characterization Theorem Definition 11. States σ and σ j are said to be topologically distinct if L(M, σ ) L(M, σ j ). Definition 12. States σ and σ j are said to be path convergent if there exists w L(M, σ ) L(M, σ j ) such that δ(σ, w) δ(σ j, w). If states σ and σ j are topologically distinct (or path convergent) we will also say the pair (σ, σ j ) is topologically distinct (or is path convergent). Theorem 3. An ɛ-machine M is exact if and only if every pair of distinct states (σ, σ j ) satisfies at least one of the following two conditions: (i) The pair (σ, σ j ) is topologically distinct. (ii) The pair (σ, σ j ) is path convergent. Proof. It was noted above that an ɛ-machine M is exact if and only if it has some sync word w of finite length. Therefore, it is enough to show that every pair of distinct states (σ, σ j ) satisfies either (i) or (ii) if and only if M has some sync word w of finite length. We establish the if first: If M has a sync word w, then every pair of distinct states (σ, σ j ) satisfies either (i) or (ii). Let w be a sync word for M. Then w L(M, σ ) for some. Tae words v j, j 1, 2,...N, such that δ(σ j, v j ) σ. Then, the word v j w w j L(M, σ j ) is also a sync word for M for each j. Therefore, for each i j either w j L(M, σ i ) or δ(σ i, w j ) δ(σ j, w j ). This establishes that the pair (σ i, σ j ) is either topologically distinct or path convergent. Since this holds for all j 1, 2,..., N and for all i j, we now every pair of distinct states is either topologically distinct or path convergent. Now, for the only if case: If every pair of distinct states (σ, σ j ) satisfies either (i) or (ii), then M has a sync word w. If each pair of distinct states (σ, σ j ) satisfies either (i) or (ii), then for all and j ( j) there is some word w σ,σ j such that one of the following three conditions is satisfied. 1. w σ,σ j L(M, σ ), but w σ,σ j L(M, σ j ). 2. w σ,σ j L(M, σ j ), but w σ,σ j L(M, σ ). 3. w σ,σ j L(M, σ ) L(M, σ j ) and δ(σ, w σ,σ j ) δ(σ j, w σ,σ j ).

9 9 We construct a sync word w w 1 w 2...w m for M, where each w i w σi,σ ji for some i and j i, as follows. Let S 0 {σ 0 1,..., σ 0 N 0 } S {σ 1,..., σ N }. Tae w 1 w σ 0 1,σ 0 2. Let S 1 {σ 1 1,..., σ 1 N 1 } δ(s 0, w 1 ). Since w 1 w σ 0 1,σ 0 2 satisfies either (1), (2), or (3), we now N 1 < N 0. Tae w 2 w σ 1 1,σ 1 2. Let S 2 {σ 2 1,..., σ 2 N 2 } δ(s 1, w 2 ). Since w 2 w σ 1 1,σ 1 2 satisfies either (1), (2), or (3) we now N 2 < N 1. Tae w 3 w σ 2 1,σ 2 2. Repeat these steps until S m 1 for some m. Note that this will happen after a finite number of steps since N N 0 is finite and N 0 > N 1 > N 2 >. By this construction w w 1 w 2...w m L(M) is a sync word for M. After observing w, an observer nows the machine must be in state σ m 1. B. A Test for Exactness We can now provide an algorithmic test for exactness using the characterization theorem of exact machines. We begin with subalgorithms to test for topological distinctness and path convergence of state pairs. Both are essentially the same algorithm and only a slight modification of the deterministic finite-automata (DFA) tablefilling algorithm that tests for pairs of equivalent states [11]. Algorithm 1. Test States for Topological Distinctness. 1. Initialization: Create a table containing boxes for all pairs of distinct states (σ, σ j ). Initially, all boxes are blan. Then, Loop over distinct pairs (σ, σ j ) Loop over x A If {x L(M, σ ) but x L(M, σ j )} or {x L(M, σ j ) but x L(M, σ )}, then mar box for pair (σ, σ j ). end end 2. Induction: If δ(σ, x) σ and δ(σ j, x) σ j and the box for the pair (σ, σ j ) is mared, then mar the box for pair (σ, σ j ). Repeat until no more inductions are possible. Algorithm 2. Test States for Path Convergence. This algorithm is identical to Algorithm 1 except that the if-statement in the initialization step is replaced with the following: If x L(M, σ ) L(M, σ j ) and δ(σ, x) δ(σ j, x), then mar box for pair (σ, σ j ). With Algorithm 1 all pairs of topologically distinct states end up with mared boxes. With Algorithm 2 all pairs of path convergent states end up with mared boxes. Both of these facts can be proved using induction on the length of the minimal distinguishing (path converging) word w for a given pair of states. The proofs are virtually identical to the proof of the standard DFA table filling algorithm, so the details have been omitted. Note also that both of these are polynomial-time algorithms. Step (1) has run time O( A N 2 ). The inductions in Step (2), if done in a reasonably efficient fashion, can also be completed in run time O( A N 2 ). (See, e.g., the analysis of DFA table filling algorithm in Ref. [11].) Therefore, the total run time of these algorithm is O( A N 2 ). Algorithm 3. Test for Exactness. 1. Use Algorithm 1 to find all pairs of topologically distinct states. 2. Use Algorithm 2 to find all pairs of path convergent states. 3. Loop over all pairs of distinct states (σ, σ j ) to chec if they are either (i) topologically distinct or (ii) path convergent. By Thm. 3, if all distinct pairs of states satisfy (i) or (ii) or both, the machine is exact and otherwise it is not. This, too, is a polynomial-time algorithm. Steps (1) and (2) have run time O( A N 2 ). Step (3) has run time O(N 2 ). Hence, the total run time for this algorithm is O( A N 2 ). V. CONCLUSION We analyzed the process of exact synchronization to finite-state ɛ-machines. In particular, we showed that for exact machines an observer synchronizes exponentially fast and that, as a result, the average uncertainty h µ (L) in an observer s predictions converges exponentially fast to the entropy rate h µ. Additionally, we found an efficient (polynomial-time) algorithm to test ɛ-machines for exactness. In Ref. [8] we similarly analyze asymptotic synchronization to nonexact ɛ-machines. It turns out that qualitatively similar results hold. That is, H(L) and h µ (L) both converge to their respective limits exponentially fast. However, the proof methods in the nonexact case are substantially different. In the future we plan to extend these results to more generalized model classes, such as to ɛ-machines with a countable number of states and to nonunifilar hidden Marov machines.

10 10 Acnowledgments NT was partially supported on a VIGRE fellowship. The wor was partially supported by the Defense Advanced Research Projects Agency (DARPA) Physical Intelligence project. The views, opinions, and/or findings contained in this article are those of the authors and should not be interpreted as representing the official views or policies, either expressed or implied, of the DARPA or the Department of Defense. [1] J. P. Crutchfield and K. Young. Inferring statistical complexity. Phys. Rev. Let., 63: , J. P. Crutchfield, The Calculi of Emergence: Computation, Dynamics, and Induction, Physica D 75: 11 54, C. R. Shalizi and J. P. Crutchfield, Computational Mechanics: Pattern and Prediction, Structure and Simplicity, J. Stat. Phys. 104: , [2] G. D. Forney Jr. The Viterbi algorithm: A personal history. CoRR, abs/cs/ , [3] A. J. Viterbi. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Info. Th., 13(2): , [4] N. Jonosa. Sofic shifts with synchronizing presentations. Theor. Comput. Sci., 158(1-2):81 115, [5] S. Sandberg. Homing and synchronizing sequences. In M. Broy et al, editor, Lect. Notes Comp. Sci., volume 3472, pages Springer, Berlin, [6] A. Paz. Introduction to Probabilistic Automata. Academic Press, New Yor, [7] J. P. Crutchfield and D. P. Feldman. Regularities unseen, randomness observed: Levels of entropy convergence. CHAOS, 13(1):25 54, [8] N. Travers and J. P. Crutchfield. Asymptotic synchronization for finite-state sources SFI Woring Paper XXX; arxiv.org:10xx.xxxx [XXXX]. [9] T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley-Interscience, New Yor, second edition, [10] C. E. Shannon. A mathematical theory of communication. Bell Sys. Tech. J., 27: , , [11] J. E. Hopcroft, R. Motwani, and J. D. Ullman. Automata Theory, Languages, and Computation. Addison-Wesley, Reading, [12] M. Reed and B. Simon. Functional Analysis. Academic Press, Appendix A We construct the possibility machine M for the threestate ɛ-machine shown in Fig. 5. The result is shown in Fig. 6. Appendix B We prove Eq. (22) in Sec. III B. Lemma 1. For any exact ɛ-machine M, lim L πb B L 1/L 1 r(b). (B1) p 21 a σ 1 p 12 a p 31 c p 13 c σ 2 σ 3 p 23 b p 11 b p 33 a FIG. 5: A three-state ɛ-machine M with alphabet A {a, b, c}. In what follows A denotes an arbitrary m m matrix and v and w denote row m-vectors. Unless otherwise specified, the entries of matrices and vectors are assumed to be complex. Definition 13. The (left) matrix p-norms (1 p ) are defined as A p max{ v A p : v p 1}. The following facts will be used in our proof. (B2) Fact 1. If A is a matrix with real non-negative entries and v (v 1,..., v m ) is a vector with real non-negative entries, then: v A 1 m (v e )A 1 1 m v e A 1, 1 where e (0,..., 1,..., 0) is the th standard basis vector. Fact 2. Let A be a matrix with real nonnegative entries, let v (v 1,..., v m ) be a vector with complex entries, and let w (w 1,..., w m ) ( v 1,..., v m ). Then: v A 1 w A 1. (B3)

11 11 p 21 a 2, ewns 123 1, ewns 123 3, ewns 123 p 33 a p 23 b p 12 a p 13 c 3, ewns 23 3, ewns 13 1, ewns 13 2, ewns 23 p 31 c p 33 a p 21 a p 12 a p 31 c p 13 c p 11 b 1, ewns 1 p 11 b p 31 c p 11 b p 13 c p 33 a p 31 c p 12 a p 21 a 2, ewns 2 3, ewns 3 p 23 b p 23 b p 33 a FIG. 6: The possibility machine M for the three-state ɛ-machine M of Fig. 5. The state names have been abbreviated for display purposes: e.g., (σ 1, {σ 1, σ 2, σ 3}) (1, 123). Fact 3. For any matrix A {a ij }, the matrix 1-norm is the largest absolute row sum: A 1 max i m a ij. (B4) j1 Fact 4. For any matrix A, L N, and 1 p : A L p A L p. (B5) Fact 5. For any matrix A and 1 p : lim L AL 1/L p r(a), (B6) where r(a) is the (left) spectral radius of A: r(a) max{ λ : λ is a (left) eigenvalue of A}. (B7) (This is, of course, the same as the right spectral radius, but we emphasize the left eigenvalues for the proof of Lemma 1 below.) Fact 1 can be proved by direct computation, and Fact 2 follows from the triangle inequality. Fact 3 is a standard result from linear algebra. Facts 4 and 5 are finitedimensional versions of more general results established in Ref. [12] for bounded linear operators on Banach spaces. Using these facts we now prove Lemma 1. Proof. By Fact 5 we now: lim sup π B B L 1/L 1 r(b). (B8) L Thus, it suffices to show that: lim inf L πb B L 1/L 1 r(b). (B9) Let us define the B-machine to be the restriction of the M machine to it s nonreccurent states. The stateto-state transition matrix for this machine is B. We call the states of this machine B-states and refer to paths in the associated graph as B-paths. Note that the rows of B {b ij } are substochastic: b ij 1, j (B10) for all i, with strict inequality for at least one value of i. By the construction of the B-machine we now that for each of its states q j there exists some initial state q i q i(j) such that q j is accessible from q i(j). Define l j to be the length of the shortest B-path from q i(j) to q j, and l max max j l j. Let c j > 0 be the probability, according to the initial distribution π B, of both starting in state q i(j) at time 0 and ending in state q j at time l j : c j (π i(j) e i(j) B lj ) j. Finally, let C 1 min j c j. Then, for any L > l max and any state q j we have: π B B L 1 π i(j) e i(j) B L 1 (B11) (π i(j) e i(j) B lj )B L lj 1 (B12) c j e j B L lj 1 C 1 e j B L lj 1 C 1 e j B L 1. (B13) (B14) (B15) Equation (B12) follows from Fact 1. The decomposition in Eq. (B13) is possible since L > l max l j. Eq. (B14) follows from Fact 1 and the definition of c j. Equation (B15) follows from the definition of C 1. Finally, Eq. (B16) follows from Fact 3, Fact 4, and Eq. (B11). Now, tae a normalized (left) eigenvector y (y 1,..., y n ) of B whose associated eigenvalue is maximal. That is, y 1 1, y B λ y, and λ r(b). Define z (z1,..., z n ) ( y 1,..., y n ). Then, for any L N: n z e B L 1 z B L 1 1 y B L 1 (B16) (B17) λ L y 1 (B18) λ L y 1 (B19) r(b) L, (B20)

12 12 where Eq. (B17) follows from Fact 1 and Eq. (B18) follows from Fact 2. Therefore, for each L we now there exists some j j(l) in {1,..., n} such that: z j(l) e j(l) B L 1 r(b)l n. (B21) Now, r(b) may be 0, but we can still choose the j(l) s such that z j(l) is never zero. And, in this case, we may divide through by z j(l) on both sides of Eq. (B22) to obtain, for each L: Therefore, for any L > l max we now: π B B L 1 C 1 e j(l) B L 1 C 1 (C 2 r(b) L) (B23) (B24) C 3 r(b) L, (B25) where C 3 C 1 C 2. Equation (B24) follows from Eq. (B16) and Eq. (B25) follows from Eq. (B23). Finally, since this holds for all L > l max, we have: e j(l) B L 1 where C 2 > 0 is defined by r(b)l n z j(l) C 2 r(b) L, 1 C 2 min. z j 0 n z j (B22) lim inf L πb B L 1/L 1 lim inf L ( C3 r(b) L) 1/L r(b). (B26)