Central Limit Theorem Approximations for the Number of Runs in Markov-Dependent Binary Sequences

Transcription

1 Central Limit Theorem Approximations for the Number of Runs in Markov-Dependent Binary Sequences George C. Mytalas, Michael A. Zazanis Department of Statistics, Athens University of Economics and Business, Athens 0434, Greece Abstract We consider Markov-dependent binary sequences and study various types of success runs overlapping, non-overlapping, exact, etc. by examining additive functionals based on state visits and transitions in an appropriate Markov chain. We establish a multivariate Central Limit Theorem for the number of these types of runs and obtain its covariance matrix by means of the recurrent potential matrix of the Markov chain. Explicit expressions for the covariance matrix are given in the Bernoulli and a simple Markovdependent case by expressing the recurrent potential matrix in terms of the stationary distribution and the mean transition times in the chain. We also obtain a multivariate Central Limit Theorem for the joint number of nonoverlapping runs of various sizes and give its covariance matrix in explicit form for Markov dependent trials. Key words: Runs, Markov Chains, Potential Matrix, Central Limit Theorem for Runs. Introduction In a sequence of binary trials, {ξ i } with ξ i {0, }, i =, 2,..., success= or failure=0 a success run of length k is the occurrence of k consecutive successes. We will not discuss here the vast array of applications of the analysis of runs and patterns in Statistics and Applied Probability; for this Corresponding author address: zazanis@aueb.gr Michael A. Zazanis Preprint submitted to Journal of Statistical Planning and Inference July 24, 202

2 we refer the reader to Balakrishnan and Koutras 2002 and Fu and Lou Given a realization of n trials there are several different ways of counting the number of success runs of length k depending, among other considerations, on whether overlapping in counting is allowed or not. A success run of length k occurs at position m k of the binary string ξ, ξ 2,..., ξ n if ξ m k+ = ξ m k+2 = = ξ m = ξ m =. Counting all the positions m = k, k =,..., n for which the above condition holds gives the count M n,k of the number of success runs with overlapping allowed. A non-overlapping success run of length k occurs at position m if holds and no non-overlapping run of length k has already occurred in positions m k +,m k + 2,..., m. The number of non overlapping runs of length k in a string of n trials is denoted by N n,k. Runs of exact size k are also of interest, i.e. k consecutive successes flanked on the left and right by failures. An exact run of size k occurs at position m k if, in addition to, ξ m k = 0 and ξ m+ = 0. We denote the number of exact runs of size k in a string of n trials by J n,k. Note that, when the values of the ξ i s are revealed sequentially, an exact run that occurs at position m will be counted when the value of ξ m+ becomes known. Finally, we say that a run of size greater than or equal to k occurs at position m if holds and ξ m k = 0. G n,k, denotes the number of success runs of size greater than or equal to k in a string of n trials. In a string of consecutive successes of length greater than k only one run of size greater than k is counted according to this definition. See Fu and Lou 2003 for further clarifications regarding these definitions. To illustrate, if n = 3 and k = 2, in the binary string 0000 we have the following success run counts: N 3,2 = 4, G 3,2 = 3, M 3,2 = 6, and J 3,2 =. In this paper we consider Markov dependent binary trials. We investigate the asymptotic form of the joint distribution of N n,k, M n,k, G n,k, J n,k, as n and k is fixed, and show that it obeys a multivariate Central Limit Theorem CLT. For this purpose we consider an appropriate Markov chain and we express the number of success runs of the kind mentioned as additive functionals based on state visits and state transitions for this chain. The multivariate CLT follows then from standard results for such functionals. The covariance matrix of the iting normal distribution is expressed in terms of the stationary distribution of the Markov chain and its recurrent 2

3 potential matrix also known as the fundamental matrix. This allows for the efficient numerical computation of the covariance matrix for quite general types of Markovian dependence of the binary trial sequence. In the special case of simple two-state Markovian dependence and of course in the case of Bernoulli trials we take advantage of the connection that exists between the potential matrix of the chain and the mean transition times between the states of the chain. By straightforward, if somewhat involved, calculations we are able to obtain explicit expressions for the potential matrix, and therefore for the covariance matrix itself, in terms of the parameters of the model. The same technique is used to obtain a multivariate CLT for the number of non-overlapping runs of different sizes, N n,k,..., N n,kν, in Markov dependent trials and to compute the corresponding covariance matrix. Besides its intrinsic interest and applications in areas including quality control, randomness tests, and reliability, we indicate applications of this result to the analysis of manufacturing systems with random yield. Many results exist on the exact distribution of these types of runs for Bernoulli trials and, in some cases, also for Markov-dependent trials. Important unified approaches based on Markov chain techniques include Stefanov and Pakes 997 where exact results are obtained for runs and more general patterns and Fu and Koutras 994 which gives the distributions of the run statistics N n,k, M n,k, G n,k, J n,k, together with the size of the longest run for non-homogeneous Bernoulli trials. There are also approximations based on it theorems that establish convergence to Poisson or compound Poisson its in certain asymptotic regimes and which are especially important in applications. For an overview of these approximations see Barbour and Chryssaphinou 200. CLT approximations for the number of runs have a long history. Despite their ited accuracy they remain an important theoretical and practical tool in the analysis of runs. Feller 968 using arguments based on the Central Limit Theorem for renewal processes, gave an normal approximation for the number of non-overlapping success runs in i.i.d. trials. Setting µ = pk and σ 2 = qpk 2k + qp k 2 qp 3k+ qp k p k 3, 2 he shows that N n,k nµ/σ n d N 0,. The same approach is essentially used in Fu et al in order to obtain the iting distribution of the number of successes in success runs of length greater than or equal k in a 3

4 sequence of Markov dependent binary trials. Fu and Lou 2007 obtain in the same fashion a CLT approximation for the number of non-overlapping occurrences of a simple or compound pattern in i.i.d. multi-type trials. A different approach towards establishing the asymptotic normality of the number of runs that has been widely used is via the Hoeffding-Robbins Central Limit Theorem for k-dependent random variables. A CLT for M n,k with i.i.d. Bernoulli trials was obtained along these lines by Godbole 992 expressing M n,k as a sum of stationary k -dependent indicators. Using a similar approach Hirano et al. 99 gave explicit results establishing that M n,k n k + p k /σ n d N0, where σ 2 = p k p k 2kp 2k + 2pk p k. 3 q Jennen-Steinmetz and Gasser 986 also use the CLT for k-dependent random variables in order to obtain a multivariate iting normal distribution for success and failure runs of various lengths with Bernoulli trials that do not necessarily have the same probability of success but which obey certain asymptotic conditions. The same approach is used in Fu and Lou 2007 in order to obtain normal approximations for the number of overlapping occurrences of simple patterns in multi-type i.i.d. trials and in Makri and Psillakis 20 which obtains a CLT approximation for J n,k. An alternative and far-reaching approach based on exponential families of random variables has been pioneered by Stefanov 995 for the analysis of the number of occurrences of runs and patterns in binary trials. By essentially constructing an exponential martingale from the transitions of an appropriate Markov chain, and a stopping time corresponding to the completion of the pattern in question, he is able to derive both exact distributional results and CLT approximations for the joint number of success runs of various sizes and to determine the corresponding iting covariance matrix. We refer the reader also to Stefanov 2000 and Stefanov and Pakes 997 for further details. A very rich literature exists on the wider problem of the number of occurrences of patterns in strings of multi-type trials, typically independent or with markovian dependence see Reinert, Schbath, and Waterman, 2000 for a review. In the study of the joint distribution of pattern frequencies in strings of multi-type trials in Rukhin 2007 the potential matrix of a Markov chain whose states are words of a given length is used explicitly in 4

5 order to obtain the asymptotic covariance matrix in the corresponding CLT. The potential matrix is determined in that case via the pattern correlation matrix as opposed to the results of this paper where it is determined using explicit computations for the mean transition times between states. We refer the reader also to Rukhin 200 for a detailed study of pattern correlation matrices, including their connection to the potential matrix, and to Rukhin 200 for explicit formulas involving the potential matrix and asymptotic results establishing convergence to the multivariate Pòlya-Aeppli law. 2. A Markov chain for success runs Throughout the paper we examine trials with two possible outcomes, 0 and. In this section we will assume that the sequence of trials {ξ n ; n N} is a two-state Markov chain with transition probability matrix [ ] q0 p P M = 0 4 q p where p 0, q 0, p, q > 0 and p 0 + q 0 = p + q =. Consider now another Markov chain {X n ; n N} with state space S = {0,,..., 2k} and transition probability matrix P = q 0 p 0 q. q. q q p... p p... p. 5 In the above matrix all elements are zero except for P i0 = q for i =, 2,..., 2k, while P 00 = q 0, P 0 = p 0, P i,i+ = p for i =, 2,..., 2k, and P 2k,k+ = p. It is easy to see that {X n } is irreducible, aperiodic, and positive recurrent. Its stationary distribution is given by q π 0 =, q + p 0 p 0 q π i = p i for i =,..., k, 6 q + p 0 π i = p 0 q q + p 0 p i p k for i = k +,..., 2k. 5

6 Suppose that P X 0 = 0 =. The total number of runs in n trials for each of the four different types of runs discussed above can be described by counting the number of visits in various states, or the number of state transitions, of the Markov chain {X n ; n N}. In fact N n,k = M n,k = n n X m {k, 2k} non-overlapping runs in [0, n ] 7 X m k overlapping runs in [0, n ] 8 G n,k = J n,k = n n X m = k runs of size k in [0, n ] 9 X m = k, X m+ = 0 exact runs in [0, n ] 0 Note that the case of Bernoulli trials is obtained within the above framework by setting p 0 = p and q 0 = q in the transition matrix P in 5. We will discuss the Bernoulli case at several places in this article, both because of its simplicity and because of the great interest it presents. 3. Potential matrices and the central it theorem for countable state-space Markov chains In this section we state for the sake of completeness some standard results that establish the connection between the recurrent potential matrix for positive recurrent Markov chains with countable state space and the variance constant in the Central Limit Theorem for additive functionals of the sample paths of such chains. Suppose that {X n ; n N} is a Markov chain with countable state space S and transition probability matrix P assumed to be irreducible and positive recurrent. Denote by π the corresponding invariant distribution. Let, as usual, Pij n := P X n = j X 0 = i and define the recurrent potential matrix, Z, also known as the fundamental matrix via Z ij = n= P n ij π j + δij, i, j S 6

7 where δ ij is the Kroneker symbol which is equal to if i = j and 0 otherwise. The convergence of the series in is a standard result see for instance Brémaud, 997. If f : S R is such that µ := i S π ifi < then the n fx m to µ is a consequence of the Strong Law of a.s. convergence of n Large Numbers for Markov chains with countable state space see Brémaud, 997. The corresponding Central Limit Theorem is given in the following Theorem. With the above assumptions on the Markov chain {X n }, let f = f,..., f d be a function S R d such that i S π if k i = µ k < and i S π ifk 2 i < for k =, 2,..., d, and define the additive functional {S n = S n,,..., S n,d ; n N} via S n = n fx m. Then, with µ = µ,..., µ d, we have n /2 S n nµ d N0, V where the covariance matrix is given by V kl = f k iγ ij f l j, k, l =,..., d, 2 i,j S S with Γ ij = π i Z ij + π j Z ji π i π j δ ij π i, i, j S. 3 For a proof we refer the reader to Aldous and Fill 997, Ch.2 and in a form where the appearance of the potential matrix is implicit in Port 994, p.823. When the state space S is finite, say consisting of n elements, the potential matrix is given by the expression where Π := Z = I P + Π 4 π π 2 π n.. π π 2 π n is a matrix with n identical rows, each equal to the stationary distribution π, and I is the identity matrix. Equation 4 provides an efficient way for the computation of the recurrent potential matrix by means of numerical methods. 7

8 An interesting connection exists between the elements of the potential matrix and the mean transition times between states of the chain. If T i := inf{n > 0; X n = i} then the following relations hold. Z ii = π i E π T i, 5 Z ij = Z jj π j E i T j. 6 For a proof we refer the reader to Brémaud, 997. In 5, 6, E i T j denotes the expectation E[T j X 0 = i] whereas E π T i denotes the expectation of T i given that X 0 is distributed according to the stationary distribution π. When the transition probability matrix has special structure one may exploit 5 and 6 in order to obtain the elements of the potential matrix in closed form. In this paper we will follow this route for Markov dependent trials with dependence given by The transition chain We now define a new Markov chain with state space S S, called the transition chain, by setting Y n := X n, X n+ for all n. The fact that {Y n ; n N} is a Markov process is immediate. The transition probability matrix of the new chain is given, in terms of the old one, by P i,j,i 2,j 2 = δ j i 2 P i2 j 2. Furthermore, the process {Y n } inherits the properties of irreducibility and positive recurrence from {X n }. The stationary distribution of the new chain, {πi, j; i, j S S}, is given in terms of the transition probability matrix and the stationary distribution of the original chain by πi, j = π i P ij. 7 Proposition 2. The potential matrix of the transition chain can be obtained in terms of that of the original chain as follows: Z ijkl = δ ik δ jl π k P kl + Z jk P kl. 8 Proof: The proposition follows by a straightforward computation which we sketch for the sake of completeness. It is simply a matter of evaluating the infinite series Z ijkl = P n ijkl πk, l + δ i,jk,l. n= 8

9 Note that where, of course, P 0 jk = δ jk. Hence Z ijkl = P n ijkl = P n jk P kl for n =, 2, 3,..., δ jk π k + P n jk π k P kl + δ ik δ jl. n= From the above 8 follows readily. Suppose now that g : S S R d is a reward function based on transitions and consider the additive functional T n = n gx m, X m+. If ν := i,j S S π ip ij gi, j exists then we have the Strong Law of Large Numbers n T n ν w.p.. The corresponding CLT for the transition chain is given in the following Theorem 3. With the above assumptions on the Markov chain {X n }, suppose that k,l S S π kp kl g 2 i k, l < for i =,..., d. Then where and Υ ij = l,l 2,k,k 2 S S n /2 T n nν d N0, Υ, g i l, l 2 Γ l,l 2,k,k 2 g j k, k 2, i, j =,..., d. 9 Γ l,l 2,k,k 2 = π l,l 2 Z l,l 2,k,k 2 + π k,k 2 Z k,k 2,l,l 2 20 π l,l 2 π k,k 2 δ l,l 2,k,k 2 π l,l 2. This is of course a direct restatement of theorem for the transition chain and its proof is omitted. If we use the expressions for π l,l 2 and Z l,l 2,k,k 2 in 7 and 8 we can express the matrix Γ of the transition chain in terms of the transition 9

10 probabilities and stationary distribution of the original Markov chain as follows Γ l,l 2,k,k 2 = π l P l l 2 P k k 2 Z l2 k π k P k k 2 + π k P k k 2 P l l 2 Z k2 l π l P l l 2 π l π k P l l 2 P k k 2, for l, l 2 k, k 2 2 Γ l,l 2,l,l 2 = 2π l P l l 2 π l P l l 2 + P l l 2 Z l2 l π l P l l 2 2 π l P l l We close this section with the consideration of the joint asymptotic distribution of two additive functionals, one based on state visits and the other based on state transitions. Let f : S R and g : S S R and consider two such S f n, and S g n defined by S f n := n fx m, S g n := n gx m, X m+. 23 Clearly, by defining the function f : S S R via fi, j = fi for all i, j S S the above case is covered by theorem 3. Nevertheless, from a computational point of view, it is worthwhile to examine this case separately, from first principles. The above argument shows that a CLT holds for S f n, S g n and the asymptotic variances of S f n and S g n can be obtained from theorems 2 and 3 respectively, so the only remaining issue is the determination of the asymptotic covariance. This is given in the following Proposition 4. Let Sn, f Sn g be defined as in 23 and suppose that i S f 2 iπ i < and i,j S g2 i, jπi, j <. Then n n CovSf n, Sn g = i,j,k S gi, jfk π i P ij Z jk + π k Z ki P ij 2π i P ij π k. 24 Proof: Begin by expressing the asymptotic covariance as n n CovSf n, Sn g = n n E πsns f n g E π SnE f π Sn, g where E π denotes expectation with respect to the stationary distribution. From 23 we can see that E π S f ns g n = n n l=m+ i,j,k S + n m l=0 k,i,j S 0 gi, jfkπ i P ij P l m jk fkgi, jπ k P ij P l m ki.

11 Also, E π SnE f π Sn g = n 2 i,j,k S gi, jfkπ kπ i,j. have Thus, subtracting, we CovS f n, S g n = n n i,j,k S l=m+ + n m k,i,j S l=0 gi, jfk π i P ij P l m jk π i P ij π k fkgi, j π k P ij P l m ki π k π i P ij. Now the following its can be easily evaluated in terms of the fundamental matrix Z in view of its definition in. n n n n l=m+ n n n l=0 π i P ij P l m jk π k = πi P ij Z jk π i P ij π k, 25 m Taking these into account we obtain The potential matrix for Bernoulli trials π k P ij P l m ki π i = πk Z ki P ij π i P ij π k. 26 In this section, for the sake of simplicity of exposition and due to its intrinsic interest, we will present the analysis for Bernoulli trials. The corresponding results for Markov dependent trials with dependence given by 4 are sketched in the appendix. The transition probability matrix for the process examined in this section is the matrix 5 modified in its first row by setting p 0 = p, q 0 = q. Conditioning on the first transition one easily obtains the mean transition times m ij := E i T j and E π T j = 2k i=0 π im ij. Using 5 and 6 we obtain the following expressions for the elements of the potential matrix. Z jj = jqp j, if 0 j k, p k jqpj p k kqpk+j p k 2, if j > k. 27

12 Z ij = jqp j + p j i, if i < j, j k, jqp j, if i > j, j k, p j i jqp j kqpk+j, i < j, j > k, p k p k 2 p k+j i jqp j kqpk+j, if i > j, j > k. p k p k Explicit expression for the covariance matrix in terms of the stationary distribution and the potential matrix In this section we evaluate the elements of the covariance matrix in the multivariate CLT for the various types of runs based on the ideas of section 3 and the results of section 4. The analysis is presented here again for Bernoulli trials and the corresponding results for two-state Markov dependent trials are relegated to the appendix. In subsection 5. we give results for the asymptotic variances and covariances of M n,k, G n,k, and N n,k whereas in subsection 5.2 we present the results for the asymptotic variance of J n,k and its asymptotic covariance with the other types of runs. 5.. Runs expressed in terms of state visits Let us use the CLT of Theorem for additive functionals of the form Sn f = n fx m with fx = A x where A is an appropriate set of states. The asymptotic variance, as given by 2, taking into account 3 becomes n n VarS n = 2 π i Z ij π i π i π j. 29 i,j A i A i,j A Equations 7, 8, and 9 show that, by choosing appropriately the set A we can obtain the asymptotic variances of the number of success runs N n,k, M n,k, and G n,k. 2

13 Asymptotic Variance of N n,k. Taking 29 with A = {k, 2k} we obtain n n VarN n,k = 2 π k Z k,k + π 2k Z 2k,2k + 2 π k Z k,2k + π 2k Z 2k,k π k π k + π 2k π 2k + 2π k π 2k = qp k 2k + qp k p 2k+ p k 3 which agrees with 2. Asymptotic Variance of M n,k. Taking 29 with A = {k, k +,..., 2k} we obtain n n VarM n,k = + p p k p k 2kp 2k. q Indeed, this agrees with 3 given in Hirano et al. 99. Asymptotic Variance G n,k. Taking 29 with A = {k} we obtain n n VarG n,k = 2π k Z kk π k π k + = qp k + 2kqp k. When k = then the random variable G n, denotes the total number of success runs of length in sample of size n. In this case we have n n VarG n, = qp 3qp which is the expression obtained in Theorem 4 in Fu and Lou, 2007, pg. 20. We next obtain the asymptotic covariances. From Theorem the asymptotic covariance of Sn f = n fx m and Sn g = n gx m when fx = A x and gx = B x is given by n n CovSf n, Sn g = i A, j B π i Z ij + π j Z ji i A π i j B π j i A B π i, 30 where we have again taken into account 3. Again, by appropriate choice of the sets A and B we can obtain the asymptotic covariances for the number of these types of runs. 3

14 Asymptotic Covariance of M n,k, N n,k. Using 30 with A = {k, 2k} and B = {k, k +,..., 2k} we obtain 2k n n CovM n,k, N n,k = + π i π k + π 2k + 2k i=k i=k = p k kqp2k p k. π i Z ik + π k Z ki + π i Z i,2k + π 2k Z 2k,i Asymptotic Covariance of G n,k, M n,k. Using 30 with A = {k} and B = {k, k +,..., 2k} we obtain n n CovG n,k, M n,k = 2k i=k+ + π i π k π k π k + + 2π k Z kk 2k i=k+ = p k p k 2kqp 2k. π i Z ik + π k Z ki Asymptotic Covariance of N n,k, G n,k. Using 30 with A = {k, 2k} and B = {k} we obtain n n CovN n,k, G n,k = qpk k + qp k kqpk. p k p k 5.2. Runs expressed in terms of state transitions Here we apply Theorem 3 on the functional Sn g := n gx m, X m+ with g : S S R an indicator function, namely gx, y = A x, y where A S S. Using 20 we have n n VarSg n = = 2 l,l 2,k,k 2 A l,l 2,k,k 2 S S π l,l 2 Z l,l 2,k,k 2 A l, l 2 Γ l,l 2,k,k 2 A k, k 2 4 l,l 2 A π l,l 2 2 l,l 2 A π l,l 2.

15 We now express the quantities referring to the transition chain in terms of the stationary distribution and the potential matrix of the original chain using 8 to obtain n n VarSg n = 2 l,l 2,k,k 2 A + l,l 2 A π l P l l 2 Z l2 k P k k 2 3 π l P l l 2. l,l 2 A π l P l l 2 The above applies immediately to J n,k, which denotes the number of runs of exact length k, in a set of length n, with A = {k, 0}. Hence we have Asymptotic Variance J n,k. n n VarJ n,k = 2π k Pk0Z 2 0k 3πkP 2 k0 2 + π k P k0 = q 2 p k + 2p q2k + q 3 p 2k. This agrees with the results of Makri and Psillakis, 20, Theorem 2.3. In order to obtain the asymptotic covariance of J n,k with the other types of runs we will use proposition 4 with fx = A x, gx, y = B x, y, where A and B are appropriate sets of states and transitions respectively A S, B S S. Then n n CovSf n, Sn g = 2 π i P ij π k 3 i,j B k A + π i P ij Z jk + π k Z ki P ij. i,j B,k A Asymptotic Covariance G n,k, J n,k. Using 3 with A = {k} and B = {k, 0} we have n n CovG n,k, J n,k = 2πkP 2 k0 + π k P k0 Z 0k + π k Z kk P k0 = q 2 p k p k 2qk + 2q. 2 5

16 Asymptotic Covariance M n,k, J n,k. Using again 3 for A = {k, k+,..., 2k} and B = {k, 0} we obtain n n CovM n,k, J n,k = 2π k P k0 2k i=k π i + 2k i=k = q 2 p k p k + 2k. π k P k0 Z 0i + π i Z ik P k0 Asymptotic Covariance N n,k, J n,k. Finally 3 with A = {k, 2k} and B = {k, 0} gives n n CovN n,k, J n,k = 2π k P k0 π k + π 2k + π k Z kk P k0 + π k P k0 Z 0k +π 2k Z 2k,k P k0 + π k P k0 Z 0,2k = q2 p k 2qp k kq3 p 2k p k p k 2 2 pk. 6. The Central Limit Theorem for the joint number of runs Define an additive functional of the transitions and the state visits of the Markov chain {X n } via a function f = f, f 2, f 3, f 4 : S, S, S, S S R 4 so that n fx m = N n,k, M n,k, G n,k, J n,k. The Strong Law of Large Numbers for Markov chains gives n n n f X m, f 2 X m, f 3 X m, n n f 4 X m, X m+ µ w.p.. where µ = µ, µ 2, µ 3, µ 4 with µ = i S π if i = π k + π 2k = qpk, p k µ 2 = i S π if 2 i = 2k i=k π i = p k, µ 3 = i S π if 3 i = π k = qp k, µ 4 = i S π i,jf 4 i, j = π k0 = qπ k = q 2 p k. Then we can summarize the results of this paper for Bernoulli trials in the following Theorem 5. With the above definitions the following Central Limit theorem holds for the number of the types of success runs of size k considered. n d N0, V n f X m f 2 X m f 3 X m f 4 X m, X m+ where V is the variance-covariance matrix whose elements were determined in the previous sections. 6 µ µ 2 µ 3 µ 4

17 A corresponding CLT holds for Markov dependent trials. The elements of the covariance matrix for dependence of the form given in 4 are those given in the Appendix. The corresponding means are given by µ = µ 2 = p 0p k q+p 0, µ 3 = qp 0p k q+p 0, µ 4 = q2 p 0 p k q+p 0. An application to manufacturing processes with random yield. qp 0p k q+p 0 p k, Consider an unreliable manufacturing cell where an item is produced when the machine successfully completes k consecutive steps, each requiring a unit of time. If a failure occurs at any given step, no item is produced and the process starts again from scratch. Thus the total number of nonoverlapping runs, N n,k gives the number of items produced in a period of n time units. Considering this cell as part of a manufacturing process, we are also interested in the flow of items downstream from the cell, to the next stage of the process. If items produced consecutively, with no failures intervening, are batched together then a stream of batches of random size, with random interarrival intervals arrives at the next station. Since the total number of batches in a time period of n units is G n,k, knowledge of the asymptotic joint distribution of G n,k, N n,k can be used to obtain approximations for the production rate and flow times in such processes. Note that, while VarN n,k gives the variability of the total number of arrivals at the downstream station in the time period in question, CovG n,k, N n,k and VarG n n, k can be used to characterize in an aggregate fashion the temporal smoothness of the arrival process which also affects queueing aspects and delays significantly. 7. A general Markov dependent model Let {ξ n ; n N} be a Markov chain with finite state space S and irreducible transition probability matrix P B. We partition the state space S into two disjoint sets, S 0 and S, such that S = S 0 S and correspondingly partition P B into a block matrix [ ] Q0 P P B = Q P Q 0, P, are square matrices of dimension m 0 := S 0 and m := S respectively while P 0 and Q are rectangular m 0 m and m m 0 matrices. In order to simplify the analysis we will assume that the substochastic matrix P is strictly positive, i.e. that it contains no zero entries. 7

18 We now consider the process { S ξ n ; n N} which is a sequence of dependent binary trials and use the approach of the previous sections in order to obtain approximations for the number of success runs of various types based on the Central Limit Theorem. Suppose we are interested in success runs of size k. Let {X n ; n N} be a Markov chain with state space consisting of m 0 + 2km elements. It will be convenient to label its states as {i, r; i = 0,, 2,..., 2k; r =, 2,..., m 0 if i = 0 and r =, 2,..., m if i > 0} and to order them lexicographically. The transition probability matrix of {X n } is P = Q 0 P 0 Q P.... Q P Q P.... Q P Q P. 33 Under the assumption of the irreducibility of the matrix P B and the additional assumption of strict positivity of P, the matrix P is again irreducible and aperiodic. The stationary distribution can be obtained by solving the system π = πp together with the normalization condition. It will be convenient to partition the stationary distribution as π = [π 0, π,..., π k ] where π 0 is a m 0 row vector and π i, i =,..., 2k, are m row vectors. We will denote these row vectors as π 0 := [π 0,, π 0,2,..., π 0,m0 ] and π i := [π i,, π i,2,..., π i,m ], i =, 2,..., 2k. If u 0 is a m 0 column vector with all entries equal to, and u a m column vector with all entries equal to, then the normalization condition can be written as π 0 u 0 + 2k i= π i u =. 34 8

19 The stationary equations are π 0 = π 0 Q 0 + k π i Q, 35 i= π = π 0 P 0, 36 π i = π i P, i = 2, 3,... k, and i = k + 2,..., 2k, 37 π k+ = π k P + π 2k P. 38 From 36, 37, and 38 we obtain and thus 2k i= π i = π 0 P 0 P i, i =, 2,..., k, 39 π i = π 0 P 0 P i I P k, i = k +, k + 2,..., 2k, 40 π i = π 0 P 0 I + P + + P k + π0 P 0 P k + + P 2k I P = π 0 P 0 I P k I P + π 0 P 0 P k I P k I P I P k = π 0 P 0 I P, where we have used the fact that I P and I P k commute. Hence 35 becomes π 0 = π 0 Q0 + P 0 I P Q. 4 The normalization condition 34 becomes π 0 u 0 + π 0 P 0 I P u =. 42 Note that P W := Q 0 +P 0 I P Q is in fact an m 0 m 0 stochastic matrix which gives the transition probability matrix of the chain {ξ n } watched in the set S 0. Indeed, if one defines a sequence of stopping times {W n } with W 0 := inf{n 0 : ξ n S 0 } and W n := inf{n > W n : ξ n S 0 } the process {Y n ; n = 0,, 2,...} defined via Y n := ξ Wn is a Markov chain with transition probability matrix given by P W. The stochastic matrix P W inherits its irreducibility from that of P B and thus 4 has an essentially unique positive solution, within a multiplicative constant which can be determined so that 42 is satisfied. In general, the stationary distribution can be computed numerically from the above equations while the recurrent potential matrix can be computed 9

20 numerically using 4 where Π is the m 0 + 2km m 0 + 2km matrix with constant rows. Its elements, Π ij, i, j =, 2,..., m 0 + m 2k, are given by Π ij = π 0,i if i m 0 and Π ij = π l,i m0 lm if m 0 + l m < i m 0 + lm, l =, 2,..., 2k. The number of the various kinds of runs considered can be expressed as additive functionals of the Markov chain {X n } given by the following expressions. N n,k = M n,k = G n,k = J n,k = n m X j = k, r + X j = 2k, r j=0 r= n 2k m j=0 i=k X j = i, r r= n m X j = k, r j=0 r= n m m 0 r= l= X m = k, r, X m+ = 0, l The multivariate CLT follows then again by Theorems and 3 while the covariance matrix is obtained as in section 5. The sets A and B that appear there are of course more complicated due to the two-dimensional representation of the state space as is clear from the above expressions for the number of runs. 8. Joint asymptotic distribution of non-overlapping runs of different sizes In this section we illustrate the generality of the method we propose by obtaining the joint asymptotic distribution of non-overlapping success runs of different lengths. This problem has been considered by Jennen-Steinmetz and Gasser 986 as mentioned in the introduction. Here we obtain analogous results for Markov dependent trials with dependence given by 4. Let {X n } be a Markov chain with countable state space S := {0,, 2,...} and transition probability matrix P given by P 00 = q 0, P 0 = p 0, P i,i+ = p, P i0 = q and all other entries equal to zero. This chain is clearly irreducible, 20

21 aperiodic, and positive recurrent with stationary distribution given by π 0 = q p 0 + q, π i = q p 0 + q p 0p i, i =, 2, The corresponding potential matrix can be obtained from 5, 6, by means of elementary calculations similar to those of section 4. Z ij = qp 0p j p p0 p 0 + q p 0 + q + j + i jp j i. 44 If k i, i =, 2,..., ν, are fixed positive integers with 0 < k < k 2 < < k ν we are interested in determining the joint asymptotic distribution of the number of non-overlapping runs N k,n, N k2,n,..., N kν,n, as the number of trials n. Since N ki,n = X m A i with A i = {lk i, l =, 2,...} 45 the joint asymptotic distribution will be multivariate Normal by the CLT for additive functionals of Markov chains in Theorem. The asymptotic variances have already been determined in section 5 for Bernoulli trials and in the Appendix in the Markov dependent case. Therefore the only remaining issue is the determination of the asymptotic covariance between, say, N k,n and N k2,n. Using 30 and taking into account 45 we have n n CovN k,n, N k2,n = π ik Z ik,jk 2 + π jk2 Z jk2,ik 46 j= i= π i lcmk,k 2 π ik i= i= j= π jk2, where lcmk, k 2 stands for the least common multiple of k and k 2. The evaluation of the asymptotic covariance using the expression for the stationary distribution 43 and the potential matrix 44 is straight forward. The only term that complicates matters is a term of the form j= i= ik jk 2 p jk 2 +jk 2 ik p ik which arises due the presence of the indicator function in 44 and which requires separate consideration according to whether k divides k 2, and if not, according to whether the remainder of the integer 2

22 division of k 2 by k divides k. We thus have three separate expressions corresponding to these three cases. where J = n n CovN k,n, N k2,n = p k 2 p k 2 + p p0 + p 0 + q k 2 k p k 2 + p k ap k 2 p k qp0 p 2 p k +k 2 q + p 0 p + k p k 2 p k a p k ac+ p k ac+ p k p k a pk ac+ p k p k 2 + qp 0p J 47 q + p 0 if k 2 = ak if k 2 = ak + b, k = bc ap k 2 + p ka pk ca p k plcmk,k 2 p k 2 2 p k p k a p k ca+ p lcmk,k 2 if k 2 = ak + b, k = bc + d. A. Explicit expressions for Markov dependent trials Here we present the results for Markov dependent trials that follow 4. A.. The elements of the potential matrix Z ij for Markov dependent trials Z jj = p k + jqp 0p j q+p 0 qp 0p j j + q+p 0 p k p p 0 q+p 0 p 0 p j + jqp 0 p j q+p 0 p 0 qp p 0 p q+p 0, if j = 0, 2 qp 0p j q+p 0 2, if j k. kqp 0p k+j q+p 0 p k 2, if j > k. qp 0p j q+p 0 2, if i = 0, j k, Z ij = q+p 0 p j i + jqp 0 p j q+p 0 jqp 0 p j q+p 0 qp 0p j q+p 0 2, qp 0 p q+p 0 2, j = 0 if i < j k qp 0p j q+p 0 2, if i > j, j k. 22

23 Z ij = p 2 0 pj + jq+p 0 qp 0 p j q+p 0 2 p k kqp 0p k+j q+p 0 p k 2, if i = 0, p j i + jqp 0p j p k q+p 0 p k qp 0p j q+p 0 2 p k kqp 0p k+j if i < j, j k +, q+p 0 p k 2 p k+j i p k + jqp 0p j q+p 0 p k qp 0p j q+p 0 2 p k kqp 0p k+j q+p 0 p k 2 if i > j, j k +. A.2. The elements of the covariance matrix for Markov dependent trials n n VarN n,k = n n VarM n,k = p 0p k p 0 + q n n VarG n,k = qp 0p k p 0 + q qp 0 p k p 0 + q p k 2 p k q2 q q 0 p 2 0 p 0 + q 2 k 2 q 2p q q p 0p k q + p 0 qp 0p k n n VarJ n,k = q2 p 0 p k + q3 p 2 0 p2k 2 p 0 + q q + p 0 2 q + 2 q + p 0 p 0 + q 2 2p 0 + qk + p + q 0 2p0 2k + q q + p 0 2qp 0 p k p 0 + q p k n n CovM n,k, N n,k = p 0 p k p 0p k p 0 + q p k n n CovG n,k, M n,k = p 0p k qp 0p k p 0 + q p 0 + q p 0 + q 2 p 0 + q + 2pp p 0 p p 0 p 0 + q 2 pk +k qpk p k 2k + p q + p + q 0 p 0 + q n n CovN n,k, G n,k = qp 0 p k p 0 + q p k qp + q 0p 0 p k p 0 + q 2 k qp 0p k 2 p k p 0 + q p k n n CovG n,k, J n,k = q2 p 0 p k q2 p 2 0 p2k 2 q + p 0 p 0 + q 2 2kq + q p 0 q + p 0 23

24 n n CovM n,k, J n,k = q2 p 0 p k p 0p k 2k + p + q 0 p 0 + q p 0 + q p 0 + q n n CovN n,k, J n,k = q2 p 0 p k p 0 + q p 0 p k p 0 + q p k 2 p k p k qk + q p 0 q + p 0 References Aldous, D., Fill, J. Reversible Markov Chains and Random Walks on Graphs. aldous/rwg/book.html Balakrishnan, N., Koutras, M. V., Runs and Scans with Applications, Wiley, New York. Barbour, A.D., Chryssaphinou, O., 200. Compound Poisson approximation: A user s guide. The Annals of Applied Probability,, 3, Brémaud, P., 997. Markov Chains, Springer Verlag. Feller, W An Introduction to Probability Theory and Its Applications vol., 3rd ed. John Wiley, New York. Fu, J.C., Koutras, M.V., 994. Distribution theory of runs: a Markov chain approach. J. Amer. Statistic. Assoc. 89, Fu, J., Lou, W.Y.W., Distribution Theory of Runs and Patterns and its Applications, World Scientific, Singapore. Fu, J., Lou, W.Y.W., On the normal approximation for the distribution of the number of simple or compound patterns in a random sequence of multi-state trials. Methodology and Computing in Applied Probability, 9, Fu, J., Lou, W.Y.W., Bai, Z.D., Li G., The exact and iting distributions for the number of successes in success runs within a sequence of Markov-dependent two-state trials. Annals of the Institute of Statistical Mathematics, 54, Godbole, A.P., 992. The exact and asymptotic distribution of overlapping success runs. Communications in Statistics Theory and Methods, 2, 4,

25 Hirano, K., Aki, S., Kashiwagi, N., Kuboki, H., 99. On Ling s binomial and negative binomial distributions of order k. Statistics and Probability Letters,, Jennen-Steinmetz, C., Gasser, T., 986. The asymptotic power of runs tests. Scandinavian Journal of Statistics, 3, Makri, F.S. and Z.M. Psillakis, 20. On success runs of a fixed length in Bernoulli sequences: Exact and asymptotic results. Computers and Mathematics with Applications, 6, Port, S.C., 994. Theoretical Probability for Applications. John Wiley. Reinert, G., Schbath, S. and Waterman, M Probabilistic and statistical properties of words: an overview. Journal of Computational Biology, 7, Rukhin, A Pattern correlation matrices and their properties. Linear Algebra and its Applications 327, Rukhin, A Pattern correlation matrices for Markov sequences and tests for randomness. Theory of Probability and its Applications 5, Rukhin, A Joint distribution of pattern frequencies and multivariate Pòlya-Aeppli law. Theory of Probability and its Applications 54, Stefanov, V. T Explicit it results for minimal sufficient statistics and maximum likelihood estimators in some Markov processes: Exponential families approach. Annals of Statistics 23, Stefanov, V.T., On run statistics for binary trials. Journal of Statistical Planning and Inference, 87, Stefanov, V.T., and A.G. Pakes 997. Explicit distributional results in pattern formation. Annals of Applied Probability 7,