Memory Loss Property for Products of Random Matrices in the Max-Plus Algebra

Transcription

1 MATHEMATICS OF OPERATIONS RESEARCH Vol. 35, No. 1, February 2010, pp issn X eissn informs doi /moor INFORMS Memory Loss Property for Products of Random Matrices in the Max-Plus Algebra Glenn Merlet Institut de Mathematiques de Luminy, Université de la Méditerranée, F Marseille, France, Products of random matrices in the max-plus algebra are used as models of a wide range of discrete event systems, including train or queueing networks, job shops, timed digital circuits, or parallel processing systems. Several mathematical models such as timed event graph or task-resources models also lead to max-plus products of matrices. Some stability and computability results, such as convergence of waiting times to a unique stationary regime or limit theorems for the throughput, have been proved under the so-called memory loss property (MLP). When the random matrices are i.i.d., we prove that this property is generic in the following sense: if it is not fulfilled, the support of the common law of the random matrices is included in a union of finitely many affine hyperplanes. Key words: max-plus algebra; random matrices; discrete event systems MSC2000 subject classification: Primary: 93C65; secondary: 93B25, 68R10, 15A52 OR/MS subject classification: Primary: probability, Markov processes; secondary: mathematics, matrices History: Received December 13, 2004; revised December 27, 2005, October , June 17, 2007, March 14, 2008, and February 15, Published online in Articles in Advance January 27, Introduction. There are many applied probability models having dynamics that can be described by products of random matrices in the max-plus algebra. We can mention computer networks (Baccelli and Hong [2]), train networks (Heidergott and de Vries [14], Braker [5]) or flexible production lines (Cohen et al. [8]). Several mathematical models, such as queueing networks (Mairesse [16], Heidergott [13]), task-resources models (Gaubert and Mairesse [11]), and timed Petri nets, including events graphs (Baccelli [1]) and 1-bounded Petri nets (Gaubert and Mairesse [12]) also lead to max-plus products of matrices. Roughly, the max operator reflects the synchronization of different events and the plus operator corresponds to the passing of time. For more developments on the max-plus modelling power, see the books by Baccelli et al. [3] or Heidergott et al. [15]. In this paper, we review the queueing networks, which are max-plus linear, following Heidergott [13], and we study in detail the example of transmission control protocol (TCP) window flow control, using a simplified version of the example presented in Baccelli and Hong [2]. We have claimed that max-plus products are relevant to modelling real-life situations. Now, the natural question is whether this modelling can be used to derive results, which are interesting from an applied point of view. For instance, max-plus linearity implies the existence of an asymptotic throughput. Unfortunately, it is, in general, very difficult to estimate it. A sufficient condition under which computational results were obtained is the so-called memory loss property (MLP). It implies the convergence of waiting times to a unique stationary regime (Mairesse [16]) and analyticity of the asymptotic throughput (Gaubert and Hong [10]). Moreover, if a max-plus linear system has the MLP and satisfies some integrability and stochasticity conditions, then the sequence of the dates of the events occurring in the system satisfies a central limit theorem (CLT), a local CLT, a renewal theorem, and a large deviations principle. These results are stated formally in 3.1. To the best of our knowledge, it is not known how to verify that the MLP holds. This brings us to the aim of this paper. When the sequence of matrices representing the model is i.i.d., we prove that the MLP is generic in the following sense: if the model does not have the MLP, then the support of the common law of the random matrices is included in a union of finitely many affine hyperplanes. Moreover, the proof leads to an explicit, although not very tractable, condition that ensures that the MLP holds (see Remark 5.1). This result implies that the TCP model of 2.2 has the MLP, as soon as the processing times in the routers are nondegenerate random variables. Therefore the throughput of the flow of packets satisfies limit theorems, the waiting times converge and the asymptotic throughput can be estimated with the methods developed in Gaubert and Hong [10]. The result also implies that aperiodic closed max-plus queueing networks have the MLP, as soon as the processing times in the nodes and the communication times on the arcs are nondegenerate random variables. Again, the throughput satisfies limit theorems, the waiting times converge, and the asymptotic throughput can be estimated with the methods developed in Gaubert and Hong [10]. If the network is simple enough, the formulation given in Remark 5.1 allows one to compute explicit conditions for the network to have the MLP. 160

2 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 161 The proof of our main theorem is based on the interpretation of entries of products of matrices as weights of paths on a finite graph. Firstly, we define a change of basis, to make one of the matrices nonpositive. Secondly, we use the nonpositivity of the entries to construct a matrix M whose iterates ultimately have max-plus rank 1. Because the MLP holds when products have rank 1 with positive probability, this concludes the proof for random matrices with discrete support. Finally, we show that products of sufficiently many matrices that are close to M also have rank 1, which concludes the proof in the general case. An outline of this paper is the following. In the first section, we introduce the max-plus algebra and illustrate its modelling power with the TCP example and a review of its applicability to queueing networks. In the second section, we define the MLP and explain its consequences in the models, then we state our results and comment on them. The asymptotic theory of matrices in the max-plus algebra is recalled in the third section, together with a useful extension. The main results are proved in the last section. 2. Modelling Max-plus algebra. Consider a stationary sequence of random matrices A n n and a random vector X 0 valued in k. We define a sequence of k valued random variables X n n by the recurrence relation X n+1 i = max A ij n + X n j (1) j Max-plus linear systems are those that can be described by such a sequence. These sequences are best understood by introducing the so-called max-plus algebra, which is actually a semiring. Definition 2.1. The max-plus semiring max is the set, equipped with the two operations max and +. Observe that the operation + is distributive with respect to the operation max. So, the additive operation of the semiring is the max, while the multiplicative operation is the +. We materialize this by using the notations a b = a + b and a b = max a b. The identity elements are and 0. We also use the matrix and vector operations induced by the semiring structure. For A, B with appropriate sizes (vectors are identified to matrices with one column), A B ij = A ij B ij = max A ij B ij, A B ij = l A il B lj = max l A il + B lj, and for a scalar a max, a A ij = a A ij = a + A ij.ifv 1 and V 2 are sets of matrices, V 1 V 2 will denote the set of all max-plus products of a matrix of V 1 by a matrix of V 2. Given an integer n, we denote by 1 n the set 1 n. Definition 2.2. Let = A n n be a stationary sequence of random k k-matrices with entries in max and at least one finite entry in each row. The sequence x n x 0 n is defined by x 0 x 0 = x 0 and x n + 1 x 0 = A n x n x 0 With this notation, the random variable X n of Equation (1) is exactly x n X 0. For more background on this framework, the reader is referred to the books by Baccelli et al. [3] or Heidergott et al. [15] Practical situations. To illustrate the modelling power of max-plus products of random matrices, we present a simplified version of the model of TCP/Internet protocol (IP) window flow control mechanism proposed by Baccelli and Hong [2]. The analysis would work for more complex networks, as will be shown in 2.3. Baccelli and Hong [2] study a single source sending packets to a single destination over a path made of routers in series, the transmission of the packets of this reference flow being TCP controlled. Each router is represented by a single-server queue. Each queue serves the packets of the reference flow as well as the ones of other flows, known as cross flows. Each router is assumed to be a first-in first-out queue for the packets of the reference flow. For the sake of simplicity, we will restrict our attention to the simplest case, with only two queues: the source and the destination (see Figure 1). The n-th packet of the reference flow requires service times and at the source and destination servers, respectively. These service times capture the processing time of the packet by the router, the propagation delays as well as the delays induced by cross traffic. We will assume that and are deterministic, to allow explicit computations. The input rate is controlled by a size-varying window, which determines the maximum number of packets sent by the source that have not been acknowledged by the destination. Let us denote the size of the window viewed by n-th packet by w n and assume this number is bounded by w. For more details about w n, which depends on the version of the TCP/IP protocol considered, see Baccelli and Hong [2].

3 162 Mathematics of Operations Research 35(1), pp , 2010 INFORMS Source Destination Input Transmission Output Acknowledgment Figure 1. A TCP-controlled simple route. Let us denote by f n and l n the departure times of the n-th packet from the first and last queues, respectively. At time f n, packet number n + 1 enters the first server. At time f n +, it is ready to leave the first queue, but it has to wait for packet number n + 1 w n+1 to be acknowledged by the last queue, which occurs at time l n + 1 w n+1. Therefore we have the following equation: f n+ 1 = max f n + l n+ 1 w n+1 Packet number n enters the last server as soon as it has arrived in the buffer of the queue (at time f n ) and packet number n 1 has left the server (at time l n 1 ). Therefore we have the following equation: l n = max f n l n 1 + = max f n + l n 1 + Thus the w -dimensional vectors X n = f n l n 1 l n w + 1 satisfy Equation (1), with matrices A n taking values A wn whose entries are equal to except for the ones defined below: A w 11 = A w 1w = 0 A w 21 = A w 22 = i 3 w 1 A w i i+1 = 0 (2) In our simplified model, the randomness lies only in w n, but it would be possible to deal with random and Queueing networks. A queueing network is a system in which items (thought of as clients, packets, or jobs) circulate between nodes called queues. Each queue consists of a buffer and possibly several servers. If an item arrives at a node at which a server is available, it remains there until its service is completed. During this service time, the server is busy. Times for an item to go from one queue to the next one are called communication times. At some nodes, called join nodes, the service can only begin when one item from each of the upstream nodes has arrived and the upstream items merge into one item. At others, called fork nodes, an item can split into several items going to different nodes after completion of service. If an item arrives at a node at which all servers are busy, then it has to wait for service in the buffer until a server becomes available. If an item has to go to an already full buffer, it is blocked. (This is called a blocking after service scheme.) To define the dynamics of the network, we still have to choose a queueing discipline (the order in which the items in the buffer are served) and a blocking discipline. (The order in which the nodes blocked by the same particular node are unblocked.) Heidergott [13] provides a detailed analysis of which queueing networks are max-plus linear and which ones are not. At the end of the present section, we summarize his main conclusion. We say that there is internal overtaking if the order in which items leave a node is different from the order in which they entered the node. A resequencing node is such that an item whose service is completed remains in the server until the service of all items that entered the queue before this particular item is finished. We say that there is no routing if the items reaching one node always originate from the same set of nodes, and the items leaving the node always go to the same set of nodes. Moreover, we state the following definitions. Definition 2.3. For a queueing network with J nodes, we denote by x i n the time of the nth departure from node i. We call the network max-plus if there exist an M and a sequence of matrices A n with size JM such that the sequence of daters X n = x n x n 1 x n M + 1 T satisfies Equation (1). We consider the probabilistic version of the network, in which all interarrival, communication and service times are random and stationary. We say that the network is max-plus with fixed support if for each i j, the entry A ij n is either finite valued a.s. or equal to a.s.

4 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 163 Theorem 2.1 (Heidergott [13]). A queueing network with only one class of items, no state-dependent service times, first-come, first-served queueing discipline and first-blocked, first-unblocked blocking discipline is max-plus with fixed support if and only if it admits no routing, no internal overtaking, and all resequencing nodes have only finitely many servers. As noticed in Heidergott [13], other blocking schemes (including blocking before service, or general blocking mechanism, see [13]) yield max-plus networks. On the other hand, networks with other queueing discipline or class-dependent and state-dependent service times are generally not max-plus. The main restriction is that, in a max-plus network, the departure time at a given node is determined by finitely many previous departure times. This is not the case with Bernoulli routings. For instance: if the items leaving node 1 are sent either to node 2 or 3, according to a Bernoulli routing, and their departure times are driven by acknowledgements from items going to node 2 only, then the n-th departure time from node 1 depends on the last departure to node 2. This departure is the n m th departure from node 1, in which m is random and can be any positive integer. 3. MLP Preliminaries. It has been known since Cohen [6], that xi n x 0 n satisfies a law of large numbers, at least when the entries are finite and integrable. The limit of 1/n max i xi n 0 n exists under the assumption that max i j 1 k A ij (see Vincent [19]) and is called the Lyapunov exponent of A n n. Definition 3.1 (Lyapunov Exponent). The limit of 1/n max i xi n 0 n is called the Lyapunov exponent of A n n. This gives the first general result about max-plus networks: the asymptotic throughput rate is well defined: it is the inverse of the Lyapunov exponent. Two natural questions arise. How to compute or approximate this throughput? Is there a stationary regime for waiting times and are the waiting times converging to this regime? As far as we know, the answer to these questions is only known under the additional assumption that the sequence A n n has the so-called MLP. Before stating the MLP, we recall a few basic facts about matrices with entries in max. All these facts can be checked by direct computation. We denote by k l max the set of matrices with k rows and l columns of entries in max. (i) A matrix A k l max is identified with the max-plus linear map from l max to k max given by x l max A x = A x and the product of the matrices corresponds to the composition of the maps. If A has at least one finite entry in each row, it also defines a map from l to k. (ii) The image of A is the set of max-plus linear combinations of the column vectors of A. More precisely, we have x k max A x = j 1 k x j A j. Definition 3.2. MLP (i) A matrix A has rank 1 if there exist a and b in k max \ k such that i j 1 k 2, A ij = a i + b j. We denote it by rk A = 1. (ii) A sequence A n n of matrices is said to have MLP if there exists an N such that rk A N A 1 = 1 >0. (iii) A pair A B of matrices is said to have the MLP if there is a matrix with rank 1 in the semigroup generated by A and B. Remark 3.1. A matrix A has rank 1 if and only if its image is the max-plus line max a, where a is the vector in the definition. A pair A B of matrices has the MLP if and only if any sequence of i.i.d. random matrices that take values A and B with positive probability has the MLP. The name memory loss property was introduced by Gaubert and Hong [10] for max-plus automata. It states that rk A n A 1 = 1 if and only if, for every pair i j, the difference xi n x 0 xj n x 0 does not depend on the initial condition x 0. Let us review the three types of results that were proved under the MLP hypothesis. Mairesse [16] introduced the MLP to show the following result. Theorem 3.1 (Mairesse [16, Theorem 6.15]). If has the MLP, then for every i j 1 k, the sequences x i n x 0 x j n x 0 n and x i n + 1 x 0 x j n x 0 n converge in total variation, uniformly in x 0, and the limits do not depend on x 0.

5 164 Mathematics of Operations Research 35(1), pp , 2010 INFORMS The other two results show that the MLP ensures a kind of stability for the max-plus Lyapunov exponent. It allows us to approximate the exponent, which is generally the best we can do, because the computation is very difficult. Indeed, Blondel et al. [4] proved that approximating the max-plus Lyapunov exponent is NP-hard. If A 1 takes only finitely many values, then the MLP obviously only depends on these values, and not on the law of A 1. Let A 1 A t be matrices in k k max with at least one finite entry in each row. For every probability vector p, let A p n n be a sequence of i.i.d. matrices such that i 1 t A p n = A i = p i. Let L p be the Lyapunov exponent of this sequence. If there is a p such that A p n n has the MLP, then for any p such that i 1 t p i > 0, the sequence A p n n has the MLP. That being the case, we have the following result on the Lyapunov exponent. Theorem 3.2 (Gaubert and Hong [10, Theorem 4.1]). Let A 1 A t be matrices in k k max with at least one finite entry in each row. If there is a probability vector p such that A p n n has the MLP, then L is an analytical function of p and the domain of analyticity includes all p such that i 1 t p i > 0. It has been proved in Merlet [17, 18] that, if = A n n has the MLP and the A n are sufficiently integrable and mixing, then the sequence x n X 0 n satisfies a CLT. If the A n are independent, then the sequence also satisfies a local CLT, a renewal theorem, and a large deviation principle. We end this section by applying these results to the example of 2.2. When does the sequence A n taking values in the set of matrices A w defined by Equation (2) have the MLP? The asymptotic theory of max-plus matrices, recalled in 4.1, ensures that there is a power of A w of rank 1, as soon as (cf. Lemma 4.1). Therefore the MLP is generic. On the other hand, let us show that if =, then the sequence does not have the MLP. In this case, let us define the vectors u = 2 3 w 1 T and v = 2 3 w T. For any w, A w maps u on u and v on v, thus the image of A n A 1 always contains u and v, which are not in the same max a. Thus A n A 1 never has rank 1. Applying the results from Mairesse [16], Gaubert and Hong [10], and Merlet [17, 18], we get the following result. Proposition 3.1. In the model of TCP-controlled flow described in 2.2, if the window sizes are assumed to be i.i.d. and, then the time between two consecutive packets at each router converges in total variation to a unique stationary distribution. Moreover, the asymptotic throughput is an analytic function of the distribution of the window s size, and the sequence of departure times satisfies a CLT with rate, a large deviation principle, and a renewal theorem. In more general models, such as the queueing networks described in 2.3, it is often unknown how to check the MLP. The aim of this paper is to prove that it generically holds. We even derive some explicit sufficient conditions in Remark 5.1. Unfortunately, these conditions are not very tractable, because there are more than k! 2 conditions to check for a network of k queues and each of these conditions involves several communication or sevice times Statements ofthe results. Let M k denote the set of k k matrices with entries in max and at least one finite entry in each row and k be the set of the primitive matrices, that is the matrices A M k such that there exists an integer n such that A n has only finite entries. Definition 3.3. For a finite set Q and Q, let f be the function from Q max to max defined by i V i f V = i 0 if i Q i 0 V i where the operations are understood in the usual sense otherwise. Such a function is called a linear form on Q max. A hyperplane (resp. affine hyperplane) of Q max is the set of zeros (resp. the level line associated to a finite level) of a nonzero linear form. Below, we see M k as a subset of k2 max and k M k as a subset of 2k2 max. A hyperplane of M k (resp. k ) is the intersection of a hyperplane of k2 max with M k (resp. k ). A hyperplane of k M k is the intersection of a hyperplane of 2k2 max with k M k. When Q = k 2, for all indices i j 1 k, we denote by A ij the function A A ij. With this situation, we write f = ij A ij, and the decomposition is unique. Our results are summarized in the following two theorems.

6 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 165 Theorem 3.3. Atomic case (i) For every k 2, the pairs of matrices that do not have the MLP is included in the union of finitely many hyperplanes of k M k. (ii) For a pair A B outside these hyperplanes, any stationary sequence A n n has the MLP, provided it satisfies the following relation: N A i i 1 N A B N i 1 N A i = A i >0 To state a similar result for continuous measures, we need to define the support of a measure. To this aim, we set the following distance on M k : d A B = max arctan A ij arctan B ij i j 1 k Theorem 3.4. General case (i) Let be a probability measure on M k with support S.IfS k is not included in a union of finitely many affine hyperplanes of k, then a sequence of i.i.d. random matrices with law has the MLP. (ii) For such a probability measure, a stationary sequence A n n has the MLP, provided A 1 has law and for every N, the product set S k N is included in the support of A n n 1 N. 4. Proofofthe theorems Asymptotic theory ofmatrices. In this section, we briefly review the spectral and asymptotic theory of max-plus matrices. For a complete exposition, see Baccelli et al. [3] or Heidergott et al. [15]. Definition 4.1. A circuit on a directed graph is a closed path of the graph. Let A be a square matrix of size k with entries in max. (i) The graph of A is the directed weighted graph whose nodes are the integers between 1 and k and whose arcs are the i j such that A ij >. The weight of i j is A ij. The graph of A will be denoted by A and the set of its elementary circuits by A. (ii) The weight of the path pth = i 1 i n i n+1 is w A pth = n j=1 A i j i j+1 Its length is pth = n (iii) The average weight of a circuit c is aw A c = w A c / c (iv) The max-plus spectral radius of A is max A = max c A aw A c. (v) The critical graph of A is obtained from A by keeping only the nodes and arcs, which belong to circuits with average weight max A. It will be denoted by c A. (vi) The cyclicity of a strongly connected graph is the greatest common divisor of the length of its circuits. The cyclicity of a general graph is the least common multiple of the cyclicities of its strongly connected components. The cyclicity of A is the cyclicity of c A and is denoted by c A. (vii) The type of A is sccn-cycc, where N is the number of strongly connected components of c A and where C is the cyclicity of A. Remark 4.1. Interpretation of powers with A. Observe that A n ij is the maximum of the weights of the paths from i to j with length n. Since the average weight of a circuit is an affine combination of the average weights of its minimal subcircuits, the max-plus spectral radius is the maximum of the average weights of all circuits. Let us recall the (max +)-spectral theory. If max and V k max \ k satisfy the equation A V = V, we say that is an eigenvalue of A and V is an eigenvector. For every A M k, the matrix à defined by à ij = A ij max A satisfies max à = 0 and for every vector V, we have A V = max A à V. In the sequel, we will therefore deal only with the case max A = 0. For every A k k max with max A 0, we set A + = n 1 A n Remark 4.2. Remark 4.1 implies that A + ij is the maximum of the weights of paths from i to j. Since max A 0, all circuits have nonpositive weights, and removing circuits from a path makes its weight greater, thus A + = n 1 k A n. Proposition 4.1. Eigenvectors (Cohen et al. [7, 8]). If c is a circuit on c A, then its average weight is max A. If A is strongly connected, then max A is the only eigenvalue of A. If moreover max A = 0, then we have the following. (i) For every i c A, A + i is an eigenvector of A with eigenvalue 0. (ii) For every eigenvector y of A with eigenvalue 0, we have y = i c A y i A + i.

7 166 Mathematics of Operations Research 35(1), pp , 2010 INFORMS (iii) For any indices i and j in the same strongly connected component of c A, the column vectors A + i and A + j are in the same max-plus line max A + i. (iv) No column vector A + i with i c A is a max-plus linear combination of the A + j with j in other strongly connected components. Proposition 4.2. Powers (Cohen et al. [7, 8]) assume A is strongly connected, max A = 0 and c A = 1. Then, there exists N such that, for all n N, we have A n = Q, where Q is defined by i j 1 k 2 Q ij = l c A A+ il A+ lj. Remark 4.3. For every i j 1 k 2, Q ij is the maximum weight of the paths from i to j that cross c A. Because we are interested in matrices of rank 1, we will use the following consequence of Propositions 4.2 and 4.1. Lemma 4.1. An scc1-cyc1 matrix with finite entries has a power of rank 1. Proof. Let à be the matrix defined by à ij = A ij max A. Then max à = 0 and c à = c A = 1. By Proposition 4.2, when n is greater than some N, the column vectors of à n are max-plus eigenvectors of Ã. Therefore, by Proposition 4.1, all these vectors are in the same max-plus line, thus à n has rank 1. But for every i j 1 k, A n ij = à n ij + n max A, therefore A n also has rank 1. We end this section with the following simple but crucial lemma. Lemma 4.2. For every A k, there exists N such that every path from i to j on A with length n N, and weight A n ij crosses c A. Because this lemma is implicit in the published proofs of Proposition 4.2 (Cohen et al. [8], Baccelli et al. [3]), we prove it for the sake of completeness. Proof. Without loss of generality, we assume max A = 0. Let c be a circuit on c A. By the definition of k, there exists N 1 such that A N 1 has only finite entries. Then A N 1 +1, A N1+2 A N1+ c 1 also have only finite entries. Let M 1 be the minimum of the entries of those c matrices. According to Proposition 4.1, there exists an index l 1 k such that A c ll = c max A = 0. For every n 2N 1, let q and r be the quotient and the remainder of the Euclidean division of n 2N 1 by c. Then, for every i j 1 k,wehave A n ij A 2N 1+m+n c ij A N 1+m il + na c ll + A N 1 lj 2M 1 Let be the maximal average weight of circuits of A not crossing c A. Let pth = i 1 i n be a path of A not crossing c A. It splits into a path with length at most k and elementary circuits, with average weight at most. Denoting by M 2, the greatest entry of A, wehave w A pth k M 2 n k Every N max 2N 1 k M 2 2M 1 / + k thus satisfies the conclusion of the lemma Outline ofthe proof. Notation. We will denote a n-tuple of matrices by i A i 1 n instead of A i i 1 n to use indices for entries of matrices. Since Lemma 4.1 states that scc1-cyc1 matrices have powers of rank 1, Theorem 3.3 readily follows from the next lemma. Lemma 4.3. For every pair A B k M k outside a union of finitely many hyperplanes, there exist two integers m and n, such that the matrix A m B A n has only finite entries and its critical graph has only one node. As a consequence, this matrix is scc1-cyc1. The proof of this lemma is postponed to 5. Theorem 3.4 will be deduced from Lemma 4.3 and the following. Lemma 4.4. If A is a matrix with finite entries whose critical graph has only one node, then there exist a neighborhood V of A and an integer n such that i A i 1 n V n rk 1 A n A = 1

8 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 167 Proof of Theorem 3.4. Every hyperplane H given by Lemma 4.3 is the kernel of a linear form f H on k M k. This linear form can be written f1 H + f2 H, where f 1 H depends only on the first matrix and f2 H on the second one. Since S k is not included in H ker f1 H, there exists a matrix A in S k such that H f1 H A 0. If f1 H = 0 or f2 H B =, then the set B M k f H A B = 0 is empty. Otherwise, it is the affine hyperplane of M k, B M k f2 H H B = f1 A. Therefore, there exists B S such that B H B M k f2 H H B = f1 A. Eventually, A B H ker f H. By Lemma 4.3, there exist m and n such that the matrix A m B A n has only finite entries and its critical graph has only one node. By Lemma 4.4, there exist a neighborhood V of A m B A n and an integer N such that every matrix in V N has rank 1. Let V 1 V 2 be a neighborhood of A B such that V m 1 V 2 V n 1 V. The matrices in V m 1 V 2 V n 1 N have rank 1. Since A and B are in the support of, wehave rk A n + m + 1 N A 1 = 1 A n + m + 1 N A 1 V m 1 V 2 V n 1 N > Almost powers. This section will be devoted to the proof of Lemma 4.4. This proof is based on ideas from the proofs of Propositions 4.1 and 4.2. To understand the powers of A, we considered their entries as the weights of paths on A, as explained in Remark 4.1. We want to do the same for products of several matrices, which means the arcs weights can be different at each step. From now on, will be the complete directed graph whose nodes are the integers between 1 and k. For every finite sequence of matrices i A i 1 n, we set the following notations: (1) The weight of a path pth = i j j 1 n+1 on (regarding i A i 1 n )isw pth = j 1 n j A ij i j+1 (2) A path is maximizing if its weight is maximal among the weights of paths with the same origin, the same end, and the same length. With this definition, i j j 1 n+1 is maximizing if and only if its weight is 1 A n A i1 i n+1. Proof of Lemma 4.4. Since c A has only one node, there exists l 1 k such that c A \ l l A ll >aw A c Thus, there exists >0 such that c A \ l l A ll aw A c > 3 Let V be the open ball of k k with center A and radius for infinity norm and M be the maximum of the infinity norm on V. Let us notice that every matrix B V has the same critical graph as A. Let B be the matrix with max-plus spectral radius 0 defined by B ij = B ij B ll. Then B à < 2, and for any elementary circuit c l l, the average weight of c satisfy aw à c < 3. From now on, i A i 1 n will be in V n and the weights of paths will always be with respect to i à i 1 n. Let pth = i j j 1 n+1 be a path with length n that does not cross l. It can split into a path with length less than k and elementary circuits. Since an elementary circuit c other than l l has weight w c aw à c c +2 c < c, wehave w pth < n k + 2kM However, for every i j 1 k, 1 à n à ij w i l l j > 2M thus there exists N such that every maximizing path with length n N crosses l. Let pth = i j j 1 n+1 be a maximizing path of length n 2N +1. Since i j j 1 N+1 is also maximizing, there exists j 0 N such that i j0 = l. Since i j n N j n+1 is maximizing for j A n N j n, there exists n N j 1 n+1 such that i j1 = l. The path i j j0 j j 1 is a circuit, thus it can split into elementary circuits. Since elementary circuits have a negative weight, except for l l, the only subcircuit of i j j0 j j 1 is l l. Consequently, for every j between j 0 and j 1, and, in particular, between N + 1 and n N,wehavei j = l, and therefore w pth = w i j 1 j N +1 + w i j n N j n+1 This means that n 2N + 1 i j 1 k 1 à n à i j = 1 à N à il + n N à n à lj thus rk 1 à n à = 1 and rk 1 A n A = 1.

9 168 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 5. Proofofthe main lemma Reduced matrices. Definition 5.1. A matrix is called reduced if it has only nonpositive entries and at least one zero in each row. It is called strictly reduced if it has only nonpositive entries and exactly one zero in each row. Lemma 5.1. (i) The set of reduced matrices is a semigroup. So is the set of strictly reduced matrices. (ii) Every reduced matrix A has max-plus spectral radius 0, and c A is made of the circuits of A whose arcs have weight 0. Proof. Point (i) is obvious. Let us prove point (ii). If A is reduced, its coefficients are nonpositive, and so is max A. It is possible to build by induction a sequence i j j 1 k such that for every j, A ij i j+1 = 0. This sequence takes twice the same value: say, in j 1 and j 2. Therefore c = i j j1 j j 2 is a circuit of A with arcs of weight 0. In particular, w A c = 0, thus max A 0. This shows that max A = 0, and, since the entries of A are nonpositive, the last statement is obvious. Definition 5.2. To every matrix A M k such that A is connected, we associate (i) the matrix à defined by i j 1 k à ij = A ij max A, (ii) the smallest elementary circuit of c A for the lexicographical order c A, (iii) the smallest node in c A : A, and (iv) the matrix Ā defined by i j 1 k Ā ij = à ij à + i A + Ã+ j A (3) We define the hyperplanes of Lemma 4.3 by linear forms. Definition 5.3. Let be the complete directed graph whose nodes are the integers between 1 and k. (i) Let E 1 be the set of nonzero linear forms aw c 1 aw c 2, where c 1 and c 2 are two elementary circuits of. (ii) Let E 2 be the set of linear forms w pth 1 pth 1 aw c w pth 2 + pth 2 aw c where pth 1 and pth 2 are two elementary paths of with the same initial node i and same final node i, and where c is an elementary circuit of that goes through. Lemma 5.2. (i) For every matrix A M k with strongly connected graph, Ā is reduced. In particular, the weights of the arcs of c Ā are 0. (ii) If no linear form of E 1 E 2 vanishes at A, then Ā is strictly reduced. (iii) The 0-form is not in E 1 E 2. Proof of Lemma 5.2 (i) à is strongly connected because so is A. Moreover, c à is the same non-weighted graph as c A, therefore A c Ã. Eventually, max à = 0, and, by Proposition 4.1, the column vector à + A is an eigenvector of à with eigenvalue 0. As already observed in Cuninghame-Green [9], this is equivalent to each of the following systems of equations: i 1 k à i A = max à ij + à + j j A j 1 k à i A à ij + à + j A i 1 k j 1 k à i A = à ij + à + j A j 1 k Ā ij 0 i 1 k j 1 k Ā ij = 0 The last system exactly means that Ā is reduced. (ii) Let us assume that no form in E 1 E 2 vanishes at A. For every i 1 k, there exists a path pth 1 = i 1 i pth1 +1 on à from i to A with weight à + i A and with a minimal length among all paths with these properties.

10 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 169 We will show that j = i 2 is the only solution of the equation Ā ij = 0; therefore Ā is strictly reduced. This equation is equivalent to à + i A = à ij + à + j A (4) Let j 1 k be a solution of this equation, and let pth = j 1 j pth +1 be a path from j to A with weight à + j A and with a minimal length among all paths with these properties. Since max à = 0, the circuits of à have a nonpositive weight; therefore the minimality of the lengths implies that pth 1 and pth are elementary. If i = A, then w à pth 1 = w à i pth = à + A A = 0; therefore i pth and pth 1 are circuits on c Ã, and also circuits on c A. i pth can be split into elementary circuits of c A. Let pth 2 be the first one. Then aw A pth 2 = aw A pth 1 = max A, and since no linear form in E 1 vanishes at A, pth 1 = pth 2, and i 2 = j. If i A, then we set pth 2 = i pth ; therefore w à pth 2 = à + i A = w à pth 1 or equivalently, or w A pth 1 pth 1 max A = w A pth 2 pth 2 max A w A pth 1 pth 1 aw c A = w A pth 2 pth 2 aw c A Since no linear form of E 2 vanishes at A, pth 1 = pth 2, and i 2 = j, provided pth 2 is elementary. Let us assume it is not. Then there exists l 1 pth +1 such that j l = i and we have w à i j 1 j l = w à i pth w à j l j pth +1 = à + i A w à i j l+1 j pth A 0 therefore aw à i j 1 j l = max à = 0. Consequently, i j 1 j l is a circuit on c A, and thus it can be split into elementary circuits on c A. Let c 1 be one of these circuits. Since aw A c 1 = max A = aw A c A and no linear form of E 1 vanishes at A, it proves c 1 = c A. Therefore A j 1 j l. Since j l = i A, wehavej pth +1 = A j 1 j l 1, and pth is not elementary. Since we already noticed that pth is elementary, the assumption that pth 2 is not elementary is false, and this concludes the proof of point (ii). (iii) By definition, the zero form is not an element of E 1. Let us prove it is not one of E 2 either. Let pth 1 = i 1 i pth1 +1, pth 2 = j 1 j pth2 +1, and c = l 1 l c l 1 be elementary paths such that i 1 = j 1 = i, i pth1 +1 = j pth2 +1 = l 1 and i 1 l 1. Let us assume that f = w pth 1 pth 1 aw c w pth 2 + pth 2 aw c = 0 and show that pth 1 = pth 2. Since pth 1 and pth 2 are elementary and not circuits, l 1 appears only once in each: as last node. Thus, there is no arc leaving l 1 on pth 1 or on pth 2, and the component in A l 1 l 2 of w pth 1 and w pth 2 is zero. The component of f is then pth 2 pth 1 / c A l 1 l 2. This is zero, thus pth 2 = pth 1 and f = w pth 1 w pth 2. Since pth 1 is elementary, i 1 i 2 is the only arc of pth 1 leaving i. Thus the component of w pth 1 in A ii 2 is A ii 2, and for every j i 2, the component of w pth 1 in A ij is zero. For the same reason, the component of w pth 2 in A ij 2 is A ij 2, and for every j j 2, the component of w pth 2 in A ij is zero. Since w pth 2 = w pth 1, it follows from the last statements that i 2 = j 2 and w i 2 i pth1 +1 = w j 2 j pth1 +1. By a finite induction, pth 1 = pth Matrix with dominating diagonal. Let A k and B M k be two matrices. By Lemma 5.2, we associate to A the reduced matrix Ā defined by Equation (3). We also define ˆB by ˆB ij = B ij à + i A + Ã+ j A (5) Observe that ˆB also depends on A. We will apply the next lemma to Ā and ˆB. We will show that, if some linear forms to be defined later do not vanish at A B, then Ā and ˆB satisfy the hypotheses of this lemma. Lemma 5.3. Let A k be a reduced matrix such that c A is strongly connected. Let N be the integer given by Lemma 4.2.

11 170 Mathematics of Operations Research 35(1), pp , 2010 INFORMS A N B A p B lm i q l m j i i Arcs in the boxes are in c (A) Let B M k be a matrix such that Figure 2. Maximal paths of A N B A p. l 1 l 2 m 1 m 2 1 k A N B l1 m 1 = A N B l2 m 2 m 1 = m 2 (6) Then, there exists s such that for every p s + c A, c A N B A p is a complete directed graph. In particular, A N B A p has type scc1-cyc1. Moreover, if A is strictly reduced, then c A N B A p has exactly one node. Proof. We first study the maximal entries of A N B A p. Such an entry A N B A p ij is the weight of a path i r r 1 N +p+2 from i to j (Figure 2). Let l be i N +1 and m be i N +2. Since A is reduced, we define step by step a sequence of indices ĩ r for r between N + 2 and N + p + 2 such that ĩ N +2 = m and for every r, Aĩr ĩ r+1 = 0. If we replace the i r with r N + 3byĩ r, we replace arcs with nonpositive weight by arcs with weight 0, thus we get a path whose weight is greater or equal to A N B A p ij. Since this weight cannot be strictly greater, and since A is reduced, it means that A ir i r+1 = 0. Eventually, the maximal entry A N B A p ij is equal to A N il + B lm and its value does not depend on p. The choice of N ensures that i r r 1 N+1 crosses c A. Let q be the first node of the path of c A. Starting from q and following backward a circuit on c A, we define a new path such that all nodes before q are in c A. Let i be the starting point of this path (Figure 2). By construction, i c A and A N B i m are greater than or equal to A N B im ; therefore it is a maximal entry of A N B. It follows from Lemma 5.1 (ii) and the strong connectivity of c A that there exists s 1 N such that A s 1 mi = 0. Since c A is the cyclicity of the only strongly connected component of c A, there exists s 2, such that for every p s 2 + c A we have A p i i = 0. Therefore, setting s = s 1 + s 2,wehaveA p mi = 0 for every p s + c A. From now on, we assume p s + c A. Matrix M = A N B A p has a maximal entry on its diagonal. This entry M i i is the max-plus spectral radius of M and the weight of every arc of c M. Moreover, M ij = M i i if and only if there exists m 1 k, such that A N B im = M i i and A p mj = 0. Equation (6) ensures that m is the same for all i. If i and j are on c M, then there exist an arc of c M from i and another one from j such that A N B im = A N B jm = M i i. There are also arcs of c M to i and to j such that A p mi = A p mj = 0. Therefore M ij = M ji = M i i and i j and j i are arcs of c M. Eventually, c M is the complete directed graph whose nodes are the i 1 k such that A N B im = M i i and A p mi = 0. If A is strictly reduced, so is A p. Therefore, there is only one i such that A p mi = 0, and c M has only one node and one arc. We set the following definition. Definition 5.4. Let be the complete directed graph whose nodes are the integers between 1 and k and A (resp. B ) the function, that maps A B k M k to A (resp. B). We denote by E 3 the set of linear forms on k M k of the following type: B j 1 m 1 B j 2 m 2 + w A pth i1 j 1 w A pth i1 + w A pth m1 w A pth i2 j 2 + w A pth i2 w A pth m2 pth i1 j 1 pth i1 + pth m1 pth i2 j 2 + pth i2 pth m2 aw A pth (7) where i 1 i 2 j 1 j 2 m 1 m 2 and are nodes of such that m 1 m 2, and for every i i 1 i 2 m 1 m 2, pth i is an elementary path of from i to, and for every l 1 2, pth il j l is a path of from i l to j l with length at most pth k. Now, we can state our last lemma. Lemma 5.4. (i) IfnoforminE 1 vanishes at A k, then c Ā is strongly connected. (ii) IfnoforminE 3 vanishes at A B k M k, then Ā ˆB satisfy relation (6).

12 Mathematics of Operations Research 35(1), pp , 2010 INFORMS 171 Proof. (i) Since c A and c Ā are equal as nonweighted graphs, we only have to show that c A is strongly connected. But the fact that no form in E 1 vanishes at A means that any two elementary circuits of A have distinct weights. Therefore c A is an elementary circuit, thus it is strongly connected. (ii) Let A B k M k be such that Ā ˆB does not satisfy relation (6). Then, there exist i 1 i 2 m 1 m 2 1 k such that m 1 m 2 and Ā N ˆB i1 m 1 = Ā N ˆB i2 m 2 For each l 1 2, we take j l 1 k such that Ā N ˆB il m l = Ā N il j l + ˆB jl m l, and we denote A by. For each i i 1 i 2 m 1 m 2, we take pth i a path of A from i to such that w à pth i = à + i with minimal length among such paths. For each l 1 2, we take pth il j l a path of A from i l to j l such that w à pth il j l = à N i l j l with minimal length among such paths. Since max à = 0, the circuits on A have nonpositive weight, and thus the minimality of length ensures that for every i i 1 i 2 m 1 m 2 the path pth i is elementary. Proposition 4.2 applied to à c A ensures that pth il j l 2kc A, and c A divides pth, thus pth il j l 2k pth. With those notations, it follows from the definition of Ā and ˆB that the linear form f E 3 defined by formula (7) vanishes at A B. Let us finish the proof of Lemma 4.3. According to Lemma 4.2, E 1 E 2 does not contain the zero form. Obviously, E 3 does not contain it either. Therefore we can take A B k M k such that no linear form in E 1 vanishes at A and no linear form in E 3 vanishes at A B. Lemma 5.4 states that matrices Ā and ˆB defined by Equations (3) and (5) satisfy the hypotheses of Lemma 5.3. Let us take the N and p given by this lemma as n and m. Since A k, we can choose n large enough to have A n k k, and therefore A m B A n k k. Now, we know that c Ā m ˆB Ā n is a complete directed graph. If, moreover, no form of E 2 vanishes at A, then according to Lemma 5.4, c Ā is strongly connected, and according to Lemma 5.3, the complete directed graph has exactly one node and one arc. However, for every circuit c, wehave aw c Ā m ˆB Ā n = aw c A m B A n + m + n max A Therefore Ā m ˆB Ā n and A m B A n have the same critical graph, which concludes the proof of Lemma 4.3. Remark 5.1. In view of the end of the proof, we get the following explicit formulation of Theorems 3.3 and 3.4: Let A B k M k be such that no linear form in E 1 E 2 vanishes at A and no linear form in E 3 vanishes at A B. Then, every sequence A n n of i.i.d. random matrices taking values A and B with positive probability has the MLP. Let be a probability measure on M k with support S. If there exists A k S at which no linear form in E 1 E 2 vanishes, and if S is not included in a union of finitely many sets of type A M k A ij A l = a, with i j l 1 k, j l and a, then every sequence A n n of i.i.d. random matrices with law has the MLP. Those formulas are explicit but not very tractable because the number of conditions to check is more than k! 2. Acknowledgments. This paper follows from the author s doctoral dissertation at Université de Rennes 1. The author thanks Jean Mairesse, Mike Keane, and the anonymous associate editor and reviewers for numerous suggestions for improvements in this presentation. References [1] Baccelli, F Ergodic theory of stochastic Petri networks. Ann. Probab. 20(1) [2] Baccelli, F., D. Hong TCP is max-plus linear and what it tells us on its throughput. Proc. SIGCOMM 2000: Proc. Conf. Appl., Tech., Architectures Protocols Comput. Comm., Stockholm, [3] Baccelli, F., G. Cohen, G. J. Olsder, J.-P. Quadrat Synchronization and linearity. An algebra for discrete event systems. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley and Sons Ltd., Chichester, UK. [4] Blondel, V. D., S. Gaubert, J. N. Tsitsiklis Approximating the spectral radius of sets of matrices in the max-algebra is NP-hard. IEEE Trans. Automatic Control 45(9)

13 172 Mathematics of Operations Research 35(1), pp , 2010 INFORMS [5] Braker, H Algorithms and applications in timed discrete event systems. Unpublished doctoral dissertation, Delft University of Technology, The Netherlands. [6] Cohen, J. E Subadditivity, generalized products of random matrices and operations research. SIAM Rev. 30(1) [7] Cohen, G., D. Dubois, J. P. Quadrat, M. Viot Analyse du comportement périodique des systèmes de production par la théorie des dioïdes. Rapport de recherche 191, Institut National de Recherche en Informatique et Automatique, Le Chesnay, France. [8] Cohen, G., D. Dubois, J. P. Quadrat, M. Viot A linear system theoretic view of discrete event processes and its use for performance evaluation in manufacturing. IEEE Trans. Automatic Control AC 30(3) [9] Cuninghame-Green, R Minimax algebra. Lecture Notes in Economics and Mathematical Systems, Vol Springer-Verlag, Berlin. [10] Gaubert, S., D. Hong Series expansions of Lyapunov exponents and forgetful monoids. Technical report, Institut National de Recherche en Informatique et Automatique, Paris. [11] Gaubert, S., J. Mairesse Task resource models and max + automata. Idempotency (Bristol, 1994), Vol. 11. Publ. Newton Inst. Cambridge University Press, Cambridge, UK, [12] Gaubert, S., J. Mairesse Modeling and analysis of timed Petri nets using heaps of pieces. IEEE Trans. Automatic Control 44(4) [13] Heidergott, B A characterisation of max + -linear queueing systems. Queueing Systems Theory Appl. 35(1 4) [14] Heidergott, B., R. de Vries Towards a (Max,+) control theory for public transportation networks. Discrete Event Dynamic Systems 11(4) [15] Heidergott, B., G. J. Oldser, J. van der Woude Max plus at work. Modeling and analysis of synchronized systems: A course on max-plus algebra and its applications. Princeton Series in Applied Mathematics. Princeton University Press, Princeton, NJ. [16] Mairesse, J Products of irreducible random matrices in the max + algebra. Adv. Appl. Probab. 29(2) [17] Merlet, G Limit theorems for iterated random topical operators. Technical report, Institut de Recherche Mathematique de Rennes, [18] Merlet, G A central limit theorem for stochastic recursive sequences of topical operators. Ann. Appl. Probab. 17(4) [19] Vincent, J.-M Some ergodic results on stochastic iterative discrete events systems. Discrete Event Dynamic Systems 7(2)