Weak lumpability of finite Markov chains and positive invariance of cones

Transcription

1 INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE Weak umpabiity of finite Markov chains and positive invariance of cones James Ledoux N 2801 Fevrier 1996 PROGRAMME 1 apport de recherche ISSN

2

3 Weak umpabiity of nite Markov chains and positive invariance of cones James Ledoux Programme 1 Architectures paraees, bases de donnees, reseaux et systemes distribues Projet Mode Rapport de recherche n2801 Fevrier pages Abstract: We consider weak umpabiity of genera nite homogeneous Markov chains evoving in discrete time, that is when a umped Markov chain with respect to a partition of the initia state space is aso a homogeneous Markov chain. We show that weak umpabiity is equivaent to the existence of a decomposabe poyhedra cone which is positivey invariant by the transition probabiity matrix of the origina chain. It aows us, in a unied way, to derive new resuts on umpabiity of reducibe Markov chains and to obtain spectra properties associated with umpabiity. Key-words: Markov chain, States aggregation, Weak umpabiity, Poyhedra cone, Positive invariance. (Resume : tsvp) INSA, Campus de Beauieu Rennes Cedex, E-mai edoux@funiv-rennes1.frg,firisa.frg Unité de recherche INRIA Rennes IRISA, Campus universitaire de Beauieu, RENNES Cedex (France) Tééphone : (33) Téécopie : (33)

4 Agregation faibe de cha^nes de Markov nies et invariance positive de c^ones Resume : Nous nous interessons a a propriete d'agregation faibe de cha^nes de Markov nies evouant en temps discret, c'est a dire quand une cha^ne de Markov, agregee seon une partition de 'espace d'etat initia, est encore markovienne homogene. Nous montrons que cette propriete est equivaente a 'existence d'un c^one poyhedrique decomposabe qui est invariant par a matrice des probabiites de transition de a cha^ne originae. Cea nous permet, d'une maniere unifee, de deriver de nouveaux resutats sur 'agregation de cha^nes de Markov reductibes et d'obtenir des proprietes spectraes associees a 'agregation. Mots-ce : Cha^ne de Markov, Agregation d'etats, C^one poyhedrique, Invariance positive

5 Weak umpabiity of nite Markov chains and positive invariance of cones 3 1 Introduction Let us consider a homogeneous Markov chain X, in discrete or continuous time, on a nite state space, denoted by S, which is assumed to be S = f1; : : :; Ng. Let P = fc(1); : : :; C(M)g be a xed partition of S in M < N casses. We associate with the given chain X the aggregated chain Y, over the state space ^S = f1; : : :; Mg, dened by: Y t = X t 2 C(); for any t. We are interested in the initia distributions of X which give an aggregated homogeneous Markov chain Y. If such a distribution exists, we say that the famiy of Markov chains sharing the same transition probabiity matrix (t.p.m.) is weaky umpabe. The former work on umpabiity was concerned with the strong umpabiity situation, that is, when any initia distribution eads to an aggregated homogeneous Markov chain. It has been characterized for irreducibe Markov chains in Kemeny and Sne (1976) and for a cass of reducibe chains in Abde-Moneim and Leysieer (1984). The weak umpabiity probem has been addressed in Kemeny and Sne (1976), Abde-Moneim and Leysieer (1982), Rubino and Sericoa (1991) for irreducibe Markov chains. Under an additiona condition, the (homogeneous) Markovian property for the aggregated chain Y has recenty been proved in Peng (1995) to be equivaent to require Y to satisfying the Chapman-Komogorov equations and then the set of a initia distributions eading to an aggregated Markov chain is expicity given. As shown in Ledoux et a. (1994), weak umpabiity of absorbing Markov chains with an ony one irreducibe transient state cass, is cosey reated with the previous works using a resetting from absorbing states on the transient ones according to the quasi-stationary distribution. Aggregation of any nite continuous time Markov chain can be repaced in the discrete time context (see Ledoux (1995)) and, therefore, wi not be discussed in the seque. If the state space S is countaby innite we refer to Ba and Yeo (1993) and Ledoux (1995). A arge amount of work on umpabiity Markov chains is concerned in proving the unicity of the transition probabiity matrix of the Markovian aggregated chains. The inear system approach, initiaized by Abde- Moneim and Leysieer (1982) and fuy deveoped in Rubino and Sericoa (1991), to compute the set of a initia distributions eading to an aggregated Markov chain is centered on this property. However, this unicity property may not hod for a genera nite Markov chain. The purpose of this paper is to pace emphasis on geometrica properties associated with the weak umpabiity condition when we are interested in aggregated Markov chains sharing the same t.p.m. In particuar, it can be used to give an unied view of the previous works and to derive new resuts for genera nite Markov chains. After reviewing some preiminaries on poyhedra cones, we anayze in Section 2, for a genera nite Markov chain with transition probabiity matrix P, the set of a initia distributions which give aggregated Markov chains sharing the same t.p.m. Pointing out the reation between umpabiity and positive invariance of cones in Section 3, we show that this set is non empty if there exists a famiy of M poyhedra cones which are \invariant" under sub-matrices of matrix P. This resut aows us to state in Section 4 that if the partition P is a renement of the partition of the state space S induced by the usua \communication" equivaence reation, then we obtain an expicit formua for the transition probabiity matrix of any Y, which depends ony on P and P. Previous works can be viewed as direct appications of this resut. Throughout Section 3 and Section 4, various properties reported in Ledoux (1993), Abde-Moneim and Leysieer (1984) and Peng (1995) are extended to genera nite Markov chains and new spectra resuts are aso derived. Notation The set of a probabiity distributions on S wi be denoted by A. The support D of a probabiity distribution is dened as the greatest subset of S apart from the distribution is zero, i.e. (i) = 0 for a i 2 E n D and (i) 6= 0 for i 2 D. By convention, vectors are row vectors. Coumn vectors are indicated by means of the transpose operator (:) T. The vector with a its components equa to 1 (resp. 0) is denoted merey by 1 (resp. 0). The vector RR n2801

6 4 James Ledoux e i denotes the ith vector of the canonica basis of IR N. We denote by I the identity matrix and by diag(v) (by diag(h i )) the (bock) diagona matrix with generic diagona (bock) entry v(i) (the matrix H i ), the dimensions being dened by the context. The cardinaity of the cass C() is denoted by n(). We assume the states of S ordered such that C() = fn(1) + + n(? 1) + 1; : : :; n(1) + + n()g for 1 M (with n(0) = 0.) For any subset C of S (whose cardinaity is n) and 2 A, the restriction of to C, i.e. the vector ((i); i 2 C), is denoted by C or R C. On the other hand, a vector on [0; 1] n() can be viewed as the vector on [0; 1] N dened by: [R?1 ](i) = 0 if i =2 C() and [R?1 ](i) = (i?n(1)??n(?1)) if i 2 C(). If C is a subset of A (resp. of [0; 1] n() ) then R C (resp. R?1 C) denotes the set f C() = 2 Cg [0; 1] n() (resp. fr?1 = 2 Cg IR N.) P If C S and C 1 T 6= 0, C is the vector of A dened by C (i) = (i)= j2c i =2 C. 2 Preiminaries on cones and weak umpabiity 2.1 Cone, poyhedra cone of IR n (j) if i 2 C and by 0 if The basic denitions on the cones are reviewed from Berman and Pemmons (1979). Throughout this subsection C denotes a subset of IR n. For any C, Span(C) (resp. A(C)) refers to the inear (resp. ane) hu of C. The set Cone(C) denotes the conica hu of C that is the set of a nite nonnegative inear combinations of the eements of C. The eements of C are caed the generators of Cone(C). If Cone(C) = C then C is caed a cone. Conv(C) is the set of a nite convex inear combinations of the eements of C. The dimension of a subset C is dened by dim(c) = dim A(C). The interior of C reative to the ane space A is denoted by int A (C). Denition 2.1 A poyhedra cone C of IR n is the soution set of a system of inear homogeneous inequaities, i.e. C = f x 2 IR n = xh 0 g where H 2 IR nm. Such a cone is a cosed convex subset of IR n. We reca that a bounded soution set of a system of inear inequaities is caed a poytope of IR n. A convex cone C is pointed if C \ (?C) = f0g and soid if int IR n(c) 6= ;. Note that a convex subset C is such that int A(C)(C) 6= ;. Finay, a cosed, pointed, soid convex cone is caed a proper cone. An extrema of a pointed poyhedra cone C is an eement which can never be written as a nonnegative inear combinations of others eements of C. A pointed poyhedra cone is nitey generated by their extremas. Denition 2.2 Let C be a cone of IR n, C 1 and C 2 be two sub-cones of C. The cone C is the direct sum of C 1 and C 2, that is denoted by C = C 1 L C2, if Span(C 1 ) \ Span(C 2 ) = f0g and C = C 1 + C 2. If such cones C 1 and C 2 exist and are distinct from f0g, then the cone C is said to be decomposabe. 2.2 Weak umpabiity of a nite Markov chain Let X = (X n ) n0 be a homogeneous Markov chain over state space S, given by its transition probabiity matrix P = (P (i; j)) i;j2e and its initia distribution ; when necessary we denote it by (; P ). Let P (i; C) denote the transition probabiity of moving in one step from state i to the subset C of S, that is P (i; C) = P j2c P (i; j). Let P C()C(m) be the n() n(m) sub-matrix of P given by (P (i; j)) i2c();j2c(m). We denote the aggregated chain constructed from (; P ) with respect to the partition P by agg(; P; P). Denition 2.3 A sequence (C 0 ; C 1 ; : : :; C j ) of subsets of S is caed possibe for the initia distribution if IP (X 0 2 C 0 ; X 1 2 C 1 ; : : :; X j 2 C j ) > 0. Given any distribution 2 A and a possibe sequence INRIA

7 Weak umpabiity of nite Markov chains and positive invariance of cones 5 (C 0 ; C 1 ; : : :; C j ) for, we can dene the vector f(; C 0 ; C 1 ; : : :; C j ) 2 A recursivey by: f(; C 0 ) = C 0 f(; C 0 ; C 1 ; : : :; C k ) = (f(; C 0 ; C 1 ; : : :; C k?1 )P ) Ck k 1: For any C 2 C, A(; C) denotes the subset of a distributions of the form f(; C 0 ; : : :; C k ; C). The approach deveoped in Kemeny and Sne (1976) and in Rubino and Sericoa (1989) consists in rewriting the conditiona expression IP (X n+1 2 C(m) j X n 2 C(); X n?1 2 C n?1 ; : : :; X 0 2 C 0 ) (dened for any (C 0 ; C 1 ; : : : ; C n?1 ; C()) possibe for ) as IP (X 1 2 C(m)) with = f(; C 0 ; : : :; C n?1 ; C()), that is, in incuding the past into the initia distribution. A necessary and sucient condition for Y to be a homogeneous Markov chain can be exhibited without any particuar assumption on X. Resut 2.4 The chain Y = agg(; P; C) is a homogeneous Markov chain if and ony if 8; m 2 ^S, the probabiity IP (X 1 2 C(m)) is the same for every 2 A(; C()). This common vaue is the transition probabiity for the chain Y to move from state to state m. Remark 1 For a chain (; P ), the set A(; C) may be empty. That impies that we can never access to the states of the cass C with as initia distribution. Consequenty, C can be eiminated of P with any repercussions on the anaysis of the aggregated process. 4 Resut 2.4 determines the transition probabiity matrix of the aggregated process which may depend on the initia distribution as shown by the foowing exampe. Exampe 1 Let us consider the two irreducibe Markov chains with respective t.p.m. P 1 and P 2 : P 1 = 1=4 1=4 1=2 0 1=6 5=6 7=8 1=8 0 1 A P2 = 0 1=6 1=6 1=3 1=3 1=3 1=3 1=6 1=6 1=4 1=4 1=4 1=4 1=4 1=4 1=4 1=4 It has been shown in Kemeny and Sne (1976) that, for P 1, the set of a initia distribution eading to an aggregated Markov chain according to the partition P 1 = ff1g; f2; 3gg is A 1 = fe 1 +(1?)(0; 1=3; 2=3) = 0 1g. Foowing the same way as in Rubino and Sericoa (1991), the corresponding set for P 2 and P 2 = ff1; 2g; f3; 4gg is A 2 = f(1=2; 1=2; 0; 0) + e 3 + (1?? )e 4 = 0 ; 1g. Let P be the matrix diag(p i ) and P = ff1; 6; 7g; f2; 3; 4; 5gg. It is easy to convince oursef that the chain agg(e 1 ; P; P) is a homogeneous Markov chain. Its transition probabiity matrix is bp (e 1) 1=4 3=4 = : 7=12 5=12 In the same way, the chain agg(e 7 ; P; P) is a homogeneous Markov chain with t.p.m. bp (e 7) 1=2 1=2 = : 1=2 1=2 We note that b Pe1 (1; 1) = 1=4 is an eigenvaue of matrix P C(1)C(1) which is distinct from the spectra radius 1=2 of the nonnegative matrix. / Let us consider a probabiity distribution and eiminate a the casses of the partition P which can never be accessed by the chain (; P ), that is the casses C() ( 2 ^S) such that A(; C()) = ;. We obtain 1 C A : RR n2801

8 6 James Ledoux a new state space and a new partition of the subset of S which are again denoted by S and P for not making the notation heavier. With that new partition P, a the sets A(; C()) are not empty and we can dene matrix b P in the foowing manner: for each 2 ^S, et be a vector in A(; C()) and set bp (; m) def = IP (X 1 2 C(m)) 8m 2 ^S: If agg(; P; P) is a homogeneous Markov process then it foows from Resut 2.4 that b P is the transition probabiity matrix of this aggregated chain. Let us now dene the set, denoted by AM( b P ), of a initia distributions eading to an aggregated homogeneous Markov process agg(; P; P) with transition probabiity matrix b P : AM( b P ) = f 2 A = agg(; P; C) is a homogeneous Markov chain with t.p.m. b P g To ighten the presentation, AM wi aso refer to AM( b P ) if there is no ambiguity. The aim of this subsection is to anayze properties of this set when it is not empty. Let us dene the foowing matrices. For any 2 ^S, P denotes the n() N sub-matrix of P : (P (i; j)) i2c();j2s. e P denotes the N M matrix dened by: 8i 2 S, 8m 2 ^S, e P (i; m) = P (i; C(m)). For any 2 ^S, we denote by e P the n() M sub-matrix of e P : ( e P (i; m))i2c();m2 ^S. The th row of the stochastic matrix b P is denoted by b P. For a 2 ^S, we set H = e P?1 T b P (n()m) and for any j 1, we dene the foowing N M j+1 bock diagona matrices H [1] = diag(h ); H [j+1] = diag(p H [j] ): (1) We are in position to adopt the inear system approach from Rubino and Sericoa (1991) and in the same manner, we have: AM( b P ) = \ j1 A j where A j = f 2 A = H [k] = 0; for k 1 g: Now, each poytope A j can be seen as the trace on the set A of the foowing poyhedra cone: C j def = f 0 = H [k] = 0; for 1 k jg; (2) that is A j = C j \ A for j 0 (with the convention T C 0 = IR N + and A 0 = A). Consequenty, we note that = j1 Cj and we have AM( b P ) = CM( b P ) \ A where CM( b P ) def AM( b P ) 6= ; CM( b P ) 6= f0g: Now, if we note that C j+1 is deduced from C j by attaching the (eventuay) additiona constraints (H [j+1] = 0) and that dim(c 1 ) N then the foowing extension of Theorem 3.4 from Rubino and Sericoa (1991) is intuitivey cear: CM( b P ) = C N (3) where N is the number of states of the origina chain. We note from the diagona structure of the matrices H [j], that, for 0, 2 C j 8 2 ^S; R?1 C() 2 C j. It aows us to derive part of the foowing emma. INRIA

9 Weak umpabiity of nite Markov chains and positive invariance of cones 7 L Lemma 2.5 Let us set C j = R C j for every 2 ^S. We have, for a j 1, C j = R?1 2 ^S C j where R?1 C j C j is a poyhedra cone of IR N (C j is a poyhedra cone of IR n().) proof. We can check from the denition of the sets C j (see (1), (2)) that for j 1, o C j = n 2 IR n() + = H = 0 and P H [k] = 0; 1 k j? 1 : (4) Consequenty, C j (resp. R?1 C j ) is a poyhedra cone of IRn() (resp. IR N ). The we-known necessary and sucient condition reported in Kemeny and Sne (1976) for having strong umpabiity of (:; P ) with an irreducibe matrix P can be extended to a genera stochastic matrix. The ony requirement is that a the aggregated chains share the same t.p.m. P. b In that case, by denition, the famiy (:; P ) of Markov chains is strongy umpabe if AM( P b ) = A or CM( P b ) = IR N + for any 2 A. In fact, it is equivaent to require that A 1 = A 0 or C 1 = C 0. Now, C 1 = IR N + is equivaent to H [1] = 0 or to (H = 0; 8 2 ^S) which are precisey the conditions given by the foowing theorem. Theorem 2.6 If we require that a the aggregated chains share the same transition probabiity matrix, then the famiy (:; P ) of Markov chains is strongy umpabe if and ony if for each pair of casses C() and C(m), P (i; C(m)) does not depend on i 2 C(). In particuar, this resut is necessary to derive some resuts in Abde-Moneim and Leysieer (1984) though the characterization expicity used is the Kemeny and Sne's one with the irreducibiity assumption. 3 Lumpabiity and positive invariance Denition 3.1 A matrix A eaves a cone C of IR N invariant or matrix A is nonnegative on the cone C, that wi be denoted by A C 0, if for every x 2 C the vector xa 2 C (i.e. CA C). The cone C is said to be positivey invariant by matrix A. Some spectra properties of matrices eaving a proper cone invariant are reviewed from Berman and Pemmons (1979). Resut 3.2 If matrix A eaves a proper cone C invariant then the spectra radius (A) is an eigenvaue of A and C contains a eft eigenvector of A corresponding to (A). Note that a nonnegative matrix is a matrix which eaves the proper cone IR N + of IR N invariant. We wi dea with cones which are not soid. Consequenty, we have to derive a weaker resut than the previous one. Lemma 3.3 If matrix A eaves a cosed, pointed convex cone C invariant then there exists an nonnegative eigenvaue of A such that C contains a eft eigenvector of A associated with. If a nonnegative matrix A is irreducibe and eaves a cosed, convex cone C IR N + invariant then C contains the positive eft eigenvector corresponding to the spectra radius (A). proof. Matrix A represents the matrix of a inear operator f on IR N with respect to the canonica basis (with the convention that f(e i ), for every i 2 N, is the ith row of matrix A, that is f(x) = xa for a x 2 IR N.) Matrix A is nonnegative on C means that f(c) C. Consequenty, f eaves the inear subspace L = Span(C) IR N invariant and it impies that the restriction of f to the subspace L, denoted by fjl, is a inear operator from L to L. The cone C is aso invariant by fjl and is soid with respect to L. Thus, the proper cone C is positivey invariant by the matrix AjL of the operator fjl. The rst part of Resut 3.2 can be appied to AjL and concusions are associated with the spectra radius of that matrix. However, the eigenvectors and the spectra radius of fjl are eigenvectors and a nonnegative eigenvaue of the initia inear operator f on domain IR N, that gives the rst part of the emma. RR n2801

10 8 James Ledoux If the nonnegative matrix A is irreducibe, then there exists an unique (up to a constant mutipe) eft eigenvector of A in IR N (in fact in + intirn + i.e. it is a positive eft eigenvector) which corresponds to the spectra radius of A. Now, for any cosed, (pointed) convex cone C IR N +, if A C 0 then we deduce from the rst part of the proof that there exists a nonnegative eft eigenvector of matrix A in C IR N +. Since there is ony one eft eigenvector of A in IR N +, it is positive and associated with the spectra radius of the matrix A. The second part of the emma hods. We want emphasize that the positive invariance of poytope, used in Lemma 3.5 from Rubino and Sericoa (1991) as a simpe stop test in their incrementa computation of AM from the A j ones, is a centra geometric invariant of the weak umpabiity property as soon as we are interested in aggregated Markov chains sharing the same transition probabiity matrix. Theorem 3.4 The set AM( b P ) 6= ; or CM( b P ) 6= f0g if and ony if there exists a poyhedra cone C C 1, dierent from f0g, such that P C L def 0 and C is the direct sum R?1 2 ^S C where C = R C for a 2 ^S. proof. Suppose that CM( P b ) 6= f0g. Let us verify that CM fus the required P conditions. We have CM = C N from reation (3). Since C N = C N+1, we have for any vector = R?1 2 ^S C() 2 C N and for any j such that 1 j N H [j+1] = ^S; C() P H [j] = 0 (by denition of system H [j+1] ) =) P H [j] = X 2 ^S C() P H [j] = 0; that is P 2 C N. The set CM is the direct sum of its M \projections" from reation (3) and Lemma 2.5. To obtain the decomposabiity of CM, it remains to estabish that these sets are distinct from f0g. If there exists 2 ^S such that C = R CM = f0g, that impies that the origina chain (; P ) can never accessed to the state cass C(). Indeed, if A(; C()) 6= ; then there exists n 0 such that R [P n ] 6= 0 and R [P n ] 2 C with the positive invariance of CM by matrix P. To concude, reca that the considered partition P contains ony the state casses of the origina state space S which are accessed by (; P ). Conversey, if there exists a poyhedra cone C C 1, which is distinct from f0g and is positivey invariant by P, such that C = L 2 ^S R?1 C then we show by induction that C C j 8j 1: The rst step is obvious. Let us assume that C C j with j > 1. For every 2 C, we have R?1 C() 2 C C j for a 2 ^S (since C is decomposabe), next [R?1 C() ]P = C() P 2 C C j for a 2 ^S (because P C 0). We concude that [8 2 ^S; C() P H [j] = 0] or H [j+1] = 0. Thus, we have C C j+1. Finay, we obtain C T j1 Cj = CM( b P ). The above proof gives a sucient condition for nding initia distributions which ead to an aggregated homogeneous Markov chain (with xed t.p.m.) Coroary 3.5 Let b P be any stochastic matrix. We dene the poyhedra cone C1 associated with b P as in (2) and suppose that it is distinct from f0g. If there exists a poyhedra cone C C 1 which is decomposabe in L 2 ^S C and such that P C 0 then C CM( b P). Using decomposabiity property, Theorem 3.4 can be reformuated with \oca" characteristics. That gives the main resut of this section. INRIA

11 Weak umpabiity of nite Markov chains and positive invariance of cones 9 Theorem 3.6 The set AM( P b ) 6= ; or CM( P b ) 6= f0g if and ony if there exists a famiy of M poyhedra cones (C ) 2 ^S, distinct from f0g, such that ( C C 1 IR n() ^S; C P C()C(m) C m 8; m 2 ^S: Remark 2 The poyhedra cone CM( b P ), when it is distinct from f0g, satises the conditions of Theorem 3.6. From Coroary 3.5, it foows that CM( b P ) is the argest poyhedra sub-cone of C 1 which is positivey invariant by P and decomposabe in M poyhedra cones. However, it may exist a smaer poyhedra sub-cone of C 1 than CM( b P ) which is ony positivey invariant. Indeed, et us return to the Exampe 1. We consider the transition probabiity matrix P 1 and the partition P = fc(1) = f1g; C(2) = f2; 3gg of f1; 2; 3g. Reca that AM( P b(e 1) ) = A 1 = CM \ A with CM = C 1 = f(1; 0; 0) + (0; 1; 2) = ; 0g. Let us dene C = CMP def = fp = 2 CMg. The two extremas of the poyhedra cone C are r 1 = (1; 1; 2) and r 2 = (21; 5; 10) that is C = fr 1 + r 2 = ; 0g. We can check that C CM = C 1 and that C is positivey invariant by P (since CP CMP = C). Theorems 3.4 and 3.6 can be associated with the Lemma 3.3 to give the foowing coroary. 4 Coroary 3.7 If CM( b P ) 6= f0g then it contains a nonnegative eft eigenvector corresponding to a nonnegative eigenvaue of P. For each 2 ^S, the cone R CM( P b ) of IR n() + contains a nonnegative eft eigenvector corresponding to the nonnegative eigenvaue P b (; ) of P C()C(). proof. The rst assertion is a direct consequence from positive invariance of CM and from Lemma 3.3. Since [R CM]P C()C() [R CM] (Theorem 3.6), there exists a nonzero eft eigenvector v in R CM associated with an eigenvaue of P C()C() with Lemma 3.3; and if so, we have with vector f(r?1 v ; C()) as initia distribution for the origina chain P b (; ) = IP f(r?1 v (X ;C()) 1 2 C()) = v P C()C() 1 T =v 1 T =. Remark 3 The fact that b P (; ) is an eigenvaue of P C()C() competey generaizes the resut given in Ledoux (1993) for an irreducibe origina chain. It was based on the fact that Markovian property induces geometric sojourn times in each casse C() and on the Jordan's canonica form of a matrix. We reca that bp (; ) may not be (P C()C() ) (see Exampe 1.) 4 From Coroary 3.7, a cone which may fu the sucient condition for weak umpabiity given in Coroary 3.5 is the one which can be formed from a famiy fv ; 2 ^Sg of nonnegative eft eigenvectors (and nonzero vectors) associated with the famiy of sub-matrices fp C()C() ; 2 ^S g. Let us set M def C v = Cone(fR?1 v ; 2 ^Sg) = Cone(R?1 v ): (5) Since v 6= 0 for every 2 ^S, the foowing stochastic matrix, denoted by b P, can be dened by Thus, we deduce from Coroary 3.5 that 2 ^S bp = f(r?1 v ; C())e P 8 2 ^S: P Cv 0 =) C v CM( b P ): RR n2801

12 10 James Ledoux Such a situation raises with the exact umpabiity property described in Schweitzer (1984). Indeed, it corresponds to assume that for a 2 ^S, P (i; j) depends ony on and m for every j 2 C(m). Pi2C() PConsequenty, for every 2 ^S, the vector v = 1 C() is a eft eigenvector corresponding to the eigenvaue P (i; j) of nonzero matrix P i2c() C()C() such that X 1 C() P C()C(m) = [ P (i; j)] 1 C(m) 8m 2 ^S; i2c() and we have C v CM( b P ) according to the previous discussion. The fact that exact umpabiity impies weak umpabiity is we known. The foowing coroary takes advantage of the identication of the sub-cone C v of CM( b P ) = C N dened in (5) and of the ane independence of the M vectors R?1 Coroary 3.8 We have CM( b P ) 6= f0g if and ony if C N?M CM( b P ) = M 2 ^S R?1 C N?M : v (i.e. dim C v = M). 6= f0g for a 2 ^S. In that case, we have When C() is is an irreducibe state cass of S then Coroary 3.7 and Lemma 3.3 give the foowing additiona assertions. The na part aso uses the positive invariance properties of cones R CM( b P ) ( 2 ^S) given in Theorem 3.6. Coroary 3.9 Let us assume that CM( b P ) 6= f0g. If P C()C() is an irreducibe matrix then R CM( b P ) contains ony one eft eigenvector v of P C()C() and this vector is positive. Moreover, P b (; ) is the spectra radius of P C()C(). Thus, we necessariy have bp = f(r?1 v ; C()) P e with any initia distribution in CM( b P ) whose support D is such that D \ C() 6= ;; moreover, for any state cass C(m) which can be accessed from a state of C() (i.e. there exists a possibe sequence (C(); C(i 1 ); : : :; C(i k ); C(m)) for some e i with i 2 C()), we have v P C()C(i1) P C(ik)C(m) 6= 0 is in R m CM( b P ) and bp m = f(r?1 m [v P C()C(i1) P C(ik)C(m) ]; C(m))e P: Remark 4 We have shown that b P (; ) is necessariy the spectra radius of P C()C() when C() is irreducibe. That improves the resuts given in Section 4 from Ledoux (1993). 4 Lumpabiity of reducibe Markov chains The previous resuts can be appied to the aggregation of Markov chains with respect to a partition P which is a renement of the partition of S corresponding to the usua communication equivaence reation. This partition is denoted by I = (I k ) k2j throughout this section. The eements of I are caed the communication casses or the irreducibiity casses and jjj denotes the cardinaity of I. Such a state cass I k induces an irreducibe sub-matrix P IkIk of P. Consequenty, we can associated with each state cass I k, the unique 4 INRIA

13 Weak umpabiity of nite Markov chains and positive invariance of cones 11 stochastic eft eigenvector v k of P IkI k corresponding to the spectra radius of P IkI k. Throughout this section, we assume that the states of S are ordered such that P is a ower bock-trianguar matrix P = 0 P I1I1 0 0 P I2I P IjJj I jjj Partition P is a renement of the partition I if 8 2 ^S, 9!k 2 J such that C() I k. For each k 2 J, there exists L k ^S such that Ik = ] 2Lk C(). Any nonnegative P vector on I k can be seen as an eement of 2Lk IR n() +. Consequenty, we denote the vector on S, 2L k R?1 C(), by R?1 I k. Denition 4.1 A famiy of communication casses (I i0 ; ; I in ) is caed a path if each cass I ik?1 has an access to the cass I ik for k = 1; : : :; n (that is there exists a state in I ik?1 which communicates with a state of I ik.) We ca I i0 the starting point and I in the end point of the path. Theorem 4.2 Let us assume that partition P is a renement of the partition I = (I k ) k2j of S. We have the famiy of vectors (v k ) k2j, v k being the stochastic eft eigenvector associated with the spectra radius of matrix P IkI k. If 2 A is such that Ik 6= 0 and agg(; P; P) is a homogeneous Markov chain, then, for any m such that I m beongs to a path with starting point I k, we have agg(r?1 I m v m ; P; P) is a homogeneous Markov chain and for a 2 ^S such that C() I m : 1 C A : bp = f(r?1 I m v m ; C()) e P ; (6) moreover the famiy F m composed of vectors R?1 (v m ) C() is such that Cone(F m ) CM( b P ). Remark 5 The previous theorem can be interpreted as foows: if a state of a cass I k is aowed to be an initia state of our Markovian mode then a the rows of matrix b P corresponding to the state casses of the P incuded in I k or in the eement of a path with starting point I k, are necessariy given by formua (6) and depend ony on I and P. 4 proof. We have I k = ] 2Lk C() for some L k ^S. We deduce from Theorem 3.6 that if agg(; L P; P) is a homogeneous Markov chain then there exists a pointed poyhedra cone, dened by C Ik = 2L k R?1 C, such that cone R Ik C Ik is positivey invariant by the irreducibe matrix P IkIk. Lemma 3.3 states that this ast cone contains the stochastic eft eigenvector v k corresponding to the spectra radius of P IkIk. Since a the distributions of cone C Ik ead to an aggregated Markov chain with the same t.p.m. P b, we derive that agg(r?1 I k v k ; P; P) is a homogeneous Markov chain and that P b = f(r?1 I k v k ; C()) P e for every 2 ^S such that C() I k. Let us now consider a path with starting point I k and assume that there exists a distribution such that Ik 6= 0 and agg(; P; P) is Markov. The chain agg(r?1 I k v k ; P; P) is aso a homogeneous Markov chain from the rst part of the proof. Since CM( P b ) is positivey invariant by P, we have that for any n 0, R?1 I k v k P n 2 CM. The cass I k communicate with any eement I i of the path. Consequenty, et i be xed, there exists n i > 0 such that for w i = R?1 I k v k P ni, R Ii w i 6= 0 and agg(w i ; P; P) is a homogeneous Markov chain. The rows of matrix P b corresponding to the casses of P incuded in Ii are necessariy given by bp v i ; C()) P; e from the rst part of the proof. = f(r?1 I i The ast part of the theorem foows from the fact that each R v m is in R CM( b P ) (since R?1 I m v m 2 CM( b P )) and from the conica property of CM( b P ). RR n2801

14 12 James Ledoux Remark 6 If we wish that a initia distributions on A ead to an aggregated homogeneous Markov chain (strong umpabiity property) then, for a k 2 ^S, there must exist such a distribution whose support contains states from cass I k. Thus, Theorem 4.2 aows us to concude that a the aggregated chains share the same transition probabiity. Consequenty, the unicity condition on this matrix required in Theorem 2.6 can be dropped. Agorithm We propose to briey specify an agorithm which can be used in the present context to compute probabiity distributions eading to an aggregated homogeneous Markov chain. However, an anaogous agorithm aso works for a genera nite Markov chain from Section 2. The agorithm beow foows the same ines as the Rubino and Sericoa's one for an irreducibe stochastic matrix P. We just want to point out here that a computation may be performed \ocay", that is in working with characteristics corresponding to the state casses of P. Note that the set CM( P b ) resuting from the agorithm contains a the initia distributions eading to an aggregated homogeneous Markov chain with L respect to P. In the process of computation, a set C j may be found to be f0g. In such a case, we have C Ik = m2l k R?1 m Cm j = f0g if C() I k = ] m2lk C(m). Consequenty, the states of I k are ignored in the seque. Preiminary step: Compute the stochastic eft eigenvectors fv k ; k 2 Jg of matrices fp IkI k ; k 2 Jg. Compute matrix b P with for any 2 ^S, b P = f(r?1 I m v m ; C()) e P if C() Im. Loop j = 1; : : :; N? M First step: Form the poyhedra cones C j ( 2 ^S) from the recursive denition (4). Compute the extremas for each C j, that is the minima nite famiy of nonzero vectors r i such that C j = Cone(r 1 ; : : :; r n ). Second step: Endoop If for each 2 ^S such that C j 6= f0g, (i = 1; : : :; n: r i P C()C() 2 Cm j ) for a m 2 ^S then CM( P b ) = R?1 L2 ^S C j exit Exampe 2 Let us consider the foowing partition P = fc(1) = f1g; C(2) = f2; 3g; C(3) = f4g; C(4) = f5; 6; 7gg of the state space S = f1; 2; 3; 4; 5; 6; 7g. The reducibe transition probabiity matrix P is given by: P = 0 1=4 1=4 1= =6 5= =8 1= = =14 3=14 3=14 3=14 1=8 1=24 0 1=6 1=6 1=6 1=3 1= =8 3=8 1=4 1= =12 3=8 1=8 1=4 1=6 The partition in communication casses is I = fi 1 = f1; 2; 3gg; I 2 = f4; 5; 6; 7gg. The stochastic eft eigenvectors corresponding to spectra radius of respective matrices P I1I1 and P I2I2 are v 1 = (7=16; 3=16; 6=16), 1 : C A 4 INRIA

15 Weak umpabiity of nite Markov chains and positive invariance of cones 13 v 2 = (1=4; 1=4; 1=4; 1=4). Matrix b P is given by Let us form the matrices H 1 ; H 2 ; H 3 ; H 4 : bp 1 = (1=4; 3=4; 0; 0); b P2 = (7=12; 5=12; 0; 0); bp 3 = (1=7; 0; 3=14; 9=14); b P4 = (5=12; 1=24; 2=9; 2=3): H 1 = H 3 = 0; H 2 =?7=12 7= =24?7= ; H 4 1=18 0?1=18 0 1=72?1=24?7=72 1=8?5=72 1=24 11=72?1=8 The nonnegative soutions to the homogeneous system associated with each previous matrix dene the four foowing poyhedra cones C 1 1; C 1 2; C 1 3; C 1 4 (see formua (4)): C 1 1 = C 1 3 = IR + ; C 1 2 = Cone(v 0 2); C 1 4 = Cone(v 0 4); with v 0 = (1; 2) = R 2 2v 1 =R 2 v 1 1 T and v 0 = (1; 1; 1) = R 4 4v 2 =R 4 v 2 1 T. Note that v 0 (resp. v0 2 4 ) is the positive eft eigenvector (up to a constant mutipicative) corresponding to the spectra radius P b (2; 2) = 5=12 (resp. P b (4; 4) = 2=3) of the irreducibe matrix PC(2)C(2) (resp. P C(4)C(4).) It is easy to check that the conditions of the Theorem 3.6 are met and thus CM( P b ) 6= f0g. If we construct the cone Cv 0 = Cone(fe 1 ; (R?1 2 v0 2); e 3 ; (R?1 4 v0 4)g), then we observe that C 1 = C v 0. It foows that CM( b P) = C1 = C v 0. 1 A : P Let us dene the foowing positive vector on S, v def = k2j R?1 I k v k, the convex subsets of IR N / C v = M 2 ^S Cone(R?1 v C() ); A v = C v \ A and matrix b P by b P = f(v; C()) e P for a 2 ^S. In the previous exampe, we found that C M( b P ) = Cv or AM( b P ) = Av. We can verify (with Theorem 3.6) that AM( b P ) = Av =) f(v; C(); C(m)) = f(v; C(m)) 8; m 2 ^S: (7) On the other hand, property in the right hand side impies that A v AM( b P ) with Coroary 3.5. Thus, it is a sucient condition for weak umpabiity with matrix b P as noted in Kemeny and Sne (1976) for irreducibe matrix P. We aso note that the right hand side in (7) gives for a 2 ^S, v C() P C()C() = b P(; )vc(). Thus, for a 2 ^S, vc() is a positive eft eigenvector of matrix P C()C() corresponding to the eigenvaue b P (; ). It can be usefu to know when the converse impication of (7) hods. It is shown to be vaid in Peng (1995) under the irreducibe assumption for the initia matrix P and the additiona condition (?): (?): the coumn vectors of matrices P k V (k 0) span IR N where V is the N M matrix dened by V (i; ) = 1 if i 2 C() and 0 otherwise. The previous comments precise some reation between the various equivaent conditions given in Theorem 3.1 from Peng (1995). Since this theorem is based ony on the condition (?) and the unicity of the t.p.m. associated with any aggregated chain from AM( b P ), it can be directy extended to our context. Note that a Peng's resuts hod in the context of Section 2. Theorem 4.3 Let us assume that partition P is a renement of the partition I of S. Under the condition (?), the foowing are equivaent: 1. agg(v=v1 T ; P; P) satises to the Chapman-Komogorov equations; RR n2801

16 14 James Ledoux 2. f(v; C(); C(m)) = f(v; C(m)) for a ; m 2 ^S; 3. AM( b P ) = Av. proof. Let be a probabiity distribution of A. Let us dene the th row, for a 2 ^S, of the M N matrix U by f(; C()). Note that U v P V U v = U v P is equivaent to the property reported in statement 2. First, we show that statement 1 impies the second one. If agg(v=v1 T ; P; P) satises to the Chapman- Komogorov equations: bp k = U v P k V 8k 1: Consequenty, we have for a k 0 U v P k+1 V = b P k+1 = b P b P k = b PUv P k V: We get 8k 0, [U v P? b PUv ]P k V = 0. Under the condition (?), we concude to U v P? b PUv = 0 U v P = U v P V U v : Now, et us assume that U v P V U v = U v P. As previousy noted, it impies that C v is positivey invariant by matrix P and C v CM( b P ). Consequenty, X is weak umpabe and Av AM( b P ). It remains to state that under the condition (?), C v = CM( b P ). Let be a vector in A M( b P ). Since agg(; P; P) is a homogeneous Markov chain, agg(; P; P) satises to the Chapman-Komogorov equations: bp k = U P k V 8k 1: Consequenty, under (?), we obtain in the same manner as in the rst part of the proof U P = U P V U : In a second part, we show that U P = U v P foows from the the unicity of the transition matrix for any aggregated chain from an initia distribution 2 AM( b P). We necessariy have for a k 0 ( b P ) k+1 = b P k+1 U P k+1 V = U v P k+1 V [U P? U v P ]P k V = 0: We deduce from the condition (?) that U P = U v P, which impies that f( C() P; C(m)) = f(v C() P; C(m)) for a ; m 2 ^S. Combining the both parts, we have for a 2 AM( b P ), 8; m 2 ^S f(; C(); C(m)) = f(; C(m)); f(; C(); C(m)) = f(v; C(); C(m)): Consequenty, we have obtained [8 2 ^S f(; C()) = f(v; C())] for a 2 A M( b P ) that is A M( b P) = Av. This proof hods aso in the context of Section 2 because, apart from the condition (?), we ony need that a aggregated chains concerned with share the same transition probabiity matrix. Theorem 4.2 of this section can be appied to derive the two main pubished resuts on weak umpabiity. The rst one deas with irreducibe matrix P, that is I reduces to ony one cass. Coroary 4.4 (Rubino and Sericoa 1991) The transition probabiity matrix P of the origina chain is assumed to be irreducibe. If agg(; P; P) is a homogeneous Markov chain then agg(; P; P) is aso a homogeneous Markov chain where is the stochastic vector soution to P =. The t.p.m. P b is the same for any aggregated homogeneous Markov chain and is given by P b = f(; C()) P e ; 2 ^S. INRIA

17 Weak umpabiity of nite Markov chains and positive invariance of cones 15 Remark 7 Let us dene the foowing poyhedra cone C = L 2 Cone(fR?1 ^S C() ). As noted previousy, the positive invariance of the set C by P impies that C CM( b P) and thus we have A = C \A AM( b P ). The interest of this condition comes from the fact that it can be checked from the \data" of the probem. A second famiy of Markov chains can aso be treated with Theorem Coroary 4.5 (Ledoux et a. 1994) Let us consider a famiy of Markov chain with transition probabiity matrix P such that the partition of S induced by the communication equivaence reation is I = fi 1 ; I 2 g: where I 1 contains one absorbing state and I 2 a the transient ones. If there exists 2 A such that I2 6= 0 and agg(; P; P) is a homogeneous Markov chain then agg((0; v); P; P) is aso a homogeneous Markov chain with v is the stochastic vector soution to vp I2I2 = v, where is the spectra radius of the matrix P I2I2. We reca that v is caed the quasi-stationary distribution associated with the famiy (:; P ). The t.p.m. b P is the same for any homogeneous Markov chain agg(; P; P) with an initia distribution whose support contains transient states. It is given by P1 b = e 1 and P b = f((0; v); C()) P e ; 2 ^S n f1g. Finay, as noted in Remark 7, if we have the opportunity to make a nontrivia aggregation with respect to the partition P then that must hod for any probabiity distribution of the poytope dened by A v = Conv(f(0; v) C() ; = 2 ^Sg). Concusion This paper extends to genera nite Markov chains the inear system approach used in Abde-Moneim and Leysieer (1982), Rubino and Sericoa (1991) for weak umpabiity probem. In adopting here the viewpoint of positive invariance of poyhedra cones, we propose new resuts on weak/strong umpabiity of a nite Markov chain. Most of our resuts are expressed with \oca" characteristics of the chain, that is to the eve of the state casses of the partition. This aows us to derive (or extend) spectra properties associated with exact aggregation. In a genera manner, our work species some (geometrica) invariant corresponding to the umpabiity requirement which are promising for study reated probems: investigate formay the weak umpabiity of strongy structured Markovian modes and anayze sensitivity to the \data" of the exact aggregation feasibiity. We do not go into further detais here. References A.M. Abde-Moneim and F.W. Leysieer, Weak umpabiity in nite Markov chains, J. App. Probab. 19 (1982) 685{691. A.M. Abde-Moneim and F.W. Leysieer, Lumpabiity for non-irreducibe nite Markov chains, J. App. Probab. 21 (1984) 567{574. F. Ba and G. Yeo, Lumpabiity and marginaisabiity for continuous-time Markov chains, J. App. Probab. 29 (1993) 518{528. A. Berman and R. Pemmons, Nonnegative Matrices in the Mathematica Sciences (Academic Press, 1979) J.G. Kemeny and J.L. Sne, Finite Markov chains (Springer-Verag, 1976). J. Ledoux, A necessary condition for weak umpabiity, Oper. Res. Letters 13 (1993) 165{168. J. Ledoux, G. Rubino and B. Sericoa, Exact aggregation of absorbing Markov processes using quasistationary distribution, J. App. Probab. 31 (1994) 626{634. J. Ledoux, On weak umpabiity of denumerabe Markov chains, Stat. and Probab. Letters 25 (1995) 329{339. RR n2801

18 16 James Ledoux N.F. Peng, On weak umpabiity of a nite Markov chain, To appear in Stat. and Probab. Letters (1995) G. Rubino and B. Sericoa, On weak umpabiity in Markov chains, J. App. Probab. 26 (1989) 446{457. G. Rubino and B. Sericoa, A nite caracterization of weak umpabe Markov processes. Part I: The discrete time case, Stochastic Process. App. 38 (1991) 195{204. P. Schweitzer, Aggregation methods for arge Markov chains, in: Iazeoa et a, eds, Mathematica Computer Performance and Reiabiity (Esevier-North Hoand, 1984). INRIA

19 Unité de recherche INRIA Lorraine, Technopôe de Nancy-Brabois, Campus scientifique, 615 rue du Jardin Botanique, BP 101, VILLERS LÈS NANCY Unité de recherche INRIA Rennes, Irisa, Campus universitaire de Beauieu, RENNES Cedex Unité de recherche INRIA Rhône-Apes, 46 avenue Féix Viaet, GRENOBLE Cedex 1 Unité de recherche INRIA Rocquencourt, Domaine de Vouceau, Rocquencourt, BP 105, LE CHESNAY Cedex Unité de recherche INRIA Sophia-Antipois, 2004 route des Lucioes, BP 93, SOPHIA-ANTIPOLIS Cedex Éditeur INRIA, Domaine de Vouceau, Rocquencourt, BP 105, LE CHESNAY Cedex (France) ISSN