Asymptotic properties of imprecise Markov chains

Transcription

1 FACULTY OF ENGINEERING Asymptotic properties of imprecise Markov chains Filip Hermans Gert De Cooman SYSTeMS Ghent University 10 September 2009, München

2

3 Imprecise Markov chain X 0 X 1 X 2 X 3 Every node is a state with finite state space X = {x 0, x 1,..., x n }. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 3 / 15

4 Imprecise Markov chain X 0 X 1 X 2 X 3 Every node is a state with finite state space X = {x 0, x 1,..., x n }. The upper transition operator T : R n R n is given by P (f x 1 ) P (f x 2 ) T f :=. so, T f (x i) = P (f x i ). P (f x n ) F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 3 / 15

5 Imprecise Markov chain X 0 X 1 X 2 X 3 T 3 f T 2 f T f f Every node is a state with finite state space X = {x 0, x 1,..., x n }. The upper transition operator T : R n R n is given by P (f x 1 ) P (f x 2 ) T f :=. so, T f (x i) = P (f x i ). P (f x n ) Doing so, P(f ) = P 0 (T 3 f ) where T 3 := T T T. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 3 / 15

6 Outline 1 Properties of the transition operator 2 Convergence 3 Structuring the state space 4 Convergence in terms of state properties F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 4 / 15

7 T has all the usual properties The properties of upper previsions are transfered to the transition operator 1 If f g then T f T g [monotonicity preserving], 2 If c R then T (f + c) = T f + c [constant additivity], 3 If α > 0 then T (αf ) = αt f [positive-homogeneity]. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 5 / 15

8 T has all the usual properties The properties of upper previsions are transfered to the transition operator 1 If f g then T f T g [monotonicity preserving], 2 If c R then T (f + c) = T f + c [constant additivity], 3 If α > 0 then T (αf ) = αt f [positive-homogeneity]. From the properties, it automatically follows that the map T is bounded, the map T is non-expansive T f T g f g. If a map has the first two properties it is also called topical. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 5 / 15

9 A lot is known for bounded and non-expansive maps 1 Edelstein [1963] showed that all elements of the ω-limit set of f, ω T (f ), are recurrent and moreover, that T acts isometrically on every ω-limit set. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 6 / 15

10 A lot is known for bounded and non-expansive maps 1 Edelstein [1963] showed that all elements of the ω-limit set of f, ω T (f ), are recurrent and moreover, that T acts isometrically on every ω-limit set. 2 Due to a result of Sine [1990] we know that T k f converges to a limit cycle for k, lim T k f = ξ f where T p ξ f = ξ f. k Moreover, the maximal period p is limited by a function of the dimension of the underlying space only. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 6 / 15

11 A lot is known for bounded and non-expansive maps 1 Edelstein [1963] showed that all elements of the ω-limit set of f, ω T (f ), are recurrent and moreover, that T acts isometrically on every ω-limit set. 2 Due to a result of Sine [1990] we know that T k f converges to a limit cycle for k, lim T k f = ξ f where T p ξ f = ξ f. k Moreover, the maximal period p is limited by a function of the dimension of the underlying space only. 3 Lemmens and Scheutzow [2005] showed for topical functions that the maximal period for a periodic point is ( n n /2 ). F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 6 / 15

12 How we interprete convergence max T k f min T k f f T f T 2 f T 3 f T 4 f T 5 f k F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 7 / 15

13 How we interprete convergence max T k f min T k f f T f T 2 f T 3 f T 4 f T 5 f k 1 An imprecise Markov chain PF-converges if max T k f min T k f as k. 2 An imprecise Markov chain converges if T k f ξ f with T ξ f = ξ f as k Clearly PF-convergence convergence. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 7 / 15

14 Convergence can be restated in terms of fixed points It can be easily seen that Proposition 1 An imprecise Markov chain converges if and only if every periodic points is also a fixed point. Remember that f is a fixed point if Tf = f. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 8 / 15

15 Convergence can be restated in terms of fixed points It can be easily seen that Proposition 1 An imprecise Markov chain converges if and only if every periodic points is also a fixed point. 2 An imprecise Markov chain PF-converges if and only if every periodic point of T is constant. Remember that f is a fixed point if Tf = f. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 8 / 15

16 Convergence can be restated in terms of fixed points It can be easily seen that Proposition 1 An imprecise Markov chain converges if and only if every periodic points is also a fixed point. 2 An imprecise Markov chain PF-converges if and only if every periodic point of T is constant. Remember that f is a fixed point if Tf = f. To conclude upon (PF)-convergence, all periodic points of T must be investigated. This is easy in the linear case, however... F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 8 / 15

17 An accessibility relation can be defined Definition We say that state y is accessible from state x, x y, if there exists n N such that T n I y (x) > 0. As is reflexive and transitive it determines a preorder on X. The binary relation on X is the associated equivalence relation. This communication relation partitions the state set X into equivalence classes called communication classes. The preorder induces a partial order on this partition, also denoted by. Because of this partial order, maximal communication classes will exist. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 9 / 15

18 The relation structures the state space M 1 M 2 M 3 X D 5 D 6 D 8 D 9 D 4 D 3 D 7 D 1 D 2 F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 10 / 15

19 Communication classes split up the imprecise Markov chain Proposition Consider a stationary imprecise Markov chain with upper transition operator T. Let C be a closed set of states, and let C be a partition of the state set X into closed sets. Then 1 T(hI B )(x) = 0 for all h L(X ), all x C and all B C c ; 2 Th(x) = T(hI C )(x) for all h L(X ) and all x C; 3 Th = C C T(I C h) = C C I C T(I C h) for all h L(X ). F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 11 / 15

20 Communication classes split up the imprecise Markov chain Proposition Consider a stationary imprecise Markov chain with upper transition operator T. Let C be a closed set of states, and let C be a partition of the state set X into closed sets. Then 1 T(hI B )(x) = 0 for all h L(X ), all x C and all B C c ; 2 Th(x) = T(hI C )(x) for all h L(X ) and all x C; 3 Th = C C T(I C h) = C C I C T(I C h) for all h L(X ). Consequently, if more communication classes exist, then PF-convergence is not possible. Example Assume there are n communication classes and take the gamble f = n k=1 ki C k then T f = n k=1 I C k T ki Ck = n k=1 I C k T k = f. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 11 / 15

21 Regularity implies PF-convergence Definition A maximal aperiodic communication class is called regular. If there is only one communication class, then X is irreducible. If X is irreducible and aperiodic, X itself is also called regular. ( n N)( k n)( x, y X )(x k y). x F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 12 / 15

22 Regularity implies PF-convergence Definition A maximal aperiodic communication class is called regular. If there is only one communication class, then X is irreducible. If X is irreducible and aperiodic, X itself is also called regular. ( n N)( k n)( x, y X )(x k y). It can be shown that regularity of X is a sufficient condition for PF-convergence. However, regularity is too strong. Example Take the precise model with transition matrix ( ) x F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 12 / 15

23 Top class regularity Definition An imprecise Markov chain is said to be top class regular if R := { x X : ( n N)( k n)( y X )T k I {x} (y) > 0 }. R is the set of maximal regular states. This means that R is a regular top class whenever the Markov chain is regular. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 13 / 15

24 Top class regularity Definition An imprecise Markov chain is said to be top class regular if R := { x X : ( n N)( k n)( y X )T k I {x} (y) > 0 }. R is the set of maximal regular states. This means that R is a regular top class whenever the Markov chain is regular. Top-class regularity is a necessary condition for PF-convergence, but not sufficient. Example ( ) f (x) Assume X = {x, y} and T f =, then the 1 is max{f (x), f (y)} belonging to the credal set to T. Therefore there is no PF-convergence. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 13 / 15

25 Necessary and sufficient conditions for PF-convergence Definition A stationary imprecise Markov chain is called regularly absorbing if it is top class regular (under ), meaning that R := { x X : ( n N)( k n)( y X )T k I {x} (y) > 0 }, and if moreover it is leaky, i.e. for all y in X \ R there is some n N such that T n I R (y) > 0. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 14 / 15

26 Necessary and sufficient conditions for PF-convergence Definition A stationary imprecise Markov chain is called regularly absorbing if it is top class regular (under ), meaning that R := { x X : ( n N)( k n)( y X )T k I {x} (y) > 0 }, and if moreover it is leaky, i.e. for all y in X \ R there is some n N such that T n I R (y) > 0. Remark that accessibility to the complete communication class R is required and not to a state of R. Example min{f (x), f (y)} Assume X = {x, y, z} and T f = min{f (x), f (y)} then min{f (x), f (y)} T n I {x} = T n I {y} = 0 and T n I {x,y} = I {x,y}. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 14 / 15

27 Necessary and sufficient conditions for PF-convergence Definition A stationary imprecise Markov chain is called regularly absorbing if it is top class regular (under ), meaning that R := { x X : ( n N)( k n)( y X )T k I {x} (y) > 0 }, and if moreover it is leaky, i.e. for all y in X \ R there is some n N such that T n I R (y) > 0. Theorem An imprecise Markov chain is PF-converging if and only if it is regularly absorbing. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 14 / 15

28 What about convergence in general? Conjecture An imprecise Markov chain converges if and only if 1 the -maximal communication classes are regular and 2 every periodic communication class leaks to the union of its -dominating classes. F. Hermans (Ghent University) Asymptotic properties of imprecise Markov chains WPMSIIP2 15 / 15