Perturbations of continuous-time Markov chains arxiv:1611.1254v1 [math.pr 4 Nov 216 Pei-Sen Li School of Mathematical Sciences, Beijing Normal University, Beijing 1875, China E-ma: peisenli@ma.bnu.edu.cn Abstract. The equivalence of regularity of a Q-matrix with its bounded perturbations is proved and a integration by parts formula is established for the associated Feller minimal transition functions. Key words and phrases. continuous-time Markov chain; integration by parts formula; perturbation; Feller minimal process; regularity. Mathematics Subject Classification (21: 6J27; 6J8. 1 Introduction One of the basic questions in studying continuous-time Markov chains is to find the regularity criterion, i.e., to investigate the conditions under which the given Q-matrix is regular, or, equivalently, the corresponding Feller minimal process is honest in the sense that the corresponding transition function P(t {j (t;i,j N} satisfies j j(t 1 for all i and t. Here we assume the chain has state space N : {,1,2,...}. We refer to Anderson (1991 and Chen (24 for the general theory of continuous-time Markov chains. In this note we show that the regularity property is preserved under a bounded perturbation of the Q-matrix. We also establish a integration by parts formula for the corresponding Feller minimal processes without the regularity condition. Given two Q-matrices R (r ij ;i,j N and A (a ij ;i,j N, we call Q (q ij ;i,j N : R+A the perturbation of R by A. Throughout this note, we assume all Q-matrices are stable and conservative. The main purpose of this note is to prove the following theorems: Theorem 1.1 Suppose that A is a bounded Q-matrix. Then Q R+A is regular if and only if R is regular. Theorem 1.2 Let Q(t {Q ij (t;i,j N} and R(t {R ij (t;i,j N} be the Feller minimal transition functions of Q and R, respectively. Then we have the following integration by parts formula k N R ik (sa k Q kj (t sds R (t va lm Q mj (vdv+r ij (t Q ij (t. (1.1 1
In particular, when k N R ik(sa k Q kj (t sds <, we can rewrite (1.1 as Q(t R(t R(sAQ(t sds. (1.2 The perturbation theory of infinitesimal generators has been a very useful tool in the hands of analysts and physicists. A considerable amount of research has been done on the perturbation of linear operators on a Banach space. The effect on a semigroup by adding a linear operator to its infinitesimal generator was studied by Phlips (1952 and Yan (1988. However, these authors did not show the equivalence of the regularity of a Q-matrix with its bounded perturbations. The integration by parts formula (1.1 was given by Chen (24, p51 under a stronger condition. The Q-matrix of the branching processes with immigration and/or resurrection introduced in Li and Chen (26 can be regarded as the perturbations of a given branching Q-matrix. Example 1.3 Let R (r ij ;i,j N be a branching Q-matrix given by { ibj i+1 j i 1,i 1; r ij otherwise, where b j (j, b j b 1. n 1 Let A (a ij ;i,j N be a bounded Q-matrix given by c j i+1 j i,i 1; a ij h j j,i ; otherwise, where { hj (j, c j (j, j1 h j h ; j1 c j c. Then the Q-matrix Q (q ij ;i,j Z+ : R + A is called a branching Q-matrix with immigration and resurrection. The corresponding continuous-time Markov chain is called a branching process with immigration and resurrection. Note that the regularity criterion of the branching Q-matrix R is given by Harris (1963. Since A is a bounded Q-matrix, by Theorem 1.1 we see Q is regular if and only R is regular. This simplifies considerably the proof of Theorem 2.1 in Li and Chen (26. 2 Bounded perturbations In this section, we assume A is a bounded Q-matrix. We shall prove that the regularity of R and Q are equivalent. Let γ sup i a i inf i a ii. Let q ii γ a i > and q ij q ij for i j. Let a ij a ij +γδ ij >. Then we have q ik a ik +(1 δ ikr ik >, 2
Proposition 2.1 The backward Kolmogorov equation of Q is equivalent to the following equation: Q ij (t k N e (r i+γ(t s q ik Q kj(sds+δ ij e (r i+γt. (2.3 Proof. Suppose that Q(t {Q ij (t;i,j N} is a solution of the backward Kolmogorov equation t Q(t QQ(t. Then t Q ij (t+(r i +γq ij (t k Nq ik Q kj(t. Multiplying both sides by the integrating factor e (r i+γt, we find t (e (r i+γt Q ij (t e (r i+γt k N q ikq kj (t. Integrating and dividing both sides by e (ri+γt give (2.3. Conversely, suppose Q ij (t is a solution of (2.3. By differentiating both sides of the equation we get the backward Kolmogorov equation t Q(t QQ(t. Let Q(t {Q ij (t;i,j N} and R(t {R ij (t;i,j N} be the minimal transition functions of Q and R, respectively. By the second successive approximation scheme; see, e.g., Chen (24, p64, we see where and R ( Q ij (t n Q (n ij (t and R ij(t ij (t δ ije rit, R (n+1 ij (t ij (t δ ije (ri+γt, Q (n+1 ij (t k N Q ( n R (n ij (t, (2.4 e ri(t s r ik R (n kj (sds (2.5 e (ri+γ(t s q ik Q(n kj (sds. (2.6 Lemma 2.2 For any n we have n 1 Q (n ij (t p with 1 p by convention. e γ(t s R (n p 1 (t sa lk Q(p kj (sds+r(n ij (te γt (2.7 3
Proof. For n, we have (2.7 trivially. Suppose that (2.7 holds for n,1,,m. Recall that q ik a ik +(1 δ ikr ik. By the second equality in (2.6 we have Q (m+1 ij (t By (2.7 and (2.5 we have I 1 p l,r N p l,r N : I 1 +I 2. [ e (ri+γ(t s r ik + + e γ(t u [ e (r i+γ(t s r ik Q (m kj (sds + k N p l,r N s e (r i+γ(t s r ik R (m kj (se γs ds u e (r i+γ(t s a ikq (m kj (sds e γ(s u R (m p 1 kl (s ua lr Q(p rj ds (udu e ri(t s r ik R (m p 1 kl (s uds a lr Q(p rj (udu e (r i+γ(t s r ik R (m kj (se γs ds e γ(t u R (m p (t ua lrq (p rj (udu+r(m+1 ij (te γt. On the other hand, using the first equality in (2.5 we obtain I 2 k N e γ(t s R ( ii (t sa ik Q(m kj (sds e γ(t s R ( (t sa lk Q(m kj (sds. Summing up the above expressions of I 1 and I 2, we see (2.7 also holds when n m+1. That gives the desired result. Proposition 2.3 Let Q(t {Q ij (t;i,j N} and R(t {R ij (t;i,j N} be the minimal transition functions of Q and R, respectively. Then Q ij (t is the unique solution of the following equation Q ij (t e γ(t s R (t sa lk Q kj(sds+r ij (te γt. (2.8 Proof. We first prove the uniqueness of (2.8. Let Q ij (t be another solution of (2.8. Let c ij (t Q ij (t Q ij (t and c j (t sup i c ij (t. Then we have c ij (t R (t sa lk c kj(sds. 4
Taking the supremum we have c j (t sup i R (t sa lkc j (sds γ c j (sds. Using Gronwall s inequality we have that c j (t. Thus (2.8 has at most one solution. Next we wl show that Q ij (t satisfies (2.8. Using (2.4 and (2.7 we have Q ij (t n n p e γ(t s R (n p (t sa lk Q(p kj (sds+ n R (n ij (te γt. Interchanging the order of summation and using (2.4 again we obtain Q ij (t That completes the proof. e γ(t s np R (n p (t sa lk p e γ(t s R (t sa lk Q kj(sds+r ij (te γt. Q (p kj (sds+r ij(te γt Proof of Theorem 1.1. Summing up both sides of (2.8 over j, we see that x i (t : j Q ij(t is a solution to the following equation: x i (t e γ(t s R (t sa lk x k(sds+e γt R ij (t. (2.9 Suppose that R is regular. Then we have j R ij(t 1, so x i (t 1 is a solution of (2.9. Let x i (t be another solution of (2.9. Set c i (t x i (t x i (t and c(t sup i c i (t. By (2.9 we obtain Taking the supremum we get c(t sup i c i (t R (t sa lkc k (sds. R (t sa lkc(sds γ j c(sds. Using Gronwall s inequality we have c(t. Then we see x i (t 1 is the unique solution to (2.9. Hence Q is regular. Conversely, suppose that Q is regular. Then x i (t j Q ij(t 1. Let y i (t j R ij(t. From (2.9 we have 1 e γt Then we must have y i (t 1, so R is regular. γe γ(t s y i (t sds. 5
3 Integration by parts formula Recall that R(t and Q(t are the Feller minimal transition functions of R and Q, respectively. By the second successive approximation scheme; see, e.g. Chen (24, p64 we have where Q ( Q ij (t n ij (t δ ije qit, Q (n+1 ij (t Q (n ij (t, (3.1 Let (Ω, F, F t,ξ t, be a realization of (R ij (t t. e qi(t s q ik Q (n kj (sds. (3.11 Lemma 3.1 Let σ t s denote the number of jumps of the trajectory t ξ t on the interval (s,t. Then for n we have n 1 Q (n ij (t p k N,l k (M t s;a n p 1,k (,t sa kl Q (p lj (sds + (M t ;A n,j(,t (3.12 with 1 p by convention, where A n,j(s,t {σ t s n,ξ t j} and M r t e r a(ξsds. Proof. For n we have (3.12 trivially. Suppose that (3.12 holds for n,1,,m. By (3.11 we have Q (m+1 ij (t e qi(t s r ik Q (m kj (sds + e q i(t s a ik Q (m kj (sds : I 1 +I 2. Denote τ inf{t : ξ t ξ }. By the Markov property we have I 1 e qi(t s r ik p r N,l r s + e qi(t s r ik P k (Ms;A m,j (,sds p r N,l r + v P k (M s v;a p,r (,s va rl Q (p lj (vdvds e q i(t s r ik P k (M s v;a p,r (,s va rl Q (p lj (vdsdv e a i(t s r 1 i r ik P k (M s ;A m,j(,sr i e r i(t s ds 6
p r N,l r r i e ris ds p r N,l r p r N,l r [ v a rl Q (p lj (vdv+ e a is r 1 i r ik P k (M t v s ;A p,r(,t v s e a is r 1 i r ik P k (M t s ;A m,j(,t sr i e r is ds [e aiτ P ξτ (Mt v τ ;A p,r(,t v τ a rl Q (p lj (vdv + [e aiτ P ξτ (Mt τ ;A m,j(,t τ [e aiτ (Mt v1 τ Am p,r (τ,t v F τ a rl Q (p lj (vdv + [e aiτ (Mt τ 1 Am+1,j (τ,t F τ p r N,l r On the other hand, we have (M t v ;A m p,r(,t va rl Q (p lj (vdv + (M t;a m+1,j (,t. I 2 l i l i r N,l r e q i(t v a Q (m lj (vdv (e a i(t v 1 {σ t v } a Q (m lj (vdv (M t v;a,r (,t va rl Q (m lj (vdv. (3.13 Summing up the above expressions of I 1 and I 2 we see (3.12 also holds when n m+1. That gives the desired result. Theorem 3.2 The Feller minimal transition functions Q(t and R(t satisfy the following equation Q ij (t k N,l k Proof. Using (3.1 and (3.12 we have (M t s 1 {ξ t s k}a kl Q lj (sds+ (M t 1 {ξ tj}. (3.14 Q ij (t n 1 n m k N,l k (M t s ;A k,n (,t sa kl Q (m lj (sds+ (M t 1 {ξ tj}. Interchanging the order of summation we see (3.14 holds. 7
Proof of Theorem 1.2. By the Markov property of {ξ t : t }, k N R ik (sa k P k (M t s1 {ξt s j}ds t [a(ξ s P ξs (Mt s1 {ξt s j} ds [ ( a(ξ s Mt s 1 {ξ tj} F s ds t [a(ξ s Mt1 s {ξtj} ds [1 {ξtj} a(ξ s e s a(ξudu ds [1 {ξtj} (1 e a(ξudu R ij (t (M t 1 {ξ tj}. On the other hand, by the Markov property, we have R (t va lm Q mj (vdv + + + + [ 1 {ξt v l}a lm Q mj (v (1 e v a(ξ udu dv (M t v 1 {ξ t vl } a lm Q mj (vdv [ v 1 {ξt v l}a lm Q mj (vdv (M t v 1 {ξ t vl } a lm Q mj (vdv [ a(ξ s { a(ξ s + k N s a(ξ s e v s a(ξ udu ds Mt v s 1 {ξ t vl }a lm Q mj (vdvds (M t v 1 {ξ t vl } a lm Q mj (vdv { [ s a(ξ s (M t v 1 {ξ t vl } a lm Q mj (vdv P ξs [ s (M t v1 {ξt vl } a lm Q mj (vdv R ik (sa k s 8 } Mt v s 1 {ξ t v l}a lm Q mj (vdv F s ds } Mt s v 1 {ξ t s vl }a lm Q mj (vdv ds P k (M t s v 1 {ξ t s v l}a lm Q mj (vdvds
+ (M t v1 {ξt vl } a lm Q mj (vdv. By the above two equations and (3.14 we obtain (1.1. Suppose that k N R ik (sa k Q kj (t sds <. Then subtracting it from both sides of (1.1 yields (1.2. Acknowledgments. The author would like to thank Professors Mu-Fa Chen, Yong-Hua Mao and Yu-Hui Zhang for their helpful comments. Research supported in part by 985 Project, NSFC(No 111313, 1151531, 1157143, SRFDP(No 213115 and the Fundamental Research Funds for the Central Universities. References [1 Anderson, W.J. (1991: Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer, New York. [2 Chen, M.F. (24: From Markov Chains to Non-Equibrium Particle Systems. 2nd Ed. World Scientific, Singapore. [3 Harris, T.E. (1963: The Theory of Branching Processes. Springer, Berlin. [4 Li, J.P. and Chen, A.Y. (26: Markov branching processes with immigration and resurrection. Markov Processes Relat. Fields 12, 139-168. [5 Phlips, R.S. (1953: Perturbation theory for semigroups of linear operators, Trans. Amer. Math. Soc 74, 199-221. [6 Yan, J.A. (1988: A perturbation theorem for semigroups of linear operators, Séminaire de Probabitiés 1321, 89-91. 9