Approximation Results for Non-Homogeneous Markov Chains and Some Applications

Transcription

1 Approximatio Results for No-Homogeeous Markov Chais ad Some Applicatios C.C.Y. Dorea 1 ad J.A. Rojas Cruz 2 Uiversidade de Brasilia ad UiCEUB, Brasilia-DF, Brazil Abstract I this ote we preset some approximatio results for o-homogeeous Markov chais that exted Hajal s results from fiite to geeral state space. These extesios will be compared to Isaacso ad Madse s strog ergodicity coditios that require the exitece of statioary distributios. AMS 1991 Subject Classificatio. Primary: 60J10; Secodary: 60J05. Key words ad Phrases: o-homogeeous Markov chai; weak ergodicity, strog ergodicity. 1. Itroductio ad Statemet of the Results The aalysis of the covergece of MCMC type algorithms such as the aealig algorithm has revived the iterest i the study of the ergodicity of o-homogeeous Markov chais, see for example, Gidas (1985) ad Aily ad Federgrue (1987). Cosider a Markov chai with state space S = {1, 2,...}, where the k-th trasitio probabilities ad the -th step trasitio probabilities are give by P (k,k+1) = P (k) =(P ij (k)) ad P (k,k+) = P (k)p (k +1)...P(k + 1). The study of the log-ru behaviour of the chai ivolves the otios of weak ad strog ergodicity with respect to a pre-selected orm. The chai is said to be weakly ergodic if for every k there exists a sequece of costat stochastic matrices {Q (k)} such that lim P (k,) Q (k) =0. (1) Ad the chai is said to be strogly ergodic if there exists a costat stochastic matrix Q such that for k =0, 1, 2,... we have lim P (k,) Q =0. (2) A matrix is said be costat if its rows are idetical ad for a matrix A =(A ij ) the orm 1 Partially supported by CNPq ad CAPES/PROCAD-Brazil. 2 Partially supported by CAPES Foudatio-Brazil. 1

2 is defied by A = sup A ij. (3) i j I sectio 3 we will prove the followig approximatio results : Theorem 1. If the chais {P ()} ad { ˆP ()} satisfy P () ˆP () < (4) 1 the they possess the same ergodic behavior, that is, if {P ()} is weakly (strogly) ergodic the same holds for { ˆP ()}. Theorem 2. Let {P ()} be a weakly ergodic chai for which there exists a sequece of costat stochastic matrices R(1), R(2),... ad R such that R()P () R( +1) < ad lim R() R =0. (5) 1 The the chai is strogly ergodic with limitig matrix R. Theorem 3. Assume there exists a weakly ergodic stochastic matrix Q such that lim P () Q = 0 (6) the the clai {P ()} is strogly ergodic. Theorems 1 ad 2 are just extesios of Hajal s (1956, 1958) results from fiite to the deumerable case. I sectio 2 we will compare our results with the extesios give by Isaacso ad Madse (1976) where the existece of the statioary distributio is assumed. More geeral settigs ca be cosidered. For that, let (S, F) be a measurable space ad let { P (k,k+1) (x, B) =P k (x, B) : x S, B F } (7) be a sequece of trasitio probability kerels with the -th step trasitio kerel give by P (k,k+) (x, B) = P (k,k+m) (x, dy)p (k+m,k+) (y,b), 1 m 1. S The replacig the orm (3) by the total variatio orm [ λ(, ) = sup x S sup λ(x, A) if λ(x, B) A F B F we ca exted Theorems 1-3 to the geeral state space case (see Remark 2). A few applicatios are icluded i sectio 2. Of particular iterest is Example 3 that exteds some results o the simulated aealig algorithm. ] (8) 2

3 2. Prelimiary Results ad Applicatios A equivalet characterizatio for the weak ergodicity (1) ca be give by the Dobrushi s ergodic δ coefficiet: δ(p )= 1 2 sup i,j k P ik P jk with lim δ(p (k,k+) )=0, k =1, 2,... (9) For A ad B ay two matrices ad for P ad Q stochastic matrices the followig iequalities hold: AB A B, δ(pq) δ(p )δ(q) ad δ(p ) δ(q) P Q. (10) Moreover, if A < ad A ik = 0 for all i, we have k=1 AP A δ(p) (11) (see Isaacso ad Madse (1976)). Propositio 1. Give two chais {P ()} ad {Q()} we have P (k,k+) Q (k,k+) 1 j=0 P (k + j) Q(k + j). (12) Proof. First write P (k,k+) Q (k,k+) = (P (k,k+ 1) Q (k,k+ 1) )P (k + 1)+ Q (k,k+ 1) (P (k + 1) Q(k + 1)). Now, for ay stochastic matrix P we have by (11) ad the fact that P =1 (P (k,k+ 1) Q (k,k+ 1) )P (k + 1) P (k,k+ 1) Q (k,k+ 1) Q (k,k+ 1) (P (k + 1) Q(k + 1)) P (k + 1) Q(k + 1). It follows that P (k,k+) Q (k,k+) P (k + 1) Q(k + 1) + P (k,k+ 1) Q (k,k+ 1). Proceedig this way we get (12). Remark 1. Assume that for each the trasitio matrix P () admits a statioary distributio π(), that is, π() = π()p (). We compare our results with the extesios give by Isaacso ad Madse (1976). (a) Theorem V.4.3 states that if {P ()} is weakly ergodic ad π j () π j ( +1) < (13) 1 j S the the chai is strogly ergodic. This follows from Theorem 2. Just take R() to be a matrix whose rows are equal to π(). Ad from (13) there exists a costat stochastic matrix R such that lim R() R =0. 3

4 (b) Theorem V.4.5 assures that if lim P () P = 0 ad P is weakly ergodic the the chai is strogly ergodic. This fact follows directly from Theorem 3 which provides a simpler ad more direct alterative proof that does ot require the assumptio of the existece of statioary distributios. (c) Note that, if P () =Q with Q weakly ergodic we ca coclude from Theorem 3 that Q is strogly ergodic so it has a uique statioary distributio. Thus, as corollary we obtai Theorem VI.4.2 : if Q is weakly ergodic the it has a uique statioary distributio. O the other had, uder assumptios of Theorem 3 the tail of the sequece {P ()} will be weakly ergodic ad Theorem 3 will the follow from Theorem V.4.5. Example 1. Let P ij () = 1 if i j, P 2 2 ii () =1 1 ad let ˆP () =I, the idetity 2 matrix. Clearly ˆP () is ot weakly ergodic ad (4) holds. It follows from Theorem 1 that {P ()} caot be weakly ergodic. Example 2. Let P 11 () =1 1, P 2 12 () = 1 ad P 2 1k () = 0 for k>2. For j 2 let P j1 () = 1, P jj() =1 1 ad P jk = 0 for k 1,j. Note that δ(p ()) = 1 1. Thus by (10) we have δ(p (k,k+) ) k+ 1 l=k ( 1 1 ) so that {P ()} is weakly ergodic. Take R() =R with R j1 = 1 ad R ji = 0 for i>1. Clearly lim R() R = 0. The for B() =R()P () R( + 1) we have B j1 () = 1 ad 2 B j2 () = 1. Also B() = 2 ad the strog ergodicity follows from Theorem Example 3. Cosider the simulated aealig algorithm where a o-homogeeous Markov chai {X } is geerated. First we have a selectio stochastic matrix Q which is assumed to be irreducible, reversible ad Q ii > 0 for all i S = {1,...,N}. At step +1 a radom variable Y is geerated with probability Q XY ad X +1 = Y with acceptace probability a +1 (X,Y), otherwise, X +1 = X. The trasitio probabilities are give by P ij () =Q ij a (i, j) ifj i ad P ii () =1 l i Q il a (i, l). These algorithms are used to estimate the global miimum f mi = mi{f(i) :i S} of a give fuctio f o S. For the Metropolis acceptace probability a (i, j) = mi{1, exp{ 1 c (f(j) f(i))} Hajek (1988) showed that if the coolig schedule c 0 ad exp( d c )=,where d is the depth of the deepest local miima, we get the desired covergece lim P (f(x )=f mi ) = 1. His argumets make use of the otios of the depth of local miima, botto of the cup, depth of the cup ad others which makes the proof laborious. Usig Theorem 2 this result ca somehow be improved. Let Q be as above (o eed of the assumptio Q ii > 0) ad let α = mi{q i,j : Q i,j > 0}. Note that sice S is fiite we have α>0.ford = max j {f(j) f mi } assume that c 0 d log 4

5 ad that c 0. It follows that a (i, j) 1. Also, if Q ii > 0 ad Q ij > 0 the P ii () =1 j i Q ij = Q ii α ad P ij () =Q ij a (i, j) α. To show the weak ergodicity ote that for i ad k fixed there exist i o ad k o such that Q iio > 0,Q kko > 0, P iko () P kko () α ad P ki o () P iio () α. Now, P ij () P kj () P ij () P kj + P iko () α j j i o,k o + P ki o () α ad δ(p ()) (1 α ). Usig the iequality (10) we get the weak ergodicity. Let π =(π 1,...,π N ) be a probability such that π i Q ij = π j Q ji ad let S mi = {j : f(j) = f mi }. Without loss of geerality we may assume that 1 S mi. The for j 1 ad l 1 we have a (1,j)=exp{ 1 c (f(j) f mi )} ad a (i, j)a (j, l) =a (1,l)a (l, j). (14) Sice S is fiite ad Q is aperiodic ad irreducible we have π j > 0, j. Defie the sequece of costat stochastic matrices R() by R ij () =π j a (1,j)b, b =[ l π l a (1,l)] 1 ad we have lim R ij() = 0 if j S mi = π j [ π l ] 1 if j S mi. l S mi Takig R = lim R() we have R() R 0. Let V () =R()P (), that is, V ij () =b {α j a (1,j)+ π l a (1,l)Q lj a (l, j)+π j a (1,j)[1 Q jr a (j, r)]}. l j r j Usig (14) ad the reversibility coditio we get V ij () =π j a (1,j)b = R ij (). It follows that R()P () R( +1)=R() R( + 1). To verify (5) ote that sice c 0 we have 1 a (1,j) a +1 (1,j), ad b b +1 b =[ l S mi π l ] 1 <. Now R ij () R ij ( +1) = π j [a (1,j) a +1 (1,j)] b + a +1 (1,j)[b b +1 ] π j [b(a (1,j) a +1 (1,j)) + (b +1 b )] ad m m R ij () R ij ( +1) π j [b (a (1,j) a +1 (1,j)) + (b +1 b )] =1 =1 π j [b(a 1 (1,j) a m+1 (1,j)) + b m+1 b 1 ] 2bπ j. 5

6 It follows that R()P () R( +1) = R ij () R ij ( +1) 2b <. 1 j S 1 Fially, from Theorem 2 we get P (k,k+) R 0 ad for k =1, 2,... 0 if j S mi lim P (k,k+) ij = π j [ π l ] 1 if j S mi. l S mi Example 4. Suppose {P i ()},...,{P N ()} are strogly ergodic with limits P 1 ( ),...,P N ( ). If α j 0 ad α α N = 1 we ca coclude from Theorem 1 that the liear combiatio { N j=1 α j P j ()} is strogly ergodic with limit N j=1 α j P j ( ). Also, suppose that {P ()} is strogly ergodic ad that {Q()} is such that Q() I < where I is the idetity matrix. The for ay subset N 1 N = {1, 2,...} the chai Q()P (), N 1 ˆP () = P (), N\N 1 is also strogly ergodic sice ˆP () P () 1 3. Proof of the Results N 1 Q() I P () = N 1 Q() I <. Proof of Theorem 1. From (4) ad Propositio 1 give ɛ>0 there exists k 0 such that P (k,k+) ˆP (k,k+) ɛ, ad k k 0. (15) (i) Let {P ()} be a weakly ergodic chai the from (9) there exists 0, such that δ(p (k,k+) ) ɛ, k a 0. (16) From (10), (15) ad (16) we get the weak ergodicity of { ˆP ()} (see also Paz (1970)). (ii) Now assume that {P ()} is strogly ergodic. Let m 0 k 0 we will show that { ˆP (m 0,m 0 +) } is a Cauchy sequece with respect to. Assume l m 0 ad without loss of geerality let l<m 0. The for large ˆP (l,l+) Q = ˆP (l,m 0) ˆP (m 0,l+) ˆP (l,m 0) Q ad sice ˆP (l,m 0) =1wehave ˆP (l,l+) Q ˆP (m 0,l+) Q 0. Sice {P ()} is strogly ergodic there exists k 1 such that Usig (15) we have P (m 0,m 0 +m) P (m 0,m 0 +) ɛ, k 1, m k 1. (17) ˆP (m 0,m 0 +) ˆP (m 0, 0 +) 3ɛ, m k 1, k 1. 6

7 Therefore { ˆP (m 0,m 0 +) } is a Cauchy sequece. Let Q be a stochastic matrix such that lim ˆP (m 0, 0 +) Q =0. (18) From (i) we have { ˆP ()} weakly ergodic so that δ( ˆP (m 0,m 0 +) ) 0. By (10) we have δ( ˆP (m 0,m 0 +) ) δ(q) ˆP (m 0,m 0 +) Q. It follows that δ(q) = 0. Thus Q is a costat matrix. Proof of Theorem 2. (i) Write V () =P () R( + 1) ad sice R( ) is a costat matrix we have for all, k, l, R()P () R( +1)=R()V () ad V (k)r(l) =0. (19) Moreover, sice V (k)v (k +1)=V (k)p (k + 1) we have V (l,l+) = V (l)p (l+1,l+). (20) (ii) From (5), (19) ad (20) give ɛ>0 there exists k 0 such that R(l)V (l,l+) R(l)V (l) ɛ, m k 0 ad. (21) l m l m Usig the followig result due to Paz (1970) : If P () =V () +S() where {S()} is a sequece of costat stochastic matrices the {P ()} is weakly ergodic if ad oly if lim V (k,k+) = 0 for every k. Our chai is weakly ergodic so give m k 0 there exists 0 = 0 (m, ɛ) such that (iii) Note that usig (19) we have ad i geeral V (m,m+) ɛ for 0. (22) P (k,k+) = (V (k)+r(k + 1))(V (k +1)+R(k + 2))P (k+2,k+) It follows from (20) that = (V (k,k+2) + R(k +1)V (k+1,k+2) + R(k + 2))P (k+2,k+) P (k,k+) = V (k,k+) + k+ 1 l=k+1 P (k,k+) R(k + ) V (k,k+) + If k k 0 the from (21) ad (22) we have R(l)V (l,l+) + R(k + ). (23) l k+1 P (k,k+) R(k + ) 2ɛ, 0. R(l)V (l) 7

8 Ad coclusio follows from the fact that R() R 0. If k<k 0 the use the fact that P (k,k0) = 1, (11) ad the idetity P (k,k+) R(k 0 + )=P [ (k,k 0) P (k 0,k 0 + ) R(k 0 + ) ] where = + k k 0. Proof of Theorem 3. Sice Q is weakly ergodic give ɛ>0 there exits 0 = 0 (ɛ) ad a costat stochastic matrix Q such that ad for large By Propositio 1 we have Q Q ɛ, o (ɛ) (24) P (k,k+ 0) Q 0 Q = P (k,k+ 0) (Q 0 Q ) ɛ. P (k,k+) P (k,k+ 0) Q 0 = P (k,k+ 0) ( P (k+ 0,k+) Q 0) 0 1 j=0 P (k j) Q. Let ɛ = ɛ/ 0 the there exists 1 (ɛ ) such that P (m) Q ɛ for m 1 (ɛ ). Take such that k (ɛ ) the 0 1 j=0 P (k j) Q 0 ɛ = ɛ. It follows that give ɛ>0 we have P (k,k+) Q 2ɛ for 0 k + 1 (ɛ ). Thus {P ()} is strogly ergodic. Remark 2. For the geeral state space we take δ(p ) = sup x,y,a P (x, A) P (y,a) ad (10) ad (11) hold with the orm (8) i place of the orm (3). The proofs of Theorem 1-3 ca be carried out similarly. Refereces : [1] Aily, S. ad Federgrue, A. (1987) - Simulated aealig methods with geeral acceptace probabilities, Joural of Applied Prob., vol. 24, [2] Dobrushi, R.L. (1956) - Cetral limit theorem for o-statioary Markov chais, Theory of Prob. ad Applicatios, vol. 1, [3] Gidas, B. (1985) - Nostatioary Markov chais ad the covergece of the aealig algorithm, Joural of Stat. Physics, vol. 39,

9 [4] Hajek, B. (1988) - Coolig schedule for optimal aealig, Mathematics of Operatio Research, vol. 13, [5] Hajal, J. (1956) - The ergodic properties of o-homogeeous fiite Markov chais, Proceedigs of the Cambridge Philosophical Society., vol. 52, [6] Hajal, J. (1958) - Weak ergodicity i o-homogeeous Markov chais, Proceedigs of the Cambridge Philosophical Soc., vol. 54, [7] Isaacso, D.L. ad Madse, R. W. (1976) - Markov Chais: Theory ad Applicatios, Wiley ad Sos, New York. [8] Madse, R.W. ad Isaacso, D.L. (1973) - Strogly ergodic behavior for o-statioary Markov processes, The Aals of Probability, vol. 1, [9] Paz, A. (1970) - Ergodic theorems for ifiite probabilistic tables, The Aals of Math. Statistics, vol. 41, [10] Rojas Cruz, J.A. (1998) - O the covergece of o-homogeeous Markov chais: weak ad strog ergodicity, Ph.D. Thesis, Departmet of Mathematics, Uiversidade de Brasilia, Brazil (i portuguese). Mailig Address: Chag C. Y. Dorea C.P Departameto de Matematica Uiversidade de Brasília Brasília-DF/BRAZIL cdorea@mat.ub.br 9