3b,12x2y + 3b,=xy2 + b22y3)..., (19) f(x, y) = bo + b,x + b2y + (1/2) (b Ix2 + 2bI2xy + b?2y2) + (1/6) (bilix' + H(u, B) = Prob (xn+1 e BIxn = u).

Transcription

1 860 MA THEMA TICS: HARRIS AND ROBBINS PROC. N. A. S. The method described here gives the development of the solution of the partial differential equation for the neighborhood of x = y = 0. It can break down only if the zero point is a singular point. Of course there is, in general, no difficulty in obtaining the approximate solution for a series around an arbitrary point. In boundary problems, there remains the question of fitting the undetermined coefficients (al, all,... ) to the boundary values. The method having been established, the reader will find no difficulty in applying it to the more common way of writing a polynomial, f(x, y) = bo + b,x + b2y + (1/2) (b Ix2 + 2bI2xy + b?2y2) + (1/6) (bilix' + 3b,12x2y + 3b,=xy2 + b22y3)..., (19) or to the problem of developing functions of several variables, instead of into a power series, into a series of other functions, having the quality that T<5PW A,.@A,= E aj*ax, (20) where every product is a linear combination of the functions with index no greater than that of the highest of the original functions. * Communication No from the Kodak Research Laboratories. Cf. the matrices Si appearing in Frobenius, G., Theorie der hyperkomplexen Grossen, Berl. Sitzungsber., 1903, and Herzberger, M., Ueber Systems hyperkomplexer Grossen, Inaugural Dissertation, Berlin, ERGODIC THEORY OF MARKO V CHAINS ADMITTING AN INFINITE INVARIANT MEASURE* By T. E. HARRISt AND HERBERT ROBBINS COLUMBIA UNIVERSITY AND THE INSTITUTE FOR ADVANCED STUDY Communicated by H. Whitney, June 3, Introduction.-We consider real valued Markov chains xo, xi,... with a stationary transition probability function..., x-, H(u, B) = Prob (xn+1 e BIxn = u). Under certain conditions there exists a probability measure Xr defined on the Borel sets B and satisfying 7r(B) = fir(du)h(u, B). (1) There is then a stationary Markov chain... x-1, xo, xi,... with the transition function H such that Prob (xn e B) = w(b), n = 0 h 1,... Birkhoff's ergodic theorem can then be applied to this chain (cf. ref. 2).

2 VOL. 39, 1953 AIA THEMA TICS: HARRIS AND ROBBINS 861 It may be that no probability measure 7r exists satisfying (1). (Such is the case, for example, if for every bounded Borel set B, lim Hk(u, B) = 0, where Hk denotes the k-step transition function, H1 = H.) There may, however, be an infinite measure 7r which satisfies (1). The purpose of this note is to show how the ergodic theorem of Hopf3 can then be applied. A similar treatment will be given elsewhere for certain stochastic processes involving a continuous time parameter (cf. ref. 5). The present note is closely related to recent work of Robbins7 and of Kallianpur and Robbins.4 2. Definition of the Measure m.-we shall assume henceforth that (1) has an "admissible" solution 7r; i.e., ir is a measure on the real Borel sets which satisfies (1), does not vanish identically, and is finite for bounded Borel sets. (Current unpublished work of C. Derman shows that this is so in a variety of circumstances.) Let Fo be the class of real Borel sets, Q the space of sequences x of real numbers, x = (..., x-1, xo, xl,...), and F the Borel extension of the cylinder sets in Q2. If A e F is determined by the coordinates Xk, Xk+1, then Q(A Xk = u) will denote the probability of A relative to the Markov chain starting with xk = u, as specified by H. A measure m may be set up on F as follows. If A is a cylinder set determined by xk,..., Xr, let m(a) = f7r(du)q(a Ix = u), (2) where (1) ensures that any value of j < k may be used indifferently in (2). It can be verified that m may be extended to F, there being no essential change in the proof of the extension theorem of Kolmogorov.6 The extended measure m satisfies (2) whenever A e F depends on Xk, Xk+l,... and j < k. Moreover, m(a) = 0 if and only if Q(A xj = u) = 0 for a.e. (7r) u; thus m(a) = 0 has a meaning in terms of ordinary Markov chain probabilities. From (1) it follows that if A e F and if T is the shift transformation, (Tx)i = xi+,, then m(a) = m(ta). 3. The Dissipative Part of Q.-For any B E Fo, let RB be the event that xn e B infinitely often for n = 1, 2,... ("x. e B i.o."). ASSUMPTION 1. If B e Fo then Q(RBI xo = u) = 1 for a.e. (7r) u in B. THEOREM 1. If Assumption 1 holds then the dissipative part of Q has m- measure 0 (see ref. 3 for terminology). Sketch of Proof: Let A be a cylinder set defined by, say, (xo, xi, Xk) e Bk+l, a (k + 1) -dimensional Borel set, and let nn12 Un ={U; 1 < Q(AIxo = u) < n n = Un=u+i 1, 2, Equation (2) ensures that A differs from E A (xo E U.) by a set of n=1

3 862 MA THEMA TICS: HARRIS A ND ROBBINS PROC. N. A. S. m-measure 0, and hence for a.e. (m)x in A, Tnx e A i.o., so that m(a E T-nA) = m(a). Next, let W be a "wandering" set with m(w) < o (it suffices to consider this case) and let A be a cylinder set such thatm(w - A) + m(a - W) < e. Then (all sumsn = ito c) 0 = m(w*etnw) > m(a ETnW) Zm[(T-nA) W] - e > m(w.et-a) - > m(a.et-8a) -2e = m(a) -2, and hence m(w) < m(a) + e < 3E, m(w) =O. 4. Metric Transitivity of T.-The following assumption is stronger than Assumption 1. ASSUMPTION 2. If B e Fo and 7r(B) > 0 then Q(RBIXO = u) = 1 for a.e. (7r) u in U. THEOREM 2. If Assumption 2 holds then the shift transformation T is metrically transitive. Sketch of Proof: First, let A e F be such that for every k = 1, 2,... A depends only on Xk, xk+l,... (call such a set a "terminal" set), and suppose A = TA. Assumption 2 and the invariance of A can be shown to imply that Q(A xo = u) is constant for a.e. (7r)u, and the 0-1 law shows that the constant is 0 or 1. It follows from (2) that m(a) = 0 or mq- A) = O. Now let g(x) be a fixed positive m-summable function which depends on a finite number of coordinates of x. is m-summable then E f(tix) lim 0 = (f; x) E g(t'x) 0 Hopf's theorem implies that if f(x) exists for a.e. (m)x. If f(x) depends on only a finite number of coordinates it follows from the result above on "terminal" invariant sets that.ff(x)m(dx)(3 I(f; x) = const. - fg(x)m(dx) (3) To prove metric transitivity it is sufficient to show that (3) holds for every m-summable f. Now let f(x) be any m-summable function and let fk(x) be a sequence of functions of a finite number of coordinates such that lim k-+a fjf(x) -fk(x)im(dx) = 0.

4 VOL.- 39, 1953 MA THEMA TICS: HARRIS A ND ROBBINS 863 Then whence AU; X) = A(f - fk; X) + ffk(x)m(dx) ffg(x)m(dx) = JAL(f; x) - ffx (dx) lim r f'g(x)m(dx) k-*+od I(f -fk; X)j. Since for any m-summable h(x), fj,(h; x)g(x)m(dx) = fh(x)m(dx), it follows that lim fij(f - fk; x)jg(x)m(dx) < lim fs(if -fkl ; x)g(x)m(dx) = k--+o k-oc lim f If-fklm(dx) = 0, and since g(x) > 0 the desired result follows. 5. Applications.-In what follows h(u) and k(u) will be 7r-summable functions of a real variable u with fk(u)ir(du) d 0. Theorems 1 and 2 in conjunction with Hopf's theorem then imply THEOREM 3.8 Under Assumption 2, for a.e. (r)xo, Prob lim h(xo) h(x.) _ f2z h(u)t(du)( Wn-- o k(xo) k(xn) f2). k(u)ir(du) COROLLARY. Under Assumption 2, if ri is another admissible solu.tion of (1) such that i(b) = 0 implies 7ri(B) = 0, thenfor some constant c > 0, ti = CTr. Next, consider a sequence of independent random variables Yi, Y2, with a common distribution function F(y). ASSUMPTION 3. The sums y, Y. are "interval-recurrent"; i.e., for any interval I, Prob (yi yn e I i.o.) = 1 (see ref. 1). We state without proof LEMMA 1. Under Assumption 3, if B e Fo has positive Lebesgue measure then for a.e. a, Prob(a + y, Yn, e B i.o.) = 1. This is true for every a if some convolution FP*(y) has an absolutely continuous component. Set H(u, B) = fb df(y - u); then Tr = Lebesgue measure satisfies (1), and from Theorem 3 and Lemma 1 we obtain THEOREM 4. Under Assumption 3, for a.e. (Lebesgue measure) a, E h(a + y1 + +yj) _ ff,h(u)du - 1 (4) Prob lim i = fi", k(u)du (A l k(a + yi +. + yj)

5 864 MATHEMATICS: HARRIS AND ROBBINS PROC. N. A. S. If some convolution Fn*(y) has an absolutely continuous component then (4) holds for every a. From Theorem 4 we can deduce the following COROLLARY. Under Assumption 3 (even if no Fn*(y) has an absolutely continuous component) for every a the limit relation in (4) holds with probability I simultaneously for every two functions h(u), k(u) which are Riemannintegrable in a finite interval and vanish elsewhere, the integral of k(u) assumed to be $0. Thus the prob 1 density of the sequence of sums yi yn implies the prob 1 equidistribution of these sums. This establishes a conjecture made in ref. 4. If the yi have only integer values then the function 7r(B) = number of integer points in B, is an admissible solution of (1). Theorem 3 then implies that under Assumption 2 (which holds whenever every integer v is a "recurrent" value of the sums y, yn), for every two functions of integers h(v), k(v) such that co co co E Ih(v)I < co, E k(v) < co, Ek(v) $ 0, -co -am -od n co ) E h(yl yi) E h(v) Prob lim % -, 1. (5) j=1 -c nl co E k(y, + yj) E k(vw) j=l -amo * Work sponsored by the Office of Scientific Research of the Air Force. t On leave from the RAND Corporation. 1 Chung, K. L, and Fuchs, W. H. J., "On the Distribution of Values of Sums of Random Variables," Mem. Am. Math. Soc., 6, (1951). 2 Doob, J. L., Stochastic Processes, Wiley & Son, New York, 1953, Chap Hopf, E., Ergodentheorie, reprinted by Chelsea, New York, 1948, pp Kallianpur, G., and Robbins, H., "On the Sequence of Partial Sums of Independent Random Variables," to appear. 6 Kallianpur, G., and Robbins, H., "Ergodic Property of the Brownian Motion Process"; to appear in PROC. NATL. ACAD. SCI. 6 Kolmogorov, A., Grundbegriffe der Warscheinlichkeitsrechnung, reprinted by Chelsea, New York, 1946, pp Robbins, H., "On the Equidistribution of Sums of Independent Random Variables"; to appear in Proc. Am. Math. Soc. 8 Chung has obtained this result independently and by a different method for the case in which the Xi have only a denumerable set of values. His results will appear in Trans. Am. Math. Soc.