Non-homogeneous random walks on a semi-infinite strip

Similar documents
Markov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued

Stability of the two queue system

STOCHASTIC PROCESSES Basic notions

8. Statistical Equilibrium and Classification of States: Discrete Time Markov Chains

Advanced Computer Networks Lecture 2. Markov Processes

Lecture 11: Introduction to Markov Chains. Copyright G. Caire (Sample Lectures) 321

Examples of Countable State Markov Chains Thursday, October 16, :12 PM

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

MATH 564/STAT 555 Applied Stochastic Processes Homework 2, September 18, 2015 Due September 30, 2015

Asymptotic behaviour of randomly reflecting billiards in unbounded tubular domains arxiv: v2 [math.pr] 12 May 2008

Stochastic Models: Markov Chains and their Generalizations

Lecture 7. µ(x)f(x). When µ is a probability measure, we say µ is a stationary distribution.

STA 624 Practice Exam 2 Applied Stochastic Processes Spring, 2008

Markov Chains CK eqns Classes Hitting times Rec./trans. Strong Markov Stat. distr. Reversibility * Markov Chains

P i [B k ] = lim. n=1 p(n) ii <. n=1. V i :=

Adventures in Stochastic Processes

= P{X 0. = i} (1) If the MC has stationary transition probabilities then, = i} = P{X n+1

Markov Chains (Part 3)

Statistics 150: Spring 2007

Chapter 7. Markov chain background. 7.1 Finite state space

Note that in the example in Lecture 1, the state Home is recurrent (and even absorbing), but all other states are transient. f ii (n) f ii = n=1 < +

Chapter 2: Markov Chains and Queues in Discrete Time

Lecture 10. Theorem 1.1 [Ergodicity and extremality] A probability measure µ on (Ω, F) is ergodic for T if and only if it is an extremal point in M.

Markov processes and queueing networks

Classification of Countable State Markov Chains

Lectures on Markov Chains

2. Transience and Recurrence

Model reversibility of a two dimensional reflecting random walk and its application to queueing network

MARKOV PROCESSES. Valerio Di Valerio

INTRODUCTION TO MARKOV CHAINS AND MARKOV CHAIN MIXING

MATH 56A: STOCHASTIC PROCESSES CHAPTER 1

Winter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo

Discrete time Markov chains. Discrete Time Markov Chains, Limiting. Limiting Distribution and Classification. Regular Transition Probability Matrices

A review of Continuous Time MC STA 624, Spring 2015

2 Discrete-Time Markov Chains

Lecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past.

Countable state discrete time Markov Chains

An Introduction to Entropy and Subshifts of. Finite Type

MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10

Transience: Whereas a finite closed communication class must be recurrent, an infinite closed communication class can be transient:

T. Liggett Mathematics 171 Final Exam June 8, 2011

DISCRETE STOCHASTIC PROCESSES Draft of 2nd Edition

Math Homework 5 Solutions

Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of. F s F t

STAT STOCHASTIC PROCESSES. Contents

Lecture 20 : Markov Chains

Ergodic Theorems. Samy Tindel. Purdue University. Probability Theory 2 - MA 539. Taken from Probability: Theory and examples by R.

FUNCTIONAL CENTRAL LIMIT (RWRE)

Asymptotic properties of the maximum likelihood estimator for a ballistic random walk in a random environment

Recap. Probability, stochastic processes, Markov chains. ELEC-C7210 Modeling and analysis of communication networks

Stability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk

Matrix analytic methods. Lecture 1: Structured Markov chains and their stationary distribution

SMSTC (2007/08) Probability.

Exact Simulation of the Stationary Distribution of M/G/c Queues

The Distribution of Mixing Times in Markov Chains

Analysis of the Dynamic Travelling Salesman Problem with Different Policies

Existence, Uniqueness and Stability of Invariant Distributions in Continuous-Time Stochastic Models

IEOR 6711, HMWK 5, Professor Sigman

Lecture 7. We can regard (p(i, j)) as defining a (maybe infinite) matrix P. Then a basic fact is

Quantitative Model Checking (QMC) - SS12

Stochastic process. X, a series of random variables indexed by t

On the Borel-Cantelli Lemma

P(X 0 = j 0,... X nk = j k )

Markov Chains. Arnoldo Frigessi Bernd Heidergott November 4, 2015

Lecture 9 Classification of States

1 Continuous-time chains, finite state space

Birth-death chain models (countable state)

Irreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1

NEW FRONTIERS IN APPLIED PROBABILITY

CDA6530: Performance Models of Computers and Networks. Chapter 3: Review of Practical Stochastic Processes

12 Markov chains The Markov property

Applied Stochastic Processes

Continuous time Markov chains

Stochastic process for macro

6 Markov Chain Monte Carlo (MCMC)

Markov Chains, Stochastic Processes, and Matrix Decompositions

On optimal stopping of autoregressive sequences

Markovian Description of Irreversible Processes and the Time Randomization (*).

Theory and Applications of Stochastic Systems Lecture Exponential Martingale for Random Walk

Homework set 3 - Solutions

Markov chains. 1 Discrete time Markov chains. c A. J. Ganesh, University of Bristol, 2015

MATH 56A SPRING 2008 STOCHASTIC PROCESSES 65

Stochastic Processes MIT, fall 2011 Day by day lecture outline and weekly homeworks. A) Lecture Outline Suggested reading

Stochastic Processes

SIMILAR MARKOV CHAINS

Two-sided Bounds for the Convergence Rate of Markov Chains

The Theory behind PageRank

Lecture 20: Reversible Processes and Queues

Markov Chains and Stochastic Sampling

An Introduction to Probability Theory and Its Applications

Necessary and sufficient conditions for strong R-positivity

Brownian Motion and the Dirichlet Problem

On the range of a two-dimensional conditioned simple random walk

TCOM 501: Networking Theory & Fundamentals. Lecture 6 February 19, 2003 Prof. Yannis A. Korilis

Stochastic Simulation of Processes, Fields and Structures

ECE-517: Reinforcement Learning in Artificial Intelligence. Lecture 4: Discrete-Time Markov Chains

On Optimal Stopping Problems with Power Function of Lévy Processes

Markov Chains. October 5, Stoch. Systems Analysis Markov chains 1

Treball final de grau GRAU DE MATEMÀTIQUES Facultat de Matemàtiques Universitat de Barcelona MARKOV CHAINS

On the Central Limit Theorem for an ergodic Markov chain

Transcription:

Non-homogeneous random walks on a semi-infinite strip Chak Hei Lo Joint work with Andrew R. Wade World Congress in Probability and Statistics 11th July, 2016

Outline Motivation: Lamperti s problem Our Model Non-homogeneous RW on semi-infinite strip Classification of the random walk Assumptions Main results Constant drift Lamperti drift Generalized Lamperti drift Examples: Correlated random walks

Motivation: Lamperti s problem Let X n be a nearest neighbour random walk on Z +.

Motivation: Lamperti s problem Let X n be a nearest neighbour random walk on Z +. Denote the mean drift at x by µ(x).

Motivation: Lamperti s problem Let X n be a nearest neighbour random walk on Z +. Denote the mean drift at x by µ(x). Lamperti s problem: µ(x) = O(1/x) when x.

Non-homogeneous RW on semi-infinite strip Let S be a finite non-empty set.

Non-homogeneous RW on semi-infinite strip Let S be a finite non-empty set. Let Σ be a subset of R + S that is locally finite, i.e., Σ ([0, r] S) is finite for all r R +.

Non-homogeneous RW on semi-infinite strip Let S be a finite non-empty set. Let Σ be a subset of R + S that is locally finite, i.e., Σ ([0, r] S) is finite for all r R +. E.g. Σ = Z + S.

Non-homogeneous RW on semi-infinite strip Let S be a finite non-empty set. Let Σ be a subset of R + S that is locally finite, i.e., Σ ([0, r] S) is finite for all r R +. E.g. Σ = Z + S. Define for each k S the line Λ k := {x R + : (x, k) Σ}.

Non-homogeneous RW on semi-infinite strip Let S be a finite non-empty set. Let Σ be a subset of R + S that is locally finite, i.e., Σ ([0, r] S) is finite for all r R +. E.g. Σ = Z + S. Define for each k S the line Λ k := {x R + : (x, k) Σ}. Suppose that for each k S the line Λ k is unbounded.

Non-homogeneous RW on semi-infinite strip Suppose that (X n, η n ), n Z +, is a time-homogeneous, irreducible Markov chain on Σ, a locally finite subset of R + S.

Non-homogeneous RW on semi-infinite strip Suppose that (X n, η n ), n Z +, is a time-homogeneous, irreducible Markov chain on Σ, a locally finite subset of R + S. Neither coordinate is assumed to be Markov.

Motivating examples We can view S as a set of internal states, influencing motion on the lines R +. E.g., Operations research: modulated queues (S = states of server)

Motivating examples We can view S as a set of internal states, influencing motion on the lines R +. E.g., Operations research: modulated queues (S = states of server) Economics: regime-switching processes (S contains market information)

Motivating examples We can view S as a set of internal states, influencing motion on the lines R +. E.g., Operations research: modulated queues (S = states of server) Economics: regime-switching processes (S contains market information) Physics: physical processes with internal degrees of freedom (S = energy/momentum states of particle)

Classification of the random walk Recall (X n, η n ) is a time-homogeneous irreducible Markov chain on the state-space Σ R + S. (i) If (X n, η n ) is transient, then X n a.s. (ii) If (X n, η n ) is recurrent, then P(X n A i.o.) = 1 for any bounded region A. (iii) Define τ = min{n 0 : X n A}. If (X n, η n ) is positive-recurrent, then E[τ] < for any bounded region A. (iv) If (X n, η n ) is recurrent but not positive recurrent, then we call it null-recurrent.

Assumptions Moments bound on jumps of X n (B p ) C p < s.t. E[ X n+1 X n p (X n, η n ) = (x, i)] C p

Assumptions Moments bound on jumps of X n (B p ) C p < s.t. E[ X n+1 X n p (X n, η n ) = (x, i)] C p Notation for moments of the displacements in the X -coordinate µ i (x) = E[X n+1 X n (X n, η n ) = (x, i)] σ i (x) = E[(X n+1 X n ) 2 (X n, η n ) = (x, i)]

Assumptions (cont.) η n is close to being Markov when X n is large Define q ij (x) = P[η n+1 = j (X n, η n ) = (x, i)] (Q ) q ij = lim x q ij (x) exists for all i, j S and (q ij ) is irreducible

Assumptions (cont.) η n is close to being Markov when X n is large Define q ij (x) = P[η n+1 = j (X n, η n ) = (x, i)] (Q ) q ij = lim x q ij (x) exists for all i, j S and (q ij ) is irreducible Let π be the unique stationary distribution on S corresponding to (q ij ).

Constant drift Constant-type moment condition (M C ) d i R for all i S such that µ i (x) = d i + o(1).

Constant drift

Constant drift

Constant drift

Constant drift

Recurrence classification for constant drift The following theorem is from Georgiou, Wade (2014), extending slightly earlier work of Malyshev (1972), Falin (1988), and Fayolle et al (1995). Theorem Suppose that (B p ) holds for some p > 1 and conditions (Q ) and (M C ) hold. The following sufficient conditions apply. If i S d iπ i > 0, then (X n, η n ) is transient. If i S d iπ i < 0, then (X n, η n ) is positive-recurrent. where π i is the unique stationary distribution on S. What about i S d iπ i = 0?

Different drifts (i) i S d iπ i 0, constant drift: µ i (x) = d i + o(1) (ii) i S d iπ i = 0 and d i = 0 for all i, Lamperti drift: µ i (x) = c i x + o(x 1 ) σ i (x) = s 2 i + o(1) (iii) i S d iπ i = 0 and d i 0 for some i, generalized Lamperti drift: µ i (x) = d i + c i x + o(x 1 ) σ i (x) = si 2 + o(1)

Lamperti drift Lamperti-type moment conditions (M L ) c i, s i R for all i S (at least one s i nonzero) such that µ i (x) = c i x + o(x 1 ); σ i (x) = s 2 i + o(1).

Lamperti drift Lamperti-type moment conditions (M L ) c i, s i R for all i S (at least one s i nonzero) such that µ i (x) = c i x + o(x 1 ); σ i (x) = s 2 i + o(1). When S is a singleton, this reduces to the classical Lamperti problem on R +.

Lamperti drift

Lamperti drift

Lamperti drift

Lamperti drift

Recurrence classification for Lamperti drift Theorem (Georgiou, Wade, 2014) Suppose that (B p ) holds for some p > 2 and conditions (Q ) and (M L ) hold. The following sufficient conditions apply. (i) If i S (2c i s 2 i )π i > 0, then (X n, η n ) is transient. (ii) If i S 2c iπ i < i S s2 i π i, then (X n, η n ) is null-recurrent. (iii) If i S (2c i + s 2 i )π i < 0, then (X n, η n ) is positive-recurrent. [With better error bounds in (Q ) and (M L ) we can also show that the boundary cases are null-recurrent.]

Idea of proof of the theorem Our general analysis is based on the Lyapunov function f ν : Σ R defined for ν R by where b i R. f ν (x, i) := x ν + ν 2 b ix ν 2

Idea of proof of the theorem Our general analysis is based on the Lyapunov function f ν : Σ R defined for ν R by where b i R. f ν (x, i) := x ν + ν 2 b ix ν 2 For appropriate choices of ν, and selecting the right b i depending on the drift and the stationary distribution, we can show that f ν (X n, η n ) is a supermartingale and so we can apply some semi-martingale theorem. Hence we obtain the last theorem shown.

Generalized Lamperti drift Generalized Lamperti type moment conditions Define µ ij (x) = E[(X n+1 X n )1{η n+1 = j} (X n, η n ) = (x, i)] (M CL ) There exist d i R, c i R, d ij R and si 2 R +, with at least one si 2 non-zero, such that all of the following is satisfied,

Generalized Lamperti drift Generalized Lamperti type moment conditions Define µ ij (x) = E[(X n+1 X n )1{η n+1 = j} (X n, η n ) = (x, i)] (M CL ) There exist d i R, c i R, d ij R and si 2 R +, with at least one si 2 non-zero, such that all of the following is satisfied, (i) For all i S, µ i (x) = d i + c i x + o(x 1 ) as x,

Generalized Lamperti drift Generalized Lamperti type moment conditions Define µ ij (x) = E[(X n+1 X n )1{η n+1 = j} (X n, η n ) = (x, i)] (M CL ) There exist d i R, c i R, d ij R and si 2 R +, with at least one si 2 non-zero, such that all of the following is satisfied, (i) For all i S, µ i (x) = d i + c i x + o(x 1 ) as x, (ii) For all i, j S, µ ij (x) = d ij + o(1) as x,

Generalized Lamperti drift Generalized Lamperti type moment conditions Define µ ij (x) = E[(X n+1 X n )1{η n+1 = j} (X n, η n ) = (x, i)] (M CL ) There exist d i R, c i R, d ij R and si 2 R +, with at least one si 2 non-zero, such that all of the following is satisfied, (i) For all i S, µ i (x) = d i + c i x + o(x 1 ) as x, (ii) For all i, j S, µ ij (x) = d ij + o(1) as x, (iii) σ i (x) = si 2 + o(1),

Generalized Lamperti drift Generalized Lamperti type moment conditions Define µ ij (x) = E[(X n+1 X n )1{η n+1 = j} (X n, η n ) = (x, i)] (M CL ) There exist d i R, c i R, d ij R and si 2 R +, with at least one si 2 non-zero, such that all of the following is satisfied, (i) For all i S, µ i (x) = d i + c i x + o(x 1 ) as x, (ii) For all i, j S, µ ij (x) = d ij + o(1) as x, (iii) σ i (x) = si 2 + o(1), (iv) i S π id i = 0.

Generalized Lamperti drift Generalized Lamperti type moment conditions Define µ ij (x) = E[(X n+1 X n )1{η n+1 = j} (X n, η n ) = (x, i)] (M CL ) There exist d i R, c i R, d ij R and si 2 R +, with at least one si 2 non-zero, such that all of the following is satisfied, (i) For all i S, µ i (x) = d i + c i x + o(x 1 ) as x, (ii) For all i, j S, µ ij (x) = d ij + o(1) as x, (iii) σ i (x) = si 2 + o(1), (iv) i S π id i = 0. Transition probability condition (Q CL ) There exist γ ij R, such that q ij (x) = q ij + γ ij x + o(x 1 )

Generalized Lamperti drift

Generalized Lamperti drift

Recurrence classification for Generalized Lamperti drift Theorem (L., Wade, 2015) Suppose that (B p ) holds for some p > 2, and conditions (Q ), (Q CL ) and (M CL ) hold. Define a i to be the unique solution up to translation of the system of equations d i + j S (a j a i )q ij = 0 i S. The following sufficient conditions apply. If i S [2c i si 2 + 2 j S a j(γ ij d ij )]π i > 0 then (X n, η n ) is transient. If i S (2c i + 2 j S a jγ ij )π i < i S (s2 i + 2 j S a jd ij )π i then (X n, η n ) is null-recurrent. If i S [2c i + si 2 + 2 j S a j(γ ij + d ij )]π i < 0 then (X n, η n ) is positive-recurrent.

Idea of proof of the theorem Transform the process (X n, η n ) with generalized Lamperti drift to a process ( X n, η n ) = (X n + a ηn, η n )

Idea of proof of the theorem Transform the process (X n, η n ) with generalized Lamperti drift to a process ( X n, η n ) = (X n + a ηn, η n ) Find the appropriate choices of the real numbers a i, i S, such that ( X n, η n ) has Lamperti drift i.e., the constant components of the drifts are eliminated

Idea of proof of the theorem Transform the process (X n, η n ) with generalized Lamperti drift to a process ( X n, η n ) = (X n + a ηn, η n ) Find the appropriate choices of the real numbers a i, i S, such that ( X n, η n ) has Lamperti drift i.e., the constant components of the drifts are eliminated Calculate the new increment moment estimates for the transformed process ( X n, η n ).

Idea of proof of the theorem Transform the process (X n, η n ) with generalized Lamperti drift to a process ( X n, η n ) = (X n + a ηn, η n ) Find the appropriate choices of the real numbers a i, i S, such that ( X n, η n ) has Lamperti drift i.e., the constant components of the drifts are eliminated Calculate the new increment moment estimates for the transformed process ( X n, η n ). Apply the results in the Lamperti drift case

Summary of cases

Summary of cases

Example: One-step Correlated random walk

Simulation results on One-step Correlated RW

Simulation results on One-step Correlated RW

Example: Two-steps Correlated random walk

Simulation results on Two-steps Correlated RW

Simulation results on Two-step Correlated RW

Simulation results on Two-steps Correlated RW

Existence and Non-existence of Moments results We also obtained some quantitative information on the nature of recurrence. We studied moments of passage times.

Existence and Non-existence of Moments results We also obtained some quantitative information on the nature of recurrence. We studied moments of passage times. For x R +, define the stopping time τ x := min{n 0 : X n x}.

Existence and Non-existence of Moments results We also obtained some quantitative information on the nature of recurrence. We studied moments of passage times. For x R +, define the stopping time τ x := min{n 0 : X n x}. We got results that gives conditions for which s such that E[τ s x ] exists or not exists.

Existence and Non-existence of Moments results We also obtained some quantitative information on the nature of recurrence. We studied moments of passage times. For x R +, define the stopping time τ x := min{n 0 : X n x}. We got results that gives conditions for which s such that E[τ s x ] exists or not exists. We used similar techniques and ideas in our proofs.

References L., A. R. Wade, Non-homogeneous random walks on a half strip with generalized Lamperti drifts, http://arxiv.org/abs/1512.04242 G. I. Falin, Ergodicity of random walks in the half-strip, Math. Notes 44 (1988) 606 608; translated from Mat. Zametki 44 (1988) 225 230 [in Russian]. G. Fayolle, V. A. Malyshev, and M. V. Menshikov, Topics in the Constructive Theory of Countable Markov Chains, Cambridge University Press, Cambridge, 1995. N. Georgiou and A. R. Wade, Non-homogeneous random walks on a semi-infinite strip, Stoch. Process. Appl. 124 (2014) 3179 3205. J. Lamperti, Criteria for the recurrence and transience of stochastic processes I, J. Math. Anal. Appl. 1 (1960) 314 330. V. A. Malyshev, Homogeneous random walks on the product of finite set and a halfline, pp. 5 13 in Veroyatnostnye Metody Issledovania (Probability Methods of Investigation) 41 [in Russian], ed. A.N. Kolmogorov, Moscow State University, Moscow, 1972.

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback