On Existence of Limiting Distribution for Time-Nonhomogeneous Countable Markov Process
|
|
- Prosper Gardner
- 6 years ago
- Views:
Transcription
1 Queueing Systems 46, , Kluwer Academic Publishers Manufactured in The Netherlands On Existence of Limiting Distribution for Time-Nonhomogeneous Countable Markov Process V ABRAMOV vyachesl@zahavnetil Department of Mathematics, The Faculty of Exact Sciences, Tel Aviv University, Tel Aviv, and College of Judea and Samaria, Ariel, Israel R LIPTSER liptser@engtauacil Department of Electrical Engineering-Systems, Tel Aviv University, Tel Aviv, Israel Received 15 October 22; Revised 1 March 23 Abstract In this paper, sufficient conditions are given for the existence of iting distribution of a nonhomogeneous countable Markov chain with time-dependent transition intensity matrix The method of proof exploits the fact that if the distribution of random process Q = (Q t t is absolutely continuous with respect to the distribution of ergodic random process Q = (Q t t,then law Q t π, where π is the invariant measure of Q We apply this result for asymptotic analysis, as t,ofa nonhomogeneous countable Markov chain which shares iting distribution with an ergodic birth-anddeath process Keywords: countable Markov process, existence of the iting distribution, birth-and-death process 1 Introduction There is a large number of papers in the queueing literature devoted to analysis of state dependent and time dependent queueing systems as M t /M t /1andM t /M t /c and associated with their Markovian queueing networks (eg, see [23 27] An analysis of such queueing system is motivated by a wide spectra of practical problems well-known in the literature For instance, a simple example corresponding to the police dispatching problem is given in [26, p 6] (see also [15] for further discussion The M t /M t / queue, used for a model of emergency ambulances and intensive care units, is considered in [6] Other applications are known for client/server computer networks, when arrival and service rates of nodes depend on amount of unfinished work and number of available tasks on server (see, eg, [2,3,19,2] Typically, an asymptotic analysis of M t /M t /1andM t /M t /c uses differential equations for transition probabilities and asymptotic analysis, as t, for their solutions This type of analysis is similar to an investigation of stability for Markov
2 354 ABRAMOV AND LIPTSER chain and is associated with a verification of stationarity (quasi-stationarity (see [1,4,5, 7 1,21,29] and many others We mention now results related to time-nonhomogeneous stochastic models converging, as t, to time-homogeneous ones Although first results were published more than 3 years ago by Gnedenko and Soloviev [12] and Gnedenko [11], a remarkable progress was achieved not a long time ago (see [13,14,16,23,3 36] In particular, Zeifman developed a number of effective tools, permitting investigate successfully ergodicity conditions for special classes of time-nonhomogeneous birth-and-death processes including M t /M t /1, M t /M t /S and M t /M t /S/ queues (for further discussion, see [14] In the present paper, we give sufficient conditions, under which a time-nonhomogeneous countable Markov chain with the transition intensity matrix (t shares the iting distribution with time-homogeneous ergodic Markov chain with the transition intensity matrix Our setting is heavily related to the above-mentioned settings and the result obtained supplements the results from [14,31,32,35,36] (more detailed comparison is given in section 6 The main difference with known approaches to this problem is that the convergence (t, t is not required In contrast to that we assume the existence of nonnegative λ ij,i j such that (here λ ij (t are entries of (t λ ij <, j i t λ ij (si (λ ij > ds = ( λij (t λ ij 2 dt <, t λ ij (s ds, t >, (11 and create the matrix with entries λ ij,i j and λ ii = j i λ ij We assume that the Markov chain with this transition intensity matrix is ergodic To explain our method with more details, notice that (11 guarantees the absolute continuity of the distribution for (t-markov chain with respect to the distribution for -Markov chain It is also assumed that -Markov chain is ergodic but the geometrical ergodicity is not required We show in theorem 21 (section 2 that the above-mentioned absolute continuity of distributions provides the iting distribution, as t, for (t-markov chain coinciding with the invariant measure of -Markov chain In section 3, we give the proof of theorem 21 In section 4, we show that not only the iting distribution but also other iting functionals are the same as for -Markov chain In section 5, an asymptotic equivalence of (t-markov chain to an ergodic birthand-death process is established
3 LIMITING DISTRIBUTION FOR TIME-NONHOMOGENEOUS MARKOV PROCESS The main result We consider a nonhomogeneous Markov chain Q = (Q t t with the countable set of states S = {, 1,} and the transition intensity matrix (t with entries λ ij (t Suppose that for any pair (i, j with i j there is a nonnegative constant λ ij such that (11 holds true, and introduce Markov chain Q = (Q t t with the set of its states S, and Q = Q, and the transition intensity matrix (see section 1 Our main assumption is that Q is ergodic, ie there is the unique probabilistic measure π on S such that π = and Q t = j Q s = i = π j, s,i,j, (21 where π j are entries of π Theorem 21 Under (11 and (21, 3 The proof of theorem Preinaries P(Q t = j Q s = i = π j, s,i,j (22 Without loss of generality one may assume that the Markov chains Q and Q have paths in the Skorokhod space D = D [, of right continuous having its to the left functions x = (x t t Let ν, ν be the distributions of Q and Q respectively, that is ν, ν are probabilistic measures on (D, G,whereG is the Borel σ -algebra on D Without loss of generality we may assume that G is completed with respect to the measure (ν + ν /2 We shall use in the sequel that ν ν Recall (see, eg, [28] that ν ν provides that ν(a = for any A G, ifν (A = For the verification of ν ν, we apply [17, theorem 24] Following this theorem, ν ν if for all i, j S (a ν (x = i = ν(x = i = ; (b t I(x s = jλ ij (s ds = t I(x s = jλ ij (si (λ ij ds, ν-as; (c with = ( [ P 1 j i ] 2 λ ij (t I(λ ij λ ij I(x t = idt < = 1, ν-as λ ij Notice that for any j i, (c is provided by the condition [ ] 2 λ ij (t 1 I(λ ij λ ij dt < λ ij equivalent to the first part in (11
4 356 ABRAMOV AND LIPTSER Introduce a stochastic basis (D, G,(G t t,ν with the general condition (see, eg, [22], where (G t t is the filtration generated by x Henceforth, E ν and E ν denote the expectations with respect to ν and ν, respectively Set Z(x = dν/dν (x and Z t (x = E ν (Z G t (x We shall use the fact that (Z t (x t is positive uniformly integrable martingale with respect to ν Throughout the paper we use the notation ( for minimum (maximum of two numbers 32 Auxiliary lemma Lemma 31 Under the assumptions of theorem 21, for any s andj S P(Q t = j Q s prob π j Proof With s<s <t, using Markov property, write P(Q t = j Q s = ν(x t = j G s = E ν( ν(x t = j G s x s According to well known formula for the conditional expectation under absolute continuous change of measure: for any integrable random variable α, E ν (α G s = E ν ((Z/Z s α G s we find ( Z(x ν(x t = j G s = E ν Z s (x I(x t = j G s = ν (x t = j x s + E ν ([ Z(x Z s (x 1 ] I(x t = j G s = j x s ν as π j and by ν ν the same convergence holds By (21, ν (x t ν-as too well So, it remains to show that ([ ] Z(x Eν Z s (x 1 I(x t = j G s Notice that ([ ] Z(x E ν E ν Z s (x 1 Z(x E ν Z s (x 1 ( Z(x E ν Z s (x 1 3 ( Z(x = E ν Z s (x 1 3 E ν ( Z(x Z s (x + 1 I I(x t = j G s ( Z(x + E ν Z s (x + 1 I ( Z(x + E ν Z s (x + 1 ( Z(x Z s (x 2 ν prob (31 s ( Z(x Z s (x > 2
5 LIMITING DISTRIBUTION FOR TIME-NONHOMOGENEOUS MARKOV PROCESS 357 As was mentioned above Z t (x is the positive uniformly integrable ν -martingale Hence s Z s (x = Z(x, ν -as Consequently, by Lebesgue dominated theorem ( Z(x s Eν Z s (x 1 3 = (32 Now, it remains to show that ( ( ( Z(x Z(x Z(x s Eν Z s (x + 1 = s Eν Z s (x + 1 I Z s (x 2 Since Z s (x Z(x,s, by the Lebesgue dominated theorem the right hand side of the above equality is equal to 2 At the same time for any s we have E ((Z(x/Z s (x G s = 1andsoforanys it holds E (Z(x/Z s (x + 1 = 2 Thus, ([ ] Z(x Eν Z s (x 1 ν I(x t = j G prob s s 33 Final part of proof By lemma 31 we have P(Q t = j Q s = ii(q s = i prob π j Hence, for any i S j= P(Q t = j Q s = ii(q s = i prob ji(q s = i (33 and the statement of theorem 21 follows 4 Asymptotic equivalence for other functionals Denote h(x t = I(x t = j Theorem 21 guarantees the asymptotic equivalence Eν( ( h(x t G s = E ν h(xt G s An analysis of the proof of theorem 21 shows that the same type of asymptotic equivalence holds for any bounded functional h(x [t, of argument x [t, ={x u,u t} provided that E ν (h(x [t, G s exists, that is under the assumptions of theorem 21 Eν( h(x [t, G s = E ν ( h(x[t, G s (41
6 358 ABRAMOV AND LIPTSER 5 Asymptotic equivalence to birth-and-death process Let λ (t λ (t µ 1 (t (λ 1 (t + µ 1 (t λ 1 (t (t = µ 2 (t (λ 2 (t + µ 2 (t λ 2 (t µ 3 (t (λ 3 (t + µ 3 (t and Assume that there exist positive numbers λ j,µ j s such that n n=1 j=1 λ j 1 µ j < (51 [ ( λj (t λ j 2 + ( µj (t µ j 2 ] dt <, j (52 We mention here that (52 does not provide λ j (t λ j,µ j (t µ j, Introduce the matrix λ λ µ 1 (λ 1 + µ 1 λ 1 = µ 2 (λ 2 + µ 2 λ 2 µ 3 (λ 3 + µ 3 and notice that Markov chain with the transition intensity matrix generates the birthand-death process It is well known (see, eg, [18, chapter 7, section 5] that under (51 the birth-anddeath process is ergodic with the unique stationary distribution on S: π = 1 + n=1 1 n n j=1 (λ j 1/µ j, π n = π j=1 λ j 1 µ j, n = 1, 2, (53 By theorem 21, the Markov chain with transition intensity matrix (t possesses the same stationary distribution It is known that the sojourn time T i in state i for the birth-and-death process is exponentially distributed P(T i x = 1 e (λ i+µ i x Under (51 and (52, for Markov chain with the transition intensity (t we have the following Let T i (t = v i (t t, where v i (t = inf{s t: Q s i, Q t = i}
7 LIMITING DISTRIBUTION FOR TIME-NONHOMOGENEOUS MARKOV PROCESS 359 Then, applying the result from section 4, we obtain P ( T i (t x = 1 e (λ i+µ i x 6 Discussion We consider here M t /M t /1 model Let A t and D t be independent and timenonhomogeneous Poisson processes with positive rates λ(t and µ(t respectively Let Q be a random variable, independent of A t and D t, taking values in S ={, 1,} We define the queue-length process Q = (Q t t in the M t /M t /1 as follows (notice that jumps of A t and D t are disjoint and so Q t S Q t = Q + A t t I(Q s > dd s Let A t and Dt be independent and homogeneous Poisson processes with positive rates λ and µ, letq = Q be independent of A t and Dt,andletQ = (Q t t be the queue-length process in the M/M/1 queue with parameters λ and µ for arrival and service of customers, respectively, The queue-length process Q t is defined as by Q t = Q + A t λ<µ t I ( Q s > dd s By theorem 21 the existence of the iting distribution for M t /M t /1 is provided [ ( λ λ(t 2 + ( µ µ(t 2 ] dt < (61 On the other hand, it is known from [14,31,32,35] that the existence of the iting distribution is provided by [ λ λ(t + µ µ(t ] = (62 Generally, (61 (62, and so (61 and (62 supplement each other If λ(t and µ(t are uniformly continuous on [, functions, then (61 (62, that is (62 is weaker than (61 Notice also that (62 (61, say, under additional condition: for small positive ε and t large enough λ λ(t + µ µ(t =O(t (1/2+ε It would be noted that in [36], it is studied an ergodicity problem, in the uniform operator topology, of time-nonhomogeneous Markov chains with transition intensity ma-
8 36 ABRAMOV AND LIPTSER trices possessing summable perturbations These results have some connection with our one too well Acknowledgement The authors indebt Prof Granovsky and the anonymous referee providing them related topics and drawing attention on some flaws References [1] VM Abramov, On the asymptotic distribution of the maximum number of infectives in epidemic models with immigrations, J Appl Probab 31 ( [2] VM Abramov, A large closed queueing network with autonomous service and bottleneck, Queueing Systems 35(1 3 ( [3] VM Abramov, Some results for large closed queueing networks with and without bottleneck: Upand down-crossings approach, Queueing Systems 38(2 ( [4] S Asmussen and H Thorisson, A Markov chain approach to periodic queues, J Appl Probab 24 ( [5] N Bambos and J Walrand, On stability of state-dependent queues and acyclic queueing networks, Adv in Appl Probab 21 ( [6] T Collings and C Stoneman, The M/M/ queue with varying arrival and departure rates, Oper Res 24 ( [7] E Gelenbe and D Finkel, Stationary deterministic flows: The single server queue, Theoret Comput Sci 52 ( [8] II Geterontidis, On certain aspects of homogeneous Markov systems in continuous time, J Appl Probab 27 ( [9] II Geterontidis, Periodic strong ergodicity in non-homogeneous Markov systems, J Appl Probab 28 ( [1] II Geterontidis, Cyclic strong ergodicity in non-homogeneous Markov systems, SIAM J Matrix Anal Appl 13 ( [11] DB Gnedenko, On a generalization of Erlang formulae, Zastos Matem 12 ( [12] BV Gnedenko and AD Soloviev, On the conditions of the existence of final probabilities for a Markov process, Math Operationsforsh Statist 4 ( [13] BL Granovsky and AI Zeifman, The decay function of nonhomogeneous birth-and-death process, with application to mean-field model, Stochastic Process Appl 72 ( [14] BL Granovsky and AI Zeifman, Nonstationary Markovian queues, J Math Sci 99(4 ( [15] L Green and P Kolesar, Testing a validity of the queueing model of a police patrol, Managm Sci 35 ( [16] J Johnson and D Isaacson, Conditions for strong ergodicity using intensity matrices, J Appl Probab 25 ( [17] YuM Kabanov, RSh Liptser and AN Shiryaev, Absolutely continuity and singularity of locally absolutely continuous probability distributions II, Math USSR Sbornik 36(1 ( [18] S Karlin, A First Course in Stochastic Processes (Academic Press, New York/London, 1968 [19] C Knessl, B Matkovsky, Z Schuss and C Tier, Asymptotic analysis of a state-dependent M/G/1 queueing system, SIAM J Appl Math 46(3 ( [2] Y Kogan and RSh Liptser, Limit non-stationary behavior of large closed queueing networks with bottlenecks, Queueing Systems 14 (
9 LIMITING DISTRIBUTION FOR TIME-NONHOMOGENEOUS MARKOV PROCESS 361 [21] AJ Lemoine, On queues with periodic Poisson input, J Appl Probab 18 ( [22] RSh Liptser and AN Shiryayev, Theory of Martingales (Kluwer Academic, Dordrecht, 1989 [23] A Mandelbaum and W Massey, Strong approximations for time dependent queues, Math Oper Res 2 ( [24] A Mandelbaum and G Pats, State-dependent queues: Approximations and applications, in: IMA Volumes in Mathematics and Its Applications, eds F Kelly and RJ Williams, Vol 71 (Springer, Berlin, 1995 pp [25] A Mandelbaum and G Pats, State-dependent stochastic networks Part I: Approximations and applications with continuous diffusion its, Ann Appl Probab 8(2 ( [26] BH Margoluis, A sample path analysis of the M t /M t /c queue, Queueing Systems 31(1 ( [27] W Massey, Asymptotic analysis of the time-dependent M/M/1 queues, Math Oper Res 1(2 ( [28] AN Shiryayev, Probability (Springer, Berlin, 1984 [29] H Thorisson, Periodic regeneration, Stochastic Process Appl 2 ( [3] A Zeifman, Some estimates of the rate of convergence for birth-and-death processes, J Appl Probab 28 ( [31] A Zeifman, On stability of continuous-time nonhomogeneous Markov chains, Soviet Math 35(7 ( [32] A Zeifman, On the ergodicity of nonhomogeneous birth and death processes, J Math Sci 72(1 ( [33] A Zeifman, On the estimation of probabilities for birth-and-death processes, J Appl Probab 32 ( [34] A Zeifman, Upper and lower bounds on the rate of convergence for nonhomogeneous birth-and-death processes, Stochastic Process Appl 59 ( [35] A Zeifman, Stability of birth and death processes, J Math Sci 91(3 ( [36] A Zeifman and D Isaacson, On strong ergodicity for nonhomogeneous continuous-time Markov chains, Stochastic Process Appl 5 (
arxiv:math/ v4 [math.pr] 12 Apr 2007
arxiv:math/612224v4 [math.pr] 12 Apr 27 LARGE CLOSED QUEUEING NETWORKS IN SEMI-MARKOV ENVIRONMENT AND ITS APPLICATION VYACHESLAV M. ABRAMOV Abstract. The paper studies closed queueing networks containing
More informationarxiv:math.pr/ v1 24 May 2005
A Large Closed Queueing Network Containing Two Types of Node and Multiple Customer Classes: One Bottleneck Station arxiv:math.pr/55489 v 24 May 25 Vyacheslav M. Abramov (vyachesl@zahav.net.il) The Sackler
More informationTwo-sided Bounds for the Convergence Rate of Markov Chains
UDC 519.217.2 Two-sided Bounds for the Convergence Rate of Markov Chains A. Zeifman, Ya. Satin, K. Kiseleva, V. Korolev Vologda State University, Institute of Informatics Problems of the FRC CSC RAS, ISEDT
More information(2m)-TH MEAN BEHAVIOR OF SOLUTIONS OF STOCHASTIC DIFFERENTIAL EQUATIONS UNDER PARAMETRIC PERTURBATIONS
(2m)-TH MEAN BEHAVIOR OF SOLUTIONS OF STOCHASTIC DIFFERENTIAL EQUATIONS UNDER PARAMETRIC PERTURBATIONS Svetlana Janković and Miljana Jovanović Faculty of Science, Department of Mathematics, University
More informationMarkov Chains. X(t) is a Markov Process if, for arbitrary times t 1 < t 2 <... < t k < t k+1. If X(t) is discrete-valued. If X(t) is continuous-valued
Markov Chains X(t) is a Markov Process if, for arbitrary times t 1 < t 2
More informationLecture 10: Semi-Markov Type Processes
Lecture 1: Semi-Markov Type Processes 1. Semi-Markov processes (SMP) 1.1 Definition of SMP 1.2 Transition probabilities for SMP 1.3 Hitting times and semi-markov renewal equations 2. Processes with semi-markov
More informationIEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18.
IEOR 6711: Stochastic Models I, Fall 23, Professor Whitt Solutions to Final Exam: Thursday, December 18. Below are six questions with several parts. Do as much as you can. Show your work. 1. Two-Pump Gas
More informationStatistics 150: Spring 2007
Statistics 150: Spring 2007 April 23, 2008 0-1 1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities
More informationContinuous-Time Markov Chain
Continuous-Time Markov Chain Consider the process {X(t),t 0} with state space {0, 1, 2,...}. The process {X(t),t 0} is a continuous-time Markov chain if for all s, t 0 and nonnegative integers i, j, x(u),
More informationExact Simulation of the Stationary Distribution of M/G/c Queues
1/36 Exact Simulation of the Stationary Distribution of M/G/c Queues Professor Karl Sigman Columbia University New York City USA Conference in Honor of Søren Asmussen Monday, August 1, 2011 Sandbjerg Estate
More informationSTABILIZATION OF AN OVERLOADED QUEUEING NETWORK USING MEASUREMENT-BASED ADMISSION CONTROL
First published in Journal of Applied Probability 43(1) c 2006 Applied Probability Trust STABILIZATION OF AN OVERLOADED QUEUEING NETWORK USING MEASUREMENT-BASED ADMISSION CONTROL LASSE LESKELÄ, Helsinki
More informationAsymptotic Coupling of an SPDE, with Applications to Many-Server Queues
Asymptotic Coupling of an SPDE, with Applications to Many-Server Queues Mohammadreza Aghajani joint work with Kavita Ramanan Brown University March 2014 Mohammadreza Aghajanijoint work Asymptotic with
More informationN.G.Bean, D.A.Green and P.G.Taylor. University of Adelaide. Adelaide. Abstract. process of an MMPP/M/1 queue is not a MAP unless the queue is a
WHEN IS A MAP POISSON N.G.Bean, D.A.Green and P.G.Taylor Department of Applied Mathematics University of Adelaide Adelaide 55 Abstract In a recent paper, Olivier and Walrand (994) claimed that the departure
More informationLIMITS FOR QUEUES AS THE WAITING ROOM GROWS. Bell Communications Research AT&T Bell Laboratories Red Bank, NJ Murray Hill, NJ 07974
LIMITS FOR QUEUES AS THE WAITING ROOM GROWS by Daniel P. Heyman Ward Whitt Bell Communications Research AT&T Bell Laboratories Red Bank, NJ 07701 Murray Hill, NJ 07974 May 11, 1988 ABSTRACT We study the
More informationA Joining Shortest Queue with MAP Inputs
The Eighth International Symposium on Operations Research and Its Applications (ISORA 09) Zhangjiajie, China, September 20 22, 2009 Copyright 2009 ORSC & APORC, pp. 25 32 A Joining Shortest Queue with
More informationSynchronized Queues with Deterministic Arrivals
Synchronized Queues with Deterministic Arrivals Dimitra Pinotsi and Michael A. Zazanis Department of Statistics Athens University of Economics and Business 76 Patission str., Athens 14 34, Greece Abstract
More informationQueueing Networks and Insensitivity
Lukáš Adam 29. 10. 2012 1 / 40 Table of contents 1 Jackson networks 2 Insensitivity in Erlang s Loss System 3 Quasi-Reversibility and Single-Node Symmetric Queues 4 Quasi-Reversibility in Networks 5 The
More informationWeighted Sums of Orthogonal Polynomials Related to Birth-Death Processes with Killing
Advances in Dynamical Systems and Applications ISSN 0973-5321, Volume 8, Number 2, pp. 401 412 (2013) http://campus.mst.edu/adsa Weighted Sums of Orthogonal Polynomials Related to Birth-Death Processes
More informationEXTINCTION PROBABILITY IN A BIRTH-DEATH PROCESS WITH KILLING
EXTINCTION PROBABILITY IN A BIRTH-DEATH PROCESS WITH KILLING Erik A. van Doorn and Alexander I. Zeifman Department of Applied Mathematics University of Twente P.O. Box 217, 7500 AE Enschede, The Netherlands
More informationLECTURE #6 BIRTH-DEATH PROCESS
LECTURE #6 BIRTH-DEATH PROCESS 204528 Queueing Theory and Applications in Networks Assoc. Prof., Ph.D. (รศ.ดร. อน นต ผลเพ ม) Computer Engineering Department, Kasetsart University Outline 2 Birth-Death
More informationNEW FRONTIERS IN APPLIED PROBABILITY
J. Appl. Prob. Spec. Vol. 48A, 209 213 (2011) Applied Probability Trust 2011 NEW FRONTIERS IN APPLIED PROBABILITY A Festschrift for SØREN ASMUSSEN Edited by P. GLYNN, T. MIKOSCH and T. ROLSKI Part 4. Simulation
More informationSENSITIVITY OF HIDDEN MARKOV MODELS
J. Appl. Prob. 42, 632 642 (2005) Printed in Israel Applied Probability Trust 2005 SENSITIVITY OF HIDDEN MARKOV MODELS ALEXANDER Yu. MITROPHANOV, ALEXANDRE LOMSADZE and MARK BORODOVSKY, Georgia Institute
More informationPERTURBATION ANALYSIS FOR DENUMERABLE MARKOV CHAINS WITH APPLICATION TO QUEUEING MODELS
Adv Appl Prob 36, 839 853 (2004) Printed in Northern Ireland Applied Probability Trust 2004 PERTURBATION ANALYSIS FOR DENUMERABLE MARKOV CHAINS WITH APPLICATION TO QUEUEING MODELS EITAN ALTMAN and KONSTANTIN
More informationSIMILAR MARKOV CHAINS
SIMILAR MARKOV CHAINS by Phil Pollett The University of Queensland MAIN REFERENCES Convergence of Markov transition probabilities and their spectral properties 1. Vere-Jones, D. Geometric ergodicity in
More informationControl of Fork-Join Networks in Heavy-Traffic
in Heavy-Traffic Asaf Zviran Based on MSc work under the guidance of Rami Atar (Technion) and Avishai Mandelbaum (Technion) Industrial Engineering and Management Technion June 2010 Introduction Network
More informationChapter 3 Balance equations, birth-death processes, continuous Markov Chains
Chapter 3 Balance equations, birth-death processes, continuous Markov Chains Ioannis Glaropoulos November 4, 2012 1 Exercise 3.2 Consider a birth-death process with 3 states, where the transition rate
More informationStochastic modelling of epidemic spread
Stochastic modelling of epidemic spread Julien Arino Centre for Research on Inner City Health St Michael s Hospital Toronto On leave from Department of Mathematics University of Manitoba Julien Arino@umanitoba.ca
More information2905 Queueing Theory and Simulation PART III: HIGHER DIMENSIONAL AND NON-MARKOVIAN QUEUES
295 Queueing Theory and Simulation PART III: HIGHER DIMENSIONAL AND NON-MARKOVIAN QUEUES 16 Queueing Systems with Two Types of Customers In this section, we discuss queueing systems with two types of customers.
More informationContents Preface The Exponential Distribution and the Poisson Process Introduction to Renewal Theory
Contents Preface... v 1 The Exponential Distribution and the Poisson Process... 1 1.1 Introduction... 1 1.2 The Density, the Distribution, the Tail, and the Hazard Functions... 2 1.2.1 The Hazard Function
More informationSolving the Poisson Disorder Problem
Advances in Finance and Stochastics: Essays in Honour of Dieter Sondermann, Springer-Verlag, 22, (295-32) Research Report No. 49, 2, Dept. Theoret. Statist. Aarhus Solving the Poisson Disorder Problem
More informationOn Reflecting Brownian Motion with Drift
Proc. Symp. Stoch. Syst. Osaka, 25), ISCIE Kyoto, 26, 1-5) On Reflecting Brownian Motion with Drift Goran Peskir This version: 12 June 26 First version: 1 September 25 Research Report No. 3, 25, Probability
More informationFINAL REPORT. Project Title: Extreme Values of Queues, Point Processes and Stochastic Networks
FINAL REPORT Project Title: Extreme Values of Queues, Point Processes and Stochastic Networks Grant No.: AFOSR 84-0367 Project Director: Professor Richard F. Serfozo Industrial and Systems Engineering
More informationStochastic relations of random variables and processes
Stochastic relations of random variables and processes Lasse Leskelä Helsinki University of Technology 7th World Congress in Probability and Statistics Singapore, 18 July 2008 Fundamental problem of applied
More informationChapter 5. Continuous-Time Markov Chains. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan
Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Continuous-Time Markov Chains Consider a continuous-time stochastic process
More informationCentral limit theorems for ergodic continuous-time Markov chains with applications to single birth processes
Front. Math. China 215, 1(4): 933 947 DOI 1.17/s11464-15-488-5 Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes Yuanyuan LIU 1, Yuhui ZHANG 2
More informationModel reversibility of a two dimensional reflecting random walk and its application to queueing network
arxiv:1312.2746v2 [math.pr] 11 Dec 2013 Model reversibility of a two dimensional reflecting random walk and its application to queueing network Masahiro Kobayashi, Masakiyo Miyazawa and Hiroshi Shimizu
More informationEXAM IN COURSE TMA4265 STOCHASTIC PROCESSES Wednesday 7. August, 2013 Time: 9:00 13:00
Norges teknisk naturvitenskapelige universitet Institutt for matematiske fag Page 1 of 7 English Contact: Håkon Tjelmeland 48 22 18 96 EXAM IN COURSE TMA4265 STOCHASTIC PROCESSES Wednesday 7. August, 2013
More informationAn M/M/1 Queue in Random Environment with Disasters
An M/M/1 Queue in Random Environment with Disasters Noam Paz 1 and Uri Yechiali 1,2 1 Department of Statistics and Operations Research School of Mathematical Sciences Tel Aviv University, Tel Aviv 69978,
More informationTHE VARIANCE CONSTANT FOR THE ACTUAL WAITING TIME OF THE PH/PH/1 QUEUE. By Mogens Bladt National University of Mexico
The Annals of Applied Probability 1996, Vol. 6, No. 3, 766 777 THE VARIANCE CONSTANT FOR THE ACTUAL WAITING TIME OF THE PH/PH/1 QUEUE By Mogens Bladt National University of Mexico In this paper we consider
More informationStochastic modelling of epidemic spread
Stochastic modelling of epidemic spread Julien Arino Department of Mathematics University of Manitoba Winnipeg Julien Arino@umanitoba.ca 19 May 2012 1 Introduction 2 Stochastic processes 3 The SIS model
More informationQueueing Theory I Summary! Little s Law! Queueing System Notation! Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K "
Queueing Theory I Summary Little s Law Queueing System Notation Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K " Little s Law a(t): the process that counts the number of arrivals
More informationStochastic process. X, a series of random variables indexed by t
Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,
More informationPart I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationMODELING WEBCHAT SERVICE CENTER WITH MANY LPS SERVERS
MODELING WEBCHAT SERVICE CENTER WITH MANY LPS SERVERS Jiheng Zhang Oct 26, 211 Model and Motivation Server Pool with multiple LPS servers LPS Server K Arrival Buffer. Model and Motivation Server Pool with
More informationModelling Complex Queuing Situations with Markov Processes
Modelling Complex Queuing Situations with Markov Processes Jason Randal Thorne, School of IT, Charles Sturt Uni, NSW 2795, Australia Abstract This article comments upon some new developments in the field
More informationMULTISERVER QUEUEING SYSTEMS WITH RETRIALS AND LOSSES
ANZIAM J. 48(27), 297 314 MULTISERVER QUEUEING SYSTEMS WITH RETRIALS AND LOSSES VYACHESLAV M. ABRAMOV 1 (Received 8 June, 26; revised 3 November, 26) Abstract The interest in retrial queueing systems mainly
More informationTHE SYNCHRONIZATION OF POISSON PROCESSES AND QUEUEING NETWORKS WITH SERVICE AND SYNCHRONIZATION NODES
Adv. Appl. Prob. 32, 824 843 (2) Printed in Northern Ireland Applied Probability Trust 2 THE SYNCHRONIZATION OF POISSON PROCESSES AND QUEUEING NETWORKS WITH SERVICE AND SYNCHRONIZATION NODES BALAJI PRABHAKAR
More informationA probabilistic proof of Perron s theorem arxiv: v1 [math.pr] 16 Jan 2018
A probabilistic proof of Perron s theorem arxiv:80.05252v [math.pr] 6 Jan 208 Raphaël Cerf DMA, École Normale Supérieure January 7, 208 Abstract Joseba Dalmau CMAP, Ecole Polytechnique We present an alternative
More informationOPTIMAL SOLUTIONS TO STOCHASTIC DIFFERENTIAL INCLUSIONS
APPLICATIONES MATHEMATICAE 29,4 (22), pp. 387 398 Mariusz Michta (Zielona Góra) OPTIMAL SOLUTIONS TO STOCHASTIC DIFFERENTIAL INCLUSIONS Abstract. A martingale problem approach is used first to analyze
More informationMarkov processes and queueing networks
Inria September 22, 2015 Outline Poisson processes Markov jump processes Some queueing networks The Poisson distribution (Siméon-Denis Poisson, 1781-1840) { } e λ λ n n! As prevalent as Gaussian distribution
More informationPerformance Evaluation of Queuing Systems
Performance Evaluation of Queuing Systems Introduction to Queuing Systems System Performance Measures & Little s Law Equilibrium Solution of Birth-Death Processes Analysis of Single-Station Queuing Systems
More informationJ. MEDHI STOCHASTIC MODELS IN QUEUEING THEORY
J. MEDHI STOCHASTIC MODELS IN QUEUEING THEORY SECOND EDITION ACADEMIC PRESS An imprint of Elsevier Science Amsterdam Boston London New York Oxford Paris San Diego San Francisco Singapore Sydney Tokyo Contents
More informationThe passage time distribution for a birth-and-death chain: Strong stationary duality gives a first stochastic proof
The passage time distribution for a birth-and-death chain: Strong stationary duality gives a first stochastic proof James Allen Fill 1 Department of Applied Mathematics and Statistics The Johns Hopkins
More informationIEOR 6711, HMWK 5, Professor Sigman
IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.
More informationAn Introduction to Stochastic Modeling
F An Introduction to Stochastic Modeling Fourth Edition Mark A. Pinsky Department of Mathematics Northwestern University Evanston, Illinois Samuel Karlin Department of Mathematics Stanford University Stanford,
More informationCONTROLLABILITY OF NONLINEAR DISCRETE SYSTEMS
Int. J. Appl. Math. Comput. Sci., 2002, Vol.2, No.2, 73 80 CONTROLLABILITY OF NONLINEAR DISCRETE SYSTEMS JERZY KLAMKA Institute of Automatic Control, Silesian University of Technology ul. Akademicka 6,
More informationAnalysis of an M/G/1 queue with customer impatience and an adaptive arrival process
Analysis of an M/G/1 queue with customer impatience and an adaptive arrival process O.J. Boxma 1, O. Kella 2, D. Perry 3, and B.J. Prabhu 1,4 1 EURANDOM and Department of Mathematics & Computer Science,
More information(implicitly assuming time-homogeneity from here on)
Continuous-Time Markov Chains Models Tuesday, November 15, 2011 2:02 PM The fundamental object describing the dynamics of a CTMC (continuous-time Markov chain) is the probability transition (matrix) function:
More informationStability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk
Stability and Rare Events in Stochastic Models Sergey Foss Heriot-Watt University, Edinburgh and Institute of Mathematics, Novosibirsk ANSAPW University of Queensland 8-11 July, 2013 1 Outline (I) Fluid
More informationOn Tandem Blocking Queues with a Common Retrial Queue
On Tandem Blocking Queues with a Common Retrial Queue K. Avrachenkov U. Yechiali Abstract We consider systems of tandem blocking queues having a common retrial queue. The model represents dynamics of short
More informationAdvanced Queueing Theory
Advanced Queueing Theory 1 Networks of queues (reversibility, output theorem, tandem networks, partial balance, product-form distribution, blocking, insensitivity, BCMP networks, mean-value analysis, Norton's
More informationSTAT 380 Continuous Time Markov Chains
STAT 380 Continuous Time Markov Chains Richard Lockhart Simon Fraser University Spring 2018 Richard Lockhart (Simon Fraser University)STAT 380 Continuous Time Markov Chains Spring 2018 1 / 35 Continuous
More informationA COMPOUND POISSON APPROXIMATION INEQUALITY
J. Appl. Prob. 43, 282 288 (2006) Printed in Israel Applied Probability Trust 2006 A COMPOUND POISSON APPROXIMATION INEQUALITY EROL A. PEKÖZ, Boston University Abstract We give conditions under which the
More informationDepartment of Applied Mathematics Faculty of EEMCS. University of Twente. Memorandum No Birth-death processes with killing
Department of Applied Mathematics Faculty of EEMCS t University of Twente The Netherlands P.O. Box 27 75 AE Enschede The Netherlands Phone: +3-53-48934 Fax: +3-53-48934 Email: memo@math.utwente.nl www.math.utwente.nl/publications
More informationContinuous Time Processes
page 102 Chapter 7 Continuous Time Processes 7.1 Introduction In a continuous time stochastic process (with discrete state space), a change of state can occur at any time instant. The associated point
More informationSTABILITY OF MULTICLASS QUEUEING NETWORKS UNDER LONGEST-QUEUE AND LONGEST-DOMINATING-QUEUE SCHEDULING
Applied Probability Trust (7 May 2015) STABILITY OF MULTICLASS QUEUEING NETWORKS UNDER LONGEST-QUEUE AND LONGEST-DOMINATING-QUEUE SCHEDULING RAMTIN PEDARSANI and JEAN WALRAND, University of California,
More informationCDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical
CDA5530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic ti process X = {X(t), t T} is a collection of random variables (rvs); one
More informationAsymptotic Irrelevance of Initial Conditions for Skorohod Reflection Mapping on the Nonnegative Orthant
Published online ahead of print March 2, 212 MATHEMATICS OF OPERATIONS RESEARCH Articles in Advance, pp. 1 12 ISSN 364-765X print) ISSN 1526-5471 online) http://dx.doi.org/1.1287/moor.112.538 212 INFORMS
More informationQueueing Theory II. Summary. ! M/M/1 Output process. ! Networks of Queue! Method of Stages. ! General Distributions
Queueing Theory II Summary! M/M/1 Output process! Networks of Queue! Method of Stages " Erlang Distribution " Hyperexponential Distribution! General Distributions " Embedded Markov Chains M/M/1 Output
More informationOn Tandem Blocking Queues with a Common Retrial Queue
On Tandem Blocking Queues with a Common Retrial Queue K. Avrachenkov U. Yechiali Abstract We consider systems of tandem blocking queues having a common retrial queue, for which explicit analytic results
More informationMatrix analytic methods. Lecture 1: Structured Markov chains and their stationary distribution
1/29 Matrix analytic methods Lecture 1: Structured Markov chains and their stationary distribution Sophie Hautphenne and David Stanford (with thanks to Guy Latouche, U. Brussels and Peter Taylor, U. Melbourne
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 15. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationCS 798: Homework Assignment 3 (Queueing Theory)
1.0 Little s law Assigned: October 6, 009 Patients arriving to the emergency room at the Grand River Hospital have a mean waiting time of three hours. It has been found that, averaged over the period of
More informationQ = (c) Assuming that Ricoh has been working continuously for 7 days, what is the probability that it will remain working at least 8 more days?
IEOR 4106: Introduction to Operations Research: Stochastic Models Spring 2005, Professor Whitt, Second Midterm Exam Chapters 5-6 in Ross, Thursday, March 31, 11:00am-1:00pm Open Book: but only the Ross
More informatione - c o m p a n i o n
OPERATIONS RESEARCH http://dx.doi.org/1.1287/opre.111.13ec e - c o m p a n i o n ONLY AVAILABLE IN ELECTRONIC FORM 212 INFORMS Electronic Companion A Diffusion Regime with Nondegenerate Slowdown by Rami
More informationHEAVY-TRAFFIC EXTREME-VALUE LIMITS FOR QUEUES
HEAVY-TRAFFIC EXTREME-VALUE LIMITS FOR QUEUES by Peter W. Glynn Department of Operations Research Stanford University Stanford, CA 94305-4022 and Ward Whitt AT&T Bell Laboratories Murray Hill, NJ 07974-0636
More informationPage 0 of 5 Final Examination Name. Closed book. 120 minutes. Cover page plus five pages of exam.
Final Examination Closed book. 120 minutes. Cover page plus five pages of exam. To receive full credit, show enough work to indicate your logic. Do not spend time calculating. You will receive full credit
More informationIrreducibility. Irreducible. every state can be reached from every other state For any i,j, exist an m 0, such that. Absorbing state: p jj =1
Irreducibility Irreducible every state can be reached from every other state For any i,j, exist an m 0, such that i,j are communicate, if the above condition is valid Irreducible: all states are communicate
More informationLet (Ω, F) be a measureable space. A filtration in discrete time is a sequence of. F s F t
2.2 Filtrations Let (Ω, F) be a measureable space. A filtration in discrete time is a sequence of σ algebras {F t } such that F t F and F t F t+1 for all t = 0, 1,.... In continuous time, the second condition
More informationA Diffusion Approximation for Stationary Distribution of Many-Server Queueing System In Halfin-Whitt Regime
A Diffusion Approximation for Stationary Distribution of Many-Server Queueing System In Halfin-Whitt Regime Mohammadreza Aghajani joint work with Kavita Ramanan Brown University APS Conference, Istanbul,
More informationQueuing Networks: Burke s Theorem, Kleinrock s Approximation, and Jackson s Theorem. Wade Trappe
Queuing Networks: Burke s Theorem, Kleinrock s Approximation, and Jackson s Theorem Wade Trappe Lecture Overview Network of Queues Introduction Queues in Tandem roduct Form Solutions Burke s Theorem What
More informationStationary remaining service time conditional on queue length
Stationary remaining service time conditional on queue length Karl Sigman Uri Yechiali October 7, 2006 Abstract In Mandelbaum and Yechiali (1979) a simple formula is derived for the expected stationary
More informationLecture 20: Reversible Processes and Queues
Lecture 20: Reversible Processes and Queues 1 Examples of reversible processes 11 Birth-death processes We define two non-negative sequences birth and death rates denoted by {λ n : n N 0 } and {µ n : n
More informationTHE ON NETWORK FLOW EQUATIONS AND SPLITTG FORMULAS TRODUCTION FOR SOJOURN TIMES IN QUEUEING NETWORKS 1 NO FLOW EQUATIONS
Applied Mathematics and Stochastic Analysis 4, Number 2, Summer 1991, III-I16 ON NETWORK FLOW EQUATIONS AND SPLITTG FORMULAS FOR SOJOURN TIMES IN QUEUEING NETWORKS 1 HANS DADUNA Institut flit Mathematische
More informationON THE NON-EXISTENCE OF PRODUCT-FORM SOLUTIONS FOR QUEUEING NETWORKS WITH RETRIALS
ON THE NON-EXISTENCE OF PRODUCT-FORM SOLUTIONS FOR QUEUEING NETWORKS WITH RETRIALS J.R. ARTALEJO, Department of Statistics and Operations Research, Faculty of Mathematics, Complutense University of Madrid,
More informationPITMAN S 2M X THEOREM FOR SKIP-FREE RANDOM WALKS WITH MARKOVIAN INCREMENTS
Elect. Comm. in Probab. 6 (2001) 73 77 ELECTRONIC COMMUNICATIONS in PROBABILITY PITMAN S 2M X THEOREM FOR SKIP-FREE RANDOM WALKS WITH MARKOVIAN INCREMENTS B.M. HAMBLY Mathematical Institute, University
More informationA Note on the Event Horizon for a Processor Sharing Queue
A Note on the Event Horizon for a Processor Sharing Queue Robert C. Hampshire Heinz School of Public Policy and Management Carnegie Mellon University hamp@andrew.cmu.edu William A. Massey Department of
More informationData analysis and stochastic modeling
Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt
More informationOperations Research Letters. Instability of FIFO in a simple queueing system with arbitrarily low loads
Operations Research Letters 37 (2009) 312 316 Contents lists available at ScienceDirect Operations Research Letters journal homepage: www.elsevier.com/locate/orl Instability of FIFO in a simple queueing
More informationContinuous Time Markov Chains
Continuous Time Markov Chains Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 2015 Outline Introduction Continuous-Time Markov
More informationThe SIS and SIR stochastic epidemic models revisited
The SIS and SIR stochastic epidemic models revisited Jesús Artalejo Faculty of Mathematics, University Complutense of Madrid Madrid, Spain jesus_artalejomat.ucm.es BCAM Talk, June 16, 2011 Talk Schedule
More information1 Continuous-time chains, finite state space
Université Paris Diderot 208 Markov chains Exercises 3 Continuous-time chains, finite state space Exercise Consider a continuous-time taking values in {, 2, 3}, with generator 2 2. 2 2 0. Draw the diagramm
More informationIntroduction to Queuing Networks Solutions to Problem Sheet 3
Introduction to Queuing Networks Solutions to Problem Sheet 3 1. (a) The state space is the whole numbers {, 1, 2,...}. The transition rates are q i,i+1 λ for all i and q i, for all i 1 since, when a bus
More informationSince D has an exponential distribution, E[D] = 0.09 years. Since {A(t) : t 0} is a Poisson process with rate λ = 10, 000, A(0.
IEOR 46: Introduction to Operations Research: Stochastic Models Chapters 5-6 in Ross, Thursday, April, 4:5-5:35pm SOLUTIONS to Second Midterm Exam, Spring 9, Open Book: but only the Ross textbook, the
More informationMARKOVIANITY OF A SUBSET OF COMPONENTS OF A MARKOV PROCESS
MARKOVIANITY OF A SUBSET OF COMPONENTS OF A MARKOV PROCESS P. I. Kitsul Department of Mathematics and Statistics Minnesota State University, Mankato, MN, USA R. S. Liptser Department of Electrical Engeneering
More informationOverload Analysis of the PH/PH/1/K Queue and the Queue of M/G/1/K Type with Very Large K
Overload Analysis of the PH/PH/1/K Queue and the Queue of M/G/1/K Type with Very Large K Attahiru Sule Alfa Department of Mechanical and Industrial Engineering University of Manitoba Winnipeg, Manitoba
More informationOn hitting times and fastest strong stationary times for skip-free and more general chains
On hitting times and fastest strong stationary times for skip-free and more general chains James Allen Fill 1 Department of Applied Mathematics and Statistics The Johns Hopkins University jimfill@jhu.edu
More informationThe Transition Probability Function P ij (t)
The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it
More informationMATH 56A: STOCHASTIC PROCESSES CHAPTER 6
MATH 56A: STOCHASTIC PROCESSES CHAPTER 6 6. Renewal Mathematically, renewal refers to a continuous time stochastic process with states,, 2,. N t {,, 2, 3, } so that you only have jumps from x to x + and
More information