Chapter 3: Markov Processes First hitting times

Transcription

1 Chapter 3: Markov Processes First hitting times L. Breuer University of Kent, UK November 3, 2010

2 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0

3 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0 Service times: exponentially distributed with rate µ > 0

4 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0 Service times: exponentially distributed with rate µ > 0 c servers

5 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0 Service times: exponentially distributed with rate µ > 0 c servers no waiting room

6 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0 Service times: exponentially distributed with rate µ > 0 c servers no waiting room If upon an arrival the system is filled, i.e. all servers are busy, this arriving user is not admitted into the system.

7 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0 Service times: exponentially distributed with rate µ > 0 c servers no waiting room If upon an arrival the system is filled, i.e. all servers are busy, this arriving user is not admitted into the system. In this case we say that the arriving user is lost.

8 Example: M/M/c/c queue Arrivals: Poisson process with rate λ > 0 Service times: exponentially distributed with rate µ > 0 c servers no waiting room If upon an arrival the system is filled, i.e. all servers are busy, this arriving user is not admitted into the system. In this case we say that the arriving user is lost. Queueing systems with the possibility of such an event are thus called loss systems.

9 Example: M/M/c/c queue (contd.) The queue described above is a skip-free Markov process Y with state space E = {0,..., c}

10 Example: M/M/c/c queue (contd.) The queue described above is a skip-free Markov process Y with state space E = {0,..., c} and generator matrix λ Q = up to the first loss. λ µ λ µ λ 2µ λ 2µ λ (c 1)µ λ (c 1)µ λ cµ cµ

11 Example: M/M/c/c queue (contd.) We wish to know the time until the first loss.

12 Example: M/M/c/c queue (contd.) We wish to know the time until the first loss. Consider the Markov process Y with state space E = {0,..., c + 1}

13 Example: M/M/c/c queue (contd.) We wish to know the time until the first loss. Consider the Markov process Y with state space E = {0,..., c + 1} and generator matrix λ Q = λ µ λ µ λ 2µ λ 2µ λ cµ cµ λ

14 Example: M/M/c/c queue (contd.) We wish to know the time until the first loss. Consider the Markov process Y with state space E = {0,..., c + 1} and generator matrix Now λ Q = λ µ λ µ λ 2µ λ 2µ λ cµ cµ λ Z := inf{t 0 : Y t = c + 1} is the random variable of the time until the first loss.

15 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1}

16 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0

17 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m,

18 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m, η a column vector and 0 the zero row vector of the same dimension.

19 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m, η a column vector and 0 the zero row vector of the same dimension. The initial distribution of Y shall be the row vector α = (α, α m+1 ),

20 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m, η a column vector and 0 the zero row vector of the same dimension. The initial distribution of Y shall be the row vector α = (α, α m+1 ), with α being a row vector of dimension m.

21 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m, η a column vector and 0 the zero row vector of the same dimension. The initial distribution of Y shall be the row vector α = (α, α m+1 ), with α being a row vector of dimension m. The first states {1,..., m} shall be transient,

22 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m, η a column vector and 0 the zero row vector of the same dimension. The initial distribution of Y shall be the row vector α = (α, α m+1 ), with α being a row vector of dimension m. The first states {1,..., m} shall be transient, while the state m + 1 is absorbing.

23 Phase type distributions Let Y = (Y t : t 0) denote an homogeneous Markov process with finite state space {1,..., m + 1} and generator ( ) T η Q = 0 0 where T is a square matrix of dimension m, η a column vector and 0 the zero row vector of the same dimension. The initial distribution of Y shall be the row vector α = (α, α m+1 ), with α being a row vector of dimension m. The first states {1,..., m} shall be transient, while the state m + 1 is absorbing. Let Z := inf{t 0 : Y t = m + 1} be the random variable of the time until absorption in state m + 1.

24 Phase type distributions The distribution of Z is called phase type distribution

25 Phase type distributions The distribution of Z is called phase type distribution (or shortly PH distribution) with parameters (α, T ).

26 Phase type distributions The distribution of Z is called phase type distribution (or shortly PH distribution) with parameters (α, T ). We write Z PH(α, T ).

27 Phase type distributions The distribution of Z is called phase type distribution (or shortly PH distribution) with parameters (α, T ). We write Z PH(α, T ). The dimension m of T is called the order of the distribution PH(α, T ).

28 Phase type distributions The distribution of Z is called phase type distribution (or shortly PH distribution) with parameters (α, T ). We write Z PH(α, T ). The dimension m of T is called the order of the distribution PH(α, T ). The states {1,..., m} are also called phases, which gives rise to the name phase type distribution.

29 Remarks Let 1 denote the column vector of dimension m with all entries equal to one.

30 Remarks Let 1 denote the column vector of dimension m with all entries equal to one. The first observations to be derived from the above definition are η = T 1 and α m+1 = 1 α1

31 Remarks Let 1 denote the column vector of dimension m with all entries equal to one. The first observations to be derived from the above definition are η = T 1 and α m+1 = 1 α1 These follow immediately from the properties that the row sums of a generator are zero and the sum of a probability vector is one.

32 Remarks Let 1 denote the column vector of dimension m with all entries equal to one. The first observations to be derived from the above definition are η = T 1 and α m+1 = 1 α1 These follow immediately from the properties that the row sums of a generator are zero and the sum of a probability vector is one. The vector η is called the exit vector of the PH distribution.

33 Theorem 9.2 Let Z PH(α, T ).

34 Theorem 9.2 Let Z PH(α, T ). Then the distribution function of Z is given by F (t) := P(Z t) = 1 αe T t 1 for all t 0,

35 Theorem 9.2 Let Z PH(α, T ). Then the distribution function of Z is given by F (t) := P(Z t) = 1 αe T t 1 for all t 0, and the density function is f (t) = αe T t η for all t > 0.

36 Theorem 9.2 Let Z PH(α, T ). Then the distribution function of Z is given by F (t) := P(Z t) = 1 αe T t 1 for all t 0, and the density function is f (t) = αe T t η for all t > 0. Here, the function e T t := exp(t t) := n=0 t n n! T n denotes a matrix exponential function.

37 Proof of theorem 9.2 For the Markov process Y with generator Q as given in the definition of Z,

38 Proof of theorem 9.2 For the Markov process Y with generator Q as given in the definition of Z, the equation ( e T t 1 e P(t) = exp(q t) = T t ) holds for the transition matrix P(t) at every time t 0.

39 Proof of theorem 9.2 For the Markov process Y with generator Q as given in the definition of Z, the equation ( e T t 1 e P(t) = exp(q t) = T t ) holds for the transition matrix P(t) at every time t 0. This implies, with e m+1 denoting the m + 1st canonical base vector, F (t) = αe Q t e m+1 = α m+1 + α (1 e T t 1)

40 Proof of theorem 9.2 For the Markov process Y with generator Q as given in the definition of Z, the equation ( e T t 1 e P(t) = exp(q t) = T t ) holds for the transition matrix P(t) at every time t 0. This implies, with e m+1 denoting the m + 1st canonical base vector, F (t) = αe Q t e m+1 = α m+1 + α (1 e T t 1) = α m+1 + α1 αe T t 1 = 1 αe T t 1

41 Proof of theorem 9.2 (contd.) For the density function we obtain

42 Proof of theorem 9.2 (contd.) For the density function we obtain f (t) = F (t) = d dt ( ) 1 αe T t 1

43 Proof of theorem 9.2 (contd.) For the density function we obtain f (t) = F (t) = d dt ( ) 1 αe T t 1 = α d dt et t 1 = αte T t 1 = αe T t ( T 1) = αe T t η which was to be proven.