MAT 4371 - SYS 5120 (Winter 2012) Assignment 5 (not to be submitted) There are 4 questions. Question 1: Consider the following generator for a continuous time Markov chain. 4 1 3 Q = 2 5 3 5 2 7 (a) Give the probability transition matrix P for the corresponding embedded Markov chain. (b) Based on the following output from R, give the stationary distribution φ for the embedded Markov chain. > P<-matrix(nrow=3,c(0,2/5,5/7,1/4,0,2/7,3/4,3/5,0)) > P [1,] 0.0000000 0.2500000 0.75 [2,] 0.4000000 0.0000000 0.60 [3,] 0.7142857 0.2857143 0.00 > # the transpose of P > Pt<-t(P) > Pt [1,] 0.00 0.4 0.7142857 [2,] 0.25 0.0 0.2857143 [3,] 0.75 0.6 0.0000000 > # the eigenvalues and vectors of P transpose > v<-eigen(pt) > v$values [1] 1.0000000-0.7390457-0.2609543 > v$vectors [1,] -0.6332372-0.67884786 0.5524690 [2,] -0.3548312-0.05348088-0.7968913 [3,] -0.6878266 0.73232874 0.2444223 (c) Use returns to state 1 of the continuous time Markov chain to define cycles for a renewal process. Define X n as the length of the nth cycle and R n 1
as the time in state 1 during the n cycle. Define N(t) as the renewal counting process, i.e. it counts the number cycles by time t. In words, what do R(t) = N(t) n=1 R n and R(t)/t represent within the context of this problem? (d) Refer to part (c). Find the expected reward of a cycle, i.e. E[R], and the expected length of a cycle, i.e. E[X]. (e) Let π 1 be the long run proportion of time in 1. Use parts (c) and (d) to find π 1. Using similar arguments, give π 2 and π 3. (f) Let π be the long-run probability vector from part (e). Show that π is the stationary distribution of the continuous time Markov chain by showing that it satisfies the general balance equations (or equilibrium equations). solution: (a) The elements of Q are the transition rates. For example, q ij is the transition rate from state i to state j. The state transition probabilities of the embedded chain are { qij /q p ij = ii, i j 0, i = j. Thus, P = 0 1/4 3/4 2/5 0 3/5 5/7 2/7 0 (b) The vector v = [ 0.6332372, 0.3548312, 0.6878266] is a left eigenvector to P with corresponding eigenvalue 1. Normalizing v to get a distribution gives the stationary distribution for the embedded chain. We get φ = 1 [ 0.6332372, 0.3548312, 0.6878266] 0.6332372 + ( 0.3548312) + ( 0.6878266) = [0.3779, 0.2117, 0.4104]. (c) R(t) is the total amount of time that the CTMC is in state i up to time t. Thus, R(t)/t is the proportion of time that the CTMC is in state i. (d) R is the sojourn time in state 1. It has an exponential distribution with rate j:j 1 q 1j. Thus, 1 E[R] = j:j 1 q = 1 1j 4 = 0.25. 2
Between visits to state 1, the following visit ratios give the mean number of visits to state 2 and state 3 for the embedded chain, respectively, ν 12 = φ 2 φ 1 = 0.2117 0.3779 = 0.56020, ν 13 = φ 3 φ 1 = 0.4104 0.3779 = 1.0860. By Wald s identity, the expected length of a cycle is E[X] = µ 1 + ν 12 µ 2 + ν 13 µ 3, where µ i represents a mean sojourn time in state i. We get E[X] = 1 ( ) ( ) ( ) ( ) 0.2117 1 0.4104 1 4 + + = 0.51718. 0.3779 5 0.3779 7 (e) From the law of large numbers for reward renewal processes, we have R(t) t E[R] E[X] almost surely as t. Therefore, the long-run proportion of time in state 1 is π 1 = 0.25 0.51718 = 0.4834. Similarly, and π 2 = π 3 = φ 2 µ 2 (0.2117)(1/5) 3 i=1 φ = i µ i (0.3779)(1/4) + (0.2117)(1/5) + (0.4104)(1/7) = 0.2166 φ 3 µ 3 (0.4104)(1/7) 3 i=1 φ = i µ i (0.3779)(1/4) + (0.2117)(1/5) + (0.4104)(1/7) = 0.3000. (f) We need to show that π Q = 0. Here is a verification with R: > Q<-matrix(nrow=3,c(-4,2,5,1,-5,2,3,3,-7)) > Q [1,] -4 1 3 [2,] 2-5 3 [3,] 5 2-7 > pi<-matrix(nrow=1,c(0.4834, 0.2166, 0.3000)) > pi [1,] 0.4834 0.2166 0.3 > pi %*% Q [1,] -4e-04 4e-04 1.665335e-16 3
Taking rounding under consideration, we have π Q = 0. Question 2: In a two-bay car wash, where cars arrive at the rate of λ = 9 cars per hour and the average duration of a car wash is twelve minutes, what is the probability that an arriving customer will find at least one of the bays occupied? Solution: We can model the car wash as an M/M/2 queueing model. The arrival rate is λ = 9 cars per hour. The mean time to service a car is 12 minutes, which is equivalent to 0.2 hour. So the service rate for a bay is µ = 1/0.2 = 5 cars per hour. We want the probability that the system is not empty, that is 1 π 0 = 1 1 ρ 1 + ρ = 1 1.9 1 +.9 = 0.947, where the traffic intensity is ρ = λ/(2 µ) = 9/(2 (5)) = 0.9. So the probability that an arriving customer will find at least one of the bays occupied is 94.7%. Question 3: At a particular exit ramp on the highway, there is a single tollbooth. It was originally thought that there would be insufficient traffic at this point to justify more than one booth, but the traffic has increased recently. There are now (during rush hour, which is the period of concern) an average of 210 cars per hours, and the average service time is fifteen seconds. Two proposals have been made. The first is to add another toll booth identical to the first (assume the traffic would split evenly and at random between the two). The second is to add an automatic (that is, no human operator) booth that accepts exact change only. That booth would have an average service time of only five seconds, but only a third of the arrivals (selected at random) would have exact change. The performance measure of concern is the average delay incurred by the people who exit at this ramp. Using an appropriate queuing model, estimate the delay under the present system, the first proposal, and the second proposal. solution: Consider the case where we add a second booth. We have two M/M/1 queues, each as an arrival rate of λ = 210/2 = 105 cars per hour. The service rate at a booth is 1 customer per 15 seconds, which is equivalent to 4 customers per minute or µ = 240 customers per hour. The traffic intensity for each M/M/1 queue is ρ = λ/µ = 105/240 = 0.4375. The mean waiting time to go through such a system (i.e. queueing time plus 4
service time) is W = 1 λ L = 0.77778 105 = 0.0074 hours (or 26.64 seconds). Remark: We used the fact that the mean number of customers in the system for an M/M/1 queue is L = and Little s Law that L = λ W. ρ 1 ρ = 0.4375 1 0.4375 = 0.77778 Now let us consider the scenario with an automatic booth. Again we have 2 M/M/1 queues, but the traffic intensity of the first queue (with the human operators) is ρ = λ (2/3) (210) = = 0.58333. µ 240 The mean waiting time to go through such a system (i.e. queueing time plus service time) is where W = 1 λ L = 1.39998 = 0.01000 hours (or 36 seconds), (2/3)(210) L = ρ 1 ρ = 0.58333 1 0.58333 = 1.39998. The traffic intensity of the second queue (with the automatic operator) is ρ = λ µ = (1/3) (210) (3600)/5 = 0.097222. The mean waiting time to go through such a system (i.e. queueing time plus service time) is where W = 1 λ L = 0.107692 = 0.001538 hours (or 5.54 seconds), (1/3)(210) L = ρ 1 ρ = 0.097222 1 0.097222 = 0.107692. Thus, one third of the customers have a mean delay of 5.54 seconds, while the others have a mean delay of 36 seconds. So the expected delay time for the second scenario is (1/3) 5.54 + (2/3) 36 = 25.84 seconds. The mean delay for the first option is 26.64 seconds. Using an automatic booth will only improve the mean delay overall customers. However, the majority of 5
the drivers in the latter case would see their mean delay go up to 36 seconds. The first scenario is probably the best choice. Question 4: Consider an M/M/2/4 queue. (a) Set up the state transition rate diagram. (b) What are the equilibrium equations? (c) Solve the equilibrium equations to obtain the stationary distribution. solution: (a) (b) From equation 0, we get Substitute into equation 1, we get state flow in = flow out 0 π 1 µ = π 0 λ 1 π 0 λ + π 2 (2 µ) = π 1 (λ + µ) 2 π 1 λ + π 3 (2 µ) = π 2 (λ + 2 µ) 3 π 2 λ + π 4 (2 µ) = π 3 (λ + 2 µ) 4 π 3 λ = π 4 (2 µ) π 1 = π 0 λ/µ. π 2 = π 0 λ/µ(λ + µ) 2 µ Substitute into equation 2, we get π 3 = π 0 (λ 2 /(2µ 2 ))(λ + µ) 2 µ Substitute into equation 3, we get Using 1 = 4 i=0 π i, gives π 0 = λ 2 π 0 λ 2 µ = π 0 2 µ 2. π 1 λ 2 µ = π 0 (λ 2 /(2µ 2 ))(λ + µ) π 0 λ 2 2 µ 2 µ 2 = π 0 λ 3 4 µ 3. π 4 = π 0 λ 4 8 µ 4. [ 1 + λ µ + λ ] 1 2 µ 2 + λ3 4µ 3 + λ4 8µ 4. 6
Note that equation 4 is also satisfied. Therefore, π 1 = π 0 λ/µ, π 2 = π 0 λ 2 2 µ 2, π 3 = π 0 λ 3 4 µ 3, π 4 = π 0 λ 4 8 µ 4, and π 0 = [ 1 + λ µ + λ ] 1 2 µ 2 + λ3 4µ 3 + λ4 8µ 4. 7