Quantitative Evaluation of Emedded Systems Solution 2: Discrete Time Markov Chains (DTMC) 2.1 Classification of DTMC states Prof. Dr. Anne Remke Design and Analysis of Communication Systems University of Twente 2016/2017 Let the probability transition matrices of the following DTMCs be given. (a) P = 0.5 0.4 0.1 0 0.5 0.5 0 0 1 (b) Q = 0.2 0.8 0 0 0.6 0.4 0 0 0.2 0.3 0.5 0 0 0 0 1 a) Draw the corresponding state-transition graphs. b) Classify the states of the following DTMCs (i.e., identify communicating classes, indicate for all classes whether they are (a)periodic, and/or positive/null-recurrent and/or transient, absorbing, etc. ) Solutions: a) The state transition graphs for P resp. Q can be found in fig. 1 resp. fig. 2 Figure 1: State transition graph for P b) Classification of states: (a) For P : all states are aperiodic states 1 and 2 are transient, whereas state 3 is recurrent and absorbing (b) For Q: All states are aperiodic, states 1, 2 and 4 are recurrent, state 3 is transient and state 4 is absorbing 1
Figure 2: State transition graph for Q 2.2 Craps The game Craps is based on betting on the outcome of the roll of two dice. The outcome of the first roll - the come-out roll - determines whether there is a need for any further rolls. On outcome 7 or 11, the game is over and the player wins. The outcomes 2, 3 or 12, however are craps ; the player loses. On any other outcome, the dice are rolled again, but the outcome of the come-out roll is remembered (the so-called point ). If the next roll yields 7 or the point, the game is over. On 7, the player loses, on point the player wins. In any other case, the dice are rolled until eventually either 7 or the point is obtained. a) Draw the state-transition graph for Craps. b) What is the probability to win? a) State transition graph (cf. fig. 3): b) 2 9 + 2 (( 1 12 )2 ( 3 4 )n + ( 1 9 )2 ( 13 18 )n + ( 5 36 )2 ( 25 36 )n ) = 2 + 0.27 = 0.492 9 n=0 2.3 Absent minded post-doc An absent minded post-doc has 2 umbrellas that he uses when commuting from home to office and back. If it rains and an umbrella is available in his location, he takes it. If it is not raining, he always forgets to take an umbrella. Suppose that it rains with probability 0.6 each time if he commutes, independently of other times. Our goal is to find the fraction of days he gets wet during a commute. a) Draw the state-transition diagram of the DTMC, modelling the umbrella problem. 2
Figure 3: State transition graph for craps game b) Derive the corresponding transition probability matrix. c) Compute the steady state distribution. d) Using the previous results, compute the fraction of days, he gets wet during a commute. a) State i represents the number of umbrellas at the current location, regardless of what the current location actually is (office or home). Figure 4: State transition graph for absent minded post-doc b) P = 0 0 1 0 0.4 0.6 0.4 0.6 0 c) The System is irreducible, aperiodic and pos. recurrent: π 0 = 0.4 π 2 π 2 = 2.5 π 0 π 1 = 0.4 π 1 + 0.6 π 2 π 1 = π 2 = 2.5 π 0 3
π 2 = π 0 + 0.6 π 1 π 0 + π 1 + π 2 = 1 π 0 + 2.5 π 0 + 2.5 π 0 = 1 6 π 0 = 1 π 0 = 1 6 π 1 = π 2 = 5 12 d) He gets wet, if he starts from a location without umbrellas and if it is raining: P r{get wet} = 0.6 π 0 = 0.6 1 6 = 1 10 2.4 Multiprocessor interference in shared memory systems Consider the following multiprocessor system: The memory modules can be accessed independently and in parallel. Each memory access has the same fixed length. All modules can handle at most one request per time unit. Each processor is waiting for at most one pending memory request. After the service of a memory request, each processor generates a new request to memory module i with probability q i (q i is independent of the number of the processor; this is the so-called uniform memory access assumption). Set m = n = 2. Compute with DTMCs a) the stationary probability that there is no memory access conflict, b) the stationary probability that there is a conflict in memory module 1, c) the mean number of served memory requests per time unit. Stated transition graph for shared memory system (cf. fig 5) π 1 q 2 π 1 + π 2 q 2 2 = 0 4
Figure 5: State transition graph for shared memory system π 1 q 1 + 2π 2 q 1 q 2 π 2 + π 3 q 2 = 0 π 2 q 2 1 + π 3 q 1 π 3 = 0 π 1 + π 2 + π 3 = 1 - π 3 = π 2 q 2 1 (q 1 1) a) π 2 b) π 1 π 1 = π 2 q 2 2 (q 2 1) π 2 ( q2 2 q 2 1 + 1 q2 1 q 1 1 ) = 1 c) mean number of served memory requests per time unit : π 1 1 + π 2 2 + π 3 1 5