MATH 446/546 Test 2 Fall 204 Note the problems are separated into two sections a set for all students and an additional set for those taking the course at the 546 level. Please read and follow all of these directions. Answer all the questions appropriate to level at which you are taking the course. You may use any calculator you have at your disposal (please not your phone). I have allowed you access to the IUP sage server if you would like to use it as a calculator. Show all of your work to get full credit. This includes algebraic expressions in order to get foll credit. Note this specifically means that you should write down symbolic matrix formulations of the computations you are doing with Sage and Matlab. Write a sentence answer for each of the word problem parts stated. Answer all questions neatly. Work alone! NAME:
All Students. Given the probability transition matrix for three states: 0. p = 0.7 0.3 0.0 0.2 0.05 0.75 (5 pts) Calculate the probability that starting from state in 4 steps the Markov chain ends in state 3. Here we look at p 4 = 0.3249 0.24545 2965 0.3038 0.2979 0.3983 0.264625 0.20825 2520625 and use the entry in row column 3. Thus the probability of starting in state that we are in state 3 after 4 steps of the chain is 2965. (5 pts) What is the steady state distribution for the given Markov chain? Here we are looking at finding a vector π such that: Thus, we solve π = πp (P T I)π = 0 with one of the rows replaced by the constraint that π i =. This gives: 0.9 0.7 0.2 0.7 0.05.0.0.0 π T = Solving for π yields the steady state distribution: π = (0.29666666667, 0.24666666667, 66666666667) Note this is very similar to the rows of p raised to a high power: 0.29666666667 0.24666666667 66666666667 p 00 = 0.29666666667 0.24666666667 66666666667. 0.29666666667 0.24666666667 66666666667 0 0 2
2. (5 pts) A police car is on patrol in a neighborhood known for its gang activities. During a patrol, there is a 60% chance of responding time to the location where help is needed, else regular patrol will continue. Upon receiving a call, there is a 0% chance for cancellation (in which case normal patrol is resumed) and a 30% chance that the car is already responding to a previous call. When the police car arrives at the scene, there is a 0% chance that the instigators will have fled (in which case the car returns back to patrol) and a 40% chance that apprehension is made immediately. Else, the officers will search the area. If apprehension occurs, there is a 60% chance of transporting the suspects to the police station, else they are released and the car returns to patrol. Express the probabilistic activities of the police patrol in the form of a diagram, and then probability transition matrix. The trick to this problem is to consider all the possible states. Thus, the car can be in any of the following five places described in the problem: Patrol-l, Call-2, Scene-3, Apprehend Transit-4, Station-5 The following diagram depicts the different transitions in the Markov Chain. Patrol 0.6 Station 0. Call 0. Fled 0.6 0.3 0.6 Apprehend Scene Search Using the numbering scheme allows for the following probability transition matrix to be created. 0.6 0.0 0.0 0.0 0. 0.3 0.6 0.0 0.0 P = 0. 0.0 0.0 0.0 0.0 0.0 0.6.0 0.0 0.0 0.0 0.0 3
3. Patients suffering form kidney failure can either get a transplant or undergo periodic dialysis. During any one year, 30% undergo cadaveric transplants, and 0% receive living-donor kidneys. In the year following a transplant, 30% of the cadaveric transplants and 5% of living-donor recipients go back to dialysis. Death percentages among the two groups are 20% and 0%, respectively. Of those in the dialysis pool, 0% die, and of those who survive more than one year after transplant, 5% die and 5% go back to dialysis. The following diagram represents this situation graphically: 0.3 0.3 Dialysis : 0. 0. 0.05 Cadaveric Transplant : 2 0.2 Living Doner Transplant : 3 0. Good Post-Transplant Year : 4 0.05 Death: 5 0.9 0.75 (a) (6 pts) What is a valid transition matrix for the described Markov Chain? The transition matrix for the given situation using the state numbers as given in the diagram is as follows: 0.3 0. 0.0 0. 0.3 0.0 0.0 0.2 P = 0.0 0.0 0.75 0. 0.05 0.0 0.0 0.9 0.05 0.0 0.0 0.0 0.0.0 (b) (5 pts) What is the expected number of years a patient stays on dialysis. The expected number of visits to state j starting in state i is given by element (i, j) in (I N) where when given probability transition matrix P we calculate N by removing the absorbing state. This gives N = 0.3 0. 0.0 0.3 0.0 0.0 0.0 0.0 0.75 0.05 0.0 0.0 0.9 4
and (I N) = 3.53982300885.0694690265 0.353982300885 7.964607699.94690265487.58407079646 0.94690265487 9.38053097345.85840707965 5752223894.8584070796.68459292.769950442 3097345327 0.769950442 3.982300885 Here it can see that a patient will stay on dialysis approximately 3.539 years. (c) (5 pts) What is the longevity of a patient who starts on dialysis? The expected time to absorption may be found by summing the values in the (I N) matrix across any row representing the number of transitions till absorption when starting in state i. (I N) = 2.9203539823 3.06946903 5.283858407 6.46076992 The expected number of years until someone who starts on dialysis dies is 2.92 years. (d) (5 pts) What is the life expectancy of a patient who survives year or longer after a transplant. Here we can use the same work we had form the previous problem. Noting that the entry in row 4 corresponding to having a good post-transplant year is approximately 6.46. Thus, we expect a patient who survives year or longer after a transplant to live 6.46 more years. (e) (5 pts) The expected number of years before an at-least--year transplant survivor goes back to dialysis or dies. Here we can consider the matrix (I N) that gives the expected time in state j starting in state i. Considering the row 4 column 4 entry this is the expected time in a post-transplant good year prior to death or going back on dialysis. Specifically we see it is expected that a person will spend 3.98 years in this particular state. To answer the question as worded we need to consider going back on dialysis as an absorbing state and place it last in the transition matrix. Mapping the states in the transition matrix as follows: 2, 3, 4, 5, Thus the new probability transition matrix is: 0.0 0.0 0.2 0.3 0.0 0.0 0.75 0. P = 0.0 0.0 0.9 0.05 0.05 0.0 0.0 0.0.0 0.0 0.0 0.0 0.0 0.0.0 5.
and can be separated into the following: 0.0 0.0 N = 0.0 0.0 0.75 and A = 0.0 0.0 0.9 The expected time to absorption is given by: 6.0 (I N) = 8.5 0.0 0.2 0.3 0. 0.05 0.05 and starting in the state corresponding to a good post transplant year we expect 0 years before death or dialysis. 6
545 Additional Problems. (0 pts) A doubly stochastic Markov chain with m states in the state space S is one where p ij = j S. Show that the vector ( m, m,..., ) m is a stationary distribution. Here we aim to show that for a matrix P with the conditions that: p ij = j S ( columns sum to ) and that p ij = i S ( rows sum to ), j S π = ( m, m,..., m ) satisfies π = πp. Proof: Let π i = m for any i S. Then consider the jth element of the row vector πp given by: π i p ij = m p ij = p ij m = m = π j Thus, π = = π = πp. ( m, m,..., ) m is the stationary distribution as was to be shown. 7