Dynamic Control of a Tandem Queueing System with Abandonments

Transcription

1 Dynamic Control of a Tandem Queueing System with Abandonments Gabriel Zayas-Cabán 1 Jungui Xie 2 Linda V. Green 3 Mark E. Lewis 1 1 Cornell University Ithaca, NY 2 University of Science and Technology of China Hefei, China 3 Columbia University New York, NY Stochastic Networks Conference, 2016 Control of an ER TTR Process StochNets / 53

2 OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

3 Introduction Motivation OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

4 Introduction Motivation Example 1: Health clinics lower income communities or universities. 1. Patients may arrive (w/o appointment). 2. Medical service provider interviews patient to get brief medical history and symptoms (triage). 3. Same medical service provider conducts a physical examination and perhaps treatment. Control of an ER TTR Process StochNets / 53

5 Introduction Motivation Example 2: Emergency Room (ER) 1. Patient can be admitted into the hospital (not this talk), or 2. Patient can be treated and discharged. 2.1 Medical service provider does the triage, i.e. determines severity status. 2.2 Same medical service provider treats and discharges patient. Control of an ER TTR Process StochNets / 53

6 Introduction Motivation In each example we have the added twists, Customers may leave while awaiting or immediately after triage, or May leave after triage, but while waiting for treatment Control of an ER TTR Process StochNets / 53

7 Introduction Motivation In each example we have the added twists, Customers may leave while awaiting or immediately after triage, or May leave after triage, but while waiting for treatment So we have two-stage stochastic service systems, having a server(s) (medical service provider(s)) {Hajek (1984), Ahn, Duenyas, Lewis (2002), Kaufman, Ahn, Lewis (2005)}, and where we allow that customers (patients) may renege or abandon before completing service at each stage. Control of an ER TTR Process StochNets / 53

8 Introduction Motivation Main issue How should one prioritize the work by a medical service provider to balance initial delay for care with the need to discharge patients in a timely fashion? Control of an ER TTR Process StochNets / 53

9 Introduction Motivation THE LUTHERAN MEDICAL CENTER OVERVIEW Image available at downloaded June Lutheran Medical Center (LMC) Triage-Treat-and-Release (TTR) program: A full service, 468-bed academic teaching hospital in Brooklyn, New York. Offers a wide range of clinical programs to diverse communities. Control of an ER TTR Process StochNets / 53

10 Introduction Motivation THE LUTHERAN MEDICAL CENTER TTR PROGRAM 1. Patients arrive to ER and are automatically registered. 2. Patients then proceed to triage (phase one service) on a FCFS basis. 3. After triage, high severity patients who are likely to be admitted to the hospital are assigned to another part of the ER for testing and/or treatment. (renege) 4. While low severity patients await treatment, called phase-two service, in the triage area. Control of an ER TTR Process StochNets / 53

11 Queueing Model OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

12 Queueing Model All of the technical results about the queueing model appear in Gabriel Zayas-Cabán, Jingui Xie, Linda V. Green and Mark E. Lewis, Dynamic control of a tandem system with abandonments, to appear in QUESTA, Analysis of the recommended policies appear in Gabriel Zayas-Cabán, Jingui Xie, Linda V. Green and Mark E. Lewis, Policies for Physician Allocation to Triage and Treatment in Emergency Departments, working paper, Control of an ER TTR Process StochNets / 53

13 Queueing Model Model specifics: Rate λ Poisson arrival process. First-come-first-served (FCFS) phase-one service (triage). Patients wait for phase-one service for an exponentially distributed amount of time with rate β 1 before abandoning. Control of an ER TTR Process StochNets / 53

14 Queueing Model Model specifics: Rate λ Poisson arrival process. First-come-first-served (FCFS) phase-one service (triage). Patients wait for phase-one service for an exponentially distributed amount of time with rate β 1 before abandoning. After phase-one: patients leave the system (w/ probability 1 p), or patients wait for FCFS phase-two service (w/ probability p). 0 p 1. Control of an ER TTR Process StochNets / 53

15 Queueing Model Model specifics: Rate λ Poisson arrival process. First-come-first-served (FCFS) phase-one service (triage). Patients wait for phase-one service for an exponentially distributed amount of time with rate β 1 before abandoning. After phase-one: patients leave the system (w/ probability 1 p), or patients wait for FCFS phase-two service (w/ probability p). 0 p 1. Patients wait for phase-two service (treatment) according to an exponentially distributed random variable with rate β 2 before abandoning. Control of an ER TTR Process StochNets / 53

16 Queueing Model Model specifics: Rate λ Poisson arrival process. First-come-first-served (FCFS) phase-one service (triage). Patients wait for phase-one service for an exponentially distributed amount of time with rate β 1 before abandoning. After phase-one: patients leave the system (w/ probability 1 p), or patients wait for FCFS phase-two service (w/ probability p). 0 p 1. Patients wait for phase-two service (treatment) according to an exponentially distributed random variable with rate β 2 before abandoning. Service in both phases by the same set of N providers, with exponential phase-one (two) service time with rate µ 1 (µ 2 ). After phase-two service, patient leaves the system. Control of an ER TTR Process StochNets / 53

17 Queueing Model Single server two-phase stochastic service system model (β 1 = 0, β 2 = β):? Control of an ER TTR Process StochNets / 53

18 Queueing Model Single server two-phase stochastic service system model: Rate λ Poisson arrival process. First-come-first-served (FCFS) phase-one service (triage). After phase-one: patients leave the system (w/ probability 1 p), or patients wait for FCFS phase-two service (w/ probability p). 0 p 1. Control of an ER TTR Process StochNets / 53

19 Queueing Model Single server two-phase stochastic service system model: Patients wait for phase-two service (treatment) according to an exponentially distributed random variable with rate β before abandoning. Service in both phases by the same provider, with exponential phase-one (two) service time with rate µ 1 (µ 2 ). After phase-two service, patient leaves the system. Control of an ER TTR Process StochNets / 53

20 Optimal Control OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

21 Optimal Control Decision-making scenario: 1. Decision-maker (medical service provider) views number of patients at each station. Control of an ER TTR Process StochNets / 53

22 Optimal Control Decision-making scenario: 1. Decision-maker (medical service provider) views number of patients at each station. 2. Decides where to serve next, assuming preemptive service disciplines and rewards R 1 and R 2. Control of an ER TTR Process StochNets / 53

23 Optimal Control Decision-making scenario: 1. Decision-maker (medical service provider) views number of patients at each station. 2. Decides where to serve next, assuming preemptive service disciplines and rewards R 1 and R 2. Main objective Want control policies that maximize total discounted expected reward or long-run average reward of the system. Control of an ER TTR Process StochNets / 53

24 Optimal Control Proposition Suppose β 1 = β 2 = 0 and µ 1 R 1 >( ) µ 2 R 2, then under the α discounted reward, it is optimal to serve at station 1 (2) whenever station 1 (2) is not empty. In the average case, any non-idling policy is optimal. Control of an ER TTR Process StochNets / 53

25 Optimal Control Consider the following two examples. Example Suppose we have the following inputs for the model: N = 1; p = 1; λ = 3; µ 1 = 60/7; µ 2 = 60/13; β 1 = 0; β 2 = 0.3; R 1 = 15; R 2 = 20. In this case, the average rewards for prioritizing station 1 (2) are given by 92 (104). Example Suppose we have the following inputs for the model: N = 1; p = 1; λ = 3; µ 1 = 60/7; µ 2 = 60/13; β 1 = 0.3; β 2 = 0; R 1 = 20; R 2 = 10. In this case, the average rewards for prioritizing station 1 (2) are given by 84 (75). Control of an ER TTR Process StochNets / 53

26 Optimal Control Consider the following two examples. Example Suppose we have the following inputs for the model: N = 1; p = 1; λ = 3; µ 1 = 60/7; µ 2 = 60/13; β 1 = 0; β 2 = 0.3; R 1 = 15; R 2 = 20. In this case, the average rewards for prioritizing station 1 (2) are given by 92 (104). Example Suppose we have the following inputs for the model: N = 1; p = 1; λ = 3; µ 1 = 60/7; µ 2 = 60/13; β 1 = 0.3; β 2 = 0; R 1 = 20; R 2 = 10. In this case, the average rewards for prioritizing station 1 (2) are given by 84 (75). In the first example, µ 1 R 1 > µ 2 R 2 so that if β 2 = 0 we might prioritize station 1. However, doing so results in average rewards that are more than 11.5% below optimal. Similarly, in the second example we have that µ 1 R 1 < µ 2 R 2 so that β 1 = 0 implies that the second station might be prioritized. Prioritizing station 2 yields average rewards 10.6% below optimal. Control of an ER TTR Process StochNets / 53

27 Optimal Control Consider the following two examples. Example Suppose we have the following inputs for the model: N = 1; p = 1; λ = 3; µ 1 = 60/7; µ 2 = 60/13; β 1 = 0; β 2 = 0.3; R 1 = 15; R 2 = 20. In this case, the average rewards for prioritizing station 1 (2) are given by 92 (104). Example Suppose we have the following inputs for the model: N = 1; p = 1; λ = 3; µ 1 = 60/7; µ 2 = 60/13; β 1 = 0.3; β 2 = 0; R 1 = 20; R 2 = 10. In this case, the average rewards for prioritizing station 1 (2) are given by 84 (75). In the first example, µ 1 R 1 > µ 2 R 2 so that if β 2 = 0 we might prioritize station 1. However, doing so results in average rewards that are more than 11.5% below optimal. Similarly, in the second example we have that µ 1 R 1 < µ 2 R 2 so that β 1 = 0 implies that the second station might be prioritized. Prioritizing station 2 yields average rewards 10.6% below optimal. Control of an ER TTR Process StochNets / 53

28 Optimal Control Continuous-Time Markov Decision Process (CTMDP) formulation: State Space. X := {(i, j) i, j Z + }, i (j) represents number of patients at station 1 (2). Decision epochs. T := {t n, n 1}, sequence of event times. Control of an ER TTR Process StochNets / 53

29 Optimal Control Continuous-Time Markov Decision Process (CTMDP) formulation: State Space. X := {(i, j) i, j Z + }, i (j) represents number of patients at station 1 (2). Decision epochs. T := {t n, n 1}, sequence of event times. Available actions in state x = (i, j). A(x) = {(n 1, n 2 ) n 1, n 2 Z +, n 1 + n 2 N}, where n 1 (n 2 ) represents the number of servers assigned to station 1 (2). A policy prescribes how many servers should be allocated to stations 1 and 2 for all states at all times. Control of an ER TTR Process StochNets / 53

30 Optimal Control Continuous-Time Markov Decision Process (CTMDP) formulation: Reward. R i received after completing phase i service, i = 1, 2. Expected reward function. min{i, n 1 }µ 1 R 1 r((i, j), (n 1, n 2 )) = λ + min{i, n 1 }µ 1 + min{j, n 2 }µ 2 + iβ 1 + jβ 2 min{j, n 2 }µ 2 R 2 +. λ + min{i, n 1 }µ 1 + min{j, n 2 }µ 2 + iβ 1 + jβ 2 Control of an ER TTR Process StochNets / 53

31 Optimal Control Discounted expected reward of policy f when system starts in state (i, j) over the horizon of length t is given by N(t) vα,t(i, f j) = E (i,j) e αtn[ r(x(t n ), f (X(t ] n ))), n=0 α := discount rate. {N(t)} counting process. {X(s)} continuous-time Markov chain (CTMC) denoting state of the system (i.e. the number of patients at each queue). Control of an ER TTR Process StochNets / 53

32 Optimal Control Infinite horizon α discounted reward: v f α(i, j) := lim t v f α,t(i, j). Optimal reward: v α(i, j) := max v α π (i, j), π Π Π := set of all (stationary, deterministic) policies. Average reward: ρ f v0,t(i, f j) (i, j) := lim inf. t t Control of an ER TTR Process StochNets / 53

33 Optimal Control Proposition Under the α discounted reward (finite or infinite horizon) or the average reward criterion, there exists a non-idling policy that is optimal. In other words, there is an optimal policy which does not idle servers whenever there are patients waiting. Control of an ER TTR Process StochNets / 53

34 Optimal Control Proposition If the number of customers at each queue exceeds the number of servers, there exists a discounted reward optimal control policy that does not split the servers. Similarly in the average reward case. Control of an ER TTR Process StochNets / 53

35 Optimal Control A single server is a proxy for an N server model off the boundary (i, j N) Off of the boundary, do not split the servers. On the boundary of an N server model, keep as many servers busy as possible. Control of an ER TTR Process StochNets / 53

36 Optimality Equations OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

37 Optimality Equations Typically, Transition rates for CTMDP are bounded. Uniformize to convert CTMDP to equivalent Discrete-Time Markov Decision Process (DTMDP), which is easier to solve and analyze. Control of an ER TTR Process StochNets / 53

38 Optimality Equations Typically, Transition rates for CTMDP are bounded. Uniformize to convert CTMDP to equivalent Discrete-Time Markov Decision Process (DTMDP), which is easier to solve and analyze. Formulate dynamic programming optimality equations for DTMDP. Use optimality equations to solve for the optimal value and control policy (e.g. successive approximations, action elimination). Control of an ER TTR Process StochNets / 53

39 Optimality Equations For our model, Transition rates for CTMDP are unbounded: Due to abandonments. Non-uniformizable CTMDP. Control of an ER TTR Process StochNets / 53

40 Optimality Equations For our model, Transition rates for CTMDP are unbounded: Due to abandonments. Non-uniformizable CTMDP. Can still formulate dynamic programming optimality equations and have the existence of stationary optimal policies. 1 Use sample path methods to obtain structure of the control policy. 2 We almost immediately can show that there exists an optimal non-idling policy 1 : X. Guo and W. Zhu : Z. Liu, P. Nain, and D. Towsley Control of an ER TTR Process StochNets / 53

41 Optimality Equations For i, j 1 and a real-valued function on X Th(i, j) = λh(i + 1, j) + jβh(i, j 1) (λ + jβ)h(i, j) work at station 1 {}}{ + max{ µ 1 [R 1 + ph(i 1, j + 1) + (1 p)h(i 1, j) h(i, j)], µ 2 [R 2 + h(i, j 1) h(i, j)] }. }{{} work at station 2 Slight adjustments are made i or j equal zero. In the uniformizable case, the terms with a negative coefficient are collected and have a non-negative coefficient Control of an ER TTR Process StochNets / 53

42 Optimality Equations Discounted reward optimality equations (DROE), αv α = T v α. Under some minor technical assumptions, the optimal value satisfies the DROE. Similarly, under the average cost criterion, there exists a constant g and function w on the state space such that (g, w) satisfies the average reward optimality equations (AROE), g1 = T w, Control of an ER TTR Process StochNets / 53

43 Sample Path Results OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

44 Sample Path Results Theorem Suppose β 1 = 0, β 2 > 0. In this case, the following hold: 1. Under the α discounted reward criterion, if µ 2 R 2 µ 1 R 1, then it is optimal to serve at station 2 whenever station 2 is not empty. 2. Under the average reward criterion, if λ( 1 µ µ 2 +β 2 ) < 1, it is optimal to serve at station 2 whenever station 2 is not empty. Control of an ER TTR Process StochNets / 53

45 Sample Path Results A few notes: In the discounted reward case, a c-µ type condition yields optimality of a priority rule (serve at 2) In the average case, no such condition is required Control of an ER TTR Process StochNets / 53

46 Sample Path Results A few notes: In the discounted reward case, a c-µ type condition yields optimality of a priority rule (serve at 2) In the average case, no such condition is required The intuition in the average case is that customers at station 1 are guaranteed to yield reward R 1, in station 2, they may or not yield any reward. The condition guarantees stability of the prioritize station 2 policy In the discounted case, we care when we get the rewards Control of an ER TTR Process StochNets / 53

47 Sample Path Results Proof : In each case, use sample path arguments to examine where h is now v α or w. max{µ 1 [R 1 + ph(i 1, j + 1) + (1 p)h(i 1, j) h(i, j)], µ 2 [R 2 + h(i, j 1) h(i, j)]}, Depending on the optimality criterion, we have the interpretation of h(y) h(x) as the difference between starting in states y and x, under the optimal policy. Follow along processes using the optimal policy until (starting in states y and x) they couple. Control of an ER TTR Process StochNets / 53

48 Sample Path Results THE CASE WITH ABANDONMENTS ONLY FROM STATION 1 Theorem Suppose β 2 = 0, β 1 > 0. The following hold: 1. Under the α discounted reward criterion, if µ 1 R 1 µ 2 R 2, then the policy that prioritizes station 1 is optimal. 2. Under the average reward criterion, if the system is stable under the policy that prioritizes station 1, then that policy is optimal. Control of an ER TTR Process StochNets / 53

49 Sample Path Results ABANDONMENT FROM BOTH STATIONS Theorem The following hold: 1. Under the α discounted and the average reward criteria, if β 1 β 2 and µ 1 R 1 2µ 2 R 2, then it is optimal to serve at station 1 whenever station 1 is not empty. 2. Under the α discounted and the average reward criteria, if β 2 β 1 and (1 µ 2 λ+µ 2 +β 2 )µ 2 R 2 µ 1 R 1, then it is optimal to serve at station 2 whenever station 2 is not empty. Control of an ER TTR Process StochNets / 53

50 Simulation Results Back to the LMC Motivation β 2 = β; β 1 = 0 Prioritizing station 2 is optimal (when it yields a stable system) The number of MSPs is probably > 1 Control of an ER TTR Process StochNets / 53

51 Simulation Results 1. Upsides to prioritize to station 2 (call it P2) Easy to implement Choose a person and stick with her/him 2. Downsides to prioritize station 2 Restrictive stability condition: let S(µ 1, µ 2, β) := 1 µ µ 2 +β 1 The stability condition is then > λ. S(µ 1,µ 2,β) 1 What about λ < µ S(µ 1,µ 2,β) 1? Service provider must change station after every service 3. Call the prioritize station 1 policy P1. Control of an ER TTR Process StochNets / 53

52 Simulation Results Threshold Policies In a threshold policy with level K, the medical service provider works at station 2 until either station 2 is empty or, the number of patients at station 1 reaches K. Control of an ER TTR Process StochNets / 53

53 Simulation Results Threshold Policies In a threshold policy with level K, the medical service provider works at station 2 until either station 2 is empty or, the number of patients at station 1 reaches K. If the prior causes the decision-maker to change stations, then it continues to follow the prioritize station 2 policy. If the threshold causes the station change, then the decision-maker works at 1 until it is empty and then returns to prioritizing station 2. Control of an ER TTR Process StochNets / 53

54 Simulation Results Note that P2 spends the highest proportion of effort at station 2, while P1 spends the least. P1 (P2) is equivalent to setting K = 1 ( ). In between these two extremes are threshold policies with higher thresholds spending more time at station 2. Control of an ER TTR Process StochNets / 53

55 Simulation Results Numerical Study OUTLINE Introduction Motivation Model Queueing Model Optimal Control Optimality Equations Sample Path Results Simulation Results Numerical Study Concluding Remarks Control of an ER TTR Process StochNets / 53

56 Simulation Results Numerical Study Parameter Symbol Value(s) µ µ β 0.15, 0.3, 0.5, 0.8 R 2 20 N 3 p 1 R 1 10, 15 λ 6, 7, 8, 9, 10, 11, 12 Table: List of Parameters and their values Control of an ER TTR Process StochNets / 53

57 Simulation Results Numerical Study Service rate µ i, i = 1, 2 are estimated from the Lutheran Medical Center TTR. Triage is assumed to take exponential time, but treatment is assumed to follow an Erlang distribution Abandonment rate range is estimated using the average wait and an assumption on the relationship between that and the probability of leaving without being seen Only within the classes of the Emergency Severity Index covered by the TTR Following closely results of Mandelbaum and Zeltyn (2007) and Batt and Terwiesch (2013) Control of an ER TTR Process StochNets / 53

58 Simulation Results Numerical Study Service rate µ i, i = 1, 2 are estimated from the Lutheran Medical Center TTR. Triage is assumed to take exponential time, but treatment is assumed to follow an Erlang distribution Abandonment rate range is estimated using the average wait and an assumption on the relationship between that and the probability of leaving without being seen Only within the classes of the Emergency Severity Index covered by the TTR Following closely results of Mandelbaum and Zeltyn (2007) and Batt and Terwiesch (2013) Arrival rates chosen to cover ranges λ < 1 S(µ 1,µ 2,β) and 1 S(µ 1,µ 2,β) λ < µ 1 Rewards chosen to cover cases µ 1 R 1 µ 2 R 2 and µ 1 R 1 > µ 2 R 2 Control of an ER TTR Process StochNets / 53

59 Simulation Results Numerical Study Change inreward,% Change inreward,% β=0.15 K=5 Change inreward,% β=0.30 K=5 10 K=10 10 K=10 8 K=15 8 K=15 6 K=20 6 K=20 4 K= 4 K= 2 E 2 E Arrival Rate λ, hours -1 Arrival Rate λ, hours -1 β=0.50 β= K=5 12 K=5 10 K=10 10 K=10 8 K=15 8 K=15 6 K= K=20 K= 4 2 K= E 2 E Arrival Rate λ, hours -1 Arrival Rate λ, hours -1 Change inreward,% Figure: Comparison of reward for a policy over the policy that prioritizes triage (K = 1). Here, R 1 = 10 and all policies yield a stable system. Control of an ER TTR Process StochNets / 53

60 Simulation Results Numerical Study Change inreward,% Change inreward,% β=0.15 K=5 K=10 K=15 6 K=20 4 K= 2 E Arrival Rate λ, hours β=0.50 Change inreward,% β=0.30 K=5 K=10 8 K=15 6 K=20 4 K= 2 E Arrival Rate λ, hours K=5 12 K=5 10 K=10 10 K=10 8 K=15 8 K=15 6 K=20 6 K=20 4 K= 4 K= 2 E 2 E Arrival Rate λ, hours -1 Arrival Rate λ, hours -1 Change inreward,% 14 β=0.80 Figure: Comparison of reward for a policy over the policy that prioritizes triage (K = 1). Here, R 1 = 15 and all policies yield a stable system. Control of an ER TTR Process StochNets / 53

61 Simulation Results Numerical Study 10 β= β=0.30 Change inreward,% K=5 K=10 K=15 K=20 K= E Change inreward,% K=5 K=10 K=15 K=20 K= E Arrival Rate λ, hours β= Arrival Rate λ, hours β=0.80 Change inreward,% K=5 K=10 K=15 K=20 K= E Change inreward,% K=5 K=10 K=15 K=20 K= E Arrival Rate λ, hours Arrival Rate λ, hours -1 Figure: Comparison of reward for a policy over the policy that prioritizes triage (K = 1). Here, R 1 = 10 and prioritizing treatment (K = ) is unstable. Control of an ER TTR Process StochNets / 53

62 Simulation Results Numerical Study 6 β= β=0.30 Change inreward,% K=5 K=10 K=15 K=20 K= E Change inreward,% K=15 K=20 K= K=5 K=10 E Change inreward,% Arrival Rate λ, hours β=0.50 K=5 K=10 K=15 K=20 K= E Change inreward,% Arrival Rate λ, hours β=0.80 K=5 K=10 K=15 K=20 K= E Arrival Rate λ, hours Arrival Rate λ, hours -1 Figure: Comparison of reward for a policy over the policy that prioritizes triage (K = 1). Prioritizing treatment (K = ) is unstable and R 1 = 15. Control of an ER TTR Process StochNets / 53

63 Simulation Results Numerical Study Average Wait, hours Waiting Time at Triage Arrival Rate λ, hours -1 K=1 K=5 K=10 K=15 K=20 K= E Figure: Comparison of average wait. Here, R 1 = 10 and β = Control of an ER TTR Process StochNets / 53

64 Simulation Results Numerical Study Switches per provider per hour 6 Switching between Triage and Treatment K=1 5 K=5 4 K=10 3 K=15 2 K=20 1 K= E Arrival Rate λ, hours -1 Figure: Comparison of reward for a policy over the policy that prioritizes triage (K = 1). Here, R 1 = 10 and prioritizing treatment (K = ) is unstable. Control of an ER TTR Process StochNets / 53

65 Simulation Results Numerical Study Takeaways via optimal control 1. When the prioritize station 2 policy is stable Loss of optimality can fall off when the thresholds are low 2. When the prioritize station 2 policy is not stable, if you then prioritize station 1 Significant gains in terms of average reward can be obtained if we are close to the stability line, by using threshold policies But at the cost of larger queuelengths. Control of an ER TTR Process StochNets / 53

66 Concluding Remarks SUMMARY Developed queueing model to approximate triage and treatment process having a single medical service provider, motivated in part by the LMC TTR program. Used a CTMDP formulation and sample path methods to determine the (structure of the) optimal dynamic schedule for service provider: Sufficient conditions for when to prioritize phase two service (i.e. treatment) ideal scenario from patient s perspective. Control of an ER TTR Process StochNets / 53

67 Concluding Remarks SOME DRAWBACKS We have assumed that decision-maker knows the state of the system at all times; arrival process is stationary; service is exponentially distributed and preemptive. In addition, prioritize phase two service ( may ) not be ideal from provider s 1 perspective even if λ + 1 < 1 µ 1 β+µ 2 Provider spends significant amount of time switching between stations. Control of an ER TTR Process StochNets / 53