The Optimizing-Simulator: Merging Optimization and Simulation Using Approximate Dynamic Programming

Transcription

1 The Optimizing-Simulator: Merging Optimization and Simulation Using Approximate Dynamic Programming Winter Simulation Conference December 5, 2005 Warren Powell CASTLE Laboratory Princeton University Warren B. Powell, Princeton University 2005 Warren B. Powell Slide 1

2 Yellow Freight System 2004 Warren B. Powell, Princeton University 2005 Warren B. Powell Slide 2

3 Yellow Freight System 2004 Warren B. Powell, Princeton University 2005 Warren B. Powell Slide 3

4 The fractional jet ownership industry 2005 Warren B. Powell Slide 4

5 NetJets Inc Warren B. Powell Slide 5

6 2005 Warren B. Powell Slide 6

7 2005 Warren B. Powell Slide 7

8 Schneider National 2005 Warren B. Powell Slide 8

9 Schneider National 2005 Warren B. Powell Slide 9

10 2005 Warren B. Powell Slide 10

11 Air Mobility Command Cargo Holding Fuel Maintenance Air Mobility Command Ramp Space Cargo Handling 2005 Warren B. Powell Slide 11

12

13 The challenges Needs for simulation:» Are we using the right mix of people and equipment?» What is the effect of new policies regarding the management of people and equipment?» What is the marginal contribution from serving customers?» What is the effect of last-minute demands on the system? 2005 Warren B. Powell Slide 13

14 The challenges We need simulation technology that accomplishes the following:» Decisions have to handle high dimensional states and actions (assigning different types of resources to different types of tasks).» The simulator has to capture behaviors that produce good behaviors not just at a point in time, but over time (decisions have to think about the future).» Performance statistics must match historical performance Warren B. Powell Slide 14

15 Outline Modeling and problem representation 2005 Warren B. Powell Slide 15

16 Modeling Resources can have a number of attributes: a = Location Equipment type Location ETA Equipment type Train priority Pool Due for maint Home shop Location ETA Bus. segment Single/team Domicile Drive hours Duty hours 8 day history Days from home Location ETA A/C type Fuel level Home shop Crew Eqpt1 Eqpt Warren B. Powell Slide 16

17 Modeling The attribute vector a t a1 a an 2 = The resource state variable R = Number of resources with attribute a at time t. ta ( ) Rt Rta a A = = Resource state variable 2005 Warren B. Powell Slide 17

18 Modeling Decision set function: D( a) = Set of decision types we can use to act on a resource with attribute a. a t a a a 1 2 = n d a t + 1 Modified resource label 2005 Warren B. Powell Slide 18

19 Modeling The modify function M ( a, W, d ) ( a, c) t 1 t t = t The information process Wt = Vector of information arriving during time interval t. Ex: new customer requests, equipment failures, weather delays Warren B. Powell Slide 19

20 Modeling Decisions x tad = Number of resources with attribute a that we can act on with decision d using the information available at time t. ( ), xt = xtad a A d D The decision function π x = X ( I ) t t t π Π =Set of decision functions (policies) Information available for making a decision 2005 Warren B. Powell Slide 20

21 Approximate dynamic programming Information and decision processes: Exogenous information process W 1 W W W W W Time x 0 x x x x x x Decisions determined by a policy 2005 Warren B. Powell Slide 21

22 Modeling System dynamics (classical view): π Given a decision function (policy) X ( S ) and exogenous information process, we can model the evolution of the state of our system using: W t t t S = f( S, X ( S ), W ) π t+ 1 t t t t Warren B. Powell Slide 22

23 Modeling X π t ( S ) t S t + t 1 x S t W t Warren B. Powell Slide 23

24 Modeling User provides: Model of physical system Our research goal: The decision function Data: Resource vector Information process Software: Decision set function ( ) Modify function M( a, d, W + 1) R t W t D a t t t Decision functions X π t ( I ) t 2005 Warren B. Powell Slide 24

25 Outline The optimizing simulator 2005 Warren B. Powell Slide 25

26 Optimizing over time Resources 2005 Warren B. Powell Slide 26

27 Optimizing over time Tasks 2005 Warren B. Powell Slide 27

28 Optimizing over time Optimizing over time t t+1 t+2 Optimizing at a point in time 2005 Warren B. Powell Slide 28

29 The optimizing simulator Classical simulation:»simple» Extremely flexible But...» Limited solution quality» Often requires extensive user defined tables to guide the simulation.» Can respond to changes in inputs in an unpredictable way. t < T??? t = 0 t = t + 1 Make decision at time t Update system state at t Warren B. Powell Slide 29

30 The optimizing simulator Optimization» Intelligent» Responds naturally to new datasets. But...» Struggles to handle complexity of real operations.» Does not model evolution of information.» Might be too intelligent? t min t t t t 1 t 1 t t t t t cx Ax B x = b Dx x u t 0 t 2005 Warren B. Powell Slide 30

31 Multicommodity flow Time Space Type 2005 Warren B. Powell Slide 31

32 The optimizing simulator To simulate or to optimize... Simulation» Strengths Extremely flexible High level of detail» Weaknesses Low level of intelligence Lower solution quality May have difficulty behaving properly with new scenarios. Difficulty adapting to random outcomes. Optimization» Strengths High level of intelligence System behaves optimally even with new datasets Reduces data set preparation.» Weaknesses Strict rules on problem structure Low level of detail Inflexible!... Why are we asking this question? 2005 Warren B. Powell Slide 32

33 Decision-making technologies Cost-based» The standard assumption of math programming.» Easily handles tradeoffs.» Easily handles high dimensions.» Can be difficult to tune to get the right behavior. Rule-based» Typically associated with AI.» Very flexible.» Difficult coding tradeoffs.» Struggles with higher dimensional states Warren B. Powell Slide 33

34 The four information classes Knowledge Kt Forecasts of exogenous events Forecasts of impacts on others Expert knowledge Ω t Vt ρ 2005 Warren B. Powell Slide 34

35 The four information classes Knowledge Kt 2005 Warren B. Powell Slide 35

36 Knowledge Rule-based: one aircraft and one requirement Aircraft Requirements California Taiwan Germany England New Jersey New Jersey Colorado 2005 Warren B. Powell Slide 36

37 Knowledge Cost based: one requirement and multiple aircraft Aircraft Requirements California Taiwan Germany England New Jersey New Jersey Colorado 2005 Warren B. Powell Slide 37

38 Knowledge Costs allow you to make tradeoffs: California Germany Issue Repositioning cost Appropriate a/c type Utilization Requires modifications Special maintenance at airbase Total cost cost / bonus -$17, Warren B. Powell Slide 38

39 Knowledge Cost based: multiple requirements and aircraft Aircraft Requirements California Taiwan Germany England New Jersey New Jersey Colorado 2005 Warren B. Powell Slide 39

40 The information classes Knowledge Kt Forecasts of exogenous events Ω t 2005 Warren B. Powell Slide 40

41 Forecasts of exogenous information Resources that are known now Aircraft Requirements California Taiwan Germany England New Jersey New Jersey Colorado X π ( I) involves solving a linear program/network model Warren B. Powell Slide 41

42 Forecasts of exogenous information Resources that are known now California California Taiwan England Taiwan New Jersey England Aircraft Aircraft Requirements Germany New Germany Jersey Colorado New Jersey New Jersey Colorado X π ( I) involves solving a linear program/network model Warren B. Powell Slide 42

43 Forecasts of exogenous information ( ) R t R > = t ' t' t and are forecasted for the future. = California Taiwan England New Jersey California Taiwan England New Jersey Aircraft Requirements Germany New Jersey Colorado Germany New Jersey Colorado 2005 Warren B. Powell Slide 43

44 The information classes The Information classes Knowledge Kt Forecasts of exogenous events Forecasts of impacts on others Ω t Vt 2005 Warren B. Powell Slide 44

45 Approximate dynamic programming Decisions now may need to know the impact on future decisions:» What is the cost of assigning this type of aircraft to move a requirement?» What is the value of having a certain number of aircraft in a region?» Should this requirement be satisfied now? Later? Never? For these questions, it is important that we optimize over time Warren B. Powell Slide 45

46 Time t V(a ) V(a ) a

47 Time t V( a ) ' 1 V( a ) ' 2 a 1 a 2

48 The optimization challenge? 2005 Warren B. Powell Slide 48

49 State variables Systems evolve through a cycle of exogenous and endogenous information ω = ˆR 1 ˆR ˆR ˆR ˆR ˆR Time x 0 x x x x x x Warren B. Powell Slide 49

50 State variables Systems evolve through a cycle of exogenous and endogenous information ˆR 1 ˆR ˆR ˆR ˆR ˆR Time x 0 x x x x x x R 0 R R R R R R Warren B. Powell Slide 50

51 Approximate dynamic programming Using this state variable, we obtain the optimality equations: { } V ( R ) = max C ( R, x ) + E V ( R ) R t t t t t t+ 1 t+ 1 t x X Three curses Problem: Curse of dimensionality State space Outcome space Action space (feasible region) 2005 Warren B. Powell Slide 51

52 Approximate dynamic programming The computational challenge: { } V ( R ) = max C ( R, x ) + E V ( R ) R t t t t t t+ 1 t+ 1 t x X How do we find V ( R )? t+ 1 t+ 1 How do we compute the expectation? How do we find the optimal solution? 2005 Warren B. Powell Slide 52

53 Approximate dynamic programming A possible approximation strategy: We start with: { ( ) } 1 1 V ( R ) = max C ( R, x ) + E V R R t t t t t t+ t+ t x t We solve this for a sample realization: Can t compute this!!! ( ω ) V ( R, ω) = max C ( R, x ) + V R ( ) x t t t t t t+ 1 t+ 1 t Now substitute in function approximations: V ( R, ω) = max C ( R, x ) + V R ( ) x ( ω ) t t t t t t+ 1 t+ 1 t Need to approximate V Don t know what this is! 2005 Warren B. Powell Slide 53

54 Approximate dynamic programming One big problem. V ( R, ω) = max C ( R, x ) + V R ( ) x ( ω ) t t t t t t+ 1 t+ 1 t Seeing is cheating! R t Warren B. Powell Slide 54

55 Approximate dynamic programming Alternative: Change the definition of the state variable: ˆR 1 ˆR ˆR ˆR ˆR ˆR Time x 0 x x x x x x R R R R R R R R R R R3 R R R R6 6 R R Warren B. Powell Slide 55

56 Approximate dynamic programming Now our optimality equation looks like: Expectation outside of the max operator. { } ω V ( R ) = E max C ( R, x ) + V ( R ( x, )) R x x x t 1, t t 1 x X t t t t t t t 1 t We drop the expectation and solve the conditional problem: Finally, we substitute in our approximation: ( ) V ( R, Rˆ ( ω)) = max C ( R ( ω), x ( ω)) + V R x, ω ( x ) x t 1 t 1 t x( ω) X ( ω) t t t t t t Post-decision state variable ( ) V ( R, Rˆ ( ω)) = max C ( R ( ω), x ( ω)) + V R x, ω ( x ) x t 1 t 1 t x( ω) X ( ω) t t t t t t Convenient value function approximation Warren B. Powell Slide 56

57 Approximate dynamic programming Approximating the value function:» We choose approximations of the form: Linear (in the resource state): V ( R ) = v R t t ta ta a A Piecewise linear, separable: V ( R ) = V ( R ) t t ta ta a A Best when assets are complex, R ta which means that is small (typically 0 or 1). Best when assets are simple, which means that may be larger. R ta 2005 Warren B. Powell Slide 57

58 Approximate dynamic programming A myopic decision rule (policy): x = arg max C ( R ( ω), x ( ω)) n t x( ω) X ( ω) t t t A decision rule that looks into the future: ( ) ( x ) x = arg max C ( R ( ω), x ( ω)) + V R x, ω n t x( ω) X ( ω) t t t t t t 2005 Warren B. Powell Slide 58

59 Approximate dynamic programming Simulating a myopic policy: t t+1 t Warren B. Powell Slide 59

60 Approximate dynamic programming A myopic decision rule (policy): x = arg max C ( R ( ω), x ( ω)) n t x( ω) X ( ω) t t t A decision rule that looks into the future: ( ) ( x ) x = arg max C ( R ( ω), x ( ω)) + V R x, ω n t x( ω) X ( ω) t t t t t t 2005 Warren B. Powell Slide 60

61 Approximate dynamic programming V( a ) ' 1 a 1 a 2 V( a ) ' Warren B. Powell Slide 61

62 2005 Warren B. Powell Slide 62

63 Classification yards Option 1: Send directly to customers Option 2: Send to regional depots Option 3: Send to classification yards

64 Approximate dynamic programming Two-stage resource allocation under uncertainty 2005 Warren B. Powell Slide 64

65 Approximate dynamic programming We obtain piecewise linear recourse functions for each regions Warren B. Powell Slide 65

66 Approximate dynamic programming The function is piecewise linear on the integers. Profits We approximate the value of cars in the future using a separable approximation Number of vehicles at a location 2005 Warren B. Powell Slide 66

67 Approximate dynamic programming To capture nonlinear behavior: Each link captures the marginal reward of an additional car Warren B. Powell Slide 67

68 Approximate dynamic programming 2005 Warren B. Powell Slide 68

69 Approximate dynamic programming 2005 Warren B. Powell Slide 69

70 Approximate dynamic programming n R1 n R2 n R3 n R4 n R Warren B. Powell Slide 70

71 Approximate dynamic programming We estimate the functions by sampling from our distributions. Marginal value: ( n v ) 1 ω ( n v ) 2 ω ( n v ) 3 ω n R1 n R2 n R3 ( n D ) 1 ω n D3 ( ω ) ( n D ) 2 ω ( n v ) 4 ω ( n v ) 5 ω n R4 n R5 DC n ( ω ) 2005 Warren B. Powell Slide 71

72 Approximate dynamic programming The time t subproblem: V ( R, R, R ) n ta t1 t 2 t3 t (i-1,t+3) Gradients: ( vˆ, vˆ ) n n+ t1 t1 ( vˆ, vˆ ) n n+ t2 t2 ( vˆ, vˆ ) n n+ t3 t3 R t1 R t 2 R t3 (i,t+1) (i+1,t+5) 2005 Warren B. Powell Slide 72

73 Approximate dynamic programming Left and right gradients are found by solving flow augmenting path problems. V ( R, R, R ) n ta t1 t 2 t3 Gradients: t (i-1,t+3) i ( ˆ ) n v + t3 R t3 The The right right derivative derivative (the (the value value of of one one more more unit unit of of that that resource) resource) is is a a flow flow augmenting augmenting path path from from that that node node to to the the supersink. supersink Warren B. Powell Slide 73

74 Approximate dynamic programming Left and right derivatives are used to build up a nonlinear approximation of the subproblem. V k it ( R ) 1t k R 1t R 1t 2005 Warren B. Powell Slide 74

75 Approximate dynamic programming Left and right derivatives are used to build up a nonlinear approximation of the subproblem. V k it ( R ) 1t Left derivative k v t Right derivative k v + t k R 1t R 1t 2005 Warren B. Powell Slide 75

76 Approximate dynamic programming Each iteration adds new segments, as well as refining old ones. V k it ( R ) 1t ( k 1) v + t ( k 1) v + + t k+1 R 1t R 1t 2005 Warren B. Powell Slide 76

77 Approximate dynamic programming 2.5 Approximate value function Functional Value, f(s) = ln(1+s) Exact 1 Iter 2 Iter 5 Iter 10 Iter 15 Iter 20 Iter Number Variable of Value, resources s 2005 Warren B. Powell Slide 77

78 Approximate dynamic programming Simulating a myopic policy t 2005 Warren B. Powell Slide 78

79 Approximate dynamic programming Simulating a myopic policy 2005 Warren B. Powell Slide 79

80 Approximate dynamic programming Using value functions to anticipate the future t Here and now Downstream impacts 2005 Warren B. Powell Slide 80

81 Approximate dynamic programming Using value functions to anticipate the future 2005 Warren B. Powell Slide 81

82 Approximate dynamic programming Using value functions to anticipate the future 2005 Warren B. Powell Slide 82

83 Approximate dynamic programming Using value functions to anticipate the future 2005 Warren B. Powell Slide 83

84 2005 Warren B. Powell Slide 84

85 2005 Warren B. Powell Slide 85

86 2005 Warren B. Powell Slide 86

87 2005 Warren B. Powell Slide 87

88 Approximate DP vs. LP Approximate dynamic programming % of Objective Upperbound The mathematical optimum Agg_PWLinear_1 Agg_PWLinear_2 Agg_PWLinear_3 DisAgg_Linear DisAgg_PWLinear Decomp_Location Iteration No Warren B. Powell Slide 88

89 Downloadable at Warren B. Powell Slide 89

90 The information classes Knowledge Kt Forecasts of exogenous events Forecasts of impacts on others Expert knowledge Ω t Vt ρ 2005 Warren B. Powell Slide 90

91 Low dimensional patterns Old modeling approach: Engineering costs Behavior Objectives x * = arg min cx Subject to : Ax = b, x 0 Physics 2005 Warren B. Powell Slide 91

92 Flows from history 2005 Warren B. Powell Slide 92

93 Flows from history Flows from the model 2005 Warren B. Powell Slide 93

94 Low dimensional patterns Bottom up/top down modeling: Patterns Specify the behaviors you want at a general level. Specify costs, driver availability, work rules, routing preferences, load avail. Engineering 2005 Warren B. Powell Slide 94

95 Low dimensional patterns Pattern matching Behavior Cost function * x cx H x = arg min +θ (, ρ) The happiness function measures the degree to which model behavior agrees with a knowledgeable expert. Hx (, ρ) = Gx ( ) ρ where Gx ( ) is an aggregation function 2005 Warren B. Powell Slide 95

96 Low dimensional patterns Patterns and aggregation:» What we do: We define patterns based on an aggregation of the attributes of a single vehicle. Patterns indicate the desirability of a single decision.» Patterns can be expressed at different levels of aggregation, simultaneously. Don t send C-5 s into Saudi Arabia Don t send C-5 s needing maintenance into Saudi Arabia Don t send C-5 s needing maintenance loaded with freight to southeast Asia into Saudi Arabia.» Patterns are not hard rules they express desirable or undesirable patterns of behavior Warren B. Powell Slide 96

97 Flows from history Flows from the model 2005 Warren B. Powell Slide 97

98 Flows from history Flows from the model 2005 Warren B. Powell Slide 98

99 Low dimensional patterns Length of haul calibration-teams With pattern Min Solo w/ pattern Solo w/o pattern Max 650 Without pattern Iteration 2005 Warren B. Powell Slide 99

100 Low dimensional patterns Patterns can come from history: 2005 Warren B. Powell Slide 100

101 Low dimensional patterns or an expert: 2005 Warren B. Powell Slide 101

102 The information classes Knowledge Kt Forecasts of exogenous events Forecasts of impacts on others Expert knowledge Ω t Vt ρ 2005 Warren B. Powell Slide 102

103 The military airlift problem 2005 Warren B. Powell Slide 103

104 Optimizing simulator Increasing information sets Policy Rule-based Myopic cost-based, one requirement to a list of aircraft, known now and actionable now Myopic cost-based, one requirement to a list of aircraft, known now and actionable in the future Myopic cost-based, a list of requirements to a list of aircraft, known now and actionable now Myopic cost-based, a list of requirements to a list of aircraft, known now and actionable in the future Rolling horizon Approximate Dynamic Programming Expert knowledge Information Classes I t = R tt I t = ( Rtt, ct ) I t = (( Rtt ) t t, ct ) I t = ( Rtt, ct ) I I I I t = (( Rtt ) t t, ct ) t = {( Rt ' t'' ) t'' t', ct t t = {( R ), c, V tt t t t tt t T t T = {( R ), c, V, ρ t T tt t t t tt ph t ph t ph t } } } Decision Functions (RB:R-A) (MP:R- AL/KNAN) (MP:R- AL/KNAF) (MP:RL- AL/KNAN) (MP:RL- AL/KNAF) (RH) (ADP) (EK) 2005 Warren B. Powell Slide 104

105 Optimizing simulator Costs of different policies Million Dollors Total cost Transportation cost Late delivery cost Repair cost 50 0 Rule Based (RB:R-A) Choice of aircraft Actionable Now Policies Actionable future (MP:RL-AL/ KNAN) Value functions (ADP) Increasing information sets 2005 Warren B. Powell Slide 105

106 Optimizing simulator Millions Throughput curves of policies Increasing information sets Pounds Cumulative expected thruput (RB:R-A) (MP:R-AL/KNAN) (MP:RL-AL/KNAN) (MP:RL-AL/KNAF) (ADP) Time periods 2005 Warren B. Powell Slide 106

107 Optimizing simulator Millions Throughput curves of policies Pounds Cumulative expected thruput (RB:R-A) (MP:R-AL/KNAN) (MP:RL-AL/KNAN) (MP:RL-AL/KNAF) (ADP) Time periods 2005 Warren B. Powell Slide 107

108 Optimizing simulator Areas between the cumulative expected thruput curve and different policy thruput curves Millions Pound * days (RB:R-A) (MP:R-AL/KNAN) (ADP) (MP:RL-AL/ KNAF) (MP:RL-AL/ KNAN) Policy Increasing information sets 2005 Warren B. Powell Slide 108

109 Outline Recent experiments with modeling airlift operations 2005 Warren B. Powell Slide 109

110 Random demands and equipment failures 2005 Warren B. Powell Slide 110

111 Pilots Aircraft Customers 2005 Warren B. Powell Slide 111

112 Case study Questions:» What is the effect of uncertain demands on a military airlift schedule?» What is the effect of equipment failures?» How does adaptive learning change the effect of randomness on the performance of the simulation?» What is the effect of advance information? 2005 Warren B. Powell Slide 112

113 Total contribution Iterative learning Determ demand No Break Learn Determ demand Break Learn Random demand No break Learn Determ demand No Break No learn Random demand Break Learn Determ demand Break No learn Random demand No Break No learn Random demand Break No learn 2005 Warren B. Powell Slide 113

114 Deterministic demands, no failures With learning Without learning Determ demand No Break Learn Determ demand Break Learn Random demand No break Learn Determ demand No Break No learn Random demand Break Learn Determ demand Break No learn Random demand No Break No learn Random demand Break No learn 2005 Warren B. Powell Slide 114

115 Deterministic demands, with failures With learning Without learning Determ demand No Break Learn Determ demand Break Learn Random demand No break Learn Determ demand No Break No learn Random demand Break Learn Determ demand Break No learn Random demand No Break No learn Random demand Break No learn 2005 Warren B. Powell Slide 115

116 Random demands, no failures With learning Without learning Determ demand No Break Learn Determ demand Break Learn Random demand No break Learn Determ demand No Break No learn Random demand Break Learn Determ demand Break No learn Random demand No Break No learn Random demand Break No learn Warren B. Powell Slide 116

117 Random demands, with failures With learning Without learning Determ demand No Break Learn Determ demand Break Learn Random demand No break Learn Determ demand No Break No learn Random demand Break Learn Determ demand Break No learn Random demand No Break No learn Random demand Break No learn 2005 Warren B. Powell Slide 117

118 Effect of advance booking Effect of advance notice Percent coverage Without learning Prebook 0 hours Prebook 2 hours Prebook 6 hours 2005 Warren B. Powell Slide 118

119 Effect of advance booking Effect of advance notice Percent coverage With learning Without learning Prebook 0 hours Prebook 2 hours Prebook 6 hours 2005 Warren B. Powell Slide 119

120 Midair refueling: initial solution 2005 Warren B. Powell Slide 120

121 Midair refueling: initial solution Path followed by tanker (moves up and down Atlantic) Warren B. Powell Slide 121

122 Midair refueling: initial solution First plane refuels Second plane crashes Green: full of fuel Yellow to red: nearing empty Black: empty (plane crashes) 2005 Warren B. Powell Slide 122

123 Midair refueling: exploration Learning over many iterations Warren B. Powell Slide 123

124 Midair refueling: final solution Planes learn to meet in the middle so both can refuel Warren B. Powell Slide 124

125 Outline Calibrating a model for a major truckload motor carrier 2005 Warren B. Powell Slide 125

126 Schneider National 2005 Warren B. Powell Slide 126

127 Schneider National 2005 Warren B. Powell Slide 127

128 2005 Warren B. Powell Slide 128

129 Truckload trucking Questions for the model:» What types of drivers should they hire? Domicile? Single drivers vs. teams?» What is the value of knowing about customer requests farther in the future?» What is the profitability of different customers?» What is the value of increasing terminal capacity? 2005 Warren B. Powell Slide 129

130 Truckload trucking LOH LOH Historical maximum Simulation Historical minimum US_SOLO US_IC US_TEAM Capacity category 2005 Warren B. Powell Slide 130

131 Truckload trucking Revenue per WU Historical maximum Simulation Historical minimum 1200 Utilization US_SOLO US_IC US_TEAM Capacity category Revenue per WU Utilization Historical maximum Simulation Historical minimum US_SOLO US_IC US_TEAM Capacity category 2005 Warren B. Powell Slide 131

132 Truckload trucking Challenge» We want to know the marginal value of each type of driver.» A driver type is determined by: Location 100 a = Domicile 100 = Driver type 3» There are 30,000 driver types!!!» We need to take the derivative of our simulation for each type Warren B. Powell Slide 132

133 Multistage problems vˆn t 1 t X π t ( R ) t t +1 t + 2 Time ˆn2 v t Resource State-Type ˆn3 v t 2005 Warren B. Powell Slide 133

134 Multistage problems v ˆn t + 1,1 t +1 X π ( R ) t+ 1 t+ 1 t + 2 Time v ˆn t + 1,2 Resource State-Type v ˆn t + 1, Warren B. Powell Slide 134

135 Multistage problems v ˆn t + 2,1 t + 2 X π ( R ) t+ 2 t+ 2 Time v ˆn t + 2,2 Resource State-Type v ˆn t + 2, Warren B. Powell Slide 135

136 Multistage problems vˆn t 1 t X π t ( R ) t t +1 t + 2 Time ˆn2 v t Resource State-Type ˆn3 v t 2005 Warren B. Powell Slide 136

137 Multistage problems v ˆn t + 1,1 t +1 X π ( R ) t+ 1 t+ 1 t + 2 Time v ˆn t + 1,2 Resource State-Type v ˆn t + 1, Warren B. Powell Slide 137

138 Multistage problems v ˆn t + 2,1 t + 2 X π ( R ) t+ 2 t+ 2 Time v ˆn t + 2,2 Resource State-Type v ˆn t + 2, Warren B. Powell Slide 138

139 Backward pass X π t ( R ) t X π ( R ) X ( R ) t+ 1 t+ 1 t+ 2 t+ 2 π 2005 Warren B. Powell Slide 139

140 Backward pass t + 2 Time v ˆn t + 2,1 Resource State-Type 2005 Warren B. Powell Slide 140

141 Backward pass t +1 t + 2 Time v ˆn t + 1,2 Resource State-Type 2005 Warren B. Powell Slide 141

142 Backward pass t t +1 t + 2 Time Resource State-Type ˆn3 v t 2005 Warren B. Powell Slide 142

143 Backward pass t t +1 t + 2 Time Resource State-Type ˆn3 v t 2005 Warren B. Powell Slide 143

144 Driver fleet optimization simulation objective function resources +40 resources +60 resources +50 resources +20 resources +10 resources Base +5 case resources # of drivers s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 avg pred 2005 Warren B. Powell Slide 144

145 Driver fleet optimization simulation objective function # of drivers s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 avg pred 2005 Warren B. Powell Slide 145

146 Driver fleet optimization simulation objective function v a # of drivers s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 avg pred 2005 Warren B. Powell Slide 146

147 Driver fleet optimization Driver types 2005 Warren B. Powell Slide 147

148 Add drivers 2005 Warren B. Powell Slide 148

149 Reduce drivers 2005 Warren B. Powell Slide 149

150 2005 Warren B. Powell Slide 150

151 Questions?