Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation

Transcription

1 Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation M.-O. Hongler, O. Gallay, R. Colmorn, P. Cordes and M. Hülsmann Ecole Polytechnique Fédérale de Lausanne (EPFL), (CH) TRANSP-OR Jacobs University - Bremen, (D) Systems Management - International Logistics EURO XXIV Lisbon - July 14 th 2010 Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

2 Smart Parts Dynamics - A Fashionable Trend in Logistics Smart Parts Dynamics - A Fashionable Trend in Logistics Highly complex decision issues tendency to decentralize the management Huge number of control parameters Feedback (i.e. non-linearity) in the underlying dynamics Ubiquitous presence of randomness in the dynamics... Decisions based on limited rationality Rigid pre-planning offers poor performance mutual interactions self-organization Autonomous agents might better perform than an effective central controller goal of today s presentation Exhibit a solvable model showing performance of decentralized control Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

3 Stylized Model for Smart Parts Dynamics A Simple Model for Competitive Dynamics Ẋ k(t) = v k(t) {z} velocity + γ k I k( X(t), X k(t)) multi agent interactions + q k(v k(t))db k,t, k = 1, 2,..., N. noise sources Multi-agent interactions: I k( X(t), X k(t)) = 1 N k N X k j k I k(x j(t)), N k := neighbourhood of agent k, 8 >< I k(x j(t)) = >: 0 if 0 X j(t) < X k(t), (velocity unchanged), 1 if X k(t) X j(t) < X k(t) + U, (U > 0), (accelerate), 0 if X j(t) > X k(t) + U, (velocity unchanged). (U := "mutual influence" interval) Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

4 Stylized Model for Smart Parts Dynamics A Simple Model for Competitive Dynamics - Applications Logistics Economy Human Mimetism... Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

5 Stylized Model for Smart Parts Dynamics Homogeneous Population of Agents h i dx k(t) = v(t) + γi( X(t), X k(t)) dt + q db k,t. indep. White Gaussian Noise := drift field D k,v (x,t) diffusion process Fokker - Planck diffusion equation: t P( x, t) = X k x k ˆDk,v( x,t) P( x, t) q2 X k 2 x 2 k [P( x, t)], P( x, t) := conditional probability density Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

6 Stylized Model for Smart Parts Dynamics Mean-Field Dynamics for Homogeneous Agents N k N Mean-Field Dynamics (MFD) dynamics for a representative effective agent trajectories point of view 1 NX I(X j(t)) N j k proportion of velocity active agents acting on k probabilistic point of view Z x+u P(x, t) dx x proportion of representative agents located in [x,x+u] Effective Fokker-Planck equation: j» v(t) + γ t P(x, t) = x Z x+u x «ff P(x, t)dx P(x, t) non linear and non local field equation q2 [P(x, t)], x2 Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

7 Stylized Model for Smart Parts Dynamics Small Influence Region - Burgers Equation Dynamics Small values of U Taylor expand up to 1 st order in U R x+u x P(x, t)dx U P(x, t) t P(x, t) = x {[v(t) + γ U P(x, t)] P(x, t)} non linear but local drift field q2 [P(x, t)] x2 t τ = γt x z = x R t 0 v(s) ds 2U Burgers Equation (to be solved with initial condition P(z, t) = δ(z)θ(z)) Ṗ(z, t) = 1 2 z [ P(z, t) 2 ] + [ q 2 8U 2 γ ] 2 z 2 [P(z, t)] Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

8 Stylized Model for Smart Parts Dynamics Burgers Eq. logarithmic transformation (Hopf - Cole ) Heat Eq. exact integration P(y, t) = q2 4γU 2 2 = 1 6 R 4 " `er y ln `er (er 1) 2 Erfc y 2 πq 2 t e q 2 t y q t 7 5 := 1 R «# y Erfc q = t (e R 1)G(y, t) E(y, t) P(y, t) R U= R U = 0.64 R U = 4 R U = 16 R U= 100 R U = 1600 Typical shape of P(y, t) for various R := 4U2 γ q 2 (viewed from the relative moving frame) factors Normalization and positivity are visually manifest!! y Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

9 Stylized Model for Smart Parts Dynamics Benefit of Competition - Noise Induced Transport Enhancement t = t = 20 1 P(y, t) t = 40 t = traveled distance y Position probability distribution: without interaction, with interactions Additional traveled distance when R = 4γU2 q 2 : X(t) t 4U 3 γt, Additional traveled distance when R = 4γU2 q 2 0: X(t) t 0. Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

10 Average Costs Estimation Optimal Effective Centralized Control Controlled diffusion process: dy t = c(y, t) dt + q db t, Y 0 = 0, (0 t T), effective central controller initial condition (Fokker-Planck equation) t Pc(y, t) = q2 2 [c(y, t)pc(y, t)] + Pc(y, t) y 2 y2 Construct a drift controller c(y, t) which, for time T, fulfills P c(y, T) Prob. density with central controller = P(y, T) Prob. density due to agent interactions Burgers exact solution Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

11 Average Costs Estimation Optimal Effective Centralized Control (continued) Introduce a utility function J central,t [c(y, t; T)] defined as: Z T J central,t [c(y, t; T)] = 0 c 2 (y, s; T) 2q 2 cost rate ρ(y,s) ds, ( := average over the realization of underlying stochastic process) Optimal Control Problem Construct an optimal drift c (y, t; T) i.e. yielding minimal cost such that: J central,t [c (y, t; T)] J central,t [c(y, t; T)] Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

12 Average Costs Estimation The Dai Pra Solution of the Optimal Control Problem Optimal drift controller: c (y, t; T) = y ln [h(y, t)], Z P(z, T) h(y, t) = G [(z y),(t t)] G(z, t) dz. R Paolo Dai Pra, "A Stochastic Control Approach to Reciprocal Diffusion Processes", Appl. Math. Optim. 23, (1991), Minimal cost: J central,t [c (y, t; T)] = 8 >< {z} N D(P G) = >: population Kullback Leibler 0 for t = 0, h i N R + N ln (e R 1) 2 R for t > 0. Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

13 Average Costs Estimation Decentralized Agent Control - Cost Estimation Cost J agents,t for decentralized evolution during time horizon T: J agents,t := Z T {z} N ρ 0 population ds Φ(s), {z} interacting agents ρ = kinetic energy z } { γ 2 U 2 /2 q 2 := individual cost rate function, {z} diffusion rate Φ(t) [0, 1] := proportion of interacting agents at time t. ******************************************************************************** Cost upper-bound, reached when Φ(t) 1 J agents,t NρT Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

14 Average Costs Estimation Costs Comparison - Centralized vs Decentralized cumulative costs upper-bounded decentralized costs actual decentralized costs centralized costs Kullback-Leibler entropy 0 Tc time horizons for which agent interactions beat the optimal effective centralized controller time horizon T Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15

15 Conclusion To Summarize and to Somehow "Philosophically" Conclude The stylized model cartoons basic and somehow "universal" features: Agents mimetic interactions produce an emergent structure - (here a "shock"- like wave), Competition enhances global transport flow - (here a t-increase of the traveled distance), Self-organization via autonomous agents interactions can reduce costs. Olivier Gallay (EPFL) Centralized Versus Decentralized Control EURO XXIV, 14/07/ / 15