Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation Logistics

Size: px
Start display at page:

Download "Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation Logistics"

Transcription

1 Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation Logistics O. Gallay, M.-O. Hongler, R. Colmorn, P. Cordes and M. Hülsmann Ecole Polytechnique Fédérale de Lausanne (EPFL), (CH) TRANSP-OR, Transport and Mobility Laboratory Jacobs University - Bremen, (D) Systems Management - International Logistics Eighth Joint Operations Research Days Fribourg - September 9 th 2010 Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

2 Smart Parts Dynamics - A Fashionable Trend in Logistics Smart Parts Dynamics - A Fashionable Trend in Logistics Highly complex decision issues tendency to decentralize the management Huge number of control parameters Feedback (i.e. non-linearity) in the underlying dynamics Ubiquitous presence of randomness in the dynamics... Decisions based on limited rationality Rigid pre-planning offers poor performance mutual interactions self-organization Autonomous agents might better perform than an effective central controller goal of today s presentation Exhibit a solvable model showing performance of decentralized control Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

3 Stylized Model for Smart Parts Dynamics A Simple Model for Local Imitation Dynamics Ẋ k(t) = v k(t) {z} velocity + γ k I k( X(t), X k(t)) multi agent interactions + q k(v k(t))db k,t, k = 1, 2,..., N. noise sources Multi-agent interactions: I k( X(t), X k(t)) = 1 N k N X k j k I k(x j(t)), N k := neighbourhood of agent k, 8 >< I k(x j(t)) = >: 0 if 0 X j(t) < X k(t), (velocity unchanged), 1 if X k(t) X j(t) < X k(t) + U, (U > 0), (accelerate), 0 if X j(t) > X k(t) + U, (velocity unchanged). (U := "mutual influence" interval) Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

4 Stylized Model for Smart Parts Dynamics A Simple Model for Imitation Dynamics - Applications Logistics Economy Human Mimetism... Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

5 Stylized Model for Smart Parts Dynamics Homogeneous Population of Agents h i dx k(t) = v(t) + γi( X(t), X k(t)) dt + q db k,t. indep. White Gaussian Noise := drift field D k,v (x,t) diffusion process Fokker - Planck diffusion equation: t P( x, t) = X k x k ˆDk,v( x,t) P( x, t) q2 X k 2 x 2 k [P( x, t)], P( x, t) := conditional probability density Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

6 Stylized Model for Smart Parts Dynamics Mean-Field Dynamics for Homogeneous Agents N k N Mean-Field Dynamics (MFD) dynamics for a representative effective agent trajectories point of view 1 NX I(X j(t)) N j k proportion of velocity active agents acting on k probabilistic point of view Z x+u P(x, t) dx x proportion of representative agents located in [x,x+u] Effective Fokker-Planck equation: j» v(t) + γ t P(x, t) = x Z x+u x «ff P(x, t)dx P(x, t) non linear and non local field equation q2 [P(x, t)], x2 Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

7 Stylized Model for Smart Parts Dynamics Small Influence Region - Burgers Equation Dynamics Small values of U Taylor expand up to 1 st order in U R x+u x P(x, t)dx U P(x, t) t P(x, t) = x {[v(t) + γ U P(x, t)] P(x, t)} non linear but local drift field q2 [P(x, t)] x2 t τ = γt x z = x R t 0 v(s) ds 2U Burgers Equation (to be solved with initial condition P(z, t) = δ(z)θ(z)) Ṗ(z, t) = 1 2 z [ P(z, t) 2 ] + [ q 2 8U 2 γ ] 2 z 2 [P(z, t)] Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

8 Stylized Model for Smart Parts Dynamics Burgers Eq. logarithmic transformation (Hopf - Cole ) Heat Eq. exact integration P(y, t) = q2 4γU 2 2 = 1 6 R 4 " `er y ln `er (er 1) 2 Erfc y 2 πq 2 t e q 2 t y q t 7 5 := 1 R «# y Erfc q = t (e R 1)G(y, t) E(y, t) P(y, t) R U= R U = 0.64 R U = 4 R U = 16 R U= 100 R U = 1600 Typical shape of P(y, t) for various R := 4U2 γ q 2 (viewed from the relative moving frame) factors Normalization and positivity are visually manifest!! y Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

9 Stylized Model for Smart Parts Dynamics Benefit of Competition - Noise Induced Transport Enhancement t = t = 20 1 P(y, t) t = 40 t = traveled distance y Position probability distribution: without interaction, with interactions Additional traveled distance when R = 4γU2 q 2 : X(t) t 4U 3 γt, Additional traveled distance when R = 4γU2 q 2 0: X(t) t 0. Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

10 Average Costs Estimation Optimal Effective Centralized Control Controlled diffusion process: dy t = c(y, t) dt + q db t, Y 0 = 0, (0 t T), effective central controller initial condition (Fokker-Planck equation) t Pc(y, t) = q2 2 [c(y, t)pc(y, t)] + Pc(y, t) y 2 y2 Construct a drift controller c(y, t) which, for time T, fulfills P c(y, T) Prob. density with central controller = P(y, T) Prob. density due to agent interactions Burgers exact solution Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

11 Average Costs Estimation Optimal Effective Centralized Control (continued) Introduce a utility function J central,t [c(y, t; T)] defined as: Z T J central,t [c(y, t; T)] = 0 c 2 (y, s; T) 2q 2 cost rate ρ(y,s) ds, ( := average over the realization of underlying stochastic process) Optimal Control Problem Construct an optimal drift c (y, t; T) i.e. yielding minimal cost such that: J central,t [c (y, t; T)] J central,t [c(y, t; T)] Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

12 Average Costs Estimation The Dai Pra Solution of the Optimal Control Problem Optimal drift controller: c (y, t; T) = y ln [h(y, t)], Z P(z, T) h(y, t) = G [(z y),(t t)] G(z, t) dz. R Paolo Dai Pra, "A Stochastic Control Approach to Reciprocal Diffusion Processes", Appl. Math. Optim. 23, (1991), Minimal cost: J central,t [c (y, t; T)] = 8 >< {z} N D(P G) = >: population Kullback Leibler 0 for t = 0, h i N R + N ln (e R 1) 2 R for t > 0. Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

13 Average Costs Estimation Decentralized Agent Control - Cost Estimation Cost J agents,t for decentralized evolution during time horizon T: J agents,t := Z T {z} N ρ 0 population ds Φ(s), {z} interacting agents ρ = kinetic energy z } { γ 2 U 2 /2 q 2 := individual cost rate function, {z} diffusion rate Φ(t) [0, 1] := proportion of interacting agents at time t. ******************************************************************************** Cost upper-bound, reached when Φ(t) 1 J agents,t NρT Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

14 Average Costs Estimation Costs Comparison - Centralized vs Decentralized cumulative costs upper-bounded decentralized costs actual decentralized costs centralized costs Kullback-Leibler entropy 0 Tc time horizons for which agent interactions beat the optimal effective centralized controller time horizon T Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

15 Conclusion Conclusion and Perspectives The stylized model exemplifies basic and somehow "universal" features: Agents mimetic interactions produce an emergent structure - (here a "shock"- like wave), Competition enhances global transport flow - (here a t-increase of the traveled distance), Self-organization via autonomous agents interactions can reduce costs. M.-O. Hongler, O. G. et al., "Centralized versus decentralized control - A solvable stylized model in transportation", Physica A, 389: , Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

16 On a Generalized Innovation Diffusion Model On a Generalized Innovation Diffusion Model Bass diffusion model: describes how a new product get adopted 2 populations of agents (2 possible states): j non adopters adopters Two ways for product adoption: { spontaneous adoption imitation Output: temporal evolution of the overall adoption rate Aggregated model, no spatial dimension Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

17 On a Generalized Innovation Diffusion Model On a Generalized Innovation Diffusion Model (continued) Introduced a spatial dimension into the original Bass model Agents now described by state and location Imitation between spatially close neighbors P(y, s=0) = Q(y, s=0) P(y, s=0.5) Q(y, s=0.5) y P(t) Original Bass model Spatial Bass Model with slightly segregated agents Spatial Bass model with highly segregated agents t F. Hashemi, M.-O. Hongler and O. G., "Spatio-Temporal Patterns for a Generalized Innovation Diffusion Model", submitted to the Journal of Economic Dynamics and Control, Olivier Gallay (EPFL, TRANSP-OR) Centralized Versus Decentralized Control 8th Joint OR Days, 09/09/ / 17

Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation

Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation M.-O. Hongler, O. Gallay, R. Colmorn, P. Cordes and M. Hülsmann Ecole Polytechnique Fédérale de Lausanne (EPFL), (CH)

More information

Centralized versus decentralized control - A solvable stylized model in transportation

Centralized versus decentralized control - A solvable stylized model in transportation Citation: Hongler M.-O., Gallay O., Hülsmann M., Cordes P., Colmorn R. (2010). Centralized Versus Decentralized Control - A Solvable Stylized Model in Transportation. Physica A: Statistical Mechanics and

More information

Physica A. Centralized versus decentralized control A solvable stylized model in transportation

Physica A. Centralized versus decentralized control A solvable stylized model in transportation Physica A 389 (1) 416 4171 Contents lists available at ScienceDirect Physica A journal homepage: www.elsevier.com/locate/physa Centralized versus decentralized control A solvable stylized model in transportation

More information

( ) ( ). ( ) " d#. ( ) " cos (%) " d%

( ) ( ). ( )  d#. ( )  cos (%)  d% Math 22 Fall 2008 Solutions to Homework #6 Problems from Pages 404-407 (Section 76) 6 We will use the technique of Separation of Variables to solve the differential equation: dy d" = ey # sin 2 (") y #

More information

Application of Batch Poisson Process to Random-sized Batch Arrival of Sediment Particles

Application of Batch Poisson Process to Random-sized Batch Arrival of Sediment Particles Application of Batch Poisson Process to Random-sized Batch Arrival of Sediment Particles Serena Y. Hung, Christina W. Tsai Department of Civil and Environmental Engineering, National Taiwan University

More information

9.3: Separable Equations

9.3: Separable Equations 9.3: Separable Equations An equation is separable if one can move terms so that each side of the equation only contains 1 variable. Consider the 1st order equation = F (x, y). dx When F (x, y) = f (x)g(y),

More information

Stochastic equations for thermodynamics

Stochastic equations for thermodynamics J. Chem. Soc., Faraday Trans. 93 (1997) 1751-1753 [arxiv 1503.09171] Stochastic equations for thermodynamics Roumen Tsekov Department of Physical Chemistry, University of Sofia, 1164 Sofia, ulgaria The

More information

Kolmogorov Equations and Markov Processes

Kolmogorov Equations and Markov Processes Kolmogorov Equations and Markov Processes May 3, 013 1 Transition measures and functions Consider a stochastic process {X(t)} t 0 whose state space is a product of intervals contained in R n. We define

More information

DETERMINATION OF MODEL VALID PREDICTION PERIOD USING THE BACKWARD FOKKER-PLANCK EQUATION

DETERMINATION OF MODEL VALID PREDICTION PERIOD USING THE BACKWARD FOKKER-PLANCK EQUATION .4 DETERMINATION OF MODEL VALID PREDICTION PERIOD USING THE BACKWARD FOKKER-PLANCK EQUATION Peter C. Chu, Leonid M. Ivanov, and C.W. Fan Department of Oceanography Naval Postgraduate School Monterey, California.

More information

Matthew Zyskowski 1 Quanyan Zhu 2

Matthew Zyskowski 1 Quanyan Zhu 2 Matthew Zyskowski 1 Quanyan Zhu 2 1 Decision Science, Credit Risk Office Barclaycard US 2 Department of Electrical Engineering Princeton University Outline Outline I Modern control theory with game-theoretic

More information

Formulas for probability theory and linear models SF2941

Formulas for probability theory and linear models SF2941 Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms

More information

Lecture 16. Theory of Second Order Linear Homogeneous ODEs

Lecture 16. Theory of Second Order Linear Homogeneous ODEs Math 245 - Mathematics of Physics and Engineering I Lecture 16. Theory of Second Order Linear Homogeneous ODEs February 17, 2012 Konstantin Zuev (USC) Math 245, Lecture 16 February 17, 2012 1 / 12 Agenda

More information

Lecture 6: Bayesian Inference in SDE Models

Lecture 6: Bayesian Inference in SDE Models Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs

More information

Optimization and Simulation

Optimization and Simulation Optimization and Simulation Variance reduction Michel Bierlaire Transport and Mobility Laboratory School of Architecture, Civil and Environmental Engineering Ecole Polytechnique Fédérale de Lausanne M.

More information

Statistical Mechanics of Active Matter

Statistical Mechanics of Active Matter Statistical Mechanics of Active Matter Umberto Marini Bettolo Marconi University of Camerino, Italy Naples, 24 May,2017 Umberto Marini Bettolo Marconi (2017) Statistical Mechanics of Active Matter 2017

More information

Chapter 2 Review of Classical Information Theory

Chapter 2 Review of Classical Information Theory Chapter 2 Review of Classical Information Theory Abstract This chapter presents a review of the classical information theory which plays a crucial role in this thesis. We introduce the various types of

More information

Brownian motion and the Central Limit Theorem

Brownian motion and the Central Limit Theorem Brownian motion and the Central Limit Theorem Amir Bar January 4, 3 Based on Shang-Keng Ma, Statistical Mechanics, sections.,.7 and the course s notes section 6. Introduction In this tutorial we shall

More information

Session 1: Probability and Markov chains

Session 1: Probability and Markov chains Session 1: Probability and Markov chains 1. Probability distributions and densities. 2. Relevant distributions. 3. Change of variable. 4. Stochastic processes. 5. The Markov property. 6. Markov finite

More information

Fokker-Planck Equation with Detailed Balance

Fokker-Planck Equation with Detailed Balance Appendix E Fokker-Planck Equation with Detailed Balance A stochastic process is simply a function of two variables, one is the time, the other is a stochastic variable X, defined by specifying: a: the

More information

ON THE FIRST TIME THAT AN ITO PROCESS HITS A BARRIER

ON THE FIRST TIME THAT AN ITO PROCESS HITS A BARRIER ON THE FIRST TIME THAT AN ITO PROCESS HITS A BARRIER GERARDO HERNANDEZ-DEL-VALLE arxiv:1209.2411v1 [math.pr] 10 Sep 2012 Abstract. This work deals with first hitting time densities of Ito processes whose

More information

Asset Pricing. Chapter IX. The Consumption Capital Asset Pricing Model. June 20, 2006

Asset Pricing. Chapter IX. The Consumption Capital Asset Pricing Model. June 20, 2006 Chapter IX. The Consumption Capital Model June 20, 2006 The Representative Agent Hypothesis and its Notion of Equilibrium 9.2.1 An infinitely lived Representative Agent Avoid terminal period problem Equivalence

More information

0.3.4 Burgers Equation and Nonlinear Wave

0.3.4 Burgers Equation and Nonlinear Wave 16 CONTENTS Solution to step (discontinuity) initial condition u(x, 0) = ul if X < 0 u r if X > 0, (80) u(x, t) = u L + (u L u R ) ( 1 1 π X 4νt e Y 2 dy ) (81) 0.3.4 Burgers Equation and Nonlinear Wave

More information

Arc Length and Surface Area in Parametric Equations

Arc Length and Surface Area in Parametric Equations Arc Length and Surface Area in Parametric Equations MATH 211, Calculus II J. Robert Buchanan Department of Mathematics Spring 2011 Background We have developed definite integral formulas for arc length

More information

Endogenous Information Choice

Endogenous Information Choice Endogenous Information Choice Lecture 7 February 11, 2015 An optimizing trader will process those prices of most importance to his decision problem most frequently and carefully, those of less importance

More information

Lecture 11: Non-linear Diffusion

Lecture 11: Non-linear Diffusion Lecture 11: Non-linear Diffusion Scribe: Lou Odette - American International Group (AIG) October 17, 006 1 Non-linear Drift In the continuum limit the PDF ρ(x, t) for the x at time t of a single random

More information

Mean-Field Games versus Exogenous Strategies for Cooperative Economic Growth

Mean-Field Games versus Exogenous Strategies for Cooperative Economic Growth Mean-Field Games versus Exogenous Strategies for Cooperative Economic Growth Max-Olivier Hongler 1, Olivier Gallay 2, and Fariba Hashemi 3 1 EPFL, Lausanne, Switzerland 2 University of Lausanne, Switzerland

More information

Uncertainty quantification and systemic risk

Uncertainty quantification and systemic risk Uncertainty quantification and systemic risk Josselin Garnier (Université Paris Diderot) with George Papanicolaou and Tzu-Wei Yang (Stanford University) February 3, 2016 Modeling systemic risk We consider

More information

Local vs. Nonlocal Diffusions A Tale of Two Laplacians

Local vs. Nonlocal Diffusions A Tale of Two Laplacians Local vs. Nonlocal Diffusions A Tale of Two Laplacians Jinqiao Duan Dept of Applied Mathematics Illinois Institute of Technology Chicago duan@iit.edu Outline 1 Einstein & Wiener: The Local diffusion 2

More information

Data-driven characterization of multidirectional pedestrian trac

Data-driven characterization of multidirectional pedestrian trac Data-driven characterization of multidirectional pedestrian trac Marija Nikoli Michel Bierlaire Flurin Hänseler TGF 5 Delft University of Technology, the Netherlands October 8, 5 / 5 Outline. Motivation

More information

Math221: HW# 2 solutions

Math221: HW# 2 solutions Math: HW# solutions Andy Royston October, 5 8..4 Integrate each side from to t: t d x dt dt dx dx (t) dt dt () g k e kt t t ge kt dt g k ( e kt ). () Since the object starts from rest, dx dx () v(). Now

More information

The first order quasi-linear PDEs

The first order quasi-linear PDEs Chapter 2 The first order quasi-linear PDEs The first order quasi-linear PDEs have the following general form: F (x, u, Du) = 0, (2.1) where x = (x 1, x 2,, x 3 ) R n, u = u(x), Du is the gradient of u.

More information

Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak

Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak Introduction to Systems Analysis and Decision Making Prepared by: Jakub Tomczak 1 Introduction. Random variables During the course we are interested in reasoning about considered phenomenon. In other words,

More information

Diffusion in the cell

Diffusion in the cell Diffusion in the cell Single particle (random walk) Microscopic view Macroscopic view Measuring diffusion Diffusion occurs via Brownian motion (passive) Ex.: D = 100 μm 2 /s for typical protein in water

More information

Neural coding Ecological approach to sensory coding: efficient adaptation to the natural environment

Neural coding Ecological approach to sensory coding: efficient adaptation to the natural environment Neural coding Ecological approach to sensory coding: efficient adaptation to the natural environment Jean-Pierre Nadal CNRS & EHESS Laboratoire de Physique Statistique (LPS, UMR 8550 CNRS - ENS UPMC Univ.

More information

5 Mutual Information and Channel Capacity

5 Mutual Information and Channel Capacity 5 Mutual Information and Channel Capacity In Section 2, we have seen the use of a quantity called entropy to measure the amount of randomness in a random variable. In this section, we introduce several

More information

Optimum CUSUM Tests for Detecting Changes in Continuous Time Processes

Optimum CUSUM Tests for Detecting Changes in Continuous Time Processes Optimum CUSUM Tests for Detecting Changes in Continuous Time Processes George V. Moustakides INRIA, Rennes, France Outline The change detection problem Overview of existing results Lorden s criterion and

More information

Introduction to asymptotic techniques for stochastic systems with multiple time-scales

Introduction to asymptotic techniques for stochastic systems with multiple time-scales Introduction to asymptotic techniques for stochastic systems with multiple time-scales Eric Vanden-Eijnden Courant Institute Motivating examples Consider the ODE {Ẋ = Y 3 + sin(πt) + cos( 2πt) X() = x

More information

Information and Physics Landauer Principle and Beyond

Information and Physics Landauer Principle and Beyond Information and Physics Landauer Principle and Beyond Ryoichi Kawai Department of Physics University of Alabama at Birmingham Maxwell Demon Lerner, 975 Landauer principle Ralf Landauer (929-999) Computational

More information

Diffusion in a Logarithmic Potential Anomalies, Aging & Ergodicity Breaking

Diffusion in a Logarithmic Potential Anomalies, Aging & Ergodicity Breaking Diffusion in a Logarithmic Potential Anomalies, Aging & Ergodicity Breaking David Kessler Bar-Ilan Univ. E. Barkai (Bar-Ilan) A. Dechant (Augsburg) E. Lutz (Augsburg) PRL, 105 120602 (2010) arxiv:1105.5496

More information

Integrodifferential Hyperbolic Equations and its Application for 2-D Rotational Fluid Flows

Integrodifferential Hyperbolic Equations and its Application for 2-D Rotational Fluid Flows Integrodifferential Hyperbolic Equations and its Application for 2-D Rotational Fluid Flows Alexander Chesnokov Lavrentyev Institute of Hydrodynamics Novosibirsk, Russia chesnokov@hydro.nsc.ru July 14,

More information

= m(0) + 4e 2 ( 3e 2 ) 2e 2, 1 (2k + k 2 ) dt. m(0) = u + R 1 B T P x 2 R dt. u + R 1 B T P y 2 R dt +

= m(0) + 4e 2 ( 3e 2 ) 2e 2, 1 (2k + k 2 ) dt. m(0) = u + R 1 B T P x 2 R dt. u + R 1 B T P y 2 R dt + ECE 553, Spring 8 Posted: May nd, 8 Problem Set #7 Solution Solutions: 1. The optimal controller is still the one given in the solution to the Problem 6 in Homework #5: u (x, t) = p(t)x k(t), t. The minimum

More information

Reinforcement Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina

Reinforcement Learning. Donglin Zeng, Department of Biostatistics, University of North Carolina Reinforcement Learning Introduction Introduction Unsupervised learning has no outcome (no feedback). Supervised learning has outcome so we know what to predict. Reinforcement learning is in between it

More information

Burgers equation in the complex plane. Govind Menon Division of Applied Mathematics Brown University

Burgers equation in the complex plane. Govind Menon Division of Applied Mathematics Brown University Burgers equation in the complex plane Govind Menon Division of Applied Mathematics Brown University What this talk contains Interesting instances of the appearance of Burgers equation in the complex plane

More information

Information Thermodynamics on Causal Networks

Information Thermodynamics on Causal Networks 1/39 Information Thermodynamics on Causal Networks FSPIP 2013, July 12 2013. Sosuke Ito Dept. of Phys., the Univ. of Tokyo (In collaboration with T. Sagawa) ariv:1306.2756 The second law of thermodynamics

More information

COMPSCI 650 Applied Information Theory Jan 21, Lecture 2

COMPSCI 650 Applied Information Theory Jan 21, Lecture 2 COMPSCI 650 Applied Information Theory Jan 21, 2016 Lecture 2 Instructor: Arya Mazumdar Scribe: Gayane Vardoyan, Jong-Chyi Su 1 Entropy Definition: Entropy is a measure of uncertainty of a random variable.

More information

Propagation of Solitons Under Colored Noise

Propagation of Solitons Under Colored Noise Propagation of Solitons Under Colored Noise Dr. Russell Herman Departments of Mathematics & Statistics, Physics & Physical Oceanography UNC Wilmington, Wilmington, NC January 6, 2009 Outline of Talk 1

More information

Name Class. 5. Find the particular solution to given the general solution y C cos x and the. x 2 y

Name Class. 5. Find the particular solution to given the general solution y C cos x and the. x 2 y 10 Differential Equations Test Form A 1. Find the general solution to the first order differential equation: y 1 yy 0. 1 (a) (b) ln y 1 y ln y 1 C y y C y 1 C y 1 y C. Find the general solution to the

More information

This is a Gaussian probability centered around m = 0 (the most probable and mean position is the origin) and the mean square displacement m 2 = n,or

This is a Gaussian probability centered around m = 0 (the most probable and mean position is the origin) and the mean square displacement m 2 = n,or Physics 7b: Statistical Mechanics Brownian Motion Brownian motion is the motion of a particle due to the buffeting by the molecules in a gas or liquid. The particle must be small enough that the effects

More information

Statistical Machine Learning from Data

Statistical Machine Learning from Data Samy Bengio Statistical Machine Learning from Data Statistical Machine Learning from Data Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique Fédérale de Lausanne (EPFL),

More information

Simulation of conditional diffusions via forward-reverse stochastic representations

Simulation of conditional diffusions via forward-reverse stochastic representations Weierstrass Institute for Applied Analysis and Stochastics Simulation of conditional diffusions via forward-reverse stochastic representations Christian Bayer and John Schoenmakers Numerical methods for

More information

Complex Systems Workshop Lecture III: Behavioral Asset Pricing Model with Heterogeneous Beliefs

Complex Systems Workshop Lecture III: Behavioral Asset Pricing Model with Heterogeneous Beliefs Complex Systems Workshop Lecture III: Behavioral Asset Pricing Model with Heterogeneous Beliefs Cars Hommes CeNDEF, UvA CEF 2013, July 9, Vancouver Cars Hommes (CeNDEF, UvA) Complex Systems CEF 2013, Vancouver

More information

A Tour of Reinforcement Learning The View from Continuous Control. Benjamin Recht University of California, Berkeley

A Tour of Reinforcement Learning The View from Continuous Control. Benjamin Recht University of California, Berkeley A Tour of Reinforcement Learning The View from Continuous Control Benjamin Recht University of California, Berkeley trustable, scalable, predictable Control Theory! Reinforcement Learning is the study

More information

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs

More information

Homework #4 Solutions

Homework #4 Solutions MAT 303 Spring 03 Problems Section.: 0,, Section.:, 6,, Section.3:,, 0,, 30 Homework # Solutions..0. Suppose that the fish population P(t) in a lake is attacked by a disease at time t = 0, with the result

More information

Systemic risk and uncertainty quantification

Systemic risk and uncertainty quantification Systemic risk and uncertainty quantification Josselin Garnier (Université Paris 7) with George Papanicolaou and Tzu-Wei Yang (Stanford University) October 6, 22 Modeling Systemic Risk We consider evolving

More information

Hands-On Learning Theory Fall 2016, Lecture 3

Hands-On Learning Theory Fall 2016, Lecture 3 Hands-On Learning Theory Fall 016, Lecture 3 Jean Honorio jhonorio@purdue.edu 1 Information Theory First, we provide some information theory background. Definition 3.1 (Entropy). The entropy of a discrete

More information

PATTERN RECOGNITION AND MACHINE LEARNING

PATTERN RECOGNITION AND MACHINE LEARNING PATTERN RECOGNITION AND MACHINE LEARNING Chapter 1. Introduction Shuai Huang April 21, 2014 Outline 1 What is Machine Learning? 2 Curve Fitting 3 Probability Theory 4 Model Selection 5 The curse of dimensionality

More information

Reading. w Foley, Section 11.2 Optional

Reading. w Foley, Section 11.2 Optional Parametric Curves w Foley, Section.2 Optional Reading w Bartels, Beatty, and Barsky. An Introduction to Splines for use in Computer Graphics and Geometric Modeling, 987. w Farin. Curves and Surfaces for

More information

Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods: Markov Chain Monte Carlo

Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods: Markov Chain Monte Carlo Group Prof. Daniel Cremers 11. Sampling Methods: Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative

More information

On Bayesian bandit algorithms

On Bayesian bandit algorithms On Bayesian bandit algorithms Emilie Kaufmann joint work with Olivier Cappé, Aurélien Garivier, Nathaniel Korda and Rémi Munos July 1st, 2012 Emilie Kaufmann (Telecom ParisTech) On Bayesian bandit algorithms

More information

Weakly interacting particle systems on graphs: from dense to sparse

Weakly interacting particle systems on graphs: from dense to sparse Weakly interacting particle systems on graphs: from dense to sparse Ruoyu Wu University of Michigan (Based on joint works with Daniel Lacker and Kavita Ramanan) USC Math Finance Colloquium October 29,

More information

Chapter Introduction. A. Bensoussan

Chapter Introduction. A. Bensoussan Chapter 2 EXPLICIT SOLUTIONS OFLINEAR QUADRATIC DIFFERENTIAL GAMES A. Bensoussan International Center for Decision and Risk Analysis University of Texas at Dallas School of Management, P.O. Box 830688

More information

Dynamical systems with Gaussian and Levy noise: analytical and stochastic approaches

Dynamical systems with Gaussian and Levy noise: analytical and stochastic approaches Dynamical systems with Gaussian and Levy noise: analytical and stochastic approaches Noise is often considered as some disturbing component of the system. In particular physical situations, noise becomes

More information

Homogenization with stochastic differential equations

Homogenization with stochastic differential equations Homogenization with stochastic differential equations Scott Hottovy shottovy@math.arizona.edu University of Arizona Program in Applied Mathematics October 12, 2011 Modeling with SDE Use SDE to model system

More information

HJB equations. Seminar in Stochastic Modelling in Economics and Finance January 10, 2011

HJB equations. Seminar in Stochastic Modelling in Economics and Finance January 10, 2011 Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics, Charles University in Prague petrasek@karlin.mff.cuni.cz Seminar in Stochastic Modelling in Economics and Finance

More information

New ideas in the non-equilibrium statistical physics and the micro approach to transportation flows

New ideas in the non-equilibrium statistical physics and the micro approach to transportation flows New ideas in the non-equilibrium statistical physics and the micro approach to transportation flows Plenary talk on the conference Stochastic and Analytic Methods in Mathematical Physics, Yerevan, Armenia,

More information

2.3. Large Scale Channel Modeling Shadowing

2.3. Large Scale Channel Modeling Shadowing c B. Chen 2.3. Large Scale Channel Modeling Shadowing Reading assignment 4.9. Multipath Channel Because of the mobility and complex environment, there are two types of channel variations: small scale variation

More information

Nested Uncertain Differential Equations and Its Application to Multi-factor Term Structure Model

Nested Uncertain Differential Equations and Its Application to Multi-factor Term Structure Model Nested Uncertain Differential Equations and Its Application to Multi-factor Term Structure Model Xiaowei Chen International Business School, Nankai University, Tianjin 371, China School of Finance, Nankai

More information

Derivation of Itô SDE and Relationship to ODE and CTMC Models

Derivation of Itô SDE and Relationship to ODE and CTMC Models Derivation of Itô SDE and Relationship to ODE and CTMC Models Biomathematics II April 23, 2015 Linda J. S. Allen Texas Tech University TTU 1 Euler-Maruyama Method for Numerical Solution of an Itô SDE dx(t)

More information

Probabilistic Graphical Models

Probabilistic Graphical Models Probabilistic Graphical Models Introduction. Basic Probability and Bayes Volkan Cevher, Matthias Seeger Ecole Polytechnique Fédérale de Lausanne 26/9/2011 (EPFL) Graphical Models 26/9/2011 1 / 28 Outline

More information

First order differential equations

First order differential equations First order differential equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T. First

More information

Dynamic Consistency for Stochastic Optimal Control Problems

Dynamic Consistency for Stochastic Optimal Control Problems Dynamic Consistency for Stochastic Optimal Control Problems Cadarache Summer School CEA/EDF/INRIA 2012 Pierre Carpentier Jean-Philippe Chancelier Michel De Lara SOWG June 2012 Lecture outline Introduction

More information

Nonlinear Single-Particle Dynamics in High Energy Accelerators

Nonlinear Single-Particle Dynamics in High Energy Accelerators Nonlinear Single-Particle Dynamics in High Energy Accelerators Part 2: Basic tools and concepts Nonlinear Single-Particle Dynamics in High Energy Accelerators This course consists of eight lectures: 1.

More information

Robust control and applications in economic theory

Robust control and applications in economic theory Robust control and applications in economic theory In honour of Professor Emeritus Grigoris Kalogeropoulos on the occasion of his retirement A. N. Yannacopoulos Department of Statistics AUEB 24 May 2013

More information

Let X and Y be two real valued stochastic variables defined on (Ω, F, P). Theorem: If X and Y are independent then. . p.1/21

Let X and Y be two real valued stochastic variables defined on (Ω, F, P). Theorem: If X and Y are independent then. . p.1/21 Multivariate transformations The remaining part of the probability course is centered around transformations t : R k R m and how they transform probability measures. For instance tx 1,...,x k = x 1 +...

More information

p 1 ( Y p dp) 1/p ( X p dp) 1 1 p

p 1 ( Y p dp) 1/p ( X p dp) 1 1 p Doob s inequality Let X(t) be a right continuous submartingale with respect to F(t), t 1 P(sup s t X(s) λ) 1 λ {sup s t X(s) λ} X + (t)dp 2 For 1 < p

More information

M/G/1 queues and Busy Cycle Analysis

M/G/1 queues and Busy Cycle Analysis queues and Busy Cycle Analysis John C.S. Lui Department of Computer Science & Engineering The Chinese University of Hong Kong www.cse.cuhk.edu.hk/ cslui John C.S. Lui (CUHK) Computer Systems Performance

More information

State Space Representation of Gaussian Processes

State Space Representation of Gaussian Processes State Space Representation of Gaussian Processes Simo Särkkä Department of Biomedical Engineering and Computational Science (BECS) Aalto University, Espoo, Finland June 12th, 2013 Simo Särkkä (Aalto University)

More information

Machine learning - HT Maximum Likelihood

Machine learning - HT Maximum Likelihood Machine learning - HT 2016 3. Maximum Likelihood Varun Kanade University of Oxford January 27, 2016 Outline Probabilistic Framework Formulate linear regression in the language of probability Introduce

More information

Speculation and the Bond Market: An Empirical No-arbitrage Framework

Speculation and the Bond Market: An Empirical No-arbitrage Framework Online Appendix to the paper Speculation and the Bond Market: An Empirical No-arbitrage Framework October 5, 2015 Part I: Maturity specific shocks in affine and equilibrium models This Appendix present

More information

Anomalous Lévy diffusion: From the flight of an albatross to optical lattices. Eric Lutz Abteilung für Quantenphysik, Universität Ulm

Anomalous Lévy diffusion: From the flight of an albatross to optical lattices. Eric Lutz Abteilung für Quantenphysik, Universität Ulm Anomalous Lévy diffusion: From the flight of an albatross to optical lattices Eric Lutz Abteilung für Quantenphysik, Universität Ulm Outline 1 Lévy distributions Broad distributions Central limit theorem

More information

08. Brownian Motion. University of Rhode Island. Gerhard Müller University of Rhode Island,

08. Brownian Motion. University of Rhode Island. Gerhard Müller University of Rhode Island, University of Rhode Island DigitalCommons@URI Nonequilibrium Statistical Physics Physics Course Materials 1-19-215 8. Brownian Motion Gerhard Müller University of Rhode Island, gmuller@uri.edu Follow this

More information

The dynamics of small particles whose size is roughly 1 µmt or. smaller, in a fluid at room temperature, is extremely erratic, and is

The dynamics of small particles whose size is roughly 1 µmt or. smaller, in a fluid at room temperature, is extremely erratic, and is 1 I. BROWNIAN MOTION The dynamics of small particles whose size is roughly 1 µmt or smaller, in a fluid at room temperature, is extremely erratic, and is called Brownian motion. The velocity of such particles

More information

Special Relativity-General Discussion

Special Relativity-General Discussion Chapter 1 Special Relativity-General Discussion Let us consider a spacetime event. By this we mean a physical occurence at some point in space at a given time. in order to characterize this event we introduce

More information

Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information.

Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information. L65 Dept. of Linguistics, Indiana University Fall 205 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission rate

More information

Dept. of Linguistics, Indiana University Fall 2015

Dept. of Linguistics, Indiana University Fall 2015 L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 28 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission

More information

Perturbative Approaches for Robust Intertemporal Optimal Portfolio Selection

Perturbative Approaches for Robust Intertemporal Optimal Portfolio Selection Perturbative Approaches for Robust Intertemporal Optimal Portfolio Selection F. Trojani and P. Vanini ECAS Course, Lugano, October 7-13, 2001 1 Contents Introduction Merton s Model and Perturbative Solution

More information

The Mathematics of Continuous Time Contract Theory

The Mathematics of Continuous Time Contract Theory The Mathematics of Continuous Time Contract Theory Ecole Polytechnique, France University of Michigan, April 3, 2018 Outline Introduction to moral hazard 1 Introduction to moral hazard 2 3 General formulation

More information

Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness

Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness Second International Conference in Memory of Carlo Giannini Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness Kenneth F. Wallis Emeritus Professor of Econometrics,

More information

Computer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression

Computer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression Group Prof. Daniel Cremers 4. Gaussian Processes - Regression Definition (Rep.) Definition: A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution.

More information

Math 211. Lecture #6. Linear Equations. September 9, 2002

Math 211. Lecture #6. Linear Equations. September 9, 2002 1 Math 211 Lecture #6 Linear Equations September 9, 2002 2 Air Resistance 2 Air Resistance Acts in the direction opposite to the velocity. 2 Air Resistance Acts in the direction opposite to the velocity.

More information

Math 225 Differential Equations Notes Chapter 1

Math 225 Differential Equations Notes Chapter 1 Math 225 Differential Equations Notes Chapter 1 Michael Muscedere September 9, 2004 1 Introduction 1.1 Background In science and engineering models are used to describe physical phenomena. Often these

More information

Research Article A Necessary Characteristic Equation of Diffusion Processes Having Gaussian Marginals

Research Article A Necessary Characteristic Equation of Diffusion Processes Having Gaussian Marginals Abstract and Applied Analysis Volume 01, Article ID 598590, 9 pages doi:10.1155/01/598590 Research Article A Necessary Characteristic Equation of Diffusion Processes Having Gaussian Marginals Syeda Rabab

More information

Economics 701 Advanced Macroeconomics I Project 1 Professor Sanjay Chugh Fall 2011

Economics 701 Advanced Macroeconomics I Project 1 Professor Sanjay Chugh Fall 2011 Department of Economics University of Maryland Economics 701 Advanced Macroeconomics I Project 1 Professor Sanjay Chugh Fall 2011 Objective As a stepping stone to learning how to work with and computationally

More information

in Bounded Domains Ariane Trescases CMLA, ENS Cachan

in Bounded Domains Ariane Trescases CMLA, ENS Cachan CMLA, ENS Cachan Joint work with Yan GUO, Chanwoo KIM and Daniela TONON International Conference on Nonlinear Analysis: Boundary Phenomena for Evolutionnary PDE Academia Sinica December 21, 214 Outline

More information

Computer Vision Group Prof. Daniel Cremers. 14. Sampling Methods

Computer Vision Group Prof. Daniel Cremers. 14. Sampling Methods Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric

More information

Lecture II: Rigid-Body Physics

Lecture II: Rigid-Body Physics Rigid-Body Motion Previously: Point dimensionless objects moving through a trajectory. Today: Objects with dimensions, moving as one piece. 2 Rigid-Body Kinematics Objects as sets of points. Relative distances

More information

Chapter 6 - Random Processes

Chapter 6 - Random Processes EE385 Class Notes //04 John Stensby Chapter 6 - Random Processes Recall that a random variable X is a mapping between the sample space S and the extended real line R +. That is, X : S R +. A random process

More information

Diffusion and random walks on graphs

Diffusion and random walks on graphs Diffusion and random walks on graphs Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural

More information

arxiv:cond-mat/ v1 [cond-mat.stat-mech] 1 Aug 2001

arxiv:cond-mat/ v1 [cond-mat.stat-mech] 1 Aug 2001 Financial Market Dynamics arxiv:cond-mat/0108017v1 [cond-mat.stat-mech] 1 Aug 2001 Fredrick Michael and M.D. Johnson Department of Physics, University of Central Florida, Orlando, FL 32816-2385 (May 23,

More information