Risk-Sensitive and Robust Mean Field Games
|
|
- Melinda Hampton
- 5 years ago
- Views:
Transcription
1 Risk-Sensitive and Robust Mean Field Games Tamer Başar Coordinated Science Laboratory Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign Urbana, IL IPAM Workshop on Mean Field Games UCLA, Los Angeles, CA August 28-September 1, 217 August 31, 217 Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
2 Outline Introduction to nonzero-sum stochastic differential games (NZS SDGs) and Nash equilibrium: role of information structures Quick overview of risk-sensitive stochastic control (RS SC): equivalence to zero-sum stochastic differential games Mean field game approach to RS NZS SDGs with local state information Problem 1 (P1) Mean field game approach to robust NZS SDGs with local state information Problem 2 (P2) Connections between P1 and P2, and ɛ-nash equilibrium Extensions and conclusions Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
3 General NZS SDGs N-player state dynamics described by a stochastic differential equation (SDE) dx t = f (t, x t, u t )dt + D(t)db t, x t t= = x, u t := (u 1t,..., u Nt ) Could also be in partitioned form: x t = (x 1t,..., x Nt ) dx it = f i (t, x it, u it ; c i (t, x i,t, u i,t ))dt + D i (t)db it, i = 1,..., N Information structures (control policy of player i: γ i Γ i ): Closed-loop perfect state for all players: u it = γ i (t; x τ, τ t), i = 1,..., N Partial (local) state: u it = γ i (t; x iτ, τ t), i = 1,..., N Measurement feedback: u it = γ i (t; y iτ, τ t), dy it = h i (t, x it, x i,t )dt + E i (t)db it, i = 1,..., N Loss function for player i (over [t, T ]): L i (x [t,t ], u [t,t ] ) := q i (x T ) + t g i (s, x s, u s )ds Take expectations (for horizon [, T ]) with u = γ( ): J i (γ i, γ i ) Nash equilibrium γ Ji(γ i, γ i) = min γ i Γ i J i (γ i, γ i) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
4 Equilibrium solution State dynamics and loss functions dx t = f (t, x t, u t )dt + D(t)db t, x t t= = x, u t := (u 1t,..., u Nt ) L i (x [t,t ], u [t,t ] ) := q i (x T ) + g i (s, x s, u s )ds, i = 1,..., N t Closed-loop perfect state for all players: assume DD is strongly positive Nash equilibrium exists and is unique, if the coupled PDEs below admit a unique smooth solution: V i,t (t; x) = min v [V i,x f (t, x, v, u i) + g i (t, x, v, u i)] Tr[V i,xx(t; x)dd ] V i (T ; x) = q i (x), u it = γ i (t, x(t)), i = 1,..., N Other dynamic information structures (s.a. local state, decentralized, measurement feedback): Extremely challenging! Possibly infinite-dimensional (even if NE exists and is unique), even in linear-quadratic (LQ) NZS SDGs. Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
5 LQ NZS SDGs with perfect state information dx(t) = J i (γ i, γ i ) = E [ Ax + [ N B i u i ]dt + Ddb(t), i N = {1, 2,..., N} i=1 x(t ) 2 W i + N ] [ x(t) 2 Qi (t) + u j (t) 2 R ij ]dt j=1 The feedback Nash equilibrium: γ i The coupled RDEs with Z i and Z i (T ) = W i Z i = F Z i + Z i F + Q i + (t, x(t)) = R 1 ii B i Z i (t)x(t), i N N j=1 Z j B j R 1 jj R ij R 1 jj Bj Z j, i N F := A N j=1 B ir 1 ii B i Z i When information is local state, or imperfect measurement (even if shared by all players), existence and characterization an open problem Any hope for N sufficiently large? MFG approach provides the answer! Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
6 Risk-sensitive (RS) formulation of the NZS SDG Replace J i with J i (γ i, γ i ) = 2 θ ln E{ exp θ 2 L i(x [,T ], u [,T ] ) } where θ > is the risk sensitivity parameter, and as before L i (x [t,t ], u [t,t ] ) := q i (x T ) + g i (s, x s, u s )ds t and u it = γ i ( ), i = 1,..., N Nash equilibrium is defined as before, and the same difficulties with regard to information structures arise as before. Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
7 Digression: Risk-sensitive (RS) stochastic control State dynamics : dx t = f (t, x t, u t ) dt + ɛ D db t ; x t t= = x b t, t, standard Wiener process; ɛ > ; u t U, t (state FB control law µ M) Objective : Choose µ to minimize : ( θ > ) J(µ; t, x t ) = 2ɛ θ ln E{ exp θ 2ɛ L(x [t,t ], u [t,t ] ) } L(x [t,t ], u [t,t ] ) := q(x T ) + ψ(t; x) value function associated with t g(s, x s, u s )ds E { exp 2ɛ[ θ q(xt ) + g(s, x s, u s ) ds ] } t 2ɛ V (t; x) := inf J(µ; t, x) =: ln ψ(t; x), µ M θ Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
8 DP and Itô differentiation rule { V t (t; x) = inf Vx (t; x) f (t, x, u) + g(t, x, u) } u U V (T ; x) q(x) + 1 4γ 2 DV x(t; x) 2 + ɛ 2 Tr[ V xx DD ] ( γ 2 := θ ) If U = R m1, f linear in u, and g quadratic in u : Optimal control law: f (x, u) = f (t, x) + B(t, x)u ; g(t, x, u) = g (t, x) + u 2 u (t) = µ (t, x) = 1 2 B (t, x) V x(t; x), t t f HJB equation : V t = V x f (t, x) + g (t, x) 1 4 [ BV x 2 γ 2 DV x 2] + ɛ 2 Tr[ V xx (t; x)dd ] ; V (T ; x) q(x) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
9 A further special case : LEQG Problem f (t, x) = A(t)x, g (t, x) = 1 2 x Qx, Q Explicit solution: q(x) = (1/2) x Q f x V (t; x) = 1 2 x Z(t)x + l ɛ (t), t Z + A Z + ZA + Q Z ( BB γ 2 DD ) Z = l ɛ (t) = ɛ 2 t Tr [ Z(s) D(s)D (s) ] ds u (t) = µ (t, x) = B (t) Z(t) x, t t f Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
10 A class of stochastic differential games Two Players : Player 1: u t ; Player 2: w t Upper-Value (UV) Function : HJI UV equation : dx t = f (x t, u t ) dt+dw t dt + ɛ D db t ; x J(µ, ν; t, x t ) := E { q(x T ) + g(s, x s, u s ) ds t inf u U γ 2 w s 2 ds } W (t; x) = inf sup J(µ, ν; t, x) µ ν sup { W t + W x ( f + Dw ) + g γ 2 w 2 w R m 2 t + ɛ 2 Tr[ Wxx DD ] } = Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
11 HJI UV equation : inf u U sup { W t + W x ( f + Dw ) + g γ 2 w 2 w R m 2 + ɛ 2 Tr[ Wxx DD ] } = Isaacs condition holds Value Function : { W t (t; x) = inf Wx (t; x) f (t, x, u) + g(t, x, u) } u U + 1 4γ 2 DW x(t; x) 2 + ɛ 2 Tr[ W xx (t; x)dd ] ; IDENTICAL with V for all permissible ɛ, γ The same holds for the time-average case (LQ) W (T ; x) q(x) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
12 LQ RS MFGs, Problem 1 (P1) Stochastic differential equation (SDE) for agent i, 1 i N dx i (t) = (A(θ i )x i (t) + B(θ i )u i (t))dt + µd(θ i )db i (t) The risk-sensitive cost function for agent i with δ > P1: J N 1,i(u i, u i ) = lim sup T φ 1 i (x, f N, u) := x i (t) 1 N δ { } T log E e 1 δ φ1 i (x,f N,u) N x i (t) 2 Q + u i (t) 2 Rdt Risk-sensitive control: Robust control w.r.t. the risk parameter δ J1,i(u N 1 [ i, u i ) = lim sup E { φ 1 } 1 i + T T 2δ var{φ1 i } + o( 1 ] δ ) f N (t) := 1 N N i=1 x i(t): Mean Field term (mass behavior) Agents are coupled with each other through the mean field term i=1 Tembine-Zhu-TB, TAC (59, 4, 214); Moon-TB, CDC (214), TAC (62, 3, 217) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
13 LQ Robust MFGs, Problem 2 (P2) Stochastic differential equation (SDE) for agent i, 1 i N dx i (t) = (A(θ i )x i (t) + B(θ i )u i (t) + D(θ i )v i (t))dt + µd(θ i )db i (t) The worst-case risk-neutral cost function for agent i P2: J2,i(u N 1 i, u i ) = sup lim sup v i V i T T E{φ2 i (x, f N, u, v)} φ 2 i (x, f N, u, v) := x i (t) 1 N x i (t) 2 Q + u i (t) 2 R γ 2 v i (t) 2 dt N i=1 v i can be viewed as a fictitious player (or adversary) of agent i, which strives for a worst-case cost function for agent i Agents are coupled with each other through the mean field term Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
14 Mean Field Analysis for P1 and P2 Solve the individual local robust control problem with g instead of f N P1: J 1 (u, g) = lim sup T δ T log E{ exp[ 1 δ P2: J 1 2 (u, v, g) = lim sup T T E{ x(t) g(t) 2 Q + u(t) 2 Rdt] } x(t) g(t) 2 Q + u(t) 2 R γ 2 v(t) 2 dt } Characterize g that is a best estimate of the mean field f N need to construct a mean field system T (g)(t) obtain a fixed point of T (g)(t), i.e., g = T (g ) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
15 Robust Tracking Control for P1 and P2 Proposition: Individual robust control problems for P1 and P2 Suppose that (A, B) is stabilizable and (A, Q 1/2 ) is detectable. Suppose that for a fixed γ = δ/2µ >, there is a matrix P that solves the following GARE Then A T P + PA + Q P(BR 1 B T 1 γ 2 DDT )P = H := A BR 1 B T P + 1 γ 2 DD T P and G := A BR 1 B T P are Hurwitz The robust decentralized controller: ū(t) = R 1 B T Px(t) R 1 B T s(t) = H T s(t) + Qg(t) Remark where ds(t) dt The worst-case disturbance (P2): v(t) = γ 2 D T Px(t) + γ 2 D T s(t) s(t) has a unique solution in Cn b : s(t) = e HT (t s) Qg(s)ds t The two robust tracking problems are identical Related to the robust (H ) control problem w.r.t. γ Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
16 Mean Field Analysis for P1 and P2 x θ (t) = E{x θ (t)} and we use h C b n for P2 Mean field system for P1 (with the robust decentralized controller) T (g)(t) := x θ (t)df (θ, x) θ Θ,x X t x θ (t) = e G(θ)t x + e G(θ)(t τ) B(θ)R 1 B T (θ) ( ) e H(θ)T (τ s) Qg(s)ds dτ τ Mean field system for P2 (with the robust decentralized controller and the worst-case disturbance) L(h)(t) := x θ (t)df (θ, x) θ Θ,x X t x θ (t) = e H(θ)t x + e H(θ)(t τ)( ) B(θ)R 1 B T (θ) γ 2 D(θ)D(θ) T ( ) e HT (θ)(τ s) Qh(s)ds dτ τ Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
17 Mean Field Analysis for P1 and P2 T (g)(t) and L(h)(t) capture the mass behavior when N is large Simplest case lim f 1 N(t) = lim N N N N x i (t) = E{x i (t)} = T (g)(t), i=1 SLLN We need to seek g and h such that g = T (g ) and h = L(h ) Sufficient condition (due to the contraction mapping theorem) P1 : R 1 Q ( P2 : θ Θ θ Θ B(θ) 2( )( e G(θ)τ dτ e H(θ)t 2 dt ) e H(θ)τ dτ df (θ) < 1 ) 2 ( B(θ) 2 R 1 + γ 2 D(θ) 2) df (θ) < 1 lim k T k (g ) = g for any g C b n g (t) and h (t) are best estimates of f N (t) when N is large Generally g h. But when γ, g h Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
18 Main Results for P1 and P2 Existence and Characterization of an ɛ-nash equilibrium There exists an ɛ-nash equilibrium with g (P1), i.e., there exist {u i, 1 i N} and ɛ N such that J1,i(u N i, u i) inf J u i 1,i(u N i, u i) + ɛ N, Ui c where ɛ N as N. For the uniform agent case, ɛ N = O(1/ N) The ɛ-nash strategy u i is decentralized, i.e., u i is a function of x i and g Law of Large Numbers: g satisfies lim N 1 N 1 lim lim sup N T T N x i=1 i (t) g (t) 2 dt =, T, a.s. 1 N N i=1 i (t) g (t) 2 dt =, a.s. g : deterministic function and can be computed offline The same results also hold for P2 with the worst-case disturbance Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1 x
19 Main Results for P1 and P2 Proof (sketch): Law of large numbers (first part) fn (t) g (t) 2 dt 2 1 N (xi (t) E{x i (t)}) 2 dt N i=1 + 2T sup E{xi (t)} g (t) 2 t The second part is zero (due to the fixed-point theorem) e i (t) = x i (t) E{x i (t)} is a mutually orthogonal random vector with E{e i (t)} = and E{ e i (t) 2 } < for all i and t Strong law of large numbers lim N (1/N) N t [, T ] i=1 e i (t) = for all (1/N) N i=1 e i (t) 2, N 1, is uniformly integrable on [, T ] for all T, we have the desired result Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
20 Partial Equivalence and Limiting Behaviors of P1 and P2 LQ-MFG,for fixed ( ) LQ-RSMFG (P1) Partial LQ-RMFG (P2) P1 and P2 share the same robust decentralized controller Partial equivalence: the mean field systems (and their fixed points) are different Limiting behaviors Large deviation (small noise) limit (µ, δ with γ = δ/2µ > ): The same results hold under this limit (SDE ODE) Risk-neutral limit (γ ): The results are identical to that of the (risk-neutral) LQ mean field game (g h ) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
21 Simulations (N = 5) A i = θ i is an i.i.d. uniform random variable with the interval [2, 5], B = D = Q = R = 1, µ = 2 γ θ = γ = 1, g (t) = 5.86e 8.49t, h (t) = 5.1e 3.37t ɛ 2 (N) := lim sup T 1 T E f N (t) g (t) 2 dt γ determines robustness of the equilibrium (due to the individual robust control problems) Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
22 Simulations (N = 5) A i = θ i is an i.i.d. uniform random variable with the interval [2, 5], B = D = Q = R = 1, µ = 2 γ θ = γ = 1, g (t) = 5.86e 8.49t, h (t) = 5.1e 3.37t ɛ 2 (N) := lim sup T 1 T E f N (t) g (t) 2 dt γ determines robustness of the equilibrium (due to the individual robust control problems) 35 3 γ = 1.5 (P1) γ = 1.5 (P2) γ = 15 (P1 and P2).9.8 γ = 1.5 (P1) γ = 15 (P1 and P2) γ = 1.5 (P2) number of agents 2 15 ε 2 (N) state value N Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
23 Conclusions Decentralized (local state-feedback) ɛ-nash equilibria for LQ risk-sensitive and LQ robust mean field games The equilibrium features robustness due to the local robust optimal control problem parametrized by γ LQ risk-sensitive and LQ robust mean field games are partially equivalent (g h ) hold the same limiting behaviors as the one-agent case Extensions to heterogenous case and nonlinear dynamics are possible, but results are not as explicit; see, Tembine, Zhu, Başar, IEEE-TAC (59, 4, 214) for RSMFG Imperfect state measurements RSMFGs on networks with agents interacting only with their neighbors Leader-Follower MFGs Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
24 THANKS! Tamer Başar (ECE/CSL, UIUC) IPAM Workshop on Mean Field Games August 31, / 1
Mean-Field Games with non-convex Hamiltonian
Mean-Field Games with non-convex Hamiltonian Martino Bardi Dipartimento di Matematica "Tullio Levi-Civita" Università di Padova Workshop "Optimal Control and MFGs" Pavia, September 19 21, 2018 Martino
More informationLinear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013
Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013 Abstract As in optimal control theory, linear quadratic (LQ) differential games (DG) can be solved, even in high dimension,
More informationMEAN FIELD GAMES WITH MAJOR AND MINOR PLAYERS
MEAN FIELD GAMES WITH MAJOR AND MINOR PLAYERS René Carmona Department of Operations Research & Financial Engineering PACM Princeton University CEMRACS - Luminy, July 17, 217 MFG WITH MAJOR AND MINOR PLAYERS
More informationStochastic optimal control with rough paths
Stochastic optimal control with rough paths Paul Gassiat TU Berlin Stochastic processes and their statistics in Finance, Okinawa, October 28, 2013 Joint work with Joscha Diehl and Peter Friz Introduction
More informationLecture 5 Linear Quadratic Stochastic Control
EE363 Winter 2008-09 Lecture 5 Linear Quadratic Stochastic Control linear-quadratic stochastic control problem solution via dynamic programming 5 1 Linear stochastic system linear dynamical system, over
More informationRobust control and applications in economic theory
Robust control and applications in economic theory In honour of Professor Emeritus Grigoris Kalogeropoulos on the occasion of his retirement A. N. Yannacopoulos Department of Statistics AUEB 24 May 2013
More informationOn continuous time contract theory
Ecole Polytechnique, France Journée de rentrée du CMAP, 3 octobre, 218 Outline 1 2 Semimartingale measures on the canonical space Random horizon 2nd order backward SDEs (Static) Principal-Agent Problem
More informationLinear Quadratic Zero-Sum Two-Person Differential Games
Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard To cite this version: Pierre Bernhard. Linear Quadratic Zero-Sum Two-Person Differential Games. Encyclopaedia of Systems and Control,
More informationBrownian Motion. 1 Definition Brownian Motion Wiener measure... 3
Brownian Motion Contents 1 Definition 2 1.1 Brownian Motion................................. 2 1.2 Wiener measure.................................. 3 2 Construction 4 2.1 Gaussian process.................................
More informationNonlinear representation, backward SDEs, and application to the Principal-Agent problem
Nonlinear representation, backward SDEs, and application to the Principal-Agent problem Ecole Polytechnique, France April 4, 218 Outline The Principal-Agent problem Formulation 1 The Principal-Agent problem
More informationOPTIMAL CONTROL. Sadegh Bolouki. Lecture slides for ECE 515. University of Illinois, Urbana-Champaign. Fall S. Bolouki (UIUC) 1 / 28
OPTIMAL CONTROL Sadegh Bolouki Lecture slides for ECE 515 University of Illinois, Urbana-Champaign Fall 2016 S. Bolouki (UIUC) 1 / 28 (Example from Optimal Control Theory, Kirk) Objective: To get from
More informationGame Theory Extra Lecture 1 (BoB)
Game Theory 2014 Extra Lecture 1 (BoB) Differential games Tools from optimal control Dynamic programming Hamilton-Jacobi-Bellman-Isaacs equation Zerosum linear quadratic games and H control Baser/Olsder,
More informationDeterministic Dynamic Programming
Deterministic Dynamic Programming 1 Value Function Consider the following optimal control problem in Mayer s form: V (t 0, x 0 ) = inf u U J(t 1, x(t 1 )) (1) subject to ẋ(t) = f(t, x(t), u(t)), x(t 0
More informationOPTIMAL CONTROL SYSTEMS
SYSTEMS MIN-MAX Harry G. Kwatny Department of Mechanical Engineering & Mechanics Drexel University OUTLINE MIN-MAX CONTROL Problem Definition HJB Equation Example GAME THEORY Differential Games Isaacs
More informationMA8109 Stochastic Processes in Systems Theory Autumn 2013
Norwegian University of Science and Technology Department of Mathematical Sciences MA819 Stochastic Processes in Systems Theory Autumn 213 1 MA819 Exam 23, problem 3b This is a linear equation of the form
More informationHJB equations. Seminar in Stochastic Modelling in Economics and Finance January 10, 2011
Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics, Charles University in Prague petrasek@karlin.mff.cuni.cz Seminar in Stochastic Modelling in Economics and Finance
More informationTheoretical Tutorial Session 2
1 / 36 Theoretical Tutorial Session 2 Xiaoming Song Department of Mathematics Drexel University July 27, 216 Outline 2 / 36 Itô s formula Martingale representation theorem Stochastic differential equations
More informationStability of Stochastic Differential Equations
Lyapunov stability theory for ODEs s Stability of Stochastic Differential Equations Part 1: Introduction Department of Mathematics and Statistics University of Strathclyde Glasgow, G1 1XH December 2010
More informationExplicit solutions of some Linear-Quadratic Mean Field Games
Explicit solutions of some Linear-Quadratic Mean Field Games Martino Bardi Department of Pure and Applied Mathematics University of Padua, Italy Men Field Games and Related Topics Rome, May 12-13, 2011
More informationMean-Field optimization problems and non-anticipative optimal transport. Beatrice Acciaio
Mean-Field optimization problems and non-anticipative optimal transport Beatrice Acciaio London School of Economics based on ongoing projects with J. Backhoff, R. Carmona and P. Wang Robust Methods in
More informationLAN property for sde s with additive fractional noise and continuous time observation
LAN property for sde s with additive fractional noise and continuous time observation Eulalia Nualart (Universitat Pompeu Fabra, Barcelona) joint work with Samy Tindel (Purdue University) Vlad s 6th birthday,
More informationSystemic Risk and Stochastic Games with Delay
Systemic Risk and Stochastic Games with Delay Jean-Pierre Fouque (with René Carmona, Mostafa Mousavi and Li-Hsien Sun) PDE and Probability Methods for Interactions Sophia Antipolis (France) - March 30-31,
More informationSystemic Risk and Stochastic Games with Delay
Systemic Risk and Stochastic Games with Delay Jean-Pierre Fouque Collaborators: R. Carmona, M. Mousavi, L.-H. Sun, N. Ning and Z. Zhang Mean Field Games 2017 IPAM at UCLA JP Fouque (UC Santa Barbara) Stochastic
More informationProduction and Relative Consumption
Mean Field Growth Modeling with Cobb-Douglas Production and Relative Consumption School of Mathematics and Statistics Carleton University Ottawa, Canada Mean Field Games and Related Topics III Henri Poincaré
More informationAn Overview of Risk-Sensitive Stochastic Optimal Control
An Overview of Risk-Sensitive Stochastic Optimal Control Matt James Australian National University Workshop: Stochastic control, communications and numerics perspective UniSA, Adelaide, Sept. 2004. 1 Outline
More informationBranching Processes II: Convergence of critical branching to Feller s CSB
Chapter 4 Branching Processes II: Convergence of critical branching to Feller s CSB Figure 4.1: Feller 4.1 Birth and Death Processes 4.1.1 Linear birth and death processes Branching processes can be studied
More informationDynamical systems with Gaussian and Levy noise: analytical and stochastic approaches
Dynamical systems with Gaussian and Levy noise: analytical and stochastic approaches Noise is often considered as some disturbing component of the system. In particular physical situations, noise becomes
More informationStochastic Differential Equations
CHAPTER 1 Stochastic Differential Equations Consider a stochastic process X t satisfying dx t = bt, X t,w t dt + σt, X t,w t dw t. 1.1 Question. 1 Can we obtain the existence and uniqueness theorem for
More informationA new approach for investment performance measurement. 3rd WCMF, Santa Barbara November 2009
A new approach for investment performance measurement 3rd WCMF, Santa Barbara November 2009 Thaleia Zariphopoulou University of Oxford, Oxford-Man Institute and The University of Texas at Austin 1 Performance
More informationControlled Diffusions and Hamilton-Jacobi Bellman Equations
Controlled Diffusions and Hamilton-Jacobi Bellman Equations Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2014 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationControl Design of a Distributed Parameter. Fixed-Bed Reactor
Workshop on Control of Distributed Parameter Systems p. 1/1 Control Design of a Distributed Parameter Fixed-Bed Reactor Ilyasse Aksikas and J. Fraser Forbes Department of Chemical and Material Engineering
More informationRough Burgers-like equations with multiplicative noise
Rough Burgers-like equations with multiplicative noise Martin Hairer Hendrik Weber Mathematics Institute University of Warwick Bielefeld, 3.11.21 Burgers-like equation Aim: Existence/Uniqueness for du
More informationIntroduction to numerical simulations for Stochastic ODEs
Introduction to numerical simulations for Stochastic ODEs Xingye Kan Illinois Institute of Technology Department of Applied Mathematics Chicago, IL 60616 August 9, 2010 Outline 1 Preliminaries 2 Numerical
More informationBernardo D Auria Stochastic Processes /12. Notes. March 29 th, 2012
1 Stochastic Calculus Notes March 9 th, 1 In 19, Bachelier proposed for the Paris stock exchange a model for the fluctuations affecting the price X(t) of an asset that was given by the Brownian motion.
More informationOn Stochastic Adaptive Control & its Applications. Bozenna Pasik-Duncan University of Kansas, USA
On Stochastic Adaptive Control & its Applications Bozenna Pasik-Duncan University of Kansas, USA ASEAS Workshop, AFOSR, 23-24 March, 2009 1. Motivation: Work in the 1970's 2. Adaptive Control of Continuous
More informationLinear-Quadratic Stochastic Differential Games with General Noise Processes
Linear-Quadratic Stochastic Differential Games with General Noise Processes Tyrone E. Duncan Abstract In this paper a noncooperative, two person, zero sum, stochastic differential game is formulated and
More informationSolution of Stochastic Optimal Control Problems and Financial Applications
Journal of Mathematical Extension Vol. 11, No. 4, (2017), 27-44 ISSN: 1735-8299 URL: http://www.ijmex.com Solution of Stochastic Optimal Control Problems and Financial Applications 2 Mat B. Kafash 1 Faculty
More informationCheap talk theoretic tools for distributed sensing in the presence of strategic sensors
1 / 37 Cheap talk theoretic tools for distributed sensing in the presence of strategic sensors Cédric Langbort Department of Aerospace Engineering & Coordinated Science Laboratory UNIVERSITY OF ILLINOIS
More information1. Stochastic Processes and filtrations
1. Stochastic Processes and 1. Stoch. pr., A stochastic process (X t ) t T is a collection of random variables on (Ω, F) with values in a measurable space (S, S), i.e., for all t, In our case X t : Ω S
More informationMean Field Consumption-Accumulation Optimization with HAR
Mean Field Consumption-Accumulation Optimization with HARA Utility School of Mathematics and Statistics Carleton University, Ottawa Workshop on Mean Field Games and Related Topics 2 Padova, September 4-7,
More informationMath 234. What you should know on day one. August 28, You should be able to use general principles like. x = cos t, y = sin t, 0 t π.
Math 234 What you should know on day one August 28, 2001 1 You should be able to use general principles like Length = ds, Area = da, Volume = dv For example the length of the semi circle x = cos t, y =
More informationUCLA Chemical Engineering. Process & Control Systems Engineering Laboratory
Constrained Innite-time Optimal Control Donald J. Chmielewski Chemical Engineering Department University of California Los Angeles February 23, 2000 Stochastic Formulation - Min Max Formulation - UCLA
More informationH -Optimal Control and Related Minimax Design Problems
Tamer Başar Pierre Bernhard H -Optimal Control and Related Minimax Design Problems A Dynamic Game Approach Second Edition 1995 Birkhäuser Boston Basel Berlin Contents Preface v 1 A General Introduction
More informationarxiv: v1 [math.pr] 18 Oct 2016
An Alternative Approach to Mean Field Game with Major and Minor Players, and Applications to Herders Impacts arxiv:161.544v1 [math.pr 18 Oct 216 Rene Carmona August 7, 218 Abstract Peiqi Wang The goal
More informationEN Applied Optimal Control Lecture 8: Dynamic Programming October 10, 2018
EN530.603 Applied Optimal Control Lecture 8: Dynamic Programming October 0, 08 Lecturer: Marin Kobilarov Dynamic Programming (DP) is conerned with the computation of an optimal policy, i.e. an optimal
More informationRough paths methods 4: Application to fbm
Rough paths methods 4: Application to fbm Samy Tindel Purdue University University of Aarhus 2016 Samy T. (Purdue) Rough Paths 4 Aarhus 2016 1 / 67 Outline 1 Main result 2 Construction of the Levy area:
More informationReal Time Stochastic Control and Decision Making: From theory to algorithms and applications
Real Time Stochastic Control and Decision Making: From theory to algorithms and applications Evangelos A. Theodorou Autonomous Control and Decision Systems Lab Challenges in control Uncertainty Stochastic
More informationBernardo D Auria Stochastic Processes /10. Notes. Abril 13 th, 2010
1 Stochastic Calculus Notes Abril 13 th, 1 As we have seen in previous lessons, the stochastic integral with respect to the Brownian motion shows a behavior different from the classical Riemann-Stieltjes
More informationLarge-population, dynamical, multi-agent, competitive, and cooperative phenomena occur in a wide range of designed
Definition Mean Field Game (MFG) theory studies the existence of Nash equilibria, together with the individual strategies which generate them, in games involving a large number of agents modeled by controlled
More informationDynamic Games with Asymmetric Information: Common Information Based Perfect Bayesian Equilibria and Sequential Decomposition
Dynamic Games with Asymmetric Information: Common Information Based Perfect Bayesian Equilibria and Sequential Decomposition 1 arxiv:1510.07001v1 [cs.gt] 23 Oct 2015 Yi Ouyang, Hamidreza Tavafoghi and
More informationLinear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters
Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 204 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationStability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games
Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games Alberto Bressan ) and Khai T. Nguyen ) *) Department of Mathematics, Penn State University **) Department of Mathematics,
More information10 Transfer Matrix Models
MIT EECS 6.241 (FALL 26) LECTURE NOTES BY A. MEGRETSKI 1 Transfer Matrix Models So far, transfer matrices were introduced for finite order state space LTI models, in which case they serve as an important
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationContinuous Time Finance
Continuous Time Finance Lisbon 2013 Tomas Björk Stockholm School of Economics Tomas Björk, 2013 Contents Stochastic Calculus (Ch 4-5). Black-Scholes (Ch 6-7. Completeness and hedging (Ch 8-9. The martingale
More informationMean Field Competitive Binary MDPs and Structured Solutions
Mean Field Competitive Binary MDPs and Structured Solutions Minyi Huang School of Mathematics and Statistics Carleton University Ottawa, Canada MFG217, UCLA, Aug 28 Sept 1, 217 1 / 32 Outline of talk The
More informationLecture 10 Linear Quadratic Stochastic Control with Partial State Observation
EE363 Winter 2008-09 Lecture 10 Linear Quadratic Stochastic Control with Partial State Observation partially observed linear-quadratic stochastic control problem estimation-control separation principle
More informationChapter 3. LQ, LQG and Control System Design. Dutch Institute of Systems and Control
Chapter 3 LQ, LQG and Control System H 2 Design Overview LQ optimization state feedback LQG optimization output feedback H 2 optimization non-stochastic version of LQG Application to feedback system design
More informationI forgot to mention last time: in the Ito formula for two standard processes, putting
I forgot to mention last time: in the Ito formula for two standard processes, putting dx t = a t dt + b t db t dy t = α t dt + β t db t, and taking f(x, y = xy, one has f x = y, f y = x, and f xx = f yy
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 15. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationNon-Zero-Sum Stochastic Differential Games of Controls and St
Non-Zero-Sum Stochastic Differential Games of Controls and Stoppings October 1, 2009 Based on two preprints: to a Non-Zero-Sum Stochastic Differential Game of Controls and Stoppings I. Karatzas, Q. Li,
More informationA Concise Course on Stochastic Partial Differential Equations
A Concise Course on Stochastic Partial Differential Equations Michael Röckner Reference: C. Prevot, M. Röckner: Springer LN in Math. 1905, Berlin (2007) And see the references therein for the original
More informationDynamic Bertrand and Cournot Competition
Dynamic Bertrand and Cournot Competition Effects of Product Differentiation Andrew Ledvina Department of Operations Research and Financial Engineering Princeton University Joint with Ronnie Sircar Princeton-Lausanne
More informationIntroduction to Algorithmic Trading Strategies Lecture 4
Introduction to Algorithmic Trading Strategies Lecture 4 Optimal Pairs Trading by Stochastic Control Haksun Li haksun.li@numericalmethod.com www.numericalmethod.com Outline Problem formulation Ito s lemma
More information2012 NCTS Workshop on Dynamical Systems
Barbara Gentz gentz@math.uni-bielefeld.de http://www.math.uni-bielefeld.de/ gentz 2012 NCTS Workshop on Dynamical Systems National Center for Theoretical Sciences, National Tsing-Hua University Hsinchu,
More informationAn Introduction to Noncooperative Games
An Introduction to Noncooperative Games Alberto Bressan Department of Mathematics, Penn State University Alberto Bressan (Penn State) Noncooperative Games 1 / 29 introduction to non-cooperative games:
More informationExercises. T 2T. e ita φ(t)dt.
Exercises. Set #. Construct an example of a sequence of probability measures P n on R which converge weakly to a probability measure P but so that the first moments m,n = xdp n do not converge to m = xdp.
More informationAsymptotics for posterior hazards
Asymptotics for posterior hazards Pierpaolo De Blasi University of Turin 10th August 2007, BNR Workshop, Isaac Newton Intitute, Cambridge, UK Joint work with Giovanni Peccati (Université Paris VI) and
More informationThe concentration of a drug in blood. Exponential decay. Different realizations. Exponential decay with noise. dc(t) dt.
The concentration of a drug in blood Exponential decay C12 concentration 2 4 6 8 1 C12 concentration 2 4 6 8 1 dc(t) dt = µc(t) C(t) = C()e µt 2 4 6 8 1 12 time in minutes 2 4 6 8 1 12 time in minutes
More informationUncertainty quantification and systemic risk
Uncertainty quantification and systemic risk Josselin Garnier (Université Paris Diderot) with George Papanicolaou and Tzu-Wei Yang (Stanford University) February 3, 2016 Modeling systemic risk We consider
More informationQuasi Stochastic Approximation American Control Conference San Francisco, June 2011
Quasi Stochastic Approximation American Control Conference San Francisco, June 2011 Sean P. Meyn Joint work with Darshan Shirodkar and Prashant Mehta Coordinated Science Laboratory and the Department of
More informationMean Field Games on networks
Mean Field Games on networks Claudio Marchi Università di Padova joint works with: S. Cacace (Rome) and F. Camilli (Rome) C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017
More informationBackward Stochastic Differential Equations with Infinite Time Horizon
Backward Stochastic Differential Equations with Infinite Time Horizon Holger Metzler PhD advisor: Prof. G. Tessitore Università di Milano-Bicocca Spring School Stochastic Control in Finance Roscoff, March
More informationAn Overview of the Martingale Representation Theorem
An Overview of the Martingale Representation Theorem Nuno Azevedo CEMAPRE - ISEG - UTL nazevedo@iseg.utl.pt September 3, 21 Nuno Azevedo (CEMAPRE - ISEG - UTL) LXDS Seminar September 3, 21 1 / 25 Background
More informationExercises for Multivariable Differential Calculus XM521
This document lists all the exercises for XM521. The Type I (True/False) exercises will be given, and should be answered, online immediately following each lecture. The Type III exercises are to be done
More informationSYSTEMTEORI - KALMAN FILTER VS LQ CONTROL
SYSTEMTEORI - KALMAN FILTER VS LQ CONTROL 1. Optimal regulator with noisy measurement Consider the following system: ẋ = Ax + Bu + w, x(0) = x 0 where w(t) is white noise with Ew(t) = 0, and x 0 is a stochastic
More informationProf. Erhan Bayraktar (University of Michigan)
September 17, 2012 KAP 414 2:15 PM- 3:15 PM Prof. (University of Michigan) Abstract: We consider a zero-sum stochastic differential controller-and-stopper game in which the state process is a controlled
More informationMathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio ( )
Mathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio (2014-2015) Etienne Tanré - Olivier Faugeras INRIA - Team Tosca November 26th, 2014 E. Tanré (INRIA - Team Tosca) Mathematical
More informationStochastic contraction BACS Workshop Chamonix, January 14, 2008
Stochastic contraction BACS Workshop Chamonix, January 14, 2008 Q.-C. Pham N. Tabareau J.-J. Slotine Q.-C. Pham, N. Tabareau, J.-J. Slotine () Stochastic contraction 1 / 19 Why stochastic contraction?
More informationA Class of Fractional Stochastic Differential Equations
Vietnam Journal of Mathematics 36:38) 71 79 Vietnam Journal of MATHEMATICS VAST 8 A Class of Fractional Stochastic Differential Equations Nguyen Tien Dung Department of Mathematics, Vietnam National University,
More informationRiccati Equations and Inequalities in Robust Control
Riccati Equations and Inequalities in Robust Control Lianhao Yin Gabriel Ingesson Martin Karlsson Optimal Control LP4 2014 June 10, 2014 Lianhao Yin Gabriel Ingesson Martin Karlsson (LTH) H control problem
More informationThermodynamic limit and phase transitions in non-cooperative games: some mean-field examples
Thermodynamic limit and phase transitions in non-cooperative games: some mean-field examples Conference on the occasion of Giovanni Colombo and Franco Ra... https://events.math.unipd.it/oscgc18/ Optimization,
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationOptimal Control. Quadratic Functions. Single variable quadratic function: Multi-variable quadratic function:
Optimal Control Control design based on pole-placement has non unique solutions Best locations for eigenvalues are sometimes difficult to determine Linear Quadratic LQ) Optimal control minimizes a quadratic
More informationOn the Stability of the Best Reply Map for Noncooperative Differential Games
On the Stability of the Best Reply Map for Noncooperative Differential Games Alberto Bressan and Zipeng Wang Department of Mathematics, Penn State University, University Park, PA, 68, USA DPMMS, University
More informationMean Field Game Theory for Systems with Major and Minor Agents: Nonlinear Theory and Mean Field Estimation
Mean Field Game Theory for Systems with Major and Minor Agents: Nonlinear Theory and Mean Field Estimation Peter E. Caines McGill University, Montreal, Canada 2 nd Workshop on Mean-Field Games and Related
More informationp 1 ( Y p dp) 1/p ( X p dp) 1 1 p
Doob s inequality Let X(t) be a right continuous submartingale with respect to F(t), t 1 P(sup s t X(s) λ) 1 λ {sup s t X(s) λ} X + (t)dp 2 For 1 < p
More informationLecture 4 Continuous time linear quadratic regulator
EE363 Winter 2008-09 Lecture 4 Continuous time linear quadratic regulator continuous-time LQR problem dynamic programming solution Hamiltonian system and two point boundary value problem infinite horizon
More informationPartial Differential Equations with Applications to Finance Seminar 1: Proving and applying Dynkin s formula
Partial Differential Equations with Applications to Finance Seminar 1: Proving and applying Dynkin s formula Group 4: Bertan Yilmaz, Richard Oti-Aboagye and Di Liu May, 15 Chapter 1 Proving Dynkin s formula
More informationMatthew Zyskowski 1 Quanyan Zhu 2
Matthew Zyskowski 1 Quanyan Zhu 2 1 Decision Science, Credit Risk Office Barclaycard US 2 Department of Electrical Engineering Princeton University Outline Outline I Modern control theory with game-theoretic
More informationStochastic Viral Dynamics with Beddington-DeAngelis Functional Response
Stochastic Viral Dynamics with Beddington-DeAngelis Functional Response Junyi Tu, Yuncheng You University of South Florida, USA you@mail.usf.edu IMA Workshop in Memory of George R. Sell June 016 Outline
More informationAn Introduction to Malliavin calculus and its applications
An Introduction to Malliavin calculus and its applications Lecture 3: Clark-Ocone formula David Nualart Department of Mathematics Kansas University University of Wyoming Summer School 214 David Nualart
More informationUCLA Chemical Engineering. Process & Control Systems Engineering Laboratory
Constrained Innite-Time Nonlinear Quadratic Optimal Control V. Manousiouthakis D. Chmielewski Chemical Engineering Department UCLA 1998 AIChE Annual Meeting Outline Unconstrained Innite-Time Nonlinear
More informationVerona Course April Lecture 1. Review of probability
Verona Course April 215. Lecture 1. Review of probability Viorel Barbu Al.I. Cuza University of Iaşi and the Romanian Academy A probability space is a triple (Ω, F, P) where Ω is an abstract set, F is
More informationControl of McKean-Vlasov Dynamics versus Mean Field Games
Control of McKean-Vlasov Dynamics versus Mean Field Games René Carmona (a),, François Delarue (b), and Aimé Lachapelle (a), (a) ORFE, Bendheim Center for Finance, Princeton University, Princeton, NJ 8544,
More informationApplications of Ito s Formula
CHAPTER 4 Applications of Ito s Formula In this chapter, we discuss several basic theorems in stochastic analysis. Their proofs are good examples of applications of Itô s formula. 1. Lévy s martingale
More informationStochastic Gradient Descent in Continuous Time
Stochastic Gradient Descent in Continuous Time Justin Sirignano University of Illinois at Urbana Champaign with Konstantinos Spiliopoulos (Boston University) 1 / 27 We consider a diffusion X t X = R m
More informationStochastic Differential Equations.
Chapter 3 Stochastic Differential Equations. 3.1 Existence and Uniqueness. One of the ways of constructing a Diffusion process is to solve the stochastic differential equation dx(t) = σ(t, x(t)) dβ(t)
More informationNonlinear Control Systems
Nonlinear Control Systems António Pedro Aguiar pedro@isr.ist.utl.pt 3. Fundamental properties IST-DEEC PhD Course http://users.isr.ist.utl.pt/%7epedro/ncs2012/ 2012 1 Example Consider the system ẋ = f
More informationin Bounded Domains Ariane Trescases CMLA, ENS Cachan
CMLA, ENS Cachan Joint work with Yan GUO, Chanwoo KIM and Daniela TONON International Conference on Nonlinear Analysis: Boundary Phenomena for Evolutionnary PDE Academia Sinica December 21, 214 Outline
More information