Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations

Transcription

1 Martino Bardi Italo Capuzzo-Dolcetta Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations Birkhauser Boston Basel Berlin

2 Contents Preface Basic notations xi xv Chapter I. Outline of the main ideas on a model problem 1 1. The infinite horizon discounted regulator 1 2. The Dynamic Programming Principle 2 3. The Hamilton-Jacobi-Bellman equation in the viscosity sense Comparison, uniqueness and stability of viscosity solutions 6 5. Synthesis of optimal controls and verification theorems Pontryagin Maximum Principle as a necessary and sufficient condition of optimality Discrete time Dynamic Programming and convergence of approximations The viscosity approximation and stochastic control Bibliographical notes 21 Chapter II. Continuous viscosity solutions of Hamilton-Jacobi equations Definitions and basic properties, Some calculus and further properties of viscosity solutions Some comparison and uniqueness results Lipschitz continuity and semiconcavity Lipschitz continuity Semiconcavity Special results for convex Hamiltonians Semiconcave generalized solutions and bilateral supersolutions Differentiability of solutions..._ A comparison theorem Solutions in the extended sense 84

3 vi CONTENTS 5.5. Differential inequalities in the viscosity sense Monotonicity of value functions along trajectories Bibliographical notes 95 Chapter III. Optimal control problems with continuous value functions: unrestricted state space The controlled dynamical system The infinite horizon problem Dynamic Programming and the Hamilton-Jacobi-Bellman equation Some simple applications: verification theorems, relaxation, stability Backward Dynamic Programming, sub- and superoptimality principles, bilateral solutions Generalized directional derivatives and equivalent notions of solution Necessary and sufficient conditions of optimality, minimum principles, and multivalued optimal feedbacks The finite horizon problem The HJB equation Local comparison and unbounded value functions Equivalent notions of solution Necessary and sufficient conditions of optimality and the Pontryagin Maximum Principle Problems whose HJB equation is a variational or quasivariational inequality The monotone control problem Optimal stopping Impulse control Optimal switching Appendix: Some results on ordinary differential equations Bibliographical notes 223 Chapter IV. Optimal control problems with continuous value functions: restricted state space Small-time controllability and minimal time functions HJB equations and boundary value problems for the minimal time function: basic theory Problems with exit times and non-zero terminal cost Compatible terminal cost and continuity of the value function The HJB equation and a superoptimality principle Free boundaries and local comparison results for undiscounted problems with exit times Problems with state constraints Bibliographical notes 282

4 CONTENTS vii Chapter V. Discontinuous viscosity solutions and applications Semicontinuous sub- and supersolutions, weak limits, and stability Non-continuous solutions Definitions, basic properties, and examples Existence of solutions by Perron's method Envelope solutions of Dirichlet problems Existence and uniqueness of e-solutions Time-optimal problems lacking controllability Boundary conditions in the viscosity sense Motivations and basic properties Comparison results and applications to exit-time problems and stability Uniqueness and complete solution for time-optimal control Bilateral supersolutions Problems with exit times and general targets Finite horizon problems with constraints on the endpoint of the trajectories ' Bibliographical notes 357 Chapter VI. Approximation and perturbation problems Semidiscrete approximation and e-optimal feedbacks Approximation of the value function and construction of optimal controls A first result on the rate of convergence Improving the rate of convergence Regular perturbations Stochastic control with small noise and vanishing viscosity Appendix: Dynamic Programming for Discrete Time Systems Bibliographical notes 395 Chapter VII. Asymptotic problems Ergodic problems Vanishing discount in the state constrained problem Vanishing discount in the unconstrained case: optimal stopping Vanishing switching costs Penalization Penalization of stopping costs Penalization of state constraints Singular perturbation problems The infinite horizon problem for systems with fast components Asymptotics for the monotone control problem Bibliographical notes 429

5 viii CONTENTS Chapter VIII. Differential Games Dynamic Programming for lower and upper values Existence of a value, relaxation, verification theorems Comparison with other information patterns and other notions of value Feedback strategies Approximation by discrete time games Bibliographical notes 468 Appendix A. Numerical Solution of Dynamic Programming Equations by Maurizio Falcone The infinite horizon problem The Dynamic Programming equation Synthesis of feedback controls Numerical tests Problems with state constraints Minimum time problems and pursuit-evasion games... y ' Time-optimal control Pursuit-evasion games Numerical tests Some hints for the construction of the algorithms Bibliographical notes 502 Appendix B. Nonlinear Hoo control by Pierpaolo Soravia Definitions Linear systems Woo control and differential games Dynamic Programming equation On the partial information problem Solving the problem Exercises 530 8r Bibliographical notes 531 Bibliography 533 Index 565