LQ Dynamic Optimization and Differential Games

Size: px

Start display at page:

Download "LQ Dynamic Optimization and Differential Games"

Veronica Mills
5 years ago
Views:

1 LQ Dynamic Optimization and Differential Games Jacob Engwerda Tilburg University, The Netherlands JOHN WILEY & SONS, LTD

3 LQ Dynamic Optimization and Differential Games

5 LQ Dynamic Optimization and Differential Games Jacob Engwerda Tilburg University, The Netherlands JOHN WILEY & SONS, LTD

6 Copyright # 2005 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone (+44) (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or ed to permreq@wiley.co.uk, or faxed to (+44) Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The Publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Other Wiley Editorial Offices John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA , USA Wiley VCH Verlag GmbH, Boschstr. 12, D Weinheim, Germany John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop # 02-01, Jin Xing Distripark, Singapore John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1 Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging-in-Publication Data p. cm. ISBN Cancer Risk factors Mathematical models. I. Edler, Lutz, II. Kitsos, Christos Par., RC Q dc British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN (HB) ISBN (HB) Typeset in 10/12pt Times by Thomson Press (India) Limited, New Delhi Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production.

7 Contents Preface Notation and symbols ix xi 1 Introduction Historical perspective How to use this book Outline of this book Notes and references 14 2 Linear algebra Basic concepts in linear algebra Eigenvalues and eigenvectors Complex eigenvalues Cayley Hamilton theorem Invariant subspaces and Jordan canonical form Semi-definite matrices Algebraic Riccati equations Notes and references Exercises Appendix 59 3 Dynamical systems Description of linear dynamical systems Existence uniqueness results for differential equations General case Control theoretic extensions Stability theory: general case Stability theory of planar systems Geometric concepts Performance specifications Examples of differential games Information, commitment and strategies Notes and references Exercises Appendix 118

8 vi CONTENTS 4 Optimization techniques Optimization of functions The Euler Lagrange equation Pontryagin s maximum principle Dynamic programming principle Solving optimal control problems Notes and references Exercises Appendix Regular linear quadratic optimal control Problem statement Finite-planning horizon Riccati differential equations Infinite-planning horizon Convergence results Notes and references Exercises Appendix Cooperative games Pareto solutions Bargaining concepts Nash bargaining solution Numerical solution Notes and references Exercises Appendix Non-cooperative open-loop information games Introduction Finite-planning horizon Open-loop Nash algebraic Riccati equations Infinite-planning horizon Computational aspects and illustrative examples Convergence results Scalar case Economics examples Asimple government debt stabilization game A game on dynamic duopolistic competition Notes and references Exercises Appendix Non-cooperative feedback information games Introduction Finite-planning horizon 362

9 CONTENTS vii 8.3 Infinite-planning horizon Two-player scalar case Computational aspects Preliminaries A scalar numerical algorithm: the two-player case The N-player scalar case Convergence results for the two-player scalar case Notes and references Exercises Appendix Uncertain non-cooperative feedback information games Stochastic approach Deterministic approach: introduction The one-player case The one-player scalar case The two-player case A fishery management game A scalar numerical algorithm Stochastic interpretation Notes and references Exercises Appendix 481 References 485 Index 495

11 Preface Interaction between processes at various levels takes place everywhere around us. Either in industry, economics, ecology or on a social level, in many places processes influence each other. Particularly in those cases where people can affect the outcome of a process, the question arises how do people come to take a particular action. To obtain a clearer view of this question within mathematics the paradigms of optimal control theory and game theory evolved, and dynamic game theory resulted as a merging of both these topics. This theory brings together the issues of optimizing behavior, the presence of multiple agents, enduring consequences of decisions and robustness with respect to variability in the environment. Within this field, linear quadratic differential games developed and these play an important role for three reasons: first, many applications of differential game theory fall into this category, secondly, there are many analytical results available and, thirdly, efficient numerical solution techniques can be used to solve these games. Going through the literature, one can find many instances of results dealing with linear quadratic differential games. Unfortunately, there is no textbook focusing on this subject and giving a rigorous self-contained treatment of this theory. This textbook intends on the one hand to fill this gap and on the other hand to show the relevance of this theory by illustrating the theoretical concepts and results by means of simple economics examples. Given this background, the organization of the book was chosen as follows. The last four chapters of the book deal with differential games. Since the theoretical development of these chapters sets out some knowledge on the one-player case and dynamic optimization techniques, these chapters are preceded by a rigorous treatment of this one-player case, the so-called regular linear quadratic control problem, and a chapter on dynamic optimization theory. To tackle these issues, however, one needs to be familiar with some preliminary work on linear algebra and dynamical systems. For that reason those subjects are dealt with first. The first chapter gives some historical developments and an outline of the book. Having worked for more than two years on this book I ask myself whether it was worth all the trouble. Was it worth spending so much spare time and research time to produce something from which all one can hope is that it will be digestible and does not contain too many flaws and, on the other hand, does not contain a number of closely related interesting issues leaving the reader with information without the theory to back it up? The main motivation for writing this book was in the hope that it somehow might contribute a small amount to people s understanding and that some people will appreciate this book and use it for this purpose...

12 x PREFACE Last, but not least, I would like to thank the people who have contributed, most of them unknowingly, to this book. First of all my wife, Carine, and children, Elsemiek, Ton and Heiko, for offering me the chance to use my spare time for doing this. Next, I am indebted to Hans Schumacher for many discussions over the past decade concerning various issues on dynamic games, and to both him and Malo Hautus for the use of their excellent course notes on dynamic optimization. Furthermore Arie Weeren, Rudy Douven and Bram van den Broek contributed indirectly to the sections on differential games through their Ph.D. theses. Finally, Hendri Adriaens is acknowledged for his technical support on LaTEX and Tomasz Michalak for his proof-reading of an early version of this book. Answers to the exercises included in this book can be found at com/go/engwerda

13 Notation and symbols Matrix (Operations) x T transpose of the vector x k x k 2 length x T x of vector x 0 zero; zero vector; zero matrix I identity matrix A 1 inverse of matrix A A T transpose of matrix A adj(a) adjoint of matrix A det(a) determinant of matrix A rank(a) rank of matrix A trace(a) trace of matrix A ðaþ spectrum of matrix A ðaj S Þ spectrum of matrix A restricted to the invariant subspace S A i ðbþ matrix A with ith column replaced by b A ji matrix A with row j and column i deleted Ker A null space of A NðAÞ null space of A Im A column space of A RðAÞ column space of A diagfa,bg diagonal matrix with diagonal entries A and B dim(s) dimension of subspace S E eigenspace corresponding with eigenvalue E g generalized eigenspace corresponding with eigenvalue E c center subspace E s stable subspace E u unstable subspace vec(a) vector of stacked columns of matrix A A B Kronecker product of matrices A and B S V direct sum subspaces S and V S? orthogonal complement of subspace S C 0 set of complex numbers with non-positive real part C þ 0 set of complex numbers with non-negative real part C n set of complex vectors with n entries C nm set of n m matrices with complex entries

14 xii NOTATION AND SYMBOLS set of continuous differentiable functions CD set of continuous functions satisfying some differentiability properties (definition 3.4) i set of control functions of player i aff set of affine functions of the state variable F set of all stabilizing feedback matrices L 2 set of all measurable Lebesgue square integrable functions on ½0; 1Þ L 2;loc set of all measurable functions that are square integrable over all finite intervals ½0; TŠ M inv set of M-invariant subspaces N set of natural numbers P pos set of n-dimensional M-invariant graph subspaces Q set of rational numbers R set of real numbers R n set of vectors with n real entries R nm set of n m matrices with real entries N d set of bargaining problems (see Chapter 6) U set of control functions for which the differential equation has a solution in the extended sense U re fu 2UjJðuÞ exists as a finite number and lim t!1 xðtþ exists g U s set of locally square integrable control functions yielding a stable closed-loop system C 1 Miscellaneous ARE ij f ðx 0 i f EðS; dþ inf IðS; dþ KðS; dþ lim NðS; dþ 0ðhÞ n i¼1 p i RDE sup jzj Re(z) x y algebraic Riccati equation Kronecker, ij ¼ 1ifi¼j, ij ¼ 0 otherwise Fréchet/Gateaux differential of f at x 0 with increment h Fréchet derivative of f ith Fréchet partial derivative of f egalitarian bargaining solution infimum ideal point Kalai Smorodinsky bargaining solution limit Nash bargaining solution higher-order terms in h product of the np i variables Riccati differential equation supremump ffiffiffiffiffiffiffiffiffiffiffiffiffiffi modulus, x 2 þ y 2, of complex number z ¼ x þ iy real part, x, of complex number z ¼ x þ iy vector inequality x i > y i ; i ¼ 1;...; N

15 1 Introduction 1.1 Historical perspective Dynamic game theory brings together four features that are key to many situations in economics, ecology and elsewhere: optimizing behavior, the presence of multiple agents/ players, enduring consequences of decisions and robustness with respect to variability in the environment. To deal with problems which have these four features the dynamic game theory methodology splits the modeling of the problem into three parts. One part is the modeling of the environment in which the agents act. To obtain a mathematical model of the agents environment a set of differential or difference equations is usually specified. These equations are assumed to capture the main dynamical features of the environment. A characteristic property of this specification is that these dynamic equations mostly contain a set of so-called input functions. These input functions model the effect of the actions taken by the agents on the environment during the course of the game. In particular, by viewing nature as a separate player in the game who can choose an input function that works against the other player(s), one can model worst-case scenarios and, consequently, analyze the robustness of the undisturbed game solution. A second part is the modeling of the agents objectives. Usually the agents objectives are formalized as cost/utility functions which have to be minimized. Since this minimization has to be performed subject to the specified dynamic model of the environment, techniques developed in optimal control theory play an important role in solving dynamic games. In fact, from a historical perspective, the theory of dynamic games arose from the merging of static game theory and optimal control theory. However, this merging cannot be done without further reflection. This is exactly what the third modeling part is about. To understand this point it is good to summarize the rudiments of static games. Most research in the field of static game theory has been and is being concentrated on the normal form of a game. In this form all possible sequences of decisions of each player are set out against each other. So, for example, for a two-player game this results in a matrix structure. Characteristic for such a game is that it takes place in one moment of LQ Dynamic Optimization and Differential Games # 2005 John Wiley & Sons, Ltd J. Engwerda

Field Geophysics THIRD EDITION. John Milsom University College London

Field Geophysics THIRD EDITION John Milsom University College London Field Geophysics The Geological Field Guide Series Basic Geological Mapping, Third edition John Barnes The Field Description of Metamorphic