Stochastic Modelling and Applied Probability

Transcription

1 Stochastic Mechanics Random Media Signal Processing and Image Synthesis Mathematical Economics and Finance Stochastic Optimization Stochastic Control Stochastic Models in Life Sciences Edited by Advisory Board Stochastic Modelling and Applied Probability (Formerly: Applications of Mathematics) 59 B. Rozovskiĭ G. Grimmett D. Dawson D. Geman I. Karatzas F. Kelly Y. Le Jan B. Øksendal G. Papanicolaou E. Pardoux

2 Mou-Hsiung Chang Stochastic Control of Hereditary Systems and Applications

3 Mou-Hsiung Chang 4300 S. Miami Blvd. U.S. Army Research Office Durham, NC USA Managing Editors B. Rozovskiĭ Division of Applied Mathematics 182 George St. Providence, RI USA G. Grimmett Centre for Mathematical Sciences Wilberforce Road Cambridge CB3 0WB UK ISBN: e-isbn: DOI: / ISSN: Stochastic Modelling and Applied Probability Library of Congress Control Number: Mathematics Subject Classification (2000): 93E20, 34K50, 90C15 c 2008 Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013,USA), except for brief excerpts in connection with reviews or scholarly analysis.use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper springer.com

4 This book is dedicated to my wife, Yuen-Man Chang.

5 Preface This research monograph develops the Hamilton-Jacobi-Bellman (HJB) theory via the dynamic programming principle for a class of optimal control problems for stochastic hereditary differential equations (SHDEs) driven by a standard Brownian motion and with a bounded or an unbounded but fading memory. These equations represent a class of infinite-dimensional stochastic systems that become increasingly important and have wide range of applications in physics, chemistry, biology, engineering, and economics/finance. The wide applicability of these systems is due to the fact that the reaction of realworld systems to exogenous effects/signals is never instantaneous and it needs some time, time that can be translated into a mathematical language by some delay terms. Therefore, to describe these delayed effects, the drift and diffusion coefficients of these stochastic equations depend not only on the current state but also explicitly on the past history of the state variable. The theory developed herein extends the finite-dimensional HJB theory of controlled diffusion processes to its infinite-dimensional counterpart for controlled SHDEs in which a certain infinite-dimensional Banach space or Hilbert space is critically involved in order to account for the bounded or unbounded memory. Another type of infinite-dimensional HJB theory that is not treated in this monograph but arises from real-world application problems can often be modeled by controlled stochastic partial differential equations. Although they are both infinite dimensional in nature and are both in the infancy of their developments, the SHDE exhibits many characteristics that are not in common with stochastic partial differential equations. Consequently, the HJB theory for controlled SHDEs is parallel to and cannot be treated as a subset of the theory developed for controlled stochastic partial differential equations. Therefore, the effort for writing this monograph is well warranted. The stochastic control problems treated herein include discounted optimal classical control and optimal stopping for SHDEs with a bounded memory over a finite time horizon. Applications of the dynamic programming principles developed specifically for control of stochastic hereditary equations yield an infinite-dimensional Hamilton-Jacobi-Bellman equation (HJBE)

6 VIII Preface for finite time horizons discounted optimal classical control problem, a HJB variational inequality (HJBVI) for optimal stopping problems, and a HJB quasi-variational inequality (HJBQVI) for combined optimal classical-impulse control problems. As an application to its theoretical developments, characterizations of pricing functions in terms of the infinite-dimensional Black-Scholes equation and an infinite-dimensional HJBVI, are derived for European and American option pricing problems in a financial market that consists of a riskless bank account and a stock whose price dynamics depends explicitly on the past historical prices instead of just the current price alone. To further illustrate the roles that the theory of stochastic control of hereditary differential systems played in real-world applications, a chapter is devoted to the development of theory of combined optimal classical-impulse control that is specifically applicable to an infinite time horizon discounted optimal investment-consumption problem in which capital gains taxes and fixed plus proportional transaction costs are taken into consideration. To address some computational issues, a chapter is devoted to Markov chain approximations and finite difference approximations of the viscosity solution of infinitedimensional HJBEs and HJBVIs. It is well known that the value functions for most of optimal control problems, deterministic or stochastic, are not smooth enough to be a classical solution of HJBEs, HJBVIs, or HJBQVIs. Therefore, the theme of this monograph is centered around development of the value function as the unique viscosity solution of these equations or inequalities. This monograph can be used as an introduction and/or a research reference for researchers and advanced graduate students who have special interest in theory and applications of optimal control of SHDEs. The monograph is intended to be as much self-contained as possible. Some knowledge in measure theory, real analysis, and functional analysis will be helpful. However, no background material is assumed beyond knowledge of the basic theory of Itô integration and stochastic (ordinary) differential equations driven by a standard Brownian motion. Although the theory developed in this monograph can be extended with additional efforts to hereditary differential equations driven by semi martingales such as Levy processes, we restrain our treatments to systems driven by Brownian motion only for the sake of clarity in theory developments. This monograph is largely based on the current account of relevant research results contributed by many researchers on controlled SHDEs and on some research done by the author during his tenure as a faculty member at the University of Alabama in Huntsville and more recently as a member of the scientific staff at the U.S. Army Research Office. Most of the material in this monograph is the product of some recently published or not-yet-published results obtained by the author and his collaborators, Roger Youree, Tao Pang, and Moustapha Pemy. The list of references is certainly not exhaustive and is likely to have omitted works done by other researchers. The author apologizes for any inadvertent omissions in this monograph of their works.

7 Preface IX The author would like to thank Boris Rozovskiĭ for motivating the submission of the manuscript to Springer and for his encouragement. Sincere thanks also go to Achi Dosanjh and Donna Lukiw, Mathematics Editors of Springer, for their editorial assistance and to Frank Holzwarth, Frank Ganz, and Frank McGuckin for their help on matters that are related to svmono.cls and LaTex. The author acknowledges partial support by a staff research grant (W911NF-04-D-0003) from the U.S. Army Research Office for the development of some of the recent research results published in journals and contained in this monograph. However, the views and conclusions contained herein are those of the author and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the U.S. Army. Research Triangle Park, North Carolina, USA Mou-Hsiung Chang October 2007

8 Contents Preface...VII Notation...XV Introduction and Summary... 1 A. BasicNotation... 2 B. StochasticControlProblemsandSummary... 3 B1. OptimalClassicalControlProblem... 5 B2. OptimalStoppingProblem... 9 B3. DiscreteApproximations B4. OptionPricing B5. Hereditary Portfolio Optimization C. OrganizationofMonograph Stochastic Hereditary Differential Equations Probabilistic Preliminaries GronwallInequality Stopping Times Regular Conditional Probability Brownian Motion and Itô Integrals BrownianMotion ItôIntegrals Itô sformula Girsanov Transformation SHDE with Bounded Memory MemoryMaps TheAssumptions Strong Solution Weak Solution SHDE with Unbounded Memory MemoryMaps... 67

9 XII Contents 1.5 MarkovianProperties ConclusionsandRemarks Stochastic Calculus PreliminaryAnalysisonBanachSpaces Bounded Linear and Bilinear Functionals Fréchet Derivatives C 0 -Semigroups Bounded and Continuous Functionals onbanachspaces TheSpaceC TheSpaceM The Weighting Function ρ The S-Operator ItôandDynkinFormulas {x s,s [t, T ]} {(S(s),S s ),s 0} Martingale Problem ConclusionsandRemarks Optimal Classical Control ProblemFormulation TheControlledSHDE Admissible Controls StatementoftheProblem ExistenceofOptimalClassicalControl Admissible Relaxed Controls ExistenceResult DynamicProgrammingPrinciple Some Probabilistic Results Continuity of the Value Function TheDDP TheInfinite-DimensionalHJBEquation Viscosity Solution Uniqueness VerificationTheorems Finite-DimensionalHJBEquation SpecialFormofHJBEquation Finite Dimensionality of HJB Equation Examples ConclusionsandRemarks Optimal Stopping TheOptimalStoppingProblem ExistenceofOptimalStopping...208

10 Contents XIII TheInfinitesimalGenerator AnAlternateFormulation ExistenceandUniqueness HJBVariationalInequality VerificationTheorem Viscosity Solution ASketchofaProofofTheorem ConclusionsandRemarks Discrete Approximations Preliminaries TemporalandSpatialDiscretizations SomeLemmas SemidiscretizationScheme First Approximation Step: Piecewise Constant Segments Second Approximation Step: Piecewise Constant Strategies OverallDiscretizationError MarkovChainApproximation ControlledMarkovChains Optimal Control of Markov Chains Embedding the Controlled Markov Chain ConvergenceofApproximations FiniteDifferenceApproximation Finite Difference Scheme DiscretizationofSegmentFunctions A Computational Algorithm ConclusionsandRemarks Option Pricing PricingwithHereditaryStructure TheFinancialMarket ContingentClaims AdmissibleTradingStrategies Risk-Neutral Martingale Measures Pricing of Contingent Claims The European Contingent Claims The American Contingent Claims Infinite-Dimensional Black-Scholes Equation Equation Derivation Viscosity Solution HJBVariationalInequality Series Solution Derivations...325

11 XIV Contents AnExample ConvergenceoftheSeries The Algorithm ConclusionsandRemarks Hereditary Portfolio Optimization The Hereditary Portfolio Optimization Problem Hereditary Price Structure with Unbounded Memory TheStockInventorySpace Consumption-TradingStrategies SolvencyRegion Portfolio Dynamics and Admissible Strategies TheProblemStatement The Controlled State Process The Properties of the Stock Prices Dynkin s Formula for the Controlled State Process TheHJBQVI TheDynamicProgrammingPrinciple DerivationoftheHJBQVI Boundary Values of the HJBQVI TheVerificationTheorem Properties of Value Function Some Simple Properties Upper Bounds of Value Function The Viscosity Solution Uniqueness ConclusionsandRemarks References Index...401

12 Notation denotes is defined as. Z, ℵ 0,andℵ denote the sets of integers, nonnegative integers, and positive integers, respectively. R and R + denote the sets of real numbers and non-negative real numbers respectively. Q denotes the set of all rational numbers and Q + the set of all nonnegative rational numbers. R n denotes the n-dimensional Euclidean space equipped with the inner product defined by x y = n i=1 x iy i and the Euclidean norm defined by x =(x x) 1/2 for x =(x 1,x 2,...,x n ), y =(y 1,y 2,...,y n ) R n. R n + = {(x 1,x 2,...,x n ) R n x i 0,i=1, 2,...,n}. R n m - the space of all n m real matrices A =[a ij ] equipped with the norm A = { n i=1 m j=1 a ij 2 } 1/2. A - the transpose of the matrix A. S n - the set of all n n symmetric matrices. trace A = n i=1 a ii, the trace of matrix A =[a ij ] S n. a b = max{a, b}, a b = min{a, b}, a + = a 0, and a = (a 0) for all real numbers a and b. m :[0,T] C([t r, T ], R n ) C denotes the memory map. Ξ- a generic real separable Banach or Hilbert space equipped with the norm Ξ. B(Ξ)- the Borel σ-algebra of subsets of Ξ, i.e., the smallest σ-algebra of subsets of Ξ that contains all open (and hence all closed) subsets of Ξ. Ξ denotes the class of bounded linear functionals on Ξ equipped with the operator norm Ξ. Ξ denotes the class of bounded bilinear functionals on Ξ and equipped with the operator norm Ξ. L(Ξ,Θ) denotes the collection of bounded linear transformations (maps) Φ : Ξ Θ and equipped with the operator norm Φ L(Ξ,Θ).

13 XVI Notation L(Ξ) =L(Ξ,Ξ). C b (Ξ) denotes the class of bounded continuous functions Φ : Ξ Rand equipped with the norm Φ Cb (Ξ) =sup x Ξ Φ(x). DΦ(φ) Ξ denotes the first order Fréchet derivative of Φ : Ξ Rat φ Ξ. D 2 Φ(φ) Ξ denotes the second order Fréchet derivative of Φ : Ξ Rat φ Ξ. C 2 (Ξ) denotes the space of twice continuously Fréchet differentiable functions Φ : Ξ R. C 2 lip (Ξ) denotes the class of Φ C2 (Ξ) satisfying the following condition: there exists a constant K>0 such that D 2 Φ(x) D 2 Φ(y) Ξ K x y Ξ, x, y Ξ. C([a, b]; R n )( <a<b< ) the separable Banach space of continuous functions φ :[a, b] R n equipped with the sup-norm defined by φ = sup φ(t). t [a,b] C 1,2 lip ([0,T] Ξ) denotes the class of functions Φ :[0,T] Ξ Rthat is continuously differentiable with respect to its first variable t [0,T] and twice continuously Fréchet differentiable with respect to its second variable x Ξ and Φ(t, ) Clip 2 (Ξ) uniformly for all t [0,T]. t Φ(t, x 1,,x n )= t Φ(t, x 1,,x n ), i Φ(t, x 1,,x n )= x i Φ(t, x 1,, x n ), and ij Φ(t, x 1,,x n )= 2 x i x j Φ(t, x 1,,x n )fori, j =1, 2,,n. L 2 (a, b; R n )( <a<b< ) the separable Hilbert space of Lebesque square integrable functions φ :[a, b] R n equipped with the L 2 -norm 2 defined by b φ 2 = φ(t) 2 dt. a D([a, b]; R n )( <a<b< ) the space of functions φ :[a, b] R n that are continuous from the right on [a, b) and have a finite left-hand-limits on (a, b]. The space D([a, b]; R n ) is a complete metric space equipped with Skorohod metric as defined in Definition ( ). r>0 is the duration of the bounded memory or delay. C = C([ r, 0]; R n ). M R L 2 ρ((, 0]; R) the ρ-weighted separable Hilbert space equipped with the inner product 0 (x, φ), (y, ϕ) M = xy + ρ(θ)φ(θ)ϕ(θ)dθ, (x, φ), (y, ϕ) M,

14 Notation XVII and the Hilbertian norm (x, φ) ρ = (x, φ), (x, φ) ρ 1/2. In the above, ρ : (, 0] [0, ) is a certain given function. (Ω,F,P,F) (wheref = {F(s),s 0}) denotes a complete filtered probability space that satisfies the usual conditions. E[X G] denotes the conditional expectation of the Ξ-valued random variable X (defined on (Ω,F,P)) given the sub-σ-algebra G F. Denote E[X] =E[X F]. L 2 (Ω,Ξ; G) denotes the collection of G-measurable (G F)andΞ-valued random variables X such that X L2 (Ω,Ξ) X(ω) 2 ΞdP (ω) <. Ω Denote L 2 (Ω,Ξ) =L 2 (Ω,Ξ; F). Ta b (F) denotes the collection of F-stopping times τ such that 0 a τ b, P -a.s.. Ta b (F) =T (F) whena =0andb =. (Ω,F,P,F,W( )) (or simply W ( ) denotes the standard Brownian motion of appropriate dimension. L w Φ(x 1,,x n )= 1 m 2 j=1 2 j Φ(x 1,,x n ). x Φ(x) and 2 xφ(x) denote, respectively, the gradient and Hessian matrix of Φ : R n R. 1 {0} :[ r, 0] R,where1 {0} (0) = 1 and 1 {0} (θ) =0for r θ<0. C B = {φ + v1 {0} φ C,v R n } equipped with the norm φ + v1 {0} C B = φ C + v. Γ is the continuous isometric extension of Γ from C (respectively, C )to (C B) (respectively, (C B) ). φ : [ r, T ] R n (respectively, φ : (,T] R n is the extension of φ :[ r, 0] R n (respectively, φ :(, 0] R n ), where φ(t) =φ(0) if t 0 and φ(t) =φ(t) ift 0. {T (t),t 0} L(Ξ) denotes C 0 -semigroup of bounded linear operators on Ξ. J(t, ψ; u( )) denotes the discounted objective functional for the optimal classical control problem. U[t, T ] denotes the class of admissible controls the optimal classical control problem. Û[t, T ] denotes the class of admissible relaxed controls the optimal classical control problem. J(t, ψ; τ) denotes the discounted objective functional for the optimal stopping problem. V (t, ψ) denotes the value function for the optimal classical control problem, the optimal stopping problem, and the pricing function, etc. A denotes the weak infinitesimal generator of a Ξ-valued Markov process with domain D(A), where Ξ = C or Ξ = M. S denotes the shift operator with domain D(S). a denotes the integral part of a R. h (N) = r N for N ℵ.

15 XVIII Notation T (N) = T. h (N) t N = h (N) t. h (N) G κ denotes the liquidating function for the hereditary portfolio optimization problem. S κ denotes the solvency region for the hereditary portfolio optimization problem. M κ denotes the intervention operator for the hereditary portfolio optimization problem. S κ denotes the boundary of S κ. (C, T ) denotes the class of admissible consumption-trading strategies.