OPTIMAL CONTROL THEORY: APPLICATIONS TO MANAGEMENT SCIENCE AND ECONOMICS

Transcription

1 OPTIMAL CONTROL THEORY: APPLICATIONS TO MANAGEMENT SCIENCE AND ECONOMICS (SECOND EDITION, 2000) Suresh P. Sethi Gerald. L. Thompson Springer Chapter 1 p. 1/37

2 CHAPTER 1 WHAT IS OPTIMAL CONTROL THEORY? Chapter 1 p. 2/37

3 WHAT IS OPTIMAL CONTROL THEORY? Dynamic Systems: Evolving over time. Chapter 1 p. 3/37

4 WHAT IS OPTIMAL CONTROL THEORY? Dynamic Systems: Evolving over time. Time: Discrete or continuous; Optimal way to control a dynamic system. Chapter 1 p. 3/37

5 WHAT IS OPTIMAL CONTROL THEORY? Dynamic Systems: Evolving over time. Time: Discrete or continuous; Optimal way to control a dynamic system. Prerequisites: Calculus, Vectors and Matrices, ODE and PDE. Chapter 1 p. 3/37

6 WHAT IS OPTIMAL CONTROL THEORY? Dynamic Systems: Evolving over time. Time: Discrete or continuous; Optimal way to control a dynamic system. Prerequisites: Calculus, Vectors and Matrices, ODE and PDE. Applications: Production, Finance, Economics, Marketing and others. Chapter 1 p. 3/37

7 BASIC CONCEPTS AND DEFINITIONS A dynamic system is described by state equation: ẋ(t) = f(x(t),u(t),t), x(0) = x 0, (1) where x(t) is state variable, u(t) is control variable. The control aim is to maximize the objective function: J = T 0 F(x(t),u(t),t)dt + S[x(T),T]. (2) Usually the control variable u(t) will be constrained as follows: u(t) Ω(t), t [0,T], (3) Chapter 1 p. 4/37

8 BASIC CONCEPTS AND DEFINITIONS CONT. Sometimes, we consider the following constraints: (1) Inequality constraints (mixed) g(x(t),u(t),t) 0, t [0,T]. (4) (2) Constraints involving only state variables (pure) h(x(t),t) 0, t [0,T]. (5) (3) Terminal state x(t) X, (6) where X is called the reachable set of the state variable at time T. Chapter 1 p. 5/37

9 FORMULATIONS OF SIMPLE CONTROL MODELS Example 1.1 A Production-Inventory Model. We consider the production and inventory storage of a given good in order to meet an exogenous demand at minimum cost. Chapter 1 p. 6/37

10 THE PRODUCTION-INVENTORY MODEL State Variable Control Variable State Equation Objective Function I(t) = Inventory Level P(t) = Production Rate I(t) = P(t) S(t), I(0) = I0 Maximize State Constraint I(t) 0 Control Constraints Terminal Condition Exogenous Functions Parameters Ò T J = 0 [h(i(t)) + Ó c(p(t))]dt Ê 0 P min P(t) P max I(T) I min S(t) = Demand Rate h(i) = Inventory Holding Cost c(p) = Production Cost T = Terminal Time I min = Minimum Ending Inventory P min = Minimum Possible Production Rate P max = Maximum Possible Production Rate I 0 = Initial Inventory Level Chapter 1 p. 7/37

11 FORMULATIONS OF SIMPLE CONTROL MODELS Example 1.2 An Advertising Model. We consider a special case of the Nerlove-Arrow advertising model. Chapter 1 p. 8/37

12 THE ADVERTISING MODEL State Variable Control Variable G(t) = Advertising Goodwill u(t) = Advertising Rate State Equation Ġ(t) = u(t) δg(t), G(0) = G 0 Objective Function Maximize J = 0 [π(g(t)) u(t)]e ρt dt State Constraint Control Constraints 0 u(t) Q Terminal Condition Exogenous Function Parameters π(g) = Gross Profit Rate δ = Goodwill Decay Constant ρ = Discount Rate Q = Upper Bound on Advertising Rate G 0 = Initial Goodwill Level Chapter 1 p. 9/37

13 FORMULATIONS OF SIMPLE CONTROL MODELS Example 1.3 A Consumption Model. We consider a problem of an agent s consumption of his wealth over time in a way that maximizes his consumption utility over his lifetime. Chapter 1 p. 10/37

14 THE CONSUMPTION MODEL State Variable Control Variable W(t) = Wealth C(t) = Consumption Rate State Equation Ẇ(t) = rw(t) C(t), W(0) = W 0 Objective Function Max State Constraint W(t) 0 Control Constraint C(t) 0 Terminal Condition Exogenous Functions Parameters Ò T Ó J = 0 U(C(t))e ρt dt + B[W(T)]e ρt Ê U(C) = Utility of Consumption B(W) = Bequest Function T = Terminal Time W 0 = Initial Wealth ρ = Discount Rate r = Interest Rate Chapter 1 p. 11/37

15 HISTORY OF OPTIMAL CONTROL THEORY Calculus of Variations. Brachistochrone problem: path of least time Newton, Leibniz, Bernoulli brothers, Jacobi, Bolza. Pontryagin et al.(1958): Maximum Principle. Chapter 1 p. 12/37

16 FIGURE 1.1 THE BRACHISTOCHRONE PROBLEM Chapter 1 p. 13/37

17 NOTATION AND CONCEPTS USED = is equal to or is defined to be equal to or is identically equal to := is defined to be equal to, is identically equal to, is approximately equal to. implies is a member of. Chapter 1 p. 14/37

18 NOTATION AND CONCEPTS USED CONT. Let y be an n-component column vector and z be an m-component row vector, i.e., y = y 1 y 2. y n = (y 1,...,y n ) T and z = (z 1,...,z m ), (7) ẏ := dy/dt and ż := dz/dt are defined as ẏ = dy dt = (ẏ 1,,ẏ n ) T and ż = dz dt = (ż 1,...,ż m ), When n = m, we can define the inner product zy = Σ n i=1z i y i. (8) Chapter 1 p. 15/37

19 NOTATION AND CONCEPTS USED CONT. More generally, if A = {a ij } = a 11 a 12 a 1k a 21 a 22 a 2k... a m1 a m2 a mk is an m k matrix and B = {b ij } is a k n matrix, we define the matrix product C = {c ij } = AB, which is an m n matrix with components c ij = Σ k r=1a ir b rj. (9) Chapter 1 p. 16/37

20 DIFFERENTIATING VECTORS AND MATRICES W.R.T. SCALARS Let f : E 1 E k be a k-dimensional function of a scalar variable t. If f is a row vector, then df dt = f t = (f 1t,f 2t,,f kt ), a row vector. If f is a column vector, then df dt = f t = f 1t f 2t. f kt = (f 1t,f 2t,,f kt ) T, a column vector. Chapter 1 p. 17/37

21 DIFFERENTIATING SCALARS W.R.T. VECTORS If F(y,z) is a scalar function of n-dimensional column vector y and m-dimensional row vector z, n 2, m 2, then the gradients F y and F z are defined, respectively, as F y = (F y1,,f yn ), a row vector, (10) and F z = F z1., a column vector, (11) F zm Chapter 1 p. 18/37

22 DIFFERENTIATING VECTORS W.R.T. VECTORS If f : E n E m E k is a k-dimensional vector function f either row or column, k 2, i.e., f = (f 1,,f k ) or f = (f 1,,f k ) T, where f i = f i (y,z) depends on column y E n and row z E m, n 2,m 2, then f z denotes k m matrix: f 1 / z 1, f 1 / z 2, f 1 / z m f 2 / z 1, f 2 / z 2, f 2 / z m f z = = { f i / z j },... f k / z 1, f k / z 2, f k / z m (12) Chapter 1 p. 19/37

23 DIFFERENTIATING VECTORS W.R.T. VECTORS CONT. f y will denote the k n matrix f y = f 1 / y 1 f 1 / y 2 f 1 / y n f 2 / y 1 f 2 / y 2 f 2 / y n... = { f i / y j }. f k / y 1 f k / y 2 f k / y n Matrices f z and f y are known as Jacobian matrices. Note that f z = fz T = f z T = f T z, T where by fz T we mean (f T ) z and not (f z ) T. (13) Chapter 1 p. 20/37

24 DIFFERENTIATING VECTORS W.R.T. VECTORS CONT. Applying the rule (11) to F y in (9) and the rule (12) to F z in (10), respectively, we obtain F yz = (F y ) z to be the n m matrix F yz = F y1 z 1 F y1 z 2 F y1 z m F y2 z 1 F y2 z 2 F y2 z m... = { 2 F y i z j }, (14) F yn z 1 F yn z 2 F yn z m Chapter 1 p. 21/37

25 DIFFERENTIATING VECTORS W.R.T. VECTORS CONT. and F zy = (F z ) y to be the m n matrix F zy = F z1 y 1 F z1 y 2 F z1 y n F z2 y 1 F z2 y 2 F z2 y n... = { 2 F z i y j }. (15) F zm y 1 F zm y 2 F zm y n Chapter 1 p. 22/37

26 PRODUCT RULE FOR DIFFERENTIATION Let g be an n-component row vector function and f be an n-component column vector function of an n-component vector x, Then (gf) x = gf x + f T gx T = gf x + f T g x. (16) If g = F x, then (gf) x =(F x f) x =F x f x + f T F xx = F x f x + (F xx f) T. (17) Chapter 1 p. 23/37

27 VECTOR NORM AND NEIGHBORHOOD The norm of an m-component row or column vector z is defined to be z = z z 2 m. (18) Neighborhood N z0 of a point, e.g., N z0 = {z z z 0 < ε}, (19) where ε > 0 is a small positive real number. Chapter 1 p. 24/37

28 LITTLE-O NOTATION AND NORM OF A FUNCTION A function F(z) : E m E 1 is said to be of the order o(z), if lim z 0 F(z) z = 0. The norm of an m-dimensional row or column vector function z(t), t [0,T], is defined to be z = [ Σ m j=1 T z 2 j(τ)dτ ] 1 2. (20) 0 Chapter 1 p. 25/37

29 SOME SPECIAL NOTATION The concepts of left and right limits x(t ) = lim ε 0 x(t ε) and x(t + ) = lim ε 0 x(t + ε). (21) Discrete-time models (in Chapter 8-9) x k : state variable at time k. u k : control variable at time k. λ k : adjoint variables, respectively, at time k, x k := x k+1 x k : difference operator. x k and u k : quantities along an optimal path. Chapter 1 p. 26/37

30 SOME SPECIAL NOTATION CONT. bang function b 1 if W < 0, bang[b 1,b 2 ;W] = undefined if W = 0, b 2 if W > 0. (22) sat function sat[y 1,y 2 ;W] = y 1 if W < y 1, W if y 1 W y 2, y 2 if W > y 2. (23) Chapter 1 p. 27/37

31 SOME SPECIAL NOTATION CONT. Impulse Control t+ε imp(x 1,x 2,t) = lim ε 0 F(x,u,τ)dτ. (24) If the impulse is applied only at time t, then we can calculate the objective function as t J = t + 0 T F(x,u,τ)dτ + imp(x 1,x 2,t) (25) t F(x,u,τ)dτ + S[x(T),T]. Chapter 1 p. 28/37

32 CONVEX SET A set D E n is a convex set if for each pair of points y,z D, the entire line segment joining these two points is also in D, i.e., py + (1 p)z D, for each p [0, 1]. Given x i E n,i = 1, 2,...,l, we define y E n to be a convex combination of x i E n, if there exists p i 0 such that l l p i = 1 and y = p i x i. i=1 i=1 Chapter 1 p. 29/37

33 CONVEX HULL The convex hull of a set D E n is { l } l cod := p i x i : p i = 1, p i 0, x i D, i = 1,2,...,l. i=1 i=1 Chapter 1 p. 30/37

34 CONCAVE FUNCTION ψ : D E 1, is concave, if for each pair of points y,z D and for all p [0, 1], ψ(py + (1 p)z) pψ(y) + (1 p)ψ(z). If is changed to > for all y,z D with y z, and 0 < p < 1, then ψ is called a strictly concave function. If ψ(x) is a differentiable function on the interval [a,b], then it is concave, if for each pair of points y,z [a,b], ψ(z) ψ(y) + ψ x (y)(z y). Chapter 1 p. 31/37

35 FIGURE 1.2 A CONCAVE FUNCTION Chapter 1 p. 32/37

36 CONCAVE AND CONVEX FUNCTIONS If the function ψ is twice differentiable, then it is concave, if at each point in [a,b], ψ xx 0. In case x is a vector, ψ xx needs to be a negative definite matrix. If ψ : D E 1 defined on a convex set D E n is a concave function, then the negative of the function ψ, i.e., ψ : D E 1, is a convex function. Chapter 1 p. 33/37

37 AFFINE AND HOMOGENEOUS FUNCTIONS ψ : E n E 1 is said to be affine, if ψ(x) ψ(0) is linear. ψ : E n E 1 is said to be homogeneous of degree one, if ψ(bx) = bψ(x), where b is a scalar constant. Saddle Point ψ : E n E m E 1. For max x min y, a point (ˆx,ŷ) E n E m is called a saddle point of ψ(x,y), if ψ(x,ŷ) ψ(ˆx,ŷ) ψ(ˆx,y) for all x E n and y E m. Note also that ψ(ˆx,ŷ) = max x ψ(x,ŷ) = min y ψ(ˆx,y). Similarly, for min x max y. Just reverse the inequalities. Chapter 1 p. 34/37

38 FIGURE 1.3 A SADDLE POINT Chapter 1 p. 35/37

39 LINEAR INDEPENDENCE AND RANK OF A MATRIX A set of vectors a 1,a 2,...,a n from E n is said to be linearly dependent if there exist scalars p i not all zero such that n p i a i = 0. (26) i=1 If the only set of p i for which (1.25) holds is p 1 = p 2 = = p n = 0, then the vectors are said to be linearly independent. Chapter 1 p. 36/37

40 LINEAR INDEPENDENCE AND RANK OF A MATRIX CONT. The rank (or more precisely the column rank) of an m n matrix A, written rank(a), is the maximum number of linearly independent columns in A. An m n matrix is of full rank if rank(a) = n. Chapter 1 p. 37/37