Optimal Stochastic Control in Continuous Time with Wiener Processes: General Results and Applications to Optimal Wildlife Management

Transcription

1 Iranian Jornal of Operaions Research Vol. 8, No., 017, pp Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] Opimal Sochasic Conrol in Coninos Time wih Wiener Processes: General Resls and Applicaions o Opimal Wildlife Managemen P. Lohmander 1 We presen a sochasic opimal conrol approach o wildlife managemen. The objecive vale is he presen vale of hning and mea, redced by he presen vale of he coss of plan damages and raffic accidens cased by he wildlife poplaion. Firs, general opimal conrol fncions and vale fncions are derived. Then, nmerically specified opimal conrol fncions and vale fncions of relevance o moose managemen in Sweden are calclaed and presened. Keywords: Sochasic opimal conrol, Wildlife managemen, Parial differenial eqaions, Moose. Manscrip was received on 06/08/017, revised on 3/11/017 and acceped for pblicaion on 30/11/ Inrodcion We sar wih a briefing on problem relevan pars of general sochasic opimal conrol heory, conine wih he derivaion of general fncion solions o he opimal wildlife managemen problem, and end p wih specific derivaions and resls of relevance o opimal moose managemen in Sweden.. The General Sochasic Opimal Conrol Problem The general opimal sochasic conrol mehodology in coninos ime is briefly presened here. Relaed inrodcions wih more deails are fond in Sehi and Thompson [5], Malliaris and Brock [4] and Winson [6]. Lohmander [1] presened conneced mehods in discree ime. We maximize he objecive fncion, T E F( X, U, ) d S( X, T) (1) 0 where X is he sae variable and U is he closed loop conrol variable. Time is represened by and T is he ime horizon. F(. ) is he insananeos profi rae and S(. ) is he salvage vale. E[. ] denoes expeced vale. z is a sandard Wiener process. dx f ( X, U, ) d G( X, U, ) dz, X x () Opimal Solions in cooperaion wih Linnaes Universiy, Umea, Sweden, peer@lohmander.com.

2 Opimal Sochasic Conrol in Coninos Time wih Wiener Processes 59 Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] According o he Bellman principle of opimaliy, we may deermine he vale fncion V(x, ) as he maximm of he sm of he ne reward dring he firs shor ime inerval, F(. )d, and he vale fncion direcly afer ha ime inerval: A Taylor approximaion gives V ( x, ) max E F( x,, ) d V ( x dx, d). (3) ( ) Vxx dx V d V ( x dx, d) V ( x, ) VxdX V d Vx ( dx )( d) (.) (4) In he Taylor approximaion, we need and Sochasic calcls ells s Hence, we ge and ( )( ) ( ) dx f d fg d dz G dz (5) dx d f d G d dz. (6) ( ) ( )( ) ( d) 0, ( d)( dz ) 0, ( dz ) d. (7) dx Frhermore, E(dz ) = 0. As a resl, we ge G d, (8) dx d 0. (9) VxxG V ( x dx, d) V ( x, ) Vx fd Vd d (.). (10) Hence, he vale fncion is approximaely given by VxxG V max E Fd V Vxfd Vd d (.), (11) which leads o

3 60 Lohmander Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] VxxG (.) 0 max E F Vxf V d. (1) d Wih d 0, we have o(.) 0, and hs d VxxG 0 max E F Vxf V. (13) Since V is no a fncion of, we obain he Hamilon-Jacobi-Bellman eqaion as follows: wih he bondary condiion VxxG V max E F Vxf, S x, T V x T 3. The Pariclar Sochasic Opimizaion Problem, (14). (15) We wan o maximize he expeced presen vale of wildlife managemen. We need he following noaions: = () is he conrol variable, he level of hning a ime, x = x() is he size of he wildlife poplaion, (k, p, f) are objecive fncion parameers. The ne revene of he hning and mea vales, k p, is a sricly concave fncion of he hning level, fx, which is proporional o he poplaion level, x, is he cos of desroyed fores planaions and cos of raffic accidens cased by he wildlife poplaion. The poplaion growh increases wih he size of he poplaion and decreases wih he hning level. The magnide of he sochasic poplaion changes depend on he sandard Wiener process, z, he size of he poplaion, and he risk parameer s. Wih r as he rae of ineres in he capial marke, we hen have he following problem: E e k p fx d 0 s.. dx ( gx ) d sx dz r max ( ) k 0, p 0, f 0, s 0. (16) The ne profi a a pariclar poin in ime is R(, x) ( k p fx). (17) The Hamilon-Jacobi-Bellman eqaion becomes

4 Opimal Sochasic Conrol in Coninos Time wih Wiener Processes 61 Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] s x J xx r ( x, ) J( x, ) e R( ( ), x( )) J x( x, )( gx( ) ( )). (18) Now, he problem is o deermine he vale fncion and he conrol fncion so ha he Hamilon- Jacobi-Bellman eqaion (HJBE) is saisfied. Le s assme ha he vale fncion can be expressed as follows: Then, by differeniaion, we ge As a resl, we can rewrie HJBE as follows: V x a bx cx (19) ( ), J x e V x e a bx cx (0) r r ( ( ), ) ( ) ( ). r J ( x( ), ) e ( b cx), (1) x r J ( x( ), ) e ( c), () xx r ( ( ), ) ( ). J x re a bx cx (3) s x Vxx ( x) rv ( x) R(, x) Vx ( x)( gx ), (4) 1 r( a bx cx ) ( k p fx) s x c ( b cx)( gx ). (5) We need o opimize he conrol, : 1 (6) max Z( ) ( k p fx) s x c ( b cx)( gx ). The firs order and second order maximal condiions are dz( ) k p b cx 0, d (7) d Z( ) p 0. d (8) The derived vale of is a niqe maximm,, which is obained o be dz ( ) k b cx d p 0, 1 Z Z( ) ( k p( ) fx) s x c ( b cx)( gx ), (30) (9)

5 6 Lohmander Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] 1 Z ( k b cx) p( ) fx s x c bgx cgx. (31) Using he opimal vales of he conrol, via he opimized conrol fncion, we ge k b cx k b cx 1 Z ( k b cx) p fx s x c bgx cgx. (3) p p The HJBE can now be wrien as s x Vxx( x) rv ( x) R(, x) Vx ( x)( gx ) (33) 1 r( a bx cx ) ( k p( ) fx) s x c ( b cx)( gx ) (34) k b cx k b cx r( a bx cx ) ( k b cx) p fx p p (35) 1 s x c bgx cgx ra rbx rcx p k b cx 4 p k b cx fx s x c bgx cgx (36) 1 1 ra rbx rcx k b cx fx s x c bgx cgx (37) 4p 1 1 ra rbx rcx ( k b 4c x bk 4ckx 4 bcx) fx s x c bgx 4p (38) cgx ra rbx rcx 1 ( k 4 p b bk 4 ckx 4 bcx 4 c x ) fx cs x bgx cgx (39) k b bk c( b k) c ra rbx rcx x x fx cs x bgx cgx. (40) 4 p p p Now, we have obained a qadraic fncion, ha always has o be zero. If he fncion is nozero, hen he HJBE is violaed. Since he fncion ms hold for all possible vales of x, he size of he poplaion, i is clear ha we have hree eqaions ha can be sed o deermine he parameers (a, b, c): k b bk c( b k) c 0 ra bg rb f x cs cg rc x. 4 p p p (41) Firs, he parameer c is deermined by solving

6 Opimal Sochasic Conrol in Coninos Time wih Wiener Processes 63 Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] Which leads o c p c p 0 cs cg rc x, x 0, c cs cg rc 0 s g r 0 c p( r g s ) p. Nex, he parameer b is deermined as follows: c( b k) 0 bg rb f x, x 0 p c( b k) c ck bg rb f 0 g r b f 0 p p p ck f p ck pf b b, c gr c p( g r) p and hs, by sbsiing for c, we ge p( r g s ) k pf ( r g s ) k f b ( ) ( ) ( ) ( ) b p r g s p g r r g s g r ( r g s ) k f k( g r s ) f b b. g s g s Finally, he parameer a is deermined: k b bk k b bk 0 ra ra 0 4p 4p ( k b) 4 ( ) 4. 4 pr k b bk pra k b pra a (4) (43) (44) (45) Now, he expression for b can be sed in he expression for a o ge

7 64 Lohmander Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] k( g r s ) f k g s k( g s ) k( g r s ) f a a 4pr 4 pr( g s ) f kg ks kg kr ks f k( g r) a a 4 pr( g s ) 4 pr( g s ) Now, we know he parameers of he vale fncion. They are explici fncions of he parameers in he iniially specified opimizaion problem: ( ),. (46) V x a bx cx (47) f k( g r) k( g r s ) f V x x p r g s x 4 pr( g s ) g s ( ) ( ). Nex, we will deermine he opimal conrol fncion. We know By sbsiing for b and c, we ge k b cx. p k g r s f g s p ( ) k p( r g s ) x p k( g r s ) f k p( g r s ) x g s k( g r s ) f k g s ( g r s ) x p k( g r s ) f k g s ( g r s ) x p (48) (49) (50) (51)

8 Opimal Sochasic Conrol in Coninos Time wih Wiener Processes 65 Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] k( g s ) ( k( g r s ) f ) ( g r s ) x p( g s ) kg ks kg kr ks f ( g r s ) x p( g s ) k( r g) f ( g r s ) x. p( g s ) Now, we know ha he opimal conrol is a linear fncion of he poplaion level. The inercep and he slope of his linear conrol fncion are explici fncions of he iniially specified parameers. 4. The Nmerically Specified Case Le s ener he pariclar nmerical parameers of a real problem. Lohmander [] esimaed revene and cos fncions and calclaed he opimal eqilibrim moose poplaion in Sweden, nder deerminisic assmpions and no disconing. However, he size of he moose poplaion is no exacly predicable and he capial marke ofen, b no always, incldes sricly posiive ineres raes. Random disrbances can have large effecs. We may deermine he opimal sochasic conrol of he moose poplaion in Sweden, based on he new general fncions derived here. The figres and fncions presened in Lohmander [] and [3] can be sed o derive he parameers of he sochasic conrol problem. Noe ha he qadraic objecive fncion in he sochasic opimal conrol problem here is an approximaion of he pariclar objecive fncion presened in [] and [3]. Boh fncions are sricly concave. The qadraic approximaion fis he original fncion very well wihin he seleced approximaion region. Of corse, he derived general eqaions can also be sed for oher animals and in oher conries of he world. These parameer vales were esimaed as g = 1/3, k = 600, p = 90, f = 90. Le r = 1/30. In Fig. 1. and Fig., we can inspec he opimal vale fncion and he opimal conrol fncion. Fig. 1 shows ha he opimal vale fncion is a sricly concave fncion of he size of he poplaion. The vale is a decreasing fncion of he sochasic parameer s. The opimal poplaion densiy, wih respec o he vale fncion, is a decreasing fncion of s. Noe ha he opimal poplaion densiy, wih respec o he vale fncion, is no eqal o he opimal poplaion densiy in he long rn. The vale may be high wih a raher high (iniial) poplaion, since he conrol (hning level) iniially can be high, gradally redcing he poplaion o a mch lower level. Fig. shows he opimal conrol, he hning level, as a fncion of he size of he poplaion. Noe ha he opimal conrol level is an increasing fncion of he sochasic parameer s. The inersecions of he alernaive conrol fncions and he line represening expeced poplaion growh wiho hning, show he poplaion levels and he hning levels where he expeced vales of he insan poplaion changes are zero. Observe ha, if s increases, hese expeced eqilibria inersecions obain lower poplaion densiies and lower hning levels. Since he firs derivaives of he alernaive conrol fncions wih respec o x are higher han he firs derivaives of he expeced poplaion growh wiho hning, wih respec o x, he expeced eqilibria are dynamically sable.

9 66 Lohmander Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] Figre 1. The opimal oal presen vale fncion, V(. ) as a fncion of he poplaion densiy, x, and he sochasic parameer s. Figre. The opimal conrol, he hning level,, as a fncion of he poplaion densiy, x, and he sochasic parameer s.

10 Opimal Sochasic Conrol in Coninos Time wih Wiener Processes 67 Downloaded from iors.ir a 1: on Wednesday November 14h 018 [ DOI: 10.95/iors ] 5. Conclsion We has derived and presened a sochasic opimal conrol approach o wildlife managemen. The objecive vale was he ne presen vale of hning and mea, redced by he presen vale of he coss of plan damages and raffic accidens cased by he wildlife poplaion. General opimal conrol fncions and vale fncions were derived. Then, nmerically specified opimal conrol fncions and vale fncions of relevance o moose managemen in Sweden were calclaed. Acknowledgemen The ahor appreciaes ravel grans from FORMAS. References [1] Lohmander, P., (007), Adapive Opimizaion of Fores Managemen in a Sochasic World, in Weinrab A. e al (Ed), Handbook of Operaions Research in Naral Resorces, Springer, Springer Science, Inernaional Series in Operaions Research and Managemen Science, New York, USA, [] Lohmander, P., (011), Hr många älgar har vi råd med? Vi Skogsägare, Deba, Nr 1. [3] Lohmander, P., (011), Älgens ekonomi och den ekonomisk opimala älgsammen, Skogen och Vile, SLU, Umea, November 4, hp:// [4] Malliaris, A.G., and Brock, W.A., (1984), Sochasic Mehods in Economics and Finance, Norh-Holland, Amserdam. [5] Sehi, S.P., and G.L.Thompson, G.L., (000), Opimal Conrol Theory, Applicaions o Managemen Science and Economics, Klwer Academic Pblishers, Boson. [6] Winson, W.L., (004), Operaions Research: Volme Two, Inrodcion o Probabiliy Models, Thomson Brooks/Cole, 4 ed.