SOLVING OPTIMAL FEEDBACK CONTROL OF CHINESE POPULATION DYNAMICS BY VISCOSITY SOLUTION APPROACH

Transcription

1 SOLVING OPTIMAL FEEDBACK CONTROL OF CHINESE POPULATION DYNAMICS BY VISCOSITY SOLUTION APPROACH Bing Sun, Bao-Zhu Guo, Insiue of Sysems Science Academy of Mahemaics and Sysem Sciences Academia Sinica, Beijing 8, China Fax: Absrac: In his paper, he opimal birh feedback conrol of a McKendrick ype age-srucured populaion dynamic sysem based on he Chinese populaion dynamics is considered. Adop he dynamic programming approach, o obain he Hamilon-Jacobi-Bellman equaion and prove ha he value funcion is is viscosiy soluion. By he derived classical verificaion heorem, he opimal birh feedback conrol is found explicily. A finie difference scheme is designed o solving numerically he opimal birh feedback conrol. Under he same consrain, by comparing wih differen conrols, he validiy of he opimaliy of he obained conrol is verified numerically. Copyrigh c 5 IFAC Keywords: Numerical soluions, Finie difference, Opimal conrol, Feedback conrol, Dynamic programming, Disribued-parameer sysems, Opimaliy.. INTRODUCTION Opimal conrol holds wo vividly developing orienaions: one is he absrac heory and he oher he compuaional mehod. For he laer, essenially speaking, here are hree approaches (Sargen, ). Firs one is ha by he necessary condiion of opimaliy, such as he Ponryagin s maximum principle, o solve a wo-poin boundary value problem mainly uilizing he muliple shooing mehod. Second is o conver he original problem ino a finie-dimensional nonlinear program by he discreizaion for he problem. The las one is by he parameerizaion of he conrol rajecory o ge a nonlinear program problem and hen adop proper seps o ackle Graduae School of he Chinese Academy of Sciences, Beijing 9, China. School of Compuaional and Applied Mahemaics, Universiy of he Wiwaersrand, Privae, Wis 5, Johannesburg, Souh Africa. i (von Sryk and Bulirsch, 99). Unforunaely, for above hree mehods, each one has is faal fauls. The Ponryagin maximum principle provides only necessary condiions for he opimal conrol and i is usually no in feedback form. The big difficuly for shooing mehod is he guess for he iniial daa o sar he ieraive numerical process. I demands ha he user undersands he essenial of he problem well in physics, which is ofen no a rivial ask (Bryson, 996). For oher wo mehods, he simplificaion for he original problem leads o he fall of he reliabiliy and accuracy, and when he degree of discreizaion and parameerizaion is very high, he work of compuaion sands ou and he solving process gives rise o curse of dimensionaliy (Bryson, 996). In his paper, differing from any one of above hree numerical mehods, a viscosiy soluion approach is adoped o ge he opimal feedback conrol of a McKendrick ype age-srucured populaion

2 dynamic sysem based on he Chinese populaion dynamics numerically. So, in some sense, our mehod can be seen as he fourh compuaional mehod o aack he opimal conrol problem. The paper is organized as follows. In Secion, he dynamic programming principle (DPP, for shor) for he value funcion of he opimal conrol problem is esablished. Some oher properies of he soluion as well as he coninuiy of he value funcion are presened. Secion is devoed o show ha he value funcion is jus he viscosiy soluion of he corresponding HJB equaion. In Secion 4, he opimal feedback conrol is formulaed by he value funcion under he smooh assumpion. In Secion 5, a finie difference scheme is formulaed o he HJB equaion of he populaion sysem wih linear quadraic opimal conrol. The numerical soluion of opimal feedback conrol is presened. The validiy of he opimaliy of he obained conrol is verified numerically in he las secion.. PROBLEM FORMULATIN AND DPP A McKendrick ype model of age-srucured populaion dynamics developed in (Song and Yu, 988) is a firs order parial differenial equaion wih nonlocal boundary condiion described by < r < a m, >, p(r, ) = p (r), r a m, p(r, ) + p(r, ) r a p(, ) = β = µ(r)p(r, ), a b(r)p(r, )dr, () where p(r, ) denoes he age densiy disribuion a ime and age r for a closed populaion. µ(r) is he relaive moraliy of he populaion, which saisfies r µ(ρ)dρ < for r < a m and a m µ(ρ)dρ =. a m is he highes age even aained by individuals of he populaion. b(r) = k(r)h(r). k(r) is he raio of females and h(r) is he feriliy paern of females saisfying a a h(r)dr =. Assume ha b(r) is bounded and measurable in [a, a ], he fecundiy period of females, < a < a < a m. p (r) is he iniial disribuion. β is he specific feriliy rae of he females a ime, which is considered o be he birh conrol of he populaion in macro-level. Le H = L (, a m ) be he sae space wih he usual inner produc, and induced norm. For any > s, le U[s, ] = β(τ) [β, β ] R + τ [s, ], β(τ) is measurable on [s, ]. Given T > and p H, he opimal conrol problem is o find an opimal conrol β ( ) U[, T ] such ha J(β ) = J(β), β( ) U[,T ] J(β) = T a m L(p(r, ), β, r, )drd + a m f (r, p(r, T ))dr () where p(r, ) is he soluion of equaion corresponding o β( ) and L, f saisfy f (r, p(r))dr, L(p(r), β, r, )dr C + C p, (, β) [, T ] [β, β ], p H, [f (r, p (r)) f (r, p (r))]dr, [L(p (r), β, r, ) L(p (r), β, r, )]dr C p p, (, β) [, T ] [β, β ], p, p H, L(p(r), β, r, )dr is coninuous in (, p, β) R + H [β, β ] for some consans C i, i =,,. () Define he ime-dependen operaors A β as follows: A β φ(r) = φ (r) µ(r)φ(r), φ D(A β ), D(A β ) = φ(r) φ, Aβ φ H, (4) a φ() = β b(r)φ(r)dr. Then he equaion can be wrien as a firs order evoluion equaion in H: a p(r, ) = A β p(r, ), p(r, ) = p (r). (5) From now on, use A β o denoe he operaor A β when β β independen of ime, and he semigroup generaed by A β will be denoed as T β. I is seen ha A β = A β.

3 Define a family of evoluion operaors T(, s; β), s < by T(, s; β)φ(r) φ(r + s)e r µ(ρ)dρ r +s, r s, a β( r) b(τ)φ(τ + s + r) = a e τ τ +s+r µ(ρ)dρ dτe r µ(ρ)dρ, r < s, φ H, s a. [ ] s T(, s; β) = T(, s + a ; β) [ s a ] n= a T(s + na, s + (n )a ; β), s > a (6) where [x] denoes he maximal ineger no exceeding x. T(, s; β) is uniquely deermined by β(τ) τ [s, ]. Limied by he lengh, heorems in his paper are given wihou proof. Theorem. Le A β be he adjoin operaor of A β in H. Then here exiss an operaor C on H ha is bounded, linear, self-adjoin and posiive definie such ha for each β [β, β ], A β C is a bounded linear operaor on H and sup A βc + β b a m. (7) β [β,β ] Furhermore, he se D defined by D = D(A β) = φ(r) φ(r), φ (r) µ(r)φ(r) H, φ(a m ) = (8) is independen of β [β, β ] and dense in H. Theorem. Le q( ) D. Then q( ), p(, ) is differeniable almos everywhere on [, T ] and d q( ), p(, ) d = A βq( ), p(, ), [, T ] a.e. (9) where p(r, ) = T(, ; β)p (r) for any p H and β( ) U[, T ]. A β = A β is he adjoin operaor of A β. By virue of heorem, p(r, ) = T(, ; β)p (r) is considered as he (weak) soluion of equaion in he sense of equaion 9, which is obained by inegraing equaion along he characerisics. The value funcion V (, p ) for he opimal conrol problem is defined by V (, p ) T = β( ) U[,T ] a m L(p(r, τ), β(τ), r, τ)drdτ () + f (r, p(r, T ))dr where p(r, τ) = T(τ, ; β)p (r) is he soluion of equaion corresponding o β( ) U[, T ]. Theorem [DPP]. For any < δ < T, V (, p ) = +δ L(p(r, τ), β(τ), r, τ)drdτ β( ) U[,+δ] () +V ( + δ, p(, + δ)), V (T, p ) = f (r, p (r))dr where p(r, τ) = T(τ, ; β)p (r). Theorem 4. Wih M and ω as in Lemma of (Guo and Yao, 996) and consans C i, i =,, in equaion, he following asserions are rue: (i). V (, p ) (+T )C +MC (+ω ) e ωt p, for all [, T ], p H. (ii). V (, p ) V (, p ) (+Mω )C e ωt p p, for all [, T ], p, p H. (iii). For any fixed p, V (, p ) is coninuous in.. VISCOSITY SOLUTIN TO HJB EQUATION For breviy in noaion, rewrie equaion 5 as dp = A β P, d () P() = P where P = p(, ), P = p ( ). Le P(τ) = T(τ, ; β)p, ψ(p(t )) = a m f (r, p(r, T ))dr, f(τ, P(τ), β(τ)) = a m L(p(r, τ), β(τ), r, τ)dr. Theorem 5. If V (, P ) C ([, T ] H), hen V saisfies he following HJB equaion V (, P )+ V p (, P ), A β P +f(, P, β) =, β [β,β ] () V (T, P )=ψ(p ), [, T ], P D(A β ). β [β,β ]

4 From now on, always assume ha A β is dissipaive for all β [β, β ]. Firs, give a definiion for he soluion of he HJB equaion in he viscosiy sense (Bardi and Capuzzo-Dolcea, 997). Le Ω be an open se of H and se USC([, T ] Ω) = upper-semiconinuous mappings u : [, T ] Ω R, LSC([, T ] Ω) = lower-semiconinuous mappings u : [, T ] Ω R. Definiion. U(, P ) USC([, T ] Ω) (respecively U(, P ) LSC([, T ] Ω)) is a subsoluion (respecively, supersoluion) of equaion on [, T ] Ω if for every es funcion Φ = ϕ + g, ϕ, ϕ p C([, T ] Ω; R), g C (Ω; R) saisfying equaion 4 and equaion 5 below: Range(ϕ p ) D, he mapping (, P ) A βϕ p (, P ), P from [, T ] Ω (4) o R is equiconinuous in β [β, β ]; here exiss a h : [, ) R such ha h is nondecreasing, h () = and (5) g(p ) = h( P ), P H and for he local maximum poin (respecively, he minimum poin) (, P ) of U Φ (respecively U + Φ), here is ϕ (, P )+ β [β,β ] (respecively, A βϕ p (, P ), P + f(, P, β). (6) ϕ (, P )+ A βϕ p (, P ), P + f(, P, β).) (7) β [β,β ] U(, P ) C([, T ] Ω) is a viscosiy soluion of equaion if i is boh a subsoluion and a supersoluion on [, T ] Ω. Theorem 6. The value funcion V (, P ) is a viscosiy soluion of he HJB equaion. 4. QUEST FOR OPTIMAL FEEDBACK CONTROL Theorem 7. Le V (, P ) be he value funcion. Then for any rajecory-conrol pair (P, β ), P = T(, ; β )P, he funcion T V (, P ) f(τ, P (τ), β (τ))dτ (8) is nondecreasing in [, T ]. Moreover, (P ( ), β ( )) is an opimal pair if and only if he above funcion is consan on [, T ]. Consequenly, if V (, P ) C ([, T ] H), β ( ) C [, T ], P () D(A β ()), hen (P ( ), β ( )) is an opimal pair if and only if V (, P ) + V p (, P ), A β P +f(, P, β ) =, (9) V (T, P (T )) = ψ(p (T )), [, T ]. The las asserion of Theorem 7 will lead o he classical verificaion heorem. Suppose he value funcion V (, P ) is smooh. Le S(Z) := arg min β [β,β ] V z (, Z), A β Z + f(, Z, β) = β [β, β ] H(Z, V z (, Z)) = V z (, Z), A β Z + f(, Z, β). () Then equaion 9 says ha β ( ) U[, T ] is an opimal conrol for he iniial sae p if and only if β S(P ) for [, T ] a.e. () where P = T(, ; β )p. Equaion is he formula for finding he opimal feedback conrol. 5. ALGORITHM AND STIMULATION In his secion, one will be limied o he following linear quadraic opimal conrol problem: J(β ) = J(β)= β( ) U[,T ] β( ) U[,T ] T [β β] () d + [p(r, T ) p(r)] dr where β denoes he mean criical feriliy rae of females in an ideal sociey and p(r) denoes he sable age densiy disribuion of he populaion wih he zero increasing rae. To solve he above equaion numerically, le j = T + j, j =,,,, N in which = T/N and N is an ineger. For given ε >, use he following approximaion: V p (, p), A β p = V p (, p), ε A βp Aβ p [ ( V, p + ε A ) βp V (, p) A β p A β p ε ] Aβ p. ε For he iniial sae p le p i = p i + A βp i A β p i ε, i =,,, M.

5 The following sufficien condiion for he sabiliy of he difference scheme will be assumed: ε max A βp i. () i M Se α j i = ε A β j p i. Alhough he convergence i of he described numerical algorihm has no been saed heoreically, he algorihm for he numerical soluion of equaion 9 is given as follows. Sep : iniializaion. Se Vi = V (T, p i ) = ψ(p i ), βi arg β [β,β ] p i = p i + A βp i A β p i ε, ψ(pi ) ψ(p i ) ε A β p i + a m [β β ] := arg H(β, p), i =,,, M. β [β,β ] (4) Sep : ieraion. V j+ i = ( α j i )V j i + α j i V j i a m (βj i β j ), V j+ β j+ i V j+ (5) i i arg A β p i β [β,β ] ε + a m [β β j+ ] for i =,,, M and j =,,,, N. From equaion, he opimal feedback conrol is βp (, p (, )) arg V p (, p (, )), β [β,β ] A β p (, ) + a (6) m [β β] in which p (, ) is he opimal rajecory of he sysem. Because i involves he opimal rajecory in equaion 6, finding he soluion of equaion is necessary. Uilize he difference scheme o compue he approximae soluion of he iniial value problem equaion, see (Song and Yu, 988). Adop he equally spaced grid mehod, o obain p i,j p i,j s + p i,j p i,j s = µ i p i,j, j J, i K, p i, = p (i s), i K, a p,j = β j s b i p i,j, j J i=a (7) in which b i = b(i s), β j = β(j s), i = a,, a, K s = a m, J s = T. Nex seps of finding he opimal feedback conrol are given in deail. I focuses on he solving he difference scheme equaion 7 o obain he opimal rajecory. Moreover, in he process, he algorihm of solving he HJB equaion will be called frequenly o ge he corresponding feedback conrol funcion. Algorihm of finding he opimal feedback conrol. Sep : Call he algorihm of solving he HJB equaion o ge he feedback conrol funcion β(, p ). Subsiue β(, p ) ino equaion 7 o ge he opimal rajecory p (, s ), s = s. Sep : Take p (, s ) as he iniial daa p as in he firs sep, and call he algorihm of solving he HJB equaion again o ge he feedback conrol β(s, p (, s )). Subsiue β(s, p (, s )) ino equaion 7 o ge he opimal rajecory p (, s ), s = s. Sep : Repea he processes above, o ge all feedback conrol funcions β(s j, p (, s j )), s j = j s, j =,,, J, ha is o say, β p (, p (, )) = β(, p ( )), (8) β(s, p (, s )),, β(t, p (, T )) ha is he opimal feedback conrol. Now i is he ime o find he numerical soluion of linear quadraic opimal conrol for he Chinese populaion based on he scheme equaion 4, 5 and 7. The iniial daa are exraced from he Chinese populaion sampling census in 989 (DDC, 99), which are ploed by MATLAB 6. as figure () (age-srucure of females, he age-srucure of whole populaion, he relaive moraliy); The age-srucure of an ideal sociey aken from (Song and Yu, 988) are lised in able, which is used o ge p(r) by muliplying he proporion in he able wih he oal ideal populaion N sum =, 4,. The feriliy paern h(r) is approximaed by he Gamma densiy disribuion curve in saisics (Song and Yu, 988) in which he peak value of he feriliy age is assumed o be 4. Oher parameers are lised as follows: T = 5, β =, β =, β =, a = 5, a = 49, a m = 99, ε =.. All compuaions are performed in Visual C++ 6. and numerical resuls are ploed by MATLAB 6.. Figure () is he value funcion V (, p (, )) and he opimal feedback conrol β p (, p (, )).

6 Table The age-srucure of an ideal sociey Age Proporion Age Proporion > β β β 5 β β β β 6 β * p (,p * (,)) v x 5 4 value funcion : v(,p * (,)) Fig.. Seven differen arbirarily chosen conrol β i and he opimal feedback conrol β p (, p (, )). x age srucure of females.5 opimal feedback conrol : β * p (,p * (,)) x 4 β.5 4 age srucure of whole populaion Fig.. The value funcion V (, p (, )) and he opimal feedback conrol β p (, p (, )). 6. CONCLUSIONS To end he paper, check he opimaliy of he numerical soluion of opimal feedback conrol is necessary. This is done by comparing he cos funcional J(β p (, p (, ))) of obained opimal conrol-rajecory pair wih ha of arbirarily chosen conrol β p, J(β p ) under he same iniial condiion, ha is, J(β p (, p (, ))) J(β p ). (9) Refer o figure (), o compue hese coss J(β i ) (i =,,, 7) for seven differen conrols β i U[, T ] respecively. The cos value J(βp (, p (, ))) is also compued. These resuls are lised in Table. I is seen ha for he opimal feedback conrol βp (, p (, )), J(βp (, p (, ))) = , which is evidenly less han oher coss J(β i ), i =,,, 7. In oher words, he numerical soluion of opimal birh feedback conrol for he Chinese populaion from is indeed go. Table Differen β and is corresponding cos J(β). β J(β) β J(β) β β β β β β β βp (, p (, )) relaive moraliy age Fig.. Age-srucures of females and whole populaion plus he relaive moraliy of Chinese populaion in 989. REFERENCES Bardi, M. and Capuzzo-Dolcea, I. (997). Opimal Conrol and Viscosiy Soluions of Hamilon-Jacobi-Bellman Equaions. Sysems & Conrol: Foundaions & Applicaions. Birkhäuser, Boson. Bryson, A. E. Jr.. (996). Opimal conrol - 95 o 985. IEEE Conrol Sysems. 6, 6. Deparmen of Demographic Census in Naional Bureau of Saisics of China. (99). China Populaion Saisics Yearbook. Scienific and Technical Documens Publishing House, Beijing (in Chinese). Guo, B. Z. and Yao, C. Z. (996). New resuls on he exponenial sabiliy of non-saionary populaion dynamics. Aca Mah. Sci. (English Ed.). 6, 7. Sargen, R. W. H. (). Opimal conrol. J. Compu. Appl. Mah.. 4, 6 7. Song, J. and Yu, J. Y. (988). Populaion Sysem Conrol. Springer-Verlag, Berlin. von Sryk, O. and Bulirsch, R. (99). Direc and indirec mehods for rajecory opimizaion. Annals of Operaions Research. 7, 57 7.