An Age-Structured Model with Immune Response of HIV Infection: Modeling and Optimal Control Approach

Transcription

1 An Age-Structured Model with Immune Response of HIV Infection: Modeling and Optimal Control Approach Hee-Dae Kwon, Jeehyun Lee, Myoung-ho Yoon March 3, 212 Abstract This paper develops and analyzes an age-structured model of HIV infection with various compartments, including target cells, infected cells, viral loads and immune effector cells, to provide a better understanding of the interaction between HIV and the immune system. We show that the proposed model has one uninfected steady state and several infected steady states. We conduct a local stability analysis of these steady states by using a generalized Jacobian matrix method in conjunction with the Laplace transform. In addition, we consider various techniques and ideas from optimal control theory to derive optimal therapy protocols by using two types of dynamic treatment methods representing reverse transcriptase inhibitors and protease inhibitors. We derive the necessary conditions (an optimality system) for optimal control functions by considering the first variations of the Lagrangian. Further, we obtain optimal therapy protocols by solving a large optimality system of equations through the use of a difference scheme based on the Runge-Kutta method. The results of numerical simulations indicate that the optimal therapy protocols can facilitate long-term control of HIV through a strong immune response after the discontinuation of the therapy. Keywords: HIV Dynamics, Age-Structured Model, Optimal Control, Gradient Method. 1 Introduction Substantial progress has been made in the mathematical modeling of HIV infection and treatment strategies [2, 2, 21, 24, 26, 29, 3. Existing models explore mainly the dynamics of CD4+ T helper cells, virus production/clearance, and the effects of various anti-retroviral drug treatments. In general, during primary HIV infection, the viral load in plasma increases, peaks, and then declines within several weeks. Stafford et al. [3 address this decline by developing various models of primary HIV infection and comparing predictions from the models with data from patients. Callaway and Perelson [8 investigate the ability of several models of HIV infection to explain how low viral loads can be sustained. Perelson and Nelson [24 explain how mathematical modeling can facilitate the discovery of some important features of HIV pathogenesis and impact the way in Department of Mathematics, Inha University, Yonghyundong, Namgu, Incheon , Republic of Korea (hdkwon@inha.ac.kr). Department of Computational Science and Engineering, Yonsei University, Shinchondong, Seodaemungu, Seoul , Republic of Korea (ezhyun@yonsei.ac.kr). Department of Mathematics, Yonsei University, Shinchondong, Seodaemungu, Seoul , Republic of Korea (youngrise@naver.com). 1

2 which HIV patients are treated with efficient anti-retroviral drugs. Nowak et al. [23 provide some analytic solutions for the emergence of a resistant virus under a single-drug therapy. Herz et al. [13 introduce a delay model for considering the time between the infection of target cells and the production of virions. Subsequently, Culshaw and Ruan [9 examine the effects of a delay on the stability of the endemical equilibrium. Age-structured models have received considerable attention from researchers interested in exploring the epidemiology of HIV. Thieme and Castillo-Chavez [31 examine the effects of infectionage-dependent infectivity on the dynamics of HIV transmission in a homogeneously mixing population. Kirschner and Webb [16 propose an age-structured model to account for the mechanism underlying AIDS chemotherapy. Recent studies have developed age-structured models for analyzing the within-host dynamics of HIV to provide a better understanding of the interaction between the immune system and HIV [22, 27. Nelson et al. [22 propose and analyze an age-structured model that allows for variations in the death rate for infected T cells and the production rate for virions. Rong et al. [27 extend the age-structured model by incorporating combination therapies and considering the effects of anti-retroviral therapy on viral dynamics. Previous studies have demonstrated that the long-term use of a single anti-retroviral drug can lead to the emergence of a resistant virus [18, 23. Thus, the most widely used strategy for HIV patients is highly active anti-retroviral therapy (HAART), which uses one or more reverse transcriptase inhibitors (RTIs) and a protease inhibitor (PI). However, although HAART can successfully reduce and maintain low viral loads in many patients, its prolonged use entails severe complications. In addition, HAART can be extremely costly in developing countries. In this regard, a number of researchers have searched for optimal treatment strategies that can reduce drug side effects, virus mutations, and complex/expensive medication burdens. There is no general consensus on optimal treatment strategies or interruption schemes. One way to consider optimal treatment schedules is to use a mathematical model for HIV infection in conjunction with control theory. Some studies have suggested continuous optimal treatment schedules based on open-loop control [2, 3, 1, 17, and others have used feedback control laws chosen on a real-time basis through the observation of the full or partial state of the system [4, 7, 28. More recently, Jang et al. [14 consider the free terminal time optimal control problem to derive the minimum treatment duration as well as the optimal multidrug treatment method. Banks et al. [5 explore some problems associated with optimal feedback control and state estimators for HIV infection by considering the state-dependent Riccati equation (SDRE) approach. Banks et al. [6 apply a receding horizon observer to an HIV feedback control problem with a long measurement time to derive optimal treatment methods for HIV progression. Kwon et al. [19 investigate an optimal control problem for an age-structured model with three compartments including target cells, infected cells, and viral loads. This paper develops an age-structured model with the immune response of HIV infection to derive an optimal multidrug therapy. The proposed model extends the age-structured models in [22, 27 by incorporating immune effector cells (cytotoxic T lymphocytes or CTLs) and combining two types of therapies (RTIs and PIs). In addition, we investigate the steady states of the model with no drug therapy and their local stability. The analytical results indicate that the proposed model has one uninfected steady state that is unstable and two infected steady states that are stable. In addition, the numerical results demonstrate that we can determine an optimal treatment scheme in which a patient moves to an immune-dominant state after discontinuing his or her therapy. The rest of this paper is organized as follows: Section 2 formulates and analyzes an agestructured HIV model that generalizes the model in [22 by incorporating immune effector cells. 2

3 Section 3 presents the formulation for the optimal control problem and derives the corresponding optimality system by considering the first variations of the Lagrangian. Section 4 introduces an algorithm based on the gradient method to solve an optimality system and presents the results of numerical simulations, demonstrating the effectiveness of the continuous optimal therapy, and Section 5 concludes. The Appendix provides detailed calculations. 2 Age-Structured Model We now introduce an age-structured model of HIV infection. The model has four state variables: uninfected CD4+ T cells T(t); infected CD4+ T cells structured by the age a of their infection, T (a,t); virions V (t); and immune effector cells E(t). A system of three ordinary differential equations and one first order hyperbolic equation describing HIV dynamics is given by dt T t dv de + T a λ dt(t) (1 ǫ 1 (t))kv (t)t(t) δ(a)t (a,t) m(a)t (a,t)e(t) (1 ǫ 2 (t)) P(a)T (a,t)da cv (t) λ E + b E T (a,t)da T E(t) d E T (a,t)da (a,t)da + K b T E(t) δ E E(t). (a,t)da + K d This model assumes that uninfected CD4+ T cells are produced at the constant rate λ and die at the rate d. Here kv T, where k denotes the infection rate, indicates the infection process in which infected cells T are produced through interactions between uninfected target cells T and virions V. The death rate δ(a) and the virion production rate P(a) for T are assumed to be functions of the age of cellular infection, a, and virions V are assumed to be cleared at the constant rate c. In addition, we assume that da 1; that is, the time unit for the age of infection is the same as that for the clock time. Because the model includes a first order hyperbolic equation, we need to introduce the boundary and initial conditions. Infected CD4+ T cells of age zero are created by infection, that is, (2.1) T (,t) (1 ǫ 1 (t))kv (t)t(t). (2.2) In addition, we impose specific initial conditions T() T, T (a,) T (a), V () V and E() E. The control term ǫ 1 (t) represents the effectiveness of RTIs, which block new infection. Thus, the infection rate k is reduced to (1 ǫ 1 (t))k, where a 1 ǫ 1 (t) b 1 < 1. The control term ǫ 2 (t) represents the effectiveness of PIs, which reduce the number of infectious virions. Thus, the production rate P(a) is reduced to (1 ǫ 2 (t))p(a), where a 2 ǫ 2 (t) b 2 < 1. Here a i and b i, (i 1, 2) represent the minimum and maximum drug efficacy, respectively. Infected CD4+ T cells are cleared by the action of immune effector cells E. This action can be expressed as m(a)t (a,t)e(t). The articles [2, 3 provide a more detailed description of the variables in the model. 3

4 Parameter Value Description λ 1(cells/µl day) Production(source) rate of CD4+T cells d.1(1/day) Death rate of CD4+T cells k (µl/virion day) Infection rate of virus δ(a) Death rate of infected cell m(a) Immune-induced clearance rate P(a) Virion production kernel c 13(1/day) Natural death rate of virus λ E , (cells/µl day) Production rate of immune effector cells b E.33(1/day) Maximum birth rate of immune effector cells K b.1(cells/µl) Saturation constant for the birth of immune effector cells d E.25(1/day) Maximum death rate of immune effector cells K d.5(cells/µl) Saturation constant for the death of immune effector cells δ E.1(1/day) Natural death rate of immune effector cells Table 1: Descriptions and numerical values of the parameters in the HIV model Remark 2.1. With the above boundary and initial conditions and a smooth enough control function, there exists a unique solution to system (2.1) that remains bounded and nonnegative for t > (see [19, 22, 32). Model (2.1) contains several constant parameters and function parameters that must be assigned for numerical simulations. Table 1 provides a summary of the descriptions and numerical values for these parameters, which are extracted mainly from [2, 3, 5. It is reasonable to expect that a a max so that the integral in (2.1) is not necessarily an infinite integral. The virus production kernel P(a) has the maximum production rate P max because cellular resources ultimately limit how rapidly virions can be produced. 2.1 Analysis of the Age-Structured Model With no drug treatment (ǫ 1 ǫ 2 ), model (2.1) has several steady states. To determine the steady states, we solve the equations λ dt e kt e V e dt e da δ(a)t e (a) m(a)e et e (a) P(a)T e (a)da cv e λ E + b E Te (a)da Te E e d E Te (a)da (a)da + K b Te E e δ E E e, (a)da + K d (2.3) 4

5 with the initial condition T e () kt ev e. Here (T e, T e (a), V e, E e ) is the steady state of the age-structured model. We can easily find a trivial or uninfected steady state: T te λ d, T te (a), V te, E te λ E δ E. (2.4) To consider other steady states, we need the following new notations: σ(a) e a δ(τ)+m(τ)eedτ, N P P(a)σ(a)da, N 1 Solving the second equation in (2.3) with the initial condition, we have The third equation in (2.3) implies that Thus, we get cv e The first equation in (2.3) implies σ(a)da, Λ λ cd kn P. (2.5) T e (a) T e ()σ(a) kt e V e σ(a). (2.6) V e λ dt e kt e P(a)kT e V e σ(a)da kt e V e N P. T e Substituting (2.7) and (2.8) into (2.6), we obtain T e (a) kt e V e σ(a) (λ c kn P. (2.7) λ d kt e k λn P c d k. (2.8) cd kn P )σ(a) Λσ(a). (2.9) In addition, substituting (2.9) into the fourth equation in (2.3) and rearranging yields which implies that λ E + b EN 1 Λ E e d EN 1 Λ E e δ E E e, N 1 Λ + K b N 1 Λ + K d (N 1 Λ + K b )(N 1 Λ + K d )(λ E δ E E e ) +b E N 1 Λ(N 1 Λ + K d )E e d E N 1 Λ(N 1 Λ + K b )E e. (2.1) Finally, we get the non-trivial or infected steady states T e and we determine E e by solving equation (2.1). c, Te kn (a) Λσ(a), V e λn P P c d k, (2.11) 5

6 A general analysis of the proposed model s steady states and their local stability is challenging because of the high nonlinearities in equation (2.1). However, we are still interested in the ability of the model to demonstrate multiple locally asymptotically stable steady states, and thus, we calculate steady states numerically and investigate their stability by using a generalized Jacobian matrix method with given numerical values for the parameters. This process requires the method of characteristics and the Laplace transform of each age-dependent term (see [22 and the references therein). Now we need an explicit functional form of the virus production kernel P(a). According to previous research [22, we may consider one functional form of kernels that may capture the features of the biology, where the maximum production rate P max is provided because cellular resources ultimately limit how rapidly virions can be produced. We can define a delayed or non-delayed exponential function as follows: { P max (1 exp β(a d1) ) if a d 1 P(a) (2.12) otherwise, where β controls how rapidly the saturation level P max is reached and d 1 denotes the delay in virus production (that is, it takes some time d 1 after the initial infection for the first virions to be produced). For simplicity, we assume that the functions δ(a) and m(a) are constants for deriving steady states and considering their stability: 2.2 Stability of the Steady States δ(a) δ.7 and m(a) m.1. In this section, we demonstrate that the model has multiple locally asymptotically stable steady states by calculating its steady states and conducting a stability analysis. The standard Jacobian matrix method is generally used to determine the stability of a system of ordinary differential equations. For an age-structured model with immune response, we employ a generalized Jacobian matrix method in conjunction with the Laplace transform for the stability analysis as follows: Note that we can derive a general solution to the second equation in (2.1): T (a,t) B(t a)e a δ+me(t τ)dτ if t a T (a t)e t δ+me(t τ)dτ if t < a, (2.13) where T (a) is the initial value and B(t) kv (t)t(t) is the boundary value of T (a,t) because we assume that ǫ 1 ǫ 2 in this section. 6

7 Substituting (2.13) into the system (2.1), we can rewrite the system (2.1) as dt dv de λ dt(t) kv (t)t(t) t + λ E P(a)B(t a)e a δ+me(t τ)dτ da t P(a)T (a t)e t δ+me(t τ)dτ da cv (t) ( t b E B(t a a)e δ+me(t τ)dτ da + t + t B(t a a)e δ+me(t τ)dτ da + t ( t d E B(t a a)e δ+me(t τ)dτ da + t t B(t a a)e δ+me(t τ)dτ da + t δ E E(t). T (a ) t t)e δ+me(t τ)dτ da T (a t)e t δ+me(t τ)dτ da + K b E(t) T (a ) t t)e δ+me(t τ)dτ da T (a t)e t δ+me(t τ)dτ da + K d E(t) (2.14) Since the functions P(a), T, and E are bounded above, we have t t T (a t)e t δ+me(t τ)dτ da as t, P(a)T (a t)e t δ+me(t τ)dτ da as t. Once the stability of (2.14) is found, it then gives the stability of (2.1). Thus, we now consider dt dv λ dt(t) kv (t)t(t) P(a)B(t a)e a δ+me(t τ)dτ da cv (t) B(t) kv (t)t(t) de B(t a)e a λ E + b E d E δ+me(t τ)dτ da B(t a)e a E(t) δ+me(t τ)dτ da + K b B(t a)e a B(t a)e a δ+me(t τ)dτ da δ+me(t τ)dτ da + K d E(t) δ E E(t). (2.15) 7

8 Let (T e,v e,b e,e e ) be the solutions to λ dt e kv e T e B e N P cv e kv e T e B e (2.16) λ E + b EB e N 1 B e N 1 + K b E e d EB e N 1 B e N 1 + K d E e δ E E e, where the notations N P and N 1 are defined in (2.5) by assuming δ(a) δ and m(a) m. Let T(t) T(t) T e, Ṽ (t) V (t) V e, B(t) B(t) B e, Ẽ(t) E(t) E e. We assume that the perturbations ( T(t), Ṽ (t), B(t), Ẽ(t)) are sufficiently small for t and that they are zero for t <. By linearizing the exponential function (e x 1 + x), we obtain where and + S P B + Similarly, we have where P(a)( B(t a) + B e )e a δ+m(ẽ(t τ)+ee)dτ da P(a)( B(t a) + B e )e a δ+meedτ e a mẽ(t τ)dτ da P(a)( B(t a) + B e )σ(a)(1 P(a) B(t a)σ(a)da + P(a)B e σ(a)da + P(a)σ(a)( S P B + B e N P + F P (Ẽ), a mẽ(t τ)dτ)da P(a) B(t a)σ(a)( P(a)B e σ(a)( a S P B F P (Ẽ) B e a a mẽ(t τ)dτ)da mẽ(t τ)dτ)da m B(t a)ẽ(t τ)dτ)da + B en P + F P (Ẽ) P(a)σ(a) B(t a)da, P(a)σ(a)( a mẽ(t τ)dτ)da. (2.17) ( B(t a) + B e )e a δ(a τ)+m(a τ)(ẽ(t τ)+e e )dτ da S 1 B + B e N 1 + F 1 (Ẽ), (2.18) S 1 B F 1 (Ẽ) B e σ(a) B(t a)da σ(a)( 8 a and mẽ(t τ)dτ)da.

9 For sufficiently small perturbations, that is, B and Ẽ 1, we may assume that S 1 B + F 1 (Ẽ) < B e N 1 + K b 1. By (2.18) and linearizing the function 1 1 x as 1 + x for x < 1, we obtain ( B(t a) + B e )e a δ+meedτ e a mẽ(t τ)dτ da ( B(t a) + B e )e a δ+meedτ e a mẽ(t τ)dτ + K b S 1 B + B e N 1 + F 1 (Ẽ) S 1 B + B e N 1 + F 1 (Ẽ) + K b ( S1 B + B e N 1 + F 1 (Ẽ) )( B e N 1 + K b S 1 B+F 1 (Ẽ) B en 1 +K b ( S1 B + F 1 (Ẽ) + B )( en 1 1 S 1 B + F 1 (Ẽ) ) B e N 1 + K b B e N 1 + K b B e N 1 + K b ) (2.19) S 1 B + F 1 (Ẽ) B e N 1 + K b + B en 1 ( S 1 B + F 1 (Ẽ) B e N 1 + K b B e N 1 + K b ) 2 B en 1 (S 1 B + F 1 (Ẽ)) (B e N 1 + K b ) 2 K b (B e N 1 + K b ) 2(S 1 B + F 1 (Ẽ)) + B en 1 B e N 1 + K b. By (2.17), (2.18) and (2.19), we can derive a system of the new variables ( T(t), Ṽ (t), B(t), Ẽ(t)) as d T (d + kv e ) T(t) kt e Ṽ (t) dṽ S p B cṽ (t) + FP(Ẽ) B(t) kt e Ṽ (t) + kv e T(t) (2.2) dẽ b E K b E e (B e N 1 + K b ) 2(S 1 B + F 1 (Ẽ)) + b EB e N 1 B e N 1 + K b Ẽ(t) d EK d E e (B e N 1 + K d ) 2(S 1 B + F 1 (Ẽ)) d EB e N 1 Ẽ(t) δ E Ẽ(t). B e N 1 + K d We now consider the Laplace transform (L) of (2.2) by denoting transforms with hats. By Fubini s theorem and changing variables, we obtain 9

10 L[F p (Ẽ) { B e P(a)σ(a)( B e m B e m +B e m B e m +B e m P(a)σ(a)( P(a)σ(a) a t t P(a)σ(a) P(a)σ(a)da a mẽ(t τ)dτ)da}e st Ẽ(τ)dτ t a t a Ẽ(τ)dτdae st t P(a)σ(a) α Ẽ(τ)dτdae st Ẽ(τ)dτe st Ẽ(τ)dτe s(α+a) dαda. Ẽ(τ)dτ)dae st Since Ẽ(τ) for τ <, we know Thus, L[F p (Ẽ) B emn P L[ B e mn P L[ B e mn P L[ a t t t P(a)σ(a) α Ẽ(τ)dτ + B e m Ẽ(τ)dτ + B e m Ẽ(τ)dτe s(α+a) dα. P(a)σ(a) α P(a)σ(a)e sa da Ẽ(τ)dτ + B e ml[p(a)σ(a)l[ B e mn P 1 s L[Ẽ + B emŝp(s) 1 s L[Ẽ 1 B e m(n P ŜP(s)) sê(s), t Ẽ(τ)dτe s(α+a) dαda α Ẽ(τ)dτ Ẽ(τ)dτe sα dα where s is the Laplace variable. 1

11 Similarly, we have L[S P B { a P(a)σ(a) B(t a)da}e st P(a)σ(a) B(α)e s(α+a) dαda P(a)σ(a) B(α)e s(α+a) dαda P(a)σ(a)e sa da L[P(a)σ(a)L[ B(α) ˆ S P (s) ˆB(s). B(α)e sα dα In addition, we have L[F 1 (Ẽ) { B e σ(a)( a 1 B e m(n 1 Ŝ1(s)) sê(s) mẽ(t τ)dτ)da}e st and L[S 1 B { σ(a) B(t a)da}e st Ŝ1(s) ˆB(s). The Laplace transform of (2.2) now form the following linear system: s d kv e kt e ˆT s c Ŝ p (s) B e m(n P ŜP(s)) 1 s ˆV kv e kt e 1 ˆB (G b G d )Ŝ1(s) Φ Ê where Φ s (G b G d )B e m(n 1 Ŝ1(s)) 1 s + (H b H d ) δ E, T Ṽ Ẽ, (2.21) and G b b E K b E e (B e N 1 + K b ) 2, G d d E K d E e (B e N 1 + K d ) 2, H b b EB e N 1 B e N 1 + K b, H d d EB e N 1 B e N 1 + K d. 11

12 State Non-infection Infection EQ EQ 1 EQ 2 EQ 3 T e Te () V e E e local stability unstable asymp. stable unstable asymp. stable Table 2: Numerical values and stability results for steady states of the age-structured model with no treatment We can solve the linear system by using Cramer s rule to determine ˆT f T( T, Ṽ, Ẽ, s) det(a(s)) ˆV f V ( T, Ṽ, Ẽ, s) det(a(s)) ˆB f B( T, Ṽ, Ẽ, s) det(a(s)) Ê f E( T, Ṽ, Ẽ, s), det(a(s)) (2.22) for some functions f T, f V, f B and f E. Here A(s) is a matrix that depends on the Laplace variable s in system (2.21). Taking the inverse Laplace transform, we determine the growth rate s of the solutions ( T(t), Ṽ (t), B(t), Ẽ(t)) at the poles of the solutions (2.22). After checking that there are no common factors in f T, f V, f B or f E, we find that these poles are at the zeroes of det(a(s)). Therefore, we conclude that if the zeroes of det(a(s)) all have negative real parts, then the equilibrium (T e, T e, V e, E e ) is locally asymptotically stable. Given the specified parameters in Table 1 and by assuming P max 8, β 5, and d 1 in equation (2.12), we determine that model (2.1) has four physical steady states and several nonphysical steady states of which at least one state variable is negative. Here we omit the nonphysical steady states. Table 2 provides a summary of the numerical values and stability results for these steady states. By the above argument, we can easily check the trivial or uninfected steady state EQ is locally unstable. In addition, we can show that there are one locally unstable equilibrium and two locally asymptotically stable equilibria for an infected patient with no treatment (ǫ 1 ǫ 2 ). In fact, the states of the system converge to an unhealthy steady state EQ 1 in which target cells are substantially depleted and the viral load is extremely high when a small amount of virus is introduced in the uninfected steady state EQ (see Figure 1). In addition, the age-structured model has a healthy stable steady state EQ 3 in which a strong immune response develops for successful control of viral infection. One of the objectives of this paper is to derive optimal strategies for moving the model system to EQ 3, not to EQ 1, as shown in Figure 1. 12

13 A EQ1 B EQ3 EQ Figure 1: EQ indicates the uninfected locally unstable equilibrium; EQ 1, the unhealthy locally asymptotically stable equilibrium with attraction region A; and EQ 3, the healthy locally asymptotically stable equilibrium with attraction region B. The dashed line indicates an uncontrolled trajectory, and the solid line, a controlled trajectory. 3 An Optimal Control Problem In this section, we formulate an optimal control problem based on model (2.1) to derive optimal treatment strategies in which HIV patients reach the healthy steady state EQ 3 (i.e., immune control of viral infection). We first define the objective functional as follows: J(ǫ 1,ǫ 2 ) 1 2 R 1 ε 2 1 (t) + R 2ε 2 2 (t) + QV 2 (t) SE 2 (t) (3.1) where R 1, R 2, Q, and S are the weight constants for controls, viral loads, and immune effector cells, respectively. The first two terms represent the systemic cost of drug treatment. This cost is based on the actual treatment cost as well as the severity of unintended side effects of drugs. Here the objective is to minimize both the systemic cost of drug treatment and the viral load while maximizing the number of immune effector cells to achieve a healthy steady state EQ 3. The two control functions ǫ 1 (t) and ǫ 2 (t) represent the efficacy of RTIs and PIs satisfying a i ǫ i (t) b i < 1, (i 1, 2), respectively. The case ǫ i (t) b i, (i 1, 2) represents maximum efficacy of drugs. We consider the following optimal control problem: Problem 3.1. We seek optimal controls ǫ 1 and ǫ 2 such that J(ǫ 1,ǫ 2 ) min{j(ǫ 1,ǫ 2 ) ǫ 1 (t) and ǫ 2 (t) are Lebesque-integrable on [,T f with values in U 1 [a 1, b 1 and U 2 [a 2, b 2 } subject to a system of the partial and ordinary differential equations (2.1) with the boundary condition T (,t) (1 ǫ 1 (t))kv (t)t(t) and the initial conditions T() T, T (a,) T (a), V () V and E() E. The basic framework of an optimal control problem is to prove the existence of an optimal control and then characterizes the optimal control by using the optimality system. Remark 3.2. We note that the existence of optimal control functions to Problem 3.1 can be shown by using the same arguments as in Theorem 3.2 in [19. 13

14 We derive an optimality system for an optimal solution by using a version of Pontryagin s Maximum Principal [11, 15, 25. For this, we introduce a Lagrange multiplier or adjoint variable Y (ξ 1 (t), ξ 2 (a,t), ξ 3 (t), ξ 4 (t)) and define the Lagrangian L(X,ǫ,Y ) 1 2 R 1 ǫ 2 1(t) + R 2 ǫ 2 2(t) + QV 2 (t) SE 2 (t) [ dt ξ 1 (t) λ + dt(t) + kt(t)v (t)(1 ǫ 1(t)) [ T ξ 2 (a,t) + T t a + δ(a)t (a,t) + m(a)t (a,t)e(t) da ξ 2 (,t)[t (,t) kt(t)v (t)(1 ǫ 1 (t)) [ dv ξ 3 (t) (1 ǫ 2(t)) [ de ξ 4 (t) λ E b E where X (T, T, V, E) and ǫ (ǫ 1, ǫ 2 ). P(a)T (a,t)da + cv (t) T (a,t)da T E(t) + d E (a,t)da + K b T (a,t)da T E(t) + δ E E(t) (a,t)da + K d Theorem 3.3. Given optimal controls and solutions to the corresponding constraint equations (2.1) that minimize the objective functional (3.1), there exists an adjoint variable Y (ξ 1 (t), ξ 2 (a,t), ξ 3 (t), ξ 4 (t)) satisfying dξ 1 ξ 1(t)d + [ξ 1 (t) ξ 2 (,t) kv (t)(1 ǫ 1 (t)), ξ 2 t + ξ 2 a ξ 2(a,t)[δ(a) + m(a)e(t) ξ 3 (t)(1 ǫ 2 (t))p(a) [ b E K b ( T (a,t)da + K b ) 2 d E K d ( T (a,t)da + K d ) 2 dξ 3 QV (t) + ξ 3(t)c + [ξ 1 (t) ξ 2 (,t) kt(t)(1 ǫ 1 (t)) dξ 4 SE(t) + ξ 2 (a,t)m(a)t (a,t)da E(t)ξ 4 (t) [ be T (a,t)da ξ 4 (t) T d E T (a,t)da (a,t)da + K b T δ E (a,t)da + K d (3.2) with the terminal conditions ξ 1 (T f ), ξ 2 (a,t f ), ξ 3 (T f ), and ξ 4 (T f ) and the boundary 14

15 condition lim ξ 2(a,t). In addition, the optimal control functions ǫ a 1 and ǫ 2 are given by ( ǫ 1 (a (t) max 1,min b 1, [ξ 1(t) ξ 2 (,t) kt(t)v (t) )), R 1 ( ǫ 2(t) max a 2,min (b 2, ξ 3(t) P(a)T (a,t)da)). R 2 (3.3) Proof. We apply the necessary conditions for finding the stationary points of L. Setting to zero the first variations of the Lagrangian with respect to Y yields the constraints (2.1). Setting to zero the first variations with respect to X yields the adjoint equations (3.2): The first adjoint equation can be obtained by taking L T. L T lim 1 [ L(T + µ µ µ T) L(T). ξ 1 T [ξ 1 d + ξ 1 kv (1 ǫ 1 ) ξ 2 (,t)kv (1 ǫ 1 ) T [ ξ 1 ξ 1 d ξ 1 kv (1 ǫ 1 ) + ξ 2 (,t)kv (1 ǫ 1 ) T ξ1 (T f ) T(T f ) + ξ 1 () T() Since variations T in the state are arbitrary, we get dξ 1 ξ 1(t)d + (ξ 1 (t) ξ 2 (,t))kv (t)(1 ǫ 1 (t)) with the terminal condition ξ 1 (T f ). Using similar arguments, the first-order necessary condition L T yields the equation for ξ 2 in the adjoint system (3.2). L T 1 [ lim L(T + µ T µ µ ) L(T ) ( ) T ξ 2 + T da t a ξ 2 (,t) T (,t) + ξ 3 (1 ǫ 2 ) b E + lim ξ 4 µ µ d E lim ξ 4 µ µ ( (T + µ T )da (T + µ T )da + K b ( (T + µ T )da (T + µ T )da + K d 15 [ξ 2 δ(a) + ξ 2 m(a)e T da P(a) T da ) T da T E da + K b T da T da + K d ) E

16 . [ ξ2 t + ξ 2 a ξ 2δ(a) ξ 2 m(a)e + ξ 3 (1 ǫ 2 )P(a) T da + K b ) 2E ξ d E K d 4 ( T da + K d ) 2E T da +ξ 4 b E K b ( ξ 2 (a,t f ) T (a,t f )da + lim ξ 2(a,t) T (a,t) + a Since variations T in the state are arbitrary, we have ξ 2 t + ξ 2 a ξ 2 (a,) T (a,)da (ξ 2 (,t) ξ 2 (,t)) T (,t) ξ 2 (a,t)[δ(a) + m(a)e(t) ξ 3 (t)(1 ǫ 2 (t))p(a) [ b E K b ( T (a,t)da + K b ) 2 d E K d ( T (a,t)da + K d ) 2 E(t)ξ 4 (t), the terminal condition ξ 2 (a,t f ), and the boundary condition lim a ξ 2(a,t). The adjoint equations for ξ 3 and ξ 4 in (3.2) may also be derived from L V 1 lim [L(V + µṽ µ µ ) L(V ) Q lim µ 2µ (V + µṽ )2 V 2 [ξ 1 kt(1 ǫ 1 ) ξ 2 (,t)kt(1 ǫ 1 ) + ξ 3 c Ṽ ξ 3 Ṽ [ QV ξ1 kt(1 ǫ 1 ) + ξ t 2kT(1 ǫ 1 ) + ξ 3 ξ 3 c Ṽ ξ 3 (T f )Ṽ (T f) + ξ 3 ()Ṽ () and L E lim 1 [L(E + µ µ µẽ) L(E) lim µ S 2µ ξ 4 Ẽ + (E + µẽ)2 E 2 ξ 2 m(a)t Ẽda b E T [ξ da d 4 E T da T ξ 4 da + K b T ξ 4 δ E da + K d 16 Ẽ

17 [ SE ξ 4 (T f )Ẽ(T f) + ξ 4 ()Ẽ() ξ 2 m(a)t da + ξ 4 b E T da d E + ξ 4 T da T ξ 4 da + K b T ξ 4 δ E Ẽ da + K d for any arbitrary variations Ṽ and Ẽ in the state. Finally, to derive the optimality conditions, we must consider the controls that adhere to the bounds. Because of the bounds on the controls, the derivatives of the objective functional may not be zero at the optimal control. However, the idea is essentially the same as that in the case without bounds, and we can derive explicit expressions for the optimal control functions (3.3) by taking the approach in [2, 3, 19. To get the explicit formulation of the optimal controls ǫ 1 and ǫ 2, we explore the necessary optimality conditions L ǫ 1 1 lim µ µ [L(ǫ 1 + µ ǫ 1 ) L(ǫ 1 ) R 1 lim µ 2µ (ǫ 1 + µ ǫ 1 ) 2 ǫ [ξ 1 ktv ξ 2 (,t)ktv ǫ 1 [R 1 ǫ + ξ 1 ktv ξ 2 (,t)ktv ǫ 1 and L ǫ 2 1 lim µ µ [L(ǫ 2 + µ ǫ 2 ) L(ǫ 2 ) R 2 lim µ 2µ (ǫ 2 + µ ǫ 2 ) 2 ǫ 2 2 [ξ 3 P(a)T da ǫ 2 P(a)T da ǫ 2 [R 2 ǫ 2 ξ 3 for any arbitrary variations ǫ 1 and ǫ 2 in the control. Using arguments similar to those for bounded controls in [2, 3, 19, we can express the optimal control ǫ 1 and ǫ 2 as ( ( ǫ 1(t) max a 1,min b 1, [ξ 1(t) ξ 2 (,t) kt(t)v (t) )) R 1 ( ǫ 2(t) max a 2,min (b 2, ξ 3(t) P(a)T (a,t)da)). R 2 17

18 To summarize, we may determine an optimal solution to problem (3.1) by solving the optimality system (2.1), (3.2) and (3.3). Note that this system is coupled and that state system (2.1) is forward in time with the initial condition, whereas the adjoint system (3.2) is backward in time with the terminal condition. One way to uncouple the system to solve the optimization problem is to use a gradient-based method. 4 Numerical Simulations We consider a simple gradient algorithm: Choose a starting guess of controls. Solve the state system (2.1) forward in time with the initial conditions. Solve the adjoint system (3.2) backward in time with the terminal conditions. Update the controls in each iteration by using the optimality conditions (3.3). Continue the iterations until convergence is achieved For more information on the gradient method, we refer the interested reader to [12. In each iteration, we solve the state and adjoint systems by using the difference schemes based on the Runge-Kutta method suggested in [1. We employ the parameter values in Table 1 for the numerical simulations and simulate early infection by introducing some virions per µl of blood plasma and very low levels of infected T cells in the uninfected unstable equilibrium EQ. That is, the initial and boundary conditions for the state are given by T() T te 1, T (,) kv ()T() 1.6e 5, V () 2.e 5, E() E te.1, and T (a,) kt te V ()σ(a). We assume that drug efficacy is bounded by a 1, a 2, b 1.7, and b 2.3. Because the magnitudes of the viral load, immune effector cells, and drug treatment functions in the objective functional (3.1) are based on different scales, we balance them by choosing the weight constants R 1 R 2 1, Q 1 3, and S 1 3. In addition, we choose a max 1, a step size of.5 for the age/time variables, and P max 8, β 5 and d 1 in the age-dependent virus production function (2.12). The optimal treatment strategies for 5 days in Figure 2 were found by solving the optimality system. Noteworthy is that the shapes of the two control functions are nearly identical and that the optimal treatment strategies have characteristics similar to those of structured treatment interruption strategies. As shown in Figure 3, we observe the corresponding system behavior by solving (2.1) under this optimal treatment regimen up to the 5th day and no treatment from the 5th day to the 8th day. In addition, Figure 3 compares the optimal solutions with full-treatment solutions (i.e., ǫ 1.7 and ǫ 2.3) and no treatment solutions (i.e., ǫ 1 ǫ 2 ) for 8 days. For the full-treatment solutions, we discontinue both drugs on the 5th day, as in the optimal solutions. The most intriguing observation from Figure 3 is that the viral load remains at low levels and that the population of uninfected T cells recovers from the virus after the 5th day under the optimal treatment strategies, even though both drugs are discontinued on the 5th day. This is because of high concentrations of immune effector cells. We also notice that both drugs taper off around the 1th day, the 1th day, the 17th day, the 21th day, and so on. Consequently, viral 18

19 ε ε Figure 2: Optimal control pair with R 1 R 2 1, Q 1 3, and S 1 3. The ǫ 1 represents reverse transcriptase inhibitors and ǫ 2 represents protease inhibitors loads and infected T cell counts are relatively high around those days. In addition, high viral loads stimulate immune effector cells, boosting the immune response. These numerical results illustrate that it is possible to move a patient from a state of early infection to a healthy stable steady state (EQ 3 ) in which a strong immune response successfully controls the viral load and consequently restores uninfected target cells without requiring drugs. This suggests one possible scenario in which optimal treatment strategies facilitate long-term control of HIV after the discontinuation of the therapy. 5 Conclusions This paper formulates and analyzes an age-structured model of HIV infection with various compartments (including target cells, infected cells, viral loads, and immune effector cells) that is subject to multiple drug treatments as control functions. The proposed model possesses an uninfected equilibrium and several infected equilibria. We conduct a detailed stability analysis of the steady states by using a generalized Jacobian matrix method in conjunction with the Laplace transform. We then apply techniques and ideas from optimal control theory to determine optimal therapy protocols. We derive an optimality system from which optimal solutions can be determined. We define finite difference approximations based on the Runge-Kutta method and implement them in conjunction with a gradient method to solve the optimality system. We design optimal treatment 19

20 1 3 Uninfected T cells 1 5 Infected T cells Virus Immune effectors Figure 3: Optimal solutions ( ); solutions with no drug treatment ( ) (i.e., ǫ 1 ǫ 2 ); and solutions with full treatment ( ) (i.e., ǫ 1.7 and ǫ 2.3) based on the discontinuation of both drugs on the 5th day in all cases (graphs of infected T cells show the function T (,t)) strategies to move the state of the system to a healthy steady state with low viral loads and high levels of immune effector cells. In addition, the results of the numerical simulations indicate that the optimal therapy protocols can facilitate long-term control of HIV by inducing a strong immune response after the discontinuation of the therapy. Acknowledgements The work of Hee-Dae Kwon was supported in part by the National Research Foundation of Korea(NRF)Grant funded by the Korean government (MEST) ( ) and in part by Inha University Research Grant. The work of Jeehyun Lee was supported by the Korea Research Foundation Grant funded by the Korean Government (KRF C12). References [1 L. M. Abia and J. C. Lopez-Marcos, Runge-Kutta methods for age-structured population models, Appl. Numer. Math., 17 (1995) pp

21 [2 B. M. Adams, H. T. Banks, M. Davidian, H. D. Kwon, H. T. Tran, S. N. Wynne, and E. S. Rosenberg, HIV dynamics: Modeling, data analysis, and optimal treatment protocols, J. Comput. Appl. Math., 184 (25) pp [3 B. M. Adams, H. T. Banks, H. D. Kwon, and H. T. Tran, Dynamic multidrug therapies for HIV: Optimal and STI control approaches, Math. Biosci. Engrg., 1 (24) pp [4 J. Alvarez-Ramirez, M. Meraz, and J. X. Velasco-Hernandez, Feedback control of the chemotherapy of HIV, Int. J. Bifur. Chaos, 1 (2) pp [5 H. T. Banks, Hee-Dae Kwon, J. A. Toivanen, and H. T. Tran, A state-dependent Riccati equation-based estimator approach for HIV feedback control, Optim. Contr. Appl. Meth., 27, (26) pp [6 H. T. Banks, Taesoo Jang, and Hee-Dae Kwon, Feedback Control of HIV Antiviral Therapy with Long Measurement Time Int. J. Pure Appl. Math., 66, (211) pp [7 M. E. Bran and G. Chen, Feedback control of a biodynamical model of HIV-1, IEEE Trans. on Biom. Engrg., 48 (21) pp [8 D. S. Callaway and A. S. Perelson, HIV-1 infection and low steady state viral loads, Bull, Math. Biol., 64 (22) pp [9 R.V. Culshaw and S. Ruan, A delay-differential equation model of HIV infection of CD4+ T-cells, Math. Biosci., 165 (2) pp [1 K. R. Fister, S. Lenhart, and J. S. McNally, Optimizing chemotherapy in an HIV model, Electronic J. of Differential Equation, 32 (1998) pp [11 W.H. Fleming and R.W. Rishel, Deterministic and stochastic optimal control, Springer-Verlag, New York, [12 M.D. Gunzburger, Perspectives in flow control and optimization, SIAM, Philadelphia, 23. [13 A.V. Herz, S. Bonhoeffer, R.M. Anderson, R.M. May, and M.A. Nowak, Viral dynamics in vivo: Limitations on estimates of intracellular delay and virus decay, Proc. Natl. Acad. Sci., 93 (1996) pp [14 T. Jang, H.-D. Kwon, and J. Lee, Free Terminal Time Optimal Control Problem of N HIV Model Based on a Conjugate Gradient Method, Bull. Math. Biol., 73 (211) pp [15 M.I. Kamien and N.L. Schwartz, Dynamic optimization, North-Holland, Amsterdam, [16 D. E. Kirschner and G. F. Webb, A model for treatment strategy in the chemotherapy of AIDS, Bull. Math. Biol., 58 (1996) pp [17 D. Kirschner, S. Lenhart, and S. Serbin, Optimal control of the chemotherapy of HIV, J. Math. Biol., 35 (1997) pp

22 [18 D. E. Kirschner and G. F. Webb, Understanding drug resistance for montherapy treatment of HIV infection, Bull. Math. Biol., 59 (1997) pp [19 H.D. Kwon, J. Lee, and S.D. Yang, Optimal control of an age-structured model of HIV infection, submitted to Appl. Math. Comput. [2 A. R. McLean and S. D. W. Frost, Zidovudine and HIV: Mathematical models of within-host population dynamics, Reviews in Medical Virology, 5 (1995) pp [21 H. Moore and W. Gu, A mathematical model for treatment-resistant mutations of HIV, Math. Biosci. Engrg., 2 (25) pp [22 P.W. Nelson, M.A. Gilchrist, D. Coombs, J.M. Hyman and A.S. Perelson, An age-structured model of HIV infection that allows for variations in the production rate of viral particles and the death rate of productively infected cells, Math. Biosci. Engrg., 1 (24) pp [23 M. A. Nowak, S. Bonhoeffer, G. M. Shaw, and R. M. May, Anti-viral drug treatment: Dynamics of resistance in free virus and infected cell populations, J. Theor. Biol, 184 (1997) pp [24 A. S. Perelson and P. W. Nelson, Mathematical anslysis of HIV-1 dynamics in vivo, SIAM Rev., 41 (1999) pp [25 L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchenko, The Mathematical Theory of Optimal Processes, Gordon and Breach, [26 D.D. Richman, D. Havlir, J. Corbeil, et al., Nevirapine resistance mutations of human immunodeficiency virus type 1 selected during therapy, J. Virol., 68 (1994), pp [27 L. Rong, Z. Feng, and A. S. Perelson, Mathematical anslysis of age-structured HIV-1 dynamics with combination antiretroviral therapy, SIAM J. Appl. Math, 67 (27) pp [28 H. Shim, S. J. Han, C. C. Chung, S. Nam, and J. H. Seo, Optimal scheduling of drug treatment for HIV infection: Continuous dose control and receding horixon control, Int. J. control Autom. Systems, 1 (23) pp [29 T. Shiri, W. Garira, and S. D. Musekwa, A two-strain HIV-1 mathematical model to assess the effects of chemotherapy on disease parameters, Math. Biosci. Engrg., 2 (25) pp [3 M. Stafford, L. Corey, Y. Cao, E Daar, D. Ho, and A. Perelson, Modeling plasma virus condentration during primary infection, J. Theor. Biol., 23 (2) pp [31 H.R. Thieme and C. Castillo-Chavez, How may the infection-age-dependent infectivity affect the dynamics of HIV/AIDS?, SIAM J. Appl. Math., 53 (1993) pp [32 G. Webb, Theory of Nonlinear Age-Dependent Population Dynamics, Marcel Dekker, New York,