MULTIOBJECTIVE DESIGN OF VACCINATION CAMPAIGNS WITH A STOCHASTIC VALIDATION

MULTIOBJECTIVE DESIGN OF VACCINATION CAMPAIGNS WITH A STOCHASTIC VALIDATION André Rodrigues da Cruz Universidade Federal de Minas Gerais Master Student in Electrical Engineering andrercruz@cpdee.ufmg.br Rodrigo Tomás Nogueira Cardoso Centro Federal de Educação Tecnológica de Minas Gerais Department of Physics and Mathematics rodrigoc@des.cefetmg.br Ricardo Hiroshi Caldeira Takahashi Universidade Federal de Minas Gerais Department of Mathematics taka@mat.ufmg.br ABSTRACT The planning of vaccination campaigns must minimize two factors: the number of infected individuals in a time horizon and the cost to implement the control. This problem is stated here as a non-linear dynamic programming optimization with impulsive control. The traditional SIR (Susceptible-Infected-Recovered) differential equation model is employed for representing the system, and the dynamic programming problem is solved in open-loop, leading to a static non-linear multiobjective optimization problem. The NSGA-II, which is a robust multiobjective genetic algorithm, is employed as the optimization machinery. A stochastic dynamic model of the epidemics is employed in order to validate the vaccination strategy, helping in the choice of the specific strategy to be implemented. The final result shows a set of interval of confidence for each optimal policy strategy. KEYWORDS. Mathematical Programming. Multicriteria Decision Analysis. Applications to Health. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1448

1 Introduction The need to understand and model the dynamics of disease proliferation has lead to an emerging new area of science, the mathematical epidemiology. This area studies quantitatively the distribution of health/disease phenomena, besides assessing the effectiveness of interventions in public health [Clancy 1999]. The epidemiological model SIR, proposed by Kermack and McKendrick in 1927, is one of the most used to represent infectious diseases. This model classifies the individuals in three states: susceptible (S), infected (I) and recovered (R), and these states are related by a system of differential equations [Anderson and May 1992, Hethcote 2000]. In the last decades, simulations methods called Individual Based Models (IBM) have been used to make more realistic assumptions than differential equation models, trying to understand how the properties of the system emerge from the interaction among each individual [Grimm 1999, Nepomuceno 2005]. A big challenge in public health is related to the promotion of vaccination campaigns [Anderson and May 1992], which aims to eradicate a disease with the minimum possible cost. For this purpose, it is necessary to: 1. determine the lowest number of campaigns; 2. set the minimum percentage of the susceptible population which must be vaccinated in each stage; and 3. determine when the campaigns (stages) must be implemented during the time horizon. Many works have applied control theory to present optimal vaccination strategies [Hethcote 2000, Moghadas 2004]. They take the pulse vaccination, which is an impulsive control defined by repeated application of a fixed vaccination ratio in discrete times with equal intervals [Zhou and Liu 2003, d Onofrio 2005]. In order to answer these questions, we propose a multiobjective optimization approach, using a non-linear dynamic programming technique with impulsive control which means that the actions of control are performed in some discrete time instants. A distinctive feature of the proposed approach is that it employs non-fixed pulse vaccination strategies, allowing different vaccination levels in arbitrary time instants. A former study, employing fixed vaccination levels, is related in [Cardoso and Takahashi 2007]. The bi-objective problem consists of minimizing both the integral of infected individuals during the time and the cost with vaccines and number of campaigns (as each campaign also involves spending on advertising, allocation of professionals, transportation, etc.). The NSGA-II [Deb et al. 2000], a very efficient multiobjective genetic algorithm, has been adjusted to find the solutions under the SIR model. The solutions in Pareto set are validated in an Individual Based Model (IBM) to study the stochastic behavior during the time. A multiobjective analysis is presented to emphasize the final solution. This paper is organized in the following order: section 2 shows the SIR model and the Individual Based Model (IBM), which describe the interaction of susceptible, infected and recovered population in a deterministic and stochastic way, respectively. Section 3 shows the optimization model. Section 4 shows the Pareto set and the simulation analysis of the optimal individuals. Section 5 presents the conclusions of this work. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1449

2 Epidemiological Models The main reasons for studying mathematical models of disease spread is the hope that improved understanding of the transmission mechanism may lead to a more effective control strategies. [Clancy 1999]. This section describes two models used in this work, the SIR and the IBM. Both models are compartmental, i.e., individuals are classified in one of the three classes [Hethcote 2000] [Anderson and May 1992]: 1. Susceptible - individuals which are not infected, but may become; 2. Infected - individuals which are sick and can transmit the disease to others; 3. Recovered - individuals which are immune to the contagion (they had been sick before, or they have been vaccinated). 2.1 SIR The SIR model uses the strategy of compartments, related by a system of three differential equations. The initial value problem are presented in Equation (1). ds dt = βis µn µs N, S(0) = S o 0 di dt = βis N γi µi, I(0) = I o 0 dr dt = γi µr, R(0) = R o 0 (1) in which S, I and R are respectively the number of susceptibles, infected and recovered; N is the number of individuals (which is supposed to be fixed: S(t) + I(t) + R(t) = N, t 0). The parameters are: β is the transmission rate; γ is the recovery rate of infected individuals; and µ is the rate of new susceptibles. To keep the number of individuals in population constant, the mortality rate and the birth rate are made equal. As N is constant along the time, the variables can be written as ratios: s(t) = S(t)/N; i(t) = I(t)/N; r(t) = R(t)/N; and r(t) = 1 s(t) i(t). This epidemiological model examines the spread of the disease in a population during a time-stage, in an average way. It is mathematically and epidemiologically well-conditioned [Hethcote 2000]. The basic reproductive number R 0 is defined as R 0 = β/(µ + γ), which represents the average number of infections produced by an infected individual. The article [Hethcote 2000] shows that the SIR system has an asymptotically stable endemic equilibrium if, and only if, R 0 > 1. The parameters values used in this work are shown in Table 1. They describe a very difficult disease to control. The number of individuals considered is N = 1000. The initial condition, in percentage, is A = (s o, i o, r o ) = (0.99; 0.01; 0.00). The time horizon adopted is T CNT RL = 50(u.t.). Figure 1 shows the behavior of the system without intervention for this configuration. In this case, the stable point is, in percentage, f = (0.0622; 0.1005; 0.8373) and for N individuals, F = (62.1988; 100.6095; 837.1918). Notice that the amount of infected individuals in the stable point (the endemic equilibrium) is very significant and, for this reason, it is necessary to control the disease. This paper employs the SIR model to find optimal vaccination campaigns in epidemic control situations, via impulsive multiobjective dynamic optimization. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1450

Parameter Value µ 1/50 β 3.0 γ 1/6 Table 1: Parameters of the SIR model used in this work. 1000 Behavior of the system during the time Number of individuals 900 800 700 600 500 400 300 200 100 Susceptible Infected Recovered 0 0 10 20 30 40 50 Time (u.t.) Figure 1: Behavior of the SIR model without intervention, considering the parameters shown in Table 1. 2.2 IBM The IBM (Individual Based Model) is a probabilistic simulation model in which individuals are discrete entities endowed with some characteristics of interest. The model simulates the interactions between such individuals [Grimm 1999]. IBM are applied in many areas such as ecology, network, and economy to study the development of animals, softwares, markets during the time. This paper uses IBM models to validate the results coming from the multiobjective optimization procedure (using the SIR model), in order to aid in the decision-making procedure that must be performed for choosing a single solution from the Pareto optimal set that is delivered from the multiobjective optimization procedure. Like a Monte Carlo simulation procedure [Ross 2002], it intends to simulate the proliferation of a disease and to test the effectiveness of a campaign, an action that the SIR model is not able to do. More specifically, IBM is used here to present an interval of confidence of each policy found. The IBM, in this work, follows some premises in order to coincide with those for SIR. The premises are: 1. Constant population: the population size is fixed; 2. Categories of an individual: An individual can be susceptible, infected or recovered; 3. Change of category: Once in a category, the individual can change to other in each instant of time, according to the rules: XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1451

(a) susceptible, infected, recovered susceptible: an individual has died and a new susceptible has just born; (b) susceptible infected: a susceptible has contracted the disease in contact with an infected; (c) infected recovered: a infected individual has become a recovered one; (d) susceptible recovered: a susceptible has been vaccinated. 4. Statistic distribution: The mortality (and birth, consequently), recover and vaccination are events that occur with uniform distribution in each time interval. 5. In IBM, each individual has the same probability of contact. All parameters must be chosen in order to make the IBM model to fit, in average, the SIR model behavior. In this way, the probability of death/birth is given by µ, the disease transmission probability for an infected individual is β, and the recovery probability is γ, as stated in Table 2. The population size is fixed at N. Parameter Value µ 0.0214 β 2.7000 γ 0.1633 Table 2: Probability values for IBM The high level code which describes IBM is shown in Algorithm 1. Algorithm 1 IBM( µ, β, γ, N, T f ) P InitialP op() for t = T 0 until T f do for i = 1 until N do AnaliseDeath(P.ind(i)) AnaliseV accine(p.ind(i)) AnaliseInf ection(p.ind(i)) AnaliseRecovery(P.ind(i)) end for end for 3 Optimization Model This paper proposes a multiobjective impulsive control using an open-loop continuousvariable dynamic optimization procedure to solve the epidemiological problem, using the SIR model as the dynamic system, coupled with a statistical analysis using the IBM. The main idea of impulsive control [Yang 1999] is to split the continuous-time interval in some stages, performing impulsively actions of control just in some time instants. The dynamic system keeps its autonomous dynamics in the time intervals between the consecutive impulsive actions of control. The solution of the dynamic programming impulsive control problems is sought here in open-loop using an evolutionary algorithm, which allows rather arbitrary objective XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1452

function and constraints, and is an alternative to using enumerative algorithms, which have prohibitive computational complexity [Bertsekas 1995]. The time horizon [0, T CNT RL] has to be partitioned in a set of time instants not previously fixed Γ = {τ 0,..., τ M }, such that: τ k < τ k+1, τ 0 = 0 and τ M = T CNT RL. These intervals do not have to be equidistant. The time instant τ + k is defined as a time instant just before the impulsive action in τ k. The optimization variables of the problem are: the number of vaccination campaigns (stages): M, the time intervals vector between the campaigns: Γ = {τ 0,..., τ M }, the percentages of vaccines of the susceptible population during the campaigns: P = {p[1],..., p[m]}, for each time instant in Γ. The following discrete-time variable notation is considered: x[k] = x(τ k ) and x[k + ] = x(τ + k ), for each state x = s, i or r. Since the vaccination acts only in the ratio of susceptible, then: s[k + ] = s[k](1 p[k]); i[k + ] = i[k]; r[k + ] = r[k]; k = 0, 1,..., M 1. (2) Each initial value problem (IVP) in (1) is valid during the time in which there is no action of control (vaccination) in the system the system presents its autonomous dynamics within such time intervals. A new initial condition for the system is established in each stage according to the difference equation (2), linking each stage with the next one. The set of initial conditions in t = 0 must be previously known, if the explicit simulation of the system is to be performed. The constraints are modeled as: each interval of time between stages τ k = τ k+1 τ k has to obey the relation: 0 < tmin τ k tmax < T CNT RL; the sum of τ k is not allowed to overcome the time of control T CNT RL; this determines the number of campaigns M; each vaccination ratio p[k] has to follow the rule: 0 < pmin p[k] pmax 1.0; it is desired that the number of infected individuals, after t = T INIC, remains up to i tol > 0, in other words, I(t) i tol, for t (T INIC, T CNT RL). Of course, in the case of some diseases, the acceptable level of infected individuals should be i tol equal to zero infected individuals, corresponding to disease eradication. The bi-objective optimization model has two functions which must be minimized: F 1 - the integral of infected population during the optimization horizon; F 2 - the total cost with the campaign (sum of vaccines added with a fanciful fixed cost of each campaign, for example, transportation, advertisement, etc.). XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1453

Therefore, the bi-objective optimization model can be formulated as in Problem (3). min M,Γ,P F 1 = T CNT RL 0 I(t)dt F 2 = c 1 M + c 2 M k=0 S(t) p[k] (3) subject to: ds dt = µn µs βis N, S(0) = S o 0; di dt = βis N γi µi, I(0) = I o 0; dr dt = γi µr, R(0) = R o 0; t (τ + k, τ k+1); s(τ + k ) = s[k+ ] = s[k](1 p[k]); i(τ + k ) = i[k+ ] = i[k]; r(τ + k ) = r[k+ ] = r[k]; k = 0, 1,..., M 1; 0 < tmin τ k tmax < T CNT RL; 0 < pmin p[k] pmax 1.0; M k=1 τ k T CNT RL; I(t) i tol, t (T INIC, T CNT RL); M N. The possible solutions for a multiobjective problem are inside the Pareto optimal or non-dominated solutions set. In a minimization problem with vector of objective function J R m, if y R n denotes the vector of decision variable of the problem and D R n the set of feasible solutions y, the Pareto optimal set Y D is characterized by: Y = {ȳ D; y D : J i (y) J i (ȳ), i = 1,..., m; J(y) J(ȳ)}. (4) The image of the Pareto optimal set Y by the objective function J, or J(Y ), is the Pareto front. The parameters considered here are: T CN T RL = 50 (u.t.), tmin = 1.0 u.t., tmax = 5.0 u.t., pmin = 0.01, pmax = 0.80, T INIC = 10 u.t., i tol = 5, c 1 = 1000, c 2 = 1. The other constant values are in Table 1. 3.1 Optimization Engine A customized version of the NSGA-II, with parameters shown in Table 3, has been developed to solve the Problem (3). The system of differential equations has been simulated numerically using the fourth-order Runge-Kutta method [Burden and Faires 2003] with precision of 10 5. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1454

Parameter Value Number of generations 1000 Number of individuals in NSGA-II 40 Combination rate 0.80 Mutation rate 0.05 Table 3: Parameters for NSGA-II The numerical integration has been made adding the total of infected individuals coming from the Runge-Kutta method. In order to avoid problems at evaluating the number of infected individuals during the control, the values have been normalized by the total points returned by the numeric algorithm. 4 Optimization Results This section is divided into two subsections. The first one shows the results found through the execution of NSGA-II under the SIR model. The following subsection takes the previous result and validates it in an IBM as a Monte Carlo simulation. A new Pareto front is then constructed through stochastic dominance (using the median value of objective functions) [Levy 1992], showing an interval of confidence of each Pareto optimal solution. In this way, it is possible to observe the deviation of the simulation under the best controls, in a multiobjective view. 4.1 Results Under SIR Model The execution of the NSGA-II has resulted in the Pareto front shown in Figure 2, in which the axes represent the integral of infected individuals during the control, F 1, normalized by the total points returned by the Runge-Kutta method, and the cost of the control during the time horizon, F 2. Each solution is represent by an asterisk and the utopian solution is represented by a diamond. All 40 individuals candidates to optimum are dispersed in the Pareto front, which shows the efficiency of the NSGA-II. One solution picked from the left side of the Pareto front is shown in Figure 3. It shows the behavior of the controlled system and total number of vaccinated individuals. Notice that the vaccination pulses employed in the vaccination campaigns are allowed to vary from one campaign to the next one, and also that the intervals between campaigns are different too. 4.2 Validation Under IBM In this subsection, a validation through Monte Carlo simulations of the result found in the last subsection is shown. Each Pareto optimal solution, in Figure 2, has been simulated in the IBM 21 times and, for each, the two objective functions in (3) were revaluated. The solutions were sorted by their F 1 values. Then, the stochastic non-domination sorting was applied using the median. The new Stochastic Pareto front with an interval of confidence is shown in Figure 4. The axes are the same as presented in Figure 2. The interval of confidence considers the median in its center and the quantiles 25% and 75% as its extremes, for each objective. Notice that only five solutions have remained non-dominated after this procedure. This occurred because the stochastic dominated solutions have high sensitivity in the XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1455

3.4 x 104 Pareto optimal set 3.2 3 F2 2.8 2.6 2.4 2.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 5 F 1 Figure 2: Pareto set for the problem 3. F 1 represents the integral of infected population during the control and F 2 represents the total cost of the vaccination control. neighborhood of the optimum. The final stochastic solution presents not only a Pareto optimal value, but also an interval of confidence. Such information can not be delivered by the SIR model only. In Figure 5, the stochastic behavior of the system of one solution is shown. The center curves represents the curve corresponding to the median behavior, and the error is represented by the 25% and 75% quantiles of obtained by simulation as described in this subsection. Note that although the SIR model points to a non-null number of infective even with vaccinations (see Figure 3), IBM simulation points that there may be a considerable chance of eradication of the disease (see Figure 5). 5 Conclusion This work has presented a multiobjective design methodology for vaccination policies which are intended to minimize both the number of infected individuals in a population and the cost of campaigns of vaccination to implement it during a time horizon. The Pareto optimal set of non-dominated policies has been found using a multiobjective genetic algorithm, the NSGA-II, executing an open-loop dynamic optimization over a SIR model of the epidemics. An individual-based model (IBM) has been used in order to validate the results obtained via the deterministic approach. A very interesting outcome of this study is: when submitted to this stochastic analysis, most of the deterministically found nondominated policies become dominated. This is due to a possibly high sensitivity in the optimal solutions, which loss the optimality for perturbed conditions. The main conclusions of this work, therefore, become: A stochastic analysis of optimal vaccination policies is needed in order to allow discarding highly sensitive solutions that would loss optimality. A reasonable perturbation to be employed in such analysis is the one coming from the IBM XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1456

1000 Behavior of the system during the time 900 800 Number of individuals 700 600 500 400 300 200 Susceptible Infected Recovered 100 0 0 10 20 30 40 50 Time (u.t.) (a) 800 Number of vaccinated susceptibles in the stages 700 Number of susceptible vaccinated 600 500 400 300 200 100 0 0 10 20 30 40 50 Time (u.t.) (b) Figure 3: Example of a solution in Figure 2. The first graphic (a) shows the behavior of the system during the time, the second one (b) shows the number of susceptibles vaccinated in each stage. system, since this model produces perturbations that are likely to be similar to the ones effectively present in the population; A multiobjective optimization approach for determining the optimal vaccination policies is interesting, since it provides a variety of different solutions that can be submitted to the stochastic analysis. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1457

3 x 104 Pareto optimal set 2.9 2.8 2.7 F2 2.6 2.5 2.4 2.3 8 10 12 14 16 18 20 F 1 Figure 4: Stochastic Pareto front representing the non-dominated solutions showed in Figure 2 in a stochastic view. The solutions were validated under IBM by Monte Carlo method. Each interval of confidence considers the median in the center and quantiles 25% and 75% by the cross (for each objective). 1000 Stochastic simulation of the IBM system 900 800 700 Number of individuals 600 500 400 Susceptible Infected Recorvered 300 200 100 0 0 5 10 15 20 25 30 35 40 45 50 Time (u.t.) Figure 5: Stochastic behavior of one solution picked from the Pareto front shown in Figure 4. Note the interval of confidence has important details to preview the behavior of the epidemic control. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1458

References [Anderson and May 1992] Anderson, R. M. and May, R. M. (1992). Infectious Diseases of Humans: Dynamics and Control. Oxford University Press. [Bertsekas 1995] Bertsekas, D. P. (1995). Dynamic Programming and Optimal Control. Athena Scientific. [Burden and Faires 2003] Burden, R. L. and Faires, J. D. (2003). Numerical Analysis. Thomson. [Cardoso and Takahashi 2007] Cardoso, R. T. N. and Takahashi, R. H. C. (2007). Solving impulsive control problems by discrete-time dynamic optimization methods. Proceedings of the XXX Congresso Nacional de Matemática Aplicada e Computacional. [Clancy 1999] Clancy, D. (1999). Optimal intervention for epidemic models with general infection and removal rate functions. Journal of Mathematical Biology. [Deb et al. 2000] Deb, K., Agrawal, S., Pratab, A., and Meyarivan, T. (2000). A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Proceedings of the VI Conference in Parallel Problem Solving from Nature, pages 849 858. [d Onofrio 2005] d Onofrio, A. (2005). On pulse vaccination strategy in the SIR epidemic model with vertical transmission. Applied Matehamtics Letters, 18(7):729 732. [Grimm 1999] Grimm, V. (1999). Ten years of individual-based modelling in ecology: what have we learned and what could we learn in the future? Ecological Modelling, pages 129 148. [Hethcote 2000] Hethcote, H. W. (2000). The mathematics of infectious diseases. SIAM Review, pages 599 653. [Levy 1992] Levy, H. (1992). Stochastic dominance and expected utility: Survey and analysis. Management Science, pages 555 593. [Moghadas 2004] Moghadas, S. M. (2004). Analysis of an epidemic model with bistable equilibria using the poincaré index. Applied Mathematics and Computation. [Nepomuceno 2005] Nepomuceno, E. G. (2005). Dinâmica, Modelagem e Controle de Epidemias. PhD thesis, Universidade Federal de Minas Gerais (UFMG). [Ross 2002] Ross, S. M. (2002). Simulation. Academic Press, San Diego, CA. [Yang 1999] Yang, T. (1999). Impulsive control. IEEE Transactions on Automatic Control, 44(5):1081 1083. [Zhou and Liu 2003] Zhou, Y. and Liu, H. (2003). Stability of periodic solutions for an SIS model with pulse vaccination. Mathematical and Computer Modelling, 38(13):299 308. XLI SBPO 2009 - Pesquisa Operacional na Gestão do Conhecimento Pág. 1459