d m = distance from te elicopter base to mission m demand location F cycle,i = set of missions assigned to veicle in cycle i H = total number of rescu

Transcription

1 PSO-Greedy Searc Algoritm for Helicopter Mission Assignment in Disaster Relief Operations Adriana Andreeva-Mori 1, Keiji Kobayasi 2 and Masato Sindo 3 Japan Aerospace Exploration Agency, Mitaka, Tokyo, Japan In te immediate aftermat of a large-scale disaster, optimal elicopter rescue mission assignment is critical to saving many lives. Te current practice in te field is mostly umancentered, owever. Japan Aerospace Exploration Agency as been developing a decisionsupport system for aircraft operation in order to promptly plan and execute rescue missions. Te current researc focuses on evacuation missions in particular and investigates te potential of particle swarm optimization (PSO) wit integrated greedy searc in aircraft resource management. We propose a robust particle model wic can reflect various elicopter properties as well as evacuation mission caracteristics. PSO parameters are modified and set based on numerical simulations and te values determined in tis way are used in an optimization of disaster relief mission assignments based on real data obtained during te Great East Japan Eartquake and Tsunami in We initialize te swarm wit a greedy searc algoritm to increase te calculation speed and improve te quality of te results. It is sown tat a ybrid PSO/ greedy searc algoritm can be successfully adapted to disaster relief support systems and provide valuable analysis and decision-making information to te autorities in carge. Nomenclature a m = mission assignment coefficient, 1 if mission m is assigned to a elicopter in te fleet; 0 oterwise a m = mission assignment variable, 1 if mission m is assigned to elicopter ; 0 oterwise c 1,2,3 = acceleration coefficient c = capacity of elicopter 1 Researcer, Operation Systems and Aviation Safety Tecnology Researc Group, andreevamori.adriana@jaxa.jp 2 Section Leader, Operation Systems and Aviation Safety Tecnology Researc Group, kkoba@cofu.jaxa.jp 3 Senior Researcer, Operation Systems and Aviation Safety Tecnology Researc Group, sindo.masato@jaxa.jp Tis document is provided by JAXA.

2 d m = distance from te elicopter base to mission m demand location F cycle,i = set of missions assigned to veicle in cycle i H = total number of rescue elicopters, wit set K = {1,, H} M = total number of evacuation missions, wit set I = {1,, M} n cycle = number of cycles for elicopter, = {1,, H} n refuel = total number of refuels for elicopter R m = number of people to be transported (evacuees) in mission m p bi = global best found at current iteration p g = global previous best found by te swarm so far p i = individual previous best found so far r 1,2,3 = random functions wit values uniformly distributed in [0,1] σ(v i ) = sigmoid function t cyclei, = total time of missions in cycle i t lim = ard deadline to bring evacuees to te base (complete te rescue missions selected) t load = time necessary for one evacuee to board veicle t m = time required by elicopter to complete rescue mission mt tl = total time necessary for take-off and landing of elicopter t range = maximum fligt time available between two refuels, elicopter specific t refuel = time necessary for a elicopter to refuel, elicopter specific V cruise = Cruising speed of elicopter V i = particle velocity at iteration i V max = maximum particle velocity x i = particle position at iteration i x ij = mission assignment model particle position, j t mission assigned to i t elicopter ω = inertia weigt Tis document is provided by JAXA.

3 D I. Introduction isaster relief operations include a large number of ground and air veicles, personnel and autorities. In te aftermat of large-scale disasters, especially, timely information saring and prompt planning and execution of rescue missions is critical for saving many lives. In tis view, Japan Aerospace Exploration Agency (JAXA) as been developing an integrated aircraft operation system for disaster relief (D-NET 2), wic overview is sown in Figure 1. D-NET 2 will combine satellites, manned and unmanned aircraft in direct searc and rescue, an unprecedented endeavor not just in Japan, but also in te world. A key component of te system is going to be te optimal resource management block, sown in red in Figure 1, wic will assign and scedule operations reconnaissance and rescue operations. Figure 1. An overview of JAXA s integrated aircraft operation system for disaster relief. In te preliminary design of te system, two blocks of te optimal management subsystem ave been defined information gatering (i.e. reconnaissance) optimization and rescue mission optimization. Details on reconnaissance optimization and satellite imagery application can be found in [1]. Te general searc and rescue flow is sown in Figure 2. It is assumed tat rigt after te disaster strikes no information about te damage and rescue needs is available. Terefore, reconnaissance is necessary in te wole stricken area. Here, suppose tis disaster area is divided into cells for easy understanding. Say information gatering is performed by elicopters wic fly continuously over te disaster area witout landing at te site. At tis initial stage, cells wit rescue needs are detected. At cell i, tis i needs location appened at time t loc. At tis point, te fact tat tere are people wo need rescue is known, but teir status and exact number is unknown. To obtain all tis information, a elicopter as to land at te site, timing defined i as t def. Tis completes te information gatering stage. Next is te rescue stage, sown in blue in Figure 2. Here, i only te evacuation process is considered. Assume tat t res is te time wen rescue was completed at cell i. It is often said, owever, tat tere is some critical time witin wic rescue sould be completed before te number of survivors Tis document is provided by JAXA.

4 drops drastically. Let us define some time t lim as a ard rescue deadline. Wen disaster relief resources available are sufficient to provide rescue to all people witin t lim, mission assignment optimization for all missions is enoug. A common practice ten is to assign resources in suc a way as to minimize te time necessary to bring all survivors to safe places. For large-scale disasters, owever, constraints on te resources become active and no feasible solution over all missions witin t lim is available. Disaster strikes Reconnaissance rescue needed? YES t loc Needs located Helicopter lands at te site Tis researc scope: rescue missions t Helicopters def rescue Needs evacuees defined t res Rescue completed NO No action needed Figure 2. Searc and rescue flow model implemented in JAXA s integrated aircraft operation system for disaster relief. Te model as been generated based on interviews wit professionals wit disaster relief operation experience. Te purpose of tis researc is to develop and apply an optimization algoritm for elicopter mission assignment wic provides optimal or sub-optimal results in near real-time environment. Here, we explore te potential of particle swarm optimization (PSO) algoritm combined wit a greedy searc algoritm as part of te optimization resource management subsystem preliminary design. Te core of te algoritm proposed consists of PSO, wit a greedy searc being implemented in order to reduce te searc space by introducing reasonably good initial swarm parameters. Surprisingly, despite te obvious need for disaster relief management systems, researc and development in tis area is relatively scarce. A good overview of available tecnologies and design requirement to suc a management system as been presented in NATO tecnical report publised in 2008 [2]. In te report, owever, no specific planning tool as been cosen as te most promising candidate for suc a system. Researcers at universities and academic institutions, on te oter and, ave analyzed particular elicopter planning strategies and tested tem on various scenarios. Veicle routing, in particular, as been a point of interest and, as suc, several optimization tecniques ave been proposed. Dynamic veicle routing approac as been reported as efficient [3], but te instances tested are limited to 10 veicles and a undred nodes. Problems of similar sizes are seen in many oter researc papers [4], [5], [6]. Te main issue tere lies in te representation of aircraft performance constraints, suc as elicopter types Tis document is provided by JAXA.

5 [7], transport capacity [8], fligt range and refueling/maintenance times. In te case of a disaster, various veicles gater in te stricken area, so te model sould be able to easily reflect properties of a eterogeneous fleet. In te recent years Ozdamar et al. proposed a novel ierarcical clustering and routing for disaster relief [9] and provided a metod to efficiently solve large scale problems wit a solution deviating from te optimal one by less tan 12% witin a reasonable calculation time. Te more complicated te algoritm is, te more parameters need to be set for sufficient results. Optimal ambulance assignment as also been investigated [10], but in tis case te number of ambulances is scalable to te number of patient notes, wereas in te case of a large-scale disaster te rescue missions outnumber greatly te available veicles. In tis researc we propose elicopter mission assignment based on particle swarm optimization wic requires relatively few parameter settings and allows for practically any veicle performance constraints and a large number of missions and veicles. To furter improve te performance of te algoritm, we initialize te positions of te particle in te swarm based on a greedy searc executed prior to te PSO itself. To te best of our knowledge, tere as been no reported researc on elicopter mission planning wit PSO application. Tis paper is organized as follows. Section II describes te problem and presents a matematical model. It also gives and overview of particle swarm optimization and te particle representation, fitness function and parameter settings proposed for our problem. Te results obtained ere are used for te numerical test runs of a scenario wic is based on data obtained during real disaster relief operations in te immediate aftermat of te Great East Japan Eartquake and Tsunami in Here, te greedy algoritm is also presented and merged into te PSO. Te test scenario, aircraft assumptions and test results are sown in Section III. Section IV summarizes tis work and outlines areas for future researc. II. Problem Setting In tis section we first describe te optimization problem and its properties and provide its matematical formulization. We ten consider particle swarm optimization wic is te core of te algoritm for elicopter mission assignment, define te necessary parameters and investigate convergence and solution quality. After tat, we introduce a greedy algoritm and discuss te advantages of its implementation in te problem. A. Problem Description Tis document is provided by JAXA.

6 Consider H rescue elicopters available and a total number of M evacuation missions wic need to be completed. Generally speaking, tis is an NP-ard problem. As JAXA s integrated aircraft operation system for disaster relief is meant mainly for large-scale disasters, we assume tat te total number of missions exceeds te number of elicopters available (H > M). Eac rescue mission m is caracterized by its distance from a elicopter base (d m ) and te number of evacuees waiting to be transported (R m ) to a safe location. Not all missions can be processed by all elicopters, i.e. tere are missions wic require special equipment suc as oist or medical equipment. Te time needed to complete a mission is elicopter specific. Tis assumption accounts for properties suc as elicopter cruising speed, take-off and landing times, time necessary to board an evacuee, etc. Based on interviews wit people involved in direct searc and rescue, we impose a one stop per trip constraint. Teoretically speaking, a large elicopter can collect evacuees from several spots before returning to its base and tus make rescue more efficient. In practice, owever, out of consideration for te safety of evacuees wo are already onboard, several stops per trip are almost never allowed. Releasing tis constraint will possibly improve te efficiency of relief operations, but at present we focus on developing a practical decision support system, so we opt for one stop per trip constraint. Besides, ere we assume tat eac rescue mission is assigned to no more tan one veicle. Tere is only one elicopter base available, i.e. a single depot problem. Eac elicopter as a certain capacity wic cannot be exceeded. Te fligt range for eac veicle is also defined in terms of fligt time. Te time for a single refueling is also fixed. As noted by Wex et al. [11], suc a problem is similar to te multiple Traveling Salesman Problem wit salesman-specific travel times or te parallel-macine sceduling problem wit unrelated macines non-batc sequence-dependent setup times. Apart from te work of Wex et al, according to Weng et al. [12] only one researc paper on te above problem as been publised, and it does not provide specific definition of te operational constraints. A major difference between all of te above work and tis researc is te ard constraint we impose on operation time. Tis constraint is analogous to te 72 ours limit for survival often stated as critical in immediate post-disaster management. Terefore, our system tries to propose a scedule wic transports as many survivors as possible to a safe place (te elicopter base in tis case) under a survival-time-based operational constraint, i.e. we do not necessarily complete all rescue missions and some nodes remain unvisited. B. Matematical Formulization Below are te notation and te matematical model of te problem described above. A cycle indicates te group of missions between two refuels. Tis document is provided by JAXA.

7 M H R m t lim Total number of rescue missions, wit set I = {1,, M} Total number of rescue elicopters, wit set K = {1,, H} Number of people to be transported (evacuees) in mission m Hard deadline to bring evacuees to te base (complete te rescue missions selected) t refuel Time necessary for a elicopter to refuel, elicopter specific t range Maximum fligt time available between two refuels, elicopter specific t cycle,i Total time of missions in cycle i a m t m Mission assignment variable, 1 if mission m is assigned to elicopter ; 0 oterwise Time required by elicopter to complete rescue mission m c Capacity of elicopter (maximum number of evacuees wic can be transported by ) n refuel Total number of refuels for elicopter n cycle Number of cycles for elicopter, = {1,, H}. One denotes a block between two refuels. F cycle,i Set of missions assigned to veicle in cycle i H M (Fit) Maximize a m R m Subject to: =1 m=1 n cycle (Con1) t cycle,i i=1 + t refuel n refuel t lim, = 1,, H a m {0,1}, m = 1, M; = 1,, H (Con2) H a m i i=1 1, m = 1,, M (Con3) a m R m c, m = 1, M; = 1,, H a m t m = m F cycle,i t cycle,i, i = 1,, n cycle ; = 1,, H (Con4) (Con5) t cycle,i t range, i = 1,, n cycle ; = 1,, H (Con6) Tis document is provided by JAXA.

8 n cycle = n refuel 1, = 1,, H (Con7) Te fitness function (Fit) maximizes te number of people transported from eac incident location to te base. Constraint (Con1) ensures tat te total time of eac elicopter does not exceed te ard deadline t lim. Constraint (Con2) indicates tat eac mission can be assigned to one elicopter (a m = 1) or not to be assigned at all (a m = 0). Constraint (Con3) ensures tat eac rescue mission is assigned to no more tan one elicopter. Constraint (Con4) imposes te elicopter capacity constraint and ensures tat te number of evacuees transported by eac veicle does not exceed its capacity. Constraint (Con5) defines te fligt time between two refuels as te sum of completion times of te missions assigned for te cycle. Constraint (Con6) makes sure tat te fuel constraint for eac elicopter is met. Constraint (Con6) sows tat tere is exactly one refuel between eac cycle. We assume tat all elicopters are fullyprepared for operation at te start of te rescue, so (Con7) is feasible. C. Particle Swarm Optimization Overview Te particle swarm optimization is an evolutionary computation paradigm originally proposed by Kennedy and Eberart in 1995 [13], [14]. Te main idea beind te algoritm is tat saring simple information can result in preforming complicated optimization. Te searc process adopted in te original algoritm is analogues to te way bird flocks searc for food. Te basic flow is sown in Figure 3. Agents, called particles ere, move in te yper searc space evaluating a fitness function wit respect to teir position. Eac particle as a memory, too, and remembers its best position (individual previous best, p i) found so far. Besides, particles sare te information among temselves so tat eac particle is aware of te best position (global previous best, p g) found in te swarm so far. Eac particle s movement from one position to anoter is decided by its velocity, wic is in turn determined based on its previous velocity and te best positions found by te particle itself and te swarm as a wole. As particles move in te searc space, tey evaluate teir position on eac iteration and wen necessary, update p i and p g. Te searc continues until a stopping criterion is met, wic can be eiter a sufficiently good fitness or a maximum number of iterations. Tis document is provided by JAXA.

9 Generate initial population Evaluate current fitness Current fitness is better tan pi NO Find te best performer in te population YES Replace pi wit current position Fitness of current best performer is better tan pg YES Replace pg wit current best performer position NO Update eac particle s position and velocity Stopping criterion is met NO YES Exit Optimal solution is fitness at pg Figure 3. Basic PSO flow. An important parameter for eac particle is its velocity, as it determines te direction of te searc. At iteration i, te velocity V i and position x i of te particle is updated as sown below. V i = V i + c 1 r 1 (p i x i ) + c 2 r 2 (p g x i ) (1) x i = x i + V i (2) Here, r 1 and r 2 are random functions wit values uniformly distributed in [0,1] randomly generated at eac iteration and for eac particle; c 1 and c 2 are te parameters wic determine ow strong te particle will be pulled towards its best position and towards te swarm s best position, respectively. Tese two parameters are often called acceleration coefficients [15] of te cognitive and social part. V i is kept witin a certain range V max, +V max depending on te problem searc space. Tis document is provided by JAXA.

10 As seen from te description above, PSO as very few parameters wic need to be set, wic is te main reason wy tis metod is adopted in te current researc. Unlike most sceduling problems, te scale of disaster relief operations, mission distribution and resource availability are difficult to predict. Terefore, te optimization algoritm needs to be robust and wit few specific settings. Even toug te original version of te PSO was developed for continuous problems, soon a discrete version was proposed [16]. Te main differences between continuous and discrete PSO consist in te particle representation and parameter tuning. Since te mission assignment problem investigated in tis researc is based on te discrete PSO, its basic caracteristics and alterations implemented ere are discussed in detail below. D. Particle Representation Most researcers dealing wit PSO applications to discrete problems agree tat a key to successfully solving te problem is to find an appropriate representation of its solution by a PSO particle [17], [18]. In te standard version of te discrete binary PSO proposed by Kennedy and Eberard [16], a particle is a sequence of 0 and 1, i.e. a bit string. Tis mapping is still widely used ( [19], [20]) as it allows a relatively straigtforward representation of te particle s position and velocity. A similar approac is adopted by Si et al. [21], wo consider an ordered integer sequence to represent te order of cities wic define te route in a traveling salesman problem, and by Salman et al., wo use define a particle as an M-dimensional vector to represent an M-task scedule [18]. In bot of te above models, te particle is again seen as a vector. Cen et al. [22] use similar particle representation and propose a ybrid discrete particle swarm optimization algoritm for capacitated veicle routing problem. Constraints tere pose a problem, toug, and te obtained solution as to be cecked for feasibility. Te main limitation of tese approaces is te difficulty to model assignments of multiple tasks (jobs) to te same task-performer. Liao et al. [17] solve tis issue by designing a job-to-position representation for a particle. Tey consider a two-dimensional array wit binary elements (0 or 1). Te first dimension indicates te position of te job and te second one te job itself. Wen te element (i,j) is 1, te i t job is assigned to te j t position. Oterwise (i,j) is zero. In Liao et al. s model, eac job is assigned a position in te sequence. In tis researc, a similar modeling is adopted. For illustration purposes, consider 2 rescue elicopters and 6 rescue missions. Wen no constraints on te mission completion time are present or active, eac mission is assigned to exactly one elicopter, so te sum of all elements in any row is exactly 1. Te particle representing a sample mission assignment [ ] [ ] is sown in Table 1. Terefore, te particle s position is defined as X = [x ij ], fori = 1: number of elicopters, j = 1: number of missions, were x ij = 1 if and Tis document is provided by JAXA.

11 only if j t mission is assigned to te i t elicopter and x ij = 0 oterwise. In te sample assignment sown in Table 1, x 11 = x 22 = x 31 = x 42 = x 52 = x 61 = 1 and te remaining elementsx 12 = x 21 = x 32 = x 41 = x 51 = x 62 = 0. For reasons related to particle velocity definition and calculation, it is important tat eac mission is assigned to exactly one veicle, i.e. no tasks are left uncompleted. In tis researc, owever, an important operational constraint is te time available for rescue missions t lim. Tis constraint leaves some tasks uncompleted, i.e. some evacuees are left beind. One way to reflect tis property is to introduce anoter variable wic sows weter a certain mission was included in final assignment. Tis will lead to zero rows in te original matrix. We ave cosen anoter approac, owever. We keep te form of te matrix sown in Table 1 and introduce a dummy veicle. Wen a mission is assigned to te dummy veicle, it will not be included in te rescue plan, i.e. all missions assigned to te dummy veicle are te leftover from te assignment to te real fleet. By including a dummy veicle in te array, te basic properties remain uncanged- te particle position representation is X = [x ij ], for i = 1: number of elicopters, j = 1: number of missions, were x ij = 1 if and only if j t mission is assigned to te i t elicopter and x ij = 0 oterwise, and te sum of te elements in eac row remains 1. In te example in Table 2, missions 1, 3 and 5 are assigned to elicopter 1, mission 6- to elicopter 2 and missions 2 and 4 are left out of te plan. Table 1.Helicopter-mission assignment model wen all missions are to be completed. Helicopter 1 Helicopter 2 Mission Mission Mission Mission Mission Mission E. Particle Velocity Table 2. Helicopter-mission assignment wit a dummy veicle. Helicopter 1 Helicopter 2 Dummy Mission Mission Mission Mission Mission Mission Tis document is provided by JAXA.

12 Te particle s velocity sould measure te probability for a certain mission to be assigned to a certain elicopter. Te original equation (1) for continuous PSO results in velocity values suc as te ones sown in Table 3. For discrete PSO, it is useful to convert tese values by te sigmoid function from equation (3) as to ave tem in te range of [0,1], as suggested by Kennedy et al. [16], [14] and adopted in te sceduling problem optimization by Liao et al. [17]. σ(v i ) = exp( V i ) (3) Table 3. Original particle velocity as determined by Equation (3). Helicopter 1 Helicopter 2 Dummy Mission Mission Mission Mission Mission Mission Keeping in mind tat by introducing te dummy veicle, te sum of all probabilities in eac row sould equal 1, te particle velocity trail becomes as te one sown in Table 4. Higer values of V i indicate iger probability of te particle coosing te value 1. For example, according to Table 4, at tis point it is most likely tat mission 3 will be assigned to elicopter 1, as te probability tere is Table 4. Particle velocity considering sigmoid conversion and 1 mission-1 veicle constraint. F. Fitness Function and Constraints Helicopter 1 Helicopter 2 Dummy Mission Mission Mission Mission Mission Mission An array representation of mission/elicopter as anoter main advantage, namely tat it allows for a very straigtforward definition of costs and rewards associated wit eac particular assignment. In te elicopter mission assignment problem, mission cost is equivalent to mission time. It includes te travel time from te base to te incident location and back, landing and take-off time, as well as time required to load/unload evacuees. Eac mission time depends on te elicopter to wic it is assigned. Some missions require special equipment suc as oist, for example, Tis document is provided by JAXA.

13 not available on all aircraft. To reflect tis constraint, we add penalty to te mission time of elicopters not capable of performing a certain mission. Te order of te penalty exceeds feasible mission times, so in practice we assign infinite mission time to tese aircraft. Tis adjustment is done to keep te convergence of te numerical simulation. Furtermore, at tis stage of te researc it is assumed tat a rescue mission cannot be assigned to more tan one veicle, i.e. if te number of evacuees exceeds te veicle s capacity, it will ave to make multiple rounds in order to complete te mission, wic is reflected in te mission time. On te oter and, te reward associated wit te completion of eac mission is te number of evacuees wo ave been transported from incident sites to te elicopter base. In tis researc, for any assignment P, te fitness function is defined as te sum of all mission rewards wic are assigned to a non-dummy veicle. Te goal is to maximize te fitness (p), i.e. te total rewards or evacuees saved. fitness(p) = Reward i i (4) for all missions i assigned to a non-dummy veicle. Tis is equivalent to M (5) fitness(p) = a m R m m=1 were a m 1, if mission i is assigned to a non dummy elicopter = { 0, if mission i is assigned to te dummy elicopter R m : Reward of completing mission i M : Total number of evacuation missions Let us consider te sample mission assignment sown in Table 2. Assume tat te cost/reward table is as sown in Table 5. Here, we assume tat elicopter 1 cannot perform mission 6, so its mission time is set to According to Table 2, missions 1, 3 and 5 are assigned to elicopter 1, mission 6- to elicopter 2 and missions 2 and 4 are assigned to te dummy veicle. Tis optimization is done under te time constraint t lim = 30. Te total mission time of elicopter 1 is 25, of elicopter 2-25, and te reward contribution of eac veicle is 31 and 50 respectively, adding up to a total reward of 81. Assigning mission 3 to elicopter 2 would ave let to te same total reward, but bot solutions are equally well-acceptable in terms of te fitness function unless furter constraints are induced. Tis document is provided by JAXA.

14 Table 5. Sample mission time/ reward table. For a two-veicle scenario, te first two columns indicate elicopter-specific mission times, te tird column sows dummy veicle mission times (in practice, any value would do), and te fort column indicates te number of people waiting to be rescued in eac mission. Helicopter 1 (small) Helicopter 2 (large) Dummy (random) Reward (evacuees) Mission Mission Mission Mission Mission Mission G. Particle Swarm Optimization Parameter Setting As mentioned earlier, one of te great advantages of PSO is te small number of parameters wic need to be adjusted. In order to determine optimal parameter settings, we use a random mission time/reward table for 5 aircraft (excluding te dummy one) and 100 missions. Te initial solution was given not completely random. We used a random subset of all missions and generated a feasible solution based on mission times for eac elicopter. A main property of te problem is tat even toug we do not know te optimal solution, we know multiple feasible solutions by assigning any missions wic meet te constraints to our fleet as long as te operational time is less tan t lim. Furter improvement in te initial solution is done by a greedy searc algoritm wic is going to be discussed in Section H. General Parameters Te first parameter tat needs to be set is te size of te population. Considering computational cost and based on our numerical experiments, we ave decided on te value of 30 particles. Furter increase does not necessarily improve te performance and increases computational time. Te second parameter is V max. Even toug early researc sowed tan V max is necessary to provide convergence, our experiments sow tat in tis particular formulation of discrete PSO small values of V max cop off unnecessarily te searc space wereas sufficiently large values do not influence te performance significantly. Terefore, we opt for leaving out V max altogeter. Te tird parameter is te so-called inertia weigt ω, proposed by Si and Eberart [23]. Using inertia weigt, te velocity originally defined in equation (1) is modified as: V i = ωv i + c 1 r 1 (p i x i ) + c 2 r 2 (p g x i ) (6) Tis document is provided by JAXA.

15 Tere ave been reports tat wit proper inertia weigt, te influence of V max can be reduced and te convergence of te algoritm as a wole can be improved [23], [24], [25]. We ave conducted various tests wit bot constant and variable inertia weigt, but we could see no obvious improvement in te solution and convergence, so for simplicity no inertia weigt is used ere (i.e., ω = 1). Neigbor Topology Parameters As for te neigbor topology, an interesting observation is made. As noted also by Liao et al. [17], wereas in te original discrete PSO te social part is obtained based on all particles istory so far, better results are obtained wen only te current iteration best position is used. Liao et al, owever, do not analyze tis penomenon or go into any detail about te possible reason. In order to investigate te most advantageous parameter setting, we introduce anoter part in te velocity trail previously sown in equation (1). Here, just like r 1 and r 2, r 3 is a random function defined in [0,1], and c 3 is te acceleration coefficient determining ow strong te particle will be pulled towards te best position p bi found at tis iteration. V i = V i + c 1 r 1 (p i x i ) + c 2 r 2 (p g x i ) + c 3 r 3 (p bi x i ) (7) In particular, te values of c 1, c 2 and c 3 are of great interest. To investigate teir optimal setting, we perform a series of numerical tests varying teir values between 0 and 5, but always using te same initial particle population, in order to eliminate te influence of te initialization parameters. [All Particles, All Iterations] Best Position Acceleration Coefficient c 2 As reported by some researcers already [17], standard acceleration coefficient values are around 2. Here, we first consider coefficients equal to 2, as to investigate te influence of eac acceleration coefficient on te convergence and fitness value obtained. All computational tests were run 10 times for 2000 iterations wit a swarm size of 30 particles. From te results sown in Figure 4 it is seen tat te best fitness value is obtained wen te best position in te current iteration only is used instead of using all particles istory (c 1 = 2, c 2 = 0, c 3 = 2). Te convergence improves sligtly wen c 1 = 2, c 2 = 2, c 3 = 2, but as seen from Figure 4, for all cases wen c 2 = 2, te fitness value is worse tan te simulations wit c 2 = 0. A possible reason for tis is convergence to a local minimum instead of global one. Tis document is provided by JAXA.

16 Figure 4. Influence of iteration best, global best and individual best. Keeping sufficient variety in te particle population is a key to avoid premature convergence. Some researcers ave proposed metods wic measure te Hamming distances among te particles at eac iteration and based on teir values create modulations wen needed. Tese tecniques require more computational time, toug, so for te purposes of our system sould be avoided wenever possible. Examining te diversity of te swarm under te current problem formulation is not easy, owever. Here, we use te Hamming distance between te missions assigned to te dummy veicle to evaluate te swarm diversity. Te Hamming distance is te number of positions wit different values in te binary string. For example, in our case, wen all 100 missions are assigned to te dummy veicle, te dummy veicle mission representation is dummy 1 = [ ] Wen no missions are assigned to te dummy veicle, dummy 2 = [ ] and te Hamming distance between dummy 1 and dummy 2 is 100. Figure 5 sows te Hamming distances for eac iteration between eac particle s te dummy-assigned missions and te best particle s dummy-assigned missions. Te diversity of te swarm is expressed by te maximum difference in te swarm s dummy Hamming distances at eac iteration, wic is easily visualized by te widt of te colored line in Figure 5. Tis document is provided by JAXA.

17 Figure 5. Hamming distances at eac iteration. Large Hamming distance indicates swarm diversity, small Hamming distance sows convergence. In all cases, te swarm tends to converge, tus reducing te average Hamming distance. In te cases of c 2 = 2, toug, te widt of te line decreases muc sooner tan in te case of c 2 = 0, i.e. te diversity of te swarm is significantly reduced at an earlier stage of te optimization. Terefore, it is safe to say tat c 2 0 limits te exploration abilities of te swarm prematurely. Based on te above analysis, in te rest of te researc we adopt a zero value for c 2, using non-zero values for c 3 instead. [Single Particle, All Iterations] Best Position Acceleration Coefficient c 1 To explore te dependence of te fitness and convergence on c 1 and c 3 values, two series of computational tests are performed- fixed value for c 3 (c 3 = 2) wit varying c 1 (Figure 6), and fixed value for c 1 (c 1 = 2) wit varying c 3 (Figure 7). Te results from te first test series do not sow a clear dependence of eiter te fitness value or te convergence properties of PSO. Te rigt grap in Figure 6 sows te minimum, maximum and average fitness values for 10 tests for eac value of c 1. As mentioned earlier, iger fitness values means more evacuation missions accomplised witin te time limit. Te disaster relief mission assignment problem as to be solved in real time, so multiple runs are not a feasible option. In oter words, te algoritm as to be robust. Te results sow, owever, tat Tis document is provided by JAXA.

18 no clear best value of c 1 is available, wic, put in anoter way, is equivalent to coosing any value of c 1 witin reasonable range wic suits te design needs. Here, we set c 1 = 2. Figure 6. Dependence of te fitness and convergence properties on c1. [All Particles, Current Iteration Only] Best Position Acceleration Coefficient c 3 In te second test series, te effects of varying c 3 value are investigated. Here, unlike in te previous test series, a relatively straigtforward dependence of te best fitness on te acceleration parameter is seen, as sown in Figure 7. Generally speaking, PSO s convergence improves for iger values of c 3, but tis worsens te average best fitness value. PSO s performance for c 3 = 0 is significantly worse tan tat for any value, so it is concluded tat non-zero values for c 1 and c 3 are appropriate. From tis perspective, a smaller value is c 3 is cosen for te rest of te tests, i.e. c 3 = 1. Tis document is provided by JAXA.

19 Figure 7. Dependence of te fitness and convergence properties on c3. H. Greedy Searc Algoritm Implementation Te first step of te PSO algoritm requires initial solution generation. In many problems, no feasible solution is known at te start of te optimization. In tis case, owever, not just a feasible, but a sub-optimal solution can be generated based on a greedy algoritm. In order to efficiently use all information available for te disaster relief missions and at te same time minimize te amount of calculation, a new parameter called profitability is introduced. Profitability is defined for eac elicopter-mission pair and is determined as te mission time/reward rate in eac case. Using te values from Table 5, te profitability values become as sown in Table 6. Very small values suc as te profitability of mission 6 wen assigned to elicopter 1 indicate tat tis veicle is incapable of performing te mission, eiter due to capacity of equipment constraints. Te profitability defined ere can easily by enanced or substituted by incident s severity, as proposed by [11]. A weigted sum of te profitability and severity of rescue missions migt furter bring te simulation closer to practical rescue operations. We believe tat suc an alteration will not cange dramatically te performance of our algoritm, so tis paper focuses on profitability only. We consider severity constraint implementation as a subject of future work. Tis document is provided by JAXA.

20 Table 6. Mission time /reward table wit profitability. Generally speaking, ig profitability indicates ig number of evacuees in a relatively close location, wereas low profitability sows few evacuees far from te base. Helicopter 1 (small) Helicopter 2 (large) Dummy (random) Reward (evacuees) Profitability (elicopter 1) Profitability (elicopter 2) Mission Mission Mission Mission Mission Mission An overview of te algoritm is briefly sown below. Figure 8. Overview on te greedy algoritm used to generate te initial population for te PSO mission assignment problem. A greedy solution determined like tis migt not be optimal. Here, te greedy-searc-based assignment is only used to initialize te positions of eac particle in te PSO, i.e. to narrow-down te searc space for better computational cost and improved PSO performance. As discussed in Section E, owever, maintaining sufficient differences in te initial amming distances is critical to avoid premature convergence to local optimum. To ensure te diversity of te swarm, te greedy-searc is performed not over te wole mission set, but on a randomly cosen subset of te wole. Once te particles initial positions are determined by te above greedy algoritms, te PSO described earlier is applied for finalizing te mission assignment optimization. Tis document is provided by JAXA.

21 III. Test Scenario A. Disaster Area and Evacuation Mission Distribution Te optimization metod described above was tested for a disaster relief scenario based on data from te Great East Japan Eartquake and Tsunami. One of te main issues related wit disaster relief mission planning is te insufficient amount of data and/or its unavailability. Because of privacy issues, information on te exact location and number of evacuees, for example is rarely released. Terefore, building a complete database from sporadic information is extremely difficult. Local autorities migt ave data about te missions performed during disaster relief, but suc information is only released in summary reports suc as [26]. To deal wit tis issue, JAXA as been developing scenarios about rescue mission location and number of evacuees based on general data released, interviews wit local rescue autorities, as well as some probabilistic assumptions. JAXA conducted researc on te activities of elicopters of local fire departments, medical assistance teams, self-defense forces, police departments and coast guard. Fligt reports were obtained and used to build up te rescue mission data base wic includes location and number of evacuees at eac site. Were possible, interviews wit pilots and rescue personnel involved in te relief missions were conducted to verify our assumptions. As a result, te scenario considered ere as information on te location of te elicopter base (a single elicopter base in tis case), eac rescue mission, and number of evacuees wic need to be transported to a safe place (ere, te elicopter base). Te modeled disaster area is Iwate Prefecture, wit Hanamaki Airport being te elicopter base were all aircraft take-off and land. Altogeter, 160 missions are considered. Eac one involves 1 to 100 evacuees. It sould be noted ere tat 100 evacuees cannot be transported at once by any of te available elicopters, so te biggest missions ave been split into blocks of 25. Te distribution by location is sown in Figure 11 and a istogram of te number of evacuees at eac incident spot is sown in Figure 10. Te missions wit evacuees describe transportation of people from facilities suc as ospitals or scools. As for te mission time distribution, 180 elicopter-mission allocations ave been penalized wit mission time 1000 to sow tat te veicles in question are incapable of conducting certain missions. A detailed istogram of te remaining mission times is sown in Figure 9. Tis document is provided by JAXA.

22 Figure 9. Histogram of te mission times potentially assigned to eac of te five elicopters available in te fleet, excluding te penalized missions. Figure 10. Histogram of te evacuees included in rescue missions in Iwate Prefecture Figure 11. Mission distribution in Iwate Prefecture Te total number of evacuees wo need to be transported to te base is 2153, i.e. if no time limit constraint is imposed, te best fitness will be B. Aircraft Assumptions We consider tree elicopter types: small, medium and large wit max fligt time between two refuels 3, 2 45 min and 2 30min, respectively. Maximum fligt time can be set to reflect te aircraft performance and oter extra requirements imposed by te operators. Eac aircraft as a cruising speed of 100 kt and needs 30 min to refuel. Small Tis document is provided by JAXA.

23 aircraft can transport up to 5 people, medium- up to 14 and large- up to 25 at a time. We assume a basic rescue fleet of five veicles- 1 large, 3 medium and 1 small. We assume tat eac organization wic is immediately involved in direct searc and rescue can contribute 1 or 2 elicopters to te fleet of tis prefecture. Once te mission locations, number of evacuees and aircraft properties are defined, te cost/reward table needed for te optimization can be determined. For eac possible mission m being assigned to elicopter, te cost index is determined as follows: t m = 2 d m + t V tl + R m t load cruise (8) were t m : Time required by elicopter to complete mission m(time [ours]) d m : Distance from te elicopter base to mission m demand location V cruise : Cruising speed of elicopter t tl : Total time necessary for take-off and landing of elicopter R m : Number of evacuees in mission m wo need to be transported t load : Time necessary for one evacuee to board veicle Under te above assumptions, te sortest time necessary for a rescue fleet of 1 small, 3 medium and 1 large veicles to complete all missions was estimated at min. To do so, we alter te fitness function of our optimization problem. Instead of te fitness function presented in Section II (B) and later modified to te one sown in Section II (E), we minimize te sum of completion times of all rescue missions. Te matematical representation is sown below. Te notation is te same as te one defined in Section II (B). Here, constraint (Con2 ) guarantees tat all missions are going to be assigned to some veicle. n cycle (Fit ) Minimize max ( t cycle,i i=1 + t refuel n refuel, = 1,, H ) Subject to: a m {0,1}, m = 1, M; = 1,, H (Con1`) H a m i i=1 = 1, m = 1,, M (Con2`) Tis document is provided by JAXA.

24 a m R m c, m = 1, M; = 1,, H a m t m = m F cycle,i, i = 1,, n cycle ; = 1,, H t cycle,i (Con3 ) (Con4 ) t cycle,i n cycle t range, i = 1,, n cycle ; = 1,, H (Con5`) = n refuel 1, = 1,, H (Con6`) It sould be noted tat min is continuous operational time of te veicles, i.e. nigt conditions and bad weater, as well as maintenance oter tat refueling, are not considered. Suc detailed operational constraints are going to be implemented in te future. C. Iwate Prefecture Mission Assignment Simulation Te greedy-searc algoritm is applied to Iwate Prefecture scenario under t lim = 18 constraint. Te ard time constraint was set to be less tan 50% of te time necessary to complete all rescue missions. Wen a greedy searc is done over all 160 missions, te fitness is To provide diversity of te swarm s particle position, 30 missions at random are excluded on eac run (altogeter 30 runs to generate initial positions for all particles are done), te fligt ranges of all aircraft are set lower by 6 to 12 min, and t lim = Te fitness ten varies between 1061 and Te positions defined tis way are used to initialize te swarm. Using te PSO setting parameters and te greedy-searc-based particle initialization discussed in te previous section, we do 10 PSO runs under t lim = 18 constraint. Te calculation results are sown in Figure 12. Te average best fitness value is , or given only 18 ours operational rescue time, 1305 out of all 2153 evacuees can be transported to a safe place. For te 10 tests run, te worst fitness value is 1301 and te best one is Te total operational time istogram of all veicles for all 10 runs (altogeter 5 veicles 10 test runs = 50) is sown in te istogram in Figure 12. As seen from te figure, most veicles assignment matces te 18 limit, so it is concluded tat te assignment distribution is feasible. Tis document is provided by JAXA.

25 Figure 12. Iwate Pref. evacuation mission assignment simulation for 5 elicopters Te missions cosen to be executed differ sligtly among iterations, but overall te solution is robust enoug, as sown in Figure 13. We found tat 61 missions out of all 160 missions are always assigned to a non-dummy elicopter wile 48 missions are never performed, i.e. always assigned to a dummy veicle. Te robustness of te solution sows tat te assignment is more or less governed by te greedy searc solution, wic in turn means tat te veicle assignment proposed by te system follows uman operator s logic and can be applied easily in practice. However, because of te euristic nature of PSO, te mission assignment differs sligtly among te runs. Introducing te PSO improves te solution by 38, so te role of te PSO is to refine te results wile complying wit all constraints. Te results also sow tat te profitability property of eac mission-elicopter is an important parameter in mission assignment. Any improvement in te mission assignment wic increases te fitness is crucial for te system, as it is equivalent to more lives saved in te immediate aftermat of te disaster. Tis document is provided by JAXA.

26 Figure 13. Dummy-veicle assignments for all simulation runs. Zero indicates tat a mission was never assigned to te dummy veicle, 10 sows tat a mission was assigned to te dummy veicle in all 10 runs. D. Comparison wit Oter Algoritms To validate te performance of te proposed PSO-greedy searc algoritm, we compare te obtained results and calculation times to a genetic algoritm solver and a mixed-integer linear programming solver. All numerical tests are done on te Iwate Pref. test scenario. Here, no fuel constraints are implemented. Instead, te total operational time is reduced from 18 ours to 15.5 ours. We set t lim to 15.5 ours because even wit absolutely no wasted time, at least 5 refuel sessions are necessary during te 18 ours operation, so te maximum pure operational time is 15.5 for medium and large veicles, and 15 for te small veicle. Terefore, 15.5 ours and no refuel is a looser constraint ten 18 ours and refuel. For reference purposes, we determine also te solution by te greedy algoritm as well. Te greedy solution fitness value is Terefore, we ran tree sets of numerical tests using te 1) developed PSOgreedy searc algoritm under no refuel/ t lim=15.5 ours constraint, 2) a genetic algoritm and 3) mixed-integer linear programming optimization. All experiments are done on Intel Core i CPU@ 3.60 GH wit 8.00 GB RAM running on Windows 7, using MATLAB 2015a, so te influence of simulation environment is eliminated. Genetic Algoritm Implementation For te problem model we refer to te work of Cu et al [27]. We use te ga function provided in te MATLAB Global Optimization Toolbox, adapted to andle integer parameters. Te parameters are set according to Deep at al [28]. Te algoritm uses tournament selection, Laplace crossover and Power mutation adapted to andle integer constraints. We set te maximum number of generations to 1000 and te number of individuals tat are guaranteed to survive to te next generation to 10% of te population number. Tis document is provided by JAXA.

27 Te genetic algoritm by itself fails to find a feasible solution to te problem. We believe tat te reason for tis outcome lies in te properties of te mission time/reward table. To reflect veicle capacities and equipment, may of te mission times are penalized. GA fails to find a solution wic does not include any penalized elicopter-specific missions. Te issue wit te way GA andles infeasible solutions as also been outlined by Kennedy et al. [14]. Simulations are done for population ranging between 10 and 300 and te maximum number of iterations and allowed constraint tolerances are varied, but in no case a feasible solution is obtained. Next, we initialize te GA solution using te greedy algoritm presented in Section II (H). Te generation of one set of suboptimal solutions (one population) takes on average 0.10 sec. Te initial best fitness is We ave to increase te population to 330 in order to ave improvement in te fitness value. We run 10 trials under te same initial conditions, but only in 5 cases we observe an improvement to 1332 from te original value of Tis value is worse tan tat obtained by pure greedy searc alone. Wen te initial solution includes te optimal greedy searc solution, no cange is observed. Te autors believe tis is due to te strong dependence of GA on te initial solution and its difficulty to avoid local minima. Many of te mission locations are close to eac oter, so tere are many local minima. Tis caracteristic makes GA inappropriate for mission assignment decision support systems like te one being developed at JAXA. Mixed-integer Linear Programming Implementation We solve te same problem as a mixed-integer linear program (MILP) referring to te general capacitated knapsack model presented by Garfinkel in [29]. Here, we apply te intlinprog function from MATLAB Optimization Toolbox. It sould be noted tat a relaxation to a linear program gives an optimal objective value of , wic means tat te optimal fitness value of our problem is somewere between 1346 (te value obtained by te greedy searc) and Te stopping criteria for te PSO-Greedy algoritm is te number of iterations (set at 2000), wile te stopping criteria for te MILP was calculation time (set at 600 sec). Te results from 10 PSO-Greedy runs and a MILP run are sown in Figure 14. As seen from te figure, te PSO-Greedy algoritm performs better in terms of bot calculation time and fitness value. MILP provides better results tan GA discussed above, owever. Tis document is provided by JAXA.