Journal of Theoretical Biology

Transcription

1 Journal of Theoretical Biology 266 (2010) Contents lists available at ScienceDirect Journal of Theoretical Biology journal homepage: Balancing organization and flexibility in foraging dynamics Michaelangelo Tabone a,b, Bard Ermentrout a,b, Brent Doiron a,b, a Department of Mathematics, University of Pittsburgh, Pittsburgh, PA, USA b Complex Biological Systems Group, University of Pittsburgh, Pittsburgh, PA, USA article info Article history: Received 20 May 2010 Received in revised form 27 June 2010 Accepted 7 July 2010 Available online 11 July 2010 Keywords: oraging behavior Social insects Swarm intelligence Biological flexibility Biological organization abstract Proper pattern organization and reorganization are central problems facing many biological networks which thrive in fluctuating environments. However, in many cases the mechanisms that organize system activity oppose those that support behavioral flexibility. Thus, a balance between pattern organization and pattern flexibility is critically important for overall biological fitness. We study this balance in the foraging strategies of ant colonies exploiting food in dynamic environments. We present discrete time and space simulations of colony activity that uses a pheromone-based recruitment strategy biasing foraging towards a food source. After food relocation, the pheromone must evaporate sufficiently before foraging can shift colony attention to a new food source. The amount of food consumed within the dynamic environment depends non-monotonically on the pheromone evaporation time constant with maximal consumption occurring at a time constant which balances trail formation and trail flexibility. A deterministic, mean field model of pheromone and foragers on trails mimics our colony simulations. This reduced framework captures the essence of the flexibility-organization balance, and relates optimal pheromone evaporation to the timescale of the dynamic environment. We expect that the principles exposed in our study will generalize and motivate novel analysis across a broad range systems biology. & 2010 Elsevier Ltd. All rights reserved. 1. Introduction Biological networks use collective strategies to survive and thrive in competitive, ever-changing environments (Nicolis and Prigogine, 1977; Camazine et al., 2001; Kauffman, 1993). Detailed organization among the components of a collective is a general principle of systems biology. or example, network organization is critical for proper gene (Kauffman, 1993), cellular (Misteli, 2001; Barabasi and Oltvai, 2004), neural (Sporns et al., 2004), and societal (Parrish and Edelstein-Keshet, 1999) network function. The network mechanisms that support this self-organization require time to establish network wide patterns. Behavioral flexibility is another important characteristic for organism survival, especially for those that thrive within unpredictable environments. or example, gene networks adapt their interactions in response to shifting environmental pressure (Kashiwagi et al., 2006), cells match their structure with shifting cellular parameters to preserve function (Barkai and Leibler, 1997), neurons modify their responses to better code dynamics sensory scenes (Wark et al., 2007), and fish schools reorganize their structure due to predator threats (Partridge, 1982). In very Corresponding author at: Department of Mathematics, University of Pittsburgh, Pittsburgh, PA, USA. address: bdoiron@pitt.edu (B. Doiron). dynamic environments, organisms must reorganize on rapid timescales, necessitating fast-acting, network-based plasticity mechanisms. How biological systems balance the memory needed for pattern organization, with the forgetting needed for flexible behavior, is an interesting, yet understudied research topic (Wagner, 2005; Inada and Kawachi, 2002). We approach this general question from the standpoint of ant colony foraging in dynamic environments. Social insects have a variety of interactions that produce emergent colony dynamics, permitting societal behaviors unavailable to the insects as individuals (Hölldobler and Wilson, 1990; ewell, 2003; Hölldobler and Wilson, 2008). The foraging behavior of ant colonies is an excellent example of self-organizing phenomena in social collectives (Camazine et al., 2001), and is a popular avenue for theoretical investigation (Deneubourg et al., 1983; Haefner and Crist, 1994; Edelstein-Keshet et al., 1995; Watmough and Edelstein-Keshet, 1995, Bonabeau et al., 1998a, b, Nicolis and Deneubourg, 1999; Beekman et al., 2001; Detrain and Deneubourg, 2006; Garnier et al., 2007; Nicolis and Dussutour, 2008). Exact foraging strategies are species specific; however, broad classification schemes have been proposed which give a general view of ant colony foraging (Oster and Wilson, 1978; Hölldobler and Wilson, 1990). Various recruitment methods are distinguished by the types of communication involved between a scout and the rest of the colony during recruitment (Beckers et al., 1989; Hölldobler and Wilson, 1990; Traniello, /$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi: /j.jtbi

2 392 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) ). Mass recruitment (hereafter simply termed recruitment) (Wilson, 1962) is a specific form of chemical communication involving a pheromone trail laid by scout ants linking a food source to the colony nest. Worker ants then follow the trail to the food source, and reinforce it as they return to the nest. This form of communication is used by many diverse ant species, across most sub-families in ormicidae (see Hölldobler and Wilson, 1990, Tables 7 5 and references within). There has been much interest, both experimental and theoretical, in recruitment strategies that best exploit multiple food sources in a foraging environment. Deneubourg et al. (1983) have predicted that ant colonies whose ants have a certain probability of failure when following a pheromone trail forage with higher efficiency in a multiple food source environment than ant colonies with perfect workers. Stickland et al. (1995) extended these results by showing that this efficiency is further increased by allowing foragers to modulate their pheromone deposit dependent upon food quality, and also allow finite memory through pheromone evaporation. Other studies (Deneubourg et al., 1986; Nicolis and Deneubourg, 1999) predict symmetry breaking bifurcations in the simultaneous exploitation of multiple food sources as colony size is increased. The latter work motivated a series of detailed experiments (Beckers et al., 1992) which verified this prediction. In total, these studies have furthered the understanding of ant foraging behavior and have grounded current adaptive optimization algorithms (Bonabeau et al., 2000). However, most studies only consider the foraging performance of a colony in a static environment. Treating a resource environment as static is certainly appropriate for certain ant species. or example, harvesting ants (Pogonomyrmex) (Hölldobler, 1976) and leafcutter ants (Atta) (Hölldobler and Wilson, 1990) exploit persistent food sources that are fixed in one location. However, many other species, e.g. honeypot ants (Myrmecocystus mimicus) (Hölldobler, 1981), exploit food sources which are unpredictable in time and space. Ant foraging in dynamic and unpredictable environments has long been a topic of interest (Traniello, 1989; Gordon, 1991). Nevertheless, there are few theoretical studies discussing the underlying mechanics by which colonies best forage in ever-changing environments (but see Dussutour et al., 2009; Bonabeau, 1996). A relevant task for ant colonies that forage in dynamic environments is to shift their attention from an old food source to a new food source as the environment changes (schematically illustrated in ig. 1). A straightforward mechanism for forgetting an old food source (presumably depleted) is via the evaporation of the pheromone trail linking the food source to the nest (Morgan, 2008) (ig. 1B and C). The evaporation of pheromone permits the colony to establish a pheromone trail to a new, viable food source (ig. 1D). We study foraging in environments where the spatial distribution of resources is dynamic. We first model ant foraging as a self-biasing random walk in a two dimensional arena containing food sources and a nest site. To simulate a dynamic resource landscape we periodically relocate a food source to random spatial locations. We show that the average total food intake over a relocation cycle depends non-monotonically on the pheromone evaporation time constant, showing optimal foraging at a specific time constant. We explore this result by constructing a mean-field model of pheromone and ant dynamics on individual trails, and in the large trail limit simplify the dynamics to a boundary value problem. As in the random walk model, a maximum in foraging efficiency occurs in this reduced system at a specific value of pheromone evaporation time constant. Using both the random walk simulations and the mean-field trail model we show how the pheromone time constant that maximizes food intake depends on the timescale of environmental changes, N N showing that colony memory should match the foraging ecology. inally, we show how an optimal pheromone evaporation timescale persists when the environmental dynamic is more realistic, allowing for the co-existence of several food sources. In total, a specific colony memory, set by pheromone evaporation, balances trail organization with trail flexibility, optimizing colony foraging in unpredictable environments. 2. Models 2.1. Self-biasing random walk model ig. 1. Schematic of the organization and flexibility of ant colony foraging in dynamic environments. (A) The colony follows a trail pheromone from the nest to a food source. (B) The exploited food source disappears, while another food source appears in an different location. Pheromone between the nest and the old food source persists, biasing ant activity to the old food source. (C) Pheromone to the old food source evaporates, allowing colony exploration of the environment. (D) The colony has found new food source and establishes a pheromone trail linking the source to the nest. Past studies have modeled pheromone-based colony foraging with self-biased random walks on structures ranging from simple binary trees (Stickland et al., 1995), to complicated simulations in realistic foraging arenas (Watmough and Edelstein-Keshet, 1995; Edelstein-Keshet et al., 1995; Haefner and Crist, 1994). We opted for an agent-based (discrete time and space) simulation of colony activity within a rectangular two dimensional arena containing 120 rows by 60 columns (ig. 2). A nest is located at a fixed position (midway up left arena wall), a food source is randomly located in the arena (and relocated, see below), and 100 ants are positioned throughout the arena. Ant locations are initialized in random cells throughout the arena. Cells are considered to be significantly larger than an ant and thus can contain multiple ants at a given time. The ant positions evolve probabilistically, where at every time step an ant moves in one of eight directions (ants always move), into either an adjacent or diagonal cell. In the N N

3 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) Arena Rows Arena Columns ig. 2. Self-biasing random walk model of colony foraging. An abridged arena raster (left) with two ants, the nest (half moon), and the current food source ( symbol). A trail of cells leading from the nest to the food source is marked (solid line). Example scenarios of pheromone concentrations in cells surrounding the two ants are shown pheromone on the trail has been increased from 1 to 10. Each ant s probability of moving in a given direction is calculated by the concentration of pheromone (Eq. (1)). absence of pheromone a cell has an attractiveness of 1, and when an ant is surrounded by 8 such cells all movements are equally probable (ig. 2 lower ant). The pheromone concentration deposited on a cell biases that cell s attractiveness, so that it has a higher probability of being occupied by nearby ants than surrounding cells with a lower concentration of pheromone (ig. 2 upper ant). Specifically, for an ant at cell ij at time t the probability of the ant moving to cell kl (jk ijr1 and jj ljr1 with i¼k and j¼l disallowed) at time t+1 is P t kl ¼ r t P kl ku,lur t, ð1þ kulu where ku and lu run over adjacent and diagonal cells to cell ij, and r kl is the pheromone concentration at cell kl. We place a single food source at a fixed radius (40 cells) and a random angle (uniform distribution) from the nest. The food source initially has 100 units of food and is kept at that location for T time units. After T time units elapse, the food source disappears (regardless if it is completely depleted or not), and a new food source appears at another random angle at the same fixed distance from the nest. This process continues for the length of the simulation. The cases when food disappears before 100 units are consumed loosely model, for instance, removal of the food source by a competitor ant colony or other animal, or that the food source actually had less than 100 units of food and removal equates to complete exploitation of the source. The collapse of these cases into a periodically shifting food source is a simplification that permits further analysis (see mean field model section). In the final section of the study we replace the periodically shifting food scenario with random food arrival times and food sources do not disappear until completely consumed (ig. 8), so as to model a more realistic spatiotemporal food distribution. When a new food source is presented the colony must locate the food source, establish a pheromone trail linking the source to the nest, and ultimately intake as much food as possible. The time step after an ant reaches the food source, the ant is placed at the nest location and the food source is reduced by one unit, and pheromone is deposited on each cell along a linear path from the nest to the food source (ig. 2). Between time steps the pheromone decays with time constant t, due to evaporation. In total, the pheromone concentration in cell ij at time step t+1,, obeys: r t þ 1 ij r t þ 1 ij ¼ r t ij rt ij 1 þm X s t k : k The sum runs over ants in the arena, and s k ¼ 1 if ant k occupies the food source cell at time step t and cell ij is part of the shortest path linking the nest to the food source, otherwise s k ¼ 0. The parameter m is the amount of pheromone deposited by a single ant. The only exceptions are the pheromone concentration in cells containing nest or the food which are kept at constant values of 1 and 1000, respectively. This guarantees that ants will be attracted to the food cell and prevent ants from biasing towards the nest. We remark that in the absence of a food source (s k ¼ 0 for all k), r ij ¼ 1 is the stable equilibrium solution ðr t þ 1 ij ¼ r t ij Þ,so that movement is still defined (Eq. (1)). All simulations were run in MATLAB (Mathworks) using custom built codes. Our model makes several simplifying reductions of actual ant foraging behavior. irst, the model parameters are qualitative, chosen to give reasonable colony-based trail formation and trail decay (see ig. 4). A quantitative connection to a specific ant species, or a specific environmental dynamic, was not attempted. Second, our model of trail following is overly simplistic. Specifically, in the absence of pheromone the model assumes that the direction of motion is unbiased (ig. 2 lower ant). urther, when an ant is on a trail it is as likely to initially follow the trail to the nest as it is to the food source. In reality, ants tend to bias their movements towards the direction their head is facing, and will have knowledge of nest location so as to bias trail following away from the nest. We expect that these simplifications reduce the speed at which our model ants explore their environment, when compared to a more realistic model of trail following. This is because our model of ants motion is, in essence, simple diffusion along a trail, while directed foraging is expected to be a biased diffusion. Nevertheless, trail formation and food exploitation occur in our simplified model (ig. 4) Mean field model Mean-field models describing the pheromone concentration on a trail have given significant intuition for foraging behavior in static environments with multiple food sources (Nicolis and Deneubourg, 1999; Nicolis et al., 2003). In this section we develop a simple mean-field model which captures the foraging behavior in dynamic environments observed with the random walk model. or every trail in the environment we model the pheromone concentration on that trail, rðtþ, the amount of food at the source, f(t), and the fraction of ants following the pheromone trail, n(t) meaning that foraging dynamics are modeled with the triplet frðtþ,f ðtþ,nðtþg. At every time t that is a multiple of the food relocation time T, a new food source appears, meaning another triplet of variables (modeling a new food quantity, pheromone trail linking the nest to the food source, and ants on that trail) must be added. Specifically, for every time t¼jt (j¼0,1,2,y) we add the triplet of differential equations to the full system: dr j dt ¼ r j t þmf jn j, dn j dt ¼ n r j þeq j ðtþ m 0 n j s, r j þeq j ðtþþm 0 r j þeq j ðtþþm 0 df j dt ¼ gf jn j : ð2þ ð3þ

4 394 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) In addition, at times t¼jt we have the conditions r j ðtþ¼0, n j (t)¼0, f j (t)¼f max, and f j 1 (t)¼0. Here, n s represents the fraction of searching ants, that is the fraction of ants that are not occupied on a trail. Since colony size is fixed, then we relate n s to the trail following ants as n s ¼ 1 P k n k. The term eq j ðtþ represents the attractivity of a new source of food to the newly appearing trail, with 0oe51 and q(t)¼1 if jt rt rðjþ1þt with q(t)¼0 otherwise (i.e. this applies only to new, active food sources). The parameter e accounts for searching ants that randomly happen upon the food, allowing a new trail to be discovered by the colony. The parameter, m 0 represents the baseline attractivity of a trail with no pheromone concentration. Since we have removed space from the model, m 0 is, in a sense, proportional to the area of the arena that has no pheromone deposit. The rate at which an ant is attracted either onto, or off of, the trail is proportional to the ratio of the pheromone on, or off, the trail to the total pheromone (as in ig. 2). The parameter t is the evaporation timescale of the pheromone, m is the rate of pheromone deposited by an ant with food, and g is the efficiency of foraging or the rate at which an ant removes food. At any given time t the simulation consists of 3n differential equations, where n is the integer part of t/t+1 (three equations for 0rt ot, six equations for T rt o2t, etc.). When t becomes large compared to T then many trails exist and the dimension of the model becomes large, lengthening the simulation time. We consider a very reduced model in which we collapse all the expired trails into a single inactive trail on which remain inactive ants (in the sense that they are not actively bringing food back to the nest, yet are also not available searchers), and pheromone which is never reinforced and only evaporates. When a trail has f j (t)¼0, as is the case when a new trail appears, the pheromone on that inactive trail decays to zero (Eq. (3)). Thus, the equations for each inactive trail can be summed to a single variable that tracks the total pheromone on all inactive trails. However, the dynamics of the ants recruitment is nonlinear since it involves a ratio of pheromone. Thus, strictly speaking, we must distinguish the number of ants on every trail, inactive or active. Indeed, if we let n I ¼ P j r J n j and r I ¼ P j r Jr j where J is the number of inactive trails, then we have the following: r j dn I dt ¼ n X s X m 0 n j : r j r J j þm 0 r j r J j þm 0 To close the system we cannot deal with pheromone summation since this requires keeping track of every inactive trail. We approximate the sums of the nonlinearities, by the corresponding nonlinearities applied to the sums: P dn I dt n j r Jr j s P m P 0 j r J n j P j r Jr j þm 0 j r Jr j þm 0 r I ¼ n s m 0n I : r I þm 0 r I þm 0 With this approximation, we need to only keep track of five variables: n A, r A, f, being the ants, pheromone, and food on an active trail, respectively, and n I, r I, the ants and pheromone on all the inactive trails. Thus we have a five dimensional system that approximates the dynamics of the original 3n dimensional system (the reduction is schematically illustrated in ig. 3A). The equations are now quite straightforward: dn A r ¼ n A þe s m 0n A dt r A þeþm 0 r A þm 0 þe, j= 1, 2, 3, 4 n-1, n Inactive Trails Active Trail 3n ODEs 5 ODEs nd interval, 6 ODEs 19 th interval, 57 ODEs BVP Approx., 5 ODEs 0.3 n I ρ I 0.2 f n I f n I f 0.1 n A ρ A ρ I n A ρa ρ I n A ρa time time time ig. 3. Trail model and related boundary value problem A. Differential equation trail model has three equations for each trail: the currently active trail as well as all previous trails. A five ODE boundary value problem was used to approximate the solution in the large trail regime. (B and C) Results from the 3n ODE model for the first (B) and 19th (C) interval, respectively. (D) Boundary value approximation of trail model. Model parameters are T¼5, m ¼ 1, g ¼ 1, e ¼ 0:01, t ¼ 1, and f max ¼2. or plotting purposes we plot f/10 rather than f.

5 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) r I dn I dt ¼ n s m 0n I, r I þm 0 m 0 þr I dr A dt ¼ mfn A r A t, dr I dt ¼ r I t, df dt ¼ gfn A: ð4þ These five equations represent the foraging dynamics for periodically appearing food after the initial transients have decayed. Here, the fraction of searching ants is simply n s ¼1 n A n I. The periodicity in the appearance and disappearance of food must be modeled with the appropriate boundary and initial conditions. We are interested in the dynamics between the appearance of two successive sources of food. Let t¼0 be the start time of a new trail and t¼t be the end time for that trail, when the food is removed and a new trail is started. Since a new pheromone trail has not formed at the moment that the food appears, we have the obvious initial conditions, n A ð0þ¼r A ð0þ¼0 and f(0)¼f max where f max is the initial food quantity placed on the new trail. or the inactive trails, the number of ants at t¼0 should be the number of ants remaining on inactive trails added to the ants that were on the active trail at the end of the last cycle, that is n I (0)¼n I (T)+n A (T). Similarly, r I ð0þ¼r I ðtþþr A ðtþ. In summary, we solve Eq. (4) subject to the five boundary conditions: n A ð0þ¼0, r A ð0þ¼0, f ð0þ¼f max, r I ð0þ¼r I ðtþþr A ðtþ, n I ð0þ¼n I ðtþþn A ðtþ: These conditions yield a periodic solution to the system in Eq. (4) in the sense that at the end a time interval T, the system is exactly in the same state as it was at the beginning. Since we want to eventually vary T, we rescale time to lie between 0 and 1 and thus let T be a parameter. Solutions of the trail model (Eq. (3)) converge to a limiting periodic solution (ig. 3C) after the decay of an initial transient (ig. 3B). That is, the solution after nineteen intervals (ig. 3C) goes essentially unchanged for successive bouts of foraging. Comparing it the periodic solution to the boundary value problem (Eq. (4)) we see there is little difference (ig. 3D). Thus, the approximation of the sum of the pheromone ratios by the ratio of the pheromone sums does not introduce significant errors. Both the differential equation and boundary value meanfield models were solved using the simulation package XPP (Ermentrout, 2002). 3. Results 3.1. Dynamic environments and foraging attention In response to relocation of food sources ant colonies must shift their foraging organization from an old food source location towards a new location (schematically illustrated in ig. 1). We first explore shifts in foraging attention with a self-biassing random walk model of colony activity (see Models). Our random walk model is similar to previous agent-based models of pheromone trail formation (Haefner and Crist, 1994; Edelstein- Keshet et al., 1995; Watmough and Edelstein-Keshet, 1995), and both trail formation and trail decay in our model are similar to results reported in these studies. However, the food relocation and subsequent trail reorganization aspect of our study (see ig. 1) is separate from the majority of past work on colony foraging dynamics. Simulations of the random walk model show dynamic shifts of colony attention as food is relocated (ig. 4). Near the end of a food cycle (t¼1900 with T¼2000) the ants 100 ood on Trails Trali 1 Trali 2 Time = 1900 Time = Time = 3000 Time = 3900 Pheromone on Trails Trali 1 Trali Time (10 3 ) ig. 4. Random walk model output for two cycles of food relocation. (A D) Ant, food, and nest locations within the arena at different time points. (E) The amount of food on the two trails. () The pheromone on each trail throughout the simulation. Since each cell in a trail has a pheromone update identical to all other cells on the trail, it suffices to show the pheromone concentration on one cell in a trail. The self-biasing random walk model was run for a total of 4000 time steps. Model parameters were, T¼2000, t ¼ 700, and g ¼ m ¼ 1.

6 396 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) establish a pheromone trail from the nest to the active food source location (ig. 4A). At the beginning of the second food cycle (t¼2100) the food source location switches, but ants remain on the trail to the old food source (ig. 4B). Halfway through the second food cycle (t¼3000), a small number of ants exploit the new food source, and fewer ants follow the trail to the old food source (ig. 4C). At the end of the second food cycle (t¼3900) ants no longer utilize the old trail and focus on exploiting the new food source (ig. 4D). Overall, the colony attention shifts from the old to new food sources promoting continued food consumption in an unpredictable environment (compare ig.4a D to the schematic illustration of ig. 1). Rather than measuring foraging with the position of the ants within the arena, it is useful to consider foraging from the perspective of the food consumption at each source (ig. 4E), and the pheromone concentration on each trail (ig. 4). During exploitation of a food source, the pheromone concentration on the trail linking the source to the nest increases in an erratic manner, owing to the stochastic nature of colony foraging. Nevertheless, once the pheromone concentration becomes sufficiently strong, a clear pheromone trail is established, and the food quantity at the source steadily decreases. When the food source connected to the first trail is removed (ig. 4E, at time t¼t, black curve), the pheromone concentration on that trail exponentially decreases in time (ig. 4, black curve) because of pheromone evaporation. Pheromone evaporation on the first trail permits colony exploration, resulting in discovery and exploitation of the new food source (ig. 4E, gray curve). Describing foraging dynamics with macro states, such as pheromone on a trail and the amount of food at a source, will facilitate a mean-field description of foraging behavior (see below, and references Nicolis and Deneubourg, 1999; Nicolis et al., 2003) ood consumption is maximized at a specific pheromone evaporation time constant Successful foraging in environments with dynamic resources requires two distinct colony behaviors. The first is for the colony to forget the old food source, allowing for subsequent exploration of the environment. The second is for the colony to organize a pheromone trail to the new food source. Both of these behaviors are significantly influenced by the pheromone time constant t. The forgetting time of the colony is directly set by t, since once a food source has disappeared then the pheromone dynamics are straightforward and depend only on t (Eq. (2) with s k ¼ 0forallk). In contrast, the creation of a new pheromone trail to an active food source requires that the pheromone deposits from individual ants remain long enough (i.e. large t). This will bias colony activity towards that source, permitting reinforcement of the pheromone trail, yielding stable exploitation of the new source. Thus, of critical interest is the overall effect of t on the colony attentional shift from a depleted food to a new viable food source. One of the principal results of our study is that the mean food captured per cycle by the colony depends non-monotonically on t (ig. 5A). Specifically, a maximum in food consumption occurs at a non-zero value of t ðt 1000Þ. Indeed, food consumption is reduced on a cycle-by-cycle basis when t is either smaller (ig. 5B), or larger (ig. 5D), than the t that maximizes consumption (ig. 5C). or small t the colony does not have sufficient time to organize a pheromone trail to a food source before that source disappears. In contrast, when t is large the memory of old, removed food sources persists and interferes with colony exploration for new sources. The t that best mitigates these two aspects of foraging results in maximal food consumption and marks a balance between the mechanisms of Average Periodic ood Consumption raction of ood Consumed τ, Pheromone Evaporation Time Constant τ = 10 2 τ = 10 3 τ = time (10 3 ) time (10 3 ) time (10 3 ) ig. 5. oraging success depends non-monotonically on the pheromone evaporation time constant, t. (A) The black line is the average fraction of food consumed over one cycle (averaged across 500 cycles). The shaded gray area is one standard deviation above and below the mean. (B D) ood consumption time series with t ¼ 100, (B) 1000, (C) and 10,000 (D), respectively. Model parameters were T¼1000, m ¼ 1, and g ¼ 1. In all cases the first food cycle was discarded to neglect transients due to the ants initial placement in the arena. colony trail organization and those of colony flexibility to unpredictable environmental changes Mean field model of dynamic foraging A maximum in the mean amount of consumed food in a single cycle at a specific t is produced in the random walk model (ig. 5A). To study this behavior, we next investigate a mean field model of dynamic foraging, and an associated boundary value problem approximation (see Models). We solve the mean field model (Eq. (4)) and consider the total food consumed by the colony over one cycle, f max f(t), as t varies. In agreement with the random walk model, the food consumed by the colony is a humped function of t (ig. 6A), with a specific t maximizing food consumption. Simulations of the full differential system (Eq. (3)) for values of t both smaller (ig. 6B), larger (ig. 6D), and at the value of t that maximizes food consumption (ig. 6C) show results mimicking those obtained with the random walk model (ig. 5B D). We remark that for vanishing t, the boundary value model shows non-zero food consumption (ig. 4A, t-0). This is due to e40, meaning that even if r A 0 (as is the case when t is

7 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) Total ood raction of ood Consumed Each Interval T = 5000 T = 3000 T = τ, Pheromone Evaporation Time Constant τ, Pheromone Evaporation Time Constant T = 10 ood Consumption τ = 0.1 τ = 1.0 τ = Total ood T = 5 ig. 6. oraging success as a function of t in the mean-field model A. Total food consumption as a function of the pheromone evaporation time constant, t, solved via the five ODE boundary value approximation. (B D) ood consumption time series computed from the full ODE model with t ¼ 0:1 (B), 1 (C), and 10 (D), respectively. Model parameters are T¼1, m ¼ 1, g ¼ 1, e ¼ 0:01, and f max ¼ T = 2.5 very small since trails cannot be formed), the number of ants on the active trail, n A, will still be non-zero and food will be consumed. This accounts for the capture of food purely by chance. In the random walk model shows minimal (but non-zero) food capture is very unlikely since the number of cells in the arena is much larger than ants in the colony, thus for t-0 the random walk model shows minimal (but non-zero) food capture. In total, despite the simplicity of the mean-field trail model and its associated boundary value problem approximation, the model replicates the findings with the random walk simulations. In particular, the boundary value model shows a balance between trail organization and flexibility occurring at a specific pheromone evaporation time constant Matching environmental timescale with pheromone evaporation It is reasonable to expect that the value of t which balances trail flexibility with stable trail organization depends on the timescale of the dynamic environment (T). Indeed, when T changes, the food consumed versus t curves are modulated in both the random walk (ig. 7A) and the boundary value (ig. 7B) models. As expected, when T increases (i.e. the environment changes more slowly) the total food captured per cycle increases, since once a trail is established there is a longer period of time for resource capture. This occurs for all values of t in both models. τ opt τ, Pheromone Evaporation Time Constant T, Interval of ood Switching ig. 7. Matching environment timescale T with pheromone timescale t. (A) The food consumption as a function of t, for three different values of T in the random walk model. (B) Same as A but the solutions are obtained with the boundary value model. (C) t opt, the value of t that maximizes food consumption, is plotted against T. The solution is obtained with the boundary value model. Model parameters are identical to those reported in igs. 2 and 4. urther, and more interesting, when T increases the value of t that maximizes food capture, labeled t opt, also increases. This indicates that longer pheromone memory is advantageous when environments change slowly, while fast changing environments are better

8 398 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) exploited with short pheromone memory. In fact, tracking t opt as a function of T using the boundary value model shows a near linear dependence of t opt on T, especially for sufficiently large T (ig. 7C). The intuition for why pheromone evaporation must match environment timescale to ensure optimal foraging is straightforward. In the limiting case of static environments ðt-1þ, excessive trail memory is not deleterious to colony foraging. This is because rapid foraging flexibility to environmental shifts is less important for colonies in slowly changing environments than for those in fast changing environments, since the penalty for residual memory of old food sources is less severe. However, stable construction of a trail once foragers are available always benefits food capture, and large t ensures reliable trail construction (recall that the initial state has no pheromone trails). As T is reduced, more importance is placed on the switching of colony attention, and t opt reduces. Thus, T sets the relative importance of flexibility against organization of pheromone trails, and t opt is sensitive to this balance More realistic random environments The environment in the random walk model is stochastic only in the angle of new food sources relative to the nest; the time of food appearance and disappearance is periodic, and the food distance from the nest is fixed. Realistic environments have additional sources of randomness, which could in principle dilute the sensitivity of food capture success on t. To test the robustness of optimal pheromone evaporation rate in a more realistic environment, the random walk simulation is adjusted to allow for variable distance of food sources from the nest, variable number of food sources available, and variable time intervals of food appearance. or simplicity, we choose the time of food appearance to follow a Poisson process (e.g. exponentially distributed inter-arrival intervals with mean interval T m ), and the location of food placement to be spatially uniform across the entire arena (as opposed to at a fixed distance from the nest). In addition, old food is not removed upon the placement of a new food source, however, the amount of food available decays exponentially with a separate time constant, t e. or reasonable t and T m the random walk model shows time epochs where multiple food sources are being simultaneously exploited (ig. 8A). As a result the variability in the amount of food captured in an interval of time is larger than the periodic food placement case. Nevertheless, when t is varied, a clear maximum in the amount of food captured is observed at a particular t (ig. 8B). The maximum is not as prominent as in the periodic food placement simulations (ig. 5A), as expected due to the increased variability in food capture (compare the standard deviation region in igs. 5A and 8B). 4. Discussion Average Periodic ood Consumption τ, pheromone evaporation time constant ig. 8. Optimal pheromone evaporation in more random environments. The random walk simulation is adjusted to allow for the co-existence of multiple food sources at random distances from the nest, appearing at random time intervals. (A) Ant, food, and nest locations in the arena during one realization of the random walk simulation. The colony exploits one food source while a closer food source as recently been placed (left). The colony exploits both food sources while a third food source has been placed in the arena (middle). The colony remains on the pheromone trail to the first food source after it has been depleted and continues to not exploit the third food source (right). (B) The average fraction of food consumed during a time interval, T m, (normalized by the initial food level at a single source) is shown for varying values of t. The black line represents the mean of computed over 4000T m at each recorded value of t. The shaded gray region represents one standard deviation above and below the mean. All simulations have T m ¼1000, g ¼ 1, m ¼ 1, and t e ¼ 330, and t ¼ 300 in A. We have shown an optimal balance between pheromone trail organization and trail flexibility at a specific pheromone memory. This principle is apparent in both random walk simulations of colony dynamics, and reduced, deterministic mean-field trail models of foraging dynamics. The exact value of pheromone evaporation time constant which optimizes food capture depends on the timescale of environment resources in slower changing environments are better exploited with longer trail memory than resources in fast changing environments. inally, our result extends to more erratic environments where food placement is random in space and time, and where multiple food sources coexist. The general principle of balancing organization and flexibility has been commented upon in several studies (Traniello, 1989; Gordon, 1991; Hölldobler and Wilson, 2008; Jeanson et al., 2003; Dussutour et al., 2009; Bonabeau, 1996; Inada and Kawachi, 2002). However, to our knowledge, this is the first study to explicitly focus on a single parameter (pheromone evaporation time constant) which controls the balance. A first prediction of our model is that ant species which forage in transient environments have pheromone deposits which evaporate more quickly than species which forage in environments with more stable resources. In agreement, Tapinoma simrothi use long lived trails to forage from predictable honeydew sources farmed from aphids (Simon and Hefetz, 1991). In contrast, the pheromone used by the Argentine ant Linepithema humile (Deneubourg et al., 1990) and the Pharoh s ant Monomorium pharaonis (Jeanson et al., 2003) lasts approximately 30 min, aiding them in exploiting ephemeral food sources. An extreme example of transient colony memory is used by army ants of the genus Onychomyrmex, which release a potent yet short-lived pheromone which evaporates within minutes, allowing them to consume small prey in one large swarm (Hölldobler and Wilson, 2008).

9 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) Thus, the pheromone evaporation timescale used by diverse ant species is often matched to the foraging ecology in which the colony must thrive. In our study, trail organization and foraging flexibility are both determined, in large part, by pheromone evaporation. While it is true that pheromone evaporation plays an important role in real ant colony foraging, our random walk and mean field treatment are oversimplifications of actual foraging strategies. It is has been shown that foraging ant colonies bias the amount of pheromone deposited on a trail depending on the quality and quantity of food that the trail leads to (Beckers et al., 1993). An extreme version of selective pheromone deposit occurs in the Pharoh s ant Monomorium pharaonis which deposit a repellent pheromone on trails to depleted food sources, facilitating the colony in ignoring unrewarding locations (Robinson et al., 2005). Modeling studies have shown how repellent pheromones increase foraging efficiency in static, multi-source environments by reducing foraging of sub-optimal resources (Robinson et al., 2008). urther, and more relevant to our study, in dynamic environments repellant pheromones ensure rapid switching of colony attention from a depleted source to a new source (Robinson et al., 2008). In addition, there is recent evidence that the ant species Pheidole megacephala uses distinct exploration and foraging pheromones, each with their own evaporation time constants (Dussutour et al., 2009). Modeling predictions, combined with experimental verification, show that exploration pheromone promote foraging pheromone deposition, and permits rapid attentional switches for colonies foraging in dynamic environments (Dussutour et al., 2009). A further model simplification of trail organization is our treatment of pheromone dynamics. In reality, pheromone itself is a complex chemical mixture (Morgan, 2008), often not evaporating as a simple exponential, yet showing several timescales of persistence. Resource dependent pheromone deposit and complex pheromone timescales would be interesting extensions to the pheromone evaporation shaping of colony foraging presented in this paper. Another reduction used by our random walk model is that ants only reinforce trail pheromone after food capture, as they return to the nest. However, several ant species have been documented to also deposit pheromone upon the trail as they follow the trail to a food source (Beckers et al., 1992; Jackson and Châline, 2007), so to ensure a guide back to the nest. While our model assumption should not significantly impact trail formation, it oversimplifies the process of a colony forgetting. This is because our model trail is never reinforced after food has been removed and trail pheromone simply evaporates, while real ants sometimes reinforce trails that do not lead to food sources. If ants were to reinforce old trails that do not lead to viable food sources, at an albeit lower pheromone deposit rate (Jackson and Châline, 2007), then trail extinction would likely be a longer process. Nevertheless, the presence of a new, viable food source would still bias colony attention away from the old food source, reducing the number of foragers that reinforce old trails. We expect that accounting for this behavior would still produce an optimal pheromone time constant, yet one that is shifted to shorter values, thus counteracting any reinforcement of worthless trails. Mean-field theory has a long history in the collective dynamics of insects (Deneubourg et al., 1983; Haefner and Crist, 1994; Edelstein-Keshet et al., 1995; Bonabeau, 1996, Bonabeau et al., 1998a, b, Nicolis and Deneubourg, 1999; Beekman et al., 2001; Detrain and Deneubourg, 2006; Garnier et al., 2007; Nicolis and Dussutour, 2008; Dussutour et al., 2009), with several studies specifically addressing the dynamic resource problem (Bonabeau, 1996; Dussutour et al., 2009). Bonabeau (1996) used specific initial conditions in a mean field to study the dynamic food source problem by using equilibrium techniques, while Dussutour et al. (2009) performed numerical simulations of two trail field equations to analyze colony responses to dynamic environments. In our treatment, we extended standard meanfield techniques to study the fully dynamic, multiple trail problem by using a simple boundary value problem approximation. The mean-field approximation introduced here can be easily adopted to study other basic questions involving colony dynamics in multiple trail, dynamic environments. Biological networks requiring a mixture of mechanisms to support function often must compromise between the fitness of individual mechanisms for the benefit of overall system behavior. Thus, while a specific sub-behavior may appear sub-optimal, the system wide function can operated near its optimum. Our study shows a simple example of this principle, where a single parameter controls optimality. We expect that similar examples can be discovered in diverse biological systems. Acknowledgements unding was provided by NS-EMSW21-RTG BD is a Sloan research ellow. We thank Leonard Maler, Andre Longtin, and Carlo Laing for motivating discussions. References Barabasi, A.-L., Oltvai, Z., Understanding the cell s functional organization. Nature Genetics 5, Barkai, N., Leibler, S., Robustness in simple biochemical networks. Nature 387, Beckers, R., Deneubourg, J.-L., Goss, S., Trail laying behaviour during food recruitment in the ant Lasius niger. Insectes Sociaux 39, Beckers, R., Deneubourg, J.L., Goss, S., Modulation of trail laying in the ant Lasius niger (hymenoptera: formicidae) and its role in the collective selection of a food source. Journal of Insect Behavior 6, Beckers, R., Goss, S., Deneubourg, J.-L., Pasteels, J., Colony size, communication and ant foraging strategy. Psyche 96, Beekman, M., Sumpter, D., Ratnieks,., Phase transition between disordered and ordered foraging in pharaoh s ants. Proceedings of the National Academy of Sciences 98 (17), Bonabeau, E., Marginally stable swarms are flexible and robust. Journal de Physique (6), Bonabeau, E., Dorigo, M., Theraulaz, G., Inspiration for optimization from social insect behaviour. Nature 406, Bonabeau, E., Theraulaz, G., Deneubourg, J.-L., 1998a. Group and mass recruitment in ant colonies: the influence of contact rates. Journal of Theoretical Biology 195, Bonabeau, E., Theraulaz, G., Deneubourg, J.-L., 1998b. The synchronization of recruitment-based activities in ants. BioSystems 45, Camazine, S., Deneubourg, J.-L., ranks, N., Sneyd, J., Theraulaz, G., Bonabeau, E., Self-Organization in Biological Systems. Princeton University Press. Deneubourg, J.-L., Aron, S., Goss, S., Pasteels, J., The self-organizing exploratory pattern of the argentine ant. Journal of Insect Behavior 3, Deneubourg, J.-L., Aron, S., Goss, S., Pasteels, J., Buerinck, G., Amplification processes and the number of participants: how they contribute to the foraging properties of ants. Physica D 22, Deneubourg, J.-L., Pasteels, J., Verhaeghe, J., Probabilistic behavior in ants: a strategy of errors? Journal of Theoretical Biology Detrain, C., Deneubourg, J.-L., Self-organized structures in a superorganism do ants behave like molecules. Physics of Life Reviews 3 (3). Dussutour, A., Nicolis, S., Shephard, G., Beekman, M., Sumpter, D., The role of multiple pheromones in food recruitment by ants. Journal of Experimental Biology 212, Edelstein-Keshet, L., Watmough, J., Ermentrout, G., Trail following in social insects: individual properties determine population behavior. Behavioral Ecology and Sociobiology 18, Ermentrout, G., Simulating, Analyzing, and Animating Dynamical Systems: A Guide to XPPAUT for Researchers and Students. SIAM Press. ewell, J., Social insect networks. Science 301, Garnier, S., Gautrais, J., Theraulaz, G., The biological principles of swarm intelligence. Swarm Intelligence 1 (1), Gordon, D., Behavioral flexibility and the foraging ecology of seed-eating ants. The American Naturalist 138, Haefner, J., Crist, T., Spatial model of movement and foraging in harvester ants (pogonomyrmex)(i): the roles of memory and communication. Journal of Theoretical Biology 166,

10 400 M. Tabone et al. / Journal of Theoretical Biology 266 (2010) Hölldobler, B., Recruitment behaviour, home range orientation anf territoriality in harvester ants Pogonomyrmex. Behavioral Ecology and Sociobiology 1, Hölldobler, B., oraging and spatiotemporal territories int he honey ant Myrmecocystus mimicus wheeler (hymenoptera: formicidae). Behavioral Ecology and Sociobiology 9, Hölldobler, B., Wilson, E., The Ants. Harvard University Press. Hölldobler, B., Wilson, E., The Superorganism: The Beauty, Elegance, and Strangeness of Insect Societies. WW Norton and Co Inc. Inada, Y., Kawachi, K., Order and flexibility in the motion of fish schools. Journal of Theoretical Biology (214), Jackson, D., Châline, Modulation of pheromone trail strength with food quality in pharaohs ant, Monomorium pharaonis. Animal Behavior 74, Jeanson, R., Ratnieks,., Deneubourg, J.-L., Pheromone trail decay rates on different substrates in the pharaoh s ant, Monomorium pharaonis. Physiological Entomology 28 (3), Kashiwagi, A., Urabe, I., Kaneko, K., Yomo, T., Adaptive response of a gene network to environmental changes by fitness-induced attractor selection. PLoS one, vol.1, e49, doi: /journal.pone Kauffman, S.A., The Origins of Order: Self-Organization and Selection in Evolution. Oxford University Press. Misteli, T., The concept of self-organization in cellular architecture. Journal of Cell Biology 155 (2), Morgan, D., Trail pheromones of ants. Physiological Entomology 34, Nicolis, G., Prigogine, I., Self-Organization in Non-equilibrium Systems. Wiley. Nicolis, S., Deneubourg, J.-L., Emerging patterns and food recruitment in ants: an analytical study. Journal of Theoretical Biology 198, Nicolis, S., Detrain, C., Demolin, D., Deneubourg, J.-L., Optimality of collective choices: a stochastic approach. Bulletin of Mathematical Biology 65 (5), Nicolis, S., Dussutour, A., Self-organization, collective decision making and resource exploitation strategies in social insects. European Physical Journal B 65 (3), Oster, G., Wilson, E., Caste and Ecology in the Social Insects. Princeton University Press. Parrish, J., Edelstein-Keshet, L., Complexity, pattern, and evolutionary tradeoffs in animal aggregation. Science 284, Partridge, B., Structure and function of fish schools. Scientific American 246, Robinson, E.J.H., Jackson, D.E., Holcombe, M., Ratnieks,.L.W., Insect communication: no entry signal in ant foraging. Nature 438, 442. Robinson, E.J.H., Ratnieks,.L.W., Holcombe, M., An agent-based model to investigate the roles of attractive and repellent pheromones in ant decision making during foraging. Journal of Theoretical Biology 255 (2), Simon, T., Hefetz, A., Trail-following of Tapinoma simrothi (formicidae: Dolichoderinae). Insectes Sociaux 38, Sporns, O., Chialvo, D., Kaiser, M., Hilgetag, C., Organization, development and function of complex brain networks. Trends in Cognitive Sciences 8, Stickland, T., Britton, N., ranks, N., Complex trails and simple algorithms in ant foraging. Proceedings of the Royal Society B 260, Traniello, J.., oraging strategies of ants. Annual Review of Entomology 34, Wagner, A., Robustness and Evolvability in Living Systems. Princeton University Press. Wark, B., Lundstrom, B., airhall, A., Sensory adaptation. Current Opinion in Neurobiology 17, Watmough, J., Edelstein-Keshet, L., Modelling the formation of trail networks by foraging ants. Journal of Theoretical Biology 176, Wilson, E., Chemical communication among workers of the fire ant Solenopsis saevissima. 1. The organization of mass-foraging; 2. an information analysis of the odor trail; 3. the experimental induction of social responses. Animal Behavior 10, ,