Evolution of Altruism

Transcription

1 Evolution of Altruism BRIAN MAYOH Department of Computer Science, University of Aarhus, Ny Munkegade, bldg. 540, 8000 Aarhus C, Denmark Summary Altruism is widespread in nature; there are many theories as to why it is beneficial and some theories why it has evolved. Some theories are based on game theoretic models of repeated interactions between individuals. Several recent papers have examined the biologically realistic model where allowed players can vary their investment in the repeated games. This much widens the players' choice of strategies and the tournaments described in this paper show that there are strategies that lead to reciprocal altruism even faster than the promising. "Raise The Stakes" strategy. This complements the results in (Killingback et al., 1999) that altruism arises in spatially distributed ecosystems, even when individuals cannot recognise each other so repeated games are inappropriate.the main innovation in this paper is a natural strategy for the biologically realistic situation when partners have very different cost-benefit ratios. Our experiments show that altruism can evolve, even when partners have very different cost-benefit ratios and must make very different investments. They also suggest

2 an explanation of why subordinates should be more altruistic than dominants, even when subordinates are not coerced by dominants. Altruism is widespread in nature; there are many theories as to why it is beneficial and some theories why it has evolved. Some theories are based on game theoretic models of repeated interactions between individuals. Usually the payoff matrix in the repeated games does not change, but some have followed May s suggestion (May, 1997) and allowed the game players to vary their investment in the repeated games. This much widens the players choice of strategies and in the tournaments described in (Roberts & Sherratt, 1999) the RTS, Raise The Stakes, strategy does very well. However in a very recent note Killingback and Doebeli (Killingback et al., 1999a) claim that RTS inevitably evolves into defection, so strategies like RTS are not the reason why reciprocal altruism has evolved. The tournaments described in this paper show that there are strategies that lead to reciprocal altruism even faster than RTS. This complements the results in (Killingback et al., 1999b)that altruism arises in spatially distributed ecosystems even when individuals cannot recognise each other so repeated games are inappropriate. However the main innovation in this paper is a natural strategy for the biologically realistic situation when partners have very different cost-benefit ratios. As in (Boyd, 1992; Leimar, 1997: Roberts & Sherratt, 1998; Sherratt & Roberts, 1999) our experiments show that altruism can evolve, even when partners have very different cost-benefit ratios and must make very different investments. They also suggest an explanation of why subordinates should be more altruistic than dominants, even when subordinates are not coerced by dominants.

3 1 Reciprocal Altruism Game In the reciprocal altruism game there are R rounds between Leader and Follower. In round i each player benefits from an altruistic payment of the other player and the payoffs are P L (i) = k L x pay F (i) - pay L (i) P F (i) = k F x pay L (i) - pay F (i) where k F and k L are the players altruistic cost-benefit constants. The strategies of the players determine the amounts pay L and pay F ; these amounts cannot be negative but they can be zero- corresponding to deception in the much studied Prisoners Dilemma game. Our game reduces to that in (Roberts & Sherratt, 1999) when the altruistic benefit is real and the same for all players: 1 < k F =k L.The total payoffs for the players are: Reward L = k L x Cost F - Cost L Reward F = k F x Cost L Cost F where Cost L = Σ pay L (i) and Cost F = Σ pay F (i). Altruism is rewarding for both players if k F > Cost F /Cost L > 1/k L, so it will be interesting to see if altruistic strategies evolve when k F x k L > 1 but k F or k L is under 1. The reciprocal altruism game reduces to the much studied Iterative Prisoners Dilemma when k F = k L = k > 1 and all pay L (i) and pay F (i) are either 0 or 1. The payoff to two cooperators is R = k-1, the payoff to two deceivers is P = 0, the traitor payoff is T = k and the sucker payoff is S = -

4 1. Clearly the crucial inequalities T > R > P > S and R >(S + T)/2 are satisfied. Our distinction between Leaders and Followers only matters if a follower strategy is allowed to use payl(i) in determining pay F (i). The reciprocal altruism game reduces to the non-stochastic version of the game in (Boyd, 1992) when k F = Benefit F /Cost L and all pay F (i) are either 0 or Cost F, k L = Benefit L /Cost F and all pay L (i) are either 0 or Cost L. In the stochastic version of the game there is a fixed probability of another round, but the last part of (Boyd, 1992) presents the arguments for why this is biologically unrealistic. Two innovations of (Killingback & Doebeli, 1998) should be mentioned. Firstly their payoffs are P L (i) = Benefit ( pay F (i) ) - Cost(pay L (i)) P F (i) = Benefit ( pay L (i) ) - Cost(pay F (i)) Secondly they assume benefits have the form A ( 1 - exp(- B pay )) and costs have the form C x pay with constants A, B and C. These innovations may be more biologically realistic but they complicate the issue of whether Raise the stakes strategies are one way altruism and symbiosis evolve in natural societies. However they do focus our attention on the units in our P L (i) and P F (i) equations. Certainly k F has a factor Utility F /Utility L, certainly k L has a factor Utility L /Utility F, but they must have other factors if

5 we are to avoid k L x k F = 1 and a game in which altruism is unlikely to evolve. Sometimes high k L and low k F can be interpreted Leader is subordinate to Follower & Utility L is high Follower dominates Leader & Utility F is low Leader benefits greatly from cooperation so Follower can expect cooperation at little cost to himself. Another interpretation is when leaders and followers have different sexes and MaleUtility FemaleUtility. The reciprocal altruism game is very similar to the game in (Leimar, 1997). That game can be formulated as: there are R rounds between Leader and Follower; in round i each player benefits from an altruistic payment of the other player and the payoffs are PL(i) = Benefit L (Q L (i)) x pay F (i) - Cost L (Q L (i)) x pay L (i) P F (i) = Benefit F (Q F (i)) x pay L (i) - Cost F (Q F (i)) x pay F (i). where pay F (i) and payl(i) are either 0 or 1, Q L (i) is the leader s quality during round i and Q F (i) is the follower s quality during round i. In (Leimar, 1997). R, QL(i) and Q F (i) are determined stochastically. Estimating partner quality seems to be biologically realistic and it is incorporated in our new strategies for the reciprocal altruism game in the form: λ = kown / kopponent.

6 2 Biological Relevance of the Reciprocal Altruism Game Interactions between biological organisms that benefit both organisms are very common in nature. If individuals cannot recognise each other the results in (Killingback et al., 1999b) can explain altruism and our repeated games are inappropriate. In social animals many actions are altruistic and the reciprocal altruism game is a good model for social animals that can recognise one another and have repeated interactions- one player in the game may represent a dominant chimpanzee or hen, the other a subordinate. Sometimes this can be explained by kinship relations and Hamilton s concept of inclusive fitness. The reciprocal altruism game may not be the best model for the mutual interactions between parents and children, but it may be a good model for grooming and other interactions between individuals that may be distant kin.nevertheless (Boyd, 1992) shows that kin cooperation can give altruistic strategies a foothold from which they can invade a population. In (Frean, 1996) the examples are: predator inspection in stickleback voluntary regurgitation of food in vampire bats. In (Roberts & Sherratt, 1999) the examples are: reciprocal interaction among guillemots preening one another (allopreening bouts vary from under a second to over a minute) allogrooming in impala the tendency of people to form friendships and to act preferentially towards friends potlatching in Pacific societies. In [Dug98] there is a new example

7 egg swapping in hermaphrodite fish but there are also examples of the closely related group selected cooperation : raiding and warfare in chipanzees queen ants jointly founding a new colony. A possible medical example is the balance between two kinds of cancer cells (Tomlinson & Bodmer, 1997) The detailed study of the queen ant example in (Bernasconi & Strassmann, 1999) shows that the assumption kf kl is particularly appropriate, because the cooperation ends with the strongest queen killing all the other queens in the joint colony. Even so the benefits of cooperation are great- colonies with more than one foundress are more successful at preventing workers from neighbouring colonies from stealing their eggs or destroying their nests. In all these examples the interactions are between organisms of the same species but the reciprocal altruism game may also be a good model for other interactions. How do cleaner fish decide which client fish to serve and which not? How do predators decide which prey to hunt and which not? How do herbivores decide which plants to eat and which not? As individuals repeatedly interact with members of another species they may be playing the reciprocal altruism game. Most interspecific relations are inherited but some are learnt from experience in the richness of any particular environment. In (Bazzan et al., 1999) experiments with the Iterated Prisoner s Dilemma game were interpreted as showing that moral sentiments, being benevolent to build up trust, play a role in altruism. Some of our experiments can also be interpreted like this, the player who benefits most

8 from mutual altruism tolerates shortchanging strategy SC and cheating by the other player. Mathematical investigations of a simple altruism game (Eshel et al. 1999) show that sometimes altruism can be an unbeatable strategy, just as if one was interacting with close kin. Many interactions between biological organisms that benefit both organisms are because of by-product mutualism (Brown, 1983) not Dilemma games; (Dugatkin, 1998) gives biological examples but we do not because altruism always evolves and by-product mutualism is not captured by our Reciprocal Altruism Game. 3 Strategies in the Reciprocal Altruism Game The strategies we consider are NA) Non-altruism: pay = 0 always GGG) Give-as-good-as-you-get: if payl(1) then a else opponent s last payment SC) Short-changer: if payl(1) then 1 else opponent s last payment - 1 RTS) Raise-the stakes: if payl(1) then a else if opponent undercut then opponent s last payment else if opponent matched then own last payment + b else own last payment + 2 b OSC) Occasional-short-changer: chance determines either RTS or RTS -1 OC) Occasional-cheat:

9 chance determines either RTS or 0 AWD) Anything-will-do: pay = a always AON) All-or-nothing if opponent undercut then 0 else a RTG) Generous raise-the stakes: if payl(1) then a else if opponent undercut then opponent s last payment else if opponent matched then own last payment + b else opponent last payment + 2 b OSG) Generous occasional-short-changer: chance determines either RTG or RTG -1 OCG) Generous occasional-cheat: chance determines either RTG or 0 RTL) Flexible Mean raise-the stakes: λ = kown / kopponent if payl(1) then a else if own last payment > λ opponent last payment then opponent s last payment else if own last payment = λ opponent last payment then own last payment + b else own last payment + 2 b OSL) Flexible raise-the stakes: λ = kown / kopponent if payl(1) then a else if own last payment > λ opponent last payment then λ opponent s last payment

10 else if own last payment = λ opponent last payment then own last payment + b else own last payment + 2 b OCL) Flexible Generous raise-the stakes: λ = kown / kopponent if payl(1) then a else if own last payment > λ opponent last payment then own last payment else if own last payment = λ opponent last payment then own last payment + b else own last payment + 2 b The first eight of these strategies are taken from (Roberts & Sherratt, 1999). Some of them are familiar from Iterative Prisoners Dilemma Game; NA is always deceive, AWD is always cooperate and the others are like Tit For Tat. The strategies in (Killingback & Doebeli, 1998) have the form : if payl(1) then a else if opponent undercut then 0 else own last payment + b. This mixture of AON and RTS is not among our strategies because such an unforgiving strategy is unlikely to have any success. The strategies in (Leimar, 1997) are very different from our s because they try to predict the partner s quality from its payments in previous rounds. In our coevolutionary implementation there was a population of 50 followers and a population of 50 leaders. In each generation every follower

11 (leader) plays at least 5 randomly chosen leaders (followers); the average number of opponents is 10. We separated followers and leaders to see if this distinction has any influence on which strategy that evolves. The parameters of the leaders were: strategy type, a,b; the parameters of the followers were:strategy type,b and the number of iteration rounds. We fixed follower s a to 1 as we wanted to see how the number of iteration rounds would evolve. In (Sherratt & Roberts, 1998) the additional parameter choosiness was introduced. Players can refuse to play next candidate if their previous payoff from the candidate was less than a certain fraction of their current highest payoff but only allowed up to GUT (giving up threshold) times. This paper showed that increasing either GUT or the number of iteration rounds (length of grooming bouts) encouraged altruism and led to higher generosity. The possibilty of making a mistake was introduced in a later paper (Roberts & Sherratt, 1999). Experiments showed that increasing the probability of making an error discourages altruism. In (Doebeli & Knowlton, 1998; Brauchli et al., 1999; Killingback et al, 1999) the influence of spatial structure on reciprocal altruism is studied The authors of (Doebeli & Knowlton, 1998) claim that spatial structure seems to be necessary for altruism to arise when species are different (kl kf).the results in (Brauchli & Killingback, 1999) show that spatial structure not only encourages altruism but it influences the preferred altruistic strategy ( if space is structured then variants of PAVLOV else variants of TITFORTAT). In (Frean, 1996) one of the first papers on varying investment it is shown that this markedly changes the advantages and disadvantages of

12 strategies. In (Wahl & Nowak, 1999a; Wahl & Nowak, 1999b) a particular family of strategies is studied:pay[i] is a linear function of opponents last payment and Pay[i-1] is irrelevant. Dynamic behaviour is very rich in this strategy family; initial investments are critical, altruism sometimes evolves sometimes not, increasing the probability of making an error discourages altruism. For a comprehensive comparison of the factors influencing the evolution of altruism see (Dugatkin, 1998) [Table3.1pp.42-44]. 4 Results To run an experiment one must fix the cost-benefit ratios, k F and k L, choose a variety of strategies and run the simulation for a number of generations. During the simulation not only can the distribution of follower and leader strategies be tracked, but one can also plot the fitness and other parameters of individuals in various ways. Among the plots produced by a typical run are Fig1 a) static fitness b) dynamic fitness

13 Fig2 a) static follower parameters b) dynamic follower parameters Fig3 a) static leader parameters b) dynamic leader parameters This run corresponds to the first two rows in the table1 below and to the cross labelled A in figure 4.

14 Fig.4 Experimental Results-error bars shortened and K-fitness made positive

15 k RTG OSG OCG RTS OSC OC RTL OSL OCL GGG AWD AON NA SC fitness rounds A) B) C) D) E) F) G) H) I) I_) J) J_) K) K_) L) L_) Table 1 Experimental Results - for each experiment upper row gives follower results, lower row gives leader results

16 The last row is not a separate run; it gives the averages over the last 10 generations so it checks that the stability of the final strategy distributions and fitnesses in rowl. Similar checks on the validity of the results are given in the apaendix. The table gives no information on the evolution of strategy parameters because the only patterns observed were initial investment a increases but large variation between 2 and 8 increment b keeps wide variation between 0 and 4 The last column of the table shows that the number of iterations does evolve, increasing appreciably from the initial 25. This is surprising as the number of iterations is determined by the follower, who is unlikely to benefit from more iterations when k F is low. It is satisfying that altruism usually evolves when it should, but it is surprising that the specially designed adjustable payment strategies RTL,OSL and OCL were never the most popular follower or leader strategy when more altruistic strategies were available. We shall see why in the next section. 5 Analysis of Altruism Game Experiments The ideal result of a game model of an ecological situation is that players strategies evolve to evolutionary stable strategies, but this rarely happens with realistic models. Even with such a classical simple game as Hawk-Dove simulations show that players strategies do not evolve to evolutionary stable strategies (Fogel et al, 1997). However one often has selforganising criticality (Bak, 1996) in that the player s strategies evolve to a critical level where each player has a reasonable fitness. The pattern of strategies played varies while at this critical level and more or less serious avalanches happen from time to time; during an avalanche one or more players get low

17 fitness and change their strategies appropriately. Sometimes evolution leads to mediocre stable states which are adequate but far from optimal for all players. Rarely does evolution lead to periodic oscillation or random wandering of strategies. There is some biological evidence for oscillation in lizard mating strategies (Sinervo & Lively, 1997), and some altruism studies show cycling from mild altruism to too-generous altruism to cheating to mild altruism again. Let us investigate whether the mediocre stable states in experiments I, I_ and J_ are because of the tensions between mildly altruistic, too-generously altruistic,and cheating strategies. The evolutionary behaviour of games is complex and depends strongly on the possible strategies available to each player, but analysis and understanding is not impossible. One approach is to consider strategies as negotiation rules (McNamara et al, 1999) and our approach is similar. One can compute the players fitnesses for particular player strategies- see the figures in this section for 14 possible strategies in the recursive altruism game. These computations can form the basis for a Theory of Moves (Brams, 1994; Brams, 1999) TOM-analysis of the evolutionary behaviour of a game. The basic idea of a TOM-analysis is to look for player strategies so no player can benefit by unilaterally changing his strategy because the other players could retaliate by changing their strategies. For our Altruism game we seek strategy pairs (S F,S L ) so that 1) for every follower strategy S F that gives greater follower payoff in the game (S F,S L ) there is leader counterstrategy S L that gives greater leader payoff in the game (S F,S L ) 2) for every leader strategy SL that gives greater leader payoff in the game (S F,S L ) there is follower counterstrategy S F that gives greater follower payoff in the game (S F,S L ).

18 This is an oversimplification of the requirements for a TOM-interesting strategy pair (S F,S L ) but it suffices for an analysis of our experiments. Figures 5 to 10 are plots over the follower and leader payoffs for K L = 2 and various values of K F. These payoffs were calculated for the 14 x 14 possible values of (S F,S L ), assuming a=2, b=1 and 30 rounds of the game. Changing S F corresponds to moving left or right in the figure and changing SL corresponds to moving into or out of the figure. First consider experiment A and figure 5. The table shows that altruism evolves; both followers and leaders choose strategies RTG and OSG. These choices correspond to the left corners of the two fitness plots in fig5. The strategy pair (RTG,RTG) is TOM-interesting; OSG is the second best strategy for both players but there is no TOM- interesting strategy- pair with OSG. The success of OSG in experiment A seems to be due to opportune choices of parameters a, b and number of rounds. fig5 kf = 1.1 kl=2 fitness plots (left) leaders (right) followers Now consider figure 6 and experiments B, G, J and J_. The table shows that altruism evolves- but followers sometimes prefer non altruistic strategies. In experiment B (G,J and J_) the most popular follower choice was

19 OSG (AWD,OC and OCG) and the most popular leader choice was RTG (RTS,RTG and RTG).These choices correspond to the (2,1), (11,4),( 6,1) and (3,1) of the two fitness plots in fig 6. In B evolution leads to the TOM interesting strategy-pair (OSG,RTG) but in the other experiments it leads to strategy-pairs that are far from being TOM-interesting. In G it leads to the mediocre stable state (AWD,RTS) as both players would benefit if followers changed to RTS. In J and J_ all 14 strategies are available, leaders agree on RTG but followers tend to OC in one experiment and OCG in the other. It looks as if the followers are punishing the leaders by not choosing RTG which would be slightly better for followers and very much better for leaders. The leaders are being punished for being too greedy, for not choosing RTL, OCL, or OSL which would give a much fairer altruism. Fig6 KF= 1 kl=2 fitness plots (left) leaders (right) followers Now consider figure 7 and experiments C, H,I and I_. The table shows that altruism evolves, but followers sometimes prefer non altruistic strategies. In experiment C (H,I and I_) the most popular follower choice was GGG (GGG or SC,RTS or OSL and GGG) and the most popular leader choice was RTG (RTG,OSL and OCG).These choices correspond to the (10,1),(14,1),

20 (4,8),(8,8)and(10,3) of the two fitness plots in fig 7. The only TOM-interesting choice is the strategy pair (OSL,OSL) in experiment I. Followers should not be so fond of GGG; in experiment I_ replacing GGG by OSL gives much better payoffs to both followers and leaders. fig 7 kf=0.95 kl=2 fitness plots (left) leaders (right) followers Now consider figure 8 and experiments D,K,K_ and L. For D,K and K_ the table shows that altruism does not evolve - both followers and leaders choose a wide variety of strategies. The most popular follower choices are SC and AWD, the most popular leader choices are the altruistic OSG, RTG (in experiments D and K), OCG (in experiment K) and OSL(in experimentk_). These choices correspond to the (11,1), (14,1),(11,2),(14,2), (11,3) and (14,3) columns of the two fitness plots in fig 9 and none of them are TOMinteresting. Experiments K and K_ give mediocre stable states because leaders become too greedy and the followers defend themselves by choosing a non-altruistic strategy. For experiment L altruism does evolve and the most popular strategies for both followers and leaders are OSL and OCL,corresponding to the (8,8), (8,9), (9,8) and (9,9) columns. These strategy pairs have become TOM-interesting because the greedy altruistic strategies have been excluded.

21 fig8 kf=0.8 kl=2 fitness plots (left) leaders (right) followers Now consider figure 9 and experiment E. The table shows that altruism does not evolve - both followers and leaders choose a wide variety of strategies. The most popular follower choice is SC and the most popular leader choices are OCG,GGG and AWD ; these choices correspond to the (3,14), (10,14) and (11,14) columns of the two fitness plots in fig 9. TOM analysis of this experiment is uninteresting because the leader fitness peaks in the figure correspond to follower fitness depths. fig9 kf=0.5 kl=2 fitness plots (left) leaders (right) followers

22 Now consider figure 10 and experiment F.The table shows that altruism does not evolve - both followers and leaders choose a wide variety of strategies. The most popular follower(leader) choice is SC (OSG) corresponding to the (14,2) columns of the two fitness plots in fig 10. TOM analysis of this experiment is uninteresting because the leader fitness peaks in the figure correspond to follower fitness depths. fig 10 kf=0 kl=2 fitness plots (left) leaders (right) followers 6 Conclusions Our experiments give surprising insights into the advantages and disadvantages of the strategy families we tried: NA never evolved even when it was best SC often evolved ( benefits more than NA in low kf) Subordinants often prefer the greedy strategies to the fair RTL, OSL and OCL so the dominants choose shortchanging strategies OC and OCG. Spatial

23 distribution of interacting individuals encourages the devolopement of altruistic strategies, but it is not a necessary precondition. Our experiments (most clearly experiment L) show that altruism can evolve, even when partners have very different cost-benefit ratios and must make very different investments. They also suggest an explanation of why subordinates should be more altruistic than dominants, even when subordinates are not coerced by dominants. References BAZZAN, A.L.C. et al. (1999) Moral sentiments in multi-agent systems, ECAL99, Springer LNAI BRAUCHLI, K. et al (1999) Evolution of cooperation in spatially structured populations J.Theor.Bio. 200, BOYD, R. (1992) The evolution of reciprocity when conditions vary in Coalitions and alliances in humans and other animals (ed.a.h.harcourt & F.B.M.de Waal) pp ,oxford University Press BRAMS, S.J. (1994) Theory of Moves Cambridge UP BRAMS, S.J. (1999) The rationality of surprise: unstable Nash equilibria and the Theory of Moves Math.Japonica 49, pp BROWN, J.L. (1983) Cooperation-a biologist s dilemma, Advances in the Study of Behaviour 13, pp BERNASCONI, G. & STRASSMANN, J.E. (1999) Cooperation among unrelated individuals: the ant foundress case TREE 14, pp BAK, P. (1996) How Nature works: the science of self-organised criticality. Springer DOEBELI, M. & KNOWLTON, N. (1998) The evolution of interspecific mutualisms. Proc.Nat.Acad.Sci.95, pp DUGATKIN, L.A. (1998) Game theory and cooperation, pp in (Eshel et al. 1999)

24 ESHEL, I. et al. (1999) The emergence of kinship behaviour in structured populations of unrelated individuals. Int.J.Game Theory 28, pp FOGEL, D.B. et al. (1997) On the instability of evolutionary stable strategies. Biosystems 44, pp FREAN, M. (1996) The evolution of degrees of cooperation. J.Theor.Bio.182 pp KILLINGBACK, T. & DOEBELI, M. (1998) Raise the stakes evolves into a defector. Nature 400 p. 518 KILLINGBACK, T. et al. (1999) Variable investment, the continuous prisoner s dilemma, and the origin of cooperation. Proc.R.Soc.Lond.B 266 pp LEIMAR, O. (1997) Reciprocity and communication of partner quality. Proc.R.Soc.Lond.B 264 pp MAY, R.M. (1987) More evolution of cooperation. Nature 327 pp MCNAMARA, J.M. (1999) et al. Incorporating rules for responding into evolutionary games. Nature 40 pp ROBERTS, G. & SHERRATT, T.N. (1998) Developement of cooperative relationships through increasing investment. Nature 394 pp ROBERTS, G. & SHERRATT, T.N. (1999) The emergence of quantitatively responsive cooperative trade. J.Theor.Bio.200 pp SHERRATT, T.N. & ROBERTS, G. (1998) The evolution of generosity and choosiness in cooperative exchanges. J.Theor.Bio.193 pp SINERVO, B & LIVELY, CM (1997) The rock-paper-scissors game and the evolution of alternative male strategies. Nature 380, pp TOMLINSON, I.P.M. & BODMER, W.F. (1997) Modelling the consequences of interactions between tumour cells. British Journal of Cancer 75 pp

25 WAHL, L.M. & NOWAK, M.A. (1999) The continuous prisoner s dilemma: I Linear reactive strategies. J.Theor.Bio.200 pp WAHL, L.M. & NOWAK, M.A. (1999) The continuous prisoner s dilemma: II Linear reactive strategies with noise. J.Theor.Bio.200 pp