arxiv:cond-mat/ v2 [cond-mat.dis-nn] 25 Oct 2003

Transcription

1 arxiv:cond-mat/ v2 [cond-mat.dis-nn] 25 Oct 2003 Replicator dynamical systems and their gradient and Hamiltonian properties A. Prykarpatsky, and V. V. Gafiychuk April 23, 2018 Abstract We consider the general properties of the replicator dynamical system from the standpoint of its evolution and stability. Vector field analysis as well as spectral properties of such system has been studied. Lyaponuv function for investigation of system evolution has been proposed. The generalization of the replicator dynamics for the case of multi-agent systems has been introduced. We propose a new mathematical model to describe the multi-agent interaction in complex system. keywords: replicator dynamics, Hamiltonian systems, gradient dynamical systems, multi-agent interaction, Lyapunov functions, complex systems. 1 Introduction Replicator equations were introduced by Fisher to capture Darwin s notion of thesurvivalofthefittest[1]andreplicatordynamicsisoneofthemostimportant dynamic models arising in biology and ecology [2, 3], evolutionary game theory [4] and economics[5, 6], traffic simulation systems and distributed computing[7] etc. It is derived from the strategies that fare better than average and thriving is on the cost of others at the expense of others (see e.g. [4]). This leads to the fundamental problem of how complex multi-agent systems widely met in nature can adapt to changes in the environment when there is no centralized control in the system. For complex living system such problem has been considered in [8]. By term complex multi-agent systems under consideration we mean one that comes into being, provides for itself, and develops pursuing its own goals [8]. In the same time replicator dynamics arises if the agents have to deal with conflicting goals and the behavior of such systems is quite different from the adaptation problem considered in [8]. In this article we develop a new dynamic Institute of Applied Mathematics at the AGH University of Science and Technology, Cracow, Polland, and Institute for Applied Problems of Mechanics and Mathematics, National Academy of Sciences of Ukraine Institute for Applied Problems of Mechanics and Mathematics, National Academy of Sciences of Ukraine, Naukova St, 3 b, Lviv 79601, Ukraine 1

2 model to describe the multi-agent interaction in complex systems of interacting agents sharing common but limited resources. Based on this model we consider a population composed of n Z + distinct competing varieties with associated fitnesses f i (v), i = 1,n, where v [0,1] is the vector of relative frequencies of the varieties (v 1,v 2,...,v n ). The evolution of relative frequencies is described by the following equations: where i = 1,n, dv i /dt = v i (f i (v) f(v) ), (1) f(v) = v i f i (v). (2) The essence of (1) and (2) is simple: varieties with an above-average fitness will expand, those with a below-average fitness will contract. Since v i [0,1],i = 1,n have to be nonnegative for all time, the system (1), (2) is defined on the nonnegative orthand R n + = {v Rn : v i 0}. (3) The replicator equation describes the relative share dynamics and thus holds on the unit a n-1 dimensional space simplex { } S n = v R n + : v i = 1. (4) Let us write the Fisher s model in the following form dv i /dt = v i a ij v j a jk v j v k, (5) j=1 j,k=1 where a ij R, v i [0,1], i,j = 1,n, n Z +. One can check the system (5) can be written in such a matrix commutative form: Here by definition, dp/dt = [[D;P],P]. (6) P = { } (v i v j ) 1 2 : i,j = 1,n, (7) { D = 1 n } 2 diag a jk v k : j = 1,n. (8) k=1 2

3 It can be checked easily that the matrix P EndE n is a projector in E n, that is P 2 = P for all t R, what appears to be very important for our further studying the structure of vector field (6) on the corresponding projector matrix manifold P [9, 10]. In particular, the expression (6) ensures that there exists the time invariant vector subspace ImP E n in the Euclidian phase vector space E n, necessary for the replicating system under regard to be the information data conserved. 2 Vector field analysis In order to study the structure of the flow (6) on the projector matrix manifold P P let us consider a functional Ψ : P R, where by definition the usual variation δψ(p) := Sp(D(P)δP), (9) with D EndE n, Sp : EndE n R 1 being the standard matrix trace. Taking into account the natural metrics on P, we consider the projection of the usual gradient vector field Ψ to the tangent space T(P) under the following conditions: ϕ(x;p) := Sp(P 2 P,X) = 0, Sp( ϕ, ϕ Ψ) P = 0, (10) holding on P for all X EndE n. The first condition is evidently equivalent to P 2 P = 0, that is P P. Thereby we can formulate such a lemma. Lemma 1. The functional gradient ϕ Ψ(P), P P, at condition (10) has the following commutator representation: ϕ Ψ(P) = [[D,P],P]. (11) Proof. Consider the projection of the usual gradient Ψ(P) to the tangent space T(P) of the manifold P having assumed that P EndE n : ϕ Ψ(P) = Ψ(P) ϕ (Λ,P), (12) where Λ EndE n is some unknown matrix. Taking into account the conditions (10), we find ϕ Ψ(P) = D Λ P(D Λ) (D Λ)P +PD+DP (13) = PD +DP +2PΛP. (14) where we made use the conditions ϕ Ψ(P) = D Λ+PΛ+ΛP) 3

4 and P(D Λ)+(D Λ)P +2PΛP = D Λ. Now one can see from (14) and the second condition in (12), that for all P P, giving rise to the final result PΛP = PDP (15) ϕ Ψ(P) = PD +DP 2PDP, (16) coinciding exactly with commutator (11). It should be noted that the manifold P is also a symplectic Grassmann manifold ([9, 10]), whose canonical symplectic structure is given by the expression: ω (2) (P) := Sp(PdP dpp), (17) where dω (2) (P) = 0 for all P P, and the differential form (17) is nondegenerate [9, 11] upon the tangent space T(P). Let us assume that ξ : P R is an arbitrary smooth function on P. Than Hamiltonian vectorfield X ξ : P T(P) on P generatedby this function relative to the symplectic structure (17) is given as follows: X ξ = [[D ξ,p],p], (18) where D ξ EndE n is a certain matrix. The vector field X ξ : P T(P) generates on P the flow dp/dt = X ξ (P) (19) being defined globally for all t R. This flow by construction is evidently compatible with the condition P 2 = P. This means in particular that X ξ +PX ξ +X ξ P = 0. (20) Thus, we stated that dynamical system (6)being considered on the Grassmann manifold P is Hamiltonian what makes it possible to formulate the following statement. Statement 1. A gradient vector field of the form (18) on Grassmann manifold P is Hamiltonian with respect to the canonical symplectic structure (17) and certain Hamiltonian function ξ : P R, satisfying ξ(p) = [D ξ,p], D ξ = D, where simultaneously ϕ Ψ(P) = X ξ (P) for all P P. Consider now the (n-1)-dimensional Riemannian space { } Mg n 1 = v i R + : i = 1,n, v i = 1 4

5 with the metrics where ds 2 (v) := d 2 Ψ P (v) = g ij (v)dv i dv j P, i,j=1 g ij (v) = 2 Ψ(v) v i v j, i,j = 1,n, v i = 1. Subject to the metrics on Mg n 1 we can calculate the gradient ϕ Ψ of the function Ψ : P R and set on Mϕ n 1 the gradient vector field where v M n 1 ψ, or dv/dt = ψ Ψ(v), (21) v i = 1 is satisfied. Having calculated (21), we can formulate the following statement. Statement 2. The gradient vector fields ϕ Ψ on P and g Ψ on Mg n 1 are equivalent or in another words vector fields and dv/dt = g Ψ(v) (22) dp(v)/dt = [[D(v),P(v)],P(v)] (23) generates the same flow on Mg n 1. As a result from the Hamiltonian property of the vector field ϕ Ψ on the Grassmann manifold P we get a new statement. Statement 3. Thegradientvectorfield g Ψ(21)onthemetricspaceMg n 1 is Hamiltonian subject to the non-degenerate symplectic structure ω (2) g (v) := ω(2) (P) M n 1 g (24) for all v Mg 2m with the Hamiltonian function ξ ψ : Mg 2m R, where ξ ψ := ξ M n 1, ξ : P R is the Hamiltonian function of the vector field X g ξ (18) on P. Otherwise if n Z + is arbitrary our two flows (22) and (23) are on P only Poissonian. 3 Spectral properties Consider the eigenvalue problem for a matrix P P, depending on evolution parameter t R: P(t)f = λf, (25) where f R n is an eigenfunction, λ R is a real eigenvalue P = P, i.e. matrix P P is symmetric. It is seen from expression (22) that specp(t) = {0,1}for 5

6 all t R. Moreover, taking into account the invariance of SpP = 1 we can conclude that only one eigenvalue of the matrix P(t), t R, is equal to 1, all others being equal to zero. So, we can formulate the next lemma. Lemma 2. The image ImP E n of the matrix P(t) P for all t R is k-dimensional k = rankp, and the kernel kerp E n is (n k)-dimensional, where k Z + is constant, not depending on t R. As a consequence of the lemma we establish that at k = 1 there exists a unique vector f 0 E n /(kerp) for which Pf 0 = f 0, f 0 E n /(kerp). (26) Due to the statement abovefor projectorp : E n E n we can write down the following invariant in time expansion in the direct sum of mutually orthogonal subspaces: E n = kerp ImP. Take now f 0 E n satisfying the condition (26). Than in accordance with (22) the next lemma holds. Lemma 3. The vector f 0 E n satisfies the following evolution equation: df 0 /dt = [D(v),P(v)]f 0 +C 0 (t)f 0, (27) where C 0 : R R is a certain function depending on the choice of the vector f 0 ImP. At some value of the vector f 0 ImP we can evidently ensure the condition C 0 0 for all t R n. Moreover one easily observes that for the matrix P(t) P one has [10] the representation P(t) = f 0 f 0, f 0,f 0 = 1, giving rise to the system (5) if f 0 := { ± v j R + : j = 1,n } E n. 4 Lyapunov Function Let us consider gradient vector fields ϕ Ψ on P and g Ψ on Mg n 1. It is easy to state that the function Ψ : P R given by (9) and equal on Mg n 1 following expression n 1 Ψ(v) = a ij v i v j c v i (28) 4 under the condition c R +, i,j=1 v i = 1 being at the same time a Lyapunov function for the vector fields ϕ Ψ on P and g Ψ on M n 1 g. Indeed: 6

7 dψ dt dψ dt = g Ψ, dv dt T(Mg n 1 ) (29) = g Ψ, g Ψ T(M n 1 g ) 0, = Sp(DdP)/dt = Sp(D dp dt ) (30) = Sp(D,[[D,P],P]) = Sp([P,D],[P,D]) = Sp([P,D],[P,D] ) 0 for all t R, where.,. T(M n 1 g ) is the scalar product on T(Mn 1 g ) obtained via the reduction of the scalar product.,. on R n upon T(Mg n 1 ) under the constraint v i = 1. 5 Multi-agent replicator system In the above we have considered the case when rankp(t) = 1, t R. It is naturally to study now the case when n rankp(t) = k > 1 being evidently constant for all t R too. This means therefore that there exists some orthonormal vectors f α ImP, α = 1,k, that for all t R. Put now f α := P(t) = { ± f α (t) f α (t) (31) } v (α) i R : i = 1,n ImP, v (α) i = 1 for all α = 1,k. The necessary orthonormality condition < f α,f β >= δ αβ, α,β = 1,k, can be automatically satisfied if one put f α,i := (exph) i,α, i = 1,n, for a generalskew-symmetricmatrixh = h intheeuclidianvectorspacee n.then, evidently, all signs at vector components ± v α i = f α,i,i = 1,n will be defined exactly for each α = 1,k. As a simple consequence of the above representation for stable vector f α E n,α = 1,k one derives that in the multiagent case one can not write down the resulting replicator dynamics equations in the terms of positive concentration frequencies. The corresponding gradient flow on P then takes modified commutator form with the Lyapunov type function variation δψ : P R as follows: δψ = Sp(D α δp (α) ), (32) where by definition Dα = D α ende n, and P (α) = f α (t) f α (t), P (α) P (β) = P (α) δ αβ, α,β = 1,k, P (α) = P. The resulting flow on P is given as dp/dt = ϕ Ψ(P), (33) 7

8 where the gradient ϕ : D(P) T(P) is calculated taking into account the set of natural constraints: ϕ α,β (X;P) := Sp[X α,β (P α P β δ α,β P β )] = 0 (34) for every α β = 1,k. Assume now that k = rankp(t) for all t R as was stated before. Then the gradient flow (33) brings about dp α /dt = [[D α,p α ],P α ] β=1, β α (P α D α P β +P β D α P α ). (35) The flow (35) one can also obtain as a Hamiltonian one with respect to the following canonical symplectic structure on P: ω (2) (P) = Sp(P α dp α dp α P α ) (36) with an element P = k P α P, natural constraints P α P β = P α δ αβ, α,β = 1,k, and some Hamiltonian function H D(P) which has to be found making use of the expression i ϕ Ψω (2) = dh. (37) Straightforward but tedious calculations of (37) give rise to the same expression (35). The system of equations (35) is a natural generalization of the replicator dynamics for description of a multi-agent interaction. They can describe for example economic communities trying to adapt to a changing environment. In this approach agents update their behavior in order to get maximum payoff under the given matrices of strategies {a (α) ik } in response to the information received from other agents. The structure of the system (35) is quite different from the system of equations (5). The first term on the right hand side describes the individual evolution of each economic agent α in accordance to its own independent replicator dynamics and the second term describes average payoffs of all others k agents except α. References [1] Fisher, R. A., The Genetical theory of natural selection, 1930, Oxford: Clarendon Press. [2] Camazine, S. et al eds., Self-Organization in Biological Systems, 2001 Princeton University Press. [3] Michod, R.E., Darwinian Dynamics, 1999, Princeton University Press, New Jersey. 8

9 [4] Hofbauer, J. and Sigmund, K. Evolutionary games and population dynamics, Cambridge University Press [5] Young, H. P. Individual strategy and Social Structure, 1998 Princeton University Press. [6] Frank, S.A., Foundations of Social Evolution, 1998, Princeton University Press. [7] Nagel, K. Distributed intelligence in large scale traffic simulations on parallel computers. intelligence.ps.gz [8] Lubashevsky, I.A., Gafiychuk, V.V. Cooperative mechanism of selfregulation in hierarchical living systems SIAM J. Appl. Math. Vol. 60, # 2, pp [9] Prykarpatsky, A.K., Mykytiuk, I.V. Algebraic integrability of dynamical systems on manifolds., Kluwer Academic Publisher, the Netherlands, [10] Prykarpatsky, A.K. Gentosh, O. On some class of gradient dynamical systems associated with a polynomial discrete probability distribution. Math. Methods & Phys. Mech. Fields, 2003 (in print, in Ukrainian). [11] Prykarpatsky, A.K., Zagrodzinski, J.A. and Blackmore, D.L. Lax type flows on Grassmann manifolds and dual momentum mappings. Report on Math. Physics,