Non-eterministic Social Laws Michael H. Coen MIT Artificial Intelligence Lab 55 Technology Square Cambrige, MA 09 mhcoen@ai.mit.eu Abstract The paper generalizes the notion of a social law, the founation of the theory of artificial social systems evelope for coorinating Multi-Agent Systems. In an artificial social system, its constituent agents are given a common social law to obey an are free to act within the confines it legislates, which are carefully esigne to avoi inter-agent conflict an ealock. In this paper, we argue that this framework can be overly restrictive in that social laws iniscriminately apply to all istributions of agent behavior, even when the probability of conflicting conitions arising is acceptably small. We efine the notion of a non-eterministic social law applicable to a family of probability istributions that escribe the expecte behaviors of a system s agents. We emonstrate that taking these istributions into account can lea to the formulation of more efficient social laws an the algorithms that ahere to them. We illustrate our approach with a traffic omain problem an emonstrate its utility through an extensive series of simulations. Introuction Agents esigne to exist in multi-agent systems in general cannot affor to be oblivious to the presence of other agents in their environment. The very notion of a multiagent system presupposes that agents, for better or worse, will have some impact on each other. A central problem in Distribute AI (DAI) has been to evelop strategies for coorinating the behaviors of these agents perhaps to cooperatively maximize some measure of the system s global utility or conversely, to insure that non-cooperative agents fin some acceptable way to peacefully coexist. Many coorination strategies have been evelope for managing multi-agent systems (MAS). One axis on which we can contrast these ifferent approaches is the egree of agent autonomy they suppose. For example, a centralize planning system [5] might globally synchronize each agent s activities in avance, taking pains to insure that conflict is avoie among them. Agents can then blinly follow these centrally arrange plans without further consieration. Alternatively, at the other en of the Copyright 000, American Association for Artificial Intelligence (www.aaai.org). All rights reserve. spectrum, agents can be wholly autonomous an pursue their iniviual goals without relying on any centralize control mechanism. In the event conflict arises among them, preformulate rules of encounter allow the agents to ynamically negotiate among themselves to resolve it []. An intermeiary approach between these extremes has been explore in the evelopment of artificial social systems [,], whose workings shoul feel familiar to anyone living in a civilize country. In an artificial social system, its constituent agents are given a common social law to obey an are free to act within the confines it legislates. A social law is explicitly esigne to prevent conflict an ealock among the agents; however, for it to be eeme useful, it shoul simultaneously allow each agent to achieve its iniviual set of goals. Thus, esigning a social law is something of a balancing act. It must be sufficiently strict to prevent conflict or ealock, an simultaneously, it must be sufficiently liberal to allow the agents to efficiently achieve their goals. Useful social laws can be esigne that not only avoi inter-agent conflict but also minimize the use of energy, time, an other resources appropriate to the problem omain. Fitoussi has examine an extension of this theory involving minimal social laws. These are social laws that minimize the set of restrictions place upon the agents, while still avoiing inter-agent conflicts. Minimal social laws allow agents to have maximum flexibility uring action selection by only isallowing those activities that woul prevent other agents from obtaining their goals; thus, they are minimally restrictive. We propose here that social laws, incluing even the minimal type escribe above, can be overly restrictive because agents must ahere to them in all circumstances even where the possibility of conflict with other agents is extremely low. By insisting that agents avoi any chance of conflict or ealock when these circumstances are highly unlikely, even minimal social laws may sometimes be overly restrictive an thereby, inherently inefficient. We will refer to this property of a social law as it being eterministic. Consier, for example, a omain consisting of a gri traverse by a group of mobile agents. A eterministic social law for this omain might institute traffic regulations to insure that agents never collie or get stuck an to be useful, it woul also allow the agents to In roceeings of the Seventeenth National Conference on Artificial Intelligence (AAAI'000), pp. 5-. Austin, Texas. 000.
reach whichever noes they neee. However, being eterministic, this social law woul be equally applicable to all istributions escribing how agents select noes to visit an how they travel between them. In this paper, we examine how knowlege of the probability istributions governing agent behavior in MAS can be applie towars more efficiently coorinating them through a non-eterministic social law. For example, in the omain above, knowing (or learning) that the agents ten to uniformly select noes in the gri to visit can rastically improve our ability to coorinate their movement. We note that from a social engineering perspective, this might appear somewhat counter-intuitive. Much of the work in artificial social systems has been motivate through analogy with how human societies function. We institute laws that govern iniviual behavior an thereby benefit the community as a whole. However, the analogy between agents an people must not be taken too far. For example, vehicular traffic laws in human society nee to be easily remembere, an are thus rarely specific to particular istributions an flows of traffic. They are even less likely to be change ynamically to reflect learne observations. Instea, epiphenomenal approaches are use: highways are constructe that implicitly reirect vehicles, signal light intervals are ynamically varie, an traffic reports are broacast via raio all of these are centralize mechanisms to reuce both congestion an the cognitive buren on human rivers. Traffic regulations themselves are essentially inviolate an for goo reason people woul fin it ifficult to rive safely otherwise. However, agents o not share this limitation. There is nothing inherently worrisome in optimizing social laws to better fit the particular MAS they are intene to govern. We woul like to clarify a point that has been somewhat unclear in the social law literature regaring the efficiency of social laws. Social laws are not algorithms they o not provie a metho for accomplishing a particular task. Rather, they are guielines that specify a class of vali algorithms (or strategies) for solving problems from a particular omain by partitioning the set of possible algorithms into law-abiing an criminal sets. Social laws are thus not necessarily instructive. Just as traffic laws in human society o not provie irections but simply legislate certain types of behavior in particular situations, social laws maintain a set of constraints that simplify writing an reasoning about algorithms. Therefore, it is not obviously meaningful to speak about a particular social law s efficiency. Instea, what shoul be consiere are the computational an other costs of the best-known algorithms the social law makes realizable. We may then refer to a social law s efficiency solely in this regar. However, others o not always clearly make this iscrimination, particularly with social laws so highly constraine an algorithmically formulate they blur the paraigmatic istinction. In referring to their work, we will sometimes fin it convenient to ignore this istinction as well. More generally, we will efine the notion of a noneterministic social law as one that oes not guarantee it is useful in the technical sense given above, although it is highly likely to be for its expecte istribution of agent goals an behaviors. We will call non-eterministic social algorithms the algorithms that ahere to these laws an only present expecte efficiency results regaring them. In the next section, we iscuss the importance of unerstaning the expecte behaviors of the agents an not simply their goal spaces while formulating social laws. After this, we examine a traffic omain originally presente in []. We formulate a non-eterministic social law for it that is more efficient than its eterministic counterpart. We then present extensive simulation results that emonstrate the efficacy of our approach. Using Distribution Information Social laws in MAS o not always provie sufficient information to write efficient control algorithms for the agents. This is not necessarily a limitation of social laws per se. However, it inicates the importance of unerstaning the expecte behavior of the agents as a group somewhere in the system s coorination mechanism, whether it be irectly incorporate into the system s social laws as we argue in the next section, or instea, into the actual control algorithms for its agents. Even though we are investigating a coorination paraigm that has no centralize controller, there is no reason to insist that iniviual agents have no knowlege of their expecte group behavior. To better unerstan this point, it will be useful to first make explicit the role of social laws from a programmer s perspective. A social law for a multi-agent system is esigne to give its agents some measure of autonomy an self-government. While it is essential that each agent follow the law, it is of no concern what the agent actually oes as long as all of its activities are legal. In other wors, social laws make no recommenations as to how agents shoul spen their time; they simply insure the agents o not unuly interfere with each other. Formulating a useful social law is computationally emaning, an even etermining whether one exists for a MAS is in general N-complete []. Therefore, the evelopment of a social law is taken to be an offline practice. However, once a social law is formulate, it can be repeately use without further computational expense. This may be contraste with negotiation protocols, in which computational effort goes into both formulating a protocol an then subsequently negotiating accoring to it each time is it employe. After a social law is esigne for a system, it is supplie to the agents programmers, who are then responsible for implementing control algorithms for the agents that obey it. However, without more information about the expecte behavior of the other agents in the system, this may be quite ifficult to o efficiently. This is because a social law inicates which set of actions is legal in any encountere situation without proviing guiance for selecting among
the legal alternatives. It can be ifficult o so without aitional information. For example, consier the following omain, taken from [], in which m agents synchronously travel circularly aroun an n-noe ring, with noes clockwise labele from n. At each time step, an agent can move to either of its two neighboring noes or it can remain immobile. A minimal social law presente in [] that permitte these agents to travel was: () Staying immobile is forbien if the noe that can be reache by a single counterclockwise movement is occupie. () Moving counterclockwise is allowe only if the two noes that woul be encountere by moving counterclockwise twice are free. While this social law provies a framework that guarantees two agents cannot collie or ealock, it oes not provie any practical guiance for how to actually move agents aroun the ring. If an agent is on noe k an wishes to travel to noe j, j>k, shoul it take a clockwise or counterclockwise path? Supposing j k < n /, the agent shoul clearly move clockwise. However, if j k > n /, it is not obvious which irection is best without knowing both the current value of m an how other agents ten to move (or stay immobile) on the ring. If the agent trie to reach noe j but was blocke k steps along the path between them, it might then have to travel clockwise aroun the ring to j, thereby incurring a k penalty for its unsuccessful counterclockwise attempt. The primary question here is how far an agent can expect to move counterclockwise without being blocke by another agent. Although we o not further analyze this problem here, this simple example makes clear the nee for an iniviual agent to have available more information than that provie irectly by its social law or sensory capabilities. The Multi-Agent Gri System We now examine the multi-agent traffic omain presente in [], which we will refer to as the Gri System. This omain consists of an nxn gri that is traverse by m mobile agents (e.g., robots), as shown in Figure. The rows an columns of the gri form lanes that the agents can navigate. We assume that time is iscrete an the system is synchronous, so at every time step, each agent is locate at some gri coorinate. In this system, agents are given goals in the form of gri coorinates to which they must navigate. Every time an agent reaches its estination, it receives a new goal to visit. For example, the agents might be transporting goos in a warehouse an are alternatively picking up an ropping off items. (We note such systems are currently in frequent commercial use.) The main consieration here is how to insure that the agents o not collie while they navigate the gri. For example, the most naïve strategy woul simply snake the agents in a Hamiltonian cycle aroun the gri, as shown in 7 5 0 5 7 Figure An 8x8 gri with three agents that will travel along the inicate paths. Notice that in three time steps, agents & will collie at coorinate (,) Figure A. Each trip between any two noes woul take O( n ) time units to complete, which oes not compare favorably with the O(n) steps an agent woul take to make the same trip in isolation i.e., with no other agents present. Because this omain oes not have cooperative goals ones that agents work together to achieve the time an agent woul take to complete a task in isolation is the optimum that any social law coul achieve. In [], a complex, eterministic social law is presente for the Gri System that guarantees agents can achieve their goals within certain time bouns. This law requires that agents only use certain rows an columns in the gri for long istance travel, much like we use a highway. When an agent reaches the neighborhoo of its goal, it then travels to it irectly along the local gri, as illustrate in Figure B. Summarizing their results, an agent that can achieve a goal in time t in isolation can achieve it using their social law in time t + n + o( n), assuming m = O( n) an m << n. For the case where m n, a variant of this law provies that each goal can be achieve in n time steps. It is helpful to keep in min that with m = O( n), the gri is very sparsely populate. For example, if n = 00, a gri containing 0,000 locations woul have on the orer of 0 agents moving on it. We are intereste in answering the following questions: by insuring generality, is the eterministic social law framework overly constraining? How can its assumptions be loosene in orer to achieve a more efficient coorination system? Can we both increase the number of agents travelling on the gri an simultaneously ecrease the amount of time they take to reach their goals? The Uniform Gri System In this section, we consier the gri system presente above uner a particular probability istribution escribing an agent s goal selection. Namely, we will assume that the goals are uniformly istribute over all points (x, y) on the gri:
r [( x, y) is a goal] =, 0 x, y < n It is important to note that this assumption will certainly not always be vali, an the non-eterministic social law we present here is not intene for systems where it is not. However, for MAS with agents escribe by this istribution, we can obtain far more efficient results than those in []. Towars etermining a lower boun for the noneterministic social law s efficiency, we first etermine the expecte istance between two ranomly selecte integral coorinates, which we call, on a line from [0, n-] inclusive: n ( n i) n E( ) = i r( = i) = i = n n i= n n i= (A) (C) (B) (D) On a two-imensional gri, the expecte istance between a pair of successive goals will be E ( ), because the total istance will be the sum of the istances along each axis inepenently. We will call this value the isolation time, enote by ; it is the expecte travel time between goals for an unconstraine, isolate agent. It is therefore also a lower boun on the time taken by any social law governing the uniform gri system. Our goal is to formulate a noneterministic social law that approaches this lower boun as closely as possible. Our approach will be to essentially allow the agents to move as they woul in isolation. They will explicitly check to make sure their moves are safe, an take corrective action if necessary. We assume that each agent has sufficient sensory capabilities to realize that other agents are in its immeiate vicinity, i.e. up to steps away. In the event a transition between noes woul cause a collision, an agent simply waits to try again on the next move. If an agent is blocke for an extene perio along its path, the social law requires that it formulate some alternate route to its estination. articularly important in this case is ensuring that the ealock recovery mechanism maintains the assume probability istribution escribing the agents movements through the gri. Notice that this approach oes not guarantee ealock will be avoie. It is possible (however unlikely) that two agents heae in opposite irections along a column or row can inefinitely block one another, even after repeately trying alternate paths to their estinations. In practice, we might try to etect such situations an formulate rules of encounter to avoi them. However, in tens of millions of simulation runs, non-recoverable ealock has never been encountere. Nonetheless, the non-eterministic social law shown below, which we call law Traffic Law U (for uniform), is not guarantee in its present form to be useful in the technical sense efine in the introuction: Traffic Law U Figure Different strategies an agent might use to traverse the gri: (A) Walk a Hamiltonian cycle; (B) Navigate through course gri until reaching appropriate neighborhoo an then use fine gri; (C) Take a minimum length path between points; (D) Loop in a figure 8 path through gri. Circles in the above figures represent goals. ) At step i+, an agent may not move to a spot occupie by another agent at step i. ) If more than one agent simultaneously wants to move to a coorinate, only one, chosen at ranom, is permitte to o so. The rest must remain where they are an wait one turn before trying again. ) If an agent has remaine immobile for more than k turns because its path has been blocke, it must pick another route to its goal. We will refer to the conition of rule of this law as a collision an the requirement of rule as rerouting. Rule in the above law is a conservative measure that prevents an agent from moving to a spot most recently occupie by another agent. While technically unnecessary, it allows us to avoi the nightmare of inter-agent communication an coorination that woul be necessary for moving an immeiately ajacent queue of agents simultaneously. Notice that the Traffic Law U leaves the precise strategy for picking an alternate route in rule unspecifie. Any particular implementation of a non-eterministic social algorithm that aheres to Traffic Law U will have to pick some mechanism for selecting this alternative route. This coul, for example, involve ynamic negotiation between the agents, ranom selection, or some other strategy. Below, we examine a metho that ranomly picks an
intermeiate goal to visit along the way to the agent s actual estination in case it gets stuck somewhere. We efine an L-path as a path between two gri points that contains at most one turn (i.e., change of irection), so calle because of its resemblance to the letter L. (See Figure (c).) oints not on the same row or column will have two L-paths connecting them. Otherwise, there will be only one. A route is efine to be a sequence of L- paths. We now give a non-eterministic social algorithm that aheres to Traffic Law U: Algorithm :. Select a new goal g.. Let be a ranom L-ath from current position to g.. Set route R =. Move along route R towars g, following rules an of Traffic Law U. 5. If blocke for more than k steps, o the following: a. Ranomly select new intermeiary goal g b. Let be a ranom L-path from current position to g c. Let be a ranom L-path from g to g.. Set R =, e. Go to step.. Upon reaching goal g, go to step. The insistence that agents travel along L-paths is well motivate for maintaining the assume istribution of agents in the gri. For example, were the agents to travel along ranom paths (i.e., completely shuffle L-paths), this woul inuce a normal istribution of the agents, more heavily favoring the center region of the gri an leaing to higher numbers of collisions an rerouting. L-paths are to be preferre because they more uniformly istribute the agents an thus, make collisions far less likely. Furthermore, assuming that turning mobile robots requires greater energy than moving them in a straight irection an aitionally interferes with ea-reckoning location strategies by introucing aitional uncertainty, L-paths are to be preferre for practical, non-istribution specific reasons as well. Analysis The efficiency of this algorithm is strictly etermine by the number of collisions an amount of rerouting an agent has to o. In the absence of these, each agent woul achieve optimal time, because the L-path to its goal is a shortest length route to it. However, in the presence of other agents, both collisions an rerouting are inevitable an can incur prohibitive time penalties. With respect to each agent, a collision has cost because of the incurre elay. Rerouting has cost of at least, because ealock may occur uring the rerouting process itself. However, there is no recursive rerouting the agent simply reroutes with respect to the original goal, not the intermeiary selecte in step (5a) of the algorithm. We will first provie a loose upper boun to the expecte running time for an agent to travel between successive goals on an nxn gri containing m agents. We use this to etermine how many agents can be allowe on the gri simultaneously given how much overhea (i.e. waste travel time) is acceptable. We then present extensive simulation results for Algorithm, ue to the ifficultly of obtaining tighter bouns for its running time. Analytic Results To etermine how many agents can simultaneously traverse the gri without incurring unreasonable elays ue to congestion, we approximately moel an agent s movement through the Uniform Gri System as if it were governe by a negative binomial istribution. This approximation will become increasingly inaccurate in systems where the gri is more heavily congeste, in which case we must turn to the simulation results given below. For Algorithm, we boun E t, the expecte travel time between goals as: Et E( time moving towars goal ) + E( time recovering from ealock ) We efine the probabilities of colliing an successful transitions as c an s respectively: m =, = n c s c Note that c woul be seem to be ouble the given value but we assume that half the time an agent is involve in a potential collision, it is the one selecte to move per rule of Traffic Law U, an no time penalty is thereby incurre. We boun the probability of ealock by consiering that it occurs only when agents collie an then subsequently block each other. Separately accounting for interior an borer regions, we have: ( n n) n c c c c = + = + c n n n Recall, the isolation time, is given by: n = E( ) = n / n We calculate E goal, the expecte time an agent spens moving towars its goal using our negative binomial istribution assumption: E goal = s We etermine E ealock, the expecte time an agent spens recovering from ealocks, explicitly noting that the agent may ealock in the mist of ealock recovery:
Eealock ( G + ( G + ( G +...)...) s = s We then have the expecte time between successive goals is: Et + s s Next, we efine c *, the ratio between the expecte an isolation times when traveling between successive goals. It is a measure of the overhea ue to agent interaction while traversing the gri: * E / t c c = + + s s c c c / Recalling the above efinition of c, we solve for the number of agents m as a function of c* an n: * * c c m ( n ) + = ( n ) + * * c + G c + n We now have a hanle on how many agents can be allowe onto an nxn gri given some level of acceptable overhea c *. For example, on a 00x00 gri, if it is acceptable for an agent to spen. times longer between successive goals than it woul on the gri alone, then we expect that roughly 87 agents can be permitte onto the gri simultaneously. Note that this is actually an unerestimate because of the non-tight boun for E t etermine above. The actual number emonstrate in simulation for c * =. is m=n, or in this case, m=00. Simulation Results A Java-base simulator was written for the Uniform Gri System employing Traffic Law U an Algorithm. Our approach for each gri of size n was to slowly increase the number of agents, m, observing how this impacte the average time of an agent to achieve its goals. We first consier the case where m = c n. As expecte, the time taken for an agent to achieve its goals on average is essentially equal to its isolation time. Tables an contain the cases for c = an 0 respectively. Each simulation was run until the agents globally achieve 0,000 goals. In the table: #S represents the number of time steps simulate; C is the total collision penalty for the simulation; R is the total rerouting penalty; Avg is the average time an agent took to achieve a goal; is the time an agent woul ieally take in isolation; c* is Avg/; an %+ is 00x(Avg )/. We note that lower c* values are better, an a value of is the best that can be achieve by any social law in this omain. We then examine cases where m = cn, where c < n. As c approaches n, the ensity of the agents increases to the point where they become hopelessly crowe, an navigation becomes extraorinarily inefficient. As this happens, it becomes more efficient to simply snake the agents aroun the gri in a Hamiltonian cycle as escribe above. Graphs an isplay the rate of change in c* (=Avg/) as a function of c (=m/n) for n=0 an 00 respectively. Finally, we examine our results for the case where c = (m=n), where we fin that empirically, c* is roughly aroun / for all values of n. Table : m = n n m #S C R Avg c* %+ 0 78 78 509 7...08 7.85 0 5 5 757.78..0.59 00 0 589 9 755 5.89. 0.99 -. 00 977 70...0. 500 508 59 9.58..00 0.7 000 5 7 8.8..0.55 Table : m = 0 n n m #S C R Avg c* %+ 0 0 7 5 558.7.0.9.9 0 0 5 5 88..0. 0. 50 70 590 708 775... 0. 00 00 7 79 7... 0. 500 0 75 00 85... 0. 000 0 7 0 557 8..7.5 0.5 Table : m = n n #S C R Avg c* %+ 0 878 758 977 8.78.0. 0. 0 8 895.9.0.7 0.7 50 85 09 0.7.. 0. 00 79 90 9 78.9..8 0.8 500 85 7 859.09.. 0. 000 9 9 5955 99..7.9 0.9 Comparison of results Algorithm is near optimal in the logarithmic cases shown in Tables an, where m = c n. Only in the case
Multiples of isolation time = c* where c m / oes the performance egrae Multiples of isolation time = c* 0 Graph : Gri System with n=0 8 5 0 0 8 5 Multiples of n agents, m=xn Graph : Gri System with n = 00 0 0 5 0 5 0 Multiples of n agents, m=xn substantially. When m = cn, we observe a near constant multiplicative cost of approximately. for c =. As c starts to increase, we note the expecte penalty observe in the average time it takes an agent to reach its goal. Finally, as c approaches n itself, the number of agents approaches n, an it woul be best to ynamically switch to the Hamiltonian path strategy. In the table below, we compare the expecte time for an agent to reach its goal in our approach an the one taken in []: Table : Comparison of non-eterministic Algorithm with the eterministic social law presente in []: Expecte Time to Goal m = Non-Deterministic Deterministic n (=n/) c, c> 0 n Approaches c, c> n.=n/5 n cn See graphs Not applicable encounter. In particular, we argue that knowlege of the unerlying istributions escribing agent behavior can give us new ways of coorinating MAS an help us formulate more efficient social laws. We emonstrate this by revisiting a previously stuie traffic omain problem. By assuming a particular istribution of both agent goals an their ealock recovery behavior, we were able to formulate a simple an more efficient strategy for coorinating the movement of agents throughout the gri. Future work in this omain inclues more precisely characterizing the runtime complexity of Algorithm, exploring how well the system works when face with other istributions, i.e., how sensitive this formulation is to the actual encountere behavior, an exploring other coorination omains that might be amenable to this approach. Acknowlegements This material is base upon work supporte by the Avance Research rojects Agency of the Department of Defense uner contract number F00 9 C 00, monitore through Rome Laboratory. Special thanks to D. Fitoussi an L. Weisman. References [] Fitoussi, D. an Tennenholtz, M. Minimal Social Laws. In roc. Of the Fifteenth National Conference on Artificial Intelligence, p-. 998. [] Moses, Y., an Tennenholtz, M. Artificial Social Systems. Computers an Artificial Intelligence. ():5-5. [] Rosenschein, J.S., an Zlotkin, G. Rules of Encounter: Design Conventions for Automate Negotiation among Computers. MIT ress. 99. [] Shoham, Y., an Tennenholtz, M. Social Laws for Artificial Agent Societies: Off-line Design. Artificial Intelligence 7. 995. [5] Stuart, C. An Implementation of a Multi-Agent lan Synchronizer. In roc. Ninth International Joint Conference on Artificial Intelligence. 985. Conclusions In this paper, we propose that general purpose, eterministic social laws appropriate for all circumstances may be inappropriate for the situations MAS actually