arxiv: v2 [cond-mat.stat-mech] 14 Nov 2008

Transcription

1 Random Booean Networks Barbara Drosse Institute of Condensed Matter Physics, Darmstadt University of Technoogy, Hochschustraße 6, Darmstadt, Germany (Dated: June 27) arxiv:76.335v2 [cond-mat.stat-mech] 4 Nov 28 This review expains in a sef-contained way the properties of random Booean networks and their attractors, with a specia focus on critica networks. Using sma exampe networks, anaytica cacuations, phenomenoogica arguments, and probems to sove, the basic concepts are introduced and important resuts concerning phase diagrams, numbers of reevant nodes and attractor properties are derived PACS numbers:. INTRODUCTION Random Booean networks (RBNs) were introduced in 969 by S. Kauffman [, 2] as a simpe mode for gene reguatory networks. Each gene was represented by a node that has two possibe states, on (corresponding to a gene that is being transcribed) and off (corresponding to a gene that is not being transcribed). There are atogether N nodes, and each node receives input from K randomy chosen nodes, which represent the genes that contro the considered gene. Furthermore, each node is assigned an update function that prescribes the state of the node in the next time step, given the state of its input nodes. This update function is chosen from the set of a possibe update functions according to some probabiity distribution. Starting from some initia configuration, the states of a nodes of the network are updated in parae. Since configuration space is finite and since dynamics is deterministic, the system must eventuay return to a configuration that it has had before, and from then on it repeats the same sequence of configurations periodicay: it is on an attractor. S. Kauffman focussed his interest on critica networks, which are at the boundary between frozen networks with ony very short attractors and chaotic networks with attractors that may incude a finite proportion of state space. He equated attractors with ce types. Since each ce contains the same DNA (i.e., the same network), ces can ony differ by the pattern of gene activity. Based on resuts of computer simuations for the network sizes possibe at that time, S. Kauffman found that the mean number of attractors in critica networks with K = 2 inputs per node increases as N. This finding was very satisfying, since the bioogica data avaiabe at that time for various species indicated that the number of ce types is proportiona to the square root of the number of genes. This woud mean that the very simpe mode of RBNs with its random wiring and its random assignment of update functions dispays the same scaing aws as the more compex reaity. The concept of universaity, famiar from equiibrium critica phenomena, appeared to work aso for this cass of nonequiibrium systems. Kauffman found aso that the mean ength of attractors increases as N. Today we know that the bioogica data and the computer simuation data are both incorrect. The sequencing of entire genomes in recent years reveaed that the number of genes is not proportiona to the mass of DNA (as was assumed at that time), but much smaer for higher organisms. The square-root aw for attractor numbers and engths in RBNs survived unti RBNs were studied with much more powerfu computers. Then it was found that for arger N the apparent square-root aw does not hod any more, but that the increase with system size is faster. The numerica work was compemented by severa beautifu anaytica papers, and today we know that the attractor number and ength of K = 2 networks increases with network size faster than any power aw. We aso know that, whie attractor numbers do not obey power aws, other properties of critica RBNs do obey power aws. It is the purpose of this review to expain in an understandabe and sef-contained way the properties of RBNs and their attractors, with a specia focus on critica networks. To this aim, this review contains exampes, short cacuations, phenomenoogica arguments, and probems to sove. Long cacuations and pots of computer simuation data were not incuded and are not necessary for the understanding of the arguments. The readers wi aso benefit from consuting the review [3], which, whie not containing the more recent findings, covers many important topics reated to Booean networks. Booean networks are used not ony to mode gene reguation networks, but aso neura networks, socia networks, and protein interaction networks. The structure of a these networks is different from RBNs with their random wiring and random assignment of update functions, and with the same number of inputs for every node. Nevertheess, understanding RBNs is a first and important step on our way to understanding the more compex rea networks. 2. MODEL A random Booean network is specified by its topoogy and its dynamica rues. The topoogy is given by the nodes and the inks between these nodes. The inks are directed, i.e., they have an arrow pointing from a node to those nodes that it infuences. The dynamica rues describe how the states of the nodes change with time.

2 2 The state of each node is on or off, and it is determined by the state of the nodes that have inks to it (i.e., that are its inputs). In the foowing, we first describe the topoogy, and then the dynamics of RBNs. A. Topoogy For a given number N of nodes and a given number K of inputs per node, a RBN is constructed by choosing the K inputs of each node at random among a nodes. If we construct a sufficienty arge number of networks in this way, we generate an ensembe of networks. In this ensembe, a possibe topoogies occur, but their statistica weights are usuay different. Let us consider the simpest possibe exampe, N = 2 and K =, shown in Figure 2 A. There are 3 possibe topoogies. Topoo- TABLE I: The 4 update functions for nodes with input. The first coumn ists the 2 possibe states of the input, the other coumns represent one update function each, faing into two casses. In F R TABLE II: The 6 update functions for nodes with 2 inputs. The first coumn ists the 4 possibe states of the two inputs, the other coumns represent one update function each, faing into four casses. In F C C 2 R (a) (b) (c) FIG. : The possibe topoogies for N = 2 and K =. gies (a) and (b) have each the statistica weight /4 in the ensembe, since each of the inks is connected in the given way with probabiity /2. Topoogy (c) has the weight /2, since there are two possibiities for reaizing this topoogy: either of the two nodes can be the one with the sef-ink. Whie the number of inputs of each node is fixed by the parameter K, the number of outputs (i.e. of outgoing inks) varies between the nodes. The mean number of outputs must be K, since there must be in tota the same number of outputs as inputs. A given node becomes the input of each of the N nodes with probabiity K/N. In the thermodynamic imit N the probabiity distribution of the number of outputs is therefore a Poisson distribution P out (k) = Kk k! e K. () B. Update functions Next, et us specify the dynamics of the networks. Each node can be in the state σ i = ( on ) or in the state σ i = ( off ), where i is the index of the node. The N nodes of the network can therefore together assume 2 N different states. An update function specifies the state of a node in the next time step, given the state of its K inputs at the present time step. Since each of the K inputs of a node can be on or off, there are M = 2 K possibe input states. The update function has to specify the new state of a node for each of these input states. Consequenty, there are 2 M different update functions. Tabe I ists the 4 possibe update functions for K =. The first two functions are constant, or frozen, i.e. the state of the node is independent of its inputs. The other two functions change whenever an input changes, i.e., they are reversibe. The third function is the copy function, the fourth is the invert function. Tabe II ists the 6 possibe update functions for K = 2. There are again two constant and two reversibe functions. Furthermore, there are canaizing functions. A function is canayzing if at east for one vaue of one of its inputs the output is fixed, irrespective of the vaues of the other inputs. The first cass of canayzing functions do not depend at a on one of the two inputs. They simpy copy or invert the vaue of one of the inputs. In Tabe II, these are the C functions. The second cass of canayzing functions has three times a or three times a in its output (the C 2 functions). For each of the two inputs there exists one vaue that fixes the output irrespective of the other input. In fact, constant functions can aso be considered as canayzing functions, because the output is fixed for any vaue of the inputs. Each node in the network is assigned an update function by randomy choosing the function from a possibe functions with K inputs according to some probabiity distribution. The simpest probabiity distribution is a constant one. For K = 2 networks, each function is then chosen with probabiity /6. In the previous section, we have introduced the concept an ensembe of networks. If we are ony interested in topoogy, an ensembe is defined by the vaues of N and K. When we want to study network dynamics, we have to assign update functions to each network, and the ensembe needs to be specified

3 3 by aso indicating which probabiity distribution of the update functions sha be used. If a 4 update functions are aowed, there are 36 different networks in the ensembe shown in Figure 2A. For topoogies (a) and (b), there are different possibities to assign update functions, for topoogy (c) there are 6 different possibiities. The determination of the statistica weight of each of the 36 networks for the case that every update function is chosen with the same probabiity is eft to the reader... In the foowing we ist severa frequenty used probabiity distributions for the update functions. Throughout this artice, we wi refer to these different update rues.. Biased functions: A function with n times the output vaue and M n times the output vaue is assigned a probabiity p n ( p) M n. Then the two frozen functions in tabe II have the probabiities p 4 and ( p) 4, each of the C functions and of the reversibe functions has the probabiity p 2 ( p) 2, and the C 2 functions have the probabiities p( p) 3 and p 3 ( p). For the specia case p = /2, a functions have the same probabiity /6. 2. Weighted casses: A functions in the same cass are assigned the same probabiity. K = networks are most interesting if the two reversibe functions occur with probabiity /2 each, and the two constant functions with probabiity. In genera K = networks, we denote the weight of the constant functions with δ. An ensembe of K = 2 networks is specified by the four parameters α, β, γ, and δ for the weight of C, reversibe, C 2 and frozen functions. The sum of the four weights must be, i.e., = α + β + γ + δ. 3. Ony canayzing functions are chosen, often incuding the constant functions. This is motivated by the finding that gene reguation networks appear to have many canayzing functions and by considerations that canayzing functions are bioogicay meaningfu [4, 5]. Severa authors create canayzing networks using three parameters [6]. One input of the node is chosen at random to be a canayzing input. The first parameter, η, is the probabiity that this input is canayzing if its vaue is. The second parameter, r, is the probabiity that the output is if the input is on its canayzing vaue. The third parameter, p, assigns update functions for the K other inputs according to rue (biased functions), for the case that the canayzing input is not on its canayzing vaue. (This notation is not uniform throughout iterature. For instance, in [6], the second and third parameter are named ρ and ρ 2.) 4. Ony threshod functions are chosen, i.e. the update rue is { if N σ i (t + ) = j= (c ij(2σ j ) + h) (2) ese The coupings c ij are zero if node i receives no input from node j, and they are ± with equa probabiity if node j is an input to node i. Negative coupings are inhibitory, positive coupings are excitatory. The parameter h is the threshod. Threshod networks are inspired by neura networks, but they are aso used in some modes for gene reguation networks [7, 8, 9]. 5. A nodes are assigned the same function. The network is then a ceuar automaton with random wiring. C. Dynamics Throughout this paper, we ony consider the case of parae update. A nodes are updated at the same time according to the state of their inputs and to their update function. Starting from some initia state, the network performs a trajectory in state space and eventuay arrives on an attractor, where the same sequence of states is periodicay repeated. Since the update rue is deterministic, the same state must aways be foowed by the same next state. If we represent the network states by points in the 2 N -dimensiona state space, each of these points has exacty one output, which is the successor state. We thus obtain a graph in state space. The size or ength of an attractor is the number of different states on the attractor. The basin of attraction of an attractor is the set of a states that eventuay end up on this attractor, incuding the attractor states themseves. The size of the basin of attraction is the number of states beonging to it. The graph of states in state space consists of unconnected components, each of them being a basin of attraction and containing an attractor, which is a oop in state space. The transient states are those that do not ie on an attractor. They are on trees eading to the attractors. Let us iustrate these concepts by studying the sma K = network shown in Figure 2.2, which consists of 4 nodes: 3 FIG. 2: A sma network with K = input per node. If we assign to the nodes,2,3,4 the functions invert, invert, copy, copy, an initia state evoves in the foowing way: 2 4

4 4 This is an attractor of period 3. If we interpret the bit sequence characterizing the state of the network as a number in binary notation, the sequence of states can aso be written as The entire state space is shown in Figure 2 C FIG. 3: The state space of the network shown in Figure 2.2, if the functions copy, copy, invert, invert are assigned to the four nodes. The numbers in the squares represent states, and arrows indicate the successor of each state. States on attractors are shaded. There are 4 attractors, two of which are fixed points (i.e., attractors of ength ). The sizes of the basins of attraction of the 4 attractors are 6,6,2,2. If the function of node is a constant function, fixing the vaue of the node at, the state of this node fixes the rest of the network, and there is ony one attractor, which is a fixed point. Its basin of attraction is of size 6. If the functions of the other nodes remain unchanged, the state space then ooks as shown in Figure FIG. 4: The state space of the network shown in Figure 2.2, if the functions, copy, invert, invert are assigned to the four nodes. Before we continue, we have to make the definition of attractor more precise: as the name says, an attractor attracts states to itsef. A periodic sequence of states (which we aso ca cyce) is an attractor if there are states outside the attractor that ead to it. However, some networks contain cyces that cannot be reached from any state that is not part of it. For instance, if we removed node 4 from the network shown in Figure 2.2, the state space woud ony contain the cyces shown in Figure 2C, and not the 8 states eading to the cyces. In the foowing, we wi use the word cyce whenever we cannot be confident that the cyce is an attractor. 5 D. Appications Let us now make use of the definitions and concepts introduced in this section in order to derive some resuts concerning cyces in state space. First, we prove that in an ensembe of networks with update rue (biased functions) or rue 2 (weighted casses), there is on an average exacty one fixed point per network. A fixed point is a cyce of ength. The proof is sighty different for rue and rue 2. Let us first choose rue 2. We make use of the property that for every update function the inverted function has the same probabiity. The inverted function has a s in the output repaced with s, and vice versa. Let us choose a network state, and et us determine for which fraction of networks in the ensembe this state is a fixed point. We choose a network at random, prepare it in the chosen state, and perform one update step. The probabiity that node remains in the same state after the update, is /2, because a network with the inverted function at node occurs equay often. The same hods for a other nodes, so that the chosen state is a fixed point of a given network with probabiity 2 N. This means that each of the 2 N states is a fixed point in the proportion 2 N of a networks, and therefore the mean number of fixed points per network is. We wi see ater that fixed points may be highy custered: a sma proportion of a networks may have many fixed points, whie the majority of networks have no fixed point. Next, we consider rue. We make now use of the property that for every update function a function with any permutation of the input states has the same probabiity. This means that networks in which state A eads to state B after one update, and networks in which another state C eads to state B after one update, occur equay often in the ensembe. Let us choose a network state with n s and N n s. The average number of states in a network eading to this state after one update is 2 N p n ( p) N n. Now, every state eads equay often to this state, and therefore this state is a fixed point in the proportion p n ( p) N n of a networks. Summation over a states gives the mean number of fixed points per network, which is. Finay, we derive a genera expression for the mean number of cyces of ength L in networks with K = 2 inputs per node. The generaization to other vaues of K is straightforward. Let C L N denote the mean number of cyces in state space of ength L, averaged over the ensembe of networks of size N. On a cyce of ength L, the state of each node goes through a sequence of s and s of period L. Let us number the 2 L possibe sequences of period L of the state of a node by the index j, ranging from to m = 2 L. Let n j denote the number of nodes that have the sequence j on a cyce of ength L, and (P L ) j,k the probabiity that a node that has the input sequences and k generates the output sequence j. This probabiity depends on the probabiity

5 5 distribution of update functions. Then C L N = N! n n k nj L n!... n m! N 2 (P L) j,k. {n j} j,k (3) The factor /L occurs because any of the L states on the cyce coud be the starting point. The sum is over a possibiities to choose the vaues {n j } such that j n j = N. The factor after the sum is the number of different ways in which the nodes can be divided into groups of the sizes n, n, n 2,...,n m. The product is the probabiity that each node with a sequence j is connected to nodes with the sequences and k and has an update function that yieds the output sequence j for the input sequences and k. This formua was first given in the beautifu paper by Samuesson and Troein []. We concude this section with a picture of the state space of a network consisting of nodes. 5. Using Equation (3), show that in an ensembe of networks with update rue or 2, there is on an average exacty one fixed point per network. 3. ANNEALED APPROXIMATION AND PHASE DIAGRAMS The anneaed approximation, which is due to Derrida and Pomeau [], is a usefu too to cacuate certain network properties. It is a mean-fied theory, which negects possibe correations between nodes. The first assumption of the anneaed approximation is that the network is infinitey arge. This means that fuctuations of goba quantities are negigibe. The second assumption of the anneaed approximation is that the inputs of each node can be assigned at every time step anew. The foowing quantities can be evauated by the anneaed approximation:. The time evoution of the proportion of s and s. 2. The time evoution of the Hamming distance between the states of two identica networks. 3. The statistics of sma perturbations. We wi discuss these in the foowing in the order given in this ist. One of the main resuts of these cacuations wi be the phase diagram, which indicates for which parameter vaues the networks are frozen, critica or chaotic. FIG. 5: The state space of a network with nodes E. Probems. Show that the fraction 3/32 of a networks in the ensembe with N = 4 and K = have the topoogy shown in Figure Show that the fraction 3 3 /2 of a networks with the topoogy shown in Figure 2.2 have the state space topoogy shown in Figure 2.3, if the distribution of update functions is given by rue with p = /4. 3. Which functions in Tabe II correspond to the threshod functions in networks with K = 2, if we set h =? 4. Consider again K = 2 networks, and choose the update rues 3 (canayzing functions), which are characterized by the parameters η, r, and p. Express the weight of each function in Tabe II in terms of η, r, and p. A. The time evoution of the proportion of s and s Let b t denote the number of nodes in state, divided by N. The proportion of nodes in state is then b t. We want to cacuate b t+ as function of b t within the anneaed approximation. Since the K inputs of each node are newy assigned at each time step, the probabiity that m inputs of a node are in state and the other inputs in state is b m t ( b t ) K m. Since we consider an infinitey arge network, this probabiity is identica to the proportion of nodes that have m inputs in state. Let p m be the probabiity that the output vaue of a node with m inputs in state is. Then we have b t+ = K m= ( ) K p m b m t ( b t ) K m. (4) m If p m is independent of m, the right-hand side is identica to p m, and b t reaches after one time step its stationary vaue, which is the fixed point of Equation (4). Among the above-isted update rues, this happens for rues (biased functions) and 2 (weighted casses ) and 4 (threshod functions). For rue, we have p m = /2, since the output vaues and occur with equa probabiity within each cass of update functions. For rue 2,

6 6 we have pm = p by definition. For rue 4, the vaue of pm is independent of m because the vaue of cij is and with equa probabiity, making each term cij (2σj ) to + and with equa probabiity. Therefore pm is identica to the probabiity that the sum of K random numbers, each of which is + or with probabiity /2, is at east as arge as h, pm K = 2.6 b (K h)/2 K. bt+ = bt ηr + ( bt )( η)r +bt ( η)p + ( bt )ηp = r + η(p r) + bt (p r)( 2η)..2 (5) The first two terms are the probabiity that the canayzing input is on its canayzing vaue, and that the output is then. The second two terms are the probabiity that the canayzing input is not on its canayzing vaue, and that the output is then. This is a one-dimensiona map. The ony fixed point of this map is r + η(p r). (p r)( 2η) Since the absoute vaue of the sope of this map is smaer than everywhere, every iteration (5) wi bring the vaue of bt coser to this fixed point. There exist aso update rues where the fixed points are unstabe and where periodic osciations or chaos occur. This occurs particuary easiy when a nodes are assigned the same function (rue 5). For instance, if a nodes are assigned the ast one of the canayzing functions occurring in the tabe of update functions II, we have the map bt+ = b2t. (6) The fixed point + b = X Here, is the number of +s, and K the number of s. For rue 3 (canayzing functions) we get [6] b = 5 is unstabe, since the sope of the map is ( 5) at this fixed point, i.e., it has an absoute vaue arger than. The iteration (6) moves bt away from this fixed point, and eventuay the network osciates between a nodes being and a nodes being. A map that aows for osciations with arger period and for chaos is obtained for the update rue that the output is ony if a inputs are equa. This map is defined for genera vaues of K and is given by K bt+ = bk t + ( bt ). (7) K FIG. 6: The vaues of bt that sti occur after the transient time for the map (3.5), as function of K. Let us consider K as a continous parameter. When it is increased, starting at, the map first has a stabe fixed point and then shows a period-doubing cascade and the Feigenbaum route to chaos shown in Figure 3 A [2]. A these resuts for bt were derived within the anneaed approximation, but they are generay beieved to appy aso to the origina networks with fixed connectivity patterns, if the thermodynamic imit is taken. If this is correct, the foowing three statements are aso correct: A (apart from a vanishing proportion of) initia states with a given vaue of b undergo the same trajectory bt with time. This trajectory is the same for a networks (apart from a vanishing proportion). When time is so arge that the dynamics have reached an attractor, the map bt+ (bt ) is the same as in the initia stage for those vaues of b that can occur on the attractors. These assumptions appear pausibe, since the paths through which a node can affect its own input nodes are infinitey ong in a randomy wired, infinitey arge network. Therefore we do not expect correations between the update function assigned to a node and the states of its input nodes. Neither do we expect a correation between the function bt+ (bt ) and the question of whether a state is on an attractor. B. The time evoution of the Hamming distance With the hep of the Hamming distance, one can distinguish between a frozen and a chaotic phase for RBNs. We make an identica copy of each network in the ensembe, and we prepare the two copies of a network in different initia states. The Hamming distance between the two networks is defined as the number of nodes that are in a different state. For the foowing, it is more convenient to use the normaized Hamming distance, which

7 7 is the Hamming distance divided by N, i.e., the proportion of nodes that are in a different state, h t = N N i= ( ) 2 σ () i σ (2) i. (8) If h t is very sma, the probabiity that more than one input of a node differ in the two copies, can be negected, and the change of h t during one time step is given by h t+ = λh t, (9) where λ is caed the sensitivity [3]. It is K times the probabiity that the output of a node changes when one of its inputs changes. For the first four update rues isted in Section 2.2, the vaue of λ is λ = 2Kp( p) (biased functions) λ = δ (weighted casses, K = ) λ = α + 2β + γ = + β δ (weighted casses, K = 2) λ = r( p) + ( r)p + (K )(η( b t ) +( η)b t )2p( p) (canayzing functions) ( ) K ( ) K λ = K (threshod functions) () 2 with in the ast ine being the argest integer smaer than or equa to (K h)/2. For rue 3 (canayzing functions), the first two terms are the probabiity that the output changes when the canayzing input is in a different state in the two network copies; the ast term is the probabiity that the output changes when one of the other inputs is in a different state in the two copies, mutipied by the number of noncanayzing inputs. This is the ony one out of the 4 rues where the vaue of λ depends on b t and therefore on time. The networks are in different phases for λ < and λ >, with the critica ine at λ = separating the two phases. In the foowing, we derive the properties of the networks in the two phases as far as possibe within the anneaed approximation. If λ <, the normaized Hamming distance decreases to. If the states of the two copies differ initiay in a sma proportion of a nodes, they become identica for a nodes, apart from possiby a imited number of nodes, which together make a contribution to the normaized Hamming distance. λ < means aso that if the two copies are initiay in identica states and the state of one node in one copy is changed, this change propagates on an average to ess than one other node. When the two copies differ initiay in a arger proportion of their nodes, we can argue that their states aso become identica after some time: we produce a arge number Q of copies of the same network and prepare their initia states such that copy number q and copy number q + (for a q =,..., Q) differ ony in a sma proportion of their nodes. Then the states of copy number q and copy number q + wi become identica after some time, and therefore the states of a Q copies become identica (again apart from possiby a imited number of nodes). The fina state at which a copies arrive must be a state where a nodes (apart from possiby a imited number) become frozen at a fixed vaue. If the fina state was an attractor where a nonvanishing proportion of nodes go through a sequence of states with a period arger than, different network copies coud be in different phases of the attractor, and the normaized Hamming distance coud not become zero. Ensembes with λ < are said to be in the frozen phase. A these considerations did not take into account that λ itsef may not be constant. For those update rues where b t assumes its fixed point vaue after the first time step, one can appy the reasoning of the previous paragraph starting at time step 2. For rue 3, the vaue b t approaches its fixed point more sowy, and therefore the vaue of λ changes over a onger time period. It is therefore possibe that the Hamming distance shows initiay another trend as during ater times. Once b t has reached its fixed point vaue, λ has become constant, and if then λ <, the normaized Hamming distance wi decrease to zero. In order to decide whether an ensembe is in the frozen phase, one must therefore evauate λ in the stationary state. For ensembes that have no stabe stationary vaue of b t, the above considerations do not appy directy, since b t cyces through different vaues, and so does λ. Furthermore, the two copies may be in different phases of the cyce and wi then never have a sma normaized Hamming distance. For ensembes with a finite osciation period T, one shoud evauate the product of a vaues of λ during one period. If this product is smaer than, a sma normaized Hamming distance created in a copy of a network with a stationary osciation, wi decrease after one period. Using a simiar reasoning as before, we concude that then the normaized Hamming distance between any two copies of the network wi decrease to zero if they have initiay the same vaue of b t. This means that a nodes (apart from possiby a imited number) go through a sequence of states that has the same period as b t. If λ > when b t has reached its stationary vaue, the normaized Hamming distance increases from then on with time and has a nonzero stationary vaue. A change in one node propagates on an average to more than one other node. If there is a fixed point or a short attractor, it is unstabe under many possibe perturbations. There is therefore no reason why a attractors shoud be short. In fact, attractors can be very ong, and the ensembe is in a phase that is usuay caed chaotic, even though this is no rea chaos because state space is finite and every trajectory becomes eventuay periodic. When b t does not become stationary but periodic, we consider again the product of a vaues of λ during one period. If this product is arger than, a sma normaized Hamming distance between two copies with the same vaue of b t

8 8 wi eventuay become arger. This means that attractors can be very ong and need not have the period of b t. For λ =, the ensembe is at the boundary between the two phases: it is critica. A change in one node propagates on an average to one other node. The critica ine can be obtained from Eqs.(), eading to the phase diagram shown in Figure 3B..8.6 p p.4 rue K rue 3, r=η = / K.8.6 δ.4 h.2 rue 2, K= β rue 4 frozen frozen K FIG. 7: Phase diagram for the first 4 update rues (biased functions, weighted casses, canayzing functions, threshod functions). Where there are more than 2 parameters, the remaining parameters were fixed at the vaues given in the respective graph tites. For theshod functions, the frozen phase is shaded. For K = 2, the mode is critica between h = 2 and 2, for arger K, there exists no critica vaue, but the mode is chaotic whenever it is not frozen. A our resuts are based on a cacuation for sma h t. When h t is not infinitesimay sma, (9) has the more genera form h t+ = λh t + νh 2 t +..., () with the highest power of h t being K, but we do not make here the effort to cacuate the coefficent ν or that of a higher-order term. We use this resut ony to obtain a reation between the stationary vaue of h t and the distance from the critica ine: In the chaotic phase, but cose to the critica ine (where λ is ony sighty arger than ), the stationary vaue of h t obtained from () is h = (λ )/ν. (2) It increases ineary with the distance from the critica ine, as ong as this distance is sma. C. The statistics of sma perturbations in critica networks Now et us have a coser ook at the propagation of a perturbation that begins at one node in a critica network. Let us consider again two identica networks, and et them be initiay in the same state. Then et us fip the state of one node in the first network. One time step ater, the nodes that receive input from this node differ in the two systems each with probabiity λ/k = /K (since λ = in a critica network). On an average, this is one node. Since the perturbation propagates to each node in the network with probabiity (K/N) (λ/k) = /N, the probabiity distribution is a Poisson distribution with mean vaue. We keep track of a nodes to which the perturbation propagates, unti no new node becomes affected by it. We denote the tota number of nodes affected by the perturbation by s. The size distribution of perturbations is a power aw n(s) s 3/2 (3) for vaues of s that are so sma that the finite system size is not yet fet, but arge enough to see the power aw. There are many ways to derive this power aw. The anneaed approximation consists in assuming that oops can be negected, so that the perturbation propagates at every step through new bonds and to new nodes. In this case, there is no difference (from the point of view of the propagating perturbation) between a network where the connections are fixed and a network where the connections are rewired at every time step. We begin our cacuation with one active node at time, n a (t = ) =, which is the node that is perturbed. At each time step (which is different from rea time!), we choose one active node and ask to how many nodes the perturbation propagates from this node in one step. These become active nodes, and the chosen node is now inactive. We therefore have a stochastic process n a (t + ) = n a (t) + ξ for the number of active nodes, with ξ being a random number with a Poisson distribution with mean vaue. The stochastic process is finished at time t = s when n a (s) =. s is the tota number of nodes affected by the perturbation. Now we define P (y, t) as the probabiity that the stochastic process has arrived at y = before or at time t, if it has started at n a = y at time t =. During the first step, y changes by y = ξ. If we denote the probabiity distribution of y with P( y), we obtain P (y, t) = d( y)p( y)p (y + y, t ) d( y)p( y) [ P (y, t) + y P y + 2 ( y)2 2 P y 2 P ]. t The first term on the right-hand side cances the efthand side. The second term on the right-hand side is the mean vaue of y, which is zero, times y P. Were are

9 9 therefore eft with the ast two terms, which give after integration P t = 2 P 2 y 2. (4) This is a diffusion equation, and we have to appy the initia and boundary conditions P (, t) = P (y, ) = P (y, ) =. (5) Expanding P in terms of eigenfunctions of the operator / t gives the genera soution P (y, t) = a+by+ dωe ω2t/4 (c ω sin(ωy)+d ω cos(ωy)). The initia and boundary conditions fix the constants to a = and d ω = and c ω = 2/πω. We therefore have P (y, t) = 2 sin ωy dω e ω2t/4, (6) π ω which becomes for y = P (, t) = 2 π dω sin ω ω e ω2 t/4 O(t /2 ) (7) for arge t. The size distribution of perturbations is obtained by taking the derivative with respect to t, eading to (3). Readers famiar with percoation theory wi notice that the spreading of a perturbation in a critica RBN is cosey reated to critica percoation on a Bethe attice. Ony the probabiity distribution of the stochastic variabe ξ is different in this case. Since the resut depends ony on the existence of the second moment of y, it is not surprising that the size distribution of critica percoation custers on the Bethe attice foows the same power aw. 3. If in a frozen network ony a imited number of nodes may not be frozen for arge times, and if in a chaotic network a nonvanishing proportion of nodes remain nonfrozen, what do you expect in a critica network? 4. NETWORKS WITH K = Many properties of networks with K = inputs per node can be derived anayticay. Nevertheess, these networks are nontrivia and share many features with networks with arger vaues of K. Therefore it is very instructive to have a coser ook at K = networks. In this review, we wi not reproduce mathematicay exact resuts that require ong cacuations, as is for instance done in [4, 5]. Instead, we wi present phenomenoogica arguments that reproduce correcty the main features of these networks and that hep to understand how these features resut from the network structure and update rues. We begin by studying the topoogy of K = networks. Then, we wi investigate the dynamics on these networks in the frozen phase and at the critica point. Finay, we wi show that the topoogy of K = networks can be mapped on the state space of K = N networks, which aows us to derive properties of the attractors of K = N networks, which are chaotic. A. Topoogy of K = networks If each node has one input, the network consists of different components, each of which has one oop and trees rooted in this oop, as shown in Figure 4 A. K = networks have the same structure as the state space pictures of other random Booean networks, ike the ones shown in Figures 2.3 and 2.4, ony the arrows are inverted. D. Probems. Expain why there is a finite critica region for K = 2 and no critica vaue of λ at a for K > 2 for update rue 4 (Figure 3 B). 2. For each of the 6 update functions for K = 2, consider an ensembe where a nodes are assigned this function. Find the function b t+ (b t ). Find a fixed points of b and determine if they are stabe. If b t becomes constant for arge times, determine whether the ensembe is frozen, critica or chaotic. If b t osciates for arge times, determine whether the normaized Hamming distance between two identica networks that start with the same vaue of b, goes to zero for arge times. Interpret the resut. FIG. 8: Exampe of a network with one input per node. It has two components, the arger component has a oop of size 3 and two trees rooted in it (one of size and one of size 6), and the smaer component has a oop of size and one tree of size. Let us first cacuate the size distribution of oops. We consider the ensembe of a networks of size N. In each network of the ensembe, each node chooses its input at random from a other nodes. The probabiity that a

10 given node is sitting on a oop of size is therefore ( P() = ) ( 2 ) (... ) N N N N e /N e 2/N... e ( )/N N = e ( )/2N N e 2 /2N N. (8) The first factor is the probabiity that the input to the first node is not this node. The second factor is the probabiity that the input to the second node is not the first or second node, etc. The ast factor is the probabiity that the input of the th node is the first node. The approximation in the second step becomes exact in the thermodynamic imit N for vaues of that satisfy im N /N =. The approximation in the ast step can be made if is arge. The probabiity that a given node is sitting on any oop is therefore proportiona to P()d N /2 e x2 /2 dx N /2. This means that the tota number of nodes sitting on oops is proportiona to N. The mean number of nodes sitting on oops of size in a network is NP() e 2 /2N. The cutoff in oop size is proportiona to N. For N, the mean number of nodes in oops of size is. This resut can aso be obtained by a simpe argument: The probabiity that a given node is sitting on a oop of size is in the imit N simpy /N, since the node amost certainy does not choose itsef as input or as input of its input etc, but in the th step the first node must be chosen as input, which happens with probabiity /N. The mean number of oops of size in a network is NP() e 2 /2N /. For N, this is simpy /. Since oops are formed independenty from each other in the imit N, the probabiity distribution of the number of oops of size is a Poisson distribution with mean vaue /. The mean number of oops per network is NP()/ e x 2 /2 N x /2 dx 2 nn for arge N. This is identica to the mean number of components. Next, et us consider the trees rooted in the oops. There are of the order of N nodes, which sit in N trees, each of which is rooted in a reevant node. This means that the average tree size is proportiona to N. The construction of a tree can be described formay exacty in the same way as we described the propagation of a perturbation in a critica network in the ast section: we begin with a node sitting in a oop. The nodes that are not sitting in oops receive their input with equa probabiity from any node in the network. Our node is therefore chosen with probabiity /N by every node outside the oops as an input, and the probabiity distribution of the number of outputs into the tree is a Poisson distribution with mean vaue (negecting terms of the order N /2 ). In the same way, we find that the number of outputs of each of the newy found tree nodes is again a Poisson distribution with mean vaue. We iterate this process unti we have identified a nodes that are part of this tree. The size distribution of trees is s 3/2. The cutoff must be s max N in order to be consistent with what we know about the mean tree size and the tota number of nodes in trees: The mean tree size is s smax ss 3/2 ds s /2 max N. The tota number of nodes in trees is proportiona to smax N ss 3/2 ds N. B. Dynamics on K = networks Knowing the topoogy of K = networks, aows us to cacuate their dynamica properties. After a transient time, the state of the nodes on the trees wi be independent of their initia state. If a node on a tree does not have a constant function, its state is determined by the state of its input node at the previous time step. A nodes that are downstream of a node with a constant function wi become frozen. If there is no constant function in the oop and the path from the oop to a node, the dynamics of this node is saved to the dynamics of the oop. If the weight of constant functions, δ, is nonzero, the probabiity that a oop of size does not contain a frozen function is ( δ), which goes to zero when is much arger than /δ. Therefore ony oops smaer than a cutoff size can have nontrivia dynamics. The number and ength of the attractors of the network are determined by the nonfrozen oops ony. Once the cyces that exist on each of the nonfrozen oops are determined, the attractors of the entire networks can be found from combinatoria arguments.. Cyces on oops Let us therefore focus on a oop that has no constant function. If the number of invert functions is odd, we

11 ca the oop an odd oop. Otherwise it is an even oop. Repacing two invert functions with copy functions and repacing the states σ i (t) of the two nodes controed by these functions and of a nodes in between with σ i (t), is a bijective mapping from one oop to another. In particuar, the number and ength of cyces on the oop is not changed. A odd oops can thus be mapped on oops with ony one invert function, and a even oops can be mapped on oops with ony copy functions. We first consider even oops with ony copy functions. These oops have two fixed points, where a nodes are in the same state. If is a prime number, a other states beong to cyces of period. Any initia state occurs again after time steps. Therefore the number of cyces on an even oop is (9) if is a prime number. The numerator counts the number of states that are not fixed points. The first term is therefore the number of cyces of ength. Adding the two fixed points gives the tota number of cyces. If is not a prime number, there exist cyces with a periods that are a divisor of. Next, et us consider odd oops with one invert function. After 2 time steps, the oop is in its origina state. If is a prime number, there is ony one cyce that has a shorter period. It is a cyce with period 2, where at each site s and s aternate. The tota number of cyces on an odd oop with a prime number is therefore (2) If is not a prime number, there are aso cyces with a period that is twice a divisor of. 2. K = networks in the frozen phase For networks with K = input per node, the parameter λ is λ = δ. (2) Thefore, ony networks without constant functions are critica. Networks with δ > are in the frozen phase. The mean number of nonfrozen nodes on nonfrozen oops is given by the sum ( δ) = δ δ. (22) We ca these oops the reevant oops. We ca the nodes on the reevant oops the reevant nodes, and we denote their number with N re. The mean number of reevant oops is given by the sum ( δ) nδ, (23) with the ast step being vaid for sma δ. The probabiity that the activity moves up the tree to the next node is δ at each step. The mean number of nonfrozen nodes on trees is therefore δ δ ( ) 2 δ ( δ) =, (24) δ and the tota mean number of nonfrozen nodes is ( δ)/δ 2. This is a finite number, which diverges as δ 2 when the critica point δ = is approached. 3. Critica K = networks If the proportion of constant functions δ is zero, the network is critica, and a oops are reevant oops. There are no nodes that are frozen on the same vaue on a attractors. A oop of size has a state that is constant in time, but in can take two different vaues. Larger oops have aso two fixed points, if they are even. Part of the nodes in a critica K = networks are therefore frozen on some attractors or even on a attractors, however, they can be frozen in different states. The network consists of n N/2 oops, each of which has of the order 2 / cyces of a ength of the order. The size of the argest oop is of the order of N. The number of attractors of the network resuts from the number of cyces on the oops. It is at east as arge as the product of a the cyce numbers of a the oops. If a cyce is not a fixed point, there are severa options to choose its phase, and the number of attractors of the network becomes arger than the product of the cyce numbers. An upper bound is the tota number of states of a the oops, which is 2 N re e a N, and a ower bound is the number of attractors on the argest oop, which is of the order e b N / N > e b N with b < b < a. From this it foows that the mean number of attractors of critica K = networks increases exponentiay with the number of reevant nodes. A compementary resut for the number of cyces C L of ength L, which is vaid for fixed L in the imit N is obtained by the foowing quick cacuation: C L N {n } = {n } c c ( ( ) e / n n! ( e / ( k ) n n! k n c e (k )/ = e R c (k )d/ e ( k ) R c d/ ) e (HL )n N = N (HL )/2. (25) Here, n is the number of oops of size, c is the cutoff in oop size N, and k is the number of states on a oop of size that beong to a cyce of ength L. This ) )

12 2 is zero for many oops. The average over an -interva of size L is identica to H L, which is the number of cyces on an even oop of size L. A more precise derivation of this reation, starting from the K = version of (3) and evauating it by making a sadde-point approximation, can be found in [6], which is inspired by the equivaent cacuation for K = 2 critica networks in []. The ength of an attractor of the network is the east common mutipe of the cyce engths of a the oops. A quick estimate gives N aog N since the ength of the arger oops is proportiona to N, and this has to be taken to a power which is the number of oops. A more precise cacuation [7] gives this expression, mutipied with a factor N b / og N, which does not modify the eading dependence on N. for the dependence on N of the number and size of the basins of attraction of the different attractors. Let us first consider networks in the frozen phase. As we have seen, there is at most a imited number of sma nonfrozen oops. Their number is independent of system size, and therefore the number of attractors is aso independent of the system size. The initia state of the nodes on these nonfrozen oops determines the attractor. The initia states of a other nodes are competey irreevant at determining the attractor. The size of the basin of attraction of an attractor is therefore 2 N N re, mutipied with the ength of the attractor, i.e., it is 2 N, divided by a factor that is independent of N. The proportion of state space beonging to a basin is therefore aso independent of N. If we define the basin entropy [8] by S = a p a np a (26) C. Dynamics on K = N networks The topoogy of a K = network is identica to the topoogy of the state space of a K = N network, when a update functions are chosen with the same weight. The reason is that in a K = N network, the state that succeeds a given state can be every state with the same probabiity. Thus, each state has one successor, which is chosen at random among a states. In the same way, in a K = network, each node has one input node, which is chosen at random among a nodes. The state space of a K = N network consists of 2 N nodes, each of which has one successor. We can now take over a resuts for the topoogy of K = networks and transate them into state space: The K = N networks have of the order of og(2 N ) N attractors. The argest attractor has a ength of the order 2N = 2 N/2, and this is proportiona to the tota number of states on attractors. A other states are transient states. An attractor of ength occurs with probabiity / if 2 N/2. Ceary, K = N networks, where a update functions are chosen with the same weight, are in the chaotic phase. The mean number of nodes to which a perturbation of one node propagates, is N/2. At each time step, haf the nodes change their state, impying aso that the network is not frozen. D. Appication: Basins of attraction in frozen, critica and chaotic networks The advantage of K = networks is that they are anayticay tractabe and can teach us at the same time about frozen, chaotic and critica behavior. We wi discuss in the next section to what extent the resuts appy to networks with other vaues of K. Based on our insights into K = networks, we derive now expressions with p a being the fraction of state space occupied by the basin of attraction of attractor a, we obtain S = const for a K = network in the frozen phase. Next, et us consider the chaotic K = N network ensembe. There are on an average / attractors of ength, with a cutoff around 2 N/2. The basin size of an attractor of ength is of the order 2 N/2, which is times the average tree size. The basin entropy is therefore S 2 og N/2 2 og xdx = const. N/2 2 N/2 (27) Finay, we evauate the basin entropy for a critica K = network. There are of the order e a N attractors with approximatey equa basin sizes, and therefore the basin entropy is S N N re. (28) Whie frozen and chaotic networks have a finite basin entropy, the basin entropy of critica networks increases as the number of reevant nodes [8]. E. Probems. How many cyces does an even (odd) oop of size 6 have? 2. Count the attractors of the network shown in Figure 4. for a four cases where oop and/or oop 2 are even/odd. 3. How does the transient time (i.e. the number of time steps unti the network reaches an attractor) increase with N for (a) K = networks in the frozen phase, (b) critica K = networks, (c) chaotic K = N networks?