Noisy Attractors and Ergodic Sets in Models. of Genetic Regulatory Networks

Transcription

1 Noisy Attractors and Ergodic Sets in Models of Genetic Regulatory Networks Andre S. Ribeiro Institute for Biocomplexity and Informatics, Univ. of Calgary, Canada Department of Physics and Astronomy, Univ. of Calgary, Canada Center for Computational Physics, Univ. of Coimbra, P Coimbra, Portugal Stuart A. Kauffman Institute for Biocomplexity and Informatics, Univ. of Calgary, Canada Department of Physics and Astronomy, University of Calgary, Canada Abstract We investigate the hypothesis that cell types are attractors. This hypothesis was criticized with the fact that real gene networks are noisy systems and thus, do not have attractors (Kadanoff et al, 2002). Given the concept of ergodic set as a set of states from which the system, once entering, does not leave when subject to internal noise, first, using the Boolean network model, we show that if all nodes of states on attractors are subject to internal state change with a probability p due to noise, multiple ergodic sets are very unlikely. Thereafter, we show that if a fraction of those nodes are locked (not subject to state fluctuations caused by internal noise), multiple ergodic sets emerge. Finally, we present an example of a gene network, modelled with a realistic model Preprint submitted to Elsevier 27 May 2013

2 of transcription and translation and gene-gene interaction, driven by a Stochastic Simulation Algorithm with multiple time-delayed reactions, which has internal noise and that we also subject to external perturbations. We show that, in this case, two distinct ergodic sets exist and are stable within a wide range of parameters variations and, to some extent, to external perturbations. Key words: Gene Regulatory Networks, Boolean Networks, Stochastic Simulation Algorithm, Attractor, Cell Type. PACS: k, Yc, Fd, a 1 Introduction Understanding the integrated behavior of genetic regulatory networks (GRN), taken in the large sense to comprise genes, RNA, proteins and other molecules that mutually interact to control the dynamical behavior of the network within and between cells, has emerged as a fundamental problem in Systems Biology. At this stage we only have partial knowledge of the regulatory network structure and logic driving the dynamical behavior of these systems. Nevertheless, we can begin to address questions about these systems using the known features of these networks, and constructing the family or ensemble, of all networks consistent with those observations. This ensemble approach (Kauffman, 2004) studies the expected properties of members of the ensemble and predicts new observables to test against the dynamical behavior of cells and tissues. It is a further profound issue whether real networks are generic The authors would like to thank icore, a funding agency of Alberta, Canada. Corresponding author. Phone: address: ARibeiro@ucalgary.ca (Andre S. Ribeiro). URL: aribeiro/index.html (Andre S. Ribeiro). 2

3 to any ensemble, given 3 billion years of evolution and natural selection. From the mathematical point of view, there are three broad frameworks in which to cast the analysis of such networks. At the most precise level one considers the chemical master equation of the detailed behavior of all components in members of some ensemble of networks. Such models are inherently stochastic. Understanding the consequences of such noise is emerging itself as a critical problem and is the focus of this article. At a second level of abstraction, one considers systems of deterministic nonlinear differential equations capturing, in some sense, the mean field behavior of the real noisy stochastic networks. Since the number of copies of regulatory molecules in the real system can be one to a few, such deterministic equations are, at best, an approximation. One approach in this framework is to add white noise in Langevin equations (Toulouse et al, 2005). However, it remains to be shown that this modelling strategy captures the real character of cellular dynamical noise. At a still higher level of abstraction, one can consider model GRN s such that genes states, time and other components are discrete variables. While furthest from the chemical master equation description, hence the most abstract, such models allow studying very large networks, with thousands of model genes or other components. In particular, random Boolean networks (RBN) have been the subject of considerable analysis (Kauffman, 1993, 1969). The links between the behaviors of RBN and chemical master equation models of the same network were recently analyzed (Zhu et al, 2006), and some parallels were found. Also, it is known that several properties of RBNs generalize to a class of piecewise linear differential equations (Glass, 1975). In particular, 3

4 a well-established transition from ordered to chaotic dynamics at a critical phase transition occurs in both systems. A generic property of many deterministic nonlinear dynamical systems is that a typical member of the ensemble has a multiplicity of dynamical attractors such as steady states, limit cycles or strange attractors, each of which drains a basin of attraction consisting of all states lying on trajectories that flow to or include that attractor. At the molecular dynamical level, we do not know what a cell type is. However, if we consider even the simplest case of a binary valued Boolean network with 30,000 genes, its state space is 2 30,000. It takes seconds to minutes for genes to turn on an off. There have been about seconds since the big bang. Thus, whatever a cell type may be, it must be a very restricted subset of the states possible in the GRN. In the deterministic framework, it is almost an inevitable hypothesis that cell types correspond to attractors in the dynamics of the network. Such attractors, in the absence of noise, are the asymptotic behaviors of the network. To be biologically plausible, such attractors need to be constrained by their dynamics to very small regions of the state space of the system. Importantly, RBNs in the ordered regime and their piecewise linear cousins have this property: their attractors are tiny subsets of state space. In the chaotic regime this is not true. This suggests that cells may be in the ordered regime. In this picture, if a cell type is an attractor, then a pathway of differentiation can be only two things: first, a perturbation from one attractor into a new basin of attraction from which the cell passes via a trajectory to the new attractor cell type. Here the perturbation can be a noise fluctuation, or an 4

5 exogenous signal. Secondly, it is possible that real GRNs have variables that change dynamically over a wide range of time scales. Then the cell types would correspond to the attractors of the fast dynamics, and differentiation would occur due to bifurcations in the fast dynamics as the slow variables change, such as morphogenesis. These two possibilities do not exclude one another. However, an important criticism (Kadanoff et al, 2002) with respect to Boolean networks is that noise may render such attractors a poor model of cell types since closure of an attractor (a state cycle) in the discrete dynamics is delicate. This is an important criticism and leads to the question of the effect of noise on the attractor as the cell type hypothesis. Some previous work was made in a first attempt to address this problem. In (Klemm and Bornholdt, 2005), Klemm and Bornholdt tested the stability of attractors with respect to infinitesimal deviations from synchronous update and found that most attractors are artifacts arising from synchronous clocking. Importantly, the remaining attractors are stable against fluctuating delays, and its average number grows with the number of nodes, within the numerically tractable range. A similar scaling law as been observed in a different approach to asynchronous Boolean networks (Greil and Drossel, 2005). The two works confirm that their models have multiple attractors assuming asynchronous updating. Yet, these works do not assume or model any probability of genes misbehaving, i.e., act contrary to what inputs states and Boolean transfer function determines. Only the time at which nodes update is assumed stochastic. We here report an analysis of the effects of minimal noise (understood has genes misbehaving with a certain probability) in the attractors of RBNs with 5

6 synchronous updating. We also do a similar study using more realistic models of GRNs. In the next section, we introduce RBN as models of GRN. The third section gives our method of analysis of ergodic sets. Next, we introduce a more realistic model of GRNs recently proposed (Ribeiro et al, 2006), based on the stochastic simulation algorithm (SSA) (Gillespie, 1977, 1976) that models transcription and translation as multiple time delayed events (Roussel and Zhu, 2006). Using this modelling strategy we show an example of such a network that indeed exhibits two ergodic sets. In the fifth section we present our results followed by the discussion. 2 Noisy Random Boolean Networks An RBN consists of N nodes, each representing a gene (Kauffman, 1969) or other variable, where the nodes can take binary values of on and off, and each node receives inputs from among them. Each gene is assigned a Boolean function from the set of possible Boolean functions of K variables. Time is discrete and here we consider that all genes update their activities synchronously. Thus, a state of the network passes to a unique successor state at each moment. Over time, the system follows a trajectory that ends on a state cycle attractor. In general, the network has many such attractors. There are several ways to introduce noise in RBN dynamics. In the Boolean modelling strategy, we introduce noise by allowing, at each time step t, that with a small probability, P, any node misbehaves its Boolean rule and assumes the opposite value to that specified. 6

7 Although we assume this as internal noise, using this noise model, external and internal noise are not distinguishable since they both consist in bit flipping a node state at a given t. Also, there is a very small probability, P N, that all nodes misbehave at a given state transition. Given this, the system is a Markov process with a hierarchy of time scales for single and multiple gene mistakes. Here, we limit the system to single gene mistakes, which is an approximation to long-term behavior in the case of very low P. I.e, we assume that multiple mistakes occur on a very long time scale and can be ignored. Since we explore all attractors, and misbehaviors on transient states would, at most, change which attractor the system falls into, we assume the system falls on a attractor (state cycle) before noise can affect its dynamics. We then consider all the states of each state cycle of the network. Each of these states of the state cycle can go to N other states, by doing a single gene state flip. We start the network on each of the states one can obtain by doing this bit flip, for each gene of each state of each state cycle, and record numerically the state cycle to which the system flows. To analyze such systems, it is essential to explore the entire state space, such that all state cycle attractors of the network are known. This limits our analysis to small networks (N < 20). Sometimes, perturbing a gene in a state of a state cycle leads the system to fall into another state cycle. That fact made us use the notion of ergodic sets, which we present in the following section. 7

8 3 Ergodic Sets Here we use the standard definition of ergodic set which can be found in the context of stochastic processes. Define an ergodic set as a closed set of state cycles such that each state cycle can be reached by a single or more genes mistakes caused by internal noise from any other state cycle of the same ergodic set. Also, the ergodic set is such that state fluctuations due to internal noise (here, single bit flips in the Boolean framework and stochastic fluctuations in the delayed SSA framework) are not sufficient to make the system leave the ergodic set. Here we assume that, in the Boolean framework, the probability P of a gene misbehaving (do the opposite of what its Boolean rule and inputs genes states demand) is small enough such that after a noise fluctuation the system dynamics progresses fast enough for the system to return to a state cycle, before another noise induced change occurs again. In the stochastic formulation no such assumption is necessary, because it is not a synchronous network and the system dynamics is inherently stochastic. As we show in the results section, the coupled gene network indeed responds to stochastic variations of genes expression levels faster than the time these need to accumulate a sufficient number of proteins to impose a state change. Also, in this framework we show examples of, when the system is in an ergodic set, it also is resistent to some extent to external perturbations, although we do not impose this as a necessary condition to be considered an ergodic set. Above we noted that, in the deterministic setting, it was almost an inevitable 8

9 hypothesis that a cell type corresponds to an attractor. If noise is included, a first possibility is that a cell type corresponds to a noisy attractor, while a second possibility is that a cell type corresponds to an ergodic set. One case does not exclude the other. But, if the second case is true, then in order to have multiple cell types, the system must have multiple ergodic sets. Here we examine this possibility. For this purpose we now describe how to determine ergodic sets in the RBN framework. We generated RBN of random and scale free topologies (scale free topologies are generated by the algorithm scale free 1 in (Airoldi and Carley, 2005), which creates directed scale free graphs according to the originally proposed algorithm in (Barabasi, 2002)). Given the inputs distribution of each node, a random Boolean function is set for all possible input values. A complete path of states matrix is generated, representing to which state any state will lead, in the system deterministic evolution. Also, initializing the system in all possible states, a cycle is found for each case. Since it is a deterministic RBN, all cycles are found using this method. We now determine which state cycles can reach other state cycles in the limit of small P, where only a single node can be perturbed at a time. We store such information in an adjacency matrix of state cycles. Having the set of state cycles, we perturb each gene of each state belonging to a cycle, one at a time, and find to which cycle the system dynamics leads. In general, if perturbing a gene of a state of cycle i, leads the system state to cycle j, then the position [i,j] of the adjacency matrix of state cycles (initialized with all zeros) takes the value 1. Given the adjacency matrix of state cycles, we now check if there is any path, 9

10 from any state of the merged sets of state cycles, which would allow leaving to a state not belonging to that set. If this occurs, the set is not an ergodic set. The remaining sets are the ergodic sets of the system. As an example, suppose a system with state cycles represented by letters A, B, C, D and E, and that according to the adjacency matrix of state cycles, there is a pathway from A to B, from B to C, from C to B, from C to D, and from D to C. Additionally, each state cycle has a pathway leaving from it onto itself. We represent this in the form of a directed graph in Fig. 1, from where one sees that the possible pathways between state cycles, due to small perturbations, result in a graph such that the system always end up on ergodic sets (B, C, D) or (E), and once entering them, cannot leave. In the stochastic modelling strategy (described in the following section), the analog is defined with the use of the K-means clustering algorithm (MacQueen, 1967), which we use to classify the products of gene expression quantity over time (proteins time series), as 0 or 1. The algorithm works as follows. Let K cluster be the number of clusters one wants to cluster the data points into: i) Place K cluster points into the space represented by the objects that are being clustered. These points are used as initial group centroids, ii) assign each object to the group that has the closest centroid, iii) when all objects have been assigned, recalculate the positions of the K cluster centroids, iv) repeat steps 2 and 3 until the centroids do not change. This separates the objects into groups from which the metric to be minimized can be calculated. The function to be minimized here is (eq. 1): 10

11 Fig. 1. A directed graph representing the possible pathways between state cycles A, B, C, D and E of an imaginary RBN with 2 ergodic sets. In this RBN, once entering the ergodic set of state cycles (B,C,D) or the ergodic set of state cycle E, the system cannot leave them. J = K cluster j = 1 n x (j) 2 i c j i=1 (1) The quantity x (j) i c j is the distance measure between each of the n data points, x (j) i, and their respective cluster centers value, c j, at each step of the algorithm. Since we intend to binarize the time series of the number of proteins of each gene, we set K cluster = 2, while n equals the number of data points of the proteins time series, and is equal to the ratio between the time series total length and the sampling period. The variables x (j) i are the values of the data points which in our case is the number of proteins in each sample of the system state, and the variables c j are the data points chosen as centroids at each step of the algorithm. We note that the temporal order of the proteins 11

12 levels is not taken in account for the clustering procedure. Because we here want to binarize the proteins expression level, choosing how many clusters one wants is not a problem (generally, this choice might be problematic when using K-means). However, the results of K-means also depend on the initial choice for centroids. We used to standard approach to minimize this problem. Since we choose the initial centroids values randomly from the set of data points, we make 1000 independent runs of the K-means algorithm to each time series and choose the solution that minimizes the distance measure. Given the time series of the number of the proteins (data points) of a gene of the GRN, the algorithm begins by choosing randomly two data points, and uses them as initial centroids. Then, it computes the distance between each data point and the centroids (using eq. 1), and places the data point (independently of the moment in time to which the data point corresponds to) into the cluster that has the smaller distance between the data point value and the centroid value. After the first clustering step, the algorithm computes the mean value of the data points in each cluster. These mean values are used as centroids in the next iteration. Once the new centroids values are computed, again all data points are placed in one of the two clusters, and again new centroids are computed. When the centroids value is constant from one step to the next of the algorithm, the process stops MacQueen (1967). Using this clustering algorithm is adequate in our case since the GRN in the stochastic framework, used as example, is such that the proteins levels of all genes have two very distinct levels ( high and low ), and when in each of these states, only vary due to stochastic fluctuations. Therefore, setting K cluster 12

13 to 2, one can distinguish clearly if a gene is on or off at a given moment. E.g., if one had a gene whose protein quantity could be in three distinct levels, K cluster should be set at 3, and binarizing this time series would not provide clear results. In general, the time series characteristics determine what is the most appropriate clustering method to use in each case. Using the K-means algorithm also allows determining in our examples, what should be considered a perturbation. Namely, here we consider that a perturbation in a protein concentration must be such that, given the threshold that defines if a state of a gene is on or off determined by the K-means algorithm, the variation of the number of proteins externally imposed must be sufficient to cross that threshold. 4 A model of a GRN dynamically driven by a multiple-delays SSA A gene network modelling strategy was proposed (Ribeiro et al, 2006) that models GRNs by coupling genes via protein-protein interactions and proteinoperator sites interactions. Also, it models transcription and translation as multiple time delayed events. Its dynamics is driven by the delay SSA (Roussel and Zhu, 2006), that consists of a modification of the original SSA (Gillespie, 1977, 1976) and uses a waiting list to store delayed output events. The waitlist consists of a list of elements (e.g., proteins being produced and occupied promoter regions), each to be released after a certain time interval (such time duration is also stored on the waitlist). The algorithm proceeds as follows (Roussel and Zhu, 2006): 1) Set t 0, t stop stop time, read initial number of molecules and reactions, 13

14 create empty waiting list L for delayed generating events. 2) Do an SSA step for reacting events to get next reacting event R 1 and corresponding occurrence time t + t 1. 3) Compare t 1 with least time in L, t min. If t 1 < t min or L is empty, set t t + t 1. Update number of molecules by performing R 1, adding delayed products (if existing) and the time delay they have to stay from appropriate distribution into L, as necessary. 4) If L is not empty and if t 1 t min, set t t + t min. Update number of molecules and L, by releasing the first element in L, else go to step 5. 5) If t < t stop, go to step 2, else stop. In these networks we represent a gene by its promoter (P ro i ) occupancy state (available to transcribe or occupied due to binding to some molecule). The system state is defined at a certain moment t by the number of the proteins (p i ) of each gene present in the system. In general, the level of expression is a function of number of RNA polymerase (RNAp ) available, promoter time delay, protein production time delay, and rate constant of transcription/translation reaction. The way to use this modelling strategy to generate ensembles of networks was proposed in (Ribeiro et al, 2006). Here, for simplicity, we model transcription and translation as single step multi-delayed reactions (Roussel and Zhu, 2006; Ribeiro et al, 2006). Using this model we here build a specific gene circuit that exhibits two ergodic sets that can be reached from the same initial condition. Once reached they are stable to all sorts of internal noise (and external perturbations of single genes states). 14

15 The gene circuit here used was built with the intent of showing that a gene network with a realistic model of noise can in fact exhibit multiple ergodic sets, via the coupling of two circuits which, if isolated, would toggle. One sub-circuit is a toggle switch (Gardner et al, 2000) without cooperative binding (a modified version of the model presented in (Lipshtat et al, 2006) that here includes time delays from transcription and translation). With internal noise present, i.e., if the dynamics is driven by the delayed SSA, the system can toggle between two states (either gene A is on and B is off, or the opposite). The other sub-circuit is a 4 genes repressilator. In a deterministic framework, a 4 genes repressilator exhibits, like the toggle switch, two attractors (odd genes on and even genes off, or the opposite). Again, using our stochastic modelling strategy due to the values chosen for the several parameters, these are not stable although toggles are less frequent than in the toggle switch. The reason to choose these two bistable circuits is that it has long been posed the hypothesis that differentiation is based on bistable mechanisms (Monod and Jacob, 1961), and recent experiments support this hypothesis (Chang et al, 2006). The toggle switch is the best know example of a bistable circuit. Also, its dynamical behavior is well known from experimental measurements (Gardner et al, 2000). The 4 genes repressilator is similar to the toggle switch, toggling between two states but at different average rates. Our goal is to show how one can attain stability simply by coupling the two known sub-circuits which, by themselves, do not settle on a single attractor. 15

16 Due to their relevance, these bistable sub-circuits dynamical behavior (if not coupled to one another) have been studied in the literature (mostly using the continuous o.d.e. s modelling strategy). See, e.g. (Muller et al, 2006; Li et al, 2006; Smith, 1987; Allwright, 1977; MacDonald, 1977; Zhu et al, 2006). As shown in the next section, the stability of the system, i.e., the resistance to internal noise and external perturbations arises from the coupling of the two circuits. The 4 genes repressilator can be described by the following reactions: RNAp + Pro i Pro i (1) + RNAp(20) + p i (100) (2) 0.1 Pro j +p i Pro j p i (3) 0.01 p i (4) Pro i p j Pro i (5) In these reactions, N = 4, i = 1,..., N and j = i + 1, except for i = N, where j = 1. In reaction 2, a time delay τ is associated to each product X of the reaction representing gene expression, using the notation: X(τ). Reaction 3 represents two independent reactions: binding and unbinding of the repressor to the promoter. The rate constants of these two reactions, represented in the numbers associated to the arrows, are not equal. The unbinding reaction allows the repressor to disassociate from the promoter. The repressor can also decay while on the promoter via reaction 5. This reaction is needed to allow the protein to decay when bound to the promoter at the same rate as if not bound. If this reaction was absent, binding to the promoter would act as a protection against decay. The toggle switch is described by the following reactions: 16

17 RNAp + Pro A 0.05 Pro A (2) + RNAp(20) + A(100) (6) RNAp + Pro B 0.05 Pro B (2) + RNAp(20) + B(100) (7) A 0.001, B 0.001, P ro B A P ro B, P ro A B P ro A (8) Pro B + A 0.1 Pro B A (9) Pro A + B 0.1 Pro A B (10) Reactions 8 and 9 are representing two reactions each: the binding and unbinding of the repressors to the genes promoter regions. We use the same values for time delays in both sub circuits gene expression reactions. Different time delays in the two sub-circuits would affect the transient to reach stability (here used in a loose sense because its a noisy system), but have no effect on long term stability. The circuits are then coupled. The coupling reactions were made assuming that genes pro A and pro 4, and genes pro B and pro 3, have very similar consensus sequences and therefore, share the same inputs. Thus, the 4 genes repressilator proteins react with the toggle switch genes promoter regions as follows: Pro A + p Pro A p 3 (11) Pro B + p 4 1 Pro B p 4 (12) P ro B p 4 P ro B, P ro A p 3 P ro A (13) Reciprocally, the toggle switch proteins react with the repressilator genes promoter regions as follows: Pro 3 + A 1 Pro 3 A (14)

18 Pro 4 + B 1 Pro 4 B (15) P ro 3 A P ro 3, P ro 4 B P ro 4 (16) A detailed justification of the values chosen for delays and number of RNAp s can be found in (Roussel and Zhu, 2006). The reactions rate constants were tuned to attain toggling behavior for each network when uncoupled (toggle switch and 4 genes repressilator), and stability when coupled (with 2 possible ergodic sets). In the next section we present our results. 5 Results We divide this section in two subsections: first, the results using the Boolean networks modelling strategy, and afterward, using the gene networks driven by the multi-delayed SSA. 5.1 Boolean networks As described above, we test on synchronous Boolean networks, when subject to noise, if these networks have 1 or more ergodic sets. We considered networks with N = 6, 10 and 14 nodes. For the RBNs we were able to examine exhaustively, if all genes are subject to perturbations, generically only a single ergodic set exists. We ran independent experiments for a 10 nodes, random topology with average connectivity of 2 and with unbiased random Boolean functions. Not a single case of multiple ergodic sets was found. We also ran, for 6, 10 and 14 genes networks, for k equal to 1,2,3,4, doing 1000 runs for each case (1000 distinct randomly 18

19 generated networks). Again, in these runs, no case of multiple ergodic sets was found. We tested the same in scale free inputs, outputs and inputsoutputs distributions with γ = 2, 2.5, generating randomly 1000 networks for each case. Again, only single ergodic sets were found. Nevertheless, in figures 2 and 3, we show two Boolean networks that do have multiple ergodic sets even when subject to all possible 1 bit perturbations once in the state cycles, hence showing that such systems exist. Notice, in Fig. 3, that one of the ergodic sets consists of two states. This leads to the conclusion that RBNs with multiple ergodic sets, even for this low level of noise, appear to be improbable. Yet, notice that, e.g., for a 10 nodes RBN of average connectivity 2, there are distinct networks as for the wiring diagram and the Boolean functions distributions, thus, it is not possible from our set of experiments to conclude that the conditions for the existence of multiple ergodic sets are very rare, since we only explored a very small sample of the state space, and networks with multiple ergodic sets are not necessarily homogeneously spread in that state space. We then asked ourselves the following question: Could the network have several ergodic sets if a few genes were not subject to noise? Cells, do in fact possess several mechanisms by which they control gene expression, usually by shutting off genes. For example, its is known that in eucaryotes, DNA forms chromatin which serves as a mechanism to control gene expression (Holde, 1989). We tested whether the mean number of ergodic sets was greater than one when only a single gene is perturbed for random k = 1, 2, 3, 4 topologies. We determined the average number of ergodic sets, for random and scale 19

20 free topologies, by perturbing the activities from 1 to N genes (the ones to be perturbed are, in all cases, randomly chosen). We tested this in random topologies with k equal to 1, 2,3 and 4, and in all cases, if a small fraction of the genes could not be perturbed, we found networks that possessed multiple ergodic sets. As an example, in Fig. 4 we plot the average number of ergodic sets, over 1000 networks, of networks with random topologies and k = 2. The number of ergodic sets falls as the number of genes able to flip increases toward N. Very similar results were obtained using scale free topologies of inputs, outputs and both, for γ equal to -2.0 and -2.5 and N = 6, 10, 14. In Fig. 5, we show the results for 14 genes networks and the 3 kinds of scale free topologies with -2 slopes. Given the results above, we went to search, using more realistic modelling strategies of GRNs if, given internal noise (stochastic dynamics) and external perturbations, a simple genetic network could be built, robust to both noise sources. 5.2 A gene network driven by the multi-delayed SSA A gene network was built, according to the reactions 2 to 16. The dynamics is simulated according to the algorithm from (Roussel and Zhu, 2006) and here described. In all cases, we started with the following initial concentrations: RNAp = 1000, all promoters free and no proteins. Also, the results are all plotted in graphs of number of molecules versus time in seconds. The gene network connections diagram can be seen in Fig. 6. The lines ending in small perpendicular lines represent repression reactions on that gene, by the 20

21 Fig. 2. Inputs diagram (top left) of a 3 genes Boolean network, Boolean functions table (bottom left), and the resulting directed graph of all state transitions (top right). The Boolean rules were chosen to obtain a system with two ergodic sets. The state cycles are {(0,0,0)} and {(1,1,1)}. Perturbing, by bit flipping, any single gene when in one of these 2 states, leads to a state of the same state cycle the system was in. Thus, state cycles {(0,0,0)} and {(1,1,1)} are ergodic sets. The boxes are drawn such that include only the states the system can go to, due to single bit perturbations of the states of the ergodic sets. gene from where the line starts. The toggle switch and the 4 genes repressilator are driven by similar reactions apart from the different number of genes, rate constants and delays values, as seen when comparing reactions 2 to 5 and 6 to 10. Notice one cannot define precisely an attractor in a noisy system. We consider 21

22 Fig. 3. Inputs diagram (top left) of a 4 genes Boolean network, Boolean functions table (bottom left), and the resulting directed graph of all state transitions (top right). The Boolean rules were chosen to obtain a system with two ergodic sets. The state cycles are {(0,0,0,0),(0,1,0,0)} and {(1,1,1,1)}. Perturbing, by bit flipping, any single gene of all the states of both state cycle, leads to a state of the same state cycle where the system was. Thus, state cycles {(0,0,0,0),(0,1,0,0)} and {(1,1,1,1)} are ergodic sets. The boxes are drawn so that they include only the states the system can go to, due to single bit perturbations of the states of the ergodic sets. 22

23 Fig. 4. Average number of ergodic sets, for k = 2 random topologies for 6,10 and 14 genes networks, allowing from 1 to N genes to flip. Each point represents the average of 1000 networks. here an attractor as a region of the state space where the system can stay for a long time. The concept of ergodic set remains as previously stated: a region of the state space that once entered, the system cannot leave if not affected by external perturbations. We begin by observing the temporal behavior of the toggle switch and the 4 genes repressilator, when independent from one another (not coupled). Both the toggle switch and the 4 genes repressilator have two unstable attractors (regions of the state space where they stay most of the time) and both toggle 23

24 Fig. 5. Average number of ergodic sets for 14 genes networks, scale free inputs, outputs, and inputs and outputs topologies with γ = 2. Each point represents the average of 1000 networks. from one unstable attractor to the other due to stochastic fluctuations (Gardner et al, 2000; Ribeiro et al, 2006). Therefore, in both cases, only one ergodic set exists (Figs 7 and 8). Comparing the two figures (Figs 7 and 8), its observable that the toggle switch is more subject to noise, i.e., it toggles more times in the same time interval between the two unstable attractors than the 4 genes repressilator. The reason for the 4 genes repressilator to toggle less is that it requires stochas- 24

25 Fig. 6. A 2 genes (A and B) toggle switch and a 4 genes (1, 2, 3 and 4) repressilator coupled by reciprocal repression reactions between genes A and 3 and between genes B and 4. The arrows ending in perpendicular lines represent repression interactions of one gene protein in the other gene promoter. tic fluctuations such that, at the same time both proteins levels of the genes on are low enough so that the other two genes (previously off ) can now start expressing and repressing the two genes previously on. This event is less likely to occur than the stochastic fluctuation of the protein level of the only gene on in the toggle switch, resulting in less frequent toggling in the same time interval. Adding more genes to the repressilator unidirectional chain (keeping the total number an even quantity) would diminish the toggling between the two alternative states even more. Given the toggle switch time series (Fig. 7) and using K-means to binarize the levels of proteins A and B, one observes from the binarized time series that the system, after an initial transient of approximately 2000 seconds, has two unstable single state attractors. One is (A,B) = (0,1) and the other is (A,B) 25

26 Fig. 7. A toggle switch proteins quantities time series. The system toggles from one unstable attractor to the other. = (1,0). These two states are by far the most represented ones in the binarized time series, out of the possible 4 binary states. We note that sometimes one detects also the states (0,0) and (1,1), if the sampling occurred during the transitions between the two unstable attractors. These transitions have very short time duration in comparison with the total time that the system remains on one of the two unstable attractors. The 4 genes repressilator behaves similar to the toggle switch. Using K-means to binarize the time series of proteins p 1, p 2, p 3, and p 4, in Fig. 8, shows that the system, after an initial transient of approximately 2500 seconds, has two unstable attractors. One is (p 1,p 2,p 3,p 4 ) = (1,0,1,0) and the other is (p 1,p 2,p 3,p 4 ) 26

27 Fig. 8. A 4 genes repressilator proteins quantities time series. The system toggles from one unstable attractor to the other. Only p 1 and p 2 time series are shown. p 3 follows the same trajectory as p 1, and p 4 as p 2. = (0,1,0,1). These two states (each a single state unstable attractor) are the most observed states in the binarized time series. Next, the two systems were coupled according to equations 11 to 16. We assume similar consensus sequences between some of the genes of the two sub circuits, which therefore share the same inputs. When coupling the two systems, two ergodic sets emerge, as opposed to the unstable attractors seen before in each of the systems. In Fig. 9, the pro- 27

28 teins expression level corresponding to one of the two possible ergodic sets is observed. Their levels are all approximately constant, aside small stochastic fluctuations, as opposed to what is observed in Figs. 7 and 8, where there is a constant toggling of proteins levels, corresponding to the toggling between the two unstable attractors of the systems. Additionally, the proteins levels of the genes in the on state fluctuate far less in the coupled system. Applying the K-means algorithm to one time series of a single simulation of this gene network, represented in Fig. 6), will result in, depending on the ergodic set reached by the system in that run, either (p 1,p 2,p 3,p 4,A,B) = (1,0,1,0,0,1) or (p 1,p 2,p 3,p 4,A,B) = (0,1,0,1,1,0). In Fig. 9, the ergodic set reached by the system after the initial transient was (1,0,1,0,0,1). We have not observed in all our experiments (above independent experiments) the coupled system leaving any of the two ergodic sets once reaching them, in the parameters range of values here used. Depending on the rates constants, e.g., the ratio between gene expression and decay, the two subsystems can be made more or less stable. For example, if the transcription rate constants are set to very small values or decays to very high, the system will never be able to reach any of the ergodic sets and will indefinitely remain on a long transient with very small total number of proteins. In this extreme regime, stochastic fluctuations of the proteins time series would be the predominant dynamical feature and no ergodic set would be observed. It is interesting to notice that, if an analogy is to be made with the Boolean networks dynamics, that is, convert the time series to a binary series, a zero corresponds to null or very near null concentrations. Higher decay would move the threshold between 0 s and 1 s to a higher value (relatively to the maxi- 28

29 mum concentrations of proteins observed). Fig. 9. Coupled repressilator and toggle switch. No toggling due to internal noise is observed. Only p 1 and B proteins are shown. Proteins p 2, p 4 and A quantities are null after a short transient, while protein p 3 has the same level as p 1. For example, the binarization using the K-means method of the system state in Fig. 9 after the initial transient, would assign the following Boolean states: p 1, p 3 and B = 1, p 2, p 4 and A = 0. These values do not change over time, since the proteins concentrations are fairly stable aside stochastic fluctuations. Figure 9 shows that this system multiple ergodic sets are resistant to the internal noise due to stochastic fluctuations. 29

30 Notice that in this more realistic model of GRNs, because it is based on the multiple delayed SSA, the gene expression levels, when observed in detail, show that proteins are produced by bursts (for a very detailed study on these GRNs models dynamics at the single gene level see (Zhu et al, 2006) and for studies with experimental measurements see (Chubb et al, 2006; Golding et al, 2005; Blake et al, 2006)). Additionally, decay (which can happen to proteins even when bound to the promoter) is also a stochastic process and so promoters states are inherently stochastic. The gene network here modelled was always resistant to all these forms of internal noise, i.e., once the ergodic set was reached, a transition to the other possible ergodic set was never observed in very long time intervals (10 7 s). Thus, it is an example of a GRN driven by the delayed SSA with multiple ergodic sets. Since the system ergodic sets proved to be resistant to any internal fluctuations (i.e. stochastic fluctuations in promoter states and proteins decay, among others), and because cells can also be affected by external perturbations, we wanted to see how our gene network reacted to external perturbations. We observed the system response to external perturbations that consist in the addition or removal of large amounts of proteins. For all proteins, we changed its concentration from high to low or vice versa, one at a time (corresponding to flipping the protein level from 0 to 1 or vice versa after binarized via K-means). Additionally, these perturbations are similar to testing what would happen if such very large (extremely unlikely) internal fluctuations of proteins concentrations occurred. Because we never observed these rare events in the cases where no external perturbation is made, we imposed them to test 30

31 if they would be sufficient to make the system leave the ergodic set. Here we present in figures 10 and 11 a perturbation to the inactive gene of the toggle switch, and another to an inactive gene of the repressilator, as examples. In all cases, the coupling was strong enough to maintain the steady state that the ergodic set consists of. In figure 10, a perturbation was introduced externally to the toggle switch, by adding 500 A molecules to the system at each seconds. As seen, due to the coupling, the perturbation was unable to remove the system from its stable state. We also tried adding at each seconds, with the same results (data not shown). In figure 11, a perturbation was introduced externally to the repressilator, by adding 500 p 2 molecules to the system at each seconds. As seen, due to the coupling to the toggle switch, the perturbation was unable to remove the system from its stable state. Notice that sometimes, protein B and p 1 went down due to the perturbation but, due to coupling they recovered the previous levels, after all proteins externally introduced, decayed. Also, we tried adding all other proteins, one at a time, and again, due to the stabilization given by the coupling, the system did not changed state (data not shown). In all cases, 100 experiments were made, all analyzed independently and with similar results, but we plot only the results of single experiments as an illustration, since plots of average behavior would not allow seeing perturbations effects. Also interesting to notice is that the time series of both repressilator and toggle switch in Figs. 10 and 11 are much less noisy than their time series when uncoupled (Figs 7 and 8). As observed from the previous cases, this GRN requires more than 1 pertur- 31

32 Fig. 10. Perturbation of the toggle switch by adding A s at each seconds. The system remains stable. bation at a time (more than one protein level significantly changed), due to the coupling between the toggle switch and the 4 genes repressilator. The ergodic sets are stable to all possible stochastic fluctuations, e.g., on proteins quantities and promoter states. Only fluctuations that persist for a long time in one direction and happening to at least three of the six proteins at the same time, would make the system change ergodic set. 32

33 Fig. 11. Perturbation of the repressilator by adding p 2 s at each seconds. The system remains stable. We tested the system response to simultaneous perturbations, i.e., adding more than 1 kind of proteins at the same time. Our simulations results are that (data not shown), only when introducing the three kinds of proteins not present in the system at the moment the perturbation is imposed, can the system be removed from the ergodic set he is in. Another possibility would be removing at a given moment, all proteins present in the system, included those bound to promoters, thus resetting the system state. For example, if 33

34 the levels of proteins p 1, p 3 and B are high, only adding to the system, at the same time, many proteins p 2, p 4 and A, will the system, in some cases, leave the ergodic set he is in. Notice that the time series of the proteins in the coupled system (Fig. 9) are much less noisier than in the two uncoupled systems (Figs. 7 and 8). For that reason, perturbing the levels of two of the three proteins holding state, is not sufficient to remove the system from its ergodic set, while, e.g., perturbing the level of 1 of the 2 proteins holding state in the uncoupled 4 genes repressilator, is sufficient to change its state. Given a detailed analysis of the set of reactions defining the coupled system, one identifies, at least, two parameters whose values must be set within a certain interval of possible values, for the system to behave as described. Also, these parameters need to be considered together (varying one may be compensated with the opposite variation of another): i) the slower is the proteins rate constant of decay, the more likely is one perturbation to affect the system state since the externally added proteins will remain in the system for a longer period, ii) the larger is the time delay of promoters release in the transcription/translation reaction, the less likely is that a perturbation causes any effect on the system dynamics since the promoters will be, on average, a longer time unavailable for reactions, resulting in more time for externally added proteins to decay without affecting the system state. Other parameters, such as the rate constants of binding and unbinding of repressors to promoters, also play an important role. In the coupled system, given our parameters values, fluctuations resulting in the events able to force the system to leave its ergodic set are very unlikely and 34

35 can be made even less likely by the coupling of more switches (or rate constants tuning, e.g.). The probability of the necessary consecutive events described to occur is thereby extremely remote and can be considered non-existent in a realistic time scale. For example, never do any proteins concentrations reach, due to stochastic fluctuations, levels as those imposed by us when adding external perturbations. Symmetrical experiments were made when the system chose the other ergodic set with the same results. In all cases, the system never went from one ergodic set to another, thereby we conclude that this system has two ergodic sets which can be reached from the same initial condition. Starting from a different initial condition, for example, non null proteins concentrations biased towards one of the ergodic sets, or one of the genes repressed, is equivalent to start from an initial state, in the Boolean framework, belong to one of ergodic sets basin of attraction. The results obtained in this section are robust to some variations of the parameters values. Changing any of the rate constants and time delays by a factor of 10 did not change the system dynamics significantly, and its only consequence is varying the proteins levels at equilibrium that depend on the relationships between rate constants of production and decay, and time delays of the transcription/translation reactions. 6 Discussion We tested whether in noisy Boolean networks, in the regime of low noise (slow dynamics versus the system fast dynamics), multiple ergodic sets exist. 35

36 If all genes can be perturbed, the results on Boolean networks we report here show that, although possible, (Fig 2), the property of having multiple ergodic sets is not generic in the ensembles examined. Yet, if a fraction of nodes are protected from perturbations, for which we presented concrete possible mechanisms, multiple ergodic sets do exist, even in the small region of the network space we searched and for very small networks (and thus with very small number of attractors even without the presence of noise). Namely, we observed in the Boolean framework that, for RBNs of 6 nodes, random topology and average connectivity of 2, one of the six genes had to be protected from noise fluctuations to attain a few networks (out of a set of 1000 randomly generated) with more than 1 ergodic set. For RBNs of 10 nodes, 2 genes needed to be protected and, for RBNs of 14 nodes, 3 nodes had to be protected. Unfortunately, due to the simulations computational complexity, we do not know if, as the number of nodes increases, it will require a larger or smaller fraction of nodes to be protected against noise for multiple ergodic sets to appear. It is likely however, that some networks will not require protection against noise of any node because even for small networks there are such cases. In this framework, as more nodes are protected against noise, the number of randomly generated networks that possess multiple ergodic sets grows significantly as our results show, but perhaps more relevant is to point out that RBNs with multiple ergodic sets do exist even when no node is protected from noise and the fact that only a few exist not necessarily makes them impossible to attain. For example, it could be the case that they can be selected for, rather than randomly attained. One could, for example, imagine 36