Structure and Dynamics of Artificial Regulatory Networks Evolved by Segmental Duplication and Divergence Model

Transcription

1 International Journal of Automation and Computing 7(1), February 2010, DOI: /s Structure and Dynamics of Artificial Regulatory Networks Evolved by Segmental Duplication and Divergence Model Xiang-Hong Lin 1,2 Tian-Wen Zhang 1 1 School of Computer Science and Technology, Harbin Institute of Technology, Harbin , PRC 2 School of Mathematics and Information Science, Northwest Normal University, Lanzhou , PRC Abstract: Based on a model of network encoding and dynamics called the artificial genome, we propose a segmental duplication and divergence model for evolving artificial regulatory networks. We find that this class of networks share structural properties with natural transcriptional regulatory networks. Specifically, these networks can display scale-free and small-world structures. We also find that these networks have a higher probability to operate in the ordered regimen, and a lower probability to operate in the chaotic regimen. That is, the dynamics of these networks is similar to that of natural networks. The results show that the structure and dynamics inherent in natural networks may be in part due to their method of generation rather than being exclusively shaped by subsequent evolution under natural selection. Keywords: Genetic regulatory network (GRN), artificial regulatory network (ARN), segmental duplication and divergence, scale-free, small-world, largest Lyapunov exponent. 1 Introduction Genetic regulatory networks (GRNs) are patterns of directed interactions between transcription factors and their target genes. Recently, new array technologies have made it possible to determine where in the genome various transcription factors bind [1]. Since transcription factor binding to the cis-regulatory region of the gene strongly influences the expression level of a given gene, these data provide links between the expression of transcription factors and other genes and allow construction of a network. The GRN of an organism plays a central role in the control of cellular processes, such as the response of a cell to environmental signals and the differentiation of cells and groups of cells in the unfolding of developmental programs. Studying models of GRNs can help us understand some of these mechanisms and provide valuable lessons for both natural and artificial systems [2]. Gene duplication and divergence mechanism is important for evolution of GRNs. Many authors have therefore considered gene duplication and divergence as a driving force in shaping the structure of GRNs [3 5]. The hypothesis underpinning these so-called duplication and divergence models is that regulatory networks are evolved by random gene duplication and by random rewiring of genetic regulatory links. Specifically, it has been noted that evolution via duplication and divergence would act as an implicit way of preferential attachment and thereby provide the observed power law degree distributions of regulatory networks [6 9]. These models generally assume that gene is a random node and assume the connections between the node and all of the neighbors in the network; thus, duplication and divergence operators are confined at the network level. However, biological duplication and divergence act upon the underlying nucleotide sequence; changes at the genome level do not necessarily have a one-to-one correspondence with network Manuscript received September 18, 2008; revised April 20, 2009 changes. Recently, comparative studies have suggested that segmental (or block ) duplication and divergence the duplication and divergence of large DNA segments have been a continuing process during evolution. Segmental duplication and divergence are those in which many genes and their upstream regions are duplicated or diverged in a single event [10 12]. In this paper, we introduce a segmental duplication and divergence model for evolving artificial regulatory networks (ARNs). The ARN can be extracted from an artificial genome that is represented as a sequence analogous to the string of nucleotides forming a real genome. An important, but currently unexplored, advantage of this approach is that it allows the segmental duplication and divergence operators to be applied to the genome sequence whose effect can then be observed at the level of the network. The aim of this paper is to explore and analyze the structure and dynamics of the ARNs evolved by the segmental duplication and divergence model. 2 Models and method 2.1 The ARN model Genes are always regulated by a number of upstream regulatory genes through their products; for instance, transcription factors bind to cis-regulatory elements in the DNA regulatory region. Certain classes of cis-regulatory elements are usually called enhancers. These are cis-regulatory elements that, at least sometime, positively regulate the gene expression. Others are called inhibitors that inhibit expression of any gene in their vicinity but all perform their specific functions when and where they do because they bind proteins that recognize their specific DNA target site sequence. In brief, gene transcription depends on the GRN architecture. Therefore, the comprehension of gene transcriptions needs to model the corresponding GRN from the

2 106 International Journal of Automation and Computing 7(1), February 2010 sequence-dependent regulatory genome. The ARN model considered here consists of a linear sequence of bases drawn from the set D = {0, 1, 2, 3} (analogous to the four bases, A, C, G, and T in DNA), which represents a genome with a directional readout comparable to the 5 3 direction in DNA [13]. The artificial genome carries genes, whose beginning is marked by a unique promoter pattern (consisting of a particular sequence of bases). In this model, the sequence 0101 is identified as a promoter signaling the start of a gene analogous to the TATA sequence in biological genomes. A promoter sequence of 4- bases was arbitrarily drawn from the set D to be 0101 ; the promoter pattern can be expected to appear with probability P = 4 4 = 0.39%. Since the promoter pattern itself is repetitive, overlapping promoters or periodic extensions of the pattern are not allowed, i.e., a sequence of (6-bases) is detected as a single promoter site starting at the first base. The g bases immediately downstream of every promoter are identified as a gene (by default, g = 30). Every gene encodes for a protein, also represented by a string of p characters drawn from the set D (by default, p = 6). Some genes encode functional proteins, and the remainders encode transcription factors that regulate other genes. In our model, a majority rule is applied to the gene yielding a shorter sequence that represents the protein. The majority rule works like this: subsequences of the gene of length p are written underneath each other. The number of occurrences of each base per position of all subsequences is counted. Therefore, the base occurring most often is the correct base of the protein in the corresponding position. If at any position two or more bases occur with the same number of times, the base being first if sorted alphabetically takes precedence over others. A complete example is shown in Table 1, where the gene has the following sequence and the protein is Table 1 Sample majority rule for translating a gene into a protein with gene length 30 and protein length 6 Gene Occurrences {0,1,2,3} 3,2,0,0 1,0,1,3 1,2,2,0 1,2,1,1 1,0,3,1 0,3,1,1 Protein In eukaryotic genomes, cis-regulatory elements can be located either upstream or downstream of the actual gene [14]. Here, cis-regulatory elements are only located upstream of the promoter; this is a simplification that does not affect the fundamental principle. As previously mentioned, matching of transcription factors to these sites results in changes to protein production for the corresponding genes. We allow binding sites to overlap but do not allow promoter sites or genes to overlap with one another. Once bound, a transcription factor regulates the closest downstream gene. The nature of this regulation is determined by a weight sequence that is a string of r characters immediately following the binding site (by default, r = 6). The regulation will activate gene expression unless base 0 occupies this first position of the weight sequence, in which case it is inhibited (hence, on average, 25% of regulation is inhibitory). In addition, the strength of the regulation is calculated by converting the base-4 weight sequence, except the first character to a real-valued weight in the range [0, W ], where W defines the interaction strength (or weight scale) of the ARN model. Previous models of ARNs primarily use Boolean representations of network dynamics [13,15,16]. Here, we show that an ARN model using standard recurrent neural network architecture [17] can display complex dynamic behaviors. The state of the ARN is updated synchronously in discrete time steps, with the activation of gene i at the time t + 1, a i(t + 1), given by ( N ) a i(t + 1) = σ w ija j(t) θ i j=1 where N is the number of genes, a j(t) is the activation of gene j at the previous time step, and w ij is the level of the regulatory interaction from gene j to gene i. θ i is the activation threshold of gene i, given by θ i = (1) N w ijδ (2) j=1 where δ is the weight bias (by default δ = 0.5). σ(x) is the sigmoid function, given by 1 σ(x) =. (3) 1 + e x Therefore, the activation of each gene is a continuous variable in the range [0, 1], where 0 represents a completely inactive gene and 1 a fully expressed gene. Fig. 1 depicts the model in hand-constructed and slightly simplified form. In Fig. 1, a simple promoter sequence 01 occurs thrice within this genome. The 9 characters immediately downstream each promoter are genes (labelled A, B and C). The three genes can be translated into proteins using the majority rule, that is, A: , B: , and C: Upon expression, the products of genes A and B are transcription factors that are free to bind to any matching sequence within the genome, and the product of gene C is a functional protein. Once bound, the nature of the regulation is determined by the 3 characters immediately following the binding site. It is simple to derive a network of gene regulatory interactions from an artificial genome (see Fig. 1 ). In Fig. 1, gene A activates the expressions of genes B, C, and itself, and gene B inhibits the expression of gene C. The interactions between genes can be represented by a weight matrix, in which the entry at row i, column j specifies the influence that gene j has on gene i. These entries may be positive or negative depending on whether the product produced by gene j is an enhancer or inhibitor in the cis-regulatory region of gene i. A zero entry indicates that there is no interaction between the two genes (see Fig. 1 (c)). In Fig. 1 (c), the matrix entry at row i, column j specifies the influence that gene j has on gene i. Given a particular ARN, the gene activations over time may be generated by the above update scheme (see Fig. 1 (d)). The state of the ARN is updated synchronously in discrete time steps, and the initial activations of genes A, B, and C are 1, 0.5, and 0, respectively.

3 X. H. Lin and T. W. Zhang / Structure and Dynamics of Artificial Regulatory Networks 107 Fig. 1 Gene expression and regulation in the ARN model (with a simplified promoter and short gene and protein length). A simple hand-constructed genome; The ARN resulting from the genome depicted in ; (c) The weight matrix of the ARN (W = 5); (d) The gene activation over time generated by the ARN For randomly generated genomes, the distance between genes cannot be derived exactly, but an approximation can be made to produce a good estimate [16]. The distance between genes can be thought of as the number of trials, moving along the genome, to obtain a success of finding a promoter sequence and hence the presence of a gene. Thus, this distance variable, d, can be shown to follow a geometric distribution with probability P. According to the geometric distribution, the mean distance between genes is given by µ d = 1 P. (4) However, genes are not allowed to overlap and hence the length of the gene should be taken into account, such that the mean distance between genes is given by µ d = 1 + g. (5) P 2.2 The segmental duplication and divergence model Segmental duplication and divergence of genome is a central mechanism in evolution of GRNs. At the macro level, where the duplicated element is of the order of kilobases of DNA, it gives rise to evolution of network architecture through the duplication and divergence of the genome. At the micro level, where the duplicated element is of oligonucleotide size, it generates novel protein and regulation coding sequences from the duplication and divergence of small segment. By modeling an artificial genome at the level of nucleotide sequences and mapping it to an ARN, biologically grounded segmental duplication and divergence operators can be mapped to the network level [18]. The simplest class of operators is a single-point mutation, in which a single random nucleotide in the genome is changed to a different value. This operator generally has a localized effect on the activity of a single gene. The second class of mutation operators is segmental duplication and divergence, which can be intragenic, affecting only one gene, or multigenic, affecting a number of genes. These operators include: 1) Segmental duplication: a random segment of the genome is duplicated. Segmental duplication events can be either tandem, in which case the copied region adjoins the original region, or non-tandem, where the copied region may be in another part of the genome. In our simulations, all duplication events are tandem. 2) Segmental deletion: a random segment of the genome is removed. 3) Segmental translocation: a random segment of the genome moves to another random location in the genome. 4) Segmental inversion: the values of a random segment of the genome rewrite in an inverse order. The final class of operators results from the actions of transposons. Transposons are discrete segments of the genome that are able to transport themselves to other locations in the genome. Segmental duplication and divergence operators can result both from the insertion of a transposon into a gene and from modifications made to the gene sequence when it leaves. In the model, we assume that the size of transposons is fixed and that the value is the mean distance between genes µ d in the genomes. So, on average, a segmental duplication or divergence event affects a single gene. In addition, we also assume that the number of transposons in the genome is l/µ d, where l is the length of the genome sequence. That is, the value equals the average number of genes in the genomes. 2.3 The evolutionary algorithm Our aim is to explore and analyze the structure and dynamics of the ARN model. For analytic simplicity, the genomes are randomly generated with the same length and evolved by the segmental duplication and divergence model. We focus on non-growing genomes, where duplication of a transposon is associated with removal of another one that

4 108 International Journal of Automation and Computing 7(1), February 2010 equals in size the duplicated transposon and combines the segmental duplication and deletion operators into a new segmental duplication-deletion operator. The evolutionary algorithm represents the time by a sequence of nonoverlapping generations in the population. In the turnover from one generation to the next, individuals reproduce to form offspring without selection pressure. That is, the individuals in the population have the same fitness. In every generation, individual evolution consists of single-point mutation with probability p point, segmental duplicationdeletion with probability p dud, segmental translocation with probability p trans, and segmental inversion with probability p inv. In the simulation conditions, the structure and dynamics of the ARNs are only determined by the mutation operators. The evolutionary algorithm with the segmental duplication and divergence model is shown in Algorithm 1. Table 2 lists the common parameter settings of segmental duplication and divergence operators in all experiments. Algorithm 1. The evolutionary algorithm with the segmental duplication and divergence model Step 1. Randomly generate an initial population of ARNs with the same genome length. Step 2. Evolve each ARN at the genome level. Step 2.1. Apply single-point mutation operator with probability p point for each nucleotide in the genome. Step 2.2. Apply segmental duplication and divergence operators for each transposon in the genome. Randomly select two transposons in the genome, apply segmental duplication-deletion operator with probability p dud ; Randomly select a transposon in the genome, apply segmental translocation operator with probability p trans; Randomly select a transposon in the genome, apply segmental inversion operator with probability p inv. Step 3. Replace the current population with the new population. Step 4. Go to Step 2 until terminating generation. Table 2 The common parameter settings of the evolutionary algorithm in all experiments Parameter Symbol Value Single-point mutation probability p point Segmental duplication-deletion probability p dud 0.1 Segmental translocation probability p trans 0.01 Segmental inversion probability p inv Network structure in the ARN model Since one of the most basic features of any complex network is its structure, it is natural to investigate network connectivity. In this section, we shall concentrate on the following two aspects of network structure, which have received most attention in the last few years: scale-free structure and small-world structure. In the simulations, we analyzed the scale-free and small-world structures of the ARNs evolved by the segmental duplication and divergence model, and the resulting configurations of genomes were analyzed after 100 generations. As a basis for comparison, we also analyzed the structures of the original ARNs, which were randomly generated with the same genome length. 3.1 Scale-free structure in the ARN model A simplistic measure of the variation of network structures from the model can be obtained from the variation of the average degree of the networks. Fig. 2 shows the average degrees of the original and evolved ARNs against the genome length, which ranges from to Each data point is averaged over 100 ARNs with a fixed genome length. The average degree of the ARNs increases lineally with the genome length, and the average degree of the evolved ARNs is smaller than that of the original ARNs. Fig. 2 Average degrees of the original and evolved ARNs against the genome length The variance of the average degree gives a scalar measure of the network structure, but it does not provide a complete description of the network structure. Much of the recent research on the structure of complex networks has focused on determining the form of their degree distributions, P (k), which measures the proportion of nodes in the network having degree k. It was reported in a variety of biological systems [19 23] that the degree distributions are described by a power law of the form, P (k) = k γ, γ > 1. Networks with degree distributions of this form are now commonly referred to as scale-free networks. The power law degree distributions observed in these biological networks are not consistent with the traditional random network models, which have been used to describe complex networks [24,25]. In these models, the degree distributions are Poisson or Gaussian. It has been noted that the scalefree structure has implications for the robustness and vulnerability of networks to failure and attack. Specifically, removing most of the nodes in a scale-free network will have little effect on the connectivity of the network [26]. Fig. 3 shows the degree distributions for the populations of the original and evolved ARNs, which consists of 100 ARNs with genome length l = (see Fig. 3 ) and (see Fig. 3 ). We see that the degree distribution of the original ARNs produces a peak at nonzero k and can be described by a characteristic bell-shaped curve, where the degrees of most genes are close to the average degree and the degree distribution decreases exponentially on ei-

5 X. H. Lin and T. W. Zhang / Structure and Dynamics of Artificial Regulatory Networks 109 ther side of the maximum. That is, the degree distribution appears to be similar to that of random networks. However, the degree distribution of the evolved ARNs appears to be similar to that of scale-free networks, with many genes having no or very few connections, and a few genes having many connections. In order to determine the influence of segmental duplication and divergence operators on the degree distribution of the evolved ARNs, investigations were undertaken into the effects of different values for p dud, p trans, and p inv. Fig. 4 shows the effects of p dud, p trans, and p inv on degree distribution of the ARNs evolved through the segmental duplication and divergence model. The results were averaged over 100 ARNs with genome length l = With p trans and p inv being constant of 0.01, the degree distributions for different parameter values of p dud are shown in Fig. 4. It is interesting to see that the degree distribution is more similar to that of scale-free networks for large p dud. As p dud increases, the degree distribution appears to fit power law decay. It falls on a straight line in a log-log plot as p dud = 1. However, with p dud being constant of 0.1, Fig. 4 shows that the degree distribution is more similar to that of random networks as p trans and p inv increase. That is, higher probability of segmental divergence operators, such as translocation and inversion, generates networks that are close to the ones having random connectivity. Fig. 3 Degree distributions of the original and evolved ARNs. The results were averaged over 100 ARNs with genome length ; The results were averaged over 100 ARNs with genome length Fig. 4 Effects of p dud, p trans, and p inv on degree distribution of the ARNs evolved through the segmental duplication and divergence model. Each different value of segmental duplicationdeletion probability (p dud = 0.001, 0.01, 0.1, and 1) or segmental translocation and inversion probability (p trans, p inv = 0.001, 0.01, 0.1, and 1) results in a different degree distribution 3.2 Small-world structure in the ARN model Another structural feature found in large complex networks is the so-called small-world structure. The first small-world property is the average shortest path length, also known as characteristic path length, defined as the average of shortest path lengths over all couples of nodes [27]. A problem with this definition is that the characteristic path length diverges if there are disconnected components in the network. Since in our simulations the network is not always strongly connected, we use an alternative way for characteristic path length: ( 1 ) 1 1 L = (6) N(N 1) d ij i,j N,i j where N is the number of nodes, N is the set of nodes in the network, and d ij is the shortest path length from node i to

6 110 International Journal of Automation and Computing 7(1), February 2010 node j. The characteristic path length L thus defined can be computed from all pairs of nodes irrespective of whether the network is connected. The second small-world property is the clustering coefficient, which measures the cliquishness of a network. The clustering coefficient is defined as the average of the local clustering coefficients over all the nodes in the network: C = 1 C i = 1 e i N N k i(k i 1) i N i N where C i is the local clustering coefficient of node i with k i neighbors sharing e i directed edges. At variance with random networks, the small-world property in real biological networks is often associated with the presence of clustering, denoted by high values of the clustering coefficients [22,28,29]. For this reason, Watts and Strogatz [27] proposed to define small-world networks as those networks having both small values of the characteristic path length L, like random networks, and high values of the clustering coefficient C, like regular lattices. To test whether the ARNs could also be classified as having small-world structure, we calculated the characteristic path length L and the clustering coefficient C for these networks and compared to random networks [24] with the same number of nodes and average number of edges per node. According to the above definition, small-world networks satisfy both L/L random 1 and C/C random 1, which are taken to be those most characteristic of the small-world structure. The distributions of L/L random and C/C random obtained from 200 ARNs with genome length l = are shown in Fig. 5. The average values of L/L random and C/C random in the evolved ARNs are 1.71 ± 0.23 and 3.69 ± 1.28, and the values are all larger than those values in the original ARNs. However, on average, L/L random increases slowly, only 38.33% larger than that of the original ARN, while C/C random increases rapidly and significantly, reaching a value that is % larger than that of the original ARN. It can be derived from the simulation that a majority of evolved ARNs approach or satisfy the definition of smallworld structure. (7) In order to analyze the small-world structure of the evolved ARNs more comprehensively, we calculated L/L random and C/C random for the different genome length. Fig. 6 shows the average values of L/L random and C/C random in the original and evolved ARNs. Each value was averaged over 100 ARNs with a fixed genome length. For the evolved ARNs, the average value of L/L random is 1.25 for l = and increases rapidly to 2.42 as l increases to before decreasing to 1.52 as l increases to The average value of C/C random has the same change trend, but its peak value is 4.64 for l = and decreases to 3.14 as l increases to (see Fig. 6 ). For the original ARNs, the average values of L/L random and C/C random are all smaller than those values of the evolved ARNs. The peak values of L/L random and C/C random are 1.37 and 2.21 for l = (see Fig. 6 ). This is a very interesting finding, indicating that the tendency to form small-world structure is clear whatever the length of genome is. Fig. 6 Average values of L/L random and C/C random for original and evolved ARNs against the genome length Fig. 5 Plot of L/L random and C/C random of the original and evolved ARNs We also investigated the effects of p dud, p trans, and p inv on small-world structure of the ARNs evolved through the segmental duplication and divergence model, and the results were averaged over 100 ARNs with genome length l = Fig. 7 shows that the average values of L/L random and C/C random increase as p dud increases, in which case p trans and p inv are constant of Inversely, with p dud being constant of 0.1, the average values of L/L random and C/C random decrease as p trans and p inv increase (see Fig. 7 ). The segmental duplication-deletion operator tends to enhance the clustering coefficient. The

7 X. H. Lin and T. W. Zhang / Structure and Dynamics of Artificial Regulatory Networks 111 segmental translocation and inversion operators tend to reduce the characteristic path length. The results show that the small-world structure inherent in the networks may be in part due to their method of generation. Specifically, the segmental duplication and divergence operators play an important role in this process. Fig. 7 Effects of p dud, p trans, and p inv on L/L random and C/C random of the ARNs evolved through the segmental duplication and divergence model. Each different value of segmental duplication-deletion probability (p dud = 0.001, 0.01, 0.1, and 1) or segmental translocation and inversion probability (p trans, p inv = 0.001, 0.01, 0.1, and 1) results in the different average values of L/L random and C/C random 4 Network dynamics in the ARN model In the previous section, the structure of the underlying ARN model was investigated. However, network structure is only one of the aspects of ARNs. The network dynamics gives rise to the myriad of functions observed in natural systems. Here, we examine the dynamics of the ARN model by attempting to analyze the two possible regimens of dynamical behavior, ordered and chaotic behavior. 4.1 Method for exploring network dynamics Dynamical systems theory organizes the state space of a system into regions called basins of attraction, which contain attractor states [30,31]. Attractors in an ordered system can be thought of as a set of equilibrium states and can exist as either point attractors or cyclic attractors. A distinguishing feature of both point and cyclic attractors is that the behavior of trajectories within their respective basins of attraction is predictable. Given any initial condition in the basin of attraction of a fixed point or limit cycle, we know that it will approach the point or cyclic attractor as t. There is a further type of attractor, the chaotic attractor, which may exist in chaotic systems. A chaotic attractor is sensitive to the initial conditions. That is, the trajectories of two initial conditions, x 0 and x 0, whose initial separation δ 0 is very small, will diverge at an exponential rate. Therefore, at some point t in the future, it will no longer be possible to predict the behavior of the trajectory x t based on x t. As described above, in a chaotic system, two trajectories that have separation δ 0 at time 0 will diverge over time. If δ t is the separation at time t, and δ t δ 0 e λt then λ is known as the largest Lyapunov exponent, and it measures the exponential rate at which the two trajectories will diverge [32]. Alternatively, in an ordered system, the largest Lyapunov exponent λ will be negative, indicating the rate at which two nearby trajectories converge to a point or cyclic attractor. The largest Lyapunov exponent λ is defined as λ = lim t ln ( δt δ 0 ). (8) The magnitude of the largest Lyapunov exponents provides a quantitative picture of system dynamics in information theoretic terms, measuring the rate at which systems create or destroy information [33]. For time series produced by dynamical systems, the magnitude of a positive characteristic exponent indicates a chaotic attractor. The magnitude of a negative characteristic exponent indicates a point or cyclic attractor. Fortunately, several methods exist for estimating the value of this exponent from time series data [33,34]. This technique has previously been applied to the analysis of high-dimensional neural networks [35]. In this paper, we adopt Sprott s method [34] to estimate the value of the largest Lyapunov exponent. For an initial state, we calculate the largest Lyapunov exponent from the time series produced by a dynamical system. In the first step, the sign of the largest Lyapunov exponent is used in classification to identify the existence of at least one chaotic attractor. Next, the set of states in the attractor is used as the basis for distinguishing between nonchaotic attractors. Point attractor can be identified by the equality of successive states (f t(x) = f t+1(x)). Cyclic attractor of length h can be identified by the equality of states h time steps apart (f t(x) = f t+h (x)). A final possibility is quasi attractor that refers to a trajectory for which the largest Lyapunov exponent is negative (indicating order), but no repeating states are observed over the duration of the calculation. Possible explanations for this case include very long transients or cyclic attractor, or estimation errors in the largest Lyapunov exponent calculation. A dynamical system frequently contains multiple attractors, which may belong to different type for example, point, cyclic, or chaotic attractor depending on the initial state. A more detailed analysis of system dynamics can be obtained by using multiple initial states. Using multiple initial states raises the issue of how to sample the state space of the system. An initial state consists of a vector of

8 112 International Journal of Automation and Computing 7(1), February 2010 continuous values; therefore, there is an infinite number of them, and no possibility of exhaustively testing the entire space of the system. In the ARN model, we use the random sampling method, in which the activation of each gene is chosen from a uniform distribution in the range [0, 1]. 4.2 Characteristics of dynamic space The dynamical behaviors of a system, including its attractors, depend on its structure. The structure of an evolved ARN, as analyzed in Section 3, differs from that of the original ARN. Therefore, we use populations of the original and evolved ARNs to explore the range of dynamical behaviors generated by the networks and to characterize how the diversity of these behaviors changes with network structure. The population consists of 100 ARNs with genome length l = For each of the base networks, 40 values of weight scale W are tested in the range [0.5, 20]. That is, networks in total are generated for the population of the evolved ARNs and networks are generated for the population of the original ARNs. Five hundred different initial states are tested for each network. The network is initially iterated for steps to ensure that the trajectory is located on an attractor, rather than a transient, and the largest Lyapunov exponent and the type of attractor are calculated over the subsequent time steps. For each network, the number and type of unique attractors are recorded. The first statistic calculation is the probability of the different attractors in a network. Fig. 8 shows how the probability of the point, cyclic, chaotic, or quasi attractors varies with W. The following trends are observed. The probability of the point attractors is 1 for W = 0.5, but drops rapidly to 0.14 for the original ARN and 0.3 for the evolved ARN as W increases to 5, and remains at this level as W increases further (see Fig. 8 ). The probability of the cyclic attractors is 0 for W = 0.5 and increases gradually as W increases, approaching 0.86 for the original ARN and 0.67 for the evolved ARN. In addition, the probability in the evolved ARN is larger than that value in the original ARN for W < 14 (see Fig. 8 ). For large weight scales (W > 3), the probability of the chaotic (see Fig. 8 (c)) or quasi (see Fig. 8 (d)) attractors in the evolved ARN is much smaller than that value in the original ARN. That is, the evolved ARN contains less chaotic and quasi attractors than the original ARN. The second statistic calculation based on the ARN populations is the average number and length of stable attractors (point or cyclic) found in a particular network. Fig. 9 shows the average number and length of stable attractors for the two ARN populations. The general trend for both populations is that the number of stable attractors increases as W increases, while the original ARN generally contains more stable attractors than the evolved ARN, below W = 15 the difference is minimal (see Fig. 9 ). However, the average length of stable attractors in the evolved ARN is much smaller than the length in the original ARN (see Fig. 9 ). It has been a plausible and long-standing hypothesis that GRNs of real cells operate in the ordered regimen or at the border between order and chaos. This hypothesis is indirectly supported by the robustness and stability observed in the phenotypic traits of living organisms under genetic (c) (d) Fig. 8 Probabilities of point, cyclic, (c) chaotic, or (d) quasi attractors for the original and evolved ARNs with genome length (Each data point corresponds to a probability calculated over 100 ARNs for the given weight scale.)

9 X. H. Lin and T. W. Zhang / Structure and Dynamics of Artificial Regulatory Networks 113 Fig. 9 Averages of number and length of stable attractors for the original and evolved ARNs with genome length (Each data point corresponds to a number or length calculated over 100 ARNs for the given weight scale.) perturbations. What is more important is that Shmulevich et al. [36] developed theoretical models to verify the hypothesis. Using the Boolean approach, they have shown that the underlying GRN of HeLa cells appears to operate either in the ordered regimen or at the border between order and chaos but does not appear to be chaotic. According to the above statistics, we find that the probability is higher to operate in the ordered regimen, and lower in the chaotic regimen for the evolved ARNs. That is, the dynamics of these networks is more similar to that of natural regulatory networks. 5 Conclusions We propose a specific ARN model based on a model of network encoding and dynamics called the artificial genome. The ARNs are evolved by a segmental duplication and divergence model. The segmental duplication and divergence model was chosen since it has been previously implicated as an important factor in the evolution of genomes and due to its simplicity. Specifically, these networks are examined from the perspective of the structural properties and dynamical behaviors. Using analytic and simulation techniques, we find that the ARNs have degree distributions between those of random networks and scale-free networks and display small-world structure. We also find that these networks are more ordered and less chaotic in their dynamical behavior. Therefore, we believe that the structure and dynamics follow naturally from the processes of generation of the networks rather than being exclusively shaped by subsequent evolution under natural selection. Why does the segmental duplication and divergence model create scale-free and small-world structures? Part of the answer is that the segmental duplication at the genome sequence level constrains the new gene to be regulated by a subset of existing cis-regulatory elements, which can be considered to be similar to the mechanism of preferential attachment at the network level. Preferential attachment has been shown to be a mechanism that can generate scalefree networks [37]. As the network evolves via the segmental duplication process, its characteristic path length increases slowly, while its clustering coefficient increases very rapidly. Because of the regularity of the connection patterns, nodes in the network remain highly connected and increase in connectivity with each segmental duplication event. However, this part of the answer neglects the mechanism of segmental divergence, such as segmental translocation or inversion. Segmental divergence only serves to perturb the structure partially randomizing some of the genes or genetic regulatory links. Thus, the formation of scale-free and smallworld structures of the networks is dominated by the effects of segmental duplication, not divergence. The dynamical behaviors of the networks depend on their structure, and the evolved networks can display scale-free and small-world structures. Therefore, the dynamics of these networks is similar to that of natural networks. References [1] T. I. Lee, N. J. Rinaldi, F. Robert, D. T. Odom, Z. Bar- Joseph, G. K. Gerber, N. M. Hannett, C. T. Harbison, C. M. Thompson, I. Simon, J. Zeitlinger, E. G. Jennings, H. L. Murray, D. B. Gordon, B. Ren, J. J. Wyrick, J. B. Tagne, T. L. Volkert, E. Fraenkel, D. K. Gifford, R. A. Young. Transcriptional regulatory networks in saccharomyces cerevisiae. Science, vol. 298, no. 5594, pp , [2] H. Hagras. Employing computational intelligence to generate more intelligent and energy efficient living spaces. International Journal of Automation and Computing, vol. 5, no. 1, pp. 1 9, [3] Z. Gu, D. Nicolae, H. H. S. Lu, W. H. Li. Rapid divergence in expression between duplicate genes inferred from microarray data. Trends in Genetics, vol. 18, no. 12, pp , [4] B. Papp, C. Pál, L. D. Hurst. Evolution of cis-regulatory elements in duplicated genes of yeast. Trends in Genetics, vol. 19, no. 8, pp , [5] S. A. Teichmann, M. M. Babu. Gene regulatory network growth by duplication. Nature Genetics, vol. 36, no. 5, pp , [6] A. Bhan, D. J. Galas, T. G. Dewey. A duplication growth model of gene expression networks. Bioinformatics, vol. 18, no. 11, pp , [7] F. Chung, L. Lu, T. G. Dewey, D. J. Galas. Duplication models for biological networks. Journal of Computational Biology, vol. 10, no. 5, pp , [8] I. Ispolatov, P. L. Krapivsky, A. Yuryev. Duplicationdivergence model of protein interaction network. Physical Review E, vol. 71, no. 6, pp , [9] D. V. Foster, S. A. Kauffman, J. E. S. Socolar. Network growth models and genetic regulatory networks. Physical Review E, vol. 73, no. 3, pp , 2006.

10 114 International Journal of Automation and Computing 7(1), February 2010 [10] R. Koszul, S. Caburet, B. Dujon, G. Fischer. Eucaryotic genome evolution through the spontaneous duplication of large chromosomal segments. The EMBO Journal, vol. 23, no. 1, pp , [11] E. V. Linardopoulou, E. M. Williams, Y. Fan, C. Friedman, J. M. Young, B. J. Trask. Human subtelomeres are hot spots of interchromosomal recombination and segmental duplication. Nature, vol. 437, no. 7055, pp , [12] X. She, G. Liu, M. Ventura, S. Zhao, D. Misceo, R. Roberto, M. F. Cardone, M. Rocchi, NISC Comparative Sequencing Program, E. D. Green, N. Archidiacano, E. E. Eichler. A preliminary comparative analysis of primate segmental duplications shows elevated substitution rates and a greatape expansion of intrachromosomal duplications. Genome Research, vol. 16, no. 5, pp , [13] T. Reil. Dynamics of gene expression in an artificial genomeimplications for biological and artificial ontogeny. In Proceedings of the 5th European Conference on Advances in Artificial Life, Lecture Notes in Computer Science, Springer, London, UK, vol. 1674, pp , [14] A. C. Bell, A. G. West, G. Felsenfeld. Insulators and boundaries: versatile regulatory elements in the eukaryotic genome. Science, vol. 291, no. 5503, pp , [15] J. Hallinan, J. Wiles. Evolving genetic regulatory networks using an artificial genome. In Proceedings of the 2nd Conference on Asia-Pacific Bioinformatics Conference, Australian Computer Society, Dunedin, New Zealand, pp , [16] A. P. Quayle, S. Bullock. Modelling the evolution of genetic regulatory networks. Journal of Theoretical Biology, vol. 238, no. 4, pp , [17] J. L. Elman. Finding structure in time. Cognitive Science, vol. 14, no. 2, pp , [18] J. Watson, N. Geard, J. Wiles. Towards more biological mutation operators in gene regulation studies. BioSystems, vol. 76, no. 1 3, pp , [19] H. Jeong, B. Tombor, R. Albert, Z. Oltvai, A. L. Barabási. The Large-scale organization of metabolic networks. Nature, vol. 407, no. 6804, pp , [20] S. Wuchty. Scale-free behavior in protein domain networks. Molecular Biology and Evolution, vol. 18, no. 9, pp , [21] N. Guelzim, S. Bottani, P. Bourgine, F. Képès. Topological and causal structure of the yeast transcriptional regulatory network. Nature Genetics, vol. 31, no. 1, pp , [22] V. Van Noort, B. Snel, M. A. Huynen. The yeast coexpression network has a small-world, scale-free architecture and can be explained by a simple model. EMBO Reports, vol. 5, no. 3, pp , [23] M. M. Babu, N. M. Luscombe, L. Aravind, M. Gerstein, S. A. Teichmann. Structure and evolution of transcriptional regulatory networks. Current Opinion in Structural Biology, vol. 14, no. 3, pp , [24] P. Erdös, A. Rényi. On the evolution of random graphs. Publications of the Mathematical Institute of the Hungarian Academy of Sciences, vol. 5, pp , [25] R. Albert, A. L. Barabási. The statistical mechanics of complex networks. Reviews of Modern Physics, vol. 74, no. 1, pp , [26] R. Albert, H. Jeong, A. L. Barabási. Error and attack tolerance of complex networks. Nature, vol. 406, no. 6794, pp , [27] D. J. Watts, S. H. Strogatz. Collective dynamics of smallworld networks. Nature, vol. 393, no. 6684, pp , [28] E. Ravasz, A. L. Somera, D. A. Mongru, Z. N. Oltvai, A. L. Barabási. Hierarchical organization of modularity in metabolic networks. Science, vol. 297, no. 5586, pp , [29] K. I. Goh, B. Kahng, D. Kim. Graph theoretic analysis of protein interaction networks of eukaryotes. Physica A: Statistical Mechanics and Its Applications, vol. 357, no. 3 4, pp , [30] A. Wuensche. Genomic regulation modelled as a network with basins of attraction. In Proceedings of the Pacific Symposium on Biocomputing, Singapore, vol. 3, pp , [31] X. B. Hu, W. H. Chen. Model predictive control of nonlinear systems: Stability region and feasible initial control. International Journal of Automation and Computing, vol. 4, no. 2, pp , [32] J. P. Eckmann, D. Ruelle. Ergodic theory of chaos and strange attractors. Reviews of Modern Physics, vol. 57, no. 3, pp , [33] A. Wolf, J. B. Swift, H. L. Swinney, J. A. Vastano. Determining Lyapunov exponents from a time series. Physica D: Nonlinear Phenomena, vol. 16, no. 3, pp , [34] J. C. Sprott. Chaos and Time-series Analysis, Oxford, USA: Oxford University Press, [35] D. J. Albers, J. C. Sprott, W. D. Dechert. Routes to chaos in neural networks with random weights. International Journal of Bifurcation and Chaos, vol. 8, no. 7, pp , [36] I. Shmulevich, S. A. Kauffman, M. Aldana. Eukaryotic cells are dynamically ordered or critical but not chaotic. Proceedings of the National Academy of Sciences, vol. 102, no. 38, pp , [37] A. L. Barabási, R. Albert. Emergence of scaling in random networks. Science, vol. 286, no. 5439, pp , Xiang-Hong Lin received the B. Sc. and M. Sc. degrees from Northwest Normal University, Lanzhou, PRC and Harbin Institute of Technology, Harbin, PRC in 1998 and 2004, respectively. He is currently a lecturer in School of Mathematics and Information Science, Northwest Normal University, and also a Ph. D. candidate in the School of Computer Science and Technology, Harbin Institute of Technology, Harbin, PRC. His research interests include the development and application of machine learning and statistical methods to problems in computational biology, especially the analysis of genetic regulatory network. waihh@hit.edu.cn (Corresponding author) Tian-Wen Zhang received the B. Sc. degree from Harbin Institute of Technology, Harbin, PRC in 1964, and was in Artificial Intelligence Laboratory at Massachusetts Institute of Technology, USA as a visiting professor in He is currently a professor with the School of Computer Science and Technology, Harbin Institute of Technology, Harbin, PRC. His research interests include active vision, wavelet signal processing, natural computation, and artificial life. twzhang@hit.edu.cn