Asynchronous Stochastic Boolean Networks as Gene Network Models

Journal of Computational Biology Journal of Computational Biology: http://mc.manuscriptcentral.com/liebert/jcb Asynchronous Stochastic Boolean Networks as Gene Network Models Journal: Journal of Computational Biology Manuscript ID: Draft Manuscript Type: Original Paper Keyword: computational molecular biology, GENE NETWORKS, pathways, probability, Statistical models Abstract: Logical models have widely been used to gain insights into the biological behavior of gene regulatory networks (GRNs). Most logical models assume a synchronous update of the genes states in a GRN. However, this may not be appropriate, because each gene may require a different period of time for changing its state. In this paper, asynchronous stochastic Boolean networks (ASBNs) are proposed for investigating various asynchronous state updating strategies in a GRN. As in stochastic computation, ASBNs use randomly permutated stochastic sequences to encode probability. A GRN is considered to be subject to noise and external perturbation, which are investigated by several stochasticity models. Hence, both stochasticity and asynchronicity are considered in the state evolution of a GRN. As a case study, ASBNs are utilized to investigate the dynamic behavior of a T helper network. It is shown that ASBNs are efficient in evaluating the steady state distributions (SSDs) of the network with random gene perturbation. The SSDs found by using ASBNs show the robustness of the attractors of the T helper network, when various stochasticity and asynchronicity models are considered to investigate its dynamic behavior. Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Asynchronous Stochastic Boolean Networks as Gene Network Models Peican Zhu and Jie Han Abstract Logical models have widely been used to gain insights into the biological behavior of gene regulatory networks (GRNs). Most logical models assume a synchronous update of the genes states in a GRN. However, this may not be appropriate, because each gene may require a different period of time for changing its state. In this paper, asynchronous stochastic Boolean networks (ASBNs) are proposed for investigating various asynchronous state updating strategies in a GRN. As in stochastic computation, ASBNs use randomly permutated stochastic sequences to encode probability. A GRN is considered to be subject to noise and external perturbation, which are investigated by several stochasticity models. Hence, both stochasticity and asynchronicity are considered in the state evolution of a GRN. As a case study, ASBNs are utilized to investigate the dynamic behavior of a T helper network. It is shown that ASBNs are efficient in evaluating the steady state distributions (SSDs) of the network with random gene perturbation. The SSDs found by using ASBNs show the robustness of the attractors of the T helper network, when various stochasticity and asynchronicity models are considered to investigate its dynamic behavior. Index Terms Stochastic Boolean networks (SBNs), stochasticity, asynchronous state update, gene regulatory networks (GRNs). Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 I I. INTRODUCTION N gene regulatory networks (GRNs), interactions such as the activation and inhibition relationships between genes, protein transcriptional factors and mrna, are often analyzed using computational models to obtain biological insights. Various methods have been proposed for modeling the genetic interactions; these include Boolean networks (BNs) (Kauffman, ), ordinary differential equation (ODE) based methods (Chen et al. 00; Qian et al., 00; Li et al., 00) and stochastic simulations at the molecular level (Gillespie, ; Gillespie, ). While the stochastic and ODE-based approaches provide a more accurate simulation of a biological system, they also require more detailed knowledge about many parameters a priori, such as the kinetic rate constants in a modeling (de Jong, 00; Karlebach and Shamir, 00). BNs model the relationships among genes through Boolean logic operations. To incorporate the effect of noise in the interactions of genes and molecules, probabilistic Boolean networks (PBNs) have been proposed and studied (Shmulevich et al., 00a; Shmulevich et al., 00b; Shmulevich and Dougherty, 00). Recently, stochastic Boolean networks (SBNs) have been proposed for an efficient modeling of GRNs (Liang and Han, 0). As in stochastic computation, an SBN employs random binary bit streams for the analysis of a PBN. To further exploit the simplicity of BNs, stochastic multiple-valued networks (SMNs) have been proposed for an efficient evaluation of gene networks with multiple discrete values (Zhu and Han, 0). For simplicity, the Boolean models usually consider a synchronous update of all genes states in a network (Shmulevich and Dougherty, 00; Naldi et al., 00; Remy et al., 00). In biology, this is not necessarily the case because the expression of a gene is seldom an instantaneous process. Instead it may require a few milliseconds or even up to a few seconds (Greil et al., 00); this makes a synchronous model less realistic. In fact, an analysis of the Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 stability of attractors has shown that some attractors may disappear in a BN model with gene perturbation (Klemm and Bornholdt, 00). This indicates that these attractors may be the result of artifacts due to the synchronous updating rules. An asynchronous state update has therefore been suggested to reduce the artifacts. However, a larger number of transitory states are usually required for deriving an attractor with an asynchronous update strategy, thus the determination of attractors becomes particularly difficult for a large network. In this paper, stochastic computational models are proposed for deriving the steady state distributions (SSDs) as estimates of the attractors of an asynchronous regulatory network. Biological systems are susceptible to stochastic fluctuations, which present an important characteristic of gene networks, because it contributes to determine the SSDs of a cell population (Zhang et al., 00). In (Ribeiro and Kauffman, 00; Willadsena and Wiles, 00), stochasticity is modeled as the perturbation of a gene s state that occurs with certain probability; however, this stochasticity in node (SIN) model over-expresses the effect of noise. A stochasticity in function (SIF) model is therefore proposed in (Garg et al., 00) and the differences between the SIN and SIF models are explicitly discussed. In a SIN model, the correlation between a node s current expression level and the probability of changing the expression is not taken into consideration, thus a model of stochasticity with propensity parameters (SPP) is proposed in (Murrugarra et al., 0). In this model, two probabilities are assigned such that each indicates the likelihood that an update leads to an increase or decrease of the current state. In this paper, asynchronous stochastic Boolean networks (ASBNs) are proposed as asynchronous gene network models. In an ASBN, a number of genes are updated simultaneously or in a different order, as considered in (Harvey and Bossomaier, ; Garg et al., 00; Luo Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 and Wang, 0; Wu et al., 00). An asynchronous update module (AUM) is first proposed to implement the asynchronous update of a gene s state. Various asynchronous update strategies that involve a different number of genes to be updated in certain order are then implemented by using randomly generated stochastic sequences. The long run behaviors of synchronous and asynchronous networks are investigated using ASBNs; comparisons between different asynchronous update strategies are performed. The efficiency of the ASBN is evaluated by a comparison to a simulation-based method (Garg et al., 00). To analyze the long run behavior of an asynchronous network, a so-called time-frame expansion technique (Liang and Han, 0; Zhu and Han, 0) is applied to efficiently compute the SSD. The simulation results of a T helper network of genes show the robustness of its attractors under various state-update strategies. II. SYNCHRONOUS AND ASYNCHRONOUS BOOLEAN NETWORKS A Boolean network (BN) can model the relationships among gene nodes by Boolean logic operations (Kauffman, ). A probabilistic Boolean network (PBN) of genes is defined by,, where,,, with 0, for,,,, a set of binary nodes, and,,,, a list of sets of update functions:,,,, (Shmulevich and Dougherty, 00; Kim et al., 00). For a BN, a node represents the state of gene ; if = (or 0), then gene is (or not) expressed. The predictor function contains the rules that determine the next state of gene. If the quantization level of the gene node is not limited to binary values, a multiple-valued network can be constructed to model the relationship among genes for reducing the inaccuracy incurred by the Boolean simplification (Kim et al., 00; Li and Cheng, 0). Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 A network model is presented in Figure for a deterministic synchronous update strategy. If,,,, consists of several update functions with different selection probabilities, this network becomes a (synchronous) probabilistic network (Shmulevich and Dougherty, 00; Kim et al., 00). Figure A general synchronous gene network model. denotes the update function for gene, and are the states of gene at time and respectively,,,,. is the number of genes in the investigated network. When asynchronicity is considered, a deterministic-asynchronous probabilistic Boolean network (DA-PBN) assumes that the state of each gene is independently updated according to its own updating period (Shmulevich and Dougherty, 00). Whether the state of gene is updated or not at a time is indicated by a binary variable (either or 0 respectively). The next state of gene,, is then given by:,,, () where is the predictor function in the DA-PBN for gene ; is a binary variable that determines whether a gene is to be updated or not. Furthermore, a different number of genes can be selected in an asynchronous update process. The genes can be updated at the same time in one time step or following certain updating order. In (Harvey and Bossomaier, ; Garg et al., 00), only one gene is randomly selected at a time for updating its state, while in (Luo and Wang, 0), m gene nodes are Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 randomly chosen for a simultaneous state update in one time step, where m is a randomly generated number. Even in one time step, however, the update of m genes is likely to be asynchronous and the state of a gene depends on the updating order of the inputs of its predictor function. In (Wu et al., 00), all the gene nodes are asynchronously updated one by one, so an updated state will subsequently affect the other components states, even within the same time step. Hence, an asynchronous update model is more realistic in modeling biological behaviors. Since the next state of a network is determined by the updated gene and the remaining genes, an updated gene state is determined by the current state vector and the corresponding predictor function (Garg et al., 00). III. STOCHASTIC LOGIC Stochastic computation has been utilized to perform probabilistic analysis through the application of Boolean logic by encoding a real number or probability into a random binary bit stream (Gaines, ). These random binary bit streams introduce inevitable stochastic fluctuations into the computation, thus making the computational result non-deterministic. However, the use of non-bernoulli sequences of fixed numbers of s and 0s for encoding initial input probabilities can reduce the stochastic fluctuation and produce more accurate results than the use of Bernoulli sequences (Han et al., 0). Moreover, a stochastic computational approach efficiently handles signal correlations in a logic network. Stochastic computational models are first utilized in stochastic Boolean networks for modeling GRNs (Liang and Han, 0). To reduce the inaccuracy incurred in a binary analysis, multiple-valued logic has been considered to generalize the stochastic analysis (Zhu and Han, 0). In a k-valued gene network, randomly permuted sequences of fixed numbers of the k Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 values are used. The use of the randomly permuted sequences as initial inputs can reduce the inevitable stochastic fluctuation and produce more accurate results than using randomly generated sequences. The stochastic logic elements used in this paper are listed in Figure, including a buffer, an inverter, AND, OR and a multiplexer (MUX) (Liang and Han, 0). For synchronous and asynchronous networks, stochastic architectures can be constructed for GRNs with update functions implemented by the stochastic logic gates in Figure. Figure Stochastic logic: (a) a buffer, (b) an inverter, (c) an AND gate, (d) an OR gate, (e) an XOR gate, and (f) a multiplexer. Stochastic logic performs a probabilistic analysis by encoding probabilities into random value streams as proportional numbers of different values. IV. STOCHASTIC COMPUTATIONAL MODELS FOR ASYNCHRONOUS UPDATE STRATEGIES In a biological system, some cells may update their states immediately while it may take longer for other cells to respond. This phenomenon corresponds to a variable reaction time or rate for each cell. To model this asynchronous process, a -to- multiplexer (MUX) is used to determine whether a gene s state is updated or not, as determined by the value of the control bit, shown in Figure. If the control bit is, where,,, and is the number of genes, then the th gene s state is determined by the input genes states and the predictor function (i.e.,, ). If 0, the gene remains at its current state, indicated by the application of the buffer in Figure (i.e. ). Hence, the stochastic architecture in Figure accurately implements the function of (). Figure An asynchronous update module (AUM) for gene, referred to as, consists of a -to- multiplexer (MUX) with a control bit. is the predictor function for gene.,,,, with Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 being the state of gene at time, while indicates the next state of gene,,,,. is the number of genes in the investigated network. Stochastic architectures for asynchronous networks can be constructed with the control sequences of the MUX that are generated according to the updating rules. The state transition matrices (STMs) are usually applied to derive the steady state distribution (SSD) by an iterative evaluation, especially for synchronous models. However, a matrix-based analysis becomes cumbersome to perform due to the size of STMs for a large network; an analysis using STMs becomes even more challenging for asynchronous networks. Hence, the SSD is evaluated by using the so-called time-frame expansion technique (Liang and Han, 0). By this technique, the temporal evolution of an asynchronous network is simulated using a spatially iterative structure of the asynchronous network model. Different strategies of updating one gene or m genes synchronously or in a random order are presented as follows: The ROG (randomly one gene) model (Garg et al., 00): At each time step, only one gene node is randomly selected for a state update, while the remaining nodes stay at their present states. The RMG (randomly m genes) model (Luo and Wang, 0): At each time step, gene nodes are randomly selected and synchronously updated, while the remaining nodes preserve their present values. Here, can be either a random number or a fixed value. If, the RMG model is simplified to the ROG model. The ARO (all genes updated in a random order) model (Wu et al., 00): At each time step, Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 all gene nodes are updated; however, the state update of each gene occurs one by one in a random order. The update is carried out immediately and a state change will affect the change of other genes states, even within the same time step. The MRO (m genes updated in a random order) model: At each time step, only gene nodes are randomly selected for updating their states and each state update occurs in a random order, while the other genes remain at their present states. If, this is simplified to the ROG model. If, the result will be the same as the ARO model. The general architectures are similar for the ROG and RMG models; the only difference lies in the control sequences {,,, } for the -to- MUXs, where indicates the control sequence for gene,,,. For the control sequences of asynchronous update modules (AUMs) in the ROG model, if the th bit in the control sequence is, i.e.,,, then S, 0 for,,,,,,,,, with, as only one gene is selected for updating its state, where indicates the length of the control sequences. In contrast, the requirement of, must be met for the RMG model as gene nodes are randomly selected for updating the states using the control sequences. The generation of the stochastic control sequences for the AUMs is explained next with an example of a -gene network using the ROG and RMG models. Let the number of genes to be updated for the RMG model, i.e., be. For each column of the generated sequence matrices, only one gene is updated while two gene nodes are randomly selected to be refreshed for,,, for the RMG model. At certain time step, let the generated control sequences be and for the ROG and RMG models respectively, as illustrated in Figure. Figure An illustrative example of the generated control sequences at a time step for a network of five genes. (a) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 The control sequences for the randomly one gene (ROG) model. (b) The control sequences for the randomly genes (RMG) model. indicates the control sequence for gene,, ; is for the jth bit in each sequence, which can be considered as the th trial of the experiments and indicates the length of the stochastic sequences and,,,., 0 indicates that the control bit is 0 for the asynchronous update module (AUM) in Figure, so the state of gene remains, while, means that the next state is determined by the predictor function. For the RMG model, the number of genes to be updated for the th trail is assumed to be two, thus, in (b). is the number of genes in the investigated network. For the ARO and MRO models, similarly, the evolution of an asynchronous network is simulated by the time-frame expansion technique. In the ARO model, as all genes are updated in a random order, each time step is divided into sub-steps and only one gene is chosen to be updated at each sub-step while the other genes maintain their present states. Assume at time, only one gene is updated at each sub-step ( sub-steps in total); hence, after the evolution of sub-steps, all the genes are updated, followed by the evolution at time. The ARO model is implemented with only one AUM with a control bit of at each sub-step for the gene to be updated (the AUM is simplified to a buffer as the control bit is 0 for the other genes). If m genes are to be updated in a different order, there will be n-m buffers (or AUMs with a control bit of 0) for the n sub-steps. This becomes an MRO model. For the ARO model, at each sub-step only one gene is updated while the others preserve their present states. If,,,,,,,,,, the th gene s state is updated by the predictor function; otherwise, the current state remains. For the MRO model, the total number of s for the sub-steps at time step is equal to the number of genes to be updated for the MRO model. All genes remain at their present states if the control bits for the AUMs are 0 at Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 a time step. The updating order is indicated by the different position of s in the generated sequences. An example of the random updating order for the AUMs at time is illustrated for a network of five genes. Following the previous analysis, time step is divided into five sub-steps. The control sequences of and for the ARO and MRO models are shown in Figure (a) and (b) respectively. Figure An illustrative example of the generated control sequences at a time step for a network of five genes. (a) The control sequences for the all genes updated in a random order (ARO) model. (b) The control sequences for the genes updated in a random order (MRO) model. is for the th time step and,, are the sub-steps at time. and are the generated control sequences at the sub-steps of time. An entry of 0 indicates the state of gene remains while a value of indicates that the next state is determined by the predictor functions. The number of genes to be updated at time is three (in a random order), thus in (b). The updating order of the ARO model at time is given by the control sequences in Figure (a), i.e.,, while the updating order of the MRO model at time step is, as given by the control sequences in Figure (b). For genes and, the present states at time remain the same. V. ASYNCHRONOUS STOCHASTIC BOOLEAN NETWORKS (ASBNS) Any gene node in a GRN can change the expression level due to noise or a gene perturbation that occurs with certain probability. The perturbation probability can be encoded as a perturbation flag vector with each element indicating whether a gene is to be perturbed or not. When a perturbation occurs, the state of the perturbed gene is determined by the present state and Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 the perturbation flag vector. In the stochasticity in node (SIN) model (Garg et al., 00), therefore, the effect of perturbation can be modeled by an XOR gate (Liang and Han, 0), by which the gene state at time will be flipped with a probability of, as shown in Figure (a). Figure Stochastic architectures for the models of stochasticity in node (SIN) and stochasticity with propensity parameter (SPP). (a) The stochastic architecture of the SIN model for a binary gene node (Liang and Han, 0). (b) The stochastic architecture of the SPP model for updating the state of a gene. In the SIN model, the correlation between a gene node s current expression level and the probability of changing the expression value is not taken into consideration. Even if the expression level of the input node for an update function guarantees the activation or degradation, there still exists a probability that the process will not occur due to stochasticity. Hence, a stochasticity model is proposed in (Murrugarra et al., 0) with propensity parameters,,,, () where indicates the update function for node, 0 and 0 denote the activation propensity and degradation propensity, and is the number of genes. A model of stochasticity with propensity parameters (SPP) is described by (Murrugarra et al., 0) In order to implement the function of (), a multiplexer is used here to simulate the behavior of stochasticity with parameters. A general structure is shown in Figure (b). Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0 ()

Page of Journal of Computational Biology 0 0 0 0 0 For the architecture in Figure (b), the output is determined by the value in the control sequence. consists of ternary values {0,, }, which indicate the three cases of, and for any,,,. and are the stochastic sequences encoding the probabilities of gene at time and respectively. is the stochastic sequence encoding the expected probability for the state of gene at time. This probability is calculated from, where indicates the update function for gene and,,,, with being the state of gene at time. and represent the stochastic sequences encoding the propensity parameters. Let, and be the th value in, and. For a ternary case, if, the output is the same as, because the transition occurs with a probability of. However, if, is determined by the output of a -to- multiplexer with and as input sequences and as the control sequence. If 0, for example, then with a probability of. If, then with a probability of or with a probability of. These functions are implemented by the -to- multiplexers in the model in Figure (b). Let a and b be the values in and respectively; the next state of the gene is obtained as,,, (),,, Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 which is equivalent to the function of (). This shows that the stochastic model in Figure (b) accurately computes the function of the SPP model. In order to analyze the long run behavior of gene networks with different stochasticity effects, synchronous and asynchronous stochastic Boolean networks (ASBNs) are presented in Figure for the SIN and SPP models. The synchronous network is obtained by implementing the stochasticity models (either the SIN in Figure (a) or the SPP in Figure (b)) into the architecture in Figure. For simplicity, only the randomly one gene (ROG) model is illustrated in Figure (b) for an ASBN. For randomly m gene (RMG) or a different updating order, the stochastic networks can be obtained by applying a stochasticity model (either the SIN in Figure (a) or the SPP in Figure (b)) into the stochastic architecture in Figure. Although discussed within the context of BNs, multiple-valued networks such as those in (Zhu and Han, 0) can also be used to construct an asynchronous network. In a stochastic network, randomly permuted sequence and perturbation vectors are generated for different signal probabilities. With these stochastic sequences as inputs to the stochastic architectures, the stochastic behavior of a gene network can be efficiently analyzed. Figure Synchronous and asynchronous stochastic Boolean networks (ASBNs) for different stochasticity models. (a) A synchronous SBN. (b) An ASBN for the randomly one gene (ROG) model. : predictor function for gene,,,,. is the number of genes in the investigated network. The detailed structures and inputs of the SIN and SPP modules are shown in Figure (a) and (b) respectively. Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 VI. RESULTS AND DISCUSSION A T helper network of genes with regulatory interaction relationships (Figure ) (Mendoza and Xenarios, 00) is considered to show the robustness of the attractors under the various asynchronous update schemes. The functional molecules or molecular complexes are listed as,,,,,,,,,,,,,,,,,,,,,,. Figure A T helper network. A positive regulatory interaction is indicated by a regular arrow, while a negative interaction is represented by a blunted arrow (Mendoza and Xenarios, 00). There is no input for four gene nodes, namely,,, and, in the T helper network. As in (Mendoza and Xenarios, 00), hence, these four elements are considered to be consistently at state 0. The network in Figure can be implemented by the BN model shown in Figure. Figure A deterministic BN model for the -gene T helper network, with 0, 0, 0 and 0.,,,,, indicates a gene node in Figure. The cell population may fall into different subgroups on exposure to input stimuli this is referred to as a cellular differentiation (Kaern et al., 00). For the -gene T helper network in Figure, functions as a key cytokine in the 0 to differentiation process while a Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 large perturbation of forces the network to switch from state 0 to, as illustrated by Figure. Figure Differentiation of T helper (adapted from (Kaern et al., 00)). cells express and, while expresses and. In the state space of the T helper network, the basins draining to different attractors are not evenly distributed; the distribution of initial states leading to the attractors is shown in Table. As revealed in Table, a random initial state evolves into with the largest probability for each state update strategy. The attractor 0 is reached with the lowest probability. For the asynchronous update strategies, the probability of reaching 0 is even lower than that for the synchronous state update. The effect of perturbation on different gene nodes has been shown by experiments of the differentiation of T helper cells (Agnello et al., 00; Bergmann et al., 00). The stochastic simulations using synchronous and asynchronous update strategies correctly identify the attractors for a wild type T helper network, in which no perturbation exists. Table shows the steady states found by the stochastic approach for several scenarios of the wild type (no gene suppression or overexpression; the initial states are randomly generated with a probability of 0.), overexpressed and overexpressed. These steady states are good estimates of the attractors found in (Garg et al., 00; Murphy and Reiner, 00). In (Garg et al., 00), at most one perturbation is considered to occur at a time. In this work, k time frames are considered with the stochasticity models (k = in Figure ). If the Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 initial state of the T helper network is 0, after the constant activation of, the state will transition into. As shown in Figure (b) and (c), the state remains even after the inactivation of if no perturbation occurs. For the SIN model, the 0 cells derive into and, while a few cells remain at their current states. The almost equally-likely cellular differentiation of steady states has been analyzed in (Kadanoff et al., 00). However, it has been shown that a 0 cell cannot transfer into a cell (Murphy and Reiner, 00). Hence, the SIN model could be the main reason for this discrepancy, because and are evolved from 0 with nearly equal probability, as revealed in Figure (d)-(f) for the synchronous and RMG models. Figure Transitions between attractors during the differentiation process of the T helper network with an external stimulus of for the synchronous and randomly genes (RMG) models. Each dot indicates a T helper cell and each cell is independent of the neighboring cells. A red dot represents an undifferentiated cell at 0, a green dot indicates the cell state of and a blue dot indicates the state of. (a) All cells are in the initial 0 state. (b) and (c) The transitions from 0 to by synchronous and asynchronous updating rules (without perturbation to the network). (d), (e) and (f) The transitions from 0 for the stochasticity in node (SIN) model with perturbation probability of 0. for each node: the percentage of the corresponding nodes indicates the percentage of the cells deriving into an attractor. For the first five time frames, each of the gene nodes is subject to the effect of SIN. The obtained transition probabilities from 0 to different attractors are similar to the values in Table, because it is equivalent to setting the initial state as the wile type by having each of the nodes under SIN for several time frames. For the SPP model, the next state of a gene is closely related with the activity of other nodes, the predictor function and the present states. The simulation results for this model are Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 shown in Table for an initial state of 0. As revealed in the results, nearly all the nodes of 0 differentiate into cells under the dosage of. Similar results can be obtained for the dosage of and for the other asynchronous update strategies. The instability of 0 is in agreement with the experimental results in (Kaern et al., 00). VII. CONCLUSION In this paper, asynchronous stochastic Boolean networks (ASBNs) are proposed for an efficient modeling of gene regulatory networks (GRNs). Various asynchronous updating rules are considered, including the models of randomly one gene (ROG), randomly m genes (RMG), all genes updated in a random order (ARO) and m genes updated in a random order (MRO). In an ASBN, if the control bits for an asynchronous update module are all 0 s, an asynchronous network becomes the same as a synchronous update network. Stochastic analyses are carried out for the asynchronous models of stochasticity in node (SIN) and stochasticity with propensity parameters (SPP). The expected long run behavior of a GRN is studied by the stochastic approach for various asynchronous state update strategies. For an asynchronous gene network, an accurate analysis becomes difficult due to the limitation of computational resources, especially when the number of genes or the quantization level of gene states increases. In the stochastic approach, a simulation-based time-frame expansion technique is used for an efficient analysis of the SSD of a network. The obtained SSDs can be used to estimate the attractors of, especially, a large network. This is shown by the study of a T helper network of genes. The robustness of the attractors for the T helper network is demonstrated for a number of synchronous and asynchronous state update strategies. Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 The effect of different asynchronous updating rules on the distribution of initial states leading to an attractor is investigated for the T helper network. It is shown that the model of stochasticity with propensity parameters (SPP) accurately reveal the state transition from 0 to under the dosage of. REFERENCES [] Agnello, D., Lankford, CSR., Bream, J., Morinobu, A., Gadina, M., O'Shea, J., Frucht, DM. 00. Cytokines and transcription factors that regulate T helper cell differentiation: new players and new insights. J Clin Immunol, :-. [] Bergmann, C., Van Hemmen, J. L., Segel, L. A. 00. Th or Th: how an appropriate T helper response can be made. Bull. Math. Biol., 0 0..0/bulm.000.0. [] Chen, K. C. et al. 00. Integrative analysis of cell cycle control in budding yeast. Mol. Biol. Cell,,. [] de Jong, H. 00. Modeling and simulation of genetic regulatory systems: a literature review. J Comput Biol, January 00, ():-. [] Gillespie, DT.. A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys, :0. [] Gillespie, DT.. Exact stochastic simulation of coupled chemical reactions. J Phys Chem, :0. [] Garg, A., Mohanram, K., Di Cara, A., De Micheli, G., and Xenarios, I. 00. Modeling stochasticity and robustness in gene regulatory networks. Bioinformatics (Oxford, England), (), i-. [] Garg, A., Di Cara, A., Xenarios, I., Mendoza, L., and De Micheli G. 00. Synchronous versus asynchronous modeling of gene regulatory networks. Bioinformatics,. [] Greil, F., Drossel, B., Sattler, J. 00. Critical Kauffman networks under deterministic asynchronous update. New Journal of Physics :. [] Han, J., Chen, H., Liang, J., Zhu, P., Yang, Z. and Lombardi, F. 0. A Stochastic Computational Approach for Accurate and Efficient Reliability Evaluation. Accepted for publication in IEEE Transactions on Computers. [] Harvey, I. and Bossomaier, T.. Time out of joint: Attractors in asynchronous random Boolean networks, in Proceedings of the Fourth European Conference on Artificial Life (ECAL, MIT Press), pp.. [] Gaines, B. R.. Stochastic Computing Systems. Advances in Information Systems Science, Vol., pp. -. Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page 0 of 0 0 0 0 0 [] Kadanoff, L. et al. 00. Boolean dynamics with random couplings. Springer Applied Mathematical Sciences Series, Special volume, Springer, New York, pp.. [] Kaern, M. et al. 00. Stochasticity in gene expression: from theories to phenotypes. Nat. Rev. Genet.,,. [] Karlebach, G., and Shamir, R. 00. Modelling and analysis of gene regulatory networks. Nat Rev Mol Cell Biol 00, :0-0. [] Kauffman, S. A.. Metabolic stability and epigenesis in randomly constructed genetic nets. Theor. Biol., vol., pp.. [] Kim, S., Li, H., Dougherty, E. R., et al. 00. Can Markov chain models mimic biological regulation? Journal of Biological Systems, vol., no., pp.. [] Klemm, K., and Bornholdt, S. 00. Stable and unstable attractors in Boolean networks. Physical Review E, Statistical, Nonlinear, and Soft Matter Physics : 0. [] Li, S. et al. 00. A quantitative study of the division cycle of Caulobacter crescentus stalked cells. PLoS Comput. Biol.,, e. [0] Li, Z., Cheng, D. 0. Algebraic Approach To Dynamics of Multivalued Networks. International Journal of Bifurcation an d Chaos, Vol. 0, No. ( 0). [] Liang, J., and Han, J. 0. Stochastic Boolean networks: an efficient approach to modeling gene regulatory networks. BMC Syst. Biol.,,. [] Luo, C., and Wang, X. 0. Dynamics of Random Boolean Networks under Fully Asynchronous Stochastic Update Based on Linear Representation. PloS one, (), e. [] Mendoza, L., and Xenarios, I. 00. A method for the generation of standardized qualitative dynamical systems of regulatory networks. Theor. Biol. Med. Model.,,. [] Murrugarra, D., Veliz-Cuba, A., and Aguilar, B., et al. 0. Modeling stochasticity and variability in gene regulatory networks. EURASIP J. Bioinform. Syst. Biol., 0 (),. [] Murphy, K. M., and Reiner, S. L. 00. The lineage decisions on helper T cells. Nat. Rev. Immunol.,,. [] Naldi, A. et al. 00. Decision diagrams for the representation and analysis of logical models of genetic networks. LNCS/LNBI,,. [] Qian, L., Wang, H., and Dougherty E. 00. Inference of noisy nonlinear differential equation models for gene regulatory networks using genetic programming and Kalman filtering. IEEE Trans. Signal Process., vol., no., pp.. 0 Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 [] Remy, E. et al. 00. From logical regulatory graphs to standard petri nets: dynamical roles and functionality of feedback circuits. Springer LNCS, 0,. [] Ribeiro, A. S., and Kauffman, S. A. 00. Noisy attractors and ergodic sets in models of gene regulatory networks. J. Theor. Biol.,,. [0] Shmulevich, I., Dougherty, E. R., and Zhang, W. 00a. From Boolean to probabilistic Boolean networks as models of genetic regulatory networks. Proc. IEEE, vol. 0, pp.. [] Shmulevich, I., Dougherty, E. R., and Zhang, W. 00b. Gene perturbation and intervention in probabilistic Boolean networks. Bioinformatics, ():-. [] Shmulevich, I., Dougherty, E. R. 00. Probabilistic Boolean Networks: The Modeling and Control of Gene Regulatory Networks. SIAM, Philadelphia. [] Willadsena, K., and Wiles, J. 00. Robustness and state-space structure of Boolean gene regulatory models. J. Theor. Biol.,,. [] Wu, M., Yang, X., and Chan, C. 00. A dynamic analysis of IRS-PKR signaling in liver cells: a discrete modeling approach. PLoS One, (), e00. [] Zhang, S. et al. 00. Simulation study in probabilistic Boolean network models for genetic regulatory networks. Int. J. Data Min. Bioinformatics,, 0. [] Zhu, P., and Han, J. 0. Stochastic Multiple-Valued Gene Networks. Accepted for publication in IEEE Transactions on Biomedical Circuits and Systems. Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 Table The distribution of initial states (in percentages) leading to the attractors of a wild type T helper cell: 00,000 states are randomly chosen for simulation. Wild type 0 (%) (%) (%) Synchronous... ROG. 0.0. RMG ( = )... RMG ( = ).. 0. ARO.0 0.0. MRO ( = ). 0..0 Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Table Steady states of the T helper network found by the stochastic approach. Perturbed genes Active genes in steady states attractors Wild type All genes are inactive 0 over expressed over expressed Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 Table The transition probability from the initial state to an attractor using the synchronous and random genes (RMG) models with an external stimulus of. If, the RMG model is simplified to the random one gene (ROG) model. The propensity parameters and are assumed to be 0. for each node in the model of stochasticity with propensity parameters (SPP). Update strategies 0 0 synchronous or RMG (m = ) 0 RMG (m = ) 0 ROG or RMG (m = ) 0 Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Figure A general synchronous gene network model. F_i denotes the update function for gene i, x_i (t) and x_i (t+) are the states of gene i at time t and t+ respectively, i {,,,n}. n is the number of genes in the investigated network. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 Figure Stochastic logic: (a) a buffer, (b) an inverter, (c) an AND gate, (d) an OR gate, (e) an XOR gate, and (f) a multiplexer. Stochastic logic performs a probabilistic analysis by encoding probabilities into random value streams as proportional numbers of different values. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Figure An asynchronous update module (AUM) for gene i, referred to as AUM _i, consists of a -to- multiplexer (MUX) with a control bit S_i. F_i is the predictor function for gene i. x(t)={x_ (t),,x_i (t),,x_n (t)} with x_i (t) being the state of gene i at time t, while x_i (t+) indicates the next state of gene i, i {,,,n}. n is the number of genes in the investigated network. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 Figure An illustrative example of the generated control sequences at a time step for a network of five genes. (a) The control sequences for the randomly one gene (ROG) model. (b) The control sequences for the randomly m genes (RMG) model. S_i indicates the control sequence for gene i, i [,]; j is for the jth bit in each sequence, which can be considered as the jth trial of the L experiments and L indicates the length of the stochastic sequences and j {,,,L}. S_(i,j)=0 indicates that the control bit is 0 for the asynchronous update module (AUM) in Figure, so the state of gene i remains, while S_(i,j)= means that the next state is determined by the predictor function F_i. For the RMG model, the number of genes to be updated for the jth trail is assumed to be two, thus _(i=)^n S_(i,j) = in (b). n is the number of genes in the investigated network. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Figure An illustrative example of the generated control sequences at a time step for a network of five genes. (a) The control sequences for the all genes updated in a random order (ARO) model. (b) The control sequences for the m genes updated in a random order (MRO) model. t is for the tth time step and t_,,t_ are the sub-steps at time t. C_ARO _t and C_MRO _t are the generated control sequences at the substeps of time t. An entry of 0 indicates the state of gene i remains while a value of indicates that the next state is determined by the predictor functions. The number of genes to be updated at time t is three (in a random order), thus C_MRO _t = in (b). xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page 0 of 0 0 0 0 0 Figure Stochastic architectures for the models of stochasticity in node (SIN) and stochasticity with propensity parameter (SPP). (a) The stochastic architecture of the SIN model for a binary gene node []. (b) The stochastic architecture of the SPP model for updating the state of a gene. 0xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Figure Synchronous and asynchronous stochastic Boolean networks (ASBNs) for different stochasticity models. (a) A synchronous SBN. (b) An ASBN for the randomly one gene (ROG) model. F_i: predictor function for gene i, i {,,,n}. n is the number of genes in the investigated network. The detailed structures and inputs of the SIN and SPP modules are shown in Figure (a) and (b) respectively. 0xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 Figure A T helper network. A positive regulatory interaction is indicated by a regular arrow, while a negative interaction is represented by a blunted arrow []. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Figure A deterministic BN model for the -gene T helper network, with X_=0, X_=0, X_=0 and X_=0. X_i, i {,,,}, indicates a gene node in Figure. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Journal of Computational Biology Page of 0 0 0 0 0 Figure Differentiation of T helper (adapted from []). Th cells express IFN-γ and T-bet, while Th expresses GATA and IL-. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0

Page of Journal of Computational Biology 0 0 0 0 0 Figure Transitions between attractors during the differentiation process of the T helper network with an external stimulus of IL- for the synchronous and randomly m genes (RMG) models. Each dot indicates a T helper cell and each cell is independent of the neighboring cells. A red dot represents an undifferentiated cell at Th0, a green dot indicates the cell state of Th and a blue dot indicates the state of Th. (a) All cells are in the initial Th0 state. (b) and (c) The transitions from Th0 to Th by synchronous and asynchronous updating rules (without perturbation to the network). (d), (e) and (f) The transitions from Th0 for the stochasticity in node (SIN) model with perturbation probability of 0. for each node: the percentage of the corresponding nodes indicates the percentage of the cells deriving into an attractor. For the first five time frames, each of the gene nodes is subject to the effect of SIN. The obtained transition probabilities from Th0 to different attractors are similar to the values in Table, because it is equivalent to setting the initial state as the wile type by having each of the nodes under SIN for several time frames. xmm (00 x 00 DPI) Mary Ann Liebert, Inc., 0 Huguenot Street, New Rochelle, NY 0