# Measures for information propagation in Boolean networks

Save this PDF as:

Size: px
Start display at page:

## Transcription

2 P. Rämö et al. / Physica D 227 (2007) Boolean function as an update rule whose inputs are determined according to the graph connections. Value 1 of the state represents an active gene and value 0 represents a non-active gene. Boolean function f associated with the node represents the regulatory rule affecting the gene in question. We denote the distribution over the set of functions in the network by F. The in-degree and the function distribution are connected in such a way that the degree distribution of the function distribution matches the network in-degree distribution. The state x of the network is the vector of state variable values x i of all the nodes at time t. The network nodes are synchronously updated. In mathematical terms, the state x i of the ith node of the network is updated as x i (t + 1) = f i (x i1 (t), x i2 (t),..., x ik (t)) where x i j is the jth connection to node i and K is the indegree of Boolean function f i. Boolean networks can be viewed also as maps between different states as x(t + 1) = F(x(t)). In quenched Boolean networks the network connections and functions do not change over time. The annealed approximation provides a probabilistic framework in which to study the dynamics of Boolean networks. In the approximation we shuffle the network connections and functions after every time step. All the analytical results in this paper are based on the annealed approximation. The functions are taken from a given function distribution F. Annealed networks can be constructed with any in-degree p in (k) and out-degree p out ( j) distribution. Of course, the two distributions must have the same expectation p in (k). In addition, we assume that the network size approaches infinity, N. In practice, these approximations mean that we can consider every node in the network as a probabilistic entity that has a probability of having a specific in-degree, outdegree, and Boolean function. To make the analysis as general as possible, we assume that the Boolean network may have any distribution of functions. The bias b t is the probability that a node has value 1 at time t. The bias-map b t+1 = g(b t ) gives the evolution of b t as b t+1 = g(b t ) = E f (x)p(x b t ), f F x {0,1} K where P(x b t ) = b t x (1 b t ) K x is the probability for the input vector x when b t is known. f is a Boolean function drawn from the distribution F and K is the in-degree of function f. We assume that all the bias-maps of the biologically realistic distributions F have a single stable fixed point b = g(b ) [13]. The average influence I is defined as the probability that a perturbation of an input to f (x) has an influence on its output value in the stationary state characterized by b : I = E 1 K f (x) f (x e i )P(x b ). f F K i=1 x {0,1} K Here, e i is the unit vector with value 1 at the ith position and refers to the exclusive or operator. We define the network order parameter λ as λ = p in (k) I, where p in (k) is the expected value of the distribution p in (k). The order parameter λ is the average amount of nodes that are perturbed one time step after we have flipped the value of a randomly chosen node, given that the network has reached the bias-map fixed point before the perturbation. Networks with λ < 1 are stable, networks with λ = 1 are critical, and networks with λ > 1 are chaotic. A Derrida map [14,15] is a one-dimensional mapping ρ t+1 = h(ρ t ), ρ {0, 1} that illustrates the propagation of perturbations in annealed Boolean networks. ρ t is the proportion of perturbed nodes in the network at time t. The Derrida map has a stable fixed point at zero for stable and critical networks and a non-zero fixed point for chaotic networks [14]. The fixed point of the Derrida map gives the final probability that a node is perturbed after the network has been run for many time steps. The slope of the Derrida map is the order parameter λ [15]. 3. Mutual information in dynamical networks Mutual information is commonly used to study how much information can be passed through a system. Beggs and Plenz [1] and Luque and Ferrara [8] proposed two different approaches that use mutual information to measure information propagation in dynamical networks. They claim that their measures are maximized in critical networks. We will find in this section that these results do not apply in many general cases. We think this is an important note since the result presented in [1] is cited by many other papers; see for example [2,16,17]. Mutual information MI can be defined as follows: MI = H 1 + H 2 H 12, where H 1 is the input entropy: H 1 = s i S P 1 (s i ) log(p 1 (s i )), H 2 is the output entropy: H 2 = s j S P 2 (s j ) log(p 2 (s j )), and H 1,2 is the joint entropy of the input and output: H 1,2 = P(s i, s j ) log(p(s i, s j )). s i S s j S Here, S refers to the set of possible symbols, P 1 (s i ) is the probability that symbol s i occurs in the input, and P 2 (s j ) is the probability that symbol s j occurs in the output. Beggs and Plenz [1] used feed-forward networks with a finite number of nodes on each layer. The nodes are randomly

3 102 P. Rämö et al. / Physica D 227 (2007) connected to the next layer, and the update rules are chosen so that a perturbation of a node is transmitted, on average, to λ nodes on the next layer. The first layer is the input layer and the last layer is the output layer. They calculated the average mutual information between the symbols in the input and output layers given the distribution of input symbols and given λ. See [1] for the exact model definition. Let us now concentrate on this approach in more detail. The setting corresponds to annealed Boolean networks with a finite number of nodes on each layer. Here, instead of the actual state the configuration of perturbed nodes constitute a symbol. We note that the probability P 2 (s 2 ) that the output symbol s 2 S occurs is only related to the probability that an output node is perturbed. This is true because the layers are randomly connected, i.e. there are no logical connections between a pair of input and output nodes. If there are enough layers, all initial perturbations finally die out in stable and critical networks because the corresponding Derrida map has a fixed point at zero. In these cases, the output contains only nonperturbed nodes. Therefore, any output symbol is uncorrelated with the input symbols. From this, we directly see that the mutual information is not maximized in critical networks. In chaotic networks the analysis becomes a bit more complicated. We have feed-forward networks with a finite number of nodes in each layer. Therefore, the perturbation size may reach zero and stay there. In addition, we know from the theory of branching processes that the perturbation size may converge also in chaotic networks when the initial perturbations have a finite size; see the next section for details. In a typical (Boolean network) case, however, the perturbation propagates so that the probability of a node being perturbed in the output layer is close to the non-zero Derrida map fixed point. This result may vary depending on how we define the case where a node gets a perturbation from multiple sources. Therefore, for some particular parameters of the model the mutual information may have a small non-zero value for chaotic networks. In conclusion, the maximum of the mutual information is not found in critical networks in this model as claimed. Luque and Ferrara [8] have used a different approach to use mutual information in annealed Boolean networks. They found that the mutual information between the consecutive states of a node in the network is MI = p log p (1 p) log(1 p) + a 1 + 2p ( ) 2 a 1 + 2p log + 1 a ( ) 1 a log + a + 1 2p 2p 2 2p 2 ( ) a + 1 2p log + 1 a ( ) 1 a log. 2(1 p) 2 2(1 p) Here, a is the probability that a node keeps its value in the Boolean network over a single time step, and p is the probability that a random Boolean function outputs one. In other words, a is one minus the fixed point of the corresponding Derrida map [15,18]. The authors found that this measure for complexity is maximized in critical Boolean networks in networks with constant in-degree K = 3. However, the measure fails to make a separation between stable networks. For example, the measure does not separate the networks with constant in-degrees K = 1 and K = 1.5, since all stable networks have the Derrida map fixed point at zero (a = 1). In addition, the analysis presented can only be directly applied to random Boolean networks with random functions. 4. Measure for information propagation based on perturbation avalanches A perturbation is a random flip to a node in the network. The size of a perturbation avalanche is the total number of nodes that have behaved differently than the nodes in the non-perturbed network. We may interpret the initial perturbation as the signal and the perturbation avalanche as the system s response to the signal. We propose to use the entropy of the avalanche size distribution as a measure for information propagation in Boolean networks. The measure illustrates how wide the scale of responses that the system can have to the initial signal is. We define the measure H as H = p n log p n, n=1 where p n is the probability that the perturbation avalanche has magnitude n. Let us assume that we have a Boolean network with a Poisson out-degree distribution p out ( j) = K j j! e K. Here, p out ( j) is the probability that a node has j out-going connections, and K is the distribution parameter. Randomly constructed graphs have a Poisson out-degree distribution. However, real gene regulatory or neural networks may have a scale-free out-degree distribution [19]. The propagation of a perturbation that has an absolute size Z t Z at time t can be studied by using the theory of branching processes [10,9]. In our analysis we use the initial perturbation size Z 0 = 1. The perturbation size Z t is the non-normalized Hamming distance after t time steps between the network states with and without the perturbation. When we use the annealed approximation in the limit N, all perturbed nodes at any time step are independent of each other. With probability one a perturbation of finite size will spread to nodes that were not previously perturbed. In addition, if Z t = 0, then Z t+a = 0 for any a > 0. Therefore, the propagation of a perturbation in Boolean networks (N ) is a branching process. For a branching process the branching probability distribution q k = P(Z 1 = k) is the probability that a node in the process produces k daughters. For a Boolean network it is given by ( ) j q k = p out ( j)i k (1 I ) j k k = ( I 1 I ) k ( ) j p out ( j)(1 I ) j. k

4 P. Rämö et al. / Physica D 227 (2007) When the out-degree distribution is Poissonian this can be calculated to be ( ) λ k j! K j q k = K λ ( j k)!k! j! e K (1 I ) j = λ k e K (K λ) n k! = λk k! e K e K λ = λk k! e λ. 1 ( j k)! (K λ) j k (K λ) k The generating function of this branching probability distribution is given by Q(s) = n=0 λ n n! e λ s n = e λ(s 1). The iterates of the generating function Q(s) are defined as Q 0 (s) = s, Q t+1 (s) = Q(Q t (s)). In general, the generating function of Z t is Q t (s) [9]. Let λ = E[Z 1 ] = Q (1) and σ 2 = E[Z1 2] λ2 = Q (1) + λ λ 2. The expectation E[Z t ] and variance V [Z t ] are E[Z t ] = Q t (1) = λt, σ 2 λ t (λ t 1) V [Z t ] = λ 2, λ 1 λ tσ 2, λ = 1. The limit lim t Q t (s) = q is q = 1 for ordered and critical networks (λ 1) and constant q < 1 for chaotic networks (λ > 1). Moreover, we can prove that Z t with probability 1 q and Z t 0 with probability q when t [9]. This means that even if the Boolean network is chaotic, there is a considerable probability q that the perturbation size will converge to zero. Perturbation spread in the network can be interpreted as a Markov random walk with adsorbing boundary at i = 0. Let us define p i, j as p i, j = P(Z t+1 = j Z t = i), i, j Z and the generating function of the random walk as P i (s) = p i, j s j. j=0 It is well known that the generating function of a sum of i independent random variables, each distributed with q k, can be expressed as P i (s) = Q(s) i. From this we get the Markov random walk for a Boolean network with a Poisson out-degree distribution as p i, j = (λi) j e λi. j! The avalanche size n is the total number of nodes generated by the branching process. The probability p n that the avalanche Fig. 1. Entropies of avalanche size distributions for Boolean networks generated with varying order parameter λ. The solid line is the theoretical estimate without noise and the broken lines are the numerical experiments with and without noise (r = 0). The amount of noise in the numerical experiments was r = 0.01, r = 0.02, and r = size is n can be conveniently expressed as [10,20] p n = 1 n p n,n 1. (1) This result applies for all Markov random walks [20]. For the Poisson case we have p n = (nλ)n 1 e nλ. n! By using Stirling s approximation n! 2πn( n e )n we have p n 1 2π e n(1 λ) λ n 1 n 3 2, from which we see that for critical Boolean networks (λ = 1) p n n 3 2. This result coincides with the well known result that in critical branching processes the avalanche size distribution is scale-free with exponent γ = 3 2. Fig. 1 shows how the measure H changes when the order parameter λ is varied. We note that the measure is maximized in critical networks. We compared the analytical result to a numerical result obtained from randomly generated quenched Boolean networks. The quenched networks had random functions chosen so that the whole network had the desired λ. In the numerical experiments, we generated 400 Boolean networks with N = 100 nodes. Then, we ran each of them for 100 time steps. The starting point was a random state and a perturbed state with one node having a different value than the random initial state. The total number of nodes that, at some time point, had a different value was the avalanche size. The experiment was repeated for different values λ in small intervals. Sometimes in quenched networks the initial perturbation causes the network to settle down in a different attractor. In

5 104 P. Rämö et al. / Physica D 227 (2007) this case, the perturbation size defined in the above-mentioned way approaches infinity when the number of steps is increased. In addition, especially in chaotic networks, the perturbation avalanche size may approach extremely large values. Therefore, we interpreted avalanche sizes that were larger than the network size as avalanches that percolate throughout the whole system. We included all such large avalanches in the probability of avalanche size n = 100. Most biological systems are stochastic and random fluctuations are common. For these reasons, it is important to study how random perturbations affect the proposed measure. As a simple experiment, we flipped the value of a node with probability r at each time step. We repeated the abovementioned numerical experiments with r = 0.01, r = 0.02, and r = See the broken lines in Fig. 1 for the numerical results. We note that, if noise is present, the maximum is shifted into the ordered regime. In addition, the peak is higher for networks with noise and the measure has much lower values for chaotic networks when noise is present. 5. Discussion We have investigated what kind of Boolean networks best propagate information. We have shown that some methods presented in the literature using mutual information do not correctly measure the information propagation capabilities of the network. We have proposed a new measure for the propagation of information in Boolean networks. The measure is based on the entropy of the size distributions of perturbation avalanches and illustrates the range of possible changes in the network as a consequence of an initial small perturbation. We have found that the measure is maximized in critical networks and in subcritical networks if noise is present. In addition, we have verified with numerical experiments the analytically derived result. Many complex dynamical systems, such as gene regulatory or neuronal networks, should have a wide range of possible ways to react to external signals. At the same time, the system should be robust against stochasticity and noise. Critical systems show perturbation avalanches with a power law distribution that compromises between these two conflicting constraints. Reactions of a large magnitude are relatively probable, while most perturbations do not lead to large avalanches. In addition, we have shown that if noise is present in the system, information propagation is maximized in slightly subcritical networks. In fact, critical networks with additional noise would be slightly chaotic and therefore difficult to control. The results suggest that in the presence of stochasticity and noise, subcritical networks could appear critical. However, this hypothesis requires further exploration. Acknowledgement This work was supported by the Academy of Finland, project No (Finnish Centre of Excellence program ). References [1] J. Beggs, D. Plenz, Neuronal avalanches in neocortical circuits, J. Neurosci. 23 (35) (2003) [2] O. Kinouchi, M. Copelli, Physics of psychophysics: Range of excitable networks is optimized at criticality, Nature Phys. 2 (2006) [3] S. Kauffman, The Origins of Order, Oxford University Press, New York, [4] I. Shmulevich, S. Kauffman, M. Aldana, Eukaryotic cells are dynamically ordered or critical but not chaotic, Proceedings of the National Academy of Sciences of the USA 102 (38) (2005) [5] R. Serra, M. Villani, A. Semeria, Genetic networks models and statistical properties of gene expression data in knock-out experiments, J. Theoret. Biol. 227 (2004) [6] P. Rämö, J. Kesseli, O. Yli-Harja, Perturbation avalanches and criticality in gene regulatory networks, J. Theoret. Biol. 242 (2006) [7] A. Ilachinski, Cellular Automata: A Discrete Universe, World Scientific, [8] B. Luque, A. Ferrara, Measuring mutual information in random Boolean networks, Complex Syst. 11. [9] T. Harris, The Theory of Branching Processes, in: Die Grundlehren Der Mathematischen Wissenschaften, vol. 119, Springer-Verlag, [10] P. Jagers, Branching Processes with Biological Applications, John Wiley and Sons, [11] M. Aldana, S. Coppersmith, L.P. Kadanoff, Boolean Dynamics with Random Couplings, in: Springer Applied Mathematical Sciences Series, Springer, New York, [12] S. Kauffman, Metabolic stability and epigenesis in randomly constructed genetic nets, J. Theoret. Biol. 22 (1969) [13] P. Rämö, J. Kesseli, O. Yli-Harja, Stability of functions in gene regulatory networks, Chaos 15 (2005) [14] B. Derrida, Y. Pomeau, Random networks of automata: A simple annealed approximation, Europhys. Lett. 1 (1986) [15] J. Kesseli, P. Rämö, O. Yli-Harja, Iterated maps for annealed Boolean networks, Phys. Rev. E 74 (2006) [16] D. Hsu, J. Beggs, Neuronal avalanches and criticality: A dynamical model for homeostasis, Neurocomputing 69 (2006) [17] C. Haldeman, J. Beggs, Critical branching captures activity in living neural networks and maximizes the number of metastable states, Phys. Rev. Lett. 94 (2005) [18] P. Rämö, J. Kesseli, O. Yli-Harja, Self-overlap in Boolean networks, in: Fourth International Workshop on Computational Systems Biology, WCSB 2006, Tampere, Finland, June [19] R. Albert, Scale-free networks in cell biology, J. Cell Sci. 118 (21) (2005) [20] M. Dwass, The total progeny in a branching process and related random walk, J. Appl. Probab. 6 (1969)

### Controlling chaos in random Boolean networks

EUROPHYSICS LETTERS 20 March 1997 Europhys. Lett., 37 (9), pp. 597-602 (1997) Controlling chaos in random Boolean networks B. Luque and R. V. Solé Complex Systems Research Group, Departament de Fisica

### Mutual information in random Boolean models of regulatory networks

PHYSICAL REVIEW E 77, 090 2008 Mutual information in random Boolean models of regulatory networks Andre S. Ribeiro,, * Stuart A. Kauffman,,2 Jason Lloyd-Price, Björn Samuelsson, 3 and Joshua E. S. Socolar

### Coupled Random Boolean Network Forming an Artificial Tissue

Coupled Random Boolean Network Forming an Artificial Tissue M. Villani, R. Serra, P.Ingrami, and S.A. Kauffman 2 DSSC, University of Modena and Reggio Emilia, via Allegri 9, I-4200 Reggio Emilia villani.marco@unimore.it,

### Introduction to Random Boolean Networks

Introduction to Random Boolean Networks Carlos Gershenson Centrum Leo Apostel, Vrije Universiteit Brussel. Krijgskundestraat 33 B-1160 Brussel, Belgium cgershen@vub.ac.be http://homepages.vub.ac.be/ cgershen/rbn/tut

### Modeling of Topological Effects in Biological Networks

Internship report Modeling of Topological Effects in Biological Networks Vincent Picard École Normale Supérieure de Cachan (Rennes) Department of Computer Science and Telecommunications Magistère of Computer

### Lyapunov exponents in random Boolean networks

Physica A 284 (2000) 33 45 www.elsevier.com/locate/physa Lyapunov exponents in random Boolean networks Bartolo Luque a;, Ricard V. Sole b;c a Centro de Astrobiolog a (CAB), Ciencias del Espacio, INTA,

### At the Edge of Chaos: Real-time Computations and Self-Organized Criticality in Recurrent Neural Networks

At the Edge of Chaos: Real-time Computations and Self-Organized Criticality in Recurrent Neural Networks Thomas Natschläger Software Competence Center Hagenberg A-4232 Hagenberg, Austria Thomas.Natschlaeger@scch.at

### Asynchronous random Boolean network model based on elementary cellular automata

Asynchronous random Boolean networ model based on elementary cellular automata Mihaela T. Matache* Jac Heidel Department of Mathematics University of Nebrasa at Omaha Omaha, NE 6882-243, USA *dmatache@mail.unomaha.edu

### Network Structure and Activity in Boolean Networks

Network Structure and Activity in Boolean Networks Abhijin Adiga 1, Hilton Galyean 1,3, Chris J. Kuhlman 1, Michael Levet 1,2, Henning S. Mortveit 1,2, and Sichao Wu 1 1 Network Dynamics and Simulation

### Attractor period distribution for critical Boolean networks

Attractor period distribution for critical Boolean networks Florian Greil Institut für Festkörperphysik, Technische Universität Darmstadt, D-64285 Darmstadt, Germany current address: Lehrstuhl für Bioinformatik,

### Classification of Random Boolean Networks

in Artificial Life VIII, Standish, Abbass, Bedau (eds)(mit Press) 2002. pp 1 8 1 Classification of Random Boolean Networks Carlos Gershenson, School of Cognitive and Computer Sciences University of Sussex

### Persistence of Activity in Random Boolean Networks

Persistence of Activity in Random Boolean Networks Shirshendu Chatterjee and Rick Durrett Cornell University December 20, 2008 Abstract We consider a model for gene regulatory networks that is a modification

### Noisy Attractors and Ergodic Sets in Models. of Genetic Regulatory Networks

Noisy Attractors and Ergodic Sets in Models of Genetic Regulatory Networks Andre S. Ribeiro Institute for Biocomplexity and Informatics, Univ. of Calgary, Canada Department of Physics and Astronomy, Univ.

### Boolean Dynamics with Random Couplings

This is page 1 Printer: Opaque this Boolean Dynamics with Random Couplings Maximino Aldana Susan Coppersmith Leo P. Kadanoff To Larry Sirovich on the occasion of his 70th birthday ABSTRACT This paper reviews

### Optimal State Estimation for Boolean Dynamical Systems using a Boolean Kalman Smoother

Optimal State Estimation for Boolean Dynamical Systems using a Boolean Kalman Smoother Mahdi Imani and Ulisses Braga-Neto Department of Electrical and Computer Engineering Texas A&M University College

### Stochastic Wilson-Cowan equations for networks of excitatory and inhibitory neurons II

Stochastic Wilson-Cowan equations for networks of excitatory and inhibitory neurons II Jack Cowan Mathematics Department and Committee on Computational Neuroscience University of Chicago 1 A simple Markov

### Balance between Noise and Information Flow Maximizes Set Complexity of Network Dynamics

Maximizes Set Complexity of Network Dynamics Tuomo Mäki-Marttunen 1,2 *, Juha Kesseli 1, Matti Nykter 1 1 Department of Signal Processing, Tampere University of Technology, Tampere, Finland, 2 Department

### 6.207/14.15: Networks Lecture 4: Erdös-Renyi Graphs and Phase Transitions

6.207/14.15: Networks Lecture 4: Erdös-Renyi Graphs and Phase Transitions Daron Acemoglu and Asu Ozdaglar MIT September 21, 2009 1 Outline Phase transitions Connectivity threshold Emergence and size of

### 14 Branching processes

4 BRANCHING PROCESSES 6 4 Branching processes In this chapter we will consider a rom model for population growth in the absence of spatial or any other resource constraints. So, consider a population of

### arxiv: v1 [cs.sy] 25 Oct 2017

Reconstruct the Logical Network from the Transition Matrix Cailu Wang, Yuegang Tao School of Control Science and Engineering, Hebei University of Technology, Tianjin, 300130, P. R. China arxiv:1710.09681v1

### Random Boolean Networks

Random Boolean Networks Boolean network definition The first Boolean networks were proposed by Stuart A. Kauffman in 1969, as random models of genetic regulatory networks (Kauffman 1969, 1993). A Random

### Self-organized Criticality and Synchronization in a Pulse-coupled Integrate-and-Fire Neuron Model Based on Small World Networks

Commun. Theor. Phys. (Beijing, China) 43 (2005) pp. 466 470 c International Academic Publishers Vol. 43, No. 3, March 15, 2005 Self-organized Criticality and Synchronization in a Pulse-coupled Integrate-and-Fire

### Effects of Interactive Function Forms in a Self-Organized Critical Model Based on Neural Networks

Commun. Theor. Phys. (Beijing, China) 40 (2003) pp. 607 613 c International Academic Publishers Vol. 40, No. 5, November 15, 2003 Effects of Interactive Function Forms in a Self-Organized Critical Model

### Information in Biology

Information in Biology CRI - Centre de Recherches Interdisciplinaires, Paris May 2012 Information processing is an essential part of Life. Thinking about it in quantitative terms may is useful. 1 Living

### Lecture 6: Entropy Rate

Lecture 6: Entropy Rate Entropy rate H(X) Random walk on graph Dr. Yao Xie, ECE587, Information Theory, Duke University Coin tossing versus poker Toss a fair coin and see and sequence Head, Tail, Tail,

### EM-algorithm for Training of State-space Models with Application to Time Series Prediction

EM-algorithm for Training of State-space Models with Application to Time Series Prediction Elia Liitiäinen, Nima Reyhani and Amaury Lendasse Helsinki University of Technology - Neural Networks Research

### State-Feedback Control of Partially-Observed Boolean Dynamical Systems Using RNA-Seq Time Series Data

State-Feedback Control of Partially-Observed Boolean Dynamical Systems Using RNA-Seq Time Series Data Mahdi Imani and Ulisses Braga-Neto Department of Electrical and Computer Engineering Texas A&M University

### Asynchronous updating of threshold-coupled chaotic neurons

PRAMANA c Indian Academy of Sciences Vol. 70, No. 6 journal of June 2008 physics pp. 1127 1134 Asynchronous updating of threshold-coupled chaotic neurons MANISH DEV SHRIMALI 1,2,3,, SUDESHNA SINHA 4 and

### Unusual Frequency Distribution Function Shape Generated by Particles Making Brownian Walk Along Line With Monotone Increasing Friction

International Journal of Mathematics and Computational Science Vol. 1, No. 3, 2015, pp. 91-97 http://www.publicscienceframework.org/journal/ijmcs Unusual Frequency Distribution Function Shape Generated

### Basic modeling approaches for biological systems. Mahesh Bule

Basic modeling approaches for biological systems Mahesh Bule The hierarchy of life from atoms to living organisms Modeling biological processes often requires accounting for action and feedback involving

### Analysis of Neural Networks with Chaotic Dynamics

Chaos, Solitonr & Fructals Vol. 3, No. 2, pp. 133-139, 1993 Printed in Great Britain @60-0779/93\$6.00 + 40 0 1993 Pergamon Press Ltd Analysis of Neural Networks with Chaotic Dynamics FRANCOIS CHAPEAU-BLONDEAU

### arxiv:cond-mat/ v1 [cond-mat.other] 4 Aug 2004

Conservation laws for the voter model in complex networks arxiv:cond-mat/0408101v1 [cond-mat.other] 4 Aug 2004 Krzysztof Suchecki, 1,2 Víctor M. Eguíluz, 1 and Maxi San Miguel 1 1 Instituto Mediterráneo

### DESIGN OF EXPERIMENTS AND BIOCHEMICAL NETWORK INFERENCE

DESIGN OF EXPERIMENTS AND BIOCHEMICAL NETWORK INFERENCE REINHARD LAUBENBACHER AND BRANDILYN STIGLER Abstract. Design of experiments is a branch of statistics that aims to identify efficient procedures

### Lecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes

Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities

### arxiv:chao-dyn/ v1 5 Mar 1996

Turbulence in Globally Coupled Maps M. G. Cosenza and A. Parravano Centro de Astrofísica Teórica, Facultad de Ciencias, Universidad de Los Andes, A. Postal 26 La Hechicera, Mérida 5251, Venezuela (To appear,

### A brief introduction to Conditional Random Fields

A brief introduction to Conditional Random Fields Mark Johnson Macquarie University April, 2005, updated October 2010 1 Talk outline Graphical models Maximum likelihood and maximum conditional likelihood

### Asynchronous Stochastic Boolean Networks as Gene Network Models

Journal of Computational Biology Journal of Computational Biology: http://mc.manuscriptcentral.com/liebert/jcb Asynchronous Stochastic Boolean Networks as Gene Network Models Journal: Journal of Computational

### Quasi-Stationary Simulation: the Subcritical Contact Process

Brazilian Journal of Physics, vol. 36, no. 3A, September, 6 685 Quasi-Stationary Simulation: the Subcritical Contact Process Marcelo Martins de Oliveira and Ronald Dickman Departamento de Física, ICEx,

### CSC321 Lecture 5: Multilayer Perceptrons

CSC321 Lecture 5: Multilayer Perceptrons Roger Grosse Roger Grosse CSC321 Lecture 5: Multilayer Perceptrons 1 / 21 Overview Recall the simple neuron-like unit: y output output bias i'th weight w 1 w2 w3

### Bayesian Inference using Markov Chain Monte Carlo in Phylogenetic Studies

Bayesian Inference using Markov Chain Monte Carlo in Phylogenetic Studies 1 What is phylogeny? Essay written for the course in Markov Chains 2004 Torbjörn Karfunkel Phylogeny is the evolutionary development

### Comparison of perturbation bounds for the stationary distribution of a Markov chain

Linear Algebra and its Applications 335 (00) 37 50 www.elsevier.com/locate/laa Comparison of perturbation bounds for the stationary distribution of a Markov chain Grace E. Cho a, Carl D. Meyer b,, a Mathematics

### A tutorial on numerical transfer operator methods. numerical analysis of dynamical systems

A tutorial on transfer operator methods for numerical analysis of dynamical systems Gary Froyland School of Mathematics and Statistics University of New South Wales, Sydney BIRS Workshop on Uncovering

### Statistical NLP: Hidden Markov Models. Updated 12/15

Statistical NLP: Hidden Markov Models Updated 12/15 Markov Models Markov models are statistical tools that are useful for NLP because they can be used for part-of-speech-tagging applications Their first

### Lecture 4: Perceptrons and Multilayer Perceptrons

Lecture 4: Perceptrons and Multilayer Perceptrons Cognitive Systems II - Machine Learning SS 2005 Part I: Basic Approaches of Concept Learning Perceptrons, Artificial Neuronal Networks Lecture 4: Perceptrons

### WHY ARE DEEP NETS REVERSIBLE: A SIMPLE THEORY,

WHY ARE DEEP NETS REVERSIBLE: A SIMPLE THEORY, WITH IMPLICATIONS FOR TRAINING Sanjeev Arora, Yingyu Liang & Tengyu Ma Department of Computer Science Princeton University Princeton, NJ 08540, USA {arora,yingyul,tengyu}@cs.princeton.edu

### Using a Hopfield Network: A Nuts and Bolts Approach

Using a Hopfield Network: A Nuts and Bolts Approach November 4, 2013 Gershon Wolfe, Ph.D. Hopfield Model as Applied to Classification Hopfield network Training the network Updating nodes Sequencing of

### Effects of Interactive Function Forms and Refractoryperiod in a Self-Organized Critical Model Based on Neural Networks

Commun. Theor. Phys. (Beijing, China) 42 (2004) pp. 121 125 c International Academic Publishers Vol. 42, No. 1, July 15, 2004 Effects of Interactive Function Forms and Refractoryperiod in a Self-Organized

### Control and synchronization of Julia sets of the complex dissipative standard system

Nonlinear Analysis: Modelling and Control, Vol. 21, No. 4, 465 476 ISSN 1392-5113 http://dx.doi.org/10.15388/na.2016.4.3 Control and synchronization of Julia sets of the complex dissipative standard system

### biologically-inspired computing lecture 12 Informatics luis rocha 2015 INDIANA UNIVERSITY biologically Inspired computing

lecture 12 -inspired Sections I485/H400 course outlook Assignments: 35% Students will complete 4/5 assignments based on algorithms presented in class Lab meets in I1 (West) 109 on Lab Wednesdays Lab 0

### Network synchronizability analysis: The theory of subgraphs and complementary graphs

Physica D 237 (2008) 1006 1012 www.elsevier.com/locate/physd Network synchronizability analysis: The theory of subgraphs and complementary graphs Zhisheng Duan a,, Chao Liu a, Guanrong Chen a,b a State

### Learning Cellular Automaton Dynamics with Neural Networks

Learning Cellular Automaton Dynamics with Neural Networks N H Wulff* and J A Hertz t CONNECT, the Niels Bohr Institute and Nordita Blegdamsvej 17, DK-2100 Copenhagen 0, Denmark Abstract We have trained

### Cellular Automata Approaches to Enzymatic Reaction Networks

Cellular Automata Approaches to Enzymatic Reaction Networks Jörg R. Weimar Institute of Scientific Computing, Technical University Braunschweig, D-38092 Braunschweig, Germany J.Weimar@tu-bs.de, http://www.jweimar.de

### The parabolic Anderson model on Z d with time-dependent potential: Frank s works

Weierstrass Institute for Applied Analysis and Stochastics The parabolic Anderson model on Z d with time-dependent potential: Frank s works Based on Frank s works 2006 2016 jointly with Dirk Erhard (Warwick),

### Multilayer Perceptron

Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Single Perceptron 3 Boolean Function Learning 4

### Written Exam 15 December Course name: Introduction to Systems Biology Course no

Technical University of Denmark Written Exam 15 December 2008 Course name: Introduction to Systems Biology Course no. 27041 Aids allowed: Open book exam Provide your answers and calculations on separate

### Evolving network with different edges

Evolving network with different edges Jie Sun, 1,2 Yizhi Ge, 1,3 and Sheng Li 1, * 1 Department of Physics, Shanghai Jiao Tong University, Shanghai, China 2 Department of Mathematics and Computer Science,

### Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training

Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training Charles Elkan elkan@cs.ucsd.edu January 17, 2013 1 Principle of maximum likelihood Consider a family of probability distributions

### Effect of number of hidden neurons on learning in large-scale layered neural networks

ICROS-SICE International Joint Conference 009 August 18-1, 009, Fukuoka International Congress Center, Japan Effect of on learning in large-scale layered neural networks Katsunari Shibata (Oita Univ.;

### HOPFIELD neural networks (HNNs) are a class of nonlinear

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II: EXPRESS BRIEFS, VOL. 52, NO. 4, APRIL 2005 213 Stochastic Noise Process Enhancement of Hopfield Neural Networks Vladimir Pavlović, Member, IEEE, Dan Schonfeld,

### WHITE NOISE APPROACH TO FEYNMAN INTEGRALS. Takeyuki Hida

J. Korean Math. Soc. 38 (21), No. 2, pp. 275 281 WHITE NOISE APPROACH TO FEYNMAN INTEGRALS Takeyuki Hida Abstract. The trajectory of a classical dynamics is detrmined by the least action principle. As

### ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD

ARTIFICIAL NEURAL NETWORK PART I HANIEH BORHANAZAD WHAT IS A NEURAL NETWORK? The simplest definition of a neural network, more properly referred to as an 'artificial' neural network (ANN), is provided

### Dynamical Systems and Deep Learning: Overview. Abbas Edalat

Dynamical Systems and Deep Learning: Overview Abbas Edalat Dynamical Systems The notion of a dynamical system includes the following: A phase or state space, which may be continuous, e.g. the real line,

### arxiv: v1 [q-bio.nc] 30 Apr 2012

Neuronal avalanches of a self-organized neural network with active-neuron-dominant structure Xiumin Li 1, 2, and Michael Small 3, 2 1 College of Automation, Chongqing University, Chongqing 444, China 2

### Recursion and Induction

Recursion and Induction Themes Recursion Recursive Definitions Recurrence Relations Induction (prove properties of recursive programs and objects defined recursively) Examples Tower of Hanoi Gray Codes

### Transport of single molecules along the periodic parallel lattices with coupling

THE JOURNAL OF CHEMICAL PHYSICS 124 204901 2006 Transport of single molecules along the periodic parallel lattices with coupling Evgeny B. Stukalin The James Franck Institute The University of Chicago

### Self Similar (Scale Free, Power Law) Networks (I)

Self Similar (Scale Free, Power Law) Networks (I) E6083: lecture 4 Prof. Predrag R. Jelenković Dept. of Electrical Engineering Columbia University, NY 10027, USA {predrag}@ee.columbia.edu February 7, 2007

### Random maps and attractors in random Boolean networks

PHYSICAL REVIEW E 72, 0462 2005 Rom maps attractors in rom Boolean networks Björn Samuelsson* Carl Troein Complex Systems Division, Department of Theoretical Physics, Lund University, Sölvegatan 4A, S-223

### A note on stochastic context-free grammars, termination and the EM-algorithm

A note on stochastic context-free grammars, termination and the EM-algorithm Niels Richard Hansen Department of Applied Mathematics and Statistics, University of Copenhagen, Universitetsparken 5, 2100

### Cellular Automata. Jason Frank Mathematical Institute

Cellular Automata Jason Frank Mathematical Institute WISM484 Introduction to Complex Systems, Utrecht University, 2015 Cellular Automata Game of Life: Simulator: http://www.bitstorm.org/gameoflife/ Hawking:

### Synchronization and Bifurcation Analysis in Coupled Networks of Discrete-Time Systems

Commun. Theor. Phys. (Beijing, China) 48 (2007) pp. 871 876 c International Academic Publishers Vol. 48, No. 5, November 15, 2007 Synchronization and Bifurcation Analysis in Coupled Networks of Discrete-Time

### Comparison of the Complex Valued and Real Valued Neural Networks Trained with Gradient Descent and Random Search Algorithms

Comparison of the Complex Valued and Real Valued Neural Networks rained with Gradient Descent and Random Search Algorithms Hans Georg Zimmermann, Alexey Minin 2,3 and Victoria Kusherbaeva 3 - Siemens AG

### Lecture 16. Lectures 1-15 Review

18.440: Lecture 16 Lectures 1-15 Review Scott Sheffield MIT 1 Outline Counting tricks and basic principles of probability Discrete random variables 2 Outline Counting tricks and basic principles of probability

### Probabilistic Models in Theoretical Neuroscience

Probabilistic Models in Theoretical Neuroscience visible unit Boltzmann machine semi-restricted Boltzmann machine restricted Boltzmann machine hidden unit Neural models of probabilistic sampling: introduction

### Comparative analysis of transport communication networks and q-type statistics

Comparative analysis of transport communication networs and -type statistics B. R. Gadjiev and T. B. Progulova International University for Nature, Society and Man, 9 Universitetsaya Street, 498 Dubna,

### Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation Ashok Patel and Bart Kosko

IEEE SIGNAL PROCESSING LETTERS, VOL. 17, NO. 12, DECEMBER 2010 1005 Optimal Mean-Square Noise Benefits in Quantizer-Array Linear Estimation Ashok Patel and Bart Kosko Abstract A new theorem shows that

### Anti-synchronization of a new hyperchaotic system via small-gain theorem

Anti-synchronization of a new hyperchaotic system via small-gain theorem Xiao Jian( ) College of Mathematics and Statistics, Chongqing University, Chongqing 400044, China (Received 8 February 2010; revised

### Perturbations in Boolean Networks

Perturbations in Boolean Networks as Model of Gene Regulatory Dynamics Von der Fakultät für Mathematik und Informatik der Universität Leipzig angenommene Dissertation zur Erlangung des akademischen Grades

### Network Science: Principles and Applications

Network Science: Principles and Applications CS 695 - Fall 2016 Amarda Shehu,Fei Li [amarda, lifei](at)gmu.edu Department of Computer Science George Mason University 1 Outline of Today s Class 2 Robustness

### Probabilistic reconstruction of the tumor progression process in gene regulatory networks in the presence of uncertainty

Probabilistic reconstruction of the tumor progression process in gene regulatory networks in the presence of uncertainty Mohammad Shahrokh Esfahani, Byung-Jun Yoon, Edward R. Dougherty,2 Department of

### arxiv:cond-mat/ v1 [cond-mat.dis-nn] 18 Nov 1996

arxiv:cond-mat/9611130v1 [cond-mat.dis-nn] 18 Nov 1996 Learning by dilution in a Neural Network B López and W Kinzel Institut für Theoretische Physik, Universität Würzburg, Am Hubland, D-97074 Würzburg

### Stochastic Processes

Stochastic Processes Stochastic Process Non Formal Definition: Non formal: A stochastic process (random process) is the opposite of a deterministic process such as one defined by a differential equation.

### An Ensemble Learning Approach to Nonlinear Dynamic Blind Source Separation Using State-Space Models

An Ensemble Learning Approach to Nonlinear Dynamic Blind Source Separation Using State-Space Models Harri Valpola, Antti Honkela, and Juha Karhunen Neural Networks Research Centre, Helsinki University

### Storage Capacity of Letter Recognition in Hopfield Networks

Storage Capacity of Letter Recognition in Hopfield Networks Gang Wei (gwei@cs.dal.ca) Zheyuan Yu (zyu@cs.dal.ca) Faculty of Computer Science, Dalhousie University, Halifax, N.S., Canada B3H 1W5 Abstract:

### Neural Networks. Nicholas Ruozzi University of Texas at Dallas

Neural Networks Nicholas Ruozzi University of Texas at Dallas Handwritten Digit Recognition Given a collection of handwritten digits and their corresponding labels, we d like to be able to correctly classify

### Activity Driven Adaptive Stochastic. Resonance. Gregor Wenning and Klaus Obermayer. Technical University of Berlin.

Activity Driven Adaptive Stochastic Resonance Gregor Wenning and Klaus Obermayer Department of Electrical Engineering and Computer Science Technical University of Berlin Franklinstr. 8/9, 187 Berlin fgrewe,obyg@cs.tu-berlin.de

### (each row defines a probability distribution). Given n-strings x X n, y Y n we can use the absence of memory in the channel to compute

ENEE 739C: Advanced Topics in Signal Processing: Coding Theory Instructor: Alexander Barg Lecture 6 (draft; 9/6/03. Error exponents for Discrete Memoryless Channels http://www.enee.umd.edu/ abarg/enee739c/course.html

### Time Series 2. Robert Almgren. Sept. 21, 2009

Time Series 2 Robert Almgren Sept. 21, 2009 This week we will talk about linear time series models: AR, MA, ARMA, ARIMA, etc. First we will talk about theory and after we will talk about fitting the models

### International Journal of Advanced Research in Computer Science and Software Engineering

Volume 3, Issue 4, April 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Application of

### Exploring the Evolution of Modularity in Gene Regulatory Networks

University of Vermont ScholarWorks @ UVM UVM Honors College Senior Theses Undergraduate Theses 2015 Exploring the Evolution of Modularity in Gene Regulatory Networks Mariko L. Totten University of Vermont,

### Last update: October 26, Neural networks. CMSC 421: Section Dana Nau

Last update: October 26, 207 Neural networks CMSC 42: Section 8.7 Dana Nau Outline Applications of neural networks Brains Neural network units Perceptrons Multilayer perceptrons 2 Example Applications

### Markov Chain Model for ALOHA protocol

Markov Chain Model for ALOHA protocol Laila Daniel and Krishnan Narayanan April 22, 2012 Outline of the talk A Markov chain (MC) model for Slotted ALOHA Basic properties of Discrete-time Markov Chain Stability

### Spatial and Temporal Behaviors in a Modified Evolution Model Based on Small World Network

Commun. Theor. Phys. (Beijing, China) 42 (2004) pp. 242 246 c International Academic Publishers Vol. 42, No. 2, August 15, 2004 Spatial and Temporal Behaviors in a Modified Evolution Model Based on Small

### On random Boolean threshold networks

On rando Boolean threshold networs Reinhard Hecel, Steffen Schober and Martin Bossert Institute of Telecounications and Applied Inforation Theory Ul University Albert-Einstein-Allee 43, 89081Ul, Gerany

### Poisson approximations

Chapter 9 Poisson approximations 9.1 Overview The Binn, p) can be thought of as the distribution of a sum of independent indicator random variables X 1 + + X n, with {X i = 1} denoting a head on the ith

### 698 Zou Yan-Li et al Vol. 14 and L 2, respectively, V 0 is the forward voltage drop across the diode, and H(u) is the Heaviside function 8 < 0 u < 0;

Vol 14 No 4, April 2005 cfl 2005 Chin. Phys. Soc. 1009-1963/2005/14(04)/0697-06 Chinese Physics and IOP Publishing Ltd Chaotic coupling synchronization of hyperchaotic oscillators * Zou Yan-Li( ΠΛ) a)y,

### Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform. Santa Fe Institute Working Paper (Submitted to Complex Systems)

Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations Melanie Mitchell 1, Peter T. Hraber 1, and James P. Crutcheld 2 Santa Fe Institute Working Paper 93-3-14 (Submitted to Complex

### Entropy Evaluation for Oscillator-based True Random Number Generators

Entropy Evaluation for Oscillator-based True Random Number Generators Yuan Ma DCS Center Institute of Information Engineering Chinese Academy of Sciences Outline RNG Modeling method Experiment Entropy

### Error Threshold for Individual Faulty Gates Using Probabilistic Transfer Matrix (PTM)

Available online at www.sciencedirect.com ScienceDirect AASRI Procedia 9 (2014 ) 138 145 2014 AASRI Conference on Circuits and Signal Processing (CSP 2014) Error Threshold for Individual Faulty Gates Using