ON THE APPROXIMATION OF STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING BY MASTER EQUATION

Size: px
Start display at page:

Download "ON THE APPROXIMATION OF STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING BY MASTER EQUATION"

Transcription

1 ON THE APPROXIMATION OF STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING BY MASTER EQUATION Luca Bortolussi 1,2 1 Department of Mathematics and Informatics University of Trieste luca@dmi.units.it 2 Center for Biomolecular Medicine, Area Science Park, Trieste QAPL 28, Budapest, 3 th March 28

2 SEMANTICS FOR SPA

3 OUTLINE 1 INTRODUCTION 2 BASICS ON SCCP 3 APPROXIMATIONS AND MASTER EQUATION 4 EXAMPLES

4 STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING CCP = CONSTRAINTS + AGENTS Constraints are formulae over an interpreted first order language (i.e. X = 1, Y > X 3); they can be added to a "container", the constraint store, but can never be removed. Agents can perform two basic operations on this store (asynchronously): tell or ask a constraint. p :- A; π = [g u] λ ; M = π.a M + M A = M p; N = A A N rw(x):- [X > X = X 1] λ(x).rw(x) + [true X = X + 1] λ(x).rw(x) STOCHASTIC CCP Each ask and tell instruction has a rate (function) attached to it: λ : C R +. The semantics of the language is given in terms of a Continuous Markov Chain. L. Bortolussi, Stochastic Concurrent Constraint Programming, QAPL, 26

5 MODELING IN SCCP OREGONATOR MODELING BIOCHEMICAL REACTIONS R R n f (R,X;k) P P m f -reaction(r, X, P, k) :- tell f (R,X;k) (R = R 1 P = P + 1). f -reaction(r, X, P, k) B k1 A A + B k2 A k3 2A + C 2A k4 C k5 B ANALYSIS TOOLS Stochastic simulation (Gillespie algorithm) Stochastic model checking and CTMC analysis Approximation with ODE s and Hybrid Automata L. Bortolussi, A. Policriti. Modeling Biological systems in sccp, Constraints, 13(1), 28.

6 FROM SCCP TO ODE ν = X G 1 G Gene(X) :- [ X = X + 1] kp.gene(x) + [X 1 ] kb X.[ ] ku.gene(x) Degrade(X) :- [X X = X 1] kd X.Degrade(X) φ = k pg 1 k b XG 1 k ug k d X Ẋ = k pg 1 k d X Φ 1 = ν φ : G 1 = k ug k b XG 1 G = k b XG 1 k ug

7 CIRCADIAN CLOCK

8 CIRCADIAN CLOCK p_gate(α A, α A, γ A, θ A, M A, A) p_gate(α R, α R, γ R, θ R, M R, A) reaction(β A, [M A ], [A]) reaction(δ MA, [M A ], []) reaction(β R, [M R ], [R]) reaction(δ MR, [M R ], []) reaction(γ C, [A, R], [AR]) reaction(δ A, [AR], [R]) reaction(δ A, [A], []) reaction(δ R, [R], []) A R A R

9 CIRCADIAN CLOCK p_gate(α A, α A, γ A, θ A, M A, A) p_gate(α R, α R, γ R, θ R, M R, A) reaction(β A, [M A ], [A]) reaction(δ MA, [M A ], []) reaction(β R, [M R ], [R]) reaction(δ MR, [M R ], []) reaction(γ C, [A, R], [AR]) reaction(δ A, [AR], [R]) reaction(δ A, [A], []) reaction(δ R, [R], []) A R

10 MASTER EQUATION FOR SCCP The master equation is equivalent to the Kolmogorov Forward Equation: it is a PDE for the time-evolution of the probability density function P(X, t). INCREASE OF P(X, t) DUE TO t j IN dt P(Y ν j, t)φ j (Y ν j )dt DECREASE OF P(X, t) DUE TO t j IN dt P(Y, t)φ j (Y)dt P(Y, t) t = j ( φj (Y ν j )P(Y ν j, t) φ j (Y)P(Y, t) )

11 FIRST-ORDER APPROXIMATION DIFFERENTIAL EQUATION FOR THE AVERAGE OF SCCP d Y i t dt = Φ 1 i (Y) t TAYLOR EXPANSION OF Φ 1 i (Y) t Φ 1 (Y) t Φ 1 ( Y t ) Y h,k=1 2 hkφ 1 ( Y t ) Y h Y k t FIRST-ORDER EQUATION FOR THE AVERAGE d Y i t dt = Φ 1 i ( Y t )

12 SECOND-ORDER APPROXIMATION EXACT EQUATION FOR COVARIANCE d Y i Y k t dt = Φ 2 ik(y) + (Y i Y i t )Φ 1 k(y) + (Y k Y k t )Φ 1 i (Y) t t t SECOND-ORDER EQUATIONS FOR AVERAGE AND COVARIANCE d Y i t dt = Φ 1 ( Y t ) T (N) h,k=1 2 hkφ 1 ( Y t ) Y h Y k t d Y i Y k t dt Y = Φ 2 ik( Y t ) + h Φ 1 k( Y t ) Y i Y h t h=1 Y + h Φ 1 i ( Y t ) Y k Y h t h=1

13 RANDOM WALK RW X :- [ X = X + 1] k.rw X + [ X = X 1] k.rw X, Φ 1 (X) = Φ 2 (X) = 2k { X = Φ 1 ( X ) X 2 XX 2 Φ 1 ( X ) = X 2 = Φ 2 ( X ) + 2 X 2 X Φ 1 ( X ) = 2k X t = X X 2 t = 2kt + X 2

14 EFFECTS OF VARIANCE R 1 :- [ X = X + 1] k.r 1 ; R 2 :- [ Y = Y + 1] k.r 2 ; R 3 :- [X > X = X 1] α1 X.R 3 R 4 :- [Y > Y = Y 1] α2 Y.R 4 ; R 5 :- [X > Y > X = X 1 Y = Y + 1] ka X Y.R 5 R 1 R 2 R 3 R 4 R 5 Stochastic SO Average/Variance X Y

15 CIRCADIAN CLOCK

16 CIRCADIAN CLOCK Stochastic FO approximation A R A R

17 CIRCADIAN CLOCK Stochastic average FO approximation A R A R

18 CIRCADIAN CLOCK Stochastic average SO approximation A R

19 CIRCADIAN CLOCK Robustness of the system: increase translation rate of R from β R = 5 to β R = 5. Stochastic, β R = 5 FO approximation, β R = A R

20 CIRCADIAN CLOCK Robustness of the system: increase translation rate of R from β R = 5 to β R = 5. Stochastic average, β R = FO approximation, β R = A R

21 CIRCADIAN CLOCK Robustness of the system: increase translation rate of R from β R = 5 to β R = 5. Stochastic average, β R = SO approximation, β R = A R

22 CONCLUSIONS Many works in statistical mechanics deal with the relation between stochastic and deterministic description of systems. The Master Equation for a SPA is the key to use these methods also for the analysis of quantitative programming languages. SPA introduce many new challenges: the main one is synchronization, which introduces discontinuities in the expression of rates. Synchronization is discrete in nature: hybrid schemes of approximation should work better.

Modeling Biological Systems in Stochastic Concurrent Constraint Programming

Modeling Biological Systems in Stochastic Concurrent Constraint Programming Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi 1 Alberto Policriti 1 1 Department of Mathematics and Computer Science University of Udine, Italy. Workshop on

More information

Biochemical simulation by stochastic concurrent constraint programming and hybrid systems

Biochemical simulation by stochastic concurrent constraint programming and hybrid systems Biochemical simulation by stochastic concurrent constraint programming and hybrid systems Luca Bortolussi 1 Alberto Policriti 2,3 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste

More information

Lecture 4 The stochastic ingredient

Lecture 4 The stochastic ingredient Lecture 4 The stochastic ingredient Luca Bortolussi 1 Alberto Policriti 2 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste Via Valerio 12/a, 34100 Trieste. luca@dmi.units.it

More information

Lecture 1 Modeling in Biology: an introduction

Lecture 1 Modeling in Biology: an introduction Lecture 1 in Biology: an introduction Luca Bortolussi 1 Alberto Policriti 2 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste Via Valerio 12/a, 34100 Trieste. luca@dmi.units.it

More information

Logic-based Multi-Objective Design of Chemical Reaction Networks

Logic-based Multi-Objective Design of Chemical Reaction Networks Logic-based Multi-Objective Design of Chemical Reaction Networks Luca Bortolussi 1 Alberto Policriti 2 Simone Silvetti 2,3 1 DMG, University of Trieste, Trieste, Italy luca@dmi.units.it 2 Dima, University

More information

Stochastic Simulation.

Stochastic Simulation. Stochastic Simulation. (and Gillespie s algorithm) Alberto Policriti Dipartimento di Matematica e Informatica Istituto di Genomica Applicata A. Policriti Stochastic Simulation 1/20 Quote of the day D.T.

More information

Modeling Biological Systems in Stochastic Concurrent Constraint Programming

Modeling Biological Systems in Stochastic Concurrent Constraint Programming Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi Alberto Policriti Abstract We present an application of stochastic Concurrent Constraint Programming (sccp) for

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

More information

MASSPA-Modeller: A Spatial Stochastic Process Algebra modelling tool

MASSPA-Modeller: A Spatial Stochastic Process Algebra modelling tool MASSPA-Modeller: A Spatial Stochastic Process Algebra modelling tool Marcel C. Guenther Jeremy T. Bradley Imperial College London, 180 Queen s Gate, London SW7 2AZ, United Kingdom, Email: {mcg05,jb}@doc.ic.ac.uk

More information

Simulation methods for stochastic models in chemistry

Simulation methods for stochastic models in chemistry Simulation methods for stochastic models in chemistry David F. Anderson anderson@math.wisc.edu Department of Mathematics University of Wisconsin - Madison SIAM: Barcelona June 4th, 21 Overview 1. Notation

More information

SPA for quantitative analysis: Lecture 6 Modelling Biological Processes

SPA for quantitative analysis: Lecture 6 Modelling Biological Processes 1/ 223 SPA for quantitative analysis: Lecture 6 Modelling Biological Processes Jane Hillston LFCS, School of Informatics The University of Edinburgh Scotland 7th March 2013 Outline 2/ 223 1 Introduction

More information

Controlled Diffusions and Hamilton-Jacobi Bellman Equations

Controlled Diffusions and Hamilton-Jacobi Bellman Equations Controlled Diffusions and Hamilton-Jacobi Bellman Equations Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2014 Emo Todorov (UW) AMATH/CSE 579, Winter

More information

Stochastic analysis of biochemical reaction networks with absolute concentration robustness

Stochastic analysis of biochemical reaction networks with absolute concentration robustness Stochastic analysis of biochemical reaction networks with absolute concentration robustness David F. Anderson anderson@math.wisc.edu Department of Mathematics University of Wisconsin - Madison Boston University

More information

Embedded Systems 5. Synchronous Composition. Lee/Seshia Section 6.2

Embedded Systems 5. Synchronous Composition. Lee/Seshia Section 6.2 Embedded Systems 5-1 - Synchronous Composition Lee/Seshia Section 6.2 Important semantic model for concurrent composition Here: composition of actors Foundation of Statecharts, Simulink, synchronous programming

More information

Introduction Probabilistic Programming ProPPA Inference Results Conclusions. Embedding Machine Learning in Stochastic Process Algebra.

Introduction Probabilistic Programming ProPPA Inference Results Conclusions. Embedding Machine Learning in Stochastic Process Algebra. Embedding Machine Learning in Stochastic Process Algebra Jane Hillston Joint work with Anastasis Georgoulas and Guido Sanguinetti, School of Informatics, University of Edinburgh 16th August 2017 quan col....

More information

Modelling Biochemical Pathways with Stochastic Process Algebra

Modelling Biochemical Pathways with Stochastic Process Algebra Modelling Biochemical Pathways with Stochastic Process Algebra Jane Hillston. LFCS, University of Edinburgh 13th April 2007 The PEPA project The PEPA project started in Edinburgh in 1991. The PEPA project

More information

Cybergenetics: Control theory for living cells

Cybergenetics: Control theory for living cells Department of Biosystems Science and Engineering, ETH-Zürich Cybergenetics: Control theory for living cells Corentin Briat Joint work with Ankit Gupta and Mustafa Khammash Introduction Overview Cybergenetics:

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

More information

Efficient Leaping Methods for Stochastic Chemical Systems

Efficient Leaping Methods for Stochastic Chemical Systems Efficient Leaping Methods for Stochastic Chemical Systems Ioana Cipcigan Muruhan Rathinam November 18, 28 Abstract. Well stirred chemical reaction systems which involve small numbers of molecules for some

More information

Stochastic models of biochemical systems

Stochastic models of biochemical systems Stochastic models of biochemical systems David F. Anderson anderson@math.wisc.edu Department of Mathematics University of Wisconsin - Madison University of Amsterdam November 14th, 212 Stochastic models

More information

MA 777: Topics in Mathematical Biology

MA 777: Topics in Mathematical Biology MA 777: Topics in Mathematical Biology David Murrugarra Department of Mathematics, University of Kentucky http://www.math.uky.edu/~dmu228/ma777/ Spring 2018 David Murrugarra (University of Kentucky) Lecture

More information

6 Continuous-Time Birth and Death Chains

6 Continuous-Time Birth and Death Chains 6 Continuous-Time Birth and Death Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology.

More information

Continuous Time Markov Chains

Continuous Time Markov Chains Continuous Time Markov Chains Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 2015 Outline Introduction Continuous-Time Markov

More information

Varieties of Stochastic Calculi

Varieties of Stochastic Calculi Research is what I'm doing when I don't know what I'm doing. Wernher Von Braun. Artificial Biochemistry Varieties of Stochastic Calculi Microsoft Research Trento, 26-5-22..26 www.luca.demon.co.uk/artificialbiochemistry.htm

More information

Extending the Tools of Chemical Reaction Engineering to the Molecular Scale

Extending the Tools of Chemical Reaction Engineering to the Molecular Scale Extending the Tools of Chemical Reaction Engineering to the Molecular Scale Multiple-time-scale order reduction for stochastic kinetics James B. Rawlings Department of Chemical and Biological Engineering

More information

Miscellaneous. Regarding reading materials. Again, ask questions (if you have) and ask them earlier

Miscellaneous. Regarding reading materials. Again, ask questions (if you have) and ask them earlier Miscellaneous Regarding reading materials Reading materials will be provided as needed If no assigned reading, it means I think the material from class is sufficient Should be enough for you to do your

More information

The Expressivity of Universal Timed CCP: Undecidability of Monadic FLTL and Closure Operators for Security

The Expressivity of Universal Timed CCP: Undecidability of Monadic FLTL and Closure Operators for Security The Expressivity of Universal Timed CCP: Undecidability of Monadic FLTL and Closure Operators for Security Carlos Olarte and Frank D. Valencia INRIA /CNRS and LIX, Ecole Polytechnique Motivation Concurrent

More information

Fast Probability Generating Function Method for Stochastic Chemical Reaction Networks

Fast Probability Generating Function Method for Stochastic Chemical Reaction Networks MATCH Communications in Mathematical and in Computer Chemistry MATCH Commun. Math. Comput. Chem. 71 (2014) 57-69 ISSN 0340-6253 Fast Probability Generating Function Method for Stochastic Chemical Reaction

More information

Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule

Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule Alan Mackworth UBC CS 322 Uncertainty 2 March 13, 2013 Textbook 6.1.3 Lecture Overview Recap: Probability & Possible World Semantics

More information

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution

More information

18.175: Lecture 30 Markov chains

18.175: Lecture 30 Markov chains 18.175: Lecture 30 Markov chains Scott Sheffield MIT Outline Review what you know about finite state Markov chains Finite state ergodicity and stationarity More general setup Outline Review what you know

More information

Some investigations concerning the CTMC and the ODE model derived from Bio-PEPA

Some investigations concerning the CTMC and the ODE model derived from Bio-PEPA FBTC 2008 Some investigations concerning the CTMC and the ODE model derived from Bio-PEPA Federica Ciocchetta 1,a, Andrea Degasperi 2,b, Jane Hillston 3,a and Muffy Calder 4,b a Laboratory for Foundations

More information

Review: control, feedback, etc. Today s topic: state-space models of systems; linearization

Review: control, feedback, etc. Today s topic: state-space models of systems; linearization Plan of the Lecture Review: control, feedback, etc Today s topic: state-space models of systems; linearization Goal: a general framework that encompasses all examples of interest Once we have mastered

More information

This is now an algebraic equation that can be solved simply:

This is now an algebraic equation that can be solved simply: Simulation of CTMC Monday, November 23, 2015 1:55 PM Homework 4 will be posted by tomorrow morning, due Friday, December 11 at 5 PM. Let's solve the Kolmogorov forward equation for the Poisson counting

More information

PROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS

PROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please

More information

Introduction to stochastic multiscale modelling of tumour growth

Introduction to stochastic multiscale modelling of tumour growth Introduction to stochastic multiscale modelling of tumour growth Tomás Alarcón ICREA & Centre de Recerca Matemàtica T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016

More information

EVOLUTIONARY DISTANCES

EVOLUTIONARY DISTANCES EVOLUTIONARY DISTANCES FROM STRINGS TO TREES Luca Bortolussi 1 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste luca@dmi.units.it Trieste, 14 th November 2007 OUTLINE 1 STRINGS:

More information

Master s Written Examination

Master s Written Examination Master s Written Examination Option: Statistics and Probability Spring 05 Full points may be obtained for correct answers to eight questions Each numbered question (which may have several parts) is worth

More information

COMP2610/COMP Information Theory

COMP2610/COMP Information Theory COMP2610/COMP6261 - Information Theory Lecture 9: Probabilistic Inequalities Mark Reid and Aditya Menon Research School of Computer Science The Australian National University August 19th, 2014 Mark Reid

More information

First Order Linear Ordinary Differential Equations

First Order Linear Ordinary Differential Equations First Order Linear Ordinary Differential Equations The most general first order linear ODE is an equation of the form p t dy dt q t y t f t. 1 Herepqarecalledcoefficients f is referred to as the forcing

More information

P i [B k ] = lim. n=1 p(n) ii <. n=1. V i :=

P i [B k ] = lim. n=1 p(n) ii <. n=1. V i := 2.7. Recurrence and transience Consider a Markov chain {X n : n N 0 } on state space E with transition matrix P. Definition 2.7.1. A state i E is called recurrent if P i [X n = i for infinitely many n]

More information

Chemical reaction network theory for stochastic and deterministic models of biochemical reaction systems

Chemical reaction network theory for stochastic and deterministic models of biochemical reaction systems Chemical reaction network theory for stochastic and deterministic models of biochemical reaction systems University of Wisconsin at Madison anderson@math.wisc.edu MBI Workshop for Young Researchers in

More information

The First Derivative and Second Derivative Test

The First Derivative and Second Derivative Test The First Derivative and Second Derivative Test James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University April 9, 2018 Outline 1 Extremal Values 2

More information

Winter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo

Winter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 9: Markov Chain Monte Carlo 9.1 Markov Chain A Markov Chain Monte

More information

Birth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes

Birth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains

More information

Population Genetics: a tutorial

Population Genetics: a tutorial : a tutorial Institute for Science and Technology Austria ThRaSh 2014 provides the basic mathematical foundation of evolutionary theory allows a better understanding of experiments allows the development

More information

Gillespie s Algorithm and its Approximations. Des Higham Department of Mathematics and Statistics University of Strathclyde

Gillespie s Algorithm and its Approximations. Des Higham Department of Mathematics and Statistics University of Strathclyde Gillespie s Algorithm and its Approximations Des Higham Department of Mathematics and Statistics University of Strathclyde djh@maths.strath.ac.uk The Three Lectures 1 Gillespie s algorithm and its relation

More information

Stochastic2010 Page 1

Stochastic2010 Page 1 Stochastic2010 Page 1 Extinction Probability for Branching Processes Friday, April 02, 2010 2:03 PM Long-time properties for branching processes Clearly state 0 is an absorbing state, forming its own recurrent

More information

The First Derivative and Second Derivative Test

The First Derivative and Second Derivative Test The First Derivative and Second Derivative Test James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 8, 2017 Outline Extremal Values The

More information

CoBiC: Context-dependent Bioambient Calculus

CoBiC: Context-dependent Bioambient Calculus QAPL 2009 CoBiC: Context-dependent Bioambient Calculus Luca Bortolussi,3 Dipartimento di Matematica ed Informatica University of Trieste, Trieste, Italy Maria G. Vigliotti 4,2 Department of Computing,

More information

Sequence Modelling with Features: Linear-Chain Conditional Random Fields. COMP-599 Oct 6, 2015

Sequence Modelling with Features: Linear-Chain Conditional Random Fields. COMP-599 Oct 6, 2015 Sequence Modelling with Features: Linear-Chain Conditional Random Fields COMP-599 Oct 6, 2015 Announcement A2 is out. Due Oct 20 at 1pm. 2 Outline Hidden Markov models: shortcomings Generative vs. discriminative

More information

Quiz 1. Name: Instructions: Closed book, notes, and no electronic devices.

Quiz 1. Name: Instructions: Closed book, notes, and no electronic devices. Quiz 1. Name: Instructions: Closed book, notes, and no electronic devices. 1. What is the difference between a deterministic model and a probabilistic model? (Two or three sentences only). 2. What is the

More information

Reaction time distributions in chemical kinetics: Oscillations and other weird behaviors

Reaction time distributions in chemical kinetics: Oscillations and other weird behaviors Introduction The algorithm Results Summary Reaction time distributions in chemical kinetics: Oscillations and other weird behaviors Ramon Xulvi-Brunet Escuela Politécnica Nacional Outline Introduction

More information

Continuum Limit of Forward Kolmogorov Equation Friday, March 06, :04 PM

Continuum Limit of Forward Kolmogorov Equation Friday, March 06, :04 PM Continuum Limit of Forward Kolmogorov Equation Friday, March 06, 2015 2:04 PM Please note that one of the equations (for ordinary Brownian motion) in Problem 1 was corrected on Wednesday night. And actually

More information

Quantitative Model Checking (QMC) - SS12

Quantitative Model Checking (QMC) - SS12 Quantitative Model Checking (QMC) - SS12 Lecture 06 David Spieler Saarland University, Germany June 4, 2012 1 / 34 Deciding Bisimulations 2 / 34 Partition Refinement Algorithm Notation: A partition P over

More information

Solving a system of Master Equations for parallel chemical interactions

Solving a system of Master Equations for parallel chemical interactions Technical Report CoSBi 21/2007 Solving a system of Master Equations for parallel chemical interactions Paola Lecca The Microsoft Research - University of Trento Centre for Computational and Systems Biology

More information

Stochastic Simulation of Biochemical Reactions

Stochastic Simulation of Biochemical Reactions 1 / 75 Stochastic Simulation of Biochemical Reactions Jorge Júlvez University of Zaragoza 2 / 75 Outline 1 Biochemical Kinetics 2 Reaction Rate Equation 3 Chemical Master Equation 4 Stochastic Simulation

More information

2 Conceptual Framework Before introducing the probabilistic concurrent constraint (PCCP) language we have to discuss a basic question: What is a proba

2 Conceptual Framework Before introducing the probabilistic concurrent constraint (PCCP) language we have to discuss a basic question: What is a proba On Probabilistic CCP Alessandra Di Pierro and Herbert Wiklicky fadp,herbertg@cs.city.ac.uk City University London, Northampton Square, London EC1V OHB Abstract This paper investigates a probabilistic version

More information

http://www.math.uah.edu/stat/markov/.xhtml 1 of 9 7/16/2009 7:20 AM Virtual Laboratories > 16. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 1. A Markov process is a random process in which the future is

More information

Note Set 5: Hidden Markov Models

Note Set 5: Hidden Markov Models Note Set 5: Hidden Markov Models Probabilistic Learning: Theory and Algorithms, CS 274A, Winter 2016 1 Hidden Markov Models (HMMs) 1.1 Introduction Consider observed data vectors x t that are d-dimensional

More information

CoBiC: Context-dependent Bioambient Calculus

CoBiC: Context-dependent Bioambient Calculus Electronic Notes in Theoretical Computer Science 253 (2009) 87 20 www.elsevier.com/locate/entcs CoBiC: Context-dependent Bioambient Calculus Luca Bortolussi,3 Dipartimento di Matematica ed Informatica

More information

Parallel Composition in Biology

Parallel Composition in Biology Parallel Composition in Biology Nir Piterman Based on joint work with: J. Fisher, A. Hajnal, B. Hall, T. A. Henzinger, M. Mateescu, D. Nickovic, A.V. Singh, and M.Y. Vardi Are there useful macros to structure

More information

M4A42 APPLIED STOCHASTIC PROCESSES

M4A42 APPLIED STOCHASTIC PROCESSES M4A42 APPLIED STOCHASTIC PROCESSES G.A. Pavliotis Department of Mathematics Imperial College London, UK LECTURE 1 12/10/2009 Lectures: Mondays 09:00-11:00, Huxley 139, Tuesdays 09:00-10:00, Huxley 144.

More information

Reinforcement Learning

Reinforcement Learning Reinforcement Learning Policy gradients Daniel Hennes 26.06.2017 University Stuttgart - IPVS - Machine Learning & Robotics 1 Policy based reinforcement learning So far we approximated the action value

More information

Eulerian (Probability-Based) Approach

Eulerian (Probability-Based) Approach Eulerian (Probability-Based) Approach Tuesday, March 03, 2015 1:59 PM Office hours for Wednesday, March 4 shifted to 5:30-6:30 PM. Homework 2 posted, due Tuesday, March 17 at 2 PM. correction: the drifts

More information

Markov Decision Processes and Dynamic Programming

Markov Decision Processes and Dynamic Programming Markov Decision Processes and Dynamic Programming A. LAZARIC (SequeL Team @INRIA-Lille) Ecole Centrale - Option DAD SequeL INRIA Lille EC-RL Course In This Lecture A. LAZARIC Markov Decision Processes

More information

Let's transfer our results for conditional probability for events into conditional probabilities for random variables.

Let's transfer our results for conditional probability for events into conditional probabilities for random variables. Kolmogorov/Smoluchowski equation approach to Brownian motion Tuesday, February 12, 2013 1:53 PM Readings: Gardiner, Secs. 1.2, 3.8.1, 3.8.2 Einstein Homework 1 due February 22. Conditional probability

More information

Lecture 6: Bayesian Inference in SDE Models

Lecture 6: Bayesian Inference in SDE Models Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs

More information

Open Problem: Approximate Planning of POMDPs in the class of Memoryless Policies

Open Problem: Approximate Planning of POMDPs in the class of Memoryless Policies Open Problem: Approximate Planning of POMDPs in the class of Memoryless Policies Kamyar Azizzadenesheli U.C. Irvine Joint work with Prof. Anima Anandkumar and Dr. Alessandro Lazaric. Motivation +1 Agent-Environment

More information

Handbook of Stochastic Methods

Handbook of Stochastic Methods C. W. Gardiner Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences Third Edition With 30 Figures Springer Contents 1. A Historical Introduction 1 1.1 Motivation I 1.2 Some Historical

More information

Frequency Spectra and Inference in Population Genetics

Frequency Spectra and Inference in Population Genetics Frequency Spectra and Inference in Population Genetics Although coalescent models have come to play a central role in population genetics, there are some situations where genealogies may not lead to efficient

More information

Modelling in Systems Biology

Modelling in Systems Biology Modelling in Systems Biology Maria Grazia Vigliotti thanks to my students Anton Stefanek, Ahmed Guecioueur Imperial College Formal representation of chemical reactions precise qualitative and quantitative

More information

Modeling biological systems with delays in Bio-PEPA

Modeling biological systems with delays in Bio-PEPA Modeling biological systems with delays in Bio-PEPA Giulio Caravagna Dipartimento di Informatica, Università di Pisa, argo Pontecorvo 3, 56127 Pisa, Italy. caravagn@di.unipi.it Jane Hillston aboratory

More information

Stochastic contraction BACS Workshop Chamonix, January 14, 2008

Stochastic contraction BACS Workshop Chamonix, January 14, 2008 Stochastic contraction BACS Workshop Chamonix, January 14, 2008 Q.-C. Pham N. Tabareau J.-J. Slotine Q.-C. Pham, N. Tabareau, J.-J. Slotine () Stochastic contraction 1 / 19 Why stochastic contraction?

More information

Fluid Limits of Queueing Networks with Batches

Fluid Limits of Queueing Networks with Batches Fluid Limits of Queueing etworks with Batches Luca Bortolussi Department of Mathematics and Computer Science University of Trieste, Italy luca@dmi.units.it Mirco Tribastone Institute of Informatics Ludwig-Maximilians-Universität

More information

Dynamical Systems and Deep Learning: Overview. Abbas Edalat

Dynamical Systems and Deep Learning: Overview. Abbas Edalat Dynamical Systems and Deep Learning: Overview Abbas Edalat Dynamical Systems The notion of a dynamical system includes the following: A phase or state space, which may be continuous, e.g. the real line,

More information

Stochastic modelling of epidemic spread

Stochastic modelling of epidemic spread Stochastic modelling of epidemic spread Julien Arino Centre for Research on Inner City Health St Michael s Hospital Toronto On leave from Department of Mathematics University of Manitoba Julien Arino@umanitoba.ca

More information

Outline. CSE 573: Artificial Intelligence Autumn Agent. Partial Observability. Markov Decision Process (MDP) 10/31/2012

Outline. CSE 573: Artificial Intelligence Autumn Agent. Partial Observability. Markov Decision Process (MDP) 10/31/2012 CSE 573: Artificial Intelligence Autumn 2012 Reasoning about Uncertainty & Hidden Markov Models Daniel Weld Many slides adapted from Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1 Outline

More information

Publications. Refereed Journal Publications

Publications. Refereed Journal Publications Publications Refereed Journal Publications [A1] [A2] [A3] [A4] [A5] [A6] [A7] [A8] [A9] C. Baier, J.-P. Katoen, H. Hermanns, and V. Wolf. Comparative branching-time semantics for Markov chains. In: Information

More information

Lecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past.

Lecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past. 1 Markov chain: definition Lecture 5 Definition 1.1 Markov chain] A sequence of random variables (X n ) n 0 taking values in a measurable state space (S, S) is called a (discrete time) Markov chain, if

More information

High-dimensional Problems in Finance and Economics. Thomas M. Mertens

High-dimensional Problems in Finance and Economics. Thomas M. Mertens High-dimensional Problems in Finance and Economics Thomas M. Mertens NYU Stern Risk Economics Lab April 17, 2012 1 / 78 Motivation Many problems in finance and economics are high dimensional. Dynamic Optimization:

More information

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods

Pattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs

More information

Switching Regime Estimation

Switching Regime Estimation Switching Regime Estimation Series de Tiempo BIrkbeck March 2013 Martin Sola (FE) Markov Switching models 01/13 1 / 52 The economy (the time series) often behaves very different in periods such as booms

More information

Brownian Motion. An Undergraduate Introduction to Financial Mathematics. J. Robert Buchanan. J. Robert Buchanan Brownian Motion

Brownian Motion. An Undergraduate Introduction to Financial Mathematics. J. Robert Buchanan. J. Robert Buchanan Brownian Motion Brownian Motion An Undergraduate Introduction to Financial Mathematics J. Robert Buchanan 2010 Background We have already seen that the limiting behavior of a discrete random walk yields a derivation of

More information

Population models from PEPA descriptions

Population models from PEPA descriptions Population models from PEPA descriptions Jane Hillston LFCS, The University of Edinburgh, Edinburgh EH9 3JZ, Scotland. Email: jeh@inf.ed.ac.uk 1 Introduction Stochastic process algebras (e.g. PEPA [10],

More information

Math 250B Final Exam Review Session Spring 2015 SOLUTIONS

Math 250B Final Exam Review Session Spring 2015 SOLUTIONS Math 5B Final Exam Review Session Spring 5 SOLUTIONS Problem Solve x x + y + 54te 3t and y x + 4y + 9e 3t λ SOLUTION: We have det(a λi) if and only if if and 4 λ only if λ 3λ This means that the eigenvalues

More information

Modelling protein trafficking: progress and challenges

Modelling protein trafficking: progress and challenges Modelling protein trafficking: progress and challenges Laboratory for Foundations of Computer Science School of Informatics University of Edinburgh 21 May 2012 Outline Protein Trafficking Modelling Biology

More information

Chapter 3 - Temporal processes

Chapter 3 - Temporal processes STK4150 - Intro 1 Chapter 3 - Temporal processes Odd Kolbjørnsen and Geir Storvik January 23 2017 STK4150 - Intro 2 Temporal processes Data collected over time Past, present, future, change Temporal aspect

More information

Introduction to Artificial Intelligence. Logical Agents

Introduction to Artificial Intelligence. Logical Agents Introduction to Artificial Intelligence Logical Agents (Logic, Deduction, Knowledge Representation) Bernhard Beckert UNIVERSITÄT KOBLENZ-LANDAU Winter Term 2004/2005 B. Beckert: KI für IM p.1 Outline Knowledge-based

More information

Synchronization Transitions in Complex Networks

Synchronization Transitions in Complex Networks Synchronization Transitions in Complex Networks Y. Moreno 1,2,3 1 Institute for Biocomputation and Physics of Complex Systems (BIFI) University of Zaragoza, Zaragoza 50018, Spain 2 Department of Theoretical

More information

Logical Agents. Knowledge based agents. Knowledge based agents. Knowledge based agents. The Wumpus World. Knowledge Bases 10/20/14

Logical Agents. Knowledge based agents. Knowledge based agents. Knowledge based agents. The Wumpus World. Knowledge Bases 10/20/14 0/0/4 Knowledge based agents Logical Agents Agents need to be able to: Store information about their environment Update and reason about that information Russell and Norvig, chapter 7 Knowledge based agents

More information

AN INTRODUCTION TO DISCRETE-EVENT SIMULATION

AN INTRODUCTION TO DISCRETE-EVENT SIMULATION AN INTRODUCTION TO DISCRETE-EVENT SIMULATION Peter W. Glynn 1 Peter J. Haas 2 1 Dept. of Management Science and Engineering Stanford University 2 IBM Almaden Research Center San Jose, CA CAVEAT: WE ARE

More information

M. Dechesne R. Sepulchre

M. Dechesne R. Sepulchre Systems and models in chronobiology A delay model for the circadian rhythm M. Dechesne R. Sepulchre Department of Electrical Engineering and Computer Science Monteore Institute University of Liège 24th

More information

18.600: Lecture 32 Markov Chains

18.600: Lecture 32 Markov Chains 18.600: Lecture 32 Markov Chains Scott Sheffield MIT Outline Markov chains Examples Ergodicity and stationarity Outline Markov chains Examples Ergodicity and stationarity Markov chains Consider a sequence

More information

LIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE

LIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE International Journal of Applied Mathematics Volume 31 No. 18, 41-49 ISSN: 1311-178 (printed version); ISSN: 1314-86 (on-line version) doi: http://dx.doi.org/1.173/ijam.v31i.6 LIMITING PROBABILITY TRANSITION

More information

COMS 4771 Probabilistic Reasoning via Graphical Models. Nakul Verma

COMS 4771 Probabilistic Reasoning via Graphical Models. Nakul Verma COMS 4771 Probabilistic Reasoning via Graphical Models Nakul Verma Last time Dimensionality Reduction Linear vs non-linear Dimensionality Reduction Principal Component Analysis (PCA) Non-linear methods

More information

Reasoning under Uncertainty: Intro to Probability

Reasoning under Uncertainty: Intro to Probability Reasoning under Uncertainty: Intro to Probability Computer Science cpsc322, Lecture 24 (Textbook Chpt 6.1, 6.1.1) March, 15, 2010 CPSC 322, Lecture 24 Slide 1 To complete your Learning about Logics Review

More information

The Markov Decision Process (MDP) model

The Markov Decision Process (MDP) model Decision Making in Robots and Autonomous Agents The Markov Decision Process (MDP) model Subramanian Ramamoorthy School of Informatics 25 January, 2013 In the MAB Model We were in a single casino and the

More information

Logic-Based Modeling in Systems Biology

Logic-Based Modeling in Systems Biology Logic-Based Modeling in Systems Biology Alexander Bockmayr LPNMR 09, Potsdam, 16 September 2009 DFG Research Center Matheon Mathematics for key technologies Outline A.Bockmayr, FU Berlin/Matheon 2 I. Systems

More information