ON THE APPROXIMATION OF STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING BY MASTER EQUATION
|
|
- Christina Higgins
- 5 years ago
- Views:
Transcription
1 ON THE APPROXIMATION OF STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING BY MASTER EQUATION Luca Bortolussi 1,2 1 Department of Mathematics and Informatics University of Trieste luca@dmi.units.it 2 Center for Biomolecular Medicine, Area Science Park, Trieste QAPL 28, Budapest, 3 th March 28
2 SEMANTICS FOR SPA
3 OUTLINE 1 INTRODUCTION 2 BASICS ON SCCP 3 APPROXIMATIONS AND MASTER EQUATION 4 EXAMPLES
4 STOCHASTIC CONCURRENT CONSTRAINT PROGRAMMING CCP = CONSTRAINTS + AGENTS Constraints are formulae over an interpreted first order language (i.e. X = 1, Y > X 3); they can be added to a "container", the constraint store, but can never be removed. Agents can perform two basic operations on this store (asynchronously): tell or ask a constraint. p :- A; π = [g u] λ ; M = π.a M + M A = M p; N = A A N rw(x):- [X > X = X 1] λ(x).rw(x) + [true X = X + 1] λ(x).rw(x) STOCHASTIC CCP Each ask and tell instruction has a rate (function) attached to it: λ : C R +. The semantics of the language is given in terms of a Continuous Markov Chain. L. Bortolussi, Stochastic Concurrent Constraint Programming, QAPL, 26
5 MODELING IN SCCP OREGONATOR MODELING BIOCHEMICAL REACTIONS R R n f (R,X;k) P P m f -reaction(r, X, P, k) :- tell f (R,X;k) (R = R 1 P = P + 1). f -reaction(r, X, P, k) B k1 A A + B k2 A k3 2A + C 2A k4 C k5 B ANALYSIS TOOLS Stochastic simulation (Gillespie algorithm) Stochastic model checking and CTMC analysis Approximation with ODE s and Hybrid Automata L. Bortolussi, A. Policriti. Modeling Biological systems in sccp, Constraints, 13(1), 28.
6 FROM SCCP TO ODE ν = X G 1 G Gene(X) :- [ X = X + 1] kp.gene(x) + [X 1 ] kb X.[ ] ku.gene(x) Degrade(X) :- [X X = X 1] kd X.Degrade(X) φ = k pg 1 k b XG 1 k ug k d X Ẋ = k pg 1 k d X Φ 1 = ν φ : G 1 = k ug k b XG 1 G = k b XG 1 k ug
7 CIRCADIAN CLOCK
8 CIRCADIAN CLOCK p_gate(α A, α A, γ A, θ A, M A, A) p_gate(α R, α R, γ R, θ R, M R, A) reaction(β A, [M A ], [A]) reaction(δ MA, [M A ], []) reaction(β R, [M R ], [R]) reaction(δ MR, [M R ], []) reaction(γ C, [A, R], [AR]) reaction(δ A, [AR], [R]) reaction(δ A, [A], []) reaction(δ R, [R], []) A R A R
9 CIRCADIAN CLOCK p_gate(α A, α A, γ A, θ A, M A, A) p_gate(α R, α R, γ R, θ R, M R, A) reaction(β A, [M A ], [A]) reaction(δ MA, [M A ], []) reaction(β R, [M R ], [R]) reaction(δ MR, [M R ], []) reaction(γ C, [A, R], [AR]) reaction(δ A, [AR], [R]) reaction(δ A, [A], []) reaction(δ R, [R], []) A R
10 MASTER EQUATION FOR SCCP The master equation is equivalent to the Kolmogorov Forward Equation: it is a PDE for the time-evolution of the probability density function P(X, t). INCREASE OF P(X, t) DUE TO t j IN dt P(Y ν j, t)φ j (Y ν j )dt DECREASE OF P(X, t) DUE TO t j IN dt P(Y, t)φ j (Y)dt P(Y, t) t = j ( φj (Y ν j )P(Y ν j, t) φ j (Y)P(Y, t) )
11 FIRST-ORDER APPROXIMATION DIFFERENTIAL EQUATION FOR THE AVERAGE OF SCCP d Y i t dt = Φ 1 i (Y) t TAYLOR EXPANSION OF Φ 1 i (Y) t Φ 1 (Y) t Φ 1 ( Y t ) Y h,k=1 2 hkφ 1 ( Y t ) Y h Y k t FIRST-ORDER EQUATION FOR THE AVERAGE d Y i t dt = Φ 1 i ( Y t )
12 SECOND-ORDER APPROXIMATION EXACT EQUATION FOR COVARIANCE d Y i Y k t dt = Φ 2 ik(y) + (Y i Y i t )Φ 1 k(y) + (Y k Y k t )Φ 1 i (Y) t t t SECOND-ORDER EQUATIONS FOR AVERAGE AND COVARIANCE d Y i t dt = Φ 1 ( Y t ) T (N) h,k=1 2 hkφ 1 ( Y t ) Y h Y k t d Y i Y k t dt Y = Φ 2 ik( Y t ) + h Φ 1 k( Y t ) Y i Y h t h=1 Y + h Φ 1 i ( Y t ) Y k Y h t h=1
13 RANDOM WALK RW X :- [ X = X + 1] k.rw X + [ X = X 1] k.rw X, Φ 1 (X) = Φ 2 (X) = 2k { X = Φ 1 ( X ) X 2 XX 2 Φ 1 ( X ) = X 2 = Φ 2 ( X ) + 2 X 2 X Φ 1 ( X ) = 2k X t = X X 2 t = 2kt + X 2
14 EFFECTS OF VARIANCE R 1 :- [ X = X + 1] k.r 1 ; R 2 :- [ Y = Y + 1] k.r 2 ; R 3 :- [X > X = X 1] α1 X.R 3 R 4 :- [Y > Y = Y 1] α2 Y.R 4 ; R 5 :- [X > Y > X = X 1 Y = Y + 1] ka X Y.R 5 R 1 R 2 R 3 R 4 R 5 Stochastic SO Average/Variance X Y
15 CIRCADIAN CLOCK
16 CIRCADIAN CLOCK Stochastic FO approximation A R A R
17 CIRCADIAN CLOCK Stochastic average FO approximation A R A R
18 CIRCADIAN CLOCK Stochastic average SO approximation A R
19 CIRCADIAN CLOCK Robustness of the system: increase translation rate of R from β R = 5 to β R = 5. Stochastic, β R = 5 FO approximation, β R = A R
20 CIRCADIAN CLOCK Robustness of the system: increase translation rate of R from β R = 5 to β R = 5. Stochastic average, β R = FO approximation, β R = A R
21 CIRCADIAN CLOCK Robustness of the system: increase translation rate of R from β R = 5 to β R = 5. Stochastic average, β R = SO approximation, β R = A R
22 CONCLUSIONS Many works in statistical mechanics deal with the relation between stochastic and deterministic description of systems. The Master Equation for a SPA is the key to use these methods also for the analysis of quantitative programming languages. SPA introduce many new challenges: the main one is synchronization, which introduces discontinuities in the expression of rates. Synchronization is discrete in nature: hybrid schemes of approximation should work better.
Modeling Biological Systems in Stochastic Concurrent Constraint Programming
Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi 1 Alberto Policriti 1 1 Department of Mathematics and Computer Science University of Udine, Italy. Workshop on
More informationBiochemical simulation by stochastic concurrent constraint programming and hybrid systems
Biochemical simulation by stochastic concurrent constraint programming and hybrid systems Luca Bortolussi 1 Alberto Policriti 2,3 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste
More informationLecture 4 The stochastic ingredient
Lecture 4 The stochastic ingredient Luca Bortolussi 1 Alberto Policriti 2 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste Via Valerio 12/a, 34100 Trieste. luca@dmi.units.it
More informationLecture 1 Modeling in Biology: an introduction
Lecture 1 in Biology: an introduction Luca Bortolussi 1 Alberto Policriti 2 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste Via Valerio 12/a, 34100 Trieste. luca@dmi.units.it
More informationLogic-based Multi-Objective Design of Chemical Reaction Networks
Logic-based Multi-Objective Design of Chemical Reaction Networks Luca Bortolussi 1 Alberto Policriti 2 Simone Silvetti 2,3 1 DMG, University of Trieste, Trieste, Italy luca@dmi.units.it 2 Dima, University
More informationStochastic Simulation.
Stochastic Simulation. (and Gillespie s algorithm) Alberto Policriti Dipartimento di Matematica e Informatica Istituto di Genomica Applicata A. Policriti Stochastic Simulation 1/20 Quote of the day D.T.
More informationModeling Biological Systems in Stochastic Concurrent Constraint Programming
Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi Alberto Policriti Abstract We present an application of stochastic Concurrent Constraint Programming (sccp) for
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationMASSPA-Modeller: A Spatial Stochastic Process Algebra modelling tool
MASSPA-Modeller: A Spatial Stochastic Process Algebra modelling tool Marcel C. Guenther Jeremy T. Bradley Imperial College London, 180 Queen s Gate, London SW7 2AZ, United Kingdom, Email: {mcg05,jb}@doc.ic.ac.uk
More informationSimulation methods for stochastic models in chemistry
Simulation methods for stochastic models in chemistry David F. Anderson anderson@math.wisc.edu Department of Mathematics University of Wisconsin - Madison SIAM: Barcelona June 4th, 21 Overview 1. Notation
More informationSPA for quantitative analysis: Lecture 6 Modelling Biological Processes
1/ 223 SPA for quantitative analysis: Lecture 6 Modelling Biological Processes Jane Hillston LFCS, School of Informatics The University of Edinburgh Scotland 7th March 2013 Outline 2/ 223 1 Introduction
More informationControlled Diffusions and Hamilton-Jacobi Bellman Equations
Controlled Diffusions and Hamilton-Jacobi Bellman Equations Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2014 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationStochastic analysis of biochemical reaction networks with absolute concentration robustness
Stochastic analysis of biochemical reaction networks with absolute concentration robustness David F. Anderson anderson@math.wisc.edu Department of Mathematics University of Wisconsin - Madison Boston University
More informationEmbedded Systems 5. Synchronous Composition. Lee/Seshia Section 6.2
Embedded Systems 5-1 - Synchronous Composition Lee/Seshia Section 6.2 Important semantic model for concurrent composition Here: composition of actors Foundation of Statecharts, Simulink, synchronous programming
More informationIntroduction Probabilistic Programming ProPPA Inference Results Conclusions. Embedding Machine Learning in Stochastic Process Algebra.
Embedding Machine Learning in Stochastic Process Algebra Jane Hillston Joint work with Anastasis Georgoulas and Guido Sanguinetti, School of Informatics, University of Edinburgh 16th August 2017 quan col....
More informationModelling Biochemical Pathways with Stochastic Process Algebra
Modelling Biochemical Pathways with Stochastic Process Algebra Jane Hillston. LFCS, University of Edinburgh 13th April 2007 The PEPA project The PEPA project started in Edinburgh in 1991. The PEPA project
More informationCybergenetics: Control theory for living cells
Department of Biosystems Science and Engineering, ETH-Zürich Cybergenetics: Control theory for living cells Corentin Briat Joint work with Ankit Gupta and Mustafa Khammash Introduction Overview Cybergenetics:
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationEfficient Leaping Methods for Stochastic Chemical Systems
Efficient Leaping Methods for Stochastic Chemical Systems Ioana Cipcigan Muruhan Rathinam November 18, 28 Abstract. Well stirred chemical reaction systems which involve small numbers of molecules for some
More informationStochastic models of biochemical systems
Stochastic models of biochemical systems David F. Anderson anderson@math.wisc.edu Department of Mathematics University of Wisconsin - Madison University of Amsterdam November 14th, 212 Stochastic models
More informationMA 777: Topics in Mathematical Biology
MA 777: Topics in Mathematical Biology David Murrugarra Department of Mathematics, University of Kentucky http://www.math.uky.edu/~dmu228/ma777/ Spring 2018 David Murrugarra (University of Kentucky) Lecture
More information6 Continuous-Time Birth and Death Chains
6 Continuous-Time Birth and Death Chains Angela Peace Biomathematics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology.
More informationContinuous Time Markov Chains
Continuous Time Markov Chains Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 2015 Outline Introduction Continuous-Time Markov
More informationVarieties of Stochastic Calculi
Research is what I'm doing when I don't know what I'm doing. Wernher Von Braun. Artificial Biochemistry Varieties of Stochastic Calculi Microsoft Research Trento, 26-5-22..26 www.luca.demon.co.uk/artificialbiochemistry.htm
More informationExtending the Tools of Chemical Reaction Engineering to the Molecular Scale
Extending the Tools of Chemical Reaction Engineering to the Molecular Scale Multiple-time-scale order reduction for stochastic kinetics James B. Rawlings Department of Chemical and Biological Engineering
More informationMiscellaneous. Regarding reading materials. Again, ask questions (if you have) and ask them earlier
Miscellaneous Regarding reading materials Reading materials will be provided as needed If no assigned reading, it means I think the material from class is sufficient Should be enough for you to do your
More informationThe Expressivity of Universal Timed CCP: Undecidability of Monadic FLTL and Closure Operators for Security
The Expressivity of Universal Timed CCP: Undecidability of Monadic FLTL and Closure Operators for Security Carlos Olarte and Frank D. Valencia INRIA /CNRS and LIX, Ecole Polytechnique Motivation Concurrent
More informationFast Probability Generating Function Method for Stochastic Chemical Reaction Networks
MATCH Communications in Mathematical and in Computer Chemistry MATCH Commun. Math. Comput. Chem. 71 (2014) 57-69 ISSN 0340-6253 Fast Probability Generating Function Method for Stochastic Chemical Reaction
More informationReasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule
Reasoning Under Uncertainty: Conditioning, Bayes Rule & the Chain Rule Alan Mackworth UBC CS 322 Uncertainty 2 March 13, 2013 Textbook 6.1.3 Lecture Overview Recap: Probability & Possible World Semantics
More informationThis article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and
This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution
More information18.175: Lecture 30 Markov chains
18.175: Lecture 30 Markov chains Scott Sheffield MIT Outline Review what you know about finite state Markov chains Finite state ergodicity and stationarity More general setup Outline Review what you know
More informationSome investigations concerning the CTMC and the ODE model derived from Bio-PEPA
FBTC 2008 Some investigations concerning the CTMC and the ODE model derived from Bio-PEPA Federica Ciocchetta 1,a, Andrea Degasperi 2,b, Jane Hillston 3,a and Muffy Calder 4,b a Laboratory for Foundations
More informationReview: control, feedback, etc. Today s topic: state-space models of systems; linearization
Plan of the Lecture Review: control, feedback, etc Today s topic: state-space models of systems; linearization Goal: a general framework that encompasses all examples of interest Once we have mastered
More informationThis is now an algebraic equation that can be solved simply:
Simulation of CTMC Monday, November 23, 2015 1:55 PM Homework 4 will be posted by tomorrow morning, due Friday, December 11 at 5 PM. Let's solve the Kolmogorov forward equation for the Poisson counting
More informationPROBABILITY: LIMIT THEOREMS II, SPRING HOMEWORK PROBLEMS
PROBABILITY: LIMIT THEOREMS II, SPRING 218. HOMEWORK PROBLEMS PROF. YURI BAKHTIN Instructions. You are allowed to work on solutions in groups, but you are required to write up solutions on your own. Please
More informationIntroduction to stochastic multiscale modelling of tumour growth
Introduction to stochastic multiscale modelling of tumour growth Tomás Alarcón ICREA & Centre de Recerca Matemàtica T. Alarcón (ICREA & CRM, Barcelona, Spain) Lecture 1 CIMPA, Santiago de Cuba, June 2016
More informationEVOLUTIONARY DISTANCES
EVOLUTIONARY DISTANCES FROM STRINGS TO TREES Luca Bortolussi 1 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste luca@dmi.units.it Trieste, 14 th November 2007 OUTLINE 1 STRINGS:
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 05 Full points may be obtained for correct answers to eight questions Each numbered question (which may have several parts) is worth
More information/,!, W l *..il.'tn. INDEPENDENT AI.L THINUS. NKUTRAI. NOTHINfi" LOWELL, MICH 10AN,.FANUAKY, 5, WHOLE NO. 288.
/ / DPD KR Y 28 «> DR B Y B D D « B 9 B P 4 R P 6 D P B P 9 R B 26 D 4 P D K 3
More informationCOMP2610/COMP Information Theory
COMP2610/COMP6261 - Information Theory Lecture 9: Probabilistic Inequalities Mark Reid and Aditya Menon Research School of Computer Science The Australian National University August 19th, 2014 Mark Reid
More informationFirst Order Linear Ordinary Differential Equations
First Order Linear Ordinary Differential Equations The most general first order linear ODE is an equation of the form p t dy dt q t y t f t. 1 Herepqarecalledcoefficients f is referred to as the forcing
More informationP i [B k ] = lim. n=1 p(n) ii <. n=1. V i :=
2.7. Recurrence and transience Consider a Markov chain {X n : n N 0 } on state space E with transition matrix P. Definition 2.7.1. A state i E is called recurrent if P i [X n = i for infinitely many n]
More informationChemical reaction network theory for stochastic and deterministic models of biochemical reaction systems
Chemical reaction network theory for stochastic and deterministic models of biochemical reaction systems University of Wisconsin at Madison anderson@math.wisc.edu MBI Workshop for Young Researchers in
More informationThe First Derivative and Second Derivative Test
The First Derivative and Second Derivative Test James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University April 9, 2018 Outline 1 Extremal Values 2
More informationWinter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo
Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 9: Markov Chain Monte Carlo 9.1 Markov Chain A Markov Chain Monte
More informationBirth and Death Processes. Birth and Death Processes. Linear Growth with Immigration. Limiting Behaviour for Birth and Death Processes
DTU Informatics 247 Stochastic Processes 6, October 27 Today: Limiting behaviour of birth and death processes Birth and death processes with absorbing states Finite state continuous time Markov chains
More informationPopulation Genetics: a tutorial
: a tutorial Institute for Science and Technology Austria ThRaSh 2014 provides the basic mathematical foundation of evolutionary theory allows a better understanding of experiments allows the development
More informationGillespie s Algorithm and its Approximations. Des Higham Department of Mathematics and Statistics University of Strathclyde
Gillespie s Algorithm and its Approximations Des Higham Department of Mathematics and Statistics University of Strathclyde djh@maths.strath.ac.uk The Three Lectures 1 Gillespie s algorithm and its relation
More informationStochastic2010 Page 1
Stochastic2010 Page 1 Extinction Probability for Branching Processes Friday, April 02, 2010 2:03 PM Long-time properties for branching processes Clearly state 0 is an absorbing state, forming its own recurrent
More informationThe First Derivative and Second Derivative Test
The First Derivative and Second Derivative Test James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 8, 2017 Outline Extremal Values The
More informationCoBiC: Context-dependent Bioambient Calculus
QAPL 2009 CoBiC: Context-dependent Bioambient Calculus Luca Bortolussi,3 Dipartimento di Matematica ed Informatica University of Trieste, Trieste, Italy Maria G. Vigliotti 4,2 Department of Computing,
More informationSequence Modelling with Features: Linear-Chain Conditional Random Fields. COMP-599 Oct 6, 2015
Sequence Modelling with Features: Linear-Chain Conditional Random Fields COMP-599 Oct 6, 2015 Announcement A2 is out. Due Oct 20 at 1pm. 2 Outline Hidden Markov models: shortcomings Generative vs. discriminative
More informationQuiz 1. Name: Instructions: Closed book, notes, and no electronic devices.
Quiz 1. Name: Instructions: Closed book, notes, and no electronic devices. 1. What is the difference between a deterministic model and a probabilistic model? (Two or three sentences only). 2. What is the
More informationReaction time distributions in chemical kinetics: Oscillations and other weird behaviors
Introduction The algorithm Results Summary Reaction time distributions in chemical kinetics: Oscillations and other weird behaviors Ramon Xulvi-Brunet Escuela Politécnica Nacional Outline Introduction
More informationContinuum Limit of Forward Kolmogorov Equation Friday, March 06, :04 PM
Continuum Limit of Forward Kolmogorov Equation Friday, March 06, 2015 2:04 PM Please note that one of the equations (for ordinary Brownian motion) in Problem 1 was corrected on Wednesday night. And actually
More informationQuantitative Model Checking (QMC) - SS12
Quantitative Model Checking (QMC) - SS12 Lecture 06 David Spieler Saarland University, Germany June 4, 2012 1 / 34 Deciding Bisimulations 2 / 34 Partition Refinement Algorithm Notation: A partition P over
More informationSolving a system of Master Equations for parallel chemical interactions
Technical Report CoSBi 21/2007 Solving a system of Master Equations for parallel chemical interactions Paola Lecca The Microsoft Research - University of Trento Centre for Computational and Systems Biology
More informationStochastic Simulation of Biochemical Reactions
1 / 75 Stochastic Simulation of Biochemical Reactions Jorge Júlvez University of Zaragoza 2 / 75 Outline 1 Biochemical Kinetics 2 Reaction Rate Equation 3 Chemical Master Equation 4 Stochastic Simulation
More information2 Conceptual Framework Before introducing the probabilistic concurrent constraint (PCCP) language we have to discuss a basic question: What is a proba
On Probabilistic CCP Alessandra Di Pierro and Herbert Wiklicky fadp,herbertg@cs.city.ac.uk City University London, Northampton Square, London EC1V OHB Abstract This paper investigates a probabilistic version
More informationhttp://www.math.uah.edu/stat/markov/.xhtml 1 of 9 7/16/2009 7:20 AM Virtual Laboratories > 16. Markov Chains > 1 2 3 4 5 6 7 8 9 10 11 12 1. A Markov process is a random process in which the future is
More informationNote Set 5: Hidden Markov Models
Note Set 5: Hidden Markov Models Probabilistic Learning: Theory and Algorithms, CS 274A, Winter 2016 1 Hidden Markov Models (HMMs) 1.1 Introduction Consider observed data vectors x t that are d-dimensional
More informationCoBiC: Context-dependent Bioambient Calculus
Electronic Notes in Theoretical Computer Science 253 (2009) 87 20 www.elsevier.com/locate/entcs CoBiC: Context-dependent Bioambient Calculus Luca Bortolussi,3 Dipartimento di Matematica ed Informatica
More informationParallel Composition in Biology
Parallel Composition in Biology Nir Piterman Based on joint work with: J. Fisher, A. Hajnal, B. Hall, T. A. Henzinger, M. Mateescu, D. Nickovic, A.V. Singh, and M.Y. Vardi Are there useful macros to structure
More informationM4A42 APPLIED STOCHASTIC PROCESSES
M4A42 APPLIED STOCHASTIC PROCESSES G.A. Pavliotis Department of Mathematics Imperial College London, UK LECTURE 1 12/10/2009 Lectures: Mondays 09:00-11:00, Huxley 139, Tuesdays 09:00-10:00, Huxley 144.
More informationReinforcement Learning
Reinforcement Learning Policy gradients Daniel Hennes 26.06.2017 University Stuttgart - IPVS - Machine Learning & Robotics 1 Policy based reinforcement learning So far we approximated the action value
More informationEulerian (Probability-Based) Approach
Eulerian (Probability-Based) Approach Tuesday, March 03, 2015 1:59 PM Office hours for Wednesday, March 4 shifted to 5:30-6:30 PM. Homework 2 posted, due Tuesday, March 17 at 2 PM. correction: the drifts
More informationMarkov Decision Processes and Dynamic Programming
Markov Decision Processes and Dynamic Programming A. LAZARIC (SequeL Team @INRIA-Lille) Ecole Centrale - Option DAD SequeL INRIA Lille EC-RL Course In This Lecture A. LAZARIC Markov Decision Processes
More informationLet's transfer our results for conditional probability for events into conditional probabilities for random variables.
Kolmogorov/Smoluchowski equation approach to Brownian motion Tuesday, February 12, 2013 1:53 PM Readings: Gardiner, Secs. 1.2, 3.8.1, 3.8.2 Einstein Homework 1 due February 22. Conditional probability
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationOpen Problem: Approximate Planning of POMDPs in the class of Memoryless Policies
Open Problem: Approximate Planning of POMDPs in the class of Memoryless Policies Kamyar Azizzadenesheli U.C. Irvine Joint work with Prof. Anima Anandkumar and Dr. Alessandro Lazaric. Motivation +1 Agent-Environment
More informationHandbook of Stochastic Methods
C. W. Gardiner Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences Third Edition With 30 Figures Springer Contents 1. A Historical Introduction 1 1.1 Motivation I 1.2 Some Historical
More informationFrequency Spectra and Inference in Population Genetics
Frequency Spectra and Inference in Population Genetics Although coalescent models have come to play a central role in population genetics, there are some situations where genealogies may not lead to efficient
More informationModelling in Systems Biology
Modelling in Systems Biology Maria Grazia Vigliotti thanks to my students Anton Stefanek, Ahmed Guecioueur Imperial College Formal representation of chemical reactions precise qualitative and quantitative
More informationModeling biological systems with delays in Bio-PEPA
Modeling biological systems with delays in Bio-PEPA Giulio Caravagna Dipartimento di Informatica, Università di Pisa, argo Pontecorvo 3, 56127 Pisa, Italy. caravagn@di.unipi.it Jane Hillston aboratory
More informationStochastic contraction BACS Workshop Chamonix, January 14, 2008
Stochastic contraction BACS Workshop Chamonix, January 14, 2008 Q.-C. Pham N. Tabareau J.-J. Slotine Q.-C. Pham, N. Tabareau, J.-J. Slotine () Stochastic contraction 1 / 19 Why stochastic contraction?
More informationFluid Limits of Queueing Networks with Batches
Fluid Limits of Queueing etworks with Batches Luca Bortolussi Department of Mathematics and Computer Science University of Trieste, Italy luca@dmi.units.it Mirco Tribastone Institute of Informatics Ludwig-Maximilians-Universität
More informationDynamical Systems and Deep Learning: Overview. Abbas Edalat
Dynamical Systems and Deep Learning: Overview Abbas Edalat Dynamical Systems The notion of a dynamical system includes the following: A phase or state space, which may be continuous, e.g. the real line,
More informationStochastic modelling of epidemic spread
Stochastic modelling of epidemic spread Julien Arino Centre for Research on Inner City Health St Michael s Hospital Toronto On leave from Department of Mathematics University of Manitoba Julien Arino@umanitoba.ca
More informationOutline. CSE 573: Artificial Intelligence Autumn Agent. Partial Observability. Markov Decision Process (MDP) 10/31/2012
CSE 573: Artificial Intelligence Autumn 2012 Reasoning about Uncertainty & Hidden Markov Models Daniel Weld Many slides adapted from Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer 1 Outline
More informationPublications. Refereed Journal Publications
Publications Refereed Journal Publications [A1] [A2] [A3] [A4] [A5] [A6] [A7] [A8] [A9] C. Baier, J.-P. Katoen, H. Hermanns, and V. Wolf. Comparative branching-time semantics for Markov chains. In: Information
More informationLecture 5. If we interpret the index n 0 as time, then a Markov chain simply requires that the future depends only on the present and not on the past.
1 Markov chain: definition Lecture 5 Definition 1.1 Markov chain] A sequence of random variables (X n ) n 0 taking values in a measurable state space (S, S) is called a (discrete time) Markov chain, if
More informationHigh-dimensional Problems in Finance and Economics. Thomas M. Mertens
High-dimensional Problems in Finance and Economics Thomas M. Mertens NYU Stern Risk Economics Lab April 17, 2012 1 / 78 Motivation Many problems in finance and economics are high dimensional. Dynamic Optimization:
More informationPattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods
Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs
More informationSwitching Regime Estimation
Switching Regime Estimation Series de Tiempo BIrkbeck March 2013 Martin Sola (FE) Markov Switching models 01/13 1 / 52 The economy (the time series) often behaves very different in periods such as booms
More informationBrownian Motion. An Undergraduate Introduction to Financial Mathematics. J. Robert Buchanan. J. Robert Buchanan Brownian Motion
Brownian Motion An Undergraduate Introduction to Financial Mathematics J. Robert Buchanan 2010 Background We have already seen that the limiting behavior of a discrete random walk yields a derivation of
More informationPopulation models from PEPA descriptions
Population models from PEPA descriptions Jane Hillston LFCS, The University of Edinburgh, Edinburgh EH9 3JZ, Scotland. Email: jeh@inf.ed.ac.uk 1 Introduction Stochastic process algebras (e.g. PEPA [10],
More informationMath 250B Final Exam Review Session Spring 2015 SOLUTIONS
Math 5B Final Exam Review Session Spring 5 SOLUTIONS Problem Solve x x + y + 54te 3t and y x + 4y + 9e 3t λ SOLUTION: We have det(a λi) if and only if if and 4 λ only if λ 3λ This means that the eigenvalues
More informationModelling protein trafficking: progress and challenges
Modelling protein trafficking: progress and challenges Laboratory for Foundations of Computer Science School of Informatics University of Edinburgh 21 May 2012 Outline Protein Trafficking Modelling Biology
More informationChapter 3 - Temporal processes
STK4150 - Intro 1 Chapter 3 - Temporal processes Odd Kolbjørnsen and Geir Storvik January 23 2017 STK4150 - Intro 2 Temporal processes Data collected over time Past, present, future, change Temporal aspect
More informationIntroduction to Artificial Intelligence. Logical Agents
Introduction to Artificial Intelligence Logical Agents (Logic, Deduction, Knowledge Representation) Bernhard Beckert UNIVERSITÄT KOBLENZ-LANDAU Winter Term 2004/2005 B. Beckert: KI für IM p.1 Outline Knowledge-based
More informationSynchronization Transitions in Complex Networks
Synchronization Transitions in Complex Networks Y. Moreno 1,2,3 1 Institute for Biocomputation and Physics of Complex Systems (BIFI) University of Zaragoza, Zaragoza 50018, Spain 2 Department of Theoretical
More informationLogical Agents. Knowledge based agents. Knowledge based agents. Knowledge based agents. The Wumpus World. Knowledge Bases 10/20/14
0/0/4 Knowledge based agents Logical Agents Agents need to be able to: Store information about their environment Update and reason about that information Russell and Norvig, chapter 7 Knowledge based agents
More informationAN INTRODUCTION TO DISCRETE-EVENT SIMULATION
AN INTRODUCTION TO DISCRETE-EVENT SIMULATION Peter W. Glynn 1 Peter J. Haas 2 1 Dept. of Management Science and Engineering Stanford University 2 IBM Almaden Research Center San Jose, CA CAVEAT: WE ARE
More informationM. Dechesne R. Sepulchre
Systems and models in chronobiology A delay model for the circadian rhythm M. Dechesne R. Sepulchre Department of Electrical Engineering and Computer Science Monteore Institute University of Liège 24th
More information18.600: Lecture 32 Markov Chains
18.600: Lecture 32 Markov Chains Scott Sheffield MIT Outline Markov chains Examples Ergodicity and stationarity Outline Markov chains Examples Ergodicity and stationarity Markov chains Consider a sequence
More informationLIMITING PROBABILITY TRANSITION MATRIX OF A CONDENSED FIBONACCI TREE
International Journal of Applied Mathematics Volume 31 No. 18, 41-49 ISSN: 1311-178 (printed version); ISSN: 1314-86 (on-line version) doi: http://dx.doi.org/1.173/ijam.v31i.6 LIMITING PROBABILITY TRANSITION
More informationCOMS 4771 Probabilistic Reasoning via Graphical Models. Nakul Verma
COMS 4771 Probabilistic Reasoning via Graphical Models Nakul Verma Last time Dimensionality Reduction Linear vs non-linear Dimensionality Reduction Principal Component Analysis (PCA) Non-linear methods
More informationReasoning under Uncertainty: Intro to Probability
Reasoning under Uncertainty: Intro to Probability Computer Science cpsc322, Lecture 24 (Textbook Chpt 6.1, 6.1.1) March, 15, 2010 CPSC 322, Lecture 24 Slide 1 To complete your Learning about Logics Review
More informationThe Markov Decision Process (MDP) model
Decision Making in Robots and Autonomous Agents The Markov Decision Process (MDP) model Subramanian Ramamoorthy School of Informatics 25 January, 2013 In the MAB Model We were in a single casino and the
More informationLogic-Based Modeling in Systems Biology
Logic-Based Modeling in Systems Biology Alexander Bockmayr LPNMR 09, Potsdam, 16 September 2009 DFG Research Center Matheon Mathematics for key technologies Outline A.Bockmayr, FU Berlin/Matheon 2 I. Systems
More information