STOCHASTIC PROCESSES FOR PHYSICISTS Understanding Noisy Systems Stochastic processes are an essential part of numerous branches of physics, as well as biology, chemistry, and finance. This textbook provides a solid understanding of stochastic processes and stochastic calculus in physics, without the need for measure theory. In avoiding measure theory, this textbook gives readers the tools necessary to use stochastic methods in research with a minimum of mathematical background. Coverage of the more exotic Levy processes is included, as is a concise account of numerical methods for simulating stochastic systems driven by Gaussian noise. The book concludes with a non-technical introduction to the concepts and jargon of measure-theoretic probability theory. With over 70 exercises, this textbook is an easily accessible introduction to stochastic processes and their applications, as well as methods for numerical simulation, for graduate students and researchers in physics. is an Assistant Professor in the Physics Department at the University of Massachusetts, Boston. He is a leading expert in the theory of quantum feedback control and the measurement and control of quantum nano-electro-mechanical systems.
STOCHASTIC PROCESSES FOR PHYSICISTS Understanding Noisy Systems KURT JACOBS University of Massachusetts
cambridge university press Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo, Delhi, Dubai, Tokyo Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK Published in the United States of America by Cambridge University Press, New York Information on this title: /9780521765428 C K. Jacobs 2010 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2010 Printed in the United Kingdom at the University Press, Cambridge A catalogue record for this publication is available from the British Library ISBN 978-0-521-76542-8 Hardback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.
To Salman Habib and Bala Sundaram, for pointing the way.
Contents Preface page xi Acknowledgments xiii 1 A review of probability theory 1 1.1 Random variables and mutually exclusive events 1 1.2 Independence 4 1.3 Dependent random variables 5 1.4 Correlations and correlation coefficients 6 1.5 Adding random variables together 7 1.6 Transformations of a random variable 8 1.7 The distribution function 10 1.8 The characteristic function 10 1.9 Moments and cumulants 12 1.10 The multivariate Gaussian 13 2 Differential equations 16 2.1 Introduction 16 2.2 Vector differential equations 17 2.3 Writing differential equations using differentials 18 2.4 Two methods for solving differential equations 18 2.4.1 A linear differential equation with driving 20 2.5 Solving vector linear differential equations 21 2.6 Diagonalizing a matrix 23 3 Stochastic equations with Gaussian noise 26 3.1 Introduction 26 3.2 Gaussian increments and the continuum limit 28 3.3 Interlude: why Gaussian noise? 31 3.4 Ito calculus 32 3.5 Ito s formula: changing variables in an SDE 35 vii
viii Contents 3.6 Solving some stochastic equations 37 3.6.1 The Ornstein Uhlenbeck process 37 3.6.2 The full linear stochastic equation 39 3.6.3 Ito stochastic integrals 40 3.7 Deriving equations for the means and variances 41 3.8 Multiple variables and multiple noise sources 42 3.8.1 Stochastic equations with multiple noise sources 42 3.8.2 Ito s formula for multiple variables 44 3.8.3 Multiple Ito stochastic integrals 45 3.8.4 The multivariate linear equation with additive noise 48 3.8.5 The full multivariate linear stochastic equation 48 3.9 Non-anticipating functions 51 4 Further properties of stochastic processes 55 4.1 Sample paths 55 4.2 The reflection principle and the first-passage time 57 4.3 The stationary auto-correlation function, g(τ) 59 4.4 Conditional probability densities 60 4.5 The power spectrum 61 4.5.1 Signals with finite energy 63 4.5.2 Signals with finite power 65 4.6 White noise 66 5 Some applications of Gaussian noise 71 5.1 Physics: Brownian motion 71 5.2 Finance: option pricing 74 5.2.1 Some preliminary concepts 75 5.2.2 Deriving the Black Scholes equation 78 5.2.3 Creating a portfolio that is equivalent to an option 81 5.2.4 The price of a European option 82 5.3 Modeling multiplicative noise in real systems: Stratonovich integrals 85 6 Numerical methods for Gaussian noise 91 6.1 Euler s method 91 6.1.1 Generating Gaussian random variables 92 6.2 Checking the accuracy of a solution 92 6.3 The accuracy of a numerical method 94 6.4 Milstien s method 95 6.4.1 Vector equations with scalar noise 95 6.4.2 Vector equations with commutative noise 96 6.4.3 General vector equations 97 6.5 Runge Kutter-like methods 98
Contents ix 6.6 Implicit methods 99 6.7 Weak solutions 99 6.7.1 Second-order weak methods 100 7 Fokker Planck equations and reaction diffusion systems 102 7.1 Deriving the Fokker Planck equation 102 7.2 The probability current 104 7.3 Absorbing and reflecting boundaries 105 7.4 Stationary solutions for one dimension 106 7.5 Physics: thermalization of a single particle 107 7.6 Time-dependent solutions 109 7.6.1 Green s functions 110 7.7 Calculating first-passage times 111 7.7.1 The time to exit an interval 111 7.7.2 The time to exit through one end of an interval 113 7.8 Chemistry: reaction diffusion equations 116 7.9 Chemistry: pattern formation in reaction diffusion systems 119 8 Jump processes 127 8.1 The Poisson process 127 8.2 Stochastic equations for jump processes 130 8.3 The master equation 131 8.4 Moments and the generating function 133 8.5 Another simple jump process: telegraph noise 134 8.6 Solving the master equation: a more complex example 136 8.7 The general form of the master equation 139 8.8 Biology: predator prey systems 140 8.9 Biology: neurons and stochastic resonance 144 9 Levy processes 151 9.1 Introduction 151 9.2 The stable Levy processes 152 9.2.1 Stochastic equations with the stable processes 156 9.2.2 Numerical simulation 157 9.3 Characterizing all the Levy processes 159 9.4 Stochastic calculus for Levy processes 162 9.4.1 The linear stochastic equation with a Levy process 163 10 Modern probability theory 166 10.1 Introduction 166 10.2 The set of all samples 167 10.3 The collection of all events 167 10.4 The collection of events forms a sigma-algebra 167 10.5 The probability measure 169
x Contents 10.6 Collecting the concepts: random variables 171 10.7 Stochastic processes: filtrations and adapted processes 174 10.7.1 Martingales 175 10.8 Translating the modern language 176 Appendix A Calculating Gaussian integrals 181 References 184 Index 186
Preface This book is intended for a one-semester graduate course on stochastic methods. It is specifically targeted at students and researchers who wish to understand and apply stochastic methods to problems in the natural sciences, and to do so without learning the technical details of measure theory. For those who want to familiarize themselves with the concepts and jargon of the modern measure-theoretic formulation of probability theory, these are described in the final chapter. The purpose of this final chapter is to provide the interested reader with the jargon necessary to read research articles that use the modern formalism. This can be useful even if one does not require this formalism in one s own research. This book contains more material than I cover in my current graduate class on the subject at UMass Boston. One can select from the text various optional paths depending on the purpose of the class. For a graduate class for physics students who will be using stochastic methods in their research work, whether in physics or interdisciplinary applications, I would suggest the following: Chapters 1, 2, 3 (with Section 3.8.5 optional), 4 (with Section 4.2 optional, as alternative methods are given in 7.7), 5 (with Section 5.2 optional), 7 (with Sections 7.8 and 7.9 optional), and 8 (with Section 8.9 optional). In the above outline I have left out Chapters 6, 9 and 10. Chapter 6 covers numerical methods for solving equations with Gaussian noise, and is the sort of thing that can be picked-up at a later point by a student if needed for research. Chapter 9 covers Levy stochastic processes, including exotic noise processes that generate probability densities with infinite variance. While this chapter is no more difficult than the preceding chapters, it is a more specialized subject in the sense that relatively few students are likely to need it in their research work. Chapter 10, as mentioned above, covers the concepts and jargon of the rigorous measure-theoretic formulation of probability theory. A brief overview of this book is as follows: Chapters 1 (probability theory without measure theory) and 2 (ordinary differential equations) give background material that is essential for understanding the rest of course. Chapter 2 will be almost xi
xii Preface all revision for students with an undergraduate physics degree. Chapter 3 covers all the basics of Ito calculus and solving stochastic differential equations. Chapter 4 introduces some further concepts such as auto-correlation functions, power spectra and white noise. Chapter 5 contains two applications (Brownian motion and option pricing), as well as a discussion of the Stratonovich formulation of stochastic equations and its role in modeling multiplicative noise. Chapter 6 covers numerical methods for solving stochastic equations. Chapter 7 covers Fokker Planck equations. This chapter also includes applications to reaction diffusion systems, and pattern formation in these systems. Chapter 8 explains jump processes and how they are described using master equations. It also contains applications to population dynamics and neuron behavior. Chapter 9 covers Levy processes. These include noise processes that generate probability densities with infinite variance, such as the Cauchy distribution. Finally Chapter 10 introduces the concepts and jargon of the modern measure-theoretic description of probability theory. While I have corrected many errors that found their way into the manuscript, it is unlikely that I eliminated them all. For the purposes of future editions I would certainly be grateful if you can let me know of any errors you find.
Acknowledgments Thanks to... Aric Hagberg for providing me with a beautiful plot of labyrinthine pattern formation in reaction diffusion systems (see Chapter 7) and Jason Ralph for bringing to my attention Edwin Jaynes discussion of mathematical style (see Chapter 10). I am also grateful to my students for being patient and helping me iron-out errors and omissions in the text. xiii