Modélisation probabiliste en biologie moléculaire et cellulaire

Transcription

1 Modélisation probabiliste en biologie moléculaire et cellulaire Romain Yvinec To cite this version: Romain Yvinec. Modélisation probabiliste en biologie moléculaire et cellulaire. Probabilités [math.pr]. Université Claude Bernard - Lyon I, 212. Français. <tel > HAL Id: tel Submitted on 7 Nov 212 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Université Claude Bernard Lyon 1 Thèse de doctorat Institut Camille Jordan N o d ordre : Laboratoire des Mathématiques UMR 528 CNRS-UCBL Modélisation probabiliste en biologie cellulaire et moléculaire Thèse de doctorat Spécialité Mathématiques présentée par Romain YVINEC sous la direction de Mostafa ADIMY, Michael C. MACKEY & Laurent PUJO-MENJOUET Soutenue publiquement le 5 octobre 212 Devant le jury composé de : Examinatrice Mostafa ADIMY Directeur de Recherches à l INRIA Dir. de thèse Ionel S. CIUPERCA Maître de Conférence à l Université Lyon 1 Examinateur Michael C. MACKEY Directeur de Recherche à l Université Mc GIll Dir. de thèse Sylvie MÉLÉARD Professeur à l Ecole Polytechnique Examinatrice Sophie MERCIER Professeur à l Université de Pau et des Pays de l Adour Rapportrice Laurent PUJO-MENJOUET Maître de Conférence à l Université Lyon 1 Dir. de thèse Marta TYRAN-KAMIŃSKA Professeur à l University of Silesia Bernard YCART Professeur à l Université de Grenoble Rapporteur Ecole Doctorale Informatique et Mathématiques - EDA 512

3

4 3 Résumé De nombreux travaux récents ont démontré l importance de la stochasticité dans l expression des gènes à différentes échelles. On passera tout d abord en revue les principaux résultats expérimentaux pour motiver l étude de modèles mathématiques prenant en compte des effets aléatoires. On étudiera ensuite deux modèles particuliers où les effets aléatoires induisent des comportements intéressants, en lien avec des résultats expérimentaux : une dynamique intermittente dans un modèle d auto-régulation de l expression d un gène; et l émergence d hétérogénéité à partir d une population homogène de protéines par modification post-traductionnelle. Dans le Chapitre I, nous avons étudié le modèle standard d expression des gènes à trois variables : ADN, ARN messager et protéine. L ADN peut être dans deux états, respectivement ON et OFF. La transcription (production d ARN messagers) peut avoir lieu uniquement dans l état ON. La traduction (production de protéines) est proportionnelle à la quantité d ARN messager. Enfin la quantité de protéines peut réguler de manière non-linéaire les taux de production précédent. Nous avons utilisé des théorèmes de convergence de processus stochastique pour mettre en évidence différents régimes de ce modèle. Nous avons ainsi prouvé rigoureusement le phénomène de production intermittente d ARN messagers et/ou de protéines. Les modèles limites obtenues sont alors des modèles hybrides, déterministes par morceaux avec sauts Markoviens. Nous avons étudié le comportement en temps long de ces modèles et prouvé la convergence vers des solutions stationnaires. Enfin, nous avons étudié en détail un modèle réduit, calculé explicitement la solution stationnaire, et étudié le diagramme de bifurcation des densités stationnaires. Ceci a permis 1) de mettre en évidence l influence de la stochasticité en comparant aux modèles déterministes; 2) de donner en retour un moyen théorique d estimer la fonction de régulation par un problème inverse. Dans le Chapitre II, nous avons étudié une version probabiliste du modèle d agrégationfragmentation. Cette version permet une définition de la nucléation en accord avec les modèles biologistes pour les maladies à Prion. Pour étudier la nucléation, nous avons utilisé une version stochastique du modèle de Becker-Döring. Dans ce modèle, l agrégation est réversible et se fait uniquement par attachement/détachement d un monomère. Le temps de nucléation est définit comme le premier temps où un noyau (c est-à-dire un agrégat de taille fixé, cette taille est un paramètre du modèle) est formé. Nous avons alors caractérisé la loi du temps de nucléation dans ce modèle. La distribution de probabilité du temps de nucléation peut prendre différente forme selon les valeurs de paramètres : exponentielle, bimodale, ou de type Weibull. Concernant le temps moyen de nucléation, nous avons mis en évidence deux phénomènes importants. D une part, le temps moyen de nucléation est une fonction non-monotone du paramètre cinétique d agrégation. D autre part, selon la valeur des autres paramètres, le temps moyen de nucléation peut dépendre fortement ou très faiblement de la quantité initiale de monomère. Ces caractérisations sont importantes pour 1) expliquer des dépendances très faible en les conditions initiales, observées expérimentalement; 2) déduire la valeur de certains paramètres d observations expérimentales. Cette étude peut donc être appliqué à des données biologiques. Enfin, concernant un modèle de polymérisation-fragmentation, nous avons montré un théorème limite d un modèle purement discret vers un modèle hybride, qui peut-être plus utile pour des simulations numériques, ainsi que pour une étude théorique.

5 4 Summary The importance of stochasticity in gene expression has been widely shown recently. We will first review the most important related work to motivate mathematical models that takes into account stochastic effects. Then, we will study two particular models where stochasticity induce interesting behavior, in accordance with experimental results : a bursting dynamic in a self-regulating gene expression model; and the emergence of heterogeneity from a homogeneous pool of protein by post-translational modification. In Chapter I, we studied a standard gene expression model, at three variables : DNA, messenger RNA and protein. DNA can be in two distinct states, ON and OFF. Transcription (production of mrna) can occur uniquely in the ON state. Translation (production of protein) is proportional to the quantity of mrna. Then, the quantity of protein can regulate in a non-linear fashion these production rates. We used convergence theorem of stochastic processes to highlight different behavior of this model. Hence, we rigorously proved the bursting phenomena of mrna and/or protein. Limiting models are then hybrid model, piecewise deterministic with Markovian jumps. We studied the long time behavior of these models and proved convergence toward a stationary state. Finally, we studied in detail a reduced model, explicitly calculated the stationary distribution and studied its bifurcation diagram. Our two main results are 1) to highlight stochastic effects by comparison with deterministic model; 2) To give back a theoretical tool to estimate non-linear regulation function through an inverse problem. In Chapter II, we studied a probabilistic version of an aggregation-fragmentation model. This version allows a definition of nucleation in agreement with biological model for Prion disease. To study the nucleation, we used a stochastic version of the Becker-Döring model. In this model, aggregation is reversible and through attachment/detachment of a monomer. The nucleation time is defined as a waiting time for a nuclei (aggregate of a fixed size, this size being a parameter of the model) to be formed. In this work, we characterized the law of the nucleation time. The probability distribution of the nucleation time can take various forms according parameter values : exponential, bimodal or Weibull. We also highlight two important phenomena for the mean nucleation time. Firstly, the mean nucleation time is a non-monotone function of the aggregation kinetic parameter. Secondly, depending of parameter values, the mean nucleation time can be strongly or very weakly correlated with the initial quantity of monomer. These characterizations are important for 1) explaining weak dependence in initial condition observed experimentally; 2) deducing some parameter values from experimental observations. Hence, this study can be directly applied to biological data. Finally, concerning a polymerization-fragmentation model, we proved a convergence theorem of a purely discrete model to hybrid model, which may be useful for numerical simulations as well as a theoretical study.

6 5 Remerciements Mes premiers remerciements vont bien sûr à mes directeurs de thèse. Tout d abord merci à Michael Mackey, qui m a initié au domaine de la recherche. Mes 3 séjours à Montréal ont été une réussite, en grande partie grâce à lui. Je remercie ensuite Laurent Pujo-Menjouet, qui a su relever le défi d un encadrement en co-direction, et qui m a ouvert de nombreuses directions de recherche. Enfin, merci à Mostafa Adimy pour la confiance qu il m a accordé et pour l encadrement de toute une équipe de recherche. L occasion pour moi de souligner l environnement inter-disciplinaire fructueux des équipes Dracula et Beagle, dont je remercie chaleureusement tous les membres. Je suis reconnaissant envers Bernard Ycart et Sophie Mercier, qui ont la patience de relire ma thèse, et qui m ont beaucoup apporté par leurs retours. Je souhaite aussi remercier Sylvie Méléard, Marta Tyran-Kaminska et Ionel Sorin Ciuperca pour avoir accepté et pris le temps de faire parti de mon Jury. C est pour moi un grand honneur. Merci également aux personnes de mon entourage qui ont pris le temps de relire (des bouts!) de ma thèse : Adriane, Julien, Erwan, Marianne et Cécile. Durant mes 3 années de thèse, j ai eu la chance de rencontrer et travailler avec de nombreuses personnes, et j aimerais les remercier ici. A Lyon, je pense notamment à Jean Bérard, Thomas Lepoutre, Olivier Gandrillon, et François Morlé. Si notre travail n a pas encore porté ses fruits, cette collaboration a été très enrichissante. Je remercie également Vincent Calvez, avec qui il est toujours un plaisir de jouer au foot comme de parler de maths, et Erwan Hingant, dont je garderai un souvenir impérissable des séances de travail. À Montréal, je suis très heureux d avoir croisé les chemins de Lennart Hilbert, Thomas Quail, Bart Borek, Guillaume Attuel, Shahed Riaz, Vahid Shahrezaei, et Changjing Zhuge. Beaucoup de pistes stimulantes ont émergé de nos nombreuses discussions et leur camaraderie m a été plus que bénéfique! Au gré des conférences à travers le monde, j ai eu le plaisir de rencontrer Alex Ramos (Sao Paulo), Tom Chou et Maria Rita D Orsogna (Los Angeles), Marta Tyran-Kaminska (Katowice), Mario Pineda-Krch (Edmonton) et de travailler avec Jinzhi Lei (Beijing)...Toutes ces personnes ont grandement contribué à l avancé de mes travaux, et à me donner l envie de poursuivre sur cette lancée. C est avec une grande motivation que je souhaite continuer à collaborer avec ces personnes. Parce que l organisation de la science est au moins aussi importante que la science elle-même, je suis content d avoir pu aborder des thèmes politiques et philosophiques avec Pierre Crépel, Nicolas Lechopier, Hervé Philippe et le MQDC... Ces 3 années de globe-trotter ont également été riche sur le plan personnel, et la fin de la thèse va de pair avec la fin d une page de ma vie. J aimerais donc remercier spécialement toutes les personnes que j ai pu côtoyer ici ou là. En premier lieu, les colocs! Elles/Ils ont su faire que l adaptation après chaque voyage se passe en douceur, et ont égayé ces 3 années. La palme pour la coloc de Mermoz, sans qui la 3e année aurait été un calvaire! Un grand merci et vive la convivialité de la colocation! Ensuite les amies, bien sûr, matheusesx ou non-matheusesx gratteux ou footeux, déboulonneuses ou déboulonneurs, cyclistes ou vélorutionnaires, dont faire la liste exhaustive me paraît risqué...un grand merci à mon ami d enfance Mathieu pour avoir suivi mon parcours avec beaucoup d intérêt; à Simon (courage pour la rédaction!); à Pierre, Pierre-Adelin, Michael, Anne, Sandrine, Anne-Sandrine, Aline, Xavier, Vincent, Laetitia que j ai toujours autant de plaisir à revoir; à Delphine et Romain, toujours enclin à se faire une petite partie; à Rémi, Catherine, Antoine, Aude, Laetitia avec qui on se sent si bien; à Julien, Erwan, Thomas, Adriane, Marianne, Mohammed, JB, Amélie, Mickaël, Alain, pour tous les moments de détente au labo (et en dehors...); Kiki, Doudou et Carole pour la poutine ou le meilleur...et à toutes cellesux que j ai oubliées! Merci la famille, toujours présente à mes côtés. Je vais pouvoir jouer davantage au tonton! Enfin un petit mot spécial pour Cécile. Merci pour tes sacrifices, merci de m avoir suivi à travers le monde, maintenant je pars sur les routes avec toi!

7 6

8 Table des matières Introduction Générale 9 1 Biologie, Rappels Historiques Modélisation Mathématique Résultats de Cette Thèse Perspectives Notations Étude Théorique de Modèles Stochastiques Chaîne de Markov à temps discret Chaîne de Markov à temps continu Processus de Markov Processus de Markov déterministes par morceaux Équation d évolution d un PDMP Théorèmes Limites Réduction de modèles par séparation d échelles de temps Réduction par passage en grande population Hybrid Models to Explain Gene Expression Variability 49 1 Introduction Standard Model Background in molecular biology The operon concept Synthetic network Prokaryotes vs Eukaryotes models The Rate Functions Transcriptional rate in inducible regulation Transcriptional rate in repressible regulation Summary Other rate functions Parameters and Time Scales Discrete Version Representation of the discrete model Long time behavior Continuous Version - Deterministic Operon Dynamics No control (single attractive steady-state) Inducible regulation (single versus multiple steady states) Repressible regulation (single steady-state versus oscillations) Bursting and Hybrid Models, a Review of Linked Models Discrete models with switch Continuous models with switch Discrete models without switch

9 8 TABLE DES MATIÈRES 7.4 Continuous models without switch Discrete models with Bursting Continuous models with Bursting Models with both switching and Bursting Hybrid discrete and continuous models More detailed models and other approaches Specific Study of the One-Dimensional Bursting Model Discrete variable model with bursting BD Continuous variable model with bursting BC Fluctuations in the degradation rate only Discussion Ergodicity and explicit convergence rate Inverse problem From One Model to Another Limiting behavior of the switching model A bursting model from a two-dimensional discrete model Adiabatic reduction in a bursting model From discrete to continuous bursting model Hybrid Models to Explain Protein Aggregation Variability Introduction Biological background: what is the prion? The Lansbury s nucleation/polymerization theory Experimental observations available Observed Dynamics Literature review Outline Formulation of the Model Dynamical models of nucleation-polymerization Misfolding process and time scale reduction First Assembly Time in a Discrete Becker-Döring model Introduction Formulation of the model Example and particular case Constant monomer formulation Irreversible limit (q ) Slow detachment limit ( q 1) Fast detachement limit (q ) - Cycle approximation Fast detachment limit (q ) - Queueing approximations Large initial monomer quantity Numerical results and analysis Application to prion Polymer Under Flow, From Discrete to Continuous Models Introduction An individual and discrete length approach Some necessary comments on the model The measure-valued stochastic process Scaling equations and the limit problem Convergence theorem

10 Chapitre Introduction Générale 9

11 1 Introduction Générale 1 Biologie, Rappels Historiques La découverte de phénomènes aléatoires en biologie est relativement récente, contrairement à d autres domaines comme la physique ou la chimie. En biologie moléculaire plus particulièrement, une vision déterministe (proche du «déterminisme Laplacien») prévalait il y a encore quelques années. En témoigne par exemple l influent livre d Erwin Schrödinger, What is Life ([75], 1944), (voir aussi [8]) pour qui l ordre macroscopique d un organisme vivant provient d un même ordre microscopique de ses constituants. Durant un demi-siècle, ces idées ont été dominantes en biologie. Cette vision déterministe en fait un domaine distinct de la physique, où la notion d ordre à partir du désordre est connue depuis longtemps (notamment grâce à Ludwig Boltzmann, James Clerk Maxwell, et la théorie cinétique des gaz, dans la deuxième moitié du 19e siècle, et plus généralement par les approches de la physique statistique). Il faut bien voir que les ordres de grandeur sont aussi radicalement différents. Dans un volume de gaz macroscopique une mole, il y a de l ordre de molécules (nombre d Avogadro). Si le nombre de cellules dans l organisme humain est estimé à environ 1 14, certaines entités biochimiques ne sont présentes que par centaines voir dizaines de copies dans une cellule! Depuis la découverte de l ADN et de son information génétique par James Watson, Maurice Wilkins et Francis Crick (1962) et depuis les travaux de Jacques Monod, François Jacob et André Lwoff (1965) sur l ARN messager et la notion d opérons, la vision dominante en biologie moléculaire est une vision mécaniste (voir par exemple [49]). Toute l information dans un organisme est contenue dans les gènes, qui la transmettent via une série (complexe) de réactions biochimiques à certaines protéines, qui vont à leur tour donner desfonctions auxcellules. Cette vision est àlabasedecequ onappelle la«cybernétique», théorie initiée par Norbert Wiener (voir par exemple [46]). Les récents progrès spectaculaires des méthodes et technologies expérimentales ont accumulé les preuves que la perception mécaniste des phénomènes biologiques ne s accorde plus aux observations expérimentales. Parmi les récentes technologies disponibles, on peut citer la PCR (réaction en chaîne par polymérase Polymerase Chain Reaction qui permet notamment de multiplier des fragments d ADN pour les étudier), les puces d ADN (qui permettent de mesurer les niveaux d expression d un grand nombre de gènes simultanément), les nombreuses techniques d observation et de détection de molécules dans une cellule (voir par exemple [7]), ainsi que de leur dynamique et structure spatiales (via notamment la spectroscopie de résonance magnétique nucléaire, voir par exemple [13]). Ces technologies ont, entre autres, permis d étudier les séquences de gènes (avec par exemple le Human Genome Project (1) ), les niveaux d expression des gènes et les interactions entre protéines. Parmi les expériences marquantes qui donnent de moins en moins d importance à l entité «gène» et de plus en plus aux interactions avec l environnement (intérieur et extérieur à la cellule), on peut citer l expérience d Elowitz et al. [28]. Ces auteurs observent l expression de deux gènes «identiques», situés à des endroits similaires dans le génome d une bactérie (en fait, l ADN d une bactérie étant circulaire, ils ont placé les deux gènes de manière symétrique par rapport à l origine de réplication). Ces deux gènes codent pour des protéines fluorescentes que l on peut distinguer. En observant une population de cellules clones, mais avec des mesures sur cellule unique, ils ont mis en évidence que les niveaux d expression de ces gènes varient considérablement d une cellule à l autre et à l intérieur d une même cellule(voir figure 1). Cette expérience, et de nombreuses autres, ont démontré les effets stochastiques de l expression des gènes. Ce phénomène a bouleversé le domaine de la biologie moléculaire. On peut citer notamment Ehrenberg et al. [26] : 1. http :// Genome/home.shtml

12 1 Biologie, Rappels Historiques 11 Figure 1: Observation expérimentale de population de bactéries. Image tirée de [27]. Le niveau de deux protéines fluorescentes (verte et rouge) est observé en simultané dans chaque cellule. Les deux protéines sont exprimées par des gènes qui possèdent la même séquence d initiation, et qui sont situés dans des endroits similaires du génome. Cette expérience démontre que les effets de l environnement sont primordiaux. There is a revolution occurring in the biological sciences ou Paldi [66] : Is it possible that in biology also, just as in the physical world, macroscopic order is based on the stochastic disorder of its elementary constituents? La précision des expériences permet de quantifier la variabilité dans l expression des gènes. Une modélisation probabiliste est donc adéquate pour interpréter au mieux les expériences. Notre contribution dans l étude d un modèle d expression des gènes va dans ce sens (Chapitre 1). Au-delà de la quantification de la stochasticité de l expression des gènes, beaucoup de questions biologiques restent en suspens. En particulier, beaucoup de biologistes se demandent si l aléatoire dans l expression des gènes a une fonction propre, ou au contraire est «inutile mais inévitable» (voir par exemple [27]). Il n est pas sûr que la modélisation mathématique puisse répondre à cette question. En revanche, beaucoup de questions concernent également les phases du développement des organismes et de la différenciation cellulaire. Certains auteurs ont proposé des théories «Darwiniennes» pour le développement (au niveau du phénotype (quelles protéines sont exprimées) plutôt que du génotype (quels gènes ou allèles sont présents), voir par exemple le travail de Kupiec et al. [47, 48]. Des modèles mathématiques «d évolution», à l échelle cellulaire, pourrait probablement apporter une meilleure compréhension des phénomènes de différenciation cellulaire. Une autre découverte importante en biologie moléculaire a été la mise en évidence d éléments pathogènes de nature protéique. Les maladies liées à ces éléments sont appelées les maladies à prion. Elles peuvent être transmissibles ou sporadiques, mais ne font pas intervenir de virus, de bactéries ou de mutation de gènes. S il y a encore de nombreux débats à ce sujet, l hypothèse la plus répandue actuellement est que les maladies à prion font intervenir uniquement une protéine (appelée prion) qui, lorsqu elle change de conformation et s agrège, devient pathogène. Cette hypothèse a d abord été avancée par Griffith [35] en 1967, puis prouvée par Prusiner [69] en Depuis, de nombreuses expériences ont été réalisées pour étudier la dynamique d agrégation de cette protéine, qui est une étape clé pour l apparition de la maladie. Ces expériences peuvent être réalisées in vivo (à l intérieur de cellules) ou in vitro (dans des tubes à essai) (voir par exemple Liautard et al. [54]). Une curiosité de ces expériences est la grande variabilité des résultats obtenus, tant au niveau de la dynamique d agrégation (temps d apparition de grands polymères, rapidité de la vitesse d agrégation, voir figure 2) que de la structure obtenue à la fin de l expérience (structure spatiale, propriétés physiques des polymères). Là encore, une mo-

13 12 Introduction Générale Figure 2: Résultats d expériences d agrégation de protéines prion, obtenus dans les mêmes conditions expérimentales et avec la même condition initiale. Les données de ces expériences sont tirées de [54]. délisation probabiliste semble donc adéquate pour prendre en compte cette variabilité, et tenter d expliquer les phénomènes sous-jacents. Notre contribution dans l étude d un modèle d agrégation-fragmentation de protéines va dans ce sens (Chapitre 2). 2 Modélisation Mathématique C est dans ce contexte de découverte de mécanismes aléatoires en biologie que s inscrivent mes travaux de thèse. La modélisation mathématique en biologie est un domaine relativement récent, qui a d abord concerné surtout la dynamique des populations. Que ce soit en dynamique des populations, ou dans les modèles de réactions biochimiques, la modélisation mathématique apporte une approche qualitative et quantitative. Dans les modèles de réactions biochimiques, la loi d action de masse permet de représenter la dynamique d un ensemble d entités biochimiques, interagissant via des réactions cinétiques, sous forme d un système d équations différentielles ordinaires. Une étude qualitative de ces équations (comportement en temps long, états d équilibre, bifurcations...) permet alors de comprendre le comportement global du système, et de valider ou non le modèle en fonction des observations expérimentales. L approche quantitative consiste à estimer les valeurs de certains paramètres, ou de variables non observables, soit grâce à une résolution explicite des équations, soit à l aide de simulations numériques. Dans le contexte des modèles d expression des gènes, le travail de Goodwin [34], rendu rigoureux mathématiquement peu après [36, 37, 65, 76, 84], est un exemple important. Cette série de travaux a montré que le niveau d expression d un gène pouvait présenter un caractère monostable, bistable ou oscillant suivant les hypothèses de régulation. Dans le contexte des modèles d agrégation de protéines, plus particulièrement le modèle de Becker-Döring [11], les travaux de [4] illustrent également l approche quantitative, en montrant les propriétés asymptotiques du modèle (convergence vers un état d équilibre, ou explosion, en fonction de la condition initiale et des paramètres). Pour une revue récente des techniques utilisées pour les modèles déterministes de réactions chimiques, voir Othmer and Lee [64]. Dès 194, le biophysicien Max Delbrück a démontré que le faible nombre de molécules enzymatiques dans une cellule pouvait donner lieu à de grandes fluctuations d entités biochimiques à l intérieur d une cellule, et avoir des impacts importants sur la physiologie des cellules. Ces idées ont été largement utilisées pour étudier des modèles de réactions chimiques et caractériser les fluctuations possibles [77]. Bartholomay [9] a établi une analogie entre ces modèles et les modèles de naissance et de mort en théorie des probabilités. Mc-

14 2 Modélisation Mathématique 13 Quarrie [56] a résumé les résultats analytiques connus, pour les réactions uni-moléculaires principalement (voir aussi les récentes contributions de [32],[3]). L approche classique traduit l évolution temporelle des entités chimiques en un système d équations sur la probabilité de trouver tel état du système au temps t (équation maîtresse). Ces équations étant généralement compliquées, on cherche en général uniquement à résoudre les deux premiers moments (moyenne et variance) pour quantifier les fluctuations. Une autre approche concerne les processus stochastiques qui décrivent l évolution temporelle du nombre de molécules. Dans les modèles biochimiques, les processus stochastiques sont des processus de saut. Les équations stochastiques peuvent ainsi s écrire à l aide de processus de Poisson standards. À chaque réaction chimique du type α 1 A 1 α 2 A 2 α n A n β 1 A 1 β 2 A 2 β n A n, on associe un processus de saut d intensité λ ÔX A1,X A2,,X An Õ et de saut X Ai X Ai Ôβ i α i Õ pour la réaction directe, et d intensité λ ÔX A1,X A2,,X An Õ et de saut X Ai X Ai Ôβ i α i Õ pour la réaction inverse, où X Ai est le nombre de molécules de type A i. Un choix usuel pour l intensité des réactions est donné par la loi d action de masse. L intensité dépend alors du nombre de rencontres de molécules, donc du nombre de α i -uplets que l on peut former avec X Ai molécules. Pour la réaction directe, par exemple, on aurait λ ÔX A1,X A2,,X An Õ k où, pour α, fôα,xõ 1, et pour tout α È N, nõ i1 fôα i,x Ai Õ, fôα,xõ XÔX 1Õ ÔX α 1Õ, α! et k représente la constante de vitesse de réaction (qui peut dépendre du volume, de la température, etc.). Exemple 1. Donnons un exemple simple, constitué des réactions A A k 1 k B 1 A À. k2 La première réaction est une transformation de deux molécules A pour donner une molécule B. La deuxième réaction est une réaction de dégradation. L évolution du nombre de molécules ÔX A,X B Õ est donnée d après la loi d action de masse par le système d équations différentielles stochastiques suivant : ³² ³± X A ÔtÕ X A ÔÕ 2Y 1 t k 1 2 X AÔsÕÔX A ÔsÕ 1Õds 2Y 2 t Y 3 t t k t X B ÔtÕ X B ÔÕ 2Y X AÔsÕÔX A ÔsÕ 1Õds 2Y 2 k 1 X BÔsÕds k 2 X A ÔsÕds k 1 X BÔsÕds où les Y i, i 1,2,3, sont des processus de Poisson standards indépendants associés à chaque réaction.,,

15 14 Introduction Générale Revenons au cas général. Si on note X le vecteur des quantités de molécules dans le système, λ i ÔXÕ l intensité de la réaction i, et α i, β i les vecteurs de stœchiométrie associés à la réaction i, l évolution du système se décrit par : XÔtÕ XÔÕ ô t Ôβ i α i ÕY i i λ i ÔXÔsÕÕds. Remarque 1. Les hypothèses physiques sous-jacentes d une telle approche sont : une diffusion rapide, un système bien mélangé, l absence de corrélation entre les positions des molécules ou entre les réactions. Nous utiliserons au cours de cette thèse ce formalisme pour décrire nos modèles (voir section 3). Notre but sera alors d obtenir une caractérisation qualitative et quantitative des modèles. En particulier, on s intéressera aux comportements en temps long (convergence vers un état d équilibre), et à la recherche de solutions analytiques, exactes ou approchées. Cette approche nous permettra en retour de pouvoir exploiter des données expérimentales. Dans la suite de cette introduction, on présente plus précisément les travaux de cette thèse (section 3), et les perspectives (section 4). Dans la dernière partie, on introduit les différents outils mathématiques sur les processus Markoviens que l on a utilisés, principalement des résultats de stabilité (section 6) et des théorèmes limites (section 7), utilisant des formalismes de semi-groupes et de martingales. 3 Résultats de Cette Thèse Au cours de cette thèse, nous étudions deux modèles probabilistes appliqués à la biologie moléculaire. Bien que faisant partie du même domaine d application, ces deux modèles sont assez distincts, et seront donc présentés séparément. Le premier modèle est un modèle d expression des gènes, et a été principalement étudié lors de mes séjours (deux fois six mois) à l Université McGill, à Montréal (Qc, Canada), sous la direction de Michael C. Mackey. Le deuxième modèle est un modèle d agrégation de protéines, et a été principalement étudié à l Université Lyon 1, sous la direction de Laurent Pujo-Menjouet. Les deux études font cependant intervenir des outils communs d analyse mathématique de modèles probabilistes (voir sections 6 et 7). Dans le Chapitre I, nous étudions le modèle standard d expression des gènes, à trois étapes : ADN, ARN messager et protéines. L ADN peut être dans deux états, respectivement «ON» et «OFF». La transcription (production d ARN messager) peut avoir lieu uniquement lorsque l ADN est dans l état «ON». La traduction (production de protéine) est proportionnelle à la quantité d ARN messager. Enfin la quantité de protéines peut réguler de manière non linéaire les taux de production précédent. La version «deterministe», sous forme de système d équations différentielles ordinaires, modélisant les concentrations des espèces biochimiques, a été étudiée dans les années 6. On connait maintenant précisément les comportements en temps long en fonction des paramètres du modèle. En particulier, on sait que si la régulation est positive, et suffisamment non linéaire, il y a une bifurcation fourche. Le système peut avoir deux états d équilibres stables. Lorsque la régulation est négative, et suffisamment non linéaire, il y a une bifurcation de Hopf. Le système peut avoir des oscillations stables. Nous avons étudié une version «stochastique» de ce modèle, sous forme d une chaîne de Markov en temps continu. La difficulté de ce modèle est due au fait que certains taux de saut de la chaîne de Markov sont non linéaires, ce qui rend l analyse mathématique plus délicate. Tout d abord, nous dérivons les cinétiques de Michaelis-Menten et de Hill, dans le formalisme des processus de saut,

16 3 Résultats de Cette Thèse 15 en utilisant des techniques de moyennisation. Ensuite nous donnons des conditions «raisonnables» pour que la chaîne de Markov soit exponentiellement ergodique, en utilisant les critères de stabilité usuels. Pour étudier quantitativement le modèle, nous utilisons une version réduite du modèle, en dimension 1, et avec une production intermittente (bursting, ce phénomène a été bien caractérisé expérimentalement). Ce modèle peut-être vu comme un modèle Markovien déterministe par morceaux. Nous donnons ici des conditions précises pour la convergence asymptotique vers un état stationnaire que l on peut calculer explicitement dans certains cas. Cette résolution explicite nous permet d abord d étudier les P-bifurcations (nombre de modes (maxima) de la densité stationnaire) et de comparer ainsi les diagrammes de bifurcations du modèle stochastique avec celui du modèle déterministe. Nous mettons notamment en évidence des phénomènes relativement généraux, de bifurcation avancée et élargie pour l apparition de deux modes sur la densité stationnaire. Cette étude du comportement en temps long nous permet également de nous intéresser au problème inverse : à partir d une densité de probabilité mesurée expérimentalement, retrouver la fonction de régulation tout entière (et pas seulement la valeur d un paramètre). Le traitement de données existantes et adaptées à notre modèle est en cours de réalisation. Enfin, pour compléter l étude de ce modèle, nous montrons rigoureusement, par des techniques de convergence de processus stochastiques, le passage du modèle initial au modèle réduit. En effectuant une mise à l échelle, réaliste du point de vue biologique, nous obtenons ainsi une convergence en loi vers le modèle limite, ce qui donne les conditions sur les paramètres pour observer le phénomène de production intermittente d ARN messagers ou de protéines. Dans le Chapitre II, nous étudions une version stochastique du modèle d agrégationfragmentation de polymères. Dans un premier temps, nous regardons le modèle sans fragmentation, de Becker-Döring, pour modéliser le phénomène de nucléation dans le processus d agrégation des protéines prion. La nucléation est le passage d un état défavorable (thermodynamiquement) pour l agrégation à un état favorable. La caractérisation quantitative de cette étape est donc essentielle pour comprendre la dynamique d agrégation des protéines. La version stochastique du modèle de Becker-Döring permet une définition de la nucléation en accord avec les modèles biologistes pour les maladies à prion : le temps d apparition du premier agrégat de taille suffisante. Ces protéines ont une conformation telle que, en-dessous d une certaine taille, les agrégats ne sont pas stables, alors qu au-dessus d une certaine taille, ils deviennent stables. La taille critique correspond à la taille du noyau. Nous caractérisons alors la distribution des temps de nucléation dans les modèles d agrégation de protéines, en utilisant la théorie des temps de passage pour les chaînes de Markov. La difficulté de ce modèle réside dans la grande taille de l espace des états de la chaîne de Markov. Nous avons alors mis en évidence plusieurs approximations analytiques, valables dans différentes régions de paramètres. Nous avons validé ces approximations à l aide de simulations numériques de la chaîne de Markov. Le comportement du temps de nucléation a alors des propriétés à priori contre-intuitives. D une part, il dépend de manière non-monotone avec les paramètres cinétiques d agrégation du modèle. D autre part, dans une certaine région de paramètre, il dépend très faiblement de la quantité initiale de protéines. Le phénomène de nucléation étant un phénomène très répandu en biophysique, ces résultats peuvent avoir un impact important (la dérivation de lois d échelles permet d éviter un grand nombre de simulations, et une analyse plus rapide et plus simple de modèles liés). Pour le modèle particulier de l agrégation des protéines prion, il permet une étude quantitative des observations expérimentales (qui reste à faire). Dans un deuxième temps, nous étudions un modèle de polymérisation-fragmentation, en présence de grands polymères déjà formés (plus grands que la taille du noyau). Cependant, sous sa forme discrète, au vu du grand nombre de protéines et des différences d échelles de

17 16 Introduction Générale temps entre la polymérisation et la fragmentation, il n est pas très adapté à une approche quantitative. Nous effectuons alors une mise à l échelle, pour obtenir un modèle limite où la polymérisation est déterministe (donné par une dérive), et la fragmentation est représentée par un processus de saut. Dans ce modèle limite, les protéines non agrégées sont représentées par une variable continue, et le nombre de polymères est discret. Ce modèle permet de prendre en compte la variabilité de la vitesse de polymérisation observée expérimentalement. Sous une forme simple, ce modèle est un processus de branchement. En général, c est un modèle individu-centré avec une compétition indirecte entre les individus. Enfin, lorsque les deux régimes sont mis bout à bout, la nucléation puis la polymérisationfragmentation, ce modèle«hybride» peut facilement incorporer un phénomène récemment observé expérimentalement : la possibilité d apparition de différentes structures de polymères. L hypothèse biologique sous-jacente est que la protéine prion peut se présenter sous différentes conformations spatiales, et mène ainsi à des agrégats de structure spatiale différente. Ces différents polymères ont des dynamiques de polymérisation et fragmentation propres à leur structure. Notre approche quantitative peut alors aider à l identification des différents paramètres de polymérisation et fragmentation, et confirmer (ou donner un poids supplémentaire à) l hypothèse biologique. 4 Perspectives Du pointdevuedela modélisation en biologie, les études des deuxmodèles quej ai menées permettent une approche quantitative des données expérimentales. Le traitement des données et l application de mes résultats par confrontation avec des données expérimentales est encore à finaliser. Pour le modèle d expression des gènes, la possibilité de trouver la fonction de régulation à partir de la densité stationnaire (et de la mesure d autres paramètres) devrait intéresser des biologistes expérimentaux. Cela permet en effet d étudier les interactions précises entre les protéines et les molécules d activation du gène, qui peuvent notamment être modifiées expérimentalement par des modifications chimiques. Le traitement de données existantes est en cours. Pour le modèle d agrégation des protéines prion, la possibilité de prendre en compte la variabilité et l émergence de différentes structures de polymères dans un même modèle permet de réinterpréter un certain nombre de résultats expérimentaux. Au cours de ce travail, j ai démontré des théorèmes de convergence pour certains modèles Markoviens, en utilisant les techniques classiques de martingales. Les théorèmes limites obtenus au chapitre I et au chapitre II sont inhabituels dans le sens où le modèle limite est un processus hybride, mêlant un comportement déterministe et un comportement stochastique. Les approximations de second ordre pour ces limites sont intéressantes à regarder. Pour le modèle d expression des gènes en particulier, la caractérisation des fluctuations autour du modèle limite permettrait une meilleure approximation du modèle initial. Une première extension, pour le modèle d expression des gènes, serait d étudier le modèle avec switch (ON-OFF) et avec production intermittente (bursting). Ces phénomènes ont été bien étudiés séparément, mais jamais (à ma connaissance) ensemble. Une étude qualitative et quantitative présenterait un intérêt non négligeable. En particulier, dans ce modèle, les temps entre production ne sont pas exponentiels (lors que le système est dans l état OFF, il faut au moins deux étapes pour obtenir un événement de production). Ceci peut en faire un modèle plus réaliste, au vu des récentes mesures expérimentales [79] des temps entre événements de production. Pour le modèle d expression des gènes toujours, la bifurcation que l on a obtenue sur le modèle réduit, de dimension un, est analogue à la bifurcation fourche du modèle détermi-

18 5 Notations 17 niste. En revanche, le modèle en dimension un ne présente pas de bifurcation de Hopf. Une étude quantitative du modèle en dimension deux, ou à l aide de simulations numériques, devrait pouvoir caractériser la bifurcation de Hopf dans le modèle stochastique. Ceci reste un problème délicat (voir par exemple dans le cas de modèles Browniens [1, 74, 14, 85]) Concernant le modèle de polymérisation-fragmentation, le modèle limite hybride que l on a obtenu est intéressant pour plusieurs raisons : d abord, il peut donner des schémas efficaces de simulation numérique; ensuite, il peut apporter des résultats quantitatifs sur la vitesse de polymérisation, qui est facilement mesurable expérimentalement. D un point de vue plus théorique, ce modèle n a pas (à ma connaissance) été étudié. En particulier, le comportement en temps long, les phénomènes de gélation (perte de masse par création d une molécule géante) et de poussière (perte de masse par création d une infinité de particules microscopiques) seraient intéressants à regarder et pourraient être comparés avec les modèles déterministes (type EDO ou EDP) et stochastiques (type chaîne de Markov) [62, 4]. Enfin, dans l étude que nous avons mené sur le premier temps d apparition d un noyau, dans le modèle de Becker-Döring, il reste encore des comportements asymptotiques intéressants à regarder. Nous avons caractérisé le temps de nucléation pour un nombre fini de molécules dans les deux asymptotiques de taux de détachement très faible et très grand. Nous avons aussi montré que le caractère discret de ce problème donne des comportements non monotones en fonction des paramètres d agrégation. Ces comportements apparaissent surtout lorsque le nombre total de molécules M est comparable avec la taille du noyau N. Une limite naturelle à regarder serait ainsi M et N avec MßN. Les modèles limites de type champ-moyen pour les modèles d agrégation-fragmentation sont connus [1], et sont des variantes de l équation de Smoluchowski. En revanche, à ma connaissance, le problème de la nucléation n a pas été étudié sur ces modèles. Par ailleurs, pour l ensemble des approximations du temps de nucléation que nous avons trouvées, et validées numériquement, il reste le problème de la quantification de l erreur, qui est un problème intéressant tant au point de vue pratique que théorique. 5 Notations Nous rappelons ici des notations usuelles et des résultats de théorie des semi-groupes. Les semi-groupes que l on regardera agiront sur les espaces de fonctions bornées (ou des sous-espaces) ou sur les espaces de fonctions intégrables (ou des sous-espaces). Soit ÔL, Ð ÐÕ un espace de Banach. On note DÔAÕ le domaine de l opérateur linéaire A. On dit que A B, ou que B est une extension de A, si DÔAÕ DÔBÕ, Bu Au pour u È DÔAÕ. On identifie un opérateur A et son graphe ØÔf,AfÕ : f È DÔAÕÙ. En particulier, un opérateur A est fermé si son graphe est fermé dans L L. Un opérateur A est dit fermable s il a une extension fermée. Si A est fermable, alors la fermeture A de A est la plus petite extension fermée de A, c est-à-dire l opérateur fermé qui a pour graphe la fermeture dans L L du graphe de A. Si A est tel que DÔAÕ est dense dans L, alors A est fermable. Si ÔA,DÔAÕÕ est un opérateur linéaire fermé, alors un sous-espace D de DÔAÕ est appelé un core pour A si la fermeture de la restriction de A à D est égale à A, c est-à-dire A D A.

19 18 Introduction Générale Un opérateur A est dissipatif si Ðλu AuÐ λðuð, pour tout u È DÔAÕ et λ. On note l image d un opérateur ImÔAÕ : AÔDÔAÕÕ. Si A est dissipatif et ImÔAÕ A, alors A est fermable, et A est encore dissipatif. Pour tout σ, on définit la résolvante de A par RÔσ,AÕ Ôσ AÕ 1. Une famille ØT ÔtÕ : t Ù d opérateurs linéaires bornés sur L est un semi-groupe si T ÔÕ I, T Ôt sõ T ÔsÕT ÔtÕ, pour tout t,s. Un semi-groupe ØT ÔtÕÙ est fortement continu si limt ÔtÕf f pour tout f È L. Un semi-groupe ØT ÔtÕÙ est un semi-groupe de contraction si ÐT ÔtÕÐ 1 pour tout t. Le générateur infinitésimal d un semi-groupe ØT ÔtÕÙ est l opérateur linéaire A défini par : t 1 Af lim ÖT ÔtÕf f. t t Le domaine DÔAÕ du générateur infinitésimal A est l ensemble des f È L tel que cette limite existe. Pour la théorie des semi-groupes, on se réfère à Engel and Nagel [29]. Dans la suite, ÔΩ,F,PÕ est un espace de probabilité, et E est l intégrale surωsuivant P. 6 Étude Théorique de Modèles Stochastiques Nous allons passer en revue dans cette section les résultats classiques mais fondamentaux sur les modèles Markoviens. Nous regarderons en particulier les problèmes d existence, d unicité et de comportement en temps long de ces modèles. Nous nous intéresserons uniquement aux modèles homogènes en temps. Nous voulons présenter dans cette partie les différents types de formalisme utilisés au cours de cette thèse. Nous citerons alors des résultats importants dans l étude du comportement de ces différents modèles, que nous utiliserons dans les chapitres de cette thèse. Nous mettrons aussi en avant les liens entre les approches probabilistes et analytiques que l on a utilisées. En aucun cas cette partie ne cherche à être exhaustive concernant l ensemble des résultats de la littérature! 6.1 Chaîne de Markov à temps discret Nous suivons dans un premier temps une référence classique pour les chaînes de Markov, le livre de Brémaud [15] ainsi que des notes de cours de Bérard [12]. En temps discret, une chaîne de Markov (homogène) est une généralisation au cas aléatoire d équations aux différences du type x n 1 f Ôx n Õ. Pour une chaîne de Markov à temps discret et à valeurs dans un espace fini ou dénombrable, la définition est plus facile car il n est pas nécessaire de prendre en compte les questions de mesurabilité. Une chaîne de Markov peut alors être définie simplement par la propriété de Markov et par une matrice (ou plus généralement un noyau) de transition. Dans toute cette partie, E est un espace dénombrable. Définition 1. [Chaîne de Markov homogène à temps discret et espace d états dénombrable] Une suite de variables aléatoires ÔX n Õ définies sur un espace de probabilité ÔΩ,F,PÕ, à valeurs dans E espace d états dénombrable, est une chaîne de Markov homogène si pour tout entier n et tous états i,i 1,,i n 1,i,j, P X n 1 j X n i,x n 1 i n 1,,X i P X n 1 j X n i,

20 6 Étude Théorique de Modèles Stochastiques 19 et si le noyau de transition (indépendant de n) défini par p ij P X n 1 j X n i vérifie les propriétés suivantes p ij, ô kèe p ik 1. Une telle chaîne de Markov est alors entièrement caractérisée par la donnée de sa loi initiale et de son noyau de transition. Soit ν la loi initiale de la chaîne de Markov, c est à dire ν ÔiÕ P X i pour tout i È E. Il vient directement de la propriété de Markov que la loi ν n de X n vérifie la relation de récurrence ô ν n 1 ÔjÕ kèe ν n ÔkÕp kj νn T PÔjÕ, j È E,n È N, où P Ôp ij Õ i,jèe, ν n Ôν n ÔiÕÕ ièe. et ν T est la transposée de ν. On a alors immédiatement ν T n νt P n, et plus généralement que la loi du k-uplet ÔX,X 1, X k 1 Õ vérifie P X i,x 1 i 1,,X k 1 i k 1 ν Ôi Õp i i 1 p ik 2i k 1. Bien qu élémentaire, la notion de chaîne de Markov est fondamentale dans toute la théorie des processus de Markov. Elle est également largement utilisée dans de nombreux modèles, notamment en biologie, avec le processus de Galton-Watson par exemple dans les modèles de dynamique des populations (voir à ce sujet Kimmel and Axelrod [45]) Pour étudier le comportement en temps long d une chaîne de Markov, il est naturel de regarder les distributions (ou lois) stationnaires (en temps). Définition 2. [Distribution stationnaire] Une loi de probabilité π sur E est dite stationnaire pour la chaîne de Markov de noyau de transition P, si π T π T P. (1) De manière plus générale, une mesure invariante est une mesure positive (non nécessairement finie) qui vérifie la relation (1). Si une chaîne de Markov ÔX n Õ, de noyau de transition P, est telle que X a pour loi π, stationnaire pour P, alors X n est de loi π pour tout temps n. Il est alors naturel de se demander ce qu il en est si la loi initiale est quelconque. Pour cela nous avons besoin de quelques définitions supplémentaires, qui sont utiles pour enlever certaines «pathologies». Premièrement, la chaîne de Markov peut visiter différents sous-ensembles de l espace d états suivant sa condition initiale. Pour cela, on définit la notion d irréductibilité. Définition 3. [Irréductibilité] Une chaîne de Markov est irréductible sur E si tous les états i,j È E communiquent, c est à dire s il existe un chemin fini i,i 1,,i k,j tel que p ii1 p i1 i 2 p ik 1i k p ik j. Deuxièmement, si tous les états de E ont une chance d être visités, une chaîne de Markov peut avoir un comportement périodique, «trop régulier» pour avoir de la densité. Pour mesurer le comportement périodique, on définit la notion de période. Définition 4. [Période] La période d i d un état i È E est par définition d i p.g.c.døn 1,p ii ÔnÕ Ù, où p ii ÔnÕ est la somme des probabilités des chemins de taille n reliant i à i, et d i si p ii ÔnÕ.

21 2 Introduction Générale Pour une chaîne de Markov irréductible, tous les états sont de même période. Si d 1, on dit alors que la chaîne est apériodique. Avec les notions d irréductibilité et d apériodicité, on est assuré que la chaîne visite tout l espace, de façon «non dégénérée». De manière informelle, on a alors la dichotomie suivante pour le comportement en temps long. Soit la chaîne «reste» essentiellement dans un compact, soit elle «part» à l infini. On définit pour cela les notions de récurrence et transience, à l aide des temps de premier retour T i inføn 1,X n i X iù. Définition 5. [Récurrence et Transience] Un état i È E est récurrent si P T i 1, et transient sinon. Un état récurrent est positivement récurrent si E T i. Ànouveau,pourunechaînedeMarkovirréductible,siunétat i È E est récurrent(respectivement positivement récurrent), alors tous les états j È E sont récurrents (respectivement positivement récurrents). On parle alors de chaîne de Markov récurrente (respectivement positivement récurrente). On a une relation forte entre la notion de récurrence et de mesure invariante, donnée par la propriété de régénération suivante : Proposition 1. [15, thm 2.1 p11] Soit ÔX n Õ une chaîne de Markov irréductible récurrente, et j È E un état quelconque. Alors νôiõ E ô n1 est une mesure invariante pour ÔX n Õ. 1 ØXniÙ1 ØnTj Ù X j, i È E, On peut alors montrer que pour une chaîne de Markov irréductible récurrente, une mesure invariante est toujours unique, à facteur multiplicatif près. L existence est donnée par le critère suivant, très utile dans la pratique : Proposition 2. [15, thm 3.1 p14] Une chaîne de Markov irréductible est positivement récurrente si et seulement s il existe une distribution stationnaire. De plus, si elle existe, la distribution stationnaire est unique et strictement positive sur E. Finalement, le principal théorème de convergence asymptotique pour les chaînes de Markov (homogènes) à temps discret sur un espace d états dénombrable s énonce ainsi : Théorème 2. [15, thm 2.1 p13] Soit ÔX n Õ une chaîne de Markov irréductible, positivement récurrente et apériodique, de noyau P. Alors, pour tous µ et ν probabilités de distribution sur E, on a lim n dôµt P n,ν T P n Õ, où dôµ,νõ ô ièe µôiõ νôiõ. Ce théorème donne donc une convergence en variation totale. Cette convergence implique bien sûr une convergence en loi. La convergence en variation totale ne fait intervenir que les distributions marginales du processus. L idée de la preuve est alors la suivante. On

22 6 Étude Théorique de Modèles Stochastiques 21 utilisedesmodifications X ½ n et X ¾ n dex n pourmontrerlaconvergenceci-dessus. Laconvergence en temps long revient à trouver deux modifications de X n tel que X ½ n X ¾ n après un temps aléatoire τ. On a alors en effet, dôx ½ n,x¾ n Õ P τ n. (2) En considérant la chaîne produit ÔXn ½,X¾ nõ, on montre qu elle est irréductible (on utilise ici l apériodicité), et possède une distribution stationnaire (donnée par le produit des deux distributionsstationnaires). Parlaproposition 2, onap τ 1, et on conclut d après l éq. (2). Cette méthode s appelle la méthode de couplage. Elle peut être étendue pour trouver la vitesse de convergence vers l état stationnaire [15]. Pour la généralisation à un espace d états quelconque, nous suivons Durrett [25]. Soit ÔS,SÕ un espace mesurable, et un espace de probabilité ÔΩ,F,PÕ muni d une suite de filtrations F n (que l on peut penser comme les filtrations générées par ÔX,X 1,,X n Õ). On définit maintenant une chaîne de Markov à espace d états quelconque. Définition 6. ÔX n Õ est une chaîne de Markov par rapport à la filtration F n si X n È F n et satisfait la propriété de Markov P X n 1 È B F n pôxn,bõ, où p : S S R est tel que : pour tout x È S, A pôx,aõ est une mesure de probabilité sur ÔS,SÕ, pour tout A È S, x pôx,aõ est une fonction mesurable. Les lois de X n sont déterminées par la propriété de Markov, comme dans le cas d un espace dénombrable. L existence des chaînes de Markov ÔX n Õ est alors donnée par le théorème d extension de Kolmogorov (voir par exemple [25, thm 7.1 p 474]). Pour une chaîne de Markov à espace d états quelconque, la notion d irréductibilité est remplacée par la notion de chaîne de Harris. Définition 7. [Chaîne de Harris] Une chaîne de Markov ÔX n Õ est une chaîne de Harris si on peut trouver deux ensembles A,B È S, une fonction q et une mesure de probabilité ρ sur B tels que : qôx,yõ ε pour tous x È A, y È B; si T A inføn : X n È AÙ, alors P T A X z pour tout z È S; si x È A et C B, alors pôx,cõ C qôx,yõρôdyõ. L avantage de cette notion est qu on peut toujours supposer (quitte à modifier l espace S et la chaîne X n ) qu une chaîne de Harris possède un point α qu elle visite avec probabilité 1. Les notions de périodicité, récurrence et transience peuvent alors s étendre aux chaînes de Harris en considérant ce point α. Nous donnerons simplement le théorème de convergence analogue au théorème 2 (légèrement moins fort) : Théorème 3. [25, thm 6.8 p 332] Soit ÔX n Õ une chaîne de Harris apériodique récurrente. Si ÔX n Õ a une distribution stationnaire π, et si α est tel que P T α X x 1, alors lim n d vôδ T x Pn,πÕ.

23 22 Introduction Générale 6.2 Chaîne de Markov à temps continu Nous allons commencer par rappeler la définition d un processus ponctuel de Poisson (sur R ), puis introduire les chaînes de Markov à temps continu, via l approche des semi-groupes de transition. Cette approche a l avantage de se généraliser «facilement» aux processus de Markov par morceaux (et à bien d autres objets), que nous introduirons ensuite. Tout comme les chaînes de Markov en temps discret sont une variante aléatoire des équations aux différences, les chaînes de Markov à temps continu peuvent être vues comme une généralisation des équations différentielles ordinaires. Le«second membre» de l équation différentielle ordinaire (autonome) dx dt f ÔxÕ se traduit par le générateur infinitésimal de la chaîne de Markov (homogène). Nous suivons à nouveau le livre de Brémaud [15]. Nous présentons d abord les chaînes de Markov à espace d états dénombrables, pour lesquelles une condition naturelle sur le générateur peut être donnée pour que le processus soit de saut pur (voir plus bas). Nous passerons enfin aux chaînes de Markov à espace d états général (on parle plus généralement de processus de Markov), et présenterons les techniques de martingales et de fonction de Lyapounov pour leur stabilité. Définition 8. [Chaîne de Markov homogène à temps continu et espace d états dénombrable] Une collection de variables aléatoires ÔX t Õ t, indexée par R, définie sur un espace de probabilité ÔΩ,F,PÕ, à valeurs dans E espace d états dénombrable est une chaîne de Markov homogène si pour tout entier n, tous états i 1,,i n,i,j, et pour tous temps t,s, s 1,,s n s P X t s j X s i,x sn i n,,x s1 i 1 P Xt s j X s i, dès que les deux membres sont bien définis, et cette quantité ne dépend pas de s. Soit PÔtÕ Øp ij ÔtÕÙ i,jèe où p ij ÔtÕ P X t s j X s i. Alors PÔtÕ est un semigroupe de transition, c est-à-dire : ô PÔtÕ est une matrice stochastique ( p ij ÔtÕ 1), j PÔÕ I, PÔt sõ PÔtÕPÔsÕ. Pour un semi-groupe continu, tel que lim PÔhÕ PÔÕ I (convergence élément par h élément), les quantités suivantes existent toujours : Définition 9. [Generateur] Pour tout état i È E, on définit et pour tout i j È E, On pose également 1 p ii ÔhÕ q i lim È Ö,, h h p ij ÔhÕ q ij lim È Ö,. h h q ii q i, et la matrice A Øq ij Ù i,jèe est appelée générateur infinitésimal du semi-groupe (ou de la chaîne de Markov). Remarque 4. En notation matricielle, on a PÔhÕ PÔÕ A lim. h h

24 6 Étude Théorique de Modèles Stochastiques 23 La notion «équivalente» de chaîne de Markov à temps discret est la notion de processus Markovien de saut pur (régulier), que l on rencontrera plusieurs fois par la suite : Définition 1 (Processus de saut pur). Un processus stochastique ÔX t Õ t à valeurs dans E (espace d état général) est un processus de saut pur si, pour presque tout ω È Ω, et t, il existe εôt,ωõ tel que XÔt s,ωõ XÔt,ωÕ, pour tout s È Öt,t εôt,ωõõ. Il est régulier si l ensemble des discontinuités DÔωÕ de t XÔt,ωÕ est σ-discret, c est-àdire, pour tout c, cardôdôωõ Ö,c Õ. Étant donné une matrice A, on peut donner une construction très simple d un processus Markovien de saut pur qui admette A pour générateur, en imposant une condition supplémentaire sur A. Cette construction est à la base des modèles de réactions chimiques, des modèles déterministes par morceaux (utilisés notamment dans le chapitre 1), et des processus ponctuels (utilisés dans le chapitre 2). Nous détaillons donc cette construction ci-dessous. L ingrédient élémentaire est le processus de Poisson (homogène). Un processus de Poisson est un processus de comptage d événements sur R, qui ont lieu successivement et indépendamment les uns des autres suivant une loi exponentielle. Plus précisément, on peut prendre la définition suivante : Définition 11. Un processus ÔN t Õ t est un processus de Poisson homogène d intensité λ si N, et pour tous temps t 1 t k, les variables aléatoires N tk 1 N t k,,n t2 N t1 sont indépendantes; pour tous a b, NÔbÕ NÔaÕ est une variable de Poisson de moyenne λôb aõ. Avec cette définition, on peut montrer qu un processus de Poisson admet la représentation équivalente, NÔtÕ ô n11 ØÔ,T Ù ÔT n Õ, où les temps d événements T n sont tels que T T1 T 2 et les variables S n T n T n 1 sont indépendantes et identiquement distribuées suivant une loi exponentielle de paramètre λ. On montre également avec cette définition que deux événements se produisent en même temps avec probabilité nulle (donc le processus de Poisson augmente de 1 en 1) et qu il n y a pas d explosion, c est-à-dire lim T n, presque sûrement. n Finalement, si on a deux (ou plus généralement une famille dénombrable) processus de Poisson indépendants, on montre aussi que deux événements ne se produisent pas en même temps (avec probabilité un) et que la somme des processus est encore un processus de Poisson, d intensité donnée par la somme des intensités (si elle est finie dans le cas dénombrable). Nous pouvons maintenant donner la construction d un processus Markovien de saut pur qui admette A pour générateur. On suppose pour cela : Hypothèse 1. q i, q i ô jiq ij.

25 24 Introduction Générale Soit ØN i,j Ù i,jèe,ij une famille de processus de Poisson d intensités respectives Øq i,j Ù i,jèe,ij, et un état initial XÔÕ indépendant de cette famille de processus. On pose alors XÔtÕ X n, pour t È ÖT n,t n 1 Õ, où les couples ÔT n,x n Õ sont définis récursivement par T, X XÔÕ, et, pour tout n, si T n, et X n XÔT n Õ i È E, alors si q i, on pose X n m (point cimetière) et T n m, pour tout m 1; sinon T n 1 est le premier événement qui a lieu après T n des processus ØN i,j Ù jièe, et X n 1 est donné par l index k i pour lequel le processus de Poisson N i,k réalise ce premier événement. Cette construction est valide (T n, X n sont bien définis donc XÔtÕ également) jusqu au temps d explosion T lim n T n. On a alors la proposition suivante : Proposition 3. [15, thm 1.2 p373] Si les conditions données par l hypothèse 1 sont valables, et si T presque sûrement, le processus construit ci-dessus est un processus Markovien de saut pur régulier de générateur infinitésimal A. La preuve repose sur le calcul de P XÔtÕ j XÔÕ i. Si j i, alors T 1 t, et, par indépendance, il vient P XÔtÕ j,t 1 t XÔÕ i Ô1 e q it Õ q ij q i. (3) Enfin, on montre que P T 2 t XÔÕ i est négligeable devant t, d où 1 lim P XÔtÕ j XÔÕ i q ij. t t Remarque 5. Cette approche des processus de saut pur est à la base des équations stochastiques dirigées par des processus de Poisson, et plus généralement des systèmes stochastiques dirigés par des processus ponctuels. Cette approche donne aussi directement une méthode de simulation des trajectoires du processus de saut pur, appelée algorithme de Gillespie [33] dans le contexte des modèles de réactions biochimiques. La méthode de construction décrite ci-dessus correspond à l algorithme de «la prochaîne réaction». A chaque événement, on simule uniquement le prochain temps d événement du processus de Poisson qui correspond à la transition que l on vient d effectuer. En gardant en mémoire tous les prochains événements possibles (pour lesquels q ij, si l on est dans l état i), on avance alors le temps au minimum de tous ces prochains événements possibles, on effectue la transition correspondante, et ainsi de suite. Cette version a l avantage d être largement généralisable à des processus ponctuels non Markoviens (avec retard, ou distribution de temps d événement non exponentielle, voir par exemple [2]). Une autre version de cet algorithme, appelée «méthode directe», vient de la formule (3) utilisée dans la preuve ci-dessus. Le prochain temps d événement est donné par une exponentielle de paramètre q i ji q ij et la transition effectuée est déterminée par un autre nombre aléatoire qui vaut j avec probabilité q ij q j. Cette méthode ne garde pas de valeurs en mémoire (autres que l état dans lequel on est) mais demande de générer deux nombres aléatoires à chaque pas de temps. Avant de passer à la description des processus de Markov plus généraux, citons un critère de convergence en temps long pour les processus Markoviens de saut pur. De la description

26 6 Étude Théorique de Modèles Stochastiques 25 trajectorielle que l on a donnée, on peut voir qu un processus Markovien de saut pur est lié à une chaîne de Markov discrète, donnée par les valeurs après les sauts X n. On étend les notions d irréductibilité, de récurrence et de positive récurrence au processus Markovien de saut pur. La même forme régénératrice (voir proposition 1) est encore valable entre les mesures invariantes (pour le semi-groupe P ÔtÕ) et les temps de premier retour, et on a alors : Théorème 6. Un processus Markovien de saut pur régulier de générateur infinitésimal A, irréductible, est positivement récurrent si et seulement s il existe une loi de probabilité π sur E telle que π T A. Dans ce cas, on a lim t p ij ÔtÕ πôjõ pour tous i,j È E. Remarque 7. Notons les différences entre les théorèmes 2 et 6. Dans le cas continu, on n a pas besoin de supposer la chaîne apériodique. Les temps de passage dans un état sont suffisamment aléatoires pour éviter le comportement périodique. Notons aussi qu il n y a pas forcément de relation entre la convergence en temps long du processus Markovien de saut pur et de sa chaîne de Markov en temps discret correspondante. En particulier, on a la relation entre une mesure invariante ν pour le processus Markovien de saut pur et µ pour la chaîne discrète µôiõ q i νôiõ, qui montre que toutes les possibilités sont ouvertes pour les valeurs respectives de ô ièeµôiõ et ô ièeνôiõ en fonction du comportement de la suite Ôq i Õ ièe. Pour une théorie équivalente sur les processus Markoviens de saut pur à valeurs dans un espace quelconque, voir par exemple [22]). Nous passons maintenant au processus de Markov plus généraux. 6.3 Processus de Markov Dans toute cette partie, E est un espace polonais (i.e. métrique séparable complet) muni de sa structure borélienne BÔEÕ. L ensemble des fonctions mesurables bornées sur E est noté BÔEÕ, que l on munit de la norme «infini» usuelle. L ensemble des fonctions à valeurs réelles, continues à droite et avec limite finie à gauche («cad-lag») sur Ö, Õ est noté D E Ö, Õ. On munit D E Ö, Õ de la topologie de Skorokhod S E. Nous suivrons dans un premier temps principalement le livre de Ethier and Kurtz [3]. On utilise la définition suivante : Définition 12 (Processus de Markov homogène). Une collection de variables aléatoires ÔX t Õ t, indexées par R, définies sur un espace de probabilité ÔΩ,F,PÕ munie d une filtration ÔF t Õ t, à valeurs dans E, un espace polonais, est un processus de Markov homogène par rapport à ÔF t Õ t si pour tous s,t et B È BÔBÕ, P X t s È B F t P Xt s È B X t : P Ôs,XÔtÕ,BÕ, La fonction P Ôt,x,BÕ, définie sur Ö, Õ E BÔBÕ est appelée fonction de transition et satisfait : P Ôt,x, Õ est une mesure de probabilité sur E, pour tous Ôt,xÕ, P Ô,x, Õ δ x, pour tout x, P Ô,,BÕ est mesurable sur Ö, Õ E, pour tout B È BÔBÕ,

27 26 Introduction Générale la relation de Chapman-Kolmogorov, pour tous s,t, x È E et B È BÔBÕ P Ôt s,x,bõ P Ôs,y,BÕP Ôt,x,dyÕ. (4) De manière similaire au cas des chaînes de Markov, les lois des n-uplets de X t sont déterminées par la relation de Chapman-Kolmogorov eq. (4). La topologie sur E (polonais) permet d assurer que ces lois (dites de dimensions finies) déterminent de manière unique un processus de Markov sur E. Comme pour le cas des chaînes de Markov, la relation de Chapman-Kolmogorov définit en un certain sens une structure de semi-groupe sur les fonctions de transition. Cependant, peu de processus stochastiques ont des formules connues pour les fonctions de transition (à l exception du mouvement Brownien, ou de quelques autres processus comme le Ornstein-Uhlenbeck), et il est plus facile de travailler avec le semi-groupe sur les fonctions bornées de E, donné par T ÔtÕf ÔxÕ f ÔyÕP Ôt,x,dyÕ E f ÔXÔtÕÕ XÔÕ x. Il est classique que le semi-groupe ØT ÔtÕÙ sur BÔEÕ (et même sur un sous-ensemble suffisamment gros), avec une loi initiale, détermine de manière unique les lois de dimensions finies de XÔtÕ. Aussi, de par sa définition, T ÔtÕ est un semi-groupe de contraction sur BÔEÕ muni de la norme infini sur E. On cherche dans quel cas le générateur infinitésimal de T ÔtÕ caractérise le semi-groupe, et donc le processus de Markov XÔtÕ. Pour utiliser la théorie classique des semi-groupes, il faut des semi-groupes fortement continus. On va voir que cela définit une sous-classe importante, mais restrictive, de processus de Markov. Ce sont les processus de Feller. Il suffit de regarder le semi-groupe ØT ÔtÕÙ sur l espace C ÔEÕ des fonctions continues sur E et de limite nulle à l infini, muni de la norme «infini», sup f ÔxÕ. Si ØT ÔtÕÙ est un semi-groupe positif de contraction sur C ÔEÕ, fortement xèe continu (limt ÔtÕf f), le théorème de Hille-Yosida caractérise alors le générateur de t T ÔtÕ et celui-ci détermine de manière unique un processus de Markov. Le résultat précis, dans le contexte des processus stochastique, est le suivant : Proposition 4 (Processus de Feller). [3, thm 2.2 p165] Soit E localement compact et séparable, et A un opérateur linéaire sur C ÔEÕ, qui vérifie le domaine de A, DÔAÕ est dense dans C ÔEÕ, A satisfait le principe du maximum positif : si f Ôx Õ sup f ÔxÕ, alors Af Ôx Õ. xèe l image de ÔλI AÕ est dense dans C ÔEÕ pour un certain λ. Soit alors T ÔtÕ le semi-groupe de contraction positif, fortement continu sur C ÔEÕ généré par la fermeture de A. Alors il existe pour tout x È E un processus de Markov X x correspondant à T ÔtÕ, de loi initiale δ x et de trajectoires dans D E Ö, Õ si et seulement si A est conservatif (c est-à-dire Ôf,gÕ Ô1,Õ est dans la fermeture de A). Un tel processus est appelé processus de Feller. Une autre classe importante de processus pour lesquels le générateur est «facilement» caractérisable sont les processus de saut pur, que l on a déjà rencontrés dans le cas d un espace d états dénombrable. Si µôx,bõ est une fonction de transition et λ È BÔEÕ, alors Af ÔxÕ λôxõ Ôf ÔyÕ f ÔxÕÕµÔx, dyõ

28 6 Étude Théorique de Modèles Stochastiques 27 est un opérateur borné sur BÔEÕ, et A est le générateur d un processus de saut pur qui peut être construit de manière analogue au cas d un espace d états dénombrable (voir proposition 3). En particulier, on peut lui associer une chaîne de Markov Y n à temps discret sur E, de fonction de transition µôx,bõ et les temps de saut sont déterminés par des lois exponentielles de paramètres λôy n Õ. Finalement, une approche plus générale, largement reconnue et utilisée actuellement (notamment pour sa commodité avec les théorèmes limites), est celle du problème de martingale, utilisé notamment par Stroock et Varadhan [78] pour caractériser les diffusions sur R d, et Jacod et Shiryaev [39] pour des processus à accroissements indépendants. Elle repose sur le générateur étendu, défini par : Définition 13 (Générateur étendu). Soit ØT ÔtÕÙ un semi-groupe de contractions sur BÔEÕ. Son générateur étendu est défini comme l opérateur (possiblement multi-valué) t µ  Ôf,gÕ È BÔEÕ BÔEÕ : T ÔtÕf f T ÔsÕgds. On a alors la proposition classique mais fondamentale : Proposition 5. [3, thm 1.7 p162] Soit XÔtÕ un processus de Markov à trajectoires dans D E Ö, Õ de fonction de transition P Ôt,x,BÕ. Soient ØT ÔtÕÙ son semi-groupe sur BÔEÕ associé, et  son générateur étendu. Alors, si Ôf,gÕ È Â, t MÔtÕ f ÔXÔtÕÕ gôxôsõõds, est une martingale par rapport à la filtration F X t canonique associée XÔtÕ. L hypothèse sur les trajectoires de XÔtÕ est suffisante pour que l intégrale définissant MÔtÕ ait un sens (mais on peut faire mieux). L idée de la preuvede cette proposition réside dans un simple calcul : E MÔt uõ F X t E f ÔXÔt uõõ F X t t u E gôxôsõõ F X t ds, u E f ÔXÔt uõõ XÔtÕ t u T ÔuÕf ÔXÔtÕÕ T ÔsÕgÔXÔtÕÕds c, f ÔXÔtÕÕ E gôxôsõõ F X t MÔtÕ. t E gôxôsõõ XÔtÕ ds t gôxôsõõds, La deuxième ligne est donnée par la propriété de Markov (pour les deux premières intégrales) et la propriété de l espérance conditionnelle (pour la troisième intégrale). Le reste suit par définition du semi-groupe et de son générateur étendu. Le problème de martingale consiste, étant donné un générateur A et une loi initiale µ sur E, à trouver une mesure de probabilité P È P D E Ö, Õ telle que le processus défini sur l espace ÔD E Ö, Õ,S E,P Õ par XÔt,ωÕ wôtõ, ω È D E Ö, Õ, t, vérifie : t f ÔXÔtÕÕ gôxôsõõds

29 28 Introduction Générale est une martingale par rapport à la filtration F X t canonique associé XÔtÕ, pour tout Ôf,gÕ È A, et XÔÕ a pour loi µ. Des conditions générales sur le générateur étendu  pour avoir existence et unicité de la solution du problème de martingale sont difficiles à obtenir. Ceci est le prix à payer pour une théorie générale. Dans la pratique, par contre, si l on se donne a priori la forme du générateur, il est souvent possible de donner des conditions sur les coefficients du générateur pour que le problème de martingale associé soit bien posé (voir par exemple le cas des diffusions traité par Stroock et Varadhan [78], et des semi-martingales comprenant les processus ponctuels, les processus à accroissements indépendants, les diffusions avec sauts traité par Jacod et Shiryaev [39]). On peut néanmoins dégager plusieurs principes généralement valables pour le problème de l existence et l unicité de la solution du problème de martingale. L existence peut être obtenue par une limite faible de solution d un problème de martingale approché, donnée par la proposition suivante : Proposition 6. [3, prop 5.1 p196] Soit A C b ÔEÕ C b ÔEÕ et A n BÔEÕ BÔEÕ, n 1,2,. On suppose que pour tout couple Ôf,gÕ È A, il existe Ôf n,g n Õ È A n tel que lim n Ðf n fð, lim n Ðg n gð. Soit alors X n une solution du problème de martingale pour A n, avec trajectoires dans D E Ö, Õ, si X n X (convergence en loi), alors X est une solution du problème de martingale pour A. Une autre technique souvent utilisée est la localisation. Elle consiste à se ramener au cas où la solution du problème de martingale est contenue dans un ouvert (que l on prendra borné en général) de E par un argument de troncature. Une solution du problème de martingale arrêtée en un ouvert U est (formellement) une solution du problème de martingale pour tout temps plus petit que le temps de sortie de U. Proposition 7. [3, thm 6.3 p219] Soit A C b ÔEÕ BÔEÕ. Soit U 1 U 2 ouvert de E. Soit ν È PÔEÕ une loi initiale, telle que pour tout k il existe une unique solution X k au problème de martingale ÔA,νÕ arrêtée en U k, avec trajectoires dans D E Ö, Õ. On pose Si pour tout t, τ k inføt : X k ÔtÕ Ê U k ou X k Ôt Õ Ê U k Ù. lim k PÔτ k tõ, alors il existe une unique solution au problème de martingale ÔA, νõ avec trajectoires dans D E Ö, Õ. Finalement, donnons un procédé qui sera utilisé dans le chapitre 2 pour obtenir l unicité de la solution du problème de martingale. Supposons que le générateur A soit le générateur infinitésimal d un semi-groupe fortement continu. Alors de manière classique l opérateur A est fermé, et la résolvante Ôλ AÕ 1 est définie pour tout λ. Supposons que pour tout x È E, il existe une solution au problème de martingale ÔA,δ x Õ (ce qui sera donné si on sait qu il existe un processus de Markov associé au semi-groupe fortement continu). Un simple calcul montre que, pour tous Ôf,gÕ È A, λ, t e λt fôx x ÔtÕÕ e λs ÔλfÔX x ÔsÕÕ gôx x ÔsÕÕÕds (5)

30 6 Étude Théorique de Modèles Stochastiques 29 est une martingale. Il vient alors que fôxõ E e λs ÔλfÔX x ÔsÕÕ gôx x ÔsÕÕÕds. On en déduit alors λðfð Ðλf gð. On a donc la proposition : Proposition 8. [3, prop 3.5 p178] Soit A opérateur linéaire, A BÔEÕ BÔEÕ. S il existe une solution au problème de martingale ÔA,δ x Õ pour tout x È E, alors A est dissipatif (voir section 5). Cette proposition permet de montrer de manière simple qu un opérateur est dissipatif. On peut alors conclure à l unicité de la solution du problème de martingale en identifiant une classe de fonctions séparatrice, comme dans le théorème suivant : Théorème 8. [3, corollaire 4.4 p187] Soit E séparable et A BÔEÕ BÔEÕ linéaire et dissipatif. On suppose que pour un (et donc tous) λ, ImÔλ AÕ DÔAÕ, et qu il existe M BÔEÕ séparatrice, M ImÔλ AÕ pour tout λ. Alors pour toute loi initiale µ, deux solutions du problème de martingale pour ÔA,µÕ à trajectoires dans D E Ö, Õ, ont même loi sur D E Ö, Õ. L ingrédient clé de cette preuve repose toujours sur l identification de la martingale donnée par l éq. (5). En particulier, pour tout h È M, si X et Y sont solutions du même problème de martingale, E e λt hôxôtõõdt Ôλ AÕ 1 hdµ E e λt hôy ÔtÕÕdt, ce qui suffit, par propriété de la transformée de Laplace et de l hypothèse sur M, pour identifier les lois de X et Y. On termine cette section en discutant de la convergence en temps long pour les processus de Markov. L approche la plus générale et utile dans la pratique est donnée par les fonctions de Lyapounov pour le générateur étendu. Voir les travaux de Meyn et Tweedie dans une série de trois papiers [58, 59, 6]. Pour des modèles particuliers, les approches par couplage peuvent s avérer également très puissantes, et donner des taux de convergence explicites très satisfaisants (voir par exemple Bardet et al. [8]). Les idées des méthodes de fonctions de Lyapounov s appuient sur des conditions de dérive du générateur pour des fonctions bien choisies, qui transmettent des propriétés au processus grâce à la formule de Dynkin. Comme pour les chaînes de Markov à temps discret et à espace d états quelconque, il faudra supposer une certaine forme de régénération supplémentaire, similaire à la propriété des chaînes de Harris énoncée dans la définition 7. La puissance des théorèmes de Meyn et Tweedie réside dans l utilisation d une chaîne discrète obtenue à partir d un échantillonnage (quelconque) du processus de Markov. Ceci rend leurs résultats largement utilisables dans beaucoup de cas. Dans tout ce qui suit, on suppose que E est un espace polonais localement compact, muni de sa structure borélienne BÔEÕ. On suppose que XÔtÕ est un processus de Markov à trajectoires dans D E Ö, Õ. On redéfinit les concepts d explosion, d irréductibilité, de récurrence, de récurrence de Harris et de récurrence de Harris positive. On note O n une famille d ouverts pré-compacts de E tel que O n E quand n, et τ n les premiers temps d entrée de X t dans O c n. On dit alors que XÔtÕ est non explosif (ou régulier) si P lim n τ n XÔÕ x 1, x È E.

31 3 Introduction Générale On note ØX t Ù si X t È C c pour tout compact C È BÔEÕ et t suffisamment grand. On dit alors que XÔtÕ est non évanescent si Pour un ensemble mesurable A, on définit P ØX t Ù XÔÕ x, x È E. T A inføt : X t È AÙ, n A XÔtÕ est φ-irréductible si pour une mesure σ-finie φ, 1 ØXtÈAÙdt. φôbõ E T B XÔÕ x, x È E. XÔtÕ est Harris récurrent si pour une mesure σ-finie φ, φôbõ P n B XÔÕ x 1, x È E. Une mesure invariante µ pour un processus de Markov XÔtÕ, de fonction de transition P Ôt,x,BÕ, est telle que µôaõ µp Ôt,,AÕ P Ôt,x,AÕµÔdxÕ. Comme pour les chaînes de Markov récurrentes, un processus de Markov Harris récurrent possède, à un facteur multiplicatif près, une unique mesure invariante. Si elle est finie, on peut alors la normaliser en une distribution de probabilité, et on parle alors de processus de Markov positivement Harris récurrent. Un échantillonnage d un processus de Markov est donné par les valeurs du processus de Markov à certains temps, déterministes ou aléatoires. L échantillonnage le plus simple est celui donné par la résolvante, R : E BÔEÕ Ö,1, RÔx,AÕ P Ôt,x,AÕe t dt. (6) Si Ôt k Õ est une suite d instants générés par des incréments indépendants entre eux (et de X t ) et distribués suivant une loi exponentielle de paramètre 1, alors ÔX tk Õ est une chaîne de Markov à temps discret, de noyau R. Plus généralement, étant donnée une loi de probabilité a sur R, on définit K a Ôx,AÕ P Ôt,x,AÕaÔdtÕ. Pour Ôt k Õ une suite d instants d accroissements indépendants suivant a, X tk est alors une chaîne de Markov à temps discret, de noyau K a. Meyn et Tweedie [59, 6, 58] ont prouvé de nombreux liens entre le processus de Markov et les K a -échantillons. Une classe importante de processus de Markov pour lesquels des résultats de stabilité existent sont les T-processus : Définition 14. Un processus de Markov est un T-processus s il existe une mesure de probabilité a sur R et une fonction non triviale T : E BÔEÕ R (T Ôx,EÕ ) tels que : pour tout B È BÔEÕ, T Ô,BÕ est semi-continu inférieurement; pour tous x È E, B È BÔEÕ, K a Ôx,BÕ T Ôx,BÕ. En lien avec cette notion, nous avons également la notion d ensemble petit :

32 6 Étude Théorique de Modèles Stochastiques 31 Définition 15. Un ensemble non vide C È BÔEÕ est dit ν-petit si ν est une mesure non triviale sur BÔEÕ, et s il existe a une mesure de probabilité sur R tel que K a Ôx, Õ νô Õ pour tout x È C. On dit simplement que C est petit si la donnée de ν n est pas importante. La relation entre ces deux notions est donnée par la proposition suivante : Proposition 9. [59, prop 4.1] Supposons P ØX t Ù XÔÕ x 1 pour un x È E. Alors tout ensemble compact est petit si et seulement si X t est irréductible et est un T- processus. Nous donnons maintenant les critères de stabilité pour un processus de Markov basé sur des fonctions de Lyapounov et sur les notions rappelées ci-dessus. On note O n une famille d ouverts pré-compacts de E tel que O n E quand n, et on note X n le processus stochastique XÔtÕ arrêté en O n, et A n son générateur. Dans toute la suite, V est une fonction de Lyapounov E R, si elle est mesurable, strictement positive et telle que V ÔxÕ quand x. Un critère de non explosion s énonce ainsi : Proposition 1. [6, thm 2.1] S il existe une fonction V de Lyapounov, et c,d tels que A n V ÔxÕ cv ÔxÕ d, x È O n, n 1, alors XÔtÕ est non explosif; il existe une variable aléatoire D finie presque sûrement tel que V ÔX t Õ De ct ; la variable aléatoire D satisfait la borne P D a XÔÕ x V ÔxÕ a, a, x È E; E V ÔX t Õ XÔÕ x e ct V ÔxÕ. Un critère de non-évanescence est donné par : Proposition 11. [6, thm 3.1] S il existe une fonction V de Lyapounov, d et C un compact tels que A n V ÔxÕ d1 ØCÙ ÔxÕ, x È O n, n 1, alors XÔtÕ est non évanescent. Un critère de récurrence est donnée par : Proposition 12. [6, thm 4.1] S il existe une fonction V de Lyapounov, d et C un compact tels que A n V ÔxÕ d1 ØCÙ ÔxÕ, x È O n, n 1, et tels que tous les ensembles compacts sont petit, alors XÔtÕ est Harris récurrent. Un critère de récurrence positive est donnée par : Proposition 13. [6, thm 4.2] S il existe c,d, C un ensemble petit fermé, f 1 et V borné sur C tels que A n V ÔxÕ cfôxõ d1 ØCÙ ÔxÕ, x È O n, n 1, alors, si XÔtÕ est non explosif, XÔtÕ est positivement Harris récurrent et sa mesure invariante est finie. On termine par un critère d ergodicité exponentielle :

33 32 Introduction Générale Proposition 14. [6, thm 6.1] S il existe une fonction V de Lyapounov, c,d, tels que A n V ÔxÕ cf ÔxÕ d, x È O n, n 1, et tels que tous les ensembles compacts sont petit, alors, il existe β 1 et B tels que ÐP Ôt,x, Õ πð f Bf ÔxÕβ t, x È E, t, avec f V 1 et où еРf supgf µôgõ. En revenant aux chaînes de Markov à temps continu et à valeurs dans un espace dénombrable, cette dernière proposition 14 donne immédiatement le critère suivant : Proposition 15. [6, thm 7.1] S il existe une fonction V de Lyapounov, c,d, tels que, ô q ij V ÔjÕ cv ÔiÕ d, i È E, j et si XÔtÕ est irréductible alors il existe π une distribution de probabilité invariante pour XÔtÕ, β 1 et B tels que avec f V 1. ÐP Ôt,i, Õ πð f Bf ÔiÕβ t, x È E, t, Nous utiliserons les propositions 14 et 15 au Chapitre 1 de cette thèse, pour donner des conditions sur nos modèles Markoviens d expression des gènes pour qu ils soient asymptotiquement stables. 6.4 Processus de Markov déterministes par morceaux Les processus de Markov déterministes par morceaux (PDMP piecewise deterministic Markov processes) ont été formalisés rigoureusement par Davis [23], qui a notamment montré qu une construction explicite d un processus déterministe par morceaux définit une solution d un certain problème de martingale. Ainsi, Davis a identifié très précisément le générateur étendu d un PDMP et son domaine. Dans la pratique, comme on a pu le voir dans les propriétés énoncées dans la partie précédente, la connaissance d un sous-ensemble de fonctions séparatrices inclus dans le domaine est cependant généralement suffisant. Nous donnons la construction d un PDMP sans bord, c est à dire que le flot déterministe reste toujours inclus dans l espace d états. Nous supposerons aussi par la suite que le flot déterministe a toujours la propriété d existence et d unicité globale. Un PDMP (sans bord) est donné en tous temps t par un couple ÔiÔtÕ,xÔtÕÕ où iôtõ È J est une variable discrète, J N et xôtõ È R d (on pourrait considérer des espaces plus généraux sans difficulté). Un PDMP est décrit par trois caractéristiques locales : un champ de vecteur H i ÔxÕ, pour tout i È J ; une intensité de saut λ i ÔxÕ, pour tout i È J ; une mesure de transition Q telle que pour tout Ôi,xÕ, QÔ, Ôi,xÕÕ est une loi de probabilité sur J R d. La construction d un PDMP suit celle d un processus de saut pur, sauf que la variable x n est pas constante entre deux sauts, mais suit une équation différentielle déterministe. On pose alors Ôi n,x n Õ ÔiÔT n Õ,xÔT n ÕÕ où ÔT n,i n,x n Õ sont définis récursivement par : T, i iôõ, x xôõ (conditions initiales données);

34 6 Étude Théorique de Modèles Stochastiques 33 si T n, et Ôi n,x n Õ ÔiÔT n Õ,xÔT n ÕÕ, alors pour tous T n t T n 1, t xôtõ g in Ôx n,t T n Õ où g in Ôx,tÕ est donnée par la solution de l équation différentielle ordinaire dy H in ÔyÕ, t, dt yôõ x. La variable discrète t iôtõ est constante égale i n, et T n 1 T n déterminé par P τ n t E t. exp λ in Ôg in Ôx n,sõõds τ n où τ n est Si τ n, on pose x n m (point cimetière) et T n m, pour tout m 1. Sinonτ n,et Ôi n 1,x n 1 ÕestdonnéparlaprobabilitédetransitionQÔ, Ôi n,xôt n 1 ÕÕ. Comme dans les processus de saut pur, cette construction est valable jusqu au temps d explosion T lim n T n. Les conditions générales pour assurer que l explosion n a pas lieu en temps fini sont difficiles à obtenir du fait de nombreuses possibilités entre les évolutions déterministes et les transitions possibles. On peut cependant montrer facilement que si : Hypothèse 2. Les intensités de saut λ i ÔxÕ sont uniformément bornées sur R d, alors T presque sûrement. Cette hypothèse est bien trop forte dans la pratique, et par la suite on supposera donc seulement que : Hypothèse 3. E N t, t, où N t ô n 1 ØtTnÙ est le nombre de sauts entre Ö,t. Pour utiliser les résultats suivants, dans la pratique, il faudra donc montrer que cette hypothèse 3 est vérifiée. Hypothèse 4. On suppose que les champs de vecteurs H i sont C 1 et tels que pour tout x È R d, ils définissent un unique flot global φ i Ôt,xÕ; les intensités de saut sont telles que pour tout couple Ôi,xÕ, λ i Ôφ i Ôt,xÕÕ est localement intégrable en, c est-à-dire qu il existe εôi,xõ tel que εôi,xõ λ i Ôφ i Ôs,xÕÕds. Ces deux conditions impliquent que la construction donnée ci-dessus a un sens. Le flot est toujours défini et on peut choisir un temps de prochain saut strictement positif. Avec les hypothèses 3 et 4, Davis a montré que le processus de Markov Ôi t,x t Õ sur J R d ainsi construit est solution du problème de martingale associé au générateur A, qui s exprime, pour toute fonction bornée de classe C 1 de x (et de dérivée bornée), AfÔi,xÕ H i ÔxÕ x f λ i ÔxÕ ÖfÔj,yÕ fôi,xõ QÔdj dy, Ôi,xÕÕ. (7) L opérateur adjoint donne (formellement) l équation d évolution sur les probabilités de densité pôi, x, tõ du processus pôi,x,tõ ÔH i ÔxÕpÔi,x,tÕÕ λ i ÔxÕpÔi,x,tÕ λ j ÔyÕpÔj,y,tÕQÔÔi,xÕ,dj dyõ. (8) t

35 34 Introduction Générale L existence de solution au problème de martingale est donc donné par la construction explicite d un processus stochastique. D après la proposition 8 et le théorème 8, si l on montre que le semi-groupe engendré par ce processus stochastique est fortement continu (ce qui est le cas si les intensités λ i sont bornées par exemple), on peut obtenir l unicité de la solution du problème de martingale. Les techniques de localisation peuvent aussi être utilisées dans la pratique. Crudu et al. [21] ont montré ainsi, avec des hypothèses fortes (mais qui peuvent être surmontées par des techniques de localisation), le résultat suivant : Théorème 9. [21, thm 2.5] Supposons les hypothèses 3 et 4 ainsi que Hypothèse 5. Les fonctions x H i ÔxÕ, x λ i ÔxÕ et x λ i ÔxÕ fôj,yõqôdj dy, Ôi,xÕÕ pour f È C 1 b, sont C1 b sur Rd. Alors, le PDMP déterminé par ÔH i,λ i,qõ est l unique solution du problème de martingale associé à A défini à l éq. (7). Toujours pour le caractère bien posé du problème de martingale, citons un résultat de perturbation qui peut s appliquer dans la pratique. L idée est de découper le générateur donné à l éq. (7) en deux parties. De manière naturelle (par rapport à la construction explicite du processus) on peut séparer la partie dérive, donnée par l évolution déterministe, de la partie saut. Notons A 1 la partie dérive, et A 2 la partie saut. Supposons que les intensités de saut λ i sont bornées. Alors l opérateur A 2 est un opérateur borné. Si l on s assure que A 1 est dissipatif, que pour un σ, BÔEÕ ImÔσ A 1 Õ, alors BÔEÕ ImÔσ ÔA 1 A 2 ÕÕ. Le théorème 8 donné ci-dessus permet donc de conclure que l unicité a lieu pour A 1 A 2. Pour l existence, on peut utiliser le résultat suivant : Proposition 16. [3, prop 1.2 p 256] Supposons que pour toute loi initiale ν sur J R d, il existe une solution au problème de martingale pour ÔA 1,νÕ à trajectoires dans D E Ö, Õ, alors il existe également une solution au problème de martingale pour ÔA 1 A 2,νÕ à trajectoires dans D E Ö, Õ (où A 2 est l opérateur de saut, avec intensités bornées). L idée de la preuve suit la construction explicite du PDMP. On se ramène d abord au cas λ constant, puis on construit successivement une solution sur tout ÖT k,t k 1Õ, avec la loi de T k 1 T k donnée par une loi exponentielle indépendante du processus, et la condition initiale donnée par la loi du saut Q en la condition finale de l étape précédente, etc. 6.5 Équation d évolution d un PDMP Nous donnons maintenant une stratégie similaire, mais en regardant le semi-groupe sur L 1, associé à l équation d évolution éq. (8). Cette stratégie sera largement utilisée au chapitre 1, sur un modèle PDMP en dimension un, lorsqu il y a uniquement des sauts dans la variable continue, et un seul champ de vecteurs (il n y a pas de variable discrète). Supposons donc pour simplifier qu on est dans un cas où le champ de vecteurs ne change pas et qu il n y a pas de dynamique sur la variable discrète. Le générateur donné dans l éq. (8) est défini par un opérateur de dérive et un opérateur de saut sur la variable continue. Rappelons quelques notions spécifiques aux semi-groupes sur L 1. Soit ÔE,E,mÕ un espace mesuré σ-fini et L 1 L 1 ÔE,E,mÕ de normeð Ð 1. Un opérateur linéaire P sur L 1 est dit sous-stochastique (respectivement stochastique) si Pu et ÐPuÐ 1 ÐuÐ 1 (respectivement ÐPuÐ 1 ÐuÐ 1 ) pour tout u, u È L 1. On note D l ensemble des densités de probabilité sur E : D Øu È L 1 : u, ÐuÐ 1 1Ù.

36 6 Étude Théorique de Modèles Stochastiques 35 Ainsi un opérateur stochastique transforme une densité en une densité. Soit P: E E Ö, 1 un noyau de transition stochastique, c est-à-dire que PÔx, Õ est une mesure de probabilité pour tout x È E et la fonction x PÔx,BÕ est mesurable pour tout B È E. Soit P un opérateur stochastique sur L 1. Si E PÔx,BÕuÔxÕmÔdxÕ B PuÔyÕmÔdyÕ pour tous B È E,u È D, alors P est l opérateur de transition associé à P. Un opérateur stochastique P sur L 1 est dit partiellement intégral s il existe une fonction mesurable p: E E Ö, Õ telle que E E pour toute densité u. De plus, si, pôx,yõmôdyõmôdxõ et PuÔyÕ alors P correspond au noyau stochastique E PÔx,BÕ pôx,yõmôdyõ 1, x È E, B pôx,yõmôdyõ, x È E,B È E, E uôxõpôx, yõ môdxõ, et on dit que P est à noyau p. Dans le cas particulier d un ensemble dénombrable E avec E la famille de tous les sous-ensembles de E et m la mesure de comptage, l espace L 1 sera noté l 1 et les densités de probabilité sont des suites. Tout opérateur stochastique sur l 1 a un noyau ÖpÔx,yÕ x,yèe qui est donné par une matrice (stochastique). Un semi-groupe ØP ÔtÕÙ t d opérateurs linéaires sur L 1 est dit sous-stochastique (respectivement stochastique) s il est fortement continu et pour tout t l opérateur P ÔtÕ est sous-stochastique (respectivement stochastique). Une densité u est invariante ou stationnaire pour ØP ÔtÕÙ t si u est un point fixe de chaque opérateur P ÔtÕ, P ÔtÕu u pour tout t. Un semi-groupe stochastique ØP ÔtÕÙ t est dit asymptotiquement stable s il existe une densité stationnaire u telle que lim ÐP ÔtÕu u Ð 1 pour u È D, t et il est partiellement intégral si, pour un t, l opérateur P Ôt Õ est partiellement intégral. Théorème 1 ([67, Thm 2]). Soit ØP ÔtÕÙ t un semi-groupe stochastique partiellement intégral. Si le semi-groupe ØP ÔtÕÙ t a une unique densité invariante u et u presque partout, alors lim t ÐP ÔtÕu u Ð 1 pour tout u È D. Dans notreétudesurunmodèle donnéparunpdmp, il neserapas tropdifficile devoir que le semi-groupe est partiellement intégral. Les conditions pour obtenir un semi-groupe stochastique (autre que le cas trivial d intensités de saut bornées) sont plus délicates. Enfin, l existence d une densité invariante(c est-à-dire une fonction mesurable invariante et intégrable, qui peut donc être renormalisée) sera donnée par des calculs sur une résolvante et une chaîne de Markov échantillonnée, que l on présente plus bas. Pour s assurer que le semi-groupe donné par le générateur de l éq. (8) est stochastique, on utilisera un résultat de perturbation. Ce résultat permet d abord de construire un semigroupe sous-stochastique, généré par une extension du générateur associé à l éq. (8). De

37 36 Introduction Générale plus, il caractérise la résolvante de ce semi-groupe, ce qui permet de déduire des critères suffisants pour le rendre stochastique. On note A l opérateur de transport associé au terme de dérive, et J l opérateur stochastique sur L 1 associé au noyau Q. L équation d évolution sur la densité peut se réécrire du dt A u λu JÔλuÕ. A étant un opérateur de transport, il est raisonnable de penser qu il est le générateur infinitésimal d un semi-groupe stochastique fortement continu (du moins on peut trouver dans la pratique des conditions pour qu il le soit). Alors, même si λ est non bornée, A 1 u A u λu est le générateur d un semi-groupe sous-stochastique. Le domaine DÔA 1 Õ est inclus dans L 1 λ Øu È L1 : λôxõ uôxõ môdxõ Ù. E Soit A 2 JÔλuÕ. L opérateur J est positif et stochastique, ÐJÔλuÕÐ 1 ÐλuÐ 1, et donc DÔA 1 Õ DÔA 2 Õ. De plus, on a clairement ÔA 1 u A 2 uõdm. E On peut alors utiliser le résultat de perturbation suivant : Théorème 11 ([43, 86, 5]). Supposons que deux opérateurs linéaires ÔA 1,DÔA 1 ÕÕ et ÔA 2,DÔA 2 ÕÕ sur L 1 vérifient les hypothèses suivantes : ÔA 1,DÔA 1 ÕÕ génère un semi-groupe sous-stochastique ØS 1 ÔtÕÙ t ; DÔA 1 Õ DÔA 2 Õ et A 2 u pour tout u È DÔA 1 Õ ; pour tout u È DÔA 1 Õ, ÔA 1 u A 2 uõdm. E Alors il existe un semi-groupe sous-stochastique ØP ÔtÕÙ t sur L 1 généré par une extension C de ÔA 1 A 2,DÔA 1 ÕÕ. Le générateur est caractérisé par RÔσ,CÕu lim N RÔσ,A 1Õ Nô ÔA 2 RÔσ,A 1 ÕÕ n u, u È L 1, σ. n De plus, ØP ÔtÕÙ t est le plus petit semi-groupe sous-stochastique dont le générateur est une extension de ÔA 1 A 2,DÔA 1 ÕÕ. Enfin, les conditions suivantes sont équivalentes : ØP ÔtÕÙ t est un semi-groupe stochastique, le générateur C est la fermeture de ÔA 1 A 2,DÔA 1 ÕÕ, pour un σ, lim n ÐÔA 2RÔσ,A 1 ÕÕ n uð, u È L 1. Tyran-Kamińska [83] a montré qu une condition suffisante pour que ØP ÔtÕÙ t soit stochastique est que l opérateur K défini par Ku lim σ A 2 RÔσ,A 1 Õu lim σ JÔλRÔσ,A 1 ÕuÕ, (9) 1 soit ergodique en moyenne, c est-à-dire lim n n N 1 ô n K n u existe. Cette proposition vient simplement de la monotonie des résolvantes RÔσ,A 1 Õ d un opérateur sous-stochastique et

38 6 Étude Théorique de Modèles Stochastiques 37 du fait que l ergodicité en moyenne s hérite par domination. En pratique, on pourra donc chercher à montrer que K possède une unique densité invariante, transférer cette propriété à l opérateur ØP ÔtÕÙ t et utiliser le théorème 1 pour conclure. Pour finir, notons les liens entre l approche probabiliste et analytique sur les PDMP donnés par la proposition suivante Proposition 17. Tyran-Kamińska [83, thm 5.2] Soient XÔtÕ le PDMP de caractéristique locale ÔH,λ,QÕ, ØP ÔtÕÙ t son semi-groupe sur L 1 associé, J l opérateur stochastique sur L 1 associé au noyau Q, et φ t ÔxÕ le flot global associé à H. On note ÔT n Õ la suite de temps de sauts de XÔtÕ, avec T lim n T n le temps d explosion pour XÔtÕ. Alors : pour tous σ, lim ÔJÔλRÔσ,A 1ÕuÕÕ n 1 ØEÙ ÔxÕ E e σt XÔÕ x p.p. x. n pour tous B È BÔEÕ, u È DÔAÕ et t B P ÔtÕuÔxÕmÔdxÕ E P XÔtÕ È B,t T XÔÕ x uôxõmôdxõ, l opérateur K défini à l éq. (9) est l opérateur de transition associé à la chaîne de Markov en temps discret ÔXÔT n ÕÕ n de noyau KÔx,BÕ On conclut avec une série de remarques QÔB;φ t ÔxÕÕλÔφ t ÔxÕÕe t λôφrôxõõdr dt, x È E,B È BÔEÕ. Remarque 12. Cet ensemble de résultats montre que l on peut ramener l étude de l équation d évolution sur les densités du PDMP (en supposant que la loi initiale a une densité) à l étude des densités d un opérateur associé à une chaîne de Markov en temps discret. On verra dans le chapitre 1 que pour un modèle simple, on peut calculer explicitement la résolvante de A 1, l opérateur K, trouver un unique candidat pour la densité invariante, et ainsi donner des conditions assez fines (sur les caractéristiques locales du PDMP) pour la stabilité asymptotique du semi-groupe associé au PDMP. Les résultats de Tyran-Kamińska [83] contiennent d autres caractérisations importantes, notamment des conditions pour que le semi-groupe soit fortement stable (perte de masse) qui ont été appliquées à différents modèles de fragmentations (voir aussi [55]). Remarque 13. L étude d un processus de Markov par une chaîne de Markov en temps discret est à la base des idées de Meyn et Tweedie présentées dans la sous-section 6.3. Notons également que ces idées ont été appliquées sur les PDMP par Costa and Dufour [2]. L importance en pratique de ces résultats est de donner des opérateurs explicitement calculables, contrairement aux résolvantes (en général). Comme on l a vu à la soussection 6.3, l échantillonnage donné par des temps aléatoires exponentiels de paramètre 1 correspond exactement à la résolvante (éq. (6)). Cependant, celui-ci est difficilement calculable dans la pratique. L approche de Marta Tyran-Kamińska donne des conditions équivalentes (voir théorème 11) pour les propriétés du semi-groupe ØP ÔtÕÙ t sur L 1 et l opérateur A 2 RÔσ,A 1 Õ. Ensuite, l opérateur K lim σ A 2 RÔσ,A 1 Õ, qui correspond à un échantillonnage aux temps de saut du PDMP, donne des conditions suffisantes pour les propriétés de stabilité du semi-groupe ØP ÔtÕÙ t. L échantillonnage utilisé par Costa et Dufour (dans un cadre un peu plus général, avec bord, et avec une approche probabiliste, en regardant le semi-groupe sur les fonctions bornées) correspond à des temps aléatoires donnés par le minimum du temps de prochain saut et d une exponentielle de paramètre 1. Les auteurs obtiennent alors des conditions d équivalence entre les propriétés de stabilité de la chaîne échantillonée et du PDMP.

39 38 Introduction Générale Remarque 14. Enfin, ces approches de type «semi-groupe» pour étudier les propriétés de stabilité d un modèle donnent en général de mauvaises estimations sur les taux de convergence vers l état d équilibre. Pour obtenir de «bons» taux de convergence explicites, on utilise généralement des techniques dites de couplage. On renvoie à de récentes études sur des PDMP dans les articles [8],[19] par exemple. On verra au chapitre 1 que cette approche permet de trouver un taux de convergence explicite pour notre modèle. 7 Théorèmes Limites Les idées des théorèmes limites en probabilités reposent sur les deux théorèmes fondamentaux que sont la loi des grands nombres (LGN) et le théorème de la limite centrale (TCL). La LGN nous dit que si on somme un grand nombre n de variables indépendantes et identiquement distribuées, intégrables, et que l on divise par ce nombre n, alors la limite est déterministe, égale à la moyenne de la loi commune des variables aléatoires. Le TCL (pour des variables L 2 ) caractérise les fluctuations autour de la limite de la LGN, qui sont alors gaussiennes, centrées en la moyenne, de variance qui tend vers en n 1ß2. Ces théorèmes ont d innombrables applications et généralisations, en particulier aux processus stochastiques. Pour le processus stochastique qui nous intéressera le plus, le processus de Poisson, ces théorèmes se traduisent par la proposition suivante : Proposition 18. Soit Y un processus de Poisson standard (d intensité 1). Alors, pour tout t, lim sup Y ÔntÕ t, presque sûrement. ntt n De plus, P Y ÔntÕ nt lim n n x x 1 2π e y2 ßÔ2tÕ dy P W ÔtÕ x, où W est un mouvement Brownien standard (de moyenne nulle et de variance t). Pour ce qui nous intéresse, les conséquences et généralisations des ces théorèmes aux processus stochastiques ont principalement pour intérêt de trouver et justifier des modèles réduits et plus abordables analytiquement. On présente ci-après deux approches de réduction de modèles, l une basée sur la séparation d échelles de temps, et l autre basée sur des passages en grandes populations (champ moyen, limite fluide, limite thermodynamique...). Ces deux approches ne sont pas forcément disjointes. Mais tout d abord expliquons les outils principaux utilisés. L approche la plus largement répandue pour prouver des théorèmes limites sur des processus stochastiques, satisfaisant une certaine équation différentielle stochastique, repose sur des arguments topologiques, et notamment de compacité. Si une suite est relativement compacte, et possède une unique valeur d adhérence, alors cette suite est convergente, vers l unique valeur d adhérence. Notons que les convergences obtenues sur les processus stochastiques seront des convergences en loi. Les processus stochastiques (sur D E Ö, Õ en général) sont vus comme des variables aléatoires d un plus grand espace, que l on notera temporairement S, muni d une certaine topologie. Notons C b ÔSÕ l ensemble des fonctions continues bornées de S. Notons PÔSÕ l ensemble des mesures de probabilités sur S. Une suite P n È PÔSÕ de mesures de probabilités sur S converge faiblement vers P si lim n fdp n fdp, f È C b ÔSÕ.

40 7 Théorèmes Limites 39 De manière équivalente, une suite de variables aléatoires X n sur S converge en loi (ou en distribution) vers X si lim n E f ÔX n Õ E f ÔXÕ, f È C b ÔSÕ. Cette convergence n est pas spécifique aux processus stochastiques. Un autre type de convergence, beaucoup plus maniable, et spécifique aux processus stochastiques, est la convergence en distribution de dimension finie. Cette convergence est la convergence en loi de tout vecteur fini de variables aléatoires données par les évaluations du processus stochastique en des temps finis. La convergence de dimension finie peut être une manière d identifier une unique limite via le résultat de Prokhorov : Proposition 19. X n converge en loi vers X si et seulement si X n converge en distribution de dimension finie et X n est relativement compact. La preuve du sens direct de cette proposition utilise le théorème de représentation de Skorokhod, qui nous dit que si on a convergence en loi, alors on peut toujours trouver (représenter) des variables aléatoires qui ont ces lois et qui convergent presque sûrement. La preuve du sens réciproque utilise le fait que les distributions de dimension finie caractérisent un processus stochastique. Une deuxième méthode pour caractériser de manière unique la loi du processus limite, largement répandue, est celle du problème de martingale. Si l on montre que toute limite de la suite de processus stochastiques doit vérifier un certain problème de martingale, et qu on a unicité (en loi) de la solution du problème de martingale, alors la loi limite est caractérisée de manière unique. On comprend alors que le caractère bien posé (en fait l unicité) d un problème de martingale est crucial pour cette approche. On verra enfin au Chapitre 1 que l on peut utiliser dans certains cas une généralisation du théorème de Lévy, le théorème de Bochner-Minlos, qui montre que sous de bonnes conditions, la fonctionnelle caractéristique d un processus stochastique caractérise sa loi. Après avoir caractérisé la loi limite, la deuxième étape consiste à montrer la relative compacité du processus stochastique (dans l espace dans lequel il vit). Cette propriété dépend fortement de la topologie que l on considère. Une notion proche de la compacité pour les lois de probabilité, et très maniable en pratique, est la tension. Définition 16 (tension). Une suite de variables aléatoires X n à valeurs dans S un espace topologique est tendue si pour tout ε, il existe un compact K È S, tel que lim inf n P X n È K 1 ε. Le fameux théorème de Prohorov caractérise la relative compacité par des critères de tension uniformes. En particulier, on peut montrer que si S est un espace métrique complet séparable, une suite est tendue si et seulement si elle est relativement compacte (voir par exemple [3, thm 2.2]). Si X n est une suite de processus stochastiques à valeurs dans D E Ö, Õ, on cherche donc si cet espace est un métrique complet séparable. Si E est métrique complet séparable, alors on peut munir D E Ö, Õ d une métrique (appelé métrique de Skorohod) qui rende D E Ö, Õ complet séparable. De plus, pour cette topologie, notée S E, on a le critère de tension suivant trouvé par Aldous (voir par exemple [39, thm 4.5 p 356]) : Proposition 2. Une suite X n est tendue dans ÔD E Ö, Õ,S E Õ si :

41 4 Introduction Générale pour tous N È N, ε, il existe n È N et K tels que pour tous N È N, ε, on a Ôn n Õ P sup Xt n K ε. tn lim limsup θ n sup ST S θ P X n T Xn S ε, où le supremum est parmi tous les temps d arrêts adaptés à la filtration canonique associée à X n, bornés par N. Citons également, toujours pour la topologie de Skorohod, le critère de Rebolledo pour les semi-martingales de dimension finie Proposition 21. [41, Cor p 41] Si X n est à valeurs dans un espace de dimension finie, et X n A n M n, avec A n un processus à variation finie, M n une martingale locale L 2, et si les suites ÔA n Õ et Ô M n Õ (processus de variation quadratique) vérifient le critère d Aldous, alors X n est tendue. Il arrive que la suite de processus ne puisse être tendue dans ÔD E Ö, Õ,S E Õ, notamment lorsque le processus limite «a plus de discontinuités» que la suite de processus. Il faut alors utiliser d autres topologies, en s assurant que le théorème de Prohorov reste vrai (ainsi que le théorème de représentation de Skorokhod), pour pouvoir utiliser les mêmes arguments de compacité. C est le cas pour la topologie de Jakubowski J sur D R Ö,1, pour laquelle on a le critère de tension suivant : Proposition 22. Une suite X n est tendue dans ÔD R Ö,1,JÕ si pour tout ε, il existe n È N et K tels que pour tous a b, il existe C tel que Ôn n Õ P sup Xt n K ε, t1 supn a,b ÔX n Õ Cn, n où N a,b est le nombre de croisements de niveau a b. Un critère similaire est valable pour l espace L p Ö,1, 1 p : Proposition 23. Une suite X n est tendue dans L p Ö,1 si pour tous N È N, ε, il existe n È N et K tels que Ôn n Õ P sup Xt n K ε, t1 pour tout ε, il existe n È N et K tels que Ôn n Õ P ÐX n t Ð BV K ε, où ÐxÐ BV ÐxÐ 1 supø i fôt i 1Õ fôt i Õ, t i subdivision de Ö,1 Ù. Enfin, si MÖ, Õ est l espace des fonctions réelles mesurables sur Ö, Õ, muni de la métrique dôx,yõ O e t maxô1, xôtõ yôtõ Õdt, alors ÔMÖ, Õ,dÕ est un espace métrique séparable, et on a le critère de tension suivant :

42 7 Théorèmes Limites 41 Proposition 24. [52, thm 4.1] Une suite X n est tendue dans ÔMÖ, Õ,dÕ si : pour tous T,ε, il existe K tel que T Pour tout T sup n 1 ØxÔtÕKÙ ε, T lim sup maxô1, xôt hõ xôtõ Õdt. h n 7.1 Réduction de modèles par séparation d échelles de temps Les théorèmes limites sont très importants dans le contexte des modèles de réactions biochimiques. En effet, il est courant que dans ces modèles certaines variables ou certaines réactions évoluent à une vitesse beaucoup plus rapide que les autres. Dans ces cas là, on peut soit «simplifier» la réaction (elle peut devenir déterministe, ou provoquer des grands sauts) ou «éliminer» la variable rapide par des techniques de moyennisation. On renvoie à deux récentes publications utilisant ce genre de techniques pour simplifier des processus de saut pur [21],[42], ainsi qu aux résultats du chapitre 1 sur la simplification du modèle d expression des gènes. Les techniques de moyennisation remontent à Kash minski et Kurtz (voir par exemple [5]). De manière heuristique, elles sont basées sur l hypothèse que la variable rapide est ergodique, et donc converge rapidement vers son état d équilibre. La variable lente, si elle dépend de la valeur de la variable rapide, ne dépendra alors à la limite que des moments asymptotiques de la variable rapide. On utilisera ces techniques de réduction dans les deux chapitres de cette thèse, soit pour prouver rigoureusement des liens entre certains modèles, soit pour réduire la dimension d un modèle et le rendre plus facile à analyser. Des techniques de réduction similaires peuvent être effectuées directement sur l équation d évolution de la densité des variables (Équation maîtresse ou Fokker-Planck) en «intégrant» sur la variable rapide, et par une hypothèse d ergodicité similaire. Voir pour cette approche [38] ou plus récemment [73]. 7.2 Réduction par passage en grande population Lorsqu on a un modèle discret, qui évolue par de petits sauts, si l on suppose que le nombre d individus à l état initial devient grand, alors par une renormalisation appropriée, on peut décrire le nombre d individus par une variable continue qui vérifiera un modèle limite. Cette idée remonte à Prokhorov [68] et Kurtz [51]. Pour une chaîne de Markov X n en temps continu à valeurs dans N, dont l évolution est décrite par des intensités de saut λ n ÔxÕ et une loi de répartition de saut µ n Ôx, Õ, le résultat classique de Kurtz [51] nous dit que si on accélère les intensités de saut par λ n ÔxÕ nλôxõ, et que l on ne change pas la loi de répartition de saut µ n Ôx, Õ µôx, Õ, alors le processus stochastique renormalisé Y n Xn n converge vers la solution de l équation différentielle ordinaire (sous réserve qu elle soit bien posée) dirigée par F ÔxÕ λôxõ R z x µôx,dzõ. Ces techniques ont été étendues à de nombreux modèles de population en biologie. La stratégie est de décrire un modèle de population discrète en utilisant des processus ponctuels (la mesure empirique), et de prouver qu ils convergent, avec une mise à l échelle

43 42 Introduction Générale adéquate et de bonnes hypothèses sur les coefficients, vers une mesure qui résout un certain problème limite. La convergence obtenue est une convergence en loi, et les preuves utilisent généralement les techniques de martingales (on montre d abord la compacité, et ensuite que toute limite est uniquement déterminée, grâce au problème de martingale). Ces idées remontent à Prokhorov [68], et ont été considérablement améliorées par de nombreux auteurs [51, 63, 41, 81, 71, 53, 24]. Les intérêts de cette approche sont : premièrement, théorique. Cette approche peut être utilisée pour prouver l existence d une solution au problème limite. Si on est capable de trouver un modèle discret particulier, qui possède une suite de solutions qui converge, et dont la limite résout nécessairement le problème limite, alors on a prouvé l existence d une solution du modèle limite ( voir par exemple [4, 62] dans le contexte de modèle d agrégationfragmentation); deuxièmement, numérique. Cette approche a été largement utilisée pour obtenir des algorithmes rapides et efficaces d un modèle continu non linéaire, comme les nombreuses variantes des équations de Poisson-McKean-Vlasov [82]. Pour une telle approche, le taux de convergence du modèle stochastique vers le modèle limite est important pour s assurer de la tolérance de l approximation réalisée [16, 61]; troisièmement, pour la modélisation. Dans un contexte physique ou biologique, cette approche permet de justifier rigoureusement les bases et les hypothèses physiques d un modèle particulier. En effet, dans les modèles de population discrets, on peut spécifier précisément chaque réaction ou les règles d évolution de la population. Ensuite, avec des hypothèses sur les coefficients décrivant cette évolution, et une mise à l échelle particulière (explicite, en général grande population, ou taux de réactions rapides, etc...), on obtient un modèle limite ou un autre. Ainsi, les hypothèses (parfois) implicites d un modèle continu sont rendues plus explicites. On peut aussi unifier certains modèles en les reliant entre eux avec des mises à l échelle particulières [44]; enfin, du point de vue pratique. Cette approche peut être utilisé pour simplifier des modèles, en particulier quand les effets discrets rendent l analyse du modèle délicate. On peut obtenir une bonne idée du comportement d un modèle initial en étudiant plusieurs comportements limites. Récemment, les approches de type «théorèmes limites» appliquées aux modèles de population en biologie mathématique ont été nombreuses, donnant un changement de point de vue à la modélisation en biologie, d une approche macroscopique à une approche microscopique. On peut donner des exemples concrets : dans les modèles de population cellulaire. Bansaye et Tran [6] ont considéré une population de cellules infectées par des parasites (le nombre de parasites donne une variable de structure pour les cellules) et ont regardé la limite quand il y a un grand nombre de parasites et une taille finie de population de cellules. On peut faire des analogies entre ce modèle et le modèle de polymérisation-fragmentation que l on étudiera au chapitre 2. On peut considérer en effet les polymères comme des cellules, et les monomères comme des parasites. On utilisera ainsi les résultats de ce papier, et on considérera aussi la limite quand le nombre de petites particules (monomères, parasites) devient grand tandis que le nombre de grandes particules (polymères, cellules) reste fini, et évolue suivant une fragmentation (ou division) aléatoire. Pour d autres études similaires de modèles hôtes-parasites, voir [7, 57]. dans les modèles d évolution. Champagnat et Méléard [17] ont étendu les modèles d évolutions (où la population est structurée par un «trait» génotypique, qui subit des mutations) avec interaction (voir [31, 18]) en rajoutant une structure d espace, typiquement une diffusion réfléchie sur un domaine borné. Les auteurs ont ainsi

44 BIBLIOGRAPHIE 43 obtenu, dans la limite de grandes populations, une équation aux dérivées partielles non- linéaire de type réaction-diffusion, avec condition au bord de Neumann. Leurs hypothèses impliquent que les taux de naissance et mort, et les coefficients de dérive et de diffusion soient bornés et Lipschitziens, pour s assurer du caractère bien posé du modèle limite. Nous utiliserons aussi cet article dans le chapitre 2, pour modéliser le système d agrégation-fragmentation de polymères avec mouvement spatial. Enfin, mentionnons juste que les approches de ce type sur l équation d évolution de la densité, très utilisées par les physiciens, portent souvent le nom d expansion de Van Kampen, ou de Kramers-Moyal (voir par exemple [72]). Bibliographie [1] D. J. Aldous. Deterministic and stochastic models for coalescence (aggregation and coagulation) : a review of the mean-field theory for probabilists. Bernoulli, 5(1) :3 48, [2] D. F. Anderson. A modified next reaction method for simulating chemical systems with time dependent propensities and delays. J. Chem. Phys., 127 :1 1, [3] D. F. Anderson and T. G. Kurtz. Design and Analysis of Biomolecular Circuits (chapter 1). Springer, [4] J M Ball, J Carr, and O Penrose. The Becker-Döring Cluster Equations : Basic Properties and Asymptotic Behaviour of Solutions. Commun. Math. Phys., 14(4) : , [5] J. Banasiak. On an extension of the kato-voigt perturbation theorem for substochastic semigroups and its application. Taiwanese J. Math., 5 : , [6] V. Bansaye and V. C. Tran. Branching Feller diffusion for cell division with parasite infection. ALEA-Lat. Am. J. Probab., 8 :81 127, [7] V. Bansaye, J.-F. Delmas, L. Marsalle, and V. C. Tran. Limit theorems for Markov processes indexed by continuous time Galton-Watson trees. Ann. Appl. Probab., 21 (6) : , [8] J.-B. Bardet, A. Christen, A. Guillin, F. Malrieu, and P.-A. Zitt. Total variation estimates for the tcp process pre-print arxiv : , 38 [9] A. F. Bartholomay. On the linear birth and death processes of biology as markoff chains. B. Math. Biophys., 2 :97 118, [1] P. H. Baxendale. A stochastic hopf bifurcation. Probab. Th. Rel. Fields, 99 : , [11] R. Becker and W. Döring. Kinetische Behandlung der Keimbildung in übersättigten Dämpfen. Ann. Phys. (Berlin), 416(8) : , [12] J. Bérard. Chaînes de markov. http ://math.univ-lyon1.fr/ jberard/notes-cmwww.pdf, [13] N. Blinov, M. Berjanskii, D. S. Wishart, and M. Stepanova. Structural domains and main-chain flexibility in prion proteins. Biochemistry, 48(7) : , 29. 1

45 44 Introduction Générale [14] R. P. Boland, T. Galla, and A. J. Mckane. How limit cycles and quasi-cycles are related in systems with intrinsic noise. J. Stat. Mech. Theor. Exp., P91 :1 27, [15] P. Brémaud. Markov Chains. Springer, , 2, 21, 22, 24 [16] E. Cepeda and N. Fournier. Smoluchowski s equation : rate of convergence of the Marcus-Lushnikov process. Stoch. Proc. Appl., 121(6) :1 34, [17] N. Champagnat and S. Méléard. Invasion and adaptive evolution for individual-based spatially structured populations. J. Math. Biol., 55(2) : , [18] N. Champagnat, R. Ferrière, and S. Méléard. Individual-based probabilistic models of adaptive evolution and various scaling approximations. Prog. Probab., 59 :75 113, [19] B. Cloez. Wasserstein decay of one dimensional jump-diffusions eprint arxiv : [2] O.L.V. Costa and F. Dufour. Stability and ergodicity of piecewise deterministic markov processes. IEEE Decis. Contr. P., 47 : , [21] A. Crudu, A. Debussche, A. Muller, and O. Radulescu. Convergence of stochastic gene networks to hybrid piecewise deterministic processes. Ann. Appl. Probab. (to appear), , 41 [22] D.J. Daley and D. Vere-Jones. An introduction to the Theory of Point Processes. Springer series in statistics, [23] M. H. A. Davis. Piecewise-deterministic markov processes : A general class of nondiffusion stochastic models. J Roy. Stat. Soc. B Met., 46(3) : , [24] D. Dawson, B. Maisonneuve, and J. Spencer. Measure-valued markov processes, volume Springer Berlin / Heidelberg, [25] R. Durrett. Probability : Theory and Examples. Cambridge U. Press, [26] M. Ehrenberg, J. Elf, E. Aurell, R. Sandberg, and J. Tegnér. Systems Biology Is Taking Off. Genome Res., 13 : , [27] A. Eldar and M. B. Elowitz. Functional roles for noise in genetic circuits. Nature, 467(7312) :167 73, [28] M.B. Elowitz, A.J. Levine, E.D. Siggia, and P.S. Swain. Stochastic gene expression in a single cell. Science, 297 : , [29] K.-J. Engel and R. Nagel. One-parameter semigroups for linear evolution equations. Springer-Verlag, [3] S. N. Ethier and T. G. Kurtz. Markov Processes : Characterization and Convergence. Wiley Interscience, , 26, 27, 28, 29, 34, 39 [31] N. Fournier and S. Méléard. A microscopic probabilistic description of a locally regulated population and macroscopic approximations. Ann. Appl. Probab., 14(4) : ,

46 BIBLIOGRAPHIE 45 [32] C. Gadgil, C. H. Lee, and H. G. Othmer. A stochastic analysis of first-order reaction networks. B. Math. Biol., 67 :91 946, [33] D. T. Gillespie. Exact stochastic simulation of coupled chemical reactions. J. Chem. Phys., 81(25) : , [34] B. C. Goodwin. Oscillatory behavior in enzymatic control processes. Adv. Enzyme. Regul., 3 : , [35] J. S. Griffith. Nature of the scrapie agent : Self-replication and scrapie. Nature, 215 (515) : , [36] J.S. Griffith. Mathematics of cellular control processes. I. Negative feedback to one gene. J. Theor. Biol., 2 :22 28, [37] J.S. Griffith. Mathematics of cellular control processes. II. Positive feedback to one gene. J. Theor. Biol., 2 :29 216, [38] H Haken. Synergetics : An introduction. Springer-Verlag, [39] J. Jacod and A. N.Shiryaev. Limit Theorems for Stochastic Processes. Springer Verlag, , 28, 39 [4] I. Jeon. Existence of Gelling Solutions for Coagulation- Fragmentation Equations. Commun. Math. Phys., 567 : , , 42 [41] A. Joffe and M. Metivier. Weak convergence of sequences of semimartingales with applications to multitype branching processes. Adv. Appl. Probab., 18(1) :2 65, , 42 [42] H.-W. Kang and T. G. Kurtz. Separation of time-scales and model reduction for stochastic reaction networks. Ann. Appl. Probab. (to appear), [43] T. Kato. On the semigroups generated by kolmogoroff s differential equations. J. Math. Soc. Japan, 6 :1 15, [44] M. Kimmel and O. Arino. Comparison of approaches to modeling of cell population dynamics. SIAM J. Appl. Math., 53(5) : , [45] M. Kimmel and D.E. Axelrod. Branching Processes in Biology. Springer-Verlag, [46] H. Kitano. Looking beyond the details : a rise in system-oriented approaches in genetics and molecular biology. Curr. Genet., 41(1) :1 1, [47] J J Kupiec. A darwinian theory for the origin of cellular differentiation. Mol. Gen. Genet., 255 :21 28, [48] J.-J. Kupiec, M. Morange, M. Silberstein, and O. Gandrillon. Le hasard au coeur de la cellule. Les Éditions Materiologiques, 29. ISBN [49] A. Kurakin. Self-organization vs watchmaker : stochastic gene expression and cell differentiation. Dev. Genes Evol., 215 :46 52, [5] T. G. Kurtz. Averaging for martingale problems and stochastic approximation. Applied Stochastic Analysis, 177 :186 29,

47 46 Introduction Générale [51] T. G. Kurtz. Solutions of Ordinary Differential Equations as Limits of Pure Jump Markov Processes. J. Appl. Probab., 7(1) :49 58, , 42 [52] T. G. Kurtz. Random time changes and convergence in distribution under the meyerzheng conditions. Ann. Probab., 19(3) :11 134, [53] T G Kurtz. Approximation of population processes. CBMS-NSF, [54] J.-P. Liautard, M.-T. Alvarez-Martinez, C. Feraudet, and J. Torrent. La protéine prion : structure, dynamique et conversion in vitro. M S-Med. Sci., 18(1) :62 69, , 12 [55] M. C. Mackey and M. Tyran-Kamińska. Dynamics and density evolution in piecewise deterministic growth processes. Ann. Polon. Math., 94(2) : , [56] D. A. McQuarrie. Stochastic approach to chemical kinetics. J. Appl. Probab., 4(3) : , [57] S. Méléard and S. Roelly. A host-parasite multilevel interacting process and continuous approximations. pages 1 31, 211. arxiv : [58] S. P. Meyn and R. L. Tweedie. Stability of markovian processes i : Criteria for discrete-time chains. Adv. Appl. Probab., 24(3) : , September , 3 [59] S. P. Meyn and R. L. Tweedie. Stability of markovian processes ii : Continuous-time processes and sampled chains. Adv. Appl. Probab., 25(3) : , , 3, 31 [6] S. P. Meyn and R. L. Tweedie. Stability of markovian processes iii : Foster-lyapunov criteria for continuous-time processes. Adv. Appl. Probab., 25(3) : , , 3, 31, 32 [61] S Mischler, C Mouhot, and B Wennberg. A new approach to quantitative chaos propagation estimates for drift, diffusion and jump processes. pages 1 45, 211. arxiv : [62] J R Norris. Smoluchowski s coagulation equation : uniqueness, nonuniqueness, and a hydrodynamic limit for stochastic coalescent. Ann. Appl. Probab., 9(1) :78 19, , 42 [63] K. Oelschlager. A Martingale Approach to the Law of Large Numbers for Weakly Interacting Stochastic processes. Ann. Appl. Probab., 12(2) : , [64] H. G. Othmer and C. H. Lee. A multi-time-scale analysis of chemical reaction networks : I. deterministic systems. J. Math. Biol., 6(3) :387 45, [65] H.G. Othmer. The qualitative dynamics of a class of biochemical control circuits. J. Math. Biol., 3 :53 78, [66] A. Paldi. Stochastic gene expression during cell differentiation : order from disorder? Cell. Mol. Life. Sci., 6(9) :1775 8, September [67] K. Pichór and R. Rudnicki. Continuous Markov semigroups and stability of transport equations. J. Math. Anal. Appl., 249(2) : , [68] Y. V. Prokhorov. Convergence of random processes and limit theorems in probability theory. Theor. Probab. Appl+, I(2), , 42

48 BIBLIOGRAPHIE 47 [69] S. B. Prusiner. Prions. Proc. Natl. Acad. Sci., 95(23) : , [7] A. Raj and A. Van Oudenaarden. Single-Molecule Approaches to Stochastic Gene Expression. Annu. Rev. Biophys., 38 :255 27, [71] S Roelly and A Rouault. Construction et propriétés de martingales des branchements spatiaux interactifs. Int. Stat. Rev., 58(2) : , [72] F. Sagués, J. Sancho, and J. García-Ojalvo. Spatiotemporal order out of noise. Rev. Mod. Phys., 79(3) : , July [73] M. Santillán and H. Qian. Irreversible thermodynamics in multiscale stochastic dynamical systems. Phys Rev E Stat Nonlin Soft Matter Phys., 83 :1 8, [74] K. R. Schenk-Hoppé. Stochastic hopf bifurcation : an example. Int. J. Non-linear Mechanics, 31(5) : , [75] Erwin Schrödinger. What Is Life? Cambridge University Press, [76] J.F. Selgrade. Mathematical analysis of a cellular control process with positive feedback. SIAM J. Appl. Math., 36 : , [77] K Singer. Application of the theory of stochastic processes to the study of irreproducible chemical reactions and nucleation processes. J Roy. Stat. Soc. B Met., 15(1) : , [78] D. W. Stroock and S. R. S. Varadhan. Multidimensional diffusion processes. Springer- Verlag, , 28 [79] D. M. Suter, N. Molina, D. Gatfield, K. Schneider, U. Schibler, and F. Naef. Mammalian genes are transcribed with widely different bursting kinetics. Science, 332(628) : 472 4, [8] N. Symonds. What is life? : Schrodinger s influence on biology. Q. Rev. Biol., 61(2) : , [81] A.-S. Sznitman. Topics in Propagation of Chaos, volume Springer, [82] V. C. Tran. Modèles particulaires stochastiques pour des problèmes d évolution adaptative et pour l approximation de solutions statistiques. PhD thesis, University Paris 1 Nanterre, [83] M. Tyran-Kamińska. Substochastic semigroups and densities of piecewise deterministic Markov processes. J. Math. Anal. Appl., 357(2) :385 42, , 37 [84] J. J. Tyson. On the existence of oscillatory solutions in negative feedback cellular control processes. J. Math. Biol., 1(4) : , [85] M. Vellela and H. Qian. On the poincaré- hill cycle map of rotational random walk : locating the stochastic limit cycle in a reversible schnakenberg model. P. Roy. Soc. A-Math. Phy., pages, [86] J. Voigt. On substochastic c -semigroups and their generators,. Transport theory Statist. Phys., 16 : ,

49

50 Chapter 1 The bursting phenomenon as a jump Markov Process 49

51 5 Hybrid Models to Explain Gene Expression Variability 1 Introduction In neurobiology, when it became clear that some of the fluctuations seen in whole nerve recording, and later in single cell recordings, were not simply measurement noise but actual fluctuations in the system being studied, researchers very quickly started wondering to what extent these fluctuations actually played a role in the operation of the nervous system. Much the same pattern of development has occurred in cellular and molecular biology as experimental techniques have allowed investigators to probe temporal behavior at ever finer levels, even to the level of individual molecules [11, 147]. Experimentalists and theoreticians alike who are interested in the regulation of gene networks are increasingly focused on trying to access the role of various types of fluctuations on the operation and fidelity of both simple and complex gene regulatory systems. Recent reviews [74, 19] give an interesting perspective on some of the issues confronting both experimentalists and modelers. Among the increasing number of paper that demonstrate stochasticity in gene expression, at the single cell level, we can quote the work of Elowitz et al. [34], who have used an elegant experimental technique to prove inherent as well as environmental stochasticity. In their work, they measure at a single cell level two different gene reporters that has equal probability to be expressed. They quantify the difference between cells and through time of the total amount of expression of both genes, as well as the difference of proportion of expression of one gene among the two. Their results clearly demonstrate variability coming from the environment as well as coming from intrinsic stochastic event inside cells. In this chapter, we deal with a model of a single gene, that is able to self-regulate its own expression. We model the dynamics of the level of expression of this gene in a single cell, without taking into account cell division. This model has been extensively used and studied in the last decades with different representations and approximations (see section 7 for a review). The aim of this minimal model is to study stochasticity in gene expression together with non-linear effect. Its advantage relies in the ability to obtain analytic results and quantitative prediction (see section 8). Recent improvements in molecular biology allow to identify and to measure precisely the level of gene expression in very small gene network, including single gene network (see the next subsection 2.1). For more complex (in the sense of large) gene network, such approach can be used as a building block to understand nonlinearity and stochasticity in higher network. Even in model of a single gene, the number of steps can vary considerably depending on the level of description chosen. We consider here a model that includes 4 steps, namely the state of the gene, the transcription, the translation and effector production. Again, improvements of molecular biology tend to identify more and more elementary steps and some model intend to take into account a more precise level of description, up to the nucleotide (see subsection 7.9). Finally, the model we consider is a purely dynamical model, and we don t consider any spatial or delay effect (even though, it is clear now that intracellular environment is not well-mixed, and that some processes inside cells take an incompressible time to proceed). Our choice of level of description allows us to include the pioneer work of Goodwin [48] together with the important recently discovered switching and bursting effect in gene expression (these terms will be made clearer in the following). The Goodwin [48] model focuses on describing the time evolution of the concentration of gene product (mrna, protein), based on the molecular basics found earlier. For describing the time evolution of a continuous variable, It is usually used an ordinary differential equation approach. When it becomes clear that the evolution of concentration of gene product in single cells could

52 1 Introduction 51 not be described by deterministic laws, one then starts to consider stochastic description. In order to take into account stochasticity, it can be used a Langevin equation (additive noise) or more generally stochastic differential equation (multiplicative noise) with either Gaussian white noise (no time correlation) or Gaussian colored noise (with positive time correlations, see Shahrezaei et al. [131]). However, in this latter representation, the variable still evolves continuously. Whereas it has been well documented experimentally [22, 47, 111, 15] that in some organisms the mrna and/or protein production is intermittent, and intense during relatively short periods of time. This phenomenon is called bursting in molecular biology. The accuracy of experiments permits to characterize the time interval between these production events, and permits to quantify the amount of molecules produced in a single burst event. In particular, in the work referred above, it has been found that in some organisms the bursting production is characterized by an exponential waiting time between production events, and the burst size is exponentially distributed as well. To reproduce such characteristics, it has recently been proposed (Friedman et al. [39], Mackey et al. [91]) to use a stochastic differential equation driven by a compound Poisson white noise, to model explicitly the discontinuous and stochastic production. Such a process can also be viewed as a piecewise-deterministic Markov process. The mathematical foundation of piecewise-deterministic Markov processes (PDMP) was given by Davis [27]. This class of stochastic process unifies deterministic processes described by ordinary differential equation, and pure jump Markov processes, described by a Markov chain. Such a class of model has found recently an important echo in mathematical biology, since it allows to take into account different dynamics into a single model (Hespanha [6]). The work of Davis [27] shows how we can use the martingale machinery to study such stochastic processes. All the tools available to study convergence of stochasticprocesses (Ethier andkurtz[36]) can thenbeusedtostudylimiting behaviorof PDMP. Two recent papers of Crudu et al. [25] and Kang and Kurtz [75] illustrate this approach, and explore various limiting cases using time-scale separation in the context of molecular reaction network. On another approach, PDMP brings new evolution equations on densities, which are typically of integro-differential types (as opposed to second-order partial differential equations associated to diffusion processes). Here, we will make use extensively of the semigroup approach to study long-time behavior of such equation, following the work of Lasota and Mackey [83], Mackey and Tyran-Kamińska [9], and Tyran-Kamińska [145]. In such approach, existence and stability of an invariant density is given by the existence and uniqueness of a solution to a fixed point problem (which presents itself as a system of algebraic equations or differential equations in our examples), associated to a discrete-time Markov chain. We can compare this approach to more traditional results in stochastic process given by Meyn and Tweedie [97], and recent contributions on convergence results of PDMP by Costa and Dufour [23]. On the other hand, the molecular basis for stochasticity in gene expression is also often attributed to low copy numbers of gene products. It is then needed to use discrete variable models rather than continuous one, and to model molecular number rather than concentration. Such ideas are widely used in biochemistry since the work of Gillespie [45]. The recent contribution of Anderson and Kurtz [3] summarizes the foundation and mathematical formulation of such models, as continuous-time Markov processes. All the different models considered here use different mathematical formulations, namely pure jump Markov process in a discrete state space, continuous state space ordinary differential equation and hybrid models. We will attach an important part to prove these different formulations relate to each other through rigorous limit theorem (see sections 9). In particular, it s quite remarkable that the so-called central dogma of molecular biology

53 52 Hybrid Models to Explain Gene Expression Variability (as a chemical reaction network) can explain much of the different experimental observed behaviors, in different parameter space regions. But first, it is important to emphasize the biochemical reaction network that is behind all these different mathematical formulations, and give some background material in molecular biology (see sections 2, 3 and 4). Once this is set up, we describe our model through a pure jump Markov process in a discrete state space and studie its qualitative behavior (section 5). Then we present its continuous deterministic version, namely the Goodwin model (section 6) and recall how we can precisely study its long time behavior. A review of (many) other linked or intermediate model is provided in section 7. Then we present an analogous study of the Goodwin model, on a stochastic reduced model (section 8), where we only keep one variable. We consider in detail the probability distribution of the molecular number (with a discrete variable) or concentrations (hence, with a continuous state variable) in generic bacterial operons in the presence of bursting using an analytical approach. As stated above, our work is motivated by the well documented production of mrna and/or protein in stochastic bursts in both prokaryotes and eukaryotes [22, 47, 111, 15], and follows other contributions by, for example, [14, 77, 39, 13, 129]. All the above mentioned work share common goal, that is to find analytic characterization of a particular stochastic gene expression model, to be able to deduce kinetic parameters from experimental observations and/or to explain qualitatively and quantitatively the amount of variability measured experimentally. It is important to also recognize the pioneering investigation of Berg [9] who first studied the statistical fluctuations of protein numbers in bacterial population (with division) through the master equation approach, and introduced the concept of what is now called bursting. The analytical solution of the steady state density of the molecular distributions in the presence of bursting was first derived by Friedman et al. [39]. Our work extends these results to show the global stability of the limiting densities and examine their bifurcation structure to give a rather complete understanding of the effect of bursting on molecular distributions. The originality of this work is then to give a bifurcation diagram for the stochastic model of gene expression, in complete analogy with the deterministic Goodwin model. As molecular distributions can now be estimated experimentally in single cells, such theoretical framework may also be of importance in practice. We show in section 8.6 how one can estimate the regulation function (rather than a single parameter) using an inverse problem approach ([29]). Such estimate may be of importance to understand detail molecular interactions that determines the regulation function (see section 3). It has been the subject of a published work (Mackey et al. [91]). Finally, our framework can be extended to a discrete variable model (see subsection 8.1), and we also investigated the fluid limit (subsection 9.4), which will be the subject of a further publication (Mackey et al. [93]). The fact that this one-dimensional bursting model relies on fundamental molecular basis of previously known mechanism in molecular biology is an important feature of this model, and has been noticed by many authors (see for instance [12] for review). Following recent theoretical contributions on reduction of stochastic hybrid system [25, 75] we rigorously prove that one limiting behavior of the (now) standard model of molecular biology gives a bursting model (see subsection 9.1 and 9.2) In our work, we can prove slight generalization of such reduction, in order to understand the key feature associated with such behavior. We also prove an adiabatic reduction for this bursting model (see subsection 9.3), which will be the subject of a further publication (Mackey et al. [92]). This work justifies the use of a reduce one-dimensional model when some variables are evolving with a fast time scale, in a context of a continuous state hybrid model. The originality of our work is to provide alternative proofs, using either partial differential

54 2 Standard Model 53 equation techniques or probabilistic techniques. Up to our knowledge, adiabatic reduction for stochastic differential equation with jumps hasn t been investigated before. 2 Standard Model 2.1 Background in molecular biology The so-called central dogma of molecular biology, based on the Nobel Prize winning work of Jacob et al. [69] in which they introduced the concept of the operon (see subsection 2.2), is simple to state in principle, but complicated in its detail. Namely through the process of transcription of DNA, messenger RNA (mrna) is produced and, in turn, through the process of translation of the mrna, proteins ( or intermediates) are produced. There is often feedback in the sense that molecules (enzymes) whose production is controlled by these proteins can modulate the translation and/or transcription processes. In what follows we will refer to these molecules as effectors (see figure 1.1). Rather astonishingly, within a few short years of the publication of the ground breaking work of Jacob et al. [69] the dynamics of this simple feedback system was studied mathematically by [48]. His formulation of the operon concept is now known as the Goodwin model. We now consider both the transcription and translation processes in detail. We first present these two processes in prokaryotes, and then explain the main differences with eukaryotes. In the transcription process an amino acid sequence in the DNA is copied by an enzyme called RNA polymerase (RNAP) to produce a complementary copy of the DNA segment encoded in the resulting RNA. Thus this is the first step in the transfer of the information encoded in the DNA. The process by which this occurs is as follows. When the DNA is in a double stranded configuration, the RNAP is able to recognize and bind to the promoter region of the DNA. (The RNAP/double stranded DNA complex is known as the closed complex.) Through the action of the RNAP, the DNA is unwound in the vicinity of the RNAP/DNA promoter site, and becomes single stranded. The RNAP/single stranded DNA is called the open complex. Once in the single stranded configuration, the transcription of the DNA into mrna commences. A lot of interactions between proteins can promote or block the closed complex formation and its binding to the promoter region of the DNA. These proteins that interact with the RNAP are called transcription factor (TF). There are many different known interactions between TF and DNA and RNAP. Some TF can stabilize or block the bindingof RNA polymerase to DNA. They can also recruit coactivator or corepressor proteins to the DNA complex, in order to increase or decrease the rate of gene transcription. In eukaryotes, TF can make the DNA more or less accessible to RNA polymerase by modifying physically its configuration. Obviously, when these TF interact with the DNA that controls its production, then they coincide with the molecules we called above effectors. The interaction between effectors and the DNA and RNAP polymerase then dictates the feedback mechanism (see section 3) and are responsible for what is called the transcriptional regulation or the gene expression regulation. All these interactions are supposedly sequence-specific meaning that specific proteins will be able to bind to specific sequence of DNA, or to specific other proteins. These concepts are however unreliable [8]. In prokaryotes, translation of the newly formed mrna starts with the binding of a ribosome to the mrna. The function of the ribosome is to read the mrna in triplets of nucleotide sequences (codons). Then through a complex sequence of events, initiation and elongation factors bring transfer RNA (trna) into contact with the ribosome-mrna complex to match the codon in the mrna to the anti-codon in the trna. The elongating peptide chain consists of these linked amino acids, and it starts folding into its

55 54 Hybrid Models to Explain Gene Expression Variability final conformation. This folding continues until the process is complete and the polypeptide chain that results is the mature protein. Although there are also many interactions between proteins at the step of translation, there are much less studies reporting for posttranscriptional regulation (see[72] that consider mrna degradation regulation mechanism and post-transcriptional regulator binding). The situation in eukaryotes differs from 2 main things. Firstly, the DNA is found in a structure that is called chromatin. The exact structure ofthechromatinismuchout ofthescopehere, andwecan keepinmindthatthechromatin packs the DNA in a smaller volume. Also, the chromatin prevents the DNA to be easily accessible. Sequence of DNA can be more or less packed, depending on the gene. The state of the chromatin (more or less packed) may also varies during time, leading to a very complex dynamics. This dynamic modification of chromatin(called chromatin remodeling) may be the result of interactions with enzymes and transcription factors (but would not be considered here). Secondly, mrna molecules are synthesized inside the nucleus, whereas the ribosomes are located outside the nucleus. Then proteins will be synthesized outside the nucleus, and will have to enter the nucleus to interact with the DNA. These facts usually lead to consider higher delays in the transcription/translation process modeling in eukaryotes than in prokaryotes. Our framework was conceived for gene expression model in bacteria (prokaryotes). However, a growing number of people argue that similar models can be used for both prokaryotes and eukaryotes, in different parameter space regions (see subsection 2.4). (a) Central Dogma (b) New Central Dogma Figure 1.1: Schematic illustration of the so-called central dogma of molecular biology. (a) Messenger RNA (mrna) are produced through the transcription of DNA, and proteins are produced through the translation of mrna. There is a feedback directly by proteins (or effectors) that can control the transcription of DNA. (b) Similar of the left panel, except that the DNA can enter in an OFF state for which transcription is not possible. 2.2 The operon concept An operon is a piece of DNA containing a cluster of genes under the control of a single promoter. The genes are transcribed together into mrna. These mrna are either translated together or separately in the cytoplasm. In most cases, genes contained in the operon are then either expressed together or not at all. Several genes must be both co-transcribed and co-regulated to define an operon. Operons were first discovered in prokaryotes but also exist in eukaryotes. From the experimental and modeling point of view, operons that contain a regulatory gene (repressor or activator) are very key concepts because they provide a very small regulatory gene network. Most famous operon are The lactose (lac) operon ([135]) in bacteria is the paradigmatic example of this concept and this much studied system consists of three structural genes named lacz,

56 2 Standard Model 55 lacy, and laca. These three genes contain the code for the ultimate production, through the translation of mrna, of the intermediates β-galactosidase, lac permease, and thiogalactoside transacetylase respectively. The enzyme β-galactosidase is active in the conversion of lactose into allolactose and then the conversion of allolactose into glucose. The lac permease is a membrane protein responsible for the transport of extracellular lactose to the interior of the cell. (Only the transacetylase plays no apparent role in the regulation of this system.) The regulatory gene laci, which is part of a different operon, codes for the lac repressor. The latter is transformed to an inactive form when it binds with allolactose. Hence, in this system, allolactose acts as the effector molecule. See figure 1.2. The tryptophan (trp) operon was also extensively studied ([58],[123],[89]). Tryptophan is an amino acid that is incorporated into proteins that are essential to bacterial growth. When tryptophan is present in the growth media, it forms a complex with the tryptophan repressor and the complex binds to the promoter of the trp operon, effectively switching off production of tryptophan biosynthetic enzymes. In the absence of tryptophan, the repressor cannot bind to the promoter and the essential tryptophan biosynthetic enzymes are produced. See figure 1.4. The bacteriophage λ system was reviewed recently ([96],[57], [58]). It is a small piece of viral DNA that encode for two proteins (ci and cro) that are mutually antagonist. When a virus infects a bacteria like E. Coli, experiments show that the system exhibits bistability. The system can be in two distinct states. Each state implies a different behavior for the cell. In one state (called lysogenic), the virus lies dormant, and is replicated only with the bacteria. In the other state, the virus expresses proteins that are able to replicate the virus itself, then lyse (kill) the host cell and release its progeny. 2.3 Synthetic network The ability of design synthetic constructed gene network, reviewed by Hasty et al. [58], provides also an excellent tool for modeling and experimental purposes. Approaches with coupled modeling/experiments were indeed used to design specific small circuits with the desired properties (bistability, oscillations etc...). Amongst the most popular synthetic networks, one can find: the genetic toggle switch, such as the λ-switch (Gardner et al. [43]). It consists of two genes that encode for proteins that are co-repressive. It has been experimentally demonstrated that this system displays bistability. the Repressilator. It consists of a loop of three genes. Each one inhibits successively the next gene ([33]). It has been experimentally demonstrated that this system can display oscillations. Synthetic positive autoregulatory gene ( tet-r system, [8], or λ-phage system [67]). It has been experimentally shown that this system displays bistability. Obviously, it has also some interest on its own (cellular control, biotechnology, genetically engineered microorganisms and so on). 2.4 Prokaryotes vs Eukaryotes models Although the quite important differences between prokaryotes and eukaryotes, it has been argued several times in the past that the standard stochastic model of gene expression

57 56 Hybrid Models to Explain Gene Expression Variability is a priori suitable for both ([95, 115, 46]). The rate constants and the meaning of the stochastic transition can be different though. In particular, the On/Off switching rate of the gene state (see figure 1.1b) on prokaryotes will usually reflect binding and unbinding events of molecule on the promoter or even pausing of RNA polymerase, while the On/Off switching rate on eukaryotes will reflect opening-closing of chromatin. Indeed, we saw that the presence of nucleosomes and the packing of DNA-nucleosome complexes into chromatin generally make promoters inaccessible to the transcriptional machinery. Transition between open and closed chromatin structures then correspond to active and inactive (repressed) promoter states, and can be fairly slow ([74],[19],[115]) compared to the dynamics of binding and unbinding event of molecules at the promoter region in prokaryotes. We refer to table 1.2 for some parameter values taken from literature. 3 The Rate Functions From what we presented above, it should be clear now that the transcription rate (and the translation rate) is function of many cellular components, and specially protein numbers/concentrations. Some modeling approaches take into account many details and many variables in order to reflect faithfully the transcription process (see subsection 7.9 for a brief review). However, these approaches increase drastically the number of parameters and the dimension of the model. With some kinetic assumptions, it is possible to reduce the complexity. The justification of it is an important stage of modeling. We detail here some classical derivation of the transcriptional regulation in the deterministic context, and (non-so) classical derivation in the stochastic context. There have been very different mechanisms (for a review in prokaryotes see [154], in yeast [53] and in higher eukaryotes [118]) proposed for the molecular basis of the regulation of the transcription rate by effector molecules. These mechanisms also depends a lot of the system considered. We focus on one particular system (feedback through complex formation) for simplicity. Depending on the model in consideration (eukaryotes or prokaryotes in particular), the feedback mechanism can be involved at different stages (activation/inactivation of the gene, or initiation of the transcription). During transcription initiation, the reversible binding of an RNAP to the promoter region and subsequent formation of an open complex achieve rapid equilibrium: initiation from the final open complex is the rate-limiting step ([142]). Transcription initiation is therefore assumed to be a pseudo-first-order reaction with rate linearly proportional to the amount of RNAP. In this section we examine the molecular dynamics of both the classical inducible and repressible operon [148] to derive expressions for the dependence of the transcription rate on effector levels. In this view, the effectors first interact with other molecules (repressors) to form a molecular complex. These interactions will modify the binding/unbinding event of repressors on the DNA, and then modify the binding/unbinding event of RNAP to the promoter region of the DNA. The effector molecules can also act by binding directly on to the promoter region and shielding it from RNAP. In all cases, the reactions with effector are considered to be in equilibrium and simply change the fraction of RNAP bound as a closed complex, thereby changing the effective transcriptional rate. See [12] and [148] for experimental evidence that such approach reproduces accurately the rate function.

58 3 The Rate Functions Transcriptional rate in inducible regulation For a typical inducible regulatory situation (such as the lac operon), in the presence of the effector molecule the repressor is inactive (is unable to bind to the operator region preceding the structural genes), and thus DNA transcription can proceed (see figure 1.2). Let R denote the repressor, E the effector molecule, and O the operator. We assume that the effector binds with the active form R of the repressor to form a complex RE n. This reaction is of the form R ne k c k RE n, (3.1) c where n is the effective number of molecules of effector required to inactivate the repressor R. Furthermore, the operator O and repressor R are assumed to interact according to O R k b k OR. (3.2) b Finally, the transcription takes place when RNAP binds the free operator O, thereby leading to the reaction O RNAP k M O RNAP M, (3.3) where M denotes the mrna. The goal of this section is to derive the effective rate of production of M in function of the effector molecules as the binding dynamics between effectors, repressors and operators quickly reach equilibrium. We first present the standard way to derive this rate, using ordinary differential equation, and then using stochastic differential equation. for simplicity, we do not include at his point the fact that effector molecules are constantly degraded and produced. Hence its total level will change over time. However, these variations will occur on a slower time scale than operator fluctuations, so that it won t change the reduction performed here. Figure 1.2: Figure taken from Wikipedia. Schematic illustration of the lac operon, an inducible operon. Top: Repressed, Bottom: Active. 1: RNA Polymerase, 2: Repressor, 3: Promoter, 4: Operator, 5: Lactose, 6: lacz, 7: lacy, 8: laca. In presence of lactose, the repressor is unable to bind to the bind to the operator, and RNA polymerase can proceed.

59 58 Hybrid Models to Explain Gene Expression Variability Deterministic description The set of chemical reactions (3.1)-(3.2)-(3.3) can be described by the following system of ODE (using standard chemical kinetics argument) ³² ³± x R k c x R x n E k c x RE n k b x O x R k b x OR, x E nk c x R x n E nk c x RE n, x REn k c x R x n E k c x RE n, x O k b x O x R k b x OR, x OR k b x O x R k b x OR, x M k M x O x RNAP, (3.4) where x entities denotes the concentration of the given biochemical entities. Note that the three following quantities are conserved through time: the total amount of operator O tot : the total amount of repressor R tot : the total amount of effector E tot : x Otot x O x OR. x Rtot x R x REn x OR. x Etot x E nx REn. We define the equilibrium rate constants K b k b k and K c k c b k. We now make specific c assumptions on reaction rates to prove the following Proposition 15. Assume the kinetic reaction rate constants satisfies Hypothesis 1. k M k c,k c,k b,k b, and the total quantity of repressors and effectors are such that Hypothesis 2. K c x Rtot x n 1 E tot 1. Then, the effective mrna production rate is a function of x Etot, given by k M k 1 Ôx Etot Õ, where if x Rtot 1, 1 K c x n E k 1 Ôx Etot Õ x RNAP x tot Otot, (3.5) K b x Rtot while if x R tot x Otot 1, k 1 Ôx Etot Õ x RNAP x Otot 1 K c x n E tot 1 K b x Rtot K c x n E tot. (3.6) Proof. By hypothesis 1, the reaction(3.3) occurs at a much slower rate than reactions(3.1)- (3.2). We then modify the last equation of eq. (3.4) on x M by x M εk M x O x RNAP,

60 3 The Rate Functions 59 where ε 1. On the slow time scale τ εt, it is a standard result [143, 38] that the fast dynamics approaches its equilibrium value as ε. The slow manifold associated is given by the system of algebraic equations ³² ³± x R 1 K c x n E K b x O x Rtot, x O 1 K b x R x Otot, x E nk c x R x n E x E tot. Now hypothesis 2 makes this system tractable, because the last equation becomes x E x Etot and the above system reduced to ² ± x R 1 K c x n E tot K b x O x Rtot, x O 1 K b x R x Otot. (3.7) It is easy to show that this system of equations has a unique strictly positive solution (it can be transformed to a second order polynomial equation), and that this solution is globally stable for the fast dynamics. Although this solution is rather complicated (as a functionof theparameters), ithas twoimportantasymptotic expressions. Whenx Rtot 1, the expression of x O has the following leading term x O x Otot 1 while when x R tot x Otot 1, the expression of x O reads K c x n E tot K b x Rtot, x O x Otot 1 K c x n E tot 1 K b x Rtot K c x n E tot. Considering that x RNAP is constant, the effective mrna production rate is then, on the slow time scale, k M k 1 Ôx Etot Õ, where in the first case, while in the second case, k 1 Ôx Etot Õ x RNAP x Otot 1 K c x n E tot K b x Rtot, 1 K c x n E k 1 Ôx Etot Õ x RNAP x tot Otot 1 K b x Rtot K c x n. E tot In both cases, there will be maximal repression when E but even then there will still be a basal level of mrna production (which we call the fractional leakage). In the first case, the production rate of mrna is unbounded with the level of effector, while it is bounded in the second case. For biological motivation, the second expression eq. (3.6) is rather used. However equation 3.5 is sometimes used with n 1 (linear regulation).

61 6 Hybrid Models to Explain Gene Expression Variability Stochastic description We can also describe the set of chemical reactions (3.1)-(3.2)-(3.3) by the following system of SDE (using standard chemical kinetics argument) ³² ³± t Å XE ÔsÕ X R ÔtÕ X R ÔÕ Y 1 k c X R ÔsÕ ds Y t 1 k c X REn ÔsÕds n t Y 2 k b X O ÔsÕX R ÔsÕds Y t 2 k b X ORÔsÕds, t Å XE ÔsÕ X E ÔtÕ X E ÔÕ ny 1 k c X R ÔsÕ ds ny t 1 k c n X RE n ÔsÕds, t Å XE ÔsÕ X REn ÔtÕ X REn ÔÕ Y 1 k c X R ÔsÕ ds Y t 1 k c n X RE n ÔsÕds, t X O ÔtÕ X O ÔÕ Y 2 k b X O ÔsÕX R ÔsÕds Y t 2 k b X ORÔsÕds, t X OR ÔtÕ X OR ÔÕ Y 2 k b X O ÔsÕX R ÔsÕds Y t 2 k b X ORÔsÕds, t X M ÔtÕ X M ÔÕ Y 3 k M X O ÔsÕX RNAP ÔsÕds, where X entities denotes the number of the given biochemical entities, and Å XE ÔsÕ X EÔsÕÔX E ÔsÕ 1Õ ÔX E ÔsÕ n 1Õ. n n! (3.8) Ineq.(3.8), Y i, i 1,2,3 referstoindependentunitpoissonprocesses, thatareassociated to reactions (3.1)-(3.2)-(3.3). For instance, Y 1 (respectively Y 1 ) gives the successive instant the forward (respectively the backward) reaction (3.1) fires. Note that the three following quantities are again conserved through time: the total amount of operator O tot : the total amount of repressor R tot : the total amount of effector E tot : X Otot X O X OR, X Rtot X R X REn X OR, X Etot X E nx REn. We now make specific assumptions on reaction rates to prove the following Proposition 16. Assume the kinetic reaction rate constants satisfies hypothesis 1 and that the following scaling holds as N, Hypothesis 3. X N E ÔÕ Nα, k c Nnα, for some α. We assume furthermore that ZE NÔÕ XN E ÔÕ N α Z Etot, lim N ZN E ÔÕ Z Etot. is such that it exists

62 3 The Rate Functions 61 Then, as N,the solution XM N ÔtÕ of eq. (3.8) converges to the solution of X M ÔtÕ X M ÔÕ Y 3 t k M E X O ÔsÕXRNAP ÔsÕds, where E X O ÔsÕ is the asymptotic first moment of XO on the fast dynamics given by reactions (3.1)-(3.2), and is given by E 1 K c Z X O E n tot XOtot 1 K b X Rtot K c ZE n. (3.9) tot Proof. By hypothesis 1, the reaction(3.3) occurs at a much slower rate than reactions(3.1)- (3.2). We then modify the last equation of eq. (3.4) on X M by X M ÔtÕ X M ÔÕ Y 3 t εk M X O ÔsÕX RNAP ÔsÕds, where ε 1. The fast dynamics consist of a closed system on a finite state space (due to mass conservation constraint) and its associated Markov chain is irreducible, so that it has a unique stationary distribution. By the averaging theorem (see [75, thm 5.1]), on the slow time scale, the dynamics can then be reduced to X M ÔtÕ X M ÔÕ Y 3 t k M E X O ÔsÕXRNAP ÔsÕds, where E X O ÔsÕ is the asymptotic first moment of XO on the fast dynamics, and is a function of K b,k c,x Rtot ÔsÕ,X Etot ÔsÕ and X Otot ÔsÕ. Its exact expression is out of reach, but we can derive analogous result as in the deterministic case. With hypothesis 3, we define ZE N XN E N and rewrite the fast system as (with a slight abuse of notation) α XR N t ÔtÕ X R ÔÕ Y 1 N nα k c XR N ÔsÕZE N Ô1 OÔ 1 N α ÕÕds ³² ³± Y t 1 N nα k c XRE N n ÔsÕds t Y 2 k b XO N ÔsÕXN R ÔsÕds Y t 2 k b XN OR, ÔsÕds ZE N t ÔtÕ ZN E ÔÕ nn α Y 1 N nα k c X R ÔsÕZE N 1 Ô1 OÔ N α ÕÕds nn α Y t 1 N nα k c XRE N n ÔsÕds, XRE N t n ÔtÕ X REn ÔÕ Y 1 N nα k c XR N ÔsÕZE N Ô1 OÔ 1 N α ÕÕds Y t 1 N nα k c XN RE n ÔsÕds, XO N t t ÔtÕ X O ÔÕ Y 2 X N ORÔtÕ X OR ÔÕ Y 2 t k b XO N ÔsÕXR N ÔsÕds k b XO N ÔsÕXR N ÔsÕds Y 2 Y 2 t k b XN ORÔsÕds, k b XN ORÔsÕds. With this scaling, the variable XR N, ZN E and X RE n then evolve at a faster time scale than XO N and XN OR, so that the averaging theorem again tells us that, at the limit N, t X O ÔtÕ X O ÔÕ Y 2 k b X O ÔsÕE X R ds Y 2 t k b ÔX O tot X O ÔsÕÕds,

63 62 Hybrid Models to Explain Gene Expression Variability so that immediately E X O Ôt Õ X Otot 1 K b E X R. To find the latter quantity E X R we look at the time scale tn Ôn 1Õα. Let then γ Ôn 1Õα. We define Z N,γ E ÔtÕ ZN E ÔtNγ Õ and similarly X N,γ R and XN,γ RE n. The fast system defined by reaction (3.1) becomes ³² ³± X N,γ t R ÔtÕ X RÔÕ Y 1 Nα k c X N,γ Y t 1 N α k c XN,γ RE n ÔsÕds, Z N,γ R ÔsÕZN,γ E t E ÔtÕ ZN E ÔÕ nn α Y 1 Nα k c XN,γ nn α Y t 1 N α k c X N,γ RE n ÔsÕds, X N,γ t RE n ÔtÕ X REn ÔÕ Y 1 Nα k c X N,γ Y t 1 N α k c XN,γ RE n ÔsÕds. 1 Ô1 OÔN ÕÕds α R ÔsÕZN,γ E R ÔsÕZN,γ E Define now Z N,γ RE n N α X N,γ RE n that satisfies the equation Z N,γ RE n ÔtÕ ZRE N n ÔÕ N α t Y 1 N α k c X N,γ so that lim N ZN,γ RE n ÔtÕ, lim N α X N,γ N lim N t RE n, k c X N,γ R ÔsÕZN,γ R ÔsÕZN,γ E E ÔsÕds t Assuming that lim N Z N E ÔÕ Z E tot, we obtain finally lim N ZN,γ E ÔtÕ Z E tot. 1 Ô1 OÔN ÕÕds α 1 Ô1 OÔN ÕÕds α Ô1 OÔ 1 N α ÕÕds N α Y 1 t k c X N,γ RE n ÔsÕds. N 2α k c Z N,γ RE n ÔsÕds, so that at this time scale, Z N,γ E is constant and contains the whole quantity of effector molecules. Stillatthistimescale, X N,γ RE n andx N,γ R arefastvaryingvariable, whosebehavior is best captured by the occupancy measure V N,γ R ÔC Ö,tÕÕ t 1 ØCÙ ÔX N,γ R ÔsÕÕds. For any bounded function f, the following quantity is a Martingale where t fôx N,γ R ÔtÕÕ fôxn,γ R ÔÕÕ Nα N C Z N,γfÔx R ÕV N,γ R Ôdx R dsõ, E C Z N,γfÔx R Õ k c x R Z N,γ E ÔfÔx R 1Õ fôx R ÕÕ k c ÔX R tot x r ÕÔfÔx R 1Õ fôx R ÕÕ. E

64 3 The Rate Functions 63 Dividing by N α, we see that its limiting measure must be solution of N t C ZEtot fôx R ÕV R Ôdx R dsõ. then V R has a binomial law of parameter ÔX Rtot, 1 1 K cz n E tot Õ. Taken all together, E X O Ôt Õ XOtot 1 K c ZE n tot 1 K b X Rtot K c ZE n, tot which is then the analog result of the deterministic description. Remark 17. Note that with the scaling we have assumed, K c X Rtot X n 1 E tot N α 1. The scaling we chose also implies that complex formation reaction occurs at a faster time scale than Repressor-Operator binding reaction. These arguments can then be used to derive operator switching rate function as a function of the effector level. We illustrate our results on figure 1.3, by calculating with a standard stochastic algorithm the statistical asymptotic mean values of X for the subsystem of reaction (3.1)-(3.2). As the scaling parameter N increases, the average values of X O, as a function of Z Etot, become closer and closer of the eq. (3.9). We also show the similar behavior of the deterministic solution of the non-linear system eq. (3.7). Remark 18. Other scalings can of course yield similar result, for instance X E N α, k c N nα, would produce another tractable limiting behavior..9 X Z Etot X Z Etot (a) n 1 (b) n 2 Figure 1.3: Numerical values of the first moment of the free operator variable X O, as a function of the effector level Z Etot. In both figures, the black lines are given by the Hill function, eq. (3.9), the dotted red lines are the numerical solution of the eq. (3.7), and the red points are the numerical mean value of X given by the system of reaction (3.1)- (3.2). Parameters are: (a) n α 1, k c k b 1,k b 1, X Otot 1, X Rtot 1, k c N nα, X Etot N α, and from down to top, N 1,1,1. (b) n 2, α 1, k c k b 1,k b 1, X O tot 1, X Rtot 1, k c Nnα, X Etot N α, and from down to top, N 1,5,1.

65 64 Hybrid Models to Explain Gene Expression Variability 3.2 Transcriptional rate in repressible regulation In the classic example of a repressible system (such as the trp operon), in the presence of effector molecules the repressor is active (able to bind to the operator region), and thus block DNA transcription (see figure 1.4). We use the same notation as before, but now note that the effector binds with the inactive form R of the repressor so it becomes active. We assume that this reaction is of the same form as in eq. (3.1). The difference now is that the operator O and repressor R are assumed to interact according to Figure 1.4: Figure taken from [123]. Schematic illustration of the Tryptophan operon, a repressible operon. In presence of Trp, the repressor is active and able to bind to the operator, which prevents RNA polymerase to bind. O R E n k b k ORE n. b Similar argument as above yields the following transcription rate function. We only state the deterministic result for simplicity. Proposition 19. Assume the kinetic reaction rate constants satisfies hypothesis 1 and that Hypothesis 4. K c x Rtot x n 1 E tot Ô1 K b x Õ 1. Then, the effective mrna production rate is a function of x Etot, given by k M k 1 Ôx Etot Õ,

66 3 The Rate Functions 65 parameter inducible repressible Λ 1 K b x Rtot K b x Rtot λ 1 k M x RNAP x Otot Table 1.1: Definition of the parameters Λ,, used in eq. (3.12), as a general case of eq. (3.6) (see subsection 3.1) and eq. (3.11) (see subsection 3.2). where if x Rtot 1, x Otot Ô1 K c x n E k 1 Ôx Etot Õ x tot Õ RNAP x Rtot K b K c x n, (3.1) E tot while if x R tot x Otot 1, k 1 Ôx Etot Õ x RNAP x Otot 1 K c x n E tot 1 Ô1 K b x Rtot ÕK c x n E tot. (3.11) 3.3 Summary The two bounded (above and below) functions given at eq. (3.6) and eq. (3.11) are most commonly used and are special cases (up to a proportional constant) of the function k 1 Ôx Etot Õ 1 K cx n E tot Λ K c x n E tot (3.12) where Λ, are given in table 1.1. We will lump all constants of proportionality that appeared previously in the derivation of the transcriptional rate function into a single parameter, thatwenameλ 1. Thetwounboundedfunctionsgivenateq.(3.5) andeq.(3.1) lead to ill-posed model, except eq. (3.5) for n 1 which has been used in the past. Itis also importanttobear inmindthat suchratefunctionsareverymodel-specific and various different form appeared in the literature, depending on the molecular dynamics considered (for a review in prokaryotes see [154], in yeast [53] and in higher eukaryotes [118]). We provide in table 1.2 some classical parameters found on the literature relevant for suchmodels. This table is not meant to beexhaustive, butto give intuition of the order of magnitude of the relevant process we look at, as well as the variation of the parameters rate one can found on different organism. Hence, the derivation of the Hill kinetics we provide might not always be justified (which explain partially the success of the on-off model which consider fluctuations at the level of the operator). In particular, we can see that for the lac operon [135] or the tryptophan operon [89] the association equilibrium constant is extremely small, making the derivation above safe, while it is not so the case for the phage λ system [67] or the TetR system [3]. Also, in the lac operon or the tryptophan operon, complex constant are scarce, but binds efficaciously the promoter. We also give some examples of number of molecules for the molecule in consideration (binding sites, RNA polymerase, ribosomes, repressor molecules) to show that in some cases, a probabilistic modeling is natural as the number of molecules is relatively small. We also highlight the fact that new experimental techniques are now used to follow individual molecules, and to characterize for example the search time of transcription factor for its binding sites!

67 66 Hybrid Models to Explain Gene Expression Variability Table 1.2: Parameters involved in the determination of the rate function. See subsections 3.1 and 3.2) for details. Note that we give all parameter values in molecule numbers, as they are required for stochastic modeling. For typical cells like E. Coli, 1 molecule per cell corresponds roughly ([142]) to a concentration of 1 nanomolar (nm) Parameters Complex formation binding constant Association Dissociation Equilibrium References and comments Large variation of order of magnitude of these rates relies on the fact that many different complexes can be involved in the interaction with promoter k c k c (min 1 ) (min 1 ) K c k c k c [135] LacI dimer (repressor) bindingto Effector molecule in the lac operon. (Fast dimerization of repressors is assumed) [3] atc binding with TetR to prevent TetR repression.5 [67] Dimer formation (λ repressor protein) in the phage λ system. Value taken from literature [89]TryptophanOperoninE.Coli. Values inferred from literature. Complex/Promoter binding constant Again large variation of order of magnitude reflects the diversity of the system considered. Experimentalist may also have the possibility to control affinity rate on promoter. Association Dissociation Equilibrium k b k b (min 1 ) (min 1 ) K b k b k b [135] LacI dimer repression by binding to the operator, in the lac operon. Taken from experimental data available on literature.

68 3 The Rate Functions [3] Direct repressor protein TetR binding to operator and other complex binding [67] Dimer (λ repressor protein) binding to the operator, in the phage λ system. Value taken from Literature. n n 1 3 [142]λ repressor protein binding to the operator, in the phage λ system, for a cooperativity constant of n. Value taken from literature [144] teta protein binding to teto promoter, in the tet-off system in S. cerevisiae. The response curve is measured experimentally and fitted to obtain kinetic parameter. n 1 2 [12] phage λ system in E. Coli. The rate of transcription is directly measured with the concentration of effector. The kinetic parameters are deduced by fitting [89]TryptophanOperoninE.Coli. Values inferred from literature. Complex affinity (Hill coefficient) n 1 3 [142] Typical biological values taken from literature. 1 [144] teta protein binding to teto promoter, in the tet-off system in S. cerevisiae. The response curve is measured experimentally and fitted to obtain kinetic parameter [12] phage λ system in E. Coli. The rate of transcription is directly measured with the concentration of effector. The kinetic parameters are deduced by fitting. 1.2 [89] Tryptophan Operon in E. Coli. Values taken from literature Number of binding sites 2-6 [144] tet-off system in S. cerevisiae Number of RNA polymerase [78] Bacteria 125 [89] E. Coli 36 [3] E. Coli 3 [113] Mammalian macrophage RNA polymerase binding constant Note that many authors consider this reaction to be responsible of the switching behavior of the gene state.

69 68 Hybrid Models to Explain Gene Expression Variability Association Dissociation Equilibrium λ λ a λ a i λ i (min 1 ) (min 1 ) [3] The promoter strength can be varied experimentally, and influence the RNA polymerase association constant [78] LacZ gene in in the Lac Operon. Values taken from literature 1 2 [89]TryptophanOperoninE.Coli. Values inferred from literature. Number of ribosomes [78] Bacteria 14 [89] E.Coli [113] Mammalian macrophage Ribosome binding constant Association Dissociation Equilibrium (min 1 ) (min 1 ) [78] Association rate given by diffusionlimited aggregation, and dissociation to reproduce translation rate faithfully 1 2 [89]TryptophanOperoninE.Coli. Values inferred from Literature. Number of Repressor molecules 5 [89] Tryptophan Operon in E. Coli. 1 [135] Repressors dimer in Lac Operon in E. Coli. Effective Diffusion constant (µm 2.min 1 ) 24 [32] Single Transcription factor detection in single cells, E Coli. Search time (min) 1 6 [32] Single Transcription factor detection in single cells, E Coli. Cell Volume (L) [89],[135] E. Coli [113] Mammalian macrophage 3.4 Other rate functions In the standard model, only the steps before (and including) the transcription usually consider nonlinear effect. In prokaryotes, ribosomes can begin binding the newly synthesized ribosome-binding site (on the mrna) almost immediately as transcription begins (whereas in eukaryotes, a delay between translation and transcription may be relevant). Analogous to transcript initiation, translation initiation of a single mrna molecule is assumed to proceed with a first-order rate λ 2. We assumed that initiation and elongation rates are such that ribosome queuing does not occur (Thattai and van Oudenaarden[142]). We therefore take each transcription and translation initiation reaction to be independent, and the translation rate would be proportional to the amount of mrna molecules. Simi-

70 4 Parameters and Time Scales 69 larly, we assume effector production rate to be proportional to the amount of intermediate protein molecules (with coefficient λ 3 ). Finally, we assume that all molecules degrade linearly with rates γ i, i 1,2,3 for mrna, proteins and effector respectively. A decay rate γ gives a half-life of lnô2õßγ. If growth in cell volume is exponential, the resulting dilution of species concentrations can be incorporated by increasing γ for all species (other than the DNA, which is replicated at a rate exactly matching cell growth). The mrna decay rate depends on the ribosome-binding rate, because actively translating ribosomes shield the mrna molecules from the action of nuclease (Thattai and van Oudenaarden [142]). 4 Parameters and Time Scales We summarize in table 1.3 the parameters used in our model, and the various range of magnitude that have been measured or fitted from experiments. Again, this table does not intend to be exhaustive, but rather to give intuitions. It is also clear that many parameters are not independent within each other, and their values then depend on the model chosen. For instance, an observation of the instantaneous rate of production of an mrna, as a first step process, or combined with an observation of the gene state kinetics, would not lead to the same transcriptional rate. The mean number of molecules, and burst statistics given at the end of this table, are also obviously function of other parameters. They can however be measured directly. For instance, as individual molecules can be measured, the authors in [2, 47, 15, 111] were able to count the number of molecules produced in each burst production event, and to deduce statistics of the burst size event. As a general trend, it can be noticed that synthesis rate of protein are usually higher than synthesis rate of mrna, while degradation rate of protein are several order of magnitude lower. Switching rate of the gene state are highly variable, but may be quite slow. Finally, the number of mrna molecules may be of only dozens, while there may have thousands or more proteins. Table 1.3: Parameters involved in the standard model of molecular biology. Note that we give all parameter values in molecule numbers, as they are required for stochastic models. For typical cells like E. Coli, 1 molecule per cell corresponds roughly [142] to a concentration of 1 nanomolar (nm) Activation rate Parameters Gene state Inactivation rate References and comments These values depend a lot on modeling choice. As we saw, transcription is a multi-step process. Activation of the gene may mean that an mrna Polymerase is bound to DNA, and then (almost) ready to start transcription. We may also consider that activation requires a(rare) transcription factor to bound. Or in eukaryotes it may requires chromatin opening.

71 7 Hybrid Models to Explain Gene Expression Variability λ a λ i (min 1 ) (min 1 ) [3] TetR system in E. Coli. The promoter strength can be varied experimentally, and influence the RNA polymerase association constant [144] tet-off system in S. cerevisiae [78] Lac operon in Bacteria [94] Interleukin protein in Lymphocytes. These rates represent opening/closing of chromatin, and were derived by fitting a stochastic model to experimental data [152]. Parameters inferred from experimental data using single mrna detection technique in yeast (S. Cerevisiae).2.1 [47] Real-time monitoring of lac/ara promoter kinetics in E. Coli [111] statistical kinetics inferred from single mrna counting in mammalian cells. mrna Synthesis rate Degradation rate Transcriptional efficiency λ λ 1 γ 1 1 λ i (min 1 ) (min 1 ) [3] TetR system in E. Coli [135] Lac operon in E. Coli. Taken from experimental data available on Literature 1.61 [89] Tryptophan Operon in E. Coli. Values inferres from literature [144] tet-off system in S. cerevisiae [113] Mammalian Macrophage [78] Lac operon in Bacteria [94] Interleukin protein in Lymphocytes. Experimentally deduced [152]. Parameters inferred from experimental data using single mrna detection technique in yeast (S. Cerevisiae) [127] global gene quantification in mammalian cells (mouse fibroblast) [111] Single mrna counting in mammalian cells. Protein Synthesis rate Degradation rate Transcriptional efficiency

72 4 Parameters and Time Scales 71 λ λ 2 γ 2 2 γ 1 (min 1 ) (min 1 ) [3] TetR system in E. Coli. Protein degradation rate equal the dilution rate [135] Lac operon in E. Coli [89] Tryptophan Operon in E. Coli. Protein degradation rate equal the dilution rate [144] tet-off system in S. cerevisiae [113] Mammalian Macrophage [78] Lac operon in Bacteria [94] Interleukin protein in Lymphocytes. Experimentally deduced [127] global gene quantification in mammalian cells (mouse fibroblast) Effector Synthesis rate Degradation rate λ 3 γ 3 (min 1 ) (min 1 ) [89] Tryptophan Operon in E. Coli. Effector degradation rate equal the dilution rate Mean Number mrna Protein X 1 X [135] Lac operon in E. Coli [113] Mammalian Macrophage [94] Interleukin protein in Lymphocytes. Experimentally deduced [152]. Parameters inferred from experimental data using single mrna detection technique in yeast (S. Cerevisiae) [127] global gene quantification in mammalian cells (mouse fibroblast) Mean Burst size mrna Protein 8-2 [2] Real-time monitoring of β- galactosidase in E. Coli. Their direct measurement also coincide with distribution fitting of a bursting model.

73 72 Hybrid Models to Explain Gene Expression Variability 4 [47] Real-time monitoring of lac/ara promoter kinetics in E. Coli 4.2 [15] T st -Venus protein controlled by the lac promoter in E. Coli. 1-3 [111] Single mrna counting in mammalian cells. Mean Burst frequency mrna Protein (min 1 ) (min 1 ) 1 3 [2] Real-time monitoring of β- galactosidase in E. Coli. Their direct measurement also coincide with distribution fitting of a bursting model [15] T st -Venus protein controlled by the lac promoter in E. Coli..2 [22] Real-time monitoring of a developmental gene in a small eukaryotes. 5 Discrete Version Based on the description above (section 2), we select 4 biochemical species involved in different chemical reactions, namely DNA, mrna, proteins and effectors. The simplest discrete stochastic description of this system is a continuous time Markov chain, with the state space being the number of each molecules of each species (or the state ON/OFF for the DNA we assume that there is a single DNA molecule), and with state transition given by the biochemical reactions (the stoichiometry of the reaction gives the state space jump, and its reaction rate gives the intensity of the jump). There are several equivalent representations of a continuous time Markov chain with discrete state space (see Introduction, part ). We present below the transition function of this Markov chain, and its generator. Then we deduce immediate consequences for the long-term behavior of this model. 5.1 Representation of the discrete model We now write for convenience X ÔX,X 1,X 2,X 3 Õ for the state of the Markov chain, with X being the state of the DNA, and X 1,X 2,X 3 respectively the numbers of mrna, proteins and effectors. Then the state space of the chain is Ø,1Ù N 3. The one-step transitions are summarized in table 1.4. Note that some reactions are catalytic reactions, that is they do not consume any species. Transition rates (or propensities) associated to first order reactions (degradation and catalytic) are derived according to the Action-Mass law and are then linear with respect to one variable. The other transition rates (k 1,k i,k a ) were derived in the previous section 3 and can be non-linear functions of the variable X 3. More detailed assumption on these rate functions will be given in the following. Let us introduce the following notation to simplify the writing. Notation 1. For any function f ÔxÕ with x Ôx,x 1,x 2,x 3 Õ, we define the following

74 5 Discrete Version 73 Table 1.4: Transitions and Parameters used for the pure jump Markov process X ÔX,X 1,X 2,X 3 Õ Biochemical Reaction State-space change vector Propensity Gene activation Ô1,,,Õ λ a 1 ØX Ùk a ÔX 3 Õ Gene inactivation Ô 1,,,Õ λ i 1 ØX 1Ùk i ÔX 3 Õ Transcription Ô,1,,Õ λ 1 1 ØX 1Ùk 1 ÔX 3 Õ mrna degradation Ô, 1,,Õ γ 1 X 1 Translation Ô,,1,Õ λ 2 X 1 Protein degradation Ô,, 1,Õ γ 2 X 2 Effector production Ô,,,1Õ λ 3 X 2 Effector degradation Ô,,, 1Õ γ 3 X 3 operators: E fôxõ f Ô,x 1,x 2,x 3 Õ E 1 f ÔxÕ f Ô1,x 1,x 2,x 3 Õ inactive state, active state, E 1 f ÔxÕ f Ôx,x 1 1,x 2,x 3 Õ mrna production, E 1 f ÔxÕ f Ôx,x 1 1,x 2,x 3 Õ mrna degradation, E 2 f ÔxÕ f Ôx,x 1,x 2 1,x 3 Õ protein production, E 2 f ÔxÕ f Ôx,x 1,x 2 1,x 3 Õ protein degradation, E 3 f ÔxÕ f Ôx,x 1,x 2,x 3 1Õ effector production, E 3 f ÔxÕ f Ôx,x 1,x 2,x 3 1Õ effector degradation. The generator associated to the Markov chain is then given by Af ÔxÕ λ a k a Ôx 3 ÕÔEf 1 f ÕÔxÕ λ i k i Ôx 3 ÕÔEf f ÕÔxÕ λ 1 1 Øx 1Ùk 1 Ôx 3 ÕÔE 1 f f ÕÔxÕ γ 1 x 1 ÔE 1 f f ÕÔxÕ λ 2 x 1 ÔE 2 f f ÕÔxÕ γ 2 x 2 ÔE 2 f f ÕÔxÕ λ 3 x 2 ÔE 3 f f ÕÔxÕ γ 3 x 3 ÔE 3 f f ÕÔxÕ. 5.2 Long time behavior Denote by τ i the i th jump times of the chain X. Firstly, we are going to show that, under reasonable assumptions, the jump times do not accumulate, that is τ. This ensures that the model is well defined for all t. Hypothesis 5. The function k 1 is linearly bounded, and specifically, there exists c such that, for any x 3 È N k 1 Ôx 3 Õ x 3 c. Now by a simple consequence of the Meyn and Tweedie [97, thm 2.1] criterion (see also part subsection 6.3, proposition 1), we obtain Proposition 2. The Markov chain defined in subsection 5.1 is non-explosive. Proof. Choose the test function f ÔxÕ x 1 x 2 x 3, which is a norm-like function, it comes directly that Af ÔxÕ maxôλ 1,λ 2,λ 3 Õf ÔxÕ c.

75 74 Hybrid Models to Explain Gene Expression Variability Secondly, we can show the irreductibility. All states communicate with each other as soon as Hypothesis 6. The function k a, and k 1 are strictly positive for x 3, and all rate constants λ a, λ i, λ k and γ k, k 1,2,3, are positive. Then it is classical that the Markov chain is irreductible. Finally, for discrete state-space Markov process, a simple criterion for exponential ergodicity is provided by [97, Theorem 7.1] (see also part subsection 6.3, proposition 15). Assuming Hypothesis 7. minγ i maxλ i, we then have, with the test function f ÔxÕ x 1 x 2 x 3, for all x, Af ÔxÕ Ômaxλ i minγ i Õf ÔxÕ λ 1 c. So the Markov process is exponentially ergodic. There exists an invariant probability measure p, B and β 1 such that the following convergence in distribution holds where P t Ôx, Õ denotes the semigroup and P t Ôx, Õ p f Bf ÔxÕβ t, P t Ôx,gÕ E x gôx t, µ f sup µôgõ. gf Despite we know the long-term behavior of this Markov chain, it s hard to deduce any quantitative information. To be able to concrete parameters values, one approach is to consider constant or linear reaction rate, thus preventing any non-linearity. Thus, analytic methods through the moment generating function can be used. With such tool, it can be computed moment equations, and stationary probability density function (or at least, its moment generating function). However, this techniques seems strictly limited to constant and linear rate functions. See [14] for a typical example. We sketch some of these results in section 7. We will see on the next section that for the continuous deterministic version of this model, namely the Goodwin model, the picture is much more complete, and can deal with non-linear rate functions. In particular, bifurcation parameter analysis can provide information on the bistability or oscillatory behavior of the model. To get analog information on the stochastic model, we will have to reduce its dimension. Hence we will study a one-dimensional stochastic model in section 8, and rigorously prove how to perform such reduction in section Continuous Version - Deterministic Operon Dynamics A continuous deterministic version of this model ignores the fluctuation in the DNA state and considers that the three other chemical species (mrna,proteins and effectors) are present in very large number. We will recall in section 9 standard results to show that the stochastic discrete model converges to the continuous deterministic model, under assumption of fast DNA switching and large molecule number. Note in particular that this model does not represent a statistical mean behavior over a large population of cells,

76 6 Continuous Version - Deterministic Operon Dynamics 75 unless all rates are assumed linear. We refer to [17, 98] for an interesting survey of techniques applicable to this deterministic approach, with in particular models that differs from Ordinary Differential Equation. We consider in this section the standard Goodwin [48] model. These results are not new but included here for convenience and to illustrate its analogy with our results on the stochastic model. Let Ôx 1,x 2,x 3 Õ denote mrna, intermediate protein, and effector concentrations respectively. Then for a generic operon with a maximal level of transcription λ 1 (in concentration over time units), we have dynamics described by the system [48, 51, 52, 1, 128] dx 1 ³² dt λ 1k 1 Ôx 3 Õ γ 1 x 1, dx 2 dt λ 2x 1 γ 2 x 2, (6.1) ³± dx 3 dt λ 3x 2 γ 3 x 3. Here we assume that the rate of mrna production is proportional to the fraction of time the operator region is active, and that the rates of intermediate and enzyme production are simply proportional to the amount of mrna and intermediate respectively. All three of the components Ôx 1,x 2,x 3 Õ are subject to linear degradation. The function k 1 was calculated in the previous section 3 and then taken in this section in the form k 1 Ôx 3 Õ 1 K cx n 3 Λ K c x n, 3 so that it s a smooth bounded function, positive everywhere. Hence global existence and uniqueness of this system is not a problem, and the solution lies in ÔR Õ 3 for all time. It will greatly simplify matters to rewrite eq. (6.1) by defining dimensionless concentrations. To this end we define the dimensionless variable y 1 λ 3λ 2 γ 3 γ 2 n K c x 1, y 2 λ 3 γ 3 n K c x 2, y 3 n K c x 3, and the system eq. 6.1 then becomes where ³² ³± dy 1 dt γ 1Öκ d f Ôy 3 Õ y 1, dy 2 dt γ 2Ôy 1 y 2 Õ, dy 3 dt γ 3Ôy 2 y 3 Õ. κ d λ 3λ 2 λ 1 n K c γ 3 γ 2 γ 1. is a dimensionless constant, and the function f is given by (6.2) f Ôy 3 Õ 1 yn 3 Λ y3 n. (6.3) In each equation, γ i for i 1,2,3 denotes a net loss rate (units of inverse time), and thus eq. 6.2 are not in dimensionless form.

77 76 Hybrid Models to Explain Gene Expression Variability The dynamics of this classic operon model can be fully analyzed. Let Y Ôy 1,y 2,y 3 Õ and denote by S t ÔY Õ the flow generated by the system eq. (6.2). For both inducible and repressible operons, for all initial conditions Y Ôy 1,y 2,y 3 Õ È R 3 the flow S tôy Õ È R 3 for t. Steady states of the system eq. (6.2) are in a one to one correspondence with solutions of the equation y κ d f ÔyÕ, (6.4) and for each solution y of eq. (6.4) there is a steady state Y Ôy 1,y 2,y 3 Õ of eq. (6.2) given by y 1 y 2 y 3 y. Whether there is a single steady state y or there are multiple steady states will depend on whether we are considering a repressible or inducible operon. The detail derivation of the steady-state and their stability is standard ([48, 146, 51, 52, 1, 133]) and is given for an interesting comparison with the stochastic model discussed in section No control (single attractive steady-state) In this case, f ÔyÕ 1, and there is a single steady state y asymptotically stable. κ d that is globally 6.2 Inducible regulation (single versus multiple steady states) For an inducible operon with f given by eq. (6.3) with 1 and Λ 1, there may be one (Y 1 or Y 3 ), two (Y 1,Y 2 Y 3 or Y 1 Y 2,Y 3 ), or three (Y 1,Y 2,Y 3 ) steady states, with the ordering Y 1 Y 2 Y 3, corresponding to the possible solutions of eq. (6.4) (cf. figure 1.5). The smaller steady state ÔY 1 Õ is typically referred to as an uninducedstate, whilethelargest steady state ÔY 3 Õ is called theinducedstate. Thesteady state values of y are easily obtained from eq. (6.4) for given parameter values, and the dependence on κ d for n 4 and a variety of values of Λ is shown in figure 1.5. Figure 1.6 shows a graph of the steady states y versus κ d for various values of the leakage parameter Λ. Analytic conditions for the existence of one or more steady states can be obtained by using eq. (6.4) in conjunction with the observation that the delineation points are marked by the values of κ d at which yßκ d is tangent to f ÔyÕ (see figure 1.5). Simple differentiation of eq. (6.4) yields the second condition 1 κ d nôλ 1Õ yn 1 ÔΛ y n Õ2. (6.5) From eq. (6.4) and eq. (6.5) we obtain the values of y at which tangency will occur: Λ 1 y n n Λ 1 n 2 Λ 1 2 2n Λ 1 1. (6.6) Λ 1 The two corresponding values of κ d at which a tangency occurs are given by (Note the deliberate use of y as opposed to y.) Λ y n κ d y 1 y n. (6.7)

78 6 Continuous Version - Deterministic Operon Dynamics x Figure 1.5: Schematic illustration of the possibility of one, two or three solutions of eq.(6.4) for varying values of κ d with inducible regulation. The monotone increasing graph is the function f of eq. (6.3), and the straight lines correspond to xßκ d for (in a clockwise direction) κ d È Ö,κ d Õ, κ d κ d,κ d È Ôκ d,κ d Õ, κ d κ d, and κ d κ d. This figure was constructed with n 4 and Λ 1 for which κ d 3.1 and κ d 5.91 as computed from eq. (6.7). See the text for further details.

79 78 Hybrid Models to Explain Gene Expression Variability x * κ d Figure 1.6: Full logarithmic plot of the steady state values of y versus κ d for an inducible system, obtainedfromeq.(6.4), forn 4andΛ 2,5,1,and 15(lefttoright)illustrating the dependence of the occurrence of bistability on Λ. See the text for details. A necessary condition for the existence of two or more steady states is obtained by requiring that the square root in in eq. (6.6) be non-negative, or n 1 Å 2 Λ. (6.8) n 1 From this a second necessary condition follows, namely κ d n 1 n n 1 n 1 n 1. (6.9) Further, from eq. (6.4) and (6.5) we can delineate the boundaries in ÔΛ,κ d Õ space in which there are one or three locally stable steady states as shown in figure 1.7. There, we have given a parametric plot (y is the parameter) of κ d versus Λ, using ΛÔyÕ yn Öy n Ôn 1Õ Ôn 1Õy n 1 and κ d ÔyÕ ÖΛÔyÕ yn 2 ny n 1 ÖΛÔyÕ 1, for n 4 obtained from eq. (6.4) and (6.5). As is clear from the figure, when leakage is appreciable (small Λ, e.g for n 4, Λ Ô5ß3Õ 2 ) then the possibility of bistable behavior is lost. Remark 21. Some general observations on the influence of n, Λ, and κ d on the appearance of bistability in the deterministic case are in order. 1. The degree of cooperativity ÔnÕ in the binding of effector to the repressor plays a significant role. Indeed, n 1 is a necessary condition for bistability. 2. If n 1 then a second necessary condition for bistability is that Λ satisfies eq. (6.8) so the fractional leakage ÔΛ 1 Õ is sufficiently small. 3. Furthermore, κ d must satisfy eq. (6.9) which is quite instructive. Namely for n the limiting lower limit is κ d 1 while for n 1 the minimal value of κ d becomes fairly large. This simply tells us that the ratio of the product of the production rates to the product of the degradation rates must always be greater than 1 for bistability to occur, and the lower the degree of cooperativity ÔnÕ the larger the ratio must be.

80 6 Continuous Version - Deterministic Operon Dynamics induced 6 κ d 4 bistable 2 uninduced K Figure 1.7: In this figure we present a parametric plot (for n 4) of the bifurcation diagram in ÔΛ,κ d Õ parameter space delineating one from three steady states in a deterministic inducible operon as obtained from eq. (6.4) and (6.5). The upper (lower) branch corresponds to κ d (κ d ), and for all values of ÔΛ,κ d Õ in the interior of the cone there are twolocally stable steady states Y 1 4,Y 3, while outside there is only one. The tip of the cone occurs at ÔΛ,κ d Õ ÔÔ5ß3Õ 2, Ô5ß3Õ 5ß3Õ as given by eq. (6.8) and (6.9). For Λ È Ö, Ô5ß3Õ 2 Õ there is but a single steady state.

81 8 Hybrid Models to Explain Gene Expression Variability 4. If n, Λ and κ d satisfy these necessary conditions then bistability is only possible if κ d È Öκ d,κ d (c.f. figure 1.7). 5. The locations of the minimal Ôy Õ and maximal Ôy Õ values of y bounding the bistable region are independent of κ d. 6. Finally (a) Ôy y Õ is a decreasing function of increasing n for constant κ d,λ (b) Ôy y Õ is an increasing function of increasing Λ for constant n,κ d. Local and global stability. The local stability of a steady state y is determined by the solutions of the eigenvalue equation [149] Ôλ γ 1 ÕÔλ γ 2 ÕÔλ γ 3 Õ γ 1 γ 2 γ 3 κ d f ½, f ½ f ½ Ôy Õ. (6.1) Set a 1 3ô γ i, a 2 3ô γ i γ j, a 3 Ô1 κ d f ½ Õ i1 ij1 i1 3õ γ i, so eq. (6.1) can be written as λ 3 a 1 λ 2 a 2 λ a 3. (6.11) By Descartes s rule of signs, eq. (6.11) will have either no positive roots for f ½ È Ö,κ 1 d Õ or one positive root otherwise. With this information and using the notation SN to denote a locally stable node, HS a half or neutrally stable steady state, and US an unstable steady state (saddle point), then there will be: A single steady state Y 1 (SN), for κ d È Ö,κ d Õ Two coexisting steady states Y 1 (SN) and Y 2 Y 3 (HS, born through a saddle node bifurcation) for κ d κ d Three coexisting steady states Y 1 ÔSN Õ,Y 2 ÔUSÕ,Y 3 (SN) for κ d È Ôκ d,κ d Õ Two coexisting steady states Y 1 Y 2 (HS at a saddle node bifurcation), and Y 3 (SN) for κ d κ d One steady state Y 3 (SN) for κ d κ d. For the inducible operon, other work extends these local stability considerations and we have the following result characterizing the global behavior: Theorem 22. Othmer [1], Smith [133, Proposition 2.1, Chapter 4] For an inducible operon with f given by eq. (6.3), define I Λ Ö1ßΛ,1. There is an attracting box B Λ R 3 defined by B Λ ØÔy 1,y 2,y 3 Õ : x i È I Λ, i 1,2,3Ù such that the flow S t is directed inward everywhere on the surface of B Λ. Furthermore, all y È B Λ and 1. If there is a single steady state, i.e. Y 1 for κ d È Ö,κ d Õ, or Y 3 for κ d κ d, then it is globally stable. 2. If there are two locally stable nodes, i.e. Y 1 and Y 3 for κ d È Ôκ d,κ d Õ, then all flows SÔY Õ are attracted to one of them. (See [128] for a delineation of the basin of attraction of Y 1 and Y 3.)

82 6 Continuous Version - Deterministic Operon Dynamics x Figure 1.8: Schematic illustration that there is only a single solution of eq. (6.4) for all values of κ d with repressible regulation. The monotone decreasing graph is f for a repressible operon, while the straight lines are xßκ d. This figure was constructed with n 4 and 1. See the text for further details.

83 82 Hybrid Models to Explain Gene Expression Variability 6.3 Repressible regulation (single steady-state versus oscillations) We now consider a repressible operon with f given by eq. (6.3) with 1 and Λ 1. As illustrated in figure 1.8, the repressible operon has a single steady state corresponding to the unique solution y of eq. (6.4). To determine its local stability we apply the Routh- Hurwitz criterion to the eigenvalue eq. (6.11). The steady state corresponding to y will be locally stable (i.e. have eigenvalues with negative real parts) if and only if a 1 (always the case) and a 1 a 2 a 3. (6.12) The well known relation between the arithmetic and geometric means 1 n nô i1 γ i n õ i1 γi«1ßn, when applied to both a 1 and a 2 gives, in conjunction with eq. (6.12), a 1 a 2 a 3 Ô8 κ d f ½ Õ 3õ i1 γ i. Thus as long as f ½ 8ßκ d, the steady state corresponding to y will be locally stable. Once condition eq. (6.12) is violated, stability of y is lost via a supercritical Hopf bifurcation and a limit cycle is born. One may even compute the Hopf period of this limit cycle by assuming that λ jω H (j 1) in eq. (6.11) where ω H is the Hopf angular frequency. Equating real and imaginary parts of the resultant yields ω H a3 ßa 1 or T H 2π ω H 2π 3 i1 γ i Ô1 κ d f ½ Õ 3 i1 γ. i These local stability results tell us nothing about the global behavior when stability is lost, but it is possible to characterize the global behavior of a repressible operon with the following Theorem 23. [133, Theorem 4.1 & Theorem 4.2, Chapter 3] For a repressible operon with ϕ given by eq. (3.11), define I Ö1ß,1. There is a globally attracting box B R 3 defined by B ØÔy 1,y 2,y 3 Õ : x i È I, i 1,2,3Ù such that the flow S is directed inward everywhere on the surface of B. Furthermore there is a single steady state y È B. If y is locally stable it is globally stable, but if y is unstable then a generalization of the Poincare-Bendixson theorem [133, Chapter 3] implies the existence of a globally stable limit cycle in B. Remark 24. There is no necessary connection between the Hopf period computed from the local stability analysis and the period of the globally stable limit cycle. 7 Bursting and Hybrid Models, a Review of Linked Models We summarize here different models that appeared in the literature and review the analytic results available on these models. For most of these models, these results concern constant or linear reaction rates. All these models are linked with the standard model we present in section 5. We also introduce our labeling for these models, that will be useful

84 7 Bursting and Hybrid Models, a Review of Linked Models 83 for naming them in section 9. Hence, capital letters D (respectively C) refers for a discrete (respectively continuous) state-space model; capital letters S (respectively B) stands for a model that includes gene switching (respectively bursting). The number (1, 2, 3) refers to the number of variables included in the model among mrna, protein or effector molecules. All variables and parameters are defined through table 1.4. Below, the stochastic models are stated using a stochastic equation formalism. All Y i are assumed to be independent unit Poisson processes, and are related to the number of times a given reaction fires (see part, subsection 6.2, remark 5). When we refer to the case in the absence of regulation, we mean that the three rate functions k a, k i and k 1 are taken constant equal to Discrete models with switch This model is considered in section 5, and takes into account the four steps described in section 2, namely gene state (X ), mrna (X 1 ), protein (X 2 ) and effector molecules (X 3 ). SD3 ³² ³± t t X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a ÔX 3 ÔsÕÕds Y 2 t t X 1 ÔtÕ X 1 ÔÕ Y 3 λ 1 1 ØX ÔsÕ1Ùk 1 ÔX 3 ÔsÕÕds Y 4 t t X 2 ÔtÕ X 2 ÔÕ Y 5 λ 2 X 1 ÔsÕds Y 6 γ 2 X 2 ÔsÕds, t t X 3 ÔtÕ X 3 ÔÕ Y 7 λ 3 X 2 ÔsÕds Y 8 γ 3 X 3 ÔsÕds. λ i 1 ØX ÔsÕ1Ùk i ÔX 3 ÔsÕÕds, γ 1 X 1 ÔsÕds, Up to our knowledge, no one considered this model! SD2 This model is more widely used, and consider three steps, namely gene state (X ), mrna (X 1 ), protein (X 2 ) (which coincide here with effector molecules). ³² ³± t t X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a ÔX 2 ÔsÕÕds Y 2 t t X 1 ÔtÕ X 1 ÔÕ Y 3 λ 1 1 ØX ÔsÕ1Ùk 1 ÔX 2 ÔsÕÕds Y 4 t t X 2 ÔtÕ X 2 ÔÕ Y 5 λ 2 X 1 ÔsÕds Y 6 γ 2 X 2 ÔsÕds. λ i 1 ØX ÔsÕ1Ùk i ÔX 2 ÔsÕÕds, γ 1 X 1 ÔsÕds, For a review of the behavior of this model without regulation, see [74],[13],[11]. In [12] the author derived asymptotic expression of the moments (and of the measure of noise)

85 84 Hybrid Models to Explain Gene Expression Variability and used it to interpret various model behavior in different kinetic parameter range X P X 1 P ON X 1 P ON λ 1 γ 1 X λ 2 2 γ 2 σ 2 1 P X 2 ON P ON σ1 2 1 X 1 2 X 1 σ2 2 1 X 2 2 X 2 λ a λ a λ i σ 2 γ 1 X 2 γ 1 λ a λ i 1 γ 2 σ 2 γ 2 γ 2 γ 1 X 1 γ 1 γ 2 X 2 γ 1 γ 2 γ 2 λ a λ i γ 1 λ a λ i γ 1 γ 2 In particular, it can be seen from the expressions above, that such model typically present higher fluctuations than a single Poissonian model. Each successive steps brings a contribution in the amount of noise (measured typically as variance over mean squared) of the protein variable for instance. SD1 This model consider a single variable among the gene products, to be either mrna or protein. It has the great advantage to be analytically solvable in the absence of nonlinearity. ³² ³± t t X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a ÔX 1 ÔsÕÕds Y 2 λ i 1 ØX ÔsÕ1Ùk i ÔX 1 ÔsÕÕds, t t X 1 ÔtÕ X 1 ÔÕ Y 3 λ 1 1 ØX ÔsÕ1Ùk 1 ÔX 1 ÔsÕÕds Y 4 γ 1 X 1 ÔsÕds. The authors in [14] computed the analytical steady-state distribution in the case without regulation (k 1,k a,k i constant) and time-dependent moment dynamics, assuming there s no gene product at time ; X 1 ÔtÕ lim t σ2 1 ÔtÕ λ a λ a λ i λ 1 γ 1 λ a λ 1 Ôλ a λ i ÕÔλ a λ i γ 1 Õ e Ôλa λ iõt λ a λ 1 γ 1 Ôλ a λ i γ 1 Õ e γ 1t, λ a λ a λ i λ 1 γ 1 λ a λ 1 Ôλ a λ i Õ 2 λ 2 1 γ 1 Ôλ a λ i γ 1 Õ, gôzõ 1 F 1 Ôc,a,bÔz 1ÕÕ, p x 1 b x 1 e b ôx 1 Å x1 Ô 1Õ i Ôa cõ i 1F 1 Ôa c i,a i,bõ, x 1! i ÔaÕ i i E X 1 ÔX 1 1Õ ÔX 1 n 1Õ b ncôc 1Õ Ôc Ôn 1ÕÕ aôa 1Õ Ôa n. 1Õ where gôzõ denotes the asymptotic moment generating function of X 1, p x 1 its asymptotic distribution and a λ i b λ 1 γ 1 c λ a γ 1 γ 1 λ a

86 7 Bursting and Hybrid Models, a Review of Linked Models 85 Still in the case without regulation, the authors in [68] derived the time-dependent probability distribution (starting with zero mrna) where gôz,tõ f 1 ÔtÕ 1 F 1 Ôc,a,bÔz 1ÕÕ f 2 ÔtÕ 1 F 1 Ô1 c a,2 a,bôz 1ÕÕ f 1 ÔtÕ 1 F 1 Ô c,1 a, be t γ 1 Ôz 1Õ Õ f 2 ÔtÕ bcô1 zõ t aô1 aõ e a γ 1 1F 1 Ôa c,1 a, be t γ 1 Ôz 1Õ Õ The authors in [63] and [112] extended the result for linear regulation (k 1,k a constant and k i ÔX 1 Õ x 1 ). All studies put in evidence that this model contains two main time scales, namely the gene switching and the gene product birth-and-death process, and that the distribution of gene product can be seen as a superposition of Poisson distribution. Roughly, when the two time scales are comparable, the probability distribution exhibits a bimodal behavior. The authors in [126] present numerical simulations of the model with non-linear negative regulation. 7.2 Continuous models with switch SC3 This model is the continuous analog of SD3. ³² ³± t t X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a Ôx 3 ÔsÕÕds Y 2 x 1 ÔtÕ 1 ØX ÔtÕ1Ùλ 1 k 1 Ôx 3 Õ γ 1 x 1, x 2 λ 2 x 1 γ 2 x 2, x 3 λ 3 x 2 γ 3 x 3. Here again, up to our knowledge, no-one considered this model! SC2 This model is the continuous analog of SD2. t ³² X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a Ôx 2 ÔsÕÕds ³± x 1 ÔtÕ 1 ØX ÔtÕ1Ùλ 1 k 1 Ôx 2 Õ γ 1 x 1, x 2 λ 2 x 1 γ 2 x 2. Y 2 t λ i 1 ØX ÔsÕ1Ùk i Ôx 3 ÔsÕÕds, λ i 1 ØX ÔsÕ1Ùk i Ôx 2 ÔsÕÕds, The authors in [13] considered this model and proved asymptotic stability of the related semi-group on L 1, for continuous function k a and k i, and constant function k 1. They used a method based on the Foguel Alternative. The authors in [87] considered numerical simulation of this model with linear regulation (k a,k 1 constant and k i Ôx 2 Õ x 2 ) SC1 This model is the continuous analog of SD1. ² t X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a Ôx 1 ÔsÕÕds ± x 1 ÔtÕ 1 ØX ÔtÕ1Ùλ 1 k 1 Ôx 1 Õ γ 1 x 1. Y 2 t λ i 1 ØX ÔsÕ1Ùk i Ôx 1 ÔsÕÕds, The authors in [87] computed the steady-state distribution of this model with linear regulation (k a,k 1 constant and k i Ôx 1 Õ x 1 ) p x1 Ae λ i λa x λ 1 γ1 1 1 x 1 Ô λ 1 λ i x 1 Õ γ 1 γ 1 1

87 86 Hybrid Models to Explain Gene Expression Variability where A is a normalizing constant. The authors in [144] computed the steady-state distribution of this model with non-linear regulation (k i,k 1 constant and k a Ôx 1 Õ ε λa γ1 ε 1 p x1 Ax λ 1 1 Ô λ i x 1 Õ γ 1 1 x γ 1 λa 1 Ô1 K Õ γ1 x 1 x 1 K ) while with (k a,k 1 constant and k i Ôx 1 Õ ε K x 1 K ) λa γ1 1 p x1 Ax λ λ 1 i ÔKÔ1 εõ εõ 1 x γ 1 Ô x 1 Õ 1 Ô1 KÕ 1 Ô1 γ 1 K Õ λik γ 1 Ô1 KÕ where A is a normalizing constant. Each expression above can be used to determine which are the conditions for the steady-state distribution to exhibit bimodality. 7.3 Discrete models without switch In these models, the gene is now assumed to stay active for all times. D3 ³² ³± t t X 1 ÔtÕ X 1 ÔÕ Y 3 λ 1 k 1 ÔX 3 ÔsÕÕds Y 4 γ 1 X 1 ÔsÕds, t t X 2 ÔtÕ X 2 ÔÕ Y 5 λ 2 X 1 ÔsÕds Y 6 γ 2 X 2 ÔsÕds, t t X 3 ÔtÕ X 3 ÔÕ Y 7 λ 3 X 2 ÔsÕds Y 8 γ 3 X 3 ÔsÕds. Note that in the absence of regulation, X 1 is independent of X 2,X 3 and follows a onedimensional Markov-process, known as the immigration and death process. Its asymptotic distribution is Poissonian. For the whole system, up to our knowledge, no study reported its asymptotic distribution (see the case for 2 variables below). However, being an open first-order reaction network, with both conversion and catalytic reaction, the study of Gadgil et al. [4] allows to derive time-dependent first and second moment. D2 ³² ³± t t X 1 ÔtÕ X 1 ÔÕ Y 3 λ 1 k 1 ÔX 2 ÔsÕÕds Y 4 γ 1 X 1 ÔsÕds, t t X 2 ÔtÕ X 2 ÔÕ Y 5 λ 2 X 1 ÔsÕds Y 6 γ 2 X 2 ÔsÕds. In the absence of regulation, asymptotic moments are given by [142]. X 1 λ 1 γ 1 X 2 λ 1λ 2 γ 1 γ 2 VarÔX λ 1 1 Õ γ 1 VarÔX λ 1λ 2 λ 2 2 Õ 1 γ 1 γ 2 γ 1 γ 2 λ 1 λ 2 CovÔX 1,X 2 γ 1 Ôγ 1 γ 2 Õ

88 7 Bursting and Hybrid Models, a Review of Linked Models 87 A complete study of the asymptotic distribution is provided in [14], whose moment generating function is given by where ϕôx,yõ exp αβ y 1 M Ô1,1 γ,βôs 1ÕÕds αôx 1ÕM Ô1,1 γ,βôy 1Õ γ γ 1 γ 2 α λ 1 γ 1 β λ 2 γ 2 From this expression, the authors in [14] derived asymptotic different behavior of the marginal protein distribution, including Poisson, Neymann, negative Binomial, Gaussian and Gamma distribution. For the non-linear regulation case, the authors in [142, 139, 14] used the linear noise expansion and simulation to study the asymptotic and transient moment behavior with respect to the regulation function. Their study show that negative regulation can increase or decrease noise strength. D1 X 1 ÔtÕ X 1 ÔÕ t t Y 3 λ 1 k 1 ÔX 1 ÔsÕÕds Y 4 γ 1 X 1 ÔsÕds The authors in [132] derived approximation of the time-dependent first moments using moment closure approximation, and successfully compared it with experimental data of the λ-repressor system. As a one-dimensional discrete Markov-chain, its asymptotic distribution can also be derived. 7.4 Continuous models without switch These models were the first one introduced to model gene self-regulation. C3 ² ± x 1 λ 1 k 1 Ôx 3 Õ γ 1 x 1, x 2 λ 2 x 1 γ 2 x 2, x 3 λ 3 x 2 γ 3 x 3. This model was originally introduced by [48]. See subsection 6 for a complete study of the asymptotic behavior of this model. C2 x 1 λ 1 k 1 Ôx 2 Õ γ 1 x 1, x 2 λ 2 x 1 γ 2 x 2. In absence of regulation, the above system can be analytically solved x 1 ÔtÕ λ 1 γ 1 x 2 ÔtÕ λ 1λ 2 γ 1 γ 2 x λ 1 1 ÔÕ γ 1 e γ 1t x 2 ÔÕ λ 1λ 2 γ 1 γ 2 e γ 2t λ 2 x λ 1 1 ÔÕ F ÔtÕ γ 1

89 88 Hybrid Models to Explain Gene Expression Variability where F ÔtÕ e γ 1 t e γ 2 t γ 2 γ 1 if γ 1 γ 2, te γ 2t if γ 1 γ 2. In the presence of positive regulation, this model has essentially similar asymptotic behavior as the previous model C3. In the presence of negative regulation, however, oscillations are not present any more when k 1 is a standard Hill function as in eq. (3.12). C1 x 1 λ 1 k 1 Ôx 1 Õ γ 1 x 1. In the presence of positive regulation, this model has essentially similar asymptotic behavior as the previous model C3. In the presence of negative regulation, however, oscillations are not present any more when k 1 is a standard Hill function as in eq. (3.12). 7.5 Discrete models with Bursting We now turn to Bursting model. Below R is the counting process associated to the number of times a bursting event happens. It is regulated by the effector or protein molecules. BD2 This model can be obtained from SD2 or D3, upon a particular scaling (see section 9). ³² ³± BD1 ³² ³± R ÔtÕ Y t λ 1 k 1 ÔX 2 ÔsÕÕds, X 1 ÔtÕ X 1 ÔÕ Y t X 2 ÔtÕ X 2 ÔÕ Z 1 t R ÔtÕ Y t γ 1 X 1 ÔsÕds λ 2 X 1 ÔsÕds λ 1 k 1 ÔX 1 ÔsÕÕds, X 1 ÔtÕ X 1 ÔÕ Y t γ 1 X 1 ÔsÕds ô t iy i i1 t Z 2 ô t iy i i 1 ØÔqi 1,q i Ù Ôξ R Ôs Õ ÕdR ÔsÕ, γ 2 X 2 ÔsÕds. 1 ØÔqi 1,q i Ù Ôξ R Ôs Õ ÕdR ÔsÕ. The authors in [129] presented stationary and time-dependent probability distribution when k 1 is constant and the jump size a geometric random variable, of mean parameter b. 1 bô1 zõe tßγ 1 a gôz,tõ 1 bô1 zõ ΓÔa nõ b n 1 be tßγ 1 a 1 b p x1 ÔtÕ 2F 1 n, a,1 a n, ΓÔn 1ÕΓÔaÕ 1 b 1 b b e tßγ 1 X 1 ÔtÕ abô1 e tßγ1 Õ σ 2 1 ÔtÕ X 1 ÔtÕÔ1 b be tßγ1 Õ wherea λ 1 γ 1 Theauthorsin[4]computedtheanalytical stationarydistributionforgeneral nonlinear regulation k 1 p p xõ 1 1 x1 a x 1 i1 a k 1ÔiÕ i b 1. b

90 7 Bursting and Hybrid Models, a Review of Linked Models Continuous models with Bursting In continuous bursting model below, N Ôds,dz,drÕ stands for a Poisson random measure, of intensity dshôzõdzdr where h is a probability density that gives the size of the burst. BC2 This model can be obtained from SC2 or BD2, upon a particular scaling (see section 9). We will consider its adiabatic reduction in subsection 9.3. BC1 ³² ³± x 1 ÔtÕ x 1 ÔÕ x 2 ÔtÕ x 2 ÔÕ x 1 ÔtÕ x 1 ÔÕ t t t γ 1 x 1 Ôs Õds λ 2 x 1 Ôs Õds γ 1 x 1 Ôs Õds t t t γ 2 x 2 Ôs Õds. z1 Ørλ1 k 1 Ôx 2 Ôs ÕÕÙN Ôds,dz,drÕ, z1 Ørλ1 k 1 Ôx 1 Ôs ÕÕÙN Ôds,dz,drÕ. The authors in [2] used this model without regulation to successfully fit data from the β-galactosidase protein in E.Coli. The asymptotic distribution is the Gamma distribution p x1 1 b a ΓÔaÕ xa 1 e xßb where a λ 1 γ 1 The authors in [39] computed the analytical expression of the steady-state distributions for non-linear regulation rate k 1, and exponential bursting size of mean b. where A is a normalizing constant. p x1 Ax 1 e xßb e a k 1 ÔzÕ z dz 7.7 Models with both switching and Bursting These models can be obtained from SD2. SBD1 t t X ÔtÕ X ÔÕ Y 1 λ a 1 ØX ÔsÕÙk a ÔX 1 ÔsÕÕds Y 2 λ i 1 ØX ÔsÕ1Ùk i ÔX 1 ÔsÕÕds, ³² t R ÔtÕ Y λ 1 1 ØX ÔsÕ1Ùk 1 ÔX 1 ÔsÕÕds, t ô t ³± X 1 ÔtÕ X 1 ÔÕ Y γ 1 X 1 ÔsÕds iy i 1 ØÔqi 1,q i Ù Ôξ R Ôs Õ ÕdR ÔsÕ. The authors in [129] presented stationary probability distribution when k a,k i,k 1 are constant, and the burst size is a geometric random variable of mean b. p x1 i1 ΓÔα nõγôβ nõγôdõ ΓÔn 1ÕΓÔαÕΓÔβÕΓÔd nõ b n 1 b α 1 b 1 b b 2 F 1 α n,d β,d n, 1 b

91 9 Hybrid Models to Explain Gene Expression Variability where a λ 1 γ 1 c λ a γ 1 d λ i γ 1 λ a α 1 Ôa d φõ 2 β 1 2 Ôa φ 2 Ôa d φõ dõ 2 4ac SBC1 t ³² X ÔtÕ X ÔÕ Y 1 ³± x 1 ÔtÕ x 1 ÔÕ t t λ a 1 ØX ÔsÕÙk a Ôx 1 ÔsÕÕds Y 2 γ 1 x 1 Ôs Õds t 7.8 Hybrid discrete and continuous models D1C1 λ i 1 ØX ÔsÕ1Ùk i Ôx 1 ÔsÕÕds, z1 Ørλ1 1 ØX ÔsÕ1Ùk 1 Ôx 1 Ôs ÕÕÙN Ôds,dz,drÕ. t t X 1 ÔtÕ X 1 ÔÕ Y 3 λ 1k 1 Ôx 2 ÔsÕÕds Y 4 γ 1X 1 ÔsÕds, x 2 λ 2 X 1 γ 2 x 2. In the absence of regulation, the asymptotic characteristic function of the protein variable x 2 has been found to be ([14]) where sβ ωôsõ exp α M Ô1,1 γ γ 1 γ 2 α λ 1 γ 1 β λ 2 γ 2 γ,zõdz This asymptotic expression include both the Gamma and Poisson distribution as limiting behavior. SD1C1 t X ÔtÕ X ÔÕ Y 1 ³² t X 1 ÔtÕ X 1 ÔÕ Y 3 ³± x 2 λ 2 X 1 γ 2 x 2. t λ a 1 ØX ÔsÕÙk a Ôx 1 ÔsÕÕds Y 2 t λ 1 1 ØX ÔsÕ1Ùk 1 Ôx 2 ÔsÕÕds Y 4 λ i 1 ØX ÔsÕ1Ùk i Ôx 2 ÔsÕÕds, γ 1 X 1 ÔsÕds, The author in [11] considered this model as an approximation of the SD2 model, and present moment calculation and numerical simulation of this model.

92 7 Bursting and Hybrid Models, a Review of Linked Models 91 Obviously, different model can again be built with similar features, and the list above is not exhaustive. Although not directly related to our work, we present in the next paragraph different approach of modeling. Such modeling review is intend to show the variety of possible choices of modeling. 7.9 More detailed models and other approaches We first review more detailed models of single gene, then models that take into account other source of noise, and finally models with interaction between genes. InitsPh.D.thesiswork, Jia[71]makesthereviewofthestandardmodelofgeneexpression and its different limiting behavior, in particular condition for occurrence of bursting. Then he generalizes the model to consider non-exponential waiting time between burst events, as well as non-geometric burst size distributions (see also Pedraza and Paulsson [15]). He gives a specific example of model of post-transcriptional regulation with small mrna (a different from but related molecule to mrna) that yields non-geometric burst size distribution. For other models taking into account post-transcriptional regulation by small mrna, see Bose and Ghosh [15], Gorban et al. [49] and for a review of biological mechanisms of post-transcriptional regulation, see Storz and Waters [136]. For models with more than two states of the promoter, see the pioneering work of Tapaswi et al. [141]. Also, Blake et al. [11] used a model with four promoter states to reproduce faithfully the GAL system in prokaryotes. In agreement with data, the main finding is that the level of noise in gene expression is non-monotonic with respect to the level of transcription efficiency. Coulon et al. [24] also considered a model with more than two states for the promoter, and extensively studied the effect of promoter transition on noise strength on protein level. For models at a much finer scale, that explicitly take into account dynamics of mrna polymerase and complex formation, see Dublanche et al. [3], while for mrna polymerase and ribosome dynamics see Kierzek et al. [78], Gorban et al. [49]. A model that goes up to the single-nucleotide level was proposed by Ribeiro [116]. For spatially extended model, see Sagués et al. [122]. In the standard model we consider here, we implicitly assume that there is only one intrinsic source of randomness. Indeed, the stochasticity in the model comes from the random occurrences of the discrete events that constitute the reaction network directly linked to the single gene model (or its product) we study. There are obviously many other sources of randomness that can influence the stochasticity in the gene expression. Firstly, the partitioning event at division is an evident source of randomness when we consider discrete number of molecules. Daughter cells may have different sizes, and each molecule then has to choose between the two daughter cells. Common model that include randomness at partition consider a binomial partition law (see pioneering work of Berg [9], and more recently Huh and Paulsson [65]), which has been supported experimentally [12, 47]. Secondly, a lot of experimental and modeling approaches have focused on extrinsic sources of noise, in particular since the experimental paper of Elowitz et al. [34]. There, the authors used two reporter genes (one with a red fluorescence, one with a green fluorescence), localized at very similar place in the genome, with the same promoter sequence, and measured the fluorescence level of these two genes in single cells. If there were only extrinsic noise, all cells should have the same proportion of red and green fluorescence, at different global intensities. The observed fluctuations in these proportions from cell to cell is attributed to the intrinsic noise. Lei [86] made a review of the different mathematical formulations of extrinsic noise. Usually, the modeling of extrinsic noise includes fluctuations of kinetic parameter, especially of the gene regulation function (see Rosenfeld et al. [12] for experimental evidence), as a Gaussian colored noise [138, 85] (with a Langevin

93 92 Hybrid Models to Explain Gene Expression Variability formalism). Noise due to randomness in the repressor molecule numbers can also be seen as an extrinsic noise. Ochab-marcinek and Tabaka [99] consider this source of noise and show that it can be responsible for bistability (using similar geometric construction-based proof as in our case, in section 8). See also [3] for an experimental evidence that extrinsic noise can have qualitative impact on the gene expression behavior. For model with two genes in interaction see for instance the pioneering work of Kepler and Elston [77], followed by instance by [87]. In such study, bifurcation characterization is of importance. Indeed, interaction of two genes has been widely used to explain cell differentiation fate, where each gene codes for a protein that is responsible of a particular cell lineage. In case of bistability, each stable state then represent a stable cell fate. See for example [79, 117, 137] for recent models applied to individuals cell data. For larger network, experiments and modeling has mostly focused on the quantification on the noise strength of the gene expression level (also called variability), as an output of the model, and as a function of the parameters and rate function or functional motif, (see Çagatay et al. [21]). Besides from extensive numerical simulations, the diffusion approximation of the discrete model has been widely used, see for instance [16]. Finally El-Samad and Khammash [31], Karlebach and Shamir [76] review other approaches of modeling of gene regulatory network, including boolean, probabilistic boolean, petri nets, discrete, continuous and hybrid models, See also the review of [1] for piecewise linear ordinary differential equation and delayed differentiation equation approach. For stochastic and delayed models, see Ribeiro [116], Galla [41] 8 Specific Study of the One-Dimensional Bursting Model We detail here the study of the one-dimensional bursting model, either in a discrete formalism (which is then a pure jump Markov process, subsection 8.1) and in a continuous formalism (which is a piecewise deterministic Markov process, subsection 8.2). For both formalism, we will recall the construction of the stochastic process (and then its existence), and study its long time behavior, using a semigroup formalism (see part subsection 6.5). Once asymptotic convergence has been proved, we study the qualitative property of the invariant probability distribution. The advantage of the one-dimensional model is to possess a probability distribution on the Gibb s form. By analogy to the deterministic modeling, we will speak of a bifurcation when the number of modes of the probability distribution change (called P-bifurcation in the literature). This analogy allows a direct comparison between bifurcation diagrams, and then to deduce the influence of the bursting production on the qualitative dynamics of gene expression. Note that such stochastic bifurcation concept has been applied to empirical measurement data by [134], where the authors obtained an experimental bifurcation diagram by controlling experimentally a parameter and estimating the probability distribution for each parameter value. Up to now, our analytic treatment is restricted to the case of exponential (or geometric in the discrete case) jump distribution. This case is probably the most interesting however, as it is (up to our knowledge) the only case measured experimentally (see [22, 47, 111, 15]). Finally, we show how can compute an explicit convergence rate towards the steady-state measureinsubsection 8.5, and as a corollary of ourstudyof theasymptotic behavior of the bursting model, we present in subsection 8.6 the inverse problem to recover the regulation function from the invariant density. This latter part is an ongoing project, where we try to collect experimental data to apply our theoretical study of the model. The inverse problem may be very interesting in the sense that it permits to deduce molecular interactions that governs the regulation function (see for instance section 3), which are not easily observable experimentally.

94 8 Specific Study of the One-Dimensional Bursting Model 93 Reaction Propensity State change vector Degradation γ n 1 Burst Production r h r λ n r Table 1.5: Definitions of the reactions, propensities and state change vector from the n state in the discrete model. See text for more details. The first subsection will be the object of a future publication ([93]), and the second one was published in 211 ([91]). 8.1 Discrete variable model with bursting BD1 In this section we model the number of gene products in a cell as a pure-jump Markov process X ØX t Ù t in the state space E Ø,1,2,...Ù. Thus a Chapman Kolmogorov governs the probabilities dynamics. A general one-dimensional bursting gene expression model[129](bd1, seesubsection7.5) maybeconstructedasfollows: letnbethenumberof gene products and P n ÔtÕ PrÔX t nõ denote the probability for finding n gene products inside the cell at a given time instant t. We shall include a loss (n n 1) and gain (n n k) of functionality processes in terms of the general rates γ n and λ n, respectively. The step size assume the values k 1,2,3,... and is a random variable (independent of the actual number of gene product) with probability mass function h, so that k1 h k 1. Therefore, the Chapman Kolmogorov equation (or master equation) describing the time evolution of the probabilities P n to have n gene products in a cell is an infinite set of differential equations dp n dt γ n 1 P n 1 γ n P n nô k1 h k λ n k P n k λ n P n, n,1,..., (8.1) where we use the convention that k1. We supplement eq. (8.1) with the initial condition P n ÔÕ v n, n,1,..., where v Ôv n Õ n È l 1 is a probability mass function oftheinitial amountx ofthegeneproduct. We giveexistenceanduniquenessof solutions of eq. (8.1) together with convergence to a stationary distribution. We assume that λ, γ, γ n, λ n,h n, n 1,2,..., ô n1 h n 1. (8.2) The process X is the minimal pure jump Markov process with the jump rate function ϕônõ λ n γ n,n, and the jump transition kernel K given by KÔn, Øn jùõ ² ± q n, if j 1,n 1, Ô1 q n Õh j, if j 1,n,, otherwise. q n γ n λ n γ n, (8.3) Firstly, we recall the construction of X. Let Øξ k Ù k, be a discrete time Markov chain in the state space E Z Ø,1,...Ù with transition kernel K and let Øε k Ù k1 be a sequence of independent random variables exponentially distributed with mean 1. Set T and define recursively the times of jumps of X as T k T k 1 ε k ϕôξ k 1 Õ, k 1,2,...

95 94 Hybrid Models to Explain Gene Expression Variability Starting from X ξ we have X t ξ k, T k t T k 1, k,1,2,..., so that the process is uniquely determined for all t T, where T lim k T k, is called the explosion time. If the explosion time is finite, we can add the point 1 to the state space and we can set X t 1 for t T. The process X is called nonexplosive if P i ÔT Õ 1 for all i È E, where P i is the law of the process starting from X i. We now rewrite eq. (8.1) as an abstract Cauchy problem in the space l 1. We make use of the results from [145]. Let K be the transition operator on l 1 corresponding to K defined as in eq. (8.3). For v Ôv n Õ n È l 1 we have ÔKvÕ q 1 v 1 and ÔKvÕ n q n 1 v n 1 Let us define the operator nô h k Ô1 q n k Õv n k, n 1,2,... k1 Gu ϕu KÔϕuÕ for u È l 1 ϕ Øu È l1 : ô ϕ n u n Ù. There is a substochastic semigroup ØP ÔtÕÙ t on l 1 such that for each initial probability mass function v È l 1 ϕ the equation du dt n GÔuÕ, t, uôõ v, (8.4) has a nonnegative solution uôtõ which is given by uôtõ P ÔtÕv for t and ÔP ÔtÕvÕ n ô P j ÔX t n,t T Õv j, n,1,... j The process X is nonexplosive if and only if the semigroup ØP ÔtÕÙ t is stochastic. Equivalently, the generator of the semigroup ØP ÔtÕÙ t is the closure of ÔG,l 1 ϕõ. In that case the solution uôtõ of eq. (8.4) is unique and it is a probability mass function for each t, if v is such. In particular, if the operator K has a strictly positive fixed point, then the semigroup ØP ÔtÕÙ t is stochastic. Thus, we now look for fixed points of K. The equation for the steady state p Ôp n Õ n of eq. (8.1) is of the form γ n 1 p n 1 γ np n nô h k λ n k p n k λ np n, n,1,... (8.5) k1 Observe that γ 1 p 1 λ p and we can rewrite eq. (8.5) as γ n 1 p n 1 γ np n λ np n 1 ô n p n 1 1 γ n 1 k k h n k λ k p k, n 1,2... Summing both sides and changing the order of summation, we obtain nô ô jn k 1 h j λk p k, n,1,..., (8.6)

96 8 Specific Study of the One-Dimensional Bursting Model 95 Thus given p eq. (8.6) uniquely determines p. Consequently, there is one, and up to a multiplicative constant only one, solution of eq. (8.5), and if p then p n for all n 1. Now, if ô n p n 1 and ô n Ôλ n γ n Õp n, (8.7) then p È l 1 ϕ, GÔp Õ, and KÔϕp Õ ϕp, which implies the semigroup ØpÔtÕÙ t is stochastic. Thus, we have proved the following result. Theorem 25. Assume condition eq. (8.2) and suppose that p Ôp nõ n given by eq. (8.6) satisfies eq. (8.7). Then for each initial probability mass function v Ôv n Õ n È l 1 ϕ eq. (8.1) has a unique solution which is a probability mass function for each t and satisfies lim ô t n ÔP ÔtÕvÕ n p n. Next, we give sufficient conditions for eq. (8.7) in the case when h is geometric with b È Ô,1Õ. Since h k Ô1 bõb k 1, k 1,2,..., (8.8) ô jn k 1 h j b n k, we obtain the following equation for p Ôp nõ n p n 1 p n λ n bγ n γ n 1, n,1... (8.9) Corollary 26. Suppose that h is geometric as in eq. (8.8). Then p Ôp nõ n is given by In particular, if p n p nõ k1 λ n lim 1 b and n γ n then the conclusions of theorem 25 hold. λ k 1 bγ k 1 γ k, n 1,2,... (8.1) lim n γ n γ n 1 1, Remark 27. [Bifurcation] The relation eq. (8.9) can be used to derive bifurcation property in terms of number of modes of the steady-state distribution as a function of parameters. The number of modes are indeed linked to the number of sign change of n λ n bγ n γ n 1. Remark 28. Usually one would consider the functionality loss γ n as a degradation rate with linear dependence on n and the bursting rate λ n to characterize the regulation the system is submitted to: external for independence on n, positive (or negative) self interaction for monotonically increasing (or decreasing) dependence with n. The functional shape of auto regulation is usually taken as a non-linear Hill function, resulting on a quasi steady state assumption of effectors and/or repressors molecules (see section 3 ) Inthefollowingexamples weassumethat his geometric withparameter bandγ n γn, n, with γ. In all examples, the conditions of corollary 26 are satisfied. The following examples are meant to show that analytical formula may be found for a variety of different jump rate function, all restricted to a geometric jump size distribution, however.

97 96 Hybrid Models to Explain Gene Expression Variability Example 1 (Negative binomial). Suppose that λ n λ λn with λ,λ. We have λ n for each n. Plugging γ k and λ k into eq. (8.1) gives p n p n 1 õ Å Å λ λ bγ n k, n,1,... n! bγ λ γ Thus p È l 1 if and only if k λ bγ γ. In that case we obtain the negative binomial distribution where p n ÔaÕ n p n Ô1 põ a, n,1,..., n! p λ and ÔaÕ n is the Pochhammer symbol defined by bγ, a λ γ bγ λ, ÔaÕ n ΓÔa ΓÔaÕ nõ aôa 1ÕÔa 2Õ... Ôa n 1Õ, ÔaÕ 1. This was previously obtained in [129]. Example 2 (Mixture of logarithmic distribution). Suppose that λ and λ n for n 1. Then p λ b n 1 n p γ n, n 1,2,..., which can be rewritten as p n The distribution b n nlnô1 bõ Ô1 p Õ, n 1,2,..., p b n p, p n nlnô1, n bõ 1,2,..., bγ bγ λ lnô1 bõ. is called a logarithmic distribution. If we assume that λ n for n m, then we obtain the following distribution and where c and p are such that p b n n p n! c n 1 õ k λk bγ p n Ô1 mô ô jm 1 b j j j and k Å, n,...,m, p j Õbn cn, n m, mô p j j mc p m b m 1. In particular, this type of distribution will be obtained if we take λ, λ, and λ n λ λn, if n λ ßλ,, otherwise.

98 8 Specific Study of the One-Dimensional Bursting Model 97 Example 3. We now look at λ n λ 1 K 1n K K 1 n, n,1,..., where λ,k 1,K 1. We find that, for each n, where and λ n bγn γ bôn a 1ÕÔn a 2 Õ n b 1, b 1 K, a 1 1 K 1 2 Ôα βõ, a 2 1 Ôα βõ, 2 α K K 1 λ bγ, β2 α 2 4λ K 1 bγ. Since K 1, we can find a nonnegative β, thus a 2 a 1. Consequently, the stationary distribution is of the form p n 1 Ôa 1 Õ n Ôa 2 Õ n b n 2F 1 Ôa 1,a 2 ;b 1 ;bõ Ôb 1 Õ n n!, n,1,..., where 2 F 1 is the Gauss s hypergeometric function 2F 1 Ôa 1,a 2 ;b 1 ;xõ ô n Ôa 1 Õ n Ôa 2 Õ n Ôb 1 Õ n Example 4 (Generalized hypergeometric distributions). The generalized hypergeometric function p F q is defined to be the real analytical function on R given by the series expansion pf q Ôa 1,...,a p ;b 1,...,b q ;xõ ô n x n n!. Ôa 1 Õ n...ôa p Õ n x n Ôb 1 Õ n...ôb q Õ n n!. The negative binomial distribution in example 1 for the case of λ has the probability generating function s 1 F Ôa 1 ;bsõß 1 F Ôa 1 ;bõ with a 1 λ ßbγ. The distribution obtained in example 3has the probability generating function s 2 F 1 Ôa 1,a 2 ;b 1 ;bsõß 2 F 1 Ôa 1,a 2 ;b 1 ;bõ. Extending both of these examples we suppose that λ n is a rational function of n satisfying λ n bγn Ôn a 1Õ... Ôn a q 1 Õb, n,1,2,... γ Ôn b 1 Õ... Ôn b q Õ Then p Ôp nõ n has the probability generating function of the form q 1F q Ôa 1,...,a q 1 ;b 1,...,b q ;bsõ q 1F q Ôa 1,...,a q 1 ;b 1,...,b q ;bõ. Example 5. Consider λ n as a Hill function of the form λ n λ 1 K 1n N K K 1 n N, where K 1,K,λ and N 1. If h is geometric and lim γ γ n n, lim 1, n n γ n 1 then irrespective of b there always exists p Ôp n Õ n satisfying eq. (8.6).

99 98 Hybrid Models to Explain Gene Expression Variability 8.2 Continuous variable model with bursting BC1 In this section we consider a continuous state space version of the model presented in section 8.1 (BC1, see subsection 7.6), which is a piecewise deterministic Markov process Y ØY t Ù t with values in E Ô, Õ where Y t denotes the amount of the gene product in a cell at time t, t. We assume that protein molecules undergo the process of degradation with rate γ that is interrupted at random times t 1 t 2... occurring with intensity λ and both λ and γ depend on the current amount of molecules. At t k a random amount of protein molecules is produced, independently of the current number of proteins, so that the process changes from Y tk to Y tk Y tk e k, k 1,2,..., where Øe k Ù k1 is a sequence of positive independent random variables with probability density function h, which are also independent of Y. The time-dependent probability density function uôt, xõ is described by the continuous analog of the master equation uôt,xõ t ÔγÔxÕuÔt,xÕÕ x λôxõuôt,xõ x λôx yõuôt, x yõhôyõdy (8.11) with the initial probability density uô,xõ vôxõ, x. We assume that γ is a continuous function and that λ is a nonnegative measurable function with λßγ being locally integrable on Ô, Õ and γôxõ for x, δ dx γôxõ, δ λôxõ dx, (8.12) γôxõ for some δ. From eq. (8.12) it follows that the differential equation x ½ ÔtÕ γôxôtõõ, xôõ x, has a unique solution which we denote by π t x, t, x. For each x we have π t x as t and t λôπ s xõds x π tx λôyõ dy, as t. γôyõ We now recall the construction of the minimal piecewise deterministic Markov process Y. Let Øε k Ù k1 be a sequence of independent random variables exponentially distributed with mean 1, which is also independent of Øe k Ù k1. Set t. For each k 1,2,... and given Y tk 1 the process evolves as Y t πt tk 1 Y t k 1, t k 1 t t k, Y tk e k, t t k, (8.13) where t k t k 1 t k and t k is a random variable such that PrÔ t k ty tk 1 xõ 1 e t λôπsxõds, t,x. The random variable t k can be defined with the help of the exponentially distributed random variable ε k trough the equality in distribution ε k tk λôπ s Y tk 1 Õds,

100 8 Specific Study of the One-Dimensional Bursting Model 99 which can be rewritten as ε k QÔπ tk Y tk 1 Õ QÔY t k 1 Õ, where the nonincreasing function Q is given by QÔxÕ x x λôyõ dy, (8.14) γôyõ and x, when the integral is finite or any x otherwise. Since Y tk π tk Y tk 1, we obtain the following stochastic recurrence equation for ØY tk Ù k Y tk Q 1 ÔQÔY tk 1 Õ ε kõ e k, k 1,2,..., where Q 1 is the generalized inverse of Q, Q 1 ÔrÕ supøx : QÔxÕ rù. Consequently, Y t is defined by eq. (8.13) for all t t, where t lim k t k is the explosion time. As in the discrete state space we can extend the state space E by adding the point 1 and define Y t 1 for t t. Let P x be the law of the process Y starting at Y x and denote by E x the expectation with respect to P x. Remark 29. Note that if QÔÕ then the amount of the gene product ØY tk Ù k at the jump times is a discrete time Markov process with transition probability function given by where KÔx,BÕ kôx,yõ e QÔxÕ x B kôx,yõdy, B È BÔÔ, ÕÕ, 1 Ô,yÕ ÔzÕhÔy zõ λôzõ γôzõ e QÔzÕ dz, x,y. (8.15) We rewrite eq. (8.11) as an abstract Cauchy problem in L 1 where the operator CuÔxÕ dôγôxõuôxõõ dx is defined on the domain du dt Cu, uôõ v, (8.16) λôxõuôxõ x λôx yõuôx yõhôyõdy D Øu È L 1 : γu È AC, ÔγuÕ ½ È L 1, lim x ÔγÔxÕuÔxÕÕ, λu È L 1 Ù, and γu È AC means that the function x γôxõuôxõ is absolutely continuous. From [9, 145] it follows that there is a substochastic semigroup ØP ÔtÕÙ t on L 1 such that for each initial density v È D eq. (8.16) has a nonnegative solution uôtõ which is given by uôtõ P ÔtÕv for t and P x ÔY t È B,t t ÕvÔxÕdx B P ÔtÕvÔxÕdx for all Borel subsets B of Ô, Õ. The semigroup ØP ÔtÕÙ t is stochastic if the transition operator K on L 1 with kernel k as in eq. (8.15) has a strictly positive fixed point. Let us consider the case of the exponential bursting size where b. hôyõ 1 b e yßb, y, (8.17)

101 1 Hybrid Models to Explain Gene Expression Variability Theorem 3. Assume that condition eq. (8.12) holds and that h is exponential as in eq. (8.17) with b. Suppose that c : 1 γôxõ e xßb QÔxÕ dx, e xßb QÔxÕ dx. (8.18) Then the semigroup ØP ÔtÕÙ t is stochastic and for each initial density v we have where lim t ÐP ÔtÕv u Ð 1, is the unique stationary density of ØP ÔtÕÙ t. u ÔxÕ 1 cγôxõ e xßb QÔxÕ (8.19) Proof. Let k be as in eq. (8.15) and let v ÔxÕ e xßb QÔxÕ, x. The functionv satisfies since for each y we have and y y v ÔyÕ v ÔxÕkÔx,yÕdx v ÔxÕkÔx,yÕdx y y v ÔxÕkÔx,yÕdx, y, hôy zõ λôzõ γôzõ e QÔzÕ dz e xßb x y e xßb dx hôy zõ λôzõ γôzõ e QÔzÕ dzdx, which, by making use of the form of h and changing the order of integration, can be transformed to y By eq. (8.18) the function v ÔxÕkÔx,yÕdx e yßb y R v ÔxÕ : 1 γôxõ Ô1 bhôy zõõ λôzõ γôzõ e QÔzÕ dz e yßb e QÔyÕ be yßb y x hôy zõ λôzõ γôzõ e QÔzÕ dz. e QÔyÕ QÔxÕ v ÔyÕdy b 1 γôxõ e xßb QÔxÕ is integrable, which implies that u È D and CÔu Õ. The rest of the proof is as in [9]. Remark 31. Note that if QÔÕ and λôxõ lim x γôxõ 1 b, then the function x e xßb QÔxÕ is integrable on Ô, Õ. If, additionally, e QÔxÕ δ lim supγôxõ, lim x x γôxõ r, and γôxõ r 1 dx for some δ,r, then condition eq. (8.18) holds.

102 8 Specific Study of the One-Dimensional Bursting Model 11 Remark 32. [Bifurcation] The relation given at eq. (8.19) can be used to derive bifurcation property in terms of number of modes of the steady-state distribution as a function of parameters. The number of extrema are indeed linked to the number of solution of (if this expression has a sense) λôxõ γôxõ 1 γ ½ ÔxÕ b γôxõ The following examples are meant to show that analytical formula may be found for a variety of different jump rate function, all restricted to an exponential jump size distribution, however. Example 6. Consider the case of linear regulation with the function λ of the form λôxõ λ λx, where λ,λ are nonnegative constants, and γôxõ γx. If 1 b λ γ and λ, then u is integrable and is the gamma distribution u ÔxÕ 1 1 ΓÔλ ßγÕ b λ Å λ ßγ x λ 1 γ e Ô1 b λ γ Õx, γ which is a continuous approximation of the negative binomial distribution previously obtained, as in [129]. Example 7. Let γôxõ γx β with γ and β 1. Suppose that λôxõ λx α with λ. Then QÔÕ if and only if α β 1. For α β 1 we have Let γôxõ γx with γ QÔxÕ λ γôβ 1 αõ xα β 1. Theorem 33. [9, Theorem 7]. The unique stationary density of eq. (8.11), with λ a measurable bounded function above and under and h an exponential distribution given by eq. (8.17), is u ÔxÕ C 1 x λôyõ x e xßb exp γ y dy, where C is a normalizing constant such that u ÔxÕdx 1. Further, uôt,xõ is asymptotically stable. Remark 34. Note also that we can also represent u as x λôyõ u ÔxÕ Cexp γy 1 b 1 y where C is a normalizing constant. Example 8.. Consider the function λ of the form where λ,k 1. Then and QÔxÕ λôxõ λ 1 K 1 x N λ γn logôx N K 1 Õ Å dy, u ÔxÕ ÔcγÕ 1 e xßb x λßγ 1 Ô1 K 1 x N Õ λßôγnõ.

103 12 Hybrid Models to Explain Gene Expression Variability Example 9. Consider the function λ of the form [91] λôxõ λ 1 Λ xn x N λ λ 1 Λ Å 1 Λ x N, where λ,λ, are positive constants and N is a positive integer. Let γôxõ γx with γ. The stationary density is given by where c is a normalizing constant and u ÔxÕ c 1 e xßb x κ bôλõ 1 1 ÔΛ x N Õ θ, (8.2) κ b λ γ θ κ b 1 Å. N Λ The solution on the last example has been extensively studied in terms of numbers of modes (P-bifurcation) in [91], which we reproduce below. We will constantly make the analogy with the deterministic bifurcation study in section 6. The first two terms of eq. (8.2) are simply proportional to the density of the gamma distribution. For κ b Λ 1 1 we have u ÔÕ while for κ b Λ 1 1, u ÔÕ and there is at least one mode at a value of x. We have u ÔxÕ for all x and from remark 34 it follows that u ½ ÔxÕ u κb λôxõ ÔxÕ 1 x b 1 Å, x. (8.21) x Observe that if κ b 1 then u is a monotone decreasing function of x, since κ b f ÔxÕ 1 for all x. Thus we assume in what follows that κ b 1. Since the analysis of the qualitative nature of the stationary density leads to different conclusions for the uncontrolled, inducible or repressible operon cases, we consider each in turn Protein distribution in the absence of control When Λ 1, the density u is that of a gamma distribution, as obtained in [39]. u ÔxÕ 1 b κ b ΓÔκb Õ xκ b 1 e xßb, where ΓÔ Õ denotes the gamma function and κ b λ γ. For κ b È Ô,1Õ, u ÔÕ and u is decreasing while for κ b 1, u ÔÕ and there is a mode at x bôκ b 1Õ Bursting in the inducible operon When 1 and Λ 1, we have θ and the third term of eq. (8.2) is a monotone increasing function of x and, consequently, there is the possibility that u may have more than one mode, indicative of the existence of bistable behavior. From eq. (8.21) it follows that we have u ½ ÔxÕ for x if and only if 1 κ b x b 1 1 xn Λ xn. (8.22) Again, graphical arguments (see figure 1.9) show that there may be up to three roots of eq. (8.22). For illustrative values of n, Λ, and b, figure 1.1 shows the graph of the values of x at which u ½ ÔxÕ as a function of κ b. When there are three roots of eq. 8.22, we label them as x 1 x 2 x 3.

104 8 Specific Study of the One-Dimensional Bursting Model x Figure 1.9: Schematic illustration of the possibility of one, two or three solutions of eq. (8.22) for varying values of κ b with bursting inducible regulation. The straight lines correspond (in a clockwise direction) to κ b È Ô,κ b Õ, κ b κ b, κ b È Ôκ b,κ b Õ (and respectively κ b Λ, κ b Λ, Λ κ b ), κ b κ b, and κ b κ b. This figure was constructed with n 4, Λ 1 and b 1 for which κ b 4.29 and κ b as computed from eq. (8.25). See the text for further details

105 14 Hybrid Models to Explain Gene Expression Variability Generally we cannot determine when there are three roots. However, we can determine when there are only two roots x 1 x 3 from the argument of subsection 6.2. At x 1 and x 3 we will not only have eq. (8.22) satisfied but the graph of the right hand side of eq. (8.22) will be tangent to the graph of the left hand side at one of them so the slopes will be equal. Differentiation of eq. (8.22) yields the second condition n xn 1 ÔΛ x n Õ 2 1 κ b bôλ 1Õ (8.23) We first show that there is an open set of parameters Ôb,Λ,κ b Õ for which the stationary density u is bimodal. From eq. (8.22) and (8.23) it follows that the value of x at which tangency will occur is given by and z are positive solutions of equation x bôκ b 1Õz z n 1 z βô1 zõ2, where β ΛÔκ b 1Õ ÔΛ 1Õκ b. We explicitly have z 1 2βn Ôn 2βn 1Õ Ôn Å 1Õ 2 4βn provided that Ôn 1Õ 2 4n β ΛÔκ b 1Õ ÔΛ 1Õκ b. (8.24) The eq. (8.24) is always satisfied when κ b Λ or when κ b Λ and Λ is as in the deterministic case, eq. (6.8). Observe also that we have z z for κ b Λ and z z for κ b Λ. The two corresponding values of b at which a tangency occurs are given by 1 Λ n b Ôκ b 1Õz βô1 z Õ Λ and z. If κ b Λ then u ÔÕ and u is decreasing for b b, while for b b there is a local maximum at x. If κ b Λ then u ÔÕ and u has one or two local maxima. As a consequence, for n 1 we have a bimodal steady state density u if and only if the parameters κ b and Λ satisfy eq. (8.24), κ b Λ, and b È Ôb,b Õ. We now want to find the analogy between the bistable behavior in the deterministic system and the existence of bimodal stationary density u. To this end we fix the parameters b and Λ 1 and vary κ b as in figure 1.9. The eq. (8.22) and (8.23) can also be combined to give an implicit equation for the value of x at which tangency will occur x 2n ÔΛ 1Õ n Λ 1 x n nbôλ 1Õx n 1 Λ Λ 1 and the corresponding values of κ b are given by Å x b Λ κ b b 1 x n x n Å. (8.25) There are two cases to distinguish. Case 1. κ b Λ. In this case, u ÔÕ. Further, the same graphical considerations as in the deterministic case show that there can be none, one, or two positive solutions

106 8 Specific Study of the One-Dimensional Bursting Model x κ b Figure 1.1: Full logarithmic plot of the values of x at which u ½ ÔxÕ versus the parameter κ b, obtained from eq. (8.22), for n 4, Λ 1, and (left to right) b 5,1 and b 1 1. Though somewhat obscured by the logarithmic scale for x, the graphs always intersect the κ b axis at κ b Λ. Additionally, it is important to note that u ½ ÔÕ for Λ κ b, and that there is always a maximum at for κ b Λ. See the text for further details.

107 16 Hybrid Models to Explain Gene Expression Variability to eq. (8.22). If κ b κ b, there are no positive solutions, u is a monotone decreasing function of x. If κ b κ b, there are two positive solutions ( x 2 and x 3 in our previous notation, x 1 has become negative and not of importance) and there will be a mode in u at x 3 with a minimum in u at x 2. Case 2. Λ κ b. Now, u ÔÕ and there may be one, two, or three positive roots of eq. (8.22). We are interested in knowing when there are three which we label as x 1 x 2 x 3 as x 1, x 3 will correspond to the location of mode in u while x 2 will be the location of the minimum between them and the condition for the existence of three roots is κ b κ b κ b. We see then that the different possibilities depend on the respective values of Λ, κ b, κ b, and κ b. To summarize, wemay characterize the stationary density u for an inducible operon in the following way: 1. Unimodal type 1: u ÔÕ andu isdecreasingfor κ b κ b and κ b Λ 2. Unimodal type 2: u ÔÕ and u has a single mode at (a) x 1 for Λ κ b κ b or (b) at x 3 for κ b κ b and Λ κ b 3. Bimodal type 1: u ÔÕ and u has a single mode at x 3 for κ b κ b Λ 4. Bimodal type 2: u ÔÕ and u has two modes at x 1, x 3, x 1 x 3 for κ b κ b κ b and Λ κ b Remark 35. Two comments are in order. 1. Remember that the case n 1 cannot display bistability in the deterministic case. However, in the case of bursting in the inducible systemwhen n 1, if Λ b 1 κ b Λ and b Λ Λ 1, then u ÔÕ and u also has a mode at x 3. Thus in this case one can have a bimodal type 1 stationary density. 2. Lipshtat et al. [88], in a numerical study of a mutually inhibitory gene arrangement (which is dynamically equivalent to an inducible operon), provided numerical evidence that bistability was possible without cooperative binding (i.e. n 1). The demonstration here of bistability gives analytic support to their conclusion. We now choose to see how the average burst size b affects bistability in the density u by looking at the parametric plot of κ b ÔxÕ versus ΛÔxÕ. Define Then F Ôx,bÕ x n 1 nx n 1 Ôx bõ. (8.26) ΛÔx,bÕ 1 xn F Ôx,bÕ 1 F Ôx,bÕ and κ b Ôx,bÕ ÖΛÔx,bÕ x n x b bôx n 1Õ. (8.27) The bifurcation diagram obtained from a parametric plot of Λ versus κ b (with x as the parameter) is illustrated in figure 1.11 for n 4 and two values of b. Note that it is necessary for Λ κ b in order to obtain Bimodal type 2 behavior. For bursting behavior in an inducible situation, there are two different bifurcation patterns that are possible. The two different cases are delineated by the respective values of Λ and κ b, as shown in figure 1.1 and figure Both bifurcation scenarios share the property that while increasing the bifurcation parameter κ b from to, the stationary density u passes from a unimodal density with a peak at a low value (either or x 1 ) to a bimodal density and then back to a unimodal density with a peak at a high value ( x 3 ).

108 8 Specific Study of the One-Dimensional Bursting Model κ b K Figure 1.11: In this figure we present two bifurcation diagrams (for n 4) in ÔΛ,κ b Õ parameter spacedelineatingunimodal frombimodal stationary densities u inan inducible operon with bursting as obtained from eq. (8.27) and (8.26). The upper cone-shaped plot is for b 1 1 while the bottom one is for b 1. In both cone shaped regions, for any situation in which the lower branch is above the line κ b Λ (lower straight line) then bimodal behavior in the stationary solution u ÔxÕ will be observed with modes in u at positive values of x, x 1 and x 3.

109 18 Hybrid Models to Explain Gene Expression Variability κ b K Figure 1.12: This figure presents an enlarged portion of figure 1.11 for b 1. The various horizontal lines mark specific values of κ b referred to in figures 1.13 and 1.14.

110 8 Specific Study of the One-Dimensional Bursting Model κ b x Figure 1.13: In this figure we illustrate Bifurcation type 1 when intrinsic bursting is present. For a variety of values of the bifurcation parameter κ b (between 3 and 6 from top to down), the stationary density u is plotted versus x between and 8. The values of the parameters used in this figure are b 1, Λ 4, and n 4. For κ b ü 3.5, u has a single mode at x. For 3.5 ü κ b 4, u has two local maxima at x and x 3 1. For 4 κ b ü 5.9, u has two local maxima at x 1 x 3. Finally, for κ b ý 5.9, u has a single mode at x 3 1. Note that for each plot of the density, the scale of the ordinate is arbitrary to improve the visualization. In what will be referred as Bifurcation type 1, the maximum at x disappears whenthereisasecondpeakatx x 3. Thesequenceofdensitiesencounteredforincreasing values of κ b is then: Unimodal type 1 to a Bimodal type 1 to a Bimodal type 2 and finally to a Unimodal type 2 density. In the Bifurcation type 2 situation, the sequence of density types for increasing values of κ b is: Unimodal type 1 to a Unimodal type 2 and then a Bimodal type 2 ending in a Unimodal type 2 density. The two different kinds of bifurcation that can occur are easily illustrated for b 1 as the parameter κ b is increased. An enlarged diagram in the region of interest is shown in figure In figure 1.13 we illustrate Bifurcation type 1, when Λ 4, and κ b increases from low to high values. As κ b increases, we pass from a Unimodal type 1 density, to a Bimodal type 1 density. Further increases in κ b lead to a Bimodal type 2 density and finally to a Unimodal type 2 density. This bifurcation cannot occur, for example, when b 1 1 and Λ 15 (see figure 1.11). In figure 1.14 we show a Bifurcation type 2, when Λ 3. As κ b increases, we pass fromaunimodal type1density, to aunimodal type2density. Thenwithfurtherincreases in κ b, we pass to a Bimodal type 2 density and finally back to a Unimodal type 2 density. Remark 36. There are several qualitative conclusions to be drawn from the analysis of

111 11 Hybrid Models to Explain Gene Expression Variability κ b x Figure 1.14: An illustration of Bifurcation type 2 for intrinsic bursting. For several valuesofthebifurcationparameterκ b (between2.8and5fromtoptodown), thestationary density u is plotted versus x between and 8. The parameters used are b 1, Λ 3, and n 4. For κ b 3, u has a single mode at x, and for 3 κ b ü 3.3, u has a single mode at x 1. For 3.3 ü κ b ü 4.45, u has two local maxima at x 1 x 3, and finally for κ b ý 4.45 u has a single mode at x 3. Note that for each plot of the density, the scale of the ordinate is arbitrary to improve the visualization. this section. 1. The presence of bursting can drastically alter the regions of parameter space in which bistability can occur relative to the deterministic case. In figure 1.15 we present the regions of bistability in the presence of bursting in the ÔΛ,b κ b Õ parameter space, which should be compared to the region of bistability in the deterministic case in the ÔΛ,κ d Õ parameter space (bκ b is the mean number of proteins produced per unit of time, as is κ d ). 2. When κ b Λ, at a fixed value of κ b, increasing the average burst size b can lead to a bifurcation from Unimodal type 1 to Bimodal type When Λ κ b, at a fixed value of κ b, increasing b can lead to a bifurcation from Unimodal type 2 to Bimodal type 2 and then back to Unimodal type Bursting in the repressible operon The possible behaviors in the stationary density u for the repressible operon are easy to delineate based on the analysis of the previous section, with eq. (8.22) replaced by 1 κ b x b 1 1 xn 1 xn. (8.28) Again graphical arguments (see figure 1.16) show that eq. (8.28) may have either none or one solution. Namely, 1. For κ b 1, u ÔÕ and u is decreasing. Eq does not have any solution (Unimodal type 1). 2. For 1 κ b, u ÔÕ and u has a single mode at a value of x determined by the single positive solution of eq. (8.28) (Unimodal type 2).

112 8 Specific Study of the One-Dimensional Bursting Model κ d or bκ b K Figure 1.15: The presence of bursting can drastically alter regions of bimodal behavior as shown in this parametric plot (for n 4) of the boundary in ÔK,b κ b Õ parameter space delineating unimodal from bimodal stationary densities u in an inducible operon with bursting and in ÔK,κ d Õ parameter space delineating one from three steady states in the deterministic inducible operon. From top to bottom, the regions are for b 1, b 1, b.1 and b.1. The lowest (heavy dashed line) is for the deterministic case. Note that for b.1, the two regions of bistability and bimodality coincide and are indistinguishable from one another Recovering the deterministic case We can recover the deterministic behavior from the bursting dynamics with a suitable scaling of the parameters and limiting procedure. With bursting production there are two important parameters (the frequency κ b and the amplitude b), while with deterministic production there is only κ d. The natural limit to consider is when b, κ b with bκ b κ d. In this limit, the implicit equations which define the maximum points of the steady state density, become the implicit eq. (6.4) and (6.5) which define the stable steady states in the deterministic case. The bifurcations will also take place at the same points, because we recover eq. (6.7) in the limit. However, Bimodality type 1 as well as the Unimodal type 1 behaviors will no longer be present, as in the deterministic case, because for κ b we have κ b Λ. Finally, from the analytical expression for the steady-state density, eq. (8.2), u will

113 112 Hybrid Models to Explain Gene Expression Variability x Figure 1.16: Schematic illustration that there can be one or no solution of eq. (8.28),depending on the value of κ b, with repressible regulation. The straight lines correspond (in a clockwise direction) to κ b 2 and κ b.8. This figure was constructed with n 4, 1 and b 1. See the text for further details. became more sharply peaked as b. Due to the normalization constant (which depends on b and κ b ), the mass will be more concentrated around the larger maximum of u. 8.3 Fluctuations in the degradation rate only We now look at a model analog to the one studied in section 8.2, but where the noise is included in the degradation rate rather than in the production rate. Such model can be justified in some sense by a limiting procedure. We then look at the stochastic differential equation in the form dx γöκ d λôxõ x dt σ xdw. Within the Ito interpretation of stochastic integration, this equation has a corresponding Fokker Planck equation for the evolution of the ensemble density uôt, xõ given by [84] u t ÖÔγκ dλôxõ γxõu x σ ÔxuÕ x 2. (8.29) As by hypothesis λôõ, it is natural to consider the boundary at x reflecting and the stationary solution of eq. (8.29) is then given by u ÔxÕ exp C 2γκd x λôyõ x e 2γxßσ2 σ 2 y dy.

114 8 Specific Study of the One-Dimensional Bursting Model 113 Set κ e 2γκ d ßσ 2, and take λôxõ λ 1 xn Λ x N Then the steady state solution is given explicitly by u ÔxÕ Ce 2γxßσ2 x κeλ 1 1 ÖΛ x n θ, (8.3) whereλ, andθ aregiven in table1.1. Note that thisdensityhas thesame expression as eq. (8.2) Remark 37. Two comments are in order. 1. Because the form of the solutions for the situation with bursting and Gaussian white noise are identical, all of the results of the previous section can be carried over here with the proviso that one replaces the average burst amplitude b with b σ 2 ß2γ b w and κ b κ e 2γκ d ßσ 2 κ d ßb w. 2. We can look for the regions of bimodality in the ÔK,κ d Õ-plane, for a fixed value of b w. We have the implicit equation for x x 2n ÔK 1Õ n K 1 K 1 and the corresponding values of κ d are given by κ d Ôx b w Õ x n nb w ÔK 1Õx n 1 K K x n Å 1 x n. Then the bimodality region in the ÔK,κ d Õ-plane with noise in the degradation rate is the same as the bimodality region for bursting in the ÔK,bκ b Õ-plane. We have also the following result. Theorem 38. [16, Theorem 2]. The unique stationary density of eq. (8.29) is given by eq. (8.3). Further uôt, xõ is asymptotically stable. 8.4 Discussion In trying to understand experimentally observed distributions of intracellular components from a modeling perspective, the norm in computational and systems biology is often to use algorithms developed initially by Gillespie [45] to solve the chemical master equation for specific situations. See [87] for a typical example. However these investigations demand long computer runs, are computationally expensive, and further offer little insight into the possible diversity of behaviors that different gene regulatory networks are capable of. There have been notable exceptions in which the problem has been treated from an analytical point of view, c.f. [77], [39], [13], and [129]. The advantage of an analytic development is that one can determine how different elements of the dynamics shape temporal and steady state results for the densities uôt,xõ and u ÔxÕ respectively. Here we have extended this analytic treatment to simple situations in which there is bursting transcription and/or translation (building on and expanding the original work of [39]), (for the fluctuations in degradation rates case, see subsection 8.3), as an alternative to the Gillespie [45] algorithm approach. The advantage of the analytic approach that we have taken is that it is possible, in some circumstances, to give precise conditions on the statistical stability of various dynamics. Even when analytic solutions are not available

115 114 Hybrid Models to Explain Gene Expression Variability for the partial integro-differential equations governing the density evolution, the numerical solution of these equations may be computationally more tractable than using the Gillespie [45] approach. The results we have reported here in section 8.2 concern convergence towards a stationary density for a continuous model in the presence of bursting noise. The source noise considered is then in the production term, and was modeled as a compound Poisson process. We have focused on qualitative properties of the stationary density, in particular the number of modes. In subsection 8.3, we have studied a continuous stochastic model where the source noise is in the degradation term, and has been modeled as multiplicative Gaussian white noise. We have focused on convergence towards steady-state, as well as qualitative properties of the stationary density. A surprising result of the work reported here is that the stationary densities in the presence of bursting noise are analytically indistinguishable from those in the presence of degradation noise. We had expected that there would be clear differences that would offer some guidance for the interpretation of experimental data to determine whether one or the other source of noise was of predominant importance. Of course, the next obvious step is to examine the problem in the presence of both noise sources simultaneously. In terms of the issue of when bistability, or a unimodal versus bimodal stationary density is to be expected, we have pointed out the analogy between the bistable behavior in the deterministic system and the existence of bimodal stationary densities in the stochastic systems. Our analysis makes clear the critical role of the dimensionless parameters n, κ (be it κ d, κ b ), b, and the fractional leakage Λ 1. The relations between these defining the various possible behaviors are subtle, and we have given these in the relevant sections of our analysis. The appearance of both unimodal and bimodal distributions of molecular constituents as well as what we have termed Bifurcation Type 1 and Bifurcation Type 2 have been extensively discussed in the applied mathematics literature (c.f. [64], [37] and others) and the bare foundations of a stochastic bifurcation theory have been laid down by [5]. Significantly, these are also well documented in the experimental literature as has been shown by many authors [43, 2, 39, 59, 151, 94, 134] for both prokaryotes and eukaryotes. If the biochemical details of a particular system are sufficiently well characterized from a quantitative point of view so that relevant parameters can be estimated, it may be possible to discriminate between whether these behaviors are due to the presence of bursting transcription/translation or extrinsic noise. 8.5 Ergodicity and explicit convergence rate In this subsection, we want to obtain an explicit convergent rate towards the asymptotic distribution. Such rate may be used experimentally to determine if the observations are at steady-state or not. We will use here probabilistic arguments. We will first present a result that shows exponential ergodicity using a classical Lyapounov criterion argument. Then, we give an explicit lower bound for the convergent rate using a coupling strategy. Here we use the semigroup defined on bounded continuous function. The semigroup associated to the BC1 model (see subsection 8.2) has for strong generator Af ÔxÕ γxf ½ ÔxÕ λôxõ x Ôf ÔyÕ f ÔxÕÕhÔy xõdy, (8.31) where we have assumed, for simplicity, that γôxõ γx is a linear function. Using Lyapounov criteria for stability of Markov processes (for an introduction of this field, see subsection 6.3), it is easy to see that under reasonable assumption such process is exponentially ergodic. Specifically, we have the

116 8 Specific Study of the One-Dimensional Bursting Model 115 b a Proposition 39. Suppose x λôxõ is continuous on Ö, Õ, λôõ, γôxõ γx, hôdyõ for all a b and that λôxõe h lim 1, (8.32) x γx then it exists β 1, B and π (invariant measure) such that ÐP Ôt,x, Õ πð V BV ÔxÕβ t, x È E, t, where еРf sup gf µôgõ and V ÔxÕ x 1. où еРf sup gf µôgõ. ÐP Ôt,x, Õ πð V BV ÔxÕβ t, x È E, t, Proof. We are going to use the criterion given by [97, thm 6.1] (see part, subsection 6.3, proposition 14). We first show that every compacts set are petite, and then exhibits a Lyapounov function that satisfy the drift condition. To show that all compact sets are petite, we show that the stochastic process is a T-process, and use [97, prop 4.1] (see part, subsection 6.3, proposition 9). We first show that the bursting process ÔX t Õ t is a T-process. Starting at x at time t, the transition function satisfies, at time t 1, for any set B È BÔRÕ, P Ô1,x,BÕ P X 1 È B,T 1 1 (8.33) where T 1 is the first instant time. Now, conditioning by the fact that T 1 1, we have Hence, we deduce λôx t Õ max λôyõ. (8.34) yèöxe γ,x P Ô1,x,BÕ e λx δ xe γ ÔBÕ : T Ôx,BÕ (8.35) where λ x max yèöxe γ,x λôyõ. By definition, X is then a T-process (with a δ 1 ). Finally, let us exhibits a Lyapounov function that satisfy the drift condition. Take V ÔxÕ x 1 in (8.31), we have AV ÔxÕ γx λôxõe h γ 1 λôxõe h V ÔxÕ γ, γv ÔxÕ so that due to condition (8.32), V is a Lyapounov function. The above criterion states that the stochastic process generated by eq. (8.31) is exponentially ergodic, with more general condition in h (but with γôxõ γx linear) than in subsection 8.2. However the convergent rate is still not explicit. For that, we are going to use a coupling technique and get an explicit convergence rate in Wasserstein distance. Let us remark that if we take fôxõ x p in eq. (8.31), we get p Ax p γpx p λôxõ Ôx yõ p hôyõdy x Then if λôxõ λ λ 1 x, we have, for p 1, Ax λ EÖh Ôγ λ 1 EÖh Õx

117 116 Hybrid Models to Explain Gene Expression Variability so that the first moment is exponentially convergent with speed Ôγ λ 1 EÖh Õ as soon as γ λ 1 EÖh. All p-moment are similarly exponentially convergent if h has finite p-moment. Now if λ, the first moment is exponentially convergent towards. This suggest that the difference between two stochastic processes generated by eq. (8.31), with a well-chosen coupling, goes to exponentially fast with an explicit speed. The p-wasserstein distance is defined by W p Ôµ 1,µ 2 Õ inf EÔ X Y p Õ 1ßp, ÔX,Y ÕÈMargÔµ 1,µ 2 Õ We can then prove the Theorem 4. Suppose λ is globally Lipschitz with Lipschitz constant Λ. If Λ γ E h, then for any µ, ν, we have W 1 ÔµP t,νp t Õ e Ôγ ΛE hõt W 1 Ôµ,νÕ. Proof. We follow similar ideas as [7]. For any x,y, we define X x t and Y y t the stochastic processes that starts at x and y and whose coupling generator is defined by Lf Ôx,yÕ Ô γx x f Ôx,yÕ γy y f Ôx,yÕÕ minôλôxõ, λôyõõ Ôf Ôx z,y zõ f Ôx,yÕÕhÔzÕdz λôxõ λôyõ Ôf Ôx z,yõ1 ØλÔxÕλÔyÕÙ f Ôx,y zõ1 ØλÔyÕλÔxÕÙ, ÕhÔzÕdz f Ôx,yÕ (8.36) that is, Xt x and Y y t jump together as most as they can, and the one that has a higher jump rate jumps alone occasionally. With f Ôx,yÕ x y : u, the drift part of the generator gives (first line of eq. 8.36) γu. The second line vanishes, and, by the triangle inequality and hypothesis on λ, the third one is dominated by Hence, Λu Lu γu gôu Λu zõhôzõdz gôuõ and the calculus on moment bounds above show that Ôu zõhôzõdz u, E X x t Xy t e Ôγ ΛE hõt x y which achieves the proof, by the definition of the Wasserstein distance. Remark 41. This coupling strategy can be adapted to get an explicit convergence rate in total variation distance (see [7]). Remark 42. The same demonstration holds for the discrete model as well. 8.6 Inverse problem In subsection 8.2, we have shown that for any set of parameters function γôxõ,λôxõ,h that satisfies particular assumption, then there exists a unique invariant density for the evolution equation, eq. (8.11). Let us summarize our condition,.

118 8 Specific Study of the One-Dimensional Bursting Model 117 Proposition 43. Assume h is an exponential distribution of mean parameter b, γ is a positive continuous function on Ô, Õ, λ a non-negative measurable function on Ô, Õ such that λ γ is locally integrable. Denote x λôyõ QÔxÕ x γôyõ dy, and, suppose that for some δ,r, δ 1 dx, γôxõ lim supγôxõ, x δ γôxõ r 1 dx, δ λôxõ dx, γôxõ e QÔxÕ lim x γôxõ r, λôxõ lim x γôxõ 1 b, then there exists a unique globally attractive invariant density for eq. (8.11) given by We can invert these property to obtain u ÔxÕ 1 cγôxõ e xßb QÔxÕ Proposition 44. Assume h is an exponential distribution of mean parameter b, γ is a positive continuous function on Ô, Õ, and u is an integrable positive function such that for some δ,r, δ 1 dx, γôxõ lim supγôxõ, x δ γôxõ r 1 dx, δ u ½ ÔxÕ γ ½ ÔxÕ dx, uôxõ γôxõ uôxõ lim x γôxõ r 1, u ½ ÔxÕ γ ½ ÔxÕ lim, x uôxõ γôxõ then the function λ defined by λôxõ 1 b γôxõ ÔγÔxÕuÔxÕÕ ½, (8.37) uôxõ is such that the function u is the invariant density for eq. (8.11) associated with h,γ,λ.

119 118 Hybrid Models to Explain Gene Expression Variability Proof. We need to invert the operator given by x dô γôxõuôxõõ dx λôxõuôxõ λôx yõuôx yõhôyõdy. Taking Laplace transform, and noting that by assumption lim xγôxõuôxõ, we obtain so that LÔλuÕÔsÕLÔh δ ÕÔsÕ slôγuõôsõ, LÔλuÕÔsÕ Ôs 1 b ÕLÔγuÕÔsÕ. By inverting the Laplace transform, we get eq. (8.37). That such λ satisfies all the properties of proposition 43 follows then by the assumption and the formula eq. (8.37). A series of remark follows. Remark 45. The assumption on admissible density u of the last proposition 44 are simply integrability condition in and exponential decay at, that can be seen from the analytical expression eq. (8.19). The result given below could have been more easily obtained by the derivation of ÔγÔxÕuÕ thanks to analytical expression eq. (8.19). However, the demonstration given here show that such inversion of the operator is not restricted to exponential jump distribution, as long as we know its Laplace transform. Hence, to be applicable for more general jump distribution, characterization of the stationary state and convergence condition of the direct problem needs to be investigated for general jump distribution. Remark 46. In practice, the formula eq. (8.37) has been shown to be tractable by using for example statistical kernel estimator of the density. The difficulty relies in estimating properly the derivatives of such function. The authors in [28] have shown statistical estimator bounds in a similar problem (for the aggregation-fragmentation problem). Estimates of the jump rate function will then be accurate in domain where the density is not near. Remark 47. Such inverse formula may have a great interest to analyze experimental data. Indeed, from the jump rate function, it is possible to guess the mechanism involved in the regulation (see for instance section 3), which is not necessarily observable experimentally. From the result in proposition 44, it can be deduced the jump rate function λôxõ if we have experimental observations in steady-state and if the other parameters γôxõ and b are known. As the steady-state is invariant by a time scale change, we cannot deduce all parameters from steady-state observations. The degradation function is however usually well caracterized experimentally using knock-out experiments. In the absence of regulation, the result in paragraph shows that, at steady-state, b VarÔXÕ X. Such relation between asymptotic moments were previously used to deduce parameter fitting in different models of gene regulation (see [14, 12]). In the presence of regulation there s no simple formula to find back the mean burst size parameter b. However, if λôxõ is assumed to be bounded, the mean burst size parameter can be found using the tail of the asymptotic probability distribution. Indeed, from the analytical expression eq. (8.19), we see that b lim x xßlogôu ÔxÕÕ

120 9 From One Model to Another From One Model to Another In this section, we are going to prove how all the models presented in section 7 are linked within each other. Briefly, the switching dynamic can lead either to an averaging behavior (if both activation and inactivation rate goes to infinity within the same order, see paragraph 9.1.1) or to a bursting behavior(large jumps appear) (if the inactivation rate and the synthesis rate go to infinity within the same order, see paragraph 9.1.2). However, the switching dynamic is not the only possible scenario to lead to bursting behavior. In the discrete state space model, the adiabatic reduction of mrna can lead to a bursting production of protein, in a similar manner than the switching model actually (see subsection 9.2). Finally, this bursting behavior can be averaged through the different variables or transmitted (when the degradation rate of a variable go to infinity, see subsection 9.3). We will make extensively use of the notation of section 7 for naming each model and its parameter. These limiting behavior are well known of modelers and experimentalists. The review paper of Kaern et al. [74] details assumptions for the ODE C2 to be a good approximation of SC2 (macrocscopic limit and fast switching kinetics), and the kinetics assumption that lead from SC2 to transcriptional bursting BD2 and translational bursting SBD1. The authors in [77] show how to take advantage of specific limiting behavior of the SD1 model (fast operator fluctuation, and large quantity of molecules) to rigorously study its qualitative behavior (bifurcation, escape time), and extend their method to the mutual repressor system. The authors in [87] considered similar techniques and validate these approximations by numerical simulations. Importantly, the authors in [115] reported that different genes in eukaryotes can have different kinetics, so that each limiting model can be applicable to different gene kinetics. On a more theoretical side, the author in [12] used a semi-group theoretical proof to showtheaveraging reductionof model SC2to C2, andtheadiabatic reductionfrom SC2to SC1. Theauthors in [25, 75] give clues to deriverigorously limiting model in the context of stochastic hybrid model. We recall the available reduction results of the switching model in the first subsection 9.1 and rely on them to extend it to the 2-dimensional variable model, in the discrete state space model in subsection 9.2 and in the continuous state space model in subsection 9.3. In this last case, we derived alternative proofs, based either on partial differential equation and on probabilistic techniques. These have been the subject of a preprint [92]. It is important to mention that the theoretical and rigorous justification of the reduction of a given model towards a bursting limit model actually follows natural ideasthatareusedbymanyauthorstoobtainasimplifiedmodel. Forinstance, theauthors in [66] show that different extensions of the standard model of gene expression (without regulation) all leads to bursting model with geometric jump size distribution, basically reasoning by how many proteins can be produced before mrna is degraded. Firsly, this reasoning suggests that such reduction is a general framework of catalytic reaction, where the reactant is needed for the reaction to occur, but is not consumed by the reaction (so that a new reaction may happen directly). The identification of the limit martingale problem we performed in subsections 9.2 and 9.3 uses a test function that exactly matches with the heuristic above. The idea is to follow the catalytic reaction up to the time the reactant is consumed. See also[129] where the authors used a reduction technique based on the characteristic method associated to the evolution equation of the moment generating equation. Again, in such models, the characteristic method exactly follows the production of the second variable up to the time the first variable vanishes. Finally, we show in subsection 9.4 how the links between the discrete and the continuous

121 12 Hybrid Models to Explain Gene Expression Variability bursting model, using well known fluid limit techniques ([36]). 9.1 Limiting behavior of the switching model Averaging results In the context of model of gene expression, the author in [12] used a result on degenerate convergence of semigroup to show the averaging reduction of model SC2 to C2. The degeneracy means here that the limiting semigroup act on a proper subspace of the starting space. The author considered the special (but biologically natural) case where the transcriptional rate function k 1 is a constant function. In such case, the deterministic part of the model can be solved exactly. But its main advantage is in fact that in such case the dynamics is constrained in a compact subset. Hence, this result could easily be extended to the case where k 1 is a smooth bounded function. With k i and k a continuous function, which are then bounded on compact set, the semigroup acting on continuous function of the full model can be constructed by the Philipps perturbation theorem (see [35]) from the deterministic semigroup. The obtained semigroup is a Feller semigroup. We rewrite the limiting theorem with our notation (section 7) below, for the reduction from SC1 (see paragraph. 7.2) to C1 (see paragraph 7.4) (which has obvious extension to 2 and 3 variables). Theorem 48. Bobrowski [12, Theorem 2 p. 356] Assume k 1 is a continuous Lipschitz on R and bounded. Then there exists a compact subset K R such that x 1 ÔtÕ È K for all t as soon as x 1 ÔÕ È K. Assume k a and k i are continuous Lipschitz functions, positive such that one of them is strictly positive. Let λ n a and λ n i sequences of positive numbers such that lim n λn a lim n λn i lim n λ n a λ n i c. For any continuous function f,g on K, i È Ø,1Ù, x È K, and t, let T n ÔtÕÔf,gÕÔi,xÕ : E Ôi,xÕ f Ôx1 ÔtÕÕ1 ØX ÔtÕÙ gôx 1 ÔtÕÕ1 ØX ÔtÕ1Ù the semigroup acting on continuous function associated to any solution of SC1 (see paragraph. 7.2), starting at Ôi,xÕ, with parameters λ n a and λn i. Similarly, write T ÔtÕÔf ÕÔxÕ the semigroup defined by C1 (see paragraph 7.4), with k 1 being replaced by λ 1 Then, using norm of uniform convergence, For any continuous function f on K, ck a Ôx 1 Õ ck a Ôx 1 Õ k i Ôx 1 Õ k 1Ôx 1 Õ lim n Tn ÔtÕÔf,f Õ T ÔtÕÔf Õ uniformly on time on all compact interval of Ö, Õ. For any continuous function f,g on K, lim n Tn ÔtÕÔf,gÕ T ÔtÕÔQÔf,gÕÕ uniformly on time on all compact interval of Ö, Õ, where QÔf,gÕ ck a k i k i f ck a g ck a k i

122 9 From One Model to Another 121 The analog result given in [25] requires only that k 1 is such that C1 defines a global flow, not necessarily restrict to evolve in a compact. However, their result requires that the fast motion given by the switch defines an ergodic semigroup, exponentially mixing, and uniformlywith respect to the slow variable x 1. Here, it is easy to see that this semigroup is λ ergodic, with unique invariant law given by a Bernoulli law of parameter ak aôx 1 Õ λ ak aôx 1 Õ λ i k i Ôx 1 Õ. Its convergent rate is exponential with rate λ a k a Ôx 1 Õ λ i k i Ôx 1 Õ. Hence, it is needed to supposeadditionally that theseratesareboundedwithrespect tox 1. As before, werewrite the limiting theorem given in [25] with our notation (section 7) below, for the reduction from SC1 to C1 (which has obvious extension to 2 and 3 variable). Theorem 49. Crudu et al. [25, Theorem 5.1 p. 13] Assume k 1 È C 1 ÔR Õ and such that the model in paragraph 7.4 defines a global flow. Assume k a and k i are C 1 on R and bounded, positive such that one of them is strictly positive. Let λ n a nλ a and λ n i nλ i with n. Let ÔXO nôtõ,xn 1 ÔtÕÕ t the stochastic process defined by SC1 (see paragraph 7.2), and Ôx 1 ÔtÕÕ t the solution of C1 (see paragraph 7.4) with k 1 being replaced by λ 1 λ a k a Ôx 1 Õ λ a k a Ôx 1 Õ λ i k i Ôx 1 Õ k 1Ôx 1 Õ Assume x n 1 ÔÕ converges in distribution to x 1ÔÕ in R, then ÔX n O ÔtÕ,xn 1 ÔtÕÕ t converges in distribution to Ôx 1 ÔtÕÕ t in DÔR ;R Õ. The restriction of bounded rate k a and k i in [25] is essentially to ensure that the fast dynamicsstayinacompact insomesense. Here, becausethefastdynamicsisonacompact state space, this assumption can be released easily. The only remaining restrictions are then that the limiting model posses a unique global solution. These results have very analog counterpart in discrete models SD1 and D1. See also [75] for general results on averaging methods Bursting The limit from a switching (SB1,SC1) model to a continuous bursting model (BC1) was treated explicitly in [25] (together with a fluid limit). Now we let λ n i nλ i and λ n 1 nλ 1. Intuitively, the switching variable X n will then spend most of its time in state. However, transition from X n to Xn 1 will still be possible (and will not vanish as n ). Convergence of X n to will hold in L1 Ô,tÕ for any finite finite time t. When X n 1, production of x 1 is suddenlyvery high, but for a brief time. Although x 1 follows a deterministic trajectory, the timing of its trajectory is stochastic. At the limit, this drastic production episode becomes a discontinuous jump, of a random size. All happen as the two successive jumps of X (from to 1 and back to ) coalesce into a single one, and create a discontinuity in x 1. In such case, convergence cannot hold in the cad-lag space DÔR ;R Õ with the Skorohod topology. The authors in [25] were able to prove tightness in L p ÔÖ,T,R Õ, 1 p. Their result requires the additional assumption that all rates k 1,k i and k a are linearly bounded, and either k a or k i is bounded with respect to x 1. This is needed to get a bound on x 1 in L ÔÖ,T,R Õ. The limiting theorem reads Theorem 5. Crudu et al. [25, Theorem 6.1 p. 17] Assume k 1 È C 1 ÔR Õ and let λ n i nλ i and λ n 1 nλ 1 with n. Let ÔX n O ÔtÕ,xn 1 ÔtÕÕ t the stochastic process defined by SC1 (see paragraph 7.2). Assume x n 1 ÔÕ converges in distribution to x 1ÔÕ in R, and X n O ÔÕ converges in distribution to. The reaction rates k 1,k i and k a are such that there exists α such that k i Ôx 1 Õ α for all x 1 ;

123 122 Hybrid Models to Explain Gene Expression Variability there exists M 1 such that k 1 Ôx 1 Õ M 1 Ôx 1 1Õ, k a Ôx 1 Õ M 1 Ôx 1 1Õ, k i Ôx 1 Õ M 1 Ôx 1 1Õ; In addition either k a or k 1 is bounded with respect to x 1. Then ÔX n ÔtÕÕ t converges in distribution to in L 1 ÔÖ,T, Ø,1ÙÕ and Ôx n 1 ÔtÕÕ t converges in distribution to the stochastic process whose generator is given by ϕ AϕÔx 1 Õ γ 1 x 1 x 1 λ a k a Ôx 1 Õ ϕôφ 1 Ôt,x 1 ÕÕ ϕôx 1 Õ λ i k i Ôφ 1 Ôt,x 1 ÕÕe t λ ik i Ôφ 1 Ôs,x 1 ÕÕds dt, (9.1) for every ϕ È C 1 b ÔR Õ and where φ 1 Ôt,x 1 Õ is the flow associated to x λ 1 k 1 ÔxÕ, xôõ x 1. Analogous result on the SD1 model holds as well. The fact that this limiting model is indeed related to BC1 is now detailed in the three following examples. Example 1. Consider the special case where both regulation rates k 1 and k i are constant, with k 1 Ôx 1 Õ k i Ôx 1 Õ 1, for all x 1. Then the flow φ 1 is easily calculated and we have t and the generator eq. (9.1) becomes AϕÔx 1 Õ γ 1 x 1 ϕ x 1 φ 1 Ôt,x 1 Õ x 1 λ 1 t, t, λ i k i Ôφ 1 Ôs,x 1 ÕÕds λ i t, λ a k a Ôx 1 Õ ϕôx 1 zõ ϕôx 1 Õ λi e λ i λ 1 λ 1 z dz, which is the BC1 model, with an exponential jump size distribution of mean parameter λ 1 λ i. Such rate has an easy interpretation, being the number of molecules created during an ON period of the gene. Other choice of regulation rate leads to different model, as illustrated in the next two examples. Example 11. Let k 1 1 and λ i k i Ôx 1 Õ λ i x 1 k (linear negative regulation), so that t and the generator eq. (9.1) becomes AϕÔx 1 Õ γ 1 x 1 ϕ x 1 φ 1 Ôt,x 1 Õ x 1 λ 1 t, t, λ i k i Ôφ 1 Ôs,x 1 ÕÕds Ôλ i x 1 λ a k a Ôx 1 Õ x 1 k Õt λ 1 λ i 2 t2, ϕôzõ ϕôx 1 Õ λi z k e z x 1 z x λ 1 λ i k 1 2 dz. λ 1 The limiting model is then a bursting model where the jump distribution is a function of the jump position, and has a Gaussian tail.

124 9 From One Model to Another 123 Example 12. Let k 1 Ôx 1 Õ x 1 and k i Ôx 1 Õ 1 (positive linear regulation), so that t and the generator eq. (9.1) becomes AϕÔx 1 Õ γ 1 x 1 ϕ x 1 φ 1 Ôt,x 1 Õ x 1 e λ 1t, t, λ i k i Ôφ 1 Ôs,x 1 ÕÕds λ i t, λ a k a Ôx 1 Õ x 1 ϕôzõ ϕôx 1 Õ λi λ 1 x λi λ 1 z 1 λ i λ 1 dz. This time, the limiting model is a bursting model where the jump distribution is a function of the jump position with a power-law tail. 9.2 A bursting model from a two-dimensional discrete model The fact that bursting models arise as a reduction procedure of a higher dimensional model was already observed in [129]-[25]. In [129], the authors show that, within an appropriate scaling, the time-dependent distribution of a 2-dimensional model converge to the time-dependent distribution of a 1-dimensional bursting model. The authors used analytics methods through the transport equation on the generating function. Their result seems to be restricted to first-order kinetics. The first variable is a fast variable that induces infrequent kicks to the second one. In [25], the authors show that, within an appropriate scaling, a fairly general discrete state space model with a binary variable converge to a bursting model with continuous state space. The authors obtained a convergence in law of the solution through martingale techniques. The binary variable is a fast variable that induces kicks to the other variable. We present below analogous result of [25] when the fast variable is similar to the one of [129]. These results are more precise than the one of [129], and more general (some kinetics rates can be non-linear). We used martingales techniques, with a proof that is similar to [25] and also inspired by results from [75]. We consider the following 2d stochastic kinetic chemical reaction model, that generalizes the D2 model (see paragraph 7.3) À λ 1k 1 ÔX 1,X 2 Õ X 1, Production of X 1 at rate λ 1 k 1 ÔX 1,X 2 Õ (9.2) X 1 γ 1 ÔX 1,X 2 Õ À, Destruction of X 1 at rate γ 1 ÔX 1,X 2 Õ (9.3) À λ 2k 2 ÔX 1,X 2 Õ X 2, Production of X 2 at rate λ 2 k 2 ÔX 1,X 2 Õ (9.4) X 2 γ 2 ÔX 1,X 2 Õ À, Destruction of X 2 at rate γ 2 ÔX 1,X 2 Õ (9.5) with γ 1 Ô,X 2 Õ γ 2 ÔX 1,Õ to ensure positivity. This model can be represented by a continuous time Markov chain in N 2, and is then a general random walk in N 2. It can be described by the following set of stochastic differential equations X 1 ÔtÕ X 1 ÔÕ X 2 ÔtÕ X 2 ÔÕ Y 1 t Y 3 t λ 1 k 1 ÔX 1 ÔsÕ,X 2 ÔsÕÕds λ 2 k 2 ÔX 1 ÔsÕ,X 2 ÔsÕÕds t Y 2 t Y 4 γ 1 ÔX 1 ÔsÕ,X 2 ÔsÕÕds γ 2 ÔX 1 ÔsÕ,X 2 ÔsÕÕds where Y i, for i are independent standard Poisson processes. The generator of this,,

125 124 Hybrid Models to Explain Gene Expression Variability process has the form BfÔX 1,X 2 Õ λ 1 k 1 ÔX 1,X 2 Õ γ 1 ÔX 1,X 2 Õ λ 2 k 2 ÔX 1,X 2 Õ γ 2 ÔX 1,X 2 Õ fôx 1 1,X 2 Õ fôx 1,X 2 Õ fôx 1 1,X 2 Õ fôx 1,X 2 Õ fôx 1,X 2 1Õ fôx 1,X 2 Õ fôx 1,X 2 1Õ fôx 1,X 2 Õ, (9.6) for every bounded function f on N 2. Example 13. We obviously have in mind the mrna-protein system given by the D2 model defined in paragraph 7.3, where γ i ÔX 1,X 2 Õ γ i X i, k 2 ÔX 1,X 2 Õ X 1 and k 1 ÔX 1,X 2 Õ k 1 ÔX 2 Õ. We suppose the following scaling holds γ N 1 ÔX 1,X 2 Õ Nγ 1 ÔX 1,X 2 Õ, λ N 2 Nλ 2, where N that is reactions eq. (9.3)- (9.4) occur at a faster time scale than the two other reactions. Then X 1 is degraded very fast, and induces also as a very fast production of X 2. The rescaled model is given by t t X1 N ÔtÕ XN 1 ÔÕ Y 1 λ 1 k 1 ÔX1 N ÔsÕ,XN 2 ÔsÕÕds Y 2 Nγ 1 ÔX1 N ÔsÕ,XN 2, ÔsÕÕds t t X2 N ÔtÕ XN 2 ÔÕ Y 3 Nλ 2 k 2 ÔX1 N ÔsÕ,XN 2 ÔsÕÕds Y 4 γ 2 ÔX1 N ÔsÕ,XN 2, ÔsÕÕds and the generator of this process has the form B N fôx 1,X 2 Õ λ 1 k 1 ÔX 1,X 2 Õ Nγ 1 ÔX 1,X 2 Õ Nλ 2 k 2 ÔX 1,X 2 Õ γ 2 ÔX 1,X 2 Õ We can prove the following reduction holds: Theorem 51. We assume that fôx 1 1,X 2 Õ fôx 1,X 2 Õ fôx 1 1,X 2 Õ fôx 1,X 2 Õ fôx 1,X 2 1Õ fôx 1,X 2 Õ fôx 1,X 2 1Õ fôx 1,X 2 Õ 1. The degradation function on X 2 satisfies γ 2 ÔX 1,Õ. 2. The degradation function on X 1 satisfies γ 1 Ô,X 2 Õ, and inf γ 1ÔX 1,X 2 Õ γ. X 1 1,X 2 3. The production rate of X 2 satisfies k 2 Ô,X 2 Õ. 4. The production rate function k 1 and k 2 are linearly bounded by X 1 X Either k 1 or k 2 is bounded.. (9.7) (9.8)

126 9 From One Model to Another 125 et ÔX1 N,XN 2 Õ the stochastic process whose generator is B N (defined in eq. (9.8)). Assume that the initial vector ÔX1 N ÔÕ,XN 2 ÔÕÕ converge in distribution to Ô,XÔÕÕ, as N. Then, for all T, ÔX1 N ÔtÕ,XN 2 ÔtÕÕ t converge in L 1 Ô,T Õ (and in L p, 1 p ) to Ô, XÔtÕÕ where XÔtÕ is the stochastic process whose generator is given by B ϕôxõ λ 1 k 1 Ô,XÕ where P t Ôγ 1 Ô1,.ÕϕÔ.ÕÕÔXÕdt ϕôxõ γ 2 Ô,XÕ P t gôxõ E gôy Ôt,XÕe t γ 1Ô1,Y Ôs,XÕÕds, ϕôx 1Õ ϕôxõ and Y Ôt,XÕ is the stochastic process starting at X at t whose generator is given by AgÔY Õ λ 2 k 2 Ô1,Y Õ gôy 1Õ gôy Õ., (9.9) Remark 52. The first three hypotheses of theorem 51 are the main characteristics of the mrna-protein system (see paragraph 7.3). Basically, they impose that quantities remains non-negative, that the first variable has always the possibility to decrease to (no matter the value of the second variable), and that the second variable cannot increase when the first variable is. Hence these three hypotheses will guarantee that (with our particular scaling) the first variable converge to, and will lead to an intermittent production of the second variable. The last two hypotheses are more technical, and guarantee that the Markov chain is not explosive, and hence well defined for all t, and that the limiting model is well defined too. We divide the proof in several steps. step 1: moment estimates Because production rates are linearly bounded, it is straightforward that with fôx 1,X 2 Õ X 1 X 2 in eq. (9.8), there is a constant C N (that depends on N and other parameters) such that Then E X N 1 ÔtÕ is a L 1 -martingale. B N fôx 1,X 2 Õ C N ÔX 1 X 2 Õ. XN 2 ÔtÕ is bounded on any time interval Ö,T and t fôx1 N ÔtÕ,XN 2 ÔtÕÕ fôxn 1 ÔÕ,XN 2 ÔÕÕ B N fôx1 N ÔsÕ,XN 2 ÔsÕÕds step 2: tightness Clearly, from the stochastic differential equation on X1 N, we must have X1 N ÔtÕ. We can show in fact that the Lebesgue measure of the set Øt T : X1 NÔtÕ Ù converge to. Indeed, taking fôx 1,X 2 Õ X 1 in eq. (9.8), we have X N 1 ÔtÕ X N 1 ÔÕ t Ôλ 1 k 1 ÔX N 1 ÔsÕ,X N 2 ÔsÕÕ Nγ 1 ÔX N 1 ÔsÕ,X N 2 ÔsÕÕÕds (9.1) is a martingale. Thanks to the lower bound assumption on γ 1, we have γe t 1 ØX N 1 ÔsÕ1Ù ds E Then, by the martingale property, we deduce t t ds ÔÕ t γne 1 ØX N 1 ÔsÕ1Ù E X1 N λ 1 γ 1 ÔX N 1 ÔsÕ,XN 2 ÔsÕÕds. E k 1 ÔX N 1 ÔsÕ,XN 2 ÔsÕÕ ds. (9.11)

127 126 Hybrid Models to Explain Gene Expression Variability And for X N 2 we obtain from the the eq. (9.7), X N 2 ÔtÕ XN 2 ÔÕ Y 3 t Let us now distinguish between the two cases. If k 2 is bounded (say by 1), we have E X N 2 ÔtÕ E X N 2 ÔÕ λ 2 N1 ØX N 1 ÔsÕ1Ù k 2ÔX N 1 ÔsÕ,XN 2 ÔsÕÕds. λ 2 NE t 1 ØX N 1 ÔsÕ1Ù ds. As k 1 is linearly bounded (say by 1) by X1 N XN 2, the upper bound eq. (9.11) becomes t ds t γne 1 ØX N 1 ÔsÕ1Ù E X1 N ÔÕ λ 1 Finally, with eq. (9.1), it is clear that E X N 1 ÔtÕ E X N 1 ÔÕ λ 1 t E X N 1 ÔsÕ E X N 2 ÔsÕ ds. E X N 1 ÔsÕ E X N 2 ÔsÕ ds. Hence, with the three last inequalities, we can conclude by the Grönwall lemma that E X N 2 ÔtÕ is bounded on Ö,T, uniformly in N. Then NE T 1 ØX N 1 ÔsÕ1Ù ds is bounded and X N 1 in L 1 ÔÖ,T,NÕ. By the law of large number, 1 N Y 3ÔNÕ is almost surely convergent, and hence almost surely bounded. We deduce then there exists a random variable C such that X N 2 ÔtÕ XN 2 ÔÕ t NC 1 ØX N ds, 1 ÔsÕ1Ù almost everywhere. By Grönwall lemma and Markov inequality K P sup X2 N ÔtÕ tèö,t as K, uniformly in N. Now if k 1 is bounded (say 1). By the martingale eq. (9.1) (and the same lower bound hypothesis on γ 1, it is clear that NE T 1 ØX N 1 ÔsÕ1Ù ds is bounded and X N 1 in L 1 ÔÖ,T,NÕ. Now, let us denote U N ÔtÕ 1 N XN 1 ÔtÕ, V N 1 N XN 2 ÔtÕ and WN N1 ØX N 1 ÔtÕ1Ù (which is then bounded in L1 ÔÖ,T ÖÕ). From eq. (9.7), and from the linear bound on k 2 (say by 1) V N ÔtÕ V N ÔÕ 1 N Y 3 t λ 2 NW N ÔU N ÔsÕ V N ÔsÕÕds Then, still by the law of the large number there exists a random variable C such that V N ÔtÕ V N ÔÕ C t W N ÔU N ÔsÕ V N ÔsÕÕds,.

128 9 From One Model to Another 127 and hence By Grönwall lemma, X N 2 ÔtÕ XN 2 ÔÕ t C W N ÔX1 N ÔsÕ XN 2. ÔsÕÕds supx2 N ÔtÕ ÔXN 1 ÔÕ XN 2 C ÔÕÕexp Ö,T which is then bounded, uniformly in N. For any subdivision of Ö,T, t t 1 t n T, n 1 ô i X N 2 Ôt i 1Õ X N 2 Ôt iõ t W N ÔsÕds, n 1 ô ti 1 Y 3 λ 2 N1 ØX N 1 ÔsÕ1Ù k 2ÔX1 N ÔsÕ,XN 2 ÔsÕÕds i t i T Y 3 λ 2 N1 ØX N 1 ÔsÕ1Ù k 2ÔX1 N ÔsÕ,X2 N ÔsÕÕds so by a similar argument as above, we also get the tightness of the BV norm (see proposition 23 part ) P ÐX N 2 Ð Ö,T K as K, independently in N. Then X N 2 is tight in Lp ÔÖ,T Õ, for any 1 p. step 3: identification of the limit We choose an adherence value Ô,X 2 ÔtÕÕ of the sequence ÔX N 1 ÔtÕ,XN 2 ÔtÕÕ in L1 ÔÖ,T Õ L p ÔÖ,T Õ. Then a subsequence (again denoted by) ÔX N 1 ÔtÕ,XN 2 ÔtÕÕ converge to Ô,X 2ÔtÕÕ, almost surely and for almost t È Ö,T. We are looking for test-functions such that t fôx1 N ÔtÕ,XN 2 ÔtÕÕ fôxn 1 ÔÕ,XN 2 ÔÕ B N fô,x2 N ÔsÕÕ1 X1 N ÔsÕds t B N fôx N 1 ÔsÕ,XN 2 ÔsÕÕ1 X N 1 ÔsÕ1ds is a martingale and B N fôx1 N ÔsÕ,XN 2 ÔsÕÕ is bounded independently of N when X 1 1. The following choice is inspired by [25]. We introduce the stochastic process Y x,y t, starting at y and whose generator is A x gôyõ λ 2 k 2 Ôx,yÕ gôy 1Õ gôyõ, for any x 1. and we introduce the semigroup Pt x defined on B b ÔR Õ, for any x 1, by Pt x gôyõ E gôy x,y t Õe t γ 1Ôx,Ys x,y Õds. (9.12) Then the semigroup P x t satisfies the equation dpt xgôyõ A x Pt x gôyõ γ 1 Ôx,yÕPt x gôyõ. dt Now for any bounded function g, define recursively fô,yõ gôyõ, fôx,yõ P x t Ôγ 1 Ôx,.ÕfÔx 1,.ÕÕÔyÕdt.

129 128 Hybrid Models to Explain Gene Expression Variability Such a test function is well defined by the assumption on γ 1. We then verify that B N fô,yõ λ 1 k 1 Ô,yÕ Pt 1 Ôγ 1Ô1,.ÕgÔ.ÕÕÔyÕdt gôyõ γ 2 Ô,yÕ gôy 1Õ gôyõ, B N fôx,yõ λ 1 k 1 Ôx,yÕ fôx 1,yÕ fôx,yõ γ 2 Ôx,yÕ fôx,y 1Õ fôx,yõ. Indeed, for any x 1, Then A x fôx,yõ γ 1 Ôx,yÕfÔx,yÕ A x P x t Ôγ 1 Ôx,.ÕfÔx 1,.ÕÕÔyÕ γ 1 Ôx,yÕP x t Ôγ 1 Ôx,.ÕfÔx 1,.ÕÕÔyÕdt, d dt Px t Ôγ 1Ôx,.ÕfÔx 1,.ÕÕÔyÕdt, lim P x t t Ôγ 1 Ôx,.ÕfÔx,.ÕÕÔyÕ γ 1 Ôx,yÕfÔx 1,yÕ, γ 1 Ôx,yÕfÔx 1,yÕ. λ 2 k 2 Ôx,yÕ fôx,y 1Õ fôx,yõ γ 1 Ôx,yÕ fôx 1,yÕ fôx,yõ. Hence B N fôx,yõ is independent of N, and, taking the limit N in we deduce fôx N 1 ÔtÕ,X N 2 ÔtÕÕ fôx N 1 ÔÕ,X N 2 ÔÕÕ is a martingale where B gôyõ λ 1 k 1 Ô,yÕ gôx 2 ÔtÕÕ gôx 2 ÔÕÕ t t B N fôx N 1 ÔsÕ,X N 2 ÔsÕÕds, B gôx 2 Õ P t Ôγ 1 Ô1,.ÕgÔ.ÕÕÔyÕdt gôyõ γ 2 Ô,yÕ gôy 1Õ gôyõ. Uniqueness Duetoassumptiononk 1 andk 2, thelimitinggenerator definesapure-jump Markov process in N which is not explosive. Uniqueness of the martingale then follows classically. Remark 53. The above expression eq. (9.9) is a generator of a bursting model for a general bursting size distribution. For instance, for constant function γ 1, and k 2 1, we have P t Ôγ 1 Ô.ÕϕÔ.ÕÕÔpÕ γ 1 P t ÔϕÕÔpÕ, γ 1 E ϕôy y t Õe γ 1t, ô γ 1 e γ 1t ϕôzõp Y y t z, zy γ 1 e γ 1t ô zy ϕôzõ Ôλ 2tÕ z y e λ 2t. Ôz yõ! It follows by integration integration by parts that P t Ôγ 1 Ô.ÕϕÔ.ÕÕÔyÕdt γ ô 1 λ 2 z, ϕôz yõ γ 1 λ 2 λ 2 γ 1 which gives then an additive geometric burst size distribution of parameter p λ 2 λ 2 γ 1, as expected. z

130 9 From One Model to Another Adiabatic reduction in a bursting model In continuous dynamical systems, considerable simplifications and insights into the behavior can be obtained by identifying fast and slow variables. This technique is especially useful when one is initially interested in the approach to a steady state. In this context a fast variable is one that relaxes much more rapidly to a conditional equilibrium than a slow variable [54]. In many systems, including chemical and biochemical ones, this is often a consequence of differences in degradation rates, with the fastest variable the one that has the largest degradation rate. We employ this strategy here to obtain approximations to the two-dimensional bursting model BC2 as a one-dimensional bursting model BC1. The adiabatic reduction technique gives results that justifies to reduce the dimension of asystemandtouseaneffectivesetofreducedequationsinlieuofdealingwithafull, higher dimensional model. This techniques essentially requires that different time scales occur in the system. Adiabatic reduction results for deterministic systems of ordinary differential equations have been available since the very precise results of [143] and [38]. The simplest results, in the hyperbolic case, give an effective construction of an uniformly asymptotically stable slow manifold(and hence a reduced equation) and prove the existence of an invariant manifold near the slow manifold, with (theoretically) any order of approximation of this invariant manifold. Such precise and geometric results have been generalized to random systems of stochastic differential equation with Gaussian white noise ([1], see also [42] for previous work on the Fokker-Planck equation). However, to the best of our knowledge, analogous results for stochastic differential equations with a jump process have not been obtained. We recall how this strategy works in ordinary differential equation, and specially in the model we consider. It is often the case that the degradation rate of mrna is much greater than the corresponding degradation rates for both the intermediate protein and the effector Ôγ 1 γ 2,γ 3 Õ so in this case the mrna dynamics are fast and we have from eq. (6.2) the relationship κ d f Ôy 3 Õ y 1. It is easy to see that such relation defines a uniformly asymptotically stable slow manifold (with eigenvalue 1). Consequently the three variables system describing the generic operon reduces to a two variables one involving the slower intermediate and effector: dy 2 dt γ 2Öκ d f Ôy 3 Õ y 2, (9.13) dy 3 dt γ 3Ôy 2 y 3 Õ. (9.14) In our considerations of specific single operon dynamics below we will also have occasion to examine two further sub-cases, namely Case 1. Intermediate (protein) dominated dynamics. If it should happen that γ 1 γ 3 γ 2 (as for the lac operon), then the effector also qualifies as a fast variable so y 2 y 3, and thus from eq.(9.13) and (9.14) we recover the one dimensional equation for the slowest variable, the intermediate: dy 2 dt γ 2Öκ d f Ôy 2 Õ y 2. Case 2. Effector (enzyme) dominated dynamics. Alternately, if γ 1 γ 2 γ 3 then the intermediate is a fast variable relative to the effector and we have κ d f Ôy 3 Õ y 2,

131 13 Hybrid Models to Explain Gene Expression Variability so our two variable system eq. (9.13) and (9.14)) reduces to a one dimensional system dy 3 dt γ 3Öκ d f Ôy 3 Õ y 3. for the relatively slow effector dynamic. The present section gives a theoretical justification of an adiabatic reduction of a particular piecewise deterministic Markov process (and has been the subject of a preprint [92]). The results we obtain do not give a bound on the error of the reduced system, but they do allow us to justify the use of a reduced system in the case of a piecewise deterministic Markov process. In that sense, the results are close to the recent ones by [25] and [75], where general convergence results for discrete models of stochastic reaction networks are given. In particular, these papers give alternative scaling of the traditional ordinary differential equation and the diffusion approximation depending on the different scaling chosen (see [6] for some examples in a reaction network model). After the scaling, the limiting models can be deterministic (ordinary differential equation), stochastic (jump Markov process), or hybrid (piecewise deterministic process). For illustrative and motivating examples given by a simulation algorithm, see [55, 114, 5]. Our particular model is meant to describe stochastic gene expression with explicit bursting [39]. The variables evolve under the action of a continuous deterministic dynamical system interrupted by positive jumps of random sizes that model the burst production. In that sense, the convergence theorems we obtain in this paper can be seen as an example in which there is a reaction with size between and, and give complementary results to those of [25] and [75]. We hope that the results here are generalizable to give insight into adiabatic reduction methods in more general stochastic hybrid systems [6, 18]. We note also that more geometrical approaches have been proposed to reduce the dimension of such systems in [17] Continuous-state bursting model The models referred to above have explicitly assumed the production of several molecules instantaneously, through a jump Markov process, in agreement with experimental observations. In line with experimental observations, it is standard to assume a Markovian hypothesis (an exponential waiting time between production jumps) and that the jump sizes are exponentially distributed (geometrically in the discrete case) as well. The intensity of the jumps can be a linearly bounded function, to allow for self-regulation. Let x 1 and x 2 denote the concentrations of mrna and protein respectively. A simple model of single gene expression with bursting in transcription is given by (SC2 model) dx 1 dt dx 2 dt γ 1 x 1 NÔh,λ1 k 1 Ôx 2 ÕÕ, (9.15) γ 2 x 2 λ 2 x 1. (9.16) Here γ 1 and γ 2 are the degradation rates for the mrna and protein respectively, λ 2 is the mrna translation rate, and NÔh,λ 1 k 1 Ôx 2 ÕÕ describes the transcription that is assumed to be a compound Poisson white noise occurring at a rate λ 1 k 1 Ôx 2 Õ with a non-negative jump size x 1 distributed with density h. The eq. (9.15) and (9.16) are a short hand notation for t x 1 ÔtÕ x 1 γ 1 x 1 Ôs Õds t x 2 ÔtÕ x 2 γ 2 x 2 Ôs Õds t t 1 Ørλ1 k 1 Ôx 2 Ôs ÕÕÙ zn Ôds,dz,drÕ, (9.17) λ 2 x 1 Ôs Õds. (9.18)

132 9 From One Model to Another 131 where X s lim ts XÔtÕ, and NÔds,dz,drÕ is a Poisson random measure on Ô, Õ Ö, Õ 2 with intensity dshôzõdzdr, where s denotes the times of the jumps, r is the statedependency in an acceptance/rejection fashion, and z the jump size. Note that Ôx 1 ÔtÕÕ is a stochastic process with almost surely finite variation on any bounded interval Ô, T Õ, so that the last integral is well defined as a Stieltjes-integral. Hypothesis 8. The following discussion is valid for general rate functions k 1 and density functions hô Õ that satisfy k 1 È C 1, k 1 is globally Lipschitz and linearly bounded with h È C and xhôxõdx. k 1 ÔxÕ c k 1 x. For a general density function h, we denote the average burst size by b xhôxõdx. (9.19) If k 1 1 is independent of the state x 2, the average transcription rate is bλ 1, and the asymptotic average mrna and protein concentrations are x eq 1 : E x 1 Ôt Õ bλ 1 γ 1, x eq 2 : E x 2 Ôt Õ λ 2 γ 2 x eq 1 bλ 1λ 2 γ 1 γ 2. (9.2) Statement of the results In the following discussion, we consider the situation when mrna degradation is a fast process, i.e. γ 1 is large enough, but the average protein concentration x eq 2 remains unchanged. In what follows, we denote by γ1 n, λn 1, λn 2 sequences of parameters, and hn sequence of density function that will replace γ 1, λ 1, λ 2, h in eq. (9.17)- (9.18). We then denote Ôx n 1,xn 2 Õ its associated solution. We will always assume one of the following three scaling relations: (S1) Frequent production rate of mrna, namely γ n 1 nγ 1, λ n 1 nλ 1, and λ n 2 λ 2 h n h are independent of n; (S2) Large burst of mrna, namely γ n 1 nγ 1, h n ÔzÕ 1 n hôz n Õ and λn 1 λ 1,λ n 2 λ 2 remain unchanged; (S3) Large production rate of protein, namely γ n 1 nγ 1, λ n 2 nλ 2, and λ n 1 λ 1 h n h are independent of n; In this section we determine an effective reduced equation for eq. (9.16) for each of the three scaling conditions (S1)-(S3). In particular, we show that under assumption (S1), eq. (9.16) can be approximated by the deterministic ordinary differential equation where dx 2 dt γ 2x 2 λ 2 kôx 2 Õ, (9.21) kôx 2 Õ bλ 1 k 1 Ôx 2 Õßγ 1. We further show that under the scaling relations (S2) or (S3), eq. (9.16) can be reduced to the stochastic differential equation dx 2 dt γ 2x 2 NÔ h,λ1 k 1 Ôx 2 ÕÕ. (9.22)

133 132 Hybrid Models to Explain Gene Expression Variability where h is a suitable density function in the jump size x 2 (to be detailed below). We first explain, using some heuristic arguments, the differences between the three scaling relations and the associated results. When n, γ1 n and applying a standard quasi-equilibrium assumption we have which yields x n 1 ÔtÕ 1 γ n 1 dx n 1 dt and therefore the second eq. (9.16) becomes, NÔh n Ô.Õ,λ n 1 k 1Ôx n 2 ÕÕ NÔγ n 1 hn Ôγ n 1 Õ,λn 1 k 1Ôx n 2 ÕÕ, dx n 2 dt γ 2 x n 2 γ 2 x n 2 N λ n 2 γ1 n γ n 1 NÔh n Ô.Õ,λ n 1k 1 Ôx n 2 ÕÕ, λ n 2 Å h n Ô γn 1 λ n Õ,λ n 1 k 1Ôx n 2 Õ. 2 Hence in eq. (9.22), hôx 2 Õ Ôλ 2 ßγ 1 Õ 1 hôôλ 2 ßγ 1 Õ 1 x 2 Õ under the scaling (S2) and (S3). Furthermore, we note that the scaling (S2) also implies nh n Ôn Õ hô Õ, while in (S1), nh n Ôn Õ nhôn Õ so that the jumps become more frequent and smaller. We denote ÔDÖ, Õ,SÕ the cad-lag function space of function defined on Ö, Õ at values in R with the usual Skorohod topology. Similarly ÔDÖ,T,JÕ is the cad-lag function space on Ö,T, with the Jakubowski topology. Also, L p Ö,T Õ the space of L p integrable function on Ö,T Õ, with T, which we endowed with total variation norm, and MÔ, Õ is the space of real measurable function on Ö, Õ with the metric dôx,yõ Our main results can be stated as follows O e t max1, xôtõ yôtõ dt. Theorem 54. Consider the eq. (9.17)-(9.18) and assume hypothesis 8. If the scaling (S1) is satisfied, i.e., λ n 1 nλ 1, and if x n 2 ÔÕ x 2, then when n, 1. The stochastic process x n 1 ÔtÕ does not converge in any functional sense; 2. The stochastic process x n 2ÔtÕ converges in law in ÔDÖ, Õ,SÕ towards the deterministic solution of the ordinary differential equation where dx 2 dt γ 2x 2 λ 2 kôx 2 Õ, x 2 ÔÕ x 2, (9.23) kôx 2 Õ bλ 1 k 1 Ôx 2 Õßγ 1. Theorem 55. Consider the eq. (9.17)-(9.18) and assume hypothesis 8. If the scaling (S2) is satisfied, i.e., h n ÔzÕ 1 n hôz n Õ, and if xn 2 ÔÕ x 2, then when n, 1. The stochastic process xn 1 ÔtÕ n converges in law in L p, 1 p and in ÔDÖ,T,JÕ to the (deterministic) fixed value ; 2. The stochastic process x n 2 ÔtÕ converges in law in Lp, 1 p and in ÔDÖ,T,JÕ to the stochastic process defined by the solution of the stochastic differential equation dx 2 dt γ 2x 2 NÔ h,λ1 k 1 Õ, x 2 ÔÕ x 2, (9.24) where hôx 2 Õ Ôλ 2 ßγ 1 Õ 1 hôôλ 2 ßγ 1 Õ 1 x 2 Õ.

134 9 From One Model to Another 133 Moreover, in the constant case k 1 1, the stochastic process x n 1 ÔtÕ converges in law in MÔ, Õ to the compound Poisson white noise NÔh,λ 1 Õ; Theorem 56. Consider the eq. (9.17)-(9.18). and assume hypothesis 8. If the scaling (S3) is satisfied, i.e., λ n 2 nλ 2, and if x n 2 ÔÕ x 2, then when n, 1. The stochastic process x n 1 ÔtÕ converges in law in Lp, 1 p and in ÔDÖ,T,JÕ to the (deterministic) fixed value ; 2. The stochastic process x n 2 ÔtÕ converges in law in Lp, 1 p and in ÔDÖ,T,JÕ to the stochastic process determined by the solution of the stochastic differential equation where hôx 2 Õ Ôλ 2 ßγ 1 Õ 1 hôôλ 2 ßγ 1 Õ 1 x 2 Õ. dx 2 dt γ 2x 2 NÔ h,ϕõ, x2 ÔÕ x 2, Remark 57. Note that scalings (S2) and (S3) give similar results for the equation governing the protein variable x 2 ÔtÕ but very different results for the asymptotic stochastic process related to the mrna. In particular, in theorem 55, very large bursts of mrna are transmitted to the protein, where in theorem 56, very rarely is mrna present but when present it is efficiently synthesized into a burst of protein. In this section, we provide three different proofs of the results mentioned above. In particular, we prove the results using a master equation approach(the Kolmogorov forward equation) as well as starting from the stochastic differential equation. Note that both techniques have been used in the past, in particular within the context of discrete models of stochastic reaction networks. For the master equation approach, see [56, 153, 124] while for the stochastic differential equation approach, we refer to [25, 75]. In paragraph we first show the tightness result for all three theorems. We then identify the limit using martingale approach in paragraph In the others section, we provide alternative proof to identify the limit. In paragraph 9.3.6, we consider the situation withoutauto-regulation so theratek 1 is independentof protein concentration x 2. In this case the two eq. (9.15)-(9.16) form a set of linear stochastic differential equations. We use then the method of characteristic functionals to identify the limit. Finally in paragraph we give a similar result on the evolution equation on densities General properties and moment estimates We first summarize the important background results on the stochastic processes used in the next One dimensional equation For the one-dimensional stochastic differential equation(9.22) perturbedbyacompoundpoissonwhitenoise, of(bounded)intensitykôx 2 Õ and jump size distribution h, the extended generator of the stochastic process Ôx 2 ÔtÕÕ t is, for any f È DÔAÕ, (see [27, Theorem 5.5]) A 1 fôxõ γ 2 x df dx kôxõ x hôz xõfôzõdz fôxõ DÔA 1 Õ Øf È MÔ, Õ : t fôxe γ 2t Õ is absolutely continuous ô for t È R and E fôx 2 ÔT i ÕÕ fôx 2 ÔT i ÕÕ for all t Ù T i t

135 134 Hybrid Models to Explain Gene Expression Variability where MÔ, Õ denotes a Borel-measurable function of Ô, Õ and the times T i are the instants of the jump of x 2. It is an extended domain containing all functions that are sufficiently smooth along the deterministic trajectories between the jumps, and with a bounded total variation induced by the jumps. The operator A 1 is the adjoint of the operator acting on densities vôt,xõ given by [9] vôt,xõ t x Öγ 2xvÔt,xÕ x kôzõvôt,zõhôx zõdz kôxõvôt,xõ. For any f È DÔA 1 Õ, we have d dt EfÔx 2ÔtÕÕ EA 1 ÔfÔx 2 ÔtÕÕÕ Two dimensional equation Consideration of the two-dimensional stochastic differential equation (9.15)-(9.16) perturbed by a compound Poisson white noise, of intensity λ 1 k 1 Ôx 2 Õ and jump size distribution h follows along similar lines. Its infinitesimal generator and extended domain are A 2 gôx 1,x 2 Õ γ 1 x 1 g x 1 λ 1 k 1 Ôx 2 Õ Ôλ 2 x 1 γ 2 x 2 Õ g x 2 x 1 hôz x 1 ÕgÔz,x 2 Õdz gôx 1,x 2 Õ «, (9.25) DÔA 2 Õ Øg È MÔÔ, Õ 2 Õ : t gôφ t Ôx 1,x 2 ÕÕ is absolutely (9.26) continuous ô for t È R and E gôx 1 ÔT i Õ,x 2 ÔT i ÕÕ gôx 1 ÔT i Õ,x 2 ÔT i ÕÕ for all t Ù T i t where φ t is the deterministic flow given by eq. (9.15) and (9.16). The evolution equation for densities uôt,x 1,x 2 Õ is uôt,x 1,x 2 Õ t Öγ 1 x 1 uôt,x 1,x 2 Õ ÖÔλ 2 x 1 γ 2 x 2 ÕuÔt,x 1,x 2 Õ x 1 x 2 x1 λ 1 k 1 Ôx 2 ÕuÔt,z,x 2 ÕhÔx 1 zõdz λ 1 k 1 Ôx 2 ÕuÔt,x 1,x 2 Õ. For any f È DÔA 2 Õ, we have d dt EfÔx 1ÔtÕ,x 2 ÔtÕÕ EA 2 ÔfÔx 1 ÔtÕ,x 2 ÔtÕÕÕ. (9.27) Using stochastic differential equations (9.17) - (9.18), we can deduce moment estimates, needed to be able to use unbounded test function (namely fôx 1,x 2 Õ x 1 and fôx 1,x 2 Õ x 2 ) in the martingale formulation. By taking the mean into eq. (9.17) - (9.18) and neglecting negatives values, E x 1 ÔtÕ E x 2 ÔtÕ t t λ 1 be k 1 Ôx 2 ÔsÕÕ ds λ 2 E x 1 ÔsÕ ds t λ 1 bôc k 1 E x 2 ÔsÕ Õds

136 9 From One Model to Another 135 where we note b E h zhôzõdz. By Grönwall inequalities, there exist a constant C such that E sup x 1 ÔtÕ CÔE x 1 ÔÕ e CT Õ tèö,t E sup x 2 ÔtÕ CÔE x 2 ÔÕ e CT Õ (9.28) tèö,t Then we claim that f Ôx 1,x 2 Õ x 1 is in the domain of the generator A 2. We only have to verify (see eq. (9.26)) ô E x 1 ÔT i Õ x 1 ÔT i Õ for all t. By eq. (9.17) E ô T i t T i t x 1 ÔT i Õ x 1 ÔT i Õ E t bλ 1 E t c k 1 x 2 ÔsÕds. which is finite according to the previous estimates Tightness 1 Ørλ1 k 1 Ôx 2 Ôs ÕÕÙ zn Ôds,dz,drÕ, S1 We first show the tightness property for the scaling (S1) corresponding to theorem 54. In such case x n 1 does no converge in any functional sense because it fluctuates very fast, as more and more jumps appears of size that stay of order 1 (given by h). However, E x n 1 ÔtÕ remains bounded, xn 1 n goes to, and by eq. (9.18), x n 2 ÔtÕ xn 2 ÔÕ t λ 2 x n 1 ÔsÕ ds. For any n, let N n be a compound Poisson process associated to eq. (9.17), with ØT n,i Ùi1 the jump times which occur at a rate nλ 1 k 1 Ôx n 2 ÔsÕÕ, and ØZ n,iù i1 the jump sizes that are iid random variables with density h (with the convention T n, and Z n, X ). Then ô x n 1 ÔtÕ Z n,i e nγ 1Ôt T n,i Õ 1 ØtTn,i Ù. T n,i t By integration, t x n 1 ÔsÕds ô T n,i t Z n,i 1 nγ 1 Ô1 e γ 1Ôt T n,i Õ Õ1 ØtTn,i Ù. Then, x n 2 ÔtÕ xn 2 ÔÕ t λ 2 x n 1 ÔsÕds Y λ 2 ô Z n,i. nγ 1 T n,i t Finally we deduce, by definition of the compound Poisson process, x n 2 ÔtÕ xn 2 ÔÕ λ 2 nγ 1 N n ÔtÕ. Now, by a time change, there exists a process Y such that N n ÔtÕ Y t nλ 1k 1 Ôx n 2 ÔsÕÕds with Y an unit rate compound Poisson process of jump size iid (with density h). As

137 136 Hybrid Models to Explain Gene Expression Variability E h, by the law of large number, 1 n Y ÔntÕ is almost surely convergent (to E h t). Then 1 ny ÔntÕ is almost surely bounded, on a compact time interval Ö,T. We deduce then that there exists a random variable C such that x n 2 ÔtÕ xn 2 ÔÕ λ 2 γ 1 C By Grönwall lemma and Markov inequality Similarly, for any t 1,t 2 È Ö,T, Again, N n Ôt 2 Õ N n Ôt 1 Õ Y t2 so that, for any ε t λ 1 k 1 Ôx n 2 ÔsÕÕds. P sup x N 2 ÔtÕ K. tèö,t x n 2 Ôt 2Õ x n 2 Ôt 1Õ λ 2 nγ 1 N n Ôt 2 Õ N n Ôt 1 Õ. t 1 nλ 1 k 1 Ôx n 2 ÔsÕÕds and, still by the law of large number x n 2 Ôt 2 Õ x n 2 Ôt 1 Õ λ 2 γ 1 C lim limsup θ n sup S 1 S 2 S 1 θ t2 t 1 λ 1 k 1 Ôx n 2 ÔsÕÕds, P x n 2 ÔS 2Õ x n 2 ÔS 1Õ ε, where the supremum is over stopping times bounded by T. Then by Aldous tightness criterion ([7, thm 4.5 p 356]), x n 2 is tight in ÔDÖ, Õ,SÕ. S3 Now we show the tightness property for the scaling (S3) corresponding to theorem 56, withλ n 2 nλ 2. In suchcase x n 1 converges toinl1 t, andweget acontrol overn xn 1 ÔsÕds. Indeed using gôx 1,x 2 Õ x 1 in eq. (9.25), we get t x n 1 ÔtÕ xn 1 ÔÕ Ô nγ 1 x n 1 ÔsÕ λ 1k 1 Ôx n 2 ÔsÕÕbdsÕ, is a martingale so that due to hypothesis 8, there is a constant C such that By eq. (9.18), then γ 1 E n t ÔsÕds ÔÕ t ÔsÕ x n 1 E x n 1 λ 1 Ôct k 1 E x n 2 dsõ x n 2 ÔtÕ E x n 2 ÔÕ λ 2 n Reporting into the estimates for x n 1 yields γ 1 E n t x n 1 ÔsÕds t x n 1 ÔsÕds. T sup x n 2 ÔtÕ E x n 2 ÔÕ λ 2 n x n 1 ÔsÕds. tèö,t E x n 1 ÔÕ λ 1 Ôct k 1 ÔE x n 2 ÔÕ tλ 2 n C 1 T C2 T E n t x n 1 ÔsÕds, t E x n 1 ÔsÕ dsõõ,

138 9 From One Model to Another 137 fortwoconstantsc 1 T,C2 T thatdependssolelyont. ByGrönwallinequality, E n t xn 1 ÔsÕds is bounded uniformly in n so that x n 1 converges to in L1 and P sup x N 2 ÔtÕ K. tèö,t Now for any subdivision of Ö,T, t t 1 t n T, n 1 ô i x n 2 Ôt i 1Õ x n 2 Ôt iõ E x n 2 ÔÕ λ 2 n so that we also get the tightness of the BV norm, P Ðx n 2 Ð Ö,T K, t x n 1 ÔsÕds, as K, independently in n. Then x n 2 is tight in Lp ÔÖ,T Õ, for any 1 p. S2 Now we show the tightness property for the scaling (S2) corresponding to theorem 55, with h n 1 n hô1 n Õ. Remark that on such case, denoting zn xn 1 n, the variables Ôz n,x n 2 Õ satisfies eq. (9.17) - (9.18) with the (S3) scaling, so we already know that x n 2 is tight in L p ÔÖ,T Õ, for any 1 p. For x n 1, note that each jumps gives a contribution for x n 1 of b γ 1 so there s no hope for a convergence to in L 1. However, we still have x n 1 ÔtÕ ô T n,i t Z n,i e nγ 1Ôt T n,i Õ 1 ttn,i. where T n,i appears with rate λ 1 k 1 Ôx n 2 ÔsÕÕ, and ØZ n,iù i1 are iid random variables with density h n. Then But for K x n 1 ÔtÕ ô T n,i t Z n,i 1 1 ØÖTn,i,T n,i n ÖÙ e nγ 1 1 n 1 ttn,i. P Z n,i e nγ 1 K nb Ke nγ 1 ε, for any ε and n sufficiently large. Then, conditioning by the jump times, t P x n 1 ÔsÕ K T n,i ô T n,i t 1 n 1 ØtTn,i Ù ô T n,i t εôt T n,i Õ1 ØtTn,i Ù ε. for n large. Because t xn 2 ÔsÕds has been shown to be bounded independently of n, we can drop the conditioning, and t P x n 1 ÔsÕ K is arbitrary small. We show also similarly that T lim sup maxô1, x n 1 Ôt hõ x n 1 ÔtÕ Õdt, h n so that x n 1 is tight in MÔ, Õ ([82, thm 4.1]).

139 138 Hybrid Models to Explain Gene Expression Variability Identification with the martingale problem The three theorems below can be proved using martingale techniques, with similar spirit. For each scaling, the generator A n 2 can be decomposed into a fast component, or order n, and a slow component, of order 1. In each case, one need to find particular condition to ensure that the fast component vanishes. For the scaling ÔS1Õ, the fast component acts only in the first variable, so ergodicity of this component will ensure that it vanishes. For the other two, the fast component acts on both variables, and we will have to find the particular relation between both variable that ensures this component vanishes Proof of theorem 54 For any B È BÔR Õ, t, we define the occupation measure t V1 n ÔB Ö,t Õ 1 ØBÙ Ôx n 1 ÔsÕÕds, and we identify V1 n as a stochastic process with value in the space of finite measure on R. Because E x n 1 ÔtÕ remains bounded uniformly in n on any Ö,T, it is stochastically bounded and V 1 then satisfies Aldous criterion of tightness. Now take a test function f that depends only on x 1, so that with Then C x2 fôx 1 Õ γ 1 x 1 f ½ Ôx 1 Õ A n 2 fôx 1Õ nc x2 fôx 1 Õ, λ 1 k 1 Ôx 2 Õ M n t fôx n 1 ÔtÕÕ fôxn 1 ÔÕÕ n R t x 1 hôz x 1 ÕfÔzÕdz fôx 1 Õ C x n 2 ÔsÕfÔx 1 ÕV n 1 Ôdx 1 dsõ is a martingale. Dividing by n, for any limiting point ÔV 1,x 2 Õ, we must have, for any f È C b ÔR Õ, E R t C x2 ÔsÕfÔx 1 ÕV 1 Ôdx 1 dsõ. Because for any x 2, the generator C x2 is (exponentially) ergodic (see paragraph 8.5) V 1 is uniquely determined by the invariant measure associated to C x2. In particular, for any t t t x 1 V1 n Ôdx bλ 1 1 dsõ k 1 Ôx 2 ÔsÕÕds. R γ 1 Then for f that depends only on x 2, converges to fôx n 2 ÔtÕÕ fôxn 2 ÔÕÕ R t Ôλ 2 x 1 γ 2 x n 2 ÔsÕÕf ½ Ôx n 2 ÔsÕÕV 1 n Ôdx 1 dsõ t fôx 2 ÔtÕÕ fôx 2 ÔÕÕ Ô bλ 1λ 2 k 1 Ôx 2 ÔsÕÕ γ 2 x 2 ÔsÕÕf ½ Ôx 2 ÔsÕÕds γ 1 Due to the assumption on k 1, there exists a unique solution associated to the (deterministic) eq. (9.21) so x 2 is uniquely determined. «.

140 9 From One Model to Another Proof of theorem 56 We already seen that x n 1 converge to in L1 ÔÖ,T Õ and x n 2 is tight in Lp ÔÖ,T Õ. Doing similarly as in subsection 9.2, we take a subsequence Ôx n 1 ÔtÕ,xn 2 ÔtÕÕ that converge to Ô,x 2ÔtÕÕ, almost surely and for almost t È Ö,T. Then we consider the fast component of the generator A n 2, given in this case by γ 1 x 1 f x 1 λ 2 x 1 f x 2. This defines a transport equation. Starting at Ôx 1,x 2 Õ at time, the asymptotic value of the flow associated to the transport equation is Ô,yÕ where We then consider y x 2 that satisfies, for any x 1,x 2, Now taking the limit n into λ 2 x 1 ÔsÕds x 2 fôx 1,x 2 Õ g x 2 γ 1 x 1 f x 1 x1 λ 2 z γ 1 z dz x 2 λ 2 x 1, γ 1 λ 2 x 1 f x 2. λ 2 γ 1 x 1 t fôx n 1ÔtÕ,x n 2 ÔtÕÕ fôx n 1ÔÕ,x n 2 ÔÕÕ A n 2fÔx n 1 ÔsÕ,x n 2 ÔsÕÕds, yields t «gôx 2 ÔtÕÕ gôx 2 ÔÕÕ γ 2 x 2 g ½ Ôx 2 ÔsÕÕ λ 1 k 1 Ôx 2 ÔsÕÕ hôzõgôx 2 ÔsÕ zõdz gôx 2 ÔsÕÕ ds, where hôx 2 Õ Ôλ 2 ßγ 1 Õ 1 hôôλ 2 ßγ 1 Õ 1 x 2 Õ. Hence the limiting process x 2 must satisfy the martingale problem associated with the generator A gôxõ γ 2 x dg dx λ 1 k 1 ÔxÕ x hôz xõfôzõdz fôxõ, for which uniqueness holds for bounded k 1 (see [25, thm 2.5] or theorem 9 in Chapter ). A truncation argument allows then to conclude Proof of theorem 55 As noticed before, Ôz n,x n 2 Õ with zn ÔtÕ xn 1 ÔtÕ n satisfies the scaling (S3) so similar conclusion holds for x n 2. The last conclusion on xn 1 is differed to the next subsection The case without auto-regulation In this subsection, we give an alternative proof of the identification of the limit, using the characteristic functional of the stochastic process. This can works when there s no nonlinearity, and eq. (9.15) - (9.16) can actually be seen as generalized Langevin equation. We consider the equations dx 1 dt dx 2 dt γ 1 x 1 NÔh,λ1 Õ, x 1 ÔÕ x 1, (9.29) γ 2 x 2 λ 2 x 1, x 2 ÔÕ x 2, (9.3)

141 14 Hybrid Models to Explain Gene Expression Variability where NÔh,λ 1 Õ is a compound Poisson white noise. The solutions x 1 ÔtÕ and x 2 ÔtÕ of eq. (9.29) - (9.3) are stochastic processes uniquely determined by the equation parameters and the stochastic process N. For a stochastic process ξ t (t ), the characteristic functional C ξ : Σ R is defined as C ξ Öf E e ifôtõξtdt, for any function f in a suitable function space Σ so that the integral ifôtõξ tdt is well defined. Before continuing, we need to introduce some topological background as well as properties of the Fourier transform in nuclear spaces (see [44]) Stochastic process as a distribution We are going to recall here the continuous correspondence between a stochastic process and a distribution. We define DÔR Õ, the space of smooth functions with compact support, with the inductive limit topology given by the family of semi-norms (k,1,2...) p k ÔfÕ sup f ÔkÕ on every DÔÖ,n Õ, n È N (c.f. [125, Example 2, page 57]). Let f È DÔR Õ, and define x in the dual space D ½ ÔR Õ such that xôfõ xôtõf ÔtÕdt (9.31) for any x in DÖ, Õ, and analogous definition for x È L p Ö,T or MÔ, Õ. Lemma 58. The map ÔDÖ, Õ,SÕ D ½ ÔR Õ Ôx t Õ t x, where x is defined by eq. (9.31), is continuous. Proof. It is a classical result that x È D has at most a countable number of discontinuity points so that x is locally integrable, the integral in eq. (9.31) is well defined for all f È DÔR Õ, x È D ½ ÔR Õ and T xôfõ xôsõ ds f, for any f with support in Ö,T [121, Section 6.11, page 142]. We conclude by noticing that xôfõ sup xôsõ f T, st and x sup st xôsõ is continuous for the Skorohod topology [7, Proposition 2.4, page 339] for all T such that T is not a discontinuity point. Similar continuity property holds respectively in D Ô, Õ,J, L p Ö,T, MÖ, Bochner-Minlos theorem for a nuclear space Let E be a nuclear space. We state a key result that will allow us to uniquely identify a measure on the dual E ½ of E. Bochner-Minlos Theorem. [44, Theorem 2, page 146] For a continuous functional C on a nuclear space E that satisfies CÔÕ 1, and for any complex z j and elements x j È A, j,k 1,...,n, nô nô z j z k CÔx j x k Õ, j1k1

142 9 From One Model to Another 141 there is a unique probability measure µ on the dual space E ½, given by CÔyÕ E ½ e iüx,yý dµôxõ. Note that the space DÔR Õ is a nuclear space [125, Example 2, page 17] The characteristic functional of a Poisson white noise The use of the characteristic functional allows us to define a generalized stochastic process that does not necessarily have a trajectory in the usual sense (like in D for instance). Indeed a (compound) Poisson white noise is seen as a random measure on the distribution space D ½, associated with the characteristic functional (given in [61], here f È DÔR Õ) C N Öf exp ϕ Ôe izfôtõ 1ÕhÔzÕdzdt, (9.32) where ϕ is the Poisson intensity and h the jump size distribution. It is not hard to see that C N Öf g and C N Ög f are conjugate to each other, C N Ö 1 and C N Ö. is continuous for h È L 1 ÔR Õ, so the conditions in the Bochner-Minlos theorem are satisfied and therefore C N uniquely defines a measure on D ½ ÔR Õ. Remark 59. To see that this measure indeed corresponds to the time derivative of the compound Poisson process, consider the following E e i N,f lim E e i j fôt jõ j N, j where j N N Ôt j 1 Õ N Ôt j Õ denotes the increment of a compound Poisson process, and Ôt j Õ is some subdivision of R of maximal step size j. Due to the independence of the increments of the Poisson process, this limit can be re-written as E e i N,f lim j õ j E e ifôt jõ j N. Now, because of the independence of the jump size and the number of jumps, and the fact that all jumps are independent and identically distributed (with distribution given by h), ô E e ifôt jõ j N E e ifôt jõ j N j N n n ô E e ifôt jõôz 1 Z nõ j N n n ô n ô n E e ifôt jõôz 1 Z nõ PÔ j N nõ e ifôt jõz npô j N nõ E e ϕ j ô n exp ϕ j e ifôt jõz hôzõdz n Ôϕ j Õ n n! e ifôt jõz hôzõdz 1, so E e i N,f lim ϕ ô exp j e ifôtjõz hôzõdz 1 j j exp ϕ Ôe izfôtõ 1ÕhÔzÕdzdt.

143 142 Hybrid Models to Explain Gene Expression Variability We refer to [18, 61, 62] for further material on characteristic functionals and generalized stochastic processes Identification of the limit using characteristic functional The proofs of theorems 54 to 56 are based on the idea of Levy s continuity theorem. However in the infinite-dimensional case, the convergence of the Fourier transform does not imply convergence in law of the random variable, and one needs to impose more restrictions, namely a compactness condition. We will use the following lemma Lemma 6. Let X n be a sequence of stochastic processes in ÔDÖ, Õ,SÕ. Suppose X n is tight in ÔDÖ, Õ,SÕ (respectively in D Ô, Õ,J, L p Ö,T, MÖ, ) and that there exists a random variable X such that, for all f in DÔR Õ, as n, Then X n converges in law to X in D MÖ, ). C X nöf C X Öf. Ô, Õ,S (respectively in D Ô, Õ,J, L p Ö,T, Proof. The convergence of the characteristic functional, the Bochner-Minlos theorem and the continuity lemma 58 ensure that the sequence X n has at most one limiting law, which has to be the law of X. The classical Prokhorov Theorem [7, Corollary 3.9, page 348] states that tightness of X n in ÔDÖ, Õ,SÕ is equivalent to relative compactness of the law of X n in PÔDÕ, the space of probability measures on D (with the topology of the weak convergence). Then X n converges in law to X in ÔDÖ, Õ,SÕ. The continuity lemma and Prokhorov theorem are also valid in D Ô, Õ,J, L p Ö,T, MÖ, (see part section 7). Note that the tightness property has already been done in paragraph Now, we give the identification property of the limit for theorems 54 through 56. The strategy is similar for each, and we only present a detailed proof for theorem 54 and sketch the main differences in the proofs for theorems 55 and 56. For any f È DÔR Õ, from eq. (9.29) - (9.3) and noting that the initial conditions x 1 and x 2 are deterministic, it is not difficult to verify that (see also [19]) where g i C x1 Öf e ig 1x 1 C NÖ f 1 ÔtÕ, C x2 Öf e ig 2x 2 Cx1 Öλ 2 f2 ÔtÕ, (9.33) e γ is fôsõds, fi ÔtÕ t e γ iôs tõ fôsõds, Ôi 1,2Õ. Note that for any function f È DÔR Õ the functions f i ÔtÕ also belong to DÔR Õ and therefore the characteristic functionals in eq. (9.33) are well-defined. Furthermore, the characteristic functional of the compound Poisson white noise has been derived in eq. (9.32). Proof of theorem 54. Recall that λ n 1 nλ 1. We omit the dependence in n of function g i and f i for simplicity. Now, we are ready to complete the proof by calculating the characteristic functionals C x n 1 and C x n 2 when n from eq. (9.33) and (9.32). Firstly, we note that g i f i ÔÕ, and when f È DÔR Õ and n, f 1 ÔtÕ 1 fôtõ OÔ 1 Õ. (9.34) nγ 1 n2

144 9 From One Model to Another 143 Furthermore, from eq. (9.33) - (9.34), we have C x n 1 Öf e ix 1 Ô 1 fôõ OÔ 1 nγ 1 n 2 ÕÕ 1 C N f 1 ÔtÕ OÔ nγ 1 n 2 Õ e i 1 x nγ 1 1 fôõ λ 1 exp i f ÔtÕxhÔxÕdxdtÕ γ 1 Thus, from eq. (9.2), we have Therefore, eq. (9.33) yields lim n C x n 1 Öf exp i lim C x n nöf eig 2x 2 2 exp iλ 2 e ig 2x 2 exp i 1 OÔ Õ. (9.35) n f ÔtÕx eq 1 dt, f È D. (9.36) f 2 ÔtÕx eq 1 dt f ÔsÕÔ1 e γ 2s Õx eq 2 ds. (9.37) Now, it is easy to verify that the right hand sides of eq. (9.36) and (9.37) give, respectively, the characteristic functional of x 1 ÔtÕ x eq 1 and x 2ÔtÕ of the solution of eq. (9.23). Hence we are done. Proof of theorem 55. Recall that h n ÔxÕ 1 n hôx n Õ. The proof is similar to the proof of theorem 55. Note simply from the scaling (S2) that eq. (9.35) becomes, still from eq. (9.33) - (9.34) C x n 1 Öf e i 1 x 1 nγ 1 1fÔÕ OÔ n 2 exp λ 1 Ôe i z fôtõ OÔ 1 nγ 1 n 2 1Õh n ÔzÕdzdt. Thus, by a change of variable x zßôγ 1 nõ, we have C x n 1 Öf e i 1 x 1 nγ 1 1fÔÕ OÔ n 2 exp λ 1 Ôe ixfôtõ OÔ 1 n 1Õ hôxõdxdt where hôxõ γ 1 hôγ 1 xõ. Then lim C x n nöf exp iλ 1 1 C N Öf. Ôe ifôtõx 1Õ hôxõdxdt where N is a compound Poisson white noise of intensity λ 1 and jump size distributed according to h. Furthermore, from eq. (9.33) lim C x n nöf eig 2x 2 2 G N Öλ 2 f2 ÔtÕ e ig 2x 2 exp iλ 1 e ig 2x 2 exp iλ 1 Ôe iλ 2x f 2 ÔtÕ 1Õ hôxõdxdt Ôe ix f 2 ÔtÕ 1Õ hôxõdxdt, (9.38) where hôx 2 Õ Ôλ 2 ßγ 1 Õ 1 hôôλ 2 ßγ 1 Õ 1 x 2 Õ It is easy to verify that eq. (9.38) is just the characteristic functional of the stochastic processes given by solutions of eq. (9.24). Proof of theorem 56 Here, λ n 2 nλ 2. we have lim C x n n Öf lim exp λ 1 1 Ôe ifôtõxßônγ 1Õ 1ÕhÔxÕdxdt 1, n,

145 144 Hybrid Models to Explain Gene Expression Variability and lim C x n nöf eig 2x 2 2 exp λ 1 e ig 2x 2 exp λ 1 Ôe iôλ 2ßγ 1 Õx f 2 ÔtÕ 1ÕhÔxÕdxdt Ôe ix f 2 ÔtÕ 1Õ hôxõdxdt, where hôxõ γ 1 λ 2 hô γ 1 λ 2 xõ Reduction on the evolution equation We conclude by a third proof for the reduction, working on the partial differential equation for the evolution equations on densities. Because we work directly on the strong form, results are weaker. In particular, Hypothesis 9. In addition of hypothesis 8, we assume that (H1) The density function h È C, and for all k 1, zk hôzõdz. (H2) The rate function k 1 È C, and k 1 is bounded above and under, k 1 k 1 ÔxÕ k 1, Theses assumption are needed to ensure that evolution equation on densities is well defined(see section 8.2), and allow us to derive scaling laws for arbitrary moments, that are needed for calculus. Regularity will allow us to derive at any order the density functions, which is also needed for the calculus. We start by a scaling property of the moments, which is crucial for the convergence results Scaling of the marginal moment Using the generator A 2 for the twodimensional stochastic process defined by eq. (9.25), we can deduce the scaling laws of the marginal moment of x 1 ÔtÕ n as γ n 1. Proposition 61. Let Ôx n 1 ÔtÕ,xn 2 ÔtÕÕ be the solutions of eq. (9.15) - (9.16), and µn k ÔtÕ E x n 1 ÔtÕk and ν n k ÔtÕ E x n 2 ÔtÕxn 1 ÔtÕk. Suppose µ n k ÔÕ and νk n ÔÕ for all n 1, then µ n k ÔtÕ and νn k ÔtÕ for all t, n 1. Moreover, for fixed t, 1. If the scaling (S1) holds, then both µ n k ÔtÕ and νn k ÔtÕ stay uniformly bounded above and below as n. 2. For the scaling (S2), then, for k 1, µ n k ÔtÕ nk 1, ν n k ÔtÕ nk 1, Ôn Õ and ν n ÔtÕ is uniformly bounded above and below as n. 3. If (S3) holds then, for k 1, µ n k ÔtÕ n 1, ν n k ÔtÕ n 1, Ôn Õ and ν n ÔtÕ is uniformly bounded above and below as n.

146 9 From One Model to Another 145 Proof. The proposition is proved using the evolution equation for the marginal moment obtained from the generator A n 2. Firstly, we claim that functions x k 1 and xk 1 x 2 (k È N ) are contained in DÔA n 2 Õ, for all n 1. To show this, we only need to verify that ô E x n 1 ÔT iõ k x n 2 ÔT i Õ l x n 1 ÔT i Õ k x n 2 ÔT i Õ l, t,k È N,l,1, T i t wherethet i are jumptimes (that also dependson the scaling n). Sincex n 2 ÔtÕ is continuous and from estimates eq. (9.28), E sup Ö,T x n 2 ÔtÕ is bounded. Then we only need to verify the case with l. Now by eq. (9.17), E ô T i t x n 1 ÔT iõ k x n 1 ÔT i Õ k E t b n k λn 1 E t k 1 Ôx n 2 ÔsÕÕds, 1 Ørλ n 1 k 1 Ôx n 2 Ôs ÕÕÙ z k N n Ôds,dz,drÕ, where we note b n k zk h n ÔzÕdz (so b n 1 and bn 1 bn ). As k 1 is assumed to be linearly bounded, still by estimates eq. (9.28) we conclude that ô E x n 1 ÔT iõ k x n 1 ÔT i Õ k, t, n 1 T i t Now, A n 2 xk 1 and An 2 xk 1 x 2 are well defined, for all k and n 1. A straightforward calculation yields «A n 2x k 1 γ1kx n k 1 λn 1k 1 Ôx 2 Õ h n Ôz x 1 ÕÔz x 1 x 1 Õ k dz x k 1 x 1 k 1 ô Å k γ1kx n k 1 λn 1k 1 Ôx 2 Õ x i 1 h n Ôz x 1 ÕÔz x 1 Õ k i dz i i x 1 k 1 ô Å k γ1kx n k 1 λn 1k 1 Ôx 2 Õ x i i 1b n k i. i Then the k th -marginal moment µ n k ÔtÕ of the first variable xn 1 depends only on the lower moment µ n i ÔtÕ, i k. We then obtain, with hypothesis eq. (9) and eq. (9.27) k 1 ô Å k γ1kµ n n k ÔtÕ λn 1k 1 µ n i i ÔtÕbn k i µ n k ÔtÕ, i k 1 ô Å µ k n k ÔtÕ γn 1kµ n k ÔtÕ λn 1k 1 µ i ÔtÕb n k i i. (9.39) Recall that in all scalings γ1 n nγ 1. Assume scaling (S1), λ n 1 nλ 1, and h n,λ n 2 are independent of n. Inequalities eq. (9.39) for k 1 yields, for all t, nλ 1 k 1 b Multiplying by e nγ 1t, a direct integration yields i µ n 1 ÔtÕ nγ 1µ n 1 ÔtÕ nλ 1k 1 b. λ 1 k 1 b γ 1 Ôe nγ 1t 1Õ e nγ 1 t µ n 1 ÔtÕ µn 1 ÔÕ λ 1k 1 b γ 1 Ôe nγ 1t 1Õ,

147 146 Hybrid Models to Explain Gene Expression Variability so finally λ 1 k 1 b γ 1 OÔ 1 n Õ µn 1 ÔtÕ λ 1k 1 b γ 1 OÔ 1 n Õ. Iteratively, for all t and k 1, there is a constant c k ÔtÕ independent of γ 1 (where c k ÔtÕ depends only on the moment of h and lower moments µ n j ÔtÕ, j k ) such that λ 1 k 1 c k ÔtÕ γ 1 OÔ 1 n Õ µn k ÔtÕ λ 1k 1 c k ÔtÕ γ 1 OÔ 1 n Õ. Assume (S2) i.e. b n k nk b k. The case k 1 follows directly from the above calculations, and for all k 1 and t, λ 1 k 1 n k b k knγ 1 OÔn k 2 Õ µ n k ÔtÕ λ 1k 1 n k b k nγ 1 OÔn k 2 Õ. Finally, assume (S3). The same method shows that for all t and k 1, there is a constant c k independent of γ 1 (c k depends of the moment of h and of λ 1 ) such that c k nγ 1 OÔ 1 n 2 Õ µn k ÔtÕ c k nγ 1 OÔ 1 n 2 Õ. A similar calculation with gôx 1,x 2 Õ x 2 x k 1 so that, for k 1, A n 2x k 1x 2 Ô γ n 1k γ 2 Õx k 1x 2 λ n 2x k 1 1 λ n 1k 1 Ôx 2 Õ while for k, we obtain k 1 ô Ô γ1k n γ 2 Õνk n ÔtÕ λn 2µ n k 1 λn 1k 1 gives analogous scaling. Namely, we have i ν k nôtõ Ô γn 1 k γ 2Õνk n ÔtÕ λn 2 µn k 1 λn 1 k 1 ν n γ 2ν λ n 2µ n 1. k 1 ô i Å k µ n i ÔtÕb n k i i k 1 ô i Å k x i i 1b n k i, Å k µ n i i ÔtÕbn k i, Then ν n is uniformly bounded for each scaling (S1),(S2), and (S3). Then, using iteratively the inequalities for ν k n, the scaling of µn k 1 and direct integration yields the desired result for each scaling Density evolution equations Let u n Ôt,x 1,x 2 Õ be the density function of Ôx n 1 ÔtÕ,xn 2 ÔtÕÕ at time t obtained from the solutions of eq. (9.15) - (9.16). The evolution of the density u n Ôt,x 1,x 2 Õ is governed by u n Ôt,x 1,x 2 Õ t Öγ1 n x x 1u n Ôt,x 1,x 2 Õ ÖÔλ n 2 1 x x γ 2x 2 Õu n Ôt,x 1,x 2 Õ 2 x1 λ n 1 k 1Ôx 2 Õu n Ôt,z,x 2 Õh n Ôx 1 zõdz λ n 1 k 1Ôx 2 Õu n Ôt,x 1,x 2 Õ (9.4) when Ôt,x 1,x 2 Õ È Ô, Õ 3. In this subsection, we prove that when n the density functionu n Ôt,x 1,x 2 ÕapproachesthedensityvÔt,x 2 Õforsolutionsofeitherthedeterministic

148 9 From One Model to Another 147 eq. (9.21) or the stochastic differential eq. (9.22) depending on the scaling. Evolution of the density function for eq. (9.21) is given by [83] vôt,x 2 Õ t x 2 Ö γ 2 x 2 u λ 2 kôx 2 Õu. (9.41) Here we note that kôx 2 Õ bλ 1 k 1 Ôx 2 Õßγ 1. x2 Evolution of the density for eq. (9.22) is given by vôt,x 2 Õ t x 2 Öγ 2 x 2 vôt,x 2 Õ λ 1 k 1 ÔzÕvÔt,zÕ hôx 2 zõdz λ 1 k 1 Ôx 2 ÕvÔt,x 2 Õ (9.42) when Ôt,x 2 Õ È Ô, Õ 2. Here h is given by hôx 2 Õ γ 1 λ 2 hô γ 1 λ 2 x 2 Õ. (9.43) When hypothesis 9 is satisfied, existence of the above densities have been rigorously proved in [9, 145]. In particular, for a given initial density that satisfies uô,x 1,x 2 Õ pôx 1,x 2 Õ, x,y (9.44) pôx 1,x 2 Õ, pôx 1,x 2 Õdx 1 dx 2 1, there is a unique solution uôt,x 1,x 2 Õ (we drop the indices n for now, the following is valid for any n 1) of eq. (9.4) that satisfies the initial condition eq. (9.44) and uôt,x 1,x 2 Õ, Moreover, if the moments of the initial density satisfy u k Ôx 2 Õ then the marginal moments uôt,x 1,x 2 Õdx 1 dx 2 1 x k 1pÔx 1,x 2 Õdx 1, x 2,k,1,, (9.45) u k Ôt,x 2 Õ x k 1uÔt,x 1,x 2 Õdx 1, are well defined for t and a.e. x 2, since moments stay finite from the discussion in paragraph Therefore lim x 1 xk 1uÔt,x 1,x 2 Õ, t,a.e x 2. (9.46) Here, we will show, using semigroup techniques as in [9, 145], that under the hypothesis 9, the densities are smooth. We will use the following result Proposition 62. [13, Corollary 5.6, page 124] Let Y be a subspace of a Banach space X, with ÔY,. Y Õ a Banach space as well. Let T ÔtÕ be a strongly continuous semigroup on X, with infinitesimal generator C. Then Y is an invariant subspace of T ÔtÕ if For sufficiently large λ, Y is an invariant subspace of RÔλ,CÕ

149 148 Hybrid Models to Explain Gene Expression Variability There exist constants c 1 and c 2 such that, for λ c 2, For λ c 2, RÔλ,CÕY is dense in Y. Then, we have RÔλ,CÕ j Y c 1 Ôλ c 2 Õ j, j 1,2... Lemma 63. Assume hypothesis 9. If the initial condition vô,x 2 Õ È C ã L 1 then the uniquesolution of eq.(9.42) (respectively eq. (9.41)) vôt,x 2 Õ È C ã L 1. Similarly if the initial condition uô,x 1,x 2 Õ È ÔC Õ 2 ã L 1 then the unique solution of eq. (9.4) uôt,x 1,x 2 Õ È ÔC Õ 2 ã L 1. Proof. Because the dynamical system given by eq. (9.21) is smooth and invertible, the result for eq. (9.41) is standard [83, Remark page 187]. We will show that the result for eq. (9.42), and the result for eq. (9.4) will follow in a similar fashion. We need to show that the subspace C L1 is invariant under the action of the semigroup defined by eq. (9.42). According to [9] (and references therein), we know that the semigroup defined by eq. (9.42) is a strongly continuous semigroup whose infinitesimal generator C is characterized by the resolvent RÔλ,CÕv lim N RÔλ,AÕ Nô ÔP Ôλ 1 k 1 RÔλ,AÕÕÕ j v, (9.47) for all v È L 1, λ, where the limit holds in L 1 and A and P are the operators given by j AvÔx dôγ 2x 2 võ 2 Õ λ 1 k 1 Ôx 2 ÕvÔx 2 Õ, dx 2 PvÔx 2 Õ x2 vôzõhôx 2 zõdz, and the resolvent RÔλ,AÕ is given by, for all v È L 1, RÔλ,AÕvÔx 2 Õ with Q λ Ôx λlnôx 2Õ x2 2 Õ γ 1 2 we have x 2 1 γ 2 x 2 e Q λôzõ Q λ Ôx 2 Õ vôzõdz, λ 1 k 1 ÔzÕ dz. We also know that for γ 2 z v È DÔAÕ Øv È L 1 : Ôx 2 võ is absolutely continuous and Å dôx2 võ È L 1 Ù, dx 2 Cv Av P Ôλ 1 k 1 võ. (9.48) We will now use the result from by proposition 62 above to complete the proof. Note that according to hypothesis 9, Q λ is a C decreasing function, so that for v È C, RÔλ,AÕv È C. Moreover, a simple computation yields, for all λ γ, RÔλ,AÕvÔx 2 Õ sup vôzõ Ôx 2,Õ 1 λ γ v 1 λ γ. Then ÔP Ôλ 1 k 1 RÔλ,AÕÕÕv v λ 1 k 1 λ γ,

150 9 From One Model to Another 149 and ÔP Ôλ 1 k 1 RÔλ,AÕÕÕ j v v λ1 k 1 j, λ γ so that convergence in eq. (9.47) holds in C and C is invariant for RÔλ,CÕ. The second condition in proposition 62 follows then by the previous calculations. Finally, because RÔλ,CÕ Ôλ CÕ 1, to show that RÔλ,CÕC is dense in C, it is enough to show that Ôλ CÕC C. According to eq. (9.48) and hypothesis 9, this is true. The main result given below shows that when n is large enough, u n Ôt,x 2 Õ u n Ôt,x 1,x 2 Õdx 1 gives an approximate solution of eq. (9.41) or eq. (9.42). O Theorem 64. Assume hypothesis 9. Let u n Ô,x 1,x 2 Õ È ÔC Õ ã 2 L 1, for all n 1. For any n 1, let u n Ôt,x 1,x 2 Õ be the associated solution of eq. (9.4), and define u n Ôt,x 2Õ u n Ôt,x 1,x 2 Õdx 1. (1) Under the scaling (S1), when n, u n Ôt,x 2Õ approaches the solution of eq. (9.41). (2) Under the scaling (S2) or (S3), when n, u n Ôt,x 2Õ approaches the solution of eq. (9.42) with h defined by eq. (9.43). In all cases, convergence holds in C, uniformly in time on any bounded time interval. Proof. Throughouttheproof, weomitindicesnonu n Ôt,x 1,x 2 Õandinthemarginaldensity u n Ôt,x 2Õ, and keep in mind that they depend on the parameter n through eq. (9.4) and the particular scaling considered. The first calculus is independent of the particular scaling chosen. Let u k Ôt,x 2 Õ x k 1uÔt,x 1,x 2 Õdx 1, k,1, which are well defined from the previous discussion. From eq. (9.4) and (9.46), we have u k t kγ1 n u k λ n u k 1 Ôx 2 u k Õ 2 γ 2 x 2 x x1 2 λ n 1 k 1Ôx 2 Õx k 1 uôt,z,x 2Õh n Ôx 1 zõdzdx 1 λ n 1 k 1Ôx 2 Õu k. Since x1 λ n 1k 1 Ôx 2 Õx k 1uÔt,z,x 2 Õh n Ôx 1 zõdzdx 1 where b n k zk h n ÔzÕdz. We have u k t kγ n 1 u k λ n 2 u k 1 x 2 γ 2 Ôx 2 u k Õ x 2 kô j λ n 1 k 1Ôx 2 Õ Å k λ n j 1k 1 Ôx 2 Õu k jb n j, kô j1 Å k u k jb n j j.

151 15 Hybrid Models to Explain Gene Expression Variability In particular, when k, When k 1, we have 1 γ n 1 u k t ku k λn 2 γ n 1 u t u 1 λn 2 x 2 u k 1 γ 2 Ôx 2 u k Õ 1 x 2 γ1 n x 2 γ1 n γ 2 Ôx 2 u Õ x 2. (9.49) λ n 1 k 1Ôx 2 Õ kô j1 Å k u k jb n j j. (9.5) Proposition 61 allows us to identify the leading terms of eq. (9.5) as n as given below. (1) When k 1 and n, note that all the right hand-side terms are bounded, and we apply the quasi-equilibrium assumption to eq. (9.5) by assuming when t t, and hence u k λn 2 u k 1 γ 2 Ôx 2 u k Õ 1 kγ1 n x 2 kγ1 n x 2 kγ1 n 1 u k n t λ n 1 k 1Ôx 2 Õ kô j1 Å k u k jb n j j OÔ1 Õ, Ôk 1Õ. n Now, we are ready to prove the results for the three different scalings. (S1). For the scaling (S1), λ n 1 nλ 1 and we have so u k 1 k λ 1 k 1 Ôx 2 Õ γ 1 kô j1 Substituting eq. (9.53) into eq. (9.49), we obtain Å k u k jb j OÔ 1 j n Õ, (9.52) u 1 bλ 1k 1 Ôx 2 Õ u OÔ 1 Õ, (9.53) γ 1 n u t x 2 Öγ 2 x 2 u λ 2 kôx 2 Õu OÔ 1 n Õ with kôx 2 Õ bλ 1 k 1 Ôx 2 Õßγ 1. Finally, note that u ÔT,x 2 Õ T u Ôt,x 2 Õ dt, t so point (1) in theorem 64 follows and convergence holds in C, uniformly in time on any bounded time interval. 1. However, to be more exact, one needs to consider the weak form associated with eq. (9.5) to have integrals of u n, as in proposition 61. The weak form reads, for any smooth function f È C 1 γ n 1 d dt u k Ôx 2Õf Ôx 2Õdx 2 k γ 2 γ1 n yu k Ôx 2Õf ½ Ôx 2Õdx 2 u k Ôx 2Õf Ôx 2Õdx 2 λn 2 γ1 «n 1 kô k k 1Ôx 2Õ b n j j γ n 1 j1 u k 1 Ôx 2Õf ½ Ôx 2Õdx 2 u k j Ôx 2Õf Ôx 2Õdx 2. (9.51) Since u k is a smooth function, there is an equivalence between the strong form (9.5) and its weak form (9.51). Here, as f (and all its derivatives) is bounded, similar estimates as in Proposition 61 can be performed, which justifies the identification of leading order terms. To keep the equations simple, we then perform our calculations on the strong form, while keeping in mind that the identification of leading terms is justified by the weak form and Proposition 61.

152 9 From One Model to Another 151 (S2). We assume the scaling (S2) so h n ÔzÕ 1 n hôz n Õ and bn k nk b k and the re-scaled k th moment b k n k b n k is independent of n. Hence, from eq. (9.52) and proposition 61, we have Therefore, n Ôk 1Õ u k λ 2 kγ 1 Ôn k u k 1 Õ x 2 γ 2 knγ 1 Ôx 2 n Ôk 1Õ u k Õ x 2 λ 1 1 knγ 1 λ 1 k 1 Ôx 2 Õ k 1 ô j1 k j Å n Ôk j 1Õ u k j b j λ 1b k kγ 1 k 1 Ôx 2 Õu λ 2 kγ 1 Ôn k u k 1 Õ x 2 OÔ 1 n Õ. u 1 b 1λ 1 γ 1 k 1 Ôx 2 Õu λ 2 γ 1 x 2 Ön 1 u 2 OÔ 1 n Õ kγ 1 k 1 Ôx 2 Õu b k b 1λ 1 γ 1 k 1 Ôx 2 Õu λ 2 γ 1 x 2 Ö λ 1b 2 2γ 1 k 1 Ôx 2 Õu λ 2 2γ 1 Ôn 2 u 3 Õ x 2 OÔ 1 n Õ b 1λ 1 γ 1 k 1 Ôx 2 Õu b 2 λ 2 2!γ 2 1 k λ !γ1 2 x 2 2 ô Ô λ 2 Õ k Ôk 1Õ!γ1 k 1 x 2 Ôλ 1 k 1 Ôx 2 Õu Õ Ö λ 1b 3 3γ 1 k 1 Ôx 2 Õu λ 2 3γ 1 Ôn 3 u 4 Õ x 2 OÔ 1 n Õ k b k 1 x k Ôλ 1 k 1 Ôx 2 Õu Õ OÔ 1 2 n Õ. Thus, when n, we have, using Taylor development series of u, λ 2 u 1 x 2 ô k1 ô k1 Ô λ 2 Õ k k! 1 k! Ô λ 2 hôx 1 Õ Ôγ k 1 b kõ k Õ k Ô γ 1 ô k1 x k 2 Ôλ 1 k 1 Ôx 2 Õu Õ x k 1hÔx 1 Õdx 1 Õ k x k Ôλ 1 k 1 Ôx 2 Õu Õ 2 1 k! Ô x 1Õ k k x k Ôλ 1 k 1 Ôx 2 Õu Õ 2 dx 1 hôx 1 ÕÔλ 1 k 1 Ôx 2 x 1 Õu Ôt,x 2 x 1 Õ λ 1 k 1 Ôx 2 Õu Ôt,x 2 ÕÕdx 1 hôx 1 Õλ 1 k 1 Ôx 2 x 1 Õu Ôt,x 2 x 1 Õdx 1 λ 1 k 1 Ôx 2 Õu Ôt,x 2 Õ x 2 x2 hôx2 zõλ 1 k 1 ÔzÕu Ôt,zÕdz λ 1 k 1 Ôx 2 Õu Ôt,x 2 Õ hôx 2 zõλ 1 k 1 ÔzÕu Ôt,zÕdz λ 1 k 1 Ôx 2 Õu Ôt,x 2 Õ. (here we note k 1 ÔzÕ when z ). Therefore, from eq. (9.49), when γ 1, u approaches to the solution of eq. (9.42), and the desired result follows.

153 152 Hybrid Models to Explain Gene Expression Variability (S3). Now, we consider the case of scaling (S3) so λ n 2 nλ 2. From eq. (9.52) and proposition 61, we have Therefore, u k 1 k λ 2 u k 1 γ 2 Ôx 2 u k Õ 1 γ 1 x 2 knγ 1 x 2 1 knγ 1 λ 1 k 1 Ôx 2 Õ k 1 ô j1 k j Å u k jb j knγ 1 λ 1 k 1 Ôx 2 Õu b k 1 λ 1 k 1 Ôx 2 Õu b k 1 λ 2 u k 1 OÔ 1 knγ 1 k γ 1 x 2 n 2 Õ. u 1 1 λ 1 k 1 Ôx 2 Õu b 1 λ 2 u 2 OÔ 1 nγ 1 γ 1 x 2 n 2 Õ 1 λ 1 k 1 Ôx 2 Õu b 1 λ 2 1 Ö λ 1 k 1 Ôx 2 Õu b 2 1 λ 2 u 3 OÔ 1 nγ 1 γ 1 x 2 2nγ 1 2γ 1 x 2 n 2 Õ 1 λ 1 k 1 Ôx 2 Õu b 1 1 λ 2 nγ 1 2! nγ1 2 b 2 Öλ 1 k 1 Ôx 2 Õu x 2 1 2! Ôλ 2 Õ 2 1 Ö λ 1 k 1 Ôx 2 Õu b 3 1 λ 2 u 4 OÔ 1 γ 1 x 2 3nγ 1 3 γ 1 x 1 n 2 Õ 1 ô 1 nλ 2 k! Ô λ 2 Õ k k 1 b k γ k1 1 x k 1 Öλ 1 k 1 Ôx 2 Õu OÔ 1 n 2 2 Õ. Thus, when n, in a manner similar to the above argument, we have nλ 2 u 1 x 2 ô k1 y 1 k! Ô λ 2 Õ k k b k γ 1 x k Öλ 1 k 1 Ôx 2 Õu 2 hôy zõλ 1 k 1 ÔzÕu Ôt,zÕdz λ 1 k 1 Ôx 2 Õu Ôt,x 2 Õ, and the result follows. 9.4 From discrete to continuous bursting model We show here that the discrete bursting model BD1, converge either to a continuous deterministic model or to a continuous bursting model, when an appropriate scaling is used. The precise result is stated in paragraph We are going to state here results of convergence of Pure-Jump Markov processes using standard techniques [36]. We will look from now on the semigroup defined on the space of bounded measurable function, rather than on L 1. While going from the discrete model to the continuous model, one needs to make the local jumps smaller and smaller so that they will eventually becomes continuous, whereas the non-local jumps will stay discontinuous. Appropriate assumptions on the coefficient needs to be made. We give here a rigorous proof of the validity of the continuous approximation, using a classical generator limit. We obtain a convergence of the stochastic process, that contains more information than solely the asymptotic distribution. As said, these techniques are not knew, but seems to have been rarely used for Piecewise deterministic process with jumps (see for instance the recent reference [25], where various limiting processes are obtained in a general settings for a finite number of reaction).

154 9 From One Model to Another 153 For the sake of completeness, and to make apparent the specificity on the choices of scaling of coefficient, we first state a mean-field limit where the pure-jump Markov process converges to the solution of an Ordinary Differential Equation (theorem 65) and then state the convergence of the pure-jump Markov process towards the Piecewise deterministic process with jumps (theorem 66) Discrete model We look at the continuous-time Markov Chain X t on the positive integer space, with transition kernel given by ô KÔx,dyÕ γôxõδ 1ÔdyÕ λôxõ h r δ r ÔdyÕ where δ i denotes the Dirac mass in i. Let F t be the natural X t -adapted filtration. Then the following expression holds X t X where M t is a F t -Martingale, and M t t ô J nt γôx s Õ X n t r1 λôx s ÕEÖh ds R ykôxs,dyõds where J n are jump times of ÔX t Õ t. Then M t has for quadratic variation M t t Normalized discrete model γôx s Õ λôx s ÕE 2 Öh ds Wechangethereactionratesγ,λandjumpsizeprobabilityh r respectivelybyγ N,λ N,h N r. We note the associated solution X N and define the process M t X N ÔtÕ 1 N X N Then it is easy to see that X N is a continuous-time Markov chain of transition kernel ô K N Ôx,dyÕ γ N ÔNxÕδ 1 ÔdyÕ λ N ÔNxÕ h N r δ r ÔdyÕ N N and X N t X N t where M N t is an L 2 -Martingale 1 N γn ÔNXs N 1 Õ N λn ÔNXs N ÕEÖhN ds Mt N (9.54) M N t ô J N n t and M N t has for quadratic variation M N t t t Xn N r1 R ykn ÔX N s,dyõds 1 N 2γN ÔNXs N 1 Õ N 2λN ÔNXs N ÕE2 Öh N ds

155 154 Hybrid Models to Explain Gene Expression Variability Limit model 1 We look at the deterministic process defined by Limit model 2 x 1 t x t γôx 1 sõ λôx 1 sõeöh ds We look at the process defined by x t x t γôx s Õ λôx s ÕEÖh ds M t (9.55) where M t is an L 2 -Martingale M t ô J nt x Jn t R ykôx s,dyõds and KÔx s,dyõ λôx s Õ1 yxs hôy x s Õ. M t has for quadratic variation Convergence theorem 1 M t t λôx s ÕE 2 Öh ds The first result concern a classical fluid limit (or thermodynamic limit) when the jump intensity is faster and faster and the jump size smaller and smaller, such that the mean velocity stays finite. Because we include unbounded jump rate function, we need to restrict to convergence on compact time interval. Theorem 65. Let λ and γ be nonnegative locally Lipschitz functions on Ö, Õ, and h be a density function on Ô, Õ with a finite first moment, i.e. EÖh xhôxõdx. Take any T such that there is a unique solution to the ordinary differential equation on Ö,T, starting at x, dx dt λôxõeöh γôxõ. Now take a closed set D that contains the trajectory up to T, i.e. Ôx t Õ tt D. Let S be a relatively open set of D, S D. Suppose we have the following scaling laws, for any N and x, γ N ÔxÕ NγÔxÕ λ N ÔxÕ NλÔxÕ h N ÔxÕ x 1 x hôyõdy For any sequence N, let X N be the associated Pure-Jump Markov Process described above by eq. (9.54). Let τ N be the exit time of S, i.e. τ N inføt,xt N Ê SÙ. Then limx N ÔÕ x implies that, for every δ, lim P sup Xsτ N N N x sτn δ tt

156 9 From One Model to Another 155 Proof. This result is contained in many text books (see for instance [81, thm 2.11], or for the corresponding martingale method [26, thm 2.8]) and is the consequence of the three followings facts (according to [81, thm 2.11]). For any N, let S N S ã Ô 1 N NÕ. The time-averaged rate of change is always finite, sup N sup N sup λ N ÔxÕ xès N y xk N Ôx,dyÕ 1 N N sup γôxõ λôxõeöh xès N There exists a positive sequence δ N such that lim sup λ N ÔxÕ y xk N Ôx,dyÕ NxÈS N y xδ N Indeed, for any η, consider δ N maxôm,1õ 1 N and M is such that supλôxõ yhôyõdy η xès M The difference between the deterministic dynamical system and the time-averaged rates of change does to zero lim sup ÔλÔxÕEÖh γôxõõ λ N ÔxÕ Ôy xõk N Ôx,dyÕ NxÈS 1 N N N Convergence theorem 2 We are now going to show that the re-scaled discrete model converge to the limiting model 2 as N under the specific assumptions Hypothesis 1. γ N ÔNxÕ NγÔxÕ, for all x, λ N ÔNxÕ λôxõ, for all x, r1 er Nh θ N N r eyθ hôyõdy. We also suppose that the rates λ and γ are linearly bounded and γôõ, namely λôxõ λ λ 1 x γôxõ γ γ 1 x and γôõ The second hypothesis guarantee that the process stay non-negative, and the first one gives non-explosion property. We finally make the additional assumption y 2 hôyõdy, which will allow us to get a control of the second moment. We prove now that Theorem 66. Under hypotheses 1, the process X N t solution of eq converges in distribution in DÔÖ,T,R Õ towards x t, solution of eq. 9.55, for any T, as N. We will use standard argument and decompose the proof in 3 steps: tightness, identification of the limit and uniqueness of the limit.

157 156 Hybrid Models to Explain Gene Expression Variability Step 1: Tightness We start by proving some moment estimates. Using the expression of the transition kernel K N, it follows that EÖX N t EÖXN t EÖh N EÖλÔX N s Õ ds Then, due to the assumption on λ, EÖX N t EÖXN t EÖh N Ôλ λ 1 EÖX N s Õds Note that due to assumption on h N, EÖh N is convergent, hence bounded. Then, by Grönwall inequality, for any T, if EX N, we have sup EXt N tèö,t For any p 2, note that ô Ôx r1 r ô ô p p N Õp h N r xp p k rk Åx k N k h N r, r1 k1 pô Å p x p k ô k N k r k h N r, k1 r1 pô Å p x p k k N k Ek Öh N. k1 Then, we deduce EÖÔX N t Õp EÖÔX N Õp t Ôλ λ 1 EÖX N s Õ p ô k1 Å p EÖÔXs N k Õp k Ek Öh N N k ds. Hence, according to the assumptionon h N and Grönwall inequality, we showby recurrence on p, if EÖÔX N Õp, then sup EÖÔXt N Õp. tèö,t We prove by similar argument that supeö sup Xt N, n tèö,t supeö sup ÔXt N Õ2. n tèö,t Now note that X N t is the semi-martingale, with finite variation part V N t t and Martingale M N t of quadratic variation M N t γôx N s Õ λôxn s ÕEÖhN N ds, t 1 N γôxn s Õ λôxn s ÕE2 Öh N N 2 ds.

158 9 From One Model to Another 157 Then using moment estimates above and assumptions on rates γ and λ, it comes supeö sup Vt N, N tèö,t supeö sup M N t. N tèö,t Similarly, for any δ, for any sequence ÔS N,T N Õ of couples of stopping times such that S n T n T and T n S n δ, we can show that supeö VT N N VS N N N supeö M N TN M N SN N Cδ, Cδ, where C is a constant that depends only of λ,λ 1,γ,γ 1,h and T. Then by Aldous-Rebolledo and Roelly s criteria ([73],[119]), this ensures that X N t is tight in DÔR,R Õ with the standard Skorokhod topology. step 2: Identification of the limit Let s consider an adherence value x of the sequence X N and denote again X N the subsequence that converges in law to x in DÔÖ,T,R Õ. For any k, let t 1... t k s t T and φ 1,...,φ k È C b ÔR,RÕ. For y È DÔÖ,T,R Õ, we define, for suitable f ΨÔyÕ φ 1 Ôy t1 Õ...φ k Ôy tk Õ fôy t Õ fôy s Õ Then EÖΨÔxÕ A B C where A EÖΨÔxÕ EÖΨÔX n Õ, B C t s Ô γôy u Õf ½ Ôy u Õ λôy u Õ EÖΨÔX n Õ EÖφ 1 ÔX N t 1 Õ...φ k ÔX N t k ÕÔM f,n t EÖφ 1 ÔX N t 1 Õ...φ k ÔX N t k ÕÔM f,n t M f,n s Õ. hôyõfôy u yõdy fôy u ÕÕdu. M f,n s Õ, By the Martingale property, C. The map y È DÔÖ,T,R Õ ΨÔyÕ is continuous as soon Ôt 1,...,t k,s,tõ does not intersect a denumerable set of points of Ö,T where y is not continuous. Then the convergence in distribution of X N to x implies that A converges to when N. Finally, B E t s γôx N u Õ N f ¾ ÔX N u λôx N u Õ ε n u n Õ ô h N r fôxu N r1 r N Õ hôyõfôxu N yõdy du, with ε n u È Ö,1. Then B as N according to the assumptions above. step 3 : Uniqueness In step 2, we have shown that adherence values of X n has to be solution of the Martingale problem associated to the generator A, AfÔxÕ γôxõ df λôxõ fôx yõhôyõdy fôxõ. dx It is known ([27]) that under our assumption we have a strong solution of eq. (9.55), so uniqueness of the solution of the martingale problem associated to A holds, using [36, corollaire 4.4 p187] (see part proposition 8).

159 158 Hybrid Models to Explain Gene Expression Variability Interpretation Lets consider the master equation eq. (8.1) in the specific example 5. We can see this master equation as a biochemical master equation ([45]). Then, the degradation reaction being a first order reaction, the propensity γ n is independent of the size of the cell. But the burst production reaction is a zero-order reaction, and hence is proportional to the size, that is λ n λv 1 KnN Λ Kn N Note that in the last expression, the Hill function occurred as an elimination procedure of the repressors molecule (see for instance [91]). Parameters Λ and are dimensionless parameters, and theparameter K is thereaction rateconstant of thebindingof N proteins to a single repressor molecule, and then is the reaction rate constant of an ÔN 1Õ-order reaction. Then K K V N Now let define the rescaled variable X ε K 1ßN P x x ε X ε t x γ ε x X t K1ßN V, we get, with ε V, P x x rε X ε t x λk1ßn ε 1 x N Ô1 bõbr 1 Λ xn Themean burstsizeof thisrescaled variableisthen b 1 bε. Hencethejumpsbecome smaller and more frequent as ε. We recover in the limit a continuous and deterministic process, the situation of the theorem 65. Now suppose the burst production rate does not increase with the size of the cell, but the burst size does. With the scaling of theorem 66, if h is an exponential distribution of mean parameter b, then h ε is a geometric distribution of parameter 1 e bßε 1 as ε, and then the mean burst size increases inversely proportional to ε. Remark 67. In practice, if we don t know a priori the size of the system, we expect the following ε to be appropriate, depending on the case, ε γ Degradation rate λ Burst frequence ε K n ÔBinding Rate constantõ n ε 1 b 1 b mean burst size Much caution must be taken while choosing the ε, because in practice the size of the system doesn t go into infinity, so that a too small ε would lead to misunderstanding. For instance, In [153], one can found the following rates taken from other literatures: λ mrna 1,min 1 γ mrna 1,min 1 γ protein,1min 1 Number of protein for one mrna 3 so that the continuous approximation with ε.1 would give a degradation rate of order 1min 1, a bursting rate of order 1min 1 and a mean burst size of.3.

160 BIBLIOGRAPHY 159 Bibliography [1] E. Pecou A. Maass, S. Martinez. Mathematical comments on basic topics in Systems Biology in Mathematical and Computational Methods in Biology [2] M. Acar, A. Becskei, and A. van Oudenaarden. Enhancement of cellular memory by reducing stochastic transitions. Nature, 435: , [3] D. F. Anderson and T. G. Kurtz. Design and Analysis of Biomolecular Circuits (chapter 1). Springer, [4] T. Aquino, E. Abranches, and A. Nunes. Stochastic single-gene auto-regulation. Phys. Rev. E, 85:14, [5] L. Arnold. Random dynamical systems. Springer-Verlag, [6] K. Ball, T. G. Kurtz, L. Popovic, and G. Rempala. Asymptotic analysis of multiscale approximations to reaction networks. Ann. Appl. Probab., 16(4): , [7] J.-B. Bardet, A. Christen, A. Guillin, F. Malrieu, and P.-A. Zitt. Total variation estimates for the tcp process arxiv: [8] A. Becskei, B. Seraphin, and L. Serrano. Positive feedback in eukaryotic gene networks: cell differentiation by graded to binary response conversion. EMBO J., 2 (1): , [9] O. G. Berg. A model for the statistical fluctuations of protein numbers in a microbial population. J. Theor. Biol., 71(4):587 63, , 91 [1] N. Berglund and B. Gentz. Noise-Induced Phenomena in Slow-Fast Dynamical Systems,A Sample-Paths Approach. Springer, [11] W.J. Blake, M. Kaern, C.R. Cantor, and J.J. Collins. Noise in eukaryotic gene expression. Nature, 422: , [12] A. Bobrowski. Degenerate convergence of semigroups related to a model of stochastic gene expression. Semigroup Forum, 73(3): , , 12 [13] A. Bobrowski, T. Lipniacki, K. Pichór, and R. Rudnicki. Asymptotic behavior of distributions of mrna and protein levels in a model of stochastic gene expression. J. Math. Anal. Appl., 333(2): , , 85, 113 [14] P. Bokes, J. R. King, A. T. Wood, and M. Loose. Exact and approximate distributions of protein and mrna levels in the low-copy regime of gene expression. J. Math. Biol., 64: , , 9 [15] I. Bose and S. Ghosh. Binary response in microrna-mediated regulation of gene expression. pages 1 7, 212. arxiv: v2. 91 [16] F. J. Bruggeman, N. Blüthgen, and H. V. Westerhoff. Noise management by molecular networks. PLoS Comput Biol, 5(9):e156, [17] M.L. Bujorianu and J.-P. Katoen. Symmetry reduction for stochastic hybrid systems. IEEE Decis. Contr. P., pages ,

161 16 Hybrid Models to Explain Gene Expression Variability [18] M.L. Bujorianu and J. Lygeros. General stochastic hybrid systems: modelling and optimal control. IEEE Decis. Contr. P., 2, [19] M. O. Cáceres and A. A. Budini. The generalized ornstein - uhlenbeck process. J. Phys. A, 3(24):8427, [2] L. Cai, N. Friedman, and X.S. Xie. Stochastic protein expression in individual cells at the single molecule level. Nature, 44: , , 71, 72, 89 [21] T. Çagatay, M. Turcotte, M. B. Elowitz, J. Garcia-Ojalvo, and G. M. Süel. Architecture-dependent noise discriminates functionally analogous differentiation circuits. Cell, 139(3):512 22, [22] J.R. Chubb, T. Trcek, S.M. Shenoy, and R.H. Singer. Transcriptional pulsing of a developmental gene. Curr. Biol., 16: , , 52, 72, 92 [23] O.L.V. Costa and F. Dufour. Stability and ergodicity of piecewise deterministic markov processes. IEEE Decis. Contr. P., 47(Dec 9-11): , [24] A. Coulon, O. Gandrillon, and G. Beslon. On the spontaneous stochastic dynamics of a single gene: complexity of the molecular interplay at the promoter. BMC Syst. Biol., 4:2, [25] A. Crudu, A. Debussche, A. Muller, and O. Radulescu. Convergence of stochastic gene networks to hybrid piecewise deterministic processes. Ann. Appl. Probab., , 52, 119, 121, 123, 127, 13, 133, 139, 152 [26] R. W. R. Darling. Fluid limits of pure jump markov processes: a practical guide. 22. arxiv:math/ [27] M. H. A. Davis. Piecewise-deterministic markov processes: A general class of nondiffusion stochastic models. J Roy. Stat. Soc. B Met., 46(3): , , 133, 157 [28] M. Doumic and L. M. Tine. Estimating the division rate of the growth-fragmentation equation. J. Math. Biol., [29] M. Doumic Jauffret, M. Hoffmann, P. Reynaud-Bouret, and V. Rivoirard. Nonparametric estimation of the division rate of a size-structured population. SIAM J. Numer. Anal., pages, [3] Y. Dublanche, K. Michalodimitrakis, N. Kümmerer, M. Foglierini, and L. Serrano. Noise in transcription negative feedback loops: simulation and experimental analysis. Mol. Sys. Biol., 2:41, , 66, 67, 68, 7, 71, 91, 92 [31] H. El-Samad and M. Khammash. Modelling and analysis of gene regulatory network using feedback control theory. Int. J. Syst. Sci., 41(1):17 33, [32] J. Elf, G.-W. Li, and X. S. Xie. Probing Transcription Factor Dynamics at the Single-Molecule Level in a Living Cell. Science, 316: , [33] M B Elowitz and S Leibler. A synthetic oscillatory network of transcriptional regulators. Nature, 43(6767):335 8, [34] M.B. Elowitz, A.J. Levine, E.D. Siggia, and P.S. Swain. Stochastic gene expression in a single cell. Science, 297: , 22. 5, 91

162 BIBLIOGRAPHY 161 [35] K.-J. Engel and R. Nagel. One-parameter semigroups for linear evolution equations. Springer-Verlag, [36] S. N. Ethier and T. G. Kurtz. Markov Processes: Characterization and Convergence. Wiley Interscience, , 12, 152, 157 [37] R. Feistel and W. Ebeling. Evolution of Complex Systems. VEB Deutscher Verlag der Wissenschaften, [38] N. Fenichel. Geometric singular perturbation theory for ordinary differential equations. J. Differ. Equations, 31(1):53 98, , 129 [39] N. Friedman, L. Cai, and X.S. Xie. Linking stochastic dynamics to population distribution: An analytical framework of gene expression. Phys. Rev. Lett., 97: /4, , 52, 89, 12, 113, 114, 13 [4] C. Gadgil, C. H. Lee, and H. G. Othmer. A stochastic analysis of first-order reaction networks. B. Math. Biol., 67:91 946, [41] T. Galla. Intrinsic fluctuations in stochastic delay systems: Theoretical description and application to a simple model of gene regulation. Phys Rev E Stat Nonlin Soft Matter Phys, 8(2):1 9, [42] C. W. Gardiner. Handbook of stochastic methods, for physics, chemistry and the natural sciences. Springer-Verlag, [43] T. S. Gardner, C. R. Cantor, and J. J. Collins. Construction of a genetic toggle switch in Escherichia coli. Nature, 43(6767):339 42, 2. 55, 114 [44] I. M. Gel fand and N. Ya. Vilenkin. Generalized Functions - vol. 4: Applications of harmonic analysis. Academic Press, [45] D. T. Gillespie. Exact stochastic simulation of coupled chemical reactions. J. Phys. Chem. A, 81(25): , , 113, 114, 158 [46] I. Golding and E. C. Cox. Eukaryotic transcription: What does it mean for a gene to be on? Curr. Biol., 16(1):R371 73, [47] I. Golding, J. Paulsson, S.M. Zawilski, and E.C. Cox. Real-time kinetics of gene activity in individual bacteria. Cell, 123: , , 52, 69, 7, 72, 91, 92 [48] B. C. Goodwin. Oscillatory behavior in enzymatic control processes. Adv. Enzyme. Regul., 3: , , 53, 75, 76, 87 [49] A. N. Gorban, A. Zinovyev, N. Morozova, and A. Harel. Modeling coupled transcription, translation and degradation and mirna-based regulation of this process. pages, 212. arxiv: v1. 91 [5] J. Goutsias. Quasiequilibrium approximation of fast reaction kinetics in stochastic biochemical systems. J. Chem. Phys., 122(18): , [51] J.S. Griffith. Mathematics of cellular control processes. I. Negative feedback to one gene. J. Theor. Biol., 2:22 28, , 76 [52] J.S. Griffith. Mathematics of cellular control processes. II. Positive feedback to one gene. J. Theor. Biol., 2:29 216, , 76

163 162 Hybrid Models to Explain Gene Expression Variability [53] S. Hahn and E. T. Young. Transcriptional regulation in saccharomyces cerevisiae: Transcription factor regulation and function, mechanisms of initiation, and roles of activators and coactivators. Genetics, 189(3):75 736, , 65 [54] H Haken. Synergetics: An introduction. Springer-Verlag, [55] E. L. Haseltine and J. B. Rawlings. Approximate simulation of coupled fast and slow reactions for stochastic chemical kinetics. J. Chem. Phys., 117(15): , [56] E. L. Haseltine and J. B. Rawlings. On the origins of approximations for stochastic chemical kinetics. J. Chem. Phys., 123(16): , [57] J Hasty, J Pradines, M Dolnik, and J J Collins. Noise-based switches and amplifiers for gene expression. Proc. Natl. Acad. Sci., 97(5):275 8, [58] J Hasty, D McMillen, F Isaacs, and J J Collins. Computational studies of gene regulatory networks: in numero molecular biology. Nat. Rev. Genet., 2(4): , [59] K.M. Hawkins and C.D. Smolke. The regulatory roles of the galactose permease and kinase in the induction response of the GAL network in Saccharomyces cerevisiae. J. Biol. Chem., 281: , [6] J. P. Hespanha. Modelling and analysis of stochastic hybrid systems. IEE P-Contr. Theor. Ap., 153(5):52 535, , 13 [61] T. Hida and Si Si. An Innovation approach to random fields, Application of White Noise Theory. World Scientific Pub. Co., , 142 [62] T. Hida and Si Si. Lectures on White Noise Functionals. World Scientific Pub. Co, [63] J. E. M. Hornos, D. Schultz, G. C. P. Innocentini, J. Wang, A. M. Walczak, J. N. Onuchic, and P. G. Wolynes. Self-regulating gene: An exact solution. Phys. Rev. E, 72(5):5197 -, [64] W. Horsthemke and R. Lefever. Noise Induced Transitions: Theory and Applications in Physics, Chemistry, and Biology. Springer-Verlag, [65] D. Huh and J. Paulsson. Non-genetic heterogeneity from stochastic partitioning at cell division. Nat. Genet., 43(2):95 1, [66] P. J. Ingram, M. P. H. Stumpf, and J. Stark. Nonidentifiability of the source of intrinsic noise in gene expression from single-burst data. PLoS Comput Biol, 4(1): e1192, [67] F. J. Isaacs, J. Hasty, C. R. Cantor, and J. J. Collins. Prediction and measurement of an autoregulatory genetic module. Proc. Natl. Acad. Sci., 1(13): , , 65, 66, 67 [68] S. Iyer-Biswas, F. Hayot, and C. Jayaprakash. Stochasticity of gene products from transcriptional pulsing. Phys Rev E Stat Nonlin Soft Matter PhysE, 79(3):1 9,

164 BIBLIOGRAPHY 163 [69] F. Jacob, D. Perrin, C. Sánchez, and J. Monod. L opéron: groupe de gènes à expression coordonnée par un opérateur. C. R. Acad. Sci. Paris, 25: , [7] J. Jacod and A. N. Shiryaev. Limit theorems for stochastic processes. Springer- Verlag, , 14, 142 [71] T. Jia. Stochastic Modeling of Gene Expression and Post-transcriptional Regulation. PhD thesis, Faculty of the Virginia Polytechnic Institute and State University, [72] T. Jia and R. Kulkarni. Post-Transcriptional Regulation of Noise in Protein Distributions during Gene Expression. Phys. Rev. Lett., 15(1):1 4, [73] A. Joffe and M. Metivier. Weak convergence of sequences of semimartingales with applications to multitype branching processes. Adv. Appl. Probab., 18(1):2 65, [74] M. Kaern, T.C. Elston, W.J. Blake, and J.J. Collins. Stochasticity in gene expression: From theories to phenotypes. Nat. Rev. Genet., 6: , 25. 5, 56, 83, 119 [75] H.-W. Kang and T. G. Kurtz. Separation of time-scales and model reduction for stochastic reaction networks. Ann. Appl. Probab. (to appear), , 52, 61, 119, 121, 123, 13, 133 [76] G. Karlebach and R. Shamir. Modelling and analysis of gene regulatory networks. Nat. Rev. Mol. Cell. Biol., 9(1):77 78, [77] T.B. Kepler and T.C. Elston. Stochasticity in transcriptional regulation: Origins, consequences, and mathematical representations. Biophys. J., 81: , , 92, 113, 119 [78] A. M. Kierzek, J. Zaim, and P. Zielenkiewicz. The effect of transcription and translation initiation frequencies on the stochastic fluctuations in prokaryotic gene expression. J. Biol. Chem., 276(11): , , 68, 7, 71, 91 [79] J. J. Kupiec. A Darwinian theory for the origin of cellular differentiation. Mol. Gen. Genet., 255:21 28, [8] J. J. Kupiec. On the lack of specificity of proteins and its consequences for a theory of biological organization. Prog. Biophys. Mol. Bio., 12(1):45 52, [81] T. G. Kurtz. Solutions of ordinary differential equations as limits of pure jump markov processes. J. Appl. Probab., 7(1):pp , [82] T. G. Kurtz. Random time changes and convergence in distribution under the meyer-zheng conditions. Ann. Prob., 19(3):11 134, [83] A. Lasota and M. C. Mackey. Probabilistic properties of deterministic systems. Cambridge University Press, , 147, 148 [84] A. Lasota and M.C. Mackey. Chaos, fractals, and noise. Springer-Verlag, [85] J. Lei. Stochasticity in single gene expression with both intrinsic noise and fluctuation in kinetic parameters. J. Theor. Biol., 256(4):485 92,

165 164 Hybrid Models to Explain Gene Expression Variability [86] J. Lei. Stochastic Modeling in Systems Biology. J. Adv. Math. Appl, in press:1 23, [87] T. Lipniacki, P. Paszek, A. Marciniak-Czochra, A. R. Brasier, and M. Kimmel. Transcriptional stochasticity in gene expression. J. Theor. Biol., 238(2): , , 92, 113, 119 [88] A. Lipshtat, A. Loinger, N. Balaban, and O. Biham. Genetic Toggle Switch without Cooperative Binding. Phys. Rev. Lett., 96(18):1 4, [89] M. C. Mackey and M. Santillán. Dynamic regulation of the tryptophan operon : A modeling study and comparison with experimental data. Proc. Natl. Acad. Sci., 98 (4): , , 65, 66, 67, 68, 7, 71 [9] M. C. Mackey and M. Tyran-Kamińska. Dynamics and density evolution in piecewise deterministic growth processes. Ann. Polon. Math., 94(2): , , 99, 1, 11, 134, 147, 148 [91] M. C. Mackey, M. Tyran-Kaminska, and R. Yvinec. Molecular distributions in gene regulatory dynamics. J. Theor. Biol., 274(1):84 96, , 52, 93, 12, 158 [92] M. C. Mackey, J. Lei, R. Yvinec, and C. Zhuge. Adiabatic reduction of jump processes in a model of stochastic gene expression with bursting transcription arxiv: , 119, 13 [93] M. C. Mackey, M. Tyran-Kaminska, A. Ramos, and R. Yvinec. Discrete and continuous stochastic gene expression models with bursting In preparation. 52, 93 [94] L. Mariani, E. G. Schulz, M. H. Lexberg, C. Helmstetter, A. Radbruch, M. Löhning, and T. Höfer. Short-term memory in gene induction reveals the regulatory principle behind stochastic IL-4 expression. Mol. Sys. Biol., 6:359, 21. 7, 71, 114 [95] H. H. McAdams and A. Arkin. Stochastic mechanisms in gene expression. Proc. Natl. Acad. Sci., 94(3): , [96] H. H. McAdams and A. Arkin. It s a noisy business! Genetic regulation at the nanomolar scale. Trends genet., 15(2):65 9, [97] S. P. Meyn and R. L. Tweedie. Stability of markovian processes iii: Foster-lyapunov criteria for continuous-time processes. Adv. Appl. Probab., 25(3): , , 73, 74, 115 [98] E. L. O Brien, E. Van Itallie, and M. R. Bennett. Modeling synthetic gene oscillators. Math. Biosci., 236(1):1 15, [99] A. Ochab-marcinek and M. Tabaka. Bimodal gene expression in noncooperative regulatory systems. Proc. Natl. Acad. Sci., 17(51): , [1] H.G. Othmer. The qualitative dynamics of a class of biochemical control circuits. J. Math. Biol., 3:53 78, , 76, 8 [11] P. Paszek. Modeling stochasticity in gene regulation: characterization in the terms of the underlying distribution function. B. Math. Biol., 69(5): , 27. 9

166 BIBLIOGRAPHY 165 [12] J. Paulsson. Models of stochastic gene expression. Phys. Life Rev., 2(2): , , 83, 118 [13] A. Pazy. Semigroups of Linear Operators and Applications to Partial Differential Equations. Springer, [14] J. Peccoud and B. Ycart. Markovian modelling of Gene Product Synthesis. Theor. Popul. Biol., 48(2): , , 74, 84, 118 [15] J. M. Pedraza and J. Paulsson. Effects of Molecular Memory and Bursting on Fluctuations in Gene Expression. Science, 319(5861): , [16] K. Pichór and R. Rudnicki. Continuous Markov semigroups and stability of transport equations. J. Math. Anal. Appl., 249: , [17] A. Polynikis, S.J. Hogan, and M. di Bernardo. Comparing differeent ODE modelling approaches for gene regulatory networks. J. Theor. Biol., 261:511 53, [18] Yu. V. Prokhorov. Convergence of random processes and limit theorems in probability theory. Theory Probab. Appl., 1(2): , [19] A. Raj and A. van Oudenaarden. Nature, nurture, or chance: Stochastic gene expression and its consequences. Cell, 135: , 28. 5, 56 [11] A. Raj and A. van Oudenaarden. Single-Molecule Approaches to Stochastic Gene Expression. Annu. Rev. Biophys., 38:255 27, 29. 5, 83 [111] A. Raj, C.S. Peskin, D. Tranchina, D.Y. Vargas, and S. Tyagi. Stochastic mrna synthesis in mammalian cells. PLoS Biol., 4: , , 52, 69, 7, 72, 92 [112] A. Ramos, G. Innocentini, and J. Hornos. Exact time-dependent solutions for a self-regulating gene. Phys Rev E Stat Nonlin Soft Matter Phys, 83(6):1 4, [113] S. Ramsey, A. Ozinsky, A. Clark, K. D. Smith, P. de Atauri, V. Thorsson, D. Orrell, and H. Bolouri. Transcriptional noise and cellular heterogeneity in mammalian macrophages. Philos. T. Roy. Soc. B, 361(1467):495 56, , 68, 7, 71 [114] C. V. Rao and A. P. Arkin. Stochastic chemical kinetics and the quasi-steady-state assumption: Application to the gillespie algorithm. J. Chem. Phys., 118(11): , [115] J.M. Raser and E.K. O Shea. Control of stochasticity in eukaryotic gene expression. Science, 34: , , 119 [116] A. S. Ribeiro. Stochastic and delayed stochastic models of gene expression and regulation. Math. Biosci., 223(1):1 11, , 92 [117] I. Roeder and I. Glauche. Towards an understanding of lineage specification in hematopoietic stem cells: A mathematical model for the interaction of transcription factors GATA-1 and PU.1. J. Theor. Biol., 241: , [118] R. G. Roeder. Transcriptional regulation and the role of diverse coactivators in animal cells. FEBS lett., 579(4):99 15, , 65 [119] S. Roelly. A criterion of convergence of measure-valued processes: application to measure branching processes. Stochastics, 17(1-2):43 65,

167 166 Hybrid Models to Explain Gene Expression Variability [12] N. Rosenfeld, J.W. Young, U. Alon, P. S. Swain, andm. B. Elowitz. Generegulation at the single-cell level. Science, 37: , , 67, 91 [121] W. Rudin. Functional analysis. McGraw-Hill Scienc, [122] F. Sagués, J. Sancho, and J. García-Ojalvo. Spatiotemporal order out of noise. Rev. Mod. Phys., 79(3): , [123] M. Santillán and M. C. Mackey. Dynamic behavior in mathematical models of the tryptophan operon. Chaos, 11(1): , , 64 [124] M. Santillán and H. Qian. Irreversible thermodynamics in multiscale stochastic dynamical systems. Phys Rev E Stat Nonlin Soft Matter Phys., 83:1 8, [125] H.H. Schaefer. Topological Vector Spaces. Springer-Verlag, , 141 [126] D. Schultz, J. N. Onuchic, and P. G. Wolynes. Understanding stochastic simulations of the smallest genetic networks. J. Chem. Phys., 126(24): , [127] B. Schwanhäusser, D.a Busse, N. Li, G. Dittmar, J. Schuchhardt, J. Wolf, W. Chen, and M. Selbach. Global quantification of mammalian gene expression control. Nature, 473(7347):337 42, , 71 [128] J.F. Selgrade. Mathematical analysis of a cellular control process with positive feedback. SIAM J. Appl. Math., 36: , , 8 [129] V. Shahrezaei and P.S. Swain. Analytic distributions for stochastic gene expression. Proc. Natl. Acad. Sci., 15: , , 88, 89, 93, 96, 11, 113, 119, 123 [13] V. Shahrezaei and P.S. Swain. The stochastic nature of biochemical networks. Curr. Opin. Biotechnol., 19: , [131] V. Shahrezaei, J.F. Ollivier, and P.S. Swain. Colored extrinsic fluctuations and stochastic gene expression. Mol. Syst. Biol., 4:196 25, [132] A. Singh and J. P. Hespanha. Noise suppression in auto-regulatory gene networks. IEEE Decis. Contr. P., 2: , [133] H.L. Smith. Monotone Dynamical Systems. American Mathematical Society, , 8, 82 [134] C. Song, H. Phenix, V. Abedi, M. Scott, B. P. Ingalls, and M. Krnand T. J. Perkins. Estimating the Stochastic Bifurcation Structure of Cellular Networks. PLoS Comput. Biol., 6:e1699/1 11, , 114 [135] M. Stamatakis and N. V. Mantzaris. Comparison of deterministic and stochastic models of the lac operon genetic network. Biophys. J., 96(3):887 96, , 65, 66, 68, 7, 71 [136] G. Storz and L. S. Waters. Regulatory rnas in bacteria. Cell, 136(4): , [137] G. M. Süel, J. Garcia-Ojalvo, L. M. Liberman, and M. B. Elowitz. An excitable gene regulatory circuit induces transient cellular differentiation. Nature, 44(783): ,

168 BIBLIOGRAPHY 167 [138] P.S. Swain, M.B. Elowitz, and E.D. Siggia. Intrinsic and extrinsic contributions to stochasticity in gene expression. Proc. Natl. Acad. Sci., 99: , [139] Y. Tao, Y. Jia, and T. G. Dewey. Stochastic fluctuations in gene expression far from equilibrium: Omega expansion and linear noise approximation. J. Chem. Phys., 122 (12):12418, [14] Y. Tao, X. Zheng, and Y. Sun. Effect of feedback regulation on stochastic gene expression. J. Theor. Biol., 247(4):827 36, [141] P. K. Tapaswi, R. K. Roychoudhury, and T. Prasad. A Stochastic Model of Gene Activation and RNA Synthesis during Embryogenesis. Sankhya Ser. A, 49(1):51 67, [142] M. Thattai and A. van Oudenaarden. Intrinsic noise in gene regulatory networks. Proc. Natl. Acad. Sci., 98(15): , , 66, 67, 68, 69, 86, 87 [143] A. N. Tikhonov. Systems of differential equations containing small parameters in the derivatives. Mat. Sb. (N.S.), 31 (73): , , 129 [144] T.-L. To and N. Maheshri. Noise can induce bimodality in positive transcriptional feedback loops without bistability. Science, 327(5969):1142 5, , 7, 71, 86 [145] M. Tyran-Kamińska. Substochastic semigroups and densities of piecewise deterministic markov processes. J. Math. Anal. Appl., 357(2):385 42, , 94, 99, 147 [146] J. J. Tyson. On the existence of oscillatory solutions in negative feedback cellular control processes. J. Math. Biol., 1(4): , [147] X. S. Xie, P. J. Choi, G.-W. Li, N. K. Lee, and G. Lia. Single-Molecule Approach to Molecular Biology in Living Bacterial Cells. Annu. Rev. Biophys., 37: , [148] G. Yagil and E. Yagil. On the relation between effector concentration and the rate of induced enzyme synthesis. Biophys. J., 11(1):11 27, [149] N. Yildirim, M. Santillán, D. Horike, and M. C. Mackey. Dynamics and bistability in a reduced model of the lac operon. Chaos, 14: , [15] J. Yu, J. Xiao, X. Ren, K. Lao, and X.S. Xie. Probing gene expression in live cells, one protein molecule at a time. Science, 311:16 163, , 52, 69, 72, 92 [151] I. Zacharioudakis, T. Gligoris, and D. Tzamarias. A yeast catabolic enzyme controls transcriptional memory. Curr. Biol., 17: , [152] D. Zenklusen, D. R. Larson, and R. H. Singer. Single-RNA counting reveals alternative modes of genes expression in yeast. Nat. Struct. Mol. Biol, 15(12): , 28. 7, 71 [153] E. S. Zeron and M. Santillán. Distributions for negative-feedback-regulated stochastic gene expression: Dimension reduction and numerical solution of the chemical master equation. J. Theor. Biol., 264(2): , , 158 [154] D. Zhou and R. Yang. Global analysis of gene transcription regulation in prokaryotes. Cell. Mol. Life. Sci., 63(19-2):226 9, , 65

169

170 Chapter 2 Study of stochastic Nucleation-Polymerization Prion Protein Model 169

171 17 Hybrid Models to Explain Protein Aggregation Variability This chapter deals with protein aggregation models. These models are dealing with the dynamics of the formation of polymers (aggregates) formed of proteins, and related to a number of applications in physics and biology. In section 1, the biological problem associated to prion diseases is presented, along with the experimental observations, obtained by the biologists who work with us, and the interesting questions they raised. We also review the literature on aggregation kinetic models. The application of our theoretical work (to be described below) to the specific model of prion diseases, was done in a collaboration with a team of biologists, directed by Jean-Pierre Liautard (Centre de Recherche sur les Pathogènes et Biologie pour la Santé (CPBS), Université Montpellier-2). In vitro nucleation-polymerization experiments has been analyzed quantitatively, and specially their heterogeneity. In section 2, the formulation of the chosen model is presented, in order to investigate the questions raised by the experimental observations. This model is composed of a discrete size Becker-Döring model with finite maximal size, and a discrete size polymerizationfragmentation model. Then, a time-scale reduction is performed, based on biological hypotheses, to reduce the complexity of the model. This reduction highlights links between a conservative form and a non-conservative form of the Becker-Döring model. In section 3, the first assembly time of a given fixed size aggregate is studied. Both a conservative and non-conservative form of a Becker-Döring model are used. Our main findings is that the stochastic and finite particle formulation gives different results from the deterministic and infinite particle formulation. In particular, we are able to characterize some discrepancies, to highlight finite system-size effect and to quantify the stochasticity in the first assembly time. In a stochastic formulation, the first assembly time may never be reached (and hence has an infinite mean time), and displays surprising non-monotonicity with respect to aggregation rates. Also, it is found that the mean first assembly time has very different relationship with respect to the initial quantity of particle, depending on the parameter region. Indeed, the mean first assembly time may be strongly correlated with the initial quantity of particle or very weakly. Finally, the distribution of this first assembly time can have various different forms(exponential, Weibull, bimodal), and may be far from a symmetric Gaussian, as a typical mean-field approach would have predicted. Then, such findings may have significant importance when analyzing aggregation experiments, and help us to understand the experimental observations on prion experiments. This study has been the subject of a preprint, with Maria R D Orsogna and Tom Chou. In section 4, the large population limit is investigated. Starting from a purely individual and stochastic polymerization-fragmentation model (sometimes called the direct simulation process), a convergence towards a hybrid infinite monomer population / finite polymer population is shown. This study follows many recent contributions on limit theorem from discrete to continuous model. In particular, standard martingale techniques are used to obtain a convergence in law of the stochastic process. The novelty lies in the fact that the asymptotic model seems to have never been applied in such field. Its hybrid structure may be a good balance between fully discrete and fully continuous model, and may be well adapted to quantify the heterogeneity of the prion proliferation observed experimentally. This work is an ongoing project with Erwan Hingant (Université Lyon 1). The aim of our analytical study developed in both sections 3-4 is to quantify the amount of stochasticity, to validate or invalidate kinetic hypotheses, and to deduce parameter values from experiments. This work is an ongoing project with Teresa Alvarez- Martinez, Samuel Bernard, Jean-Pierre Liautard and Laurent Pujo-Menjouet.

172 1 Introduction Introduction In this chapter, mathematical models of protein aggregation kinetics are studied. These models are conceived to represent faithfully the aggregation dynamics of a particular protein, the prion protein, and to explain the experimental observations. Thus, we start to introduce the necessary biological concepts and motivations, before going to the mathematical study. Firstly, the diseases linked to the dynamics of aggregation of this prion protein are reviewed in section 1.1. Secondly, the main kinetic hypotheses for this protein aggregation model are introduced in subsections Thirdly, to motivate the mathematical study of such a model, the different experimental techniques used for prion modeling are presented in subsection 1.3. The specific in vitro experiments we used on prion aggregation kinetic are described in subsection 1.4, and the main unusual feature associated to it is explained. Finally, we end up this introduction by a mini literature review on coagulation-fragmentation model, in order to give an overall picture of the field. 1.1 Biological background: what is the prion? Diseases such as Creutzfeldt-Jacob or Kuru for human, and bovine spongiform encephalopathies (BSE), scrapie (in sheep) or chronic-wasting disease for animals are all spongiform encephalopathies and belong to a larger class of neurodegenerative disorders ([13]). The key features of spongiform encephalopathies are the followings: they are transmissible, and the agent responsible for such transmission is a protein (rather than a virus, bacteria...), called prion. It is usually referred to the proteinonly hypothesis, and to any disease related to it as a prion disease ; they are characterized by a long incubation time (up to 5 years in humans). This phaseisfollowedbyarapidanddramaticclinicalphase(somemonthsorafewyears), leading to brain damage and death. Symptoms are convulsions, dementia, ataxia (balance and coordination dysfunction), and behavioral or personality changes; they affect the structure of the brain or other neural tissue, and amyloid plaques, formed of protein aggregates, are observed. Such region are spongiform. No immune response has been detected; No treatments are known, and no diagnostic during the incubation time are known. From an historical point of view, the biologist Tikvah Alper and mathematician John Stanley Griffith ([64]) first developed the hypothesis during the 196s that some transmissible spongiform encephalopathies are caused by an infectious agent consisting solely of proteins. This hypothesis had lots of impact, in molecular biology, for its potential contradiction with the so-called central dogma (see chapter 1). It was in 1982 that Stanley B. Prusiner announced that his team had purified the hypothetical infectious prion, and that the infectious agent consisted only of a specific protein ([123]). Nowadays, prion diseases are still a major public health issue. Such diseases are then transmissible, within a same species or from species to species (including from animals to human), or can also appear spontaneously. The control of occurrences and transmissions of such diseases is related to a better understanding of involved mechanism inside organisms. The difficulty is that the mechanisms involved occur at very different time scale, including large time scale, hardly captured by experimental observations. Then there have been numerous theoretical modeling approaches to help understanding such mechanisms (see subsection 1.5 for a small review). It has generally been accepted that spongiform encephalopathies result from the aggregation of an ubiquitous protein, the so-called prion protein, into amyloids ([29], [39], [123]). It is also believed that the formation of prion amyloid is due to a change of the

173 172 Hybrid Models to Explain Protein Aggregation Variability prion protein conformation ( [97], [38]). The normal (or non-pathological) conformation of this protein is called PrP C (standing for cellular Prion Protein). This protein can misfold (change conformation), and the misfolded protein has a tendency to form aggregates. These aggregates are referred as PrP Sc (standing for Scrapie Prion Protein). The aggregation process leads to a decrease of PrP C level by a conversion mechanism. One difficulty of understanding the cause of the pathology relies on the very different form prion aggregates can take, and the many different possible kinetic pathways that lead to such aggregates (see the next paragraph for aggregation kinetics controversy). In particular, to the best of our knowledge, it is not sure what is the exact cause of the disease. It could be due either to some specific form of aggregates it is not known actually which of the different aggregate forms of the prion could be toxic, and what are the exact pathogenic mechanisms leading to the disease [74] or, as said above, it could be due to a PrP C monomer decay. The protein population decreases is indeed the consequence of protein polymerization to the PrP Sc polymers after a specific conformation change. However, in any case, the overall dynamic of the process is still relevant to understand the main features of the disease Debates on different aggregation kinetics. In the previous decades, the kinetic of amyloid formations has been the subject to extensive researches and is still currently under investigation. For a good review on protein aggregation kinetics, see [11] for instance. One of the particularity of prion protein aggregation is that the different and many possible pathways leading to the formation of amyloid fibers from single proteins (monomers) or pre-formed seeds (polymers) are not fully understood and still subject to controversy [83], [72]. The early process of transconformation of prion protein is also subject to debate. It is generally accepted that this process does not involve any other molecules although it could be mediated by another misfolded protein ([94], [123], [5]). Recent studies using dynamic models tried to explain possible routes of spontaneous protein folding ([2],[41]). 1.2 The Lansbury s nucleation/polymerization theory The main stream molecular theory to explain the prion polymer dynamic is the one introduced by Lansbury et al. in 1995 [29]. In this paper, the authors investigate the formation of large aggregates of proteins ordered by specific contacts. The model, based on nucleation-dependent protein polymerization, describes various well-characterized processes, including protein crystallization, microtubule assembly, flagellum assembly, sicklecell hemoglobin fibril formation, bacteriophage procapsid assembly, actin polymerization and amyloid polymerization. Inspiring different groups of biologists and mathematicians who tried later on to improve this first model, their ideas are based on the following biological assumptions. The normal PrP C protein does not aggregate by itself. But a misfolded form of it is able to aggregate, and the aggregates are called PrP Sc. Such misfolded form can appear spontaneously from spatial and chemical modification of PrP C. When PrP Sc are present, they start to aggregate the misfolded protein by addition of one by one protein. Firstly, the early aggregation formation requires a series of association steps that are thermodynamically unfavorable (with an association constant K 1). These aggregation steps are unfavorable up to a given size (that is not currently known), which is referred to the nucleus size. Secondly, once a nucleus is formed, further addition of monomer becomes thermodynamically favorable (with an association constant K 1) resulting in rapid polymerization/growth ([49], [26], [4], [6]). The model is the named nucleation-dependent polymerization model, be-

174 1 Introduction 173 cause the overall polymerization dynamic depend strongly on whether a nucleus is present or not. Starting from a homogeneous pool of monomer, the formation of the first nucleus (an event called nucleation), leads to a drastic change in the dynamic. The first step, corresponding to nucleation, is a very unstable process and can be more stochastic than deterministic, while the second and further steps would be quite straightforward and more deterministic. According to this theory, because of its high stochasticity, nucleus formation would be considered as a kinetic barrier to sporadic prion diseases. But this barrier could be overcome by infection with a large polymer. The disease would not be spontaneous anymore, it could be transmitted (on purposeor not) by a PrP Sc polymer (called seed) which would directly lead to the second deterministic step since no formation of the first nucleus would be required. Finally, long PrP Sc polymers are also subject to fragmentation. They can break to smaller polymers, which lead to a multiplication of aggregation sites, and then to an exponential growing phase of the total protein mass contained in polymers [29]. 1.3 Experimental observations available There are mainly four levels on which experimental data on prion diseases can be collected. A first level is a population level. The number of infected people can be recorded and followed along time. For humans, due to the difficulty of the diagnostic and the long incubation time, few significant and robust data exists. The situation is slightly better for animals, specifically on bovines (mostly in Europe) or deers (North-America) [143]. A second level is the cellular level. It is possible to follow an in vivo cell population in animals, or to make a culture of cells, infected by PrP Sc aggregate. However, for both, the great complexity of cell dynamics (extra cellular interactions, different feedbacks, etc.) make it hard to collect pertinent information on the dynamics of the event that lead to cell infection. An open question concerned the interaction between the prion amyloids and the subcellular environment (where the prions are formed? how does it depends on the cell behaviour? and so on...). See [11] for some related questions. A third level, which we will be interested in, is the protein level. The progress of physical methods and techniques has made possible to partially study the structure of prion protein, for both the PrP C and the PrP Sc. Then a variety of different structures of prion amyloids have been characterized (see [19, 121] for some review of what is known on the molecular basis). However, due to the highly unstable form of the misfolded prion monomer, and its small size aggregates, the intermediate form (between the monomer to large polymer) are not well characterized. Still at this level, recent techniques allow to perform in vitro conversion of prion protein into PrP Sc polymer, and to follow the dynamic of this conversion through fluorescence markers. These techniques requires to use a modified form of the PrP C, called the recombinant PrP C. From a homogeneous pool of recombinant PrP C protein, the formation of polymer and larger amyloids is observable. The amount of mass (or rather the intensity of fluorescence, supposedly linearly correlated) that is present in polymers can be recorded trough time, within a time scale that is conceivable in a laboratory (typically 24h or aweek). Themaindrawbacksof suchmethod is that the recombinant PrP C protein has been modified chemically, and may not hence repro-

175 174 Hybrid Models to Explain Protein Aggregation Variability duce faithfully the feature of the original prion PrP C protein. It also requires high protein concentration, to a level that exceeds physiological concentration. Whether or not the obtained amyloids are able to generate infectiousity is also still unclear [138]. Finally, let us mention that some techniques also permit to measure the size of the amyloid obtained experimentally. A fourth level, even smaller, concern the atomic level of the protein. The idea is to precisely understand the physical and spatial structure of the protein, to characterize its stability and investigate all possible transconformation [2]. In vitro polymerization experiments of prion protein give some interesting insights of what could be the different mechanisms involved in the process. Interestingly, a main dynamical characteristic of the mechanism is used experimentally. Indeed, the PMCA (Protein Misfolding Cyclic Amplification) consists of successive phase of incubation and sonication in order to obtain lot of polymer fragments. During incubation, the polymer are supposed to growth by aggregation, and the sonication breaks large polymers, and hence speed up the next incubation phase, and so on. Agitating during polymerization experiments also speed up the polymerization process. We discern between two kinds of in vitro polymerization experiments: Those started with a homogeneous pool of protein recombinant are called nucleation experiments. In these experiment, the time required for the polymerization to truly start can be measured. According to Lansbury s theory, such time is related to the waiting time for one nucleus to appear. We refer either to the first assembly time, to the nucleation time, or to the lag time. A second kind of experiments is the seeding experiment. In such experiments, a preformed seed (a large polymer) is present initially with and a pool of recombinant prion protein. In both experiments, as well as in nucleation experiments, we can record through time the intensity of fluorescence, which relates to the total mass present in polymers. Such measures allow in particular to look at the speed of the polymerization process. We present more in detail in the next section the qualitative and quantitative behavior of the nucleation-polymerization process. For in vitro polymerization experiments, one of the challenges resides in the low sensitivity to the dynamical properties of the polymerization on initial concentration of prion protein ([13], [54], [115], [12]), as well as to the high heterogeneity of the outcomes. But before precisely defining such concept, the result of polymerization experiments are shown in details. 1.4 Observed Dynamics We present here the in vitro polymerization experiments performed by the biologists who work with us. All experiments were previously published [1], [3]. Firstly, we give details about the experimental set up. Secondly, we present a typical outcome of a polymerization experiments. Thirdly, we show statistics on the nucleation time and polymerization speed deduced from the nucleation experiments. Finally, we explain the qualitative features of the seeding experiments, and the information that can be extracted from it. Nucleation-Polymerisation experiments were performed with an initial population of recombinant Prion protein (rp rp) from Syrian hamster (Misocricetus auratus) and produced as described previously([1]). Protein concentrations were determined by spectrophotometry (Beckman spectrophometer) using an extinction coefficient of M- 1cm-1 at 278 nm and a molecular mass of 16,227 kda. Samples containing.4 to 1.2

176 1 Introduction 175 mg/ml of the oxidized form of HaPrP9-231 (recombinant PrP C, rprp) were incubated for 1-5 days with phosphate-buffered saline (PBS), 1M GdnHCl, 2.44 M urea, 15 mm NaCl (Buffer B). The rp rp spontaneously converted into the fibrillar isoform upon continuous shaking at 25 rpm in conical plastic tubes (Eppendorf). The kinetics of amyloid formation was monitored in SpectraMax Gemini XS (Molecular Devices). Samples containing.1 to 1.2 mg/ml of the oxidized form of HaPrP9-231 (rprp) were incubated upon continuous shaking at 135 rpm in 96-well plates and in the presence of ThT (1 µm). The kinetics was monitored by measuring the fluorescence intensity using 445 nm excitation and 485 and 5 nm emission. Every set of measurements was performed in triplicates, and the results were averaged. In figure 2.1a are presented results of several nucleation experiments performed as described above. The T ht fluorescence is used as a measurable quantity, correlated (supposedly linearly) to the total mass of polymers during experiments. A population of monomer recombinant Prion protein (rprp) at a given concentration (from.1 to 1.2 mg/ml) is present initially, together with ThT fluorescent. The rp rp spontaneously converts into fibrillar isoform (polymer), upon which the ThT binds. Then the polymerization kinetic is monitored by measuring the fluorescence intensity for 1 5 days. From figure 2.1a the diversity and heterogeneity (to be explained further) of the experimental results can be immediately observed. However, experiments were performed in same experimental conditions, with the same recombinant prion protein. The aim of quantitative analysis of polymerization kinetics is to validate or invalidate kinetic hypotheses and to determine parameters values. For this, quantitative information has to be determined from experimental results. For this, the experimental curve is fitted with the general equation of a sigmoid (figure 2.1b) a+y ThT Fluorescence ThT Fluorescence v max =a/4τ Hours Hours (a) Time Experimental Series y T lag Time (b) Fitting Curve Figure 2.1: (a) Time (in hours) evolution of the ThT fluorescence (arbitraty units) in various spontaneous polymerization. The T ht fluorescence is used as a measurable quantity, correlated to the total mass of polymers. The experiments were performed in two different conditions (left and right panel), with an initial population of recombinant prion protein (PrPc). Each type of symbol corresponds to one experiment, and each symbol corresponds to a time measurement. For each experiment, the experimental set of measurements was fitted according to a sigmoid given by eq. (1.1) and shown in solid lines. (b) The solid line is a sigmoid function given by eq. (1.1). We can see the definition of the key parameters on this curve: v max is the maximal slope of the sigmoid, which is achieved at the inflexion point. The tangent at this point is represented in dotted line. We note 1 τ the maximal speed, normalized by the mass that polymerized, which is named by a on the figure. Then T lag is the waiting time for the polymerization to start. See the text for more details.

177 176 Hybrid Models to Explain Protein Aggregation Variability We first note that the mass of polymer follows an evolution shaped as a sigmoid (figure 2.1b) given by the general following sigmoid equation, F y a 1 e Ôt T i Õ τ. (1.1) This equation is phenomenological but gives a rather good estimate of some parameters used to compare the models with the experiments. Four quantities appear to be characteristic of the prion aggregation dynamic. Firstly, F max a y is the maximal fluorescent value reached asymptotically, at the end of the experiment (while y is the initial level of fluorescence). Secondly, 1 τ is the (normalized) maximal polymerization rate, which is achieved at t T i, the inflexion point. Finally, the lag time T lag is the waiting time for the true start of polymerization. In our stochastic model, the start of the polymerization is due to a discrete event (the first nucleation). However, this supposedly discrete event is not observable experimentally, and the continuous and smooth sigmoid curve we used to fit experimental results cannot give such information. Then, in agreement with the literature, the lag time is defined as the time required to measure a given fraction of the maximum value, say 1%. This time can only be measured on the sigmoid curve. This time is actually very close (1) to the formula given by Lee et al. ([95]), which linked the lag time to T i and τ by the equation (see figure 2.1b) as T lag T i 2τ. All these quantities (F max, y, a, τ, T i, T lag ) can be measured on each experimental curve as sampled in figure 2.1a. We can see on figure 2.1a that the dynamic of prion amyloid formation on each experiment is high heterogeneous, even if they were obtained under the same experimental conditions. Namely, each of the three quantities T lag, τ and F max, on which we mainly focus, are highly variable from one experiment to another. Let us first present statistics for each one, how they correlate with the initial concentration of protein, and finally how they correlated within each other. We will see that such analysis suggests a stochastic formulation of a nucleation-polymerization model, which gives rise to a heterogeneity in the dynamics of polymerization, as well as in the obtained structure of polymers. This analysis is partially described in a recent paper [3] Nucleation Time Statistics The initial concentration of protein and the lag time are usually inversely correlated in protein nucleation experiments ([54], [4], [13]). This feature is common in different fields of physics and biology (polymer, crystallization). However, in these experiments, these two quantities are very poorly correlated: we found a correlation coefficient of.8 and a p-value of.49. (figure 2.2 A). These results show that the lag time and the initial concentration are not correlated between each other. Such a phenomenon has been observed previously for prion protein nucleation experiments ([4], [13]). We look also at the variability of the lag time while repeating experiments in the same conditions. The coefficient of variability (standard deviation over the mean) is respectively.77,.72 and.55 for m.4,.8, 1.2 mgßl, over 29, 24 and 19 experiments. The distributions of lag time in experiments are shown in figure 2.2 B. As the initial concentration increase, the main peak is sharper and the tail is fatter (the Kurtosis coefficient varies from.7, 4.46 and.64). The distribution is very asymmetric for intermediate concentration (the skewness varies from.91, 2.1 and 1.3). We note however that the number of experiments is too small to deduce any distribution fitting. 1. *note: the ten percent value is actually given by T i lnô9õτ

178 1 Introduction 177 Figure 2.2: Analysis of the T lag in spontaneous polymerization in vitro experiments. A Each triangle represents the T lag (in hours) found by fitting one experimental curve with eq. (1.1), as shown in figure 2.1b. Experiments are performed with the same condition, with respectively initial concentration of.4,.8 and 1.2 mg/l of rp rp protein. The black squares represent the mean and the dashed line is obtained by a linear fit of these means as a function of the initial concentration. The slope is.13 hours 1.mg 1.L. The correlation coefficient between the lag time and the initial concentration is.8, with a p-value of.49. B Histograms of the lag time in spontaneous polymerization experiments. From left to right, the initial concentration of protein is.4,.8 and 1.2 mg/l. The histograms are constructed based on the points on the left figure, with respectively 29, 24 and 19 experiments Polymerization Speed Statistics The(normalized) maximal polymerization rate is also poorly correlated with the initial concentration of protein (see figure 2.3a). The high heterogeneity of the growth rate (the coefficient of variability are respectively.58,.23 and.55 for.4,.8 and 1.2 mg/l initial concentration) may explain this weak relationship. We also compute the distributions of polymerization rate in experiments (figure 2.3b) Maximal Fluorescence Statistics For a specific set of experiment, the maximal fluorescence get concentrated in two distinct regions, whatever the initial concentration protein is (figure 2.4). Indeed, in independent samples obtained in the same experimental conditions, the histogram of the final fluorescence value was bimodal, with peaks around 52 or 228 (arbitrary units). We showed that segregating experiments with those giving a low F max value and those giving a high F max value, increased significantly the correlation coefficient (from.42 to.7 and.6, see figure 2.4A) between F max and the initial concentration Correlation with each other The figures and analysis presented here were the subject of a publication [3]. It has been shown that the maximum value F max is not correlated with the remain quantity of monomers at the end of the experiment, neither with the lag time or the maximum growth rate (figure 2.5a - 2.5b). We also note that the lag time and the maximal growth rate are apparently uncorrelated (figure 2.5c)

179 178 Hybrid Models to Explain Protein Aggregation Variability Frequency (a) Experimental data /τ 1 2 1/τ (b) Experimental histogram 1 2 1/τ Figure 2.3: Normalized maximal polymerization rate. (a) Normalized maximal polymerization rate with initial quantity of P rp protein (in log scale). Each triangle represents the rate 1ßτ (in hours 1 ) found by fitting the experimental curve with eq. (1.1), as explained in the subsection 1.4. Experiments are performed with the same condition, with respectively.4,.8 and 1.2 mg/l of PrP protein. The black squares represent the mean of the experimental values, for each concentration. The dashed line is obtained by a linear fit of these means as a function of the initial concentration. The slope is.33 hours 1.mg 1.L. (b) Histograms of the polymerization rate in spontaneous polymerization experiments. From left to right, the initial concentration of protein is.4,.8 and 1.2 mg/l. The histograms are constructed based on respectively 29, 24 and 19 experiments Seeding experiments and conclusion Heterogeneity of the structure. Such a difference in the F max value, obtained in repeated experiments, cannot be explained by a difference in the polymerized mass, but only by a difference in the final polymer structure, as argued in [3]. The electron microscopy analysis gives a clue to interpret this heterogeneity: we can clearly see that different polymers may appear (figure 2.6a). Actually, it has been shown that different polymers with different structures have a different binding affinity with the ThTfluorescence. Direct measurements of the size of polymers have indeed confirmed that the relation between the size of polymer with its fluorescence response to ThT highly depends on the structure of the polymer (figure 2.6b). This explains why we observed in paragraph two distinct peaks for the final fluorescence value F max in polymerization experiments. Intermediate values within this two ranges of values can be explained either by an additional structure or the presence of both structures (figure 2.4B) Seeding experiments. We have seen that there is an heterogeneity in the polymer structure. Further analysis of the experimental results reveals that the different polymer structures are the result of a heterogeneous process before nucleation takes place. For this, we need to look at results of seeding experiments. It has long been suggested that the seeding experiments explain the infectiousity of the prion disease. Indeed, experiments with increased initial quantity of seed exhibit subsequent reduction of lag time (figure 2.7a). It is also interesting to note how these seeding experiments bring some information into the overall polymerization process. Firstly, it has to be noticed that this lag time does not disappear, suggesting that it exists a conformational mechanism that could not be suppressed before the polymerization can take place. Secondly, successive seeding experiments (the polymers obtained at the end of an experiment is used as seeds for the next seeding experiment) increase the polymerization growth rate (figure 2.7b). However, it has been shown that successive seedings

180 1 Introduction 179 Figure 2.4: Maximal fluorescent values in spontaneous polymerization. A Each point represent the experimentally measured final value of fluorescence (arbitrary unit), as a function of the initial concentration of proteins. All experiments are performed in the same conditions with initial concentration of proteins respectively.4,.8 and 1.2 mg.l 1. We then segregate arbitrarily the values in two categories: the highest values and the lowest values. The higher dashed line shows a linear fit of the mean among the highest value (as a function of the initial concentration), and the lower solid line shows a linear fit of the mean among the lowest value (as a function of the initial concentration). The slopes are respectively and L.mg 1. We also calculated the correlation coefficientbetweenthefinalvalueoffluorescencef max andtheinitialconcentration. Before separating the values, the correlation value is.42 (p-value ). After separating the values in two distinct sets, correlation values are.72 (p values ) for the lowest F max value set, and.6 (p value ) for the highest F max value set. B. Histogram of final value of fluorescence of the same data set as in the left figure. We then fit this histogram with the superposition of two Gaussians, centered in the two peaks, namely 52 and 228. The fitted variance are respectively 252 and 1362 (arbitrary units). do not change the structure of the polymers, which suggest that the nucleation formation is predominant in the choice of structure of prion amyloids. The structure of polymers depends on the nucleation process more than on the polymerization process Conclusion: suggested model All these observations suggest that an intrinsic conformational change process takes place before the nucleation, and is determinant for the following kinetic. As different polymers structure may appear, it is reasonable that different misfolded monomers may be present. Then a possible mechanism is that each kind of misfolded protein only aggregates with a similar misfolded protein, and lead to possibly different nucleus structures. The first nucleus formed dictates the dynamic and probably the polymers structure (figure 2.8). Because the nucleation process is longer than the polymerization, if there is already a given formed polymer, it grows and leads (by fragmentation) to multiple growing polymers of the same structure, making more and more unlikely the formation of a nucleus of a different structure.

181 18 Hybrid Models to Explain Protein Aggregation Variability (a) F max and Ti (b) F max and τ (c) T lag and 1 τ Figure 2.5: Correlation between F max, T lag and 1 τ in nucleation experiments. The figures are taken from [3]. For each experiment, the time data series are fitted according to eq. (1.1), and the values of F max, T lag and 1 τ are then deduced as explained in subsection 1.4. The values of these parameters are plotted in : (a))f max (arbitrary unit) as a function of Ti (hours) (b) F max (arbitrary unit) as a function of τ (hours) (c) T lag (hours) as a function of 1 τ (hours 1 ). See [3] for more details. (a) Electron microscopy analysis (b) Relation between Fluorescence and Polymer size Figure 2.6: Heterogeneity of the observed structure. The figures are taken from [3]. (a) Electron microscopy analysis that shows pictures of the polymers obtained at the end of nucleation experiments. (b) Each point corresponds to the measurement of the fluorescence versus the size of an individual polymer. See [3] for more details. Thus, the nucleation experiment would lead to a possible coexistence of different strains in theory while the seeding experiment has small chance to lead to such a phenomenon. A stochastic formulation of the Lansbury s nucleation-polymerization model (subsection 1.2) can easily incorporate the possibility of different structures in competition for the apparition of the first nucleus, and then seems appropriate for the mathematical formulation of

182 1 Introduction 181 (a) Increased initial quantity of seed (b) Successive seeding experiment Figure 2.7: Seeding experiments. The figures are taken from [3]. Each type of symbol corresponds to a time data series of a seeding experiment. The time data series was fitted according to eq. (1.1), and the obtained curve is reported here. (a) For down (red line) to up (green line), the initial amount of polymers used as seeds is increased. (b) From right (black line) to left (blue line), the polymers used as a seed come from an increasing number of successive seeding experiments. See [3] for more details. Figure 2.8: Model suggested by [3]. Figure taken from [3]. Each color corresponds to a particular misfolded protein or a polymer structure. This figure illustrates that a conformational change occurs before the polymerization, and during the nucleation process. This conformational change is determinant for the kinetic of the polymerization. the model shown in figure 2.8 and given by Alvarez-Martinez et al. [3]. What kind of different information a stochastic model gives compare to a deterministic model? Is it more appropriate to describe the dynamic of Prion nucleation? Is it possible to get coexistence of several strains in a same experiment? Is it possible to reproduce this withamathematical model, startingfromanhomogeneouspopulation ofprp C monomer?

183 182 Hybrid Models to Explain Protein Aggregation Variability Answering these question is the purpose of this work. These questions are fundamental for the next goal: to understand the toxicity of different strains, and to estimate useful parameters. Indeed different strains would cause different levels of toxicity for the systems, and their dynamics could be totally different from one to another. That is why the overall behavior should be deeply investigated since it may be strongly correlated to the parameters involved in the process, each set of parameters representing a specific strain. A primary necessary step to the study of a model with multiple strains structure is the study of a stochastic model with one single structure.thus, we start by studying in section 3 a stochastic formulation of a nucleation model, in order to understand the stochasticity in the nucleation time, as a function of parameters (initial quantity of monomers, aggregation kinetic rates, nucleus size). We continue by studying the polymerization-fragmentation model in section Literature review For each of the four levels of experimental observations mentioned in subsection 1.3, some theoretical mathematical modeling have been used, for which we now briefly give some references. Then, we spend more time on coagulation-fragmentation model, and finally review the specific literature on nucleation modeling that is useful for us. For the smallest scale, the atomic description of protein configuration, people mostly use molecular dynamic simulations (coarse-grained model, random-coil peptides) for which we can refer to [16, 117, 112, 66]. These techniques allow to combine precise chemical and physical properties of the protein conformation and spatial mechanistic rule of the attachment/detachment of proteins within each other. Hence, in such models, both physical properties and mechanic rule influence the aggregation dynamic. For the cellular level, models usually take into account the spatial dynamic inside cells, and the cell characteristics (protein synthesis rate, cellular density, cell cycle, cell death...) together with prion strains characteristics (aggregation dynamic, diffusivity,...). See for instance [116, 131]. If these models usually lead to interesting modeling and mathematical questions, the lack of experimental data, however, is quite problematic (this may change quickly). For the population level, epidemiologist model can be used to represent the propagation of the disease in an animal population, taking into account possible rules of transmission between animals, within their environment. For an example on a deer population, see [2] General Coagulation-Fragmentation model We now review coagulation-fragmentation models, that are mostly adapted to the protein level experimental data. In general, in a coagulation-fragmentation model, each particle is characterized by its size (or mass). It can hence be seen as a structured population model, where the structure variable is the size (or the mass) of the particle. Population model are usually defined in terms of birth and death of particles. In coagulationfragmentation model, two particles die simultaneously when they coagulate (attach) with each other, and a new particle is born also simultaneously. If the two old particles are of size respectively x and y, such event appears with rate given by a coagulation kernel KÔx,yÕ, and the new particle is of size x y. The fragmentation process is the reverse process. A particle of size x die and gives birth to two new particles of size y and x y, at a rate F Ôx,yÕ. The mathematical formulation of these mechanistic rule can be deterministic, as a systems of ordinary differential equations or partial differential equations,

184 1 Introduction 183 or stochastic, as a finite particle model (given by point process) or a superprocess. For every formalism, the typical questions that arise in a mathematical study are the conditions for well-posedness of the model (depending on condition on kernel K, F and initial condition), its long-time behaviour, and particular phenomenon of gelling and dusting solution: while reasonable conditions on the initial condition and on the kernel K,F can be given to ensure that the solution is mass-conservative for all time (the sum of mass of all particles of the system stays constant over time), some degeneracy cases have been shown to lead to solutions for which the mass is not conserved during a finite time interval. The gelling phenomena corresponds to the (physical) situation where a single giant particle is created, and a phase transition lead to a gel. The dust phenomena corresponds to the situation where an infinity of particle of mass are created. Apart from deterministic and stochastic models, the size of particles may be of different nature between models. Namely in systems of ordinary differential equations, the size is treated as a discrete variable, and there is one equation for each size of particle. While in partial differential equation model, the size is treated as a continuous variable (the model is usually refer to the Smoluchowski model). The same dichotomy holds as well for stochastic model. For a review of results on deterministic discrete coagulation-fragmentation model, we refer to Wattis [139]. General results on existence, uniqueness and mass conservation has been first derived by Ball and Carr [8], while Hendriks et al. [7] considered the case of purely coagulation and gave condition for gelation. Since then, results have been improved by Laurençot and Mischler [92], while Cañizo [24], Fournier and Mischler [59] gave conditions for exponential trend to equilibrium. The study of stochastic pure-coagulation model was first developed by Hendriks et al. [71], Lushnikov [98], Marcus [12]. Such models are usually refer to the stochastic coalescent model or the Marcus-Lushnikov model. For an interesting survey of results on pure-coagulation model, see the very popular work of [1], which contains a wide variety of applications, reviews available exact solutions, gelation phenomena, various examples and types of coagulation kernel, and mean-field limit. This author raises a certain number of interesting open problems related to these model. In [113], the author derived the fluid limit of the stochastic coalescent model, namely the Smoluchowski s coagulation equation. The author used such approach to derive a general result of existence of the mean-field Smoluchowski model (KÔx,yÕ ϕôxõϕôyõ, with sub-linear function ϕ, and ϕôxõ 1 ϕôyõ 1 KÔx,yÕ as Ôx,yÕ ). The author also provided a review and new result of uniqueness of the mean-field Smoluchowski model for similar aggregation kernel, with an extra assumption on the initial distribution of particle mass. Importantly, he also gave an example of an aggregation kernel for which uniqueness does not hold, by exhibiting two conservative solution of the same equation. Finally, in the special case of discrete mass particle, the author provided a bound of the convergence rate of the stochastic coalescent to the mean-field Smoluchowski model. See also [56] for other results on well-posedness of Smoluchowski s coagulation model, with homogeneous kernel and [3] for a convergence rate of the Marcus-Lushnikov model towards the Smoluchowski s coagulation model, in Wasserstein distance (in 1 n ). For pure-fragmentation model we refer to Wagner [136, 137]. The author considers a general pure fragmentation model (with example including binary fragmentation, homogeneous fragmentation). In particular, the author reviews conditions on the fragmentation kernel so that the discrete stochastic model (and its deterministic counterpart) almost surely undergoes an explosion in finite time. As in the pure aggregation model, these conditions involved a lower bound condition, such as the fragmentation kernel explodes sufficiently rapidly in. See also [1] for a review on analytical techniques to characterize such phenomenon.

185 184 Hybrid Models to Explain Protein Aggregation Variability Finally, for the general coagulation-fragmentation model, the first rigorous results seems to have been obtained by Jeon [78]. This author used the stochastic formulation model to study the gelling phenomena of the mean-field Smoluchowski s coagulationfragmentation equation. In particular, he derived conditions on coagulation kernel KÔx, yõ and fragmentation kernel F Ôx,yÕ to show the tightness of the stochastic coagulationfragmentation model, and hence existence of solution of Smoluchowski s coagulationfragmentation equation. His condition on the kernel involved lim KÔx,yÕßxy and there exists G such that F Ôx,yÕ GÔx yõ with lim x x y GÔxÕ. Results on gelation phenomena involve a lower bound condition such as the existence of M,ε, and εij KÔi,jÕ Mij. Fluid limit results in the case where gelation occurs were recently obtained in [55, 57] where the authors show that different limiting models are possible, namely the Smoluchowski model and a modified version, named Flory s model Becker-Döring Model A special case of the coagulation-fragmentation model is the Becker-Döring Model, which was originally used by [14]. In such model, aggregation and fragmentation occur only one monomer by one monomer, that is, in a discrete-size description, KÔx,yÕ x 1, or y 1 and similarly for the fragmentation kernel. The theoretical foundations of such models have been laid down by Ball et al. [9], followed by other contributions [7, 28, 127] for the well posedness of the model and its asymptotic behaviour. Convergence rates towards equilibrium have been obtained by Jabin and Niethammer [75] Prion model According to the Lansbury s theory, during the nucleation phase, addition of monomer occurs one-by-one but are unfavorable, so that detachment of monomer are also important. Then the Becker-Döring Model seems the most adapted to the nucleation phase. For the polymerization phase, when nuclei are already there, the coagulation still occurs one-byone, but detachment is negligible. However fragmentation of large polymer does occur. Thus, we use a coagulation-fragmentation model, where coagulation occurs only with single monomer, and fragmentation occurs with a general kernel Finite maximal size and Stochastic nucleation models All the models quoted above do not use any maximal size for the particles, and mostly study the long-time behavior of the system. However, to capture the nucleation phase, it seems more natural to study a model where there is a maximal size, and to study the waiting time for the solution to reach this maximal size. Such approach has been taken in [12] using a maximal size deterministic Becker-Döring Model. In particular, the authors derive general scaling laws for the nucleation, as a function of initial condition and kinetic parameters. Our approach in section 3 can be seen as a generalization of their study to the stochastic version of the Becker-Döring Model. Previous stochastic models have been used to study the nucleation time, within protein aggregation fields ([132], [53], [73], [87]). In [53], they use a simple autocatalyic conversion kinetic model to get the distribution of incubation time. Under the assumption that the involved constant rate is a stochastic variable, log normally distributed, the incubation time is then also shown to be log normal. In [73],[132], the authors get the distribution

186 2 Formulation of the Model 185 shape of lag time using assumptions on probabilities of nucleus formation event. Hofrichter [73] end up with a delay exponential distribution, while Szabo [132] found a β-distribution, useful to experimentally deduce the rate of single nucleation formation. In [87] the authors used a phenomenological model to get the mean waiting time to reach a certain amount a polymer, from one initial seed and under assumptions on distribution of aggregation and fissioning times. This expression allows them to discuss the influence of initial dose or other parameters on the incubation time. Using a purely stochastic model for sequential aggregation of monomers and dimers, they obtain different waiting time distributions, as a γ-distribution, a β-distribution or a convolution of both. Our approach is rather different, also close to that last one exposed in [87]. Indeed, for the nucleation phase, we use a purely stochastic Becker-Döring kinetic model, under the assumption that the first polymer is formed by successive additions and disassociations of one misfolded monomer. This discrete stochastic model allows us to define the nucleation time as the waiting time to reach the first nucleus (a polymer of a given size). After the first nucleus is formed, our stochastic kinetic model includes aggregation through monomer additions and fragmentations of polymers (similar to previous prion model). 1.6 Outline We present in detail the formulation of our model in the next subsection 2. There we give the biochemical reaction steps underlying this model, and its deterministic and stochastic version (both with discrete size). Then, we focus on the misfolding process, and obtain two limiting models by performing a time-scale reduction. These limiting models are easier to handle, in particular to study the nucleation time. In section 3, we study the nucleation time in a stochastic version of the Becker-Döring Model. We attach importance in finding analytical solutions, either exact or approximate, in order to get general scalings laws as well as quantitative informations on the behavior of the system, with respect to parameters. We show that the stochastic formulation leads to several unexpected features for the nucleation time. Finally, we apply this study to the prion modeling and compare our theoretical results to the experimental data. In section 4, we focus on the polymerization-fragmentation phase of the model. We consider a slight generalization of the model, including spatial movement, and study the limit when the number of monomer is very large compared to the number of polymer. Using stochastic limit theorem, we show that our purely discrete model converge to a hybrid model, where polymerization is deterministic and fragmentation is a jump process. 2 Formulation of the Model 2.1 Dynamical models of nucleation-polymerization We use a simplified version of the model introduced by Lansbury et al. in 1995 ([29]). The dynamic is composed of a set of chemical reactions involving only the prion protein. Firstly, it is based on the assumption that the protein is able to spontaneously misfold and unfold again(figure 2.9a). The misfolded form is supposedly very unstable, and this process of folding/unfolding very fast. The misfolded protein is the only form able to actively contribute to the aggregation process, by addition of one monomer at each step [4]. Secondly, the early steps of the aggregation process (figure 2.9b) are thermodynamically unfavorable, meaning that the forward polymerization reaction rate is several orders of magnitude lower than the backward depolymerization reaction rate. These reaction rates, p, q, are supposed to be independent of the size of the aggregates. We called the species formed during this process the oligomers. There are small aggregates of size less than a

187 186 Hybrid Models to Explain Protein Aggregation Variability (a) Spontaneous Misfoling (b) Nucleation steps (n 5) (c) rapid polymerization (d) polymerization/fragmentation Figure 2.9: In this figure we present the successive reactions steps of the nucleationpolymerization model. (a) Fast equilibrium between normal and transconformed monomer. (b)nucleation reaction steps. Heren 5. All thestepsarecomposedofunfavorableaddition of a single monomer. (c) Polymerization reaction steps. All the steps are composed of irreversible addition of a single monomer. (d) Fragmentation process. The fragmentation rate is proportional to the mass of the polymer. The two parts have equal probability to be of a size between one and the size of the initial polymer minus one. When it gives birth to an oligomer (size less than n) this last one is supposed to break into small monomers immediately due to the instability of the oligomer). given number, n. At this size, the kinetic steps change, and the aggregation of monomer is irreversible. The particular oligomer size n at which the kinetic steps change is called the nucleus. We emphasize that we use a constant-size nucleus model, which does not necessarily correspond to the most unstable species, as it has been well explained [12]. Finally, the rest of the dynamic(figure 2.9c - 2.9d) is followed by a classical polymerizationfragmentation model [11], resulting in rapid polymerization/growth. The fragmentation process is responsible of the auto-catalytic form of the prion polymerization. We focus on the lag time, so on the early steps of the nucleation-polymerization process. Because we are interested in the time scale of the monomer disappearance (and not of the polymer relaxation), the irreversibility hypothesis on the polymer growth is fairly acceptable [62] (the depolymerization reactions are negligible after the first nucleus is formed because the polymerization reactions are fast). Table 2.1 summarizes the different parameters involved in this model. According to this theory, because of its high stochasticity, nucleus formation would be considered as a kinetic barrier to sporadic prion diseases. But this barrier could be overcome by infection. The disease would not be spontaneous anymore, it could be transmitted on purpose or not by a PrP Sc polymer seeding which would directly lead to the second step since no formation of the first nucleus would be required. Once again, our main focus here is the sporadic appearance of the first nucleus, rather than its transmission.

188 2 Formulation of the Model 187 Table 2.1: Definitions of variables and parameters. We use small letters for the continuous variables involved in the deterministic model, and capital letters for the discrete variables involved in the stochastic model. We keep the same notation for the parameters in both models, in order to avoid many different notations, although the parameters for secondorder reaction has different units. Name mßm f 1 ßF 1 f i ßF i N γ γ c γ γ p q σ q p k p k b Definition Concentration/Number of Native Monomer Concentration/Number of Misfolded Monomer i 2..N 1, Concentration/Number of aggregates of size i Nucleus size Folding rate Unfolding rate Equilibrium constant between monomers Elongation rate in nucleation steps Dissociation rate in nucleation steps Dissociation equilibrium constant in nucleation steps Elongation rate in polymerization steps Fragmentation rate in polymerization steps We look at the following set of chemical reactions defined by (variable and parameter are defined in table 2.1): m γ f 1 (spontaneous conformation) (2.1) γ f 1 f 1 p q f 2 (dimerization) (2.2). f k 1 f 1 p q f k ((k)-mer formation) (2.3). f N 1 f 1 p un (nucleus formation) (2.4) k p u i f 1 u i 1 Ôi NÕ (elongation) (2.5) k u b i ui j u j Ôi N,1 j i 1Õ (polymer break) (2.6) and u k kf 1 Ô if k N 1Õ (oligomer instability) (2.7) The system of chemical reactions (2.1) - (2.7) defines our full model and consists of four steps: misfolding, nucleation, polymerization, and fragmentation. All reaction rates are assumed to follow the law of Mass-Action, with kinetic constant indicated on each reaction. The reversible reaction (2.1) represents the misfolding process between normal and misfolded protein, occurring at rate γ and γ. The reaction (2.2) - (2.3) represent the aggregation process during the nucleation phase, and consist of reversible attachment/detachment of misfolded monomer to aggregate of size k, k 1...N 2, at rate respectively p and q. Such rates are assumed to be independent of the size of the aggregate. The reaction (2.4) is irreversible and represents the formation of a nucleus, by attachment of one misfolded monomer to an aggregate of size N 1, at rate p. The irreversibility hypothesis comes from the assumption that all aggregates of size greater than the nucleus size N are stable. These aggregates are called polymers. Then, reaction (2.5)

189 188 Hybrid Models to Explain Protein Aggregation Variability consists of irreversible polymerization, by addition of one by one misfolded monomer, at rate k p, also assumed to be independent of the size of the polymer. Reaction (2.6) is the fragmentation process, occurring at rate linearly proportional to the size of the polymer. For a linear polymer of size i, there is i 1 connection between monomer, and we take the fragmentation rate to be k b Ôi 1Õ. The size repartition kernel of the new-formed polymer is taken uniform along all possible pairs of polymers. Thus, the total fragmentation kernel F Ôi,jÕ, which gives the probability per unit of time that a polymer of size i breaks into two polymers of size j and i j, is 2 F Ôi,jÕ k b Ôi 1Õ i 1 1 ØjiÙ 2k b1 ØjiÙ. (2.8) The factor 2 comes from the symmetry condition between the pairs Ôj,i jõ and Ôi j,jõ. Finally, if due to a fragmentation event, a polymer of size less than N appears, we suppose that it breaks instantaneously in monomers, which is represented by reaction (2.7). This model can be seen as a coagulation-fragmentation model, where the coagulation kernel KÔx,yÕ is constant equal to p for x 1, y 1..N (and vice-versa), and constant equal to k p for x 1 and y N 1 (and vice-versa), and zero otherwise. The fragmentation kernel is F Ôi,jÕ 2k b 1 ØjiÙ Deterministic model of prion polymerization The above chemical reactions system can be quantitatively studied by law of actionmass and transformed into a set of ordinary differential equations. Although it involves an infinite number of species (one for each size), it is known that this system can be reduced to a finite set of differential equations, as we recall below. It has one equation for each species of size lower than the nucleus size, in addition to two equations for the number of polymers and their mass. Firstly, a system of an infinite number of differential equations is built based on the reactions ( ) with the action-mass law. We get, with the same notations as above: ³² ³± dm dt df 1 dt df 2 dt df i dt df N 1 dt du N dt du i dt γm γ f 1, N 1 ô «γm γ f 1 pf 1 f 1 f k q 2f 2 k2 ô «ô k p f 1 u k N ÔN 1Õk b u k, kn pf 1 Ô f 1 2 f 2Õ qôf 2 f 3 Õ, kn N 1 ô pf 1 Ôf i 1 f i Õ qôf i f i 1 Õ, 3 i N 2, pf 1 Ôf N 2 f N 1 Õ qf N 1, ô pf 1 f N 1 k p f 1 u N k b ÔN 1Õu N 2k b k p f 1 Ôu i 1 u i Õ k b Ôi 1Õu i 2k b ô ki 1 k3 kn 1 f k «u k, u k, i N 1. Secondly, with the variables y in u i, z in iu i, it is standard ([14]) to transform this set of infinite number of differential equations into the following finite set of differential

190 2 Formulation of the Model 189 equations ³² ³± dm dt df 1 dt df 2 dt df i dt df N 1 dt dy dt dz dt γm γ f 1, γm γ f 1 pf 1 f 1 k p f 1 y N ÔN 1Õk b y, N ô 1 k2 pf 1 Ô f 1 2 f 2Õ qôf 2 f 3 Õ, f k «q 2f 2 N ô 1 pf 1 Ôf i 1 f i Õ qôf i f i 1 Õ, 3 i N 2, pf 1 Ôf N 2 f N 1 Õ qf N 1, pf 1 f N 1 k b z Ô2N 1Õk b y, Npf 1 f N 1 k p f 1 y N ÔN 1Õk b z. k3 f k «(2.9) In this model, the lag time Tlag det is defined as the waiting time for the mass of polymer to reach ten percent of the total initial mass (cf figure 2.1a). In our simulation, a sigmoid shape is observed for the time evolution of the mass of polymers, which is qualitatively in good agreement with the experiment and previous studies. Note that this model is a slight modification of the deterministic model studied by Masel ( [14] ) adapted to the in vitro experiments. In this deterministic framework, ordinary differential equations are used to model the evolution of concentrations of the species. Based on biological observations, we introduce a concentration of abnormal monomer (f 1 ) corresponding to a small proportion of the concentration of normal monomer (m). This low concentration of misfolded protein actively contributes to the aggregation process while the high concentration of normal protein still remains inactive Stochastic model of prion polymerization Let us now give an insight of the stochastic model. To that purpose, we take the same reactions steps as previously explained, but use now a continuous time Markov chain to describe its time evolution. This stochastic model can be treated using the theory of Markov processes. From the reaction (2.1) -(2.7), we can write down a system of stochastic differential equation driven by Poisson processes. However, its complete expression is complicated due to the fragmentation term for small aggregate. We only write down the system for reaction (2.1) - (2.4), that is before nucleation takes places. In that case, the system is described by

191 19 Hybrid Models to Explain Protein Aggregation Variability ³² ³± t t M ÔtÕ M ÔÕ Y 1 γm ÔsÕds Y 2 γ F 1 ÔsÕds, t t F 1 ÔtÕ F 1 ÔÕ Y 1 γm ÔsÕds Y 2 γ F 1 ÔsÕds N 1 t p ô t 2Y 3 2 F 1ÔsÕÔF 1 ÔsÕ 1Õds Y 2i 1 i2 t Nô t 2Y 4 qf 2 ÔsÕds Y 2i qf i ÔsÕds, i3 pf 1 ÔsÕF i ÔsÕds t p t F 2 ÔtÕ N 2 ÔÕ Y 3 2 F 1ÔsÕÔF 1 ÔsÕ 1Õds Y 5 pf 1 ÔsÕF 2 ÔsÕds t t Y 4 qf 2 ÔsÕds Y 6 qf 3 ÔsÕds, t t F i ÔtÕ F i ÔÕ Y 2i 1 pf 1 ÔsÕF i 1 ds Y 2i 1 pf 1 ÔsÕF i ÔsÕds t t Y 2i qf i ÔsÕds Y 2i 2 qf i 1 ÔsÕds, 3 i N 2, t t F N 1 ÔtÕ F i ÔÕ Y 2N 4 pf 1 ÔsÕF N 2 ds Y 2N 1 pf 1 ÔsÕF N 1 ÔsÕds t Y 2N 2 qf N 1 ÔsÕds, t U N ÔtÕ U N ÔÕ Y 2N 1 pf 1 ÔsÕF N 1 ds. (2.1) where Y i, 1 i 2N 1, are independent standard Poisson process. This system may be simulated through a standard stochastic simulation algorithm or Gillespie algorithm ([6]). The details of the stochastic model allow us to exactly identify the first discrete nucleation event (figure 2.1b ). Then, in the stochastic model, the lag time is defined as the waiting time to obtain one nucleus, that is one aggregate of the critical size at which the dynamic entirely changes, due to the irreversibility of the nucleus and larger polymers. In our simulation, we can observe how the dynamic drastically changes after the first nucleation event (figure 2.1b). This is solely due to the hypothesis of parameters change at that point, and in particular to the irreversible aggregation hypothesis. We notice also that the time evolution of the mass of polymers follows roughly a sigmoid, due to the polymer breaks. 2.2 Misfolding process and time scale reduction The introduction of the misfolding protein makes the analysis of the nucleation time more delicate. Thus, we use a time scale reduction, based on two different biological hypothesis, to eliminate one of the two variables between the normal and the misfolded protein. Firstly, if the misfolding process occurs at a very fast time scale, compared to the other time scale of the system, both normal and misfolded protein equilibrate within each other. At the slow time scale, the system only sees the averaged quantity of each protein. In particular, in the deterministic model, the rate of aggregation depends of a fraction of the total quantity of monomers. In the stochastic model, the fast subsystem made up of normal and misfolded monomers converges to a binomial distribution, and the slow system only depends on the first two moments of this binomial distribution. We note that the

192 2 Formulation of the Model 191 (a) Deterministic Simulation (b) Stochastic Simulation Figure 2.1: (a)deterministic Simulation and definition of the lag time in the deterministic model. One simulation of the deterministic model, with the concentration of normal and folded protein, concentration of oligomers and polymers. The lag time is defined as the waiting time to convert a given fraction of the initial monomers into polymers, here 1%. We used here môõ 1, γ ßγ 1, σ 1, n 7. The time (in log scale) has been rescaled by τ pt. (b) Stochastic Simulation and definition of the lag time in the stochastic model. One simulation of the stochastic model, with the numbers of normal and misfolded protein, the mass of oligomers and the mass of polymers. The lag time is defined as the waiting time for the formation of the first nucleus. We used MÔÕ 1, γ ßγ 1, σ 1, n 7. The time (in log scale) has been rescaled by τ pt. reduced model can be seen as an original Becker-Döring model where the total mass is conserved. Secondly, another biological hypothesis is to assume that the misfolded protein is very unstable and hence present in very small quantity compared to the normal protein. Specifically, if we assume that the total quantity of protein is very large, and that the misfolded protein is highly unstable, we obtain a further reduced model where the quantity of misfolded protein is constant over time, and aggregation takes place with constant monomer quantity. Such reduced model can be seen as a Becker-Döring model where the quantity of monomer is conserved (but not the total mass). For both scaling, we present the derivation of the limiting model in the deterministic and stochastic formulation Deterministic equation Fast misfolding process From the initial system of differential equation(2.9), we first consider the following scaling γ γn γ γ n where n and all other parameters remain unchanged. We define the free monomer variable m free ÔtÕ môtõ f 1 ÔtÕ. Then môtõ and f 1 ÔtÕ are fast variable, but m free ÔtÕ (and all other variables f i, i 2, p and u) are slow variables. To see that, consider the fast

193 192 Hybrid Models to Explain Protein Aggregation Variability time scale τ tn, so that the previous system writes dm γm γ f 1, dτ df 1 γm γ 1 f 1 pf 1 f 1 dτ n ³² ³± dm free dτ df 2 dt df i dτ df N 1 dτ dy dτ dz dτ kn N 1 ô k2 f k «k p f 1 y NÔN 1Õk b y, ««1 N 1 ô N 1 ô pf 1 f 1 f k q 2f 2 f k n k2 k3 «ô ô k p f 1 u k NÔN 1Õk b u k, kn q 2f 2 pf 1 Ô f 1 2 f 2Õ qôf 2 f 3 Õ, 1 pf 1 Ôf i 1 f i Õ qôf i f i 1 Õ, 3 i N 2, n 1 pf 1 Ôf N 2 f N 1 Õ qf N 1, n 1 pf 1 f N 1 k b z Ô2N 1Õk b y, n 1 Npf 1 f N 1 k p f 1 y NÔN 1Õk b z. n Due to the total mass conservation, all concentrations remain bounded as n, and the fast subsystem becomes N 1 ô k3 f k «dm dτ γm γ f 1, (2.11) df 1 dτ γm γ f 1. (2.12) This system has a unique asymptotic equilibrium, that depends solely on m free ÔÕ môõ f 1 ÔÕ ans is given by ³² ³± môτ Õ γ γ γ m freeôõ, f 1 Ôτ Õ γ γ γ m freeôõ. Going back to the original time scale, the slow system becomes now «dm free γp N 1 γ ô dt γ γ m free γ γ m free f k q 2f 2 df 2 dt df i dt df N 1 dt dy dt dz dt k2 N 1 ô k3 γk p γ γ m freey NÔN 1Õk b y, γp γ γ γ m freeô 2Ôγ γ Õ m free f 2 Õ qôf 2 f 3 Õ, γp γ γ m freeôf i 1 f i Õ qôf i f i 1 Õ, 3 i N 2, γp γ γ m freeôf N 2 f N 1 Õ qf N 1, γp γ γ m freef N 1 k b z Ô2N 1Õk b y, N γp γk p γ γ m freef N 1 γ γ m freey NÔN 1Õk b z. f k «

194 2 Formulation of the Model 193 Remark 68. In the slow scale system, the variables f 1 and m are instantaneously equilibrated with each other and with m free following relation eq. (2.11) - (2.12). Re-writing the system in terms of the variable f 1, we obtain an original Becker-Döring system where the γ monomer variable evolves at a slower time scale (given by γ γ t) than all other species. Finally, with the time change τ γp γ γ t, and with the following notations σ q p, the system becomes ³² ³± dm free dτ df 2 dτ df i dτ df N 1 dτ dy dτ dz dτ c γ γ, σ σô1 c Õ, K b k b p Ô1 c Õ, K k p p, 1 m free m free 1 c N ô 1 k2 f k «σ 2f 2 Km free y NÔN 1ÕK b y, 1 m free Ô 2Ô1 c Õ m free f 2 Õ σ Ôf 2 f 3 Õ, N ô 1 k3 m free Ôf i 1 f i Õ σ Ôf i f i 1 Õ, 3 i N 2, m free Ôf N 2 f N 1 Õ σ f N 1, m free f N 1 K b z Ô2N 1ÕK b y, Nm free f N 1 Km free y NÔN 1ÕK b z. f k «(2.13) (2.14) This system can be seen as a Becker-Döring system where the dimerization occurs at as slower rate than all other aggregation rates. This comes from the fact that this reaction is a second-order reaction, and hence depends on the square of the available quantity of active monomers, while other reaction solely depends linearly on the quantity of active monomers Very large normal monomer and rare transconformed monomer We continue from the system of eq. (2.14), and assume a further scaling, namely that m free is a large quantity and the rate of de-transconformation γ is also very large. We specifically suppose m free ÔÕ m free ÔÕn, γ γ n. and n. The system of eq. (2.14) is best described in the time scale τ pt and with the variable f1 n γ γ nγ m free,

195 194 Hybrid Models to Explain Protein Aggregation Variability so that we get df n 1 dτ γ f n γ nγ 1 f1 n N ô 1 «f k k2 σ 2f 2 N ô 1 k3 f k «Kf n 1 y NÔN 1ÕK by, ³² ³± df 2 dτ df i dτ df N 1 dτ dy dτ dz dτ f1 n Ô fn 1 2 f 2Õ σôf 2 f 3 Õ, f1 n Ôf i 1 f i Õ σôf i f i 1 Õ, 3 i N 2, f1 n Ôf N 2 f N 1 Õ σf N 1, f1f n N 1 K b z Ô2N 1ÕK b y, Nf1f n N 1 Kf1y n NÔN 1ÕK b z. Then, as n, dfn 1 dτ and so fn 1 ÔtÕ limfn 1 ÔÕ is constant over time. So the system behaves as the quantity of active monomers is constant over time. The resulting equations are f 1 ÔtÕ f 1 ÔÕ, df 2 f 1 Ô dτ ³² f 1 2 f 2Õ σôf 2 f 3 Õ, df i f 1 Ôf i 1 f i Õ σôf i f i 1 Õ, 3 i N 2 dτ df N 1 (2.15) f 1 Ôf N 2 f N 1 Õ σf N 1, dτ dy f 1 f N 1 K b z Ô2N 1ÕK b y, dτ ³± dz nf 1 f N 1 Kf 1 y NÔN 1ÕK b z. dτ Note that these equations do not have any more the mass conservation property. We expectthemtofaithfullyreproducetheearlystepofthenucleationprocesswhenσ f 1 ÔÕ, because in such case the mass created during nucleation is negligible. The latter condition is easily verified when there are a small amount of transconformed protein. The nucleation part of the system of eq. (2.15) is a linear system with a source term. namely df dt Af B, with f 1 σ σ f 1 f 1 σ σ A Æ f 1 f 1 σ and B f Æ where f Ôf i Õ i2,,n 1.

196 2 Formulation of the Model Stochastic equation The same two scalings can be applied similarly to the stochastic formulation. As the system of equation becomes quite unfriendly, we only sketch the main differences Fast misfolding process From the system of eq. (2.1), we now consider the following scaling γ γn γ γ n where n and all other parameters remain unchanged. We define the free monomer variable M free ÔtÕ MÔtÕ F 1 ÔtÕ. Then MÔtÕ and F 1 ÔtÕ are fast variable, but M free ÔtÕ (and all other variables F i, i 2, U N ) are slow variables. To see that, consider the fast time scale M n ÔtÕ MÔtn 1 Õ, Fi n F i Ôtn 1 Õ. Due to the total mass conservation, all quantities remains bounded as n, and, neglecting terms in OÔ 1 nõ, the fast subsystem becomes t M n ÔtÕ M n ÔÕ Y 1 γm n ÔsÕds F n 1 ÔtÕ F n 1 ÔÕ Y 1 t γm n sõds t Y 2 t Y 2 γ F n 1 ÔsÕds, γ F n 1 ÔsÕds This system has a unique asymptotic equilibrium distribution, that depends solely on Mfree n ÔÕ Mn ÔÕ ÔÕ ans is given by a Binomial distribution F n 1 M n BÔMfree n ÔÕ, γ γ γ Õ, F1 n Mn free ÔÕ M BÔMn free ÔÕ, γ γ γ Õ.. Thus F n 1 is a fast switching variable and the asymptotic first two moments of interest are F1 n Mn free ÔÕ γ γ γ, F n 1 ÔF n 1 1Õ M n free ÔÕÔMn free ÔÕ 1Õ γ γ γ 2. Going back to the original time scale, with the time change τ following notations σ q p, γp γ γ t, and with the c γ γ, σ σô1 c Õ, K b k b p Ô1 c Õ, K k p p,

197 196 Hybrid Models to Explain Protein Aggregation Variability the slow system becomes now (see Theorem 5.1 Kang and Kurtz 211) ³² ³± τ 1 M free ÔτÕ M free ÔÕ 2Y 3 2Ô1 c Õ M freeôsõôm free ÔsÕ 1Õds N ô 1 τ Y 2i 1 M free ÔsÕF i ÔsÕds i2 τ Nô τ 2Y 4 σ F 2 ÔsÕds Y 2i σ F i ÔsÕds, i3 τ 1 F 2 ÔτÕ N 2 ÔÕ Y 3 2Ô1 c Õ M freeôsõôm free ÔsÕ 1Õds τ τ τ Y 5 M free ÔsÕF 2 ÔsÕds Y 4 σ F 2 ÔsÕds Y 6 σ F 3 ÔsÕds, τ τ F i ÔτÕ F i ÔÕ Y 2i 1 M free ÔsÕF i 1 ds Y 2i 1 M free ÔsÕF i ÔsÕds τ τ Y 2i σ F i ÔsÕds Y 2i 2 σ F i 1 ÔsÕds 3 i N 2, τ τ F N 1 ÔτÕ F i ÔÕ Y 2N 4 M free ÔsÕF N 2 ds Y 2N 1 M free ÔsÕF N 1 ÔsÕds, τ Y 2N 2 σ F N 1 ÔsÕds, τ U N ÔτÕ U N ÔÕ Y 2N 1 M freeôsõf N 1 ds, which is, as in the deterministic case, a Becker-Döring model where the dimerization occurs at a slower time scale than other reaction Very large normal monomer and rare transconformed monomer As in the deterministic case, we now make the additional assumption that M free is a large quantity and the rate of de-transconformation γ is also very large, i.e. M free ÔÕ M free ÔÕn, γ γ n. Then, as n, The resulting equations are ³² ³± F 1 ÔτÕ F 1 ÔÕ, F 2 τ F 2 ÔτÕ N 2 ÔÕ Y τ Y 5 F 1 F 2 ÔsÕds τ τ Y 4 σf 2 ÔsÕds Y 6 σf 3 ÔsÕds, τ τ F i ÔτÕ F i ÔÕ Y 2i 1 F 1 F i 1 ÔsÕds Y 2i 1 F 1 F i ÔsÕds τ τ Y 2i σf i ÔsÕds Y 2i 2 σf i 1 ÔsÕds 3 i N 2, τ τ F N 1 ÔτÕ F i ÔÕ Y 2N 4 F 1 F N 2 ÔsÕds Y 2N 1 F 1 F N 1 ÔsÕds, τ Y 2N 2 σf N 1 ÔsÕds, τ U N ÔτÕ U N ÔÕ Y 2N 1 F 1F N 1 ÔsÕds. The system of eq. (2.16) is a first-order reaction network, namely (2.16)

198 3 First Assembly Time in a Discrete Becker-Döring model 197 À F k 1 F σ F 2 (dimerization) (2.17). F 1 F k ((k)-mer formation) (2.18) σ. F F 1 N 1 U N (nucleus formation) (2.19) where À denotes the fact that monomers are not consumed. The time-dependent solution of such a system has been solved by Kingman [85], and is known as a linear Jackson queueing network. We show in the next section 3 that this allows us to deduce the analytical solution of the first assembly time for this model. 3 First Assembly Time in a Discrete Becker-Döring model This work has been done in collaboration with Maria R. D Orsogna and Tom Chou, and have been the subject of a preprint. During this section we deal with the Becker-Döring model (with a fixed maximal size). We deeply study the first assembly time problem, which is defined as a waiting time problem. We use classical tools for such study (scaling laws, dimension reduction methods, time-scale reduction, linear approximation). With the help of analytic approximations and extensive numerical simulations, we end up with a general picture for the different behavior of the first assembly time, as a function of the model parameters. Particularly, we are able to characterize parameter space regions where the first assembly time has distinct properties. Our main findings implies the non-monotonicity of the mean first assembly time as a function of the aggregation rate, and give rise to three different behavior (the following will be made clearer in the next subsections): for small quantity of initial particles, the first assembly time follows an exponential distribution, and the mean first assembly time is strongly correlated to the initial quantity of particles; for intermediate quantity of initial particles (and large enough nucleus size), the first assembly time has a bimodal distribution, and the mean first assembly time is almost independent of the initial quantity of particles; for large quantity of initial particles, the first assembly time has a Weibull distribution, and the mean first assembly time is weakly correlated to the initial quantity of particles 3.1 Introduction The self-assembly of macromolecules and particles is a fundamental process in physical and chemical systems. Although particle nucleation and assembly have been studied for many decades, interest in this field has recently been intensified due to engineering, biotechnological and imaging advances at the nanoscale level [141, 142, 65]. Aggregating atoms and molecules can lead to the design of new materials useful for surface coatings [35], electronics [145], drug delivery [52] and catalysis [81]. Examples include the self-assembly of DNA structures [34, 17] into polyedric nanocapsules useful for transporting drugs [17] or the self-assembly of semiconducting quantum dots to be used as quantum computing bits [86].

199 198 Hybrid Models to Explain Protein Aggregation Variability Other important realizations of molecular self-assembly may be found in physiology or virology. One example is the rare self-assembly of fibrous protein aggregates such as β amyloid that has long been suspected to play a role in neurodegenerative conditions such as Alzheimer s, Parkinson s, and Huntington s disease [129]. Here, individual PrP C proteinsmisfoldintoprp Sc prionswhichsubsequentlyself-assembleintofibrils. Theaggregation of misfolded proteins in neurodegenerative diseases is a rare event, usually involving a very low concentration of prions. Fibril nucleation also appears to occur slowly; however once a critical size of about 1-2 proteins is reached, the fibril growth process accelerates dramatically. Figure 2.11: Illustration of an homogeneous self-assembly and growth in a closed unit volume initiated with M 3 free monomers. At a specific intermediate time in this depicted realization, there are six free monomers, four dimers, four trimers, and one cluster of size four. For each realization of this process, there is a specific time t at which a maximum cluster (N 6 in this example) is first formed (blue cluster). Viral proteins may also self-assemble to form capsid shells in the form of helices, icosahedra, dodecahedra, depending on virus type. A typical assembly process involves several steps where dozens of dimers aggregate to form more complex subunits which later cooperatively assemble into the capsid shell. Usually, capsid formation requires hundreds of protein subunits that self-assemble over a period of seconds to hours, depending on experimental conditions [147, 148]. Aside from these two illustrative cases, many other biological processes involve a fixed maximum cluster size of tens or hundreds of units at which the process is completed or beyond which the dynamic change [99]. Developing a stochastic self-assembly model with a fixed maximum cluster size is thus important for our understanding of a large class of biological phenomena. Theoretical models for self-assembly have typically described mean-field concentrations of clusters of all possible sizes using the well-studied mass-action, Becker-Döring equations [119, 14, 128, 36]. While Master equations for the fully stochastic nucleation and growth problem have been derived, and initial analyses and simulations performed [18, 125], there has been relatively less work on the stochastic self-assembly problem. Two collaborators of this present work have recently shown that in finite systems, where the maximum cluster size is capped, results from mean-field mass-action equations are inaccurate and that in this case a stochastic treatment is necessary [47]. In previous work of equilibrium cluster size distributions derived from a discrete, stochastic model, the authors in [47] found that a striking finite-size effect arises when the total mass is not divisible by the maximum cluster size. In particular, they identified the discreteness of the system as the major source of divergence between mean-field, mass action equations and the fully stochastic model. Moreover, discrepancies between the two approaches are most apparent in the strong binding limit where monomer detachment is slow. Before the system reaches equilibrium, or when the detachment is appreciable, the