Switching Regulatory Models of Cellular Stress Response

Transcription

1 Bonformacs Advance Access publshed March, 9 Swchng Regulaory Models of Cellular Sress Response Gudo Sangune a,b, Andreas Ruor c, Manfred Opper c and Cedrc Archambeau d a Deparmen of Compuer Scence, Unversy of Sheffeld, Regen Cour, Porobello Road, Sheffeld, S 4DP, U.K., b Bologcal and Envronmenal Sysems Group, Deparmen of Chemcal and Process Engneerng, Unversy of Sheffeld, Mappn Sree, Sheffeld, S 3JD, U.K. c Deparmen of Compuer Scence, Technsche Unversä Berln D-587 Berln, Germany d Deparmen of Compuer Scence Unversy College London, Gower Sree, WCE 6BT, U.K. Assocae Edor: Prof Davd Rocke ABSTRACT Movaon Sress response n cells s ofen medaed by quck acvaon of ranscrpon facors. Gven he dffculy n expermenally assayng ranscrpon facor acves, several sascal approaches have been proposed o nfer hem from mcroarray me-courses. However, hese approaches ofen rely on pror assumpons whch rule ou he rapd responses observed durng sress response. Resuls We presen a novel sascal model o nfer how ranscrpon facors medae sress response n cells. The model s based on he assumpon ha sensory ranscrpon facors quckly rans beween acve and nacve saes. We herefore model mrna producon usng a b-sable dynamcal sysems whose behavour s descrbed by a sysem of dfferenal equaons drven by a laen sochasc process. We assume he sochasc process o be a wo sae connuous me jump process, and devse boh an exac soluon for he nference problem as well as an effcen approxmae algorhm. We evaluae he mehod on boh smulaed daa and real daa descrbng E. col s response o sudden oxygen sarvaon. Ths hghlghs boh he accuracy of he proposed mehod and s poenal for generang novel hypoheses and esable predcons. Avalably MATLAB and C++ code used n he paper can be downloaded from hp:// gudo/. Conac gudo@dcs.shef.ac.uk INTRODUCTION Undersandng he molecular bass of sress reacon n cells s one of he mos mporan asks n Sysems Bology. Sress reacon mechansms are key n a number of bomedcal and bo-engneerng applcaons, rangng from drug desgn o genec engneerng of drough-ressan crops. Whle mos expermenal sudes have radonally focused on comparng seady saes before and afer he mposon of he sress, s becomng ncreasngly clear ha he dynamcs of he mmedae reacon o he sress hold mporan bologcal nformaon [see e.g. Parrdge e al., 7]. Cells respond o exernal smul n a varey of ways; perhaps he mos fundamenal s he ranscrponal one. The smulus s medaed by ranscrpon facors (TFs) whch rans from nacve o acve sae and bnd o specfc genes o acvae or nhb her ranscrpon. Despe s mporance, ranscrponal regulaon s far from beng wholly undersood. In parcular, s expermenal exploraon s severely hampered by he fac ha some of he fundamenal key players are very hard o measure: expermenal echnques o measure acve TF concenraons or o quanae her effec on arge genes are dffcul and me consumng. In response o hese expermenal lmaons, here has been a sgnfcan amoun of effor n he modellng communy n order o produce sascal models of ranscrpon o nfer he acvy levels of TFs from me-seres measuremens of he arges expresson levels. Broadly speakng, here are wo caegores of models for TF acvy nference: coarse models whch aemp o capure he smulaneous behavour of all TFs and all genes n an organsm [e.g. Lao e al., 3, Sangune e al., 6, Saba and James, 6] and dealed, ordnary dfferenal equaons (ODE) based models of small subneworks nvolvng only a handful of genes and one TF (Sngle Inpu Mofs, SIM) [e.g. Barenco e al., 6, Lawrence e al., 6, Rogers e al., 7, Khann e al., 7, Gao e al., 8]. Whle hs may seem an overly smple sysem, should be poned ou ha SIMs are amongs he mos over-represened nework mofs n baceral ranscrponal neworks [Alon, 6]. Mos of hese models dscrese me assumng TF acvy o be consan beween observaon pons; o our knowledge, he only approach o nfer a connuous me TF acvy profle s Lawrence e al. [6], where a Gaussan process (GP) pror dsrbuon s placed over TF acvy (hs was exended furher n Gao e al. [8]). Whle hese approaches have ceranly se mporan groundwork o undersand ranscrponal regulaon, sress reacon poses new challenges o he sascal modeller. Rapd adapaon o envronmenal changes s ofen key o he survval of he cell; n order o cope wh hs, TFs are ofen pos-ranscrponally regulaed va fas processes such as dmersaon and phosphorylaon, so ha hey can be urned on or off as soon as he sgnal s receved [see e.g. Alon, 6]. Ths s clearly a problem for dscree me models as he pecewse consan assumpon canno be jusfed n hs case. Whle he connuous me approach of Lawrence e al. [6] s more appealng, usng a GP pror for TF acvy nroduces a srong connuy consran (ndeed a smoohness consran n many cases). Moreover, f a saonary covarance s employed, hs wll auomacally deermne a characersc me-scale over whch he laen process can change. Ths s ll sued o model ranscrpon facors whch occasonally vary very quckly, whle remanng n a seady sae for he majory of me. Of course, hs could be The Auhor (9). Publshed by Oxford Unversy Press. All rghs reserved. For Permssons, please emal: journals.permssons@oxfordjournals.org

2 Gudo Sangune a,b, Andreas Ruor c, Manfred Opper c and Cedrc Archambeau d avoded by usng a non-saonary covarance (as for example n Archambeau e al. [7]), bu he compuaonal overheads ncurred would be very sgnfcan. In hs conrbuon, we presen a new approach for connuous me TFacvy nference whch specfcally models quck response o sress sgnals. We buld on he model-based approach naed n Barenco e al. [6], where mrna expresson for some genes s conrolled by an unobserved TFva a sysem of ordnary dfferenal equaons. We hen model he laen process as a Markovan sochascdynamcal process whch performs ransons beween wo saes (he elegraph process). Thus, our model of ranscrpon s an approxmaon o he classcal Mchaels-Menen dynamcs where he me aken for TF concenraons o change from neglgble o sauraon level s assumed o be exremely shor (compared o he me beween observaons). The Markovan naure of he process means ha exac nference s possble for hs sysem. However, he compuaonal overheads are sgnfcan, and we devse an effcen approxmae nference scheme based on a varaonal approach. In addon o nferrng TF profles, our mehod also gves an effecve way o esmae a number of parameers, such as mrna producon and decay raes, whch ofen play a crcal role n sysems bology models bu are rarely precsely known. The res of he paper s organsed as follows. In he model and mehods secon, we descrbe our ranscrponal regulaon model, brefly revew nference approaches for connuous me Markov processes, and descrbe boh our exac and varaonal soluons. In he resuls secon, we es boh he exac and approxmae nference on a smulaed daa se, and compare our approach o Lawrence e al. [6] o underlne he mporance of approprae pror assumpons. We hen use our approach o sudy he behavour of he maser regulaor FNR n he adapaon of E. col durng he sudden change from aerobc o anaerobc condons. FNR acvaon coordnaes he acon of hundreds of genes nvolved n swchng E. col meabolsm from aerobc o nrc; our analyss leads o bologcally measurable predcons, such as he exsence of a fne (measurable) me-lag beween sress mposon and FNR acvaon, and a predced decrease n acvy when he new anaerobc seady sae s acheved. In he dscusson secon, we dscuss relaed approaches and evaluae he relave mers of our approach, as well as hghlghng poenal exensons. MODEL AND METHODS The sarng pon for our model s he Mchaels-Menen model of ranscrpon for a sngle-npu mof model wh =,..., m gene arges [Alon, 6, Barenco e al., 6] dx () d = A c () κ + c () + b λ x () Here x () s mrna concenraon of arge gene as a funcon of me, b s s baselne ranscrpon rae, λ s he mrna decay rae and c() s he acve TF concenraon, self a funcon of me. The remanng parameers A and κ deermne he amplude and shape of he acvaon curve; A can be nerpreed as he sensvy of x o he TF, and κ represens he concenraon a whch half he sauraon level of acvaon s acheved. Our am s o model a suaon where a rapd response o a sgnal makes he TFacvy quckly swch beween he sauraon level and zero. We wll herefore smplfy he model as dx () d = A µ () + b λ x () () where µ() {, }. The model herefore s a b-sable model wh a hgher seady sae x = A +b and a lower seady sae for x λ = b. λ Gven he me dependence of he drvng process µ, equaon () s easy o solve n closed form, and he parameers can be esmaed by sandard mehods (e.g. leas squares). However, we are neresed n he suaon where he process µ s no observed. To encode he fac ha µ can perform an arbrary number of swches beween s wo saes, we wll place a pror on n he form of a wo-saes Markov jump process, also known as a elegraph process. The elegraph process s characersed by s ranson raes f ± (), whch gve he rae a whch he process swches beween he wo saes. To perform nference, we wll be neresed n he sngle me margnal probably p (), gvng he probably ha he process s n he on sae a me. Gven ranson raes f ± for he process, he probably of he sysem beng n a parcular sae a a gven me s gven by he followng Maser equaon dp () d = f p () + f + p () dp () = f + p () + f p (). d Usng he fac ha p + p = a all mes, one can reduce he Maser equaon o a sngle equaon on he probably p as dp () = (f + + f )p () + f +. () d We wll assume ha we have nose corruped observaons ˆx ( j ) of he oupu x () a dscree me pons j, j =,..., N wh x ( ) servng as nal condons for he problem. The observaons (condoned on he rue sae) wll be assumed o be..d. wh normal nose model wh varance σ. Thus he probably of makng a sngle observaon ˆx() gven x() a me s descrbed by a Gaussan lkelhood p(ˆx() x()) exp «! ˆx () x (). (3) σ = Alhough hs s an ncorrec nose model as he quany x s clearly posve a all mes, he error made wll be small for b /λ much larger han he observaon nose σ. The remanng parameers of he model A, b and λ are consraned posve gven her physcal meanng as producon and decay raes and wll be gven exponenal or fla prors. The nference ask consss of wo pars: sae nference, where we use he nosy observaons ˆx o nfer he poseror dsrbuon over he rue sae of he sysem (boh x and µ), and parameer esmaon, where we learn he model parameers A, b, λ and σ. In he followng subsecons, we wll presen wo approaches o performng nference n hs model. Frs, we oulne an exac nference approach whch explos he causal srucure of he model o derve a forward-backward algorhm for he jon poseror over x and µ. Ths s closely relaed o he famlar algorhm for Hdden Markov Models, he man dfferences beng ha our sae vecor s hybrd connuous-dscree, and ha me s connuous. The man drawback of hs approach s compuaonal: he forward-backward pass requres solvng numercally paral dfferenal equaons n poenally hgh dmensons. We hen presen a more effcen approxmae nference algorhm whch avods hese problems by drecly modellng he poseror dsrbuon over he TF acvy µ.. Exac nference Alhough he process x() wh observaons ˆx( j ) looks lke a sandard Hdden Markov Model, hs assumpon s no correc. In fac, x() s an negral over he Markov jump process µ(), as shown by obanng he general soluon of equaon () usng Laplace ransform:» x () = e λ ( ) x( ) + e λ (s ) (A µ(s) + b )ds. (4) Therefore x() depends on he whole hsory of he process µ() up o me. However, he combned process (µ(), x()) s Markovan, as he dynamcs descrbed n () and () depend only on he curren sae of he sysem. Consequenly, we can base our exac soluon o he sae nference problem on he forward-backward algorhm for Markovan sochasc processes,

3 Swchng regulaory models f we use boh µ() and x() as sae varables. There are however sll wo key dfferences beween our swchng model and a sandard HMM: he sae varable (x(), µ()) s hybrd connuous-dscree, and me s a connuous varable. Therefore he well-known forward and backward recurson rules for dscree Hdden Markov Models are replaced by paral dfferenal equaons (PDEs), he Chapman-Kolmogorov equaons [ e.g. Gardner, 996]. Our model s somehow smpler han he general case: jumps only occur n µ() and here s no dffuson, as x() s a deermnsc funcon f µ() s known. We wll use he Chapman-Kolmogorov equaons o calculae he margnal probably dsrbuon q µ(x, ) of he poseror process of µ and x = (x, x,... x m). Usng he Markovan srucure of hs jon process one can show ha q µ(x, ) = pµ(x, )Ψµ(x, ). (5) Ths decomposon of he poseror margnal s he connuous me verson of he well known decomposon n erms of forward and backward messages for Hdden Markov Models [see e.g. Bshop, 6]. Here p µ(x, ) denoes he margnal probably dsrbuon of he process condoned on he daa before me,.e. he flered process or forward message, and s a me-ndependen normalsaon consan, whch equals he lkelhood p(ˆx,..., ˆx N θ) of he daa gven he model parameers θ = (λ, f, f, A, b). The las par of (5), Ψ µ(x, ) = p ({ˆx( j ) j > } x() = x, µ() = µ), (6) s he lkelhood of all observaons afer me under he condon ha he process has sae (x, µ) a me (he backward message). Adapng he general form of he dfferenal Chapman-Kolmogorov equaons [Gardner, 996] o our case, we oban he followng backward equaon sasfed by Ψ µ(x, ), Ψ Ψ + (A + b λ x ) Ψ = = f (Ψ (x, ) Ψ (x, )), + (A + b λ x ) Ψ = = f + (Ψ (x, ) Ψ (x, )). (7) These PDEs mus be solved backward n me sarng a he las observaon ˆx( N ) usng he nal condon Ψ µ(x, N ) = p(ˆx( N ) x( N ) = x). (8) The oher observaons are aken no accoun by jump condons Ψ µ(x, j ) = Ψµ(x, + j ) p(ˆx( j) x( j ) = x) (9) wh Ψ µ(x, j ) beng he values of Ψµ(x, ) before and afer he j-h observaon. Here we use he propery of he nose model ha he observaons ˆx( j ) are ndependen condoned on he process (µ(), x()). Therefore he lkelhood Ψ µ(x, j ) ncludng ˆx( j) s he produc of Ψ µ(x, + j ) for observaons a laer me pons and he probably p(ˆx( j ) x( j ) = x) gven by (3). In order o calculae q µ(x, ) we need o consder he flered process descrbed by p µ(x, ), oo. Is me evoluon s gven by he forward Chapman-Kolmogorov equaon p m + X (A + b λ x ) p (x, ) = = f + p (x, ) f p (x, ) p + = (A + b λ x ) p (x, ) = f p (x, ) f + p (x, ) () and he poseror q µ(x, ) fulfls a smlar PDE. Ths can be seen by calculang he me dervave q µ = «Ψ µ(x, ) pµ + p µ(x, ) Ψµ () of he poseror dsrbuon. Usng he PDEs gven n (7) and () we fnd q = (A + b λ x )Ψ (x, )p (x, ) = q + (f +Ψ (x, )p (x, ) f Ψ (x, )p (x, )), = = (A + b λ x )Ψ (x, )p (x, ) + (f Ψ (x, )p (x, ) f + Ψ (x, )p (x, )). () Ths equaon can be furher smplfed by nroducng me and sae dependen poseror jump raes g + (x, ) = Ψ (x, ) Ψ (x, ) f + g (x, ) = Ψ (x, ) Ψ (x, ) f (3) and applyng (5). We hen fnd q m + X (A + b λ x )q (x, ) = = g + (x, ) q (x, ) g (x, ) q (x, ), q + = (A + b λ x )q (x, ) = g (x, ) q (x, ) g + (x, ) q (x, ), (4) whch s also he forward Chapman-Kolmogorov equaon. Consequenly, he only dfferences beween pror and poseror process are he jump raes for he elegraph process µ(). In he case of a sngle arge gene numercal negraon of he PDEs (7) and (4) s compuaonally feasble. We use he Lax algorhm [Vesely, 994] for ha purpose, because prevens negave values for Ψ µ(x, ) and q µ(x, ) as long as he sep szes fulfl he condon x > A. The boundares are deermned by he wo seady saes x low = b/λ and x hgh = (A + b)/λ. In he forward negraon hese boundares are closed, as he process canno leave he nerval beween x low and x hgh. Bu can come from he ousde, so ha we have o use open boundares n he backward negraon. Parameer esmaon based on he exac soluon of he sae nference problem s also possble. For ha purpose we use he free energy F ln p(ˆx( ),..., ˆx( N ) θ),.e. he negave log-lkelhood of he daa as a funcon of he model parameers θ = (λ, f, f, A, b). Ths quany s gven by F = ln E pror [Ψ µ(x, )] = ln [Ψ (x, )p (x, ) + Ψ (x, )p (x, )] dx (5) and only a sngle backward negraon s necessary n order o oban Ψ µ(x, ). Here E pror denoes expecaon under he pror dsrbuon for he frs observaon a. Mnmsng he free energy wh respec o he model parameers hen leads o her ype II maxmum lkelhood esmaes θ = arg mn F (θ).. Varaonal approxmaon As dscussed above, he exac soluon for he nference problem for he swchng model suffers from he curse of dmensonaly, so ha exac nference n hgher dmensons becomes prohbvely expensve. Varaonal nference [see e.g. Jordan e al., 999] s a powerful approach o solvng 3

4 Gudo Sangune a,b, Andreas Ruor c, Manfred Opper c and Cedrc Archambeau d approxmaely he nference problem. Gven an nracable probably dsrbuon p, he varaonal approach fnds an opmal approxmaon o p whn a ceran famly of dsrbuons. Ths s usually done by mnmzng he Kullback-Lebler (KL) dvergence beween he wo dsrbuon KL [q p] = E q» log q p = dq log q p. By selecng a suable famly of approxmang dsrbuons, he nference problem s hen urned no an opmsaon problem. We wll resrc he dscusson o he case of a sngle arge gene, he generalsaon o more genes beng sraghforward. In he followng, we wll vew he sochasc process as a probably measure over he (nfne dmensonal) space of possble pahs of he TF µ :T (he noaon µ :T ndcaes a specfc realsaon of he process beween and T ). Gven a pror elegraph process p (µ :T f ± ) and a nose model for he observaons p (ˆx µ :T ), Bayes heorem allows n prncple he compuaon of a poseror process as p pos (µ :T ˆx) = p (ˆx µ :T ) p pror (µ :T f ± ). (6) As he soluon (4) of he model () depends on he whole hsory of he process µ, so wll he lkelhood facor n (6). Ths means ha he poseror process wll no be a Markov process. However, sll makes sense o seek a Markov process ha approxmaes opmally he poseror process. To do hs, we wll compue he Kullback-Lebler (KL) dvergence beween he poseror process n (6) and an approxmang Markov process q (µ g ± ). Ths s gven by KL [q p pos] = ln + KL [q p pror ] NX E q [ln p (ˆx j x ( j ))]. j= (7) The KL dvergence beween wo Markov jump processes was compued n he general case n Opper and Sangune [7] maeral). A dervaon of he KL dvergence n he specal case of he elegraph process can be found n he supplemenary maeral, he fnal resul beng gven by T» KL [q p pror ] = dq () g () ln g () f () + f () g () + T» d [ q ()] g + () ln g +() f + () + f +() g + (). The esmaon of he lkelhood erm n (7) s more challengng; under he assumpon of Gaussan nose, requres he compuaon of he frs wo momens of he random varable x() under he approxmang process q. These are gven by» x ( ) = exp ( λ ) x () + b λ (exp (λ ) ) +A exp (λs) q (s) ds x ( ) = exp ( λ ) jx () + b λ [exp (λ ) ] + + x () b λ (exp (λ ) ) + x () AI + b λ (exp (λ ) ) AI + + A ff exp [λ ( + s)] q (, s) dds. (8) Here, I = R exp (λs) q (s) ds and we have used he fac ha µ() q = q (). In general, hese negrals are analycally nracable when he raes for he approxmang process g ± are funcons of me. We wll herefore solve he opmsaon problem on a grd, assumng he approxmang process raes o be consan beween pons n he grd. Ths allows us o solve explcly he Maser equaon on he grd, and herefore allows he calculaon of he negrals needed n (8) (he explc calculaons are gven n he supplemenary maeral). By akng he grd o be suffcenly fne, he numercal soluon can approxmae he rue mnmum o arbrary precson. Ths algorhm scales lnearly wh he number of genes (snce we need o compue a se of momens per gene), makng compuaonally much more effcen han he exac nference soluon..3 Parameer esmaon The varaonal procedure oulned above wll oban an approxmaon o he poseror dsrbuon of he swchng TF µ gven he observaons and he model parameers. Ths can hen be used o esmae he parameers of he model, n an EM-lke scheme (ofen called Varaonal Bayes EM). In he E- sep, an approxmae poseror s compued by mnmsng he KLdvergence wh respec o he rae parameers g±. Ths can be done by graden descen or usng oher search sraeges. In he M-sep, we use he approxmae poseror q(µ) o margnalse he process µ, obanng an approxmaon o he margnal lkelhood of he daa as p (ˆx Θ) exp E q(µ) [log p (ˆx, µ Θ)] (9) where = exp( H[q]) and H[q] s he enropy of he approxmae poseror. Ths can be shown o be a lower bound on he rue lkelhood by nvokng Jensen s nequaly [see e.g. Jordan e al., 999]. Equaon (9) can be used n Bayes heorem o compue poseror dsrbuons over he model parameers. The poseror dsrbuon over he parameers A, b and σ s obaned analycally (for suable choces of prors) n he form of runcaed Gaussan and nverse Wshar dsrbuons. Unforunaely, hs s no possble for he decay parameer λ; esmaon of s poseror dsrbuon s done by drec evaluaon of he un-normalsed poseror over a grd n dmenson. RESULTS. Synhec daa To benchmark he model and assess he valdy of our approxmaon, we ran he model on a smple synhec example. We consruced a smple oy daa se made up of en equally spaced observaons drawn from he model wh npu sgnal (TF acvy) µ() = j [, 69] [66, ] [7, 659] The dfferenal equaon parameers were chosen as A = 3.7 3, b = 8 4 (producon raes) and λ = 5 3 (decay rae). Gaussan nose wh a sandard devaon of.3 was added o he heorecal values of x o gve he observaons. The resuls of he nference are shown n Fgure. Fgure (ab) show he nferred poseror mean (dashed black) compared o he npu mpulse (hn black) n he approxmae and exac case respecvely. In order no o cluer he fgures, we omed he confdence nervals for he poseror TF acvy; snce a each me he TF s a bnary varable, hese can be obaned from he mean value as p (q q ). Also shown s he poseror frs momen for x() (hck black) wh confdence nervals and he daa (red crosses). The grd used had fve grd pons for every observaon pon (for a dscusson of how grd sze affecs model resul, see he supplemenary maeral). Boh he reconsrucons are reasonable alhough he exac one has much gher uncerany. Fgure (c) shows he resuls of applyng Lawrence e al. [6] o he daa usng a squared exponenal covarance funcon for he GP pror. Alhough he model produces a good f o he daa, he saonary. 4

5 Swchng regulaory models (a),9,8,7,6,5,4,3,, (b) (c) Fgure. Resuls of nference on oy daa se. (a) Resuls from varaonal approach: nferred poseror mean (dashed black) compared o he npu mpulse (hn black). Also shown he poseror frs momen for x() (hck black) wh confdence nervals and he daa (red crosses). (b) Resuls of exac nference. (c) Reconsruced frs momen usng Lawrence e al. [6]. covarance used forces he nferred TF profle o have bologcally meanngless fas oscllaons. Boh he exac and approxmae nference slghly underesmae he value of he parameer λ a 4 ±.3 3 (rue value 5 3 ). The esmaes for he model parameers are good boh n he approxmae and exac case, wh A =.8 ±.3 3 and b =.8 ±. 3 (resuls from approxmae nference) and A = 3. ±. 3 and b =.8 ±.6 3 (resuls from exac nference). The exac nference was carred ou wh σ fxed o he rue value, whle he approxmae nference obaned σ =.5. The whole process ook approxmaely en mnues on a sandard PC for he approxmae case and hours for he exac case.. Mcroaerobc shf n E.col As a real example on whch o es our approach, we consdered ranscrpomc measuremens of he reacon of E.col o sudden oxygen sarvaon [Parrdge e al., 7]. Eschercha col sa robus organsm ha can adap remarkably well o changes n s envronmen. One of he mos dramac such changes rounely encounerd by he bacerum s he change n he avalably of oxygen: he bacerum can be expelled from he hos s gu and very rapdly moves from an envronmen wh vrually no oxygen o an aerobc envronmen. Ths change enals a whole shf n he meabolsm of he bacerum from a nrc meabolsm o a much more energecally favourable aerobc meabolsm. The se of enzymes nvolved n he wo dfferen phases of E.col meabolsm s only parly overlappng; n order operform hs shf, a large number of genes mus be urned on and off n a coordnaed manner. Ths acon s carred ou by a few TFs whch respond o oxygen sgnals. Perhaps one of he mos mporan oxygen sensors n he cell s he ron-sulphur cluser proen FNR. FNR s a maser regulaor (.e., one of a dozen TFs whch arge mos of he baceral genes) whch can exs n wo saes. Is sae n he presence of oxygen s an nacve monomer. When oxygen s removed, he proen s dmerzed whch can n urn bnd DNA and acvae or repress ranscrpon of a large number of genes. Therefore, one expecs ha, afer a ceran me lag, FNR undergoes a fas ranson from nacve o acve sae. Ineresngly, oal proen concenraon of FNR (dmer + monomer) s approxmaely consan beween aerobc and anaerobc condons [Jervs and Green, 7]. In he expermenal seng consdered n Parrdge e al. [7], an aerobcally grown culure of E.col K was rapdly deprved of oxygen. Mcroarray measuremens were hen aken a 5,, 5 and 6 mnues followng he mposon of he sress. The arrays measure he change n concenraon of mrna relave o he nal pon. Ths mples ha one of he parameers n our model, b, becomes undenfable as he lower seady sae becomes. Ths s easly resolved by seng b = λ n he model. Parrdge e al. [7] also performed genome-wde TF acvy nference usng he probablsc model descrbed n [Sangune e al., 6]. Ths showed a rapd response of FNR o he sgnal ha appeared o al off when he sysem reached seady sae. We consdered a subse of fve genes whch are known o be acvaed by FNR and wh a reasonably smple promoer srucure: hese are ompw, yjd, hypb, moaa and aspa. In pracce, all of hese genes, wh he excepon of yjd, are also regulaed by oher ranscrpon facors. However, hese oher ranscrpon facors are furher downsream han FNR n he sress response cascade, so ha one can assume ha he nal response o oxygen whdrawal s well modelled as a SIM. Performng he exac nference n hs case would requre numercally solvng a paral dfferenal equaon (PDE) wh fve spaal dmensons, whch s nfeasble. The resuls of he approxmae nference of he FNR acve profle are shown n Fgure (a). The sysem appears o undergo a sharp ranson beween nacve and acve sae a around 3 o 6 mnues. Ths resuls n an neresng predcon: removng oxygen for a perod shorer han 3 mnues wll no lead o an FNR-medaed ranscrponal response. Therefore, one may vew hs as an ndrec measuremen of he me akes E.col o comm self o change s meabolc regme beween aerobc and nrc. The model also predcs ha he acvy of FNR wll al off slghly afer approxmaely mnues. Whle, here s no smple bochemcal explanaon for hs, as FNR wll reman dmerzed and hence acve as long as oxygen s absen, a decrease n acvy owards seady sae was also predced by Parrdge e al. [7] usng a dfferen compuaonal model. The mos plausble explanaon s he acon of oher ranscrpon facors whch are downsream arges of FNR and whch become acve afer a reasonable ranscrponal delay. Fgure (b) shows he nferred half lves of he fve arges (rangles on he rgh) agans her expermenally measured values 5

6 Gudo Sangune a,b, Andreas Ruor c, Manfred Opper c and Cedrc Archambeau d Selnger e al. [3]. In wo cases, yjd and moaa, he expermenal value of he half lfe of he ranscrp was no avalable. In general, here s a good agreemen beween he nferred values and he expermenal measuremens, alhough should be noced ha he expermenal measuremens are exremely nosy n some cases. The ably o provde a reasonable ndrec esmae of mrna half lves spoenally precous o bologss: s known ha mrna decay s a regulaed process, mplyng ha mrna half lves measured n dfferen expermenal condons wll n general be dfferen. As s dffcul o measure expermenally decay raes n a dynamc seng lke sress reacon, s essenal o be able o denfy hese parameers n he model. We can gan furher nsgh no he workngs of he model by comparng predced expresson profles wh he observed dscree me pons. Fgure 3 shows hs for hree genes, ompw (a), hypb (b) and aspa (c). The sold lnes represen he mean of he sochasc process, and he doed lnes are he confdence nervals obaned by addng ± sandard devaon of he me margnals (hese error bars nclude only he varably n he sochasc process, he knec parameers were fxed a her maxmum lkelhood value for he plo). In general, all reconsruced profles show a saurang behavour, as mpled by he nferred TF acvy (Fgure (a)). The specfc form of he profle hough s deermned by he knec parameers nferred. I s also neresng o noce ha he f of he model o he hypb expresson profle s no as good as n he oher wo cases. In parcular, hypb expresson markedly decreases from 5 mnues o6 mnues, whch s ncompable wh he oher profles and can hardly be accommodaed by he model. Ths s probably due o he effec of he ranscrpon facor IHF, whch also acvaes hypb bu s repressed by FNR. I s herefore plausble ha, afer a ceran amoun of me, he SIM approxmaon breaks down n hs case, explanng he poor f o hs profle. 3 DISCUSSION In hs paper, we presened a novel model-based approach o nfer TF acvy profles from mcroarray me seres daa. The cenral assumpons underpnnng he model are he SIM assumpon (all he genes are arges of a sngle TF) and he pror model ha TFs rans quckly from acve o nacve sae. Ths second assumpon s lkely o be reasonable n many sress reacon expermens, parcularly when he sress s appled a he meabolc level, rggerng a pos-ranslaonal modfcaon of he TFs. Whle hs s a farly broad class of condons, s mporan o pon ou ha here are bologcal sresses (e.g. hea shock) ha do no f well n hs caegory. I would be neresng o explore sochasc models ha can combne fas and slowly reacng componens. The work ha s perhaps mos closely relaed o ours s Lawrence e al. [6], where ranscrponal regulaon s modelled wh a lnear sysem of ODEs wh a laen drvng facor. However, he choce of a Gaussan process pror for he TF acvy has as a naural consequence ha he nferred poseror TF profles vary connuously wh me. Whle hs may be a reasonable assumpon n ceran sengs, clearly s unenable when modellng fas bologcal processes such as sress reacon. Anoher mporan advanage of our model over Lawrence e al. [6] s he ably o denfy he decay parameers. Lawrence e al. need o fx a leas one of he decay raes o he expermenally measured value (a weakness shared by Barenco e al. [6]). Gven he very low accuracy of such measuremens (see Fgure b), he bas nroduced by fxng a parameer could poenally lead oserous errors nhe esmaon of he laen process. There are also several oher papers ha aack he problem of nferrng TF acvy profles from mrna me seres daa. For example, Rogers e al. [7], Khann e al. [7] use he same ODE-based model of ranscrpon, bu hen resrc hemselves o pecewse consan TF level, effecvely dscresng me. These models are also lmed o he SIM case. Many oher models address he global case, where hundreds of TFs regulae housands of genes [e.g. Lao e al., 3, Sangune e al., 6, Saba and James, 6]; however, n order o conan complexy, hey adop a smplsc model of ranscrpon where only lnear and addve effecs of TFs are reaned. Whle genome-wde modellng s sll an ambous arge for he model we developed, should be poned ou ha presens sgnfcan compuaonal advanages over oher ODE-based approaches. In parcular, he complexy of our algorhm scales lnearly wh he number of arge genes and me pons, whle for example he Gaussan Process-based approach of Lawrence e al. [6] s cubc n he produc of he number of genes and me pons. In hs lgh, sascal modellng of a moderae sze pahway s ceranly whn reach. There s an neresng relaonshp beween our work and he work on re-wrng neworks of Guo e al. [7]. There, lnks n a bologcal nework were swched on and off accordng o a dscree me Markov process. In our approach, s he acvy of he regulaory nodes n he nework ha swches on and off as a (connuous me) Markov process. Anoher poenally neresng generalsaon of our work s o he case where he sochasc behavour of gene expresson s no solely due o he TFs acvy, bu also o he nrnsc sochascy of ranscrpon. A varaonal approach o nference n b-sable (or more generally, non-gaussan) sysems of SDEs was recenly proposed n [Archambeau e al., 7]. ACKNOWLEDGEMENTS G.S. would lke o hank Jeff Green and Rober Poole for many useful dscussons on he physologcal funcons of FNR, and for nroducng hm o he fascnang world of baceral sress reacon. C.A. and M.O. graefully acknowledge suppor from he EPSRC under he VISDEM projec (EP/C5848/). We would lke o hank he anonymous revewers for her consrucve crcsm. REFERENCES U. Alon. An nroducon o sysems bology. Chapman and Hall, London, 6. C. Archambeau, D. Cornford, M. Opper, and J. Shawe-Taylor. Gaussan process approxmaons of sochasc dfferenal equaons. Journal of Machne Learnng Research Workshop and Conference Proceedngs, (): 6, 7. M. Barenco, D. Tomescu, D. Brewer, R. Callard, J. STark, and M. Hubank. Ranked predcon of p53 arges usng hdden varable dynamcal modellng. Genome Bology, 7(3), 6. C. M. Bshop. Paern Recognon and Machne Learnng. Sprnger, Sngapore, 6. 6

7 Swchng regulaory models µ mn (a) ompw yjd hypb moaa aspa (b) Fgure. Resuls on E.col daa: (a) poseror mean FNR profle; (b) half lves of arges (n mnues) wh uncerany, nferred (rangles on he rgh) versus expermenally measured. No measuremen of he half lfe of yjd or moaa s avalable (a) (b) (c) Fgure 3. Reconsruced expresson profles versus observed daa: (a) ompw, (b) hypb, (c) aspa. Sold lnes are mean predcon, doed black lnes confdence nervals (± sandard devaon), red crosses measured expresson levels P. Gao, A. Honkela, M. Raray, and N. D. Lawrence. Gaussan process modellng of laen chemcal speces: applcaons o nferrng ranscrpon facor acves. Bonformacs, 4(6): 7 75, 8. C. W. Gardner. Handbook of Sochasc Mehods. Sprnger, Berln, second edon, 996. F. Guo, S. Hanneke, W. Fu, and E. P. Xng. Recoverng emporally rewrng neworks: a model-based approach. In Proceedngs of he 4h Inernaonal Conference on Machne Learnng, 7. A. J. Jervs and J. Green. In vvo demosraon of FNR dmers n response o lower O avalably. Journal of Bacerology, 89: 93 93, 7. M. I. Jordan,. Ghahraman, T. S. Jaakkola, and L. K. Saul. An nroducon o varaonal mehods for graphcal models. Machne Learnng, 37:83 33, 999. R. Khann, V. Vnco, V. Mersnas, C. Smh, and E. W. Sascal reconsrucon of ranscrpon facor acvy usng mchaels-menen knecs. Bomercs, 63:86 83, 7. N. D. Lawrence, G. Sangune, and M. Raray. Modellng ranscrponal regulaon usng gaussan processes. In Advances n Neural Informaon Processng Sysems 9, 6. J. C. Lao, R. Boscolo, Y.-L. Yang, L. M. Tran, C. Saba, and V. P. Roychowdhury. Nework componen analyss: Reconsrucon of regulaory sgnals n bologcal sysems. Proceedngs of he Naonal Academy of Scences USA, (6):55 557, 3. M. Opper and G. Sangune. Varaonal nference for markov jump processes. In Advances n Neural Informaon Processng Sysems, 7. J. D. Parrdge, G. Sangune, D. Dbden, R. E. Robers, R. K. Poole, and J. Green. Transon of eschercha col from aerobc o mcro-aerobc condons nvolves fas and slow reacng regulaory componens. Journal of Bologcal Chemsry, 8(5): 3 37, 7. S. Rogers, R. Khann, and M. Grolam. Bayesan model-based nference of ranscrpon facor acvy. BMC Bonformacs, 8 (S), 7. C. Saba and G. M. James. Bayesan sparse hdden componens analyss for ranscrpon regulaon neworks. Bonformacs, (6): , 6. G. Sangune, N. D. Lawrence, and M. Raray. Probablsc nference of ranscrpon facor concenraons and gene-specfc regulaory acves. Bonformacs, ():775 78, 6. D. W. Selnger, R. M. Saxena, K. J. Cheung, G. M. Church, and C. Rosenow. Global RNA half-lfe analyss n Eschercha col reveals posonal paerns of ranscrp degradaon. Genome Research, 3():6 3, 3. F. J. Vesely. Compuaonal Physcs: An Inroducon. Plenum Publshng Corporaon, New York,