Bayesian networks for reconstructing transcriptional regulatory networks

Size: px

Start display at page:

Download "Bayesian networks for reconstructing transcriptional regulatory networks"

Silvia Thornton
5 years ago
Views:

1 Bayesian networks for reconstructing transcriptional regulatory networks Prof Su-In Lee Computer Science & Genome Sciences University of Washington, Seattle GENOME 541 Introduction to Computational Molecular Biology II 1 All living organisms are composed of cells 2 1

2 From DNA to protein 22,000 genes in human! DNA Gene RNA Protein person x s cell AGATATGTGGATTGTTAGGATTTATGCGCGTCAGTGACTACGCATGTTACGCACCTACGACTAGGTAATGATTGATC A AUGUGGAUUGUU MWIV Transcription factor C B MWIV a switch! ( transcription factor binding site ) Gene regulation AUGCGCGUC AUGUUACGCACCUAC AUGAUUAU AUGAUUGAU AUGCGCGUC MRV MRV A and B regulate the expression of C transcription translation MLRTY Transcription factor Gene Expression MLRTY MID Genes regulate each others expression some biological processes gene regulatory network 3 Inferring the regulatory network DNA (3 billion-long string) AGCTATAGCATAGCACTACAGAC AGCATACACACCATTTTAAAACGCG CACAAAAATCAGCTAAACCAGGGTT ACTACGACACTTACAACTACATT For example, Given gene expression data measuring RNA levels of all genes Infer the regulatory network that controls gene expression gene regulatory network g 1 g 5 g Q gene expression data N instances Different conditions 4 2

3 Outline Mathematical tools Gene expression data and basic analysis techniques Reconstructing regulatory networks Individual sequence variation and gene regulation 5 Learning goals Be familiar with: a well-known Comp Bio problem Reconstructing the regulatory network from expression data How to evaluate the learned network probabilistic models in ML Bayesian network representation Parameter/structure learning of a Bayesian network Feature selection problem 6 3

4 Outline Mathematical tools Bayesian network representation Learning the structure of Bayesian networks Gene expression data and basic analysis techniques Reconstructing regulatory networks Individual sequence variation and gene regulation 7 Probability theory review Assume random variables Val(A)={a 1,a 2,a 3 }, Val(B)={b 1,b 2 } Conditional probability Definition Chain rule Bayes rule Probabilistic independence 8 4

Bayesian network semantics [koller and Friedman] A Bayesian network structure G is a directed acyclic graph whose nodes represent random variables X 1,,X n.

G encodes the following set of conditional independence assumptions, called the local Markov assumptions, and denoted by I L (G): x 2 For each variable X i : x 1

5 Bayesian network semantics [koller and Friedman] A Bayesian network structure G is a directed acyclic graph whose nodes represent random variables X 1,,X n. PaX i : parents of X i in G NonDescendantsX i : variables in G that are not descendants of X i. G encodes the following set of conditional independence assumptions, called the local Markov assumptions, and denoted by I L (G): x 2 For each variable X i : x 1 x 4 x 3 x 3 x 10 x 11 x 5 x 7 x 8 x 9 x 6 9 The Genetics example [koller and Friedman] Variables B: blood type (a phenotype) G: genotype of the gene that encodes a person s blood type; <A,A>, <A,B>, <A,O>, <B,B>, <B,O>, <O,O> 10 5

6 Bayesian network joint distribution [koller and Friedman] Let G be a Bayesian network graph over the variables X 1,,X n. We say that a distribution P factorizes according to G if P can be expressed as: A Bayesian network is a pair (G,P) where P factorizes over G, and where P is specified as a set of CPDs associated with G s nodes. 11 The Student example [koller and Friedman] An intuitive example Course difficulty (D), Intelligence (I), Grade (G), quality of the recommendation letter (L), SAT (S) Val(D) = {easy, hard}, Val(L) = {strong, weak}, Val (I) = {i 1,i 0 }, Val (S) = {s 1,s 0 }, Val (G) = {g 1,g 2,g 3 } Conditional independence L ㅗ I, D, S G S ㅗ I, D, G, L I G ㅗ S I, D I ㅗ D : 12 6

7 The Student Bayesian network Joint distribution P(I,D,G,S,L) = 13 Parameter estimation Assumptions Fixed network structure Fully observed instances of the network variables: D={d[1],,d[M]} Maximum likelihood estimation (MLE)! Parameters of the Bayesian network For example, {i0,d1,g1,l0,s0} 14 7

8 The Thumbtack example [koller and Friedman] Parameter learning for a single variable. Variable X: an outcome of a thumbtack toss Val(X) = {head, tail} X Data A set of thumbtack tosses: x[1], x[m] 15 Maximum likelihood estimation Say that P(x=head) = Θ, P(x=tail) = 1-Θ P(HHTTHHH <M h heads, M t tails>; Θ) = Definition: The likelihood function L(Θ : D) = P(D; Θ) Maximum likelihood estimation (MLE) Given data D=HHTTHHH <M h heads, M t tails>, find Θ that maximizes the likelihood function L(Θ : D). 16 8

9 MLE for the Thumbtack problem Given data D=HHTTHHH <M h heads, M t tails>, MLE solution Θ* = M h / (M h +M t ). Proof: 17 MLE for Bayesian networks Likelihood decomposition: The local likelihood function for X i is: Bayesian network with table CPDs 18 9

10 Learning the regulatory network A toy example with 4 genes Activator, Repressor, Target, Target2 Bayesian network representation Conditional independence structure Estimate the parameters & structure that give the best fit to data. Challenges Too many genes (22,000 genes)! Too many possible structures too large search space A=high R=high A=high R=low A=low R=high A=low R=low T=high Activator s RNA level T=low Target s RNA level Target2 s RNA level parameters structure T=high 0.99 A R T=lowT T2 0.1 Repressor s RNA level T2=high T2=low How to resolve? A new type of probabilistic model Biologically plausible assumptions N instances 19 gene expression data Outline Mathematical tools Gene expression data and basic analysis techniques Reconstructing the regulatory network Individual sequence variation and gene regulation 20 10

Gene expression data Induced i Experiments (samples) Downregulated Upregulated Genes j Repressed E ij - RNA level of gene j in experiment i 21 From Dr Segal s slides Why care about clustering?

11 Gene expression data Induced i Experiments (samples) Downregulated Upregulated Genes j Repressed E ij - RNA level of gene j in experiment i 21 From Dr Segal s slides Why care about clustering? Gene 1 Gene 2 E1 E2 E3 Gene N E1 E2 E3 Discover functional relation Similar expression functionally related Assign function to unknown gene Gene 1 Gene 2 Find which gene controls which other genes Gene N Find regulatory motifs (more later) 22 11

Hierarchical agglomerative Compute all pairwise distances Merge closest pair Data instances 23 From Dr Segal s slides Clustering expression profiles Limitations: Data

12 Hierarchical agglomerative Compute all pairwise distances Merge closest pair Data instances 23 From Dr Segal s slides Clustering expression profiles Limitations: Data instances No explanation on what caused expression of each gene (No regulatory mechanism) Co-regulated genes cluster together Infer gene function 24 From Dr Segal s slides 12

Outline Mathematical tools Gene expression data and basic analysis techniques Reconstructing the regulatory network Individual sequence variation and gene regulation 25 Goal: Inferring regulatory

13 Outline Mathematical tools Gene expression data and basic analysis techniques Reconstructing the regulatory network Individual sequence variation and gene regulation 25 Goal: Inferring regulatory networks Expression data Experimental conditions Q 2x10 4 (for human) Infer the regulatory network that controls gene expression e 1 e 6 e Q Causality relationships among e 1-Q A and B regulate the expression of C (A and B are regulators of C) B A C Bayesian networks 26 13

14 Regulatory network Bayesian network representation Xi: expression level of gene i Val(Xi): continuous Interpretation Conditional independence Causal relationships X2 X 1 X3 Joint distribution P(X) = Conditional probability distribution (CPD)? X5 X4 X6 27 Context specificity of gene expression Context A Basal expression level RNA level X2 X 1 Context B Activator induces expression Upstream region of target gene (X5) Activator (X3)? X3 X5 X4 X6 activator binding site Context C Activator + repressor decrease expression Repressor (X4) Activator (X3) repressor binding site activator binding site Segal et al. Nat Genet

15 Context specificity of gene expression Context A Basal expression level RNA level X2 X 1 Context B Activator induces expression Upstream region of target gene (X5) Activator (X3)? X3 X5 X3 X4 X6 Context C Activator + repressor decrease expression Repressor (X4) repressor binding site activator binding site Activator (X3) activator binding site false true X4 false true 0 Context A Context B Context C Parameterization Tree conditional probability distributions (CPDs) mean (µ) & variance (σ 2 ) of the normal distribution in each context Tree CPD X 1 false X3 true X2 (μ A,σ A ) X4 (μ B,σ B ) (μ C,σ C ) false true X3 X Context A 3 Context B -3 Context C X5 X

Reconstructing the network Training data Gene expression data Goal Learn the structure & tree CPDs parameters X 1 X3 false true X2 X4 X3 X4 false true (μ A,σ A ) 0 Context A (μ B,σ B ) 3 Context B.

16 Reconstructing the network Training data Gene expression data Goal Learn the structure & tree CPDs parameters X 1 X3 false true X2 X4 X3 X4 false true (μ A,σ A ) 0 Context A (μ B,σ B ) 3 Context B Context C (μ C,σ C ) X5 X6 31 Learning Structure learning [Koller & Friedman] Constraint based approaches Score based approaches Bayesian model averaging Given a set of all possible network structures and the scoring function that measures how well the model fits the observed data, we try to select the highest scoring network structure. X 1 Scoring function Likelihood score Bayesian score X3 X5 X2 X4 32 X6 16

17 Scoring functions Let S: structure, Θ S : parameters for S, D: data Likelihood score Bayesian score X 1 X3 X2 X4 X5 33 X6 Modularity of regulatory networks Genes tend to be co-regulated with others by the same factors. Biologically more relevant Module 1 More compact representation Smaller number of parameters Reduced search space for structure learning X3 X2 X 1 Module 2 X4 Candidate regulators A small set of genes that can be parents of other modules. Same module Share the CPD X5 Module 3 X

18 Reconstructing the regulatory network Training data Gene expression data Goal Learn the structure & tree CPDs parameters genes X1 X2 : X6 : experiments (arrays) Context specificity X 1 Module 1 false X3 true X2 Module 2 X4 X3 X4 (μ A,σ A ) 0 Context A (μ B,σ B ) false 3 Context B true Context C (μ C,σ C ) X5 X6 Module 3 35 Review: Structure learning Find the structure S that maximizes P(S D) ML score: max P(Structure Data) Θ log P(D S,Θ) More prone to overfitting P(D S) P(S) maximize S log P(D S) + log P(S) P(D S) = P(D S,Θ S ) P(Θ S S) dθ S P(S): prior distribution on the structure X1 X2 : X6 : genes Data D experiments (arrays) Decomposability For a certain structure S, log P(D S) Structure S? X 1 Module 1 = log P(D S,Θ S ) P(Θ S S) dθ S P(X1 Θ m1 )P(X2,X3,X4 X1,Θ m2 ) P(X5,X6 X3,X4,Θ m3 ) P(Θ m1 )P(Θ m2 ) P(Θ m3 ) module 1 score dθ m1 dθ m2 dθ m3 X3 X2 Module 2 X4 module 2 score module 3 score = log P(X1 Θ m1 ) P(Θ m1 ) dθ m1 + log P(X2,X3,X4 X1,Θ m2 ) P(Θ m2 ) dθ m2 + log P(X5,X6 X3,X4,Θ m3 ) P(Θ m3 ) dθ m3 X5 Module 3 X

false Goal false YGR043C true HAP4 true Identify modules (member genes) Discover module regulation program Regulation program genes (X s) experiments Candidate regulators X 1

.. -3 Context C X5 Module 3 X6 37 Learning Structure learning Find the structure that maximizes Bayesian score log P(S D).

19 false Goal false YGR043C true HAP4 true Identify modules (member genes) Discover module regulation program Regulation program genes (X s) experiments Candidate regulators X 1 Module 1 (μ A,σ A ) false X3 true X4 X3 X2 Module 2 X4 0 Context A fals e 3 Context B true Context C X5 Module 3 X6 37 Learning Structure learning Find the structure that maximizes Bayesian score log P(S D). Expectation Maximization (EM) algorithm M-step: Given a partition of the genes into modules, learn the best regulation program (tree CPD) for each module. E-step: Given the inferred regulatory programs, we reassign genes into modules such that the associated regulation program best predicts each gene s behavior

20 Learning regulatory network Iterative procedure Cluster genes into modules (E-step) Learn a regulatory program for each module (M-step) Candidate regulators Maximum increase in Bayesian score M1 M22 KEM1 MKT1 MSK1 DHH1 UTH1 ECM18 TEC1 GPA1 HAP4 ASG7 MFA1 MEC3 HAP1 SGS1 SEC59 RIM15 SAS5 PHO5 PHO2 PHO3 PHM6 SPL2 PHO84 PHO4 GIT1 VTC3 39 From expression to regulation (M-step) Combinatorial search over the space of trees Arrays sorted in original order HAP4 PHO5 PHO2 PHO3 PHM6 PHO84 PHO4 HAP4 GIT1 VTC3 Arrays sorted according to expression of HAP4 Segal et al. Nat Genet

21 From expression to regulation (M-step) HAP4 SIP4 PHO5 PHO2 PHO3 PHM6 PHO84 PHO4 GIT1 VTC3 SIP4 HAP4 41 Segal et al. Nat Genet 2003 Learning control programs Score: log P(M D) log P(D M,µ,σ )P(µ,σ) dµdσ 0 Score of HAP4 split: log P(M D) log P(D HAP4 M,µ,σ )P(µ,σ) dµdσ + log P(D HAP4 M,µ,σ )P(µ,σ) dµdσ HAP4 Segal et al. Nat Genet

22 Learning control programs HAP4 Split as long as the score improves HAP4 YGR043C Score of HAP4 split: log P(M D) 0 0 log P(D HAP4 M,µ,σ )P(µ,σ) dµdσ + log P(D HAP4 M,µ,σ )P(µ,σ) dµdσ Score of HAP4/YGR043C split: log P(M D) log P(D HAP4 M,µ,σ)P(µ,σ) dµdσ + log P(D HAP4 D YGR043C M,µ,σ)P(µ,σ) dµdσ 43 + log P(D HAP4 D YGR043C M,µ,σ)P(µ,σ) dµdσ Selecting candidate regulators Direct/Indirect transcriptional regulators Selected: Experimentally verified controllers (known) Putative controllers (by domain) Transcription factors, signaling molecules, receptors Yeast: ~8% genome 44 From Dr Segal s slides 22

Module network procedure Pre-processing Conditions Genes regulator selection M-step Module network procedure Motifs Modules Candidate regulators Regulation program learning Expression data clustering

23 Module network procedure Pre-processing Conditions Genes regulator selection M-step Module network procedure Motifs Modules Candidate regulators Regulation program learning Expression data clustering Gene partition data selection Gene reassignment to modules Functional modules E-step Graphic presentation GeneXPress.stanford.edu (TRANSFAC) Post-processing Annotations Experimental tests of computational predictions Annotations (GO, KEGG) 45 Segal et al. Nat Genet 2003 Yeast Stress Data (Gasch et al) Genes Selected 2355 genes that showed activity Experiments Diverse environmental stress conditions (heat shock, hydrogen peroxide, menadione, ) Time series for each condition Used all 173 arrays 46 From Dr Segal s slides 23

Module Evaluation Criteria Are the module genes functionally coherent? Do the regulators have regulatory roles in the predicted conditions?

24 Module Evaluation Criteria Are the module genes functionally coherent? Do the regulators have regulatory roles in the predicted conditions? Are the genes in the module known targets of the predicted regulators? Are the regulators consistent with the regulatory motifs found in upstream of the module genes? 47 Module functional coherence Coherence (%) Modules >60% Coherent 41 Modules >40% Coherent Metabolic: AA, respiration, glycolysis, galactose Stress: Oxidative stress, osmotic stress Cellular localization: Nucleas, ER Cellular processes: Cell cycle, sporulation, mating Molecular functions: Protein folding, RNA & DNA processing, trafficking 48 From Dr Segal s slides 24

25 Respiration module From Dr Segal s slides HAP4 known to up regulate Oxid. Phos. 49 Respiration Module From Dr Segal s slides HAP4 known to up regulate Oxid. Phos. HAP4, MSN4, XBP1 known to be regulators under predicted conditions 50 25

26 Respiration Module From Dr Segal s slides HAP4 known to up regulate Oxid. Phos. HAP4, MSN4, XBP1 known to be regulators under predicted conditions HAP4 Binding site found in 39/55 genes 51 Respiration Module From Dr Segal s slides HAP4 known to up regulate Oxid. Phos. HAP4, MSN4, XBP1 known to be regulators under predicted conditions HAP4 Binding site found in 39/55 genes MSN4 Binding site found in 28/55 genes 52 26

27 M: enrichment for motif known to participate in regulation by C: Respective regulator known to have a role under G: Respective regulator known to regulate module genes or their implied process respective regulator the predicted condition Complete module evaluation # Module a # G b C (%) c Reg. d MCGReg. d MCGReg. d MCGReg. d MC GMotif Signature e 1 Respiration and carbon regulation Hap4 Alpha2 Cmk1 Gac1 [HAP234, ADR1] [HAP234, STRE] Xbp1 Msn4 2 Energy, osmolarity and camp signaling Tpk1 Kin82 Yer184c Cmk1 [STRE, ADR1] [STRE, CAT8] Ppt1 Kns1 3 Energy and Osmotic stress I Xbp1 Kin82 Tpk1 [STRE, MIG1] [STRE, ADR1] 4 Energy and Osmotic stress II Ypl230w Yap6 Gac1 Wsc4 [REPCAR,ADR1] [REPCAR, STRE] [ADR1, CAT8] 5 Glycolysis and Folding Gcn20 Ecm22 Bmh1 Bas1 [GCR1, CBF1] [GCR1, STRE] 6 Galactose metabolism Gal4 Gac1 Hir3 Ime4 [GAL4, REPCAR] [GAL4, ADR1] 7 Snf kinase regulated processes Ypl230w Yap6 Tos8 Sip2 [N7, ADR1] [N7, STRE] [N7, GCR1] 8 Nitrogen catabolite repression Gat1 Plp2 [GATA, STRE] [GATA,CBF1] [GATA,GCN4] 9 AA metabolism I Gat1 Ime4 Cdc20 Slt2 [GCN4,CBF1] [GCN4,GATA] 10 AA metabolism II Xbp1 Hap4 Afr1 Uga3 [GCN4,STE12] [GCN4,STRE] Ppt1 11 AA and purine metabolism Gat1 Ppz2 Rim11 [N11,GCN4] [N11,STRE] 12 Nuclear Alpha2 Ino2 [N12,STRE] 13 Mixed I Pph3 Ras2 Tpk1 [N13,ADR1] 14 Ribosomal and phosphate metabolism Ppt1 Sip2 Cad1 [N13, LEU3] 15 mrna,rrna and trna processing Lsg1 Tpk2 Ppt1 [N13,REB1] [N13,ABF1] 16 RNA Processing and Cell Cycle Ypl230w Ime4 Ppt1 Tpk2 [N16, GCR1] Rho2 Mcm1 17 DNA and RNA processing Tpk1 Gis1 Ppt1 [N13, GCN4] 18 TFs and RNA processing Gis1 Pph3 Tpk2 Lsg1 [N18, N13] 19 TFs and nuclear transport Ypl230w Met18 Ppt1 [N19, N38] [N19, ABF1] 20 TFs I Cdc14 Mcm1 Ksp1 [N20,N48] 21 TFs II [N21, ADR1] 22 TFs, cell wall and mating Ptc3 Sps1 [MSE, CBF1] [MSE, MATA] 23 TFs and sporulation Rcs1 Ypl133c [N23, REB1] 24 Sporulation and TFs Gcn20 Gat1 Ste5 [N24,N46] 25 Sporulation and camp pathway Xbp1 Ypl230w Sip2 Not3 [ADR1, STRE] 26 Sporulation and Cell wall Ypl230w Yap6 Msn4 [N26, ADR1] [N26, STRE] 27 Cell wall and transport I Shp1 Bcy1 Gal80 Ime1 [N27,ADR1] [N27,STRE] Yak1 28 Cell wall and Transport II Ypl230w Kin82 Msn4 [STRE, MCM1] 29 Cell differentiation Ypl230w Ypk1 Cna1 [STE12, XBP1] 30 Cell cycle (G2/M) Cdc14 Clb1 Far1 [N30,MCM1] 31 Cell cycle, TFs and DNA metabolism Gis1 Ste5 Clb5 32 Cell cycle and general TFs Ime4 Ume1 Xbp1 Prr1 [N32,ABF1] Cnb1 Arp9 33 Mitochondrial and Signaling Tpk1 Cmk1 Yer184c Gis1 [N33,STRE] [N33, ADR1] [STRE, ADR1] 34 Mitochondrial and Protein fate Ypk1 Sds22 Rsc3 [N34, STRE] [N34, XBP1] 35 Trafficking and Mitochondrial Tpk1 Sds22 Mrf1' [N35, N30] 36 ER and Nuclear Gcn20 Yjl103c Not3 Tup1 [N36, ABF1] [N36, XBP1] 37 Proteasome and Endocytosis Ime4 Cup9 Bmh2 Hrt1 [N37, REB1] [N37, ABF1] [N37, ADR1] [N37, HSF] 38 Protein modification and trafficking Ypl230w Ptc3 Cdc42 [N38, N7] 39 Protein folding Bmh1 Bcy1 Ypl230w [HSF, XBP1] [HSF, ADR1] [HSF, GCN4] [HSF, CAT8] 40 Oxidative stress I Yap1 Sko1 Far1 [YAP1, STRE] [YAP1, HSF] [YAP1, XBP1] 41 Oxidative stress II Tos8 Flo8 [YAP1, STRE] 42 Unkown (sub-telomeric) Gcn20 43 Unknown genes I [STRE, PHO4] [STRE, STE12] 44 Unknown genes II Apg1 Pcl10 [N44, STRE] 45 Unknown genes III 39 5 Xbp1 Kar4 [N45, STRE] 46 Mixed II Gcn20 Tos8 Sip2 [N46, STRE] 47 Mixed III Gcn20 Ume1 Cnb1 [XBP1, HAC1] [XBP1, REB1] 48 Mixed IV Fkh1 Sho1 [N48, N34] 49 Ty ORFs Missing values [N50, STRE] 53 Segal et al. Nat Genet 2003 Results summarized Modules span a wide variety of processes including metabolic pathways, stress responses, cell-cycle related processes, molecular functions, and cellular compartments. 46/50 functionally coherent modules 30/50 modules included genes known to be regulated by the module s predicted regulators 15/50 modules had a match between the predicted regulator and its known motif (transcription binding sites) 54 From Dr Segal s slides 27

Inferring Transcriptional Regulatory Networks from High-throughput Data

Inferring Transcriptional Regulatory Networks from High-throughput Data Lectures 9 Oct 26, 2011 CSE 527 Computational Biology, Fall 2011 Instructor: Su-In Lee TA: Christopher Miles Monday & Wednesday 12:00-1:20