Information Transfer in Biological Systems

Transcription

1 Information Transfer in Biological Systems W. Szpankowski Department of Computer Science Purdue University W. Lafayette, IN May 19, 2009 AofA and IT logos Cergy Pontoise, 2009 Joint work with M. Aktulga, A. Grama, M. Koyuturk, I. Kontoyiannis, L. Lyznik, L. Szpankowski.

2 Outline 1. What is Information? Information in Biology Beyond Shannon Information 2. Information in Correlation: Finding Dependency Biological Signals Mutual Information Alternative Splicing 3. Information in Structures: Network Motifs Biological Networks Finding Biologically Significant Structures Structural Compression (Bonus)

3 What is Information? C. Shannon: These semantic aspects of communication are irrelevant... C. F. Von Weizsäcker: Information is only that which produces information (relativity). Information is only that which is understood (rationality) Information has no absolute meaning. F. Brooks, jr. (JACM, 2003, Three Great Challenges... ): We have no theory however that gives us a metric for the Information embodied in structure... this is the most fundamental gap in the theoretical underpinning of Information and computer science. P. Nurse, (Nature, 2008, Life, Logic, and Information ):... opines that the successes of system biology must be supplemented by deeper investigations into how living systems gather, process, store and use Information.

4 Information in Biology M. Eigen, Treatise on Matter, Information, Life and Thought : The differentiable characteristic of the living systems is Information. Information assures the controlled reproduction of all constituents, ensuring conservation of viability.... Information theory, pioneered by Claude Shannon, cannot answer this question... in principle, the answer was formulated 130 years ago by Charles Darwin. Some Fundamental Questions: How information is generated and transferred through underlying mechanisms of variation and selection (e.g., Darwin channel?); How information in biomolecules (sequences and structures) relates to the organization of the cell; What structure of a biological network significantly inform us of the presence of a phenotype (disease)? Whether there are error correcting mechanisms (codes) in biomolecules; How organisms survive and thrive in noisy environments. Life is a delicate interplay of energy, entropy, and information; essential functions of living beings correspond to the generation, consumption, processing, preservation, and duplication of information.

5 Beyond Shannon Participants of 2005&2008 Information Beyond Shannon workshop realize: Time: When information is transmitted over networks of gene regulation, protein interactions, immune response, the associated delay is an important factor. Space: In molecular interaction networks, spatially distributed components raise fundamental issues of limitations in information exchange since the available resources must be shared, allocated, and re-used. Information and Control: Information is exchanged in space and time for decision making, thus timeliness of information delivery along with reliability and complexity constitute the basic objective. Semantics: How to represent and measure biological Information that is context-dependent? Learnable Information: How much information can be extracted from a data repository? What information residing in living systems is meaningful? Limited Resources: Ability to communicate information is often limited by available resources (e.g., bandwidth of signaling channels). How much information can be extracted and processed with limited resources? Dynamic information: In a complex dynamic network Information is not only communicated but also processed and generated (human brain).

6 Science of Information

7 Outline Update 1. What is Information? 2. Information in Correlation: Finding Dependency Biological Signals Mutual Information Alternative Splicing 3. Information in Structures: Network Motifs

8 Biological Signals Biological world is highly stochasticand inhomogeneous (S. Salzberg). Physical limits within a cell are quite severe so it likely it has evolved to maximize information capacity (W. Bialek et. al.). Start codon codons Donor site CGCCATGCCCTTCTCCAACAGGTGAGTGAGC Transcription start Exon Promoter 5 UTR CCTCCCAGCCCTGCCCAG Acceptor site Intron Poly-A site Stop codon GATCCCCATGCCTGAGGGCCCCTC GGCAGAAACAATAAAACCAC 3 UTR

9 How to Measure Dependency: Theoretical Background Question: How to measure dependency between two parts of DNA (say whether introns error-controlled well-preserved exons)? Suppose we have two strings of unequal lengths X n 1 = X 1, X 2,..., X n Y M 1 = Y 1, Y 2, Y 3, , Y M, where M n, taking values in a common finite alphabet A such that: X 1 n Y 1 M I(X;Y) X n 1 are independent and identically distributed with distribution P(x); Y i are also i.i.d. with a possibly different distribution Q(y); W(y x) is a channel such that P x A P(x)W(y x) = Q(y) for all y. We wish to differentiate between the following two scenarios: (I) Independence: X n 1 and Y M 1 are independent. (II) Dependence. An index J {1, 2,..., M n+1} is chosen and Y J+n 1 J is generated as the output of the channel W with input X n 1.

10 Mutual Information To distinguish (I) and (II), we compute mutual information defined as I(X; Y ) = X x,y A V (x, y)log V (x, y) P(x)Q(y) where V (x, y) is joint distribution of X and Y. More precisely: we compute the empirical mutual information Î j (n) between X n j+n 1 1 and Yj defined as Î j (n) = X x,y A ˆp j (x, y)log ˆp j(x, y) ˆp(x)ˆq j (y). where ˆp j (x, y) is the joint empirical distribution of (X n 1, Y j+n 1 j ), ˆp(x) and ˆq j (y) are the empirical distributions of X n j+n 1 1 and Yj. We conclude: (I) Independence implies Î j (n) 0, (II) Dependence implies Î j (n)> 0.

11 Scenario (I): Independence The multivariate central limit theorem (for ˆp j ) shows that ni(n) converges in distribution to a (scaled) χ 2 chi-squared distribution, that is, (2 ln 2)nI(n) D Z χ 2 (( A 1) 2 ), where Z has a χ 2 distribution with k = ( A 1) 2 ) degrees of freedom. Therefore, P e,1 = Pr{declare dependence independent strings} = Pr{I(n) > θ independent strings} Pr{Z > (2 ln 2)θn}, hence P e,1 1 γ(k, (θ ln 2)n) Γ(k) exp (θ ln 2)n.

12 Scenario (II): Dependence Here, I(n) I with probability one at rate 1/ n, that is, n[i(n) I] D V N(0, σ 2 ), where the resulting variance σ 2 is given by, σ 2 = Var log Therefore: W(Y X) = X Q(Y ) x,y A P(x)W(y x) log W(y x) Q(y) I 2. P e,2 = Pr{declare independence W -dependent strings} = Pr{I(n) θ W -dependent strings} Pr{V [θ I] n} (I θ)2 exp n, 2σ 2

13 Discussion 1. The threshold θ needs to be strictly between 0 and I = I(X; Y ). 2. Both probabilities of error decay exponentially fast, but P e,1 exp( (θ ln 2) n) and P e,2 exp I θ 2σ 2n 3. To estimate P e,2 knowledge of the value of σ 2 is required, but practically impossible to compute. 4. (Practical Approach). Since in practice declaring false positives is much more undesirable than overlooking potential dependence, in our experiments we decide on an acceptably small false-positive probability ǫ, and then select θ such that P e,1 ǫ.

14 Alternative Splicing Dependency exists between alternative splicing locations? How to measure and discover them? Note: Only genes must produce 100,000 proteins.

15 Alternative Splicing in zmsrp32 Figure 1: Alternative splicings of the zmsrp32 gene in maize. The gene consists of a number of exons (shaded boxes) and introns (lines) flanked by the 5 and 3 untranslated regions, UTR (white boxes). The gene zmsrp32 is coded by 4735 nucleotides and has four alternative splicing variants. Two of these four variants are due to different splicings of this gene, between positions and

16 Results: Various Grouping Mutual Information Mutual Information Base position on zmsrp32 gene seq (a) Base position on zmsrp32 gene seq (b) Figure 2: Estimated mutual information between the exon located between bases and each contiguous subsequence of length 369 in the intron between bases The estimates were computed for the standard four-letter alphabet {A, C, G, T } (shown in (a)), and for the corresponding transformed sequences for the two-letter purine/pyrimidine grouping {AG, CT } (shown in (b)).

17 Results: Finding Dependency Mutual Information Mutual Information Base position on zmsrp32 gene seq Base position on zmsrp32 gene seq (a) (b) Mutual Information Mutual Information Base position on zmsrp32 gene seq Base position on zmsrp32 gene seq (c) (d) Figure 3: Î j versus j for various groupings of nucleotides]: In (a) and (c), mutual information between the exon found between 1 78 and the intron located between , over: (a) four letter-alphabet {A, C, G, T }; and (b) the Watson-Crick pairs {AT, CG}. In (b) and (d), mutual information between the [exon located between and the intron between , for the (b) four letter alphabet and the (d) two letter grouping of purine/pyrimidine {AG, CT }.

18 Conclusions There results strongly suggests that: there is significant dependence between the bases in positions and certain substrings of the bases in positions The region contains the 5 untranslated sequences (UTR), an intron, and the first protein coding exon, the sequence encodes an intron that undergoes alternative splicing We narrow down the mutual information calculations to the 5 untranslated region (5 UTR) in positions 1 78 and the 5 UTR intron in positions A close inspection of the resulting mutual information graphs indicates that the dependency is restricted to the alternative exons embedded into the intron sequences in positions and

19 Outline Update 1. What is Information? 2. Information in Correlation 3. Information in Structures: Network Motifs Biological Networks Finding Biologically Significant Structures

20 Protein Interaction Networks Molecular Interaction Networks: Graph theoretical abstraction for the organization of the cell Protein-protein interactions (PPI Network) Proteins signal to each other, form complexes to perform a particular function, transport each other in the cell... It is possible to detect interacting proteins through high-throughput screening, small scale experiments, and in silico predictions Protein Interaction Undirected Graph Model S. Cerevisiae PPI network hspace0.3in [Jeong et al., Nature, 2001]

21 Modularity in PPI Networks A functionally modular group of proteins (e.g. a protein complex) is likely to induce a dense subgraph Algorithmic approaches target identification of dense subgraphs An important problem: How do we define dense? Statistical approach: What is significantly dense? RNA Polymerase II Complex Corresponding induced subgraph

22 Significance of Dense Subgraphs A subnet of r proteins is said to be ρ-dense if the number of interactions, F(r), between these r proteins is ρr 2, that is, F(r) ρr 2 What is the expected size, R ρ, of the largest ρ-dense subgraph in a random graph? Any ρ-dense subgraph with larger size is statistically significant! Maximum clique is a special case of this problem (ρ = 1) G(n, p) model n proteins, each interaction occurs with probability p Simple enough to facilitate rigorous analysis Piecewise G(n, p) model Captures the basic characteristics of PPI networks Power-law model

23 Largest Dense Subgraph on G(n, p) Theorem 1 (Koyuturk, Grama, W.S., 2006). If G is a random graph with n nodes, where every edge exists with probability p, then lim n R ρ log n = 1 H p (ρ) (pr.), where H p (ρ) = ρlog ρ 1 ρ + (1 ρ)log p 1 p denotes divergence. More precisely, log n P(R ρ r 0 ) O n 1/H p(ρ) «, where for large n. r 0 = log n log log n + log H p(ρ) H p (ρ)

24 Proof X r,ρ : number of subgraphs of size r with density at least ρ X r,ρ = {U V (G) : U = r F(U) ρr 2 } Y r : number of edges induced by a set U of r vertices E[X r ] = `n r P(Yr ρr 2 ) P(Y r ρr 2 ) ` r 2 ρr 2 (r 2 ρr 2 )p ρr2 (1 p) r2 ρr 2 Upper bound: By the first moment method: P(R ρ r) P(X r,ρ 1) E[X r,ρ ] Plug in Stirling s formula for appropriate regimes Lower bound: To use the second moment method, we have to account for dependencies in terms of nodes and existing edges Use Stirling s formula, plug in continuous variables for range of dependencies

25 Piecewise G(n, p) Model Few proteins with many interacting partners, many proteins with few interacting partners p h > p b > p 1 Captures the basic characteristics of PPI networks, where p l < p b < p h. G(V, E), V = V h V l such that n h = V h V l = n l P(uv E(G)) = 8 < : p h p l If n h = O(1), then P(R ρ r 1 ) O V h V 1 if u, v V h if u, v V l p b if u V h, v V l or u V l, v V h. log n n 1/H p l (ρ) «where r 1 = log n log log n + 2n h log B + log H pl (ρ) log e + 1 H pl (ρ) and B = (p b (1 p l ))/p l + (1 p b ).

26 SIDES An algorithm for identification of Significantly Dense Subgraphs (SIDES) Based on Highly Connected Subgraphs algorithm (Hartuv & Shamir, 2000) Recursive min-cut partitioning heuristic We use statistical significance as stopping criterion p << 1 p << 1 p << 1

27 Behavior of Largest Dense Subgraph Across Species Observed Gnp model Piecewise model HS SC 12 Observed Gnp model Piecewise model SC Size of largest dense subgraph BT AT OS RNHP EC MM CE DM Size of largest dense subgraph BT AT OS RN HP EC MM CE HS DM Number of proteins (log-scale) Number of proteins (log-scale) ρ = 0.5 ρ = 1.0 Number of nodes vs. Size of largest dense subgraph for PPI networks belonging to 9 Eukaryotic species

28 Behavior of Largest Dense Subgraph w.r.t Density Size of largest dense subgraph Observed Gnp model Piecewise model Size of largest dense subgraph Observed Gnp model Piecewise model Density Density S. cerevisiae H. sapiens Density threshold vs. Size of largest dense subgraph for Yeast and Human PPI networks

29 Performance of SIDES Biological relevance of identified clusters is assessed with respect to Gene Ontology (GO) Find GO Terms that are significantly enriched in each cluster Quality of the clusters For GO Term t that is significantly enriched in cluster C, let T be the set of proteins associated with t PPI GO Cluster specificity = 100 C T / C sensitivity = 100 C T / T P 1 P 2 P 4 P 5 P 3 P 6 C T SIDES MCODE Min. Max. Avg. Min. Max. Avg. Specificity (%) Sensitivity (%) Comparison of SIDES with MCODE (Bader & Hogue, 2003) on yeast PPI network derived from DIP and BIND databases

30 Performance of SIDES Specificity (%) Sensitivity (%) Cluster size (number of proteins) SIDES MCODE Cluster size (number of proteins) SIDES MCODE Correlation SIDES: 0.22 SIDES: 0.27 MCODE: MCODE: 0.36

31 THANK YOU

32 Outline Update 1. What is Information? 2. Information in Correlation 3. Information in Structures: Network Motifs Biological Networks Finding Biologically Significant Structures Structural Compression

33 where the summation is over all distinct structures. Structural Information The (descriptive) entropy of a random (labeled) graph process G is H G = E[ log P(G)] = X G G P(G) log P(G), where P(G) is the probability of a graph G. A random structure model is defined for an unlabeled version. labeled graphs have the same structure. Some G 1 1 G 2 1 G 3 1 G 4 1 S 1 S G 5 1 G 6 1 G 7 1 G 8 1 S 3 S Random Graph Model Random Structure Model The probability of a structure S is: P(S) = N(S) P(G) N(S) is the number of different labeled graphs having the same structure. The entropy of a random structure S can be defined as H S = E[ log P(S)] = X S S P(S) log P(S),

34 Relationship between H G and H S Two labeled graphs G 1 and G 2 are called isomorphic if and only if there is a one-to-one map from V (G 1 ) onto V (G 2 ) which preserves the adjacency. Graph Automorphism: a For a graph G its automorphism is adjacency preserving permutation of vertices of G. b d c e The collection Aut(G) of all automorphism of G is called the automorphism group of G.

35 Relationship between H G and H S Two labeled graphs G 1 and G 2 are called isomorphic if and only if there is a one-to-one map from V (G 1 ) onto V (G 2 ) which preserves the adjacency. Graph Automorphism: a For a graph G its automorphism is adjacency preserving permutation of vertices of G. b d c e The collection Aut(G) of all automorphism of G is called the automorphism group of G. Lemma 1. If all isomorphic graphs have the same probability, then H S = H G log n! + X S S P(S) log Aut(S), where Aut(S) is the automorphism group of S. Proof idea: Using the fact that N(S) = n! Aut(S).

36 Erdös-Rényi Graph Model Our random structure model is the unlabeled version of the binomial random graph model known also as the Erdös and Rényi model. The binomial random graph model G(n, p) generates graphs with n vertices, where edges are chosen independently with probability p. If a graph G in G(n, p) has k edges, then P(G) = p k q (n 2) k, where q = 1 p. Theorem 2 (Y. Choi and W.S., 2008). For large n and all p satisfying ln n n and 1 p ln n n (i.e., the graph is connected w.h.p.), n H S = h(p) log n! + o(1), 2 p where h(p) = p log p (1 p) log (1 p) is the entropy rate. Thus H S = n 2 h(p) n log n + n log e 1 2 log n 1 2 log (2π) + o(1). Proof idea: 1. H S = H G log n! + P S S P(S) log Aut(S). 2. H G = `n 2 h(p) 3. P S S P(S) log Aut(S) = o(1) by asymmetry of G(n, p).

37 Compression Algorithm Compression Algorithm called Structural zip, in short SZIP Demo.

38 Compression Algorithm Compression Algorithm called Structural zip, in short SZIP Demo. We can prove the following estimate on the compression ratio of S(p, n) for our algorithm SZIP. Theorem 3 (Y. Choi and W.S., 2008). The total expected length of encoding produced by our algorithm SZIP for a structure S from S(n, p), is at most n 2 h(p) n log n + n (c + Φ(log n)) + O(n 1 η ), where h(p) = p log p (1 p)log (1 p), c is an explicitly computable constant, η is a positive constant, and Φ(x) is a fluctuating function with a small amplitude or zero. Our algorithm is asymptotically optimal up to the second largest term, and works quite fine in practise.

39 Experimental Results Real-world and random graphs. Real-world Random Table 1: The length of encodings (in bits) Networks # of # of our adjacency adjacency arithmetic nodes edges algorithm matrix list coding US Airports 332 2,126 8,118 54,946 38,268 12,991 Protein interaction (Yeast) 2,361 6,646 46,912 2,785, ,504 67,488 Collaboration (Geometry) 6,167 21, ,365 19,012, , ,811 Collaboration (Erdös) 6,935 11,857 62,617 24,043, , ,377 Genetic interaction (Human) 8,605 26, ,199 37,018, , ,569 Internet (AS level) 25,881 52, , ,900,140 1,572, ,060 S(n, p) 1,000 p = , ,500 99,900 40,350 S(n, p) 1,000 p = , , , ,392 S(n, p) 1,000 p = , ,500 2,997, ,252 `n2 2e log n `n2 h(p) n : number of vertices e : number of edges Adjacency matrix : `n 2 bits Adjacency list : 2e log n bits Arithmetic coding : `n 2 h(p) bits (compressing the adjacency matrix)