in Computer Simulations for Bioinformatics

Transcription

1 Bi04a_ Unit 04a: Stochastic Processes and their Applications in Computer Simulations for Bioinformatics Stochastic Processes and their Applications in Computer Simulations for Bioinformatics Basic Probability Concepts revisited Crude Monte Carlo Markov-Chain Monte Carlo a) Metropolis-Hastings b) Gibbs-Sampling Real Example: How to find the Consensus Sequence (Unit 4c, Start) Simulated Annealing using Markov-Chain-Monte Carlo Real Example: How to find the Consensus Sequence (Unit 4c, ctd) Genetic Algorithms a) General applications used to optimize structure of multidimensional objects: molecule conformations, sequence alignments, etc. Real example: Multiple sequence alignment by GA Program SAGA b) Genetic Computing: Genetic Algorithms used to optimize computer programs Bi04a_2

2 Basic Probability Concepts revisited Bi04a_3 Discrete distributions absolute frequencies relative frequencies limiting behaviours as N probabilites & cumulative probabilities Continuous distributions absolute frequencies relative frequencies limiting behaviours as N probabilites & cumulative probabilities Basic Concepts revisited Bi04a_4. Discrete Distributions random variable takes only a few possible values example: ξ {, 2, 3, 4, 5, 6} for (normal) dice N-times throwing a dice: {, 2, 3, 4, 5, 6, } 6 i = i= H H H H H H absolute Frequences H 3 -times dice shows 3 H N Summation Condition for absolute Frequences 2 6 h h2 h6 relative frequences 6 i= H H H =, =,... = N N N h = i Normalization Condition for relative Frequences " irgendein Wert muß ja schließlich auftreten..." 2

3 Basic Concepts revisited Bi04a_5 Limiting behaviour as N {h,...h 6 } {p...p 6 } Hi p Probability is limit of i = lim = lim hi N N N relative frequencies as N 6 6 hi i= i= = p = i Normalization condition to Generalize: a) Consider an arbitrary number of values (L instead of 6) b) Consider unequal probabilities p i p j for i j L L hi i= i= = p = i Basic Concepts revisited Bi04a_6 Example for limiting behaviour of relative frequencies as N {h, h 2,...h 6 } N {p, p 2,...p 6 } small N: {h, h 2,...h 6 } we obtain very different results for each lap large N: {h, h 2,...h 6 } we obtain more similar results for each lap 3

4 Basic Concepts revisited Bi04a_7 p(i) = Pr(ξ=i) probability distribution F(i) = Pr(ξ i) () p() i F i = i l= cumulative probability Basic Concepts revisited Bi04a_8 A simple example: normal dice Mafia dice p F / p F /6 L l= () = () = p() l F i p l i l=

5 Basic Concepts revisited Bi04a_9 2. Continuous Distributions random variable takes arbitrary (real) values within given intervall ex.: ξ, 0 ξ finite intervals -2 ξ 5 0 ξ infinite intervals - ξ Basic Concepts revisited Bi04a_0 Continuous Distributions, ctd. p(x) uniform distribution x x+dx p(x) is defined such that Pr(x ξ x + dx) = p(x)dx dx... infinitesimally small interval shaded area ( ) b Pr a ξ b = p ( ξ ) dξ a ξ Since the random variable can take any (in between) value, the concept of discrete probability values must be generalized to a continuous probability distribution function ( ) ( ) Pr ξ = p ξ dξ = [a,b]... real intervall Normalization for probability density function (p.d.f.) irgendwo muß ξ ja liegen 5

6 Basic Concepts revisited Bi04a_ Continuous Distributions, ctd. specifically consider interval [-,x] p p(x) F(x) if this holds, we know from calculus: x P(- ξ x ) = x ξ ( ξ) ξ = ( ) = Pr ( ξ ) p d F x x ( ) F( x) p x = d dx F(- ) = 0 F(+ ) = F(x) is increasing since p(x) 0 probability density function (p.d.f.) cumulative distribution function Basic Concepts revisited Bi04a_2 Example: linearly increasing p.d.f. on [0,]: p(x) = 2x p(x) F(x) Normalization: Cumulative distribution function: 2 2x p( x) dx= = 0= x 2 x 2 p( ξ) dξ = ξ = x 0 0 6

7 Basic Concepts revisited Bi04a_3 from Kalos (986) Basic Concepts revisited Bi04a_4 how to get a random variable ξ? physically: throw 0 sided prism to construct real number as 0.n n 2 n 3 n 4...n L [0,) from Morgan (984) on a computer: use random number generator! (see expendix) 7

8 Expendix : How to generate random numbers Bi04a_5 Real random numbers (from nature, i.e. physical process) from Morgan (984) Pseudo random numbers (PRNs) (from computer programs) x n+ =a x n +b (mod m) congruential method e.g. a = 573, b = 9, m = 000, x 0 =89 x ξ = n n [0,] m seed uniform distributed Really random versus Pseudo random. Bi04a_6 Real random numbers (from nature, i.e. physical process) no prediction possible randomness (of e.g. dice) due to uncertaincy principle of quantum physics statistical tests will not reveal systematic predictability awkward for simulation Pseudo random numbers (PRNs) (from computer programs) strictly predictable randomness due to algorithm, its parameters & seed statistical tests will not easily reveal systematic predictability however: beware of loops handy for simulation 8

9 Expendix : PRNs Quality Bi04a_7 Bad Generator from Morgan (984) Improved Generator Expendix : PRNs from most common distributions Bi04a_8 ξ [0,] uniformly distributed is available in (almost) every programing language (library) ξ N(µ,σ) normal (Gaussian) distribution mean standard deviation is available in many programing language (libraries) from Kalos (986) from Kalos (986) 9

10 Expendix : Ho to get PRNs from other distributions Bi04a_9 statistical and mathematical program libraries deriving desired distribution from uniform PRNs on [0,]: inversion method other, even more sophisticated methods... rejection method (works for arbitrary distributions wanted!) table lookup-method Bi04a_20 Expendix : Linear transformations of PRNs uniform PRNs uniform PRNs Gaussian PRNs Gaussian PRNs arbitrary distributed PRNs (? not generally predictable) 0 N 0 - trafo 0 trafo + ξ N. ξ [0,N] CAUTION:! what happens at start & end of interval! ξ 2 ξ- [-,+] 0 A B- trafo ξ A+(B-A) ξ [A,B] 0 0

11 Expendix : General transformations of PRNs Bi04a_2 from Morgan (984) consider cumulative distribution function for argumentation And for details: see Morgan (984), p.29 or Kalos (986), p.40 y ( ) = Pr ( ) = ( ) = ( ) F y Y y X x F X ( ) dfx x dx fy ( y) = dx dy dx fy( y) = fx( x) dy dy = f ( x) dx x d dy Apply differentiatiation operator to both sides of equation! Use chain-rule for differentiation taking absolute values will make formula valid for decreasing and increasing transformation functions! Expendix : General transformations of PRNs Bi04a_22 An illustration of the transformation y = x relating the densities f X (x) = e -x, f Y (y) = 2ye -y2. The shaded regions have equal areas. from Morgan (984)

12 Expendix : Random numbers from libraries Bi04a_23 SAS NAG IMSL NORMAL generates a normally distributed G05CAF uniform over (0,) nextbeta beta distribution pseudo-random variate RANBIN generates an observation from a G05DAF uniform over (a,b) nextbinomial binomial distribution binomial distribution RANCAU generates a Cauchy deviate G05DBF exponential nextcauchy Cauchy distribution RANEXP generates an exponential deviate G05DDF Normal nextchisquared Chi-squared distribution RANGAM generates an observation from a gamma distribution G05DYF discrete uniform nextexponential standard exponential distribution RANNOR generates a normal deviate G05DRF Poisson nextexponentialmix mixture of two exponential distributions RANPOI generates an observation from a Poisson distribution G05FEF Beta distribution (multiple) nextgamma standard gamma distribution RANTBL generates deviates from a tabled G05DFF Cauchy distribution nextgeometric geometric distribution probability mass function RANTRI generates an observation from a triangular distribution G05DHF Chi-square distribution nexthypergeometric hypergeometric distribution RANUNI generates a uniform deviate G05DKF F-distribution nextlogarithmic logarithmic distribution UNIFORM generates a pseudo-random variate G05FFF Gamma distribution (multiple) nextlognormal lognormal distribution uniformly distributed on the interval (0,) G05DCF Logistic distribution nextmultivariatenormal multivariate normal distribution G05DEF Lognormal distribution nextnegativebinomial negative binomial distribution G05DJF Student's t-distribution nextnormal standard normal distribution using an inverse CDF method G05FSF von Mises distribution nextnormalar standard normal distribution using an acceptance/rejection method G05DPF Weibull distribution nextpoisson Poisson distribution nextstudentst Student's t distribution next Triangular triangular distribution nextvonmises von Mises distribution nextweibull Weibull distribution Expendix : Rejection method Bi04a_24 p wanted distribution (for some peculiar reason...) a recipe: generate ξ [a,b], uniform generate η [0, max(p)], uniform b ξ Rejection method is simple! works for the most fancy distributions normalizes itself! if η<p(ξ) take ξ as next random number ( Points) otherwise discard (ξ, η) ( Points) 2

13 Expendix : Rejection method, ctd. Bi04a_25 rejection method is simple but: may be inefficient: Simple but dead slow! Expendix : Select randomly from a group of n individuals according to weights (probabilities) ( table lookup method for a discrete random variable ) Bi04a_26 ) Case : equal probabilities (=weights) ξ individual # 2 3 n equidistant intervals 0 /n 2/n 3/n (n-)/n n/n = recipe: draw pseudo random number ξ from uniform distribution over [0,] ξ individual selected: i = int +, = min (, ) / i i n n 3

14 Expendix : Select randomly from a group of n individuals according to weights (probabilities) ( table lookup method for a discrete random variable ) Bi04a_27 2) Case 2: different probabilities (=weights), e.g. p i proportional e -E(s i )/τ ξ individual # n equidistant intervals 0 n ( )/ n Esi τ Es ( i) / τ pi = e e Normalization : p i= i= i= recipe: draw pseudo random number ξ from uniform distribution over [0,] individual i selected so that: i p < ξ l l= l= i p l Literature on Random Numbers Bi04a_28 Morgan,B.J.T Elements of Simulation. Chapman and Hall, New York. Rosanow,J.A Wahrscheinlichkeitstheorie. Rowohlt Taschenbuch Verlag, Hamburg. Kalos,M.H. and P.A.Whitlock 986. Monte Carlo methods. Wiley, New York. SAS Institute SAS User's Guide: Basics. Cary. NAg Fortran Library Mark 6. Oxford. JMSL Reference Manual Visual Numerics, San Ramon. 4