Bayesian Scientific Computing D. Calvetti & E. Sommersalo

Transcription

1 Bayesian Scientific Computing D. Calvetti & E. Sommersalo Quinn Winters Week : -7 January 8 Lecture - Inverse Problems and Subjective Computing. Example.7: Assume every day except sundays a train for your destination leaves every S minutes from the station. On Sundays, the train leaves every S minutes from the station. You arrive at the station with no information about the train s time table. What is your expected weighting time? Let T be the waiting time. T π(t working day χ S S(t and T π(t sunday χ S S(t. Therefore one obtains the formula: E[T ] t(π(t working dayχ working day + π(t sundayχ sunday dp Ω E[T working day]p (working day + E[T sunday]p (sunday 6 ( 7 S tχ [,S](t dt + ( 7 S tχ [,S](t dt 6 7S R S 6 7S S S t dt + / t dt 7S + / 4S 7S 3S 7 + S 7 4S 7 (Consider the following modification to Example.7: Instead of every S minutes the train leaves the station every day at the same, unevenly distributed times t j, j n. Assume that you have the train time table, but no watch, nor any idea what time of day it is. What is the expected waiting time in this case? R

2 Solution If T is the time the train leaves each time, then for any given interval [t i, t i+ ] we obtain a density formula for T just like before given by T in [t i, t i+ ] π(t t [t i, t i+ ] n t i+ t i χ [ti,t i+ ](t t i+ t i Therefore, we can see that T has density χ [ti,t t i i+ t i+ ], since i t n t probability of landing in an interval is assumed to be uniformly distributed and therefore t i+ t i t n t is P (t [t i, t i+ ]. This can be reduced to show that the density function of T is given by: n i t i+ t i n χ [ti,t t i+ t i+ ] i t n t i Thus, to obtain expected waiting time, integrate to get: ( n E[T ] t χ [ti,t t n t i+ ] dt R t n t t n t i n ti+ i n i t i t n t t n t t n t t dt t i+ t i t n t χ [ti,t i+ ](t ( t t + t 3 t t n t n + t n t n (t n + t (t n t (t n t t n + t

3 . Let A (X, Y R where X, Y N (, σ. Then, R A is said to be Rayleigh Distributed. Derive an analytic expression of the Rayleigh distribution and write a program that generates points from the Rayleigh distribution and plots a historgram. Solution One can see that calculating the Rayleigh distribution s density function can easily be obtaine by calculating d P (R t where R is Rayleigh distributed. Notice dt that {R t} is equivalent to asking if X + Y t, which is equivalent to asking if a multi-variate normal is contained in the dialated unit disk td. Therefore: P (R t d P (R t dt d ( dt πσ d ( dt π td π t t /σ e t σ e (x +y /σ da re r /σ dr dθ To obtain the distribution function for the Rayleigh distribution, one simply integrates the density function. Recall that Therefore: P (R x x ( d /σ e t dt t /σ e t σ t /σ e t σ ( dt e x /σ e t /σ To obtain a program that draws points from the Rayleigh distribution consider the following program written in python from numpy. l i n a l g import norm from numpy. m a t l i b import randn 3 import m a t p l o t l i b. p y p l o t as p l t 4 5 d e f r a y l e i g h ( sigma : 6 r e t u r n norm ( sigma randn (, 7 3

4 8 d e f h i s t o g r a m r a y l e i g h p o i n t s ( n p o i n t s [,,, ], sigma [.5,, 5, ] : 9 # i n t i a l i z e s t o r a g e and c r e a t e random draws samples d i c t. fromkeys ( sigma f o r k i n samples : samples [ k ] d i c t. fromkeys ( n p o i n t s 3 f o r s i z e i n n p o i n t s : 4 f o r sg i n sigma : 5 samples [ sg ] [ s i z e ] [ r a y l e i g h ( sg f o r i i n range ( s i z e ] 6 # make p l o t s and s a v e them as a c o l l e c t i v e p l o t 7 p l t. c l o s e ( a l l 8 f, a x e s p l t. s u b p l o t s ( l e n ( n p o i n t s, l e n ( sigma 9 a x e sa x e s. r e s h a p e ( l e n ( n p o i n t s, l e n ( sigma f o r i i n range ( l e n ( n p o i n t s : f o r j i n range ( l e n ( sigma : sz, sg n p o i n t s [ i ], sigma [ j ] 3 a x e s [ i, j ]. h i s t ( samples [ sg ] [ s z ], auto 4 a x e s [ i, j ]. s e t t i t l e ( {} p t s o f R a y l e i g h ({}. format ( sz, sg 5 f. s u b p l o t s a d j u s t ( hspace.7 6 p l t. show ( h i s t o g r a m r a y l e i g h p o i n t s ( n p o i n t s [], sigma [.,,, ] The results of which are: 4

5 3. Calculate the third and fourth moments of a Guassian density. Solution One recalls that a random variable that has Guassian density with mean µ and varience σ is written as X N (µ, σ and has density given by f(x σ π exp ( (x µ σ Compute the n-th (centered-guassian moment (i.e. X N (, σ by E[X n ] σ π xn exp ( x dx σ ( xσt σ π σn t n exp t σ dt σn t n e t / dt π Now all that remains is to compute for n 3, 4. Notice that: ( t 3 e ( t / t 3 e t / So by symmetry, for n 3 the integral vanishes giving us E[X 3 ]. Now suppose that n 4, and compute via integration by parts the following: σ 4 ( t 4 e t / dt σ4 s 3 π π 3t e t / dt 3 e t / 3σ4 t e t / dt π (te 3σ4 t / π 3σ4 π π 3σ 4 e t / dt The justification that e t / dt π is a classical argument which goes 5

6 like: ( e dt t / e (t +s / da R π re r / dθ dr π re r / dr πe r / lim n π( e n / π To complete the arugment, just take square roots. Moreover, one can see that in general the n-th moment of a centered Guassian can be computed via integration by parts applied iteratively. Namely, we have already shown a trivial inductive case and noting that t (n+ e t / dt One can see that t n+ te t / dt tn+ e t / }{{} (n + t n e t / dt e tn / (n + t n dt t n e t / dt (n!!, which gives the closed form for the n-th centered Gaussian moment as { E[X n if n is odd ] σ n (n!! if n is even Now, by the Binomial Theorem, we can compute for Y N (µ, σ by noting 6

7 that Y d X + µ, so E[Y n ] E[(X + µ n ] E n/ j [ n k ( ] n µ n k X k k n k n/ n! (j!(n j! µn j σ j (j!! which gives us a general formula. k n! k!(n k! µn k E[X k ] µ n j σ j n! (n j!(j!! 7