ψ(t) := log E[e tx ] (1) P X j x e Nψ (x) j=1

Transcription

1 Seminars 15/01/018 Ex 1. Exponential estimate Let X be a real valued random variable, and suppose that there exists r > 0 such that E[e rx ] <. For t [0, r define the function 1a Prove that ψ is convex, namely that for α [0, 1] ψt := log E[e tx ] 1 ψ1 αt + αs 1 αψt + αψs 1b Let ψ be the convex conjugate or Legendre transform of ψ, namely ψ x = sup t x ψt, t [0,r x R Prove that ψ 0 and that, whenever E[X] >, ψx = 0 for x E[X]. 1c Prove that for all x R PX x e ψ x Deduce that if X 1,..., X N are i.i.d. random variables and for each of them ψ is defined as in 1, then N P X j x e Nψ x 1 N j=1 { } 1d Let r := sup r 0 : E[e r X ] < +. Prove that ψ extends to an analytic function in the complex disk { z < r}. 1e Give an example of a random variable with r > 0 and such that lim t r ψt = +, and one example where lim t r ψt < +. Ex. In a Math department there are two lecturers teaching Stochastic Processes. One lecturer already knows that he has n students in his class. The other one however missed his first class, so he still considers the number N of his students as a random variable with expectation n. At the end of the semester, an exam will take place and each student has a probability p 0, 1 of passing the exam independently of the other students, and of the total number of students attending the classes. Which class has the higher expectation of students passing the exam? Which class has the higher variance? Ex 3. A man has r roubles, and he plays a game many times betting 1 rouble each time. He wins thus earning one extra rouble with probability p 0, 1, and loses with probability 1 p thus losing the rouble he bet. He stops once he is ruined 0 roubles remaining, or once he has enough money to afford a fancy yacht, which costs R > r roubles. 3a Find the probability that he gets the yacht as a function of n and p. After some time, he is broke 0 roubles. However he finds out that his infinitely rich uncle will borrow him money indefinitely, and he continues to play indefinitely a real gambling addict. Let now p n be the probability that the man starting at time 0 with 0 roubles and using his uncle s money, will have again 0 roubles at some point before the n-th bet. We want to prove that { 1 if p = 1/. lim p n = 3 n < 1 if p 1/. Let X k be the amount of roubles after k bets, and u n := PX n = 0, v n := PX 1 0, X 0,... X n 1 0, X n = 0. 3b Find u n explicitely, and prove that p n = k n v k, and u n = k n u n kv k. Recall that, given two sequences of positive real numbers a n and b n with a 0 = 1 and such that a n = k b ka n k, it holds b k 1 a k = 3c Use Stirling s formula to deduce 3. k 0 Ex 4. The random variables Y and Y are equal a.s.. Prove that E[X Y ] = E[X Y ] a.s.. Ex 5. Let µ and ν be positive, σ-finite measures on the measurable spaces E and F. Let X and Y be random variables taking values in E and F respectively, and such that the law θ PE F of X, Y admits a density w.r.t. µ ν, namely k 0 dθx, y = ϱx, ydµxdνy Given a bounded measurable function f : E E R, write E[fX, Y Y ] in terms of ϱ, µ and f. Ex 6. Fourier Transform on Banach spaces Let E be a real separable Banach space equipped with the norm and the associated Borel σ-algebra, and let E be its dual. For µ PE, define the characteristic function of µ as ϕ µ : E C ϕ µy := dµxe i x,y E 1

2 In this exercise, we want to prove that pretty much like in the case E = R d, the characteristic function characterizes. Recall that if E is separable, then it admits a countable norming set, namely there exists a countable {y 1, y,...} E, such that the y i have dual norm 1 and sup x, y i = x 4 i N 6a For {y 1, y,...} as in 4, check that the Borel σ-algebra of E is generated by sets of the form {x E : x, y 1 A 1,..., x, y n A n} 5 where n runs over N and A 1,..., A n run over Borelian sets of R. 6b Recall that, if two probability measures coincide on a π-system 1, they coincide on the σ-algebra generated by such a π-system. Using 3a, prove that, for µ, ν PE, if ϕ µ = ϕ ν, then µ = ν. Ex 7. Gaussian random variables Recall that a real-valued random variable X is called a symmetric Gaussian random variable if E[e itx ] = e Qt / for some Q 0. If E is a separable real Banach space, we say that an E-valued random variable X is symmetric Gaussian, if for every y E, the real-valued r.v. X, y is symmetric Gaussian. Hereafter in this exercise, E is a separable Banach space. We want to prove that if X is an E-valued Gaussian r.v., then the exists a q > 0 such that E[e q X ] < +. 7a Let E be a separable real Banach space, and X and Y be E-valued, independent, identically distributed symmetric Gaussian r.v.. Prove that X + Y / and X Y / are independent symmetric Gaussian r.v., still with the same distribution. 7b For E and X as above, prove that for every a, b 0 it holds P X ap X b P X b a 7c Fix a 0 > 0 such that PX a 0 1/3, and set for n 1 a n = a 0 + a n 1 = a 0 n/ 1 1 a 0 n/ 6 Prove that P X a n P X a n 0 7d Conclude that there exists q > 0 such that E[e q X ] <. Ex 8. Pushforward Let E be a measurable space and let PE be the set of probability measures on E. 8a For µ, ν PE and α [0, 1], check that αµ + 1 αν PE. This corresponds to sampling according to µ or ν on E depending on the result of tossing a coin, that is sample with µ with probaiblity α and ν with probability 1 α. This is called a convex combination of µ and ν. A convex combination αµ + 1 αν is trivial if it coincides with either µ or ν, in other words if α = 0 or α = 1 or µ = ν. An element in PE is extremal if it cannot be expressed as a non-trivial convex combination. 8b Characterize the extremal elements of PE. Check that if E is a metric or metrizable space equipped with the Borel σ-algebra, then µ PE is extremal iff it is a Dirac mass, µ = δ x for some x E, where δ x is the probability measure defined as δ xa = 1 Ax, A measurable Suppose that E is a good topological spaces, say it is a compact metric space 3. We can endow PE with the narrow topology, which is the weakest topology such that the map PE µ fxdµx is continuous for all continuous functions f CE. 8c Check that the immersion map ı: E PE defined as ıx = δ x is continuous. Suppose now that E and E are good topological spaces as above and equip PE and PE with the narrow topology. If π : E E is measurable, we can lift π to a map T π : PE PE as T πµ = µ π 1, namely T πµb = µπ 1 B for all Borelian B E. This is called the pushforward of µ via π. 8d Suppose that π CE, E. Check that ı T π = π ı, where ı and ı are defined as in 8c. T παµ + 1 αν = αt πµ + 1 αt πν. T π is continuous from PE to PE. 8e Suppose π CE, E and that a map U : PE PE satisfies the same conditions as in 4d. Prove that U = T π. 1 A collection of subsets of E is a π system if it contains and is closed under finite intersection. Hint: Prove that for X and Y as in 3a, it holds P X ap Y b P X a b P Y b a. 3 We will be interested in E = R d or E = C[0, ; R d in the class. These are not compact, but only minor modifications are needed.

3 Ex 9. Let X X t t 0 be a real-valued stochastic process, and let τ be a real, positive random variable. Define the process Y by Y t = X t + 1 τ=t. Check that PX = Y = 0. When does it hold PX t = Y t = 0 for all t 0? Sol 1. Notice that E[e rx ] < only entails a bound for the positive part of X. 1a By the Cauchy-Schwarz inequality e ψ1 αt+αs = E [ e 1 αt+αsx] E [ e 1 αtx 1 1 α ] 1 α E [ e αsx 1 α ] α = e 1 αψt+αψs 1b Since ψ0 = 0, one gets the inequality ψ 0 taking t = 0 in the supremum in. On the other hand, by Jensen inequality ψt = log E[e tx ] log e te[x] = te[x] This means that for x E[X], one has t x ψt 0. Therefore, for such an x, the supremum in is achieved for t = 0. 1c For all t [0, r it holds PX x Pe tx e tx e tx E[e tx ] = e tx ψt Optimize over t namely take the inf in the r.h.s., to get the result. It is then easy to check that ψ N t := log E[e tx X N /N ] = Nψt/N, so that ψ N t = Nψ t. 1d Assume r > 0 or there is nothing to prove. Since x n r n n! e r x, we have that E[ X n ] is finite, and for any r 0, r one has Therefore the power series m n := E[X n ] r n n! E[e r X ] φz := m n n! z n is dominated by a geometric series and converges uniformly in any ball { z R} of radius R < r. Thus φ is analytic in { z < r}. By dominated and monotone convergence, φ coincides with ψ on the real segment 0, r. 1e For the first example, consider an exponential random variable of parameter λ > 0. One has r = λ and E[e tx ] = λe R t λx dx = λ + as t λ. + t λ For the second example, consider a random variable with density ϱx = 1 Z constant. Then r = λ but for t < λ E[e tx 1 ] dx < R + Z1+x so that lim t r ψt exists ψ is increasing in this case and is finite 4. e λx 1+x 1 x 0, where Z is a suitable Sol. Let Y 1,... Y n {0, 1} be the results of the students in the first class 1 means they passed the exam, and let X 1,..., X N the the results for the second class. Then Y = n Y i and X = N X i are the numbers of students that passed the exams. Then by linearity E[Y ] = n p and by independence E[Y ] = n p + nn 1p. On the other hand E[X] = E[X N = k]pn = k = k p PN = k = E[N] p = n p = E[Y ] Similarly E[X ] = = E[X N = k]pn = k k p + kk 1p PN = k = E[N] p1 p + E[N ]p = Var[Y ] + E[N ]p Therefore Var[X] = Var[Y ] + p Var[N] Var[Y ]. Sol 3. Let s call X n the random variables corresponding to the amount of money negative if he got a debt that the man has after gambling n times. 4 In order to produce these examples, somebody suggested to take a function F on C with a singularity at z = r R, and then to calculate the Laplace transform of e f to get the law of a random variable: this will not work since in general, the Laplace transform of a given function will not produce a probability mesure.

4 3a Let s call p r the probability that the man gets the yacht if he starts with n roubles. Then, by conditioning on the result of the first bet we gather p 0 = 0 This system admits the unique solution p R = 1 p r = p p r pp r 1 p r = r R if p = 1/. p r = 1 1 p/pr 1 1 p/p R if p 1/. 3b X n = 0 iff the man wins exactly n times and loses n times in the first n bets. Thus u n = C n np n 1 p n 7 The other formulas follow from the fact that the man can be in 0 only after an even number of steps, and that the probability of being at 0 after n bets is the sum over k of the probabilities of getting back to 0 for the first time after k bets, and then earning 0 roubles in the remaining n k bets. 3c Recall that n! = πnn n e n exp θn where θn 0, 1. Then from 7 we have for some constant cn 1/, 1n 4πnn n e n exp θ n 4n u n = πn n n e n exp θn p n 1 p n = cn πn 4p1 p n 6n Notice that 4p1 p 0, 1], and it equals 1 iff p = 1/. Therefore the series n un diverges for p = 1/ and converges otherwise. Sol 4. This may look obvious, but sets of measure 0 are known to be tricky! So let s give all the details. Let G, G be the σ-algebras generated by Y and Y respectively. The smallest σ-algebra G G containing both G and G. Z := E[X Y ] and Z := E[X Y ] are respectively G- and G -measurable random variables, but they both are F- measurable, so it makes sense to ask whether Z = Z a.s.. Since they are both G G -measurable, it is enough to check that E[1 AZ] = E[1 AZ ] 8 for every A G G. Since this σ-algebra is generated by Y, Y, it follows that every it is enough to check 8 for such A of the form A = {Y I, Y I } for measurable I, I. Setting B = {Y I I } and C = {Y I I } we have that PA B = PA C = 0. Therefore E[1 AZ] PA B=0 = E[1 BE[X G]] B G = E[E[1 B X G]] = E[1 B X] =E[1 C X ] = E[E[1 C X G ]] C G = E[1 CE[X G ]] PA C=0 = E[1 AZ ] Sol 5. By the definition of conditional expectation, we need to find a measurable map a.e. equivalence q : F R such that gyfx, ydθx, y = gyqydθe, y E F for all bounded measurable g. It follows Sol 6. 6a By 4, we have that E[fX, Y Y ] = qy = E F fx, yϱx, ydµx ϱx, ydµx {x E, x 1} = n {x E : x, y 1 [ 1, 1],..., x, y n [ 1, 1]} so the closed ball of radius 1 is in the σ-algebra generated by sets of the type 5. By translations and homotheties, all the closed balls are in this σ-algebra, and thus it coincides with the Borel σ-algebra. 6b Let µ, ν PE be such that ϕ µ = ϕ ν. Then for every y 1,..., y n E and t 1,..., t n R we have that n dµx exp i t j x, y j = ϕ µ t jy j = ϕ n ν t jy j = dµx exp i t j x, y j j=1 j j j=1 Therefore, since on R n we know that a probability is characterized by its characteristic function, µ and ν coincide on sets of the form 5. Since this is a π-system, they coincide on the Borel σ-algebra by 6a. Sol 7. 7a By Exercise6, it is enough to prove that the characteristic functions coincide. And indeed E[e itx+y / e isx Y / ] = E[e it+sx/ e it sy / ] = e Q t+s / e Q t s / = e Q t +s = E[e itx e isy ] E

5 7b From point 7a, assuming b a X+Y P X a P X b = P P X b a, Y b a = P a where from the first to the second line we used X = 1 X+Y P X Y b X b a P + X Y 1 X+Y = P X+Y a, X Y Y b a = P X Y b X b a and similarly for Y. 7c Let u n = P X a n/p X a 0. Then since P X a 0 /3 and from the 7b applied with a = a 0 and b = a n+1 so that b a = a n u 0 1/3 /3 = 1 u n+1 = P X an+1 P X a 0 P X an P X a 0 P X a 0 which entails u n n. 7d The estimate in 7c implies that the probability that X is large decays fast as fast as for a real-valued Gaussian. One can conclude as follows, recalling 6: E[e q X ] e qa 0 P X a 0 + e qa n+1 P X a n P X a 0 e qa 0 + e q a 0 n/ n n 0 n 0 = u n e qa 0 + n 0 e 4 q a 0 logn Sol 8. which converges provided q < log/4a 0. 8a This is immediate as αµe + 1 ανe = 1, and also σ-additivity follows easily. 8b We claim that a probability µ PE is extremal iff µa {0, 1} for every measurable A. Indeed, if there exists A with µa 0, 1, then we can write µ = αν αν 0 where α = µa, while the probabilities ν 0, ν 1 are defined by ν 1B = µb A and ν 0B = µb A c. Since ν 1A = 1 0 = ν 0A and α 0, 1, the convex combination is non-trivial. Conversely, if µ can be written as a non-trivial convex combination µ = αν αν 0, then there exists a measurable A such that ν 1A ν 0A, and since α 0, 1 it follows µa 0, 1. Now, if E is metrizable and µ is extremal, any ball Bx, r has either measure 1 or 0. First notice that there exists at least one x E such that µbx, r > 0 for every r > 0, otherwise µ would vanish on every open set and it would be the null measure not a probability. On the other hand, by additivity there a unique x with such a property if y has also this property, it must happen y Bx, r for every r > 0, so that x = y. Therefore by continuity on monotone sequences µ{x} = lim r 0 µbx, r = 1 and thus µ = δ x. 8c It is enough to check that for every f CE the map x fdıx is continuous, which is true. 8d We need to check Since δ xπ 1 A = δ πx A, we have T π ıx = T πδ x = δ πx = ı π. This follows from the definition of T π. For each g CE one has gydt πµy = gπxdµx E thus T π is continuous since g π CE. 8e Since U ı = ı π, we have that for every x E, Uδ x = δ πx = T πδ x. From the linearity of U second condition we gather that U coincides with T π of finite convex combinations of Dirac masses δ xi. Since U and T π are both continuous, we only need to check that finite convex combinations of Dirac masses are dense in PE. To this aim, since E is compact, for every ε > 0 there exists a finite partition of E, say {A ε 1, A ε,..., A ε N ε} in non-empty Borel sets of diameter smaller than ε this means that if x, y A ε i, then dx, y ε. For each i = 1,..., N ε, pick an arbitrary x ε i A ε i and for a µ PE define µ ε := αi ε δ x ε i, αi ε := µa ε i, N ε i=1

6 Since f CE are uniformly continuous, it is easily seen that for every f CE one has lim fdµ ε = fdµ ε 0 This implies that for every open neighborhood O of µ in PE, there is an ε > 0 small enough such that µ ε O, and thus finite convex combinations of Dirac masses are dense in PE. Sol 9. We have that PX = Y = P1 τ = 0 = Pτ = 0. On the other hand, PX t = Y t = Pτ t. Therefore Y is a version of X iff τ does not charge any point e.g. any law which admits a density w.r.t. the Lebesgue measure.