Notes on Snell Envelops and Examples

Transcription

1 Notes o Sell Evelops ad Examples Example (Secretary Problem): Coside a pool of N cadidates whose qualificatios are represeted by ukow umbers {a > a 2 > > a N } from best to last. They are iterviewed sequetially i a radom order ad we deote the correspodig qualificatio by {Y,Y 2,...,Y N }. That is, {Y,Y 2,...,Y N } is a radom permutatio of {a,a 2,...,a N } ad we assume all N! permutatios are equally likely. Cadidates that are dismissed caot be recalled. The values of {Y,Y 2,...,Y N } are ot observable, what oe observes is the relative rak. X = {Y Y k }. So X is the rak of the -th cadidate amog the cadidates that have bee iterviewed. The questio is how oe chooses a stoppig time τ with respect to the iformatio geerated by {X,X 2,...,X N } so as to maximize the probability P (Y τ = a ) of selectig the best cadidate. solutio: Observe that {X,X 2,...,X N } are idepedet radom variables ad that P (X =)=P (X =2)= = P (X = ) =/ for every. Sice Y caot be determied by the value of {X,X 2,...,X } (i.e., Y is ot measurable wrt to the σ-algebra geerated by {X,X 2,...,X }), the first thig we wat to do is to rewrite P (Y τ = a ) i terms of X i s. To this ed, ote that P (Y = a X,...,X )=f (X ) where f (x). = { /N ; if x = 0 ; if x>

2 Therefore P (Y τ = a ) = = = = E[ {Y=a } {τ=} ] E [ E[ {Y=a } {τ=} X,...,X ] ] E [ {τ=} E[ {Y=a } X,...,X ] ] E [ {τ=} f (X ) ] = Ef τ (X τ ). Now we ca write dow the DPE V N (x) f N (x) { } + V (x) = max f (x), V + (k). + It is ot difficult to show (by a verificatio argumet) that sup P (Y τ = a )=V () τ ad the optimal stoppig time is σ. = if{ :V (X )=f (X )}. Sice the solutio to the DPE has a explicit form { /N ; if x = V (x) = [ ] N N + + ; if x> ; m(n) ad V (x) m(n) N [ ] N + + ; <m(n), m(n) where m(n) is the uique iteger such that N k=m(n) k < N k=m(n) k, 2

3 it is ot difficult to check that σ = if{ m(n):x =}. That is, the optimal policy is to let the first m(n) cadidates go ad cotiue iterviewig cadidated util the first time at which the curret cadidate is the best amog the those examied so far. The maximal probability is sup P (Y τ = a )=V () = m(n) τ N [ N + + m(n) Whe N is large, it is ot difficult to see that m(n)/n /e ad V () /e, where /e 36.8%. Sell evelope Cosider a sequece of itegrable radom variables Y = {Y : =0,,...} adapted to a filtratio F = {F } defied o a complete probability space (Ω, F, P). For simplicity we assume {Y } is uiformly bouded. This coditio ca be relaxed ad all the results below hold as log as sup Y + is itegrable. Remark. I this sectio, wheever we metio stoppig times, uless explicitly poited out otherwise, we are assumig that the stoppig time is fiite with probability oe. We cosider the ifiite horizo problem. v 0 = sup E[Y τ ]. τ I a utshell, we will prove that there exists a supermartigale {Z } such that v 0 = E[Z ] for every 0, ad the optimal stoppig time is σ = if{ 0:Y = Z }, wheever σ is fiite with probability oe. As we will see, the supermartigale ca be represeted i the form Z = esssup τ E[Y τ F ]. This supermartigale Z is called Sell evelope ad will tur out to be the smallest supermartigale that domiates Y. 3 ].

4 . Essetial supremum I this sectio we will itroduce the cocept of essetial supremum, which is very useful to resolve the measurability issue whe the supremum of a collectio of ucoutable fuctios are cocered. Cosider a arbitrary family of radom variables g :Ω R. = R {+ } defied o a complete probability space (Ω, F, P). The essetial supremum of G is a radom variable l :Ω R such that. For every g G, l g almost surely. 2. If h g almost surely for every g G, the h l almost surely. Propositio. Essetial supremum always exists, ad there exists a sequece {g } G such that esssup(g) = sup g, a.s. Proof. We ca assume all the radom variables g takig values i [0, ] (otherwise, use a mootoe bijectio from R to [0, ]). Let C be the collectio of all coutable subset of G. For every C C, defie poitwise l C. = sup g. g C Let v =. sup E[l C ]. C C We claim that the supremum i the defiitio of v is attaied. cosider a maximizig sequece {C,C 2,...} so that Ideed, v = lim E[l C ]. Defie C. = C C 2, the C C, ad E[l C ] E[l C ] v. We coclude E[l C ]=v. Let l. = lc. We wat to show that l is the essetial supremum. Cosider ay g G, defie C =. C {g} C. We have v E[l C ]=E[l g] E[l ]=v. 4

5 It follows that l g = l,orl g, almost surely. Moreover, for ay h such that h g almost surely, we have h l C almost surely for every C C. I particular, h l almost surely. Fially, sice C is coutable, oe ca arrage it i a sequece {g } so that l = l C = sup g = sup g. g C This completes the proof. Remark 2. It is ot difficult to see that essetial supremum is uique up to almost sure equivalece. Remark 3. Whe the set G is directed upwards i the sese that for every pair g,g 2 G, there exists g G such that g g g 2 almost surely, oe ca argue that there exists a o-decreasig sequece {g } G such that esssup(g) = lim g..2 Sell evelope The Sell evelope is defied as the process Z = {Z } give by We have the followig result. Z. = esssupτ E[Y τ F ]. Theorem. The process Z satisfies the DPE Z = max{y,e[z + F ]} ad. v = sup E[Y τ ]=E[Z ] τ for every 0. I particular, Z is the smallest supermartigale that domiates Y. Proof. The radom variable Z is bouded, F -measurable ad satisfies Y Z. We claim that the collectio of radom variables {E[Y τ F ]:τ } are directed upwards. I fact, let τ, τ 2 be two stoppig times. Let A. = {E[Y τ F ] E[Y τ2 F ]}. 5

6 The A F ad the radom time τ. = τ A + τ 2 A c is ideed a stoppig time. It follows that E[Y τ F ]= A E[Y τ F ]+ A ce[y τ2 F ] = max{e[y τ F ],E[Y τ2 F ]}. Thaks to Remark 3, there exists a sequece of stoppig times {τ k } such that τ k, ad Z = lim k E[Y τk F ]. Usig MCT (the coditioal expectatio versio), we have E[Z F ]=lim k E[E[Y τk F ] F ]=lim k E[Y τk F ] Z. It follows that Z max{y,e[z + F ]}. O the other had, for ay stoopig time τ, we write Ad Y τ = Y {τ=} + Y τ (+) {τ>}. E[Y τ F ] = {τ=} Y + {τ>} E[Y τ (+) F ] = {τ=} Y + {τ>} E[E[Y τ (+) F + ] F ] {τ=} Y + {τ>} E[Z + F ] max{y,e[z + F ]}. Takig supremum o the left-had-side, we have Z max{y,e[z + F ]}. Therefore equality holds. Furthermore, by MCT, E[Z ] = lim k E[E[Y τk F ]] = lim k E[Y τk ], we have E[Z ] sup E[Y τ ]=v. τ But by defiitio of Z, E[Z ] E[Y τ ] for every τ. It follows that E[Z ]=supe[y τ ]=v. τ Fially, ote the DPE implies that {Z } is a supermartigale. Let Z be aother supermartigale that domiates Y (thus bouded from below). By optioal samplig theorem, for every τ, Z E[Z τ F ] E[Y τ F ]. Thus Z Z. This completes the proof. 6

7 Theorem 2. The supremum v 0 = sup τ E[Y τ ] is attaied if ad oly if the stoppig time σ. = if{ 0:Z = Y } is fiite with probability oe. I this case, σ is ideed optimal. Proof. We first assume that σ is fiite almost surely. We claim that the process {Z σ : 0} is a martigale wrt to {F }. Ideed, E[Z σ (+) F ] = {σ }Z σ + {σ >} E[Z + F ] = {σ }Z σ + {σ >} Z = Z σ. I particular, we have, thaks to DCT ad Theorem, v 0 = E[Z 0 ]=lim E[Z σ ] =E[Z σ ]=E[Y σ ], or σ is optimal. Now assume that there exists a optimal stoppig time ρ, ad we wish to show σ is fiite. It suffices to show that Z ρ = Y ρ. But Y ρ Z ρ ad by optioal samplig theorem, Therefore, Y ρ = Z ρ almost surely. E[Y ρ ]=v 0 = E[Z 0 ] E[Z ρ ]. Exercise. For every stoppig time ρ, show that Z ρ = esssup τ ρ E[Y τ F ρ ] ad E[Z ρ ] = sup E[Y τ ]. τ ρ Exercise (A pathwise approach to optimal stoppig). Z = {Z } admits the Doob s Decompositio A supermartigale Z = M A where M = {M } is a martigale wrt to {F }, ad A = {A } is a odecreasig process such that A 0 = 0 ad A is F -measurable. The decompsitio is uique, ad it is determied recursively by M 0 = Z 0, M = M + Z E[Z F ] 7

8 ad A 0 =0, A = A + Z E[Z F ]. Let M. = lim M. Defie a o-adapted process λ. = M M. The v 0 = E[sup(Y + λ )]. Moreover, if σ is fiite with probability oe, the the supremum sup (Y + λ ) is attaied at σ almost surely. Exercise (Prophet Iequality). Let Y = {Y } be a sequece of o-egative, idepedet radom variables, ad F is the σ-algebra geerated by (Y 0,Y,...,Y ). Show that E[sup Y ] 2v 0. 8