ADAPTIVE POISSON DISORDER PROBLEM

Transcription

1 ADAPTIVE POISSON DISORDER PROBLEM ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS Abstrat. We study the quikest detetion problem of a sudden hange in the arrival rate of a Poisson proess from a known value to an unknown and unobservable value at an unknown and unobservable disorder time. Our objetive is to design an alarm time whih is adapted to the history of the arrival proess and detets the disorder time as soon as possible. In previous solvable versions of the Poisson disorder problem, the arrival rate after the disorder has been assumed a known onstant. In reality, however, we may at most have some prior information on the likely values of the new arrival rate before the disorder atually happens, and insuffiient estimates of the new rate after the disorder happens. Consequently, we assume in this paper that the new arrival rate after the disorder is a random variable. The detetion problem is shown to admit a finite-dimensional Markovian suffiient statisti if the new rate has a disrete distribution with finitely-many atoms. Furthermore, the detetion problem is ast as a disounted optimal stopping problem with running ost for a finite-dimensional pieewise-deterministi Markov proess. This optimal stopping problem is studied in detail in the speial ase where the new arrival rate has Bernoulli distribution. This is a non-trivial optimal stopping problem for a two-dimensional pieewise-deterministi Markov proess driven by the same point proess. Using a suitable single-jump operator, we solve it expliitly, desribe the analyti properties of the value funtion and the stopping region, and present methods for their numerial alulation. We provide a onrete example where the value funtion does not satisfy the smooth-fit priniple on a proper subset of the onneted, ontinuously differentiable optimal stopping boundary, whereas it does on the rest. Contents 1. Introdution and Synopsis 2 Part 1. ANALYSIS: PROBLEM DESCRIPTION, MODEL, AND APPROXIMATION 11 Date: November 6, Mathematis Subjet Classifiation. Primary 62L1; Seondary 62L15, 62C1, 6G4. Key words and phrases. Poisson disorder problem, quikest detetion, optimal stopping. 1

2 2 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS 2. Problem desription Suffiient statistis for the adaptive Poisson disorder problem Poisson disorder problem with a Bernoulli post-disorder arrival rate An optimal stopping problem for the quikest detetion of the Poisson disorder The sample-paths of the suffiient-statisti proess Φ = ( Φ (), Φ (1) ) Case I: A large disorder arrival rate Case II: A small disorder arrival rate 2 5. A family of related optimal stopping problems A bound on the alarm time Appendix: proofs of seleted results in Part 1 3 Part 2. SYNTHESIS: SOLUTION AND NUMERICAL METHODS The solution The struture of the stopping regions The boundaries of the stopping regions The entrane boundary Γ e n The exit boundary Γ x n Case I revisited: effiient methods for large post-disorder arrival rates The smoothness of the value funtions and the stopping boundaries The interplay between the exit and entrane boundaries The regularity of the value funtions and the optimal stopping boundaries when the disorder arrival rate λ is large Failure of the smooth-fit priniple: a onrete example Appendix: proofs of seleted results in Part 2 75 Referenes Introdution and Synopsis Suppose that arrivals of ertain events onstitute a Poisson proess N = {N t : t } with a known rate µ >. At some time θ, the arrival rate suddenly hanges from µ to Λ. Both the disorder time θ and the post-disorder arrival rate Λ of the Poisson proess are unknown and unobservable. Our problem is to find an alarm time τ whih depends only on the past and the present observations of the proess N, and detets the disorder time θ as soon as possible.

3 ADAPTIVE POISSON DISORDER PROBLEM 3 More preisely, we shall assume that θ and Λ are random variables on some probability spae (Ω, H, P), on whih the proess N is also defined; the variables θ, Λ are independent of eah other and of the proess N. An alarm time is a stopping time τ of the history of the proess N. We shall try to hoose suh a stopping time so as to minimize the Bayes risk (1.1) P {τ < θ} + E(τ θ) +, namely, the sum of the frequeny P{τ < θ} of the false alarms and the expeted ost E(τ θ) + of the detetion delay. We shall assume that the post-disorder arrival rate Λ has some general prior distribution ν( ). Similarly, the disorder time θ will be assumed to have an exponential distribution onditionally on that the disorder has not happened yet, i.e., for some π [, 1) and λ > (1.2) P{θ = } = π and P{θ > t θ > } = e λt, t. The Poisson disorder problem with a known post-disorder rate (namely, Λ equals a known onstant with probability one) was studied first by Galhuk and Rozovskii (1971) and was solved ompletely by Peskir and Shiryaev (22). In the meantime, Davis (1976) notied that several forms of Bayes risks inluding (1.1) admit similar solutions. He alled this lass of problems standard Poisson disorder problems, and found a partial solution. Reently, Bayraktar and Dayanik (23) solved the Poisson disorder problem when the detetion delay is penalized exponentially. Bayraktar, Dayanik and Karatzas (24b) showed that the exponential detetion delay penalty in fat leads to another variant of standard Poisson disorder problems if the standards suggested by M. Davis are restated under a suitable referene probability measure. It was also shown that use of a suitable referene probability measure redues the dimension of the Markovian suffiient statisti for the detetion problem, and the solution of the standard Poisson disorder problem was desribed fully. We believe that unknown and unobservable post-disorder arrival rate Λ aptures quite well real-life appliations of hange-point detetion theory. Before the onset of the new regime, past experiene may help us at most to fit an apriori distribution ν( ) on the likely values of the new arrival rate of N after the disorder. Even after the disorder happens, we may not have enough observations to get a reliable statistial estimate of the post-disorder rate. Indeed, sine a good alarm is expeted to sound as soon as the disorder happens, we may have very few observations of N sampled from the new regime sine the disorder. Let us highlight our approah to the problem and our main results. We show that the most general suh detetion problem is equivalent, under a referene probability measure,

4 4 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS to a disounted optimal stopping problem with a running ost for an infinite-dimensional Markovian suffiient statisti. However, the dimension beomes finite as soon as the prior probability distribution ν( ) of the post-disorder arrival rate Λ harges only a finite number of atoms. This lass of problems is of great interest sine, in many appliations, we have typially an empirial distribution of the post-disorder arrival rate, onstruted either from finite past data or from expert opinions on the most signifiant likely values. We then study in detail the ase where the new arrival rate after the disorder is expeted either to inrease or to derease by the same amount. The detetion problem turns in this ase into an optimal stopping problem for a two-dimensional pieewise-deterministi Markov proess, driven by the same point proess. We solve this optimal stopping problem fully by desribing ε-optimal and optimal stopping times and identifing expliitly the non-trivial shape of the optimal ontinuation region. The ommon approah to an optimal stopping problem for a ontinuous-time Markov proess is to reformulate it as a free-boundary problem in terms of the infinitesimal generator of the Markov proess. The free-boundary problems sometimes turn out to be quite hard, even in one dimension; see, for example, Galhuk and Rozovskii (1971), Peskir and Shiryaev (22), Bayraktar and Dayanik (23). Here the infinitesimal operator gets ompliated further and beomes a singular partial differential-delay operator. Moreover, it is a non-trivial task, even in two dimensions, to guess the loation, shape, and smoothness of the free-boundary separating the ontinuation and stopping regions, as well as the behavior of the value funtion along the boundary. Instead, we follow a diret approah and work with integral operators rather than differential operators. As in Gugerli (1986) and Davis (1993), we use a suitable single-jump operator to strip the jumps off the original two-dimensional pieewise-deterministi Markov proess and turn the original optimal stopping problem into a sequene of optimal stopping problems for a deterministi proess with ontinuous paths. Using diret arguments, we are able to infer from the properties of the single-jump operator the loation and shape of the optimal ontinuation region, as well as the smoothness of the swithing boundary and the value funtion. The single-jump operator also provides a straightforward numerial method for alulating the value funtion and the ontinuation region. The deterministi proess obtained after removing the jumps from the original Markov proess has two fundamentally different types of behavior. We tailor the naive numerial method to eah ase, by exploiting the behavior of the paths.

5 ADAPTIVE POISSON DISORDER PROBLEM 5 We also raise the question when the value funtion should be a lassial solution of the relevant free-boundary problem. For a large range of onfigurations of parameters, both the value funtion and the boundary of the ontinuation region turn out to be ontinuously differentiable, and one may also hoose to use finite-differene methods for differential-differene equations to solve the problem numerially. For a few other ases, we annot qualify ompletely the degree of the smoothness of the value funtion. Visosity approahes or some other tehniques of non-smooth analysis are very likely to fill the gap, but we do not pursue this diretion here. We report one onrete example on partial failure of the smooth-fit priniple: in ertain ases, the value funtion is ontinuously differentiable everywhere on the state spae exept on a proper subset of the onneted and ontinuously differentiable optimal stopping boundary. Synopsis. In Setion 2, we present the model and the preise statement of the problem. A Poisson proess N with arrival rate µ and two independent random variables θ and Λ with given prior probability distributions are introdued on a suitable probability spae (Ω, H, P ). Under a new probability measure P obtained from P by an absolutely ontinuous hange of measure, (i) the proess N beomes a Poisson proess whose arrival rate hanges from onstant µ to the random Λ at time θ, and (ii) the random variables Λ and θ are independent and have the same distributions as those under P. Working under the referene probability measure P turns out to be onvenient. In Setion 3.1, the generalized Bayes theorem gives the new form [ τ ( R τ (π) = 1 π + (1 π) E e λt Φ () t λ ) (3.2) dt, τ S, for the Bayes risk in (1.1), this time expressed under P in terms of a suitable proess Φ () adapted to the history of N. Unfortunately, this proess is not Markovian in general. In fat, the dynamis of a family { Φ (k)} k N in (3.8) dφ (k) t ( = λ m (k) + Φ (k) t ) of adapted proesses inluding Φ () are nested as dt + 1 µ Φ(k+1) t (dn t µdt), t >, Φ (k) = π 1 π m(k). Hene the Poisson disorder problem is equivalent to the minimization of the Bayes risk in (3.2) over all stopping times τ of the infinite-dimensional Markovian suffiient statisti { Φ (k)} k N. However, Corollary 3.3 shows that only finitely many of the Φ (k) s (as many as the number of atoms of the distribution ν( )) ontain all relevant information if the distribution ν( ) is onentrated on a finite number of atoms.

6 6 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS In Setion 4, we speialize to the detetion problem where the post-disorder arrival rate Λ is expeted either to inrease by one unit or to derease by one unit; namely, µ > 1 and ν({µ 1, µ + 1}) = 1. The suffiient statisti for the detetion problem beomes the proess {Φ (), Φ (1) } with the dynamis ( ) (4.2) dφ () t = λ 1 + Φ () t dt + 1 µ Φ(1) t (dn t µdt), Φ () = π (4.3) dφ (1) t ( = λ m + Φ (1) t 1 π, ) dt + 1 µ Φ() t (dn t µdt), Φ (1) = π 1 π m, where m E [Λ µ = ν({µ + 1}) ν({µ 1}). If we rotate the oordinate system by 45 lokwise, then the dynamis of the suffiient statisti in the new oordinates Φ { Φ (), Φ (1) } [ d Φ () t = (λ + 1) Φ () () (1 m)π t + Φ t dn t,, 2(1 π) (4.6) d Φ (1) t = λ(1 m) dt 1 2 µ [ (λ 1) Φ (1) λ(1 + m) t + dt µ Φ (1) t dn t, Φ() = Φ(1) = (1 + m)π 2(1 π) are autonomous. Between onseutive jumps of the Poisson proess N, the sample paths of the proess Φ = { Φ (), Φ (1) } follow the integral urves t (x(t, φ ), y(t, φ 1 )), t R + of the differential equations (4.7) d dt x(t, φ λ(1 m) ) = (λ + 1)x(t, φ ) +, x(, φ ) = φ, 2 d dt y(t, φ λ(1 + m) 1) = (λ 1)y(t, φ 1 ) +, y(, φ 1 ) = φ 1. 2 More preisely, if σ and σ n, n N is the nth jump time of the Poisson proess N, then (see also Figures 1 and 2 on page 19) (4.1) ( ) Φ () () t = x t σ n, Φ σ n ) and Φ () σ n = ( 1 1 µ Φ (1) t Φ () σ n and ( = y Φ (1) σ n = t σ n, ( µ ) (1) Φ σ n ), σ n t < σ n+1, n N, Φ (1) σ n, n N. Moreover, the detetion problem redues to the disounted optimal stopping problem [ τ V (φ, φ 1 ) inf τ S Eφ,φ ( Φ() ) 1 e λt (1) g t, Φ t dt (4.12) with g(φ, φ 1 ) φ + φ 1 λ 2, (φ, φ 1 ) R 2 + for the two-dimensional pieewise-deterministi Markov proess Φ = { Φ (), Φ (1) } in (4.6) or (4.1).

7 ADAPTIVE POISSON DISORDER PROBLEM 7 To solve (4.12), we introdue in Setion 5 the family of optimal stopping problems [ τ σn V n (φ, φ 1 ) inf τ S Eφ,φ ( Φ() ) 1 e λt (1) (5.1) g t, Φ t dt, (φ, φ 1 ) R 2 +, n N. We show that the sequene {V n (, )} n N onverges to the value funtion V (, ) uniformly on R 2 +. More preisely, 2 (5.2) ( µ λ + µ ) n V n (φ, φ 1 ) V (φ, φ 1 ), n N, (φ, φ 1 ) R 2 +. Following Gugerli (1986) and Davis (1993), we define the single-jump operator J on bounded Borel funtions w : R 2 + R by (5.3) (5.7) Jw(t, φ, φ 1 ) E φ,φ 1 = t [ t σ1 ( Φ() e λu g u, ) (1) Φ ( Φ() ) u du + 1 {t σ1 }e λσ 1 (1) w σ 1, Φ σ 1, e (λ+µ)u( g + µ w S )( x(u, φ ), y(u, φ 1 ) ) du, ) where S(x, y) ((1 1µ ) x, (1 + 1µ ) y R 2 + for every (x, y) R 2 +, and (5.4) J t w(φ, φ 1 ) inf Jw(u, (φ, φ 1 )), u [t, t [,. t [,, The value funtions V n, n N in (5.1) oinide with the funtions v n, n N defined sequentially by (5.6) v n J v n 1 n N, and v, and V (, ) = v(, ) lim n v n (, ) on R 2 +. Moreover, the form of the single-jump operator Jw(,, ) in (5.7) allows us to prove that the funtions v n (, ) N and v(, ) are inreasing in eah argument, uniformly bounded, and onave. This information beomes ruial later in Setion 9 as we study the shape of the ontinuation regions and the smoothness of the optimal stopping boundaries. For every n N, the value funtion V n+1 (, ) in (5.1) is attained by the stopping time S n+1 defined sequentially by r n (φ, φ 1 ) inf{t > : v n+1 (x(t, φ ), y(t, φ 1 )) = }, n N, (φ, φ 1 ) R 2 +, ( ) ( ) ( ) r n Φ, if σ1 > r n Φ S 1 r Φ σ1, and S n+1 ( ) σ 1 + S n θ σ1, if σ 1 r n Φ, n N, where θ s is the shift-operator on Ω: N t θ s = N s+t ; see Proposition 5.5 and Corollary 5.8. In other words, the best ation before the (n + 1)st Poisson arrival is to stop if the ontinuous

8 8 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS parametri urve t ( x(t, Φ () (ω)), y(t, Φ (1) (ω)) ), t R + in (4.7) enters the (n + 1)st stopping region {(x, y) R 2 + : v n+1 (x, y) = } before the arrival of the first Poisson event. Otherwise, it is better to wait, and stop if the urve t ( x(t, Φ () σ 1 (ω)(ω)), y(t, Φ(1) σ 1 (ω) (ω))), t R + enters the nth stopping region {(x, y) R 2 + : v n (x, y) = } before the arrival of the next Poisson event, and so on; see also Figure 4(b) on page 39, Setion 9. The expliit optimal stopping rules S n for V n (, ), n N and the uniform onvergene in (5.2) give an ε-optimal stopping time for V (, ) in (4.12). For every ε >, ( ) n [ 2 µ Sn < ε = E φ,φ 1 e λt g ( ) Φt dt V (φ, φ 1 ) < ε, (φ, φ 1 ) R 2 λ + µ +. In Proposition 5.12 of Setion 5, we also prove that the problem with the value funtion V (, ) in (4.12) admits an optimal stopping time, and the lassial stopping times U ε inf{t : V ( Φ t ) ε} are ε-optimal for every ε. In Setion 6, we show that the optimal ontinuation region {(φ, φ 1 ) R 2 + : V (φ, φ 1 ) < } for the problem V (, ) in (4.12) is a bounded subset of R 2 +. The boundedness of this region and the onavity of the value funtion V (, ) will help us desribe expliitly the struture of the ontinuation region in Setion 9. As a by-produt of the results in Setion 6, we obtain some simple bounds on the alarm time. When these are tight, they may serve as good approximate detetion rules. The optimal stopping time U inf{t : V ( Φ () (1) t, Φ t ) = } is bounded by { () (1) τ C inf t : Φ t + Φ t λ } () (1) (6.2) 2 U τ D inf{t : Φ t + Φ t ξ } for a suitable onstant ξ. If λ λ+µ (1 λ), then ξ = λ+µ 2; otherwise, it is the expliit solution of (6.1), i.e., where ξ λ(1 + m) = + 2(λ 1) φ [ φ 1 + (λ + µ)(1 λ) λ 2 λ(1 + m) 2(λ 1) [ 1 + φ and φ 1 λ 1 λ+1 2(λ + 1) λ(1 m) (λ + µ)(1 + λ) + λ. 2 > λ + µ 2 For small values of the ratio µ/ of the pre-disorder arrival rate and the detetion delay ost per unit time, the thresholds of the lower and upper bounds in (6.2) on the optimal stopping time U are lose. Then the upper bound τ D may serve as a simple approximate alarm time.

9 ADAPTIVE POISSON DISORDER PROBLEM 9 Setion 8 starts with the desription of the general sequential/numerial solution method. Eah funtion V n (, ) in (5.1) vanishes outside the region D {(x, y) R 2 + : x + y ξ }, where ξ is defined as above. On the bounded set D, we an find the funtions V n (, ) v n (, ) by repeatedly applying the operator J in (5.4). In pratie, the uniform onvergene in (5.2) lets us ontrol the number of iterations needed to ahieve any given level of auray. The exponential rate of onvergene also suggests that this sequential algorithm should be omputationally feasible and aurate. We tailor the general method mainly to two distint ases (see below) of the detetion problem. In the meantime, we also shed light on the struture of the solutions of the optimal stopping problems in (4.12) and (5.1). We show that the stopping regions Γ n {(φ, φ 1 ) R 2 + : v n (φ, φ 1 ) = } = {(φ, φ 1 ) R 2 + : γ n (φ ) φ 1 }, n N, Γ {(φ, φ 1 ) R 2 + : v(φ, φ 1 ) = } = {(φ, φ 1 ) R 2 + : γ(φ ) φ 1 } are onvex epigraphs of some boundary funtions γ n : R + R and γ : R + R, respetively. These boundary funtions are stritly dereasing, ontinuous, and onvex on their ompat supports. The sequene {γ n ( )} n N is inreasing and onverges to γ( ); see Figure 4(a) on page 39. Case I: A large disorder arrival rate λ (1 + m)(/2). After some preparations in Setions 9 and 1, we prove in Setion 11 that { (x, y) R 2 + : v(x, y) = v n (x, y) } inreases to R 2 +, and {x R + : γ(x) = γ n (x)} inreases to R +. Namely, every iteration v n (, ) of the suessive approximations in (5.6) gives the exat value funtion v(, ) on a subset of the state spae R 2 + whih inreases to the whole spae as the iterations progress. Based on this fat, Methods B and C on pages 5 and 52, respetively, gradually alulate the value funtion v(, ) on R 2 + and the optimal stopping boundary γ( ) on R + ; see also Figure 6. In Setion 12.2, we prove that the value funtions v n (, ), n N and v(, ) are ontinuously differentiable everywhere, and that the boundary funtions γ n ( ), n N and γ( ) are ontinuously differentiable on their respetive supports. The value funtion v(, ) obtained as the limit of the suessive approximations in (5.6) satisfies the variational inequalities in (12.1)-(12.4) assoiated with the optimal stopping problem (4.12) and is the unique solution (together with the boundary Γ {(x, γ(x)) : x supp(γ)}) of the orresponding freeboundary problem. Finally, the smooth-fit priniple also holds for the funtion v(, ) aross

10 1 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS the boundary Γ. These results ensure that the value funtion v(, ) an be alulated by using finite-differene methods for partial differential-differene equations. Case II: A small disorder arrival rate < λ < (1 + m)(/2). In this ase we have to pay more attention to the struture of the boundaries Γ n+1 = {(x, γ n+1 (x)) : x supp(γ n+1 )} of the stopping regions Γ n+1, n N. For every n N, the boundary Γ n+1 is divided into the entrane and exit boundaries { (x(rn Γ e n+1 (φ, φ 1 ), φ ), γ n+1 (y(r n (φ, φ 1 ), φ 1 ) ) }, for some (φ, φ 1 ) C n+1, (1.1) { } Γ x n+1 (φ, φ 1 ) Γ n+1 : (x(t, φ ), y(t, φ 1 )) C n+1, t (, δ for some δ >, respetively (in Case I, we always have Γ n+1 Γ e n+1 for every n N ). We show that the value funtion v n+1 (, ) and the exit boundary Γ x n+1 are ompletely determined by the entrane boundary Γ e n+1 (see Lemma 1.7), and the value funtion v n (, ) from the previous iteration determines the entrane boundary Γ e n+1. Sine v is already available, the general solution method outlined above an be enhaned as in Method D on page 62. For ertain onfigurations of parameters, we are able to prove that the value funtions v n+1 (, ), n N are ontinuously differentiable everywhere on R 2 +\ Γ x n+1 and are not differentiable on the exit boundary Γ x n+1, n N, see Proposition and Setion In Setion 12.3, we give a onrete example for a ase where the value funtion v(, ) of the optimal stopping problem in (4.12) is ontinuously differentiable everywhere on R 2 +\ Γ x and is not differentiable on the exit boundary Γ x of the optimal stopping region Γ. The interesting feature of this example is that the smooth-fit priniple fails on some proper subset (namely, the exit boundary Γ x ) of the onneted and ontinuously differentiable optimal stopping boundary Γ, while this priniple holds on the rest; see Figure 9(d). This work is divided naturally in two parts. In Part 1, we desribe the problem, formulate a onvenient model, and develop an important approximation. In Part 2, we use the approximation of Part 1 to develop the solution and study its properties. Both parts are aompanied by independent appendies whih are the homes for long proofs.

11 ADAPTIVE POISSON DISORDER PROBLEM 11 Part 1. ANALYSIS: PROBLEM DESCRIPTION, MODEL, AND APPROXIMATION 2. Problem desription Let N = {N t ; t } be a homogeneous Poisson proess with some rate µ > on a fixed probability spae (Ω, H, P ), whih also supports two random variables θ and Λ independent of eah other and of the proess N. We shall denote by ν( ) the distribution of the random variable Λ, assume that (2.1) m (k) (v µ) k ν(dv), k N are well-defined and finite, and that R (2.2) P {θ = } = π and P {θ > t} = (1 π)e λt, t hold for some onstants λ > and π [, 1). Let us denote by F = {F t } t the right-ontinuous enlargement with P -null sets of the natural filtration σ(n s ; s t) of N. We also define a larger filtration G = {G t } t by setting G t F t σ{θ, Λ}, t. The G-adapted, right-ontinuous (hene, G-progressively measurable) proess (2.3) h(t) µ1 {t<θ} + Λ1 {t θ}, t indues the (P, G)-martingale (see Brémaud (1981, pp )) { t ( ) h(s ) t } (2.4) Z t exp log dn s (h(s) µ)ds, t. µ This martingale defines a new probability measure P on every (Ω, G t ) by dp (2.5) dp = Z t, t. Gt Sine P and P oinide on G = σ{θ, Λ}, the random variables θ and Λ are independent and have the same distributions under both P and P. Under the new probability measure P the ounting proess N has G-progressively measurable intensity given by h( ) of (2.3), namely N t t h(s)ds, t is a (P, G)-martingale. In other words, the G-adapted proess N is a Poisson proess whose rate hanges at time θ from µ to Λ. In the Poisson disorder problem, only the proess N is observable, and our objetive is to detet the disorder time θ as quikly as possible. More preisely, we want to find an

12 12 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS F-stopping time τ that minimizes the Bayes risk (2.6) R τ (π) P{τ < θ} + E(τ θ) +, where > is a onstant, and the expetation E is taken under the probability measure P. Hene, we are interested in an alarm time τ whih is adapted to the history of the proess N, and minimizes the tradeoff between the frequeny of false alarms P{τ < θ} and the expeted time of delay E(τ θ) + between the alarm time and the unobservable disorder time. In the next setion, we shall formulate the quikest detetion problem as a problem of optimal stopping for a suitable Markov proess. 3. Suffiient statistis for the adaptive Poisson disorder problem Let S be the olletion of all F-stopping times, and introdue the F-adapted proesses (3.1) Π t P{θ t F t }, and Φ (k) t E[(Λ µ)k 1 {θ t} F t 1 Π t, k N, t. Sine Λ has the same distribution ν( ) under P and P, eah Φ (k), k N is well-defined by (2.1). The proess Π = {Π t, t } traks the likelihood that a hange in the intensity of N has already ourred, given past and present observations of the proess. Eah Φ (k) = {Φ (k) t, t }, k N may be regarded as a (weighted) odds-ratio proess. Our first lemma below shows that the minimum Bayes risk an be found by solving a disounted optimal stopping problem, with disount rate λ and running ost funtion f(x) = x λ/ for the F-adapted proess Φ (). The alulations are onsiderably easier when the proess Φ () has the Markov property. Unfortunately, this is not true in general. However, the expliit dynamis of Φ () in Lemma 3.2 reveal that the infinite-dimensional sequene {Φ (k) } k N of the proesses in (3.1) is always a suffiient Markovian statisti for the quikest detetion problem. The same result also suggests suffiient onditions for the existene of a finite-dimensional suffiient Markovian statisti, a ase amenable to onrete analysis Lemma. The Bayes risk in (2.6) equals (3.2) R τ (π) = 1 π + (1 π) E [ τ ( e λt Φ () t λ ) dt, τ S, where the expetation E is taken under the (referene) probability measure P. For several proofs below, the following observations will be useful. Every Z t in (2.4) an be written as (3.3) Z t = 1 {t<θ} + L t L θ 1 {t θ}

13 ADAPTIVE POISSON DISORDER PROBLEM 13 in terms of the likelihood ratio proess ( ) Nt Λ (3.4) L t e (Λ µ)t, t. µ Then the generalized Bayes theorem (see, e.g., Liptser and Shiryaev (21, Setion 7.9)) and (3.3) imply (3.5) 1 Π t = E [Z t 1 {θ>t} F t E [Z t F t = P {θ > t F t } E [Z t F t = (1 π)e λt, E [Z t F t sine θ is independent of the proess N under P and has the distribution (2.2). Proof of Lemma 3.1. By (3.1), the generalized Bayes theorem and (3.5), we have Φ () t = E[1 {θ t} F t 1 Π t = E [Z t 1 {θ t} F t (1 Π t )E [Z t F t = E [Z t 1 {θ t} F t (1 π)e λt, t, whih gives [ (3.6) E[(τ θ) + = E = 1 {θ t<τ} dt = E [Z t 1 {τ>t} 1 θ t dt E [ 1{τ>t} E [Z t 1 {θ t} F t dt = (1 π) E [ τ e λt Φ () t dt, τ S. Suppose that τ S takes ountably many distint values {t n, n N} for some t n R + {+ }. Then (3.7) P{τ < θ} = n P{t n < θ, τ = t n } = n E [Z tn 1 {tn<θ}1 {τ=tn} = E [1 {θ>tn}1 {τ=tn} = (1 π)e λtn E [1 {τ=tn} n n [( t n ) [ τ = (1 π) E 1 λe λt dt 1 {τ=tn} = (1 π) (1 π)λ E e λt dt. n An arbitrary stopping time τ S is the almost-sure limit of a dereasing sequene {τ n } n 1 S of stopping times whih take ountably many values. For every τ n, n N, (3.7) holds. Sine t 1 {t<θ} and t t e λs ds are right-ontinuous and bounded, passage to limit as n and the bounded onvergene theorem verifies (3.7) for every τ S. The sum of (3.6) and (3.7) gives (3.2) Lemma. Let m (k), k N be defined as in (2.1) Then every Φ (k), k N in (3.1) satisfies the equation ( (3.8) = λ dφ (k) t m (k) + Φ (k) t ) dt + 1 µ Φ(k+1) t (dn t µdt), t >, Φ (k) = π 1 π m(k).

14 14 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS Proof. For every k N, let us introdue the funtion ( ) x v (3.9) F (k) (t, x) (v µ) k e (v µ)t ν(dv), t R +, x R. µ The generalized Bayes theorem, (3.5), and the independene of the random variables θ, Λ and the proess N under P imply (3.1) Φ (k) t = E [ (Λ µ) k Z t 1 {θ t} F t (1 Π t )E [Z t F t = πeλt 1 π F (k) (t, N t ) + λ for every k N and t R +, where (3.11) U (k) t t πeλt 1 π F (k) (t, N t ) and V (k) t λ = E [ (Λ µ) k (L t 1 {θ=} + Lt L θ 1 {<θ t} ) F t (1 π)e λt e λ(t s) F (k) (t s, N t N s )ds = U (k) t Every F (k) (t, x), k N in (3.9) is ontinuously differentiable, and t + V (k) t e λ(t s) F (k) (t s, N t N s )ds. (3.12) t F (k) (t, x) = F (k+1) (t, x), t >, x R, k N. The hange of variable formula for jump proesses gives (3.13) t F (k) F (k) (t, N t ) = F (k) (, ) + (s, N s )ds + t + [F (k) (s, N s ) F (k) (s, N s ) <s t t = m (k) F (k+1) (s, N s )ds + <s t t F (k) x (s, N s )dn s F (k) x (s, N s ) N s [ F (k) (s, N s ) F (k) (s, N s ), where N s N s N s {, 1} for every s >, and the last equality follows from (3.12), F (k) (, ) = m (k), k N, and t F (k) However, F (k) (s, N s ) F (k) (s, N s ) is equal to x (s, N s )dn s = <s t F (k) ( ) Ns + N v s ( ) Ns v (v µ) k e (v µ)s ν(dv) (v µ) k e (v µ)s ν(dv) µ µ ( ) [ Ns ( ) Ns v v = 1 (v µ) k e (v µ)t ν(dv) = N s µ µ µ x (s, N s ) N s. ( ) Ns v (v µ) k+1 e (v µ)t ν(dv) = 1 µ µ F (k+1) (s, N s ) N s,

15 ADAPTIVE POISSON DISORDER PROBLEM 15 sine [(v/µ) Ns 1 = ( N s /µ)(v µ). The last displayed equation and (3.13) imply (3.14) F (k) (t, N t ) = m (k) = m (k) + t t F (k+1) (s, N s )ds + <s t 1 µ F (k+1) (s, N s ) N s 1 µ F (k+1) (s, N s )(dn s µds), t R +, k N. This identity will help us derive the dynamis of U (k) and V (k) in (3.11). Note that ( ) 1 π d U (k) t = d ( e λt F (k) (t, N t ) ) = e λt F (k) (t, N t )λdt + e λt df (k) (t, N t ) π Therefore, (3.15) N (s) du (k) t = λu (k) t = λ 1 π π U (k) t dt + eλt µ F (k+1) (t, N t )(dn t µdt). + 1 µ U (k+1) t (dn t µdt), t >, U (k) = π 1 π m(k). The derivation of the dynamis of V (k) is trikier. For every fixed s [, t), let us define u N s+u N s, u t s. This is also a Poisson proess under P. As in (3.14), ( ) t s F (k) t s, N (s) t s = m (k) 1 ( ) (dn + µ F (k+1) u, N (s) (s) u u µdu ). Changing the variable of integration and substituting N (s) = N s+ N s into this equality gives F (k) (t s, N t N s ) = m (k) + 1 µ Let us plug this identity into V (k) t order of integration. Then e λt V (k) t = t λe λs ( m (k) + 1 µ t = m (k) λe λs ds + λ µ t = m (k) λe λs ds + 1 µ t s F (k+1) (v s, N v N s )(dn v µdv). in (3.11), multiply both sides by e λt, and hange the t t t ) F (k+1) (v s, N v N s )(dn v µdv) ds ( v ) e λs F (k+1) (v s, N v N s )ds (dn v µdv) e λv V v (k+1) (dn v µdv). Differentiating both sides and rearranging terms, we obtain ( ) (3.16) dv (k) t = λ m (k) + V (k) t dt + 1 µ V (k+1) t (dn t µdt), t >, V (k) =. Adding (3.15) and (3.16) as in (3.1) gives the dynamis (3.8) of the proess Φ (k).

16 16 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS Lemma 3.2 shows that the proess Φ () does not have the Markov property in general. This is beause, as (3.8) shows, Φ () depends on Φ (1), then Φ (1) depends on Φ (2), and so on ad infinitum. However, a finite-dimensional suffiient Markovian statisti emerges if the system of stohasti differential equations in (3.8) is loseable, namely, if the proess Φ (k) an be expressed in terms of the proesses Φ (),..., Φ (k 1), for some k N. Our next orollary shows that this is true if Λ takes finitely many distint values Corollary. Suppose that ν({λ 1,, λ k }) = 1 for some positive numbers λ 1,..., λ k. Consider the polynomial p(v) k k 1 (v λ i + µ) v k + i v i, i=1 i= v R for suitable real numbers,..., k 1. Then {Φ (), Φ (1),, Φ (k 1) } is a k-dimensional suffiient Markov statisti, with Φ (k) = k 1 i= i Φ (i). Proof. Under the hypothesis, the random variable p(λ µ) = (Λ µ) k + k 1 i= i(λ µ) i is equal to zero almost surely. Therefore, (3.1) implies k 1 Φ (k) t + i= i Φ (i) t = E [ p(λ µ)1 {θ t} F t 1 Π t =, P-a.s., for every t. The proess on the lefthand side has right-ontinuous sample paths, by (3.8). Therefore, Φ (k) t + k 1 i= i Φ (i) t = for all t R + almost surely, i.e., the proess Φ (k) is a linear ombination of the proesses Φ (),..., Φ (k 1) outside a null set. In appliations, one may onstrut an a prior distribution for the random variable Λ by using empirial distributions obtained from past data, if available, and/or from expert opinions in the field. Therefore, it is reasonable to expet that a prior distribution for Λ will typially be disrete with finite support. In suh a ase, we an set up the detetion problem in the form of an optimal stopping problem for a finite-dimensional Markov proess, thanks to Corollary 3.3. In the remainder of the paper, we shall study the ase where the arrival rate of the observations after the disorder has a Bernoulli prior distribution. 4. Poisson disorder problem with a Bernoulli post-disorder arrival rate We shall assume heneforth µ > 1 and that the random variable Λ has Bernoulli distribution (4.1) ν({µ 1, µ + 1}) = 1.

17 ADAPTIVE POISSON DISORDER PROBLEM 17 Namely, the rate of the Poisson proess N is expeted to inrease or derease by one unit after the disorder. Corollary 3.3 implies that Φ (2) = Φ (), and the suffiient statisti (Φ (), Φ (1) ) is a Markov proess. Aording to Lemma 3.2, the pair satisfies ( ) (4.2) dφ () t = λ 1 + Φ () t dt + 1 µ Φ(1) t (dn t µdt), Φ () = π 1 π, ( ) (4.3) dφ (1) t = λ m + Φ (1) t dt + 1 µ Φ() t (dn t µdt), Φ (1) = π 1 π m, where, as in (2.1), we set (4.4) m m (1) = E [Λ µ = P{Λ = µ + 1} P{Λ = µ 1}. The dynamis of the proesses Φ () and Φ (1) in (4.2) and (4.3) are interdependent. However, if we define a new proess (4.5) Φ [ Φ() Φ (1) [ 1 Φ () Φ (1), 2 Φ () + Φ (1) then eah of the new proesses Φ () and Φ (1) is autonomous: [ (λ + 1) Φ () (4.6) d Φ () t = d Φ (1) t = t + λ(1 m) dt 1 2 µ [ (λ 1) Φ (1) λ(1 + m) t + dt µ Φ () t dn t, Φ (1) t dn t, Φ() = Φ(1) = (1 m)π 2(1 π), (1 + m)π 2(1 π). The new oordinates Φ () and Φ (1) are in fat the onditional odds-ratio proesses as in Φ () t = 2 P{Λ = µ 1, θ t F t} P{θ > t F t } and t = 2 P{Λ = µ + 1, θ t F t}. P{θ > t F t } Φ(1) Therefore, both Φ () and Φ (1) are nonnegative proesses. Note that m [ 1, 1 in (4.4). The ases m = ±1 degenerate to Poisson disorder problems with known post-disorder rates, and were studied by Bayraktar, Dayanik, and Karatzas (24b). Therefore, we will assume that m ( 1, 1) in the remainder Remark. For every φ R and φ 1 R, let us denote by x(t, φ ), t R and y(t, φ 1 ), t R the solutions of the differential equations (4.7) d dt x(t, φ λ(1 m) ) = (λ + 1)x(t, φ ) +, x(, φ ) = φ, 2 d dt y(t, φ λ(1 + m) 1) = (λ 1)y(t, φ 1 ) +, y(, φ 1 ) = φ 1, 2

18 18 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS respetively. These solutions are given by [ λ(1 m) x(t, φ ) = + e (λ+1)t λ(1 m) φ +, t R, 2(λ + 1) 2(λ + 1) (4.8) [ λ(1 + m) + e (λ 1)t λ(1 + m) φ 1 +, λ 1 2(λ 1) 2(λ 1) y(t, φ 1 ) = φ m, t R. t, λ = 1 2 Both x(, φ ) and y(, φ 1 ) have the semi-group property, i.e., for every t R and s R (4.9) x(t + s, φ ) = x(s, x(t, φ )) and y(t + s, φ 1 ) = y(s, y(t, φ 1 )). Note from (4.6) and (4.7) that ( ) Φ () () (4.1) t = x t σ n, Φ σ n and Φ (1) t ( ) (1) = y t σ n, Φ σ n, σ n t < σ n+1, n N An optimal stopping problem for the quikest detetion of the Poisson disorder. In terms of the new suffiient statistis Φ (1) and Φ () in (4.5, 4.6), the Bayes risk of (2.6, 3.2) an be rewritten as R τ (π) = 1 π + (1 π) 2 E [ τ (1 π) 2 ( e λt Φ () (1) t + Φ t λ ) 2 dt, τ S. Therefore, the minimum Bayes risk U(π) inf τ S R τ (π), π [, 1) is given by ( (1 m)π (4.11) U(π) = 1 π + V, 2(1 π) ) (1 + m)π, π [, 1), 2(1 π) where m is as in (4.4), the funtion V (, ) is the value funtion of the optimal stopping problem (4.12) [ τ V (φ, φ 1 ) inf τ S Eφ,φ ( Φ() 1 e λt g t, ) (1) Φ t dt, g(φ, φ 1 ) φ + φ 1 λ 2, (φ, φ 1 ) R 2 +, Φ () and E φ,φ 1 is the onditional P -expetation given that = φ and an optimal stopping time for (4.12) is a minimum Bayes alarm time. Φ (1) = φ 1. Moreover, It is lear from (4.12) that it is never optimal to stop before the proess Φ leaves the region (4.13) C { (φ, φ 1 ) R 2 + : φ + φ 1 < λ } 2.

19 ADAPTIVE POISSON DISORDER PROBLEM 19 t t t Φ () t (ω) Φ (1) t (ω) Φ (1) t (ω) φ d Φ (1) Φ (1) Φ () (a) Φ (1) with < λ < 1 (b) Φ (1) with λ 1 () Φ () with λ > Figure 1. The sample-paths of the proesses Φ (1) and Φ (). In the next subsetion we shall disuss the pathwise behavior of the proess Φ; this will give insight into the solution of the optimal stopping problem in (4.12) The sample-paths of the suffiient-statisti proess Φ = ( Φ (), Φ (1) ). The proess Φ () jumps downwards and inreases between jumps; see (4.6) and Figure 1(). On the other hand, the proess Φ (1) jumps upwards, and its behavior between jumps depends on the sign of 1 λ. If λ 1, then the proess Φ (1) inreases between jumps; see Figure 1(b). If < λ < 1, then Φ (1) reverts to the (positive) mean-level (4.14) φ d λ(1 + m) (1 λ) 2 between jumps; it never visits φ d unless it starts there; and in this latter ase, it stays at φ d until the first jump and never omes bak to φ d later (i.e., φ d > is an entrane boundary for Φ (1) ); see Figure 1(a). Finally, note that φ d and 1 λ have the same signs. As for the solution of the optimal stopping problem in (4.12), it is worth waiting if the proess Φ is in the region C of (4.13), or is likely to return to C shortly. The sample-paths of the proess Φ are deterministi between jumps, and tend towards, or away from, the region

20 2 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS Φ (1) Φ t (ω) Φ (1) Φ t (ω) α α = 9 α λ 2 λ 2 α C C φ d λ Φ () λ 2 2 (a) Case I: λ 1 or < (λ/) 2 φ d (b) Case II: < φ d < (λ/) 2 Φ () Figure 2. The sample-paths of Φ C. These two ases are desribed separately below. In both ases, however, the proess Φ jumps in the same diretion relative to its position before the jump. A jump at (φ, φ 1 ) is an instantaneous displaement (1/µ)[ φ φ 1 T in Φ. Therefore, the jump diretion is away from (respetively, towards) the region C if φ < φ 1 (respetively, φ > φ 1 ). Along a quarter of a irle in Figure 2(a), the diretions of jumps at an equal distane from the origin are illustrated by the arrows. Note also that, along any fixed half-ray in R 2 +, the jump diretion (namely, the angle α in Figure 2(a)) does not hange, but the size of the jump does Case I: A large disorder arrival rate. Suppose that λ 1 or < (λ/) 2 φ d. Equivalently, λ [1 (1 + m)(/2) + is large. Between jumps, the proess Φ gets farther away from the region C. It may return to C by jumps only, and only if the jump originates in the region L {(φ, φ 1 ) : φ > φ 1 }; see Figure 2(a). But, if Φ (1) reahes at or above (λ/) 2, then Φ will never return to C Case II: A small disorder arrival rate. Now suppose that < φ d < (λ/) 2. Equivalently, < λ < 1 (1 + m)(/2) is small. If the proess Φ finds itself in a very lose neighborhood of the upper-left orner of the triangular region C, then it will drift into C

21 ADAPTIVE POISSON DISORDER PROBLEM 21 before the next jump with positive probability. Otherwise, the behavior of the sample-paths of Φ relative to C is very similar to that in Case I; see Figure 2(b). 5. A family of related optimal stopping problems Let us introdue the family of optimal stopping problems (5.1) [ τ σn V n (φ, φ 1 ) inf τ S Eφ,φ ( Φ() 1 e λt g t, ) (1) Φ t dt, (φ, φ 1 ) R 2 +, n N, obtained from (4.12) by stopping the proess Φ at the nth jump time σ n of the proess N. Sine g(, ) in (4.12) is bounded from below by the onstant (λ/) 2, the expetation in (5.1) is well-defined for every stopping time τ S. In fat, 2/ V n for every n N. Sine the sequene (σ n ) n 1 of jump times of the proess N is inreasing almost surely, the sequene (V n ) n 1 is dereasing. Therefore, lim n V n exists everywhere. It is also obvious that V n V, n N Proposition. As n, the sequene V n (φ, φ 1 ) onverges to V (φ, φ 1 ) uniformly in (φ, φ 1 ) R 2 +. In fat, for every n N and (φ, φ 1 ) R 2 +, we have (5.2) ( ) n 2 µ V n (φ, φ 1 ) V (φ, φ 1 ). λ + µ Proof. Let us fix (φ, φ 1 ) R 2 +. For every τ S and n N, we have E φ,φ 1 [ τ E φ,φ 1 E φ,φ 1 [ e λs g( Φ τ σn s )ds = E φ,φ 1 [ τ σn [ τ σn e λs g( Φ s )ds [ τ e λs g( Φ s )ds + E φ,φ 1 1 {τ σn} λ 2 E φ,φ 1 e λs g( Φ 2 s )ds Eφ,φ 1 [ 1 {τ σn} τ e λs ds σ n [ e λσ n 2 Vn (φ, φ 1 ) e λs g( Φ s )ds σ n ( µ ) n λ + µ We have used the bound g(φ, φ 1 ) (λ/) 2 from (4.12), as well as the fat that N is a Poisson proess with rate µ under P, and σ n is the n-th jump time of N. Taking the infimum over τ S gives the first inequality in (5.2). We shall try to alulate now the funtions V n ( ) of (5.1), following a method of Gugerli (1986) and Davis (1993). Let us start by defining on the olletion of bounded Borel funtions

22 22 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS w : R 2 + R the operators (5.3) (5.4) Jw(t, φ, φ 1 ) E φ,φ 1 J t w(φ, φ 1 ) [ t σ1 inf Jw(u, φ, φ 1 ) u [t, ( Φ() e λu g u, ) (1) Φ ( Φ() ) u du + 1 {t σ1 }e λσ 1 (1) w σ 1, Φ σ 1, for every t [,. The speial struture of the stopping times of jump proesses (see Lemma 7.1 below) implies [ τ σ1 J w(φ, φ 1 ) = inf τ S Eφ,φ ( Φ() ) 1 e λt (1) g t, Φ ( Φ() ) t dt + 1 {τ σ1 }e λσ 1 (1) (5.5) w σ 1, Φ σ 1. By relying on the strong Markov property of the proess N at its first jump time σ 1, one expets that the value funtion V of (4.12) satisfies the equation V = J V. Below, we show that this is indeed the ase. In fat, if we define v n : R 2 + R, n N sequentially by (5.6) v, and v n J v n 1 n N, then every v n is bounded and idential to V n of (5.1), lim n v n exists and equals the value funtion V in (4.12). Under P, the first jump time σ 1 of the proess N has exponential distribution with rate µ. Using the Fubini theorem and (4.1), we an write (5.3) as (5.7) Jw(t, φ, φ 1 ) = t e (λ+µ)u( g + µ w S )( x(u, φ ), y(u, φ 1 ) ) du, t [,, where x(, φ ) and y(, φ 1 ) are the solutions (4.8) of the ordinary differential equations in (4.7), and S : R 2 + R 2 + is the linear mapping (( S(φ, φ 1 ) 1 1 ) ( φ, ) ) (5.8) φ 1. µ µ 5.2. Remark. Using µ > 1 and the expliit forms of x(u, φ ) and y(u, φ 1 ) in (4.8), it is easy to hek that the integrand in (5.7) is absolutely integrable on R +. Therefore, lim Jw(t, φ, φ 1 ) = Jw(, φ, φ 1 ) <, t and the mapping t Jw(t, φ, φ 1 ) : [, R is ontinuous. The infimum J t w(φ, φ 1 ) in (5.4) is attained for every t [, Lemma. For every bounded Borel funtion w : R 2 + R, the mapping J w is bounded. If we define w sup (φ,φ 1 ) R w(φ, φ ) <, then ( ) λ 2 (5.9) λ + µ + µ λ + µ w J w(φ, φ 1 ), (φ, φ 1 ) R 2 +.

23 ADAPTIVE POISSON DISORDER PROBLEM 23 If the funtion w(φ, φ 1 ) is onave, then so is J w(φ, φ 1 ). If w 1 w 2 are real-valued and bounded Borel funtions defined on R 2 +, then J w 1 J w Corollary. Every v n, n N in (5.6) is bounded and onave, and 2/... v n v n 1 v 1 v. The limit (5.1) exists, and is also bounded and onave. v(φ, φ 1 ) lim n v n (φ, φ 1 ), (φ, φ 1 ) R 2 + Both v n : R 2 + R, n N and v : R 2 + R are ontinuous, inreasing in eah of their arguments, and their left and right partial derivatives are bounded on every ompat subset of R Proposition. For every n N, the funtions v n of (5.6) and V n of (5.1) oinide. For every ε, let rn(φ ε, φ 1 ) inf { ( ) s (, : Jv n s, φ, φ 1 J v n (φ, φ 1 ) + ε }, n N, (φ, φ 1 ) R 2 +, ( ) ( ) ) r S1 ε r( ε/2 ε Φ σ1, and Sn+1 ε n Φ, if σ1 > rn ε/2 Φ ( ) σ 1 + Sn ε/2 θ σ1, if σ 1 rn ε/2 Φ, n N, where θ s is the shift-operator on Ω: N t θ s = N s+t. Then (5.11) E φ,φ 1 [ S ε n e λt g ( ) Φt dt v n (φ, φ 1 ) + ε, n N, ε Proposition. We have v(φ, φ 1 ) = V (φ, φ 1 ) for every (φ, φ 1 ) R 2 +. Moreover, V is the largest nonpositive solution U of the equation U = J U Lemma. Let w : R 2 + R be a bounded funtion. For every t R + and (φ, φ 1 ) R 2 +, (5.12) J t w(φ, φ 1 ) = Jw(t, φ, φ 1 ) + e (λ+µ)t J w ( x(t, φ ), y(t, φ 1 )) ) Corollary. Let (5.13) r n (φ, φ 1 ) = inf { s (, : Jv n ( s, (φ, φ 1 ) ) = J v n (φ, φ 1 ) } be the same as rn(φ ε, φ 1 ) in Proposition 5.5 with ε =. Then r n (φ, φ 1 ) = inf { ( (5.14) t > : v n+1 x(t, φ ), y(t, φ 1 ) ) = } Proof. Let us fix (φ, φ 1 ) R 2 +, and denote r n (φ, φ 1 ) by r n. Jv n (r n, φ, φ 1 ) = J v n (φ, φ 1 ) = J rn v n (φ, φ 1 ). (inf ). By Remark 5.2, we have

24 24 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS Suppose first that r n <. Sine J v n = v n+1, taking t = r n and w = v n in (5.7) gives Jv n (r n, φ, φ 1 ) = J rn v n (φ, φ 1 ) = Jv n (r n, φ, φ 1 ) + e (λ+µ)rn v n+1 (x(r n, φ ), y(r n, φ 1 )). Therefore, v n+1 (x(r n, φ ), y(r n, φ 1 )) =. If < t < r n, then Jv n (t, φ, φ 1 ) > J v n (φ, φ 1 ) = J rn v n (φ, φ 1 ) = J t v n (φ, φ 1 ) sine u J u v n (φ, φ 1 ) is nondereasing. Taking t (, r n ) and w = v n in (5.7) imply J v n (φ, φ 1 ) = J t v n (φ, φ 1 ) = Jv n (t, φ, φ 1 ) + e (λ+µ)t v n+1 (x(t, φ ), y(t, φ 1 )). Therefore, v n+1 (x(t, φ ), y(t, φ 1 )) < for every t (, r n ), and (5.14) follows. Suppose now that r n =. Then we have v n+1 (x(t, φ ), y(t, φ 1 )) < for every t (, ) by the same argument in the last paragraph above. Hene, {t > : v n+1 (x(t, φ ), y(t, φ 1 )) = } =, and (5.14) still holds Remark. For every t [, r n (φ, φ 1 ), we have J t v n (φ, φ 1 ) = J v n (φ, φ 1 ) = v n+1 (φ, φ 1 ). Then substituting w(, ) = v n (, ) in (5.12) gives the dynami programming equation for the family {v k (, )} k N : for every (φ, φ 1 ) R 2 + and n N (5.15) v n+1 (φ, φ 1 ) = Jv n (t, φ, φ 1 ) + e (λ+µ)t v n+1 (x(t, φ ), y(t, φ 1 )), t [, r n (φ, φ 1 ) Remark (Dynami Programming Equation for V (, )). Sine V (, ) is bounded, and V = J V by Proposition 5.6, Lemma 5.7 gives (5.16) J t V (φ, φ 1 ) = JV (t, φ, φ 1 ) + e (λ+µ)t V ( x(t, φ ), y(t, φ 1 )) ), t R + for every (φ, φ 1 ) R 2 +; and if we define (5.17) r(φ, φ 1 ) inf{t > : JV (t, φ, φ 1 ) = J V (φ, φ 1 )}, (φ, φ 1 ) R 2 +, then arguments similar to those in the proof of Corollary 5.8, and (5.16), give (5.18) r(φ, φ 1 ) = inf{t > : V (x(t, φ ), y(t, φ 1 )) = }, (φ, φ 1 ) R 2 +, as well as the Dynami Programming equation (5.19) V (φ, φ 1 ) = JV (t, φ, φ 1 ) + e (λ+µ)t V (x(t, φ ), y(t, φ 1 )), t [, r(φ, φ 1 ) for the funtion V (, ) of (4.12). Beause t Jw(t, (φ, φ 1 )) and t J t w(φ, φ 1 ) are ontinuous for every bounded w : R 2 + R (see, e.g., (5.7)), the identity (5.16) implies that t V (x(t, φ ), y(t, φ 1 )) is ontinuous. Therefore, every realization of t V ( Φ t ) is right-ontinuous and has left-limits.

25 Let us define the F-stopping times ADAPTIVE POISSON DISORDER PROBLEM 25 (5.2) U ε inf{t : V ( Φ t ) ε}, ε. By Remark 5.1, we have (5.21) V ( ΦUε ) ε on the event {Uε < } Proposition. Let M t e λt V ( Φ t ) + t e λs g( Φ s )ds, t. For every n N, ε, and (φ, φ 1 ) R 2 +, we have E φ,φ 1 [M = E φ,φ 1 [M Uε σn, i.e., (5.22) V (φ, φ 1 ) = E φ,φ 1 [ Uε σ e λ(uε σn) V ( Φ n Uε σn ) + e λs g( Φ s )ds Proposition. For every ε, the stopping time U ε in (5.2) is ε-optimal for the problem (4.12), i.e., E φ,φ 1 [ Uε e λs g( Φ s )ds V (φ, φ 1 ) + ε, for every (φ, φ 1 ) R 2 +. Proposition 5.5 above shows how we an alulate the V n s sequentially; it also identifies expliitly ε-optimal times for every optimal stopping problem in (5.1). Together with Proposition 5.1, it suggests a way to alulate ε-optimal alarm times: Let ε 1 > and ε 2 be two arbitrary numbers suh that ε 1 + ε 2 = ε >. Let us hoose n N suh that (5.23) ( ) n 2 µ < ε 1. λ + µ Then we have V n (φ, φ 1 ) ε 1 V (φ, φ 1 ) V n (φ, φ 1 ) for every (φ, φ 1 ) R 2 +, and the stopping time S ε 2 n of Proposition 5.5 is an ε-optimal stopping time for our original optimal stopping problem (4.12) in the sense that V (φ, φ 1 ) E φ,φ 1 [ S ε 2 n e λt g ( Φt ) dt < V (φ, φ 1 ) + ε, (φ, φ 1 ) R 2 +. Sine n N satisfies (5.23), and S ε 2 n (ω) σ n (ω) for all ω Ω, setting the alarm at the nth jump of the proess N (if this has not been triggered by S ε 2 n (ω) earlier) is not in error more than ε. In this setion, we showed that the (more lassial) stopping times U ε of (5.2) are also ε-optimal for (4.12); espeially the stopping time U is optimal, see Proposition 5.12.

26 26 ERHAN BAYRAKTAR, SAVAS DAYANIK, AND IOANNIS KARATZAS 6. A bound on the alarm time We shall show that the optimal ontinuation region C = {(φ, φ 1 ) R 2 + : V (φ, φ 1 ) < } is ontained in some set (6.1) D = {(φ, φ 1 ) R 2 + : φ + φ 1 < ξ } [ ) λ + µ for a suitable ξ 2,. Therefore, the region C has ompat losure; this will be very useful in proving in the next setion that C has a stritly dereasing onvex boundary. Reall from Setion 4.1 that it is not optimal to stop before the proess Φ leaves the region C in (4.13). Thus the optimal stopping time U of Proposition 5.12 is bounded by { () (1) τ C inf t : Φ t + Φ t λ } () (1) (6.2) 2 U τ D inf{t : Φ t + Φ t ξ } the exit times τ C and τ D of the proess Φ from the regions C and D, respetively. The onstant threshold ξ in (6.1) is essentially determined by the number (λ + µ) 2/ (see (6.5), (6.9) and (6.11)), and our alulations below suggest that they are lose. Therefore, the bounds in (6.2) may prove useful in pratie. The differene [(λ + µ)/ 2 (λ/) 2 = (µ/) 2 between the thresholds that determine the latest and the earliest alarm times is also meaningful. It inreases as µ/ inreases: waiting longer is enouraged if the new information arrives at a higher rate than we pay for detetion delay per unit time when the disorder has already happened. Finally, we prove in Lemma 6.1 that τ D in (6.2) has finite expetation. Therefore, E φ,φ 1 [U E φ,φ 1 [τ D < for every (φ, φ 1 ) R 2 +. Let τ S be any F-stopping time. By Lemma 7.1, there is a onstant t suh that τ σ 1 = t σ 1 almost surely. Therefore [ τ (6.3) E φ,φ 1 e λs g ( ) Φs ds = E φ,φ 1 E φ,φ 1 = t [ τ σ1 [ t e λs g ( Φs ) ds + E φ,φ 1 [ τ 1 {τ σ1 } 1 {s σ1 }e λs g ( x(s, φ ), y(s, φ 1 ) ) ds e (λ+µ)s [ g ( x(s, φ ), y(s, φ 1 ) ) µ 2 ds. e λs g ( ) Φs ds σ 1 2 Eφ,φ 1 The inequality follows from g(φ, φ 1 ) g(, ) = (λ/) 2, see (4.12). [ 1{t σ1 }e λσ 1 The funtions x(, φ ) and y(, φ 1 ) are the solutions of (4.7) (see Remark 4.1), and σ 1 has exponential

27 ADAPTIVE POISSON DISORDER PROBLEM 27 y λ+µ 2 R 2 y y +\D 1 l R 2 +\D 1 l R 2 +\D 2 (φ, φ 1 ) λ ξ (x( t, φ ), y( t, φ 1)) 1 λ 2 λ+µ λ+µ 2 2 (x(t, φ ), y(t, φ 1 )) (φ, φ 1) λ 2 (x(t, φ ), y(t, φ 1 )) C (φ, φ 1 ) λ 2 λ+µ 2 (a) λ 1 (φ d ) x φ d λ 2 C (b) λ 2 λ+µ 2 λ 1 λ λ+µ (λ < 1) λ 2 λ 1 λ 2 φ d x C () < ξ λ λ+µ 2 2 λ 1 λ < λ+µ (λ < 1) x Figure 3. Region D distribution with rate µ under P. Clearly, if (6.4) < g ( x(s, φ ), y(s, φ 1 ) ) µ 2 = x(s, φ ) + y(s, φ 1 ) λ + µ 2, < s <, [ then (6.3) implies that E φ,φ 1 τ e λs g ( ) Φs ds > for every F-stopping time τ almost surely (sine the filtration F is right-ontinuous, the probability of {τ } F equals zero or one). Thus, stopping immediately is optimal at every (φ, φ 1 ) where (6.4) holds. If λ 1, then s x(s, φ ) and s y(s, φ 1 ) are inreasing for every (φ, φ 1 ) R 2 +, see (4.7) and Figure 3(a). Therefore, x(s, φ ) + y(s, φ 1 ) > x(, φ ) + y(, φ 1 ) = φ + φ 1 for every < s <. Hene, (6.4) holds, and therefore it is optimal to stop immediately outside the region (6.5) { D 1 (φ, φ 1 ) R 2 + : φ + φ 1 < λ + µ } 2 if λ 1. Suppose now that < λ < 1; equivalently, φ d of (4.14) is positive. Then s x(s, φ ) is inreasing for every φ R +. For φ 1 = φ d, the derivative dy(s, φ d )/ds in (4.7) vanishes for every < s <. The mapping s y(s, φ 1 ) is inreasing if φ 1 [, φ d ), dereasing if φ 1 (φ d, ), and φ(s, φ d ) = φ d for every s < ; see (4.7) and Figures 3(b,). The derivative (6.6) d [ x(t, φ ) + y(t, φ 1 ) = (λ + 1)x(t, φ ) + (λ 1)y(t, φ 1 ) + λ 2 dt