Quickest detection of drift change for Brownian motion in generalized Bayesian and minimax settings

Transcription

1 Statistics & Decisions 24, (26) / DOI /stnd c Oldenorg Wissenschaftsverlag, München 26 Qickest detection of drift change for Brownian motion in generalized Bayesian and minima settings Egene A. Feinerg, Alert N. Shiryaev Received: Novemer 17, 26; Accepted: Ferary 14, 27 Smmary: he paper deals with the qickest detection of a change of the drift of the Brownian motion. We show that the generalized Bayesian formlation of the qickest detection prolem can e redced to the optimal stopping prolem for a diffsion Markov process. For this prolem the optimal procedre is descried and its characteristics are fond. We show also that the same procedre is asymptotically optimal for the minima formlation of the qickest detection prolem. 1 Introdction and prolem formlation 1. his paper deals with the prolem of the qickest detection of a change of a drift for a Brownian motion formlated and stdied in Shiryaev [17, 18, 19] in the Bayesian and generalized Bayesian settings. Let B = (B t ) t e a standard Brownian motion defined on a filtered proaility space (, F,(F t ) t, P). Withot loss of generality we shall assme that is the space of continos fnctions ω = (ω t ) t. Everywhere in this paper, the eqality etween two random variale defined on this proaility space means that these random variales are eqal P-a.s. Sppose we oserve a stochastic process X = (X t ) t that has the following strctre: or, eqivalently, dx t = X t = µ(t θ) + + σb t (1.1) { σ db t, t <θ, µ dt + σ db t, t θ, with X =. Here µ and σ are known nmers, where µ = andσ>, and θ is an nknown time; θ [, ]. We interpret θ< as a time when the disorder appears, and θ = means that the disorder never happens. he appearance of a disorder shold e detected as soon as possile trying to avoid false alarms. AMS 2 sject classification: Primary: 6G35, 6G4; Secondary: 93E1 Key words and phrases: Brownian motion, disorder, generalized Bayesian and minima formlations of the qickest detection prolem, optimal stopping, asymptotical optimality his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

2 446 Feinerg -- Shiryaev Let P θ = Law(X θ) e the distrition of the process X nder the assmption that the disorder happened at the deterministic time θ. In particlar, P is the distrition of X nder the assmption that the disorder never happened, i.e. P = Law(σB t, t ), and P is the distrition of the process µt + σb t, t. Let τ = τ(ω) e a finite stopping stopping time with respect to the filtration F X = (Ft X ) t generated y the process X t. We interpret τ as the decision that the disorder has happened at the time τ(ω). If the system is controlled y a stopping time τ, occrrences of false alarms can e characterized in several ways. For eample, they can e characterized either y the proaility of the false alarm α θ = P θ (τ < θ), or y the mean time ntil the false alarm = E τ, θ [, ), or y their cominations, where E θ denotes the epectation with respect to the proaility P θ,θ [, ]. 2. Recall the following two variants (A) and (B) of the prolem of qickest detection, the so-called Bayesian and generalized Bayesian formlations, proposed in [17, 18, 19]. Variant (A). Sppose that θ is a random variale, θ = θ(ω), independent of B = (B t ) t and having an eponential distrition with an atom at, P(θ = ) = π, P(θ > t θ>) = e λt, where π [, 1) and λ>are known constants. For a constant α (, 1) denote M (α) ={τ : P(τ < θ) α}. Variant (A) of the qickest detection prolem is to find, for agivenα (, 1), a stopping time τ(α), if it eists, sch that E(τ (α) θ τ (α) θ) = inf E(τ θ τ θ). τ M (α) Variant (B). According to this variant, θ is a parameter in [, ] rather than a random variale considered in Variant (A). For every > we denote y M ={τ : E τ = } the set of stopping times with the P -mean. Variant (B) is, for a given (, ), to find a stopping time τ, if it eists, sch that E θ (τ θ)+ dθ = inf τ M E θ (τ θ) + dθ. (1.2) his variant of the qickest detection prolem is called generalized Bayesian, sinceθ can e interpreted as a generalized random variale with the niform distrition on [, ). his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

3 Qickest detection 447 Remark 1.1 It is natral to consider a igger class M ={τ : E τ< } M andtotrytofindapolicy τ sch that E θ ( τ θ)+ dθ = inf E θ (τ θ) + dθ. (1.3) τ M However, we shall see in Corollary 4.3 elow that the infima on the right-hand sides of eqations (1.2) and (1.3) are eqal and therefore there is no advantage in considering the class M instead of the class M. As was shown in [17, 18, 19], there is an optimal detection stopping time for Variant (A) and it can e descried in the following way. Let ( ) π t = P θ t Ft X e the a posteriori proaility that the disorder has appeared efore time t. In particlar, π = π. he stopping time τ (α) = inf{t : π t 1 α} (1.4) is optimal in the class M (α). he process (π t ) t satisfies the stochastic differential eqation ) dπ t = (λ µ2 σ 2 π2 t (1 π t ) dt + µ σ 2 π t(1 π t ) dx t with π = π; see [17, 18, 19] for details. Set According to Itô s formla with ϕ = π /(1 π ). Let ϕ t = π t 1 π t. dϕ t = λ(1 + ϕ t ) dt + µ σ 2 ϕ t dx t (1.5) L t = d ( P Ft X ) d ( P Ft X ) e a Radon Nikodým derivative, also called the likelihood, of P Ft X with respect to P Ft X. It is well known [9, 21] that L t = e µ σ 2 X t 1 µ 2 2 σ 2 t his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

4 448 Feinerg -- Shiryaev and y Itô s formla dl t = µ σ 2 L t dx t, L = 1. (1.6) By sing eqation (1.6) and applying Itô s formla to the fnction ( λe λt ) ( ) t d L t, e λ L we find the following representation for the strong soltion ϕ t of eqation (1.5): when ϕ =. In a general case, Denote hen, if ϕ =, we see from (1.7) that and from (1.5) we find that t e λt L t ϕ t = λ e λ d, (1.7) L t ϕ t = ϕ e λt e λt L t L t + λ e λ d. L ψ t (λ) = ψ t (λ) = ϕ t λ. t e λt L t e λ L d (1.8) dψ t (λ) = (1 + λψ t (λ)) dt + µ σ 2 ψ t(λ) dx t. (1.9) Using the processes (ϕ t ) t and (ψ t (λ)) t, the stopping time τ(α) defined y (1.4) can e presented as { τ(α) = inf t : ϕ t 1 α } α and, in the case π = ϕ =, { τ(α) = inf t : ψ t (λ) 1 α }. (1.1) αλ Following [17, 18, 19], let and where is a fied constant. λ, α 1, (1.11) 1 α λ >, (1.12) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

5 Qickest detection 449 Eqations (1.8) and (1.9) imply that nder (1.11) and (1.12) the limit eists, allows the representation and satisfies the eqation ψ t = lim λ ψ t (λ) ψ t = t L t L d, (1.13) dψ t = dt + µ σ 2 ψ t dx t, ψ =. (1.14) Moreover, nder (1.11) and (1.12), we define the stopping time τ = lim τ α 1, (α). λ his stopping time has the following representation: τ = inf{t : ψ t }. (1.15) It is interesting and important for or frther considerations to notice that E τ =. (1.16) Indeed, nder the measre P the differential eqation (1.14) has the form Since ψ t τ So, from (1.17) dψ t = dt + µ σ ψ t db t, ψ =. (1.17), y the martingale properties of stochastic integrals, E ψ t db t =. = E ψ τ = E τ. In other words, = lim 1 α λ, where the limit is taken as indicated in (1.11) and (1.12), has a simple meaning of the epected time ntil the process (ψt ) t reaches the level, taken nder the assmption that a disorder never happens. It was shown in [17, 18, 19] that, nder the limits (1.11) and (1.12), the optimal stopping times τ(α) for the Bayesian Variant (A) converge to an optimal stopping time for the generalized Bayesian prolem formlated in Variant (B). Namely, the stopping time τ = lim α 1,λ τ(α) is optimal in the sense of (1.2). 3. One of the main reslts of the present paper is the direct proof in Section 2 of the optimality of the stopping time τ = inf{t : ψ t } in Variant (B). his his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

6 45 Feinerg -- Shiryaev direct proof clarifies why the process (ψ t ) t is a sfficient statistics for this prolem. (For the discrete time case, the corresponding method is called the Shiryaev Roerts procedre.) his direct proof is also sefl for or analysis in Section 3 of the qickest detection prolem in the following formlation. Variant (C). If it eists, in the class M ={τ : E τ = } find a stopping time σ for which sp E θ (σ θ σ θ) = θ inf sp E θ (τ θ τ θ), (1.18) τ M θ where, similarly to Variant (B), θ [, ). Althogh optimal policies for this minima prolem are still nknown, we shall see in Section 3 that or approach, ased on the reslts for Variant (B), implies that, at least for large, the stopping time τ = inf{t : ψ t } is asymptotically optimal. We remark that asymptotic optimality has een estalished in the statistical literatre for many change point models; see, for eample, collection of papers [2]. In addition to asymptotic optimality, or approach leads to two-sided ineqalities (3.5) for the vale fnction on the right-hand side of (1.18). hese ineqalities provide estimates how close the performance of the asymptotically optimal stopping time is to the optimal vale. We shall discss another interesting and poplar minima formlation of the qickest detection prolem, Variant (D), in Section 5. 2 Variant (B) 1. Since for any stopping time τ we oserve that (τ θ) + = θ I( τ) d, L τ E I( τ) d = E L θ θ L L θ d, E θ (τ θ) + = E θ I( τ) d = θ θ (2.1) where the second eqality in (2.1) holds since for θ ( dp θ d Pθ F X ) E θ I( τ) = E I( τ) = E dp d ( P F X )I ( τ) (2.2) L L = E I ( τ) = E I ( τ). L θ L θ Here the first eqality in (2.2) follows from the property P θ P, the second eqality follows from { τ} F X, and the last eqality follows from the property P (A) = P (A) for all A F X. o prove the third eqality in (2.2), we notice that for θ d ( P θ F X ) d ( P F X ) = d ( P θ F X ) d ( P F X ) = d ( P θ F X ) d(p F X ) d(p F X ) d(p F X ) = L d(p F X ) d(p θ F X ) = L L θ, his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

7 Qickest detection 451 where the eqality d ( P F X ) d ( P θ F X ) = L θ (2.3) follows from the specific property of the model (1.1) that P F X is the measre of the process X t = µt + σb t, t, and P θ F X is the measre for the process X = t µi(s θ) ds + σb t, t. Since the increments of these two processes coincide on the time interval [θ, ], the Radon Nikodým derivative of the measre for the first process with respect to the measre for the second process is eqal to the Radon Nikodým derivative of the measre for the process X t = µt+σb t, t θ, with respect to the measre for the process X t = σb t, t θ. hs, d ( P F X ) d ( P θ F X ) = d ( P Fθ X ) d ( P θ Fθ X ) = L θ. We remark that formla (2.3) also follows from direct calclations of the Radon Nikodým derivative in (2.3) y applying, for eample, the Corollary from [9, heorem 7.18]. After integrating in θ the first and the last epressionsin (2.1), we find that [ ] E θ (τ θ) + L dθ = E d dθ L θ where [ = E Hence, the following statement holds. ψ = Lemma 2.1 (a) For any stopping time τ L L θ dθ θ ]d = E ψ d, L L θ dθ. (2.4) E θ (τ θ) + dθ = E ψ d. (2.5) () he vale 1 B( ) = inf E θ (τ θ) + dθ τ M of the generalized Bayesian prolem eqals to the vale of the conditional-etremal optimal stopping prolem for the process (ψ t ) t : B( ) = 1 inf τ M E ψ d. (2.6) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

8 452 Feinerg -- Shiryaev In Section 4 we shall show that the optimal stopping time for this prolem is τ, defined in (1.15), that prescries to stop when the process (ψ t ) t hits the level. So, B( ) = 1 E 2. o compte E ψ d, we introdce the fnction U() = E () ψ d. (2.7) ψ d, (2.8) where E () is the epectation for the distrition P () for the Markov process (ψ t ) t satisfying the stochastic differential eqation (1.17) with ψ =, [, ). he infinitesimal operator L of this Markov process is L = + ρ2 2 2, >, with the signal to noise ratio ρ = µ2. Note, y the way, that the Kllack Leiler 2σ 2 divergence E log d( P F X ) d(p F X ) etween the measres P F X and P F X is eactly ρ. he fnction U = U(), (, ), satisfies Kolmogorov s ackward eqation i.e. L U() =, U + ρ 2 U = ; (2.9) see [3] for details on this eqation. o compte the fnction U = U(), [, ], we introdce for the fnction where F() = e ( Ei( )), (2.1) Ei( ) = is the integral eponential fnction [4, 5]. Let G() = e d, F() 2 d. (2.11) Soltions of the prolems considered in this paper depend on µ and σ only via the signal to noise ratio ρ = µ2. Withot loss of generality we shall primarily consider 2σ in this paper the case ρ = 1 and 2 formlate statements of theorems and lemmas for this case. In Section 4 we shall eplain how the major characteristics shold e recalclated when ρ = 1. Under ρ = 1 we have the following reslt. his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

9 Qickest detection 453 Lemma 2.2 For all > and [, ] In particlar, for > and = U() = G ( 1 U() = G ) G Proof: According to (1.16), E τ =. hs, for all. (2.12). (2.13) U() = E () ψ s ds E () τ E() τ = E τ = 2. herefore, U() is a onded soltion of eqation (2.9) with the ondary condition U( ) =. It is easy to find all the onded soltions of eqation (2.9). hese soltions are U() = C 1 1 e 2 ( e z z ) dz d = C 1 1 F() 2 d = C 1 G. he ondary condition U( ) = implies C 1 = G( 1 ). his gives (2.12). Since G(z) asz, (2.12) implies (2.13). 3. Set = 1. By (2.13), U() = G(). Integrating (2.11) y parts, we find G() = F() 2 d = wherewesedtheformla that follows directly from (2.1). Formla (2.14) implies that where Since Ei( ) = F() d 1 = F() F () = F() 1, + F() d 1, (2.14) G() = F() (), (2.15) F() () = 1 d. (2.16) e t t dt = e e t + t dt his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

10 454 Feinerg -- Shiryaev and F() = e ( Ei( )),wefindthat F() e ( Ei( )) [ e t ] d = d = ( + t) dt d [ ] = e t d dt = e t log ( 1 + t ) log(1 + ) dt = e d. ( + t) t Hence, aking into accont that B( ) = 1 E () = 1 log(1 + ) e d. (2.17) ψ d = 1 U() = 1 G = G(), we have that formlae (2.15) (2.17) imply the following reslt. heorem 2.3 In the Variant (B), B( ) = F() (),where= 1/, and ths [ ] [ F() ] log(1 + ) B( ) = F() 1 d = F() 1 e d. (2.18) 4. Let s stdy the asymptotics of B( ) for the case ρ = 1when and when. We recall that the policy τ defined in (1.15) is optimal in the class M. heorem 2.4 In the Variant (B), { ( ) log 1 C + O log 2 B( ) =,, 2 + O( 2 ),, where C = is Eler s constant. (2.19) Proof: We shall se the following known formlas [8, (3.1.6) and (3.2.4)] for the integral eponential fnction: for all > Ei( ) = log 1 C + ( 1) n+1 n n n! n=1 and for large > and any n 1 [ 1 n ( Ei( ) = e + ( 1) k k! 1 k+1 + O k=1 n+1 (2.2) )]. (2.21) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

11 Qickest detection 455 First, we consider the case of small,i.e. is large. By (2.21), with n = 1, we have F() = e ( Ei( )) = 1 + O ( 1 2 ) = + O( 2 ). (2.22) Letsshownowthatfor () = 1 ( O 2 ). (2.23) Formla (2.23) is important ecase (2.22) and (2.23) imply that for B( ) = F() () = O 2 = 2 + O( 2 ). (2.24) o prove (2.23), we write log(1 + ) e d (2.25) = 1 log(1 + ) e d + 2 Since log(1 + ) log(2 1) for 2, 2 log(1 + ) e d 1 1 log(1 + ) e d + 2 log(1 + ) e d. log(2 1) e d = 1 [ ( Ei 2, (2.26) 2 2)] where the last epression follows from the Laplace transform epression [4, p. 148] of the fnction {, < < 1, f() = (2.27) log(2 1), 1. From (2.26) and from (2.21) with n =, for log(1 + ) e d 1 [ ( Ei )] 2 [ ( )] ( ) = 2e O = o 2. (2.28) Net, for 2 1 log(1 + ) e d e 2 1 log(1 + ) d < e = o 2. (2.29) Also, since 1 log(1 + ) = O(2 ),for< 1andfor 1 log(1 + ) e d 1 [ e 1 ] 2 + O(2 ) d = 1 1 ( O 2 ). (2.3) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

12 456 Feinerg -- Shiryaev Hence, for, formlae (2.25) (2.3) imply () = 1 log(1 + ) e d = O 2. (2.31) hs, (2.23) and therefore (2.24) are proved. Finally, consider the case of large,i.e. is small. By (2.1) and (2.2), the first term in (2.18) has the following asymptotic as : F() = e ( Ei( )) = log 1 ( C + O log 1 ) = log C + O ( log ). (2.32) he remaining part of the last epression in (2.18) can e estimated in the following way. Since log(1 + ) for, for 2 2 log(1 + ) e d 2 ( 1 e d 2 = O() = O ). (2.33) From (2.26) and (2.2) we have for log(1 + ) e d [ ( Ei 2 ( = O log 2 2)] 2 1 ) ( ) log 2 = O. (2.34) Hence, from (2.31) (2.34) we conclde that log(1 + ) B( ) = F() () = F() 1 + e d ( = F() 1 + O() + O log 2 1 ) ( ) log 2 = log 1 C + O. 3 Variant (C) 1. he qickest detection prolems in Variant (C) are poplar in the statistical and qality control literatre, especially in the case of discrete time; see [2, 6, 22] and references therein. We investigate a continos-time version of Variant (C) for scheme (1.1) in this section. he following theorem provides pper and lower estimates for C( ) = inf τ M sp θ E θ (τ θ τ θ). heorem 3.1 For > and for τ = inf{t : ψ t } where C ( ) = E τ. B( ) C( ) C ( ), (3.1) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

13 Qickest detection 457 Lemma 3.3 elow implies that C ( ) = E τ = F = F(). (3.2) heorems 2.3, 2.4, 3.1 and formla (3.2) imply the following reslt. heorem 3.2 (a) he following lower and pper estimates hold for the fnction C( ): for all > F() () C( ) F(), (3.3) where = 1. () For large (i.e. is small) ( log 2 ) ( log log 1 C + O = B( ) C( ) C 2 ) ( ) = log C + O, (3.4) which implies for large ( log C 2 ) ( ) 1 + O C( ) C ( ). (3.5) (c) For small (i.e. is large) 2 + O( 2 ) C( ) + O( 2 ), (3.6) which implies for small C ( ) + O( 2 ) C( ) C ( ) he rest of this section is the proofs of heorems 3.1 and 3.2. Proof of heorem 3.1: Denote B θ (τ) = E θ (τ θ τ θ). hen C( ) = inf sp B θ (τ). τ M θ Since E I(θ τ) = E θ I(θ τ) for any stopping time τ, then for any θ and any τ M sp B θ (τ) E I(θ τ) B θ (τ) E θ I(θ τ) = E θ (τ θ) +. (3.7) θ Since E I(θ τ) = P (τ θ), y integrating in θ the first and the last epressions in (3.7), we find sp B θ (τ) E τ E θ (τ θ) + dθ. θ his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

14 458 Feinerg -- Shiryaev Since E τ = for all τ M,wehaveforτ M and sp θ B θ (τ) 1 θ E θ (τ θ) + dθ C( ) = inf sp B θ (τ) inf 1 E θ (τ θ) + dθ = B( ), τ M τ M i.e. the left ineqality in (3.1) is proved. Now we shall prove the right ineqality in (3.1). Consider the stopping time τ defined in (1.15). Since τ M, C( ) sp E θ (τ θ τ θ). θ Under the measre P, the Markov diffsion process (ψ t ) t satisfies (1.14) with dx t = µ dt + σ db t, X =. (3.8) For A we consider stopping times τ A = inf{t A : ψ t A}. hen for each s wehave E s [( τ A s ) I ( τ A s)] = E s [( τ A θ s ) I ( τ A s )], where θ s is the right shift y s operator, i.e. (τa θ s)(ω) = τa (θ sω) and θ s ω means the shifted trajectory (ω +s ) for ω = (ω ). We denote y P () the distrition of the process (ψ t ) t nder the assmptions that a disorder takes place at time θ = andψ =, where. We also denote y E () the epectation operator with respect to the measre P () ; E() = E. Since {τa s} ={τ A s, ψ s A}, y the Markov property of the homogeneos process (ψ t ) t and (3.8), we find that for s A E s (τa s τ A s) = ( E s τ A θ s τa,ψ s = ) ( P s ψs d τa s) = = A A ( ) E () ( τ A P ψs d τa s) ( ) E () ( τ A P ψs d τa s) E τa, ( ) where τa τ A P () -a.s. for any [, A]. Hence, sp E θ (τ θ τ θ) = E τ θ his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

15 Qickest detection 459 and C( ) = inf sp τ M θ E θ (τ θ τ θ) sp E θ (τ θ τ θ) = E τ = C ( ). θ 3. Recall the fnction F defined in (2.1). Lemma 3.3 Let ρ = 1. he fnction has for all the representation In particlar, V() = F V() = E () τ V() = F F.. (3.9) Proof: With respect to the measre P (), the Markov diffsion process (ψ t) t is defined y (1.14) with ψ = and (3.8). Hence, for < < the fnction V() = E () τ satisfies Kolmogorov s ackward eqation (1 + 2ρ)V + ρ 2 V = 1. (3.1) In addition, V() = for. Also, it is easy to see that E () τ E() τ. Let s show that E () τ =. Indeed, with respect to the measre P(), t ψ t = + t + ρ ψ s db s. (3.11) Since ψ s for all s τ, we find y taking the epectation in (3.11) with respect to the measre P () that E () τ =. hs, we have that E () τ E() τ =. his implies that V() is onded and we mst consider only onded on [, ] soltions of the eqation (3.1). It is easy to check that all sch soltions have the following form: where C 1 is a constant. V() = (C 1 ) + e ρ 1 e 1 ρ d, his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

16 46 Feinerg -- Shiryaev he condition V( ) = leads to By setting y = 1 ρ,weget In addition, a e d = 2 V() = + e 1 ρ a e ρ 1 d = 1 ρ 1 ρ e d( 1 ) = e a a When ρ = 1, formlae (3.12) (3.14) imply (3.9). e ρ 1 d e ρ 1 e ρ 1 d. (3.12) e d. (3.13) 2 e a d = e a ( Ei( a)). (3.14) a Proof of heorem 3.2: he ineqalities (3.3) follow directly from (3.1), (3.2), and heorem 2.3. he ineqalities ( ) ((3.4) ) and (3.6) follow from the asymptotics (2.19) for the fnction B( ) = F Conditional-etremal prolem 1. he conditional-etremal prolem is to find, for a given >, a stopping time τ M, if it eists, sch that E ψ s ds = inf E ψ s ds. τ M Recall that according to Lemma inf τ M E ψ s ds = B( ), 1 where B( ) = inf τ M E θ (τ θ) + dθ. o solve this prolem, we se the traditional method of Lagrange mltipliers. In or case this means that we shold solve first the following etremal prolem: find [ ] inf E ψ s ds cτ, (4.1) where the infimm is taken over the class of all stopping times τ with E τ<. he constant c >, called a Lagrange mltiplier, can e interpreted as the oservation cost per nit time. 2. With respect to the measre P, the process (ψ t ) t satisfies the stochastic differential eqation (1.17). his process possesses the following properties: (i) it is a his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

17 Qickest detection 461 nonnegative smartingale, and (ii) it is a Markov diffsion process. o solve (4.1) we se a Markovian approach; see [19, 13] for details on the Markovian approach in the optimal stopping theory. Let P () e a distrition of the process (ψ t ) t defined y the stochastic differential (1.17) with the initial condition ψ =. Let E () e the epectation operator with respect to this measre. Denote Sc () = inf E() (ψ s c) ds, (4.2) where inf is taken over all τ M with E () τ<. According to the general theory of optimal stopping [19, 13], there eists an optimal stopping rle for the prolem (4.1) and this optimal stopping rle τ (c) has the form τ (c) = inf{t : ψ t (c)}, (4.3) where (c) is a nonnegative nmer. he oservations shold e contined when ψ t [, (c)), and the oservations shold e stopped when ψ t [ (c), ). Moreover, L Sc () = + c, [, (c)), with L = d d d2 + ρ2 and S d 2 c () = if (c). It is clear that Sc (), since the process can e stopped at time. Since (ψ s c) < when ψ s < c, it is easy to see from (4.2) that (c) c. (4.4) Asymptotic vales of (c) for c andforc are presented in Lemma 4.6. According to the Markovian approach in the theory of optimal stopping [19, 13], to find the fnction Sc (), it is necessary to solve the following Stefan prolem with an nknown ondary (c): L S c () = + c, < (c), S c () =, (c), (4.5) S c () =, = (c). Let s nderline that the nknown elements in (4.5) are the fnction S c () and the ondary point (c). Bonded soltions of the eqation L S c () = + c have the form (we consider ρ = 1 for simplicity) S() = C 1 + c G, where G is defined in (2.11). he conditions S c ((c)) = ands c ((c)) = provide a possiility to find the constant C 1 and the vale of (c) y solving F = c (4.6) (c) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

18 462 Feinerg -- Shiryaev and C 1 = G c(c). (c) herefore, the onded soltion S c () of (4.5) has the following strctre: [ ) S c () = G G In particlar, ( 1 (c) ( 1 S c () = G c(c). (c) )] + c( (c)). (4.7) he soltions S c () and (c) from (4.7) and (4.6) are eactly the optimal vale Sc () and the optimal ondary (c) for the optimal stopping prolem (4.1) respectively. his follows from the general theory of optimal stopping for Markov processes [19, 13]. he optimality of the stopping time τ (c) defined in (4.3) with 1 (c) sch that F( (c) ) = c can also e estalished withot sing the general theory. In fact, the optimality of this stopping time can e proved y sing the otained soltion of the free ondary prolem (4.5) and y applying the verification theorems and Itô s formla in the same way as in [21, Chapter 7, Section 2a] or in [13, heorem 22.1 in Chapter 6]. hs, we have the following reslt. heorem 4.1 Consider the optimal stopping prolem (4.1) for ρ = 1. hen the optimal vale fnction is [ ( ) ( )] 1 1 S () = G G + c( (c)), (c) where (c) is a niqe root of the eqation F = c. (4.8) (c) he optimal stopping time is τ (c) = inf{t : ψ t (c)}. 3. heorem 4.1 implies that for each c > the stopping time τ (c) is optimal in the class M. his means that for any stopping time τ M with E () τ< E () (c) (ψ s c) ds E () (ψ s c) ds. (4.9) ( ) For a given constant >, let s set now c = c,wherec = F 1. Eqation (4.8) implies that (c ) =. herefore, τ (c ) M and (4.9) yields that for any τ M his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

19 Qickest detection 463 with E () τ< E () (c ) ψ s ds E () = If E () τ<, i.e. τ M, then (4.1) implies E () (c ) ( ) ψ s ds + c E () τ (c ) E () τ ψ s ds + c ( E () τ ). (4.1) ψ s ds E () ψ s ds. (4.11) hs, we have the following corollary from heorem 4.1. ( ) Corollary 4.2 Let c = F 1. he stopping time τ (c ) M is optimal within the class M, i.e. the ineqality (4.11) holds for any τ with E () τ<. Corollary 4.2 and (2.5) imply the following statement. Corollary 4.3 For any > inf τ M 4. According to heorem 3.1, E θ (τ θ) + dθ = inf E θ (τ θ) + dθ. τ M B( ) C( ) C ( ), (4.12) where C( ) = inf τ M sp θ E θ (τ θ τ θ) and C ( ) = E τ. he following theorem states the similar ineqalities for the class M. heorem 4.4 For any > where C( ) = inf τ M sp θ E θ (τ θ τ θ). B( ) C( ) C ( ), (4.13) Proof: he right ineqality in (4.13) follows from the right ineqality in (4.12) and from M M. For the proof of the left ineqality in (4.13), note that for any stopping time τ sp E θ (τ θ τ θ) E τ = θ = θ sp E θ (τ θ τ θ ) P θ (τ > θ) dθ E θ (τ θ τ θ) P θ (τ > θ) dθ E θ (τ θ) + dθ = E ψ d, his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

20 464 Feinerg -- Shiryaev where the last eqality follows from (2.4). herefore, for > inf sp τ M θ Since τ M = a M +a, inf τ M E ψ s ds = inf E τ a E θ (τ θ τ θ) ψ s ds E inf τ M +a E τ inf τ M E ψ s ds E τ. (4.14) = inf B( + a) = B( ), (4.15) a where the second eqality in (4.15) follows from (2.6), and the last eqality follows from the monotonicity of the fnction B stated in Lemma 4.5 elow. Hence, (4.14) and (4.15) imply the left ineqality in (4.13). Lemma 4.5 he fnction B( ), >, is increasing. Proof: Consider the fnction F() = e e that F() <1/. Indeed, this ineqality is eqivalent to rewritten as d introdced in (2.1). We oserve e t t t e t dt < 1. he last ineqality holds ecase t e t dt = + e d < dt < e, which can e e d = 1. According to heorem 2.3, B( ) = F() (),where = 1/. o complete the proof, we shall show that the fnction F() () is decreasing. Indeed, we have F () F() () = d 1 ( F() = 1 ) 2 d <, where the last ineqality follows from F() <1/. 5. For large and small vales of c >, the following statement provides the asymptotics for the threshold (c), that defines y eqation (4.3) the optimal stopping time τ (c), and for the vale of Sc () defined in (4.2). Lemma 4.6 (a) For c, and () For c, (c) = c + c 2 + O(c 3 ) (4.16) S c () = 1 2 c2 + O(c 3 ). (4.17) (c) e c+c (4.18) his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

21 Qickest detection 465 and where C = is Eler s constant. S c () ec+c, (4.19) Proof: (a) According to (4.8), the threshold (c) is the soltion of the eqation F(1/ (c)) = c. Let = 1/ (c). hen (2.22) implies that when c. herefore (c) and, in view of (2.1) and (2.21), ( ) F() = e ( Ei( )) = O 1. (4.2) 2 3 hs for small c > or eqivalently and therefore c = (c) ( (c)) 2 + O ([ (c)] 3) ( (c) = c + ( (c)) 2 + O [ (c)] 3) (4.21) ( ( (c)) 2 = c 2 + O [ (c)] 3). (4.22) Eqations (4.21) and (4.22) imply (4.16). o find the asymptotic for Sc () when c, we notice from heorem 4.1 that Sc () = G c (c), (4.23) (c) where G() = F() 2 he last eqation and (4.2) imply that for large G() = O 3. (4.24) d. From (4.23), (4.24), and (4.16), we have [ ( Sc () = 12 [ (c)] 2 + O [ (c)] 3)] c (c) = 1 2 (c + c2 ) 2 c(c + c 2 ) 2 + O(c 3 ) = 1 2 c2 + O(c 3 ). his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

22 466 Feinerg -- Shiryaev () Let c. hen the eqation F(1/ (c)) = c and (2.2), (2.22), and (2.32) imply that (c) and ( log log ) (c) (c) C + O = c. (4.25) (c) hs for c (c) e c+c. (4.26) o find the asymptotic of Sc () for c, we rewrite (4.23) as [ ( ) ] 1 1 Sc () = (c) (c) G c. (4.27) (c) ( ) As follows from (2.7), (2.8), and (2.13), B( ) = 1 G 1 and, according to (2.19), for large ( log 2 ) B( ) = log 1 C + O. Hence for large (c) ( ) ( 1 1 log (c) G = log 2 ) (c) (c) 1 C + O (c) (c) and (4.27) and (4.25) imply [ ( log Sc () = (c) log 2 (c) (c) 1 C + O (c) [ ( log = 2 (c) (c) 1 + O Formlae (4.28) and (4.26) imply (4.19). (c) 6. Consider asymptotics for (c) fond in Lemma 4.6. ) ] c )]. (4.28) Remark 4.7 Formla (4.16) shows that (c) insignificantly eceeds c when c is small: (c) c c 2. However, if c is large, (c) e c+c. At the first glance, this reslt seems strange ecase, when the vales of ψ s are close to (c), the vales of ψ s c are large and their contritions to the ojective fnction are also positive, while it appears that for an optimal policy these contritions shold e negative. In fact, there is no contradiction here, since the process (ψ t ) t is positive recrrent with respect to the measre P,i.e. E () σ < for any for any >, where σ = inf{t > : ψ t = }. For the first time, this was pointed ot y Pollak and Siegmnd [14], where it was also noticed that this process has an invariant distrition F(y) = lim t P () (ψ t y), y >, for any initial state. his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

23 Qickest detection 467 o find this distrition, we write Kolmogorov s forward eqation for the density f(t, y) of the distrition F(t, y) = P () (ψ t y), f t = f y + ρ 2 y 2 (y2 f ), y >. herefore, the density f = f(y) of the invariant distrition F = F(y) satisfies the eqation df dy = ρ d2 dy 2 (y2 f ), y >, whose nonnegativesoltion with the condition F(+) = (which follows from positivity a.s. of ψ t for all t > ) and natral condition F( ) = 1, as it is easy to find, has the form f(y) = 1 y 2 e 1/(ρy), y > ; see [15, and ]). herefore the invariant distrition F = F(y) is given y the formla F(y) = e 1/(ρy), y >. his is the Fréchet-type distrition, which is well known in the theory of etreme vale distritions [7]. 5 Comparison of two minima variants 1. Recall that in Variant (C) the vale of the criterion is defined as C( ) = inf sp E θ (τ θ τ θ). τ M θ In the statistical literatre there are many investigations of another minima criterion presented in Variant (D). Variant (D). In the class M ={τ : E τ = } find a stopping time σ,ifit eists, sch that sp θ ess sp E θ ((σ θ)+ F θ )(ω) = ω inf sp τ M θ ess sp E θ ((τ θ) + F θ )(ω). (5.1) ω aking ess sp of the conditional epectations E θ ( F θ )(ω) essentially means that we optimize for the worst possile sitation at the time θ when the disorder happens. For discrete time, criterion (5.1) was introdced y Lorden [1] who proved that the so-called CUSUM method of Page [12] is asymptotically optimal for this criterion when. Later Mostakides [11] and Ritov [16] proved that the CUSUM method his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

24 468 Feinerg -- Shiryaev is indeed optimal. he continos time model (1.1) was investigated y Beiel [1] and Shiryaev [2]. hey proved that the eponential CUSUM process γ t = sp θ t L t L θ, t, (cf. ψ t = t L t L θ dθ) is the sfficient statistics and the stopping time σ = inf{t : γ t D},whereDisthe root of D 1 log D =, is optimal in the class M ; we recall that here ρ = 1. Denote and recall that D( ) = B( ) = C( ) = inf sp τ M θ 1 inf τ M inf ess sp E θ ((τ θ) + F θ )(ω) ω sp τ M θ E θ (τ θ) + dθ, E θ (τ θ τ θ), C ( ( ) = sp E θ τ θ τ θ) = E τ. θ he following ineqalities smmarize for any ρ> the relationship etween B( ), C( ), C ( ), and D( ): For large B( ) C( ) C ( ), B( ) C( ) D( ). B( ) = 1 [ ( ) ] log 2 ρ log(ρ ) 1 C + O, ρ ρ C ( ) = 1 [ ( ) ] log 2 ρ log(ρ ) C + O, ρ ρ D( ) = 1 [ ( )] 1 log(ρ ) 1 + O, ρ ρ (5.2) (5.3) where C = is Eler s constant; see (2.19) for B( ), (3.4) and (3.6) for C( ) and C ( ), and [1, 2] for D( ). he ineqalities (5.2) and asymptotics (5.3) imply that C ( ) 1 ρ + O ( log ρ ρ ) C( ) C ( ), which shows that the stopping time τ is asymptotically optimal for Variant (C). Note that D( ) C ( ) for large. As follows from the definitions of C( ) and D( ), it is his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

25 Qickest detection 469 always tre that C( ) D( ). So, for large we have not only the ineqality C( ) 1 [ ( )] log 2 ρ log(ρ ) C + O, ρ ρ t a slightly etter ineqality C( ) 1 ρ [ log(ρ ) 1 + O ( )] log 2 ρ follows from (5.2) and (5.3). In connection to formlas (5.3), it is sefl to remark that ρ has the dimension sec 1 and ρ is dimension free. Acknowledgements. his research was partially spported y NSF (National Science Fondation) Grants DMI-3121 and DMI-6538, y a grant from NYSAR (New York State Office of Science, echnology, and Academic Research), and y RFBR (Rssian Fondation for Basic Research), Grant a. References [1] M. Beiel. A note on Ritov s Bayes approach to the minima property of the CUSUM procedre. Annals of Statistics, 24: , [2] E. Carlstein, H.-G. Müller, and D. Siegmnd, editors. Change-Point Prolems. Institte of Mathematical Statistics, IMS Lectre Notes MonographSeries 23, [3] E. B. Dynkin. Markov Processes. Volmes I, II. Springer-Verlag, [4] A. Erdélyi, W. Magns, F. Oerhettinger, and F. G. ricomi. ales of Integral ransforms. Volme I. McGraw-Hill, [5] I. S. Gradshteyn and I. M. Ryzhik. ales of Integrals, Series, and Prodcts. Academic Press, [6] D. M. Hawkins and D. H. Olwell. Cmlative Sm Charts and Charting for Qality Improvement. Springer, [7] S. Kotz and S. Nadarajah. Etreme Vale Distritions. Imperial College Press, 2. [8] N. N. Leedev. Special Fnctions and heir Applications. Prentice-Hall, [9] R. Sh. Liptser and A. N. Shiryaev. Statistics of Random Processes. I. General heory; II. Applications. Springer-Verlag, 1977, [1] G. Lorden. Procedres for reacting to a change in distrition. Annals of Mathematical Statistics, 42: , ρ his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.

26 47 Feinerg -- Shiryaev [11] G. V. Mostakides. Optimal stopping times for detecting changes in distritions. Annals of Statistics, 14: , [12] E. S. Page. Continos inspection schemes. Biometrika, 41:1 115, [13] G. Peskir and A. N. Shiryaev. Optimal Stopping and Free-Bondary Prolems. Birkhäser, 26. [14] M. Pollak and D. Siegmnd. A diffsion process and its applications to detecting a change in the drift of Brownian motion. Biometrika, 72:267 28, [15] A. D. Polyanin and V. F. Zaitsev. Handook of Eact Soltions for Ordinary Differential Eqations. CRC Press, [16] Y. Ritov. Decision theoretic optimality of the CUSUM procedre. Annals of Statistics, 18: , 199. [17] A. N. Shiryaev. he prolem of the most rapid detection of a distrance in a stationary process. Soviet Mathematics, Doklady, 2: , [18] A. N. Shiryaev. On optimm methods in qickest detection prolems. heory of Proaility and Its Applications, 8:22 46, [19] A. N. Shiryayev [Shiryaev]. Optimal Stopping Rles. Springer-Verlag, [2] A. N. Shiryaev. Minima optimality of the method of cmlative sms (CUSUM) in the continos time case. Rssian Mathematical Srveys, 51:75 751, [21] A. N. Shiryaev. Essentials of Stochastic Finance. World Scientific, [22] A. G. artakovsky and V. V. Veeravalli. General asymptotic Bayesian theory of qickest change detection. heory of Proaility and Its Applications, 49: , 25. Egene A. Feinerg Department of Applied Mathematics and Statistics Stony Brook University Stony Brook, NY USA Egene.Feinerg@snys.ed Alert N. Shiryaev Steklov Mathematical Institte Gkina Str. 8 Moscow Rssia alertsh@mi.ras.r his article is protected y German copyright law. Yo may copy and distrite this article for yor personal se only. Other se is only allowed with written permission y the copyright holder.