arxiv: v2 [math.pr] 4 Feb 2009

Transcription

1 Optimal detection of homogeneous segment of observations in stochastic sequence arxiv: v2 [math.pr] 4 Feb 2009 Abstract Wojciech Sarnowski a, Krzysztof Szajowski b,a a Wroc law University of Technology, Institute of Mathematics, Wybrzeże Wyspiańskiego 27, Wroc law, Poland b Institute of Mathematics, Polish Academy of Science, Śniadeckich 8, Warszawa, P.O. Box 21, Poland A Markov process is registered. At random moment θ the distribution of observed sequence changes. Using probability maximizing approach the optimal stopping rule for detecting the change is identified. Some explicit solution is obtained. Key words: Disorder problem, sequential detection, optimal stopping, Markov process, change point MSC: Primar 60G40, 60K99; Secondary 90D60 1 INTRODUCTION In this paper we present a generalization of disorder problem. Typically, disorder problem is limited to the case of switching between sequences of independent random variables (see Bojdecki [1]). Some developments of basic model can be found in [6] where the optimal rule is obtained for finite statespace Markov chains. Our approach admits Markovian dependence structure for observed sequence as well, however with possibly uncountable state-space. Corresponding author addresses: Wojciech.Sarnowski@pwr.wroc.pl (Wojciech Sarnowski), Krzysztof.Szajowski@pwr.wroc.pl (Krzysztof Szajowski). URLs: (Wojciech Sarnowski), (Krzysztof Szajowski). 1

2 Formulation of the problem can be found in section 2. The main result is presented in section 3. Section 4 provides example of application of detection method. In appendix we derive useful formulas for conditional probabilities. 2 FORMULATION OF THE PROBLEM Let (X n, n N) be an observed sequence of random variables defined on the space (Ω, F, P) with values in (E, B), where E is a subset of R. For (X n, n N) denote filtration F n = σ(x 0, X 1,..., X n ). On (E, B) there are defined σ-additive measures {µ x } x E. Space (Ω, F, P) supports also unobservable (hence not measurable with respect to F n ) variable θ which has geometrical distribution: P(θ = j) = p j 1 q, q = 1 p (0, 1), j = 1, 2,... (1) On (Ω, F, P) we introduce additionally two Markov processes (X 0 n, G0 n, µ0 x ), (X 1 n, G 1 n, µ 1 x), which are connected with (X) and θ by the following equation: X n = X 0 n I {θ>n} + X 1 n I {θ n}. (2) σ-fields Gn, 0 Gn 1 are the smallest σ-fields for which (Xn, 0 n N), ((Xn, 1 n N) are adapted (respectively). We assume that θ, (Xn 0, n N), (X1 n, n N) are mutually independent. Measures µ i x satisfy the relations: µ 0 x (dy) = f 0(y)µ x (dy), µ 1 x (dy) = f1 x (y)µ x(dy) where f 0 x(.) f 1 x(.). Furthermore µ 0 x, µ 1 x, x E are known in advance. Shortly speaking our model assumes that process (X n, n N) is obtained by switching at random and unknown instant θ between two Markov processes (X 0 n, n N) and (X1 n, n N). During the on-line observation of (X n, n N) we aim in detection of switching time θ in optimal way, according to the maximum probability criterium. For any fixed d {0, 1, 2,...} we look for the stopping time τ T such that P( θ τ d) = sup P( θ τ d) (3) τ T where T denotes the set of all stopping times with respect to the filtration {F n } n N. Using parameter d we control the precision level of detection. The most rigorous case: d = 0 will be studied in details. 2

3 3 SOLUTION OF THE PROBLEM Let us define: Z n = P( θ n d F n ), n = 0, 1, 2,..., V n = esssup {τ T, τ n} P( θ n d F n ), n = 0, 1, 2,... τ 0 = inf{n : Z n = V n } (4) Notice that, if Z = 0, then Z τ = P( θ τ d F τ ) for τ T. Since F n F τ (when n τ) we have V n =ess sup τ n =ess sup τ n P( θ τ d F n ) = ess sup E(I { θ τ d} F n ) τ n E(Z τ F n ) The following lemma ensure existence of the solution Lemma 1 The stopping time τ 0 defined by formula (4) is the solution of problem (3). Proof. From the theorems presented in [1] it is enough to show that lim n Z n = 0. For all natural numbers n, k, where n k we have: Z n = E(I { θ n d} F n ) E(sup I { θ j d} F n ) j k From Levy s theorem lim sup n Z n E(sup j k I { θ j d} F ) where F = σ ( n=1 F n ). It is true that: lim sup j k, k I { θ j d} = 0 a.s. and by the dominated convergence theorem we get lim E(sup I { θ j d} F ) = 0 a.s. k j k what ends the proof of the lemma. For further considerations it will be convenient to introduce the following notation which will make our formulas more compact and clear Π n = P (θ n F n ) x k,n = (x k, x k+1,..., x n 1, x n ), k n k 1 d l j,k,d,n = fx 1 n 1 i (x n i ) fx 0 n 1 i (x n i ) i=j k 1 L j,k,d,n = i=j f 1 X n 1 i (X n i ) i=k d i=k f 0 X n 1 i (X n i ) 3

4 where d n, j 2 i=j1 u i = 1 if only j 1 > j 2 for any u i R. Additionally, for a sequence of sets A i F i and variables X i, i = 0, 1, 2,... we will write probability P(X k A k,..., X n A n X 0 A 0,..., X k 1 A k 1 ); k n as P(X k,n A k,n X 0,k 1 A 0,k 1 ) Next lemma transforms payoff function to the more convenient form. Lemma 2 Let h(x 1,d+2, α) = ( where x 1,..., x d+2 E, α (0, 1) then ) l 0,m,d,d+2 (1 α) (5) p m l 0,0,d,d+2 P( θ n d) = E [ h(x n 1 d,n, Π n ) ] (6) Proof. We rewrite initial criterion as the expectation P( θ n d)=e [P( θ n d F n )] = E [P(θ n + d F n ) P(θ n d 1 F n )] (7) Probabilities under expectation can be transformed to the convenient form using formulas (A.1) and (A.3). Next, solving equation (A.9) for Π n d 1 and substituting the result in equation (A.3) we obtain the lemma: P( θ n d F n ) = ( ) (1 Π p m n ) L 0,0,d,n Lemma 3 Process η n = {X n d 1,n, Π n } forms a random Markov function. Proof. According to lemma 17 pp in [3] it is enough to show that η n+1 is a function of previous stage η n and variable X n+1 (property of a system of transitive statistics) and that conditional distribution of X n+1 given F n is a function of η n. 4

5 For x 1,..., x d+2, y E, α (0, 1) let us consider a function ( f 1 ) x ϕ(x 1,d+2, α, y) = x 2,d+2, y, d+2 (y)(q + pα) fx 1 d+2 (y)(q + pα) + fx 0 d+2 (y)p(1 α) We will show that η n+1 = ϕ(η n, X n+1 ). It is enough to use formula (A.9). Putting d = 0 and substituting n by n + 1 in (A.9) we get Π n+1 as a function of Π n : Π n+1 = f1 X n (X n+1 )(q + pπ n ) H(X n, X n+1, Π n ) (8) where H(x, y, α) = f 1 x(y)(q + pα) + f 0 x(y)p(1 α). To see that conditional distribution of X n+1 given F n is a function of η n, for any Borel function u : E R let us consider the conditional expectation of u(x n+1 ) given F n : E( u(x n+1 ) F n ) =E (u(x n+1 )(1 Π n+1 ) F n ) + E (u(x n+1 )Π n+1 F n ) ( ) ( =E u(x n+1 ) pf0 X n (X n+1 )(1 Π n ) F n + E u(x n+1 ) f1 X n (X n+1 )(q + pπ n ) H(X n, X n+1, Π n ) H(X n, X n+1, Π n ) =p(1 Π n ) u(y)fx 0 n (y)dµ Xn (y) + (q + pπ n ) u(y)fx 1 n (y)dµ Xn (y) F n ) We use here equation (8). Thus, conditional distribution of X n+1 given F n depends only on components of η n, what ends the proof of lemma. Lemmas (2) and (3) are crucial for the solution of posed problem (3). They show that initial problem can be replaced with the problem of stopping Markov random function η n = (X n d 1,n, Π n ) with the payoff given by equation (5). In consequence we can use standard tools for finding stopping time τ such that E [ h(x τ 1 d,τ, Π τ )] = sup E [ h(x τ 1 d,τ, Π τ ) ] (9) τ T To solve reduced problem (9) for any Borel function u : E d+2 [0, 1] R let us define operators: Tu(x 1,d+2, α)=e [ u(x n d,n+1, Π n+1 ) X n 1 d,n = x 1,d+2, Π n = α ] Q k u(x 1,d+2, α)=max{u(x 1,d+2, α), TQ k 1 u(x 1,d+2, α)}, k 1 Q 0 u(x 1,d+2, α)=u(x 1,d+2, α) 5

6 Lemma 4 For the payoff function h(x 1,d+2, α) characterized by (5) the following formulas hold: Q k h(x 1,d+2, α)=(1 α) max { TQ k h(x 1,d+2, α)=(1 α)r k (x 2,d+2 ), k 0 where r k (x 2,d+2 )=p r 0 (x 2,d+2 )=p [ f 0 x d+2 (x d+3 ) max { ] l 0,m 1,d 1,d+2 p m l 0,0,d 1,d+2 } l 0,m,d,d+2 ; r p m k 1 (x l 2,d+2 ), k 1 0,0,d,d+2 } l 0,m,d,d+3 ; r p m k 1 (x l 3,d+3 ) dµ xd+2 (x d+3 ) 0,0,d,d+3 Proof. By the definition of operator T and using (8) we have Th( X n 1 d,n, Π n ) = E [ ] h(x n d,n+1, Π n+1 ) X n d 1,..., X n, Π n [ ] d+1 = E (1 p d +1 + q )(1 Π p m n+1 ) X n d 1,..., X n, Π n L 0,0,d,n+1 = p(1 Π n ) ( H(X n, u, Π n )dµ Xn (u) [ = p(1 Π n ) = (1 Π n )p [ (1 p d ) + q L 1,m,d,n+1 fx 1 n (u) p m L 1,1,d,n+1 fx 0 n (u) L 1,m,d,n+1 ) f 0 X n (u) H(X n, u, Π n ) f p m X 1 L n (u)dµ Xn (u) 1,1,d,n+1 ] L 0,m 1,d 1,n p m L 0,0,d 1,n Directly from the definition of Q results that ] = (1 Π n )r 0 (X n d,n ) Qh(X n 1 d,n, Π n )=max { h(x n 1 d,n, Π n ); Th(X n 1 d,n, Π n ) } = (1 Π n ) max { } ; r p m 0 (X L n d,n ) 0,0,d,n Suppose now that lemma (4) holds for TQ k 1 h and Q k h for some k > 1. Then using similar transformation as in the case of k = 0 we get 6

7 TQ k h( X n 1 d,n, Π n ) = E [ Q k h(x n d,n+1, Π n+1 ) X n d 1,..., X n, Π n ] = [ max{ (1 Π n )pf 0 X n (u) ] dµ Xn (u) = (1 Π n )r k (X n d,n ) L 1,m,d,n+1 fx 1 n (u) p m L 1,1,d,n+1 fx 0 n (u) ; r k 1(X n d+1,n, u)} Moreover Q k+1 h( X n 1 d,n, Π n ) = max { h(x n 1 d,n, Π n ); TQ k h(x n 1 d,n, Π n ) } = (1 Π n ) max This completes the proof. { The following theorem is the main result of the paper. Theorem 1 The solution of problem (3) is given by: } ; r p m k (X L n d,n ) 0,0,d,n τ = inf{n d + 1 : where r (X n d,n ) = lim k r k (X n d,n ) p m L 0,0,d,n r (X n d,n )} (10) Proof. It is easy to notice that for any τ T stopping time τ = max{τ, d+1} is at least as good as τ. Thus can limit our considerations to the rules not smaller than d + 1. This is because for the case τ = d + 1 > τ we have P(θ τ) = P(θ τ) = P(θ 1). From optimal stopping theory (c.f [3]) we know that τ 0 defined by (4) can be expressed as τ 0 = inf{n d + 1 : h(x n 1 d,n, Π n ) Q h(x n 1 d,n, Π n )} where Q h(x n 1 d,n, Π n ) = lim k Q k h(x n 1 d,n, Π n ). According to lemma (4): 7

8 τ 0 =inf =inf =τ { { What ends the proof. n d + 1 : max{ n d + 1 : p m L 0,0,d,n p m L 0,0,d,n ; r (X n d,n )} } p m L 0,0,d,n r (X n d,n ) Remark. Obtained result is a generalization of theorem 2 in [1] for the case of Markovian sequences. Assuming that X 0 and X 1 are i.i.d sequences we get exactly the same rule as Bojdecki did. } 4 EXAMPLE Let us consider the case d = 0. Then, optimal rule (10) reduces to simpler form with τ = inf{n : f 1 X n 1 (X n ) pf 0 X n 1 (X n ) r (X n )} r (X n ) = p fx 0 n (u) max{ f1 X n (u) E pfx 0 n (u), r (u)}dµ Xn (u) Moreover suppose that the state space E = {0, 1}. Matrices of transition probabilities and conditional densities are as follow [ µ 0 i (j) ] i=0,1 j=0,1 = , [ µ 1 i (j) ] i=0,1 = j=0, [ f 0 i (j) ] i=0,1 j=0,1 = 1 1, [ f 1 i (j) ] i=0,1 = 7 1/3 j=0, /2 3 8

9 For such model we find threshold r (i), i = 0, 1 solving the system of equations r (i) = j=0,1 pf 0 i (j) max{ f1 i (j) pf 0 i (j), r (j)}µ i (j); i = 0, 1 Treating r as a function of parameter p we obtain: r p (0) =1 [0,p 1 ](p) p 10 1 (p 1,p 2 ](p) + r p(1) =1 [0,p2 ](p) p 50 36p 21 (p 2,p 3 ](p) p 50 36p 21 (p 2,p 3 ](p) + 14p p 21 (p 3,1](p) 35 7p 50 10p 36p 21 (p 3,1](p) where: p 1 = 1, p 3 2 = 229 7, p 18 3 = The most interesting case takes 136 the place when p > p 3 0, 946 because then the average disorder time is not too small. Obtained stopping rule τ depends on observations collected at times τ 1 and τ. Thus, to make optimal rule more clear we need to analyze all possible sequences of (X τ 1, X τ ) i.e. {0, 0}, {0, 1}, {1, 0}, {1, 1}. Sequence {0, 0}: In this case we stop if only p. Solving the inequality for p, we p 50 10p 36p 2 get that stopping time takes the place for all p (p 3, 1). Sequence {0, 1}: It reduces to inequality 1 3p a set of solutions is empty. Sequence {1, 0}: Pair {1, 0} implies the stopping time if 7 p solution for p (p 3, 1). Sequence {1, 1}: This sequence rises the alarm if only 3 p inequality is satisfied for any p (p 3, 1). 14p 25 50p 18p 2. Taking into account that p (p 3, 1) 35 7p 50 10p 36p 2. However there is no 14p 25 50p 18p 2. It turns out that the The analysis shows that we obtain very clear and simple optimal rule for case p > p 3 : stop at the first moment when two zeros or two ones occur in a row. A USEFUL FORMULAS In appendix we present useful formulas for conditional probabilities of various events defined by disorder time θ. 9

10 Formula 1. P(θ n + d F n ) = 1 p d (1 Π n ) (A.1) Proof. We will show the equality on set A = {ω : X 0,n A 0,n } P(θ n + d X 0,n A 0,n ) = 1 P(θ > n + d X 0,n A 0,n ) = 1 P(θ > n, θ n + 1,..., θ n + d X 0,n A 0,n ) = 1 P(θ n + d X 0,n A 0,n, θ > n, θ n + 1,..., θ n + d 1)... P(θ n + 1 X 0,n A 0,n, θ > n)p(θ > n X 0,n A 0,n ) = 1 p d (1 P(θ n X 0,n A 0,n )) (A.2) Formula 2. P(θ n d 1 F n ) Π n d 1 L 0,d+1,d,n = Π n d 1 L 0,d+1,d,n + (1 Π n d 1 ) [ q ] d k=0 p d k L 0,k+1,d,n + p d+1 L 0,0,d,n Proof. On set A = {ω : X 0,n A 0,n } we have (A.3) P(θ n d 1 X 0,n A 0,n ) = P(θ n d 1, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) P(X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) (A.4) The nominator of (A.4) can be rewritten as P(θ n d 1, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) = P(X n A n X 0,n 1 A 0,n 1, θ n d 1)... P(X n d A n X 0,n d 1 A 0,n d 1, θ n d 1) P(θ n d 1 X 0,n d 1 A 0,n d 1 ) = µ 1 X n 1 (A n )... µ 1 X n d 1 (A n d )P(θ n d 1 X 0,n d 1 A 0,n d 1 ) (A.5) 10

11 To deal with the denominator of (A.4) we divide space Ω into disjoin sets 0 Ω = {ω : θ(ω) n d 1} {ω : θ(ω) = n + i} {ω : θ(ω) > n} i= d P(X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) =P(θ n d 1, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) +P(θ = n d, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) P(θ = n, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) +P(θ > n, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) (A.6) Notice that first component of (A.6) is just (A.5). The second part is P(θ =n d, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) =P(θ > n d 1, θ = n d, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) =P(X n A n X 0,n 1 A 0,n 1, θ > n d 1, θ = n d)... P(X n d A n d X 0,n d 1 A 0,n d 1, θ > n d 1, θ = n d) P(θ = n d X 0,n d 1 A 0,n d 1, θ > n d 1) P(θ > n d 1 X 0,n d 1 A 0,n d 1 ) =µ 1 X n 1 (A n )... µ 1 X n d 1 (A n d )qp(θ > n d 1 X 0,n d 1 A 0,n d 1 ) In the same manner we deal with following components. The very last component can be rearranged as (A.7) P(θ >n, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) =P(θ > n d 1, θ > n, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) =P(X n A n X 0,n 1 A 0,n 1, θ > n d 1, θ > n)... P(X n d A n d X 0,n d 1 A 0,n d 1, θ > n d 1, θ > n) P(θ > n X 0,n d 1 A 0,n d 1, θ > n d 1) P(θ > n d 1 X 0,n d 1 A 0,n d 1 ) =µ 0 X n 1 (A n )... µ 0 X n d 1 (A n d )p d+1 P(θ > n d 1 X 0,n d 1 A 0,n d 1 ) (A.8) Combining equations (A.4)-(A.8) we get the thesis. Formula 3. 11

12 Π n d 1 L 0,d+1,d,n + (1 Π n d 1 )q d Π n = k=0 p d k L 0,k+1,d,n Π n d 1 L 0,d+1,d,n + (1 Π n d 1 ) [ q ] d k=0 p d k L 0,k+1,d,n + p d+1 L 0,0,d,n Proof. On set A = {ω : X 0,n A 0,n } we have (A.9) P(θ > n X 0,n A 0,n ) = 1 P(θ > n, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) P(X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) (A.10) Let us transform the nominator of (A.10) P(θ >n, X n d,n A n d,n X 0,n d 1 A 0,n d 1 ) =P(X n A n X 0,n 1 A 0,n 1, θ > n)... P(X n d A n X 0,n d 1 A 0,n d 1, θ > n) P(θ > n X 0,n d 1 A 0,n d 1 ) =µ 0 X n 1 (A n )... µ 0 X n d 1 (A n d ) P(θ > n d 1, θ n d,..., θ n X 0,n d 1 A 0,n d 1 ) =µ 0 X n 1 (A n )... µ 0 X n d 1 (A n d ) P(θ n X 0,n d 1 A 0,n d 1, θ > n d 1, θ n d,..., θ n 1)... P(θ n d X 0,n d 1 A 0,n d 1, θ > n d 1) P(θ > n d 1 X 0,n d 1 A 0,n d 1 ) =µ 0 X n 1 (A n )... µ 0 X n d 1 (A n d )p d+1 P(θ > n d 1 X 0,n d 1 A 0,n d 1 ) (A.11) To compute the denominator of (A.10) we again apply formulas (A.5)-(A.8). Thus using (A.11) and (A.5)-(A.8) we prove the lemma. References [1] T. Bojdecki, Probability maximizing approach to optimal stopping and its application to a disorder problem, Stochastics, 3:61 71, [2] T. Bojdecki and J. Hosza, On a generalized disorder problem, Stochastic Processes Appl., 18: , [3] A.N. Shiryaev, Optimal Stopping Rules, Springer-Verlag, New York, Heidelberg, Berlin,

13 [4] K. Szajowski, Optimal on-line detection of outside observation, J.Stat. Planning and Inference, 30: , [5] M. Yoshida, Probability maximizing approach for a quickest detection problem with complocated Markov chain, J. Inform. Optimization Sci., 4: , [6] B. Yakir, Optimal detection of a change in distribution when the observations form a Markov chain with a finite state spacem, IMS Lecture Notes - Monograph Series, Vol. 23,