On Strongly Equivalent Nonrandomized Transition Probabilities

Transcription

1 On Strongly Equivalent Nonrandomized Transition Probabilities Eugene. Feinberg 1 and lexey B. Piunovskiy bstract Dvoretzky, Wald and Wolfowitz proved in 1951 the existence of equivalent and strongly equivalent mappings for a given transition probability when the number of nonatomic measures is finite and the decision set is finite. This paper introduces a notion of strongly equivalent transition probabilities with respect to a finite collection of functions. This notion contains the notions of equivalent and strongly equivalent transition probabilities as particular cases. This paper shows that a strongly equivalent mapping with respect to a finite collection of functions exists for a finite number of nonatomic distributions and finite decision set. It also provides a condition when this is true for a countable decision set. ccording to a recent example by Loeb and Sun, a strongly equivalent mapping may not exist under these conditions when the decision set is uncountable. This paper also provides two additional counterexamples and shows that strongly equivalent mappings exist for homogeneous transition probabilities. 1 Introduction Dvoretzky, Wald and Wolfowits [, 3] studied the following problem. Consider a standard Borel space, i.e. is a one-to-one measurable image of a Borel subset of a Polish space, and consider another standard Borel space. Sometimes is called a state space (or set) and is called a decision space (or set). Let a finite number of nonatomic probability measures µ 1,..., µ K on be given. We recall that a measure µ on a standard Borel space is called nonatomic if µ({x}) = for all x. We follow the terminology that a finite set is countable. Of course, if is countable then f(a)ν(da) = a f(a)ν(a) for any measure ν and for any function f on. Denote by π(da x) a regular transition probability from to. Since we consider only regular transition probabilities and only measurable functions, we sometimes omit the terms regular and measurable. The following two facts were proved in [3]. Proposition 1. ([3, Theorem 3.1]) If is a finite set then for any finite number K, for any measurable functions r k (x, a), k = 1,..., K, and for any transition probability π from to, there exists a measurable mapping ϕ of to such that r k (x, a)π(da x)µ k (dx) = r k (x, ϕ(x))µ k (dx), k = 1,..., K. 1 Department of pplied Mathematics and Statistics, State University of New York at Stony Brook, Stony Brook, NY , US. Department of Mathematical Sciences, University of Liverpool, Liverpool, L69 7ZL, UK. 1

2 Proposition. ([3, Theorem 3.]) If is a finite set then for any transition probability π from to there exists a measurable mapping ϕ : such that π(a x)µ k (dx) = µ k ({x : ϕ(x) = a}), a, k = 1,..., K. We remark that Dvoretzky, Wald, and Wolfowitz [, 3] considered = R n but their proofs are valid for a standard Borel space. Let the finite integer K and measurable functions r k (x, a) on be fixed, k = 1,..., K. ccording to [3], two transition probabilities π 1 and π are called equivalent if r k (x, a)π 1 (da x)µ k (dx) = r k (x, a)π (da x)µ k (dx), k = 1,..., K; (1) and π 1 and π are called strongly equivalent if for any measurable subset B of π 1 (B x)µ k (dx) = π (B x)µ k (dx), k = 1,..., K. () Thus, for any transition probability, Proposition 1 states the existence of an equivalent measurable mapping when the set is finite. Proposition claims the existence of a strongly equivalent measurable mapping when is finite. s mentioned in [3], strongly equivalent transition probabilities are equivalent for any set of measurable functions {r k } on, but they may not be equivalent for a given set of measurable functions on. The natural question is whether, for a given transition probability, equivalent and strongly equivalent measurable mappings exist for infinite. Dvoretzky, Wald, and Wolfowitz [3, Section 4] studied this question by considering a compact and approximating it by finite sets. Feinberg and Piunovskiy [4, Theorem 1] proved the existence of an equivalent measurable mapping for an arbitrary Borel space. Loeb and Sun [6, Example.7] provided a surprising example when a strongly equivalent measurable mapping does not exist when K = and = [ 1, 1]. In particular, in this example the transition probability, for which there is no equivalent mapping, is a stochastic mixture of two mappings. In this paper we introduce a definition of strongly equivalent transition probabilities with respect to (wrt) a finite collection of functions on. By selecting an appropriate collection of functions, strongly equivalent transition probabilities wrt to a collection become strongly equivalent in the sense of (). By selecting another collection, strongly equivalent transition probabilities wrt to that collection become equivalent. Theorem 1 below states that if the set is finite, then, for a transition probability and a finite collection of measurable functions on, there exists a mapping that is strongly equivalent wrt to that collection. Theorem states that, if the transition probability does not depend on x, then the corresponding generalization of Proposition holds for an arbitrary Borel space. Theorem 3 provides a sufficient condition for a countable set when strongly equivalent wrt to a given finite collection of functions mapping exists for a given transition probability. Corollary 1 states, for a countable, the existence of a strongly equivalent mapping wrt a given collection of functions for a transition probability being a finite mixture of deterministic mappings. In addition, we construct examples that illustrate, for a given transition probability, the following possibilities: (i) there is no strongly equivalent mapping wrt to a function f(x, a) on when K = 1 (if f = f(x), such a mapping exists), and (ii) equivalent and strongly equivalent mappings do not exist for a countable number of nonatomic measures.

3 Definitions and preliminary result Consider nonempty standard Borel spaces and. Let P be a probability measure on and π be a transition probability from to. transition probability π is deterministic if for each x there exists a such that π({a} x) = 1. Each deterministic transition probability π is identified by a measurable mapping ϕ : such that π({ϕ(x)} x) = 1 for all x. Definition 1. Suppose a natural K and a collection of measurable functions {f k (x, a)} k=1,,...,k on are given. Two transition probabilities π 1 and π are called strongly equivalent wrt {f k } k=1,,...,k if for each k = 1,,..., K and for any measurable function ρ(a) on ρ(a)f k (x, a)π 1 (da x)p (dx) = ρ(a)f k (x, a)π (da x)p (dx). We remark that in this paper we follow a standard agreement that a finite or infinite Lebesgue - Stieltjes integral gdp is defined for any nonnegative measurable function g. In general, gdp = g + dp g dp, and this integral is defined only if either g + dp < or g dp <, where g + (x) = max{, g(x)} and g (x) = max{, g(x)}. If two integrals are not defined, they are considered to be equal. Notice that if π is a deterministic transition probability defined by a measurable mapping ϕ then ρ(a)f k (x, a)π(da x)p (dx) = ρ(ϕ(x))f k (x, ϕ(x))p (dx). We observe that, for P = K 1 K k=1 µ k, the strong equivalence of transition probabilities wrt {f k } k=1,,...,k for the properly selected functions f k implies the described in the Introduction strong equivalence and equivalence in the sense of Dvoretzky, Wald and Wolfowitz [, 3]. Indeed, since µ k P, k = 1,..., K, by the Radon-Nykodým theorem there are nonnegative measurable functions g k, k = 1,..., K, such that µ k (dx) = g k (x)p (dx). Thus, if transition probabilities π 1 and π are strongly equivalent wrt {f k } k=1,,...,k with f k (x, a) = g k (x)r k (x, a) then, by setting ρ(a) = 1 we have that they are equivalent in the sense of Dvoretzky, Wald and Wolfowitz [3]; (see (1) ). Transition probabilities π 1 and π are strongly equivalent wrt {g k } k=1,,...,k if and only if they are strongly equivalent in the sense of Dvoretzky, Wald and Wolfowitz [3]; see (). The following theorem, whose proof is based on the method introduced in [3], states the existence of strongly equivalent mappings wrt to a given finite collection of functions. Theorem 1. Let P (dx) be a non-atomic probability measure, be finite, K be a natural number, and a collection {f k (x, a)} k=1,,...,k of measurable functions on be given. Then, for any transition probability π from to, there exists a measurable mapping ϕ such that π and ϕ are strongly equivalent wrt {f k } k=1,,...,k. Proof. For any b consider the functions r bk (x, a) = I{a = b}f k (x, a), k = 1,..., K. Then {r bk } {b,k=1,...,k} is a finite collections of functions and, therefore, Proposition 1 implies that ϕ I{ϕ(x) = b}f k (x, ϕ(x))p (dx) = f k (x, b)π(b x)p (dx), b, k = 1,..., K. By multiplying each side of this equality by ρ(b), adding them in b, and replacing the notation b with a, we have ρ(ϕ(x))f k (x, ϕ(x))p (dx) = ρ(a)f k (x, a)π(a x)p (dx), k = 1,..., K. a 3

4 Remark 1. can be considered as a state space and can be considered as an action space. Sometimes it is natural to consider the set of available actions (x) for each state x. measurable mapping ϕ : is called a selector if ϕ(x) (x) for all x. The natural assumptions are that the graph Gr () = {(x, a) : x, a (x)} of the setvalued mapping (x) is a measurable subset of and there exists a selector. transition probability π must satisfy π((x) x) = 1 for all x. Functions f k (x, a), with respect to which a strong equivalence is considered, should satisfy f k (x, a) = for a / (x). Though such a model looks more general, in fact, it is not more general for the purpose of this paper, because, given a transition probability π, the existence of a strongly equivalent measurable function ϕ wrt {f k } k=1,,...,k implies the existence of a strongly equivalent selector ϕ wrt {f k } k=1,,...,k for the new model. Let Y = {x ϕ(x) / (x)}. Since the mapping g(x) = (x, ϕ(x)) is Borel for a Borel mapping ϕ and the graph of a Borel function ϕ is a Borel set, Y is a measurable subset of. Let φ be a selector. Define the selector ϕ: ϕ(x) = ϕ(x) for x \ Y and ϕ(x) = φ(x) for x Y. Then the transition probability π and selector ϕ are strongly equivalent wrt {f k } k=1,,...,k. 3 Uncountable set Let be an uncountable standard Borel space. The following example shows that, for a given transition probability π, the strongly equivalent function may not exist. Therefore, Theorem 1 does not hold if is uncountable. Example 1. (Loeb and Sun [6, Example.7]) For K > 1, Proposition cannot be extended to an uncountable. Let = [, 1], = [ 1, 1], µ 1 (dx) = dx, µ (dx) = xdx, and π(γ x) = 1[I{x Γ} + I{( x) Γ}] for any Borel subset Γ of [ 1, 1]. In this example there is no measurable mapping ϕ : strongly equivalent to π; see Loeb and Sun [6, Example.7] for detail. Therefore, Proposition cannot be extended to a continuum. Thus, when is not countable, Proposition and Theorem 1 do not hold for K > 1. Consider the case K = 1. Let f = f 1. If f does not depend on a, i.e. f = f(x), then Theorem 1 holds for any standard Borel space. Indeed, since all Borel spaces are isomorphic to Borel subsets of [, 1], it is sufficient to consider = = [, 1]. By replacing P with µ(dx) = f(x)p (dx), the statement of Theorem 1 for K = 1 and an arbitrary standard Borel space means that any random variable on [, 1] can be presented as a measurable mapping ϕ of a random variable with the nonatomic distribution µ. This is a well-known fact broadly used in simulation. In particular, when µ is the Lebesgue measure, the existence of such mapping, that we denote by ϕ, was proved by umann [1, Lemma F]. For a general random variable defined by a nonatomic measure µ, observe that F ()) is uniformly distributed on [, 1], where F is the distribution function of. Thus, ϕ(x) = ϕ(f (x)) is the required mapping. However, the following example shows that Theorem 1 does not hold for K = 1 when f = f(x, a) and is uncountable. Example. For K = 1 there is no strongly equivalent mapping wrt to f depending on a. Consider the same,, and π as in Example 1. Let f(x, a) = x a, and P be the Lebesgue measure on [, 1]. Then, 1 ρ(a)f(x, a)π(da x) =.5(ρ(x) x + ρ( x) x) for any measurable function 1 4

5 ρ(a) and therefore ρ(a)f(x, a)π(da x)dx = 1 For any measurable function ϕ on [, 1] 1 ρ(ϕ(x))f(x, ϕ(x))dx = [ρ(x) x + ρ( x) x]dx. (3) ρ(ϕ(x))i{ϕ(x) > }[x ϕ(x)]dx ρ(ϕ(x))i{ϕ(x) }[x + ϕ(x)]dx. Let ϕ be strongly equivalent to π wrt to {f}. Consider ρ 1 (a) = a I{a > }. Since the lefthand parts of (3) and (4) are qual, 1 1 x dx = 1 Hence 1 I{ϕ(x) > }[x ϕ(x)] dx = Consider ρ (a) = a I{a }. Then 1 1 x dx = 1 ϕ(x)i{ϕ(x) > }[x ϕ(x)]dx. 1 I{ϕ(x) > }x dx 1 ϕ(x)i{ϕ(x) }[x + ϕ(x)]dx. 1 (4) x dx. (5) Hence 1 1 I{ϕ(x) }[x + ϕ(x)] dx = I{ϕ(x) }x dx 1 1 x dx. (6) Sum (5) and (6). The righthand side of this sum is. The lefthand side is the sum of two nonnegative numbers. Therefore, and 1 I{ϕ(x) > }[x ϕ(x)] dx = ϕ(x) = 1 I{ϕ(x) }[x + ϕ(x)] dx = { x, if ϕ(x) >, x, if ϕ(x), where in (7) and in all the relations for ϕ(x) in this example we mean that they hold P -a.s. Observe that P ({ϕ(x) = }) = P ({x = }) =. Consider { ρ 3 (a) = I{ϕ(a) = a, a > }/a, if a,, otherwise. Since the lefthand sides of (3) and (4) are qual, 1 1 I{ϕ(x) = x}dx = 1 = I{ϕ(ϕ(x)) = ϕ(x), ϕ(x) > } 1 I{ϕ(x) = x}dx, 5 [ ] x ϕ(x) 1 dx (7) (8)

6 where the last equality holds because, in view of (7), the following is true: (i) ϕ(x) = x, when ϕ(x), (ii) I{ϕ(x) = x} = I{ϕ(ϕ(x)) = ϕ(x), ϕ(x) }, and (iii) P ({ϕ(x) = }) =. Equality (8) implies that P ({ϕ(x) = x}) = and, in view of (7), P ({ϕ(x) = x}) = 1. This is impossible, because for ρ 4 (a) = I{a < } we have 1 1 ρ 1 4(a)f(x, a)π(da x)dx = 1 and 4 1 ρ 4(ϕ(x))f(x, ϕ(x))dx = 1, where the former follows from (3) and the latter follows from (4). Consider a particular situation when the transition probability π does not depend on x, i.e., π(da x) = π(da y) for all x, y. Though, according to Example 1, strongly equivalent policies may not exist, they exist when the transition probability does not depend on x. Theorem. If all the measures µ 1,..., µ K are nonatomic and the transition probability π does not depend on x then for π there exists a strongly equivalent mapping. Proof. Let π(b) = π(b x), where x and B is a measurable subset of. We observe that π(b x)µ k(dx) = π(b), k = 1,..., K. Therefore, we need to prove that there exists a measurable mapping ϕ of to such that for any k = 1,..., K and for any measurable subset B of µ k ({x : ϕ(x) B}) = π(b). (9) Since all uncountable Borel spaces are isomorphic, we assume without loss of generality that = (, 1]. We shall construct a strongly equivalent mapping ϕ : (, 1]. In particular, we shall construct a sequence of measurable step functions ϕ n, n = 1,,..., such that ϕ n ϕ as n. First, we shall construct the function ϕ 1. To do this, we consider a decision rule π 1 that takes only two values: 1 and 1; π1 ( 1 x) = π((, 1]) and π1 (1 x) = π(( 1, 1]). ccording to Proposition, for the decision rule π 1 there exists a strongly equivalent mapping ϕ 1. Let We observe that, for i = 1, 1,i = {x ϕ 1 (x) = i }, i = 1,. µ k ( 1,i ) = π(( i 1, i ]), k = 1,..., K. Let for some n = 1,,..., a partition n,1,..., n, n of be given such that all elements of this partition are measurable sets and for i = 1,..., n µ k ( n,i ) = π(( i 1, i ]), k = 1,..., K. (1) n n We construct a partition n+1,1,..., n+1, n+1 of such that: (i) all elements of this partition are measurable, (ii) n,i = n+1,i 1 n+1,i, and (iii) for i = 1,..., n+1 µ k ( n+1,i ) = π(( i 1, i ]), k = 1,..., K. (11) n+1 n+1 To construct such a partition, we consider an arbitrary set n,j, j = 1,..., n. If π(( j 1 n, j we set n+1,j 1 = n,j and n+1,j =. If π(( j 1, j n measures µ n,j 1,..., µ n,j K on n,j: µ n,j k (B) = µ k(b n,j )/π(( j 1 n, j n ]) 6 ]) =, n ]) >, we consider the probability n

7 for measurable subsets B of n,j. We also define the probability measure π n,j on ( j 1 n, j n ] : π n,j (D) = π(d ( j 1 n, j n ])/π((j 1 n, j n ]) for measurable subsets D of ( j 1, j ]. The arguments used to partition the set into n n 1,1 and 1, imply that the set n,j can be partitioned into two measurable subsets n+1,j 1 and n+1,j such that (11) holds for i = j 1, j. Thus, the construction of n+1,1,..., n+1, n+1 is completed. For each n = 1,,... we set ϕ n (x) = i n when x n,i, i = 1,..., n. For n = 1 this definition is consistent with the previously given definitions. Since < ϕ n+1 (x) ϕ n (x), we can define ϕ(x) = lim n ϕ n (x), x. To complete the proof of the theorem, we need to prove (9). To verify (9), it is sufficient to show that for any n = 1,,... and for any i = 1,..., n µ k {ϕ(x) < i } = π((, i )), k = 1,..., K. (1) n n We observe that for n = 1,,... and for i = 1,..., n and for j = 1,,... Therefore, for k = 1,..., K {x ϕ n (x) < i n } = j=1{x ϕ n+j (x) ij 1 n+j } {x ϕ n+j (x) ij 1 } {x ϕ n+j+1 (x) ij+1 1 n+j n+j+1 }. µ k {ϕ(x) < i n } = lim j µ k{ϕ n+j (x) ij 1 n+j } = lim j π((, ij 1 n+j ]) = π((, i 1 i )) lim n j π((ij, n+j )) = π((, i n )), n where the second equality follows from (1). Thus, (1) is proved. 4 Countable set In this section, we prove a particular case of Theorem 1 when is an infinite countable set. Since, in view of Example 1, strongly equivalent mappings may not exist when is uncountable, the natural question is whether strongly equivalent mappings exist for a countable set. We do not know the answer to this question. The following Theorem 3 states the existence of strongly equivalent mappings wrt to a finite family of functions under the additional assumption that the transition probability π( x) is concentrated on a finite set B(x) for every x. We remark that, if is uncountable, then Example 1 demonstrates that the statements of neither Theorem 3 nor Corollary 1 are correct. 7

8 Theorem 3. Let P (dx) be a non-atomic probability measure, be a countable set, K be a natural number, and {f k (x, a)} k=1,,...,k be a collection of measurable functions on. If for P -almost all x the set B(x) = {a π(a x) > } is finite then there exists a measurable mapping strongly equivalent to π wrt {f k (x, a)} k=1,,...,k. Proof. For each finite subset B of, consider the measurable set B = {x B(x) = B}. Theorem 1 implies that, for the problem with the set replaced with B, there exists a strongly equivalent mapping wrt {f k (x, a)} k=1,,...,k. Therefore, the strongly equivalent mapping ϕ wrt {f k (x, a)} k=1,,...,k is defined on the measurable set Y = B B. This set is measurable, because there is only a countable number of terms in this union. Since P ( \ Y ) =, we can set ϕ(x) b for x \ Y, where b is an arbitrary element of. We recall that a transition probability π from a measurable set to a measurable set is called a finite mixture of mappings if there exist a finite number of measurable mappings φ 1,..., φ n of to such that, for some nonnegative numbers α 1,..., α n with their sum equal to 1, the equality π(γ x) = n i=1 α ii{φ i (x) Γ} holds for all measurable subsets Γ of and for all x. Corollary 1. Let be a countable set, K be a natural number, {f k } k=1,...,k be measurable functions on, and π be a finite mixture of measurable mappings. Then there exists a measurable mapping strongly equivalent to π wrt {f k } k=1,...,k. 5 Counterexample when the number of measures µ k is countable Dvotetzky, Wald, and Wolfowitz [3, Examples 1 and ] provide examples when Propositions 1 and do not hold when the set of parameters k, on which the functions r k and measures µ k depend, is continuum; see also Ferguson [5, p. 14]. s we see from Example 1, Theorem 3, and Corollary 1, some facts on the existence of equivalent mappings may hold for countable sets and do not hold for continuum sets. The following example demonstrates that Propositions 1, and Theorem 1 do not hold when k belongs to a countable set. Example 3. Let = [, 1], = {, 1}, and π( x) = π(1 x) 1/. ccording to [7, Theorem 6.6 in Chapter II], there exists a sequence {f k (x)} k=1,,... of nonnegative bounded uniformly continuous functions on such that, if for all k = 1,... f k (x)ν 1 (dx) = f k (x)ν (dx), (13) then measures ν 1 and ν coincide. (i) Consider r k (x, a) = f k (x)i{a = 1}, k = 1,..., and suppose there exists a measurable mapping ϕ such that r k (x, a)π(da x)dx = r k (x, ϕ(x))dx, k = 1,,... i.e. in this example all µ k are Lebesgue measures. This is equivalent to f k (x) 1 dx = f k (x)i{ϕ(x) = 1}dx, k = 1,,.... (14) 8

9 Therefore, the measures ν 1 (dx) = 1dx and ν (dx) = I{ϕ(x) = 1}dx are equal. This is impossible, because their densities 1 and I{ϕ(x) = 1} are not equal for all x. Thus, Proposition 1 does not hold when K is not finite. (ii) Since (13) remains valid if f k is multiplied by any constant c k, we select f k in the way f k(x)dx = 1, k = 1,,.... Then (14) is equivalent to π(1 x)µ k (dx) = I{ϕ(x) = 1}µ k (dx), k = 1,,..., (15) where µ k (dx) = f k (x)dx. Since (14) is impossible, Proposition does not hold when K is not finite. (iii) Let P (dx) = k=1 ( 1 )k µ k (dx) and g k (x) = µ k (dx)/p (dx). Then (15) becomes ag k (x)π(da x)p (dx) = ϕ(x)g k (x)p (dx), k = 1,,.... Since (15) is impossible, Theorem 1 does not hold either when K is countable. cknowledgement. This research was partially supported by NSF grant DMI References [1] R.J. umann, Mixed and behavior strategies in infinite games. nn. Math.Studies, 53(1964), pp []. Dvoretzky,. Wald, J. Wolfowitz, Elimination of randomization in certain problems of statistics and of the theory of games, Proc. Nat. cad. Sci., 36(195), pp [3]. Dvoretzky,. Wald, J. Wolfowitz, Elimination of randomization in certain statistical decision procedures and zero-sum two-person games, nn. Math. Stat., (1951), pp [4] E.. Feinberg and.b. Piunovskiy. On Dvoretzky-Wald-Wolfowitz theorem on nonrandomized statistical decisions. SIM Theory Probability ppl., 5(6), pp [5] T.S. Ferguson. Mathematical Statistics. Decision Theoretic pproach. cademic Press, New York, [6] P. Loeb and Y. Sun. Purification of measure-valued maps. Illinois J. of Mathematics, 5(6), pp [7] K.R.Parthasarathy. Probability Measures on Metric Spaces. MS Chelsea Publishing, Providence, Rhode Island, 5. 9