Weak convergence in metric spaces

Transcription

1 Chapter 1 Weak convergence in metric spaces 1.1 Definition of weak convergence Let (X, d) be a metric space. An obvious candidate for a σ-algebra on X having some relation to the metric is the Borel algebra B(X)=σ ( {G X G is an open set} ). Experience from ther k -case suggest that this is a natural choice ofσ-algebra. Indeed this is true if the metric space is separable (meaning that there exists a countable dense subsetc X). But as luck will have it, we will mainly be working in nonseparable spaces, and among the surprising features of such spaces is the discovery that the Borel algebra contains too many sets to be manegeable. Example 1.1 Pollards example IV.2 considers the mapφ :R D[0, 1] given by φ(x)(t)=1 [x,1] (t) for x, t [0, 1] The graph ofφ(x) for some x [0, 1] is shown in the figure below. Note thatφ(x) is the distribution function for the one-point measure in the point x. Note also that the natural extensionφ n :R n D[0, 1] given by φ n (x 1,..., x n )= 1 n φ(x i ) n 1 i=1

2 2 Chapter 1. Weak convergence in metric spaces gives the empiricial distribution function of the observations x 1,..., x n. Hence the mapφis a cornerstone of the theory we are about to build. φ(x)(t) x 1 t Figure 1.1: The graph of φ(x) for a fixed x [0, 1]. Note that this is a distribution function, namely the distribution function for the one-point measure in x. If x 1 < x 2 we see that φ(x 1 )(t) φ(x 2 )(t)= 0 if t [0, x 1 ) 1 if t [x 1, x 2 ) 0 if t [x 2, 1] If denotes the uniform norm on D[0, 1] it follows that φ(x 1 ) φ(x 2 ) =1 for x 1 x 2 We conclude that a ball of radius 1/2 in D[0, 1] can at most contain a singleφ(x)- function. Consider for any set A [0, 1], Borel-measurable or not, the D[0, 1]-subset ( G A = B φ(x), 1 ). 2 x A As a union of open balls we see that G A is an open subset of D[0, 1]. And from the above considerations on ball of radius 1/2 it follows thatφ 1 (G A )=A. Hence, if we equip D[0, 1] with the Borel algebra related to the uniform norm, then the onlyσ-algebra on [0, 1] makingφmeasurable is theσ-algebra of all subsets. In particularφ is non-measurable ifris equipped with the usual Borel algebraband D[0, 1] is equipped with the Boral algebra arising from the uniform norm. If we want to sayφis measurable (and we certainly do), we will have to reduce the σ-algebra on D[0, 1] by excluding many (perhaps even most) open sets.

3 1.1. Definition of weak convergence 3 To circumvent the problem ofb(x) being too large, we will fix a sub-σ-algebra A B(X) as the basis for the subsequent measurability considerations. The typical choice is the ball algebra B 0 (X)=σ ( {B(x, r) X x X, r>0} ). We have defined the ball algebra in terms of open balls. But it is easy to establish that the family of closed balls generate the sameσ-algebra as the family of open balls, so we could equally well have used closed balls for the definition. It is easy to show that ifxis separable, then it holds thatb 0 (X)=B(X) - just copy the standard proof that the Borel algebrab k onr k is generated by balls (or by boxes or by virtually any class of nice sets). But if X is non-separable, the ball algebra will be significantly smaller than the Borel algebra. In particular, most open sets will not be ball-measurable. In many cases, X will be a space of real-valued functions of some sort, say a subset of the functionsy R satisfying some conditions. HereYis just the set on which these particular functions are defined. The structure of a function space gives an alternative possibility for the choice of a σ-algebra, namely the projection algebra, usually denoted by P(X). This is defined as the σ-algebra generated by all the projections π y :X R given by π y (x)= x(y) for x X, y Y. Very frequently it happens that the projection algebra and the ball algebra coincides. But this has to be established on a case-by-case basis, and in some function spaces it will fail. When equality fails, it is usually because the projection algebra is strictly smaller than the ball algebra. In the sequel we consider a fixed metric space (X, d) and a fixed choice ofσ-algebra A B(X). These choices will be taken tacitly in the formulation of the definitions and theorems. Definition 1.2 Letµ,µ 1,µ 2,... be probability measures on (X,A). We say thatµ n wk converges weakly toµ, and we writeµ n µ for n, if it holds that f dµ n f dµ for n (1.1) for every function f : X R which is continuous, bounded and A B measurable.

4 4 Chapter 1. Weak convergence in metric spaces The surprising feature about this defintion is that we do not require (1.1) to hold for every continuous bonded function, but only for those that are A-measurable. Indeed, the condition does not even make sense for a function that is non-measurable with respect toa: the measures involved are only assumed to be defined ona, making the integrals non-sensical for integrands that are not A-measurable. Depending on the choice of A, we will expect quite a few continuous maps to be non-measurable with respect toa. If we are not careful, this can be taken to extremes: Example 1.3 Consider a metric space (X, d) equipped with the trivialσ-algebra, A={,X}. With this choice, only constant functions are A-measurable. In particular, the only continuous, bounded, A-measurable functions are the constants. This has the perculiar consequence that (1.1) is always satisfied! Thus, any sequence of probability measures on (X,A) will converge weakly to any limit. The whole theory becomes vacuous in this case. For practical handling of weak converges, we will usually rely on a formulation with stochastic variables: Definition 1.4 Let X, X 1, X 2,... be a sequence of stochastic variables defined on a background space (X,F, P) with values in (X,A). We say that X n converges in distribution to X, written X n D X for n if it holds that f (X n ) dp f (X) dp for n (1.2) for every function f : X R which is continuous, bounded and A B measurable. Clearly it holds that X n D X Xn (P) wk X(P) The real thing is the sequence of measures converging weakly. The concept of stochastic variables converging in distribution is mainly introduced because it is an efficient language.

5 1.2. Regularity conditions Regularity conditions We noted that the role ofawas to prevent us from having too many measurable sets and functions. But for the concept of weak convergence to be interesting at all, it is important that sufficiently many continuous functions are A-measurable - we have to protect ourselves from the ridiculous behavior exhibited in exampel 1.3. So it is necessary that A is small, but not too small. Pollard formulates this requirement through the notion of completely regular points. It is not a good name, and Pollard makes fun of it himself - the word regularity can mean anything, and hence there is no intuition attached to it. It is also a problematic concept in Pollards version: it is slightly too mild (at least to my taste), and it makes some of the subsequent arguments quite hard. Hence we will introduce our own variant, along with Pollards. Recall the bump function notation: If A B are two subsets ofx, then a function f :X R is said to be A f B if 1 A (x) f (x) 1 B (x) for every x X Let us spell it out: if A f B it holds that f (x) [0, 1] f (x)=1 f (x)=0 for every x X for x A for x B In many applications a bump function will tacitly assumed to be continuous (or smooth), but we will usually take some effort to spell out exactly what we requires of the bump functions we bring into play. Definition 1.5 A point x X is DP-regular (or completely regular in the sense of Pollard) if there for any r > 0 exists a function f : X R which satisfies {x} f B(x, r) (1.3) and which is f is uniformly continuous anda-measurable. For an illustration of the bump function required by the notion of DP-regularity, see the left hand panel of figure 1.2. Note that the real requirement is not so much the existence of a bump function of this shape - they exist around every point in every

6 6 Chapter 1. Weak convergence in metric spaces metric space. It is not even the requirement that the function is uniformly continuous. The key is the combined claim that the function is uniformly continuous as well as A-measurable. Uniform continuity makes certain arguments possible. But often an even stronger form of regularity can make a world of difference. Recall that a function f :X R is Lipschitz if there is a constant C> 0 such that f (x) f (y) C d(x, y) for every x, y X. A Lipschitz function is of course uniformly continuous, but in a very specific way. The class of Lipschitz functions is stable under formation of linear combination, and also under the formation of pointwise minimum and maximum of two functions. Definition 1.6 A point x X is EH-regular if there for any r > s>0 exists a function f :X Rwhich satisfies and which is f is Lipschitz anda-measurable. B(x, s) f B(x, r) (1.4) DP regularity EH regularity x r x x+r t x r x s x x+s x+r t Figure 1.2: The shape of the bump functions required to exist by the two regularity conditions, illustrated withx=r. On the left is a function which satisfies that{x} f B(x, r). On the right is a function which which satisfies that B(x, s) f B(x, r). The function on the right of course also satisfies the condition on the left. For an illustration of the bump function required by the notion of EH-regularity, see the right hand panel of figure 1.2. Note that again that the real requirement is not so much the existence of a bump function of this shape - it is the combination of the shape, the regularity and the measurability.

7 1.2. Regularity conditions 7 It is obvious that a point which is EH-regular is also DP-regular. It is conceivable that DP-regular points need not be EH-regular. However, we have the following result: Lemma 1.7 Let (X, d) be a matric space. IfA=B 0 (X) is the ball algebra onxthen every point x X is EH-regular. Proof: The first key observation is that for fixed x 0 X the map x d(x, x 0 ) is A B measurable. This follows because B is generated by the paving {(, r) r } and we can easily compute the pre-images of this paving: { B(x0, r) if r>0 {x d(x, x 0 ) (, r)}= if r 0 which in both cases is ball measurable. Note also that the backwards triangle inequality, d(x, x 0 ) d(y, x 0 ) d(x, y) for every x, y X shows that x d(x, x 0 ) is Lipschitz with Lipschitz-constant 1. The second key observation is that the function g :R R with the graph below is B B measurable, bounded and Lipschitz. g(t) s r t Figure 1.3: The graph of g(t).

8 8 Chapter 1. Weak convergence in metric spaces We can write a formula for g as and we can observe that g(x)= 1 if x s r x r s if x [s, r] 0 if x r { { r x }} g(x)=min 1, max r s, 0 We have constructed g from three affine functions (well, two of them are actually constants). Affine functions are obviously Lipschitz, and due to the stability of the Lipschitz property under pointwise minimum and maximum, it follows that g is Lipschitz. Combining the pieces, we see that f (x)=g(d(x, x 0 )) isa-measurable and Lipschitz and satisfies that B(x 0, s) f B(x 0, r), and we conclude that x 0 is EH-regular. A very impontant lesson from lemma 1.7 is that as long as we stick to the ball algebra, all points are regular, even in the strong EH-sense. Hence regularity is not really an issue at all. Still, Pollard chooses to keep the concept in all his theorem to pretend that he is arguing with a potentially smaller choice of A than the ball algebra. I have personally not seen a single example, where this extra generality is relevant, so I consider Pollards gain as fictive. However, we will keep our theorems in line with Pollards formulation. Theorem 1.8 Let h : X R be a function. Suppose that 1) h(x) 0 for every x X 2) h is continuous in every point x C where C X is a separable set, consisting of EH-regular points. Then there is a sequence 0 f 1 f 2... of bounded and LipschitzA-measurable functions such that a) f n (x) h(x) for every x X b) f n (x)րh(x) for every x C

9 1.2. Regularity conditions 9 Proof: Let C 0 C be a countable, dense subset. For each x C 0 and each s, r Q + with s < r we can apply EH-regularity to choose a A-measurable Lipschitz function f x,s,r such that B(x, s) f x0,s,r B(x, r). Consider the collection of functions D={q f x,s,r x C, s, r, q Q +, s<r} These functions are of course A-measurable, Lipschitz and bounded. Clearly D is countable. Consider the subset D ={ f D f h} ThenD is also countable, and can be ordered in some way, say Define D ={g 1, g 2,...} f n = max{g 1,...,g n } Clearly 0 f 1 f 2... and clearly each f n isa-measurable, Lipschitz, bounded and dominated by h. All we need is to establish property b) of the theorem. So take y C. If h(y)=0there is nothing to prove, so we may assume that h(y)>0. Take q Q + such that h(y)>q. Since h is continuous in y, we can find r Q + such that h(z)>q for every z B(y, r) Find x C 0 B(y, r/3). Observe that y B(x, r/3) B(x, 2r/3) B(y, r) by the triangle inequality. This implies that q f x,r/3,2r/3 h and so q f x,r/3,2r/3 D. In particular we can find n 0 N such that q f x,r/3,2r/3 = g n0. Since f n is the pointwise maximum of a number of g n -functions, it follows that f n (y) g n0 (y)=q for n n 0. In conclusion we see that f n (y)րh(y).

10 10 Chapter 1. Weak convergence in metric spaces We could actually obtain a slightly stronger statements from the above proof. This stronger statement is occasionally useful. Observe that D depends on C, but not on the function h. If we let D { = max{g1,...,g n } g 1,...,g n D } n=1 consist of the pointwise maximum of finitely manyd-functions, thend is also countable. We observe that the sequence f 1, f 2... obtained in the proof of theorem 1.8 is indeed a sequence ofd -functions. So we can assume that the approximating functions are chosen from a fixed countable family, applicable to every h. If we start out with a separable spacexwe may take C=X. If every point inxis EH-regular (for instance because A is chosen as the ball algebra, which in this separable setting equals the Borel algebra), then for every nonnegative continous function h, we can find a sequence ofd -functions f 1, f 2,... such that f n ր h in every point. We shall in the exercises see how this statement implies that C(X), the space of all continuous real-valued functions, is separable in the uniform norm whenxis a compact metric space. 1.3 The continuous mapping theorem Under theorem 1.8 we can reduce the burden when checking weak congergence quite a bit: Lemma 1.9 (Convergence lemma) Let X 1, X 2,... be stochastic variables, defined on (Ω, F, P) and with values in (X, A). Suppose there exists a separable subset C X consisting of EH-regular points such that P(X C) = 1. If f (X n ) dp f (X) dp for n for every bounded,a-measurable Lipschitz function f, then it holds that X n D X. The convergence lemma shows that when testing for convegence in distribution, we only need to check (1.2) for Lipschitz functions, which is a considerable reduction in complexity.

11 1.3. The continuous mapping theorem 11 Instead of proving the convergence lemma, we will proceed directly to prove an even stronger result which essentially has the same proof: Lemma 1.10 (Extended convergence lemma) Let X 1, X 2,... be stochastic variables, defined on (Ω,F, P) and with values in (X,A). Suppose there exists a separable subset C X consisting of EH-regular points such that P(X C) = 1. Assume that f (X n ) dp f (X) dp for n (1.5) for every bounded,a-measurable Lipschitz function f. Let h :X R be bounded and A-measurable. If h is continuous in every point x C, then it holds that h(x n ) dp h(x) dp for n Proof: Find c>0 such that h+c 0. We can use theorem 1.8 to find bounded, A-measurable Lipschitz functions 0 f 1 f 2... bounded by h+c and in particular satisfying that f n ր h+c on C. By the general domination we find that for any fixed m it holds that f m (X n ) dp h(x n ) dp+ c Since the f n -functions by assumption satisfy (1.5), it follows by letting n that f m (X) dp lim inf h(x n ) dp+ c n for any fixed m. But observe that f m (X)րh(X)+c P-almost surely. The monotone convergence theorem thus implies that f m (X) dp h(x) dp+ c Combining these two sources of information, we obtain that h(x) dp lim inf h(x n ) dp n where the constant c has disappeared. Note that h satisfies the same conditions as h, so we also have that h(x) dp lim inf h(x n ) dp= lim sup h(x n ) dp n n

12 12 Chapter 1. Weak convergence in metric spaces Multiplying by 1 and collecting the two inequalities, we obtain that h(x) dp lim inf h(x n) dp lim sup h(x n ) dp h(x) dp n which of course shows that h(x n ) dp n h(x) dp for n as desired. As an example of the use of the extended convergence lemma, suppose that X n D X where X is concentrated on a separable subset ofx, consisting of EH-regular points. Consider ana-measurable subset A X with the property that P(X A)=0. Observe that 1 A is continous in every point not in A and that X is concentrated on C\ A. Using the extended convergence lemma, it follows that P(X n A) P(X A) for n. This the abstract anaogue of the classical result in one dimension that if a sequence of real-valued stochastic variable converge in distribution, then the corresponding distribution functions will converge pointwise in every point, except possibly in the points of positive probability for the limit distribution. Corollary 1.11 (Continuous mapping theorem) Let (X, d) and (X, d ) be two metric spaces, equipped withσ-algebrasaanda (both sub-σ-algebras of the respective Borel algebras). Let X, X 1, X 2,... be stochastic variables with values in (X, d) and suppose that X n D X. Suppose there exists a separable subset C X consisting of EH-regular points such that P(X C)=1. Let H :X X bea A measurable, and suppose H is continuous in every point x C. Then it holds that H(X n ) D H(X). Proof: Trivial from the extended convergence lemma. Note the somewhat unusual choice of language: the continuous mapping theorem is primarily concerned with non-continuous maps.... If the transformation H is properly continuous, the content of corollary 1.11 is trivial, and not worth a name - the

13 1.3. The continuous mapping theorem 13 power of the result lies in the fact that it allows transformations with some amount of non-continuity - the non-continuity only has to be located in regions of space which are irrelevant, seen from the perspective of the limit distribution.