Lecture 3: Probability Measures - 2 1. Continuation of measures 1.1 Problem of continuation of a probability measure 1.2 Outer measure 1.3 Lebesgue outer measure 1.4 Lebesgue continuation of an elementary measure 1.5 Continuation of an elementary measure to the minimal σ- algebra 2. Measures and distribution functions 2.1 Probability measure and its distribution function 2.2 Lebesgue measure 3. Example of non-borel set 1. Continuation of measures 1.1 Problem of continuation of a probability measure Let A be an algebra of subsets of a space Ω. Definition 3.1. An elementary probability measure is a function P (A) defined on sets A A and satisfying the following conditions: (a) P (A) 0, A F; (b) If A 1, A 2,..., A n is a finite sequence of disjoint events, i.e., 1
A i A j =, i j then P ( n k=1a k ) = n k=1 P (A k ); (c) P (Ω) = 1. Definition 3.2. A probability measure P (A), A σ(a) is a continuation of an elementary measure P (A), A A if P (A) = P (A), A A. The following fundamental questions arise: (1) Does exist a continuation P (A), A σ(a) of an elementary measure P (A), A A? (2) Is this continuation unique? Example Let F (x), x R 1 is a distribution function on a real line, i.e., a real-valued function defined on R 1 satisfying conditions: (a) F (x) is monotonically non-decreasing function; (b) F ( ) = lim x F (x) = 0, F ( ) = lim x F (x) = 1; (c) F (x) is continuous from the right, i.e., lim x y x F (y) = F (x), x R 1. Let also A be the class of all finite unions of disjoint intervals A = n i=1(a i, b i ], a i b i + ((a, ] = (a, )), (a i, b i ] (a j, b j ] =, i j, n = 1, 2,.... Lemma 3.1. The class A is an algebra and the function P (A) = n i=1 (F (b i ) F (a i )) is an elementary measure defined on algebra A. The above questions take the following form: (1) Does exist a probability measure P (A), A σ(a) such that 2
P ((, x]) = F (x), x? (2) Is this continuation unique? 1.2 Outer measure Definition 3.3. A function of sets λ(a) defined on all subsets A Ω taking values in the interval [0, ] is an outer measure if it satisfies conditions: (a) λ( ) = 0; (b) If A, A 1,... Ω is a finite or countable sequence of sets such A n A n then λ(a) λ(a n ). n The property (b) is so-called sub-additivity property. Definition 3.4. A set E Ω is called measurable if, λ(a) = λ(a E) + λ(a \ E), A Ω. Theorem 3.1**. The class F λ of all measurable sets is a σ- algebra and the outer measure λ(a) is a measure on this σ- algebra. (a) Lemma 3.2. Sets, Ω F λ. (b) Lemma 3.3. If E is a measurable set then E is a measurable set. (c) Lemma 3.4. If E is a measurable set, A, B Ω and E B =, then, λ(a (B E)) = λ(a B) + λ(a E). 3
(d) Lemma 3.5. If E 1,..., E n are measurable sets, A, B Ω and E k B =, E k E r =, k, r = 1,..., n, then, λ(a (B ( n k=1e k ))) = λ(a B) + n λ(a E k ). (e) Lemma 3.6. The class F λ is an algebra. (f) Lemma 3.7. If E 1, E 2,... and E k E r =, k, r = 1,..., then, λ(a ( k=1e k )) = λ(a E k ). k=1 k=1 (g) Lemma 3.8. The class F λ is an σ-algebra. (h) Theorem 3.1 is a corollary of Lemmas 3.7 and 3.8. Lemma 3.9. The σ-algebra F λ is full, i.e., any subset A A of a measurable set A such that λ(a ) = 0 is also a measurable set and λ(a) = 0. 1.3 Lebesgue outer measure Let A ba a class of subsets of Ω such that, Ω A. Let also µ(a), A A is a function of sets taking values in the interval [0, ] such that µ( ) = 0. Let now define a function of sets λ(a), A Ω as, λ(a) = inf{ n=1 µ(a n ) : A n A, A n=1a n }. 4
Theorem 3.2. The function λ(a), A Ω is an outer measure. (a) λ( ) = 0 follows from relation µ( ) = 0; (b) λ(a), A Ω is sub-additive function; (c) If λ(a n ) = for some n then the property of sub-additivity is obvious; (d) If λ(a n ) <, n = 1, 2,... then, by definition of λ(a), for every ε > 0, there exist sets A n,m A, n, m = 1, 2,... such that A n m A n,m, n = 1, 2,... and m µ(a n,m ) λ(a n ) + ε (e) λ(a) n m µ(a n,m ) n λ(a n ) + ε; (f) Passing ε 0 we get λ(a) n λ(a n ). Definition 3.5. The outer measure λ(a), A Ω is called Lebesgue outer measure and the measure λ(a), A F λ is called Lebesgue measure (constructed from the function of sets µ(a), A A). 1.4 Lebesgue continuation of an elementary measure Let µ (A), A A and µ (A), A A be two functions of sets. Definition 3.6. The function µ (A), A A is a continuation of function µ (A), A A if: (a) A A ; (b) µ (A) = µ (A), A A. Let A be an algebra of subsets of a space Ω. 2 n ; 5
Definition 3.7. An elementary measure is a function µ(a) defined on sets A A and taking values in the interval [0, ] is am elementary measure if it satisfies the following conditions: (a) µ( ) = 0; (b) If A 1, A 2,..., A n is a finite sequence of disjoint events, i.e., A i A j =, i j then µ( n k=1a k ) = n k=1 µ(a k ). Theorem 3.3**. The Lebesgue measure λ(a), A F λ constructed from an elementary measure µ(a), A A is a continuation of this elementary measure, i.e., A F λ and λ(a) = µ(a), A A, if and only if the elementary measure µ(a), A A is subadditive, i.e., for any finite or countable sequence of sets A, A 1, A 2,... A, A n A n, µ(a) µ(a n ). n (a) Necessity: If λ(a), A F λ is a continuation of the elementary measure µ(a), A A then the sub-additivity of µ(a) follows from sub-additivity of the outer measure λ(a); (b) Sufficiency is not trivial. Definition 3.8. An elementary measure µ(a), A A is σ- additive if it satisfies the following condition: (a) If A 1, A 2,..., is a infinite sequence of disjoint events, i.e., A i A j =, i j such that n=1a n A then, µ( n=1a n ) = n=1 µ(a n ). 6
Lemma 3.10. If an elementary measure µ(a), A A is σ- additive then it is also a sub-additive. (a) Let A, A k, k = 1, 2,... A, A k A k. Then A = k (A A k ); (b) B n = n k=1(a A k ). Then A = n B n = n (B n \ B n 1 ), where B 0 = ; (c) B n, B n \ B n 1 A, n = 1, 2,..., B n \ B n 1, n = 1, 2,... are disjoint sets; (d) µ(a) = k=1 µ(b k \ B k 1 ) = lim n nk=1 µ(b k \ B k 1 ) = lim n µ(b n ) = lim n nk=1 µ(a A k ) lim n nk=1 µ(a k ) = n µ(a k ). 1.5 Continuation of an elementary measure to the minimal σ-algebra Let A be an algebra of subsets of a space Ω and µ(a), A A be an elementary mesure defined on this algebra. Definition 3.9. An elementary measure µ(a), A A is σ- finite if there exist a the sequence of sets A 1, A 2,... A such that n A n = Ω and µ(a n ) <, n = 1, 2,.... Theorem 3.4*. Let µ (A), A σ(a) and µ (A), A σ(a) are two continuations of a σ-finite elementary measure µ(a), A A on the minimal σ-algebra σ(a). Then µ (A) = µ (A), A σ(a), 7
i.e., the continuation of an elementary σ-finite measure from an algebra to the corresponding minimal σ-algebra is unique. (a) The proof may be reduced to the case of finite measure µ(a), A A, i.e., where µ(ω) < ; (b) Let R be the class of sets A σ(a) such that µ (A) = µ (A); (c) R is a monotonic class; (d) Also A R; (e) Thus, σ(a) R. The following theorem is the direct corollary of Theorems 3.3 and 3.4. Theorem 3.5. If µ(a), A A is a σ-finite and sub-additive elementary measure defined on algebra A, then A σ(a) F λ and the Lebesgue measure λ(a), A F λ constructed from an elementary measure µ(a), A A is a continuation of this elementary measure defining also the measure λ(a), A σ(a) that is the unique continuation of the elementary measure µ(a), A A to the minimal σ-algebra σ(a). The following theorem describes the relationship between the σ- algebras σ(a) and F λ and the Lebesgue measure λ(a), A F λ and the measure λ(a), A σ(a). Let us introduce the class of zero-sets, N λ = {A Ω : A A, A σ(a), λ(a ) = 0}. Theorem 3.6. If µ(a), A A is a σ-finite and sub-additive 8
elementary measure defined on algebra A, then the σ-algebras σ(a) and F λ are connected by the following relation, F λ = {B = (A A ) \ A, A σ(a), A, A N λ }, while the measures λ(a), A F λ and λ(a), A σ(a) are connected by the relation, λ(b) = λ(a), B = (A A ) \ A, A σ(a), A, A N λ. Measure λ(a), A F λ is called a replenishment of the measure λ(a), A σ(a). Corollary 1. Let A σ(a) and ε > 0. Then there exist sets A 1, A 2,... A such that A n=1a n such that µ(a) n=1 µ(a n ) µ(a) + ε. 2. Measures and distribution functions 2.1 Probability measure and its distribution function Let F (x), x R 1 is a distribution function on a real line, i.e., a real-valued function defined on R 1 ; A be the algebra of all finite unions of disjoint intervals A = n i=1(a i, b i ] and P (A) be the elementary measure defined on algebra A by formulas P (A) = n (F (b i ) F (a i )), A A. (1) i=1 Lemma 3.11*. The elementary measure P (A) is σ-additive. The following theorem is a corollary of Theorem 3.6 and Lemma 3.11. 9
Theorem 3.7. If P (A), A A is an elementary measure defined on algebra A by relation (1), then the Lebesgue measure F (A), A F F constructed from an elementary measure P (A), A A is a continuation of this elementary measure defining also the measure F (A), A σ(a) that is the unique continuation of the elementary measure P (A), A A to the Borel σ-algebra B 1 = σ(a). Let F be the family of all distribution functions, i.e., the family of real-valued functions F (x) defined on R 1 satisfying conditions: (a) F (x) is monotonically non-decreasing function; (b) F ( ) = lim x F (x) = 0, F ( ) = lim x F (x) = 1; (c) F (x) is continuous from the right, i.e., lim x y x F (y) = F (x), x R 1. Let also P be the class of all probability measures, on Borel σ- algebra B 1. The following theorem is a corollary of Theorem 3.7. Theorem 3.8. There is one-to-one mapping between the families F and P. The mapping P F is given by the relations F (x) = P ((, x]), x R 1 while the inverse mapping F P is given by the algorithm described in the Theorem 3.7. 2.2 Lebesgue measure Let m(x), x R 1 is a function on a real line satisfying conditions: (a) m(x) is monotonically non-decreasing function; (b) m(x) is continuous from the right, i.e., lim x y x m(y) = m(x), x R 1. Let also A be the algebra of all finite unions of disjoint intervals 10
A = n i=1(a i, b i ] and m(a) be the function of sets defined on algebra A by formulas, m(a) = n (m(b i ) m(a i )), A A. (2) i=1 Note that m(r 1 ) < iff both m( ) = lim x m(x) > and m( ) = lim x m(x) <. Lemma 3.12*. The function m(a) is a σ-finite and σ-additive elementary measure defined on algebra A. The following theorem is a corollary of Theorem 3.6 and Lemma 3.11. Theorem 3.9. If m(a), A A is an elementary measure defined on algebra A by relation (2), then the Lebesgue measure M(A), A F M constructed from an elementary measure of sets m(a), A A is a continuation of this elementary measure defining also the measure M(A), A σ(a) that is the unique continuation of the elementary measure m(a), A A to the Borel σ-algebra B 1 = σ(a). The case of the standard Lebesgue measure λ(a), A B 1 on Borel σ-algebra B 1 corresponds to the model with function m(x) = x, x R 1. 3. Example of a non-borel set Let denote the λ(a), A F λ the replenishment of the standard Lebesgue measure λ(a), A B 1. 11
Let also N be the set of integer numbers. (1) Take some irrational number a and define the set A = {b = n + ma : n, m N. (2) Define equivalence relation between any to real numbers x y if x y A. This equivalence relation is reflective (x x), symmetric (y x if x y) and transitive (if x y and y z then x z). (3) This follows from the listed above properties of the equivalence relation that the set of real numbers splits in the family E a of disjoint sets such that every set from E a contains all real numbers that are connected by the relation with each others. The family E a has a continuum elements (it admit a one-to-one mapping with the set of all real numbers). (4) According so-called axiom of choice there exists the set E 0 that contains exactly one point from every set E E a. Theorem 3.10**. The set E 0 does not belong to the σ-algebra F λ. (a) If E 0 F λ then λ(e 0 ) = 0; (b) λ(e 0 + b) = λ(e 0 ) = 0, b A; (c) E 0 + b E 0 + b =, b, b A; (d) b A (E 0 + b) = R 1 ; (e) λ(r 1 ) = 0?? (f) Thus E 0 F λ. 12
Halmos P.R. (1950) Measure Theory, Van Nostrand, New York. LN Problem 1. Prove of find in the R, G or GS* the proofs of Lemmas 3.2 3.8. * GS: Gikman, I.I., Skorokhod, A.V. (1996) Introduction to the Theory of Random Processes, Dover, Mineola. 2. Let λ(a) = I A (a) where a Ω. Is λ(a) a outer measure? What is the σ-algebra F λ of measurable sets? 3. Prove or find in R, G or GS the proof of the sufficiency statement of Theorem 3.3. 4. Complete the sketch of the proof for Theorem 3.4. 5. Prove or find in R, G or GS the proof of Theorem 3.6. 6. The probability space < Ω, F, P > ia called non-atomic if for every A F with P (A) > 0 there exist B F, B A such that o < P (B) < P (A). Prove that the set of values P (A), A F coincides with the whole interval [0, 1]. 7. Prove or find in R, G or GS the proof of Lemmas 3.11 and 3.12. 13