ABSTRACT CONDITIONAL EXPECTATION IN L 2 Abstract. We prove that conditional expecations exist in the L 2 case. The L 2 treatment also gives us a geometric interpretation for conditional expectation. 1. Hilbert spaces Let H be a vector space with a real inner-product,. For h H, set h, h = h 2, so that distance between g, h H is given by g h. Many familar geometric facts hold, such as the Pythagorean theorem: g 2 + h 2 = g + h 2 if g, h = 0, and the paralleogram law: g + h 2 + g h 2 = 2 g 2 + 2 h 2. The Cauchy-Schwarz inequality: g, h g h can be used to show that H endowed with is in fact a metric space. Recall that if the metric induced by the norm makes H a complete metric space, then we call H a Hilbert space. Recall that a subset of C of H is convex if for every g, h C, we have that αg +(1 α)h C for every α (0, 1). Theorem 1 (Hilbert projection theorem). If C is a closed convex subset of a Hilbert space H, then for every h H, there exists a unique z C that realizes the infimum: δ := inf w h. w C In the case that C is a closed subspace of H, then it is necessary and sufficent that h z, w for every w C, for z C to be the minimizer. Proof. Recall that every closed subset of a complete space is also complete. Let w n be any minimizing sequence; that is, w n h δ. Note that 1 2 (w n + w m ) C, so that 1 2 (w n + w m ) h δ 2 ; so that the paralleogram law applied with 1 2 (w n + w m ) and h 1 2 (w n + w m ) implies that w n is a Cauchy sequence, which must have a limit z C for which z h = δ. To show unqiueness, let z, z C be two points that achieve the infimum, then the alternating sequence of z and z is
a minimizing sequence, and from the previous argument has a limit ẑ: clearly, ẑ = z = z. If C is also a subspace, then for any w C and any t R, we have z + tw C, so that h z tw h z. This implies that 2t w, h z + t 2 w 2 0. The expression is a convex quadratic in t, from which we deduce that w, h z = 0. For the converse, we assume z C is such that z h, w = 0 for all w C. Let w C. Note that z w C. The Pythagorean theorem with z w and h z imply that h w h z. Exercise 1.1 (Conditions on Theorem 1). Show that the condition that C is closed can not removed. Show that if C is not a subspace of H, then it might not be true that h z is orthogonal to all elements of C. Find an example where C is closed, but not convex, and there is more than one minimizer. Under the condition that C is closed, but not convex, must there exists a minimizer? 2. Conditional expectation via projection Let (Ω, F, P) be a probability space. Let L 2 the space of all random variables X such that E X 2 <. We will identify two random variables X and Y if P(X = Y ) = 1. Consider the inner-product on L 2 defined by X, Y = E(XY ). Theorem 2. Let (Ω, F, P) be a probability space. The space L 2 is a Hilbert space. Theorem 2 is usually proved in a functional analysis course. Remark 1. Let F be sub-sigma field of F. Let L 2 (F) denote the space of all X F such that E X 2 <. Applying Theorem 2 to the probability space (Ω, F, P), we have that L 2 (F) is a Hilbert space in its own right, so it is closed space of L 2. Exercise 2.1. Argue directly why L 2 (F) is closed subspace of L 2. Exercise 2.2. Let C[0, 1] be the space of all continuous functions f : [0, 1] R. Show that f, g := 1 0 f(x)g(x)dx defines an inner-product on C[0, 1]. However, show that C[0, 1] is not a Hilbert space.
Theorem 3 (Conditional expectation in L 2 ). Let (Ω, F, P) be a probability space, F be a sub-sigma algebra. Let Z be the minimizer of If X L 2, then E(X F) = Z. inf X Y. Y L 2 (F) Remark 2. Since Theorem 1, along with Remark 1 gives the existence of E(X F), we can use Theorem 3 to define conditional expectation in the case that X L 2. Exercise 2.3 (Coupling). Let X, Y, Z L 2, be such that (X, Z) d = (Y, Z). Show using Theorem 3 that E(X Z) = E(Y Z). Proof of Theorem 3. Let Z L 2 (F) be the minimizer given by Theorem 1. It suffices to verifty that E(ZY ) = E(XY ) for all Y L 2 (F).We have that E(ZY ) E(XY ) = Z, Y X, Y = Z X, Y = 0, Exercise 2.4. Show that if G F F are sub-sigma algebras, then for X L 2, we have that E ( X E(X F) ) 2 E ( X E(X G) ) 2. Exercise 2.5. Let X L 2 and W be random variables defined on the same probability space. Let F = σ(w ). Let Z be the minimizer of inf Y L 2 (F) X Y. Show that there exists a measurable function g : R R such that g(w ) = Z. We can use Theorem 3 to define conditional expecation in L 1. Recall that L 2 L 1 for probability spaces. Exercise 2.6 (Conditional expecation in L 1 ). Let (Ω, F, P) be a probability space and F be a sub-sigma algebra. Let 0 X L 1. Show that there exist 0 X n L 2, such that X n X in almost surely and in L 1. Pretend you only know how to do conditional expecation in L 2. Show that if we define Z := lim n E(X n F), then Z is a version of E(X F). Remark 3. With conditional expectation it is possible to give a proof of the Radon-Nikodym theorem. This really is a proof, provided that we use the L 2 construction of conditional expectation.
3. Von Neumann s mean ergodic theorem Let (Ω, F, µ) be a probability space and let T : Ω Ω be a measurepreserving map. The invariant sigma-algebra, is the set I of all events A F such that µ(a T 1 (A)) = 0. We say that T is ergodic if the invariant sigma-algebra is trivial. Let f : Ω R. We say that f is invariant if f T = f almost surely. Exercise 3.1. Show that T is ergodic if and only if every invariant measurable function is a constant almost surely. Theorem 4 (von Neumann). Let (Ω, F, µ) be a probability space and let T : Ω Ω be a measure-preserving map. For all real-valued f L 2, we have that 1 n 1 f T i L2 E(f I). n i=0 Here the conditional expecation is taken with respect to the probability measure µ. Exercise 3.2. Apply Theorem 4 to show that if X i L 2 are i.i.d. random variables and S n = X 1 + + X n, then S n /n EX 1 in L 2. In the next exercises we will prove this Theorem 4. Exercise 3.3. Let H be a Hilbert space, and suppose that C is a closed subspace of H. Let C = {h H : h, g = 0 for all g C}. Show that H = C C, so that for all h H, there exists unique c C and c C so that h = c + c. Exercise 3.4. Let B be the set of all f L 2 such that there exists g L 2 with f = g T g; the function f is called be coboundary, and g a transfer function. Show that the expectation of a coboundary is always zero. Exercise 3.5. Let f B. Show that for all h I, we have f, h = 0. Let C be closure of B in L 2. Show that C = I. Here we are abusing notation, and using I to denote the set of all invariant functions. Exercise 3.6. Prove Theorem 4 for functions in B, and its closure C. Exercise 3.7. Prove Theorem 4 for functions in I. Exercise 3.8. Prove Theorem 4.
Exercise 3.9. Let (Ω, F, µ) be a probability space and T : Ω Ω be measure-preserving. Show that T is ergodic if and only if 1 n 1 µ(a T k B) µ(a)µ(b). n k=0 Hint: Use the fact that if f n f in L 2, then f n, g f, g, for all g L 2.