THE HILBERT SPACE L
Definition: Let (Ω,A,P be a probability space. The set of all random variables X:Ω satisfying is denoted as L. EX < Remark: EX < implies that E X < (or equivalently that EX, because X X + E X EX +. Proposition: The set L together with the pointwise scalar multiplication defined for X L and λ by (λx(ω=λ(x(ω, ω Ω and the pointwise addition defined for X,Y L by is a vector space. (X+Y(ω=X(ω+Y(ω, ω Ω Proof: (i The two operations are closed because and X L, λ EX < E(λX =λ EX < λx L X,Y L EX, EY < E(X+Y E(X +Y < X+Y L.
(ii The associative, commutative, and distributive properties (X+Y+Z=X+(Y+Z, (λµx=λ(µx, X+Y=Y+X, λ(x+y=(λx+(λy, (λ+µx=(λx+(µx follow immediately from the pointwise definitions of the two operations. For example, if X,Y,Z L then ((X+Y+Z(ω =(X+Y(ω+Z(ω =(X(ω+Y(ω+Z(ω =X(ω+(Y(ω+Z(ω =X(ω+(Y+Z(ω =(X+(Y+Z(ω, ω Ω. (iii The random variable 0 which is identically zero on Ω satisfies the property of a zero vector. X+0=X X L (iv For all X S there exists an inverse vector -X defined by (-X(ω=-(X(ω, ω Ω, satisfying -X+X=0. (v X=X
Exercise: Show that a function < >:L L can be defined by <X,Y>=EXY, which satisfies for X,Y,Z L and λ <X+Y,Z>=<X,Z>+<Y,Z>, <λx,y>=λ<x,y>, <X,Y>=<Y,X>, <X,X> 0. Solution: - <E(-X -Y EXY E(X +Y < EXY, <X+Y,Z>=E(X+YZ=EXZ+EYZ=<X,Z>+<Y,Z>, <λx,y>=e(λxy=λexy=λ<x,y>, <X,Y>=EXY=EYX=<Y,X>, <X,X>=EXX=EX 0. 3
The function < > satisfies all the properties of an inner product except for <X,X>=0 X=0, because EX =0 implies only that P(X=0=, but not that X(ω=0 for all ω Ω. Analogously, the function satisfies all the properties of a norm except for X =0 X=0. To circumvent this problem we identify two random variables if they are equal almost surely, i.e., we switch from the individual random variables X L to equivalence classes [X]={Y L : P(Y=X=} of random variables which agree almost everywhere. Definition: Defining for equivalence classes [X], [Y] of almost surely equal elements of L and λ [X]+[Y]=[X+Y], λ[x]=[λx], <[X],[Y]>=<X,Y> we obtain an inner product space, which is denoted by L. 4
Proposition: The inner product space L of equivalence classes of almost surely equal random variables with finite variances is complete, i.e., X n L for all n, Thus L is a Hilbert space. Remark: Norm convergence X X 0 X L : X n X 0. m n X n X 0 is equivalent to mean square convergence n X X =E(X n -X 0. Exercise: Show that the relation ~ defined by X~Y P(X=Y= is indeed an equivalence relation by verifying the reflexive, symmetric, and transitive properties X~X, X~Y Y~X, X~Y,Y~Z X~Z X,Y,Z L. 5
Solution: The transitive property is satisfied, because {ω:x(ω=z(ω} {ω:x(ω=y(ω=z(ω} {ω:x(ω=z(ω} C {ω:x(ω=y(ω=z(ω} C = ({ω:x(ω=y(ω} {ω:y(ω=z(ω} C = {ω:x(ω=y(ω} C {ω:y(ω=z(ω} C P({ω:X(ω=Z(ω} C P({ω:X(ω=Y(ω} C + P({ω:Y(ω=Z(ω} C. Proposition: If E(X n -X 0 and E(Y n -Y 0, then Proof: (i EX n EX, (ii EX n Y n EXY, (iii Cov(X n Y n Cov(X,Y, (iv Var(X n Var(X. (i EX n =EX n = X n, X, =EX =EX (ii EX n Y n = X n, Y n X, Y =EXY (iii Cov(X n,y n =EX n Y n -EX n EY n EXY-EXEY=Cov(X,Y (iv Var(X n =Cov(X n,x n Cov(X,X=Var(X 6
Definition: The conditional expectation of X L given a closed subspace S L, which contains the constant function, is defined to be the projection of X onto S, i.e., E(X S=P S (X. Remark: The conditional expectation satisfies X E(XS < X Y for all other elements of S. Definition: The conditional expectation of X L given X,,X n L is defined to be the projection of X onto the closed subspace M(X,,X n spanned by all random variables of the form g(x,,x n, where g is some measurable function g: n, i.e., Remarks: (i It follows from E(X X,...,Xn = P M(X,...,X (X. n that X E(X X span(,x,,x n M(X,,X n,...,xn X E(X span(,x.,...,xn (ii For elements of L the definition of E(X X,...,Xn above coincides with the more general defnition of conditional expectation as the mean of the conditional distribution. 7
Exercise: Show that the bivariate normal density f(x=f(x,x = (π det Σ T µ exp( (x µ Σ (x with mean vector µ=(µ,µ T and covariance matrix Σ= = ρ ρ factors into two univariate normal densities, the marginal density f with mean µ and variance and the conditional x density f with mean µ + ρ and variance (-ρ. x Solution: Putting z =, z = we obtain (x-µ T Σ - (x-µ= = (x x x ρ (x T ( ρ x ρ (x ρ ( ρ + and completing squares (x x x z ρzz = ρ + z z ρ = ρ z ρ z ρz + ρ z + z = (z ρz z + ρ. Thus, f(x,x = π exp(- z π ( ρ exp(- (z ρz ρ. 8
Remark: The last exercise shows that in the case of a bivariate normal random vector (X,X the mean of the conditional distribution of X given X is a linear function of and X. More generally, if (X,X,,X n T has a multivariate normal distribution, then E(X X,...,Xn =E(X span(,x,...,xn. 9