Probability and Measure

Transcription

1 Probability ad Measure Stefa Grosskisky Cambridge, Michaelmas 2005 These otes ad other iformatio about the course are available o stefa/teachig/probmeas.html The text is based o ad partially copied from otes o the same course by James Norris, Ala Stacey ad Geoffrey Grimmett. Cotets 1 Itroductio 3 2 Set systems ad measures Set systems Measures xtesio ad uiqueess Lebesgue-Stieltjes measure Idepedece ad Borel-Catelli lemmas Measurable Fuctios ad Radom Variables Measurable Fuctios Radom Variables Covergece of measurable fuctios Itegratio Defiitio ad basic properties Itegrals ad limits Itegratio ad differetiatio Product measure ad Fubii s theorem L p -spaces Norms ad iequalities Completeess of L p L 2 as a Hilbert space Covergece i L

2 6 Characteristic fuctios ad Gaussia radom variables Defiitios Properties of characteristic fuctios Gaussia radom variables rgodic theory ad sums of radom variables Motivatio Measure-preservig trasformatios rgodic Theorems Sums of idepedet radom variables

3 1 Itroductio Motivatio from two perspectives: 1. Probability Probability space Ω, PΩ, P, where Ω is a set, PΩ the set of evets power set i this case ad P : PΩ [0, 1] is the probability measure. If Ω is coutable, for every A PΩ we have PA = ω A P {ω}. So calculatig probabilities just ivolves possibly ifiite sums. If Ω = 0, 1] ad P is the uiform probability measure o 0, 1] the for every ω Ω it is Pω = 0. So 1 = P 0, 1] P {ω}. w 0,1] 2. Itegratio Aalysis I geeral, for cotiuous Ω, the probability of a set A Ω ca be computed as PA = 1 A x ρx dx, Ω provided this Riema-itegral exists. Here 1 A is the idicator fuctio of the set A ad ρ is the probability desity. I the example above ρ 1 ad this leads to P a, b] = a,b]x dx = b a dx = b a. Usig stadard properties of Riema-itegrals this approach works fie, as log as A is a fiite uio or itersectio of itervals. But e.g. for A = 0, 1] Q, the Riema-itegral Ax dx is ot defied, although the probability for this evet is ituitively 0. Moreover, sice A is coutable, it ca be writte as A = { a : N }. Defie f := 1 {a1,...,a } with f 1 A for. For every, f is Riema-itegrable ad 1 0 f x dx = 0. So it should be the case that lim 1 0 f x dx = 0? = A x dx, but the latter itegral is ot defied. Thus the cocept of Riema-itegrals is ot satisfactory for two reasos: The set of Riema-itegrable fuctios is ot closed, ad there are too may fuctios which are ot Riema-itegrable. Goals of this course Geeralisatio of Riema-itegratio to Lebesgue usig measure theory Usig measure theory as the basis of advaced probability ad discussig applicatios i that area 3

4 Official schedule Measure spaces, σ-algebras, π-systems ad uiqueess of extesio, statemet * ad proof * of Carathéodory s extesio theorem. Costructio of Lebesgue measure o R. The Borel σ-algebra of R. xistece of o-measurable subsets of R. Lebesgue-Stieltjes measures ad probability distributio fuctios. Idepedece of evets, idepedece of σ-algebras. The Borel-Catelli lemmas. Kolmogorov s zero-oe law. [6] Measurable fuctios, radom variables, idepedece of radom variables. Costructio of the itegral, expectatio. Covergece i measure ad covergece almost everywhere. Fatou s lemma, mootoe ad domiated covergece, differetiatio uder the itegral sig. Discussio of product measure ad statemet of Fubii s theorem. [6] Chebyshev s iequality, tail estimates. Jese s iequality. Completeess of L p for 1 p. The Hölder ad Mikowski iequalities, uiform itegrability. [4] L 2 as a Hilbert space. Orthogoal projectio, relatio with elemetary coditioal probability. Variace ad covariace. Gaussia radom variables, the multivariate ormal distributio. [2] The strog law of large umbers, proof for idepedet radom variables with bouded fourth momets. Measure preservig trasformatios, Beroulli shifts. Statemets * ad proofs * of maximal ergodic theorem ad Birkhoff s almost everywhere ergodic theorem, proof of the strog law. [4] The Fourier trasform of a fiite measure, characteristic fuctios, uiqueess ad iversio. Weak covergece, statemet of Lévy s covergece theorem for characteristic fuctios. The cetral limit theorem. [2] Appropriate books P. Billigsley Probability ad Measure. Wiley 1995 hardback. R.M. Dudley Real Aalysis ad Probability. Cambridge Uiversity Press 2002 paperback. R.T. Durrett Probability: Theory ad xamples. Wadsworth ad Brooks/Cole 1991 hardback. D. Williams Probability with Martigales. Cambridge Uiversity Press paperback. From the poit of view of aalysis, the first chapters of this book might be iterestig: W. Rudi Real ad Complex Aalysis. McGraw-Hill paperback 4

5 2 Set systems ad measures 2.1 Set systems Let Ω be a arbitrary set ad A PΩ a set of subsets. Defiitio 2.1. Say that A is a rig, if for all A, B A i A ii B \ A A iii A B A. Say that A is a algebra or field, if for all A, B A i A ii A c = Ω \ A A iii A B A. Say that A is a σ-algebra or σ-field, if for all A ad A 1, A 2,... A i A ii A c A iii A A. N Remarks. i A σ-algebra is closed uder coutably fiitely may set operatios, sice A B = A c B c c c A, A = A c, 1 N N A \ B = A B c A, A B = A \ B B \ A A. ii Thus: A is a σ-algebra A is a algebra A is a rig I geeral the iverse statemetss are false, but i special cases they hold: if Ω is fiite if Ω A xamples. i PΩ ad {, Ω} are obviously σ-algebras { } ii Ω = R, R = i=1 a i, b i ] : a i < b i, i = 1,...,, N is the rig of half-ope itervals. R is a algebra if we allow for ifiite itervals. Lemma 2.1. Let { A i : i I } be a possibly ucoutable collectio of σ-algebras. The i I A i is a σ-algebra, whereas i I A i i geeral is ot. Proof. see example sheet 1 Defiitio 2.2. Let PΩ. The the σ-algebra geerated by is σ := A, the smallest σ-algebra cotaiig. A A σ alg. 1 as log as ii is fulfilled ad are equivalet 5

6 Remarks. i If A is a σ-algebra, the σa = A. ii Let 1, 2 PΩ with 1 2. The σ 1 σ 2. xample. Let A Ω. The σ {A} = {, Ω, A, A c }. The ext example is so importat, that we sped a extra defiitio. Defiitio 2.3. Let Ω, τ be a topological space with topology τ PΩ set of ope sets 2. The στ is called the Borel σ-algebra of Ω, deoted by BΩ. A BΩ is called a Borel set. Oe usually deotes BR = B. Lemma 2.2. Let Ω = R, R the rig of half-ope itervals ad I = { a, b] : a < b } the set of all half-ope itervals. The σr = σi = B. Proof. i I R σi σr. O the other had, each A R ca be writte as A = i=1 a i, b i ] σi. Thus R σi σr σi. ii ach A I ca be writte as A = a, b] = =1 a, b + 1 B σi B. Let A R be ope, i.e. x A ɛ x >0 : x ɛ x, x + ɛ x A. Thus x A a x, b x Q : {x} a x, b x ] A. The A = x A a x, b x ] which is a coutable uio, sice a x, b x Q. Thus A σi B I. { d } Aalogously, σr d is geerated by I d = a i, b i ] : a i < b i, 1 = 1,..., d. i=1 Defiitio 2.4. Let A PΩ be a σ-algebra. The pair Ω, A is a measurable space ad elemets of A are measurable sets. If Ω is fiite or coutably ifiite, oe usually takes A = PΩ as relevat σ-algebra. 2.2 Measures Defiitio 2.5. Let A be a rig o Ω. A set fuctio is ay fuctio µ : A [0, ] with µ = 0. µ is called additive if for all A, B A with A B = : µa B = µa + µb. µ is called coutably additive or σ-additive if for all sequeces A N A i A j = for i j ad A A: µ A = µa. N N N with 2 Havig a direct defiitio of ope sets for Ω = R d, there is also a axiomatic defiitio of a topology, amely i τ ad Ω τ ii A, B τ : A B τ iii i I Ai τ, give Ai τ for all i I 6

7 Note. µ coutably additive µ additive [ A1 = A, A 2 = B, A 3 = A 4 =... = ] Defiitio 2.6. Let Ω, A be a measurable space. A coutably additive set fuctio µ : A [0, ] is called a measure, the triple Ω, A, µ is called measure space. If µω <, µ is called fiite. If µω = 1, µ is a probability measure ad Ω, A, µ is a probability space. If Ω is a topological space ad A = BΩ, the µ is called Borel measure. Basic properties. Let Ω, A, µ be a measure space. i µ is o-decreasig: For all A, B A, A B it is µb = µb \ A + µa µa. Note: The versio µb \ A = µb µa oly makes sese if µa <. ii µ is subadditive: For all A, B A µa + µb = µa \ B + µa B + µb \ A +µa B = }{{} =µa B = µa B + µa B µa B. Agai: µa B = µa + µb µa B if µa B <. iii µ is also coutably subadditive see example sheet 1. Remark. These properties also hold o a rig, i ad ii also for additive set fuctios. xamples. i Discrete measure theory: Let Ω be coutable. Ay m : Ω [0, ] is called mass fuctio. There is a oe-to-oe correspodace betwee mass fuctios ad measures µ o Ω, PΩ, give by x Ω : µ {x} = mx ad A Ω : µa = x A µ {x} = x A mx. If µ {x} = 1 for all x Ω, µ is called coutig measure. ii Let Ω = R ad R be the rig of half-ope itervals. For A R write A = i=1 a i, b i ] with disjoit itervals ad set µa := i=1 b i a i. The represetatio of A is ot uique, but the defiitio of µ is idepedet of the particular choice. Further, µ is additive ad traslatio ivariat, i.e. x R : µa + x = µa, where A + x := i=1 a i + x, b i + x]. The key questio is: Ca µ be exteded to a measure o B? I order to attack this questio i the ext subsectio, it is useful to itroduce the followig property of set fuctios. 7

8 Defiitio 2.7. Let A be a rig o Ω ad µ : A [0, ] a additive set fuctio. µ is cotiuous at A A, if i µ is cotiuous from below: give ay A 1 A 2 A 3... i A with N A = A A A A, the lim µa = µa ii µ is cotiuous from above: give ay A 1 A 2 A 3... i A with N A = A A A A ad µa < for some N, the lim µa = µa Lemma 2.3. Let A be a rig o Ω ad µ : A [0, ] a additive set fuctio. The: i µ is coutably additive µ is cotiuous at all A A ii µ is cotiuous from below at all A A µ is coutably additive iii µ is cot. from above at ad µa < for all A A µ is coutably additive Proof. i Give A A i A, the A = A 1 \ A 0 A 2 \ A 1 A 3 \ A 2... A 0 = µa = m 1 µa +1 \ A = lim µ m =0 =0 A +1 \ A = lim m µa m. Give A A i A ad µa m < for some m N. Let B := A m \ A for m. The B A m \ A for ad thus, followig the above, µa m µa = µb µa m \ A = µa m µa. Sice µa m < this implies lim µa = µa. ii see example sheet 1 iii aalogous to i ad ii 2.3 xtesio ad uiqueess Theorem 2.4. Carathéodory s extesio theorem Let A be a rig o Ω ad µ : A [0, ] be a coutably additive set fuctio. The there exists a measure µ o Ω, σa such that µ A = µa for all A A. The proof is give at the ed of this sectio. To formulate a result o uiqueess two further otios are useful. Defiitio 2.8. Let Ω be a set. A PΩ os a π-system if i A, ii A, B A : A B A. 8

9 A is a d-system if i Ω A, ii A, B A, A B : B \ A A, iii A 1, A 2,... A : A 1 A 2... N A A. Remarks. i The set I = { a, b] a b } of half-ope itervals is a π-system o R. ii A is a σ-algebra A is a π- ad a d-system see example sheet. Lemma 2.5. Dyki s π-system lemma Let A be a π-system. The for ay d-system D A it is D σa. Theorem 2.6. Uiqueess of extesio Let A PΩ be a π-system. Suppose that µ 1, µ 2 are measures o σa with µ 1 Ω = µ 2 Ω <. If µ 1 = µ 2 o A the µ 1 = µ 2 o σa. Proof. Cosider D = { A σa : µ 1 A = µ 2 A }. By hypothesis, Ω D. For A, B D with A B we have µ i A + µ i B \ A = µ i B <, i = 1, 2, thus also B \ A D. If A D, N, with A A, the µ 1 A = lim µ 1A = lim µ 2A = µ 2 A, so A D. Thus D σa is a d-system cotaiig the π-system A, so D = σa by Dyki s lemma. These theorems provide geeral tools for the costructio ad characterisatio of measures. Before we apply them i a specific cotext i the ext subsectio we give the missig proofs. Proof of Carathéodory s extesio theorem. For ay B Ω, defie the outer measure µ B = if µa, where the ifimum is take over all sequeces A N i A such that B A ad is take to be if there is o such sequece. Note that µ is icreasig ad µ = 0. Let us say that A Ω is µ -measurable if, for all B Ω, µ B = µ B A + µ B A c. Write M for the set of all µ -measurable sets. We shall show that M is a σ-algebra cotaiig A ad that µ is a measure o M, extedig µ. This will prove the theorem. Step I. We show that µ is coutably subadditive. Suppose that B B. If µ B < for all, the, give ɛ > 0, there exist sequeces A m m N i A, with B m A m, µ B + ɛ/2 m µa m. 9

10 The B A m ad thus µ B m Hece, i ay case µ B µ B. µa m m µ B + ɛ. Step II. We show that µ exteds µ. Sice A is a rig ad µ is coutably additive, µ is coutably subadditive. Hece, for A A ad ay sequece A N i A with A A, we have µa µa. O takig the ifimum over all such sequeces, we see that µa µ A. O the other had, it is obvious that µ A µa for A A. Step III. We show that M cotais A. Let A A ad B Ω. We have to show that µ B = µ B A + µ B A c. By subadditivity of µ, it is eough to show that µ B µ B A + µ B A c. If µ B =, this is trivial, so let us assume that µ B <. The, give ɛ > 0, we ca fid a sequece A N i A such that B A, µ B + ɛ µa. The B A A A, B A c A A c, so that µ B A + µ B A c µa A + µa A c = µa µ B + ɛ. Sice ɛ > 0 was arbitrary, we are doe. Step IV. We show that M is a algebra. Clearly Ω M ad A c A wheever A A. Suppose that A 1, A 2 M ad B Ω. The µ B = µ B A 1 + µ B A c 1 Hece A 1 A 2 M. = µ B A 1 A 2 + µ B A 1 A c 2 + µ B A c 1 = µ B A 1 A 2 + µ B A 1 A 2 c A 1 + µ B A 1 A 2 c A c 1 = µ B A 1 A 2 + µ B A 1 A 2 c. Step V. We show that M is a σ-algebra ad that µ is a measure o M. We already kow that M is a algebra, so it suffices to show that, for ay sequece of disjoit sets A N i M, for A = A we have A M, µ A = µ A. So, take ay B Ω, the µ B = µ B A 1 + µ B A c 1 = µ B A 1 + µ B A 2 + µ B A c 1 A c 2 =... = µ B A i + µ B A c 1... A c. i=1 10

11 Note that µ B A c 1... Ac µ B A c for all. Hece, o lettig ad usig coutable subadditivity, we get µ B µ B A + µ B A c µ B A + µ B A c. =1 The reverse iequality holds by subadditivity, so we have equality. Hece A M ad, settig B = A, we get µ A = µ A. =1 Proof of Dyki s π-system lemma. Deote by D the itersectio of all d-systems cotaiig A. The D is itself a d-system. We shall show that D is also a π-system ad hece a σ-algebra, thus provig the lemma. Cosider D = { B D : B A D for all A A }. The A D because A is a π-system. Let us check that D is a d-system: clearly Ω D ; ext, suppose B 1, B 2 D with B 1 B 2, the for A A we have B 2 \ B 1 A = B 2 A \ B 1 A D, because D is a d-system, so B 2 \ B 1 D ; fially, if B D, N, ad B B, the for A A we have B A B A, so B A D ad B D. Hece D = D. Now cosider D = { B D : B A D for all A D }. The A D because D = D. We ca check that D is a d-system, just as we did for D. Hece D = D which shows that D is a π-system. 2.4 Lebesgue-Stieltjes measure Theorem 2.7. There exists a uique Borel measure µ o R, B such that µ a, b] = b a, for all a, b R with a < b. Proof. xistece Let R be the rig of fiite uios of disjoit itervals of the form A = a 1, b 1 ]... a, b ] ad cosider the set fuctio µa = i=1 b i a i give above. We aim to show that µ is coutably additive o A, which the proves existece by Carathéodory s extesio theorem. Sice µa < for all A R it suffices to show that µ is cotiuous from above by Lemma 2.3 iii. Suppose ot. The there exists ɛ > 0 ad A with µa 2ɛ for all. For each we ca fid C A with C B ad µa \ C ɛ2. The µc 1... C = µa µa \ C 1... µa \ C µa µa 1 \ C 1... µa \ C 2ɛ N ɛ2 = ɛ, 11

12 ad i particular C 1... C. Thus K = C 1... C, N is a mootoe sequece of compact o-empty sets, so N K N A which is a cotradictio to A. Uiqueess For each cosider µ A := µ, + 1] A. The µ is a probability measure o R, B, so, by Theorem 2.6, µ is uiquely determided by its values o the π-system I geeratig B. Sice µa = N µ A it follows that µ is also uiquely determied. Defiitio 2.9. A R is called ull if A B B with µb = 0. Deote by N the set of all ull sets. The L = { B N : b B, N N } is the Lebesgue σ-algebra. The sets i L are called Lebesgue-measurable sets or Lebesgue sets. The Borel measure µ ca be exteded to L via λb N := µb for all B B, N N. λ is called Lebesgue measure. see example sheet 1.7 Theorem 2.8. B L PR. Proof. i We argue that B ad L have differet cardiality: Accordig to example sheet 1.4, B is separable ad thus cardb = c. 3 With example sheet 1.10 we have for the Cator set C: cardc = c ad µc = 0. Sice with Defiitio 2.9 PC L we have cardl = 2 c > c. ii Usig the axiom of choice we costruct a subset of U = [0, 1] which is ot L.-measurable: Defie the equivalece relatio o U by x y if x y Q. Write = { i : i I} for the equivalece classes of ad let R be a collectio of represetatives of the i, chose by the axiom of choice. The U ca be partitioed U = i = R + q = {r + q : r R}, i I q Q [0,1 q Q [0,1 where + is to be uderstood modulo 1. Suppose R L, the R + q L ad λr = λr + q for all q Q by traslatio ivariace of λ. But by coutable additivity of λ this meas λr = λu = 1, q Q [0,1 which leads to a cotradictio for either λr = 0 or λr > 0. Thus R L. Remarks. i very set of positive measure has o-measurable subsets ad, moreover: every subset of A is Lebesgue-measurable λa = 0. ii There exists o traslatio ivariat, coutably additive set fuctio o PR with µa 0, for at least oe A R. 3 Notatio: The cardiality of a coutable set is cardn = ℵ 0, of a cotiuous set cardr = c ad of the power set card PR = 2 c. 12

13 Defiitio F : R R is called distributio fuctio if i F is o-decreasig F is a probability distributio fuctio if i additio ii F is right-cotiuous, i.e. lim x x 0 F x = F x 0. iii lim F x = 1, x lim F x = 0. x Propositio 2.9. Let µ be a Rado measure 4 o R, B. The { µ 0, x], x > 0 F x := µ x, 0], x 0 is a distr. fct. with µ a, b] = F b F a, a < b. A distributio fuctio with this property is uiquely determied up to a additive costat. Proof. F ad µ a, b] = F b F a for b > a by defiitio. For x x > 0 F x = µ 0, x ] µ 0, x] = F x by cotiuity of measures. Let F ad G be two such distributio fuctios. The for all a < b µ = µ a, b] µ a, b] = F b F a Gb + Ga = 0, ad F = G + c for some costat c R. Remarks. i The distributio fuctios for the Lebesgue measure are F x = x + c, c R. ii Shiftig the referece poit r i the defiitio of F, which is take to r = 0 i Propositio 2.9, adds a costat to F. iii If µ is a fiite measure, e.g. i probability, oe usually uses the cumulative distributio fuctio with r =, F x = µ, x]. O the other had, Rado measures are uiquely determied by their distributio fuctio. Theorem Let F : R R be a distributio fuctio. The there exists a uique Rado measure µ F o R, B, such that µ F a, b] = F b F a for all a < b. The completio λ F of this measure, defied aalogously to λ o a exteded σ-algebra L F, is called the Lebesgue-Stieltjes measure of F. Proof. Same as for Lebesgue measure. But ote that the ull sets for µ F might be differet, so that L F is i geeral a differet σ-algebra tha L. 4 i.e. µk < for K B compact 13

14 xamples of distributio fuctios. i If a probability distributio fuctio F is differetiable o R, the f = F is called the probability desity fuctio, ad for A L F it is λ F A = A dλ F x = A fx dx. For istace, fx = 1 for the Lebesgue measure, ad fx = e x2 /2 / 2π for the Gaussia ormal distributio. ii Let = {e 1, e 2,...} R be coutable ad p : [0, ] a mass fuctio with correspodig measure µb = e i B pe i defied for all B PR. The F x = e i x pe i is a distributio fuctio for the measure µ, ad the cotiuatio px = { px, x 0, x is a discrete aalogo of the desity fuctio. iii Let F be the uiform probability distributio fuctio o the Cator set C see example sheet The F is cotiuous o R, but differetiable oly o R\C with F x = 0, so there exists o probability desity. 2.5 Idepedece ad Borel-Catelli lemmas Let Ω, A, P be a probability space. It provides a model for a experimet whose outcome is radom. Ω describes the set of possible outcomes, A the set of evets observable sets of outcomes ad PA is the probability of a evet A A. Defiitio The evets A i i I, A i A, are said to be idepedet if P A i = PA i for all fiite J I. i J i J The σ-algebras A i i I, A i A are said to be idepedet if the evets A i i I are idepedet for ay choice A i A i. A useful way to establish idepedece of two σ-algebra is give below. Theorem Let A 1, A 2 A be π-systems ad suppose that PA 1 A 2 = PA 1 PA 2 wheever A 1 A 1, A 2 A 2. The σa 1 ad σa 2 are idepedet. Proof. Fix A 1 A 1 ad defie for A A µa := PA 1 A, νa := PA 1 PA. 14

15 µ ad ν agree o the π-system A 2 with µω = νω = PA 1 <. So, by uiqueess of extesio Theorem 2.6, for all A 1 A 1 ad A 2 σa 2 PA 1 A 2 = µa 2 = νa 2 = PA 1 PA 2. Now fix A 2 σa 2 ad repeat the same argumet with µ A := PA A 2, ν A := PA PA 2 to show that for all A 1 σa 1 we have PA 1 A 2 = PA 1 PA 2. Remark. I particular, the σ-algebras σ {A 1 } ad σ {A 2 } geerated by sigle evets are idepedet if A 1 ad A 2 are idepedet. Defiitio Let A N be a sequece of sets i Ω. The lim if A := { ω Ω : ω A for all but fiitely may } = A k = A ev., N k lim sup A := { ω Ω : ω A for ifiitely may } = A k = A i.o.. N Remarks. i If A A for all N the lim if A, lim sup A A. ii lim if A lim sup A, sice for all m, : k m A k A m k A k. iii lim sup A c = lim if A c k Lemma First Borel-Catelli lemma If PA <, the PA i.o. = 0. N Proof. PA i.o. = P N k A k P k This argumet is also valid if P is ot a probability measure. A k PA k 0 for. k Lemma Secod Borel-Catelli lemma Suppose that A N are idepedet. If PA =, the PA i.o. = 1. N Proof. We use the iequality 1 a e a. Set a = PA. The we have for all N P A c k = [ a k exp k k 1 ] a k = 0. k Hece PA i.o. = 1 Plim if A c = 1 P = 1. N k A c k 15

16 3 Measurable Fuctios ad Radom Variables 3.1 Measurable Fuctios Defiitio 3.1. Let, ad F, F be measurable spaces. A fuctio f : F is called measurable with respect to ad F or /F-measurable if A F : f 1 A = { x : fx A }. Remarks. i If = P every fuctio f : F is measurable w.r.t. ad F. ii Usually F, F = R, B or R, B with the exteded real lie R = R {, } ad B = { B C : b B, C {, } }. If i additio is a topological space with = B, f is called Borel fuctio. iii Preimages of fuctios preserve the set operatios f 1 A = f 1 A, f 1 A c = f 1 A c. N N Hece for ay f : F : If is a σ-algebra o the { A F : f 1 A } is a σ-algebra o F. If F is a σ-algebra o F the σf := { f 1 A : A F } is a σ-algebra o, called σ-algebra geerated by f. This is the smallest σ-algebra o w.r.t. which f is measurable. Lemma 3.1. Let f : F ad F = σa for some A PF. If f 1 A for all A A the f is measurable w.r.t. ad F. Proof. Cosider C := { A F : f 1 A }. C is a σ-algebra o F ad A C by hypethesis. Thus σa = F C ad f is measurable. Lemma 3.2. f : R is /B-measurable if oe of the followig holds: i f 1, c] = { x : fx c } for all c R, ii f 1, c = { x : fx < c } for all c R, iii f 1 [c, = { x : fx c } for all c R, iv f 1 c, = { x : fx > c } for all c R. Proof. i O example sheet 1.3 it was show that B = σ {, c] : c R}. The statemet the follows with Lemma 3.1. ii iv Show amalogously that B is geerated by the respective sets. Lemma 3.3. Let ad F be topological spaces ad f : F be cotiuous i.e. f 1 U ope wheever U F ope. The f is measurable w.r.t. B ad BF. 16

17 Proof. Let A = { U F : U ope } ad remember that BF = σa. The for all U A, f 1 U is ope ad thus f 1 U B. Sice BF = σa, f is measurable with Lemma 3.1. Lemma 3.4. Let f 1 : 1 2 ad f 2 : 2 3. If f 1 is 1 / 2 -measurable ad f 2 is 2 / 3 -measurable, the f 2 f 1 is 1 / 3 -measurable. Proof. For every A 3, f 2 f 1 1 A = f1 1 f 1 2 A 1 sice f2 1 A 2. Propositio 3.5. Let f : R, N be /B-measurable. The so are i c f 1 for all c R ii f 1 + f 2 iii f 1 f 2 iv if N f Remarks. i Notatio: v sup f vi lim if f vii lim sup f. N if f { x := if f x : N } R... N N ii I particular f 1 f 2 = max{f 1, f 2 } ad f 1 f 2 = mi{f 1, f 2 } are measurable. iii f measurable f + = f 0 ad f = f 0 measurable f measurable Proof. i If c 0 we have for all y R { x : c fx y } = { x : fx y/c }. If c = 0 it is { x : 0 y } {, y 0 =, y < 0. ii see example sheet iii f 1 f 2 = 4 1 f1 + f 2 2 f 1 f 2 2 is measurable with i, ii ad Lemma 3.4, sice g : R R, gx = x 2 is cotiuous ad thus measurable. iv vii see example sheet Defiitio 3.2. Let 1 A : R be the idicator fuctio of A. f : R is called simple if f = c i 1 Ai for some N, c i R ad A 1,..., A. i=1 Remark. The above represetatio is ot uique. I the followig we are always goig to use a ice represetatio with c i c j ad A i A j = if i j. Lemma 3.6. Let, be a measurable space. i A simple fuctio f : R with ice represetatio f = i=1 c i 1 Ai is /B-measurable if ad oly if A i for all i = 1,...,. Such a fuctio is called -simple fuctio. 17

18 ii Let f 1, f 2 be -simple fuctios. The λ 1 f 1 + λ 2 f 2, for all λ 1, λ 2 R, ad f 1 f 2 are -simple fuctios. So the set of -simple fuctios is a vector space. Proof. i Let A i for i = 1,...,. The for all B B : f 1 B = A i. i:c i B O the other had, if A i A i for some i the f 1 {c i } = A i ad f is ot measurable. ii Let f 1 = i=1 c i 1 Ai ad f 2 = i=1 d i 1 Bi. Defie C ij = A i B j. The the {C ij } are disjoit ad each C ij as well as uios of C ij s. f 1 f 2 ad λ 1 f 1 + λ 2 f 2 are costat o each C ij ad thus -simple. Remark. I particular, for A, 1 A ad f 1 A f measurable are measurable. Defiitio 3.3. Let, ad G, G be measurable spaces ad let µ be a measure o,. The ay measurable fuctio f : G iduces the image measure ν := µ f 1 o G, give by νa := µ f 1 A. Remark. ν is a measure sice f 1 A is measurable for all A G ad f 1 preserves set operatios, i.e. f 1 A B = f 1 A f 1 B see above. 3.2 Radom Variables Let Ω, A, P be a probability space ad, a measurable space. Defiitio 3.4. A measurable fuctio X : Ω is called radom variable i or simply radom variable if = R. The image measure µ X = P X 1 o, is called law or distributio of X. For = R the cumulative probability distributio fuctio of µ X, F X : R [0, 1] with F X x = µ X, x] = P {ω Ω : Xω x} = PX x is called distributio fuctio of X. Remark. By Theorem 2.10, µ X is uiquly determied by F X. Usually radom variables are characterised by their probability distributio F X without specifyig Ω, A, P ad X. Theorem 3.7. For every probability distributio fuctio F there exists a probability space Ω, A, P ad a radom variable X : Ω R such that F is the distributio fuctio of X. Proof. see example sheet. Defiitio 3.5. The radom variables X N i are called idepedet if the σ-algebras σx = { X 1 A : A } are idepedet. Lemma 3.8. Real radom variables X N are idepedet if ad oly if P X 1 x 1,..., X k x k = P X1 x 1 P Xk x k 18

19 for all x 1,..., x k R, k N. Proof. see example sheet Remark. X 0 is ofte regarded as a stochastic process with discrete time. The σ- algebra geerated by X 0,..., X, σx 0,..., X = σ {X 1 i A : A, i = 0,..., }, cotais evets depedig measurably o X 0,..., X ad represets what is kow about the process by time. Defiitio 3.6. Let X N be a sequece of radom variables. Defie T := σx +1, X +2,..., T := N T. The T is called the tail σ-algebra of X N ad elemets i T are called tail evets. xamples. { ω Ω : lim N X ω exists } or { ω Ω : lim sup X ω = }. Theorem 3.9. Kolmogorov s 0-1-law Suppose X N is a sequece of idepedet radom variables. The every tail evet has probability 0 or 1. Moreover, ay T -measurable radom variable Y is almost surely costat, i.e. PY = c = 1 for some c R. Proof. The σ-algebra F = σx 1,..., X is geerated by the π-system of evets A = { X 1 x 1,..., X x }, whereas T is geerated by the π-system of evets B = { X +1 x +1,..., X +k x +k }, k N. Sice PA B = PA PB for all such A ad B by idepedece, F ad T are idepedet by Theorem 2.11 for all N. Hece F ad T are idepedet, sice T T +1. Sice F is a π-system geeratig the σ-algebra F = σx : N, F ad T are idepedet, agai by Theorem But T F ad thus every A T is idepedet of itself, i.e. PA = PA A = PA PA PA {0, 1}. Let Y be a T -measurable radom variable. The F Y y = PY y takes values i {0, 1}, so PY = c = 1 for c = if{y R : F Y y = 1}. Remark. Kolmogorov s 0-1-law ivolves the σ-algebras geerated by radom variables, rather tha the radom variables themselves. Thus it ca be formulated without usig r.v. s: Let F N be a sequece of idepedet σ-algebras i A.Let A be a tail evet, i.e. A T, where T = T with T = σ F m. N The PA = 0 or PA = 1. m 19

20 3.3 Covergece of measurable fuctios Let Ω, A, µ be a measure space. Defiitio 3.7. We say that A A holds almost everywhere or a.e., if µa c = 0. If µ is a probability measure oe ofte uses almost surely or a.s. istead. Let f 1, f 2,... : Ω R be a sequece of measurable fuctios. Say that i f f everywhere or poitwise if f ω fω for all ω Ω, ii f f almost everywhere a.e. if µ {ω Ω : f ω fω} = 0, agai, if µ is a prob. meas. oe uses almost surely or a.s. istead iii f f i measure or i probability i case µ is a prob. meas. if ɛ > 0 : µ f f > ɛ = µ {ω Ω : f ω fω > ɛ} 0 for. Theorem Let f N be a sequece of measurable fuctios. i Assume that µω <. If f f a.e. the f f i measure. ii If f f i measure the f k f a.e. for some subsequece k k N. Proof. i Set g = f f ad suppose g 0 a.e.. The for every ɛ > 0 µ g ɛ µ { g m ɛ} µ g ɛ ev. µg 0 = µω. m Hece µ g > ɛ 0 as ad f f i measure. ii Suppose g = f f 0 i measure. Thus µ g > 1/k 0 for every k N ad we ca fid a subsequece k k N such that µ g k > 1/k < 2 k ad thus µ g k > 1/k <. So, by the first Borel-Catelli lemma Lemma 2.12 µ g k > 1/k i.o. = 0. The complemet of this evet is cotaied i g k 0, ad thus k N µ g k 0 µ g k > 1/k i.o. = 0, so g k 0 a.e.. xample. Let X 1, X 2,... {0, 1} be idepedet radom variables with PX = 0 = 1 1/ ad PX = 1 = 1/. The ɛ > 0 : P X > ɛ = 1/ 0 as, i.e. X 0 i measure. O the other had, PX = 1 = ad {X = 1} are idepedet evets. Thus with the secod Borel-Catelli lemma PX 0 PX = 1 i.o. = 1, ad thus X 0 a.s.. 20

21 4 Itegratio 4.1 Defiitio ad basic properties Let,, µ be a measure space. Theorem 4.1. Let f : [0, ] be /B -measurable. The there exists a sequece f N of -simple, o-egative fuctios, such that f f poitwise as. { fx 2 Proof. Defie f x := /2, 0 fx <, fx = 2 1 Ak, x 2 k k=0 where A k, = f 1 [ 2 k, 2 k + 1, k = 0,..., 2 1, ad A 2, = f 1 [, ]. Sice f is measurable, so are the sets A k,, ad thus f is -simple for all N. From the first represetatio it follows immediately that f +1 x f x for all x ad that f x fx 2 for fx, or fx for fx =. Thus f f. This motivates the followig defiitio. Defiitio 4.1. Let f : [0, ] be a /B -measurable fuctio. We defie the itegral of f, writte as µf = f dµ = f dµ = fx µdx, by { } f dµ := sup g dµ : g -simple, 0 g f The itegral of a -simple fuctio g : R with ice represetatio gx = is defied as g dµ := c k µa k where we adopt 0 = 0 = 0. k=1 Remarks. i g dµ is idepedet of the represetatio of the -simple fuctio g. ii If f, g : R are -simple the: f g f dµ g dµ ad c1 f + c 2 g dµ = c 1 f dµ + c 2. g dµ for all c 1, c 2 R. c k 1 Ak Lemma 4.2. Let f : [0, ] be measurable ad f N a sequece of -simple fuctios o with 0 f f. The f dµ f dµ. Proof. f dµ f dµ for all N by defiitio of f dµ. It remais to show that for ay -simple fuctio g = k=1 a k1 Ak f with ice represetatio ad a k 0 f dµ g dµ. lim 21 k=1

22 Choose ɛ > 0 ad set B := { x : f x+ɛ gx }. Thus B ad for ay A : µb A µa. Case i: g dµ = µa r = for some r {1,..., }. The f dµ f 1 B Ar dµ g ɛ 1 B Ar dµ = a r ɛ µb A r as, provided ɛ < a r. Case ii: g dµ < for A = k=1 A k it is µa <. The f dµ f 1 B A dµ g ɛ 1 B A dµ = = g 1 B A dµ ɛ 1 B A g dµ ɛ µa as. This is true for ɛ arbitrarily small ad thus lim f dµ g dµ. Defiitio 4.2. Let f : R be a /B -measurable fuctio. f is called itegrable if f + dµ < ad f dµ < 5 ad the itegral is defied as f dµ = f + dµ f dµ. For radom variables X : Ω R the itegral X dp = X is also called expectatio. Ω Remarks. i The itegral remais well defied if either f + dµ = or f dµ =, possibly takig the values ±. ii For A ad f itegrable, f 1 A is itegrable ad we write Theorem 4.3. Basic properties of itegratio Let f, g : R be itegrable fuctios o,, µ. i Liearity: f + g ad, for ay c R, c f are itegrable with f + g dµ = ii Mootoicity: f g iii f 0 ad Let f : R be measurable. The f dµ + g dµ, f dµ g dµ. f dµ = 0 f = 0 a.e. iv f itegrable f itegrable, ad i this case 5 with Propositio 3.5 f + ad f are measurable A f dµ := c f dµ = c f dµ. f dµ = 0. f dµ f dµ. f 1 A dµ. 22

23 Proof. i If f, g 0 choose sequeces of -simple fuctios with 0 f f ad 0 g g. The f + g is -simple for all N ad f + g dµ = f dµ + g dµ. Sice 0 f + g f + g it follows by Lemma 4.2: f + g dµ = f dµ + For f, g : R we have f + g + f + g = f + f + g + g ad thus f + g + + f + g = f + g + f + + g +. Sice each of the terms is o-egative we also have f + g + dµ + f dµ + g dµ = g dµ. f + g dµ + f + dµ + g + dµ ad the statemet follows by reorderig the terms. For c f, c 0, aalogously, ad f dµ = f dµ f + dµ = f dµ. ii With f g usig i: f dµ = f g dµ + g dµ g dµ. }{{} 0 iii Let f 0 ad suppose that µf > 0 > 0. The, sice {f > 0} = m N {f 1/m}, we have µf 1/ = ɛ > 0 for some N. Thus f 1 1 f 1/ ad f dµ ɛ/ > 0. O the other had let f = 0 a.e. f +, f = 0 a.e. f + dµ = f dµ = 0 by defiitio. { f iv Follows with f = f + + f + f = f f f + ad i, ii. = f 4.2 Itegrals ad limits Let,, µ be a measure space. We are iterested uder which coditios o f we have f dµ lim f dµ. Theorem 4.4. Mootoe covergece Let f N, f : [0, ], be a sequece of measurable fuctios with f f poitwise. The f 0 is measurable ad f dµ f dµ. Proof. f is measurable by Propositio 3.5 sice f = sup N f. For each N let f k k N be a sequece of -simple fuctios with 0 f k f as k ad let g := max { f1,..., f }. The g is a icreasig sequece of -simple fuctios with fr g f for each r, N. Takig the limit we get f r g f for each r N with g = lim g. 23

24 Takig the limit r gives g = f. Hece fr dµ : r : f dµ = lim g dµ by Lemma 4.2. But g dµ f dµ for each r, N ad so with f r dµ f dµ lim f dµ lim f r dµ f dµ lim f dµ. r Remark. [ f 0 a.e. ad f f a.e. is sufficiet for Theorem 4.4 to hold. N = {f < 0} {f > f +1 }, µn = 0. Use mootoe covergece o N c. ] Lemma 4.5. Fatou s lemma Let f N, f : [0, ], be a sequece of measurable fuctios. The lim if f dµ lim if f dµ. Proof. Let g := if f k. The the g are measurable by Propositio 3.5 ad g lim if f. k So sice f g ad by mootoe covergece: f dµ g dµ lim if f dµ which proves the statemet, takig lim if o the left-had side. Theorem 4.6. Domiated covergece Let f N, f : R, be a sequece of measurable fuctios with f f poitwise. Suppose that for some itegrable g : [0, ], f g poitwise for all N. The f ad the f are itegrable ad f dµ f dµ. Proof. f is measurable with Propositio 3.5 ad f, f are itegrable sice with f, f g, f dµ, f dµ g dµ <. We have 0 g ± f g ± f, so lim if g ± f = g ± f. By Fatou s lemma, g dµ+ f dµ = lim if g+f dµ lim if g dµ f dµ = lim if g f dµ lim if g+f dµ = g dµ+ lim if f dµ, g f dµ = g dµ lim sup f dµ. Sice g dµ < it follows that f dµ lim if f dµ lim sup f dµ f dµ, provig that f dµ f dµ as. Remark. f f a.e. ad f g a.e. is sufficiet for Theorem 4.6 to hold. 24

25 Corollary 4.7. Bouded covergece Let µ < ad f N, f : R, be a sequece of measurable fuctios with f f, satisfyig f C for some C R ad all N. The f ad the f are itegrable ad f dµ f dµ. Proof. Apply domiated covergece with g C, otig that g dµ = C µ <. The followig example shows that the iequality i Lemma 4.5 ca be strict ad that domiatio by a itegrable fuctio i Theorem 4.6 is crucial. xample. O R, L, λ take f = 2 1 0,1/. The f f 0 poitwise, but f dλ = 0 < f dλ =. Remarks. quivalet series versios of Theorems 4.4 ad 4.6: i Let f N, f : [0, ] measurable. The =1 f dµ = f dµ. ii Let f N, f : R measurable. If f coverges ad f k g, where g =1 k=1 is itegrable, the f, f are itegrable ad f dµ = f dµ. =1 4.3 Itegratio ad differetiatio =1 Theorem 4.8. Differetiatio uder the itegral sig Let U R be ope ad suppose that f : U R satisfies: i x ft, x is itegrable for all t U, ii t fx, t is differetiable for all x, iii f t t, x gx for some itegrable g : R ad all x, t U. t,. is itegrable for all t, the fuctio F : U R defied by F d is differetiable ad dt F t = f t, x µdx. t The f t Proof. Take a sequece h 0 ad set g t, x := ft + h, x ft, x h f t, x. t =1 =1 t = ft, x µdx The g x 0 for all x ad g t, x 2 gx for all N, t U by the MVT. f t t,. is measurable as the limit of measurable fuctios, ad itegrable sice f t g. The by domiated covergece F t + h F t f t, x µdx = g x µdx 0. h t 25

26 Remarks. i For the Lebesgue itegral we write f 1 a,b] dλ = fx dx = R a,b] ii Liearity of the itegral the implies: c R, usig the covetio a b b a fx dx = Theorem 4.9. Fudametal theorem of calculus fx dx = b a c a fx dx. i Let f : [a, b] R be a cotiuous fuctio ad set F a t = The F a is differetiable o [a, b] with F a = f. fx dx + t a b c fx dx. ii Let F : [a, b] R be differetiable with cotiuous derivative f. The b a fx dx = F b F a. b a fx dx. fx dx for all Proof. i Fix t [a, b. ɛ>0 δ>0 : x y <δ fx ft <ɛ. So for 0<h δ, F at + h F a t h ft = 1 h t+h Aalogous for egative h ad t a, b], thus F a = f. ii F F a t = 0 for all t a, b so by the MVT F b F a = F a b F a a = t b a fx ft dx ɛ h fx dx. t+h t dx = ɛ. So to the calculatio of Lebesgue itegrals the methods of elemetary calculus apply. Chage of variable ad partial itegratio are o the example sheet problems 2.11 ad Product measure ad Fubii s theorem Let 1, 1, µ 1 ad 2, 2, µ 2 be fiite measure spaces ad = 1 2. Defiitio 4.3. The product σ-algebra = 1 2 := σa is geerated by the π-system A = { } A 1 A 2 : A 1 1, A 2 2. xample. If 1 = 2 = R ad 1 = 2 = B the 1 2 = BR 2. Lemma Let f : R be -measurable. The the followig holds: i fx 1,. : 2 R is 2 -measurable for all x 1 1. ii If f is bouded, f 1 x 1 := fx 1, x 2 µ 2 dx 2 is bouded ad 1 -measurable. 2 26

27 Proof. i For fixed x 1 1 defie T x1 { : 2 by T x1 x 2 = x 1, x 2. For A = A 1 A 2 A, Tx 1 A2, x 1 A = 1 A 1, x 1 A 2 ad thus with Lemma T x1 is 2 /-measurable. So fx 1,. = ft x1. is 2 /B-measurable with Lemma 3.4. ii f 1 is well defied by i ad bouded sice f is bouded ad µ 2 2 <. For f = 1 A, f 1 x 1 = µ 2 T 1 x 1 A. Deote D = { A : f 1 is measurable }, which ca be checked to be a d-system. Sice f 1 x 1 = 1 A1 x 1 µ 2 A 2 for A = A 1 A 2, A D ad thus = σa = D with Dyki s lemma 2.5. By liearity of itegratio the statemet also holds for o-egative -simple fuctios, ad by mootoe covergece for all measurable f usig f 1 x 1 := f + x 1, x 2 µ 2 dx 2 2 f x 1, x 2 µ 2 dx 2. 2 Theorem Product measure There exists a uique measure µ = µ 1 µ 2 o, such that µa 1 A 2 = µ 1 A 1 µ 2 A 2 for all A 1 1 ad A 2 2, defied as µa := 1 A x 1, x 2 µ 2 dx 2 µ 1 dx Proof. With Lemma 4.10, µ is a well defied fuctio of A. Usig mootoe covergece µ ca be see to be coutably additive ad is thus a measure. Sice 1 A1 A 2 = 1 A1 1 A2 the above property is fulfilled for all A 1 1 ad A 2 2. Sice A i Defiitio 4.3 is a π-system geeratig ad µ <, µ is uiquely determied by its values o A followig Theorem 2.6 Uiqueess of extesio. Remark. f : 1 2 R is measurable if ad oly if ˆf : 2 1 R with ˆfx 2, x 1 = fx 1, x 2 is measurable ad for itegrable f: ˆf dµ2 µ 1 = f dµ 1 µ Theorem Fubii s theorem i Let f : [0, ] be -measurable. The f dµ = fx 1, x 2 µ 2 dx 2 µ 1 dx ii Let f : R be µ-itegrable. The fx 1,. is µ 2 -itegrable for µ 1 -almost all x 1. f., x 2 µ 2 dx 2 is µ 1 -itegrable ad the formula i i holds. 2 Proof. i If f = 1 A for some A the formula holds by defiitio of µ ad ca be exteded to o-egative measurable f as i the proof of Lemma 4.10 ii. ii If f : R is µ-itegrable, the by i fx 1, x 2 µ 2 dx 2 µ 1 dx 1 = f dµ <. 2 1 The result follows aalogous to i usig domiated covergece. 27

28 Remarks. i Product measures ad Fubii ca be exteded to σ-fiite measure spaces. ii The operatio of takig products of measure spaces is associative := = also for measures. So products ca be take without specifyig the order, e.g. R d, BR d, µ d. xample. I = R e x2 dx = π, I 2 = e x2 +y2 dx dy = R 2 sice by Fubii s theorem ad polar-doordiates 2π r=0 φ=0 e r2 r dr dφ = 2π [ e r2 /2 ] 0 = π. 28

29 5 L p -spaces 5.1 Norms ad iequalities Let,, µ be a measure space. Theorem 5.1. Chebyshev s iequality Let f : [0, ] be measurable. The for ay λ 0: Proof. Itegrate λ 1 {f λ} f. λ µf λ f dµ. Remark. For g L p ad λ > 0 we have µ g λ = µ g p λ p λ p g p dµ <, leadig to the tail estimate µ g λ = Oλ p as λ. xample. Let X be a r.v. with m = X <. The P X m λ VarX/λ 2. Defiitio 5.1. For 1 p we deote by L p = L p,, µ the set of measurable fuctios f : R with fiite L p -orm: 1/p f p = f dµ p for p <, f = if { λ R : f λ a.e. }. We say that f coverges to f i L p if f f p 0 as. Remark. For 1 p < : f p µ 1/p f. Defiitio 5.2. A fuctio f : R R is covex if, for all x, y R ad t [0, 1] f t x + 1 t y t fx + 1 t fy. Remark. Let f : R R be covex. The f is cotiuous i particular measurable ad x 0 R a R : fx a x x 0 + fx 0. Theorem 5.2. Jese s iequality Let X be a itegrable r.v. ad f : R R a covex. The fx f X. Proof. With m = X choose a R such that fx a X m + fm. I particular fx a X + fm < ad fx R is well defied. Moreover fx a X m + fm = fm = f X. Corollary 5.3. Mootoicity of L p -orms Let 1 p < q <. The for ay radom variable X L p P: X p X q, i particular L p P L q P. 29

30 Proof. Set fx = x q/p, the f is covex. Thus with Jese s iequality X p = X p 1/p = f X p 1/q f X p 1/q = X q 1/q = X q. Theorem 5.4. Hölder s iequality Let p, q [1, ] be cojugate idices, i.e. 1 p + 1 q f g dµ = f g 1 f p g q. = 1. The for all measurable f, g : R Proof. For p = 1, q = the result follows easily with f g f g a.e.. If f p, g q = 0 or the result is trivial, so i the followig p, q 1, ad f p, f g q 0,. For give 0 < a, b < let a = e s/p, b = e t/q ad by covexity of e x we get e s/p+t/q es p + et q ad thus a b ap p + bq q. Now isert a = f / f p ad b = g / g q ad itegrate f p dµ g q dµ f g 1 f p g q p f p + p q g q = f p g 1 q p + 1 q = f p g q. q Theorem 5.5. Mikowski s iequality For p [1, ] ad measurable f, g : R we have f + g p f p + g p. Proof. The cases p = 1, are easy, so assume p 1, ad f p, g p <, f +g p > 0. The f + g p 2 f g p 2 p f p + g p so f + g p <. The f + g p dµ f f + g p 1 dµ + g f + g p 1 dµ f f p + g p + g p 1 q with q = p p 1 usig Hölder. But sice p = q p 1 we have f + g p 1 q = ad the result follows, isertig i the above iequality. f + g p dµ 1 1/p, 5.2 Completeess of L p Defiitio 5.3. Let V be a vector space.. : V [0, is a orm if f.a. v, u V, λ R i u + v u + v ii λ v = λ v iii v = 0 v = 0. A symmetric biliear map.,. : V V R is a ier product if v, v 0 with equality if ad oly if v = 0. A pair V,. is called ormed vector space, ad V,.,. ier product space. 30

31 Remarks. i very ier product.,. iduces a orm.,. by Cauchy-Schwarz ieq. 6 ii By Mikowski s iequality each L p is a vector space ad i, ii hold. iii For 1 p : f p = 0 f = 0 a.e. by Theorem 4.3. For f, g L p we say that g is a versio of f if g = f a.e.. So iii oly holds if we idetify each f L p with all its versios. 7 iv L 2 is also a ier product space with f, g = f g dµ, sice f 2 = f, f. Defiitio 5.4. A ormed vector space is complete if every Cauchy sequece i V coverges, v N V with v v m,m 0 v V : v v 0 as. A complete ormed vector space is called a Baach space ad a ier product space which is complete w.r.t. the iduced orm is called a Hilbert space. Remark. If v v 0 the also v v, by the triagle iequality. Theorem 5.6. Completeess of L p L p is complete for every p [1, ], i.e. L p is a Baach space ad L 2 is a Hilbert space. Proof. The case p = is left as a exercise, i the followig p <. Let f N be a sequece i L p such that f f m p 0 as, m. Choose a subsequece k k N such that S := f k+1 f k p <. By Mikowski s iequality, for ay K N, k=1 K f k+1 f k S. p k=1 By mootoe covergece this boud holds also for K =, so f k+1 f k < a.e. So for a.e. x R, f k x is Cauchy ad thus coverges by completeess of R. We defie { limk f fx := k, if the limit exists. 0, otherwise Give ɛ > 0, we ca fid N N such that f f m p dµ < ɛ for all m N, ad i particular f f k p dµ < ɛ for sufficietly large k. Hece by Fatou s Lemma f f p dµ = lim if f f k p dµ lim if k k f f k p dµ < ɛ for all N. Hece f L p ad f f p 0 sice ɛ > 0 was arbitrary. This result is very importat, sice Baach ad Hilbert spaces have may useful properties, as ca be see i the ext subsectio. k=1 6 u, v V : u, v 2 u 2 v 2. 7 Formally, the L p -orm is the defied o the quotiet space L p = L p /Z, where Z L p deotes the subspace of all fuctios vaishig almost everywhere. 31

32 5.3 L 2 as a Hilbert space Let,, µ be a measure space ad cosider the Hilbert space L 2 = L 2,, µ with ier product f, g = f g dµ ad orm f = f, f. Propositio 5.7. For f, g L 2 we have Pythagoras rule f + g 2 2 = f f, g + g 2 2, ad the parallelogram law f + g f g 2 2 = 2 f g 2 2. Proof. see example sheet Defiitio 5.5. We say f, g L 2 are orthogoal if f, g = 0. For V L 2, we defie the subspace V = { f L 2 : f, v = 0 for all v V }. V L 2 is called closed if, for every sequece f N i V, with f f i L 2, we have f = v a.e., for some v V. Theorem 5.8. Orthogoal projectio Let V be a closed subspace of L 2. The each f L 2 has a decompositio f = v + u, with v V ad u V. Moreover, f v 2 f g 2 for all g V, with equality iff g = v a.e.. Remark. The decompositio is uique up to a versio, sice for ṽ V, ũ U with v + u = ṽ + ũ a.e. we have with Pythagoras rule 0 = v ṽ + u ũ 2 2 = v ṽ u ũ 2 2. Thus ṽ = v ad ũ = u a.e., ad v is called the orthogoal projectio of f o V. Proof. Choose a sequece g V such that, as, f g 2 df, V := if { f g 2 : g V }. By the parallelogram law, 2 f g + g m / g g m 2 2 = 2 f g f g m 2 2. But 2 f g + g m / df, V 2, so we must have g g m 2 0 as, m. By completeess, g g 2 0, for some g L 2, ad by closure g = v a.e., for some v V. Hece f v 2 = lim f g 2 = df, V f h 2 for all h V. I particular, for all t R, h V, we have df, V 2 f v + t h 2 2 = df, V 2 2t f v, h + t 2 h 2 2. So we must have f v, h = 0, ad u = f v V, as required. 32

33 Defiitio 5.6. For X, Y L 2 P with meas m X = X ad m Y = Y we defie variace, covariace ad correlatio by varx = X m X 2, covx, Y = X m X Y m Y, corrx, Y = covx, Y / varx vary. For a radom variable X = X 1,..., X i R we defie the covariace matrix varx = covx i, X j i,j=1,..,. Remarks. i varx = 0 if ad oly if X = m X a.s.. ii covx, X = varx ad if X ad Y are idepedet, the covx, Y = 0. iii By Hölder covx, Y varx vary ad thus corrx, Y [ 1, 1]. Propositio 5.9. very covariace matrix is o-egative defiite. Proof. see example sheet. Revisio. Let Ω, A, P be a probability space ad let G A be some evet. For PG > 0 the coditioal probability P. G, give by PA G = PA G PG for all A A, is a probability measure o Ω, A. For a radom variable X : Ω R we deote / X G = X dp. G = X 1 G dp PG = X 1 G /PG, Ω Ω wheever PG > 0, ad we set X G = 0 whe PG = 0. Defiitio 5.7. Let G i iıi be a coutable family of disjoit evets with i G i = Ω ad set G = σg i : i I. The the coditioal expectatio of a r.v. X give G is give by X G = i I X G i 1 Gi. Remarks. i X G is a G/B-measurable r.v., takig costat values o each G i. I particular, for G = σω = {, Ω}, X {, Ω} = X Ω 1 Ω = X. ii For every A G it is A = i J G i for some J I. Thus A X G dp = i I X 1 Gi 1 Gi dp / PG i = X 1 Gi = X dp. A i J A I particular, if X is itegrable, X G is itegrable ad X G = X. iii L 2 G, P is complete ad therefore a closed subspace of L 2 A, P. If X L 2 A, P the X G L 2 G, P. 33

34 Propositio If X L 2 A, P the X G is a versio of the orthogoal projectio of X o L 2 G, P. Proof. see example sheet Remarks o the geeral case i For a geeral σ-algebra F A oe ca show, that for every itegrable r.v. X there exists a F-measurable, itegrable r.v. Y with F Y dp = F X dp for every F F. It is uique up to a versio, defiig the coditioal expectatio Y = X F. ii If X is F-measurable, X F = X. I particular X A = X. iii For σ-algebras G 1 G 2 A we have X G 2 G 1 = X G1 = X G 1 G Covergece i L 1 Let Ω, A, P be a probability space ad cosider L 1 = L 1 Ω, A, P. From the example sheet 3.2 we kow that if X coverges to X i L 1, the also i probability. Theorem Bouded covergece Let X N be a sequece of radom variables with X X i probability. If i additio X C a.s. for all N ad some C <, the X X i L 1. Proof. By Theorem 3.10ii X is the almost sure limit of a subsequece, so X C a.s.. For ɛ > 0 there exists N N such that for all N: P X X > ɛ/2 ɛ/4c. The X X = X X 1 X X >ɛ/2 + X X 1 X X ɛ/2 2 C ɛ/4c + ɛ/2 = ɛ. Lemma For X L 1 A, P set I X δ = sup { X 1 A : A A, PA δ }. The I X δ 0 as δ 0. Proof. Suppose ot. The, for some ɛ > 0, there exist A A, with PA 2 ad X 1 A ɛ for all N. By the first Borel-Catelli lemma, PA i.o. = 0. But the by domiated covergece ɛ X 1 m A m X 1{A i.o.} = 0 as, which is a cotradictio. Defiitio 5.8. Let X be a family of radom variables. For 1 p we say that X is bouded i L p if sup { X p : X X } <. Defie I X δ = sup { X 1 A : X X, A A, PA δ }. 34

35 We say that X is uiformly itegrable or UI if X is bouded i L 1 ad I X δ 0, as δ 0. Remarks. i X is bouded i L 1 if ad oly if I X 1 = sup { X 1 : X X } <. ii With Lemma 5.12, ay sigle, itegrable radom variable is UI, which ca easily be exteded to fiitely may. iii X is UI, if there exists Y L 1 A, P such that X Y for all X X [ X 1A Y 1A for all A A, the use ii ] there exists p > 1 such that X is bouded i L p [ by Hölder, for cojugate idices p ad q <, X 1A X p PA 1/q ] xample. X = 1 0,1/ is bouded i L 1 for Lebesgue measure o 0, 1], but ot UI. The followig result provides a alterative characterizatio of uiform itegrability. Propositio A family X of radom variables is UI if ad oly if sup { } X 1 X K : X X 0, as K. Proof. Suppose X is UI. Give ɛ > 0, choose δ > 0 so that I X δ < ɛ, the choose K < so that I X 1 Kδ. The with A = { X K } we have PA δ, so X 1 A < ɛ for all X X. Hece, as K, sup { X 1 X K : X X } 0. O the other had, if this coditio holds, I X 1 <, sice X K + X 1 X K. Give ɛ > 0, choose K < so that X 1 X K < ɛ/2 for all X X. The choose δ > 0 so that Kδ < ɛ/2. For all X X ad A A with PA < δ, we have X 1 A X 1 X K + K PA < ɛ. Hece X is UI. Theorem Let X, N ad X be radom variables. The followig are equivalet: i X L 1 for all N, X L 1 ad X X i L 1, ii {X : N} is UI ad X X i probability. Proof. Suppose i holds. The X X i probability, followig problem 3.2. Moreover, give ɛ > 0, there exists N such that X X < ɛ/2 wheever N. The we ca fid δ > 0 so that PA δ implies, usig Lemma 5.12, X 1 A ɛ/2, X 1 A ɛ, for all = 1,..., N. The, also for N ad PA δ, X 1 A X X + X 1 A ɛ. Hece {X : N} is UI ad we have show that i implies ii. 35