A getle itroductio to Measure Theory Gaurav Chadalia Departmet of Computer ciece ad Egieerig UNY - Uiversity at Buffalo, Buffalo, NY gsc4@buffalo.edu March 12, 2007 Abstract This ote itroduces the basic cocepts ad defiitios of measure theory relevat to probability theory. It is meat to be a simplified tutorial o measure theory. 1 Itroductio The Method of Exhaustio is a techique to fid out the area of a shape (for example, circle) by approximatig (lower boud) it by iscribig it usig a polygo. The idea is that as the umber of sides of the polygo teds to ifiity the area of the polygo will get arbitrarily close to the area of the circle. This method was used by Archimedes to fid out the value of π. Thus, by exhaustig the values of the area of the polygo, the area of the circle ca be obtaied. I fact, the Riema itegral is a applicatio of the method of exhaustio. Measure theory is cocered with geeralizig the otios of area o arbitrary sets of Euclidea spaces ad otios of legth of subsets of R. Essetially, it is a commo groud for aalysis of real fuctios ad set theory. For the otio of area, there are certai properties like o-egativity ad additivity that hold true. imilarly, i measure theory there is a stroger assumptio of coutable additivity. Measure theory ivolves study of σ-algebras (abstract mathematical structures), measures, measurable fuctios ad itegrals. Itegratio i the cotext of measure theory ivolves aalogous sums ad is based o fuctios costat o sets of some σ-algebra ad ot o fuctios costat o itervals (as is doe i the traditioal maer). I will clearly defie a space so that there is somethig to measure i ectios 2.3 ad 2.4; the I will itroduce the otio of a measure o this space i ectio 3 ad show how measures are used to defie a itegral i ectio 3.2. 2 Basics Before lookig at measure theory more formally, I will review some of the basic cocepts icludig some ideas from abstract algebra that defie the uderlyig mathematical structures o which measures are defied. 1
2.1 Limit of a sequece A sequece x 1, x 2,... x is said to coverge to the poit or has limit x if for every ɛ > 0 there is a atural umber N ɛ such that the eighborhood of O(x, ɛ) (a circle with ceter x ad radius ɛ) has all the poits x with > N ɛ. Ituitively, it meas that the elemets of the sequece evetually become arbitrarily close to the poit x. 2.2 Cauchy equece A sequece x 1, x 2,... x is said to be a Cauchy sequece if for every ɛ > 0 there exists a N ɛ such that d(x, x ) < ɛ for all, N ɛ. Ituitively, it meas that the elemets of the sequeces evetually become close to each other. However, Cauchy sequeces are ot the same as coverget sequeces (havig a limit) except i certai cases (for example i R). ome of the importat properties of Cauchy sequeces are: Ay Cauchy sequece is bouded. Ay Cauchy sequece i R is coverget. 2.3 Algebraic tructure I pure mathematics, a algebraic structure is a collectio of oe or more sets that are operated upo by oe or more operatios. These operatios satisfy certai axioms. A algebraic structure cosists of all the possible models that are characterized by the axioms. Algebraic structures are categorized by the umber of sets ad biary / uary operatios they require. Followig are some of the relevat structures: Groups - A group (, ) is a set uder a biary operatio : that satisfies certai axioms like associativity, closure, e.g. Itegers uder additio which also happes to be a abelia group. A abelia groups is a group i which the order of operatio does ot matter, i.e. a b = b a. Rigs - A rig is a set uder two biary operatios : addictio ad multiplicatio. A rig uder additio is a abelia group. e.g. itegers, polyomials. A commutative rig is a rig i which the members satisfy the commutative law. Fields - A field is a structure uder which the operatios of additio, multiplicatio, divisio ad subtractio are performed. A field is a commutative rig. 2.4 σ-algebra ad Borel algebra σ-algebra is a collectio of subsets of a set that is closed uder coutable set operatios i.e. the complemet of a member (subset), the uio or itersectio of coutably may members are also members. Formally, a algebra of subsets of a set is a σ-algebra if cotais the limit of every mootoe sequece of its sets. The pair (, ) is the a measurable space ad the sets i are said to be measurable. e.g. 2. Borel algebra of a set is the miimal σ-algebra that cotais all the ope sets (or closed sets) o the real lie. The elemets of Borel algebra are called Borel ets. σ-algebras (i particular, Borel algebras) allow us to cocetrate o sets that are (i some way) easy to use. I other words, these mathematical structures hold certai importat properties (coutable additivity - 2.5) that allow us to defie the cocept of a measure o seemigly arbitrary sets. 2
2.5 Coutably Additive Let φ be a real fuctio, ad be a σ-algebra. The fuctio φ is coutably additive or σ-additive whe φ() = φ( i ) where ; = i ; i j = 0 (1) It meas that for fiite or coutably ifiite sequece of disjoit sets, the legth of the uio of these sets is equal to the sum of the legths of these sets. The fuctio φ is coutably sub-additive whe φ() φ( i ) where ; i (2) 2.6 Metric pace A metric space is a set wherei the cocept of legth or distace betwee elemets is defied. It is deoted by (, d) where is a set ad d is the metric (a sigle-valued, o-egative distace fuctio) such that d : R. It satisfies the properties of positivity, symmetry, idetity ad triagle iequality. If every Cauchy sequece i a metric space (, d) coverges to a elemet i the space, the the space (, d) is said to be complete, e.g. The ope iterval (0, 1) is ot complete whereas the closed iterval [0, 1] is complete. Every compact (closed ad bouded) metric space is complete. Every ormed space (distace fuctio set to the absolute differece betwee two elemets of the space) is also a metric space. As a side ote, a complete ormed space is siad to be a Baach space. 3 Measure pace ad Measure If there is a coutably additive set fuctio µ defied o σ-algebra of subsets of, the the triplet (,, µ) is a measure space. A example of measure space is a Euclidea space with Lebesgue measure. The sets of are called measurable sets ad the fuctio µ is called a measure ad has the followig properties: µ is o-egative, i.e. µ(x) 0 for all X. µ is coutably additive (as defied i sectio 2.5). µ( ) = 0. µ obeys mootoicity : µ(x) µ(y ) for all X, Y ad X Y. The cocept of measure has coectios with itegratio over arbitrary sets. Moreover, if µ() = 1, the the measure space is called a probability space ad µ is a probability measure. The sets are called evets. Oe says that a property holds almost everywhere if the set for which the property does ot hold is a ull set or a set with measure 0. I probability theory, aalogous to almost everywhere, almost certai or almost sure meas except for a evet of probability measure 0. This meas that the evet has zero probability of ot occurrig although it is still possible that they might ot occur. There are differet kids of measures - Borel measure, Lebesgue measure, Haar measure, Dirac measure to ame a few. 3
The measure space (,, µ) is complete if every subset of a set of measure 0 is measurable ad has measure 0. Formally, for all, if µ() = 0 the for all subsets X, µ(x) = 0. If ot, the a space ca be made complete. The Borel measure is ot complete ad hece the complete Lebesgue measure is preferred. Every Borel set is also Lebesgue measurable. 3.1 Lebesgue Measure The defiitio of measure as give above is of a geeralized form. However, i practice it is ofte the case that oly those sets that are Lebesgue measurable eed to be cosidered. To get to that, let s defie a outer measure µ (X) ad a ier measure µ (X) o a set X. The outer measure of a set X is defied as: { µ (X) = if m(b ) : m is a measure o iterval B ad X } B The outer measure as defied above is agai of geeralized form. Cosider a iterval B with edpoits a ad b such that a < b istead of the set A, the the outer measure for the iterval is defied as b a. The ier measure is defied as : (3) If the followig is true, µ (X) = m() µ ( \ X) (4) µ (X) = µ (X) (5) the µ(x) (which deotes the value of µ (X) or µ (X)) is said to be a Lebesgue measure o the set X. 3.2 Lebesgue Itegral A measure µ of a measure space (,, µ) is determied by its values µ(s i ) where s. is a fiite or ifiite sequece i the coutable space. Here, = 2. If f is a fuctio from this coutable space ito R ad is µ-itegrable over the set (i.e. the sequece evaluated uder f is absolutely coverget), the the Lebesgue itegral of f o is defied as a sum of the sequece evaluated uder f as show below : f(s)dµ = f(s. )µ(s. ) (6) If µ() = 1 i.e. the σ-algebra is a probability space ad µ is a probability measure, the i probability theory, Equatio 6 is referred to as the expectatio of f ad is deoted by E(f). A geeralized defiitio of the itegral of f whe there exists a sequece of fuctios {f } µ-itegrable over ad uiformly coverget to f is give by: f(s)dµ = lim f (s)dµ (7) 4
The defiitio of a itegral i this form is i cotrast with that of the Riema itegral which is oly applicable to itervals ad uios of itervals. For Riema itegral (followig the idea of method of exhaustio), the space over which f is defied is divided ito smaller sub-itervals ad represeted i the followig way : f(η )(x +1 x ) (8) where the value of f(x) is substituted by the fuctio evaluated at a arbitrarily chose poit η withi the iterval [x, x +1 ]. However, this ca be doe oly whe f(x) is cotiuous or whe it does ot have may poits of discotiuity. The Riema itegral of f is obtaied by takig the limit of to ifiity i the above sum. I order to distiguish betwee the Lebesgue ad Reima itegrals cosider the values that the fuctio f ca take to be o the x-axis (called the domai of the fuctio) ad the values of the fuctio evaluated at the chose poits to be o the y-axis (called the rage of the fuctio). The Lebesgue itegral defies the sub-itervals alog the rage of the fuctio ad the Riema itegral defies them alog the domai of the fuctio. Thus, the Riema itegral requires the values of the domai to be very close to each other (fuctio should be cotiuous). This is a limitatio sice it restricts the class of fuctios o which the itegral ca be defied. Thus the Lebesgue itegral is preferred because it is applicable to arbitrary classes of fuctios. Aother reaso is that if there is a sequece of Riema itegrable fuctios {f }, the it is ot true that the fuctio f defied as f = lim f (absolutely coverget) is Riema itegrable. Now, I preset two theorems from Kolmogorov ad Fomi [1999] that explicitly state the coditios uder which the limit of a itegral is equal to the itegral of the limit as give i the defiitio i Equatio 7. The eed to take the limit iside the itegral or to cosider piecewise itegratio of a coverget sequece of fuctios arises i serveral applicatios, most otable beig Fourier series. Theorem 3.1 (Domiated Covergece Theorem). If {f } is a sequece of measurable fuctios from a measure space (,, µ) ito R ad the sequece coverges to f ad f (s) g(s) almost everywhere, for all, where g is a o-egative fuctio itegrable over, the the limit fuctio f is itegrable over ad f(s)dµ = lim f (s)dµ Theorem 3.2 (Mootoe Covergece Theorem). If {f } is a sequece of measurable fuctios from a measure space (,, µ) ito R such that f 1 f 2... f... ad the sequece coverges to f, the the limit fuctio f is itegrable over ad f(s)dµ = lim f (s)dµ The theorem 3.1 is so called because of the use of the itegrable fuctio g for statig the coditio of itegrability of piecewise fuctios {f } ad the theorem 3.1 is so called because it assumes that the sequece of piecewise fuctios is mootoically icreasig. 5
4 Measurable Fuctio It has bee prove that it is impossible to assig a measure to all subsets of R without preservig some of the properties like additivity. Hece, as see i the previous sectio I will cocetrate oly o certai measurable sets. Cosequetly, I will geeralize the cocept of a fuctio adapted to the class of measurable sets ad hece to their correspodig σ-algebras. uch fuctios are called measurable fuctios. They are defied as follows: Let (, 2 ) be a coutable measurable space ad cosider fuctios from a measurable space (, ) ito. uch a fuctio f is measurable if the followig is true for every set X : f 1 [X] = {s f(s) X} (9) Equatio 9 is also called the preimage (ot a iverse fuctio) of set X uder the measurable fuctio f. If (, ) is equipped with a probability measure, a measurable fuctio is a radom variable i the cotext of probability theory. 5 Fial Commet Measure theory is a fasciatig area of study i its ow right ad at the same time plays a importat role i fuctioal aalysis, advaced probability theory ad statistics. I this ote, I have oly laid dow a framework that gives a ituitive isight ito the world of measures ad eables us to defie itegrals, probability distributios ad radom variables. For more rigorous proofs, covergece properties ad for explorig the use of Lebesgue itegral i the cotext of square itegrable fuctios ad Hilbert spaces, the reader is referred to Kolmogorov ad Fomi [1999]. A Few defiitios A.1 Power et A power set of a set is the set of all possible subsets of. It is deoted by 2 or P. If there are elemets i the set, the P = 2. 2 defies the set of all fuctios from to {0, 1}. A power set forms a abelia group whe cosidered with the operatio of symmetric differece ad a commutative rig whe cosidered with the operatios of symmetric differece ad itersectio. A.2 Covex et, Covex Closure, implex A set X i the liear space is said to be covex if give ay two arbitrary poits x ad y from the set, the segmet joiig them also belogs to X. The covex closure of the set X is the smallest closed covex set cotaiig X. The covex closure ca also be obtaied as the itersectio of all the closed covex set which cotai the set X. A set of poits x 1, x 2,... x +1 i a ormed liear space is said to be i geeral positio if o k + 1(k < ) of these poits lie i a subspace of dimesio less tha k (if it does, the it is the degeerate case). The covex closure of such a set of poits x 1, x 2,... x +1 that are i geeral positio is called a -dimesioal simplex. I other words, it is the miimal covex set. The -dimesioal simplex with + 1 poits is also represeted i the followig form: +1 x = α k x k ; α k 0 ad +1 α k = 1 (10) 6
A.3 Iequalities Let X be a measurable space with dimesio ; a 1, a 2,... a ad b 1, b 2,... b are real or complex umbers; 1 p, q ad 1 p + 1 q = 1. A.3.1 Cauchy-chwarz Iequality ( ) 2 a k b k a 2 k The Cauchy-chwarz iequality is used to prove triagle iequality. b 2 k (11) A.3.2 Hölder s Iequality ( ) 1 ( ) p 1 q a k b k a k p b k q (12) Hölders iequality is a geeralized case of Cauchy-chwarz iequality ad is used to prove the geeralizatio of triagle iequality ad the Mikowskis iequality. A.3.3 Mikowski s Iequality ( ) 1 ( p ) 1 a k + b k p a k p p ( b k q ) 1 q (13) A.3.4 Weak ad trog Covergece A sequece {x } i a ormed liear space L is said to coverge weakly to a elemet x whe the followig coditios are true: The orms of the elemets {x } are uiformly bouded : x M. f(x ) f(x) for all f L The sequece {x } is said to coverge strogly whe its orm coverges to x such that x x 0 as. Obviously, for fiite dimesioal spaces, strog covergece ad weak covergece is the same. We have aalogous defiitios for covergece of liear fuctioals. Refereces A. N. Kolmogorov ad. V. Fomi. Elemets of the Theory of Fuctios ad Fuctioal Aalysis, volume I ad II. Dover Publicatios, Mieola, NY, 1999. IBN 0-486-40683-0. 7