NOTES ON PROBABILITY THEORY FOR ENEE 620. Adrian Papamarcou. (with references to Probability and Measure by Patrick Billingsley)

Transcription

1 NOTES ON PROBABILITY THEORY FOR ENEE 620 Adria Papamarcou (with refereces to Probability ad Measure by Patrick Billigsley) Revised February

2 1. Itroductio to radom processes A radom process is a collectio of radom variables (r.v. s for short) that arise i the same probability experimet (the last clause ca be replaced by the exact statemet that are defied o a commo probability space; the term probability space will be defied i the ext sectio). Thus a radom process is mathematically represeted by the collectio {X t, t I}, where X t deotes the t th radom variable i the process, ad the idex t rus over a idex set I which is arbitrary. A radom process is a mathematical idealizatio of a set of radom measuremets obtaied i a physical experimet. This radomess ca be quatified by a probabilistic or statistical descriptio of the process, ad the complexity of this descriptio depeds largely o the size of the idex set I. I briefly discussig this issue of complexity, we cosider idex set sizes, which cover most cases of iterest. (a) I cosists of oe idex oly. I this case we are measurig a sigle radom quatity, represeted by the r.v. X. From elemetary probability, we kow that a simple way of describig X statistically is through its cumulative distributio fuctio (or cdf ) F X, which is defied by the relatioship F X (x) = Pr{X x} (Notatio. Throughout this course, radom variables will be deoted by upper case letters, ad fixed (o-radom) umbers by lower case letters.) F X is always odecreasig, cotiuous from the right, ad such that F X ( ) = 0, F X (+ ) = 1. Thus typically it looks like this: 1 F (x) X 0 x We also kow that i most cases of iterest, we ca alteratively specify the statistics of X by a probability desity fuctio (or pdf ) f X, which is a oegative fuctio that itegrates to uity over the etire real lie ad is related to F X by F X (x) = x 1 f X (u)du.

3 This also covers r.v. s with discrete compoets, i which case f X cotais δ fuctios. (b) I cosists of idices, e.g. I = {1,..., }. I this case the process variables form a radom vector i R, deoted by X = (X 1,..., X ). The statistical descriptio of the process ca be accomplished by specifyig the cdf F X (or F X1,...,X ) of the radom vector X; this is a real-valued fuctio o R defied by the relatioship F X (x 1,..., x ) = Pr{X 1 x 1,..., X x }. The dimesio of their argumet otwithstadig, the fuctios F X ad F X have quite similar behavior. F X, too, is odecreasig: if y k x k for all values of k, the F X (y 1,..., y ) F X (y 1,..., y ). Furthermore, i most cases of iterest, we ca write F X (x 1,..., x ) = x for a suitable pdf f X, also defied o R. x1... f X (u 1,..., u )du 1 du, Recall that the cdf of ay sub-vector of X ca be easily determied from the cdf F X by settig the redudat argumets of F X equal to +. Thus for example, the cdf of the r.v. X 1 is computed via F X1 (x 1 ) = F X (x 1,,..., ). This procedure is ot reversible: kowledge of the margial distributios of X does ot i geeral suffice to determie F X. Oe importat exceptio is the case where the compoets of the radom vector are idepedet ; the the cdf of X is give by the product of the cdf s of the idividual compoets, i.e., F X (x 1,..., x ) = F X1 (x 1 ) F X (x ), ad the same relatioship is true if we replace cdf s by pdf s (F by f). I the last two cases, we cosider ifiite idex sets I. (c) I is coutably ifiite, say I = N (the set of positive itegers or atural umbers). Here the process is equivalet to a sequece of radom variables X 1, X 2,.... The problem of describig the statistics of ifiitely may radom variables is most ecoomically solved by specifyig the so-called fiite-dimesioal distributios, amely the 2

4 distributios of all fiite-dimesioal vectors that ca be formed with these variables. I this case, the stated procedure amouts to specifyig the cdf F Xt1,...,X t for every choice of ad (distict) itegers t 1,..., t. Although the above specificatio of fiite-dimesioal distributios suffices to describe statistically what happes i i the radom process over ay fiite idex- (or time-) widow, it is ot clear whether it also determies properties of the process that effectively ivolve the etire idex set (or discrete-time axis). Cosider for example the radom variable X X X = lim which gives the asymptotic value of the time average of the radom observatios. X is clearly a property of the process, yet its value is ot determied by ay fiite umber of variables of the process. Thus, X g(x t1,..., X t ) for ay choice of g ad argumets t 1,..., t, ad we caot use a sigle fiite-dimesioal distributio of the process to determie the cdf of X. As it turs out (this is a rather profoud fact i probability theory), most ifiitary properties of the process are determied by the set of all fiite-dimesioal distributios. Such properties iclude radom quatities such as limits of time averages, ad thus the statistics of X are i priciple deducible from the fiite-dimesioal distributios (i practice, the task is usually formidable!). Put differetly, if two distict radom processes {X k, k N} ad {Y k, k N} have idetical fiite-dimesioal distributios, the the variables X ad Y will also have idetical statistics. I summary, augmetatio of a fiite idex set to a coutably ifiite oe ecessitates the specificatio of a ifiite set of fiite-dimesioal distributios. This etails a cosiderable jump i complexity, but esures that all importat properties of the process (icludig asymptotic oes) are statistically specified. (d) I this last case we cosider a ucoutably ifiite idex set, amely I = R. If we thik of the process as evolvig i time, the we are effectively dealig with a cotiuous-time process observed at all times. I cotiuig our previous discussio o fiite-dimesioal distributios, we ote aother rather profoud fact: fiite-dimesioal distributios o loger suffice to determie all saliet characteristics of the process. As a example, suppose oe wishes to model the umber of calls hadled by a telephoe exhage up to time t, where t R. A typical graph of this radom time-varyig quatity would be 3

5 t It is ow possible to costruct two models {X t, t R} ad {Y t, t R} that have idetical fiite dimesioal distributios, yet differ i the followig importat aspect: {X t, t R} (almost) always gives observatios of the above typical form, whereas {Y k, k Z} is ot kow to do the same. More precisely, equals uity, whereas the quatity Pr{X t is iteger-valued ad odecreasig for all t} Pr{Y t is iteger-valued ad odecreasig for all t} caot be defied, ad hece does ot exist.* Of the two processes, oly {X t, t R} is (possibly) suitable for modelig the radom physical system i had. The reaso for the above discrepacy is that the two radom processes {X t, t R} ad {Y t, t R} are costructed i etirely differet ways. This illustrates the geeral priciple that radom processes are ot fully characterized by distributios aloe; their costructio amouts to the specificatio of a family of radom variables o the same probability space. Precise uderstadig of the cocepts probability space ad radom variable is therefore essetial. 2. A simple stochastic process Billigsley, Sec. 1, The uit iterval. Cosider the probability experimet i which we choose a poit ω at radom from the uit iterval (0, 1]. (Notatio. A parethesis implies that the edpoit lies outside the iterval; a square bracket that it lies iside.) We assume that the selectio of ω is uiform, i that { } Pr ω (a, b] = b a. * The pivotal differece betwee the statemets X 0 i (c) ad X t is itegervalued ad odecreasig for all t i (d) is that the former ivolves a coutable ifiity of time idices, whereas the latter ivolves a ucoutable oe. Agreemet of two processes over fiite-dimesioal distributios implies agreemet over coutably expressible properties, but does ot guaratee the same for ucoutably expressible oes. 4

6 { } As expected, Pr ω (0, 1] = 1. We ow cosider the biary expasio of the radom poit ω. We ca write X k (ω) ω =.X 1 (ω)x 2 (ω)... = 2 k, where X k (ω) stads for the k th digit i the biary expasio. A iterative algorithm for derivig these digits is as follows. We divide the uit iterval ito two equal subitervals, ad set the first digit equal to 0 if ω falls i the left-had subiterval, 1 otherwise. O the subiterval cotaiig ω we perform a similar divisio to obtai the secod digit; ad so forth. This is illustrated i the figure below. k=1 0 ω 1 X (ω) = 1 1 X (ω) = 0 2 X (ω) = 1 3 1/2 1 1/2 3/4 5/8 3/4 (Notatio. Edpoits marked lie outside, those marked iside, the set or curve depicted) The variatio of the k th digit X k (ω) with ω has the followig distictive feature. Startig from the left, X k alterates i value betwee 0 ad 1 o adjacet itervals of legth 2 k : Thus the graphs of X 1 ad X 2 look like this: X 1(ω) X 2(ω) /2 ω 1 0 1/4 1/2 3/4 ω 1 From the above observatio we deduce that the vector X = (X 1,..., X ) has the followig behavior as ω varies: ω (0, 2 ] ω (2, 2 2 ] ω (2 2, 3 2 ]. ω ( (2 1)2, 1 ] : : :.. : X(ω) = X(ω) = X(ω) = X(ω) =

7 Thus each of the 2 biary words of legth is obtaied over a iterval of legth 2. I terms of probabilities (here legth=probability), all biary words of legth are equally likely cadidates for the trucated expasio of a poit draw uiformly at radom from the uit iterval. Notig also that ay fixed digit X k is 0 or 1 with equal probability, we coclude that for a biary word (a 1,..., a ), Pr{X 1 = a 1,..., X = a } = 2 = Pr{X 1 = a 1 } Pr{X = a }. Now compare the above with the situatio i which Y 1, Y 2,... are the outcomes of a sequece of idepedet tosses of a fair coi labeled 0 ad 1. By idepedece, we have Pr{Y 1 = a 1,..., Y = a } = 2 = Pr{Y 1 = a 1 } Pr{Y = a }. Thus we have two sequeces of radom quatities with idetical probabilistic descriptios (it is easy to verify that the processes {X k, k N} ad {Y k, k N} have the same fiite-dimesioal distributios). Sice drawig a poit from a iterval is i a sese simpler tha tossig a coi ifiitely may times, we ca use the process {X k, k N} istead of {Y k, k N} to model the outcomes of idepedet coi tosses. This choice has the iterestig implicatio that oe ca costruct ifiitely may radom quatities without explicit referece to their (joit) statistical descriptio by defiig these quatities as fuctios of the outcome of a sigle probability experimet. The above leads to the followig iterpretatio, which will prevail i this course: a radom variable is a real-valued fuctio of the outcome of a probability experimet. A radom process is a collectio of such fuctios, all of which are defied i terms of the same probability experimet. 3. The otio of a probability space For refereces, see subsequet sectios. A probability space is a mathematical model for a radom experimet (or probability experimet). It cosists of three etities. (i) A abstract set of poits, called sample space, ad usually deoted by Ω. The poits, or elemets, of Ω are usually deoted by ω. Iterpretatio: Ω is the set of all possible outcomes of the radom experimet. (ii) A collectio of subsets of Ω, called evet space, ad usually deoted by a upper case script character such as F. The sets that costitute the evet space are called evets. Iterpretatio: The evet space essetially represets all possible modes of observig the experimet. A subset A of Ω is a evet if we ca set up a observatio mechaism to detect whether the outcome ω of the experimet lies i A or ot, i.e., whether A occurs or ot. (iii) A fuctio P, called probability measure, which is defied o the evet space ad takes values i the iterval [0, 1]. Iterpretatio: For every evet A, P (A) provides a umerical assessmet of the likelihood that A occurs; the quatity P (A) is the probability of A. 6

8 The stadard represetatio of a probability space is a triple with the above three etities i their respective order, i.e., (Ω, F, P ). The pair (Ω, F) is referred to as a measurable space. It describes the outcomes ad modes of observatio of the experimet without referece to the likelihood of the observables. I geeral, the same measurable space ca give rise to may differet probability spaces. 4. Evet spaces ad fields Billigsley, Sec. 2, Spaces ad Classes of Sets. From a mathematical viewpoit, the sample space Ω is etirely ucostraied; it is a arbitrary set of poits. Costraits o Ω are imposed oly by modelig cosideratios: Ω should be rich eough to represet all outcomes of the physical experimet that we wish to model. This does ot mea that a poit ω should be of the same form as the outcome of the physical experimet; it merely suggests that oe should be able to set up a correspodece betwee the actual outcomes ad the poits i Ω. Thus i the Example of Sec. 2 above, the sample space Ω = (0, 1] adequately represeted the outcomes of a sequece of coi tosses, i spite of the fact that the poits i Ω were ot themselves biary sequeces. This was because it was possible to idetify every ω with a distict biary sequece by takig its biary expasio (coversely, every biary sequece that does ot coverge to 0 ca be idetified with a distict poit i (0, 1]). I cotrast to the above, the mathematical costraits o the evet space F are rigid; they stem from the earlier iterpretatio of evets as sets of outcomes that are observable by available mechaisms. Three such costraits are give below. 1. F, Ω F. This is reasoable i view of the fact that o observatio is eeded to determie whether the outcome lies i (impossible) of Ω (certai). 2. A F A c F (closure uder complemetatio) Obvious, sice the same observatio mechaism is used for both A ad A c. 3. A F, B F A B F (closure uder uio) By combiig the two observatio mechaisms (A versus A c ad B versus B c, oe obtais a sigle observatio mechaism for A B versus (A B) c. Defiitio. A algebra or field is a collectio of subsets of Ω satisfyig coditios (1) (3) above. Examples of fields. (i) Ω arbitrary. F = {, Ω}. F is easily see to satisfy coditios (1) (3). It is the smallest field that ca be built from a sample space Ω, ad is ofte referred to as the trivial field. Clearly, o useful observatios ca be made i the experimet represeted here. 7

9 (ii) Ω arbitrary. F = power set of Ω = the collectio of all subsets of Ω. Agai F is easily see to satisfy coditios (1) (3): by covetio, the empty set is a subset of every set, ad set operatios o subsets of Ω always yield subsets of Ω. I the experimet modeled here, every subset of Ω ca be tested for occurrece; we thus have the exact opposite of example (i). (Notatio. The power set of Ω is deoted by 2 Ω.) (iii) Here Ω is agai arbitrary, ad we cosider sets C 1,..., C M partitio or decompositio of Ω; that is, that form a fiite ( i, j s.t. i j) C i C j = ad M C i = Ω. The sets C i are referred to as cells or atoms of the partitio. The defiitio of F is as follows: { F = A : A = } C i, I {1,..., M}. i I Thus F cosists of all uios of sets C i ; by covetio, we let C i =. To see whether F is a field, we check coditios (1) (3). i (1) = i C i F, Ω = (2) A = i I C i A c = (3) A = i J ad thus F is a field. C i, B = i K i {1,...,M} i {1,...,M} I C i A B = C i F ; C i F ; i J K C i F, (iv) Here we take Ω = (0, 1], ad we defie F as the collectio cosistig of the empty set ad all fiite uios of semi-ope subitervals of (0,1], i.e., F = { } { A : A = M } (a i, b i ], M <, (a i, b i ] (0, 1]. Here coditio (1) is easily see to be satisfied: is explicitly icluded i F, ad the choice M = 1, a 1 = 0, b 1 = 1 yields Ω F. The same is true of coditio (3), sice the uio of two fiite uios of itervals is itself a fiite uio of itervals. 8

10 To check coditio (2), we first ote that if two itervals (a 1, b 1 ] ad (a 2, b 2 ] overlap, their uio is a sigle semi-ope iterval (c, d]. Based o this observatio, we ca use a simple iductive argumet to show that a fiite uio of semi-ope itervals ca be expressed as a fiite uio of o-overlappig semi-ope itervals. I other words, M (a i, b i ] = N (c i, d i ], where 0 c 1 < d 1 < c 2 < d 2 <... < c N < d N 1 ad N M. Now ( N c (c i, d i ]) = (0, c1 ] (d 1, c 2 ]... (d N 1, c N ] (d N, 1], where both (0, 0] ad (1, 1] are take to be the empty set. This equality (illustrated i the figure below) verifies coditio (2), thereby provig that F is a field. A: 0 = c 1 d 1 c 2 d 2 c 3 d 3 1 A c : Two further properties of fields (4) Closure uder itersectio: A F, B F A B F. To see this, recall de Morga s law: (A B) c = A c B c. Suppose ow that A ad B lie i F. By axioms (2) ad (3), the same is true of the sets A c, B c, A c B c ad (A c B c ) c. The last set is precisely A B. (5) Closure uder fiite uios: A 1,..., A F A 1 A F. We prove this by a easy iductio: suppose the statemet is true for ay sets A 1,..., A F, ad that A +1 is also a set i F. The by axiom (3), we have that A 1 A +1 = (A 1 A ) A +1 also lies i F, which proves that the statemet is true for ay + 1 sets i F. As the statemet is obviously true i the case = 1, the iductio is complete. Remark. From (4) ad (5) it easily follows that every field is closed uder fiite itersectios. 9

11 5. Evet spaces ad sigma-fields Billigsley, Sec. 1., Classes of Sets. Uios ad itersectios over arbitrary idex sets Suppose {A i, i I} is a collectio of subsets of Ω; here the idex set I is etirely arbitrary. The uio of the sets A i over I is defied as the set of poits ω that lie i at least oe of the sets i the collectio; i.e., A i = {ω : ( i I) ω A i }. i I The itersectio of the sets A i over I is defied as the set of poits ω that lie i every oe of the sets i the collectio, i.e., A i = {ω : ( i I) ω A i }. i I (Notatio. The symbol reads for all, ad reads there exists oe. ) Fields ad coutable uios We saw that fields are closed uder the operatio of takig uios of fiitely may costituet sets. However, closure does ot always hold if we take uios of ifiitely may such sets. Thus if we have a sequece A 1, A 2,... of sets i a field F, the uio will ot always lie i F. A i def = To see a istace where such a coutable uio lies outside the field, cosider Example (iv) itroduced earlier. If we take A i = i N A i ( 0, ], 3i the the coutable uio will be a subset of (0, 1/2), sice each of the A i s is a subset of that ope iterval. We claim that this uio is actually equal to (0, 1/2). Ideed, if ω is ay poit i (0, 1/2), the for a sufficietly large value of i we will have A i ω i, 10

12 ad thus ω will lie i A i for that value of i. This is illustated i the figure below. A i ω 0 1/2 1 1/2-1/3i We have therefore show that the uio of all A i s is give by the ope iterval (0, 1/2), which caot be expressed as a fiite uio of semi-ope itervals ad hece lies outside F. Remark. A ofte asked questio is: what happes for i =? The aswer is, i ever takes ifiity as a value, ad the iclusio of i the symbol for the above coutable uio is purely a matter of covetio (just as i the case of a ifiite series). Thus the defiitio of the sequece A 1, A 2,... does ot ecompass a set such as A, which could be aïvely take as (0, 1/2 1/ ] = (0, 1/2]. The defiitio of a sigma-field A σ-field or σ-algebra is a field that is closed uder coutable uios. Thus a collectio F of subsets of Ω is a σ-field if it satisfies the followig axioms. 1. F, Ω F. 2. A F A c F (closure uder complemetatio). 3. ( i N) A i F A i F (closure uder coutable uios). Remark. Coutable meas either fiite or coutably ifiite; a set is coutably ifiite if its elemets ca be arraged i the form of a ifiite sequece, or equivaletly, put i a oe-to-oe correspodece with the atural umbers. Thus strictly speakig, (3 ) should be labeled closure uder coutably ifiite uios. Yet the distictio is uimportat, sice a fiite uio is a coutably ifiite uio where all but fiitely may sets are empty. I particular, (3 ) readily implies axiom (3) i the defiitio of a field (closure uder uio), as well as property (5) of the previous sectio (closure uder fiite uios). The followig statemet is a direct cosequece of the above cosideratios. Corollary. If a field cosists of fiitely may sets, it is also a σ-field. Examples of sigma fields Let us agai cosider the examples of fields give i Sectio 4. (i) F = {, Ω} is a σ-field by the above Corollary. (ii) F = 2 Ω is a σ-field sice ay set operatio (icludig takig coutable uios) yields a subset of Ω. 11

13 (iii) I this case F cosists of all uios of cells i a fiite partitio of Ω. Clearly F is fiite (if there are M cells, the there are 2 M sets i F) ad thus by the earlier corollary, F is a σ-field. If we take a coutably ifiite partitio of Ω ito sets C 1, C 2,..., the the collectio { F = A : A = } C i, I N, i I will also be a σ-field. It is easy to check the first two axioms; for closure uder coutable uios, we ote that give ay sequece of sets A 1, A 2,... i F such that A k = i I k C i, we ca write where I = I 1 I 2. k=1 A k = i I Remark. A measurable space (Ω, F) i which F is a σ-field cosistig of all uios of atoms i a coutable partitio of Ω is called discrete. (iv). I this example, F cosisted of the empty set ad all fiite uios of semi-ope subitervals of (0, 1]. As we saw earlier i this sectio, there exists sequece of semi-ope itervals i F, the coutable uio of which is a ope iterval lyig outside F. Thus F is ot closed uder coutable uios, ad hece it is ot a σ-field. 6. Geerated sigma-fields ad the Borel field Billigsley, Sec. 1, Classes of Sets. The sigma-field geerated by a collectio As we saw i the previous sectio, a field of subsets of Ω is ot always a σ-field. A questio that arises aturally is i what ways such a field (or more geerally, a arbitrary collectio of subsets of Ω) ca be augmeted so as to form a σ-field. It is easy to see that this is always possible, sice the power set 2 Ω is a σ-field which cotais (as a subcollectio) every collectio G of subsets of Ω. A far more iterestig fact is that there also exists a miimal such augmetatio: that is, give ay G, there exists a uique σ-field of subsets of Ω that both cotais G ad is cotaied i every σ-field cotaiig G. (Notatio. The term cotais ca mea either cotais as a subset or cotais as a elemet; which meaig is pertiet depeds o the cotext.) Before provig the existece of a miimal such σ-field, it is worth givig a simple example. Suppose Ω = (0, 1], ad defie the collectio G by { } G = (0, 1/3], (2/3, 1]. C i, 12

14 As poited out above, the power set F 1 = 2 Ω is (trivially) a σ-field that cotais G. To fid the smallest σ-field with this property, we reaso as follows. The sets ad (0, 1] clearly lie i every σ-field cotaiig G, as do (0, 1/3] ad (2/3, 1]. Hece the uio (0, 1/3] (2/3, 1] also lies i every such σ-field, ad so does its complemet (1/3, 2/3]. By closure uder uio, the same is true of the sets (0, 2/3] ad (1/3, 1]. Thus every σ-field cotaiig G must also cotai the collectio { } F 2 =, Ω, (0, 1/3], (1/3, 2/3], (2/3, 1], (0, 2/3], (1/3, 1], (0, 1/3] (2/3, 1]. Sice F 2 is itself a σ-field, we coclude that F 2 is the smallest σ-field cotaiig G. I the case of G cosistig of ifiitely may sets, the costructio of a miimal σ-field cotaiig G is ofte impossible. I cotrast, the proof of its existece is quite straightforward, ad relies o the simple fact that the itersectio of a arbitrary class of σ-fields is itself a σ-field. (Remark. I takig the itersectio of two collectios of subsets of Ω, we idetify those subsets of Ω that are commo to both collectios; we do ot take itersectios of subsets of Ω. For example, if Ω = (0, 1] ad G, F 1 ad F 2 are defied as above, the If also F 3 = {(0, 1/2]}, the G F 2 = G F 1 = G, F 2 F 1 = F 2. F 3 G = F 3 F 2 =, F 3 F 1 = F 3. Aalogous statemets ca be made for every set operatio ad relatio applied to collectios. Thus for example, G F 2 F 1, while F 3 is a subset of either G or F 2.) To show that a arbitrary itersectio of σ-fields is itself a σ-field, cosider F = k K where each F k is a σ-field of subsets of Ω ad the idex set K is arbitrary. We check each of the three axioms i tur. F k, (1) Both ad Ω lie i every F k, thus also i F. 13

15 (2) If A lies i F k for some k, the A c also lies i that F k. If A lies i every F k, the so does A c, i.e., A c F. (3 ) The argumet here is essetially the same as above: if A 1, A 2,... lie i every F k, the by closure of each F k uder coutable uios, the same will be true of A i. With the above fact i mid, we ca easily show that the miimal σ-field cotaiig a collectio G is the itersectio of all σ-fields cotaiig G. Ideed, the said itersectio (i) is itself a σ-field (by the above fact); (ii) is cotaied i every σ-field cotaiig G; ad (iii) cotais G. We summarize the above iformatio i the followig defiitio. Give a collectio G of subsets of Ω, the miimal σ-field cotaiig G, or equivaletly the σ-field geerated by G, is defied as the itersectio of all σ-field s cotaiig G, ad is deoted by σ(g). Oe last remark before proceedig to the ext topic is the followig: it is possible for two or more distict collectios to geerate the same σ-field. A simple illustratio of this fact ca be give i terms of our earlier example, where { } G = (0, 1/3], (2/3, 1]. If we ow take { } G = (0, 1/3], (1/3, 2/3], (2/3, 1] ad G = { } (0, 1/3], (0, 2/3], the oe ca easily verify that σ(g) = σ(g ) = σ(g ). The Borel field As oted above, the σ-fields geerated by fiite collectios of subsets of Ω are rather easy to costruct; such σ-fields are simply described i terms of fiite partitios of Ω (cf. Sectio 4, Example (iii)). For a coutably ifiite G, oe might be tempted to extrapolate that σ(g) will cosist of uios of cells i a coutable partitio of Ω. This will always be true if the sample space Ω is coutably ifiite, but ot so if Ω is ucoutable. To elaborate o the last statemet, we retur to the ucoutable space Ω = (0, 1]. 14

16 As we saw i Sectio 4, the field F = { } { A : A = M } (a i, b i ], M <, (a i, b i ] (0, 1] is ot a σ-field. By the foregoig discussio, F ca be augmeted to a miimal σ-field σ(f). This σ-field is called the Borel field of the uit iterval, ad is deoted by B((0, 1]). Thus B((0, 1]) def = σ(f). The Borel field is of crucial importace i probability theory, beig the basis for the defiitio of a radom variable ad its distributio. It cotais, amog others, all sets that arise from itervals by coutably may set operatios. It does ot, however, cotai every subset of the uit iterval; it is possible to give (admittedly cotrived) couterexamples to that effect. What other collectios of subsets of (0, 1] geerate B((0, 1])? The aswer is may, icludig some that are easier to describe tha F. I what follows we give a example of such a alterative collectio, pricipally i order to illustrate a geeral method of provig that two give collectios geerate the same σ-field. We claim that the σ-field geerated by the collectio { } G = A : A = (0, a), a < 1, is B((0, 1]), i.e., that σ(g) = σ(f). To prove the above equality, we must prove each of the iclusios σ(g) σ(f) ad σ(f) σ(g). For the former iclusio, it is sufficiet to show that G is cotaied i σ(f). This is because ay σ-field that cotais G will, by defiitio of σ( ), also cotai σ(g). The same argumet ca be made with F ad G iterchaged, ad thus we coclude that G σ(f) ad F σ(g) σ(g) = σ(f). To prove G σ(f): By the method give uder Fields ad coutable uios (Sectio 5), we ca write a arbitrary iterval (0, a) i G as (0, a) = ( 0, (1 (i + 1) 1 )a ]. Each of the sets i the above uio lies i F. Thus (0, a) is expressible as a coutable uio of sets i F, ad hece lies i σ(f). To prove F σ(g): The empty set trivially lies i σ(g). It thus remais to prove that every fiite uio of semi-ope itervals (a i, b i ] lies i σ(g); this is equivalet to provig that every sigle semi-ope iterval (a, b] lies i σ(g). 15

17 We express (a, b] as (a, b] = (a, 1] (b, 1] = (a, 1] (b, 1] c, so that it suffices to show that every (a, 1] lies i σ(g). We ow write (a, 1] as (a, 1] = [ a + (1 a)i 1, 1 ]. Each of the sets i the above uio is a complemet of a set i G, ad hece lies i σ(g). Thus (a, 1] also lies i σ(g), ad the proof is complete. We emphasize agai that the choice of the alterative geeratig collectio G is ot uique; oe ca easily show that substitutio of the geeric set (0, a) by ay of the itervals (0, a], [a, b], etc., still yields the Borel field. More importatly, G ca be replaced by a (sub-)collectio of itervals (0, a) such that a is a ratioal umber (expressible as a ratio of itegers). Sice ay real umber ca be writte as a icreasig or decreasig sequece of ratioals, we ca easily adapt the above proof to suit the modified G by usig ratioal edpoits i the appropriate uios. Ad sice the set of ratioals is coutable, this implies that the Borel field ca be geerated by a coutable collectio of itervals. We ca ow justify our earlier statemet that σ-fields geerated by coutable collectios o ucoutable sample spaces are ot always described i terms of coutable partitios. We do so by otig that the Borel field cotais (amog others) all sets that cosist of sigle poits o the uit iterval; these sets aloe form a ucoutable partitio of that iterval. The Borel field of the etire real lie ca be defied i a similar fashio: ({ }) B(R) def = σ (, a] : a R. Here agai the choice of geeratig itervals is ot uique, ad ratioal edpoits are fully acceptable. We ca also defie the Borel field of a arbitrary subset Ω of the real lie by B(Ω) def = {A : A = C Ω, C B(R)}. A iterestig exercise is to prove that i the case of the uit iterval, the above defiitio of B((0, 1]) is cosistet with the oe give origially. 7. Defiitio of the evet space Billigsley, Sec. 4, Limit sets. As we argued i Sectio 4, it is desirable that every evet space cotai ad Ω, ad be closed uder complemetatio ad fiite uios. Thus every evet space should at least be a field. That it should also be a σ-field is ot so obvious. The axiom of 16

18 closure uder coutable uios implies the followig: if we have a sequece of observatio mechaisms M 1, M 2,... (where M i observes the occurece of evet A i ), the we ca effectively combie these mechaisms ito oe that will decide whether the uio occurred or ot. This implicatio is ot always true i practice, as the example give i this sectio illustrates. Despite this shortcomig, the structure of the σ-field is chose for the evet space because it allows us to use the powerful mathematical machiery associated with the probability measure (which will be formally defied i the followig sectio). For the remaider of this course, the evet space F will always be a σ-field. Cosider ow the followig example which ivolves a ifiite sequece of evets A 1, A 2,... i a measurable space (Ω, F). We are iterested i descriptios of the set A of sample poits ω that lie i ifiitely may (but ot ecessarily all) of the sets A i. Thus A i A = {ω : ω A i for ifiitely may i}. I derivig a alterative descriptio of A, we argue as follows. If a poit ω lies i fiitely may sets A i, the there exists a idex k such that ω does ot lie i ay of the sets A k, A k+1,.... Coversely, if a poit ω lies i ifiitely may A i s, the for every k that poit will lie i at least oe of the sets A k, A k+1,.... Thus A = {ω : ( k) ω at least oe of A k, A k+1,...} { = ω : ( k) ω } A i. i k Writig B k for i k A i, we have A = {ω : ( k) ω B k } = k 1 B k = k 1 A i. To show that A is a evet, i.e., A F, we argue as follows. Every B k is a evet, sice it ca be writte as a coutable uio of evets A i ; ad thus A, which is the itersectio of the B k s, is also a evet. (Remark. De Morga s law is true for arbitrary collectios of evets, ad thus closure uder coutable uios is equivalet to closure uder coutable itersectios.) The evet A is called the limit superior (lim sup) of the sequece A 1, A 2,..., ad is ofte described i words as A i occurs ifiitely ofte (i.o.). Thus def lim sup A i = {A i i.o.} def = i k 1 17 A i. i k i k

19 We ca thik of the above situatio i terms of a radom experimet whose actual outcome ω is ukow to us. Our iformatio about ω is limited to a sequece of partial observatios of the experimet: for every i we kow whether ω A i or ω A c i, i.e., whether the evet A i has occurred or ot. Sice the set A of outcomes is expressible i terms of the sequece A 1, A 2,..., it is reasoable to assume that we ca process our observatios so as to determie whether or ot ω A i.e., whether or ot ifiitely may of the evets A i have occurred. Ufortuately, this is easier said tha doe. Cosider for istace the case i which the observatios are made sequetially i discrete time. If we assume that the A i s are such that every itersectio of the form C i, (C i is either A i or A c i), i 1 is oempty, the we have o meas of determiig i fiite time whether ifiitely may of the A i s have occurred. Thus the set of outcomes A = lim sup i A i does ot correspod to ay real observatio of the experimet; it is a evet oly because F is a σ-field. Remark. I a similar maer we ca defie the limit iferior (lim if) of the sequece A 1, A 2,... as the evet that A i occurs evetually, or equivaletly, A i occurs for all but fiitely may values of i. It is easy to check that this defiitio is cosistet with the represetatio lim if A i = A i, i k 1 i k ad that (lim if i A i ) c = lim sup i A c i. 8. Probability measures Billigsley, Sec. 2, Probability Measures. Defiitio A probability measure o the measurable space (Ω, F) is a real-valued fuctio P defied o F that satisfies the followig axioms: (P1) Noegativity: ( A F) P (A) 0. (P2) Normalizatio: P (Ω) = 1. (P3) Coutable Additivity: if A 1, A 2,... are pairwise disjoit evets (i.e., A i A j = for i j), the ( ) P A i = P (A i ). Note that i (P3) above, the uio of the A i s lies i F by the assumptio that the evet space is a σ-field. (P1 3) are also kow as the Kolmogorov axioms. Simple properties 18

20 From (P1) ad (P2) we ca deduce that P is fiitely additive ad that the probability of the empty set is 0. Ideed, if A 1,..., A are pairwise disjoit evets, we ca write A i = A i, where A +1 = A +2 =... =. The sequece A 1, A 2,... still cosists of disjoit evets, so we ca apply (P3) to obtai ( P A i ) = P (A i ) + P (A i ). i=+1 The ifiite sum o the right-had side cosists of terms equal to P ( ), ad hece it will be equal to 0 if P ( ) = 0, ad + if P ( ) > 0. As the probability measure caot take ifiity as a value (it is assumed to be a real-valued fuctio), it must be that i=+1 P (A i ) = P ( ) = 0. Therefore ( P A i ) = P (A i ). We thus obtai (P4) P ( ) = 0; ad (P5) Fiite additivity: if A 1,..., A are pairwise disjoit evets, ( P A i ) = P (A i ). (Remark. We ca obtai (P4) without assumig that P (A) < by applyig (P3) to a sequece where A 1 = Ω ad the remaiig A i s equal to ((P1) ad (P2) are also eeded here). Havig obtaied (P4), we ca apply (P3) ad (P1) as before to obtai (P5).) (P6) For all A F, P (A) + P (A c ) = 1. This follows from (P2) ad (P5) sice A A c =, A A c = Ω. (P7) Mootoicity uder iclusio: B A P (B) P (A). This is because we ca write B as A (B A c ), which is a disjoit uio. Hece by (P5) ad (P1), P (B) = P (A) + P (B A c ) P (A). 19

21 I particular, Ω A for every evet A, so (P8) For all A F, P (A) 1. Notatio. The set differece A B or A \ B is defied by A B def = A \ B def = A B c. The symmetric set differece A B is defied by A B = B A = (A B c ) (B A c ). These operatios are illustrated i the figure below. A : A B 1: A\ B = A-B B 3: B\ A = B-A 1 3: A B = B A We say that a sequece of evets (A ) N is icreasig if A 1 A 2... ; it is decreasig if A 1 A For such sequeces (which are also called mootoe) we ca defie a limitig evet lim A as follows: if (A ) N is icreasig, lim A def = A ; =1 whereas if (A ) N is decreasig, lim A def = A. =1 For brevity we write A A ad A A (respectively), where A = lim A. Probability measures are cotiuous o mootoe sequeces of evets; i other words, P (lim A ) = lim P (A ). 20

22 To prove this for a icreasig sequece A 1, A 2,..., we geerate a sequece of pairwise disjoit evets B 1, B 2,... as follows: B 1 =A 1 B 2 =A 2 A 1. B =A A 1 A 1 =B 1 A 2 =B 1 B 2. A =B 1 B A 3 B 3 B 2 B 1 A 1 A 2 From the above costructio, it is easy to see that A i = B i. This is because ay ω that lies i oe of the A i s will also lie i oe of the B i s, ad vice versa. Hece ( ) ( ) P A i = P B i, ad sice the B i s are disjoit, coutable additivity gives ( P A i ) = P (B i ) = lim Now sice A is the uio of the first B i s, we also have P (B i ). P (A ) = P (B i ), 21

23 ad hece ( P A i ) = lim P (A ). The aalogous result for decreasig sequeces of sets follows easily. If A 1, A 2,... is decreasig, the A c 1, A c 2,... is icreasig; by De Morga s law ad the above result we the have ( P A i ) ( = P A c i) c = 1 lim P (Ac ) = 1 [1 lim P (A )] = lim P (A ). We have thus obtaied (P9) Mootoe cotiuity from below: If A A, the lim P (A ) = P (A). (P10) Mootoe cotiuity from above: If A A, the lim P (A ) = P (A). Remark. As we have see, (P9) ad (P10) follow directly from the Kolmogorov axioms (P1 3). It is ot difficult to show that uder the assumptio of oegativity (P1), the coutable additivity axiom (P3) is actually equivalet to the two axioms of fiite additivity (P5) ad mootoe cotiuity from below (P9) combied. Thus a alterative to the Kolmogorov axioms is the set of axioms cosistig of (P 1,2,5) ad either (P9) or (P10): Noegativity Noegativity Normalizatio Normalizatio Fiite additivity Coutable additivity Cotiuity from above or below Covex mixtures of probability measures Let P 1, P 2,... be probability measures o the same measurable space (Ω, F). We say that the set fuctio P is a covex mixture (or covex combiatio) of these measures if it ca be expressed as a weighted sum of the P i s with oegative weights that add to uity. I other words, for every A F, P (A) is defied as P (A) = λ i P i (A), where the real coefficiets λ i satisfy ( i) λ i 0, λ i = 1. Claim. P is a probability measure o (Ω, F). 22

24 Proof. Noegativity of P follows directly from that of the λ i s ad P i s. Normalizatio is also easily established via P (Ω) = λ i P i (Ω) = λ i = 1. To prove coutable additivity, cosider a sequece of disjoit evets A 1, A 2,.... We have ( P j=1 A j ) = ( ) λ i P i A j j=1 = λ i j=1 P i (A j ), where the last equality follows by coutable additivity of the measures P i. The iterated sum has positive summads, so we ca chage the order of summatio to obtai ( P j=1 A j ) Thus P is a probability measure. = j=1 9. Specificatio of probability measures λ i P i (A j ) = P (A j ). Billigsley, Sec. 3, Lebesgue measure o the uit iterval; Sec. 4. Discrete spaces As we saw i Sectio 5, a measurable space (Ω, F) is discrete if the σ-field F is geerated by a coutable partitio of Ω ito atoms C 1, C 2,.... The for every evet A there exists a idex set I N such that A = i I C i, j=1 ad if P is a probability measure o (Ω, F), we have P (A) = i I P (C i ). The above demostrates that i order to defie a probability measure P o (Ω, F), it suffices to specify the quatity p i = P (C i ) for every atomic evet C i. Clearly, the p i s satisfy ( i) p i 0, p i = 1. 23

25 That ay sequece (p i ) i N satisfyig this oegativity/ormalizatio coditio geerates a probability measure is ot difficult to see: if we let P (A) = i I p i with A ad I defied as before, the the set fuctio P is oegative ad such that P (Ω) = 1. To establish coutable additivity, we simply ote that disjoit evets ca be expressed as uios of cells over likewise disjoit idex sets. Defiitio. A probability mass fuctio (pmf) is a sequece of oegative umbers whose sum equals uity. I the cotext of a give discrete probability space, the pmf is the fuctio that assigs probability to each atomic evet. Example. Ω = {0, 1,...}, F = 2 Ω. It is easy to see i this case that F is geerated by the oe poit sets (or sigletos) {k}. To defie the measure P, we use the Poisso pmf: where k = 0, 1,.... We have p k λ λk = e k!, p k = e λ k=0 k=0 λ k k! = e λ e λ = 1 as required. If we let P {k} = p k, the the probability that the outcome of the experimet is odd is ( ) ( P {1, 3, 5,...} = e λ λ + λ3 3! + λ5 e 5! +... = e λ λ e λ ) = 1 e 2λ. 2 2 No-discrete spaces As we saw above, we ca cocisely specify a probability measure o a discrete space by quotig the probabilities of the atomic evets. This is ot possible i the case of a o-discrete space, sice the evet space is o loger geerated by a coutable partitio of Ω ad thus there is o coutable family of miimal evets. Yet a cocise specificatio of measures o o-discrete spaces is absolutely ecessary if such spaces are to be used i probability models. Without such specificatio, the task of explicitly defiig a set fuctio o all (ucoutably may) evets ad subsequetly testig it for the Kolmogorov axioms becomes impracticable. A method ofte used for defiig a probability measure o a o-discrete σ-field F ivolves the costructio of a prelimiary set fuctio Q o a field F 0 that geerates F. If the fuctio Q, as costructed o F 0, satisfies certai coditios similar (but ot idetical) to the Kolmogorov axioms, the it is possible to exted Q to a uique probability 24

26 measure P o F = σ(f 0 ). Thus uder these coditios, specificatio of Q o F 0 suffices to determie the probability measure P o F without ambiguity. (Oe word of cautio: that Q uiquely determies P does ot imply that for a arbitrary evet A / F 0 oe ca easily compute P (A) based o the values take by Q; i may cases, this computatio will be highly complex or eve ifeasible.) The above method of defiig measures is based o the followig theorem, whose proof ca be foud i Billigsley, Sectio 3. Theorem. Let F 0 be a field of subsets of Ω, ad let Q be a oegative coutably additive set fuctio o F 0 such that Q(Ω) = 1. The there exists a uique probability measure o σ(f 0 ) such that P Q o F 0. Remark. As F o will ot i geeral be closed uder coutable uios, the statemet Q is coutably additive o F 0 is uderstood as if A 1, A 2,... lie i F 0 ad i A i also lies i F 0, the ( ) Q A i = P (A i ). i i The Lebesgue measure o the uit iterval To illustrate the extesio techique outlied i the previous subsectio, we briefly cosider the problem of defiig a probability measure P o the Borel field of the uit iterval such that P (a, b] = b a for every (a, b] (0, 1]. We first eed to idetify a field F 0 that geerates the Borel field, i.e., such that σ(f 0 ) = B((0, 1]). As we saw i Sectio 6, oe such choice of F 0 cosists of the empty set ad all fiite disjoit uios of semi-ope itervals N (c i, d i ], where 0 c 1 < d 1 < c 2 < d 2 <... < c N < d N 1. Sice we would like the probability of ay semi-ope iterval to be equal to its legth, we must defie the set fuctio Q o F 0 by ( N ) N Q( ) = 0, (c i, d i ] = (d i c i ). The set fuctio Q is clearly oegative ad satisfies Q(0, 1] = 1. It is ot difficult to show that Q is fiitely additive: if two sets i F 0 are disjoit, the their costituet itervals are o-overlappig. Coutable additivity of Q o F 0 ca be also established, but the proof is slightly more ivolved (see Billigsley, Sec. 2, Lebesgue measure o the uit iterval). By the extesio theorem of the previous subsectio, there exists a uique probability measure P o σ(f 0 ) = B((0, 1]) such that P = Q o F 0. P is called the Lebesgue 25

27 measure o the uit iterval. It is the oly probability measure o B((0, 1]) that assigs to every semi-ope iterval a probability equal to its legth. What is the probability of a sigleto {x} uder the Lebesgue measure? Ituitively, it should be zero (a poit has o legth). This is easily verified by writig {x} as the limit of a decresig sequece of semi-ope itervals, {x} = ( (1 (i + 1) 1 )x, x ], ad ivokig mootoe cotiuity from above: P {x} = lim P ( (1 ( + 1) 1 )x, x ] = lim ( + 1) 1 x = 0. Thus the Lebesgue measure of ay iterval (whether ope, semi-ope or closed) equals the iterval legth. Questio. Do coutable subsets of the uit iterval (e.g., the set of ratioals i that iterval) lie i its Borel field? What is the Lebesgue measure of such sets? 10. Defiitio of radom variable Billigsley, Sec. 5, Defiitio; Sec. 13, Measurable Mappigs, Mappigs ito R k. Prelimiaries I this sectio we cosider real valued fuctios f defied o a sample space Ω, i.e., f : Ω R, We recall the defiitio of the image of a set A Ω uder f as f(a) def = {x R : ( ω A) f(ω) = x} ; the iverse image of a set H R uder f is defied by f 1 (H) def = {ω Ω : f(ω) H}. I developig the cocept of the radom variable, we will employ images ad iverse images (i particular) extesively. The followig simple property will be quite useful: if {H i, i I} is a arbitrary collectio of subsets of R, the ( ) f 1 H i = f 1 (H i ). i I i I To see this, let ω lie i the iverse image of the uio. The f(ω) H i for some i = i(ω), ad thus ω f 1 (H i ). Coversely, if ω lies i the uio of the iverse images, the f(ω ) H j for some j = j(ω ), ad thus f(ω ) also lies i the uio of all H s. 26

28 By similar reasoig, we ca show that ( ) f 1 H i = f 1 (H i ), i I i I ad for (forward) images, ( ) f A i i I = ( f(a i ), f i I i I A i ) i I f(a i ). Defiitio A radom variable (r.v.) o a measurable space (Ω, F) is a real-valued fuctio X = X( ) o Ω such that for all a R, the set lies i F (i.e., is a evet). X 1 (, a] = {ω : X(ω) a} We ca thik of X(ω) as the result of a measuremet take i the course of a radom experimet: if the outcome of the experimet is ω, we obtai a readig X(ω) which provides partial iformatio about ω. A more precise iterpretatio will be give i Sectio 11. For the momet, we cosider some simple examples of radom variables. Examples of radom variables (i) Let (Ω, F) be arbitrary, ad A F. The idicator fuctio I A (.) of the evet A is defied by { 1, if ω A; I A (ω) = 0, if ω A c. We illustrate this defiitio i the figure below, where for the sake of simplicity, the sample space Ω is represeted by a iterval o the real lie. 1 R I A (.) 0 A Ω To see that X = I A defies a radom variable o (Ω, F), ote that as a rages over the real lie, the set X 1 (, a] = {ω : X(ω) a} is give by {, if a < 0; X 1 = A c if 0 a < 1; Ω if a 1. 27

29 Thus X 1 (, a] is always a evet, ad hece X = I A is a r.v. by the above defiitio. If B / F, the also B c / F, ad thus I B is ot a r.v. (ii) (Ω, F) is arbitrary, E 1, E 2,... is a coutable partitio of Ω, ad (c j ) j N is a sequece of distict real umbers. We defie X( ) as the fuctio that takes the costat value c j o every evet E j. I terms of idicator fuctios, X(ω) = c j I Ej (ω). j=1 R c 3 X(.) c 1 0 E 1 E 2 E 3... Ω c 2 We the have for a R X 1 (, a] = j J a E j, where J a = {j : a c j }. As the above iverse image is a coutable uio of evets, it is itself a evet ad thus X is a r.v. o (Ω, F). Remark. X as defied above takes the geeral form of a discrete radom variable o the measurable space (Ω, F). Thus a discrete r.v. is oe whose rage is coutable. A simple radom variable is oe whose rage is fiite. (iii) Let (Ω, F) = (R, B(R)) ad X( ) be a cotiuous icreasig real-valued fuctio such that lim X(ω) =. lim X(ω) = +. ω ω + R a X(.) X -1 (a) Ω=R 28

30 The X 1 (, a] = (, X 1 (a)]. Sice every iterval lies i B(R), X is a r.v. Remark. There are may possible variatios o this example. If we merely assume that X is odecreasig (as opposed to strictly icreasig) the iverse images are agai of the form (, b] for suitable values of b. If we remove the costrait that the limits as ω ± be ifiite, we admit ad R as possible iverse images. Fially, if we drop the cotiuity assumptio, the the iverse images will take the form (, b] or (, b). I all these cases, X remais a r.v. o (Ω, F). 11. The sigma field geerated by a radom variable Billigsley, Sec. 5, Defiitio; Sec. 13, Measurable Mappigs, Mappigs ito R k ; Sec. 20, Subfields. As oted i the previous sectio, we ca thik of the radom variable X as a measuremet take i the course of the radom experimet (Ω, F, P ). Thus if the outcome of the experimet is ω, we ca observe X(ω) with the aid of a measurig istrumet M X. I itroducig the importat cocept of the σ-field geerated by X, we will assume that the istrumet M X ca be set i a ifiity of observatio modes idexed by the real umbers a ad deoted by M X,a. I mode M X,a, the istrumet determies whether X(ω) a or X(ω) > a; that is, it decides which of the complemetary evets X 1 (, a] = {ω : X(ω) a} ad X 1 (a, ) = {ω : X(ω) > a} occurs. (Note that the above subsets of Ω are evets by defiitio of the radom variable; this is also cosistet with our earlier iterpretatio of a evet as a set of outcomes whose occurrece ca be determied.) The class of evets observable by M X is ot limited to iverse images of itervals. As i our earlier discussio of the evet space, we may assume that this class cotais the empty set ad the sample space, ad is closed uder complemetatio ad coutable uios, i.e., it is a σ-field. Sice the omial evets observable by M X are the iverse images of the itervals (, a], it is reasoable to postulate that the σ-field associated with the istrumet M X is the smallest σ-field that cotais these evets. Defiitio. Let X be a radom variable o (Ω, F, P ). We deote by σ(x) the σ-field geerated by the evets {ω : X(ω) a} as a varies over the real lie. Thus ({ }) σ(x) def = σ X 1 (, a] : a R. σ(x) is referred to as the σ-field geerated by X. Corollary. σ(x) F. To see this, ote that the geeratig collectio { } G = X 1 (, a] : a R 29

31 is cotaied i F. Thus σ(x) = σ(g) F. Examples Cosider the three examples of the previous sectio. (i) (Ω, F) arbitrary, A F, X = I A. We have see that the iverse image X 1 (, a] is oe of the three sets, A c, Ω. Thus σ(x) = σ({, A c, Ω}) = {, Ω, A, A c }, ad I A allows us to determie the occurrece of a sigle otrivial evet, amely A. (ii) (Ω, F) is agai arbitrary, ad X = c j I Ej, j=1 where the c j s are distict ad the evets E j form a coutable partitio of Ω. I this case the iverse images satisfy X 1 (, a] = j J a E j for J a = {j : a c j }. Thus every iverse image lies i the σ-field geerated by the partitio {E 1, E 2,...}, ad hece σ(x) σ({e 1, E 2,...}). We claim that the reverse iclusio is also true. To show this, it suffices to prove that every atom E j of the partitio lies i σ(x). Notig that (by distictess of the c j s) E j = X 1 {c j }, ad that {c j } = (, c j ] (, c j 1 ], =1 we obtai E j = X 1 (, c j ] X 1 (, c j 1 ]. =1 Sice the above expressio ivolves coutably may operatios o iverse images of itervals (, a], we have that E j σ(x). Hece we coclude that σ(x) = σ({e 1, E 2,...}). Thus the σ-field of a discrete r.v. is the σ-field geerated by the correspodig coutable partitio. 30

32 (iii) (Ω, F) = (R, B(R)), ad X( ) is cotiuous, strictly icreasig ad ubouded both from above ad below. I this case a iverse fuctio X 1 exists, ad its rage equals R. The X 1 (, a] = (, X 1 (a)] ad sice X 1 (a) takes all possible real values, we have { } X 1 (, a] : a R = { } (, a] : a R. As the collectio o the r.h.s. geerates the Borel field of the real lie, we have σ(x) = B(R) = F. Thus the r.v. X is completely iformative about the uderlyig experimet: the occurrece or ot of ay evet ca be determied usig X. Remark. As it turs out, the above result is true eve without assumig that X( ) is cotiuous ad ubouded. However, it is essetial that X( ) be strictly icreasig. If it were ot, e.g., if X( ) were costat over a iterval (c, d), the the istrumet M X would ot be able to distiguish betwee outcomes lyig i that iterval. Thus o proper subset of (c, d) could lie i σ(x), ad σ(x) B(R). A alterative, somewhat more direct, represetatio of σ(x) exists by virtue of the followig theorem. Theorem. The σ-field geerated by a radom variable is the collectio of all iverse images of Borel sets o the real lie. Thus if X is a r.v. o (Ω, F, P ), σ(x) = {X 1 (H) : H B(R)}. To prove this theorem, we use the followig simple lemma. Lemma. Let Ω be arbitrary, ad f a real-valued fuctio o Ω. The (i) if B is a σ-field of subsets of R, the collectio {f 1 (H) : H B} is a σ-field of subsets of Ω; (ii) if A is a σ-field of subsets of Ω, the collectio {H R : f 1 (H) A} is a σ-field of subsets of R. The proof of the lemma is straightforward ad is left as a exercise. The idetity f 1 ( i H i) = i f 1 (H i ) is useful i establishig closure uder coutable uios. Proof of Theorem. For coveiece let L = {X 1 (H) : H B(R)}. L σ(x): Sice B(R) is a σ-field of subsets of R, the collectio L is a σ-field of subsets of Ω by statemet (i) of the above lemma. Sice all itervals lie i B(R), we have that { } L X 1 (, a] : a R, 31