RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS FOR DEPENDENT DATA
|
|
- Dominick Scott
- 5 years ago
- Views:
Transcription
1 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS FOR DEPENDENT DATA PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO Abstract. This paper deals with empirical processes of the type C B = { µ B P X +1 B X 1,..., X }, where X is a sequece of radom variables ad µ = 1/ i=1 δ X i the empirical measure. Coditios for sup B C B to coverge stably i particular, i distributio are give, where B rages over a suitable class of measurable sets. These coditios apply whe X is exchageable, or, more geerally, coditioally idetically distributed i the sese of [6]. By such coditios, i some relevat situatios, oe obtais that sup B C B P 0, or eve that sup B C B coverges a.s.. Results of this type are useful i Bayesia statistics. 1. Itroductio ad motivatios A umber of real problems reduce to evaluate the predictive distributio a = P X +1 X 1,..., X for a sequece X 1, X 2,... of radom variables. Here, we focus o those situatios where a ca ot be calculated i closed form, ad oe decides to estimate it basig o the available data X 1,..., X. Related refereces are [1], [2], [3], [5], [6], [8], [10], [15], [18], [20]. For otatioal reasos, it is coveiet to work i the coordiate probability space. Accordigly, we fix a measurable space S, B, a probability P o S, B, ad we let X be the -th caoical projectio o S, B, P, 1. We also let G = σx 1,..., X ad X = X 1, X 2,.... Sice we are cocered with predictive distributios, it is reasoable to make some qualitative assumptios o them. I [6], X is said to be coditioally idetically distributed c.i.d. i case E I B X k G = E IB X +1 G, a.s., for all B B ad k > 0, where G 0 is the trivial σ-field. Thus, at each time 0, the future observatios X k : k > are idetically distributed give the past G. I a sese, this is a weak form of exchageability. I fact, X is exchageable if ad oly if it is statioary ad c.i.d., ad various examples of o exchageable c.i.d. sequeces are available. I the sequel, X = X 1, X 2,... is a c.i.d. sequece of radom variables Mathematics Subject Classificatio. 60G09, 60B10, 60A10, 62F15. Key words ad phrases. Bayesia predictive iferece Cetral limit theorem Coditioal idetity i distributio Empirical distributio Exchageability Predictive distributio Stable covergece. 1
2 2 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO I that case, a soud estimate of a is the empirical distributio µ = 1 δ Xi. The choice of µ ca be defeded as follows. Let D B ad let deote the suporm o D. Suppose also that D is coutably determied, as defied i Sectio 2. The latter is a mild coditio, oly eeded to hadle measurability issues. The, i=1 µ a = sup µ B a B a.s. 0 1 B D provided X is c.i.d. ad µ coverges uiformly o D with probability 1; see [5]. For istace, µ a a.s. 0 wheever X is exchageable ad D a Gliveko- Catelli class. Or else, µ a a.s. 0 if S = R, D = {, t] : t R}, ad X 1 has a discrete distributio or if ɛ>0 lim if P X +1 X < ɛ = 0; see [4]. To sum up, uder mild assumptios, µ is a cosistet estimate of a with respect to uiform distace for c.i.d. data. This is i lie with de Fietti [10] i the particular case of exchageable idicators. Takig 1 as a startig poit, the ext step is to ivestigate the covergece rate. That is, to ivestigate whether α µ a coverges i distributio, possibly to a ull limit, for suitable costats α > 0. This is precisely the purpose of this paper. A first piece of iformatio o the covergece rate of µ a ca be gaied as follows. For B B, defie µb = lim sup µ B, W B = {µ B µb}. By the SLLN for c.i.d. sequeces, µ B a.s. µb; see [6]. Hece, for fixed 0 ad B B, oe obtais E µb G = limk E µ k B G = limk 1 k k i=+1 = E I B X +1 G = a B a.s.. E I B X i G I tur, this implies { µ B a B } = EW B G a.s., so that µ a 1 sup E 1 W B G E W G a.s.. B D If sup E W k < for some k 1, it follows that E { α µ a k} α k E W k 0 wheever α 0. Eve if obvious, this fact is potetially useful, as sup E W k < for all k 1, if X is exchageable, 2 for various choices of D; see Remark 3. I particular, 2 holds if D is fiite.
3 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS 3 The itriguig case, however, is α =. For each B B ad probability Q o S, B, write C Q B = E Q W B G ad C B = C P B = { µ B a B }. I Theorem 3.3 of [6], the asymptotic behaviour of C B is ivestigated for fixed B. Here, istead, we are iterested i C = sup C B = µ a. B D Our mai result Theorem 1 is the followig. Fix a radom probability measure N o R ad a probability Q o S, B such that The, C Q N stably uder Q ad W is uiformly itegrable uder both P ad Q. C N stably wheever P Q. 3 A remarkable particular case is N = δ 0. Suppose i fact that, for some Q, oe has C Q Q 0 ad W uiformly itegrable uder P ad Q. The, C P 0 wheever P Q. Stable covergece i the sese of Reyi is a stroger form of covergece i distributio. The defiitio is recalled i Sectio 2. I geeral, oe caot dispese with the uiform itegrability coditio. However, the latter is ofte true. For istace, W is uiformly itegrable uder P ad Q provided D meets 2 ad X is exchageable uder P ad Q. To make 3 cocrete, a large list of referece probabilities Q is eeded. Various examples are available i the Bayesia oparametrics framework; see e.g. [16] ad refereces therei. The most popular is perhaps the Ferguso-Dirichlet law, deoted by Q 0. If P = Q 0, the X is exchageable ad a B = α P X 1 B + µ B α + a.s. for some costat α > 0. Sice µ a α/ whe P = Q 0, somethig more tha C P 0 ca be expected i case P Q 0. Ideed, we prove that µ a = C coverges a.s. wheever P Q 0 with a desity satisfyig a certai coditio; see Theorem 2 ad Corollary 5. Oe more example should be metioed. Let X = Y, Z, where Z > 0 ad P α P Y 1 B + Y +1 B G = i=1 Z i I B Y i α + i=1 Z a.s. i for some costat α > 0. Uder some coditios, X is c.i.d. but ot ecessarily exchageable, W is uiformly itegrable ad C coverges stably. See Sectio 4. The above material takes a icer form whe the coditio P Q ca be give a simple characterizatio. This happes, for istace, if S = {x 1,..., x k, x k+1 } is
4 4 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO fiite, X exchageable ad P X 1 = x > 0 for all x S. The, P Q 0 for some choice of Q 0 if ad oly if µ{x1 },..., µ{x k } has a absolutely cotiuous distributio with respect to Lebesgue measure. I this particular case, however, a part of our results ca also be obtaied through Berstei - vo Mises theorem; see Sectio 3. Fially, we make two remarks. i If X is exchageable, our results apply to Bayesia predictive iferece. Suppose i fact S is Polish ad B the Borel σ-field, so that de Fietti s theorem applies. The, P is a uique mixture of product probabilities o B ad the mixig measure is called prior distributio i a Bayesia framework. Now, give Q, P Q is just a assumptio o the prior distributio. This is plai i the last example where S = {x 1,..., x k, x k+1 }. I Bayesia terms, such a example ca be summarized as follows. For a multiomial statistical model, C P 0 if the prior is absolutely cotiuous with respect to Lebesgue measure, ad C coverges a.s. if the prior desity satisfies a certai coditio. ii To our kowledge, there is o geeral represetatio for the predictive distributios of a exchageable sequece. Such a represetatio would be very useful. Eve if partially, results like 3 cotribute to fill the gap. As a example, for fixed B B, oe obtais a B = µ B + o P 1 as far as X is exchageable ad P Q for some Q such that C Q B Q 0 ad W B is uiformly itegrable. 2. Mai results A few defiitios eed to be recalled. Let T be a metric space, B T the Borel σ-field o T ad Ω, A, P a probability space. A radom probability measure o T is a mappig N o Ω B T such that: i Nω, is a probability o B T for each ω Ω; ii N, B is A-measurable for each B B T. Let Z be a sequece of T -valued radom variables ad N a radom probability measure o T. Both Z ad N are defied o Ω, A, P. Say that Z coverges stably to N i case P Z H E N H weakly for all H A such that P H > 0. Clearly, if Z N stably, the Z coverges i distributio to the probability law E N just let H = Ω. Stable covergece has bee itroduced by Reyi i [17] ad subsequetly ivestigated by various authors. See [9] for more iformatio. Next, say that D B is coutably determied i case, for some fixed coutable subclass D 0 D, oe obtais sup B D0 ν 1 B ν 2 B = sup B D ν 1 B ν 2 B for every couple ν 1, ν 2 of probabilities o B. A sufficiet coditio is that, for some coutable D 0 D, ad for every ɛ > 0, B D ad probability ν o B, there is B 0 D 0 satisfyig ν B B 0 < ɛ. Most classes D ivolved i applicatios are coutably determied. For istace, D = {, t] : t R k } ad D = {closed balls} are coutably determied if S = R k ad B the Borel σ-field. Or else, D = B is coutably determied if B is coutably geerated.
5 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS 5 We are ow i a positio to state our mai result. Let N be a radom probability measure o R, defied o the measurable space S, B, ad let Q be a probability o S, B. Theorem 1. Let D be coutably determied. Suppose C Q N stably uder Q, ad W : 1 is uiformly itegrable uder P ad Q. The, C = µ a N stably wheever P Q. Proof. Sice D is coutably determied, there are o measurability problems i takig sup B D. I particular, W ad C are radom variables ad C is G -measurable. Let f be a versio of dp dq ad U = f E Q f G. The, C B = E W B G = E Q f W B G Lettig M = E Q E Q f G = C Q B + E Q U W B G, P -a.s., for each B B. E Q f G U W G E Q f G ad takig sup B D, it follows that C Q M C C Q + M, P -a.s.. We first assume f bouded. Sice C Q N stably uder Q, give a bouded radom variable Z o S, B, oe obtais φ C Q Z dq Nφ Z dq, for each bouded cotiuous φ : R R, where Nφ = φx N, dx. Lettig Z = f I H /P H, with H B ad P H > 0, it follows that C Q N stably uder P. Therefore, it suffices to prove EM 0. Give ɛ > 0, sice W is uiformly itegrable uder Q, there is c > 0 such that { } ɛ E Q W I { W >c} < for all. sup f Sice M is G -measurable, EM = E Q f M = EQ EQ f G M = EQ U W c E Q U + sup f E Q W I { W >c} < c E Q U + ɛ for all. Therefore, the martigale covergece theorem implies lim sup EM c lim sup E Q U + ɛ = ɛ. This cocludes the proof whe f is bouded. Next, let f be ay desity. Fix k > 0 such that P f k > 0 ad defie K = {f k} ad P K = P K. The, P K has the bouded desity f I K /P K with respect to Q. By what already proved, C P k N stably uder P K, where C P E { } I k K W B G B = E PK W B G =, P K -a.s.. EI K G
6 6 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO Lettig R = I K EI K G, it follows that E { I K C C P k } = E { I K sup E{ } R W B G } B D EI K G E { E { } R W G } { I K = E R W } EI K G c E R + E { W I { W >c}} for all c > 0. Sice E R 0 ad W is uiformly itegrable uder P, arguig as above implies E PK C C P k E{ I K C C P k } 0. P K Therefore, C N stably uder P K. Fially, fix H B, P H > 0, ad a bouded cotiuous fuctio φ : R R. The P H K = P H {f k} > 0, for k large eough, ad 2 sup φ P f > k + P H E φ C H E Nφ H E φ C H K E Nφ H K. Sice E φ C H K E Nφ H K as, ad P f > k 0 as k, this cocludes the proof. We ext deal with the particular case Q = Q 0, where Q 0 is a Ferguso-Dirichlet law o S, B. If P Q 0 with a desity satisfyig a certai coditio, the covergece rate of µ a ca be remarkably improved. Theorem 2. Suppose D is coutably determied ad sup E Q0 W 2 <. The, C = µ a coverges a.s. provided P Q 0 ad E Q0 f 2 E Q0 { EQ0 f G 2} = O 1, for some versio f of dp dq 0. Proof. Let D B = C B. The, D is G -measurable as D is coutably determied ad E +1 D +1 G = E sup I B X i + 1EµB G +1 G B D i=1 sup E +1 I B X i G + 1E µb G B D i=1 = sup I B X i EµB G = D B D i=1 Sice D is a G -submartigale, it suffices to prove that sup E D <. Let U = f E 0 f G, where E 0 stads for E Q0. By assumptio, there are c 1, c 2 > 0 such that a.s.. E 0 W 2 c 1, E 0 U 2 = { E 0 f 2 E 0 E 0 f G 2 } c 2 for all.
7 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS 7 As oted i Sectio 1, sice Q 0 is a Ferguso-Dirichlet law, there is α > 0 such that C Q 0 = sup E 0 W B G α for all. B D Defie M = E0 U W G ad recall that C C Q0 + M, P -a.s.; see the E 0 f G proof of Theorem 1. The, for all, oe obtais E D = E C E C Q0 + EM α + E0 f M = α + E 0 U W α + E 0 U 2 E 0 W 2 α + c 1 E 0 U 2 α + c 1 c 2. Fially, we specify a poit raised i Sectio 1. Remark 3. There is a log list of coutably determied choices of D such that sup E W k ck, for all k 1, if X is i.i.d., where ck is some uiversal costat; see e.g. Subsectios ad of [21]. Fix oe such D, k 1, ad suppose S is Polish ad B the Borel σ-field. If X is exchageable, de Fietti s theorem yields E W k T ck a.s. for all, where T is the tail σ-field of X. Hece, E W k = E { E W k T } ck for all. This proves iequality Exchageable data with fiite state space Whe X is exchageable ad S fiite, there is some overlappig betwee Theorem 1 ad a result of Berstei ad vo Mises Coectios with Berstei - vo Mises theorem. For each θ i a ope set Θ R k, let P θ be a product probability o S, B that is, X is i.i.d. uder P θ. Suppose the map θ P θ B is Borel measurable for fixed B B. Give a prior probability π o the Borel subsets of Θ, defie P B = P θ B πdθ, B B. Roughly speakig, Berstei - vo Mises BVM theorem ca be stated as follows. Suppose π is absolutely cotiuous with respect to Lebesgue measure ad the statistical model P θ : θ Θ is suitably smooth we refer to [13] for a detailed expositio of what smooth meas. For each, suppose θ admits a cosistet maximum likelihood estimator θ. Further, suppose the prior π possesses the first momet ad deote θ the posterior mea of θ. The, θ θ P θ0 0 for each θ 0 Θ such that the desity of π is strictly positive ad cotiuous at θ 0. Actually, BVM-theorem yields much more tha asserted, what reported above beig just the corollary coected to this paper. We refer to [13] ad [14] for more iformatio ad historical otes. See also [18].
8 8 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO Assumig a smooth, fiite-dimesioal statistical model is fudametal; see e.g. [11]. Ideed, BVM-theorem does ot apply whe the oly iformatio is X exchageable or eve c.i.d. ad P Q for some referece probability Q. Oe exceptio, however, is S fiite. Let us suppose S = {x 1,..., x k, x k+1 }, X exchageable, P X 1 = x > 0 for all x S, ad D = B = power set of S. Also, let λ deote Lebesgue measure o R k ad π the probability distributio of θ = µ{x 1 },..., µ{x k }. As oted i Sectio 1, π λ if ad oly if P Q 0 for some choice of Q 0. Sice D is fiite ad X exchageable uder P ad Q 0, the W is uiformly itegrable uder P ad Q 0. Thus, Theorem 1 yields C P 0 wheever π λ. O the other had, π is the prior distributio for this problem. The uderlyig statistical model is smooth ad fiite-dimesioal it is just a multiomial model. Further, for each, the maximum likelihood estimator ad the posterior mea of θ are, respectively, θ = µ {x 1 },..., µ {x k }, θ = a {x 1 },..., a {x k }. Thus, BVM-theorem implies C P 0 as far as π λ ad the desity of π is cotiuous o the complemet of a π-ull set. To sum up, i this particular case, the same coclusios as Theorem 1 ca be draw from BVM-theorem. Ulike the latter, however, Theorem 1 does ot require ay coditio o the desity of π Some cosequeces of Theorems 1 ad 2. I this subsectio, we focus o S = {0, 1}. Thus, D = B = power set of S ad λ is Lebesgue measure o R. Let N 0, a deote the oe-dimesioal Gaussia law with mea 0 ad variace a 0 where N 0, 0 = δ 0. Our first result allows π to have a discrete part. Corollary 4. With S = {0, 1}, let π be the probability distributio of µ{1} ad = {θ [0, 1] : π{θ} > 0}, Defie the radom probability measure N o R as A = {ω S : µω, {1} }. N = 1 I A δ 0 + I A N 0, µ{1}1 µ{1}. If X is exchageable ad π does ot have a sigular cotiuous part, the C {1} N stably ad C N h 1 stably where hx = x, x R, is the modulus fuctio. Proof. By stadard argumets, the Corollary holds whe π 0, 1 provided it holds whe π = 0 ad π = 1. Let π = 0. The π λ, as π does ot have a sigular cotiuous part, ad the Corollary follows from Theorem 1. Thus, it ca be assumed π = 1. Sice C {0} = C {1}, C = C {1} ad the modulus fuctio is cotiuous, it suffices to prove that C {1} N stably. Next, exchageability of X implies W {1} N 0, µ{1}1 µ{1} stably; see e.g. Theorem 3.1 of [6]. Sice π = 1, the N = N 0, µ{1}1 µ{1} a.s.. Hece, it is eough to show that E C {1} W {1} 0.
9 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS 9 Fix ɛ > 0 ad let M = W {1}. Sice X is exchageable, M is uiformly itegrable. Therefore, there is c > 0 such that sup E ɛ M I { M >c} < 4. Defie φx = x if x c, φx = c if x > c, ad φx = c if x < c. Sice C {1} = EM G a.s., it follows that E C {1} W {1} E EM G EφM G + +E EφM G φm + E φm M E EφM G φm + 4 E M I { M >c} < E EφM G φm + ɛ for all. Write = {a 1, a 2,...} ad M,j = µ {1} a j. Sice σm,j G ad P µ{1} = π = 1, oe also obtais E EφM G φm = E E φm,j I {µ{1}=aj} G φm,j I {µ{1}=aj} j = E φm,j { } P µ{1} = a j G I {µ{1}=aj} j m c E P µ{1} = a j G I {µ{1}=aj} + 2 c π{a j } for all m,. j>m j=1 By the martigale covergece theorem, E P µ{1} = a j G I {µ{1}=aj} 0, as, for each j. Thus, lim sup E C {1} W {1} ɛ + 2 c π{a j } for all m. j>m Takig the limit as m cocludes the proof. If π is sigular cotiuous, we cojecture that C {1} coverges stably to a o ull limit. But we have ot a proof. I the ext result, a real fuctio g o 0, 1 is said to be almost Lipschitz i case x gxx a 1 x b is Lipschitz o 0, 1 for some reals a, b < 1. Corollary 5. Suppose S = {0, 1}, X is exchageable ad π is the probability distributio of µ{1}. If π admits a almost Lipschitz desity with respect to λ, the C coverges a.s. to a real radom variable. Proof. Let V = µ{1}. By assumptio, there are a, b < 1 ad a versio g of dπ dλ such that φθ = gθθ a 1 θ b is Lipschitz o 0, 1. For each u 1, u 2 > 0, we ca take Q 0 such that V has a beta-distributio with parameters u 1, u 2 uder Q 0. Let Q 0 be such that V has a beta-distributio with parameters u 1 = 1 a ad u 2 = 1 b uder Q 0. The, for ay 1 ad x 1,..., x {0, 1}, oe obtais P 1 X 1 = x 1,..., X = x = θ r 1 θ r πdθ = = c V r 1 V r φv dq 0 where r = θ r a 1 θ r b φθ dθ x i ad c > 0 is a costat. i=1
10 10 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO Let h = c φ. The, h is Lipschitz ad f = hv is a versio of dp dq 0. Let V = E 0 V G, where E 0 stads for E Q0. Sice h is Lipschitz, f E 0 f G hv hv + E 0 hv hv G d V V + d E 0 V V G where d is the Lipschitz costat of h. Sice E 0 C Q0 2 E 0 W 2 ad V V = C Q0 {1} W {1} C Q0 + W, it follows that E 0 f 2 E 0 E0 f G 2 = E 0 { f E0 f G 2} 4 d 2 E 0 { V V 2} 4 d2 E { 0 C Q 0 + W 2 } 16 d2 E 0 W 2. Sice sup E 0 W 2 <, the E 0 f 2 E 0 E0 f G 2 = O1/. A applicatio of Theorem 2 cocludes the proof. Corollaries 4 ad 5 deal with S = {0, 1} but similar results ca be proved for ay fiite S. See also [12] ad [19]. 4. Geeralized Polya urs I this sectio, basig o Examples 1.3 ad 3.5 of [6], the asymptotic behaviour of C is ivestigated for a certai c.i.d. sequece. Let Y, B Y be a measurable space, B + the Borel σ-field o 0, ad S = Y 0,, B = B Y B +, X = Y, Z, where Y ω = y, Z ω = z for all ω = y 1, z 1, y 2, z 2,... S. Give a law P o B, it is assumed that P Y +1 B G = α P Y 1 B + i=1 Z i I B Y i α + i=1 Z i a.s., 1, 4 P Z +1 C X 1,..., X, Y +1 = P Z1 C a.s., 0, 5 for some costat α > 0 ad all B B Y ad C B +. Note that Z is i.i.d. ad Z +1 is idepedet of Y 1, Z 1,..., Y, Z, Y +1 for all 0. I real problems, the Z should be viewed as weights while the Y describe the pheomeo of iterest. As a example, cosider a ur cotaiig white ad black balls. At each time 1, a ball is draw ad the replaced together with Z more balls of the same colour. Let Y be the idicator of the evet {white ball at time } ad suppose Z is chose accordig to a fixed distributio o the itegers, idepedetly of Y 1, Z 1,..., Y 1, Z 1, Y. The, the predictive distributios of X are give by 4-5. Note also that the probability law of Y is Ferguso- Dirichlet i case Z = 1 for all. It is ot hard to prove that X is c.i.d.. We state this fact as a lemma. Lemma 6. The sequece X assessed accordig to 4-5 is c.i.d.. Proof. Fix k > 0 ad A B Y B +. By a mootoe class argumet, it ca be assumed A = B C where B B Y ad C B +. Further, it ca be assumed
11 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS 11 k = + 2. Let = 0 ad G 0 the trivial σ-field. Sice X 2 X 1 as it is easily see, E I B Y 2 I C Z 2 G 0 = E IB Y 1 I C Z 1 G 0 a.s.. If 1, defie G = σx 1,..., X, Z +1. O otig that E I B Y +1 G = E IB Y +1 G a.s., oe obtais E I B Y +2 G { = E EIB Y +2 G +1 G } = α P Y 1 B + i=1 Z i I B Y i + Z +1 EI B Y +1 G α + +1 i=1 Z i = α + i=1 Z i EI B Y +1 G + Z +1 EI B Y +1 G α + +1 i=1 Z. i = EI B Y +1 G = EI B Y +1 G a.s.. Fially, sice G G, the previous equality implies E I B Y +2 I C Z +2 G = P Z1 C E { EI B Y +2 G G } = P Z 1 C E { EI B Y +1 G G } = E IB Y +1 I C Z +1 G Therefore, X is c.i.d.. Usually, oe is iterested i predictig Y more tha Z. Thus, i the sequel, we focus o P Y +1 B G. For each B B Y, we write C B = C B 0,, a B = a B 0, = P Y+1 B G, ad so o. I Example 3.5 of [6], assumig EZ 2 1 <, it is show that C B N 0, σ 2 B stably, where σ 2 B = varz 1 EZ 1 2 µb 1 µb. Here, we prove that C coverges stably whe regarded as a map C : S l D, where l D is the space of real bouded fuctios o D equipped with uiform distace; see Sectio 1.5 of [21]. I particular, stable covergece of C as a radom elemet of l D implies stable covergece of C = sup B D C B. Ituitively, the stable limit of C whe it exists is coected to Browia bridge. Let B 1, B 2,... be pairwise disjoit elemets of B Y ad k D = {B k 0, : k 1}, T 0 = 0, T k = µb i. Also, let G be a stadard Browia bridge process o some probability space Ω 0, A 0, P 0. For fixed ω S, varz1 { Lω, B k = GTk ω GT k 1 ω } EZ 1 is a real radom variable o Ω 0, A 0, P 0. Sice the B k are pairwise disjoit ad G has cotiuous paths, Lω, B k 0 as k. So, it makes sese to defie Mω, as the probability distributio of Lω = Lω, B 1, Lω, B 2,..., that is, Mω, A = P 0 Lω A i=1 for each Borel set A l D. Similarly, let Nω, be the probability distributio of sup k 1 Lω, B k, i.e., Nω, A = P 0 sup Lω, B k A for each Borel set A R. k 1 a.s..
12 12 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO Theorem 7. Suppose B 1, B 2,... B Y are pairwise disjoit ad D, M, N are defied as above. Let X be assessed accordig to 4-5 with a Z 1 b a.s. for some costats 0 < a < b. The sup E W 2 c P Y 1 k B k, 6 for some costat c idepedet of the B k, ad C M stably i the metric space l D. I particular, C N stably. Let Q 1 deote the probability law of a sequece X satisfyig 4-5 ad a Z 1 b a.s.. I view of Theorem 7, Q 1 ca play the role of Q i Theorem 1. That is, for a arbitrary c.i.d. sequece X with distributio P, oe has C N stably provided P Q 1 ad W is uiformly itegrable uder P. The coditio of pairwise disjoit B k is actually rather strog. However, it holds i at least two relevat situatios: whe a sigle set B is ivolved ad whe S = {x 1, x 2,...} is coutable ad B k = {x k } for all k. Proof of Theorem 7. This proof ivolves some simple but log calculatios. Accordigly, we just give a sketch of the proof ad we refer to [7] for details. Sice X is c.i.d., for fixed B B Y oe has a B = E µb G a.s.. Hece, a B : 1 is a G -martigale with a B a.s. µb, ad this implies E { a +1 B µb 2} = E { a j B a j+1 B 2} = E { a j B a j+1 B 2}. j> Replacig a j B by 4 ad usig that a Z i b a.s. for all i, a log but straightforward calculatio yields j> E{ a j B a j+1 B 2} c 1 P Y 1 B where c 1 is a costat idepedet of B. It follows that E a +1 µ 2 = E { sup a+1 B k µb k 2} E { a +1 B k µb k 2} k k = E { a j B k a j+1 B k 2} c 1 P Y 1 B k k j> k = c 1 P Y 1 k B k as the B k are pairwise disjoit. Precisely as above, after some algebra, oe obtais E µ a +1 2 c 2 P Y 1 k B k for some costat c 2 idepedet of B 1, B 2,.... Therefore, E W 2 = E µ µ 2 2 E µ a E a +1 µ 2 c P Y 1 k B k where c = 2 c 1 + c 2. This proves iequality 6. It remais to prove that C M stably i the metric space l D. For each m 1, let Σ m be the m m matrix with elemets j> σ k,j = varz 1 EZ 1 2 µbk B j µb k µb j, k, j = 1,..., m. By Theorems ad of [21], for C M stably, it is eough that i-fiite dimesioal covergece: C B 1,..., C B m N m 0, Σm stably for each m 1,
13 RATE OF CONVERGENCE OF PREDICTIVE DISTRIBUTIONS 13 where N m 0, Σm is the m-dimesioal Gaussia law with mea 0 ad covariace matrix Σ m ; ii-asymptotic tightess: For each ɛ, δ > 0, there is m 1 such that lim sup P sup C B r C B s > ɛ < δ. r,s>m Fix m 1, b 1,..., b m R, ad defie R = m k=1 b ki Bk Y. Sice R : 1 is c.i.d., arguig exactly as i Example 3.5 of [6], oe obtais m b k C B k = i=1{ Ri ER +1 G } N 0, b k b j σ k,j stably. k,j k=1 Sice b 1,..., b m are arbitrary, i holds. To check ii, give ɛ, δ > 0, take m such that P ɛ 2 δ 2 Y 1 r>m B r < 4 c where c is the costat ivolved i 6. By what already proved, P sup C B r C B s > ɛ P 2 sup C B r > ɛ r,s>m r>m P 2 E 4 sup W B r G > ɛ r>m ɛ 2 E{ sup W B r 2} r>m 4 c P Y 1 r>m B r < δ. Thus, ii holds, ad this cocludes the proof. ɛ 2 Ackowledgmets: This paper beefited from the helpful suggestios of two aoymous referees. Refereces [1] Algoet P.H Uiversal schemes for predictio, gamblig ad portfolio selectio, A. Probab., 20, [2] Algoet P.H Uiversal predictio schemes Correctio, A. Probab., 23, [3] Berti P. ad Rigo P A uiform limit theorem for predictive distributios, Statist. Probab. Letters, 56, [4] Berti P., Pratelli L. ad Rigo P Almost sure uiform covergece of empirical distributio fuctios, Iterat. Math. J., 2, [5] Berti P., Mattei A. ad Rigo P Uiform covergece of empirical ad predictive measures, Atti Sem. Mat. Fis. Uiv. Modea, L, [6] Berti P., Pratelli L. ad Rigo P Limit theorems for a class of idetically distributed radom variables, A. Probab., 32, [7] Berti P., Crimaldi I., Pratelli L. ad Rigo P Rate of covergece of predictive distributios for depedet data, Techical Report, available at: [8] Blackwell D. ad Dubis L.E.1962 Mergig of opiios with icreasig iformatio, A. Math. Statist., 33, [9] Crimaldi I., Letta G. ad Pratelli L A strog form of stable covergece, Semiaire de Probabilites XL, Lect. Notes i Math., 1899,
14 14 PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO [10] de Fietti B La previsio: ses lois logiques, ses sources subjectives, Aales Istit. Poicare, 7, [11] Freedma D O the Berstei - vo Mises theorem with ifiitedimesioal parameters, A. Statist., 27, [12] Ghosh J.K., Siha B.K. ad Joshi S.N Expasios for posterior probability ad itegrated Bayes risk, Statistical Decisio Theory ad Related Topics III, Vol. 1, Academic Press, [13] Ghosh J.K. ad Ramamoorthi R.V Bayesia oparametrics, Spriger. [14] Le Cam L. ad Yag G.L Asymptotics i statistics: some basic cocepts, Spriger. [15] Morvai G. ad Weiss B Forward estimatio for ergodic time series, A. Ist. H. Poicare Probab. Statist., 41, [16] Pitma J Some developmets of the Blackwell-MacQuee ur scheme, I: Statistics, Probability ad Game Theory Ferguso, Shapley ad Mac Quee Eds., IMS Lecture Notes Moogr. Ser., 30, [17] Reyi A O stable sequeces of evets, Sakhya A, 25, [18] Romaovsky V Sulle probabilita a posteriori, Giorale dell Istituto Italiao degli Attuari,. 4, [19] Strasser H Improved bouds for equivalece of Bayes ad maximum likelihood estimatio, Theor. Probab. Appl., 22, [20] Stute W O almost sure covergece of coditioal empirical distributio fuctios, A. Probab., 14, [21] va der Vaart A. ad Weller J.A Weak covergece ad empirical processes, Spriger. Patrizia Berti, Dipartimeto di Matematica Pura ed Applicata G. Vitali, Uiversita di Modea e Reggio-Emilia, via Campi 213/B, Modea, Italy address: patrizia.berti@uimore.it Iree Crimaldi, Dipartimeto di Matematica, Uiversita di Bologa, Piazza di Porta Sa Doato 5, Bologa, Italy address: crimaldi@dm.uibo.it Luca Pratelli, Accademia Navale, viale Italia 72, Livoro, Italy address: pratel@mail.dm.uipi.it Pietro Rigo correspodig author, Dipartimeto di Ecoomia Politica e Metodi Quatitativi, Uiversita di Pavia, via S. Felice 5, Pavia, Italy address: prigo@eco.uipv.it
Asymptotics of Predictive Distributions
Asymptotics of redictive Distributios atrizia Berti, Luca ratelli ad ietro Rigo Abstract Let (X ) be a sequece of radom variables, adapted to a filtratio (G ), ad let μ = (1/) i=1 δ X i ad a ( ) = (X +1
More informationA CENTRAL LIMIT THEOREM AND ITS APPLICATIONS TO MULTICOLOR RANDOMLY REINFORCED URNS
A CENTRAL LIMIT THEOREM AND ITS APPLICATIONS TO MULTICOLOR RANDOMLY REINFORCED URNS PATRIZIA BERTI, IRENE CRIMALDI, LUCA PRATELLI, AND PIETRO RIGO Abstract Let X be a sequece of itegrable real radom variables,
More informationQuaderni di Dipartimento. Exchangeable Sequences Driven by an Absolutely Continuous Random Measure
Quaderi di Dipartimeto Exchageable Sequeces Drive by a Absolutely Cotiuous Radom Measure Patrizia Berti (Uiversità di Modea e Reggio Emilia) Luca Pratelli (Accademia Navale di Livoro) Pietro Rigo (Uiversità
More informationQuaderni di Dipartimento. A Central Limit Theorem and Its Applications to Multicolor Randomly Reinforced Urns
Quaderi di Dipartimeto A Cetral Limit Theorem ad Its Applicatios to Multicolor Radomly Reiforced Urs Patrizia Berti Uiversità di Modea e Reggio Emilia Iree Crimaldi Uiversità di Bologa Luca Pratelli Accademia
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationAn alternative proof of a theorem of Aldous. concerning convergence in distribution for martingales.
A alterative proof of a theorem of Aldous cocerig covergece i distributio for martigales. Maurizio Pratelli Dipartimeto di Matematica, Uiversità di Pisa. Via Buoarroti 2. I-56127 Pisa, Italy e-mail: pratelli@dm.uipi.it
More informationASYMPTOTIC PREDICTIVE INFERENCE WITH EXCHANGEABLE DATA PATRIZIA BERTI, LUCA PRATELLI, AND PIETRO RIGO
ASYMTOTIC REDICTIVE INFERENCE WITH EXCHANGEABLE DATA ATRIZIA BERTI, LUCA RATELLI, AND IETRO RIGO Abstract. Let (X ) be a sequece of radom variables, adapted to a filtratio (G ), ad let µ = (1/) i=1 δ X
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationRates of Convergence by Moduli of Continuity
Rates of Covergece by Moduli of Cotiuity Joh Duchi: Notes for Statistics 300b March, 017 1 Itroductio I this ote, we give a presetatio showig the importace, ad relatioship betwee, the modulis of cotiuity
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationMeasure and Measurable Functions
3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationMi-Hwa Ko and Tae-Sung Kim
J. Korea Math. Soc. 42 2005), No. 5, pp. 949 957 ALMOST SURE CONVERGENCE FOR WEIGHTED SUMS OF NEGATIVELY ORTHANT DEPENDENT RANDOM VARIABLES Mi-Hwa Ko ad Tae-Sug Kim Abstract. For weighted sum of a sequece
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationON POINTWISE BINOMIAL APPROXIMATION
Iteratioal Joural of Pure ad Applied Mathematics Volume 71 No. 1 2011, 57-66 ON POINTWISE BINOMIAL APPROXIMATION BY w-functions K. Teerapabolar 1, P. Wogkasem 2 Departmet of Mathematics Faculty of Sciece
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationThe Borel hierarchy classifies subsets of the reals by their topological complexity. Another approach is to classify them by size.
Lecture 7: Measure ad Category The Borel hierarchy classifies subsets of the reals by their topological complexity. Aother approach is to classify them by size. Filters ad Ideals The most commo measure
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationA Note on the Kolmogorov-Feller Weak Law of Large Numbers
Joural of Mathematical Research with Applicatios Mar., 015, Vol. 35, No., pp. 3 8 DOI:10.3770/j.iss:095-651.015.0.013 Http://jmre.dlut.edu.c A Note o the Kolmogorov-Feller Weak Law of Large Numbers Yachu
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationRiesz-Fischer Sequences and Lower Frame Bounds
Zeitschrift für Aalysis ud ihre Aweduge Joural for Aalysis ad its Applicatios Volume 1 (00), No., 305 314 Riesz-Fischer Sequeces ad Lower Frame Bouds P. Casazza, O. Christese, S. Li ad A. Lider Abstract.
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationMAXIMAL INEQUALITIES AND STRONG LAW OF LARGE NUMBERS FOR AANA SEQUENCES
Commu Korea Math Soc 26 20, No, pp 5 6 DOI 0434/CKMS20265 MAXIMAL INEQUALITIES AND STRONG LAW OF LARGE NUMBERS FOR AANA SEQUENCES Wag Xueju, Hu Shuhe, Li Xiaoqi, ad Yag Wezhi Abstract Let {X, } be a sequece
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationRandom Walks on Discrete and Continuous Circles. by Jeffrey S. Rosenthal School of Mathematics, University of Minnesota, Minneapolis, MN, U.S.A.
Radom Walks o Discrete ad Cotiuous Circles by Jeffrey S. Rosethal School of Mathematics, Uiversity of Miesota, Mieapolis, MN, U.S.A. 55455 (Appeared i Joural of Applied Probability 30 (1993), 780 789.)
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationBerry-Esseen bounds for self-normalized martingales
Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,
More informationThe Boolean Ring of Intervals
MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationEstimation of the essential supremum of a regression function
Estimatio of the essetial supremum of a regressio fuctio Michael ohler, Adam rzyżak 2, ad Harro Walk 3 Fachbereich Mathematik, Techische Uiversität Darmstadt, Schlossgartestr. 7, 64289 Darmstadt, Germay,
More informationA survey on penalized empirical risk minimization Sara A. van de Geer
A survey o pealized empirical risk miimizatio Sara A. va de Geer We address the questio how to choose the pealty i empirical risk miimizatio. Roughly speakig, this pealty should be a good boud for the
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationComplete Convergence for Asymptotically Almost Negatively Associated Random Variables
Applied Mathematical Scieces, Vol. 12, 2018, o. 30, 1441-1452 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2018.810142 Complete Covergece for Asymptotically Almost Negatively Associated Radom
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More informationLecture 4. We also define the set of possible values for the random walk as the set of all x R d such that P(S n = x) > 0 for some n.
Radom Walks ad Browia Motio Tel Aviv Uiversity Sprig 20 Lecture date: Mar 2, 20 Lecture 4 Istructor: Ro Peled Scribe: Lira Rotem This lecture deals primarily with recurrece for geeral radom walks. We preset
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More informationSieve Estimators: Consistency and Rates of Convergence
EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes
More informationThe Wasserstein distances
The Wasserstei distaces March 20, 2011 This documet presets the proof of the mai results we proved o Wasserstei distaces themselves (ad ot o curves i the Wasserstei space). I particular, triagle iequality
More informationNotes on Snell Envelops and Examples
Notes o Sell Evelops ad Examples Example (Secretary Problem): Coside a pool of N cadidates whose qualificatios are represeted by ukow umbers {a > a 2 > > a N } from best to last. They are iterviewed sequetially
More informationDetailed proofs of Propositions 3.1 and 3.2
Detailed proofs of Propositios 3. ad 3. Proof of Propositio 3. NB: itegratio sets are geerally omitted for itegrals defied over a uit hypercube [0, s with ay s d. We first give four lemmas. The proof of
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationL = n i, i=1. dp p n 1
Exchageable sequeces ad probabilities for probabilities 1996; modified 98 5 21 to add material o mutual iformatio; modified 98 7 21 to add Heath-Sudderth proof of de Fietti represetatio; modified 99 11
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationA Weak Law of Large Numbers Under Weak Mixing
A Weak Law of Large Numbers Uder Weak Mixig Bruce E. Hase Uiversity of Wiscosi Jauary 209 Abstract This paper presets a ew weak law of large umbers (WLLN) for heterogeous depedet processes ad arrays. The
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationf n (x) f m (x) < ɛ/3 for all x A. By continuity of f n and f m we can find δ > 0 such that d(x, x 0 ) < δ implies that
Lecture 15 We have see that a sequece of cotiuous fuctios which is uiformly coverget produces a limit fuctio which is also cotiuous. We shall stregthe this result ow. Theorem 1 Let f : X R or (C) be a
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationCentral limit theorem and almost sure central limit theorem for the product of some partial sums
Proc. Idia Acad. Sci. Math. Sci. Vol. 8, No. 2, May 2008, pp. 289 294. Prited i Idia Cetral it theorem ad almost sure cetral it theorem for the product of some partial sums YU MIAO College of Mathematics
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationBIRKHOFF ERGODIC THEOREM
BIRKHOFF ERGODIC THEOREM Abstract. We will give a proof of the poitwise ergodic theorem, which was first proved by Birkhoff. May improvemets have bee made sice Birkhoff s orgial proof. The versio we give
More informationJournal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula
Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationEFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS
EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS Ryszard Zieliński Ist Math Polish Acad Sc POBox 21, 00-956 Warszawa 10, Polad e-mail: rziel@impagovpl ABSTRACT Weak laws of large umbers (W LLN), strog
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationHyun-Chull Kim and Tae-Sung Kim
Commu. Korea Math. Soc. 20 2005), No. 3, pp. 531 538 A CENTRAL LIMIT THEOREM FOR GENERAL WEIGHTED SUM OF LNQD RANDOM VARIABLES AND ITS APPLICATION Hyu-Chull Kim ad Tae-Sug Kim Abstract. I this paper we
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More information