STAT331. Example of Martingale CLT with Cox s Model
|
|
- Adela Harrington
- 6 years ago
- Views:
Transcription
1 STAT33 Example of Martigale CLT with Cox s Model I this uit we illustrate the Martigale Cetral Limit Theorem by applyig it to the partial likelihood score fuctio from Cox s model. For simplicity of presetatio we assume that we have a scalar covariate Z; however, the same methods apply for vector Z. Suppose that Z deotes a scalar covariate ad assume that the hazard fuctio for someoe with covariate value Z is h(t Z) = λ (t)e βz. The observatios cosist of triplets (U i, δ i, Z i ), for i =, 2,...,, arisig from i.i.d. (T i, C i, Z i ). As usual, we assume that cesorig is oiformative (T i C i Z i ). Let U = 2 ll (β), β where L is Cox s partial likelihood. Recall that if L were a real likelihood fuctio based o i.i.d. observatios, the uder some regularity coditios we would have that 2 ll (β) β = ( ) ll / β L N(, v), sice the bracketed term is a average of i.i.d. zero-mea radom variables (say, with some variace v). This is why we stadardize ll (β)/ β by /2.
2 Returig to Cox s partial likelihood, we idicated i Uit 2 that we ca express U as where ΣU (t) = U = ΣU (t) t=, ( Z i l= Z le βz l Y ) l(s) dm i (s), l= eβz l Yl (s) ad where N i (t) = [U i t, δ i = ], Y i (t) = (U i t), M i (t) = N i (t) A i (t), ad Lettig we ca thus write A i (t) = ( H i (s) def = 2 Z i λ (s)e βz l Y i (s)ds. Zl e βz l Y ) l(s), e βz lyl (s) ΣU (t) = 2 H i (s)dm i (s). Settig this equal to zero is the same as settig H i (s)dm i (s) to zero. Note that A i ( ) is cotiuous because T i is cotiuous, so λ is cotiuous. To apply the martigale CLT, we eed H i ( ) to be predictable ad locally bouded. It is clear that H i ( ) is predictable (it is left cotiuous ad adapted). Oe way to esure that it is locally bouded is to assume that Z is bouded (Exercise 8 of Uit 2). Also, because the survival times T, T 2,..., T for the subjects are assumed to be idepedet ad cotiuously distributed, they caot jump at the same time, ad so the M i ( ) are orthogoal. Thus, the settig of the martigale cetral limit theorem, (3.3)-(3.5), has bee established. 2
3 We ow evaluate coditios (a) ad (b) of the Martigale CLT. (a) Does < ΣU, ΣU? > (t) P α(t) (some determiistic fuctio) as Because A i ( ) is cotiuous ad M i ad M j are orthogoal for i j, < ΣU, ΣU > (t) = H i (s)h j (s)d < M i, M j > (s) where = = = j= Q (s) = H 2 i (s)da i (s) λ (s) (Z i Q (s)) 2 λ (s) e βz i Y i (s)ds ( ) (Z i Q (s)) 2 e βz i Y i (s) ds, l= Z le βz l Y l(s) l= eβz l Yl (s). Cosider the term i large brackets for fixed s. This has the same probability limit as { } (Z i µ(s)) 2 e βz i Y i (s), (4.) where µ(s) = E ( Z l e βz l Y l(s) ) E (e βz l Yl (s)). To see this, ote that their differece equals (Z i Q (s)) 2 e βz i Y i (s) (Z i µ(s)) 2 e βz i Y i (s) = ( Y i (s)e βz i Q 2 (s) µ 2 (s) + 2Z i (µ(s) Q (s)) ) 3
4 = (Q 2 (s) µ 2 (s)) Y i (s)e βz i 2 (Q (s) µ(s)) By the Law of Large Numbers, ad Y i (s)e βz P i E(Yi (s)e βz i ) Y i (s)e βz i Z i. (4.2) Y i (s)e βz i P Z i E(Yi (s)e βz i Z i ). Sice Q (s) coverges i probability to µ(s) (Law of Large Numbers ad Slutzky), Q 2 (s) µ 2 (s) coverges i probability to zero (Cotiuous Mappig Theorem), ad hece (4.2) coverges to zero (repeatedly apply Slutzky s lemma). Now let s cosider (4.) for ay arbitrary s. Sice it is a average of i.i.d. radom variables, it coverges i probability to ( ) m(s) def = E (Z i µ(s)) 2 e βz i Y i (s). It follows from this that < ΣU, ΣU > (t) = λ (s) ( ) (Z i Q (s)) 2 e βz i Y i (s) ds P λ (s)m(s)ds. (4.3) That is, if the limit of the itegral is the itegral of the limit (we will show this more rigorously i Uit 5). Thus, i the otatio of the Martigale CLT, α(t) = λ (s)m(s)ds. 4
5 Now cosider coditio (b) of the Martigale Cetral Limit Theorem: Does < ΣU,ϵ, ΣU,ϵ > (t) P for all ϵ >? We have < ΣU,ϵ, ΣU,ϵ > (t) = = λ (s){ H 2 i (s) [ H i (s) ϵ]da i (s) = ( j= Z i Z ) [ j(s)e βz j Y j (s) 2 Z i j= eβz j Yj (s) Note that the limit, i probability, of [ Z i j= Z je βz j Y j(s) j= eβz j Yj (s) j= Z je βz j Y j (s) j= eβz j Yj (s) ϵ ϵ is zero because the term i absolute value is bouded (still assume Z is bouded) whereas ϵ (Exercise ). It follows that the etire itegrad i the above itegral coverges to zero i probability, ad hece that < ΣU,ϵ, ΣU,ϵ > (t) P for every t ad ϵ as (implicit i this statemet is that the limit of the itegral equals the itegral of the limit. This requires some assumptios that we will retur to i Uit 5). Thus, the coditios of Martigale CLT hold, so that ] ] e βz i Y i (s)}ds. ΣU ( ) w Q( ), as, where Q( ) is a zero-mea Gaussia process with idepedet icremets ad var (Q(t)) = α(t). By takig t =, it follows that U = ΣU ( ) L N(, σ 2 ) as, where σ 2 = α( ) = λ (s)m(s)ds. Note: ot clear why we ca evaluate these processes at ifiity. Usually, oe assumes that the support of U does ot exted beyod some time τ, so that the process evaluated at time τ is same as at ifiity. 5
6 The expressio (4.3) for α(t) caot be simplified much more without beig more specific about Z ad the cesorig distributio. For example, cosider the 2-sample problem, where { for group Z i = for group ad suppose that P (Z i = ) = 2, ad that the distributio of C i does ot deped o (T i, Z i ), ad has c.d.f. G( ). The it is ot difficult to show that uder H : β =, ad where E ( (Z l e βz l Y l (s) ) = 2 ( F (s))( G(s)) E ( e βz l Y l (s) ) = ( F (s))( G(s)), F (t) = e λ (s)ds. Thus, µ(s) = /2. It ca also be easily show that, uder H, m(s) = E ( (Z i 2 )2 Y i (s) ) Thus α(t) = 4 = 4 = 4 ( F (s))( G(s)). where f (s) = λ (s)( F (s)). Whe t =, where p = P r(δ i = ). σ 2 = α( ) = 4 λ (s)( F (s))( G(s))ds f (s)( G(s))ds, f (s)( G(s))ds = p 4, 6
7 Hece we have show that uder H : β =, U = /2 ll (β)/ β coverges i distributio to a ormal distributio with mea zero ad variace σ 2. Therefore, the partial likelihood score test of H, obtaied by evaluatig this at β = ad stadardizig it by σ, is asymptotically N(, ) uder H. We later use very similar techiques to fid the o-ull distributio of Cox s partial likelihood score test whe β. Note: this result shows that lograk test is asymptotically N(,) uder the ull: recall that it ca be viewed as arisig from a Cox proportioal hazards model as a score test! 7
8 Brief Review of Some Key Martigale Theory Results X( ) is a martigale if it is adapted to F t ad: X( ) is right cotiuous with left-had limits E X(t) < t E[X(t + s) F t ] a.s. = X(t) s, t. X( ) is a sub-martigale if the = is replaced by. If X( ) is a martigale, the E(X(t)) is costat i t; without loss of geerality we ca take E(X(t)) =. N( ) is a coutig process if N() =, N(t) < N( ) is right-cotiuous N( ) is a step fuctio with jumps of size +. H( ) is a predictable process if its value at t determied by F t (e.g., left cotiuous ad adapted). Doob-Meyer decompositio: X( ) = o-egative sub-martigale right-cotiuous,, predictable process A( ) s.t. E(A(t)) < t, ad M( ) = X( ) A( ) is a martigale. A( ) is called the compesator for X( ). If A() a.s. =, A( ) is almost surely uique. Ay coutig process N( ) such that E(N(t)) < is a sub-martigale, A( ) such that N( ) A( ) is a martigale. If M( ) is ay martigale ad EM 2 (t) <, M 2 ( ) is a sub-martigale! predictable process < M, M > ( ) such that M 2 ( ) < M, M > ( ) is a martigale. The process < M, M > ( ) is called the predictable quadratic variatio. Note: If E(M(t)) =, var (M(t)) = E(M 2 (t)) = E(< M, M > (t)). If the zero mea martigale M = N A satisfies E(M 2 (t)) < ad A( ) is cotiuous, the < M, M > ( ) as = A( ). Thus, V ar(m(t)) = E(A(t)). 8
9 Next suppose that M ( ), M 2 ( ), are zero-mea martigales defied o the same filtratio. The a M ( ) + bm 2 ( ) is a martigale (for ay costats a, b). If E(Mj 2 (t)) < for j =, 2, a right-cotiuous predictable process < M, M 2 > ( ) such that < M, M 2 > () =, E < M, M 2 > (t) <, ad M ( ) M 2 ( ) < M, M 2 > ( ) is a martigale. If < M, M 2 > as =, M ( ) ad M 2 ( ) are called orthogoal. If M j = N j A j, where N j is a coutig process ad A j is its cotiuous compesator, the if N i ad N j caot jump at the same time, < M i, M j > ( ) = as. Next suppose that N( ) is a bouded coutig process, that A( ) is the compesator for N( ), ad E(N A) 2 (t) < for every t. Moreover, H( ) is a bouded, predictable process. Defie Q( ) by The Q(t) = H(s)dM(s) Q( ) is a zero-mea martigale. V ar(q(t)) = E(Q 2 (t)). where M( ) = N( ) A( ). Q 2 ( ) is a sub-martigale, ad thus predictable < Q, Q > ( ) such that Q 2 ( ) < Q, Q > ( ) is a martigale. Hece, var (Q(t)) = E(< Q, Q > (t)). If A( ) is cotiuous, < Q, Q > (t) a.s. = H2 (s)da(s), ad thus V ar(q(t)) = E( H2 (s)da(s)). If M, M 2 are 2 martigales defied similarly to M( ), H ad H 2 are bouded predictable processes, ad Q ( ) ad Q 2 ( ) are defied i the same way as Q( ), the predictable quadratic covariace process < Q, Q 2 > satisfies < Q, Q 2 > (t) = H (s) H 2 ( )d < M, M 2 > (s). 9
10 Martigale CLT: For,2,...,, let Defie N i ( ) = coutig processes that ca t jump at the same time A i ( ) = cotiuous compesators H i ( ) = locally bouded predictable processes Suppose that M i( ) = N i ( ) A i ( ) U i (t) = H i(s)dm i (s) Hi (s) = H i(s)[ H i (s) ϵ] U i,ϵ (t) = H i (s)dm i(s) ΣU ( ) = i U i( ), ad ΣU i,ϵ ( ) = i U i,ϵ( ). The (a) < ΣU, ΣU > (t) P α(t) for all t, for some fuctio α( ). (b) < ΣU,ϵ, ΣU,ϵ >(t) P for all t ad all ϵ >. ΣU ( ) L U( ), where U( ) is a zero-mea Gaussia process with idepedet icremets ad variace fuctio V ar(u(t)) = α(t). Note: By the defiitios of N i ( ), A i ( ), ad H i ( ): < ΣU, ΣU > (t) = Hi(s)dA 2 i (s) < ΣU,ϵ, ΣU,ϵ > (t) = where H i (s) = H i(s) [ H i (s) ϵ]. H 2 i (s)da i (s), Ituitio: While the proof of the Martigale CLT is rather detailed, the result is ot surprisig. Oe forms a sum, U ( ) = U i ( ), of orthogoal
11 martigales, where each has zero mea, ad where the variace of the sum coverges i probability to some costat fuctio α( ). I the classical CLT we are used to seeig a sum of i.i.d. radom variables multiplied by /2. Thus, the variace of the stadardized sum is (ad coverges to) σ2 = σ 2, where σ 2 is the variace of each term i the sum. I the way we have set up the Martigale CLT, the multiplier /2 is buried i the itegrads, H i ( ) of the martigales Q i, so the aalogy is really the same as i the classical CLT. If we cosider the asymptotic distributio of U (t) for some fixed t, it is ot surprisig that the result is a ormal distributio with mea. From this oe would expect the fiite-dimesioal distributios of U ( ) to coverge to zero mea multivariate ormal distributios. Thus, if tightess holds, we would expect U ( ) to coverge to a zero mea Gaussia process. The idepedet icremets of this limitig process are assured from the ucorrelated icremet property of the idividual martigales U i ( ).
12 Exercises. Prove that (see page 5) Q (s) P as, where Q (s) = {( Z i j= Z je βz j Y j(s) j= eβz j Yj (s) ) ϵ }. 2. Prove the expressios for µ, m ad σ 2 o page 6. Do ot take ay of the expressios o page 6 for grated. 2
17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationKolmogorov-Smirnov type Tests for Local Gaussianity in High-Frequency Data
Proceedigs 59th ISI World Statistics Cogress, 5-30 August 013, Hog Kog (Sessio STS046) p.09 Kolmogorov-Smirov type Tests for Local Gaussiaity i High-Frequecy Data George Tauche, Duke Uiversity Viktor Todorov,
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationChapter 13: Tests of Hypothesis Section 13.1 Introduction
Chapter 13: Tests of Hypothesis Sectio 13.1 Itroductio RECAP: Chapter 1 discussed the Likelihood Ratio Method as a geeral approach to fid good test procedures. Testig for the Normal Mea Example, discussed
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationWeb-based Supplementary Materials for A Modified Partial Likelihood Score Method for Cox Regression with Covariate Error Under the Internal
Web-based Supplemetary Materials for A Modified Partial Likelihood Score Method for Cox Regressio with Covariate Error Uder the Iteral Validatio Desig by David M. Zucker, Xi Zhou, Xiaomei Liao, Yi Li,
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationAn alternative proof of a theorem of Aldous. concerning convergence in distribution for martingales.
A alterative proof of a theorem of Aldous cocerig covergece i distributio for martigales. Maurizio Pratelli Dipartimeto di Matematica, Uiversità di Pisa. Via Buoarroti 2. I-56127 Pisa, Italy e-mail: pratelli@dm.uipi.it
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:
Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal
More informationUnderstanding Samples
1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More information6a Time change b Quadratic variation c Planar Brownian motion d Conformal local martingales e Hints to exercises...
Tel Aviv Uiversity, 28 Browia motio 59 6 Time chage 6a Time chage..................... 59 6b Quadratic variatio................. 61 6c Plaar Browia motio.............. 64 6d Coformal local martigales............
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationConfidence Level We want to estimate the true mean of a random variable X economically and with confidence.
Cofidece Iterval 700 Samples Sample Mea 03 Cofidece Level 095 Margi of Error 0037 We wat to estimate the true mea of a radom variable X ecoomically ad with cofidece True Mea μ from the Etire Populatio
More informationBerry-Esseen bounds for self-normalized martingales
Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More information18.S096: Homework Problem Set 1 (revised)
8.S096: Homework Problem Set (revised) Topics i Mathematics of Data Sciece (Fall 05) Afoso S. Badeira Due o October 6, 05 Exteded to: October 8, 05 This homework problem set is due o October 6, at the
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationRef. Gallager, Stochastic Processes. Notation a vector. All vectors are row vectors. k k. jωx. Φ joint chacteristic function of.
Gaussia Radom ector Ref. Gallager, Stochastic Processes. Notatio a vector. All vectors are row vectors a a aa ω matrix scalar matrix ( ) ( ) -dim radom vector,, -dim real vector ω,, ω Φ oit chacteristic
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationThe Poisson Process *
OpeStax-CNX module: m11255 1 The Poisso Process * Do Johso This work is produced by OpeStax-CNX ad licesed uder the Creative Commos Attributio Licese 1.0 Some sigals have o waveform. Cosider the measuremet
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationSTAT 350 Handout 19 Sampling Distribution, Central Limit Theorem (6.6)
STAT 350 Hadout 9 Samplig Distributio, Cetral Limit Theorem (6.6) A radom sample is a sequece of radom variables X, X 2,, X that are idepedet ad idetically distributed. o This property is ofte abbreviated
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More information(b) What is the probability that a particle reaches the upper boundary n before the lower boundary m?
MATH 529 The Boudary Problem The drukard s walk (or boudary problem) is oe of the most famous problems i the theory of radom walks. Oe versio of the problem is described as follows: Suppose a particle
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationLecture 11 October 27
STATS 300A: Theory of Statistics Fall 205 Lecture October 27 Lecturer: Lester Mackey Scribe: Viswajith Veugopal, Vivek Bagaria, Steve Yadlowsky Warig: These otes may cotai factual ad/or typographic errors..
More informationLECTURE 14 NOTES. A sequence of α-level tests {ϕ n (x)} is consistent if
LECTURE 14 NOTES 1. Asymptotic power of tests. Defiitio 1.1. A sequece of -level tests {ϕ x)} is cosistet if β θ) := E θ [ ϕ x) ] 1 as, for ay θ Θ 1. Just like cosistecy of a sequece of estimators, Defiitio
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More informationMa 530 Infinite Series I
Ma 50 Ifiite Series I Please ote that i additio to the material below this lecture icorporated material from the Visual Calculus web site. The material o sequeces is at Visual Sequeces. (To use this li
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationDirection: This test is worth 250 points. You are required to complete this test within 50 minutes.
Term Test October 3, 003 Name Math 56 Studet Number Directio: This test is worth 50 poits. You are required to complete this test withi 50 miutes. I order to receive full credit, aswer each problem completely
More informationModule 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationEstimation of the Mean and the ACVF
Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationSequences. Notation. Convergence of a Sequence
Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it
More informationMatrix Representation of Data in Experiment
Matrix Represetatio of Data i Experimet Cosider a very simple model for resposes y ij : y ij i ij, i 1,; j 1,,..., (ote that for simplicity we are assumig the two () groups are of equal sample size ) Y
More informationApply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.
Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α
More informationChapter 8. Euler s Gamma function
Chapter 8 Euler s Gamma fuctio The Gamma fuctio plays a importat role i the fuctioal equatio for ζ(s that we will derive i the ext chapter. I the preset chapter we have collected some properties of the
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationIE 230 Seat # Name < KEY > Please read these directions. Closed book and notes. 60 minutes.
IE 230 Seat # Name < KEY > Please read these directios. Closed book ad otes. 60 miutes. Covers through the ormal distributio, Sectio 4.7 of Motgomery ad Ruger, fourth editio. Cover page ad four pages of
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More information6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.
6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationStochastic Integration and Ito s Formula
CHAPTER 3 Stochastic Itegratio ad Ito s Formula I this chapter we discuss Itô s theory of stochastic itegratio. This is a vast subject. However, our goal is rather modest: we will develop this theory oly
More informationProperties and Hypothesis Testing
Chapter 3 Properties ad Hypothesis Testig 3.1 Types of data The regressio techiques developed i previous chapters ca be applied to three differet kids of data. 1. Cross-sectioal data. 2. Time series data.
More informationMath 152. Rumbos Fall Solutions to Review Problems for Exam #2. Number of Heads Frequency
Math 152. Rumbos Fall 2009 1 Solutios to Review Problems for Exam #2 1. I the book Experimetatio ad Measuremet, by W. J. Youde ad published by the by the Natioal Sciece Teachers Associatio i 1962, the
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationSection 11.8: Power Series
Sectio 11.8: Power Series 1. Power Series I this sectio, we cosider geeralizig the cocept of a series. Recall that a series is a ifiite sum of umbers a. We ca talk about whether or ot it coverges ad i
More information